Azure Service Bus as Job Queue
Loading...
LlamaCloud’s job queue is AMQP-compatible. You can use Azure Service Bus instead of a self-managed RabbitMQ by supplying a Service Bus connection string in your Helm values.
Prerequisites
Section titled “Prerequisites”- An Azure Service Bus namespace.
- Namespace tier: Standard or higher (Basic is not supported).
- A connection string with Manage, Send, and Listen permissions.
- Manage is needed so LlamaCloud can validate and, if necessary, create the queues on startup.
- How to find it: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-get-started-with-queues?tabs=connection-string
Configure via values.yaml (Recommended)
Section titled “Configure via values.yaml (Recommended)”Update your values.yaml as follows:
global: config: rabbitmq: external: enabled: true # Paste your Azure Service Bus connection string here connectionString: "Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=<policy>;SharedAccessKey=<key>;EntityPath=<queue-name>"
# Disable the RabbitMQ subchart when using an external queuerabbitmq: enabled: falseConfigure via Kubernetes Secret (Optional)
Section titled “Configure via Kubernetes Secret (Optional)”For production, you may prefer a secret instead of inlining credentials.
:::note Environment variable name
LlamaCloud reads the Azure Service Bus connection string from the JOB_QUEUE_CONNECTION_STRING environment variable. When supplying a Kubernetes Secret via global.config.rabbitmq.external.existingSecretName, ensure the secret contains a key named JOB_QUEUE_CONNECTION_STRING whose value is the full Service Bus connection string.
:::
- Create a secret with the connection string:
kubectl create secret generic my-queue-secret \ --from-literal=JOB_QUEUE_CONNECTION_STRING='Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=<policy>;SharedAccessKey=<key>;EntityPath=<queue-name>'- Reference the secret in
values.yaml:
global: config: rabbitmq: external: enabled: true existingSecretName: my-queue-secret
rabbitmq: enabled: falseVerify
Section titled “Verify”- Upgrade your release and ensure pods start successfully.
- Services will log successful connections to the job queue; check
jobs-workerandllamaparsepods for confirmation.