Azure OpenAI Setup
Loading...
LlamaCloud supports Azure OpenAI as an enterprise-grade alternative to OpenAI for organizations requiring enhanced compliance, security, and private deployments. This page guides you through configuring Azure OpenAI integration with your self-hosted LlamaCloud deployment.
Prerequisites
Section titled âPrerequisitesâ- A valid Azure OpenAI account
- Azure OpenAI resource deployed in Azure
- Access and quota for the supported models:
gpt-4ogpt-4o-minigpt-4.1gpt-4.1-minigpt-4.1-nanogpt-5gpt-5-minigpt-5-nanotext-embedding-3-smalltext-embedding-3-largewhisper-1
Environment Variables
Section titled âEnvironment VariablesâAzure OpenAI supports per-model configuration using environment variables. Each model uses its own set of credentials:
Environment Variable Pattern
Section titled âEnvironment Variable PatternâAZURE_OPENAI_<MODEL_NAME>_API_KEY- Deployment API keyAZURE_OPENAI_<MODEL_NAME>_BASE_URL- Azure OpenAI endpointAZURE_OPENAI_<MODEL_NAME>_DEPLOYMENT_NAME- Deployment name (not model name)AZURE_OPENAI_<MODEL_NAME>_API_VERSION- API version
Supported Models
Section titled âSupported Modelsâ- GPT-4o:
AZURE_OPENAI_GPT_4O_* - GPT-4o Mini:
AZURE_OPENAI_GPT_4O_MINI_* - GPT-4.1:
AZURE_OPENAI_GPT_4_1_* - GPT-4.1 Mini:
AZURE_OPENAI_GPT_4_1_MINI_* - GPT-4.1 Nano:
AZURE_OPENAI_GPT_4_1_NANO_* - GPT-5:
AZURE_OPENAI_GPT_5_* - GPT-5 Mini:
AZURE_OPENAI_GPT_5_MINI_* - GPT-5 Nano:
AZURE_OPENAI_GPT_5_NANO_* - Text Embedding 3 Small:
AZURE_OPENAI_TEXT_EMBEDDING_3_SMALL_* - Text Embedding 3 Large:
AZURE_OPENAI_TEXT_EMBEDDING_3_LARGE_* - Whisper 1:
AZURE_OPENAI_WHISPER_1_*
Configuration
Section titled âConfigurationâAzure OpenAI uses per-model configuration with separate credentials for each model. Follow these steps:
Step 1: Gather Azure OpenAI Information
Section titled âStep 1: Gather Azure OpenAI InformationâFor each model you want to use, collect:
- API Key: From your Azure OpenAI resourceâs âKeys and Endpointâ section
- Base URL: Your Azure endpoint (e.g.,
https://your-resource.openai.azure.com) - Deployment Name: The name of your model deployment (not the model name)
- API Version: Current version is
2024-12-01-preview
Step 2: Create Kubernetes Secret
Section titled âStep 2: Create Kubernetes SecretâCreate a secret with your Azure OpenAI credentials for each model:
apiVersion: v1kind: Secretmetadata: name: azure-openai-credentialstype: OpaquestringData: # GPT-4o configuration AZURE_OPENAI_GPT_4O_API_KEY: "your-gpt-4o-api-key" AZURE_OPENAI_GPT_4O_BASE_URL: "https://your-resource.openai.azure.com" AZURE_OPENAI_GPT_4O_DEPLOYMENT_NAME: "your-gpt-4o-deployment" AZURE_OPENAI_GPT_4O_API_VERSION: "2024-12-01-preview"
# GPT-4o Mini configuration AZURE_OPENAI_GPT_4O_MINI_API_KEY: "your-gpt-4o-mini-api-key" AZURE_OPENAI_GPT_4O_MINI_BASE_URL: "https://your-resource.openai.azure.com" AZURE_OPENAI_GPT_4O_MINI_DEPLOYMENT_NAME: "your-gpt-4o-mini-deployment" AZURE_OPENAI_GPT_4O_MINI_API_VERSION: "2024-12-01-preview"
# Add other models as needed...Apply the secret:
kubectl apply -f azure-openai-secret.yamlStep 3: Configure Helm Values
Section titled âStep 3: Configure Helm ValuesâReference the secret in your Helm configuration:
backend: config: azureOpenAi: enabled: true existingSecret: "azure-openai-credentials"
llamaParse: config: azureOpenAi: enabled: true existingSecret: "azure-openai-credentials"Direct Configuration (Deprecated)
Direct Configuration
Section titled âDirect Configurationâ# this method is being deprecated in the near future, please use the external secret method insteadbackend: config: azureOpenAi: enabled: true key: "your-api-key" endpoint: "https://your-resource.openai.azure.com" deploymentName: "your-deployment-name" apiVersion: "2024-12-01-preview"
llamaParse: config: azureOpenAi: enabled: true key: "your-api-key" endpoint: "https://your-resource.openai.azure.com" deploymentName: "your-deployment-name" apiVersion: "2024-12-01-preview"Finding Azure OpenAI Configuration Values
Section titled âFinding Azure OpenAI Configuration ValuesâBase URL
Section titled âBase URLâ- Go to your Azure OpenAI resource in the Azure Portal
- Navigate to Resource Management â Keys and Endpoint
- Copy the Endpoint value (format:
https://your-resource.openai.azure.com)
API Key
Section titled âAPI Keyâ- In the same Keys and Endpoint section
- Copy either KEY 1 or KEY 2
Deployment Name
Section titled âDeployment Nameâ- Go to Model deployments in your Azure OpenAI resource
- Use the Deployment name (not the Model name)
- Example: If you have a deployment called âmy-gpt-4oâ using the âgpt-4oâ model, use âmy-gpt-4oâ
API Version
Section titled âAPI Versionâ- Use the latest stable version:
2024-12-01-preview - Check Azure OpenAI API reference for current versions
Verification
Section titled âVerificationâAfter configuration, verify your Azure OpenAI integration:
-
Verify in Admin UI: Check available models in the LlamaCloud admin interface
-
Test functionality: Upload a document to confirm Azure OpenAI models are working
Troubleshooting
Section titled âTroubleshootingâCommon Issues
Section titled âCommon IssuesâAuthentication Errors
Section titled âAuthentication ErrorsâError: Access denied due to invalid subscription keySolution:
- Verify your API key is correct
- Ensure the key hasnât expired
- Check that youâre using the right key for the deployment
Deployment Not Found
Section titled âDeployment Not FoundâError: The API deployment for this resource does not existSolution:
- Verify the deployment name exactly matches whatâs in Azure
- Check that the deployment is in the same region as your resource
- Ensure the deployment is not paused or stopped
API Version Issues
Section titled âAPI Version IssuesâError: Invalid API version specifiedSolution:
- Use a supported API version (e.g.,
2024-12-01-preview) - Check Azure OpenAI documentation for current versions
Rate Limiting
Section titled âRate LimitingâError: Rate limit exceededSolution:
- Check your Azure OpenAI quotas
- Consider upgrading your deployment tier
- Implement request throttling
Debug Steps
Section titled âDebug Stepsâ-
Test endpoint directly:
Terminal window curl -H "api-key: YOUR_KEY" \"https://YOUR_RESOURCE.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT/completions?api-version=2024-12-01-preview" -
Verify secret mounting:
Terminal window kubectl get secret your-azure-openai-secret -o yamlkubectl describe pod <pod-name> | grep -A 20 Environment -
Check network connectivity: Ensure your cluster can reach your Azure OpenAI endpoint