AWS Bedrock Setup
Loading...
LlamaCloud supports AWS Bedrock for accessing Anthropic Claude models as part of its multimodal AI capabilities. AWS Bedrock provides enterprise-grade access with advanced security, compliance, and AWS ecosystem integration. This page guides you through configuring AWS Bedrock integration with your self-hosted LlamaCloud deployment.
Prerequisites
Section titled “Prerequisites”- A valid AWS account
- AWS Bedrock access enabled in your region
- Access and quota to the supported models:
- For LlamaParse Advanced Features:
- Anthropic Claude 3.7 Sonnet (
anthropic.claude-3-7-sonnet-20250219-v1:0) - Anthropic Claude 3.5 Sonnet (
anthropic.claude-3-5-sonnet-20240620-v1:0) - Anthropic Claude 3.5 Haiku (
anthropic.claude-3-5-haiku-20241022-v1:0) - Anthropic Claude 4 Sonnet (
anthropic.claude-sonnet-4-20250514-v1:0)
- Anthropic Claude 3.7 Sonnet (
- For LlamaCloud Playground:
- Anthropic Claude 3.5 Sonnet V2
- Cohere Rerank 3.5 (Optional)
- For LlamaParse Advanced Features:
Environment Variables
Section titled “Environment Variables”Basic Configuration
Section titled “Basic Configuration”AWS_ACCESS_KEY_ID- AWS access key IDAWS_SECRET_ACCESS_KEY- AWS secret access keyAWS_REGION- AWS region (e.g.,us-east-1)
Model Version Overrides (Optional)
Section titled “Model Version Overrides (Optional)”BEDROCK_ANTHROPIC_SONNET_3_5_VERSION_NAME- Override for Claude 3.5 Sonnet versionBEDROCK_ANTHROPIC_SONNET_3_7_VERSION_NAME- Override for Claude 3.7 Sonnet versionBEDROCK_ANTHROPIC_HAIKU_3_5_VERSION_NAME- Override for Claude 3.5 Haiku versionBEDROCK_ANTHROPIC_SONNET_4_0_VERSION_NAME- Override for Claude 4 Sonnet version
Configuration
Section titled “Configuration”Choose one of these methods to configure AWS Bedrock integration:
Method 1: IAM Roles for Service Accounts (Recommended)
Section titled “Method 1: IAM Roles for Service Accounts (Recommended)”This method uses AWS IAM roles for secure, credential-free authentication.
Step 1: Create IAM Policy
Section titled “Step 1: Create IAM Policy”Create an IAM policy with the following permissions:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream", "bedrock:CreateModelInvocationJob", "bedrock:ListInferenceProfiles" ], "Resource": [ "arn:aws:bedrock:*:*:foundation-model/*", "arn:aws:bedrock:*:*:inference-profile/*" ] } ]}Step 2: Create IAM Role
Section titled “Step 2: Create IAM Role”- Create an IAM role for EKS service accounts
- Attach the policy to the role
- Configure trust relationship for your EKS cluster
Step 3: Configure Helm Values
Section titled “Step 3: Configure Helm Values”llamaParse: config: awsBedrock: enabled: true serviceAccount: annotations: eks.amazonaws.com/role-arn: "arn:aws:iam::ACCOUNT-ID:role/ROLE-NAME"Step 4: Override Model Versions (Optional)
Section titled “Step 4: Override Model Versions (Optional)”If you need to override the default model versions, configure them directly in your Helm values:
llamaParse: config: awsBedrock: enabled: true # Optional: Override default model versions sonnet3_5ModelVersionName: "anthropic.claude-3-5-sonnet-20240620-v1:0" sonnet3_7ModelVersionName: "anthropic.claude-3-7-sonnet-20250219-v1:0" haiku3_5ModelVersionName: "anthropic.claude-3-5-haiku-20241022-v1:0" sonnet4_0ModelVersionName: "anthropic.claude-sonnet-4-20250514-v1:0" # Or use inference profile IDs # sonnet3_5ModelVersionName: "us.anthropic.claude-3-5-sonnet-20241022-v2:0" # sonnet3_7ModelVersionName: "us.anthropic.claude-3-7-sonnet-20250219-v1:0" # haiku3_5ModelVersionName: "us.anthropic.claude-3-5-haiku-20241022-v1:0" # sonnet4_0ModelVersionName: "us.anthropic.claude-sonnet-4-20250514-v1:0" serviceAccount: annotations: eks.amazonaws.com/role-arn: "arn:aws:iam::ACCOUNT-ID:role/ROLE-NAME"For complete setup instructions, see the AWS EKS IAM Roles documentation.
Method 2: Static AWS Credentials
Section titled “Method 2: Static AWS Credentials”Step 1: Create Kubernetes Secret
Section titled “Step 1: Create Kubernetes Secret”apiVersion: v1kind: Secretmetadata: name: aws-bedrock-credentialstype: OpaquestringData: AWS_ACCESS_KEY_ID: "AKIA..." AWS_SECRET_ACCESS_KEY: "your-secret-key" AWS_REGION: "us-east-1" # Optional: Override default model versions BEDROCK_ANTHROPIC_SONNET_3_5_VERSION_NAME: "anthropic.claude-3-5-sonnet-20240620-v1:0" BEDROCK_ANTHROPIC_SONNET_3_7_VERSION_NAME: "anthropic.claude-3-7-sonnet-20250219-v1:0" BEDROCK_ANTHROPIC_HAIKU_3_5_VERSION_NAME: "anthropic.claude-3-5-haiku-20241022-v1:0" BEDROCK_ANTHROPIC_SONNET_4_0_VERSION_NAME: "anthropic.claude-sonnet-4-20250514-v1:0"Step 2: Configure Helm Values
Section titled “Step 2: Configure Helm Values”# External SecretllamaParse: config: awsBedrock: enabled: true existingSecret: "aws-bedrock-credentials"
# or direct configuration (not recommended for production)llamaParse: config: awsBedrock: enabled: true region: "us-east-1" accessKeyId: "AKIA..." secretAccessKey: "your-secret-key" sonnet3_5ModelVersionName: "anthropic.claude-3-5-sonnet-20240620-v1:0" sonnet3_7ModelVersionName: "anthropic.claude-3-7-sonnet-20250219-v1:0" haiku3_5ModelVersionName: "anthropic.claude-3-5-haiku-20241022-v1:0" sonnet4_0ModelVersionName: "anthropic.claude-sonnet-4-20250514-v1:0"AWS Bedrock Inference Profiles
Section titled “AWS Bedrock Inference Profiles”AWS Bedrock inference profiles provide enhanced functionality for routing model requests across regions and cost tracking. Some newer Anthropic models require inference profiles instead of direct model IDs.
When to Use Inference Profiles
Section titled “When to Use Inference Profiles”- Required for certain models: Depending on the model and region, you may need to use inference profiles instead of direct model IDs
- Please refer to the AWS Inference Profiles Support Docs for more information
- Cross-region routing: Automatic request routing across multiple regions for improved availability
- Cost tracking: Detailed usage and cost tracking for applications
- Enhanced reliability: Better fault tolerance and availability
Inference Profile ID Format
Section titled “Inference Profile ID Format”Inference profile IDs use a regional prefix format:
- Claude 3.5 Sonnet:
us.anthropic.claude-3-5-sonnet-20241022-v2:0 - Claude 3.7 Sonnet:
us.anthropic.claude-3-7-sonnet-20250219-v1:0 - Claude 3.5 Haiku:
us.anthropic.claude-3-5-haiku-20241022-v1:0 - Claude 4 Sonnet:
us.anthropic.claude-sonnet-4-20250514-v1:0
Finding Inference Profile IDs
Section titled “Finding Inference Profile IDs”- Navigate to the Amazon Bedrock console
- Go to Inference and Assessment → Cross-region Inference
- Copy the Inference Profile ID for your desired model
Configuration with Inference Profiles
Section titled “Configuration with Inference Profiles”Use inference profile IDs instead of model IDs in your configuration:
apiVersion: v1kind: Secretmetadata: name: aws-bedrock-credentialstype: OpaquestringData: AWS_ACCESS_KEY_ID: "AKIA..." AWS_SECRET_ACCESS_KEY: "your-secret-key" AWS_REGION: "us-east-1" # Use inference profiles for newer models BEDROCK_ANTHROPIC_SONNET_3_5_VERSION_NAME: "us.anthropic.claude-3-5-sonnet-20241022-v2:0" BEDROCK_ANTHROPIC_SONNET_3_7_VERSION_NAME: "us.anthropic.claude-3-7-sonnet-20250219-v1:0" BEDROCK_ANTHROPIC_HAIKU_3_5_VERSION_NAME: "us.anthropic.claude-3-5-haiku-20241022-v1:0" BEDROCK_ANTHROPIC_SONNET_4_0_VERSION_NAME: "us.anthropic.claude-sonnet-4-20250514-v1:0"Verification
Section titled “Verification”After configuration, verify your AWS Bedrock integration:
- Check Admin UI: Verify Claude models appear in LlamaCloud admin interface
- Test functionality: Upload a document and use advanced parsing modes
Troubleshooting
Section titled “Troubleshooting”Common Issues
Section titled “Common Issues”Access Denied
Section titled “Access Denied”Error: User is not authorized to perform bedrock:InvokeModelSolution:
- Verify IAM permissions include all required actions
- Check that the resource ARNs include both foundation models and inference profiles
- Ensure the role is properly attached to the service account (for IRSA)
Model Not Found
Section titled “Model Not Found”Error: Could not resolve foundation model from model identifierSolution:
- Verify the model ID or inference profile ID is correct
- Check that the model is available in your AWS region
- Try using an inference profile ID instead of direct model ID
On-Demand Throughput Error
Section titled “On-Demand Throughput Error”Error: On-demand throughput is not supported for this modelSolution:
- This typically means the model requires an inference profile
- Switch to using the inference profile ID format
- Check the Bedrock console for the correct inference profile ID
Region Issues
Section titled “Region Issues”Error: Model not available in regionSolution:
- Check model availability in your specified AWS region
- Consider switching to a region where the model is available
- Use cross-region inference profiles for better availability
Quota Exceeded
Section titled “Quota Exceeded”Error: Throttling exception or quota exceededSolution:
- Check your Bedrock quotas in AWS Console
- Request quota increases if needed
- Implement request throttling in your application
Debug Steps
Section titled “Debug Steps”-
Test AWS credentials directly:
Terminal window aws bedrock list-foundation-models --region us-east-1 -
Test model invocation:
Terminal window aws bedrock-runtime invoke-model \--model-id anthropic.claude-3-5-sonnet-20240620-v1:0 \--body '{"messages":[{"role":"user","content":[{"type":"text","text":"Hello"}]}],"max_tokens":100,"anthropic_version":"bedrock-2023-05-31"}' \--cli-binary-format raw-in-base64-out \output.json -
Verify service account annotations (for IRSA):
Terminal window kubectl describe serviceaccount llamacloud-llamaparse -
Check secret mounting:
Terminal window kubectl describe pod <pod-name> | grep -A 20 Environment