AWS Bedrock Setup
Loading...
LlamaCloud supports AWS Bedrock for accessing Anthropic Claude models as part of its multimodal AI capabilities. AWS Bedrock provides enterprise-grade access with advanced security, compliance, and AWS ecosystem integration. This page guides you through configuring AWS Bedrock integration with your self-hosted LlamaCloud deployment.
Prerequisites
Section titled âPrerequisitesâ- A valid AWS account
- AWS Bedrock access enabled in your region
- Access and quota to the supported models:
- For LlamaParse Advanced Features:
- Anthropic Claude 3.7 Sonnet (
anthropic.claude-3-7-sonnet-20250219-v1:0) - Anthropic Claude 3.5 Sonnet (
anthropic.claude-3-5-sonnet-20240620-v1:0) - Anthropic Claude 3.5 Haiku (
anthropic.claude-3-5-haiku-20241022-v1:0) - Anthropic Claude 4 Sonnet (
anthropic.claude-sonnet-4-20250514-v1:0)
- Anthropic Claude 3.7 Sonnet (
- For LlamaCloud Playground:
- Anthropic Claude 3.5 Sonnet V2
- Cohere Rerank 3.5 (Optional)
- For LlamaParse Advanced Features:
Environment Variables
Section titled âEnvironment VariablesâBasic Configuration
Section titled âBasic ConfigurationâAWS_ACCESS_KEY_ID- AWS access key IDAWS_SECRET_ACCESS_KEY- AWS secret access keyAWS_REGION- AWS region (e.g.,us-east-1)
Model Version Overrides (Optional)
Section titled âModel Version Overrides (Optional)âBEDROCK_ANTHROPIC_SONNET_3_5_VERSION_NAME- Override for Claude 3.5 Sonnet versionBEDROCK_ANTHROPIC_SONNET_3_7_VERSION_NAME- Override for Claude 3.7 Sonnet versionBEDROCK_ANTHROPIC_HAIKU_3_5_VERSION_NAME- Override for Claude 3.5 Haiku versionBEDROCK_ANTHROPIC_SONNET_4_0_VERSION_NAME- Override for Claude 4 Sonnet version
Configuration
Section titled âConfigurationâChoose one of these methods to configure AWS Bedrock integration:
Method 1: IAM Roles for Service Accounts (Recommended)
Section titled âMethod 1: IAM Roles for Service Accounts (Recommended)âThis method uses AWS IAM roles for secure, credential-free authentication.
Step 1: Create IAM Policy
Section titled âStep 1: Create IAM PolicyâCreate an IAM policy with the following permissions:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream", "bedrock:CreateModelInvocationJob", "bedrock:ListInferenceProfiles" ], "Resource": [ "arn:aws:bedrock:*:*:foundation-model/*", "arn:aws:bedrock:*:*:inference-profile/*" ] } ]}Step 2: Create IAM Role
Section titled âStep 2: Create IAM Roleâ- Create an IAM role for EKS service accounts
- Attach the policy to the role
- Configure trust relationship for your EKS cluster
Step 3: Configure Helm Values
Section titled âStep 3: Configure Helm ValuesâllamaParse: config: awsBedrock: enabled: true serviceAccount: annotations: eks.amazonaws.com/role-arn: "arn:aws:iam::ACCOUNT-ID:role/ROLE-NAME"Step 4: Override Model Versions (Optional)
Section titled âStep 4: Override Model Versions (Optional)âIf you need to override the default model versions, configure them directly in your Helm values:
llamaParse: config: awsBedrock: enabled: true # Optional: Override default model versions sonnet3_5ModelVersionName: "anthropic.claude-3-5-sonnet-20240620-v1:0" sonnet3_7ModelVersionName: "anthropic.claude-3-7-sonnet-20250219-v1:0" haiku3_5ModelVersionName: "anthropic.claude-3-5-haiku-20241022-v1:0" sonnet4_0ModelVersionName: "anthropic.claude-sonnet-4-20250514-v1:0" # Or use inference profile IDs # sonnet3_5ModelVersionName: "us.anthropic.claude-3-5-sonnet-20241022-v2:0" # sonnet3_7ModelVersionName: "us.anthropic.claude-3-7-sonnet-20250219-v1:0" # haiku3_5ModelVersionName: "us.anthropic.claude-3-5-haiku-20241022-v1:0" # sonnet4_0ModelVersionName: "us.anthropic.claude-sonnet-4-20250514-v1:0" serviceAccount: annotations: eks.amazonaws.com/role-arn: "arn:aws:iam::ACCOUNT-ID:role/ROLE-NAME"For complete setup instructions, see the AWS EKS IAM Roles documentation.
Method 2: Static AWS Credentials
Section titled âMethod 2: Static AWS CredentialsâStep 1: Create Kubernetes Secret
Section titled âStep 1: Create Kubernetes SecretâapiVersion: v1kind: Secretmetadata: name: aws-bedrock-credentialstype: OpaquestringData: AWS_ACCESS_KEY_ID: "AKIA..." AWS_SECRET_ACCESS_KEY: "your-secret-key" AWS_REGION: "us-east-1" # Optional: Override default model versions BEDROCK_ANTHROPIC_SONNET_3_5_VERSION_NAME: "anthropic.claude-3-5-sonnet-20240620-v1:0" BEDROCK_ANTHROPIC_SONNET_3_7_VERSION_NAME: "anthropic.claude-3-7-sonnet-20250219-v1:0" BEDROCK_ANTHROPIC_HAIKU_3_5_VERSION_NAME: "anthropic.claude-3-5-haiku-20241022-v1:0" BEDROCK_ANTHROPIC_SONNET_4_0_VERSION_NAME: "anthropic.claude-sonnet-4-20250514-v1:0"Step 2: Configure Helm Values
Section titled âStep 2: Configure Helm Valuesâ# External SecretllamaParse: config: awsBedrock: enabled: true existingSecret: "aws-bedrock-credentials"
# or direct configuration (not recommended for production)llamaParse: config: awsBedrock: enabled: true region: "us-east-1" accessKeyId: "AKIA..." secretAccessKey: "your-secret-key" sonnet3_5ModelVersionName: "anthropic.claude-3-5-sonnet-20240620-v1:0" sonnet3_7ModelVersionName: "anthropic.claude-3-7-sonnet-20250219-v1:0" haiku3_5ModelVersionName: "anthropic.claude-3-5-haiku-20241022-v1:0" sonnet4_0ModelVersionName: "anthropic.claude-sonnet-4-20250514-v1:0"AWS Bedrock Inference Profiles
Section titled âAWS Bedrock Inference ProfilesâAWS Bedrock inference profiles provide enhanced functionality for routing model requests across regions and cost tracking. Some newer Anthropic models require inference profiles instead of direct model IDs.
When to Use Inference Profiles
Section titled âWhen to Use Inference Profilesâ- Required for certain models: Depending on the model and region, you may need to use inference profiles instead of direct model IDs
- Please refer to the AWS Inference Profiles Support Docs for more information
- Cross-region routing: Automatic request routing across multiple regions for improved availability
- Cost tracking: Detailed usage and cost tracking for applications
- Enhanced reliability: Better fault tolerance and availability
Inference Profile ID Format
Section titled âInference Profile ID FormatâInference profile IDs use a regional prefix format:
- Claude 3.5 Sonnet:
us.anthropic.claude-3-5-sonnet-20241022-v2:0 - Claude 3.7 Sonnet:
us.anthropic.claude-3-7-sonnet-20250219-v1:0 - Claude 3.5 Haiku:
us.anthropic.claude-3-5-haiku-20241022-v1:0 - Claude 4 Sonnet:
us.anthropic.claude-sonnet-4-20250514-v1:0
Finding Inference Profile IDs
Section titled âFinding Inference Profile IDsâ- Navigate to the Amazon Bedrock console
- Go to Inference and Assessment â Cross-region Inference
- Copy the Inference Profile ID for your desired model
Configuration with Inference Profiles
Section titled âConfiguration with Inference ProfilesâUse inference profile IDs instead of model IDs in your configuration:
apiVersion: v1kind: Secretmetadata: name: aws-bedrock-credentialstype: OpaquestringData: AWS_ACCESS_KEY_ID: "AKIA..." AWS_SECRET_ACCESS_KEY: "your-secret-key" AWS_REGION: "us-east-1" # Use inference profiles for newer models BEDROCK_ANTHROPIC_SONNET_3_5_VERSION_NAME: "us.anthropic.claude-3-5-sonnet-20241022-v2:0" BEDROCK_ANTHROPIC_SONNET_3_7_VERSION_NAME: "us.anthropic.claude-3-7-sonnet-20250219-v1:0" BEDROCK_ANTHROPIC_HAIKU_3_5_VERSION_NAME: "us.anthropic.claude-3-5-haiku-20241022-v1:0" BEDROCK_ANTHROPIC_SONNET_4_0_VERSION_NAME: "us.anthropic.claude-sonnet-4-20250514-v1:0"Verification
Section titled âVerificationâAfter configuration, verify your AWS Bedrock integration:
- Check Admin UI: Verify Claude models appear in LlamaCloud admin interface
- Test functionality: Upload a document and use advanced parsing modes
Troubleshooting
Section titled âTroubleshootingâCommon Issues
Section titled âCommon IssuesâAccess Denied
Section titled âAccess DeniedâError: User is not authorized to perform bedrock:InvokeModelSolution:
- Verify IAM permissions include all required actions
- Check that the resource ARNs include both foundation models and inference profiles
- Ensure the role is properly attached to the service account (for IRSA)
Model Not Found
Section titled âModel Not FoundâError: Could not resolve foundation model from model identifierSolution:
- Verify the model ID or inference profile ID is correct
- Check that the model is available in your AWS region
- Try using an inference profile ID instead of direct model ID
On-Demand Throughput Error
Section titled âOn-Demand Throughput ErrorâError: On-demand throughput is not supported for this modelSolution:
- This typically means the model requires an inference profile
- Switch to using the inference profile ID format
- Check the Bedrock console for the correct inference profile ID
Region Issues
Section titled âRegion IssuesâError: Model not available in regionSolution:
- Check model availability in your specified AWS region
- Consider switching to a region where the model is available
- Use cross-region inference profiles for better availability
Quota Exceeded
Section titled âQuota ExceededâError: Throttling exception or quota exceededSolution:
- Check your Bedrock quotas in AWS Console
- Request quota increases if needed
- Implement request throttling in your application
Debug Steps
Section titled âDebug Stepsâ-
Test AWS credentials directly:
Terminal window aws bedrock list-foundation-models --region us-east-1 -
Test model invocation:
Terminal window aws bedrock-runtime invoke-model \--model-id anthropic.claude-3-5-sonnet-20240620-v1:0 \--body '{"messages":[{"role":"user","content":[{"type":"text","text":"Hello"}]}],"max_tokens":100,"anthropic_version":"bedrock-2023-05-31"}' \--cli-binary-format raw-in-base64-out \output.json -
Verify service account annotations (for IRSA):
Terminal window kubectl describe serviceaccount llamacloud-llamaparse -
Check secret mounting:
Terminal window kubectl describe pod <pod-name> | grep -A 20 Environment