AWS Bedrock Setup

Self-Hosting Documentation Access Granted

LlamaCloud supports AWS Bedrock for accessing Anthropic Claude models as part of its multimodal AI capabilities. AWS Bedrock provides enterprise-grade access with advanced security, compliance, and AWS ecosystem integration. This page guides you through configuring AWS Bedrock integration with your self-hosted LlamaCloud deployment.

Prerequisites

A valid AWS account
AWS Bedrock access enabled in your region
Access and quota to the supported models:
- For LlamaParse Advanced Features:
  - Anthropic Claude 3.7 Sonnet (anthropic.claude-3-7-sonnet-20250219-v1:0)
  - Anthropic Claude 3.5 Sonnet (anthropic.claude-3-5-sonnet-20240620-v1:0) (deprecated)
  - Anthropic Claude 3.5 Haiku (anthropic.claude-3-5-haiku-20241022-v1:0)
  - Anthropic Claude 4 Sonnet (anthropic.claude-sonnet-4-20250514-v1:0)
- For LlamaCloud Playground:
  - Anthropic Claude 3.5 Sonnet V2 (deprecated)
  - Cohere Rerank 3.5 (Optional)

Environment Variables

Basic Configuration

AWS_ACCESS_KEY_ID - AWS access key ID
AWS_SECRET_ACCESS_KEY - AWS secret access key
AWS_REGION - AWS region (e.g., us-east-1)

Model Version Overrides (Optional)

BEDROCK_ANTHROPIC_SONNET_3_5_VERSION_NAME - Override for Claude 3.5 Sonnet version (deprecated)
BEDROCK_ANTHROPIC_SONNET_3_7_VERSION_NAME - Override for Claude 3.7 Sonnet version
BEDROCK_ANTHROPIC_HAIKU_3_5_VERSION_NAME - Override for Claude 3.5 Haiku version
BEDROCK_ANTHROPIC_HAIKU_4_5_VERSION_NAME - Override for Claude 4.5 Haiku version
BEDROCK_ANTHROPIC_SONNET_4_0_VERSION_NAME - Override for Claude 4 Sonnet version
PREFERED_PREMIUM_MODE_MODEL - Override the premium_mode model.

Configuration

Choose one of these methods to configure AWS Bedrock integration:

Method 1: IAM Roles for Service Accounts (Recommended)

This method uses AWS IAM roles for secure, credential-free authentication.

Step 1: Create IAM Policy

Create an IAM policy with the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:CreateModelInvocationJob",
        "bedrock:ListInferenceProfiles"
      ],
      "Resource": [
        "arn:aws:bedrock:*:*:foundation-model/*",
        "arn:aws:bedrock:*:*:inference-profile/*"
      ]
    }
  ]
}

Step 2: Create IAM Role

Create an IAM role for EKS service accounts
Attach the policy to the role
Configure trust relationship for your EKS cluster

Step 3: Configure Helm Values

llamaParse:
  serviceAccountAnnotations:
    eks.amazonaws.com/role-arn: "arn:aws:iam::ACCOUNT-ID:role/ROLE-NAME"

Step 4: Override Model Versions (Optional)

If you need to override the default model versions, configure them directly in your Helm values:

config:
  llms:
    awsBedrock:
      # Optional: Override default model versions
      sonnet3_5ModelVersionName: "anthropic.claude-3-5-sonnet-20240620-v1:0"
      sonnet3_7ModelVersionName: "anthropic.claude-3-7-sonnet-20250219-v1:0"
      haiku3_5ModelVersionName: "anthropic.claude-3-5-haiku-20241022-v1:0"
      sonnet4_0ModelVersionName: "anthropic.claude-sonnet-4-20250514-v1:0"
      # Or use inference profile IDs
      # sonnet3_5ModelVersionName: "us.anthropic.claude-3-5-sonnet-20241022-v2:0"
      # sonnet3_7ModelVersionName: "us.anthropic.claude-3-7-sonnet-20250219-v1:0"
      # haiku3_5ModelVersionName: "us.anthropic.claude-3-5-haiku-20241022-v1:0"
      # sonnet4_0ModelVersionName: "us.anthropic.claude-sonnet-4-20250514-v1:0"

For complete setup instructions, see the AWS EKS IAM Roles documentation.

Method 2: Static AWS Credentials

Step 1: Create Kubernetes Secret

apiVersion: v1
kind: Secret
metadata:
  name: aws-bedrock-credentials
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: "AKIA..."
  AWS_SECRET_ACCESS_KEY: "your-secret-key"
  AWS_REGION: "us-east-1"
  # Optional: Override default model versions
  BEDROCK_ANTHROPIC_SONNET_3_5_VERSION_NAME: "anthropic.claude-3-5-sonnet-20240620-v1:0"
  BEDROCK_ANTHROPIC_SONNET_3_7_VERSION_NAME: "anthropic.claude-3-7-sonnet-20250219-v1:0"
  BEDROCK_ANTHROPIC_HAIKU_3_5_VERSION_NAME: "anthropic.claude-3-5-haiku-20241022-v1:0"
  BEDROCK_ANTHROPIC_HAIKU_4_5_VERSION_NAME: "anthropic.claude-haiku-4-5-20251001-v1:0"
  BEDROCK_ANTHROPIC_SONNET_4_0_VERSION_NAME: "anthropic.claude-sonnet-4-20250514-v1:0"
  PREFERED_PREMIUM_MODE_MODEL: "anthropic-sonnet-4.0"

Step 2: Configure Helm Values

# External Secret
config:
  llms:
    awsBedrock:
      secret: "aws-bedrock-credentials"

# or direct configuration (not recommended for production)
config:
  llms:
    awsBedrock:
      region: "us-east-1"
      accessKeyId: "AKIA..."
      secretAccessKey: "your-secret-key"
      sonnet3_5ModelVersionName: "anthropic.claude-3-5-sonnet-20240620-v1:0"
      sonnet3_7ModelVersionName: "anthropic.claude-3-7-sonnet-20250219-v1:0"
      haiku3_5ModelVersionName: "anthropic.claude-3-5-haiku-20241022-v1:0"
      sonnet4_0ModelVersionName: "anthropic.claude-sonnet-4-20250514-v1:0"

AWS Bedrock Inference Profiles

AWS Bedrock inference profiles provide enhanced functionality for routing model requests across regions and cost tracking. Some newer Anthropic models require inference profiles instead of direct model IDs.

When to Use Inference Profiles

Required for certain models: Depending on the model and region, you may need to use inference profiles instead of direct model IDs
- Please refer to the AWS Inference Profiles Support Docs for more information
Cross-region routing: Automatic request routing across multiple regions for improved availability
Cost tracking: Detailed usage and cost tracking for applications
Enhanced reliability: Better fault tolerance and availability

Inference Profile ID Format

Inference profile IDs use a regional prefix format:

Claude 3.5 Sonnet: us.anthropic.claude-3-5-sonnet-20241022-v2:0
Claude 3.7 Sonnet: us.anthropic.claude-3-7-sonnet-20250219-v1:0
Claude 3.5 Haiku: us.anthropic.claude-3-5-haiku-20241022-v1:0
Claude 4 Sonnet: us.anthropic.claude-sonnet-4-20250514-v1:0

Finding Inference Profile IDs

Navigate to the Amazon Bedrock console
Go to Inference and Assessment → Cross-region Inference
Copy the Inference Profile ID for your desired model

Configuration with Inference Profiles

Use inference profile IDs instead of model IDs in your configuration:

apiVersion: v1
kind: Secret
metadata:
  name: aws-bedrock-credentials
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: "AKIA..."
  AWS_SECRET_ACCESS_KEY: "your-secret-key"
  AWS_REGION: "us-east-1"
  # Use inference profiles for newer models
  BEDROCK_ANTHROPIC_SONNET_3_5_VERSION_NAME: "us.anthropic.claude-3-5-sonnet-20241022-v2:0"
  BEDROCK_ANTHROPIC_SONNET_3_7_VERSION_NAME: "us.anthropic.claude-3-7-sonnet-20250219-v1:0"
  BEDROCK_ANTHROPIC_HAIKU_3_5_VERSION_NAME: "us.anthropic.claude-3-5-haiku-20241022-v1:0"
  BEDROCK_ANTHROPIC_HAIKU_4_5_VERSION_NAME: "us.anthropic.claude-haiku-4-5-20251001-v1:0"
  BEDROCK_ANTHROPIC_SONNET_4_0_VERSION_NAME: "us.anthropic.claude-sonnet-4-20250514-v1:0"
  PREFERED_PREMIUM_MODE_MODEL: "anthropic-sonnet-4.0"

Verification

After configuration, verify your AWS Bedrock integration:

Check Admin UI: Verify Claude models appear in LlamaCloud admin interface
Test functionality: Upload a document and use advanced parsing modes

Troubleshooting

Common Issues

Access Denied

Error: User is not authorized to perform bedrock:InvokeModel

Solution:

Verify IAM permissions include all required actions
Check that the resource ARNs include both foundation models and inference profiles
Ensure the role is properly attached to the service account (for IRSA)

Model Not Found

Error: Could not resolve foundation model from model identifier

Solution:

Verify the model ID or inference profile ID is correct
Check that the model is available in your AWS region
Try using an inference profile ID instead of direct model ID

On-Demand Throughput Error

Error: On-demand throughput is not supported for this model

Solution:

This typically means the model requires an inference profile
Switch to using the inference profile ID format
Check the Bedrock console for the correct inference profile ID

Region Issues

Error: Model not available in region

Solution:

Check model availability in your specified AWS region
Consider switching to a region where the model is available
Use cross-region inference profiles for better availability

Quota Exceeded

Error: Throttling exception or quota exceeded

Solution:

Check your Bedrock quotas in AWS Console
Request quota increases if needed
Implement request throttling in your application

Debug Steps

Test AWS credentials directly:

aws bedrock list-foundation-models --region us-east-1

Test model invocation:

aws bedrock-runtime invoke-model \
  --model-id anthropic.claude-3-5-sonnet-20240620-v1:0 \
  --body '{"messages":[{"role":"user","content":[{"type":"text","text":"Hello"}]}],"max_tokens":100,"anthropic_version":"bedrock-2023-05-31"}' \
  --cli-binary-format raw-in-base64-out \
  output.json

Verify service account annotations (for IRSA):
Terminal window
```
kubectl describe serviceaccount llamacloud-llamaparse
```

Check secret mounting:

kubectl describe pod <pod-name> | grep -A 20 Environment