S3 Connector

The S3 connector syncs objects from a bucket into a connected folder. It signs in with an IAM user’s access key pair and works with S3-compatible stores too, as long as they’re reachable from the internet.

This page covers the web UI. To set up the same connection, a synced folder, and an index or batch over it entirely over the REST API, see Set up a connector with the API.

Create an IAM user for the S3 connector

Make an IAM user for LlamaCloud and give it read access to the buckets you want to sync. It needs two actions:

• s3:ListBucket on the bucket, to see what’s there
• s3:GetObject on the objects, to download them

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "LlamaCloudList",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::your-bucket-name"
    },
    {
      "Sid": "LlamaCloudRead",
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::your-bucket-name/*"
    }
  ]
}

If you only want a folder to sync one prefix, you can narrow both statements to it: add an s3:prefix condition on s3:ListBucket and change the GetObject resource to arn:aws:s3:::your-bucket-name/your-prefix/*. The connector always sends the prefix when it lists, so the condition works.

One key pair can feed folders in any number of buckets, in any region. There’s no region to enter; the connector finds each bucket’s region when you create a folder from it.

Then create an access key for the user. AWS shows the secret once, so copy it before closing the page.

Add the S3 connection

On Files → Connections, click Add connection, pick S3, and fill in:

Field	Required	What to enter
Access key ID	Yes	The IAM user’s access key, e.g. AKIAIOSFODNN7EXAMPLE
Secret access key	Yes	The secret shown when the key was created
Endpoint URL	No	Leave empty for AWS. For an S3-compatible store, its base URL, e.g. https://s3-compatible.example.com

Click Connect. LlamaCloud checks the key with AWS before saving it:

• AWS S3: it calls sts:GetCallerIdentity, which every IAM user can call, and shows the user’s ARN as the account on the Connections tab. A wrong key fails here. A key that’s valid but can’t read your bucket won’t fail until you create a folder.
• S3-compatible stores: it calls ListBuckets, so the key needs permission to list buckets on that store. The account column shows the access key ID.

Use an S3-compatible store with the S3 connector

Set Endpoint URL to the store’s base URL and use the store’s own access key pair. MinIO and Cloudflare R2 both work this way.

The endpoint has to be an http:// or https:// URL that resolves to a public address. A MinIO running on a private network or behind a VPN can’t be reached, and the connection is refused with “S3 endpoint must resolve to a public address.”

Create a folder from an S3 connection

From Files → New folder → Connected folder, pick your S3 connection and fill in:

• Bucket (required), e.g. acme-documents
• Prefix (optional), e.g. invoices/2024/. Only objects whose keys start with it are synced. Leave it empty to sync the whole bucket.

You can paste an s3://bucket/prefix/ URI into the bucket field and it’ll be split for you.

Before the folder is created, the connector lists the bucket with the key. If the bucket doesn’t exist you’ll see a “not found” error, and if the key isn’t allowed to list it you’ll see “access denied”.

How the S3 connector syncs

Every sync lists everything under the bucket and prefix and compares it with what’s already in the folder. New objects are added, objects whose ETag changed are downloaded again, and deleted objects are removed from the folder. Zero-byte “folder” marker objects (keys ending in /) are skipped.

Because every sync is a full listing, a bucket with a very large number of objects takes longer to sync even if nothing changed. Narrowing the folder to a prefix helps.

Troubleshooting the S3 connector

Message	What to check
”AWS STS rejected this access key pair”	The key ID or secret is wrong, or the key has been deactivated.
”s3 bucket ’…’ not found”	The bucket name is misspelled, or the bucket was deleted.
”s3 denied access to bucket ’…’”	The IAM policy is missing s3:ListBucket on the bucket, or an s3:prefix condition doesn’t match the prefix you entered.
Folder syncs but files fail to import	The policy is missing s3:GetObject, or the object resource doesn’t cover the prefix.