Syncing

After you add, update, or remove files in a directory, you need to sync the index to reflect those changes. Syncing re-parses changed files and re-exports updated chunks to the vector store.

The initial sync is triggered automatically when you create an index. Use the sync endpoint for subsequent updates.

Trigger a sync

Python

await client.beta.indexes.sync("<your-index-id>")

TypeScript

await client.beta.indexes.sync("<your-index-id>");

Go

_, err := client.Beta.Indexes.Sync(ctx, "<your-index-id>", llamacloud.BetaIndexSyncParams{})
if err != nil {
  log.Fatal(err)
}

Java

client.beta().indexes().sync("<your-index-id>");

CLI

llp beta:indexes sync --index-id "<your-index-id>"

The sync runs asynchronously. Poll the index to know when it completes — see Typical workflow below for a race-safe polling loop.

Note

Only one sync can run at a time for a given index. If a sync is already in progress, the API returns 409 Conflict. The sync endpoint is also rate limited — repeated calls in quick succession return 429 Too Many Requests. Trigger the sync once and poll the index status; don’t call the sync endpoint in a loop.

The workflow below assumes your sync call was accepted, so the run that finishes is the one covering your changes. If you get a 409 or 429, the sync already running may have started before your files were added — when it finishes, your files can still be missing. The Index a file cookbook shows how to handle that by checking for the file itself.

Typical workflow

1. Upload new files or remove outdated files from the directory.
2. Read the index’s current last_synced_at, then call the sync endpoint once.
3. Poll the index until status is ready and last_synced_at has advanced past the value from step 2. Resume retrieval or chat.

Caution

Don’t poll for status == "ready" alone right after triggering a sync. The index keeps reporting its previous ready state until the new sync starts, so a bare status check can return immediately and report success before any work has happened. Comparing last_synced_at against the value you read before the sync confirms the new run actually finished. Give the loop a timeout and back off between polls so it can’t spin forever.

Only status values ready and failed are stable across releases — treat any other value (pending, syncing, exporting, …) simply as “not done yet”.

Python

import asyncio
import time

# Add a new file to the directory
with open("new-report.pdf", "rb") as f:
    file_obj = await client.files.create(file=f, purpose="user_data")

await client.beta.directories.files.add(
    directory_id,
    file_id=file_obj.id,
)

# Note when the index was last synced. A freshly triggered sync keeps
# reporting the previous "ready" state until it starts, so we wait for
# both a "ready" status and a newer last_synced_at.
before = (await client.beta.indexes.get(index_id)).last_synced_at

# Trigger a sync (call this once — the endpoint is rate limited)
await client.beta.indexes.sync(index_id)

# Poll until this sync finishes, backing off between attempts and
# giving up after a deadline.
deadline = time.monotonic() + 600  # 10 minutes
delay = 2
while True:
    idx = await client.beta.indexes.get(index_id)
    status = idx.metadata["status"] if idx.metadata else "unknown"

    if status == "ready" and idx.last_synced_at != before:
        print("Sync complete!")
        break
    if status == "failed":
        print("Sync failed:", idx.metadata["error_message"])
        break
    if time.monotonic() >= deadline:
        raise TimeoutError("Sync did not complete within 10 minutes")

    await asyncio.sleep(delay)
    delay = min(delay * 2, 30)  # exponential backoff, capped at 30s

TypeScript

import fs from "fs";

// Add a new file
const fileObj = await client.files.create({
  file: fs.createReadStream("new-report.pdf"),
  purpose: "user_data",
});

await client.beta.directories.files.add(directoryId, {
  file_id: fileObj.id,
});

// Note when the index was last synced. A freshly triggered sync keeps
// reporting the previous "ready" state until it starts, so we wait for
// both a "ready" status and a newer last_synced_at.
const before = (await client.beta.indexes.get(indexId)).last_synced_at;

// Trigger a sync (call this once — the endpoint is rate limited)
await client.beta.indexes.sync(indexId);

// Poll until this sync finishes, backing off between attempts and
// giving up after a deadline.
const deadline = Date.now() + 600_000; // 10 minutes
let delay = 2000;
while (true) {
  const idx = await client.beta.indexes.get(indexId);
  const status = (idx.metadata?.status as string) ?? "unknown";

  if (status === "ready" && idx.last_synced_at !== before) {
    console.log("Sync complete!");
    break;
  } else if (status === "failed") {
    console.error("Sync failed:", idx.metadata?.error_message);
    break;
  } else if (Date.now() >= deadline) {
    throw new Error("Sync did not complete within 10 minutes");
  }

  await new Promise((r) => setTimeout(r, delay));
  delay = Math.min(delay * 2, 30_000); // exponential backoff, capped at 30s
}

Go

// Add a new file to the directory
f, err := os.Open("new-report.pdf")
if err != nil {
  log.Fatal(err)
}
defer f.Close()

fileObj, err := client.Files.New(ctx, llamacloud.FileNewParams{
  File:    f,
  Purpose: "user_data",
})
if err != nil {
  log.Fatal(err)
}

_, err = client.Beta.Directories.Files.Add(ctx, directoryID, llamacloud.BetaDirectoryFileAddParams{
  FileID: fileObj.ID,
})
if err != nil {
  log.Fatal(err)
}

// Note when the index was last synced. A freshly triggered sync keeps
// reporting the previous "ready" state until it starts, so we wait for
// both a "ready" status and a newer LastSyncedAt.
idxBefore, err := client.Beta.Indexes.Get(ctx, indexID, llamacloud.BetaIndexGetParams{})
if err != nil {
  log.Fatal(err)
}
before := idxBefore.LastSyncedAt

// Trigger a sync (call this once — the endpoint is rate limited)
_, err = client.Beta.Indexes.Sync(ctx, indexID, llamacloud.BetaIndexSyncParams{})
if err != nil {
  log.Fatal(err)
}

// Poll until this sync finishes, backing off between attempts and
// giving up after a deadline.
deadline := time.Now().Add(10 * time.Minute)
delay := 2 * time.Second
for {
  idx, err := client.Beta.Indexes.Get(ctx, indexID, llamacloud.BetaIndexGetParams{})
  if err != nil {
    log.Fatal(err)
  }

  status, ok := idx.Metadata["status"].(string)
  if !ok {
    status = "unknown"
  }

  if status == "ready" && !idx.LastSyncedAt.Equal(before) {
    fmt.Println("Sync complete!")
    break
  } else if status == "failed" {
    fmt.Println("Sync failed:", idx.Metadata["error_message"])
    break
  } else if time.Now().After(deadline) {
    log.Fatal("Sync did not complete within 10 minutes")
  }

  time.Sleep(delay)
  if delay < 30*time.Second {
    delay *= 2
    if delay > 30*time.Second {
      delay = 30 * time.Second
    }
  }
}

Java

import java.nio.file.Paths;
import java.time.Duration;
import java.time.Instant;
import java.time.OffsetDateTime;
import java.util.Optional;

import ai.llamaindex.llamacloud.core.JsonValue;
import ai.llamaindex.llamacloud.models.beta.directories.files.FileAddParams;
import ai.llamaindex.llamacloud.models.beta.indexes.IndexGetResponse;
import ai.llamaindex.llamacloud.models.files.FileCreateParams;
import ai.llamaindex.llamacloud.models.files.FileCreateResponse;

// Add a new file to the directory
FileCreateResponse fileObj = client.files().create(
    FileCreateParams.builder()
        .file(Paths.get("new-report.pdf"))
        .purpose("user_data")
        .build());

client.beta().directories().files().add(
    directoryId,
    FileAddParams.builder()
        .fileId(fileObj.id())
        .build());

// Note when the index was last synced. A freshly triggered sync keeps
// reporting the previous "ready" state until it starts, so we wait for
// both a "ready" status and a newer lastSyncedAt.
Optional<OffsetDateTime> before = client.beta().indexes().get(indexId).lastSyncedAt();

// Trigger a sync (call this once — the endpoint is rate limited)
client.beta().indexes().sync(indexId);

// Poll until this sync finishes, backing off between attempts and
// giving up after a deadline.
Instant deadline = Instant.now().plus(Duration.ofMinutes(10));
long delayMs = 2000;
while (true) {
    IndexGetResponse idx = client.beta().indexes().get(indexId);
    String status = idx.metadata()
        .map(m -> m._additionalProperties().get("status"))
        .flatMap(v -> ((JsonValue) v).asString())
        .orElse("unknown");

    if (status.equals("ready") && !idx.lastSyncedAt().equals(before)) {
        System.out.println("Sync complete!");
        break;
    } else if (status.equals("failed")) {
        Optional<String> errOpt = idx.metadata()
            .map(m -> m._additionalProperties().get("error_message"))
            .flatMap(v -> ((JsonValue) v).asString());
        System.out.println("Sync failed: " + errOpt.orElse(""));
        break;
    } else if (Instant.now().isAfter(deadline)) {
        throw new RuntimeException("Sync did not complete within 10 minutes");
    }

    Thread.sleep(delayMs);
    delayMs = Math.min(delayMs * 2, 30000); // exponential backoff, capped at 30s
}

CLI

# Add a new file to the directory
FILE_ID=$(llp files create \
  --file ./new-report.pdf \
  --purpose user_data | jq -r '.id')

llp beta:directories:files add \
  --directory-id "$DIRECTORY_ID" \
  --file-id "$FILE_ID"

# Note when the index was last synced. A freshly triggered sync keeps
# reporting the previous "ready" state until it starts, so we wait for
# both a "ready" status and a newer last_synced_at.
BEFORE=$(llp beta:indexes get --index-id "$INDEX_ID" | jq -r '.last_synced_at // ""')

# Trigger a sync (call this once — the endpoint is rate limited)
llp beta:indexes sync --index-id "$INDEX_ID"

# Poll until this sync finishes, backing off between attempts and
# giving up after a deadline.
DEADLINE=$(( $(date +%s) + 600 ))  # 10 minutes
DELAY=2
while true; do
  IDX=$(llp beta:indexes get --index-id "$INDEX_ID")
  STATUS=$(echo "$IDX" | jq -r '.metadata.status // "unknown"')
  SYNCED_AT=$(echo "$IDX" | jq -r '.last_synced_at // ""')

  if [ "$STATUS" = "ready" ] && [ "$SYNCED_AT" != "$BEFORE" ]; then
    echo "Sync complete!"
    break
  elif [ "$STATUS" = "failed" ]; then
    echo "Sync failed: $(echo "$IDX" | jq -r '.metadata.error_message // ""')"
    break
  elif [ "$(date +%s)" -ge "$DEADLINE" ]; then
    echo "Sync did not complete within 10 minutes" >&2
    exit 1
  fi

  sleep "$DELAY"
  DELAY=$(( DELAY * 2 ))
  [ "$DELAY" -gt 30 ] && DELAY=30
done

Managing indexes

Get an index

Python

# Get an index by ID
idx = await client.beta.indexes.get("<your-index-id>")
print(f"{idx.id}: status={idx.metadata['status']}")

TypeScript

// Get an index by ID
const idx = await client.beta.indexes.get("<your-index-id>");
console.log(`${idx.id}: status=${idx.metadata?.status}`);

Go

// Get an index by ID
idx, err := client.Beta.Indexes.Get(ctx, "<your-index-id>", llamacloud.BetaIndexGetParams{})
if err != nil {
  log.Fatal(err)
}
fmt.Printf("%s: status=%v\n", idx.ID, idx.Metadata["status"])

Java

import ai.llamaindex.llamacloud.core.JsonValue;
import java.util.Optional;
import ai.llamaindex.llamacloud.models.beta.indexes.IndexGetResponse;

// Get an index by ID
IndexGetResponse idx = client.beta().indexes().get("<your-index-id>");
Optional<String> statusOpt = idx.metadata()
    .map(m -> m._additionalProperties().get("status"))
    .flatMap(v -> ((JsonValue) v).asString());
String status = statusOpt.orElse("unknown");
System.out.println(idx.id() + ": status=" + status);

CLI

# Get an index by ID
llp beta:indexes get --index-id "<your-index-id>" \
  | jq -r '"\(.id): status=\(.metadata.status)"'

Delete an index

Deleting an index removes the sync and export configuration. The source directory and its files are not affected.

Python

await client.beta.indexes.delete("<your-index-id>")

TypeScript

await client.beta.indexes.delete("<your-index-id>");

Go

err := client.Beta.Indexes.Delete(ctx, "<your-index-id>", llamacloud.BetaIndexDeleteParams{})
if err != nil {
  log.Fatal(err)
}

Java

client.beta().indexes().delete("<your-index-id>");

CLI

llp beta:indexes delete --index-id "<your-index-id>"