Chat | Developer Documentation

Guide

Index V2 (Beta)

Built-in chat agent that performs RAG over your indexes with session management and streaming responses.

Overview

The chat API provides a built-in RAG agent that can answer questions over one or more of your indexes. It manages conversation sessions, streams responses as Server-Sent Events, and handles retrieval, tool use, and reasoning internally.

Create a session

A session holds the conversation history. You can optionally bind it to specific indexes at creation time — once the first message is sent, the bound indexes are locked for the session’s lifetime.

Python
TypeScript

# Session bound to specific indexes
session = await client.beta.chat.create(
    index_ids=["<index-id-1>", "<index-id-2>"],
)
print(session.session_id)

# Unbound session (indexes specified per-message)
session = await client.beta.chat.create()

// Session bound to specific indexes
const session = await client.beta.chat.create({
  index_ids: ["<index-id-1>", "<index-id-2>"],
});
console.log(session.session_id);

// Unbound session
const unboundSession = await client.beta.chat.create({});

Stream messages

Send a message and receive the agent’s response as a stream of Server-Sent Events. The stream includes thinking steps, tool calls (retrieval), and the final text response.

Use with_streaming_response to get the raw SSE stream and parse events line by line.

Python
TypeScript

import json

response = client.beta.chat.with_streaming_response.stream(
    session.session_id,
    index_ids=["<index-id>"],
    prompt="What are the key findings in the Q3 report?",
)

async with response as stream:
    async for line in stream.iter_lines():
        if not line or line.startswith(":"):
            continue
        if line.startswith("data: "):
            data = json.loads(line[6:])
            if data.get("type") == "text_delta":
                print(data["content"], end="", flush=True)
            elif data.get("type") == "stop":
                print("\n--- Done ---")
            else:
                print(data)  # handle other event types as needed

const response = await client.beta.chat.stream(session.session_id, {
  index_ids: ["<index-id>"],
  prompt: "What are the key findings in the Q3 report?",
}).asResponse();

const reader = response.body!.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  const chunk = decoder.decode(value, { stream: true });
  for (const line of chunk.split("\n")) {
    if (line.startsWith("data: ")) {
      const data = JSON.parse(line.slice(6));
      if (data.type === "text_delta") {
        process.stdout.write(data.content);
      }
      else if (data.type === "stop") {
        console.log("\n--- Done ---");
      } else {
        console.log(data); // handle other event types as needed
      }
    }
  }
}

Event types

The stream emits two categories of events:

Delta events (event: delta) — Incremental content for real-time display:

Type	Description
`text_delta`	A chunk of the agent’s response text.
`thinking_delta`	A chunk of the agent’s reasoning (if exposed).

Message events (event: message) — Complete, discrete events:

Type	Description
`text`	Complete agent response text.
`thinking`	Complete agent reasoning block.
`user_input`	The user’s original message.
`tool_call`	A tool invocation by the agent (e.g., retrieval).
`tool_result`	The result of a tool call.
`stop`	End of the stream, includes token usage.
`warning`	Informational warning (e.g., skipped indexes).

sessions = await client.beta.chat.list()

for s in sessions.items:
    print(f"{s.session_id}: {s.generated_title or '(untitled)'}")

const sessions = await client.beta.chat.list();

for (const s of sessions.items) {
  console.log(`${s.session_id}: ${s.generated_title ?? "(untitled)"}`);
}

Get session details

Retrieve a session summary or the full session with its event history.

Python
TypeScript

# Summary only
summary = await client.beta.chat.get_summary(session.session_id)
print(summary.generated_title)
print(summary.job_metadata)

# Full session with event history
full = await client.beta.chat.retrieve(session.session_id)
for event in full.events:
    print(event.type, event.content[:100] if hasattr(event, "content") else "")

// Summary only
const summary = await client.beta.chat.getSummary(session.session_id);
console.log(summary.generated_title);

// Full session with events
const full = await client.beta.chat.retrieve(session.session_id);
for (const event of full.events) {
  console.log(event.type);
}

Delete a session

Python
TypeScript

await client.beta.chat.delete(session.session_id)

await client.beta.chat.delete(session.session_id);