Skip to content
LlamaIndex OSS Documentation
API Reference

Pipelines

Search Pipelines
pipelines.list(PipelineListParams**kwargs) -> PipelineListResponse
GET/api/v1/pipelines
Create Pipeline
pipelines.create(PipelineCreateParams**kwargs) -> Pipeline
POST/api/v1/pipelines
Get Pipeline
pipelines.get(strpipeline_id) -> Pipeline
GET/api/v1/pipelines/{pipeline_id}
Update Existing Pipeline
pipelines.update(strpipeline_id, PipelineUpdateParams**kwargs) -> Pipeline
PUT/api/v1/pipelines/{pipeline_id}
Delete Pipeline
pipelines.delete(strpipeline_id)
DELETE/api/v1/pipelines/{pipeline_id}
Get Pipeline Status
pipelines.get_status(strpipeline_id, PipelineGetStatusParams**kwargs) -> ManagedIngestionStatusResponse
GET/api/v1/pipelines/{pipeline_id}/status
Upsert Pipeline
pipelines.upsert(PipelineUpsertParams**kwargs) -> Pipeline
PUT/api/v1/pipelines
Run Search
pipelines.retrieve(strpipeline_id, PipelineRetrieveParams**kwargs) -> PipelineRetrieveResponse
POST/api/v1/pipelines/{pipeline_id}/retrieve
ModelsExpand Collapse
class AdvancedModeTransformConfig: …
chunking_config: Optional[ChunkingConfig]

Configuration for the chunking.

One of the following:
class ChunkingConfigNoneChunkingConfig: …
mode: Optional[Literal["none"]]
class ChunkingConfigCharacterChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["character"]]
class ChunkingConfigTokenChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["token"]]
separator: Optional[str]
class ChunkingConfigSentenceChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["sentence"]]
paragraph_separator: Optional[str]
separator: Optional[str]
class ChunkingConfigSemanticChunkingConfig: …
breakpoint_percentile_threshold: Optional[int]
buffer_size: Optional[int]
mode: Optional[Literal["semantic"]]
mode: Optional[Literal["advanced"]]
segmentation_config: Optional[SegmentationConfig]

Configuration for the segmentation.

One of the following:
class SegmentationConfigNoneSegmentationConfig: …
mode: Optional[Literal["none"]]
class SegmentationConfigPageSegmentationConfig: …
mode: Optional[Literal["page"]]
page_separator: Optional[str]
class SegmentationConfigElementSegmentationConfig: …
mode: Optional[Literal["element"]]
class AutoTransformConfig: …
chunk_overlap: Optional[int]

Chunk overlap for the transformation.

chunk_size: Optional[int]

Chunk size for the transformation.

exclusiveMinimum0
mode: Optional[Literal["auto"]]
class AzureOpenAIEmbedding: …
additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for Azure deployment.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for Azure OpenAI API.

azure_deployment: Optional[str]

The Azure deployment to use.

azure_endpoint: Optional[str]

The Azure endpoint to use.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
class AzureOpenAIEmbeddingConfig: …
component: Optional[AzureOpenAIEmbedding]

Configuration for the Azure OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for Azure deployment.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for Azure OpenAI API.

azure_deployment: Optional[str]

The Azure deployment to use.

azure_endpoint: Optional[str]

The Azure endpoint to use.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["AZURE_EMBEDDING"]]

Type of the embedding model.

class BedrockEmbedding: …
additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the bedrock client.

aws_access_key_id: Optional[str]

AWS Access Key ID to use

aws_secret_access_key: Optional[str]

AWS Secret Access Key to use

aws_session_token: Optional[str]

AWS Session Token to use

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

The maximum number of API retries.

exclusiveMinimum0
model_name: Optional[str]

The modelId of the Bedrock model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

profile_name: Optional[str]

The name of aws profile to use. If not given, then the default profile is used.

region_name: Optional[str]

AWS region name to use. Uses region configured in AWS CLI if not passed

timeout: Optional[float]

The timeout for the Bedrock API request in seconds. It will be used for both connect and read timeouts.

class BedrockEmbeddingConfig: …
component: Optional[BedrockEmbedding]

Configuration for the Bedrock embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the bedrock client.

aws_access_key_id: Optional[str]

AWS Access Key ID to use

aws_secret_access_key: Optional[str]

AWS Secret Access Key to use

aws_session_token: Optional[str]

AWS Session Token to use

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

The maximum number of API retries.

exclusiveMinimum0
model_name: Optional[str]

The modelId of the Bedrock model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

profile_name: Optional[str]

The name of aws profile to use. If not given, then the default profile is used.

region_name: Optional[str]

AWS region name to use. Uses region configured in AWS CLI if not passed

timeout: Optional[float]

The timeout for the Bedrock API request in seconds. It will be used for both connect and read timeouts.

type: Optional[Literal["BEDROCK_EMBEDDING"]]

Type of the embedding model.

class CohereEmbedding: …
api_key: Optional[str]

The Cohere API key.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embedding_type: Optional[str]

Embedding type. If not provided float embedding_type is used when needed.

input_type: Optional[str]

Model Input type. If not provided, search_document and search_query are used when needed.

model_name: Optional[str]

The modelId of the Cohere model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

truncate: Optional[str]

Truncation type - START/ END/ NONE

class CohereEmbeddingConfig: …
component: Optional[CohereEmbedding]

Configuration for the Cohere embedding model.

api_key: Optional[str]

The Cohere API key.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embedding_type: Optional[str]

Embedding type. If not provided float embedding_type is used when needed.

input_type: Optional[str]

Model Input type. If not provided, search_document and search_query are used when needed.

model_name: Optional[str]

The modelId of the Cohere model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

truncate: Optional[str]

Truncation type - START/ END/ NONE

type: Optional[Literal["COHERE_EMBEDDING"]]

Type of the embedding model.

class DataSinkCreate: …

Schema for creating a data sink.

component: Component

Component that implements the data sink

One of the following:
Dict[str, object]
class CloudPineconeVectorStore: …

Cloud Pinecone Vector Store.

This class is used to store the configuration for a Pinecone vector store, so that it can be created and used in LlamaCloud.

Args: api_key (str): API key for authenticating with Pinecone index_name (str): name of the Pinecone index namespace (optional[str]): namespace to use in the Pinecone index insert_kwargs (optional[dict]): additional kwargs to pass during insertion

api_key: str

The API key for authenticating with Pinecone

formatpassword
index_name: str
class_name: Optional[str]
insert_kwargs: Optional[Dict[str, object]]
namespace: Optional[str]
supports_nested_metadata_filters: Optional[Literal[true]]
class CloudPostgresVectorStore: …
database: str
embed_dim: int
host: str
password: str
port: int
schema_name: str
table_name: str
user: str
class_name: Optional[str]
hnsw_settings: Optional[PgVectorHnswSettings]

HNSW settings for PGVector.

distance_method: Optional[Literal["l2", "ip", "cosine", 3 more]]

The distance method to use.

One of the following:
"l2"
"ip"
"cosine"
"l1"
"hamming"
"jaccard"
ef_construction: Optional[int]

The number of edges to use during the construction phase.

minimum1

The number of edges to use during the search phase.

minimum1
m: Optional[int]

The number of bi-directional links created for each new element.

minimum1
vector_type: Optional[Literal["vector", "half_vec", "bit", "sparse_vec"]]

The type of vector to use.

One of the following:
"vector"
"half_vec"
"bit"
"sparse_vec"
perform_setup: Optional[bool]
supports_nested_metadata_filters: Optional[bool]
class CloudQdrantVectorStore: …

Cloud Qdrant Vector Store.

This class is used to store the configuration for a Qdrant vector store, so that it can be created and used in LlamaCloud.

Args: collection_name (str): name of the Qdrant collection url (str): url of the Qdrant instance api_key (str): API key for authenticating with Qdrant max_retries (int): maximum number of retries in case of a failure. Defaults to 3 client_kwargs (dict): additional kwargs to pass to the Qdrant client

api_key: str
collection_name: str
url: str
class_name: Optional[str]
client_kwargs: Optional[Dict[str, object]]
max_retries: Optional[int]
supports_nested_metadata_filters: Optional[Literal[true]]
class CloudAzureAISearchVectorStore: …

Cloud Azure AI Search Vector Store.

search_service_api_key: str
search_service_endpoint: str
class_name: Optional[str]
client_id: Optional[str]
client_secret: Optional[str]
embedding_dimension: Optional[int]
filterable_metadata_field_keys: Optional[Dict[str, object]]
index_name: Optional[str]
search_service_api_version: Optional[str]
supports_nested_metadata_filters: Optional[Literal[true]]
tenant_id: Optional[str]

Cloud MongoDB Atlas Vector Store.

This class is used to store the configuration for a MongoDB Atlas vector store, so that it can be created and used in LlamaCloud.

Args: mongodb_uri (str): URI for connecting to MongoDB Atlas db_name (str): name of the MongoDB database collection_name (str): name of the MongoDB collection vector_index_name (str): name of the MongoDB Atlas vector index fulltext_index_name (str): name of the MongoDB Atlas full-text index

class CloudMilvusVectorStore: …

Cloud Milvus Vector Store.

uri: str
token: Optional[str]
class_name: Optional[str]
collection_name: Optional[str]
embedding_dimension: Optional[int]
supports_nested_metadata_filters: Optional[bool]
class CloudAstraDBVectorStore: …

Cloud AstraDB Vector Store.

This class is used to store the configuration for an AstraDB vector store, so that it can be created and used in LlamaCloud.

Args: token (str): The Astra DB Application Token to use. api_endpoint (str): The Astra DB JSON API endpoint for your database. collection_name (str): Collection name to use. If not existing, it will be created. embedding_dimension (int): Length of the embedding vectors in use. keyspace (optional[str]): The keyspace to use. If not provided, ‘default_keyspace’

token: str

The Astra DB Application Token to use

formatpassword
api_endpoint: str

The Astra DB JSON API endpoint for your database

collection_name: str

Collection name to use. If not existing, it will be created

embedding_dimension: int

Length of the embedding vectors in use

class_name: Optional[str]
keyspace: Optional[str]

The keyspace to use. If not provided, ‘default_keyspace’

supports_nested_metadata_filters: Optional[Literal[true]]
name: str

The name of the data sink.

sink_type: Literal["PINECONE", "POSTGRES", "QDRANT", 4 more]
One of the following:
"PINECONE"
"POSTGRES"
"QDRANT"
"AZUREAI_SEARCH"
"MONGODB_ATLAS"
"MILVUS"
"ASTRA_DB"
class GeminiEmbedding: …
api_base: Optional[str]

API base to access the model. Defaults to None.

api_key: Optional[str]

API key to access the model. Defaults to None.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
model_name: Optional[str]

The modelId of the Gemini model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

output_dimensionality: Optional[int]

Optional reduced dimension for output embeddings. Supported by models/text-embedding-004 and newer (e.g. gemini-embedding-001). Not supported by models/embedding-001.

task_type: Optional[str]

The task for embedding model.

title: Optional[str]

Title is only applicable for retrieval_document tasks, and is used to represent a document title. For other tasks, title is invalid.

transport: Optional[str]

Transport to access the model. Defaults to None.

class GeminiEmbeddingConfig: …
component: Optional[GeminiEmbedding]

Configuration for the Gemini embedding model.

api_base: Optional[str]

API base to access the model. Defaults to None.

api_key: Optional[str]

API key to access the model. Defaults to None.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
model_name: Optional[str]

The modelId of the Gemini model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

output_dimensionality: Optional[int]

Optional reduced dimension for output embeddings. Supported by models/text-embedding-004 and newer (e.g. gemini-embedding-001). Not supported by models/embedding-001.

task_type: Optional[str]

The task for embedding model.

title: Optional[str]

Title is only applicable for retrieval_document tasks, and is used to represent a document title. For other tasks, title is invalid.

transport: Optional[str]

Transport to access the model. Defaults to None.

type: Optional[Literal["GEMINI_EMBEDDING"]]

Type of the embedding model.

class HuggingFaceInferenceAPIEmbedding: …
token: Optional[Union[str, bool, null]]

Hugging Face token. Will default to the locally saved token. Pass token=False if you don’t want to send your token to the server.

One of the following:
str
bool
class_name: Optional[str]
cookies: Optional[Dict[str, str]]

Additional cookies to send to the server.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
headers: Optional[Dict[str, str]]

Additional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values.

model_name: Optional[str]

Hugging Face model name. If None, the task will be used.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

pooling: Optional[Literal["cls", "mean", "last"]]

Enum of possible pooling choices with pooling behaviors.

One of the following:
"cls"
"mean"
"last"
query_instruction: Optional[str]

Instruction to prepend during query embedding.

task: Optional[str]

Optional task to pick Hugging Face’s recommended model, used when model_name is left as default of None.

text_instruction: Optional[str]

Instruction to prepend during text embedding.

timeout: Optional[float]

The maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.

class HuggingFaceInferenceAPIEmbeddingConfig: …
component: Optional[HuggingFaceInferenceAPIEmbedding]

Configuration for the HuggingFace Inference API embedding model.

token: Optional[Union[str, bool, null]]

Hugging Face token. Will default to the locally saved token. Pass token=False if you don’t want to send your token to the server.

One of the following:
str
bool
class_name: Optional[str]
cookies: Optional[Dict[str, str]]

Additional cookies to send to the server.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
headers: Optional[Dict[str, str]]

Additional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values.

model_name: Optional[str]

Hugging Face model name. If None, the task will be used.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

pooling: Optional[Literal["cls", "mean", "last"]]

Enum of possible pooling choices with pooling behaviors.

One of the following:
"cls"
"mean"
"last"
query_instruction: Optional[str]

Instruction to prepend during query embedding.

task: Optional[str]

Optional task to pick Hugging Face’s recommended model, used when model_name is left as default of None.

text_instruction: Optional[str]

Instruction to prepend during text embedding.

timeout: Optional[float]

The maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.

type: Optional[Literal["HUGGINGFACE_API_EMBEDDING"]]

Type of the embedding model.

class LlamaParseParameters: …
adaptive_long_table: Optional[bool]
aggressive_table_extraction: Optional[bool]
auto_mode: Optional[bool]
auto_mode_configuration_json: Optional[str]
auto_mode_trigger_on_image_in_page: Optional[bool]
auto_mode_trigger_on_regexp_in_page: Optional[str]
auto_mode_trigger_on_table_in_page: Optional[bool]
auto_mode_trigger_on_text_in_page: Optional[str]
azure_openai_api_version: Optional[str]
azure_openai_deployment_name: Optional[str]
azure_openai_endpoint: Optional[str]
azure_openai_key: Optional[str]
bbox_bottom: Optional[float]
bbox_left: Optional[float]
bbox_right: Optional[float]
bbox_top: Optional[float]
bounding_box: Optional[str]
compact_markdown_table: Optional[bool]
complemental_formatting_instruction: Optional[str]
content_guideline_instruction: Optional[str]
continuous_mode: Optional[bool]
disable_image_extraction: Optional[bool]
disable_ocr: Optional[bool]
disable_reconstruction: Optional[bool]
do_not_cache: Optional[bool]
do_not_unroll_columns: Optional[bool]
enable_cost_optimizer: Optional[bool]
extract_charts: Optional[bool]
extract_layout: Optional[bool]
extract_printed_page_number: Optional[bool]
fast_mode: Optional[bool]
formatting_instruction: Optional[str]
gpt4o_api_key: Optional[str]
gpt4o_mode: Optional[bool]
guess_xlsx_sheet_name: Optional[bool]
hide_footers: Optional[bool]
hide_headers: Optional[bool]
high_res_ocr: Optional[bool]
html_make_all_elements_visible: Optional[bool]
html_remove_fixed_elements: Optional[bool]
html_remove_navigation_elements: Optional[bool]
http_proxy: Optional[str]
ignore_document_elements_for_layout_detection: Optional[bool]
images_to_save: Optional[List[Literal["screenshot", "embedded", "layout"]]]
One of the following:
"screenshot"
"embedded"
"layout"
inline_images_in_markdown: Optional[bool]
input_s3_path: Optional[str]
input_s3_region: Optional[str]
input_url: Optional[str]
internal_is_screenshot_job: Optional[bool]
invalidate_cache: Optional[bool]
is_formatting_instruction: Optional[bool]
job_timeout_extra_time_per_page_in_seconds: Optional[float]
job_timeout_in_seconds: Optional[float]
keep_page_separator_when_merging_tables: Optional[bool]
languages: Optional[List[ParsingLanguages]]
One of the following:
"af"
"az"
"bs"
"cs"
"cy"
"da"
"de"
"en"
"es"
"et"
"fr"
"ga"
"hr"
"hu"
"id"
"is"
"it"
"ku"
"la"
"lt"
"lv"
"mi"
"ms"
"mt"
"nl"
"no"
"oc"
"pi"
"pl"
"pt"
"ro"
"rs_latin"
"sk"
"sl"
"sq"
"sv"
"sw"
"tl"
"tr"
"uz"
"vi"
"ar"
"fa"
"ug"
"ur"
"bn"
"as"
"mni"
"ru"
"rs_cyrillic"
"be"
"bg"
"uk"
"mn"
"abq"
"ady"
"kbd"
"ava"
"dar"
"inh"
"che"
"lbe"
"lez"
"tab"
"tjk"
"hi"
"mr"
"ne"
"bh"
"mai"
"ang"
"bho"
"mah"
"sck"
"new"
"gom"
"sa"
"bgc"
"th"
"ch_sim"
"ch_tra"
"ja"
"ko"
"ta"
"te"
"kn"
layout_aware: Optional[bool]
line_level_bounding_box: Optional[bool]
markdown_table_multiline_header_separator: Optional[str]
max_pages: Optional[int]
max_pages_enforced: Optional[int]
merge_tables_across_pages_in_markdown: Optional[bool]
model: Optional[str]
outlined_table_extraction: Optional[bool]
output_pdf_of_document: Optional[bool]
output_s3_path_prefix: Optional[str]
output_s3_region: Optional[str]
output_tables_as_html: Optional[bool]
page_error_tolerance: Optional[float]
page_footer_prefix: Optional[str]
page_footer_suffix: Optional[str]
page_header_prefix: Optional[str]
page_header_suffix: Optional[str]
page_prefix: Optional[str]
page_separator: Optional[str]
page_suffix: Optional[str]
parse_mode: Optional[ParsingMode]

Enum for representing the mode of parsing to be used.

One of the following:
"parse_page_without_llm"
"parse_page_with_llm"
"parse_page_with_lvm"
"parse_page_with_agent"
"parse_page_with_layout_agent"
"parse_document_with_llm"
"parse_document_with_lvm"
"parse_document_with_agent"
parsing_instruction: Optional[str]
precise_bounding_box: Optional[bool]
premium_mode: Optional[bool]
presentation_out_of_bounds_content: Optional[bool]
presentation_skip_embedded_data: Optional[bool]
preserve_layout_alignment_across_pages: Optional[bool]
preserve_very_small_text: Optional[bool]
preset: Optional[str]
priority: Optional[Literal["low", "medium", "high", "critical"]]

The priority for the request. This field may be ignored or overwritten depending on the organization tier.

One of the following:
"low"
"medium"
"high"
"critical"
project_id: Optional[str]
remove_hidden_text: Optional[bool]
replace_failed_page_mode: Optional[FailPageMode]

Enum for representing the different available page error handling modes.

One of the following:
"raw_text"
"blank_page"
"error_message"
replace_failed_page_with_error_message_prefix: Optional[str]
replace_failed_page_with_error_message_suffix: Optional[str]
save_images: Optional[bool]
skip_diagonal_text: Optional[bool]
specialized_chart_parsing_agentic: Optional[bool]
specialized_chart_parsing_efficient: Optional[bool]
specialized_chart_parsing_plus: Optional[bool]
specialized_image_parsing: Optional[bool]
spreadsheet_extract_sub_tables: Optional[bool]
spreadsheet_force_formula_computation: Optional[bool]
spreadsheet_include_hidden_sheets: Optional[bool]
strict_mode_buggy_font: Optional[bool]
strict_mode_image_extraction: Optional[bool]
strict_mode_image_ocr: Optional[bool]
strict_mode_reconstruction: Optional[bool]
structured_output: Optional[bool]
structured_output_json_schema: Optional[str]
structured_output_json_schema_name: Optional[str]
system_prompt: Optional[str]
system_prompt_append: Optional[str]
take_screenshot: Optional[bool]
target_pages: Optional[str]
tier: Optional[str]
use_vendor_multimodal_model: Optional[bool]
user_prompt: Optional[str]
vendor_multimodal_api_key: Optional[str]
vendor_multimodal_model_name: Optional[str]
version: Optional[str]
webhook_configurations: Optional[List[WebhookConfiguration]]

Outbound webhook endpoints to notify on job status changes

webhook_events: Optional[List[Literal["extract.pending", "extract.success", "extract.error", 14 more]]]

Events to subscribe to (e.g. ‘parse.success’, ‘extract.error’). If null, all events are delivered.

One of the following:
"extract.pending"
"extract.success"
"extract.error"
"extract.partial_success"
"extract.cancelled"
"parse.pending"
"parse.running"
"parse.success"
"parse.error"
"parse.partial_success"
"parse.cancelled"
"classify.pending"
"classify.success"
"classify.error"
"classify.partial_success"
"classify.cancelled"
"unmapped_event"
webhook_headers: Optional[Dict[str, str]]

Custom HTTP headers sent with each webhook request (e.g. auth tokens)

webhook_output_format: Optional[str]

Response format sent to the webhook: ‘string’ (default) or ‘json’

webhook_url: Optional[str]

URL to receive webhook POST notifications

webhook_url: Optional[str]
class LlmParameters: …
class_name: Optional[str]
model_name: Optional[Literal["GPT_4O", "GPT_4O_MINI", "GPT_4_1", 10 more]]

The name of the model to use for LLM completions.

One of the following:
"GPT_4O"
"GPT_4O_MINI"
"GPT_4_1"
"GPT_4_1_NANO"
"GPT_4_1_MINI"
"AZURE_OPENAI_GPT_4O"
"AZURE_OPENAI_GPT_4O_MINI"
"AZURE_OPENAI_GPT_4_1"
"AZURE_OPENAI_GPT_4_1_MINI"
"AZURE_OPENAI_GPT_4_1_NANO"
"CLAUDE_4_5_SONNET"
"BEDROCK_CLAUDE_3_5_SONNET_V1"
"BEDROCK_CLAUDE_3_5_SONNET_V2"
system_prompt: Optional[str]

The system prompt to use for the completion.

maxLength3000
temperature: Optional[float]

The temperature value for the model.

use_chain_of_thought_reasoning: Optional[bool]

Whether to use chain of thought reasoning.

use_citation: Optional[bool]

Whether to show citations in the response.

class ManagedIngestionStatusResponse: …
status: Literal["NOT_STARTED", "IN_PROGRESS", "SUCCESS", 3 more]

Status of the ingestion.

One of the following:
"NOT_STARTED"
"IN_PROGRESS"
"SUCCESS"
"ERROR"
"PARTIAL_SUCCESS"
"CANCELLED"
deployment_date: Optional[datetime]

Date of the deployment.

formatdate-time
effective_at: Optional[datetime]

When the status is effective

formatdate-time
error: Optional[List[Error]]

List of errors that occurred during ingestion.

job_id: str

ID of the job that failed.

formatuuid
message: str

List of errors that occurred during ingestion.

step: Literal["MANAGED_INGESTION", "DATA_SOURCE", "FILE_UPDATER", 4 more]

Name of the job that failed.

One of the following:
"MANAGED_INGESTION"
"DATA_SOURCE"
"FILE_UPDATER"
"PARSE"
"TRANSFORM"
"INGESTION"
"METADATA_UPDATE"
job_id: Optional[str]

ID of the latest job.

formatuuid
Literal["system", "developer", "user", 5 more]

Message role.

One of the following:
"system"
"developer"
"user"
"assistant"
"function"
"tool"
"chatbot"
"model"
class MetadataFilters: …

Metadata filters for vector stores.

filters: List[Filter]
One of the following:
class FilterMetadataFilter: …

Comprehensive metadata filter for vector stores to support more operators.

Value uses Strict types, as int, float and str are compatible types and were all converted to string before.

See: https://docs.pydantic.dev/latest/usage/types/#strict-types

key: str
value: Union[float, str, List[str], 3 more]
One of the following:
float
str
List[str]
List[float]
List[int]
operator: Optional[Literal["==", ">", "<", 11 more]]

Vector store filter operator.

One of the following:
"=="
">"
"<"
"!="
">="
"<="
"in"
"nin"
"any"
"all"
"text_match"
"text_match_insensitive"
"contains"
"is_empty"
condition: Optional[Literal["and", "or", "not"]]

Vector store filter conditions to combine different filters.

One of the following:
"and"
"or"
"not"
class OpenAIEmbedding: …
additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for OpenAI API.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for OpenAI API.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
class OpenAIEmbeddingConfig: …
component: Optional[OpenAIEmbedding]

Configuration for the OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for OpenAI API.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for OpenAI API.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["OPENAI_EMBEDDING"]]

Type of the embedding model.

class PageFigureNodeWithScore: …

Page figure metadata with score

node: Node
confidence: float

The confidence of the figure

maximum1
minimum0
figure_name: str

The name of the figure

figure_size: int

The size of the figure in bytes

minimum0
file_id: str

The ID of the file that the figure was taken from

formatuuid
page_index: int

The index of the page for which the figure is taken (0-indexed)

minimum0
is_likely_noise: Optional[bool]

Whether the figure is likely to be noise

metadata: Optional[Dict[str, object]]

Metadata for the figure

score: float

The score of the figure node

class_name: Optional[str]
class PageScreenshotNodeWithScore: …

Page screenshot metadata with score

node: Node
file_id: str

The ID of the file that the page screenshot was taken from

formatuuid
image_size: int

The size of the image in bytes

minimum0
page_index: int

The index of the page for which the screenshot is taken (0-indexed)

minimum0
metadata: Optional[Dict[str, object]]

Metadata for the screenshot

score: float

The score of the screenshot node

class_name: Optional[str]
class Pipeline: …

Schema for a pipeline.

id: str

Unique identifier

formatuuid
embedding_config: EmbeddingConfig
One of the following:
class EmbeddingConfigManagedOpenAIEmbeddingConfig: …
component: Optional[EmbeddingConfigManagedOpenAIEmbeddingConfigComponent]

Configuration for the Managed OpenAI embedding model.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
model_name: Optional[Literal["openai-text-embedding-3-small"]]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

type: Optional[Literal["MANAGED_OPENAI_EMBEDDING"]]

Type of the embedding model.

class AzureOpenAIEmbeddingConfig: …
component: Optional[AzureOpenAIEmbedding]

Configuration for the Azure OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for Azure deployment.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for Azure OpenAI API.

azure_deployment: Optional[str]

The Azure deployment to use.

azure_endpoint: Optional[str]

The Azure endpoint to use.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["AZURE_EMBEDDING"]]

Type of the embedding model.

class CohereEmbeddingConfig: …
component: Optional[CohereEmbedding]

Configuration for the Cohere embedding model.

api_key: Optional[str]

The Cohere API key.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embedding_type: Optional[str]

Embedding type. If not provided float embedding_type is used when needed.

input_type: Optional[str]

Model Input type. If not provided, search_document and search_query are used when needed.

model_name: Optional[str]

The modelId of the Cohere model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

truncate: Optional[str]

Truncation type - START/ END/ NONE

type: Optional[Literal["COHERE_EMBEDDING"]]

Type of the embedding model.

class GeminiEmbeddingConfig: …
component: Optional[GeminiEmbedding]

Configuration for the Gemini embedding model.

api_base: Optional[str]

API base to access the model. Defaults to None.

api_key: Optional[str]

API key to access the model. Defaults to None.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
model_name: Optional[str]

The modelId of the Gemini model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

output_dimensionality: Optional[int]

Optional reduced dimension for output embeddings. Supported by models/text-embedding-004 and newer (e.g. gemini-embedding-001). Not supported by models/embedding-001.

task_type: Optional[str]

The task for embedding model.

title: Optional[str]

Title is only applicable for retrieval_document tasks, and is used to represent a document title. For other tasks, title is invalid.

transport: Optional[str]

Transport to access the model. Defaults to None.

type: Optional[Literal["GEMINI_EMBEDDING"]]

Type of the embedding model.

class HuggingFaceInferenceAPIEmbeddingConfig: …
component: Optional[HuggingFaceInferenceAPIEmbedding]

Configuration for the HuggingFace Inference API embedding model.

token: Optional[Union[str, bool, null]]

Hugging Face token. Will default to the locally saved token. Pass token=False if you don’t want to send your token to the server.

One of the following:
str
bool
class_name: Optional[str]
cookies: Optional[Dict[str, str]]

Additional cookies to send to the server.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
headers: Optional[Dict[str, str]]

Additional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values.

model_name: Optional[str]

Hugging Face model name. If None, the task will be used.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

pooling: Optional[Literal["cls", "mean", "last"]]

Enum of possible pooling choices with pooling behaviors.

One of the following:
"cls"
"mean"
"last"
query_instruction: Optional[str]

Instruction to prepend during query embedding.

task: Optional[str]

Optional task to pick Hugging Face’s recommended model, used when model_name is left as default of None.

text_instruction: Optional[str]

Instruction to prepend during text embedding.

timeout: Optional[float]

The maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.

type: Optional[Literal["HUGGINGFACE_API_EMBEDDING"]]

Type of the embedding model.

class OpenAIEmbeddingConfig: …
component: Optional[OpenAIEmbedding]

Configuration for the OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for OpenAI API.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for OpenAI API.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["OPENAI_EMBEDDING"]]

Type of the embedding model.

class VertexAIEmbeddingConfig: …
component: Optional[VertexTextEmbedding]

Configuration for the VertexAI embedding model.

client_email: Optional[str]

The client email for the VertexAI credentials.

location: str

The default location to use when making API calls.

private_key: Optional[str]

The private key for the VertexAI credentials.

private_key_id: Optional[str]

The private key ID for the VertexAI credentials.

project: str

The default GCP project to use when making Vertex API calls.

token_uri: Optional[str]

The token URI for the VertexAI credentials.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the Vertex.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embed_mode: Optional[Literal["default", "classification", "clustering", 2 more]]

The embedding mode to use.

One of the following:
"default"
"classification"
"clustering"
"similarity"
"retrieval"
model_name: Optional[str]

The modelId of the VertexAI model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

type: Optional[Literal["VERTEXAI_EMBEDDING"]]

Type of the embedding model.

class BedrockEmbeddingConfig: …
component: Optional[BedrockEmbedding]

Configuration for the Bedrock embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the bedrock client.

aws_access_key_id: Optional[str]

AWS Access Key ID to use

aws_secret_access_key: Optional[str]

AWS Secret Access Key to use

aws_session_token: Optional[str]

AWS Session Token to use

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

The maximum number of API retries.

exclusiveMinimum0
model_name: Optional[str]

The modelId of the Bedrock model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

profile_name: Optional[str]

The name of aws profile to use. If not given, then the default profile is used.

region_name: Optional[str]

AWS region name to use. Uses region configured in AWS CLI if not passed

timeout: Optional[float]

The timeout for the Bedrock API request in seconds. It will be used for both connect and read timeouts.

type: Optional[Literal["BEDROCK_EMBEDDING"]]

Type of the embedding model.

name: str
project_id: str
config_hash: Optional[ConfigHash]

Hashes for the configuration of a pipeline.

embedding_config_hash: Optional[str]

Hash of the embedding config.

parsing_config_hash: Optional[str]

Hash of the llama parse parameters.

transform_config_hash: Optional[str]

Hash of the transform config.

created_at: Optional[datetime]

Creation datetime

formatdate-time
data_sink: Optional[DataSink]

Schema for a data sink.

id: str

Unique identifier

formatuuid
component: Component

Component that implements the data sink

One of the following:
Dict[str, object]
class CloudPineconeVectorStore: …

Cloud Pinecone Vector Store.

This class is used to store the configuration for a Pinecone vector store, so that it can be created and used in LlamaCloud.

Args: api_key (str): API key for authenticating with Pinecone index_name (str): name of the Pinecone index namespace (optional[str]): namespace to use in the Pinecone index insert_kwargs (optional[dict]): additional kwargs to pass during insertion

api_key: str

The API key for authenticating with Pinecone

formatpassword
index_name: str
class_name: Optional[str]
insert_kwargs: Optional[Dict[str, object]]
namespace: Optional[str]
supports_nested_metadata_filters: Optional[Literal[true]]
class CloudPostgresVectorStore: …
database: str
embed_dim: int
host: str
password: str
port: int
schema_name: str
table_name: str
user: str
class_name: Optional[str]
hnsw_settings: Optional[PgVectorHnswSettings]

HNSW settings for PGVector.

distance_method: Optional[Literal["l2", "ip", "cosine", 3 more]]

The distance method to use.

One of the following:
"l2"
"ip"
"cosine"
"l1"
"hamming"
"jaccard"
ef_construction: Optional[int]

The number of edges to use during the construction phase.

minimum1

The number of edges to use during the search phase.

minimum1
m: Optional[int]

The number of bi-directional links created for each new element.

minimum1
vector_type: Optional[Literal["vector", "half_vec", "bit", "sparse_vec"]]

The type of vector to use.

One of the following:
"vector"
"half_vec"
"bit"
"sparse_vec"
perform_setup: Optional[bool]
supports_nested_metadata_filters: Optional[bool]
class CloudQdrantVectorStore: …

Cloud Qdrant Vector Store.

This class is used to store the configuration for a Qdrant vector store, so that it can be created and used in LlamaCloud.

Args: collection_name (str): name of the Qdrant collection url (str): url of the Qdrant instance api_key (str): API key for authenticating with Qdrant max_retries (int): maximum number of retries in case of a failure. Defaults to 3 client_kwargs (dict): additional kwargs to pass to the Qdrant client

api_key: str
collection_name: str
url: str
class_name: Optional[str]
client_kwargs: Optional[Dict[str, object]]
max_retries: Optional[int]
supports_nested_metadata_filters: Optional[Literal[true]]
class CloudAzureAISearchVectorStore: …

Cloud Azure AI Search Vector Store.

search_service_api_key: str
search_service_endpoint: str
class_name: Optional[str]
client_id: Optional[str]
client_secret: Optional[str]
embedding_dimension: Optional[int]
filterable_metadata_field_keys: Optional[Dict[str, object]]
index_name: Optional[str]
search_service_api_version: Optional[str]
supports_nested_metadata_filters: Optional[Literal[true]]
tenant_id: Optional[str]

Cloud MongoDB Atlas Vector Store.

This class is used to store the configuration for a MongoDB Atlas vector store, so that it can be created and used in LlamaCloud.

Args: mongodb_uri (str): URI for connecting to MongoDB Atlas db_name (str): name of the MongoDB database collection_name (str): name of the MongoDB collection vector_index_name (str): name of the MongoDB Atlas vector index fulltext_index_name (str): name of the MongoDB Atlas full-text index

class CloudMilvusVectorStore: …

Cloud Milvus Vector Store.

uri: str
token: Optional[str]
class_name: Optional[str]
collection_name: Optional[str]
embedding_dimension: Optional[int]
supports_nested_metadata_filters: Optional[bool]
class CloudAstraDBVectorStore: …

Cloud AstraDB Vector Store.

This class is used to store the configuration for an AstraDB vector store, so that it can be created and used in LlamaCloud.

Args: token (str): The Astra DB Application Token to use. api_endpoint (str): The Astra DB JSON API endpoint for your database. collection_name (str): Collection name to use. If not existing, it will be created. embedding_dimension (int): Length of the embedding vectors in use. keyspace (optional[str]): The keyspace to use. If not provided, ‘default_keyspace’

token: str

The Astra DB Application Token to use

formatpassword
api_endpoint: str

The Astra DB JSON API endpoint for your database

collection_name: str

Collection name to use. If not existing, it will be created

embedding_dimension: int

Length of the embedding vectors in use

class_name: Optional[str]
keyspace: Optional[str]

The keyspace to use. If not provided, ‘default_keyspace’

supports_nested_metadata_filters: Optional[Literal[true]]
name: str

The name of the data sink.

project_id: str
sink_type: Literal["PINECONE", "POSTGRES", "QDRANT", 4 more]
One of the following:
"PINECONE"
"POSTGRES"
"QDRANT"
"AZUREAI_SEARCH"
"MONGODB_ATLAS"
"MILVUS"
"ASTRA_DB"
created_at: Optional[datetime]

Creation datetime

formatdate-time
updated_at: Optional[datetime]

Update datetime

formatdate-time
embedding_model_config: Optional[EmbeddingModelConfig]

Schema for an embedding model config.

id: str

Unique identifier

formatuuid
embedding_config: EmbeddingModelConfigEmbeddingConfig

The embedding configuration for the embedding model config.

One of the following:
class AzureOpenAIEmbeddingConfig: …
component: Optional[AzureOpenAIEmbedding]

Configuration for the Azure OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for Azure deployment.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for Azure OpenAI API.

azure_deployment: Optional[str]

The Azure deployment to use.

azure_endpoint: Optional[str]

The Azure endpoint to use.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["AZURE_EMBEDDING"]]

Type of the embedding model.

class CohereEmbeddingConfig: …
component: Optional[CohereEmbedding]

Configuration for the Cohere embedding model.

api_key: Optional[str]

The Cohere API key.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embedding_type: Optional[str]

Embedding type. If not provided float embedding_type is used when needed.

input_type: Optional[str]

Model Input type. If not provided, search_document and search_query are used when needed.

model_name: Optional[str]

The modelId of the Cohere model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

truncate: Optional[str]

Truncation type - START/ END/ NONE

type: Optional[Literal["COHERE_EMBEDDING"]]

Type of the embedding model.

class GeminiEmbeddingConfig: …
component: Optional[GeminiEmbedding]

Configuration for the Gemini embedding model.

api_base: Optional[str]

API base to access the model. Defaults to None.

api_key: Optional[str]

API key to access the model. Defaults to None.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
model_name: Optional[str]

The modelId of the Gemini model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

output_dimensionality: Optional[int]

Optional reduced dimension for output embeddings. Supported by models/text-embedding-004 and newer (e.g. gemini-embedding-001). Not supported by models/embedding-001.

task_type: Optional[str]

The task for embedding model.

title: Optional[str]

Title is only applicable for retrieval_document tasks, and is used to represent a document title. For other tasks, title is invalid.

transport: Optional[str]

Transport to access the model. Defaults to None.

type: Optional[Literal["GEMINI_EMBEDDING"]]

Type of the embedding model.

class HuggingFaceInferenceAPIEmbeddingConfig: …
component: Optional[HuggingFaceInferenceAPIEmbedding]

Configuration for the HuggingFace Inference API embedding model.

token: Optional[Union[str, bool, null]]

Hugging Face token. Will default to the locally saved token. Pass token=False if you don’t want to send your token to the server.

One of the following:
str
bool
class_name: Optional[str]
cookies: Optional[Dict[str, str]]

Additional cookies to send to the server.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
headers: Optional[Dict[str, str]]

Additional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values.

model_name: Optional[str]

Hugging Face model name. If None, the task will be used.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

pooling: Optional[Literal["cls", "mean", "last"]]

Enum of possible pooling choices with pooling behaviors.

One of the following:
"cls"
"mean"
"last"
query_instruction: Optional[str]

Instruction to prepend during query embedding.

task: Optional[str]

Optional task to pick Hugging Face’s recommended model, used when model_name is left as default of None.

text_instruction: Optional[str]

Instruction to prepend during text embedding.

timeout: Optional[float]

The maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.

type: Optional[Literal["HUGGINGFACE_API_EMBEDDING"]]

Type of the embedding model.

class OpenAIEmbeddingConfig: …
component: Optional[OpenAIEmbedding]

Configuration for the OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for OpenAI API.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for OpenAI API.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["OPENAI_EMBEDDING"]]

Type of the embedding model.

class VertexAIEmbeddingConfig: …
component: Optional[VertexTextEmbedding]

Configuration for the VertexAI embedding model.

client_email: Optional[str]

The client email for the VertexAI credentials.

location: str

The default location to use when making API calls.

private_key: Optional[str]

The private key for the VertexAI credentials.

private_key_id: Optional[str]

The private key ID for the VertexAI credentials.

project: str

The default GCP project to use when making Vertex API calls.

token_uri: Optional[str]

The token URI for the VertexAI credentials.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the Vertex.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embed_mode: Optional[Literal["default", "classification", "clustering", 2 more]]

The embedding mode to use.

One of the following:
"default"
"classification"
"clustering"
"similarity"
"retrieval"
model_name: Optional[str]

The modelId of the VertexAI model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

type: Optional[Literal["VERTEXAI_EMBEDDING"]]

Type of the embedding model.

class BedrockEmbeddingConfig: …
component: Optional[BedrockEmbedding]

Configuration for the Bedrock embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the bedrock client.

aws_access_key_id: Optional[str]

AWS Access Key ID to use

aws_secret_access_key: Optional[str]

AWS Secret Access Key to use

aws_session_token: Optional[str]

AWS Session Token to use

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

The maximum number of API retries.

exclusiveMinimum0
model_name: Optional[str]

The modelId of the Bedrock model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

profile_name: Optional[str]

The name of aws profile to use. If not given, then the default profile is used.

region_name: Optional[str]

AWS region name to use. Uses region configured in AWS CLI if not passed

timeout: Optional[float]

The timeout for the Bedrock API request in seconds. It will be used for both connect and read timeouts.

type: Optional[Literal["BEDROCK_EMBEDDING"]]

Type of the embedding model.

name: str

The name of the embedding model config.

project_id: str
created_at: Optional[datetime]

Creation datetime

formatdate-time
updated_at: Optional[datetime]

Update datetime

formatdate-time
embedding_model_config_id: Optional[str]

The ID of the EmbeddingModelConfig this pipeline is using.

formatuuid
llama_parse_parameters: Optional[LlamaParseParameters]

Settings that can be configured for how to use LlamaParse to parse files within a LlamaCloud pipeline.

adaptive_long_table: Optional[bool]
aggressive_table_extraction: Optional[bool]
auto_mode: Optional[bool]
auto_mode_configuration_json: Optional[str]
auto_mode_trigger_on_image_in_page: Optional[bool]
auto_mode_trigger_on_regexp_in_page: Optional[str]
auto_mode_trigger_on_table_in_page: Optional[bool]
auto_mode_trigger_on_text_in_page: Optional[str]
azure_openai_api_version: Optional[str]
azure_openai_deployment_name: Optional[str]
azure_openai_endpoint: Optional[str]
azure_openai_key: Optional[str]
bbox_bottom: Optional[float]
bbox_left: Optional[float]
bbox_right: Optional[float]
bbox_top: Optional[float]
bounding_box: Optional[str]
compact_markdown_table: Optional[bool]
complemental_formatting_instruction: Optional[str]
content_guideline_instruction: Optional[str]
continuous_mode: Optional[bool]
disable_image_extraction: Optional[bool]
disable_ocr: Optional[bool]
disable_reconstruction: Optional[bool]
do_not_cache: Optional[bool]
do_not_unroll_columns: Optional[bool]
enable_cost_optimizer: Optional[bool]
extract_charts: Optional[bool]
extract_layout: Optional[bool]
extract_printed_page_number: Optional[bool]
fast_mode: Optional[bool]
formatting_instruction: Optional[str]
gpt4o_api_key: Optional[str]
gpt4o_mode: Optional[bool]
guess_xlsx_sheet_name: Optional[bool]
hide_footers: Optional[bool]
hide_headers: Optional[bool]
high_res_ocr: Optional[bool]
html_make_all_elements_visible: Optional[bool]
html_remove_fixed_elements: Optional[bool]
html_remove_navigation_elements: Optional[bool]
http_proxy: Optional[str]
ignore_document_elements_for_layout_detection: Optional[bool]
images_to_save: Optional[List[Literal["screenshot", "embedded", "layout"]]]
One of the following:
"screenshot"
"embedded"
"layout"
inline_images_in_markdown: Optional[bool]
input_s3_path: Optional[str]
input_s3_region: Optional[str]
input_url: Optional[str]
internal_is_screenshot_job: Optional[bool]
invalidate_cache: Optional[bool]
is_formatting_instruction: Optional[bool]
job_timeout_extra_time_per_page_in_seconds: Optional[float]
job_timeout_in_seconds: Optional[float]
keep_page_separator_when_merging_tables: Optional[bool]
languages: Optional[List[ParsingLanguages]]
One of the following:
"af"
"az"
"bs"
"cs"
"cy"
"da"
"de"
"en"
"es"
"et"
"fr"
"ga"
"hr"
"hu"
"id"
"is"
"it"
"ku"
"la"
"lt"
"lv"
"mi"
"ms"
"mt"
"nl"
"no"
"oc"
"pi"
"pl"
"pt"
"ro"
"rs_latin"
"sk"
"sl"
"sq"
"sv"
"sw"
"tl"
"tr"
"uz"
"vi"
"ar"
"fa"
"ug"
"ur"
"bn"
"as"
"mni"
"ru"
"rs_cyrillic"
"be"
"bg"
"uk"
"mn"
"abq"
"ady"
"kbd"
"ava"
"dar"
"inh"
"che"
"lbe"
"lez"
"tab"
"tjk"
"hi"
"mr"
"ne"
"bh"
"mai"
"ang"
"bho"
"mah"
"sck"
"new"
"gom"
"sa"
"bgc"
"th"
"ch_sim"
"ch_tra"
"ja"
"ko"
"ta"
"te"
"kn"
layout_aware: Optional[bool]
line_level_bounding_box: Optional[bool]
markdown_table_multiline_header_separator: Optional[str]
max_pages: Optional[int]
max_pages_enforced: Optional[int]
merge_tables_across_pages_in_markdown: Optional[bool]
model: Optional[str]
outlined_table_extraction: Optional[bool]
output_pdf_of_document: Optional[bool]
output_s3_path_prefix: Optional[str]
output_s3_region: Optional[str]
output_tables_as_html: Optional[bool]
page_error_tolerance: Optional[float]
page_footer_prefix: Optional[str]
page_footer_suffix: Optional[str]
page_header_prefix: Optional[str]
page_header_suffix: Optional[str]
page_prefix: Optional[str]
page_separator: Optional[str]
page_suffix: Optional[str]
parse_mode: Optional[ParsingMode]

Enum for representing the mode of parsing to be used.

One of the following:
"parse_page_without_llm"
"parse_page_with_llm"
"parse_page_with_lvm"
"parse_page_with_agent"
"parse_page_with_layout_agent"
"parse_document_with_llm"
"parse_document_with_lvm"
"parse_document_with_agent"
parsing_instruction: Optional[str]
precise_bounding_box: Optional[bool]
premium_mode: Optional[bool]
presentation_out_of_bounds_content: Optional[bool]
presentation_skip_embedded_data: Optional[bool]
preserve_layout_alignment_across_pages: Optional[bool]
preserve_very_small_text: Optional[bool]
preset: Optional[str]
priority: Optional[Literal["low", "medium", "high", "critical"]]

The priority for the request. This field may be ignored or overwritten depending on the organization tier.

One of the following:
"low"
"medium"
"high"
"critical"
project_id: Optional[str]
remove_hidden_text: Optional[bool]
replace_failed_page_mode: Optional[FailPageMode]

Enum for representing the different available page error handling modes.

One of the following:
"raw_text"
"blank_page"
"error_message"
replace_failed_page_with_error_message_prefix: Optional[str]
replace_failed_page_with_error_message_suffix: Optional[str]
save_images: Optional[bool]
skip_diagonal_text: Optional[bool]
specialized_chart_parsing_agentic: Optional[bool]
specialized_chart_parsing_efficient: Optional[bool]
specialized_chart_parsing_plus: Optional[bool]
specialized_image_parsing: Optional[bool]
spreadsheet_extract_sub_tables: Optional[bool]
spreadsheet_force_formula_computation: Optional[bool]
spreadsheet_include_hidden_sheets: Optional[bool]
strict_mode_buggy_font: Optional[bool]
strict_mode_image_extraction: Optional[bool]
strict_mode_image_ocr: Optional[bool]
strict_mode_reconstruction: Optional[bool]
structured_output: Optional[bool]
structured_output_json_schema: Optional[str]
structured_output_json_schema_name: Optional[str]
system_prompt: Optional[str]
system_prompt_append: Optional[str]
take_screenshot: Optional[bool]
target_pages: Optional[str]
tier: Optional[str]
use_vendor_multimodal_model: Optional[bool]
user_prompt: Optional[str]
vendor_multimodal_api_key: Optional[str]
vendor_multimodal_model_name: Optional[str]
version: Optional[str]
webhook_configurations: Optional[List[WebhookConfiguration]]

Outbound webhook endpoints to notify on job status changes

webhook_events: Optional[List[Literal["extract.pending", "extract.success", "extract.error", 14 more]]]

Events to subscribe to (e.g. ‘parse.success’, ‘extract.error’). If null, all events are delivered.

One of the following:
"extract.pending"
"extract.success"
"extract.error"
"extract.partial_success"
"extract.cancelled"
"parse.pending"
"parse.running"
"parse.success"
"parse.error"
"parse.partial_success"
"parse.cancelled"
"classify.pending"
"classify.success"
"classify.error"
"classify.partial_success"
"classify.cancelled"
"unmapped_event"
webhook_headers: Optional[Dict[str, str]]

Custom HTTP headers sent with each webhook request (e.g. auth tokens)

webhook_output_format: Optional[str]

Response format sent to the webhook: ‘string’ (default) or ‘json’

webhook_url: Optional[str]

URL to receive webhook POST notifications

webhook_url: Optional[str]
managed_pipeline_id: Optional[str]

The ID of the ManagedPipeline this playground pipeline is linked to.

formatuuid
metadata_config: Optional[PipelineMetadataConfig]

Metadata configuration for the pipeline.

excluded_embed_metadata_keys: Optional[List[str]]

List of metadata keys to exclude from embeddings

excluded_llm_metadata_keys: Optional[List[str]]

List of metadata keys to exclude from LLM during retrieval

pipeline_type: Optional[PipelineType]

Type of pipeline. Either PLAYGROUND or MANAGED.

One of the following:
"PLAYGROUND"
"MANAGED"
preset_retrieval_parameters: Optional[PresetRetrievalParams]

Preset retrieval parameters for the pipeline.

alpha: Optional[float]

Alpha value for hybrid retrieval to determine the weights between dense and sparse retrieval. 0 is sparse retrieval and 1 is dense retrieval.

maximum1
minimum0
class_name: Optional[str]
dense_similarity_cutoff: Optional[float]

Minimum similarity score wrt query for retrieval

maximum1
minimum0
dense_similarity_top_k: Optional[int]

Number of nodes for dense retrieval.

maximum100
minimum1
enable_reranking: Optional[bool]

Enable reranking for retrieval

files_top_k: Optional[int]

Number of files to retrieve (only for retrieval mode files_via_metadata and files_via_content).

maximum5
minimum1
rerank_top_n: Optional[int]

Number of reranked nodes for returning.

maximum100
minimum1
retrieval_mode: Optional[RetrievalMode]

The retrieval mode for the query.

One of the following:
"chunks"
"files_via_metadata"
"files_via_content"
"auto_routed"
Deprecatedretrieve_image_nodes: Optional[bool]

Whether to retrieve image nodes.

retrieve_page_figure_nodes: Optional[bool]

Whether to retrieve page figure nodes.

retrieve_page_screenshot_nodes: Optional[bool]

Whether to retrieve page screenshot nodes.

search_filters: Optional[MetadataFilters]

Metadata filters for vector stores.

filters: List[Filter]
One of the following:
class FilterMetadataFilter: …

Comprehensive metadata filter for vector stores to support more operators.

Value uses Strict types, as int, float and str are compatible types and were all converted to string before.

See: https://docs.pydantic.dev/latest/usage/types/#strict-types

key: str
value: Union[float, str, List[str], 3 more]
One of the following:
float
str
List[str]
List[float]
List[int]
operator: Optional[Literal["==", ">", "<", 11 more]]

Vector store filter operator.

One of the following:
"=="
">"
"<"
"!="
">="
"<="
"in"
"nin"
"any"
"all"
"text_match"
"text_match_insensitive"
"contains"
"is_empty"
condition: Optional[Literal["and", "or", "not"]]

Vector store filter conditions to combine different filters.

One of the following:
"and"
"or"
"not"
search_filters_inference_schema: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

JSON Schema that will be used to infer search_filters. Omit or leave as null to skip inference.

One of the following:
Dict[str, object]
List[object]
str
float
bool
sparse_similarity_top_k: Optional[int]

Number of nodes for sparse retrieval.

maximum100
minimum1
sparse_model_config: Optional[SparseModelConfig]

Configuration for sparse embedding models used in hybrid search.

This allows users to choose between Splade and BM25 models for sparse retrieval in managed data sinks.

class_name: Optional[str]
model_type: Optional[Literal["splade", "bm25", "auto"]]

The sparse model type to use. ‘bm25’ uses Qdrant’s FastEmbed BM25 model (default for new pipelines), ‘splade’ uses HuggingFace Splade model, ‘auto’ selects based on deployment mode (BYOC uses term frequency, Cloud uses Splade).

One of the following:
"splade"
"bm25"
"auto"
status: Optional[Literal["CREATED", "DELETING"]]

Status of the pipeline.

One of the following:
"CREATED"
"DELETING"
transform_config: Optional[TransformConfig]

Configuration for the transformation.

One of the following:
class AutoTransformConfig: …
chunk_overlap: Optional[int]

Chunk overlap for the transformation.

chunk_size: Optional[int]

Chunk size for the transformation.

exclusiveMinimum0
mode: Optional[Literal["auto"]]
class AdvancedModeTransformConfig: …
chunking_config: Optional[ChunkingConfig]

Configuration for the chunking.

One of the following:
class ChunkingConfigNoneChunkingConfig: …
mode: Optional[Literal["none"]]
class ChunkingConfigCharacterChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["character"]]
class ChunkingConfigTokenChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["token"]]
separator: Optional[str]
class ChunkingConfigSentenceChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["sentence"]]
paragraph_separator: Optional[str]
separator: Optional[str]
class ChunkingConfigSemanticChunkingConfig: …
breakpoint_percentile_threshold: Optional[int]
buffer_size: Optional[int]
mode: Optional[Literal["semantic"]]
mode: Optional[Literal["advanced"]]
segmentation_config: Optional[SegmentationConfig]

Configuration for the segmentation.

One of the following:
class SegmentationConfigNoneSegmentationConfig: …
mode: Optional[Literal["none"]]
class SegmentationConfigPageSegmentationConfig: …
mode: Optional[Literal["page"]]
page_separator: Optional[str]
class SegmentationConfigElementSegmentationConfig: …
mode: Optional[Literal["element"]]
updated_at: Optional[datetime]

Update datetime

formatdate-time
class PipelineCreate: …

Schema for creating a pipeline.

name: str
data_sink: Optional[DataSinkCreate]

Schema for creating a data sink.

component: Component

Component that implements the data sink

One of the following:
Dict[str, object]
class CloudPineconeVectorStore: …

Cloud Pinecone Vector Store.

This class is used to store the configuration for a Pinecone vector store, so that it can be created and used in LlamaCloud.

Args: api_key (str): API key for authenticating with Pinecone index_name (str): name of the Pinecone index namespace (optional[str]): namespace to use in the Pinecone index insert_kwargs (optional[dict]): additional kwargs to pass during insertion

api_key: str

The API key for authenticating with Pinecone

formatpassword
index_name: str
class_name: Optional[str]
insert_kwargs: Optional[Dict[str, object]]
namespace: Optional[str]
supports_nested_metadata_filters: Optional[Literal[true]]
class CloudPostgresVectorStore: …
database: str
embed_dim: int
host: str
password: str
port: int
schema_name: str
table_name: str
user: str
class_name: Optional[str]
hnsw_settings: Optional[PgVectorHnswSettings]

HNSW settings for PGVector.

distance_method: Optional[Literal["l2", "ip", "cosine", 3 more]]

The distance method to use.

One of the following:
"l2"
"ip"
"cosine"
"l1"
"hamming"
"jaccard"
ef_construction: Optional[int]

The number of edges to use during the construction phase.

minimum1

The number of edges to use during the search phase.

minimum1
m: Optional[int]

The number of bi-directional links created for each new element.

minimum1
vector_type: Optional[Literal["vector", "half_vec", "bit", "sparse_vec"]]

The type of vector to use.

One of the following:
"vector"
"half_vec"
"bit"
"sparse_vec"
perform_setup: Optional[bool]
supports_nested_metadata_filters: Optional[bool]
class CloudQdrantVectorStore: …

Cloud Qdrant Vector Store.

This class is used to store the configuration for a Qdrant vector store, so that it can be created and used in LlamaCloud.

Args: collection_name (str): name of the Qdrant collection url (str): url of the Qdrant instance api_key (str): API key for authenticating with Qdrant max_retries (int): maximum number of retries in case of a failure. Defaults to 3 client_kwargs (dict): additional kwargs to pass to the Qdrant client

api_key: str
collection_name: str
url: str
class_name: Optional[str]
client_kwargs: Optional[Dict[str, object]]
max_retries: Optional[int]
supports_nested_metadata_filters: Optional[Literal[true]]
class CloudAzureAISearchVectorStore: …

Cloud Azure AI Search Vector Store.

search_service_api_key: str
search_service_endpoint: str
class_name: Optional[str]
client_id: Optional[str]
client_secret: Optional[str]
embedding_dimension: Optional[int]
filterable_metadata_field_keys: Optional[Dict[str, object]]
index_name: Optional[str]
search_service_api_version: Optional[str]
supports_nested_metadata_filters: Optional[Literal[true]]
tenant_id: Optional[str]

Cloud MongoDB Atlas Vector Store.

This class is used to store the configuration for a MongoDB Atlas vector store, so that it can be created and used in LlamaCloud.

Args: mongodb_uri (str): URI for connecting to MongoDB Atlas db_name (str): name of the MongoDB database collection_name (str): name of the MongoDB collection vector_index_name (str): name of the MongoDB Atlas vector index fulltext_index_name (str): name of the MongoDB Atlas full-text index

class CloudMilvusVectorStore: …

Cloud Milvus Vector Store.

uri: str
token: Optional[str]
class_name: Optional[str]
collection_name: Optional[str]
embedding_dimension: Optional[int]
supports_nested_metadata_filters: Optional[bool]
class CloudAstraDBVectorStore: …

Cloud AstraDB Vector Store.

This class is used to store the configuration for an AstraDB vector store, so that it can be created and used in LlamaCloud.

Args: token (str): The Astra DB Application Token to use. api_endpoint (str): The Astra DB JSON API endpoint for your database. collection_name (str): Collection name to use. If not existing, it will be created. embedding_dimension (int): Length of the embedding vectors in use. keyspace (optional[str]): The keyspace to use. If not provided, ‘default_keyspace’

token: str

The Astra DB Application Token to use

formatpassword
api_endpoint: str

The Astra DB JSON API endpoint for your database

collection_name: str

Collection name to use. If not existing, it will be created

embedding_dimension: int

Length of the embedding vectors in use

class_name: Optional[str]
keyspace: Optional[str]

The keyspace to use. If not provided, ‘default_keyspace’

supports_nested_metadata_filters: Optional[Literal[true]]
name: str

The name of the data sink.

sink_type: Literal["PINECONE", "POSTGRES", "QDRANT", 4 more]
One of the following:
"PINECONE"
"POSTGRES"
"QDRANT"
"AZUREAI_SEARCH"
"MONGODB_ATLAS"
"MILVUS"
"ASTRA_DB"
data_sink_id: Optional[str]

Data sink ID. When provided instead of data_sink, the data sink will be looked up by ID.

formatuuid
embedding_config: Optional[EmbeddingConfig]
One of the following:
class AzureOpenAIEmbeddingConfig: …
component: Optional[AzureOpenAIEmbedding]

Configuration for the Azure OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for Azure deployment.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for Azure OpenAI API.

azure_deployment: Optional[str]

The Azure deployment to use.

azure_endpoint: Optional[str]

The Azure endpoint to use.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["AZURE_EMBEDDING"]]

Type of the embedding model.

class CohereEmbeddingConfig: …
component: Optional[CohereEmbedding]

Configuration for the Cohere embedding model.

api_key: Optional[str]

The Cohere API key.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embedding_type: Optional[str]

Embedding type. If not provided float embedding_type is used when needed.

input_type: Optional[str]

Model Input type. If not provided, search_document and search_query are used when needed.

model_name: Optional[str]

The modelId of the Cohere model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

truncate: Optional[str]

Truncation type - START/ END/ NONE

type: Optional[Literal["COHERE_EMBEDDING"]]

Type of the embedding model.

class GeminiEmbeddingConfig: …
component: Optional[GeminiEmbedding]

Configuration for the Gemini embedding model.

api_base: Optional[str]

API base to access the model. Defaults to None.

api_key: Optional[str]

API key to access the model. Defaults to None.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
model_name: Optional[str]

The modelId of the Gemini model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

output_dimensionality: Optional[int]

Optional reduced dimension for output embeddings. Supported by models/text-embedding-004 and newer (e.g. gemini-embedding-001). Not supported by models/embedding-001.

task_type: Optional[str]

The task for embedding model.

title: Optional[str]

Title is only applicable for retrieval_document tasks, and is used to represent a document title. For other tasks, title is invalid.

transport: Optional[str]

Transport to access the model. Defaults to None.

type: Optional[Literal["GEMINI_EMBEDDING"]]

Type of the embedding model.

class HuggingFaceInferenceAPIEmbeddingConfig: …
component: Optional[HuggingFaceInferenceAPIEmbedding]

Configuration for the HuggingFace Inference API embedding model.

token: Optional[Union[str, bool, null]]

Hugging Face token. Will default to the locally saved token. Pass token=False if you don’t want to send your token to the server.

One of the following:
str
bool
class_name: Optional[str]
cookies: Optional[Dict[str, str]]

Additional cookies to send to the server.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
headers: Optional[Dict[str, str]]

Additional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values.

model_name: Optional[str]

Hugging Face model name. If None, the task will be used.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

pooling: Optional[Literal["cls", "mean", "last"]]

Enum of possible pooling choices with pooling behaviors.

One of the following:
"cls"
"mean"
"last"
query_instruction: Optional[str]

Instruction to prepend during query embedding.

task: Optional[str]

Optional task to pick Hugging Face’s recommended model, used when model_name is left as default of None.

text_instruction: Optional[str]

Instruction to prepend during text embedding.

timeout: Optional[float]

The maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.

type: Optional[Literal["HUGGINGFACE_API_EMBEDDING"]]

Type of the embedding model.

class OpenAIEmbeddingConfig: …
component: Optional[OpenAIEmbedding]

Configuration for the OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for OpenAI API.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for OpenAI API.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["OPENAI_EMBEDDING"]]

Type of the embedding model.

class VertexAIEmbeddingConfig: …
component: Optional[VertexTextEmbedding]

Configuration for the VertexAI embedding model.

client_email: Optional[str]

The client email for the VertexAI credentials.

location: str

The default location to use when making API calls.

private_key: Optional[str]

The private key for the VertexAI credentials.

private_key_id: Optional[str]

The private key ID for the VertexAI credentials.

project: str

The default GCP project to use when making Vertex API calls.

token_uri: Optional[str]

The token URI for the VertexAI credentials.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the Vertex.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embed_mode: Optional[Literal["default", "classification", "clustering", 2 more]]

The embedding mode to use.

One of the following:
"default"
"classification"
"clustering"
"similarity"
"retrieval"
model_name: Optional[str]

The modelId of the VertexAI model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

type: Optional[Literal["VERTEXAI_EMBEDDING"]]

Type of the embedding model.

class BedrockEmbeddingConfig: …
component: Optional[BedrockEmbedding]

Configuration for the Bedrock embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the bedrock client.

aws_access_key_id: Optional[str]

AWS Access Key ID to use

aws_secret_access_key: Optional[str]

AWS Secret Access Key to use

aws_session_token: Optional[str]

AWS Session Token to use

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

The maximum number of API retries.

exclusiveMinimum0
model_name: Optional[str]

The modelId of the Bedrock model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

profile_name: Optional[str]

The name of aws profile to use. If not given, then the default profile is used.

region_name: Optional[str]

AWS region name to use. Uses region configured in AWS CLI if not passed

timeout: Optional[float]

The timeout for the Bedrock API request in seconds. It will be used for both connect and read timeouts.

type: Optional[Literal["BEDROCK_EMBEDDING"]]

Type of the embedding model.

embedding_model_config_id: Optional[str]

Embedding model config ID. When provided instead of embedding_config, the embedding model config will be looked up by ID.

formatuuid
llama_parse_parameters: Optional[LlamaParseParameters]

Settings that can be configured for how to use LlamaParse to parse files within a LlamaCloud pipeline.

adaptive_long_table: Optional[bool]
aggressive_table_extraction: Optional[bool]
auto_mode: Optional[bool]
auto_mode_configuration_json: Optional[str]
auto_mode_trigger_on_image_in_page: Optional[bool]
auto_mode_trigger_on_regexp_in_page: Optional[str]
auto_mode_trigger_on_table_in_page: Optional[bool]
auto_mode_trigger_on_text_in_page: Optional[str]
azure_openai_api_version: Optional[str]
azure_openai_deployment_name: Optional[str]
azure_openai_endpoint: Optional[str]
azure_openai_key: Optional[str]
bbox_bottom: Optional[float]
bbox_left: Optional[float]
bbox_right: Optional[float]
bbox_top: Optional[float]
bounding_box: Optional[str]
compact_markdown_table: Optional[bool]
complemental_formatting_instruction: Optional[str]
content_guideline_instruction: Optional[str]
continuous_mode: Optional[bool]
disable_image_extraction: Optional[bool]
disable_ocr: Optional[bool]
disable_reconstruction: Optional[bool]
do_not_cache: Optional[bool]
do_not_unroll_columns: Optional[bool]
enable_cost_optimizer: Optional[bool]
extract_charts: Optional[bool]
extract_layout: Optional[bool]
extract_printed_page_number: Optional[bool]
fast_mode: Optional[bool]
formatting_instruction: Optional[str]
gpt4o_api_key: Optional[str]
gpt4o_mode: Optional[bool]
guess_xlsx_sheet_name: Optional[bool]
hide_footers: Optional[bool]
hide_headers: Optional[bool]
high_res_ocr: Optional[bool]
html_make_all_elements_visible: Optional[bool]
html_remove_fixed_elements: Optional[bool]
html_remove_navigation_elements: Optional[bool]
http_proxy: Optional[str]
ignore_document_elements_for_layout_detection: Optional[bool]
images_to_save: Optional[List[Literal["screenshot", "embedded", "layout"]]]
One of the following:
"screenshot"
"embedded"
"layout"
inline_images_in_markdown: Optional[bool]
input_s3_path: Optional[str]
input_s3_region: Optional[str]
input_url: Optional[str]
internal_is_screenshot_job: Optional[bool]
invalidate_cache: Optional[bool]
is_formatting_instruction: Optional[bool]
job_timeout_extra_time_per_page_in_seconds: Optional[float]
job_timeout_in_seconds: Optional[float]
keep_page_separator_when_merging_tables: Optional[bool]
languages: Optional[List[ParsingLanguages]]
One of the following:
"af"
"az"
"bs"
"cs"
"cy"
"da"
"de"
"en"
"es"
"et"
"fr"
"ga"
"hr"
"hu"
"id"
"is"
"it"
"ku"
"la"
"lt"
"lv"
"mi"
"ms"
"mt"
"nl"
"no"
"oc"
"pi"
"pl"
"pt"
"ro"
"rs_latin"
"sk"
"sl"
"sq"
"sv"
"sw"
"tl"
"tr"
"uz"
"vi"
"ar"
"fa"
"ug"
"ur"
"bn"
"as"
"mni"
"ru"
"rs_cyrillic"
"be"
"bg"
"uk"
"mn"
"abq"
"ady"
"kbd"
"ava"
"dar"
"inh"
"che"
"lbe"
"lez"
"tab"
"tjk"
"hi"
"mr"
"ne"
"bh"
"mai"
"ang"
"bho"
"mah"
"sck"
"new"
"gom"
"sa"
"bgc"
"th"
"ch_sim"
"ch_tra"
"ja"
"ko"
"ta"
"te"
"kn"
layout_aware: Optional[bool]
line_level_bounding_box: Optional[bool]
markdown_table_multiline_header_separator: Optional[str]
max_pages: Optional[int]
max_pages_enforced: Optional[int]
merge_tables_across_pages_in_markdown: Optional[bool]
model: Optional[str]
outlined_table_extraction: Optional[bool]
output_pdf_of_document: Optional[bool]
output_s3_path_prefix: Optional[str]
output_s3_region: Optional[str]
output_tables_as_html: Optional[bool]
page_error_tolerance: Optional[float]
page_footer_prefix: Optional[str]
page_footer_suffix: Optional[str]
page_header_prefix: Optional[str]
page_header_suffix: Optional[str]
page_prefix: Optional[str]
page_separator: Optional[str]
page_suffix: Optional[str]
parse_mode: Optional[ParsingMode]

Enum for representing the mode of parsing to be used.

One of the following:
"parse_page_without_llm"
"parse_page_with_llm"
"parse_page_with_lvm"
"parse_page_with_agent"
"parse_page_with_layout_agent"
"parse_document_with_llm"
"parse_document_with_lvm"
"parse_document_with_agent"
parsing_instruction: Optional[str]
precise_bounding_box: Optional[bool]
premium_mode: Optional[bool]
presentation_out_of_bounds_content: Optional[bool]
presentation_skip_embedded_data: Optional[bool]
preserve_layout_alignment_across_pages: Optional[bool]
preserve_very_small_text: Optional[bool]
preset: Optional[str]
priority: Optional[Literal["low", "medium", "high", "critical"]]

The priority for the request. This field may be ignored or overwritten depending on the organization tier.

One of the following:
"low"
"medium"
"high"
"critical"
project_id: Optional[str]
remove_hidden_text: Optional[bool]
replace_failed_page_mode: Optional[FailPageMode]

Enum for representing the different available page error handling modes.

One of the following:
"raw_text"
"blank_page"
"error_message"
replace_failed_page_with_error_message_prefix: Optional[str]
replace_failed_page_with_error_message_suffix: Optional[str]
save_images: Optional[bool]
skip_diagonal_text: Optional[bool]
specialized_chart_parsing_agentic: Optional[bool]
specialized_chart_parsing_efficient: Optional[bool]
specialized_chart_parsing_plus: Optional[bool]
specialized_image_parsing: Optional[bool]
spreadsheet_extract_sub_tables: Optional[bool]
spreadsheet_force_formula_computation: Optional[bool]
spreadsheet_include_hidden_sheets: Optional[bool]
strict_mode_buggy_font: Optional[bool]
strict_mode_image_extraction: Optional[bool]
strict_mode_image_ocr: Optional[bool]
strict_mode_reconstruction: Optional[bool]
structured_output: Optional[bool]
structured_output_json_schema: Optional[str]
structured_output_json_schema_name: Optional[str]
system_prompt: Optional[str]
system_prompt_append: Optional[str]
take_screenshot: Optional[bool]
target_pages: Optional[str]
tier: Optional[str]
use_vendor_multimodal_model: Optional[bool]
user_prompt: Optional[str]
vendor_multimodal_api_key: Optional[str]
vendor_multimodal_model_name: Optional[str]
version: Optional[str]
webhook_configurations: Optional[List[WebhookConfiguration]]

Outbound webhook endpoints to notify on job status changes

webhook_events: Optional[List[Literal["extract.pending", "extract.success", "extract.error", 14 more]]]

Events to subscribe to (e.g. ‘parse.success’, ‘extract.error’). If null, all events are delivered.

One of the following:
"extract.pending"
"extract.success"
"extract.error"
"extract.partial_success"
"extract.cancelled"
"parse.pending"
"parse.running"
"parse.success"
"parse.error"
"parse.partial_success"
"parse.cancelled"
"classify.pending"
"classify.success"
"classify.error"
"classify.partial_success"
"classify.cancelled"
"unmapped_event"
webhook_headers: Optional[Dict[str, str]]

Custom HTTP headers sent with each webhook request (e.g. auth tokens)

webhook_output_format: Optional[str]

Response format sent to the webhook: ‘string’ (default) or ‘json’

webhook_url: Optional[str]

URL to receive webhook POST notifications

webhook_url: Optional[str]
managed_pipeline_id: Optional[str]

The ID of the ManagedPipeline this playground pipeline is linked to.

formatuuid
metadata_config: Optional[PipelineMetadataConfig]

Metadata configuration for the pipeline.

excluded_embed_metadata_keys: Optional[List[str]]

List of metadata keys to exclude from embeddings

excluded_llm_metadata_keys: Optional[List[str]]

List of metadata keys to exclude from LLM during retrieval

pipeline_type: Optional[PipelineType]

Type of pipeline. Either PLAYGROUND or MANAGED.

One of the following:
"PLAYGROUND"
"MANAGED"
preset_retrieval_parameters: Optional[PresetRetrievalParams]

Preset retrieval parameters for the pipeline.

alpha: Optional[float]

Alpha value for hybrid retrieval to determine the weights between dense and sparse retrieval. 0 is sparse retrieval and 1 is dense retrieval.

maximum1
minimum0
class_name: Optional[str]
dense_similarity_cutoff: Optional[float]

Minimum similarity score wrt query for retrieval

maximum1
minimum0
dense_similarity_top_k: Optional[int]

Number of nodes for dense retrieval.

maximum100
minimum1
enable_reranking: Optional[bool]

Enable reranking for retrieval

files_top_k: Optional[int]

Number of files to retrieve (only for retrieval mode files_via_metadata and files_via_content).

maximum5
minimum1
rerank_top_n: Optional[int]

Number of reranked nodes for returning.

maximum100
minimum1
retrieval_mode: Optional[RetrievalMode]

The retrieval mode for the query.

One of the following:
"chunks"
"files_via_metadata"
"files_via_content"
"auto_routed"
Deprecatedretrieve_image_nodes: Optional[bool]

Whether to retrieve image nodes.

retrieve_page_figure_nodes: Optional[bool]

Whether to retrieve page figure nodes.

retrieve_page_screenshot_nodes: Optional[bool]

Whether to retrieve page screenshot nodes.

search_filters: Optional[MetadataFilters]

Metadata filters for vector stores.

filters: List[Filter]
One of the following:
class FilterMetadataFilter: …

Comprehensive metadata filter for vector stores to support more operators.

Value uses Strict types, as int, float and str are compatible types and were all converted to string before.

See: https://docs.pydantic.dev/latest/usage/types/#strict-types

key: str
value: Union[float, str, List[str], 3 more]
One of the following:
float
str
List[str]
List[float]
List[int]
operator: Optional[Literal["==", ">", "<", 11 more]]

Vector store filter operator.

One of the following:
"=="
">"
"<"
"!="
">="
"<="
"in"
"nin"
"any"
"all"
"text_match"
"text_match_insensitive"
"contains"
"is_empty"
condition: Optional[Literal["and", "or", "not"]]

Vector store filter conditions to combine different filters.

One of the following:
"and"
"or"
"not"
search_filters_inference_schema: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

JSON Schema that will be used to infer search_filters. Omit or leave as null to skip inference.

One of the following:
Dict[str, object]
List[object]
str
float
bool
sparse_similarity_top_k: Optional[int]

Number of nodes for sparse retrieval.

maximum100
minimum1
sparse_model_config: Optional[SparseModelConfig]

Configuration for sparse embedding models used in hybrid search.

This allows users to choose between Splade and BM25 models for sparse retrieval in managed data sinks.

class_name: Optional[str]
model_type: Optional[Literal["splade", "bm25", "auto"]]

The sparse model type to use. ‘bm25’ uses Qdrant’s FastEmbed BM25 model (default for new pipelines), ‘splade’ uses HuggingFace Splade model, ‘auto’ selects based on deployment mode (BYOC uses term frequency, Cloud uses Splade).

One of the following:
"splade"
"bm25"
"auto"
status: Optional[str]

Status of the pipeline deployment.

transform_config: Optional[TransformConfig]

Configuration for the transformation.

One of the following:
class AutoTransformConfig: …
chunk_overlap: Optional[int]

Chunk overlap for the transformation.

chunk_size: Optional[int]

Chunk size for the transformation.

exclusiveMinimum0
mode: Optional[Literal["auto"]]
class AdvancedModeTransformConfig: …
chunking_config: Optional[ChunkingConfig]

Configuration for the chunking.

One of the following:
class ChunkingConfigNoneChunkingConfig: …
mode: Optional[Literal["none"]]
class ChunkingConfigCharacterChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["character"]]
class ChunkingConfigTokenChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["token"]]
separator: Optional[str]
class ChunkingConfigSentenceChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["sentence"]]
paragraph_separator: Optional[str]
separator: Optional[str]
class ChunkingConfigSemanticChunkingConfig: …
breakpoint_percentile_threshold: Optional[int]
buffer_size: Optional[int]
mode: Optional[Literal["semantic"]]
mode: Optional[Literal["advanced"]]
segmentation_config: Optional[SegmentationConfig]

Configuration for the segmentation.

One of the following:
class SegmentationConfigNoneSegmentationConfig: …
mode: Optional[Literal["none"]]
class SegmentationConfigPageSegmentationConfig: …
mode: Optional[Literal["page"]]
page_separator: Optional[str]
class SegmentationConfigElementSegmentationConfig: …
mode: Optional[Literal["element"]]
class PipelineMetadataConfig: …
excluded_embed_metadata_keys: Optional[List[str]]

List of metadata keys to exclude from embeddings

excluded_llm_metadata_keys: Optional[List[str]]

List of metadata keys to exclude from LLM during retrieval

Literal["PLAYGROUND", "MANAGED"]

Enum for representing the type of a pipeline

One of the following:
"PLAYGROUND"
"MANAGED"
class PresetRetrievalParams: …

Schema for the search params for an retrieval execution that can be preset for a pipeline.

alpha: Optional[float]

Alpha value for hybrid retrieval to determine the weights between dense and sparse retrieval. 0 is sparse retrieval and 1 is dense retrieval.

maximum1
minimum0
class_name: Optional[str]
dense_similarity_cutoff: Optional[float]

Minimum similarity score wrt query for retrieval

maximum1
minimum0
dense_similarity_top_k: Optional[int]

Number of nodes for dense retrieval.

maximum100
minimum1
enable_reranking: Optional[bool]

Enable reranking for retrieval

files_top_k: Optional[int]

Number of files to retrieve (only for retrieval mode files_via_metadata and files_via_content).

maximum5
minimum1
rerank_top_n: Optional[int]

Number of reranked nodes for returning.

maximum100
minimum1
retrieval_mode: Optional[RetrievalMode]

The retrieval mode for the query.

One of the following:
"chunks"
"files_via_metadata"
"files_via_content"
"auto_routed"
Deprecatedretrieve_image_nodes: Optional[bool]

Whether to retrieve image nodes.

retrieve_page_figure_nodes: Optional[bool]

Whether to retrieve page figure nodes.

retrieve_page_screenshot_nodes: Optional[bool]

Whether to retrieve page screenshot nodes.

search_filters: Optional[MetadataFilters]

Metadata filters for vector stores.

filters: List[Filter]
One of the following:
class FilterMetadataFilter: …

Comprehensive metadata filter for vector stores to support more operators.

Value uses Strict types, as int, float and str are compatible types and were all converted to string before.

See: https://docs.pydantic.dev/latest/usage/types/#strict-types

key: str
value: Union[float, str, List[str], 3 more]
One of the following:
float
str
List[str]
List[float]
List[int]
operator: Optional[Literal["==", ">", "<", 11 more]]

Vector store filter operator.

One of the following:
"=="
">"
"<"
"!="
">="
"<="
"in"
"nin"
"any"
"all"
"text_match"
"text_match_insensitive"
"contains"
"is_empty"
condition: Optional[Literal["and", "or", "not"]]

Vector store filter conditions to combine different filters.

One of the following:
"and"
"or"
"not"
search_filters_inference_schema: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

JSON Schema that will be used to infer search_filters. Omit or leave as null to skip inference.

One of the following:
Dict[str, object]
List[object]
str
float
bool
sparse_similarity_top_k: Optional[int]

Number of nodes for sparse retrieval.

maximum100
minimum1
Literal["chunks", "files_via_metadata", "files_via_content", "auto_routed"]
One of the following:
"chunks"
"files_via_metadata"
"files_via_content"
"auto_routed"
class SparseModelConfig: …

Configuration for sparse embedding models used in hybrid search.

This allows users to choose between Splade and BM25 models for sparse retrieval in managed data sinks.

class_name: Optional[str]
model_type: Optional[Literal["splade", "bm25", "auto"]]

The sparse model type to use. ‘bm25’ uses Qdrant’s FastEmbed BM25 model (default for new pipelines), ‘splade’ uses HuggingFace Splade model, ‘auto’ selects based on deployment mode (BYOC uses term frequency, Cloud uses Splade).

One of the following:
"splade"
"bm25"
"auto"
class VertexAIEmbeddingConfig: …
component: Optional[VertexTextEmbedding]

Configuration for the VertexAI embedding model.

client_email: Optional[str]

The client email for the VertexAI credentials.

location: str

The default location to use when making API calls.

private_key: Optional[str]

The private key for the VertexAI credentials.

private_key_id: Optional[str]

The private key ID for the VertexAI credentials.

project: str

The default GCP project to use when making Vertex API calls.

token_uri: Optional[str]

The token URI for the VertexAI credentials.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the Vertex.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embed_mode: Optional[Literal["default", "classification", "clustering", 2 more]]

The embedding mode to use.

One of the following:
"default"
"classification"
"clustering"
"similarity"
"retrieval"
model_name: Optional[str]

The modelId of the VertexAI model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

type: Optional[Literal["VERTEXAI_EMBEDDING"]]

Type of the embedding model.

class VertexTextEmbedding: …
client_email: Optional[str]

The client email for the VertexAI credentials.

location: str

The default location to use when making API calls.

private_key: Optional[str]

The private key for the VertexAI credentials.

private_key_id: Optional[str]

The private key ID for the VertexAI credentials.

project: str

The default GCP project to use when making Vertex API calls.

token_uri: Optional[str]

The token URI for the VertexAI credentials.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the Vertex.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embed_mode: Optional[Literal["default", "classification", "clustering", 2 more]]

The embedding mode to use.

One of the following:
"default"
"classification"
"clustering"
"similarity"
"retrieval"
model_name: Optional[str]

The modelId of the VertexAI model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

List[Pipeline]
id: str

Unique identifier

formatuuid
embedding_config: EmbeddingConfig
One of the following:
class EmbeddingConfigManagedOpenAIEmbeddingConfig: …
component: Optional[EmbeddingConfigManagedOpenAIEmbeddingConfigComponent]

Configuration for the Managed OpenAI embedding model.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
model_name: Optional[Literal["openai-text-embedding-3-small"]]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

type: Optional[Literal["MANAGED_OPENAI_EMBEDDING"]]

Type of the embedding model.

class AzureOpenAIEmbeddingConfig: …
component: Optional[AzureOpenAIEmbedding]

Configuration for the Azure OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for Azure deployment.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for Azure OpenAI API.

azure_deployment: Optional[str]

The Azure deployment to use.

azure_endpoint: Optional[str]

The Azure endpoint to use.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["AZURE_EMBEDDING"]]

Type of the embedding model.

class CohereEmbeddingConfig: …
component: Optional[CohereEmbedding]

Configuration for the Cohere embedding model.

api_key: Optional[str]

The Cohere API key.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embedding_type: Optional[str]

Embedding type. If not provided float embedding_type is used when needed.

input_type: Optional[str]

Model Input type. If not provided, search_document and search_query are used when needed.

model_name: Optional[str]

The modelId of the Cohere model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

truncate: Optional[str]

Truncation type - START/ END/ NONE

type: Optional[Literal["COHERE_EMBEDDING"]]

Type of the embedding model.

class GeminiEmbeddingConfig: …
component: Optional[GeminiEmbedding]

Configuration for the Gemini embedding model.

api_base: Optional[str]

API base to access the model. Defaults to None.

api_key: Optional[str]

API key to access the model. Defaults to None.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
model_name: Optional[str]

The modelId of the Gemini model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

output_dimensionality: Optional[int]

Optional reduced dimension for output embeddings. Supported by models/text-embedding-004 and newer (e.g. gemini-embedding-001). Not supported by models/embedding-001.

task_type: Optional[str]

The task for embedding model.

title: Optional[str]

Title is only applicable for retrieval_document tasks, and is used to represent a document title. For other tasks, title is invalid.

transport: Optional[str]

Transport to access the model. Defaults to None.

type: Optional[Literal["GEMINI_EMBEDDING"]]

Type of the embedding model.

class HuggingFaceInferenceAPIEmbeddingConfig: …
component: Optional[HuggingFaceInferenceAPIEmbedding]

Configuration for the HuggingFace Inference API embedding model.

token: Optional[Union[str, bool, null]]

Hugging Face token. Will default to the locally saved token. Pass token=False if you don’t want to send your token to the server.

One of the following:
str
bool
class_name: Optional[str]
cookies: Optional[Dict[str, str]]

Additional cookies to send to the server.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
headers: Optional[Dict[str, str]]

Additional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values.

model_name: Optional[str]

Hugging Face model name. If None, the task will be used.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

pooling: Optional[Literal["cls", "mean", "last"]]

Enum of possible pooling choices with pooling behaviors.

One of the following:
"cls"
"mean"
"last"
query_instruction: Optional[str]

Instruction to prepend during query embedding.

task: Optional[str]

Optional task to pick Hugging Face’s recommended model, used when model_name is left as default of None.

text_instruction: Optional[str]

Instruction to prepend during text embedding.

timeout: Optional[float]

The maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.

type: Optional[Literal["HUGGINGFACE_API_EMBEDDING"]]

Type of the embedding model.

class OpenAIEmbeddingConfig: …
component: Optional[OpenAIEmbedding]

Configuration for the OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for OpenAI API.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for OpenAI API.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["OPENAI_EMBEDDING"]]

Type of the embedding model.

class VertexAIEmbeddingConfig: …
component: Optional[VertexTextEmbedding]

Configuration for the VertexAI embedding model.

client_email: Optional[str]

The client email for the VertexAI credentials.

location: str

The default location to use when making API calls.

private_key: Optional[str]

The private key for the VertexAI credentials.

private_key_id: Optional[str]

The private key ID for the VertexAI credentials.

project: str

The default GCP project to use when making Vertex API calls.

token_uri: Optional[str]

The token URI for the VertexAI credentials.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the Vertex.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embed_mode: Optional[Literal["default", "classification", "clustering", 2 more]]

The embedding mode to use.

One of the following:
"default"
"classification"
"clustering"
"similarity"
"retrieval"
model_name: Optional[str]

The modelId of the VertexAI model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

type: Optional[Literal["VERTEXAI_EMBEDDING"]]

Type of the embedding model.

class BedrockEmbeddingConfig: …
component: Optional[BedrockEmbedding]

Configuration for the Bedrock embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the bedrock client.

aws_access_key_id: Optional[str]

AWS Access Key ID to use

aws_secret_access_key: Optional[str]

AWS Secret Access Key to use

aws_session_token: Optional[str]

AWS Session Token to use

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

The maximum number of API retries.

exclusiveMinimum0
model_name: Optional[str]

The modelId of the Bedrock model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

profile_name: Optional[str]

The name of aws profile to use. If not given, then the default profile is used.

region_name: Optional[str]

AWS region name to use. Uses region configured in AWS CLI if not passed

timeout: Optional[float]

The timeout for the Bedrock API request in seconds. It will be used for both connect and read timeouts.

type: Optional[Literal["BEDROCK_EMBEDDING"]]

Type of the embedding model.

name: str
project_id: str
config_hash: Optional[ConfigHash]

Hashes for the configuration of a pipeline.

embedding_config_hash: Optional[str]

Hash of the embedding config.

parsing_config_hash: Optional[str]

Hash of the llama parse parameters.

transform_config_hash: Optional[str]

Hash of the transform config.

created_at: Optional[datetime]

Creation datetime

formatdate-time
data_sink: Optional[DataSink]

Schema for a data sink.

id: str

Unique identifier

formatuuid
component: Component

Component that implements the data sink

One of the following:
Dict[str, object]
class CloudPineconeVectorStore: …

Cloud Pinecone Vector Store.

This class is used to store the configuration for a Pinecone vector store, so that it can be created and used in LlamaCloud.

Args: api_key (str): API key for authenticating with Pinecone index_name (str): name of the Pinecone index namespace (optional[str]): namespace to use in the Pinecone index insert_kwargs (optional[dict]): additional kwargs to pass during insertion

api_key: str

The API key for authenticating with Pinecone

formatpassword
index_name: str
class_name: Optional[str]
insert_kwargs: Optional[Dict[str, object]]
namespace: Optional[str]
supports_nested_metadata_filters: Optional[Literal[true]]
class CloudPostgresVectorStore: …
database: str
embed_dim: int
host: str
password: str
port: int
schema_name: str
table_name: str
user: str
class_name: Optional[str]
hnsw_settings: Optional[PgVectorHnswSettings]

HNSW settings for PGVector.

distance_method: Optional[Literal["l2", "ip", "cosine", 3 more]]

The distance method to use.

One of the following:
"l2"
"ip"
"cosine"
"l1"
"hamming"
"jaccard"
ef_construction: Optional[int]

The number of edges to use during the construction phase.

minimum1

The number of edges to use during the search phase.

minimum1
m: Optional[int]

The number of bi-directional links created for each new element.

minimum1
vector_type: Optional[Literal["vector", "half_vec", "bit", "sparse_vec"]]

The type of vector to use.

One of the following:
"vector"
"half_vec"
"bit"
"sparse_vec"
perform_setup: Optional[bool]
supports_nested_metadata_filters: Optional[bool]
class CloudQdrantVectorStore: …

Cloud Qdrant Vector Store.

This class is used to store the configuration for a Qdrant vector store, so that it can be created and used in LlamaCloud.

Args: collection_name (str): name of the Qdrant collection url (str): url of the Qdrant instance api_key (str): API key for authenticating with Qdrant max_retries (int): maximum number of retries in case of a failure. Defaults to 3 client_kwargs (dict): additional kwargs to pass to the Qdrant client

api_key: str
collection_name: str
url: str
class_name: Optional[str]
client_kwargs: Optional[Dict[str, object]]
max_retries: Optional[int]
supports_nested_metadata_filters: Optional[Literal[true]]
class CloudAzureAISearchVectorStore: …

Cloud Azure AI Search Vector Store.

search_service_api_key: str
search_service_endpoint: str
class_name: Optional[str]
client_id: Optional[str]
client_secret: Optional[str]
embedding_dimension: Optional[int]
filterable_metadata_field_keys: Optional[Dict[str, object]]
index_name: Optional[str]
search_service_api_version: Optional[str]
supports_nested_metadata_filters: Optional[Literal[true]]
tenant_id: Optional[str]

Cloud MongoDB Atlas Vector Store.

This class is used to store the configuration for a MongoDB Atlas vector store, so that it can be created and used in LlamaCloud.

Args: mongodb_uri (str): URI for connecting to MongoDB Atlas db_name (str): name of the MongoDB database collection_name (str): name of the MongoDB collection vector_index_name (str): name of the MongoDB Atlas vector index fulltext_index_name (str): name of the MongoDB Atlas full-text index

class CloudMilvusVectorStore: …

Cloud Milvus Vector Store.

uri: str
token: Optional[str]
class_name: Optional[str]
collection_name: Optional[str]
embedding_dimension: Optional[int]
supports_nested_metadata_filters: Optional[bool]
class CloudAstraDBVectorStore: …

Cloud AstraDB Vector Store.

This class is used to store the configuration for an AstraDB vector store, so that it can be created and used in LlamaCloud.

Args: token (str): The Astra DB Application Token to use. api_endpoint (str): The Astra DB JSON API endpoint for your database. collection_name (str): Collection name to use. If not existing, it will be created. embedding_dimension (int): Length of the embedding vectors in use. keyspace (optional[str]): The keyspace to use. If not provided, ‘default_keyspace’

token: str

The Astra DB Application Token to use

formatpassword
api_endpoint: str

The Astra DB JSON API endpoint for your database

collection_name: str

Collection name to use. If not existing, it will be created

embedding_dimension: int

Length of the embedding vectors in use

class_name: Optional[str]
keyspace: Optional[str]

The keyspace to use. If not provided, ‘default_keyspace’

supports_nested_metadata_filters: Optional[Literal[true]]
name: str

The name of the data sink.

project_id: str
sink_type: Literal["PINECONE", "POSTGRES", "QDRANT", 4 more]
One of the following:
"PINECONE"
"POSTGRES"
"QDRANT"
"AZUREAI_SEARCH"
"MONGODB_ATLAS"
"MILVUS"
"ASTRA_DB"
created_at: Optional[datetime]

Creation datetime

formatdate-time
updated_at: Optional[datetime]

Update datetime

formatdate-time
embedding_model_config: Optional[EmbeddingModelConfig]

Schema for an embedding model config.

id: str

Unique identifier

formatuuid
embedding_config: EmbeddingModelConfigEmbeddingConfig

The embedding configuration for the embedding model config.

One of the following:
class AzureOpenAIEmbeddingConfig: …
component: Optional[AzureOpenAIEmbedding]

Configuration for the Azure OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for Azure deployment.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for Azure OpenAI API.

azure_deployment: Optional[str]

The Azure deployment to use.

azure_endpoint: Optional[str]

The Azure endpoint to use.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["AZURE_EMBEDDING"]]

Type of the embedding model.

class CohereEmbeddingConfig: …
component: Optional[CohereEmbedding]

Configuration for the Cohere embedding model.

api_key: Optional[str]

The Cohere API key.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embedding_type: Optional[str]

Embedding type. If not provided float embedding_type is used when needed.

input_type: Optional[str]

Model Input type. If not provided, search_document and search_query are used when needed.

model_name: Optional[str]

The modelId of the Cohere model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

truncate: Optional[str]

Truncation type - START/ END/ NONE

type: Optional[Literal["COHERE_EMBEDDING"]]

Type of the embedding model.

class GeminiEmbeddingConfig: …
component: Optional[GeminiEmbedding]

Configuration for the Gemini embedding model.

api_base: Optional[str]

API base to access the model. Defaults to None.

api_key: Optional[str]

API key to access the model. Defaults to None.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
model_name: Optional[str]

The modelId of the Gemini model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

output_dimensionality: Optional[int]

Optional reduced dimension for output embeddings. Supported by models/text-embedding-004 and newer (e.g. gemini-embedding-001). Not supported by models/embedding-001.

task_type: Optional[str]

The task for embedding model.

title: Optional[str]

Title is only applicable for retrieval_document tasks, and is used to represent a document title. For other tasks, title is invalid.

transport: Optional[str]

Transport to access the model. Defaults to None.

type: Optional[Literal["GEMINI_EMBEDDING"]]

Type of the embedding model.

class HuggingFaceInferenceAPIEmbeddingConfig: …
component: Optional[HuggingFaceInferenceAPIEmbedding]

Configuration for the HuggingFace Inference API embedding model.

token: Optional[Union[str, bool, null]]

Hugging Face token. Will default to the locally saved token. Pass token=False if you don’t want to send your token to the server.

One of the following:
str
bool
class_name: Optional[str]
cookies: Optional[Dict[str, str]]

Additional cookies to send to the server.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
headers: Optional[Dict[str, str]]

Additional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values.

model_name: Optional[str]

Hugging Face model name. If None, the task will be used.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

pooling: Optional[Literal["cls", "mean", "last"]]

Enum of possible pooling choices with pooling behaviors.

One of the following:
"cls"
"mean"
"last"
query_instruction: Optional[str]

Instruction to prepend during query embedding.

task: Optional[str]

Optional task to pick Hugging Face’s recommended model, used when model_name is left as default of None.

text_instruction: Optional[str]

Instruction to prepend during text embedding.

timeout: Optional[float]

The maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.

type: Optional[Literal["HUGGINGFACE_API_EMBEDDING"]]

Type of the embedding model.

class OpenAIEmbeddingConfig: …
component: Optional[OpenAIEmbedding]

Configuration for the OpenAI embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the OpenAI API.

api_base: Optional[str]

The base URL for OpenAI API.

api_key: Optional[str]

The OpenAI API key.

api_version: Optional[str]

The version for OpenAI API.

class_name: Optional[str]
default_headers: Optional[Dict[str, str]]

The default headers for API requests.

dimensions: Optional[int]

The number of dimensions on the output embedding vectors. Works only with v3 embedding models.

embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

Maximum number of retries.

minimum0
model_name: Optional[str]

The name of the OpenAI embedding model.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

reuse_client: Optional[bool]

Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.

timeout: Optional[float]

Timeout for each request.

minimum0
type: Optional[Literal["OPENAI_EMBEDDING"]]

Type of the embedding model.

class VertexAIEmbeddingConfig: …
component: Optional[VertexTextEmbedding]

Configuration for the VertexAI embedding model.

client_email: Optional[str]

The client email for the VertexAI credentials.

location: str

The default location to use when making API calls.

private_key: Optional[str]

The private key for the VertexAI credentials.

private_key_id: Optional[str]

The private key ID for the VertexAI credentials.

project: str

The default GCP project to use when making Vertex API calls.

token_uri: Optional[str]

The token URI for the VertexAI credentials.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the Vertex.

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
embed_mode: Optional[Literal["default", "classification", "clustering", 2 more]]

The embedding mode to use.

One of the following:
"default"
"classification"
"clustering"
"similarity"
"retrieval"
model_name: Optional[str]

The modelId of the VertexAI model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

type: Optional[Literal["VERTEXAI_EMBEDDING"]]

Type of the embedding model.

class BedrockEmbeddingConfig: …
component: Optional[BedrockEmbedding]

Configuration for the Bedrock embedding model.

additional_kwargs: Optional[Dict[str, object]]

Additional kwargs for the bedrock client.

aws_access_key_id: Optional[str]

AWS Access Key ID to use

aws_secret_access_key: Optional[str]

AWS Secret Access Key to use

aws_session_token: Optional[str]

AWS Session Token to use

class_name: Optional[str]
embed_batch_size: Optional[int]

The batch size for embedding calls.

maximum2048
exclusiveMinimum0
max_retries: Optional[int]

The maximum number of API retries.

exclusiveMinimum0
model_name: Optional[str]

The modelId of the Bedrock model to use.

num_workers: Optional[int]

The number of workers to use for async embedding calls.

profile_name: Optional[str]

The name of aws profile to use. If not given, then the default profile is used.

region_name: Optional[str]

AWS region name to use. Uses region configured in AWS CLI if not passed

timeout: Optional[float]

The timeout for the Bedrock API request in seconds. It will be used for both connect and read timeouts.

type: Optional[Literal["BEDROCK_EMBEDDING"]]

Type of the embedding model.

name: str

The name of the embedding model config.

project_id: str
created_at: Optional[datetime]

Creation datetime

formatdate-time
updated_at: Optional[datetime]

Update datetime

formatdate-time
embedding_model_config_id: Optional[str]

The ID of the EmbeddingModelConfig this pipeline is using.

formatuuid
llama_parse_parameters: Optional[LlamaParseParameters]

Settings that can be configured for how to use LlamaParse to parse files within a LlamaCloud pipeline.

adaptive_long_table: Optional[bool]
aggressive_table_extraction: Optional[bool]
auto_mode: Optional[bool]
auto_mode_configuration_json: Optional[str]
auto_mode_trigger_on_image_in_page: Optional[bool]
auto_mode_trigger_on_regexp_in_page: Optional[str]
auto_mode_trigger_on_table_in_page: Optional[bool]
auto_mode_trigger_on_text_in_page: Optional[str]
azure_openai_api_version: Optional[str]
azure_openai_deployment_name: Optional[str]
azure_openai_endpoint: Optional[str]
azure_openai_key: Optional[str]
bbox_bottom: Optional[float]
bbox_left: Optional[float]
bbox_right: Optional[float]
bbox_top: Optional[float]
bounding_box: Optional[str]
compact_markdown_table: Optional[bool]
complemental_formatting_instruction: Optional[str]
content_guideline_instruction: Optional[str]
continuous_mode: Optional[bool]
disable_image_extraction: Optional[bool]
disable_ocr: Optional[bool]
disable_reconstruction: Optional[bool]
do_not_cache: Optional[bool]
do_not_unroll_columns: Optional[bool]
enable_cost_optimizer: Optional[bool]
extract_charts: Optional[bool]
extract_layout: Optional[bool]
extract_printed_page_number: Optional[bool]
fast_mode: Optional[bool]
formatting_instruction: Optional[str]
gpt4o_api_key: Optional[str]
gpt4o_mode: Optional[bool]
guess_xlsx_sheet_name: Optional[bool]
hide_footers: Optional[bool]
hide_headers: Optional[bool]
high_res_ocr: Optional[bool]
html_make_all_elements_visible: Optional[bool]
html_remove_fixed_elements: Optional[bool]
html_remove_navigation_elements: Optional[bool]
http_proxy: Optional[str]
ignore_document_elements_for_layout_detection: Optional[bool]
images_to_save: Optional[List[Literal["screenshot", "embedded", "layout"]]]
One of the following:
"screenshot"
"embedded"
"layout"
inline_images_in_markdown: Optional[bool]
input_s3_path: Optional[str]
input_s3_region: Optional[str]
input_url: Optional[str]
internal_is_screenshot_job: Optional[bool]
invalidate_cache: Optional[bool]
is_formatting_instruction: Optional[bool]
job_timeout_extra_time_per_page_in_seconds: Optional[float]
job_timeout_in_seconds: Optional[float]
keep_page_separator_when_merging_tables: Optional[bool]
languages: Optional[List[ParsingLanguages]]
One of the following:
"af"
"az"
"bs"
"cs"
"cy"
"da"
"de"
"en"
"es"
"et"
"fr"
"ga"
"hr"
"hu"
"id"
"is"
"it"
"ku"
"la"
"lt"
"lv"
"mi"
"ms"
"mt"
"nl"
"no"
"oc"
"pi"
"pl"
"pt"
"ro"
"rs_latin"
"sk"
"sl"
"sq"
"sv"
"sw"
"tl"
"tr"
"uz"
"vi"
"ar"
"fa"
"ug"
"ur"
"bn"
"as"
"mni"
"ru"
"rs_cyrillic"
"be"
"bg"
"uk"
"mn"
"abq"
"ady"
"kbd"
"ava"
"dar"
"inh"
"che"
"lbe"
"lez"
"tab"
"tjk"
"hi"
"mr"
"ne"
"bh"
"mai"
"ang"
"bho"
"mah"
"sck"
"new"
"gom"
"sa"
"bgc"
"th"
"ch_sim"
"ch_tra"
"ja"
"ko"
"ta"
"te"
"kn"
layout_aware: Optional[bool]
line_level_bounding_box: Optional[bool]
markdown_table_multiline_header_separator: Optional[str]
max_pages: Optional[int]
max_pages_enforced: Optional[int]
merge_tables_across_pages_in_markdown: Optional[bool]
model: Optional[str]
outlined_table_extraction: Optional[bool]
output_pdf_of_document: Optional[bool]
output_s3_path_prefix: Optional[str]
output_s3_region: Optional[str]
output_tables_as_html: Optional[bool]
page_error_tolerance: Optional[float]
page_footer_prefix: Optional[str]
page_footer_suffix: Optional[str]
page_header_prefix: Optional[str]
page_header_suffix: Optional[str]
page_prefix: Optional[str]
page_separator: Optional[str]
page_suffix: Optional[str]
parse_mode: Optional[ParsingMode]

Enum for representing the mode of parsing to be used.

One of the following:
"parse_page_without_llm"
"parse_page_with_llm"
"parse_page_with_lvm"
"parse_page_with_agent"
"parse_page_with_layout_agent"
"parse_document_with_llm"
"parse_document_with_lvm"
"parse_document_with_agent"
parsing_instruction: Optional[str]
precise_bounding_box: Optional[bool]
premium_mode: Optional[bool]
presentation_out_of_bounds_content: Optional[bool]
presentation_skip_embedded_data: Optional[bool]
preserve_layout_alignment_across_pages: Optional[bool]
preserve_very_small_text: Optional[bool]
preset: Optional[str]
priority: Optional[Literal["low", "medium", "high", "critical"]]

The priority for the request. This field may be ignored or overwritten depending on the organization tier.

One of the following:
"low"
"medium"
"high"
"critical"
project_id: Optional[str]
remove_hidden_text: Optional[bool]
replace_failed_page_mode: Optional[FailPageMode]

Enum for representing the different available page error handling modes.

One of the following:
"raw_text"
"blank_page"
"error_message"
replace_failed_page_with_error_message_prefix: Optional[str]
replace_failed_page_with_error_message_suffix: Optional[str]
save_images: Optional[bool]
skip_diagonal_text: Optional[bool]
specialized_chart_parsing_agentic: Optional[bool]
specialized_chart_parsing_efficient: Optional[bool]
specialized_chart_parsing_plus: Optional[bool]
specialized_image_parsing: Optional[bool]
spreadsheet_extract_sub_tables: Optional[bool]
spreadsheet_force_formula_computation: Optional[bool]
spreadsheet_include_hidden_sheets: Optional[bool]
strict_mode_buggy_font: Optional[bool]
strict_mode_image_extraction: Optional[bool]
strict_mode_image_ocr: Optional[bool]
strict_mode_reconstruction: Optional[bool]
structured_output: Optional[bool]
structured_output_json_schema: Optional[str]
structured_output_json_schema_name: Optional[str]
system_prompt: Optional[str]
system_prompt_append: Optional[str]
take_screenshot: Optional[bool]
target_pages: Optional[str]
tier: Optional[str]
use_vendor_multimodal_model: Optional[bool]
user_prompt: Optional[str]
vendor_multimodal_api_key: Optional[str]
vendor_multimodal_model_name: Optional[str]
version: Optional[str]
webhook_configurations: Optional[List[WebhookConfiguration]]

Outbound webhook endpoints to notify on job status changes

webhook_events: Optional[List[Literal["extract.pending", "extract.success", "extract.error", 14 more]]]

Events to subscribe to (e.g. ‘parse.success’, ‘extract.error’). If null, all events are delivered.

One of the following:
"extract.pending"
"extract.success"
"extract.error"
"extract.partial_success"
"extract.cancelled"
"parse.pending"
"parse.running"
"parse.success"
"parse.error"
"parse.partial_success"
"parse.cancelled"
"classify.pending"
"classify.success"
"classify.error"
"classify.partial_success"
"classify.cancelled"
"unmapped_event"
webhook_headers: Optional[Dict[str, str]]

Custom HTTP headers sent with each webhook request (e.g. auth tokens)

webhook_output_format: Optional[str]

Response format sent to the webhook: ‘string’ (default) or ‘json’

webhook_url: Optional[str]

URL to receive webhook POST notifications

webhook_url: Optional[str]
managed_pipeline_id: Optional[str]

The ID of the ManagedPipeline this playground pipeline is linked to.

formatuuid
metadata_config: Optional[PipelineMetadataConfig]

Metadata configuration for the pipeline.

excluded_embed_metadata_keys: Optional[List[str]]

List of metadata keys to exclude from embeddings

excluded_llm_metadata_keys: Optional[List[str]]

List of metadata keys to exclude from LLM during retrieval

pipeline_type: Optional[PipelineType]

Type of pipeline. Either PLAYGROUND or MANAGED.

One of the following:
"PLAYGROUND"
"MANAGED"
preset_retrieval_parameters: Optional[PresetRetrievalParams]

Preset retrieval parameters for the pipeline.

alpha: Optional[float]

Alpha value for hybrid retrieval to determine the weights between dense and sparse retrieval. 0 is sparse retrieval and 1 is dense retrieval.

maximum1
minimum0
class_name: Optional[str]
dense_similarity_cutoff: Optional[float]

Minimum similarity score wrt query for retrieval

maximum1
minimum0
dense_similarity_top_k: Optional[int]

Number of nodes for dense retrieval.

maximum100
minimum1
enable_reranking: Optional[bool]

Enable reranking for retrieval

files_top_k: Optional[int]

Number of files to retrieve (only for retrieval mode files_via_metadata and files_via_content).

maximum5
minimum1
rerank_top_n: Optional[int]

Number of reranked nodes for returning.

maximum100
minimum1
retrieval_mode: Optional[RetrievalMode]

The retrieval mode for the query.

One of the following:
"chunks"
"files_via_metadata"
"files_via_content"
"auto_routed"
Deprecatedretrieve_image_nodes: Optional[bool]

Whether to retrieve image nodes.

retrieve_page_figure_nodes: Optional[bool]

Whether to retrieve page figure nodes.

retrieve_page_screenshot_nodes: Optional[bool]

Whether to retrieve page screenshot nodes.

search_filters: Optional[MetadataFilters]

Metadata filters for vector stores.

filters: List[Filter]
One of the following:
class FilterMetadataFilter: …

Comprehensive metadata filter for vector stores to support more operators.

Value uses Strict types, as int, float and str are compatible types and were all converted to string before.

See: https://docs.pydantic.dev/latest/usage/types/#strict-types

key: str
value: Union[float, str, List[str], 3 more]
One of the following:
float
str
List[str]
List[float]
List[int]
operator: Optional[Literal["==", ">", "<", 11 more]]

Vector store filter operator.

One of the following:
"=="
">"
"<"
"!="
">="
"<="
"in"
"nin"
"any"
"all"
"text_match"
"text_match_insensitive"
"contains"
"is_empty"
condition: Optional[Literal["and", "or", "not"]]

Vector store filter conditions to combine different filters.

One of the following:
"and"
"or"
"not"
search_filters_inference_schema: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

JSON Schema that will be used to infer search_filters. Omit or leave as null to skip inference.

One of the following:
Dict[str, object]
List[object]
str
float
bool
sparse_similarity_top_k: Optional[int]

Number of nodes for sparse retrieval.

maximum100
minimum1
sparse_model_config: Optional[SparseModelConfig]

Configuration for sparse embedding models used in hybrid search.

This allows users to choose between Splade and BM25 models for sparse retrieval in managed data sinks.

class_name: Optional[str]
model_type: Optional[Literal["splade", "bm25", "auto"]]

The sparse model type to use. ‘bm25’ uses Qdrant’s FastEmbed BM25 model (default for new pipelines), ‘splade’ uses HuggingFace Splade model, ‘auto’ selects based on deployment mode (BYOC uses term frequency, Cloud uses Splade).

One of the following:
"splade"
"bm25"
"auto"
status: Optional[Literal["CREATED", "DELETING"]]

Status of the pipeline.

One of the following:
"CREATED"
"DELETING"
transform_config: Optional[TransformConfig]

Configuration for the transformation.

One of the following:
class AutoTransformConfig: …
chunk_overlap: Optional[int]

Chunk overlap for the transformation.

chunk_size: Optional[int]

Chunk size for the transformation.

exclusiveMinimum0
mode: Optional[Literal["auto"]]
class AdvancedModeTransformConfig: …
chunking_config: Optional[ChunkingConfig]

Configuration for the chunking.

One of the following:
class ChunkingConfigNoneChunkingConfig: …
mode: Optional[Literal["none"]]
class ChunkingConfigCharacterChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["character"]]
class ChunkingConfigTokenChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["token"]]
separator: Optional[str]
class ChunkingConfigSentenceChunkingConfig: …
chunk_overlap: Optional[int]
chunk_size: Optional[int]
mode: Optional[Literal["sentence"]]
paragraph_separator: Optional[str]
separator: Optional[str]
class ChunkingConfigSemanticChunkingConfig: …
breakpoint_percentile_threshold: Optional[int]
buffer_size: Optional[int]
mode: Optional[Literal["semantic"]]
mode: Optional[Literal["advanced"]]
segmentation_config: Optional[SegmentationConfig]

Configuration for the segmentation.

One of the following:
class SegmentationConfigNoneSegmentationConfig: …
mode: Optional[Literal["none"]]
class SegmentationConfigPageSegmentationConfig: …
mode: Optional[Literal["page"]]
page_separator: Optional[str]
class SegmentationConfigElementSegmentationConfig: …
mode: Optional[Literal["element"]]
updated_at: Optional[datetime]

Update datetime

formatdate-time
class PipelineRetrieveResponse: …

Schema for the result of an retrieval execution.

pipeline_id: str

The ID of the pipeline that the query was retrieved against.

formatuuid
retrieval_nodes: List[RetrievalNode]

The nodes retrieved by the pipeline for the given query.

node: TextNode

Provided for backward compatibility.

class_name: Optional[str]
embedding: Optional[List[float]]

Embedding of the node.

end_char_idx: Optional[int]

End char index of the node.

excluded_embed_metadata_keys: Optional[List[str]]

Metadata keys that are excluded from text for the embed model.

excluded_llm_metadata_keys: Optional[List[str]]

Metadata keys that are excluded from text for the LLM.

extra_info: Optional[Dict[str, object]]

A flat dictionary of metadata fields

id: Optional[str]

Unique ID of the node.

metadata_seperator: Optional[str]

Separator between metadata fields when converting to string.

metadata_template: Optional[str]

Template for how metadata is formatted, with {key} and {value} placeholders.

mimetype: Optional[str]

MIME type of the node content.

relationships: Optional[Dict[str, Relationships]]

A mapping of relationships to other node information.

One of the following:
class RelationshipsRelatedNodeInfo: …
node_id: str
class_name: Optional[str]
hash: Optional[str]
metadata: Optional[Dict[str, object]]
node_type: Optional[Union[Literal["1", "2", "3", 2 more], str, null]]
One of the following:
Literal["1", "2", "3", 2 more]
One of the following:
"1"
"2"
"3"
"4"
"5"
str
List[RelationshipsUnionMember1]
node_id: str
class_name: Optional[str]
hash: Optional[str]
metadata: Optional[Dict[str, object]]
node_type: Optional[Union[Literal["1", "2", "3", 2 more], str, null]]
One of the following:
Literal["1", "2", "3", 2 more]
One of the following:
"1"
"2"
"3"
"4"
"5"
str
start_char_idx: Optional[int]

Start char index of the node.

text: Optional[str]

Text content of the node.

text_template: Optional[str]

Template for how text is formatted, with {content} and {metadata_str} placeholders.

class_name: Optional[str]
score: Optional[float]
class_name: Optional[str]
Deprecatedimage_nodes: Optional[List[PageScreenshotNodeWithScore]]

The image nodes retrieved by the pipeline for the given query. Deprecated - will soon be replaced with ‘page_screenshot_nodes’.

node: Node
file_id: str

The ID of the file that the page screenshot was taken from

formatuuid
image_size: int

The size of the image in bytes

minimum0
page_index: int

The index of the page for which the screenshot is taken (0-indexed)

minimum0
metadata: Optional[Dict[str, object]]

Metadata for the screenshot

score: float

The score of the screenshot node

class_name: Optional[str]
inferred_search_filters: Optional[MetadataFilters]

Metadata filters for vector stores.

filters: List[Filter]
One of the following:
class FilterMetadataFilter: …

Comprehensive metadata filter for vector stores to support more operators.

Value uses Strict types, as int, float and str are compatible types and were all converted to string before.

See: https://docs.pydantic.dev/latest/usage/types/#strict-types

key: str
value: Union[float, str, List[str], 3 more]
One of the following:
float
str
List[str]
List[float]
List[int]
operator: Optional[Literal["==", ">", "<", 11 more]]

Vector store filter operator.

One of the following:
"=="
">"
"<"
"!="
">="
"<="
"in"
"nin"
"any"
"all"
"text_match"
"text_match_insensitive"
"contains"
"is_empty"
condition: Optional[Literal["and", "or", "not"]]

Vector store filter conditions to combine different filters.

One of the following:
"and"
"or"
"not"
metadata: Optional[Dict[str, str]]

Metadata associated with the retrieval execution

page_figure_nodes: Optional[List[PageFigureNodeWithScore]]

The page figure nodes retrieved by the pipeline for the given query.

node: Node
confidence: float

The confidence of the figure

maximum1
minimum0
figure_name: str

The name of the figure

figure_size: int

The size of the figure in bytes

minimum0
file_id: str

The ID of the file that the figure was taken from

formatuuid
page_index: int

The index of the page for which the figure is taken (0-indexed)

minimum0
is_likely_noise: Optional[bool]

Whether the figure is likely to be noise

metadata: Optional[Dict[str, object]]

Metadata for the figure

score: float

The score of the figure node

class_name: Optional[str]
retrieval_latency: Optional[Dict[str, float]]

The end-to-end latency for retrieval and reranking.

PipelinesSync

Sync Pipeline
pipelines.sync.create(strpipeline_id) -> Pipeline
POST/api/v1/pipelines/{pipeline_id}/sync
Cancel Pipeline Sync
pipelines.sync.cancel(strpipeline_id) -> Pipeline
POST/api/v1/pipelines/{pipeline_id}/sync/cancel

PipelinesData Sources

List Pipeline Data Sources
pipelines.data_sources.get_data_sources(strpipeline_id) -> DataSourceGetDataSourcesResponse
GET/api/v1/pipelines/{pipeline_id}/data-sources
Add Data Sources To Pipeline
pipelines.data_sources.update_data_sources(strpipeline_id, DataSourceUpdateDataSourcesParams**kwargs) -> DataSourceUpdateDataSourcesResponse
PUT/api/v1/pipelines/{pipeline_id}/data-sources
Update Pipeline Data Source
pipelines.data_sources.update(strdata_source_id, DataSourceUpdateParams**kwargs) -> PipelineDataSource
PUT/api/v1/pipelines/{pipeline_id}/data-sources/{data_source_id}
Get Pipeline Data Source Status
pipelines.data_sources.get_status(strdata_source_id, DataSourceGetStatusParams**kwargs) -> ManagedIngestionStatusResponse
GET/api/v1/pipelines/{pipeline_id}/data-sources/{data_source_id}/status
Sync Pipeline Data Source
pipelines.data_sources.sync(strdata_source_id, DataSourceSyncParams**kwargs) -> Pipeline
POST/api/v1/pipelines/{pipeline_id}/data-sources/{data_source_id}/sync
ModelsExpand Collapse
class PipelineDataSource: …

Schema for a data source in a pipeline.

id: str

Unique identifier

formatuuid
component: Component

Component that implements the data source

One of the following:
Dict[str, object]
class CloudS3DataSource: …
bucket: str

The name of the S3 bucket to read from.

aws_access_id: Optional[str]

The AWS access ID to use for authentication.

aws_access_secret: Optional[str]

The AWS access secret to use for authentication.

formatpassword
class_name: Optional[str]
prefix: Optional[str]

The prefix of the S3 objects to read from.

regex_pattern: Optional[str]

The regex pattern to filter S3 objects. Must be a valid regex pattern.

s3_endpoint_url: Optional[str]

The S3 endpoint URL to use for authentication.

supports_access_control: Optional[bool]
class CloudAzStorageBlobDataSource: …
account_url: str

The Azure Storage Blob account URL to use for authentication.

container_name: str

The name of the Azure Storage Blob container to read from.

account_key: Optional[str]

The Azure Storage Blob account key to use for authentication.

formatpassword
account_name: Optional[str]

The Azure Storage Blob account name to use for authentication.

blob: Optional[str]

The blob name to read from.

class_name: Optional[str]
client_id: Optional[str]

The Azure AD client ID to use for authentication.

client_secret: Optional[str]

The Azure AD client secret to use for authentication.

formatpassword
prefix: Optional[str]

The prefix of the Azure Storage Blob objects to read from.

supports_access_control: Optional[bool]
tenant_id: Optional[str]

The Azure AD tenant ID to use for authentication.

class CloudGoogleDriveDataSource: …
folder_id: str

The ID of the Google Drive folder to read from.

class_name: Optional[str]
service_account_key: Optional[Dict[str, str]]

A dictionary containing secret values

supports_access_control: Optional[bool]
class CloudOneDriveDataSource: …
client_id: str

The client ID to use for authentication.

client_secret: str

The client secret to use for authentication.

formatpassword
tenant_id: str

The tenant ID to use for authentication.

user_principal_name: str

The user principal name to use for authentication.

class_name: Optional[str]
folder_id: Optional[str]

The ID of the OneDrive folder to read from.

folder_path: Optional[str]

The path of the OneDrive folder to read from.

required_exts: Optional[List[str]]

The list of required file extensions.

supports_access_control: Optional[Literal[true]]
class CloudSharepointDataSource: …
client_id: str

The client ID to use for authentication.

client_secret: str

The client secret to use for authentication.

formatpassword
tenant_id: str

The tenant ID to use for authentication.

class_name: Optional[str]
drive_name: Optional[str]

The name of the Sharepoint drive to read from.

exclude_path_patterns: Optional[List[str]]

List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: [‘/temp/’, ‘/backup/’, ‘.git/’, ‘.tmp$’, ’^~’]

folder_id: Optional[str]

The ID of the Sharepoint folder to read from.

folder_path: Optional[str]

The path of the Sharepoint folder to read from.

get_permissions: Optional[bool]

Whether to get permissions for the sharepoint site.

include_path_patterns: Optional[List[str]]

List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: [‘/reports/’, ‘/docs/..pdf$’, ‘^Report..pdf$’]

required_exts: Optional[List[str]]

The list of required file extensions.

site_id: Optional[str]

The ID of the SharePoint site to download from.

site_name: Optional[str]

The name of the SharePoint site to download from.

supports_access_control: Optional[Literal[true]]
class CloudSlackDataSource: …
slack_token: str

Slack Bot Token.

formatpassword
channel_ids: Optional[str]

Slack Channel.

channel_patterns: Optional[str]

Slack Channel name pattern.

class_name: Optional[str]
earliest_date: Optional[str]

Earliest date.

earliest_date_timestamp: Optional[float]

Earliest date timestamp.

latest_date: Optional[str]

Latest date.

latest_date_timestamp: Optional[float]

Latest date timestamp.

supports_access_control: Optional[bool]
class CloudNotionPageDataSource: …
integration_token: str

The integration token to use for authentication.

formatpassword
class_name: Optional[str]
database_ids: Optional[str]

The Notion Database Id to read content from.

page_ids: Optional[str]

The Page ID’s of the Notion to read from.

supports_access_control: Optional[bool]
class CloudConfluenceDataSource: …
authentication_mechanism: str

Type of Authentication for connecting to Confluence APIs.

server_url: str

The server URL of the Confluence instance.

api_token: Optional[str]

The API token to use for authentication.

formatpassword
class_name: Optional[str]
cql: Optional[str]

The CQL query to use for fetching pages.

failure_handling: Optional[FailureHandlingConfig]

Configuration for handling failures during processing. Key-value object controlling failure handling behaviors.

Example: { “skip_list_failures”: true }

Currently supports:

  • skip_list_failures: Skip failed batches/lists and continue processing
skip_list_failures: Optional[bool]

Whether to skip failed batches/lists and continue processing

index_restricted_pages: Optional[bool]

Whether to index restricted pages.

keep_markdown_format: Optional[bool]

Whether to keep the markdown format.

label: Optional[str]

The label to use for fetching pages.

page_ids: Optional[str]

The page IDs of the Confluence to read from.

space_key: Optional[str]

The space key to read from.

supports_access_control: Optional[bool]
user_name: Optional[str]

The username to use for authentication.

class CloudJiraDataSource: …

Cloud Jira Data Source integrating JiraReader.

authentication_mechanism: str

Type of Authentication for connecting to Jira APIs.

query: str

JQL (Jira Query Language) query to search.

api_token: Optional[str]

The API/ Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
class_name: Optional[str]
cloud_id: Optional[str]

The cloud ID, used in case of OAuth2.

email: Optional[str]

The email address to use for authentication.

server_url: Optional[str]

The server url for Jira Cloud.

supports_access_control: Optional[bool]
class CloudJiraDataSourceV2: …

Cloud Jira Data Source integrating JiraReaderV2.

authentication_mechanism: str

Type of Authentication for connecting to Jira APIs.

query: str

JQL (Jira Query Language) query to search.

server_url: str

The server url for Jira Cloud.

api_token: Optional[str]

The API Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
api_version: Optional[Literal["2", "3"]]

Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF).

One of the following:
"2"
"3"
class_name: Optional[str]
cloud_id: Optional[str]

The cloud ID, used in case of OAuth2.

email: Optional[str]

The email address to use for authentication.

expand: Optional[str]

Fields to expand in the response.

fields: Optional[List[str]]

List of fields to retrieve from Jira. If None, retrieves all fields.

get_permissions: Optional[bool]

Whether to fetch project role permissions and issue-level security

requests_per_minute: Optional[int]

Rate limit for Jira API requests per minute.

supports_access_control: Optional[bool]
class CloudBoxDataSource: …
authentication_mechanism: Literal["developer_token", "ccg"]

The type of authentication to use (Developer Token or CCG)

One of the following:
"developer_token"
"ccg"
class_name: Optional[str]
client_id: Optional[str]

Box API key used for identifying the application the user is authenticating with

client_secret: Optional[str]

Box API secret used for making auth requests.

formatpassword
developer_token: Optional[str]

Developer token for authentication if authentication_mechanism is ‘developer_token’.

formatpassword
enterprise_id: Optional[str]

Box Enterprise ID, if provided authenticates as service.

folder_id: Optional[str]

The ID of the Box folder to read from.

supports_access_control: Optional[bool]
user_id: Optional[str]

Box User ID, if provided authenticates as user.

data_source_id: str

The ID of the data source.

formatuuid
last_synced_at: datetime

The last time the data source was automatically synced.

formatdate-time
name: str

The name of the data source.

pipeline_id: str

The ID of the pipeline.

formatuuid
project_id: str
source_type: Literal["S3", "AZURE_STORAGE_BLOB", "GOOGLE_DRIVE", 8 more]
One of the following:
"S3"
"AZURE_STORAGE_BLOB"
"GOOGLE_DRIVE"
"MICROSOFT_ONEDRIVE"
"MICROSOFT_SHAREPOINT"
"SLACK"
"NOTION_PAGE"
"CONFLUENCE"
"JIRA"
"JIRA_V2"
"BOX"
created_at: Optional[datetime]

Creation datetime

formatdate-time
custom_metadata: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Custom metadata that will be present on all data loaded from the data source

One of the following:
Dict[str, object]
List[object]
str
float
bool
status: Optional[Literal["NOT_STARTED", "IN_PROGRESS", "SUCCESS", 2 more]]

The status of the data source in the pipeline.

One of the following:
"NOT_STARTED"
"IN_PROGRESS"
"SUCCESS"
"ERROR"
"CANCELLED"
status_updated_at: Optional[datetime]

The last time the status was updated.

formatdate-time
sync_interval: Optional[float]

The interval at which the data source should be synced.

sync_schedule_set_by: Optional[str]

The id of the user who set the sync schedule.

updated_at: Optional[datetime]

Update datetime

formatdate-time
version_metadata: Optional[DataSourceReaderVersionMetadata]

Version metadata for the data source

reader_version: Optional[Literal["1.0", "2.0", "2.1"]]

The version of the reader to use for this data source.

One of the following:
"1.0"
"2.0"
"2.1"
id: str

Unique identifier

formatuuid
component: Component

Component that implements the data source

One of the following:
Dict[str, object]
class CloudS3DataSource: …
bucket: str

The name of the S3 bucket to read from.

aws_access_id: Optional[str]

The AWS access ID to use for authentication.

aws_access_secret: Optional[str]

The AWS access secret to use for authentication.

formatpassword
class_name: Optional[str]
prefix: Optional[str]

The prefix of the S3 objects to read from.

regex_pattern: Optional[str]

The regex pattern to filter S3 objects. Must be a valid regex pattern.

s3_endpoint_url: Optional[str]

The S3 endpoint URL to use for authentication.

supports_access_control: Optional[bool]
class CloudAzStorageBlobDataSource: …
account_url: str

The Azure Storage Blob account URL to use for authentication.

container_name: str

The name of the Azure Storage Blob container to read from.

account_key: Optional[str]

The Azure Storage Blob account key to use for authentication.

formatpassword
account_name: Optional[str]

The Azure Storage Blob account name to use for authentication.

blob: Optional[str]

The blob name to read from.

class_name: Optional[str]
client_id: Optional[str]

The Azure AD client ID to use for authentication.

client_secret: Optional[str]

The Azure AD client secret to use for authentication.

formatpassword
prefix: Optional[str]

The prefix of the Azure Storage Blob objects to read from.

supports_access_control: Optional[bool]
tenant_id: Optional[str]

The Azure AD tenant ID to use for authentication.

class CloudGoogleDriveDataSource: …
folder_id: str

The ID of the Google Drive folder to read from.

class_name: Optional[str]
service_account_key: Optional[Dict[str, str]]

A dictionary containing secret values

supports_access_control: Optional[bool]
class CloudOneDriveDataSource: …
client_id: str

The client ID to use for authentication.

client_secret: str

The client secret to use for authentication.

formatpassword
tenant_id: str

The tenant ID to use for authentication.

user_principal_name: str

The user principal name to use for authentication.

class_name: Optional[str]
folder_id: Optional[str]

The ID of the OneDrive folder to read from.

folder_path: Optional[str]

The path of the OneDrive folder to read from.

required_exts: Optional[List[str]]

The list of required file extensions.

supports_access_control: Optional[Literal[true]]
class CloudSharepointDataSource: …
client_id: str

The client ID to use for authentication.

client_secret: str

The client secret to use for authentication.

formatpassword
tenant_id: str

The tenant ID to use for authentication.

class_name: Optional[str]
drive_name: Optional[str]

The name of the Sharepoint drive to read from.

exclude_path_patterns: Optional[List[str]]

List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: [‘/temp/’, ‘/backup/’, ‘.git/’, ‘.tmp$’, ’^~’]

folder_id: Optional[str]

The ID of the Sharepoint folder to read from.

folder_path: Optional[str]

The path of the Sharepoint folder to read from.

get_permissions: Optional[bool]

Whether to get permissions for the sharepoint site.

include_path_patterns: Optional[List[str]]

List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: [‘/reports/’, ‘/docs/..pdf$’, ‘^Report..pdf$’]

required_exts: Optional[List[str]]

The list of required file extensions.

site_id: Optional[str]

The ID of the SharePoint site to download from.

site_name: Optional[str]

The name of the SharePoint site to download from.

supports_access_control: Optional[Literal[true]]
class CloudSlackDataSource: …
slack_token: str

Slack Bot Token.

formatpassword
channel_ids: Optional[str]

Slack Channel.

channel_patterns: Optional[str]

Slack Channel name pattern.

class_name: Optional[str]
earliest_date: Optional[str]

Earliest date.

earliest_date_timestamp: Optional[float]

Earliest date timestamp.

latest_date: Optional[str]

Latest date.

latest_date_timestamp: Optional[float]

Latest date timestamp.

supports_access_control: Optional[bool]
class CloudNotionPageDataSource: …
integration_token: str

The integration token to use for authentication.

formatpassword
class_name: Optional[str]
database_ids: Optional[str]

The Notion Database Id to read content from.

page_ids: Optional[str]

The Page ID’s of the Notion to read from.

supports_access_control: Optional[bool]
class CloudConfluenceDataSource: …
authentication_mechanism: str

Type of Authentication for connecting to Confluence APIs.

server_url: str

The server URL of the Confluence instance.

api_token: Optional[str]

The API token to use for authentication.

formatpassword
class_name: Optional[str]
cql: Optional[str]

The CQL query to use for fetching pages.

failure_handling: Optional[FailureHandlingConfig]

Configuration for handling failures during processing. Key-value object controlling failure handling behaviors.

Example: { “skip_list_failures”: true }

Currently supports:

  • skip_list_failures: Skip failed batches/lists and continue processing
skip_list_failures: Optional[bool]

Whether to skip failed batches/lists and continue processing

index_restricted_pages: Optional[bool]

Whether to index restricted pages.

keep_markdown_format: Optional[bool]

Whether to keep the markdown format.

label: Optional[str]

The label to use for fetching pages.

page_ids: Optional[str]

The page IDs of the Confluence to read from.

space_key: Optional[str]

The space key to read from.

supports_access_control: Optional[bool]
user_name: Optional[str]

The username to use for authentication.

class CloudJiraDataSource: …

Cloud Jira Data Source integrating JiraReader.

authentication_mechanism: str

Type of Authentication for connecting to Jira APIs.

query: str

JQL (Jira Query Language) query to search.

api_token: Optional[str]

The API/ Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
class_name: Optional[str]
cloud_id: Optional[str]

The cloud ID, used in case of OAuth2.

email: Optional[str]

The email address to use for authentication.

server_url: Optional[str]

The server url for Jira Cloud.

supports_access_control: Optional[bool]
class CloudJiraDataSourceV2: …

Cloud Jira Data Source integrating JiraReaderV2.

authentication_mechanism: str

Type of Authentication for connecting to Jira APIs.

query: str

JQL (Jira Query Language) query to search.

server_url: str

The server url for Jira Cloud.

api_token: Optional[str]

The API Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
api_version: Optional[Literal["2", "3"]]

Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF).

One of the following:
"2"
"3"
class_name: Optional[str]
cloud_id: Optional[str]

The cloud ID, used in case of OAuth2.

email: Optional[str]

The email address to use for authentication.

expand: Optional[str]

Fields to expand in the response.

fields: Optional[List[str]]

List of fields to retrieve from Jira. If None, retrieves all fields.

get_permissions: Optional[bool]

Whether to fetch project role permissions and issue-level security

requests_per_minute: Optional[int]

Rate limit for Jira API requests per minute.

supports_access_control: Optional[bool]
class CloudBoxDataSource: …
authentication_mechanism: Literal["developer_token", "ccg"]

The type of authentication to use (Developer Token or CCG)

One of the following:
"developer_token"
"ccg"
class_name: Optional[str]
client_id: Optional[str]

Box API key used for identifying the application the user is authenticating with

client_secret: Optional[str]

Box API secret used for making auth requests.

formatpassword
developer_token: Optional[str]

Developer token for authentication if authentication_mechanism is ‘developer_token’.

formatpassword
enterprise_id: Optional[str]

Box Enterprise ID, if provided authenticates as service.

folder_id: Optional[str]

The ID of the Box folder to read from.

supports_access_control: Optional[bool]
user_id: Optional[str]

Box User ID, if provided authenticates as user.

data_source_id: str

The ID of the data source.

formatuuid
last_synced_at: datetime

The last time the data source was automatically synced.

formatdate-time
name: str

The name of the data source.

pipeline_id: str

The ID of the pipeline.

formatuuid
project_id: str
source_type: Literal["S3", "AZURE_STORAGE_BLOB", "GOOGLE_DRIVE", 8 more]
One of the following:
"S3"
"AZURE_STORAGE_BLOB"
"GOOGLE_DRIVE"
"MICROSOFT_ONEDRIVE"
"MICROSOFT_SHAREPOINT"
"SLACK"
"NOTION_PAGE"
"CONFLUENCE"
"JIRA"
"JIRA_V2"
"BOX"
created_at: Optional[datetime]

Creation datetime

formatdate-time
custom_metadata: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Custom metadata that will be present on all data loaded from the data source

One of the following:
Dict[str, object]
List[object]
str
float
bool
status: Optional[Literal["NOT_STARTED", "IN_PROGRESS", "SUCCESS", 2 more]]

The status of the data source in the pipeline.

One of the following:
"NOT_STARTED"
"IN_PROGRESS"
"SUCCESS"
"ERROR"
"CANCELLED"
status_updated_at: Optional[datetime]

The last time the status was updated.

formatdate-time
sync_interval: Optional[float]

The interval at which the data source should be synced.

sync_schedule_set_by: Optional[str]

The id of the user who set the sync schedule.

updated_at: Optional[datetime]

Update datetime

formatdate-time
version_metadata: Optional[DataSourceReaderVersionMetadata]

Version metadata for the data source

reader_version: Optional[Literal["1.0", "2.0", "2.1"]]

The version of the reader to use for this data source.

One of the following:
"1.0"
"2.0"
"2.1"
id: str

Unique identifier

formatuuid
component: Component

Component that implements the data source

One of the following:
Dict[str, object]
class CloudS3DataSource: …
bucket: str

The name of the S3 bucket to read from.

aws_access_id: Optional[str]

The AWS access ID to use for authentication.

aws_access_secret: Optional[str]

The AWS access secret to use for authentication.

formatpassword
class_name: Optional[str]
prefix: Optional[str]

The prefix of the S3 objects to read from.

regex_pattern: Optional[str]

The regex pattern to filter S3 objects. Must be a valid regex pattern.

s3_endpoint_url: Optional[str]

The S3 endpoint URL to use for authentication.

supports_access_control: Optional[bool]
class CloudAzStorageBlobDataSource: …
account_url: str

The Azure Storage Blob account URL to use for authentication.

container_name: str

The name of the Azure Storage Blob container to read from.

account_key: Optional[str]

The Azure Storage Blob account key to use for authentication.

formatpassword
account_name: Optional[str]

The Azure Storage Blob account name to use for authentication.

blob: Optional[str]

The blob name to read from.

class_name: Optional[str]
client_id: Optional[str]

The Azure AD client ID to use for authentication.

client_secret: Optional[str]

The Azure AD client secret to use for authentication.

formatpassword
prefix: Optional[str]

The prefix of the Azure Storage Blob objects to read from.

supports_access_control: Optional[bool]
tenant_id: Optional[str]

The Azure AD tenant ID to use for authentication.

class CloudGoogleDriveDataSource: …
folder_id: str

The ID of the Google Drive folder to read from.

class_name: Optional[str]
service_account_key: Optional[Dict[str, str]]

A dictionary containing secret values

supports_access_control: Optional[bool]
class CloudOneDriveDataSource: …
client_id: str

The client ID to use for authentication.

client_secret: str

The client secret to use for authentication.

formatpassword
tenant_id: str

The tenant ID to use for authentication.

user_principal_name: str

The user principal name to use for authentication.

class_name: Optional[str]
folder_id: Optional[str]

The ID of the OneDrive folder to read from.

folder_path: Optional[str]

The path of the OneDrive folder to read from.

required_exts: Optional[List[str]]

The list of required file extensions.

supports_access_control: Optional[Literal[true]]
class CloudSharepointDataSource: …
client_id: str

The client ID to use for authentication.

client_secret: str

The client secret to use for authentication.

formatpassword
tenant_id: str

The tenant ID to use for authentication.

class_name: Optional[str]
drive_name: Optional[str]

The name of the Sharepoint drive to read from.

exclude_path_patterns: Optional[List[str]]

List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: [‘/temp/’, ‘/backup/’, ‘.git/’, ‘.tmp$’, ’^~’]

folder_id: Optional[str]

The ID of the Sharepoint folder to read from.

folder_path: Optional[str]

The path of the Sharepoint folder to read from.

get_permissions: Optional[bool]

Whether to get permissions for the sharepoint site.

include_path_patterns: Optional[List[str]]

List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: [‘/reports/’, ‘/docs/..pdf$’, ‘^Report..pdf$’]

required_exts: Optional[List[str]]

The list of required file extensions.

site_id: Optional[str]

The ID of the SharePoint site to download from.

site_name: Optional[str]

The name of the SharePoint site to download from.

supports_access_control: Optional[Literal[true]]
class CloudSlackDataSource: …
slack_token: str

Slack Bot Token.

formatpassword
channel_ids: Optional[str]

Slack Channel.

channel_patterns: Optional[str]

Slack Channel name pattern.

class_name: Optional[str]
earliest_date: Optional[str]

Earliest date.

earliest_date_timestamp: Optional[float]

Earliest date timestamp.

latest_date: Optional[str]

Latest date.

latest_date_timestamp: Optional[float]

Latest date timestamp.

supports_access_control: Optional[bool]
class CloudNotionPageDataSource: …
integration_token: str

The integration token to use for authentication.

formatpassword
class_name: Optional[str]
database_ids: Optional[str]

The Notion Database Id to read content from.

page_ids: Optional[str]

The Page ID’s of the Notion to read from.

supports_access_control: Optional[bool]
class CloudConfluenceDataSource: …
authentication_mechanism: str

Type of Authentication for connecting to Confluence APIs.

server_url: str

The server URL of the Confluence instance.

api_token: Optional[str]

The API token to use for authentication.

formatpassword
class_name: Optional[str]
cql: Optional[str]

The CQL query to use for fetching pages.

failure_handling: Optional[FailureHandlingConfig]

Configuration for handling failures during processing. Key-value object controlling failure handling behaviors.

Example: { “skip_list_failures”: true }

Currently supports:

  • skip_list_failures: Skip failed batches/lists and continue processing
skip_list_failures: Optional[bool]

Whether to skip failed batches/lists and continue processing

index_restricted_pages: Optional[bool]

Whether to index restricted pages.

keep_markdown_format: Optional[bool]

Whether to keep the markdown format.

label: Optional[str]

The label to use for fetching pages.

page_ids: Optional[str]

The page IDs of the Confluence to read from.

space_key: Optional[str]

The space key to read from.

supports_access_control: Optional[bool]
user_name: Optional[str]

The username to use for authentication.

class CloudJiraDataSource: …

Cloud Jira Data Source integrating JiraReader.

authentication_mechanism: str

Type of Authentication for connecting to Jira APIs.

query: str

JQL (Jira Query Language) query to search.

api_token: Optional[str]

The API/ Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
class_name: Optional[str]
cloud_id: Optional[str]

The cloud ID, used in case of OAuth2.

email: Optional[str]

The email address to use for authentication.

server_url: Optional[str]

The server url for Jira Cloud.

supports_access_control: Optional[bool]
class CloudJiraDataSourceV2: …

Cloud Jira Data Source integrating JiraReaderV2.

authentication_mechanism: str

Type of Authentication for connecting to Jira APIs.

query: str

JQL (Jira Query Language) query to search.

server_url: str

The server url for Jira Cloud.

api_token: Optional[str]

The API Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
api_version: Optional[Literal["2", "3"]]

Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF).

One of the following:
"2"
"3"
class_name: Optional[str]
cloud_id: Optional[str]

The cloud ID, used in case of OAuth2.

email: Optional[str]

The email address to use for authentication.

expand: Optional[str]

Fields to expand in the response.

fields: Optional[List[str]]

List of fields to retrieve from Jira. If None, retrieves all fields.

get_permissions: Optional[bool]

Whether to fetch project role permissions and issue-level security

requests_per_minute: Optional[int]

Rate limit for Jira API requests per minute.

supports_access_control: Optional[bool]
class CloudBoxDataSource: …
authentication_mechanism: Literal["developer_token", "ccg"]

The type of authentication to use (Developer Token or CCG)

One of the following:
"developer_token"
"ccg"
class_name: Optional[str]
client_id: Optional[str]

Box API key used for identifying the application the user is authenticating with

client_secret: Optional[str]

Box API secret used for making auth requests.

formatpassword
developer_token: Optional[str]

Developer token for authentication if authentication_mechanism is ‘developer_token’.

formatpassword
enterprise_id: Optional[str]

Box Enterprise ID, if provided authenticates as service.

folder_id: Optional[str]

The ID of the Box folder to read from.

supports_access_control: Optional[bool]
user_id: Optional[str]

Box User ID, if provided authenticates as user.

data_source_id: str

The ID of the data source.

formatuuid
last_synced_at: datetime

The last time the data source was automatically synced.

formatdate-time
name: str

The name of the data source.

pipeline_id: str

The ID of the pipeline.

formatuuid
project_id: str
source_type: Literal["S3", "AZURE_STORAGE_BLOB", "GOOGLE_DRIVE", 8 more]
One of the following:
"S3"
"AZURE_STORAGE_BLOB"
"GOOGLE_DRIVE"
"MICROSOFT_ONEDRIVE"
"MICROSOFT_SHAREPOINT"
"SLACK"
"NOTION_PAGE"
"CONFLUENCE"
"JIRA"
"JIRA_V2"
"BOX"
created_at: Optional[datetime]

Creation datetime

formatdate-time
custom_metadata: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Custom metadata that will be present on all data loaded from the data source

One of the following:
Dict[str, object]
List[object]
str
float
bool
status: Optional[Literal["NOT_STARTED", "IN_PROGRESS", "SUCCESS", 2 more]]

The status of the data source in the pipeline.

One of the following:
"NOT_STARTED"
"IN_PROGRESS"
"SUCCESS"
"ERROR"
"CANCELLED"
status_updated_at: Optional[datetime]

The last time the status was updated.

formatdate-time
sync_interval: Optional[float]

The interval at which the data source should be synced.

sync_schedule_set_by: Optional[str]

The id of the user who set the sync schedule.

updated_at: Optional[datetime]

Update datetime

formatdate-time
version_metadata: Optional[DataSourceReaderVersionMetadata]

Version metadata for the data source

reader_version: Optional[Literal["1.0", "2.0", "2.1"]]

The version of the reader to use for this data source.

One of the following:
"1.0"
"2.0"
"2.1"

PipelinesImages

List File Page Screenshots
pipelines.images.list_page_screenshots(strid, ImageListPageScreenshotsParams**kwargs) -> ImageListPageScreenshotsResponse
GET/api/v1/files/{id}/page_screenshots
Get File Page Screenshot
pipelines.images.get_page_screenshot(intpage_index, ImageGetPageScreenshotParams**kwargs) -> object
GET/api/v1/files/{id}/page_screenshots/{page_index}
Get File Page Figure
pipelines.images.get_page_figure(strfigure_name, ImageGetPageFigureParams**kwargs) -> object
GET/api/v1/files/{id}/page-figures/{page_index}/{figure_name}
List File Pages Figures
pipelines.images.list_page_figures(strid, ImageListPageFiguresParams**kwargs) -> ImageListPageFiguresResponse
GET/api/v1/files/{id}/page-figures
ModelsExpand Collapse
List[ImageListPageScreenshotsResponseItem]
file_id: str

The ID of the file that the page screenshot was taken from

formatuuid
image_size: int

The size of the image in bytes

minimum0
page_index: int

The index of the page for which the screenshot is taken (0-indexed)

minimum0
metadata: Optional[Dict[str, object]]

Metadata for the screenshot

List[ImageListPageFiguresResponseItem]
confidence: float

The confidence of the figure

maximum1
minimum0
figure_name: str

The name of the figure

figure_size: int

The size of the figure in bytes

minimum0
file_id: str

The ID of the file that the figure was taken from

formatuuid
page_index: int

The index of the page for which the figure is taken (0-indexed)

minimum0
is_likely_noise: Optional[bool]

Whether the figure is likely to be noise

metadata: Optional[Dict[str, object]]

Metadata for the figure

PipelinesFiles

Get Pipeline File Status Counts
pipelines.files.get_status_counts(strpipeline_id, FileGetStatusCountsParams**kwargs) -> FileGetStatusCountsResponse
GET/api/v1/pipelines/{pipeline_id}/files/status-counts
Get Pipeline File Status
pipelines.files.get_status(strfile_id, FileGetStatusParams**kwargs) -> ManagedIngestionStatusResponse
GET/api/v1/pipelines/{pipeline_id}/files/{file_id}/status
Add Files To Pipeline Api
pipelines.files.create(strpipeline_id, FileCreateParams**kwargs) -> FileCreateResponse
PUT/api/v1/pipelines/{pipeline_id}/files
Update Pipeline File
pipelines.files.update(strfile_id, FileUpdateParams**kwargs) -> PipelineFile
PUT/api/v1/pipelines/{pipeline_id}/files/{file_id}
Delete Pipeline File
pipelines.files.delete(strfile_id, FileDeleteParams**kwargs)
DELETE/api/v1/pipelines/{pipeline_id}/files/{file_id}
List Pipeline Files2
Deprecated
pipelines.files.list(strpipeline_id, FileListParams**kwargs) -> SyncPaginatedPipelineFiles[PipelineFile]
GET/api/v1/pipelines/{pipeline_id}/files2
ModelsExpand Collapse
class PipelineFile: …

A file associated with a pipeline.

id: str

Unique identifier for the pipeline file.

formatuuid
pipeline_id: str

The ID of the pipeline that the file is associated with.

formatuuid
config_hash: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Hashes for the configuration of the pipeline.

One of the following:
Dict[str, object]
List[object]
str
float
bool
created_at: Optional[datetime]

When the pipeline file was created.

formatdate-time
custom_metadata: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Custom metadata for the file.

One of the following:
Dict[str, object]
List[object]
str
float
bool
data_source_id: Optional[str]

The ID of the data source that the file belongs to.

formatuuid
external_file_id: Optional[str]

The ID of the file in the external system.

file_id: Optional[str]

The ID of the file.

formatuuid
file_size: Optional[int]

Size of the file in bytes.

file_type: Optional[str]

File type (e.g. pdf, docx, etc.).

indexed_page_count: Optional[int]

The number of pages that have been indexed for this file.

last_modified_at: Optional[datetime]

The last modified time of the file.

formatdate-time
name: Optional[str]

Name of the file.

permission_info: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Permission information for the file.

One of the following:
Dict[str, object]
List[object]
str
float
bool
project_id: Optional[str]

The ID of the project that the file belongs to.

formatuuid
resource_info: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Resource information for the file.

One of the following:
Dict[str, object]
List[object]
str
float
bool
status: Optional[Literal["NOT_STARTED", "IN_PROGRESS", "SUCCESS", 2 more]]

Status of the pipeline file.

One of the following:
"NOT_STARTED"
"IN_PROGRESS"
"SUCCESS"
"ERROR"
"CANCELLED"
status_updated_at: Optional[datetime]

The last time the status was updated.

formatdate-time
updated_at: Optional[datetime]

When the pipeline file was last updated.

formatdate-time
class FileGetStatusCountsResponse: …
counts: Dict[str, int]

The counts of files by status

total_count: int

The total number of files

data_source_id: Optional[str]

The ID of the data source that the files belong to

formatuuid
only_manually_uploaded: Optional[bool]

Whether to only count manually uploaded files

pipeline_id: Optional[str]

The ID of the pipeline that the files belong to

formatuuid
id: str

Unique identifier for the pipeline file.

formatuuid
pipeline_id: str

The ID of the pipeline that the file is associated with.

formatuuid
config_hash: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Hashes for the configuration of the pipeline.

One of the following:
Dict[str, object]
List[object]
str
float
bool
created_at: Optional[datetime]

When the pipeline file was created.

formatdate-time
custom_metadata: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Custom metadata for the file.

One of the following:
Dict[str, object]
List[object]
str
float
bool
data_source_id: Optional[str]

The ID of the data source that the file belongs to.

formatuuid
external_file_id: Optional[str]

The ID of the file in the external system.

file_id: Optional[str]

The ID of the file.

formatuuid
file_size: Optional[int]

Size of the file in bytes.

file_type: Optional[str]

File type (e.g. pdf, docx, etc.).

indexed_page_count: Optional[int]

The number of pages that have been indexed for this file.

last_modified_at: Optional[datetime]

The last modified time of the file.

formatdate-time
name: Optional[str]

Name of the file.

permission_info: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Permission information for the file.

One of the following:
Dict[str, object]
List[object]
str
float
bool
project_id: Optional[str]

The ID of the project that the file belongs to.

formatuuid
resource_info: Optional[Dict[str, Union[Dict[str, object], List[object], str, 3 more]]]

Resource information for the file.

One of the following:
Dict[str, object]
List[object]
str
float
bool
status: Optional[Literal["NOT_STARTED", "IN_PROGRESS", "SUCCESS", 2 more]]

Status of the pipeline file.

One of the following:
"NOT_STARTED"
"IN_PROGRESS"
"SUCCESS"
"ERROR"
"CANCELLED"
status_updated_at: Optional[datetime]

The last time the status was updated.

formatdate-time
updated_at: Optional[datetime]

When the pipeline file was last updated.

formatdate-time

PipelinesMetadata

Import Pipeline Metadata
pipelines.metadata.create(strpipeline_id, MetadataCreateParams**kwargs) -> MetadataCreateResponse
PUT/api/v1/pipelines/{pipeline_id}/metadata
Delete Pipeline Files Metadata
pipelines.metadata.delete_all(strpipeline_id)
DELETE/api/v1/pipelines/{pipeline_id}/metadata
ModelsExpand Collapse
Dict[str, str]

PipelinesDocuments

Create Batch Pipeline Documents
pipelines.documents.create(strpipeline_id, DocumentCreateParams**kwargs) -> DocumentCreateResponse
POST/api/v1/pipelines/{pipeline_id}/documents
Paginated List Pipeline Documents
pipelines.documents.list(strpipeline_id, DocumentListParams**kwargs) -> SyncPaginatedCloudDocuments[CloudDocument]
GET/api/v1/pipelines/{pipeline_id}/documents/paginated
Get Pipeline Document
pipelines.documents.get(strdocument_id, DocumentGetParams**kwargs) -> CloudDocument
GET/api/v1/pipelines/{pipeline_id}/documents/{document_id}
Delete Pipeline Document
pipelines.documents.delete(strdocument_id, DocumentDeleteParams**kwargs)
DELETE/api/v1/pipelines/{pipeline_id}/documents/{document_id}
Get Pipeline Document Status
pipelines.documents.get_status(strdocument_id, DocumentGetStatusParams**kwargs) -> ManagedIngestionStatusResponse
GET/api/v1/pipelines/{pipeline_id}/documents/{document_id}/status
Sync Pipeline Document
pipelines.documents.sync(strdocument_id, DocumentSyncParams**kwargs) -> object
POST/api/v1/pipelines/{pipeline_id}/documents/{document_id}/sync
List Pipeline Document Chunks
pipelines.documents.get_chunks(strdocument_id, DocumentGetChunksParams**kwargs) -> DocumentGetChunksResponse
GET/api/v1/pipelines/{pipeline_id}/documents/{document_id}/chunks
Upsert Batch Pipeline Documents
pipelines.documents.upsert(strpipeline_id, DocumentUpsertParams**kwargs) -> DocumentUpsertResponse
PUT/api/v1/pipelines/{pipeline_id}/documents
ModelsExpand Collapse
class CloudDocument: …

Cloud document stored in S3.

id: str
metadata: Dict[str, object]
text: str
excluded_embed_metadata_keys: Optional[List[str]]
excluded_llm_metadata_keys: Optional[List[str]]
page_positions: Optional[List[int]]

indices in the CloudDocument.text where a new page begins. e.g. Second page starts at index specified by page_positions[1].

status_metadata: Optional[Dict[str, object]]
class CloudDocumentCreate: …

Create a new cloud document.

metadata: Dict[str, object]
text: str
id: Optional[str]
excluded_embed_metadata_keys: Optional[List[str]]
excluded_llm_metadata_keys: Optional[List[str]]
page_positions: Optional[List[int]]

indices in the CloudDocument.text where a new page begins. e.g. Second page starts at index specified by page_positions[1].

class TextNode: …

Provided for backward compatibility.

class_name: Optional[str]
embedding: Optional[List[float]]

Embedding of the node.

end_char_idx: Optional[int]

End char index of the node.

excluded_embed_metadata_keys: Optional[List[str]]

Metadata keys that are excluded from text for the embed model.

excluded_llm_metadata_keys: Optional[List[str]]

Metadata keys that are excluded from text for the LLM.

extra_info: Optional[Dict[str, object]]

A flat dictionary of metadata fields

id: Optional[str]

Unique ID of the node.

metadata_seperator: Optional[str]

Separator between metadata fields when converting to string.

metadata_template: Optional[str]

Template for how metadata is formatted, with {key} and {value} placeholders.

mimetype: Optional[str]

MIME type of the node content.

relationships: Optional[Dict[str, Relationships]]

A mapping of relationships to other node information.

One of the following:
class RelationshipsRelatedNodeInfo: …
node_id: str
class_name: Optional[str]
hash: Optional[str]
metadata: Optional[Dict[str, object]]
node_type: Optional[Union[Literal["1", "2", "3", 2 more], str, null]]
One of the following:
Literal["1", "2", "3", 2 more]
One of the following:
"1"
"2"
"3"
"4"
"5"
str
List[RelationshipsUnionMember1]
node_id: str
class_name: Optional[str]
hash: Optional[str]
metadata: Optional[Dict[str, object]]
node_type: Optional[Union[Literal["1", "2", "3", 2 more], str, null]]
One of the following:
Literal["1", "2", "3", 2 more]
One of the following:
"1"
"2"
"3"
"4"
"5"
str
start_char_idx: Optional[int]

Start char index of the node.

text: Optional[str]

Text content of the node.

text_template: Optional[str]

Template for how text is formatted, with {content} and {metadata_str} placeholders.

id: str
metadata: Dict[str, object]
text: str
excluded_embed_metadata_keys: Optional[List[str]]
excluded_llm_metadata_keys: Optional[List[str]]
page_positions: Optional[List[int]]

indices in the CloudDocument.text where a new page begins. e.g. Second page starts at index specified by page_positions[1].

status_metadata: Optional[Dict[str, object]]
List[TextNode]
class_name: Optional[str]
embedding: Optional[List[float]]

Embedding of the node.

end_char_idx: Optional[int]

End char index of the node.

excluded_embed_metadata_keys: Optional[List[str]]

Metadata keys that are excluded from text for the embed model.

excluded_llm_metadata_keys: Optional[List[str]]

Metadata keys that are excluded from text for the LLM.

extra_info: Optional[Dict[str, object]]

A flat dictionary of metadata fields

id: Optional[str]

Unique ID of the node.

metadata_seperator: Optional[str]

Separator between metadata fields when converting to string.

metadata_template: Optional[str]

Template for how metadata is formatted, with {key} and {value} placeholders.

mimetype: Optional[str]

MIME type of the node content.

relationships: Optional[Dict[str, Relationships]]

A mapping of relationships to other node information.

One of the following:
class RelationshipsRelatedNodeInfo: …
node_id: str
class_name: Optional[str]
hash: Optional[str]
metadata: Optional[Dict[str, object]]
node_type: Optional[Union[Literal["1", "2", "3", 2 more], str, null]]
One of the following:
Literal["1", "2", "3", 2 more]
One of the following:
"1"
"2"
"3"
"4"
"5"
str
List[RelationshipsUnionMember1]
node_id: str
class_name: Optional[str]
hash: Optional[str]
metadata: Optional[Dict[str, object]]
node_type: Optional[Union[Literal["1", "2", "3", 2 more], str, null]]
One of the following:
Literal["1", "2", "3", 2 more]
One of the following:
"1"
"2"
"3"
"4"
"5"
str
start_char_idx: Optional[int]

Start char index of the node.

text: Optional[str]

Text content of the node.

text_template: Optional[str]

Template for how text is formatted, with {content} and {metadata_str} placeholders.

id: str
metadata: Dict[str, object]
text: str
excluded_embed_metadata_keys: Optional[List[str]]
excluded_llm_metadata_keys: Optional[List[str]]
page_positions: Optional[List[int]]

indices in the CloudDocument.text where a new page begins. e.g. Second page starts at index specified by page_positions[1].

status_metadata: Optional[Dict[str, object]]