Data Connectors

Concept

A data connector (aka Reader) ingest data from different data sources and data formats into a simple Document representation (text and simple metadata).

Tip

Once you’ve ingested your data, you can build an Index on top, ask questions using a Query Engine, and have a conversation using a Chat Engine.

Integration packages

Each data connector is published as its own package, so you only install the ones you need. The full set lives in the readers directory of the LlamaIndex repo, and every package is named llama-index-readers-<source> on PyPI.

Usage Pattern

Get started with:

from llama_index.readers.google import GoogleDocsReader

loader = GoogleDocsReader()
documents = loader.load_data(document_ids=[...])

See the full usage pattern guide for more details.

Modules

Some sample data connectors:

• local file directory (SimpleDirectoryReader). Can support parsing a wide range of file types: .pdf, .jpg, .png, .docx, etc.
• Notion (NotionPageReader)
• Google Docs (GoogleDocsReader)
• Slack (SlackReader)
• Discord (DiscordReader)
• Apify Actors (ApifyActor). Can crawl the web, scrape webpages, extract text content, download files including .pdf, .jpg, .png, .docx, etc.

See the modules guide for more details.