Jina Embeddings

If you’re opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

%pip install llama-index-embeddings-jinaai
%pip install llama-index-llms-openai

!pip install llama-index

You may also need other packages that do not come direcly with llama-index

!pip install Pillow

For this example, you will need an API key which you can get from https://jina.ai/embeddings/

# Initilise with your api key
import os

jinaai_api_key = "YOUR_JINAAI_API_KEY"
os.environ["JINAAI_API_KEY"] = jinaai_api_key

Embed text and queries with Jina embedding models through JinaAI API

You can encode your text and your queries using the JinaEmbedding class. Jina offers a range of models adaptable to various use cases.

Model	Dimension	Language	MRL (matryoshka)	Context
jina-embeddings-v3	1024	Multilingual (89 languages)	Yes	8192
jina-embeddings-v2-base-en	768	English	No	8192
jina-embeddings-v2-base-de	768	German & English	No	8192
jina-embeddings-v2-base-es	768	Spanish & English	No	8192
jina-embeddings-v2-base-zh	768	Chinese & English	No	8192

Recommended Model: jina-embeddings-v3 :

We recommend jina-embeddings-v3 as the latest and most performant embedding model from Jina AI. This model features 5 task-specific adapters trained on top of its backbone, optimizing various embedding use cases.

By default JinaEmbedding class uses jina-embeddings-v3. On top of the backbone, jina-embeddings-v3 has been trained with 5 task-specific adapters for different embedding uses.

Task-Specific Adapters:

Include task in your request to optimize your downstream application:

• retrieval.query: Used to encode user queries or questions in retrieval tasks.
• retrieval.passage: Used to encode large documents in retrieval tasks at indexing time.
• classification: Used to encode text for text classification tasks.
• text-matching: Used to encode text for similarity matching, such as measuring similarity between two sentences.
• separation: Used for clustering or reranking tasks.

Matryoshka Representation Learning:

jina-embeddings-v3 supports Matryoshka Representation Learning, allowing users to control the embedding dimension with minimal performance loss.
Include dimensions in your request to select the desired dimension.
By default, dimensions is set to 1024, and a number between 256 and 1024 is recommended.
You can reference the table below for hints on dimension vs. performance:

Dimension	32	64	128	256	512	768	1024
Average Retrieval Performance (nDCG@10)	52.54	58.54	61.64	62.72	63.16	63.3	63.35

Late Chunking in Long-Context Embedding Models

jina-embeddings-v3 supports Late Chunking, the technique to leverage the model’s long-context capabilities for generating contextual chunk embeddings. Include late_chunking=True in your request to enable contextual chunked representation. When set to true, Jina AI API will concatenate all sentences in the input field and feed them as a single string to the model. Internally, the model embeds this long concatenated string and then performs late chunking, returning a list of embeddings that matches the size of the input list.

from llama_index.embeddings.jinaai import JinaEmbedding

text_embed_model = JinaEmbedding(
    api_key=jinaai_api_key,
    model="jina-embeddings-v3",
    # choose `retrieval.passage` to get passage embeddings
    task="retrieval.passage",
)

embeddings = text_embed_model.get_text_embedding("This is the text to embed")
print("Text dim:", len(embeddings))
print("Text embed:", embeddings[:5])

query_embed_model = JinaEmbedding(
    api_key=jinaai_api_key,
    model="jina-embeddings-v3",
    # choose `retrieval.query` to get query embeddings, or choose your desired task type
    task="retrieval.query",
    # `dimensions` allows users to control the embedding dimension with minimal performance loss. by default it is 1024.
    # A number between 256 and 1024 is recommended.
    dimensions=512,
)

embeddings = query_embed_model.get_query_embedding(
    "This is the query to embed"
)
print("Query dim:", len(embeddings))
print("Query embed:", embeddings[:5])

Embed images and queries with Jina CLIP through JinaAI API

You can also encode your images and your queries using the JinaEmbedding class

from llama_index.embeddings.jinaai import JinaEmbedding
from PIL import Image
import requests
from numpy import dot
from numpy.linalg import norm

embed_model = JinaEmbedding(
    api_key=jinaai_api_key,
    model="jina-clip-v1",
)

image_url = "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcStMP8S3VbNCqOQd7QQQcbvC_FLa1HlftCiJw&s"
im = Image.open(requests.get(image_url, stream=True).raw)
print("Image:")
display(im)

image_embeddings = embed_model.get_image_embedding(image_url)
print("Image dim:", len(image_embeddings))
print("Image embed:", image_embeddings[:5])

text_embeddings = embed_model.get_text_embedding(
    "Logo of a pink blue llama on dark background"
)
print("Text dim:", len(text_embeddings))
print("Text embed:", text_embeddings[:5])

cos_sim = dot(image_embeddings, text_embeddings) / (
    norm(image_embeddings) * norm(text_embeddings)
)
print("Cosine similarity:", cos_sim)

Embed in batches

You can also embed text in batches, the batch size can be controlled by setting the embed_batch_size parameter (the default value will be 10 if not passed, and it should not be larger than 2048)

embed_model = JinaEmbedding(
    api_key=jinaai_api_key,
    model="jina-embeddings-v3",
    embed_batch_size=16,
    task="retrieval.passage",
)

embeddings = embed_model.get_text_embedding_batch(
    ["This is the text to embed", "More text can be provided in a batch"]
)

print(len(embeddings))
print(embeddings[0][:5])

Let’s build a RAG pipeline using Jina AI Embeddings

Download Data

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

Imports

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

from llama_index.llms.openai import OpenAI
from llama_index.core.response.notebook_utils import display_source_node

from IPython.display import Markdown, display

Load Data

documents = SimpleDirectoryReader("./data/paul_graham/").load_data()

Build index

your_openai_key = "YOUR_OPENAI_KEY"
llm = OpenAI(api_key=your_openai_key)
embed_model = JinaEmbedding(
    api_key=jinaai_api_key,
    model="jina-embeddings-v3",
    embed_batch_size=16,
    task="retrieval.passage",
)

index = VectorStoreIndex.from_documents(
    documents=documents, embed_model=embed_model
)

Build retriever

search_query_retriever = index.as_retriever()

search_query_retrieved_nodes = search_query_retriever.retrieve(
    "What happened after the thesis?"
)

for n in search_query_retrieved_nodes:
    display_source_node(n, source_length=2000)