Embedders
Embedders in Haystack transform texts or Documents into vector representations using pre-trained models. You can then use the embeddings in your pipeline for tasks like question answering, information retrieval, and more.
The embeddings produced by Haystack embedders are fixed-length vectors. They capture contextual information and semantic relationships within the text. The primary goal of embeddings is to transform text into a format the language model can understand and process in a more nuanced and contextually aware way.
Text and Document Embedders
There are two types of Embedders: text and document.
Text Embedders work with text strings and are most often used at the beginning of query pipelines to convert query text into vectors and send it to a Retriever.
Document Embedders embed Document objects and are most often used in indexing pipelines, after Converters, and before a DocumentWriter.
You must use the same embedding model for text and documents. This means that if you use CohereDocumentEmbedder in your indexing pipeline, you must then use CohereTextEmbedder with the same model in your query pipeline.
These are the Embedders available in Haystack:
Embedder | Description |
---|---|
AmazonBedrockTextEmbedder | Computes embeddings for text (such as a query) using models through Amazon Bedrock API. |
AmazonBedrockDocumentEmbedder | Computes embeddings for documents using models through Amazon Bedrock API. |
AzureOpenAITextEmbedder | Computes embeddings for text (such as a query) using OpenAI models deployed through Azure. |
AzureOpenAIDocumentEmbedder | Computes embeddings for documents using OpenAI models deployed through Azure. |
CohereTextEmbedder | Embeds a simple string (such as a query) with a Cohere model. Requires an API key from Cohere |
CohereDocumentEmbedder | Embeds a list of documents with a Cohere model. Requires an API key from Cohere. |
FastembedTextEmbedder | Computes the embeddings of a string using embedding models supported by Fastembed. |
FastembedDocumentEmbedder | Computes the embeddings of a list of documents using the models supported by Fastembed. |
FastembedSparseTextEmbedder | Embeds a simple string (such as a query) into a sparse vector using the models supported by Fastembed. |
FastembedSparseDocumentEmbedder | Enriches a list of documents with their sparse embeddings using the models supported by Fastembed. |
HuggingFaceAPIDocumentEmbedder | Computes document embeddings using various Hugging Face APIs. |
HuggingFaceAPITextEmbedder | Embeds strings using various Hugging Face APIs. |
HuggingFaceTEIDocumentEmbedder | Computes embeddings for documents using the TEI library. |
HuggingFaceTEITextEmbedder | Cmputes embeddings for text using the TEI library. |
JinaTextEmbedder | Embeds a simple string (such as a query) with a Jina AI Embeddings model. Requires an API key from Jina AI. |
JinaDocumentEmbedder | Embeds a list of documents with a Jina AI Embeddings model. Requires an API key from Jina AI. |
MistralTextEmbedder | Transforms a string into a vector using the Mistral API and models. |
MistralDocumentEmbedder | Computes the embeddings of a list of documents using the Mistral API and models. |
NvidiaTextEmbedder | Embeds a simple string (such as a query) into a vector. |
NvidiaDocumentEmbedder | Enriches the metadata of documents with an embedding of their content. |
OllamaTextEmbedder | Computes the embeddings of a string using embedding models compatible with the Ollama Library. |
OllamaDocumentEmbedder | Computes the embeddings of a list of documents using embedding models compatible with the Ollama Library. |
OpenAIDocumentEmbedder | Embeds a list of documents with an OpenAI embedding model. Requires an API key from an active OpenAI account. |
OpenAITextEmbedder | Embeds a simple string (such as a query) with an OpenAI embedding model. Requires an API key from an active OpenAI account. |
OptimumTextEmbedder | Embeds text using models loaded with the Hugging Face Optimum library. |
OptimumDocumentEmbedder | Computes documents’ embeddings using models loaded with the Hugging Face Optimum library. |
SentenceTransformersTextEmbedder | Embeds a simple string (such as a query) using a Sentence Transformer model. |
SentenceTransformersDocumentEmbedder | Embeds a list of documents with a Sentence Transformer model. |
Updated 4 months ago