CohereTextEmbedder

This component transforms a string into a vector that captures its semantics using a Cohere embedding model. When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant Documents.

Suggest Edits


Name	CohereTextEmbedder
Type	Text Embedder
Most common Position in a Pipeline	Before an embedding Retriever in a Query/RAG pipeline
Inputs	“text”: a string
Outputs	“embedding”: a List of float numbers (vectors) “meta”: a Dictionary of metadata strings

Overview

CohereTextEmbedder embeds a simple string (such as a query) into a vector. For embedding lists of Documents, use the use the CohereDocumentEmbedder, which enriches the Document with the computed embedding, also known as vector.

The component supports the following Cohere models:
"embed-english-v3.0", "embed-english-light-v3.0", "embed-multilingual-v3.0",
"embed-multilingual-light-v3.0", "embed-english-v2.0", "embed-english-light-v2.0",
"embed-multilingual-v2.0". The default model is embed-english-v2.0. This list of all supported models can be found in Cohere’s model documentation.

To start using this integration with Haystack, install it with:

pip install cohere-haystack

The component uses a COHERE_API_KEY or CO_API_KEY environment variable by default. Otherwise, you can pass an API key at initialization with api_key:

embedder = CohereTextEmbedder(api_key=Secret.from_token("<your-api-key>"))

To get a Cohere API key, head over to https://cohere.com/.

Usage

On its own

Here is how you can use the component on its own. You’ll need to pass in your Cohere API key as shown below, or set it as an environment variable called COHERE_API_KEY.

from cohere_haystack.embedders.text_embedder import CohereTextEmbedder

text_to_embed = "I love pizza!"

text_embedder = CohereTextEmbedder("YOUR COHERE API KEY")

print(text_embedder.run(text_to_embed))
# {'embedding': [-0.453125, 1.2236328, 2.0058594, 0.67871094...], 
#  'meta': {'api_version': {'version': '1'}, 'billed_units': {'input_tokens': 4}}}

In a Pipeline

from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from cohere_haystack.embedders.text_embedder import CohereTextEmbedder
from cohere_haystack.embedders.document_embedder import CohereDocumentEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever

document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")

documents = [Document(content="My name is Wolfgang and I live in Berlin"),
             Document(content="I saw a black horse running"),
             Document(content="Germany has many big cities")]

document_embedder = CohereDocumentEmbedder("YOUR COHERE API KEY")
documents_with_embeddings = document_embedder.run(documents)['documents']
document_store.write_documents(documents_with_embeddings)

query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", CohereTextEmbedder("YOUR COHERE API KEY"))
query_pipeline.add_component("retriever", InMemoryEmbeddingRetriever(document_store=document_store))
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")

query = "Who lives in Berlin?"

result = query_pipeline.run({"text_embedder":{"text": query}})

print(result['retriever']['documents'][0])

# Document(id=..., content: 'My name is Wolfgang and I live in Berlin')

Updated 9 months ago