OtariTextEmbedder

This component computes the embedding of a string using embedding models from many providers through an Otari LLM gateway.

Most common position in a pipeline	Before an embedding Retriever in a query/RAG pipeline
Mandatory init variables	api_key: An Otari API key. Can be set with OTARI_API_KEY env var.
Mandatory run variables	text: A string
Output variables	embedding: A list of float numbers meta: A dictionary of metadata
API reference	Otari
GitHub link	https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/otari
Package name	otari-haystack

This component should be used to embed a string, such as a query. To embed a list of documents, use the OtariDocumentEmbedder.

Overview

Otari is an open-source, OpenAI-compatible LLM gateway by mozilla.ai. It routes requests to over 40 model providers and adds API keys, budgets, and usage and cost tracking.

OtariTextEmbedder computes the embedding of a string through the gateway's embeddings endpoint. Name the model with a selector that the gateway accepts, such as openai:text-embedding-3-small (default). Set dimensions to shorten the embeddings of models that support it.

By default, OtariTextEmbedder calls otari.ai at https://api.otari.ai/api/v1. For an account in otari.ai's EU region, whose API keys start with otk_v1_eu_, set api_base_url to https://eu.api.otari.ai/api/v1. For a gateway you run yourself, set api_base_url to its address followed by /api/v1, for example http://localhost:8000/api/v1.

To start using this integration with Haystack, install it with:

shell

pip install otari-haystack

OtariTextEmbedder needs an Otari API key to work. It uses an OTARI_API_KEY environment variable by default. Otherwise, you can pass an API key at initialization with api_key:

python

from haystack.utils import Secret
from haystack_integrations.components.embedders.otari import OtariTextEmbedder

embedder = OtariTextEmbedder(api_key=Secret.from_token("<your-api-key>"))

When the gateway prices a request, the usage entry in the output meta contains the cost in US dollars as cost_usd, a decimal string, and the price list it came from as pricing_source.

Provider credentials

The gateway needs a credential for the provider of the embedding model. On otari.ai, add a provider key on your organization's Providers page. On a gateway you run yourself, add the provider on the Providers page of its dashboard, or declare it in the gateway's config.yml:

yaml

providers:
  openai:
    api_key: ${OPENAI_API_KEY}

The gateway reads a provider's environment variable, such as OPENAI_API_KEY, only if its config names the variable like this. For details, see Configuring a provider in the Otari docs.

Usage

On its own

python

from haystack_integrations.components.embedders.otari import OtariTextEmbedder

text_embedder = OtariTextEmbedder()

print(text_embedder.run("I love pizza!"))
# >> {'embedding': [0.017020374536514282, -0.023255806416273117, ...],
# >>  'meta': {'model': 'text-embedding-3-small', 'usage': {...}}}

In a pipeline

python

from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.otari import (
    OtariDocumentEmbedder,
    OtariTextEmbedder,
)

document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")

documents = [
    Document(content="My name is Wolfgang and I live in Berlin"),
    Document(content="I saw a black horse running"),
    Document(content="Germany has many big cities"),
]

document_embedder = OtariDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)

query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", OtariTextEmbedder())
query_pipeline.add_component(
    "retriever", InMemoryEmbeddingRetriever(document_store=document_store)
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")

result = query_pipeline.run({"text_embedder": {"text": "Who lives in Berlin?"}})

print(result["retriever"]["documents"][0])
# >> Document(id=..., content: 'My name is Wolfgang and I live in Berlin', score: ...)