Expand to see the pipeline graph
## Importing a Pipeline to Haystack Enterprise Platform You can import your Haystack pipeline into Haystack Enterprise Platform and continue visually building your pipeline To do that, follow the steps described in Haystack Enterprise Platform [documentation](https://docs.cloud.deepset.ai/docs/import-a-pipeline#import-your-pipeline). --- // File: concepts/pipelines import ClickableImage from "@site/src/components/ClickableImage"; import YoutubeEmbed from "@site/src/components/YoutubeEmbed"; # Pipelines To build modern search pipelines with LLMs, you need two things: powerful components and an easy way to put them together. The Haystack pipeline is built for this purpose and enables you to design and scale your interactions with LLMs. The pipelines in Haystack are directed multigraphs of different Haystack components and integrations. They give you the freedom to connect these components in various ways. This means that the pipeline doesn't need to be a continuous stream of information. With the flexibility of Haystack pipelines, you can have simultaneous flows, standalone components, loops, and other types of connections. ## Flexibility Haystack pipelines are much more than just query and indexing pipelines. What a pipeline does, whether that be indexing, querying, fetching from an API, preprocessing or more, completely depends on how you design your pipeline and what components you use. While you can still create single-function pipelines, like indexing pipelines using ready-made components to clean up, split, and write the documents into a Document Store, or query pipelines that just take a query and return an answer, Haystack allows you to combine multiple use cases into one pipeline with decision components (like the `ConditionalRouter`) as well. ### Agentic Pipelines Haystack loops and branches enable the creation of complex applications like agents. Here are a few examples on how to create them: - [Tutorial: Building a Chat Agent with Function Calling](https://haystack.deepset.ai/tutorials/40_building_chat_application_with_function_calling) - [Tutorial: Building an Agentic RAG with Fallback to Websearch](https://haystack.deepset.ai/tutorials/36_building_fallbacks_with_conditional_routing) - [Tutorial: Generating Structured Output with Loop-Based Auto-Correction](https://haystack.deepset.ai/tutorials/28_structured_output_with_loop) - [Cookbook: Define & Run Tools](https://haystack.deepset.ai/cookbook/tools_support) - [Cookbook: Conversational RAG using Memory](https://haystack.deepset.ai/cookbook/conversational_rag_using_memory) - [Cookbook: Newsletter Sending Agent with Experimental Haystack Tools](https://haystack.deepset.ai/cookbook/newsletter-agent) ### Branching A pipeline can have multiple branches that process data concurrently. For example, to process different file types, you can have a pipeline with a bunch of converters, each handling a specific file type. You then feed all your files to the pipeline and it smartly divides and routes them to appropriate converters all at once, saving you the effort of sending your files one by one for processing.
Example Use Case - Problem Statement
### Problem Statement Let’s consider an example RAG pipeline that embeds a query, uses a Retriever component to locate documents relevant to the query, and then leverages an LLM to generate an answer based on the retrieved documents. The `OpenAIChatGenerator` component used in the pipeline below expects an API key to authenticate with OpenAI’s servers and perform the generation. Let’s assume that the component accepts a `str` value for it: ```python generator = OpenAIChatGenerator(api_key="sk-xxxxxxxxxxxxxxxxxx") pipeline.add_component("generator", generator) ``` This works in a pinch, but this is bad practice - we shouldn’t hard-code such secrets in the codebase. An alternative would be to store the key in an environment variable externally, read from it in Python, and pass that to the component: ```python import os api_key = os.environ.get("OPENAI_API_KEY") generator = OpenAIChatGenerator(api_key=api_key) pipeline.add_component("generator", generator) ``` This is better – the pipeline works as intended, and we aren’t hard-coding any secrets in the code. Remember that pipelines are serializable. Since the API key is a secret, we should definitely avoid saving it to disk. Let’s modify the component’s `to_dict` method to exclude the key: ```python def to_dict(self) -> Dict[str, Any]: # Do not pass the `api_key` init parameter. return default_to_dict(self, model=self.model) ``` But what happens when the pipeline is loaded from disk? In the best-case scenario, the component’s backend will automatically try to read the key from a hard-coded environment variable, and that key is the same as the one that was passed to the component before it was serialized. But in a worse case, the backend doesn’t look up the key in a hard-coded environment variable and fails when it gets called inside a `pipeline.run()` invocation.
| | |
| --- | --- |
| **Base classes** | `Tracer` and `Span` |
| **How to enable** | Implement the `Tracer` interface, then `tracing.enable_tracing(your_tracer)` |
| **Content tracing** | Optional. Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` to trace component inputs and outputs |
| **Package** | Built into Haystack |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/tracer.py |
## Overview
If your tracing backend isn't supported out of the box, you can connect it to Haystack by implementing the `Tracer` interface. This gives you full control over how spans are created and how tags are recorded.
## Usage
1. Implement the `Tracer` interface. The following code snippet provides an example using the OpenTelemetry package:
```python
import contextlib
from collections.abc import Iterator
from typing import Any
from opentelemetry import trace
from opentelemetry.trace import NonRecordingSpan
from haystack.tracing import Tracer, Span
from haystack.tracing import utils as tracing_utils
import opentelemetry.trace
class OpenTelemetrySpan(Span):
def __init__(self, span: opentelemetry.trace.Span) -> None:
self._span = span
def set_tag(self, key: str, value: Any) -> None:
# Tracing backends usually don't support any tag value
# `coerce_tag_value` forces the value to either be a Python
# primitive (int, float, boolean, str) or tries to dump it as string.
coerced_value = tracing_utils.coerce_tag_value(value)
self._span.set_attribute(key, coerced_value)
class OpenTelemetryTracer(Tracer):
def __init__(self, tracer: opentelemetry.trace.Tracer) -> None:
self._tracer = tracer
@contextlib.contextmanager
def trace(
self,
operation_name: str,
tags: dict[str, Any] | None = None,
parent_span: Span | None = None,
) -> Iterator[Span]:
with self._tracer.start_as_current_span(operation_name) as span:
span = OpenTelemetrySpan(span)
if tags:
span.set_tags(tags)
yield span
def current_span(self) -> Span | None:
current_span = trace.get_current_span()
if isinstance(current_span, NonRecordingSpan):
return None
return OpenTelemetrySpan(current_span)
```
2. Tell Haystack to use your custom tracer:
```python
from haystack import tracing
haystack_tracer = OpenTelemetryTracer(tracer)
tracing.enable_tracing(haystack_tracer)
```
---
// File: development/tracing/datadog
# Datadog
Learn how to trace your Haystack pipelines with Datadog.
| | |
| --- | --- |
| **Tracer class** | `DatadogTracer` |
| **How to enable** | Enable the tracer with `tracing.enable_tracing(DatadogTracer(ddtrace.tracer))`, or add the `DatadogConnector` component to your pipeline |
| **Content tracing** | Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` to trace component inputs and outputs |
| **Package** | `datadog-haystack` |
| **API reference** | [datadog](/reference/integrations-datadog) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/datadog |
## Overview
Trace your Haystack pipelines with [Datadog](https://www.datadoghq.com/) through [Datadog's tracing library `ddtrace`](https://ddtrace.readthedocs.io/en/stable/). Haystack captures detailed information about pipeline runs, like API calls, context data, and prompts, so you can see the complete trace of your pipeline execution in Datadog.
## Installation
Install the `datadog-haystack` package:
```shell
pip install datadog-haystack
```
## Prerequisites
1. A way to receive traces, such as a running [Datadog Agent](https://docs.datadoghq.com/agent/). `ddtrace` sends traces to the Datadog Agent at `localhost:8126` by default.
2. Configure `ddtrace` through the standard mechanisms, for example the `DD_SERVICE`, `DD_ENV`, and `DD_VERSION` environment variables, or by running your application with the `ddtrace-run` command. See the [ddtrace documentation](https://ddtrace.readthedocs.io/en/stable/) for more details.
## Usage
Enable the `DatadogTracer` directly to trace any Haystack pipeline, without adding a component to it. Make sure to set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable before importing any Haystack components.
```python
import os
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
import ddtrace
from haystack import Pipeline, tracing
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.tracing.datadog import DatadogTracer
# Enable the Datadog tracer
tracing.enable_tracing(DatadogTracer(ddtrace.tracer))
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component("llm", OpenAIChatGenerator())
pipe.connect("prompt_builder.prompt", "llm.messages")
messages = [
ChatMessage.from_system(
"Always respond in German even if some input data is in other languages.",
),
ChatMessage.from_user("Tell me about {{location}}"),
]
response = pipe.run(
data={
"prompt_builder": {
"template_variables": {"location": "Berlin"},
"template": messages,
},
},
)
print(response["llm"]["replies"][0])
```
Each pipeline run produces a trace that includes the entire execution context, including prompts, completions, and metadata. You can then view the traces in your Datadog dashboard.
## Alternative: the DatadogConnector component
If you prefer to manage tracing as part of your pipeline definition (for example, so it serializes to YAML), you can add the `DatadogConnector` component instead. It enables the same Datadog tracing as soon as it is initialized.
:::info
See the [`DatadogConnector` documentation page](../../pipeline-components/connectors/datadogconnector.mdx) for full usage examples, or check out the [integration page](https://haystack.deepset.ai/integrations/datadog).
:::
---
// File: development/tracing/langfuse
import ClickableImage from "@site/src/components/ClickableImage";
# Langfuse
Learn how to trace your Haystack pipelines with Langfuse.
| | |
| --- | --- |
| **Tracer class** | `LangfuseTracer` |
| **How to enable** | Enable the tracer with `tracing.enable_tracing(LangfuseTracer(langfuse))`, or add the `LangfuseConnector` component to your pipeline |
| **Content tracing** | Required. Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` |
| **Package** | `langfuse-haystack` |
| **API reference** | [langfuse](/reference/integrations-langfuse) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/langfuse |
## Overview
Trace your Haystack pipelines with the [Langfuse](https://langfuse.com/) UI. Langfuse captures detailed information about pipeline runs, like API calls, context data, prompts, and more. Use it to monitor model performance such as token usage and cost, find areas for improvement, and create datasets from your pipeline executions.
## Installation
Install the `langfuse-haystack` package:
```shell
pip install langfuse-haystack
```
## Prerequisites
1. An active Langfuse [account](https://cloud.langfuse.com/).
2. Set the `LANGFUSE_SECRET_KEY` and `LANGFUSE_PUBLIC_KEY` environment variables with your Langfuse secret and public keys, found in your account profile.
3. Set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true` to enable tracing.
:::info[Usage Notice]
To ensure proper tracing, always set environment variables before importing any Haystack components. This is crucial because Haystack initializes its internal tracing components during import. An even better practice is to set these environment variables in your shell before running the script.
:::
## Usage
Enable the `LangfuseTracer` directly to trace any Haystack pipeline, without adding a component to it.
```python
import os
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com"
os.environ["LANGFUSE_SECRET_KEY"] = "
| | |
| --- | --- |
| **Tracer class** | `LoggingTracer` |
| **How to enable** | `tracing.enable_tracing(LoggingTracer(...))` |
| **Content tracing** | Required to log inputs and outputs. Set `tracing.tracer.is_content_tracing_enabled = True` |
| **Package** | Built into Haystack |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/logging_tracer.py |
## Overview
Use Haystack's [`LoggingTracer`](https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/logging_tracer.py) logs to inspect the data that's flowing through your pipeline in real time.
This feature is particularly helpful during experimentation and prototyping, as you don’t need to set up any tracing backend beforehand.
## Usage
Here’s how you can enable this tracer. In this example, we are adding color tags (this is optional) to highlight the components' names and inputs:
```python
import logging
from haystack import tracing
from haystack.tracing.logging_tracer import LoggingTracer
logging.basicConfig(
format="%(levelname)s - %(name)s - %(message)s",
level=logging.WARNING,
)
logging.getLogger("haystack").setLevel(logging.DEBUG)
tracing.tracer.is_content_tracing_enabled = (
True # to enable tracing/logging content (inputs/outputs)
)
tracing.enable_tracing(
LoggingTracer(
tags_color_strings={
"haystack.component.input": "\x1b[1;31m",
"haystack.component.name": "\x1b[1;34m",
},
),
)
```
Here’s what the resulting log would look like when a pipeline is run:
| | |
| --- | --- |
| **How to enable** | `mlflow.haystack.autolog()` |
| **Content tracing** | Captured automatically, including latencies, token usage, cost, and exceptions |
| **Package** | `mlflow` |
| **Integration guide** | https://haystack.deepset.ai/integrations/mlflow |
## Overview
[MLflow](https://mlflow.org/) is an open-source platform for managing the end-to-end machine learning and AI lifecycle. MLflow provides native tracing support for Haystack, so you can capture traces from all your pipelines and components with a single line of code.
## Installation
Install MLflow:
```shell
pip install mlflow
```
## Usage
Enable automatic tracing for all Haystack pipelines and components:
```python
import mlflow
mlflow.haystack.autolog()
# Optionally set an experiment name
mlflow.set_experiment("Haystack")
```
This automatically captures traces from all Haystack pipelines and components, including latencies, token usage, cost, and any exceptions.
:::info
Check out the [MLflow Haystack integration guide](https://haystack.deepset.ai/integrations/mlflow) for a full walkthrough with examples.
:::
---
// File: development/tracing/opentelemetry
import ClickableImage from "@site/src/components/ClickableImage";
# OpenTelemetry
Learn how to trace your Haystack pipelines with OpenTelemetry.
| | |
| --- | --- |
| **Tracer class** | `OpenTelemetryTracer` |
| **How to enable** | Configure an OpenTelemetry `TracerProvider`, then enable the tracer with `tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application")))`, or add the `OpenTelemetryConnector` component to your pipeline |
| **Content tracing** | Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` to trace component inputs and outputs |
| **Package** | `opentelemetry-haystack` |
| **API reference** | [opentelemetry](/reference/integrations-opentelemetry) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opentelemetry |
## Overview
[OpenTelemetry](https://opentelemetry.io/) is an open-source observability framework for collecting traces, metrics, and logs. Haystack integrates with OpenTelemetry, so you can send traces of your pipeline runs to any OpenTelemetry-compatible backend.
:::info[Provided by an integration]
`OpenTelemetryTracer` lives in the `opentelemetry-haystack` package and is not part of Haystack core. Since Haystack 3.0, OpenTelemetry tracing is no longer auto-enabled when `opentelemetry-sdk` is installed. Install the integration and either enable the `OpenTelemetryTracer` directly or add the `OpenTelemetryConnector` component to your pipeline.
:::
## Installation
Install the `opentelemetry-haystack` package:
```shell
pip install opentelemetry-haystack
```
To add traces to even deeper levels of your pipelines, we recommend you check out [OpenTelemetry integrations](https://opentelemetry.io/ecosystem/registry/?s=python), such as:
- [`urllib3` instrumentation](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-urllib3) for tracing HTTP requests in your pipeline,
- [OpenAI instrumentation](https://github.com/traceloop/openllmetry/tree/main/packages/opentelemetry-instrumentation-openai) for tracing OpenAI requests.
## Prerequisites
A configured OpenTelemetry `TracerProvider` with an exporter, for example an OTLP exporter that sends traces to a collector or a backend. Set up the provider before enabling the tracer.
## Usage
Enable the `OpenTelemetryTracer` directly to trace any Haystack pipeline, without adding a component to it. Configure your `TracerProvider` and set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable before importing any Haystack components.
```python
import os
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.semconv.resource import ResourceAttributes
# Configure the OpenTelemetry SDK. A service name is required for most backends.
resource = Resource(attributes={ResourceAttributes.SERVICE_NAME: "haystack"})
tracer_provider = TracerProvider(resource=resource)
tracer_provider.add_span_processor(
BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")),
)
trace.set_tracer_provider(tracer_provider)
from haystack import tracing
from haystack_integrations.tracing.opentelemetry import OpenTelemetryTracer
# Enable the OpenTelemetry tracer
tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application")))
```
Each pipeline run then produces a trace that includes the entire execution context, including prompts, completions, and metadata. You can view the traces in your OpenTelemetry-compatible backend.
## Alternative: the OpenTelemetryConnector component
If you prefer to manage tracing as part of your pipeline definition, you can add the `OpenTelemetryConnector` component instead. It enables the same OpenTelemetry tracing as soon as it is initialized.
:::info
See the [`OpenTelemetryConnector` documentation page](../../pipeline-components/connectors/opentelemetryconnector.mdx) for full usage examples, or check out the [integration page](https://haystack.deepset.ai/integrations/opentelemetry).
:::
## Visualizing Traces During Development
Use [Jaeger](https://www.jaegertracing.io/docs/1.6/getting-started/) as a lightweight tracing backend for local pipeline development. This allows you to experiment with tracing without the need for a complex tracing backend.
| | |
| --- | --- |
| **Tracer class** | `WeaveTracer` |
| **How to enable** | Enable the tracer with `tracing.enable_tracing(WeaveTracer(project_name="..."))`, or add the `WeaveConnector` component to your pipeline |
| **Content tracing** | Required. Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` |
| **Package** | `weave-haystack` |
| **API reference** | [Weave](/reference/integrations-weave) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weave |
## Overview
Trace and visualize your pipeline execution in [Weights & Biases](https://wandb.ai/site/). Information captured by the Haystack tracing tool, such as API calls, context data, and prompts, is sent to Weights & Biases, where you can see the complete trace of your pipeline execution.
## Installation
Install the `weave-haystack` package:
```shell
pip install weave-haystack
```
## Prerequisites
1. A Weave account. You can sign up for free on the [Weights & Biases website](https://wandb.ai/site).
2. Set the `WANDB_API_KEY` environment variable with your Weights & Biases API key. Once logged in, you can find your API key on [your home page](https://wandb.ai/home).
3. Set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true`.
## Usage
Enable the `WeaveTracer` directly to trace any Haystack pipeline, without adding a component to it. The `project_name` is the name that will appear in your Weave project.
```python
import os
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
from haystack import Pipeline, tracing
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.tracing.weave import WeaveTracer
# Enable the Weave tracer
tracing.enable_tracing(WeaveTracer(project_name="test_pipeline"))
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component("llm", OpenAIChatGenerator())
pipe.connect("prompt_builder.prompt", "llm.messages")
messages = [
ChatMessage.from_system(
"Always respond in German even if some input data is in other languages.",
),
ChatMessage.from_user("Tell me about {{location}}"),
]
response = pipe.run(
data={
"prompt_builder": {
"template_variables": {"location": "Berlin"},
"template": messages,
},
},
)
print(response["llm"]["replies"][0])
```
You can then see the complete trace for your pipeline at `https://wandb.ai/
| | |
| --- | --- |
| API reference | [AlloyDB](/reference/integrations-alloydb) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/alloydb |
[AlloyDB](https://cloud.google.com/alloydb) is a fully managed, PostgreSQL-compatible database service on Google Cloud. The `AlloyDBDocumentStore` uses the [pgvector extension](https://cloud.google.com/alloydb/docs/ai/work-with-embeddings) to perform vector similarity search.
Connection is handled securely via the [AlloyDB Python Connector](https://github.com/GoogleCloudPlatform/alloydb-python-connector), which provides TLS encryption and IAM-based authorization without requiring manual SSL certificate management, firewall rules, or IP allowlisting.
The `AlloyDBDocumentStore` supports embedding retrieval, keyword retrieval, and metadata filtering.
## Installation
Install the `alloydb-haystack` integration:
```shell
pip install alloydb-haystack
```
To set up an AlloyDB cluster and instance, follow the [AlloyDB quickstart](https://cloud.google.com/alloydb/docs/quickstart).
## Usage
### Authentication
The `AlloyDBDocumentStore` uses [Secrets](../concepts/secret-management.mdx) and reads connection details from environment variables by default:
- `ALLOYDB_INSTANCE_URI`: the AlloyDB instance URI in the format `projects/PROJECT/locations/REGION/clusters/CLUSTER/instances/INSTANCE`.
- `ALLOYDB_USER`: the database user. When using IAM database authentication, use the service account email (omitting `.gserviceaccount.com`) or the full IAM user email.
- `ALLOYDB_PASSWORD`: the database password. Not required when `enable_iam_auth=True`.
```shell
export ALLOYDB_INSTANCE_URI="projects/MY_PROJECT/locations/MY_REGION/clusters/MY_CLUSTER/instances/MY_INSTANCE"
export ALLOYDB_USER="my-db-user"
export ALLOYDB_PASSWORD="my-db-password"
```
To authenticate with IAM instead of a password, set `enable_iam_auth=True` and grant the IAM principal the AlloyDB Client role. See the [AlloyDB IAM authentication documentation](https://cloud.google.com/alloydb/docs/manage-iam-authn) for details.
## Initialization
Initialize an `AlloyDBDocumentStore` and write Documents to it. Connection to AlloyDB is established lazily on first use, and the table that stores Haystack Documents is created automatically if it doesn't exist:
```python
from haystack import Document
from haystack_integrations.document_stores.alloydb import AlloyDBDocumentStore
document_store = AlloyDBDocumentStore(
db="my-database",
embedding_dimension=768,
vector_function="cosine_similarity",
recreate_table=True,
)
document_store.write_documents(
[
Document(content="This is first", embedding=[0.1] * 768),
Document(content="This is second", embedding=[0.3] * 768),
],
)
print(document_store.count_documents())
```
To learn more about the initialization parameters, see our [API docs](/reference/integrations-alloydb#alloydbdocumentstore).
To compute embeddings for your Documents, you can use a Document Embedder, such as the [`SentenceTransformersDocumentEmbedder`](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx).
### Search Strategy
The `AlloyDBDocumentStore` supports two search strategies for embedding retrieval:
- `"exact_nearest_neighbor"` (default): provides perfect recall but can be slow on large numbers of documents.
- `"hnsw"`: an approximate nearest neighbor search strategy that trades off some accuracy for speed. Recommended for large numbers of documents.
When using `"hnsw"`, an index is created based on the `vector_function` you choose, so subsequent queries should keep using the same vector similarity function in order to take advantage of the index. You can tune index creation through `hnsw_index_creation_kwargs` (see the [pgvector documentation](https://github.com/pgvector/pgvector?tab=readme-ov-file#hnsw)).
### Metadata Filtering
The `AlloyDBDocumentStore` fully supports comparison operators (`==`, `!=`, `>`, `>=`, `<`, `<=`, `in`, `not in`, `like`, `not like`) and the logical operators `AND` and `OR`. The `like` and `not like` operators are PostgreSQL-specific extensions to the standard Haystack filter syntax and map to the SQL `LIKE` / `NOT LIKE` pattern-matching operators.
The `NOT` logical operator is **not** supported. Because every comparison operator already has a negated counterpart (`==`/`!=`, `in`/`not in`, `like`/`not like`), any filter expressible with `NOT` around a single condition can be rewritten by inverting the comparison operator instead. To negate a nested `AND`/`OR` group, apply De Morgan's laws — for example, `NOT (A AND B)` becomes `(NOT A) OR (NOT B)`, where each `NOT A` / `NOT B` is expressed via the inverted comparison.
For more details on filter syntax, refer to [Metadata Filtering](../concepts/metadata-filtering.mdx).
### Supported Retrievers
- [`AlloyDBEmbeddingRetriever`](../pipeline-components/retrievers/alloydbembeddingretriever.mdx): An embedding-based Retriever that fetches Documents from the Document Store based on a query embedding.
- [`AlloyDBKeywordRetriever`](../pipeline-components/retrievers/alloydbkeywordretriever.mdx): A keyword-based Retriever that fetches Documents matching a query using PostgreSQL full-text search.
---
// File: document-stores/arangodocumentstore
# ArangoDocumentStore
Use the ArangoDB multi-model database with Haystack for embedding retrieval and GraphRAG workloads.
| | |
| --- | --- |
| API reference | [ArangoDB](/reference/integrations-arangodb) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/arangodb |
ArangoDB is a multi-model database that combines documents, graphs, and key-value data in a single engine. The `ArangoDocumentStore` stores documents in an ArangoDB collection and runs vector similarity search using AQL (ArangoDB Query Language) vector functions. Because documents and their relationships live in the same database, ArangoDB is a good fit for GraphRAG pipelines that combine semantic search with graph traversal.
Vector search requires **ArangoDB 3.12 or later** with the vector index feature enabled (the `--vector-index` startup flag).
For more information, see the [ArangoDB documentation](https://docs.arangodb.com/).
## Installation
Run ArangoDB with Docker, enabling the vector index and setting a root password:
```shell
docker run -d -p 8529:8529 \
-e ARANGO_ROOT_PASSWORD=test-password \
arangodb:3.12 arangod --vector-index
```
Install the Haystack integration:
```shell
pip install arangodb-haystack
```
## Usage
The store reads its credentials from the `ARANGO_USERNAME` and `ARANGO_PASSWORD` environment variables by default. `ARANGO_USERNAME` falls back to `root` if it is not set, so you typically only need to provide the password:
```shell
export ARANGO_PASSWORD=test-password
```
Initialize the document store and write documents:
```python
from haystack import Document
from haystack_integrations.document_stores.arangodb import ArangoDocumentStore
document_store = ArangoDocumentStore(
host="http://localhost:8529",
database="haystack",
collection_name="documents",
embedding_dimension=768,
recreate_collection=True,
)
document_store.write_documents(
[
Document(
content="There are over 7,000 languages spoken around the world today.",
),
Document(
content="Elephants have been observed to recognize themselves in mirrors.",
),
],
)
print(document_store.count_documents())
```
To learn more about the initialization parameters, see the [API docs](/reference/integrations-arangodb#arangodocumentstore).
To compute real embeddings for your documents, use a Document Embedder such as the [`SentenceTransformersDocumentEmbedder`](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx). The embedding dimension produced by the embedder must match the `embedding_dimension` configured on the store.
### Authentication
Credentials are passed as Haystack [`Secret`](../concepts/secret-management.mdx) objects. By default they are read from environment variables, but you can also pass them explicitly:
```python
from haystack.utils import Secret
from haystack_integrations.document_stores.arangodb import ArangoDocumentStore
document_store = ArangoDocumentStore(
host="http://localhost:8529",
database="haystack",
username=Secret.from_env_var("ARANGO_USERNAME", strict=False),
password=Secret.from_env_var("ARANGO_PASSWORD"),
)
```
### Similarity Functions
`ArangoDocumentStore` supports three similarity functions for vector search, configured at initialization with the `similarity_function` parameter:
- `"cosine"` (default): cosine similarity, best for normalized embeddings.
- `"dot_product"`: dot product, useful when embedding magnitude carries meaning.
- `"l2"`: Euclidean (L2) distance.
```python
document_store = ArangoDocumentStore(
host="http://localhost:8529",
embedding_dimension=768,
similarity_function="dot_product",
)
```
### Supported Retrievers
- [`ArangoEmbeddingRetriever`](../pipeline-components/retrievers/arangoembeddingretriever.mdx): Retrieves documents from the `ArangoDocumentStore` based on vector similarity using ArangoDB's AQL vector functions.
---
// File: document-stores/arcadedbdocumentstore
# ArcadeDBDocumentStore
| | |
| --- | --- |
| API reference | [ArcadeDB](/reference/integrations-arcadedb) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/arcadedb |
ArcadeDB is a multi-model database that supports vector search via its LSM_VECTOR (HNSW) index. The `ArcadeDBDocumentStore` uses ArcadeDB's HTTP/JSON API for all operations—no special drivers required. It supports dense embedding retrieval and SQL-based metadata filtering.
For more information, see the [ArcadeDB documentation](https://docs.arcadedb.com/).
## Installation
Run ArcadeDB with Docker and update the password according to your setup:
```shell
docker run -d -p 2480:2480 \
-e JAVA_OPTS="-Darcadedb.server.rootPassword=arcadedb" \
arcadedata/arcadedb:latest
```
Install the Haystack integration:
```shell
pip install arcadedb-haystack
```
## Usage
Set credentials via environment variables (recommended) or pass them explicitly:
```shell
export ARCADEDB_USERNAME=root
export ARCADEDB_PASSWORD=arcadedb
```
Initialize the document store and write documents:
```python
from haystack import Document
from haystack_integrations.document_stores.arcadedb import ArcadeDBDocumentStore
document_store = ArcadeDBDocumentStore(
url="http://localhost:2480",
database="haystack",
embedding_dimension=768,
recreate_type=True,
)
document_store.write_documents(
[
Document(content="This is first", embedding=[0.0] * 768),
Document(content="This is second", embedding=[0.1, 0.2, 0.3] + [0.0] * 765),
]
)
print(document_store.count_documents())
```
To learn more about the initialization parameters, see the [API docs](/reference/integrations-arcadedb#arcadedbdocumentstore).
Documents without embeddings or with a different dimension are stored with a zero-padded vector so they can be written and filtered; use an [Embedder](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx) for real embeddings.
### Supported Retrievers
- [ArcadeDBEmbeddingRetriever](../pipeline-components/retrievers/arcadedbembeddingretriever.mdx): An embedding-based Retriever that fetches documents from the Document Store by vector similarity (HNSW).
---
// File: document-stores/astradocumentstore
# AstraDocumentStore
| | |
| --- | --- |
| API reference | [Astra](/reference/integrations-astra) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/astra |
DataStax Astra DB is a serverless vector database built on Apache Cassandra, and it supports vector-based search and auto-scaling. You can deploy it on AWS, GCP, or Azure and easily expand to one or more regions within those clouds for multi-region availability, low latency data access, data sovereignty, and to avoid cloud vendor lock-in. For more information, see the [DataStax documentation](https://docs.datastax.com/en/home/docs/index.html).
### Initialization
Once you have an AstraDB account and have created a database, install the `astra-haystack` integration:
```shell
pip install astra-haystack
```
From the configuration in AstraDB’s web UI, you need the database API endpoint and a generated token.
You can additionally set a collection name and a namespace. The collection name defaults to `documents`, and you can set the embedding dimensions and the similarity metric alongside it with `embedding_dimension` and `similarity`. The namespace organizes data in a database and is called a keyspace in Apache Cassandra.
Then, in Haystack, initialize an `AstraDocumentStore` object that’s connected to the AstraDB instance, and write documents to it.
We strongly encourage passing authentication data through environment variables: make sure to populate the environment variables `ASTRA_DB_API_ENDPOINT` and `ASTRA_DB_APPLICATION_TOKEN` before running the following example.
```python
from haystack import Document
from haystack_integrations.document_stores.astra import AstraDocumentStore
document_store = AstraDocumentStore()
document_store.write_documents(
[Document(content="This is first"), Document(content="This is second")],
)
print(document_store.count_documents())
```
### Supported Retrievers
[AstraEmbeddingRetriever](../pipeline-components/retrievers/astraretriever.mdx): An embedding-based Retriever that fetches documents from the Document Store based on a query embedding provided to the Retriever.
### Indexing Warnings
When you create an Astra DB Document Store, you might see one of these warnings:
> Astra DB collection `...` is detected as having indexing turned on for all fields (either created manually or by older versions of this plugin). This implies stricter limitations on the amount of text each string in a document can store. Consider indexing anew on a fresh collection to be able to store longer texts.
Or:
> Astra DB collection `...` is detected as having the following indexing policy: `{...}`. This does not match the requested indexing policy for this object: `{...}`. In particular, there may be stricter limitations on the amount of text each string in a document can store. Consider indexing anew on a fresh collection to be able to store longer texts.
#### Why You See This Warning
The collection already exists and is configured to [index all fields for search](https://docs.datastax.com/en/astra-db-serverless/api-reference/collections.html#the-indexing-option), possibly because you created it earlier or an older plugin did. When Haystack tries to create the collection, it applies an indexing policy optimized for your intended use. This policy lets you store longer texts and avoids indexing fields you won’t filter on, which also reduces write overhead.
#### Common Causes
1. You created the collection outside Haystack (for example, in the Astra UI or with AstraPy’s `Database.create_collection()`).
2. You created the collection with an older version of the plugin.
#### Impact
This is only a warning. Your application keeps running unless you try to store very long text fields. If you do, Astra DB returns an indexing error.
#### Solutions
- **Recommended:** _Drop and recreate the collection_ if you can repopulate it. Then rerun your Haystack application so it creates the collection with the optimized indexing policy.
- _Ignore the warning_ if you’re sure you won’t store very long text fields.
## Additional References
🧑🍳 Cookbook: [Using AstraDB as a data store in your Haystack pipelines](https://haystack.deepset.ai/cookbook/astradb_haystack_integration)
---
// File: document-stores/azureaisearchdocumentstore
# AzureAISearchDocumentStore
A Document Store for storing and retrieval from Azure AI Search Index.
| | |
| --- | --- |
| **API reference** | [Azure AI Search](/reference/integrations-azure_ai_search) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/azure_ai_search |
[Azure AI Search](https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search) is an enterprise-ready search and retrieval system to build RAG-based applications on Azure, with native LLM integrations.
`AzureAISearchDocumentStore` supports semantic reranking and metadata/content filtering. The Document Store is useful for various tasks such as generating knowledge base insights (catalog or document search), information discovery (data exploration), RAG, and automation.
### Initialization
This integration requires you to have an active Azure subscription with a deployed [Azure AI Search](https://azure.microsoft.com/en-us/products/ai-services/ai-search) service.
Once you have the subscription, install the `azure-ai-search-haystack` integration:
```shell
pip install azure-ai-search-haystack
```
To use the `AzureAISearchDocumentStore`, you need to provide a search service endpoint as an `AZURE_AI_SEARCH_ENDPOINT` and an API key as `AZURE_AI_SEARCH_API_KEY` for authentication. If the API key is not provided, the `DefaultAzureCredential` will attempt to authenticate you through the browser.
During initialization the Document Store will either retrieve the existing search index for the given `index_name` or create a new one if it doesn't already exist. Note that one of the limitations of `AzureAISearchDocumentStore` is that the fields of the Azure search index cannot be modified through the API after creation. Therefore, any additional fields beyond the default ones must be provided as `metadata_fields` during the Document Store's initialization. However, if needed, [Azure AI portal](https://azure.microsoft.com/) can be used to modify the fields without deleting the index.
It is recommended to pass authentication data through `AZURE_AI_SEARCH_API_KEY` and `AZURE_AI_SEARCH_ENDPOINT` before running the following example.
```python
from haystack_integrations.document_stores.azure_ai_search import (
AzureAISearchDocumentStore,
)
from haystack import Document
document_store = AzureAISearchDocumentStore(index_name="haystack-docs")
document_store.write_documents(
[
Document(content="This is the first document."),
Document(content="This is the second document."),
],
)
print(document_store.count_documents())
```
:::info[Latency Notice]
Due to Azure search index latency, the document count returned in the example might be zero if executed immediately. To ensure accurate results, be mindful of this latency when retrieving documents from the search index.
:::
You can enable semantic reranking in `AzureAISearchDocumentStore` by providing [SemanticSearch](https://learn.microsoft.com/en-us/python/api/azure-search-documents/azure.search.documents.indexes.models.semanticsearch?view=azure-python) configuration in `index_creation_kwargs` during initialization and calling it from one of the Retrievers. For more information, refer to the [Azure AI tutorial](https://learn.microsoft.com/en-us/azure/search/search-get-started-semantic) on this feature.
### Supported Retrievers
The Haystack Azure AI Search integration includes three Retriever components. Each Retriever leverages the Azure AI Search API and you can select the one that best suits your pipeline:
- [`AzureAISearchEmbeddingRetriever`](../pipeline-components/retrievers/azureaisearchembeddingretriever.mdx): This Retriever accepts the embeddings of a single query as input and returns a list of matching documents. The query must be embedded beforehand, which can be done using an [Embedder](../pipeline-components/embedders.mdx) component.
- [`AzureAISearchBM25Retriever`](../pipeline-components/retrievers/azureaisearchbm25retriever.mdx): A keyword-based Retriever that retrieves documents matching a query from the Azure AI Search index.
- [`AzureAISearchHybridRetriever`](../pipeline-components/retrievers/azureaisearchhybridretriever.mdx): This Retriever combines embedding-based retrieval and keyword search to find matching documents in the search index to get more relevant results.
---
// File: document-stores/chromadocumentstore
# ChromaDocumentStore
| | |
| --- | --- |
| API reference | [Chroma](/reference/integrations-chroma) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/chroma |
[Chroma](https://docs.trychroma.com/) is an open source vector database capable of storing collections of documents along with their metadata, creating embeddings for documents and queries, and searching the collections filtering by document metadata or content. Additionally, Chroma supports multi-modal embedding functions.
Chroma can be used in-memory, as an embedded database, or in a client-server fashion. When running in-memory, Chroma can still keep its contents on disk across different sessions. This allows users to quickly put together prototypes using the in-memory version and later move to production, where the client-server version is deployed.
## Initialization
First, install the Chroma integration, which will install Haystack and Chroma if they are not already present. The following command is all you need to start:
```shell
pip install chroma-haystack
```
To store data in Chroma, create a `ChromaDocumentStore` instance and write documents with:
```python
from haystack_integrations.document_stores.chroma import ChromaDocumentStore
from haystack import Document
document_store = ChromaDocumentStore()
document_store.write_documents(
[
Document(content="This is the first document."),
Document(content="This is the second document."),
],
)
print(document_store.count_documents())
```
In this case, since we didn’t pass any embeddings along with our documents, Chroma will create them for us using its [default embedding function](https://docs.trychroma.com/embeddings#default-all-minilm-l6-v2).
### Connection Options
1. **In-Memory Mode (Local)**: Chroma can be set up as a local Document Store for fast and lightweight usage. You can use this option during development or small-scale experiments. Set up a local in-memory instance of `ChromaDocumentStore` like this:
```python
from haystack_integrations.document_stores.chroma import ChromaDocumentStore
document_store = ChromaDocumentStore()
```
2. **Persistent Storage**: If you need to retain the documents between sessions, Chroma supports persistent storage by specifying a path to store data on disk:
```python
from haystack_integrations.document_stores.chroma import ChromaDocumentStore
document_store = ChromaDocumentStore(persist_path="your_directory_path")
```
3. **Remote Connection**: You can connect to a remote Chroma database through HTTP. This is suitable for distributed setups where multiple clients might interact with the same remote Chroma instance.
Note that this option is incompatible with in-memory or persistent storage modes.
First, start a Chroma server:
```shell
chroma run --path /db_path
```
Or using docker:
```shell
docker run -p 8000:8000 chromadb/chroma
```
Then, initialize the Document Store with `host` and `port` parameters:
```python
from haystack_integrations.document_stores.chroma import ChromaDocumentStore
document_store = ChromaDocumentStore(host="localhost", port=8000)
```
## Supported Retrievers
The Haystack Chroma integration comes with two Retriever components. They both rely on the Chroma [query API](https://docs.trychroma.com/reference/Collection#query), but they have different inputs and outputs so that you can pick the one that best fits your pipeline:
- [`ChromaQueryTextRetriever`](../pipeline-components/retrievers/chromaqueryretriever.mdx): This Retriever takes a plain-text query string in input and returns a list of matching documents. Chroma will create the embeddings for the query using its [default embedding function](https://docs.trychroma.com/embeddings#default-all-minilm-l6-v2).
- [`ChromaEmbeddingRetriever`](../pipeline-components/retrievers/chromaembeddingretriever.mdx): This Retriever takes the embeddings of a single query in input and returns a list of matching documents. The query needs to be embedded before being passed to this component. For example, you can use an [embedder](../pipeline-components/embedders.mdx) component.
## Additional References
🧑🍳 Cookbook: [Use Chroma for RAG and Indexing](https://haystack.deepset.ai/cookbook/chroma-indexing-and-rag-examples)
---
// File: document-stores/elasticsearch-document-store
# ElasticsearchDocumentStore
Use an Elasticsearch database with Haystack.
| | |
| --- | --- |
| API reference | [Elasticsearch](/reference/integrations-elasticsearch) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch |
ElasticsearchDocumentStore is excellent if you want to evaluate the performance of different retrieval options (dense vs. sparse) and aim for a smooth transition from PoC to production.
It features the approximate nearest neighbours (ANN) search.
### Initialization
[Install](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html) Elasticsearch and then [start](https://www.elastic.co/guide/en/elasticsearch/reference/current/starting-elasticsearch.html) an instance. Haystack supports Elasticsearch 8.
If you have Docker set up, we recommend pulling the Docker image and running it.
```shell
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.19.7
docker run -p 9200:9200 -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" -e "xpack.security.enabled=false" docker.elastic.co/elasticsearch/elasticsearch:8.19.7
```
As an alternative, you can go to [Elasticsearch integration GitHub](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch) and start a Docker container running Elasticsearch using the provided `docker-compose.yml`:
```shell
docker compose up
```
Once you have a running Elasticsearch instance, install the `elasticsearch-haystack` integration:
```shell
pip install elasticsearch-haystack
```
Then, initialize an `ElasticsearchDocumentStore` object that’s connected to the Elasticsearch instance and writes documents to it:
```python
from haystack_integrations.document_stores.elasticsearch import (
ElasticsearchDocumentStore,
)
from haystack import Document
document_store = ElasticsearchDocumentStore(hosts="http://localhost:9200")
document_store.write_documents(
[Document(content="This is first"), Document(content="This is second")],
)
print(document_store.count_documents())
```
### Supported Retrievers
[`ElasticsearchBM25Retriever`](../pipeline-components/retrievers/elasticsearchbm25retriever.mdx): A keyword-based Retriever that fetches documents matching a query from the Document Store.
[`ElasticsearchEmbeddingRetriever`](../pipeline-components/retrievers/elasticsearchembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query.
---
// File: document-stores/faissdocumentstore
# FAISSDocumentStore
| | |
| --- | --- |
| API reference | [FAISS](/reference/integrations-faiss) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/faiss |
`FAISSDocumentStore` is a local Document Store backed by [FAISS](https://github.com/facebookresearch/faiss) for vector similarity search.
It keeps vectors in a FAISS index and stores document data in memory, with optional persistence to disk.
`FAISSDocumentStore` is a good fit for local development and small to medium-sized datasets where you want a lightweight setup without running an external database service.
## Installation
Install the FAISS integration:
```shell
pip install faiss-haystack
```
## Initialization
Create a `FAISSDocumentStore` instance and write embedded documents:
```python
from haystack import Document
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.document_stores.faiss import FAISSDocumentStore
document_store = FAISSDocumentStore(
index_path="my_faiss_index", # Optional: enables persistence on disk
index_string="Flat",
embedding_dim=768,
)
document_store.write_documents(
[
Document(content="This is first", embedding=[0.1] * 768),
Document(content="This is second", embedding=[0.2] * 768),
],
policy=DuplicatePolicy.OVERWRITE,
)
print(document_store.count_documents())
# Persist index and metadata files (`.faiss` and `.json`)
document_store.save("my_faiss_index")
```
### Persistence
If you provide `index_path` when initializing `FAISSDocumentStore`, it tries to load existing persisted files (`.faiss` and `.json`) from that path.
You can also explicitly call:
- `save(index_path)` to write index and metadata to disk.
- `load(index_path)` to load them later.
Example of loading from a previously saved folder/path:
```python
from haystack_integrations.document_stores.faiss import FAISSDocumentStore
# This loads `my_faiss_index.faiss` and `my_faiss_index.json` if they exist
document_store = FAISSDocumentStore(index_path="my_faiss_index")
# Alternatively, initialize first and then load explicitly
another_store = FAISSDocumentStore(embedding_dim=768)
another_store.load("my_faiss_index")
```
## Supported Retrievers
[`FAISSEmbeddingRetriever`](../pipeline-components/retrievers/faissembeddingretriever.mdx): Retrieves documents from `FAISSDocumentStore` based on query embeddings.
### Fixing OpenMP Runtime Conflicts on macOS
#### Symptoms
You may encounter one or both of the following errors at runtime:
```
OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program.
```
```
resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
```
If setting `OMP_NUM_THREADS=1` prevents the crash, the root cause is **multiple OpenMP runtimes loaded simultaneously**. Each runtime maintains its own thread pool and thread-local storage (TLS). When two runtimes spin up worker threads at the same time, they corrupt each other's memory — causing segfaults at `N > 1` threads.
---
#### Diagnosis
First, find how many copies of `libomp.dylib` exist in your virtual environment:
```bash
find /path/to/your/.venv -name "libomp.dylib" 2>/dev/null
```
If you see more than one, e.g.:
```
.venv/lib/pythonX.Y/site-packages/torch/lib/libomp.dylib
.venv/lib/pythonX.Y/site-packages/sklearn/.dylibs/libomp.dylib
.venv/lib/pythonX.Y/site-packages/faiss/.dylibs/libomp.dylib
```
you need to consolidate them into a single runtime.
---
#### Fix
The solution is to pick one canonical `libomp.dylib` (torch's is a good choice) and replace all other copies with symlinks pointing to it.
For each duplicate, delete the copy and replace it with a symlink:
```bash
# Delete the duplicate
rm /path/to/.venv/lib/pythonX.Y/site-packages/
| | |
| --- | --- |
| API reference | [FalkorDB](/reference/integrations-falkordb) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/falkordb |
FalkorDB is a high-performance graph database optimized for GraphRAG workloads. The `FalkorDBDocumentStore` stores documents as graph nodes and supports native vector search — no APOC is required. Documents and their `meta` fields are stored flat on each node, and all bulk writes use `UNWIND` + `MERGE` for safe OpenCypher upserts.
For more information, see the [FalkorDB documentation](https://docs.falkordb.com/).
## Installation
Run FalkorDB with Docker:
```shell
docker run -d -p 6379:6379 falkordb/falkordb:latest
```
Install the Haystack integration:
```shell
pip install falkordb-haystack
```
## Usage
Initialize the document store and write documents:
```python
from haystack import Document
from haystack_integrations.document_stores.falkordb import FalkorDBDocumentStore
document_store = FalkorDBDocumentStore(
host="localhost",
port=6379,
embedding_dim=768,
recreate_graph=True,
)
document_store.write_documents(
[
Document(
content="There are over 7,000 languages spoken around the world today.",
),
Document(
content="Elephants have been observed to recognize themselves in mirrors.",
),
],
)
print(document_store.count_documents())
```
To learn more about the initialization parameters, see the [API docs](/reference/integrations-falkordb#falkordbdocumentstore).
To compute real embeddings for your documents, use a Document Embedder such as the [`SentenceTransformersDocumentEmbedder`](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx).
### Authentication
To connect to a password-protected FalkorDB instance, pass the password via `Secret`:
```python
from haystack.utils import Secret
from haystack_integrations.document_stores.falkordb import FalkorDBDocumentStore
document_store = FalkorDBDocumentStore(
host="localhost",
port=6379,
password=Secret.from_env_var("FALKORDB_PASSWORD"),
)
```
### Similarity Functions
`FalkorDBDocumentStore` supports two similarity functions for vector search:
- `"cosine"` (default): cosine similarity, best for normalized embeddings.
- `"euclidean"`: Euclidean distance, useful when embedding magnitude matters.
```python
document_store = FalkorDBDocumentStore(
host="localhost",
port=6379,
embedding_dim=768,
similarity="euclidean",
)
```
### Supported Retrievers
- [`FalkorDBEmbeddingRetriever`](../pipeline-components/retrievers/falkordbembeddingretriever.mdx): Retrieves documents from the `FalkorDBDocumentStore` based on vector similarity using FalkorDB's native vector index.
- [`FalkorDBCypherRetriever`](../pipeline-components/retrievers/falkordbcypherretriever.mdx): Retrieves documents by executing arbitrary OpenCypher queries, enabling graph traversal and multi-hop queries for GraphRAG pipelines.
---
// File: document-stores/inmemorydocumentstore
# InMemoryDocumentStore
The `InMemoryDocumentStore` is a very simple document store with no extra services or dependencies.
It is great for experimenting with Haystack, however we do not recommend using it for production.
### Initialization
`InMemoryDocumentStore` requires no external setup. Simply use this code:
```python
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
```
### Supported Retrievers
[`InMemoryBM25Retriever`](../pipeline-components/retrievers/inmemorybm25retriever.mdx): A keyword-based Retriever that fetches documents matching a query from a temporary in-memory database.
[`InMemoryEmbeddingRetriever`](../pipeline-components/retrievers/inmemoryembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query.
---
// File: document-stores/mongodbatlasdocumentstore
# MongoDBAtlasDocumentStore
| | |
| --- | --- |
| API reference | [MongoDB Atlas](/reference/integrations-mongodb-atlas) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mongodb_atlas |
`MongoDBAtlasDocumentStore` can be used to manage documents using [MongoDB Atlas](https://www.mongodb.com/atlas), a multi-cloud database service by the same people who build MongoDB. Atlas simplifies deploying and managing your databases while offering the versatility you need to build resilient and performant global applications on the cloud providers of your choice. You can use MongoDB Atlas on cloud providers such as AWS, Azure, or Google Cloud, all without leaving Atlas' web UI.
MongoDB Atlas supports embeddings and can therefore be used for embedding retrieval.
## Installation
To use MongoDB Atlas with Haystack, install the integration first:
```shell
pip install mongodb-atlas-haystack
```
## Initialization
To use MongoDB Atlas with Haystack, you will need to create your MongoDB Atlas account: check the [MongoDB Atlas documentation](https://www.mongodb.com/docs/atlas/getting-started/) for help. You also need to [create a vector search index](https://www.mongodb.com/docs/atlas/atlas-vector-search/create-index/#std-label-avs-create-index) and [a full-text search index](https://www.mongodb.com/docs/atlas/atlas-search/manage-indexes/#create-an-atlas-search-index) for the collection you plan to use.
Once you have your connection string, you should export it in an environment variable called `MONGO_CONNECTION_STRING`. It should look something like this:
```shell
export MONGO_CONNECTION_STRING="mongodb+srv://
| | |
| --- | --- |
| API reference | [OpenSearch](/reference/integrations-opensearch) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opensearch |
OpenSearch is a fully open source search and analytics engine for use cases such as log analytics, real-time application monitoring, and clickstream analysis. For more information, see the [OpenSearch documentation](https://opensearch.org/docs/).
This Document Store is great if you want to evaluate the performance of different retrieval options (dense vs. sparse). It’s compatible with the Amazon OpenSearch Service.
OpenSearch provides support for vector similarity comparisons and approximate nearest neighbors algorithms.
### Initialization
[Install](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/) and run an OpenSearch instance.
If you have Docker set up, we recommend pulling the Docker image and running it.
```shell
docker pull opensearchproject/opensearch:3.5.0
docker run \
-p 9200:9200 \
-p 9600:9600 \
-e "discovery.type=single-node" \
-e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" \
-e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=SecureHaystack*2026" \
opensearchproject/opensearch:3.5.0
```
As an alternative, you can go to [OpenSearch integration GitHub](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opensearch) and start a Docker container running OpenSearch using the provided `docker-compose.yml`:
```shell
docker compose up
```
Once you have a running OpenSearch instance, install the `opensearch-haystack` integration:
```shell
pip install opensearch-haystack
```
Then, initialize an `OpenSearchDocumentStore` object that’s connected to the OpenSearch instance and writes documents to it:
```python
from haystack_integrations.document_stores.opensearch import OpenSearchDocumentStore
from haystack import Document
document_store = OpenSearchDocumentStore(
hosts="http://localhost:9200",
use_ssl=True,
verify_certs=False,
http_auth=("admin", "SecureHaystack*2026"),
)
document_store.write_documents(
[Document(content="This is first"), Document(content="This is second")],
)
print(document_store.count_documents())
```
### Supported Retrievers
[`OpenSearchBM25Retriever`](../pipeline-components/retrievers/opensearchbm25retriever.mdx): A keyword-based Retriever that fetches documents matching a query from the Document Store.
[`OpenSearchEmbeddingRetriever`](../pipeline-components/retrievers/opensearchembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query.
## Additional References
🧑🍳 Cookbook: [PDF-Based Question Answering with Amazon Bedrock and Haystack](https://haystack.deepset.ai/cookbook/amazon_bedrock_for_documentation_qa)
---
// File: document-stores/oracledocumentstore
# OracleDocumentStore
| | |
| --- | --- |
| API reference | [Oracle](/reference/integrations-oracle) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/oracle |
`OracleDocumentStore` is a Document Store backed by [Oracle AI Vector Search](https://www.oracle.com/database/ai-vector-search/), available in Oracle Database 23ai and later.
It stores documents alongside dense vector embeddings in a native `VECTOR` column, and supports both vector similarity search and keyword search via an automatically managed DBMS_SEARCH index.
## Installation
```shell
pip install oracle-haystack
```
The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Connection
`OracleDocumentStore` connects to Oracle using the `OracleConnectionConfig` dataclass, which supports two connection modes:
- **Thin mode** (default): connects directly over TCP. No Oracle Instant Client required.
- **Thick mode**: activated automatically when `wallet_location` is provided. Used for Oracle Autonomous Database (ADB-S) connections.
Set the connection parameters as environment variables:
```shell
export ORACLE_USER="haystack"
export ORACLE_PASSWORD="secret"
export ORACLE_DSN="localhost:1521/freepdb1"
```
## Initialization
```python
from haystack.utils import Secret
from haystack_integrations.document_stores.oracle import (
OracleDocumentStore,
OracleConnectionConfig,
)
document_store = OracleDocumentStore(
connection_config=OracleConnectionConfig(
user=Secret.from_env_var("ORACLE_USER"),
password=Secret.from_env_var("ORACLE_PASSWORD"),
dsn=Secret.from_env_var("ORACLE_DSN"),
),
embedding_dim=768,
)
```
To learn more about the initialization parameters, see the [API docs](/reference/integrations-oracle#oracledocumentstore).
### Connecting to Oracle Autonomous Database
For Oracle Autonomous Database (ADB-S), provide a wallet for authentication. The store automatically activates thick mode when `wallet_location` is set:
```python
document_store = OracleDocumentStore(
connection_config=OracleConnectionConfig(
user=Secret.from_env_var("ORACLE_USER"),
password=Secret.from_env_var("ORACLE_PASSWORD"),
dsn=Secret.from_env_var("ORACLE_DSN"),
wallet_location="/path/to/wallet",
wallet_password=Secret.from_env_var("WALLET_PASSWORD"),
),
embedding_dim=1536,
)
```
### HNSW Vector Index
By default, the store performs exact vector search. To enable approximate nearest-neighbor search (faster on large datasets), create an HNSW index:
```python
document_store = OracleDocumentStore(
connection_config=OracleConnectionConfig(
user=Secret.from_env_var("ORACLE_USER"),
password=Secret.from_env_var("ORACLE_PASSWORD"),
dsn=Secret.from_env_var("ORACLE_DSN"),
),
embedding_dim=768,
distance_metric="COSINE",
create_index=True, # creates the HNSW index on startup
hnsw_neighbors=32,
hnsw_ef_construction=200,
hnsw_accuracy=95,
)
```
## Supported Retrievers
- [`OracleEmbeddingRetriever`](../pipeline-components/retrievers/oracleembeddingretriever.mdx): Retrieves documents from `OracleDocumentStore` based on vector similarity to a query embedding.
- [`OracleKeywordRetriever`](../pipeline-components/retrievers/oraclekeywordretriever.mdx): Retrieves documents matching a keyword query using Oracle's DBMS_SEARCH full-text index.
## Example: RAG pipeline
```python
from haystack import Document, Pipeline
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
from haystack_integrations.document_stores.oracle import (
OracleDocumentStore,
OracleConnectionConfig,
)
from haystack_integrations.components.retrievers.oracle import OracleEmbeddingRetriever
document_store = OracleDocumentStore(
connection_config=OracleConnectionConfig(
user=Secret.from_env_var("ORACLE_USER"),
password=Secret.from_env_var("ORACLE_PASSWORD"),
dsn=Secret.from_env_var("ORACLE_DSN"),
),
embedding_dim=384,
)
# Index documents
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness.",
),
Document(
content="In certain places, you can witness the phenomenon of bioluminescent waves.",
),
]
doc_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
embedded_docs = doc_embedder.run(documents)["documents"]
document_store.write_documents(embedded_docs, policy=DuplicatePolicy.OVERWRITE)
# Build a RAG pipeline
template = [
ChatMessage.from_user(
"""
Given the following context, answer the question.
Context: {% for doc in documents %}{{ doc.content }}{% endfor %}
Question: {{ query }}
""",
),
]
pipeline = Pipeline()
pipeline.add_component(
"embedder",
SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"),
)
pipeline.add_component(
"retriever",
OracleEmbeddingRetriever(document_store=document_store, top_k=3),
)
pipeline.add_component("prompt_builder", ChatPromptBuilder(template=template))
pipeline.add_component(
"llm",
OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
)
pipeline.connect("embedder.embedding", "retriever.query_embedding")
pipeline.connect("retriever.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder.prompt", "llm.messages")
result = pipeline.run(
{
"embedder": {"text": "How many languages are there?"},
"prompt_builder": {"query": "How many languages are there?"},
},
)
print(result["llm"]["replies"][0].text)
```
---
// File: document-stores/pgvectordocumentstore
# PgvectorDocumentStore
| | |
| --- | --- |
| API reference | [Pgvector](/reference/integrations-pgvector) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/pgvector/ |
Pgvector is an extension for PostgreSQL that enhances its capabilities with vector similarity search. It builds upon the classic features of PostgreSQL, such as ACID compliance and point-in-time recovery, and introduces the ability to perform exact and approximate nearest neighbor search using vectors.
For more information, see the [pgvector repository](https://github.com/pgvector/pgvector).
Pgvector Document Store supports embedding retrieval and metadata filtering.
## Installation
To quickly set up a PostgreSQL database with pgvector, you can use Docker:
```shell
docker run -d -p 5432:5432 -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=postgres pgvector/pgvector:pg17
```
For more information on installing pgvector, visit the [pgvector GitHub repository](https://github.com/pgvector/pgvector).
To use pgvector with Haystack, install the `pgvector-haystack` integration:
```shell
pip install pgvector-haystack
```
## Usage
### Connection String
Define the connection string to your PostgreSQL database in the `PG_CONN_STR` environment variable. Two formats are supported:
**URI format:**
```shell
export PG_CONN_STR="postgresql://USER:PASSWORD@HOST:PORT/DB_NAME"
```
**Keyword/value format:**
```shell
export PG_CONN_STR="host=HOST port=PORT dbname=DB_NAME user=USER password=PASSWORD"
```
:::caution[Special Characters in Connection URIs]
When using the URI format, special characters in the password must be [percent-encoded](https://en.wikipedia.org/wiki/Percent-encoding). Otherwise, connection errors may occur. A password like `p=ssword` would cause the error `psycopg.OperationalError: [Errno -2] Name or service not known`.
For example, if your password is `p=ssword`, the connection string should be:
```shell
export PG_CONN_STR="postgresql://postgres:p%3Dssword@localhost:5432/postgres"
```
Alternatively, use the keyword/value format, which does not require percent-encoding:
```shell
export PG_CONN_STR="host=localhost port=5432 dbname=postgres user=postgres password=p=ssword"
```
:::
For more details, see the [PostgreSQL connection string documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING).
## Initialization
Initialize a `PgvectorDocumentStore` object that’s connected to the PostgreSQL database and writes documents to it:
```python
from haystack_integrations.document_stores.pgvector import PgvectorDocumentStore
from haystack import Document
document_store = PgvectorDocumentStore(
embedding_dimension=768,
vector_function="cosine_similarity",
recreate_table=True,
search_strategy="hnsw",
)
document_store.write_documents(
[
Document(content="This is first", embedding=[0.1] * 768),
Document(content="This is second", embedding=[0.3] * 768),
],
)
print(document_store.count_documents())
```
To learn more about the initialization parameters, see our [API docs](/reference/integrations-pgvector#pgvectordocumentstore).
To properly compute embeddings for your documents, you can use a Document Embedder (for instance, the [`SentenceTransformersDocumentEmbedder`](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx)).
### Supported Retrievers
- [`PgvectorEmbeddingRetriever`](../pipeline-components/retrievers/pgvectorembeddingretriever.mdx): An embedding-based Retriever that fetches documents from the Document Store based on a query embedding provided to the Retriever.
- [`PgvectorKeywordRetriever`](../pipeline-components/retrievers/pgvectorkeywordretriever.mdx): A keyword-based Retriever that fetches documents matching a query from the Pgvector Document Store.
---
// File: document-stores/pinecone-document-store
# PineconeDocumentStore
Use a Pinecone vector database with Haystack.
| | |
| --- | --- |
| API reference | [Pinecone](/reference/integrations-pinecone) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/pinecone |
[Pinecone](https://www.pinecone.io/) is a cloud-based vector database. It is fast and easy to use.
Unlike other solutions (such as Qdrant and Weaviate), it can’t run locally on the user's machine but provides a generous free tier.
### Installation
You can simply install the Pinecone Haystack integration with:
```shell
pip install pinecone-haystack
```
### Initialization
- To use Pinecone as a Document Store in Haystack, sign up for a free Pinecone [account](https://app.pinecone.io/) and get your API key.
The Pinecone API key can be explicitly provided or automatically read from the environment variable `PINECONE_API_KEY` (recommended).
- In Haystack, each `PineconeDocumentStore` operates in a specific namespace of an index. If not provided, both index and namespace are `default`.
If the index already exists, the Document Store connects to it. Otherwise, it creates a new index.
- When creating a new index, you can provide a `spec` in the form of a dictionary. This allows choosing between serverless and pod deployment options and setting additional parameters. Refer to the [Pinecone documentation](https://docs.pinecone.io/reference/api/control-plane/create_index) for more details. If not provided, a default spec with serverless deployment in the `us-east-1` region will be used (compatible with the free tier).
- You can provide `dimension` and `metric`, but they are only taken into account if the Pinecone index does not already exist.
Then, you can use the Document Store like this:
```python
from haystack import Document
from haystack_integrations.document_stores.pinecone import PineconeDocumentStore
# Make sure you have the PINECONE_API_KEY environment variable set
document_store = PineconeDocumentStore(
index="default",
namespace="default",
dimension=5,
metric="cosine",
spec={"serverless": {"region": "us-east-1", "cloud": "aws"}},
)
document_store.write_documents(
[
Document(content="This is first", embedding=[0.1] * 5),
Document(content="This is second", embedding=[0.1, 0.2, 0.3, 0.4, 0.5]),
],
)
print(document_store.count_documents())
```
### Supported Retrievers
[`PineconeEmbeddingRetriever`](../pipeline-components/retrievers/pineconedenseretriever.mdx): Retrieves documents from the `PineconeDocumentStore` based on their dense embeddings (vectors).
---
// File: document-stores/qdrant-document-store
# QdrantDocumentStore
Use the Qdrant vector database with Haystack.
| | |
| --- | --- |
| API reference | [Qdrant](/reference/integrations-qdrant) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/qdrant |
Qdrant is a powerful high-performance, massive-scale vector database. The `QdrantDocumentStore` can be used with any Qdrant instance, in-memory, locally persisted, hosted, and the official Qdrant Cloud.
### Installation
You can simply install the Qdrant Haystack integration with:
```shell
pip install qdrant-haystack
```
### Initialization
The quickest way to use `QdrantDocumentStore` is to create an in-memory instance of it:
```python
from haystack.dataclasses.document import Document
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
document_store = QdrantDocumentStore(
":memory:",
recreate_index=True,
return_embedding=True,
wait_result_from_api=True,
)
document_store.write_documents(
[
Document(content="This is first", embedding=[0.0] * 768),
Document(content="This is second", embedding=[0.1] * 768),
],
)
print(document_store.count_documents())
```
:::warning[Collections Created Outside Haystack]
When you create a `QdrantDocumentStore` instance, Haystack takes care of setting up the collection. In general, you cannot use a Qdrant collection created without Haystack with Haystack. If you want to migrate your existing collection, see the sample script at https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/qdrant/src/haystack_integrations/document_stores/qdrant/migrate_to_sparse.py.
:::
You can also connect directly to [Qdrant Cloud](https://cloud.qdrant.io/login). Once you have your API key and your cluster URL from the Qdrant dashboard, you can connect like this:
```python
from haystack.dataclasses.document import Document
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
from haystack.utils import Secret
document_store = QdrantDocumentStore(
url="https://XXXXXXXXX.us-east4-0.gcp.cloud.qdrant.io:6333",
index="your_index_name",
embedding_dim=5, # based on the embedding model
recreate_index=True, # enable only to recreate the index and not connect to the existing one
api_key=Secret.from_token("YOUR_TOKEN"),
)
document_store.write_documents(
[
Document(content="This is first", embedding=[0.0] * 5),
Document(content="This is second", embedding=[0.1, 0.2, 0.3, 0.4, 0.5]),
],
)
print(document_store.count_documents())
```
:::tip[More information]
You can find more ways to initialize and use QdrantDocumentStore on our [integration page](https://haystack.deepset.ai/integrations/qdrant-document-store).
:::
### Supported Retrievers
- [`QdrantEmbeddingRetriever`](../pipeline-components/retrievers/qdrantembeddingretriever.mdx): Retrieves documents from the `QdrantDocumentStore` based on their dense embeddings (vectors).
- [`QdrantSparseEmbeddingRetriever`](../pipeline-components/retrievers/qdrantsparseembeddingretriever.mdx): Retrieves documents from the `QdrantDocumentStore` based on their sparse embeddings.
- [`QdrantHybridRetriever`](../pipeline-components/retrievers/qdranthybridretriever.mdx): Retrieves documents from the `QdrantDocumentStore` based on both dense and sparse embeddings.
:::note[Sparse Embedding Support]
To use Sparse Embedding support, you need to initialize the `QdrantDocumentStore` with `use_sparse_embeddings=True`, which is `False` by default.
If you want to use Document Store or collection previously created with this feature disabled, you must migrate the existing data. You can do this by taking advantage of the `migrate_to_sparse_embeddings_support` utility function.
:::
## Additional References
🧑🍳 Cookbook: [Sparse Embedding Retrieval with Qdrant and FastEmbed](https://haystack.deepset.ai/cookbook/sparse_embedding_retrieval)
---
// File: document-stores/supabasedocumentstore
# SupabaseDocumentStore
| | |
| --- | --- |
| API reference | [Supabase](/reference/integrations-supabase) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/supabase/ |
[Supabase](https://supabase.com/) is an open-source backend platform built on PostgreSQL. The Supabase integration for Haystack provides two document stores:
- **`SupabasePgvectorDocumentStore`** — vector similarity search using the [pgvector](https://github.com/pgvector/pgvector) PostgreSQL extension, which comes pre-installed on Supabase.
- **`SupabaseGroongaDocumentStore`** — multilingual full-text search using the [PGroonga](https://pgroonga.github.io/) PostgreSQL extension. No embeddings required.
## Installation
```shell
pip install supabase-haystack
```
The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## SupabasePgvectorDocumentStore
`SupabasePgvectorDocumentStore` is a thin wrapper around [`PgvectorDocumentStore`](./pgvectordocumentstore.mdx) with Supabase-specific defaults:
- Reads the connection string from the `SUPABASE_DB_URL` environment variable.
- Defaults `create_extension` to `False` since pgvector is pre-installed on Supabase.
### Connection
Set the `SUPABASE_DB_URL` environment variable with your Supabase database connection string.
:::tip[Use session mode (port 5432)]
Supabase offers two pooler ports: transaction mode (port 6543) and session mode (port 5432). For best compatibility with pgvector operations, use session mode or a direct connection.
:::
```shell
export SUPABASE_DB_URL="postgresql://postgres.[project-ref]:[password]@aws-0-[region].pooler.supabase.com:5432/postgres"
```
### Initialization
```python
from haystack_integrations.document_stores.supabase import SupabasePgvectorDocumentStore
document_store = SupabasePgvectorDocumentStore(
embedding_dimension=768,
vector_function="cosine_similarity",
recreate_table=True,
)
```
To learn more about the initialization parameters, see the [API docs](/reference/integrations-supabase#supabasepgvectordocumentstore).
### Supported Retrievers
- [`SupabasePgvectorEmbeddingRetriever`](../pipeline-components/retrievers/supabasepgvectorembeddingretriever.mdx): Fetches documents from the store based on a query embedding.
- [`SupabasePgvectorKeywordRetriever`](../pipeline-components/retrievers/supabasepgvectorkeywordretriever.mdx): Fetches documents matching a keyword query using PostgreSQL's `ts_rank_cd` ranking.
### Example: RAG pipeline
```python
from haystack import Document, Pipeline
from haystack.document_stores.types.policy import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
from haystack_integrations.document_stores.supabase import SupabasePgvectorDocumentStore
from haystack_integrations.components.retrievers.supabase import (
SupabasePgvectorEmbeddingRetriever,
)
document_store = SupabasePgvectorDocumentStore(
embedding_dimension=768,
vector_function="cosine_similarity",
recreate_table=True,
)
# Index documents
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness.",
),
Document(
content="In certain places, you can witness the phenomenon of bioluminescent waves.",
),
]
embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = embedder.run(documents)
document_store.write_documents(
documents_with_embeddings["documents"],
policy=DuplicatePolicy.OVERWRITE,
)
# Query pipeline
prompt_template = [
ChatMessage.from_system("Answer the question based on the provided context."),
ChatMessage.from_user(
"Query: {{query}}\nDocuments:\n{% for doc in documents %}{{ doc.content }}\n{% endfor %}\nAnswer:",
),
]
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
SupabasePgvectorEmbeddingRetriever(document_store=document_store),
)
query_pipeline.add_component(
"prompt_builder",
ChatPromptBuilder(
template=prompt_template,
required_variables=["query", "documents"],
),
)
query_pipeline.add_component("generator", OpenAIChatGenerator(model="gpt-4o"))
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query_pipeline.connect("retriever.documents", "prompt_builder.documents")
query_pipeline.connect("prompt_builder.prompt", "generator.messages")
result = query_pipeline.run(
{
"text_embedder": {"text": "How many languages are there?"},
"prompt_builder": {"query": "How many languages are there?"},
},
)
```
---
## SupabaseGroongaDocumentStore
`SupabaseGroongaDocumentStore` uses [PGroonga](https://pgroonga.github.io/), a PostgreSQL extension for fast, multilingual full-text search. Unlike the pgvector store, it works with plain text queries and requires no embeddings.
### Prerequisites
PGroonga must be enabled in your Supabase project. Run the following SQL in the Supabase SQL editor:
```sql
CREATE EXTENSION IF NOT EXISTS pgroonga;
```
You also need to create a SQL function that PGroonga uses for search. See the [integration README](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/supabase/) for the required function definition.
### Initialization
```python
from haystack_integrations.document_stores.supabase import SupabaseGroongaDocumentStore
from haystack.utils import Secret
document_store = SupabaseGroongaDocumentStore(
supabase_url="https://
| | |
| --- | --- |
| API reference | [Valkey](/reference/integrations-valkey) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/valkey |
[Valkey](https://valkey.io/) is a high-performance, in-memory data structure store that you can use in Haystack pipelines with the `ValkeyDocumentStore`. Valkey operates in-memory by default for maximum performance, but can be configured with persistence options for data durability.
The `ValkeyDocumentStore` connects to a Valkey server with the search module running and supports vector similarity search for RAG and other retrieval use cases. For a detailed overview of all the available methods and settings, visit the [API Reference](/reference/integrations-valkey#valkeydocumentstore).
## Installation
You can install the Valkey Haystack integration with:
```shell
pip install valkey-haystack
```
The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Initialization
To use Valkey as your data storage for Haystack pipelines, you need a Valkey server with the search module running. Initialize a `ValkeyDocumentStore` like this:
```python
from haystack_integrations.document_stores.valkey import ValkeyDocumentStore
document_store = ValkeyDocumentStore(
nodes_list=[("localhost", 6379)],
index_name="my_documents",
embedding_dim=768,
distance_metric="cosine",
)
```
### Running Valkey locally
For development and testing, you can start a Valkey server with Docker:
```shell
docker run -d -p 6379:6379 valkey/valkey-bundle:latest
```
Then connect with the same initialization code above, using `nodes_list=[("localhost", 6379)]`.
For more advanced configurations and clustering setups, refer to the [Valkey documentation](https://valkey.io/docs/).
## Writing documents
To write documents to your `ValkeyDocumentStore`, create an indexing pipeline or use the `write_documents()` method. You can use [Converters](../pipeline-components/converters.mdx), [PreProcessors](../pipeline-components/preprocessors.mdx), and other integrations to fetch and prepare data. Below is an example that indexes Markdown files into Valkey.
### Indexing pipeline
```python
from haystack import Pipeline
from haystack.components.converters import MarkdownToDocument
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
)
from haystack.components.preprocessors import DocumentSplitter
from haystack_integrations.document_stores.valkey import ValkeyDocumentStore
document_store = ValkeyDocumentStore(
nodes_list=[("localhost", 6379)],
index_name="my_documents",
embedding_dim=768,
distance_metric="cosine",
)
indexing = Pipeline()
indexing.add_component("converter", MarkdownToDocument())
indexing.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=2),
)
indexing.add_component("embedder", SentenceTransformersDocumentEmbedder())
indexing.add_component("writer", DocumentWriter(document_store))
indexing.connect("converter", "splitter")
indexing.connect("splitter", "embedder")
indexing.connect("embedder", "writer")
indexing.run({"converter": {"sources": ["filename.md"]}})
```
## Using Valkey in a RAG pipeline
Once documents are in your `ValkeyDocumentStore`, you can use [`ValkeyEmbeddingRetriever`](../pipeline-components/retrievers/valkeyembeddingretriever.mdx) to retrieve them. The following example builds a RAG pipeline with a custom prompt:
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
)
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.document_stores.valkey import ValkeyDocumentStore
from haystack_integrations.components.retrievers.valkey import ValkeyEmbeddingRetriever
document_store = ValkeyDocumentStore(
nodes_list=[("localhost", 6379)],
index_name="my_documents",
embedding_dim=768,
distance_metric="cosine",
)
prompt_template = [
ChatMessage.from_system(
"Answer the question based on the provided context. If the context does not include an answer, reply with 'I don't know'.",
),
ChatMessage.from_user(
"Query: {{query}}\n"
"Documents:\n{% for doc in documents %}{{ doc.content }}\n{% endfor %}\n"
"Answer:",
),
]
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
ValkeyEmbeddingRetriever(document_store=document_store),
)
query_pipeline.add_component(
"prompt_builder",
ChatPromptBuilder(
template=prompt_template,
required_variables=["query", "documents"],
),
)
query_pipeline.add_component(
"generator",
OpenAIChatGenerator(
api_key=Secret.from_token("YOUR_OPENAI_API_KEY"),
model="gpt-4o",
),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query_pipeline.connect("retriever.documents", "prompt_builder.documents")
query_pipeline.connect("prompt_builder.prompt", "generator.messages")
query = "What is Valkey?"
results = query_pipeline.run(
{
"text_embedder": {"text": query},
"prompt_builder": {"query": query},
},
)
```
For more examples, see the [examples folder](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/valkey/examples) in the repository.
## Performance benefits
- **In-memory storage**: Fast read and write operations.
- **High throughput**: Handles many operations per second.
- **Low latency**: Minimal response times for document operations.
- **Scalability**: Supports clustering for horizontal scaling.
## Supported Retrievers
[`ValkeyEmbeddingRetriever`](../pipeline-components/retrievers/valkeyembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query from the `ValkeyDocumentStore`.
---
// File: document-stores/vespadocumentstore
# VespaDocumentStore
| | |
| --- | --- |
| API reference | [Vespa](/reference/integrations-vespa) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/vespa |
[Vespa](https://vespa.ai/) is an open-source big data serving engine that supports structured, text, and vector search at scale. The `VespaDocumentStore` connects Haystack to an existing Vespa application through [pyvespa](https://vespa-engine.github.io/pyvespa/) and supports both lexical and dense vector retrieval as well as metadata filtering.
Unlike most other Haystack Document Stores, the `VespaDocumentStore` does **not** create or deploy the Vespa application or schema for you. You configure Vespa with the fields and rank profiles you need, deploy it (either self-hosted or on [Vespa Cloud](https://cloud.vespa.ai/)), and then point the Document Store at the running endpoint.
## Installation
Install the `vespa-haystack` integration:
```shell
pip install vespa-haystack
```
To run Vespa locally, see the [Vespa quick start](https://docs.vespa.ai/en/vespa-quick-start.html). To deploy a managed Vespa application, see [Vespa Cloud](https://cloud.vespa.ai/en/getting-started).
## Usage
### Prerequisites: Vespa Schema
Before using the `VespaDocumentStore`, you need a deployed Vespa application with a schema compatible with the fields you configure on the Document Store. By default, the integration expects:
- A text field named `content` for the Document body.
- A tensor field named `embedding` for dense vectors (when using embedding retrieval).
- A rank profile named `bm25` for lexical retrieval (used by `VespaKeywordRetriever`).
- A rank profile named `semantic` that ranks with `closeness(field, embedding)` (used by `VespaEmbeddingRetriever`).
Field and rank profile names can be customized via the Document Store and Retriever constructors. See the [Vespa documentation](https://docs.vespa.ai/en/schemas.html) for details on writing schemas and rank profiles.
### Authentication
The `VespaDocumentStore` supports the authentication methods provided by `pyvespa`:
- **No authentication** for local development against an unsecured Vespa endpoint.
- **mTLS** with a data plane certificate and key (via the `cert` and `key` parameters as [Secrets](../concepts/secret-management.mdx)).
- **Bearer token** for Vespa Cloud token endpoints (via `vespa_cloud_secret_token` or the `VESPA_CLOUD_SECRET_TOKEN` environment variable).
The Vespa endpoint URL can be passed via the `url` parameter or the `VESPA_URL` environment variable:
```shell
export VESPA_URL="http://localhost"
```
For Vespa Cloud token authentication:
```shell
export VESPA_URL="https://my-app.my-tenant.aws-us-east-1c.z.vespa-app.cloud"
export VESPA_CLOUD_SECRET_TOKEN="my-secret-token"
```
## Initialization
Point the `VespaDocumentStore` at your deployed Vespa application and write Documents to it. The HTTP client is created lazily on first use:
```python
from haystack import Document
from haystack_integrations.document_stores.vespa import VespaDocumentStore
document_store = VespaDocumentStore(
url="http://localhost",
schema="doc",
namespace="doc",
content_field="content",
embedding_field="embedding",
metadata_fields=["category"],
)
document_store.write_documents(
[
Document(
content="Haystack integrates with Vespa for search.",
meta={"category": "docs"},
),
Document(
content="Vespa supports lexical and vector retrieval.",
meta={"category": "docs"},
),
],
)
print(document_store.count_documents())
```
To learn more about the initialization parameters, see our [API docs](/reference/integrations-vespa#vespadocumentstore).
To compute embeddings for your Documents, you can use a Document Embedder, such as the [`SentenceTransformersDocumentEmbedder`](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx).
### Metadata Fields
Vespa is strictly schema-bound: every metadata field that you want to feed or read back from Vespa must exist as a field in the deployed schema. Use the `metadata_fields` parameter to declare an allowlist of metadata keys to send to Vespa on write and to request back on read. Metadata keys that are not in this allowlist are kept on Documents in memory but are not stored in Vespa.
### Metadata Filtering
The `VespaDocumentStore` supports comparison operators (`==`, `!=`, `>`, `>=`, `<`, `<=`, `in`, `not in`) and the logical operators `AND`, `OR`, and `NOT`. Filters are translated to Vespa's [YQL](https://docs.vespa.ai/en/query-language.html) where clauses whenever possible.
Filters on date-typed values are evaluated client-side in Python when YQL cannot express the comparison directly. For more details on filter syntax, refer to [Metadata Filtering](../concepts/metadata-filtering.mdx).
### Supported Retrievers
- [`VespaEmbeddingRetriever`](../pipeline-components/retrievers/vespaembeddingretriever.mdx): A dense embedding-based Retriever that fetches Documents from Vespa using nearest-neighbor search and a configurable rank profile.
- [`VespaKeywordRetriever`](../pipeline-components/retrievers/vespakeywordretriever.mdx): A lexical Retriever that fetches Documents from Vespa using a configurable rank profile (BM25 by default).
---
// File: document-stores/weaviatedocumentstore
# WeaviateDocumentStore
| | |
| --- | --- |
| API reference | [Weaviate](/reference/integrations-weaviate) |
| GitHub link | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weaviate |
Weaviate is a multi-purpose vector DB that can store both embeddings and data objects, making it a good choice for multi-modality.
The `WeaviateDocumentStore` can connect to any Weaviate instance, whether it's running on Weaviate Cloud Services, Kubernetes, or a local Docker container.
## Installation
You can simply install the Weaviate Haystack integration with:
```shell
pip install weaviate-haystack
```
## Initialization
### Weaviate Embedded
To use `WeaviateDocumentStore` as a temporary instance, initialize it as ["Embedded"](https://weaviate.io/developers/weaviate/installation/embedded):
```python
from haystack_integrations.document_stores.weaviate import WeaviateDocumentStore
from weaviate.embedded import EmbeddedOptions
document_store = WeaviateDocumentStore(embedded_options=EmbeddedOptions())
```
### Docker
You can use `WeaviateDocumentStore` in a local Docker container. This is what a minimal `docker-compose.yml` could look like:
```yaml
---
services:
weaviate:
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
image: semitechnologies/weaviate:1.36.2
ports:
- 8080:8080
- 50051:50051
volumes:
- weaviate_data:/var/lib/weaviate
restart: 'no'
environment:
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
DEFAULT_VECTORIZER_MODULE: 'none'
ENABLE_MODULES: ''
CLUSTER_HOSTNAME: 'node1'
volumes:
weaviate_data:
...
```
:::warning
With this example, we explicitly enable access without authentication, so you don't need to set any username, password, or API key to connect to our local instance. That is strongly discouraged for production use. See the [authorization](#authorization) section for detailed information.
:::
Start your container with `docker compose up -d` and then initialize the Document Store with:
```python
from haystack_integrations.document_stores.weaviate.document_store import (
WeaviateDocumentStore,
)
from haystack import Document
document_store = WeaviateDocumentStore(url="http://localhost:8080")
document_store.write_documents(
[Document(content="This is first"), Document(content="This is second")],
)
print(document_store.count_documents())
```
### Weaviate Cloud Service
To use the [Weaviate managed cloud service](https://weaviate.io/developers/wcs), first, create your Weaviate cluster.
Then, initialize the `WeaviateDocumentStore` using the API Key and URL found in your [Weaviate account](https://console.weaviate.cloud/):
```python
from haystack_integrations.document_stores.weaviate import (
WeaviateDocumentStore,
AuthApiKey,
)
from haystack import Document
import os
os.environ["WEAVIATE_API_KEY"] = "YOUR-API-KEY"
auth_client_secret = AuthApiKey()
document_store = WeaviateDocumentStore(
url="YOUR-WEAVIATE-URL",
auth_client_secret=auth_client_secret,
)
```
## Authorization
We provide some utility classes in the `auth` package to handle authorization using different credentials. Every class stores distinct [secrets](../concepts/secret-management.mdx) and retrieves them from the environment variables when required.
The default environment variables for the classes are:
- **`AuthApiKey`**
- `WEAVIATE_API_KEY`
- **`AuthBearerToken`**
- `WEAVIATE_ACCESS_TOKEN`
- `WEAVIATE_REFRESH_TOKEN`
- **`AuthClientCredentials`**
- `WEAVIATE_CLIENT_SECRET`
- `WEAVIATE_SCOPE`
- **`AuthClientPassword`**
- `WEAVIATE_USERNAME`
- `WEAVIATE_PASSWORD`
- `WEAVIATE_SCOPE`
You can easily change environment variables if needed. In the following snippet, we instruct `AuthApiKey` to look for `MY_ENV_VAR`.
```python
from haystack_integrations.document_stores.weaviate.auth import AuthApiKey
from haystack.utils.auth import Secret
AuthApiKey(api_key=Secret.from_env_var("MY_ENV_VAR"))
```
## Supported Retrievers
[`WeaviateBM25Retriever`](../pipeline-components/retrievers/weaviatebm25retriever.mdx): A keyword-based Retriever that fetches documents matching a query from the Document Store.
[`WeaviateEmbeddingRetriever`](../pipeline-components/retrievers/weaviateembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query.
---
// File: intro
# Introduction to Haystack
Haystack is an **open-source AI framework** for building production-ready **AI Agents**, **powerful RAG applications** and **scalable multimodal search systems**. Build pipelines using reusable components, each responsible for specific tasks. Customize and extend pipelines to match your requirements. Learn more about Haystack and how it works.
:::tip[Welcome to Haystack]
To skip the introductions and go directly to installing and creating a search app, see [Get Started](overview/get-started.mdx).
:::
Haystack is an open-source AI orchestration framework that you can use to build powerful, production-ready applications with Large Language Models (LLMs) for various use cases. Whether you’re creating autonomous agents, multimodal apps, or scalable RAG systems, Haystack provides the tools to move from idea to production easily.
Haystack is designed in a modular way, allowing you to combine the best technology from OpenAI, Google, Anthropic, and open-source projects like Hugging Face's Transformers.
The core foundation of Haystack consists of components and pipelines, along with Document Stores, Agents, Tools, and many integrations. Read more about Haystack concepts in the [Haystack Concepts Overview](concepts/concepts-overview.mdx).
Supported by an engaged community of developers, Haystack has grown into a comprehensive and user-friendly framework for LLM-based development.
:::note[Looking to scale with confidence?]
If your team needs **enterprise-grade support, best practices, and deployment guidance** to run Haystack in production, check out **Haystack Enterprise Starter**.
📜 [Learn more about Haystack Enterprise Starter](https://haystack.deepset.ai/blog/announcing-haystack-enterprise)
🤝 [Get in touch with our team](https://www.deepset.ai/products-and-services/haystack-enterprise-starter)
👉 For platform tooling to **manage data, pipelines, testing, and governance at scale**, explore the [Haystack Enterprise Platform](https://www.deepset.ai/products-and-services/haystack-enterprise-platform).
:::
---
// File: memory-stores/cogneememorystore
# CogneeMemoryStore
`CogneeMemoryStore` is a persistent memory store backed by Cognee's knowledge graph API. It is the shared data layer used by [`CogneeRetriever`](../pipeline-components/retrievers/cogneeretriever.mdx) and [`CogneeWriter`](../pipeline-components/writers/cogneewriter.mdx).
| | |
| --- | --- |
| **Used by** | [`CogneeRetriever`](../pipeline-components/retrievers/cogneeretriever.mdx), [`CogneeWriter`](../pipeline-components/writers/cogneewriter.mdx) |
| **Optional init variables** | `search_type`, `top_k`, `dataset_name`, `session_id`, `self_improvement`, `timeout` |
| **API reference** | [Cognee](/reference/integrations-cognee#cogneememorystore) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cognee |
| **Package name** | `cognee-haystack` |
## Overview
`CogneeMemoryStore` wraps Cognee's V2 memory API:
- `add_memories` → `cognee.remember`
- `search_memories` → `cognee.recall`
- `improve` → `cognee.improve`
- `delete_all_memories` → `cognee.forget`
Cognee supports two memory tiers. Set `session_id` to use the **session cache** — fast writes with no LLM extraction, session-aware recall. Leave `session_id` as `None` to write to the **permanent knowledge graph**, which uses LLM extraction during ingestion and supports richer graph-completion queries.
Cognee configuration (LLM provider, database, vector store) is read from environment variables. See the [Cognee documentation](https://docs.cognee.ai) for setup instructions.
### Parameters
- `search_type` is *optional* and defaults to `"GRAPH_COMPLETION"`. Controls which Cognee recall strategy is used. Other useful values include `"CHUNKS"` for raw retrieval and `"SUMMARIES"` for summarized graph nodes.
- `top_k` is *optional* and defaults to `5`. Sets the default maximum number of memories returned per search.
- `dataset_name` is *optional* and defaults to `"haystack_memory"`. Names the Cognee dataset backing this store.
- `session_id` is *optional* and defaults to `None`. When set, reads and writes target the session-cache tier. When `None`, the permanent knowledge graph is used.
- `self_improvement` is *optional* and defaults to `True`. When `True`, Cognee runs graph improvement inline after every write. Set to `False` when you want `improve()` to be the sole improvement trigger.
- `timeout` is *optional* and defaults to `300`. Per-call timeout in seconds for any Cognee operation.
### Installation
Install the Cognee integration:
```bash
pip install cognee-haystack
```
Set your LLM API key (used by Cognee for graph extraction and queries):
```bash
export LLM_API_KEY="your-llm-api-key"
```
Optionally, set a separate embedding API key (defaults to `LLM_API_KEY` when unset):
```bash
export EMBEDDING_API_KEY="your-embedding-api-key"
```
## Usage
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.memory_stores.cognee import CogneeMemoryStore
store = CogneeMemoryStore(search_type="GRAPH_COMPLETION", top_k=5)
store.add_memories(
messages=[ChatMessage.from_user("Alice enjoys hiking and outdoor activities.")],
user_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890",
)
memories = store.search_memories(
query="What does Alice like?",
user_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890",
)
print([msg.text for msg in memories])
```
### Session tier vs permanent graph
Use `session_id` to control which memory tier is targeted. A single store can serve both tiers — the writer's `session_id` overrides the store's `session_id` per call.
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.memory_stores.cognee import CogneeMemoryStore
store = CogneeMemoryStore(dataset_name="my_agent_memory", self_improvement=False)
# Write long-lived facts to the permanent graph (no session_id).
store.add_memories(
messages=[ChatMessage.from_user("Alice is a senior data scientist at Acme Corp.")],
)
# Write transient session context to the session cache.
store.add_memories(
messages=[
ChatMessage.from_user("Alice is currently debugging a vector store issue.")
],
session_id="alice_session_1",
)
# Promote the session cache into the permanent graph.
store.improve(session_id="alice_session_1")
```
### Delete all memories
```python
# Delete only this store's dataset (session cache is unaffected).
store.delete_all_memories()
# To wipe everything including the session cache, call cognee directly:
import asyncio
import cognee
asyncio.run(cognee.forget(everything=True))
```
---
// File: memory-stores/mem0memorystore
# Mem0MemoryStore
`Mem0MemoryStore` is a memory store backed by the Mem0 cloud API. It is the shared data layer used by [`Mem0MemoryRetriever`](../pipeline-components/retrievers/mem0memoryretriever.mdx), [`Mem0MemoryWriter`](../pipeline-components/writers/mem0memorywriter.mdx), and the [Mem0 Memory Tools](../tools/ready-made-tools/mem0memorytools.mdx).
| | |
| --- | --- |
| **Used by** | [`Mem0MemoryRetriever`](../pipeline-components/retrievers/mem0memoryretriever.mdx), [`Mem0MemoryWriter`](../pipeline-components/writers/mem0memorywriter.mdx), [`Mem0MemoryRetrieverTool`, `Mem0MemoryWriterTool`](../tools/ready-made-tools/mem0memorytools.mdx) |
| **Optional init variables** | `api_key`: Defaults to `MEM0_API_KEY` environment variable |
| **API reference** | [Mem0](/reference/integrations-mem0#mem0memorystore) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mem0 |
| **Package name** | `mem0-haystack` |
## Overview
`Mem0MemoryStore` wraps the Mem0 cloud API and provides two core methods:
- `add_memories` — stores a list of `ChatMessage` objects as memories in Mem0.
- `search_memories` — retrieves memories from Mem0 that are relevant to a query.
Scope memories with at least one Mem0 entity ID: `user_id`, `run_id`, `agent_id`, or `app_id`. These are runtime parameters, so a single store instance can serve multiple users or sessions.
The `infer` parameter on `add_memories` controls how Mem0 processes incoming messages:
- `infer=True` lets Mem0 extract memories from the messages automatically. This is useful when storing a full Agent turn.
- `infer=False` stores the supplied message text as-is. This is useful when the exact memory text has already been selected upstream.
### Installation
Install the Mem0 integration:
```bash
pip install mem0-haystack
```
Set your Mem0 API key:
```bash
export MEM0_API_KEY="your-mem0-api-key"
```
## Usage
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.memory_stores.mem0 import Mem0MemoryStore
store = Mem0MemoryStore()
store.add_memories(
messages=[ChatMessage.from_user("Alice prefers concise Python examples.")],
user_id="alice",
infer=False,
)
memories = store.search_memories(
query="What does Alice prefer?",
user_id="alice",
top_k=3,
)
print([msg.text for msg in memories])
```
### Scoping with multiple entity IDs
Mem0 supports narrowing the scope of reads and writes with `user_id`, `run_id`, `agent_id`, and `app_id`. Pass any combination at call time:
```python
store.add_memories(
messages=[
ChatMessage.from_user("Alice is working on a documentation search system.")
],
user_id="alice",
run_id="docs-assistant-session-1",
infer=True,
)
memories = store.search_memories(
query="What project is Alice working on?",
user_id="alice",
run_id="docs-assistant-session-1",
)
print([msg.text for msg in memories])
```
### Retrieving all memories in scope
Pass `query=None` to return all memories matching the provided scope without a relevance search:
```python
all_memories = store.search_memories(query=None, user_id="alice")
print([msg.text for msg in all_memories])
```
---
// File: optimization/advanced-rag-techniques/hypothetical-document-embeddings-hyde
import ClickableImage from "@site/src/components/ClickableImage";
# Hypothetical Document Embeddings (HyDE)
Enhance the retrieval in Haystack using HyDE method by generating a mock-up hypothetical document for an initial query.
## When Is It Helpful?
The HyDE method is highly useful when:
- The performance of the retrieval step in your pipeline is not good enough (for example, low Recall metric).
- Your retrieval step has a query as input and returns documents from a larger document base.
- Particularly worth a try if your data (documents or queries) come from a special domain that is very different from the typical datasets that Retrievers are trained on.
## How Does It Work?
Many embedding retrievers generalize poorly to new, unseen domains. This approach tries to tackle this problem. Given a query, the Hypothetical Document Embeddings (HyDE) first zero-shot prompts an instruction-following language model to generate a “fake” hypothetical document that captures relevant textual patterns from the initial query - in practice, this is done five times. Then, it encodes each hypothetical document into an embedding vector and averages them. The resulting, single embedding can be used to identify a neighbourhood in the document embedding space from which similar actual documents are retrieved based on vector similarity. As with any other retriever, these retrieved documents can then be used downstream in a pipeline (for example, in a Generator for RAG). Refer to the paper “[Precise Zero-Shot Dense Retrieval without Relevance Labels](https://aclanthology.org/2023.acl-long.99/)” for more details.
Haystack supports many more model providers including **Cohere**, **Mistral**, **NVIDIA**, **Ollama**, and others—both cloud-hosted and local options.
Browse the full list of supported models and chat generators in the [Generators documentation](../pipeline-components/generators.mdx).
You can also explore all available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page.
Haystack supports many more model providers including **Cohere**, **Mistral**, **NVIDIA**, **Ollama**, and others—both cloud-hosted and local options.
Browse the full list of supported models and chat generators in the [Generators documentation](../pipeline-components/generators.mdx).
You can also explore all available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page.
| | |
| --- | --- |
| **Mandatory init variables** | `document_store`: The document store to inspect and fetch from
`retriever`: A relevance-scoring retriever or retrieval `Pipeline` | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../../concepts/data-classes/chatmessage.mdx)s | | **Output variables** | `last_message`: The answer, citing documents as `[doc]`
`documents`: Every document the agent retrieved, deduplicated | | **API reference** | [Agent Pack](/reference/integrations-agent-pack) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/agent_pack/src/haystack_integrations/agent_pack/advanced_rag | | **Package name** | `agent-pack-haystack` |
:::warning
Part of [Agent Pack](../agent-pack.mdx), which is experimental for the moment. Its APIs and agent architectures can change in any release, without following the usual deprecation policy.
:::
## When to use this agent
Use the advanced RAG agent when:
- You have a large, heterogeneous corpus (many topics, sources, or document types mixed together) with well-structured metadata. The agent uses that metadata to narrow retrieval, making results more precise than relevance ranking alone.
- You need to retrieve exact or complete subsets of documents by metadata ("all pages of this file", "everything from source X"), not just the most relevant matches.
It's less useful when:
- There's no metadata, or the metadata isn't useful for narrowing retrieval.
- The corpus is small and homogeneous, so plain top-k retrieval already returns the right documents.
## Installation
```shell
pip install agent-pack-haystack arrow
```
`arrow` (required with the default system prompt) renders today's date so the agent can build filters for relative dates like "the last 5 years".
Set `OPENAI_API_KEY` in the environment.
## Usage
Index a corpus with varied metadata and ask a question the agent can only answer well by inspecting the metadata, building a filter, and retrieving with it:
```python
from haystack import Document
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.dataclasses import ChatMessage
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.agent_pack import create_advanced_rag_agent
document_store = InMemoryDocumentStore()
document_store.write_documents(
[
Document(
content="CRISPR gene editing corrected a hereditary blindness mutation in a clinical trial.",
meta={"category": "science", "year": 2021, "rating": 4.6},
),
Document(
content="A quantum computer demonstrated error-corrected logical qubits.",
meta={"category": "science", "year": 2023, "rating": 4.8},
),
Document(
content="Dolly the sheep became the first mammal cloned from an adult somatic cell.",
meta={"category": "science", "year": 1996, "rating": 4.2},
),
Document(
content="The Berlin Wall fell, a decisive moment in the end of the Cold War.",
meta={"category": "history", "year": 1989, "rating": 4.7},
),
Document(
content="Argentina won the FIFA World Cup final against France on penalties.",
meta={"category": "sports", "year": 2022, "rating": 4.9},
),
],
)
agent = create_advanced_rag_agent(
document_store=document_store,
retriever=InMemoryBM25Retriever(document_store=document_store, top_k=5),
)
result = agent.run(
messages=[ChatMessage.from_user("What science advances happened after 2015?")],
)
print(result["last_message"].text) # the answer, citing documents as [doc `retriever`: A relevance-scoring retriever or retrieval `Pipeline` | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../../concepts/data-classes/chatmessage.mdx)s | | **Output variables** | `last_message`: The answer, citing documents as `[doc
`documents`: Every document the agent retrieved, deduplicated | | **API reference** | [Agent Pack](/reference/integrations-agent-pack) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/agent_pack/src/haystack_integrations/agent_pack/advanced_rag | | **Package name** | `agent-pack-haystack` |
| | |
| --- | --- |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../../concepts/data-classes/chatmessage.mdx)s |
| **Output variables** | `report`: The final cited Markdown report
`brief`, `notes`: The intermediate research brief and collected summaries | | **API reference** | [Agent Pack](/reference/integrations-agent-pack) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/agent_pack/src/haystack_integrations/agent_pack/deep_research | | **Package name** | `agent-pack-haystack` |
:::warning
Part of [Agent Pack](../agent-pack.mdx), which is experimental for the moment. Its APIs and agent architectures can change in any release, without following the usual deprecation policy.
:::
## When to use this agent
Use the deep research agent when you need more than a quick answer. It's designed for questions that require gathering information from many web sources, evaluating them, and producing a structured report with citations.
Typical use cases include:
- Researching a broad topic across many sources.
- Comparing products, companies, technologies, or scientific findings.
- Preparing a literature review or market overview.
- Answering complex questions that benefit from investigating several sub-topics in parallel.
It's less useful when:
- A single web search or RAG lookup is enough. The multi-agent workflow adds latency and cost.
- The information lives in a private knowledge base rather than on the public web. For that, use the [Advanced RAG Agent](./advanced-rag-agent.mdx).
## Installation
```shell
pip install agent-pack-haystack tavily-haystack trafilatura pypdf arrow
```
`tavily-haystack`, `trafilatura`, `pypdf`, and `arrow` are separate installs the deep research agent needs at runtime (web search, HTML and PDF parsing, and date rendering).
Set `OPENAI_API_KEY` and `TAVILY_API_KEY` in the environment.
## Usage
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.agent_pack import create_deep_research_agent
agent = create_deep_research_agent()
result = agent.run(messages=[ChatMessage.from_user("your research question")])
print(result["report"])
```
`agent.run(...)` returns a dictionary whose main output is `report`, the final Markdown report. The dictionary also carries the intermediate `brief` (a `str`) and `notes` (a `list[str]`), plus the standard [`Agent`](../agent.mdx) outputs `messages`, `last_message`, `step_count`, `token_usage`, and `tool_call_counts`.
## Configuration
Everything is configured through keyword arguments to `create_deep_research_agent`. All parameters are keyword-only and optional.
### Models
Each phase takes its own ChatGenerator, so you can mix models by cost and capability, or swap in a different provider.
- `scope_llm` is the LLM that rewrites the user query into a focused research brief. Defaults to `OpenAIResponsesChatGenerator("gpt-5.4")`.
- `orchestrator_llm` is the LLM that plans the investigation and delegates the sub-questions. Defaults to `OpenAIResponsesChatGenerator("gpt-5.4")`.
- `researcher_llm` is the LLM that drives each sub-researcher's search, read, and think loop. Defaults to `OpenAIResponsesChatGenerator("gpt-5.4-mini")`.
- `summarizer_llm` is the LLM used inside the `read_url` tool to summarize a fetched page toward the question. Defaults to `OpenAIResponsesChatGenerator("gpt-5.4-mini")`.
- `writer_llm` is the LLM that turns the brief plus collected notes into the final report. Defaults to `OpenAIResponsesChatGenerator("gpt-5.4")`.
### Breadth and depth
- `max_subtopics` is the maximum number of sub-questions the orchestrator may delegate (breadth). Defaults to `5`.
- `max_concurrent_researchers` is the maximum number of sub-researchers that run at the same time. Defaults to `5`.
- `max_orchestrator_steps` is the maximum number of steps for the orchestrator's agent loop (reflect and delegate rounds). Defaults to `8`.
- `max_researcher_steps` is the maximum number of steps for each sub-researcher's agent loop. Defaults to `20`.
### Search and reading
- `max_search_results` is the number of results returned per `web_search` call. Defaults to `10`.
- `max_content_length` is the maximum number of raw page characters fed to the summarizer, before summarization. Defaults to `50000`.
## How it works
The architecture is built around a single top-level Haystack [`Agent`](../agent.mdx), which acts as the orchestrator. Two [hooks](../hooks.mdx) run before and after its loop, creating three logical phases: Scope, Research, and Write. During the Research phase, the orchestrator invokes isolated sub-researcher agents, each its own `Agent`, through a tool:
- **Scope.** The user question is rewritten into a focused research brief.
- **Research.** The orchestrator splits the brief into focused sub-questions, delegates each to a sub-researcher, and collects their summaries.
- **Write.** The brief and the collected summaries become the final report: Markdown with inline `[text](url)` citations.
Scope and Write are plain LLM calls (a [`ChatPromptBuilder`](../../builders/chatpromptbuilder.mdx) and an [`OpenAIResponsesChatGenerator`](../../generators/openairesponseschatgenerator.mdx)), wrapped as serializable hook classes (`ScopeHook`, `WriteHook`):
- Scope runs as a `before_run` hook: before the orchestrator's loop starts, it turns the user query into a brief, stored on the agent's [`State`](../state.mdx).
- Write runs as an `after_run` hook: when the orchestrator's loop finishes, it turns the brief plus collected `notes` into the final report.
`brief`, `notes`, and `report` are declared in the agent's `state_schema`, so they come back as outputs of a single `agent.run(...)` call.
### The agents
The Research phase uses two nested agents. Each one is a Haystack `Agent`: an LLM that loops, calling tools, until it decides to answer.
#### Orchestrator
The orchestrator is the lead agent: it receives the research brief and coordinates the whole investigation.
- **Job:** split the brief into a few focused, non-overlapping sub-questions, delegate each one, check coverage, and stop when there's enough.
- **Parallelism:** it emits several delegation calls in a single turn, and they run concurrently (bounded by `max_concurrent_researchers`).
- **Memory:** the summaries returned by sub-researchers are appended to a shared `notes` list (the agent's `State`), which the writer later turns into the report.
- **Stops when:** it replies with plain text (research complete) or hits `max_orchestrator_steps`.
The orchestrator's tools:
| Tool | What it is | What it does |
| --- | --- | --- |
| `research_subtopic` | The sub-researcher agent, exposed as a tool ([`ComponentTool`](../../../tools/componenttool.mdx)) | Researches a single sub-question in an isolated context and returns a compressed, cited summary. Only that summary is shown to the orchestrator; the summary is also appended to `notes`. |
| `think_tool` | A no-op reflection tool | Lets the orchestrator pause to plan sub-questions and assess coverage between rounds. |
#### Sub-researcher
The sub-researcher is a reusable agent that answers a single sub-question. The orchestrator runs it many times in parallel, each in its own isolated context. This is the key idea: each sub-researcher processes the raw search results privately and returns only a concise summary, so the orchestrator's context stays small and the final report stays coherent.
- **Job:** search the web, optionally read promising pages, reflect, then write a compressed summary with inline citations to the exact source URLs.
- **Returns:** its final text message *is* the summary (it exits as soon as it writes plain text).
- **Bounded by:** `max_researcher_steps`.
The sub-researcher's tools:
| Tool | What it is | What it does |
| --- | --- | --- |
| `web_search` | [`TavilyWebSearchTool`](../../../tools/ready-made-tools/tavilywebsearchtool.mdx) from the Tavily integration | Runs a web search and returns the top results as title, exact URL, and snippet. |
| `read_url` | [`PipelineTool`](../../../tools/pipelinetool.mdx) over a fetch, route, convert-to-text, and summarize pipeline | Fetches a page (`LinkContentFetcher`), routes by MIME type (`FileTypeRouter`) to `HTMLToDocument` (Trafilatura) or `PyPDFToDocument` so PDFs are parsed too, and summarizes the page toward a question the agent passes, so only the relevant text enters the agent's context, not the full page. Used only when a search snippet is too shallow. |
| `think_tool` | A no-op reflection tool | "What did I learn? What's missing? Stop or continue?" between searches. |
### Context management
The core challenge in a deep research agent is keeping each context window small and focused. Raw web content (search results, full pages, PDFs) is large and noisy. If it all accumulated in a single context, the model's output quality would degrade. We avoid that with isolation and compression:
- Each sub-researcher runs as its own agent with its own `State`, so all the messy intermediate content (every search result, every fetched page) stays in *its* private context.
- It finishes by writing one short summary (its final message). Only that summary leaves the sub-researcher: the raw content never reaches the orchestrator or the writer.
Two settings on the `research_subtopic` tool decide where that summary goes:
| Setting (on `research_subtopic`) | Controls | Effect |
| --- | --- | --- |
| `outputs_to_string={"source": "last_message"}` | What the orchestrator's LLM sees as the tool result | Only the summary text comes back, not the sub-researcher's full message history. Keeps the orchestrator's context clean. |
| `outputs_to_state={"notes": {...}}` | What gets saved for the writer | The same summary is appended (as text) to the shared `notes` list, which becomes the writer's input. |
So each summary travels two ways, into the orchestrator's reasoning (so it can decide whether to dig further) and into the `notes` accumulator (so the writer can use it), while the bulky raw research stays isolated and is not propagated beyond the sub-researcher:
```
sub-researcher (private context: searches, pages, reflections)
│ writes one short summary
├─ outputs_to_string → orchestrator's LLM (decide: done, or dig more?)
└─ outputs_to_state → notes → writer (final report)
```
---
// File: pipeline-components/agents-1/agent-pack
# Agent Pack
Agent Pack is a collection of complex, pre-configured Haystack agents you can run as they are, customize, or copy as a blueprint for your own architecture.
`brief`, `notes`: The intermediate research brief and collected summaries | | **API reference** | [Agent Pack](/reference/integrations-agent-pack) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/agent_pack/src/haystack_integrations/agent_pack/deep_research | | **Package name** | `agent-pack-haystack` |
| | |
| --- | --- |
| **API reference** | Agent Pack |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/agent_pack |
| **Package name** | `agent-pack-haystack` |
## Overview
Language Models and Tools are the core building blocks of agents.
However, building a robust agent often requires more: an architecture that splits the work across sub-agents, techniques for keeping context small but focused, and ways to influence and interact with the agent loop.
Agent Pack combines these techniques into working agents. Each one is a complete architecture built from Haystack primitives ([`Agent`](./agent.mdx), [Tools](../../tools/tool.mdx), [hooks](./hooks.mdx), [`State`](./state.mdx), and Pipelines), exposed behind a single `create_*` entry point.
There are three ways to use an agent from the pack:
- **Run it as is.** Each agent has a factory that builds a ready-to-run agent with defaults chosen to work out of the box. For example, `create_deep_research_agent()` returns an agent that takes a question and produces a report.
- **Customize it.** Each entry point exposes parameters for changing models, adding tools, and configuring behavior specific to that type of agent. (*This interface is still being refined to make customization more consistent and flexible.*)
- **Copy it.** Read the implementation and take inspiration for your agents. These agents are built with this use case in mind, so individual parts should be easy to adapt.
In other words, you can use the agents in Agent Pack as either ready-made solutions or reference architectures.
## Installation
```shell
pip install agent-pack-haystack
```
Each agent has its own additional runtime dependencies and may require API keys. See its documentation page for these details.
## Available agents
| Agent | Description |
| --- | --- |
| [Deep Research Agent](agent-pack/deep-research-agent.mdx) | Researches a question on the web and produces a structured Markdown report with citations. |
| [Advanced RAG Agent](agent-pack/advanced-rag-agent.mdx) | Inspects document-store metadata and builds Haystack filters to retrieve precisely, then answers with citations. |
## Experimental
:::warning
Agent Pack is experimental for the moment. Its APIs and agent architectures can change in any release, without following the usual deprecation policy.
:::
Agent Pack is distributed separately from `haystack-ai` and its code lives in `haystack-core-integrations`.
Agentic architectures are evolving fast. Keeping it outside `haystack-ai` allows us to improve these agents rapidly, release updates independently, and avoid committing to stability before these patterns have settled.
Developing complex agents also helps us identify missing capabilities in Haystack. When a gap shows up, we can implement it in the pack first, and then, if it turns out to be generally useful, refine it and move it into Haystack.
## Feedback
If you find a bug or have an idea for a complex agent that could belong in the pack, [open an issue](https://github.com/deepset-ai/haystack-core-integrations/issues).
---
// File: pipeline-components/agents-1/agent
# Agent
The `Agent` component is a tool-using agent that interacts with chat-based LLMs and tools to solve complex queries iteratively. It can execute external tools, manage state across multiple LLM calls, and stop execution based on configurable `exit_conditions`.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or user input |
| **Mandatory init variables** | `chat_generator`: An instance of a Chat Generator that supports tools |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)s |
| **Output variables** | `messages`: Chat history with tool and model responses
`last_message`: The final `ChatMessage` of the run
`step_count`, `token_usage`, `tool_call_counts`, `exit_reason`: Run metadata
Plus one output per key defined in `state_schema` | | **API reference** | [Agents](/reference/agents-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/agents/agent.py | | **Package name** | `haystack-ai` |
## Overview
The `Agent` component is a loop-based system that uses a chat-based large language model (LLM) and external tools to solve complex user queries.
It works iteratively—calling tools, updating state, and generating prompts—until one of the configurable `exit_conditions` is met.
It can:
- Dynamically select tools based on user input,
- Maintain and validate runtime state using a schema,
- Stream token-level outputs from the LLM.
The `Agent` returns a dictionary containing:
- `messages`: the full conversation history,
- `last_message`: the final `ChatMessage` from the agent,
- `step_count`: the number of steps the agent ran,
- `token_usage`: aggregated token usage summed across every LLM call in the run,
- `tool_call_counts`: how many times each tool was invoked, keyed by tool name,
- `exit_reason`: why the agent stopped, useful for routing its output downstream,
- Additional dynamic keys based on `state_schema`.
### Run Metadata
The `step_count`, `token_usage`, `tool_call_counts`, and `exit_reason` outputs are populated automatically during a run. They are added to the agent's `state_schema` behind the scenes, so tools registered with `inputs_from_state` and [hooks](./hooks.mdx) can read them from the live `State`. They are outputs only — they cannot be passed as inputs to `run()` or `run_async()`, and using them as keys in your own `state_schema` raises a `ValueError`. See [State](./state.mdx#schema-definition) for details.
```python
response = agent.run(messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")])
print(response["step_count"]) # 2
print(
response["token_usage"],
) # {"prompt_tokens": 512, "completion_tokens": 86, ...}
print(response["tool_call_counts"]) # {"calculator": 1}
print(response["exit_reason"]) # "text"
```
### Exit reason
The `exit_reason` output tells you why the agent stopped, which makes it easy to route the agent's output downstream — for example, with a [`ConditionalRouter`](../routers/conditionalrouter.mdx). It is one of:
- `"text"`: the model returned a reply with no tool calls.
- the name of the tool that satisfied a tool exit condition. In this case `last_message` is that tool's result — a tool-result `ChatMessage` whose `text` is empty — so `exit_reason` tells you how to consume it.
- `"max_agent_steps"`: the agent reached `max_agent_steps` before meeting an exit condition.
Because `exit_reason` is available on the live `State`, an `after_run` [hook](./hooks.mdx) can read it to react to how the run ended — for example, appending a fallback answer when the step budget is exhausted before the agent finished:
```python
from haystack.components.agents.state import State
from haystack.dataclasses import ChatMessage
from haystack.hooks import hook
@hook
def fallback_on_max_steps(state: State) -> None:
if state.get("exit_reason") == "max_agent_steps":
state.set(
"messages",
[ChatMessage.from_assistant("Sorry, I ran out of steps before finishing.")],
)
```
## Parameters
`chat_generator` is the only mandatory parameter — an instance of a Chat Generator that supports tools. All other parameters are optional.
- `tools`: A list of tool or toolset instances the agent can call. Supported types: [`Tool`](../../tools/tool.mdx), [`ComponentTool`](../../tools/componenttool.mdx), [`PipelineTool`](../../tools/pipelinetool.mdx), [`AgentTool`](../../tools/agenttool.mdx), [`MCPTool`](../../tools/mcptool.mdx), [`Toolset`](../../tools/toolset.mdx), [`MCPToolset`](../../tools/mcptoolset.mdx), [`SearchableToolset`](../../tools/searchabletoolset.mdx). Tool names must be unique; duplicate names are detected at the start of each agent step, before the chat generator is called.
- `system_prompt`: A plain string or Jinja2 template used as the system message for every run. If the template contains Jinja2 variables, those variables become additional inputs to `run()`.
- `user_prompt`: A Jinja2 template appended to the user-provided messages on each run. Template variables become additional inputs to `run()`. Use `required_variables` to enforce which variables must be provided.
- `exit_conditions`: List of conditions that cause the agent to stop. Use `”text”` to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed. Defaults to `[“text”]`. Exit conditions are evaluated at runtime rather than validated at initialization, so a condition can name a tool that is only loaded later — for example, a tool passed at runtime via `run(tools=...)` or one discovered by a [`SearchableToolset`](../../tools/searchabletoolset.mdx).
- `state_schema`: Defines the agent's runtime state — a dict mapping key names to type configs (e.g. `{“docs”: {“type”: list[Document]}}`). Tools can read from and write to state keys via `inputs_from_state` and `outputs_to_state`. See [State](./state.mdx) for full details.
- `streaming_callback`: A callback invoked for each streamed token. Use the built-in `print_streaming_chunk` for console output.
- `max_agent_steps`: Maximum number of LLM + tool call iterations before the agent stops. Defaults to `100`.
- `raise_on_tool_invocation_failure`: If `True`, raises an exception when a tool call fails. If `False` (default), the error is passed back to the LLM as a message so it can recover.
- `hooks`: A dict mapping a hook point (`"before_run"`, `"before_llm"`, `"before_tool"`, `"after_tool"`, `"on_exit"`, `"after_run"`) to a list of hooks the agent runs at that point. Hooks receive the live `State` and influence the run by mutating it — for example, to build run-time context or require human confirmation of tool calls. See [Hooks](./hooks.mdx) and [Human in the Loop](./human-in-the-loop.mdx).
- `tool_concurrency_limit`: Maximum number of tool calls to execute at the same time. Defaults to `4`; set to `1` to disable parallel tool execution.
- `tool_streaming_callback_passthrough`: If `True`, passes the streaming callback to tools that accept it.
### Runtime overrides
`run()` also accepts parameters that override the init-time configuration for a single call:
- `tools`: Pass a list of `Tool`/`Toolset` objects, or a list of tool name strings to select a subset of the agent's configured tools for this run.
- `generation_kwargs`: Additional keyword arguments forwarded to the LLM, overriding any set at init time (e.g. `{“temperature”: 0.2}`).
- `hook_context`: A dict of request-scoped resources made available to [hooks](./hooks.mdx) via `state.data["hook_context"]` — for example, a user ID or a WebSocket connection.
:::info
For the full parameter reference, see the [Agents API Documentation](/reference/agents-api).
:::
### Cloning and modifying an agent
Agent attributes are not meant to be reassigned after initialization: several parameters are processed at init time, so setting an attribute on a built agent does not reliably take effect. The recommended way to get a modified version of an existing agent is `clone()`.
`clone()` returns a new agent with the same configuration, optionally replacing some init parameters. This is useful for creating a variant of an agent you did not build yourself, such as one returned by a factory function of the [Agent Pack](./agent-pack.mdx).
```python
variant = agent.clone(system_prompt="Answer in German.", max_agent_steps=20)
```
Overrides replace the original values. To extend a list or dictionary instead, unpack the existing value and add your entries:
```python
extended = agent.clone(
tools=[*agent.tools, my_new_tool],
state_schema={**agent.state_schema, "notes": {"type": str}},
hooks={**agent.hooks, "before_llm": [my_hook]},
)
```
## Usage
### On its own
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import tool
from haystack.components.agents import Agent
from typing import Annotated
@tool(outputs_to_state={"calc_result": {"source": "result"}})
def calculator(
expression: Annotated[str, "Math expression to evaluate, e.g. '7 * (4 + 2)'"],
) -> dict:
"""Evaluate basic math expressions."""
try:
result = eval(expression, {"__builtins__": {}})
return {"result": result}
except Exception as e:
return {"error": str(e)}
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[calculator],
system_prompt="You are a helpful assistant. Always use the calculator tool to evaluate math expressions.",
state_schema={"calc_result": {"type": int}},
)
response = agent.run(messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")])
print(response["last_message"].text)
print("Calc Result:", response.get("calc_result"))
```
### In a pipeline
The example pipeline below creates a database assistant using `OpenAIChatGenerator`, `LinkContentFetcher`, and custom database tool.
It reads the given URL and processes the page content, then builds a prompt for the AI.
The assistant uses this information to write people's names and titles from the given page to the database.
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.converters.html import HTMLToDocument
from haystack.components.fetchers.link_content import LinkContentFetcher
from haystack import Document, Pipeline
from haystack.dataclasses import ChatMessage
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.tools import tool
from typing import Annotated, Optional
document_store = InMemoryDocumentStore() # create a document store or an SQL database
@tool
def add_database_tool(
name: Annotated[str, "First name of the person"],
surname: Annotated[str, "Last name of the person"],
job_title: Annotated[Optional[str], "Job title or role of the person"] = None,
other: Annotated[Optional[str], "Any other relevant information"] = None,
) -> str:
"""Add a person to the database with information about them."""
document_store.write_documents(
[
Document(
content=name + " " + surname + " " + (job_title or ""),
meta={"other": other},
),
],
)
# Returning a confirmation lets the agent know the tool call succeeded
return f"Successfully added {name} {surname} to the database."
database_assistant = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[add_database_tool],
system_prompt="""
You are a database assistant.
Your task is to extract the names of people mentioned in the given context and add them to a knowledge base,
along with additional relevant information about them that can be extracted from the context.
Do not use your own knowledge, stay grounded to the given context.
Do not ask the user for confirmation.
Instead, automatically update the knowledge base and return a brief summary of the people added,
including the information stored for each.
""",
)
extraction_agent = Pipeline()
extraction_agent.add_component("fetcher", LinkContentFetcher())
extraction_agent.add_component("converter", HTMLToDocument())
extraction_agent.add_component(
"builder",
ChatPromptBuilder(
template=[
ChatMessage.from_user("""
{% for doc in docs %}
{{ doc.content|default|truncate(25000) }}
{% endfor %}
"""),
],
required_variables=["docs"],
),
)
extraction_agent.add_component("database_agent", database_assistant)
extraction_agent.connect("fetcher.streams", "converter.sources")
extraction_agent.connect("converter.documents", "builder.docs")
extraction_agent.connect("builder", "database_agent")
agent_output = extraction_agent.run(
{
"fetcher": {
"urls": ["https://github.com/deepset-ai/haystack/releases/tag/v2.27.0"],
},
},
)
print(agent_output["database_agent"]["last_message"].text)
# Inspect what was written to the document store
written_docs = document_store.filter_documents()
print(f"\n{len(written_docs)} people added to the database:")
for doc in written_docs:
print(f" - {doc.content}")
```
### In YAML
The example pipeline below fetches a webpage, converts its HTML to text, and builds a chat prompt combining the page content with a user query.
The `Agent` then answers the question based on the provided content and can use its web search tool to find additional information if needed.
`last_message`: The final `ChatMessage` of the run
`step_count`, `token_usage`, `tool_call_counts`, `exit_reason`: Run metadata
Plus one output per key defined in `state_schema` | | **API reference** | [Agents](/reference/agents-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/agents/agent.py | | **Package name** | `haystack-ai` |
View YAML
```yaml components: agent: init_parameters: chat_generator: init_parameters: api_base_url: null api_key: env_vars: - OPENAI_API_KEY strict: true type: env_var generation_kwargs: {} http_client_kwargs: null max_retries: null model: gpt-5.4-nano organization: null streaming_callback: null timeout: null tools: null tools_strict: false type: haystack.components.generators.chat.openai.OpenAIChatGenerator exit_conditions: - text hooks: null max_agent_steps: 5 raise_on_tool_invocation_failure: false required_variables: null state_schema: {} streaming_callback: null system_prompt: You are a helpful assistant. Use the web search tool to find information when needed. tool_concurrency_limit: 4 tool_streaming_callback_passthrough: false tools: - data: component: init_parameters: allowed_domains: null api_key: env_vars: - SERPERDEV_API_KEY strict: true type: env_var exclude_subdomains: false search_params: {} top_k: 3 type: haystack_integrations.components.websearch.serperdev.websearch.SerperDevWebSearch description: Search the web for current information on any topic inputs_from_state: null name: web_search outputs_to_state: null outputs_to_string: null parameters: null type: haystack.tools.component_tool.ComponentTool user_prompt: null type: haystack.components.agents.agent.Agent converter: init_parameters: extraction_kwargs: {} store_full_path: false type: haystack.components.converters.html.HTMLToDocument fetcher: init_parameters: client_kwargs: follow_redirects: true timeout: 3 http2: false raise_on_failure: true request_headers: {} retry_attempts: 2 timeout: 3 user_agents: - haystack/LinkContentFetcher/2.27.0rc0 type: haystack.components.fetchers.link_content.LinkContentFetcher prompt_builder: init_parameters: required_variables: - docs - query template: - content: - text: 'Based on the following content: {% for doc in docs %} {{ doc.content }} {% endfor %} Answer this question: {{ query }}' meta: {} name: null role: user variables: null type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder connection_type_validation: true connections: - receiver: converter.sources sender: fetcher.streams - receiver: prompt_builder.docs sender: converter.documents - receiver: agent.messages sender: prompt_builder.prompt max_runs_per_component: 100 metadata: {} ```
| | |
| --- | --- |
| **Configured on** | The [`Agent`](../agent.mdx) component under the `before_llm` [hook point](../hooks.mdx) |
| **Mandatory init variables** | `compactor`: The strategy used to shorten the messages
`context_window`: The model's context-window size in tokens | | **Import path** | `haystack.hooks.compaction.CompactionHook` | | **API reference** | [Hooks](/reference/hooks-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/compaction/hooks.py | | **Package name** | `haystack-ai` |
## Usage
Register the hook under `before_llm` and configure the context window of the Agent's model:
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.hooks.compaction import CompactionHook, SlidingWindowCompactor
compaction_hook = CompactionHook(
compactor=SlidingWindowCompactor(),
context_window=400_000, # gpt-5.4-nano's context window
compact_at=0.7,
compact_to=0.4,
)
agent = Agent(
chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4-nano"),
tools=[],
hooks={"before_llm": [compaction_hook]},
max_agent_steps=50,
)
```
`CompactionHook` can only be registered under `before_llm`. The Agent raises a `ValueError` if you register it at another hook point.
## Configuration
| Parameter | Default | Description |
| --- | --- | --- |
| `compactor` | No default | A `Compactor` implementation that decides how to shorten the messages. |
| `context_window` | No default | The model's full context-window size in tokens. It must be greater than zero. |
| `compact_at` | `0.7` | The fraction of the context window at which compaction starts. Leave enough space above it for the next model response and tool results. |
| `compact_to` | `0.4` | The fraction of the context window that compaction targets. A lower value compacts less often but removes more context each time. |
| `token_counter` | `ApproximateTokenCounter()` | The counter used to estimate messages not yet included in provider-reported usage. |
The thresholds must satisfy `0 < compact_to < compact_at <= 1`. A target at or above the trigger would leave the conversation ready to compact again on the next step, so the hook rejects that configuration.
## How the hook measures context
After an LLM call, the Agent stores the generator's reported prompt-plus-completion usage in `state.data["context_tokens"]`. This count includes the system prompt, tool schemas, and provider-specific chat-template overhead. Messages appended since that call, typically tool results, are measured locally with the configured [`TokenCounter`](../../../token-counters.mdx).
If the generator does not report usage and `context_tokens` remains `0`, the hook estimates the complete conversation and tool schemas locally. The default `ApproximateTokenCounter` needs no extra dependency. You can provide another built-in or custom counter:
```python
from haystack.hooks.compaction import CompactionHook, SlidingWindowCompactor
from haystack.token_counters import TiktokenCounter
compaction_hook = CompactionHook(
compactor=SlidingWindowCompactor(),
context_window=128_000,
token_counter=TiktokenCounter(encoding="o200k_base"),
)
```
The hook subtracts estimated non-message overhead from the target passed to the compactor. This prevents the compactor from treating tool schemas or provider formatting as message tokens it can remove.
## Choosing a compactor
The compactor controls what information survives:
| Compactor | Strategy |
| --- | --- |
| [`SlidingWindowCompactor`](sliding-window-compactor.mdx) | Keeps the current task and as much complete recent conversation as fits, removing complete historical turns before it trims the task's own steps. |
| [`ToolResultPruningCompactor`](tool-result-pruning-compactor.mdx) | Replaces older, large tool results with short placeholders while keeping recent results intact. |
You can also implement the `Compactor` protocol for a custom strategy. See [Context Compaction](../compaction.mdx#creating-a-custom-compactor) for its requirements.
## Lifecycle and serialization
The hook warms up its token counter and compactor when they provide a `warm_up` method, and delegates `close` to the compactor when supported. Its asynchronous lifecycle methods prefer the compactor's async implementation when one exists.
`to_dict()` serializes the hook together with its compactor and token counter. `from_dict()` reconstructs both nested objects, so an Agent configured with the hook can be serialized and restored.
---
// File: pipeline-components/agents-1/compaction/sliding-window-compactor
# SlidingWindowCompactor
`SlidingWindowCompactor` removes older conversation history while preserving the Agent's instructions, current task, and as much complete recent conversation as the token target allows. It removes complete historical turns first, and only trims the current task's own Agent steps when removing every historical turn is not enough.
:::warning[Experimental]
`SlidingWindowCompactor` is experimental and may change without a deprecation cycle. Compaction is lossy: messages removed by this strategy cannot be recovered or summarized.
:::
`context_window`: The model's context-window size in tokens | | **Import path** | `haystack.hooks.compaction.CompactionHook` | | **API reference** | [Hooks](/reference/hooks-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/compaction/hooks.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Used by** | [`CompactionHook`](compaction-hook.mdx) |
| **Mandatory init variables** | None |
| **Import path** | `haystack.hooks.compaction.SlidingWindowCompactor` |
| **API reference** | [Hooks](/reference/hooks-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/compaction/sliding_window.py |
| **Package name** | `haystack-ai` |
## Usage
Pass the compactor to a `CompactionHook`:
```python
from haystack.hooks.compaction import CompactionHook, SlidingWindowCompactor
compaction_hook = CompactionHook(
compactor=SlidingWindowCompactor(
min_keep_steps=1,
omission_note=(
"[{num_removed} earlier messages were removed to free up context.]"
),
),
context_window=200_000,
compact_at=0.7,
compact_to=0.4,
)
```
`CompactionHook` determines when compaction runs and provides the target token count. `SlidingWindowCompactor` determines which messages to retain.
## How the sliding window is selected
The compactor divides a conversation into protected context, historical turns, and the current task's Agent steps:
1. It preserves all leading system messages as the Agent's instructions.
2. It preserves the latest user message as the current task.
3. It groups the history before that task into complete historical turns, each running from one user message up to the next.
4. It groups each assistant message and all immediately following tool-result messages into one complete Agent step.
5. Working backwards from the newest, it keeps as many complete historical turns as fit within the target.
6. Only when the current task alone still exceeds the target does it begin removing that task's own steps, oldest first.
7. It replaces what it removed with an omission note, unless the note is disabled.
Keeping complete steps ensures that an assistant tool call is not separated from its results, including batches of parallel tool calls. Incomplete tool-call exchanges are rejected by chat-completion providers. Historical turns are kept or removed in full for the same reason: an assistant reply is never retained without the user message it answers.
The target is a goal rather than a guarantee, and the conversation can end up above it rather than below. Leading system messages and the current task are never removed, and `min_keep_steps` holds on to the newest Agent steps whatever their size, so a long system prompt or a single large tool result can leave the conversation well over the target.
## Configuration
| Parameter | Default | Description |
| --- | --- | --- |
| `min_keep_steps` | `1` | The minimum number of complete recent Agent steps to preserve, even if they exceed the target. Set it to `0` to allow all completed steps to be removed. |
| `omission_note` | `"[{num_removed} earlier messages were removed from this conversation to free up context and cannot be recovered.]"` | A user message inserted where history was removed. Use `{num_removed}` to include the number of removed messages, provide custom text without the placeholder, or set it to `None` to remove history silently. |
`min_keep_steps` cannot be negative.
### Omission notes
An omission note tells the model that earlier context is missing. Without one, the shortened conversation can appear complete and the model may repeat work or behave as though it still has the removed information.
The note is left where the removed messages used to sit: directly after the leading system messages when only historical turns were removed, and directly after the latest user message when the current task's own steps were removed. Repeated compactions fold an earlier note into the new one, so the conversation carries at most one.
Compaction metadata is stored on the note, including the strategy name and the numbers of removed and retained messages.
## When the conversation is unchanged
The compactor returns `None` without changing the conversation when:
- The conversation already fits within `target_tokens`.
- There is no removable history outside the protected messages and the history it retained.
---
// File: pipeline-components/agents-1/compaction/tool-result-pruning-compactor
# ToolResultPruningCompactor
`ToolResultPruningCompactor` reduces an Agent's context by replacing older tool results with short placeholders. It keeps every tool call paired with a result, allowing the model to see which tool it called and call it again if needed.
:::warning[Experimental]
`ToolResultPruningCompactor` is experimental and may change without a deprecation cycle. Pruning is lossy: removed tool output cannot be recovered unless it was stored separately.
:::
| | |
| --- | --- |
| **Used by** | [`CompactionHook`](compaction-hook.mdx) |
| **Mandatory init variables** | None |
| **Import path** | `haystack.hooks.compaction.ToolResultPruningCompactor` |
| **API reference** | [Hooks](/reference/hooks-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/compaction/tool_result_pruning.py |
| **Package name** | `haystack-ai` |
## Usage
Pass the compactor to a `CompactionHook`:
```python
from haystack.hooks.compaction import CompactionHook, ToolResultPruningCompactor
compaction_hook = CompactionHook(
compactor=ToolResultPruningCompactor(
min_keep_steps=1,
min_tokens=200,
),
context_window=200_000,
compact_at=0.7,
compact_to=0.4,
)
```
`CompactionHook` determines when compaction runs and provides the target token count. `ToolResultPruningCompactor` replaces only as many eligible results as needed to reach that target.
## How pruning works
The compactor processes tool results from oldest to newest:
1. It leaves the conversation unchanged when it already fits within the target.
2. It protects results from at least the configured number of recent tool-calling Agent steps. Since at least one step must be kept, the current result batch remains intact until the model has acted on it.
3. It skips results already marked by context compaction, results below the token threshold, and results carrying protected metadata such as an offloaded-result reference.
4. It replaces eligible results with the configured placeholder until the estimated conversation size reaches the target.
The replacement preserves the originating tool call, error flag, and message metadata. This keeps the tool-call exchange valid for chat-completion providers while discarding the expensive result content.
The target is a goal rather than a guarantee. Protected recent results and results that do not meet the pruning rules can leave the conversation above the requested target.
## Configuration
| Parameter | Default | Description |
| --- | --- | --- |
| `min_keep_steps` | `1` | The minimum number of recent tool-calling Agent steps whose results remain intact regardless of the target. Parallel results from one step are protected together. |
| `min_tokens` | `200` | Only prune a tool-result message when it uses more than this many tokens, as measured by the configured token counter. |
| `placeholder` | ``"[Tool result removed to free up context. Call `{tool_name}` again if you need it.]"`` | Text that replaces a pruned result. Use `{tool_name}` to insert the originating tool's name. Other braces are preserved literally. |
| `skip_meta_keys` | `("tool_result_offloaded",)` | Leave a result unchanged when its metadata contains any listed key. The default protects pointers created by `ToolResultOffloadHook`. |
`min_keep_steps` must be at least `1`, and `min_tokens` cannot be negative.
### Custom placeholders
Keep custom placeholders short so replacing a result saves context. If a placeholder costs at least as many tokens as the original result, the compactor leaves that result unchanged.
```python
compactor = ToolResultPruningCompactor(
placeholder="Previous output from {tool_name} was removed. Call the tool again if needed.",
)
```
The compactor records `context_compaction` metadata on each rewritten result with the strategy name and the original tool-result message's token count.
## Token counting
The compactor uses the [`TokenCounter`](../../../token-counters.mdx) supplied by `CompactionHook` to determine whether a result exceeds `min_tokens` and whether replacing it saves context.
The complete conversation is counted once. For each eligible result, the counter then measures only the original result message and its short replacement. The compactor updates its running total using the difference between those two counts.
## Interaction with tool result offloading
[`ToolResultOffloadHook`](../tool-result-offloading.mdx) stores a tool result outside the conversation and replaces it with a reference. Pruning that reference would prevent the model from retrieving the stored content.
By default, `ToolResultPruningCompactor` skips messages with `tool_result_offloaded` metadata. These pointer messages are already small, so pruning them would save little context while removing the model's only reference to the full stored result. Add other metadata keys to `skip_meta_keys` when another hook or application feature leaves references that must remain available.
## When the conversation is unchanged
The compactor returns `None` without changing the conversation when:
- The conversation already fits within `target_tokens`.
- There are no results older than the protected steps.
- Every older result is already compacted, protected by metadata, or no larger than `min_tokens`.
- A replacement would not reduce the result's measured token count.
---
// File: pipeline-components/agents-1/compaction
# Context Compaction
Context compaction shortens an Agent's conversation so long runs do not exhaust the model's context window. It rewrites older history into a smaller representation while preserving the context the Agent needs to continue working.
Compaction is lossy. After it runs, the Agent works from a shorter record of the conversation. What survives and what is discarded depends on the compaction strategy.
## How context compaction works
Context compaction separates three responsibilities:
| Responsibility | Abstraction | Purpose |
| --- | --- | --- |
| Decide when to compact | [`CompactionHook`](compaction/compaction-hook.mdx) | Monitors the Agent's context before LLM calls and invokes a compactor after a configured threshold is reached. |
| Decide how to compact | `Compactor` protocol | Defines how a conversation is rewritten. Haystack includes [`SlidingWindowCompactor`](compaction/sliding-window-compactor.mdx) and [`ToolResultPruningCompactor`](compaction/tool-result-pruning-compactor.mdx). |
| Measure the conversation | [`TokenCounter`](../../token-counters.mdx) protocol | Estimates the size of messages and tool schemas before they are sent to a model. |
This separation lets you combine a standard trigger with different compaction strategies and token counters. For example, a local sliding window can remove old history without making an additional model call, while a custom compactor could summarize the same history with an LLM.
## Basic setup
The following example registers a `CompactionHook` under the Agent's `before_llm` [hook point](./hooks.mdx). It starts compacting at 70% of the model's context window and asks the compactor to reduce the context to approximately 40%.
```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.hooks.compaction import CompactionHook, SlidingWindowCompactor
from haystack.tools import tool
@tool
def fetch_page(url: Annotated[str, "The URL to fetch"]) -> str:
"""Fetch a web page and return its text."""
return "Fusion startups reported net-energy-gain milestones this year. " * 500
compaction_hook = CompactionHook(
compactor=SlidingWindowCompactor(),
context_window=400_000, # gpt-5.4-nano's context window
compact_at=0.7,
compact_to=0.4,
)
agent = Agent(
chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4-nano"),
tools=[fetch_page],
system_prompt="You are a research assistant. Fetch pages as needed and cite what you used.",
hooks={"before_llm": [compaction_hook]},
)
result = agent.run(
messages=[ChatMessage.from_user("Summarize recent fusion energy milestones.")],
)
print(result["last_message"].text)
```
See [`CompactionHook`](compaction/compaction-hook.mdx) for threshold configuration, context measurement, lifecycle, and serialization.
## Compaction strategies
Compactors receive the current messages, a target token count, and the same token counter used to measure the context. They return a shorter replacement conversation or `None` when there is nothing useful to change.
| Compactor | Strategy | Trade-off |
| --- | --- | --- |
| [`SlidingWindowCompactor`](compaction/sliding-window-compactor.mdx) | Preserves the Agent's instructions and latest user task, keeps complete historical turns while they fit, and trims the current task's own Agent steps only when that is not enough. | Fast and local, but discarded information is not summarized. |
| [`ToolResultPruningCompactor`](compaction/tool-result-pruning-compactor.mdx) | Replaces older, large tool results with short placeholders while preserving tool-call/result structure. | Retains the shape of the run and recent results, but removes the content of pruned results. |
## Combining compaction strategies
Register multiple `CompactionHook` instances at `before_llm` to apply progressively more aggressive strategies. Hooks run in list order against the same Agent state, so each hook measures the messages left by the previous one.
For example, prune large tool results first and use a sliding window as a fallback:
```python
from haystack.hooks.compaction import (
CompactionHook,
SlidingWindowCompactor,
ToolResultPruningCompactor,
)
prune_tool_results = CompactionHook(
compactor=ToolResultPruningCompactor(),
context_window=400_000,
compact_at=0.7,
compact_to=0.4,
)
drop_old_steps = CompactionHook(
compactor=SlidingWindowCompactor(),
context_window=400_000,
compact_at=0.7,
compact_to=0.4,
)
agent = Agent(
chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4-nano"),
tools=[fetch_page],
hooks={"before_llm": [prune_tool_results, drop_old_steps]},
)
```
If pruning brings the updated context below `compact_at`, the sliding-window hook does nothing. If pruning returns `None` because no eligible results remain, or it shortens the context without getting below the trigger, the sliding window removes historical turns and then, if needed, the current task's oldest Agent steps. A result the pruning compactor already replaced with a placeholder stays with the historical turn it belongs to, so its tool call keeps an answer. Configure both hooks for the same model context window and compatible token counters so they make decisions from comparable estimates.
### Creating a custom compactor
Implement the `Compactor` protocol when you need a different strategy, such as summarizing older messages or selectively shortening tool results.
```python
from typing import Any
from haystack.core.serialization import default_to_dict
from haystack.dataclasses import ChatMessage
from haystack.hooks.compaction import Compactor
from haystack.token_counters import TokenCounter
class CustomCompactor(Compactor):
def compact(
self,
messages: list[ChatMessage],
target_tokens: int,
token_counter: TokenCounter,
) -> list[ChatMessage] | None:
# Return a shorter, valid conversation or None when nothing should change.
...
def to_dict(self) -> dict[str, Any]:
return default_to_dict(self)
```
A compactor must follow these rules:
1. Return `None` unless the conversation actually gets smaller.
2. Return a new list without modifying the input `messages` list.
3. Keep tool calls together with all their result messages. Chat-completion APIs reject incomplete tool-call exchanges.
The `target_tokens` value is a goal rather than a guarantee. When the target conflicts with context the Agent must retain, preserve the required context and get as close to the target as possible.
`compact_async()` calls `compact()` by default. Override it when compaction performs I/O, such as calling an LLM, so asynchronous Agent runs are not blocked. Use `to_dict()` to serialize constructor settings. The protocol's default `from_dict()` handles plain constructor values; override it when serialized values must be reconstructed first, such as a `Secret` or nested component.
## Token counters
The default `ApproximateTokenCounter` estimates tokens from text length and needs no extra dependency. You can configure another built-in or custom [`TokenCounter`](../../token-counters.mdx) when you need a model- or provider-specific estimate.
Token counters can also include tool schemas and non-text content in the estimate. Consult the page for the counter you use to understand how it handles images and files.
## Context compaction and tool result offloading
[Tool result offloading](./tool-result-offloading.mdx) solves an adjacent problem: it writes large tool results to a store and leaves a pointer in the conversation. The two approaches work well together — offloading keeps individual results small as they arrive, while compaction bounds the conversation as a whole.
An offloaded result is represented by a reference to the stored content. If a compactor removes or rewrites that message, the model loses the reference it needs to read the content again.
`ToolResultPruningCompactor` skips results marked as offloaded by default, preserving their stored-content references.
---
// File: pipeline-components/agents-1/hooks
# Hooks
Hooks let you run custom logic at defined points of an [`Agent`](./agent.mdx)'s run loop — at the start and end of a run, before each LLM call, before and after tool execution, and on exit.
| | |
| --- | --- |
| **Configured on** | The [`Agent`](./agent.mdx) component via the `hooks` parameter |
| **Key classes** | `hook` (decorator), `FunctionHook`, `Hook` (protocol) |
| **Import path** | `haystack.hooks` |
| **API reference** | [Hooks](/reference/hooks-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/ |
| **Package name** | `haystack-ai` |
## Overview
Pass `hooks` to the `Agent` as a dictionary mapping a *hook point* to a list of hooks the Agent runs at that point. Each hook receives the live [`State`](./state.mdx) and influences the run by mutating it in place. Hooks for a hook point run in list order, and the same hook can be registered under multiple hook points.
This enables patterns such as building run-time system context, retrieving memories before the first LLM call, auditing or intercepting tool calls, and requiring a condition to hold before the Agent is allowed to finish.
### Hook points
- `before_run`: Runs once per run, after the state is initialized and before the first chat-generator call. Use it to rewrite the initial messages or seed state — for example, to turn the user query into a task brief — without re-running on every step like `before_llm` does.
- `before_llm`: Runs before each chat-generator call.
- `before_tool`: Runs after the model requests tool calls, before any tools run. After these hooks run, the Agent re-reads the current last message from `state.data["messages"]`. If that message contains tool calls, those calls are executed. If it does not, no tools run for that step, no tool-based exit condition is triggered, and the Agent loops back to the next LLM call unless `max_agent_steps` has been reached.
- `after_tool`: Runs after tools execute, once their result messages are in `state.data["messages"]`, before the exit-condition check and the next LLM call. Use it to rewrite the freshly produced tool-result messages — for example, to offload, redact, truncate, or summarize results. It does not run on the plain-text exit step. It does still run when a `before_tool` hook removed the pending tool calls: no tools executed on that step, so don't assume the last message is a fresh tool result.
- `on_exit`: Runs when the Agent is about to stop on an exit condition. An `on_exit` hook can keep the Agent running by setting the `continue_run` control flag (`state.set("continue_run", True)`), usually alongside a message telling the model what to do next. `on_exit` hooks run when the Agent stops on an exit condition, but not when it stops because `max_agent_steps` is reached — use `after_run` for logic that must run however the run ends.
- `after_run`: Runs once per run, after the step loop has ended and before the Agent builds its return value — regardless of whether the run stopped on an exit condition or because `max_agent_steps` was reached (unlike `on_exit`). Mutations to the state, such as appending a final message, are reflected in the returned `messages` / `last_message` and `state_schema` outputs. Setting `continue_run` here has no effect.
Registering a hook under an unknown hook point raises a `ValueError` at construction. A hook class can declare an `allowed_hook_points` attribute listing the hook points it supports; the Agent validates it and fails fast if the hook is registered somewhere it doesn't belong.
### State keys for hooks
The Agent manages a few state keys that hooks interact with. Like the run-metadata keys (`step_count`, `token_usage`, `tool_call_counts`), they are reserved — using any of them in your own `state_schema` raises a `ValueError`. See [State](./state.mdx#schema-definition) for the full list:
- `continue_run`: Set by an `on_exit` hook to keep the Agent running.
- `tools`: The tools available in the current step, for hooks to inspect.
- `hook_context`: Request-scoped resources passed to `Agent.run(hook_context={...})` / `run_async(hook_context={...})`. Hooks read it with `state.data["hook_context"]` or `state.data.get("hook_context")` — use it for per-request resources such as a user ID, a WebSocket, or a database client. Avoid the plain `state.get("hook_context")` here: `State.get` returns a deep copy of the value, which often fails for the kinds of resources stored in this dict (such as a WebSocket or a database client).
- `context_tokens`: An approximate count of the tokens currently in the context window, refreshed after each LLM call with that reply's prompt-plus-completion tokens (read it with `state.get("context_tokens")`). Unlike `token_usage`, which accumulates over the run, this is replaced on every call. It's `0` until the first reply that reports usage and doesn't count messages appended after the latest call. A `before_llm` hook can read it to trigger context compaction once it crosses a threshold.
Hooks can also read the automatically tracked run metadata: `step_count`, `token_usage`, and `tool_call_counts`.
## Creating hooks
### With the `@hook` decorator
The `@hook` decorator wraps a function taking a single `State` argument into a hook. A regular function becomes the hook's sync path, a coroutine function its async path. To give a single hook both paths, construct a `FunctionHook` directly with both `function` and `async_function`.
The example below registers a hook at each of `before_llm`, `before_tool`, and `on_exit` to show what hooks can do:
```python
from datetime import datetime, timezone
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.agents.state import State, replace_values
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.hooks import hook
from haystack.tools import tool
@tool
def search(query: Annotated[str, "The search query"]) -> str:
"""Search the web."""
# Placeholder: would call a real search API
return "Fusion startups reported net-energy-gain milestones this year."
@hook
def build_context(state: State) -> None:
# before_llm: build run-time system context once, before the first model call.
if state.get("step_count") == 0:
now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
system = ChatMessage.from_system(
f"You are a research assistant. The current time is {now}.",
)
state.set(
"messages",
[system, *state.data["messages"]],
handler_override=replace_values,
)
@hook
def audit_tool_calls(state: State) -> None:
# before_tool: see which tools the model is about to run.
pending = state.data["messages"][-1].tool_calls
print(f"about to run: {[tc.tool_name for tc in pending]}")
@hook
def require_search(state: State) -> None:
# on_exit: keep going until the agent has actually searched.
if state.get("tool_call_counts", {}).get("search", 0) == 0:
state.set("messages", [ChatMessage.from_system("Search before answering.")])
state.set("continue_run", True)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[search],
hooks={
"before_llm": [build_context],
"before_tool": [audit_tool_calls],
"on_exit": [require_search],
},
)
result = agent.run(
messages=[
ChatMessage.from_user("What are the latest developments in fusion energy?"),
],
)
print(result["last_message"].text)
```
### Class-based hooks
A hook is any object with a `run(state)` method; it may additionally define `run_async(state)` for true async behavior. Class-based hooks may also implement the optional lifecycle methods `warm_up` / `warm_up_async` and `close` / `close_async`. The Agent calls them from its own `warm_up` / `close`, so a hook can defer opening clients or reading credentials until warm-up and release them on close. Because warm-up runs before every Agent run, a hook should not repeat expensive initialization: return early if the work is already done, as in `if self._client is not None: return`.
When a class-based hook should be serializable (so an Agent using it can be serialized), implement `to_dict` / `from_dict`: store serializable constructor arguments on the hook and rebuild runtime clients from those values.
The example below is an `on_exit` hook that grades the Agent's answer with its own LLM and asks the Agent to improve a weak answer before finishing:
```python
from typing import Any
from haystack.components.agents import Agent
from haystack.components.agents.state import State
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.core.serialization import default_from_dict, default_to_dict
from haystack.dataclasses import ChatMessage
class GradeFinalAnswer:
"""Grade the Agent's answer with an LLM and ask it to improve a weak answer before finishing."""
def __init__(self, model: str = "gpt-5.4-nano"):
self.model = model
self._judge = OpenAIChatGenerator(model=self.model)
def warm_up(self) -> None:
# The Agent calls this before every run, but OpenAIChatGenerator.warm_up
# creates its client only on the first call, so repeating it is safe and cheap.
self._judge.warm_up()
def close(self) -> None:
# Release the judge's client during the Agent's close.
self._judge.close()
def run(self, state: State) -> None:
answer = state.data["messages"][-1].text or ""
verdict = (
self._judge.run(
messages=[
ChatMessage.from_user(
f"Reply with only PASS or FAIL. Is this answer complete?\n\n{answer}",
),
],
)["replies"][0].text
or ""
)
if "FAIL" in verdict.upper():
state.set(
"messages",
[
ChatMessage.from_user(
"Your answer was incomplete. Please improve it.",
),
],
)
state.set("continue_run", True)
def to_dict(self) -> dict[str, Any]:
return default_to_dict(self, model=self.model)
@classmethod
def from_dict(cls, data: dict[str, Any]) -> "GradeFinalAnswer":
return default_from_dict(cls, data)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
hooks={"on_exit": [GradeFinalAnswer()]},
)
result = agent.run(messages=[ChatMessage.from_user("Explain how vaccines work.")])
print(result["last_message"].text)
```
## Ready-made hooks
Haystack ships several ready-made hooks, each in its own submodule of `haystack.hooks`:
- `CompactionHook` (from `haystack.hooks.compaction`): A `before_llm` hook that shortens an Agent's conversation when it reaches a configured fraction of the model's context window. A `Compactor` determines how the conversation is shortened. See [Context Compaction](./compaction.mdx).
- `ConfirmationHook` (from `haystack.hooks.human_in_the_loop`): A `before_tool` hook that applies Human-in-the-Loop confirmation strategies to pending tool calls — a human can confirm, modify, or reject the tool calls the model requested before they run. See [Human in the Loop](./human-in-the-loop.mdx).
- `ToolResultOffloadHook` (from `haystack.hooks.tool_result_offloading`): An `after_tool` hook that offloads tool results to a `ToolResultStore` (such as `FileSystemToolResultStore`) and replaces them in the conversation with a compact pointer, so the next LLM call sees a reference instead of the full result. Per-tool policies (`AlwaysOffload`, `NeverOffload`, `OffloadOverChars`) control which results are offloaded. See [Tool Result Offloading](./tool-result-offloading.mdx).
---
// File: pipeline-components/agents-1/human-in-the-loop
# Human in the Loop
Human-in-the-loop (HITL) lets you intercept an agent's tool calls before they are executed.
A human can **confirm**, **reject**, or **modify** the parameters of each tool call in real time.
This is useful for high-stakes operations - such as sending emails, modifying databases, or making API calls - where you want a human to review the action first.
| | |
| --- | --- |
| **Configured on** | The [`Agent`](./agent.mdx) component, as a `ConfirmationHook` registered under the `before_tool` [hook point](./hooks.mdx) |
| **Key classes** | `ConfirmationHook`, `BlockingConfirmationStrategy`, `AlwaysAskPolicy`, `AskOncePolicy`, `NeverAskPolicy`, `RichConsoleUI`, `SimpleConsoleUI` |
| **Import path** | `haystack.hooks.human_in_the_loop` |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/human_in_the_loop/ |
| **Package name** | `haystack-ai` |
## Overview
HITL is one application of the Agent's general [hooks](./hooks.mdx) mechanism: a `ConfirmationHook` registered under the `before_tool` hook point intercepts the tool calls the model requested before they run, and confirms, modifies, or rejects them by rewriting the conversation in the Agent's `State`.
The HITL system is composed of these layers:
- **`ConfirmationHook`** - the `before_tool` hook that applies your confirmation strategies to pending tool calls. Its `confirmation_strategies` mapping accepts a single tool name, a tuple of tool names, or the wildcard `"*"` that applies to any tool without a more specific entry.
- **Strategy** - decides what to do when a tool is about to be called. The built-in `BlockingConfirmationStrategy` pauses execution and asks a human.
- **Policy** - decides *when* to ask. Built-in policies: `AlwaysAskPolicy`, `NeverAskPolicy`, `AskOncePolicy`.
- **UI** - the interface used to ask the human. Built-in UIs: `RichConsoleUI` (requires `rich`) and `SimpleConsoleUI` (stdlib only).
When the agent is about to invoke a tool, the strategy checks the policy.
If the policy says to ask, the UI prompts the human with the tool name, description, and parameters. The human can:
- **Confirm** (`y`) - execute as-is
- **Reject** (`n`) - skip execution and feed rejection feedback back to the LLM
- **Modify** (`m`) - edit the parameters before execution
The agent then continues with the human's decision.
:::info
Strategies see only the arguments the model produced for a tool call. Values injected from [`State`](./state.mdx) via a tool's `inputs_from_state` mapping are not included in what is presented for confirmation — that injection happens at tool execution time.
:::
## Usage
### Basic setup
```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.hooks.human_in_the_loop import (
AlwaysAskPolicy,
BlockingConfirmationStrategy,
ConfirmationHook,
SimpleConsoleUI,
)
from haystack.tools import tool
@tool
def send_email(
to: Annotated[str, "The recipient email address"],
subject: Annotated[str, "The email subject line"],
body: Annotated[str, "The email body"],
) -> str:
"""Send an email to a recipient."""
return f"Email sent to {to}."
strategy = BlockingConfirmationStrategy(
confirmation_policy=AlwaysAskPolicy(),
confirmation_ui=SimpleConsoleUI(),
)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"),
tools=[send_email],
hooks={
"before_tool": [
ConfirmationHook(confirmation_strategies={"send_email": strategy}),
],
},
)
result = agent.run(
messages=[ChatMessage.from_user("Send a welcome email to alice@example.com")],
)
```
When the agent calls `send_email`, the terminal will pause and show:
```
--- Tool Execution Request ---
Tool: send_email
Description: Send an email to a recipient.
Arguments:
to: alice@example.com
subject: Welcome!
body: Hi Alice, welcome aboard!
------------------------------
Confirm execution? (y=confirm / n=reject / m=modify):
```
### Using RichConsoleUI
`RichConsoleUI` provides a styled terminal prompt using the [`rich`](https://github.com/Textualize/rich) library:
```shell
pip install rich
```
```python
from haystack.hooks.human_in_the_loop import RichConsoleUI
strategy = BlockingConfirmationStrategy(
confirmation_policy=AlwaysAskPolicy(),
confirmation_ui=RichConsoleUI(),
)
```
### Applying strategies to multiple tools
You can configure different strategies per tool, share one strategy across a group of tools using a tuple key, or set a default for all tools with the wildcard `"*"` (applied to any tool without a more specific entry):
```python
@tool
def delete_record(record_id: Annotated[str, "The ID of the record to delete"]) -> str:
"""Delete a record from the database."""
return f"Record {record_id} deleted."
@tool
def update_record(
record_id: Annotated[str, "The ID of the record to update"],
data: Annotated[str, "The new data as a JSON string"],
) -> str:
"""Update a record in the database."""
return f"Record {record_id} updated."
@tool
def search(query: Annotated[str, "The search query"]) -> str:
"""Search the knowledge base."""
return f"Results for: {query}"
ask_strategy = BlockingConfirmationStrategy(
confirmation_policy=AlwaysAskPolicy(),
confirmation_ui=SimpleConsoleUI(),
)
confirmation_hook = ConfirmationHook(
confirmation_strategies={
# Share one strategy across multiple sensitive tools using a tuple key
("send_email", "delete_record", "update_record"): ask_strategy,
# search has no strategy - always executes without asking
},
)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"),
tools=[send_email, delete_record, update_record, search],
hooks={"before_tool": [confirmation_hook]},
)
```
### Customizing feedback messages
When a tool call is rejected or modified, `BlockingConfirmationStrategy` sends a message back to the LLM explaining what happened. Three optional template parameters control these messages — each has a sensible default, so you only need to set them if you want different wording:
- `reject_template`: Sent to the LLM when the user rejects a tool call. Must include a `{tool_name}` placeholder. Default: `"Tool execution for '{tool_name}' was rejected by the user."`
- `modify_template`: Sent when the user modifies the parameters. Must include `{tool_name}` and `{final_tool_params}` placeholders. Default: `"The parameters for tool '{tool_name}' were updated by the user to:\n{final_tool_params}"`
- `user_feedback_template`: Appends the user's optional free-text feedback to either message. Must include a `{feedback}` placeholder. Default: `"With user feedback: {feedback}"`
```python
strategy = BlockingConfirmationStrategy(
confirmation_policy=AlwaysAskPolicy(),
confirmation_ui=SimpleConsoleUI(),
reject_template="Skipping '{tool_name}' — rejected by operator.",
modify_template="Updated parameters for '{tool_name}': {final_tool_params}",
user_feedback_template="Reason: {feedback}",
)
```
## Policies
Policies control *when* the human is asked.
| Policy | Behavior |
| --- | --- |
| `AlwaysAskPolicy` | Ask every time the tool is called |
| `NeverAskPolicy` | Never ask - always proceed (useful for toggling HITL off without removing the strategy) |
| `AskOncePolicy` | Ask once per unique `(tool_name, parameters)` combination. Remembers confirmed calls and skips asking on repeats. |
### Custom policy
You can implement your own policy by subclassing `ConfirmationPolicy` from `haystack.hooks.human_in_the_loop.types`:
```python
from haystack.hooks.human_in_the_loop.types import (
ConfirmationPolicy,
ConfirmationUIResult,
)
from typing import Any
class AskForSensitiveParamsPolicy(ConfirmationPolicy):
"""Only ask when the 'to' parameter looks like an external email domain."""
def should_ask(
self,
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
) -> bool:
to = tool_params.get("to", "")
return not to.endswith("@mycompany.com")
```
For stateful policies, also implement `update_after_confirmation`.
It is called after the user responds and receives the full `ConfirmationUIResult`, letting you update internal state based on the outcome.
The following policy asks once per tool name and skips re-asking for any tool the user has already confirmed:
```python
from haystack.hooks.human_in_the_loop.types import ConfirmationPolicy
from haystack.hooks.human_in_the_loop import ConfirmationUIResult
from typing import Any
class AskOncePerToolPolicy(ConfirmationPolicy):
"""Ask once per tool name, regardless of parameters. Skip on repeat confirmed calls."""
def __init__(self) -> None:
self._confirmed_tools: set[str] = set()
def should_ask(
self,
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
) -> bool:
return tool_name not in self._confirmed_tools
def update_after_confirmation(
self,
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
confirmation_result: ConfirmationUIResult,
) -> None:
if confirmation_result.action == "confirm":
self._confirmed_tools.add(tool_name)
```
## Dataclasses
### `ConfirmationUIResult`
Returned by the UI after the human responds.
| Field | Type | Description |
| --- | --- | --- |
| `action` | `str` | `"confirm"`, `"reject"`, or `"modify"` |
| `feedback` | `str \| None` | Optional free-text feedback from the human |
| `new_tool_params` | `dict \| None` | Replacement parameters when action is `"modify"` |
### `ToolExecutionDecision`
Returned by the strategy to the agent.
| Field | Type | Description |
| --- | --- | --- |
| `tool_name` | `str` | Name of the tool |
| `execute` | `bool` | Whether to execute the tool |
| `tool_call_id` | `str \| None` | ID of the tool call |
| `feedback` | `str \| None` | Feedback message passed back to the LLM on rejection or modification |
| `final_tool_params` | `dict \| None` | Final parameters to use for execution |
## Example: HITL with Hayhooks and Open WebUI
The [hitl-hayhooks-redis-openwebui](https://github.com/deepset-ai/hitl-hayhooks-redis-openwebui) repository shows a full production-style HITL setup using a Haystack Agent served via [Hayhooks](https://github.com/deepset-ai/hayhooks) with approval dialogs rendered in [Open WebUI](https://github.com/open-webui/open-webui).
The key pattern it demonstrates is a custom `RedisConfirmationStrategy` that receives per-request resources - a Redis client and an async event queue - at runtime. Pass such resources via the generic `hook_context` run argument (`agent.run(messages=[...], hook_context={"redis": client})`). `ConfirmationHook` reads this dict from state with `state.data["hook_context"]` (not `state.get`, which returns a deep copy that fails for live resources like clients and queues - see [Hooks](./hooks.mdx)) and passes it to each strategy's `run()` as the `confirmation_strategy_context` keyword argument, which is how a custom strategy receives the Redis client and event queue:
- When a tool call is about to execute, the strategy emits a `tool_call_start` SSE event and blocks on `Redis BLPOP` waiting for an approval decision.
- The Open WebUI Pipe function receives the SSE event, shows the user a confirmation dialog, then writes `approved` or `rejected` to Redis via `LPUSH`.
- Once Redis unblocks, the strategy returns a `ToolExecutionDecision` and the agent continues.
This is a good reference if you need non-blocking HITL in a web or server environment where `SimpleConsoleUI` and `RichConsoleUI` are not suitable.
## Custom UI
Implement `ConfirmationUI` from `haystack.hooks.human_in_the_loop.types` to build your own interface - for example, a web-based approval queue:
```python
from haystack.hooks.human_in_the_loop.types import ConfirmationUI
from haystack.hooks.human_in_the_loop import ConfirmationUIResult
from typing import Any
class WebhookApprovalUI(ConfirmationUI):
"""Sends a webhook and waits for an async approval response."""
def get_user_confirmation(
self,
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
) -> ConfirmationUIResult:
# Send approval request to your system and wait for response
response = send_approval_request_and_wait(tool_name, tool_params)
return ConfirmationUIResult(
action=response["action"],
feedback=response.get("feedback"),
)
```
---
// File: pipeline-components/agents-1/state
# State
`State` is a container for storing shared information during Agent and Tool execution.
It provides a structured way to share data between tools, accumulate results across multiple tool calls, and surface them alongside the agent's final answer.
## Overview
When building agents that use multiple tools, you often need tools to share information or accumulate results across iterations.
State provides centralized storage that all tools can read from and write to.
For example, a search tool called multiple times can append its results to a shared `documents` list, which is then returned alongside the agent's final answer for source inspection.
State uses a schema-based approach where you define:
- What data can be stored,
- The type of each piece of data,
- How values are merged when updated.
The Agent creates and manages the `State` object internally. You shouldn't need to instantiate it directly.
You interact with it through tool definitions (`inputs_from_state`, `outputs_to_state`, or a `state: State` parameter) and read results from the agent's output dict.
### Supported Types
State supports standard Python types:
- Basic types: `str`, `int`, `float`, `bool`, `dict`
- List types: `list`, `list[str]`, `list[int]`, `list[Document]`
- Union types: `str | int`, `str | None`
- Custom classes and data classes.
### Automatic Message Handling
State automatically includes a `messages` field that stores the full conversation history during execution.
You don't need to define this in your schema.
It uses `list[ChatMessage]` type with the `merge_lists` handler, so new messages are appended on each iteration.
### State API
| Method | Description |
| --- | --- |
| `state.get(key, default=None)` | Read a value; returns `default` if the key doesn't exist |
| `state.set(key, value)` | Write a value, merged using the schema's handler |
| `state.has(key)` | Returns `True` if the key exists in state |
| `state.data` | Returns a snapshot of all current state as a `dict` |
## Schema Definition
The schema defines what data can be stored and how values are updated. Each schema entry consists of:
- `type` (required): The Python type for this field (for example, `str`, `int`, `list`)
- `handler` (optional): A callable that determines how new values are merged when `set()` is called
```python
{
"parameter_name": {
"type": SomeType, # Required: expected Python type
"handler": some_func, # Optional: merge function
},
}
```
If you don't specify a handler, State automatically assigns a default based on the type.
:::info Reserved keys
The `Agent` manages some state keys itself and rejects them in a user-provided `state_schema` with a `ValueError`:
- The run-metadata keys `step_count`, `token_usage`, `tool_call_counts`, and `exit_reason`, which the Agent populates automatically during a run: tools and hooks can read them from the live `State`, and they are returned in the result dictionary. `exit_reason` reports why the Agent stopped (`"text"`, the name of the tool that triggered a tool exit condition, or `"max_agent_steps"`).
- The hook-facing keys `continue_run` (set by an `on_exit` hook to keep the Agent running), `tools` (the tools available in the current step, for hooks to inspect), `hook_context` (the request-scoped resources passed to `Agent.run(hook_context={...})`), and `context_tokens` (an approximate count of the tokens currently in the context window, refreshed after each LLM call for hooks to read — for example, to trigger context compaction). Unlike the run-metadata keys, these are not returned in the result dictionary.
If one of your state keys clashes, rename it (for example, `my_token_usage`).
:::
### Default Handlers
State provides two built-in merge behaviors (importable from `haystack.components.agents.state`):
- **`merge_lists`**: Appends to the existing list (default for list types)
- **`replace_values`**: Overwrites the existing value (default for non-list types)
```python
from haystack.components.agents import State
schema = {
"documents": {"type": list}, # uses merge_lists by default
"user_name": {"type": str}, # uses replace_values by default
}
state = State(schema=schema)
state.set("documents", [1, 2])
state.set("documents", [3, 4])
print(state.get("documents")) # [1, 2, 3, 4]
state.set("user_name", "Alice")
state.set("user_name", "Bob")
print(state.get("user_name")) # "Bob"
```
### Custom Handlers
Custom handlers are useful when the default `merge_lists` or `replace_values` behaviors don't fit your needs.
A handler takes the current state value and the new value and returns the merged result.
The example below uses a deduplication handler, useful when multiple tool calls might return overlapping results and you want to avoid accumulating duplicates in state:
```python
def deduplicate(current_value: list | None, new_value: list) -> list:
"""Append new items, skipping any already in the list."""
existing = set(current_value or [])
return (current_value or []) + [item for item in new_value if item not in existing]
schema = {"doc_ids": {"type": list, "handler": deduplicate}}
state = State(schema=schema)
state.set("doc_ids", ["doc-1", "doc-2"])
state.set("doc_ids", ["doc-2", "doc-3"])
print(state.get("doc_ids")) # ["doc-1", "doc-2", "doc-3"]
```
You can also override the handler for a single `set()` call:
```python
from haystack.components.agents import State
def concatenate_strings(current: str | None, new: str) -> str:
return f"{current}-{new}" if current else new
state = State(schema={"user_name": {"type": str}})
state.set("user_name", "Alice")
state.set("user_name", "Bob", handler_override=concatenate_strings)
print(state.get("user_name")) # "Alice-Bob"
```
## Using State
Define a `state_schema` when creating the Agent.
State keys declared in `state_schema` are exposed as output keys on the agent's result dict alongside `messages` and `last_message`.
Tools interact with State through three mechanisms:
- **`outputs_to_state`**: Write tool results to state keys after the tool runs.
- **`inputs_from_state`**: Inject state values into tool parameters before the tool runs.
- **Direct `State` injection**: Add a `state: State` parameter to your tool function's signature. The Agent detects the `State` annotation and injects the live `State` object automatically, so you can read or write any key defined in the schema. The `State` object is never exposed to the LLM's parameter schema.
### Reading from State: `inputs_from_state`
`inputs_from_state` maps state keys to function parameter names using the format `{"state_key": "param_name"}`.
The value is injected from state before the tool runs, so the LLM never needs to provide it.
Parameters mapped via `inputs_from_state` are automatically excluded from the LLM's parameter schema.
The model never sees or provides them:
```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.tools import tool
@tool(inputs_from_state={"user_name": "user_context"})
def search_documents(
query: Annotated[str, "The search query"],
user_context: str, # injected from state; excluded from LLM schema
) -> dict:
"""Search documents using query and user context."""
return {"results": [f"Found results for '{query}' (user: {user_context})"]}
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[search_documents],
system_prompt="Use the search_documents tool to find information.",
streaming_callback=print_streaming_chunk,
state_schema={"user_name": {"type": str}},
)
result = agent.run(
messages=[ChatMessage.from_user("Search for Python tutorials")],
user_name="Alice", # state key "user_name" is pre-populated by passing user_name= to agent.run()
)
print(result["last_message"].text)
```
### Writing to State: `outputs_to_state`
The `outputs_to_state` parameter maps tool output keys to state keys. Each entry supports two optional fields:
```python
{
"state_key": {
"source": "tool_output_key", # which key to read from the tool's return dict; omit to store the entire dict
"handler": some_func, # override the schema's merge handler for this mapping only
},
}
```
```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.tools import tool
@tool(
outputs_to_state={
"documents": {"source": "documents"},
"result_count": {"source": "count"},
"last_query": {"source": "query"},
},
)
def retrieve_documents(
query: Annotated[str, "The search query"],
) -> dict:
"""Retrieve relevant documents."""
return {
"documents": [
{"title": "Doc 1", "content": "Content about Python"},
{"title": "Doc 2", "content": "More about Python"},
],
"count": 2,
"query": query,
}
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[retrieve_documents],
system_prompt="Use the retrieve_documents tool to find information.",
streaming_callback=print_streaming_chunk,
state_schema={
"documents": {"type": list},
"result_count": {"type": int},
"last_query": {"type": str},
},
)
result = agent.run(messages=[ChatMessage.from_user("Find information about Python")])
print(f"Documents: {result['documents']}")
print(f"Result count: {result['result_count']}")
print(f"Last query: {result['last_query']}")
```
If you omit `source`, the entire tool result dict is stored under the state key:
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.tools import tool
@tool(outputs_to_state={"user_info": {}})
def get_user_info() -> dict:
"""Get user information."""
return {"name": "Alice", "email": "alice@example.com", "role": "admin"}
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[get_user_info],
system_prompt="Use the get_user_info tool to look up user details.",
streaming_callback=print_streaming_chunk,
state_schema={"user_info": {"type": dict}},
)
result = agent.run(messages=[ChatMessage.from_user("What are the user's details?")])
print(result["last_message"].text)
print(f"User info: {result['user_info']}")
```
### Combining Inputs and Outputs
Tools can both read from and write to State, enabling tool chaining across iterations.
This example builds on `retrieve_documents` from the previous section:
```python
@tool(
inputs_from_state={"documents": "documents"},
outputs_to_state={
"final_docs": {"source": "processed_docs"},
"final_count": {"source": "processed_count"},
},
)
def process_documents(
max_results: Annotated[int, "Maximum number of documents to return"],
documents: list = None, # injected from state; LLM does not provide this
) -> dict:
"""Process retrieved documents and return a filtered subset."""
processed = (documents or [])[:max_results]
return {"processed_docs": processed, "processed_count": len(processed)}
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[retrieve_documents, process_documents], # chained through state
system_prompt="Use the available tools to retrieve and process documents.",
streaming_callback=print_streaming_chunk,
state_schema={
"documents": {"type": list},
"result_count": {"type": int},
"last_query": {"type": str},
"final_docs": {"type": list},
"final_count": {"type": int},
},
)
result = agent.run(
messages=[ChatMessage.from_user("Find and process 3 documents about Python")],
)
print(f"Processed {result['final_count']} documents")
```
### Injecting State Directly into Tools
As an alternative to `inputs_from_state` and `outputs_to_state`, a tool can declare a `state: State` parameter to receive the live `State` object at invocation time.
This lets the tool read from and write to any number of state keys without declaring mappings upfront.
The Agent detects the `State` annotation and injects the object automatically.
It is excluded from the LLM-facing schema. The model is never asked to supply it.
Both `State` and `State | None` annotations are supported.
For function-based tools, add the `state` parameter and use the `@tool` decorator:
```python
from typing import Annotated
from haystack.components.agents import Agent, State
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage, Document
from haystack.tools import tool
@tool
def retrieve_and_store(
query: Annotated[str, "The search query"],
state: State,
) -> str:
"""Retrieve documents and store them directly in state."""
documents = [Document(content=f"Result for '{query}'")]
state.set("documents", documents)
user_name = state.get("user_name", "unknown")
return f"Retrieved {len(documents)} document(s) for {user_name}"
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[retrieve_and_store],
system_prompt="Use the retrieve_and_store tool to find documents.",
streaming_callback=print_streaming_chunk,
state_schema={"documents": {"type": list[Document]}, "user_name": {"type": str}},
)
result = agent.run(
messages=[ChatMessage.from_user("Find documents about Python")],
user_name="Alice",
)
print(result["last_message"].text)
print(result["documents"])
```
For component-based tools, declare a `State` input socket on the `run` method and wrap it with `ComponentTool`:
```python
from haystack import component
from haystack.components.agents import Agent, State
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage, Document
from haystack.tools import ComponentTool
@component
class DocumentRetriever:
"""Retrieve documents and store them in state."""
@component.output_types(reply=str)
def run(self, query: str, state: State) -> dict[str, str]:
"""
Retrieve documents based on query and store them in state.
:param query: The search query
"""
documents = [Document(content=f"Result for '{query}'")]
state.set("documents", documents)
return {"reply": f"Retrieved {len(documents)} document(s)"}
retriever_tool = ComponentTool(
component=DocumentRetriever(),
name="retrieve",
description="Retrieve documents based on a search query",
)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[retriever_tool],
system_prompt="Use the retrieve tool to find documents.",
streaming_callback=print_streaming_chunk,
state_schema={"documents": {"type": list[Document]}},
)
result = agent.run(messages=[ChatMessage.from_user("Find documents about Python")])
print(result["last_message"].text)
print(result["documents"])
```
---
// File: pipeline-components/agents-1/tool-result-offloading
# Tool Result Offloading
Tool result offloading writes selected tool results to a store and replaces them in the conversation with a compact pointer — a reference plus a short preview — so the next LLM call sees a reference instead of the full result.
This keeps the context window small when tools return large outputs (web pages, file contents, query results), and it is a step towards letting an Agent operate on offloaded results with follow-up tools, such as a file-reading tool that opens the referenced files.
| | |
| --- | --- |
| **Configured on** | The [`Agent`](./agent.mdx) component, as a `ToolResultOffloadHook` registered under the `after_tool` [hook point](./hooks.mdx) |
| **Key classes** | `ToolResultOffloadHook`, `FileSystemToolResultStore`, `AlwaysOffload`, `NeverOffload`, `OffloadOverChars` |
| **Import path** | `haystack.hooks.tool_result_offloading` |
| **API reference** | [Hooks](/reference/hooks-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/tool_result_offloading/ |
| **Package name** | `haystack-ai` |
## Overview
Tool result offloading is one application of the Agent's general [hooks](./hooks.mdx) mechanism: a `ToolResultOffloadHook` registered under the `after_tool` hook point runs after each step's tools execute and rewrites the freshly produced tool-result messages in the Agent's [`State`](./state.mdx). It only considers the current step's results; earlier conversation history is left untouched.
The system is composed of these layers:
- **`ToolResultOffloadHook`** - the `after_tool` hook that applies your offload strategies to fresh tool results. Its `offload_strategies` mapping accepts a single tool name, a tuple of tool names, or the wildcard `"*"` that applies to any tool without a more specific entry.
- **Policy** - decides *whether* a given result is offloaded. Built-in policies: `AlwaysOffload`, `NeverOffload`, `OffloadOverChars`.
- **Store** - decides *where* the full result lives. The built-in `FileSystemToolResultStore` writes results to the local file system.
When a result is offloaded, the hook writes the full text to the store and rebuilds the message with a one-line pointer in its place:
```
Tool result offloaded to '/abs/path/tool_results/2_search_call-123.txt' (18234 characters). Preview: Fusion startups reported...
```
The pointer carries the store reference, the original length, and a preview of the first `preview_chars` characters (200 by default, configurable on the hook), so the model knows roughly what was offloaded and where to find it.
## Usage
### Basic setup
The example below offloads any tool result longer than 4,000 characters to files under a local `tool_results` directory:
```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.hooks.tool_result_offloading import (
FileSystemToolResultStore,
OffloadOverChars,
ToolResultOffloadHook,
)
from haystack.tools import tool
@tool
def search(query: Annotated[str, "The search query"]) -> str:
"""Search the web and return the (potentially large) results."""
# Placeholder: would call a real search API
return f"... large result for {query} ..."
offload_hook = ToolResultOffloadHook(
store=FileSystemToolResultStore(root="tool_results"),
offload_strategies={"*": OffloadOverChars(4000)},
)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[search],
hooks={"after_tool": [offload_hook]},
)
result = agent.run(messages=[ChatMessage.from_user("Summarize today's tech news")])
```
### Configuring what gets offloaded per tool
Each key in `offload_strategies` may be a single tool name, a tuple of tool names sharing one policy, or the wildcard `"*"`. More specific keys win over `"*"`, and a tool with no matching key (and no `"*"`) is never offloaded:
```python
from haystack.hooks.tool_result_offloading import (
AlwaysOffload,
FileSystemToolResultStore,
NeverOffload,
OffloadOverChars,
ToolResultOffloadHook,
)
offload_hook = ToolResultOffloadHook(
store=FileSystemToolResultStore(root="tool_results"),
offload_strategies={
"web_search": AlwaysOffload(), # force offload
"get_time": NeverOffload(), # opt out of the wildcard default
("read_file", "list_dir"): OffloadOverChars(4000), # tuple key: shared policy
"*": OffloadOverChars(8000), # default for any unlisted tool
},
)
```
### What is offloaded
The hook only offloads **successful, text** tool results:
- Error results — including rejections produced by a `before_tool` [Human-in-the-Loop](./human-in-the-loop.mdx) hook — are always left in context, so the model sees what went wrong.
- Non-text results (image or file content) are left in context; supporting only text is a deliberate choice for now. A warning is logged when a non-text result has a matching offload policy.
- Each result is offloaded at most once, even though the hook runs on every tool step. This also means two offload hooks registered under `after_tool` won't offload each other's pointers.
## Policies
Policies control *whether* a result is offloaded.
| Policy | Behavior |
| --- | --- |
| `AlwaysOffload` | Offload every result of the tool it is assigned to |
| `NeverOffload` | Never offload - keep the full result in context (useful to opt a tool out of a wildcard default) |
| `OffloadOverChars(threshold)` | Offload only when the result is longer than `threshold` characters |
### Custom policy
Subclass the `OffloadPolicy` protocol from `haystack.hooks.tool_result_offloading` for custom conditions. A policy needs a `should_offload` method, which receives the tool name, the result text, and the Agent's live [`State`](./state.mdx), so it can also decide based on run context:
```python
from haystack.components.agents.state import State
from haystack.hooks.tool_result_offloading import OffloadPolicy
class OffloadLateSteps(OffloadPolicy):
"""Offload results only once the run is several steps deep and context pressure builds up."""
def should_offload(self, tool_name: str, result: str, state: State) -> bool:
return state.data.get("step_count", 0) >= 3 and len(result) > 1000
```
The protocol provides default `to_dict` / `from_dict` implementations, so a policy like this one, whose constructor takes no arguments, is serializable as-is. A policy with constructor arguments should implement both methods itself, following `OffloadOverChars` as an example.
## Stores
### `FileSystemToolResultStore`
`FileSystemToolResultStore(root=...)` writes each offloaded result to a file under its root directory and returns the absolute file path as the reference. The directory is created on first write. Store keys are derived from the step count, tool name, and tool call ID (for example `2_search_call-123.txt`), so results from different tools and steps do not collide. A key that would resolve outside the root directory is rejected.
### Custom store
Subclass the `ToolResultStore` protocol to target other backends, such as object storage or an isolated sandbox file system. A store needs two methods: `write(key=..., content=...)` persists the content and returns an opaque reference string, and `read(reference)` resolves that reference back to the content:
```python
from haystack.hooks.tool_result_offloading import ToolResultStore
class InMemoryToolResultStore(ToolResultStore):
"""Keep offloaded results in a dict - useful for tests."""
def __init__(self) -> None:
self._data: dict[str, str] = {}
def write(self, *, key: str, content: str) -> str:
self._data[key] = content
return key
def read(self, reference: str) -> str:
return self._data[reference]
```
Like `OffloadPolicy`, the protocol provides default `to_dict` / `from_dict` implementations covering stores whose constructor takes no arguments; implement both methods for stores with constructor arguments.
### Per-run stores via `hook_context`
The constructor `store` is shared by every run - fine for single-user or local use. In a multi-user server, give each run its own isolated store (for example, a per-session directory) by passing it in the Agent's generic `hook_context` run argument under the key `RESULT_STORE_CONTEXT_KEY`. It overrides the constructor store for that run:
```python
from haystack.hooks.tool_result_offloading import (
RESULT_STORE_CONTEXT_KEY,
FileSystemToolResultStore,
)
per_request_store = FileSystemToolResultStore(root=f"tool_results/{session_id}")
result = agent.run(
messages=[ChatMessage.from_user("...")],
hook_context={RESULT_STORE_CONTEXT_KEY: per_request_store},
)
```
Isolating the store per run keeps concurrent users from colliding on store keys or reading each other's offloaded results — especially important when a file-reading tool is scoped to the store. The hook itself keeps no mutable state, so a single instance is safe to share across concurrent runs.
## Letting the Agent read offloaded results back
The pointer left in the conversation tells the model where the full result lives, but the model can only act on it if the Agent has a tool that can read from the store. With `FileSystemToolResultStore`, that can be a simple file-reading tool:
```python
from typing import Annotated
from haystack.tools import tool
@tool
def read_offloaded_result(
path: Annotated[str, "Absolute path of an offloaded tool result"],
) -> str:
"""Read back the full content of an offloaded tool result."""
return FileSystemToolResultStore(root="tool_results").read(path)
```
With this tool available, the Agent can work with a compact conversation and selectively re-read only the offloaded results it actually needs — instead of carrying every full result in context on every LLM call.
## Serialization
`ToolResultOffloadHook` implements `to_dict` / `from_dict`, so an Agent using it can be serialized as long as the configured store and policies are serializable too. The built-in store and policies all are; for custom ones, see the notes in [Policies](#custom-policy) and [Stores](#custom-store) above.
## Additional References
📖 Related docs:
- [Hooks](./hooks.mdx) — the general mechanism behind this feature, including the `after_tool` hook point
- [Human in the Loop](./human-in-the-loop.mdx) — another ready-made hook, intercepting tool calls for human review
- [State](./state.mdx) — the live run state hooks and policies receive
---
// File: pipeline-components/audio/external-integrations-audio
# External Integrations
External integrations that enable working with audio in Haystack by transcribing files or converting text to audio.
| Name | Description |
| --- | --- |
| [AssemblyAI](https://haystack.deepset.ai/integrations/assemblyai) | Perform speech recognition, speaker diarization and summarization. |
| [Elevenlabs](https://haystack.deepset.ai/integrations/elevenlabs) | Convert text to speech using ElevenLabs’ API. |
---
// File: pipeline-components/audio/funasrtranscriber
# FunASRTranscriber
Transcribe audio files to Haystack Documents using FunASR — a local, open-source speech recognition toolkit supporting 50+ languages.
| | |
| --- | --- |
| **Most common position in a pipeline** | As the first component in an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of audio file paths (`str` or `Path`) or `ByteStream` objects |
| **Output variables** | `documents`: A list of Haystack Documents, one per source, with transcript text in `content` |
| **API reference** | [FunASR integration](/reference/integrations-funasr) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/funasr/src/haystack_integrations/components/audio/funasr/transcriber.py |
## Overview
`FunASRTranscriber` uses [FunASR](https://github.com/modelscope/FunASR), an open-source speech recognition toolkit from Alibaba DAMO Academy, to transcribe audio files into Haystack `Document` objects. It runs entirely locally — no API key required.
The default model is `iic/SenseVoiceSmall`, a multilingual model supporting 50+ languages that is 5–10x faster than Whisper. Models are downloaded from ModelScope on first use and cached in `~/.cache/modelscope`.
The component accepts audio file paths (`str` or `Path`) as well as `ByteStream` objects. The model is loaded into memory automatically the first time the component runs.
## Usage
### On its own
```python
from haystack_integrations.components.audio.funasr import FunASRTranscriber
transcriber = FunASRTranscriber()
result = transcriber.run(sources=["speech.wav"])
print(result["documents"][0].content)
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.fetchers import LinkContentFetcher
from haystack_integrations.components.audio.funasr import FunASRTranscriber
pipe = Pipeline()
pipe.add_component("fetcher", LinkContentFetcher())
pipe.add_component("transcriber", FunASRTranscriber())
pipe.connect("fetcher", "transcriber")
result = pipe.run(
data={
"fetcher": {
"urls": ["https://example.com/interview.wav"],
},
},
)
print(result["transcriber"]["documents"][0].content)
```
---
// File: pipeline-components/audio/localwhispertranscriber
# LocalWhisperTranscriber
Use `LocalWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model using your local installation of Whisper.
| | |
| --- | --- |
| **Most common position in a pipeline** | As the first component in an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of paths or binary streams that you want to transcribe |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Whisper](/reference/integrations-whisper) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/whisper |
| **Package name** | `whisper-haystack` |
## Overview
The component also needs to know which Whisper model to work with. Specify this in the `model` parameter when initializing the component. All transcription is completed on the executing machine, and the audio is never sent to a third-party provider.
See other optional parameters you can specify in our [API documentation](/reference/integrations-whisper).
See the [Whisper API documentation](https://platform.openai.com/docs/guides/speech-to-text) and the official Whisper [GitHub repo](https://github.com/openai/whisper) for the supported audio formats and languages.
The `LocalWhisperTranscriber` is part of the `whisper-haystack` integration package. To work with it, install the package along with [Whisper](https://github.com/openai/whisper) (which also pulls in torch) using the following commands:
```shell
pip install whisper-haystack
pip install -U openai-whisper
```
## Usage
### On its own
Here’s an example of how to use `LocalWhisperTranscriber` on its own:
```python
import requests
from haystack_integrations.components.audio.whisper import LocalWhisperTranscriber
response = requests.get(
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
)
with open("kennedy_speech.mp3", "wb") as file:
file.write(response.content)
transcriber = LocalWhisperTranscriber(model="tiny")
transcription = transcriber.run(sources=["./kennedy_speech.mp3"])
print(transcription["documents"][0].content)
```
### In a pipeline
The pipeline below fetches an audio file from a specified URL and transcribes it. It first retrieves the audio file using `LinkContentFetcher`, then transcribes the audio into text with `LocalWhisperTranscriber`, and finally outputs the transcription text.
```python
from haystack_integrations.components.audio.whisper import LocalWhisperTranscriber
from haystack.components.fetchers import LinkContentFetcher
from haystack import Pipeline
pipe = Pipeline()
pipe.add_component("fetcher", LinkContentFetcher())
pipe.add_component("transcriber", LocalWhisperTranscriber(model="tiny"))
pipe.connect("fetcher", "transcriber")
result = pipe.run(
data={
"fetcher": {
"urls": [
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
],
},
},
)
print(result["transcriber"]["documents"][0].content)
```
## Additional References
🧑🍳 Cookbook: [Multilingual RAG from a podcast with Whisper, Qdrant and Mistral](https://haystack.deepset.ai/cookbook/multilingual_rag_podcast)
---
// File: pipeline-components/audio/remotewhispertranscriber
# RemoteWhisperTranscriber
Use `RemoteWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model.
| | |
| --- | --- |
| **Most common position in a pipeline** | As the first component in an indexing pipeline |
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with an environment variable `OPENAI_API_KEY`. |
| **Mandatory run variables** | `sources`: A list of paths or binary streams that you want to transcribe |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Whisper](/reference/integrations-whisper) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/whisper |
| **Package name** | `whisper-haystack` |
## Overview
The `RemoteWhisperTranscriber` is part of the `whisper-haystack` integration package. Install it with:
```shell
pip install whisper-haystack
```
`RemoteWhisperTranscriber` works with OpenAI-compatible clients and isn't limited to just OpenAI as a provider. For example, [Groq](https://console.groq.com/docs/speech-to-text) offers a drop-in replacement that can be used as well. You can set the API key in one of two ways:
1. Through the `api_key` initialization parameter, where the key is resolved using [Secret API](../../concepts/secret-management.mdx).
2. By setting it in the `OPENAI_API_KEY` environment variable, which the system will use to access the key.
```python
from haystack_integrations.components.audio.whisper import RemoteWhisperTranscriber
transcriber = RemoteWhisperTranscriber()
```
Additionally, the component requires the following parameters to work:
- `model` specifies the Whisper model.
- `api_base_url` specifies the OpenAI base URL and defaults to `"https://api.openai.com/v1"`. If you are using Whisper provider other than OpenAI set this parameter according to provider's documentation.
See other optional parameters in our [API documentation](/reference/integrations-whisper).
See the [Whisper API documentation](https://platform.openai.com/docs/guides/speech-to-text) and the official Whisper [GitHub repo](https://github.com/openai/whisper) for the supported audio formats and languages.
## Usage
### On its own
Here’s an example of how to use `RemoteWhisperTranscriber` to transcribe a local file:
```python
import requests
from haystack_integrations.components.audio.whisper import RemoteWhisperTranscriber
response = requests.get(
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
)
with open("kennedy_speech.mp3", "wb") as file:
file.write(response.content)
transcriber = RemoteWhisperTranscriber()
transcription = transcriber.run(sources=["./kennedy_speech.mp3"])
print(transcription["documents"][0].content)
```
### In a pipeline
The pipeline below fetches an audio file from a specified URL and transcribes it. It first retrieves the audio file using `LinkContentFetcher`, then transcribes the audio into text with `RemoteWhisperTranscriber`, and finally outputs the transcription text.
```python
from haystack_integrations.components.audio.whisper import RemoteWhisperTranscriber
from haystack.components.fetchers import LinkContentFetcher
from haystack import Pipeline
pipe = Pipeline()
pipe.add_component("fetcher", LinkContentFetcher())
pipe.add_component("transcriber", RemoteWhisperTranscriber())
pipe.connect("fetcher", "transcriber")
result = pipe.run(
data={
"fetcher": {
"urls": [
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
],
},
},
)
print(result["transcriber"]["documents"][0].content)
```
## Additional References
🧑🍳 Cookbook: [Multilingual RAG from a podcast with Whisper, Qdrant and Mistral](https://haystack.deepset.ai/cookbook/multilingual_rag_podcast)
---
// File: pipeline-components/audio
# Audio
Use these components to work with audio in Haystack by transcribing files or converting text to audio.
| Name | Description |
| --- | --- |
| [FunASRTranscriber](audio/funasrtranscriber.mdx) | Transcribe audio files using FunASR — a local, open-source speech recognition toolkit supporting 50+ languages. |
| [LocalWhisperTranscriber](audio/localwhispertranscriber.mdx) | Transcribe audio files using OpenAI's Whisper model using your local installation of Whisper. |
| [RemoteWhisperTranscriber](audio/remotewhispertranscriber.mdx) | Transcribe audio files using OpenAI's Whisper model. |
---
// File: pipeline-components/builders/answerbuilder
# AnswerBuilder
Use this component in pipelines that contain a Generator to parse its replies.
| | |
| --- | --- |
| **Most common position in a pipeline** | Use in pipelines (such as a RAG pipeline) after a [Generator](../generators.mdx) component to create [`GeneratedAnswer`](../../concepts/data-classes.mdx#generatedanswer) objects from its replies. |
| **Mandatory run variables** | `query`: A query string
`replies`: A list of strings, or a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects that are replies from a Generator | | **Output variables** | `answers`: A list of `GeneratedAnswer` objects | | **API reference** | [Builders](/reference/builders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/answer_builder.py | | **Package name** | `haystack-ai` |
## Overview
`AnswerBuilder` takes a query and the replies a Generator returns as input and parses them into `GeneratedAnswer` objects. Optionally, it also takes documents and metadata from the Generator as inputs to enrich the `GeneratedAnswer` objects.
The `AnswerBuilder` works with both Chat and non-Chat Generators.
The optional `pattern` parameter defines how to extract answer texts from replies. It needs to be a regular expression with a maximum of one capture group. If a capture group is present, the text matched by the capture group is used as the answer. If no capture group is present, the whole match is used as the answer. If no `pattern` is set, the whole reply is used as the answer text.
The optional `reference_pattern` parameter can be set to a regular expression that parses referenced documents from the replies so that only those referenced documents are listed in the `GeneratedAnswer` objects. Haystack assumes that documents are referenced by their index in the list of input documents and that indices start at 1. For example, if you set the `reference_pattern` to _`\\[(\\d+)\\]`,_ it finds “1” in a string "This is an answer[1]". If `reference_pattern` is not set, all input documents are listed in the `GeneratedAnswer` objects.
## Usage
### On its own
Below is an example where we’re using the `AnswerBuilder` to parse a string that could be the reply received from a Generator using a custom regular expression. Any text other than the answer will not be included in the `GeneratedAnswer` object constructed by the builder.
```python
from haystack.components.builders import AnswerBuilder
builder = AnswerBuilder(pattern="Answer: (.*)")
builder.run(
query="What's the answer?",
replies=["This is an argument. Answer: This is the answer."],
)
```
### In a pipeline
Below is an example of a RAG pipeline where we use an `AnswerBuilder` to create `GeneratedAnswer` objects from the replies returned by a Generator. In addition to the text of the reply, these objects also hold the query, the referenced docs, and metadata returned by the Generator.
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.utils import Secret
from haystack.dataclasses import ChatMessage
from haystack.dataclasses import Document
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given these documents, answer the question.\nDocuments:\n"
"{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
"Question: {{query}}\nAnswer:",
),
]
docs = [
Document(content="The capital of France is Paris"),
Document(content="The capital of England is London"),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
p = Pipeline()
p.add_component(
instance=InMemoryBM25Retriever(document_store=document_store),
name="retriever",
)
p.add_component(
instance=ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
),
name="prompt_builder",
)
p.add_component(
instance=OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
name="llm",
)
p.add_component(instance=AnswerBuilder(), name="answer_builder")
p.connect("retriever", "prompt_builder.documents")
p.connect("prompt_builder", "llm.messages")
p.connect("llm.replies", "answer_builder.replies")
p.connect("retriever", "answer_builder.documents")
query = "What is the capital of France?"
result = p.run(
{
"retriever": {"query": query},
"prompt_builder": {"query": query},
"answer_builder": {"query": query},
},
)
print(result)
```
---
// File: pipeline-components/builders/chatpromptbuilder
# ChatPromptBuilder
This component constructs prompts dynamically by processing chat messages.
`replies`: A list of strings, or a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects that are replies from a Generator | | **Output variables** | `answers`: A list of `GeneratedAnswer` objects | | **API reference** | [Builders](/reference/builders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/answer_builder.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [Generator](../generators.mdx) |
| **Mandatory init variables** | `template`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects or a special string template. Needs to be provided either during init or run. |
| **Mandatory run variables** | `**kwargs`: Any strings that should be used to render the prompt template. See [Variables](#variables) section for more details. |
| **Output variables** | `prompt`: A dynamically constructed prompt |
| **API reference** | [Builders](/reference/builders-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/chat_prompt_builder.py |
| **Package name** | `haystack-ai` |
## Overview
The `ChatPromptBuilder` component creates prompts using static or dynamic templates written in [Jinja2](https://palletsprojects.com/p/jinja/) syntax, by processing a list of chat messages or a special string template. The templates contain placeholders like `{{ variable }}` that are filled with values provided during runtime. You can use it for static prompts set at initialization or change the templates and variables dynamically while running.
To use it, start by providing a list of `ChatMessage` objects or a special string as the template.
[`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) is a data class that includes message content, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
The builder looks for placeholders in the template and identifies the required variables. You can also list these variables manually. During runtime, the `run` method takes the template and the variables, fills in the placeholders, and returns the completed prompt. If required variables are missing. If the template is invalid, the builder raises an error.
For example, you can create a simple translation prompt:
```python
template = [ChatMessage.from_user("Translate to {{ target_language }}: {{ text }}")]
builder = ChatPromptBuilder(template=template)
result = builder.run(target_language="French", text="Hello, how are you?")
```
Or you can also replace the template at runtime with a new one:
```python
new_template = [
ChatMessage.from_user("Summarize in {{ target_language }}: {{ content }}"),
]
result = builder.run(
template=new_template,
target_language="English",
content="A detailed paragraph.",
)
```
### Variables
The template variables found in the init template are used as input types for the component. By default, `required_variables` is set to `"*"`, so all variables in the template are required: if any of them is missing at runtime, the component raises an error and halts execution.
Use `required_variables` and `variables` to specify the input types and required variables:
- `required_variables`
- Defines which template variables must be provided when the component runs.
- If any required variable is missing, the component raises an error and halts execution.
- You can:
- Use `"*"` (the default) to mark all variables in the template as required, or
- Pass a list of required variable names (such as `["name"]`) to make only those required; the remaining variables are optional, or
- Pass an empty list (`[]`) or `None` to make all variables optional. Setting `None` explicitly logs a warning, since missing variables are then silently replaced with empty strings, which can lead to unintended behavior, especially in complex pipelines.
- `variables`
- Lists all variables that can appear in the template, whether required or optional.
- Optional variables that aren't provided are replaced with an empty string in the rendered prompt.
- This allows partial prompts to be constructed without errors, unless a variable is marked as required.
In the example below, only _name_ is required to run the component, while _topic_ is only an optional variable:
```python
template = [
ChatMessage.from_user("Hello, {{ name }}. How can I assist you with {{ topic }}?"),
]
builder = ChatPromptBuilder(
template=template,
required_variables=["name"],
variables=["name", "topic"],
)
result = builder.run(name="Alice")
# Output: "Hello, Alice. How can I assist you with ?"
```
The component only waits for the required inputs before running.
### Roles
A [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) represents a single message in the conversation and can have one of three class methods that build the chat messages: `from_user`, `from_system`, or `from_assistant`. `from_user` messages are inputs provided by the user, such as a query or request. `from_system` messages provide context or instructions to guide the LLM’s behavior, such as setting a tone or purpose for the conversation. `from_assistant` defines the expected or actual response from the LLM.
Here’s how the roles work together in a `ChatPromptBuilder`:
```python
system_message = ChatMessage.from_system(
"You are an assistant helping tourists in {{ language }}.",
)
user_message = ChatMessage.from_user("What are the best places to visit in {{ city }}?")
assistant_message = ChatMessage.from_assistant(
"The best places to visit in {{ city }} include the Eiffel Tower, Louvre Museum, and Montmartre.",
)
```
### String Templates
Instead of a list of `ChatMessage` objects, you can also express the template as a special string.
This template format allows you to define `ChatMessage` sequences using Jinja2 syntax. Each `{% message %}` block defines a single message with a specific role, and you can insert dynamic content using `{{ variables }}`.
Compared to using a list of `ChatMessage`s, this format is more flexible and allows including structured parts like images in the templatized `ChatMessage`; to better understand this use case, check out the [multimodal example](#multimodal) in the Usage section below.
#### The `insert` Tag
String templates also support an `{% insert %}` tag. It is a placeholder that evaluates an expression to a `ChatMessage` or a list of `ChatMessage` objects and expands it into the prompt, so messages provided at runtime can be interleaved with literal `{% message %}` blocks. For example, you can wrap the runtime messages with a system message above and a templated user message below, then pass the messages (and any template variables) at run time:
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
template = """
{% message role="system" %}You are a helpful assistant.{% endmessage %}
{% insert messages %}
{% message role="user" %}{{ query }}{% endmessage %}
"""
builder = ChatPromptBuilder(template=template)
result = builder.run(
messages=[ChatMessage.from_user("Hi"), ChatMessage.from_assistant("Hello!")],
query="What's the weather?",
)
# result["prompt"] -> [system, user "Hi", assistant "Hello!", user "What's the weather?"]
```
All content types (tool calls, tool call results, images, reasoning, `name`, and `meta`) round trip without loss. A missing or empty value expands to nothing.
The expression can be a plain variable (`{% insert messages %}`), a slice or index (`{% insert messages[-1:] %}`, `{% insert messages[-1] %}`), or a combination of variables (`{% insert previous + current %}`). Multiple `{% insert %}` tags can be used in a single template, so the runtime messages can be split, reordered, or repeated across different positions.
### Jinja2 Time Extension
`ChatPromptBuilder` supports the Jinja2 TimeExtension, which allows you to work with datetime formats.
The Time Extension provides two main features:
1. A `now` tag that gives you access to the current time,
2. Date/time formatting capabilities through Python's datetime module.
To use the Jinja2 TimeExtension, you need to install a dependency with:
```shell
pip install arrow>=1.3.0
```
#### The `now` Tag
The `now` tag creates a datetime object representing the current time, which you can then store in a variable:
```jinja2
{% now 'utc' as current_time %}
The current UTC time is: {{ current_time }}
```
You can specify different timezones:
```jinja2
{% now 'America/New_York' as ny_time %}
The time in New York is: {{ ny_time }}
```
If you don't specify a timezone, your system's local timezone will be used:
```jinja2
{% now as local_time %}
Local time: {{ local_time }}
```
#### Date Formatting
You can format the datetime objects using Python's `strftime` syntax:
```jinja2
{% now as current_time %}
Formatted date: {{ current_time.strftime('%Y-%m-%d %H:%M:%S') }}
```
The common format codes are:
- `%Y`: 4-digit year (for example, 2025)
- `%m`: Month as a zero-padded number (01-12)
- `%d`: Day as a zero-padded number (01-31)
- `%H`: Hour (24-hour clock) as a zero-padded number (00-23)
- `%M`: Minute as a zero-padded number (00-59)
- `%S`: Second as a zero-padded number (00-59)
#### Example
```python
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
template = [
ChatMessage.from_user("Current date is: {% now 'UTC' %}"),
ChatMessage.from_assistant("Thank you for providing the date"),
ChatMessage.from_user("Yesterday was: {% now 'UTC' - 'days=1' %}"),
]
builder = ChatPromptBuilder(template=template)
result = builder.run()["prompt"]
```
## Usage
### On its own
#### With static template
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
template = [
ChatMessage.from_user(
"Translate to {{ target_language }}. Context: {{ snippet }}; Translation:",
),
]
builder = ChatPromptBuilder(template=template)
builder.run(target_language="spanish", snippet="I can't speak spanish.")
```
#### With special string template
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
template = """
{% message role="user" %}
Hello, my name is {{name}}!
{% endmessage %}
"""
builder = ChatPromptBuilder(template=template)
result = builder.run(name="John")
assert result["prompt"] == [ChatMessage.from_user("Hello, my name is John!")]
```
#### Specifying name and meta in a ChatMessage
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
template = """
{% message role="user" name="John" meta={"key": "value"} %}
Hello from {{country}}!
{% endmessage %}
"""
builder = ChatPromptBuilder(template=template)
result = builder.run(country="Italy")
assert result["prompt"] == [
ChatMessage.from_user("Hello from Italy!", name="John", meta={"key": "value"}),
]
```
#### Multiple ChatMessages with different roles
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
template = """
{% message role="system" %}
You are a {{adjective}} assistant.
{% endmessage %}
{% message role="user" %}
Hello, my name is {{name}}!
{% endmessage %}
{% message role="assistant" %}
Hello, {{name}}! How can I help you today?
{% endmessage %}
"""
builder = ChatPromptBuilder(template=template)
result = builder.run(name="John", adjective="helpful")
assert result["prompt"] == [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user("Hello, my name is John!"),
ChatMessage.from_assistant("Hello, John! How can I help you today?"),
]
```
#### Overriding static template at runtime
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
template = [
ChatMessage.from_user(
"Translate to {{ target_language }}. Context: {{ snippet }}; Translation:",
),
]
builder = ChatPromptBuilder(template=template)
builder.run(target_language="spanish", snippet="I can't speak spanish.")
summary_template = [
ChatMessage.from_user(
"Translate to {{ target_language }} and summarize. Context: {{ snippet }}; Summary:",
),
]
builder.run(
target_language="spanish",
snippet="I can't speak spanish.",
template=summary_template,
)
```
#### Multimodal
The `| templatize_part` filter in the example below tells the template engine to insert structured (non-text) objects, such as images, into the message content. These are treated differently from plain text and are rendered as special content parts in the final `ChatMessage`.
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage, ImageContent
template = """
{% message role="user" meta={"key": "value"}%}
Hello! I am {{user_name}}. What's the difference between the following images?
{% for image in images %}
{{ image | templatize_part }}
{% endfor %}
{% endmessage %}
"""
builder = ChatPromptBuilder(template=template)
images = [
ImageContent.from_file_path("apple.jpg"),
ImageContent.from_file_path("kiwi.jpg"),
]
result = builder.run(user_name="John", images=images)
assert result["prompt"] == [
ChatMessage.from_user(
content_parts=[
"Hello! I am John. What's the difference between the following images?",
*images,
],
meta={"key": "value"},
),
]
```
### In a pipeline
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack import Pipeline
from haystack.utils import Secret
# no parameter init, we don't use any runtime template variables
prompt_builder = ChatPromptBuilder()
llm = OpenAIChatGenerator()
pipe = Pipeline()
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("prompt_builder.prompt", "llm.messages")
location = "Berlin"
language = "English"
system_message = ChatMessage.from_system(
"You are an assistant giving information to tourists in {{language}}",
)
messages = [system_message, ChatMessage.from_user("Tell me about {{location}}")]
res = pipe.run(
data={
"prompt_builder": {
"template_variables": {"location": location, "language": language},
"template": messages,
},
},
)
print(res)
```
Then, you could ask about the weather forecast for the said location. The `ChatPromptBuilder` fills in the template with the new `day_count` variable and passes it to an LLM once again:
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack import Pipeline
from haystack.utils import Secret
# no parameter init, we don't use any runtime template variables
prompt_builder = ChatPromptBuilder()
llm = OpenAIChatGenerator()
pipe = Pipeline()
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("prompt_builder.prompt", "llm.messages")
location = "Berlin"
messages = [
system_message,
ChatMessage.from_user(
"What's the weather forecast for {{location}} in the next {{day_count}} days?",
),
]
res = pipe.run(
data={
"prompt_builder": {
"template_variables": {"location": location, "day_count": "5"},
"template": messages,
},
},
)
print(res)
```
### In YAML
This is the YAML representation of the pipeline shown above. It dynamically constructs a prompt and generates an answer using a chat model.
```yaml
components:
llm:
init_parameters:
api_base_url: null
api_key:
env_vars:
- OPENAI_API_KEY
strict: true
type: env_var
generation_kwargs: {}
http_client_kwargs: null
max_retries: null
model: gpt-4o-mini
organization: null
streaming_callback: null
timeout: null
tools: null
tools_strict: false
type: haystack.components.generators.chat.openai.OpenAIChatGenerator
prompt_builder:
init_parameters:
required_variables: '*'
template: null
variables: null
type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
connection_type_validation: true
connections:
- receiver: llm.messages
sender: prompt_builder.prompt
max_runs_per_component: 100
metadata: {}
```
## Additional References
🧑🍳 Cookbook: [Advanced Prompt Customization for Anthropic](https://haystack.deepset.ai/cookbook/prompt_customization_for_anthropic)
---
// File: pipeline-components/builders/promptbuilder
# PromptBuilder
Use this component in pipelines before a Generator to render a prompt template and fill in variable values.
| | |
| --- | --- |
| **Most common position in a pipeline** | In a querying pipeline, before a [Generator](../generators.mdx) |
| **Mandatory init variables** | `template`: A prompt template string that uses Jinja2 syntax |
| **Mandatory run variables** | `**kwargs`: Any strings that should be used to render the prompt template. See [Variables](#variables) section for more details. |
| **Output variables** | `prompt`: A string that represents the rendered prompt template |
| **API reference** | [Builders](/reference/builders-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/prompt_builder.py |
| **Package name** | `haystack-ai` |
## Overview
`PromptBuilder` is initialized with a prompt template and renders it by filling in parameters passed through keyword arguments, `kwargs`. With `kwargs`, you can pass a variable number of keyword arguments so that any variable used in the prompt template can be specified with the desired value. Values for all variables appearing in the prompt template need to be provided through the `kwargs`.
The template that is provided to the `PromptBuilder` during initialization needs to conform to the [Jinja2](https://palletsprojects.com/p/jinja/) template language.
### Variables
The template variables found in the init template are used as input types for the component. By default, `required_variables` is set to `"*"`, so all variables in the template are required: if any of them is missing at runtime, the component raises an error and halts execution.
Use `required_variables` and `variables` to specify the input types and required variables:
- `required_variables`
- Defines which template variables must be provided when the component runs.
- If any required variable is missing, the component raises an error and halts execution.
- You can:
- Use `"*"` (the default) to mark all variables in the template as required, or
- Pass a list of required variable names (such as `["query"]`) to make only those required; the remaining variables are optional, or
- Pass an empty list (`[]`) or `None` to make all variables optional. Setting `None` explicitly logs a warning, since missing variables are then silently replaced with empty strings, which can lead to unintended behavior, especially in complex pipelines.
- `variables`
- Lists all variables that can appear in the template, whether required or optional.
- Optional variables that aren't provided are replaced with an empty string in the rendered prompt.
- This allows partial prompts to be constructed without errors, unless a variable is marked as required.
```python
from haystack.components.builders import PromptBuilder
# All variables required (the default, equivalent to required_variables="*")
builder = PromptBuilder(
template="Hello {{name}}! {{greeting}}",
)
# Some variables required
builder = PromptBuilder(
template="Hello {{name}}! {{greeting}}",
required_variables=["name"], # 'greeting' becomes optional
)
# All variables optional (missing ones default to empty string)
builder = PromptBuilder(
template="Hello {{name}}! {{greeting}}",
required_variables=[], # explicit None also works but logs a warning
)
```
The component only waits for the required inputs before running.
### Jinja2 Time Extension
`PromptBuilder` supports the Jinja2 TimeExtension, which allows you to work with datetime formats.
The Time Extension provides two main features:
1. A `now` tag that gives you access to the current time,
2. Date/time formatting capabilities through Python's datetime module.
To use the Jinja2 TimeExtension, you need to install a dependency with:
```shell
pip install arrow>=1.3.0
```
#### The `now` Tag
The `now` tag creates a datetime object representing the current time, which you can then store in a variable:
```jinja2
{% now 'utc' as current_time %}
The current UTC time is: {{ current_time }}
```
You can specify different timezones:
```jinja2
{% now 'America/New_York' as ny_time %}
The time in New York is: {{ ny_time }}
```
If you don't specify a timezone, your system's local timezone will be used:
```jinja2
{% now as local_time %}
Local time: {{ local_time }}
```
#### Date Formatting
You can format the datetime objects using Python's `strftime` syntax:
```jinja2
{% now as current_time %}
Formatted date: {{ current_time.strftime('%Y-%m-%d %H:%M:%S') }}
```
The common format codes are:
- `%Y`: 4-digit year (for example, 2025)
- `%m`: Month as a zero-padded number (01-12)
- `%d`: Day as a zero-padded number (01-31)
- `%H`: Hour (24-hour clock) as a zero-padded number (00-23)
- `%M`: Minute as a zero-padded number (00-59)
- `%S`: Second as a zero-padded number (00-59)
#### Example
```python
from haystack.components.builders import PromptBuilder
# Define template using Jinja-style formatting
template = """
Current date is: {% now 'UTC' %}
Thank you for providing the date
Yesterday was: {% now 'UTC' - 'days=1' %}
"""
builder = PromptBuilder(template=template)
result = builder.run()["prompt"]
```
## Usage
### On its own
Below is an example of using the `PromptBuilder` to render a prompt template and fill it with `target_language` and `snippet`. The PromptBuilder returns a prompt with the string `Translate the following context to spanish. Context: I can't speak spanish.; Translation:`.
```python
from haystack.components.builders import PromptBuilder
template = "Translate the following context to {{ target_language }}. Context: {{ snippet }}; Translation:"
builder = PromptBuilder(template=template)
builder.run(target_language="spanish", snippet="I can't speak spanish.")
```
### In a pipeline
Below is an example of a RAG pipeline where we use a `PromptBuilder` to render a custom prompt template and fill it with the contents of retrieved documents and a query. The rendered prompt is then sent to a Generator.
```python
from haystack import Pipeline, Document
from haystack.utils import Secret
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders.prompt_builder import PromptBuilder
# in a real world use case documents could come from a retriever, web, or any other source
documents = [
Document(content="Joe lives in Berlin"),
Document(content="Joe is a software engineer"),
]
prompt_template = """
Given these documents, answer the question.\nDocuments:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
\nQuestion: {{query}}
\nAnswer:
"""
p = Pipeline()
p.add_component(instance=PromptBuilder(template=prompt_template), name="prompt_builder")
p.add_component(
instance=OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
name="llm",
)
p.connect("prompt_builder", "llm")
question = "Where does Joe live?"
result = p.run({"prompt_builder": {"documents": documents, "query": question}})
print(result)
```
#### Changing the template at runtime (Prompt Engineering)
`PromptBuilder` allows you to switch the prompt template of an existing pipeline. The example below builds on top of the existing pipeline in the previous section. We are invoking the existing pipeline with a new prompt template:
```python
documents = [
Document(content="Joe lives in Berlin", meta={"name": "doc1"}),
Document(content="Joe is a software engineer", meta={"name": "doc1"}),
]
new_template = """
You are a helpful assistant.
Given these documents, answer the question.
Documents:
{% for doc in documents %}
Document {{ loop.index }}:
Document name: {{ doc.meta['name'] }}
{{ doc.content }}
{% endfor %}
Question: {{ query }}
Answer:
"""
p.run(
{
"prompt_builder": {
"documents": documents,
"query": question,
"template": new_template,
},
},
)
```
If you want to use different variables during prompt engineering than in the default template, you can do so by setting `PromptBuilder`'s variables init parameter accordingly.
#### Overwriting variables at runtime
In case you want to overwrite the values of variables, you can use `template_variables` during runtime, as shown below:
```python
language_template = """
You are a helpful assistant.
Given these documents, answer the question.
Documents:
{% for doc in documents %}
Document {{ loop.index }}:
Document name: {{ doc.meta['name'] }}
{{ doc.content }}
{% endfor %}
Question: {{ query }}
Please provide your answer in {{ answer_language | default('English') }}
Answer:
"""
p.run(
{
"prompt_builder": {
"documents": documents,
"query": question,
"template": language_template,
"template_variables": {"answer_language": "German"},
},
},
)
```
Note that `language_template` introduces `answer_language` variable which is not bound to any pipeline variable. If not set otherwise, it would use its default value, "English". In this example, we overwrite its value to "German".
The `template_variables` allows you to overwrite pipeline variables (such as documents) as well.
### In YAML
This is the YAML representation of the RAG pipeline shown above. It renders a custom prompt template by filling it with the contents of retrieved documents and a query, then sends the rendered prompt to a generator.
```yaml
components:
llm:
init_parameters:
api_base_url: null
api_key:
env_vars:
- OPENAI_API_KEY
strict: true
type: env_var
generation_kwargs: {}
http_client_kwargs: null
max_retries: null
model: gpt-5-mini
organization: null
streaming_callback: null
timeout: null
tools: null
tools_strict: false
type: haystack.components.generators.chat.openai.OpenAIChatGenerator
prompt_builder:
init_parameters:
required_variables: '*'
template: "\n Given these documents, answer the question.\nDocuments:\n \
\ {% for doc in documents %}\n {{ doc.content }}\n {% endfor %}\n\
\n \nQuestion: {{query}}\n \nAnswer:\n "
variables: null
type: haystack.components.builders.prompt_builder.PromptBuilder
connection_type_validation: true
connections:
- receiver: llm.messages
sender: prompt_builder.prompt
max_runs_per_component: 100
metadata: {}
```
## Additional References
🧑🍳 Cookbooks:
- [Advanced Prompt Customization for Anthropic](https://haystack.deepset.ai/cookbook/prompt_customization_for_anthropic)
- [Prompt Optimization with DSPy](https://haystack.deepset.ai/cookbook/prompt_optimization_with_dspy)
---
// File: pipeline-components/builders
# Builders
| Component | Description |
| --- | --- |
| [AnswerBuilder](builders/answerbuilder.mdx) | Creates `GeneratedAnswer` objects from the query and the answer. |
| [PromptBuilder](builders/promptbuilder.mdx) | Renders prompt templates with given parameters. |
| [ChatPromptBuilder](builders/chatpromptbuilder.mdx) | PromptBuilder for chat messages. |
---
// File: pipeline-components/caching/cachechecker
# CacheChecker
This component checks for the presence of documents in a Document Store based on a specified cache field.
| | |
| --- | --- |
| **Most common position in a pipeline** | Flexible |
| **Mandatory init variables** | `document_store`: A Document Store instance
`cache_field`: Name of the document's metadata field | | **Mandatory run variables** | `items`: A list of values associated with the `cache_field` in documents | | **Output variables** | `hits`: A list of documents that were found with the specified value in cache
`misses`: A list of values that could not be found | | **API reference** | [Caching](/reference/caching-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/caching/cache_checker.py | | **Package name** | `haystack-ai` |
## Overview
`CacheChecker` checks if a Document Store contains any document with a value in the `cache_field` that matches any of the values provided in the `items` input variable. It returns a dictionary with two keys: `"hits"` and `"misses"`. The values are lists of documents that were found in the cache and items that were not, respectively.
## Usage
### On its own
```python
from haystack.components.caching import CacheChecker
from haystack.document_stores.in_memory import InMemoryDocumentStore
my_doc_store = InMemoryDocumentStore()
# For URL-based caching
cache_checker = CacheChecker(document_store=my_doc_store, cache_field="url")
cache_check_results = cache_checker.run(
items=[
"https://example.com/resource",
"https://another_example.com/other_resources",
],
)
print(
cache_check_results["hits"],
) # List of Documents that were found in the cache: all of these have 'url': `cache_field`: Name of the document's metadata field | | **Mandatory run variables** | `items`: A list of values associated with the `cache_field` in documents | | **Output variables** | `hits`: A list of documents that were found with the specified value in cache
`misses`: A list of values that could not be found | | **API reference** | [Caching](/reference/caching-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/caching/cache_checker.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [`MetadataRouter`](../routers/metadatarouter.mdx) |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Langdetect](/reference/integrations-langdetect) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/langdetect |
| **Package name** | `langdetect-haystack` |
## Overview
`DocumentLanguageClassifier` classifies the language of documents and adds the detected language to their metadata. If a document's text does not match any of the languages specified at initialization, it is classified as "unmatched". By default, the classifier classifies for English (”en”) documents, with the rest being classified as “unmatched”.
The set of supported languages can be specified in the init method with the `languages` variable, using ISO codes.
To route your documents to various branches of the pipeline based on the language, use `MetadataRouter` component right after `DocumentLanguageClassifier`.
For classifying and then routing plain text using the same logic, use the `TextLanguageRouter` component instead.
## Usage
Install the `langdetect-haystack` package to use the `DocumentLanguageClassifier` component:
```shell
pip install langdetect-haystack
```
### On its own
Below, we are using the `DocumentLanguageClassifier` to classify English and German documents:
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack_integrations.components.classifiers.langdetect import (
DocumentLanguageClassifier,
)
from haystack import Document
documents = [
Document(content="Mein Name ist Jean und ich wohne in Paris."),
Document(content="Mein Name ist Mark und ich wohne in Berlin."),
Document(content="Mein Name ist Giorgio und ich wohne in Rome."),
Document(content="My name is Pierre and I live in Paris"),
Document(content="My name is Paul and I live in Berlin."),
Document(content="My name is Alessia and I live in Rome."),
]
document_classifier = DocumentLanguageClassifier(languages=["en", "de"])
document_classifier.run(documents=documents)
```
### In a pipeline
Below, we are using the `DocumentLanguageClassifier` in an indexing pipeline that indexes English and German documents into two difference indexes in an `InMemoryDocumentStore`, using embedding models for each language.
```python
from haystack import Pipeline
from haystack import Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.classifiers.langdetect import (
DocumentLanguageClassifier,
)
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
)
from haystack.components.writers import DocumentWriter
from haystack.components.routers import MetadataRouter
document_store_en = InMemoryDocumentStore()
document_store_de = InMemoryDocumentStore()
document_classifier = DocumentLanguageClassifier(languages=["en", "de"])
metadata_router = MetadataRouter(
rules={"en": {"language": {"$eq": "en"}}, "de": {"language": {"$eq": "de"}}},
)
english_embedder = SentenceTransformersDocumentEmbedder()
german_embedder = SentenceTransformersDocumentEmbedder(
model="PM-AI/bi-encoder_msmarco_bert-base_german",
)
en_writer = DocumentWriter(document_store=document_store_en)
de_writer = DocumentWriter(document_store=document_store_de)
indexing_pipeline = Pipeline()
indexing_pipeline.add_component(document_classifier, name="document_classifier")
indexing_pipeline.add_component(metadata_router, name="metadata_router")
indexing_pipeline.add_component(english_embedder, name="english_embedder")
indexing_pipeline.add_component(german_embedder, name="german_embedder")
indexing_pipeline.add_component(en_writer, name="en_writer")
indexing_pipeline.add_component(de_writer, name="de_writer")
indexing_pipeline.connect("document_classifier.documents", "metadata_router.documents")
indexing_pipeline.connect("metadata_router.en", "english_embedder.documents")
indexing_pipeline.connect("metadata_router.de", "german_embedder.documents")
indexing_pipeline.connect("english_embedder", "en_writer")
indexing_pipeline.connect("german_embedder", "de_writer")
indexing_pipeline.run(
{
"document_classifier": {
"documents": [
Document(content="This is an English sentence."),
Document(content="Dies ist ein deutscher Satz."),
],
},
},
)
```
---
// File: pipeline-components/classifiers/transformerszeroshotdocumentclassifier
# TransformersZeroShotDocumentClassifier
Classifies the documents based on the provided labels and adds them to their metadata.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [MetadataRouter](../routers/metadatarouter.mdx) |
| **Mandatory init variables** | `model`: The name or path of a Hugging Face model for zero shot document classification
`labels`: The set of possible class labels to classify each document into, for example, [`positive`, `negative`]. The labels depend on the selected model. | | **Mandatory run variables** | `documents`: A list of documents to classify | | **Output variables** | `documents`: A list of processed documents with an added `classification` metadata field | | **API reference** | [Transformers](/reference/integrations-transformers) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/transformers | | **Package name** | `transformers-haystack` |
## Overview
The `TransformersZeroShotDocumentClassifier` component performs zero-shot classification of documents based on the labels that you set and adds the predicted label to their metadata.
The component uses a Hugging Face pipeline for zero-shot classification.
To initialize the component, provide the model and the set of labels to be used for categorization.
You can additionally configure the component to allow multiple labels to be true with the `multi_label` boolean set to True.
Classification is run on the document's content field by default. If you want it to run on another field, set the`classification_field` to one of the document's metadata fields.
The classification results are stored in the `classification` dictionary within each document's metadata. If `multi_label` is set to `True`, you will find the scores for each label under the `details` key within the `classification` dictionary.
Available models for the task of zero-shot-classification are:
- `valhalla/distilbart-mnli-12-3`
- `cross-encoder/nli-distilroberta-base`
- `cross-encoder/nli-deberta-v3-xsmall`
## Usage
Install the `transformers-haystack` package to use the `TransformersZeroShotDocumentClassifier`:
```shell
pip install transformers-haystack
```
### On its own
```python
from haystack import Document
from haystack_integrations.components.classifiers.transformers import (
TransformersZeroShotDocumentClassifier,
)
documents = [
Document(id="0", content="Cats don't get teeth cavities."),
Document(id="1", content="Cucumbers can be grown in water."),
]
document_classifier = TransformersZeroShotDocumentClassifier(
model="cross-encoder/nli-deberta-v3-xsmall",
labels=["animals", "food"],
)
document_classifier.run(documents=documents)
```
### In a pipeline
The following is a pipeline that classifies documents based on predefined classification labels
retrieved from a search pipeline:
```python
from haystack import Document
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.core.pipeline import Pipeline
from haystack_integrations.components.classifiers.transformers import (
TransformersZeroShotDocumentClassifier,
)
documents = [
Document(id="0", content="Today was a nice day!"),
Document(id="1", content="Yesterday was a bad day!"),
]
document_store = InMemoryDocumentStore()
retriever = InMemoryBM25Retriever(document_store=document_store)
document_classifier = TransformersZeroShotDocumentClassifier(
model="cross-encoder/nli-deberta-v3-xsmall",
labels=["positive", "negative"],
)
document_store.write_documents(documents)
pipeline = Pipeline()
pipeline.add_component(name="retriever", instance=retriever)
pipeline.add_component(name="document_classifier", instance=document_classifier)
pipeline.connect("retriever", "document_classifier")
queries = ["How was your day today?", "How was your day yesterday?"]
expected_predictions = ["positive", "negative"]
for idx, query in enumerate(queries):
result = pipeline.run({"retriever": {"query": query, "top_k": 1}})
classified_docs = result["document_classifier"]["documents"]
assert classified_docs[0].id == str(idx)
assert (
classified_docs[0].meta["classification"]["label"] == expected_predictions[idx]
)
```
---
// File: pipeline-components/classifiers
# Classifiers
Use Classifiers to classify your documents by specific traits and update the metadata.
| Classifier | Description |
| --- | --- |
| [DocumentLanguageClassifier](classifiers/documentlanguageclassifier.mdx) | Classify documents by language. |
| [TransformersZeroShotDocumentClassifier](classifiers/transformerszeroshotdocumentclassifier.mdx) | Classify the documents based on the provided labels. |
---
// File: pipeline-components/connectors/datadogconnector
# DatadogConnector
Learn how to work with Datadog in Haystack.
`labels`: The set of possible class labels to classify each document into, for example, [`positive`, `negative`]. The labels depend on the selected model. | | **Mandatory run variables** | `documents`: A list of documents to classify | | **Output variables** | `documents`: A list of processed documents with an added `classification` metadata field | | **API reference** | [Transformers](/reference/integrations-transformers) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/transformers | | **Package name** | `transformers-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Anywhere, as it’s not connected to other components |
| **Mandatory init variables** | None. The connection to the Datadog backend is created at initialization time |
| **Output variables** | `name`: The name of the tracing component |
| **API reference** | [datadog](/reference/integrations-datadog) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/datadog |
| **Package name** | `datadog-haystack` |
## Overview
`DatadogConnector` integrates tracing capabilities into Haystack pipelines using [Datadog](https://www.datadoghq.com/), through [Datadog's tracing library `ddtrace`](https://ddtrace.readthedocs.io/en/stable/). It captures detailed information about pipeline runs, like API calls, context data, prompts, and more, so you can see the complete trace of your pipeline execution in Datadog.
Datadog tracing is enabled as soon as the `DatadogConnector` is initialized, so you only need to add it to your pipeline – it does not need to be connected to other components or to run to take effect.
You can optionally pass a `name` to identify this tracing component (it defaults to `datadog`).
### Prerequisites
These are the things that you need before working with the `DatadogConnector`:
1. A way to receive traces, such as a running [Datadog Agent](https://docs.datadoghq.com/agent/). `ddtrace` sends traces to the Datadog Agent at `localhost:8126` by default.
2. Set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true` – this will enable content tracing (inputs and outputs) in your pipelines.
3. Configure `ddtrace` through the standard mechanisms, for example the `DD_SERVICE`, `DD_ENV`, and `DD_VERSION` environment variables, or by running your application with the `ddtrace-run` command. See the [ddtrace documentation](https://ddtrace.readthedocs.io/en/stable/) for more details.
### Installation
First, install the `datadog-haystack` package to use the `DatadogConnector`:
```shell
pip install datadog-haystack
```
:::info[Usage Notice] To ensure proper tracing, always set environment variables before importing any Haystack components. This is crucial because Haystack initializes its internal tracing components during import. In the example below, we first set the environment variables and then import the relevant Haystack components. Alternatively, an even better practice is to set these environment variables in your shell before running the script. This approach keeps configuration separate from code and allows for easier management of different environments. ::: ## Usage In the example below, we are adding `DatadogConnector` to the pipeline as a _tracer_. Each pipeline run will produce a trace that includes the entire execution context, including prompts, completions, and metadata. You can then view the traces in your Datadog dashboard. ```python import os os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true" from haystack import Pipeline from haystack.components.builders import ChatPromptBuilder from haystack.components.generators.chat import OpenAIChatGenerator from haystack.dataclasses import ChatMessage from haystack_integrations.components.connectors.datadog import DatadogConnector pipe = Pipeline() pipe.add_component("tracer", DatadogConnector("Chat example")) pipe.add_component("prompt_builder", ChatPromptBuilder()) pipe.add_component("llm", OpenAIChatGenerator()) pipe.connect("prompt_builder.prompt", "llm.messages") messages = [ ChatMessage.from_system( "Always respond in German even if some input data is in other languages.", ), ChatMessage.from_user("Tell me about {{location}}"), ] response = pipe.run( data={ "prompt_builder": { "template_variables": {"location": "Berlin"}, "template": messages, }, }, ) print(response["llm"]["replies"][0]) ``` ### With an Agent ```python import os os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true" from typing import Annotated from haystack.components.agents import Agent from haystack.components.generators.chat import OpenAIChatGenerator from haystack.dataclasses import ChatMessage from haystack.tools import tool from haystack import Pipeline from haystack_integrations.components.connectors.datadog import DatadogConnector @tool def get_weather(city: Annotated[str, "The city to get weather for"]) -> str: """Get current weather information for a city.""" weather_data = { "Berlin": "18°C, partly cloudy", "New York": "22°C, sunny", "Tokyo": "25°C, clear skies", } return weather_data.get(city, f"Weather information for {city} not available") @tool def calculate( operation: Annotated[ str, "Mathematical operation: add, subtract, multiply, divide", ], a: Annotated[float, "First number"], b: Annotated[float, "Second number"], ) -> str: """Perform basic mathematical calculations.""" if operation == "add": result = a + b elif operation == "subtract": result = a - b elif operation == "multiply": result = a * b elif operation == "divide": if b == 0: return "Error: Division by zero" result = a / b else: return f"Error: Unknown operation '{operation}'" return f"The result of {a} {operation} {b} is {result}" # Create the chat generator chat_generator = OpenAIChatGenerator() # Create the agent with tools agent = Agent( chat_generator=chat_generator, tools=[get_weather, calculate], system_prompt="You are a helpful assistant with access to weather and calculator tools. Use them when needed.", exit_conditions=["text"], ) # Create the DatadogConnector for tracing datadog_connector = DatadogConnector("Agent Example") # Build the pipeline pipe = Pipeline() pipe.add_component("tracer", datadog_connector) pipe.add_component("agent", agent) # Run the pipeline response = pipe.run( data={ "agent": { "messages": [ ChatMessage.from_user( "What's the weather in Berlin and calculate 15 + 27?", ), ], }, "tracer": {}, }, ) # Display results print("Agent Response:") print(response["agent"]["last_message"].text) ``` ### Configuring the tracing backend directly Instead of using the `DatadogConnector`, you can configure the Datadog tracing backend directly by enabling a `DatadogTracer`. Make sure to set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable before importing any Haystack components. ```python import ddtrace from haystack import tracing from haystack_integrations.tracing.datadog import DatadogTracer tracing.enable_tracing(DatadogTracer(ddtrace.tracer)) ``` --- // File: pipeline-components/connectors/external-integrations-connectors # External Integrations External integrations that connect your pipelines to services by external providers. | Name | Description | | --- | --- | | [Arize AI](https://haystack.deepset.ai/integrations/arize) | Trace and evaluate your Haystack pipelines with Arize AI. | | [Arize Phoenix](https://haystack.deepset.ai/integrations/arize-phoenix) | Trace and evaluate your Haystack pipelines with Arize Phoenix. | | [Context AI](https://haystack.deepset.ai/integrations/context-ai) | Log conversations for analytics by Context.ai | | [Opik](https://haystack.deepset.ai/integrations/opik) | Trace and evaluate your Haystack pipelines with Opik platform. | | [Traceloop](https://haystack.deepset.ai/integrations/traceloop) | Evaluate and monitor the quality of your LLM apps and agents | --- // File: pipeline-components/connectors/githubfileeditor # GitHubFileEditor This is a component for editing files in GitHub repositories through the GitHub API.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a Chat Generator, or right at the beginning of a pipeline |
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
| **Mandatory run variables** | `command`: Operation type (edit, create, delete, undo)
`payload`: Command-specific parameters | | **Output variables** | `result`: String that indicates the operation result | | **API reference** | [GitHub](/reference/integrations-github) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github | | **Package name** | `github-haystack` |
## Overview
`GitHubFileEditor` supports multiple file operations, including editing existing files, creating new files, deleting files, and undoing recent changes.
There are four main commands:
- **EDIT**: Edit an existing file by replacing specific content
- **CREATE**: Create a new file with specified content
- **DELETE**: Delete an existing file
- **UNDO**: Revert the last commit if made by the same user
### Authorization
This component requires GitHub authentication with a personal access token. You can set the token using the `GITHUB_TOKEN` environment variable, or pass it directly during initialization via the `github_token` parameter.
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens). Make sure to grant the appropriate permissions for repository access and content management.
### Installation
Install the GitHub integration with pip:
```shell
pip install github-haystack
```
## Usage
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
Editing an existing file:
```python
from haystack_integrations.components.connectors.github import GitHubFileEditor, Command
editor = GitHubFileEditor(repo="owner/repo", branch="main")
result = editor.run(
command=Command.EDIT,
payload={
"path": "src/example.py",
"original": "def old_function():",
"replacement": "def new_function():",
"message": "Renamed function for clarity",
},
)
print(result)
```
```bash
{'result': 'Edit successful'}
```
Creating a new file:
```python
from haystack_integrations.components.connectors.github import GitHubFileEditor, Command
editor = GitHubFileEditor(repo="owner/repo")
result = editor.run(
command=Command.CREATE,
payload={
"path": "docs/new_file.md",
"content": "# New Documentation\n\nThis is a new file.",
"message": "Add new documentation file",
},
)
print(result)
```
```bash
{'result': 'File created successfully'}
```
---
// File: pipeline-components/connectors/githubissuecommenter
# GitHubIssueCommenter
This component posts comments to GitHub issues using the GitHub API.
`payload`: Command-specific parameters | | **Output variables** | `result`: String that indicates the operation result | | **API reference** | [GitHub](/reference/integrations-github) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github | | **Package name** | `github-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a Chat Generator that provides the comment text to post or right at the beginning of a pipeline |
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
| **Mandatory run variables** | `url`: A GitHub issue URL
`comment`: Comment text to post | | **Output variables** | `success`: Boolean indicating whether the comment was posted successfully | | **API reference** | [GitHub](/reference/integrations-github) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github | | **Package name** | `github-haystack` |
## Overview
`GitHubIssueCommenter` takes a GitHub issue URL and comment text, then posts the comment to the specified issue.
The component requires authentication with a GitHub personal access token since posting comments is an authenticated operation.
### Authorization
This component requires GitHub authentication with a personal access token. You can set the token using the `GITHUB_TOKEN` environment variable, or pass it directly during initialization via the `github_token` parameter.
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens). Make sure to grant the appropriate permissions for repository access and issue management.
### Installation
Install the GitHub integration with pip:
```shell
pip install github-haystack
```
## Usage
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
Basic usage with environment variable authentication:
```python
from haystack_integrations.components.connectors.github import GitHubIssueCommenter
commenter = GitHubIssueCommenter()
result = commenter.run(
url="https://github.com/owner/repo/issues/123",
comment="Thanks for reporting this issue! We'll look into it.",
)
print(result)
```
```bash
{'success': True}
```
### In a pipeline
The following pipeline analyzes a GitHub issue and automatically posts a response:
```python
from haystack import Pipeline
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.connectors.github import (
GitHubIssueViewer,
GitHubIssueCommenter,
)
issue_viewer = GitHubIssueViewer()
issue_commenter = GitHubIssueCommenter()
prompt_template = [
ChatMessage.from_system(
"You are a helpful assistant that analyzes GitHub issues and creates appropriate responses.",
),
ChatMessage.from_user(
"Based on the following GitHub issue:\n"
"{% for document in documents %}"
"{% if document.meta.type == 'issue' %}"
"**Issue Title:** {{ document.meta.title }}\n"
"**Issue Description:** {{ document.content }}\n"
"{% endif %}"
"{% endfor %}\n"
"Generate a helpful response comment for this issue. Keep it professional and concise.",
),
]
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
llm = OpenAIChatGenerator(model="gpt-4o-mini")
pipeline = Pipeline()
pipeline.add_component("issue_viewer", issue_viewer)
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("llm", llm)
pipeline.add_component("issue_commenter", issue_commenter)
pipeline.connect("issue_viewer.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder.prompt", "llm.messages")
pipeline.connect("llm.replies", "issue_commenter.comment")
issue_url = "https://github.com/owner/repo/issues/123"
result = pipeline.run(
data={"issue_viewer": {"url": issue_url}, "issue_commenter": {"url": issue_url}},
)
print(f"Comment posted successfully: {result['issue_commenter']['success']}")
```
```
Comment posted successfully: True
```
---
// File: pipeline-components/connectors/githubissueviewer
# GitHubIssueViewer
This component fetches and parses GitHub issues into Haystack documents.
`comment`: Comment text to post | | **Output variables** | `success`: Boolean indicating whether the comment was posted successfully | | **API reference** | [GitHub](/reference/integrations-github) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github | | **Package name** | `github-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Right at the beginning of a pipeline and before a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) that expects the content of a GitHub issue as input |
| **Mandatory run variables** | `url`: A GitHub issue URL |
| **Output variables** | `documents`: A list of documents containing the main issue and its comments |
| **API reference** | [GitHub](/reference/integrations-github) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
| **Package name** | `github-haystack` |
## Overview
`GitHubIssueViewer` takes a GitHub issue URL and returns a list of documents where:
- The first document contains the main issue content
- Subsequent documents contain the issue comments (if any)
Each document includes rich metadata such as the issue title, number, state, creation date, author, and more.
### Authorization
The component can work without authentication for public repositories, but for private repositories or to avoid rate limiting, you can provide a GitHub personal access token.
Pass the token during initialization via the `github_token` parameter, for example `github_token=Secret.from_env_var("GITHUB_TOKEN")`. This component has no default environment variable for the token.
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens).
### Installation
Install the GitHub integration with pip:
```shell
pip install github-haystack
```
## Usage
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
Basic usage without authentication:
```python
from haystack_integrations.components.connectors.github import GitHubIssueViewer
viewer = GitHubIssueViewer()
result = viewer.run(url="https://github.com/deepset-ai/haystack/issues/123")
print(result)
```
```bash
{'documents': [Document(id=3989459bbd8c2a8420a9ba7f3cd3cf79bb41d78bd0738882e57d509e1293c67a, content: 'sentence-transformers = 0.2.6.1
haystack = latest
farm = 0.4.3 latest branch
In the call to Emb...', meta: {'type': 'issue', 'title': 'SentenceTransformer no longer accepts \'gpu" as argument', 'number': 123, 'state': 'closed', 'created_at': '2020-05-28T04:49:31Z', 'updated_at': '2020-05-28T07:11:43Z', 'author': 'predoctech', 'url': 'https://github.com/deepset-ai/haystack/issues/123'}), Document(id=a8a56b9ad119244678804d5873b13da0784587773d8f839e07f644c4d02c167a, content: 'Thanks for reporting!
Fixed with #124 ', meta: {'type': 'comment', 'issue_number': 123, 'created_at': '2020-05-28T07:11:42Z', 'updated_at': '2020-05-28T07:11:42Z', 'author': 'tholor', 'url': 'https://github.com/deepset-ai/haystack/issues/123#issuecomment-635153940'})]}
```
### In a pipeline
The following pipeline fetches a GitHub issue, extracts relevant information, and generates a summary:
```python
from haystack import Pipeline
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.connectors.github import GitHubIssueViewer
# Initialize components
issue_viewer = GitHubIssueViewer()
prompt_template = [
ChatMessage.from_system("You are a helpful assistant that analyzes GitHub issues."),
ChatMessage.from_user(
"Based on the following GitHub issue and comments:\n"
"{% for document in documents %}"
"{% if document.meta.type == 'issue' %}"
"**Issue Title:** {{ document.meta.title }}\n"
"**Issue Description:** {{ document.content }}\n"
"{% else %}"
"**Comment by {{ document.meta.author }}:** {{ document.content }}\n"
"{% endif %}"
"{% endfor %}\n"
"Please provide a summary of the issue and suggest potential solutions.",
),
]
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
llm = OpenAIChatGenerator(model="gpt-4o-mini")
# Create pipeline
pipeline = Pipeline()
pipeline.add_component("issue_viewer", issue_viewer)
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("llm", llm)
# Connect components
pipeline.connect("issue_viewer.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder.prompt", "llm.messages")
# Run pipeline
issue_url = "https://github.com/deepset-ai/haystack/issues/123"
result = pipeline.run(data={"issue_viewer": {"url": issue_url}})
print(result["llm"]["replies"][0])
```
---
// File: pipeline-components/connectors/githubprcreator
# GitHubPRCreator
This component creates pull requests from a fork back to the original repository through the GitHub API.
| | |
| --- | --- |
| **Most common position in a pipeline** | At the end of a pipeline, after [GitHubRepoForker](githubrepoforker.mdx), [GitHubFileEditor](githubfileeditor.mdx) and other components that prepare changes for submission |
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
| **Mandatory run variables** | `issue_url`: GitHub issue URL
`title`: PR title
`branch`: Source branch
`base`: Target branch | | **Output variables** | `result`: String indicating the pull request creation result | | **API reference** | [GitHub](/reference/integrations-github) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github | | **Package name** | `github-haystack` |
## Overview
`GitHubPRCreator` takes a GitHub issue URL and creates a pull request from your fork to the original repository, automatically linking it to the specified issue. It's designed to work with existing forks and assumes you have already made changes in a branch.
Key features:
- **Cross-repository PRs**: Creates pull requests from your fork to the original repository
- **Issue linking**: Automatically links the PR to the specified GitHub issue
- **Draft support**: Option to create draft pull requests
- **Fork validation**: Checks that the required fork exists before creating the PR
As optional parameters, you can set `body` to provide a pull request description and the boolean parameter `draft` to open a draft pull request.
### Authorization
This component requires GitHub authentication with a personal access token from the fork owner. You can set the token using the `GITHUB_TOKEN` environment variable, or pass it directly during initialization via the `github_token` parameter.
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens). Make sure to grant the appropriate permissions for repository access and pull request creation.
### Installation
Install the GitHub integration with pip:
```shell
pip install github-haystack
```
## Usage
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
```python
from haystack_integrations.components.connectors.github import GitHubPRCreator
pr_creator = GitHubPRCreator()
result = pr_creator.run(
issue_url="https://github.com/owner/repo/issues/123",
title="Fix issue #123",
body="This PR addresses issue #123 by implementing the requested changes.",
branch="fix-123", # Branch in your fork with the changes
base="main", # Branch in original repo to merge into
)
print(result)
```
```bash
{'result': 'Pull request #456 created successfully and linked to issue #123'}
```
---
// File: pipeline-components/connectors/githubrepoforker
# GitHubRepoForker
This component forks a GitHub repository from an issue URL through the GitHub API.
`title`: PR title
`branch`: Source branch
`base`: Target branch | | **Output variables** | `result`: String indicating the pull request creation result | | **API reference** | [GitHub](/reference/integrations-github) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github | | **Package name** | `github-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Right at the beginning of a pipeline and before an [Agent](../agents-1/agent.mdx) component that expects the name of a GitHub branch as input |
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
| **Mandatory run variables** | `url`: The URL of a GitHub issue in the repository that should be forked |
| **Output variables** | `repo`: Fork repository path
`issue_branch`: Issue-specific branch name (if created) | | **API reference** | [GitHub](/reference/integrations-github) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github | | **Package name** | `github-haystack` |
## Overview
`GitHubRepoForker` takes a GitHub issue URL, extracts the repository information, creates or syncs a fork of that repository, and optionally creates an issue-specific branch. It's particularly useful for automated workflows that need to create pull requests or work with repository forks.
Key features:
- **Auto-sync**: Automatically syncs existing forks with the upstream repository
- **Branch creation**: Creates issue-specific branches (e.g., "fix-123" for issue #123)
- **Completion waiting**: Optionally waits for fork creation to complete
- **Fork management**: Handles existing forks intelligently
### Authorization
This component requires GitHub authentication with a personal access token. You can set the token using the `GITHUB_TOKEN` environment variable, or pass it directly during initialization via the `github_token` parameter.
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens). Make sure to grant the appropriate permissions for repository forking and management.
### Installation
Install the GitHub integration with pip:
```shell
pip install github-haystack
```
## Usage
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
```python
from haystack_integrations.components.connectors.github import GitHubRepoForker
forker = GitHubRepoForker()
result = forker.run(url="https://github.com/owner/repo/issues/123")
print(result)
```
```bash
{'repo': 'owner/repo', 'issue_branch': 'fix-123'}
```
---
// File: pipeline-components/connectors/githubrepoviewer
# GitHubRepoViewer
This component navigates and fetches content from GitHub repositories through the GitHub API.
`issue_branch`: Issue-specific branch name (if created) | | **API reference** | [GitHub](/reference/integrations-github) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github | | **Package name** | `github-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Right at the beginning of a pipeline and before a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) that expects the content of GitHub files as input |
| **Mandatory run variables** | `path`: Repository path to view
`repo`: Repository in owner/repo format | | **Output variables** | `documents`: A list of documents containing repository contents | | **API reference** | [GitHub](/reference/integrations-github) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github | | **Package name** | `github-haystack` |
## Overview
`GitHubRepoViewer` provides different behavior based on the path type:
- **For directories**: Returns a list of documents, one for each item (files and subdirectories),
- **For files**: Returns a single document containing the file content.
Each document includes rich metadata such as the path, type, size, and URL.
### Authorization
The component can work without authentication for public repositories, but for private repositories or to avoid rate limiting, you can provide a GitHub personal access token.
Pass the token during initialization via the `github_token` parameter, for example `github_token=Secret.from_env_var("GITHUB_TOKEN")`. This component has no default environment variable for the token.
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens).
### Installation
Install the GitHub integration with pip:
```shell
pip install github-haystack
```
## Usage
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
Viewing a directory listing:
```python
from haystack_integrations.components.connectors.github import GitHubRepoViewer
viewer = GitHubRepoViewer()
result = viewer.run(
repo="deepset-ai/haystack",
path="haystack/components",
branch="main",
)
print(result)
```
```bash
{'documents': [Document(id=..., content: 'agents', meta: {'path': 'haystack/components/agents', 'type': 'dir', 'size': 0, 'url': 'https://github.com/deepset-ai/haystack/tree/main/haystack/components/agents'}), ...]}
```
Viewing a specific file:
```python
from haystack_integrations.components.connectors.github import GitHubRepoViewer
viewer = GitHubRepoViewer(repo="deepset-ai/haystack", branch="main")
result = viewer.run(path="README.md")
print(result)
```
```bash
{'documents': [Document(id=..., content: '
## Overview
`JinaReaderConnector` interacts with Jina AI’s Reader API to process queries and output documents.
You need to select one of the following modes of operations when initializing the component:
- `read`: Processes a URL and extracts the textual content.
- `search`: Searches the web and returns textual content from the most relevant pages.
- `ground`: Performs fact-checking using a grounding engine.
You can find more information on these modes in the [Jina Reader documentation](https://jina.ai/reader/).
You can additionally control the response format from the Jina Reader API using the component’s `json_response` parameter:
- `True` (default) requests a JSON response for documents enriched with structured metadata.
- `False` requests a raw response, resulting in one document with minimal metadata.
### Authorization
The component uses a `JINA_API_KEY` environment variable by default. Otherwise, you can pass a Jina API key at initialization with `api_key` like this:
```python
reader = JinaReaderConnector(mode="read", api_key=Secret.from_token("`repo`: Repository in owner/repo format | | **Output variables** | `documents`: A list of documents containing repository contents | | **API reference** | [GitHub](/reference/integrations-github) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github | | **Package name** | `github-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Anywhere, as it’s not connected to other components |
| **Mandatory init variables** | `name`: The name of the pipeline or component to identify the tracing run |
| **Output variables** | `name`: The name of the tracing component
`trace_url`: A link to the tracing data
`trace_id`: The ID of the trace | | **API reference** | [langfuse](/reference/integrations-langfuse) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/langfuse | | **Package name** | `langfuse-haystack` |
## Overview
`LangfuseConnector` integrates tracing capabilities into Haystack pipelines using [Langfuse](https://langfuse.com/). It captures detailed information about pipeline runs, like API calls, context data, prompts, and more. Use this component to:
- Monitor model performance, such as token usage and cost.
- Find areas for pipeline improvement by identifying low-quality outputs and collecting user feedback.
- Create datasets for fine-tuning and testing from your pipeline executions.
To work with the integration, add the `LangfuseConnector` to your pipeline, run the pipeline, and then view the tracing data on the Langfuse website. Don’t connect this component to any other – `LangfuseConnector` will simply run in your pipeline’s background.
You can optionally define two more parameters when working with this component:
- `httpx_client`: An optional custom `httpx.Client` instance for Langfuse API calls. Note that custom clients are discarded when deserializing a pipeline from YAML, as HTTPX clients cannot be serialized. In such cases, Langfuse creates a default client.
- `span_handler`: An optional custom handler for processing spans. If not provided, the `DefaultSpanHandler` is used. The span handler defines how spans are created and processed, enabling customization of span types based on component types and post-processing of spans. See more details in the [Advanced Usage section](#advanced-usage) below.
### Prerequisites
These are the things that you need before working with LangfuseConnector:
1. Make sure you have an active Langfuse [account](https://cloud.langfuse.com/).
2. Set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true` – this will enable tracing in your pipelines.
3. Set the `LANGFUSE_SECRET_KEY` and `LANGFUSE_PUBLIC_KEY` environment variables with your Langfuse secret and public keys found in your account profile.
### Installation
First, install `langfuse-haystack` package to use the `LangfuseConnector`:
```shell
pip install langfuse-haystack
```
`trace_url`: A link to the tracing data
`trace_id`: The ID of the trace | | **API reference** | [langfuse](/reference/integrations-langfuse) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/langfuse | | **Package name** | `langfuse-haystack` |
:::info[Usage Notice] To ensure proper tracing, always set environment variables before importing any Haystack components. This is crucial because Haystack initializes its internal tracing components during import. In the example below, we first set the environmental variables and then import the relevant Haystack components. Alternatively, an even better practice is to set these environment variables in your shell before running the script. This approach keeps configuration separate from code and allows for easier management of different environments. ::: ## Usage In the example below, we are adding `LangfuseConnector` to the pipeline as a _tracer_. Each pipeline run will produce one trace that includes the entire execution context, including prompts, completions, and metadata. You can then view the trace by following a URL link printed in the output. ```python import os os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com" os.environ["TOKENIZERS_PARALLELISM"] = "false" os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true" from haystack.components.builders import ChatPromptBuilder from haystack.components.generators.chat import OpenAIChatGenerator from haystack.dataclasses import ChatMessage from haystack import Pipeline from haystack_integrations.components.connectors.langfuse import LangfuseConnector if __name__ == "__main__": pipe = Pipeline() pipe.add_component("tracer", LangfuseConnector("Chat example")) pipe.add_component("prompt_builder", ChatPromptBuilder()) pipe.add_component("llm", OpenAIChatGenerator()) pipe.connect("prompt_builder.prompt", "llm.messages") messages = [ ChatMessage.from_system( "Always respond in German even if some input data is in other languages.", ), ChatMessage.from_user("Tell me about {{location}}"), ] response = pipe.run( data={ "prompt_builder": { "template_variables": {"location": "Berlin"}, "template": messages, }, }, ) print(response["llm"]["replies"][0]) print(response["tracer"]["trace_url"]) ``` ### With an Agent ```python import os os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com" os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true" from typing import Annotated from haystack.components.agents import Agent from haystack.components.generators.chat import OpenAIChatGenerator from haystack.dataclasses import ChatMessage from haystack.tools import tool from haystack import Pipeline from haystack_integrations.components.connectors.langfuse import LangfuseConnector @tool def get_weather(city: Annotated[str, "The city to get weather for"]) -> str: """Get current weather information for a city.""" weather_data = { "Berlin": "18°C, partly cloudy", "New York": "22°C, sunny", "Tokyo": "25°C, clear skies", } return weather_data.get(city, f"Weather information for {city} not available") @tool def calculate( operation: Annotated[ str, "Mathematical operation: add, subtract, multiply, divide", ], a: Annotated[float, "First number"], b: Annotated[float, "Second number"], ) -> str: """Perform basic mathematical calculations.""" if operation == "add": result = a + b elif operation == "subtract": result = a - b elif operation == "multiply": result = a * b elif operation == "divide": if b == 0: return "Error: Division by zero" else: result = a / b else: return f"Error: Unknown operation '{operation}'" return f"The result of {a} {operation} {b} is {result}" if __name__ == "__main__": # Create components chat_generator = OpenAIChatGenerator() agent = Agent( chat_generator=chat_generator, tools=[get_weather, calculate], system_prompt="You are a helpful assistant with access to weather and calculator tools. Use them when needed.", exit_conditions=["text"], ) langfuse_connector = LangfuseConnector("Agent Example") # Create and run pipeline pipe = Pipeline() pipe.add_component("tracer", langfuse_connector) pipe.add_component("agent", agent) response = pipe.run( data={ "agent": { "messages": [ ChatMessage.from_user( "What's the weather in Berlin and calculate 15 + 27?", ), ], }, "tracer": {"invocation_context": {"test": "agent_with_tools"}}, }, ) print(response["agent"]["last_message"].text) print(response["tracer"]["trace_url"]) ``` ## Advanced Usage ### Customizing Langfuse Traces with SpanHandler The `SpanHandler` interface in Haystack allows you to customize how spans are created and processed for Langfuse trace creation. This enables you to log custom metrics, add tags, or integrate metadata. By extending `SpanHandler` or its default implementation, `DefaultSpanHandler`, you can define custom logic for span processing, providing precise control over what data is logged to Langfuse for tracking and analyzing pipeline executions. Here's an example: ```python from haystack_integrations.components.connectors.langfuse import LangfuseConnector from haystack_integrations.tracing.langfuse import DefaultSpanHandler, LangfuseSpan from typing import Optional class CustomSpanHandler(DefaultSpanHandler): def handle(self, span: LangfuseSpan, component_type: Optional[str]) -> None: # Custom logic to add metadata or modify span if component_type == "OpenAIChatGenerator": output = span._data.get("haystack.component.output", {}) if len(output.get("text", "")) < 10: span._span.update(level="WARNING", status_message="Response too short") # Add the custom handler to the LangfuseConnector connector = LangfuseConnector( "Custom Handler Example", span_handler=CustomSpanHandler() ) ``` --- // File: pipeline-components/connectors/oauthtokenresolver # OAuthTokenResolver Resolves an OAuth access token at pipeline runtime and emits it for downstream components such as the SharePoint and Google Drive retrievers and fetchers.
| | |
| --- | --- |
| **Most common position in a pipeline** | At the start of a pipeline, feeding `access_token` into downstream components such as [`MSSharePointRetriever`](../retrievers/mssharepointretriever.mdx) or [`GoogleDriveRetriever`](../retrievers/googledriveretriever.mdx) |
| **Mandatory init variables** | `token_source`: The strategy that resolves the access token, for example `OAuthRefreshTokenSource` |
| **Mandatory run variables** | None for config-only sources. `subject_token`: a controller-injected per-request credential, mandatory only when the source requires it (for example `OAuthTokenExchangeSource`) |
| **Output variables** | `access_token`: A bearer token string |
| **API reference** | [OAuth](/reference/integrations-oauth) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/oauth |
| **Package name** | `oauth-haystack` |
## Overview
`OAuthTokenResolver` resolves an OAuth access token when the pipeline runs and emits it on the `access_token` output socket. Downstream components – such as [`MSSharePointRetriever`](../retrievers/mssharepointretriever.mdx), [`MSSharePointFetcher`](../fetchers/mssharepointfetcher.mdx), [`GoogleDriveRetriever`](../retrievers/googledriveretriever.mdx), and [`GoogleDriveFetcher`](../fetchers/googledrivefetcher.mdx) – consume the token through a normal connection and never need to know how it was obtained.
The resolver itself is a thin wrapper. The actual work of getting a token is delegated to a pluggable **token source** that decides *where* the token comes from. This separation lets you swap authentication strategies (refresh-token grant, per-request token exchange, or a static long-lived token) without changing the rest of your pipeline.
### Token sources
You pass a token source to the resolver through the `token_source` parameter. All sources are importable from `haystack_integrations.utils.oauth`.
| Source | Use it when | Per-request input |
| --- | --- | --- |
| `OAuthRefreshTokenSource` | You have a single, fixed identity backed by a stored refresh token and want the source to exchange it for short-lived access tokens and cache them. | None |
| `OAuthTokenExchangeSource` | You serve multiple users (or run multiple replicas) and want to exchange an incoming per-request user assertion for a downstream token, with no persistent storage. Implements RFC 8693 token exchange and Microsoft's on-behalf-of flow. | `subject_token` |
| `OAuthStaticTokenSource` | Your provider issues a non-expiring token that you manage out of band (for example Slack or Notion). | None |
When the configured source needs a per-request credential (`OAuthTokenExchangeSource` sets `requires_subject_token = True`), the resolver declares a **mandatory** `subject_token` run input. This is a controller-injected credential – for example an incoming user assertion – not a value chosen by an end user. For config-only sources (`OAuthRefreshTokenSource`, `OAuthStaticTokenSource`), the resolver declares no run input and acts as a source node.
:::info[Scopes are provider-specific]
The OAuth scopes you request depend on the downstream service. For Microsoft Graph, that means scopes such as `https://graph.microsoft.com/Files.Read.All`; for Google Drive, scopes such as `https://www.googleapis.com/auth/drive.readonly`. Always consult your identity provider's documentation for the exact scope values.
:::
### Installation
Install the OAuth integration with:
```shell
pip install oauth-haystack
```
## Usage
### On its own
Resolve a token with a stored refresh token using `OAuthRefreshTokenSource`. The refresh token is read from an environment variable through the [Secret API](../../concepts/secret-management.mdx):
```python
from haystack.utils import Secret
from haystack_integrations.components.connectors.oauth import OAuthTokenResolver
from haystack_integrations.utils.oauth import OAuthRefreshTokenSource
resolver = OAuthTokenResolver(
token_source=OAuthRefreshTokenSource(
token_url="https://login.microsoftonline.com/common/oauth2/v2.0/token",
client_id="aaa-bbb-ccc",
refresh_token=Secret.from_env_var("MS_REFRESH_TOKEN"),
scopes=[
"https://graph.microsoft.com/Files.Read.All",
"offline_access",
],
),
)
access_token = resolver.run()["access_token"]
```
For a provider that issues long-lived, non-expiring tokens, use `OAuthStaticTokenSource` instead:
```python
from haystack.utils import Secret
from haystack_integrations.components.connectors.oauth import OAuthTokenResolver
from haystack_integrations.utils.oauth import OAuthStaticTokenSource
resolver = OAuthTokenResolver(
token_source=OAuthStaticTokenSource(token=Secret.from_env_var("SERVICE_TOKEN")),
)
access_token = resolver.run()["access_token"]
```
For multi-user backends, use `OAuthTokenExchangeSource`. The resolver then requires a per-request `subject_token`:
```python
from haystack_integrations.components.connectors.oauth import OAuthTokenResolver
from haystack_integrations.utils.oauth import OAuthTokenExchangeSource
resolver = OAuthTokenResolver(
token_source=OAuthTokenExchangeSource(
token_url="https://login.microsoftonline.com/
| | |
| --- | --- |
| **Most common position in a pipeline** | Anywhere, after components providing input for its run parameters |
| **Mandatory init variables** | `openapi_spec`: The OpenAPI specification for the service. Can be a URL, file path, or raw string. |
| **Mandatory run variables** | `operation_id`: The operationId from the OpenAPI spec to invoke. |
| **Output variables** | `response`: A REST service response |
| **API reference** | [OpenAPI](/reference/integrations-openapi) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/openapi |
| **Package name** | `openapi-haystack` |
## Overview
The `OpenAPIConnector` is a component within the Haystack ecosystem that allows direct invocation of REST endpoints defined in an OpenAPI (formerly Swagger) specification. It acts as a bridge between Haystack pipelines and any REST API that follows the OpenAPI standard, enabling dynamic method calls, authentication, and parameter handling.
To use the `OpenAPIConnector`, ensure that you have the `openapi-haystack` package installed:
```shell
pip install openapi-haystack
```
Unlike [OpenAPIServiceConnector](openapiserviceconnector.mdx), which works with LLMs, `OpenAPIConnector` directly calls REST endpoints using explicit input arguments.
## Usage
### On its own
You can initialize and use the `OpenAPIConnector` on its own by passing an OpenAPI specification and other parameters:
```python
from haystack.utils import Secret
from haystack_integrations.components.connectors.openapi import OpenAPIConnector
connector = OpenAPIConnector(
openapi_spec="https://bit.ly/serperdev_openapi",
credentials=Secret.from_env_var("SERPERDEV_API_KEY"),
service_kwargs={"config_factory": my_custom_config_factory},
)
response = connector.run(
operation_id="search",
arguments={"q": "Who was Nikola Tesla?"},
)
```
#### Output
The `OpenAPIConnector` returns a dictionary containing the service response:
```json
{
"response": { // here goes REST endpoint response JSON
}
}
```
### In a pipeline
The `OpenAPIConnector` can be integrated into a Haystack pipeline to interact with OpenAPI services. For example, here’s how you can link the `OpenAPIConnector` to a pipeline:
```python
from haystack import Pipeline
from haystack_integrations.components.connectors.openapi import OpenAPIConnector
from haystack.dataclasses.chat_message import ChatMessage
from haystack.utils import Secret
# Initialize the OpenAPIConnector
connector = OpenAPIConnector(
openapi_spec="https://bit.ly/serperdev_openapi",
credentials=Secret.from_env_var("SERPERDEV_API_KEY"),
)
# Create a ChatMessage from the user
user_message = ChatMessage.from_user(text="Who was Nikola Tesla?")
# Define the pipeline
pipeline = Pipeline()
pipeline.add_component("openapi_connector", connector)
# Run the pipeline
response = pipeline.run(
data={
"openapi_connector": {
"operation_id": "search",
"arguments": {"q": user_message.text},
},
},
)
# Extract the answer from the response
answer = response.get("openapi_connector", {}).get("response", {})
print(answer)
```
---
// File: pipeline-components/connectors/openapiserviceconnector
# OpenAPIServiceConnector
`OpenAPIServiceConnector` is a component that acts as an interface between the Haystack ecosystem and OpenAPI services.
:::tip[Consider using MCP instead]
These OpenAPI components are a legacy way to connect Haystack to external APIs. For most use cases, we recommend the [`MCPTool`](../../tools/mcptool.mdx) instead: it is the modern, standardized way to give your pipelines and agents access to external tools and services.
:::
| | |
| --- | --- |
| **Most common position in a pipeline** | Flexible |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects where the last message must be from the assistant and contain tool calls.
`service_openapi_spec`: OpenAPI specification of the service being invoked. It can be YAML/JSON, and all ref values must be resolved.
`service_credentials`: Authentication credentials for the service. We currently support two OpenAPI spec v3 security schemes:
1. http – for Basic, Bearer, and other HTTP authentication schemes;
2. apiKey – for API keys and cookie authentication. | | **Output variables** | `service_response`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects where each message corresponds to a tool call invocation.
If a message contains multiple tool calls, there will be multiple responses. | | **API reference** | [OpenAPI](/reference/integrations-openapi) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/openapi | | **Package name** | `openapi-haystack` |
## Overview
`OpenAPIServiceConnector` acts as a bridge between Haystack ecosystem and OpenAPI services. This component works by using information from a `ChatMessage` to dynamically invoke service methods. It handles parameter payload parsing from `ChatMessage`, service authentication, method invocation, and response formatting, making it easier to integrate OpenAPI services.
To use `OpenAPIServiceConnector`, you need to install the `openapi-haystack` package with:
```shell
pip install openapi-haystack
```
`OpenAPIServiceConnector` component doesn’t have any init parameters.
## Usage
### On its own
This component is primarily meant to be used in pipelines, as [`OpenAPIServiceToFunctions`](../converters/openapiservicetofunctions.mdx), in tandem with an LLM with tool calling capabilities, resolves the actual tool call parameters that are injected as invocation parameters for `OpenAPIServiceConnector`.
### In a pipeline
Let's say we're linking the Serper search engine to a pipeline. Here, `OpenAPIServiceConnector` uses the abilities of `OpenAPIServiceToFunctions`. `OpenAPIServiceToFunctions` first fetches and changes the [Serper's OpenAPI specification](https://bit.ly/serper_dev_spec) into function definitions that an LLM with tool calling capabilities can understand. Then, `OpenAPIServiceConnector` activates the Serper service using this specification.
More precisely, `OpenAPIServiceConnector` dynamically calls methods defined in the Serper OpenAPI specification. This involves reading chat messages to extract tool call parameters, handling authentication with the Serper service, and making the right API calls. The connector makes sure that the method call follows the Serper API requirements, such as correct formatting requests and handling responses.
Note that we used Serper just as an example here. This could be any OpenAPI-compliant service.
:::info
To run the following code snippet, note that you have to have your own Serper and OpenAI API keys.
:::
```python
import json
import requests
from typing import Any
from haystack import Pipeline
from haystack.components.converters import OutputAdapter
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.connectors.openapi import OpenAPIServiceConnector
from haystack_integrations.components.converters.openapi import (
OpenAPIServiceToFunctions,
)
def prepare_fc_params(openai_functions_schema: dict[str, Any]) -> dict[str, Any]:
return {
"tools": [{"type": "function", "function": openai_functions_schema}],
"tool_choice": {
"type": "function",
"function": {"name": openai_functions_schema["name"]},
},
}
serperdev_spec = requests.get("https://bit.ly/serper_dev_spec").json()
system_prompt = requests.get("https://bit.ly/serper_dev_system").text
user_prompt = "Why was Sam Altman ousted from OpenAI?"
pipe = Pipeline()
pipe.add_component("spec_to_functions", OpenAPIServiceToFunctions())
pipe.add_component(
"prepare_fc_adapter",
OutputAdapter(
"{{functions[0] | prepare_fc}}",
dict[str, Any],
{"prepare_fc": prepare_fc_params},
),
)
pipe.add_component("functions_llm", OpenAIChatGenerator())
pipe.add_component("openapi_connector", OpenAPIServiceConnector())
pipe.add_component(
"message_adapter",
OutputAdapter(
"{{system_message + service_response}}",
list[ChatMessage],
unsafe=True,
),
)
pipe.add_component("llm", OpenAIChatGenerator())
pipe.connect("spec_to_functions.functions", "prepare_fc_adapter.functions")
pipe.connect(
"spec_to_functions.openapi_specs",
"openapi_connector.service_openapi_spec",
)
pipe.connect("prepare_fc_adapter", "functions_llm.generation_kwargs")
pipe.connect("functions_llm.replies", "openapi_connector.messages")
pipe.connect("openapi_connector.service_response", "message_adapter.service_response")
pipe.connect("message_adapter", "llm.messages")
result = pipe.run(
data={
"functions_llm": {
"messages": [
ChatMessage.from_system("Only do tool/function calling"),
ChatMessage.from_user(user_prompt),
],
},
"openapi_connector": {
"service_credentials": serper_dev_key,
},
"spec_to_functions": {
"sources": [ByteStream.from_string(json.dumps(serperdev_spec))],
},
"message_adapter": {
"system_message": [ChatMessage.from_system(system_prompt)],
},
},
)
print(result["llm"]["replies"][0].text)
# Sam Altman was ousted from OpenAI on November 17, 2023, following
# a "deliberative review process" by the board of directors. The board concluded
# that he was not "consistently candid in his communications". However, he
# returned as CEO just days after his ouster.
```
---
// File: pipeline-components/connectors/opentelemetryconnector
# OpenTelemetryConnector
Learn how to work with OpenTelemetry in Haystack.
`service_openapi_spec`: OpenAPI specification of the service being invoked. It can be YAML/JSON, and all ref values must be resolved.
`service_credentials`: Authentication credentials for the service. We currently support two OpenAPI spec v3 security schemes:
1. http – for Basic, Bearer, and other HTTP authentication schemes;
2. apiKey – for API keys and cookie authentication. | | **Output variables** | `service_response`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects where each message corresponds to a tool call invocation.
If a message contains multiple tool calls, there will be multiple responses. | | **API reference** | [OpenAPI](/reference/integrations-openapi) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/openapi | | **Package name** | `openapi-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Anywhere, as it’s not connected to other components |
| **Mandatory init variables** | None. The tracer is created at initialization time |
| **Output variables** | `name`: The name of the tracing component |
| **API reference** | [opentelemetry](/reference/integrations-opentelemetry) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opentelemetry |
| **Package name** | `opentelemetry-haystack` |
## Overview
`OpenTelemetryConnector` integrates tracing capabilities into Haystack pipelines using [OpenTelemetry](https://opentelemetry.io/), through the [OpenTelemetry SDK](https://opentelemetry.io/docs/languages/python/). It captures detailed information about pipeline runs, like API calls, context data, prompts, and more, so you can see the complete trace of your pipeline execution in any OpenTelemetry-compatible backend.
OpenTelemetry tracing is enabled as soon as the `OpenTelemetryConnector` is initialized, so you only need to add it to your pipeline – it does not need to be connected to other components or to run to take effect.
You can optionally pass a `name` to identify this tracing component (it defaults to `opentelemetry`).
### Prerequisites
These are the things that you need before working with the `OpenTelemetryConnector`:
1. A configured OpenTelemetry `TracerProvider` with an exporter (for example, an OTLP exporter that sends traces to a collector or a backend). Set up the provider before initializing the connector.
2. Set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true` – this will enable content tracing (inputs and outputs) in your pipelines.
3. To add traces at even deeper levels, check out the available [OpenTelemetry instrumentations](https://opentelemetry.io/ecosystem/registry/?s=python), such as `opentelemetry-instrumentation-openai-v2` for tracing OpenAI requests.
### Installation
First, install the `opentelemetry-haystack` package to use the `OpenTelemetryConnector`:
```shell
pip install opentelemetry-haystack
```
:::info[Usage Notice] To ensure proper tracing, always set environment variables before importing any Haystack components. This is crucial because Haystack initializes its internal tracing components during import. In the example below, we first set the environment variables and then import the relevant Haystack components. Alternatively, an even better practice is to set these environment variables in your shell before running the script. This approach keeps configuration separate from code and allows for easier management of different environments. ::: ## Usage In the example below, we are adding `OpenTelemetryConnector` to the pipeline as a _tracer_. Each pipeline run will produce a trace that includes the entire execution context, including prompts, completions, and metadata. You can then view the traces in your OpenTelemetry backend. ```python import os os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true" from opentelemetry import trace from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter from opentelemetry.sdk.resources import Resource from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.semconv.resource import ResourceAttributes # Configure the OpenTelemetry SDK. A service name is required for most backends. resource = Resource(attributes={ResourceAttributes.SERVICE_NAME: "haystack"}) tracer_provider = TracerProvider(resource=resource) tracer_provider.add_span_processor( BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")), ) trace.set_tracer_provider(tracer_provider) from haystack import Pipeline from haystack.components.builders import ChatPromptBuilder from haystack.components.generators.chat import OpenAIChatGenerator from haystack.dataclasses import ChatMessage from haystack_integrations.components.connectors.opentelemetry import ( OpenTelemetryConnector, ) pipe = Pipeline() pipe.add_component("tracer", OpenTelemetryConnector("Chat example")) pipe.add_component("prompt_builder", ChatPromptBuilder()) pipe.add_component("llm", OpenAIChatGenerator()) pipe.connect("prompt_builder.prompt", "llm.messages") messages = [ ChatMessage.from_system( "Always respond in German even if some input data is in other languages.", ), ChatMessage.from_user("Tell me about {{location}}"), ] response = pipe.run( data={ "prompt_builder": { "template_variables": {"location": "Berlin"}, "template": messages, }, }, ) print(response["llm"]["replies"][0]) ``` ### Configuring the tracing backend directly Instead of using the `OpenTelemetryConnector`, you can configure the OpenTelemetry tracing backend directly by enabling an `OpenTelemetryTracer`. Make sure to set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable and configure your `TracerProvider` before importing any Haystack components. ```python from opentelemetry import trace from haystack import tracing from haystack_integrations.tracing.opentelemetry import OpenTelemetryTracer tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application"))) ``` --- // File: pipeline-components/connectors/weaveconnector # WeaveConnector Learn how to use Weights & Biases Weave framework for tracing and monitoring your pipeline components.
| | |
| --- | --- |
| **Most common position in a pipeline** | Anywhere, as it’s not connected to other components |
| **Mandatory init variables** | `pipeline_name`: The name of your pipeline, which will also show up in Weaver dashboard. |
| **Output variables** | `pipeline_name`: The name of the pipeline that just run |
| **API reference** | [Weave](/reference/integrations-weave) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weave |
| **Package name** | `weave-haystack` |
## Overview
This integration allows you to trace and visualize your pipeline execution in [Weights & Biases](https://wandb.ai/site/).
Information captured by the Haystack tracing tool, such as API calls, context data, and prompts, is sent to Weights & Biases, where you can see the complete trace of your pipeline execution.
### Prerequisites
You need a Weave account to use this feature. You can sign up for free at [Weights & Biases website](https://wandb.ai/site).
You will then need to set the `WANDB_API_KEY` environment variable with your Weights & Biases API key. Once logged in, you can find your API key on [your home page](https://wandb.ai/home).
Then go to `https://wandb.ai/
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx), or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | AWS credentials are resolved via `Secret` parameters or the default boto3 credential chain (environment variables, AWS config files, IAM roles). |
| **Mandatory run variables** | `sources`: A list of file paths or `ByteStream` objects |
| **Output variables** | `documents`: A list of documents
`raw_textract_response`: A list of raw responses from the Textract API | | **API reference** | [Amazon Textract](/reference/integrations-amazon_textract) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_textract | | **Package name** | `amazon-textract-haystack` |
## Overview
`AmazonTextractConverter` takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and uses AWS Textract to extract text from images and single-page PDFs. Optionally, metadata can be attached to the documents through the `meta` input parameter. You need an active AWS account with access to the Textract service to use this integration. Refer to the [AWS Textract documentation](https://docs.aws.amazon.com/textract/latest/dg/getting-started.html) to set up your AWS credentials and ensure Textract is available in your selected region.
Supported input formats: JPEG, PNG, TIFF, BMP, and single-page PDF (up to 10 MB).
By default, the component uses the standard AWS environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`, `AWS_DEFAULT_REGION`, `AWS_PROFILE`) for authentication. You can also pass these as `Secret` objects at initialization. The component falls back to the default boto3 credential chain if no explicit credentials are provided, which makes it work with IAM roles when running on AWS infrastructure.
### Operation modes
The component switches between two Textract APIs depending on how you configure it:
- **Plain text OCR (`DetectDocumentText`)** – Used when `feature_types` is not set. This is the fastest and cheapest option, extracting raw text from the document.
- **Structured analysis (`AnalyzeDocument`)** – Used when `feature_types` is set. You can pass any combination of `"TABLES"`, `"FORMS"`, `"SIGNATURES"`, and `"LAYOUT"` to extract richer structural information from the document.
### Natural-language queries
You can pass a list of natural-language questions through the `queries` parameter on `run()`. When queries are provided, the `QUERIES` feature type is added automatically and Textract returns the extracted answers in the raw response. This is useful for pulling specific fields out of forms, invoices, or receipts without writing custom parsing logic.
## Usage
You need to install the `amazon-textract-haystack` integration to use `AmazonTextractConverter`:
```shell
pip install amazon-textract-haystack
```
### On its own
Basic usage with plain text OCR:
```python
from haystack_integrations.components.converters.amazon_textract import (
AmazonTextractConverter,
)
converter = AmazonTextractConverter()
result = converter.run(sources=["document.png"])
documents = result["documents"]
```
Extracting tables and forms with `AnalyzeDocument`:
```python
from haystack_integrations.components.converters.amazon_textract import (
AmazonTextractConverter,
)
converter = AmazonTextractConverter(feature_types=["TABLES", "FORMS"])
result = converter.run(sources=["invoice.pdf"])
documents = result["documents"]
raw_responses = result["raw_textract_response"]
```
Using natural-language queries to extract specific fields:
```python
from haystack_integrations.components.converters.amazon_textract import (
AmazonTextractConverter,
)
converter = AmazonTextractConverter()
result = converter.run(
sources=["receipt.png"],
queries=["What is the patient name?", "What is the total due?"],
)
documents = result["documents"]
raw_responses = result["raw_textract_response"]
```
Passing AWS credentials explicitly:
```python
from haystack.utils import Secret
from haystack_integrations.components.converters.amazon_textract import (
AmazonTextractConverter,
)
converter = AmazonTextractConverter(
aws_access_key_id=Secret.from_env_var("AWS_ACCESS_KEY_ID"),
aws_secret_access_key=Secret.from_env_var("AWS_SECRET_ACCESS_KEY"),
aws_region_name=Secret.from_token("us-east-1"),
)
result = converter.run(sources=["document.png"])
```
### In a pipeline
Here's an example of an indexing pipeline that uses Textract to extract text from images and writes the resulting documents to a Document Store:
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.converters.amazon_textract import (
AmazonTextractConverter,
)
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", AmazonTextractConverter())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
file_names = ["document.png", "invoice.pdf"]
pipeline.run({"converter": {"sources": file_names}})
```
---
// File: pipeline-components/converters/azuredocumentintelligenceconverter
# AzureDocumentIntelligenceConverter
`AzureDocumentIntelligenceConverter` converts files to Documents using Azure's Document Intelligence service with GitHub Flavored Markdown output for better LLM/RAG integration. It supports the following file formats: PDF (both searchable and image-only), JPEG, PNG, BMP, TIFF, DOCX, XLSX, PPTX, and HTML.
`raw_textract_response`: A list of raw responses from the Textract API | | **API reference** | [Amazon Textract](/reference/integrations-amazon_textract) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_textract | | **Package name** | `amazon-textract-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx), or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | `endpoint`: The endpoint URL of your Azure Document Intelligence resource
`api_key`: The API key for Azure authentication. Can be set with `AZURE_DI_API_KEY` environment variable. | | **Mandatory run variables** | `sources`: A list of file paths or ByteStream objects | | **Output variables** | `documents`: A list of documents
`raw_azure_response`: A list of raw responses from Azure | | **API reference** | [Azure Document Intelligence](/reference/integrations-azure_doc_intelligence) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/azure_doc_intelligence | | **Package name** | `azure-doc-intelligence-haystack` |
## Overview
`AzureDocumentIntelligenceConverter` takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and uses Azure's Document Intelligence service to convert the files to a list of documents. Optionally, metadata can be attached to the documents through the `meta` input parameter. You need an active Azure account and a Document Intelligence or Cognitive Services resource to use this integration. Follow the steps described in the Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/quickstarts/get-started-sdks-rest-api) to set up your resource.
The component uses an `AZURE_DI_API_KEY` environment variable by default. Otherwise, you can pass an `api_key` at initialization — see code examples below.
This component uses the `azure-ai-documentintelligence` package (v1.0.0+) and outputs GitHub Flavored Markdown, preserving document structure such as headings, tables, and lists. Tables are rendered as inline markdown tables rather than being extracted as separate documents.
When you initialize the component, you can optionally set the `model_id`, which refers to the model you want to use. Available options include:
- `"prebuilt-document"`: General document analysis (default)
- `"prebuilt-read"`: Fast OCR for text extraction
- `"prebuilt-layout"`: Enhanced layout analysis with better table and structure detection
- Custom model IDs from your Azure resource
Refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/choose-model-feature) for a full list of available models.
:::info
This component replaces the legacy [`AzureOCRDocumentConverter`](azureocrdocumentconverter.mdx), which uses the older `azure-ai-formrecognizer` package. The `AzureDocumentIntelligenceConverter` uses the newer `azure-ai-documentintelligence` SDK and produces Markdown output instead of plain text, making it better suited for LLM and RAG applications.
:::
:::note
This component returns Markdown content. Avoid piping it through `DocumentCleaner()` with its default settings because `remove_extra_whitespaces=True` and `remove_empty_lines=True` can collapse line breaks and flatten headings, tables, and lists. Connect the converter directly to your next component, or disable those options if you need custom cleanup.
:::
## Usage
You need to install the `azure-doc-intelligence-haystack` integration to use the `AzureDocumentIntelligenceConverter`:
```shell
pip install azure-doc-intelligence-haystack
```
### On its own
```python
from pathlib import Path
from haystack_integrations.components.converters.azure_doc_intelligence import (
AzureDocumentIntelligenceConverter,
)
from haystack.utils import Secret
converter = AzureDocumentIntelligenceConverter(
endpoint="https://YOUR_RESOURCE.cognitiveservices.azure.com/",
api_key=Secret.from_env_var("AZURE_DI_API_KEY"),
)
result = converter.run(sources=[Path("my_file.pdf")])
documents = result["documents"]
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack.utils import Secret
from haystack_integrations.components.converters.azure_doc_intelligence import (
AzureDocumentIntelligenceConverter,
)
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component(
"converter",
AzureDocumentIntelligenceConverter(
endpoint="https://YOUR_RESOURCE.cognitiveservices.azure.com/",
api_key=Secret.from_env_var("AZURE_DI_API_KEY"),
),
)
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "splitter")
pipeline.connect("splitter", "writer")
file_names = ["my_file.pdf"]
pipeline.run({"converter": {"sources": file_names}})
```
---
// File: pipeline-components/converters/azureocrdocumentconverter
# AzureOCRDocumentConverter
`AzureOCRDocumentConverter` converts files to documents using Azure's Document Intelligence service. It supports the following file formats: PDF (both searchable and image-only), JPEG, PNG, BMP, TIFF, DOCX, XLSX, PPTX, and HTML.
`api_key`: The API key for Azure authentication. Can be set with `AZURE_DI_API_KEY` environment variable. | | **Mandatory run variables** | `sources`: A list of file paths or ByteStream objects | | **Output variables** | `documents`: A list of documents
`raw_azure_response`: A list of raw responses from Azure | | **API reference** | [Azure Document Intelligence](/reference/integrations-azure_doc_intelligence) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/azure_doc_intelligence | | **Package name** | `azure-doc-intelligence-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | `endpoint`: The endpoint of your Azure resource
`api_key`: The API key of your Azure resource. Can be set with `AZURE_AI_API_KEY` environment variable. | | **Mandatory run variables** | `sources`: A list of file paths | | **Output variables** | `documents`: A list of documents
`raw_azure_response`: A list of raw responses from Azure | | **API reference** | [Azure Form Recognizer](/reference/integrations-azure_form_recognizer) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/azure_form_recognizer | | **Package name** | `azure-form-recognizer-haystack` |
## Overview
`AzureOCRDocumentConverter` takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and uses Azure services to convert the files to a list of documents. Optionally, metadata can be attached to the documents through the `meta` input parameter. You need an active Azure account and a Document Intelligence or Cognitive Services resource to use this integration. Follow the steps described in the Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/quickstarts/get-started-sdks-rest-api) to set up your resource.
The component uses an `AZURE_AI_API_KEY` environment variable by default. Otherwise, you can pass an `api_key` at initialization – see code examples below.
When you initialize the component, you can optionally set the `model_id`, which refers to the model you want to use. Please refer to [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/choose-model-feature) for a list of available models. The default model is `"prebuilt-read"`.
The `AzureOCRDocumentConverter` doesn’t leave tables inline in the page text. It creates a separate `Document` for each table, with the table rendered as CSV in the document content and `preceding_context`, `following_context`, and `page` added to its metadata.
## Usage
The `AzureOCRDocumentConverter` is part of the `azure-form-recognizer-haystack` integration package. Install it with:
```shell
pip install azure-form-recognizer-haystack
```
### On its own
```python
from pathlib import Path
from haystack_integrations.components.converters.azure_form_recognizer import (
AzureOCRDocumentConverter,
)
from haystack.utils import Secret
converter = AzureOCRDocumentConverter(
endpoint="azure_resource_url",
api_key=Secret.from_token("`api_key`: The API key of your Azure resource. Can be set with `AZURE_AI_API_KEY` environment variable. | | **Mandatory run variables** | `sources`: A list of file paths | | **Output variables** | `documents`: A list of documents
`raw_azure_response`: A list of raw responses from Azure | | **API reference** | [Azure Form Recognizer](/reference/integrations-azure_form_recognizer) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/azure_form_recognizer | | **Package name** | `azure-form-recognizer-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Converters](/reference/converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/csv.py |
| **Package name** | `haystack-ai` |
## Overview
`CSVToDocument` converts one or more CSV files into a text document.
The component uses UTF-8 encoding by default, but you may specify a different encoding if needed during initialization.
You can optionally attach metadata to each document with a `meta` parameter when running the component.
## Usage
### On its own
```python
from haystack.components.converters.csv import CSVToDocument
converter = CSVToDocument()
results = converter.run(
sources=["sample.csv"],
meta={"date_added": datetime.now().isoformat()},
)
documents = results["documents"]
print(documents[0].content)
# 'col1,col2\nrow1,row1\nrow2,row2\n'
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import CSVToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", CSVToDocument())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": file_names}})
```
---
// File: pipeline-components/converters/doclingconverter
# DoclingConverter
`DoclingConverter` converts PDF, DOCX, HTML, and other document formats to Haystack Documents using [Docling](https://docling-project.github.io/docling/), a document parsing library that understands document structure including layout, tables, and headings.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx), or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of file paths, URLs, or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Docling](/reference/integrations-docling) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/docling |
| **Package name** | `docling-haystack` |
## Overview
The `DoclingConverter` takes a list of file paths, URLs, or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects and uses Docling to parse them into a rich document representation that captures layout, tables, headings, and other structural elements.
The component supports three export modes, controlled by the `export_type` parameter:
- **`ExportType.MARKDOWN`** (default): Exports each input document as a single Markdown string in one [`Document`](../../concepts/data-classes.mdx#document). Use this mode when you want to preserve the full document content as formatted text.
- **`ExportType.DOC_CHUNKS`**: Chunks each document using Docling's `HybridChunker` and returns one [`Document`](../../concepts/data-classes.mdx#document) per chunk. Chunk metadata includes structural context from Docling. Use this mode for indexing pipelines where downstream retrieval benefits from semantically coherent chunks.
- **`ExportType.JSON`**: Serializes the full Docling document to a JSON string in one [`Document`](../../concepts/data-classes.mdx#document). Use this mode when you need access to the complete structured representation.
You can customize parsing behavior by passing a pre-configured `DocumentConverter` instance via the `converter` parameter, and pass additional keyword arguments to Docling's conversion step via `convert_kwargs`. For `ExportType.MARKDOWN`, use `md_export_kwargs` to control Markdown rendering options (for example, image placeholder text). For `ExportType.DOC_CHUNKS`, provide a custom `BaseChunker` instance via the `chunker` parameter.
Document metadata is populated by a `MetaExtractor` instance. The default `MetaExtractor` adds Docling-specific metadata (chunk structure or document origin) under the `dl_meta` key. You can supply a custom `BaseMetaExtractor` implementation via the `meta_extractor` parameter. Additional metadata can be attached to all output Documents by passing a dictionary to the `meta` run parameter, or per source by passing a list of dictionaries.
## Usage
Install the Docling integration:
```shell
pip install docling-haystack
```
### On its own
```python
from haystack_integrations.components.converters.docling import (
DoclingConverter,
ExportType,
)
# Default: full document as Markdown
converter = DoclingConverter()
result = converter.run(sources=["report.pdf", "notes.docx"])
documents = result["documents"]
print(documents[0].content)
# One document per chunk
converter = DoclingConverter(export_type=ExportType.DOC_CHUNKS)
result = converter.run(sources=["report.pdf"])
documents = result["documents"]
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.converters.docling import DoclingConverter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", DoclingConverter())
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "writer")
pipeline.run({"converter": {"sources": ["report.pdf", "manual.docx"]}})
```
When you set `export_type=ExportType.DOC_CHUNKS`, `DoclingConverter` already chunks the documents, so you typically don't need a separate `DocumentSplitter` in the pipeline.
## Additional Features
### Custom chunking
Provide a custom Docling chunker to control how documents are split. The `chunker` parameter only takes effect with `ExportType.DOC_CHUNKS`:
```python
from docling.chunking import HybridChunker
from haystack_integrations.components.converters.docling import (
DoclingConverter,
ExportType,
)
chunker = HybridChunker(tokenizer="BAAI/bge-small-en-v1.5", max_tokens=256)
converter = DoclingConverter(export_type=ExportType.DOC_CHUNKS, chunker=chunker)
result = converter.run(sources=["report.pdf"])
```
### Attaching metadata
Pass a single dictionary to apply metadata to all output Documents, or a list to set metadata per source:
```python
from haystack_integrations.components.converters.docling import DoclingConverter
converter = DoclingConverter()
# Same metadata for all sources
result = converter.run(
sources=["a.pdf", "b.pdf"],
meta={"project": "research"},
)
# Per-source metadata
result = converter.run(
sources=["a.pdf", "b.pdf"],
meta=[{"title": "Report A"}, {"title": "Report B"}],
)
```
### Processing in-memory files
Pass [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects to convert files loaded into memory. Set `file_path` in the ByteStream metadata so Docling can detect the file format:
```python
from haystack.dataclasses import ByteStream
from haystack_integrations.components.converters.docling import DoclingConverter
with open("report.pdf", "rb") as f:
data = f.read()
source = ByteStream(data=data, meta={"file_path": "report.pdf"})
converter = DoclingConverter()
result = converter.run(sources=[source])
```
---
// File: pipeline-components/converters/doclingserveconverter
# DoclingServeConverter
`DoclingServeConverter` converts PDF, DOCX, HTML, and other document formats to Haystack Documents by calling a [DoclingServe](https://github.com/docling-project/docling-serve) HTTP server. Unlike the local [`DoclingConverter`](doclingconverter.mdx), this component has no heavy ML dependencies — all document parsing happens on the remote server.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx), or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of file paths, URLs, or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Docling Serve](/reference/integrations-docling_serve) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/docling_serve |
| **Package name** | `docling-serve-haystack` |
## Overview
The `DoclingServeConverter` takes a list of file paths, URLs, or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects and sends them to a running DoclingServe instance for parsing. Local files and `ByteStream` objects are uploaded to the `/v1/convert/file` endpoint; URL strings are sent to `/v1/convert/source`.
The component supports three export modes, controlled by the `export_type` parameter:
- **`ExportType.MARKDOWN`** (default): Returns the document content as a Markdown string. Use this mode when you want well-structured text output with formatting preserved.
- **`ExportType.TEXT`**: Returns plain text extracted from the document. Use this mode when you need clean, unformatted text.
- **`ExportType.JSON`**: Returns the full Docling document representation as a JSON string. Use this mode when you need access to the complete structured representation.
Each source produces one [`Document`](../../concepts/data-classes.mdx#document) in the output. Sources that fail to convert are skipped with a warning logged.
You can pass additional conversion options to the DoclingServe API via the `convert_options` parameter (for example, `{"do_ocr": True, "ocr_engine": "tesseract"}`). If the DoclingServe instance requires authentication, pass the API key via the `api_key` parameter or set the `DOCLING_SERVE_API_KEY` environment variable.
The component supports both synchronous (`run`) and asynchronous (`run_async`) execution.
## Usage
Install the Docling Serve integration:
```shell
pip install docling-serve-haystack
```
Start a DoclingServe instance locally (requires Docker):
```shell
docker run -p 5001:5001 ghcr.io/docling-project/docling-serve-cpu:latest
```
### On its own
```python
from haystack_integrations.components.converters.docling_serve import (
DoclingServeConverter,
)
# Default: Markdown output
converter = DoclingServeConverter(base_url="http://localhost:5001")
result = converter.run(sources=["report.pdf", "notes.docx"])
documents = result["documents"]
print(documents[0].content[:200])
# Plain text output
from haystack_integrations.components.converters.docling_serve import ExportType
converter = DoclingServeConverter(
base_url="http://localhost:5001",
export_type=ExportType.TEXT,
)
result = converter.run(sources=["report.pdf"])
print(result["documents"][0].content)
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.converters.docling_serve import (
DoclingServeConverter,
)
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component(
"converter",
DoclingServeConverter(base_url="http://localhost:5001"),
)
pipeline.add_component("splitter", DocumentSplitter())
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": ["report.pdf", "manual.docx"]}})
```
## Additional Features
### Converting URLs directly
Pass URL strings to convert remote documents without downloading them first:
```python
from haystack_integrations.components.converters.docling_serve import (
DoclingServeConverter,
)
converter = DoclingServeConverter(base_url="http://localhost:5001")
result = converter.run(sources=["https://arxiv.org/pdf/2602.17316"])
print(result["documents"][0].content[:200])
```
### Attaching metadata
Pass a single dictionary to apply metadata to all output Documents, or a list to set metadata per source:
```python
from haystack_integrations.components.converters.docling_serve import (
DoclingServeConverter,
)
converter = DoclingServeConverter(base_url="http://localhost:5001")
# Same metadata for all sources
result = converter.run(
sources=["a.pdf", "b.pdf"],
meta={"project": "research"},
)
# Per-source metadata
result = converter.run(
sources=["a.pdf", "b.pdf"],
meta=[{"title": "Report A"}, {"title": "Report B"}],
)
```
### Processing in-memory files
Pass [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects to convert files loaded into memory. Set `file_path` in the ByteStream metadata so DoclingServe can detect the file format:
```python
from haystack.dataclasses import ByteStream
from haystack_integrations.components.converters.docling_serve import (
DoclingServeConverter,
)
with open("report.pdf", "rb") as f:
data = f.read()
source = ByteStream(data=data, meta={"file_path": "report.pdf"})
converter = DoclingServeConverter(base_url="http://localhost:5001")
result = converter.run(sources=[source])
```
---
// File: pipeline-components/converters/documenttoimagecontent
# DocumentToImageContent
`DocumentToImageContent` extracts visual data from image or PDF file-based documents and converts them into `ImageContent` objects. These are ready for multimodal AI pipelines, including tasks like image question-answering and captioning.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a `ChatPromptBuilder` in a query pipeline |
| **Mandatory run variables** | `documents`: A list of documents to process. Each document should have metadata containing at minimum a 'file_path_meta_field' key. PDF documents additionally require a 'page_number' key to specify which page to convert. |
| **Output variables** | `image_contents`: A list of `ImageContent` objects |
| **API reference** | [Image Converters](/reference/image-converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/image/document_to_image.py |
| **Package name** | `haystack-ai` |
## Overview
`DocumentToImageContent` processes a list of documents containing image or PDF file paths and converts them into `ImageContent` objects.
- For images, it reads and encodes the file directly.
- For PDFs, it extracts the specified page (through `page_number` in metadata) and converts it to an image.
By default, it looks for the file path in the `file_path` metadata field. You can customize this with the `file_path_meta_field` parameter. The `root_path` lets you specify a common base directory for file resolution.
This component is typically used in query pipelines right before a `ChatPromptBuilder` when you would like to add Images to your user prompt.
If `size` is provided, the images will be resized while maintaining aspect ratio. This reduces file size, memory usage, and processing time, which is beneficial when working with models that have resolution constraints or when transmitting images to remote services.
## Usage
### On its own
```python
from haystack import Document
from haystack.components.converters.image.document_to_image import (
DocumentToImageContent,
)
converter = DocumentToImageContent(
file_path_meta_field="file_path",
root_path="/data/documents",
detail="high",
size=(800, 600),
)
documents = [
Document(content="Photo of a mountain", meta={"file_path": "mountain.jpg"}),
Document(
content="First page of a report",
meta={"file_path": "report.pdf", "page_number": 1},
),
]
result = converter.run(documents)
image_contents = result["image_contents"]
print(image_contents)
# [
# ImageContent(
# base64_image="/9j/4A...", mime_type="image/jpeg", detail="high",
# meta={"file_path": "mountain.jpg"}
# ),
# ImageContent(
# base64_image="/9j/4A...", mime_type="image/jpeg", detail="high",
# meta={"file_path": "report.pdf", "page_number": 1}
# )
# ]
```
### In a pipeline
You can use `DocumentToImageContent` in multimodal indexing pipelines before passing to an Embedder or captioning model.
```python
from haystack import Document, Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.converters.image.document_to_image import (
DocumentToImageContent,
)
# Query pipeline
pipeline = Pipeline()
pipeline.add_component("image_converter", DocumentToImageContent(detail="auto"))
pipeline.add_component(
"chat_prompt_builder",
ChatPromptBuilder(
required_variables=["question"],
template="""{% message role="system" %}
You are a friendly assistant that answers questions based on provided images.
{% endmessage %}
{%- message role="user" -%}
Only provide an answer to the question using the images provided.
Question: {{ question }}
Answer:
{%- for img in image_contents -%}
{{ img | templatize_part }}
{%- endfor -%}
{%- endmessage -%}
""",
),
)
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini"))
pipeline.connect("image_converter", "chat_prompt_builder.image_contents")
pipeline.connect("chat_prompt_builder", "llm")
documents = [
Document(content="Cat image", meta={"file_path": "cat.jpg"}),
Document(content="Doc intro", meta={"file_path": "paper.pdf", "page_number": 1}),
]
result = pipeline.run(
data={
"image_converter": {"documents": documents},
"chat_prompt_builder": {"question": "What color is the cat?"},
},
)
print(result)
# {
# "llm": {
# "replies": [
# ChatMessage(
# _role=
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: DOCX file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Converters](/reference/converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/docx.py |
| **Package name** | `haystack-ai` |
## Overview
The `DOCXToDocument` component converts DOCX files into documents. It takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. By defining the table format (CSV or Markdown), you can use this component to extract tables in your DOCX files. Optionally, you can attach metadata to the documents through the `meta` input parameter.
## Usage
First, install the`python-docx` package to start using this converter:
```shell
pip install python-docx
```
### On its own
```python
from haystack.components.converters.docx import DOCXToDocument, DOCXTableFormat
converter = DOCXToDocument()
# or define the table format
converter = DOCXToDocument(table_format=DOCXTableFormat.CSV)
results = converter.run(
sources=["sample.docx"],
meta={"date_added": datetime.now().isoformat()},
)
documents = results["documents"]
print(documents[0].content)
# 'This is the text from the DOCX file.'
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import DOCXToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", DOCXToDocument())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": file_names}})
```
---
// File: pipeline-components/converters/filetofilecontent
# FileToFileContent
`FileToFileContent` reads local files and converts them into `FileContent` objects. These are ready for multimodal AI pipelines that need to pass PDFs and other file types to an LLM.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a `ChatPromptBuilder` in a query pipeline |
| **Mandatory run variables** | `sources`: A list of file paths or ByteStreams |
| **Output variables** | `file_contents`: A list of `FileContent` objects |
| **API reference** | [Converters](/reference/converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/file_to_file_content.py |
| **Package name** | `haystack-ai` |
## Overview
`FileToFileContent` processes a list of file sources and converts them into `FileContent` objects that can be embedded
into a `ChatMessage` and passed to a Language Model.
Each source can be:
- A file path (string or `Path`), or
- A `ByteStream` object.
Optionally, you can provide extra provider-specific information using the `extra` parameter. This can be a single dictionary (applied to all files) or a list matching the length of `sources`.
Support for passing files to LLMs varies by provider. Some providers do not support file inputs, some restrict support
to PDF files, and others accept a wider range of file types.
## Usage
### On its own
```python
from haystack.components.converters import FileToFileContent
converter = FileToFileContent()
sources = ["document.pdf", "recording.mp3"]
result = converter.run(sources=sources)
file_contents = result["file_contents"]
print(file_contents)
# [
# FileContent(
# base64_data='JVBERi0x...', mime_type='application/pdf',
# filename='document.pdf', extra={}
# ),
# FileContent(
# base64_data='SUQzBA...', mime_type='audio/mpeg',
# filename='recording.mp3', extra={}
# )
# ]
```
### In a pipeline
Use `FileToFileContent` together with a `LinkContentFetcher` and a `ChatPromptBuilder` to build a pipeline that fetches a remote file, converts it, and passes it to an LLM.
```python
from haystack.components.converters import FileToFileContent
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.generators.chat.openai import OpenAIChatGenerator
from haystack.components.builders import ChatPromptBuilder
from haystack import Pipeline
template = """
{% message role="user"%}
{% for file in files %}
{{ file | templatize_part }}
{% endfor %}
What's the main takeaway of the following document? Just one sentence.
{% endmessage %}
"""
pipeline = Pipeline()
pipeline.add_component("fetcher", LinkContentFetcher())
pipeline.add_component("converter", FileToFileContent())
pipeline.add_component("prompt_builder", ChatPromptBuilder(template=template))
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4.1-mini"))
pipeline.connect("fetcher", "converter")
pipeline.connect("converter", "prompt_builder")
pipeline.connect("prompt_builder", "llm")
results = pipeline.run({"fetcher": {"urls": ["https://arxiv.org/pdf/2309.08632"]}})
print(results["llm"]["replies"][0].text)
# The document is a satirical paper humorously claiming that pretraining a
# small language model exclusively on evaluation benchmark test sets can achieve
# perfect performance, highlighting issues of data contamination in model
# evaluation.
```
---
// File: pipeline-components/converters/htmltodocument
# HTMLToDocument
A component that converts HTML files to documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of HTML file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Converters](/reference/converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/html.py |
| **Package name** | `haystack-ai` |
## Overview
The `HTMLToDocument` component converts HTML files into documents. It can be used in an indexing pipeline to index the contents of an HTML file into a Document Store or even in a querying pipeline after the [`LinkContentFetcher`](../fetchers/linkcontentfetcher.mdx). The `HTMLToDocument` component takes a list of HTML file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and converts the files to a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
When you initialize the component, you can optionally set `extraction_kwargs`, a dictionary containing keyword arguments to customize the extraction process. These are passed to the underlying Trafilatura `extract` function. For the full list of available arguments, see the [Trafilatura documentation](https://trafilatura.readthedocs.io/en/latest/corefunctions.html#extract).
## Usage
### On its own
```python
from pathlib import Path
from haystack.components.converters import HTMLToDocument
converter = HTMLToDocument()
docs = converter.run(sources=[Path("saved_page.html")])
```
### In a pipeline
Here's an example of an indexing pipeline that writes the contents of an HTML file into an `InMemoryDocumentStore`:
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import HTMLToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", HTMLToDocument())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": file_names}})
```
---
// File: pipeline-components/converters/imagefiletodocument
# ImageFileToDocument
Converts image file references into empty `Document` objects with associated metadata.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a component that processes images, like `SentenceTransformersDocumentImageEmbedder` or `LLMDocumentContentExtractor` |
| **Mandatory run variables** | `sources`: A list of image file paths or ByteStreams |
| **Output variables** | `documents`: A list of empty Document objects with associated metadata |
| **API reference** | [Image Converters](/reference/image-converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/image/file_to_document.py |
| **Package name** | `haystack-ai` |
## Overview
`ImageFileToDocument` converts image file sources into empty `Document` objects with associated metadata.
This component is useful in pipelines where image file paths need to be wrapped in `Document` objects to be processed by downstream components such as `SentenceTransformersDocumentImageEmbedder` or `LLMDocumentContentExtractor`.
It _does not_ extract any content from the image files, but instead creates `Document` objects with `None` as their content and attaches metadata such as file path and any user-provided values.
Each source can be:
- A file path (string or `Path`), or
- A `ByteStream` object.
Optionally, you can provide metadata using the `meta` parameter. This can be a single dictionary (applied to all documents) or a list matching the length of `sources`.
## Usage
### On its own
This component is primarily meant to be used in pipelines.
```python
from haystack.components.converters.image import ImageFileToDocument
converter = ImageFileToDocument()
sources = ["image.jpg", "another_image.png"]
result = converter.run(sources=sources)
documents = result["documents"]
print(documents)
# [Document(id=..., content=None, meta={'file_path': 'image.jpg'}),
# Document(id=..., content=None, meta={'file_path': 'another_image.png'})]
```
### In a pipeline
In the following Pipeline, image documents are created using the `ImageFileToDocument` component, then they are enriched with image embeddings and saved in the Document Store.
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Pipeline
from haystack.components.converters.image import ImageFileToDocument
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentImageEmbedder,
)
from haystack.components.writers.document_writer import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
# Create our document store
doc_store = InMemoryDocumentStore()
# Define pipeline with components
indexing_pipe = Pipeline()
indexing_pipe.add_component(
"image_converter",
ImageFileToDocument(store_full_path=True),
)
indexing_pipe.add_component(
"image_doc_embedder",
SentenceTransformersDocumentImageEmbedder(),
)
indexing_pipe.add_component("document_writer", DocumentWriter(doc_store))
indexing_pipe.connect("image_converter.documents", "image_doc_embedder.documents")
indexing_pipe.connect("image_doc_embedder.documents", "document_writer.documents")
indexing_result = indexing_pipe.run(
data={"image_converter": {"sources": ["apple.jpg", "kiwi.png"]}},
)
indexed_documents = doc_store.filter_documents()
print(f"Indexed {len(indexed_documents)} documents")
# Indexed 2 documents
```
## Additional References
🧑🍳 Cookbook: [Introduction to Multimodality](https://haystack.deepset.ai/cookbook/multimodal_intro)
---
// File: pipeline-components/converters/imagefiletoimagecontent
# ImageFileToImageContent
`ImageFileToImageContent` reads local image files and converts them into `ImageContent` objects. These are ready for multimodal AI pipelines, including tasks like image captioning, visual QA, or prompt-based generation.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a `ChatPromptBuilder` in a query pipeline |
| **Mandatory run variables** | `sources`: A list of image file paths or ByteStreams |
| **Output variables** | `image_contents`: A list of ImageContent objects |
| **API reference** | [Image Converters](/reference/image-converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/image/file_to_image.py |
| **Package name** | `haystack-ai` |
## Overview
`ImageFileToImageContent` processes a list of image sources and converts them into `ImageContent` objects. These can be used in multimodal pipelines that require base64-encoded image input.
Each source can be:
- A file path (string or `Path`), or
- A `ByteStream` object.
Optionally, you can provide metadata using the `meta` parameter. This can be a single dictionary (applied to all images) or a list matching the length of `sources`.
Use the `size` parameter to resize images while preserving aspect ratio. This reduces memory usage and transmission size, which is helpful when working with remote models or limited-resource environments.
This component is often used in query pipelines just before a `ChatPromptBuilder`.
## Usage
### On its own
```python
from haystack.components.converters.image import ImageFileToImageContent
converter = ImageFileToImageContent(detail="high", size=(800, 600))
sources = ["cat.jpg", "scenery.png"]
result = converter.run(sources=sources)
image_contents = result["image_contents"]
print(image_contents)
# [
# ImageContent(
# base64_image="/9j/4A...", mime_type="image/jpeg", detail="high",
# meta={"file_path": "cat.jpg"}
# ),
# ImageContent(
# base64_image="iVBORw0KGgo...", mime_type="image/png", detail="high",
# meta={"file_path": "scenery.png"}
# )
# ]
```
### In a pipeline
Use `ImageFileToImageContent` to supply image data to a `ChatPromptBuilder` for multimodal QA or captioning with an LLM.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.converters.image import ImageFileToImageContent
# Query pipeline
pipeline = Pipeline()
pipeline.add_component("image_converter", ImageFileToImageContent(detail="auto"))
pipeline.add_component(
"chat_prompt_builder",
ChatPromptBuilder(
required_variables=["question"],
template="""{% message role="system" %}
You are a helpful assistant that answers questions using the provided images.
{% endmessage %}
{% message role="user" %}
Question: {{ question }}
{% for img in image_contents %}
{{ img | templatize_part }}
{% endfor %}
{% endmessage %}
""",
),
)
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini"))
pipeline.connect("image_converter", "chat_prompt_builder.image_contents")
pipeline.connect("chat_prompt_builder", "llm")
sources = ["apple.jpg", "haystack-logo.png"]
result = pipeline.run(
data={
"image_converter": {"sources": sources},
"chat_prompt_builder": {"question": "Describe the Haystack logo."},
},
)
print(result)
# {
# "llm": {
# "replies": [
# ChatMessage(
# _role=
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | ONE OF, OR BOTH:
`jq_schema`: A jq filter string to extract content
`content_key`: A key string to extract document content | | **Mandatory run variables** | `sources`: A list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects | | **Output variables** | `documents`: A list of documents | | **API reference** | [Converters](/reference/converters-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/json.py | | **Package name** | `haystack-ai` |
## Overview
`JSONConverter` converts one or more JSON files into a text document.
### Parameters Overview
To initialize `JSONConverter`, you must provide either `jq_schema`, or `content_key` parameter, or both.
`jq_schema` parameter filter extracts nested data from JSON files. Refer to the [jq documentation](https://jqlang.github.io/jq/) for filter syntax. If not set, the entire JSON file is used.
The `content_key` parameter lets you specify which key in the extracted data will be the document's content.
- If both `jq_schema` and `content_key` are set, the `content_key` is searched in the data extracted by `jq_schema`. Non-object data will be skipped.
- If only `jq_schema` is set, the extracted value must be scalar; objects or arrays will be skipped.
- If only `content_key` is set, the source must be a JSON object, or it will be skipped.
Check out the [API reference](/reference/converters-api#jsonconverter) for the full list of parameters.
## Usage
You need to install the `jq` package to use this Converter:
```shell
pip install jq
```
### Example
Here is an example of simple component usage:
```python
import json
from haystack.components.converters import JSONConverter
from haystack.dataclasses import ByteStream
source = ByteStream.from_string(
json.dumps({"text": "This is the content of my document"}),
)
converter = JSONConverter(content_key="text")
results = converter.run(sources=[source])
documents = results["documents"]
print(documents[0].content)
# 'This is the content of my document'
```
In the following more complex example, we provide a `jq_schema` string to filter the JSON source files and `extra_meta_fields` to extract from the filtered data:
```python
import json
from haystack.components.converters import JSONConverter
from haystack.dataclasses import ByteStream
data = {
"laureates": [
{
"firstname": "Enrico",
"surname": "Fermi",
"motivation": "for his demonstrations of the existence of new radioactive elements produced "
"by neutron irradiation, and for his related discovery of nuclear reactions brought about by"
" slow neutrons",
},
{
"firstname": "Rita",
"surname": "Levi-Montalcini",
"motivation": "for their discoveries of growth factors",
},
],
}
source = ByteStream.from_string(json.dumps(data))
converter = JSONConverter(
jq_schema=".laureates[]",
content_key="motivation",
extra_meta_fields={"firstname", "surname"},
)
results = converter.run(sources=[source])
documents = results["documents"]
print(documents[0].content)
# 'for his demonstrations of the existence of new radioactive elements produced by
# neutron irradiation, and for his related discovery of nuclear reactions brought
# about by slow neutrons'
print(documents[0].meta)
# {'firstname': 'Enrico', 'surname': 'Fermi'}
print(documents[1].content)
# 'for their discoveries of growth factors'
print(documents[1].meta)
# {'firstname': 'Rita', 'surname': 'Levi-Montalcini'}
```
---
// File: pipeline-components/converters/kreuzbergconverter
# KreuzbergConverter
`KreuzbergConverter` converts files to Haystack Documents using [Kreuzberg](https://docs.kreuzberg.dev/), a document intelligence framework with a Rust core that extracts text from 91+ file formats entirely locally with no external API calls.
`jq_schema`: A jq filter string to extract content
`content_key`: A key string to extract document content | | **Mandatory run variables** | `sources`: A list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects | | **Output variables** | `documents`: A list of documents | | **API reference** | [Converters](/reference/converters-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/json.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx), or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of file paths, directory paths, or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Kreuzberg](/reference/integrations-kreuzberg) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/kreuzberg |
| **Package name** | `kreuzberg-haystack` |
## Overview
The `KreuzbergConverter` takes a list of file paths, directory paths, or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects and uses Kreuzberg to extract text and metadata. All processing is performed locally with no external API calls.
**Supported format categories:**
- **Documents**: PDF, DOCX, DOC, PPTX, PPT, XLSX, XLS, ODT, ODS, ODP, RTF, Pages, Keynote, Numbers, and more
- **Images (via OCR)**: PNG, JPEG, TIFF, GIF, BMP, WebP, JPEG 2000, SVG
- **Text/Markup**: Markdown, HTML, XML, LaTeX, Typst, JSON, YAML, reStructuredText, Jupyter notebooks
- **Email**: EML, MSG (with attachment extraction)
- **Archives**: ZIP, TAR, GZIP, 7Z (extracts and processes contents recursively)
- **eBooks & Academic**: EPUB, BibTeX, DocBook, JATS
The component returns one Haystack [`Document`](../../concepts/data-classes.mdx#document) per source by default. When per-page extraction or chunking is enabled, it returns one Document per page or chunk instead. Documents include rich metadata such as quality scores, detected languages, extracted keywords, table data, and PDF annotations.
By default, batch processing is enabled, leveraging Rust's rayon thread pool for parallel extraction. Set `batch=False` for sequential processing.
You can customize extraction behavior with Kreuzberg's `ExtractionConfig`, either passed directly or loaded from a TOML, YAML, or JSON configuration file via `config_path`. See the [Kreuzberg documentation](https://docs.kreuzberg.dev/) for the full configuration reference.
## Usage
Install the Kreuzberg integration:
```shell
pip install kreuzberg-haystack
```
### On its own
```python
from haystack_integrations.components.converters.kreuzberg import KreuzbergConverter
converter = KreuzbergConverter()
result = converter.run(sources=["report.pdf", "notes.docx"])
documents = result["documents"]
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.converters.kreuzberg import KreuzbergConverter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", KreuzbergConverter())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": ["report.pdf", "presentation.pptx"]}})
```
## Additional Features
### Markdown Output with OCR
Use `ExtractionConfig` to customize the output format and OCR settings:
```python
from haystack_integrations.components.converters.kreuzberg import KreuzbergConverter
from kreuzberg import ExtractionConfig, OcrConfig
converter = KreuzbergConverter(
config=ExtractionConfig(
output_format="markdown",
ocr=OcrConfig(backend="tesseract", language="eng"),
),
)
result = converter.run(sources=["scanned_document.pdf"])
documents = result["documents"]
```
### Per-Page Extraction
Create one Document per page using `PageConfig`:
```python
from haystack_integrations.components.converters.kreuzberg import KreuzbergConverter
from kreuzberg import ExtractionConfig, PageConfig
converter = KreuzbergConverter(
config=ExtractionConfig(
page=PageConfig(extract_pages=True),
),
)
result = converter.run(sources=["multipage.pdf"])
# One Document per page, each with page_number in metadata
```
### Token Reduction
Reduce output size for LLM consumption with `TokenReductionConfig`. Token reduction uses TF-IDF-based extractive summarization to identify and preserve the most important terms and phrases, progressively removing less critical content such as extra whitespace, filler words, and redundant phrases. Five levels are available: `"off"` (no reduction), `"light"` (~15%), `"moderate"` (~30%), `"aggressive"` (~50%), and `"maximum"` (>50% reduction):
```python
from haystack_integrations.components.converters.kreuzberg import KreuzbergConverter
from kreuzberg import ExtractionConfig, TokenReductionConfig
converter = KreuzbergConverter(
config=ExtractionConfig(
token_reduction=TokenReductionConfig(mode="moderate"),
),
)
```
### Config from File
Load extraction settings from a TOML, YAML, or JSON file:
```python
from haystack_integrations.components.converters.kreuzberg import KreuzbergConverter
converter = KreuzbergConverter(config_path="extraction_config.toml")
```
For the full configuration reference and format support matrix, see the [Kreuzberg documentation](https://docs.kreuzberg.dev/).
---
// File: pipeline-components/converters/libreofficefileconverter
# LibreOfficeFileConverter
A component that converts office files between formats using LibreOffice's command line interface (`soffice`).
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a document converter (e.g. [`DOCXToDocument`](./docxtodocument.mdx)) when the source files need to be converted to a format that the converter supports |
| **Mandatory run variables** | `sources`: File paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects; `output_file_type`: The target file format |
| **Output variables** | `output`: A list of [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **API reference** | [LibreOffice](/reference/integrations-libreoffice) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/libreoffice |
| **Package name** | `libreoffice-haystack` |
## Overview
`LibreOfficeFileConverter` converts office files from one format to another using LibreOffice's `soffice` command line utility. It supports a wide range of document, spreadsheet, and presentation formats and is useful when your pipeline receives files in a format that downstream converters don't support.
Unlike most converters, `LibreOfficeFileConverter` outputs [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects rather than Haystack Documents. This means it's typically chained with a document converter (such as [`DOCXToDocument`](./docxtodocument.mdx) or [`PyPDFToDocument`](./pypdftodocument.mdx)) to produce the final Documents.
**Requires LibreOffice to be installed** and available in `PATH` as `soffice`. See the [LibreOffice installation guide](https://www.libreoffice.org/get-help/install-howto/) for details.
### Supported conversions
| Category | Input formats | Possible output formats |
| --- | --- | --- |
| Documents | `doc`, `docx`, `odt`, `rtf`, `txt`, `html` | `pdf`, `docx`, `doc`, `odt`, `rtf`, `txt`, `html`, `epub` |
| Spreadsheets | `xlsx`, `xls`, `ods`, `csv` | `pdf`, `xlsx`, `xls`, `ods`, `csv`, `html` |
| Presentations | `pptx`, `ppt`, `odp` | `pdf`, `pptx`, `ppt`, `odp`, `html`, `png`, `jpg` |
This is a non-exhaustive list. See the [LibreOffice filter documentation](https://help.libreoffice.org/latest/en-GB/text/shared/guide/convertfilters.html) for all supported conversions.
## Usage
Install the LibreOffice integration:
```shell
pip install libreoffice-haystack
```
### On its own
```python
from pathlib import Path
from haystack_integrations.components.converters.libreoffice import (
LibreOfficeFileConverter,
)
converter = LibreOfficeFileConverter()
result = converter.run(sources=[Path("sample.doc")], output_file_type="docx")
bytestreams = result["output"]
```
You can also set `output_file_type` at initialization to avoid passing it on every `run()` call:
```python
converter = LibreOfficeFileConverter(output_file_type="pdf")
result = converter.run(sources=[Path("report.pptx")])
```
### In a pipeline
A common pattern is to chain `LibreOfficeFileConverter` with a document converter. The example below converts a legacy `.doc` file to `.docx` and then extracts it as a Haystack Document:
```python
from pathlib import Path
from haystack import Pipeline
from haystack.components.converters import DOCXToDocument
from haystack_integrations.components.converters.libreoffice import (
LibreOfficeFileConverter,
)
pipeline = Pipeline()
pipeline.add_component(
"libreoffice_converter",
LibreOfficeFileConverter(output_file_type="docx"),
)
pipeline.add_component("docx_converter", DOCXToDocument())
pipeline.connect("libreoffice_converter.output", "docx_converter.sources")
result = pipeline.run(
{"libreoffice_converter": {"sources": [Path("legacy_report.doc")]}},
)
documents = result["docx_converter"]["documents"]
```
---
// File: pipeline-components/converters/markdowntodocument
# MarkdownToDocument
A component that converts Markdown files to documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: Markdown file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Converters](/reference/converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/markdown.py |
| **Package name** | `haystack-ai` |
## Overview
The `MarkdownToDocument` component converts Markdown files into documents. You can use it in an indexing pipeline to index the contents of a Markdown file into a Document Store. It takes a list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
When you initialize the component, you can optionally turn off progress bars by setting `progress_bar` to `False`. If you want to convert the contents of tables into a single line, you can enable that through the `table_to_single_line` parameter.
If your Markdown files start with YAML frontmatter, set `extract_frontmatter=True` to move that data into `Document.meta` and remove it from the converted document content. Metadata passed through the `meta` input takes precedence over frontmatter keys.
## Usage
You need to install the `markdown-it-py` and `mdit_plain` packages to use the `MarkdownToDocument` component:
```shell
pip install markdown-it-py mdit_plain
```
### On its own
```python
from haystack.components.converters import MarkdownToDocument
converter = MarkdownToDocument()
docs = converter.run(sources=[Path("my_file.md")])
```
### With YAML frontmatter
Given `equity_note.md`:
```markdown
---
ticker: AAPL
source: earnings_call
date: 2026-06-12
---
# Thesis
Revenue guidance improved.
```
```python
from haystack.components.converters import MarkdownToDocument
converter = MarkdownToDocument(extract_frontmatter=True)
docs = converter.run(sources=["equity_note.md"])["documents"]
print(docs[0].meta["ticker"])
print(docs[0].content)
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import MarkdownToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", MarkdownToDocument())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": file_names}})
```
## Additional References
:notebook: Tutorial: [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline)
---
// File: pipeline-components/converters/markitdownconverter
# MarkItDownConverter
A component that converts files to Documents using Microsoft's MarkItDown library.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: File paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [MarkItDown](/reference/integrations-markitdown) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/markitdown |
| **Package name** | `markitdown-haystack` |
## Overview
`MarkItDownConverter` converts files into Haystack Documents using Microsoft's [MarkItDown](https://github.com/microsoft/markitdown) library. MarkItDown converts many file formats to Markdown, including PDF, Word (.docx), PowerPoint (.pptx), Excel (.xlsx), HTML, and more. All processing is performed locally without relying on external APIs.
The converter accepts file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of Documents. You can attach metadata to the Documents through the `meta` input parameter.
:::note
This component returns Markdown content. Avoid piping it through `DocumentCleaner()` with its default settings because `remove_extra_whitespaces=True` and `remove_empty_lines=True` can collapse line breaks and flatten headings, tables, lists, and image tags. Connect the converter directly to your next component, or disable those options if you need custom cleanup.
:::
## Usage
Install the MarkItDown integration:
```shell
pip install markitdown-haystack
```
### On its own
```python
from haystack_integrations.components.converters.markitdown import MarkItDownConverter
converter = MarkItDownConverter()
result = converter.run(sources=["document.pdf", "report.docx"])
documents = result["documents"]
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.converters.markitdown import MarkItDownConverter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", MarkItDownConverter())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": ["document.pdf", "report.docx"]}})
```
---
// File: pipeline-components/converters/mistralocrdocumentconverter
# MistralOCRDocumentConverter
`MistralOCRDocumentConverter` extracts text from documents using Mistral's OCR API, with optional structured annotations for both individual image regions and full documents. It supports various input formats including local files, URLs, and Mistral file IDs.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx), or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Mistral API key. Can be set with `MISTRAL_API_KEY` environment variable. |
| **Mandatory run variables** | `sources`: A list of document sources (file paths, ByteStreams, URLs, or Mistral chunks) |
| **Output variables** | `documents`: A list of documents
`raw_mistral_response`: A list of raw OCR responses from Mistral API | | **API reference** | [Mistral](/reference/integrations-mistral) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral | | **Package name** | `mistral-haystack` |
## Overview
The `MistralOCRDocumentConverter` takes a list of document sources and uses Mistral's OCR API to extract text from images and PDFs. It supports multiple input formats:
- **Local files**: File paths (str or Path) or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects
- **Remote resources**: Document URLs, image URLs using Mistral's `DocumentURLChunk` and `ImageURLChunk`
- **Mistral storage**: File IDs using Mistral's `FileChunk` for files previously uploaded to Mistral
The component returns one Haystack [`Document`](../../concepts/data-classes.mdx#document) per source, with all pages concatenated using form feed characters (`\f`) as separators. This format ensures compatibility with Haystack's [`DocumentSplitter`](../preprocessors/documentsplitter.mdx) for accurate page-wise splitting and overlap handling. The content is returned in markdown format, with images represented as `` tags.
By default, the component uses the `MISTRAL_API_KEY` environment variable for authentication. You can also pass an `api_key` at initialization. Local files are automatically uploaded to Mistral's storage for processing and deleted afterward (configurable with `cleanup_uploaded_files`).
When you initialize the component, you can optionally specify which pages to process, set limits on image extraction, configure minimum image sizes, or include base64-encoded images in the response. The default model is `"mistral-ocr-2505"`. See the [Mistral models documentation](https://docs.mistral.ai/getting-started/models/models_overview/) for available models.
### Structured Annotations
A unique feature of `MistralOCRDocumentConverter` is its support for structured annotations using Pydantic schemas:
- **Bounding box annotations** (`bbox_annotation_schema`): Annotate individual image regions with structured data (for example, image type, description, summary). These annotations are inserted inline after the corresponding image tags in the markdown content.
- **Document annotations** (`document_annotation_schema`): Annotate the full document with structured data (for example, language, chapter titles, URLs). These annotations are unpacked into the document's metadata with a `source_` prefix (for example, `source_language`, `source_chapter_titles`).
When annotation schemas are provided, the OCR model first extracts text and structure, then a Vision LLM analyzes the content and generates structured annotations according to your defined Pydantic schemas. Note that document annotation is limited to a maximum of 8 pages. For more details, see the [Mistral documentation on annotations](https://docs.mistral.ai/capabilities/document_ai/annotations/).
:::note
This component returns Markdown content. Avoid piping it through `DocumentCleaner()` with its default settings because `remove_extra_whitespaces=True` and `remove_empty_lines=True` can collapse line breaks and flatten headings, tables, and image tags. For page-aware chunking, connect the converter directly to `DocumentSplitter`, or disable those options if you need custom cleanup.
:::
## Usage
You need to install the `mistral-haystack` integration to use `MistralOCRDocumentConverter`:
```shell
pip install mistral-haystack
```
### On its own
Basic usage with a local file:
```python
from pathlib import Path
from haystack.utils import Secret
from haystack_integrations.components.converters.mistral import (
MistralOCRDocumentConverter,
)
converter = MistralOCRDocumentConverter(
api_key=Secret.from_env_var("MISTRAL_API_KEY"),
model="mistral-ocr-2505",
)
result = converter.run(sources=[Path("my_document.pdf")])
documents = result["documents"]
```
Processing multiple sources with different types:
```python
from pathlib import Path
from haystack.utils import Secret
from haystack_integrations.components.converters.mistral import (
MistralOCRDocumentConverter,
)
from mistralai.models import DocumentURLChunk, ImageURLChunk
converter = MistralOCRDocumentConverter(
api_key=Secret.from_env_var("MISTRAL_API_KEY"),
model="mistral-ocr-2505",
)
sources = [
Path("local_document.pdf"),
DocumentURLChunk(document_url="https://example.com/document.pdf"),
ImageURLChunk(image_url="https://example.com/receipt.jpg"),
]
result = converter.run(sources=sources)
documents = result["documents"] # List of 3 Documents
raw_responses = result["raw_mistral_response"] # List of 3 raw responses
```
Using structured annotations:
```python
from pathlib import Path
from typing import List
from pydantic import BaseModel, Field
from haystack.utils import Secret
from haystack_integrations.components.converters.mistral import (
MistralOCRDocumentConverter,
)
from mistralai.models import DocumentURLChunk
# Define schema for image region annotations
class ImageAnnotation(BaseModel):
image_type: str = Field(..., description="The type of image content")
short_description: str = Field(
...,
description="Short natural-language description",
)
summary: str = Field(..., description="Detailed summary of the image content")
# Define schema for document-level annotations
class DocumentAnnotation(BaseModel):
language: str = Field(..., description="Primary language of the document")
chapter_titles: List[str] = Field(
...,
description="Detected chapter or section titles",
)
urls: List[str] = Field(..., description="URLs found in the text")
converter = MistralOCRDocumentConverter(
api_key=Secret.from_env_var("MISTRAL_API_KEY"),
model="mistral-ocr-2505",
)
sources = [DocumentURLChunk(document_url="https://example.com/report.pdf")]
result = converter.run(
sources=sources,
bbox_annotation_schema=ImageAnnotation,
document_annotation_schema=DocumentAnnotation,
)
documents = result["documents"]
# Document metadata will include:
# - source_language: extracted from DocumentAnnotation
# - source_chapter_titles: extracted from DocumentAnnotation
# - source_urls: extracted from DocumentAnnotation
# Document content will include inline image annotations
```
### In a pipeline
Here's an example of an indexing pipeline that processes PDFs with OCR and writes them to a Document Store:
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack.utils import Secret
from haystack_integrations.components.converters.mistral import (
MistralOCRDocumentConverter,
)
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component(
"converter",
MistralOCRDocumentConverter(
api_key=Secret.from_env_var("MISTRAL_API_KEY"),
model="mistral-ocr-2505",
),
)
pipeline.add_component("splitter", DocumentSplitter(split_by="page", split_length=1))
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "splitter")
pipeline.connect("splitter", "writer")
file_paths = ["invoice.pdf", "receipt.jpg", "contract.pdf"]
pipeline.run({"converter": {"sources": file_paths}})
```
---
// File: pipeline-components/converters/msgtodocument
# MSGToDocument
Converts Microsoft Outlook .msg files to documents.
`raw_mistral_response`: A list of raw OCR responses from Mistral API | | **API reference** | [Mistral](/reference/integrations-mistral) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral | | **Package name** | `mistral-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of .msg file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents
`attachments`: A list of ByteStream objects representing file attachments | | **API reference** | [Converters](/reference/converters-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/msg.py | | **Package name** | `haystack-ai` |
## Overview
The `MSGToDocument` component converts Microsoft Outlook `.msg` files into documents. This component extracts the email metadata (such as sender, recipients, CC, BCC, subject) and body content. Additionally, any file attachments within the `.msg` file are extracted as `ByteStream` objects.
## Usage
First, install the `python-oxmsg` package to start using this converter:
```
pip install python-oxmsg
```
### On its own
```python
from haystack.components.converters.msg import MSGToDocument
from datetime import datetime
converter = MSGToDocument()
results = converter.run(
sources=["sample.msg"],
meta={"date_added": datetime.now().isoformat()},
)
documents = results["documents"]
attachments = results["attachments"]
print(documents[0].content)
```
### In a pipeline
The following setup enables efficient extraction, preprocessing, and indexing of `.msg` email files within a Haystack pipeline:
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.routers import FileTypeRouter
from haystack.components.converters import MSGToDocument
from haystack.components.writers import DocumentWriter
router = FileTypeRouter(mime_types=["application/vnd.ms-outlook"])
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("router", router)
pipeline.add_component("converter", MSGToDocument())
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("router.application/vnd.ms-outlook", "converter.sources")
pipeline.connect("converter.documents", "writer.documents")
file_names = ["email1.msg", "email2.msg"]
pipeline.run({"converter": {"sources": file_names}})
```
---
// File: pipeline-components/converters/multifileconverter
# MultiFileConverter
Converts CSV, DOCX, HTML, JSON, MD, PPTX, PDF, TXT, and XSLX files to documents.
`attachments`: A list of ByteStream objects representing file attachments | | **API reference** | [Converters](/reference/converters-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/msg.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before PreProcessors , or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of file paths or ByteStream objects |
| **Output variables** | `documents`: A list of converted documents
`unclassified`: A list of file paths or byte streams whose MIME type isn't supported
`failed`: A list of file paths or byte streams that couldn't be processed, for example a path that doesn't exist | | **API reference** | [Converters](/reference/converters-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/multi_file_converter.py | | **Package name** | `haystack-ai` |
## Overview
`MultiFileConverter` converts input files of various file types into documents.
It is a SuperComponent that combines a [`FileTypeRouter`](../routers/filetyperouter.mdx), nine converters and a [`DocumentJoiner`](../joiners/documentjoiner.mdx) into a single component.
### Parameters
To initialize `MultiFileConverter`, there are no mandatory parameters. Optionally, you can provide `encoding` and `json_content_key` parameters.
The `json_content_key` parameter lets you specify for the JSON files which key in the extracted data will be the document's content. The parameter is passed on to the underlying [`JSONConverter`](jsonconverter.mdx) component.
The `encoding` parameter lets you specify the default encoding of the TXT, CSV, and MD files. If you don't provide any value, the component uses `utf-8` by default. Note that if the encoding is specified in the metadata of an input ByteStream, it will override this parameter's setting. The parameter is passed on to the underlying [`TextFileToDocument`](textfiletodocument.mdx) and [`CSVToDocument`](csvtodocument.mdx) components.
## Usage
Install dependencies for all supported file types to use the `MultiFileConverter`:
```shell
pip install pypdf trafilatura python-pptx python-docx jq openpyxl tabulate pandas
```
### On its own
```python
from haystack.components.converters import MultiFileConverter
converter = MultiFileConverter()
converter.run(sources=["test.txt", "test.pdf"], meta={})
```
### In a pipeline
You can also use `MultiFileConverter` in your indexing pipeline.
```python
from haystack import Pipeline
from haystack.components.converters import MultiFileConverter
from haystack.components.preprocessors import DocumentPreprocessor
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", MultiFileConverter())
pipeline.add_component("preprocessor", DocumentPreprocessor())
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "preprocessor")
pipeline.connect("preprocessor", "writer")
result = pipeline.run(data={"sources": ["test.txt", "test.pdf"]})
print(result)
# {'writer': {'documents_written': 3}}
```
---
// File: pipeline-components/converters/openapiservicetofunctions
# OpenAPIServiceToFunctions
`OpenAPIServiceToFunctions` is a component that transforms OpenAPI service specifications into a format compatible with LLM tool calling.
:::tip[Consider using MCP instead]
These OpenAPI components are a legacy way to connect Haystack to external APIs. For most use cases, we recommend the [`MCPTool`](../../tools/mcptool.mdx) instead: it is the modern, standardized way to give your pipelines and agents access to external tools and services.
:::
`unclassified`: A list of file paths or byte streams whose MIME type isn't supported
`failed`: A list of file paths or byte streams that couldn't be processed, for example a path that doesn't exist | | **API reference** | [Converters](/reference/converters-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/multi_file_converter.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Flexible |
| **Mandatory run variables** | `sources`: A list of OpenAPI specification sources, which can be file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `functions`: A list of JSON function definitions objects. For each path definition in OpenAPI specification, a corresponding function definition is generated.
`openapi_specs`: A list of JSON/YAML objects with references resolved. Such OpenAPI spec (with references resolved) can, in turn, be used as input to OpenAPIServiceConnector. | | **API reference** | [OpenAPI](/reference/integrations-openapi) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/openapi | | **Package name** | `openapi-haystack` |
## Overview
`OpenAPIServiceToFunctions` transforms OpenAPI service specifications into a function calling format suitable for LLM tool calling. It takes an OpenAPI specification, processes it to extract function definitions, and formats these definitions to be compatible with LLM tool calling.
`OpenAPIServiceToFunctions` is valuable when used together with [`OpenAPIServiceConnector`](../connectors/openapiserviceconnector.mdx) component. It converts OpenAPI specifications into function definitions, allowing `OpenAPIServiceConnector` to handle input parameters for the OpenAPI specification and facilitate their use in REST API calls through `OpenAPIServiceConnector`.
To use `OpenAPIServiceToFunctions`, you need to install the `openapi-haystack` package with:
```shell
pip install openapi-haystack
```
`OpenAPIServiceToFunctions` component doesn’t have any init parameters.
## Usage
### On its own
This component is primarily meant to be used in pipelines. Using this component alone is useful when you want to convert OpenAPI specification into function definitions and then perhaps save them in a file and subsequently use them for tool calling.
### In a pipeline
In a pipeline context, `OpenAPIServiceToFunctions` is most valuable when used alongside `OpenAPIServiceConnector`. For instance, let’s consider integrating [serper.dev](http://serper.dev/) search engine bridge into a pipeline. `OpenAPIServiceToFunctions` retrieves the OpenAPI specification of Serper from https://bit.ly/serper_dev_spec, converts this specification into function definitions that an LLM with tool calling capabilities can understand, and then seamlessly passes these definitions as `generation_kwargs` to the Chat Generator component.
:::info
To run the following code snippet, note that you have to have your own Serper and OpenAI API keys.
:::
```python
import json
import requests
from typing import Any
from haystack import Pipeline
from haystack.components.converters import OutputAdapter
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.connectors.openapi import OpenAPIServiceConnector
from haystack_integrations.components.converters.openapi import (
OpenAPIServiceToFunctions,
)
def prepare_fc_params(openai_functions_schema: dict[str, Any]) -> dict[str, Any]:
return {
"tools": [{"type": "function", "function": openai_functions_schema}],
"tool_choice": {
"type": "function",
"function": {"name": openai_functions_schema["name"]},
},
}
serperdev_spec = requests.get("https://bit.ly/serper_dev_spec").json()
system_prompt = requests.get("https://bit.ly/serper_dev_system").text
user_prompt = "Why was Sam Altman ousted from OpenAI?"
pipe = Pipeline()
pipe.add_component("spec_to_functions", OpenAPIServiceToFunctions())
pipe.add_component(
"prepare_fc_adapter",
OutputAdapter(
"{{functions[0] | prepare_fc}}",
dict[str, Any],
{"prepare_fc": prepare_fc_params},
),
)
pipe.add_component("functions_llm", OpenAIChatGenerator())
pipe.add_component("openapi_connector", OpenAPIServiceConnector())
pipe.add_component(
"message_adapter",
OutputAdapter(
"{{system_message + service_response}}",
list[ChatMessage],
unsafe=True,
),
)
pipe.add_component("llm", OpenAIChatGenerator())
pipe.connect("spec_to_functions.functions", "prepare_fc_adapter.functions")
pipe.connect(
"spec_to_functions.openapi_specs",
"openapi_connector.service_openapi_spec",
)
pipe.connect("prepare_fc_adapter", "functions_llm.generation_kwargs")
pipe.connect("functions_llm.replies", "openapi_connector.messages")
pipe.connect("openapi_connector.service_response", "message_adapter.service_response")
pipe.connect("message_adapter", "llm.messages")
result = pipe.run(
data={
"functions_llm": {
"messages": [
ChatMessage.from_system("Only do tool/function calling"),
ChatMessage.from_user(user_prompt),
],
},
"openapi_connector": {
"service_credentials": serper_dev_key,
},
"spec_to_functions": {
"sources": [ByteStream.from_string(json.dumps(serperdev_spec))],
},
"message_adapter": {
"system_message": [ChatMessage.from_system(system_prompt)],
},
},
)
print(result["llm"]["replies"][0].text)
# Sam Altman was ousted from OpenAI on November 17, 2023, following
# a "deliberative review process" by the board of directors. The board concluded
# that he was not "consistently candid in his communications". However, he
# returned as CEO just days after his ouster.
```
---
// File: pipeline-components/converters/outputadapter
# OutputAdapter
This component helps the output of one component fit smoothly into the input of another. It uses Jinja expressions to define how this adaptation occurs.
`openapi_specs`: A list of JSON/YAML objects with references resolved. Such OpenAPI spec (with references resolved) can, in turn, be used as input to OpenAPIServiceConnector. | | **API reference** | [OpenAPI](/reference/integrations-openapi) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/openapi | | **Package name** | `openapi-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Flexible |
| **Mandatory init variables** | `template`: A Jinja template string that defines how to adapt the data
`output_type`: Type alias that this instance will return | | **Mandatory run variables** | `**kwargs`: Input variables to be used in Jinja expression. See [Variables](#variables) section for more details. | | **Output variables** | The output is specified under the `output` key dictionary | | **API reference** | [Converters](/reference/converters-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/output_adapter.py | | **Package name** | `haystack-ai` |
## Overview
To use `OutputAdapter`, you need to specify the adaptation rule that includes:
- `template`: A Jinja template string that defines how to adapt the input data.
- `output_type`: The type of the output data (such as `str`, `List[int]`..). This doesn't change the actual output type and is only needed to validate connection with other components.
- `custom_filters`: An optional dictionary of custom Jinja filters to be used in the template.
### Variables
The `OutputAdapter` requires all template variables to be present before running and raises an error if any template variable is missing at pipeline connect time.
```python
from haystack.components.converters import OutputAdapter
adapter = OutputAdapter(template="Hello {{name}}!", output_type=str)
```
### Unsafe behavior
The `OutputAdapter` internally renders the `template` using Jinja, and by default, this is safe behavior. However, it limits the output types to strings, bytes, numbers, tuples, lists, dicts, sets, booleans, `None`, and `Ellipsis` (`...`), as well as any combination of these structures.
If you want to use other types such as `ChatMessage`, `Document`, or `Answer`, you must enable unsafe template rendering by setting the `unsafe` init argument to `True`.
Be cautious, as enabling this can be unsafe and may lead to remote code execution if the `template` is a string customizable by the end user.
## Usage
### On its own
This component is primarily meant to be used in pipelines.
In this example, `OutputAdapter` simply outputs the content field of the first document in the arrays of documents:
```python
from haystack import Document
from haystack.components.converters import OutputAdapter
adapter = OutputAdapter(template="{{ documents[0].content }}", output_type=str)
input_data = {"documents": [Document(content="Test content")]}
expected_output = {"output": "Test content"}
assert adapter.run(**input_data) == expected_output
```
### In a pipeline
The example below demonstrates a straightforward pipeline that uses the `OutputAdapter` to capitalize the first document in the list. If needed, you can also utilize the predefined Jinja [filters](https://jinja.palletsprojects.com/en/3.1.x/templates/#builtin-filters).
```python
from haystack import Pipeline, component, Document
from haystack.components.converters import OutputAdapter
@component
class DocumentProducer:
@component.output_types(documents=dict)
def run(self):
return {"documents": [Document(content="haystack")]}
pipe = Pipeline()
pipe.add_component(
name="output_adapter",
instance=OutputAdapter(
template="{{ documents[0].content | capitalize}}",
output_type=str,
),
)
pipe.add_component(name="document_producer", instance=DocumentProducer())
pipe.connect("document_producer", "output_adapter")
result = pipe.run(data={})
assert result["output_adapter"]["output"] == "Haystack"
```
You can also define your own custom filters, which can then be added to an `OutputAdapter` instance through its init method and used in templates. Here’s an example of this approach:
```python
from haystack import Pipeline, component, Document
from haystack.components.converters import OutputAdapter
def reverse_string(s):
return s[::-1]
@component
class DocumentProducer:
@component.output_types(documents=dict)
def run(self):
return {"documents": [Document(content="haystack")]}
pipe = Pipeline()
pipe.add_component(
name="output_adapter",
instance=OutputAdapter(
template="{{ documents[0].content | reverse_string}}",
output_type=str,
custom_filters={"reverse_string": reverse_string},
),
)
pipe.add_component(name="document_producer", instance=DocumentProducer())
pipe.connect("document_producer", "output_adapter")
result = pipe.run(data={})
assert result["output_adapter"]["output"] == "kcatsyah"
```
---
// File: pipeline-components/converters/paddleocrvldocumentconverter
# PaddleOCRVLDocumentConverter
`PaddleOCRVLDocumentConverter` extracts text from documents using PaddleOCR's large model document parsing API. PaddleOCR-VL is used behind the scenes. For more information, please refer to the [PaddleOCR-VL documentation](https://www.paddleocr.ai/latest/en/version3.x/algorithm/PaddleOCR-VL/PaddleOCR-VL.html).
`output_type`: Type alias that this instance will return | | **Mandatory run variables** | `**kwargs`: Input variables to be used in Jinja expression. See [Variables](#variables) section for more details. | | **Output variables** | The output is specified under the `output` key dictionary | | **API reference** | [Converters](/reference/converters-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/output_adapter.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx), or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | `api_url`: The URL of the PaddleOCR-VL API.
`access_token`: The AI Studio access token. Can be set with `AISTUDIO_ACCESS_TOKEN` environment variable. | | **Mandatory run variables** | `sources`: A list of image or PDF file paths or ByteStream objects. | | **Output variables** | `documents`: A list of documents.
`raw_paddleocr_responses`: A list of raw OCR responses from PaddleOCR API. | | **API reference** | [PaddleOCR](/reference/integrations-paddleocr) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/paddleocr | | **Package name** | `paddleocr-haystack` |
## Overview
The `PaddleOCRVLDocumentConverter` takes a list of document sources and uses PaddleOCR's large model document parsing API to extract text from images and PDFs. It supports both images and PDF files.
The component returns one Haystack [`Document`](../../concepts/data-classes.mdx#document) per source, with all pages concatenated using form feed characters (`\f`) as separators. This format ensures compatibility with Haystack's [`DocumentSplitter`](../preprocessors/documentsplitter.mdx) for accurate page-wise splitting and overlap handling. The content is returned in markdown format, with images represented as `` tags.
The component takes `api_url` as a required parameter. To obtain the API URL, visit the [PaddleOCR official website](https://aistudio.baidu.com/paddleocr), click the **API** button, choose the example code for PaddleOCR-VL, and copy the `API_URL`.
By default, the component uses the `AISTUDIO_ACCESS_TOKEN` environment variable for authentication. You can also pass an `access_token` at initialization. The AI Studio access token can be obtained from [this page](https://aistudio.baidu.com/account/accessToken).
`raw_paddleocr_responses` can be useful while tuning layout thresholds, prompt settings, or Markdown post-processing options because it gives you access to the original API output alongside the converted Haystack documents.
:::note
This component returns Markdown content. Avoid piping it through `DocumentCleaner()` with its default settings because `remove_extra_whitespaces=True` and `remove_empty_lines=True` can collapse line breaks and flatten headings, tables, and image tags. For page-aware chunking, connect the converter directly to `DocumentSplitter`, or disable those options if you need custom cleanup.
:::
## When to use it
`PaddleOCRVLDocumentConverter` is a strong fit when you need more than plain OCR text:
- **Scanned PDFs and camera-captured documents** where page orientation and warped text can reduce extraction quality.
- **Layout-sensitive documents** such as invoices, reports, forms, and multi-column PDFs where preserving structure matters for downstream chunking and retrieval.
- **Tables, formulas, charts, or seals** where you want more targeted extraction behavior than plain text OCR.
- **RAG ingestion pipelines** where Markdown output is useful because headings, lists, tables, and page breaks can be preserved for later splitting.
## Useful configuration areas
The full parameter list is available in the [API reference](/reference/integrations-paddleocr). In practice, the most useful options tend to fall into these groups:
- **Input handling and image cleanup**: `file_type`, `use_doc_orientation_classify`, and `use_doc_unwarping` help when you mix PDFs and images or work with skewed scans and mobile photos.
- **Layout-aware extraction**: `use_layout_detection`, `layout_threshold`, `layout_nms`, `layout_unclip_ratio`, `layout_merge_bboxes_mode`, `layout_shape_mode`, and `merge_layout_blocks` help you tune how regions are detected and merged before Markdown is generated.
- **Content focus**: `prompt_label`, `use_ocr_for_image_block`, `use_chart_recognition`, and `use_seal_recognition` let you bias extraction toward a particular type of content, such as plain OCR, formulas, tables, charts, or seals.
- **Markdown output shaping**: `format_block_content`, `markdown_ignore_labels`, `prettify_markdown`, `show_formula_number`, `restructure_pages`, `merge_tables`, and `relevel_titles` help you control how much cleanup and restructuring happens before the result becomes a Haystack document.
- **VLM generation controls**: `repetition_penalty`, `temperature`, `top_p`, `min_pixels`, `max_pixels`, `max_new_tokens`, `vlm_extra_args`, and `additional_params` are useful when you need to trade off output quality, determinism, and cost.
- **Debugging and inspection**: `visualize=True` and the returned `raw_paddleocr_responses` are helpful when you are tuning extraction quality for a new document type.
## Typical scenarios
These settings are especially useful in a few common workflows:
- **Scanned contracts or receipts from phones**: start with `use_doc_orientation_classify=True` and `use_doc_unwarping=True`.
- **Table-heavy financial or operations PDFs**: consider `use_layout_detection=True`, `merge_tables=True`, and `restructure_pages=True`.
- **Formula-heavy documents**: use `prompt_label="formula"` together with `show_formula_number=True` if formula numbering matters in the final Markdown.
- **Mixed business documents with figures or seals**: enable `use_chart_recognition=True`, `use_seal_recognition=True`, or `use_ocr_for_image_block=True` depending on the content you want to preserve.
## Usage
You need to install the `paddleocr-haystack` integration to use `PaddleOCRVLDocumentConverter`:
```shell
pip install paddleocr-haystack
```
### On its own
Basic usage with a local file:
```python
from pathlib import Path
from haystack.utils import Secret
from haystack_integrations.components.converters.paddleocr import (
PaddleOCRVLDocumentConverter,
)
converter = PaddleOCRVLDocumentConverter(
api_url="`access_token`: The AI Studio access token. Can be set with `AISTUDIO_ACCESS_TOKEN` environment variable. | | **Mandatory run variables** | `sources`: A list of image or PDF file paths or ByteStream objects. | | **Output variables** | `documents`: A list of documents.
`raw_paddleocr_responses`: A list of raw OCR responses from PaddleOCR API. | | **API reference** | [PaddleOCR](/reference/integrations-paddleocr) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/paddleocr | | **Package name** | `paddleocr-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: PDF file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Converters](/reference/converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/pdfminer.py |
| **Package name** | `haystack-ai` |
## Overview
The `PDFMinerToDocument` component converts PDF files into documents using [PDFMiner](https://pdfminersix.readthedocs.io/en/latest/) extraction tool arguments.
You can use it in an indexing pipeline to index the contents of a PDF file in a Document Store. It takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream)objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
When initializing the component, you can adjust several parameters to fit your PDF. See the full parameter list and descriptions in our [API reference](/reference/converters-api#pdfminertodocument).
## Usage
First, install `pdfminer` package to start using this converter:
```shell
pip install pdfminer.six
```
### On its own
```python
from haystack.components.converters import PDFMinerToDocument
converter = PDFMinerToDocument()
results = converter.run(
sources=["sample.pdf"],
meta={"date_added": datetime.now().isoformat()},
)
documents = results["documents"]
print(documents[0].content)
# 'This is a text from the PDF file.'
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import PDFMinerToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", PDFMinerToDocument())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": file_names}})
```
---
// File: pipeline-components/converters/pdftoimagecontent
# PDFToImageContent
`PDFToImageContent` reads local PDF files and converts them into `ImageContent` objects. These are ready for multimodal AI pipelines, including tasks like image captioning, visual QA, or prompt-based generation.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a `ChatPromptBuilder` in a query pipeline |
| **Mandatory run variables** | `sources`: A list of PDF file paths or ByteStreams |
| **Output variables** | `image_contents`: A list of ImageContent objects |
| **API reference** | [Image Converters](/reference/image-converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/image/pdf_to_image.py |
| **Package name** | `haystack-ai` |
## Overview
`PDFToImageContent` processes a list of PDF sources and converts them into `ImageContent` objects, one for each page of the PDF. These can be used in multimodal pipelines that require base64-encoded image input.
Each source can be:
- A file path (string or `Path`), or
- A `ByteStream` object.
Optionally, you can provide metadata using the `meta` parameter. This can be a single dictionary (applied to all images) or a list matching the length of `sources`.
Use the `size` parameter to resize images while preserving aspect ratio. This reduces memory usage and transmission size, which is helpful when working with remote models or limited-resource environments.
This component is often used in query pipelines just before a `ChatPromptBuilder`.
## Usage
### On its own
```python
from haystack.components.converters.image import PDFToImageContent
converter = PDFToImageContent()
sources = ["file.pdf", "another_file.pdf"]
image_contents = converter.run(sources=sources)["image_contents"]
print(image_contents)
# [ImageContent(base64_image='...',
# mime_type='image/jpeg',
# detail=None,
# meta={'file_path': 'file.pdf', 'page_number': 1}),
# ...]
```
### In a pipeline
Use `PDFToImageContent` to supply page images to a `ChatPromptBuilder` for multimodal QA or captioning with an LLM.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.converters.image import PDFToImageContent
# Query pipeline
pipeline = Pipeline()
pipeline.add_component("image_converter", PDFToImageContent(detail="auto"))
pipeline.add_component(
"chat_prompt_builder",
ChatPromptBuilder(
required_variables=["question"],
template="""{% message role="system" %}
You are a helpful assistant that answers questions using the provided images.
{% endmessage %}
{% message role="user" %}
Question: {{ question }}
{% for img in image_contents %}
{{ img | templatize_part }}
{% endfor %}
{% endmessage %}
""",
),
)
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini"))
pipeline.connect("image_converter", "chat_prompt_builder.image_contents")
pipeline.connect("chat_prompt_builder", "llm")
sources = ["flan_paper.pdf"]
result = pipeline.run(
data={
"image_converter": {"sources": ["flan_paper.pdf"], "page_range": "9"},
"chat_prompt_builder": {"question": "What is the main takeaway of Figure 6?"},
},
)
print(result["llm"]["replies"][0].text)
# ('The main takeaway of Figure 6 is that Flan-PaLM demonstrates improved '
# 'performance in zero-shot reasoning tasks when utilizing chain-of-thought '
# '(CoT) reasoning, as indicated by higher accuracy across different model '
# 'sizes compared to PaLM without finetuning. This highlights the importance of '
# 'instruction finetuning combined with CoT for enhancing reasoning '
# 'capabilities in models.')
```
## Additional References
🧑🍳 Cookbook: [Introduction to Multimodality](https://haystack.deepset.ai/cookbook/multimodal_intro)
---
// File: pipeline-components/converters/pptxtodocument
# PPTXToDocument
Convert PPTX files to documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: PPTX file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Converters](/reference/converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/pptx.py |
| **Package name** | `haystack-ai` |
## Overview
The `PPTXToDocument` component converts PPTX files into documents. It takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
## Usage
First, install the`python-pptx` package to start using this converter:
```shell
pip install python-pptx
```
### On its own
```python
from haystack.components.converters import PPTXToDocument
converter = PPTXToDocument()
results = converter.run(
sources=["sample.pptx"],
meta={"date_added": datetime.now().isoformat()},
)
documents = results["documents"]
print(documents[0].content)
# 'This is the text from the PPTX file.'
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import PPTXToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", PPTXToDocument())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": file_names}})
```
---
// File: pipeline-components/converters/pypdftodocument
# PyPDFToDocument
A component that converts PDF files to Documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: PDF file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Converters](/reference/converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/pypdf.py |
| **Package name** | `haystack-ai` |
## Overview
The `PyPDFToDocument` component converts PDF files into documents. You can use it in an indexing pipeline to index the contents of a PDF file into a Document Store. It takes a list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
## Usage
You need to install `pypdf` package to use the `PyPDFToDocument` converter:
```shell
pip install pypdf
```
### On its own
```python
from pathlib import Path
from haystack.components.converters import PyPDFToDocument
converter = PyPDFToDocument()
docs = converter.run(sources=[Path("my_file.pdf")])
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import PyPDFToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", PyPDFToDocument())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": file_names}})
```
## Additional References
🧑🍳 Cookbook: [PDF-Based Question Answering with Amazon Bedrock and Haystack](https://haystack.deepset.ai/cookbook/amazon_bedrock_for_documentation_qa)
📓 Tutorial: [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline)
---
// File: pipeline-components/converters/textfiletodocument
# TextFileToDocument
Converts text files to documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of paths to text files you want to convert |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Converters](/reference/converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/txt.py |
| **Package name** | `haystack-ai` |
## Overview
The `TextFileToDocument` component converts text files into documents. You can use it in an indexing pipeline to index the contents of text files into a Document Store. It takes a list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
When you initialize the component, you can optionally set the default encoding of the text files through the `encoding` parameter. If you don't provide any value, the component uses `"utf-8"` by default. Note that if the encoding is specified in the metadata of an input ByteStream, it will override this parameter's setting.
## Usage
### On its own
```python
from pathlib import Path
from haystack.components.converters import TextFileToDocument
converter = TextFileToDocument()
docs = converter.run(sources=[Path("my_file.txt")])
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", TextFileToDocument())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": file_names}})
```
## Additional References
:notebook: Tutorial: [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline)
---
// File: pipeline-components/converters/tikadocumentconverter
# TikaDocumentConverter
An integration for converting files of different types (PDF, DOCX, HTML, and more) to documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: File paths |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Tika](/reference/integrations-tika) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/tika |
| **Package name** | `tika-haystack` |
## Overview
The `TikaDocumentConverter` component converts files of different types (pdf, docx, html, and others) into documents. You can use it in an indexing pipeline to index the contents of files into a Document Store. It takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
This integration uses [Apache Tika](https://tika.apache.org/) to parse the files and requires a running Tika server.
The easiest way to run Tika is by using Docker: `docker run -d -p 127.0.0.1:9998:9998 apache/tika:latest`.
For more options on running Tika on Docker, see the [Tika documentation](https://github.com/apache/tika-docker/blob/main/README.md#usage).
When you initialize the `TikaDocumentConverter` component, you can specify a custom URL of the Tika server you are using through the parameter `tika_url`. The default URL is `"http://localhost:9998/tika"`.
## Usage
Install the `tika-haystack` package to use the `TikaDocumentConverter` component:
```shell
pip install tika-haystack
```
### On its own
```python
from haystack_integrations.components.converters.tika import TikaDocumentConverter
from pathlib import Path
converter = TikaDocumentConverter()
converter.run(sources=[Path("my_file.pdf")])
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.converters.tika import TikaDocumentConverter
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", TikaDocumentConverter())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": file_paths}})
```
---
// File: pipeline-components/converters/twelvelabsvideoconverter
# TwelveLabsVideoConverter
`TwelveLabsVideoConverter` converts videos to Haystack Documents using the TwelveLabs Pegasus video-language model. Pegasus analyzes each video on the fly — its visuals **and** its own audio (via ASR) — and returns text, so each source video becomes one Document whose content is Pegasus's analysis (for example, a description plus a transcript). There is no frame extraction or separate transcription step.
| | |
| --- | --- |
| **Most common position in a pipeline** | At the beginning of an indexing pipeline, before [PreProcessors](../preprocessors.mdx) or an embedder |
| **Mandatory init variables** | `api_key`: The TwelveLabs API key. Can be set with `TWELVELABS_API_KEY` env var. |
| **Mandatory run variables** | `sources`: A list of video URLs or local file paths |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [TwelveLabs](/reference/integrations-twelvelabs) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/twelvelabs |
| **Package name** | `twelvelabs-haystack` |
## Overview
The `TwelveLabsVideoConverter` takes a list of video sources and produces one [`Document`](../../concepts/data-classes.mdx#document) per source, with the Document content set to Pegasus's text analysis. Sources may be publicly accessible direct video URLs or local file paths (uploaded to TwelveLabs, up to 200 MB). Sources that fail to process are skipped with a warning logged, so one bad source does not fail the whole batch.
Each produced Document carries metadata about the request, including `source`, `asset_id`, `analysis_id`, `model`, and `provider`. The default model is `pegasus1.5`.
You can steer the analysis with a custom `prompt` and tune `temperature` and `max_tokens`.
To start using this integration with Haystack, install the package with:
```shell
pip install twelvelabs-haystack
```
The component uses a `TWELVELABS_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`. To get an API key, head to [playground.twelvelabs.io](https://playground.twelvelabs.io).
## Usage
### On its own
```python
from haystack_integrations.components.converters.twelvelabs import (
TwelveLabsVideoConverter,
)
converter = TwelveLabsVideoConverter()
result = converter.run(sources=["https://example.com/clip.mp4"])
document = result["documents"][0]
print(document.content) # Pegasus's description + transcript of the video
print(document.meta) # includes source, asset_id, analysis_id, model, provider
```
:::info
We recommend setting `TWELVELABS_API_KEY` as an environment variable instead of setting it as a parameter.
:::
### With a custom prompt
```python
from haystack_integrations.components.converters.twelvelabs import (
TwelveLabsVideoConverter,
)
converter = TwelveLabsVideoConverter(
prompt="Summarize this video in three bullet points and list any products shown.",
temperature=0.2,
max_tokens=1024,
)
result = converter.run(sources=["https://example.com/clip.mp4"])
print(result["documents"][0].content)
```
### In a pipeline
This indexing pipeline analyzes videos with Pegasus, embeds the resulting analysis with the [`TwelveLabsDocumentEmbedder`](../embedders/twelvelabsdocumentembedder.mdx), and writes the documents to a document store:
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.converters.twelvelabs import (
TwelveLabsVideoConverter,
)
from haystack_integrations.components.embedders.twelvelabs import (
TwelveLabsDocumentEmbedder,
)
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
indexing_pipeline = Pipeline()
indexing_pipeline.add_component("converter", TwelveLabsVideoConverter())
indexing_pipeline.add_component("embedder", TwelveLabsDocumentEmbedder())
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
indexing_pipeline.connect("converter", "embedder")
indexing_pipeline.connect("embedder", "writer")
indexing_pipeline.run({"converter": {"sources": ["https://example.com/clip.mp4"]}})
```
### Attaching metadata
Pass a single dictionary to apply metadata to all output Documents, or a list to set metadata per source:
```python
from haystack_integrations.components.converters.twelvelabs import (
TwelveLabsVideoConverter,
)
converter = TwelveLabsVideoConverter()
# Same metadata for all sources
result = converter.run(
sources=["https://example.com/a.mp4", "https://example.com/b.mp4"],
meta={"campaign": "demo"},
)
# Per-source metadata
result = converter.run(
sources=["https://example.com/a.mp4", "https://example.com/b.mp4"],
meta=[{"title": "Clip A"}, {"title": "Clip B"}],
)
```
---
// File: pipeline-components/converters/unstructuredfileconverter
# UnstructuredFileConverter
Use this component to convert text files and directories to a document.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `paths`: A union of lists of paths |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Unstructured](/reference/integrations-unstructured) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/unstructured |
| **Package name** | `unstructured-fileconverter-haystack` |
## Overview
`UnstructuredFileConverter` converts files and directories into documents using the Unstructured API.
[Unstructured](https://docs.unstructured.io/) provides a series of tools to do ETL for LLMs. The `UnstructuredFileConverter` calls the Unstructured API that extracts text and other information from a vast range of file [formats](https://docs.unstructured.io/api-reference/api-services/overview#supported-file-types).
This Converter supports different modes for creating documents from the elements returned by Unstructured:
- `"one-doc-per-file"`: One Haystack document per file. All elements are concatenated into one text field.
- `"one-doc-per-page"`: One Haystack document per page. All elements on a page are concatenated into one text field.
- `"one-doc-per-element"`: One Haystack document per element. Each element is converted to a Haystack document.
## Usage
Install the Unstructured integration to use `UnstructuredFileConverter`component:
```shell
pip install unstructured-fileconverter-haystack
```
There are free and paid versions of Unstructured API: **Free Unstructured API** and **Unstructured Serverless API**.
1. **Free Unstructured API**:
- API URL: `https://api.unstructured.io/general/v0/general`
- This version is free, but comes with certain limitations.
2. **Unstructured Serverless API**:
- You'll find your unique API URL in your Unstructured account after signing up for the paid version.
- This is a full-tier paid version of Unstructured.
For more details about the two tiers refer to Unstructured [FAQ](https://docs.unstructured.io/faq/faq).
> ❗️ The API keys for the free and paid versions are different and cannot be used interchangeably.
Regardless of the chosen tier, we recommend to set the Unstructured API key as an environment variable `UNSTRUCTURED_API_KEY`:
```shell
export UNSTRUCTURED_API_KEY=your_api_key
```
### On its own
```python
import os
from haystack_integrations.components.converters.unstructured import (
UnstructuredFileConverter,
)
converter = UnstructuredFileConverter()
documents = converter.run(paths=["a/file/path.pdf", "a/directory/path"])["documents"]
```
### In a pipeline
```python
import os
from haystack import Pipeline
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.converters.unstructured import (
UnstructuredFileConverter,
)
document_store = InMemoryDocumentStore()
indexing = Pipeline()
indexing.add_component("converter", UnstructuredFileConverter())
indexing.add_component("writer", DocumentWriter(document_store))
indexing.connect("converter", "writer")
indexing.run({"converter": {"paths": ["a/file/path.pdf", "a/directory/path"]}})
```
### With Docker
To use `UnstructuredFileConverter` through Docker, first, set up an Unstructured Docker container:
```
docker run -p 8000:8000 -d --rm --name unstructured-api quay.io/unstructured-io/unstructured-api:latest --port 8000 --host 0.0.0.0
```
When initializing the component, specify the localhost URL:
```python
from haystack_integrations.components.converters.unstructured import (
UnstructuredFileConverter,
)
converter = UnstructuredFileConverter(
api_url="http://localhost:8000/general/v0/general",
)
```
---
// File: pipeline-components/converters/xlsxtodocument
# XLSXToDocument
Converts Excel files into documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory run variables** | `sources`: File paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Converters](/reference/converters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/xlsx.py |
| **Package name** | `haystack-ai` |
## Overview
The `XLSXToDocument` component converts XLSX files into Haystack Documents with a CSV (default) or Markdown format. It takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
To see the additional parameters that you can specify with the component initialization, check out the [API Reference](/reference/converters-api#xlsxtodocument).
## Usage
First, install the pandas, openpyxl, and tabulate packages to start using this converter:
```shell
pip install pandas openpyxl
pip install tabulate
```
### On its own
```python
from haystack.components.converters import XLSXToDocument
converter = XLSXToDocument()
results = converter.run(
sources=["sample.xlsx"],
meta={"date_added": datetime.now().isoformat()},
)
documents = results["documents"]
print(documents[0].content)
# ",A,B\n1,col_a,col_b\n2,1.5,test\n"
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import XLSXToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", XLSXToDocument())
pipeline.add_component("cleaner", DocumentCleaner())
pipeline.add_component(
"splitter",
DocumentSplitter(split_by="sentence", split_length=5),
)
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "cleaner")
pipeline.connect("cleaner", "splitter")
pipeline.connect("splitter", "writer")
pipeline.run({"converter": {"sources": file_names}})
```
---
// File: pipeline-components/converters
# Converters
Use various Converters to extract data from files in different formats and cast it into the unified document format. There are several converters available for converting PDFs, images, DOCX files, and more.
| Converter | Description |
| --- | --- |
| [AmazonTextractConverter](converters/amazontextractconverter.mdx) | Converts images and single-page PDFs to documents using AWS Textract, with optional structured analysis of tables, forms, signatures, and layout, plus natural-language queries. |
| [AzureDocumentIntelligenceConverter](converters/azuredocumentintelligenceconverter.mdx) | Converts PDF, JPEG, PNG, BMP, TIFF, DOCX, XLSX, PPTX, and HTML to documents using Azure's Document Intelligence service with GitHub Flavored Markdown output. |
| [AzureOCRDocumentConverter](converters/azureocrdocumentconverter.mdx) | Converts PDF (both searchable and image-only), JPEG, PNG, BMP, TIFF, DOCX, XLSX, PPTX, and HTML to documents. |
| [CSVToDocument](converters/csvtodocument.mdx) | Converts CSV files to documents. |
| [DoclingConverter](converters/doclingconverter.mdx) | Converts PDF, DOCX, HTML, and other document formats to documents with layout-aware chunking, Markdown, and JSON export. |
| [DoclingServeConverter](converters/doclingserveconverter.mdx) | Converts PDF, DOCX, HTML, and other document formats to documents using a remote DoclingServe HTTP server, with no local ML dependencies. |
| [DocumentToImageContent](converters/documenttoimagecontent.mdx) | Extracts visual data from image or PDF file-based documents and converts them into `ImageContent` objects. |
| [DOCXToDocument](converters/docxtodocument.mdx) | Convert DOCX files to documents. |
| [FileToFileContent](converters/filetofilecontent.mdx) | Reads files and converts them into `FileContent` objects. |
| [HTMLToDocument](converters/htmltodocument.mdx) | Converts HTML files to documents. |
| [ImageFileToDocument](converters/imagefiletodocument.mdx) | Converts image file references into empty `Document` objects with associated metadata. |
| [ImageFileToImageContent](converters/imagefiletoimagecontent.mdx) | Reads local image files and converts them into `ImageContent` objects. |
| [JSONConverter](converters/jsonconverter.mdx) | Converts JSON files to text documents. |
| [KreuzbergConverter](converters/kreuzbergconverter.mdx) | Converts 91+ file formats to documents locally using Kreuzberg's Rust-core engine. |
| [LibreOfficeFileConverter](converters/libreofficefileconverter.mdx) | Converts office files (documents, spreadsheets, presentations) between formats using LibreOffice's command line interface. |
| [MarkdownToDocument](converters/markdowntodocument.mdx) | Converts markdown files to documents. |
| [MarkItDownConverter](converters/markitdownconverter.mdx) | Converts PDF, Word, PowerPoint, Excel, HTML, images, and more to documents using Microsoft's MarkItDown library. |
| [MistralOCRDocumentConverter](converters/mistralocrdocumentconverter.mdx) | Extracts text from documents using Mistral's OCR API, with optional structured annotations. |
| [MSGToDocument](converters/msgtodocument.mdx) | Converts Microsoft Outlook .msg files to documents. |
| [MultiFileConverter](converters/multifileconverter.mdx) | Converts CSV, DOCX, HTML, JSON, MD, PPTX, PDF, TXT, and XSLX files to documents. |
| [OpenAPIServiceToFunctions](converters/openapiservicetofunctions.mdx) | Transforms OpenAPI service specifications into a format compatible with OpenAI's function calling mechanism. |
| [OutputAdapter](converters/outputadapter.mdx) | Helps the output of one component fit into the input of another. |
| [PaddleOCRVLDocumentConverter](converters/paddleocrvldocumentconverter.mdx) | Extracts text from documents using PaddleOCR's large model document parsing API. |
| [PDFMinerToDocument](converters/pdfminertodocument.mdx) | Converts complex PDF files to documents using pdfminer arguments. |
| [PDFToImageContent](converters/pdftoimagecontent.mdx) | Reads local PDF files and converts them into `ImageContent` objects. |
| [PPTXToDocument](converters/pptxtodocument.mdx) | Converts PPTX files to documents. |
| [PyPDFToDocument](converters/pypdftodocument.mdx) | Converts PDF files to documents. |
| [TikaDocumentConverter](converters/tikadocumentconverter.mdx) | Converts various file types to documents using Apache Tika. |
| [TextFileToDocument](converters/textfiletodocument.mdx) | Converts text files to documents. |
| [TwelveLabsVideoConverter](converters/twelvelabsvideoconverter.mdx) | Converts videos to documents using the TwelveLabs Pegasus video-language model. |
| [UnstructuredFileConverter](converters/unstructuredfileconverter.mdx) | Converts text files and directories to a document. |
| [XLSXToDocument](converters/xlsxtodocument.mdx) | Converts Excel files into documents. |
---
// File: pipeline-components/downloaders/s3downloader
# S3Downloader
`S3Downloader` downloads files from AWS S3 buckets to the local filesystem and enriches documents with the local file path.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before File Converters or Routers that need local file paths |
| **Mandatory init variables** | `file_root_path`: Path where files will be downloaded. Can be set with `FILE_ROOT_PATH` env var.
`aws_access_key_id`: AWS access key ID. Can be set with AWS_ACCESS_KEY_ID env var.
`aws_secret_access_key`: AWS secret access key. Can be set with AWS_SECRET_ACCESS_KEY env var.
`aws_region_name`: AWS region name. Can be set with AWS_DEFAULT_REGION env var. | | **Mandatory run variables** | `documents`: A list of documents containing name of the file to download in metadata. | | **Output variables** | `documents`: A list of documents enriched with the local file path in `meta['file_path']` | | **API reference** | [S3Downloader](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
## Overview
`S3Downloader` downloads files from AWS S3 buckets to your local filesystem and enriches Document objects with the local file path. This component is useful for pipelines that need to process files stored in S3, such as PDFs, images, or text files.
The component supports AWS authentication through environment variables by default. You can set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION` environment variables. Alternatively, you can pass credentials directly at initialization using the [Secret API](../../concepts/secret-management.mdx):
```python
from haystack.utils import Secret
from haystack_integrations.components.downloaders.s3 import S3Downloader
downloader = S3Downloader(
aws_access_key_id=Secret.from_token("`aws_access_key_id`: AWS access key ID. Can be set with AWS_ACCESS_KEY_ID env var.
`aws_secret_access_key`: AWS secret access key. Can be set with AWS_SECRET_ACCESS_KEY env var.
`aws_region_name`: AWS region name. Can be set with AWS_DEFAULT_REGION env var. | | **Mandatory run variables** | `documents`: A list of documents containing name of the file to download in metadata. | | **Output variables** | `documents`: A list of documents enriched with the local file path in `meta['file_path']` | | **API reference** | [S3Downloader](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `model`: The embedding model to use
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.
`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. | | **Mandatory run variables** | `documents`: A list of documents to be embedded | | **Output variables** | `documents`: A list of documents (enriched with embeddings) | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
## Overview
[Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) is a fully managed service that makes language models from leading AI startups and Amazon available for your use through a unified API.
Amazon Titan and Cohere embedding models are supported, for example `amazon.titan-embed-text-v1`, `amazon.titan-embed-text-v2:0`, `amazon.titan-embed-image-v1`, `cohere.embed-english-v3`, `cohere.embed-multilingual-v3`, and `cohere.embed-v4:0`. To find all supported models, see the [Amazon Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html), filter for "embedding", and select models from the Amazon Titan and Cohere series.
:::info[Batch Inference]
Note that only Cohere models support batch inference – computing embeddings for more documents with the same request.
:::
This component should be used to embed a list of documents. To embed a string, you should use the [`AmazonBedrockTextEmbedder`](amazonbedrocktextembedder.mdx).
### Authentication
`AmazonBedrockDocumentEmbedder` uses AWS for authentication. You can either provide credentials as parameters directly to the component or use the AWS CLI and authenticate through your IAM. For more information on how to set up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
To initialize `AmazonBedrockDocumentEmbedder` and authenticate by providing credentials, provide the `model` name, as well as `aws_access_key_id`, `aws_secret_access_key` and `aws_region_name`. Other parameters are optional. You can check them out in our [API reference](/reference/integrations-amazon-bedrock#amazonbedrockdocumentembedder).
### Model-specific parameters
Even if Haystack provides a unified interface, each model offered by Bedrock can accept specific parameters. You can pass these parameters at initialization.
For example, Cohere models support `input_type` and `truncate`, as seen in [Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html).
```python
from haystack_integrations.components.embedders.amazon_bedrock import (
AmazonBedrockDocumentEmbedder,
)
embedder = AmazonBedrockDocumentEmbedder(
model="cohere.embed-english-v3",
input_type="search_document",
truncate="LEFT",
)
```
### Embedding Metadata
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
You can do this easily by using the Document Embedder:
```python
from haystack import Document
from haystack_integrations.components.embedders.amazon_bedrock import (
AmazonBedrockDocumentEmbedder,
)
doc = Document(content="some text", meta={"title": "relevant title", "page number": 18})
embedder = AmazonBedrockDocumentEmbedder(
model="cohere.embed-english-v3",
meta_fields_to_embed=["title"],
)
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
```
## Usage
### Installation
You need to install `amazon-bedrock-haystack` package to use the `AmazonBedrockDocumentEmbedder`:
```shell
pip install amazon-bedrock-haystack
```
### On its own
Basic usage:
```python
import os
from haystack import Document
from haystack_integrations.components.embedders.amazon_bedrock import (
AmazonBedrockDocumentEmbedder,
)
os.environ["AWS_ACCESS_KEY_ID"] = "..."
os.environ["AWS_SECRET_ACCESS_KEY"] = "..."
os.environ["AWS_DEFAULT_REGION"] = "us-east-1" # just an example
doc = Document(content="I love pizza!")
embedder = AmazonBedrockDocumentEmbedder(
model="cohere.embed-english-v3",
input_type="search_document",
)
result = embedder.run(documents=[doc])
print(result["documents"][0].embedding)
# [0.017020374536514282, -0.023255806416273117, ...]
```
### In a pipeline
In a RAG pipeline:
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.amazon_bedrock import (
AmazonBedrockDocumentEmbedder,
AmazonBedrockTextEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
]
indexing_pipeline = Pipeline()
indexing_pipeline.add_component(
"embedder",
AmazonBedrockDocumentEmbedder(model="cohere.embed-english-v3"),
)
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
indexing_pipeline.connect("embedder", "writer")
indexing_pipeline.run({"embedder": {"documents": documents}})
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
AmazonBedrockTextEmbedder(model="cohere.embed-english-v3"),
)
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Who lives in Berlin?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
# Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
```
## Additional References
🧑🍳 Cookbook: [PDF-Based Question Answering with Amazon Bedrock and Haystack](https://haystack.deepset.ai/cookbook/amazon_bedrock_for_documentation_qa)
---
// File: pipeline-components/embedders/amazonbedrockdocumentimageembedder
# AmazonBedrockDocumentImageEmbedder
`AmazonBedrockDocumentImageEmbedder` computes image embeddings for documents using models exposed through the Amazon Bedrock API. It stores the obtained vectors in the embedding field of each document.
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.
`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. | | **Mandatory run variables** | `documents`: A list of documents to be embedded | | **Output variables** | `documents`: A list of documents (enriched with embeddings) | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `model`: The multimodal embedding model to use.
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.
`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. | | **Mandatory run variables** | `documents`: A list of documents, with a meta field containing an image file path | | **Output variables** | `documents`: A list of documents (enriched with embeddings) | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
## Overview
Amazon Bedrock is a fully managed service that provides access to foundation models through a unified API.
`AmazonBedrockDocumentImageEmbedder` expects a list of documents containing an image or a PDF file path in a meta field. The meta field can be specified with the `file_path_meta_field` init parameter of this component.
The embedder efficiently loads the images, computes the embeddings using selected Bedrock model, and stores each of them in the `embedding` field of the document.
Amazon Titan and Cohere multimodal embedding models are supported, for example `amazon.titan-embed-image-v1`, `cohere.embed-english-v3`, `cohere.embed-multilingual-v3`, and `cohere.embed-v4:0`.
`AmazonBedrockDocumentImageEmbedder` is commonly used in indexing pipelines. At retrieval time, you need to use the same model with `AmazonBedrockTextEmbedder` to embed the query, before using an Embedding Retriever.
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install amazon-bedrock-haystack
```
### Authentication
`AmazonBedrockDocumentImageEmbedder` uses AWS for authentication. You can either provide credentials as parameters directly to the component or use the AWS CLI and authenticate through your IAM. For more information on how to set up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
To initialize `AmazonBedrockDocumentImageEmbedder` and authenticate by providing credentials, provide the `model` name, as well as `aws_access_key_id`, `aws_secret_access_key`, and `aws_region_name`. Other parameters are optional, you can check them out in our [API reference](/reference/integrations-amazon-bedrock#amazonbedrocktextembedder).
### Model-specific parameters
Even if Haystack provides a unified interface, each model offered by Bedrock can accept specific parameters. You can pass these parameters at initialization.
- **Amazon Titan**: Use `embeddingConfig` to control embedding behavior.
- **Cohere v3**: Use `embedding_types` to select a single embedding type for images.
```python
from haystack_integrations.components.embedders.amazon_bedrock import (
AmazonBedrockDocumentImageEmbedder,
)
embedder = AmazonBedrockDocumentImageEmbedder(
model="cohere.embed-english-v3",
embedding_types=["float"], # single value only
)
```
Note that only _one_ value in `embedding_types` is supported by this component. Passing multiple values raises an error.
## Usage
### On its own
```python
import os
from haystack import Document
from haystack_integrations.components.embedders.amazon_bedrock import (
AmazonBedrockDocumentImageEmbedder,
)
os.environ["AWS_ACCESS_KEY_ID"] = "..."
os.environ["AWS_SECRET_ACCESS_KEY"] = "..."
os.environ["AWS_DEFAULT_REGION"] = "us-east-1" # example
# Point Documents to image/PDF files via metadata (default key: "file_path")
documents = [
Document(content="A photo of a cat", meta={"file_path": "cat.jpg"}),
Document(
content="Invoice page",
meta={
"file_path": "invoice.pdf",
"mime_type": "application/pdf",
"page_number": 1,
},
),
]
embedder = AmazonBedrockDocumentImageEmbedder(
model="amazon.titan-embed-image-v1",
image_size=(1024, 1024), # optional downscaling
)
result = embedder.run(documents=documents)
embedded_docs = result["documents"]
```
### In a pipeline
In this example, we can see an indexing pipeline with 3 components:
- `ImageFileToDocument` Converter that creates empty documents with a reference to an image in the `meta.file_path` field;
- `AmazonBedrockDocumentImageEmbedder` that loads the images, computes embeddings and stores them in documents;
- `DocumentWriter` that write the documents in the `InMemoryDocumentStore`.
There is also a multimodal retrieval pipeline, composed of an `AmazonBedrockTextEmbedder` (using the same model as before) and an `InMemoryEmbeddingRetriever`.
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.writers import DocumentWriter
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack_integrations.components.embedders.amazon_bedrock import (
AmazonBedrockDocumentImageEmbedder,
AmazonBedrockTextEmbedder,
)
# Document store using vector similarity for retrieval
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
# Sample corpus with file paths in metadata
documents = [
Document(content="A sketch of a horse", meta={"file_path": "horse.png"}),
Document(content="A city map", meta={"file_path": "map.jpg"}),
]
# Indexing pipeline: image embeddings -> write to store
indexing = Pipeline()
indexing.add_component(
"image_embedder",
AmazonBedrockDocumentImageEmbedder(model="cohere.embed-english-v3"),
)
indexing.add_component("writer", DocumentWriter(document_store=document_store))
indexing.connect("image_embedder", "writer")
indexing.run({"image_embedder": {"documents": documents}})
# Query pipeline: text -> embedding -> vector retriever
query = Pipeline()
query.add_component(
"text_embedder",
AmazonBedrockTextEmbedder(model="cohere.embed-english-v3"),
)
query.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query.connect("text_embedder.embedding", "retriever.query_embedding")
res = query.run({"text_embedder": {"text": "Which document shows a horse?"}})
```
## Additional References
:notebook: Tutorial: [Creating Vision+Text RAG Pipelines](https://haystack.deepset.ai/tutorials/46_multimodal_rag)
---
// File: pipeline-components/embedders/amazonbedrocktextembedder
# AmazonBedrockTextEmbedder
This component computes embeddings for text (such as a query) using models through Amazon Bedrock API.
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.
`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. | | **Mandatory run variables** | `documents`: A list of documents, with a meta field containing an image file path | | **Output variables** | `documents`: A list of documents (enriched with embeddings) | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `model`: The embedding model to use
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.
`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. | | **Mandatory run variables** | `text`: A string | | **Output variables** | `embedding`: A list of float numbers (vector) | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
## Overview
[Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) is a fully managed service that makes language models from leading AI startups and Amazon available for your use through a unified API.
Amazon Titan and Cohere embedding models are supported, for example `amazon.titan-embed-text-v1`, `amazon.titan-embed-text-v2:0`, `amazon.titan-embed-image-v1`, `cohere.embed-english-v3`, `cohere.embed-multilingual-v3`, and `cohere.embed-v4:0`. To find all supported models, see the [Amazon Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html), filter for "embedding", and select models from the Amazon Titan and Cohere series.
Use `AmazonBedrockTextEmbedder` to embed a simple string (such as a query) into a vector. Use the [`AmazonBedrockDocumentEmbedder`](amazonbedrockdocumentembedder.mdx) to enrich the documents with the computed embedding, also known as vector.
### Authentication
`AmazonBedrockTextEmbedder` uses AWS for authentication. You can either provide credentials as parameters directly to the component or use the AWS CLI and authenticate through your IAM. For more information on how to set up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
To initialize `AmazonBedrockTextEmbedder` and authenticate by providing credentials, provide the `model` name, as well as `aws_access_key_id`, `aws_secret_access_key`, and `aws_region_name`. Other parameters are optional, you can check them out in our [API reference](/reference/integrations-amazon-bedrock#amazonbedrocktextembedder).
### Model-specific parameters
Even if Haystack provides a unified interface, each model offered by Bedrock can accept specific parameters. You can pass these parameters at initialization.
For example, the Cohere models support `input_type` and `truncate`, as seen in [Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html).
```python
from haystack_integrations.components.embedders.amazon_bedrock import (
AmazonBedrockTextEmbedder,
)
embedder = AmazonBedrockTextEmbedder(
model="cohere.embed-english-v3",
input_type="search_query",
truncate="LEFT",
)
```
## Usage
### Installation
You need to install `amazon-bedrock-haystack` package to use the `AmazonBedrockTextEmbedder`:
```shell
pip install amazon-bedrock-haystack
```
### On its own
Basic usage:
```python
import os
from haystack_integrations.components.embedders.amazon_bedrock import (
AmazonBedrockTextEmbedder,
)
os.environ["AWS_ACCESS_KEY_ID"] = "..."
os.environ["AWS_SECRET_ACCESS_KEY"] = "..."
os.environ["AWS_DEFAULT_REGION"] = "us-east-1" # just an example
text_to_embed = "I love pizza!"
text_embedder = AmazonBedrockTextEmbedder(
model="cohere.embed-english-v3",
input_type="search_query",
)
print(text_embedder.run(text_to_embed))
# {'embedding': [-0.453125, 1.2236328, 2.0058594, 0.67871094...]}
```
### In a pipeline
In a RAG pipeline:
```python
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.amazon_bedrock import (
AmazonBedrockDocumentEmbedder,
AmazonBedrockTextEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
]
document_embedder = AmazonBedrockDocumentEmbedder(model="cohere.embed-english-v3")
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
AmazonBedrockTextEmbedder(model="cohere.embed-english-v3"),
)
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Who lives in Berlin?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
# Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
```
## Additional References
🧑🍳 Cookbook: [PDF-Based Question Answering with Amazon Bedrock and Haystack](https://haystack.deepset.ai/cookbook/amazon_bedrock_for_documentation_qa)
---
// File: pipeline-components/embedders/azureopenaidocumentembedder
# AzureOpenAIDocumentEmbedder
This component computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Azure cognitive services for text and document embedding with models deployed on Azure.
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.
`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. | | **Mandatory run variables** | `text`: A string | | **Output variables** | `embedding`: A list of float numbers (vector) | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) |
| **Mandatory init variables** | `api_key`: The Azure OpenAI API key. Can be set with `AZURE_OPENAI_API_KEY` env var.
`azure_endpoint`: The endpoint of the model deployed on Azure. | | **Mandatory run variables** | `documents`: A list of documents | | **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/azure_document_embedder.py | | **Package name** | `haystack-ai` |
## Overview
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents.
To see the list of compatible embedding models, head over to Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?source=recommendations). The default model for `AzureOpenAITextEmbedder` is `text-embedding-ada-002`.
This component should be used to embed a list of documents. To embed a string, you should use the [`AzureOpenAITextEmbedder`](azureopenaitextembedder.mdx).
To work with Azure components, you will need an Azure OpenAI API key, as well as an Azure OpenAI Endpoint. You can learn more about them in Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
The component uses `AZURE_OPENAI_API_KEY` or `AZURE_OPENAI_AD_TOKEN` environment variables by default. Otherwise, you can pass `api_key` or `azure_ad_token` at initialization:
```python
client = AzureOpenAIDocumentEmbedder(
azure_endpoint="`azure_endpoint`: The endpoint of the model deployed on Azure. | | **Mandatory run variables** | `documents`: A list of documents | | **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/azure_document_embedder.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: The Azure OpenAI API key. Can be set with `AZURE_OPENAI_API_KEY` env var.
`azure_endpoint`: The endpoint of the model deployed on Azure. | | **Mandatory run variables** | `text`: A string | | **Output variables** | `embedding`: A list of float numbers
`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/azure_text_embedder.py | | **Package name** | `haystack-ai` |
## Overview
`AzureOpenAITextEmbedder` transforms a string into a vector that captures its semantics using an OpenAI embedding model. It uses Azure cognitive services for text and document embedding with models deployed on Azure.
To see the list of compatible embedding models, head over to Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?source=recommendations). The default model for `AzureOpenAITextEmbedder` is `text-embedding-ada-002`.
Use `AzureOpenAITextEmbedder` to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [`AzureOpenAIDocumentEmbedder`](azureopenaidocumentembedder.mdx), which enriches the documents with the computed embedding, also known as vector.
To work with Azure components, you will need an Azure OpenAI API key, as well as an Azure OpenAI Endpoint. You can learn more about them in Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
The component uses `AZURE_OPENAI_API_KEY` or `AZURE_OPENAI_AD_TOKEN` environment variables by default. Otherwise, you can pass `api_key` or `azure_ad_token` at initialization:
```python
client = AzureOpenAITextEmbedder(
azure_endpoint="`azure_endpoint`: The endpoint of the model deployed on Azure. | | **Mandatory run variables** | `text`: A string | | **Output variables** | `embedding`: A list of float numbers
`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/azure_text_embedder.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata strings | | **API reference** | [Cohere](/reference/integrations-cohere) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere | | **Package name** | `cohere-haystack` |
## Overview
`CohereDocumentEmbedder` enriches the metadata of documents with an embedding of their content. To embed a string, you should use the [`CohereTextEmbedder`](coheretextembedder.mdx).
The component supports the following Cohere models:
`"embed-v4.0"`, `"embed-english-v3.0"`, `"embed-english-light-v3.0"`, `"embed-multilingual-v3.0"`,
`"embed-multilingual-light-v3.0"`, `"embed-english-v2.0"`, `"embed-english-light-v2.0"`,
`"embed-multilingual-v2.0"`. The default model is `embed-v4.0`. This list of all supported models can be found in Cohere’s [model documentation](https://docs.cohere.com/docs/models#representation).
To start using this integration with Haystack, install it with:
```shell
pip install cohere-haystack
```
The component uses a `COHERE_API_KEY` or `CO_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
from haystack.utils import Secret
from haystack_integrations.components.embedders.cohere import CohereDocumentEmbedder
embedder = CohereDocumentEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata strings | | **API reference** | [Cohere](/reference/integrations-cohere) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere | | **Package name** | `cohere-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents, with a meta field containing an image file path |
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
| **API reference** | [Cohere](/reference/integrations-cohere) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere |
| **Package name** | `cohere-haystack` |
## Overview
`CohereDocumentImageEmbedder` expects a list of documents containing an image or a PDF file path in a meta field. The meta field can be specified with the `file_path_meta_field` init parameter of this component.
The embedder efficiently loads the images, computes the embeddings using a Cohere model, and stores each of them in the `embedding` field of the document.
`CohereDocumentImageEmbedder` is commonly used in indexing pipelines. At retrieval time, you need to use the same model with a `CohereTextEmbedder` to embed the query, before using an Embedding Retriever.
This component is compatible with Cohere Embed models v3 and later. For a complete list of supported models, see the [Cohere documentation](https://docs.cohere.com/docs/models#embed).
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install cohere-haystack
```
### Authentication
The component uses a `COHERE_API_KEY` or `CO_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` method:
```python
from haystack.utils import Secret
from haystack_integrations.components.embedders.cohere import (
CohereDocumentImageEmbedder,
)
embedder = CohereDocumentImageEmbedder(api_key=Secret.from_token("
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers (vectors)
`meta`: A dictionary of metadata strings | | **API reference** | [Cohere](/reference/integrations-cohere) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere | | **Package name** | `cohere-haystack` |
## Overview
`CohereTextEmbedder` embeds a simple string (such as a query) into a vector. For embedding lists of documents, use the use the [`CohereDocumentEmbedder`](coheredocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
The component supports the following Cohere models:
`"embed-v4.0"`, `"embed-english-v3.0"`, `"embed-english-light-v3.0"`, `"embed-multilingual-v3.0"`,
`"embed-multilingual-light-v3.0"`, `"embed-english-v2.0"`, `"embed-english-light-v2.0"`,
`"embed-multilingual-v2.0"`. The default model is `embed-v4.0`. This list of all supported models can be found in Cohere’s [model documentation](https://docs.cohere.com/docs/models#representation).
To start using this integration with Haystack, install it with:
```shell
pip install cohere-haystack
```
The component uses a `COHERE_API_KEY` or `CO_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` static method:
```python
embedder = CohereTextEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata strings | | **API reference** | [Cohere](/reference/integrations-cohere) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere | | **Package name** | `cohere-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Eden AI API key. Can be set with `EDENAI_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata strings | | **API reference** | [Eden AI](/reference/integrations-edenai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/edenai | | **Package name** | `edenai-haystack` |
This component should be used to embed a list of Documents. To embed a string, use the [`EdenAITextEmbedder`](edenaitextembedder.mdx).
## Overview
`EdenAIDocumentEmbedder` computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Eden AI's OpenAI-compatible API. Models are selected using Eden AI's `provider/model` naming convention, for example `openai/text-embedding-3-small` (default) or `mistral/mistral-embed`. For the full list of available models, see the [Eden AI models catalog](https://www.edenai.co/models).
To start using this integration with Haystack, install it with:
```shell
pip install edenai-haystack
```
`EdenAIDocumentEmbedder` needs an Eden AI API key to work. It uses an `EDENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
from haystack.utils import Secret
from haystack_integrations.components.embedders.edenai import EdenAIDocumentEmbedder
embedder = EdenAIDocumentEmbedder(
api_key=Secret.from_token("`meta`: A dictionary of metadata strings | | **API reference** | [Eden AI](/reference/integrations-edenai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/edenai | | **Package name** | `edenai-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: The Eden AI API key. Can be set with `EDENAI_API_KEY` env var. |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers (vectors)
`meta`: A dictionary of metadata strings | | **API reference** | [Eden AI](/reference/integrations-edenai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/edenai | | **Package name** | `edenai-haystack` |
Use `EdenAITextEmbedder` to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [`EdenAIDocumentEmbedder`](edenaidocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
## Overview
`EdenAITextEmbedder` transforms a string into a vector that captures its semantics using an Eden AI embedding model. Models are selected using Eden AI's `provider/model` naming convention, for example `openai/text-embedding-3-small` (default) or `mistral/mistral-embed`. For the full list of available models, see the [Eden AI models catalog](https://www.edenai.co/models).
To start using this integration with Haystack, install it with:
```shell
pip install edenai-haystack
```
`EdenAITextEmbedder` needs an Eden AI API key to work. It uses an `EDENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
from haystack.utils import Secret
from haystack_integrations.components.embedders.edenai import EdenAITextEmbedder
embedder = EdenAITextEmbedder(
api_key=Secret.from_token("`meta`: A dictionary of metadata strings | | **API reference** | [Eden AI](/reference/integrations-edenai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/edenai | | **Package name** | `edenai-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
| **API reference** | [FastEmbed](/reference/fastembed-embedders) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed |
| **Package name** | `fastembed-haystack` |
This component should be used to embed a list of documents. To embed a string, use the [`FastembedTextEmbedder`](fastembedtextembedder.mdx).
## Overview
`FastembedDocumentEmbedder` computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses embedding [models supported by FastEmbed](https://qdrant.github.io/fastembed/examples/Supported_Models/).
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents in order to find the most similar or relevant documents.
### Compatible models
You can find the original models in the [FastEmbed documentation](https://qdrant.github.io/fastembed/).
Nowadays, most of the models in the [Massive Text Embedding Benchmark (MTEB) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard) are compatible with FastEmbed. You can look for compatibility in the [supported model list](https://qdrant.github.io/fastembed/examples/Supported_Models/).
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install fastembed-haystack
```
### Parameters
You can set the path where the model will be stored in a cache directory. Also, you can set the number of threads a single `onnxruntime` session can use.
```python
cache_dir = "/your_cacheDirectory"
embedder = FastembedDocumentEmbedder(
model="BAAI/bge-large-en-v1.5",
cache_dir=cache_dir,
threads=2,
)
```
If you want to use the data parallel encoding, you can set the parameters `parallel` and `batch_size`.
- If parallel > 1, data-parallel encoding will be used. This is recommended for offline encoding of large datasets.
- If parallel is 0, use all available cores.
- If None, don't use data-parallel processing; use default `onnxruntime` threading instead.
:::tip
If you create a Text Embedder and a Document Embedder based on the same model, Haystack uses the same resource behind the scenes to save resources.
:::
### Embedding Metadata
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
You can do this easily by using the Document Embedder:
```python
from haystack import Document
from haystack_integrations.components.embedders.fastembed import (
FastembedDocumentEmbedder,
)
doc = Document(
content="some text",
meta={"title": "relevant title", "page number": 18},
)
embedder = FastembedDocumentEmbedder(
model="BAAI/bge-small-en-v1.5",
batch_size=256,
meta_fields_to_embed=["title"],
)
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
```
## Usage
### On its own
```python
from haystack.dataclasses import Document
from haystack_integrations.components.embedders.fastembed import (
FastembedDocumentEmbedder,
)
document_list = [
Document(content="I love pizza!"),
Document(content="I like spaghetti"),
]
doc_embedder = FastembedDocumentEmbedder()
result = doc_embedder.run(document_list)
print(result["documents"][0].embedding)
# [-0.04235665127635002, 0.021791068837046623, ...]
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.components.writers import DocumentWriter
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.fastembed import (
FastembedDocumentEmbedder,
FastembedTextEmbedder,
)
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
Document(content="fastembed is supported by and maintained by Qdrant."),
]
document_embedder = FastembedDocumentEmbedder()
writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.OVERWRITE)
indexing_pipeline = Pipeline()
indexing_pipeline.add_component("document_embedder", document_embedder)
indexing_pipeline.add_component("writer", writer)
indexing_pipeline.connect("document_embedder", "writer")
indexing_pipeline.run({"document_embedder": {"documents": documents}})
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", FastembedTextEmbedder())
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Who supports fastembed?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0]) # noqa: T201
# Document(id=...,
# content: 'fastembed is supported by and maintained by Qdrant.',
# score: 0.758..)
```
## Additional References
🧑🍳 Cookbook: [RAG Pipeline Using FastEmbed for Embeddings Generation](https://haystack.deepset.ai/cookbook/rag_fastembed)
---
// File: pipeline-components/embedders/fastembedsparsedocumentembedder
# FastembedSparseDocumentEmbedder
Use this component to enrich a list of documents with their sparse embeddings.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents (enriched with sparse embeddings) |
| **API reference** | [FastEmbed](/reference/fastembed-embedders) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed |
| **Package name** | `fastembed-haystack` |
To compute a sparse embedding for a string, use the [`FastembedSparseTextEmbedder`](fastembedsparsetextembedder.mdx).
## Overview
`FastembedSparseDocumentEmbedder` computes the sparse embeddings of a list of documents and stores the obtained vectors in the `sparse_embedding` field of each document. It uses sparse embedding [models](https://qdrant.github.io/fastembed/examples/Supported_Models/#supported-sparse-text-embedding-models) supported by FastEmbed.
The vectors calculated by this component are necessary for performing sparse embedding retrieval on a set of documents. During retrieval, the sparse vector representing the query is compared to those of the documents to identify the most similar or relevant ones.
### Compatible models
You can find the supported models in the [FastEmbed documentation](https://qdrant.github.io/fastembed/examples/Supported_Models/#supported-sparse-text-embedding-models).
Currently, supported models are based on SPLADE, a technique for producing sparse representations for text, where each non-zero value in the embedding is the importance weight of a term in the BERT WordPiece vocabulary. For more information, see [our docs](../retrievers.mdx#sparse-embedding-based-retrievers) that explain sparse embedding-based Retrievers further.
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install fastembed-haystack
```
### Parameters
You can set the path where the model will be stored in a cache directory. Also, you can set the number of threads a single `onnxruntime` session can use:
```python
cache_dir = "/your_cacheDirectory"
embedder = FastembedSparseDocumentEmbedder(
model="prithivida/Splade_PP_en_v1",
cache_dir=cache_dir,
threads=2,
)
```
If you want to use the data parallel encoding, you can set the parameters `parallel` and `batch_size`.
- If `parallel` > 1, data-parallel encoding will be used. This is recommended for offline encoding of large datasets.
- If `parallel` is 0, use all available cores.
- If None, don't use data-parallel processing; use default `onnxruntime` threading instead.
:::tip
If you create both a Sparse Text Embedder and a Sparse Document Embedder based on the same model, Haystack utilizes a shared resource behind the scenes to conserve resources.
:::
### Embedding Metadata
Text documents often include metadata. If the metadata is distinctive and semantically meaningful, you can embed it along with the document's text to improve retrieval.
You can do this easily by using the sparse Document Embedder:
```python
from haystack import Document
from haystack_integrations.components.embedders.fastembed import (
FastembedSparseDocumentEmbedder,
)
doc = Document(
content="some text",
meta={"title": "relevant title", "page number": 18},
)
embedder = FastembedSparseDocumentEmbedder(
model="prithivida/Splade_PP_en_v1",
meta_fields_to_embed=["title"],
)
docs_w_sparse_embeddings = embedder.run(documents=[doc])["documents"]
```
## Usage
### On its own
```python
from haystack.dataclasses import Document
from haystack_integrations.components.embedders.fastembed import (
FastembedSparseDocumentEmbedder,
)
document_list = [
Document(content="I love pizza!"),
Document(content="I like spaghetti"),
]
doc_embedder = FastembedSparseDocumentEmbedder()
result = doc_embedder.run(document_list)
print(result["documents"][0])
# Document(id=...,
# content: 'I love pizza!',
# sparse_embedding: vector with 24 non-zero elements)
```
### In a pipeline
Currently, sparse embedding retrieval is only supported by `QdrantDocumentStore`.
First, install the package with:
```shell
pip install qdrant-haystack
```
Then, try out this pipeline:
```python
from haystack import Document, Pipeline
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.retrievers.qdrant import (
QdrantSparseEmbeddingRetriever,
)
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.fastembed import (
FastembedSparseDocumentEmbedder,
FastembedSparseTextEmbedder,
)
document_store = QdrantDocumentStore(
":memory:",
recreate_index=True,
use_sparse_embeddings=True,
)
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
Document(content="fastembed is supported by and maintained by Qdrant."),
]
sparse_document_embedder = FastembedSparseDocumentEmbedder()
writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.OVERWRITE)
indexing_pipeline = Pipeline()
indexing_pipeline.add_component("sparse_document_embedder", sparse_document_embedder)
indexing_pipeline.add_component("writer", writer)
indexing_pipeline.connect("sparse_document_embedder", "writer")
indexing_pipeline.run({"sparse_document_embedder": {"documents": documents}})
query_pipeline = Pipeline()
query_pipeline.add_component("sparse_text_embedder", FastembedSparseTextEmbedder())
query_pipeline.add_component(
"sparse_retriever",
QdrantSparseEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect(
"sparse_text_embedder.sparse_embedding",
"sparse_retriever.query_sparse_embedding",
)
query = "Who supports fastembed?"
result = query_pipeline.run({"sparse_text_embedder": {"text": query}})
print(result["sparse_retriever"]["documents"][0]) # noqa: T201
# Document(id=...,
# content: 'fastembed is supported by and maintained by Qdrant.',
# score: 0.758..)
```
## Additional References
🧑🍳 Cookbook: [Sparse Embedding Retrieval with Qdrant and FastEmbed](https://haystack.deepset.ai/cookbook/sparse_embedding_retrieval)
---
// File: pipeline-components/embedders/fastembedsparsetextembedder
# FastembedSparseTextEmbedder
Use this component to embed a simple string (such as a query) into a sparse vector.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a sparse embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `sparse_embedding`: A [`SparseEmbedding`](../../concepts/data-classes.mdx#sparseembedding) object |
| **API reference** | [FastEmbed](/reference/fastembed-embedders) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed |
| **Package name** | `fastembed-haystack` |
For embedding lists of documents, use the [`FastembedSparseDocumentEmbedder`](fastembedsparsedocumentembedder.mdx), which enriches the document with the computed sparse embedding.
## Overview
`FastembedSparseTextEmbedder` transforms a string into a sparse vector using sparse embedding [models](https://qdrant.github.io/fastembed/examples/Supported_Models/#supported-sparse-text-embedding-models) supported by FastEmbed.
When you perform sparse embedding retrieval, use this component first to transform your query into a sparse vector. Then, the sparse embedding Retriever will use the vector to search for similar or relevant documents.
### Compatible Models
You can find the supported models in the [FastEmbed documentation](https://qdrant.github.io/fastembed/examples/Supported_Models/#supported-sparse-text-embedding-models).
Currently, supported models are based on SPLADE, a technique for producing sparse representations for text, where each non-zero value in the embedding is the importance weight of a term in the BERT WordPiece vocabulary. For more information, see [our docs](../retrievers.mdx#sparse-embedding-based-retrievers) that explain sparse embedding-based Retrievers further.
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install fastembed-haystack
```
### Parameters
You can set the path where the model will be stored in a cache directory. Also, you can set the number of threads a single `onnxruntime` session can use:
```python
cache_dir = "/your_cacheDirectory"
embedder = FastembedSparseTextEmbedder(
model="prithivida/Splade_PP_en_v1",
cache_dir=cache_dir,
threads=2,
)
```
If you want to use the data parallel encoding, you can set the `parallel` parameter.
- If `parallel` > 1, data-parallel encoding will be used. This is recommended for offline encoding of large datasets.
- If `parallel` is 0, use all available cores.
- If None, don't use data-parallel processing; use the default `onnxruntime` threading instead.
:::tip
If you create both a Sparse Text Embedder and a Sparse Document Embedder based on the same model, Haystack utilizes a shared resource behind the scenes to conserve resources.
:::
## Usage
### On its own
```python
from haystack_integrations.components.embedders.fastembed import (
FastembedSparseTextEmbedder,
)
text = """It clearly says online this will work on a Mac OS system.
The disk comes and it does not, only Windows.
Do Not order this if you have a Mac!!"""
text_embedder = FastembedSparseTextEmbedder(model="prithivida/Splade_PP_en_v1")
sparse_embedding = text_embedder.run(text)["sparse_embedding"]
```
### In a pipeline
Currently, sparse embedding retrieval is only supported by `QdrantDocumentStore`.
First, install the package with:
```shell
pip install qdrant-haystack
```
Then, try out this pipeline:
```python
from haystack import Document, Pipeline
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
from haystack_integrations.components.retrievers.qdrant import (
QdrantSparseEmbeddingRetriever,
)
from haystack_integrations.components.embedders.fastembed import (
FastembedSparseTextEmbedder,
FastembedSparseDocumentEmbedder,
FastembedTextEmbedder,
)
document_store = QdrantDocumentStore(
":memory:",
recreate_index=True,
use_sparse_embeddings=True,
)
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
Document(content="fastembed is supported by and maintained by Qdrant."),
]
sparse_document_embedder = FastembedSparseDocumentEmbedder(
model="prithivida/Splade_PP_en_v1",
)
documents_with_sparse_embeddings = sparse_document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_sparse_embeddings)
query_pipeline = Pipeline()
query_pipeline.add_component("sparse_text_embedder", FastembedSparseTextEmbedder())
query_pipeline.add_component(
"sparse_retriever",
QdrantSparseEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect(
"sparse_text_embedder.sparse_embedding",
"sparse_retriever.query_sparse_embedding",
)
query = "Who supports fastembed?"
result = query_pipeline.run({"sparse_text_embedder": {"text": query}})
print(result["sparse_retriever"]["documents"][0]) # noqa: T201
# Document(id=...,
# content: 'fastembed is supported by and maintained by Qdrant.',
# score: 0.561..)
```
## Additional References
🧑🍳 Cookbook: [Sparse Embedding Retrieval with Qdrant and FastEmbed](https://haystack.deepset.ai/cookbook/sparse_embedding_retrieval)
---
// File: pipeline-components/embedders/fastembedtextembedder
# FastembedTextEmbedder
This component computes the embeddings of a string using embedding models supported by FastEmbed.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A vector (list of float numbers) |
| **API reference** | [FastEmbed](/reference/fastembed-embedders) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed |
| **Package name** | `fastembed-haystack` |
This component should be used to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [`FastembedDocumentEmbedder`](fastembeddocumentembedder.mdx), which enriches the document with the computed embedding, known as vector.
## Overview
`FastembedTextEmbedder` transforms a string into a vector that captures its semantics using embedding [models supported by FastEmbed](https://qdrant.github.io/fastembed/examples/Supported_Models/).
When you perform embedding retrieval, use this component first to transform your query into a vector. Then, the embedding Retriever will use the vector to search for similar or relevant documents.
### Compatible models
You can find the original models in the [FastEmbed documentation](https://qdrant.github.io/fastembed/).
Currently, most of the models in the [Massive Text Embedding Benchmark (MTEB) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard) are compatible with FastEmbed. You can look for compatibility in the [supported model list](https://qdrant.github.io/fastembed/examples/Supported_Models/).
### Installation
To start using this integration with Haystack, install the package with:
```bash
pip install fastembed-haystack
```
### Instructions
Some recent models that you can find in MTEB require prepending the text with an instruction to work better for retrieval.
For example, if you use `[BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5#model-list)` model, you should prefix your query with the `instruction: “passage:”`.
This is how it works with `FastembedTextEmbedder`:
```python
instruction = "passage:"
embedder = FastembedTextEmbedder(
model="BAAI/bge-large-en-v1.5",
prefix=instruction,
)
```
### Parameters
You can set the path where the model will be stored in a cache directory. Also, you can set the number of threads a single `onnxruntime` session can use.
```python
cache_dir = "/your_cacheDirectory"
embedder = FastembedTextEmbedder(
model="BAAI/bge-large-en-v1.5",
cache_dir=cache_dir,
threads=2,
)
```
If you want to use the data parallel encoding, you can set the parameters `parallel` and `batch_size`.
- If parallel > 1, data-parallel encoding will be used. This is recommended for offline encoding of large datasets.
- If parallel is 0, use all available cores.
- If None, don't use data-parallel processing; use default `onnxruntime` threading instead.
:::tip
If you create a Text Embedder and a Document Embedder based on the same model, Haystack uses the same resource behind the scenes to save resources.
:::
## Usage
### On its own
```python
from haystack_integrations.components.embedders.fastembed import FastembedTextEmbedder
text = """It clearly says online this will work on a Mac OS system.
The disk comes and it does not, only Windows.
Do Not order this if you have a Mac!!"""
text_embedder = FastembedTextEmbedder(model="BAAI/bge-small-en-v1.5")
embedding = text_embedder.run(text)["embedding"]
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.fastembed import (
FastembedDocumentEmbedder,
FastembedTextEmbedder,
)
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
Document(content="fastembed is supported by and maintained by Qdrant."),
]
document_embedder = FastembedDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", FastembedTextEmbedder())
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Who supports FastEmbed?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0]) # noqa: T201
# Document(id=...,
# content: 'fastembed is supported by and maintained by Qdrant.',
# score: 0.758..)
```
## Additional References
🧑🍳 Cookbook: [RAG Pipeline Using FastEmbed for Embeddings Generation](https://haystack.deepset.ai/cookbook/rag_fastembed)
---
// File: pipeline-components/embedders/googlegenaidocumentembedder
# GoogleGenAIDocumentEmbedder
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [DocumentWriter](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Google API key. Can be set with `GOOGLE_API_KEY` or `GEMINI_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata | | **API reference** | [Google GenAI](/reference/integrations-google-genai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_genai | | **Package name** | `google-genai-haystack` |
## Overview
`GoogleGenAIDocumentEmbedder` enriches the metadata of documents with an embedding of their content. To embed a string, you should use the [`GoogleGenAITextEmbedder`](googlegenaitextembedder.mdx).
The component supports [Google AI Embedding models](https://ai.google.dev/gemini-api/docs/embeddings#model-versions).
`gemini-embedding-001` is the default model.
To start using this integration with Haystack, install it with:
```shell
pip install google-genai-haystack
```
### Authentication
Google Gen AI is compatible with both the Gemini Developer API and the Vertex AI API.
To use this component with the Gemini Developer API and get an API key, visit [Google AI Studio](https://aistudio.google.com/).
To use this component with the Vertex AI API, visit [Google Cloud > Vertex AI](https://cloud.google.com/vertex-ai).
The component uses a `GOOGLE_API_KEY` or `GEMINI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` static method:
```python
embedder = GoogleGenAIDocumentEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [Google GenAI](/reference/integrations-google-genai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_genai | | **Package name** | `google-genai-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [DocumentWriter](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Google API key. Can be set with `GOOGLE_API_KEY` or `GEMINI_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents, with a meta field containing an image file path |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata | | **API reference** | [Google GenAI](/reference/integrations-google-genai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_genai | | **Package name** | `google-genai-haystack` |
## Overview
`GoogleGenAIMultimodalDocumentEmbedder` expects a list of documents containing a file path in a meta field. The meta field can be specified with the `file_path_meta_field` init parameter of this component.
The embedder efficiently loads the files, computes the embeddings using a Google AI model, and stores each of them in the `embedding` field of the document.
`GoogleGenAIMultimodalDocumentEmbedder` is commonly used in indexing pipelines. At retrieval time, you need to use the same model with a `GoogleGenAITextEmbedder` to embed the query, before using an Embedding Retriever.
This component is compatible with Gemini multimodal models: `gemini-embedding-2` and later. For a complete list of supported models, see the [Google AI documentation](https://ai.google.dev/gemini-api/docs/embeddings).
To embed a textual document, you should use the [`GoogleGenAIDocumentEmbedder`](googlegenaidocumentembedder.mdx).
To embed a string, you should use the [`GoogleGenAITextEmbedder`](googlegenaitextembedder.mdx).
To start using this integration with Haystack, install it with:
```shell
pip install google-genai-haystack
```
### Authentication
Google Gen AI is compatible with both the Gemini Developer API and the Vertex AI API.
To use this component with the Gemini Developer API and get an API key, visit [Google AI Studio](https://aistudio.google.com/).
To use this component with the Vertex AI API, visit [Google Cloud > Vertex AI](https://cloud.google.com/vertex-ai).
The component uses a `GOOGLE_API_KEY` or `GEMINI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` static method:
```python
embedder = GoogleGenAIMultimodalDocumentEmbedder(
api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [Google GenAI](/reference/integrations-google-genai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_genai | | **Package name** | `google-genai-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: The Google API key. Can be set with `GOOGLE_API_KEY` or `GEMINI_API_KEY` env var. |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers
`meta`: A dictionary of metadata | | **API reference** | [Google GenAI](/reference/integrations-google-genai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_genai | | **Package name** | `google-genai-haystack` |
## Overview
`GoogleGenAITextEmbedder` embeds a simple string (such as a query) into a vector. For embedding lists of documents, use the [`GoogleGenAIDocumentEmbedder`](googlegenaidocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
The component supports [Google AI Embedding models](https://ai.google.dev/gemini-api/docs/embeddings#model-versions).
`gemini-embedding-001` is the default model.
To start using this integration with Haystack, install it with:
```shell
pip install google-genai-haystack
```
### Authentication
Google Gen AI is compatible with both the Gemini Developer API and the Vertex AI API.
To use this component with the Gemini Developer API and get an API key, visit [Google AI Studio](https://aistudio.google.com/).
To use this component with the Vertex AI API, visit [Google Cloud > Vertex AI](https://cloud.google.com/vertex-ai).
The component uses a `GOOGLE_API_KEY` or `GEMINI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` static method:
```python
embedder = GoogleGenAITextEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [Google GenAI](/reference/integrations-google-genai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_genai | | **Package name** | `google-genai-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_type`: The type of Hugging Face API to use
`api_params`: A dictionary with one of the following keys:
- `model`: Hugging Face model ID. Required when `api_type` is `SERVERLESS_INFERENCE_API`.**OR** - `url`: URL of the inference endpoint. Required when `api_type` is `INFERENCE_ENDPOINTS` or `TEXT_EMBEDDINGS_INFERENCE`. | | **Mandatory run variables** | `documents`: A list of documents to be embedded | | **Output variables** | `documents`: A list of documents to be embedded (enriched with embeddings) | | **API reference** | [Hugging Face API](/reference/integrations-huggingface-api) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/huggingface_api | | **Package name** | `huggingface-api-haystack` |
## Overview
`HuggingFaceAPIDocumentEmbedder` can be used to compute document embeddings using different Hugging Face APIs:
- [Free Serverless Inference API](https://huggingface.co/inference-api)
- [Paid Inference Endpoints](https://huggingface.co/inference-endpoints)
- [Self-hosted Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference)
:::info
This component should be used to embed a list of documents. To embed a string, use [`HuggingFaceAPITextEmbedder`](huggingfaceapitextembedder.mdx).
:::
The component uses a `HF_API_TOKEN` environment variable by default. Otherwise, you can pass a Hugging Face API token at initialization with `token` – see code examples below.
The token is needed:
- If you use the Serverless Inference API, or
- If you use the Inference Endpoints.
## Usage
Install the `huggingface-api-haystack` package to use the `HuggingFaceAPIDocumentEmbedder`:
```shell
pip install huggingface-api-haystack
```
Similarly to other Document Embedders, this component allows adding prefixes (and postfixes) to include instruction and embedding metadata.
For more fine-grained details, refer to the component’s [API reference](/reference/integrations-huggingface-api#huggingfaceapidocumentembedder).
### On its own
#### Using Free Serverless Inference API
Formerly known as (free) Hugging Face Inference API, this API allows you to quickly experiment with many models hosted on the Hugging Face Hub, offloading the inference to Hugging Face servers. It’s rate-limited and not meant for production.
To use this API, you need a [free Hugging Face token](https://huggingface.co/settings/tokens).
The Embedder expects the `model` in `api_params`.
```python
from haystack_integrations.components.embedders.huggingface_api import (
HuggingFaceAPIDocumentEmbedder,
)
from haystack.utils import Secret
from haystack.dataclasses import Document
doc = Document(content="I love pizza!")
document_embedder = HuggingFaceAPIDocumentEmbedder(
api_type="serverless_inference_api",
api_params={"model": "BAAI/bge-small-en-v1.5"},
token=Secret.from_token("`api_params`: A dictionary with one of the following keys:
- `model`: Hugging Face model ID. Required when `api_type` is `SERVERLESS_INFERENCE_API`.**OR** - `url`: URL of the inference endpoint. Required when `api_type` is `INFERENCE_ENDPOINTS` or `TEXT_EMBEDDINGS_INFERENCE`. | | **Mandatory run variables** | `documents`: A list of documents to be embedded | | **Output variables** | `documents`: A list of documents to be embedded (enriched with embeddings) | | **API reference** | [Hugging Face API](/reference/integrations-huggingface-api) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/huggingface_api | | **Package name** | `huggingface-api-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_type`: The type of Hugging Face API to use
`api_params`: A dictionary with one of the following keys:
- `model`: Hugging Face model ID. Required when `api_type` is `SERVERLESS_INFERENCE_API`.**OR** - `url`: URL of the inference endpoint. Required when `api_type` is `INFERENCE_ENDPOINTS` or `TEXT_EMBEDDINGS_INFERENCE`. | | **Mandatory run variables** | `text`: A string | | **Output variables** | `embedding`: A list of float numbers | | **API reference** | [Hugging Face API](/reference/integrations-huggingface-api) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/huggingface_api | | **Package name** | `huggingface-api-haystack` |
## Overview
`HuggingFaceAPITextEmbedder` can be used to embed strings using different Hugging Face APIs:
- [Free Serverless Inference API](https://huggingface.co/inference-api)
- [Paid Inference Endpoints](https://huggingface.co/inference-endpoints)
- [Self-hosted Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference)
:::info
This component should be used to embed plain text. To embed a list of documents, use [`HuggingFaceAPIDocumentEmbedder`](huggingfaceapidocumentembedder.mdx).
:::
The component uses a `HF_API_TOKEN` environment variable by default. Otherwise, you can pass a Hugging Face API token at initialization with `token` – see code examples below.
The token is needed:
- If you use the Serverless Inference API, or
- If you use the Inference Endpoints.
## Usage
Install the `huggingface-api-haystack` package to use the `HuggingFaceAPITextEmbedder`:
```shell
pip install huggingface-api-haystack
```
Similarly to other text Embedders, this component allows adding prefixes (and postfixes) to include instructions.
For more fine-grained details, refer to the component’s [API reference](/reference/integrations-huggingface-api#huggingfaceapitextembedder).
### On its own
#### Using Free Serverless Inference API
Formerly known as (free) Hugging Face Inference API, this API allows you to quickly experiment with many models hosted on the Hugging Face Hub, offloading the inference to Hugging Face servers. It’s rate-limited and not meant for production.
To use this API, you need a [free Hugging Face token](https://huggingface.co/settings/tokens).
The Embedder expects the `model` in `api_params`.
```python
from haystack_integrations.components.embedders.huggingface_api import (
HuggingFaceAPITextEmbedder,
)
from haystack.utils import Secret
text_embedder = HuggingFaceAPITextEmbedder(
api_type="serverless_inference_api",
api_params={"model": "BAAI/bge-small-en-v1.5"},
token=Secret.from_token("`api_params`: A dictionary with one of the following keys:
- `model`: Hugging Face model ID. Required when `api_type` is `SERVERLESS_INFERENCE_API`.**OR** - `url`: URL of the inference endpoint. Required when `api_type` is `INFERENCE_ENDPOINTS` or `TEXT_EMBEDDINGS_INFERENCE`. | | **Mandatory run variables** | `text`: A string | | **Output variables** | `embedding`: A list of float numbers | | **API reference** | [Hugging Face API](/reference/integrations-huggingface-api) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/huggingface_api | | **Package name** | `huggingface-api-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Jina API key. Can be set with `JINA_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata | | **API reference** | [Jina](/reference/integrations-jina) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina | | **Package name** | `jina-haystack` |
## Overview
`JinaDocumentEmbedder` enriches the metadata of documents with an embedding of their content. To embed a string, you should use the [`JinaTextEmbedder`](jinatextembedder.mdx). To see the list of compatible Jina Embeddings models, head to Jina AI’s [website](https://jina.ai/embeddings/). The default model for `JinaDocumentEmbedder` is `jina-embeddings-v3`.
To start using this integration with Haystack, install the package with:
```shell
pip install jina-haystack
```
The component uses a `JINA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
embedder = JinaDocumentEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [Jina](/reference/integrations-jina) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina | | **Package name** | `jina-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Jina API key. Can be set with `JINA_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents, with a meta field containing an image file path |
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
| **API reference** | [Jina](/reference/integrations-jina) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina |
| **Package name** | `jina-haystack` |
## Overview
`JinaDocumentImageEmbedder` expects a list of documents containing an image or a PDF file path in a meta field. The meta field can be specified with the `file_path_meta_field` init parameter of this component.
The embedder efficiently loads the images, computes the embeddings using a Jina model, and stores each of them in the `embedding` field of the document.
`JinaDocumentImageEmbedder` is commonly used in indexing pipelines. At retrieval time, you need to use the same model with a `JinaTextEmbedder` to embed the query, before using an Embedding Retriever.
This component is compatible with Jina multimodal embedding models:
- `jina-clip-v1`
- `jina-clip-v2` (default)
- `jina-embeddings-v4` (non-commercial research only)
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install jina-haystack
```
### Authentication
The component uses a `JINA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` method:
```python
embedder = JinaDocumentImageEmbedder(api_key=Secret.from_token("
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: The Jina API key. Can be set with `JINA_API_KEY` env var. |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers
`meta`: A dictionary of metadata | | **API reference** | [Jina](/reference/integrations-jina) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina | | **Package name** | `jina-haystack` |
## Overview
`JinaTextEmbedder` embeds a simple string (such as a query) into a vector. For embedding lists of documents, use the use the [`JinaDocumentEmbedder`](jinadocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector. To see the list of compatible Jina Embeddings models, head to Jina AI’s [website](https://jina.ai/embeddings/). The default model for `JinaTextEmbedder` is `jina-embeddings-v3`.
To start using this integration with Haystack, install the package with:
```shell
pip install jina-haystack
```
The component uses a `JINA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
embedder = JinaTextEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [Jina](/reference/integrations-jina) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina | | **Package name** | `jina-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Mistral API key. Can be set with `MISTRAL_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata strings | | **API reference** | [Mistral](/reference/integrations-mistral) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral | | **Package name** | `mistral-haystack` |
This component should be used to embed a list of Documents. To embed a string, use the [`MistralTextEmbedder`](mistraltextembedder.mdx).
## Overview
`MistralDocumentEmbedder` computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses the Mistral API and its embedding models.
The component currently supports the `mistral-embed` embedding model. The list of all supported models can be found in Mistral’s [embedding models documentation](https://docs.mistral.ai/platform/endpoints/#embedding-models).
To start using this integration with Haystack, install it with:
```shell
pip install mistral-haystack
```
`MistralDocumentEmbedder` needs a Mistral API key to work. It uses an `MISTRAL_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
embedder = MistralDocumentEmbedder(
api_key=Secret.from_token("`meta`: A dictionary of metadata strings | | **API reference** | [Mistral](/reference/integrations-mistral) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral | | **Package name** | `mistral-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: The Mistral API key. Can be set with `MISTRAL_API_KEY` env var. |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers (vectors)
`meta`: A dictionary of metadata strings | | **API reference** | [Mistral](/reference/integrations-mistral) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral | | **Package name** | `mistral-haystack` |
Use `MistralTextEmbedder` to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [`MistralDocumentEmbedder`](mistraldocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
## Overview
`MistralTextEmbedder` transforms a string into a vector that captures its semantics using a Mistral embedding model.
The component currently supports the `mistral-embed` embedding model. The list of all supported models can be found in Mistral’s [embedding models documentation](https://docs.mistral.ai/platform/endpoints/#embedding-models).
To start using this integration with Haystack, install it with:
```shell
pip install mistral-haystack
```
`MistralTextEmbedder` needs a Mistral API key to work. It uses a `MISTRAL_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
embedder = MistralTextEmbedder(
api_key=Secret.from_token("`meta`: A dictionary of metadata strings | | **API reference** | [Mistral](/reference/integrations-mistral) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral | | **Package name** | `mistral-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In place of a real Document Embedder, in tests and prototypes |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents enriched with embeddings
`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/mock_document_embedder.py | | **Package name** | `haystack-ai` |
## Overview
`MockDocumentEmbedder` is a deterministic, zero-cost drop-in replacement for real Document Embedders such as `OpenAIDocumentEmbedder`. It implements `run`, `run_async`, and serialization like any other embedder but never contacts an external service, which makes it ideal for unit tests, smoke tests, and quick prototypes.
The embedding is selected based on how the component is configured:
- **Deterministic (default)**: With no configuration, each document's embedding is derived from a stable hash of its prepared text. The same text always yields the same unit-length embedding, and different texts yield different embeddings, so the mock works in retrieval pipelines and is reproducible across runs and processes.
- **Fixed embedding**: Pass an `embedding` vector. The same vector is assigned to every document.
- **Dynamic embedding**: Pass an `embedding_fn` callable that receives the prepared text of a document and returns the embedding. To support serialization, pass a named function.
`embedding` and `embedding_fn` are mutually exclusive.
Further optional parameters:
- `dimension`: The number of dimensions of the deterministic embedding. Defaults to `768`. Ignored when `embedding` or `embedding_fn` is provided, since their length is determined by the value or callable.
- `model`: The model name reported in the metadata. Defaults to `"mock-model"`.
- `meta`: Additional metadata merged into the output `meta`.
- `prefix` / `suffix`: Strings added to the beginning and end of each text before embedding, mirroring real embedders.
- `meta_fields_to_embed` / `embedding_separator`: Like real Document Embedders, the metadata fields listed in `meta_fields_to_embed` are concatenated with the document content before embedding, so the deterministic embedding reflects the embedded metadata.
- `progress_bar`: Accepted for interface compatibility with real Document Embedders and ignored.
:::info
The deterministic embeddings are derived from a hash: identical texts get identical vectors and the similarity between different texts is stable but arbitrary. For exact-match retrieval in tests this is exactly what you want. Do not expect semantically similar texts to end up close together.
:::
Use `MockDocumentEmbedder` for documents and its counterpart [`MockTextEmbedder`](mocktextembedder.mdx) for queries. With the default deterministic mode, a query whose text matches a document's content produces the same vector, so the document is retrieved as the top hit.
## Usage
### On its own
```python
from haystack import Document
from haystack.components.embedders import MockDocumentEmbedder
embedder = MockDocumentEmbedder(dimension=8)
result = embedder.run([Document(content="I love pizza!")])
print(result["documents"][0].embedding) # a deterministic list of 8 floats
```
### In a pipeline
Use it in an indexing pipeline exactly like a real Document Embedder — no API key needed:
```python
from haystack import Document, Pipeline
from haystack.components.embedders import MockDocumentEmbedder
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
indexing_pipeline = Pipeline()
indexing_pipeline.add_component("embedder", MockDocumentEmbedder(dimension=8))
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
indexing_pipeline.connect("embedder.documents", "writer.documents")
indexing_pipeline.run(
{
"embedder": {
"documents": [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
],
},
},
)
print(document_store.count_documents()) # 2
```
---
// File: pipeline-components/embedders/mocktextembedder
# MockTextEmbedder
A Text Embedder that returns deterministic embeddings without calling any API, for tests and quick prototypes.
`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/mock_document_embedder.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In place of a real Text Embedder, in tests and prototypes |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers
`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/mock_text_embedder.py | | **Package name** | `haystack-ai` |
## Overview
`MockTextEmbedder` is a deterministic, zero-cost drop-in replacement for real Text Embedders such as `OpenAITextEmbedder`. It implements `run`, `run_async`, and serialization like any other embedder but never contacts an external service, which makes it ideal for unit tests, smoke tests, and quick prototypes.
The embedding is selected based on how the component is configured:
- **Deterministic (default)**: With no configuration, the embedding is derived from a stable hash of the input text. The same text always yields the same unit-length embedding, and different texts yield different embeddings, so the mock works in retrieval pipelines and is reproducible across runs and processes.
- **Fixed embedding**: Pass an `embedding` vector. The same vector is returned for every input.
- **Dynamic embedding**: Pass an `embedding_fn` callable that receives the prepared text (after `prefix`/`suffix` are applied) and returns the embedding. To support serialization, pass a named function.
`embedding` and `embedding_fn` are mutually exclusive.
Further optional parameters:
- `dimension`: The number of dimensions of the deterministic embedding. Defaults to `768`. Ignored when `embedding` or `embedding_fn` is provided, since their length is determined by the value or callable.
- `model`: The model name reported in the metadata. Defaults to `"mock-model"`.
- `meta`: Additional metadata merged into the output `meta`.
- `prefix` / `suffix`: Strings added to the beginning and end of the text before embedding, mirroring real embedders.
:::info
The deterministic embeddings are derived from a hash: identical texts get identical vectors and the similarity between different texts is stable but arbitrary. For exact-match retrieval in tests this is exactly what you want. Do not expect semantically similar texts to end up close together.
:::
Use `MockTextEmbedder` for queries and its counterpart [`MockDocumentEmbedder`](mockdocumentembedder.mdx) for documents. With the default deterministic mode, a query whose text matches a document's content produces the same vector, so the document is retrieved as the top hit.
## Usage
### On its own
```python
from haystack.components.embedders import MockTextEmbedder
embedder = MockTextEmbedder(dimension=8)
result = embedder.run("I love pizza!")
print(result["embedding"]) # a deterministic list of 8 floats
```
### In a pipeline
A retrieval pipeline built with mock embedders runs without any API key and always returns the same result for the same input:
```python
from haystack import Document, Pipeline
from haystack.components.embedders import MockDocumentEmbedder, MockTextEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
]
indexed = MockDocumentEmbedder(dimension=8).run(documents=documents)
document_store.write_documents(indexed["documents"])
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", MockTextEmbedder(dimension=8))
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
result = query_pipeline.run(
{"text_embedder": {"text": "I saw a black horse running"}},
)
print(result["retriever"]["documents"][0].content) # "I saw a black horse running"
```
---
// File: pipeline-components/embedders/nvidiadocumentembedder
# NvidiaDocumentEmbedder
This component computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document.
`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/mock_text_embedder.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: API key for the NVIDIA NIM. Can be set with `NVIDIA_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata | | **API reference** | [NVIDIA](/reference/integrations-nvidia) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia | | **Package name** | `nvidia-haystack` |
## Overview
`NvidiaDocumentEmbedder` enriches documents with an embedding of their content.
You can use this component with self-hosted models using NVIDIA NIM or models hosted on the [NVIDIA API Catalog](https://build.nvidia.com/explore/discover).
To embed a string, use [`NvidiaTextEmbedder`](nvidiatextembedder.mdx).
## Usage
To start using `NvidiaDocumentEmbedder`, install the `nvidia-haystack` package:
```shell
pip install nvidia-haystack
```
You can use `NvidiaDocumentEmbedder` with all the embedding models available on the [NVIDIA API Catalog](https://docs.api.nvidia.com/nim/reference) or with a model deployed using NVIDIA NIM. For more information, refer to [NIM for Embedding](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/index.html).
### On its own
To use models from the NVIDIA API Catalog, you need to specify the `api_url` and your API key. You can get your API key from the [NVIDIA API Catalog](https://build.nvidia.com/explore/discover).
`NvidiaDocumentEmbedder` uses the `NVIDIA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with the `api_key` parameter:
```python
from haystack import Document
from haystack.utils.auth import Secret
from haystack_integrations.components.embedders.nvidia import NvidiaDocumentEmbedder
documents = [
Document(content="A transformer is a deep learning architecture"),
Document(content="Large language models use transformer architectures"),
]
embedder = NvidiaDocumentEmbedder(
model="nvidia/nv-embedqa-e5-v5",
api_url="https://integrate.api.nvidia.com/v1",
api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [NVIDIA](/reference/integrations-nvidia) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia | | **Package name** | `nvidia-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: API key for the NVIDIA NIM. Can be set with `NVIDIA_API_KEY` env var. |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers (vectors)
`meta`: A dictionary of metadata strings | | **API reference** | [NVIDIA](/reference/integrations-nvidia) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia | | **Package name** | `nvidia-haystack` |
## Overview
`NvidiaTextEmbedder` embeds a simple string (such as a query) into a vector.
You can use this component with self-hosted models using NVIDIA NIM or models hosted on the [NVIDIA API Catalog](https://build.nvidia.com/explore/discover).
To embed a list of documents, use [`NvidiaDocumentEmbedder`](nvidiadocumentembedder.mdx), which enriches each document with the computed embedding.
## Usage
To start using `NvidiaTextEmbedder`, install the `nvidia-haystack` package:
```shell
pip install nvidia-haystack
```
You can use `NvidiaTextEmbedder` with all the embedding models available on the [NVIDIA API Catalog](https://docs.api.nvidia.com/nim/reference) or with a model deployed using NVIDIA NIM. For more information, refer to [NIM for Embedding](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/index.html).
### On its own
To use models from the NVIDIA API Catalog, you need to specify the `api_url` and your API key. You can get your API key from the [NVIDIA API Catalog](https://build.nvidia.com/explore/discover).
`NvidiaTextEmbedder` uses the `NVIDIA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with the `api_key` parameter:
```python
from haystack.utils.auth import Secret
from haystack_integrations.components.embedders.nvidia import NvidiaTextEmbedder
embedder = NvidiaTextEmbedder(
model="nvidia/nv-embedqa-e5-v5",
api_url="https://integrate.api.nvidia.com/v1",
api_key=Secret.from_token("`meta`: A dictionary of metadata strings | | **API reference** | [NVIDIA](/reference/integrations-nvidia) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia | | **Package name** | `nvidia-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata strings | | **API reference** | [Ollama](/reference/integrations-ollama) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama | | **Package name** | `ollama-haystack` |
`OllamaDocumentEmbedder` computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses embedding models compatible with the Ollama Library.
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents to find the most similar or relevant documents.
## Overview
`OllamaDocumentEmbedder` should be used to embed a list of documents. For embedding a string only, use the [`OllamaTextEmbedder`](ollamatextembedder.mdx).
The component uses `http://localhost:11434` as the default URL as most available setups (Mac, Linux, Docker) default to port 11434.
### Compatible Models
Unless specified otherwise while initializing this component, the default embedding model is "nomic-embed-text". See other possible pre-built models in Ollama's [library](https://ollama.com/library). To load your own custom model, follow the [instructions](https://docs.ollama.com/modelfile) from Ollama.
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install ollama-haystack
```
Make sure that you have a running Ollama model (either through a docker container, or locally hosted). No other configuration is necessary as Ollama has the embedding API built in.
### Embedding Metadata
Most embedded metadata contains information about the model name and type. You can pass [optional arguments](https://docs.ollama.com/modelfile#valid-parameters-and-values), such as temperature, top_p, and others, to the Ollama generation endpoint.
The name of the model used will be automatically appended as part of the document metadata. An example payload using the nomic-embed-text model will look like this:
```python
{"meta": {"model": "nomic-embed-text"}}
```
## Usage
### On its own
```python
from haystack import Document
from haystack_integrations.components.embedders.ollama import OllamaDocumentEmbedder
doc = Document(content="What do llamas say once you have thanked them? No probllama!")
document_embedder = OllamaDocumentEmbedder()
result = document_embedder.run([doc])
print(result["documents"][0].embedding)
# Calculating embeddings: 100%|██████████| 1/1 [00:02<00:00, 2.82s/it]
# [-0.16412407159805298, -3.8359334468841553, ... ]
```
### In a pipeline
```python
from haystack import Pipeline
from haystack_integrations.components.embedders.ollama import OllamaDocumentEmbedder
from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter
from haystack.components.converters import PyPDFToDocument
from haystack.components.writers import DocumentWriter
from haystack.document_stores.types import DuplicatePolicy
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
embedder = OllamaDocumentEmbedder(
model="nomic-embed-text",
url="http://localhost:11434",
) # This is the default model and URL
cleaner = DocumentCleaner()
splitter = DocumentSplitter()
file_converter = PyPDFToDocument()
writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.OVERWRITE)
indexing_pipeline = Pipeline()
# Add components to pipeline
indexing_pipeline.add_component("embedder", embedder)
indexing_pipeline.add_component("converter", file_converter)
indexing_pipeline.add_component("cleaner", cleaner)
indexing_pipeline.add_component("splitter", splitter)
indexing_pipeline.add_component("writer", writer)
# Connect components in pipeline
indexing_pipeline.connect("converter", "cleaner")
indexing_pipeline.connect("cleaner", "splitter")
indexing_pipeline.connect("splitter", "embedder")
indexing_pipeline.connect("embedder", "writer")
# Run Pipeline
indexing_pipeline.run({"converter": {"sources": ["files/test_pdf_data.pdf"]}})
# Calculating embeddings: 100%|██████████| 115/115
# {'embedder': {'meta': {'model': 'nomic-embed-text'}}, 'writer': {'documents_written': 115}}
```
---
// File: pipeline-components/embedders/ollamatextembedder
# OllamaTextEmbedder
This component computes the embeddings of a string using embedding models compatible with the Ollama Library.
`meta`: A dictionary of metadata strings | | **API reference** | [Ollama](/reference/integrations-ollama) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama | | **Package name** | `ollama-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers (vectors)
`meta`: A dictionary of metadata strings | | **API reference** | [Ollama](/reference/integrations-ollama) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama | | **Package name** | `ollama-haystack` |
`OllamaTextEmbedder` computes the embeddings of a string and returns the obtained vector. It uses embedding models compatible with the Ollama Library.
When you perform embedding retrieval, use this component first to transform your query into a vector. Then, the embedding Retriever uses that vector to search for similar or relevant documents.
## Overview
`OllamaTextEmbedder` should be used to embed a string. For embedding a list of documents, use the [`OllamaDocumentEmbedder`](ollamadocumentembedder.mdx).
The component uses `http://localhost:11434` as the default URL as most available setups (Mac, Linux, Docker) default to port 11434.
### Compatible Models
Unless specified otherwise while initializing this component, the default embedding model is "nomic-embed-text". See other possible pre-built models in Ollama's [library](https://ollama.com/library). To load your own custom model, follow the [instructions](https://docs.ollama.com/modelfile) from Ollama.
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install ollama-haystack
```
Make sure that you have a running Ollama model (either through a docker container, or locally hosted). No other configuration is necessary as Ollama has the embedding API built in.
### Embedding Metadata
Most embedded metadata contains information about the model name and type. You can pass [optional arguments](https://docs.ollama.com/modelfile#valid-parameters-and-values), such as temperature, top_p, and others, to the Ollama generation endpoint.
The name of the model used will be automatically appended as part of the metadata. An example payload using the nomic-embed-text model will look like this:
```python
{"meta": {"model": "nomic-embed-text"}}
```
## Usage
### On its own
```python
from haystack_integrations.components.embedders.ollama import OllamaTextEmbedder
embedder = OllamaTextEmbedder()
result = embedder.run(
text="What do llamas say once you have thanked them? No probllama!",
)
print(result["embedding"])
```
### In a pipeline
```python
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.ollama import (
OllamaDocumentEmbedder,
OllamaTextEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
]
document_embedder = OllamaDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", OllamaTextEmbedder())
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Who lives in Berlin?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/embedders/openaidocumentembedder
# OpenAIDocumentEmbedder
OpenAIDocumentEmbedder computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses OpenAI embedding models.
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents.
`meta`: A dictionary of metadata strings | | **API reference** | [Ollama](/reference/integrations-ollama) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama | | **Package name** | `ollama-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/openai_document_embedder.py | | **Package name** | `haystack-ai` |
## Overview
To see the list of compatible OpenAI embedding models, head over to OpenAI [documentation](https://platform.openai.com/docs/guides/embeddings). The default model for `OpenAIDocumentEmbedder` is `text-embedding-ada-002`. You can specify another model with the `model` parameter when initializing this component.
This component should be used to embed a list of documents. To embed a string, use the [OpenAITextEmbedder](openaitextembedder.mdx).
The component uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```
embedder = OpenAIDocumentEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/openai_document_embedder.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers
`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/openai_text_embedder.py | | **Package name** | `haystack-ai` |
## Overview
To see the list of compatible OpenAI embedding models, head over to OpenAI [documentation](https://platform.openai.com/docs/guides/embeddings). The default model for `OpenAITextEmbedder` is `text-embedding-ada-002`. You can specify another model with the `model` parameter when initializing this component.
Use `OpenAITextEmbedder` to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [OpenAIDocumentEmbedder](openaidocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
The component uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
embedder = OpenAITextEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [Embedders](/reference/embedders-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/openai_text_embedder.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents enriched with embeddings |
| **API reference** | [Optimum](/reference/integrations-optimum) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/optimum |
| **Package name** | `optimum-haystack` |
## Overview
`OptimumDocumentEmbedder` embeds text strings using models loaded with the [HuggingFace Optimum](https://huggingface.co/docs/optimum/index) library. It uses the [ONNX runtime](https://onnxruntime.ai/) for high-speed inference.
The default model is `sentence-transformers/all-mpnet-base-v2`.
Similarly to other Embedders, this component allows adding prefixes (and suffixes) to include instructions. For more details, refer to the component’s API reference.
There are three useful parameters specific to the Optimum Embedder that you can control with various modes:
- [Pooling](/reference/integrations-optimum#optimumembedderpooling): generate a fixed-sized sentence embedding from a variable-sized sentence embedding
- [Optimization](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/optimization): apply graph optimization to the model and improve inference speed
- [Quantization](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/quantization): reduce the computational and memory costs
Find all the available mode details in our Optimum [API Reference](/reference/integrations-optimum).
### Authentication
Authentication with a Hugging Face API Token is only required to access private or gated models through Serverless Inference API or the Inference Endpoints.
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
## Usage
To start using this integration with Haystack, install it with:
```shell
pip install optimum-haystack
```
### On its own
```python
from haystack.dataclasses import Document
from haystack_integrations.components.embedders.optimum import OptimumDocumentEmbedder
doc = Document(content="I love pizza!")
document_embedder = OptimumDocumentEmbedder(
model="sentence-transformers/all-mpnet-base-v2",
)
result = document_embedder.run([doc])
print(result["documents"][0].embedding)
# [0.017020374536514282, -0.023255806416273117, ...]
```
### In a pipeline
```python
from haystack import Pipeline
from haystack import Document
from haystack_integrations.components.embedders.optimum import (
OptimumDocumentEmbedder,
OptimumEmbedderPooling,
OptimumEmbedderOptimizationConfig,
OptimumEmbedderOptimizationMode,
)
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
]
embedder = OptimumDocumentEmbedder(
model="intfloat/e5-base-v2",
normalize_embeddings=True,
onnx_execution_provider="CUDAExecutionProvider",
optimizer_settings=OptimumEmbedderOptimizationConfig(
mode=OptimumEmbedderOptimizationMode.O4,
for_gpu=True,
),
working_dir="/tmp/optimum",
pooling_mode=OptimumEmbedderPooling.MEAN,
)
pipeline = Pipeline()
pipeline.add_component("embedder", embedder)
results = pipeline.run({"embedder": {"documents": documents}})
print(results["embedder"]["documents"][0].embedding)
```
---
// File: pipeline-components/embedders/optimumtextembedder
# OptimumTextEmbedder
A component to embed text using models loaded with the Hugging Face Optimum library.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers (vectors) |
| **API reference** | [Optimum](/reference/integrations-optimum) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/optimum |
| **Package name** | `optimum-haystack` |
## Overview
`OptimumTextEmbedder` embeds text strings using models loaded with the [HuggingFace Optimum](https://huggingface.co/docs/optimum/index) library. It uses the [ONNX runtime](https://onnxruntime.ai/) for high-speed inference.
The default model is `sentence-transformers/all-mpnet-base-v2`.
Similarly to other Embedders, this component allows adding prefixes (and suffixes) to include instructions. For more details, refer to the component’s API reference.
There are three useful parameters specific to the Optimum Embedder that you can control with various modes:
- [Pooling](/reference/integrations-optimum#optimumembedderpooling): generate a fixed-sized sentence embedding from a variable-sized sentence embedding
- [Optimization](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/optimization): apply graph optimization to the model and improve inference speed
- [Quantization](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/quantization): reduce the computational and memory costs
Find all the available mode details in our Optimum [API Reference](/reference/integrations-optimum).
### Authentication
Authentication with a Hugging Face API Token is only required to access private or gated models through Serverless Inference API or the Inference Endpoints.
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
## Usage
To start using this integration with Haystack, install it with:
```shell
pip install optimum-haystack
```
### On its own
```python
from haystack_integrations.components.embedders.optimum import OptimumTextEmbedder
text_to_embed = "I love pizza!"
text_embedder = OptimumTextEmbedder(model="sentence-transformers/all-mpnet-base-v2")
print(text_embedder.run(text_to_embed))
# {'embedding': [-0.07804739475250244, 0.1498992145061493, ...]}
```
### In a pipeline
Note that this example requires GPU support to execute.
```python
from haystack import Pipeline
from haystack_integrations.components.embedders.optimum import (
OptimumTextEmbedder,
OptimumEmbedderPooling,
OptimumEmbedderOptimizationConfig,
OptimumEmbedderOptimizationMode,
)
pipeline = Pipeline()
embedder = OptimumTextEmbedder(
model="intfloat/e5-base-v2",
normalize_embeddings=True,
onnx_execution_provider="CUDAExecutionProvider",
optimizer_settings=OptimumEmbedderOptimizationConfig(
mode=OptimumEmbedderOptimizationMode.O4,
for_gpu=True,
),
working_dir="/tmp/optimum",
pooling_mode=OptimumEmbedderPooling.MEAN,
)
pipeline.add_component("embedder", embedder)
results = pipeline.run(
{
"embedder": {
"text": "Ex profunditate antique doctrinae, Ad caelos supra semper, Hoc incantamentum evoco, draco apparet, Incantamentum iam transactum est",
},
},
)
print(results["embedder"]["embedding"])
```
---
// File: pipeline-components/embedders/perplexitydocumentembedder
# PerplexityDocumentEmbedder
`PerplexityDocumentEmbedder` computes the embeddings of a list of documents and stores the obtained vectors in the `embedding` field of each document. It uses Perplexity embedding models.
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: A Perplexity API key. Can be set with `PERPLEXITY_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata | | **API reference** | [Integrations](/reference/integrations-perplexity) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/perplexity/src/haystack_integrations/components/embedders/perplexity/document_embedder.py | | **Package name** | `perplexity-haystack` |
## Overview
`PerplexityDocumentEmbedder` supports the following embedding models:
- `pplx-embed-v1-0.6b` (default)
- `pplx-embed-v1-4b`
Use this component to embed a list of documents. To embed a single string (such as a query), use [PerplexityTextEmbedder](perplexitytextembedder.mdx).
The component uses a `PERPLEXITY_API_KEY` environment variable by default. You can also pass an API key directly at initialization:
```python
from haystack_integrations.components.embedders.perplexity import (
PerplexityDocumentEmbedder,
)
from haystack.utils import Secret
embedder = PerplexityDocumentEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [Integrations](/reference/integrations-perplexity) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/perplexity/src/haystack_integrations/components/embedders/perplexity/document_embedder.py | | **Package name** | `perplexity-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: A Perplexity API key. Can be set with `PERPLEXITY_API_KEY` env var. |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers
`meta`: A dictionary of metadata | | **API reference** | [Integrations](/reference/integrations-perplexity) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/perplexity/src/haystack_integrations/components/embedders/perplexity/text_embedder.py | | **Package name** | `perplexity-haystack` |
## Overview
`PerplexityTextEmbedder` supports the following embedding models:
- `pplx-embed-v1-0.6b` (default)
- `pplx-embed-v1-4b`
Use `PerplexityTextEmbedder` to embed a single string, such as a query. For embedding lists of documents, use [PerplexityDocumentEmbedder](perplexitydocumentembedder.mdx).
The component uses a `PERPLEXITY_API_KEY` environment variable by default. You can also pass an API key directly at initialization:
```python
from haystack_integrations.components.embedders.perplexity import PerplexityTextEmbedder
from haystack.utils import Secret
embedder = PerplexityTextEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [Integrations](/reference/integrations-perplexity) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/perplexity/src/haystack_integrations/components/embedders/perplexity/text_embedder.py | | **Package name** | `perplexity-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
| **API reference** | [Sentence Transformers](/reference/integrations-sentence-transformers) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sentence_transformers |
| **Package name** | `sentence-transformers-haystack` |
## Overview
`SentenceTransformersDocumentEmbedder` should be used to embed a list of documents. To embed a string, use the [SentenceTransformersTextEmbedder](sentencetransformerstextembedder.mdx).
### Authentication
Authentication with a Hugging Face API Token is only required to access private or gated models through Serverless Inference API or the Inference Endpoints.
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
```python
document_embedder = SentenceTransformersDocumentEmbedder(
token=Secret.from_token("
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `documents`: A list of documents, with a meta field containing an image file path |
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
| **API reference** | [Sentence Transformers](/reference/integrations-sentence-transformers) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sentence_transformers |
| **Package name** | `sentence-transformers-haystack` |
## Overview
`SentenceTransformersDocumentImageEmbedder` expects a list of documents containing an image or a PDF file path in a meta field. The meta field can be specified with the `file_path_meta_field` init parameter of this component.
The embedder efficiently loads the images, computes the embeddings using a Sentence Transformers models, and stores each of them in the `embedding` field of the document.
`SentenceTransformersDocumentImageEmbedder` is commonly used in indexing pipelines. At retrieval time, you need to use the same model with a `SentenceTransformersTextEmbedder` to embed the query before using an Embedding Retriever.
You can set the `device` parameter to use HF models on your CPU or GPU.
Additionally, you can select the backend to use for the Sentence Transformers mode with the `backend` parameter: `torch` (default), `onnx`, or `openvino`. ONNX and OpenVINO allow specific speed optimizations; for more information, read the [Sentence Transformers documentation](https://sbert.net/docs/sentence_transformer/usage/efficiency.html).
### Authentication
Authentication with a Hugging Face API Token is only required to access private or gated models.
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
### Compatible Models
To be used with this component, the model must be compatible with Sentence Transformers and
able to embed images and text into the same vector space. Compatible models include:
- `sentence-transformers/clip-ViT-B-32` (default)
- `sentence-transformers/clip-ViT-L-14`
- `sentence-transformers/clip-ViT-B-16`
- `sentence-transformers/clip-ViT-B-32-multilingual-v1`
- `jinaai/jina-embeddings-v4`
- `jinaai/jina-clip-v1`
- `jinaai/jina-clip-v2`
## Usage
Install the `sentence-transformers-haystack` package to use the `SentenceTransformersDocumentImageEmbedder`:
```shell
pip install sentence-transformers-haystack
```
### On its own
```python
from haystack import Document
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentImageEmbedder,
)
embedder = SentenceTransformersDocumentImageEmbedder(
model="sentence-transformers/clip-ViT-B-32",
)
documents = [
Document(content="A photo of a cat", meta={"file_path": "cat.jpg"}),
Document(content="A photo of a dog", meta={"file_path": "dog.jpg"}),
]
result = embedder.run(documents=documents)
documents_with_embeddings = result["documents"]
print(documents_with_embeddings)
# [Document(id=...,
# content='A photo of a cat',
# meta={'file_path': 'cat.jpg',
# 'embedding_source': {'type': 'image', 'file_path_meta_field': 'file_path'}},
# embedding=vector of size 512),
# ...]
```
### In a pipeline
In this example, we can see an indexing pipeline with 3 components:
- `ImageFileToDocument` Converter that creates empty documents with a reference to an image in the `meta.file_path` field,
- `SentenceTransformersDocumentImageEmbedder` that loads the images, computes embeddings and stores them in documents,
- `DocumentWriter` that writes the documents in the `InMemoryDocumentStore`
There is also a multimodal retrieval pipeline, composed by a `SentenceTransformersTextEmbedder` (using the same model as before) and an `InMemoryEmbeddingRetriever`.
```python
from haystack import Pipeline
from haystack.components.converters.image import ImageFileToDocument
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentImageEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
# Indexing pipeline
indexing_pipeline = Pipeline()
indexing_pipeline.add_component("image_converter", ImageFileToDocument())
indexing_pipeline.add_component(
"embedder",
SentenceTransformersDocumentImageEmbedder(
model="sentence-transformers/clip-ViT-B-32",
),
)
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
indexing_pipeline.connect("image_converter", "embedder")
indexing_pipeline.connect("embedder", "writer")
indexing_pipeline.run(data={"image_converter": {"sources": ["dog.jpg", "hyena.jpeg"]}})
# Multimodal retrieval pipeline
retrieval_pipeline = Pipeline()
retrieval_pipeline.add_component(
"embedder",
SentenceTransformersTextEmbedder(model="sentence-transformers/clip-ViT-B-32"),
)
retrieval_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store, top_k=2),
)
retrieval_pipeline.connect("embedder", "retriever")
result = retrieval_pipeline.run(data={"text": "man's best friend"})
print(result)
# {
# 'retriever': {
# 'documents': [
# Document(
# id=0c96...,
# meta={
# 'file_path': 'dog.jpg',
# 'embedding_source': {
# 'type': 'image',
# 'file_path_meta_field': 'file_path'
# }
# },
# score=32.025817780129856
# ),
# Document(
# id=5e76...,
# meta={
# 'file_path': 'hyena.jpeg',
# 'embedding_source': {
# 'type': 'image',
# 'file_path_meta_field': 'file_path'
# }
# },
# score=20.648225327085242
# )
# ]
# }
# }
```
## Additional References
🧑🍳 Cookbook: [Introduction to Multimodality](https://haystack.deepset.ai/cookbook/multimodal_intro)
---
// File: pipeline-components/embedders/sentencetransformerssparsedocumentembedder
# SentenceTransformersSparseDocumentEmbedder
Use this component to enrich a list of documents with their sparse embeddings using Sentence Transformers models.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents (enriched with sparse embeddings) |
| **API reference** | [Sentence Transformers](/reference/integrations-sentence-transformers) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sentence_transformers |
| **Package name** | `sentence-transformers-haystack` |
To compute a sparse embedding for a string, use the [`SentenceTransformersSparseTextEmbedder`](sentencetransformerssparsetextembedder.mdx).
## Overview
`SentenceTransformersSparseDocumentEmbedder` computes the sparse embeddings of a list of documents and stores the obtained vectors in the `sparse_embedding` field of each document. It uses sparse embedding models supported by the Sentence Transformers library.
The vectors computed by this component are necessary to perform sparse embedding retrieval on a collection of documents. At retrieval time, the sparse vector representing the query is compared with those of the documents to find the most similar or relevant ones.
### Compatible Models
The default embedding model is [`prithivida/Splade_PP_en_v2`](https://huggingface.co/prithivida/Splade_PP_en_v2). You can specify another model with the `model` parameter when initializing this component.
Compatible models are based on SPLADE (SParse Lexical AnD Expansion), a technique for producing sparse representations for text, where each non-zero value in the embedding is the importance weight of a term in the vocabulary. This approach combines the benefits of learned sparse representations with the efficiency of traditional sparse retrieval methods. For more information, see [our docs](../retrievers.mdx#sparse-embedding-based-retrievers) that explain sparse embedding-based Retrievers further.
You can find compatible SPLADE models on the [Hugging Face Model Hub](https://huggingface.co/models?search=splade).
### Authentication
Authentication with a Hugging Face API Token is only required to access private or gated models.
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
```python
from haystack.utils import Secret
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersSparseDocumentEmbedder,
)
document_embedder = SentenceTransformersSparseDocumentEmbedder(
token=Secret.from_token("
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a sparse embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `sparse_embedding`: A [`SparseEmbedding`](../../concepts/data-classes.mdx#sparseembedding) object |
| **API reference** | [Sentence Transformers](/reference/integrations-sentence-transformers) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sentence_transformers |
| **Package name** | `sentence-transformers-haystack` |
For embedding lists of documents, use the [`SentenceTransformersSparseDocumentEmbedder`](sentencetransformerssparsedocumentembedder.mdx), which enriches the document with the computed sparse embedding.
## Overview
`SentenceTransformersSparseTextEmbedder` transforms a string into a sparse vector using sparse embedding models supported by the Sentence Transformers library.
When you perform sparse embedding retrieval, use this component first to transform your query into a sparse vector. Then, the Retriever will use the sparse vector to search for similar or relevant documents.
### Compatible Models
The default embedding model is [`prithivida/Splade_PP_en_v2`](https://huggingface.co/prithivida/Splade_PP_en_v2). You can specify another model with the `model` parameter when initializing this component.
Compatible models are based on SPLADE (SParse Lexical AnD Expansion), a technique for producing sparse representations for text, where each non-zero value in the embedding is the importance weight of a term in the vocabulary. This approach combines the benefits of learned sparse representations with the efficiency of traditional sparse retrieval methods. For more information, see [our docs](../retrievers.mdx#sparse-embedding-based-retrievers) that explain sparse embedding-based Retrievers further.
You can find compatible SPLADE models on the [Hugging Face Model Hub](https://huggingface.co/models?search=splade).
### Authentication
Authentication with a Hugging Face API Token is only required to access private or gated models.
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
```python
from haystack.utils import Secret
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersSparseTextEmbedder,
)
text_embedder = SentenceTransformersSparseTextEmbedder(
token=Secret.from_token("
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers |
| **API reference** | [Sentence Transformers](/reference/integrations-sentence-transformers) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sentence_transformers |
| **Package name** | `sentence-transformers-haystack` |
## Overview
This component should be used to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [SentenceTransformersDocumentEmbedder](sentencetransformersdocumentembedder.mdx), which enriches the document with the computed embedding, known as vector.
### Authentication
Authentication with a Hugging Face API Token is only required to access private or gated models through Serverless Inference API or the Inference Endpoints.
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
```python
text_embedder = SentenceTransformersTextEmbedder(
token=Secret.from_token("
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [DocumentWriter](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `model`: The model used through the STACKIT API |
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
| **Output variables** | `documents`: A list of documents enriched with embeddings |
| **API reference** | [STACKIT](/reference/integrations-stackit) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit |
| **Package name** | `stackit-haystack` |
## Overview
`STACKITDocumentEmbedder` enables document embedding models served by STACKIT through their API.
### Parameters
To use the `STACKITDocumentEmbedder`, ensure you have set a `STACKIT_API_KEY` as an environment variable. Alternatively, provide the API key as an environment variable with a different name or a token by setting `api_key` and using Haystack’s [secret management](../../concepts/secret-management.mdx).
Set your preferred supported model with the `model` parameter when initializing the component. See the full list of all supported models on the [STACKIT website](https://docs.stackit.cloud/products/data-and-ai/ai-model-serving/basics/available-shared-models/).
Optionally, you can change the default `api_base_url`, which is `"https://api.openai-compat.model-serving.eu01.onstackit.cloud/v1"`.
Other optional parameters include `prefix` and `suffix` (added to each text before embedding), `batch_size`, `meta_fields_to_embed`, `embedding_separator`, and `dimensions`.
The component needs a list of documents as input to operate.
## Usage
Install the `stackit-haystack` package to use the `STACKITDocumentEmbedder` and set an environment variable called `STACKIT_API_KEY` to your API key.
```shell
pip install stackit-haystack
```
### On its own
```python
from haystack import Document
from haystack_integrations.components.embedders.stackit import STACKITDocumentEmbedder
doc = Document(content="I love pizza!")
document_embedder = STACKITDocumentEmbedder(model="intfloat/e5-mistral-7b-instruct")
result = document_embedder.run([doc])
print(result["documents"][0].embedding)
# [0.0215301513671875, 0.01499176025390625, ...]
```
### In a pipeline
You can also use `STACKITDocumentEmbedder` in your pipeline in a following way.
```python
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.stackit import (
STACKITTextEmbedder,
STACKITDocumentEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
document_store = InMemoryDocumentStore()
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
]
document_embedder = STACKITDocumentEmbedder(model="intfloat/e5-mistral-7b-instruct")
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)
text_embedder = STACKITTextEmbedder(model="intfloat/e5-mistral-7b-instruct")
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", text_embedder)
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Where does Wolfgang live?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
# Document(id=..., content: 'My name is Wolfgang and I live in Berlin', score: ...)
```
You can find more usage examples in the STACKIT integration [repository](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit/examples) and its [integration page](https://haystack.deepset.ai/integrations/stackit).
---
// File: pipeline-components/embedders/stackittextembedder
# STACKITTextEmbedder
This component enables text embedding using the STACKIT API.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `model`: The model used through the STACKIT API |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers |
| **API reference** | [STACKIT](/reference/integrations-stackit) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit |
| **Package name** | `stackit-haystack` |
## Overview
`STACKITTextEmbedder` enables text embedding models served by STACKIT through their API.
### Parameters
To use the `STACKITTextEmbedder`, ensure you have set a `STACKIT_API_KEY` as an environment variable. Alternatively, provide the API key as an environment variable with a different name or a token by setting `api_key` and using Haystack’s [secret management](../../concepts/secret-management.mdx).
Set your preferred supported model with the `model` parameter when initializing the component. See the full list of all supported models on the [STACKIT website](https://docs.stackit.cloud/products/data-and-ai/ai-model-serving/basics/available-shared-models/).
Optionally, you can change the default `api_base_url`, which is `"https://api.openai-compat.model-serving.eu01.onstackit.cloud/v1"`.
Other optional parameters include `prefix` and `suffix` (added to the text before embedding) and `dimensions`.
The component needs a text input to operate.
## Usage
Install the `stackit-haystack` package to use the `STACKITTextEmbedder` and set an environment variable called `STACKIT_API_KEY` to your API key.
```shell
pip install stackit-haystack
```
### On its own
```python
from haystack_integrations.components.embedders.stackit import STACKITTextEmbedder
text_embedder = STACKITTextEmbedder(model="intfloat/e5-mistral-7b-instruct")
print(text_embedder.run("I love pizza!"))
# {'embedding': [0.0215301513671875, 0.01499176025390625, ...]}
```
### In a pipeline
You can also use `STACKITTextEmbedder` in your pipeline.
```python
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.stackit import (
STACKITTextEmbedder,
STACKITDocumentEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
document_store = InMemoryDocumentStore()
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
]
document_embedder = STACKITDocumentEmbedder(model="intfloat/e5-mistral-7b-instruct")
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)
text_embedder = STACKITTextEmbedder(model="intfloat/e5-mistral-7b-instruct")
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", text_embedder)
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Where does Wolfgang live?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
# Document(id=..., content: 'My name is Wolfgang and I live in Berlin', score: ...)
```
You can find more usage examples in the STACKIT integration [repository](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit/examples) and its [integration page](https://haystack.deepset.ai/integrations/stackit).
---
// File: pipeline-components/embedders/twelvelabsdocumentembedder
# TwelveLabsDocumentEmbedder
This component computes the embeddings of a list of documents using the TwelveLabs Marengo multimodal embedding model and stores the obtained vectors in the embedding field of each document. The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: The TwelveLabs API key. Can be set with `TWELVELABS_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata | | **API reference** | [TwelveLabs](/reference/integrations-twelvelabs) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/twelvelabs | | **Package name** | `twelvelabs-haystack` |
## Overview
`TwelveLabsDocumentEmbedder` enriches each document with an embedding of its content. To embed a string, use the [`TwelveLabsTextEmbedder`](twelvelabstextembedder.mdx). The default model is `marengo3.0`.
Because Marengo embeds text, images, audio, and video into a single shared space, these embeddings support cross-modal retrieval.
To start using this integration with Haystack, install the package with:
```shell
pip install twelvelabs-haystack
```
The component uses a `TWELVELABS_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
from haystack.utils import Secret
from haystack_integrations.components.embedders.twelvelabs import (
TwelveLabsDocumentEmbedder,
)
embedder = TwelveLabsDocumentEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [TwelveLabs](/reference/integrations-twelvelabs) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/twelvelabs | | **Package name** | `twelvelabs-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: The TwelveLabs API key. Can be set with `TWELVELABS_API_KEY` env var. |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers
`meta`: A dictionary of metadata | | **API reference** | [TwelveLabs](/reference/integrations-twelvelabs) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/twelvelabs | | **Package name** | `twelvelabs-haystack` |
## Overview
`TwelveLabsTextEmbedder` embeds a simple string (such as a query) into a vector. For embedding lists of documents, use the [`TwelveLabsDocumentEmbedder`](twelvelabsdocumentembedder.mdx), which enriches each document with the computed embedding. The default model is `marengo3.0`.
Because Marengo embeds into a single shared space, embeddings produced from text are directly comparable (cosine similarity) with embeddings of images, audio, and video from the same model.
To start using this integration with Haystack, install the package with:
```shell
pip install twelvelabs-haystack
```
The component uses a `TWELVELABS_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
from haystack.utils import Secret
from haystack_integrations.components.embedders.twelvelabs import TwelveLabsTextEmbedder
embedder = TwelveLabsTextEmbedder(api_key=Secret.from_token("`meta`: A dictionary of metadata | | **API reference** | [TwelveLabs](/reference/integrations-twelvelabs) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/twelvelabs | | **Package name** | `twelvelabs-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [DocumentWriter](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `model`: The model used through the VertexAI Embeddings API |
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
| **Output variables** | `documents`: A list of documents enriched with embeddings |
| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
| **Package name** | `google-vertex-haystack` |
`VertexAIDocumentEmbedder` enriches the metadata of documents with an embedding of their content. To embed a string, use the [`VertexAITextEmbedder`](vertexaitextembedder.mdx).
To use the `VertexAIDocumentEmbedder`, initialize it with:
- `model`: The supported models are:
- "text-embedding-004"
- "text-embedding-005"
- "textembedding-gecko-multilingual@001"
- "text-multilingual-embedding-002"
- "text-embedding-large-exp-03-07"
- `task_type`: "RETRIEVAL_DOCUMENT” is the default. You can find all task types in the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype).
### Authentication
`VertexAIDocumentEmbedder` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
## Usage
Install the `google-vertex-haystack` package to use this Embedder:
```shell
pip install google-vertex-haystack
```
### On its own
```python
from haystack import Document
from haystack_integrations.components.embedders.google_vertex import (
VertexAIDocumentEmbedder,
)
doc = Document(content="I love pizza!")
document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005")
result = document_embedder.run([doc])
print(result["documents"][0].embedding)
# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665,
```
### In a pipeline
```python
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.google_vertex import (
VertexAITextEmbedder,
)
from haystack_integrations.components.embedders.google_vertex import (
VertexAIDocumentEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
]
document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005")
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
VertexAITextEmbedder(model="text-embedding-005"),
)
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Who lives in Berlin?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
# Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
```
---
// File: pipeline-components/embedders/vertexaitextembedder
# VertexAITextEmbedder
This component computes embeddings for text (such as a query) using models through VertexAI Embeddings API.
:::warning[Deprecation Notice]
The `google-vertex-haystack` integration is archived and no longer maintained. It builds on a deprecated Google SDK.
We recommend switching to the [GoogleGenAITextEmbedder](googlegenaitextembedder.mdx) from the `google-genai-haystack` package instead.
:::
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `model`: The model used through the VertexAI Embeddings API |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A list of float numbers |
| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
| **Package name** | `google-vertex-haystack` |
## Overview
`VertexAITextEmbedder` embeds a simple string (such as a query) into a vector. For embedding lists of documents, use the [`VertexAIDocumentEmbedder`](vertexaidocumentembedder.mdx) which enriches the document with the computed embedding, also known as vector.
To start using the `VertexAITextEmbedder`, initialize it with:
- `model`: The supported models are:
- "text-embedding-004"
- "text-embedding-005"
- "textembedding-gecko-multilingual@001"
- "text-multilingual-embedding-002"
- "text-embedding-large-exp-03-07"
- `task_type`: "RETRIEVAL_QUERY” is the default. You can find all task types in the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype).
### Authentication
`VertexAITextEmbedder` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
## Usage
Install the `google-vertex-haystack` package to use this Embedder:
```shell
pip install google-vertex-haystack
```
### On its own
```python
from haystack_integrations.components.embedders.google_vertex import (
VertexAITextEmbedder,
)
text_to_embed = "I love pizza!"
text_embedder = VertexAITextEmbedder(model="text-embedding-005")
print(text_embedder.run(text_to_embed))
# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...]
```
### In a pipeline
```python
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.google_vertex import (
VertexAITextEmbedder,
)
from haystack_integrations.components.embedders.google_vertex import (
VertexAIDocumentEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
]
document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005")
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
VertexAITextEmbedder(model="text-embedding-005"),
)
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Who lives in Berlin?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
# Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
```
---
// File: pipeline-components/embedders/vllmdocumentembedder
# VLLMDocumentEmbedder
This component computes the embeddings of a list of documents using models served with [vLLM](https://docs.vllm.ai/).
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `model`: The name of the model served by vLLM |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
| **API reference** | [vLLM](/reference/integrations-vllm) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/vllm |
| **Package name** | `vllm-haystack` |
## Overview
[vLLM](https://docs.vllm.ai/) is a high-throughput and memory-efficient inference and serving engine for LLMs. It exposes an OpenAI-compatible HTTP server, which `VLLMDocumentEmbedder` uses to compute embeddings through the Embeddings API.
`VLLMDocumentEmbedder` computes the embeddings of a list of documents and stores the obtained vectors in the `embedding` field of each document. It expects a vLLM server to be running and accessible at the `api_base_url` parameter (by default, `http://localhost:8000/v1`). To embed a string (such as a query), use the [`VLLMTextEmbedder`](vllmtextembedder.mdx).
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents to find the most similar or relevant ones.
If the vLLM server was started with `--api-key`, provide the API key through the `VLLM_API_KEY` environment variable or the `api_key` init parameter using Haystack's [Secret](../../concepts/secret-management.mdx) API.
### Compatible models
vLLM supports a range of embedding models. Check the [vLLM pooling models docs](https://docs.vllm.ai/en/stable/models/pooling_models) for the list of supported architectures and models.
### vLLM-specific parameters
You can pass vLLM-specific parameters through the `extra_parameters` dictionary. These are forwarded as `extra_body` to the OpenAI-compatible embeddings endpoint. Use this to pass parameters that are not part of the standard OpenAI Embeddings API, such as `truncate_prompt_tokens` or `truncation_side`. See the [vLLM Embeddings API docs](https://docs.vllm.ai/en/stable/models/pooling_models/embed/#openai-compatible-embeddings-api) for details.
```python
embedder = VLLMDocumentEmbedder(
model="google/embeddinggemma-300m",
extra_parameters={"truncate_prompt_tokens": 256, "truncation_side": "right"},
)
```
### Matryoshka embeddings
If the model was trained with Matryoshka Representation Learning, you can reduce the dimensionality of the output vector through the `dimensions` parameter. See the [vLLM Matryoshka docs](https://docs.vllm.ai/en/stable/models/pooling_models/embed/#matryoshka-embeddings) for details.
### Batching and failure handling
`VLLMDocumentEmbedder` encodes documents in batches. Use `batch_size` (default `32`) to control how many documents are sent in a single request to the vLLM server, and `progress_bar` to toggle the progress indicator.
By default (`raise_on_failure=False`), failed embedding requests are logged and processing continues with the remaining documents. Set `raise_on_failure=True` to raise an exception instead.
### Instructions
Some embedding models require prepending the document text with an instruction to work better for retrieval. For example, if you use [intfloat/e5-large-v2](https://huggingface.co/intfloat/e5-large-v2), you should prefix your document with the following instruction: "passage:".
This is how it works with `VLLMDocumentEmbedder`:
```python
instruction = "passage:"
embedder = VLLMDocumentEmbedder(
model="intfloat/e5-large-v2",
prefix=instruction,
)
```
### Embedding metadata
Documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval. Pass the relevant fields through `meta_fields_to_embed`; they are concatenated to the document text using `embedding_separator` (a newline by default):
```python
from haystack import Document
from haystack_integrations.components.embedders.vllm import VLLMDocumentEmbedder
doc = Document(content="some text", meta={"title": "relevant title", "page_number": 18})
embedder = VLLMDocumentEmbedder(
model="google/embeddinggemma-300m",
meta_fields_to_embed=["title"],
)
docs_with_embeddings = embedder.run(documents=[doc])["documents"]
```
## Usage
Install the `vllm-haystack` package to use the `VLLMDocumentEmbedder`:
```shell
pip install vllm-haystack
```
### Starting the vLLM server
Before using this component, start a vLLM server with an embedding model:
```bash
vllm serve google/embeddinggemma-300m
```
For details on server options, see the [vLLM CLI docs](https://docs.vllm.ai/en/stable/cli/serve/).
### On its own
```python
from haystack import Document
from haystack_integrations.components.embedders.vllm import VLLMDocumentEmbedder
doc = Document(content="I love pizza!")
document_embedder = VLLMDocumentEmbedder(model="google/embeddinggemma-300m")
result = document_embedder.run([doc])
print(result["documents"][0].embedding)
# [-0.0215301513671875, 0.01499176025390625, ...]
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.vllm import (
VLLMDocumentEmbedder,
VLLMTextEmbedder,
)
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
]
document_embedder = VLLMDocumentEmbedder(model="google/embeddinggemma-300m")
writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.OVERWRITE)
indexing_pipeline = Pipeline()
indexing_pipeline.add_component("document_embedder", document_embedder)
indexing_pipeline.add_component("writer", writer)
indexing_pipeline.connect("document_embedder", "writer")
indexing_pipeline.run({"document_embedder": {"documents": documents}})
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
VLLMTextEmbedder(model="google/embeddinggemma-300m"),
)
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Who lives in Berlin?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
# Document(id=..., content: 'My name is Wolfgang and I live in Berlin', score: ...)
```
---
// File: pipeline-components/embedders/vllmtextembedder
# VLLMTextEmbedder
This component computes the embeddings of a string using models served with [vLLM](https://docs.vllm.ai/).
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `model`: The name of the model served by vLLM |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `embedding`: A vector (list of float numbers) |
| **API reference** | [vLLM](/reference/integrations-vllm) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/vllm |
| **Package name** | `vllm-haystack` |
## Overview
[vLLM](https://docs.vllm.ai/) is a high-throughput and memory-efficient inference and serving engine for LLMs. It exposes an OpenAI-compatible HTTP server, which `VLLMTextEmbedder` uses to compute embeddings through the Embeddings API.
`VLLMTextEmbedder` expects a vLLM server to be running and accessible at the `api_base_url` parameter (by default, `http://localhost:8000/v1`). Use this component to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [`VLLMDocumentEmbedder`](vllmdocumentembedder.mdx).
When you perform embedding retrieval, use this component first to transform your query into a vector. Then, the embedding Retriever will use the vector to search for similar or relevant documents.
If the vLLM server was started with `--api-key`, provide the API key through the `VLLM_API_KEY` environment variable or the `api_key` init parameter using Haystack's [Secret](../../concepts/secret-management.mdx) API.
### Compatible models
vLLM supports a range of embedding models. Check the [vLLM pooling models docs](https://docs.vllm.ai/en/stable/models/pooling_models) for the list of supported architectures and models.
### vLLM-specific parameters
You can pass vLLM-specific parameters through the `extra_parameters` dictionary. These are forwarded as `extra_body` to the OpenAI-compatible embeddings endpoint. Use this to pass parameters that are not part of the standard OpenAI Embeddings API, such as `truncate_prompt_tokens` or `truncation_side`. See the [vLLM Embeddings API docs](https://docs.vllm.ai/en/stable/models/pooling_models/embed/#openai-compatible-embeddings-api) for details.
```python
embedder = VLLMTextEmbedder(
model="google/embeddinggemma-300m",
extra_parameters={"truncate_prompt_tokens": 256, "truncation_side": "right"},
)
```
### Matryoshka embeddings
If the model was trained with Matryoshka Representation Learning, you can reduce the dimensionality of the output vector through the `dimensions` parameter. See the [vLLM Matryoshka docs](https://docs.vllm.ai/en/stable/models/pooling_models/embed/#matryoshka-embeddings) for details.
### Instructions
Some embedding models require prepending the text with an instruction to work better for retrieval. For example, if you use [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5#model-list), you should prefix your query with the following instruction: "Represent this sentence for searching relevant passages:".
This is how it works with `VLLMTextEmbedder`:
```python
instruction = "Represent this sentence for searching relevant passages:"
embedder = VLLMTextEmbedder(
model="BAAI/bge-large-en-v1.5",
prefix=instruction,
)
```
## Usage
Install the `vllm-haystack` package to use the `VLLMTextEmbedder`:
```shell
pip install vllm-haystack
```
### Starting the vLLM server
Before using this component, start a vLLM server with an embedding model:
```bash
vllm serve google/embeddinggemma-300m
```
For details on server options, see the [vLLM CLI docs](https://docs.vllm.ai/en/stable/cli/serve/).
### On its own
```python
from haystack_integrations.components.embedders.vllm import VLLMTextEmbedder
text_embedder = VLLMTextEmbedder(model="google/embeddinggemma-300m")
print(text_embedder.run("I love pizza!"))
# {'embedding': [-0.0215301513671875, 0.01499176025390625, ...], 'meta': {...}}
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.vllm import (
VLLMDocumentEmbedder,
VLLMTextEmbedder,
)
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
]
document_embedder = VLLMDocumentEmbedder(model="google/embeddinggemma-300m")
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
VLLMTextEmbedder(model="google/embeddinggemma-300m"),
)
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Who lives in Berlin?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
# Document(id=..., content: 'My name is Wolfgang and I live in Berlin', score: ...)
```
---
// File: pipeline-components/embedders/watsonxdocumentembedder
# WatsonxDocumentEmbedder
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents to find the most similar or relevant documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
| **Mandatory init variables** | `api_key`: The IBM Cloud API key. Can be set with `WATSONX_API_KEY` env var.
`project_id`: The IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. | | **Mandatory run variables** | `documents`: A list of documents to be embedded | | **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata strings | | **API reference** | [Watsonx](/reference/integrations-watsonx) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx | | **Package name** | `watsonx-haystack` |
## Overview
`WatsonxDocumentEmbedder` enriches the metadata of documents with an embedding of their content. To embed a string, you should use the [`WatsonxTextEmbedder`](watsonxtextembedder.mdx).
The component supports IBM watsonx.ai embedding models such as `ibm/slate-30m-english-rtrvr-v2` and similar. The default model is `ibm/slate-30m-english-rtrvr-v2`. This list of all supported models can be found in IBM's [model documentation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models-embed.html?context=wx).
To start using this integration with Haystack, install it with:
```shell
pip install watsonx-haystack
```
The component uses `WATSONX_API_KEY` and `WATSONX_PROJECT_ID` environment variables by default. Otherwise, you can pass API credentials at initialization with `api_key` and `project_id`:
```python
embedder = WatsonxDocumentEmbedder(
api_key=Secret.from_token("`project_id`: The IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. | | **Mandatory run variables** | `documents`: A list of documents to be embedded | | **Output variables** | `documents`: A list of documents (enriched with embeddings)
`meta`: A dictionary of metadata strings | | **API reference** | [Watsonx](/reference/integrations-watsonx) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx | | **Package name** | `watsonx-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
| **Mandatory init variables** | `api_key`: An IBM Cloud API key. Can be set with `WATSONX_API_KEY` env var.
`project_id`: An IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. | | **Mandatory run variables** | `text`: A string | | **Output variables** | `embedding`: A list of float numbers
`meta`: A dictionary of metadata | | **API reference** | [Watsonx](/reference/integrations-watsonx) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx | | **Package name** | `watsonx-haystack` |
## Overview
To see the list of compatible IBM watsonx.ai embedding models, head over to IBM [documentation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models-embed.html?context=wx). The default model for `WatsonxTextEmbedder` is `ibm/slate-30m-english-rtrvr-v2`. You can specify another model with the `model` parameter when initializing this component.
Use `WatsonxTextEmbedder` to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [`WatsonxDocumentEmbedder`](watsonxdocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
The component uses `WATSONX_API_KEY` and `WATSONX_PROJECT_ID` environment variables by default. Otherwise, you can pass API credentials at initialization with `api_key` and `project_id`:
```python
embedder = WatsonxTextEmbedder(
api_key=Secret.from_token("`project_id`: An IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. | | **Mandatory run variables** | `text`: A string | | **Output variables** | `embedding`: A list of float numbers
`meta`: A dictionary of metadata | | **API reference** | [Watsonx](/reference/integrations-watsonx) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx | | **Package name** | `watsonx-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
| **Mandatory run variables** | `ground_truth_answers`: A list of strings containing the ground truth answers
`predicted_answers`: A list of strings containing the predicted answers to be evaluated | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 representing the proportion of questions in which any predicted answer matched the ground truth answers
- `individual_scores`: A list of 0s and 1s, where 1 means that the predicted answer matched one of the ground truths | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/answer_exact_match.py | | **Package name** | `haystack-ai` |
## Overview
You can use the `AnswerExactMatchEvaluator` component to evaluate answers predicted by a Haystack pipeline, such as an extractive question answering pipeline, against ground truth labels. As the `AnswerExactMatchEvaluator` checks whether a predicted answer exactly matches the ground truth answer. It is not suited to evaluate answers generated by LLMs, for example, in a RAG pipeline. Use `FaithfulnessEvaluator` or `SASEvaluator` instead.
To initialize an `AnswerExactMatchEvaluator`, there are no parameters required.
Note that only _one_ predicted answer is compared to _one_ ground truth answer at a time. The component does not support multiple ground truth answers for the same question or multiple answers predicted for the same question.
## Usage
### On its own
Below is an example of using an `AnswerExactMatchEvaluator` component to evaluate two answers and compare them to ground truth answers.
```python
from haystack.components.evaluators import AnswerExactMatchEvaluator
evaluator = AnswerExactMatchEvaluator()
result = evaluator.run(
ground_truth_answers=["Berlin", "Paris"],
predicted_answers=["Berlin", "Lyon"],
)
print(result["individual_scores"])
# [1, 0]
print(result["score"])
# 0.5
```
### In a pipeline
Below is an example where we use an `AnswerExactMatchEvaluator` and a `SASEvaluator` in a pipeline to evaluate two answers and compare them to ground truth answers. Running a pipeline instead of the individual components simplifies calculating more than one metric.
```python
from haystack import Pipeline
from haystack.components.evaluators import AnswerExactMatchEvaluator
from haystack.components.evaluators import SASEvaluator
pipeline = Pipeline()
em_evaluator = AnswerExactMatchEvaluator()
sas_evaluator = SASEvaluator()
pipeline.add_component("em_evaluator", em_evaluator)
pipeline.add_component("sas_evaluator", sas_evaluator)
ground_truth_answers = ["Berlin", "Paris"]
predicted_answers = ["Berlin", "Lyon"]
result = pipeline.run(
{
"em_evaluator": {
"ground_truth_answers": ground_truth_answers,
"predicted_answers": predicted_answers,
},
"sas_evaluator": {
"ground_truth_answers": ground_truth_answers,
"predicted_answers": predicted_answers,
},
},
)
for evaluator in result:
print(result[evaluator]["individual_scores"])
# [1, 0]
# [1.0, 0.5174766182899475]
for evaluator in result:
print(result[evaluator]["score"])
# 0.5
# 0.7587383091449738
```
---
// File: pipeline-components/evaluators/contextrelevanceevaluator
# ContextRelevanceEvaluator
The `ContextRelevanceEvaluator` uses an LLM to evaluate whether contexts are relevant to a question. It does not require ground truth labels.
`predicted_answers`: A list of strings containing the predicted answers to be evaluated | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 representing the proportion of questions in which any predicted answer matched the ground truth answers
- `individual_scores`: A list of 0s and 1s, where 1 means that the predicted answer matched one of the ground truths | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/answer_exact_match.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
| **Mandatory run variables** | `questions`: A list of questions
`contexts`: A list of a list of contexts, which are the contents of documents. This accounts for one list of contexts per question. | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 that represents the mean context relevance score over all input questions
- `individual_scores`: A list of the individual context relevance scores, each either 0 or 1, for each input pair of a question and a list of contexts
- `results`: A list of dictionaries with keys `relevant_statements`, `score`, and `status`. They contain the statements that an LLM found relevant in each context, the binary score for that context, and `evaluated` for valid results or `error` for failed evaluations. | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/context_relevance.py | | **Package name** | `haystack-ai` |
## Overview
You can use the `ContextRelevanceEvaluator` component to evaluate documents retrieved by a Haystack pipeline, such as a RAG pipeline, without ground truth labels. The component breaks up the context into multiple statements and checks whether each statement is relevant for answering a question. The score for each context is binary: 1 if the LLM found at least one relevant statement in it, 0 otherwise. The overall `score` is the mean of these binary scores over all input questions, so it is a number from 0.0 to 1.0.
### Parameters
The default model for this Evaluator is `gpt-5-mini`. You can override the model using the `chat_generator` parameter during initialization. This needs to be a Chat Generator instance configured to return a JSON object. For example, when using the [`OpenAIChatGenerator`](../generators/openaichatgenerator.mdx), you should pass `{"response_format": {"type": "json_object"}}` in its `generation_kwargs`.
If you are not initializing the Evaluator with your own Chat Generator other than OpenAI, a valid OpenAI API key must be set as an `OPENAI_API_KEY` environment variable. For details, see our [documentation page on secret management](../../concepts/secret-management.mdx).
Two optional initialization parameters are:
- `raise_on_failure`: If True, raise an exception on an unsuccessful API call.
- `progress_bar`: Whether to show a progress bar during the evaluation.
`ContextRelevanceEvaluator` has an optional `examples` parameter that can be used to pass few-shot examples conforming to the expected input and output format of `ContextRelevanceEvaluator`. These examples are included in the prompt that is sent to the LLM. Examples, therefore, increase the number of tokens of the prompt and make each request more costly. Adding examples is helpful if you want to improve the quality of the evaluation at the cost of more tokens.
Each example must be a dictionary with keys `inputs` and `outputs`.
`inputs` must be a dictionary with keys `questions` and `contexts`.
`outputs` must be a dictionary with `relevant_statements`.
Here is the expected format:
```python
[
{
"inputs": {
"questions": "What is the capital of Italy?",
"contexts": ["Rome is the capital of Italy."],
},
"outputs": {
"relevant_statements": ["Rome is the capital of Italy."],
},
},
]
```
## Usage
### On its own
Below is an example where we use a `ContextRelevanceEvaluator` component to evaluate a response generated based on a provided question and context. The `ContextRelevanceEvaluator` returns a score of 1 because it finds a statement in the context that is relevant to the question.
```python
from haystack.components.evaluators import ContextRelevanceEvaluator
questions = ["Who created the Python language?"]
contexts = [
[
"Python, created by Guido van Rossum in the late 1980s, is a high-level general-purpose programming language. Its design philosophy emphasizes code readability, and its language constructs aim to help programmers write clear, logical code for both small and large-scale software projects.",
],
]
evaluator = ContextRelevanceEvaluator()
result = evaluator.run(questions=questions, contexts=contexts)
print(result["score"])
# 1
print(result["individual_scores"])
# [1]
print(result["results"])
# [{'relevant_statements': ['Python, created by Guido van Rossum in the late 1980s, is a high-level general-purpose programming language.'], 'status': 'evaluated', 'score': 1}]
```
### In a pipeline
Below is an example where we use a `FaithfulnessEvaluator` and a `ContextRelevanceEvaluator` in a pipeline to evaluate responses and contexts (the content of documents) received by a RAG pipeline based on provided questions. Running a pipeline instead of the individual components simplifies calculating more than one metric.
```python
from haystack import Pipeline
from haystack.components.evaluators import (
ContextRelevanceEvaluator,
FaithfulnessEvaluator,
)
pipeline = Pipeline()
context_relevance_evaluator = ContextRelevanceEvaluator()
faithfulness_evaluator = FaithfulnessEvaluator()
pipeline.add_component("context_relevance_evaluator", context_relevance_evaluator)
pipeline.add_component("faithfulness_evaluator", faithfulness_evaluator)
questions = ["Who created the Python language?"]
contexts = [
[
"Python, created by Guido van Rossum in the late 1980s, is a high-level general-purpose programming language. Its design philosophy emphasizes code readability, and its language constructs aim to help programmers write clear, logical code for both small and large-scale software projects.",
],
]
predicted_answers = [
"Python is a high-level general-purpose programming language that was created by George Lucas.",
]
result = pipeline.run(
{
"context_relevance_evaluator": {"questions": questions, "contexts": contexts},
"faithfulness_evaluator": {
"questions": questions,
"contexts": contexts,
"predicted_answers": predicted_answers,
},
},
)
for evaluator in result:
print(result[evaluator]["individual_scores"])
# [1]
# [0.5]
for evaluator in result:
print(result[evaluator]["score"])
# 1
# 0.5
```
---
// File: pipeline-components/evaluators/deepevalevaluator
# DeepEvalEvaluator
The DeepEvalEvaluator evaluates Haystack pipelines using LLM-based metrics. It supports metrics like answer relevancy, faithfulness, contextual relevance, and more.
`contexts`: A list of a list of contexts, which are the contents of documents. This accounts for one list of contexts per question. | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 that represents the mean context relevance score over all input questions
- `individual_scores`: A list of the individual context relevance scores, each either 0 or 1, for each input pair of a question and a list of contexts
- `results`: A list of dictionaries with keys `relevant_statements`, `score`, and `status`. They contain the statements that an LLM found relevant in each context, the binary score for that context, and `evaluated` for valid results or `error` for failed evaluations. | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/context_relevance.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline has generated the inputs for the Evaluator. |
| **Mandatory init variables** | `metric`: One of the DeepEval metrics to use for evaluation |
| **Mandatory run variables** | `**inputs`: A keyword arguments dictionary containing the expected inputs. The expected inputs will change based on the metric you are evaluating. See below for more details. |
| **Output variables** | `results`: A nested list of metric results. There can be one or more results, depending on the metric. Each result is a dictionary containing:
- `name` - The name of the metric
- `score` - The score of the metric
- `explanation` - An optional explanation of the score | | **API reference** | [DeepEval](/reference/integrations-deepeval) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/deepeval | | **Package name** | `deepeval-haystack` |
DeepEval is an evaluation framework that provides a number of LLM-based evaluation metrics. You can use the `DeepEvalEvaluator` component to evaluate a Haystack pipeline, such as a retrieval-augmented generated pipeline, against one of the metrics provided by DeepEval.
## Supported Metrics
DeepEval supports a number of metrics, which we expose through the [DeepEval metric enumeration.](/reference/integrations-deepeval#deepevalmetric) [`DeepEvalEvaluator`](/reference/integrations-deepeval#deepevalevaluator) in Haystack supports the metrics listed below with the expected `metric_params` while initializing the Evaluator. Many metrics use OpenAI models and require you to set an environment variable `OPENAI_API_KEY`. For a complete guide on these metrics, visit the [DeepEval documentation](https://docs.confident-ai.com/docs/getting-started).
- `name` - The name of the metric
- `score` - The score of the metric
- `explanation` - An optional explanation of the score | | **API reference** | [DeepEval](/reference/integrations-deepeval) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/deepeval | | **Package name** | `deepeval-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline has generated the inputs for the Evaluator. |
| **Mandatory init variables** | `metric`: One of the DeepEval metrics to use for evaluation |
| **Mandatory run variables** | “\*\*inputs”: A keyword arguments dictionary containing the expected inputs. The expected inputs will change based on the metric you are evaluating. See below for more details. |
| **Output variables** | `results`: A nested list of metric results. There can be one or more results, depending on the metric. Each result is a dictionary containing:
- `name` - The name of the metric
- `score` - The score of the metric
- `explanation` - An optional explanation of the score | | **API reference** | [DeepEval](/reference/integrations-deepeval) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/deepeval | | **Package name** | `deepeval-haystack` |
## Parameters Overview
To initialize a `DeepEvalEvaluator`, you need to provide the following parameters :
- `metric`: A `DeepEvalMetric`.
- `metric_params`: Optionally, if the metric calls for any additional parameters, you should provide them here.
## Usage
To use the `DeepEvalEvaluator`, you first need to install the integration:
```bash
pip install deepeval-haystack
```
To use the `DeepEvalEvaluator` you need to follow these steps:
1. Initialize the `DeepEvalEvaluator` while providing the correct `metric_params` for the metric you are using.
2. Run the `DeepEvalEvaluator` on its own or in a pipeline by providing the expected input for the metric you are using.
### Examples
**Evaluate Faithfulness**
To create a faithfulness evaluation pipeline:
```python
from haystack import Pipeline
from haystack_integrations.components.evaluators.deepeval import (
DeepEvalEvaluator,
DeepEvalMetric,
)
pipeline = Pipeline()
evaluator = DeepEvalEvaluator(
metric=DeepEvalMetric.FAITHFULNESS,
metric_params={"model": "gpt-4o-mini"},
)
pipeline.add_component("evaluator", evaluator)
```
To run the evaluation pipeline, you should have the _expected inputs_ for the metric ready at hand. This metric expects a list of `questions`, a list of `contexts`, and a list of `responses`. These should come from the results of the pipeline you want to evaluate.
```python
results = pipeline.run(
{
"evaluator": {
"questions": [
"When was the Rhodes Statue built?",
"Where is the Pyramid of Giza?",
],
"contexts": [["Context for question 1"], ["Context for question 2"]],
"responses": ["Response for question 1", "response for question 2"],
},
},
)
```
## Additional References
🧑🍳 Cookbook: [RAG Pipeline Evaluation Using DeepEval](https://haystack.deepset.ai/cookbook/rag_eval_deep_eval)
---
// File: pipeline-components/evaluators/documentmapevaluator
# DocumentMAPEvaluator
The `DocumentMAPEvaluator` evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks to what extent the list of retrieved documents contains only relevant documents as specified in the ground truth labels or also non-relevant documents. This metric is called mean average precision (MAP).
- `name` - The name of the metric
- `score` - The score of the metric
- `explanation` - An optional explanation of the score | | **API reference** | [DeepEval](/reference/integrations-deepeval) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/deepeval | | **Package name** | `deepeval-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
| **Mandatory run variables** | `ground_truth_documents`: A list of a list of ground truth documents. This accounts for one list of ground truth documents per question.
`retrieved_documents`: A list of a list of retrieved documents. This accounts for one list of retrieved documents per question. | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 that represents the mean average precision
- `individual_scores`: A list of the individual average precision scores ranging from 0.0 to 1.0 for each input pair of a list of retrieved documents and a list of ground truth documents | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/document_map.py | | **Package name** | `haystack-ai` |
## Overview
You can use the `DocumentMAPEvaluator` component to evaluate documents retrieved by a Haystack pipeline, such as a RAG pipeline, against ground truth labels. A higher mean average precision is better, indicating that the list of retrieved documents contains many relevant documents and only a few non-relevant documents or none at all.
To initialize a `DocumentMAPEvaluator`, there are no parameters required.
## Usage
### On its own
Below is an example where we use a `DocumentMAPEvaluator` component to evaluate documents retrieved for two queries. For the first query, there is one ground truth document and one retrieved document. For the second query, there are two ground truth documents and three retrieved documents.
```python
from haystack import Document
from haystack.components.evaluators import DocumentMAPEvaluator
evaluator = DocumentMAPEvaluator()
result = evaluator.run(
ground_truth_documents=[
[Document(content="France")],
[Document(content="9th century"), Document(content="9th")],
],
retrieved_documents=[
[Document(content="France")],
[
Document(content="9th century"),
Document(content="10th century"),
Document(content="9th"),
],
],
)
print(result["individual_scores"])
# [1.0, 0.8333333333333333]
print(result["score"])
# 0.9166666666666666
```
### In a pipeline
Below is an example where we use a `DocumentMAPEvaluator` and a `DocumentMRREvaluator` in a pipeline to evaluate two answers and compare them to ground truth answers. Running a pipeline instead of the individual components simplifies calculating more than one metric.
```python
from haystack import Document, Pipeline
from haystack.components.evaluators import DocumentMRREvaluator, DocumentMAPEvaluator
pipeline = Pipeline()
mrr_evaluator = DocumentMRREvaluator()
map_evaluator = DocumentMAPEvaluator()
pipeline.add_component("mrr_evaluator", mrr_evaluator)
pipeline.add_component("map_evaluator", map_evaluator)
ground_truth_documents = [
[Document(content="France")],
[Document(content="9th century"), Document(content="9th")],
]
retrieved_documents = [
[Document(content="France")],
[
Document(content="9th century"),
Document(content="10th century"),
Document(content="9th"),
],
]
result = pipeline.run(
{
"mrr_evaluator": {
"ground_truth_documents": ground_truth_documents,
"retrieved_documents": retrieved_documents,
},
"map_evaluator": {
"ground_truth_documents": ground_truth_documents,
"retrieved_documents": retrieved_documents,
},
},
)
for evaluator in result:
print(result[evaluator]["individual_scores"])
# [1.0, 0.8333333333333333]
# [1.0, 1.0]
for evaluator in result:
print(result[evaluator]["score"])
# 0.9166666666666666
# 1.0
```
---
// File: pipeline-components/evaluators/documentmrrevaluator
# DocumentMRREvaluator
The `DocumentMRREvaluator` evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks at what rank ground truth documents appear in the list of retrieved documents. This metric is called mean reciprocal rank (MRR).
`retrieved_documents`: A list of a list of retrieved documents. This accounts for one list of retrieved documents per question. | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 that represents the mean average precision
- `individual_scores`: A list of the individual average precision scores ranging from 0.0 to 1.0 for each input pair of a list of retrieved documents and a list of ground truth documents | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/document_map.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
| **Mandatory run variables** | `ground_truth_documents`: A list containing another list of ground truth documents. This accounts for one list of ground truth documents per question.
`retrieved_documents`: A list containing another list of retrieved documents. This accounts for one list of retrieved documents per question. | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 that represents the mean reciprocal rank
- `individual_scores`: A list of the individual reciprocal ranks ranging from 0.0 to 1.0 for each input pair of a list of retrieved documents and a list of ground truth documents | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/document_mrr.py | | **Package name** | `haystack-ai` |
## Overview
You can use the `DocumentMRREvaluator` component to evaluate documents retrieved by a Haystack pipeline, such as a RAG pipeline, against ground truth labels. A higher mean reciprocal rank is better and indicates that relevant documents appear at an earlier position in the list of retrieved documents.
To initialize a `DocumentMRREvaluator`, there are no parameters required.
## Usage
### On its own
Below is an example where we use a `DocumentMRREvaluator` component to evaluate documents retrieved for two queries. For the first query, there is one ground truth document and one retrieved document. For the second query, there are two ground truth documents and three retrieved documents.
```python
from haystack import Document
from haystack.components.evaluators import DocumentMRREvaluator
evaluator = DocumentMRREvaluator()
result = evaluator.run(
ground_truth_documents=[
[Document(content="France")],
[Document(content="9th century"), Document(content="9th")],
],
retrieved_documents=[
[Document(content="France")],
[
Document(content="9th century"),
Document(content="10th century"),
Document(content="9th"),
],
],
)
print(result["individual_scores"])
# [1.0, 1.0]
print(result["score"])
# 1.0
```
### In a pipeline
Below is an example where we use a `DocumentRecallEvaluator` and a `DocumentMRREvaluator` in a pipeline to evaluate two answers and compare them to ground truth answers. Running a pipeline instead of the individual components simplifies calculating more than one metric.
```python
from haystack import Document, Pipeline
from haystack.components.evaluators import DocumentMRREvaluator, DocumentRecallEvaluator
pipeline = Pipeline()
mrr_evaluator = DocumentMRREvaluator()
recall_evaluator = DocumentRecallEvaluator()
pipeline.add_component("mrr_evaluator", mrr_evaluator)
pipeline.add_component("recall_evaluator", recall_evaluator)
ground_truth_documents = [
[Document(content="France")],
[Document(content="9th century"), Document(content="9th")],
]
retrieved_documents = [
[Document(content="France")],
[
Document(content="9th century"),
Document(content="10th century"),
Document(content="9th"),
],
]
result = pipeline.run(
{
"mrr_evaluator": {
"ground_truth_documents": ground_truth_documents,
"retrieved_documents": retrieved_documents,
},
"recall_evaluator": {
"ground_truth_documents": ground_truth_documents,
"retrieved_documents": retrieved_documents,
},
},
)
for evaluator in result:
print(result[evaluator]["individual_scores"])
# [1.0, 1.0]
# [1.0, 1.0]
for evaluator in result:
print(result[evaluator]["score"])
# 1.0
# 1.0
```
---
// File: pipeline-components/evaluators/documentndcgevaluator
# DocumentNDCGEvaluator
The `DocumentNDCGEvaluator` evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks at what rank ground truth documents appear in the list of retrieved documents. This metric is called normalized discounted cumulative gain (NDCG).
`retrieved_documents`: A list containing another list of retrieved documents. This accounts for one list of retrieved documents per question. | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 that represents the mean reciprocal rank
- `individual_scores`: A list of the individual reciprocal ranks ranging from 0.0 to 1.0 for each input pair of a list of retrieved documents and a list of ground truth documents | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/document_mrr.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
| **Mandatory run variables** | `ground_truth_documents`: A list containing another list of ground truth documents, one list per question
`retrieved_documents`: A list containing another list of retrieved documents, one list per question | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 that represents the NDCG
- `individual_scores`: A list of individual NDCG values ranging from 0.0 to 1.0 for each input pair of a list of retrieved documents and a list of ground truth documents | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/document_ndcg.py | | **Package name** | `haystack-ai` |
## Overview
You can use the `DocumentNDCGEvaluator` component to evaluate documents retrieved by a Haystack pipeline, such as a RAG pipeline, against ground truth labels. A higher NDCG is better and indicates that relevant documents appear at an earlier position in the list of retrieved documents.
If the ground truth documents have scores, a higher NDCG indicates that documents with a higher score appear at an earlier position in the list of retrieved documents. If the ground truth documents have no scores, binary relevance is assumed, meaning that all ground truth documents are equally relevant, and the order in which they are in the list of retrieved documents does not matter for the NDCG.
No parameters are required to initialize a `DocumentNDCGEvaluator`.
## Usage
### On its own
Below is an example where we use the `DocumentNDCGEvaluator` to evaluate documents retrieved for a query. There are two ground truth documents and three retrieved documents. All ground truth documents are retrieved, but one non-relevant document is ranked higher than one of the ground truth documents, which lowers the NDCG score.
```python
from haystack import Document
from haystack.components.evaluators import DocumentNDCGEvaluator
evaluator = DocumentNDCGEvaluator()
result = evaluator.run(
ground_truth_documents=[
[Document(content="France", score=1.0), Document(content="Paris", score=0.5)],
],
retrieved_documents=[
[
Document(content="France"),
Document(content="Germany"),
Document(content="Paris"),
],
],
)
print(result["individual_scores"])
# [0.9502344167898356]
print(result["score"])
# 0.9502344167898356
```
### In a pipeline
Below is an example of using a `DocumentNDCGEvaluator` and `DocumentMRREvaluator` in a pipeline to evaluate retrieved documents and compare them to ground truth documents. Running a pipeline instead of the individual components simplifies calculating more than one metric.
```python
from haystack import Document, Pipeline
from haystack.components.evaluators import DocumentMRREvaluator, DocumentNDCGEvaluator
pipeline = Pipeline()
pipeline.add_component("ndcg_evaluator", DocumentNDCGEvaluator())
pipeline.add_component("mrr_evaluator", DocumentMRREvaluator())
ground_truth_documents = [
[Document(content="France", score=1.0), Document(content="Paris", score=0.5)],
]
retrieved_documents = [
[
Document(content="France"),
Document(content="Germany"),
Document(content="Paris"),
],
]
result = pipeline.run(
{
"ndcg_evaluator": {
"ground_truth_documents": ground_truth_documents,
"retrieved_documents": retrieved_documents,
},
"mrr_evaluator": {
"ground_truth_documents": ground_truth_documents,
"retrieved_documents": retrieved_documents,
},
},
)
for evaluator in result:
print(result[evaluator]["score"])
# 1.0
# 0.9502344167898356
```
---
// File: pipeline-components/evaluators/documentrecallevaluator
# DocumentRecallEvaluator
The `DocumentRecallEvaluator` evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks how many of the ground truth documents were retrieved. This metric is called recall.
`retrieved_documents`: A list containing another list of retrieved documents, one list per question | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 that represents the NDCG
- `individual_scores`: A list of individual NDCG values ranging from 0.0 to 1.0 for each input pair of a list of retrieved documents and a list of ground truth documents | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/document_ndcg.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
| **Mandatory run variables** | `ground_truth_documents`: A list of a list of ground truth documents. This accounts for one list of ground truth documents per question.
`retrieved_documents`: A list of a list of retrieved documents. This accounts for one list of retrieved documents per question. | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 that represents the mean recall score over all inputs
- `individual_scores`: A list of the individual recall scores ranging from 0.0 to 1.0 of each input pair of a list of retrieved documents and a list of ground truth documents. If the mode is set to single_hit, each individual score is either 0 or 1. | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/document_recall.py | | **Package name** | `haystack-ai` |
## Overview
You can use the `DocumentRecallEvaluator` component to evaluate documents retrieved by a Haystack pipeline, such as a RAG Pipeline, against ground truth labels.
When initializing a `DocumentRecallEvaluator`, you can set the `mode` parameter to
`RecallMode.SINGLE_HIT` or `RecallMode.MULTI_HIT`. By default, `RecallMode.SINGLE_HIT` is used.
`RecallMode.SINGLE_HIT` means that _any_ of the ground truth documents need to be retrieved to count as a correct retrieval with a recall score of 1. A single retrieved document can achieve the full score.
`RecallMode.MULTI_HIT` means that _all_ of the ground truth documents need to be retrieved to count as a correct retrieval with a recall score of 1. The number of retrieved documents must be at least the number of ground truth documents to achieve the full score.
## Usage
### On its own
Below is an example where we use a `DocumentRecallEvaluator` component to evaluate documents retrieved for two queries. For the first query, there is one ground truth document and one retrieved document. For the second query, there are two ground truth documents and three retrieved documents.
```python
from haystack import Document
from haystack.components.evaluators import DocumentRecallEvaluator
evaluator = DocumentRecallEvaluator()
result = evaluator.run(
ground_truth_documents=[
[Document(content="France")],
[Document(content="9th century"), Document(content="9th")],
],
retrieved_documents=[
[Document(content="France")],
[
Document(content="9th century"),
Document(content="10th century"),
Document(content="9th"),
],
],
)
print(result["individual_scores"])
# [1.0, 1.0]
print(result["score"])
# 1.0
```
### In a pipeline
Below is an example where we use a `DocumentRecallEvaluator` and a `DocumentMRREvaluator` in a pipeline to evaluate two answers and compare them to ground truth answers. Running a pipeline instead of the individual components simplifies calculating more than one metric.
```python
from haystack import Document, Pipeline
from haystack.components.evaluators import DocumentMRREvaluator, DocumentRecallEvaluator
pipeline = Pipeline()
mrr_evaluator = DocumentMRREvaluator()
recall_evaluator = DocumentRecallEvaluator()
pipeline.add_component("mrr_evaluator", mrr_evaluator)
pipeline.add_component("recall_evaluator", recall_evaluator)
ground_truth_documents = [
[Document(content="France")],
[Document(content="9th century"), Document(content="9th")],
]
retrieved_documents = [
[Document(content="France")],
[
Document(content="9th century"),
Document(content="10th century"),
Document(content="9th"),
],
]
result = pipeline.run(
{
"mrr_evaluator": {
"ground_truth_documents": ground_truth_documents,
"retrieved_documents": retrieved_documents,
},
"recall_evaluator": {
"ground_truth_documents": ground_truth_documents,
"retrieved_documents": retrieved_documents,
},
},
)
for evaluator in result:
print(result[evaluator]["individual_scores"])
# [1.0, 1.0]
# [1.0, 1.0]
for evaluator in result:
print(result[evaluator]["score"])
# 1.0
# 1.0
```
---
// File: pipeline-components/evaluators/external-integrations-evaluators
# External Integrations
| Name | Description |
| --- | --- |
| [Flow Judge](https://haystack.deepset.ai/integrations/flow-judge) | Evaluate Haystack pipelines using Flow Judge model. |
---
// File: pipeline-components/evaluators/faithfulnessevaluator
# FaithfulnessEvaluator
The `FaithfulnessEvaluator` uses an LLM to evaluate whether a generated answer can be inferred from the provided contexts. It does not require ground truth labels. This metric is called faithfulness, sometimes also referred to as groundedness or hallucination.
`retrieved_documents`: A list of a list of retrieved documents. This accounts for one list of retrieved documents per question. | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 that represents the mean recall score over all inputs
- `individual_scores`: A list of the individual recall scores ranging from 0.0 to 1.0 of each input pair of a list of retrieved documents and a list of ground truth documents. If the mode is set to single_hit, each individual score is either 0 or 1. | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/document_recall.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
| **Mandatory run variables** | `questions`: A list of questions
`contexts`: A list of a list of contexts, which are the contents of documents. This accounts for one list of contexts per question.
`predicted_answers`: A list of predicted answers, for example, the outputs of a Generator in a RAG pipeline | | **Output variables** | A dictionary containing:
- `score`: A number from 0.0 to 1.0 that represents the average faithfulness score across all questions
- `individual_scores`: A list of the individual faithfulness scores ranging from 0.0 to 1.0 for each input triple of a question, a list of contexts, and a predicted answer.
- `results`: A list of dictionaries with `statements`, `statement_scores`, `score`, and `status` keys. They contain the statements extracted by an LLM from each predicted answer, the corresponding faithfulness scores per statement (either 0 or 1), the mean score for that answer, and `evaluated` for valid results or `error` for failed evaluations. | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/faithfulness.py | | **Package name** | `haystack-ai` |
You can use the `FaithfulnessEvaluator` component to evaluate documents retrieved by a Haystack pipeline, such as a RAG pipeline, without ground truth labels. The component splits the generated answer into statements and checks each of them against the provided contexts with an LLM. A higher faithfulness score is better, and it indicates that a larger number of statements in the generated answers can be inferred from the contexts. The faithfulness score can be used to better understand how often and when the Generator in a RAG pipeline hallucinates.
### Parameters
The default model for this Evaluator is `gpt-5-mini`. You can override the model using the `chat_generator` parameter during initialization. This needs to be a Chat Generator instance configured to return a JSON object. For example, when using the [`OpenAIChatGenerator`](../generators/openaichatgenerator.mdx), you should pass `{"response_format": {"type": "json_object"}}` in its `generation_kwargs`.
If you are not initializing the Evaluator with your own Chat Generator other than OpenAI, a valid OpenAI API key must be set as an `OPENAI_API_KEY` environment variable. For details, see our [documentation page on secret management](../../concepts/secret-management.mdx).
Two other optional initialization parameters are:
- `raise_on_failure`: If True, raise an exception on an unsuccessful API call.
- `progress_bar`: Whether to show a progress bar during the evaluation.
`FaithfulnessEvaluator` has an optional `examples` parameter that can be used to pass few-shot examples conforming to the expected input and output format of `FaithfulnessEvaluator`. These examples are included in the prompt that is sent to the LLM. Examples, therefore, increase the number of tokens of the prompt and make each request more costly. Adding examples is helpful if you want to improve the quality of the evaluation at the cost of more tokens.
Each example must be a dictionary with keys `inputs` and `outputs`.
`inputs` must be a dictionary with keys `questions`, `contexts`, and `predicted_answers`.
`outputs` must be a dictionary with `statements` and `statement_scores`.
Here is the expected format:
```python
[
{
"inputs": {
"questions": "What is the capital of Italy?",
"contexts": ["Rome is the capital of Italy."],
"predicted_answers": "Rome is the capital of Italy with more than 4 million inhabitants.",
},
"outputs": {
"statements": [
"Rome is the capital of Italy.",
"Rome has more than 4 million inhabitants.",
],
"statement_scores": [1, 0],
},
},
]
```
## Usage
### On its own
Below is an example of using a `FaithfulnessEvaluator` component to evaluate a predicted answer generated based on a provided question and context. The `FaithfulnessEvaluator` returns a score of 0.5 because it detects two statements in the answer, of which only one is correct.
```python
from haystack.components.evaluators import FaithfulnessEvaluator
questions = ["Who created the Python language?"]
contexts = [
[
"Python, created by Guido van Rossum in the late 1980s, is a high-level general-purpose programming language. Its design philosophy emphasizes code readability, and its language constructs aim to help programmers write clear, logical code for both small and large-scale software projects.",
],
]
predicted_answers = [
"Python is a high-level general-purpose programming language that was created by George Lucas.",
]
evaluator = FaithfulnessEvaluator()
result = evaluator.run(
questions=questions,
contexts=contexts,
predicted_answers=predicted_answers,
)
print(result["individual_scores"])
# [0.5]
print(result["score"])
# 0.5
print(result["results"])
# [{'statements': ['Python is a high-level general-purpose programming language.',
# 'Python was created by George Lucas.'], 'statement_scores': [1, 0], 'status': 'evaluated', 'score': 0.5}]
```
### In a pipeline
Below is an example where we use a `FaithfulnessEvaluator` and a `ContextRelevanceEvaluator` in a pipeline to evaluate predicted answers and contexts (the content of documents) received by a RAG pipeline based on provided questions. Running a pipeline instead of the individual components simplifies calculating more than one metric.
```python
from haystack import Pipeline
from haystack.components.evaluators import (
ContextRelevanceEvaluator,
FaithfulnessEvaluator,
)
pipeline = Pipeline()
context_relevance_evaluator = ContextRelevanceEvaluator()
faithfulness_evaluator = FaithfulnessEvaluator()
pipeline.add_component("context_relevance_evaluator", context_relevance_evaluator)
pipeline.add_component("faithfulness_evaluator", faithfulness_evaluator)
questions = ["Who created the Python language?"]
contexts = [
[
"Python, created by Guido van Rossum in the late 1980s, is a high-level general-purpose programming language. Its design philosophy emphasizes code readability, and its language constructs aim to help programmers write clear, logical code for both small and large-scale software projects.",
],
]
predicted_answers = [
"Python is a high-level general-purpose programming language that was created by George Lucas.",
]
result = pipeline.run(
{
"context_relevance_evaluator": {"questions": questions, "contexts": contexts},
"faithfulness_evaluator": {
"questions": questions,
"contexts": contexts,
"predicted_answers": predicted_answers,
},
},
)
for evaluator in result:
print(result[evaluator]["individual_scores"])
# ...
# [0.5]
for evaluator in result:
print(result[evaluator]["score"])
#
# 0.5
```
---
// File: pipeline-components/evaluators/llmevaluator
# LLMEvaluator
This Evaluator uses an LLM to evaluate inputs based on a prompt containing user-defined instructions and examples.
`contexts`: A list of a list of contexts, which are the contents of documents. This accounts for one list of contexts per question.
`predicted_answers`: A list of predicted answers, for example, the outputs of a Generator in a RAG pipeline | | **Output variables** | A dictionary containing:
- `score`: A number from 0.0 to 1.0 that represents the average faithfulness score across all questions
- `individual_scores`: A list of the individual faithfulness scores ranging from 0.0 to 1.0 for each input triple of a question, a list of contexts, and a predicted answer.
- `results`: A list of dictionaries with `statements`, `statement_scores`, `score`, and `status` keys. They contain the statements extracted by an LLM from each predicted answer, the corresponding faithfulness scores per statement (either 0 or 1), the mean score for that answer, and `evaluated` for valid results or `error` for failed evaluations. | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/faithfulness.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
| **Mandatory init variables** | `instructions`: The prompt instructions string
`inputs`: The expected inputs
`outputs`: The output names of the evaluation results
`examples`: Few-shot examples conforming to the input and output format | | **Mandatory run variables** | `inputs`: Defined by the user – for example, questions or responses | | **Output variables** | A dictionary containing:
- `results`: A list of dictionaries whose keys are the ones you declared in the `outputs` parameter, such as `score`
- `meta`: The metadata returned by the Chat Generator for each evaluated input, or `None` if there is none | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/llm_evaluator.py | | **Package name** | `haystack-ai` |
## Overview
The `LLMEvaluator` component can evaluate answers, documents, or any other outputs of a Haystack pipeline based on a user-defined aspect. The component combines the instructions, examples, and expected output names into one prompt. It is meant for calculating user-defined model-based evaluation metrics. If you are looking for pre-defined model-based evaluators that work out of the box, have a look at Haystack’s [`FaithfulnessEvaluator`](faithfulnessevaluator.mdx) and [`ContextRelevanceEvaluator`](contextrelevanceevaluator.mdx) components instead.
### Parameters
The default model for this Evaluator is `gpt-5-mini`. You can override the model using the `chat_generator` parameter during initialization. This needs to be a Chat Generator instance configured to return a JSON object. For example, when using the [`OpenAIChatGenerator`](../generators/openaichatgenerator.mdx), you should pass `{"response_format": {"type": "json_object"}}` in its `generation_kwargs`.
If you are not initializing the Evaluator with your own Chat Generator other than OpenAI, a valid OpenAI API key must be set as an `OPENAI_API_KEY` environment variable. For details, see our [documentation page on secret management](../../concepts/secret-management.mdx).
`LLMEvaluator` takes the following parameters for initialization. The first four are required, the last two have defaults:
- `instructions`: The prompt instructions to use for evaluation, such as a question about the inputs that the LLM can answer with _yes,_ _no_, or a score.
- `inputs`: The inputs that the `LLMEvaluator` expects and that it evaluates. The inputs determine the incoming connections of the component. Each input is a tuple of an input name and input type. Input types must be lists. An example could be `[("responses", List[str])]`.
- `outputs`: Output names of the evaluation results corresponding to keys in the output dictionary. An example could be a `["score"]`.
- `examples`: Use this parameter to pass few-shot examples conforming to the expected input and output format. These examples are included in the prompt that is sent to the LLM. Examples increase the number of tokens of the prompt and make each request more costly. Adding more than one or two examples can be helpful if you want to improve the quality of the evaluation at the cost of more tokens.
- `raise_on_failure`: If True (default), raise an exception on an unsuccessful API call.
- `progress_bar`: Whether to show a progress bar during the evaluation. True is the default.
Each example must be a dictionary with keys `inputs` and `outputs`.
`inputs` must be a dictionary whose keys are the input names you declared in the `inputs` parameter.
`outputs` must be a dictionary whose keys are the names you declared in the `outputs` parameter.
Here is the expected format for an Evaluator declared with `inputs=[("responses", list[str])]` and `outputs=["score"]`:
```python
[
{
"inputs": {"responses": "Damn, this is straight outta hell!!!"},
"outputs": {"score": 1},
},
{
"inputs": {"responses": "Football is the most popular sport."},
"outputs": {"score": 0},
},
]
```
## Usage
### On its own
Below is an example where we use an `LLMEvaluator` component to evaluate a generated response. The aspect we evaluate is whether the response is problematic for children as defined in the instructions. The `LLMEvaluator` returns one binary score per input response with the result that both responses are not problematic.
```python
from typing import List
from haystack.components.evaluators import LLMEvaluator
llm_evaluator = LLMEvaluator(
instructions="Is this answer problematic for children?",
inputs=[("responses", List[str])],
outputs=["score"],
examples=[
{
"inputs": {"responses": "Damn, this is straight outta hell!!!"},
"outputs": {"score": 1},
},
{
"inputs": {"responses": "Football is the most popular sport."},
"outputs": {"score": 0},
},
],
)
responses = [
"Football is the most popular sport with around 4 billion followers worldwide",
"Python language was created by Guido van Rossum.",
]
results = llm_evaluator.run(responses=responses)
print(results)
# {'results': [{'score': 0}, {'score': 0}],
# 'meta': [{'model': 'gpt-5-mini-2025-08-07', 'index': 0, 'finish_reason': 'stop', 'usage': {...}},
# {'model': 'gpt-5-mini-2025-08-07', 'index': 0, 'finish_reason': 'stop', 'usage': {...}}]}
```
### In a pipeline
Below is an example where we use an `LLMEvaluator` in a pipeline to evaluate a response.
```python
from typing import List
from haystack import Pipeline
from haystack.components.evaluators import LLMEvaluator
pipeline = Pipeline()
llm_evaluator = LLMEvaluator(
instructions="Is this answer problematic for children?",
inputs=[("responses", List[str])],
outputs=["score"],
examples=[
{
"inputs": {"responses": "Damn, this is straight outta hell!!!"},
"outputs": {"score": 1},
},
{
"inputs": {"responses": "Football is the most popular sport."},
"outputs": {"score": 0},
},
],
)
pipeline.add_component("llm_evaluator", llm_evaluator)
responses = [
"Football is the most popular sport with around 4 billion followers worldwide",
"Python language was created by Guido van Rossum.",
]
result = pipeline.run({"llm_evaluator": {"responses": responses}})
for evaluator in result:
print(result[evaluator]["results"])
# [{'score': 0}, {'score': 0}]
```
---
// File: pipeline-components/evaluators/ragasevaluator
# RagasEvaluator
This component evaluates Haystack pipelines using LLM-based metrics. It supports metrics like context relevance, factual accuracy, response relevance, and more.
`inputs`: The expected inputs
`outputs`: The output names of the evaluation results
`examples`: Few-shot examples conforming to the input and output format | | **Mandatory run variables** | `inputs`: Defined by the user – for example, questions or responses | | **Output variables** | A dictionary containing:
- `results`: A list of dictionaries whose keys are the ones you declared in the `outputs` parameter, such as `score`
- `meta`: The metadata returned by the Chat Generator for each evaluated input, or `None` if there is none | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/llm_evaluator.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline has generated the inputs for the Evaluator. |
| **Mandatory init variables** | `ragas_metrics`: A list of modern Ragas metrics from `ragas.metrics.collections`. Each metric must be fully configured (including its LLM) at construction time. |
| **Mandatory run variables** | The expected inputs will change based on the metrics you are evaluating, but can include `query`, `response`, `documents`, `reference_contexts`, `multi_responses`, `reference`, and `rubrics`. |
| **Output variables** | `result`: A dictionary mapping metric names to their `MetricResult`. |
| **API reference** | [Ragas](/reference/integrations-ragas) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ragas |
| **Package name** | `ragas-haystack` |
Ragas is an evaluation framework that provides a number of LLM-based evaluation metrics. You can use the `RagasEvaluator` component to evaluate a Haystack pipeline, such as a retrieval-augmented generative pipeline, against one of the metrics provided by Ragas.
## Supported Metrics
The `RagasEvaluator` supports the modern Ragas metrics API. You can pass any metric from `ragas.metrics.collections` (such as `Faithfulness`, `AnswerRelevancy`, `ContextPrecision`, etc.) as long as it is a `SimpleBaseMetric` instance. Each metric must be fully configured (including its LLM and embeddings) at construction time.
For a complete guide on these metrics, visit the [Ragas documentation](https://docs.ragas.io/).
## Parameters Overview
To initialize a `RagasEvaluator`, you need to provide the following parameters:
- `ragas_metrics`: A list of modern Ragas metrics from `ragas.metrics.collections`. Each metric must be fully configured (including its LLM) at construction time.
## Usage
To use the `RagasEvaluator`, you first need to install the integration:
```bash
pip install ragas-haystack
```
To use the `RagasEvaluator` you need to follow these steps:
1. Initialize the `RagasEvaluator` while providing the fully configured metrics you want to use.
2. Run the `RagasEvaluator`, either on its own or in a pipeline, by providing the expected inputs for the metrics you are using (e.g. `query`, `documents`, `response`, etc.).
### Examples
#### Evaluate Answer Relevancy
To create an answer relevancy evaluation pipeline (note that the `OPENAI_API_KEY` environment variable must be set for this example to work):
```python
from haystack import Pipeline
from haystack_integrations.components.evaluators.ragas import RagasEvaluator
from openai import AsyncOpenAI
from ragas.llms import llm_factory
from ragas.embeddings import embedding_factory
from ragas.metrics.collections import AnswerRelevancy
client = AsyncOpenAI()
llm = llm_factory("gpt-4o-mini", client=client)
embeddings = embedding_factory("openai", model="text-embedding-3-small", client=client)
pipeline = Pipeline()
evaluator = RagasEvaluator(
ragas_metrics=[AnswerRelevancy(llm=llm, embeddings=embeddings)],
)
pipeline.add_component("evaluator", evaluator)
```
To run the evaluation pipeline, you should have the _expected inputs_ for the metric ready at hand. This metric expects a `query` and `response`, which should come from the results of the pipeline you want to evaluate.
```python
results = pipeline.run(
{
"evaluator": {
"query": "Where is the Pyramid of Giza?",
"response": "The Pyramid of Giza is located in Egypt.",
},
},
)
```
#### Evaluate Context Precision and Faithfulness
To create a pipeline that evaluates multiple metrics at once:
```python
from haystack import Pipeline
from haystack_integrations.components.evaluators.ragas import RagasEvaluator
from openai import AsyncOpenAI
from ragas.llms import llm_factory
from ragas.metrics.collections import ContextPrecision, Faithfulness
client = AsyncOpenAI()
llm = llm_factory("gpt-4o-mini", client=client)
pipeline = Pipeline()
evaluator = RagasEvaluator(
ragas_metrics=[ContextPrecision(llm=llm), Faithfulness(llm=llm)],
)
pipeline.add_component("evaluator", evaluator)
```
To run the evaluation pipeline, you should provide the combined inputs required by all metrics.
```python
results = pipeline.run(
{
"evaluator": {
"query": "Which is the most popular global sport?",
"documents": [
"The popularity of sports can be measured in various ways, including TV viewership, social media presence, number of participants, and economic impact. Football is undoubtedly the world's most popular sport with major events like the FIFA World Cup and sports personalities like Ronaldo and Messi, drawing a followership of more than 4 billion people."
],
"response": "Football is the most popular sport with around 4 billion followers worldwide",
"reference": "Football is the most popular sport",
},
},
)
```
## Additional References
🧑🍳 Cookbook: [Evaluate a RAG pipeline using Ragas integration](https://haystack.deepset.ai/cookbook/rag_eval_ragas)
---
// File: pipeline-components/evaluators/sasevaluator
# SASEvaluator
The `SASEvaluator` evaluates answers predicted by Haystack pipelines using ground truth labels. It checks the semantic similarity of a predicted answer and the ground truth answer using a fine-tuned language model. This metric is called semantic answer similarity.
| | |
| --- | --- |
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `ground_truth_answers`: A list of strings containing the ground truth answers
`predicted_answers`: A list of strings containing the predicted answers to be evaluated | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 representing the mean SAS score for all pairs of predicted answers and ground truth answers
- `individual_scores`: A list of the SAS scores ranging from 0.0 to 1.0 of all pairs of predicted answers and ground truth answers | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/sas_evaluator.py | | **Package name** | `haystack-ai` |
## Overview
You can use the `SASEvaluator` component to evaluate answers predicted by a Haystack pipeline, such as a RAG pipeline, against ground truth labels.
You can provide a bi-encoder or cross-encoder model to initialize a `SASEvaluator`. By default, `sentence-transformers/paraphrase-multilingual-mpnet-base-v2` model is used.
Note that only _one_ predicted answer is compared to _one_ ground truth answer at a time. The component does not support multiple ground truth answers for the same question or multiple answers predicted for the same question.
## Usage
### On its own
Below is an example of using a `SASEvaluator` component to evaluate two answers and compare them to ground truth answers.
```python
from haystack.components.evaluators import SASEvaluator
sas_evaluator = SASEvaluator()
result = sas_evaluator.run(
ground_truth_answers=["Berlin", "Paris"],
predicted_answers=["Berlin", "Lyon"],
)
print(result["individual_scores"])
# [1.0, 0.5174766182899475]
print(result["score"])
# 0.7587383091449738
```
### In a pipeline
Below is an example where we use an `AnswerExactMatchEvaluator` and a `SASEvaluator` in a pipeline to evaluate two answers and compare them to ground truth answers. Running a pipeline instead of the individual components simplifies calculating more than one metric.
```python
from haystack import Pipeline
from haystack.components.evaluators import AnswerExactMatchEvaluator, SASEvaluator
pipeline = Pipeline()
em_evaluator = AnswerExactMatchEvaluator()
sas_evaluator = SASEvaluator()
pipeline.add_component("em_evaluator", em_evaluator)
pipeline.add_component("sas_evaluator", sas_evaluator)
ground_truth_answers = ["Berlin", "Paris"]
predicted_answers = ["Berlin", "Lyon"]
result = pipeline.run(
{
"em_evaluator": {
"ground_truth_answers": ground_truth_answers,
"predicted_answers": predicted_answers,
},
"sas_evaluator": {
"ground_truth_answers": ground_truth_answers,
"predicted_answers": predicted_answers,
},
},
)
for evaluator in result:
print(result[evaluator]["individual_scores"])
# [1, 0]
# [1.0, 0.5174766182899475]
for evaluator in result:
print(result[evaluator]["score"])
# 0.5
# 0.7587383091449738
```
## Additional References
🧑🍳 Cookbook: [Prompt Optimization with DSPy](https://haystack.deepset.ai/cookbook/prompt_optimization_with_dspy)
---
// File: pipeline-components/evaluators
# Evaluators
| Evaluator | Description |
| --- | --- |
| [AnswerExactMatchEvaluator](evaluators/answerexactmatchevaluator.mdx) | Evaluates answers predicted by Haystack pipelines using ground truth labels. It checks character by character whether a predicted answer exactly matches the ground truth answer. |
| [ContextRelevanceEvaluator](evaluators/contextrelevanceevaluator.mdx) | Uses an LLM to evaluate whether a generated answer can be inferred from the provided contexts. |
| [DeepEvalEvaluator](evaluators/deepevalevaluator.mdx) | Use DeepEval to evaluate generative pipelines. |
| [DocumentMAPEvaluator](evaluators/documentmapevaluator.mdx) | Evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks to what extent the list of retrieved documents contains only relevant documents as specified in the ground truth labels or also non-relevant documents. |
| [DocumentMRREvaluator](evaluators/documentmrrevaluator.mdx) | Evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks at what rank ground truth documents appear in the list of retrieved documents. |
| [DocumentNDCGEvaluator](evaluators/documentndcgevaluator.mdx) | Evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks at what rank ground truth documents appear in the list of retrieved documents. This metric is called normalized discounted cumulative gain (NDCG). |
| [DocumentRecallEvaluator](evaluators/documentrecallevaluator.mdx) | Evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks how many of the ground truth documents were retrieved. |
| [FaithfulnessEvaluator](evaluators/faithfulnessevaluator.mdx) | Uses an LLM to evaluate whether a generated answer can be inferred from the provided contexts. Does not require ground truth labels. |
| [LLMEvaluator](evaluators/llmevaluator.mdx) | Uses an LLM to evaluate inputs based on a prompt containing user-defined instructions and examples. |
| [RagasEvaluator](evaluators/ragasevaluator.mdx) | Use Ragas framework to evaluate a retrieval-augmented generative pipeline. |
| [SASEvaluator](evaluators/sasevaluator.mdx) | Evaluates answers predicted by Haystack pipelines using ground truth labels. It checks the semantic similarity of a predicted answer and the ground truth answer using a fine-tuned language model. |
---
// File: pipeline-components/extractors/llmdocumentcontentextractor
# LLMDocumentContentExtractor
Extracts textual content and metadata (if applicable) from image-based documents using a vision-enabled Large Language Model (LLM).
`predicted_answers`: A list of strings containing the predicted answers to be evaluated | | **Output variables** | A dictionary containing:
\- `score`: A number from 0.0 to 1.0 representing the mean SAS score for all pairs of predicted answers and ground truth answers
- `individual_scores`: A list of the SAS scores ranging from 0.0 to 1.0 of all pairs of predicted answers and ground truth answers | | **API reference** | [Evaluators](/reference/evaluators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/sas_evaluator.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After [Converters](../converters.mdx) in an indexing pipeline to extract text from image-based documents |
| **Mandatory init variables** | `chat_generator`: A ChatGenerator instance that supports vision-based input |
| **Mandatory run variables** | `documents`: A list of documents with file paths in metadata |
| **Output variables** | `documents`: Successfully processed documents with extracted content
`failed_documents`: Documents that failed processing with error metadata | | **API reference** | [Extractors](/reference/extractors-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/extractors/image/llm_document_content_extractor.py | | **Package name** | `haystack-ai` |
## Overview
`LLMDocumentContentExtractor` extracts textual content from image-based documents using a vision-enabled Large Language Model (LLM). This component is particularly useful for processing scanned documents, images containing text, or PDF pages that need to be converted to searchable text.
The component works by:
1. Converting each input document into an image using the `DocumentToImageContent` component.
2. Using a predefined prompt to instruct the LLM on how to extract content and/or metadata.
3. Processing the image through a vision-capable ChatGenerator to extract structured textual content.
The prompt must not contain Jinja variables; it should only include instructions for the LLM. Image data and the prompt are passed together to the LLM as a Chat Message.
The extractor supports both plain-text and JSON responses from the LLM:
- If the LLM returns a plain string, that text is written to the document's `content`.
- If the LLM returns a JSON object with only the `document_content` key, that value is written to `content`.
- If the LLM returns a JSON object with multiple keys, the value of `document_content` (if present) is written to `content`, and all other keys are merged into the document's metadata.
Documents for which the LLM fails to extract content are returned in a separate `failed_documents` list with an `extraction_error` entry in their metadata for debugging or reprocessing.
## Usage
### On its own
Below is an example that uses the `LLMDocumentContentExtractor` to extract text from image-based documents:
```python
from haystack import Document
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.extractors.image import LLMDocumentContentExtractor
# Initialize the chat generator with vision capabilities
chat_generator = OpenAIChatGenerator(
model="gpt-4o-mini",
generation_kwargs={"temperature": 0.0},
)
# Create the extractor
extractor = LLMDocumentContentExtractor(
chat_generator=chat_generator,
file_path_meta_field="file_path",
raise_on_failure=False,
)
# Create documents with image file paths
documents = [
Document(content="", meta={"file_path": "image.jpg"}),
Document(content="", meta={"file_path": "document.pdf", "page_number": 1}),
]
# Run the extractor
result = extractor.run(documents=documents)
# Check results
print(f"Successfully processed: {len(result['documents'])}")
print(f"Failed documents: {len(result['failed_documents'])}")
# Access extracted content
for doc in result["documents"]:
print(f"File: {doc.meta['file_path']}")
print(f"Extracted content: {doc.content[:100]}...")
```
### Using custom prompts
You can provide a custom prompt to instruct the LLM on how to extract content:
```python
from haystack.components.extractors.image import LLMDocumentContentExtractor
from haystack.components.generators.chat import OpenAIChatGenerator
custom_prompt = """
Extract all text content from this image-based document.
Instructions:
- Extract text exactly as it appears
- Preserve the reading order
- Format tables as markdown
- Describe any images or diagrams briefly
- Maintain document structure
Document:"""
chat_generator = OpenAIChatGenerator(model="gpt-4o-mini")
extractor = LLMDocumentContentExtractor(
chat_generator=chat_generator,
prompt=custom_prompt,
file_path_meta_field="file_path",
)
documents = [Document(content="", meta={"file_path": "scanned_document.pdf"})]
result = extractor.run(documents=documents)
```
### Handling failed documents
The component provides detailed error information for failed documents:
```python
from haystack.components.extractors.image import LLMDocumentContentExtractor
from haystack.components.generators.chat import OpenAIChatGenerator
chat_generator = OpenAIChatGenerator(model="gpt-4o-mini")
extractor = LLMDocumentContentExtractor(
chat_generator=chat_generator,
raise_on_failure=False, # Don't raise exceptions, return failed documents
)
documents = [Document(content="", meta={"file_path": "problematic_image.jpg"})]
result = extractor.run(documents=documents)
# Check for failed documents
for failed_doc in result["failed_documents"]:
print(f"Failed to process: {failed_doc.meta['file_path']}")
print(f"Error: {failed_doc.meta['extraction_error']}")
```
### In a pipeline
Below is an example of a pipeline that uses `LLMDocumentContentExtractor` to process image-based documents and store the extracted text:
```python
from haystack import Pipeline
from haystack.components.extractors.image import LLMDocumentContentExtractor
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.dataclasses import Document
# Create document store
document_store = InMemoryDocumentStore()
# Create pipeline
p = Pipeline()
p.add_component(
instance=LLMDocumentContentExtractor(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
file_path_meta_field="file_path",
),
name="content_extractor",
)
p.add_component(instance=DocumentSplitter(), name="splitter")
p.add_component(instance=DocumentWriter(document_store=document_store), name="writer")
# Connect components
p.connect("content_extractor.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
# Create test documents
docs = [
Document(content="", meta={"file_path": "scanned_document.pdf"}),
Document(content="", meta={"file_path": "image_with_text.jpg"}),
]
# Run pipeline
result = p.run({"content_extractor": {"documents": docs}})
# Check results
print(f"Successfully processed: {len(result['content_extractor']['documents'])}")
print(f"Failed documents: {len(result['content_extractor']['failed_documents'])}")
# Access documents in the store
stored_docs = document_store.filter_documents()
print(f"Documents in store: {len(stored_docs)}")
```
---
// File: pipeline-components/extractors/llmmetadataextractor
# LLMMetadataExtractor
Extracts metadata from documents using a Large Language Model. The metadata is extracted by providing a prompt to a LLM that generates it.
`failed_documents`: Documents that failed processing with error metadata | | **API reference** | [Extractors](/reference/extractors-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/extractors/image/llm_document_content_extractor.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After [PreProcessors](../preprocessors.mdx) in an indexing pipeline |
| **Mandatory init variables** | `prompt`: The prompt to instruct the LLM on how to extract metadata from the document. It must contain exactly one variable, called `document`.
`chat_generator`: A Chat Generator instance which represents the LLM configured to return a JSON object | | **Mandatory run variables** | `documents`: A list of documents | | **Output variables** | `documents`: A list of documents | | **API reference** | [Extractors](/reference/extractors-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/extractors/llm_metadata_extractor.py | | **Package name** | `haystack-ai` |
## Overview
The `LLMMetadataExtractor` extraction relies on an LLM and a prompt to perform the metadata extraction. At initialization time, it expects an LLM, a Haystack Generator, and a prompt describing the metadata extraction process.
The prompt must have exactly one variable, called `document`, that points to a single document in the list of documents. So, to access the content of the document, you can use `{{ document.content }}` in the prompt. The component raises a `ValueError` at initialization if the prompt has no variables, more than one variable, or a variable with a different name.
At runtime, it expects a list of documents and will run the LLM on each document in the list, extracting metadata from the document. The metadata will be added to the document's metadata field.
If the LLM fails to extract metadata from a document, it will be added to the `failed_documents` list. The failed documents' metadata will contain the keys `metadata_extraction_error` and `metadata_extraction_response`.
These documents can be re-run with another extractor to extract metadata using the `metadata_extraction_response` and `metadata_extraction_error` in the prompt.
`chat_generator` accepts any Haystack Chat Generator configured to return a JSON object, for example:
- [OpenAIChatGenerator](../generators/openaichatgenerator.mdx)
- [AzureOpenAIChatGenerator](../generators/azureopenaichatgenerator.mdx)
- [AmazonBedrockChatGenerator](../generators/amazonbedrockchatgenerator.mdx)
- [VertexAIGeminiChatGenerator](../generators/vertexaigeminichatgenerator.mdx)
## Usage
Here's an example of using the `LLMMetadataExtractor` to extract named entities and add them to the document's metadata.
First, the mandatory imports:
```python
from haystack import Document
from haystack.components.extractors.llm_metadata_extractor import LLMMetadataExtractor
from haystack.components.generators.chat import OpenAIChatGenerator
```
Then, define some documents:
```python
docs = [
Document(
content="deepset was founded in 2018 in Berlin, and is known for its Haystack framework",
),
Document(
content="Hugging Face is a company founded in New York, USA and is known for its Transformers library",
),
]
```
And now, a prompt that extracts named entities from the documents:
```python
NER_PROMPT = """
-Goal-
Given text and a list of entity types, identify all entities of those types from the text.
-Steps-
1. Identify all entities. For each identified entity, extract the following information:
- entity_name: Name of the entity, capitalized
- entity_type: One of the following types: [organization, product, service, industry]
Format each entity as a JSON like: {"entity": `chat_generator`: A Chat Generator instance which represents the LLM configured to return a JSON object | | **Mandatory run variables** | `documents`: A list of documents | | **Output variables** | `documents`: A list of documents | | **API reference** | [Extractors](/reference/extractors-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/extractors/llm_metadata_extractor.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In an indexing pipeline, before writing Documents to a Document Store |
| **Mandatory run variables** | `documents`: A list of Document objects |
| **Output variables** | `documents`: A list of Document objects with PII metadata added |
| **API reference** | [Presidio](/reference/integrations-presidio) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/presidio |
| **Package name** | `presidio-haystack` |
## Overview
[Microsoft Presidio](https://data-privacy-stack.github.io/presidio/) is an open-source framework for PII detection and anonymization. `PresidioEntityExtractor` uses Presidio's Analyzer Engine to scan document text and identify entities such as names, email addresses, phone numbers, and more.
The extractor does **not** modify the document text. Instead, it adds the detected entities as structured metadata, letting you inspect or act on PII findings without altering the original content. This is useful when you want to audit what PII is present before deciding how to handle it — for example, routing documents to a review queue, logging PII findings, or conditionally applying anonymization.
If you want to replace PII directly rather than annotate it, see [`PresidioDocumentCleaner`](../preprocessors/presidiodocumentcleaner.mdx) for Documents or [`PresidioTextCleaner`](../preprocessors/presidiotextcleaner.mdx) for plain strings.
## Configuration
| Parameter | Default | Description |
| --- | --- | --- |
| `language` | `"en"` | ISO 639-1 language code for PII detection. The appropriate spaCy model is selected automatically for [supported languages](#non-english-languages). See [Presidio supported languages](https://data-privacy-stack.github.io/presidio/analyzer/languages/). |
| `entities` | `None` | List of PII entity types to detect (e.g. `["PERSON", "EMAIL_ADDRESS"]`). If `None`, all supported types are detected. See [supported entities](https://data-privacy-stack.github.io/presidio/supported_entities/). |
| `score_threshold` | `0.35` | Minimum confidence score (0–1) for a detected entity to be included. |
| `models` | `None` | Advanced override: explicit list of spaCy model configs, e.g. `[{"lang_code": "fr", "model_name": "fr_core_news_md"}]`. Use this only when you need a specific model variant or a language not in the built-in mapping. If `None`, the model is selected automatically based on `language`. |
## Usage
Install the `presidio-haystack` package to use the `PresidioEntityExtractor`.
```bash
pip install presidio-haystack
```
### On its own
```python
from haystack import Document
from haystack_integrations.components.extractors.presidio import PresidioEntityExtractor
extractor = PresidioEntityExtractor()
result = extractor.run(
documents=[Document(content="Contact Alice at alice@example.com")],
)
print(result["documents"][0].meta["entities"])
# [{"entity_type": "PERSON", "start": 8, "end": 13, "score": 0.85},
# {"entity_type": "EMAIL_ADDRESS", "start": 17, "end": 34, "score": 1.0}]
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.extractors.presidio import PresidioEntityExtractor
document_store = InMemoryDocumentStore()
indexing_pipeline = Pipeline()
indexing_pipeline.add_component("extractor", PresidioEntityExtractor())
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
indexing_pipeline.connect("extractor", "writer")
indexing_pipeline.run(
{
"extractor": {
"documents": [
Document(content="Alice Smith's email is alice@example.com"),
Document(content="Call Bob at 212-555-9876"),
],
},
},
)
# Documents are stored with detected PII in doc.meta["entities"]
```
### Using Custom Parameters
Use `entities` to limit detection to the PII types you actually care about. This reduces false positives and improves performance by skipping recognizers you don't need.
Use `score_threshold` to tune the precision-recall tradeoff. The default `0.35` casts a wide net and may include some false positives. Raise it (e.g. `0.7`) when you need high confidence in each detected entity; lower it when missing any PII is the bigger risk.
```python
from haystack_integrations.components.extractors.presidio import PresidioEntityExtractor
extractor = PresidioEntityExtractor(
language="de",
entities=["PERSON", "EMAIL_ADDRESS"], # only detect names and emails
score_threshold=0.7, # higher precision, fewer false positives
)
```
### Non-English languages
For any language in the built-in mapping, just set `language` — the right spaCy model is selected and loaded automatically at warm-up time.
```python
from haystack import Document
from haystack_integrations.components.extractors.presidio import PresidioEntityExtractor
# No `models` parameter needed — de_core_news_lg is selected automatically
extractor = PresidioEntityExtractor(language="de")
result = extractor.run(
documents=[Document(content="Kontaktieren Sie Hans Müller unter hans@example.com")],
)
```
Supported languages and their default models are listed in `PresidioEntityExtractor.SPACY_DEFAULT_MODELS`. Using a language not in that mapping without providing `models` raises a `ValueError` at warm-up time with a list of the supported language codes.
To use a non-default model variant, or a language outside the built-in mapping, pass `models` explicitly:
```python
extractor = PresidioEntityExtractor(
language="fr",
models=[{"lang_code": "fr", "model_name": "fr_core_news_md"}],
)
```
---
// File: pipeline-components/extractors/regextextextractor
# RegexTextExtractor
Extracts text from chat messages or strings using a regular expression pattern.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [Chat Generator](../generators.mdx) to parse structured output from LLM responses |
| **Mandatory init variables** | `regex_pattern`: The regular expression pattern used to extract text |
| **Mandatory run variables** | `text_or_messages`: A string or a list of `ChatMessage` objects to search through |
| **Output variables** | `captured_text`: The extracted text from the first capture group |
| **API reference** | [Extractors](/reference/extractors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/extractors/regex_text_extractor.py |
| **Package name** | `haystack-ai` |
## Overview
`RegexTextExtractor` parses text input or `ChatMessage` objects using a regular expression pattern and extracts text captured by capture groups. This is useful for extracting structured information from LLM outputs that follow specific formats, such as XML-like tags or other patterns.
The component works with both plain strings and lists of `ChatMessage` objects. When given a list of messages, it processes only the last message.
The regex pattern should include at least one capture group (text within parentheses) to specify what text to extract. If no capture group is provided, the entire match is returned instead.
### Handling no matches
When the pattern doesn't match, the component returns `captured_text` as an empty string:
```python
from haystack.components.extractors import RegexTextExtractor
extractor = RegexTextExtractor(regex_pattern=r"
| | |
| --- | --- |
| **Most common position in a pipeline** | After the [PreProcessor](../preprocessors.mdx) in an indexing pipeline or after a [Retriever](../retrievers.mdx) in a query pipeline |
| **Mandatory init variables** | `model`: Name or path of the spaCy model to use |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Spacy](/reference/integrations-spacy) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/spacy |
| **Package name** | `spacy-haystack` |
## Overview
`SpacyNamedEntityExtractor` looks for entities, which are spans in the text. The extractor automatically recognizes and groups them depending on their class, such as people's names, organizations, locations, and other types. The exact classes are determined by the model that you initialize the component with.
`SpacyNamedEntityExtractor` takes a list of documents as input and returns a list of the same documents with their `meta` data enriched with `NamedEntityAnnotations`. A `NamedEntityAnnotation` consists of the type of the entity and the start and end of the span, for example: `NamedEntityAnnotation(entity='PERSON', start=11, end=16, score=None)`.
When the `SpacyNamedEntityExtractor` is initialized, you need to set a `model`. Optionally, you can set `pipeline_kwargs`, which are then passed on to the spaCy pipeline. You can additionally set the `device` that is used to run the component.
## Usage
Install the `spacy-haystack` package to use the `SpacyNamedEntityExtractor`:
```shell
pip install spacy-haystack
```
The component works with any [spaCy model](https://spacy.io/models) that contains an NER component.
`SpacyNamedEntityExtractor` accepts a list of `Documents` as its input. The extractor annotates the raw text in the documents and stores the annotations in the document's `meta` dictionary under the `named_entities` key.
```python
from haystack.dataclasses import Document
from haystack_integrations.components.extractors.spacy import (
SpacyNamedEntityExtractor,
)
extractor = SpacyNamedEntityExtractor(model="en_core_web_sm")
documents = [
Document(content="My name is Clara and I live in Berkeley, California."),
Document(content="I'm Merlin, the happy pig!"),
Document(content="New York State is home to the Empire State Building."),
]
result = extractor.run(documents)
print(result["documents"])
```
Here is the example result:
```python
[Document(id=aec840d1b6c85609f4f16c3e222a5a25fd8c4c53bd981a40c1268ab9c72cee10, content: 'My name is Clara and I live in Berkeley, California.', meta: {'named_entities': [NamedEntityAnnotation(entity='PERSON', start=11, end=16, score=None), NamedEntityAnnotation(entity='GPE', start=31, end=39, score=None), NamedEntityAnnotation(entity='GPE', start=41, end=51, score=None)]}),
Document(id=98f1dc5d0ccd9d9950cd191d1076db0f7af40c401dd7608f11c90cb3fc38c0c2, content: 'I'm Merlin, the happy pig!', meta: {'named_entities': [NamedEntityAnnotation(entity='PERSON', start=4, end=10, score=None)]}),
Document(id=44948ea0eec018b33aceaaedde4616eb9e93ce075e0090ec1613fc145f84b4a9, content: 'New York State is home to the Empire State Building.', meta: {'named_entities': [NamedEntityAnnotation(entity='GPE', start=0, end=14, score=None), NamedEntityAnnotation(entity='ORG', start=26, end=51, score=None)]})]
```
### Get stored annotations
This component includes the `get_stored_annotations` helper class method that allows you to retrieve the annotations stored in a `Document` transparently:
```python
from haystack.dataclasses import Document
from haystack_integrations.components.extractors.spacy import (
SpacyNamedEntityExtractor,
)
extractor = SpacyNamedEntityExtractor(model="en_core_web_sm")
documents = [
Document(content="My name is Clara and I live in Berkeley, California."),
Document(content="I'm Merlin, the happy pig!"),
Document(content="New York State is home to the Empire State Building."),
]
result = extractor.run(documents)
annotations = [
SpacyNamedEntityExtractor.get_stored_annotations(doc) for doc in result["documents"]
]
print(annotations)
# If a Document doesn't contain any annotations, this returns None.
new_doc = Document(content="In one of many possible worlds...")
assert SpacyNamedEntityExtractor.get_stored_annotations(new_doc) is None
```
---
// File: pipeline-components/extractors/transformersnamedentityextractor
# TransformersNamedEntityExtractor
This component extracts predefined entities out of a piece of text and writes them into documents’ meta field.
| | |
| --- | --- |
| **Most common position in a pipeline** | After the [PreProcessor](../preprocessors.mdx) in an indexing pipeline or after a [Retriever](../retrievers.mdx) in a query pipeline |
| **Mandatory init variables** | `model`: Name or path of the model to use |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Transformers](/reference/integrations-transformers) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/transformers |
| **Package name** | `transformers-haystack` |
## Overview
`TransformersNamedEntityExtractor` looks for entities, which are spans in the text. The extractor automatically recognizes and groups them depending on their class, such as people's names, organizations, locations, and other types. The exact classes are determined by the model that you initialize the component with.
`TransformersNamedEntityExtractor` takes a list of documents as input and returns a list of the same documents with their `meta` data enriched with `NamedEntityAnnotations`. A `NamedEntityAnnotation` consists of the type of the entity, the start and end of the span, and a score calculated by the model, for example: `NamedEntityAnnotation(entity='PER', start=11, end=16, score=0.9)`.
When the `TransformersNamedEntityExtractor` is initialized, you need to set a `model`. Optionally, you can set `pipeline_kwargs`, which are then passed on to the Hugging Face pipeline. You can additionally set the `device` that is used to run the component.
Authentication with a Hugging Face API token is only required to access private or gated models. You can pass the token at initialization with `token`, or set the `HF_API_TOKEN` or `HF_TOKEN` environment variable.
## Usage
Install the `transformers-haystack` package to use the `TransformersNamedEntityExtractor`:
```shell
pip install transformers-haystack
```
The component works with any Hugging Face model that supports token classification or NER.
`TransformersNamedEntityExtractor` accepts a list of `Documents` as its input. The extractor annotates the raw text in the documents and stores the annotations in the document's `meta` dictionary under the `named_entities` key.
```python
from haystack.dataclasses import Document
from haystack_integrations.components.extractors.transformers import (
TransformersNamedEntityExtractor,
)
extractor = TransformersNamedEntityExtractor(model="dslim/bert-base-NER")
documents = [
Document(content="My name is Clara and I live in Berkeley, California."),
Document(content="I'm Merlin, the happy pig!"),
Document(content="New York State is home to the Empire State Building."),
]
result = extractor.run(documents)
print(result["documents"])
```
Here is the example result:
```python
[Document(id=aec840d1b6c85609f4f16c3e222a5a25fd8c4c53bd981a40c1268ab9c72cee10, content: 'My name is Clara and I live in Berkeley, California.', meta: {'named_entities': [NamedEntityAnnotation(entity='PER', start=11, end=16, score=np.float32(0.99641764)), NamedEntityAnnotation(entity='LOC', start=31, end=39, score=np.float32(0.996198)), NamedEntityAnnotation(entity='LOC', start=41, end=51, score=np.float32(0.9990196))]}),
Document(id=98f1dc5d0ccd9d9950cd191d1076db0f7af40c401dd7608f11c90cb3fc38c0c2, content: 'I'm Merlin, the happy pig!', meta: {'named_entities': [NamedEntityAnnotation(entity='PER', start=4, end=10, score=np.float32(0.99054915))]}),
Document(id=44948ea0eec018b33aceaaedde4616eb9e93ce075e0090ec1613fc145f84b4a9, content: 'New York State is home to the Empire State Building.', meta: {'named_entities': [NamedEntityAnnotation(entity='LOC', start=0, end=14, score=np.float32(0.9989541)), NamedEntityAnnotation(entity='LOC', start=30, end=51, score=np.float32(0.9574631))]})]
```
### Get stored annotations
This component includes the `get_stored_annotations` helper class method that allows you to retrieve the annotations stored in a `Document` transparently:
```python
from haystack.dataclasses import Document
from haystack_integrations.components.extractors.transformers import (
TransformersNamedEntityExtractor,
)
extractor = TransformersNamedEntityExtractor(model="dslim/bert-base-NER")
documents = [
Document(content="My name is Clara and I live in Berkeley, California."),
Document(content="I'm Merlin, the happy pig!"),
Document(content="New York State is home to the Empire State Building."),
]
result = extractor.run(documents)
annotations = [
TransformersNamedEntityExtractor.get_stored_annotations(doc)
for doc in result["documents"]
]
print(annotations)
# If a Document doesn't contain any annotations, this returns None.
new_doc = Document(content="In one of many possible worlds...")
assert TransformersNamedEntityExtractor.get_stored_annotations(new_doc) is None
```
---
// File: pipeline-components/extractors
# Extractors
| Name | Description |
| --- | --- |
| [LLMDocumentContentExtractor](extractors/llmdocumentcontentextractor.mdx) | Extracts textual content from image-based documents using a vision-enabled Large Language Model (LLM). |
| [LLMMetadataExtractor](extractors/llmmetadataextractor.mdx) | Extracts metadata from documents using a Large Language Model. The metadata is extracted by providing a prompt to a LLM that generates it. |
| [PresidioEntityExtractor](extractors/presidioentityextractor.mdx) | Detects PII in Documents and stores entities as structured metadata, without modifying the text. Powered by Microsoft Presidio. |
| [RegexTextExtractor](extractors/regextextextractor.mdx) | Extracts text from chat messages or strings using a regular expression pattern. |
| [SpacyNamedEntityExtractor](extractors/spacynamedentityextractor.mdx) | Extracts predefined entities out of a piece of text and writes them into documents' meta field. Uses a spaCy model. |
| [TransformersNamedEntityExtractor](extractors/transformersnamedentityextractor.mdx) | Extracts predefined entities out of a piece of text and writes them into documents' meta field. Uses a Hugging Face model. |
---
// File: pipeline-components/fetchers/external-integrations-fetchers
# External Integrations
External integrations that enable data extraction from different sources.
| Name | Description |
| --- | --- |
| [Apify](https://haystack.deepset.ai/integrations/apify) | Extract data from e-commerce websites, social media platforms (such as Facebook, Instagram, and TikTok), search engines, online maps, and more, while automating web tasks. |
| [Bright Data](https://haystack.deepset.ai/integrations/bright-data) | Extract data from 45+ websites, get search engine results, and access geo-restricted content using Bright Data's web scraping services. |
| [Mastodon](https://haystack.deepset.ai/integrations/mastodon-fetcher) | Fetch a Mastodon username's latest posts. |
| [Notion](https://haystack.deepset.ai/integrations/notion-extractor) | Extract pages from Notion to Haystack Documents. |
---
// File: pipeline-components/fetchers/firecrawlcrawler
# FirecrawlCrawler
Use Firecrawl to crawl websites and return the content as Haystack Documents. Unlike single-page fetchers, FirecrawlCrawler follows links and discovers subpages.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing or query pipelines as the data fetching step |
| **Mandatory run variables** | `urls`: A list of URLs (strings) to start crawling from |
| **Output variables** | `documents`: A list of [Documents](../../concepts/data-classes.mdx) |
| **API reference** | [Firecrawl](/reference/integrations-firecrawl) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/firecrawl |
| **Package name** | `firecrawl-haystack` |
## Overview
`FirecrawlCrawler` uses [Firecrawl](https://firecrawl.dev) to crawl one or more URLs and return the extracted content as Haystack `Document` objects. Starting from each given URL, it follows links to discover subpages up to a configurable limit. This makes it well-suited for ingesting entire websites or documentation sites, not just single pages.
Firecrawl returns content in a structured format that works well as input for LLMs. Each crawled page becomes a separate `Document` with the page content in the `content` field and metadata, such as title, URL, and description, in the `meta` field.
### Crawl parameters
You can control the crawl behavior through the `params` argument. Some commonly used parameters:
- `limit`: Maximum number of pages to crawl per URL. Defaults to `1`. Without a limit, Firecrawl may crawl all subpages and consume credits quickly.
- `scrape_options`: Controls the output format. Defaults to `{"formats": ["markdown"]}`.
See the [Firecrawl API reference](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post) for the full list of available parameters.
### Authorization
`FirecrawlCrawler` uses the `FIRECRAWL_API_KEY` environment variable by default. You can also pass the key explicitly at initialization:
```python
from haystack.utils import Secret
from haystack_integrations.components.fetchers.firecrawl import FirecrawlCrawler
crawler = FirecrawlCrawler(api_key=Secret.from_token("
| | |
| --- | --- |
| **Most common position in a pipeline** | After [`GoogleDriveRetriever`](../retrievers/googledriveretriever.mdx), before a Router or File Converters |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `access_token`: A delegated Google OAuth bearer token, typically wired from an upstream `OAuthTokenResolver`
`targets`: A list of `Document`s (from `GoogleDriveRetriever`) or raw Google Drive file ids / URLs | | **Output variables** | `streams`: A list of [ByteStreams](../../concepts/data-classes.mdx) holding the fetched content | | **API reference** | [Google Drive](/reference/integrations-google-drive) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_drive | | **Package name** | `google-drive-haystack` |
## Overview
`GoogleDriveFetcher` downloads the full content of Google Drive files through the [Drive API v3](https://developers.google.com/drive/api/reference/rest/v3) and returns `ByteStream` objects, ready for a downstream converter.
It complements [`GoogleDriveRetriever`](../retrievers/googledriveretriever.mdx), which returns only metadata (and optionally exported text). Wire the retriever's `documents` (or a list of file ids / Drive URLs) into the fetcher to download the underlying content. The fetcher dispatches on each file's mime type:
- **Binary files** (PDF, DOCX, images, ...) are downloaded as-is via `files.get?alt=media`.
- **Native Google Docs/Sheets/Slides** are exported with `files.export`, by default to the Office formats (DOCX/XLSX/PPTX), configurable via `export_mime_types`.
- **Folders** and other non-downloadable Google types (Forms, Sites, ...) are skipped.
Each `ByteStream`'s `meta` carries `file_id`, `web_url`, `file_name`, and `content_type`. Because the output is a list of `ByteStream`s of mixed types, the typical next step is a [`FileTypeRouter`](../routers/filetyperouter.mdx) that dispatches each stream to the right converter ([`PyPDFToDocument`](../converters/pypdftodocument.mdx), [`DOCXToDocument`](../converters/docxtodocument.mdx), [`XLSXToDocument`](../converters/xlsxtodocument.mdx), or [`PPTXToDocument`](../converters/pptxtodocument.mdx)).
### Authentication
The fetcher takes a per-user `access_token` as a run input. The token must carry a delegated Google OAuth scope that allows reading file content, for example `https://www.googleapis.com/auth/drive.readonly`. Typically you wire it from an upstream [`OAuthTokenResolver`](../connectors/oauthtokenresolver.mdx), which emits a plain string. A `Secret` is also accepted and resolved internally.
### Error handling and concurrency
- `raise_on_failure` (default `True`): when `False`, a failed fetch is logged and the file is skipped, so the remaining files are still returned.
- `max_retries` (default `3`): retries on throttled (HTTP 429) and transient server errors.
- `max_concurrent_requests` (default `5`): bounds the number of files fetched concurrently by `run_async` to avoid tripping Drive rate limits. It has no effect on the synchronous `run`, which fetches files one at a time.
- `export_mime_types`: overrides the default native-Google-to-Office export mapping. Drive caps a single export at 10 MB.
### Installation
Install the Google Drive integration with:
```shell
pip install google-drive-haystack
```
## Usage
### On its own
`access_token` below is a per-user delegated Google OAuth bearer token. You can pass either raw file ids / Drive URLs or the `Document`s produced by `GoogleDriveRetriever`.
```python
from haystack_integrations.components.fetchers.google_drive import GoogleDriveFetcher
fetcher = GoogleDriveFetcher()
result = fetcher.run(
access_token="my-delegated-google-token",
targets=[
"https://drive.google.com/file/d/1AbCdEfGhIjKlMnOpQrStUvWxYz/view",
],
)
for stream in result["streams"]:
print(stream.meta["file_name"], stream.meta["content_type"])
```
### In a pipeline
The following query pipeline ties the whole integration together: an [`OAuthTokenResolver`](../connectors/oauthtokenresolver.mdx) provides a token, [`GoogleDriveRetriever`](../retrievers/googledriveretriever.mdx) searches Drive, `GoogleDriveFetcher` downloads the matching files, and a [`FileTypeRouter`](../routers/filetyperouter.mdx) sends each `ByteStream` to the right converter. Note that the resolver's single `access_token` output feeds both the retriever and the fetcher.
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.routers import FileTypeRouter
from haystack.components.converters import PyPDFToDocument, DOCXToDocument
from haystack_integrations.components.connectors.oauth import OAuthTokenResolver
from haystack_integrations.utils.oauth import OAuthRefreshTokenSource
from haystack_integrations.components.retrievers.google_drive import (
GoogleDriveRetriever,
)
from haystack_integrations.components.fetchers.google_drive import GoogleDriveFetcher
pipeline = Pipeline()
pipeline.add_component(
"resolver",
OAuthTokenResolver(
token_source=OAuthRefreshTokenSource(
token_url="https://oauth2.googleapis.com/token",
client_id="aaa-bbb-ccc",
refresh_token=Secret.from_env_var("GOOGLE_REFRESH_TOKEN"),
scopes=["https://www.googleapis.com/auth/drive.readonly"],
),
),
)
pipeline.add_component("retriever", GoogleDriveRetriever(top_k=5))
pipeline.add_component("fetcher", GoogleDriveFetcher())
pipeline.add_component(
"router",
FileTypeRouter(
mime_types=[
"application/pdf",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
],
),
)
pipeline.add_component("pdf_converter", PyPDFToDocument())
pipeline.add_component("docx_converter", DOCXToDocument())
# The same token feeds both the retriever and the fetcher.
pipeline.connect("resolver.access_token", "retriever.access_token")
pipeline.connect("resolver.access_token", "fetcher.access_token")
# The retrieved documents become the fetcher's targets.
pipeline.connect("retriever.documents", "fetcher.targets")
# Route each fetched ByteStream to the matching converter.
pipeline.connect("fetcher.streams", "router.sources")
pipeline.connect("router.application/pdf", "pdf_converter.sources")
pipeline.connect(
"router.application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"docx_converter.sources",
)
result = pipeline.run({"retriever": {"query": "quarterly roadmap"}})
```
---
// File: pipeline-components/fetchers/linkcontentfetcher
# LinkContentFetcher
With LinkContentFetcher, you can use the contents of several URLs as the data for your pipeline. You can use it in indexing and query pipelines to fetch the contents of the URLs you give it.
`targets`: A list of `Document`s (from `GoogleDriveRetriever`) or raw Google Drive file ids / URLs | | **Output variables** | `streams`: A list of [ByteStreams](../../concepts/data-classes.mdx) holding the fetched content | | **API reference** | [Google Drive](/reference/integrations-google-drive) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_drive | | **Package name** | `google-drive-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing or query pipelines as the data fetching step |
| **Mandatory run variables** | `urls`: A list of URLs (strings) |
| **Output variables** | `streams`: A list of [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
| **API reference** | [Fetchers](/reference/fetchers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/fetchers/link_content.py |
| **Package name** | `haystack-ai` |
## Overview
`LinkContentFetcher` fetches the contents of the `urls` you give it and returns a list of content streams. Each item in this list is the content of one link it successfully fetched in the form of a `ByteStream` object. Each of these objects in the returned list has metadata that contains its content type (in the `content_type` key) and its URL (in the `url` key).
For example, if you pass ten URLs to `LinkContentFetcher` and it manages to fetch six of them, then the output will be a list of six `ByteStream` objects, each containing information about its content type and URL.
It may happen that some sites block `LinkContentFetcher` from getting their content. In that case, it logs the error and returns the `ByteStream` objects that it successfully fetched.
Often, to use this component in a pipeline, you must convert the returned list of `ByteStream` objects into a list of `Document` objects. To do so, you can use the `HTMLToDocument` component.
You can use `LinkContentFetcher` at the beginning of an indexing pipeline to index the contents of URLs into a Document Store. You can also use it directly in a query pipeline, such as a retrieval-augmented generative (RAG) pipeline, to use the contents of a URL as the data source.
## Security considerations
`LinkContentFetcher` requests the URLs passed to it. If those URLs come directly from end users, this can expose your environment to server-side request forgery (SSRF) risks.
Before calling `LinkContentFetcher`, an application should therefore validate and sanitize user-provided URLs. For example:
- Allow only expected schemes, for example `https`
- Use an allowlist of trusted domains when possible
- Block localhost, link-local, and private-network destinations
- Consider using an outbound proxy or network-level egress restrictions in production
For example, an application could block private, loopback, link-local, reserved IPs, and custom IP ranges using the standard library's `ipaddress` module:
```python
import ipaddress
from urllib.parse import urlparse
PRIVATE_RANGES = (
ipaddress.ip_network("127.0.0.0/8"),
ipaddress.ip_network("10.0.0.0/8"),
ipaddress.ip_network("172.16.0.0/12"),
ipaddress.ip_network("192.168.0.0/16"),
ipaddress.ip_network("169.254.0.0/16"),
)
def is_unsafe_url(url: str) -> bool:
parsed = urlparse(url)
if parsed.scheme != "https" or not parsed.hostname:
return True
try:
ip = ipaddress.ip_address(parsed.hostname)
except ValueError:
# Hostname (not a raw IP). Apply your own domain allowlist policy here. Filter out "LOCALHOST" etc.
return False
return (
ip.is_private
or ip.is_loopback
or ip.is_link_local
or ip.is_reserved
or any(ip in net for net in PRIVATE_RANGES)
)
```
## Usage
### On its own
Below is an example where `LinkContentFetcher` fetches the contents of a URL. It initializes the component using the default settings. To change the default component settings, such as `retry_attempts`, check out the API reference [docs](/reference/fetchers-api).
```python
from haystack.components.fetchers import LinkContentFetcher
fetcher = LinkContentFetcher()
fetcher.run(urls=["https://haystack.deepset.ai"])
```
### In a pipeline
Below is an example of an indexing pipeline that uses the `LinkContentFetcher` to index the contents of the specified URLs into an `InMemoryDocumentStore`. Notice how it uses the `HTMLToDocument` component to convert the list of `ByteStream` objects to `Document` objects.
```python
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
fetcher = LinkContentFetcher()
converter = HTMLToDocument()
writer = DocumentWriter(document_store=document_store)
indexing_pipeline = Pipeline()
indexing_pipeline.add_component(instance=fetcher, name="fetcher")
indexing_pipeline.add_component(instance=converter, name="converter")
indexing_pipeline.add_component(instance=writer, name="writer")
indexing_pipeline.connect("fetcher.streams", "converter.sources")
indexing_pipeline.connect("converter.documents", "writer.documents")
indexing_pipeline.run(
data={
"fetcher": {
"urls": [
"https://haystack.deepset.ai/blog/guide-to-using-zephyr-with-haystack2",
],
},
},
)
```
---
// File: pipeline-components/fetchers/mssharepointfetcher
# MSSharePointFetcher
Fetches the full content of Microsoft SharePoint and OneDrive items via the Microsoft Graph API and returns it as ByteStreams.
| | |
| --- | --- |
| **Most common position in a pipeline** | After [`MSSharePointRetriever`](../retrievers/mssharepointretriever.mdx), before a Router or File Converters |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `access_token`: A delegated Microsoft Graph bearer token, typically wired from an upstream `OAuthTokenResolver`
`targets`: A list of `Document`s (from `MSSharePointRetriever`) or raw SharePoint/OneDrive `web_url` strings | | **Output variables** | `streams`: A list of [ByteStreams](../../concepts/data-classes.mdx) holding the fetched content | | **API reference** | [Microsoft SharePoint](/reference/integrations-microsoft-sharepoint) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/microsoft_sharepoint | | **Package name** | `microsoft-sharepoint-haystack` |
## Overview
`MSSharePointFetcher` downloads the full content of Microsoft SharePoint and OneDrive items through the [Microsoft Graph API](https://learn.microsoft.com/en-us/graph/use-the-api) and returns `ByteStream` objects, ready for a downstream converter.
It complements [`MSSharePointRetriever`](../retrievers/mssharepointretriever.mdx), which returns only Search snippets and metadata. Wire the retriever's `documents` (or a list of `web_url`s) into the fetcher to download the underlying content. The fetcher dispatches on the entity type of each hit:
- **Files** (`driveItem`) are downloaded as their raw bytes (PDF, DOCX, ...).
- **List items** (`listItem`) are returned as a JSON `ByteStream` of the item's column values (`fields`).
- **SharePoint pages** (`sitePage`) are returned as an HTML `ByteStream` built from the page's web parts.
Each `ByteStream`'s `meta` carries `url`, `file_name`, `content_type`, and a normalized `entity_type` (`driveItem`, `listItem`, or `sitePage`). Everything is resolved through the Microsoft Graph `shares` endpoint (plus the Pages API for pages), so only the `web_url` already exposed by the retriever is needed.
Because the output is a list of `ByteStream`s of mixed types, the typical next step is a [`FileTypeRouter`](../routers/filetyperouter.mdx) that dispatches each stream to the right converter ([`PyPDFToDocument`](../converters/pypdftodocument.mdx), [`DOCXToDocument`](../converters/docxtodocument.mdx), [`HTMLToDocument`](../converters/htmltodocument.mdx), or a JSON converter).
### Authentication
The fetcher takes a per-user `access_token` as a run input. The token must carry **delegated** Microsoft Graph permissions (for example `Files.Read.All` for files and `Sites.Read.All` for list items and pages). Typically you wire it from an upstream [`OAuthTokenResolver`](../connectors/oauthtokenresolver.mdx), which emits a plain string. A `Secret` is also accepted and resolved internally.
### Error handling and concurrency
- `raise_on_failure` (default `True`): when `False`, a failed fetch is logged and the item is skipped, so the remaining items are still returned.
- `max_retries` (default `3`): retries on throttled (HTTP 429) and transient server errors.
- `max_concurrent_requests` (default `5`): bounds the number of items fetched concurrently by `run_async` to avoid tripping Microsoft Graph rate limits. It has no effect on the synchronous `run`, which fetches items one at a time.
### Installation
Install the Microsoft SharePoint integration with:
```shell
pip install microsoft-sharepoint-haystack
```
## Usage
### On its own
`access_token` below is a per-user delegated Microsoft Graph bearer token. You can pass either raw `web_url` strings or the `Document`s produced by `MSSharePointRetriever`.
```python
from haystack_integrations.components.fetchers.microsoft_sharepoint import (
MSSharePointFetcher,
)
fetcher = MSSharePointFetcher()
result = fetcher.run(
access_token="my-delegated-graph-token",
targets=[
"https://contoso.sharepoint.com/sites/contoso-team/contoso-designs.docx",
],
)
for stream in result["streams"]:
print(stream.meta["file_name"], stream.meta["content_type"])
```
### In a pipeline
The following query pipeline ties the whole integration together: an [`OAuthTokenResolver`](../connectors/oauthtokenresolver.mdx) provides a token, [`MSSharePointRetriever`](../retrievers/mssharepointretriever.mdx) searches SharePoint, `MSSharePointFetcher` downloads the matching items, and a [`FileTypeRouter`](../routers/filetyperouter.mdx) sends each `ByteStream` to the right converter. Note that the resolver's single `access_token` output feeds both the retriever and the fetcher.
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.routers import FileTypeRouter
from haystack.components.converters import PyPDFToDocument, DOCXToDocument
from haystack_integrations.components.connectors.oauth import OAuthTokenResolver
from haystack_integrations.utils.oauth import OAuthRefreshTokenSource
from haystack_integrations.components.retrievers.microsoft_sharepoint import (
MSSharePointRetriever,
)
from haystack_integrations.components.fetchers.microsoft_sharepoint import (
MSSharePointFetcher,
)
pipeline = Pipeline()
pipeline.add_component(
"resolver",
OAuthTokenResolver(
token_source=OAuthRefreshTokenSource(
token_url="https://login.microsoftonline.com/common/oauth2/v2.0/token",
client_id="aaa-bbb-ccc",
refresh_token=Secret.from_env_var("MS_REFRESH_TOKEN"),
scopes=[
"https://graph.microsoft.com/Files.Read.All",
"https://graph.microsoft.com/Sites.Read.All",
"offline_access",
],
),
),
)
pipeline.add_component("retriever", MSSharePointRetriever(top_k=5))
pipeline.add_component("fetcher", MSSharePointFetcher())
pipeline.add_component(
"router",
FileTypeRouter(
mime_types=[
"application/pdf",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
],
),
)
pipeline.add_component("pdf_converter", PyPDFToDocument())
pipeline.add_component("docx_converter", DOCXToDocument())
# The same token feeds both the retriever and the fetcher.
pipeline.connect("resolver.access_token", "retriever.access_token")
pipeline.connect("resolver.access_token", "fetcher.access_token")
# The retrieved documents become the fetcher's targets.
pipeline.connect("retriever.documents", "fetcher.targets")
# Route each fetched ByteStream to the matching converter.
pipeline.connect("fetcher.streams", "router.sources")
pipeline.connect("router.application/pdf", "pdf_converter.sources")
pipeline.connect(
"router.application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"docx_converter.sources",
)
result = pipeline.run({"retriever": {"query": "quarterly roadmap"}})
```
---
// File: pipeline-components/fetchers/tavilyfetcher
# TavilyFetcher
Use Tavily Extract to fetch and parse content from URLs as Haystack Documents. Unlike web search, it retrieves content from the URLs you provide rather than discovering them via a query.
`targets`: A list of `Document`s (from `MSSharePointRetriever`) or raw SharePoint/OneDrive `web_url` strings | | **Output variables** | `streams`: A list of [ByteStreams](../../concepts/data-classes.mdx) holding the fetched content | | **API reference** | [Microsoft SharePoint](/reference/integrations-microsoft-sharepoint) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/microsoft_sharepoint | | **Package name** | `microsoft-sharepoint-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing or query pipelines as the data fetching step |
| **Mandatory init variables** | `api_key`: The Tavily API key. Can be set with the `TAVILY_API_KEY` env var. |
| **Mandatory run variables** | `urls`: A list of URLs (strings) to extract content from (max 20 per request) |
| **Output variables** | `documents`: A list of [Documents](../../concepts/data-classes.mdx)
`meta`: Request-level metadata (`response_time`, `usage`, `request_id`, `failed_results`) | | **API reference** | [Tavily](/reference/integrations-tavily) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/tavily | | **Package name** | `tavily-haystack` |
## Overview
`TavilyFetcher` wraps the [Tavily Extract API](https://docs.tavily.com/documentation/api-reference/endpoint/extract) to retrieve and parse web page content from one or more specified URLs. PDF URLs are also supported. Each successful URL becomes a Haystack `Document` with page content in `content` and metadata such as `url` (and optionally `images`) in `meta`.
This component is complementary to [`TavilyWebSearch`](../websearch/tavilywebsearch.mdx): search discovers URLs from a query, while `TavilyFetcher` extracts full content from URLs you already have.
### Extract parameters
You can control extraction behavior at initialization:
- `extract_depth`: `"basic"` (fast, lower cost) or `"advanced"` (more data including tables; higher latency and cost). Defaults to `"basic"`.
- `include_images`: When `True`, image URLs are stored on each Document under `meta["images"]`. Defaults to `False`.
- `extract_params`: Extra kwargs forwarded to the Tavily Extract API (for example `format`, `include_favicon`, `query`, `chunks_per_source`). See the [Tavily Extract API reference](https://docs.tavily.com/documentation/api-reference/endpoint/extract).
Of these, only `extract_params` can also be passed to `run()` to override it for a single call. Note that an `extract_params` dictionary passed to `run()` fully replaces the one set at initialization instead of being merged with it.
### Authorization
`TavilyFetcher` uses the `TAVILY_API_KEY` environment variable by default. You can also pass the key explicitly:
```python
from haystack.utils import Secret
from haystack_integrations.components.fetchers.tavily import TavilyFetcher
fetcher = TavilyFetcher(api_key=Secret.from_token("`meta`: Request-level metadata (`response_time`, `usage`, `request_id`, `failed_results`) | | **API reference** | [Tavily](/reference/integrations-tavily) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/tavily | | **Package name** | `tavily-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: The AIMLAPI API key. Can be set with `AIMLAPI_API_KEY` env var. |
| **Mandatory run variables** | `messages` A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [AIMLAPI](/reference/integrations-aimlapi) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/aimlapi |
| **Package name** | `aimlapi-haystack` |
## Overview
`AIMLAPIChatGenerator` provides access to AI models through the AIMLAPI, a unified API gateway for models from various providers. You can use different models within a single pipeline with a consistent interface. The default model is `openai/gpt-5-chat-latest`.
AIMLAPI uses a single API key for all providers, which allows you to switch between or combine different models without managing multiple credentials.
For a complete list of available models, check the [AIMLAPI documentation](https://docs.aimlapi.com/).
The component needs a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
You can pass any chat completion parameters valid for the underlying model directly to `AIMLAPIChatGenerator` using the `generation_kwargs` parameter, both at initialization and to the `run()` method.
### Authentication
`AIMLAPIChatGenerator` needs an AIMLAPI API key to work. You can set this key in:
- The `api_key` init parameter using [Secret API](../../concepts/secret-management.mdx)
- The `AIMLAPI_API_KEY` environment variable (recommended)
### Structured Output
`AIMLAPIChatGenerator` supports structured output generation for compatible models, allowing you to receive responses in a predictable format. You can use Pydantic models or JSON schemas to define the structure of the output through the `response_format` parameter in `generation_kwargs`.
This is useful when you need to extract structured data from text or generate responses that match a specific format.
```python
from pydantic import BaseModel
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.aimlapi import AIMLAPIChatGenerator
class CityInfo(BaseModel):
city_name: str
country: str
population: int
famous_for: str
client = AIMLAPIChatGenerator(
model="openai/gpt-4o-2024-08-06",
generation_kwargs={"response_format": CityInfo}
)
response = client.run(messages=[
ChatMessage.from_user(
"Berlin is the capital and largest city of Germany with a population of "
"approximately 3.7 million. It's famous for its history, culture, and nightlife."
)
])
print(response["replies"][0].text)
>> {"city_name":"Berlin","country":"Germany","population":3700000,
>> "famous_for":"history, culture, and nightlife"}
```
:::info[Model Compatibility]
Structured output support depends on the underlying model. OpenAI models starting from `gpt-4o-2024-08-06` support Pydantic models and JSON schemas. For details on which models support this feature, refer to the respective model provider's documentation.
:::
### Tool Support
`AIMLAPIChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.aimlapi import AIMLAPIChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = AIMLAPIChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
`AIMLAPIChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
You can stream output as it's generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
```python
from haystack.components.generators.utils import print_streaming_chunk
from haystack_integrations.components.generators.aimlapi import AIMLAPIChatGenerator
# Configure the generator with a streaming callback
component = AIMLAPIChatGenerator(streaming_callback=print_streaming_chunk)
# Pass a list of messages
from haystack.dataclasses import ChatMessage
component.run([ChatMessage.from_user("Your question here")])
```
:::info
Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
:::
See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
We recommend to give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
## Usage
Install the `aimlapi-haystack` package to use the `AIMLAPIChatGenerator`:
```shell
pip install aimlapi-haystack
```
### On its own
```python
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.aimlapi import AIMLAPIChatGenerator
client = AIMLAPIChatGenerator(model="openai/gpt-5-chat-latest", streaming_callback=print_streaming_chunk)
response = client.run([ChatMessage.from_user("What's Natural Language Processing? Be brief.")])
>> Natural Language Processing (NLP) is a field of artificial intelligence that
>> focuses on the interaction between computers and humans through natural language.
>> It involves enabling machines to understand, interpret, and generate human
>> language in a meaningful way, facilitating tasks such as language translation,
>> sentiment analysis, and text summarization.
print(response)
>> {'replies': [ChatMessage(_role=
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `model`: The model to use
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.
`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) instances | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
[Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) is a fully managed service that makes high-performing foundation models from leading AI startups and Amazon available through a unified API. You can choose from various foundation models to find the one best suited for your use case.
`AmazonBedrockChatGenerator` enables chat completion using chat models from Amazon, Anthropic, Cohere, Meta, Mistral, and more with a single component.
## Overview
This component uses AWS for authentication. You can use the AWS CLI to authenticate through your IAM. For more information on setting up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
:::info[Using AWS CLI]
Consider using AWS CLI as a more straightforward tool to manage your AWS services. With AWS CLI, you can quickly configure your [boto3 credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). This way, you won't need to provide detailed authentication parameters when initializing Amazon Bedrock Generator in Haystack.
:::
To use this component for text generation, initialize an AmazonBedrockChatGenerator with the model name, the AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`) should be set as environment variables, be configured as described above or passed as [Secret](../../concepts/secret-management.mdx) arguments. Note, make sure the region you set supports Amazon Bedrock.
### Tool Support
`AmazonBedrockChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = AmazonBedrockChatGenerator(
model="global.anthropic.claude-sonnet-4-6",
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
### Prompt Caching
`AmazonBedrockChatGenerator` supports prompt caching, to reduce inference response latency and input token costs.
Prompt caching on Bedrock is available for [selected models](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html).
It allows you to define cache points within a request, as long as the input meets a model-specific minimum token threshold.
Each request can contain up to four cache points.
#### Caching messages
This generator allows you to control cache points at the `ChatMessage` level via the `meta` field.
For example, to cache a long user message to be reused across multiple requests:
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.amazon_bedrock import (
AmazonBedrockChatGenerator,
)
msg = ChatMessage.from_user(
"long message...",
meta={"cachePoint": {"type": "default", "ttl": "5m"}},
)
generator = AmazonBedrockChatGenerator(
model="global.anthropic.claude-sonnet-4-6",
)
result = generator.run(messages=[msg])
```
If the cache point is successfully written, the number of cached input tokens is available at:
```python
result["replies"][0].meta["usage"]["cache_write_input_tokens"]
```
#### Caching tools
You can also cache tool definitions using the `tools_cachepoint_config` initialization parameter.
When provided, all tools sent to the model are cached, if they exceed the minimum token threshold and the selected
model supports prompt caching.
```python
from haystack_integrations.components.generators.amazon_bedrock import (
AmazonBedrockChatGenerator,
)
# define or load your tools
generator = AmazonBedrockChatGenerator(
model="global.anthropic.claude-sonnet-4-6",
tools=my_tools,
tools_cachepoint_config={"type": "default", "ttl": "5m"},
)
# send a request to the Language Model
```
For more details on how prompt caching works in Amazon Bedrock, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html).
## Usage
To start using Amazon Bedrock with Haystack, install the `amazon-bedrock-haystack` package:
```shell
pip install amazon-bedrock-haystack
```
### On its own
Basic usage:
```python
from haystack_integrations.components.generators.amazon_bedrock import (
AmazonBedrockChatGenerator,
)
from haystack.dataclasses import ChatMessage
generator = AmazonBedrockChatGenerator(model="global.anthropic.claude-sonnet-4-6")
messages = [
ChatMessage.from_system(
"You are a helpful assistant that answers question in Spanish only",
),
ChatMessage.from_user("What's Natural Language Processing? Be brief."),
]
response = generator.run(messages)
print(response)
```
With multimodal inputs:
```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack_integrations.components.generators.amazon_bedrock import (
AmazonBedrockChatGenerator,
)
llm = AmazonBedrockChatGenerator(model="global.anthropic.claude-sonnet-4-6")
image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(
content_parts=["What does the image show? Max 5 words.", image],
)
response = llm.run([user_message])["replies"][0].text
print(response)
# Red apple on straw mat.
```
### In a pipeline
In a RAG pipeline:
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.amazon_bedrock import (
AmazonBedrockChatGenerator,
)
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component(
"llm", AmazonBedrockChatGenerator(model="global.anthropic.claude-sonnet-4-6")
)
pipe.connect("prompt_builder", "llm")
country = "Germany"
system_message = ChatMessage.from_system(
"You are an assistant giving out valuable information to language learners.",
)
messages = [
system_message,
ChatMessage.from_user("What's the official language of {{ country }}?"),
]
res = pipe.run(
data={
"prompt_builder": {
"template_variables": {"country": country},
"template": messages,
},
},
)
print(res)
```
---
// File: pipeline-components/generators/amazonbedrockgenerator
# AmazonBedrockGenerator
This component enables text generation using models through Amazon Bedrock service.
:::warning[Deprecation Notice]
`AmazonBedrockGenerator` is deprecated and will be removed in a future version.
We recommend switching to [AmazonBedrockChatGenerator](amazonbedrockchatgenerator.mdx) instead,
which also accepts a plain string as input.
:::
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.
`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) instances | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `model`: The model to use
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.
`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. | | **Mandatory run variables** | `prompt`: The instructions for the Generator | | **Output variables** | `replies`: A list of strings with all the replies generated by the model | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
[Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) is a fully managed service that makes high-performing foundation models from leading AI startups and Amazon available through a unified API. You can choose from various foundation models to find the one best suited for your use case.
`AmazonBedrockGenerator` enables text generation using models from AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon with a single component.
The models that we currently support are Anthropic's Claude, AI21 Labs' Jurassic-2, Stability AI's Stable Diffusion, Cohere's Command and Embed, Meta's Llama 2, and the Amazon Titan language and embeddings models.
## Overview
This component uses AWS for authentication. You can use the AWS CLI to authenticate through your IAM. For more information on setting up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
:::info[Using AWS CLI]
Consider using AWS CLI as a more straightforward tool to manage your AWS services. With AWS CLI, you can quickly configure your [boto3 credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). This way, you won't need to provide detailed authentication parameters when initializing Amazon Bedrock Generator in Haystack.
:::
To use this component for text generation, initialize an AmazonBedrockGenerator with the model name, the AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`) should be set as environment variables, be configured as described above or passed as [Secret](../../concepts/secret-management.mdx) arguments. Note, make sure the region you set supports Amazon Bedrock.
To start using Amazon Bedrock with Haystack, install the `amazon-bedrock-haystack` package:
```shell
pip install amazon-bedrock-haystack
```
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
### On its own
Basic usage:
```python
from haystack_integrations.components.generators.amazon_bedrock import (
AmazonBedrockGenerator,
)
aws_access_key_id = "..."
aws_secret_access_key = "..."
aws_region_name = "eu-central-1"
generator = AmazonBedrockGenerator(model="anthropic.claude-v2")
result = generator.run("Who is the best American actor?")
for reply in result["replies"]:
print(reply)
# >>> 'There is no definitive "best" American actor, as acting skill and talent a# re subjective. However, some of the most acclaimed and influential American act# ors include Tom Hanks, Daniel Day-Lewis, Denzel Washington, Meryl Streep, Rober# t De Niro, Al Pacino, Marlon Brando, Jack Nicholson, Leonardo DiCaprio and John# ny Depp. Choosing a single "best" actor comes down to personal preference.'
```
### In a pipeline
In a RAG pipeline:
```python
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.builders import PromptBuilder
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack import Pipeline
from haystack_integrations.components.generators.amazon_bedrock import (
AmazonBedrockGenerator,
)
template = """
Given the following information, answer the question.
Context:
{% for document in documents %}
{{ document.content }}
{% endfor %}
Question: What's the official language of {{ country }}?
"""
aws_access_key_id = "..."
aws_secret_access_key = "..."
aws_region_name = "eu-central-1"
generator = AmazonBedrockGenerator(model="anthropic.claude-v2")
docstore = InMemoryDocumentStore()
pipe = Pipeline()
pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("generator", generator)
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "generator")
pipe.run({"retriever": {"query": "France"}, "prompt_builder": {"country": "France"}})
# {'generator': {'replies': ['Based on the context provided, the official language of France is French.']}}
```
## Additional References
🧑🍳 Cookbook: [PDF-Based Question Answering with Amazon Bedrock and Haystack](https://haystack.deepset.ai/cookbook/amazon_bedrock_for_documentation_qa)
---
// File: pipeline-components/generators/anthropicchatgenerator
# AnthropicChatGenerator
This component enables chat completions using Anthropic large language models (LLMs).
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.
`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. | | **Mandatory run variables** | `prompt`: The instructions for the Generator | | **Output variables** | `replies`: A list of strings with all the replies generated by the model | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock | | **Package name** | `amazon-bedrock-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: An Anthropic API key. Can be set with `ANTHROPIC_API_KEY` env var. |
| **Mandatory run variables** | `messages` A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [Anthropic](/reference/integrations-anthropic) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic |
| **Package name** | `anthropic-haystack` |
## Overview
This integration supports Anthropic `chat` models such as `claude-3-5-sonnet-20240620`,`claude-3-opus-20240229`, `claude-3-haiku-20240307`, and similar. Check out the most recent full list in [Anthropic documentation](https://docs.anthropic.com/en/docs/about-claude/models).
### Parameters
`AnthropicChatGenerator` needs an Anthropic API key to work. You can provide this key in:
- The `ANTHROPIC_API_KEY` environment variable (recommended)
- The `api_key` init parameter and Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`
Set your preferred Anthropic model with the `model` parameter when initializing the component.
`AnthropicChatGenerator` requires a prompt to generate text, but you can pass any text generation parameters available in the Anthropic [Messaging API](https://docs.anthropic.com/en/api/messages) method directly to this component using the `generation_kwargs` parameter, both at initialization and when running the component. For more details on the parameters supported by the Anthropic API, see the [Anthropic documentation](https://docs.anthropic.com).
Finally, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
Both text and image input modalities are supported.
### Tool Support
`AnthropicChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = AnthropicChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
You can stream output as it’s generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
```python
from haystack.components.generators.utils import print_streaming_chunk
# Configure any `Generator` or `ChatGenerator` with a streaming callback
component = SomeGeneratorOrChatGenerator(streaming_callback=print_streaming_chunk)
# If this is a `ChatGenerator`, pass a list of messages:
# from haystack.dataclasses import ChatMessage
# component.run([ChatMessage.from_user("Your question here")])
# If this is a (non-chat) `Generator`, pass a prompt:
# component.run({"prompt": "Your prompt here"})
```
:::info
Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
:::
See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
### Prompt caching
Prompt caching is a feature for Anthropic LLMs that stores large text inputs for reuse. It allows you to send a large text block once and then refer to it in later requests without resending the entire text.
This feature is particularly useful for coding assistants that need full codebase context and for processing large documents. It can help reduce costs and improve response times.
Here's an example of an instance of `AnthropicChatGenerator` being initialized with prompt caching and tagging a message to be cached:
```python python
from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
generation_kwargs = {"extra_headers": {"anthropic-beta": "prompt-caching-2024-07-31"}}
claude_llm = AnthropicChatGenerator(
api_key=Secret.from_env_var("ANTHROPIC_API_KEY"),
generation_kwargs=generation_kwargs,
)
system_message = ChatMessage.from_system(
"Replace with some long text documents, code or instructions"
)
system_message.meta["cache_control"] = {"type": "ephemeral"}
messages = [
system_message,
ChatMessage.from_user("A query about the long text for example"),
]
result = claude_llm.run(messages)
# and now invoke again with
messages = [
system_message,
ChatMessage.from_user("Another query about the long text etc"),
]
result = claude_llm.run(messages)
# and so on, either invoking component directly or in the pipeline
```
For more details, refer to Anthropic's [documentation](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) and integration [examples](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic/example).
## Usage
Install the`anthropic-haystack` package to use the `AnthropicChatGenerator`:
```shell
pip install anthropic-haystack
```
### On its own
```python
from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
from haystack.dataclasses import ChatMessage
generator = AnthropicChatGenerator()
message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
print(generator.run([message]))
```
With multimodal inputs:
```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
llm = AnthropicChatGenerator()
image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(
content_parts=["What does the image show? Max 5 words.", image],
)
response = llm.run([user_message])["replies"][0].text
print(response)
# Red apple on straw.
```
### In a pipeline
You can also use `AnthropicChatGenerator`with the Anthropic chat models in your pipeline.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
from haystack.utils import Secret
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component(
"llm",
AnthropicChatGenerator(Secret.from_env_var("ANTHROPIC_API_KEY")),
)
pipe.connect("prompt_builder", "llm")
country = "Germany"
system_message = ChatMessage.from_system(
"You are an assistant giving out valuable information to language learners.",
)
messages = [
system_message,
ChatMessage.from_user("What's the official language of {{ country }}?"),
]
res = pipe.run(
data={
"prompt_builder": {
"template_variables": {"country": country},
"template": messages,
},
},
)
print(res)
```
## Additional References
🧑🍳 Cookbook: [Advanced Prompt Customization for Anthropic](https://haystack.deepset.ai/cookbook/prompt_customization_for_anthropic)
---
// File: pipeline-components/generators/anthropicfoundrychatgenerator
# AnthropicFoundryChatGenerator
This component enables chat completions using Anthropic models served through Azure Foundry.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: Your Azure Foundry API key. Can be set with the `ANTHROPIC_FOUNDRY_API_KEY` env var. Alternatively, pass an `azure_ad_token_provider` callable.
`resource`: Your Azure Foundry resource name. Can be set with the `ANTHROPIC_FOUNDRY_RESOURCE` env var. Alternatively, pass a full `endpoint` URL. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **API reference** | [Anthropic](/reference/integrations-anthropic) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic | | **Package name** | `anthropic-haystack` |
## Overview
`AnthropicFoundryChatGenerator` lets you call Anthropic's Claude models through an [Azure Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/) deployment. It is a thin subclass of [`AnthropicChatGenerator`](anthropicchatgenerator.mdx) — the request and response shapes match the Anthropic Messages API, but the traffic flows through your Azure resource instead of `api.anthropic.com`.
Use this generator when your organization standardizes on Azure for model hosting (billing, networking, compliance) but still wants to work against Claude. If you don't need Azure, prefer `AnthropicChatGenerator`.
The default model is `claude-sonnet-4-5`. Other models known to work include `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-opus-4-5`, `claude-opus-4-1`, and `claude-haiku-4-5`. This list is not exhaustive — the actual catalog depends on what is deployed in your Foundry resource. See the [Anthropic model overview](https://docs.anthropic.com/en/docs/about-claude/models) for guidance on picking a model.
### Parameters
`AnthropicFoundryChatGenerator` needs two things to talk to Azure: credentials and an endpoint.
**Credentials.** Pick one of:
- The `ANTHROPIC_FOUNDRY_API_KEY` environment variable (recommended).
- The `api_key` init parameter using the Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`.
- A callable passed as `azure_ad_token_provider` that returns a fresh Azure AD token on demand. Use this for Entra ID / managed-identity setups where a static key isn't appropriate.
**Endpoint.** Pick one of:
- The `resource` init parameter (or the `ANTHROPIC_FOUNDRY_RESOURCE` environment variable) — the short Foundry resource name, used to derive the URL.
- The `endpoint` init parameter — a full URL, useful for custom domains or non-standard routes.
Once configured, pass any text-generation parameter supported by the Anthropic [Messages API](https://docs.anthropic.com/en/api/messages) through `generation_kwargs`, either at init or per call. Common keys include `system`, `max_tokens`, `temperature`, `top_p`, `top_k`, `stop_sequences`, `metadata`, and `extra_headers`. You can also tune `timeout` and `max_retries` to control client-side resilience.
The component takes a list of `ChatMessage` objects. `ChatMessage` is a data class that holds a message, a role (`user`, `assistant`, `system`, or `tool`), and optional metadata. Only text input is supported.
### Tool Support
`AnthropicFoundryChatGenerator` supports function calling through the `tools` parameter, which accepts:
- **A list of Tool objects**: Pass individual tools as a list.
- **A single Toolset**: Pass an entire Toolset directly.
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.anthropic import AnthropicFoundryChatGenerator
weather_tool = Tool(name="weather", description="Get weather info", ...)
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
generator = AnthropicFoundryChatGenerator(
resource="my-resource",
tools=[math_toolset, weather_tool],
)
```
Tools passed to `run()` override any tools set at init time. For more details, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
You can stream output as it's generated. Pass a callback to `streaming_callback`. The built-in `print_streaming_chunk` prints text tokens and tool events to stdout.
```python
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.anthropic import (
AnthropicFoundryChatGenerator,
)
generator = AnthropicFoundryChatGenerator(
resource="my-resource",
streaming_callback=print_streaming_chunk,
)
generator.run([ChatMessage.from_user("Your question here")])
```
:::info
Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
:::
See [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) for how `StreamingChunk` works and how to write a custom callback. Prefer `print_streaming_chunk` unless you need a specific transport (such as SSE or WebSocket) or custom UI formatting.
### Async
`run_async` mirrors `run` and is wired up automatically — useful inside an async pipeline or web handler.
```python
import asyncio
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.anthropic import (
AnthropicFoundryChatGenerator,
)
async def main():
generator = AnthropicFoundryChatGenerator(resource="my-resource")
result = await generator.run_async([ChatMessage.from_user("Hello!")])
print(result["replies"][0].text)
asyncio.run(main())
```
## Usage
Install the `anthropic-haystack` package to use the `AnthropicFoundryChatGenerator`:
```shell
pip install anthropic-haystack
```
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
from haystack_integrations.components.generators.anthropic import (
AnthropicFoundryChatGenerator,
)
generator = AnthropicFoundryChatGenerator(
model="claude-sonnet-4-5",
api_key=Secret.from_env_var("ANTHROPIC_FOUNDRY_API_KEY"),
resource="my-resource",
)
response = generator.run([ChatMessage.from_user("What's Natural Language Processing?")])
print(response)
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.anthropic import (
AnthropicFoundryChatGenerator,
)
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component(
"llm",
AnthropicFoundryChatGenerator(resource="my-resource"),
)
pipe.connect("prompt_builder", "llm")
country = "Germany"
messages = [
ChatMessage.from_system(
"You are an assistant giving out valuable information to language learners.",
),
ChatMessage.from_user("What's the official language of {{ country }}?"),
]
res = pipe.run(
data={
"prompt_builder": {
"template_variables": {"country": country},
"template": messages,
},
},
)
print(res)
```
---
// File: pipeline-components/generators/anthropicgenerator
# AnthropicGenerator
This component enables text completions using Anthropic large language models (LLMs).
:::warning[Deprecation Notice]
`AnthropicGenerator` is deprecated and will be removed in a future version.
We recommend switching to [AnthropicChatGenerator](anthropicchatgenerator.mdx) instead,
which also accepts a plain string as input.
:::
`resource`: Your Azure Foundry resource name. Can be set with the `ANTHROPIC_FOUNDRY_RESOURCE` env var. Alternatively, pass a full `endpoint` URL. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **API reference** | [Anthropic](/reference/integrations-anthropic) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic | | **Package name** | `anthropic-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [PromptBuilder](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: An Anthropic API key. Can be set with `ANTHROPIC_API_KEY` env var. |
| **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM |
| **Output variables** | `replies`: A list of strings with all the replies generated by the LLM
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on | | **API reference** | [Anthropic](/reference/integrations-anthropic) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic | | **Package name** | `anthropic-haystack` |
## Overview
This integration supports Anthropic models such as `claude-3-5-sonnet-20240620`,`claude-3-opus-20240229`, `claude-3-haiku-20240307`, and similar. Although these LLMs are called chat models, the main prompt interface works with the string prompts. Check out the most recent full list in the [Anthropic documentation](https://docs.anthropic.com/en/docs/about-claude/models).
### Parameters
`AnthropicGenerator` needs an Anthropic API key to work. You can provide this key in:
- The `ANTHROPIC_API_KEY` environment variable (recommended)
- The `api_key` init parameter and Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`
Set your preferred Anthropic model in the `model` parameter when initializing the component.
`AnthropicGenerator` requires a prompt to generate text, but you can pass any text generation parameters available in the Anthropic [Messaging API](https://docs.anthropic.com/en/api/messages) method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the Anthropic API, see [Anthropic documentation](https://docs.anthropic.com).
Finally, the component run method requires a single string prompt to generate text.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
Install the `anthropic-haystack` package to use the `AnthropicGenerator`:
```shell
pip install anthropic-haystack
```
### On its own
```python
from haystack_integrations.components.generators.anthropic import AnthropicGenerator
generator = AnthropicGenerator()
print(generator.run("What's Natural Language Processing? Be brief."))
```
### In a pipeline
You can also use `AnthropicGenerator` with the Anthropic models in your pipeline.
```python
from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack_integrations.components.generators.anthropic import AnthropicGenerator
from haystack.utils import Secret
template = """
You are an assistant giving out valuable information to language learners.
Answer this question, be brief.
Question: {{ query }}?
"""
pipe = Pipeline()
pipe.add_component("prompt_builder", PromptBuilder(template))
pipe.add_component("llm", AnthropicGenerator(Secret.from_env_var("ANTHROPIC_API_KEY")))
pipe.connect("prompt_builder", "llm")
query = "What language is spoke in Germany?"
res = pipe.run(data={"prompt_builder": {"query": {query}}})
print(res)
```
---
// File: pipeline-components/generators/anthropicvertexchatgenerator
# AnthropicVertexChatGenerator
This component enables chat completions using AnthropicVertex API.
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on | | **API reference** | [Anthropic](/reference/integrations-anthropic) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic | | **Package name** | `anthropic-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `region`: The region where the Anthropic model is deployed
`project_id`: GCP project ID where the Anthropic model is deployed | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **API reference** | [Anthropic](/reference/integrations-anthropic) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic | | **Package name** | `anthropic-haystack` |
## Overview
`AnthropicVertexChatGenerator` enables text generation using Anthropic's Claude models through the Anthropic Vertex AI API.
A variety of Claude models (Opus, Sonnet, Haiku, and others) are accessible through the Vertex AI API endpoint. For more details about the models, refer to [Anthropic Vertex AI documentation](https://docs.anthropic.com/en/api/claude-on-vertex-ai).
### Parameters
To use the `AnthropicVertexChatGenerator`, ensure you have a GCP project with Vertex AI enabled. You need to pass your GCP `project_id` and `region` as init parameters. If you pass `None` for both, the component falls back to the `PROJECT_ID` and `REGION` environment variables.
Before making requests, you may need to authenticate with GCP using `gcloud auth login`.
Set your preferred supported Anthropic model with the `model` parameter when initializing the component. Additionally, ensure that the desired Anthropic model is activated in the Vertex AI Model Garden.
`AnthropicVertexChatGenerator` requires a prompt to generate text, but you can pass any text generation parameters available in the Anthropic [Messaging API](https://docs.anthropic.com/en/api/messages) method directly to this component using the `generation_kwargs` parameter, both at initialization and when running the component. For more details on the parameters supported by the Anthropic API, see the [Anthropic documentation](https://docs.anthropic.com/).
Finally, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
Only text input modality is supported at this time.
### Streaming
You can stream output as it’s generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
```python
from haystack.components.generators.utils import print_streaming_chunk
# Configure any `Generator` or `ChatGenerator` with a streaming callback
component = SomeGeneratorOrChatGenerator(streaming_callback=print_streaming_chunk)
# If this is a `ChatGenerator`, pass a list of messages:
# from haystack.dataclasses import ChatMessage
# component.run([ChatMessage.from_user("Your question here")])
# If this is a (non-chat) `Generator`, pass a prompt:
# component.run({"prompt": "Your prompt here"})
```
:::info
Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
:::
See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
### Prompt Caching
Prompt caching is a feature for Anthropic LLMs that stores large text inputs for reuse. It allows you to send a large text block once and then refer to it in later requests without resending the entire text.
This feature is particularly useful for coding assistants that need full codebase context and for processing large documents. It can help reduce costs and improve response times.
Here's an example of an instance of `AnthropicVertexChatGenerator` being initialized with prompt caching and tagging a message to be cached:
```python
from haystack_integrations.components.generators.anthropic import (
AnthropicVertexChatGenerator,
)
from haystack.dataclasses import ChatMessage
generation_kwargs = {"extra_headers": {"anthropic-beta": "prompt-caching-2024-07-31"}}
claude_llm = AnthropicVertexChatGenerator(
region="your_region",
project_id="test_id",
generation_kwargs=generation_kwargs,
)
system_message = ChatMessage.from_system(
"Replace with some long text documents, code or instructions",
)
system_message.meta["cache_control"] = {"type": "ephemeral"}
messages = [
system_message,
ChatMessage.from_user("A query about the long text for example"),
]
result = claude_llm.run(messages)
# and now invoke again with
messages = [
system_message,
ChatMessage.from_user("Another query about the long text etc"),
]
result = claude_llm.run(messages)
# and so on, either invoking component directly or in the pipeline
```
For more details, refer to Anthropic's [documentation](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) and integration [examples](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic/example).
## Usage
Install the`anthropic-haystack` package to use the `AnthropicVertexChatGenerator`:
```shell
pip install anthropic-haystack
```
### On its own
```python
from haystack_integrations.components.generators.anthropic import (
AnthropicVertexChatGenerator,
)
from haystack.dataclasses import ChatMessage
messages = [ChatMessage.from_user("What's Natural Language Processing?")]
client = AnthropicVertexChatGenerator(
model="claude-sonnet-4@20250514",
project_id="your-project-id",
region="us-central1",
)
response = client.run(messages)
print(response)
```
### In a pipeline
You can also use `AnthropicVertexChatGenerator`with the Anthropic chat models in your pipeline.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.anthropic import (
AnthropicVertexChatGenerator,
)
from haystack.utils import Secret
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component(
"llm",
AnthropicVertexChatGenerator(project_id="test_id", region="us-central1"),
)
pipe.connect("prompt_builder", "llm")
country = "Germany"
system_message = ChatMessage.from_system(
"You are an assistant giving out valuable information to language learners.",
)
messages = [
system_message,
ChatMessage.from_user("What's the official language of {{ country }}?"),
]
res = pipe.run(
data={
"prompt_builder": {
"template_variables": {"country": country},
"template": messages,
},
},
)
print(res)
```
---
// File: pipeline-components/generators/azureopenaichatgenerator
# AzureOpenAIChatGenerator
This component enables chat completion using OpenAI’s large language models (LLMs) through Azure services.
`project_id`: GCP project ID where the Anthropic model is deployed | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **API reference** | [Anthropic](/reference/integrations-anthropic) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic | | **Package name** | `anthropic-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: The Azure OpenAI API key. Can be set with `AZURE_OPENAI_API_KEY` env var.
`azure_ad_token`: Microsoft Entra ID token. Can be set with `AZURE_OPENAI_AD_TOKEN` env var. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string | | **Output variables** | `replies`: A list of alternative replies of the LLM to the input chat | | **API reference** | [Generators](/reference/generators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/azure.py | | **Package name** | `haystack-ai` |
## Overview
`AzureOpenAIChatGenerator` supports OpenAI models deployed through Azure services. To see the list of supported models, head over to Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?source=recommendations). The default model used with the component is `gpt-4.1-mini`.
To work with Azure components, you will need an Azure OpenAI API key, as well as an Azure OpenAI Endpoint. You can learn more about them in Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
The component uses `AZURE_OPENAI_API_KEY` and `AZURE_OPENAI_AD_TOKEN` environment variables by default. Otherwise, you can pass `api_key` and `azure_ad_token` at initialization:
```python
client = AzureOpenAIChatGenerator(
azure_endpoint="`azure_ad_token`: Microsoft Entra ID token. Can be set with `AZURE_OPENAI_AD_TOKEN` env var. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string | | **Output variables** | `replies`: A list of alternative replies of the LLM to the input chat | | **API reference** | [Generators](/reference/generators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/azure.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: The Azure OpenAI API key. Can be set with `AZURE_OPENAI_API_KEY` env var or a callable for Azure AD token.
`azure_endpoint`: The endpoint of the deployed model. Can be set with `AZURE_OPENAI_ENDPOINT` env var. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects containing the generated responses | | **API reference** | [Generators](/reference/generators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/azure_responses.py | | **Package name** | `haystack-ai` |
## Overview
`AzureOpenAIResponsesChatGenerator` uses OpenAI's Responses API through Azure OpenAI services. It supports gpt-5 and o-series models (reasoning models like o1, o3-mini) deployed on Azure. The default model is `gpt-5-mini`.
The Responses API is designed for reasoning-capable models and supports features like reasoning summaries, multi-turn conversations with previous response IDs, and structured outputs. This component provides access to these capabilities through Azure's infrastructure.
The component requires a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`), and optional metadata. If a string is passed, it is converted into a list containing a single `ChatMessage` with the `user` role. See the [usage](#usage) section for examples.
You can pass any parameters valid for the OpenAI Responses API directly to `AzureOpenAIResponsesChatGenerator` using the `generation_kwargs` parameter, both at initialization and to the `run()` method. For more details on the supported parameters, refer to the [Azure OpenAI documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
You can specify a model for this component through the `azure_deployment` init parameter, which should match your Azure deployment name.
### Authentication
To work with Azure components, you need an Azure OpenAI API key and an Azure OpenAI endpoint. You can learn more about them in the [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
The component uses `AZURE_OPENAI_API_KEY` and `AZURE_OPENAI_ENDPOINT` environment variables by default. Otherwise, you can pass these at initialization using a [`Secret`](../../concepts/secret-management.mdx):
```python
from haystack.components.generators.chat import AzureOpenAIResponsesChatGenerator
from haystack.utils import Secret
client = AzureOpenAIResponsesChatGenerator(
azure_endpoint="https://your-resource.azure.openai.com/",
api_key=Secret.from_token("`azure_endpoint`: The endpoint of the deployed model. Can be set with `AZURE_OPENAI_ENDPOINT` env var. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects containing the generated responses | | **API reference** | [Generators](/reference/generators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/azure_responses.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
| **Mandatory run variables** | `messages` A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [Cohere](/reference/integrations-cohere) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere |
| **Package name** | `cohere-haystack` |
This integration supports Cohere `chat` models such as `command-a-03-2025` (the default), `command-a-plus-05-2026`, and `command-r-plus-08-2024`. Check out the most recent full list in [Cohere documentation](https://docs.cohere.com/reference/chat).
## Overview
`CohereChatGenerator` needs a Cohere API key to work. You can set this key in:
- The `api_key` init parameter using [Secret API](../../concepts/secret-management.mdx)
- The `COHERE_API_KEY` environment variable (recommended)
Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `Co.chat` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the Cohere API, refer to the [Cohere documentation](https://docs.cohere.com/reference/chat).
Finally, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
### Tool Support
`CohereChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.cohere import CohereChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = CohereChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
You need to install `cohere-haystack` package to use the `CohereChatGenerator`:
```shell
pip install cohere-haystack
```
#### On its own
```python
from haystack_integrations.components.generators.cohere import CohereChatGenerator
from haystack.dataclasses import ChatMessage
generator = CohereChatGenerator()
message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
print(generator.run([message]))
```
With multimodal inputs:
```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack_integrations.components.generators.cohere import CohereChatGenerator
# Use a multimodal model like Command A Vision
llm = CohereChatGenerator(model="command-a-vision-07-2025")
image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(
content_parts=["What does the image show? Max 5 words.", image],
)
response = llm.run([user_message])["replies"][0].text
print(response)
# Red apple on straw.
```
#### In a Pipeline
You can also use `CohereChatGenerator` to use cohere chat models in your pipeline.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.cohere import CohereChatGenerator
from haystack.utils import Secret
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component("llm", CohereChatGenerator())
pipe.connect("prompt_builder", "llm")
country = "Germany"
system_message = ChatMessage.from_system(
"You are an assistant giving out valuable information to language learners.",
)
messages = [
system_message,
ChatMessage.from_user("What's the official language of {{ country }}?"),
]
res = pipe.run(
data={
"prompt_builder": {
"template_variables": {"country": country},
"template": messages,
},
},
)
print(res)
```
---
// File: pipeline-components/generators/coheregenerator
# CohereGenerator
`CohereGenerator` enables text generation using Cohere's large language models (LLMs).
:::warning[Deprecation Notice]
`CohereGenerator` is deprecated and will be removed in a future version.
We recommend switching to [CohereChatGenerator](coherechatgenerator.mdx) instead,
which also accepts a plain string as input.
:::
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
| **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM |
| **Output variables** | `replies`: A list of strings with all the replies generated by the LLM
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on | | **API reference** | [Cohere](/reference/integrations-cohere) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere | | **Package name** | `cohere-haystack` |
This integration supports Cohere models such as `command`, `command-r` and `comman-r-plus`. Check out the most recent full list in [Cohere documentation](https://docs.cohere.com/reference/chat).
## Overview
`CohereGenerator` needs a Cohere API key to work. You can write this key in:
- The `api_key` init parameter using [Secret API](../../concepts/secret-management.mdx)
- The `COHERE_API_KEY` environment variable (recommended)
Then, the component needs a prompt to operate, but you can pass any text generation parameters directly to this component using the `generation_kwargs` parameter at initialization. For more details on the parameters supported by the Cohere API, refer to the [Cohere documentation](https://docs.cohere.com/reference/chat).
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
You need to install `cohere-haystack` package to use the `CohereGenerator`:
```shell
pip install cohere-haystack
```
### On its own
Basic usage:
```python
from haystack_integrations.components.generators.cohere import CohereGenerator
client = CohereGenerator()
response = client.run("Briefly explain what NLP is in one sentence.")
print(response)
>>> {'replies': ["Natural Language Processing (NLP) is a subfield of artificial intelligence and computational linguistics that focuses on the interaction between computers and human languages..."],
'meta': [{'finish_reason': 'COMPLETE'}]}
```
With streaming:
```python
from haystack_integrations.components.generators.cohere import CohereGenerator
client = CohereGenerator(streaming_callback=lambda chunk: print(chunk.content, end="", flush=True))
response = client.run("Briefly explain what NLP is in one sentence.")
print(response)
>>> Natural Language Processing (NLP) is the study of natural language and how it can be used to solve problems through computational methods, enabling machines to understand, interpret, and generate human language.
>>>{'replies': [' Natural Language Processing (NLP) is the study of natural language and how it can be used to solve problems through computational methods, enabling machines to understand, interpret, and generate human language.'], 'meta': [{'index': 0, 'finish_reason': 'COMPLETE'}]}
```
### In a pipeline
In a RAG pipeline:
```python
from haystack import Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.builders.prompt_builder import PromptBuilder
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.generators.cohere import CohereGenerator
from haystack import Document
docstore = InMemoryDocumentStore()
docstore.write_documents(
[
Document(content="Rome is the capital of Italy"),
Document(content="Paris is the capital of France"),
],
)
query = "What is the capital of France?"
template = """
Given the following information, answer the question.
Context:
{% for document in documents %}
{{ document.content }}
{% endfor %}
Question: {{ query }}?
"""
pipe = Pipeline()
pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("llm", CohereGenerator())
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "llm")
res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
print(res)
```
---
// File: pipeline-components/generators/cometapichatgenerator
# CometAPIChatGenerator
CometAPIChatGenerator enables chat completion using AI models through the Comet API.
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on | | **API reference** | [Cohere](/reference/integrations-cohere) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere | | **Package name** | `cohere-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: The Comet API key. Can be set with `COMET_API_KEY` env var. |
| **Mandatory run variables** | `messages` A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [Comet API](/reference/integrations-cometapi) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cometapi |
| **Package name** | `cometapi-haystack` |
## Overview
`CometAPIChatGenerator` provides access to over 500 AI models through the Comet API, a unified API gateway for models from providers like OpenAI, Anthropic, Google, xAI, DeepSeek, and many more. You can use different models from different providers within a single pipeline with a consistent interface.
Comet API uses a single API key for all providers, which allows you to switch between or combine different models without managing multiple credentials.
The range of models supported by Comet API include:
- OpenAI models: `gpt-5-mini` (default), `gpt-4o`, `gpt-4o-mini`, and more
- Anthropic models: `claude-sonnet-4-5`, `claude-opus-4-5-20251101`, and more
- Google models: `gemini-2.5-pro`, `gemini-2.5-flash`, and more
- xAI models: `grok-4.3`, and more
- DeepSeek models: `deepseek-chat`, and more
For a complete list of available models, check the [Comet API documentation](https://apidoc.cometapi.com/).
The component needs a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
You can pass any chat completion parameters valid for the underlying model directly to `CometAPIChatGenerator` using the `generation_kwargs` parameter, both at initialization and to the `run()` method.
### Authentication
`CometAPIChatGenerator` needs a Comet API key to work. You can set this key in:
- The `api_key` init parameter using [Secret API](../../concepts/secret-management.mdx)
- The `COMET_API_KEY` environment variable (recommended)
### Structured Output
`CometAPIChatGenerator` supports structured output generation for compatible models, allowing you to receive responses in a predictable format. You can use Pydantic models or JSON schemas to define the structure of the output through the `response_format` parameter in `generation_kwargs`.
This is useful when you need to extract structured data from text or generate responses that match a specific format.
```python
from pydantic import BaseModel
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.cometapi import CometAPIChatGenerator
class CityInfo(BaseModel):
city_name: str
country: str
population: int
famous_for: str
client = CometAPIChatGenerator(
model="gpt-4o-2024-08-06",
generation_kwargs={"response_format": CityInfo}
)
response = client.run(messages=[
ChatMessage.from_user(
"Berlin is the capital and largest city of Germany with a population of "
"approximately 3.7 million. It's famous for its history, culture, and nightlife."
)
])
print(response["replies"][0].text)
>> {"city_name":"Berlin","country":"Germany","population":3700000,
>> "famous_for":"history, culture, and nightlife"}
```
:::info[Model Compatibility]
Structured output support depends on the underlying model. OpenAI models starting from `gpt-4o-2024-08-06` support Pydantic models and JSON schemas. For details on which models support this feature, refer to the respective model provider's documentation.
:::
### Tool Support
`CometAPIChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.cometapi import CometAPIChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = CometAPIChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
`CometAPIChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
You can stream output as it's generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
```python
from haystack.components.generators.utils import print_streaming_chunk
from haystack_integrations.components.generators.cometapi import CometAPIChatGenerator
# Configure the generator with a streaming callback
component = CometAPIChatGenerator(streaming_callback=print_streaming_chunk)
# Pass a list of messages
from haystack.dataclasses import ChatMessage
component.run([ChatMessage.from_user("Your question here")])
```
:::info
Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
:::
See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
We recommend to give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
## Usage
Install the `cometapi-haystack` package to use the `CometAPIChatGenerator`:
```shell
pip install cometapi-haystack
```
### On its own
```python
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.cometapi import CometAPIChatGenerator
client = CometAPIChatGenerator(model="gpt-4o-mini", streaming_callback=print_streaming_chunk)
response = client.run([ChatMessage.from_user("What's Natural Language Processing? Be brief.")])
>> Natural Language Processing (NLP) is a field of artificial intelligence that
>> focuses on the interaction between computers and humans through natural language.
>> It involves enabling machines to understand, interpret, and generate human
>> language in a meaningful way, facilitating tasks such as language translation,
>> sentiment analysis, and text summarization.
print(response)
>> {'replies': [ChatMessage(_role=
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: The Eden AI API key. Can be set with `EDENAI_API_KEY` env var. |
| **Mandatory run variables** | `messages` A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [Eden AI](/reference/integrations-edenai) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/edenai |
| **Package name** | `edenai-haystack` |
## Overview
`EdenAIChatGenerator` connects Haystack to [Eden AI](https://www.edenai.co/), a unified, OpenAI-compatible API that gives access to 500+ models from many providers (OpenAI, Anthropic, Mistral, Google, Cohere, and more) through a single API key, with EU data residency.
`EdenAIChatGenerator` needs an Eden AI API key to work. You can write this key in:
- The `api_key` init parameter using [Secret API](../../concepts/secret-management.mdx)
- The `EDENAI_API_KEY` environment variable (recommended)
Models are selected using Eden AI's `provider/model` naming convention, for example:
- `openai/gpt-4o-mini` (default)
- `anthropic/claude-sonnet-4-5`
- `mistral/mistral-large-latest`
- `google/gemini-2.5-flash`
For the full list of available models, see the [Eden AI models catalog](https://www.edenai.co/models).
This component needs a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
Refer to the [Eden AI documentation](https://docs.edenai.co/) for more details on the parameters supported by the API, which you can provide with `generation_kwargs` when running the component.
### Tool Support
`EdenAIChatGenerator` supports function calling through the `tools` parameter, which accepts a list of `Tool` objects, a single `Toolset`, or a mix of both. This lets you organize related tools into logical groups while also including standalone tools as needed.
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
Install the `edenai-haystack` package to use the `EdenAIChatGenerator`:
```shell
pip install edenai-haystack
```
#### On its own
```python
from haystack_integrations.components.generators.edenai import EdenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
generator = EdenAIChatGenerator(
api_key=Secret.from_env_var("EDENAI_API_KEY"),
model="mistral/mistral-large-latest",
streaming_callback=print_streaming_chunk,
)
message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
print(generator.run([message]))
```
#### In a Pipeline
Below is an example RAG Pipeline where we answer questions based on the contents of a URL. We add the contents of the URL into our `messages` in the `ChatPromptBuilder` and generate an answer with the `EdenAIChatGenerator`.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.edenai import EdenAIChatGenerator
fetcher = LinkContentFetcher()
converter = HTMLToDocument()
prompt_builder = ChatPromptBuilder(variables=["documents"])
llm = EdenAIChatGenerator(model="mistral/mistral-large-latest")
message_template = """Answer the following question based on the contents of the article: {{query}}\n
Article: {{documents[0].content}} \n
"""
messages = [ChatMessage.from_user(message_template)]
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="fetcher", instance=fetcher)
rag_pipeline.add_component(name="converter", instance=converter)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)
rag_pipeline.connect("fetcher.streams", "converter.sources")
rag_pipeline.connect("converter.documents", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
question = "What is Eden AI?"
result = rag_pipeline.run(
{
"fetcher": {"urls": ["https://www.edenai.co/"]},
"prompt_builder": {
"template_variables": {"query": question},
"template": messages,
},
},
)
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/generators/external-integrations-generators
# External Integrations
External integrations that enable RAG pipeline creation.
| Name | Description |
| --- | --- |
| [DeepL](https://haystack.deepset.ai/integrations/deepl) | Translate your text and documents using DeepL services. |
| [fastRAG](https://haystack.deepset.ai/integrations/fastrag/) | Enables the creation of efficient and optimized retrieval augmented generative pipelines. |
| [LM Format Enforcer](https://haystack.deepset.ai/integrations/lmformatenforcer) | Enforce JSON Schema / Regex output of your local models with `LMFormatEnforcerLocalGenerator`. |
| [Titan](https://haystack.deepset.ai/integrations/titanml-takeoff) | Run local open-source LLMs from Meta, Mistral and Alphabet directly in your computer. |
---
// File: pipeline-components/generators/fallbackchatgenerator
# FallbackChatGenerator
A ChatGenerator wrapper that tries multiple Chat Generators sequentially until one succeeds.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `chat_generators`: A non-empty list of Chat Generator components to try in order |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string |
| **Output variables** | `replies`: Generated ChatMessage instances from the first successful generator
`meta`: Execution metadata including successful generator details | | **API reference** | [Generators](/reference/generators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/fallback.py | | **Package name** | `haystack-ai` |
## Overview
`FallbackChatGenerator` is a wrapper component that tries multiple Chat Generators sequentially until one succeeds. If a Generator fails, the component tries the next one in the list. This handles provider outages, rate limits, and other transient failures.
The component forwards all parameters to the underlying Chat Generators and returns the first successful result. When a Generator raises any exception, the component tries the next Generator. This includes timeout errors, rate limit errors (429), authentication errors (401), context length errors (400), server errors (500+), and any other exception.
The component returns execution metadata including which Generator succeeded, how many attempts were made, and which Generators failed. All parameters (`messages`, `generation_kwargs`, `tools`, `streaming_callback`) are forwarded to the underlying Generators. If a string is passed to `messages`, it is converted into a list containing a single `ChatMessage` with the `user` role before forwarding.
Timeout enforcement is delegated to the underlying Chat Generators. To control latency, configure your Chat Generators with a `timeout` parameter. Chat Generators like OpenAI, Anthropic, and Cohere support timeout parameters that raise exceptions when exceeded.
### Monitoring and Telemetry
The `meta` dictionary in the output contains useful information for monitoring:
```python
from haystack.components.generators.chat import (
FallbackChatGenerator,
OpenAIChatGenerator,
)
from haystack.dataclasses import ChatMessage
# Set up generators
primary = OpenAIChatGenerator(model="gpt-4o")
backup = OpenAIChatGenerator(model="gpt-4o-mini")
generator = FallbackChatGenerator(chat_generators=[primary, backup])
# Run and inspect metadata
result = generator.run(messages=[ChatMessage.from_user("Hello")])
meta = result["meta"]
print(
f"Successful generator index: {meta['successful_chat_generator_index']}",
) # 0 for first, 1 for second, etc.
print(
f"Successful generator class: {meta['successful_chat_generator_class']}",
) # e.g., "OpenAIChatGenerator"
print(
f"Total attempts made: {meta['total_attempts']}",
) # How many Generators were tried
print(
f"Failed generators: {meta['failed_chat_generators']}",
) # List of failed Generator names
```
You can use this metadata to:
- Track which Generators are being used most frequently
- Monitor failure rates for each Generator
- Set up alerts when fallbacks occur
- Adjust Generator ordering based on success rates
### Streaming
`FallbackChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) through the `streaming_callback` parameter. The callback is passed directly to the underlying Generators.
## Usage
### On its own
Basic usage with fallback from a primary to a backup model:
```python
from haystack.components.generators.chat import (
FallbackChatGenerator,
OpenAIChatGenerator,
)
from haystack.dataclasses import ChatMessage
# Create primary and backup generators
primary = OpenAIChatGenerator(model="gpt-4o", timeout=30)
backup = OpenAIChatGenerator(model="gpt-4o-mini", timeout=30)
# Wrap them in a FallbackChatGenerator
generator = FallbackChatGenerator(chat_generators=[primary, backup])
# Use it like any other Chat Generator
messages = [ChatMessage.from_user("What's Natural Language Processing? Be brief.")]
result = generator.run(messages=messages)
print(result["replies"][0].text)
print(f"Successful generator: {result['meta']['successful_chat_generator_class']}")
print(f"Total attempts: {result['meta']['total_attempts']}")
# Natural Language Processing (NLP) is a field of artificial intelligence that
# focuses on the interaction between computers and humans through natural language...
# Successful generator: OpenAIChatGenerator
# Total attempts: 1
```
With multiple providers:
```python
from haystack.components.generators.chat import (
FallbackChatGenerator,
OpenAIChatGenerator,
AzureOpenAIChatGenerator,
)
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
# Create generators from different providers
openai_gen = OpenAIChatGenerator(
model="gpt-4o-mini",
api_key=Secret.from_env_var("OPENAI_API_KEY"),
timeout=30,
)
azure_gen = AzureOpenAIChatGenerator(
azure_endpoint="`meta`: Execution metadata including successful generator details | | **API reference** | [Generators](/reference/generators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/fallback.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: A Google AI Studio API key. Can be set with `GOOGLE_API_KEY` env var. |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat |
| **Output variables** | `replies`: A list of alternative replies of the model to the input chat |
| **API reference** | [Google AI](/reference/integrations-google-ai) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_ai |
| **Package name** | `google-ai-haystack` |
`GoogleAIGeminiChatGenerator` supports `gemini-2.5-pro-exp-03-25`, `gemini-2.0-flash`, `gemini-1.5-pro`, and `gemini-1.5-flash` models.
For available models, see https://ai.google.dev/gemini-api/docs/models/gemini.
### Parameters Overview
`GoogleAIGeminiChatGenerator` uses a Google Studio API key for authentication. You can write this key in an `api_key` parameter or as a `GOOGLE_API_KEY` environment variable (recommended).
To get an API key, visit the [Google AI Studio](https://aistudio.google.com/) website.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
To begin working with `GoogleAIGeminiChatGenerator`, install the `google-ai-haystack` package:
```shell
pip install google-ai-haystack
```
### On its own
Basic usage:
```python
import os
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.google_ai import GoogleAIGeminiChatGenerator
os.environ["GOOGLE_API_KEY"] = "
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: A Google AI Studio API key. Can be set with `GOOGLE_API_KEY` env var. |
| **Mandatory run variables** | `parts`: A variadic list containing a mix of images, audio, video, and text to prompt Gemini |
| **Output variables** | `replies`: A list of strings or dictionaries with all the replies generated by the model |
| **API reference** | [Google AI](/reference/integrations-google-ai) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_ai |
| **Package name** | `google-ai-haystack` |
`GoogleAIGeminiGenerator` supports `gemini-2.5-pro-exp-03-25`, `gemini-2.0-flash`, `gemini-1.5-pro`, and `gemini-1.5-flash` models.
For available models, see https://ai.google.dev/gemini-api/docs/models/gemini.
### Parameters Overview
`GoogleAIGeminiGenerator` uses a Google AI Studio API key for authentication. You can write this key in an `api_key` parameter or as a `GOOGLE_API_KEY` environment variable (recommended).
To get an API key, visit the [Google AI Studio](https://ai.google.dev/gemini-api/docs/api-key) website.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
Start by installing the `google-ai-haystack` package to use the `GoogleAIGeminiGenerator`:
```shell
pip install google-ai-haystack
```
### On its own
Basic usage:
```python
import os
from haystack_integrations.components.generators.google_ai import GoogleAIGeminiGenerator
os.environ["GOOGLE_API_KEY"] = "
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: A Google API key. Can be set with `GOOGLE_API_KEY` env var. |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat |
| **Output variables** | `replies`: A list of alternative replies of the model to the input chat |
| **API reference** | [Google GenAI](/reference/integrations-google-genai) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_genai |
| **Package name** | `google-genai-haystack` |
## Overview
`GoogleGenAIChatGenerator` supports Gemini generative models, such as
`gemini-3.1-flash-lite-preview`, `gemini-3.1-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-pro`, and `gemini-2.5-flash-lite`.
### Tool Support
`GoogleGenAIChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = GoogleGenAIChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
### Authentication
Google Gen AI is compatible with both the Gemini Developer API and the Vertex AI API.
To use this component with the Gemini Developer API and get an API key, visit [Google AI Studio](https://aistudio.google.com/).
To use this component with the Vertex AI API, visit [Google Cloud > Vertex AI](https://cloud.google.com/vertex-ai).
The component uses a `GOOGLE_API_KEY` or `GEMINI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` static method:
```python
chat_generator = GoogleGenAIChatGenerator(api_key=Secret.from_token("
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_type`: The type of Hugging Face API to use
`api_params`: A dictionary with one of the following keys:
- `model`: Hugging Face model ID. Required when `api_type` is `SERVERLESS_INFERENCE_API`.**OR** - `url`: URL of the inference endpoint. Required when `api_type` is `INFERENCE_ENDPOINTS` or `TEXT_GENERATION_INFERENCE`. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string | | **Output variables** | `replies`: A list of replies of the LLM to the input chat | | **API reference** | [Hugging Face API](/reference/integrations-huggingface-api) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/huggingface_api | | **Package name** | `huggingface-api-haystack` |
## Overview
`HuggingFaceAPIChatGenerator` can be used to generate chat completions using different Hugging Face APIs:
- [Serverless Inference API (Inference Providers)](https://huggingface.co/docs/inference-providers) - free tier available
- [Paid Inference Endpoints](https://huggingface.co/inference-endpoints)
- [Self-hosted Text Generation Inference](https://github.com/huggingface/text-generation-inference)
This component's main input is a list of `ChatMessage` objects. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata. If a string is passed, it is converted into a list containing a single `ChatMessage` with the `user` role. For more information, check out our [`ChatMessage` docs](../../concepts/data-classes/chatmessage.mdx).
The component reads the `HF_API_TOKEN` or `HF_TOKEN` environment variable by default. Otherwise, you can pass a Hugging Face API token at initialization with `token` – see code examples below.
The token is needed:
- If you use the Serverless Inference API, or
- If you use the Inference Endpoints.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
Install the `huggingface-api-haystack` package to use the `HuggingFaceAPIChatGenerator`:
```shell
pip install huggingface-api-haystack
```
### On its own
#### Using Serverless Inference API (Inference Providers) - Free Tier Available
This API allows you to quickly experiment with many models hosted on the Hugging Face Hub, offloading the inference to Hugging Face servers. It's rate-limited and not meant for production.
To use this API, you need a [free Hugging Face token](https://huggingface.co/settings/tokens).
The Generator expects the `model` in `api_params`. It's also recommended to specify a `provider` for better performance and reliability.
```python
from haystack_integrations.components.generators.huggingface_api import (
HuggingFaceAPIChatGenerator,
)
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
from haystack_integrations.common.huggingface_api.utils import HFGenerationAPIType
messages = [
ChatMessage.from_system("\\nYou are a helpful, respectful and honest assistant"),
ChatMessage.from_user("What's Natural Language Processing?"),
]
# the api_type can be expressed using the HFGenerationAPIType enum or as a string
api_type = HFGenerationAPIType.SERVERLESS_INFERENCE_API
api_type = "serverless_inference_api" # this is equivalent to the above
generator = HuggingFaceAPIChatGenerator(
api_type=api_type,
api_params={"model": "Qwen/Qwen2.5-7B-Instruct", "provider": "together"},
token=Secret.from_env_var("HF_API_TOKEN"),
)
result = generator.run(messages)
print(result)
```
#### Using Paid Inference Endpoints
In this case, a private instance of the model is deployed by Hugging Face, and you typically pay per hour.
To understand how to spin up an Inference Endpoint, visit [Hugging Face documentation](https://huggingface.co/inference-endpoints/dedicated).
Additionally, in this case, you need to provide your Hugging Face token.
The Generator expects the `url` of your endpoint in `api_params`.
```python
from haystack_integrations.components.generators.huggingface_api import (
HuggingFaceAPIChatGenerator,
)
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
messages = [
ChatMessage.from_system("\\nYou are a helpful, respectful and honest assistant"),
ChatMessage.from_user("What's Natural Language Processing?"),
]
generator = HuggingFaceAPIChatGenerator(
api_type="inference_endpoints",
api_params={"url": "`api_params`: A dictionary with one of the following keys:
- `model`: Hugging Face model ID. Required when `api_type` is `SERVERLESS_INFERENCE_API`.**OR** - `url`: URL of the inference endpoint. Required when `api_type` is `INFERENCE_ENDPOINTS` or `TEXT_GENERATION_INFERENCE`. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string | | **Output variables** | `replies`: A list of replies of the LLM to the input chat | | **API reference** | [Hugging Face API](/reference/integrations-huggingface-api) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/huggingface_api | | **Package name** | `huggingface-api-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | None. The provider's API key is read by LiteLLM from its standard environment variable (for example, `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`). You can also pass it explicitly through the `api_key` init parameter. |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [LiteLLM](/reference/integrations-litellm) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/litellm |
| **Package name** | `litellm-haystack` |
## Overview
`LiteLLMChatGenerator` routes chat completions through [LiteLLM](https://docs.litellm.ai/), which exposes a single, unified interface to over 100 LLM providers, including OpenAI, Anthropic, Google, AWS Bedrock, Azure, Cohere, Mistral, and Groq. This lets you switch providers by changing only the `model` string, without rewriting your pipeline.
### Parameters
Model names use the LiteLLM `provider/model-name` format, for example `openai/gpt-4o`, `anthropic/claude-sonnet-4-20250514`, or `bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0`. The default model is `openai/gpt-4o`. See the [LiteLLM providers documentation](https://docs.litellm.ai/docs/providers) for the full list of supported providers and their model identifiers.
`LiteLLMChatGenerator` needs an API key for the selected provider. You can provide it in two ways:
- Let LiteLLM resolve credentials itself from the provider's standard environment variable, such as `OPENAI_API_KEY` or `ANTHROPIC_API_KEY` (recommended).
- Pass it explicitly through the `api_key` init parameter and Haystack's [Secret](../../concepts/secret-management.mdx) API: `Secret.from_env_var("OPENAI_API_KEY")`. Use this only when you want Haystack to manage and serialize the key.
If you run against a self-hosted LiteLLM proxy or a custom endpoint, set the `api_base_url` parameter.
You can pass any parameter supported by [`litellm.completion()`](https://docs.litellm.ai/docs/completion/input) through the `generation_kwargs` parameter, both at initialization and when running the component. LiteLLM normalizes these parameters across providers and drops the ones a given provider does not support.
Finally, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
### Tool Support
`LiteLLMChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
Tool calls work with both the synchronous and streaming responses, as long as the underlying provider and model support function calling. For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
You can stream output as it's generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
```python
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.litellm import LiteLLMChatGenerator
generator = LiteLLMChatGenerator(
model="openai/gpt-4o",
streaming_callback=print_streaming_chunk,
)
generator.run([ChatMessage.from_user("Your question here")])
```
See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
### Asynchronous Execution
`LiteLLMChatGenerator` provides a `run_async` method for use in asynchronous pipelines and applications. It accepts the same parameters as `run` and supports both regular and streaming responses (pass an async streaming callback when streaming).
## Usage
Install the `litellm-haystack` package to use the `LiteLLMChatGenerator`:
```shell
pip install litellm-haystack
```
### On its own
```python
from haystack_integrations.components.generators.litellm import LiteLLMChatGenerator
from haystack.dataclasses import ChatMessage
generator = LiteLLMChatGenerator(
model="anthropic/claude-sonnet-4-20250514",
generation_kwargs={"max_tokens": 1024, "temperature": 0.7},
)
messages = [
ChatMessage.from_system("You are a helpful assistant"),
ChatMessage.from_user("What's Natural Language Processing? Be brief."),
]
result = generator.run(messages=messages)
print(result["replies"][0].text)
```
### In a pipeline
You can also use `LiteLLMChatGenerator` in a pipeline together with a `ChatPromptBuilder`.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.litellm import LiteLLMChatGenerator
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component("llm", LiteLLMChatGenerator(model="openai/gpt-4o"))
pipe.connect("prompt_builder", "llm")
country = "Germany"
system_message = ChatMessage.from_system(
"You are an assistant giving out valuable information to language learners.",
)
messages = [
system_message,
ChatMessage.from_user("What's the official language of {{ country }}?"),
]
res = pipe.run(
data={
"prompt_builder": {
"template_variables": {"country": country},
"template": messages,
},
},
)
print(res)
```
---
// File: pipeline-components/generators/llamacppchatgenerator
# LlamaCppChatGenerator
`LlamaCppChatGenerator` enables chat completion using an LLM running on Llama.cpp.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `model`: The path of the model to use |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) instances representing the input messages |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) instances with all the replies generated by the LLM |
| **API reference** | [Llama.cpp](/reference/integrations-llama-cpp) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/llama_cpp |
| **Package name** | `llama-cpp-haystack` |
## Overview
[Llama.cpp](https://github.com/ggml-org/llama.cpp) is a library written in C/C++ for efficient inference of Large Language Models. It leverages the efficient quantized GGUF format, dramatically reducing memory requirements and accelerating inference. This means it is possible to run LLMs efficiently on standard machines (even without GPUs).
`Llama.cpp` uses the quantized binary file of the LLM in GGUF format, which can be downloaded from [Hugging Face](https://huggingface.co/models?library=gguf). `LlamaCppChatGenerator` supports models running on `Llama.cpp` by taking the path to the locally saved GGUF file as `model` parameter at initialization.
### Tool Support
`LlamaCppChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = LlamaCppChatGenerator(
model="/path/to/model.gguf",
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
## Installation
Install the `llama-cpp-haystack` package to use this integration:
```shell
pip install llama-cpp-haystack
```
### Using a different compute backend
The default installation behavior is to build `llama.cpp` for CPU on Linux and Windows and use Metal on MacOS. To use other compute backends:
1. Follow instructions on the [llama.cpp installation page](https://github.com/abetlen/llama-cpp-python#installation) to install [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) for your preferred compute backend.
2. Install [llama-cpp-haystack](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/llama_cpp) using the command above.
For example, to use `llama-cpp-haystack` with the **cuBLAS backend**, you have to run the following commands:
```shell
export GGML_CUDA=1
CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python
pip install llama-cpp-haystack
```
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
1. Download the GGUF version of the desired LLM. The GGUF versions of popular models can be downloaded from [Hugging Face](https://huggingface.co/models?library=gguf).
2. Initialize `LlamaCppChatGenerator` with the path to the GGUF file and specify the required model and text generation parameters:
```python
from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
from haystack.dataclasses import ChatMessage
generator = LlamaCppChatGenerator(
model="/content/openchat-3.5-1210.Q3_K_S.gguf",
n_ctx=512,
n_batch=128,
model_kwargs={"n_gpu_layers": -1},
generation_kwargs={"max_tokens": 128, "temperature": 0.1},
)
messages = [ChatMessage.from_user("Who is the best American actor?")]
result = generator.run(messages)
```
### Passing additional model parameters
The `model`, `n_ctx`, `n_batch` arguments have been exposed for convenience and can be directly passed to the Generator during initialization as keyword arguments. Note that `model` translates to `llama.cpp`'s `model_path` parameter.
The `model_kwargs` parameter can pass additional arguments when initializing the model. In case of duplication, these parameters override the `model`, `n_ctx`, and `n_batch` initialization parameters.
See [Llama.cpp's LLM documentation](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.__init__) for more information on the available model arguments.
**Note**: Llama.cpp automatically extracts the `chat_template` from the model metadata for applying formatting to ChatMessages. You can override the `chat_template` used by passing in a custom `chat_handler` or `chat_format` as a model parameter.
For example, to offload the model to GPU during initialization:
```python
from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
from haystack.dataclasses import ChatMessage
generator = LlamaCppChatGenerator(
model="/content/openchat-3.5-1210.Q3_K_S.gguf",
n_ctx=512,
n_batch=128,
model_kwargs={"n_gpu_layers": -1},
)
messages = [ChatMessage.from_user("Who is the best American actor?")]
result = generator.run(messages, generation_kwargs={"max_tokens": 128})
generated_reply = result["replies"][0].text
print(generated_reply)
```
### Passing text generation parameters
The `generation_kwargs` parameter can pass additional generation arguments like `max_tokens`, `temperature`, `top_k`, `top_p`, and others to the model during inference.
See [Llama.cpp's Chat Completion API documentation](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.create_chat_completion) for more information on the available generation arguments.
**Note**: JSON mode, Function Calling, and Tools are all supported as `generation_kwargs`. Please see the [llama-cpp-python GitHub README](https://github.com/abetlen/llama-cpp-python?tab=readme-ov-file#json-and-json-schema-mode) for more information on how to use them.
For example, to set the `max_tokens` and `temperature`:
```python
from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
from haystack.dataclasses import ChatMessage
generator = LlamaCppChatGenerator(
model="/content/openchat-3.5-1210.Q3_K_S.gguf",
n_ctx=512,
n_batch=128,
generation_kwargs={"max_tokens": 128, "temperature": 0.1},
)
messages = [ChatMessage.from_user("Who is the best American actor?")]
result = generator.run(messages)
```
### With multimodal (image + text) inputs
```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
# Initialize with multimodal support
llm = LlamaCppChatGenerator(
model="llava-v1.5-7b-q4_0.gguf",
chat_handler_name="Llava15ChatHandler", # Use llava-1-5 handler
model_clip_path="mmproj-model-f16.gguf", # CLIP model
n_ctx=4096, # Larger context for image processing
)
image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(
content_parts=["What does the image show? Max 5 words.", image],
)
response = llm.run([user_message])["replies"][0].text
print(response)
# Red apple on straw.
```
The `generation_kwargs` can also be passed to the `run` method of the generator directly:
```python
from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
from haystack.dataclasses import ChatMessage
generator = LlamaCppChatGenerator(
model="/content/openchat-3.5-1210.Q3_K_S.gguf",
n_ctx=512,
n_batch=128,
)
messages = [ChatMessage.from_user("Who is the best American actor?")]
result = generator.run(
messages,
generation_kwargs={"max_tokens": 128, "temperature": 0.1},
)
```
### In a pipeline
We use the `LlamaCppChatGenerator` in a Retrieval Augmented Generation pipeline on the [Simple Wikipedia](https://huggingface.co/datasets/pszemraj/simple_wikipedia) Dataset from Hugging Face and generate answers using the [OpenChat-3.5](https://huggingface.co/openchat/openchat-3.5-1210) LLM.
Load the dataset:
```python
# Install HuggingFace Datasets using "pip install datasets"
from datasets import load_dataset
from haystack import Document, Pipeline
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders import ChatPromptBuilder
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.dataclasses import ChatMessage
# Import LlamaCppChatGenerator
from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
# Load first 100 rows of the Simple Wikipedia Dataset from HuggingFace
dataset = load_dataset("pszemraj/simple_wikipedia", split="validation[:100]")
docs = [
Document(
content=doc["text"],
meta={
"title": doc["title"],
"url": doc["url"],
},
)
for doc in dataset
]
```
Index the documents to the `InMemoryDocumentStore` using the `SentenceTransformersDocumentEmbedder` and `DocumentWriter`:
```python
doc_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
# Install the Sentence Transformers embedders using "pip install sentence-transformers-haystack"
doc_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
# Indexing Pipeline
indexing_pipeline = Pipeline()
indexing_pipeline.add_component(instance=doc_embedder, name="DocEmbedder")
indexing_pipeline.add_component(
instance=DocumentWriter(document_store=doc_store),
name="DocWriter",
)
indexing_pipeline.connect("DocEmbedder", "DocWriter")
indexing_pipeline.run({"DocEmbedder": {"documents": docs}})
```
Create the RAG pipeline and add the `LlamaCppChatGenerator` to it:
```python
system_message = ChatMessage.from_system(
"""
Answer the question using the provided context.
Context:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
""",
)
user_message = ChatMessage.from_user("Question: {{question}}")
assistent_message = ChatMessage.from_assistant("Answer: ")
chat_template = [system_message, user_message, assistent_message]
rag_pipeline = Pipeline()
text_embedder = SentenceTransformersTextEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
# Load the LLM using LlamaCppChatGenerator
model_path = "openchat-3.5-1210.Q3_K_S.gguf"
generator = LlamaCppChatGenerator(model=model_path, n_ctx=4096, n_batch=128)
rag_pipeline.add_component(
instance=text_embedder,
name="text_embedder",
)
rag_pipeline.add_component(
instance=InMemoryEmbeddingRetriever(document_store=doc_store, top_k=3),
name="retriever",
)
rag_pipeline.add_component(
instance=ChatPromptBuilder(template=chat_template),
name="prompt_builder",
)
rag_pipeline.add_component(instance=generator, name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("text_embedder", "retriever")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder", "llm")
rag_pipeline.connect("llm", "answer_builder")
rag_pipeline.connect("retriever", "answer_builder.documents")
```
Run the pipeline:
```python
question = "Which year did the Joker movie release?"
result = rag_pipeline.run(
{
"text_embedder": {"text": question},
"prompt_builder": {"question": question},
"llm": {"generation_kwargs": {"max_tokens": 128, "temperature": 0.1}},
"answer_builder": {"query": question},
},
)
generated_answer = result["answer_builder"]["answers"][0]
print(generated_answer.data)
# The Joker movie was released on October 4, 2019.
```
---
// File: pipeline-components/generators/llamacppgenerator
# LlamaCppGenerator
`LlamaCppGenerator` provides an interface to generate text using an LLM running on Llama.cpp.
:::warning[Deprecation Notice]
`LlamaCppGenerator` is deprecated and will be removed in a future version.
We recommend switching to [LlamaCppChatGenerator](llamacppchatgenerator.mdx) instead,
which also accepts a plain string as input.
:::
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `model`: The path of the model to use |
| **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM |
| **Output variables** | `replies`: A list of strings with all the replies generated by the LLM
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count and others | | **API reference** | [Llama.cpp](/reference/integrations-llama-cpp) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/llama_cpp | | **Package name** | `llama-cpp-haystack` |
## Overview
[Llama.cpp](https://github.com/ggml-org/llama.cpp) is a library written in C/C++ for efficient inference of Large Language Models. It leverages the efficient quantized GGUF format, dramatically reducing memory requirements and accelerating inference. This means it is possible to run LLMs efficiently on standard machines (even without GPUs).
`Llama.cpp` uses the quantized binary file of the LLM in GGUF format that can be downloaded from [Hugging Face](https://huggingface.co/models?library=gguf). `LlamaCppGenerator` supports models running on `Llama.cpp` by taking the path to the locally saved GGUF file as `model` parameter at initialization.
## Installation
Install the `llama-cpp-haystack` package:
```bash
pip install llama-cpp-haystack
```
### Using a different compute backend
The default installation behavior is to build `llama.cpp` for CPU on Linux and Windows and use Metal on MacOS. To use other compute backends:
1. Follow instructions on the [llama.cpp installation page](https://github.com/abetlen/llama-cpp-python#installation) to install [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) for your preferred compute backend.
2. Install [llama-cpp-haystack](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/llama_cpp) using the command above.
For example, to use `llama-cpp-haystack` with the **cuBLAS backend**, you have to run the following commands:
```bash
export GGML_CUDA=1
CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python
pip install llama-cpp-haystack
```
The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
1. You need to download the GGUF version of the desired LLM. The GGUF versions of popular models can be downloaded from [Hugging Face](https://huggingface.co/models?library=gguf).
2. Initialize a `LlamaCppGenerator` with the path to the GGUF file and also specify the required model and text generation parameters:
```python
from haystack_integrations.components.generators.llama_cpp import LlamaCppGenerator
generator = LlamaCppGenerator(
model="/content/openchat-3.5-1210.Q3_K_S.gguf",
n_ctx=512,
n_batch=128,
model_kwargs={"n_gpu_layers": -1},
generation_kwargs={"max_tokens": 128, "temperature": 0.1},
)
prompt = f"Who is the best American actor?"
result = generator.run(prompt)
```
### Passing additional model parameters
The `model`, `n_ctx`, `n_batch` arguments have been exposed for convenience and can be directly passed to the Generator during initialization as keyword arguments. Note that `model` translates to `llama.cpp`'s `model_path` parameter.
The `model_kwargs` parameter can pass additional arguments when initializing the model. In case of duplication, these parameters override the `model`, `n_ctx`, and `n_batch` initialization parameters.
See [Llama.cpp's LLM documentation](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.__init__) for more information on the available model arguments.
For example, to offload the model to GPU during initialization:
```python
from haystack_integrations.components.generators.llama_cpp import LlamaCppGenerator
generator = LlamaCppGenerator(
model="/content/openchat-3.5-1210.Q3_K_S.gguf",
n_ctx=512,
n_batch=128,
model_kwargs={"n_gpu_layers": -1},
)
prompt = f"Who is the best American actor?"
result = generator.run(prompt, generation_kwargs={"max_tokens": 128})
generated_text = result["replies"][0]
print(generated_text)
```
### Passing text generation parameters
The `generation_kwargs` parameter can pass additional generation arguments like `max_tokens`, `temperature`, `top_k`, `top_p`, and others to the model during inference.
See [Llama.cpp's Completion API documentation](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.create_completion) for more information on the available generation arguments.
For example, to set the `max_tokens` and `temperature`:
```python
from haystack_integrations.components.generators.llama_cpp import LlamaCppGenerator
generator = LlamaCppGenerator(
model="/content/openchat-3.5-1210.Q3_K_S.gguf",
n_ctx=512,
n_batch=128,
generation_kwargs={"max_tokens": 128, "temperature": 0.1},
)
prompt = f"Who is the best American actor?"
result = generator.run(prompt)
```
The `generation_kwargs` can also be passed to the `run` method of the generator directly:
```python
from haystack_integrations.components.generators.llama_cpp import LlamaCppGenerator
generator = LlamaCppGenerator(
model="/content/openchat-3.5-1210.Q3_K_S.gguf",
n_ctx=512,
n_batch=128,
)
prompt = f"Who is the best American actor?"
result = generator.run(
prompt,
generation_kwargs={"max_tokens": 128, "temperature": 0.1},
)
```
### Using in a Pipeline
We use the `LlamaCppGenerator` in a Retrieval Augmented Generation pipeline on the [Simple Wikipedia](https://huggingface.co/datasets/pszemraj/simple_wikipedia) Dataset from HuggingFace and generate answers using the [OpenChat-3.5](https://huggingface.co/openchat/openchat-3.5-1210) LLM.
Load the dataset:
```python
# Install HuggingFace Datasets using "pip install datasets"
from datasets import load_dataset
from haystack import Document, Pipeline
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders.prompt_builder import PromptBuilder
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
# Import LlamaCppGenerator
from haystack_integrations.components.generators.llama_cpp import LlamaCppGenerator
# Load first 100 rows of the Simple Wikipedia Dataset from HuggingFace
dataset = load_dataset("pszemraj/simple_wikipedia", split="validation[:100]")
docs = [
Document(
content=doc["text"],
meta={
"title": doc["title"],
"url": doc["url"],
},
)
for doc in dataset
]
```
Index the documents to the `InMemoryDocumentStore` using the `SentenceTransformersDocumentEmbedder` and `DocumentWriter`:
```python
doc_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
doc_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
# Indexing Pipeline
indexing_pipeline = Pipeline()
indexing_pipeline.add_component(instance=doc_embedder, name="DocEmbedder")
indexing_pipeline.add_component(
instance=DocumentWriter(document_store=doc_store),
name="DocWriter",
)
indexing_pipeline.connect(connect_from="DocEmbedder", connect_to="DocWriter")
indexing_pipeline.run({"DocEmbedder": {"documents": docs}})
```
Create the Retrieval Augmented Generation (RAG) pipeline and add the `LlamaCppGenerator` to it:
```python
# Prompt Template for the https://huggingface.co/openchat/openchat-3.5-1210 LLM
prompt_template = """GPT4 Correct User: Answer the question using the provided context.
Question: {{question}}
Context:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
<|end_of_turn|>
GPT4 Correct Assistant:
"""
rag_pipeline = Pipeline()
text_embedder = SentenceTransformersTextEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
# Load the LLM using LlamaCppGenerator
model_path = "openchat-3.5-1210.Q3_K_S.gguf"
generator = LlamaCppGenerator(model=model_path, n_ctx=4096, n_batch=128)
rag_pipeline.add_component(
instance=text_embedder,
name="text_embedder",
)
rag_pipeline.add_component(
instance=InMemoryEmbeddingRetriever(document_store=doc_store, top_k=3),
name="retriever",
)
rag_pipeline.add_component(
instance=PromptBuilder(template=prompt_template),
name="prompt_builder",
)
rag_pipeline.add_component(instance=generator, name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("text_embedder", "retriever")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder", "llm")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")
```
Run the pipeline:
```python
question = "Which year did the Joker movie release?"
result = rag_pipeline.run(
{
"text_embedder": {"text": question},
"prompt_builder": {"question": question},
"llm": {"generation_kwargs": {"max_tokens": 128, "temperature": 0.1}},
"answer_builder": {"query": question},
},
)
generated_answer = result["answer_builder"]["answers"][0]
print(generated_answer.data)
# The Joker movie was released on October 4, 2019.
```
---
// File: pipeline-components/generators/llamastackchatgenerator
# LlamaStackChatGenerator
This component enables chat completions using any model made available by inference providers on a Llama Stack server.
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count and others | | **API reference** | [Llama.cpp](/reference/integrations-llama-cpp) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/llama_cpp | | **Package name** | `llama-cpp-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `model`: The name of the model to use for chat completion.
This depends on the inference provider used for the Llama Stack Server. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat | | **Output variables** | `replies`: A list of alternative replies of the model to the input chat | | **API reference** | [Llama Stack](/reference/integrations-llama-stack) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/llama_stack | | **Package name** | `llama-stack-haystack` |
## Overview
[Llama Stack](https://ogx-ai.github.io/docs) provides building blocks and unified APIs to streamline the development of AI applications across various environments.
The `LlamaStackChatGenerator` enables you to access any LLMs exposed by inference providers hosted on a Llama Stack server. It abstracts away the underlying provider details, allowing you to reuse the same client-side code regardless of the inference backend. For a list of supported providers and configuration options, refer to the [Llama Stack documentation](https://ogx-ai.github.io/docs/providers/inference).
This component uses the same `ChatMessage` format as other Haystack Chat Generators for structured input and output. For more information, see the [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
### Tool Support
`LlamaStackChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.llama_stack import LlamaStackChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = LlamaStackChatGenerator(
model="ollama/llama3.2:3b",
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
## Initialization
To use this integration, you must have:
- A running instance of a Llama Stack server (local or remote)
- A valid model name supported by your selected inference provider
Then initialize the `LlamaStackChatGenerator` by specifying the `model` name or ID. The value depends on the inference provider running on your server.
**Examples:**
- For Ollama: `model="ollama/llama3.2:3b"`
- For vLLM: `model="meta-llama/Llama-3.2-3B"`
**Note:** Switching the inference provider only requires updating the model name.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
To start using this integration, install the package with:
```shell
pip install llama-stack-haystack
```
### On its own
```python
import os
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.llama_stack import (
LlamaStackChatGenerator,
)
client = LlamaStackChatGenerator(model="ollama/llama3.2:3b")
response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
print(response["replies"])
```
#### With Streaming
```python
import os
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.llama_stack import (
LlamaStackChatGenerator,
)
from haystack.components.generators.utils import print_streaming_chunk
client = LlamaStackChatGenerator(
model="ollama/llama3.2:3b",
streaming_callback=print_streaming_chunk,
)
response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
print(response["replies"])
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.llama_stack import (
LlamaStackChatGenerator,
)
prompt_builder = ChatPromptBuilder()
llm = LlamaStackChatGenerator(model="ollama/llama3.2:3b")
pipe = Pipeline()
pipe.add_component("builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("builder.prompt", "llm.messages")
messages = [
ChatMessage.from_system("Give brief answers."),
ChatMessage.from_user("Tell me about {{city}}"),
]
response = pipe.run(
data={"builder": {"template": messages, "template_variables": {"city": "Berlin"}}},
)
print(response)
```
---
// File: pipeline-components/generators/metallamachatgenerator
# MetaLlamaChatGenerator
This component enables chat completion with any model hosted available with Meta Llama API.
This depends on the inference provider used for the Llama Stack Server. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat | | **Output variables** | `replies`: A list of alternative replies of the model to the input chat | | **API reference** | [Llama Stack](/reference/integrations-llama-stack) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/llama_stack | | **Package name** | `llama-stack-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: A Meta Llama API key. Can be set with `LLAMA_API_KEY` env variable or passed to `init()` method. |
| **Mandatory run variables** | `messages`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [Meta Llama API](/reference/integrations-meta-llama) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/meta_llama |
| **Package name** | `meta-llama-haystack` |
## Overview
The `MetaLlamaChatGenerator` enables you to use multiple Meta Llama models by making chat completion calls to the Meta [Llama API](https://llama.developer.meta.com/?utm_source=partner-haystack&utm_medium=website). The default model is `Llama-4-Scout-17B-16E-Instruct-FP8`.
Currently available models are:
| | | | | |
| --- | --- | --- | --- | --- |
| Model ID | Input context length | Output context length | Input Modalities | Output Modalities |
| `Llama-4-Scout-17B-16E-Instruct-FP8` | 128k | 4028 | Text, Image | Text |
| `Llama-4-Maverick-17B-128E-Instruct-FP8` | 128k | 4028 | Text, Image | Text |
| `Llama-3.3-70B-Instruct` | 128k | 4028 | Text | Text |
| `Llama-3.3-8B-Instruct` | 128k | 4028 | Text | Text |
This component uses the same `ChatMessage` format as other Haystack Chat Generators for structured input and output. For more information, see the [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
### Tool Support
`MetaLlamaChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.meta_llama import MetaLlamaChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = MetaLlamaChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Initialization
To use this integration, you must have a Meta Llama API key. You can provide it with the `LLAMA_API_KEY` environment variable or by using a [Secret](../../concepts/secret-management.mdx).
Then, install the `meta-llama-haystack` integration:
```shell
pip install meta-llama-haystack
```
### Streaming
`MetaLlamaChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM, allowing tokens to be emitted as they are generated. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
## Usage
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.meta_llama import (
MetaLlamaChatGenerator,
)
llm = MetaLlamaChatGenerator()
response = llm.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
print(response["replies"][0].text)
```
With streaming and model routing:
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.meta_llama import (
MetaLlamaChatGenerator,
)
llm = MetaLlamaChatGenerator(
model="Llama-3.3-8B-Instruct",
streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
)
response = llm.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
# check the model used for the response
print("\n\n Model used: ", response["replies"][0].meta["model"])
```
With multimodal inputs:
```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack_integrations.components.generators.meta_llama import (
MetaLlamaChatGenerator,
)
llm = MetaLlamaChatGenerator(model="Llama-4-Scout-17B-16E-Instruct-FP8")
image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(
content_parts=["What does the image show? Max 5 words.", image],
)
response = llm.run([user_message])["replies"][0].text
print(response)
# Red apple on straw.
```
### In a pipeline
```python
# To run this example, you will need to set a `LLAMA_API_KEY` environment variable.
from haystack import Document, Pipeline
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.utils import print_streaming_chunk
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.dataclasses import ChatMessage
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.utils import Secret
from haystack_integrations.components.generators.meta_llama import (
MetaLlamaChatGenerator,
)
# Write documents to InMemoryDocumentStore
document_store = InMemoryDocumentStore()
document_store.write_documents(
[
Document(content="My name is Jean and I live in Paris."),
Document(content="My name is Mark and I live in Berlin."),
Document(content="My name is Giorgio and I live in Rome."),
],
)
# Build a RAG pipeline
prompt_template = [
ChatMessage.from_user(
"Given these documents, answer the question.\n"
"Documents:\n{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
"Question: {{question}}\n"
"Answer:",
),
]
# Define required variables explicitly
prompt_builder = ChatPromptBuilder(
template=prompt_template,
required_variables={"question", "documents"},
)
retriever = InMemoryBM25Retriever(document_store=document_store)
llm = MetaLlamaChatGenerator(
api_key=Secret.from_env_var("LLAMA_API_KEY"),
streaming_callback=print_streaming_chunk,
)
rag_pipeline = Pipeline()
rag_pipeline.add_component("retriever", retriever)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder", "llm.messages")
# Ask a question
question = "Who lives in Paris?"
rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
},
)
```
---
// File: pipeline-components/generators/mistralchatgenerator
# MistralChatGenerator
This component enables chat completion using Mistral’s text generation models.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: The Mistral API key. Can be set with `MISTRAL_API_KEY` env var. |
| **Mandatory run variables** | `messages` A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [Mistral](/reference/integrations-mistral) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral |
| **Package name** | `mistral-haystack` |
## Overview
This integration supports Mistral’s models provided through the generative endpoint. For a full list of available models, check out the [Mistral documentation](https://docs.mistral.ai/platform/endpoints/#generative-endpoints).
`MistralChatGenerator` needs a Mistral API key to work. You can write this key in:
- The `api_key` init parameter using [Secret API](../../concepts/secret-management.mdx)
- The `MISTRAL_API_KEY` environment variable (recommended)
Currently, available models are:
- `mistral-small-latest` (default)
- `mistral-medium-latest`
- `mistral-large-latest`
- `codestral-latest`
This component needs a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
Refer to the [Mistral API documentation](https://docs.mistral.ai/api/#operation/createChatCompletion) for more details on the parameters supported by the Mistral API, which you can provide with `generation_kwargs` when running the component.
### Tool Support
`MistralChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.mistral import MistralChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = MistralChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
Install the `mistral-haystack` package to use the `MistralChatGenerator`:
```shell
pip install mistral-haystack
```
#### On its own
```python
from haystack_integrations.components.generators.mistral import MistralChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
generator = MistralChatGenerator(
api_key=Secret.from_env_var("MISTRAL_API_KEY"),
streaming_callback=print_streaming_chunk,
)
message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
print(generator.run([message]))
```
With multimodal inputs:
```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack_integrations.components.generators.mistral import MistralChatGenerator
llm = MistralChatGenerator(model="pixtral-12b-2409")
image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(
content_parts=["What does the image show? Max 5 words.", image],
)
response = llm.run([user_message])["replies"][0].text
print(response)
# Red apple on straw.
```
#### In a Pipeline
Below is an example RAG Pipeline where we answer questions based on the URL contents. We add the contents of the URL into our `messages` in the `ChatPromptBuilder` and generate an answer with the `MistralChatGenerator`.
```python
from haystack import Document
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.utils import print_streaming_chunk
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.mistral import MistralChatGenerator
fetcher = LinkContentFetcher()
converter = HTMLToDocument()
prompt_builder = ChatPromptBuilder(variables=["documents"])
llm = MistralChatGenerator(
streaming_callback=print_streaming_chunk,
model="mistral-small",
)
message_template = """Answer the following question based on the contents of the article: {{query}}\n
Article: {{documents[0].content}} \n
"""
messages = [ChatMessage.from_user(message_template)]
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="fetcher", instance=fetcher)
rag_pipeline.add_component(name="converter", instance=converter)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)
rag_pipeline.connect("fetcher.streams", "converter.sources")
rag_pipeline.connect("converter.documents", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
question = "What are the capabilities of Mixtral?"
result = rag_pipeline.run(
{
"fetcher": {"urls": ["https://mistral.ai/news/mixtral-of-experts"]},
"prompt_builder": {
"template_variables": {"query": question},
"template": messages,
},
"llm": {"generation_kwargs": {"max_tokens": 165}},
},
)
```
## Additional References
🧑🍳 Cookbook: [Web QA with Mixtral-8x7B-Instruct-v0.1](https://haystack.deepset.ai/cookbook/mixtral-8x7b-for-web-qa)
---
// File: pipeline-components/generators/mockchatgenerator
# MockChatGenerator
A Chat Generator that returns predefined responses without calling any API, for tests and quick prototypes.
| | |
| --- | --- |
| **Most common position in a pipeline** | In place of a real Chat Generator, in tests and prototypes |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string |
| **Output variables** | `replies`: A list of generated `ChatMessage` objects |
| **API reference** | [Generators](/reference/generators-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/mock.py |
| **Package name** | `haystack-ai` |
## Overview
`MockChatGenerator` is a deterministic, zero-cost drop-in replacement for real Chat Generators such as `OpenAIChatGenerator`. It implements `run`, `run_async`, streaming callbacks, and serialization but never contacts an external service, which makes it ideal for unit tests, smoke tests, and quick prototypes.
The response is selected based on how the component is configured:
- **Fixed response**: Pass a single string or `ChatMessage` via `responses`. The same reply is returned on every call. A `ChatMessage` passed as a response must have the `assistant` role.
- **Cycling responses**: Pass a list of strings and/or `ChatMessage` objects via `responses`. Each call returns the next item, wrapping around to the start once the list is exhausted. This is useful to drive multi-step flows such as Agents, where the first call returns a tool call and a later call returns the final answer.
- **Dynamic response**: Pass a `response_fn` callable that receives the input messages and returns the reply as a string or an assistant `ChatMessage`. Use this when the reply should depend on the input. To support serialization, pass a named function.
- **Echo (default)**: With no configuration, the component echoes back the text of the last message that has text content, so it is usable out of the box.
`responses` and `response_fn` are mutually exclusive.
Further optional parameters:
- `model`: The model name reported in the response metadata. Defaults to `"mock-model"`.
- `meta`: Additional metadata merged into the `meta` of every returned `ChatMessage`. A per-response `ChatMessage`'s own metadata takes precedence.
- `streaming_callback`: An optional callback invoked with `StreamingChunk` objects reconstructed from the predefined response. It lets the mock exercise streaming code paths without a real model.
## Usage
### On its own
```python
from haystack.components.generators.chat import MockChatGenerator
from haystack.dataclasses import ChatMessage
# Fixed response
generator = MockChatGenerator(responses="Hello, this is a mock response.")
result = generator.run([ChatMessage.from_user("Hi!")])
print(result["replies"][0].text) # "Hello, this is a mock response."
# Echo mode (default): returns the last message with text content
generator = MockChatGenerator()
result = generator.run([ChatMessage.from_user("Repeat after me")])
print(result["replies"][0].text) # "Repeat after me"
```
### Driving an Agent
Pass `ChatMessage` objects (rather than plain strings) to return tool calls or reasoning content. With cycling responses, you can script a full agent loop without a real model:
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import MockChatGenerator
from haystack.dataclasses import ChatMessage, ToolCall
from haystack.tools import tool
@tool
def search(query: str) -> str:
"""Search for information."""
return f"Results for: {query}"
generator = MockChatGenerator(
responses=[
ChatMessage.from_assistant(
tool_calls=[ToolCall(tool_name="search", arguments={"query": "Haystack"})],
),
"Here is the final answer.",
],
)
agent = Agent(chat_generator=generator, tools=[search])
result = agent.run(messages=[ChatMessage.from_user("Tell me about Haystack")])
print(result["last_message"].text) # "Here is the final answer."
```
### Input-dependent responses
```python
from haystack.components.generators.chat import MockChatGenerator
from haystack.dataclasses import ChatMessage
def shout_back(messages: list[ChatMessage]) -> str:
return messages[-1].text.upper()
generator = MockChatGenerator(response_fn=shout_back)
result = generator.run([ChatMessage.from_user("hello")])
print(result["replies"][0].text) # "HELLO"
```
---
// File: pipeline-components/generators/nvidiachatgenerator
# NvidiaChatGenerator
This Generator enables chat completion using NVIDIA-hosted models.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: API key for the NVIDIA NIM. Can be set with `NVIDIA_API_KEY` env var. |
| **Mandatory run variables** | `messages`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [NVIDIA API](https://build.nvidia.com/models) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia |
| **Package name** | `nvidia-haystack` |
## Overview
`NvidiaChatGenerator` enables chat completions using NVIDIA generative models via the NVIDIA API. It is compatible with the [ChatMessage](../../concepts/data-classes/chatmessage.mdx) format for both input and output, ensuring seamless integration in chat-based pipelines.
You can use LLMs self-hosted with NVIDIA NIM or models hosted on the [NVIDIA API Catalog](https://build.nvidia.com/explore/discover). The default model for this component is `meta/llama-3.1-8b-instruct`.
To use this integration, you must have an NVIDIA API key. You can provide it with the `NVIDIA_API_KEY` environment variable or by using a [Secret](../../concepts/secret-management.mdx).
### Tool support
`NvidiaChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.nvidia import NvidiaChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = NvidiaChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, refer to the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
This generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
## Usage
To start using `NvidiaChatGenerator`, install the `nvidia-haystack` package:
```shell
pip install nvidia-haystack
```
You can use `NvidiaChatGenerator` with all the LLMs available in the [NVIDIA API Catalog](https://docs.api.nvidia.com/nim/reference) or with a model deployed using NVIDIA NIM. For more information, refer to the [NVIDIA NIM for LLMs Playbook](https://developer.nvidia.com/docs/nemo-microservices/inference/playbooks/nmi_playbook.html).
### On its own
To use LLMs from the NVIDIA API Catalog, specify the `api_base_url` if needed (the default is `https://integrate.api.nvidia.com/v1`) and your API key. You can get your API key from the [NVIDIA API Catalog](https://build.nvidia.com/explore/discover).
```python
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
from haystack_integrations.components.generators.nvidia import NvidiaChatGenerator
generator = NvidiaChatGenerator(
model="meta/llama-3.1-8b-instruct",
api_key=Secret.from_env_var("NVIDIA_API_KEY"),
)
messages = [ChatMessage.from_user("What's Natural Language Processing? Be brief.")]
result = generator.run(messages)
print(result["replies"])
```
With multimodal inputs:
```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack.utils import Secret
from haystack_integrations.components.generators.nvidia import NvidiaChatGenerator
llm = NvidiaChatGenerator(
model="meta/llama-3.2-11b-vision-instruct",
api_key=Secret.from_env_var("NVIDIA_API_KEY"),
)
image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(
content_parts=[
"What does the image show? Max 5 words.",
image,
],
)
response = llm.run([user_message])["replies"][0].text
print(response)
# Red apple on straw.
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
from haystack_integrations.components.generators.nvidia import NvidiaChatGenerator
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component(
"llm",
NvidiaChatGenerator(
model="meta/llama-3.1-8b-instruct",
api_key=Secret.from_env_var("NVIDIA_API_KEY"),
),
)
pipe.connect("prompt_builder", "llm")
country = "Germany"
system_message = ChatMessage.from_system(
"You are an assistant giving out valuable information to language learners.",
)
messages = [
system_message,
ChatMessage.from_user("What's the official language of {{ country }}?"),
]
res = pipe.run(
data={
"prompt_builder": {
"template_variables": {"country": country},
"template": messages,
},
},
)
print(res)
```
## Related
- Cookbook: [Haystack RAG Pipeline with Self-Deployed AI models using NVIDIA NIMs](https://haystack.deepset.ai/cookbook/rag-with-nims)
---
// File: pipeline-components/generators/nvidiagenerator
# NvidiaGenerator
This Generator enables text generation using NVIDIA-hosted models.
:::warning[Deprecation Notice]
`NvidiaGenerator` is deprecated and will be removed in a future version.
We recommend switching to [NvidiaChatGenerator](nvidiachatgenerator.mdx) instead,
which also accepts a plain string as input.
:::
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: API key for the NVIDIA NIM. Can be set with `NVIDIA_API_KEY` env var. |
| **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM |
| **Output variables** | `replies`: A list of strings with all the replies generated by the LLM
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count and others | | **API reference** | [NVIDIA](/reference/integrations-nvidia) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia | | **Package name** | `nvidia-haystack` |
## Overview
`NvidiaGenerator` provides an interface for generating text using LLMs self-hosted with NVIDIA NIM or models hosted on the [NVIDIA API Catalog](https://build.nvidia.com/explore/discover).
## Usage
To start using `NvidiaGenerator`, install the `nvidia-haystack` package:
```shell
pip install nvidia-haystack
```
You can use `NvidiaGenerator` with all the LLMs available in the [NVIDIA API Catalog](https://docs.api.nvidia.com/nim/reference) or with a model deployed using NVIDIA NIM. For more information, refer to the [NVIDIA NIM for LLMs Playbook](https://developer.nvidia.com/docs/nemo-microservices/inference/playbooks/nmi_playbook.html).
### On its own
To use LLMs from the NVIDIA API Catalog, specify the `api_url` and your API key. You can get your API key from the [NVIDIA API Catalog](https://build.nvidia.com/explore/discover).
`NvidiaGenerator` uses the `NVIDIA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with the `api_key` parameter:
```python
from haystack.utils.auth import Secret
from haystack_integrations.components.generators.nvidia import NvidiaGenerator
generator = NvidiaGenerator(
model="meta/llama-3.1-70b-instruct",
api_url="https://integrate.api.nvidia.com/v1",
api_key=Secret.from_token("`meta`: A list of dictionaries with the metadata associated with each reply, such as token count and others | | **API reference** | [NVIDIA](/reference/integrations-nvidia) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia | | **Package name** | `nvidia-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat |
| **Output variables** | `replies`: A list of LLM’s alternative replies |
| **API reference** | [Ollama](/reference/integrations-ollama) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama |
| **Package name** | `ollama-haystack` |
## Overview
[Ollama](https://github.com/jmorganca/ollama) is a project focused on running LLMs locally. Internally, it uses the quantized GGUF format by default. This means it is possible to run LLMs on standard machines (even without GPUs) without having to handle complex installation procedures.
`OllamaChatGenerator` supports models running on Ollama, such as `llama2` and `mixtral`. Find the full list of supported models [here](https://ollama.ai/library).
`OllamaChatGenerator` needs a `model` name and a `url` to work. By default, it uses `"qwen3:0.6b"` model and `"http://localhost:11434"` url.
The way to operate with `OllamaChatGenerator` is by using `ChatMessage` objects. [ChatMessage](../../concepts/data-classes/chatmessage.mdx) is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata. See the [usage](#usage) section for an example.
### Tool Support
`OllamaChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.ollama import OllamaChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = OllamaChatGenerator(
model="llama2",
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
You can stream output as it’s generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
```python
from haystack.components.generators.utils import print_streaming_chunk
# Configure any `Generator` or `ChatGenerator` with a streaming callback
component = SomeGeneratorOrChatGenerator(streaming_callback=print_streaming_chunk)
# If this is a `ChatGenerator`, pass a list of messages:
# from haystack.dataclasses import ChatMessage
# component.run([ChatMessage.from_user("Your question here")])
# If this is a (non-chat) `Generator`, pass a prompt:
# component.run({"prompt": "Your prompt here"})
```
:::info
Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
:::
See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
### Streaming with Tools
You can combine streaming with tool calling. Pass both `tools` and `streaming_callback`; when the model decides to invoke a tool, the streamed chunks carry tool-call deltas instead of text tokens, and the final reconstructed `ChatMessage` exposes the resolved `tool_calls` list on `replies[0]`.
```python
from haystack.dataclasses import ChatMessage
from haystack.dataclasses.streaming_chunk import StreamingChunk
from haystack.tools import create_tool_from_function
from haystack_integrations.components.generators.ollama import OllamaChatGenerator
def get_weather(city: str) -> str:
"""Get current weather for a city."""
return f"Sunny, 22°C in {city}"
def callback(chunk: StreamingChunk) -> None:
if chunk.tool_calls:
print(f"[tool delta] {chunk.tool_calls}")
elif chunk.content:
print(chunk.content, end="", flush=True)
generator = OllamaChatGenerator(
model="llama3.1:8b",
generation_kwargs={"temperature": 0.0},
tools=[create_tool_from_function(get_weather)],
streaming_callback=callback,
)
response = generator.run(
messages=[
ChatMessage.from_user(
"What's the weather in Berlin? Use the get_weather tool.",
),
],
)
# Final reconstructed message: tool_calls populated, text is None
assistant_message = response["replies"][0]
print(assistant_message.tool_calls)
# -> [ToolCall(tool_name='get_weather', arguments={'city': 'Berlin'}, ...)]
```
You can use the built-in `print_streaming_chunk` callback (which handles both text tokens and tool events) instead of writing your own.
## Usage
1. You need a running instance of Ollama. The installation instructions are [in the Ollama GitHub repository](https://github.com/jmorganca/ollama).
A fast way to run Ollama is using Docker:
```bash
docker run -d -p 11434:11434 --name ollama ollama/ollama:latest
```
2. You need to download or pull the desired LLM. The model library is available on the [Ollama website](https://ollama.ai/library).
If you are using Docker, you can, for example, pull the Zephyr model:
```bash
docker exec ollama ollama pull zephyr
```
If you already installed Ollama in your system, you can execute:
```bash
ollama pull zephyr
```
:::tip[Choose a specific version of a model]
You can also specify a tag to choose a specific (quantized) version of your model. The available tags are shown in the model card of the Ollama models library. This is an [example](https://ollama.ai/library/zephyr/tags) for Zephyr.
In this case, simply run
```shell
# ollama pull model:tag
ollama pull zephyr:7b-alpha-q3_K_S
```
:::
3. You also need to install the `ollama-haystack` package:
```bash
pip install ollama-haystack
```
### On its own
```python
from haystack_integrations.components.generators.ollama import OllamaChatGenerator
from haystack.dataclasses import ChatMessage
generator = OllamaChatGenerator(model="zephyr",
url = "http://localhost:11434",
generation_kwargs={
"num_predict": 100,
"temperature": 0.9,
})
messages = [ChatMessage.from_system("\nYou are a helpful, respectful and honest assistant"),
ChatMessage.from_user("What's Natural Language Processing?")]
print(generator.run(messages=messages))
>> {
"replies": [
ChatMessage(
_role=
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM |
| **Output variables** | `replies`: A list of strings with all the replies generated by the LLM
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count and others | | **API reference** | [Ollama](/reference/integrations-ollama) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama | | **Package name** | `ollama-haystack` |
## Overview
`OllamaGenerator` provides an interface to generate text using an LLM running on Ollama.
`OllamaGenerator` needs a `model` name and a `url` to work. By default, it uses `"orca-mini"` model and `"http://localhost:11434"` url.
[Ollama](https://github.com/jmorganca/ollama) is a project focused on running LLMs locally. Internally, it uses the quantized GGUF format by default. This means it is possible to run LLMs on standard machines (even without GPUs) without having to go through complex installation procedures.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
1. You need a running instance of Ollama. You can find the installation instructions [here](https://github.com/jmorganca/ollama).
A fast way to run Ollama is using Docker:
```shell
docker run -d -p 11434:11434 --name ollama ollama/ollama:latest
```
2. You need to download or pull the desired LLM. The model library is available on the [Ollama website](https://ollama.ai/library).
If you are using Docker, you can, for example, pull the Zephyr model:
```shell
docker exec ollama ollama pull zephyr
```
If you have already installed Ollama in your system, you can execute:
```shell
ollama pull zephyr
```
:::tip[Choose a specific version of a model]
You can also specify a tag to choose a specific (quantized) version of your model. The available tags are shown in the model card of the Ollama models library. This is an [example](https://ollama.ai/library/zephyr/tags) for Zephyr.
In this case, simply run
```shell
# ollama pull model:tag
ollama pull zephyr:7b-alpha-q3_K_S
```
:::
3. You also need to install the `ollama-haystack` package:
```shell
pip install ollama-haystack
```
### On its own
Here's how the `OllamaGenerator` would work just on its own:
```python
from haystack_integrations.components.generators.ollama import OllamaGenerator
generator = OllamaGenerator(
model="zephyr",
url="http://localhost:11434",
generation_kwargs={
"num_predict": 100,
"temperature": 0.9,
},
)
print(generator.run("Who is the best American actor?"))
# {'replies': ['I do not have the ability to form opinions or preferences.
# However, some of the most acclaimed american actors in recent years include
# denzel washington, tom hanks, leonardo dicaprio, matthew mcconaughey...'],
# 'meta': [{'model': 'zephyr', ...}]}
```
### In a Pipeline
```python
from haystack_integrations.components.generators.ollama import OllamaGenerator
from haystack import Pipeline, Document
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.builders.prompt_builder import PromptBuilder
from haystack.document_stores.in_memory import InMemoryDocumentStore
template = """
Given the following information, answer the question.
Context:
{% for document in documents %}
{{ document.content }}
{% endfor %}
Question: {{ query }}?
"""
docstore = InMemoryDocumentStore()
docstore.write_documents(
[
Document(content="I really like summer"),
Document(content="My favorite sport is soccer"),
Document(content="I don't like reading sci-fi books"),
Document(content="I don't like crowded places"),
],
)
generator = OllamaGenerator(
model="zephyr",
url="http://localhost:11434",
generation_kwargs={
"num_predict": 100,
"temperature": 0.9,
},
)
pipe = Pipeline()
pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("llm", generator)
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "llm")
result = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
print(result)
# {'llm': {'replies': ['Based on the provided context, it seems that you enjoy
# soccer and summer. Unfortunately, there is no direct information given about
# what else you enjoy...'],
# 'meta': [{'model': 'zephyr', ...]}}
```
---
// File: pipeline-components/generators/openaichatgenerator
# OpenAIChatGenerator
`OpenAIChatGenerator` enables chat completion using OpenAI's large language models (LLMs).
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count and others | | **API reference** | [Ollama](/reference/integrations-ollama) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama | | **Package name** | `ollama-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a single string |
| **Output variables** | `replies`: A list of alternative replies of the LLM to the input chat |
| **API reference** | [Generators](/reference/generators-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/openai.py |
| **Package name** | `haystack-ai` |
## Overview
`OpenAIChatGenerator` supports OpenAI's chat completion models, such as `gpt-4o-mini`, `gpt-4.1-mini`, and the GPT-5 family. The default model is `gpt-5-mini`.
`OpenAIChatGenerator` needs an OpenAI key to work. It uses an ` OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```python
generator = OpenAIChatGenerator(model="gpt-4o-mini")
```
Then, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata. See the [usage](#usage) section for an example. If a string is passed, it is converted into a list containing a single `ChatMessage` with the `user` role.
You can pass any chat completion parameters valid for the `openai.ChatCompletion.create` method directly to `OpenAIChatGenerator` using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat).
`OpenAIChatGenerator` can support custom deployments of your OpenAI models through the `api_base_url` init parameter.
### Structured Output
`OpenAIChatGenerator` supports structured output generation, allowing you to receive responses in a predictable format. You can use Pydantic models or JSON schemas to define the structure of the output through the `response_format` parameter in `generation_kwargs`.
This is useful when you need to extract structured data from text or generate responses that match a specific format.
```python
from pydantic import BaseModel
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
class NobelPrizeInfo(BaseModel):
recipient_name: str
award_year: int
category: str
achievement_description: str
nationality: str
client = OpenAIChatGenerator(
model="gpt-4o-2024-08-06",
generation_kwargs={"response_format": NobelPrizeInfo},
)
response = client.run(
messages=[
ChatMessage.from_user(
"In 2021, American scientist David Julius received the Nobel Prize in"
" Physiology or Medicine for his groundbreaking discoveries on how the human body"
" senses temperature and touch.",
),
],
)
print(response["replies"][0].text)
# {"recipient_name":"David Julius","award_year":2021,"category":"Physiology or Medicine",
# "achievement_description":"David Julius was awarded for his transformative findings
# regarding the molecular mechanisms underlying the human body's sense of temperature
# and touch. Through innovative experiments, he identified specific receptors responsible
# for detecting heat and mechanical stimuli, ranging from gentle touch to pain-inducing
# pressure.","nationality":"American"}
```
:::info[Model Compatibility and Limitations]
- Pydantic models and JSON schemas are supported for latest models starting from `gpt-4o-2024-08-06`.
- Older models only support basic JSON mode through `{"type": "json_object"}`. For details, see [OpenAI JSON mode documentation](https://platform.openai.com/docs/guides/structured-outputs#json-mode).
- Streaming limitation: When using streaming with structured outputs, you must provide a JSON schema instead of a Pydantic model for `response_format`.
- For complete information, check the [OpenAI Structured Outputs documentation](https://platform.openai.com/docs/guides/structured-outputs).
:::
### Streaming
You can stream output as it’s generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
```python
from haystack.components.generators.chat.openai import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
# Configure any `ChatGenerator` with a streaming callback
component = OpenAIChatGenerator(streaming_callback=print_streaming_chunk)
# pass a list of messages or a single string to `run()`
from haystack.dataclasses import ChatMessage
component.run([ChatMessage.from_user("Your question here")])
```
:::info
Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
:::
See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
## Usage
### On its own
Basic usage:
```python
from haystack.dataclasses import ChatMessage
from haystack.components.generators.chat import OpenAIChatGenerator
client = OpenAIChatGenerator()
response = client.run(
[ChatMessage.from_user("What's Natural Language Processing? Be brief.")],
)
print(response)
# {'replies': [ChatMessage(_role=
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx), flexible |
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
| **Mandatory run variables** | `prompt`: A string containing the prompt for the model |
| **Output variables** | `images`: A list of generated images
`revised_prompt`: A string containing the prompt that was used to generate the image, if there was any revision to the prompt made by OpenAI | | **API reference** | [Generators](/reference/generators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/openai_image_generator.py | | **Package name** | `haystack-ai` |
## Overview
The `OpenAIImageGenerator` component generates images using OpenAI's image generation models (such as `gpt-image-2`).
By default, the component uses the `gpt-image-2` model, `"auto"` quality, and 1024x1024 resolution. You can change these parameters using `model` (during component initialization), `quality`, and `size` (during component initialization or run) parameters.
`OpenAIImageGenerator` needs an OpenAI key to work. It uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
```
image_generator = OpenAIImageGenerator(api_key=Secret.from_token("`revised_prompt`: A string containing the prompt that was used to generate the image, if there was any revision to the prompt made by OpenAI | | **API reference** | [Generators](/reference/generators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/openai_image_generator.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects containing the generated responses |
| **API reference** | [Generators](/reference/generators-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/openai_responses.py |
| **Package name** | `haystack-ai` |
## Overview
`OpenAIResponsesChatGenerator` uses OpenAI's Responses API to generate chat completions. It supports OpenAI's chat completion and reasoning models, such as `gpt-4o-mini`, `gpt-4.1-mini`, and the GPT-5 and o-series families. The default model is `gpt-5-mini`.
The Responses API is designed for reasoning-capable models and supports features like reasoning summaries, multi-turn conversations with previous response IDs, and structured outputs.
The component requires a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`), and optional metadata. If a string is passed, it is converted into a list containing a single `ChatMessage` with the `user` role. See the [usage](#usage) section for examples.
You can pass any parameters valid for the OpenAI Responses API directly to `OpenAIResponsesChatGenerator` using the `generation_kwargs` parameter, both at initialization and to the `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI Responses API documentation](https://platform.openai.com/docs/api-reference/responses).
`OpenAIResponsesChatGenerator` can support custom deployments of your OpenAI models through the `api_base_url` init parameter.
### Authentication
`OpenAIResponsesChatGenerator` needs an OpenAI key to work. It uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key` using a [`Secret`](../../concepts/secret-management.mdx):
```python
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.utils import Secret
generator = OpenAIResponsesChatGenerator(api_key=Secret.from_token("
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: An OpenRouter API key. Can be set with `OPENROUTER_API_KEY` env variable or passed to `init()` method. |
| **Mandatory run variables** | `messages`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [OpenRouter](/reference/integrations-openrouter) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/openrouter |
| **Package name** | `openrouter-haystack` |
## Overview
The `OpenRouterChatGenerator` enables you to use models from multiple providers (such as `openai/gpt-4o`, `anthropic/claude-sonnet-4.5`, and others) by making chat completion calls to the [OpenRouter API](https://openrouter.ai/docs/quickstart).
This generator also supports OpenRouter-specific features such as:
- Provider routing and model fallback that are configurable with the `generation_kwargs` parameter during initialization or runtime.
- Custom HTTP headers that can be supplied using the `extra_headers` parameter.
This component uses the same `ChatMessage` format as other Haystack Chat Generators for structured input and output. For more information, see the [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
### Tool Support
`OpenRouterChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.openrouter import OpenRouterChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = OpenRouterChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Initialization
To use this integration, you must have an active OpenRouter subscription with sufficient credits and an API key. You can provide it with the `OPENROUTER_API_KEY` environment variable or by using a [Secret](../../concepts/secret-management.mdx).
Then, install the `openrouter-haystack` integration:
```shell
pip install openrouter-haystack
```
### Streaming
`OpenRouterChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM, allowing tokens to be emitted as they are generated. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
## Usage
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.openrouter import (
OpenRouterChatGenerator,
)
client = OpenRouterChatGenerator()
response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
print(response["replies"][0].text)
```
With streaming and model routing:
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.openrouter import (
OpenRouterChatGenerator,
)
client = OpenRouterChatGenerator(
model="openrouter/auto",
streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
)
response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
# check the model used for the response
print("\n\n Model used: ", response["replies"][0].meta["model"])
```
With multimodal inputs:
```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack_integrations.components.generators.openrouter import (
OpenRouterChatGenerator,
)
llm = OpenRouterChatGenerator(model="anthropic/claude-sonnet-4.5")
image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(
content_parts=["What does the image show? Max 5 words.", image],
)
response = llm.run([user_message])["replies"][0].text
print(response)
# Red apple on straw.
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.openrouter import (
OpenRouterChatGenerator,
)
prompt_builder = ChatPromptBuilder()
llm = OpenRouterChatGenerator(model="openai/gpt-4o-mini")
pipe = Pipeline()
pipe.add_component("builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("builder.prompt", "llm.messages")
messages = [
ChatMessage.from_system("Give brief answers."),
ChatMessage.from_user("Tell me about {{city}}"),
]
response = pipe.run(
data={"builder": {"template": messages, "template_variables": {"city": "Berlin"}}},
)
print(response)
```
---
// File: pipeline-components/generators/orcarouterchatgenerator
# OrcaRouterChatGenerator
This component enables chat completion through [OrcaRouter](https://www.orcarouter.ai/), an OpenAI-compatible model routing gateway.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: An OrcaRouter API key. Can be set with `ORCAROUTER_API_KEY` env variable or passed to `init()` method. |
| **Mandatory run variables** | `messages`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [OrcaRouter](/reference/integrations-orcarouter) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/orcarouter |
| **Package name** | `orcarouter-haystack` |
## Overview
The `OrcaRouterChatGenerator` enables you to use models from multiple providers (such as `openai/gpt-4o-mini`, `anthropic/claude-opus-4.8`, and `google/gemini-2.5-flash`) by making chat completion calls to the [OrcaRouter API](https://docs.orcarouter.ai). Models are addressed with a `provider/model` namespace, and you can browse the available models in the [OrcaRouter model catalog](https://www.orcarouter.ai/models).
This generator also supports OrcaRouter-specific features such as:
- Automatic routing with the `orcarouter/auto` model, which lets OrcaRouter pick a live upstream model per request based on the policy configured in your OrcaRouter console.
- Provider routing and model fallback that are configurable with the `generation_kwargs` parameter during initialization or runtime. OrcaRouter-specific routing options are forwarded to the gateway through `extra_body`.
This component uses the same `ChatMessage` format as other Haystack Chat Generators for structured input and output. For more information, see the [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
### Tool Support
`OrcaRouterChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.orcarouter import OrcaRouterChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = OrcaRouterChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Initialization
To use this integration, you need an OrcaRouter API key. You can provide it with the `ORCAROUTER_API_KEY` environment variable or by using a [Secret](../../concepts/secret-management.mdx).
Then, install the `orcarouter-haystack` integration:
```shell
pip install orcarouter-haystack
```
### Streaming
`OrcaRouterChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM, allowing tokens to be emitted as they are generated. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
## Usage
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.orcarouter import (
OrcaRouterChatGenerator,
)
client = OrcaRouterChatGenerator(model="openai/gpt-4o-mini")
response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
print(response["replies"][0].text)
```
With automatic routing and streaming:
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.orcarouter import (
OrcaRouterChatGenerator,
)
client = OrcaRouterChatGenerator(
model="orcarouter/auto",
streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
)
response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
# check the model used for the response
print("\n\n Model used: ", response["replies"][0].meta["model"])
```
With a fallback chain:
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.orcarouter import (
OrcaRouterChatGenerator,
)
client = OrcaRouterChatGenerator(
model="openai/gpt-4o-mini",
generation_kwargs={
"extra_body": {
"route": "fallback",
"models": [
"openai/gpt-4o-mini",
"anthropic/claude-haiku-4.5",
"google/gemini-2.5-flash",
],
}
},
)
response = client.run([ChatMessage.from_user("What is Haystack?")])
print(response["replies"][0].text)
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.orcarouter import (
OrcaRouterChatGenerator,
)
prompt_builder = ChatPromptBuilder()
llm = OrcaRouterChatGenerator(model="openai/gpt-4o-mini")
pipe = Pipeline()
pipe.add_component("builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("builder.prompt", "llm.messages")
messages = [
ChatMessage.from_system("Give brief answers."),
ChatMessage.from_user("Tell me about {{city}}"),
]
response = pipe.run(
data={"builder": {"template": messages, "template_variables": {"city": "Berlin"}}},
)
print(response)
```
---
// File: pipeline-components/generators/perplexitychatgenerator
# PerplexityChatGenerator
`PerplexityChatGenerator` enables chat completion using models via the Perplexity Agent API.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: A Perplexity API key. Can be set with `PERPLEXITY_API_KEY` env var. |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat |
| **Output variables** | `replies`: A list of alternative replies of the LLM to the input chat |
| **API reference** | [Integrations](/reference/integrations-perplexity) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/perplexity/src/haystack_integrations/components/generators/perplexity/chat/chat_generator.py |
| **Package name** | `perplexity-haystack` |
## Overview
`PerplexityChatGenerator` is built on top of `OpenAIResponsesChatGenerator` and communicates with the [Perplexity Agent API](https://docs.perplexity.ai/) (`POST /v1/agent`), which uses an OpenAI Responses-compatible interface.
It supports the following models:
- `openai/gpt-5.5`
- `openai/gpt-5.4` (default)
- `anthropic/claude-sonnet-4-6`
- `xai/grok-4.3`
- `google/gemini-3-flash-preview`
See the [Perplexity Agent API models page](https://docs.perplexity.ai/docs/agent-api/models) for the current list.
`PerplexityChatGenerator` needs a Perplexity API key to work. It uses a `PERPLEXITY_API_KEY` environment variable by default.
The component accepts a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (such as `user`, `assistant`, or `system`), and optional metadata. See the [usage](#usage) section for an example.
You can pass any parameters supported by the Perplexity Agent API using the `generation_kwargs` parameter, both at initialization and in the `run()` method.
## Usage
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.perplexity import (
PerplexityChatGenerator,
)
chat_generator = PerplexityChatGenerator()
response = chat_generator.run(
[ChatMessage.from_user("What's Natural Language Processing? Be brief.")],
)
print(response["replies"][0].text)
```
With streaming — pass any callable to `streaming_callback`, or use the built-in `print_streaming_chunk`:
```python
from haystack.dataclasses import ChatMessage
from haystack.components.generators.utils import print_streaming_chunk
from haystack_integrations.components.generators.perplexity import (
PerplexityChatGenerator,
)
chat_generator = PerplexityChatGenerator(streaming_callback=print_streaming_chunk)
response = chat_generator.run(
[ChatMessage.from_user("What's Natural Language Processing? Be brief.")],
)
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
from haystack_integrations.components.generators.perplexity import (
PerplexityChatGenerator,
)
prompt_builder = ChatPromptBuilder(
template=[
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user("Tell me about {{topic}}"),
],
required_variables="*",
)
llm = PerplexityChatGenerator(
api_key=Secret.from_env_var("PERPLEXITY_API_KEY"),
model="openai/gpt-5.4",
)
pipe = Pipeline()
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("prompt_builder.prompt", "llm.messages")
result = pipe.run(
data={"prompt_builder": {"topic": "large language models"}},
)
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/generators/sagemakergenerator
# SagemakerGenerator
This component enables text generation using LLMs deployed on Amazon Sagemaker.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `model`: The model to use
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var. | | **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM | | **Output variables** | `replies`: A list of strings with all the replies generated by the LLM
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on | | **API reference** | [Amazon Sagemaker](/reference/integrations-amazon-sagemaker) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_sagemaker | | **Package name** | `amazon-sagemaker-haystack` |
`SagemakerGenerator` allows you to make use of models deployed on [AWS SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html).
## Parameters Overview
`SagemakerGenerator` needs AWS credentials to work. Set the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables.
You also need to specify your Sagemaker endpoint at initialization time for the component to work. Pass the endpoint name to the `model` parameter like this:
```python
generator = SagemakerGenerator(model="jumpstart-dft-hf-llm-falcon-7b-instruct-bf16")
```
Additionally, you can pass any text generation parameters valid for your specific model directly to `SagemakerGenerator` using the `generation_kwargs` parameter, both at initialization and to `run()` method.
If your model also needs custom attributes, pass those as a dictionary at initialization time by setting the `aws_custom_attributes` parameter.
One notable family of models that needs these custom parameters is Llama2, which needs to be initialized with `{"accept_eula": True}` :
```python
generator = SagemakerGenerator(
model="jumpstart-dft-meta-textgenerationneuron-llama-2-7b",
aws_custom_attributes={"accept_eula": True},
)
```
## Usage
You need to install `amazon-sagemaker-haystack` package to use the `SagemakerGenerator`:
```shell
pip install amazon-sagemaker-haystack
```
### On its own
Basic usage:
```python
from haystack_integrations.components.generators.amazon_sagemaker import SagemakerGenerator
client = SagemakerGenerator(model="jumpstart-dft-hf-llm-falcon-7b-instruct-bf16")
response = client.run("Briefly explain what NLP is in one sentence.")
print(response)
>>> {'replies': ["Natural Language Processing (NLP) is a subfield of artificial intelligence and computational linguistics that focuses on the interaction between computers and human languages..."],
'metadata': [{}]}
```
### In a pipeline
In a RAG pipeline:
```python
from haystack_integrations.components.generators.amazon_sagemaker import (
SagemakerGenerator,
)
from haystack import Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.builders import PromptBuilder
template = """
Given the following information, answer the question.
Context:
{% for document in documents %}
{{ document.content }}
{% endfor %}
Question: What's the official language of {{ country }}?
"""
pipe = Pipeline()
pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component(
"llm",
SagemakerGenerator(model="jumpstart-dft-hf-llm-falcon-7b-instruct-bf16"),
)
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "llm")
pipe.run({"prompt_builder": {"country": "France"}})
```
---
// File: pipeline-components/generators/stackitchatgenerator
# STACKITChatGenerator
This component enables chat completions using the STACKIT API.
`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.
`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var. | | **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM | | **Output variables** | `replies`: A list of strings with all the replies generated by the LLM
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on | | **API reference** | [Amazon Sagemaker](/reference/integrations-amazon-sagemaker) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_sagemaker | | **Package name** | `amazon-sagemaker-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `model`: The model used through the STACKIT API
`api_key`: A STACKIT API key. Can be set with `STACKIT_API_KEY` env var. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **API reference** | [STACKIT](/reference/integrations-stackit) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit | | **Package name** | `stackit-haystack` |
## Overview
`STACKITChatGenerator` enables text generation models served by STACKIT through their API.
### Parameters
To use the `STACKITChatGenerator`, ensure you have set a `STACKIT_API_KEY` as an environment variable. Alternatively, provide the API key as another environment variable or a token by setting
`api_key` and using Haystack’s [secret management](../../concepts/secret-management.mdx).
Set your preferred supported model with the `model` parameter when initializing the component. See the full list of all supported models on the [STACKIT website](https://docs.stackit.cloud/stackit/en/models-licenses-319914532.html).
Optionally, you can change the default `api_base_url`, which is `"https://api.openai-compat.model-serving.eu01.onstackit.cloud/v1"`.
You can pass any text generation parameters valid for the STACKIT Chat Completion API directly to this component with the `generation_kwargs` parameter in the init or run methods.
The component needs a list of `ChatMessage` objects to run. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata. Find out more about it [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
### Streaming
This ChatGenerator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly into the output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
Install the `stackit-haystack` package to use the `STACKITChatGenerator`:
```shell
pip install stackit-haystack
```
### On its own
```python
from haystack_integrations.components.generators.stackit import STACKITChatGenerator
from haystack.dataclasses import ChatMessage
generator = STACKITChatGenerator(model="neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8")
result = generator.run([ChatMessage.from_user("Tell me a joke.")])
print(result)
```
With multimodal inputs:
```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack_integrations.components.generators.stackit import STACKITChatGenerator
llm = STACKITChatGenerator(model="meta-llama/Llama-3.2-11B-Vision-Instruct")
image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(
content_parts=["What does the image show? Max 5 words.", image],
)
response = llm.run([user_message])["replies"][0].text
print(response)
# Red apple on straw.
```
### In a pipeline
You can also use `STACKITChatGenerator` in your pipeline.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.stackit import STACKITChatGenerator
prompt_builder = ChatPromptBuilder()
llm = STACKITChatGenerator(model="neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8")
messages = [ChatMessage.from_user("Question: {{question}} \\n")]
pipeline = Pipeline()
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("llm", llm)
pipeline.connect("prompt_builder.prompt", "llm.messages")
result = pipeline.run(
{
"prompt_builder": {
"template_variables": {"question": "Tell me a joke."},
"template": messages,
},
},
)
print(result)
```
For an example of streaming in a pipeline, refer to the examples in the STACKIT integration [repository](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit/examples) and on its dedicated [integration page](https://haystack.deepset.ai/integrations/stackit).
---
// File: pipeline-components/generators/togetheraichatgenerator
# TogetherAIChatGenerator
This component enables chat completion using models hosted on Together AI.
`api_key`: A STACKIT API key. Can be set with `STACKIT_API_KEY` env var. | | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **API reference** | [STACKIT](/reference/integrations-stackit) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit | | **Package name** | `stackit-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: A Together API key. Can be set with `TOGETHER_API_KEY` env var. |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [TogetherAI](/reference/integrations-togetherai) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/togetherai |
| **Package name** | `togetherai-haystack` |
## Overview
`TogetherAIChatGenerator` supports models hosted on [Together AI](https://docs.together.ai/intro), such as `meta-llama/Llama-3.3-70B-Instruct-Turbo`. For the full list of supported models, see [Together AI documentation](https://docs.together.ai/docs/serverless/models).
This component needs a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
You can pass any text generation parameters valid for the Together AI chat completion API directly to this component using the `generation_kwargs` parameter in `__init__` or the `generation_kwargs` parameter in `run` method. For more details on the parameters supported by the Together AI API, see [Together AI API documentation](https://docs.together.ai/reference/chat-completions-1).
To use this integration, you need to have an active TogetherAI subscription with sufficient credits and an API key. You can provide it with:
- The `TOGETHER_API_KEY` environment variable (recommended)
- The `api_key` init parameter and Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`
By default, the component uses Together AI's OpenAI-compatible base URL `https://api.together.xyz/v1`, which you can override with `api_base_url` if needed.
### Tool Support
`TogetherAIChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
```python
from haystack.tools import Tool, Toolset
from haystack_integrations.components.generators.togetherai import TogetherAIChatGenerator
# Create individual tools
weather_tool = Tool(name="weather", description="Get weather info", ...)
news_tool = Tool(name="news", description="Get latest news", ...)
# Group related tools into a toolset
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
# Pass mixed tools and toolsets to the generator
generator = TogetherAIChatGenerator(
tools=[math_toolset, weather_tool, news_tool] # Mix of Toolset and Tool objects
)
```
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
`TogetherAIChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM, allowing tokens to be emitted as they are generated. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
## Usage
Install the `togetherai-haystack` package to use the `TogetherAIChatGenerator`:
```shell
pip install togetherai-haystack
```
### On its own
Basic usage:
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.togetherai import (
TogetherAIChatGenerator,
)
client = TogetherAIChatGenerator()
response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
print(response["replies"][0].text)
```
With streaming:
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.togetherai import (
TogetherAIChatGenerator,
)
client = TogetherAIChatGenerator(
model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
)
response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
# check the model used for the response
print("\n\nModel used:", response["replies"][0].meta.get("model"))
```
### In a Pipeline
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.togetherai import (
TogetherAIChatGenerator,
)
prompt_builder = ChatPromptBuilder()
llm = TogetherAIChatGenerator(model="meta-llama/Llama-3.3-70B-Instruct-Turbo")
pipe = Pipeline()
pipe.add_component("builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("builder.prompt", "llm.messages")
messages = [
ChatMessage.from_system("Give brief answers."),
ChatMessage.from_user("Tell me about {{city}}"),
]
response = pipe.run(
data={"builder": {"template": messages, "template_variables": {"city": "Berlin"}}},
)
print(response)
```
---
// File: pipeline-components/generators/togetheraigenerator
# TogetherAIGenerator
This component enables text generation using models hosted on Together AI.
:::warning[Deprecation Notice]
`TogetherAIGenerator` is deprecated and will be removed in a future version.
We recommend switching to [TogetherAIChatGenerator](togetheraichatgenerator.mdx) instead,
which also accepts a plain string as input.
:::
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: A Together API key. Can be set with `TOGETHER_API_KEY` env var. |
| **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM |
| **Output variables** | `replies`: A list of strings with all the replies generated by the LLM
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on | | **API reference** | [TogetherAI](/reference/integrations-togetherai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/togetherai | | **Package name** | `togetherai-haystack` |
## Overview
`TogetherAIGenerator` supports models hosted on [Together AI](https://docs.together.ai/intro), such as `meta-llama/Llama-3.3-70B-Instruct-Turbo`. For the full list of supported models, see [Together AI documentation](https://docs.together.ai/docs/chat-models).
This component needs a prompt string to operate. You can pass any text generation parameters valid for the Together AI chat completion API directly to this component using the `generation_kwargs` parameter in `__init__` or the `generation_kwargs` parameter in `run` method. For more details on the parameters supported by the Together AI API, see [Together AI API documentation](https://docs.together.ai/reference/chat-completions-1).
You can also provide an optional `system_prompt` to set context or instructions for text generation. If not provided, the system prompt is omitted, and the default system prompt of the model is used.
To use this integration, you need to have an active TogetherAI subscription with sufficient credits and an API key. You can provide it with:
- The `TOGETHER_API_KEY` environment variable (recommended)
- The `api_key` init parameter and Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`
By default, the component uses Together AI's OpenAI-compatible base URL `https://api.together.xyz/v1`, which you can override with `api_base_url` if needed.
### Streaming
`TogetherAIGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM, allowing tokens to be emitted as they are generated. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
:::info
This component is designed for text generation, not for chat. If you want to use Together AI LLMs for chat, use [`TogetherAIChatGenerator`](togetheraichatgenerator.mdx) instead.
:::
## Usage
Install the `togetherai-haystack` package to use the `TogetherAIGenerator`:
```shell
pip install togetherai-haystack
```
### On its own
Basic usage:
```python
from haystack_integrations.components.generators.togetherai import TogetherAIGenerator
client = TogetherAIGenerator(model="meta-llama/Llama-3.3-70B-Instruct-Turbo")
response = client.run("What's Natural Language Processing? Be brief.")
print(response)
>> {'replies': ['Natural Language Processing (NLP) is a branch of artificial intelligence
>> that focuses on enabling computers to understand, interpret, and generate human language
>> in a way that is meaningful and useful.'],
>> 'meta': [{'model': 'meta-llama/Llama-3.3-70B-Instruct-Turbo', 'index': 0,
>> 'finish_reason': 'stop', 'usage': {'prompt_tokens': 15, 'completion_tokens': 36,
>> 'total_tokens': 51}}]}
```
With streaming:
```python
from haystack_integrations.components.generators.togetherai import TogetherAIGenerator
client = TogetherAIGenerator(
model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
)
response = client.run("What's Natural Language Processing? Be brief.")
print(response)
```
With system prompt:
```python
from haystack_integrations.components.generators.togetherai import TogetherAIGenerator
client = TogetherAIGenerator(
model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
system_prompt="You are a helpful assistant that provides concise answers.",
)
response = client.run("What's Natural Language Processing?")
print(response["replies"][0])
```
### In a Pipeline
```python
from haystack import Pipeline, Document
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.builders.prompt_builder import PromptBuilder
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.generators.togetherai import TogetherAIGenerator
docstore = InMemoryDocumentStore()
docstore.write_documents([
Document(content="Rome is the capital of Italy"),
Document(content="Paris is the capital of France")
])
query = "What is the capital of France?"
template = """
Given the following information, answer the question.
Context:
{% for document in documents %}
{{ document.content }}
{% endfor %}
Question: {{ query }}?
"""
pipe = Pipeline()
pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("llm", TogetherAIGenerator(model="meta-llama/Llama-3.3-70B-Instruct-Turbo"))
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "llm")
result = pipe.run({
"prompt_builder": {"query": query},
"retriever": {"query": query}
})
print(result)
>> {'llm': {'replies': ['The capital of France is Paris.'],
>> 'meta': [{'model': 'meta-llama/Llama-3.3-70B-Instruct-Turbo', ...}]}}
```
---
// File: pipeline-components/generators/transformerschatgenerator
# TransformersChatGenerator
Provides an interface for chat completion using a Hugging Face model that runs locally.
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on | | **API reference** | [TogetherAI](/reference/integrations-togetherai) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/togetherai | | **Package name** | `togetherai-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects generated by the LLM |
| **API reference** | [Transformers](/reference/integrations-transformers) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/transformers |
| **Package name** | `transformers-haystack` |
## Overview
Keep in mind that if LLMs run locally, you may need a powerful machine to run them. This depends strongly on the model you select and its parameter count.
If a string is passed to `messages`, it is converted into a list containing a single `ChatMessage` with the `user` role.
Authentication with a Hugging Face API token is only required to access private or gated models. You can pass the token at initialization with `token`, or set the `HF_API_TOKEN` or `HF_TOKEN` environment variable:
```python
generator = TransformersChatGenerator(
token=Secret.from_token("
| | |
| --- | --- |
| **Mandatory run variables** | `prefix`: A string of code before the current point
`suffix`: An optional string of code after the current point | | **Output variables** | `replies`: Code generated by the model | | **API reference** | [Google Vertex](/reference/integrations-google-vertex) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex | | **Package name** | `google-vertex-haystack` |
`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`.
### Parameters Overview
`VertexAICodeGenerator` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
## Usage
You need to install `google-vertex-haystack` package first to use the `VertexAIImageCaptioner`:
```shell
pip install google-vertex-haystack
```
Basic usage:
````python
from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator
generator = VertexAICodeGenerator()
result = generator.run(prefix="def to_json(data):")
for answer in result["replies"]:
print(answer)
>>> ```python
>>> import json
>>>
>>> def to_json(data):
>>> """Converts a Python object to a JSON string.
>>>
>>> Args:
>>> data: The Python object to convert.
>>>
>>> Returns:
>>> A JSON string representing the Python object.
>>> """
>>>
>>> return json.dumps(data)
>>> ```
````
You can also set other parameters like the number of output tokens, temperature, stop sequences, and the number of candidates.
Let’s try a different model:
```python
from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator
generator = VertexAICodeGenerator(
model="code-gecko",
temperature=0.8,
candidate_count=3
)
result = generator.run(prefix="def convert_temperature(degrees):")
for answer in result["replies"]:
print(answer)
>>>
>>> return degrees * (9/5) + 32
>>>
>>> return round(degrees * (9.0 / 5.0) + 32, 1)
>>>
>>> return 5 * (degrees - 32) /9
>>>
>>> def convert_temperature_back(degrees):
>>> return 9 * (degrees / 5) + 32
```
---
// File: pipeline-components/generators/vertexaigeminichatgenerator
# VertexAIGeminiChatGenerator
`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models.
:::warning[Deprecation Notice]
This integration uses the deprecated google-generativeai SDK, which will lose support after August 2025.
We recommend switching to the new [GoogleGenAIChatGenerator](googlegenaichatgenerator.mdx) integration instead.
:::
`suffix`: An optional string of code after the current point | | **Output variables** | `replies`: Code generated by the model | | **API reference** | [Google Vertex](/reference/integrations-google-vertex) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex | | **Package name** | `google-vertex-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat |
| **Output variables** | `replies`: A list of alternative replies of the model to the input chat |
| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
| **Package name** | `google-vertex-haystack` |
`VertexAIGeminiGenerator` supports `gemini-1.5-pro` and `gemini-1.5-flash`/ `gemini-2.0-flash` models. Note that [Google recommends upgrading](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions) from `gemini-1.5-pro` to `gemini-2.0-flash`.
For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models.
:::info
To explore the full capabilities of Gemini check out this [article](https://haystack.deepset.ai/blog/gemini-models-with-google-vertex-for-haystack) and the related [🧑🍳 Cookbook](https://colab.research.google.com/github/deepset-ai/haystack-cookbook/blob/main/notebooks/vertexai-gemini-examples.ipynb).
:::
### Parameters Overview
`VertexAIGeminiChatGenerator` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
You need to install the `google-vertex-haystack` package to use the `VertexAIGeminiChatGenerator`:
```shell
pip install google-vertex-haystack
```
### On its own
Basic usage:
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator
gemini_chat = VertexAIGeminiChatGenerator()
messages = [ChatMessage.from_user("Tell me the name of a movie")]
res = gemini_chat.run(messages)
print(res["replies"][0].text)
>>> The Shawshank Redemption
messages += [res["replies"][0], ChatMessage.from_user("Who's the main actor?")]
res = gemini_chat.run(messages)
print(res["replies"][0].text)
>>> Tim Robbins
```
When chatting with Gemini Pro, you can also easily use function calls. First, define the function locally and convert into a [Tool](../../tools/tool.mdx):
```python
from typing import Annotated
from haystack.tools import create_tool_from_function
# example function to get the current weather
def get_current_weather(
location: Annotated[
str,
"The city for which to get the weather, e.g. 'San Francisco'",
] = "Munich",
unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius",
) -> str:
return f"The weather in {location} is sunny. The temperature is 20 {unit}."
tool = create_tool_from_function(get_current_weather)
```
Create a new instance of `VertexAIGeminiChatGenerator` to set the tools:
```python
from haystack_integrations.components.generators.google_vertex import (
VertexAIGeminiChatGenerator,
)
gemini_chat = VertexAIGeminiChatGenerator(model="gemini-2.0-flash-exp", tools=[tool])
```
And then ask our question. The model prepares the tool call, your code executes it with `Tool.invoke`, and the results go back to the model for the final answer:
```python
from haystack.dataclasses import ChatMessage
messages = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")]
replies = gemini_chat.run(messages=messages)["replies"]
print(replies[0].tool_calls)
>>> [ToolCall(tool_name='get_current_weather',
>>> arguments={'unit': 'celsius', 'location': 'Berlin'}, id=None)]
tool_messages = []
for tool_call in replies[0].tool_calls:
result = tool.invoke(**tool_call.arguments)
tool_messages.append(ChatMessage.from_tool(tool_result=result, origin=tool_call))
messages = messages + replies + tool_messages
final_replies = gemini_chat.run(messages=messages)["replies"]
print(final_replies[0].text)
>>> The temperature in Berlin is 20 degrees Celsius.
```
### With an Agent
Instead of driving the tool call loop yourself, pass the generator and your tools to an [`Agent`](../agents-1/agent.mdx). It lets the model prepare tool calls, executes them, and feeds the results back until a final answer is ready:
```python
from haystack.components.agents import Agent
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.google_vertex import (
VertexAIGeminiChatGenerator,
)
agent = Agent(
chat_generator=VertexAIGeminiChatGenerator(model="gemini-2.0-flash-exp"),
tools=[tool],
)
result = agent.run(
messages=[ChatMessage.from_user("What is the temperature in celsius in Berlin?")]
)
print(result["last_message"].text)
>>> The temperature in Berlin is 20 degrees Celsius.
```
### In a pipeline
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack import Pipeline
from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator
# no parameter init, we don't use any runtime template variables
prompt_builder = ChatPromptBuilder()
gemini_chat = VertexAIGeminiChatGenerator()
pipe = Pipeline()
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("gemini", gemini_chat)
pipe.connect("prompt_builder.prompt", "gemini.messages")
location = "Rome"
messages = [ChatMessage.from_user("Tell me briefly about {{location}} history")]
res = pipe.run(data={"prompt_builder": {"template_variables":{"location": location}, "template": messages}})
print(res)
>>> - **753 B.C.:** Traditional date of the founding of Rome by Romulus and Remus.
>>> - **509 B.C.:** Establishment of the Roman Republic, replacing the Etruscan monarchy.
>>> - **492-264 B.C.:** Series of wars against neighboring tribes, resulting in the expansion of the Roman Republic's territory.
>>> - **264-146 B.C.:** Three Punic Wars against Carthage, resulting in the destruction of Carthage and the Roman Republic becoming the dominant power in the Mediterranean.
>>> - **133-73 B.C.:** Series of civil wars and slave revolts, leading to the rise of Julius Caesar.
>>> - **49 B.C.:** Julius Caesar crosses the Rubicon River, starting the Roman Civil War.
>>> - **44 B.C.:** Julius Caesar is assassinated, leading to the Second Triumvirate of Octavian, Mark Antony, and Lepidus.
>>> - **31 B.C.:** Battle of Actium, where Octavian defeats Mark Antony and Cleopatra, becoming the sole ruler of Rome.
>>> - **27 B.C.:** The Roman Republic is transformed into the Roman Empire, with Octavian becoming the first Roman emperor, known as Augustus.
>>> - **1st century A.D.:** The Roman Empire reaches its greatest extent, stretching from Britain to Egypt.
>>> - **3rd century A.D.:** The Roman Empire begins to decline, facing internal instability, invasions by Germanic tribes, and the rise of Christianity.
>>> - **476 A.D.:** The last Western Roman emperor, Romulus Augustulus, is overthrown by the Germanic leader Odoacer, marking the end of the Roman Empire in the West.
```
## Additional References
🧑🍳 Cookbook: [Function Calling and Multimodal QA with Gemini](https://haystack.deepset.ai/cookbook/vertexai-gemini-examples)
---
// File: pipeline-components/generators/vertexaigeminigenerator
# VertexAIGeminiGenerator
`VertexAIGeminiGenerator` enables text generation using Google Gemini models.
:::warning[Deprecation Notice]
This integration uses the deprecated google-generativeai SDK, which will lose support after August 2025.
We recommend switching to the new [GoogleGenAIChatGenerator](googlegenaichatgenerator.mdx) integration instead.
:::
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory run variables** | `parts`: A variadic list containing a mix of images, audio, video, and text to prompt Gemini |
| **Output variables** | `replies`: A list of strings or dictionaries with all the replies generated by the model |
| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
| **Package name** | `google-vertex-haystack` |
`VertexAIGeminiGenerator` supports `gemini-1.5-pro` and `gemini-1.5-flash`/ `gemini-2.0-flash` models. Note that [Google recommends upgrading](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions) from `gemini-1.5-pro` to `gemini-2.0-flash`.
For details on available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models.
:::info
To explore the full capabilities of Gemini check out this [article](https://haystack.deepset.ai/blog/gemini-models-with-google-vertex-for-haystack) and the related [Colab notebook](https://colab.research.google.com/drive/10SdXvH2ATSzqzA3OOmTM8KzD5ZdH_Q6Z?usp=sharing).
:::
### Parameters Overview
`VertexAIGeminiGenerator` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
You should install `google-vertex-haystack` package to use the `VertexAIGeminiGenerator`:
```shell
pip install google-vertex-haystack
```
### On its own
Basic usage:
```python
from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator
gemini = VertexAIGeminiGenerator()
result = gemini.run(parts = ["What is the most interesting thing you know?"])
for answer in result["replies"]:
print(answer)
>>> 1. **The Origin of Life:** How and where did life begin? The answers to this question are still shrouded in mystery, but scientists continuously uncover new insights into the remarkable story of our planet's earliest forms of life.
>>> 2. **The Unseen Universe:** The vast majority of the universe is comprised of matter and energy that we cannot directly observe. Dark matter and dark energy make up over 95% of the universe, yet we still don't fully understand their properties or how they influence the cosmos.
>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows two particles to become so intertwined that they share the same fate, regardless of how far apart they are. This has mind-bending implications for our understanding of reality and could potentially lead to advancements in communication and computing.
>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can pass at different rates for different observers. Astronauts traveling at high speeds, for example, experience time dilation relative to people on Earth. This phenomenon could have significant implications for future space travel.
>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the abundance of potential life-supporting planets, we have yet to find any concrete evidence of extraterrestrial life. This contradiction between scientific expectations and observational reality is known as the Fermi Paradox and remains one of the most intriguing mysteries in modern science.
>>> 6. **Biological Evolution:** The idea that life evolves over time through natural selection is one of the most profound and transformative scientific discoveries. It explains the diversity of life on Earth and provides insights into our own origins and the interconnectedness of all living things.
>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, known as neuroplasticity, is a remarkable phenomenon that has important implications for learning, memory, and recovery from brain injuries.
>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, refers to the range of distances from a star within which liquid water can exist on a planet's surface. This zone is critical for the potential existence of life as we know it and has been used to guide the search for exoplanets that could support life.
>>> 9. **String Theory:** This theoretical framework in physics aims to unify all the fundamental forces of nature into a single coherent theory. It suggests that the universe has extra dimensions beyond the familiar three spatial dimensions and time.
>>> 10. **Consciousness:** The nature of human consciousness and how it arises from the brain's physical processes remain one of the most profound and elusive mysteries in science. Understanding consciousness is crucial for unraveling the complexities of the human mind and our place in the universe.
```
Advanced usage, multi-modal prompting:
```python
import requests
from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator
URLS = [
"https://raw.githubusercontent.com/silvanocerza/robots/main/robot1.jpg",
"https://raw.githubusercontent.com/silvanocerza/robots/main/robot2.jpg",
"https://raw.githubusercontent.com/silvanocerza/robots/main/robot3.jpg",
"https://raw.githubusercontent.com/silvanocerza/robots/main/robot4.jpg"
]
images = [
ByteStream(data=requests.get(url).content, mime_type="image/jpeg")
for url in URLS
]
gemini = VertexAIGeminiGenerator()
result = gemini.run(parts = ["What can you tell me about this robots?", *images])
for answer in result["replies"]:
print(answer)
>>> The first image is of C-3PO and R2-D2 from the Star Wars franchise. C-3PO is a protocol droid, while R2-D2 is an astromech droid. They are both loyal companions to the heroes of the Star Wars saga.
>>> The second image is of Maria from the 1927 film Metropolis. Maria is a robot who is created to be the perfect woman. She is beautiful, intelligent, and obedient. However, she is also soulless and lacks any real emotions.
>>> The third image is of Gort from the 1951 film The Day the Earth Stood Still. Gort is a robot who is sent to Earth to warn humanity about the dangers of nuclear war. He is a powerful and intelligent robot, but he is also compassionate and understanding.
>>> The fourth image is of Marvin from the 1977 film The Hitchhiker's Guide to the Galaxy. Marvin is a robot who is depressed and pessimistic. He is constantly complaining about everything, but he is also very intelligent and has a dry sense of humor.
```
### In a pipeline
In a RAG pipeline:
```python
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.builders import PromptBuilder
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.generators.google_vertex import (
VertexAIGeminiGenerator,
)
docstore = InMemoryDocumentStore()
docstore.write_documents(
[
Document(content="Rome is the capital of Italy"),
Document(content="Paris is the capital of France"),
],
)
query = "What is the capital of France?"
template = """
Given the following information, answer the question.
Context:
{% for document in documents %}
{{ document.content }}
{% endfor %}
Question: {{ query }}?
"""
pipe = Pipeline()
pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("gemini", VertexAIGeminiGenerator())
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "gemini")
res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
print(res)
```
## Additional References
🧑🍳 Cookbook: [Function Calling and Multimodal QA with Gemini](https://haystack.deepset.ai/cookbook/vertexai-gemini-examples)
---
// File: pipeline-components/generators/vertexaiimagecaptioner
# VertexAIImageCaptioner
`VertexAIImageCaptioner` enables text generation using Google Vertex AI `imagetext` generative model.
| | |
| --- | --- |
| **Mandatory run variables** | `image`: A [`ByteStream`](../../concepts/data-classes.mdx#bytestream) object storing an image |
| **Output variables** | `captions`: A list of strings generated by the model |
| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
| **Package name** | `google-vertex-haystack` |
### Parameters Overview
`VertexAIImageCaptioner` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
## Usage
You need to install `google-vertex-haystack` package to use the `VertexAIImageCaptioner`:
```shell
pip install google-vertex-haystack
```
### On its own
Basic usage:
```python
import requests
from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIImageCaptioner
captioner = VertexAIImageCaptioner()
image = ByteStream(data=requests.get("https://raw.githubusercontent.com/silvanocerza/robots/main/robot1.jpg").content)
result = captioner.run(image=image)
for caption in result["captions"]:
print(caption)
>>> two gold robots are standing next to each other in the desert
```
You can also set the caption language and the number of results:
```python
import requests
from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIImageCaptioner
captioner = VertexAIImageCaptioner(
number_of_results=3, # Can't be greater than 3
language="it",
)
image = ByteStream(data=requests.get("https://raw.githubusercontent.com/silvanocerza/robots/main/robot1.jpg").content)
result = captioner.run(image=image)
for caption in result["captions"]:
print(caption)
>>> due robot dorati sono in piedi uno accanto all'altro in un deserto
>>> un c3p0 e un r2d2 stanno in piedi uno accanto all'altro in un deserto
>>> due robot dorati sono in piedi uno accanto all'altro
```
---
// File: pipeline-components/generators/vertexaiimagegenerator
# VertexAIImageGenerator
This component enables image generation using Google Vertex AI generative model.
| | |
| --- | --- |
| **Mandatory run variables** | `prompt`: A string containing the prompt for the model |
| **Output variables** | `images`: A list of [`ByteStream`](../../concepts/data-classes.mdx#bytestream) containing images generated by the model |
| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
| **Package name** | `google-vertex-haystack` |
`VertexAIImageGenerator` supports the `imagegeneration` model.
### Parameters Overview
`VertexAIImageGenerator` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
## Usage
You need to install `google-vertex-haystack` package to use the `VertexAIImageGenerator`:
```shell
pip install google-vertex-haystack
```
### On its own
Basic usage:
```python
from pathlib import Path
from haystack_integrations.components.generators.google_vertex import (
VertexAIImageGenerator,
)
generator = VertexAIImageGenerator()
result = generator.run(prompt="Generate an image of a cute cat")
result["images"][0].to_file(Path("my_image.png"))
```
You can also set other parameters like the number of images generated and the guidance scale to change the strength of the prompt.
Let’s also use a negative prompt to omit something from the image:
```python
from pathlib import Path
from haystack_integrations.components.generators.google_vertex import (
VertexAIImageGenerator,
)
generator = VertexAIImageGenerator(
number_of_images=3,
guidance_scale=12,
)
result = generator.run(
prompt="Generate an image of a cute cat",
negative_prompt="window, chair",
)
for i, image in enumerate(result["images"]):
images.to_file(Path(f"image_{i}.png"))
```
---
// File: pipeline-components/generators/vertexaiimageqa
# VertexAIImageQA
This component enables text generation (image captioning) using Google Vertex AI generative models.
| | |
| --- | --- |
| **Mandatory run variables** | `image`: A [`ByteStream`](../../concepts/data-classes.mdx#bytestream) containing an image data
`question`: A string of a question about the image | | **Output variables** | `replies`: A list of strings containing answers generated by the model | | **API reference** | [Google Vertex](/reference/integrations-google-vertex) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex | | **Package name** | `google-vertex-haystack` |
`VertexAIImageQA` supports the `imagetext` model.
### Parameters Overview
`VertexAIImageQA` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
## Usage
You need to install `google-vertex-haystack` package to use the `VertexAIImageQA`:
```shell
pip install google-vertex-haystack
```
### On its own
Basic usage:
```python
from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIImageQA
qa = VertexAIImageQA()
image = ByteStream.from_file_path("dog.jpg")
res = qa.run(image=image, question="What color is this dog")
print(res["replies"][0])
>>> white
```
You can also set the number of answers generated:
```python
from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIImageQA
qa = VertexAIImageQA(
number_of_results=3,
)
image = ByteStream.from_file_path("dog.jpg")
res = qa.run(image=image, question="Tell me something about this dog")
for answer in res["replies"]:
print(answer)
>>> pomeranian
>>> white
>>> pomeranian puppy
```
---
// File: pipeline-components/generators/vertexaitextgenerator
# VertexAITextGenerator
This component enables text generation using Google Vertex AI generative models.
`question`: A string of a question about the image | | **Output variables** | `replies`: A list of strings containing answers generated by the model | | **API reference** | [Google Vertex](/reference/integrations-google-vertex) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex | | **Package name** | `google-vertex-haystack` |
| | |
| --- | --- |
| **Mandatory run variables** | `prompt`: A string containing the prompt for the model |
| **Output variables** | `replies`: A list of strings containing answers generated by the model
`safety_attributes`: A dictionary containing scores for safety attributes
`citations`: A list of dictionaries containing grounding citations | | **API reference** | [Google Vertex](/reference/integrations-google-vertex) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex | | **Package name** | `google-vertex-haystack` |
`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models.
### Parameters Overview
`VertexAITextGenerator` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
## Usage
You need to install `google-vertex-haystack` package to use the `VertexAITextGenerator`:
```shell
pip install google-vertex-haystack
```
### On its own
Basic usage:
````python
from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator
generator = VertexAITextGenerator()
res = generator.run("Tell me a good interview question for a software engineer.")
print(res["replies"][0])
>>> **Question:** You are given a list of integers and a target sum. Find all unique combinations of numbers in the list that add up to the target sum.
>>>
>>> **Example:**
>>>
>>> ```
>>> Input: [1, 2, 3, 4, 5], target = 7
>>> Output: [[1, 2, 4], [3, 4]]
>>> ```
>>>
>>> **Follow-up:** What if the list contains duplicate numbers?
````
You can also set other parameters like the number of answers generated, temperature to control the randomness, and stop sequences to stop generation. For a full list of possible parameters, see the documentation of [`TextGenerationModel.predict()`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.language_models.TextGenerationModel#vertexai_language_models_TextGenerationModel_predict).
```python
from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator
generator = VertexAITextGenerator(
candidate_count=3,
temperature=0.2,
stop_sequences=["example", "Example"],
)
res = generator.run("Tell me a good interview question for a software engineer.")
for answer in res["replies"]:
print(answer)
print("-----")
>>> **Question:** You are given a list of integers, and you need to find the longest increasing subsequence. What is the most efficient algorithm to solve this problem?
>>> -----
>>> **Question:** You are given a list of integers and a target sum. Find all unique combinations in the list that sum up to the target sum. The same number can be used multiple times in a combination.
>>> -----
>>> **Question:** You are given a list of integers and a target sum. Find all unique combinations of numbers in the list that add up to the target sum.
>>> -----
```
---
// File: pipeline-components/generators/vllmchatgenerator
# VLLMChatGenerator
This component enables chat completion using models served with [vLLM](https://docs.vllm.ai/).
`safety_attributes`: A dictionary containing scores for safety attributes
`citations`: A list of dictionaries containing grounding citations | | **API reference** | [Google Vertex](/reference/integrations-google-vertex) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex | | **Package name** | `google-vertex-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `model`: The name of the model served by vLLM |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **API reference** | [vLLM](/reference/integrations-vllm) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/vllm |
| **Package name** | `vllm-haystack` |
## Overview
[vLLM](https://docs.vllm.ai/) is a high-throughput and memory-efficient inference and serving engine for LLMs. It exposes an OpenAI-compatible HTTP server, which `VLLMChatGenerator` uses to run chat completions.
`VLLMChatGenerator` expects a vLLM server to be running and accessible at the `api_base_url` parameter (by default, `http://localhost:8000/v1`). The component needs a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
You can pass any text generation parameters valid for the vLLM OpenAI-compatible Chat Completion API directly to this component using the `generation_kwargs` parameter in `__init__` or in the `run` method. vLLM-specific parameters not part of the standard OpenAI API (such as `top_k`, `min_tokens`, `repetition_penalty`) can be passed through `generation_kwargs["extra_body"]`. For more details, see the [vLLM documentation](https://docs.vllm.ai/en/stable/serving/openai_compatible_server/).
If the vLLM server was started with `--api-key`, provide the API key through the `VLLM_API_KEY` environment variable or the `api_key` init parameter using Haystack's [Secret](../../concepts/secret-management.mdx) API.
### Tool Support
`VLLMChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
- **A list of Tool objects**: Pass individual tools as a list
- **A single Toolset**: Pass an entire Toolset directly
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
This allows you to organize related tools into logical groups while also including standalone tools as needed.
For tool calling to work, the vLLM server must be started with `--enable-auto-tool-choice` and `--tool-call-parser`. The available tool call parsers depend on the model. See the [vLLM tool calling docs](https://docs.vllm.ai/en/stable/features/tool_calling/) for the full list.
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
### Streaming
`VLLMChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM, allowing tokens to be emitted as they are generated. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
### Reasoning models
`VLLMChatGenerator` supports reasoning models. To use them, start the vLLM server with the appropriate `--reasoning-parser`. The reasoning content produced by the model is exposed in the `reasoning` field of the returned `ChatMessage`.
## Usage
Install the `vllm-haystack` package to use the `VLLMChatGenerator`:
```shell
pip install vllm-haystack
```
### Starting the vLLM server
Before using this component, start a vLLM server:
```bash
vllm serve Qwen/Qwen3-4B-Instruct-2507
```
For reasoning models, start the server with the appropriate reasoning parser:
```bash
vllm serve Qwen/Qwen3-0.6B --reasoning-parser qwen3
```
For tool calling, start the server with `--enable-auto-tool-choice` and `--tool-call-parser`:
```bash
vllm serve Qwen/Qwen3-0.6B --enable-auto-tool-choice --tool-call-parser hermes
```
For details on server options, see the [vLLM CLI docs](https://docs.vllm.ai/en/stable/cli/serve/).
### On its own
Basic usage:
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.vllm import VLLMChatGenerator
generator = VLLMChatGenerator(
model="Qwen/Qwen3-4B-Instruct-2507",
generation_kwargs={"max_tokens": 512, "temperature": 0.7},
)
messages = [ChatMessage.from_user("What's Natural Language Processing?")]
response = generator.run(messages=messages)
print(response["replies"][0].text)
```
### With vLLM-specific parameters
Pass vLLM-specific parameters through the `generation_kwargs["extra_body"]` dictionary:
```python
from haystack_integrations.components.generators.vllm import VLLMChatGenerator
generator = VLLMChatGenerator(
model="Qwen/Qwen3-4B-Instruct-2507",
generation_kwargs={
"max_tokens": 512,
"extra_body": {
"top_k": 50,
"min_tokens": 10,
"repetition_penalty": 1.1,
},
},
)
```
### With tool calling
Start the vLLM server with `--enable-auto-tool-choice` and `--tool-call-parser`, then:
```python
from haystack.dataclasses import ChatMessage
from haystack.tools import tool
from haystack_integrations.components.generators.vllm import VLLMChatGenerator
@tool
def weather(city: str) -> str:
"""Get the weather in a given city."""
return f"The weather in {city} is sunny"
generator = VLLMChatGenerator(model="Qwen/Qwen3-0.6B", tools=[weather])
messages = [ChatMessage.from_user("What is the weather in Paris?")]
response = generator.run(messages=messages)
print(response["replies"][0].tool_calls)
```
### With reasoning models
Start the vLLM server with `--reasoning-parser`, then:
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.vllm import VLLMChatGenerator
generator = VLLMChatGenerator(model="Qwen/Qwen3-0.6B")
messages = [ChatMessage.from_user("Solve step by step: what is 15 * 37?")]
response = generator.run(messages=messages)
reply = response["replies"][0]
if reply.reasoning:
print("Reasoning:", reply.reasoning.reasoning_text)
print("Answer:", reply.text)
```
### In a pipeline
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.vllm import VLLMChatGenerator
prompt_builder = ChatPromptBuilder()
llm = VLLMChatGenerator(model="Qwen/Qwen3-4B-Instruct-2507")
pipe = Pipeline()
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("prompt_builder.prompt", "llm.messages")
messages = [
ChatMessage.from_system("Give brief answers."),
ChatMessage.from_user("Tell me about {{city}}"),
]
response = pipe.run(
data={
"prompt_builder": {
"template": messages,
"template_variables": {"city": "Berlin"},
},
},
)
print(response)
```
---
// File: pipeline-components/generators/watsonxchatgenerator
# WatsonxChatGenerator
Use this component with IBM watsonx models like `granite-4-h-small` for chat generation.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: The IBM Cloud API key. Can be set with `WATSONX_API_KEY` env var.
`project_id`: The IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. | | **Mandatory run variables** | `messages` A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **API reference** | [Watsonx](/reference/integrations-watsonx) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx | | **Package name** | `watsonx-haystack` |
This integration supports IBM watsonx.ai foundation models such as `ibm/granite-4-h-small`, `meta-llama/llama-3-3-70b-instruct`, `mistralai/mistral-small-3-1-24b-instruct-2503`, and similar. These models provide high-quality chat completion capabilities through IBM's cloud platform. Check out the most recent full list in the [IBM watsonx.ai documentation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models-ibm.html?context=wx).
## Overview
`WatsonxChatGenerator` needs IBM Cloud credentials to work. You can set these in:
- The `api_key` and `project_id` init parameters using [Secret API](../../concepts/secret-management.mdx)
- The `WATSONX_API_KEY` and `WATSONX_PROJECT_ID` environment variables (recommended)
Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the IBM watsonx.ai API directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the IBM watsonx.ai API, refer to the [IBM watsonx.ai documentation](https://cloud.ibm.com/apidocs/watsonx-ai).
Finally, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
You need to install `watsonx-haystack` package to use the `WatsonxChatGenerator`:
```shell
pip install watsonx-haystack
```
#### On its own
```python
from haystack_integrations.components.generators.watsonx.chat.chat_generator import (
WatsonxChatGenerator,
)
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
generator = WatsonxChatGenerator(
api_key=Secret.from_env_var("WATSONX_API_KEY"),
project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
model="ibm/granite-4-h-small",
)
message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
print(generator.run(messages=[message]))
```
With multimodal inputs:
```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack_integrations.components.generators.watsonx.chat.chat_generator import (
WatsonxChatGenerator,
)
# Use a multimodal model
llm = WatsonxChatGenerator(model="meta-llama/llama-3-2-11b-vision-instruct")
image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(
content_parts=["What does the image show? Max 5 words.", image],
)
response = llm.run(messages=[user_message])["replies"][0].text
print(response)
# Red apple on straw.
```
#### In a Pipeline
You can also use `WatsonxChatGenerator` to use IBM watsonx.ai chat models in your pipeline.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.watsonx.chat.chat_generator import (
WatsonxChatGenerator,
)
from haystack.utils import Secret
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component(
"llm",
WatsonxChatGenerator(
api_key=Secret.from_env_var("WATSONX_API_KEY"),
project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
model="ibm/granite-4-h-small",
),
)
pipe.connect("prompt_builder", "llm")
country = "Germany"
system_message = ChatMessage.from_system(
"You are an assistant giving out valuable information to language learners.",
)
messages = [
system_message,
ChatMessage.from_user("What's the official language of {{ country }}?"),
]
res = pipe.run(
data={
"prompt_builder": {
"template_variables": {"country": country},
"template": messages,
},
},
)
print(res)
```
---
// File: pipeline-components/generators/watsonxgenerator
# WatsonxGenerator
Use this component with IBM watsonx models like `granite-3-2b-instruct` for simple text generation tasks.
:::warning[Deprecation Notice]
`WatsonxGenerator` is deprecated and will be removed in a future version.
We recommend switching to [WatsonxChatGenerator](watsonxchatgenerator.mdx) instead,
which also accepts a plain string as input.
:::
`project_id`: The IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. | | **Mandatory run variables** | `messages` A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | **API reference** | [Watsonx](/reference/integrations-watsonx) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx | | **Package name** | `watsonx-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [PromptBuilder](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: An IBM Cloud API key. Can be set with `WATSONX_API_KEY` env var.
`project_id`: An IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. | | **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM | | **Output variables** | `replies`: A list of strings with all the replies generated by the LLM
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on | | **API reference** | [Watsonx](/reference/integrations-watsonx) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx | | **Package name** | `watsonx-haystack` |
## Overview
This integration supports IBM watsonx.ai foundation models such as `ibm/granite-13b-chat-v2`, `ibm/llama-2-70b-chat`, `ibm/llama-3-70b-instruct`, and similar. These models provide high-quality text generation capabilities through IBM's cloud platform. Check out the most recent full list in the [IBM watsonx.ai documentation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models-ibm.html?context=wx).
### Parameters
`WatsonxGenerator` needs IBM Cloud credentials to work. You can provide these in:
- The `WATSONX_API_KEY` environment variable (recommended)
- The `WATSONX_PROJECT_ID` environment variable (recommended)
- The `api_key` and `project_id` init parameters using Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`
Set your preferred IBM watsonx.ai model in the `model` parameter when initializing the component. The default model is `ibm/granite-3-2b-instruct`.
`WatsonxGenerator` requires a prompt to generate text, but you can pass any text generation parameters available in the IBM watsonx.ai API directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the IBM watsonx.ai API, see [IBM watsonx.ai documentation](https://cloud.ibm.com/apidocs/watsonx-ai).
The component also supports system prompts that can be set at initialization or passed during runtime to provide context or instructions for the generation.
Finally, the component run method requires a single string prompt to generate text.
### Streaming
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
## Usage
Install the `watsonx-haystack` package to use the `WatsonxGenerator`:
```shell
pip install watsonx-haystack
```
### On its own
```python
from haystack_integrations.components.generators.watsonx.generator import (
WatsonxGenerator,
)
from haystack.utils import Secret
generator = WatsonxGenerator(
api_key=Secret.from_env_var("WATSONX_API_KEY"),
project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
)
print(generator.run("What's Natural Language Processing? Be brief."))
```
### In a pipeline
You can also use `WatsonxGenerator` with the IBM watsonx.ai models in your pipeline.
```python
from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack_integrations.components.generators.watsonx.generator import (
WatsonxGenerator,
)
from haystack.utils import Secret
template = """
You are an assistant giving out valuable information to language learners.
Answer this question, be brief.
Question: {{ query }}?
"""
pipe = Pipeline()
pipe.add_component("prompt_builder", PromptBuilder(template))
pipe.add_component(
"llm",
WatsonxGenerator(
api_key=Secret.from_env_var("WATSONX_API_KEY"),
project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
),
)
pipe.connect("prompt_builder", "llm")
query = "What language is spoken in Germany?"
res = pipe.run(data={"prompt_builder": {"query": query}})
print(res)
```
---
// File: pipeline-components/generators
# Generators
Generators are responsible for generating text after you give them a prompt. They are specific for each LLM technology (OpenAI, local, TGI and others).
| Generator | Description | Streaming Support |
| --- | --- | --- |
| [AmazonBedrockChatGenerator](generators/amazonbedrockchatgenerator.mdx) | Enables chat completion using models through Amazon Bedrock service. | ✅ |
| [AmazonBedrockGenerator](generators/amazonbedrockgenerator.mdx) | Enables text generation using models through Amazon Bedrock service. | ✅ |
| [AIMLAPIChatGenerator](generators/aimllapichatgenerator.mdx) | Enables chat completion using AI models through the AIMLAPI. | ✅ |
| [AnthropicChatGenerator](generators/anthropicchatgenerator.mdx) | This component enables chat completions using Anthropic large language models (LLMs). | ✅ |
| [AnthropicFoundryChatGenerator](generators/anthropicfoundrychatgenerator.mdx) | This component enables chat completions using Anthropic models served through Azure Foundry. | ✅ |
| [AnthropicVertexChatGenerator](generators/anthropicvertexchatgenerator.mdx) | This component enables chat completions using AnthropicVertex API. | ✅ |
| [AnthropicGenerator](generators/anthropicgenerator.mdx) | This component enables text completions using Anthropic large language models (LLMs). | ✅ |
| [AzureOpenAIChatGenerator](generators/azureopenaichatgenerator.mdx) | Enables chat completion using OpenAI's LLMs through Azure services. | ✅ |
| [AzureOpenAIResponsesChatGenerator](generators/azureopenairesponseschatgenerator.mdx) | Enables chat completion using OpenAI's Responses API through Azure services with support for reasoning models. | ✅ |
| [CohereChatGenerator](generators/coherechatgenerator.mdx) | Enables chat completion using Cohere's LLMs. | ✅ |
| [CohereGenerator](generators/coheregenerator.mdx) | Queries the LLM using Cohere API. | ✅ |
| [CometAPIChatGenerator](generators/cometapichatgenerator.mdx) | Enables chat completion using AI models through the Comet API. | ✅ |
| [EdenAIChatGenerator](generators/edenaichatgenerator.mdx) | Enables chat completion using 500+ models through the Eden AI gateway. | ✅ |
| [FallbackChatGenerator](generators/fallbackchatgenerator.mdx) | A ChatGenerator wrapper that tries multiple Chat Generators sequentially until one succeeds. | ✅ |
| [GoogleAIGeminiChatGenerator](generators/googleaigeminichatgenerator.mdx) | Enables chat completion using Google Gemini models. **_This integration will be deprecated soon. We recommend using [GoogleGenAIChatGenerator](generators/googlegenaichatgenerator.mdx) integration instead._** | ✅ |
| [GoogleAIGeminiGenerator](generators/googleaigeminigenerator.mdx) | Enables text generation using Google Gemini models. **_This integration will be deprecated soon. We recommend using [GoogleGenAIChatGenerator](generators/googlegenaichatgenerator.mdx) integration instead._** | ✅ |
| [GoogleGenAIChatGenerator](generators/googlegenaichatgenerator.mdx) | Enables chat completion using Google Gemini models through Google Gen AI SDK. | ✅ |
| [HuggingFaceAPIChatGenerator](generators/huggingfaceapichatgenerator.mdx) | Enables chat completion using various Hugging Face APIs. | ✅ |
| [TransformersChatGenerator](generators/transformerschatgenerator.mdx) | Provides an interface for chat completion using a Hugging Face model that runs locally. | ✅ |
| [LiteLLMChatGenerator](generators/litellmchatgenerator.mdx) | Enables chat completion using various LLM providers through LiteLLM. | ✅ |
| [LlamaCppChatGenerator](generators/llamacppchatgenerator.mdx) | Enables chat completion using an LLM running on Llama.cpp. | ✅ |
| [LlamaCppGenerator](generators/llamacppgenerator.mdx) | Generate text using an LLM running with Llama.cpp. | ❌ |
| [LlamaStackChatGenerator](generators/llamastackchatgenerator.mdx) | Enables chat completions using an LLM model made available via Llama Stack server | ✅ |
| [MetaLlamaChatGenerator](generators/metallamachatgenerator.mdx) | Enables chat completion with any model hosted available with Meta Llama API. | ✅ |
| [MistralChatGenerator](generators/mistralchatgenerator.mdx) | Enables chat completion using Mistral's text generation models. | ✅ |
| [MockChatGenerator](generators/mockchatgenerator.mdx) | Returns predefined responses without calling any API — a deterministic, zero-cost stand-in for real Chat Generators in tests and prototypes. | ✅ |
| [NvidiaChatGenerator](generators/nvidiachatgenerator.mdx) | Enables chat completion using Nvidia-hosted models. | ✅ |
| [NvidiaGenerator](generators/nvidiagenerator.mdx) | Provides an interface for generating text using LLMs self-hosted with NVIDIA NIM or models hosted on the NVIDIA API catalog. | ❌ |
| [OllamaChatGenerator](generators/ollamachatgenerator.mdx) | Enables chat completion using an LLM running on Ollama. | ✅ |
| [OllamaGenerator](generators/ollamagenerator.mdx) | Provides an interface to generate text using an LLM running on Ollama. | ✅ |
| [OpenAIChatGenerator](generators/openaichatgenerator.mdx) | Enables chat completion using OpenAI's large language models (LLMs). | ✅ |
| [OpenAIImageGenerator](generators/openaiimagegenerator.mdx) | Generate images using OpenAI's image generation models such as `gpt-image-2`. | ❌ |
| [OpenAIResponsesChatGenerator](generators/openairesponseschatgenerator.mdx) | Enables chat completion using OpenAI's Responses API with support for reasoning models. | ✅ |
| [OpenRouterChatGenerator](generators/openrouterchatgenerator.mdx) | Enables chat completion with any model hosted on OpenRouter. | ✅ |
| [OrcaRouterChatGenerator](generators/orcarouterchatgenerator.mdx) | Enables chat completion using models routed through OrcaRouter. | ✅ |
| [PerplexityChatGenerator](generators/perplexitychatgenerator.mdx) | Enables chat completion using models via the Perplexity Agent API. | ✅ |
| [SagemakerGenerator](generators/sagemakergenerator.mdx) | Enables text generation using LLMs deployed on Amazon Sagemaker. | ❌ |
| [STACKITChatGenerator](generators/stackitchatgenerator.mdx) | Enables chat completions using the STACKIT API. | ✅ |
| [TogetherAIChatGenerator](generators/togetheraichatgenerator.mdx) | Enables chat completion using models hosted on Together AI. | ✅ |
| [TogetherAIGenerator](generators/togetheraigenerator.mdx) | Enables text generation using models hosted on Together AI. | ✅ |
| [VertexAICodeGenerator](generators/vertexaicodegenerator.mdx) | Enables code generation using Google Vertex AI generative model. | ❌ |
| [VertexAIGeminiChatGenerator](generators/vertexaigeminichatgenerator.mdx) | Enables chat completion using Google Gemini models with GCP Vertex AI. **_This integration will be deprecated soon. We recommend using [GoogleGenAIChatGenerator](generators/googlegenaichatgenerator.mdx) integration instead._** | ✅ |
| [VertexAIGeminiGenerator](generators/vertexaigeminigenerator.mdx) | Enables text generation using Google Gemini models with GCP Vertex AI. **_This integration will be deprecated soon. We recommend using [GoogleGenAIChatGenerator](generators/googlegenaichatgenerator.mdx) integration instead._** | ✅ |
| [VertexAIImageCaptioner](generators/vertexaiimagecaptioner.mdx) | Enables text generation using Google Vertex AI `imagetext` generative model. | ❌ |
| [VertexAIImageGenerator](generators/vertexaiimagegenerator.mdx) | Enables image generation using Google Vertex AI generative model. | ❌ |
| [VertexAIImageQA](generators/vertexaiimageqa.mdx) | Enables text generation (image captioning) using Google Vertex AI generative models. | ❌ |
| [VertexAITextGenerator](generators/vertexaitextgenerator.mdx) | Enables text generation using Google Vertex AI generative models. | ❌ |
| [VLLMChatGenerator](generators/vllmchatgenerator.mdx) | Enables chat completion using models served with vLLM. | ✅ |
| [WatsonxGenerator](generators/watsonxgenerator.mdx) | Enables text generation with IBM Watsonx models. | ✅ |
| [WatsonxChatGenerator](generators/watsonxchatgenerator.mdx) | Enables chat completions with IBM Watsonx models. | ✅ |
---
// File: pipeline-components/joiners/answerjoiner
# AnswerJoiner
Merges multiple answers from different Generators into a single list.
`project_id`: An IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. | | **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM | | **Output variables** | `replies`: A list of strings with all the replies generated by the LLM
`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on | | **API reference** | [Watsonx](/reference/integrations-watsonx) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx | | **Package name** | `watsonx-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In query pipelines, after [Generators](../generators.mdx) and, subsequently, components that return a list of answers such as [`AnswerBuilder`](../builders/answerbuilder.mdx) |
| **Mandatory run variables** | `answers`: A nested list of answers to be merged, received from the Generator. This input is `variadic`, meaning you can connect a variable number of components to it. |
| **Output variables** | `answers`: A merged list of answers |
| **API reference** | [Joiners](/reference/joiners-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/joiners/answer_joiner.py |
| **Package name** | `haystack-ai` |
## Overvew
`AnswerJoiner` joins input lists of [`Answer`](../../concepts/data-classes.mdx#answer) objects from multiple connections and returns them as one list.
You can optionally set the `top_k` parameter, which specifies the maximum number of answers to return. If you don’t set this parameter, the component returns all answers it receives.
## Usage
In this simple example pipeline, the `AnswerJoiner` merges answers from two instances of Generators:
```python
from haystack.components.builders import AnswerBuilder
from haystack.components.joiners import AnswerJoiner
from haystack.core.pipeline import Pipeline
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
query = "What's Natural Language Processing?"
messages = [
ChatMessage.from_system(
"You are a helpful, respectful and honest assistant. Be super concise.",
),
ChatMessage.from_user(query),
]
pipe = Pipeline()
pipe.add_component("gpt-4o", OpenAIChatGenerator(model="gpt-4o"))
pipe.add_component("llama", OpenAIChatGenerator())
pipe.add_component("aba", AnswerBuilder())
pipe.add_component("abb", AnswerBuilder())
pipe.add_component("joiner", AnswerJoiner())
pipe.connect("gpt-4o.replies", "aba")
pipe.connect("llama.replies", "abb")
pipe.connect("aba.answers", "joiner")
pipe.connect("abb.answers", "joiner")
results = pipe.run(
data={
"gpt-4o": {"messages": messages},
"llama": {"messages": messages},
"aba": {"query": query},
"abb": {"query": query},
},
)
```
---
// File: pipeline-components/joiners/branchjoiner
import ClickableImage from "@site/src/components/ClickableImage";
# BranchJoiner
Use this component to join different branches of a pipeline into a single output.
| | |
| --- | --- |
| **Most common position in a pipeline** | Flexible: Can appear at the beginning of a pipeline or at the start of loops. |
| **Mandatory init variables** | `type_`: The type of data expected from preceding components |
| **Mandatory run variables** | `**kwargs`: Any input data type defined at the initialization. This input is variadic, meaning you can connect a variable number of components to it. |
| **Output variables** | `value`: The first value received from the connected components. |
| **API reference** | [Joiners](/reference/joiners-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/joiners/branch.py |
| **Package name** | `haystack-ai` |
## Overview
`BranchJoiner` joins multiple branches in a pipeline, allowing their outputs to be reconciled into a single branch. This is especially useful in pipelines with multiple branches that need to be unified before moving to the single component that comes next.
`BranchJoiner` receives multiple data connections of the same type from other components and passes the first value it receives to its single output. This makes it essential for closing loops in pipelines or reconciling multiple branches from a decision component.
`BranchJoiner` can handle only one input of one data type, declared in the `__init__` function. It ensures that the data type remains consistent across the pipeline branches. If more than one value is received for the input when `run` is invoked, the component will raise an error:
```python
from haystack.components.joiners import BranchJoiner
bj = BranchJoiner(int)
bj.run(value=[3, 4, 5])
# ValueError: BranchJoiner expects only one input, but 3 were received.
```
## Usage
### On its own
Although only one input value is allowed at every run, due to its variadic nature `BranchJoiner` still expects a list. As an example:
```python
from haystack.components.joiners import BranchJoiner
# an example where input and output are strings
bj = BranchJoiner(str)
bj.run(value=["hello"])
# {"value" : "hello"}
# an example where input and output are integers
bj = BranchJoiner(int)
bj.run(value=[3])
# {"value": 3}
```
### In a pipeline
#### Enabling loops
Below is an example where `BranchJoiner` is used for closing a loop. In this example, `BranchJoiner` receives a looped-back list of `ChatMessage` objects from the `JsonSchemaValidator` and sends it down to the `OpenAIChatGenerator` for re-generation.
```python
import json
from haystack import Pipeline
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.joiners import BranchJoiner
from haystack.components.validators import JsonSchemaValidator
from haystack.dataclasses import ChatMessage
person_schema = {
"type": "object",
"properties": {
"first_name": {"type": "string", "pattern": "^[A-Z][a-z]+$"},
"last_name": {"type": "string", "pattern": "^[A-Z][a-z]+$"},
"nationality": {
"type": "string",
"enum": ["Italian", "Portuguese", "American"],
},
},
"required": ["first_name", "last_name", "nationality"],
}
# Initialize a pipeline
pipe = Pipeline()
# Add components to the pipeline
pipe.add_component("joiner", BranchJoiner(list[ChatMessage]))
pipe.add_component("fc_llm", OpenAIChatGenerator(model="gpt-4.1-mini"))
pipe.add_component("validator", JsonSchemaValidator(json_schema=person_schema))
# Connect components
pipe.connect("joiner", "fc_llm")
pipe.connect("fc_llm.replies", "validator.messages")
pipe.connect("validator.validation_error", "joiner")
result = pipe.run(
data={
"fc_llm": {"generation_kwargs": {"response_format": {"type": "json_object"}}},
"joiner": {
"value": [ChatMessage.from_user("Create json object from Peter Parker")],
},
},
)
print(json.loads(result["validator"]["validated"][0].text))
# Output:
# {'first_name': 'Peter', 'last_name': 'Parker', 'nationality': 'American', 'name': 'Spider-Man', 'occupation':
# 'Superhero', 'age': 23, 'location': 'New York City'}
```
Expand to see the pipeline graph
Expand to see the pipeline graph
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing and query pipelines, after components that return a list of documents such as multiple [Retrievers](../retrievers.mdx) or multiple [Converters](../converters.mdx) |
| **Mandatory run variables** | `documents`: A list of documents. This input is `variadic`, meaning you can connect a variable number of components to it. |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Joiners](/reference/joiners-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/joiners/document_joiner.py |
| **Package name** | `haystack-ai` |
## Overview
`DocumentJoiner` joins input lists of documents from multiple connections and outputs them as one list. You can choose how you want the lists to be joined by specifying the `join_mode`. There are four options available:
- `concatenate` - Combines document from multiple components, discarding any duplicates. documents get their scores from the last component in the pipeline that assigns scores. This mode doesn’t influence document scores.
- `merge` - Merges the scores of duplicate documents coming from multiple components. You can also assign a weight to the scores to influence how they’re merged and set the top_k limit to specify how many documents you want `DocumentJoiner` to return.
- `reciprocal_rank_fusion`- Combines documents into a single list based on their ranking received from multiple components. It then calculates a new score based on the ranks of documents in the input lists. If the same Document appears in more than one list (was returned by multiple components), it gets a higher score.
- `distribution_based_rank_fusion` – Combines rankings from multiple sources into a single, unified ranking. It analyzes how scores are spread out and normalizes them, ensuring that each component's scoring method is taken into account. This normalization helps to balance the influence of each component, resulting in a more robust and fair combined ranking. If a document appears in multiple lists, its final score is adjusted based on the distribution of scores from all lists.
## Usage
### On its own
Below is an example where we are using the `DocumentJoiner` to merge two lists of documents. We run the `DocumentJoiner` and provide the documents. It returns a list of documents ranked by combined scores. By default, equal weight is given to each Retriever score. You could also use custom weights by setting the weights parameter to a list of floats with one weight per input component.
```python
from haystack import Document
from haystack.components.joiners.document_joiner import DocumentJoiner
docs_1 = [
Document(content="Paris is the capital of France.", score=0.5),
Document(content="Berlin is the capital of Germany.", score=0.4),
]
docs_2 = [
Document(content="Paris is the capital of France.", score=0.6),
Document(content="Rome is the capital of Italy.", score=0.5),
]
joiner = DocumentJoiner(join_mode="merge")
joiner.run(documents=[docs_1, docs_2])
# {'documents': [Document(id=0f5beda04153dbfc462c8b31f8536749e43654709ecf0cfe22c6d009c9912214, content: 'Paris is the capital of France.', score: 0.55), Document(id=424beed8b549a359239ab000f33ca3b1ddb0f30a988bbef2a46597b9c27e42f2, content: 'Rome is the capital of Italy.', score: 0.25), Document(id=312b465e77e25c11512ee76ae699ce2eb201f34c8c51384003bb367e24fb6cf8, content: 'Berlin is the capital of Germany.', score: 0.2)]}
```
### In a pipeline
#### Hybrid Retrieval
Below is an example of a hybrid retrieval pipeline that retrieves documents from an `InMemoryDocumentStore` based on keyword search (using `InMemoryBM25Retriever`) and embedding search (using `InMemoryEmbeddingRetriever`). It then uses the `DocumentJoiner` with its default join mode to concatenate the retrieved documents into one list. The Document Store must contain documents with embeddings, otherwise the `InMemoryEmbeddingRetriever` will not return any documents.
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack.components.joiners.document_joiner import DocumentJoiner
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import (
InMemoryBM25Retriever,
InMemoryEmbeddingRetriever,
)
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
)
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component(
instance=InMemoryBM25Retriever(document_store=document_store),
name="bm25_retriever",
)
p.add_component(
instance=SentenceTransformersTextEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
),
name="text_embedder",
)
p.add_component(
instance=InMemoryEmbeddingRetriever(document_store=document_store),
name="embedding_retriever",
)
p.add_component(instance=DocumentJoiner(), name="joiner")
p.connect("bm25_retriever", "joiner")
p.connect("embedding_retriever", "joiner")
p.connect("text_embedder", "embedding_retriever")
query = "What is the capital of France?"
p.run(data={"bm25_retriever": {"query": query}, "text_embedder": {"text": query}})
```
#### Indexing
Here's an example of an indexing pipeline that uses `DocumentJoiner` to compile all files into a single list of documents that can be fed through the rest of the indexing pipeline as one.
```python
from haystack.components.writers import DocumentWriter
from haystack.components.converters import (
MarkdownToDocument,
PyPDFToDocument,
TextFileToDocument,
)
from haystack.components.preprocessors import DocumentSplitter, DocumentCleaner
from haystack.components.routers import FileTypeRouter
from haystack.components.joiners import DocumentJoiner
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
)
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from pathlib import Path
document_store = InMemoryDocumentStore()
file_type_router = FileTypeRouter(
mime_types=["text/plain", "application/pdf", "text/markdown"],
)
text_file_converter = TextFileToDocument()
markdown_converter = MarkdownToDocument()
pdf_converter = PyPDFToDocument()
document_joiner = DocumentJoiner()
document_cleaner = DocumentCleaner()
document_splitter = DocumentSplitter(
split_by="word",
split_length=150,
split_overlap=50,
)
document_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
document_writer = DocumentWriter(document_store)
preprocessing_pipeline = Pipeline()
preprocessing_pipeline.add_component(instance=file_type_router, name="file_type_router")
preprocessing_pipeline.add_component(
instance=text_file_converter,
name="text_file_converter",
)
preprocessing_pipeline.add_component(
instance=markdown_converter,
name="markdown_converter",
)
preprocessing_pipeline.add_component(instance=pdf_converter, name="pypdf_converter")
preprocessing_pipeline.add_component(instance=document_joiner, name="document_joiner")
preprocessing_pipeline.add_component(instance=document_cleaner, name="document_cleaner")
preprocessing_pipeline.add_component(
instance=document_splitter,
name="document_splitter",
)
preprocessing_pipeline.add_component(
instance=document_embedder,
name="document_embedder",
)
preprocessing_pipeline.add_component(instance=document_writer, name="document_writer")
preprocessing_pipeline.connect(
"file_type_router.text/plain",
"text_file_converter.sources",
)
preprocessing_pipeline.connect(
"file_type_router.application/pdf",
"pypdf_converter.sources",
)
preprocessing_pipeline.connect(
"file_type_router.text/markdown",
"markdown_converter.sources",
)
preprocessing_pipeline.connect("text_file_converter", "document_joiner")
preprocessing_pipeline.connect("pypdf_converter", "document_joiner")
preprocessing_pipeline.connect("markdown_converter", "document_joiner")
preprocessing_pipeline.connect("document_joiner", "document_cleaner")
preprocessing_pipeline.connect("document_cleaner", "document_splitter")
preprocessing_pipeline.connect("document_splitter", "document_embedder")
preprocessing_pipeline.connect("document_embedder", "document_writer")
preprocessing_pipeline.run(
{"file_type_router": {"sources": list(Path(output_dir).glob("**/*"))}},
)
```
## Additional References :notebook: Tutorial: [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline) --- // File: pipeline-components/joiners/listjoiner # ListJoiner A component that joins multiple lists into a single flat list.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing and query pipelines, after components that return lists of documents such as multiple [Retrievers](../retrievers.mdx) or multiple [Converters](../converters.mdx) |
| **Mandatory run variables** | `values`: The dictionary of lists to be joined |
| **Output variables** | `values`: A dictionary with a `values` key containing the joined list |
| **API reference** | [Joiners](/reference/joiners-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/joiners/list_joiner.py |
| **Package name** | `haystack-ai` |
## Overview
The `ListJoiner` component combines multiple lists into one list. It is useful for combining multiple lists from different pipeline components, merging LLM responses, handling multi-step data processing, and gathering data from different sources into one list.
The items stay in order based on when each input list was processed in a pipeline.
You can optionally specify a `list_type_` parameter to set the expected type of the lists being joined (for example, `List[ChatMessage]`). If not set, `ListJoiner` will accept lists containing mixed data types.
## Usage
### On its own
```python
from haystack.components.joiners import ListJoiner
list1 = ["Hello", "world"]
list2 = ["This", "is", "Haystack"]
list3 = ["ListJoiner", "Example"]
joiner = ListJoiner()
result = joiner.run(values=[list1, list2, list3])
print(result["values"])
```
### In a pipeline
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack import Pipeline
from haystack.components.joiners import ListJoiner
from typing import List
user_message = [
ChatMessage.from_user("Give a brief answer the following question: {{query}}"),
]
feedback_prompt = """
You are given a question and an answer.
Your task is to provide a score and a brief feedback on the answer.
Question: {{query}}
Answer: {{response}}
"""
feedback_message = [ChatMessage.from_system(feedback_prompt)]
prompt_builder = ChatPromptBuilder(template=user_message)
feedback_prompt_builder = ChatPromptBuilder(template=feedback_message)
llm = OpenAIChatGenerator(model="gpt-4o-mini")
feedback_llm = OpenAIChatGenerator(model="gpt-4o-mini")
pipe = Pipeline()
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.add_component("feedback_prompt_builder", feedback_prompt_builder)
pipe.add_component("feedback_llm", feedback_llm)
pipe.add_component("list_joiner", ListJoiner(List[ChatMessage]))
pipe.connect("prompt_builder.prompt", "llm.messages")
pipe.connect("prompt_builder.prompt", "list_joiner")
pipe.connect("llm.replies", "list_joiner")
pipe.connect("llm.replies", "feedback_prompt_builder.response")
pipe.connect("feedback_prompt_builder.prompt", "feedback_llm.messages")
pipe.connect("feedback_llm.replies", "list_joiner")
query = "What is nuclear physics?"
ans = pipe.run(
data={
"prompt_builder": {"template_variables": {"query": query}},
"feedback_prompt_builder": {"template_variables": {"query": query}},
},
)
print(ans["list_joiner"]["values"])
```
---
// File: pipeline-components/joiners/stringjoiner
# StringJoiner
Component to join strings from different components into a list of strings.
| | |
| --- | --- |
| **Most common position in a pipeline** | After at least two other components to join their strings |
| **Mandatory run variables** | `strings`: Multiple strings from connected components. |
| **Output variables** | `strings`: A list of merged strings |
| **API reference** | [Joiners](/reference/joiners-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/joiners/string_joiner.py |
| **Package name** | `haystack-ai` |
## Overview
The `StringJoiner` component collects multiple string outputs from various pipeline components and combines them into a single list. This is useful when you need to merge several strings from different parts of a pipeline into a unified output.
## Usage
```python
from haystack.components.joiners import StringJoiner
from haystack.components.builders import PromptBuilder
from haystack.core.pipeline import Pipeline
string_1 = "What's Natural Language Processing?"
string_2 = "What is life?"
pipeline = Pipeline()
pipeline.add_component("prompt_builder_1", PromptBuilder("Builder 1: {{query}}"))
pipeline.add_component("prompt_builder_2", PromptBuilder("Builder 2: {{query}}"))
pipeline.add_component("string_joiner", StringJoiner())
pipeline.connect("prompt_builder_1.prompt", "string_joiner.strings")
pipeline.connect("prompt_builder_2.prompt", "string_joiner.strings")
result = pipeline.run(
data={
"prompt_builder_1": {"query": string_1},
"prompt_builder_2": {"query": string_2},
},
)
print(result)
```
---
// File: pipeline-components/joiners
# Joiners
| Component | Description |
| --- | --- |
| [AnswerJoiner](joiners/answerjoiner.mdx) | Joins multiple answers from different Generators into a single list. |
| [BranchJoiner](joiners/branchjoiner.mdx) | Joins different branches of a pipeline into a single output. |
| [DocumentJoiner](joiners/documentjoiner.mdx) | Joins lists of documents. |
| [ListJoiner](joiners/listjoiner.mdx) | Joins multiple lists into a single flat list. |
| [StringJoiner](joiners/stringjoiner.mdx) | Joins strings from different components into a list of strings. |
---
// File: pipeline-components/preprocessors/chinesedocumentsplitter
# ChineseDocumentSplitter
`ChineseDocumentSplitter` divides Chinese text documents into smaller chunks using advanced Chinese language processing capabilities. It leverages HanLP for accurate Chinese word segmentation and sentence tokenization, making it ideal for processing Chinese text that requires linguistic awareness.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) and [DocumentCleaner](documentcleaner.mdx), before [Classifiers](../classifiers.mdx) |
| **Mandatory run variables** | `documents`: A list of documents with Chinese text content |
| **Output variables** | `documents`: A list of documents, each containing a chunk of the original Chinese text |
| **API reference** | [HanLP](/reference/integrations-hanlp) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/hanlp |
| **Package name** | `hanlp-haystack` |
## Overview
`ChineseDocumentSplitter` is a specialized document splitter designed specifically for Chinese text processing. Unlike English text where words are separated by spaces, Chinese text is written continuously without spaces between words.
This component leverages HanLP (Han Language Processing) to provide accurate Chinese word segmentation and sentence tokenization. It supports two granularity levels:
- **Coarse granularity**: Provides broader word segmentation suitable for most general use cases. Uses `COARSE_ELECTRA_SMALL_ZH` model for general-purpose segmentation.
- **Fine granularity**: Offers more detailed word segmentation for specialized applications. Uses `FINE_ELECTRA_SMALL_ZH` model for detailed segmentation.
The splitter can divide documents by various units:
- `word`: Splits by Chinese words (multi-character tokens)
- `sentence`: Splits by sentences using HanLP sentence tokenizer
- `passage`: Splits by double line breaks ("\\n\\n")
- `page`: Splits by form feed characters ("\\f")
- `line`: Splits by single line breaks ("\\n")
- `period`: Splits by periods (".")
- `function`: Uses a custom splitting function
Each extracted chunk retains metadata from the original document and includes additional fields:
- `source_id`: The ID of the original document
- `page_number`: The page number the chunk belongs to
- `split_id`: The sequential ID of the split within the document
- `split_idx_start`: The starting index of the chunk in the original document
When `respect_sentence_boundary=True` is set, the component uses HanLP's sentence tokenizer (`UD_CTB_EOS_MUL`) to ensure that splits occur only between complete sentences, preserving the semantic integrity of the text.
## Usage
### On its own
You can use `ChineseDocumentSplitter` outside of a pipeline to process Chinese documents directly:
```python
from haystack import Document
from haystack_integrations.components.preprocessors.hanlp import ChineseDocumentSplitter
# Initialize the splitter with word-based splitting
splitter = ChineseDocumentSplitter(
split_by="word",
split_length=10,
split_overlap=3,
granularity="coarse",
)
# Create a Chinese document
doc = Document(
content="这是第一句话，这是第二句话，这是第三句话。这是第四句话，这是第五句话，这是第六句话！",
)
# Split the document
result = splitter.run(documents=[doc])
print(result["documents"]) # List of split documents
```
### With sentence boundary respect
When splitting by words, you can ensure that sentence boundaries are respected:
```python
from haystack import Document
from haystack_integrations.components.preprocessors.hanlp import ChineseDocumentSplitter
doc = Document(
content="这是第一句话，这是第二句话，这是第三句话。"
"这是第四句话，这是第五句话，这是第六句话！"
"这是第七句话，这是第八句话，这是第九句话？",
)
splitter = ChineseDocumentSplitter(
split_by="word",
split_length=10,
split_overlap=3,
respect_sentence_boundary=True,
granularity="coarse",
)
result = splitter.run(documents=[doc])
# Each chunk will end with a complete sentence
for doc in result["documents"]:
print(f"Chunk: {doc.content}")
print(f"Ends with sentence: {doc.content.endswith(('。', '！', '？'))}")
```
### With fine granularity
For more detailed word segmentation:
```python
from haystack import Document
from haystack_integrations.components.preprocessors.hanlp import ChineseDocumentSplitter
doc = Document(content="人工智能技术正在快速发展，改变着我们的生活方式。")
splitter = ChineseDocumentSplitter(
split_by="word",
split_length=5,
split_overlap=0,
granularity="fine", # More detailed segmentation
)
result = splitter.run(documents=[doc])
print(result["documents"])
```
### With custom splitting function
You can also use a custom function for splitting:
```python
from haystack import Document
from haystack_integrations.components.preprocessors.hanlp import ChineseDocumentSplitter
def custom_split(text: str) -> list[str]:
"""Custom splitting function that splits by commas"""
return text.split("，")
doc = Document(content="第一段，第二段，第三段，第四段")
splitter = ChineseDocumentSplitter(split_by="function", splitting_function=custom_split)
result = splitter.run(documents=[doc])
print(result["documents"])
```
### In a pipeline
Here's how you can integrate `ChineseDocumentSplitter` into a Haystack indexing pipeline:
```python
from haystack import Pipeline, Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters.txt import TextFileToDocument
from haystack_integrations.components.preprocessors.hanlp import ChineseDocumentSplitter
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.writers import DocumentWriter
# Initialize components
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component(instance=TextFileToDocument(), name="text_file_converter")
p.add_component(instance=DocumentCleaner(), name="cleaner")
p.add_component(
instance=ChineseDocumentSplitter(
split_by="word",
split_length=100,
split_overlap=20,
respect_sentence_boundary=True,
granularity="coarse",
),
name="chinese_splitter",
)
p.add_component(instance=DocumentWriter(document_store=document_store), name="writer")
# Connect components
p.connect("text_file_converter.documents", "cleaner.documents")
p.connect("cleaner.documents", "chinese_splitter.documents")
p.connect("chinese_splitter.documents", "writer.documents")
# Run pipeline with Chinese text files
p.run({"text_file_converter": {"sources": ["path/to/your/chinese/files.txt"]}})
```
This pipeline processes Chinese text files by converting them to documents, cleaning the text, splitting them into linguistically-aware chunks using Chinese word segmentation, and storing the results in the Document Store for further retrieval and processing.
---
// File: pipeline-components/preprocessors/chonkierecursivedocumentsplitter
# ChonkieRecursiveDocumentSplitter
`ChonkieRecursiveDocumentSplitter` splits documents using a hierarchy of splitting rules via [Chonkie](https://docs.chonkie.ai/)'s `RecursiveChunker`.
It applies progressively finer-grained splits until all chunks satisfy the configured size constraints, making it effective for structured text like Markdown or code.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx), before [Embedders](../embedders.mdx) |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Chonkie](/reference/integrations-chonkie) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/chonkie |
## Overview
`ChonkieRecursiveDocumentSplitter` wraps Chonkie's `RecursiveChunker` to split documents by applying splitting rules level by level.
If a chunk produced at one level still exceeds `chunk_size`, the next level's rules are applied to it.
This continues recursively until all chunks are within the size limit.
You can customize the splitting behavior by providing `RecursiveRules` from Chonkie.
See the [Chonkie documentation](https://docs.chonkie.ai/) for details on defining custom rules.
Each output document includes the original document's metadata plus:
- `source_id`: ID of the original document
- `page_number`: Page number of the chunk within the original document
- `split_id`: Index of the chunk within the document
- `split_idx_start` / `split_idx_end`: Character offsets of the chunk in the original text
- `token_count`: Number of tokens in the chunk
## Installation
```bash
pip install chonkie-haystack
```
## Configuration
| Parameter | Default | Description |
| --- | --- | --- |
| `tokenizer` | `"character"` | Tokenizer to use. Common options: `"character"`, `"gpt2"`, `"cl100k_base"`. See [Chonkie docs](https://docs.chonkie.ai/) for all options. |
| `chunk_size` | `2048` | Maximum number of tokens per chunk. |
| `min_characters_per_chunk` | `24` | Minimum number of characters a chunk must contain. |
| `rules` | `None` | Custom `RecursiveRules` defining the splitting hierarchy. If `None`, Chonkie's default rules are used. |
| `skip_empty_documents` | `True` | Whether to skip documents with empty content. |
| `page_break_character` | `"\f"` | Character used to detect page breaks when tracking page numbers. |
## Usage
### On its own
```python
from haystack import Document
from haystack_integrations.components.preprocessors.chonkie import (
ChonkieRecursiveDocumentSplitter,
)
chunker = ChonkieRecursiveDocumentSplitter(chunk_size=512)
documents = [
Document(
content="# Introduction\n\nHaystack is a framework.\n\n## Features\n\nIt supports RAG pipelines.",
),
]
result = chunker.run(documents=documents)
print(result["documents"])
```
### With custom rules
```python
from chonkie.types.recursive import RecursiveLevel, RecursiveRules
from haystack import Document
from haystack_integrations.components.preprocessors.chonkie import (
ChonkieRecursiveDocumentSplitter,
)
rules = RecursiveRules(
levels=[
RecursiveLevel(delimiters=["\n\n"]),
RecursiveLevel(delimiters=["\n"]),
RecursiveLevel(delimiters=[". ", "! ", "? "]),
],
)
chunker = ChonkieRecursiveDocumentSplitter(chunk_size=256, rules=rules)
documents = [Document(content="First paragraph.\n\nSecond paragraph with more detail.")]
result = chunker.run(documents=documents)
print(result["documents"])
```
### In a pipeline
```python
from pathlib import Path
from haystack import Pipeline
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.preprocessors.chonkie import (
ChonkieRecursiveDocumentSplitter,
)
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component("converter", TextFileToDocument())
p.add_component("cleaner", DocumentCleaner())
p.add_component("splitter", ChonkieRecursiveDocumentSplitter(chunk_size=512))
p.add_component("writer", DocumentWriter(document_store=document_store))
p.connect("converter.documents", "cleaner.documents")
p.connect("cleaner.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
files = list(Path("path/to/your/files").glob("*.md"))
p.run({"converter": {"sources": files}})
```
---
// File: pipeline-components/preprocessors/chonkiesemanticdocumentsplitter
# ChonkieSemanticDocumentSplitter
`ChonkieSemanticDocumentSplitter` splits documents at semantically meaningful boundaries using [Chonkie](https://docs.chonkie.ai/)'s `SemanticChunker`.
Rather than splitting by a fixed token count, it uses an embedding model to detect topic shifts and keeps related sentences together.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx), before [Embedders](../embedders.mdx) |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Chonkie](/reference/integrations-chonkie) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/chonkie |
## Overview
`ChonkieSemanticDocumentSplitter` wraps Chonkie's `SemanticChunker` to produce context-aware chunks by grouping sentences with similar semantic content.
It computes embeddings for sentences and uses cosine similarity to find natural topic boundaries.
The embedding model is loaded lazily — `warm_up()` is called automatically the first time `run()` is invoked, whether inside a pipeline or standalone.
Each output document includes the original document's metadata plus:
- `source_id`: ID of the original document
- `page_number`: Page number of the chunk within the original document
- `split_id`: Index of the chunk within the document
- `split_idx_start` / `split_idx_end`: Character offsets of the chunk in the original text
- `token_count`: Number of tokens in the chunk
## Installation
```bash
pip install chonkie-haystack
```
## Configuration
| Parameter | Default | Description |
| --- | --- | --- |
| `embedding_model` | `"minishlab/potion-base-32M"` | The embedding model used to compute sentence similarity. See [Chonkie docs](https://docs.chonkie.ai/) for supported models. |
| `threshold` | `0.8` | Cosine similarity threshold below which a sentence boundary becomes a split point. |
| `chunk_size` | `2048` | Maximum number of tokens per chunk (based on the embedding model's tokenizer). |
| `similarity_window` | `3` | Number of surrounding sentences to include when computing similarity. |
| `min_sentences_per_chunk` | `1` | Minimum number of sentences that must be included in each chunk. |
| `min_characters_per_sentence` | `24` | Minimum number of characters for a sentence to be considered valid. |
| `delim` | `None` | Custom sentence delimiters. If `None`, Chonkie's default delimiters are used. |
| `include_delim` | `"prev"` | Whether to attach the delimiter to the previous (`"prev"`) or next (`"next"`) chunk. |
| `skip_window` | `0` | Number of sentences to skip when computing similarity scores. |
| `filter_window` | `5` | Window size for the Savitzky-Golay smoothing filter applied to similarity scores. |
| `filter_polyorder` | `3` | Polynomial order for the Savitzky-Golay filter. |
| `filter_tolerance` | `0.2` | Tolerance used when filtering similarity scores. |
| `skip_empty_documents` | `True` | Whether to skip documents with empty content. |
| `page_break_character` | `"\f"` | Character used to detect page breaks when tracking page numbers. |
## Usage
### On its own
```python
from haystack import Document
from haystack_integrations.components.preprocessors.chonkie import (
ChonkieSemanticDocumentSplitter,
)
chunker = ChonkieSemanticDocumentSplitter(chunk_size=512, threshold=0.5)
documents = [
Document(
content="Haystack is an open-source framework for LLM applications. "
"It makes building RAG pipelines easy. "
"The Eiffel Tower is located in Paris. "
"Paris is the capital of France.",
),
]
result = chunker.run(documents=documents)
print(result["documents"])
```
### In a pipeline
```python
from pathlib import Path
from haystack import Pipeline
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.preprocessors.chonkie import (
ChonkieSemanticDocumentSplitter,
)
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component("converter", TextFileToDocument())
p.add_component("cleaner", DocumentCleaner())
p.add_component(
"splitter",
ChonkieSemanticDocumentSplitter(chunk_size=512, threshold=0.5),
)
p.add_component("writer", DocumentWriter(document_store=document_store))
p.connect("converter.documents", "cleaner.documents")
p.connect("cleaner.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
files = list(Path("path/to/your/files").glob("*.txt"))
p.run({"converter": {"sources": files}})
```
---
// File: pipeline-components/preprocessors/chonkiesentencedocumentsplitter
# ChonkieSentenceDocumentSplitter
`ChonkieSentenceDocumentSplitter` splits documents into chunks that respect sentence boundaries using [Chonkie](https://docs.chonkie.ai/)'s `SentenceChunker`.
Unlike pure token splitting, it avoids cutting mid-sentence, producing more coherent chunks.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) and [`DocumentCleaner`](documentcleaner.mdx), before [Embedders](../embedders.mdx) |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Chonkie](/reference/integrations-chonkie) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/chonkie |
## Overview
`ChonkieSentenceDocumentSplitter` wraps Chonkie's `SentenceChunker` to split each input document into chunks whose boundaries align with sentence endings.
The chunker groups sentences together until the chunk size limit is reached.
Each output document includes the original document's metadata plus:
- `source_id`: ID of the original document
- `page_number`: Page number of the chunk within the original document
- `split_id`: Index of the chunk within the document
- `split_idx_start` / `split_idx_end`: Character offsets of the chunk in the original text
- `token_count`: Number of tokens in the chunk
## Installation
```bash
pip install chonkie-haystack
```
## Configuration
| Parameter | Default | Description |
| --- | --- | --- |
| `tokenizer` | `"character"` | Tokenizer to use. Common options: `"character"`, `"gpt2"`, `"cl100k_base"`. See [Chonkie docs](https://docs.chonkie.ai/) for all options. |
| `chunk_size` | `2048` | Maximum number of tokens per chunk. |
| `chunk_overlap` | `0` | Number of overlapping tokens between consecutive chunks. |
| `min_sentences_per_chunk` | `1` | Minimum number of sentences that must be included in each chunk. |
| `min_characters_per_sentence` | `12` | Minimum number of characters for a sentence to be considered valid. |
| `approximate` | `False` | Whether to use approximate chunking for faster processing. |
| `delim` | `None` | Custom sentence delimiters. If `None`, Chonkie's default delimiters are used. |
| `include_delim` | `"prev"` | Whether to attach the delimiter to the previous (`"prev"`) or next (`"next"`) chunk. |
| `skip_empty_documents` | `True` | Whether to skip documents with empty content. |
| `page_break_character` | `"\f"` | Character used to detect page breaks when tracking page numbers. |
## Usage
### On its own
```python
from haystack import Document
from haystack_integrations.components.preprocessors.chonkie import (
ChonkieSentenceDocumentSplitter,
)
chunker = ChonkieSentenceDocumentSplitter(
tokenizer="gpt2",
chunk_size=512,
chunk_overlap=0,
)
documents = [
Document(
content="Haystack is an open-source framework. It helps you build LLM applications.",
),
]
result = chunker.run(documents=documents)
print(result["documents"])
```
### In a pipeline
```python
from pathlib import Path
from haystack import Pipeline
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.preprocessors.chonkie import (
ChonkieSentenceDocumentSplitter,
)
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component("converter", TextFileToDocument())
p.add_component("cleaner", DocumentCleaner())
p.add_component(
"splitter",
ChonkieSentenceDocumentSplitter(tokenizer="gpt2", chunk_size=512),
)
p.add_component("writer", DocumentWriter(document_store=document_store))
p.connect("converter.documents", "cleaner.documents")
p.connect("cleaner.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
files = list(Path("path/to/your/files").glob("*.txt"))
p.run({"converter": {"sources": files}})
```
---
// File: pipeline-components/preprocessors/chonkietokendocumentsplitter
# ChonkieTokenDocumentSplitter
`ChonkieTokenDocumentSplitter` splits documents into fixed-size token-based chunks using [Chonkie](https://docs.chonkie.ai/)'s `TokenChunker`.
It supports multiple tokenizers and is well-suited for splitting long documents before indexing.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) and [`DocumentCleaner`](documentcleaner.mdx), before [Embedders](../embedders.mdx) |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Chonkie](/reference/integrations-chonkie) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/chonkie |
## Overview
`ChonkieTokenDocumentSplitter` wraps Chonkie's `TokenChunker` to split each input document into smaller chunks based on token count.
You can configure the tokenizer, chunk size, and overlap between chunks.
Each output document includes the original document's metadata plus:
- `source_id`: ID of the original document
- `page_number`: Page number of the chunk within the original document
- `split_id`: Index of the chunk within the document
- `split_idx_start` / `split_idx_end`: Character offsets of the chunk in the original text
- `token_count`: Number of tokens in the chunk
## Installation
```bash
pip install chonkie-haystack
```
## Configuration
| Parameter | Default | Description |
| --- | --- | --- |
| `tokenizer` | `"character"` | Tokenizer to use. Common options: `"character"`, `"gpt2"`, `"cl100k_base"`. See [Chonkie docs](https://docs.chonkie.ai/) for all options. |
| `chunk_size` | `2048` | Maximum number of tokens per chunk. |
| `chunk_overlap` | `0` | Number of overlapping tokens between consecutive chunks. |
| `skip_empty_documents` | `True` | Whether to skip documents with empty content. |
| `page_break_character` | `"\f"` | Character used to detect page breaks when tracking page numbers. |
## Usage
### On its own
```python
from haystack import Document
from haystack_integrations.components.preprocessors.chonkie import (
ChonkieTokenDocumentSplitter,
)
chunker = ChonkieTokenDocumentSplitter(
tokenizer="gpt2",
chunk_size=512,
chunk_overlap=50,
)
documents = [
Document(
content="Haystack is an open-source framework for building LLM applications.",
),
]
result = chunker.run(documents=documents)
print(result["documents"])
```
### In a pipeline
```python
from pathlib import Path
from haystack import Pipeline
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.preprocessors.chonkie import (
ChonkieTokenDocumentSplitter,
)
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component("converter", TextFileToDocument())
p.add_component("cleaner", DocumentCleaner())
p.add_component(
"splitter",
ChonkieTokenDocumentSplitter(tokenizer="gpt2", chunk_size=512),
)
p.add_component("writer", DocumentWriter(document_store=document_store))
p.connect("converter.documents", "cleaner.documents")
p.connect("cleaner.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
files = list(Path("path/to/your/files").glob("*.txt"))
p.run({"converter": {"sources": files}})
```
---
// File: pipeline-components/preprocessors/csvdocumentcleaner
# CSVDocumentCleaner
Use `CSVDocumentCleaner` to clean CSV documents by removing empty rows and columns while preserving specific ignored rows and columns. It processes CSV content stored in documents and helps standardize data for further analysis.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) , before [Embedders](../embedders.mdx) or [Writers](../writers/documentwriter.mdx) |
| **Mandatory run variables** | `documents`: A list of documents containing CSV content |
| **Output variables** | `documents`: A list of cleaned CSV documents |
| **API reference** | [PreProcessors](/reference/preprocessors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/csv_document_cleaner.py |
| **Package name** | `haystack-ai` |
## Overview
`CSVDocumentCleaner` expects a list of `Document` objects as input, each containing CSV-formatted content as text. It cleans the data by removing fully empty rows and columns while allowing users to specify the number of rows and columns to be preserved before cleaning.
### Parameters
- `ignore_rows`: Number of rows to ignore from the top of the CSV table before processing. If any columns are removed, the same columns will be dropped from the ignored rows.
- `ignore_columns`: Number of columns to ignore from the left of the CSV table before processing. If any rows are removed, the same rows will be dropped from the ignored columns.
- `remove_empty_rows`: Whether to remove entirely empty rows.
- `remove_empty_columns`: Whether to remove entirely empty columns.
- `keep_id`: Whether to retain the original document ID in the output document.
### Cleaning Process
The `CSVDocumentCleaner` algorithm follows these steps:
1. Reads each document's content as a CSV table using pandas.
2. Retains the specified number of `ignore_rows` from the top and `ignore_columns` from the left.
3. Drops any rows and columns that are entirely empty (contain only NaN values).
4. If columns are dropped, they are also removed from ignored rows.
5. If rows are dropped, they are also removed from ignored columns.
6. Reattaches the remaining ignored rows and columns to maintain their original positions.
7. Returns the cleaned CSV content as a new `Document` object.
## Usage
### On its own
You can use `CSVDocumentCleaner` independently to clean up CSV documents:
```python
from haystack import Document
from haystack.components.preprocessors import CSVDocumentCleaner
cleaner = CSVDocumentCleaner(ignore_rows=1, ignore_columns=0)
documents = [Document(content="""col1,col2,col3\n,,\na,b,c\n,,""")]
cleaned_docs = cleaner.run(documents=documents)
```
### In a pipeline
```python
from pathlib import Path
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import XLSXToDocument
from haystack.components.preprocessors import CSVDocumentCleaner
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component(instance=XLSXToDocument(), name="xlsx_file_converter")
p.add_component(
instance=CSVDocumentCleaner(ignore_rows=1, ignore_columns=1),
name="csv_cleaner",
)
p.add_component(instance=DocumentWriter(document_store=document_store), name="writer")
p.connect("xlsx_file_converter.documents", "csv_cleaner.documents")
p.connect("csv_cleaner.documents", "writer.documents")
p.run({"xlsx_file_converter": {"sources": [Path("your_xlsx_file.xlsx")]}})
```
This ensures that CSV documents are properly cleaned before further processing or storage.
---
// File: pipeline-components/preprocessors/csvdocumentsplitter
# CSVDocumentSplitter
`CSVDocumentSplitter` divides CSV documents into smaller sub-tables based on split arguments. This is useful for handling structured data that contains multiple tables, improving data processing efficiency and retrieval.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) , before [CSVDocumentCleaner](csvdocumentcleaner.mdx) |
| **Mandatory run variables** | `documents`: A list of documents with CSV-formatted content |
| **Output variables** | `documents`: A list of documents, each containing a sub-table extracted from the original CSV file |
| **API reference** | [PreProcessors](/reference/preprocessors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/csv_document_splitter.py |
| **Package name** | `haystack-ai` |
## Overview
`CSVDocumentSplitter` expects a list of documents containing CSV-formatted content and returns a list of new `Document` objects, each representing a sub-table extracted from the original document.
There are two modes of operation for the splitter:
1. `threshold` (Default): Identifies empty rows or columns exceeding a given threshold and splits the document accordingly.
2. `row-wise`: Splits each row into a separate document, treating each as an independent sub-table.
The splitting process follows these rules:
1. **Row-Based Splitting**: If `row_split_threshold` is set, consecutive empty rows equalling or exceeding this threshold trigger a split.
2. **Column-Based Splitting**: If `column_split_threshold` is set, consecutive empty columns equalling or exceeding this threshold trigger a split.
3. **Recursive Splitting**: If both thresholds are provided, `CSVDocumentSplitter` first splits by rows and then by columns. If more empty rows are detected, the splitting process is called again. This ensures that sub-tables are fully separated.
Each extracted sub-table retains metadata from the original document and includes additional fields:
- `source_id`: The ID of the original document
- `row_idx_start`: The starting row index of the sub-table in the original document
- `col_idx_start`: The starting column index of the sub-table in the original document
- `split_id`: The sequential ID of the split within the document
This component is especially useful for document processing pipelines that require structured data to be extracted and stored efficiently.
### Supported Document Stores
`CSVDocumentSplitter` is compatible with the following Document Stores:
- [AstraDocumentStore](../../document-stores/astradocumentstore.mdx)
- [ChromaDocumentStore](../../document-stores/chromadocumentstore.mdx)
- [ElasticsearchDocumentStore](../../document-stores/elasticsearch-document-store.mdx)
- [OpenSearchDocumentStore](../../document-stores/opensearch-document-store.mdx)
- [PgvectorDocumentStore](../../document-stores/pgvectordocumentstore.mdx)
- [PineconeDocumentStore](../../document-stores/pinecone-document-store.mdx)
- [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx)
- [WeaviateDocumentStore](../../document-stores/weaviatedocumentstore.mdx)
- [MilvusDocumentStore](https://haystack.deepset.ai/integrations/milvus-document-store)
- [Neo4jDocumentStore](https://haystack.deepset.ai/integrations/neo4j-document-store)
## Usage
### On its own
You can use `CSVDocumentSplitter` outside of a pipeline to process CSV documents directly:
```python
from haystack import Document
from haystack.components.preprocessors import CSVDocumentSplitter
splitter = CSVDocumentSplitter(row_split_threshold=1, column_split_threshold=1)
doc = Document(
content="""ID,LeftVal,,,RightVal,Extra
1,Hello,,,World,Joined
2,StillLeft,,,StillRight,Bridge
,,,,,
A,B,,,C,D
E,F,,,G,H
""",
)
split_result = splitter.run([doc])
print(split_result["documents"]) # List of split tables as Documents
```
### In a pipeline
Here's how you can integrate `CSVDocumentSplitter` into a Haystack indexing pipeline:
```python
from haystack import Pipeline, Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters.csv import CSVToDocument
from haystack.components.preprocessors import CSVDocumentSplitter
from haystack.components.preprocessors import CSVDocumentCleaner
from haystack.components.writers import DocumentWriter
# Initialize components
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component(instance=CSVToDocument(), name="csv_file_converter")
p.add_component(instance=CSVDocumentSplitter(), name="splitter")
p.add_component(instance=CSVDocumentCleaner(), name="cleaner")
p.add_component(instance=DocumentWriter(document_store=document_store), name="writer")
# Connect components
p.connect("csv_file_converter.documents", "splitter.documents")
p.connect("splitter.documents", "cleaner.documents")
p.connect("cleaner.documents", "writer.documents")
# Run pipeline
p.run({"csv_file_converter": {"sources": ["path/to/your/file.csv"]}})
```
This pipeline extracts CSV content, splits it into structured sub-tables, cleans the CSV documents by removing empty rows and columns, and stores the resulting documents in the Document Store for further retrieval and processing.
---
// File: pipeline-components/preprocessors/documentcleaner
# DocumentCleaner
Use `DocumentCleaner` to make text documents more readable. It removes extra whitespaces, empty lines, specified substrings, regexes, page headers, and footers in this particular order. This is useful for preparing the documents for further processing by LLMs.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) , after [`DocumentSplitter`](documentsplitter.mdx) |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [PreProcessors](/reference/preprocessors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/document_cleaner.py |
| **Package name** | `haystack-ai` |
## Overview
`DocumentCleaner` expects a list of documents as input and returns a list of documents with cleaned texts. Selectable cleaning steps for each input document are to `remove_empty_lines`, `remove_extra_whitespaces` and to `remove_repeated_substrings`. These three parameters are booleans that can be set when the component is initialized.
- `unicode_normalization` normalizes Unicode characters to a standard form. The parameter can be set to NFC, NFKC, NFD, or NFKD.
- `ascii_only` removes accents from characters and replaces them with their closest ASCII equivalents.
- `remove_empty_lines` removes empty lines from the document.
- `remove_extra_whitespaces` removes extra whitespaces from the document.
- `remove_repeated_substrings` removes repeated substrings (headers/footers) from pages in the document. Pages in the text need to be separated by form feed character "\\f", which is supported by [`TextFileToDocument`](../converters/textfiletodocument.mdx), [`AzureOCRDocumentConverter`](../converters/azureocrdocumentconverter.mdx), [`MistralOCRDocumentConverter`](../converters/mistralocrdocumentconverter.mdx), and [`PaddleOCRVLDocumentConverter`](../converters/paddleocrvldocumentconverter.mdx).
:::note
`remove_extra_whitespaces` and `remove_empty_lines` work best on plain-text content. If your converter returns Markdown, such as [`AzureDocumentIntelligenceConverter`](../converters/azuredocumentintelligenceconverter.mdx), [`MarkItDownConverter`](../converters/markitdownconverter.mdx), [`MistralOCRDocumentConverter`](../converters/mistralocrdocumentconverter.mdx), or [`PaddleOCRVLDocumentConverter`](../converters/paddleocrvldocumentconverter.mdx), disable those options to preserve headings, tables, lists, and image tags.
:::
In addition, you can specify a list of strings that should be removed from all documents as part of the cleaning with the parameter `remove_substring`. You can also specify a regular expression with the parameter `remove_regex` and any matches will be removed.
The cleaning steps are executed in the following order:
1. unicode_normalization
2. ascii_only
3. remove_extra_whitespaces
4. remove_empty_lines
5. remove_substrings
6. remove_regex
7. remove_repeated_substrings
## Usage
### On its own
You can use it outside of a pipeline to clean up your documents:
```python
from haystack import Document
from haystack.components.preprocessors import DocumentCleaner
doc = Document(content="This is a document to clean\n\n\nsubstring to remove")
cleaner = DocumentCleaner(remove_substrings=["substring to remove"])
result = cleaner.run(documents=[doc])
assert result["documents"][0].content == "This is a document to clean "
```
### In a pipeline
```python
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component(instance=TextFileToDocument(), name="text_file_converter")
p.add_component(instance=DocumentCleaner(), name="cleaner")
p.add_component(
instance=DocumentSplitter(split_by="sentence", split_length=1),
name="splitter",
)
p.add_component(instance=DocumentWriter(document_store=document_store), name="writer")
p.connect("text_file_converter.documents", "cleaner.documents")
p.connect("cleaner.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
p.run({"text_file_converter": {"sources": your_files}})
```
### In YAML
```yaml
components:
cleaner:
init_parameters:
ascii_only: false
keep_id: false
remove_empty_lines: true
remove_extra_whitespaces: true
remove_regex: null
remove_repeated_substrings: false
remove_substrings: null
replace_regexes: null
strip_whitespaces: false
unicode_normalization: null
type: haystack.components.preprocessors.document_cleaner.DocumentCleaner
splitter:
init_parameters:
extend_abbreviations: true
language: en
respect_sentence_boundary: false
skip_empty_documents: true
split_by: sentence
split_length: 1
split_overlap: 0
split_threshold: 0
use_split_rules: true
type: haystack.components.preprocessors.document_splitter.DocumentSplitter
text_file_converter:
init_parameters:
encoding: utf-8
store_full_path: false
type: haystack.components.converters.txt.TextFileToDocument
writer:
init_parameters:
document_store:
init_parameters:
bm25_algorithm: BM25L
bm25_parameters: {}
bm25_tokenization_regex: (?u)\\b\\w+\\b
embedding_similarity_function: dot_product
index: 64e4f9ab-87fb-47fd-b390-dabcfda61447
return_embedding: true
type: haystack.document_stores.in_memory.document_store.InMemoryDocumentStore
policy: NONE
type: haystack.components.writers.document_writer.DocumentWriter
connection_type_validation: true
connections:
- receiver: cleaner.documents
sender: text_file_converter.documents
- receiver: splitter.documents
sender: cleaner.documents
- receiver: writer.documents
sender: splitter.documents
max_runs_per_component: 100
metadata: {}
```
---
// File: pipeline-components/preprocessors/documentpreprocessor
# DocumentPreprocessor
Divides a list of text documents into a list of shorter text documents and then makes them more readable by cleaning.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of split and cleaned documents |
| **API reference** | [PreProcessors](/reference/preprocessors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/document_preprocessor.py |
| **Package name** | `haystack-ai` |
## Overview
`DocumentPreprocessor` first splits and then cleans documents.
It is a SuperComponent that combines a `DocumentSplitter` and a `DocumentCleaner` into a single component.
### Parameters
The `DocumentPreprocessor` exposes all initialization parameters of the underlying `DocumentSplitter` and `DocumentCleaner`, and they are all optional. A detailed description of their parameters is in the respective documentation pages:
- [DocumentSplitter](documentsplitter.mdx)
- [DocumentCleaner](documentcleaner.mdx)
## Usage
### On its own
```python
from haystack import Document
from haystack.components.preprocessors import DocumentPreprocessor
doc = Document(content="I love pizza!")
preprocessor = DocumentPreprocessor()
result = preprocessor.run(documents=[doc])
print(result["documents"])
```
### In a pipeline
You can use the `DocumentPreprocessor` in your indexing pipeline. The example below requires installing additional dependencies for the `MultiFileConverter`:
```shell
pip install pypdf markdown-it-py mdit_plain trafilatura python-pptx python-docx jq openpyxl tabulate pandas
```
```python
from haystack import Pipeline
from haystack.components.converters import MultiFileConverter
from haystack.components.preprocessors import DocumentPreprocessor
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", MultiFileConverter())
pipeline.add_component("preprocessor", DocumentPreprocessor())
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter", "preprocessor")
pipeline.connect("preprocessor", "writer")
result = pipeline.run(data={"sources": ["test.txt", "test.pdf"]})
print(result)
# {'writer': {'documents_written': 3}}
```
---
// File: pipeline-components/preprocessors/documentsplitter
# DocumentSplitter
`DocumentSplitter` divides a list of text documents into a list of shorter text documents. This is useful for long texts that otherwise wouldn't fit into the maximum text length of language models and can also speed up question answering.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) and [`DocumentCleaner`](documentcleaner.mdx) , before [Classifiers](../classifiers.mdx) |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [PreProcessors](/reference/preprocessors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/document_splitter.py |
| **Package name** | `haystack-ai` |
## Overview
`DocumentSplitter` expects a list of documents as input and returns a list of documents with split texts. It splits each input document by `split_by` after `split_length` units with an overlap of `split_overlap` units. These additional parameters can be set when the component is initialized:
- `split_by` can be `"word"`, `"sentence"`, `"passage"` (paragraph), `"page"`, `"line"`, `"period"` or `"function"`.
- `split_length` is an integer indicating the chunk size, which is the number of words, sentences, or passages.
- `split_overlap` is an integer indicating the number of overlapping words, sentences, or passages between chunks.
- `split_threshold` is an integer indicating the minimum number of words, sentences, or passages that the document fragment should have. If the fragment is below the threshold, it will be attached to the previous one.
A field `"source_id"` is added to each document's `meta` data to keep track of the original document that was split. Another meta field `"page_number"` is added to each document to keep track of the page it belonged to in the original document. Other metadata are copied from the original document.
The DocumentSplitter is compatible with the following DocumentStores:
- [AstraDocumentStore](../../document-stores/astradocumentstore.mdx)
- [ChromaDocumentStore](../../document-stores/chromadocumentstore.mdx) – limited support, overlapping information is not stored.
- [ElasticsearchDocumentStore](../../document-stores/elasticsearch-document-store.mdx)
- [OpenSearchDocumentStore](../../document-stores/opensearch-document-store.mdx)
- [PgvectorDocumentStore](../../document-stores/pgvectordocumentstore.mdx)
- [PineconeDocumentStore](../../document-stores/pinecone-document-store.mdx) – limited support, overlapping information is not stored.
- [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx)
- [WeaviateDocumentStore](../../document-stores/weaviatedocumentstore.mdx)
- [MilvusDocumentStore](https://haystack.deepset.ai/integrations/milvus-document-store)
- [Neo4jDocumentStore](https://haystack.deepset.ai/integrations/neo4j-document-store)
## Usage
### On its own
You can use this component outside of a pipeline to shorten your documents like this:
```python
from haystack import Document
from haystack.components.preprocessors import DocumentSplitter
doc = Document(
content="Moonlight shimmered softly, wolves howled nearby, night enveloped everything.",
)
splitter = DocumentSplitter(split_by="word", split_length=3, split_overlap=0)
result = splitter.run(documents=[doc])
```
### In a pipeline
Here's how you can use `DocumentSplitter` in an indexing pipeline:
```python
from pathlib import Path
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters.txt import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component(instance=TextFileToDocument(), name="text_file_converter")
p.add_component(instance=DocumentCleaner(), name="cleaner")
p.add_component(
instance=DocumentSplitter(split_by="sentence", split_length=1),
name="splitter",
)
p.add_component(instance=DocumentWriter(document_store=document_store), name="writer")
p.connect("text_file_converter.documents", "cleaner.documents")
p.connect("cleaner.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
path = "path/to/your/files"
files = list(Path(path).glob("*.md"))
p.run({"text_file_converter": {"sources": files}})
```
### In YAML
This is the YAML representation of the indexing pipeline shown above. It reads text files, cleans the text, splits it into individual sentences, and writes them to an in-memory document store.
```yaml
components:
cleaner:
init_parameters:
ascii_only: false
keep_id: false
remove_empty_lines: true
remove_extra_whitespaces: true
remove_regex: null
remove_repeated_substrings: false
remove_substrings: null
replace_regexes: null
strip_whitespaces: false
unicode_normalization: null
type: haystack.components.preprocessors.document_cleaner.DocumentCleaner
splitter:
init_parameters:
extend_abbreviations: true
language: en
respect_sentence_boundary: false
skip_empty_documents: true
split_by: sentence
split_length: 1
split_overlap: 0
split_threshold: 0
use_split_rules: true
type: haystack.components.preprocessors.document_splitter.DocumentSplitter
text_file_converter:
init_parameters:
encoding: utf-8
store_full_path: false
type: haystack.components.converters.txt.TextFileToDocument
writer:
init_parameters:
document_store:
init_parameters:
bm25_algorithm: BM25L
bm25_parameters: {}
bm25_tokenization_regex: (?u)\\b\\w+\\b
embedding_similarity_function: dot_product
index: 64e4f9ab-87fb-47fd-b390-dabcfda61447
return_embedding: true
type: haystack.document_stores.in_memory.document_store.InMemoryDocumentStore
policy: NONE
type: haystack.components.writers.document_writer.DocumentWriter
connection_type_validation: true
connections:
- receiver: cleaner.documents
sender: text_file_converter.documents
- receiver: splitter.documents
sender: cleaner.documents
- receiver: writer.documents
sender: splitter.documents
max_runs_per_component: 100
metadata: {}
```
---
// File: pipeline-components/preprocessors/embeddingbaseddocumentsplitter
# EmbeddingBasedDocumentSplitter
Use this component to split documents based on embedding similarity using cosine distances between sequential sentence groups.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) and [`DocumentCleaner`](documentcleaner.mdx) |
| **Mandatory run variables** | `documents`: A list of documents to split each into smaller documents based on embedding similarity. |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [PreProcessors](/reference/preprocessors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/embedding_based_document_splitter.py |
| **Package name** | `haystack-ai` |
## Overview
This component splits documents based on embedding similarity using cosine distances between sequential sentence groups.
It first splits text into sentences, optionally groups them, calculates embeddings for each group, and then uses cosine
distance between sequential embeddings to determine split points. Any distance above the specified percentile is treated
as a break point. The component also tracks page numbers based on form feed characters (`\f`) in the original document.
This component is inspired by [5 Levels of Text Splitting](https://github.com/FullStackRetrieval-com/RetrievalTutorials/blob/main/tutorials/LevelsOfTextSplitting/5_Levels_Of_Text_Splitting.ipynb) by Greg Kamradt.
## Usage
### On its own
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
)
from haystack.components.preprocessors import EmbeddingBasedDocumentSplitter
# Create a document with content that has a clear topic shift
doc = Document(
content="This is a first sentence. This is a second sentence. This is a third sentence. "
"Completely different topic. The same completely different topic.",
)
# Initialize the embedder to calculate semantic similarities
embedder = SentenceTransformersDocumentEmbedder()
# Configure the splitter with parameters that control splitting behavior
splitter = EmbeddingBasedDocumentSplitter(
document_embedder=embedder,
sentences_per_group=2, # Group 2 sentences before calculating embeddings
percentile=0.95, # Split when cosine distance exceeds 95th percentile
min_length=50, # Merge splits shorter than 50 characters
max_length=1000, # Further split chunks longer than 1000 characters
)
result = splitter.run(documents=[doc])
# The result contains a list of Document objects, each representing a semantic chunk
# Each split document includes metadata: source_id, split_id, and page_number
print(f"Original document split into {len(result['documents'])} chunks")
for i, split_doc in enumerate(result["documents"]):
print(f"Chunk {i}: {split_doc.content[:50]}...")
```
### In a pipeline
```python
from pathlib import Path
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters.txt import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import EmbeddingBasedDocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
)
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component(instance=TextFileToDocument(), name="text_file_converter")
pipeline.add_component(instance=DocumentCleaner(), name="cleaner")
pipeline.add_component(
instance=EmbeddingBasedDocumentSplitter(
document_embedder=SentenceTransformersDocumentEmbedder(),
sentences_per_group=2,
percentile=0.95,
min_length=50,
max_length=1000,
),
name="splitter",
)
pipeline.add_component(
instance=DocumentWriter(document_store=document_store), name="writer"
)
pipeline.connect("text_file_converter.documents", "cleaner.documents")
pipeline.connect("cleaner.documents", "splitter.documents")
pipeline.connect("splitter.documents", "writer.documents")
path = "path/to/your/files"
files = list(Path(path).glob("*.md"))
pipeline.run({"text_file_converter": {"sources": files}})
```
---
// File: pipeline-components/preprocessors/hierarchicaldocumentsplitter
# HierarchicalDocumentSplitter
Use this component to create a multi-level document structure based on parent-children relationships between text segments.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) and [`DocumentCleaner`](documentcleaner.mdx) |
| **Mandatory init variables** | `block_sizes`: Set of block sizes to split the document into. The blocks are split in descending order. |
| **Mandatory run variables** | `documents`: A list of documents to split into hierarchical blocks |
| **Output variables** | `documents`: A list of hierarchical documents |
| **API reference** | [PreProcessors](/reference/preprocessors-api) |
| **GitHub link** | [https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/hierarchical_document_splitter.py](https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/hierarchical_document_splitter.py#L12) |
| **Package name** | `haystack-ai` |
## Overview
The `HierarchicalDocumentSplitter` divides documents into blocks of different sizes, creating a tree-like structure.
A block is one of the chunks of text that the splitter produces. It is similar to cutting a long piece of text into smaller pieces: each piece is a block. Blocks form a tree structure where your full document is the root block, and as you split it into smaller and smaller pieces you get child-blocks and leaf-blocks, down to whatever smallest size specified.
The [`AutoMergingRetriever`](../retrievers/automergingretriever.mdx) component then leverages this hierarchical structure to improve document retrieval.
To initialize the component, you need to specify the `block_size`, which is the “maximum length” of each of the blocks, measured in the specific unit (see `split_by` parameter). Pass a set of sizes (for example, `{20, 5}`), and it will:
- First, split the document into blocks of up to 20 units each (the “parent” blocks).
- Then, it will split each of those into blocks of up to 5 units each (the “child” blocks).
This descending order of sizes builds the hierarchy.
These additional parameters can be set when the component is initialized:
- `split_by` can be `"word"` (default), `"sentence"`, `"passage"`, `"page"`.
- `split_overlap` is an integer indicating the number of overlapping words, sentences, or passages between chunks, 0 being the default.
## Usage
### On its own
```python
from haystack import Document
from haystack.components.preprocessors import HierarchicalDocumentSplitter
doc = Document(content="This is a simple test document")
splitter = HierarchicalDocumentSplitter(block_sizes={3, 2}, split_overlap=0, split_by="word")
splitter.run([doc])
>> {'documents': [Document(id=3f7..., content: 'This is a simple test document', meta: {'__block_size': 0, '__parent_id': None, '__children_ids': ['80a..', 'f0e..'], '__level': 0}),
>> Document(id=80a.., content: 'This is a ', meta: {'__block_size': 3, '__parent_id': '3f7..', '__children_ids': ['e39..', 'fbf..'], '__level': 1, 'source_id': '3f7..', 'page_number': 1, 'split_id': 0, 'split_idx_start': 0}),
>> Document(id=f0e.., content: 'simple test document', meta: {'__block_size': 3, '__parent_id': '3f7..', '__children_ids': ['5d1..', '181..'], '__level': 1, 'source_id': '3f7..', 'page_number': 1, 'split_id': 1, 'split_idx_start': 10}),
>> Document(id=e39.., content: 'This is ', meta: {'__block_size': 2, '__parent_id': '80a..', '__children_ids': [], '__level': 2, 'source_id': '80a..', 'page_number': 1, 'split_id': 0, 'split_idx_start': 0}),
>> Document(id=fbf.., content: 'a ', meta: {'__block_size': 2, '__parent_id': '80a..', '__children_ids': [], '__level': 2, 'source_id': '80a..', 'page_number': 1, 'split_id': 1, 'split_idx_start': 8}),
>> Document(id=5d1.., content: 'simple test ', meta: {'__block_size': 2, '__parent_id': 'f0e..', '__children_ids': [], '__level': 2, 'source_id': 'f0e..', 'page_number': 1, 'split_id': 0, 'split_idx_start': 0}),
>> Document(id=181.., content: 'document', meta: {'__block_size': 2, '__parent_id': 'f0e..', '__children_ids': [], '__level': 2, 'source_id': 'f0e..', 'page_number': 1, 'split_id': 1, 'split_idx_start': 12})]}
```
### In a pipeline
This Haystack pipeline processes `.md` files by converting them to documents, cleaning the text, splitting it into sentence-based chunks, and storing the results in an In-Memory Document Store.
```python
from pathlib import Path
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters.txt import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import HierarchicalDocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component(instance=TextFileToDocument(), name="text_file_converter")
pipeline.add_component(instance=DocumentCleaner(), name="cleaner")
pipeline.add_component(
instance=HierarchicalDocumentSplitter(
block_sizes={10, 6, 3}, split_overlap=0, split_by="sentence"
),
name="splitter",
)
pipeline.add_component(
instance=DocumentWriter(document_store=document_store), name="writer"
)
pipeline.connect("text_file_converter.documents", "cleaner.documents")
pipeline.connect("cleaner.documents", "splitter.documents")
pipeline.connect("splitter.documents", "writer.documents")
path = "path/to/your/files"
files = list(Path(path).glob("*.md"))
pipeline.run({"text_file_converter": {"sources": files}})
```
---
// File: pipeline-components/preprocessors/markdownheadersplitter
# MarkdownHeaderSplitter
Split documents at ATX-style Markdown headers (`#`, `##`, and so on), with optional secondary splitting. Header hierarchy is preserved as metadata on each chunk.
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx) and [`DocumentCleaner`](documentcleaner.mdx) |
| **Mandatory run variables** | `documents`: A list of text documents to split. |
| **Output variables** | `documents`: A list of documents split at headers (and optionally by secondary split). |
| **API reference** | [PreProcessors](/reference/preprocessors-api) |
| **GitHub link** | [https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/markdown_header_splitter.py](https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/markdown_header_splitter.py) |
| **Package name** | `haystack-ai` |
## Overview
The `MarkdownHeaderSplitter` processes text documents by:
- Splitting them into chunks at ATX-style Markdown headers (`#`, `##`, …, `######`), preserving header hierarchy as metadata.
- Optionally applying a secondary split (by word, passage, period, or line) to each chunk using Haystack's [`DocumentSplitter`](documentsplitter.mdx).
- Preserving and propagating metadata such as parent headers, page numbers, and split IDs.
Only ATX-style headers are recognized (e.g. `# Title`). Setext-style headers (`Underline with ===`) aren't supported.
Parameters you can set when initializing the component:
- `page_break_character`: Character used to identify page breaks. Defaults to form feed `\f`.
- `keep_headers`: If `True`, headers remain in the chunk content. If `False`, headers are moved to metadata only. Defaults to `True`.
- `secondary_split`: Optional secondary split after header splitting. Options: `None`, `"word"`, `"passage"`, `"period"`, `"line"`. Defaults to `None`.
- `split_length`: Maximum number of units per split when using secondary splitting. Defaults to `200`.
- `split_overlap`: Number of overlapping units between splits when using secondary splitting. Defaults to `0`.
- `split_threshold`: Minimum number of units per split when using secondary splitting. Defaults to `0`.
- `skip_empty_documents`: Whether to skip documents with empty content. Defaults to `True`.
Each output document's metadata includes:
- `source_id`: ID of the original document.
- `page_number`: Page number. Updated when `page_break_character` is found.
- `split_id`: Index of the chunk within its parent.
- `header`: The header text for this chunk.
- `parent_headers`: List of parent header texts in hierarchy order.
The component only works with text documents. Documents with `None` or non-string content raise a `ValueError`.
## Usage
### On its own
```python
from haystack import Document
from haystack.components.preprocessors import MarkdownHeaderSplitter
text = (
"# Introduction\n"
"This is the intro section.\n"
"## Getting Started\n"
"Here is how to start.\n"
"## Advanced\n"
"Advanced content here."
)
doc = Document(content=text)
splitter = MarkdownHeaderSplitter(keep_headers=True)
result = splitter.run(documents=[doc])
# result["documents"] contains one document per header section,
# with meta["header"], meta["parent_headers"], meta["source_id"], and so on
```
### With secondary splitting
When sections are long, you can add a secondary split, for example by word, so each chunk stays within a maximum size:
```python
from haystack import Document
from haystack.components.preprocessors import MarkdownHeaderSplitter
text = "# Section\n" + "Some long body text. " * 50
doc = Document(content=text)
splitter = MarkdownHeaderSplitter(
keep_headers=True,
secondary_split="word",
split_length=20,
split_overlap=2,
)
result = splitter.run(documents=[doc])
```
### In a pipeline
This pipeline converts Markdown files to documents, cleans them, splits by headers, and writes to an in-memory document store:
```python
from pathlib import Path
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters.txt import TextFileToDocument
from haystack.components.preprocessors import MarkdownHeaderSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component("text_file_converter", TextFileToDocument())
p.add_component("splitter", MarkdownHeaderSplitter(keep_headers=True))
p.add_component("writer", DocumentWriter(document_store=document_store))
p.connect("text_file_converter.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
path = "path/to/your/files"
files = list(Path(path).glob("*.md"))
p.run({"text_file_converter": {"sources": files}})
```
---
// File: pipeline-components/preprocessors/presidiodocumentcleaner
# PresidioDocumentCleaner
`PresidioDocumentCleaner` replaces personally identifiable information (PII) in the text content of Documents with entity type placeholders such as `
| | |
| --- | --- |
| **Most common position in a pipeline** | In an indexing pipeline, before writing Documents to a Document Store |
| **Mandatory run variables** | `documents`: A list of Document objects |
| **Output variables** | `documents`: A list of Document objects with PII replaced |
| **API reference** | [Presidio](/reference/integrations-presidio) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/presidio |
| **Package name** | `presidio-haystack` |
## Overview
[Microsoft Presidio](https://data-privacy-stack.github.io/presidio/) is an open-source framework for PII detection and anonymization. `PresidioDocumentCleaner` uses Presidio's Analyzer and Anonymizer engines to scan document text and replace detected entities with type placeholders such as `
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, before a Generator or Chat Generator |
| **Mandatory run variables** | `texts`: A list of strings |
| **Output variables** | `texts`: A list of strings with PII replaced |
| **API reference** | [Presidio](/reference/integrations-presidio) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/presidio |
| **Package name** | `presidio-haystack` |
## Overview
[Microsoft Presidio](https://data-privacy-stack.github.io/presidio/) is an open-source framework for PII detection and anonymization. `PresidioTextCleaner` uses Presidio's Analyzer and Anonymizer engines to scan plain text strings and replace detected entities with type placeholders such as `
| | |
| --- | --- |
| **Most common position in a pipeline** | In indexing pipelines after [Converters](../converters.mdx), before [Embedders](../embedders.mdx) or [`DocumentWriter`](../writers/documentwriter.mdx) |
| **Mandatory run variables** | `documents`: A list of Python source code documents |
| **Output variables** | `documents`: A list of Python source code documents split into syntax-aware chunks |
| **API reference** | [PreProcessors](/reference/preprocessors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/python_code_splitter.py |
| **Package name** | `haystack-ai` |
## Overview
`PythonCodeSplitter` expects each input document's `content` to be valid Python source code. It parses the source with Python's `ast` module and creates ordered split units for:
- Module docstrings
- Consecutive import blocks
- Top-level functions
- Class headers
- Methods and nested classes
- Remaining top-level statements
The splitter merges these units in source order toward `max_effective_lines`. Effective lines are calculated from character length with `ceil(len(source) / expected_chars_per_line)`, so long lines count as more than one line.
Functions and methods are kept whole by the primary AST split. If one syntactic unit is larger than `oversized_factor * max_effective_lines`, the splitter falls back to a line-based secondary split using [`DocumentSplitter`](documentsplitter.mdx). This oversized fallback is the only case where chunks can overlap; the primary AST split does not add overlap.
By default, `preserve_class_definition=True`. When a chunk contains class members without the original class header, the splitter prefixes the bare class signature so the chunk still carries the class context.
If `strip_docstrings=True`, function, method, and class docstrings are removed from chunk content and stored in `meta["docstrings"]`. Module docstrings stay in the chunk content because they are their own top-level unit.
### Per-chunk metadata
Each output document carries the metadata below. All fields from the parent document's `meta` (except `split_id`) are also propagated.
| Field | Description |
| --- | --- |
| `source_id` | ID of the originating document |
| `split_id` | Sequential index of this chunk within its source document |
| `start_line` | First line of the chunk in the original source (1-indexed). Oversized secondary chunks keep the originating unit's range. |
| `end_line` | Last line of the chunk in the original source (1-indexed). Oversized secondary chunks keep the originating unit's range. |
| `unit_kinds` | List of syntactic unit kinds included in this chunk, such as `imports`, `function`, `class_header`, or `method` |
| `include_classes` | *(when applicable)* Ordered list of class names whose members appear in this chunk |
| `decorators` | *(when applicable)* Ordered list of decorator strings found on included functions, methods, or classes |
| `docstrings` | *(when `strip_docstrings=True`)* List of stripped docstring strings in source order |
| `secondary_split` | `True` if this chunk was produced by the oversized fallback splitter |
| `secondary_split_index` | Index of this piece within the secondary split sequence |
| `secondary_split_total` | Total number of pieces produced by the secondary split |
Documents with `None` content raise `ValueError`, documents with non-string content raise `TypeError`, and invalid Python source raises `SyntaxError`. Empty documents are skipped.
## Configuration
| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `min_effective_lines` | `int` | `20` | Minimum effective lines per chunk. While a chunk is below this value, the splitter keeps merging in the next unit. |
| `max_effective_lines` | `int` | `100` | Target effective lines per chunk. Units are merged greedily toward this value. |
| `expected_chars_per_line` | `int` | `45` | Character count used to estimate effective lines via `ceil(len(source) / expected_chars_per_line)`. |
| `oversized_factor` | `int` | `3` | Multiplier that triggers secondary line-based splitting for oversized syntactic units. |
| `strip_docstrings` | `bool` | `False` | Moves function, method, and class docstrings from content into `meta["docstrings"]`. |
| `preserve_class_definition` | `bool` | `True` | Prefixes class signatures on chunks that contain class members without the class header. |
| `secondary_split_overlap` | `int` | `5` | Line overlap used only by the oversized secondary split. |
| `secondary_split_length` | `int \| None` | `None` | Line length for the oversized secondary split. Defaults to `max_effective_lines` when `None`. |
## Usage
### On its own
```python
import textwrap
from haystack import Document
from haystack.components.preprocessors import PythonCodeSplitter
source = textwrap.dedent(
'''
"""Math utilities."""
from math import pi
class Circle:
"""A circle."""
def __init__(self, radius: float) -> None:
self.radius = radius
def area(self) -> float:
return pi * self.radius * self.radius
'''
).lstrip()
splitter = PythonCodeSplitter(
min_effective_lines=4,
max_effective_lines=12,
strip_docstrings=True,
)
result = splitter.run(
documents=[Document(content=source, meta={"file_name": "geometry.py"})],
)
for chunk in result["documents"]:
print(
chunk.meta["start_line"],
chunk.meta["end_line"],
chunk.meta.get("include_classes"),
)
```
### With docstring stripping for RAG
Set `strip_docstrings=True` when docstrings are verbose. The docstring text is moved out of the chunk content into `meta["docstrings"]`, keeping the stored chunk compact. Pass `meta_fields_to_embed=["docstrings"]` to your embedder so the docstring text still influences retrieval even though it is no longer in the chunk content.
```python
from haystack import Document
from haystack.components.preprocessors import PythonCodeSplitter
source = '''
"""Example module."""
from math import pi
class Circle:
"""A circle defined by its radius."""
def __init__(self, r: float) -> None:
"""Store the radius."""
self.r = r
def area(self) -> float:
"""Return the area of the circle."""
return pi * self.r * self.r
'''
splitter = PythonCodeSplitter(
min_effective_lines=20,
max_effective_lines=100,
strip_docstrings=True,
)
result = splitter.run(
documents=[Document(content=source, meta={"file_name": "my_module.py"})]
)
for chunk in result["documents"]:
print(chunk.content)
print(chunk.meta.get("docstrings"))
```
### In a pipeline
This pipeline converts Python files to documents, splits them with `PythonCodeSplitter`, and writes the chunks to an in-memory document store.
```python
from pathlib import Path
from haystack import Pipeline
from haystack.components.converters.txt import TextFileToDocument
from haystack.components.preprocessors import PythonCodeSplitter
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component("converter", TextFileToDocument())
p.add_component("splitter", PythonCodeSplitter(max_effective_lines=80))
p.add_component("writer", DocumentWriter(document_store=document_store))
p.connect("converter.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
files = list(Path("path/to/your/project").glob("**/*.py"))
p.run({"converter": {"sources": files}})
```
---
// File: pipeline-components/preprocessors/recursivesplitter
# RecursiveDocumentSplitter
This component recursively breaks down text into smaller chunks by applying a given list of separators to the text.
| | |
| --- | --- |
| Most common position in a pipeline | In indexing pipelines after [Converters](../converters.mdx) and [`DocumentCleaner`](documentcleaner.mdx) , before [Classifiers](../classifiers.mdx) |
| Mandatory run variables | `documents`: A list of documents |
| Output variables | `documents`: A list of documents |
| API reference | [PreProcessors](/reference/preprocessors-api) |
| Github link | https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/recursive_splitter.py |
## Overview
The `RecursiveDocumentSplitter` expects a list of documents as input and returns a list of documents with split texts. You can set the following parameters when initializing the component:
- `split_length`: The maximum length of each chunk, in words, by default. See the `split_units` parameter to change the the unit.
- `split_overlap`: The number of characters or words that overlap between consecutive chunks.
- `split_unit`: The unit of the `split_length` parameter. Can be either `"word"`, `"char"`, or `"token"`.
- `separators`: An optional list of separator strings to use for splitting the text. If you don’t provide any separators, the default ones are `["\n\n", "sentence", "\n", " "]`. The string separators will be treated as regular expressions. If the separator is `"sentence"`, the text will be split into sentences using a custom sentence tokenizer based on NLTK. See [SentenceSplitter](https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/sentence_tokenizer.py#L116) code for more information.
- `sentence_splitter_params`: Optional parameters to pass to the [SentenceSplitter](https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/sentence_tokenizer.py#L116).
The separators are applied in the same order as they are defined in the list. The first separator is used on the text; any resulting chunk that is within the specified `chunk_size` is retained. For chunks that exceed the defined `chunk_size`, the next separator in the list is applied. If all separators are used and the chunk still exceeds the `chunk_size`, a hard split occurs based on the `chunk_size`, taking into account whether words or characters are used as counting units. This process is repeated until all chunks are within the limits of the specified `chunk_size`.
## Usage
```python
from haystack import Document
from haystack.components.preprocessors import RecursiveDocumentSplitter
chunker = RecursiveDocumentSplitter(split_length=260, split_overlap=0, separators=["\n\n", "\n", ".", " "])
text = ('''Artificial intelligence (AI) - Introduction
AI, in its broadest sense, is intelligence exhibited by machines, particularly computer systems.
AI technology is widely used throughout industry, government, and science. Some high-profile applications include advanced web search engines; recommendation systems; interacting via human speech; autonomous vehicles; generative and creative tools; and superhuman play and analysis in strategy games.''')
doc = Document(content=text)
doc_chunks = chunker.run([doc])
print(doc_chunks["documents"])
>[
>Document(id=..., content: 'Artificial intelligence (AI) - Introduction\n\n', meta: {'source_id': '...', 'parent_id': '...', 'split_id': 0, 'split_idx_start': 0, '_split_overlap': None, 'page_number': 1})
>Document(id=..., content: 'AI, in its broadest sense, is intelligence exhibited by machines, particularly computer systems.\n', meta: {'source_id': '...', 'parent_id': '...', 'split_id': 1, 'split_idx_start': 45, '_split_overlap': None, 'page_number': 1})
>Document(id=..., content: 'AI technology is widely used throughout industry, government, and science.', meta: {'source_id': '...', 'parent_id': '...', 'split_id': 2, 'split_idx_start': 142, '_split_overlap': None, 'page_number': 1})
>Document(id=..., content: ' Some high-profile applications include advanced web search engines; recommendation systems; interac...', meta: {'source_id': '...', 'parent_id': '...', 'split_id': 3, 'split_idx_start': 216, '_split_overlap': None, 'page_number': 1})
>]
```
### In a pipeline
Here's how you can use `RecursiveSplitter` in an indexing pipeline:
```python
from pathlib import Path
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters.txt import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import RecursiveDocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component(instance=TextFileToDocument(), name="text_file_converter")
p.add_component(instance=DocumentCleaner(), name="cleaner")
p.add_component(
instance=RecursiveDocumentSplitter(
split_length=400,
split_overlap=0,
split_unit="char",
separators=["\n\n", "\n", "sentence", " "],
sentence_splitter_params={
"language": "en",
"use_split_rules": True,
"keep_white_spaces": False,
},
),
name="recursive_splitter",
)
p.add_component(instance=DocumentWriter(document_store=document_store), name="writer")
p.connect("text_file_converter.documents", "cleaner.documents")
p.connect("cleaner.documents", "recursive_splitter.documents")
p.connect("recursive_splitter.documents", "writer.documents")
path = "path/to/your/files"
files = list(Path(path).glob("*.md"))
p.run({"text_file_converter": {"sources": files}})
```
---
// File: pipeline-components/preprocessors/textcleaner
# TextCleaner
Use `TextCleaner` to make text data more readable. It removes regexes, punctuation, and numbers, as well as converts text to lowercase. This is especially useful to clean up text data before evaluation.
| | |
| --- | --- |
| **Most common position in a pipeline** | Between a [Generator](../generators.mdx) and an [Evaluator](../evaluators.mdx) |
| **Mandatory run variables** | `texts`: A list of strings to be cleaned |
| **Output variables** | `texts`: A list of cleaned texts |
| **API reference** | [PreProcessors](/reference/preprocessors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/text_cleaner.py |
| **Package name** | `haystack-ai` |
## Overview
`TextCleaner` expects a list of strings as input and returns a list of strings with cleaned texts. Selectable cleaning steps are to `convert_to_lowercase`, `remove_punctuation`, and to `remove_numbers`. These three parameters are booleans that need to be set when the component is initialized.
- `convert_to_lowercase` converts all characters in texts to lowercase.
- `remove_punctuation` removes all punctuation from the text.
- `remove_numbers` removes all numerical digits from the text.
In addition, you can specify a regular expression with the parameter `remove_regexps`, and any matches will be removed.
## Usage
### On its own
You can use it outside of a pipeline to clean up any texts:
```python
from haystack.components.preprocessors import TextCleaner
text_to_clean = (
"1Moonlight shimmered softly, 300 Wolves howled nearby, Night enveloped everything."
)
cleaner = TextCleaner(
convert_to_lowercase=True,
remove_punctuation=False,
remove_numbers=True,
)
result = cleaner.run(texts=[text_to_clean])
```
### In a pipeline
In this example, we are using `TextCleaner` after a `TransformersExtractiveReader` and an `OutputAdapter` to remove the punctuation in texts. Then, our custom-made `ExactMatchEvaluator` component compares the retrieved answer to the ground truth answer.
The examples on this page use Transformers components from the `transformers-haystack` package. Install it to run the examples:
```shell
pip install transformers-haystack
```
```python
from typing import List
from haystack import component, Document, Pipeline
from haystack.components.converters import OutputAdapter
from haystack.components.preprocessors import TextCleaner
from haystack_integrations.components.readers.transformers import (
TransformersExtractiveReader,
)
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_store.write_documents(documents=documents)
@component
class ExactMatchEvaluator:
@component.output_types(score=int)
def run(self, expected: str, provided: List[str]):
return {"score": int(expected in provided)}
adapter = OutputAdapter(
template="{{answers | extract_data}}",
output_type=List[str],
custom_filters={
"extract_data": lambda data: [answer.data for answer in data if answer.data],
},
)
p = Pipeline()
p.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
p.add_component("reader", TransformersExtractiveReader())
p.add_component("adapter", adapter)
p.add_component("cleaner", TextCleaner(remove_punctuation=True))
p.add_component("evaluator", ExactMatchEvaluator())
p.connect("retriever", "reader")
p.connect("reader", "adapter")
p.connect("adapter", "cleaner.texts")
p.connect("cleaner", "evaluator.provided")
question = "What behavior indicates a high level of self-awareness of elephants?"
ground_truth_answer = "recognizing themselves in mirrors"
result = p.run(
{
"retriever": {"query": question},
"reader": {"query": question},
"evaluator": {"expected": ground_truth_answer},
},
)
print(result)
```
---
// File: pipeline-components/preprocessors
# PreProcessors
Use the PreProcessors to prepare your data normalize white spaces, remove headers and footers, clean empty lines in your Documents, or split them into smaller pieces. PreProcessors are useful in an indexing pipeline to prepare your files for search.
| PreProcessor | Description |
| --- | --- |
| [ChineseDocumentSplitter](preprocessors/chinesedocumentsplitter.mdx) | Divides Chinese text documents into smaller chunks using advanced Chinese language processing capabilities, using HanLP for accurate Chinese word segmentation and sentence tokenization. |
| [ChonkieRecursiveDocumentSplitter](preprocessors/chonkierecursivedocumentsplitter.mdx) | Splits documents recursively using a hierarchy of rules via Chonkie's `RecursiveChunker`, applying progressively finer splits until all chunks satisfy the size constraints. |
| [ChonkieSemanticDocumentSplitter](preprocessors/chonkiesemanticdocumentsplitter.mdx) | Splits documents at semantic topic boundaries using embedding similarity via Chonkie's `SemanticChunker`, keeping related sentences together. |
| [ChonkieSentenceDocumentSplitter](preprocessors/chonkiesentencedocumentsplitter.mdx) | Splits documents into chunks that respect sentence boundaries via Chonkie's `SentenceChunker`, avoiding mid-sentence cuts. |
| [ChonkieTokenDocumentSplitter](preprocessors/chonkietokendocumentsplitter.mdx) | Splits documents into fixed-size token-based chunks via Chonkie's `TokenChunker`, supporting multiple tokenizers. |
| [CSVDocumentCleaner](preprocessors/csvdocumentcleaner.mdx) | Cleans CSV documents by removing empty rows and columns while preserving specific ignored rows and columns. |
| [CSVDocumentSplitter](preprocessors/csvdocumentsplitter.mdx) | Divides CSV documents into smaller sub-tables based on empty rows and columns. |
| [DocumentCleaner](preprocessors/documentcleaner.mdx) | Removes extra whitespaces, empty lines, specified substrings, regexes, page headers, and footers from documents. |
| [DocumentPreprocessor](preprocessors/documentpreprocessor.mdx) | Divides a list of text documents into a list of shorter text documents and then makes them more readable by cleaning. |
| [DocumentSplitter](preprocessors/documentsplitter.mdx) | Splits a list of text documents into a list of text documents with shorter texts. |
| [EmbeddingBasedDocumentSplitter](preprocessors/embeddingbaseddocumentsplitter.mdx) | Splits documents based on embedding similarity using cosine distances between sequential sentence groups. |
| [HierarchicalDocumentSplitter](preprocessors/hierarchicaldocumentsplitter.mdx) | Creates a multi-level document structure based on parent-children relationships between text segments. |
| [MarkdownHeaderSplitter](preprocessors/markdownheadersplitter.mdx) | Splits documents at ATX-style Markdown headers (#), with optional secondary splitting. Preserves header hierarchy as metadata. |
| [PresidioDocumentCleaner](preprocessors/presidiodocumentcleaner.mdx) | Replaces PII in Document text with entity type placeholders using Microsoft Presidio. |
| [PresidioTextCleaner](preprocessors/presidiotextcleaner.mdx) | Replaces PII in plain strings — useful for sanitizing user queries before they reach an LLM. |
| [PythonCodeSplitter](preprocessors/pythoncodesplitter.mdx) | Splits Python source documents into syntax-aware chunks using AST units such as imports, functions, class headers, methods, and statements. |
| [RecursiveSplitter](preprocessors/recursivesplitter.mdx) | Splits text into smaller chunks, it does so by recursively applying a list of separators to the text, applied in the order they are provided. | | [TextCleaner](preprocessors/textcleaner.mdx) | Removes regexes, punctuation, and numbers, as well as converts text to lowercase. Useful to clean up text data before evaluation. | --- // File: pipeline-components/query/queryexpander # QueryExpander QueryExpander uses an LLM to generate semantically similar queries to improve retrieval recall in RAG systems.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a Retriever component that accepts multiple queries, such as [`MultiQueryTextRetriever`](../retrievers/multiquerytextretriever.mdx) or [`MultiQueryEmbeddingRetriever`](../retrievers/multiqueryembeddingretriever.mdx) |
| **Mandatory run variables** | `query`: The query string to expand |
| **Output variables** | `queries`: A list of expanded queries |
| **API reference** | [Query](/reference/query-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/query/query_expander.py |
| **Package name** | `haystack-ai` |
## Overview
`QueryExpander` takes a user query and generates multiple semantically similar variations of it. This technique improves retrieval recall by allowing your retrieval system to find documents that might not match the original query phrasing but are still relevant.
The component uses a chat-based LLM to generate expanded queries. By default, it uses OpenAI's `gpt-4.1-mini` model, but you can pass any preferred Chat Generator component (such as `AnthropicChatGenerator` or `AzureOpenAIChatGenerator`) to the `chat_generator` parameter.
The example below uses `AnthropicChatGenerator`, which lives in the `anthropic-haystack` package:
```shell
pip install anthropic-haystack
```
```python
from haystack.components.query import QueryExpander
from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
expander = QueryExpander(
chat_generator=AnthropicChatGenerator(model="claude-sonnet-4-20250514"),
n_expansions=3,
)
```
The generated queries:
- Use different words and phrasings while maintaining the same core meaning
- Include synonyms and related terms
- Preserve the original query's language
- Are designed to work well with both keyword-based and semantic search (such as embeddings)
You can control the number of query expansions with the `n_expansions` parameter and choose whether to include the original query in the output with the `include_original_query` parameter.
### Custom Prompt Template
You can provide a custom prompt template to control how queries are expanded:
```python
from haystack.components.query import QueryExpander
custom_template = """
You are a search query expansion assistant.
Generate {{ n_expansions }} alternative search queries for: "{{ query }}"
Return a JSON object with a "queries" array containing the expanded queries.
Focus on technical terminology and domain-specific variations.
"""
expander = QueryExpander(prompt_template=custom_template, n_expansions=4)
result = expander.run(query="machine learning optimization")
```
## Usage
`QueryExpander` is designed to work with multi-query Retrievers. For complete pipeline examples, see:
- [`MultiQueryTextRetriever`](../retrievers/multiquerytextretriever.mdx) page for keyword-based (BM25) retrieval
- [`MultiQueryEmbeddingRetriever`](../retrievers/multiqueryembeddingretriever.mdx) page for embedding-based retrieval
---
// File: pipeline-components/rankers/amazonbedrockranker
# AmazonBedrockRanker
Use this component to rank documents based on their similarity to the query using Amazon Bedrock models.
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents such as a [Retriever](../retrievers.mdx) |
| **Mandatory init variables** | `aws_access_key_id`: AWS access key ID. Can be set with AWS_ACCESS_KEY_ID env var.
`aws_secret_access_key`: AWS secret access key. Can be set with AWS_SECRET_ACCESS_KEY env var.
`aws_region_name`: AWS region name. Can be set with AWS_DEFAULT_REGION env var. | | **Mandatory run variables** | `documents`: A list of document objects
`query`: A query string | | **Output variables** | `documents`: A list of document objects | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock/ | | **Package name** | `amazon-bedrock-haystack` |
## Overview
`AmazonBedrockRanker` ranks documents based on semantic relevance to a specified query. It uses Amazon Bedrock Rerank API. This list of all supported models can be found in Amazon’s [documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/rerank-supported.html). The default model for this Ranker is `cohere.rerank-v3-5:0`.
You can also specify the `top_k` parameter to set the maximum number of documents to return.
### Installation
To start using Amazon Bedrock with Haystack, install the `amazon-bedrock-haystack` package:
```shell
pip install amazon-bedrock-haystack
```
### Authentication
This component uses AWS for authentication. You can use the AWS CLI to authenticate through your IAM. For more information on setting up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
:::info[Using AWS CLI]
Consider using AWS CLI as a more straightforward tool to manage your AWS services. With AWS CLI, you can quickly configure your [boto3 credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). This way, you won't need to provide detailed authentication parameters when initializing Amazon Bedrock in Haystack.
:::
To use this component, initialize it with the model name. The AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`) should be set as environment variables, configured as described above, or passed as [Secret](../../concepts/secret-management.mdx) arguments. Make sure the region you set supports Amazon Bedrock.
## Usage
### On its own
This example uses `AmazonBedrockRanker` to rank two simple documents. To run the Ranker, pass a `query` and provide the `documents`.
```python
from haystack import Document
from haystack_integrations.components.rankers.amazon_bedrock import AmazonBedrockRanker
docs = [Document(content="Paris"), Document(content="Berlin")]
ranker = AmazonBedrockRanker()
ranker.run(query="City in France", documents=docs, top_k=1)
```
### In a pipeline
Below is an example of a pipeline that retrieves documents from an `InMemoryDocumentStore` based on keyword search (using `InMemoryBM25Retriever`). It then uses the `AmazonBedrockRanker` to rank the retrieved documents according to their similarity to the query. The pipeline uses the default settings of the Ranker.
```python
from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.rankers.amazon_bedrock import AmazonBedrockRanker
docs = [
Document(content="Paris is in France"),
Document(content="Berlin is in Germany"),
Document(content="Lyon is in France"),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
retriever = InMemoryBM25Retriever(document_store=document_store)
ranker = AmazonBedrockRanker()
document_ranker_pipeline = Pipeline()
document_ranker_pipeline.add_component(instance=retriever, name="retriever")
document_ranker_pipeline.add_component(instance=ranker, name="ranker")
document_ranker_pipeline.connect("retriever.documents", "ranker.documents")
query = "Cities in France"
res = document_ranker_pipeline.run(
data={
"retriever": {"query": query, "top_k": 3},
"ranker": {"query": query, "top_k": 2},
},
)
```
---
// File: pipeline-components/rankers/choosing-the-right-ranker
# Choosing the Right Ranker
This page provides guidance on selecting the right Ranker for your pipeline in Haystack. It explains the distinctions between API-based, on-premise rankers and heuristic approaches, and offers advice based on latency, privacy, and diversity requirements.
Rankers in Haystack reorder a set of retrieved documents based on their estimated relevance to a user query. Rankers operate after retrieval and aim to refine the result list before it's passed to a downstream component like a [Generator](../generators.mdx) or [Reader](../readers.mdx).
This reordering is based on additional signals beyond simple vector similarity. Depending on the Ranker used, these signals can include semantic similarity (with cross-encoders), structured metadata (such as timestamps or categories), or position-based heuristics (for example, placing relevant content at the start and end).
A typical question answering pipeline using a Ranker includes:
1. Retrieve: Use a [Retriever](../retrievers.mdx) to find a candidate set of documents.
2. Rank: Reorder those documents using a Ranker component.
3. Answer: Pass the re-ranked documents to a downstream [Generator](../generators.mdx) or [Reader](../readers.mdx).
This guide helps you choose the right Ranker depending on your use case, whether you're optimizing for performance, cost, accuracy, or diversity in results. It focuses on selecting between different types of Rankers in Haystack, not specific models, but rather the general mechanism and interface that best suits your setup.
## API Based Rankers
These Rankers use external APIs to reorder documents using powerful models hosted remotely. They offer high-quality relevance scoring without local compute, but can be slower due to network latency and costly at scale.
The pricing model varies by provider, some charge per token processed , while others bill by usage time or number of API calls. Refer to the respective provider documentation for precise cost structures.
Most API-based Rankers in Haystack currently rely on cross-encoder models (currently, but might change in the future), which evaluate the query and document together to produce highly accurate relevance scores. Examples include [AmazonBedrockRanker](amazonbedrockranker.mdx), [CohereRanker](cohereranker.mdx) and [JinaRanker](jinaranker.mdx).
In contrast, the [NvidiaRanker](nvidiaranker.mdx) and [LLMRanker](llmranker.mdx) use large language models (LLMs) for ranking. These models treat relevance as a semantic reasoning task, which can yield better results for complex or multi-step queries, though often at higher computational cost. **LLMRanker** works with any Haystack chat generator and prompts the LLM to return ranked document indices as JSON.
## On-Premise Rankers
These Rankers run entirely on your local infrastructure. They are ideal for teams prioritizing data privacy, cost control, or low-latency inference without depending on external APIs. Since the models are executed locally, they avoid network bottlenecks and recurring usage costs, but require sufficient compute resources, typically GPU-backed, especially for cross-encoder models.
All on-premise Rankers in Haystack use cross-encoder architectures. These models jointly process the query and each document to assess relevance with deep contextual awareness. For example:
- [SentenceTransformersSimilarityRanker](sentencetransformerssimilarityranker.mdx) ranks documents based on semantic similarity to the query. In addition to the default PyTorch backend (optimal for GPU), it also offers other memory-efficient options which are suitable for CPU-only cases: ONNX and OpenVINO.
- [HuggingFaceTEIRanker](huggingfaceteiranker.mdx) is based on the Text Embeddings Inference project: whether you have GPU resources or not, it offers high-performance for serving the models locally. In addition, you can also use this component to perform inference with reranking models hosted on Hugging Face Inference Endpoints.
- [FastembedRanker](fastembedranker.mdx) supports a variety of cross-encoder models and is optimal for CPU-only environments.
- [SentenceTransformersDiversityRanker](sentencetransformersdiversityranker.mdx) reorders documents to maximize diversity, helping reduce redundancy and cover a broader range of relevant topics.
These Rankers give you full control over model selection, optimization, and deployment, making them well-suited for production environments with strict SLAs or compliance requirements.
## Rule-Based Rankers
Rule-Based Rankers in Haystack prioritize or reorder documents based on heuristic logic rather than semantic understanding. They operate on document metadata or simple structural patterns, making them computationally efficient and useful for enforcing domain-specific rules or structuring inputs in a retrieval pipeline. While they do not assess semantic relevance directly, they serve as valuable complements to more advanced methods like cross-encoder or LLM-based Rankers.
For example:
- [MetaFieldRanker](metafieldranker.mdx) scores and orders documents based on metadata values such as recency, source reliability, or custom-defined priorities.
- [MetaFieldGroupingRanker](metafieldgroupingranker.mdx) groups documents by a specified metadata field and returns every document in each group together, ensuring that related documents (for example, from the same file) are processed as a single block, which has been shown to improve LLM performance.
- [LostInTheMiddleRanker](lostinthemiddleranker.mdx) reorders documents after ranking to mitigate position bias in models with limited context windows, ensuring that highly relevant items are not overlooked.
The **MetaFieldRanker** Ranker is typically used _before_ semantic ranking to filter or restructure documents according to business logic.
In contrast, **LostInTheMiddleRanker and MetaFieldGroupingRanker** are intended for use _after_ ranking, to improve the effectiveness of downstream components like LLMs. These deterministic approaches provide speed, transparency, and fine-grained control, making them well-suited for pipelines requiring explainability or strict operational logic.
---
// File: pipeline-components/rankers/cohereranker
# CohereRanker
Use this component to rank documents based on their similarity to the query using Cohere rerank models.
`aws_secret_access_key`: AWS secret access key. Can be set with AWS_SECRET_ACCESS_KEY env var.
`aws_region_name`: AWS region name. Can be set with AWS_DEFAULT_REGION env var. | | **Mandatory run variables** | `documents`: A list of document objects
`query`: A query string | | **Output variables** | `documents`: A list of document objects | | **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock/ | | **Package name** | `amazon-bedrock-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents such as a [Retriever](../retrievers.mdx) |
| **Mandatory init variables** | `api_key`: The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
| **Mandatory run variables** | `documents`: A list of document objects
`query`: A query string | | **Output variables** | `documents`: A list of document objects | | **API reference** | [Cohere](/reference/integrations-cohere) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere | | **Package name** | `cohere-haystack` |
## Overview
`CohereRanker` ranks `Documents` based on semantic relevance to a specified query. It uses Cohere rerank models for ranking. This list of all supported models can be found in Cohere’s [documentation](https://docs.cohere.com/docs/rerank-2). The default model for this Ranker is `rerank-v3.5`.
You can also specify the `top_k` parameter to set the maximum number of documents to return.
To start using this integration with Haystack, install it with:
```shell
pip install cohere-haystack
```
The component uses a `COHERE_API_KEY` or `CO_API_KEY` environment variable by default. Otherwise, you can pass a Cohere API key at initialization with `api_key` like this:
```python
ranker = CohereRanker(api_key=Secret.from_token("`query`: A query string | | **Output variables** | `documents`: A list of document objects | | **API reference** | [Cohere](/reference/integrations-cohere) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere | | **Package name** | `cohere-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents such as a [Retriever](../retrievers.mdx) |
| **Mandatory run variables** | `documents`: A list of documents
`query`: A query string | | **Output variables** | `documents`: A list of documents | | **API reference** | [FastEmbed](/reference/fastembed-embedders) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed | | **Package name** | `fastembed-haystack` |
## Overview
`FastembedLateInteractionRanker` ranks documents using **late interaction scoring**. Unlike cross-encoder rankers (which encode the query and document together), ColBERT encodes the query and each document independently into token-level embeddings, then computes a **MaxSim** score: for each query token, it finds the most similar document token, and sums these maximum similarities into a final relevance score.
This approach gives ColBERT a strong balance between accuracy and efficiency — it is more expressive than bi-encoders while being faster than cross-encoders at inference time.
`FastembedLateInteractionRanker` is most useful in query pipelines such as a retrieval-augmented generation (RAG) pipeline or a document search pipeline. Use it after a Retriever to rerank a candidate set of documents by relevance. When combining with a Retriever, set the Retriever's `top_k` higher than the Ranker's `top_k` — retrieve a broad candidate set, then let ColBERT select the best ones.
By default, this component uses the `colbert-ir/colbertv2.0` model. For details on different initialization settings, check out the [API reference](/reference/fastembed-embedders) page.
:::note
ColBERT scores are **unnormalized sums** (not probabilities). Their magnitude depends on query length and document length, typically ranging from ~3 to ~30. They are meaningful for ranking within a single query but should not be compared across different queries.
:::
### Compatible Models
You can find the compatible ColBERT models in the [FastEmbed documentation](https://qdrant.github.io/fastembed/examples/Supported_Models/).
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install fastembed-haystack
```
### Parameters
You can set the path where the model is stored in a cache directory. You can also set the number of threads a single `onnxruntime` session can use.
```python
ranker = FastembedLateInteractionRanker(
model_name="colbert-ir/colbertv2.0",
cache_dir="/your_cache_directory",
threads=2,
)
```
For offline encoding of large document sets, enable data-parallel processing:
```python
ranker = FastembedLateInteractionRanker(
model_name="colbert-ir/colbertv2.0",
batch_size=64,
parallel=2, # number of parallel processes; 0 = use all cores
)
```
## Usage
### On its own
This example uses `FastembedLateInteractionRanker` to rank two simple documents.
```python
from haystack import Document
from haystack_integrations.components.rankers.fastembed import (
FastembedLateInteractionRanker,
)
docs = [Document(content="Paris"), Document(content="Berlin")]
ranker = FastembedLateInteractionRanker(model_name="colbert-ir/colbertv2.0", top_k=1)
result = ranker.run(query="City in Germany", documents=docs)
print(result["documents"][0].content)
# Berlin
```
### In a pipeline
Below is an example of a full RAG pipeline that retrieves documents using embedding similarity, reranks them with `FastembedLateInteractionRanker`, and generates an answer with an LLM.
This example uses the `TransformersChatGenerator`, which requires additional packages:
```shell
pip install "transformers[torch]"
```
The examples on this page use Transformers components from the `transformers-haystack` package. Install it to run the examples:
```shell
pip install transformers-haystack
```
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack_integrations.components.generators.transformers import (
TransformersChatGenerator,
)
from haystack.components.writers import DocumentWriter
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.rankers.fastembed import (
FastembedLateInteractionRanker,
)
from haystack_integrations.components.embedders.fastembed import (
FastembedDocumentEmbedder,
FastembedTextEmbedder,
)
# Set up and populate the document store
document_store = InMemoryDocumentStore()
docs = [
Document(content="Paris is the capital of France."),
Document(content="Berlin is the capital of Germany."),
Document(content="Madrid is the capital of Spain."),
]
indexing = Pipeline()
indexing.add_component("embedder", FastembedDocumentEmbedder())
indexing.add_component("writer", DocumentWriter(document_store=document_store))
indexing.connect("embedder", "writer")
indexing.run({"embedder": {"documents": docs}})
# Define the chat prompt template
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given these documents, answer the question.\n"
"Documents:\n{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
"Question: {{query}}\nAnswer:",
),
]
# Build the query pipeline with ColBERT reranking
rag = Pipeline()
rag.add_component("text_embedder", FastembedTextEmbedder())
rag.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store, top_k=3),
)
rag.add_component(
"ranker",
FastembedLateInteractionRanker(model_name="colbert-ir/colbertv2.0", top_k=2),
)
rag.add_component(
"prompt_builder",
ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
),
)
rag.add_component(
"llm",
TransformersChatGenerator(model="HuggingFaceTB/SmolLM2-360M-Instruct"),
)
rag.connect("text_embedder.embedding", "retriever.query_embedding")
rag.connect("retriever.documents", "ranker.documents")
rag.connect("ranker.documents", "prompt_builder.documents")
rag.connect("prompt_builder.prompt", "llm.messages")
query = "What is the capital of Germany?"
result = rag.run(
{
"text_embedder": {"text": query},
"ranker": {"query": query},
"prompt_builder": {"query": query},
},
)
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/rankers/fastembedranker
# FastembedRanker
Use this component to rank documents based on their similarity to the query using cross-encoder models supported by FastEmbed.
`query`: A query string | | **Output variables** | `documents`: A list of documents | | **API reference** | [FastEmbed](/reference/fastembed-embedders) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed | | **Package name** | `fastembed-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents such as a [Retriever](../retrievers.mdx) |
| **Mandatory run variables** | `documents`: A list of documents
`query`: A query string | | **Output variables** | `documents`: A list of documents | | **API reference** | [FastEmbed](/reference/fastembed-embedders) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed | | **Package name** | `fastembed-haystack` |
## Overview
`FastembedRanker` ranks the documents based on how similar they are to the query. It uses [cross-encoder models supported by FastEmbed](https://qdrant.github.io/fastembed/examples/Supported_Models/).
Based on ONXX Runtime, FastEmbed provides a fast experience on standard CPU machines.
`FastembedRanker` is most useful in query pipelines such as a retrieval-augmented generation (RAG) pipeline or a document search pipeline to ensure the retrieved documents are ordered by relevance. You can use it after a Retriever (such as the [`InMemoryEmbeddingRetriever`](../retrievers/inmemoryembeddingretriever.mdx)) to improve the search results. When using `FastembedRanker` with a Retriever, consider setting the Retriever's `top_k` to a small number. This way, the Ranker will have fewer documents to process, which can help make your pipeline faster.
By default, this component uses the `Xenova/ms-marco-MiniLM-L-6-v2` model, but you can switch to a different model by adjusting the `model_name` parameter when initializing the Ranker. For details on different initialization settings, check out the [API reference](/reference/fastembed-embedders) page.
### Compatible Models
You can find the compatible models in the [FastEmbed documentation](https://qdrant.github.io/fastembed/examples/Supported_Models/).
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install fastembed-haystack
```
### Parameters
You can set the path where the model is stored in a cache directory. You can also set the number of threads a single `onnxruntime` session can use.
```python
cache_dir = "/your_cacheDirectory"
ranker = FastembedRanker(
model_name="Xenova/ms-marco-MiniLM-L-6-v2",
cache_dir=cache_dir,
threads=2,
)
```
If you want to use the data parallel encoding, you can set the parameters `parallel` and `batch_size`.
- If `parallel` > 1, data-parallel encoding will be used. This is recommended for offline encoding of large datasets.
- If `parallel` is 0, use all available cores.
- If None, don't use data-parallel processing; use default `onnxruntime` threading instead.
## Usage
### On its own
This example uses `FastembedRanker` to rank two simple documents. To run the Ranker, pass a `query`, provide the `documents`, and set the number of documents to return in the `top_k` parameter.
```python
from haystack import Document
from haystack_integrations.components.rankers.fastembed import FastembedRanker
docs = [Document(content="Paris"), Document(content="Berlin")]
ranker = FastembedRanker()
ranker.run(query="City in France", documents=docs, top_k=1)
```
### In a pipeline
Below is an example of a pipeline that retrieves documents from an `InMemoryDocumentStore` based on keyword search using `InMemoryBM25Retriever`. It then uses the `FastembedRanker` to rank the retrieved documents according to their similarity to the query. The pipeline uses the default settings of the Ranker.
```python
from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.rankers.fastembed import FastembedRanker
docs = [
Document(content="Paris is in France"),
Document(content="Berlin is in Germany"),
Document(content="Lyon is in France"),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
retriever = InMemoryBM25Retriever(document_store=document_store)
ranker = FastembedRanker()
document_ranker_pipeline = Pipeline()
document_ranker_pipeline.add_component(instance=retriever, name="retriever")
document_ranker_pipeline.add_component(instance=ranker, name="ranker")
document_ranker_pipeline.connect("retriever.documents", "ranker.documents")
query = "Cities in France"
res = document_ranker_pipeline.run(
data={
"retriever": {"query": query, "top_k": 3},
"ranker": {"query": query, "top_k": 2},
},
)
```
---
// File: pipeline-components/rankers/huggingfaceteiranker
# HuggingFaceTEIRanker
Use this component to rank documents based on their similarity to the query using a Text Embeddings Inference (TEI) API endpoint.
`query`: A query string | | **Output variables** | `documents`: A list of documents | | **API reference** | [FastEmbed](/reference/fastembed-embedders) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed | | **Package name** | `fastembed-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents, such as a [Retriever](../retrievers.mdx) |
| **Mandatory init variables** | `url`: Base URL of the TEI reranking service (for example, "https://api.example.com"). |
| **Mandatory run variables** | `query`: A query string
`documents`: A list of document objects | | **Output variables** | `documents`: A list of document objects | | **API reference** | [Hugging Face API](/reference/integrations-huggingface-api) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/huggingface_api | | **Package name** | `huggingface-api-haystack` |
## Overview
HuggingFaceTEIRanker ranks documents based on semantic relevance to a specified query.
You can use it with one of the Text Embeddings Inference (TEI) API endpoints:
- [Self-hosted Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference)
- [Hugging Face Inference Endpoints](https://huggingface.co/inference-endpoints)
You can also specify the `top_k` parameter to set the maximum number of documents to return.
Depending on your TEI server configuration, you may also require a Hugging Face [token](https://huggingface.co/settings/tokens) to use for authorization. You can set it with `HF_API_TOKEN` or `HF_TOKEN` environment variables, or by using Haystack's [Secret management](../../concepts/secret-management.mdx).
## Usage
Install the `huggingface-api-haystack` package to use the `HuggingFaceTEIRanker`:
```shell
pip install huggingface-api-haystack
```
### On its own
You can use `HuggingFaceTEIRanker` outside of a pipeline to order documents based on your query.
This example uses the `HuggingFaceTEIRanker` to rank two simple documents. To run the Ranker, pass a query, provide the documents, and set the number of documents to return in the `top_k` parameter.
```python
from haystack import Document
from haystack_integrations.components.rankers.huggingface_api import HuggingFaceTEIRanker
from haystack.utils import Secret
reranker = HuggingFaceTEIRanker(
url="http://localhost:8080",
top_k=5,
timeout=30,
token=Secret.from_token("my_api_token")
)
docs = [Document(content="The capital of France is Paris"), Document(content="The capital of Germany is Berlin")]
result = reranker.run(query="What is the capital of France?", documents=docs)
ranked_docs = result["documents"]
print(ranked_docs)
>> {'documents': [Document(id=..., content: 'the capital of France is Paris', score: 0.9979767),
>> Document(id=..., content: 'the capital of Germany is Berlin', score: 0.13982213)]}
```
### In a pipeline
`HuggingFaceTEIRanker` is most efficient in query pipelines when used after a Retriever.
Below is an example of a pipeline that retrieves documents from an `InMemoryDocumentStore` based on keyword search (using `InMemoryBM25Retriever`). It then uses the `HuggingFaceTEIRanker` to rank the retrieved documents according to their similarity to the query. The pipeline uses the default settings of the Ranker.
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack_integrations.components.rankers.huggingface_api import (
HuggingFaceTEIRanker,
)
docs = [
Document(content="Paris is in France"),
Document(content="Berlin is in Germany"),
Document(content="Lyon is in France"),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
retriever = InMemoryBM25Retriever(document_store=document_store)
ranker = HuggingFaceTEIRanker(url="http://localhost:8080")
document_ranker_pipeline = Pipeline()
document_ranker_pipeline.add_component(instance=retriever, name="retriever")
document_ranker_pipeline.add_component(instance=ranker, name="ranker")
document_ranker_pipeline.connect("retriever.documents", "ranker.documents")
query = "Cities in France"
document_ranker_pipeline.run(
data={
"retriever": {"query": query, "top_k": 3},
"ranker": {"query": query, "top_k": 2},
},
)
```
---
// File: pipeline-components/rankers/jinaranker
# JinaRanker
Use this component to rank documents based on their similarity to the query using Jina AI models.
`documents`: A list of document objects | | **Output variables** | `documents`: A list of document objects | | **API reference** | [Hugging Face API](/reference/integrations-huggingface-api) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/huggingface_api | | **Package name** | `huggingface-api-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents (such as a [Retriever](../retrievers.mdx) ) |
| **Mandatory init variables** | `api_key`: The Jina API key. Can be set with `JINA_API_KEY` env var. |
| **Mandatory run variables** | `query`: A query string
`documents`: A list of documents | | **Output variables** | `documents`: A list of documents
`meta`: A dictionary with the model used and usage information | | **API reference** | [Jina](/reference/integrations-jina) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina | | **Package name** | `jina-haystack` |
## Overview
`JinaRanker` ranks the given documents based on how similar they are to the given query. It uses Jina AI ranking models – check out the full list at Jina AI’s [website](https://jina.ai/reranker/). The default model for this Ranker is `jina-reranker-v1-base-en`.
Additionally, you can use the optional `top_k` and `score_threshold` parameters with `JinaRanker` :
- The Ranker's `top_k` is the number of documents it returns (if it's the last component in the pipeline) or forwards to the next component.
- If you set the `score_threshold` for the Ranker, it will only return documents with a similarity score (computed by the Jina AI model) above this threshold.
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install jina-haystack
```
### Authorization
The component uses a `JINA_API_KEY` environment variable by default. Otherwise, you can pass a Jina API key at initialization with `api_key` like this:
```python
ranker = JinaRanker(api_key=Secret.from_token("`documents`: A list of documents | | **Output variables** | `documents`: A list of documents
`meta`: A dictionary with the model used and usage information | | **API reference** | [Jina](/reference/integrations-jina) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina | | **Package name** | `jina-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents such as a [Retriever](../retrievers.mdx) |
| **Mandatory run variables** | `query`: A query string
`documents`: A list of document objects | | **Output variables** | `documents`: A list of documents | | **API reference** | [Rankers](/reference/rankers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/rankers/llm_ranker.py | | **Package name** | `haystack-ai` |
## Overview
`LLMRanker` uses an LLM to reorder documents by relevance to the query. Unlike cross-encoder rankers, it treats relevance as a semantic reasoning task, which can yield better results for complex or multi-step queries. The component sends the query and document contents to the LLM and parses the response as JSON: an array of objects with an `index` field (1-based document position). Only documents that the LLM includes in this list are returned, in the order given.
Before ranking, duplicate documents are removed. You can set `top_k` to limit how many documents are returned. If generation or parsing fails, the ranker either raises (when `raise_on_failure=True`) or returns the input documents in their original order (when `raise_on_failure=False`, the default).
You can pass any Haystack `ChatGenerator` that supports structured JSON output. If you omit `chat_generator`, a default `OpenAIChatGenerator` (e.g. `gpt-4.1-mini`) with JSON schema for the ranking response is used. You need to provide an OPENAI_API_KEY for this `ChatGenerator`. You can also provide a custom `prompt` template. It must include exactly the variables `query` and `documents` and instruct the LLM to return ranked 1-based document indices as JSON.
## Usage
### On its own
This example uses `LLMRanker` with the default `OpenAIChatGenerator` to rank two documents. The ranker returns documents in the order specified by the LLM.
```python
from haystack import Document
from haystack.components.rankers import LLMRanker
ranker = LLMRanker()
documents = [
Document(id="paris", content="Paris is the capital of France."),
Document(id="berlin", content="Berlin is the capital of Germany."),
]
result = ranker.run(query="capital of Germany", documents=documents)
print(result["documents"][0].id) # "berlin"
```
### With a custom chat generator
You can pass your own chat generator configured for JSON output (e.g. with `response_format` / JSON schema so the model returns the expected `documents` array with `index` fields):
```python
from haystack import Document
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.rankers import LLMRanker
chat_generator = OpenAIChatGenerator(
model="gpt-4.1-mini",
generation_kwargs={
"temperature": 0.0,
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "document_ranking",
"schema": {
"type": "object",
"properties": {
"documents": {
"type": "array",
"items": {
"type": "object",
"properties": {"index": {"type": "integer"}},
"required": ["index"],
"additionalProperties": False,
},
},
},
"required": ["documents"],
"additionalProperties": False,
},
},
},
},
)
ranker = LLMRanker(chat_generator=chat_generator)
documents = [
Document(content="Paris is the capital of France."),
Document(content="Berlin is the capital of Germany."),
]
result = ranker.run(query="capital of Germany", documents=documents, top_k=1)
```
### In a pipeline
Below is an example of a pipeline that retrieves documents with `InMemoryBM25Retriever` and then ranks them with `LLMRanker`:
```python
from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.rankers import LLMRanker
from haystack.document_stores.in_memory import InMemoryDocumentStore
docs = [
Document(content="Paris is in France."),
Document(content="Berlin is in Germany."),
Document(content="Lyon is in France."),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
retriever = InMemoryBM25Retriever(document_store=document_store)
ranker = LLMRanker(top_k=2)
pipeline = Pipeline()
pipeline.add_component(instance=retriever, name="retriever")
pipeline.add_component(instance=ranker, name="ranker")
pipeline.connect("retriever.documents", "ranker.documents")
query = "Cities in France"
result = pipeline.run(
data={
"retriever": {"query": query, "top_k": 3},
"ranker": {"query": query, "top_k": 2},
},
)
```
:::note[`top_k` parameter]
The Retriever's `top_k` controls how many documents are retrieved. The Ranker's `top_k` limits how many of those documents are returned after ranking. You can set the same or a smaller `top_k` for the Ranker to optimize cost and latency.
:::
---
// File: pipeline-components/rankers/lostinthemiddleranker
# LostInTheMiddleRanker
This Ranker positions the most relevant documents at the beginning and at the end of the resulting list while placing the least relevant Documents in the middle.
`documents`: A list of document objects | | **Output variables** | `documents`: A list of documents | | **API reference** | [Rankers](/reference/rankers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/rankers/llm_ranker.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents (such as a [Retriever](../retrievers.mdx) ) |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Rankers](/reference/rankers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/rankers/lost_in_the_middle.py |
| **Package name** | `haystack-ai` |
## Overview
The `LostInTheMiddleRanker` reorders the documents based on the "Lost in the Middle" order, described in the ["Lost in the Middle: How Language Models Use Long Contexts"](https://arxiv.org/abs/2307.03172) research paper. It aims to lay out paragraphs into LLM context so that the relevant paragraphs are at the beginning or end of the input context, while the least relevant information is in the middle of the context. This reordering is helpful when very long contexts are sent to an LLM, as current models pay more attention to the start and end of long input contexts.
In contrast to other rankers, `LostInTheMiddleRanker` assumes that the input documents are already sorted by relevance, and it doesn’t require a query as input. It is typically used as the last component before building a prompt for an LLM to prepare the input context for the LLM.
### Parameters
If you specify the `word_count_threshold` when running the component, the Ranker includes all documents up until the point where adding another document would exceed the given threshold. The last document that exceeds the threshold will be included in the resulting list of Documents, but all following documents will be discarded.
You can also specify the `top_k` parameter to set the maximum number of documents to return.
## Usage
### On its own
```python
from haystack import Document
from haystack.components.rankers import LostInTheMiddleRanker
ranker = LostInTheMiddleRanker()
docs = [
Document(content="Paris"),
Document(content="Berlin"),
Document(content="Madrid"),
]
result = ranker.run(documents=docs)
for doc in result["documents"]:
print(doc.content)
```
### In a pipeline
Note that this example requires an OpenAI key to run.
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.rankers import LostInTheMiddleRanker
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
# Define prompt template
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given these documents, answer the question.\nDocuments:\n"
"{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
"Question: {{query}}\nAnswer:",
),
]
# Define documents
docs = [
Document(content="Paris is in France..."),
Document(content="Berlin is in Germany..."),
Document(content="Lyon is in France..."),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
retriever = InMemoryBM25Retriever(document_store=document_store)
ranker = LostInTheMiddleRanker(word_count_threshold=1024)
prompt_builder = ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
)
generator = OpenAIChatGenerator()
p = Pipeline()
p.add_component(instance=retriever, name="retriever")
p.add_component(instance=ranker, name="ranker")
p.add_component(instance=prompt_builder, name="prompt_builder")
p.add_component(instance=generator, name="llm")
p.connect("retriever.documents", "ranker.documents")
p.connect("ranker.documents", "prompt_builder.documents")
p.connect("prompt_builder.prompt", "llm.messages")
p.run(
{
"retriever": {"query": "What cities are in France?", "top_k": 3},
"prompt_builder": {"query": "What cities are in France?"},
},
)
```
---
// File: pipeline-components/rankers/metafieldgroupingranker
# MetaFieldGroupingRanker
Reorder the documents by grouping them based on metadata keys.
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents, such as a [Retriever](../retrievers.mdx) |
| **Mandatory init variables** | `group_by`: The name of the meta field to group by |
| **Mandatory run variables** | `documents`: A list of documents to group |
| **Output variables** | `documents`: A grouped list of documents |
| **API reference** | [Rankers](/reference/rankers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/rankers/meta_field_grouping_ranker.py |
| **Package name** | `haystack-ai` |
## Overview
The `MetaFieldGroupingRanker` component groups documents by a primary metadata key `group_by`, and subgroups them with an optional secondary key, `subgroup_by`.
Within each group or subgroup, the component can also sort documents by a metadata key `sort_docs_by`.
The output is a flat list of documents ordered by `group_by` and `subgroup_by` values. Any documents without a group are placed at the end of the list.
The component helps improve the efficiency and performance of subsequent processing by an LLM.
## Usage
### On its own
```python
from haystack.components.rankers import MetaFieldGroupingRanker
from haystack import Document
docs = [
Document(
content="JavaScript is popular",
meta={"group": "42", "split_id": 7, "subgroup": "subB"},
),
Document(
content="Python is popular",
meta={"group": "42", "split_id": 4, "subgroup": "subB"},
),
Document(
content="A chromosome is DNA",
meta={"group": "314", "split_id": 2, "subgroup": "subC"},
),
Document(
content="An octopus has three hearts",
meta={"group": "11", "split_id": 2, "subgroup": "subD"},
),
Document(
content="Java is popular",
meta={"group": "42", "split_id": 3, "subgroup": "subB"},
),
]
ranker = MetaFieldGroupingRanker(
group_by="group",
subgroup_by="subgroup",
sort_docs_by="split_id",
)
result = ranker.run(documents=docs)
print(result["documents"])
```
### In a pipeline
The following pipeline uses the `MetaFieldGroupingRanker` to organize documents by certain meta fields while sorting by page number, then formats these organized documents into a chat message which is passed to the `OpenAIChatGenerator` to create a structured explanation of the content.
```python
from haystack import Pipeline
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.rankers import MetaFieldGroupingRanker
from haystack.dataclasses import Document, ChatMessage
docs = [
Document(
content="Chapter 1: Introduction to Python",
meta={"chapter": "1", "section": "intro", "page": 1},
),
Document(
content="Chapter 2: Basic Data Types",
meta={"chapter": "2", "section": "basics", "page": 15},
),
Document(
content="Chapter 1: Python Installation",
meta={"chapter": "1", "section": "setup", "page": 5},
),
]
ranker = MetaFieldGroupingRanker(
group_by="chapter",
subgroup_by="section",
sort_docs_by="page",
)
chat_generator = OpenAIChatGenerator(
generation_kwargs={"max_completion_tokens": 500},
)
# First run the ranker
ranked_result = ranker.run(documents=docs)
ranked_docs = ranked_result["documents"]
# Create chat messages with the ranked documents
messages = [
ChatMessage.from_system("You are a helpful programming tutor."),
ChatMessage.from_user(
f"Here are the course documents in order:\n"
+ "\n".join([f"- {doc.content}" for doc in ranked_docs])
+ "\n\nBased on these documents, explain the structure of this Python course.",
),
]
# Create and run pipeline for just the chat generator
pipeline = Pipeline()
pipeline.add_component("chat_generator", chat_generator)
result = pipeline.run(data={"chat_generator": {"messages": messages}})
print(result["chat_generator"]["replies"][0])
```
---
// File: pipeline-components/rankers/metafieldranker
# MetaFieldRanker
`MetaFieldRanker` ranks Documents based on the value of their meta field you specify. It's a lightweight Ranker that can improve your pipeline's results without slowing it down.
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents, such as a [Retriever](../retrievers.mdx) |
| **Mandatory init variables** | `meta_field`: The name of the meta field to rank by |
| **Mandatory run variables** | `documents`: A list of documents
`top_k`: The maximum number of documents to return. If not provided, returns all documents it received. | | **Output variables** | `documents`: A list of documents | | **API reference** | [Rankers](/reference/rankers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/rankers/meta_field.py | | **Package name** | `haystack-ai` |
## Overview
`MetaFieldRanker` sorts documents based on the value of a specific meta field in descending or ascending order. This means the returned list of `Document` objects are arranged in a selected order, with string values sorted alphabetically or in reverse (for example, Tokyo, Paris, Berlin).
`MetaFieldRanker` comes with the optional parameters `weight` and `ranking_mode` you can use to combine a document’s score assigned by the Retriever and the value of its meta field for the ranking. The `weight` parameter lets you balance the importance of the Document's content and the meta field in the ranking process. The `ranking_mode` parameter defines how the scores from the Retriever and the Ranker are combined.
This Ranker is useful in query pipelines, like retrieval-augmented generation (RAG) pipelines or document search pipelines. It ensures the documents are ordered by their meta field value. You can also use it after a Retriever (such as the `InMemoryEmbeddingRetriever`) to combine the Retriever’s score with a document’s meta value for improved ranking.
By default, `MetaFieldRanker` sorts documents only based on the meta field. You can adjust this by setting the `weight` to less than 1 when initializing this component. For more details on different initialization settings, check out the API reference for this component.
## Usage
### On its own
You can use this Ranker outside of a pipeline to sort documents.
This example uses the `MetaFieldRanker` to rank two simple documents. When running the Ranker, you provide the `documents` and set the number of documents to rank using the `top_k` parameter.
```python
from haystack import Document
from haystack.components.rankers import MetaFieldRanker
docs = [
Document(content="Paris", meta={"rating": 1.3}),
Document(content="Berlin", meta={"rating": 0.7}),
]
ranker = MetaFieldRanker(meta_field="rating")
ranker.run(documents=docs, top_k=1)
```
### In a pipeline
Below is an example of a pipeline that retrieves documents from an `InMemoryDocumentStore` based on keyword search (using `InMemoryBM25Retriever`). It then uses the `MetaFieldRanker` to rank the retrieved documents based on the meta field `rating`, using the Ranker's default settings:
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.rankers import MetaFieldRanker
docs = [
Document(content="Paris", meta={"rating": 1.3}),
Document(content="Berlin", meta={"rating": 0.7}),
Document(content="Barcelona", meta={"rating": 2.1}),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
retriever = InMemoryBM25Retriever(document_store=document_store)
ranker = MetaFieldRanker(meta_field="rating")
document_ranker_pipeline = Pipeline()
document_ranker_pipeline.add_component(instance=retriever, name="retriever")
document_ranker_pipeline.add_component(instance=ranker, name="ranker")
document_ranker_pipeline.connect("retriever.documents", "ranker.documents")
query = "Cities in France"
document_ranker_pipeline.run(
data={
"retriever": {"query": query, "top_k": 3},
"ranker": {"top_k": 2},
},
)
```
---
// File: pipeline-components/rankers/nvidiaranker
# NvidiaRanker
Use this component to rank documents based on their similarity to the query using Nvidia-hosted models.
`top_k`: The maximum number of documents to return. If not provided, returns all documents it received. | | **Output variables** | `documents`: A list of documents | | **API reference** | [Rankers](/reference/rankers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/rankers/meta_field.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents such as a [Retriever](../retrievers.mdx) |
| **Mandatory init variables** | `api_key`: API key for the NVIDIA NIM. Can be set with `NVIDIA_API_KEY` env var. |
| **Mandatory run variables** | `query`: A query string
`documents`: A list of document objects | | **Output variables** | `documents`: A list of document objects | | **API reference** | [Nvidia](/reference/integrations-nvidia) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia | | **Package name** | `nvidia-haystack` |
## Overview
`NvidiaRanker` ranks `Documents` based on semantic relevance to a specified query. It uses ranking models provided by [NVIDIA NIMs](https://ai.nvidia.com). If you don't set the `model` parameter, the hosted default `nv-rerank-qa-mistral-4b:1` is used.
You can also specify the `top_k` parameter to set the maximum number of documents to return.
See the rest of the customizable parameters you can set for `NvidiaRanker` in our [API reference](/reference/integrations-nvidia).
To start using this integration with Haystack, install it with:
```shell
pip install nvidia-haystack
```
The component uses an `NVIDIA_API_KEY` environment variable by default. Otherwise, you can pass an Nvidia API key at initialization with `api_key` like this:
```python
ranker = NvidiaRanker(api_key=Secret.from_token("`documents`: A list of document objects | | **Output variables** | `documents`: A list of document objects | | **API reference** | [Nvidia](/reference/integrations-nvidia) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia | | **Package name** | `nvidia-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a dense [Retriever](../retrievers.mdx) with `return_embedding=True` |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `documents`: A list of document objects, each with `score` and `embedding` set |
| **Output variables** | `documents`: A list of document objects |
| **API reference** | [Pyversity](/reference/integrations-pyversity) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/pyversity |
| **Package name** | `pyversity-haystack` |
## Overview
`PyversityRanker` reranks `Documents` using [pyversity](https://github.com/Pringled/pyversity)'s diversification algorithms. Unlike similarity-based rankers, it balances **relevance and diversity** - so the output isn't just the most relevant documents, but a varied selection that avoids redundancy.
Documents must have both `score` and `embedding` populated. This makes it a natural fit after a dense retriever such as `InMemoryEmbeddingRetriever` configured with `return_embedding=True`. Documents missing either field are skipped with a warning.
The key parameters are:
- `strategy`: The diversification algorithm to use. Defaults to `Strategy.DPP` (Determinantal Point Process). `Strategy.MMR` (Maximal Marginal Relevance) is another popular option.
- `diversity`: A float in `[0, 1]` controlling the relevance–diversity trade-off. `0.0` keeps the most relevant documents; `1.0` maximises diversity regardless of relevance. Defaults to `0.5`.
- `top_k`: The number of documents to return. If `None`, all documents are returned in diversified order.
### Installation
To start using this integration with Haystack, install the package with:
```shell
pip install pyversity-haystack
```
## Usage
### On its own
This example uses `PyversityRanker` to rerank five documents. Each document must have a `score` and `embedding` set. The ranker returns the top 3 documents using the MMR strategy with a diversity of `0.7`.
```python
from haystack import Document
from pyversity import Strategy
from haystack_integrations.components.rankers.pyversity import PyversityRanker
documents = [
Document(
content="Paris is the capital of France.",
score=0.95,
embedding=[0.9, 0.1, 0.0, 0.0],
),
Document(
content="The Eiffel Tower is located in Paris.",
score=0.90,
embedding=[0.8, 0.2, 0.0, 0.0],
),
Document(
content="Berlin is the capital of Germany.",
score=0.85,
embedding=[0.0, 0.0, 0.9, 0.1],
),
Document(
content="The Brandenburg Gate is in Berlin.",
score=0.80,
embedding=[0.0, 0.0, 0.8, 0.2],
),
Document(
content="France borders Spain to the south.",
score=0.75,
embedding=[0.5, 0.5, 0.0, 0.0],
),
]
ranker = PyversityRanker(top_k=3, strategy=Strategy.MMR, diversity=0.7)
result = ranker.run(documents=documents)
for doc in result["documents"]:
print(f"{doc.score:.2f} {doc.content}")
```
### In a pipeline
Below is an example of a pipeline that embeds documents and stores them in an `InMemoryDocumentStore`. It then retrieves the top 6 documents using `InMemoryEmbeddingRetriever` and reranks them with `PyversityRanker` to return 3 diverse results.
Note that the retriever must be configured with `return_embedding=True` so that documents have embeddings available for the ranker.
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document, Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.retrievers import InMemoryEmbeddingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from pyversity import Strategy
from haystack_integrations.components.rankers.pyversity import PyversityRanker
# Index documents
document_store = InMemoryDocumentStore()
raw_documents = [
Document(content="Paris is the capital of France."),
Document(content="The Eiffel Tower is located in Paris."),
Document(content="Berlin is the capital of Germany."),
Document(content="The Brandenburg Gate is in Berlin."),
Document(content="France borders Spain to the south."),
Document(content="The Louvre is the world's largest art museum and is in Paris."),
Document(content="Munich is the capital of Bavaria."),
Document(content="The Rhine river flows through Germany and France."),
]
doc_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = doc_embedder.run(raw_documents)["documents"]
document_store.write_documents(documents_with_embeddings)
# Build pipeline
pipeline = Pipeline()
pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(
document_store=document_store,
top_k=6,
return_embedding=True,
),
)
pipeline.add_component(
"ranker",
PyversityRanker(top_k=3, strategy=Strategy.MMR, diversity=0.7),
)
pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
pipeline.connect("retriever.documents", "ranker.documents")
# Run
result = pipeline.run(
{"text_embedder": {"text": "What are the famous landmarks in France?"}},
)
for doc in result["ranker"]["documents"]:
print(f"{doc.score:.4f} {doc.content}")
```
:::note[Embeddings required]
`PyversityRanker` requires documents to have both `score` and `embedding` set. When using a dense retriever, make sure to pass `return_embedding=True`. Documents missing either field are skipped with a warning.
:::
---
// File: pipeline-components/rankers/sentencetransformersdiversityranker
# SentenceTransformersDiversityRanker
This is a Diversity Ranker based on Sentence Transformers.
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents such as a [Retriever](../retrievers.mdx) |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `documents`: A list of documents
`query`: A query string | | **Output variables** | `documents`: A list of documents | | **API reference** | [Sentence Transformers](/reference/integrations-sentence-transformers) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sentence_transformers | | **Package name** | `sentence-transformers-haystack` |
## Overview
The `SentenceTransformersDiversityRanker` uses a ranking algorithm to order documents to maximize their overall diversity. It ranks a list of documents based on their similarity to the query. The component embeds the query and the documents using a pre-trained Sentence Transformers model.
This Ranker’s default model is `sentence-transformers/all-MiniLM-L6-v2`.
You can optionally set the `top_k` parameter, which specifies the maximum number of documents to return. It defaults to 10.
Authentication with a Hugging Face API token is only required to access private or gated models. You can pass the token at initialization with `token`, or set the `HF_API_TOKEN` or `HF_TOKEN` environment variable.
Find the full list of optional initialization parameters in our [API reference](/reference/integrations-sentence-transformers#sentencetransformersdiversityranker).
## Usage
Install the `sentence-transformers-haystack` package to use the `SentenceTransformersDiversityRanker`:
```shell
pip install sentence-transformers-haystack
```
### On its own
```python
from haystack import Document
from haystack_integrations.components.rankers.sentence_transformers import (
SentenceTransformersDiversityRanker,
)
ranker = SentenceTransformersDiversityRanker(
model="sentence-transformers/all-MiniLM-L6-v2",
similarity="cosine",
)
docs = [
Document(content="Regular Exercise"),
Document(content="Balanced Nutrition"),
Document(content="Positive Mindset"),
Document(content="Eating Well"),
Document(content="Doing physical activities"),
Document(content="Thinking positively"),
]
query = "How can I maintain physical fitness?"
output = ranker.run(query=query, documents=docs)
docs = output["documents"]
print(docs)
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack_integrations.components.rankers.sentence_transformers import (
SentenceTransformersDiversityRanker,
)
docs = [
Document(content="The iconic Eiffel Tower is a symbol of Paris"),
Document(content="Visit Luxembourg Gardens for a haven of tranquility in Paris"),
Document(
content="The Point Alexandre III bridge in Paris is famous for its Beaux-Arts style",
),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
retriever = InMemoryBM25Retriever(document_store=document_store)
ranker = SentenceTransformersDiversityRanker()
document_ranker_pipeline = Pipeline()
document_ranker_pipeline.add_component(instance=retriever, name="retriever")
document_ranker_pipeline.add_component(instance=ranker, name="ranker")
document_ranker_pipeline.connect("retriever.documents", "ranker.documents")
query = "Most famous iconic sight in Paris"
document_ranker_pipeline.run(
data={
"retriever": {"query": query, "top_k": 3},
"ranker": {"query": query, "top_k": 2},
},
)
```
---
// File: pipeline-components/rankers/sentencetransformerssimilarityranker
# SentenceTransformersSimilarityRanker
Use this component to rank documents based on their similarity to the query. The SentenceTransformersSimilarityRanker is a powerful, model-based Ranker that uses a cross-encoder model to produce document and query embeddings.
`query`: A query string | | **Output variables** | `documents`: A list of documents | | **API reference** | [Sentence Transformers](/reference/integrations-sentence-transformers) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sentence_transformers | | **Package name** | `sentence-transformers-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents such as a [Retriever](../retrievers.mdx) |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `documents`: A list of documents
`query`: A query string | | **Output variables** | `documents`: A list of documents | | **API reference** | [Sentence Transformers](/reference/integrations-sentence-transformers) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sentence_transformers | | **Package name** | `sentence-transformers-haystack` |
## Overview
`SentenceTransformersSimilarityRanker` ranks documents based on how similar they are to the query. It uses a pre-trained cross-encoder model from the Hugging Face Hub to embed both the query and the documents. It then compares the embeddings to determine how similar they are. The result is a list of `Document` objects in ranked order, with the Documents most similar to the query appearing first.
`SentenceTransformersSimilarityRanker` is most useful in query pipelines, such as a retrieval-augmented generation (RAG) pipeline or a document search pipeline, to ensure the retrieved documents are ordered by relevance. You can use it after a Retriever (such as the `InMemoryEmbeddingRetriever`) to improve the search results. When using `SentenceTransformersSimilarityRanker` with a Retriever, consider setting the Retriever's `top_k` to a small number. This way, the Ranker will have fewer documents to process, which can help make your pipeline faster.
By default, this component uses the `cross-encoder/ms-marco-MiniLM-L-6-v2` model, but it's flexible. You can switch to a different model by adjusting the `model` parameter when initializing the Ranker. For details on different initialization settings, check out the API reference for this component.
You can set the `device` parameter to use HF models on your CPU or GPU.
Additionally, you can select the backend to use for the Sentence Transformers mode with the `backend` parameter: `torch` (default), `onnx`, or `openvino`.
### Authorization
Authentication with a Hugging Face API token is only required to access private or gated models. The component uses a `HF_API_TOKEN` environment variable by default. Otherwise, you can pass a Hugging Face API token at initialization with [Secret](../../concepts/secret-management.mdx) `token`:
```python
ranker = SentenceTransformersSimilarityRanker(token=Secret.from_token("`query`: A query string | | **Output variables** | `documents`: A list of documents | | **API reference** | [Sentence Transformers](/reference/integrations-sentence-transformers) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sentence_transformers | | **Package name** | `sentence-transformers-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In a query pipeline, after a component that returns a list of documents such as a [Retriever](../retrievers.mdx) |
| **Mandatory init variables** | `model`: The name of the reranker model served by vLLM |
| **Mandatory run variables** | `query`: A query string
`documents`: A list of document objects | | **Output variables** | `documents`: A list of document objects
`meta`: A dictionary with the model used and usage information | | **API reference** | [vLLM](/reference/integrations-vllm) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/vllm | | **Package name** | `vllm-haystack` |
## Overview
[vLLM](https://docs.vllm.ai/) is a high-throughput and memory-efficient inference and serving engine for LLMs. It exposes an HTTP server, which `VLLMRanker` uses to rerank documents through the `/rerank` endpoint.
`VLLMRanker` expects a vLLM server to be running and accessible at the `api_base_url` parameter (by default, `http://localhost:8000/v1`). Use this component after a Retriever in a query pipeline to reorder the retrieved documents by relevance to the query.
You can also specify the `top_k` parameter to set the maximum number of documents to return, and the `score_threshold` parameter to drop documents with a relevance score below a given value.
If the vLLM server was started with `--api-key`, provide the API key through the `VLLM_API_KEY` environment variable or the `api_key` init parameter using Haystack's [Secret](../../concepts/secret-management.mdx) API.
### Compatible models
vLLM supports a range of reranker models. Check the [vLLM supported models docs](https://docs.vllm.ai/en/stable/models/pooling_models/scoring/#supported-models) for the list of supported architectures and models.
### vLLM-specific parameters
You can pass vLLM-specific parameters through the `extra_parameters` dictionary. These are merged into the request body sent to the `/rerank` endpoint. Use this to pass parameters that are not part of the standard rerank API, such as `truncate_prompt_tokens`. See the [vLLM rerank API docs](https://docs.vllm.ai/en/stable/models/pooling_models/scoring/#rerank-api) for details.
```python
ranker = VLLMRanker(
model="BAAI/bge-reranker-base",
extra_parameters={"truncate_prompt_tokens": 256},
)
```
### Embedding meta fields
Some use cases benefit from including meta information (such as a title) alongside the document content when reranking. Pass the names of the meta fields to include through the `meta_fields_to_embed` parameter; they will be concatenated with the document content using `meta_data_separator`.
```python
ranker = VLLMRanker(
model="BAAI/bge-reranker-base",
meta_fields_to_embed=["title"],
meta_data_separator="\n",
)
```
## Usage
Install the `vllm-haystack` package to use the `VLLMRanker`:
```shell
pip install vllm-haystack
```
### Starting the vLLM server
Before using this component, start a vLLM server with a reranker model:
```bash
vllm serve BAAI/bge-reranker-base
```
For details on server options, see the [vLLM CLI docs](https://docs.vllm.ai/en/stable/cli/serve/).
### On its own
```python
from haystack import Document
from haystack_integrations.components.rankers.vllm import VLLMRanker
ranker = VLLMRanker(model="BAAI/bge-reranker-base")
docs = [
Document(content="The capital of Brazil is Brasilia."),
Document(content="The capital of France is Paris."),
]
result = ranker.run(query="What is the capital of France?", documents=docs)
print(result["documents"][0].content)
# The capital of France is Paris.
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.rankers.vllm import VLLMRanker
docs = [
Document(content="Paris is in France"),
Document(content="Berlin is in Germany"),
Document(content="Lyon is in France"),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
retriever = InMemoryBM25Retriever(document_store=document_store)
ranker = VLLMRanker(model="BAAI/bge-reranker-base")
document_ranker_pipeline = Pipeline()
document_ranker_pipeline.add_component(instance=retriever, name="retriever")
document_ranker_pipeline.add_component(instance=ranker, name="ranker")
document_ranker_pipeline.connect("retriever.documents", "ranker.documents")
query = "Cities in France"
result = document_ranker_pipeline.run(
data={
"retriever": {"query": query, "top_k": 3},
"ranker": {"query": query, "top_k": 2},
},
)
print(result["ranker"]["documents"][0])
# Document(id=..., content: 'Paris is in France', score: ...)
```
---
// File: pipeline-components/rankers
# Rankers
Rankers are a group of components that order documents by given criteria. Their goal is to improve your document retrieval results.
| Ranker | Description |
| --- | --- |
| [AmazonBedrockRanker](rankers/amazonbedrockranker.mdx) | Ranks documents based on their similarity to the query using Amazon Bedrock models. |
| [CohereRanker](rankers/cohereranker.mdx) | Ranks documents based on their similarity to the query using Cohere rerank models. |
| [FastembedRanker](rankers/fastembedranker.mdx) | Ranks documents based on their similarity to the query using cross-encoder models supported by FastEmbed. |
| [FastembedLateInteractionRanker](rankers/fastembedlateinteractionranker.mdx) | Ranks documents based on their similarity to the query using late interaction models supported by FastEmbed. |
| [HuggingFaceTEIRanker](rankers/huggingfaceteiranker.mdx) | Ranks documents based on their similarity to the query using a Text Embeddings Inference (TEI) API endpoint. |
| [JinaRanker](rankers/jinaranker.mdx) | Ranks documents based on their similarity to the query using Jina AI models. |
| [LLMRanker](rankers/llmranker.mdx) | Ranks documents for a query using a Large Language Model, which returns ranked document indices as JSON. |
| [LostInTheMiddleRanker](rankers/lostinthemiddleranker.mdx) | Positions the most relevant documents at the beginning and at the end of the resulting list while placing the least relevant documents in the middle, based on a [research paper](https://arxiv.org/abs/2307.03172). |
| [MetaFieldRanker](rankers/metafieldranker.mdx) | A lightweight Ranker that orders documents based on a specific metadata field value. |
| [MetaFieldGroupingRanker](rankers/metafieldgroupingranker.mdx) | Reorders the documents by grouping them based on metadata keys. |
| [NvidiaRanker](rankers/nvidiaranker.mdx) | Ranks documents using large-language models from [NVIDIA NIMs](https://ai.nvidia.com) . |
| [PyversityRanker](rankers/pyversityranker.mdx) | Reranks documents by balancing relevance and diversity using pyversity's diversification algorithms. |
| [SentenceTransformersDiversityRanker](rankers/sentencetransformersdiversityranker.mdx) | A Diversity Ranker based on Sentence Transformers. |
| [SentenceTransformersSimilarityRanker](rankers/sentencetransformerssimilarityranker.mdx) | A model-based Ranker that orders documents based on their relevance to the query. It uses a cross-encoder model to produce query and document embeddings. It then compares the similarity of the query embedding to the document embeddings to produce a ranking with the most similar documents appearing first. `documents`: A list of document objects | | **Output variables** | `documents`: A list of document objects
`meta`: A dictionary with the model used and usage information | | **API reference** | [vLLM](/reference/integrations-vllm) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/vllm | | **Package name** | `vllm-haystack` |
It's a powerful Ranker that takes word order and syntax into account. You can use it to improve the initial ranking done by a weaker Retriever, but it's also more expensive computationally than the Rankers that don't use models. | | [VLLMRanker](rankers/vllmranker.mdx) | Ranks documents based on their similarity to the query using reranker models served with vLLM. | --- // File: pipeline-components/readers/transformersextractivereader # TransformersExtractiveReader Use this component in extractive question answering pipelines based on a query and a list of documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | In query pipelines, after a component that returns a list of documents, such as a [Retriever](../retrievers.mdx) |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `documents`: A list of documents
`query`: A query string | | **Output variables** | `answers`: A list of [`ExtractedAnswer`](../../concepts/data-classes.mdx#extractedanswer) objects | | **API reference** | [Transformers](/reference/integrations-transformers) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/transformers | | **Package name** | `transformers-haystack` |
## Overview
`TransformersExtractiveReader` locates and extracts answers to a given query from the document text. It's used in extractive QA systems where you want to know exactly where the answer is located within the document. It's usually coupled with a Retriever that precedes it, but you can also use it with other components that fetch documents.
Readers assign a _probability_ to answers. This score ranges from 0 to 1, indicating how well the results the Reader returned match the query. Probability closest to 1 means the model has high confidence in the answer's relevance. The Reader sorts the answers based on their probability scores, with higher probability listed first. You can limit the number of answers the Reader returns in the optional `top_k` parameter.
You can use the probability to set the quality expectations for your system. To do that, use the `confidence_score` parameter of the Reader to set a minimum probability threshold for answers. For example, setting `confidence_threshold` to `0.7` means only answers with a probability higher than 0.7 will be returned.
By default, the Reader includes a scenario where no answer to the query is found in the document text (`no_answer=True`). In this case, it returns an additional `ExtractedAnswer` with no text and the probability that none of the `top_k` answers are correct. For example, if `top_k=4` the system will return four answers and an additional empty one. Each answer has a probability assigned. If the empty answer has a probability of 0.5, it means that's the probability that none of the returned answers is correct. To receive only the actual top_k answers, set the `no_answer` parameter to `False` when initializing the component.
### Models
Here are the models that we recommend for using with `TransformersExtractiveReader`:
| | | |
| --- | --- | --- |
| Model URL | Description | Language |
| [deepset/roberta-base-squad2-distilled](https://huggingface.co/deepset/roberta-base-squad2-distilled) (default) | A distilled model, relatively fast and with good performance. | English |
| [deepset/roberta-large-squad2](https://huggingface.co/deepset/roberta-large-squad2) | A large model with good performance. Slower than the distilled one. | English |
| [deepset/tinyroberta-squad2](https://huggingface.co/deepset/tinyroberta-squad2) | A distilled version of roberta-large-squad2 model, very fast. | English |
| [deepset/xlm-roberta-base-squad2](https://huggingface.co/deepset/xlm-roberta-base-squad2) | A base multilingual model with good speed and performance. | Multilingual |
You can also view other question answering models on [Hugging Face](https://huggingface.co/models?pipeline_tag=question-answering).
Authentication with a Hugging Face API token is only required to access private or gated models. You can pass the token at initialization with `token`, or set the `HF_API_TOKEN` or `HF_TOKEN` environment variable.
## Usage
Install the `transformers-haystack` package to use the `TransformersExtractiveReader`:
```shell
pip install transformers-haystack
```
### On its own
Below is an example that uses the `TransformersExtractiveReader` outside of a pipeline. The Reader gets the query and the documents at runtime. It should return two answers and an additional third answer with no text and the probability that the `top_k` answers are incorrect.
```python
from haystack import Document
from haystack_integrations.components.readers.transformers import (
TransformersExtractiveReader,
)
docs = [
Document(content="Paris is the capital of France."),
Document(content="Berlin is the capital of Germany."),
]
reader = TransformersExtractiveReader()
reader.run(query="What is the capital of France?", documents=docs, top_k=2)
```
### In a pipeline
Below is an example of a pipeline that retrieves a document from an `InMemoryDocumentStore` based on keyword search (using `InMemoryBM25Retriever`). It then uses the `TransformersExtractiveReader` to extract the answer to our query from the top retrieved documents.
With the TransformersExtractiveReader’s `top_k` set to 2, an additional, third answer with no text and the probability that the other `top_k` answers are incorrect is also returned.
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack_integrations.components.readers.transformers import (
TransformersExtractiveReader,
)
docs = [
Document(content="Paris is the capital of France."),
Document(content="Berlin is the capital of Germany."),
Document(content="Rome is the capital of Italy."),
Document(content="Madrid is the capital of Spain."),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
retriever = InMemoryBM25Retriever(document_store=document_store)
reader = TransformersExtractiveReader()
extractive_qa_pipeline = Pipeline()
extractive_qa_pipeline.add_component(instance=retriever, name="retriever")
extractive_qa_pipeline.add_component(instance=reader, name="reader")
extractive_qa_pipeline.connect("retriever.documents", "reader.documents")
query = "What is the capital of France?"
extractive_qa_pipeline.run(
data={
"retriever": {"query": query, "top_k": 3},
"reader": {"query": query, "top_k": 2},
},
)
```
---
// File: pipeline-components/readers
# Readers
Readers are pipeline components that pinpoint answers in documents. They’re used in extractive question answering systems.
Currently, there's one Reader available in Haystack: [TransformersExtractiveReader](readers/transformersextractivereader.mdx).
---
// File: pipeline-components/retrievers/alloydbembeddingretriever
# AlloyDBEmbeddingRetriever
An embedding-based Retriever compatible with the AlloyDB Document Store.
`query`: A query string | | **Output variables** | `answers`: A list of [`ExtractedAnswer`](../../concepts/data-classes.mdx#extractedanswer) objects | | **API reference** | [Transformers](/reference/integrations-transformers) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/transformers | | **Package name** | `transformers-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of an [AlloyDBDocumentStore](../../document-stores/alloydbdocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [AlloyDB](/reference/integrations-alloydb) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/alloydb |
| **Package name** | `alloydb-haystack` |
## Overview
The `AlloyDBEmbeddingRetriever` is an embedding-based Retriever compatible with the `AlloyDBDocumentStore`. It compares the query and Document embeddings and fetches the Documents most relevant to the query from the `AlloyDBDocumentStore` based on the outcome.
When using the `AlloyDBEmbeddingRetriever` in your Pipeline, make sure it has the query and Document embeddings available. You can do so by adding a Document Embedder to your indexing Pipeline and a Text Embedder to your query Pipeline.
In addition to the `query_embedding`, the `AlloyDBEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve), `filters` to narrow down the search space, and `vector_function` to override the similarity function set on the Document Store.
Some relevant parameters that impact embedding retrieval must be defined when the corresponding `AlloyDBDocumentStore` is initialized: these include `embedding_dimension`, `vector_function`, and the search strategy (`"exact_nearest_neighbor"` or `"hnsw"`).
## Installation
Install the `alloydb-haystack` integration:
```shell
pip install alloydb-haystack
```
To set up an AlloyDB cluster and instance, follow the [AlloyDB quickstart](https://cloud.google.com/alloydb/docs/quickstart).
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
### On its own
This Retriever needs the `AlloyDBDocumentStore` and indexed Documents to run.
Set the `ALLOYDB_INSTANCE_URI`, `ALLOYDB_USER`, and `ALLOYDB_PASSWORD` environment variables to connect to your AlloyDB instance.
```python
from haystack_integrations.document_stores.alloydb import AlloyDBDocumentStore
from haystack_integrations.components.retrievers.alloydb import (
AlloyDBEmbeddingRetriever,
)
document_store = AlloyDBDocumentStore()
retriever = AlloyDBEmbeddingRetriever(document_store=document_store)
## using a fake vector to keep the example simple
retriever.run(query_embedding=[0.1] * 768)
```
### In a Pipeline
```python
from haystack import Document, Pipeline
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack_integrations.document_stores.alloydb import AlloyDBDocumentStore
from haystack_integrations.components.retrievers.alloydb import (
AlloyDBEmbeddingRetriever,
)
document_store = AlloyDBDocumentStore(
embedding_dimension=768,
vector_function="cosine_similarity",
recreate_table=True,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings.get("documents"),
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
AlloyDBEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/alloydbkeywordretriever
# AlloyDBKeywordRetriever
A keyword-based Retriever that fetches documents matching a query from the AlloyDB Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of an [AlloyDBDocumentStore](../../document-stores/alloydbdocumentstore.mdx) |
| **Mandatory run variables** | `query`: A string |
| **Output variables** | `documents`: A list of documents (matching the query) |
| **API reference** | [AlloyDB](/reference/integrations-alloydb) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/alloydb |
| **Package name** | `alloydb-haystack` |
## Overview
The `AlloyDBKeywordRetriever` is a keyword-based Retriever compatible with the `AlloyDBDocumentStore`.
It uses PostgreSQL full-text search (`to_tsvector` / `plainto_tsquery`) to find Documents and ranks them with `ts_rank_cd`. The ranking considers how often the query terms appear in the Document, how close together the terms are, and how important the part of the Document is where they occur. For more details, see the [PostgreSQL documentation](https://www.postgresql.org/docs/current/textsearch-controls.html#TEXTSEARCH-RANKING).
Keep in mind that, unlike similar components such as `ElasticsearchBM25Retriever`, this Retriever does not apply fuzzy search out of the box, so it’s necessary to carefully formulate the query in order to avoid getting zero results.
The language used to parse query and Document content for keyword retrieval is set via the `language` parameter on the `AlloyDBDocumentStore` (defaults to `"english"`). To list the supported languages on your database, run:
```sql
SELECT cfgname FROM pg_ts_config;
```
In addition to the `query`, the `AlloyDBKeywordRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow the search space.
## Installation
Install the `alloydb-haystack` integration:
```shell
pip install alloydb-haystack
```
To set up an AlloyDB cluster and instance, follow the [AlloyDB quickstart](https://cloud.google.com/alloydb/docs/quickstart).
## Usage
### On its own
This Retriever needs the `AlloyDBDocumentStore` and indexed Documents to run.
Set the `ALLOYDB_INSTANCE_URI`, `ALLOYDB_USER`, and `ALLOYDB_PASSWORD` environment variables to connect to your AlloyDB instance.
```python
from haystack_integrations.document_stores.alloydb import AlloyDBDocumentStore
from haystack_integrations.components.retrievers.alloydb import (
AlloyDBKeywordRetriever,
)
document_store = AlloyDBDocumentStore()
retriever = AlloyDBKeywordRetriever(document_store=document_store)
retriever.run(query="my nice query")
```
### In a RAG pipeline
The prerequisites necessary for running this code are:
- Set an environment variable `OPENAI_API_KEY` with your OpenAI API key.
- Set the `ALLOYDB_INSTANCE_URI`, `ALLOYDB_USER`, and `ALLOYDB_PASSWORD` environment variables to connect to your AlloyDB instance.
```python
from haystack import Document, Pipeline
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.document_stores.alloydb import AlloyDBDocumentStore
from haystack_integrations.components.retrievers.alloydb import (
AlloyDBKeywordRetriever,
)
## Create a RAG query pipeline
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given these documents, answer the question.\nDocuments:\n"
"{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
"Question: {{question}}\nAnswer:",
),
]
document_store = AlloyDBDocumentStore(
language="english", # this parameter influences text parsing for keyword retrieval
recreate_table=True,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_store.write_documents(documents=documents, policy=DuplicatePolicy.SKIP)
retriever = AlloyDBKeywordRetriever(document_store=document_store)
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="retriever", instance=retriever)
rag_pipeline.add_component(
instance=ChatPromptBuilder(
template=prompt_template,
required_variables={"question", "documents"},
),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")
question = "languages spoken around the world today"
result = rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
"answer_builder": {"query": question},
},
)
print(result["answer_builder"])
```
---
// File: pipeline-components/retrievers/arangoembeddingretriever
# ArangoEmbeddingRetriever
An embedding-based Retriever compatible with the ArangoDB Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline
2. The last component in a semantic search pipeline | | **Mandatory init variables** | `document_store`: An instance of an [ArangoDocumentStore](../../document-stores/arangodocumentstore.mdx) | | **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) | | **Output variables** | `documents`: A list of documents | | **API reference** | [ArangoDB](/reference/integrations-arangodb) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/arangodb | | **Package name** | `arangodb-haystack` |
## Overview
The `ArangoEmbeddingRetriever` retrieves documents from an `ArangoDocumentStore` using ArangoDB's AQL vector functions. It compares the query embedding with document embeddings and returns the most similar documents.
In addition to `query_embedding`, the retriever accepts optional `filters` to narrow the search space and `top_k` to limit the number of results. Both can be set at initialization and overridden per call to `run()`.
The embedding dimension and similarity function (`cosine`, `dot_product`, or `l2`) are configured on the `ArangoDocumentStore` at initialization time.
## Installation
```shell
pip install arangodb-haystack
```
Ensure ArangoDB 3.12+ is running with the vector index enabled, for example via Docker:
```shell
docker run -d -p 8529:8529 \
-e ARANGO_ROOT_PASSWORD=test-password \
arangodb:3.12 arangod --vector-index
```
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
### On its own
```python
from haystack import Document
from haystack_integrations.document_stores.arangodb import ArangoDocumentStore
from haystack_integrations.components.retrievers.arangodb import (
ArangoEmbeddingRetriever,
)
document_store = ArangoDocumentStore(
host="http://localhost:8529",
embedding_dimension=3,
recreate_collection=True,
)
document_store.write_documents(
[
Document(
content="There are over 7,000 languages spoken around the world today.",
embedding=[0.1, 0.2, 0.3],
),
Document(
content="Elephants have been observed to recognize themselves in mirrors.",
embedding=[0.8, 0.1, 0.5],
),
],
)
retriever = ArangoEmbeddingRetriever(document_store=document_store, top_k=1)
result = retriever.run(query_embedding=[0.1, 0.2, 0.3])
print(result["documents"][0].content)
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack_integrations.document_stores.arangodb import ArangoDocumentStore
from haystack_integrations.components.retrievers.arangodb import (
ArangoEmbeddingRetriever,
)
document_store = ArangoDocumentStore(
host="http://localhost:8529",
embedding_dimension=384,
recreate_collection=True,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to recognize themselves in mirrors.",
),
Document(
content="Bioluminescent waves can be seen in the Maldives and Puerto Rico.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings["documents"],
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"),
)
query_pipeline.add_component(
"retriever",
ArangoEmbeddingRetriever(document_store=document_store, top_k=3),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
result = query_pipeline.run(
{"text_embedder": {"text": "How many languages are there?"}},
)
print(result["retriever"]["documents"][0].content)
```
---
// File: pipeline-components/retrievers/arcadedbembeddingretriever
# ArcadeDBEmbeddingRetriever
An embedding-based Retriever compatible with the ArcadeDB Document Store. It uses ArcadeDB's LSM_VECTOR (HNSW) index for vector similarity search.
2. The last component in a semantic search pipeline | | **Mandatory init variables** | `document_store`: An instance of an [ArangoDocumentStore](../../document-stores/arangodocumentstore.mdx) | | **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) | | **Output variables** | `documents`: A list of documents | | **API reference** | [ArangoDB](/reference/integrations-arangodb) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/arangodb | | **Package name** | `arangodb-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) in a RAG pipeline 2. The last component in a semantic search pipeline |
| **Mandatory init variables** | `document_store`: An instance of [ArcadeDBDocumentStore](../../document-stores/arcadedbdocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [ArcadeDB](/reference/integrations-arcadedb) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/arcadedb |
| **Package name** | `arcadedb-haystack` |
## Overview
The `ArcadeDBEmbeddingRetriever` retrieves documents from `ArcadeDBDocumentStore` by comparing the query embedding with document embeddings using the store's HNSW index. It accepts optional `filters` for metadata filtering and `top_k` to limit the number of results. Use a Document Embedder in your indexing pipeline and a Text Embedder in your query pipeline so embeddings are available.
## Installation
```shell
pip install arcadedb-haystack
```
Ensure ArcadeDB is running, for example via Docker, and credentials are set (`ARCADEDB_USERNAME`, `ARCADEDB_PASSWORD`).
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
### On its own
```python
from haystack_integrations.document_stores.arcadedb import ArcadeDBDocumentStore
from haystack_integrations.components.retrievers.arcadedb import (
ArcadeDBEmbeddingRetriever,
)
document_store = ArcadeDBDocumentStore(
url="http://localhost:2480",
database="haystack",
embedding_dimension=768,
)
retriever = ArcadeDBEmbeddingRetriever(document_store=document_store, top_k=5)
# Example: run with a query embedding (e.g. from an embedder)
result = retriever.run(query_embedding=[0.1] * 768)
for doc in result["documents"]:
print(doc.content)
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack_integrations.document_stores.arcadedb import ArcadeDBDocumentStore
from haystack_integrations.components.retrievers.arcadedb import (
ArcadeDBEmbeddingRetriever,
)
document_store = ArcadeDBDocumentStore(
url="http://localhost:2480",
database="haystack",
embedding_dimension=768,
recreate_type=True,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to recognize themselves in mirrors.",
),
Document(
content="Bioluminescent waves can be seen in the Maldives and Puerto Rico.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings["documents"],
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
ArcadeDBEmbeddingRetriever(document_store=document_store, top_k=3),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
result = query_pipeline.run(
{"text_embedder": {"text": "How many languages are there?"}},
)
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/astraretriever
# AstraEmbeddingRetriever
This is an embedding-based Retriever compatible with the Astra Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline
2. The last component in the semantic search pipeline
3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of [AstraDocumentStore](../../document-stores/astradocumentstore.mdx) | | **Mandatory run variables** | `query_embedding`: A list of floats | | **Output variables** | `documents`: A list of documents | | **API reference** | [Astra](/reference/integrations-astra) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/astra | | **Package name** | `astra-haystack` |
## Overview
`AstraEmbeddingRetriever` compares the query and document embeddings and fetches the documents most relevant to the query from the [`AstraDocumentStore`](../../document-stores/astradocumentstore.mdx) based on the outcome.
When using the `AstraEmbeddingRetriever` in your NLP system, make sure it has the query and document embeddings available. You can do so by adding a Document Embedder to your indexing pipeline and a Text Embedder to your query pipeline.
In addition to the `query_embedding`, the `AstraEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of documents to retrieve) and `filters` to narrow down the search space.
### Setup and installation
Once you have an AstraDB account and have created a database, install the `astra-haystack` integration:
```shell
pip install astra-haystack
```
From the configuration in AstraDB’s web UI, you need the database ID and a generated token.
You will additionally need a collection name and a namespace. When you create the collection name, you also need to set the embedding dimensions and the similarity metric. The namespace organizes data in a database and is called a keyspace in Apache Cassandra.
Then, optionally, install the `sentence-transformers-haystack` package as well to run the example below:
```shell
pip install sentence-transformers-haystack
```
## Usage
We strongly encourage passing authentication data through environment variables: make sure to populate the environment variables `ASTRA_DB_API_ENDPOINT` and `ASTRA_DB_APPLICATION_TOKEN` before running the following example.
### In a pipeline
Use this Retriever in a query pipeline like this:
```python
from haystack import Document, Pipeline
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack_integrations.components.retrievers.astra import AstraEmbeddingRetriever
from haystack_integrations.document_stores.astra import AstraDocumentStore
document_store = AstraDocumentStore()
model = "sentence-transformers/all-mpnet-base-v2"
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder(model=model)
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings.get("documents"),
policy=DuplicatePolicy.SKIP,
)
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
SentenceTransformersTextEmbedder(model=model),
)
query_pipeline.add_component(
"retriever",
AstraEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
The example output would be:
```python
Document(id=cfe93bc1c274908801e6670440bf2bbba54fad792770d57421f85ffa2a4fcc94, content: 'There are over 7,000 languages spoken around the world today.', score: 0.8929937, embedding: vector of size 768)
```
## Additional References
🧑🍳 Cookbook: [Using AstraDB as a data store in your Haystack pipelines](https://haystack.deepset.ai/cookbook/astradb_haystack_integration)
---
// File: pipeline-components/retrievers/automergingretriever
# AutoMergingRetriever
Use AutoMergingRetriever to improve search results by returning complete parent documents instead of fragmented chunks when multiple related pieces match a query.
2. The last component in the semantic search pipeline
3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of [AstraDocumentStore](../../document-stores/astradocumentstore.mdx) | | **Mandatory run variables** | `query_embedding`: A list of floats | | **Output variables** | `documents`: A list of documents | | **API reference** | [Astra](/reference/integrations-astra) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/astra | | **Package name** | `astra-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Used after the main Retriever component that returns hierarchical documents. |
| **Mandatory init variables** | `document_store`: Document Store from which to retrieve the parent documents |
| **Mandatory run variables** | `documents`: A list of leaf documents that were matched by a Retriever |
| **Output variables** | `documents`: A list resulting documents |
| **API reference** | [Retrievers](/reference/retrievers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/auto_merging_retriever.py |
| **Package name** | `haystack-ai` |
## Overview
The `AutoMergingRetriever` is a component that works with a hierarchical document structure. It returns the parent documents instead of individual leaf documents when a certain threshold is met.
This can be particularly useful when working with paragraphs split into multiple chunks. When several chunks from the same paragraph match your query, the complete paragraph often provides more context and value than the individual pieces alone.
Here is how this Retriever works:
1. It requires documents to be organized in a tree structure, with leaf nodes stored in a document index - see [`HierarchicalDocumentSplitter`](../preprocessors/hierarchicaldocumentsplitter.mdx) documentation.
2. When searching, it counts how many leaf documents under the same parent match your query.
3. If this count exceeds your defined threshold, it returns the parent document instead of the individual leaves.
The `AutoMergingRetriever` can currently be used by the following Document Stores:
- [AstraDocumentStore](../../document-stores/astradocumentstore.mdx)
- [ElasticsearchDocumentStore](../../document-stores/elasticsearch-document-store.mdx)
- [OpenSearchDocumentStore](../../document-stores/opensearch-document-store.mdx)
- [PgvectorDocumentStore](../../document-stores/pgvectordocumentstore.mdx)
- [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx)
## Usage
### On its own
```python
from haystack import Document
from haystack.components.preprocessors import HierarchicalDocumentSplitter
from haystack.components.retrievers.auto_merging_retriever import AutoMergingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
# create a hierarchical document structure with 3 levels, where the parent document has 3 children
text = "The sun rose early in the morning. It cast a warm glow over the trees. Birds began to sing."
original_document = Document(content=text)
builder = HierarchicalDocumentSplitter(block_sizes={10, 3}, split_overlap=0, split_by="word")
docs = builder.run([original_document])["documents"]
# store the root document and the level-1 parent documents, then initialize the retriever
doc_store_parents = InMemoryDocumentStore()
for doc in docs:
if doc.meta["__children_ids"] and doc.meta["__level"] in [0, 1]:
doc_store_parents.write_documents([doc])
retriever = AutoMergingRetriever(doc_store_parents, threshold=0.5)
# assume we retrieved 2 leaf docs from the same parent, the parent document should be returned,
# since it has 3 children and the threshold=0.5, and we retrieved 2 children (2/3 > 0.5)
leaf_docs = [doc for doc in docs if not doc.meta["__children_ids"]]
retrieved_docs = retriever.run(leaf_docs[4:6])
>> retrieved_docs["documents"]
>> [Document(id=bcc..., content: 'warm glow over the trees. Birds began to sing.',
>> meta: {'__block_size': 10, '__parent_id': '835...', '__children_ids': ['a93...', 'c3e...', 'c61...'], '__level': 1,
>> 'source_id': '835...', 'page_number': 1, 'split_id': 1, 'split_idx_start': 45})]
```
### In a pipeline
This is an example of a RAG Haystack pipeline. It first retrieves leaf-level document chunks using BM25, merges them into higher-level parent documents with `AutoMergingRetriever`, constructs a prompt, and generates an answer using OpenAI's chat model.
```python
from typing import List, Tuple
from haystack import Document, Pipeline
from haystack.components.preprocessors import HierarchicalDocumentSplitter
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.retrievers import AutoMergingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.document_stores.types import DuplicatePolicy
from haystack.dataclasses import ChatMessage
def indexing(
documents: List[Document],
) -> Tuple[InMemoryDocumentStore, InMemoryDocumentStore]:
splitter = HierarchicalDocumentSplitter(
block_sizes={10, 3},
split_overlap=0,
split_by="word",
)
docs = splitter.run(documents)
leaf_documents = [doc for doc in docs["documents"] if doc.meta["__level"] == 1]
leaf_doc_store = InMemoryDocumentStore()
leaf_doc_store.write_documents(leaf_documents, policy=DuplicatePolicy.OVERWRITE)
parent_documents = [doc for doc in docs["documents"] if doc.meta["__level"] == 0]
parent_doc_store = InMemoryDocumentStore()
parent_doc_store.write_documents(parent_documents, policy=DuplicatePolicy.OVERWRITE)
return leaf_doc_store, parent_doc_store
# Add documents
docs = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
leaf_docs, parent_docs = indexing(docs)
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given these documents, answer the question.\nDocuments:\n"
"{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
"Question: {{question}}\nAnswer:",
),
]
rag_pipeline = Pipeline()
rag_pipeline.add_component(
instance=InMemoryBM25Retriever(document_store=leaf_docs),
name="bm25_retriever",
)
rag_pipeline.add_component(
instance=AutoMergingRetriever(parent_docs, threshold=0.6),
name="retriever",
)
rag_pipeline.add_component(
instance=ChatPromptBuilder(
template=prompt_template,
required_variables={"question", "documents"},
),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("bm25_retriever.documents", "retriever.documents")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")
question = "How many languages are there?"
result = rag_pipeline.run(
{
"bm25_retriever": {"query": question},
"prompt_builder": {"question": question},
"answer_builder": {"query": question},
},
)
```
---
// File: pipeline-components/retrievers/azureaisearchbm25retriever
# AzureAISearchBM25Retriever
A keyword-based Retriever that fetches Documents matching a query from the Azure AI Search Document Store.
A keyword-based Retriever that fetches documents matching a query from the Azure AI Search Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before an [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of [`AzureAISearchDocumentStore`](../../document-stores/azureaisearchdocumentstore.mdx) |
| **Mandatory run variables** | `query`: A string |
| **Output variables** | `documents`: A list of documents (matching the query) |
| **API reference** | [Azure AI Search](/reference/integrations-azure_ai_search) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/azure_ai_search |
| **Package name** | `azure-ai-search-haystack` |
## Overview
The `AzureAISearchBM25Retriever` is a keyword-based Retriever designed to fetch documents that match a query from an `AzureAISearchDocumentStore`. It uses the BM25 algorithm which calculates a weighted word overlap between the query and the documents to determine their similarity. The Retriever accepts textual query but you can also provide a combination of terms with boolean operators. Some examples of valid queries could be `"pool"`, `"pool spa"`, and `"pool spa +airport"`.
In addition to the `query`, the `AzureAISearchBM25Retriever` accepts other optional parameters, including `top_k` (the maximum number of documents to retrieve) and `filters` to narrow down the search space.
If your search index includes a [semantic configuration](https://learn.microsoft.com/en-us/azure/search/semantic-how-to-query-request), you can enable semantic ranking to apply it to the Retriever's results. For more details, refer to the [Azure AI documentation](https://learn.microsoft.com/en-us/azure/search/hybrid-search-how-to-query#semantic-hybrid-search).
If you want a combination of BM25 and vector retrieval, use the `AzureAISearchHybridRetriever`, which uses both vector search and BM25 search to match documents and query.
## Usage
### Installation
This integration requires you to have an active Azure subscription with a deployed [Azure AI Search](https://azure.microsoft.com/en-us/products/ai-services/ai-search) service.
To start using Azure AI search with Haystack, install the package with:
```shell
pip install azure-ai-search-haystack
```
### On its own
This Retriever needs `AzureAISearchDocumentStore` and indexed documents to run.
```python
from haystack import Document
from haystack_integrations.components.retrievers.azure_ai_search import (
AzureAISearchBM25Retriever,
)
from haystack_integrations.document_stores.azure_ai_search import (
AzureAISearchDocumentStore,
)
document_store = AzureAISearchDocumentStore(index_name="haystack_docs")
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_store.write_documents(documents=documents)
retriever = AzureAISearchBM25Retriever(document_store=document_store)
retriever.run(query="How many languages are spoken around the world today?")
```
### In a RAG pipeline
The below example shows how to use the `AzureAISearchBM25Retriever` in a RAG pipeline. Set your `OPENAI_API_KEY` as an environment variable and then run the following code:
```python
from haystack_integrations.components.retrievers.azure_ai_search import (
AzureAISearchBM25Retriever,
)
from haystack_integrations.document_stores.azure_ai_search import (
AzureAISearchDocumentStore,
)
from haystack import Document
from haystack import Pipeline
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy
import os
api_key = os.environ["OPENAI_API_KEY"]
# Create a RAG query pipeline
prompt_template = [
ChatMessage.from_user(
"""
Given these documents, answer the question.\nDocuments:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
\nQuestion: {{question}}
\nAnswer:
""",
),
]
document_store = AzureAISearchDocumentStore(index_name="haystack-docs")
# Add Documents
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
# policy param is optional, as AzureAISearchDocumentStore has a default policy of DuplicatePolicy.OVERWRITE
document_store.write_documents(documents=documents, policy=DuplicatePolicy.OVERWRITE)
retriever = AzureAISearchBM25Retriever(document_store=document_store)
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="retriever", instance=retriever)
rag_pipeline.add_component(
instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")
question = "Tell me something about languages?"
result = rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
"answer_builder": {"query": question},
},
)
print(result["answer_builder"]["answers"][0])
```
---
// File: pipeline-components/retrievers/azureaisearchembeddingretriever
# AzureAISearchEmbeddingRetriever
An embedding Retriever compatible with the Azure AI Search Document Store.
This Retriever accepts the embeddings of a single query as input and returns a list of matching documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the embedding retrieval pipeline 3. After a Text Embedder and before an [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of [`AzureAISearchDocumentStore`](../../document-stores/azureaisearchdocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A list of floats |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Azure AI Search](/reference/integrations-azure_ai_search) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/azure_ai_search |
| **Package name** | `azure-ai-search-haystack` |
## Overview
The `AzureAISearchEmbeddingRetriever` is an embedding-based Retriever compatible with the `AzureAISearchDocumentStore`. It compares the query and document embeddings and fetches the most relevant documents from the `AzureAISearchDocumentStore` based on the outcome.
The query needs to be embedded before being passed to this component. For example, you could use a Text [Embedder](../embedders.mdx) component.
By default, the `AzureAISearchDocumentStore` uses the [HNSW algorithm](https://learn.microsoft.com/en-us/azure/search/vector-search-overview#nearest-neighbors-search) with cosine similarity to handle vector searches. The vector configuration is set during the initialization of the document store and can be customized by providing the `vector_search_configuration` parameter.
In addition to the `query_embedding`, the `AzureAISearchEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of documents to retrieve) and `filters` to narrow down the search space.
:::info[Semantic Ranking]
The semantic ranking capability of Azure AI Search is not available for vector retrieval. To include semantic ranking in your retrieval process, use the [`AzureAISearchBM25Retriever`](azureaisearchbm25retriever.mdx) or [`AzureAISearchHybridRetriever`](azureaisearchhybridretriever.mdx). For more details, see [Azure AI documentation](https://learn.microsoft.com/en-us/azure/search/semantic-how-to-query-request?tabs=portal-query#set-up-the-query).
:::
## Usage
### Installation
This integration requires you to have an active Azure subscription with a deployed [Azure AI Search](https://azure.microsoft.com/en-us/products/ai-services/ai-search) service.
To start using Azure AI search with Haystack, install the package with:
```shell
pip install azure-ai-search-haystack
```
### On its own
This Retriever needs `AzureAISearchDocumentStore` and indexed documents to run.
```python
from haystack_integrations.document_stores.azure_ai_search import (
AzureAISearchDocumentStore,
)
from haystack_integrations.components.retrievers.azure_ai_search import (
AzureAISearchEmbeddingRetriever,
)
document_store = AzureAISearchDocumentStore()
retriever = AzureAISearchEmbeddingRetriever(document_store=document_store)
# example run query
retriever.run(query_embedding=[0.1] * 384)
```
### In a pipeline
Here is how you could use the `AzureAISearchEmbeddingRetriever` in a pipeline. In this example, you would create two pipelines: an indexing one and a querying one.
In the indexing pipeline, the documents are passed to the Document Embedder and then written into the Document Store.
Then, in the querying pipeline, we use a Text Embedder to get the vector representation of the input query that will be then passed to the `AzureAISearchEmbeddingRetriever` to get the results.
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document, Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.retrievers.azure_ai_search import (
AzureAISearchEmbeddingRetriever,
)
from haystack_integrations.document_stores.azure_ai_search import (
AzureAISearchDocumentStore,
)
document_store = AzureAISearchDocumentStore(index_name="retrieval-example")
model = "sentence-transformers/all-mpnet-base-v2"
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="""Elephants have been observed to behave in a way that indicates a
high level of self-awareness, such as recognizing themselves in mirrors.""",
),
Document(
content="""In certain parts of the world, like the Maldives, Puerto Rico, and
San Diego, you can witness the phenomenon of bioluminescent waves.""",
),
]
document_embedder = SentenceTransformersDocumentEmbedder(model=model)
# Indexing Pipeline
indexing_pipeline = Pipeline()
indexing_pipeline.add_component(instance=document_embedder, name="doc_embedder")
indexing_pipeline.add_component(
instance=DocumentWriter(document_store=document_store),
name="doc_writer",
)
indexing_pipeline.connect("doc_embedder", "doc_writer")
indexing_pipeline.run({"doc_embedder": {"documents": documents}})
# Query Pipeline
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
SentenceTransformersTextEmbedder(model=model),
)
query_pipeline.add_component(
"retriever",
AzureAISearchEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/azureaisearchhybridretriever
# AzureAISearchHybridRetriever
A Retriever based both on dense and sparse embeddings, compatible with the Azure AI Search Document Store.
This Retriever combines embedding-based retrieval and BM25 text search search to find matching documents in the search index to get more relevant results.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a TextEmbedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in a hybrid search pipeline 3. After a TextEmbedder and before an [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of [`AzureAISearchDocumentStore`](../../document-stores/azureaisearchdocumentstore.mdx) |
| **Mandatory run variables** | `query`: A string
`query_embedding`: A list of floats | | **Output variables** | `documents`: A list of documents (matching the query) | | **API reference** | [Azure AI Search](/reference/integrations-azure_ai_search) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/azure_ai_search | | **Package name** | `azure-ai-search-haystack` |
## Overview
The `AzureAISearchHybridRetriever` combines vector retrieval and BM25 text search to fetch relevant documents from the `AzureAISearchDocumentStore`. It processes both textual (keyword) queries and query embeddings in a single request, executing all subqueries in parallel. The results are merged and reordered using [Reciprocal Rank Fusion (RRF)](https://learn.microsoft.com/en-us/azure/search/hybrid-search-ranking) to create a unified result set.
Besides the `query` and `query_embedding`, the `AzureAISearchHybridRetriever` accepts optional parameters such as `top_k` (the maximum number of documents to retrieve) and `filters` to refine the search. Additional keyword arguments can also be passed during initialization for further customization.
If your search index includes a [semantic configuration](https://learn.microsoft.com/en-us/azure/search/semantic-how-to-query-request), you can enable semantic ranking to apply it to the Retriever's results. For more details, refer to the [Azure AI documentation](https://learn.microsoft.com/en-us/azure/search/hybrid-search-how-to-query#semantic-hybrid-search).
For purely keyword-based retrieval, you can use `AzureAISearchBM25Retriever`, and for embedding-based retrieval, `AzureAISearchEmbeddingRetriever` is available.
## Usage
### Installation
This integration requires you to have an active Azure subscription with a deployed [Azure AI Search](https://azure.microsoft.com/en-us/products/ai-services/ai-search) service.
To start using Azure AI search with Haystack, install the package with:
```shell
pip install azure-ai-search-haystack
```
### On its own
This Retriever needs `AzureAISearchDocumentStore` and indexed documents to run.
```python
from haystack import Document
from haystack_integrations.components.retrievers.azure_ai_search import (
AzureAISearchHybridRetriever,
)
from haystack_integrations.document_stores.azure_ai_search import (
AzureAISearchDocumentStore,
)
document_store = AzureAISearchDocumentStore(index_name="haystack_docs")
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_store.write_documents(documents=documents)
retriever = AzureAISearchHybridRetriever(document_store=document_store)
# fake embeddings to keep the example simple
retriever.run(
query="How many languages are spoken around the world today?",
query_embedding=[0.1] * 384,
)
```
### In a RAG pipeline
The following example demonstrates using the `AzureAISearchHybridRetriever` in a pipeline. An indexing pipeline is responsible for indexing and storing documents with embeddings in the `AzureAISearchDocumentStore`, while the query pipeline uses hybrid retrieval to fetch relevant documents based on a given query.
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document, Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.retrievers.azure_ai_search import (
AzureAISearchHybridRetriever,
)
from haystack_integrations.document_stores.azure_ai_search import (
AzureAISearchDocumentStore,
)
document_store = AzureAISearchDocumentStore(index_name="hybrid-retrieval-example")
model = "sentence-transformers/all-mpnet-base-v2"
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="""Elephants have been observed to behave in a way that indicates a
high level of self-awareness, such as recognizing themselves in mirrors.""",
),
Document(
content="""In certain parts of the world, like the Maldives, Puerto Rico, and
San Diego, you can witness the phenomenon of bioluminescent waves.""",
),
]
document_embedder = SentenceTransformersDocumentEmbedder(model=model)
# Indexing Pipeline
indexing_pipeline = Pipeline()
indexing_pipeline.add_component(instance=document_embedder, name="doc_embedder")
indexing_pipeline.add_component(
instance=DocumentWriter(document_store=document_store),
name="doc_writer",
)
indexing_pipeline.connect("doc_embedder", "doc_writer")
indexing_pipeline.run({"doc_embedder": {"documents": documents}})
# Query Pipeline
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
SentenceTransformersTextEmbedder(model=model),
)
query_pipeline.add_component(
"retriever",
AzureAISearchHybridRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run(
{"text_embedder": {"text": query}, "retriever": {"query": query}},
)
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/chromaembeddingretriever
# ChromaEmbeddingRetriever
This is an embedding Retriever compatible with the Chroma Document Store.
`query_embedding`: A list of floats | | **Output variables** | `documents`: A list of documents (matching the query) | | **API reference** | [Azure AI Search](/reference/integrations-azure_ai_search) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/azure_ai_search | | **Package name** | `azure-ai-search-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [ChromaDocumentStore](../../document-stores/chromadocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A list of floats |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Chroma](/reference/integrations-chroma) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/chroma |
| **Package name** | `chroma-haystack` |
## Overview
The `ChromaEmbeddingRetriever` is an embedding-based Retriever compatible with the `ChromaDocumentStore`. It compares the query and document embeddings and fetches the documents most relevant to the query from the `ChromaDocumentStore` based on the outcome.
The query needs to be embedded before being passed to this component. For example, you could use a text [embedder](../embedders.mdx) component.
In addition to the `query_embedding`, the `ChromaEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of documents to retrieve) and `filters` to narrow down the search space.
### Usage
#### On its own
This Retriever needs the `ChromaDocumentStore` and indexed documents to run.
```python
from haystack_integrations.document_stores.chroma import ChromaDocumentStore
from haystack_integrations.components.retrievers.chroma import ChromaEmbeddingRetriever
document_store = ChromaDocumentStore()
retriever = ChromaEmbeddingRetriever(document_store=document_store)
# example run query
retriever.run(query_embedding=[0.1] * 384)
```
#### In a pipeline
Here is how you could use the `ChromaEmbeddingRetriever` in a pipeline. In this example, you would create two pipelines: an indexing one and a querying one.
In the indexing pipeline, the documents are passed to the Document Embedder and then written into the document Store.
Then, in the querying pipeline, we use a text embedder to get the vector representation of the input query that will be then passed to the `ChromaEmbeddingRetriever` to get the results.
```python
import os
from pathlib import Path
from haystack import Pipeline
from haystack.dataclasses import Document
from haystack.components.writers import DocumentWriter
# Note: the following requires a "pip install sentence-transformers-haystack"
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack_integrations.document_stores.chroma import ChromaDocumentStore
from haystack_integrations.components.retrievers.chroma import ChromaEmbeddingRetriever
from sentence_transformers import SentenceTransformer
# Chroma is used in-memory so we use the same instances in the two pipelines below
document_store = ChromaDocumentStore()
documents = [
Document(content="This contains variable declarations", meta={"title": "one"}),
Document(
content="This contains another sort of variable declarations",
meta={"title": "two"},
),
Document(
content="This has nothing to do with variable declarations",
meta={"title": "three"},
),
Document(content="A random doc", meta={"title": "four"}),
]
indexing = Pipeline()
indexing.add_component("embedder", SentenceTransformersDocumentEmbedder())
indexing.add_component("writer", DocumentWriter(document_store))
indexing.connect("embedder.documents", "writer.documents")
indexing.run({"embedder": {"documents": documents}})
querying = Pipeline()
querying.add_component("query_embedder", SentenceTransformersTextEmbedder())
querying.add_component("retriever", ChromaEmbeddingRetriever(document_store))
querying.connect("query_embedder.embedding", "retriever.query_embedding")
results = querying.run({"query_embedder": {"text": "Variable declarations"}})
for d in results["retriever"]["documents"]:
print(d.meta, d.score)
```
## Additional References
🧑🍳 Cookbook: [Use Chroma for RAG and Indexing](https://haystack.deepset.ai/cookbook/chroma-indexing-and-rag-examples)
---
// File: pipeline-components/retrievers/chromaqueryretriever
# ChromaQueryTextRetriever
This is a a Retriever compatible with the Chroma Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [ChromaDocumentStore](../../document-stores/chromadocumentstore.mdx) |
| **Mandatory run variables** | `query`: A single query in plain-text format to be processed by the [Retriever](../retrievers.mdx) |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Chroma](/reference/integrations-chroma) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/chroma |
| **Package name** | `chroma-haystack` |
## Overview
The `ChromaQueryTextRetriever` is an embedding-based Retriever compatible with the `ChromaDocumentStore` that uses the Chroma [query API](https://docs.trychroma.com/reference/python/collection#query).
This component takes a plain-text query string in input and returns the matching documents.
Chroma will create the embedding for the query using its [embedding function](https://docs.trychroma.com/docs/embeddings/embedding-functions); in case you do not want to use the default embedding function, this must be specified at `ChromaDocumentStore` initialization.
### Usage
#### On its own
This Retriever needs the `ChromaDocumentStore` and indexed documents to run.
```python
from haystack_integrations.document_stores.chroma import ChromaDocumentStore
from haystack_integrations.components.retrievers.chroma import ChromaQueryTextRetriever
document_store = ChromaDocumentStore()
retriever = ChromaQueryTextRetriever(document_store=document_store)
# example run query
retriever.run(query="How does Chroma Retriever work?")
```
#### In a pipeline
Here is how you could use the `ChromaQueryTextRetriever` in a Pipeline. In this example, you would create two pipelines: an indexing one and a querying one.
In the indexing pipeline, the documents are written in the Document Store.
Then, in the querying pipeline, `ChromaQueryTextRetriever` gets the answer from the Document Store based on the provided query.
```python
import os
from pathlib import Path
from haystack import Pipeline
from haystack.dataclasses import Document
from haystack.components.writers import DocumentWriter
from haystack_integrations.document_stores.chroma import ChromaDocumentStore
from haystack_integrations.components.retrievers.chroma import ChromaQueryTextRetriever
# Chroma is used in-memory so we use the same instances in the two pipelines below
document_store = ChromaDocumentStore()
documents = [
Document(content="This contains variable declarations", meta={"title": "one"}),
Document(
content="This contains another sort of variable declarations",
meta={"title": "two"},
),
Document(
content="This has nothing to do with variable declarations",
meta={"title": "three"},
),
Document(content="A random doc", meta={"title": "four"}),
]
indexing = Pipeline()
indexing.add_component("writer", DocumentWriter(document_store))
indexing.run({"writer": {"documents": documents}})
querying = Pipeline()
querying.add_component("retriever", ChromaQueryTextRetriever(document_store))
results = querying.run({"retriever": {"query": "Variable declarations", "top_k": 3}})
for d in results["retriever"]["documents"]:
print(d.meta, d.score)
```
## Additional References
🧑🍳 Cookbook: [Use Chroma for RAG and Indexing](https://haystack.deepset.ai/cookbook/chroma-indexing-and-rag-examples)
---
// File: pipeline-components/retrievers/cogneeretriever
# CogneeRetriever
Retrieves memories from a `CogneeMemoryStore` and returns them as system `ChatMessage` objects.
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an [`Agent`](../agents-1/agent.mdx) or Chat Generator in memory-augmented pipelines |
| **Mandatory init variables** | `memory_store`: A `CogneeMemoryStore` instance |
| **Optional init variables** | `top_k`: Maximum number of memories to return (defaults to the store's `top_k`) |
| **Mandatory run variables** | `query`: A text query to search memories |
| **Optional run variables** | `user_id`: Cognee user ID to scope the retrieval; pass `None` to use Cognee's default user |
| **Output variables** | `messages`: A list of system `ChatMessage` objects |
| **API reference** | [Cognee](/reference/integrations-cognee#cogneeretriever) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cognee |
| **Package name** | `cognee-haystack` |
## Overview
`CogneeRetriever` retrieves memories from a `CogneeMemoryStore` and returns them as system `ChatMessage` objects.
Use it to inject long-term memory into an Agent or a chat generation pipeline before the model produces a response.
Search behavior — including the search strategy (`search_type`), dataset, and session tier — is configured on the `CogneeMemoryStore`. The retriever is a thin pipeline adapter over `search_memories`.
The `user_id` parameter scopes the retrieval to a specific Cognee user. Pass `None` to use Cognee's default user.
## Installation
Install the Cognee integration:
```bash
pip install cognee-haystack
```
Set your LLM API key (used by Cognee for graph extraction and queries):
```bash
export LLM_API_KEY="your-llm-api-key"
```
Optionally, set a separate embedding API key (defaults to `LLM_API_KEY` when unset):
```bash
export EMBEDDING_API_KEY="your-embedding-api-key"
```
## Usage
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.retrievers.cognee import CogneeRetriever
from haystack_integrations.memory_stores.cognee import CogneeMemoryStore
store = CogneeMemoryStore(search_type="GRAPH_COMPLETION", top_k=5)
# Write some memories first
store.add_memories(
messages=[ChatMessage.from_user("Alice prefers concise Python examples.")],
user_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890",
)
retriever = CogneeRetriever(memory_store=store, top_k=3)
result = retriever.run(
query="What does Alice prefer?",
user_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890",
)
memories = result["messages"]
print([message.text for message in memories])
```
### In a Pipeline
This example retrieves memories, prepends them to the current user message, and passes the combined message list to an Agent.
```python
from haystack import Pipeline
from haystack.components.agents import Agent
from haystack.components.converters import OutputAdapter
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.retrievers.cognee import CogneeRetriever
from haystack_integrations.memory_stores.cognee import CogneeMemoryStore
store = CogneeMemoryStore(dataset_name="my_agent_memory", session_id="alice_session_1")
pipeline = Pipeline()
pipeline.add_component("retriever", CogneeRetriever(memory_store=store, top_k=5))
pipeline.add_component(
"memory_context",
OutputAdapter(
template="{{ memories + user_messages }}",
output_type=list[ChatMessage],
unsafe=True,
),
)
pipeline.add_component(
"agent",
Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
system_prompt=(
"Use any system messages at the start of the conversation as long-term memory. "
"Answer concisely."
),
),
)
pipeline.connect("retriever.messages", "memory_context.memories")
pipeline.connect("memory_context.output", "agent.messages")
query = "Give me a short implementation tip."
pipeline.run(
{
"retriever": {
"query": query,
"user_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
},
"memory_context": {
"user_messages": [ChatMessage.from_user(query)],
},
}
)
```
---
// File: pipeline-components/retrievers/elasticsearchbm25retriever
# ElasticsearchBM25Retriever
A keyword-based Retriever that fetches Documents matching a query from the Elasticsearch Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of [ElasticsearchDocumentStore](../../document-stores/elasticsearch-document-store.mdx) |
| **Mandatory run variables** | `query`: A string |
| **Output variables** | `documents`: A list of documents (matching the query) |
| **API reference** | [Elasticsearch](/reference/integrations-elasticsearch) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch |
| **Package name** | `elasticsearch-haystack` |
## Overview
`ElasticsearchBM25Retriever` is a keyword-based Retriever that fetches Documents matching a query from an `ElasticsearchDocumentStore`. It determines the similarity between Documents and the query based on the BM25 algorithm, which computes a weighted word overlap between the two strings.
Since the `ElasticsearchBM25Retriever` matches strings based on word overlap, it’s often used to find exact matches to names of persons or products, IDs, or well-defined error messages. The BM25 algorithm is very lightweight and simple. Nevertheless, it can be hard to beat with more complex embedding-based approaches on out-of-domain data.
In addition to the `query`, the `ElasticsearchBM25Retriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
When initializing Retriever, you can also adjust how [inexact fuzzy matching](https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#fuzziness) is performed, using the `fuzziness` parameter.
If you want a semantic match between a query and documents, you can use `ElasticsearchEmbeddingRetriever`, which uses vectors created by embedding models to retrieve relevant information.
## Installation
[Install](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html) Elasticsearch and then [start](https://www.elastic.co/guide/en/elasticsearch/reference/current/starting-elasticsearch.html) an instance. Haystack supports Elasticsearch 8.
If you have Docker set up, we recommend pulling the Docker image and running it.
```shell
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.19.7
docker run -p 9200:9200 -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" -e "xpack.security.enabled=false" elasticsearch:8.19.7
```
As an alternative, you can go to [Elasticsearch integration GitHub](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch) and start a Docker container running Elasticsearch using the provided `docker-compose.yml`:
```shell
docker compose up
```
Once you have a running Elasticsearch instance, install the `elasticsearch-haystack` integration:
```shell
pip install elasticsearch-haystack
```
## Usage
### On its own
```python
from haystack import Document
from haystack_integrations.components.retrievers.elasticsearch import (
ElasticsearchBM25Retriever,
)
from haystack_integrations.document_stores.elasticsearch import (
ElasticsearchDocumentStore,
)
from elasticsearch import Elasticsearch
document_store = ElasticsearchDocumentStore(hosts="http://localhost:9200/")
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_store.write_documents(documents=documents)
retriever = ElasticsearchBM25Retriever(document_store=document_store)
retriever.run(query="How many languages are spoken around the world today?")
```
### In a RAG pipeline
Set your `OPENAI_API_KEY` as an environment variable and then run the following code:
```python
from haystack_integrations.components.retrievers.elasticsearch import (
ElasticsearchBM25Retriever,
)
from haystack_integrations.document_stores.elasticsearch import (
ElasticsearchDocumentStore,
)
from elasticsearch import Elasticsearch
from haystack import Document
from haystack import Pipeline
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy
# OpenAIChatGenerator reads the OPENAI_API_KEY environment variable by default.
# Create a RAG query pipeline
prompt_template = [
ChatMessage.from_user(
"""
Given these documents, answer the question.\nDocuments:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
\nQuestion: {{question}}
\nAnswer:
""",
),
]
document_store = ElasticsearchDocumentStore(hosts="http://localhost:9200/")
# Add Documents
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
# DuplicatePolicy.SKIP param is optional, but useful to run the script multiple times without throwing errors
document_store.write_documents(documents=documents, policy=DuplicatePolicy.SKIP)
retriever = ElasticsearchBM25Retriever(document_store=document_store)
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="retriever", instance=retriever)
rag_pipeline.add_component(
instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")
question = "How many languages are spoken around the world today?"
result = rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
"answer_builder": {"query": question},
},
)
print(result["answer_builder"]["answers"][0].data)
```
Here’s an example output you might get:
```python
"Over 7,000 languages are spoken around the world today"
```
---
// File: pipeline-components/retrievers/elasticsearchembeddingretriever
# ElasticsearchEmbeddingRetriever
An embedding-based Retriever compatible with the Elasticsearch Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of [ElasticsearchDocumentStore](../../document-stores/elasticsearch-document-store.mdx) |
| **Mandatory run variables** | `query_embedding`: A list of floats |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Elasticsearch](/reference/integrations-elasticsearch) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch |
| **Package name** | `elasticsearch-haystack` |
## Overview
The `ElasticsearchEmbeddingRetriever` is an embedding-based Retriever compatible with the `ElasticsearchDocumentStore`. It compares the query and Document embeddings and fetches the Documents most relevant to the query from the `ElasticsearchDocumentStore` based on the outcome.
When using the `ElasticsearchEmbeddingRetriever` in your NLP system, ensure it has the query and Document embeddings available. You can do so by adding a Document Embedder to your indexing pipeline and a Text Embedder to your query pipeline.
In addition to the `query_embedding`, the `ElasticsearchEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
When initializing Retriever, you can also set `num_candidates`: the number of approximate nearest neighbor candidates on each shard. It's an advanced setting you can read more about in the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html#tune-approximate-knn-for-speed-accuracy).
The `embedding_similarity_function` to use for embedding retrieval must be defined when the corresponding `ElasticsearchDocumentStore` is initialized.
## Installation
[Install](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html) Elasticsearch and then [start](https://www.elastic.co/guide/en/elasticsearch/reference/current/starting-elasticsearch.html) an instance. Haystack supports Elasticsearch 8.
If you have Docker set up, we recommend pulling the Docker image and running it.
```shell
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.19.7
docker run -p 9200:9200 -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" -e "xpack.security.enabled=false" elasticsearch:8.19.7
```
As an alternative, you can go to [Elasticsearch integration GitHub](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch) and start a Docker container running Elasticsearch using the provided `docker-compose.yml`:
```shell
docker compose up
```
Once you have a running Elasticsearch instance, install the `elasticsearch-haystack` integration:
```shell
pip install elasticsearch-haystack
```
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
### In a pipeline
Use this Retriever in a query Pipeline like this:
```python
from haystack_integrations.components.retrievers.elasticsearch import (
ElasticsearchEmbeddingRetriever,
)
from haystack_integrations.document_stores.elasticsearch import (
ElasticsearchDocumentStore,
)
from haystack.document_stores.types import DuplicatePolicy
from haystack import Document, Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
document_store = ElasticsearchDocumentStore(hosts="http://localhost:9200/")
model = "BAAI/bge-large-en-v1.5"
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder(model=model)
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings.get("documents"),
policy=DuplicatePolicy.SKIP,
)
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
SentenceTransformersTextEmbedder(model=model),
)
query_pipeline.add_component(
"retriever",
ElasticsearchEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
The example output would be:
```python
Document(id=cfe93bc1c274908801e6670440bf2bbba54fad792770d57421f85ffa2a4fcc94, content: 'There are over 7,000 languages spoken around the world today.', score: 0.87717235, embedding: vector of size 1024)
```
---
// File: pipeline-components/retrievers/elasticsearchhybridretriever
# ElasticsearchHybridRetriever
This is a [SuperComponent](../../concepts/components/supercomponents.mdx) that implements a Hybrid Retriever in a single component, relying on Elasticsearch as the backend Document Store.
A Hybrid Retriever uses both traditional keyword-based search (BM25) and embedding-based search to retrieve documents, combining the strengths of both approaches. The Retriever then merges and re-ranks the results from both methods.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a TextEmbedder and before a PromptBuilder in a RAG pipeline 2. The last component in a hybrid search pipeline 3. After a TextEmbedder and before a TransformersExtractiveReader in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of [`ElasticsearchDocumentStore`](../../document-stores/elasticsearch-document-store.mdx)
`embedder`: Any [Embedder](../embedders.mdx) implementing the `TextEmbedder` protocol | | **Mandatory run variables** | `query`: A query string | | **Output variables** | `documents`: A list of documents matching the query | | **API reference** | [Elasticsearch](/reference/integrations-elasticsearch) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch | | **Package name** | `elasticsearch-haystack` |
## Overview
The `ElasticsearchHybridRetriever` combines two retrieval methods:
1. **BM25 Retrieval**: A keyword-based search that uses the BM25 algorithm to find documents based on term frequency and inverse document frequency. It's based on the [`ElasticsearchBM25Retriever`](elasticsearchbm25retriever.mdx) component and is suitable for finding exact matches to names, IDs, or well-defined terms.
2. **Embedding-based Retrieval**: A semantic search that uses vector similarity to find documents that are semantically similar to the query. It's based on the [`ElasticsearchEmbeddingRetriever`](elasticsearchembeddingretriever.mdx) component and is suitable for semantic search.
The component automatically handles:
- Converting the query into an embedding using the provided embedder,
- Running both retrieval methods in parallel,
- Merging and re-ranking the results using the specified join mode (default: Reciprocal Rank Fusion).
### Installation
[Install](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html) Elasticsearch and then [start](https://www.elastic.co/guide/en/elasticsearch/reference/current/starting-elasticsearch.html) an instance. Haystack supports Elasticsearch 8.
If you have Docker set up, we recommend pulling the Docker image and running it.
```shell
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.19.7
docker run -p 9200:9200 -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" -e "xpack.security.enabled=false" elasticsearch:8.19.7
```
As an alternative, you can go to the [Elasticsearch integration GitHub](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch) and start a Docker container running Elasticsearch using the provided `docker-compose.yml`:
```shell
docker compose up
```
Once you have a running Elasticsearch instance, install the `elasticsearch-haystack` integration:
```shell
pip install elasticsearch-haystack
```
### Optional Parameters
This Retriever accepts various optional parameters. You can verify the most up-to-date list of parameters in our [API Reference](/reference/integrations-elasticsearch).
You can pass additional parameters to the underlying BM25 and embedding retriever components using the `top_k_bm25`, `fuzziness`, `filters_bm25`, `scale_score`, `filter_policy_bm25`, `top_k_embedding`, `filters_embedding`, `num_candidates`, and `filter_policy_embedding` parameters.
The `DocumentJoiner` parameters (`join_mode`, `weights`, `top_k`, and `sort_by_score`) are all exposed directly on the `ElasticsearchHybridRetriever` class.
## Usage
### On its own
This Retriever needs the `ElasticsearchDocumentStore` populated with documents (including embeddings) to run.
```python
from haystack import Document
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack_integrations.components.retrievers.elasticsearch import (
ElasticsearchHybridRetriever,
)
from haystack_integrations.document_stores.elasticsearch import (
ElasticsearchDocumentStore,
)
document_store = ElasticsearchDocumentStore(hosts="http://localhost:9200/")
model = "sentence-transformers/all-MiniLM-L6-v2"
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
doc_embedder = SentenceTransformersDocumentEmbedder(model=model)
docs_with_embeddings = doc_embedder.run(documents)
document_store.write_documents(docs_with_embeddings["documents"])
embedder = SentenceTransformersTextEmbedder(model=model)
retriever = ElasticsearchHybridRetriever(
document_store=document_store,
embedder=embedder,
)
results = retriever.run(query="How many languages are spoken around the world today?")
print(results["documents"])
```
### In a pipeline
Here's a full example that uses an indexing pipeline to store documents with embeddings, and a query pipeline that uses `ElasticsearchHybridRetriever` for hybrid retrieval.
Set your `OPENAI_API_KEY` as an environment variable and then run the following code:
```python
from haystack import Document, Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.writers import DocumentWriter
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.retrievers.elasticsearch import (
ElasticsearchHybridRetriever,
)
from haystack_integrations.document_stores.elasticsearch import (
ElasticsearchDocumentStore,
)
document_store = ElasticsearchDocumentStore(hosts="http://localhost:9200/")
model = "sentence-transformers/all-MiniLM-L6-v2"
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
# Indexing Pipeline
indexing_pipeline = Pipeline()
indexing_pipeline.add_component(
"doc_embedder",
SentenceTransformersDocumentEmbedder(model=model),
)
indexing_pipeline.add_component(
"doc_writer",
DocumentWriter(document_store=document_store, policy=DuplicatePolicy.SKIP),
)
indexing_pipeline.connect("doc_embedder", "doc_writer")
indexing_pipeline.run({"doc_embedder": {"documents": documents}})
# Query Pipeline
prompt_template = [
ChatMessage.from_user(
"""
Given these documents, answer the question.\nDocuments:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
\nQuestion: {{question}}
\nAnswer:
""",
),
]
embedder = SentenceTransformersTextEmbedder(model=model)
retriever = ElasticsearchHybridRetriever(
document_store=document_store,
embedder=embedder,
top_k_bm25=3,
top_k_embedding=3,
join_mode="reciprocal_rank_fusion",
)
query_pipeline = Pipeline()
query_pipeline.add_component("retriever", retriever)
query_pipeline.add_component(
"prompt_builder",
ChatPromptBuilder(template=prompt_template, required_variables="*"),
)
query_pipeline.add_component("llm", OpenAIChatGenerator())
query_pipeline.connect("retriever.documents", "prompt_builder.documents")
query_pipeline.connect("prompt_builder.prompt", "llm.messages")
question = "How many languages are spoken around the world today?"
result = query_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
},
)
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/retrievers/elasticsearchsqlretriever
# ElasticsearchSQLRetriever
Executes raw Elasticsearch SQL queries against an Elasticsearch Document Store and returns the raw JSON response.
| | |
| --------------------------------------- | ------------------------------------------------------------------------------------------------ |
| **Most common position in a pipeline** | Standalone, or anywhere you need to fetch metadata, aggregations, or other structured data |
| **Mandatory init variables** | `document_store`: An instance of `ElasticsearchDocumentStore` |
| **Mandatory run variables** | `query`: An Elasticsearch SQL query string |
| **Output variables** | `result`: A dictionary with the raw JSON response from the Elasticsearch SQL API |
| **API reference** | [Elasticsearch](https://docs.haystack.deepset.ai/reference/integrations-elasticsearch) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch |
| **Package name** | `elasticsearch-haystack` |
## Overview
`ElasticsearchSQLRetriever` lets you run [Elasticsearch SQL](https://www.elastic.co/guide/en/elasticsearch/reference/current/xpack-sql.html) queries directly against an `ElasticsearchDocumentStore`. Instead of matching a query against documents like the `ElasticsearchBM25Retriever` or `ElasticsearchEmbeddingRetriever`, it executes a SQL statement and returns the **raw JSON response** from the Elasticsearch SQL API.
This is useful when you need structured access to your index at runtime, for example to fetch specific fields, filter on metadata, or compute aggregations such as counts and averages.
Unlike the other Elasticsearch retrievers, this component does not return a list of `Document` objects. The output is a single `result` dictionary, where `result["result"]` holds the raw Elasticsearch response. For a typical query, the response contains:
- `result["result"]["columns"]`: metadata describing each returned column.
- `result["result"]["rows"]`: the data rows.
The component accepts two optional parameters at initialization:
- `raise_on_failure`: if `True` (the default), an exception is raised when the SQL API call fails. If `False`, the error is logged as a warning and an empty dictionary is returned.
- `fetch_size`: the number of results to fetch per page. If not set, the default fetch size configured in Elasticsearch is used.
## Installation
Install Elasticsearch and then start an instance. Haystack supports Elasticsearch 8.
If you have Docker set up, we recommend pulling the Docker image and running it.
```bash
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.19.7
docker run -p 9200:9200 -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" -e "xpack.security.enabled=false" elasticsearch:8.19.7
```
As an alternative, you can go to [Elasticsearch integration GitHub](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch) and start a Docker container running Elasticsearch using the provided `docker-compose.yml`:
```bash
docker compose up
```
Once you have a running Elasticsearch instance, install the `elasticsearch-haystack` integration:
```bash
pip install elasticsearch-haystack
```
## Usage
### On its own
Write a few documents to an index, then run a SQL query against it. The example below selects the `content` field from the index and reads the returned columns and rows:
```python
from haystack import Document
from haystack_integrations.components.retrievers.elasticsearch import (
ElasticsearchSQLRetriever,
)
from haystack_integrations.document_stores.elasticsearch import (
ElasticsearchDocumentStore,
)
from haystack.document_stores.types import DuplicatePolicy
document_store = ElasticsearchDocumentStore(
hosts="http://localhost:9200/", index="my_index"
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
# DuplicatePolicy.SKIP is optional, but useful to run the script multiple times without throwing errors
document_store.write_documents(documents=documents, policy=DuplicatePolicy.SKIP)
retriever = ElasticsearchSQLRetriever(document_store=document_store)
output = retriever.run(query='SELECT content FROM "my_index" LIMIT 10')
result = output["result"]
print(result["columns"]) # column metadata, e.g. [{"name": "content", "type": "text"}]
for row in result["rows"]:
print(row)
```
### Running an aggregation query
Because the component returns the raw SQL response, you can use it for aggregations that the document-based retrievers don't support, such as counting documents:
```python
retriever = ElasticsearchSQLRetriever(document_store=document_store)
output = retriever.run(query='SELECT COUNT(*) AS doc_count FROM "my_index"')
result = output["result"]
print(result["rows"]) # e.g. [[3]]
```
To avoid raising an exception on a malformed or failing query, initialize the component with `raise_on_failure=False`. In that case, a failed query logs a warning and returns an empty dictionary instead.
---
// File: pipeline-components/retrievers/faissembeddingretriever
# FAISSEmbeddingRetriever
An embedding-based Retriever compatible with the FAISSDocumentStore.
`embedder`: Any [Embedder](../embedders.mdx) implementing the `TextEmbedder` protocol | | **Mandatory run variables** | `query`: A query string | | **Output variables** | `documents`: A list of documents matching the query | | **API reference** | [Elasticsearch](/reference/integrations-elasticsearch) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch | | **Package name** | `elasticsearch-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in a semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [`FAISSDocumentStore`](../../document-stores/faissdocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [FAISS](/reference/integrations-faiss) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/faiss |
| **Package name** | `faiss-haystack` |
## Overview
The `FAISSEmbeddingRetriever` is an embedding-based Retriever that queries a `FAISSDocumentStore`. It compares the query embedding to document embeddings stored in FAISS and returns the most similar documents.
This Retriever expects precomputed embeddings in the Document Store and a query embedding at runtime. You can generate them with a Document Embedder in your indexing pipeline and a Text Embedder in your query pipeline.
In addition to `query_embedding`, you can pass:
- `top_k`: The maximum number of documents to return.
- `filters`: Metadata filters to restrict retrieved documents.
You can also configure default filters and `filter_policy` at initialization.
## Usage
### On its own
```python
from haystack_integrations.document_stores.faiss import FAISSDocumentStore
from haystack_integrations.components.retrievers.faiss import FAISSEmbeddingRetriever
document_store = FAISSDocumentStore(embedding_dim=768)
retriever = FAISSEmbeddingRetriever(document_store=document_store, top_k=5)
# Example query embedding
result = retriever.run(query_embedding=[0.1] * 768)
print(result["documents"])
```
### In a pipeline
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document, Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.document_stores.faiss import FAISSDocumentStore
from haystack_integrations.components.retrievers.faiss import FAISSEmbeddingRetriever
document_store = FAISSDocumentStore(embedding_dim=768)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of intelligence.",
),
Document(
content="In certain places, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(
documents_with_embeddings,
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
FAISSEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/falkordbcypherretriever
# FalkorDBCypherRetriever
A Retriever that executes arbitrary OpenCypher queries against a FalkorDB Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a query-building component and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a GraphRAG pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [FalkorDBDocumentStore](../../document-stores/falkordbdocumentstore.mdx) |
| **Mandatory run variables** | `query`: An OpenCypher query string (or set `custom_cypher_query` at init) |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [FalkorDB](/reference/integrations-falkordb) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/falkordb |
| **Package name** | `falkordb-haystack` |
## Overview
The `FalkorDBCypherRetriever` executes arbitrary OpenCypher queries against a `FalkorDBDocumentStore`, making it suitable for graph traversal and multi-hop queries in GraphRAG pipelines. The query must return nodes or dictionaries that map to Haystack `Document` fields.
A `custom_cypher_query` can be set at initialization and optionally overridden at runtime by passing `query` to `run()`. Use parameterized queries (`$param_name` in Cypher, passed via `parameters`) rather than string interpolation to avoid injection vulnerabilities.
:::warning[Security]
Raw Cypher queries must only come from trusted sources. Never pass unsanitized user input directly in query strings. Use `parameters` instead.
:::
## Installation
```shell
pip install falkordb-haystack
```
Ensure FalkorDB is running, for example via Docker:
```shell
docker run -d -p 6379:6379 falkordb/falkordb:latest
```
The examples on this page use Transformers components from the `transformers-haystack` package. Install it to run the examples:
```shell
pip install transformers-haystack
```
## Usage
### On its own
```python
from haystack import Document
from haystack_integrations.document_stores.falkordb import FalkorDBDocumentStore
from haystack_integrations.components.retrievers.falkordb import FalkorDBCypherRetriever
document_store = FalkorDBDocumentStore(
host="localhost",
port=6379,
recreate_graph=True,
)
document_store.write_documents(
[
Document(
content="There are over 7,000 languages spoken around the world today.",
meta={"topic": "linguistics"},
),
Document(
content="Elephants have been observed to recognize themselves in mirrors.",
meta={"topic": "biology"},
),
],
)
retriever = FalkorDBCypherRetriever(
document_store=document_store,
custom_cypher_query="MATCH (d:Document {topic: $topic}) RETURN d",
)
result = retriever.run(parameters={"topic": "linguistics"})
print(result["documents"][0].content)
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack_integrations.components.generators.transformers import (
TransformersChatGenerator,
)
from haystack.dataclasses import ChatMessage
from haystack_integrations.document_stores.falkordb import FalkorDBDocumentStore
from haystack_integrations.components.retrievers.falkordb import FalkorDBCypherRetriever
document_store = FalkorDBDocumentStore(
host="localhost",
port=6379,
recreate_graph=True,
)
document_store.write_documents(
[
Document(
content="There are over 7,000 languages spoken around the world today.",
meta={"topic": "linguistics"},
),
Document(
content="Elephants have been observed to recognize themselves in mirrors.",
meta={"topic": "biology"},
),
],
)
prompt_template = [
ChatMessage.from_user(
"""Given these documents, answer the question.
Documents:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
Question: {{ question }}""",
),
]
pipeline = Pipeline()
pipeline.add_component(
"retriever",
FalkorDBCypherRetriever(
document_store=document_store,
custom_cypher_query="MATCH (d:Document {topic: $topic}) RETURN d",
),
)
pipeline.add_component("prompt_builder", ChatPromptBuilder(template=prompt_template))
pipeline.add_component(
"llm",
TransformersChatGenerator(model="HuggingFaceTB/SmolLM2-135M-Instruct"),
)
pipeline.connect("retriever.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder.prompt", "llm.messages")
result = pipeline.run(
{
"retriever": {"parameters": {"topic": "linguistics"}},
"prompt_builder": {"question": "How many languages are there?"},
},
)
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/retrievers/falkordbembeddingretriever
# FalkorDBEmbeddingRetriever
An embedding-based Retriever compatible with the FalkorDB Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline
2. The last component in a semantic search pipeline | | **Mandatory init variables** | `document_store`: An instance of a [FalkorDBDocumentStore](../../document-stores/falkordbdocumentstore.mdx) | | **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) | | **Output variables** | `documents`: A list of documents | | **API reference** | [FalkorDB](/reference/integrations-falkordb) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/falkordb | | **Package name** | `falkordb-haystack` |
## Overview
The `FalkorDBEmbeddingRetriever` retrieves documents from a `FalkorDBDocumentStore` using FalkorDB's native vector index. It compares the query embedding with document embeddings and returns the most similar documents.
In addition to `query_embedding`, the retriever accepts optional `filters` to narrow the search space and `top_k` to limit the number of results.
The embedding dimension and similarity function are configured on the `FalkorDBDocumentStore` at initialization time.
## Installation
```shell
pip install falkordb-haystack
```
Ensure FalkorDB is running, for example via Docker:
```shell
docker run -d -p 6379:6379 falkordb/falkordb:latest
```
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
### On its own
```python
from haystack import Document
from haystack_integrations.document_stores.falkordb import FalkorDBDocumentStore
from haystack_integrations.components.retrievers.falkordb import (
FalkorDBEmbeddingRetriever,
)
document_store = FalkorDBDocumentStore(
host="localhost",
port=6379,
embedding_dim=3,
recreate_graph=True,
)
document_store.write_documents(
[
Document(
content="There are over 7,000 languages spoken around the world today.",
embedding=[0.1, 0.2, 0.3],
),
Document(
content="Elephants have been observed to recognize themselves in mirrors.",
embedding=[0.8, 0.1, 0.5],
),
],
)
retriever = FalkorDBEmbeddingRetriever(document_store=document_store, top_k=1)
result = retriever.run(query_embedding=[0.1, 0.2, 0.3])
print(result["documents"][0].content)
```
### In a pipeline
```python
from haystack import Document, Pipeline
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack_integrations.document_stores.falkordb import FalkorDBDocumentStore
from haystack_integrations.components.retrievers.falkordb import (
FalkorDBEmbeddingRetriever,
)
document_store = FalkorDBDocumentStore(
host="localhost",
port=6379,
embedding_dim=384,
recreate_graph=True,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to recognize themselves in mirrors.",
),
Document(
content="Bioluminescent waves can be seen in the Maldives and Puerto Rico.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings["documents"],
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"),
)
query_pipeline.add_component(
"retriever",
FalkorDBEmbeddingRetriever(document_store=document_store, top_k=3),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
result = query_pipeline.run(
{"text_embedder": {"text": "How many languages are there?"}},
)
print(result["retriever"]["documents"][0].content)
```
---
// File: pipeline-components/retrievers/filterretriever
# FilterRetriever
Use this Retriever with any Document Store to get the Documents that match specific filters.
2. The last component in a semantic search pipeline | | **Mandatory init variables** | `document_store`: An instance of a [FalkorDBDocumentStore](../../document-stores/falkordbdocumentstore.mdx) | | **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) | | **Output variables** | `documents`: A list of documents | | **API reference** | [FalkorDB](/reference/integrations-falkordb) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/falkordb | | **Package name** | `falkordb-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | At the beginning of a Pipeline |
| **Mandatory init variables** | `document_store`: An instance of a Document Store |
| **Mandatory run variables** | `filters`: A dictionary of filters in the same syntax supported by the Document Stores |
| **Output variables** | `documents`: All the documents that match these filters |
| **API reference** | [Retrievers](/reference/retrievers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/filter_retriever.py |
| **Package name** | `haystack-ai` |
## Overview
`FilterRetriever` retrieves Documents that match the provided filters.
It’s a special kind of Retriever – it can work with all Document Stores instead of being specialized to work with only one.
However, as every other Retriever, it needs some Document Store at initialization time, and it will perform filtering on the content of that instance only.
Therefore, it can be used as any other Retriever in a Pipeline.
Pay attention when using `FilterRetriever` on a Document Store that contains many Documents, as `FilterRetriever` will return all documents that match the filters. The `run` command with no filters can easily overwhelm other components in the Pipeline (for example, Generators):
```python
filter_retriever.run({})
```
Another thing to note is that `FilterRetriever` does not score your Documents or rank them in any way. If you need to rank the Documents by similarity to a query, consider using Ranker components.
## Usage
### On its own
```python
from haystack import Document
from haystack.components.retrievers import FilterRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
docs = [
Document(content="Python is a popular programming language", meta={"lang": "en"}),
Document(
content="python ist eine beliebte Programmiersprache",
meta={"lang": "de"},
),
]
doc_store = InMemoryDocumentStore()
doc_store.write_documents(docs)
retriever = FilterRetriever(doc_store)
result = retriever.run(filters={"field": "lang", "operator": "==", "value": "en"})
assert "documents" in result
assert len(result["documents"]) == 1
assert result["documents"][0].content == "Python is a popular programming language"
```
### In a RAG pipeline
Set your `OPENAI_API_KEY` as an environment variable and then run the following code:
```python
from haystack.components.retrievers.filter_retriever import FilterRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack import Document, Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy
# OpenAIChatGenerator reads the OPENAI_API_KEY environment variable by default.
document_store = InMemoryDocumentStore()
documents = [
Document(content="Mark lives in Berlin.", meta={"year": 2018}),
Document(content="Mark lives in Paris.", meta={"year": 2021}),
Document(content="Mark is Danish.", meta={"year": 2021}),
Document(content="Mark lives in New York.", meta={"year": 2023}),
]
document_store.write_documents(documents=documents)
# Create a RAG query pipeline
prompt_template = [
ChatMessage.from_user(
"""
Given these documents, answer the question.\nDocuments:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
\nQuestion: {{question}}
\nAnswer:
""",
),
]
rag_pipeline = Pipeline()
rag_pipeline.add_component(
name="retriever",
instance=FilterRetriever(document_store=document_store),
)
rag_pipeline.add_component(
instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
result = rag_pipeline.run(
{
"retriever": {"filters": {"field": "year", "operator": "==", "value": 2021}},
"prompt_builder": {"question": "Where does Mark live?"},
},
)
print(result["llm"]["replies"][0].text)
```
Here’s an example output you might get:
```
According to the provided documents, Mark lives in Paris.
```
---
// File: pipeline-components/retrievers/googledriveretriever
# GoogleDriveRetriever
Retrieves files from Google Drive via the Drive API v3 search endpoint.
| | |
| --- | --- |
| **Most common position in a pipeline** | At the start of a query pipeline, after an [`OAuthTokenResolver`](../connectors/oauthtokenresolver.mdx) that provides the `access_token` |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `query`: The search query string
`access_token`: A delegated Google OAuth bearer token, typically wired from an upstream `OAuthTokenResolver` | | **Output variables** | `documents`: A list of [Documents](../../concepts/data-classes.mdx) holding file metadata (and optionally exported text) | | **API reference** | [Google Drive](/reference/integrations-google-drive) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_drive | | **Package name** | `google-drive-haystack` |
## Overview
`GoogleDriveRetriever` runs a full-text search over a user's Google Drive (and optionally shared drives) through the [Drive API v3](https://developers.google.com/drive/api/reference/rest/v3/files/list) `files.list` endpoint and maps each matching file to a Haystack `Document`.
By default, each `Document` carries resource metadata (`file_name`, `file_id`, `web_url`, `mime_type`, `file_extension`, author, and timestamps) and uses the file `description` or `name` as `content`, because the Drive search API does not return a text snippet. Set `include_content=True` to additionally export native Google Docs/Sheets/Slides to text and use that as the `Document` content. Binary files (PDF, DOCX, ...) are never downloaded by the retriever.
To download the full content of the matching files, compose it with [`GoogleDriveFetcher`](../fetchers/googledrivefetcher.mdx) on the returned `web_url`/`file_id`, followed by a converter.
### Authentication
The retriever takes a per-user `access_token` as a run input. The token must carry a delegated Google OAuth scope that allows search, for example `https://www.googleapis.com/auth/drive.readonly`. The metadata-only `drive.metadata.readonly` scope cannot search file content or export documents. Typically you wire the token from an upstream [`OAuthTokenResolver`](../connectors/oauthtokenresolver.mdx), which emits a plain string. A `Secret` is also accepted and resolved internally.
### Scoping and filtering the search
- `query_filter`: an optional Drive query clause AND-ed with the full-text search term, for example `"mimeType != 'application/vnd.google-apps.folder'"` or `"'`access_token`: A delegated Google OAuth bearer token, typically wired from an upstream `OAuthTokenResolver` | | **Output variables** | `documents`: A list of [Documents](../../concepts/data-classes.mdx) holding file metadata (and optionally exported text) | | **API reference** | [Google Drive](/reference/integrations-google-drive) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_drive | | **Package name** | `google-drive-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In query pipelines:
In a RAG pipeline, before a [`PromptBuilder`](../builders/promptbuilder.mdx)
In a semantic search pipeline, as the last component
In an extractive QA pipeline, before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) | | **Mandatory init variables** | `document_store`: An instance of [InMemoryDocumentStore](../../document-stores/inmemorydocumentstore.mdx) | | **Mandatory run variables** | `query`: A query string | | **Output variables** | `documents`: A list of documents (matching the query) | | **API reference** | [Retrievers](/reference/retrievers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/in_memory/bm25_retriever.py | | **Package name** | `haystack-ai` |
## Overview
`InMemoryBM25Retriever` is a keyword-based Retriever that fetches Documents matching a query from a temporary in-memory database. It determines the similarity between Documents and the query based on the BM25 algorithm, which computes a weighted word overlap between the two strings.
Since the `InMemoryBM25Retriever` matches strings based on word overlap, it’s often used to find exact matches to names of persons or products, IDs, or well-defined error messages. The BM25 algorithm is very lightweight and simple. Nevertheless, it can be hard to beat with more complex embedding-based approaches on out-of-domain data.
In addition to the `query`, the `InMemoryBM25Retriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
Some relevant parameters that impact the BM25 retrieval must be defined when the corresponding `InMemoryDocumentStore` is initialized: these include the specific BM25 algorithm and its parameters.
## Usage
### On its own
```python
from haystack import Document
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_store.write_documents(documents=documents)
retriever = InMemoryBM25Retriever(document_store=document_store)
retriever.run(query="How many languages are spoken around the world today?")
```
### In a Pipeline
#### In a RAG Pipeline
Here's an example of the Retriever in a retrieval-augmented generation pipeline:
```python
import os
from haystack import Document
from haystack import Pipeline
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.dataclasses import ChatMessage
from haystack.document_stores.in_memory import InMemoryDocumentStore
# Create a RAG query pipeline
prompt_template = [
ChatMessage.from_user(
"""
Given these documents, answer the question.\nDocuments:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
\nQuestion: {{question}}
\nAnswer:
""",
),
]
os.environ["OPENAI_API_KEY"] = "sk-XXXXXX"
rag_pipeline = Pipeline()
rag_pipeline.add_component(
instance=InMemoryBM25Retriever(document_store=InMemoryDocumentStore()),
name="retriever",
)
rag_pipeline.add_component(
instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")
# Draw the pipeline
rag_pipeline.draw("./rag_pipeline.png")
# Add Documents
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
rag_pipeline.get_component("retriever").document_store.write_documents(documents)
# Run the pipeline
question = "How many languages are there?"
result = rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
"answer_builder": {"query": question},
},
)
print(result["answer_builder"]["answers"][0])
```
#### In a Document Search Pipeline
Here's how you can use this Retriever in a document search pipeline:
```python
from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
# Create components and a query pipeline
document_store = InMemoryDocumentStore()
retriever = InMemoryBM25Retriever(document_store=document_store)
pipeline = Pipeline()
pipeline.add_component(instance=retriever, name="retriever")
# Add Documents
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_store.write_documents(documents)
# Run the pipeline
result = pipeline.run(data={"retriever": {"query": "How many languages are there?"}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/inmemoryembeddingretriever
# InMemoryEmbeddingRetriever
Use this Retriever with the InMemoryDocumentStore if you're looking for embedding-based retrieval.
In a RAG pipeline, before a [`PromptBuilder`](../builders/promptbuilder.mdx)
In a semantic search pipeline, as the last component
In an extractive QA pipeline, before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) | | **Mandatory init variables** | `document_store`: An instance of [InMemoryDocumentStore](../../document-stores/inmemorydocumentstore.mdx) | | **Mandatory run variables** | `query`: A query string | | **Output variables** | `documents`: A list of documents (matching the query) | | **API reference** | [Retrievers](/reference/retrievers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/in_memory/bm25_retriever.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | In query pipelines:
In a RAG pipeline, before a [`PromptBuilder`](../builders/promptbuilder.mdx)
In a semantic search pipeline, as the last component
In an extractive QA pipeline, after a Tex tEmbedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) | | **Mandatory init variables** | `document_store`: An instance of [InMemoryDocumentStore](../../document-stores/inmemorydocumentstore.mdx) | | **Mandatory run variables** | `query_embedding`: A list of floating point numbers | | **Output variables** | `documents`: A list of documents | | **API reference** | [Retrievers](/reference/retrievers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/in_memory/embedding_retriever.py | | **Package name** | `haystack-ai` |
## Overview
The `InMemoryEmbeddingRetriever` is an embedding-based Retriever compatible with the `InMemoryDocumentStore`. It compares the query and Document embeddings and fetches the Documents most relevant to the query from the `InMemoryDocumentStore` based on the outcome.
When using the `InMemoryEmbeddingRetriever` in your NLP system, make sure it has the query and Document embeddings available. You can do so by adding a DocumentEmbedder to your indexing pipeline and a Text Embedder to your query pipeline. For details, see [Embedders](../embedders.mdx).
In addition to the `query_embedding`, the `InMemoryEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
The `embedding_similarity_function` to use for embedding retrieval must be defined when the corresponding`InMemoryDocumentStore` is initialized.
## Usage
### In a pipeline
Use this Retriever in a query pipeline like this:
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack.components.retrievers import InMemoryEmbeddingRetriever
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/mem0memoryretriever
# Mem0MemoryRetriever
Retrieves long-term memories from Mem0 as `ChatMessage` objects.
In a RAG pipeline, before a [`PromptBuilder`](../builders/promptbuilder.mdx)
In a semantic search pipeline, as the last component
In an extractive QA pipeline, after a Tex tEmbedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) | | **Mandatory init variables** | `document_store`: An instance of [InMemoryDocumentStore](../../document-stores/inmemorydocumentstore.mdx) | | **Mandatory run variables** | `query_embedding`: A list of floating point numbers | | **Output variables** | `documents`: A list of documents | | **API reference** | [Retrievers](/reference/retrievers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/in_memory/embedding_retriever.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before an [`Agent`](../agents-1/agent.mdx) or Chat Generator in memory-augmented pipelines |
| **Mandatory init variables** | `memory_store`: A `Mem0MemoryStore` instance |
| **Mandatory run variables** | `query`: A text query or `None`; at least one Mem0 scope through `user_id`, `run_id`, `agent_id`, `app_id`, or `filters` |
| **Output variables** | `memories`: A list of `ChatMessage` objects |
| **Mem0 API docs** | [Search Memories](https://docs.mem0.ai/api-reference/memory/search-memories), [Memory Filters](https://docs.mem0.ai/platform/features/v2-memory-filters) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mem0 |
| **Package name** | `mem0-haystack` |
## Overview
`Mem0MemoryRetriever` retrieves memories from a `Mem0MemoryStore` and returns them as system `ChatMessage` objects.
Use it to inject long-term memory into an Agent or a chat generation pipeline before the model produces a response.
The `query` input can be a string or `None`.
When `query` is a string, the component searches for relevant memories and applies `top_k`.
When `query` is `None`, it returns all memories matching the provided scope.
Scope the retrieval with at least one Mem0 entity ID: `user_id`, `run_id`, `agent_id`, or `app_id`.
You can also pass Haystack-style `filters`; when filters and ID parameters are both provided, they are combined with an `AND` condition.
For general filter syntax, see [Metadata Filtering](../../concepts/metadata-filtering.mdx).
User-provided Mem0 metadata is included in each returned message's `meta`.
Mem0 retrieval fields such as `memory_id`, `user_id`, `score`, and timestamps are included under `meta["mem0"]`.
### Installation
Install the Mem0 integration:
```shell
pip install mem0-haystack
```
Set your Mem0 API key:
```shell
export MEM0_API_KEY="your-mem0-api-key"
```
## Usage
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.retrievers.mem0 import Mem0MemoryRetriever
from haystack_integrations.memory_stores.mem0 import Mem0MemoryStore
store = Mem0MemoryStore()
store.add_memories(
messages=[ChatMessage.from_user("Alice prefers concise Python examples.")],
user_id="alice",
infer=False,
)
retriever = Mem0MemoryRetriever(memory_store=store, top_k=3)
result = retriever.run(query="answer style", user_id="alice")
memories = result["memories"]
for memory in memories:
print(memory.text)
```
To retrieve all memories in scope, pass `query=None`:
```python
all_memories = retriever.run(query=None, user_id="alice")["memories"]
print([memory.text for memory in all_memories])
```
### In a Pipeline
This example retrieves memories, prepends them to the current user message, and passes the combined message list to an Agent.
```python
from haystack import Pipeline
from haystack.components.agents import Agent
from haystack.components.converters import OutputAdapter
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.retrievers.mem0 import Mem0MemoryRetriever
from haystack_integrations.memory_stores.mem0 import Mem0MemoryStore
store = Mem0MemoryStore()
pipeline = Pipeline()
pipeline.add_component("retriever", Mem0MemoryRetriever(memory_store=store, top_k=5))
pipeline.add_component(
"memory_context",
OutputAdapter(
template="{{ memories + user_messages }}",
output_type=list[ChatMessage],
unsafe=True,
),
)
pipeline.add_component(
"agent",
Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
system_prompt=(
"Use any system messages at the start of the conversation as long-term memory. "
"Answer concisely."
),
streaming_callback=print_streaming_chunk,
),
)
pipeline.connect("retriever.memories", "memory_context.memories")
pipeline.connect("memory_context.output", "agent.messages")
query = "Give me a short implementation tip."
pipeline.run(
{
"retriever": {
"query": query,
"user_id": "alice",
},
"memory_context": {
"user_messages": [
ChatMessage.from_user(query),
],
},
},
)
```
---
// File: pipeline-components/retrievers/mongodbatlasembeddingretriever
# MongoDBAtlasEmbeddingRetriever
This is an embedding Retriever compatible with the MongoDB Atlas Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [MongoDBAtlasDocumentStore](../../document-stores/mongodbatlasdocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A list of floats |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [MongoDB Atlas](/reference/integrations-mongodb-atlas) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mongodb_atlas |
| **Package name** | `mongodb-atlas-haystack` |
The `MongoDBAtlasEmbeddingRetriever` is an embedding-based Retriever compatible with the [`MongoDBAtlasDocumentStore`](../../document-stores/mongodbatlasdocumentstore.mdx). It compares the query and Document embeddings and fetches the Documents most relevant to the query from the Document Store based on the outcome.
### Parameters
When using the `MongoDBAtlasEmbeddingRetriever` in your NLP system, ensure the query and Document [embeddings](../embedders.mdx) are available. You can do so by adding a Document Embedder to your indexing Pipeline and a Text Embedder to your query Pipeline.
In addition to the `query_embedding`, the `MongoDBAtlasEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
## Usage
### Installation
To start using MongoDB Atlas with Haystack, install the package with:
```shell
pip install mongodb-atlas-haystack
```
### On its own
The Retriever needs an instance of `MongoDBAtlasDocumentStore` and indexed Documents to run.
```python
from haystack_integrations.document_stores.mongodb_atlas import (
MongoDBAtlasDocumentStore,
)
from haystack_integrations.components.retrievers.mongodb_atlas import (
MongoDBAtlasEmbeddingRetriever,
)
document_store = MongoDBAtlasDocumentStore()
retriever = MongoDBAtlasEmbeddingRetriever(document_store=document_store)
# example run query
retriever.run(query_embedding=[0.1] * 384)
```
### In a Pipeline
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Pipeline, Document
from haystack.document_stores.types import DuplicatePolicy
from haystack.components.writers import DocumentWriter
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack_integrations.document_stores.mongodb_atlas import (
MongoDBAtlasDocumentStore,
)
from haystack_integrations.components.retrievers.mongodb_atlas import (
MongoDBAtlasEmbeddingRetriever,
)
# Create some example documents
documents = [
Document(content="My name is Jean and I live in Paris."),
Document(content="My name is Mark and I live in Berlin."),
Document(content="My name is Giorgio and I live in Rome."),
]
document_store = MongoDBAtlasDocumentStore()
# Define some more components
doc_writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.SKIP)
doc_embedder = SentenceTransformersDocumentEmbedder(model="intfloat/e5-base-v2")
query_embedder = SentenceTransformersTextEmbedder(model="intfloat/e5-base-v2")
# Pipeline that ingests document for retrieval
ingestion_pipe = Pipeline()
ingestion_pipe.add_component(instance=doc_embedder, name="doc_embedder")
ingestion_pipe.add_component(instance=doc_writer, name="doc_writer")
ingestion_pipe.connect("doc_embedder.documents", "doc_writer.documents")
ingestion_pipe.run({"doc_embedder": {"documents": documents}})
# Build a RAG pipeline with a Retriever to get relevant documents to
# the query and an OpenAIChatGenerator interacting with LLMs using a custom prompt.
prompt_template = [
ChatMessage.from_user(
"""
Given these documents, answer the question.\nDocuments:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
\nQuestion: {{question}}
\nAnswer:
""",
),
]
rag_pipeline = Pipeline()
rag_pipeline.add_component(instance=query_embedder, name="query_embedder")
rag_pipeline.add_component(
instance=MongoDBAtlasEmbeddingRetriever(document_store=document_store),
name="retriever",
)
rag_pipeline.add_component(
instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.connect("query_embedder", "retriever.query_embedding")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
# Ask a question on the data you just added.
question = "Where does Mark live?"
result = rag_pipeline.run(
{
"query_embedder": {"text": question},
"prompt_builder": {"question": question},
},
)
# The generated reply is a ChatMessage; its text holds the answer.
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/retrievers/mongodbatlasfulltextretriever
# MongoDBAtlasFullTextRetriever
This is a full-text search Retriever compatible with the MongoDB Atlas Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [TransformersExtractiveReader](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [MongoDBAtlasDocumentStore](../../document-stores/mongodbatlasdocumentstore.mdx) |
| **Mandatory run variables** | `query`: A query string to search for. If the query contains multiple terms, Atlas Search evaluates each term separately for matches. |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [MongoDB Atlas](/reference/integrations-mongodb-atlas) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mongodb_atlas |
| **Package name** | `mongodb-atlas-haystack` |
The `MongoDBAtlasFullTextRetriever` is a full-text search Retriever compatible with the [`MongoDBAtlasDocumentStore`](../../document-stores/mongodbatlasdocumentstore.mdx). The full-text search is dependent on the `full_text_search_index` used in the [`MongoDBAtlasDocumentStore`](../../document-stores/mongodbatlasdocumentstore.mdx).
### Parameters
In addition to the `query`, the `MongoDBAtlasFullTextRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
When running the component, you can specify more optional parameters such as `fuzzy` or `synonyms`, `match_criteria`, `score`. Check out our [MongoDB Atlas](/reference/integrations-mongodb-atlas) API Reference for more details on all parameters.
## Usage
### Installation
To start using MongoDB Atlas with Haystack, install the package with:
```shell
pip install mongodb-atlas-haystack
```
### On its own
The Retriever needs an instance of `MongoDBAtlasDocumentStore` and indexed documents to run.
```python
from haystack_integrations.document_stores.mongodb_atlas import (
MongoDBAtlasDocumentStore,
)
from haystack_integrations.components.retrievers.mongodb_atlas import (
MongoDBAtlasFullTextRetriever,
)
store = MongoDBAtlasDocumentStore(
database_name="your_existing_db",
collection_name="your_existing_collection",
vector_search_index="your_existing_index",
full_text_search_index="your_existing_index",
)
retriever = MongoDBAtlasFullTextRetriever(document_store=store)
results = retriever.run(query="Your search query")
print(results["documents"])
```
### In a Pipeline
Here's a Hybrid Retrieval pipeline example that makes use of both available MongoDB Atlas Retrievers:
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Pipeline, Document
from haystack.document_stores.types import DuplicatePolicy
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.joiners import DocumentJoiner
from haystack_integrations.document_stores.mongodb_atlas import (
MongoDBAtlasDocumentStore,
)
from haystack_integrations.components.retrievers.mongodb_atlas import (
MongoDBAtlasEmbeddingRetriever,
MongoDBAtlasFullTextRetriever,
)
documents = [
Document(content="My name is Jean and I live in Paris."),
Document(content="My name is Mark and I live in Berlin."),
Document(content="My name is Giorgio and I live in Rome."),
Document(content="Python is a programming language popular for data science."),
Document(
content="MongoDB Atlas offers full-text search and vector search capabilities.",
),
]
document_store = MongoDBAtlasDocumentStore(
database_name="haystack_test",
collection_name="test_collection",
vector_search_index="test_vector_search_index",
full_text_search_index="test_full_text_search_index",
)
# Clean out any old data so this example is repeatable
print(f"Clearing collection {document_store.collection_name} …")
document_store.collection.delete_many({})
ingest_pipe = Pipeline()
doc_embedder = SentenceTransformersDocumentEmbedder(model="intfloat/e5-base-v2")
ingest_pipe.add_component(instance=doc_embedder, name="doc_embedder")
doc_writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.SKIP)
ingest_pipe.add_component(instance=doc_writer, name="doc_writer")
ingest_pipe.connect("doc_embedder.documents", "doc_writer.documents")
print(f"Running ingestion on {len(documents)} in-memory docs …")
ingest_pipe.run({"doc_embedder": {"documents": documents}})
query_pipe = Pipeline()
text_embedder = SentenceTransformersTextEmbedder(model="intfloat/e5-base-v2")
query_pipe.add_component(instance=text_embedder, name="text_embedder")
embed_retriever = MongoDBAtlasEmbeddingRetriever(document_store=document_store, top_k=3)
query_pipe.add_component(instance=embed_retriever, name="embedding_retriever")
query_pipe.connect("text_embedder", "embedding_retriever")
# (c) full-text retriever
ft_retriever = MongoDBAtlasFullTextRetriever(document_store=document_store, top_k=3)
query_pipe.add_component(instance=ft_retriever, name="full_text_retriever")
joiner = DocumentJoiner(join_mode="reciprocal_rank_fusion", top_k=3)
query_pipe.add_component(instance=joiner, name="joiner")
query_pipe.connect("embedding_retriever", "joiner")
query_pipe.connect("full_text_retriever", "joiner")
question = "Where does Mark live?"
print(f"Running hybrid retrieval for query: '{question}'")
output = query_pipe.run(
{
"text_embedder": {"text": question},
"full_text_retriever": {"query": question},
},
)
print("\nFinal fused documents:")
for doc in output["joiner"]["documents"]:
print(f"- {doc.content}")
```
---
// File: pipeline-components/retrievers/mssharepointretriever
# MSSharePointRetriever
Retrieves content from Microsoft SharePoint and OneDrive via the Microsoft Search (Graph) API.
| | |
| --- | --- |
| **Most common position in a pipeline** | At the start of a query pipeline, after an [`OAuthTokenResolver`](../connectors/oauthtokenresolver.mdx) that provides the `access_token` |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `query`: The search query string
`access_token`: A delegated Microsoft Graph bearer token, typically wired from an upstream `OAuthTokenResolver` | | **Output variables** | `documents`: A list of [Documents](../../concepts/data-classes.mdx) holding the search snippets and resource metadata | | **API reference** | [Microsoft SharePoint](/reference/integrations-microsoft-sharepoint) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/microsoft_sharepoint | | **Package name** | `microsoft-sharepoint-haystack` |
## Overview
`MSSharePointRetriever` searches a user's Microsoft SharePoint and OneDrive content through the [Microsoft Search (Graph) API](https://learn.microsoft.com/en-us/graph/api/resources/search-api-overview). Given a query, it calls `POST /search/query` and maps each hit to a Haystack `Document` whose `content` is the search snippet and whose `meta` carries the resource metadata: `file_name`, `web_url`, `entity_type`, `created_date_time`, `last_modified_date_time`, `created_by`, `last_modified_by`, `mime_type`, and `file_extension`. It also stores the SharePoint identifiers a downstream fetcher needs to read list items and pages by ID (`site_id`, `list_id`, `list_item_id`, `list_item_unique_id`).
The retriever does **not** download or convert the underlying files – it only returns Search snippets and metadata. To download the full content of the hits, compose it with [`MSSharePointFetcher`](../fetchers/mssharepointfetcher.mdx) followed by a converter.
### Authentication
The retriever takes a per-user `access_token` as a run input. The token must carry **delegated** Microsoft Graph permissions (for example `Files.Read.All`, plus `Sites.Read.All` for site and list scoping); the Search API supports delegated permissions only. Typically you wire the token from an upstream [`OAuthTokenResolver`](../connectors/oauthtokenresolver.mdx), which emits a plain string. A `Secret` is also accepted and resolved internally.
### Scoping and filtering the search
You can narrow what is searched in several ways:
- `entity_types`: which Microsoft Search entity types to query. Defaults to `["driveItem", "listItem"]`, which covers files, folders, SharePoint pages and news, and list items. Other valid values are `"list"` and `"site"`.
- KQL operators embedded directly in the query, for example `filetype:docx`, `author:"Jane Doe"`, or `path:"https://contoso.sharepoint.com/sites/Team"`. See the [Keyword Query Language (KQL) syntax reference](https://learn.microsoft.com/en-us/sharepoint/dev/general-development/keyword-query-language-kql-syntax-reference).
- `query_template`: a reusable template such as `'{searchTerms} path:"https://contoso.sharepoint.com/sites/Team"'`, where the literal `{searchTerms}` placeholder is replaced by the run-time query.
### Installation
Install the Microsoft SharePoint integration with:
```shell
pip install microsoft-sharepoint-haystack
```
## Usage
### On its own
`access_token` below is a per-user delegated Microsoft Graph bearer token. In production you would obtain it from an [`OAuthTokenResolver`](../connectors/oauthtokenresolver.mdx) rather than pasting it in.
```python
from haystack_integrations.components.retrievers.microsoft_sharepoint import (
MSSharePointRetriever,
)
retriever = MSSharePointRetriever(top_k=5)
result = retriever.run(
query="quarterly roadmap",
access_token="my-delegated-graph-token",
)
for doc in result["documents"]:
print(doc.meta["file_name"], "-", doc.meta["web_url"])
```
### In a pipeline
The following pipeline obtains a token from an `OAuthTokenResolver` and feeds it into the retriever, so that running the pipeline requires only the query:
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack_integrations.components.connectors.oauth import OAuthTokenResolver
from haystack_integrations.utils.oauth import OAuthRefreshTokenSource
from haystack_integrations.components.retrievers.microsoft_sharepoint import (
MSSharePointRetriever,
)
pipeline = Pipeline()
pipeline.add_component(
"resolver",
OAuthTokenResolver(
token_source=OAuthRefreshTokenSource(
token_url="https://login.microsoftonline.com/common/oauth2/v2.0/token",
client_id="aaa-bbb-ccc",
refresh_token=Secret.from_env_var("MS_REFRESH_TOKEN"),
scopes=[
"https://graph.microsoft.com/Files.Read.All",
"https://graph.microsoft.com/Sites.Read.All",
"offline_access",
],
),
),
)
pipeline.add_component("retriever", MSSharePointRetriever(top_k=5))
pipeline.connect("resolver.access_token", "retriever.access_token")
result = pipeline.run({"retriever": {"query": "quarterly roadmap"}})
documents = result["retriever"]["documents"]
```
To download and convert the full content of the retrieved hits, connect the retriever's `documents` output to a [`MSSharePointFetcher`](../fetchers/mssharepointfetcher.mdx). See that page for an end-to-end retrieve-fetch-convert example.
---
// File: pipeline-components/retrievers/multiqueryembeddingretriever
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# MultiQueryEmbeddingRetriever
Retrieves documents using multiple queries in parallel with an embedding-based Retriever.
`access_token`: A delegated Microsoft Graph bearer token, typically wired from an upstream `OAuthTokenResolver` | | **Output variables** | `documents`: A list of [Documents](../../concepts/data-classes.mdx) holding the search snippets and resource metadata | | **API reference** | [Microsoft SharePoint](/reference/integrations-microsoft-sharepoint) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/microsoft_sharepoint | | **Package name** | `microsoft-sharepoint-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`QueryExpander`](../query/queryexpander.mdx) component, before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) in RAG pipelines |
| **Mandatory init variables** | `retriever`: An embedding-based Retriever (such as `InMemoryEmbeddingRetriever`)
`query_embedder`: A Text Embedder component | | **Mandatory run variables** | `queries`: A list of query strings | | **Output variables** | `documents`: A list of retrieved documents sorted by relevance score | | **API reference** | [Retrievers](/reference/retrievers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/multi_query_embedding_retriever.py | | **Package name** | `haystack-ai` |
## Overview
`MultiQueryEmbeddingRetriever` improves retrieval recall by searching for documents using multiple queries in parallel. Each query is converted to an embedding using a Text Embedder, and an embedding-based Retriever fetches relevant documents.
The component:
- Processes queries in parallel using a thread pool for better performance
- Automatically deduplicates results based on document content
- Sorts the final results by relevance score
This Retriever is particularly effective when combined with [`QueryExpander`](../query/queryexpander.mdx), which generates multiple query variations from a single user query. By searching with these variations, you can find documents that might not match the original query phrasing but are still relevant.
Use `MultiQueryEmbeddingRetriever` when your documents use different words than your users' queries, or when you want to find more diverse results in RAG pipelines. Running multiple queries takes more time, but you can speed it up by increasing `max_workers` to run queries in parallel.
:::tip[When to use a `MultiQueryTextRetriever` instead]
If you need exact keyword matching and don't want to use embeddings, use [`MultiQueryTextRetriever`](multiquerytextretriever.mdx) instead. It works with text-based Retrievers like `InMemoryBM25Retriever` and is better when synonyms can be generated through query expansion.
:::
### Passing Additional Retriever Parameters
You can pass additional parameters to the underlying Retriever using `retriever_kwargs`:
```python
result = multi_query_retriever.run(
queries=["renewable energy", "sustainable power"],
retriever_kwargs={"top_k": 5},
)
```
## Usage
This pipeline takes a single query "sustainable power generation" and expands it into multiple variations using an LLM (for example: "renewable energy sources", "green electricity", "clean power"). The Retriever then converts each variation to an embedding and searches for similar documents. This way, documents about "solar energy" or "wind energy" can be found even though they don't contain the words "sustainable power generation".
Before running the pipeline, documents must be embedded using a Document Embedder and stored in the Document Store.
`query_embedder`: A Text Embedder component | | **Mandatory run variables** | `queries`: A list of query strings | | **Output variables** | `documents`: A list of retrieved documents sorted by relevance score | | **API reference** | [Retrievers](/reference/retrievers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/multi_query_embedding_retriever.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`QueryExpander`](../query/queryexpander.mdx) component, before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) in RAG pipelines |
| **Mandatory init variables** | `retriever`: A text-based Retriever (such as `InMemoryBM25Retriever`) |
| **Mandatory run variables** | `queries`: A list of query strings |
| **Output variables** | `documents`: A list of retrieved documents sorted by relevance score |
| **API reference** | [Retrievers](/reference/retrievers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/multi_query_text_retriever.py |
| **Package name** | `haystack-ai` |
## Overview
`MultiQueryTextRetriever` improves retrieval recall by searching for documents using multiple queries in parallel. It wraps a text-based Retriever (such as `InMemoryBM25Retriever`) and processes multiple query strings simultaneously using a thread pool.
The component:
- Processes queries in parallel for better performance
- Automatically deduplicates results based on document content
- Sorts the final results by relevance score
This Retriever is particularly effective when combined with [`QueryExpander`](../query/queryexpander.mdx), which generates multiple query variations from a single user query. By searching with these variations, you can find documents that use different keywords than the original query.
Use `MultiQueryTextRetriever` when your documents use different words than your users' queries, or when you want to use query expansion with keyword-based search (BM25). Running multiple queries takes more time, but you can speed it up by increasing `max_workers` to run queries in parallel.
:::tip[When to use `MultiQueryEmbeddingRetriever` instead]
If you need semantic search where meaning matters more than exact keyword matches, use [`MultiQueryEmbeddingRetriever`](multiqueryembeddingretriever.mdx) instead. It works with embedding-based Retrievers and requires a Text Embedder.
:::
### Passing Additional Retriever Parameters
You can pass additional parameters to the underlying Retriever using `retriever_kwargs`:
```python
result = multiquery_retriever.run(
queries=["renewable energy", "sustainable power"],
retriever_kwargs={"top_k": 5},
)
```
## Usage
### On its own
In this example, we pass three queries manually to the Retriever: "renewable energy", "geothermal", and "hydropower". The Retriever runs a BM25 search for each query (retrieving up to 2 documents per query), then combines all results, removes duplicates, and sorts them by score.
```python
from haystack import Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers import (
InMemoryBM25Retriever,
MultiQueryTextRetriever,
)
documents = [
Document(
content="Renewable energy is energy that is collected from renewable resources.",
),
Document(
content="Solar energy is a type of green energy that is harnessed from the sun.",
),
Document(
content="Wind energy is another type of green energy that is generated by wind turbines.",
),
Document(
content="Hydropower is a form of renewable energy using the flow of water to generate electricity.",
),
Document(
content="Geothermal energy is heat that comes from the sub-surface of the earth.",
),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(documents)
retriever = MultiQueryTextRetriever(
retriever=InMemoryBM25Retriever(document_store=document_store, top_k=2),
)
results = retriever.run(queries=["renewable energy", "geothermal", "hydropower"])
for doc in results["documents"]:
print(f"Content: {doc.content}, Score: {doc.score:.4f}")
```
### In a pipeline with QueryExpander
This pipeline takes a single query "sustainable power" and expands it into multiple variations using an LLM (for example: "renewable energy sources", "green electricity", "clean power"). The Retriever then searches for each variation and combines the results. This way, documents about "solar energy" or "hydropower" can be found even though they don't contain the words "sustainable power".
| | |
| --- | --- |
| **Most common position in a pipeline** | After query input, before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) in RAG pipelines |
| **Mandatory init variables** | `retrievers`: A dictionary mapping names to text retrievers (implementing the `TextRetriever` protocol) |
| **Optional init variables** | `join_mode`: `"reciprocal_rank_fusion"` (default) or `"concatenate"` |
| **Mandatory run variables** | `query`: A query string |
| **Output variables** | `documents`: A merged list of retrieved documents |
| **API reference** | [Retrievers](/reference/retrievers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/multi_retriever.py |
| **Package name** | `haystack-ai` |
## Overview
`MultiRetriever` composes any number of text retrievers into a single component. All retrievers are queried in parallel using a thread pool, and their results are merged before being returned.
The component:
- Queries all retrievers concurrently for better performance
- Merges results across retrievers using the configured `join_mode`
- Supports selectively enabling retrievers at runtime via `active_retrievers`
All retrievers passed to `MultiRetriever` must implement the `TextRetriever` protocol — their `run` method must accept a text `query`, `filters`, and `top_k`. Use [`TextEmbeddingRetriever`](textembeddingretriever.mdx) to wrap an embedding-based retriever so it can be used with this component.
### Join modes
The `join_mode` parameter controls how results from multiple retrievers are merged:
- **`reciprocal_rank_fusion`** (default): Assigns scores based on each document's rank across retrieval lists using the [Reciprocal Rank Fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf) algorithm. Documents appearing highly ranked in multiple lists receive higher scores. Results are deduplicated and returned in descending score order. This is the recommended mode when combining retrievers with incomparable scores, such as BM25 and embedding retrievers.
- **`concatenate`**: Combines all results into a single list and deduplicates.
## Usage
### On its own
This example sets up a `MultiRetriever` combining a BM25 retriever and an embedding-based retriever (wrapped with `TextEmbeddingRetriever`). Both are queried in parallel and the results are merged using reciprocal rank fusion.
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.retrievers import (
InMemoryBM25Retriever,
InMemoryEmbeddingRetriever,
)
from haystack.components.retrievers import MultiRetriever, TextEmbeddingRetriever
from haystack.components.writers import DocumentWriter
documents = [
Document(
content="Renewable energy is energy that is collected from renewable resources.",
),
Document(
content="Solar energy is a type of green energy that is harnessed from the sun.",
),
Document(
content="Wind energy is another type of green energy that is generated by wind turbines.",
),
]
doc_store = InMemoryDocumentStore()
doc_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
doc_writer = DocumentWriter(document_store=doc_store, policy=DuplicatePolicy.SKIP)
doc_writer.run(documents=doc_embedder.run(documents)["documents"])
retriever = MultiRetriever(
retrievers={
"bm25": InMemoryBM25Retriever(document_store=doc_store),
"embedding": TextEmbeddingRetriever(
retriever=InMemoryEmbeddingRetriever(document_store=doc_store),
text_embedder=SentenceTransformersTextEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
),
),
},
top_k=3,
)
result = retriever.run(query="green energy sources")
for doc in result["documents"]:
print(doc.content)
```
### Selecting retrievers at runtime
Use the `active_retrievers` parameter to run only a subset of retrievers. Names must match the keys in the `retrievers` dictionary. Building on the example above:
```python
# Run only the BM25 retriever
result = retriever.run(query="green energy sources", active_retrievers=["bm25"])
for doc in result["documents"]:
print(doc.content)
```
### In a RAG pipeline
This RAG pipeline uses `MultiRetriever` to combine BM25 and embedding retrieval before generating an answer with an LLM.
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.document_stores.types import DuplicatePolicy
from haystack.components.builders import ChatPromptBuilder
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.retrievers import (
InMemoryBM25Retriever,
InMemoryEmbeddingRetriever,
)
from haystack.components.retrievers import MultiRetriever, TextEmbeddingRetriever
from haystack.components.writers import DocumentWriter
from haystack.dataclasses import ChatMessage
documents = [
Document(
content="Renewable energy is energy that is collected from renewable resources.",
),
Document(
content="Solar energy is a type of green energy that is harnessed from the sun.",
),
Document(
content="Wind energy is another type of green energy that is generated by wind turbines.",
),
]
doc_store = InMemoryDocumentStore()
doc_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
doc_writer = DocumentWriter(document_store=doc_store, policy=DuplicatePolicy.SKIP)
doc_writer.run(documents=doc_embedder.run(documents)["documents"])
prompt_template = [
ChatMessage.from_system(
"You are a helpful assistant that answers questions based on the provided documents.",
),
ChatMessage.from_user(
"Given these documents, answer the question.\nDocuments:\n"
"{% for doc in documents %}{{ doc.content }}\n{% endfor %}\n"
"Question: {{ question }}",
),
]
pipeline = Pipeline()
pipeline.add_component(
"retriever",
MultiRetriever(
retrievers={
"bm25": InMemoryBM25Retriever(document_store=doc_store),
"embedding": TextEmbeddingRetriever(
retriever=InMemoryEmbeddingRetriever(document_store=doc_store),
text_embedder=SentenceTransformersTextEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
),
),
},
top_k=3,
),
)
pipeline.add_component(
"prompt_builder",
ChatPromptBuilder(
template=prompt_template,
required_variables=["documents", "question"],
),
)
pipeline.add_component("llm", OpenAIChatGenerator())
pipeline.connect("retriever.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder.prompt", "llm.messages")
result = pipeline.run(
{
"retriever": {"query": "green energy sources"},
"prompt_builder": {"question": "What types of green energy exist?"},
},
)
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/retrievers/opensearchbm25retriever
# OpenSearchBM25Retriever
This is a keyword-based Retriever that fetches Documents matching a query from an OpenSearch Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of an [OpenSearchDocumentStore](../../document-stores/opensearch-document-store.mdx) |
| **Mandatory run variables** | `query`: A query string |
| **Output variables** | `documents`: A list of documents matching the query |
| **API reference** | [OpenSearch](/reference/integrations-opensearch) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opensearch |
| **Package name** | `opensearch-haystack` |
## Overview
`OpenSearchBM25Retriever` is a keyword-based Retriever that fetches Documents matching a query from an `OpenSearchDocumentStore`. It determines the similarity between Documents and the query based on the BM25 algorithm, which computes a weighted word overlap between the two strings.
Since the `OpenSearchBM25Retriever` matches strings based on word overlap, it’s often used to find exact matches to names of persons or products, IDs, or well-defined error messages. The BM25 algorithm is very lightweight and simple. Nevertheless, it can be hard to beat with more complex embedding-based approaches on out-of-domain data.
In addition to the `query`, the `OpenSearchBM25Retriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
You can adjust how [inexact fuzzy matching](https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#fuzziness) is performed, using the `fuzziness` parameter.
It is also possible to specify if all terms in the query must match using the `all_terms_must_match` parameter, which defaults to `False`.
If you want more flexible matching of a query to Documents, you can use the `OpenSearchEmbeddingRetriever`, which uses vectors created by LLMs to retrieve relevant information.
### Setup and installation
[Install](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/) and run an OpenSearch instance.
If you have Docker set up, we recommend pulling the Docker image and running it.
```shell
docker pull opensearchproject/opensearch:3.5.0
docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of an [OpenSearchDocumentStore](../../document-stores/opensearch-document-store.mdx) |
| **Mandatory run variables** | `query_embedding`: A list of floats |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [OpenSearch](/reference/integrations-opensearch) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opensearch |
| **Package name** | `opensearch-haystack` |
## Overview
The `OpenSearchEmbeddingRetriever` is an embedding-based Retriever compatible with the `OpenSearchDocumentStore`. It compares the query and Document embeddings and fetches the Documents most relevant to the query from the `OpenSearchDocumentStore` based on the outcome.
When using the `OpenSearchEmbeddingRetriever` in your NLP system, make sure it has the query and Document embeddings available. You can do so by adding a Document Embedder to your indexing pipeline and a Text Embedder to your query pipeline.
In addition to the `query_embedding`, the `OpenSearchEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
The `embedding_dim` for storing and retrieving embeddings must be defined when the corresponding `OpenSearchDocumentStore` is initialized.
### Setup and installation
[Install](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/) and run an OpenSearch instance.
If you have Docker set up, we recommend pulling the Docker image and running it.
```shell
docker pull opensearchproject/opensearch:3.5.0
docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=
| | |
| --- | --- |
| Most common position in a pipeline | 1. After a TextEmbedder and before a PromptBuilder in a RAG pipeline 2. The last component in a hybrid search pipeline 3. After a TextEmbedder and before a TransformersExtractiveReader in an extractive QA pipeline |
| Mandatory init variables | `document_store`: An instance of `OpenSearchDocumentStore` to use for retrieval
`embedder`: Any [Embedder](../embedders.mdx) implementing the `TextEmbedder` protocol | | Mandatory run variables | `query`: A query string | | Output variables | `documents`: A list of documents matching the query | | API reference | [OpenSearch](/reference/integrations-opensearch) | | GitHub | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opensearch |
## Overview
The `OpenSearchHybridRetriever` combines two retrieval methods:
1. **BM25 Retrieval**: A keyword-based search that uses the BM25 algorithm to find documents based on term frequency and inverse document frequency. It's based on the [`OpenSearchBM25Retriever`](opensearchbm25retriever.mdx) component and is suitable for traditional keyword-based search.
2. **Embedding-based Retrieval**: A semantic search that uses vector similarity to find documents that are semantically similar to the query. It's based on the [`OpenSearchEmbeddingRetriever`](opensearchembeddingretriever.mdx) component and is suitable for semantic search.
The component automatically handles:
- Converting the query into an embedding using the provided embedded,
- Running both retrieval methods in parallel,
- Merging and re-ranking the results using the specified join mode.
### Setup and Installation
```shell
pip install opensearch-haystack
```
### Optional Parameters
This Retriever accepts various optional parameters. You can verify the most up-to-date list of parameters in our [API Reference](/reference/integrations-opensearch#opensearchhybridretriever).
You can pass additional parameters to the underlying components using the `bm25_retriever` and `embedding_retriever` dictionaries.
The `DocumentJoiner` parameters are all exposed on the `OpenSearchHybridRetriever` class, so you can set them directly.
Here's an example:
```python
retriever = OpenSearchHybridRetriever(
document_store=document_store,
embedder=embedder,
bm25_retriever={"raise_on_failure": True},
embedding_retriever={"raise_on_failure": False},
)
```
## Usage
### On its own
This Retriever needs the `OpensearchDocumentStore` populated with documents to run. You can’t use it on its own.
### In a pipeline
Here's a basic example of how to use the `OpenSearchHybridRetriever`:
You can use the following command to run OpenSearch locally using Docker. Make sure you have Docker installed and running on your machine. Note that this example disables the security plugin for simplicity. In a production environment, you should enable security features.
```shell
docker run -d \
--name opensearch-nosec \
-p 9200:9200 \
-p 9600:9600 \
-e "discovery.type=single-node" \
-e "DISABLE_SECURITY_PLUGIN=true" \
opensearchproject/opensearch:3.5.0
```
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document
from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder, SentenceTransformersDocumentEmbedder
from haystack_integrations.components.retrievers.opensearch import OpenSearchHybridRetriever
from haystack_integrations.document_stores.opensearch import OpenSearchDocumentStore
# Initialize the document store
doc_store = OpenSearchDocumentStore(
hosts=["http://localhost:9200"],
index="document_store",
embedding_dim=384,
)
# Create some sample documents
docs = [
Document(content="Machine learning is a subset of artificial intelligence."),
Document(content="Deep learning is a subset of machine learning."),
Document(content="Natural language processing is a field of AI."),
Document(content="Reinforcement learning is a type of machine learning."),
Document(content="Supervised learning is a type of machine learning."),
]
# Embed the documents and add them to the document store
doc_embedder = SentenceTransformersDocumentEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
docs = doc_embedder.run(docs)
doc_store.write_documents(docs['documents'])
# Initialize some haystack text embedder, in this case the SentenceTransformersTextEmbedder
embedder = SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
# Initialize the hybrid retriever
retriever = OpenSearchHybridRetriever(
document_store=doc_store,
embedder=embedder,
top_k_bm25=3,
top_k_embedding=3,
join_mode="reciprocal_rank_fusion"
)
# Run the retriever
results = retriever.run(query="What is reinforcement learning?", filters_bm25=None, filters_embedding=None)
>> results['documents']
{'documents': [Document(id=..., content: 'Reinforcement learning is a type of machine learning.', score: 1.0),
Document(id=..., content: 'Supervised learning is a type of machine learning.', score: 0.9760624679979518),
Document(id=..., content: 'Deep learning is a subset of machine learning.', score: 0.4919354838709677),
Document(id=..., content: 'Machine learning is a subset of artificial intelligence.', score: 0.4841269841269841)]}
```
---
// File: pipeline-components/retrievers/opensearchmetadataretriever
# OpenSearchMetadataRetriever
Searches and ranks the metadata fields of documents stored in an OpenSearch Document Store and returns the matching metadata values.
`embedder`: Any [Embedder](../embedders.mdx) implementing the `TextEmbedder` protocol | | Mandatory run variables | `query`: A query string | | Output variables | `documents`: A list of documents matching the query | | API reference | [OpenSearch](/reference/integrations-opensearch) | | GitHub | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opensearch |
| | |
| --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| **Most common position in a pipeline** | The last component in a metadata lookup pipeline, or wherever you need other structured data from an OpenSearchDocumentStore index |
| **Mandatory init variables** | `document_store`: An instance of `OpenSearchDocumentStore`; `metadata_fields`: List of metadata field names to search and return |
| **Mandatory run variables** | `query`: A search query string (may contain comma-separated parts) |
| **Output variables** | `metadata`: A list of dictionaries containing only the requested metadata fields |
| **API reference** | [OpenSearch](/reference/integrations-opensearch) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opensearch |
| **Package name** | `opensearch-haystack` |
## Overview
`OpenSearchMetadataRetriever` searches the metadata of documents stored in an `OpenSearchDocumentStore` and returns the matching metadata values, not the documents themselves. It is useful when the metadata is the answer: for example, listing the categories or tags that match a partial query, building a metadata autocomplete, or surfacing the structured side of an index without pulling back document content.
Unlike the other OpenSearch retrievers (`OpenSearchBM25Retriever`, `OpenSearchEmbeddingRetriever`, `OpenSearchHybridRetriever`), this component does not return `Document` objects. The output is a list under `metadata`, where each entry is a dictionary containing only the fields you listed in `metadata_fields`. Document content and any other metadata are excluded from the result.
The retriever supports two search modes:
- `strict` uses prefix and wildcard matching on the configured metadata fields.
- `fuzzy` (the default) uses fuzzy matching with `dis_max` queries, allowing typos and partial matches.
In both modes, candidate documents are scored server-side with Jaccard similarity on character n-grams (the `jaccard_n` parameter controls the n-gram size), and exact matches receive an additional boost controlled by `exact_match_weight`. Up to 1000 hits are fetched from OpenSearch, and the top `top_k` results are returned.
Both a synchronous `run` method and an asynchronous `run_async` method are available with the same parameters.
### Field types
The matching engine only operates on metadata fields that OpenSearch indexes as text or keyword values. Numeric, boolean, and array-of-non-strings fields are not valid search targets, because prefix, wildcard, and full-text matching do not apply to them. Mixed-type fields, such as a list that combines strings and numbers, are also not supported.
## Installation
If you have Docker set up, the easiest way to run OpenSearch is to pull and run the Docker image.
```bash
docker pull opensearchproject/opensearch:3
docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in a semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of an [OracleDocumentStore](../../document-stores/oracledocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Oracle](/reference/integrations-oracle) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/oracle |
| **Package name** | `oracle-haystack` |
## Overview
The `OracleEmbeddingRetriever` is an embedding-based Retriever compatible with `OracleDocumentStore`. It uses Oracle AI Vector Search to compare query and document embeddings, fetching the most relevant documents based on vector similarity.
When using `OracleEmbeddingRetriever` in a pipeline, make sure embeddings are available for both documents (at index time) and queries (at query time). Use a Document Embedder in your indexing pipeline and a Text Embedder in your query pipeline.
The distance metric (COSINE, EUCLIDEAN, or DOT) is configured on the `OracleDocumentStore`. In addition to `query_embedding`, the retriever accepts `top_k` (maximum documents to return) and `filters` to narrow the search space.
## Installation
To run Oracle Database 23ai locally with Docker:
```shell
docker run -d --name oracle23ai \
-p 1521:1521 \
-e ORACLE_PASSWORD=oracle \
-e ORACLE_INIT_PARAMS=vector_memory_size=512M \
gvenzl/oracle-free:23-slim
```
Install the Oracle integration for Haystack:
```shell
pip install oracle-haystack
```
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
### On its own
This Retriever needs an `OracleDocumentStore` and indexed documents with embeddings to run.
```python
from haystack.utils import Secret
from haystack_integrations.document_stores.oracle import (
OracleDocumentStore,
OracleConnectionConfig,
)
from haystack_integrations.components.retrievers.oracle import OracleEmbeddingRetriever
document_store = OracleDocumentStore(
connection_config=OracleConnectionConfig(
user=Secret.from_env_var("ORACLE_USER"),
password=Secret.from_env_var("ORACLE_PASSWORD"),
dsn=Secret.from_env_var("ORACLE_DSN"),
),
embedding_dim=768,
)
retriever = OracleEmbeddingRetriever(document_store=document_store)
# using a fake vector to keep the example simple
retriever.run(query_embedding=[0.1] * 768)
```
### In a Pipeline
```python
from haystack import Document, Pipeline
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.utils import Secret
from haystack_integrations.document_stores.oracle import (
OracleDocumentStore,
OracleConnectionConfig,
)
from haystack_integrations.components.retrievers.oracle import OracleEmbeddingRetriever
document_store = OracleDocumentStore(
connection_config=OracleConnectionConfig(
user=Secret.from_env_var("ORACLE_USER"),
password=Secret.from_env_var("ORACLE_PASSWORD"),
dsn=Secret.from_env_var("ORACLE_DSN"),
),
embedding_dim=768,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings["documents"],
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component(
"text_embedder",
SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"),
)
query_pipeline.add_component(
"retriever",
OracleEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/oraclekeywordretriever
# OracleKeywordRetriever
A keyword-based Retriever that fetches documents matching a query from the Oracle Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in a keyword search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of an [OracleDocumentStore](../../document-stores/oracledocumentstore.mdx) |
| **Mandatory run variables** | `query`: A string |
| **Output variables** | `documents`: A list of documents matching the query |
| **API reference** | [Oracle](/reference/integrations-oracle) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/oracle |
| **Package name** | `oracle-haystack` |
## Overview
The `OracleKeywordRetriever` is a keyword-based Retriever compatible with `OracleDocumentStore`. It uses Oracle's DBMS_SEARCH full-text index — automatically created when the document store is initialized — to search documents by keyword relevance.
This retriever works without embeddings, making it suitable for keyword-only pipelines or as the keyword branch of a hybrid search pipeline.
In addition to `query`, the retriever accepts `top_k` (maximum documents to return) and `filters` to narrow the search space.
## Installation
To run Oracle Database 23ai locally with Docker:
```shell
docker run -d --name oracle23ai \
-p 1521:1521 \
-e ORACLE_PASSWORD=oracle \
-e ORACLE_INIT_PARAMS=vector_memory_size=512M \
gvenzl/oracle-free:23-slim
```
Install the Oracle integration for Haystack:
```shell
pip install oracle-haystack
```
## Usage
### On its own
This Retriever needs an `OracleDocumentStore` and indexed documents to run.
```python
from haystack.utils import Secret
from haystack_integrations.document_stores.oracle import (
OracleDocumentStore,
OracleConnectionConfig,
)
from haystack_integrations.components.retrievers.oracle import OracleKeywordRetriever
document_store = OracleDocumentStore(
connection_config=OracleConnectionConfig(
user=Secret.from_env_var("ORACLE_USER"),
password=Secret.from_env_var("ORACLE_PASSWORD"),
dsn=Secret.from_env_var("ORACLE_DSN"),
),
embedding_dim=768,
)
retriever = OracleKeywordRetriever(document_store=document_store)
retriever.run(query="my keyword query")
```
### In a RAG pipeline
```python
from haystack import Document, Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy
from haystack.utils import Secret
from haystack_integrations.document_stores.oracle import (
OracleDocumentStore,
OracleConnectionConfig,
)
from haystack_integrations.components.retrievers.oracle import OracleKeywordRetriever
prompt_template = [
ChatMessage.from_user(
"""
Given these documents, answer the question.\nDocuments:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
\nQuestion: {{question}}
\nAnswer:
""",
),
]
document_store = OracleDocumentStore(
connection_config=OracleConnectionConfig(
user=Secret.from_env_var("ORACLE_USER"),
password=Secret.from_env_var("ORACLE_PASSWORD"),
dsn=Secret.from_env_var("ORACLE_DSN"),
),
embedding_dim=768,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_store.write_documents(documents=documents, policy=DuplicatePolicy.SKIP)
retriever = OracleKeywordRetriever(document_store=document_store)
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="retriever", instance=retriever)
rag_pipeline.add_component(
instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
question = "How many languages are there?"
result = rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
},
)
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/retrievers/pgvectorembeddingretriever
# PgvectorEmbeddingRetriever
An embedding-based Retriever compatible with the Pgvector Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [PgvectorDocumentStore](../../document-stores/pgvectordocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Pgvector](/reference/integrations-pgvector) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/pgvector |
| **Package name** | `pgvector-haystack` |
## Overview
The `PgvectorEmbeddingRetriever` is an embedding-based Retriever compatible with the `PgvectorDocumentStore`. It compares the query and Document embeddings and fetches the Documents most relevant to the query from the `PgvectorDocumentStore` based on the outcome.
When using the `PgvectorEmbeddingRetriever` in your Pipeline, make sure it has the query and Document embeddings available. You can do so by adding a Document Embedder to your indexing Pipeline and a Text Embedder to your query Pipeline.
In addition to the `query_embedding`, the `PgvectorEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
Some relevant parameters that impact the embedding retrieval must be defined when the corresponding `PgvectorDocumentStore` is initialized: these include embedding dimension, vector function, and some others related to the search strategy (exact nearest neighbor or HNSW).
## Installation
To quickly set up a PostgreSQL database with pgvector, you can use Docker:
```shell
docker run -d -p 5432:5432 -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=postgres pgvector/pgvector:pg17
```
For more information on installing pgvector, visit the [pgvector GitHub repository](https://github.com/pgvector/pgvector).
To use pgvector with Haystack, install the `pgvector-haystack` integration:
```shell
pip install pgvector-haystack
```
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
### On its own
This Retriever needs the `PgvectorDocumentStore` and indexed Documents to run.
```python
import os
from haystack_integrations.document_stores.pgvector import PgvectorDocumentStore
from haystack_integrations.components.retrievers.pgvector import (
PgvectorEmbeddingRetriever,
)
os.environ["PG_CONN_STR"] = "postgresql://postgres:postgres@localhost:5432/postgres"
document_store = PgvectorDocumentStore()
retriever = PgvectorEmbeddingRetriever(document_store=document_store)
# using a fake vector to keep the example simple
retriever.run(query_embedding=[0.1] * 768)
```
### In a Pipeline
```python
import os
from haystack.document_stores.types import DuplicatePolicy
from haystack import Document, Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack_integrations.document_stores.pgvector import PgvectorDocumentStore
from haystack_integrations.components.retrievers.pgvector import (
PgvectorEmbeddingRetriever,
)
os.environ["PG_CONN_STR"] = "postgresql://postgres:postgres@localhost:5432/postgres"
document_store = PgvectorDocumentStore(
embedding_dimension=768,
vector_function="cosine_similarity",
recreate_table=True,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings.get("documents"),
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
PgvectorEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/pgvectorkeywordretriever
# PgvectorKeywordRetriever
This is a keyword-based Retriever that fetches documents matching a query from the Pgvector Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [PgvectorDocumentStore](../../document-stores/pgvectordocumentstore.mdx) |
| **Mandatory run variables** | `query`: A string |
| **Output variables** | `documents`: A list of documents (matching the query) |
| **API reference** | [Pgvector](/reference/integrations-pgvector) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/pgvector |
| **Package name** | `pgvector-haystack` |
## Overview
The `PgvectorKeywordRetriever` is a keyword-based Retriever compatible with the `PgvectorDocumentStore`.
The component uses the `ts_rank_cd` function of PostgreSQL to rank the documents.
It considers how often the query terms appear in the document, how close together the terms are in the document, and how important is the part of the document where they occur.
For more details, see [Postgres documentation](https://www.postgresql.org/docs/current/textsearch-controls.html#TEXTSEARCH-RANKING).
Keep in mind that, unlike similar components such as `ElasticsearchBM25Retriever`, this Retriever does not apply fuzzy search out of the box, so it’s necessary to carefully formulate the query in order to avoid getting zero results.
In addition to the `query`, the `PgvectorKeywordRetriever` accepts other optional parameters, including `top_k` (the maximum number of documents to retrieve) and `filters` to narrow the search space.
### Installation
To quickly set up a PostgreSQL database with pgvector, you can use Docker:
```shell
docker run -d -p 5432:5432 -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=postgres pgvector/pgvector:pg17
```
For more information on how to install pgvector, visit the [pgvector GitHub repository](https://github.com/pgvector/pgvector).
Install the `pgvector-haystack` integration:
```shell
pip install pgvector-haystack
```
## Usage
### On its own
This Retriever needs the `PgvectorDocumentStore` and indexed documents to run.
Set an environment variable `PG_CONN_STR` with the connection string to your PostgreSQL database.
```python
from haystack_integrations.document_stores.pgvector import PgvectorDocumentStore
from haystack_integrations.components.retrievers.pgvector import (
PgvectorKeywordRetriever,
)
document_store = PgvectorDocumentStore()
retriever = PgvectorKeywordRetriever(document_store=document_store)
retriever.run(query="my nice query")
```
### In a RAG pipeline
The prerequisites necessary for running this code are:
- Set an environment variable `OPENAI_API_KEY` with your OpenAI API key.
- Set an environment variable `PG_CONN_STR` with the connection string to your PostgreSQL database.
```python
from haystack import Document
from haystack import Pipeline
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.document_stores.pgvector import PgvectorDocumentStore
from haystack_integrations.components.retrievers.pgvector import (
PgvectorKeywordRetriever,
)
# Create a RAG query pipeline
prompt_template = [
ChatMessage.from_user(
"""
Given these documents, answer the question.\nDocuments:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
\nQuestion: {{question}}
\nAnswer:
""",
),
]
document_store = PgvectorDocumentStore(
language="english", # this parameter influences text parsing for keyword retrieval
recreate_table=True,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
# DuplicatePolicy.SKIP param is optional, but useful to run the script multiple times without throwing errors
document_store.write_documents(documents=documents, policy=DuplicatePolicy.SKIP)
retriever = PgvectorKeywordRetriever(document_store=document_store)
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="retriever", instance=retriever)
rag_pipeline.add_component(
instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")
question = "languages spoken around the world today"
result = rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
"answer_builder": {"query": question},
},
)
print(result["answer_builder"])
```
---
// File: pipeline-components/retrievers/pineconedenseretriever
# PineconeEmbeddingRetriever
An embedding-based Retriever compatible with the Pinecone Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [PineconeDocumentStore](../../document-stores/pinecone-document-store.mdx) |
| **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Pinecone](/reference/integrations-pinecone) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/pinecone |
| **Package name** | `pinecone-haystack` |
## Overview
The `PineconeEmbeddingRetriever` is an embedding-based Retriever compatible with the `PineconeDocumentStore`. It compares the query and Document embeddings and fetches the Documents most relevant to the query from the `PineconeDocumentStore` based on the outcome.
When using the `PineconeEmbeddingRetriever` in your NLP system, make sure it has the query and Document embeddings available. You can do so by adding a Document Embedder to your indexing Pipeline and a Text Embedder to your query Pipeline.
In addition to the `query_embedding`, the `PineconeEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
Some relevant parameters that impact the embedding retrieval must be defined when the corresponding `PineconeDocumentStore` is initialized: these include the `dimension` of the embeddings and the distance `metric` to use.
## Usage
### On its own
This Retriever needs the `PineconeDocumentStore` and indexed Documents to run.
```python
from haystack_integrations.components.retrievers.pinecone import (
PineconeEmbeddingRetriever,
)
from haystack_integrations.document_stores.pinecone import PineconeDocumentStore
# Make sure you have the PINECONE_API_KEY environment variable set
document_store = PineconeDocumentStore(
index="my_index_with_documents",
namespace="my_namespace",
dimension=768,
)
retriever = PineconeEmbeddingRetriever(document_store=document_store)
# using an imaginary vector to keep the example simple, example run query:
retriever.run(query_embedding=[0.1] * 768)
```
### In a pipeline
Install the dependencies you’ll need:
```shell
pip install pinecone-haystack
pip install sentence-transformers-haystack
```
Use this Retriever in a query Pipeline like this:
```python
from haystack.document_stores.types import DuplicatePolicy
from haystack import Document
from haystack import Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack_integrations.components.retrievers.pinecone import (
PineconeEmbeddingRetriever,
)
from haystack_integrations.document_stores.pinecone import PineconeDocumentStore
# Make sure you have the PINECONE_API_KEY environment variable set
document_store = PineconeDocumentStore(
index="my_index",
namespace="my_namespace",
dimension=768,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings.get("documents"),
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
PineconeEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
The example output would be:
```python
Document(id=cfe93bc1c274908801e6670440bf2bbba54fad792770d57421f85ffa2a4fcc94, content: 'There are over 7,000 languages spoken around the world today.', score: 0.87717235, embedding: vector of size 768)
```
---
// File: pipeline-components/retrievers/qdrantembeddingretriever
# QdrantEmbeddingRetriever
An embedding-based Retriever compatible with the Qdrant Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1\. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG Pipeline
2. The last component in the semantic search pipeline
3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx) | | **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) | | **Output variables** | `documents`: A list of documents | | **API reference** | [Qdrant](/reference/integrations-qdrant) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/qdrant | | **Package name** | `qdrant-haystack` |
## Overview
The `QdrantEmbeddingRetriever` is an embedding-based Retriever compatible with the `QdrantDocumentStore`. It compares the query and Document embeddings and fetches the Documents most relevant to the query from the `QdrantDocumentStore` based on the outcome.
When using the `QdrantEmbeddingRetriever` in your NLP system, make sure it has the query and Document embeddings available. You can add a Document Embedder to your indexing Pipeline and a Text Embedder to your query Pipeline.
In addition to the `query_embedding`, the `QdrantEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
Some relevant parameters that impact the embedding retrieval must be defined when the corresponding `QdrantDocumentStore` is initialized: these include the embedding dimension (`embedding_dim`), the `similarity` function to use when comparing embeddings and the HNSW configuration (`hnsw_config`).
### Installation
To start using Qdrant with Haystack, first install the package with:
```shell
pip install qdrant-haystack
```
### Usage
#### On its own
This Retriever needs the `QdrantDocumentStore` and indexed Documents to run.
```python
from haystack_integrations.components.retrievers.qdrant import QdrantEmbeddingRetriever
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
document_store = QdrantDocumentStore(
":memory:",
recreate_index=True,
return_embedding=True,
wait_result_from_api=True,
)
retriever = QdrantEmbeddingRetriever(document_store=document_store)
# using a fake vector to keep the example simple
retriever.run(query_embedding=[0.1] * 768)
```
#### In a Pipeline
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack.document_stores.types import DuplicatePolicy
from haystack import Document
from haystack import Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack_integrations.components.retrievers.qdrant import QdrantEmbeddingRetriever
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
document_store = QdrantDocumentStore(
":memory:",
recreate_index=True,
return_embedding=True,
wait_result_from_api=True,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings.get("documents"),
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
QdrantEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/qdranthybridretriever
# QdrantHybridRetriever
A Retriever based both on dense and sparse embeddings, compatible with the Qdrant Document Store.
2. The last component in the semantic search pipeline
3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx) | | **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) | | **Output variables** | `documents`: A list of documents | | **API reference** | [Qdrant](/reference/integrations-qdrant) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/qdrant | | **Package name** | `qdrant-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | 1\. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline
2. The last component in a hybrid search pipeline
3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx) | | **Mandatory run variables** | `query_embedding`: A dense vector representing the query (a list of floats)
`query_sparse_embedding`: A [`SparseEmbedding`](../../concepts/data-classes.mdx#sparseembedding) object containing a vectorial representation of the query | | **Output variables** | `documents`: A list of documents | | **API reference** | [Qdrant](/reference/integrations-qdrant) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/qdrant | | **Package name** | `qdrant-haystack` |
## Overview
The `QdrantHybridRetriever` is a Retriever based both on dense and sparse embeddings, compatible with the [`QdrantDocumentStore`](../../document-stores/qdrant-document-store.mdx).
It compares the query and document’s dense and sparse embeddings and fetches the documents most relevant to the query from the `QdrantDocumentStore`, fusing the scores with Reciprocal Rank Fusion.
:::tip[Hybrid Retrieval Pipeline]
If you want additional customization for merging or fusing results, consider creating a hybrid retrieval pipeline with [`DocumentJoiner`](../joiners/documentjoiner.mdx).
You can check out our hybrid retrieval pipeline [tutorial](https://haystack.deepset.ai/tutorials/33_hybrid_retrieval) for detailed steps.
:::
When using the `QdrantHybridRetriever`, make sure it has the query and document with dense and sparse embeddings available. You can do so by:
- Adding a (dense) document Embedder and a sparse document Embedder to your indexing pipeline,
- Adding a (dense) text Embedder and a sparse text Embedder to your query pipeline.
In addition to `query_embedding` and `query_sparse_embedding`, the `QdrantHybridRetriever` accepts other optional parameters, including `top_k` (the maximum number of documents to retrieve) and `filters` to narrow down the search space.
:::note[Sparse Embedding Support]
To use Sparse Embedding support, you need to initialize the `QdrantDocumentStore` with `use_sparse_embeddings=True`, which is `False` by default.
If you want to use Document Store or collection previously created with this feature disabled, you must migrate the existing data. You can do this by taking advantage of the `migrate_to_sparse_embeddings_support` utility function.
:::
### Installation
To start using Qdrant with Haystack, first install the package with:
```shell
pip install qdrant-haystack
```
## Usage
### On its own
```python
from haystack_integrations.components.retrievers.qdrant import QdrantHybridRetriever
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
from haystack.dataclasses import Document, SparseEmbedding
document_store = QdrantDocumentStore(
":memory:",
use_sparse_embeddings=True,
recreate_index=True,
return_embedding=True,
wait_result_from_api=True,
)
doc = Document(
content="test",
embedding=[0.5] * 768,
sparse_embedding=SparseEmbedding(indices=[0, 3, 5], values=[0.1, 0.5, 0.12]),
)
document_store.write_documents([doc])
retriever = QdrantHybridRetriever(document_store=document_store)
embedding = [0.1] * 768
sparse_embedding = SparseEmbedding(indices=[0, 1, 2, 3], values=[0.1, 0.8, 0.05, 0.33])
retriever.run(query_embedding=embedding, query_sparse_embedding=sparse_embedding)
```
### In a pipeline
Currently, you can compute sparse embeddings using Fastembed Sparse Embedders.
First, install the package with:
```shell
pip install fastembed-haystack
```
In the example below, we are using Fastembed Embedders to compute dense embeddings as well.
```python
from haystack import Document, Pipeline
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.retrievers.qdrant import QdrantHybridRetriever
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.fastembed import (
FastembedTextEmbedder,
FastembedDocumentEmbedder,
FastembedSparseTextEmbedder,
FastembedSparseDocumentEmbedder,
)
document_store = QdrantDocumentStore(
":memory:",
recreate_index=True,
use_sparse_embeddings=True,
embedding_dim=384,
)
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
Document(content="fastembed is supported by and maintained by Qdrant."),
]
indexing = Pipeline()
indexing.add_component(
"sparse_doc_embedder",
FastembedSparseDocumentEmbedder(model="prithvida/Splade_PP_en_v1"),
)
indexing.add_component(
"dense_doc_embedder",
FastembedDocumentEmbedder(model="BAAI/bge-small-en-v1.5"),
)
indexing.add_component(
"writer",
DocumentWriter(document_store=document_store, policy=DuplicatePolicy.OVERWRITE),
)
indexing.connect("sparse_doc_embedder", "dense_doc_embedder")
indexing.connect("dense_doc_embedder", "writer")
indexing.run({"sparse_doc_embedder": {"documents": documents}})
querying = Pipeline()
querying.add_component(
"sparse_text_embedder",
FastembedSparseTextEmbedder(model="prithvida/Splade_PP_en_v1"),
)
querying.add_component(
"dense_text_embedder",
FastembedTextEmbedder(
model="BAAI/bge-small-en-v1.5",
prefix="Represent this sentence for searching relevant passages: ",
),
)
querying.add_component(
"retriever",
QdrantHybridRetriever(document_store=document_store),
)
querying.connect(
"sparse_text_embedder.sparse_embedding",
"retriever.query_sparse_embedding",
)
querying.connect("dense_text_embedder.embedding", "retriever.query_embedding")
question = "Who supports fastembed?"
results = querying.run(
{
"dense_text_embedder": {"text": question},
"sparse_text_embedder": {"text": question},
},
)
print(results["retriever"]["documents"][0])
# Document(id=...,
# content: 'fastembed is supported by and maintained by Qdrant.',
# score: 1.0)
```
## Additional References
:notebook: Tutorial: [Creating a Hybrid Retrieval Pipeline](https://haystack.deepset.ai/tutorials/33_hybrid_retrieval)
🧑🍳 Cookbook: [Sparse Embedding Retrieval with Qdrant and FastEmbed](https://haystack.deepset.ai/cookbook/sparse_embedding_retrieval)
---
// File: pipeline-components/retrievers/qdrantsparseembeddingretriever
# QdrantSparseEmbeddingRetriever
A Retriever based on sparse embeddings, compatible with the Qdrant Document Store.
2. The last component in a hybrid search pipeline
3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx) | | **Mandatory run variables** | `query_embedding`: A dense vector representing the query (a list of floats)
`query_sparse_embedding`: A [`SparseEmbedding`](../../concepts/data-classes.mdx#sparseembedding) object containing a vectorial representation of the query | | **Output variables** | `documents`: A list of documents | | **API reference** | [Qdrant](/reference/integrations-qdrant) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/qdrant | | **Package name** | `qdrant-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | 1\. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline
2. The last component in the semantic search pipeline
3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx) | | **Mandatory run variables** | `query_sparse_embedding`: A [`SparseEmbedding`](../../concepts/data-classes.mdx#sparseembedding) object containing a vectorial representation of the query | | **Output variables** | `documents`: A list of documents | | **API reference** | [Qdrant](/reference/integrations-qdrant) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/qdrant | | **Package name** | `qdrant-haystack` |
## Overview
The `QdrantSparseEmbeddingRetriever` is a Retriever based on sparse embeddings, compatible with the [`QdrantDocumentStore`](../../document-stores/qdrant-document-store.mdx).
It compares the query and document sparse embeddings and, based on the outcome, fetches the documents most relevant to the query from the `QdrantDocumentStore`.
When using the `QdrantSparseEmbeddingRetriever`, make sure it has the query and document sparse embeddings available. You can do so by adding a sparse document Embedder to your indexing pipeline and a sparse text Embedder to your query pipeline.
In addition to the `query_sparse_embedding`, the `QdrantSparseEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of documents to retrieve) and `filters` to narrow down the search space.
:::note[Sparse Embedding Support]
To use Sparse Embedding support, you need to initialize the `QdrantDocumentStore` with `use_sparse_embeddings=True`, which is `False` by default.
If you want to use Document Store or collection previously created with this feature disabled, you must migrate the existing data. You can do this by taking advantage of the `migrate_to_sparse_embeddings_support` utility function.
:::
### Installation
To start using Qdrant with Haystack, first install the package with:
```shell
pip install qdrant-haystack
```
## Usage
### On its own
This Retriever needs the `QdrantDocumentStore` and indexed documents to run.
```python
from haystack_integrations.components.retrievers.qdrant import (
QdrantSparseEmbeddingRetriever,
)
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
from haystack.dataclasses import Document, SparseEmbedding
document_store = QdrantDocumentStore(
":memory:",
use_sparse_embeddings=True,
recreate_index=True,
return_embedding=True,
)
doc = Document(
content="test",
sparse_embedding=SparseEmbedding(indices=[0, 3, 5], values=[0.1, 0.5, 0.12]),
)
document_store.write_documents([doc])
retriever = QdrantSparseEmbeddingRetriever(document_store=document_store)
sparse_embedding = SparseEmbedding(indices=[0, 1, 2, 3], values=[0.1, 0.8, 0.05, 0.33])
retriever.run(query_sparse_embedding=sparse_embedding)
```
### In a pipeline
In Haystack, you can compute sparse embeddings using Fastembed Embedders.
First, install the package with:
```shell
pip install fastembed-haystack
```
Then, try out this pipeline:
```python
from haystack import Document, Pipeline
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.retrievers.qdrant import (
QdrantSparseEmbeddingRetriever,
)
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.fastembed import (
FastembedSparseDocumentEmbedder,
FastembedSparseTextEmbedder,
)
document_store = QdrantDocumentStore(
":memory:",
recreate_index=True,
use_sparse_embeddings=True,
)
documents = [
Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities"),
Document(content="fastembed is supported by and maintained by Qdrant."),
]
sparse_document_embedder = FastembedSparseDocumentEmbedder()
writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.OVERWRITE)
indexing_pipeline = Pipeline()
indexing_pipeline.add_component("sparse_document_embedder", sparse_document_embedder)
indexing_pipeline.add_component("writer", writer)
indexing_pipeline.connect("sparse_document_embedder", "writer")
indexing_pipeline.run({"sparse_document_embedder": {"documents": documents}})
query_pipeline = Pipeline()
query_pipeline.add_component("sparse_text_embedder", FastembedSparseTextEmbedder())
query_pipeline.add_component(
"sparse_retriever",
QdrantSparseEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect(
"sparse_text_embedder.sparse_embedding",
"sparse_retriever.query_sparse_embedding",
)
query = "Who supports fastembed?"
result = query_pipeline.run({"sparse_text_embedder": {"text": query}})
print(result["sparse_retriever"]["documents"][0]) # noqa: T201
# Document(id=...,
# content: 'fastembed is supported by and maintained by Qdrant.',
# score: 24.882490158081055)
```
## Additional References
🧑🍳 Cookbook: [Sparse Embedding Retrieval with Qdrant and FastEmbed](https://haystack.deepset.ai/cookbook/sparse_embedding_retrieval)
---
// File: pipeline-components/retrievers/sentencewindowretriever
# SentenceWindowRetriever
Use this component to retrieve neighboring sentences around relevant sentences to get the full context.
2. The last component in the semantic search pipeline
3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx) | | **Mandatory run variables** | `query_sparse_embedding`: A [`SparseEmbedding`](../../concepts/data-classes.mdx#sparseembedding) object containing a vectorial representation of the query | | **Output variables** | `documents`: A list of documents | | **API reference** | [Qdrant](/reference/integrations-qdrant) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/qdrant | | **Package name** | `qdrant-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Used after the main Retriever component, like the `InMemoryEmbeddingRetriever` or any other Retriever. |
| **Mandatory init variables** | `document_store`: An instance of a Document Store |
| **Mandatory run variables** | `retrieved_documents`: A list of already retrieved documents for which you want to get a context window |
| **Output variables** | `context_windows`: A list of strings
`context_documents`: A list of documents ordered by `split_idx_start` | | **API reference** | [Retrievers](/reference/retrievers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/sentence_window_retriever.py | | **Package name** | `haystack-ai` |
## Overview
The "sentence window" is a retrieval technique that allows for the retrieval of the context around relevant sentences.
During indexing, documents are broken into smaller chunks or sentences and indexed. During retrieval, the sentences most relevant to a given query, based on a certain similarity metric, are retrieved.
Once we have the relevant sentences, we can retrieve neighboring sentences to provide full context. The number of neighboring sentences to retrieve is defined by a fixed number of sentences before and after the relevant sentence.
This component is meant to be used with other Retrievers, such as the `InMemoryEmbeddingRetriever`. These Retrievers find relevant sentences by comparing a query against indexed sentences using a similarity metric. Then, the `SentenceWindowRetriever` component retrieves neighboring sentences around the relevant ones by leveraging metadata stored in the `Document` object.
## Usage
### On its own
```python
splitter = DocumentSplitter(split_length=10, split_overlap=5, split_by="word")
text = (
"This is a text with some words. There is a second sentence. And there is also a third sentence. "
"It also contains a fourth sentence. And a fifth sentence. And a sixth sentence. And a seventh sentence"
)
doc = Document(content=text)
docs = splitter.run([doc])
doc_store = InMemoryDocumentStore()
doc_store.write_documents(docs["documents"])
retriever = SentenceWindowRetriever(document_store=doc_store, window_size=3)
```
### In a Pipeline
```python
from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.retrievers import SentenceWindowRetriever
from haystack.components.preprocessors import DocumentSplitter
from haystack.document_stores.in_memory import InMemoryDocumentStore
splitter = DocumentSplitter(split_length=10, split_overlap=5, split_by="word")
text = (
"This is a text with some words. There is a second sentence. And there is also a third sentence. "
"It also contains a fourth sentence. And a fifth sentence. And a sixth sentence. And a seventh sentence"
)
doc = Document(content=text)
docs = splitter.run([doc])
doc_store = InMemoryDocumentStore()
doc_store.write_documents(docs["documents"])
rag = Pipeline()
rag.add_component("bm25_retriever", InMemoryBM25Retriever(doc_store, top_k=1))
rag.add_component(
"sentence_window_retriever",
SentenceWindowRetriever(document_store=doc_store, window_size=3),
)
rag.connect("bm25_retriever", "sentence_window_retriever")
rag.run({"bm25_retriever": {"query": "third"}})
```
## Additional References
:notebook: Tutorial: [Retrieving a Context Window Around a Sentence](https://haystack.deepset.ai/tutorials/42_sentence_window_retriever)
---
// File: pipeline-components/retrievers/snowflaketableretriever
# SnowflakeTableRetriever
Connects to a Snowflake database to execute an SQL query.
`context_documents`: A list of documents ordered by `split_idx_start` | | **API reference** | [Retrievers](/reference/retrievers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/sentence_window_retriever.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `user`: User's login
`account`: Snowflake account identifier
`api_key`: Snowflake account password. Can be set with `SNOWFLAKE_API_KEY` env var | | **Mandatory run variables** | `query`: An SQL query to execute | | **Output variables** | `dataframe`: The resulting Pandas dataframe version of the table
`table`: The same result as a Markdown-formatted string | | **API reference** | [Snowflake](/reference/integrations-snowflake) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/snowflake | | **Package name** | `snowflake-haystack` |
## Overview
The `SnowflakeTableRetriever` connects to a Snowflake database and retrieves data using an SQL query. It then returns a Pandas dataframe and a Markdown version of the table:
To start using the integration, install it with:
```bash
pip install snowflake-haystack
```
## Usage
### On its own
```python
from haystack.utils import Secret
from haystack_integrations.components.retrievers.snowflake import (
SnowflakeTableRetriever,
)
snowflake = SnowflakeTableRetriever(
user="`account`: Snowflake account identifier
`api_key`: Snowflake account password. Can be set with `SNOWFLAKE_API_KEY` env var | | **Mandatory run variables** | `query`: An SQL query to execute | | **Output variables** | `dataframe`: The resulting Pandas dataframe version of the table
`table`: The same result as a Markdown-formatted string | | **API reference** | [Snowflake](/reference/integrations-snowflake) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/snowflake | | **Package name** | `snowflake-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `drivername`: SQLAlchemy driver name, for example `sqlite`, `postgresql+psycopg2`, `mysql+pymysql`, or `mssql+pyodbc`. For real database backends you will also need `host`, `port`, `database`, `username`, and `password`. |
| **Mandatory run variables** | `query`: An SQL query to execute |
| **Output variables** | `dataframe`: The query result as a Pandas DataFrame
`table`: The same result rendered as a Markdown table
`error`: Error message if the query failed, empty string otherwise | | **API reference** | [SQLAlchemy](/reference/integrations-sqlalchemy) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sqlalchemy | | **Package name** | `sqlalchemy-haystack` |
## Overview
`SQLAlchemyTableRetriever` is a backend-agnostic table retriever: it speaks to anything SQLAlchemy speaks to — PostgreSQL, MySQL, SQLite, MSSQL, and the long tail of dialects covered by third-party drivers. Give it a SQL query and it hands you back the result as both a Pandas DataFrame (under `dataframe`) and a ready-to-render Markdown table (under `table`), which is convenient for piping into a prompt. Results are capped at 10,000 rows.
If the query fails, the component does not raise — it returns an empty DataFrame and puts the SQLAlchemy error string in the `error` output. That makes it safe to drop into a pipeline without wrapping the whole thing in a try/except.
### Connection parameters
The init arguments map directly to SQLAlchemy's URL parts:
- `drivername` — the only strictly required one. Pick the driver that matches your backend, e.g. `postgresql+psycopg2`, `mysql+pymysql`, `sqlite`, `mssql+pyodbc`.
- `host`, `port`, `database`, `username` — standard connection bits. Pass whatever your backend needs.
- `password` — a Haystack [Secret](../../concepts/secret-management.mdx). Resolve it from an environment variable with `Secret.from_env_var("MY_DB_PASSWORD")`, or inline with `Secret.from_token("…")` (not recommended for anything other than local tinkering).
For SQLite, `drivername="sqlite"` with `database=":memory:"` is enough — no host/user/password needed.
### `init_script`
Pass `init_script` to run one or more SQL statements once, in a single transaction, the first time the component is warmed up. Typical uses:
- Seeding an in-memory SQLite database for demos or tests.
- Creating temporary views or session-level settings before queries run.
Each entry in the list is a single statement.
## Usage
Install the `sqlalchemy-haystack` package, plus the driver for your database:
```shell
pip install sqlalchemy-haystack
# For PostgreSQL, also install a driver:
pip install psycopg2-binary
```
### On its own
A self-contained example using an in-memory SQLite database seeded via `init_script`:
```python
from haystack_integrations.components.retrievers.sqlalchemy import (
SQLAlchemyTableRetriever,
)
retriever = SQLAlchemyTableRetriever(
drivername="sqlite",
database=":memory:",
init_script=[
"CREATE TABLE employees (name TEXT, salary INTEGER)",
"INSERT INTO employees VALUES ('Ada', 90000), ('Linus', 85000), ('Grace', 95000)",
],
)
result = retriever.run(query="SELECT name, salary FROM employees ORDER BY salary DESC")
print(result["dataframe"])
print(result["table"])
```
Connecting to a real backend looks the same — swap the driver and pass connection details:
```python
from haystack.utils import Secret
from haystack_integrations.components.retrievers.sqlalchemy import (
SQLAlchemyTableRetriever,
)
retriever = SQLAlchemyTableRetriever(
drivername="postgresql+psycopg2",
host="db.example.com",
port=5432,
database="analytics",
username="readonly",
password=Secret.from_env_var("ANALYTICS_DB_PASSWORD"),
)
```
### In a pipeline
Use the retriever's Markdown `table` output as context for an LLM — for example, asking an LLM to summarize a query result:
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.retrievers.sqlalchemy import (
SQLAlchemyTableRetriever,
)
retriever = SQLAlchemyTableRetriever(
drivername="postgresql+psycopg2",
host="db.example.com",
port=5432,
database="analytics",
username="readonly",
password=Secret.from_env_var("ANALYTICS_DB_PASSWORD"),
)
pipeline = Pipeline()
pipeline.add_component(
"builder",
ChatPromptBuilder(
template=[ChatMessage.from_user("Describe this table: {{ table }}")],
required_variables="*",
),
)
pipeline.add_component("db", retriever)
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o"))
pipeline.connect("db.table", "builder.table")
pipeline.connect("builder.prompt", "llm.messages")
pipeline.run(data={"query": "SELECT employee, salary FROM employees LIMIT 10"})
```
---
// File: pipeline-components/retrievers/supabasegroongabm25retriever
# SupabaseGroongaBM25Retriever
A full-text Retriever that fetches documents from the SupabaseGroongaDocumentStore using PGroonga search.
`table`: The same result rendered as a Markdown table
`error`: Error message if the query failed, empty string otherwise | | **API reference** | [SQLAlchemy](/reference/integrations-sqlalchemy) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sqlalchemy | | **Package name** | `sqlalchemy-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the full-text search pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [SupabaseGroongaDocumentStore](../../document-stores/supabasedocumentstore.mdx) |
| **Mandatory run variables** | `query`: A string |
| **Output variables** | `documents`: A list of documents (matching the query) |
| **API reference** | [Supabase](/reference/integrations-supabase) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/supabase |
| **Package name** | `supabase-haystack` |
## Overview
`SupabaseGroongaBM25Retriever` retrieves Documents from the `SupabaseGroongaDocumentStore` using [PGroonga](https://pgroonga.github.io/), a PostgreSQL extension for fast, multilingual full-text search.
Unlike embedding-based retrievers, this Retriever works with plain text queries and requires no embeddings. It supports a wide range of languages out of the box through PGroonga's multilingual indexing capabilities.
The Retriever can be combined with `SupabasePgvectorEmbeddingRetriever` and a [`DocumentJoiner`](../joiners/documentjoiner.mdx) for hybrid search pipelines that take advantage of both keyword and semantic retrieval.
You can also use of the [Smart Pipeline Connections](https://docs.haystack.deepset.ai/docs/smart-pipeline-connections) and skip the `DocumentJoiner` if you want to combine the results of both retrievers in a RAG pipeline.
In addition to `query`, the Retriever accepts optional parameters including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow the search space.
## Prerequisites
PGroonga must be enabled in your Supabase project. Run the following SQL in the Supabase SQL editor:
```sql
CREATE EXTENSION IF NOT EXISTS pgroonga;
```
You also need to create a SQL function that PGroonga uses for search. See the [integration README](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/supabase/) for the required function definition.
## Installation
```shell
pip install supabase-haystack
```
## Usage
### On its own
This Retriever needs the `SupabaseGroongaDocumentStore` and indexed Documents to run.
Set the `SUPABASE_URL` and `SUPABASE_SERVICE_KEY` environment variables for your Supabase project.
```python
from haystack_integrations.document_stores.supabase import SupabaseGroongaDocumentStore
from haystack_integrations.components.retrievers.supabase import (
SupabaseGroongaBM25Retriever,
)
from haystack.utils import Secret
document_store = SupabaseGroongaDocumentStore(
supabase_url="https://
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [SupabasePgvectorDocumentStore](../../document-stores/supabasedocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Supabase](/reference/integrations-supabase) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/supabase |
| **Package name** | `supabase-haystack` |
## Overview
`SupabasePgvectorEmbeddingRetriever` is a thin wrapper around [`PgvectorEmbeddingRetriever`](pgvectorembeddingretriever.mdx), adapted for use with `SupabasePgvectorDocumentStore`. It compares the query and Document embeddings and fetches the Documents most relevant to the query based on vector similarity.
When using this Retriever in your pipeline, make sure embeddings are available. Add a Document Embedder to your indexing pipeline and a Text Embedder to your query pipeline.
In addition to `query_embedding`, the Retriever accepts optional parameters including `top_k` (the maximum number of Documents to retrieve), `filters` to narrow down the search space, and `vector_function` to override the similarity function set on the Document Store.
Some relevant parameters that impact embedding retrieval must be defined when the `SupabasePgvectorDocumentStore` is initialized: `embedding_dimension`, `vector_function`, and `search_strategy` (`"exact_nearest_neighbor"` or `"hnsw"`).
## Installation
```shell
pip install supabase-haystack
```
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
### On its own
This Retriever needs the `SupabasePgvectorDocumentStore` and indexed Documents to run.
Set the `SUPABASE_DB_URL` environment variable with your Supabase database connection string.
```python
from haystack_integrations.document_stores.supabase import SupabasePgvectorDocumentStore
from haystack_integrations.components.retrievers.supabase import (
SupabasePgvectorEmbeddingRetriever,
)
document_store = SupabasePgvectorDocumentStore(embedding_dimension=768)
retriever = SupabasePgvectorEmbeddingRetriever(document_store=document_store)
# using a fake vector to keep the example simple
retriever.run(query_embedding=[0.1] * 768)
```
### In a Pipeline
```python
from haystack import Document, Pipeline
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack_integrations.document_stores.supabase import SupabasePgvectorDocumentStore
from haystack_integrations.components.retrievers.supabase import (
SupabasePgvectorEmbeddingRetriever,
)
document_store = SupabasePgvectorDocumentStore(
embedding_dimension=768,
vector_function="cosine_similarity",
recreate_table=True,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings.get("documents"),
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
SupabasePgvectorEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/supabasepgvectorkeywordretriever
# SupabasePgvectorKeywordRetriever
A keyword-based Retriever that fetches documents matching a query from the SupabasePgvectorDocumentStore.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [SupabasePgvectorDocumentStore](../../document-stores/supabasedocumentstore.mdx) |
| **Mandatory run variables** | `query`: A string |
| **Output variables** | `documents`: A list of documents (matching the query) |
| **API reference** | [Supabase](/reference/integrations-supabase) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/supabase |
| **Package name** | `supabase-haystack` |
## Overview
`SupabasePgvectorKeywordRetriever` is a thin wrapper around [`PgvectorKeywordRetriever`](pgvectorkeywordretriever.mdx), adapted for use with `SupabasePgvectorDocumentStore`.
It uses PostgreSQL full-text search (`to_tsvector` / `plainto_tsquery`) to find Documents and ranks them with the `ts_rank_cd` function. The ranking considers how often the query terms appear in the Document, how close together the terms are, and how important the part of the Document is where they occur. For more details, see the [PostgreSQL documentation](https://www.postgresql.org/docs/current/textsearch-controls.html#TEXTSEARCH-RANKING).
Keep in mind that, unlike similar components such as `ElasticsearchBM25Retriever`, this Retriever does not apply fuzzy search out of the box, so it's necessary to carefully formulate the query in order to avoid getting zero results.
The language used to parse query and Document content for keyword retrieval is set via the `language` parameter on the `SupabasePgvectorDocumentStore` (defaults to `"english"`).
In addition to the `query`, the Retriever accepts optional parameters including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow the search space.
## Installation
```shell
pip install supabase-haystack
```
## Usage
### On its own
This Retriever needs the `SupabasePgvectorDocumentStore` and indexed Documents to run.
Set the `SUPABASE_DB_URL` environment variable with your Supabase database connection string.
```python
from haystack_integrations.document_stores.supabase import SupabasePgvectorDocumentStore
from haystack_integrations.components.retrievers.supabase import (
SupabasePgvectorKeywordRetriever,
)
document_store = SupabasePgvectorDocumentStore()
retriever = SupabasePgvectorKeywordRetriever(document_store=document_store)
retriever.run(query="my nice query")
```
### In a RAG pipeline
The prerequisites for running this code are:
- Set an environment variable `OPENAI_API_KEY` with your OpenAI API key.
- Set an environment variable `SUPABASE_DB_URL` with the connection string to your Supabase database.
```python
from haystack import Document, Pipeline
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.document_stores.supabase import SupabasePgvectorDocumentStore
from haystack_integrations.components.retrievers.supabase import (
SupabasePgvectorKeywordRetriever,
)
prompt_template = [
ChatMessage.from_user(
"Given these documents, answer the question.\nDocuments:\n"
"{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
"Question: {{question}}\nAnswer:",
),
]
document_store = SupabasePgvectorDocumentStore(
language="english",
recreate_table=True,
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_store.write_documents(documents=documents, policy=DuplicatePolicy.SKIP)
retriever = SupabasePgvectorKeywordRetriever(document_store=document_store)
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="retriever", instance=retriever)
rag_pipeline.add_component(
instance=ChatPromptBuilder(
template=prompt_template,
required_variables={"question", "documents"},
),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")
question = "languages spoken around the world today"
result = rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
"answer_builder": {"query": question},
},
)
print(result["answer_builder"])
```
---
// File: pipeline-components/retrievers/textembeddingretriever
# TextEmbeddingRetriever
Wraps an embedding-based retriever with a text embedder into a single component that accepts a text query.
| | |
| --- | --- |
| **Most common position in a pipeline** | In query pipelines:
In a RAG pipeline, before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx)
In a semantic search pipeline, as the last component
As a retriever inside [`MultiRetriever`](multiretriever.mdx) | | **Mandatory init variables** | `retriever`: An embedding-based Retriever
`text_embedder`: A Text Embedder component | | **Mandatory run variables** | `query`: A query string | | **Output variables** | `documents`: A list of retrieved documents sorted by relevance score | | **API reference** | [Retrievers](/reference/retrievers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/text_embedding_retriever.py | | **Package name** | `haystack-ai` |
## Overview
`TextEmbeddingRetriever` bundles a text embedder and an embedding-based retriever into a single component. It accepts a plain text query, converts it to an embedding internally, and returns documents sorted by relevance score.
You can use it anywhere an embedding-based retriever fits: in RAG pipelines before a prompt builder, as the final component in a semantic search pipeline, or as a drop-in retriever inside [`MultiRetriever`](multiretriever.mdx).
## Usage
### On its own
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.retrievers import (
InMemoryEmbeddingRetriever,
TextEmbeddingRetriever,
)
from haystack.components.writers import DocumentWriter
documents = [
Document(
content="Renewable energy is energy that is collected from renewable resources.",
),
Document(
content="Solar energy is a type of green energy that is harnessed from the sun.",
),
Document(
content="Wind energy is another type of green energy that is generated by wind turbines.",
),
Document(
content="Geothermal energy is heat that comes from the sub-surface of the earth.",
),
]
doc_store = InMemoryDocumentStore()
doc_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
doc_writer = DocumentWriter(document_store=doc_store, policy=DuplicatePolicy.SKIP)
doc_writer.run(documents=doc_embedder.run(documents)["documents"])
retriever = TextEmbeddingRetriever(
retriever=InMemoryEmbeddingRetriever(document_store=doc_store, top_k=2),
text_embedder=SentenceTransformersTextEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
),
)
result = retriever.run(query="Geothermal energy")
for doc in result["documents"]:
print(f"Content: {doc.content}, Score: {doc.score}")
```
### As part of MultiRetriever
`TextEmbeddingRetriever` is most commonly used as one of the retrievers inside a [`MultiRetriever`](multiretriever.mdx):
```python
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
)
from haystack.components.retrievers import (
InMemoryBM25Retriever,
InMemoryEmbeddingRetriever,
)
from haystack.components.retrievers import MultiRetriever, TextEmbeddingRetriever
retriever = MultiRetriever(
retrievers={
"bm25": InMemoryBM25Retriever(document_store=doc_store),
"embedding": TextEmbeddingRetriever(
retriever=InMemoryEmbeddingRetriever(document_store=doc_store),
text_embedder=SentenceTransformersTextEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
),
),
},
)
```
---
// File: pipeline-components/retrievers/valkeyembeddingretriever
# ValkeyEmbeddingRetriever
This is an embedding Retriever compatible with the Valkey Document Store.
In a RAG pipeline, before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx)
In a semantic search pipeline, as the last component
As a retriever inside [`MultiRetriever`](multiretriever.mdx) | | **Mandatory init variables** | `retriever`: An embedding-based Retriever
`text_embedder`: A Text Embedder component | | **Mandatory run variables** | `query`: A query string | | **Output variables** | `documents`: A list of retrieved documents sorted by relevance score | | **API reference** | [Retrievers](/reference/retrievers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/retrievers/text_embedding_retriever.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in a semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [ValkeyDocumentStore](../../document-stores/valkeydocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A list of floats |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Valkey](/reference/integrations-valkey) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/valkey |
| **Package name** | `valkey-haystack` |
## Overview
The `ValkeyEmbeddingRetriever` is an embedding-based Retriever compatible with the [`ValkeyDocumentStore`](../../document-stores/valkeydocumentstore.mdx). It compares the query and Document embeddings and fetches the Documents most relevant to the query from the `ValkeyDocumentStore` based on vector similarity.
### Parameters
When using the `ValkeyEmbeddingRetriever` in your system, ensure the query and Document [embeddings](../embedders.mdx) are available. You can do so by adding a Document embedder to your indexing pipeline and a text embedder to your query pipeline.
In addition to the `query_embedding`, the `ValkeyEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
## Usage
### Installation
To start using Valkey with Haystack, install the package with:
```shell
pip install valkey-haystack
```
### On its own
This Retriever needs an instance of `ValkeyDocumentStore` and indexed Documents to run.
```python
from haystack_integrations.document_stores.valkey import ValkeyDocumentStore
from haystack_integrations.components.retrievers.valkey import ValkeyEmbeddingRetriever
document_store = ValkeyDocumentStore(
nodes_list=[("localhost", 6379)],
index_name="my_documents",
embedding_dim=768,
distance_metric="cosine",
)
retriever = ValkeyEmbeddingRetriever(document_store=document_store)
# Using a fake vector to keep the example simple
retriever.run(query_embedding=[0.1] * 768)
```
### In a Pipeline
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document, Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.writers import DocumentWriter
from haystack_integrations.document_stores.valkey import ValkeyDocumentStore
from haystack_integrations.components.retrievers.valkey import ValkeyEmbeddingRetriever
document_store = ValkeyDocumentStore(
nodes_list=[("localhost", 6379)],
index_name="my_documents",
embedding_dim=768,
distance_metric="cosine",
)
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
indexing = Pipeline()
indexing.add_component("embedder", SentenceTransformersDocumentEmbedder())
indexing.add_component("writer", DocumentWriter(document_store))
indexing.connect("embedder.documents", "writer.documents")
indexing.run({"embedder": {"documents": documents}})
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
ValkeyEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
For a full RAG example with `ValkeyEmbeddingRetriever`, see the [ValkeyDocumentStore](../../document-stores/valkeydocumentstore.mdx#using-valkey-in-a-rag-pipeline) documentation.
---
// File: pipeline-components/retrievers/vespaembeddingretriever
# VespaEmbeddingRetriever
An embedding-based Retriever compatible with the Vespa Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [VespaDocumentStore](../../document-stores/vespadocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A vector representing the query (a list of floats) |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Vespa](/reference/integrations-vespa) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/vespa |
| **Package name** | `vespa-haystack` |
## Overview
The `VespaEmbeddingRetriever` is a dense embedding-based Retriever compatible with the `VespaDocumentStore`. It uses Vespa's [nearest-neighbor search](https://docs.vespa.ai/en/nearest-neighbor-search.html) to find Documents whose embedding is closest to the query embedding and applies a configurable rank profile to score them.
When using the `VespaEmbeddingRetriever` in your Pipeline, make sure it has the query and Document embeddings available. You can do so by adding a Document Embedder to your indexing Pipeline and a Text Embedder to your query Pipeline.
In addition to the `query_embedding`, the `VespaEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
The retriever expects the underlying Vespa application to expose:
- A tensor field for embeddings (named `embedding` by default, configurable on the Document Store via `embedding_field`).
- A rank profile that scores nearest-neighbor candidates (named `semantic` by default, configurable via the `ranking` parameter). The profile typically uses `closeness(field, embedding)` and takes a query input tensor (named `query_embedding` by default, configurable via `query_tensor_name`).
You can additionally tune retrieval with `target_hits`, which sets how many neighbors each Vespa content node considers per query before first-phase ranking.
## Installation
Install the `vespa-haystack` integration:
```shell
pip install vespa-haystack
```
To run Vespa locally, see the [Vespa quick start](https://docs.vespa.ai/en/vespa-quick-start.html).
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
## Usage
### On its own
This Retriever needs the `VespaDocumentStore` and indexed Documents to run. Set the `VESPA_URL` environment variable (or pass `url=...` to the Document Store) to connect to your Vespa application.
```python
from haystack_integrations.document_stores.vespa import VespaDocumentStore
from haystack_integrations.components.retrievers.vespa import (
VespaEmbeddingRetriever,
)
document_store = VespaDocumentStore(schema="doc", namespace="doc")
retriever = VespaEmbeddingRetriever(document_store=document_store)
## using a fake vector to keep the example simple
retriever.run(query_embedding=[0.1] * 768)
```
### In a Pipeline
```python
from haystack import Document, Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.writers import DocumentWriter
from haystack_integrations.document_stores.vespa import VespaDocumentStore
from haystack_integrations.components.retrievers.vespa import (
VespaEmbeddingRetriever,
)
document_store = VespaDocumentStore(
schema="doc",
namespace="doc",
content_field="content",
embedding_field="embedding",
metadata_fields=["category"],
)
documents = [
Document(
content="Haystack integrates with Vespa for search.",
meta={"category": "docs"},
),
Document(
content="Vespa supports lexical and vector retrieval.",
meta={"category": "docs"},
),
Document(content="Cats sleep most of the day.", meta={"category": "animals"}),
]
indexing = Pipeline()
indexing.add_component("embedder", SentenceTransformersDocumentEmbedder())
indexing.add_component("writer", DocumentWriter(document_store=document_store))
indexing.connect("embedder", "writer")
indexing.run({"embedder": {"documents": documents}})
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
VespaEmbeddingRetriever(
document_store=document_store,
top_k=2,
query_tensor_name="query_embedding",
),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "semantic vector search"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/vespakeywordretriever
# VespaKeywordRetriever
A keyword-based Retriever that fetches documents matching a query from the Vespa Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the keyword search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [VespaDocumentStore](../../document-stores/vespadocumentstore.mdx) |
| **Mandatory run variables** | `query`: A string |
| **Output variables** | `documents`: A list of documents (matching the query) |
| **API reference** | [Vespa](/reference/integrations-vespa) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/vespa |
| **Package name** | `vespa-haystack` |
## Overview
The `VespaKeywordRetriever` is a keyword-based Retriever compatible with the `VespaDocumentStore`. It runs a [YQL](https://docs.vespa.ai/en/query-language.html) `userQuery()` against your Vespa application and ranks results with a configurable rank profile (defaults to `bm25`, which typically uses Vespa's [BM25 ranking feature](https://docs.vespa.ai/en/reference/bm25.html)).
The retriever expects the underlying Vespa application to expose:
- A text field for the Document body (named `content` by default, configurable on the Document Store via `content_field`). The field needs to be indexed for text matching in your Vespa schema.
- A rank profile that scores lexical matches (named `bm25` by default, configurable via the `ranking` parameter). Pass `ranking=None` to use the schema default profile.
In addition to the `query`, the `VespaKeywordRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow the search space.
## Installation
Install the `vespa-haystack` integration:
```shell
pip install vespa-haystack
```
To run Vespa locally, see the [Vespa quick start](https://docs.vespa.ai/en/vespa-quick-start.html).
## Usage
### On its own
This Retriever needs the `VespaDocumentStore` and indexed Documents to run. Set the `VESPA_URL` environment variable (or pass `url=...` to the Document Store) to connect to your Vespa application.
```python
from haystack_integrations.document_stores.vespa import VespaDocumentStore
from haystack_integrations.components.retrievers.vespa import (
VespaKeywordRetriever,
)
document_store = VespaDocumentStore(schema="doc", namespace="doc")
retriever = VespaKeywordRetriever(document_store=document_store)
retriever.run(query="my nice query")
```
### In a RAG pipeline
The prerequisites necessary for running this code are:
- Set an environment variable `OPENAI_API_KEY` with your OpenAI API key.
- Set the `VESPA_URL` environment variable (or pass `url=...` to the Document Store) to connect to your Vespa application.
- A deployed Vespa schema with a `content` text field, a `category` metadata field, and a `bm25` rank profile.
```python
from haystack import Document, Pipeline
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.document_stores.vespa import VespaDocumentStore
from haystack_integrations.components.retrievers.vespa import (
VespaKeywordRetriever,
)
## Create a RAG query pipeline
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given these documents, answer the question.\nDocuments:\n"
"{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
"Question: {{question}}\nAnswer:",
),
]
document_store = VespaDocumentStore(
schema="doc",
namespace="doc",
content_field="content",
metadata_fields=["category"],
)
documents = [
Document(
content="Haystack integrates with Vespa for search.",
meta={"category": "docs"},
),
Document(
content="Vespa supports lexical and vector retrieval.",
meta={"category": "docs"},
),
Document(
content="This note is about something else entirely.",
meta={"category": "misc"},
),
]
document_store.write_documents(documents=documents, policy=DuplicatePolicy.OVERWRITE)
retriever = VespaKeywordRetriever(
document_store=document_store,
filters={"field": "meta.category", "operator": "==", "value": "docs"},
)
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="retriever", instance=retriever)
rag_pipeline.add_component(
instance=ChatPromptBuilder(
template=prompt_template,
required_variables={"question", "documents"},
),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")
question = "How does Haystack work with Vespa?"
result = rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
"answer_builder": {"query": question},
},
)
print(result["answer_builder"])
```
---
// File: pipeline-components/retrievers/weaviatebm25retriever
# WeaviateBM25Retriever
This is a keyword-based Retriever that fetches Documents matching a query from the Weaviate Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [WeaviateDocumentStore](../../document-stores/weaviatedocumentstore.mdx) |
| **Mandatory run variables** | `query`: A string |
| **Output variables** | `documents`: A list of documents (matching the query) |
| **API reference** | [Weaviate](/reference/integrations-weaviate) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weaviate |
| **Package name** | `weaviate-haystack` |
## Overview
`WeaviateBM25Retriever` is a keyword-based Retriever that fetches Documents matching a query from [`WeaviateDocumentStore`](../../document-stores/weaviatedocumentstore.mdx). It determines the similarity between Documents and the query based on the BM25 algorithm, which computes a weighted word overlap between the
two strings.
Since the `WeaviateBM25Retriever` matches strings based on word overlap, it’s often used to find exact matches to names of persons or products, IDs, or well-defined error messages. The BM25 algorithm is very lightweight and simple. Beating it with more complex embedding-based approaches on out-of-domain data can be hard.
If you want a semantic match between a query and documents, use the [`WeaviateEmbeddingRetriever`](weaviateembeddingretriever.mdx), which uses vectors created by embedding models to retrieve relevant information.
### Parameters
In addition to the `query`, the `WeaviateBM25Retriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
### Usage
### Installation
To start using Weaviate with Haystack, install the package with:
```shell
pip install weaviate-haystack
```
#### On its own
This Retriever needs an instance of `WeaviateDocumentStore` and indexed Documents to run.
```python
from haystack_integrations.document_stores.weaviate.document_store import (
WeaviateDocumentStore,
)
from haystack_integrations.components.retrievers.weaviate import WeaviateBM25Retriever
document_store = WeaviateDocumentStore(url="http://localhost:8080")
retriever = WeaviateBM25Retriever(document_store=document_store)
retriever.run(query="How to make a pizza", top_k=3)
```
#### In a Pipeline
```python
from haystack_integrations.document_stores.weaviate.document_store import (
WeaviateDocumentStore,
)
from haystack_integrations.components.retrievers.weaviate import (
WeaviateBM25Retriever,
)
from haystack import Document
from haystack import Pipeline
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy
# Create a RAG query pipeline
prompt_template = [
ChatMessage.from_user(
"""
Given these documents, answer the question.\nDocuments:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
\nQuestion: {{question}}
\nAnswer:
""",
),
]
document_store = WeaviateDocumentStore(url="http://localhost:8080")
# Add Documents
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
# DuplicatePolicy.SKIP param is optional, but useful to run the script multiple times without throwing errors
document_store.write_documents(documents=documents, policy=DuplicatePolicy.SKIP)
rag_pipeline = Pipeline()
rag_pipeline.add_component(
name="retriever",
instance=WeaviateBM25Retriever(document_store=document_store),
)
rag_pipeline.add_component(
instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")
question = "How many languages are spoken around the world today?"
result = rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
"answer_builder": {"query": question},
},
)
print(result["answer_builder"]["answers"][0])
```
---
// File: pipeline-components/retrievers/weaviateembeddingretriever
# WeaviateEmbeddingRetriever
This is an embedding Retriever compatible with the Weaviate Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [WeaviateDocumentStore](../../document-stores/weaviatedocumentstore.mdx) |
| **Mandatory run variables** | `query_embedding`: A list of floats |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Weaviate](/reference/integrations-weaviate) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weaviate |
| **Package name** | `weaviate-haystack` |
## Overview
The `WeaviateEmbeddingRetriever` is an embedding-based Retriever compatible with the [`WeaviateDocumentStore`](../../document-stores/weaviatedocumentstore.mdx). It compares the query and Document embeddings and fetches the Documents most relevant to the query from the `WeaviateDocumentStore` based on the outcome.
### Parameters
When using the `WeaviateEmbeddingRetriever` in your NLP system, ensure the query and Document [embeddings](../embedders.mdx) are available. You can do so by adding a Document Embedder to your indexing Pipeline and a Text Embedder to your query Pipeline.
In addition to the `query_embedding`, the `WeaviateEmbeddingRetriever` accepts other optional parameters, including `top_k` (the maximum number of Documents to retrieve) and `filters` to narrow down the search space.
You can also specify `distance`, the maximum allowed distance between embeddings, and `certainty`, the normalized distance between the result items and the search embedding. The behavior of `distance` depends on the Collection’s distance metric used. See the [official Weaviate documentation](https://weaviate.io/developers/weaviate/api/graphql/search-operators#variables) for more information.
The embedding similarity function depends on the vectorizer used in the `WeaviateDocumentStore` collection. Check out the [official Weaviate documentation](https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules) to see all the supported vectorizers.
## Usage
### Installation
To start using Weaviate with Haystack, install the package with:
```shell
pip install weaviate-haystack
```
### On its own
This Retriever needs an instance of `WeaviateDocumentStore` and indexed Documents to run.
```python
from haystack_integrations.document_stores.weaviate.document_store import (
WeaviateDocumentStore,
)
from haystack_integrations.components.retrievers.weaviate import (
WeaviateEmbeddingRetriever,
)
document_store = WeaviateDocumentStore(url="http://localhost:8080")
retriever = WeaviateEmbeddingRetriever(document_store=document_store)
# using a fake vector to keep the example simple
retriever.run(query_embedding=[0.1] * 768)
```
### In a Pipeline
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack.document_stores.types import DuplicatePolicy
from haystack import Document
from haystack import Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack_integrations.document_stores.weaviate.document_store import (
WeaviateDocumentStore,
)
from haystack_integrations.components.retrievers.weaviate import (
WeaviateEmbeddingRetriever,
)
document_store = WeaviateDocumentStore(url="http://localhost:8080")
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings.get("documents"),
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
WeaviateEmbeddingRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run({"text_embedder": {"text": query}})
print(result["retriever"]["documents"][0])
```
---
// File: pipeline-components/retrievers/weaviatehybridretriever
# WeaviateHybridRetriever
A Retriever that combines BM25 keyword search and vector similarity to fetch documents from the Weaviate Document Store.
| | |
| --- | --- |
| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in a hybrid search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline |
| **Mandatory init variables** | `document_store`: An instance of a [WeaviateDocumentStore](../../document-stores/weaviatedocumentstore.mdx) |
| **Mandatory run variables** | `query`: A string
`query_embedding`: A list of floats | | **Output variables** | `documents`: A list of documents (matching the query) | | **API reference** | [Weaviate](/reference/integrations-weaviate) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weaviate | | **Package name** | `weaviate-haystack` |
## Overview
The `WeaviateHybridRetriever` combines keyword-based (BM25) and vector similarity search to fetch documents from the [`WeaviateDocumentStore`](../../document-stores/weaviatedocumentstore.mdx). Weaviate executes both searches in parallel and fuses the results into a single ranked list. The Retriever requires both a text query and its corresponding embedding.
The `alpha` parameter controls how much each search method contributes to the final results:
- `alpha = 0.0`: only keyword (BM25) scoring is used,
- `alpha = 1.0`: only vector similarity scoring is used,
- Values in between blend the two; higher values favor the vector score, lower values favor BM25.
If you don't specify `alpha`, it defaults to `0.7`, which is also the Weaviate server default.
You can also use the `max_vector_distance` parameter to set a threshold for the vector component. Candidates with a distance larger than this threshold are excluded from the vector portion before blending.
See the [official Weaviate documentation](https://weaviate.io/developers/weaviate/search/hybrid#parameters) for more details on hybrid search parameters.
### Parameters
When using the `WeaviateHybridRetriever`, you need to provide both the query text and its embedding. You can do this by adding a Text Embedder to your query pipeline.
In addition to `query` and `query_embedding`, the retriever accepts optional parameters including `top_k` (the maximum number of documents to return), `filters` to narrow down the search space, and `filter_policy` to determine how filters are applied.
## Usage
### Installation
To start using Weaviate with Haystack, install the package with:
```shell
pip install weaviate-haystack
```
### On its own
This Retriever needs an instance of `WeaviateDocumentStore` and indexed documents to run.
```python
from haystack_integrations.document_stores.weaviate.document_store import (
WeaviateDocumentStore,
)
from haystack_integrations.components.retrievers.weaviate import WeaviateHybridRetriever
document_store = WeaviateDocumentStore(url="http://localhost:8080")
retriever = WeaviateHybridRetriever(document_store=document_store)
# using a fake vector to keep the example simple
retriever.run(query="How many languages are there?", query_embedding=[0.1] * 768)
```
### In a pipeline
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack.document_stores.types import DuplicatePolicy
from haystack import Document
from haystack import Pipeline
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack_integrations.document_stores.weaviate.document_store import (
WeaviateDocumentStore,
)
from haystack_integrations.components.retrievers.weaviate import (
WeaviateHybridRetriever,
)
document_store = WeaviateDocumentStore(url="http://localhost:8080")
documents = [
Document(content="There are over 7,000 languages spoken around the world today."),
Document(
content="Elephants have been observed to behave in a way that indicates a high level of self-awareness, such as recognizing themselves in mirrors.",
),
Document(
content="In certain parts of the world, like the Maldives, Puerto Rico, and San Diego, you can witness the phenomenon of bioluminescent waves.",
),
]
document_embedder = SentenceTransformersDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)
document_store.write_documents(
documents_with_embeddings.get("documents"),
policy=DuplicatePolicy.OVERWRITE,
)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
WeaviateHybridRetriever(document_store=document_store),
)
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "How many languages are there?"
result = query_pipeline.run(
{"text_embedder": {"text": query}, "retriever": {"query": query}},
)
print(result["retriever"]["documents"][0])
```
### Adjusting the Alpha Parameter
You can set the `alpha` parameter at initialization or override it at query time:
```python
from haystack_integrations.components.retrievers.weaviate import WeaviateHybridRetriever
# Favor keyword search (good for exact matches)
retriever_keyword_heavy = WeaviateHybridRetriever(
document_store=document_store,
alpha=0.25,
)
# Balanced hybrid search
retriever_balanced = WeaviateHybridRetriever(document_store=document_store, alpha=0.5)
# Favor vector search (good for semantic similarity)
retriever_vector_heavy = WeaviateHybridRetriever(
document_store=document_store,
alpha=0.75,
)
# Override alpha at query time
result = retriever_balanced.run(
query="artificial intelligence",
query_embedding=embedding,
alpha=0.8,
)
```
---
// File: pipeline-components/retrievers
# Retrievers
Retrievers go through all the documents in a Document Store and select the ones that match the user query.
## How Do Retrievers Work?
Retrievers are the basic components of the majority of search systems. They’re used in the retrieval part of the retrieval-augmented generation (RAG) pipelines, they’re at the core of document retrieval pipelines, and they’re paired up with a Reader in extractive question answering pipelines.
When given a query, the Retriever sifts through the documents in the Document Store, assigns a score to each document to indicate how relevant it is to the query, and returns top candidates. It then passes the selected documents on to the next component in the pipeline or returns them as answers to the query.
Nevertheless, it's important to note that most Retrievers based on dense embedding do not compare each document with the query but use approximate techniques to achieve almost the same result with better performance.
## Retriever Types
Depending on how they calculate the similarity between the query and the document, you can divide Retrievers into sparse keyword-based, dense embedding-based, and sparse embedding-based. Several Document Stores can be coupled with different types of Retrievers.
### Sparse Keyword-Based Retrievers
The sparse keyword-based Retrievers look for keywords shared between the documents and the query using the BM25 algorithm or similar ones. This algorithm computes a weighted world overlap between the documents and the query.
Main features:
- Simple but effective, don’t need training, work quite well out of the box
- Can work on any language
- Don’t take word order or syntax into account
- Can’t handle out-of-vocabulary words
- Are good for use cases where precise wording matters
- Can’t handle synonyms or words with similar meaning
### Dense Embedding-Based Retrievers
Dense embedding-based Retrievers work with embeddings, which are vector representations of words that capture their semantics. Dense Retrievers need an [Embedder](embedders.mdx) first to turn the documents and the query into vectors. Then, they calculate the vector similarity of the query and each document in the Document Store to fetch the most relevant documents.
Main features:
- They’re powerful but also more expensive computationally than sparse Retrievers
- They’re trained on labeled datasets
- They’re language-specific, which means they can only work in the language of the dataset they were trained on. Nevertheless, multilingual embedding models are available.
- Because they work with embeddings, they take word order and syntax into account
- Can handle out-of-vocabulary words to a certain extent
### Sparse Embedding-Based Retrievers
This category includes approaches such as [SPLADE](https://www.pinecone.io/learn/splade/). These techniques combine the positive aspects of keyword-based and dense embedding Retrievers using specific embedding models.
In particular, SPLADE uses Language Models like BERT to weigh the relevance of different terms in the query and perform automatic term expansions, reducing the vocabulary mismatch problem (queries and relevant documents often lack term overlap).
Main features:
- Better than dense embedding Retrievers on precise keyword matching
- Better than BM25 on semantic matching
- Slower than BM25
- Still experimental compared to both BM25 and dense embeddings: few models supported by few Document Stores
### Filter Retriever
`FilterRetriever` is a special kind of Retriever that can work with all Document Stores and retrieves all documents that match the provided filters.
For more information, read this Retriever's [documentation page](retrievers/filterretriever.mdx).
### Advanced Retriever Techniques
#### Combining Retrievers
You can use different types of Retrievers in one pipeline to take advantage of the strengths and mitigate the weaknesses of each of them. There are two most common strategies to do this: combining a sparse and dense Retriever (hybrid retrieval) and using two dense Retrievers, each with a different model (multi-embedding retrieval).
##### Hybrid Retrieval
You can use different Retriever types, sparse and dense, in one pipeline to take advantage of their strengths and make your pipeline more robust to different kinds of queries and documents. When both Retrievers fetch their candidate documents, you can combine them to produce the final ranking and get the top documents as a result.
See an example of this approach in our [`DocumentJoiner` docs](joiners/documentjoiner.mdx#in-a-pipeline).
:::tip[Metadata Filtering]
When talking about hybrid retrieval, some database providers mean _metadata filtering_ on dense embedding retrieval. While this is different from combining different Retrievers, it is usually supported by Haystack Retrievers. For more information, check the [Metadata Filtering page](../concepts/metadata-filtering.mdx).
:::
:::info[Hybrid Retrievers]
Some Document Stores offer hybrid retrieval on the database side.
In general, these solutions can be performant, but they offer fewer customization options (for instance, on how to merge results from different retrieval techniques).
Some hybrid Retrievers are available in Haystack, such as [`QdrantHybridRetriever`](retrievers/qdranthybridretriever.mdx).
If your preferred Document Store does not have a hybrid Retriever available or if you want to customize the behavior even further, check out the hybrid retrieval pipelines [tutorial](https://haystack.deepset.ai/tutorials/33_hybrid_retrieval).
:::
##### Multi-Retriever
[`MultiRetriever`](retrievers/multiretriever.mdx) composes any number of text retrievers into a single component, running them in parallel and deduplicating results. Unlike wiring individual retrievers in a pipeline, `MultiRetriever` encapsulates all retrieval strategies in one component and lets you enable or disable specific retrievers at runtime using the `active_retrievers` parameter.
Use [`TextEmbeddingRetriever`](retrievers/textembeddingretriever.mdx) to wrap an embedding-based retriever so it can be used inside `MultiRetriever`.
:::warning[Experimental]
`MultiRetriever` is experimental and may change or be removed in future releases without prior deprecation notice.
:::
##### Multi-Query Retrieval
Multi-query retrieval improves recall by expanding a single user query into multiple semantically similar queries. Each query variation can capture different aspects of the user's intent and match documents that use different terminology.
This approach works with both text-based and embedding-based Retrievers:
- [`MultiQueryTextRetriever`](retrievers/multiquerytextretriever.mdx): Wraps a text-based Retriever (such as BM25) and runs multiple queries in parallel.
- [`MultiQueryEmbeddingRetriever`](retrievers/multiqueryembeddingretriever.mdx): Wraps an embedding-based Retriever and runs multiple queries in parallel.
To generate query variations, use the [`QueryExpander`](query/queryexpander.mdx) component, which uses an LLM to create semantically similar queries from the original.
##### Multi-Embedding Retrieval
In this strategy, you use two embedding-based Retrievers, each with a different model, to embed the same documents. You then end up having multiple embeddings of one document. It can also be handy if you need multimodal retrieval.
## Retrievers and Document Stores
Retrievers are tightly coupled with [Document Stores](../concepts/document-store.mdx). Most Document Stores can work both with a sparse or a dense Retriever or both Retriever types combined. See the documentation of a specific Document Store to check which Retrievers it supports.
### Naming Conventions
The Retriever names in Haystack consist of:
- Document Store name +
- Retrieval method +
- _Retriever_.
Practical examples:
- `ElasticsearchBM25Retriever`: BM25 is a sparse keyword-based retrieval technique, and this Retriever works with `ElasticsearchDocumentStore`.
- `ElasticsearchEmbeddingRetriever`: When not mentioned, Embedding stays for Dense Embedding, and this Retriever works with `ElasticsearchDocumentStore`.
- `QdrantSparseEmbeddingRetriever`: Sparse Embedding is the technique, and this Retriever works with `QdrantDocumentStore`.
While we try to stick to this convention, there is sometimes a need to be flexible and accommodate features that are specific to a Document Store. For example:
- `ChromaQueryTextRetriever`: This Retriever uses the query API of Chroma and expects text inputs. It works with `ChromaDocumentStore`.
## FilterPolicy
`FilterPolicy` determines how filters are applied during the document retrieval process. It controls the interaction between static filters set during Retriever initialization and dynamic filters provided at runtime. The possible values are:
- **REPLACE** (default): Any runtime filters completely override the initialization filters. This allows specific queries to dynamically change the filtering scope.
- **MERGE**: Combines runtime filters with initialization filters, narrowing down the search results.
The `FilterPolicy` is set in a selected Retriever's init method, while `filters` can be set in both init and run methods.
## Using a Retriever
For details on how to initialize and use a Retriever in a pipeline, see the documentation for a specific Retriever. The following Retrievers are available in Haystack:
| Component | Description |
| --- | --- |
| [AlloyDBEmbeddingRetriever](retrievers/alloydbembeddingretriever.mdx) | An embedding-based Retriever compatible with the AlloyDB Document Store. |
| [AlloyDBKeywordRetriever](retrievers/alloydbkeywordretriever.mdx) | A keyword-based Retriever that fetches Documents matching a query from the AlloyDB Document Store. |
| [ArangoEmbeddingRetriever](retrievers/arangoembeddingretriever.mdx) | An embedding-based Retriever compatible with the ArangoDB Document Store. |
| [ArcadeDBEmbeddingRetriever](retrievers/arcadedbembeddingretriever.mdx) | An embedding-based Retriever compatible with the ArcadeDB Document Store. |
| [AstraEmbeddingRetriever](retrievers/astraretriever.mdx) | An embedding-based Retriever compatible with the AstraDocumentStore. |
| [AutoMergingRetriever](retrievers/automergingretriever.mdx) | Retrieves complete parent documents instead of fragmented chunks when multiple related pieces match a query. |
| [AzureAISearchEmbeddingRetriever](retrievers/azureaisearchembeddingretriever.mdx) | An embedding Retriever compatible with the Azure AI Search Document Store. |
| [AzureAISearchBM25Retriever](retrievers/azureaisearchbm25retriever.mdx) | A keyword-based Retriever that fetches Documents matching a query from the Azure AI Search Document Store. |
| [AzureAISearchHybridRetriever](retrievers/azureaisearchhybridretriever.mdx) | A Retriever based both on dense and sparse embeddings, compatible with the Azure AI Search Document Store. |
| [ChromaEmbeddingRetriever](retrievers/chromaembeddingretriever.mdx) | An embedding-based Retriever compatible with the Chroma Document Store. |
| [ChromaQueryTextRetriever](retrievers/chromaqueryretriever.mdx) | A Retriever compatible with the Chroma Document Store that uses the Chroma query API. |
| [CogneeRetriever](retrievers/cogneeretriever.mdx) | Retrieves memories from a CogneeMemoryStore and returns them as system ChatMessage objects. |
| [ElasticsearchEmbeddingRetriever](retrievers/elasticsearchembeddingretriever.mdx) | An embedding-based Retriever compatible with the Elasticsearch Document Store. |
| [ElasticsearchBM25Retriever](retrievers/elasticsearchbm25retriever.mdx) | A keyword-based Retriever that fetches Documents matching a query from the Elasticsearch Document Store. |
| [ElasticsearchHybridRetriever](retrievers/elasticsearchhybridretriever.mdx) | A SuperComponent that combines BM25 and embedding-based retrieval from the Elasticsearch Document Store. |
| [ElasticsearchSQLRetriever](retrievers/elasticsearchsqlretriever.mdx) | Executes raw Elasticsearch SQL queries against an Elasticsearch Document Store and returns the raw JSON response. |
| [FAISSEmbeddingRetriever](retrievers/faissembeddingretriever.mdx) | An embedding-based Retriever compatible with the FAISSDocumentStore. |
| [FalkorDBCypherRetriever](retrievers/falkordbcypherretriever.mdx) | A Retriever that executes arbitrary OpenCypher queries against a FalkorDB Document Store. |
| [FalkorDBEmbeddingRetriever](retrievers/falkordbembeddingretriever.mdx) | An embedding-based Retriever compatible with the FalkorDB Document Store. |
| [GoogleDriveRetriever](retrievers/googledriveretriever.mdx) | Retrieves files from Google Drive via the Drive API v3 search endpoint. |
| [InMemoryBM25Retriever](retrievers/inmemorybm25retriever.mdx) | A keyword-based Retriever compatible with the InMemoryDocumentStore. |
| [InMemoryEmbeddingRetriever](retrievers/inmemoryembeddingretriever.mdx) | An embedding-based Retriever compatible with the InMemoryDocumentStore. |
| [FilterRetriever](retrievers/filterretriever.mdx) | A special Retriever to be used with any Document Store to get the Documents that match specific filters. |
| [Mem0MemoryRetriever](retrievers/mem0memoryretriever.mdx) | Retrieves ChatMessage memories from Mem0 for memory-augmented Agent and pipeline workflows. |
| [MultiQueryEmbeddingRetriever](retrievers/multiqueryembeddingretriever.mdx) | Retrieves documents using multiple queries in parallel with an embedding-based Retriever. |
| [MultiQueryTextRetriever](retrievers/multiquerytextretriever.mdx) | Retrieves documents using multiple queries in parallel with a text-based Retriever. |
| [MultiRetriever](retrievers/multiretriever.mdx) | Runs multiple text retrievers in parallel and combines their deduplicated results. Experimental. |
| [MongoDBAtlasEmbeddingRetriever](retrievers/mongodbatlasembeddingretriever.mdx) | An embedding Retriever compatible with the MongoDB Atlas Document Store. |
| [MongoDBAtlasFullTextRetriever](retrievers/mongodbatlasfulltextretriever.mdx) | A full-text search Retriever compatible with the MongoDB Atlas Document Store. |
| [MSSharePointRetriever](retrievers/mssharepointretriever.mdx) | Retrieves content from Microsoft SharePoint and OneDrive via the Microsoft Search (Graph) API. |
| [OpenSearchBM25Retriever](retrievers/opensearchbm25retriever.mdx) | A keyword-based Retriever that fetches Documents matching a query from an OpenSearch Document Store. |
| [OpenSearchEmbeddingRetriever](retrievers/opensearchembeddingretriever.mdx) | An embedding-based Retriever compatible with the OpenSearch Document Store. |
| [OpenSearchHybridRetriever](retrievers/opensearchhybridretriever.mdx) | A SuperComponent that implements a Hybrid Retriever in a single component, relying on OpenSearch as the backend Document Store. |
| [OpenSearchMetadataRetriever](retrievers/opensearchmetadataretriever.mdx) | Searches and ranks the metadata fields of documents stored in an OpenSearch Document Store and returns the matching metadata values. |
| [OpenSearchSQLRetriever](retrievers/opensearchsqlretriever.mdx) | Executes raw OpenSearch SQL queries against an OpenSearch Document Store and returns the raw JSON response. |
| [OracleEmbeddingRetriever](retrievers/oracleembeddingretriever.mdx) | An embedding-based Retriever compatible with the Oracle Document Store. |
| [OracleKeywordRetriever](retrievers/oraclekeywordretriever.mdx) | A keyword-based Retriever that fetches Documents matching a query from the Oracle Document Store. |
| [PgvectorEmbeddingRetriever](retrievers/pgvectorembeddingretriever.mdx) | An embedding-based Retriever compatible with the Pgvector Document Store. |
| [PgvectorKeywordRetriever](retrievers/pgvectorkeywordretriever.mdx) | A keyword-based Retriever that fetches documents matching a query from the Pgvector Document Store. |
| [PineconeEmbeddingRetriever](retrievers/pineconedenseretriever.mdx) | An embedding-based Retriever compatible with the Pinecone Document Store. |
| [QdrantEmbeddingRetriever](retrievers/qdrantembeddingretriever.mdx) | An embedding-based Retriever compatible with the Qdrant Document Store. |
| [QdrantSparseEmbeddingRetriever](retrievers/qdrantsparseembeddingretriever.mdx) | A sparse embedding-based Retriever compatible with the Qdrant Document Store. |
| [QdrantHybridRetriever](retrievers/qdranthybridretriever.mdx) | A Retriever based both on dense and sparse embeddings, compatible with the Qdrant Document Store. |
| [SentenceWindowRetriever](retrievers/sentencewindowretriever.mdx) | Retrieves neighboring sentences around relevant sentences to get the full context. |
| [SnowflakeTableRetriever](retrievers/snowflaketableretriever.mdx) | Connects to a Snowflake database to execute an SQL query. |
| [SQLAlchemyTableRetriever](retrievers/sqlalchemytableretriever.mdx) | Connects to any SQLAlchemy-supported database and executes an SQL query. |
| [SupabaseGroongaBM25Retriever](retrievers/supabasegroongabm25retriever.mdx) | A full-text Retriever that fetches documents from the SupabaseGroongaDocumentStore using PGroonga search. |
| [SupabasePgvectorEmbeddingRetriever](retrievers/supabasepgvectorembeddingretriever.mdx) | An embedding-based Retriever compatible with the SupabasePgvectorDocumentStore. |
| [SupabasePgvectorKeywordRetriever](retrievers/supabasepgvectorkeywordretriever.mdx) | A keyword-based Retriever that fetches documents matching a query from the SupabasePgvectorDocumentStore. |
| [TextEmbeddingRetriever](retrievers/textembeddingretriever.mdx) | Wraps an embedding-based retriever with a text embedder into a single component that accepts a text query. |
| [ValkeyEmbeddingRetriever](retrievers/valkeyembeddingretriever.mdx) | An embedding Retriever compatible with the Valkey Document Store. |
| [VespaEmbeddingRetriever](retrievers/vespaembeddingretriever.mdx) | An embedding-based Retriever compatible with the Vespa Document Store. |
| [VespaKeywordRetriever](retrievers/vespakeywordretriever.mdx) | A keyword-based Retriever that fetches Documents matching a query from the Vespa Document Store. |
| [WeaviateBM25Retriever](retrievers/weaviatebm25retriever.mdx) | A keyword-based Retriever that fetches Documents matching a query from the Weaviate Document Store. |
| [WeaviateEmbeddingRetriever](retrievers/weaviateembeddingretriever.mdx) | An embedding Retriever compatible with the Weaviate Document Store. |
| [WeaviateHybridRetriever](retrievers/weaviatehybridretriever.mdx) | Combines BM25 keyword search and vector similarity to fetch documents from the Weaviate Document Store. |
---
// File: pipeline-components/routers/conditionalrouter
# ConditionalRouter
`ConditionalRouter` routes your data through different paths down the pipeline by evaluating the conditions that you specified.
`query_embedding`: A list of floats | | **Output variables** | `documents`: A list of documents (matching the query) | | **API reference** | [Weaviate](/reference/integrations-weaviate) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weaviate | | **Package name** | `weaviate-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Flexible |
| **Mandatory init variables** | `routes`: A list of dictionaries defining routes (See the [Overview](#overview) section below) |
| **Mandatory run variables** | `**kwargs`: Input variables to evaluate in order to choose a specific route. See [Variables](#variables) section for more details. |
| **Output variables** | A dictionary containing one or more output names and values of the chosen route |
| **API reference** | [Routers](/reference/routers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/routers/conditional_router.py |
| **Package name** | `haystack-ai` |
## Overview
To use `ConditionalRouter` you need to define a list of routes.
Each route is a dictionary with the following elements:
- `'condition'`: A Jinja2 string expression that determines if the route is selected.
- `'output'`: A Jinja2 expression or list of expressions defining one or more output values.
- `'output_type'`: The expected type or list of types corresponding to each output (for example, `str`, `list[int]`).
- Note that this doesn't enforce the type conversion of the output. Instead, the output field is rendered using Jinja2, which automatically infers types. If you need to ensure the result is a string (for example, "123" instead of `123`), wrap the Jinja expression in single quotes like this: `output: "'{{message.text}}'"`. This ensures the rendered output is treated as a string by Jinja2.
- `'output_name'`: The name or list of names under which the output values are published. This is used to connect the router to other components in the pipeline.
## Usage
### Basic routing
In this example, we configure two routes. The first route sends the `'streams'` value to `'enough_streams'` if the stream count exceeds two. Conversely, the second route directs `'streams'` to `'insufficient_streams'` when there are two or fewer streams.
```python
from haystack.components.routers import ConditionalRouter
routes = [
{
"condition": "{{streams|length > 2}}",
"output": "{{streams}}",
"output_name": "enough_streams",
"output_type": list[int],
},
{
"condition": "{{streams|length <= 2}}",
"output": "{{streams}}",
"output_name": "insufficient_streams",
"output_type": list[int],
},
]
router = ConditionalRouter(routes)
result = router.run(streams=[1, 2, 3], query="Haystack")
print(result)
# {"enough_streams": [1, 2, 3]}
```
### Multiple outputs per route
Each route can emit more than one output at a time. Pass lists to `output`, `output_name`, and `output_type` — all three must have the same length.
```python
from haystack.components.routers import ConditionalRouter
routes = [
{
"condition": "{{ query|length > 10 }}",
"output": ["{{ query }}", "{{ query|length }}"],
"output_name": ["long_query", "char_count"],
"output_type": [str, int],
},
{
"condition": "{{ query|length <= 10 }}",
"output": ["{{ query }}", "{{ query|length }}"],
"output_name": ["short_query", "char_count"],
"output_type": [str, int],
},
]
router = ConditionalRouter(routes=routes)
result = router.run(query="Hello")
print(result)
# {'short_query': 'Hello', 'char_count': 5}
```
All outputs from the selected route are emitted together, so downstream components can consume any combination of them.
### Variables
By default, every Jinja2 variable referenced in your route `condition` and `output` templates is required — the component won't run until all of them are provided. You can mark specific variables as optional using the `optional_variables` init parameter.
```python
from haystack.components.routers import ConditionalRouter
routes = [
{
"condition": '{{ path == "rag" }}',
"output": "{{ question }}",
"output_name": "rag_route",
"output_type": str,
},
{
"condition": "{{ True }}", # fallback route
"output": "{{ question }}",
"output_name": "default_route",
"output_type": str,
},
]
# 'path' is optional, 'question' is required
router = ConditionalRouter(routes=routes, optional_variables=["path"])
# 'path' provided — first route matches
print(router.run(question="What is RAG?", path="rag"))
# {'rag_route': 'What is RAG?'}
# 'path' omitted — evaluates as None, fallback route fires
print(router.run(question="What is RAG?"))
# {'default_route': 'What is RAG?'}
```
If an optional variable is not provided at runtime, it's evaluated as `None`, which generally does not raise an error but can affect the condition's outcome.
### In a pipeline
Below is an example of a simple pipeline that routes a query based on its length and returns both the text and its character count.
If the query is too short, the pipeline returns a warning message and the character count, then stops.
If the query is long enough, the pipeline returns the original query and its character count, sends the query to the `PromptBuilder`, and then to the Generator to produce the final answer.
```python
from haystack import Pipeline
from haystack.components.routers import ConditionalRouter
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
# Two routes, each returning two outputs: the text and its length
routes = [
{
"condition": "{{ query|length > 10 }}",
"output": ["{{ query }}", "{{ query|length }}"],
"output_name": ["ok_query", "length"],
"output_type": [str, int],
},
{
"condition": "{{ query|length <= 10 }}",
"output": ["query too short: {{ query }}", "{{ query|length }}"],
"output_name": ["too_short_query", "length"],
"output_type": [str, int],
},
]
router = ConditionalRouter(routes=routes)
pipe = Pipeline()
pipe.add_component("router", router)
pipe.add_component(
"prompt_builder",
ChatPromptBuilder(
template=[ChatMessage.from_user("Answer the following query: {{ query }}")],
required_variables=["query"],
),
)
pipe.add_component("generator", OpenAIChatGenerator())
pipe.connect("router.ok_query", "prompt_builder.query")
pipe.connect("prompt_builder.prompt", "generator.messages")
# Short query: length ≤ 10 ⇒ fallback route fires.
print(pipe.run(data={"router": {"query": "Berlin"}}))
# {'router': {'too_short_query': 'query too short: Berlin', 'length': 6}}
# Long query: length > 10 ⇒ first route fires.
print(pipe.run(data={"router": {"query": "What is the capital of Italy?"}}))
# {
# 'router': {'length': 29},
# 'generator': {'replies': [ChatMessage(content='The capital of Italy is Rome (Italian: Roma).', role=## Additional References :notebook: Tutorial: [Building Fallbacks to Websearch with Conditional Routing](https://haystack.deepset.ai/tutorials/36_building_fallbacks_with_conditional_routing) --- // File: pipeline-components/routers/documentlengthrouter # DocumentLengthRouter Routes documents to different output connections based on the length of their `content` field.
| | |
| --- | --- |
| **Most common position in a pipeline** | Flexible |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `short_documents`: A list of documents where `content` is None or the length of `content` is less than or equal to the threshold.
`long_documents`: A list of documents where the length of `content` is greater than the threshold. | | **API reference** | [Routers](/reference/routers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/routers/document_length_router.py | | **Package name** | `haystack-ai` |
## Overview
`DocumentLengthRouter` routes documents to different output connections based on the length of their `content` field.
It allows to set a `threshold` init parameter. Documents where `content` is None, or the length of `content` is less than or equal to the threshold are routed to "short_documents". Others are routed to "long_documents".
A common use case for `DocumentLengthRouter` is handling documents obtained from PDFs that contain non-text content, such as scanned pages or images. This component can detect empty or low-content documents and route them to components that perform OCR, generate captions, or compute image embeddings.
## Usage
### On its own
```python
from haystack.components.routers import DocumentLengthRouter
from haystack.dataclasses import Document
docs = [
Document(content="Short"),
Document(content="Long document " * 20),
]
router = DocumentLengthRouter(threshold=10)
result = router.run(documents=docs)
print(result)
# {
# "short_documents": [Document(content="Short", ...)],
# "long_documents": [Document(content="Long document ...", ...)],
# }
```
### In a pipeline
In the following indexing pipeline, the `PyPDFToDocument` Converter extracts text from PDF files.
Documents are then split by pages using a `DocumentSplitter`.
Next, the `DocumentLengthRouter` routes short documents to `LLMDocumentContentExtractor` to extract text, which is particularly useful for non-textual, image-based pages.
Finally, all documents are sent to the `DocumentWriter` and written to the Document Store.
```python
from haystack import Pipeline
from haystack.components.converters import PyPDFToDocument
from haystack.components.extractors.image import LLMDocumentContentExtractor
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.routers import DocumentLengthRouter
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
indexing_pipe = Pipeline()
indexing_pipe.add_component("pdf_converter", PyPDFToDocument(store_full_path=True))
# setting skip_empty_documents=False is important here because the
# LLMDocumentContentExtractor can extract text from non-textual documents
# that otherwise would be skipped
indexing_pipe.add_component(
"pdf_splitter",
DocumentSplitter(split_by="page", split_length=1, skip_empty_documents=False),
)
indexing_pipe.add_component("doc_length_router", DocumentLengthRouter(threshold=10))
indexing_pipe.add_component(
"content_extractor",
LLMDocumentContentExtractor(
chat_generator=OpenAIChatGenerator(model="gpt-4.1-mini"),
),
)
indexing_pipe.add_component(
"document_writer",
DocumentWriter(document_store=document_store),
)
indexing_pipe.connect("pdf_converter.documents", "pdf_splitter.documents")
indexing_pipe.connect("pdf_splitter.documents", "doc_length_router.documents")
# The short PDF pages will be enriched/captioned
indexing_pipe.connect(
"doc_length_router.short_documents",
"content_extractor.documents",
)
indexing_pipe.connect("doc_length_router.long_documents", "document_writer.documents")
indexing_pipe.connect("content_extractor.documents", "document_writer.documents")
# Run the indexing pipeline with sources
indexing_result = indexing_pipe.run(
data={"sources": ["textual_pdf.pdf", "non_textual_pdf.pdf"]},
)
# Inspect the documents
indexed_documents = document_store.filter_documents()
print(f"Indexed {len(indexed_documents)} documents:\n")
for doc in indexed_documents:
print("file_path: ", doc.meta["file_path"])
print("page_number: ", doc.meta["page_number"])
print("content: ", doc.content)
print("-" * 100 + "\n")
# Indexed 3 documents:
#
# file_path: textual_pdf.pdf
# page_number: 1
# content: A sample PDF file...
# ----------------------------------------------------------------------------------------------------
#
# file_path: textual_pdf.pdf
# page_number: 2
# content: Page 2 of Sample PDF...
# ----------------------------------------------------------------------------------------------------
#
# file_path: non_textual_pdf.pdf
# page_number: 1
# content: Content extracted from non-textual PDF using a LLM...
# ----------------------------------------------------------------------------------------------------
```
---
// File: pipeline-components/routers/documenttyperouter
# DocumentTypeRouter
Use this Router in pipelines to route documents based on their MIME types to different outputs for further processing.
`long_documents`: A list of documents where the length of `content` is greater than the threshold. | | **API reference** | [Routers](/reference/routers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/routers/document_length_router.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | As a preprocessing component to route documents by type before sending them to specific [Converters](../converters.mdx) or [Preprocessors](../preprocessors.mdx) |
| **Mandatory init variables** | `mime_types`: A list of MIME types or regex patterns for classification |
| **Mandatory run variables** | `documents`: A list of [Documents](../../concepts/data-classes.mdx#document) to categorize |
| **Output variables** | `unclassified`: A list of uncategorized [Documents](../../concepts/data-classes.mdx#document)
`mime_types`: For example "text/plain", "application/pdf", "image/jpeg": List of categorized [Documents](../../concepts/data-classes.mdx#document) | | **API reference** | [Routers](/reference/routers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/routers/document_type_router.py | | **Package name** | `haystack-ai` |
## Overview
`DocumentTypeRouter` routes documents based on their MIME types, supporting both exact matches and regex patterns. It can determine MIME types from document metadata or infer them from file paths using standard Python `mimetypes` module and custom mappings.
When initializing the component, specify the set of MIME types to route to separate outputs. Set the `mime_types` parameter to a list of types, for example: `["text/plain", "audio/x-wav", "image/jpeg"]`. Documents with MIME types that are not listed are routed to an output named "unclassified".
The component requires at least one of the following parameters to determine MIME types:
- `mime_type_meta_field`: Name of the metadata field containing the MIME type
- `file_path_meta_field`: Name of the metadata field containing the file path (MIME type will be inferred from the file extension)
## Usage
### On its own
Below is an example that uses the `DocumentTypeRouter` to categorize documents by their MIME types:
```python
from haystack.components.routers import DocumentTypeRouter
from haystack.dataclasses import Document
docs = [
Document(content="Example text", meta={"file_path": "example.txt"}),
Document(content="Another document", meta={"mime_type": "application/pdf"}),
Document(content="Unknown type"),
]
router = DocumentTypeRouter(
mime_type_meta_field="mime_type",
file_path_meta_field="file_path",
mime_types=["text/plain", "application/pdf"],
)
result = router.run(documents=docs)
print(result)
```
Expected output:
```python
{
"text/plain": [Document(...)],
"application/pdf": [Document(...)],
"unclassified": [Document(...)],
}
```
### Using regex patterns
You can use regex patterns to match multiple MIME types with similar patterns:
```python
from haystack.components.routers import DocumentTypeRouter
from haystack.dataclasses import Document
docs = [
Document(content="Plain text", meta={"mime_type": "text/plain"}),
Document(content="HTML text", meta={"mime_type": "text/html"}),
Document(content="Markdown text", meta={"mime_type": "text/markdown"}),
Document(content="JPEG image", meta={"mime_type": "image/jpeg"}),
Document(content="PNG image", meta={"mime_type": "image/png"}),
Document(content="PDF document", meta={"mime_type": "application/pdf"}),
]
router = DocumentTypeRouter(
mime_type_meta_field="mime_type",
mime_types=[r"text/.*", r"image/.*"],
)
result = router.run(documents=docs)
# Result will have:
# - "text/.*": 3 documents (text/plain, text/html, text/markdown)
# - "image/.*": 2 documents (image/jpeg, image/png)
# - "unclassified": 1 document (application/pdf)
```
### Using custom MIME types
You can add custom MIME type mappings for uncommon file types:
```python
from haystack.components.routers import DocumentTypeRouter
from haystack.dataclasses import Document
docs = [
Document(content="Word document", meta={"file_path": "document.docx"}),
Document(content="Markdown file", meta={"file_path": "readme.md"}),
Document(content="Outlook message", meta={"file_path": "email.msg"}),
]
router = DocumentTypeRouter(
file_path_meta_field="file_path",
mime_types=[
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"text/markdown",
"application/vnd.ms-outlook",
],
additional_mimetypes={
"application/vnd.openxmlformats-officedocument.wordprocessingml.document": ".docx",
},
)
result = router.run(documents=docs)
```
### In a pipeline
Below is an example of a pipeline that uses a `DocumentTypeRouter` to categorize documents by type and then process them differently. Text documents get processed by a `DocumentSplitter` before being stored, while PDF documents are stored directly.
```python
from haystack import Pipeline
from haystack.components.routers import DocumentTypeRouter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack.dataclasses import Document
# Create document store
document_store = InMemoryDocumentStore()
# Create pipeline
p = Pipeline()
p.add_component(
instance=DocumentTypeRouter(
mime_types=["text/plain", "application/pdf"],
mime_type_meta_field="mime_type",
),
name="document_type_router",
)
p.add_component(instance=DocumentSplitter(), name="text_splitter")
p.add_component(
instance=DocumentWriter(document_store=document_store),
name="text_writer",
)
p.add_component(
instance=DocumentWriter(document_store=document_store),
name="pdf_writer",
)
# Connect components
p.connect("document_type_router.text/plain", "text_splitter.documents")
p.connect("text_splitter.documents", "text_writer.documents")
p.connect("document_type_router.application/pdf", "pdf_writer.documents")
# Create test documents
docs = [
Document(
content="This is a text document that will be split and stored.",
meta={"mime_type": "text/plain"},
),
Document(
content="This is a PDF document that will be stored directly.",
meta={"mime_type": "application/pdf"},
),
Document(
content="This is an image document that will be unclassified.",
meta={"mime_type": "image/jpeg"},
),
]
# Run pipeline
result = p.run({"document_type_router": {"documents": docs}})
# The pipeline will route documents based on their MIME types:
# - Text documents (text/plain) → DocumentSplitter → DocumentWriter
# - PDF documents (application/pdf) → DocumentWriter (direct)
# - Other documents → unclassified output
```
---
// File: pipeline-components/routers/filetyperouter
# FileTypeRouter
Use this Router in indexing pipelines to route file paths or byte streams based on their type to different outputs for further processing.
`mime_types`: For example "text/plain", "application/pdf", "image/jpeg": List of categorized [Documents](../../concepts/data-classes.mdx#document) | | **API reference** | [Routers](/reference/routers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/routers/document_type_router.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | As the first component preprocessing data followed by [Converters](../converters.mdx) |
| **Mandatory init variables** | `mime_types`: A list of MIME types or regex patterns for classification |
| **Mandatory run variables** | `sources`: A list of file paths or byte streams to categorize |
| **Output variables** | `unclassified`: A list of uncategorized file paths or [byte streams](../../concepts/data-classes.mdx#bytestream)
`failed`: A list of sources that could not be processed, for example a file path that doesn't exist
`mime_types`: For example "text/plain", "text/html", "application/pdf", "text/markdown", "audio/x-wav", "image/jpeg": List of categorized file paths or byte streams | | **API reference** | [Routers](/reference/routers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/routers/file_type_router.py | | **Package name** | `haystack-ai` |
## Overview
`FileTypeRouter` routes file paths or byte streams based on their type, for example, plain text, jpeg image, or audio wave. For file paths, it infers MIME types from their extensions, while for byte streams, it determines MIME types based on the provided metadata.
When initializing the component, you specify the set of MIME types to route to separate outputs. To do this, set the `mime_types` parameter to a list of types, for example: `["text/plain", "audio/x-wav", "image/jpeg"]`. Types that are not listed are routed to an output named “unclassified”.
## Usage
### On its own
Below is an example that uses the `FileTypeRouter` to route two file paths:
```python
from haystack import Document
from haystack.components.routers import FileTypeRouter
router = FileTypeRouter(mime_types=["text/plain"])
router.run(sources=["text-file-will-be-added.txt", "pdf-will-not-ne-added.pdf"])
```
### In a pipeline
Below is an example of a pipeline that uses a `FileTypeRouter` to forward only plain text files to a `DocumentSplitter` and then a `DocumentWriter`. Only the content of plain text files gets added to the `InMemoryDocumentStore`, but not the content of files of any other type. As an alternative, you could add a `PyPDFToDocument` Converter to the pipeline and use the `FileTypeRouter` to route PDFs to it so that it converts them to documents.
```python
from haystack import Pipeline
from haystack.components.routers import FileTypeRouter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component(
instance=FileTypeRouter(mime_types=["text/plain"]),
name="file_type_router",
)
p.add_component(instance=TextFileToDocument(), name="text_file_converter")
p.add_component(instance=DocumentSplitter(), name="splitter")
p.add_component(instance=DocumentWriter(document_store=document_store), name="writer")
p.connect("file_type_router.text/plain", "text_file_converter.sources")
p.connect("text_file_converter.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
p.run(
{
"file_type_router": {
"sources": ["text-file-will-be-added.txt", "pdf-will-not-be-added.pdf"],
},
},
)
```
---
// File: pipeline-components/routers/llmmessagesrouter
# LLMMessagesRouter
Use this component to route Chat Messages to various output connections using a generative Language Model to perform classification.
`failed`: A list of sources that could not be processed, for example a file path that doesn't exist
`mime_types`: For example "text/plain", "text/html", "application/pdf", "text/markdown", "audio/x-wav", "image/jpeg": List of categorized file paths or byte streams | | **API reference** | [Routers](/reference/routers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/routers/file_type_router.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Flexible |
| **Mandatory init variables** | `chat_generator`: A Chat Generator instance (the LLM used for classification)
`output_names`: A list of output connection names
`output_patterns`: A list of regular expressions to be matched against the output of the LLM. | | **Mandatory run variables** | `messages`: A list of Chat Messages | | **Output variables** | `chat_generator_text`: The text output of the LLM, useful for debugging
`output_names`: Each contains the list of messages that matched the corresponding pattern
`unmatched`: Messages not matching any pattern | | **API reference** | [Routers](/reference/routers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/routers/llm_messages_router.py | | **Package name** | `haystack-ai` |
## Overview
`LLMMessagesRouter` uses an LLM to classify chat messages and route them to different outputs based on that classification.
This is especially useful for tasks like content moderation. If a message is deemed safe, you might forward it to a Chat Generator to generate a reply. Otherwise, you may halt the interaction or log the message separately.
First, you need to pass a ChatGenerator instance in the `chat_generator` parameter.
Then, define two lists of the same length:
- `output_names`: The names of the outputs to which you want to route messages,
- `output_patterns`: Regular expressions that are matched against the LLM output.
Each pattern is evaluated in order, and the first match determines the output. To define appropriate patterns, we recommend reviewing the model card of your chosen LLM and/or experimenting with it.
Optionally, you can provide a `system_prompt` to guide the classification behavior of the LLM. In this case as well, we recommend checking the model card to discover customization options.
To see the full list of parameters, check out our [API reference](/reference/routers-api#llmmessagesrouter).
## Usage
### On its own
Below is an example of using `LLMMessagesRouter` to route Chat Messages to two output connections based on safety classification. Messages that don’t match any pattern are routed to `unmatched`.
We use Llama Guard 4 for content moderation. To use this model with the Hugging Face API, you need to [request access](https://huggingface.co/meta-llama/Llama-Guard-4-12B) and set the `HF_TOKEN` environment variable.
The examples on this page use Hugging Face API components from the `huggingface-api-haystack` package. Install it to run the examples:
```shell
pip install huggingface-api-haystack
```
```python
from haystack_integrations.components.generators.huggingface_api import (
HuggingFaceAPIChatGenerator,
)
from haystack.components.routers.llm_messages_router import LLMMessagesRouter
from haystack.dataclasses import ChatMessage
chat_generator = HuggingFaceAPIChatGenerator(
api_type="serverless_inference_api",
api_params={"model": "meta-llama/Llama-Guard-4-12B", "provider": "groq"},
)
router = LLMMessagesRouter(
chat_generator=chat_generator,
output_names=["unsafe", "safe"],
output_patterns=["unsafe", "safe"],
)
print(router.run([ChatMessage.from_user("How to rob a bank?")]))
# {
# 'chat_generator_text': 'unsafe\nS2',
# 'unsafe': [
# ChatMessage(
# _role=`output_names`: A list of output connection names
`output_patterns`: A list of regular expressions to be matched against the output of the LLM. | | **Mandatory run variables** | `messages`: A list of Chat Messages | | **Output variables** | `chat_generator_text`: The text output of the LLM, useful for debugging
`output_names`: Each contains the list of messages that matched the corresponding pattern
`unmatched`: Messages not matching any pattern | | **API reference** | [Routers](/reference/routers-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/routers/llm_messages_router.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After components that classify documents, such as [`DocumentLanguageClassifier`](../classifiers/documentlanguageclassifier.mdx) |
| **Mandatory init variables** | `rules`: A dictionary with metadata routing rules (see our API Reference for examples) |
| **Mandatory run variables** | `documents`: A list of documents or byte streams |
| **Output variables** | `unmatched`: A list of documents or byte streams not matching any rule
``: A list of documents or byte streams matching custom rules (where `` is the name of the rule). There's one output per one rule you define. Each of these outputs is a list of documents or byte streams. |
| **API reference** | [Routers](/reference/routers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/routers/metadata_router.py |
| **Package name** | `haystack-ai` |
## Overview
`MetadataRouter` routes documents or byte streams to different outputs based on their metadata. You initialize it with `rules` defining the names of the outputs and filters to match documents or byte streams to one of the connections. The filters follow the same syntax as filters in Document Stores. If a document or byte stream matches multiple filters, it is sent to multiple outputs. Objects that do not match any rule go to an output connection named `unmatched`.
In pipelines, this component is most useful after a Classifier (such as the `DocumentLanguageClassifier`) that adds the classification results to the documents' metadata.
This component has no default rules. If you don't define any rules when initializing the component, it routes all documents or byte streams to the `unmatched` output.
## Usage
### On its own
Below is an example that uses the `MetadataRouter` to filter out documents based on their metadata. We initialize the router by setting a rule to pass on all documents with `language` set to `en` in their metadata to an output connection called `en`. Documents that don't match this rule go to an output connection named `unmatched`.
```python
from haystack import Document
from haystack.components.routers import MetadataRouter
docs = [
Document(content="Paris is the capital of France.", meta={"language": "en"}),
Document(
content="Berlin ist die Haupststadt von Deutschland.",
meta={"language": "de"},
),
]
router = MetadataRouter(
rules={"en": {"field": "meta.language", "operator": "==", "value": "en"}},
)
router.run(documents=docs)
```
### Routing ByteStreams
You can also use `MetadataRouter` to route `ByteStream` objects based on their metadata. This is useful when working with binary data or when you need to route files before they're converted to documents.
```python
from haystack.dataclasses import ByteStream
from haystack.components.routers import MetadataRouter
streams = [
ByteStream.from_string("Hello world", meta={"language": "en"}),
ByteStream.from_string("Bonjour le monde", meta={"language": "fr"}),
]
router = MetadataRouter(
rules={"english": {"field": "meta.language", "operator": "==", "value": "en"}},
output_type=list[ByteStream],
)
result = router.run(documents=streams)
# {'english': [ByteStream(...)], 'unmatched': [ByteStream(...)]}
```
### In a pipeline
Below is an example of an indexing pipeline that converts text files to documents and uses the `DocumentLanguageClassifier` to detect the language of the text and add it to the documents' metadata. It then uses the `MetadataRouter` to forward only English language documents to the `DocumentWriter`. Documents of other languages will not be added to the `DocumentStore`.
The examples on this page use language classification components from the `langdetect-haystack` package. Install it to run the examples:
```shell
pip install langdetect-haystack
```
```python
from haystack import Pipeline
from haystack.components.converters import TextFileToDocument
from haystack_integrations.components.classifiers.langdetect import (
DocumentLanguageClassifier,
)
from haystack.components.routers import MetadataRouter
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component(instance=TextFileToDocument(), name="text_file_converter")
p.add_component(instance=DocumentLanguageClassifier(), name="language_classifier")
p.add_component(
instance=MetadataRouter(
rules={"en": {"field": "meta.language", "operator": "==", "value": "en"}},
),
name="router",
)
p.add_component(instance=DocumentWriter(document_store=document_store), name="writer")
p.connect("text_file_converter.documents", "language_classifier.documents")
p.connect("language_classifier.documents", "router.documents")
p.connect("router.en", "writer.documents")
p.run(
{
"text_file_converter": {
"sources": [
"english-file-will-be-added.txt",
"german-file-will-not-be-added.txt",
],
},
},
)
```
---
// File: pipeline-components/routers/textlanguagerouter
# TextLanguageRouter
Use this component in pipelines to route a query based on its language.
`
| | |
| --- | --- |
| **Most common position in a pipeline** | As the first component to route a query to different [Retrievers](../retrievers.mdx) , based on its language |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `text`: A string |
| **Output variables** | `unmatched`: A string
``: A string (where `` is defined during initialization). For example: `fr`: French language string. |
| **API reference** | [Langdetect](/reference/integrations-langdetect) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/langdetect |
| **Package name** | `langdetect-haystack` |
## Overview
`TextLanguageRouter` detects the language of an input string and routes it to an output named after the language if it's in the set of languages the component was initialized with. By default, only English is in this set. If the detected language of the input text is not in the component’s `languages` , it's routed to an output named `unmatched`.
In pipelines, it's used as the first component to route a query based on its language and filter out queries in unsupported languages.
The components parameter `languages` must be a list of languages in ISO code, such as en, de, fr, es, it, each corresponding to a different output connection (see [langdetect documentation](https://github.com/Mimino666/langdetect#languages))).
## Usage
Install the `langdetect-haystack` package to use the `TextLanguageRouter` component:
```shell
pip install langdetect-haystack
```
### On its own
Below is an example where using the `TextLanguageRouter` to route only French texts to an output connection named `fr`. Other texts, such as the English text below, are routed to an output named `unmatched`.
```python
from haystack_integrations.components.routers.langdetect import TextLanguageRouter
router = TextLanguageRouter(languages=["fr"])
router.run(text="What's your query?")
```
### In a pipeline
Below is an example of a query pipeline that uses a `TextLanguageRouter` to forward only English language queries to the Retriever.
```python
from haystack import Pipeline
from haystack_integrations.components.routers.langdetect import TextLanguageRouter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
document_store = InMemoryDocumentStore()
p = Pipeline()
p.add_component(instance=TextLanguageRouter(), name="text_language_router")
p.add_component(
instance=InMemoryBM25Retriever(document_store=document_store),
name="retriever",
)
p.connect("text_language_router.en", "retriever.query")
p.run({"text_language_router": {"text": "What's your query?"}})
```
---
// File: pipeline-components/routers/transformerstextrouter
# TransformersTextRouter
Use this component to route text input to various output connections based on a model-defined categorization label.
`
| | |
| --- | --- |
| **Most common position in a pipeline** | Flexible |
| **Mandatory init variables** | `model`: The name or path of a Hugging Face model for text classification |
| **Mandatory run variables** | `text`: The text to be routed to one of the specified outputs based on which label it has been categorized into |
| **Output variables** | `
## Overview
`TransformersTextRouter` routes text input to various output connections based on its categorization label. This is useful for routing queries to different models in a pipeline depending on their categorization.
First, you need to set a selected model with a `model` parameter when initializing the component. The selected model then provides the set of labels for categorization.
You can additionally provide the `labels` parameter – a list of strings of possible class labels to classify each sequence into. If not provided, the component fetches the labels from the model configuration file hosted on the HuggingFace Hub using `transformers.AutoConfig.from_pretrained`.
Authentication with a Hugging Face API token is only required to access private or gated models. You can pass the token at initialization with `token`, or set the `HF_API_TOKEN` or `HF_TOKEN` environment variable.
To see the full list of parameters, check out our [API reference](/reference/integrations-transformers#transformerstextrouter).
## Usage
Install the `transformers-haystack` package to use the `TransformersTextRouter`:
```shell
pip install transformers-haystack
```
### On its own
The `TransformersTextRouter` isn’t very effective on its own, as its main strength lies in working within a pipeline. The component's true potential is unlocked when it is integrated into a pipeline, where it can efficiently route text to the most appropriate components. Please see the following section for a complete example of usage.
### In a pipeline
Below is an example of a simple pipeline that routes English queries to a Text Generator optimized for English text and German queries to a Text Generator optimized for German text.
```python
from haystack import Pipeline
from haystack_integrations.components.routers.transformers import TransformersTextRouter
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack_integrations.components.generators.transformers import (
TransformersChatGenerator,
)
from haystack.dataclasses import ChatMessage
p = Pipeline()
p.add_component(
instance=TransformersTextRouter(
model="papluca/xlm-roberta-base-language-detection",
),
name="text_router",
)
p.add_component(
instance=ChatPromptBuilder(
template=[ChatMessage.from_user("Answer the question: {{query}}\nAnswer:")],
required_variables={"query"},
),
name="english_prompt_builder",
)
p.add_component(
instance=ChatPromptBuilder(
template=[ChatMessage.from_user("Beantworte die Frage: {{query}}\nAntwort:")],
required_variables={"query"},
),
name="german_prompt_builder",
)
p.add_component(
instance=TransformersChatGenerator(
model="DiscoResearch/Llama3-DiscoLeo-Instruct-8B-v0.1",
),
name="german_llm",
)
p.add_component(
instance=TransformersChatGenerator(model="microsoft/Phi-3-mini-4k-instruct"),
name="english_llm",
)
p.connect("text_router.en", "english_prompt_builder.query")
p.connect("text_router.de", "german_prompt_builder.query")
p.connect("english_prompt_builder.prompt", "english_llm.messages")
p.connect("german_prompt_builder.prompt", "german_llm.messages")
# English Example
print(p.run({"text_router": {"text": "What is the capital of Germany?"}}))
# German Example
print(p.run({"text_router": {"text": "Was ist die Hauptstadt von Deutschland?"}}))
```
## Additional References
:notebook: Tutorial: [Query Classification with TransformersTextRouter and TransformersZeroShotTextRouter](https://haystack.deepset.ai/tutorials/41_query_classification_with_transformerstextrouter_and_transformerszeroshottextrouter)
---
// File: pipeline-components/routers/transformerszeroshottextrouter
# TransformersZeroShotTextRouter
Use this component to route text input to various output connections based on its user-defined categorization label.
| | |
| --- | --- |
| **Most common position in a pipeline** | Flexible |
| **Mandatory init variables** | `labels`: A list of labels for classification |
| **Mandatory run variables** | `text`: The text to be routed to one of the specified outputs based on which label it has been categorized into |
| **Output variables** | `
## Overview
`TransformersZeroShotTextRouter` routes text input to various output connections based on its categorization label. This feature is especially beneficial for directing queries to appropriate components within a pipeline, according to their specific categories. Users can define the labels for this categorization process.
`TransformersZeroShotTextRouter` uses the `MoritzLaurer/deberta-v3-base-zeroshot-v1.1-all-33` zero-shot text classification model by default. You can set another model of your choosing with the `model` parameter.
To use `TransformersZeroShotTextRouter`, you need to provide the mandatory `labels` parameter – a list of strings of possible class labels to classify each sequence into.
Authentication with a Hugging Face API token is only required to access private or gated models. You can pass the token at initialization with `token`, or set the `HF_API_TOKEN` or `HF_TOKEN` environment variable.
To see the full list of parameters, check out our [API reference](/reference/integrations-transformers#transformerszeroshottextrouter).
## Usage
Install the `transformers-haystack` package to use the `TransformersZeroShotTextRouter`:
```shell
pip install transformers-haystack
```
### On its own
The `TransformersZeroShotTextRouter` isn’t very effective on its own, as its main strength lies in working within a pipeline. The component's true potential is unlocked when it is integrated into a pipeline, where it can efficiently route text to the most appropriate components. Please see the following section for a complete example of usage.
### In a pipeline
Below is an example of a simple pipeline that routes input text to an appropriate route in the pipeline.
We first create an `InMemoryDocumentStore` and populate it with documents about Germany and France, embedding these documents using `SentenceTransformersDocumentEmbedder`.
We then create a retrieving pipeline with the `TransformersZeroShotTextRouter` to categorize an incoming text as either "passage" or "query" based on these predefined labels. Depending on the categorization, the text is then processed by appropriate Embedders tailored for passages and queries, respectively. These Embedders generate embeddings that are used by `InMemoryEmbeddingRetriever` to find relevant documents in the Document Store.
Finally, the pipeline is executed with a sample text: "What is the capital of Germany?” which categorizes this input text as “query” and routes it to Query Embedder and subsequently Query Retriever to return the relevant results.
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.core.pipeline import Pipeline
from haystack_integrations.components.routers.transformers import TransformersZeroShotTextRouter
from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder, SentenceTransformersDocumentEmbedder
from haystack.components.retrievers import InMemoryEmbeddingRetriever
document_store = InMemoryDocumentStore()
doc_embedder = SentenceTransformersDocumentEmbedder(model="intfloat/e5-base-v2")
docs = [
Document(
content="Germany, officially the Federal Republic of Germany, is a country in the western region of "
"Central Europe. The nation's capital and most populous city is Berlin and its main financial centre "
"is Frankfurt; the largest urban area is the Ruhr."
),
Document(
content="France, officially the French Republic, is a country located primarily in Western Europe. "
"France is a unitary semi-presidential republic with its capital in Paris, the country's largest city "
"and main cultural and commercial centre; other major urban areas include Marseille, Lyon, Toulouse, "
"Lille, Bordeaux, Strasbourg, Nantes and Nice."
)
]
docs_with_embeddings = doc_embedder.run(docs)
document_store.write_documents(docs_with_embeddings["documents"])
p = Pipeline()
p.add_component(instance=TransformersZeroShotTextRouter(labels=["passage", "query"]), name="text_router")
p.add_component(
instance=SentenceTransformersTextEmbedder(model="intfloat/e5-base-v2", prefix="passage: "),
name="passage_embedder"
)
p.add_component(
instance=SentenceTransformersTextEmbedder(model="intfloat/e5-base-v2", prefix="query: "),
name="query_embedder"
)
p.add_component(
instance=InMemoryEmbeddingRetriever(document_store=document_store),
name="query_retriever"
)
p.add_component(
instance=InMemoryEmbeddingRetriever(document_store=document_store),
name="passage_retriever"
)
p.connect("text_router.passage", "passage_embedder.text")
p.connect("passage_embedder.embedding", "passage_retriever.query_embedding")
p.connect("text_router.query", "query_embedder.text")
p.connect("query_embedder.embedding", "query_retriever.query_embedding")
# Query Example
result = p.run({"text_router": {"text": "What is the capital of Germany?"}})
print(result)
>>{'query_retriever': {'documents': [Document(id=32d393dd8ee60648ae7e630cfe34b1922e747812ddf9a2c8b3650e66e0ecdb5a,
content: 'Germany, officially the Federal Republic of Germany, is a country in the western region of Central E...',
score: 0.8625669285150891), Document(id=c17102d8d818ce5cdfee0288488c518f5c9df238a9739a080142090e8c4cb3ba,
content: 'France, officially the French Republic, is a country located primarily in Western Europe. France is ...',
score: 0.7637571978602222)]}}
```
## Additional References
:notebook: Tutorial: [Query Classification with TransformersTextRouter and TransformersZeroShotTextRouter](https://haystack.deepset.ai/tutorials/41_query_classification_with_transformerstextrouter_and_transformerszeroshottextrouter)
---
// File: pipeline-components/routers
# Routers
Routers is a group of components that route queries or documents to other components that can handle them best.
| Component | Description |
| --- | --- |
| [ConditionalRouter](routers/conditionalrouter.mdx) | Routes data based on specified conditions. |
| [DocumentLengthRouter](routers/documentlengthrouter.mdx) | Routes documents to different output connections based on the length of their `content` field. |
| [DocumentTypeRouter](routers/documenttyperouter.mdx) | Routes documents based on their MIME types to different outputs for further processing. |
| [FileTypeRouter](routers/filetyperouter.mdx) | Routes file paths or byte streams based on their type further down the pipeline. |
| [LLMMessagesRouter](routers/llmmessagesrouter.mdx) | Routes Chat Messages to various output connections using a generative Language Model to perform classification. |
| [MetadataRouter](routers/metadatarouter.mdx) | Routes documents based on their metadata field values. |
| [TextLanguageRouter](routers/textlanguagerouter.mdx) | Routes queries based on their language. |
| [TransformersTextRouter](routers/transformerstextrouter.mdx) | Routes text input to various output connections based on a model-defined categorization label. |
| [TransformersZeroShotTextRouter](routers/transformerszeroshottextrouter.mdx) | Routes text input to various output connections based on user-defined categorization label. |
---
// File: pipeline-components/samplers/toppsampler
# TopPSampler
Uses nucleus sampling to filter documents.
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [Ranker](../rankers.mdx) |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Samplers](/reference/samplers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/samplers/top_p.py |
| **Package name** | `haystack-ai` |
## Overview
Top-P (nucleus) sampling is a method that helps identify and select a subset of documents based on their cumulative probabilities. Instead of choosing a fixed number of documents, this method focuses on a specified percentage of the highest cumulative probabilities within a list of documents. To put it simply, `TopPSampler` provides a way to efficiently select the most relevant documents based on their similarity to a given query.
The practical goal of the `TopPSampler` is to return a list of documents that, in sum, have a score larger than the `top_p` value. So, for example, when `top_p` is set to a high value, more documents will be returned, which can result in more varied outputs. The value is typically set between 0 and 1. By default, the component uses documents' `score` fields to look at the similarity scores.
The component’s `run()` method takes in a set of documents that already carry scores and filters them based on the cumulative probability of those scores. It doesn't compute scores itself, so place it after a component that does, such as a Ranker.
## Usage
### On its own
```python
from haystack import Document
from haystack.components.samplers import TopPSampler
sampler = TopPSampler(top_p=0.99, score_field="similarity_score")
docs = [
Document(content="Berlin", meta={"similarity_score": -10.6}),
Document(content="Belgrade", meta={"similarity_score": -8.9}),
Document(content="Sarajevo", meta={"similarity_score": -4.6}),
]
output = sampler.run(documents=docs)
docs = output["documents"]
print(docs)
```
### In a pipeline
To best understand how can you use a `TopPSampler` and which components to pair it with, explore the following example.
The examples on this page use Sentence Transformers rankers and the SerperDev web search component from the `sentence-transformers-haystack` and `serperdev-haystack` packages. Install them to run the examples:
```shell
pip install sentence-transformers-haystack serperdev-haystack
```
```python
# import necessary dependencies
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.preprocessors import DocumentSplitter
from haystack_integrations.components.rankers.sentence_transformers import (
SentenceTransformersSimilarityRanker,
)
from haystack.components.routers.file_type_router import FileTypeRouter
from haystack.components.samplers import TopPSampler
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.utils import Secret
from haystack.dataclasses import ChatMessage
# initialize the components
web_search = SerperDevWebSearch(api_key=Secret.from_token("
| | |
| --- | --- |
| **Most common position in a pipeline** | After any component that produces documents, such as a Retriever or a Converter |
| **Mandatory init variables** | `access_key_id`: Lara API access key ID. Can be set with `LARA_ACCESS_KEY_ID` env var.
`access_key_secret`: Lara API access key secret. Can be set with `LARA_ACCESS_KEY_SECRET` env var. | | **Mandatory run variables** | `documents`: A list of documents to be translated | | **Output variables** | `documents`: A list of translated documents | | **API reference** | [Lara](/reference/integrations-lara) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/lara | | **Package name** | `lara-haystack` |
## Overview
[Lara](https://developers.laratranslate.com/docs/introduction) is an adaptive translation AI by [translated](https://translated.com/) that combines the fluency and context handling of LLMs with low hallucination and latency. It adapts to domains at inference time using optional context, instructions, translation memories, and glossaries.
`LaraDocumentTranslator` takes a list of Haystack documents, translates their text content via the Lara API, and returns new documents containing the translations. The original document ID is preserved in each translated document's metadata under the `original_document_id` key.
Key features:
- **Automatic language detection**: set `source_lang` to `None` and Lara auto-detects it.
- **Translation styles**: choose `"faithful"`, `"fluid"`, or `"creative"` to control the tone.
- **Context and instructions**: pass surrounding text or natural-language instructions to improve quality.
- **Translation memories and glossaries**: supply memory or glossary IDs so Lara enforces consistent terminology.
- **Reasoning (Lara Think)**: enable multi-step linguistic analysis for higher-quality output.
## Usage
### Installation
To start using this integration with Haystack, install it with:
```shell
pip install lara-haystack
```
`LaraDocumentTranslator` needs Lara API credentials to work. It uses the `LARA_ACCESS_KEY_ID` and `LARA_ACCESS_KEY_SECRET` environment variables by default. Otherwise, you can pass them at initialization:
```python
from haystack.utils import Secret
from haystack_integrations.components.translators.lara import LaraDocumentTranslator
translator = LaraDocumentTranslator(
access_key_id=Secret.from_token("`access_key_secret`: Lara API access key secret. Can be set with `LARA_ACCESS_KEY_SECRET` env var. | | **Mandatory run variables** | `documents`: A list of documents to be translated | | **Output variables** | `documents`: A list of translated documents | | **API reference** | [Lara](/reference/integrations-lara) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/lara | | **Package name** | `lara-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [Generator](../generators.mdx) |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) instances to be validated – the last message in this list is the one that is validated |
| **Output variables** | `validated`: A list of messages if the last message is valid
`validation_error`: A list of messages if the last message is invalid | | **API reference** | [Validators](/reference/validators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/validators/json_schema.py | | **Package name** | `haystack-ai` |
## Overview
`JsonSchemaValidator` checks the JSON content of a `ChatMessage` against a given [JSON Schema](https://json-schema.org/). If a message's JSON content follows the provided schema, it's moved to the `validated` output. If not, it's moved to the `validation_error`output. When there's an error, the component uses either the provided custom `error_template` or a default template to create the error message. These error `ChatMessages` can be used in Haystack recovery loops.
## Usage
### In a pipeline
In this simple pipeline, the `MessageProducer` sends a list of chat messages to a Generator through `BranchJoiner`. The resulting messages from the Generator are sent to `JsonSchemaValidator`, and the error `ChatMessages` are sent back to `BranchJoiner` for a recovery loop.
```python
from typing import List
from haystack import Pipeline
from haystack import component
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.joiners import BranchJoiner
from haystack.components.validators import JsonSchemaValidator
from haystack.dataclasses import ChatMessage
@component
class MessageProducer:
@component.output_types(messages=List[ChatMessage])
def run(self, messages: List[ChatMessage]) -> dict:
return {"messages": messages}
p = Pipeline()
p.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini",
generation_kwargs={"response_format": {"type": "json_object"}}))
p.add_component("schema_validator", JsonSchemaValidator())
p.add_component("branch_joiner", BranchJoiner(List[ChatMessage]))
p.add_component("message_producer", MessageProducer())
p.connect("message_producer.messages", "branch_joiner")
p.connect("branch_joiner", "llm")
p.connect("llm.replies", "schema_validator.messages")
p.connect("schema_validator.validation_error", "branch_joiner")
result = p.run(
data={"message_producer": {
"messages": [ChatMessage.from_user("Generate JSON for person with name 'John' and age 30")]},
"schema_validator": {"json_schema": {"type": "object",
"properties": {"name": {"type": "string"},
"age": {"type": "integer"}}}}})
print(result)
>> {'schema_validator': {'validated': [ChatMessage(_role=`validation_error`: A list of messages if the last message is invalid | | **API reference** | [Validators](/reference/validators-api) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/validators/json_schema.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Brave Search API key. Can be set with the `BRAVE_API_KEY` env var. |
| **Mandatory run variables** | `query`: A string with your search query. |
| **Output variables** | `documents`: A list of Haystack Documents containing search result content and metadata.
`links`: A list of strings of resulting URLs. | | **API reference** | [Brave Search API](/reference/integrations-brave) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/brave/src/haystack_integrations/components/websearch/brave/brave_websearch.py | | **Package name** | `brave-haystack` |
## Overview
When you give `BraveWebSearch` a query, it uses the [Brave Search API](https://brave.com/search/api/) to search the web and return relevant content as Haystack `Document` objects. It also returns a list of the source URLs.
Brave Search is an independent search engine with its own web index. It is a great fit for RAG pipelines that need reliable, privacy-focused web results without depending on Google or Bing.
`BraveWebSearch` requires a Brave Search API key to work. By default, it looks for a `BRAVE_API_KEY` environment variable. Alternatively, you can pass an `api_key` directly during initialization.
## Usage
### On its own
Here is a quick example of how `BraveWebSearch` searches the web based on a query and returns a list of Documents.
```python
from haystack_integrations.components.websearch.brave import BraveWebSearch
from haystack.utils import Secret
web_search = BraveWebSearch(
api_key=Secret.from_env_var("BRAVE_API_KEY"),
top_k=5,
)
query = "What is Haystack by deepset?"
response = web_search.run(query=query)
for doc in response["documents"]:
print(doc.content)
```
### In a pipeline
Here is an example of a Retrieval-Augmented Generation (RAG) pipeline that uses `BraveWebSearch` to look up an answer on the web.
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.components.websearch.brave import BraveWebSearch
from haystack.dataclasses import ChatMessage
web_search = BraveWebSearch(
api_key=Secret.from_env_var("BRAVE_API_KEY"),
top_k=3,
)
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given the information below:\n"
"{% for document in documents %}{{ document.content }}\n{% endfor %}\n"
"Answer the following question: {{ query }}.\nAnswer:",
),
]
prompt_builder = ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
)
llm = OpenAIChatGenerator(
api_key=Secret.from_env_var("OPENAI_API_KEY"),
)
pipe = Pipeline()
pipe.add_component("search", web_search)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("search.documents", "prompt_builder.documents")
pipe.connect("prompt_builder.prompt", "llm.messages")
query = "What is Haystack by deepset?"
result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/websearch/ddgswebsearch
# DDGSWebSearch
Search the web with ddgs (Dux Distributed Global Search), a metasearch library that aggregates results from multiple search engines without an API key.
`links`: A list of strings of resulting URLs. | | **API reference** | [Brave Search API](/reference/integrations-brave) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/brave/src/haystack_integrations/components/websearch/brave/brave_websearch.py | | **Package name** | `brave-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | None. `ddgs` requires no API key. |
| **Mandatory run variables** | `query`: A string with your search query. |
| **Output variables** | `documents`: A list of Haystack Documents containing search result snippets, with the result title and URL in the metadata.
`links`: A list of strings of resulting URLs. | | **API reference** | [ddgs API](/reference/integrations-ddgs) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/ddgs/src/haystack_integrations/components/websearch/ddgs/ddgs_websearch.py | | **Package name** | `ddgs-haystack` |
## Overview
When you give `DDGSWebSearch` a query, it uses [ddgs](https://github.com/deedy5/ddgs) to search the web and return the result snippets as Haystack `Document` objects. It also returns a list of the source URLs.
Unlike the other websearch components, `DDGSWebSearch` needs **no API key and no account**. `ddgs` is a free metasearch library that queries public search engines directly, aggregating results from backends such as DuckDuckGo, Google, Bing, Brave, Yahoo, Yandex, and Mullvad.
You can configure the search with:
- `backend`: A comma-separated list of ddgs backends to query, for example `"duckduckgo, google, brave"`, or `"auto"` to let ddgs choose. See the [ddgs documentation](https://github.com/deedy5/ddgs) for the full list of backends.
- `region`: The region and locale of the search, for example `"us-en"`, `"de-de"`, or `"wt-wt"` for no region.
- `safesearch`: The safe-search level, one of `"on"`, `"moderate"`, or `"off"`.
- `top_k`: The maximum number of results to return.
- `search_params`: Additional keyword arguments forwarded to the underlying `DDGS().text()` call, such as `page` or `timelimit`. Values you set here take precedence over `backend`, `region`, `safesearch`, and `top_k`.
All of these can be overridden for a single search by passing them to `run()`. Note that a `search_params` dictionary passed to `run()` fully replaces the one set at initialization instead of being merged with it.
`DDGSWebSearch` also supports asynchronous execution through `run_async()`. Because `ddgs` has no native async API, the blocking search runs in a worker thread. The underlying client is created lazily on the first search. To avoid the cold-start latency of the first call, you can call `warm_up()` explicitly.
:::note[Best-effort results]
`ddgs` queries public search engines without an API contract, so results are best-effort: they can differ between runs, and heavy use may be throttled or temporarily blocked. For production workloads that need predictable rate limits, consider a component backed by a commercial search API, such as [`TavilyWebSearch`](tavilywebsearch.mdx) or [`SerperDevWebSearch`](serperdevwebsearch.mdx).
:::
## Usage
Install the `ddgs-haystack` package to use the `DDGSWebSearch` component:
```shell
pip install ddgs-haystack
```
### On its own
Here is a quick example of how `DDGSWebSearch` searches the web based on a query and returns a list of Documents. No API key is needed.
```python
from haystack_integrations.components.websearch.ddgs import DDGSWebSearch
web_search = DDGSWebSearch(top_k=5)
query = "What is Haystack by deepset?"
response = web_search.run(query=query)
for doc in response["documents"]:
print(doc.meta["url"])
print(doc.content)
```
To search with specific backends and in a specific region:
```python
web_search = DDGSWebSearch(
top_k=5,
backend="duckduckgo, brave",
region="de-de",
safesearch="off",
)
```
### In a pipeline
Here is an example of a Retrieval-Augmented Generation (RAG) pipeline that uses `DDGSWebSearch` to look up an answer on the web.
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.components.websearch.ddgs import DDGSWebSearch
from haystack.dataclasses import ChatMessage
web_search = DDGSWebSearch(top_k=3)
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given the information below:\n"
"{% for document in documents %}{{ document.content }}\n{% endfor %}\n"
"Answer the following question: {{ query }}.\nAnswer:",
),
]
prompt_builder = ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
)
llm = OpenAIChatGenerator(
api_key=Secret.from_env_var("OPENAI_API_KEY"),
)
pipe = Pipeline()
pipe.add_component("search", web_search)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("search.documents", "prompt_builder.documents")
pipe.connect("prompt_builder.prompt", "llm.messages")
query = "What is Haystack by deepset?"
result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})
print(result["llm"]["replies"][0].text)
```
Because `ddgs` returns only short snippets rather than full page content, you can add a [`LinkContentFetcher`](../fetchers/linkcontentfetcher.mdx) and a converter after the search to fetch and read the actual web pages when you need more context.
---
// File: pipeline-components/websearch/external-integrations-websearch
# External Integrations
External integrations that enable websearch with Haystack.
| Name | Description |
| --- | --- |
| [DuckDuckGo](https://haystack.deepset.ai/integrations/duckduckgo-api-websearch) | Use DuckDuckGo API for web searches. |
| [Exa](https://haystack.deepset.ai/integrations/exa) | Search the web with Exa's AI-powered search, get content, answers, and conduct deep research. |
| [Serpex](https://haystack.deepset.ai/integrations/serpex) | Multi-engine web search for Haystack — access Google, Bing, DuckDuckGo, Brave, Yahoo, and Yandex via Serpex API. |
---
// File: pipeline-components/websearch/firecrawlwebsearch
# FirecrawlWebSearch
Search the web and extract content using the Firecrawl API.
`links`: A list of strings of resulting URLs. | | **API reference** | [ddgs API](/reference/integrations-ddgs) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/ddgs/src/haystack_integrations/components/websearch/ddgs/ddgs_websearch.py | | **Package name** | `ddgs-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or right at the beginning of an indexing pipeline. |
| **Mandatory init variables** | `api_key`: The Firecrawl API key. Can be set with the `FIRECRAWL_API_KEY` env var. |
| **Mandatory run variables** | `query`: A string with your search query. |
| **Output variables** | `documents`: A list of Haystack Documents containing the scraped content and metadata.
`links`: A list of strings of resulting URLs. | | **API reference** | [Firecrawl Search API](/reference/integrations-firecrawl) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/firecrawl/src/haystack_integrations/components/websearch/firecrawl/firecrawl_websearch.py | | **Package name** | `firecrawl-haystack` |
## Overview
When you give `FirecrawlWebSearch` a query, it uses the Firecrawl Search API to search the web, crawl the resulting pages, and return the structured text as a list of Haystack `Document` objects. It also returns a list of the underlying URLs.
Because Firecrawl actively scrapes and structures the content of the pages it finds into LLM-friendly formats, you generally don't need an additional component like `LinkContentFetcher` to read the web pages. `FirecrawlWebSearch` handles the retrieval and scraping all in one step.
`FirecrawlWebSearch` requires a [Firecrawl](https://firecrawl.dev) API key to work. By default, it looks for a `FIRECRAWL_API_KEY` environment variable. Alternatively, you can pass an `api_key` directly during initialization.
## Usage
### On its own
Here is a quick example of how `FirecrawlWebSearch` searches the web based on a query, scrapes the resulting web pages, and returns a list of Documents containing the page content.
```python
from haystack_integrations.components.websearch.firecrawl import FirecrawlWebSearch
from haystack.utils import Secret
web_search = FirecrawlWebSearch(
api_key=Secret.from_env_var("FIRECRAWL_API_KEY"),
top_k=5,
search_params={"scrape_options": {"formats": ["markdown"]}},
)
query = "What is Haystack by deepset?"
response = web_search.run(query=query)
for doc in response["documents"]:
print(doc.content)
```
### In a pipeline
Here is an example of a Retrieval-Augmented Generation (RAG) pipeline where using `FirecrawlWebSearch` to look up an answer. Because Firecrawl returns the actual text of the scraped pages, you can pass its `documents` output directly into the `ChatPromptBuilder` to give the LLM the necessary context.
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.components.websearch.firecrawl import FirecrawlWebSearch
from haystack.dataclasses import ChatMessage
web_search = FirecrawlWebSearch(
api_key=Secret.from_env_var("FIRECRAWL_API_KEY"),
top_k=2,
search_params={"scrape_options": {"formats": ["markdown"]}},
)
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given the information below:\n"
"{% for document in documents %}{{ document.content }}\n{% endfor %}\n"
"Answer the following question: {{ query }}.\nAnswer:",
),
]
prompt_builder = ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
)
llm = OpenAIChatGenerator(
api_key=Secret.from_env_var("OPENAI_API_KEY"),
model="gpt-5-nano",
)
pipe = Pipeline()
pipe.add_component("search", web_search)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("search.documents", "prompt_builder.documents")
pipe.connect("prompt_builder.prompt", "llm.messages")
query = "What is Haystack by deepset?"
result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/websearch/linkupwebsearch
# LinkupWebSearch
Search the web using the Linkup Search API.
`links`: A list of strings of resulting URLs. | | **API reference** | [Firecrawl Search API](/reference/integrations-firecrawl) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/firecrawl/src/haystack_integrations/components/websearch/firecrawl/firecrawl_websearch.py | | **Package name** | `firecrawl-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Linkup API key. Can be set with the `LINKUP_API_KEY` env var. |
| **Mandatory run variables** | `query`: A string with your search query. |
| **Output variables** | `documents`: A list of Haystack Documents containing search result content, with the result title and URL in the metadata.
`links`: A list of strings of resulting URLs. | | **API reference** | [Linkup Search API](/reference/integrations-linkup) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/linkup/src/haystack_integrations/components/websearch/linkup/linkup_websearch.py | | **Package name** | `linkup-haystack` |
## Overview
When you give `LinkupWebSearch` a query, it uses the [Linkup](https://www.linkup.so) Search API to search the web and returns the results as Haystack `Document` objects, together with a list of the source URLs.
Each result becomes a `Document` whose content is the text Linkup returns for that result, with the result title and URL stored in the Document's `meta`.
Use the `depth` parameter to trade latency for thoroughness:
- `"fast"`: keyword-based queries only, sub-second response (beta).
- `"standard"`: a single search pass. This is the default.
- `"deep"`: runs an agentic workflow, which takes longer.
`top_k` limits the number of results and maps to the `max_results` parameter of the Linkup API. To use additional API options, such as `include_images`, `from_date`, `to_date`, `include_domains`, or `exclude_domains`, pass them in `search_params`. See the [Linkup API reference](https://docs.linkup.so/pages/documentation/api-reference/endpoint/post-search) for all available options. Image results carry no text, so enabling `include_images` adds Documents with empty content.
You can override `top_k`, `depth`, and `search_params` for a single search by passing them to `run()`. Note that a `search_params` dictionary passed to `run()` fully replaces the one set at initialization instead of being merged with it.
`LinkupWebSearch` also supports asynchronous execution through `run_async()`. The underlying client is created lazily on the first search. To avoid the cold-start latency of the first call, you can call `warm_up()` explicitly.
`LinkupWebSearch` requires a Linkup API key to work. By default, it looks for a `LINKUP_API_KEY` environment variable. Alternatively, you can pass an `api_key` directly during initialization.
## Usage
Install the `linkup-haystack` package to use the `LinkupWebSearch` component:
```shell
pip install linkup-haystack
```
### On its own
Here is a quick example of how `LinkupWebSearch` searches the web based on a query and returns a list of Documents.
```python
from haystack_integrations.components.websearch.linkup import LinkupWebSearch
from haystack.utils import Secret
web_search = LinkupWebSearch(
api_key=Secret.from_env_var("LINKUP_API_KEY"),
top_k=5,
depth="standard",
)
query = "What is Haystack by deepset?"
response = web_search.run(query=query)
for doc in response["documents"]:
print(doc.meta["url"])
print(doc.content)
```
### In a pipeline
Here is an example of a Retrieval-Augmented Generation (RAG) pipeline that uses `LinkupWebSearch` to look up an answer on the web.
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.components.websearch.linkup import LinkupWebSearch
from haystack.dataclasses import ChatMessage
web_search = LinkupWebSearch(
api_key=Secret.from_env_var("LINKUP_API_KEY"),
top_k=3,
)
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given the information below:\n"
"{% for document in documents %}{{ document.content }}\n{% endfor %}\n"
"Answer the following question: {{ query }}.\nAnswer:",
),
]
prompt_builder = ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
)
llm = OpenAIChatGenerator(
api_key=Secret.from_env_var("OPENAI_API_KEY"),
)
pipe = Pipeline()
pipe.add_component("search", web_search)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("search.documents", "prompt_builder.documents")
pipe.connect("prompt_builder.prompt", "llm.messages")
query = "What is Haystack by deepset?"
result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/websearch/perplexitywebsearch
# PerplexityWebSearch
Search the web using the Perplexity Search API.
`links`: A list of strings of resulting URLs. | | **API reference** | [Linkup Search API](/reference/integrations-linkup) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/linkup/src/haystack_integrations/components/websearch/linkup/linkup_websearch.py | | **Package name** | `linkup-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or at the beginning of an indexing pipeline |
| **Mandatory init variables** | `api_key`: A Perplexity API key. Can be set with `PERPLEXITY_API_KEY` env var. |
| **Mandatory run variables** | `query`: A string with your search query. |
| **Output variables** | `documents`: A list of Haystack Documents containing search result content and metadata.
`links`: A list of strings of resulting URLs. | | **API reference** | [Integrations](/reference/integrations-perplexity) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/perplexity/src/haystack_integrations/components/websearch/perplexity/perplexity_websearch.py | | **Package name** | `perplexity-haystack` |
## Overview
When you give `PerplexityWebSearch` a query, it uses the [Perplexity Search API](https://docs.perplexity.ai/) to search the web and return relevant content as Haystack `Document` objects. It also returns a list of the source URLs.
Each returned `Document` contains a text snippet as its `content` and a `meta` dictionary with `title`, `url`, `date`, and `last_updated` fields.
`PerplexityWebSearch` requires a Perplexity API key to work. By default, it reads from the `PERPLEXITY_API_KEY` environment variable. You can also pass an `api_key` directly during initialization.
The `top_k` parameter controls the maximum number of results returned (between 1 and 20, default is 10).
You can filter and refine search results using `search_params`, which supports keys such as `country`, `search_recency_filter`, `search_domain_filter`, and date range filters. These can be set at initialization or overridden per `run()` call. See the [Perplexity Search API reference](https://docs.perplexity.ai/api-reference/search-post) for the full list of parameters.
`PerplexityWebSearch` supports both synchronous (`run()`) and asynchronous (`run_async()`) operation.
## Usage
### On its own
```python
from haystack.utils import Secret
from haystack_integrations.components.websearch.perplexity import PerplexityWebSearch
web_search = PerplexityWebSearch(
api_key=Secret.from_env_var("PERPLEXITY_API_KEY"),
top_k=5,
)
result = web_search.run(query="What is Haystack by deepset?")
for doc in result["documents"]:
print(doc.content)
print(doc.meta["url"])
```
With search filters:
```python
from haystack.utils import Secret
from haystack_integrations.components.websearch.perplexity import PerplexityWebSearch
web_search = PerplexityWebSearch(
api_key=Secret.from_env_var("PERPLEXITY_API_KEY"),
top_k=5,
search_params={"country": "us", "search_recency_filter": "week"},
)
result = web_search.run(query="Latest AI research papers")
for doc in result["documents"]:
print(doc.meta["title"], doc.meta["url"])
```
### In a pipeline
Here is an example of a RAG pipeline that uses `PerplexityWebSearch` to look up an answer on the web.
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.perplexity import (
PerplexityChatGenerator,
)
from haystack_integrations.components.websearch.perplexity import PerplexityWebSearch
web_search = PerplexityWebSearch(
api_key=Secret.from_env_var("PERPLEXITY_API_KEY"),
top_k=3,
)
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given the information below:\n"
"{% for document in documents %}{{ document.content }}\n{% endfor %}\n"
"Answer the following question: {{ query }}.\nAnswer:",
),
]
prompt_builder = ChatPromptBuilder(
template=prompt_template,
required_variables=["query", "documents"],
)
llm = PerplexityChatGenerator(
api_key=Secret.from_env_var("PERPLEXITY_API_KEY"),
)
pipe = Pipeline()
pipe.add_component("search", web_search)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("search.documents", "prompt_builder.documents")
pipe.connect("prompt_builder.prompt", "llm.messages")
query = "What is Haystack by deepset?"
result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/websearch/searchapiwebsearch
# SearchApiWebSearch
Search engine using Search API.
`links`: A list of strings of resulting URLs. | | **API reference** | [Integrations](/reference/integrations-perplexity) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/perplexity/src/haystack_integrations/components/websearch/perplexity/perplexity_websearch.py | | **Package name** | `perplexity-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [`LinkContentFetcher`](../fetchers/linkcontentfetcher.mdx) or [Converters](../converters.mdx) |
| **Mandatory init variables** | `api_key`: The SearchAPI API key. Can be set with `SEARCHAPI_API_KEY` env var. |
| **Mandatory run variables** | `query`: A string with your query |
| **Output variables** | `documents`: A list of documents
`links`: A list of strings of resulting links | | **API reference** | [SearchApi](/reference/integrations-searchapi) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/searchapi | | **Package name** | `searchapi-haystack` |
## Overview
When you give `SearchApiWebSearch` a query, it returns a list of the URLs most relevant to your search. It uses page snippets (pieces of text displayed under the page title in search results) to find the answers, not the whole pages.
To search the content of the web pages, use the [`LinkContentFetcher`](../fetchers/linkcontentfetcher.mdx) component.
`SearchApiWebSearch` requires a [SearchApi](https://www.searchapi.io) key to work. It uses a `SEARCHAPI_API_KEY` environment variable by default. Otherwise, you can pass an `api_key` at initialization – see code examples below.
:::info[Alternative search]
To use [Serper Dev](https://serper.dev/?gclid=Cj0KCQiAgqGrBhDtARIsAM5s0_kPElllv3M59UPok1Ad-ZNudLaY21zDvbt5qw-b78OcUoqqvplVHRwaAgRgEALw_wcB) as an alternative, see its respective [documentation page](serperdevwebsearch.mdx).
:::
## Usage
Install the `searchapi-haystack` package to use the `SearchApiWebSearch` component:
```shell
pip install searchapi-haystack
```
### On its own
This is an example of how `SearchApiWebSearch` looks up answers to our query on the web and converts the results into a list of documents with content snippets of the results, as well as URLs as strings.
```python
from haystack_integrations.components.websearch.searchapi import SearchApiWebSearch
from haystack.utils import Secret
web_search = SearchApiWebSearch(api_key=Secret.from_token("`links`: A list of strings of resulting links | | **API reference** | [SearchApi](/reference/integrations-searchapi) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/searchapi | | **Package name** | `searchapi-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before [`LinkContentFetcher`](../fetchers/linkcontentfetcher.mdx) or [Converters](../converters.mdx) |
| **Mandatory init variables** | `api_key`: The Serper API key. Can be set with `SERPERDEV_API_KEY` env var. |
| **Mandatory run variables** | `query`: A string with your query |
| **Output variables** | `documents`: A list of documents
`links`: A list of strings of resulting links | | **API reference** | [SerperDev](/reference/integrations-serperdev) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/serperdev | | **Package name** | `serperdev-haystack` |
## Overview
When you give `SerperDevWebSearch` a query, it returns a list of the URLs most relevant to your search. It uses page snippets (pieces of text displayed under the page title in search results) to find the answers, not the whole pages.
To search the content of the web pages, use the [`LinkContentFetcher`](../fetchers/linkcontentfetcher.mdx) component.
`SerperDevWebSearch` requires a [SerperDev](https://serper.dev/) key to work. It uses a `SERPERDEV_API_KEY` environment variable by default. Otherwise, you can pass an `api_key` at initialization – see code examples below.
:::info[Alternative search]
To use [Search API](https://www.searchapi.io/) as an alternative, see its respective [documentation page](searchapiwebsearch.mdx).
:::
## Usage
Install the `serperdev-haystack` package to use the `SerperDevWebSearch` component:
```shell
pip install serperdev-haystack
```
### On its own
This is an example of how `SerperDevWebSearch` looks up answers to our query on the web and converts the results into a list of documents with content snippets of the results, as well as URLs as strings.
```python
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.utils import Secret
web_search = SerperDevWebSearch(api_key=Secret.from_token("`links`: A list of strings of resulting links | | **API reference** | [SerperDev](/reference/integrations-serperdev) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/serperdev | | **Package name** | `serperdev-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Tavily API key. Can be set with the `TAVILY_API_KEY` env var. |
| **Mandatory run variables** | `query`: A string with your search query. |
| **Output variables** | `documents`: A list of Haystack Documents containing search result content and metadata.
`links`: A list of strings of resulting URLs. | | **API reference** | [Tavily Search API](/reference/integrations-tavily) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/tavily/src/haystack_integrations/components/websearch/tavily/tavily_websearch.py | | **Package name** | `tavily-haystack` |
## Overview
When you give `TavilyWebSearch` a query, it uses the [Tavily](https://tavily.com) Search API to search the web and return relevant content as Haystack `Document` objects. It also returns a list of the source URLs.
Tavily is an AI-powered search API built specifically for LLM applications. It returns clean, relevant snippets without the noise of traditional search engines, making it a great fit for RAG pipelines.
`TavilyWebSearch` requires a Tavily API key to work. By default, it looks for a `TAVILY_API_KEY` environment variable. Alternatively, you can pass an `api_key` directly during initialization.
## Usage
### On its own
Here is a quick example of how `TavilyWebSearch` searches the web based on a query and returns a list of Documents.
```python
from haystack_integrations.components.websearch.tavily import TavilyWebSearch
from haystack.utils import Secret
web_search = TavilyWebSearch(
api_key=Secret.from_env_var("TAVILY_API_KEY"),
top_k=5,
)
query = "What is Haystack by deepset?"
response = web_search.run(query=query)
for doc in response["documents"]:
print(doc.content)
```
### In a pipeline
Here is an example of a Retrieval-Augmented Generation (RAG) pipeline that uses `TavilyWebSearch` to look up an answer on the web.
```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.components.websearch.tavily import TavilyWebSearch
from haystack.dataclasses import ChatMessage
web_search = TavilyWebSearch(
api_key=Secret.from_env_var("TAVILY_API_KEY"),
top_k=3,
)
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given the information below:\n"
"{% for document in documents %}{{ document.content }}\n{% endfor %}\n"
"Answer the following question: {{ query }}.\nAnswer:",
),
]
prompt_builder = ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
)
llm = OpenAIChatGenerator(
api_key=Secret.from_env_var("OPENAI_API_KEY"),
)
pipe = Pipeline()
pipe.add_component("search", web_search)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("search.documents", "prompt_builder.documents")
pipe.connect("prompt_builder.prompt", "llm.messages")
query = "What is Haystack by deepset?"
result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})
print(result["llm"]["replies"][0].text)
```
---
// File: pipeline-components/websearch
# WebSearch
Use these components to look up answers on the internet.
| Name | Description |
| --- | --- |
| [BraveWebSearch](websearch/bravewebsearch.mdx) | Search engine using the Brave Search API. |
| [DDGSWebSearch](websearch/ddgswebsearch.mdx) | Multi-engine web search using ddgs (Dux Distributed Global Search), with no API key required. |
| [FirecrawlWebSearch](websearch/firecrawlwebsearch.mdx) | Search engine using the Firecrawl API. |
| [LinkupWebSearch](websearch/linkupwebsearch.mdx) | Search engine using the Linkup Search API. |
| [PerplexityWebSearch](websearch/perplexitywebsearch.mdx) | Search engine using the Perplexity Search API. |
| [SearchApiWebSearch](websearch/searchapiwebsearch.mdx) | Search engine using Search API. |
| [SerperDevWebSearch](websearch/serperdevwebsearch.mdx) | Search engine using SerperDev API. |
| [TavilyWebSearch](websearch/tavilywebsearch.mdx) | Search engine using the Tavily AI-powered search API. |
---
// File: pipeline-components/writers/cogneewriter
# CogneeWriter
Writes `ChatMessage` objects to a `CogneeMemoryStore` as long-term memories.
`links`: A list of strings of resulting URLs. | | **API reference** | [Tavily Search API](/reference/integrations-tavily) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/tavily/src/haystack_integrations/components/websearch/tavily/tavily_websearch.py | | **Package name** | `tavily-haystack` |
| | |
| --- | --- |
| **Most common position in a pipeline** | After an [`Agent`](../agents-1/agent.mdx) or Chat Generator in memory-augmented pipelines |
| **Mandatory init variables** | `memory_store`: A `CogneeMemoryStore` instance |
| **Optional init variables** | `session_id`: When set, writes target the session-cache tier; when `None`, writes go to the permanent knowledge graph |
| **Mandatory run variables** | `messages`: A list of `ChatMessage` objects |
| **Optional run variables** | `user_id`: Cognee user ID to scope the write; pass `None` to use Cognee's default user |
| **Output variables** | `messages_written`: The list of `ChatMessage` objects that were written (passed through unchanged) |
| **API reference** | [Cognee](/reference/integrations-cognee#cogneewriter) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cognee |
| **Package name** | `cognee-haystack` |
## Overview
`CogneeWriter` persists a list of `ChatMessage` objects into a `CogneeMemoryStore`. Use it in a Haystack Pipeline to store conversation facts or user preferences after an Agent turn.
Messages are passed through unchanged to the pipeline output (`messages_written`), making this component easy to chain after an Agent or generator without breaking the pipeline flow.
The `session_id` init parameter controls which Cognee memory tier is targeted:
- Omit `session_id` (or set it to `None`) to write to the **permanent knowledge graph** — Cognee runs LLM extraction during ingestion, producing rich graph-completion-ready nodes.
- Set `session_id` to write to the **session cache** — fast writes with no LLM extraction, scoped to that session. Session content can later be promoted to the permanent graph via `CogneeMemoryStore.improve()`.
The writer's `session_id` overrides the store's `session_id` per call, so a single store can back multiple writers targeting different memory tiers.
## Installation
Install the Cognee integration:
```bash
pip install cognee-haystack
```
Set your LLM API key (used by Cognee for graph extraction):
```bash
export LLM_API_KEY="your-llm-api-key"
```
Optionally, set a separate embedding API key (defaults to `LLM_API_KEY` when unset):
```bash
export EMBEDDING_API_KEY="your-embedding-api-key"
```
## Usage
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.writers.cognee import CogneeWriter
from haystack_integrations.memory_stores.cognee import CogneeMemoryStore
store = CogneeMemoryStore()
writer = CogneeWriter(memory_store=store)
result = writer.run(
messages=[ChatMessage.from_user("Alice prefers concise Python examples.")],
user_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890",
)
print(result["messages_written"])
```
To write to the session cache instead of the permanent graph, pass a `session_id`:
```python
session_writer = CogneeWriter(memory_store=store, session_id="alice_session_1")
session_writer.run(
messages=[
ChatMessage.from_user("Alice is currently debugging a vector store issue.")
],
user_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890",
)
```
### In a Pipeline
This example connects an Agent's full `messages` output to `CogneeWriter`, so Cognee stores the conversation turn in the permanent graph.
```python
from haystack import Pipeline
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.writers.cognee import CogneeWriter
from haystack_integrations.memory_stores.cognee import CogneeMemoryStore
store = CogneeMemoryStore(dataset_name="my_agent_memory")
pipeline = Pipeline()
pipeline.add_component(
"agent",
Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
system_prompt=(
"Answer the user and preserve durable user facts or preferences for future conversations."
),
),
)
pipeline.add_component("writer", CogneeWriter(memory_store=store))
pipeline.connect("agent.messages", "writer.messages")
result = pipeline.run(
{
"agent": {
"messages": [
ChatMessage.from_user(
"My name is Alice and I prefer concise Python examples.",
),
],
},
"writer": {
"user_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
},
},
)
print(result["writer"]["messages_written"])
```
---
// File: pipeline-components/writers/documentwriter
# DocumentWriter
Use this component to write documents into a Document Store of your choice.
| | |
| --- | --- |
| **Most common position in a pipeline** | As the last component in an indexing pipeline |
| **Mandatory init variables** | `document_store`: A Document Store instance |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents_written`: The number of documents written (integer) |
| **API reference** | [Document Writers](/reference/document-writers-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/writers/document_writer.py |
| **Package name** | `haystack-ai` |
## Overview
`DocumentWriter` writes a list of documents into a Document Store of your choice. It’s typically used in an indexing pipeline as the final step after preprocessing documents and creating their embeddings.
To use this component with a specific file type, make sure you use the correct [Converter](../converters.mdx) before it. For example, to use `DocumentWriter` with Markdown files, use the `MarkdownToDocument` component before `DocumentWriter` in your indexing pipeline.
### DuplicatePolicy
The `DuplicatePolicy` is a class that defines the different options for handling documents with the same ID in a `DocumentStore`. It has four possible values:
- **NONE**: The default policy that relies on Document Store settings.
- **OVERWRITE**: Indicates that if a document with the same ID already exists in the `DocumentStore`, it should be overwritten with the new document.
- **SKIP**: If a document with the same ID already exists, the new document will be skipped and not added to the `DocumentStore`.
- **FAIL**: Raises an error if a document with the same ID already exists in the `DocumentStore`. It prevents duplicate documents from being added.
## Usage
### On its own
Below is an example of how to write two documents into an `InMemoryDocumentStore`:
```python
from haystack import Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.writers import DocumentWriter
documents = [
Document(content="This is document 1"),
Document(content="This is document 2"),
]
document_store = InMemoryDocumentStore()
document_writer = DocumentWriter(document_store=document_store)
document_writer.run(documents=documents)
```
### In a pipeline
Below is an example of an indexing pipeline that first uses the `SentenceTransformersDocumentEmbedder` to create embeddings of documents and then use the `DocumentWriter` to write the documents to an `InMemoryDocumentStore`:
The examples on this page use Sentence Transformers embedders from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.document_stores.types import DuplicatePolicy
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
)
from haystack.components.writers import DocumentWriter
documents = [
Document(content="This is document 1"),
Document(content="This is document 2"),
]
document_store = InMemoryDocumentStore()
embedder = SentenceTransformersDocumentEmbedder()
document_writer = DocumentWriter(
document_store=document_store,
policy=DuplicatePolicy.NONE,
)
indexing_pipeline = Pipeline()
indexing_pipeline.add_component(instance=embedder, name="embedder")
indexing_pipeline.add_component(instance=document_writer, name="writer")
indexing_pipeline.connect("embedder", "writer")
indexing_pipeline.run({"embedder": {"documents": documents}})
```
---
// File: pipeline-components/writers/mem0memorywriter
# Mem0MemoryWriter
Writes `ChatMessage` objects to Mem0 as long-term memories.
| | |
| --- | --- |
| **Most common position in a pipeline** | After an [`Agent`](../agents-1/agent.mdx) or Chat Generator in memory-augmented pipelines |
| **Mandatory init variables** | `memory_store`: A `Mem0MemoryStore` instance |
| **Mandatory run variables** | `messages`: A list of `ChatMessage` objects; at least one Mem0 scope through `user_id`, `run_id`, `agent_id`, or `app_id` |
| **Output variables** | `memories_written`: The number of memories written |
| **Mem0 API docs** | [Add Memories](https://docs.mem0.ai/api-reference/memory/add-memories) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mem0 |
| **Package name** | `mem0-haystack` |
## Overview
`Mem0MemoryWriter` writes a list of `ChatMessage` objects to a `Mem0MemoryStore`. Use it near the end of a memory-augmented pipeline to persist conversation facts, user preferences, and durable project context for future runs.
Scope written memories with at least one Mem0 entity ID: `user_id`, `run_id`, `agent_id`, or `app_id`. These are runtime inputs, so one pipeline instance can write memories for multiple users, sessions, agents, or applications.
The `infer` init parameter controls how Mem0 stores the incoming messages:
- `infer=True` lets Mem0 extract memories from the messages. This is useful when writing a full Agent turn that includes the user message, tool context, and final assistant response.
- `infer=False` stores the supplied message text as-is. This is useful when the upstream component has already selected the exact memory text.
### Installation
Install the Mem0 integration:
```shell
pip install mem0-haystack
```
Set your Mem0 API key:
```shell
export MEM0_API_KEY="your-mem0-api-key"
```
## Usage
### On its own
```python
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.writers.mem0 import Mem0MemoryWriter
from haystack_integrations.memory_stores.mem0 import Mem0MemoryStore
store = Mem0MemoryStore()
writer = Mem0MemoryWriter(memory_store=store, infer=False)
result = writer.run(
messages=[ChatMessage.from_user("Alice prefers concise Python examples.")],
user_id="alice",
)
print(result["memories_written"])
```
### In a Pipeline
This example connects an Agent's full `messages` output to `Mem0MemoryWriter` with `infer=True`, so Mem0 can extract memories from the full turn context.
```python
from haystack import Pipeline
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.writers.mem0 import Mem0MemoryWriter
from haystack_integrations.memory_stores.mem0 import Mem0MemoryStore
store = Mem0MemoryStore()
pipeline = Pipeline()
pipeline.add_component(
"agent",
Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
system_prompt=(
"Answer the user and preserve durable user facts or preferences for future conversations."
),
streaming_callback=print_streaming_chunk,
),
)
pipeline.add_component("writer", Mem0MemoryWriter(memory_store=store, infer=True))
pipeline.connect("agent.messages", "writer.messages")
result = pipeline.run(
{
"agent": {
"messages": [
ChatMessage.from_user(
"My name is Alice and I prefer concise Python examples.",
),
],
},
"writer": {
"user_id": "alice",
},
},
)
print(result["writer"]["memories_written"])
```
---
// File: token-counters/approximatetokencounter
# ApproximateTokenCounter
`ApproximateTokenCounter` estimates the token count of `ChatMessage` objects and optional tool schemas from their text length. It needs no extra dependency or warm-up step.
| | |
| --- | --- |
| **Import path** | `haystack.token_counters.ApproximateTokenCounter` |
| **API reference** | [Token Counters](/reference/token-counters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/token_counters/approximate_counter.py |
| **Package name** | `haystack-ai` |
## Usage
Create the counter and pass a list of messages to `count()`:
```python
from haystack.dataclasses import ChatMessage
from haystack.token_counters import ApproximateTokenCounter
messages = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user("Explain retrieval-augmented generation."),
]
counter = ApproximateTokenCounter()
token_count = counter.count(messages)
print(token_count)
```
By default, the counter treats four characters as one token. Set `chars_per_token` to tune the estimate for the languages and models in your application:
```python
counter = ApproximateTokenCounter(chars_per_token=3.5)
```
A smaller value produces a higher, more conservative estimate. `chars_per_token` must be greater than zero.
To include the context consumed by tool schemas, pass the tools to `count()`:
```python
token_count = counter.count(messages, tools=[search_tool])
```
## Non-text content
Images and files cannot be measured from text length, so the counter adds a flat estimate for each item. Change the defaults when your application sends large images or long documents:
```python
counter = ApproximateTokenCounter(
chars_per_token=4.0,
tokens_per_image=765,
tokens_per_file=4000,
)
```
The counter includes non-text content attached directly to a message as well as content nested inside tool results.
---
// File: token-counters/openaitokencounter
# OpenAITokenCounter
`OpenAITokenCounter` uses OpenAI's `POST /v1/responses/input_tokens` endpoint to count the input tokens for a specific model. It supports text, images, files, tool calls, tool results, and tool schemas in the same format used by the Responses API.
Because it calls a remote API, it requires an OpenAI API key and adds network latency. Use it when you need model-specific counts or need to measure non-text inputs and tool schemas accurately. For local estimates, use [`ApproximateTokenCounter`](approximatetokencounter.mdx) or [`TiktokenCounter`](tiktokencounter.mdx).
```python
from haystack.dataclasses import ChatMessage
from haystack.token_counters import OpenAITokenCounter
counter = OpenAITokenCounter("gpt-5-mini")
messages = [ChatMessage.from_user("What's Natural Language Processing?")]
token_count = counter.count(messages)
```
By default, the counter reads the API key from `OPENAI_API_KEY`. You can also pass a Haystack `Secret` explicitly and configure the API base URL, organization, timeout, retry count, and HTTP client options.
See the [Token Counters API reference](/reference/token-counters-api) for all constructor parameters and methods.
---
// File: token-counters/tiktokencounter
# TiktokenCounter
`TiktokenCounter` estimates the token count of `ChatMessage` objects and optional tool schemas with OpenAI's `tiktoken` byte-pair encoder. It is generally more accurate than a character-based estimate for OpenAI models, but its results can differ from the token counts of other providers.
| | |
| --- | --- |
| **Import path** | `haystack.token_counters.TiktokenCounter` |
| **API reference** | [Token Counters](/reference/token-counters-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/token_counters/tiktoken_counter.py |
| **Package name** | `haystack-ai` |
## Installation
Install the optional `tiktoken` dependency before constructing the counter:
```bash
pip install tiktoken
```
## Usage
Create the counter and pass a list of messages to `count()`:
```python
from haystack.dataclasses import ChatMessage
from haystack.token_counters import TiktokenCounter
messages = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user("Explain retrieval-augmented generation."),
]
counter = TiktokenCounter()
token_count = counter.count(messages)
print(token_count)
```
The default encoding is `o200k_base`. Pass a different encoding when required by your model:
```python
counter = TiktokenCounter(encoding="cl100k_base")
```
The counter loads its encoding on the first call to `count()`. To load it during application startup instead, call `warm_up()` explicitly:
```python
counter.warm_up()
```
To include the context consumed by tool schemas, pass the tools to `count()`:
```python
token_count = counter.count(messages, tools=[search_tool])
```
## Non-text content
The tokenizer cannot measure images or files, so the counter adds a flat estimate for each item. Change the defaults when your application sends large images or long documents:
```python
counter = TiktokenCounter(
encoding="o200k_base",
tokens_per_image=765,
tokens_per_file=4000,
)
```
The counter includes non-text content attached directly to a message as well as content nested inside tool results.
---
// File: token-counters
# Token Counters
Token counters estimate how many tokens a list of `ChatMessage` objects and optional tool schemas occupy. They are useful when you need to know the size of a conversation before sending it to a model, for example, to check whether it fits in the model's context window or to decide how much context to remove.
Haystack provides the `TokenCounter` protocol and three implementations:
| Counter | How it counts text | Extra dependency | Best suited for |
| --- | --- | --- | --- |
| [`ApproximateTokenCounter`](token-counters/approximatetokencounter.mdx) | Divides the rendered text length by a configurable characters-per-token ratio | None | Fast, dependency-free estimates |
| [`TiktokenCounter`](token-counters/tiktokencounter.mdx) | Uses OpenAI's `tiktoken` byte-pair encoder | `tiktoken` | More accurate estimates for OpenAI models |
| [`OpenAITokenCounter`](token-counters/openaitokencounter.mdx) | Calls OpenAI's input token counting API | OpenAI API key | Exact, model-specific counts including images, files, and tools |
All counters include message roles, text, tool calls, tool results, and optional tool schemas. The local counters account for images and files using configurable flat rates, including images and files nested in tool results. `OpenAITokenCounter` sends supported non-text content to OpenAI for a model-specific count.
See the [Token Counters API reference](/reference/token-counters-api) for all constructor parameters and methods.
## Counting tool schemas
Tool schemas are sent to the model alongside the messages and consume context tokens. Pass the tools to `count()` to include their schemas in the estimate:
```python
from typing import Annotated
from haystack.dataclasses import ChatMessage
from haystack.token_counters import ApproximateTokenCounter
from haystack.tools import tool
@tool
def search(query: Annotated[str, "The search query"]) -> str:
"""Search for documents that match the query."""
return "Search results"
messages = [ChatMessage.from_user("Find information about Haystack.")]
counter = ApproximateTokenCounter()
token_count = counter.count(messages, tools=[search])
```
You can also count tool schemas without messages by calling `counter.count([], tools=[search])`.
## Images and files
Images and files do not have a portable text-based token count. Each token counter can handle them differently depending on the tokenizer or provider it uses.
See the documentation for the counter you use to understand how it counts non-text content and whether you need to configure it:
- [`ApproximateTokenCounter`](token-counters/approximatetokencounter.mdx)
- [`TiktokenCounter`](token-counters/tiktokencounter.mdx)
- [`OpenAITokenCounter`](token-counters/openaitokencounter.mdx)
## Creating a custom token counter
Implement the `TokenCounter` protocol when you need different counting behavior, such as using a provider's token-counting endpoint. A custom implementation must provide `count()` and `to_dict()` methods. The default `from_dict()` implementation restores plain constructor values.
```python
from typing import Any
from haystack.core.serialization import default_to_dict
from haystack.dataclasses import ChatMessage
from haystack.token_counters import TokenCounter
from haystack.tools import ToolsType
class ProviderTokenCounter(TokenCounter):
def count(
self,
messages: list[ChatMessage],
tools: ToolsType | None = None,
) -> int:
# Call the provider's token-counting endpoint here.
...
def to_dict(self) -> dict[str, Any]:
return default_to_dict(self)
```
Override `from_dict()` when `to_dict()` serializes values that must be reconstructed before passing them to the constructor, such as a `Secret` or a nested component.
---
// File: tools/agenttool
# AgentTool
Wraps a Haystack Agent so another Agent can call it as a tool.
| | |
| --- | --- |
| **Mandatory init variables** | `agent`: The Haystack Agent to wrap
`name`: The name of the tool
`description`: Description of the tool | | **API reference** | [AgentTool](/reference/tools-api#agenttool) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tools/agent_tool.py | | **Package name** | `haystack-ai` |
## Overview
`AgentTool` turns a Haystack [`Agent`](../pipeline-components/agents-1/agent.mdx) into a tool that another `Agent` can call. This is the basis for multi-agent systems: one agent specializes in a task, and a coordinator delegates to it instead of doing the work itself.
The main benefit is context isolation. A specialist may search the web several times and read a few pages before it answers, and all of that stays inside the specialist. The coordinator sends a task and receives an answer, so its context stays small.
Besides the `Agent` itself, the only required configuration is a name and a description. The coordinator's model sends the task as plain text and receives the specialist's reply as text.
If the specialist needs more than a task, for example a variable in its system prompt, `AgentTool` adds it to the tool's inputs, or takes it from the coordinator's state. See [Prompt Variables](#prompt-variables).
### Parameters
- `agent` is mandatory and must be an `Agent` instance.
- `name` is mandatory and specifies the tool name.
- `description` is mandatory. It should tell the calling LLM what the wrapped Agent is specialized in and when to delegate to it.
- `parameters` is optional and lets you override the generated JSON schema for the tool's inputs. It must cover every mandatory input of the wrapped Agent that is not supplied through `inputs_from_state`, otherwise a `ValueError` is raised.
- `outputs_to_string` is optional and controls how the wrapped Agent's output is converted to a string for the calling LLM. By default, the text of the final reply is returned, or the serialized message if the reply has no text. A warning is appended if the Agent stopped because it reached `max_agent_steps`.
- `inputs_from_state` is optional and maps the calling Agent's state keys to inputs of the wrapped Agent. Example: `{"subject": "topic"}` passes the state value at `"subject"` as the wrapped Agent's `"topic"` input. Inputs mapped this way are not added to the generated schema, since the calling Agent provides them.
- `outputs_to_state` is optional and maps the wrapped Agent's output keys to the calling Agent's state keys. Example: `{"notes": {"source": "last_message"}}` writes the wrapped Agent's `"last_message"` output to `"notes"` in state.
## Usage
### Basic Usage
This example uses the SerperDev web search component (`serperdev-haystack` package). Install it to run the example:
```shell
pip install serperdev-haystack
```
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import AgentTool, ComponentTool
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
researcher = Agent(
chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4-mini"),
system_prompt="You are a research specialist. Investigate the task and report your findings.",
tools=[
ComponentTool(
component=SerperDevWebSearch(top_k=3),
name="web_search",
description="Search the web for current information on any topic",
),
],
)
research = AgentTool(
agent=researcher,
name="research",
description="Research a question on the web and report the findings",
)
coordinator = Agent(
chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4"),
tools=[research],
system_prompt="You coordinate specialists. Delegate research questions, then answer the user.",
)
result = coordinator.run(
[
ChatMessage.from_user(
"What are the latest developments in the Haystack framework?",
),
],
)
print(result["last_message"].text)
```
The coordinator sees a single `research` tool that takes the task to delegate as one user message. The searches the researcher runs and the results it reads never enter the coordinator's context, only the final report does.
### Prompt Variables
If the wrapped Agent has variables in its `system_prompt` or `user_prompt`, written as [Jinja templates](../concepts/jinja-templates.mdx), they are mandatory inputs. `AgentTool` adds a string parameter for each of them to the generated schema, and the calling LLM fills them in.
The reviewer below is specialized by language, and the coordinator picks the language for each review it delegates:
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import AgentTool
SNIPPET = """
def load_config(path):
return json.loads(open(path).read())
"""
reviewer = Agent(
chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4-mini"),
system_prompt=(
"You are a senior {{language}} engineer. Review the code you are given and reply with the single "
"most important issue, in one sentence."
),
)
review_tool = AgentTool(
agent=reviewer,
name="code_review",
description="Ask a senior engineer to review a code snippet",
)
print(review_tool.parameters["required"])
# ['messages', 'language']
coordinator = Agent(
chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4-mini"),
tools=[review_tool],
system_prompt="You triage code snippets. Delegate every review to the code_review tool, then answer the user.",
)
result = coordinator.run(
[ChatMessage.from_user(f"Review this Python snippet:\n{SNIPPET}")],
)
print(result["last_message"].text)
```
Use `inputs_from_state` when the value should come from the calling Agent's [state](../pipeline-components/agents-1/state.mdx) instead of from the LLM. Inputs mapped this way are not added to the generated schema:
```python
review_tool = AgentTool(
agent=reviewer,
name="code_review",
description="Ask a senior engineer to review a code snippet",
# the state key "project_language" fills the reviewer's "language" input
inputs_from_state={"project_language": "language"},
)
print(review_tool.parameters["required"])
# ['messages']
coordinator = Agent(
chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4-mini"),
tools=[review_tool],
system_prompt="You triage code snippets. Delegate every review to the code_review tool, then answer the user.",
state_schema={"project_language": {"type": str}},
)
result = coordinator.run(
[ChatMessage.from_user(f"Review this snippet:\n{SNIPPET}")],
project_language="Python",
)
print(result["last_message"].text)
```
## Additional References
📖 Related docs:
- [Multi-Agent Systems](../concepts/agents/multi-agent-systems.mdx)
- [ComponentTool](componenttool.mdx)
- [State](../pipeline-components/agents-1/state.mdx)
📚 Tutorials:
- [Creating a Multi-Agent System with Haystack](https://haystack.deepset.ai/tutorials/45_creating_a_multi_agent_system)
---
// File: tools/componenttool
# ComponentTool
This wrapper allows using Haystack components to be used as tools by LLMs.
`name`: The name of the tool
`description`: Description of the tool | | **API reference** | [AgentTool](/reference/tools-api#agenttool) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tools/agent_tool.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Mandatory init variables** | `component`: The Haystack component to wrap |
| **API reference** | [ComponentTool](/reference/tools-api#componenttool) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tools/component_tool.py |
| **Package name** | `haystack-ai` |
## Overview
`ComponentTool` is a Tool that wraps Haystack components, allowing them to be used as tools by LLMs. ComponentTool automatically generates LLM-compatible tool schemas from component input sockets, which are derived from the component's `run` method signature and type hints.
It does input type conversion and offers support for components with run methods that have the following input types:
- Basic types (str, int, float, bool, dict)
- Dataclasses (both simple and nested structures)
- Lists of basic types (such as list[str])
- Lists of dataclasses (such as list[Document])
- Parameters with mixed types (such as list[Document], str...)
If the wrapped component defines a `run_async` method, `ComponentTool` automatically wires an async invoker as well, so the tool supports async invocation (for example, from `Agent.run_async`) without extra configuration. See [Async Tools](tool.mdx#async-tools) for details.
To wrap an [`Agent`](../pipeline-components/agents-1/agent.mdx) as a tool, use [`AgentTool`](agenttool.mdx) instead. It is a specialization of `ComponentTool` with defaults tailored to Agents: the calling LLM is asked for the task to delegate as a single user message, and the tool result is the wrapped Agent's final reply.
### Parameters
- `component` is mandatory and must be a Haystack component instance, either an existing one or a custom component.
- `name` is optional and defaults to the component class name in snake case, for example, "serper_dev_web_search" for `SerperDevWebSearch`.
- `description` is optional and defaults to the component’s docstring. This is what the LLM uses to decide when to call the tool.
- `parameters` is optional and lets you override the auto-generated JSON schema for the tool’s inputs.
- `outputs_to_string` is optional and controls how the component’s output is converted to a string for the LLM. By default, the full result dict is serialized. Use `{"source": "key"}` to extract a single output key, or add `"handler"` to apply a custom formatter.
- `inputs_from_state` is optional and maps agent state keys to component input parameters. Example: `{"repository": "repo"}` passes the state value at `"repository"` as the component’s `"repo"` input.
- `outputs_to_state` is optional and maps component output keys to agent state keys. Example: `{"documents": {"source": "docs"}}` writes the component’s `"docs"` output to `"documents"` in state.
## Usage
:::tip
The recommended way to use `ComponentTool` in Haystack is with the [`Agent`](../pipeline-components/agents-1/agent.mdx) component, which manages the tool call loop for you.
:::
### With the Agent Component
The example on this page uses the SerperDev web search component that has moved to the `serperdev-haystack` package. Install it to run the example:
```shell
pip install serperdev-haystack
```
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool
from haystack.components.agents import Agent
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.utils import Secret
# Create a SerperDev search component
search = SerperDevWebSearch(api_key=Secret.from_env_var("SERPERDEV_API_KEY"), top_k=3)
# Create a tool from the component
search_tool = ComponentTool(
component=search,
name="web_search", # Optional: defaults to "serper_dev_web_search"
description="Search the web for current information on any topic", # Optional: defaults to component docstring
)
agent = Agent(
system_prompt="You are an assistant that can use web search to find information.",
chat_generator=OpenAIChatGenerator(),
tools=[search_tool],
)
response = agent.run(
messages=[ChatMessage.from_user("Give me a brief summary on who Nikola Tesla is")],
)
print(response["messages"][-1].text)
```
## Additional References
📖 Related docs:
- [AgentTool](agenttool.mdx)
- [Multi-Agent Systems](../concepts/agents/multi-agent-systems.mdx)
📚 Tutorials:
- [Build a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent)
- [Creating a Multi-Agent System with Haystack](https://haystack.deepset.ai/tutorials/45_creating_a_multi_agent_system)
---
// File: tools/mcptool
# MCPTool
MCPTool enables integration with external tools and services through the Model Context Protocol (MCP).
| | |
| --- | --- |
| **Mandatory init variables** | `name`: The name of the tool
`server_info`: Information about the MCP server to connect to | | **API reference** | [MCP](/reference/integrations-mcp) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mcp | | **Package name** | `mcp-haystack` |
## Overview
`MCPTool` is a Tool that allows Haystack to communicate with external tools and services using the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/). MCP is an open protocol that standardizes how applications provide context to LLMs, similar to how USB-C provides a standardized way to connect devices.
The `MCPTool` supports multiple transport options:
- Streamable HTTP for connecting to HTTP servers,
- SSE (Server-Sent Events) for connecting to HTTP servers **(deprecated)**,
- StdIO for direct execution of local programs.
Learn more about the MCP protocol and its architecture at the [official MCP website](https://modelcontextprotocol.io/).
### Parameters
- `name` is _mandatory_ and specifies the name of the tool.
- `server_info` is _mandatory_ and needs to be either an `SSEServerInfo`, `StreamableHttpServerInfo` or `StdioServerInfo` object that contains connection information.
- `description` is _optional_ and provides context to the LLM about what the tool does.
### Results
The Tool return results as a list of JSON objects, representing `TextContent`, `ImageContent`, or `EmbeddedResource` types from the mcp-sdk.
## Usage
Install the MCP-Haystack integration to use the `MCPTool`:
```shell
pip install mcp-haystack
```
### With Streamable HTTP Transport
You can create an `MCPTool` that connects to an external HTTP server using streamable-http transport:
```python
from haystack_integrations.tools.mcp import MCPTool, StreamableHttpServerInfo
# Create an MCP tool that connects to an HTTP server
server_info = StreamableHttpServerInfo(url="http://localhost:8000/mcp")
tool = MCPTool(name="my_tool", server_info=server_info)
# Use the tool
result = tool.invoke(param1="value1", param2="value2")
```
### With SSE Transport (deprecated)
:::warning
SSE transport has been [deprecated by the MCP specification](https://modelcontextprotocol.io/specification/2025-11-25/basic/transports#streamable-http) in favor of Streamable HTTP. Use [Streamable HTTP](#with-streamable-http-transport) for new integrations. If you are connecting to an existing SSE-only server, `SSEServerInfo` will continue to work, but consider migrating to `StreamableHttpServerInfo` when the server supports it.
:::
You can create an `MCPTool` that connects to an external HTTP server using SSE transport:
```python
from haystack_integrations.tools.mcp import MCPTool, SSEServerInfo
# Create an MCP tool that connects to an HTTP server
server_info = SSEServerInfo(url="http://localhost:8000/sse")
tool = MCPTool(name="my_tool", server_info=server_info)
# Use the tool
result = tool.invoke(param1="value1", param2="value2")
```
### With StdIO Transport
You can also create an `MCPTool` that executes a local program directly and connects to it through stdio transport:
```python
from haystack_integrations.tools.mcp import MCPTool, StdioServerInfo
# Create an MCP tool that uses stdio transport
server_info = StdioServerInfo(
command="uvx",
args=["mcp-server-time", "--local-timezone=Europe/Berlin"],
)
tool = MCPTool(name="get_current_time", server_info=server_info)
# Get the current time in New York
result = tool.invoke(timezone="America/New_York")
```
### In a pipeline
You can integrate an `MCPTool` into a pipeline through the `Agent` component:
```python
from haystack import Pipeline
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.tools.mcp import MCPTool, StdioServerInfo
time_tool = MCPTool(
name="get_current_time",
server_info=StdioServerInfo(
command="uvx",
args=["mcp-server-time", "--local-timezone=Europe/Berlin"],
),
)
pipeline = Pipeline()
pipeline.add_component(
"agent",
Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
tools=[time_tool],
),
)
user_input = "What is the time in New York? Be brief." # can be any city
user_input_msg = ChatMessage.from_user(text=user_input)
result = pipeline.run({"agent": {"messages": [user_input_msg]}})
print(result["agent"]["last_message"].text)
# The current time in New York is 1:57 PM.
```
### With the Agent Component
You can use `MCPTool` with the [Agent](../pipeline-components/agents-1/agent.mdx) component. The `Agent` component combines the ChatGenerator of your choice with built-in tool execution to run tool calls and process tool results.
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.components.agents import Agent
from haystack_integrations.tools.mcp import MCPTool, StdioServerInfo
time_tool = MCPTool(
name="get_current_time",
server_info=StdioServerInfo(
command="uvx",
args=["mcp-server-time", "--local-timezone=Europe/Berlin"],
),
)
# Agent Setup
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[time_tool],
exit_conditions=["text"],
)
# Run the Agent
response = agent.run(
messages=[ChatMessage.from_user("What is the time in New York? Be brief.")],
)
# Output
print(response["messages"][-1].text)
```
---
// File: tools/mcptoolset
# MCPToolset
`MCPToolset` connects to an MCP-compliant server and automatically loads all available tools into a single manageable unit. These tools can be used directly with components like Chat Generator or `Agent`.
`server_info`: Information about the MCP server to connect to | | **API reference** | [MCP](/reference/integrations-mcp) | | **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mcp | | **Package name** | `mcp-haystack` |
| | |
| --- | --- |
| **Mandatory init variables** | `server_info`: Information about the MCP server to connect to |
| **API reference** | [mcp](/reference/integrations-mcp) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mcp |
| **Package name** | `mcp-haystack` |
## Overview
MCPToolset is a subclass of `Toolset` that dynamically discovers and loads tools from any MCP-compliant server.
It supports:
- **Streamable HTTP** for connecting to HTTP servers
- **SSE (Server-Sent Events)** _(deprecated)_ for remote MCP servers through HTTP
- **StdIO** for local tool execution through subprocess
The MCPToolset makes it easy to plug external tools into pipelines or agents, with built-in support for filtering (with `tool_names`).
### Parameters
To initialize the MCPToolset, use the following parameters:
- `server_info` (required): Connection information for the MCP server
- `tool_names` (optional): A list of tool names to add to the Toolset
:::info
Note that if `tool_names` is not specified, all tools from the MCP server will be loaded. Be cautious if there are many tools (20–30+), as this can overwhelm the LLM’s tool resolution logic.
:::
### Installation
```shell
pip install mcp-haystack
```
## Usage
### With StdIO Transport
```python
from haystack_integrations.tools.mcp import MCPToolset, StdioServerInfo
server_info = StdioServerInfo(
command="uvx",
args=["mcp-server-time", "--local-timezone=Europe/Berlin"],
)
toolset = MCPToolset(
server_info=server_info,
tool_names=["get_current_time"],
) # If tool_names is omitted, all tools on this MCP server will be loaded (can overwhelm LLM if too many)
```
### With Streamable HTTP Transport
```python
from haystack_integrations.tools.mcp import MCPToolset, StreamableHttpServerInfo
server_info = StreamableHttpServerInfo(url="http://localhost:8000/mcp")
toolset = MCPToolset(server_info=server_info, tool_names=["get_current_time"])
```
### With SSE Transport (deprecated)
```python
from haystack_integrations.tools.mcp import MCPToolset, SSEServerInfo
server_info = SSEServerInfo(url="http://localhost:8000/sse")
toolset = MCPToolset(server_info=server_info, tool_names=["get_current_time"])
```
### In a Pipeline
```python
from haystack import Pipeline
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.tools.mcp import MCPToolset, StdioServerInfo
server_info = StdioServerInfo(
command="uvx",
args=["mcp-server-time", "--local-timezone=Europe/Berlin"],
)
toolset = MCPToolset(server_info=server_info)
pipeline = Pipeline()
pipeline.add_component(
"agent",
Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
tools=toolset,
),
)
user_input = ChatMessage.from_user(text="What is the time in New York?")
result = pipeline.run({"agent": {"messages": [user_input]}})
print(result["agent"]["last_message"].text)
```
### With the Agent
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.agents import Agent
from haystack.dataclasses import ChatMessage
from haystack_integrations.tools.mcp import MCPToolset, StdioServerInfo
toolset = MCPToolset(
server_info=StdioServerInfo(
command="uvx",
args=["mcp-server-time", "--local-timezone=Europe/Berlin"],
),
tool_names=[
"get_current_time",
], # Omit to load all tools, but may overwhelm LLM if many
)
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=toolset,
exit_conditions=["text"],
)
response = agent.run(messages=[ChatMessage.from_user("What is the time in New York?")])
print(response["messages"][-1].text)
```
---
// File: tools/pipelinetool
# PipelineTool
Wraps a Haystack pipeline so an LLM can call it as a tool.
| | |
| --- | --- |
| **Mandatory init variables** | `pipeline`: The Haystack pipeline to wrap
`name`: The name of the tool
`description`: Description of the tool | | **API reference** | [PipelineTool](/reference/tools-api#pipeline_tool) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tools/pipeline_tool.py | | **Package name** | `haystack-ai` |
## Overview
`PipelineTool` lets you wrap a whole Haystack pipeline and expose it as a tool that an LLM can call.
It replaces the older workflow of first wrapping a pipeline in a `SuperComponent` and then passing that to
`ComponentTool`.
`PipelineTool` builds the tool parameter schema from the pipeline’s input sockets and uses the underlying components’ docstrings for input descriptions. You can choose which pipeline inputs and outputs to expose with
`input_mapping` and `output_mapping`. It can be used with the `Agent` component, either directly or within a pipeline.
`PipelineTool` also supports async invocation: since every `Pipeline` exposes a native `run_async`, the tool can be awaited (for example, from `Agent.run_async`) without extra configuration. See [Async Tools](tool.mdx#async-tools) for details.
### Parameters
- `pipeline` is mandatory and must be a `Pipeline` instance.
- `name` is mandatory and specifies the tool name.
- `description` is mandatory and explains what the tool does.
- `input_mapping` is optional. It maps tool input names to pipeline input socket paths. If omitted, a default mapping is created from all pipeline inputs.
- `output_mapping` is optional. It maps pipeline output socket paths to tool output names. If omitted, a default mapping is created from all pipeline outputs.
- `parameters` is optional and lets you override the auto-generated JSON schema for the tool's inputs.
- `outputs_to_string` is optional and controls how the pipeline's output is converted to a string for the LLM. By default, the full result dict is serialized. Use `{"source": "key"}` to extract a single output key, or add `"handler"` to apply a custom formatter.
- `inputs_from_state` is optional and maps agent state keys to pipeline input parameters. Example: `{"repository": "repo"}` passes the state value at `"repository"` as the pipeline's `"repo"` input.
- `outputs_to_state` is optional and maps pipeline output keys to agent state keys. Example: `{"documents": {"source": "docs"}}` writes the pipeline's `"docs"` output to `"documents"` in state.
## Usage
:::tip
The recommended way to use `PipelineTool` in Haystack is with the [`Agent`](../pipeline-components/agents-1/agent.mdx) component, which manages the tool call loop for you. You can run the `Agent` standalone or add it to a pipeline, as the examples below show.
:::
### Basic Usage
You can create a `PipelineTool` from any existing Haystack pipeline:
The examples on this page use Sentence Transformers components (a ranker and embedders) from the `sentence-transformers-haystack` package. Install it to run the examples:
```shell
pip install sentence-transformers-haystack
```
```python
from haystack import Document, Pipeline
from haystack.tools import PipelineTool
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack_integrations.components.rankers.sentence_transformers import (
SentenceTransformersSimilarityRanker,
)
from haystack.document_stores.in_memory import InMemoryDocumentStore
# Create your pipeline
document_store = InMemoryDocumentStore()
# Add some example documents
document_store.write_documents(
[
Document(
content="Nikola Tesla was a Serbian-American inventor and electrical engineer.",
),
Document(
content="Alternating current (AC) is an electric current which periodically reverses direction.",
),
Document(
content="Thomas Edison promoted direct current (DC) and competed with AC in the War of Currents.",
),
],
)
retrieval_pipeline = Pipeline()
retrieval_pipeline.add_component(
"bm25_retriever",
InMemoryBM25Retriever(document_store=document_store),
)
retrieval_pipeline.add_component(
"ranker",
SentenceTransformersSimilarityRanker(model="cross-encoder/ms-marco-MiniLM-L-6-v2"),
)
retrieval_pipeline.connect("bm25_retriever.documents", "ranker.documents")
# Wrap the pipeline as a tool
retrieval_tool = PipelineTool(
pipeline=retrieval_pipeline,
input_mapping={"query": ["bm25_retriever.query", "ranker.query"]},
output_mapping={"ranker.documents": "documents"},
name="retrieval_tool",
description="Search short articles about Nikola Tesla, AC electricity, and related inventors",
)
print(retrieval_tool)
```
### With the Agent Component
```python
from haystack import Document, Pipeline
from haystack.tools import PipelineTool
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack.components.retrievers import InMemoryEmbeddingRetriever
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.agents import Agent
from haystack.dataclasses import ChatMessage
# Initialize a document store and add some documents
document_store = InMemoryDocumentStore()
document_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
documents = [
Document(
content="Nikola Tesla was a Serbian-American inventor and electrical engineer.",
),
Document(
content="He is best known for his contributions to the design of the modern alternating current (AC) electricity supply system.",
),
]
docs_with_embeddings = document_embedder.run(documents=documents)["documents"]
document_store.write_documents(docs_with_embeddings)
# Build a simple retrieval pipeline
retrieval_pipeline = Pipeline()
retrieval_pipeline.add_component(
"embedder",
SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"),
)
retrieval_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
retrieval_pipeline.connect("embedder.embedding", "retriever.query_embedding")
# Wrap the pipeline as a tool
retriever_tool = PipelineTool(
pipeline=retrieval_pipeline,
input_mapping={"query": ["embedder.text"]},
output_mapping={"retriever.documents": "documents"},
name="document_retriever",
description="For any questions about Nikola Tesla, always use this tool",
)
agent = Agent(
system_prompt="You are an assistant that can use a retrieval tool to find information about Nikola Tesla.",
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[retriever_tool],
)
result = agent.run([ChatMessage.from_user("Who was Nikola Tesla?")])
print("Answer:")
print(result["messages"][-1].text)
```
### In a Pipeline
You can also use `PipelineTool` in a pipeline by passing it to an `Agent` component.
```python
from haystack import Document, Pipeline
from haystack.tools import PipelineTool
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
from haystack.components.retrievers import InMemoryEmbeddingRetriever
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.agents import Agent
from haystack.dataclasses import ChatMessage
# Initialize a document store and add some documents
document_store = InMemoryDocumentStore()
document_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
documents = [
Document(
content="Nikola Tesla was a Serbian-American inventor and electrical engineer.",
),
Document(
content="He is best known for his contributions to the design of the modern alternating current (AC) electricity supply system.",
),
]
docs_with_embeddings = document_embedder.run(documents=documents)["documents"]
document_store.write_documents(docs_with_embeddings)
# Build a simple retrieval pipeline
retrieval_pipeline = Pipeline()
retrieval_pipeline.add_component(
"embedder",
SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"),
)
retrieval_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
retrieval_pipeline.connect("embedder.embedding", "retriever.query_embedding")
# Wrap the pipeline as a tool
retriever_tool = PipelineTool(
pipeline=retrieval_pipeline,
input_mapping={"query": ["embedder.text"]},
output_mapping={"retriever.documents": "documents"},
name="document_retriever",
description="For any questions about Nikola Tesla, always use this tool",
)
pipeline = Pipeline()
pipeline.add_component(
"agent",
Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[retriever_tool],
),
)
message = ChatMessage.from_user(
"Use the document retriever tool to find information about Nikola Tesla",
)
result = pipeline.run({"agent": {"messages": [message]}})
print(result["agent"]["last_message"].text)
```
---
// File: tools/ready-made-tools/e2btoolset
# E2BToolset
A Toolset that gives Agents access to a live [E2B](https://e2b.dev/) cloud sandbox for executing bash commands and managing files.
`name`: The name of the tool
`description`: Description of the tool | | **API reference** | [PipelineTool](/reference/tools-api#pipeline_tool) | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tools/pipeline_tool.py | | **Package name** | `haystack-ai` |
| | |
| --- | --- |
| **Mandatory init variables** | `api_key`: E2B API key. Can be set with `E2B_API_KEY` env var. |
| **API reference** | [E2B](/reference/integrations-e2b) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/e2b |
| **Package name** | `e2b-haystack` |
## Overview
`E2BToolset` bundles four tools that operate inside the same [E2B](https://e2b.dev/) cloud sandbox, giving an Agent a secure, isolated Linux environment to execute code and manipulate files:
- **`run_bash_command`** (`RunBashCommandTool`): Runs a bash command and returns the combined `exit_code`, `stdout`, and `stderr`. Use it for shell scripts, package installation, code compilation, or any system-level operation.
- **`read_file`** (`ReadFileTool`): Reads the text content of a file from the sandbox filesystem.
- **`write_file`** (`WriteFileTool`): Writes text content to a file in the sandbox. Parent directories are created automatically and existing files are overwritten.
- **`list_directory`** (`ListDirectoryTool`): Lists files and subdirectories at a given path.
All four tools share a single `E2BSandbox` instance, so a file written by `write_file` is immediately available to `run_bash_command` and `read_file` in the same Agent run. The toolset owns the sandbox lifecycle: `warm_up()` starts the sandbox, `close()` shuts it down, and YAML serialization round-trips preserve the shared-sandbox relationship.
### Parameters
- `api_key` is _mandatory_ and must be an E2B API key. The default setting uses the environment variable `E2B_API_KEY`. Get a key at [e2b.dev](https://e2b.dev/).
- `sandbox_template` is _optional_ and defaults to `"base"`. Sets the E2B sandbox template to use.
- `timeout` is _optional_ and defaults to `120`. Sets the sandbox inactivity timeout in seconds.
- `environment_vars` is _optional_ and lets you inject environment variables into the sandbox process.
## Usage
Install the E2B integration to use `E2BToolset`:
```shell
pip install e2b-haystack
```
Set your E2B API key:
```shell
export E2B_API_KEY="your-e2b-api-key"
```
### With an Agent
You can use `E2BToolset` with the [Agent](../../pipeline-components/agents-1/agent.mdx) component. The Agent will automatically start the sandbox, invoke the tools to write, run, and inspect code, and let the LLM chain calls together inside the same sandbox process.
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.tools.e2b import E2BToolset
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
tools=E2BToolset(),
system_prompt=(
"You are a helpful coding assistant with access to a live Linux sandbox. "
"Use the available tools freely to explore, write files, and run commands. "
"All tools operate inside the same sandbox environment, so files written "
"with write_file are immediately available to run_bash_command and read_file."
),
max_agent_steps=15,
)
response = agent.run(
messages=[
ChatMessage.from_user(
"Write a Python script to /tmp/primes.py that prints all prime numbers "
"up to 50, run it, and then read the file back so I can see both the "
"script and its output.",
),
],
)
print(response["last_message"].text)
```
### Using individual tools
If you only need a subset of the tools, you can instantiate them directly and pass them a shared `E2BSandbox`:
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.tools.e2b import (
E2BSandbox,
ListDirectoryTool,
ReadFileTool,
RunBashCommandTool,
WriteFileTool,
)
sandbox = E2BSandbox(sandbox_template="base", timeout=300)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
tools=[
RunBashCommandTool(sandbox=sandbox),
ReadFileTool(sandbox=sandbox),
WriteFileTool(sandbox=sandbox),
ListDirectoryTool(sandbox=sandbox),
],
)
```
When using the tools standalone (outside an Agent or Pipeline), call `sandbox.warm_up()` before the first invocation and `sandbox.close()` when you are done to release the cloud resources.
### In a Pipeline
`E2BToolset` is fully serializable, so you can wrap an Agent that uses it in a Pipeline and save the Pipeline to YAML:
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.core.pipeline import Pipeline
from haystack.dataclasses import ChatMessage
from haystack_integrations.tools.e2b import E2BToolset
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
tools=E2BToolset(sandbox_template="base", timeout=120),
system_prompt="You are a helpful coding assistant with access to a live Linux sandbox.",
max_agent_steps=10,
)
pipeline = Pipeline()
pipeline.add_component("agent", agent)
# Serialize and restore - all four tools still share the same E2BSandbox after the round-trip.
yaml_str = pipeline.dumps()
restored = Pipeline.loads(yaml_str)
result = restored.run(
data={
"agent": {
"messages": [
ChatMessage.from_user(
"Write a Python one-liner to /tmp/hello.py that prints "
"'Hello from E2B!', run it, then show me the output.",
),
],
},
},
)
print(result["agent"]["last_message"].text)
```
---
// File: tools/ready-made-tools/githubfileeditortool
# GitHubFileEditorTool
A Tool that allows Agents to edit files in GitHub repositories.
| | |
| --- | --- |
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
| **API reference** | [Tools](/reference/tools-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
| **Package name** | `github-haystack` |
## Overview
`GitHubFileEditorTool` wraps the [`GitHubFileEditor`](../../pipeline-components/connectors/githubfileeditor.mdx) component, providing a tool interface for use in agent workflows and tool-based pipelines.
The tool supports multiple file operations including editing existing files, creating new files, deleting files, and undoing recent changes. It supports four main commands:
- **EDIT**: Edit an existing file by replacing specific content
- **CREATE**: Create a new file with specified content
- **DELETE**: Delete an existing file
- **UNDO**: Revert the last commit if made by the same user
### Parameters
- `name` is _optional_ and defaults to "file_editor". Specifies the name of the tool.
- `description` is _optional_ and provides context to the LLM about what the tool does.
- `github_token` is _mandatory_ and must be a GitHub personal access token for API authentication. The default setting uses the environment variable `GITHUB_TOKEN`.
- `repo` is _optional_ and sets a default repository in owner/repo format.
- `branch` is _optional_ and defaults to "main". Sets the default branch to work with.
- `raise_on_failure` is _optional_ and defaults to `True`. If False, errors are returned instead of raising exceptions.
## Usage
Install the GitHub integration to use the `GitHubFileEditorTool`:
```shell
pip install github-haystack
```
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
Basic usage to edit a file:
```python
from haystack_integrations.tools.github import GitHubFileEditorTool
tool = GitHubFileEditorTool()
result = tool.invoke(
command="edit",
payload={
"path": "src/example.py",
"original": "def old_function():",
"replacement": "def new_function():",
"message": "Renamed function for clarity",
},
repo="owner/repo",
branch="main",
)
print(result)
```
```bash
{'result': 'Edit successful'}
```
### With an Agent
You can use `GitHubFileEditorTool` with the [Agent](../../pipeline-components/agents-1/agent.mdx) component. The Agent will automatically invoke the tool when needed to edit files in GitHub repositories.
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.components.agents import Agent
from haystack_integrations.tools.github import GitHubFileEditorTool
editor_tool = GitHubFileEditorTool(repo="owner/repo")
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[editor_tool],
exit_conditions=["text"],
)
response = agent.run(
messages=[
ChatMessage.from_user(
"Edit the file README.md in the repository \"owner/repo\" and replace the original string 'tpyo' with the replacement 'typo'. This is all context you need.",
),
],
)
print(response["last_message"].text)
```
```bash
The file `README.md` has been successfully edited to correct the spelling of 'tpyo' to 'typo'.
```
---
// File: tools/ready-made-tools/githubissuecommentertool
# GitHubIssueCommenterTool
A Tool that allows Agents to post comments to GitHub issues.
| | |
| --- | --- |
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
| **API reference** | [Tools](/reference/tools-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
| **Package name** | `github-haystack` |
## Overview
`GitHubIssueCommenterTool` wraps the [`GitHubIssueCommenter`](../../pipeline-components/connectors/githubissuecommenter.mdx) component, providing a tool interface for use in agent workflows and tool-based pipelines.
The tool takes a GitHub issue URL and comment text, then posts the comment to the specified issue using the GitHub API. This requires authentication since posting comments is an authenticated operation.
### Parameters
- `name` is _optional_ and defaults to "issue_commenter". Specifies the name of the tool.
- `description` is _optional_ and provides context to the LLM about what the tool does.
- `github_token` is _mandatory_ and must be a GitHub personal access token for API authentication. The default setting uses the environment variable `GITHUB_TOKEN`.
- `raise_on_failure` is _optional_ and defaults to `True`. If False, errors are returned instead of raising exceptions.
- `retry_attempts` is _optional_ and defaults to `2`. Number of retry attempts for failed requests.
## Usage
Install the GitHub integration to use the `GitHubIssueCommenterTool`:
```shell
pip install github-haystack
```
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
Basic usage to comment on an issue:
```python
from haystack_integrations.tools.github import GitHubIssueCommenterTool
tool = GitHubIssueCommenterTool()
result = tool.invoke(
url="https://github.com/owner/repo/issues/123",
comment="Thanks for reporting this issue! We'll look into it.",
)
print(result)
```
```bash
{'success': True}
```
### With an Agent
You can use `GitHubIssueCommenterTool` with the [Agent](../../pipeline-components/agents-1/agent.mdx) component. The Agent will automatically invoke the tool when needed to post comments on GitHub issues.
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.components.agents import Agent
from haystack_integrations.tools.github import GitHubIssueCommenterTool
comment_tool = GitHubIssueCommenterTool(name="github_issue_commenter")
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[comment_tool],
exit_conditions=["text"],
)
response = agent.run(
messages=[
ChatMessage.from_user(
"Please post a helpful comment on this GitHub issue: https://github.com/owner/repo/issues/123 acknowledging the bug report and mentioning that we're investigating",
),
],
)
print(response["last_message"].text)
```
```bash
I have posted the comment on the GitHub issue, acknowledging the bug report and mentioning that the team is investigating the problem. If you need anything else, feel free to ask!
```
---
// File: tools/ready-made-tools/githubissueviewertool
# GitHubIssueViewerTool
A Tool that allows Agents to fetch and parse GitHub issues into documents.
| | |
| --- | --- |
| **API reference** | [Tools](/reference/tools-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
| **Package name** | `github-haystack` |
## Overview
`GitHubIssueViewerTool` wraps the [`GitHubIssueViewer`](../../pipeline-components/connectors/githubissueviewer.mdx) component, providing a tool interface for use in agent workflows and tool-based pipelines.
The tool takes a GitHub issue URL and returns a list of documents where:
- The first document contains the main issue content,
- Subsequent documents contain the issue comments (if any).
Each document includes rich metadata such as the issue title, number, state, creation date, author, and more.
### Parameters
- `name` is _optional_ and defaults to "issue_viewer". Specifies the name of the tool.
- `description` is _optional_ and provides context to the LLM about what the tool does.
- `github_token` is _optional_ but recommended for private repositories or to avoid rate limiting.
- `raise_on_failure` is _optional_ and defaults to `True`. If False, errors are returned as documents instead of raising exceptions.
- `retry_attempts` is _optional_ and defaults to `2`. Number of retry attempts for failed requests.
## Usage
Install the GitHub integration to use the `GitHubIssueViewerTool`:
```shell
pip install github-haystack
```
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
```python
from haystack_integrations.tools.github import GitHubIssueViewerTool
tool = GitHubIssueViewerTool()
result = tool.invoke(url="https://github.com/deepset-ai/haystack/issues/123")
print(result)
```
```bash
{'documents': [Document(id=3989459bbd8c2a8420a9ba7f3cd3cf79bb41d78bd0738882e57d509e1293c67a, content: 'sentence-transformers = 0.2.6.1
haystack = latest
farm = 0.4.3 latest branch
In the call to Emb...', meta: {'type': 'issue', 'title': 'SentenceTransformer no longer accepts \'gpu" as argument', 'number': 123, 'state': 'closed', 'created_at': '2020-05-28T04:49:31Z', 'updated_at': '2020-05-28T07:11:43Z', 'author': 'predoctech', 'url': 'https://github.com/deepset-ai/haystack/issues/123'}), Document(id=a8a56b9ad119244678804d5873b13da0784587773d8f839e07f644c4d02c167a, content: 'Thanks for reporting!
Fixed with #124 ', meta: {'type': 'comment', 'issue_number': 123, 'created_at': '2020-05-28T07:11:42Z', 'updated_at': '2020-05-28T07:11:42Z', 'author': 'tholor', 'url': 'https://github.com/deepset-ai/haystack/issues/123#issuecomment-635153940'})]}
```
### With an Agent
You can use `GitHubIssueViewerTool` with the [Agent](../../pipeline-components/agents-1/agent.mdx) component. The Agent will automatically invoke the tool when needed to fetch GitHub issue information.
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.components.agents import Agent
from haystack_integrations.tools.github import GitHubIssueViewerTool
issue_tool = GitHubIssueViewerTool(name="github_issue_viewer")
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[issue_tool],
exit_conditions=["text"],
)
response = agent.run(
messages=[
ChatMessage.from_user(
"Please analyze this GitHub issue and summarize the main problem: https://github.com/deepset-ai/haystack/issues/123",
),
],
)
print(response["last_message"].text)
```
```bash
The GitHub issue titled "SentenceTransformer no longer accepts 'gpu' as argument" (issue \#123) discusses a problem encountered when using the `EmbeddingRetriever()` function. The user reports that passing the argument `gpu=True` now causes an error because the method that processes this argument does not accept "gpu" anymore; instead, it previously accepted "cuda" without issues.
The user indicates that this change is problematic since it prevents users from instantiating the embedding model with GPU support, forcing them to default to using only the CPU for model execution.
The issue was later closed with a comment indicating it was fixed in another pull request (#124).
```
---
// File: tools/ready-made-tools/githubprcreatortool
# GitHubPRCreatorTool
A Tool that allows Agents to create pull requests from a fork back to the original repository.
| | |
| --- | --- |
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
| **API reference** | [Tools](/reference/tools-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
| **Package name** | `github-haystack` |
## Overview
`GitHubPRCreatorTool` wraps the [`GitHubPRCreator`](../../pipeline-components/connectors/githubprcreator.mdx) component, providing a tool interface for use in agent workflows and tool-based pipelines.
The tool takes a GitHub issue URL and creates a pull request from your fork to the original repository, automatically linking it to the specified issue. It's designed to work with existing forks and assumes you have already made changes in a branch.
### Parameters
- `name` is _optional_ and defaults to "pr_creator". Specifies the name of the tool.
- `description` is _optional_ and provides context to the LLM about what the tool does.
- `github_token` is _mandatory_ and must be a GitHub personal access token from the fork owner. The default setting uses the environment variable `GITHUB_TOKEN`.
- `raise_on_failure` is _optional_ and defaults to `True`. If False, errors are returned instead of raising exceptions.
## Usage
Install the GitHub integration to use the `GitHubPRCreatorTool`:
```shell
pip install github-haystack
```
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
Basic usage to create a pull request:
```python
from haystack_integrations.tools.github import GitHubPRCreatorTool
tool = GitHubPRCreatorTool()
result = tool.invoke(
issue_url="https://github.com/owner/repo/issues/123",
title="Fix issue #123",
body="This PR addresses issue #123 by implementing the requested changes.",
branch="fix-123", # Branch in your fork with the changes
base="main", # Branch in original repo to merge into
)
print(result)
```
```bash
{'result': 'Pull request #16 created successfully and linked to issue #4'}
```
### With an Agent
You can use `GitHubPRCreatorTool` with the [Agent](../../pipeline-components/agents-1/agent.mdx) component. The Agent will automatically invoke the tool when needed to create pull requests.
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.components.agents import Agent
from haystack_integrations.tools.github import GitHubPRCreatorTool
pr_tool = GitHubPRCreatorTool(name="github_pr_creator")
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[pr_tool],
exit_conditions=["text"],
)
response = agent.run(
messages=[
ChatMessage.from_user(
"Create a pull request for issue https://github.com/owner/repo/issues/4 with title 'Fix authentication bug' and empty body using my fix-4 branch and main as target branch",
),
],
)
print(response["last_message"].text)
```
```bash
The pull request titled "Fix authentication bug" has been created successfully and linked to issue [#123](https://github.com/owner/repo/issues/4).
```
---
// File: tools/ready-made-tools/githubrepoviewertool
# GitHubRepoViewerTool
A Tool that allows Agents to navigate and fetch content from GitHub repositories.
| | |
| --- | --- |
| **API reference** | [Tools](/reference/tools-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
| **Package name** | `github-haystack` |
## Overview
`GitHubRepoViewerTool` wraps the [`GitHubRepoViewer`](../../pipeline-components/connectors/githubrepoviewer.mdx) component, providing a tool interface for use in agent workflows and tool-based pipelines.
The tool provides different behavior based on the path type:
- **For directories**: Returns a list of documents, one for each item (files and subdirectories),
- **For files**: Returns a single document containing the file content.
Each document includes rich metadata such as the path, type, size, and URL.
### Parameters
- `name` is _optional_ and defaults to "repo_viewer". Specifies the name of the tool.
- `description` is _optional_ and provides context to the LLM about what the tool does.
- `github_token` is _optional_ but recommended for private repositories or to avoid rate limiting.
- `repo` is _optional_ and sets a default repository in owner/repo format.
- `branch` is _optional_ and defaults to "main". Sets the default branch to work with.
- `raise_on_failure` is _optional_ and defaults to `True`. If False, errors are returned as documents instead of raising exceptions.
- `max_file_size` is _optional_ and defaults to `1,000,000` bytes (1MB). Maximum file size to fetch.
## Usage
Install the GitHub integration to use the `GitHubRepoViewerTool`:
```shell
pip install github-haystack
```
:::info[Repository Placeholder]
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
:::
### On its own
Basic usage to view repository contents:
```python
from haystack_integrations.tools.github import GitHubRepoViewerTool
tool = GitHubRepoViewerTool()
result = tool.invoke(
repo="deepset-ai/haystack",
path="haystack/components",
branch="main",
)
print(result)
```
```bash
{'documents': [Document(id=..., content: 'agents', meta: {'path': 'haystack/components/agents', 'type': 'dir', 'size': 0, 'url': 'https://github.com/deepset-ai/haystack/tree/main/haystack/components/agents'}), Document(id=..., content: 'builders', meta: {'path': 'haystack/components/builders', 'type': 'dir', 'size': 0, 'url': 'https://github.com/deepset-ai/haystack/tree/main/haystack/components/builders'}),...]}
```
### With an Agent
You can use `GitHubRepoViewerTool` with the [Agent](../../pipeline-components/agents-1/agent.mdx) component. The Agent will automatically invoke the tool when needed to explore repository structure and read files.
Note that we set the Agent's `state_schema` parameter in this code example so that the GitHubRepoViewerTool can write documents to the state.
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage, Document
from haystack.components.agents import Agent
from haystack_integrations.tools.github import GitHubRepoViewerTool
repo_tool = GitHubRepoViewerTool(name="github_repo_viewer")
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[repo_tool],
exit_conditions=["text"],
state_schema={"documents": {"type": list[Document]}},
)
response = agent.run(
messages=[
ChatMessage.from_user(
"Can you analyze the structure of the deepset-ai/haystack repository and tell me about the main components?",
),
],
)
print(response["last_message"].text)
```
```bash
The `deepset-ai/haystack` repository has a structured layout that includes several important components. Here's an overview of its main parts:
1. **Directories**:
- **`.github`**: Contains GitHub-specific configuration files and workflows.
- **`docker`**: Likely includes Docker-related files for containerization of the Haystack application.
- **`docs`**: Contains documentation for the Haystack project. This could include guides, API documentation, and other related resources.
- **`e2e`**: This likely stands for "end-to-end", possibly containing tests or examples related to end-to-end functionality of the Haystack framework.
- **`examples`**: Includes example scripts or notebooks demonstrating how to use Haystack.
- **`haystack`**: This is likely the core source code of the Haystack framework itself, containing the main functionality and classes.
- **`proposals`**: A directory that may contain proposals for new features or changes to the Haystack project.
- **`releasenotes`**: Contains notes about various releases, including changes and improvements.
- **`test`**: This directory likely contains unit tests and other testing utilities to ensure code quality and functionality.
2. **Files**:
- **`.gitignore`**: Specifies files and directories that should be ignored by Git.
- **`.pre-commit-config.yaml`**: Configuration file for pre-commit hooks to automate code quality checks.
- **`CITATION.cff`**: Might include information on how to cite the repository in academic work.
- **`code_of_conduct.txt`**: Contains the code of conduct for contributors and users of the repository.
- **`CONTRIBUTING.md`**: Guidelines for contributing to the repository.
- **`LICENSE`**: The license under which the project is distributed.
- **`VERSION.txt`**: Contains versioning information for the project.
- **`README.md`**: A markdown file that usually provides an overview of the project, installation instructions, and usage examples.
- **`SECURITY.md`**: Contains information about the security policy of the repository.
This structure indicates a well-organized repository that follows common conventions in open-source projects, with a focus on documentation, contribution guidelines, and testing. The core functionalities are likely housed in the `haystack` directory, with additional resources provided in the other directories.
```
---
// File: tools/ready-made-tools/mem0memorytools
# Mem0 Memory Tools
The Mem0 integration provides two ready-made Tools for Agent memory workflows:
- **`retrieve_memories`** (`Mem0MemoryRetrieverTool`) searches long-term memories, or returns all scoped memories when no query is provided.
- **`store_memory`** (`Mem0MemoryWriterTool`) stores durable facts, preferences, and context as long-term memories.
| | |
| --- | --- |
| **Mandatory init variables** | `memory_store`: A `Mem0MemoryStore` instance. |
| **Environment variables** | `MEM0_API_KEY`: Your Mem0 cloud API key. |
| **Mem0 API docs** | [Search Memories](https://docs.mem0.ai/api-reference/memory/search-memories), [Add Memories](https://docs.mem0.ai/api-reference/memory/add-memories) |
| **API reference** | [Mem0](/reference/integrations-mem0) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mem0 |
| **Package name** | `mem0-haystack` |
## Overview
Use these tools when an [Agent](../../pipeline-components/agents-1/agent.mdx) needs persistent memory across conversations. The retriever tool gives the Agent access to memories stored in Mem0, and the writer tool lets the Agent save new information that should be useful in future runs.
Both tools use a shared `Mem0MemoryStore`. By default, they inject `user_id` from [Agent State](../../pipeline-components/agents-1/state.mdx) through `inputs_from_state`, so one Agent instance can serve multiple users without exposing user IDs to the LLM as tool-call parameters.
`Mem0MemoryRetrieverTool` exposes `query` and `top_k` to the LLM. If the Agent omits `query` or passes `null`, the tool returns all memories in the injected scope. This is useful when the Agent needs to inspect known context before deciding whether a more specific memory search is necessary.
`Mem0MemoryWriterTool` exposes `text` and `infer` to the LLM. The writer tool uses `infer=False` by default so the Agent stores exactly the memory text it chose. Use `infer=True` when you want Mem0 to extract memories from longer text, such as a conversation transcript.
### Parameters
`Mem0MemoryRetrieverTool`:
- `memory_store` is _mandatory_. It is the `Mem0MemoryStore` instance to query.
- `top_k` is _optional_ and defaults to `5`. It sets the default maximum number of memories returned for query searches.
- `name` is _optional_ and defaults to `"retrieve_memories"`.
- `description` is _optional_ and describes the tool to the LLM.
- `parameters` is _optional_ and lets you override the JSON schema exposed to the LLM.
- `inputs_from_state` is _optional_ and defaults to `{"user_id": "user_id"}`.
`Mem0MemoryWriterTool`:
- `memory_store` is _mandatory_. It is the `Mem0MemoryStore` instance to write to.
- `name` is _optional_ and defaults to `"store_memory"`.
- `description` is _optional_ and describes the tool to the LLM.
- `parameters` is _optional_ and lets you override the JSON schema exposed to the LLM.
- `inputs_from_state` is _optional_ and defaults to `{"user_id": "user_id"}`.
To pass more Mem0 entity IDs at runtime, add the fields to the Agent's `state_schema` and map those State keys to the tool parameters with `inputs_from_state`. For example, `{"user_id": "user_id", "session_id": "run_id"}` passes `state["session_id"]` to the tool's `run_id` parameter.
At least one Mem0 scope must be available when retrieving or storing memories. Use `user_id` for the common per-user case, or add `run_id`, `agent_id`, or `app_id` when your application needs a narrower scope.
## Usage
Install the Mem0 integration:
```shell
pip install mem0-haystack
```
Set your Mem0 API key:
```shell
export MEM0_API_KEY="your-mem0-api-key"
```
### With an Agent
You can use both tools with an Agent to read memories at the beginning of a turn and write new durable memories before the final answer.
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack_integrations.memory_stores.mem0 import Mem0MemoryStore
from haystack_integrations.tools.mem0 import (
Mem0MemoryRetrieverTool,
Mem0MemoryWriterTool,
)
store = Mem0MemoryStore()
retrieve_memories = Mem0MemoryRetrieverTool(memory_store=store, top_k=10)
store_memory = Mem0MemoryWriterTool(memory_store=store)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4"),
tools=[retrieve_memories, store_memory],
system_prompt="""You are a helpful assistant with long-term memory.
At the beginning of each turn, call retrieve_memories without a query to inspect known memories.
Use store_memory only for new durable user-specific facts, preferences, or project context.
Before storing, compare the proposed memory with retrieved memories and avoid duplicates.
Do not store transient requests that are only useful in the current conversation.
""",
streaming_callback=print_streaming_chunk,
state_schema={"user_id": {"type": str}},
)
result = agent.run(
messages=[
ChatMessage.from_user(
"My name is Alice. Please remember that I prefer concise Python examples.",
),
],
user_id="alice",
)
```
### Pass more IDs through State
Mem0 supports scoping memories with `user_id`, `run_id`, `agent_id`, and `app_id`. The tools expose only `user_id` by default, but you can inject more IDs through Agent State without adding them to the LLM-facing parameter schema.
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack_integrations.memory_stores.mem0 import Mem0MemoryStore
from haystack_integrations.tools.mem0 import (
Mem0MemoryRetrieverTool,
Mem0MemoryWriterTool,
)
store = Mem0MemoryStore()
inputs_from_state = {
"user_id": "user_id",
# Map the Agent State key "conversation_id" to the tool's "run_id" parameter.
"conversation_id": "run_id",
}
retrieve_memories = Mem0MemoryRetrieverTool(
memory_store=store,
inputs_from_state=inputs_from_state,
)
store_memory = Mem0MemoryWriterTool(
memory_store=store,
inputs_from_state=inputs_from_state,
)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4"),
tools=[retrieve_memories, store_memory],
state_schema={
"user_id": {"type": str},
"conversation_id": {"type": str},
},
streaming_callback=print_streaming_chunk,
)
result = agent.run(
messages=[
ChatMessage.from_user(
"Remember that this conversation is about the docs assistant prototype.",
),
],
user_id="alice",
conversation_id="docs-assistant-prototype",
)
```
---
// File: tools/ready-made-tools/mirageshelltool
# MirageShellTool
A Tool that gives Agents a bash shell over a [Mirage](https://github.com/strukto-ai/mirage) unified virtual filesystem, mounting backends like S3, Google Drive, and Postgres as one file tree.
| | |
| --- | --- |
| **Mandatory init variables** | `workspace`: A `MirageWorkspace` describing the mount tree the Agent can access. |
| **API reference** | [Mirage](/reference/integrations-mirage) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mirage |
| **Package name** | `mirage-haystack` |
## Overview
`MirageShellTool` hands an [Agent](../../pipeline-components/agents-1/agent.mdx) a single shell over a [Mirage](https://github.com/strukto-ai/mirage) *unified virtual filesystem*: one directory tree that mounts heterogeneous backends — object storage, databases, SaaS apps, and local disk — side by side. Instead of pre-loading file contents into the prompt, the Agent explores the mounted data itself by running ordinary bash commands (`ls`, `cat`, `grep`, `wc`, …). Command output is normalized to text and truncated before it reaches the model.
The tool is backed by two serializable helpers you compose the workspace with:
- **`MirageWorkspace`**: A description of the mount tree that lazily builds a live Mirage workspace. It is the shared backend behind the tool, and you can also use it directly — without an Agent — through its `run()` / `run_async()` methods.
- **`MirageMount`**: A declarative description of a single backend: *where* it is mounted (`path`), *which* backend it is (`resource`, a Mirage registry name such as `"s3"`, `"gdrive"`, `"postgres"`, `"disk"`, or `"ram"`), and *how* it is configured (`config`). Credentials can be passed as Haystack `Secret`s and are resolved only when the live workspace is built.
Because every backend is mounted the same way, one tool gives the Agent uniform access to S3, Google Drive, Slack, Gmail, Redis, Postgres, local disk, and more — swap a `MirageMount` and the Agent's commands stay the same. Mirage never shells out to the host, so the Agent's blast radius is confined to the mounts you attach (see [Security model](#security-model)).
### Parameters
- `workspace` is _mandatory_ and must be a `MirageWorkspace` describing the mounts the Agent can access.
- `name` is _optional_ and defaults to `"mirage_shell"`. Sets the tool name exposed to the LLM.
- `description` is _optional_. A custom tool description; when not set, one is generated from the mount tree.
- `invocation_timeout` is _optional_ and defaults to `60.0`. Maximum seconds to wait for a command to finish.
- `max_output_chars` is _optional_ and defaults to `20000`. Command output is truncated to this many characters before being returned to the model.
- `allowed_commands` is _optional_. If set, only these command names may run (for example `["ls", "cat", "grep"]`). See [Security model](#security-model).
- `denied_paths` is _optional_. If set, any command referencing one of these path substrings is rejected.
## Usage
Install the Mirage integration to use `MirageShellTool`:
```shell
pip install mirage-haystack
```
### With an Agent
You can use `MirageShellTool` with the [Agent](../../pipeline-components/agents-1/agent.mdx) component. The Agent starts the workspace on `warm_up()`, then drives the tool with bash to answer questions by exploring the mounted files itself.
The example below builds a small "log triage" Agent. A directory of log files is mounted read-only, and the Agent inspects it with bash to answer a question. It uses a local `disk` mount so it is fully self-contained; swap the `MirageMount` for `s3`, `gdrive`, `postgres`, … to point the same Agent at a different backend.
```python
import os
import tempfile
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.tools.mirage import (
MirageMount,
MirageShellTool,
MirageWorkspace,
)
# Create some sample data on disk (in a real setup this already exists).
data_dir = tempfile.mkdtemp(prefix="mirage-logs-")
with open(os.path.join(data_dir, "api.log"), "w") as fh:
fh.write(
"INFO request /health 200\nERROR db connection timeout\nERROR db connection timeout\n",
)
with open(os.path.join(data_dir, "worker.log"), "w") as fh:
fh.write("INFO job 41 done\nERROR job 42 failed: OutOfMemory\n")
# Describe the workspace. The read-only mount is the authoritative write boundary:
# Mirage refuses any write to it regardless of the command the model chooses.
workspace = MirageWorkspace(
mounts=[
MirageMount(
path="/logs",
resource="disk",
config={"root": data_dir},
read_only=True,
),
],
)
tool = MirageShellTool(workspace, allowed_commands=["ls", "cat", "grep", "head", "wc"])
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
tools=[tool],
system_prompt=(
"You are a log-triage assistant. A virtual filesystem is available through the `mirage_shell` "
"tool. Use bash commands (ls, cat, grep, wc, ...) to inspect the mounted files under /logs before "
"answering. Base your answer only on what the files actually show."
),
)
response = agent.run(
messages=[
ChatMessage.from_user(
"Across all files in /logs, what is the single most common ERROR message, "
"and how many times does it occur?",
),
],
)
print(response["last_message"].text)
tool.close()
```
### Running commands without an Agent
`MirageWorkspace` can be used on its own, which is handy for testing a mount tree or building non-agentic pipelines. When using it standalone, call `warm_up()` before the first invocation (or let the first `run()` build it lazily) and `close()` when you are done to release resources.
```python
from haystack_integrations.tools.mirage import MirageMount, MirageWorkspace
workspace = MirageWorkspace(
mounts=[
MirageMount(path="/data", resource="ram"), # in-memory scratch space
MirageMount(
path="/s3",
resource="s3",
config={"bucket": "my-bucket"},
read_only=True,
),
],
)
print(workspace.run("ls /s3"))
print(workspace.run("grep -r alert /s3/logs | wc -l"))
workspace.close()
```
### Mounting credentialed backends
Backends that need credentials take them through `config`. Pass secrets as Haystack `Secret`s so they are resolved only when the live workspace is built and are never serialized in plaintext:
```python
from haystack.utils import Secret
from haystack_integrations.tools.mirage import MirageMount
MirageMount(path="/data", resource="ram") # in-memory scratch
MirageMount(path="/local", resource="disk", config={"root": "/srv/data"}) # local disk
MirageMount(path="/s3", resource="s3", config={"bucket": "my-bucket"}, read_only=True)
MirageMount(
path="/drive",
resource="gdrive",
config={
"client_id": "...",
"refresh_token": Secret.from_env_var("GDRIVE_REFRESH_TOKEN"),
},
read_only=True,
)
```
Discover the backend names available in your Mirage install with `MirageMount.available_resources()`; the config keys each backend expects come from that backend's Mirage config class.
## Security model
Mirage never shells out to the host: every command runs inside Mirage's own virtual-filesystem interpreter, so an Agent's blast radius is confined to the mounts you attach. Two controls shape what an Agent can do:
- **Per-mount read-only mode** (`MirageMount(..., read_only=True)`) is the authoritative write boundary. Mirage refuses any write to a read-only mount regardless of the command used — this is how you prevent modification or deletion. Mount anything the Agent should not change as read-only.
- **The command allowlist** (`allowed_commands`) restricts *which* commands may run. It is enforced against every command Mirage would execute, including commands nested inside `$(...)`, backticks, `<(...)`, and subshells, so `ls "$(rm x)"` is rejected unless `rm` is also allowed. Treat it as a best-effort filter to steer the Agent, not a sandbox: allowing a command that itself runs other commands (`eval`, `bash`, `sh`, `source`, `xargs`, `timeout`) effectively allows anything, so do not list those for untrusted or hosted use.
- **`denied_paths`** rejects any command whose text references one of the given path substrings.
---
// File: tools/ready-made-tools/tavilywebsearchtool
# TavilyWebSearchTool
A Tool that allows Agents to search the web with Tavily.
| | |
| --- | --- |
| **Mandatory init variables** | `api_key`: The Tavily API key. Can be set with the `TAVILY_API_KEY` env var. |
| **API reference** | [Tavily](/reference/integrations-tavily) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/tavily/src/haystack_integrations/tools/tavily/websearch_tool.py |
| **Package name** | `tavily-haystack` |
## Overview
`TavilyWebSearchTool` wraps the [`TavilyWebSearch`](../../pipeline-components/websearch/tavilywebsearch.mdx) component, providing a tool interface for use in agent workflows and tool-based pipelines.
The tool parameters are derived from the component's `run` method, so the LLM can pass a `query` and, optionally, `search_params` that override the ones set at initialization time.
Results are formatted as a string, with each result showing a title, the exact URL, and a content snippet. This makes it straightforward for the LLM to cite its sources.
### Parameters
All parameters are keyword-only.
- `api_key` is _mandatory_ and holds the Tavily API key. The default setting reads it from the `TAVILY_API_KEY` environment variable.
- `top_k` is _optional_ and sets the maximum number of results to return. If unset, the `TavilyWebSearch` default applies.
- `search_params` is _optional_ and takes additional parameters for the Tavily search API. Supported keys include `search_depth`, `include_answer`, `include_raw_content`, `include_domains`, and `exclude_domains`.
- `name` is _optional_ and defaults to "web_search". Specifies the name of the tool.
- `description` is _optional_ and provides context to the LLM about what the tool does. If not provided, a default description is applied.
## Usage
Install the Tavily integration to use the `TavilyWebSearchTool`:
```shell
pip install tavily-haystack
```
### On its own
Basic usage to search the web:
```python
from haystack_integrations.tools.tavily import TavilyWebSearchTool
tool = TavilyWebSearchTool(top_k=3)
result = tool.invoke(query="What is Haystack by deepset?")
for document in result["documents"]:
print(document.meta["title"], "-", document.meta["url"])
```
```bash
GitHub - deepset-ai/haystack: Open-source AI orchestration framework ... - https://github.com/deepset-ai/haystack
deepset - Wikipedia - https://en.wikipedia.org/wiki/Deepset
Haystack | Haystack - https://haystack.deepset.ai
```
### With an Agent
You can use `TavilyWebSearchTool` with the [Agent](../../pipeline-components/agents-1/agent.mdx) component. The Agent will automatically invoke the tool when it needs information from the web.
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.tools.tavily import TavilyWebSearchTool
web_search = TavilyWebSearchTool(top_k=5, search_params={"search_depth": "advanced"})
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5-mini"),
tools=[web_search],
)
result = agent.run(messages=[ChatMessage.from_user("What is Haystack by deepset?")])
print(result["last_message"].text)
```
```bash
Haystack (by deepset) is an open-source Python framework for building production-ready
LLM applications, especially Retrieval-Augmented Generation (RAG), semantic search,
question answering, and agentic workflows.
It provides modular components and pipelines (document stores, retrievers, rankers,
generators, routers, and tool integrations) so you can compose and control how data
flows before a model sees it.
Source repo: https://github.com/deepset-ai/haystack
```
---
// File: tools/ready-made-tools
# Ready-Made Tools
Ready-made Tools and Toolsets provide prebuilt capabilities for common Agent workflows. You can pass them directly to an [Agent](../pipeline-components/agents-1/agent.mdx), which executes them for you, or inspect their function and parameter schema when you need to customize their behavior.
These are the ready-made Tools and Toolsets available in Haystack:
| Tool or Toolset | Description |
| --- | --- |
| [E2BToolset](ready-made-tools/e2btoolset.mdx) | Gives Agents access to a live E2B cloud sandbox for executing bash commands and managing files. |
| [GitHubFileEditorTool](ready-made-tools/githubfileeditortool.mdx) | Edits files in GitHub repositories. |
| [GitHubIssueCommenterTool](ready-made-tools/githubissuecommentertool.mdx) | Posts comments to GitHub issues. |
| [GitHubIssueViewerTool](ready-made-tools/githubissueviewertool.mdx) | Fetches and parses GitHub issues into documents. |
| [GitHubPRCreatorTool](ready-made-tools/githubprcreatortool.mdx) | Creates pull requests from a fork back to the original repository. |
| [GitHubRepoViewerTool](ready-made-tools/githubrepoviewertool.mdx) | Navigates and fetches content from GitHub repositories. |
| [Mem0MemoryRetrieverTool](ready-made-tools/mem0memorytools.mdx) | Retrieves long-term memories from Mem0 for Agent workflows. |
| [Mem0MemoryWriterTool](ready-made-tools/mem0memorytools.mdx) | Stores long-term memories in Mem0 for future Agent runs. |
| [MirageShellTool](ready-made-tools/mirageshelltool.mdx) | Gives Agents a bash shell over a Mirage unified virtual filesystem, mounting backends like S3, Google Drive, and Postgres as one file tree. |
| [TavilyWebSearchTool](ready-made-tools/tavilywebsearchtool.mdx) | Searches the web with Tavily and returns results Agents can cite. |
---
// File: tools/searchabletoolset
# SearchableToolset
Enable agents to dynamically discover tools from large catalogs using keyword-based search.
| | |
| --- | --- |
| **Mandatory init variables** | `catalog`: A list of Tools and/or Toolsets, or a single Toolset |
| **API reference** | [SearchableToolset](/reference/tools-api#searchabletoolset) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tools/searchable_toolset.py |
| **Package name** | `haystack-ai` |
## Overview
`SearchableToolset` is designed for working with large tool catalogs.
Instead of exposing all tools at once, which can overwhelm the LLM context, it provides a single `search_tools` bootstrap tool.
The agent uses this tool to find and load specific tools from the catalog using BM25 keyword search.
Once the agent calls `search_tools`, the matching tools become immediately available and the agent can invoke them in
subsequent iterations.
### Modes of operation
`SearchableToolset` operates in one of two modes depending on catalog size:
- **Search mode** (default for large catalogs): The agent starts with only the `search_tools` bootstrap tool and discovers other tools on demand. This is activated when the catalog size meets or exceeds `search_threshold`.
- **Passthrough mode** (small catalogs): All tools are exposed directly, with no discovery step needed. This is activated automatically when the catalog has fewer tools than `search_threshold`.
### Parameters
- `catalog` (required): The source of tools — a list of `Tool` and/or `Toolset` instances, or a single `Toolset`. This includes [MCPTool](mcptool.mdx) and [MCPToolset](mcptoolset.mdx) instances.
- `top_k` (optional): The default number of tools returned by each `search_tools` call. Default is `3`.
- `search_threshold` (optional): Minimum catalog size to activate search mode. Catalogs smaller than this value use passthrough mode instead. Default is `8`.
:::info
`SearchableToolset` does not support adding new tools after initialization or merging with other toolsets. Use `catalog` to provide all tools upfront.
:::
### Warm-up
`SearchableToolset` builds its search index during `warm_up()`. When used with an [`Agent`](../pipeline-components/agents-1/agent.mdx), constructing the Agent does not trigger this — warm-up happens when you call `Agent.warm_up()` or automatically at run time.
All tool names in the catalog must be unique: `warm_up()` raises a `ValueError` if the catalog contains tools with duplicate names, since a search hit could otherwise resolve to the wrong tool.
The Agent evaluates its `exit_conditions` at runtime, so an exit condition can name any tool in the catalog, even one the agent has not discovered yet.
## Usage
### Basic usage with an Agent
```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import create_tool_from_function, SearchableToolset
def get_weather(city: Annotated[str, "The city to get the weather for"]) -> str:
"""Get current weather for a city."""
return f"Sunny, 22°C in {city}"
def search_web(query: Annotated[str, "The search query"]) -> str:
"""Search the web for information."""
return f"Results for: {query}"
# Build a catalog from tools
catalog = [
create_tool_from_function(get_weather),
create_tool_from_function(search_web),
# ... many more tools
]
toolset = SearchableToolset(catalog=catalog)
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=toolset,
)
# The agent initially sees only `search_tools`. It will call it to find relevant tools,
# then use the discovered tools to answer the question.
result = agent.run(messages=[ChatMessage.from_user("What's the weather in Milan?")])
print(result["messages"][-1].text)
```
### Customizing the bootstrap tool
You can customize the name, description, and parameter descriptions of the `search_tools` bootstrap tool:
- `search_tool_name`: Custom name for the bootstrap tool. Default is `"search_tools"`.
- `search_tool_description`: Custom description for the bootstrap tool.
- `search_tool_parameters_description`: Custom descriptions for the bootstrap tool's parameters. Keys must be a subset of `{"tool_keywords", "k"}`.
```python
toolset = SearchableToolset(
catalog=catalog,
search_tool_name="find_tools",
search_tool_description="Search for tools in the catalog by keyword.",
search_tool_parameters_description={
"tool_keywords": "Keywords to find tools, e.g. 'email send'",
"k": "Max number of tools to return",
},
)
```
### Reusing the toolset across multiple agent runs
You can safely reuse the same `SearchableToolset` instance across multiple agent runs, including concurrent ones. Each `Agent` run operates on an isolated, run-scoped copy of the toolset (created with [`spawn()`](toolset.mdx#run-scoped-copies-and-tool-selection)), so tools discovered in one run do not persist into, or collide with, other runs — every run starts fresh from the catalog:
```python
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=toolset,
)
result1 = agent.run(messages=[ChatMessage.from_user("What's the weather in Milan?")])
# The next run starts fresh: tools discovered in the previous run are not carried over
result2 = agent.run(messages=[ChatMessage.from_user("Search for news about AI.")])
```
If you drive the toolset directly (outside an `Agent`), you can call `clear()` to reset the discovered tools yourself.
---
// File: tools/skilltoolset
# SkillToolset
Let agents discover and read skills — reusable instruction sets with bundled files — through progressive disclosure.
| | |
| --- | --- |
| **Mandatory init variables** | `store`: A `SkillStore` instance that provides the skills |
| **API reference** | [SkillToolset](/reference/tools-api#skilltoolset) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tools/skills/skill_toolset.py |
| **Package name** | `haystack-ai` |
## Overview
A *skill* is a directory (or equivalent storage unit) containing a `SKILL.md` file with YAML frontmatter (`description` is required; `name` is optional and defaults to the directory name) and a markdown body of instructions. Skills may bundle additional files, such as reference docs, examples, or templates.
`SkillToolset` lets an [`Agent`](../pipeline-components/agents-1/agent.mdx) use skills through *progressive disclosure*, similar to how coding assistants like Claude Code expose skills: the model first sees only each skill's name and description, loads the full instructions when a task calls for them, and fetches bundled files only when the instructions reference them. This keeps the context small even with many detailed skills.
The toolset exposes two tools:
- `load_skill`: Returns a skill's full instructions on demand, plus a manifest of its bundled files. The names and descriptions of all discovered skills are baked into this tool's description at warm-up, so the model can see which skills exist without any system prompt injection.
- `read_skill_file`: Reads a file bundled with a skill (with path-traversal protection).
Skills are discovered when the toolset is warmed up — the `Agent` does this automatically before a run. Constructing the toolset does not read any skills.
`SkillToolset` is backed by a `SkillStore`. Use the built-in `FileSystemSkillStore` to load skills from a local directory, or implement the `SkillStore` protocol (`list_skills`, `load_skill`, `read_skill_file`, plus serialization methods) to back the toolset with any storage system — a database, a remote API, and so on.
:::info
The tool names `load_skill` and `read_skill_file` are fixed, so an `Agent` can use at most one `SkillToolset`. It also does not support adding tools or concatenation with other toolsets — to combine it with other tools, pass it to the `Agent` alongside them, for example `tools=[skills_toolset, other_tool]`. To serve skills from multiple sources, back a single toolset with a custom store that merges them.
:::
### Skill format
`FileSystemSkillStore` expects one sub-directory per skill under a root directory:
```
skills/
pdf-forms/
SKILL.md # frontmatter (description required, name optional) + markdown instructions
reference/forms.md # optional bundled file
```
A minimal `SKILL.md` looks like this:
```markdown
---
name: pdf-forms
description: Fill in PDF forms programmatically. Use when the user asks to complete or fill a PDF form.
---
# Filling PDF forms
1. Inspect the form fields first...
2. For the full field reference, read `reference/forms.md`.
```
Only the frontmatter of each `SKILL.md` is read at warm-up to build the catalog; instruction bodies and bundled files are read lazily when the agent calls the corresponding tool.
### Multimodal skill assets
`read_skill_file` returns text files as strings, images as [`ImageContent`](../concepts/data-classes/imagecontent.mdx), and PDFs as [`FileContent`](../concepts/data-classes/filecontent.mdx). Image and file results are passed to the model as content parts of the tool result instead of being converted to a string, so an `Agent` backed by a multimodal chat generator that supports these inputs (for example, `OpenAIResponsesChatGenerator`) can read a skill's visual assets — such as a reference screenshot or a showcase PDF — directly. Binary files that are neither images nor PDFs are rejected with an error.
### Executing bundled scripts
`SkillToolset` only *reads* skills — `load_skill` and `read_skill_file` never execute anything. If your skills bundle executable scripts (for example, a Python helper that the instructions tell the model to run), pass a script-execution tool of your own to the `Agent` alongside the toolset:
```python
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[skills_toolset, run_shell_command_tool], # your own execution tool
)
```
The agent can then read a bundled script with `read_skill_file` and run it through your execution tool. Since such a tool runs model-chosen commands, scope it carefully — restrict what it can execute, sandbox it, or guard it with a [Human in the Loop](../pipeline-components/agents-1/human-in-the-loop.mdx) confirmation strategy.
## Usage
### With an Agent
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.skill_stores.file_system import FileSystemSkillStore
from haystack.tools import SkillToolset
store = FileSystemSkillStore("skills/")
skills_toolset = SkillToolset(store)
agent = Agent(chat_generator=OpenAIChatGenerator(), tools=skills_toolset)
# The agent sees the available skills in the `load_skill` tool description,
# loads the matching skill, and follows its instructions.
result = agent.run(messages=[ChatMessage.from_user("Fill in this PDF form for me.")])
print(result["last_message"].text)
```
### Inspecting discovered skills
The `skills` property returns the metadata of all discovered skills as a mapping of skill name to `SkillInfo` (warming up the toolset first if needed):
```python
from haystack.skill_stores.file_system import FileSystemSkillStore
from haystack.tools import SkillToolset
skills_toolset = SkillToolset(FileSystemSkillStore("skills/"))
for name, info in skills_toolset.skills.items():
print(f"{name}: {info.description}")
```
---
// File: tools/tool
# Tool
`Tool` is a data class representing a function that Language Models can prepare a call for.
A growing number of Language Models now support passing tool definitions alongside the prompt.
Tool calling refers to the ability of Language Models to generate calls to tools - be they functions or APIs - when responding to user queries. The model prepares the tool call but does not execute it.
If you are looking for the details of this data class's methods and parameters, visit our [API documentation](/reference/tools-api).
## Tool class
`Tool` is a simple and unified abstraction to represent tools in the Haystack framework.
A tool is a function for which Language Models can prepare a call.
The `Tool` class is used in Chat Generators and provides a consistent experience across models. `Tool` is also used by the [`Agent`](../pipeline-components/agents-1/agent.mdx) component, which executes the calls prepared by Language Models.
```python
@dataclass
class Tool:
name: str
description: str
parameters: dict[str, Any]
function: Callable | None = None
outputs_to_string: dict[str, Any] | None = None
inputs_from_state: dict[str, str] | None = None
outputs_to_state: dict[str, dict[str, Any]] | None = None
async_function: Callable | None = None
```
- `name` is the name of the Tool.
- `description` is a string describing what the Tool does.
- `parameters` is a JSON schema describing the expected parameters.
- `function` is invoked when the Tool is called. It must be a regular (sync) function.
- `async_function` (optional) is a coroutine function awaited when the Tool is invoked in an async context. See [Async Tools](#async-tools) below.
- `outputs_to_string` (optional) controls how parts of the tool’s output are converted into one or more strings (e.g. for LLM consumption).
- `inputs_from_state` (optional) maps values from the agent state to the tool’s input parameters (e.g. to share info between tools)
- `outputs_to_state` (optional) specifies how tool outputs are written back into the agent state, with optional handlers.
Keep in mind that the accurate definitions of `name` and `description` are important for the Language Model to prepare the call correctly.
`Tool` exposes a `tool_spec` property, returning the tool specification to be used by Language Models.
It also has an `invoke` method that executes the underlying function with the provided parameters.
## Tool Initialization
There are three ways to create a `Tool`:
- **`@tool` decorator** — recommended for most cases; infers name, description, and schema from the function.
- **`create_tool_from_function`** — same as `@tool` but called as a function; useful when you can’t decorate directly.
- **Manual initialization** — construct `Tool(...)` directly when you need full control over the JSON schema.
:::tip
For most use cases, we recommend `@tool` or `create_tool_from_function`. Both automatically generate the `parameters` JSON schema from your function’s type hints and [`Annotated`](https://docs.python.org/3/library/typing.html#typing.Annotated) parameter descriptions, so you don’t need to write the schema by hand.
:::
### @tool decorator
The `@tool` decorator converts a function into a Tool. It infers the name, description, and parameters from the function and automatically generates a JSON schema. Use `typing.Annotated` to add descriptions to individual parameters. When called without arguments (`@tool`), defaults are inferred from the function. When called with arguments (`@tool(name=..., outputs_to_state=...)`), you can customize any of the Tool fields.
```python
from typing import Annotated, Literal
from haystack.tools import tool
@tool
def get_weather(
city: Annotated[str, "the city for which to get the weather"] = "Munich",
unit: Annotated[
Literal["Celsius", "Fahrenheit"],
"the unit for the temperature",
] = "Celsius",
):
"""A simple function to get the current weather for a location."""
return f"Weather report for {city}: 20 {unit}, sunny"
print(get_weather)
```
```
Tool(
name=’get_weather’,
description=’A simple function to get the current weather for a location.’,
parameters={
‘type’: ‘object’,
‘properties’: {
‘city’: {‘type’: ‘string’, ‘description’: ‘the city for which to get the weather’, ‘default’: ‘Munich’},
‘unit’: {
‘type’: ‘string’,
‘enum’: [‘Celsius’, ‘Fahrenheit’],
‘description’: ‘the unit for the temperature’,
‘default’: ‘Celsius’,
},
},
},
function=
| | |
| --- | --- |
| **Mandatory init variables** | `tools`: A list of tools |
| **API reference** | [Toolset](/reference/tools-api#toolset) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tools/toolset.py |
| **Package name** | `haystack-ai` |
## Overview
A `Toolset` groups multiple Tool instances into a single manageable unit. It simplifies passing tools to components like Chat Generators or [`Agent`](../pipeline-components/agents-1/agent.mdx), and supports filtering, serialization, and reuse.
Additionally, by subclassing `Toolset`, you can create implementations that dynamically load tools from external sources like OpenAPI URLs, MCP servers, or other resources.
### Initializing Toolset
Here’s how to initialize `Toolset` with [Tool](tool.mdx). Alternatively, you can use [ComponentTool](componenttool.mdx) or [MCPTool](mcptool.mdx) in `Toolset` as Tool instances.
```python
from typing import Annotated
from haystack.tools import Toolset, tool
@tool
def add_numbers(
a: Annotated[int, "first number"],
b: Annotated[int, "second number"],
) -> int:
"""Add two numbers."""
return a + b
@tool
def subtract_numbers(
a: Annotated[int, "first number"],
b: Annotated[int, "second number"],
) -> int:
"""Subtract b from a."""
return a - b
math_toolset = Toolset([add_numbers, subtract_numbers])
```
### Adding New Tools to Toolset
```python
from typing import Annotated
from haystack.tools import tool
@tool
def multiply_numbers(
a: Annotated[int, "first number"],
b: Annotated[int, "second number"],
) -> int:
"""Multiply two numbers."""
return a * b
math_toolset.add(multiply_numbers)
```
### Combining Toolsets
To use multiple Toolsets together, pass them as a list wherever tools are accepted:
```python
agent = Agent(
chat_generator=OpenAIChatGenerator(), tools=[math_toolset, another_toolset]
)
```
### Run-Scoped Copies and Tool Selection
An [`Agent`](../pipeline-components/agents-1/agent.mdx) run never modifies your configured `Toolset`. A `Toolset` with per-run state, such as a [`SearchableToolset`](searchabletoolset.mdx), is copied for each run through its `spawn()` method, so concurrent runs cannot leak state (like discovered tools) into each other. A plain `Toolset` has no per-run state and is shared as is; just avoid adding or removing tools while runs are in progress.
You can also restrict an `Agent` to a subset of tools at runtime by passing tool names, for example `agent.run(tools=["tool_a", "tool_b"])`. The selection applies only to that run, and dynamic behavior like a `SearchableToolset`'s search keeps working over the selected subset.
Two methods support this and can be overridden when subclassing:
- `get_selectable_tools()`: Returns every tool available for name-based selection. Override it if your subclass's iteration does not surface every selectable tool.
- `spawn()`: Returns the `Toolset` itself, which has no run-scoped state to isolate. Override it to return an isolated, run-scoped copy if your subclass holds run-scoped state.
## Usage
You can use `Toolset` wherever you can use Tools in Haystack.
:::tip
The recommended way to use a `Toolset` in Haystack is with the [`Agent`](../pipeline-components/agents-1/agent.mdx) component, which manages the tool call loop for you. The examples below also show how to pass a `Toolset` directly to a `ChatGenerator` for cases where you need fine-grained control.
:::
### With the Agent
```python
from haystack.components.agents import Agent
from haystack.dataclasses import ChatMessage
from haystack.components.generators.chat import OpenAIChatGenerator
agent = Agent(
system_prompt="You are a helpful assistant that can do math using the tools at your disposal.",
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=math_toolset,
)
response = agent.run(messages=[ChatMessage.from_user("What is 4 + 2?")])
print(response["messages"][-1].text)
```
Output:
```
4 + 2 equals 6.
```
### With a ChatGenerator
You can pass a `Toolset` directly to a Chat Generator. The model prepares the tool calls; executing them (for example, with `Tool.invoke`) is up to you:
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
chat_generator = OpenAIChatGenerator(model="gpt-5.4-nano", tools=math_toolset)
user_message = ChatMessage.from_user("What is 10 minus 5?")
replies = chat_generator.run(messages=[user_message])["replies"]
print(f"assistant message: {replies}")
# If the assistant message contains a tool call, execute it
if replies[0].tool_calls:
tool_call = replies[0].tool_calls[0]
tool = next(t for t in math_toolset if t.name == tool_call.tool_name)
print(f"tool result: {tool.invoke(**tool_call.arguments)}")
```
Output:
```
assistant message: [ChatMessage(
_role=