TransformersTextRouter

Use this component to route text input to various output connections based on a model-defined categorization label.

Most common position in a pipeline	Flexible
Mandatory init variables	model: The name or path of a Hugging Face model for text classification
Mandatory run variables	text: The text to be routed to one of the specified outputs based on which label it has been categorized into
Output variables	<label>: The input text, on the output connection named after the label the model assigned to it. There's one output per label.
API reference	Transformers
GitHub link	https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/transformers
Package name	transformers-haystack

Overview

TransformersTextRouter routes text input to various output connections based on its categorization label. This is useful for routing queries to different models in a pipeline depending on their categorization.

First, you need to set a selected model with a model parameter when initializing the component. The selected model then provides the set of labels for categorization.

You can additionally provide the labels parameter – a list of strings of possible class labels to classify each sequence into. If not provided, the component fetches the labels from the model configuration file hosted on the HuggingFace Hub using transformers.AutoConfig.from_pretrained.

Authentication with a Hugging Face API token is only required to access private or gated models. You can pass the token at initialization with token, or set the HF_API_TOKEN or HF_TOKEN environment variable.

To see the full list of parameters, check out our API reference.

Usage

Install the transformers-haystack package to use the TransformersTextRouter:

shell

pip install transformers-haystack

On its own

The TransformersTextRouter isn’t very effective on its own, as its main strength lies in working within a pipeline. The component's true potential is unlocked when it is integrated into a pipeline, where it can efficiently route text to the most appropriate components. Please see the following section for a complete example of usage.

In a pipeline

Below is an example of a simple pipeline that routes English queries to a Text Generator optimized for English text and German queries to a Text Generator optimized for German text.

python

from haystack import Pipeline
from haystack_integrations.components.routers.transformers import TransformersTextRouter
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack_integrations.components.generators.transformers import (
    TransformersChatGenerator,
)
from haystack.dataclasses import ChatMessage

p = Pipeline()

p.add_component(
    instance=TransformersTextRouter(
        model="papluca/xlm-roberta-base-language-detection",
    ),
    name="text_router",
)
p.add_component(
    instance=ChatPromptBuilder(
        template=[ChatMessage.from_user("Answer the question: {{query}}\nAnswer:")],
        required_variables={"query"},
    ),
    name="english_prompt_builder",
)
p.add_component(
    instance=ChatPromptBuilder(
        template=[ChatMessage.from_user("Beantworte die Frage: {{query}}\nAntwort:")],
        required_variables={"query"},
    ),
    name="german_prompt_builder",
)
p.add_component(
    instance=TransformersChatGenerator(
        model="DiscoResearch/Llama3-DiscoLeo-Instruct-8B-v0.1",
    ),
    name="german_llm",
)
p.add_component(
    instance=TransformersChatGenerator(model="microsoft/Phi-3-mini-4k-instruct"),
    name="english_llm",
)

p.connect("text_router.en", "english_prompt_builder.query")
p.connect("text_router.de", "german_prompt_builder.query")
p.connect("english_prompt_builder.prompt", "english_llm.messages")
p.connect("german_prompt_builder.prompt", "german_llm.messages")

# English Example
print(p.run({"text_router": {"text": "What is the capital of Germany?"}}))

# German Example
print(p.run({"text_router": {"text": "Was ist die Hauptstadt von Deutschland?"}}))

Additional References

📓 Tutorial: Query Classification with TransformersTextRouter and TransformersZeroShotTextRouter