Skip to main content
Version: 3.5

Guardrails

Guardrails help keep your pipelines and Agents within limits you set: what content goes in and out of the model, how much a run spends, how long it runs, which tools it may call without approval, and what it returns. This page lists guardrails Haystack offers, what each guards against, and where it plugs in.

A guardrail commonly plugs in at one of these places:

  • Provider setting: The model provider checks the request and the reply. You turn it on in the chat generator.
  • Pipeline component: Runs as its own step in a pipeline, for example to clean a query or route a message before it reaches a chat generator or an Agent.
  • Agent hook: Runs inside an Agent's run loop. Pass it to the Agent's hooks parameter. See Hooks.

Guardrails for pipelines​

These guardrails can be used in a pipeline, whether it calls a chat generator directly or runs an Agent.

GuardrailGuards againstPlugs in asLearn more
Provider moderationHarmful content in the prompt or the replyProvider settingContent moderation by the provider
LLMMessagesRouter with a guard modelHarmful content, checked by a model you choosePipeline componentLLMMessagesRouter
PresidioTextCleaner, PresidioDocumentCleanerPII reaching the model or the indexPipeline componentPresidioTextCleaner, PresidioDocumentCleaner
PresidioEntityExtractorPII you need to detect and act on yourselfPipeline componentPresidioEntityExtractor

Content moderation by the provider​

Several model providers check prompts and replies for harmful content. When a provider blocks a reply, most of the chat generators listed below set the reply's finish_reason to "content_filter". The reply may hold partial text. In a pipeline, read it from the reply's meta:

python
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.dataclasses import ChatMessage

generator = OpenAIResponsesChatGenerator(model="gpt-6-luna")
result = generator.run(messages=[ChatMessage.from_user("Your question here")])
reply = result["replies"][0]
if reply.meta.get("finish_reason") == "content_filter":
print("The provider blocked this reply.")

By default, an Agent stops on a blocked reply and returns exit_reason="content_filter":

python
from haystack.components.agents import Agent

agent = Agent(chat_generator=generator)
result = agent.run(messages=[ChatMessage.from_user("Your question here")])
if result["exit_reason"] == "content_filter":
print("The provider blocked this reply.")

These examples use OpenAIResponsesChatGenerator, so they need the OPENAI_API_KEY environment variable. With another provider from the table below, swap in its chat generator and set that provider's credentials instead.

ProviderBuilt-in moderationHow to turn it on
Amazon BedrockGuardrailsguardrail_config init parameter
Google GeminiSafety settingssafety_settings init parameter
MistralGuardrails, safety system promptguardrails in extra_body, safe_prompt in generation_kwargs
Azure OpenAIContent filtersConfigured on the Azure deployment, see AzureOpenAIChatGenerator and AzureOpenAIResponsesChatGenerator
AnthropicThe model refuses on its ownNothing to set, see AnthropicChatGenerator
CohereSafety modessafety_mode in generation_kwargs

Content moderation with a guard model​

To moderate content with a model you choose, put an LLMMessagesRouter with a guard model, such as Llama Guard, Granite Guardian, ShieldGemma, or NemoGuard, in front of a chat generator or an Agent. The router can send safe messages on and route unsafe ones to a separate output. This is useful for self-hosted models, or to apply one moderation policy across several providers.

PII​

The Presidio integration finds and replaces personally identifiable information (PII), such as names, emails, and phone numbers:

All three take a score_threshold that sets how confident Presidio must be before it treats text as PII.

Guardrails for Agents​

These guardrails are specific to the Agent.

GuardrailGuards againstPlugs in asLearn more
max_agent_stepsA run that loops without finishingAgent init parameterAgent
stop_run state keyA condition you check in your own hookState key, set from your own hook (for example in before_llm)Stopping a run from a hook
TokenBudgetHookA run using more tokens than you want to pay forAgent hook (before_llm)Token Budget
StepBudgetReminderHookThe Agent hitting max_agent_steps before it answersAgent hook (before_llm)Step Budget Reminder
ConfirmationHookTools running without a human's approvalAgent hook (before_tool)Human in the Loop
StructuredOutputHookA final reply that doesn't match a Pydantic model or JSON schemaAgent hook (on_exit)Structured Output

Budgets and step limits​

TokenBudgetHook stops the run before the next LLM call once the run has used max_total_tokens. The run ends with exit_reason="token_budget_exceeded".

max_agent_steps caps the number of LLM and tool call iterations. It defaults to 100. When the Agent reaches it, the run ends with exit_reason="max_agent_steps". StepBudgetReminderHook tells the model how many steps it has left, so it can wrap up in time.

Stopping a run from a hook​

A hook can stop the run by setting the stop_run state key. The Agent reads it before each LLM call and returns its value as the exit_reason. Use it for checks that no ready-made hook covers. This hook stops the run once the Agent has called the search tool five times:

python
from haystack.components.agents.state import State
from haystack.hooks import hook


@hook
def limit_searches(state: State) -> None:
if state.get("tool_call_counts", {}).get("search", 0) >= 5:
state.set("stop_run", "search_limit_reached")

Register it with hooks={"before_llm": [limit_searches]}. See State keys for hooks.

Tool approval​

ConfirmationHook lets a human confirm, modify, or reject a tool call before it runs. A policy decides when to ask: AlwaysAskPolicy asks on every call, AskOncePolicy asks once per tool and set of arguments, and NeverAskPolicy never asks. You set a strategy per tool, so you can require approval only for risky tools such as send_email.

Output validation​

StructuredOutputHook parses the Agent's final reply as JSON and validates it against a Pydantic model or a JSON schema. When the reply is invalid, the hook sends the error back to the model and asks it to fix the reply, up to max_retries times.