Guardrails
Guardrails help keep your pipelines and Agents within limits you set: what content goes in and out of the model, how much a run spends, how long it runs, which tools it may call without approval, and what it returns. This page lists guardrails Haystack offers, what each guards against, and where it plugs in.
A guardrail commonly plugs in at one of these places:
- Provider setting: The model provider checks the request and the reply. You turn it on in the chat generator.
- Pipeline component: Runs as its own step in a pipeline, for example to clean a query or route a message before it reaches a chat generator or an Agent.
- Agent hook: Runs inside an
Agent's run loop. Pass it to the Agent'shooksparameter. See Hooks.
Guardrails for pipelines
These guardrails can be used in a pipeline, whether it calls a chat generator directly or runs an Agent.
| Guardrail | Guards against | Plugs in as | Learn more |
|---|---|---|---|
| Provider moderation | Harmful content in the prompt or the reply | Provider setting | Content moderation by the provider |
LLMMessagesRouter with a guard model | Harmful content, checked by a model you choose | Pipeline component | LLMMessagesRouter |
PresidioTextCleaner, PresidioDocumentCleaner | PII reaching the model or the index | Pipeline component | PresidioTextCleaner, PresidioDocumentCleaner |
PresidioEntityExtractor | PII you need to detect and act on yourself | Pipeline component | PresidioEntityExtractor |
Content moderation by the provider
Several model providers check prompts and replies for harmful content. When a provider blocks a reply, most of the chat generators listed below set the reply's finish_reason to "content_filter". The reply may hold partial text. In a pipeline, read it from the reply's meta:
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.dataclasses import ChatMessage
generator = OpenAIResponsesChatGenerator(model="gpt-6-luna")
result = generator.run(messages=[ChatMessage.from_user("Your question here")])
reply = result["replies"][0]
if reply.meta.get("finish_reason") == "content_filter":
print("The provider blocked this reply.")
By default, an Agent stops on a blocked reply and returns exit_reason="content_filter":
from haystack.components.agents import Agent
agent = Agent(chat_generator=generator)
result = agent.run(messages=[ChatMessage.from_user("Your question here")])
if result["exit_reason"] == "content_filter":
print("The provider blocked this reply.")
These examples use OpenAIResponsesChatGenerator, so they need the OPENAI_API_KEY environment variable. With another provider from the table below, swap in its chat generator and set that provider's credentials instead.
| Provider | Built-in moderation | How to turn it on |
|---|---|---|
| Amazon Bedrock | Guardrails | guardrail_config init parameter |
| Google Gemini | Safety settings | safety_settings init parameter |
| Mistral | Guardrails, safety system prompt | guardrails in extra_body, safe_prompt in generation_kwargs |
| Azure OpenAI | Content filters | Configured on the Azure deployment, see AzureOpenAIChatGenerator and AzureOpenAIResponsesChatGenerator |
| Anthropic | The model refuses on its own | Nothing to set, see AnthropicChatGenerator |
| Cohere | Safety modes | safety_mode in generation_kwargs |
Content moderation with a guard model
To moderate content with a model you choose, put an LLMMessagesRouter with a guard model, such as Llama Guard, Granite Guardian, ShieldGemma, or NemoGuard, in front of a chat generator or an Agent. The router can send safe messages on and route unsafe ones to a separate output. This is useful for self-hosted models, or to apply one moderation policy across several providers.
PII
The Presidio integration finds and replaces personally identifiable information (PII), such as names, emails, and phone numbers:
PresidioTextCleanerreplaces PII in strings, for example in a user query before it reaches the model.PresidioDocumentCleanerreplaces PII in documents, for example before you index them.PresidioEntityExtractorfinds PII and returns the entities, so you can decide what to do with them.
All three take a score_threshold that sets how confident Presidio must be before it treats text as PII.
Guardrails for Agents
These guardrails are specific to the Agent.
| Guardrail | Guards against | Plugs in as | Learn more |
|---|---|---|---|
max_agent_steps | A run that loops without finishing | Agent init parameter | Agent |
stop_run state key | A condition you check in your own hook | State key, set from your own hook (for example in before_llm) | Stopping a run from a hook |
TokenBudgetHook | A run using more tokens than you want to pay for | Agent hook (before_llm) | Token Budget |
StepBudgetReminderHook | The Agent hitting max_agent_steps before it answers | Agent hook (before_llm) | Step Budget Reminder |
ConfirmationHook | Tools running without a human's approval | Agent hook (before_tool) | Human in the Loop |
StructuredOutputHook | A final reply that doesn't match a Pydantic model or JSON schema | Agent hook (on_exit) | Structured Output |
Budgets and step limits
TokenBudgetHook stops the run before the next LLM call once the run has used max_total_tokens. The run ends with exit_reason="token_budget_exceeded".
max_agent_steps caps the number of LLM and tool call iterations. It defaults to 100. When the Agent reaches it, the run ends with exit_reason="max_agent_steps". StepBudgetReminderHook tells the model how many steps it has left, so it can wrap up in time.
Stopping a run from a hook
A hook can stop the run by setting the stop_run state key. The Agent reads it before each LLM call and returns its value as the exit_reason. Use it for checks that no ready-made hook covers. This hook stops the run once the Agent has called the search tool five times:
from haystack.components.agents.state import State
from haystack.hooks import hook
@hook
def limit_searches(state: State) -> None:
if state.get("tool_call_counts", {}).get("search", 0) >= 5:
state.set("stop_run", "search_limit_reached")
Register it with hooks={"before_llm": [limit_searches]}. See State keys for hooks.
Tool approval
ConfirmationHook lets a human confirm, modify, or reject a tool call before it runs. A policy decides when to ask: AlwaysAskPolicy asks on every call, AskOncePolicy asks once per tool and set of arguments, and NeverAskPolicy never asks. You set a strategy per tool, so you can require approval only for risky tools such as send_email.
Output validation
StructuredOutputHook parses the Agent's final reply as JSON and validates it against a Pydantic model or a JSON schema. When the reply is invalid, the hook sends the error back to the model and asks it to fix the reply, up to max_retries times.