DocumentationAPI Reference📓 Tutorials🧑‍🍳 Cookbook🤝 Integrations💜 Discord🎨 Studio
API Reference

Agent API

Uses a large language model to answer complex queries that require multiple steps to find the correct answer.

Module base

Tool

class Tool()

Agent uses tools to find the best answer. A tool is a pipeline or a node. When you add a tool to an Agent, the Agent

can invoke the underlying pipeline or node to answer questions.

You must provide a name and a description for each tool. The name should be short and should indicate what the tool can do. The description should explain what the tool is useful for. The Agent uses the description to decide when to use a tool, so the wording you use is important.

Arguments:

  • name: The name of the tool. The Agent uses this name to refer to the tool in the text the Agent generates. The name should be short, ideally one token, and a good description of what the tool can do, for example: "Calculator" or "Search". Use only letters (a-z, A-Z), digits (0-9) and underscores (_)."
  • pipeline_or_node: The pipeline or node to run when the Agent invokes this tool.
  • description: A description of what the tool is useful for. The Agent uses this description to decide when to use which tool. For example, you can describe a tool for calculations by "useful for when you need to

answer questions about math".

ToolsManager

class ToolsManager()

The ToolsManager manages tools for an Agent.

ToolsManager.__init__

def __init__(
    tools: Optional[List[Tool]] = None,
    tool_pattern:
    str = r"Tool:\s*(\w+)\s*Tool Input:\s*(?:\"([\s\S]*?)\"|((?:.|\n)*))\s*")

Arguments:

  • tools: A list of tools to add to the ToolManager. Each tool must have a unique name.
  • tool_pattern: A regular expression pattern that matches the text that the Agent generates to invoke a tool.

ToolsManager.add_tool

def add_tool(tool: Tool)

Add a tool to the Agent. This also updates the PromptTemplate for the Agent's PromptNode with the tool name.

Arguments:

  • tool: The tool to add to the Agent. Any previously added tool with the same name will be overwritten. Example: `agent.add_tool( Tool( name="Calculator", pipeline_or_node=calculator description="Useful when you need to answer questions about math." ) )

ToolsManager.get_tool_names

def get_tool_names() -> str

Returns a string with the names of all registered tools.

ToolsManager.get_tools

def get_tools() -> List[Tool]

Returns a list of all registered tool instances.

ToolsManager.get_tool_names_with_descriptions

def get_tool_names_with_descriptions() -> str

Returns a string with the names and descriptions of all registered tools.

ToolsManager.extract_tool_name_and_tool_input

def extract_tool_name_and_tool_input(
        llm_response: str) -> Tuple[Optional[str], Optional[str]]

Parse the tool name and the tool input from the PromptNode response.

Arguments:

  • llm_response: The PromptNode response.

Returns:

A tuple containing the tool name and the tool input.

Agent

class Agent()

An Agent answers queries using the tools you give to it. The tools are pipelines or nodes. The Agent uses a large language model (LLM) through the PromptNode you initialize it with. To answer a query, the Agent follows this sequence:

  1. It generates a thought based on the query.
  2. It decides which tool to use.
  3. It generates the input for the tool.
  4. Based on the output it gets from the tool, the Agent can either stop if it now knows the answer or repeat the process of 1) generate thought, 2) choose tool, 3) generate input.

Agents are useful for questions containing multiple sub questions that can be answered step-by-step (Multi-hop QA) using multiple pipelines and nodes as tools.

Agent.__init__

def __init__(prompt_node: PromptNode,
             prompt_template: Optional[Union[str, PromptTemplate]] = None,
             tools_manager: Optional[ToolsManager] = None,
             memory: Optional[Memory] = None,
             prompt_parameters_resolver: Optional[Callable] = None,
             max_steps: int = 8,
             final_answer_pattern: str = r"Final Answer\s*:\s*(.*)",
             streaming: bool = True)

Creates an Agent instance.

Arguments:

  • prompt_node: The PromptNode that the Agent uses to decide which tool to use and what input to provide to it in each iteration.
  • prompt_template: The name of a PromptTemplate for the PromptNode. It's used for generating thoughts and choosing tools to answer queries step-by-step. You can use the default zero-shot-react template or create a new template in a similar format. with add_tool() before running the Agent.
  • tools_manager: A ToolsManager instance that the Agent uses to run tools. Each tool must have a unique name. You can also add tools with add_tool() before running the Agent.
  • memory: A Memory instance that the Agent uses to store information between iterations.
  • prompt_parameters_resolver: A callable that takes query, agent, and agent_step as parameters and returns a dictionary of parameters to pass to the prompt_template. The default is a callable that returns a dictionary of keys and values needed for the React agent prompt template.
  • max_steps: The number of times the Agent can run a tool +1 to let it infer it knows the final answer. Set it to at least 2, so that the Agent can run one a tool once and then infer it knows the final answer. The default is 8.
  • final_answer_pattern: A regular expression to extract the final answer from the text the Agent generated.
  • streaming: Whether to use streaming or not. If True, the Agent will stream response tokens from the LLM. If False, the Agent will wait for the LLM to finish generating the response and then process it. The default is True.

Agent.update_hash

def update_hash()

Used for telemetry. Hashes the tool classnames to send an event only when they change. See haystack/telemetry.py::send_event

Agent.add_tool

def add_tool(tool: Tool)

Add a tool to the Agent. This also updates the PromptTemplate for the Agent's PromptNode with the tool name.

Arguments:

  • tool: The tool to add to the Agent. Any previously added tool with the same name will be overwritten. Example: `agent.add_tool( Tool( name="Calculator", pipeline_or_node=calculator description="Useful when you need to answer questions about math" ) )

Agent.has_tool

def has_tool(tool_name: str) -> bool

Check whether the Agent has a tool with the name you provide.

Arguments:

  • tool_name: The name of the tool for which you want to check whether the Agent has it.

Agent.run

def run(query: str,
        max_steps: Optional[int] = None,
        params: Optional[dict] = None) -> Dict[str, Union[str, List[Answer]]]

Runs the Agent given a query and optional parameters to pass on to the tools used. The result is in the

same format as a pipeline's result: a dictionary with a key answers containing a list of answers.

Arguments:

  • query: The search query
  • max_steps: The number of times the Agent can run a tool +1 to infer it knows the final answer. If you want to set it, make it at least 2 so that the Agent can run a tool once and then infer it knows the final answer.
  • params: A dictionary of parameters you want to pass to the tools that are pipelines. To pass a parameter to all nodes in those pipelines, use the format: {"top_k": 10}. To pass a parameter to targeted nodes in those pipelines, use the format: {"Retriever": {"top_k": 10}, "Reader": {"top_k": 3}}. You can only pass parameters to tools that are pipelines, but not nodes.

Agent.create_agent_step

def create_agent_step(max_steps: Optional[int] = None) -> AgentStep

Create an AgentStep object. Override this method to customize the AgentStep class used by the Agent.

Agent.prepare_data_for_memory

def prepare_data_for_memory(**kwargs) -> dict

Prepare data for saving to the Agent's memory. Override this method to customize the data saved to the memory.