
Most common position in a pipeline	Before a Generator
Mandatory init variables	"template": A list of `ChatMessage` objects. Needs to be provided either during init or run.
Mandatory run variables	“**kwargs”: Any strings that should be used to render the prompt template
Output variables	“prompt”: A dynamically constructed prompt
API reference	Builders
GitHub link	https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/chat_prompt_builder.py

Overview

The ChatPromptBuilder component creates prompts using static or dynamic templates written in Jinja2 syntax, by processing a list of chat messages. The templates contain placeholders like {{ variable }} that are filled with values provided during runtime. You can use it for static prompts set at initialization or change the templates and variables dynamically while running.

To use it, start by providing a list of ChatMessage objects as the template. ChatMessage is a data class that includes message content, a role (who generated the message, such as user, assistant, system, function), and optional metadata.

The builder looks for placeholders in the template and identifies the required variables. You can also list these variables manually. During runtime, the run method takes the template and the variables, fills in the placeholders, and returns the completed prompt. If required variables are missing. If the template is invalid, the builder raises an error.

For example, you can create a simple translation prompt:

template = [ChatMessage.from_user("Translate to {{ target_language }}: {{ text }}")]
builder = ChatPromptBuilder(template=template)
result = builder.run(target_language="French", text="Hello, how are you?")

Or you can also replace the template at runtime with a new one:

new_template = [ChatMessage.from_user("Summarize in {{ target_language }}: {{ content }}")]
result = builder.run(template=new_template, target_language="English", content="A detailed paragraph.")

Variables

The template variables found in the init template are used as input types for the component and are all optional unless explicitly specified. If an optional template variable is not provided as an input, it will be replaced with an empty string in the rendered prompt. Use variables and required_variables to specify the input types and required variables.

template = [ChatMessage.from_user("Hello, {{ name }}. How can I assist you with {{ topic }}?")]

builder = ChatPromptBuilder(template=template, required_variables=["name"], variables=["name", "topic"])

result = builder.run(name="Alice")
# Output: "Hello, Alice. How can I assist you with ?"

Roles

A ChatMessage represents a single message in the conversation and can have one of three roles: from_user, from_system, or from_assistant. Messages with the from_user role are inputs provided by the user, such as a query or request. Messages with the from_system role provide context or instructions to guide the LLM’s behavior, such as setting a tone or purpose for the conversation. The from_assistant role defines the expected or actual response from the LLM.

Here’s how the roles work together in a ChatPromptBuilder:

system_message = ChatMessage.from_system("You are an assistant helping tourists in {{ language }}.")

user_message = ChatMessage.from_user("What are the best places to visit in {{ city }}?")

assistant_message = ChatMessage.from_assistant("The best places to visit in {{ city }} include the Eiffel Tower, Louvre Museum, and Montmartre.")

Usage

On its own

With static template

from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage

template = [ChatMessage.from_user("Translate to {{ target_language }}. Context: {{ snippet }}; Translation:")]
builder = ChatPromptBuilder(template=template)
builder.run(target_language="spanish", snippet="I can't speak spanish.")

Overriding static template at runtime

from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage

template = [ChatMessage.from_user("Translate to {{ target_language }}. Context: {{ snippet }}; Translation:")]
builder = ChatPromptBuilder(template=template)
builder.run(target_language="spanish", snippet="I can't speak spanish.")

summary_template = [ChatMessage.from_user("Translate to {{ target_language }} and summarize. Context: {{ snippet }}; Summary:")]
builder.run(target_language="spanish", snippet="I can't speak spanish.", template=summary_template)

In a pipeline

from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack import Pipeline
from haystack.utils import Secret

# no parameter init, we don't use any runtime template variables
prompt_builder = ChatPromptBuilder()
llm = OpenAIChatGenerator()

pipe = Pipeline()
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("prompt_builder.prompt", "llm.messages")

location = "Berlin"
language = "English"
system_message = ChatMessage.from_system("You are an assistant giving information to tourists in {{language}}")
messages = [system_message, ChatMessage.from_user("Tell me about {{location}}")]

res = pipe.run(data={"prompt_builder": {"template_variables": {"location": location, "language": language},
                                    "template": messages}})
print(res)

Then, you could ask about the weather forecast for the said location. The ChatPromptBuilder fills in the template with the new day_count variable and passes it to an LLM once again:

from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack import Pipeline
from haystack.utils import Secret

# no parameter init, we don't use any runtime template variables
prompt_builder = ChatPromptBuilder()
llm = OpenAIChatGenerator()

pipe = Pipeline()
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("prompt_builder.prompt", "llm.messages")

location = "Berlin"

messages = [system_message, ChatMessage.from_user("What's the weather forecast for {{location}} in the next {{day_count}} days?")]
res = pipe.run(data={"prompt_builder": {"template_variables": {"location": location, "day_count": "5"},
                                    "template": messages}})

print(res)

Additional References

🧑‍🍳 Cookbook: Advanced Prompt Customization for Anthropic