Skip to main content
Version: 2.32-unstable

Generators

chat/azure​

AzureOpenAIChatGenerator​

Bases: OpenAIChatGenerator

Generates text using OpenAI's models on Azure.

It works with the gpt-4 - type models and supports streaming responses from OpenAI API. It uses ChatMessage format in input and output.

You can customize how the text is generated by passing parameters to the OpenAI API. Use the **generation_kwargs argument when you initialize the component or when you run it. Any parameter that works with openai.ChatCompletion.create will work here too.

For details on OpenAI API parameters, see OpenAI documentation.

Usage example​

python
from haystack.components.generators.chat import AzureOpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret

messages = [ChatMessage.from_user("What's Natural Language Processing?")]

client = AzureOpenAIChatGenerator(
    azure_endpoint="<Your Azure endpoint e.g. `https://your-company.azure.openai.com/>",
    api_key=Secret.from_token("<your-api-key>"),
    azure_deployment="<this is a model name, e.g. gpt-4.1-mini>")
response = client.run(messages)
print(response)
{'replies':
    [ChatMessage(_role=<ChatRole.ASSISTANT: 'assistant'>, _content=[TextContent(text=
    "Natural Language Processing (NLP) is a branch of artificial intelligence that focuses on
     enabling computers to understand, interpret, and generate human language in a way that is useful.")],
     _name=None,
     _meta={'model': 'gpt-4.1-mini', 'index': 0, 'finish_reason': 'stop',
     'usage': {'prompt_tokens': 15, 'completion_tokens': 36, 'total_tokens': 51}})]
}

SUPPORTED_MODELS​

python
SUPPORTED_MODELS: list[str] = [
    "gpt-5.4",
    "gpt-5.4-pro",
    "gpt-5.3-codex",
    "gpt-5.2",
    "gpt-5.2-codex",
    "gpt-5.2-chat",
    "gpt-5.1",
    "gpt-5.1-chat",
    "gpt-5.1-codex",
    "gpt-5.1-codex-mini",
    "gpt-5",
    "gpt-5-mini",
    "gpt-5-nano",
    "gpt-5-chat",
    "gpt-4.1",
    "gpt-4.1-mini",
    "gpt-4.1-nano",
    "gpt-4o",
    "gpt-4o-mini",
    "gpt-4o-audio-preview",
    "gpt-realtime-1.5",
    "gpt-audio-1.5",
    "o1",
    "o1-mini",
    "o3",
    "o3-mini",
    "o4-mini",
    "codex-mini",
    "gpt-4",
    "gpt-35-turbo",
    "gpt-oss-120b",
    "computer-use-preview",
]

A non-exhaustive list of chat models supported by this component. See https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure for the full list.

init​

python
__init__(
    azure_endpoint: str | Secret | None = None,
    api_version: str | Secret | None = "2024-12-01-preview",
    azure_deployment: str | None = "gpt-4.1-mini",
    api_key: Secret | None = Secret.from_env_var(
        "AZURE_OPENAI_API_KEY", strict=False
    ),
    azure_ad_token: Secret | None = Secret.from_env_var(
        "AZURE_OPENAI_AD_TOKEN", strict=False
    ),
    organization: str | None = None,
    streaming_callback: StreamingCallbackT | None = None,
    timeout: float | None = None,
    max_retries: int | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    default_headers: dict[str, str] | None = None,
    tools: ToolsType | None = None,
    tools_strict: bool = False,
    *,
    azure_ad_token_provider: (
        AzureADTokenProvider | AsyncAzureADTokenProvider | None
    ) = None,
    http_client_kwargs: dict[str, Any] | None = None
) -> None

Initialize the Azure OpenAI Chat Generator component.

Parameters:

  • azure_endpoint (str | Secret | None) – The endpoint of the deployed model, for example "https://example-resource.azure.openai.com/". Can also be a Secret, for example Secret.from_env_var("AZURE_OPENAI_ENDPOINT"), to resolve the value from an environment variable at runtime. This is useful to switch endpoints between environments (e.g. dev and prod) without changing the serialized pipeline.
  • api_version (str | Secret | None) – The version of the API to use. Defaults to 2024-12-01-preview. Can also be a Secret, for example Secret.from_env_var("AZURE_OPENAI_API_VERSION"), to resolve the value from an environment variable at runtime.
  • azure_deployment (str | None) – The deployment of the model, usually the model name.
  • api_key (Secret | None) – The API key to use for authentication.
  • azure_ad_token (Secret | None) – Azure Active Directory token.
  • organization (str | None) – Your organization ID, defaults to None. For help, see Setting up your organization.
  • streaming_callback (StreamingCallbackT | None) – A callback function called when a new token is received from the stream. It accepts StreamingChunk as an argument.
  • timeout (float | None) – Timeout for OpenAI client calls. If not set, it defaults to either the OPENAI_TIMEOUT environment variable, or 30 seconds.
  • max_retries (int | None) – Maximum number of retries to contact OpenAI after an internal error. If not set, it defaults to either the OPENAI_MAX_RETRIES environment variable, or set to 5.
  • generation_kwargs (dict[str, Any] | None) – Other parameters to use for the model. These parameters are sent directly to the OpenAI endpoint. For details, see OpenAI documentation. Some of the supported parameters:
  • max_completion_tokens: An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.
  • temperature: The sampling temperature to use. Higher values mean the model takes more risks. Try 0.9 for more creative applications and 0 (argmax sampling) for ones with a well-defined answer.
  • top_p: Nucleus sampling is an alternative to sampling with temperature, where the model considers tokens with a top_p probability mass. For example, 0.1 means only the tokens comprising the top 10% probability mass are considered.
  • n: The number of completions to generate for each prompt. For example, with 3 prompts and n=2, the LLM will generate two completions per prompt, resulting in 6 completions total.
  • stop: One or more sequences after which the LLM should stop generating tokens.
  • presence_penalty: The penalty applied if a token is already present. Higher values make the model less likely to repeat the token.
  • frequency_penalty: Penalty applied if a token has already been generated. Higher values make the model less likely to repeat the token.
  • logit_bias: Adds a logit bias to specific tokens. The keys of the dictionary are tokens, and the values are the bias to add to that token.
  • response_format: A JSON schema or a Pydantic model that enforces the structure of the model's response. If provided, the output will always be validated against this format (unless the model returns a tool call). For details, see the OpenAI Structured Outputs documentation. Notes:
    • This parameter accepts Pydantic models and JSON schemas for latest models starting from GPT-4o. Older models only support basic version of structured outputs through {"type": "json_object"}. For detailed information on JSON mode, see the OpenAI Structured Outputs documentation.
    • For structured outputs with streaming, the response_format must be a JSON schema and not a Pydantic model.
  • default_headers (dict[str, str] | None) – Default headers to use for the AzureOpenAI client.
  • tools (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls.
  • tools_strict (bool) – Whether to enable strict schema adherence for tool calls. If set to True, the model will follow exactly the schema provided in the parameters field of the tool definition, but this may increase latency.
  • azure_ad_token_provider (AzureADTokenProvider | AsyncAzureADTokenProvider | None) – A function that returns an Azure Active Directory token, will be invoked on every request.
  • http_client_kwargs (dict[str, Any] | None) – A dictionary of keyword arguments to configure a custom httpx.Clientor httpx.AsyncClient. For more information, see the HTTPX documentation.

warm_up​

python
warm_up() -> None

Warm up the tools and initialize the synchronous Azure OpenAI client.

warm_up_async​

python
warm_up_async() -> None

Warm up the tools and initialize the asynchronous Azure OpenAI client on the serving event loop.

close​

python
close() -> None

Releases the synchronous Azure OpenAI client.

close_async​

python
close_async() -> None

Releases the asynchronous Azure OpenAI client.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize this component to a dictionary.

Returns:

  • dict[str, Any] – The serialized component as a dictionary.

from_dict​

python
from_dict(data: dict[str, Any]) -> AzureOpenAIChatGenerator

Deserialize this component from a dictionary.

Parameters:

  • data (dict[str, Any]) – The dictionary representation of this component.

Returns:

  • AzureOpenAIChatGenerator – The deserialized component instance.

chat/azure_responses​

AzureOpenAIResponsesChatGenerator​

Bases: OpenAIResponsesChatGenerator

Completes chats using OpenAI's Responses API on Azure.

It works with the gpt-5 and o-series models and supports streaming responses from OpenAI API. It uses ChatMessage format in input and output.

You can customize how the text is generated by passing parameters to the OpenAI API. Use the **generation_kwargs argument when you initialize the component or when you run it. Any parameter that works with openai.Responses.create will work here too.

For details on OpenAI API parameters, see OpenAI documentation.

Usage example​

python
from haystack.components.generators.chat import AzureOpenAIResponsesChatGenerator
from haystack.dataclasses import ChatMessage

messages = [ChatMessage.from_user("What's Natural Language Processing?")]

client = AzureOpenAIResponsesChatGenerator(
    azure_endpoint="https://example-resource.azure.openai.com/",
    generation_kwargs={"reasoning": {"effort": "low", "summary": "auto"}}
)
response = client.run(messages)
print(response)

SUPPORTED_MODELS​

python
SUPPORTED_MODELS: list[str] = [
    "gpt-5.4-pro",
    "gpt-5.4",
    "gpt-5.3-chat",
    "gpt-5.3-codex",
    "gpt-5.2-codex",
    "gpt-5.2",
    "gpt-5.2-chat",
    "gpt-5.1-codex-max",
    "gpt-5.1",
    "gpt-5.1-chat",
    "gpt-5.1-codex",
    "gpt-5.1-codex-mini",
    "gpt-5-pro",
    "gpt-5-codex",
    "gpt-5",
    "gpt-5-mini",
    "gpt-5-nano",
    "gpt-5-chat",
    "gpt-4o",
    "gpt-4o-mini",
    "computer-use-preview",
    "gpt-4.1",
    "gpt-4.1-nano",
    "gpt-4.1-mini",
    "gpt-image-1",
    "gpt-image-1-mini",
    "gpt-image-1.5",
    "o1",
    "o3-mini",
    "o3",
    "o4-mini",
]

A non-exhaustive list of chat models supported by this component. See https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/responses#model-support for the full list.

init​

python
__init__(
    *,
    api_key: (
        Secret | Callable[[], str] | Callable[[], Awaitable[str]]
    ) = Secret.from_env_var("AZURE_OPENAI_API_KEY", strict=False),
    azure_endpoint: str | None = None,
    azure_deployment: str = "gpt-5-mini",
    streaming_callback: StreamingCallbackT | None = None,
    organization: str | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    timeout: float | None = None,
    max_retries: int | None = None,
    tools: ToolsType | None = None,
    tools_strict: bool = False,
    http_client_kwargs: dict[str, Any] | None = None
) -> None

Initialize the AzureOpenAIResponsesChatGenerator component.

Parameters:

  • api_key (Secret | Callable[[], str] | Callable[[], Awaitable[str]]) – The API key to use for authentication. Can be:
  • A Secret object containing the API key.
  • A Secret object containing the Azure Active Directory token.
  • A function that returns an Azure Active Directory token.
  • azure_endpoint (str | None) – The endpoint of the deployed model, for example "https://example-resource.azure.openai.com/".
  • azure_deployment (str) – The deployment of the model, usually the model name.
  • organization (str | None) – Your organization ID, defaults to None. For help, see Setting up your organization.
  • streaming_callback (StreamingCallbackT | None) – A callback function called when a new token is received from the stream. It accepts StreamingChunk as an argument.
  • timeout (float | None) – Timeout for OpenAI client calls. If not set, it defaults to either the OPENAI_TIMEOUT environment variable, or 30 seconds.
  • max_retries (int | None) – Maximum number of retries to contact OpenAI after an internal error. If not set, it defaults to either the OPENAI_MAX_RETRIES environment variable, or set to 5.
  • generation_kwargs (dict[str, Any] | None) – Other parameters to use for the model. These parameters are sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some of the supported parameters:
  • temperature: What sampling temperature to use. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
  • top_p: An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. For example, 0.1 means only the tokens comprising the top 10% probability mass are considered.
  • previous_response_id: The ID of the previous response. Use this to create multi-turn conversations.
  • text_format: A Pydantic model that enforces the structure of the model's response. If provided, the output will always be validated against this format (unless the model returns a tool call). For details, see the OpenAI Structured Outputs documentation.
  • text: A JSON schema that enforces the structure of the model's response. If provided, the output will always be validated against this format (unless the model returns a tool call). Notes:
    • Both JSON Schema and Pydantic models are supported for latest models starting from GPT-4o.
    • If both are provided, text_format takes precedence and json schema passed to text is ignored.
    • Currently, this component doesn't support streaming for structured outputs.
    • Older models only support basic version of structured outputs through {"type": "json_object"}. For detailed information on JSON mode, see the OpenAI Structured Outputs documentation.
  • reasoning: A dictionary of parameters for reasoning. For example:
    • summary: The summary of the reasoning.
    • effort: The level of effort to put into the reasoning. Can be low, medium or high.
    • generate_summary: Whether to generate a summary of the reasoning. Note: OpenAI does not return the reasoning tokens, but we can view summary if its enabled. For details, see the OpenAI Reasoning documentation.
  • tools (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls.
  • tools_strict (bool) – Whether to enable strict schema adherence for tool calls. If set to True, the model will follow exactly the schema provided in the parameters field of the tool definition, but this may increase latency.
  • http_client_kwargs (dict[str, Any] | None) – A dictionary of keyword arguments to configure a custom httpx.Clientor httpx.AsyncClient. For more information, see the HTTPX documentation.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize this component to a dictionary.

Returns:

  • dict[str, Any] – The serialized component as a dictionary.

from_dict​

python
from_dict(data: dict[str, Any]) -> AzureOpenAIResponsesChatGenerator

Deserialize this component from a dictionary.

Parameters:

  • data (dict[str, Any]) – The dictionary representation of this component.

Returns:

  • AzureOpenAIResponsesChatGenerator – The deserialized component instance.

chat/fallback​

FallbackChatGenerator​

A chat generator wrapper that tries multiple chat generators sequentially.

It forwards all parameters transparently to the underlying chat generators and returns the first successful result. Calls chat generators sequentially until one succeeds. Falls back on any exception raised by a generator. If all chat generators fail, it raises a RuntimeError with details.

Timeout enforcement is fully delegated to the underlying chat generators. The fallback mechanism will only work correctly if the underlying chat generators implement proper timeout handling and raise exceptions when timeouts occur. For predictable latency guarantees, ensure your chat generators:

  • Support a timeout parameter in their initialization
  • Implement timeout as total wall-clock time (shared deadline for both streaming and non-streaming)
  • Raise timeout exceptions (e.g., TimeoutError, asyncio.TimeoutError, httpx.TimeoutException) when exceeded

Note: Most well-implemented chat generators (OpenAI, Anthropic, Cohere, etc.) support timeout parameters with consistent semantics. For HTTP-based LLM providers, a single timeout value (e.g., timeout=30) typically applies to all connection phases: connection setup, read, write, and pool. For streaming responses, read timeout is the maximum gap between chunks. For non-streaming, it's the time limit for receiving the complete response.

Fail over is automatically triggered when a generator raises any exception, including:

  • Timeout errors (if the generator implements and raises them)
  • Rate limit errors (429)
  • Authentication errors (401)
  • Context length errors (400)
  • Server errors (500+)
  • Any other exception

init​

python
__init__(chat_generators: list[ChatGenerator]) -> None

Creates an instance of FallbackChatGenerator.

Parameters:

  • chat_generators (list[ChatGenerator]) – A non-empty list of chat generator components to try in order.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize the component, including nested chat generators.

from_dict​

python
from_dict(data: dict[str, Any]) -> FallbackChatGenerator

Rebuild the component from a serialized representation, restoring nested chat generators.

warm_up​

python
warm_up() -> None

Warm up all underlying chat generators.

warm_up_async​

python
warm_up_async() -> None

Warm up all underlying chat generators on the serving event loop.

close​

python
close() -> None

Release the underlying chat generators' resources.

close_async​

python
close_async() -> None

Release the underlying chat generators' async resources.

run​

python
run(
    messages: list[ChatMessage] | str,
    generation_kwargs: dict[str, Any] | None = None,
    tools: ToolsType | None = None,
    streaming_callback: StreamingCallbackT | None = None,
) -> dict[str, list[ChatMessage] | dict[str, Any]]

Execute chat generators sequentially until one succeeds.

Parameters:

  • messages (list[ChatMessage] | str) – The conversation history as a list of ChatMessage instances.
  • generation_kwargs (dict[str, Any] | None) – Optional parameters for the chat generator (e.g., temperature, max_tokens).
  • tools (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for function calling capabilities.
  • streaming_callback (StreamingCallbackT | None) – Optional callable for handling streaming responses.

Returns:

  • dict[str, list[ChatMessage] | dict[str, Any]] – A dictionary with:
  • "replies": Generated ChatMessage instances from the first successful generator.
  • "meta": Execution metadata including successful_chat_generator_index, successful_chat_generator_class, total_attempts, failed_chat_generators, plus any metadata from the successful generator.

Raises:

  • RuntimeError – If all chat generators fail.

run_async​

python
run_async(
    messages: list[ChatMessage] | str,
    generation_kwargs: dict[str, Any] | None = None,
    tools: ToolsType | None = None,
    streaming_callback: StreamingCallbackT | None = None,
) -> dict[str, list[ChatMessage] | dict[str, Any]]

Asynchronously execute chat generators sequentially until one succeeds.

Parameters:

  • messages (list[ChatMessage] | str) – The conversation history as a list of ChatMessage instances.
  • generation_kwargs (dict[str, Any] | None) – Optional parameters for the chat generator (e.g., temperature, max_tokens).
  • tools (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for function calling capabilities.
  • streaming_callback (StreamingCallbackT | None) – Optional callable for handling streaming responses.

Returns:

  • dict[str, list[ChatMessage] | dict[str, Any]] – A dictionary with:
  • "replies": Generated ChatMessage instances from the first successful generator.
  • "meta": Execution metadata including successful_chat_generator_index, successful_chat_generator_class, total_attempts, failed_chat_generators, plus any metadata from the successful generator.

Raises:

  • RuntimeError – If all chat generators fail.

chat/llm​

LLM​

Bases: Agent

A text generation component powered by a large language model.

The LLM component is a simplified version of the Agent that focuses solely on text generation without tool usage. It processes messages and returns a single response from the language model.

Usage examples​

python
from haystack.components.generators.chat import LLM
from haystack.components.generators.chat import OpenAIChatGenerator

llm = LLM(
    chat_generator=OpenAIChatGenerator(),
    system_prompt="You are a helpful translation assistant.",
    user_prompt="Summarize the following document: {{ document }}",
    required_variables=["document"],
)

result = llm.run(document="The weather is lovely today and the sun is shining. ")
print(result["last_message"].text)

init​

python
__init__(
    *,
    chat_generator: ChatGenerator,
    system_prompt: str | None = None,
    user_prompt: str | None = None,
    required_variables: list[str] | Literal["*"] = "*",
    streaming_callback: StreamingCallbackT | None = None
) -> None

Initialize the LLM component.

Parameters:

  • chat_generator (ChatGenerator) – An instance of the chat generator that the LLM should use.
  • system_prompt (str | None) – System prompt for the LLM. Can be a plain string template or a Jinja2 message template.
  • user_prompt (str | None) – User prompt for the LLM. This prompt is appended to the messages provided at runtime. Can be a plain string template or a Jinja2 message template. If it contains template variables (e.g., {{ variable_name }}), they become inputs to the component. If omitted or if there are no template variables, messages must be provided at runtime instead.
  • required_variables (list[str] | Literal['*']) – Variables that must be provided as input to user_prompt or system_prompt. If a variable listed as required is not provided, an exception is raised. If set to "*", all variables found in the prompt are required. Defaults to "*". Only relevant when user_prompt or system_prompt contains template variables.
  • streaming_callback (StreamingCallbackT | None) – A callback that will be invoked when a response is streamed from the LLM.

Raises:

  • ValueError – If user_prompt contains template variables but required_variables is an empty list.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize the LLM component to a dictionary.

Returns:

  • dict[str, Any] – Dictionary with serialized data.

from_dict​

python
from_dict(data: dict[str, Any]) -> LLM

Deserialize the LLM from a dictionary.

Parameters:

  • data (dict[str, Any]) – Dictionary to deserialize from.

Returns:

  • LLM – Deserialized LLM instance.

run​

python
run(
    *,
    streaming_callback: StreamingCallbackT | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    **kwargs: Any
) -> dict[str, Any]

Process messages and generate a response from the language model.

Parameters:

  • messages – Optional list of ChatMessage objects to prepend to the conversation. Whether this is required or optional depends on the user_prompt configuration: if user_prompt has no template variables, messages must be provided. Passed via **kwargs.
  • streaming_callback (StreamingCallbackT | None) – A callback that will be invoked when a response is streamed from the LLM.
  • generation_kwargs (dict[str, Any] | None) – Additional keyword arguments for the underlying chat generator. These parameters will override the parameters passed during component initialization.
  • kwargs (Any) – Additional keyword arguments. These are used to fill template variables in user_prompt or system_prompt (the keys must match template variable names).

Returns:

  • dict[str, Any] – A dictionary with the following keys:
  • "messages": List of all messages exchanged during the LLM's run.
  • "last_message": The last message exchanged during the LLM's run.
  • "token_usage": Token usage from the LLM call (e.g. prompt_tokens, completion_tokens). Empty if the chat generator did not return usage data.

run_async​

python
run_async(
    *,
    streaming_callback: StreamingCallbackT | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    **kwargs: Any
) -> dict[str, Any]

Asynchronously process messages and generate a response from the language model.

Parameters:

  • messages – Optional list of ChatMessage objects to prepend to the conversation. Whether this is required or optional depends on the user_prompt configuration: if user_prompt has no template variables, messages must be provided. Passed via **kwargs.
  • streaming_callback (StreamingCallbackT | None) – An asynchronous callback that will be invoked when a response is streamed from the LLM.
  • generation_kwargs (dict[str, Any] | None) – Additional keyword arguments for the underlying chat generator. These parameters will override the parameters passed during component initialization.
  • kwargs (Any) – Additional keyword arguments. These are used to fill template variables in user_prompt or system_prompt (the keys must match template variable names).

Returns:

  • dict[str, Any] – A dictionary with the following keys:
  • "messages": List of all messages exchanged during the LLM's run.
  • "last_message": The last message exchanged during the LLM's run.
  • "token_usage": Token usage from the LLM call (e.g. prompt_tokens, completion_tokens). Empty if the chat generator did not return usage data.

chat/mock​

MockChatGenerator​

A Chat Generator that returns predefined responses without calling any API.

It is a drop-in replacement for real Chat Generators (such as OpenAIChatGenerator) in tests, smoke tests, and quick prototypes. It implements the same interface (run, run_async, streaming, serialization) but never contacts an external service, so it is fully deterministic and free to run.

The response is selected based on how the component is configured:

  • Fixed response: pass a single string or ChatMessage. The same reply is returned on every call. Any ChatMessage passed as a response must have the assistant role.
  • Cycling responses: pass a list of strings and/or ChatMessage objects. Each call returns the next item, wrapping around to the start once the list is exhausted. This is useful to drive multi-step flows such as Agents, where the first call returns a tool call and a later call returns the final answer.
  • Dynamic response: pass a response_fn callable that receives the input messages and returns the reply. This is useful when the reply should depend on the input, for example to echo back part of the prompt.
  • Echo (default): with no configuration, the component echoes back the text of the last message that has text content. This makes it usable out of the box for quick prototyping.

Pass ChatMessage objects (rather than plain strings) to return tool calls or reasoning content, which is handy for exercising tool-calling pipelines without a real model.

Usage example​

python
from haystack.components.generators.chat import MockChatGenerator
from haystack.dataclasses import ChatMessage, ToolCall

# Fixed response
generator = MockChatGenerator(responses="Hello, this is a mock response.")
result = generator.run([ChatMessage.from_user("Hi!")])
print(result["replies"][0].text)  # "Hello, this is a mock response."

# Cycling responses to drive an Agent-like loop
generator = MockChatGenerator(
    responses=[
        ChatMessage.from_assistant(tool_calls=[ToolCall(tool_name="search", arguments={"query": "Haystack"})]),
        "Here is the final answer.",
    ]
)

init​

python
__init__(
    responses: str | ChatMessage | Sequence[str | ChatMessage] | None = None,
    *,
    response_fn: ResponseFn | None = None,
    model: str = "mock-model",
    meta: dict[str, Any] | None = None,
    streaming_callback: StreamingCallbackT | None = None
) -> None

Creates an instance of MockChatGenerator.

Parameters:

  • responses (str | ChatMessage | Sequence[str | ChatMessage] | None) – The predefined response(s) to return. Accepts a single string or ChatMessage (returned on every call), or a non-empty list of strings and/or ChatMessage objects that are returned in order, cycling back to the start once exhausted. Strings are wrapped into assistant ChatMessage objects, and any ChatMessage passed must have the assistant role. Mutually exclusive with response_fn. If neither is provided, the component echoes the last message with text content.
  • response_fn (ResponseFn | None) – An optional callable that receives the input messages and returns the reply as a string or an assistant ChatMessage. Use this for input-dependent responses. Mutually exclusive with responses. To support serialization, pass a named function (lambdas and nested functions cannot be serialized).
  • model (str) – The model name reported in the response metadata. Purely cosmetic; no model is loaded.
  • meta (dict[str, Any] | None) – Additional metadata merged into the meta of every returned ChatMessage. A per-response ChatMessage's own metadata takes precedence over this value.
  • streaming_callback (StreamingCallbackT | None) – An optional callback invoked with StreamingChunk objects reconstructed from the predefined response. It lets the mock exercise streaming code paths without a real model.

Raises:

  • ValueError – If both responses and response_fn are provided, if responses is an empty list, or if a ChatMessage response does not have the assistant role.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize the component to a dictionary.

from_dict​

python
from_dict(data: dict[str, Any]) -> MockChatGenerator

Deserialize the component from a dictionary.

warm_up​

python
warm_up() -> None

No-op warm up, provided for interface compatibility with real Chat Generators.

run​

python
run(
    messages: list[ChatMessage] | str,
    streaming_callback: StreamingCallbackT | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    *,
    tools: ToolsType | None = None,
    tools_strict: bool | None = None
) -> dict[str, list[ChatMessage]]

Return a predefined reply for the given messages without calling any API.

The signature mirrors OpenAIChatGenerator.run so the mock can be used as a positional drop-in replacement.

Parameters:

  • messages (list[ChatMessage] | str) – The conversation history as a list of ChatMessage instances or a single string.
  • streaming_callback (StreamingCallbackT | None) – An optional callback invoked with reconstructed StreamingChunk objects. Overrides the callback set at initialization.
  • generation_kwargs (dict[str, Any] | None) – Accepted for interface compatibility and ignored.
  • tools (ToolsType | None) – Accepted for interface compatibility and ignored.
  • tools_strict (bool | None) – Accepted for interface compatibility and ignored.

Returns:

  • dict[str, list[ChatMessage]] – A dictionary with a single key replies containing the predefined reply as a list of one ChatMessage (empty in echo mode when there is no message to echo).

run_async​

python
run_async(
    messages: list[ChatMessage] | str,
    streaming_callback: StreamingCallbackT | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    *,
    tools: ToolsType | None = None,
    tools_strict: bool | None = None
) -> dict[str, list[ChatMessage]]

Asynchronously return a predefined reply for the given messages without calling any API.

The signature mirrors OpenAIChatGenerator.run_async so the mock can be used as a positional drop-in replacement.

Parameters:

  • messages (list[ChatMessage] | str) – The conversation history as a list of ChatMessage instances or a single string.
  • streaming_callback (StreamingCallbackT | None) – An optional callback invoked with reconstructed StreamingChunk objects. Overrides the callback set at initialization.
  • generation_kwargs (dict[str, Any] | None) – Accepted for interface compatibility and ignored.
  • tools (ToolsType | None) – Accepted for interface compatibility and ignored.
  • tools_strict (bool | None) – Accepted for interface compatibility and ignored.

Returns:

  • dict[str, list[ChatMessage]] – A dictionary with a single key replies containing the predefined reply as a list of one ChatMessage (empty in echo mode when there is no message to echo).

chat/openai​

OpenAIChatGenerator​

Completes chats using OpenAI's large language models (LLMs).

It works with the gpt-4 and gpt-5 series models and supports streaming responses from OpenAI API. It uses ChatMessage format in input and output.

You can customize how the text is generated by passing parameters to the OpenAI API. Use the **generation_kwargs argument when you initialize the component or when you run it. Any parameter that works with openai.ChatCompletion.create will work here too.

For details on OpenAI API parameters, see OpenAI documentation.

Usage example​

python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage

messages = [ChatMessage.from_user("What's Natural Language Processing?")]

client = OpenAIChatGenerator()
response = client.run(messages)
print(response)

Output:

{'replies':
    [ChatMessage(_role=<ChatRole.ASSISTANT: 'assistant'>, _content=
    [TextContent(text="Natural Language Processing (NLP) is a branch of artificial intelligence
        that focuses on enabling computers to understand, interpret, and generate human language in
        a way that is meaningful and useful.")],
     _name=None,
     _meta={'model': 'gpt-5-mini', 'index': 0, 'finish_reason': 'stop',
     'usage': {'prompt_tokens': 15, 'completion_tokens': 36, 'total_tokens': 51}})
    ]
}

SUPPORTED_MODELS​

python
SUPPORTED_MODELS: list[str] = [
    "gpt-5-mini",
    "gpt-5-nano",
    "gpt-5",
    "gpt-5.1",
    "gpt-5.2",
    "gpt-5.2-pro",
    "gpt-5.4",
    "gpt-5-pro",
    "gpt-4.1",
    "gpt-4.1-mini",
    "gpt-4.1-nano",
    "gpt-4o",
    "gpt-4o-mini",
    "gpt-4-turbo",
    "gpt-4",
    "gpt-3.5-turbo",
]

A non-exhaustive list of chat models supported by this component. See https://developers.openai.com/api/docs/models for the full list and snapshot IDs.

init​

python
__init__(
    api_key: Secret = Secret.from_env_var("OPENAI_API_KEY"),
    model: str = "gpt-5-mini",
    streaming_callback: StreamingCallbackT | None = None,
    api_base_url: str | None = None,
    organization: str | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    timeout: float | None = None,
    max_retries: int | None = None,
    tools: ToolsType | None = None,
    tools_strict: bool = False,
    http_client_kwargs: dict[str, Any] | None = None,
) -> None

Creates an instance of OpenAIChatGenerator. Unless specified otherwise in model, uses OpenAI's gpt-5-mini

Before initializing the component, you can set the 'OPENAI_TIMEOUT' and 'OPENAI_MAX_RETRIES' environment variables to override the timeout and max_retries parameters respectively in the OpenAI client.

Parameters:

  • api_key (Secret) – The OpenAI API key. You can set it with an environment variable OPENAI_API_KEY, or pass with this parameter during initialization.
  • model (str) – The name of the model to use.
  • streaming_callback (StreamingCallbackT | None) – A callback function that is called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument.
  • api_base_url (str | None) – An optional base URL.
  • organization (str | None) – Your organization ID, defaults to None. See production best practices.
  • generation_kwargs (dict[str, Any] | None) – Other parameters to use for the model. These parameters are sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some of the supported parameters:
  • max_completion_tokens: An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.
  • temperature: What sampling temperature to use. Higher values mean the model will take more risks. Try 0.9 for more creative applications and 0 (argmax sampling) for ones with a well-defined answer.
  • top_p: An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. For example, 0.1 means only the tokens comprising the top 10% probability mass are considered.
  • n: How many completions to generate for each prompt. For example, if the LLM gets 3 prompts and n is 2, it will generate two completions for each of the three prompts, ending up with 6 completions in total.
  • stop: One or more sequences after which the LLM should stop generating tokens.
  • presence_penalty: What penalty to apply if a token is already present at all. Bigger values mean the model will be less likely to repeat the same token in the text.
  • frequency_penalty: What penalty to apply if a token has already been generated in the text. Bigger values mean the model will be less likely to repeat the same token in the text.
  • logit_bias: Add a logit bias to specific tokens. The keys of the dictionary are tokens, and the values are the bias to add to that token.
  • response_format: A JSON schema or a Pydantic model that enforces the structure of the model's response. If provided, the output will always be validated against this format (unless the model returns a tool call). For details, see the OpenAI Structured Outputs documentation. Notes:
    • This parameter accepts Pydantic models and JSON schemas for latest models starting from GPT-4o. Older models only support basic version of structured outputs through {"type": "json_object"}. For detailed information on JSON mode, see the OpenAI Structured Outputs documentation.
    • For structured outputs with streaming, the response_format must be a JSON schema and not a Pydantic model.
  • timeout (float | None) – Timeout for OpenAI client calls. If not set, it defaults to either the OPENAI_TIMEOUT environment variable, or 30 seconds.
  • max_retries (int | None) – Maximum number of retries to contact OpenAI after an internal error. If not set, it defaults to either the OPENAI_MAX_RETRIES environment variable, or set to 5.
  • tools (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls.
  • tools_strict (bool) – Whether to enable strict schema adherence for tool calls. If set to True, the model will follow exactly the schema provided in the parameters field of the tool definition, but this may increase latency.
  • http_client_kwargs (dict[str, Any] | None) – A dictionary of keyword arguments to configure a custom httpx.Clientor httpx.AsyncClient. For more information, see the HTTPX documentation.

warm_up​

python
warm_up() -> None

Warm up the tools and initialize the synchronous OpenAI client.

warm_up_async​

python
warm_up_async() -> None

Warm up the tools and initialize the asynchronous OpenAI client on the serving event loop.

close​

python
close() -> None

Releases the synchronous OpenAI client.

close_async​

python
close_async() -> None

Releases the asynchronous OpenAI client.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize this component to a dictionary.

Returns:

  • dict[str, Any] – The serialized component as a dictionary.

from_dict​

python
from_dict(data: dict[str, Any]) -> OpenAIChatGenerator

Deserialize this component from a dictionary.

Parameters:

  • data (dict[str, Any]) – The dictionary representation of this component.

Returns:

  • OpenAIChatGenerator – The deserialized component instance.

run​

python
run(
    messages: list[ChatMessage] | str,
    streaming_callback: StreamingCallbackT | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    *,
    tools: ToolsType | None = None,
    tools_strict: bool | None = None
) -> dict[str, list[ChatMessage]]

Invokes chat completion based on the provided messages and generation parameters.

Parameters:

  • messages (list[ChatMessage] | str) – A list of ChatMessage instances representing the input messages. If a string is provided, it is converted to a list containing a ChatMessage with user role.
  • streaming_callback (StreamingCallbackT | None) – A callback function that is called when a new token is received from the stream.
  • generation_kwargs (dict[str, Any] | None) – Additional keyword arguments for text generation. These parameters will override the parameters passed during component initialization. For details on OpenAI API parameters, see OpenAI documentation.
  • tools (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls. If set, it will override the tools parameter provided during initialization.
  • tools_strict (bool | None) – Whether to enable strict schema adherence for tool calls. If set to True, the model will follow exactly the schema provided in the parameters field of the tool definition, but this may increase latency. If set, it will override the tools_strict parameter set during component initialization.

Returns:

  • dict[str, list[ChatMessage]] – A dictionary with the following key:
  • replies: A list containing the generated responses as ChatMessage instances.

run_async​

python
run_async(
    messages: list[ChatMessage] | str,
    streaming_callback: StreamingCallbackT | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    *,
    tools: ToolsType | None = None,
    tools_strict: bool | None = None
) -> dict[str, list[ChatMessage]]

Asynchronously invokes chat completion based on the provided messages and generation parameters.

This is the asynchronous version of the run method. It has the same parameters and return values but can be used with await in async code.

Parameters:

  • messages (list[ChatMessage] | str) – A list of ChatMessage instances representing the input messages. If a string is provided, it is converted to a list containing a ChatMessage with user role.
  • streaming_callback (StreamingCallbackT | None) – A callback function that is called when a new token is received from the stream. Async callbacks are preferred; a sync callback is accepted but will run synchronously on the event loop and may block it.
  • generation_kwargs (dict[str, Any] | None) – Additional keyword arguments for text generation. These parameters will override the parameters passed during component initialization. For details on OpenAI API parameters, see OpenAI documentation.
  • tools (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls. If set, it will override the tools parameter provided during initialization.
  • tools_strict (bool | None) – Whether to enable strict schema adherence for tool calls. If set to True, the model will follow exactly the schema provided in the parameters field of the tool definition, but this may increase latency. If set, it will override the tools_strict parameter set during component initialization.

Returns:

  • dict[str, list[ChatMessage]] – A dictionary with the following key:
  • replies: A list containing the generated responses as ChatMessage instances.

chat/openai_responses​

OpenAIResponsesChatGenerator​

Completes chats using OpenAI's Responses API.

It works with the gpt-4 and o-series models and supports streaming responses from OpenAI API. It uses ChatMessage format in input and output.

You can customize how the text is generated by passing parameters to the OpenAI API. Use the **generation_kwargs argument when you initialize the component or when you run it. Any parameter that works with openai.Responses.create will work here too.

For details on OpenAI API parameters, see OpenAI documentation.

Usage example​

python
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.dataclasses import ChatMessage

messages = [ChatMessage.from_user("What's Natural Language Processing?")]

client = OpenAIResponsesChatGenerator(generation_kwargs={"reasoning": {"effort": "low", "summary": "auto"}})
response = client.run(messages)
print(response)

SUPPORTED_MODELS​

python
SUPPORTED_MODELS: list[str] = [
    "gpt-5-mini",
    "gpt-5-nano",
    "gpt-5",
    "gpt-5.1",
    "gpt-5.2",
    "gpt-5.2-pro",
    "gpt-5.4",
    "gpt-5-pro",
    "gpt-4.1",
    "gpt-4.1-mini",
    "gpt-4.1-nano",
    "gpt-4o",
    "gpt-4o-mini",
    "o1",
    "o1-mini",
    "o1-pro",
    "o3",
    "o3-mini",
    "o3-pro",
    "o4-mini",
]

A non-exhaustive list of chat models supported by this component. See https://platform.openai.com/docs/models for the full list and snapshot IDs.

init​

python
__init__(
    *,
    api_key: Secret = Secret.from_env_var("OPENAI_API_KEY"),
    model: str = "gpt-5-mini",
    streaming_callback: StreamingCallbackT | None = None,
    api_base_url: str | None = None,
    organization: str | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    timeout: float | None = None,
    max_retries: int | None = None,
    tools: ToolsType | list[dict] | None = None,
    tools_strict: bool = False,
    http_client_kwargs: dict[str, Any] | None = None
) -> None

Creates an instance of OpenAIResponsesChatGenerator. Uses OpenAI's gpt-5-mini by default.

Before initializing the component, you can set the 'OPENAI_TIMEOUT' and 'OPENAI_MAX_RETRIES' environment variables to override the timeout and max_retries parameters respectively in the OpenAI client.

Parameters:

  • api_key (Secret) – The OpenAI API key. You can set it with an environment variable OPENAI_API_KEY, or pass with this parameter during initialization.
  • model (str) – The name of the model to use.
  • streaming_callback (StreamingCallbackT | None) – A callback function that is called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument.
  • api_base_url (str | None) – An optional base URL.
  • organization (str | None) – Your organization ID, defaults to None. See production best practices.
  • generation_kwargs (dict[str, Any] | None) – Other parameters to use for the model. These parameters are sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some of the supported parameters:
  • temperature: What sampling temperature to use. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
  • top_p: An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. For example, 0.1 means only the tokens comprising the top 10% probability mass are considered.
  • previous_response_id: The ID of the previous response. Use this to create multi-turn conversations.
  • text_format: A Pydantic model that enforces the structure of the model's response. If provided, the output will always be validated against this format (unless the model returns a tool call). For details, see the OpenAI Structured Outputs documentation.
  • text: A JSON schema that enforces the structure of the model's response. If provided, the output will always be validated against this format (unless the model returns a tool call). Notes:
    • Both JSON Schema and Pydantic models are supported for latest models starting from GPT-4o.
    • If both are provided, text_format takes precedence and json schema passed to text is ignored.
    • Currently, this component doesn't support streaming for structured outputs.
    • Older models only support basic version of structured outputs through {"type": "json_object"}. For detailed information on JSON mode, see the OpenAI Structured Outputs documentation.
  • reasoning: A dictionary of parameters for reasoning. For example:
    • summary: The summary of the reasoning.
    • effort: The level of effort to put into the reasoning. Can be low, medium or high.
    • generate_summary: Whether to generate a summary of the reasoning. Note: OpenAI does not return the reasoning tokens, but we can view summary if its enabled. For details, see the OpenAI Reasoning documentation.
  • timeout (float | None) – Timeout for OpenAI client calls. If not set, it defaults to either the OPENAI_TIMEOUT environment variable, or 30 seconds.
  • max_retries (int | None) – Maximum number of retries to contact OpenAI after an internal error. If not set, it defaults to either the OPENAI_MAX_RETRIES environment variable, or set to 5.
  • tools (ToolsType | list[dict] | None) – The tools that the model can use to prepare calls. This parameter can accept either a mixed list of Haystack Tool objects and Haystack Toolset. Or you can pass a dictionary of OpenAI/MCP tool definitions. Note: You cannot pass OpenAI/MCP tools and Haystack tools together. For details on tool support, see OpenAI documentation.
  • tools_strict (bool) – Whether to enable strict schema adherence for tool calls. If set to False, the model may not exactly follow the schema provided in the parameters field of the tool definition. In Response API, tool calls are strict by default.
  • http_client_kwargs (dict[str, Any] | None) – A dictionary of keyword arguments to configure a custom httpx.Clientor httpx.AsyncClient. For more information, see the HTTPX documentation.

warm_up​

python
warm_up() -> None

Warm up the tools and initialize the synchronous OpenAI client.

warm_up_async​

python
warm_up_async() -> None

Warm up the tools and initialize the asynchronous OpenAI client on the serving event loop.

close​

python
close() -> None

Releases the synchronous OpenAI client.

close_async​

python
close_async() -> None

Releases the asynchronous OpenAI client.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize this component to a dictionary.

Returns:

  • dict[str, Any] – The serialized component as a dictionary.

from_dict​

python
from_dict(data: dict[str, Any]) -> OpenAIResponsesChatGenerator

Deserialize this component from a dictionary.

Parameters:

  • data (dict[str, Any]) – The dictionary representation of this component.

Returns:

  • OpenAIResponsesChatGenerator – The deserialized component instance.

run​

python
run(
    messages: list[ChatMessage] | str,
    *,
    streaming_callback: StreamingCallbackT | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    tools: ToolsType | list[dict] | None = None,
    tools_strict: bool | None = None
) -> dict[str, list[ChatMessage]]

Invokes response generation based on the provided messages and generation parameters.

Parameters:

  • messages (list[ChatMessage] | str) – A list of ChatMessage instances representing the input messages.
  • streaming_callback (StreamingCallbackT | None) – A callback function that is called when a new token is received from the stream.
  • generation_kwargs (dict[str, Any] | None) – Additional keyword arguments for text generation. These parameters will override the parameters passed during component initialization. For details on OpenAI API parameters, see OpenAI documentation.
  • tools (ToolsType | list[dict] | None) – The tools that the model can use to prepare calls. If set, it will override the tools parameter set during component initialization. This parameter can accept either a mixed list of Haystack Tool objects and Haystack Toolset. Or you can pass a dictionary of OpenAI/MCP tool definitions. Note: You cannot pass OpenAI/MCP tools and Haystack tools together. For details on tool support, see OpenAI documentation.
  • tools_strict (bool | None) – Whether to enable strict schema adherence for tool calls. If set to False, the model may not exactly follow the schema provided in the parameters field of the tool definition. In Response API, tool calls are strict by default. If set, it will override the tools_strict parameter set during component initialization.

Returns:

  • dict[str, list[ChatMessage]] – A dictionary with the following key:
  • replies: A list containing the generated responses as ChatMessage instances.

run_async​

python
run_async(
    messages: list[ChatMessage] | str,
    *,
    streaming_callback: StreamingCallbackT | None = None,
    generation_kwargs: dict[str, Any] | None = None,
    tools: ToolsType | list[dict] | None = None,
    tools_strict: bool | None = None
) -> dict[str, list[ChatMessage]]

Asynchronously invokes response generation based on the provided messages and generation parameters.

This is the asynchronous version of the run method. It has the same parameters and return values but can be used with await in async code.

Parameters:

  • messages (list[ChatMessage] | str) – A list of ChatMessage instances representing the input messages.
  • streaming_callback (StreamingCallbackT | None) – A callback function that is called when a new token is received from the stream. Async callbacks are preferred; a sync callback is accepted but will run synchronously on the event loop and may block it.
  • generation_kwargs (dict[str, Any] | None) – Additional keyword arguments for text generation. These parameters will override the parameters passed during component initialization. For details on OpenAI API parameters, see OpenAI documentation.
  • tools (ToolsType | list[dict] | None) – A list of tools or a Toolset for which the model can prepare calls. If set, it will override the tools parameter set during component initialization. This parameter can accept either a list of mixed list of Haystack Tool objects and Haystack Toolset. Or you can pass a dictionary of OpenAI/MCP tool definitions. Note: You cannot pass OpenAI/MCP tools and Haystack tools together.
  • tools_strict (bool | None) – Whether to enable strict schema adherence for tool calls. If set to True, the model will follow exactly the schema provided in the parameters field of the tool definition, but this may increase latency. If set, it will override the tools_strict parameter set during component initialization.

Returns:

  • dict[str, list[ChatMessage]] – A dictionary with the following key:
  • replies: A list containing the generated responses as ChatMessage instances.

openai_image_generator​

OpenAIImageGenerator​

Generates images using OpenAI's image generation models such as gpt-image-2.

For details on OpenAI API parameters, see OpenAI documentation.

Usage example​

python
from haystack.components.generators import OpenAIImageGenerator
image_generator = OpenAIImageGenerator()
response = image_generator.run("Show me a picture of a black cat.")
print(response)

init​

python
__init__(
    model: str = "gpt-image-2",
    quality: Literal["auto", "high", "medium", "low"] = "auto",
    size: Literal["1024x1024", "1024x1536", "1536x1024", "auto"] = "1024x1024",
    response_format: Literal["b64_json"] = "b64_json",
    api_key: Secret = Secret.from_env_var("OPENAI_API_KEY"),
    api_base_url: str | None = None,
    organization: str | None = None,
    timeout: float | None = None,
    max_retries: int | None = None,
    http_client_kwargs: dict[str, Any] | None = None,
) -> None

Creates an instance of OpenAIImageGenerator. Unless specified otherwise in model, uses OpenAI's gpt-image-2.

Parameters:

  • model (str) – The model to use for image generation. Model names can be found in the OpenAI documentation.
  • quality (Literal['auto', 'high', 'medium', 'low']) – The quality of the generated image. Can be "auto", "high", "medium", or "low".
  • size (Literal['1024x1024', '1024x1536', '1536x1024', 'auto']) – The size of the generated images. One of 1024x1024, 1024x1536, 1536x1024, or "auto". gpt-image-2 also supports arbitrary sizes. You can find more information about supported sizes in the OpenAI documentation.
  • response_format (Literal['b64_json']) – This parameter is ignored and only kept for backward compatibility.
  • api_key (Secret) – The OpenAI API key to connect to OpenAI.
  • api_base_url (str | None) – An optional base URL.
  • organization (str | None) – The Organization ID, defaults to None.
  • timeout (float | None) – Timeout for OpenAI Client calls. If not set, it is inferred from the OPENAI_TIMEOUT environment variable or set to 30.
  • max_retries (int | None) – Maximum retries to establish contact with OpenAI if it returns an internal error. If not set, it is inferred from the OPENAI_MAX_RETRIES environment variable or set to 5.
  • http_client_kwargs (dict[str, Any] | None) – A dictionary of keyword arguments to configure a custom httpx.Clientor httpx.AsyncClient. For more information, see the HTTPX documentation.

warm_up​

python
warm_up() -> None

Initializes the synchronous OpenAI client.

warm_up_async​

python
warm_up_async() -> None

Initializes the asynchronous OpenAI client on the serving event loop.

close​

python
close() -> None

Releases the synchronous OpenAI client.

close_async​

python
close_async() -> None

Releases the asynchronous OpenAI client.

run​

python
run(
    prompt: str,
    size: Literal["1024x1024", "1024x1536", "1536x1024", "auto"] | None = None,
    quality: Literal["auto", "high", "medium", "low"] | None = None,
    response_format: Literal["b64_json"] | None = None,
) -> dict[str, Any]

Invokes the image generation inference based on the provided prompt and generation parameters.

Parameters:

  • prompt (str) – The prompt to generate the image.
  • size (Literal['1024x1024', '1024x1536', '1536x1024', 'auto'] | None) – If provided, overrides the size provided during initialization.
  • quality (Literal['auto', 'high', 'medium', 'low'] | None) – If provided, overrides the quality provided during initialization.
  • response_format (Literal['b64_json'] | None) – This parameter is ignored and only kept for backward compatibility.

Returns:

  • dict[str, Any] – A dictionary containing the generated list of images as base64 encoded JSON strings and the revised prompt. The revised prompt is the prompt that was used to generate the image, if there was any revision to the prompt made by OpenAI.

run_async​

python
run_async(
    prompt: str,
    size: Literal["1024x1024", "1024x1536", "1536x1024", "auto"] | None = None,
    quality: Literal["auto", "high", "medium", "low"] | None = None,
    response_format: Literal["b64_json"] | None = None,
) -> dict[str, Any]

Asynchronously invokes the image generation inference based on the provided prompt and generation parameters.

This is the asynchronous version of the run method. It has the same parameters and return values but can be used with await in an async code.

Parameters:

  • prompt (str) – The prompt to generate the image.
  • size (Literal['1024x1024', '1024x1536', '1536x1024', 'auto'] | None) – If provided, overrides the size provided during initialization.
  • quality (Literal['auto', 'high', 'medium', 'low'] | None) – If provided, overrides the quality provided during initialization.
  • response_format (Literal['b64_json'] | None) – This parameter is ignored and only kept for backward compatibility.

Returns:

  • dict[str, Any] – A dictionary containing the generated list of images as base64 encoded JSON strings and the revised prompt. The revised prompt is the prompt that was used to generate the image, if there was any revision to the prompt made by OpenAI.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize this component to a dictionary.

Returns:

  • dict[str, Any] – The serialized component as a dictionary.

from_dict​

python
from_dict(data: dict[str, Any]) -> OpenAIImageGenerator

Deserialize this component from a dictionary.

Parameters:

  • data (dict[str, Any]) – The dictionary representation of this component.

Returns:

  • OpenAIImageGenerator – The deserialized component instance.

utils​

python
print_streaming_chunk(chunk: StreamingChunk) -> None

Callback function to handle and display streaming output chunks.

This function processes a StreamingChunk object by:

  • Printing tool call metadata (if any), including function names and arguments, as they arrive.
  • Printing tool call results when available.
  • Printing the main content (e.g., text tokens) of the chunk as it is received.

The function outputs data directly to stdout and flushes output buffers to ensure immediate display during streaming.

Parameters:

  • chunk (StreamingChunk) – A chunk of streaming data containing content and optional metadata, such as tool calls and tool results.