Generators
Generators are responsible for generating text after you give them a prompt. They are specific for each LLM technology (OpenAI, local, TGI and others).
| Generator | Description | Streaming Support |
|---|---|---|
| AmazonBedrockChatGenerator | Enables chat completion using models through Amazon Bedrock service. | ✅ |
| AIMLAPIChatGenerator | Enables chat completion using AI models through the AIMLAPI. | ✅ |
| AnthropicChatGenerator | This component enables chat completions using Anthropic large language models (LLMs). | ✅ |
| AnthropicFoundryChatGenerator | This component enables chat completions using Anthropic models served through Azure Foundry. | ✅ |
| AnthropicVertexChatGenerator | This component enables chat completions using AnthropicVertex API. | ✅ |
| AzureOpenAIChatGenerator | Enables chat completion using OpenAI's LLMs through Azure services. | ✅ |
| AzureOpenAIGenerator | Enables text generation using OpenAI's LLMs through Azure services. | ✅ |
| AzureOpenAIResponsesChatGenerator | Enables chat completion using OpenAI's Responses API through Azure services with support for reasoning models. | ✅ |
| CohereChatGenerator | Enables chat completion using Cohere's LLMs. | ✅ |
| CometAPIChatGenerator | Enables chat completion using AI models through the Comet API. | ✅ |
| DALLEImageGenerator | Generate images using OpenAI's image generation models such as gpt-image-2. | ❌ |
| FallbackChatGenerator | A ChatGenerator wrapper that tries multiple Chat Generators sequentially until one succeeds. | ✅ |
| GoogleAIGeminiChatGenerator | Enables chat completion using Google Gemini models. This integration will be deprecated soon. We recommend using GoogleGenAIChatGenerator integration instead. | ✅ |
| GoogleAIGeminiGenerator | Enables text generation using Google Gemini models. This integration will be deprecated soon. We recommend using GoogleGenAIChatGenerator integration instead. | ✅ |
| GoogleGenAIChatGenerator | Enables chat completion using Google Gemini models through Google Gen AI SDK. | ✅ |
| HuggingFaceAPIChatGenerator | Enables chat completion using various Hugging Face APIs. | ✅ |
| HuggingFaceAPIGenerator | Enables text generation using various Hugging Face APIs. | ✅ |
| HuggingFaceLocalChatGenerator | Provides an interface for chat completion using a Hugging Face model that runs locally. | ✅ |
| HuggingFaceLocalGenerator | Provides an interface to generate text using a Hugging Face model that runs locally. | ✅ |
| LiteLLMChatGenerator | Enables chat completion using any of 100+ LLM providers through LiteLLM. | ✅ |
| LlamaCppChatGenerator | Enables chat completion using an LLM running on Llama.cpp. | ❌ |
| LlamaStackChatGenerator | Enables chat completions using an LLM model made available via Llama Stack server | ✅ |
| MetaLlamaChatGenerator | Enables chat completion with any model hosted available with Meta Llama API. | ✅ |
| MistralChatGenerator | Enables chat completion using Mistral's text generation models. | ✅ |
| NvidiaChatGenerator | Enables chat completion using Nvidia-hosted models. | ✅ |
| OllamaChatGenerator | Enables chat completion using an LLM running on Ollama. | ✅ |
| OpenAIChatGenerator | Enables chat completion using OpenAI's large language models (LLMs). | ✅ |
| OpenAIGenerator | Enables text generation using OpenAI's large language models (LLMs). | ✅ |
| OpenAIResponsesChatGenerator | Enables chat completion using OpenAI's Responses API with support for reasoning models. | ✅ |
| OpenRouterChatGenerator | Enables chat completion with any model hosted on OpenRouter. | ✅ |
| PerplexityChatGenerator | Enables chat completion using models via the Perplexity Agent API. | ✅ |
| SagemakerGenerator | Enables text generation using LLMs deployed on Amazon Sagemaker. | ❌ |
| STACKITChatGenerator | Enables chat completions using the STACKIT API. | ✅ |
| TogetherAIChatGenerator | Enables chat completion using models hosted on Together AI. | ✅ |
| VertexAICodeGenerator | Enables code generation using Google Vertex AI generative model. | ❌ |
| VertexAIGeminiChatGenerator | Enables chat completion using Google Gemini models with GCP Vertex AI. This integration will be deprecated soon. We recommend using GoogleGenAIChatGenerator integration instead. | ✅ |
| VertexAIGeminiGenerator | Enables text generation using Google Gemini models with GCP Vertex AI. This integration will be deprecated soon. We recommend using GoogleGenAIChatGenerator integration instead. | ✅ |
| VertexAIImageCaptioner | Enables text generation using Google Vertex AI imagetext generative model. | ❌ |
| VertexAIImageGenerator | Enables image generation using Google Vertex AI generative model. | ❌ |
| VertexAIImageQA | Enables text generation (image captioning) using Google Vertex AI generative models. | ❌ |
| VertexAITextGenerator | Enables text generation using Google Vertex AI generative models. | ❌ |
| VLLMChatGenerator | Enables chat completion using models served with vLLM. | ✅ |
| WatsonxChatGenerator | Enables chat completions with IBM Watsonx models. | ✅ |