Module haystack_integrations.components.generators.anthropic.generator

AnthropicGenerator

@component
class AnthropicGenerator()

Enables text generation using Anthropic large language models (LLMs). It supports the Claude family of models.

Although Anthropic natively supports a much richer messaging API, we have intentionally simplified it in this component so that the main input/output interface is string-based. For more complete support, consider using the AnthropicChatGenerator.

from haystack_integrations.components.generators.anthropic import AnthropicGenerator

client = AnthropicGenerator(model="claude-3-sonnet-20240229")
response = client.run("What's Natural Language Processing? Be brief.")
print(response)
>>{'replies': ['Natural language processing (NLP) is a branch of artificial intelligence focused on enabling
>>computers to understand, interpret, and manipulate human language. The goal of NLP is to read, decipher,
>> understand, and make sense of the human languages in a manner that is valuable.'], 'meta': {'model':
>> 'claude-2.1', 'index': 0, 'finish_reason': 'end_turn', 'usage': {'input_tokens': 18, 'output_tokens': 58}}}

AnthropicGenerator.__init__

def __init__(api_key: Secret = Secret.from_env_var("ANTHROPIC_API_KEY"),
             model: str = "claude-3-sonnet-20240229",
             streaming_callback: Optional[Callable[[StreamingChunk],
                                                   None]] = None,
             system_prompt: Optional[str] = None,
             generation_kwargs: Optional[Dict[str, Any]] = None)

Initialize the AnthropicGenerator.

Arguments:

• api_key: The Anthropic API key.
• model: The name of the Anthropic model to use.
• streaming_callback: An optional callback function to handle streaming chunks.
• system_prompt: An optional system prompt to use for generation.
• generation_kwargs: Additional keyword arguments for generation.

AnthropicGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serialize this component to a dictionary.

Returns:

The serialized component as a dictionary.

AnthropicGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "AnthropicGenerator"

Deserialize this component from a dictionary.

Arguments:

• data: The dictionary representation of this component.

Returns:

The deserialized component instance.

AnthropicGenerator.run

@component.output_types(replies=List[str], meta=List[Dict[str, Any]])
def run(prompt: str, generation_kwargs: Optional[Dict[str, Any]] = None)

Generate replies using the Anthropic API.

Arguments:

• prompt: The input prompt for generation.
• generation_kwargs: Additional keyword arguments for generation.

Returns:

A dictionary containing:

• replies: A list of generated replies.
• meta: A list of metadata dictionaries for each reply.

Module haystack_integrations.components.generators.anthropic.chat.chat_generator

AnthropicChatGenerator

@component
class AnthropicChatGenerator()

Completes chats using Anthropic's large language models (LLMs).

It uses ChatMessage format in input and output.

You can customize how the text is generated by passing parameters to the Anthropic API. Use the **generation_kwargs argument when you initialize the component or when you run it. Any parameter that works with anthropic.Message.create will work here too.

For details on Anthropic API parameters, see Anthropic documentation.

Usage example:

from haystack_experimental.components.generators.anthropic import AnthropicChatGenerator
from haystack_experimental.dataclasses import ChatMessage

generator = AnthropicChatGenerator(model="claude-3-5-sonnet-20240620",
                                   generation_kwargs={
                                       "max_tokens": 1000,
                                       "temperature": 0.7,
                                   })

messages = [ChatMessage.from_system("You are a helpful, respectful and honest assistant"),
            ChatMessage.from_user("What's Natural Language Processing?")]
print(generator.run(messages=messages))

<a id="haystack_integrations.components.generators.anthropic.chat.chat_generator.AnthropicChatGenerator.__init__"></a>

#### AnthropicChatGenerator.\_\_init\_\_

```python
def __init__(api_key: Secret = Secret.from_env_var("ANTHROPIC_API_KEY"),
             model: str = "claude-3-5-sonnet-20240620",
             streaming_callback: Optional[Callable[[StreamingChunk],
                                                   None]] = None,
             generation_kwargs: Optional[Dict[str, Any]] = None,
             ignore_tools_thinking_messages: bool = True,
             tools: Optional[List[Tool]] = None)

Creates an instance of AnthropicChatGenerator.

Arguments:

• api_key: The Anthropic API key
• model: The name of the model to use.
• streaming_callback: A callback function that is called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument.
• generation_kwargs: Other parameters to use for the model. These parameters are all sent directly to the Anthropic endpoint. See Anthropic documentation for more details.

Supported generation_kwargs parameters are:

• system: The system message to be passed to the model.
• max_tokens: The maximum number of tokens to generate.
• metadata: A dictionary of metadata to be passed to the model.
• stop_sequences: A list of strings that the model should stop generating at.
• temperature: The temperature to use for sampling.
• top_p: The top_p value to use for nucleus sampling.
• top_k: The top_k value to use for top-k sampling.
• extra_headers: A dictionary of extra headers to be passed to the model (i.e. for beta features).
• ignore_tools_thinking_messages: Anthropic's approach to tools (function calling) resolution involves a "chain of thought" messages before returning the actual function names and parameters in a message. If ignore_tools_thinking_messages is True, the generator will drop so-called thinking messages when tool use is detected. See the Anthropic tools for more details.
• tools: A list of Tool objects that the model can use. Each tool should have a unique name.

AnthropicChatGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serialize this component to a dictionary.

Returns:

The serialized component as a dictionary.

AnthropicChatGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "AnthropicChatGenerator"

Deserialize this component from a dictionary.

Arguments:

• data: The dictionary representation of this component.

Returns:

The deserialized component instance.

AnthropicChatGenerator.run

@component.output_types(replies=List[ChatMessage])
def run(messages: List[ChatMessage],
        streaming_callback: Optional[Callable[[StreamingChunk], None]] = None,
        generation_kwargs: Optional[Dict[str, Any]] = None,
        tools: Optional[List[Tool]] = None)

Invokes the Anthropic API with the given messages and generation kwargs.

Arguments:

• messages: A list of ChatMessage instances representing the input messages.
• streaming_callback: A callback function that is called when a new token is received from the stream.
• generation_kwargs: Optional arguments to pass to the Anthropic generation endpoint.
• tools: A list of tools for which the model can prepare calls. If set, it will override the tools parameter set during component initialization.

Returns:

A dictionary with the following keys:

• replies: The responses from the model

Module haystack_integrations.components.generators.anthropic.chat.vertex_chat_generator

AnthropicVertexChatGenerator

@component
class AnthropicVertexChatGenerator(AnthropicChatGenerator)

Enables text generation using state-of-the-art Claude 3 LLMs via the Anthropic Vertex AI API. It supports models such as Claude 3.5 Sonnet, Claude 3 Opus, Claude 3 Sonnet, and Claude 3 Haiku, accessible through the Vertex AI API endpoint.

To use AnthropicVertexChatGenerator, you must have a GCP project with Vertex AI enabled. Additionally, ensure that the desired Anthropic model is activated in the Vertex AI Model Garden. Before making requests, you may need to authenticate with GCP using gcloud auth login. For more details, refer to the [guide] (https://docs.anthropic.com/en/api/claude-on-vertex-ai).

Any valid text generation parameters for the Anthropic messaging API can be passed to the AnthropicVertex API. Users can provide these parameters directly to the component via the generation_kwargs parameter in __init__ or the run method.

For more details on the parameters supported by the Anthropic API, refer to the Anthropic Message API documentation.

from haystack_integrations.components.generators.anthropic import AnthropicVertexChatGenerator
from haystack.dataclasses import ChatMessage

messages = [ChatMessage.from_user("What's Natural Language Processing?")]
client = AnthropicVertexChatGenerator(
            model="claude-3-sonnet@20240229",
            project_id="your-project-id", region="your-region"
        )
response = client.run(messages)
print(response)

>> {'replies': [ChatMessage(content='Natural Language Processing (NLP) is a field of artificial intelligence that
>> focuses on enabling computers to understand, interpret, and generate human language. It involves developing
>> techniques and algorithms to analyze and process text or speech data, allowing machines to comprehend and
>> communicate in natural languages like English, Spanish, or Chinese.', role=<ChatRole.ASSISTANT: 'assistant'>,
>> name=None, meta={'model': 'claude-3-sonnet@20240229', 'index': 0, 'finish_reason': 'end_turn',
>> 'usage': {'input_tokens': 15, 'output_tokens': 64}})]}

For more details on supported models and their capabilities, refer to the Anthropic documentation.

AnthropicVertexChatGenerator.__init__

def __init__(region: str,
             project_id: str,
             model: str = "claude-3-5-sonnet@20240620",
             streaming_callback: Optional[Callable[[StreamingChunk],
                                                   None]] = None,
             generation_kwargs: Optional[Dict[str, Any]] = None,
             ignore_tools_thinking_messages: bool = True,
             tools: Optional[List[Tool]] = None)

Creates an instance of AnthropicVertexChatGenerator.

Arguments:

• region: The region where the Anthropic model is deployed. Defaults to "us-central1".
• project_id: The GCP project ID where the Anthropic model is deployed.
• model: The name of the model to use.
• streaming_callback: A callback function that is called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument.
• generation_kwargs: Other parameters to use for the model. These parameters are all sent directly to the AnthropicVertex endpoint. See Anthropic documentation for more details.

Supported generation_kwargs parameters are:

• system: The system message to be passed to the model.
• max_tokens: The maximum number of tokens to generate.
• metadata: A dictionary of metadata to be passed to the model.
• stop_sequences: A list of strings that the model should stop generating at.
• temperature: The temperature to use for sampling.
• top_p: The top_p value to use for nucleus sampling.
• top_k: The top_k value to use for top-k sampling.
• extra_headers: A dictionary of extra headers to be passed to the model (i.e. for beta features).
• ignore_tools_thinking_messages: Anthropic's approach to tools (function calling) resolution involves a "chain of thought" messages before returning the actual function names and parameters in a message. If ignore_tools_thinking_messages is True, the generator will drop so-called thinking messages when tool use is detected. See the Anthropic tools for more details.
• tools: A list of Tool objects that the model can use. Each tool should have a unique name.

AnthropicVertexChatGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serialize this component to a dictionary.

Returns:

The serialized component as a dictionary.

AnthropicVertexChatGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "AnthropicVertexChatGenerator"

Deserialize this component from a dictionary.

Arguments:

• data: The dictionary representation of this component.

Returns:

The deserialized component instance.