Overview

This integration supports Anthropic chat models such as claude-3-5-sonnet-20240620,claude-3-opus-20240229, claude-3-haiku-20240307, and similar. Check out the most recent full list in Anthropic documentation.

Parameters

AnthropicChatGenerator needs an Anthropic API key to work. You can provide this key in:

• The ANTHROPIC_API_KEY environment variable (recommended)
• The api_key init parameter and Haystack Secret API: Secret.from_token("your-api-key-here")

AnthropicChatGenerator requires a prompt to generate text, but you can pass any text generation parameters available in the Anthropic Messaging API method directly to this component using the generation_kwargs parameter, both at initialization and when running the component. For more details on the parameters supported by the Anthropic API, see the Anthropic documentation.

Finally, the component needs a list of ChatMessage objects to operate. ChatMessage is a data class that contains a message, a role (who generated the message, such as user, assistant, system, function), and optional metadata.

Only text input modality is supported at this time.

Streaming

AnthropicChatGenerator supports streaming the tokens from the LLM directly in output. To do so, pass a function to the streaming_callback init parameter.

Usage

Install theanthropic-haystack package to use the AnthropicChatGenerator:

pip install anthropic-haystack

On its own

from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
from haystack.dataclasses import ChatMessage

generator = AnthropicChatGenerator()
message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
print(generator.run([message]))

In a pipeline

You can also use AnthropicChatGeneratorwith the Anthropic chat models in your pipeline.

from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
from haystack.utils import Secret

pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component("llm", AnthropicChatGenerator(Secret.from_env_var("ANTHROPIC_API_KEY")))
pipe.connect("prompt_builder", "llm")

country = "Germany"
system_message = ChatMessage.from_system("You are an assistant giving out valuable information to language learners.")
messages = [system_message, ChatMessage.from_user("What's the official language of {{ country }}?")]

res = pipe.run(data={"prompt_builder": {"template_variables": {"country": country}, "template": messages}})
print(res)