Overview

This integration supports Anthropic models such as claude-3-5-sonnet-20240620,claude-3-opus-20240229, claude-3-haiku-20240307, and similar. Although these LLMs are called chat models, the main prompt interface works with the string prompts. Check out the most recent full list in the Anthropic documentation.

Parameters

AnthropicGenerator needs an Anthropic API key to work. You can provide this key in:

• The ANTHROPIC_API_KEY environment variable (recommended)
• The api_key init parameter and Haystack Secret API: Secret.from_token("your-api-key-here")

AnthropicGenerator requires a prompt to generate text, but you can pass any text generation parameters available in the Anthropic Messaging API method directly to this component using the generation_kwargs parameter, both at initialization and to run() method. For more details on the parameters supported by the Anthropic API, see Anthropic documentation.

Finally, the component run method requires a single string prompt to generate text.

Streaming

AnthropicGenerator supports streaming the tokens from the LLM directly in output. To do so, pass a function to the streaming_callback init parameter.

Usage

Install the anthropic-haystack package to use the AnthropicGenerator:

pip install anthropic-haystack

On its own

from haystack_integrations.components.generators.anthropic import AnthropicGenerator

generator = AnthropicGenerator()
print(generator.run("What's Natural Language Processing? Be brief."))

In a pipeline

You can also use AnthropicGenerator with the Anthropic models in your pipeline.

from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack_integrations.components.generators.anthropic import AnthropicGenerator
from haystack.utils import Secret

template = """
You are an assistant giving out valuable information to language learners.
Answer this question, be brief.

Question: {{ query }}?
"""

pipe = Pipeline()
pipe.add_component("prompt_builder", PromptBuilder(template))
pipe.add_component("llm", AnthropicGenerator(Secret.from_env_var("ANTHROPIC_API_KEY")))
pipe.connect("prompt_builder", "llm")

query = "What language is spoke in Germany?"
res = pipe.run(data={"prompt_builder": {"query": {query}}})
print(res)