AnthropicGenerator
This component enables text completions using Anthropic large language models (LLMs).
| Name | AnthropicGenerator |
| Source | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic |
| Most common position in a pipeline | After a PromptBuilder |
| Mandatory input variables | “prompt”: A string containing the prompt for the LLM |
| Output variables | “replies”: A list of strings with all the replies generated by the LLM ”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
Overview
This integration supports Anthropic models such as claude-3-5-sonnet-20240620,claude-3-opus-20240229, claude-3-haiku-20240307, and similar. Although these LLMs are called chat models, the main prompt interface works with the string prompts. Check out the most recent full list in the Anthropic documentation.
Parameters
AnthropicGenerator needs an Anthropic API key to work. You can provide this key in:
- The
ANTHROPIC_API_KEYenvironment variable (recommended) - The
api_keyinit parameter and Haystack Secret API:Secret.from_token("your-api-key-here")
Set your preferred Anthropic model in the model parameter when initializing the component.
AnthropicGenerator requires a prompt to generate text, but you can pass any text generation parameters available in the Anthropic Messaging API method directly to this component using the generation_kwargs parameter, both at initialization and to run() method. For more details on the parameters supported by the Anthropic API, see Anthropic documentation.
Finally, the component run method requires a single string prompt to generate text.
Streaming
This Generator supports streaming the tokens from the LLM directly in output. To do so, pass a function to the streaming_callback init parameter.
Usage
Install the anthropic-haystack package to use the AnthropicGenerator:
pip install anthropic-haystack
On its own
from haystack_integrations.components.generators.anthropic import AnthropicGenerator
generator = AnthropicGenerator()
print(generator.run("What's Natural Language Processing? Be brief."))
In a pipeline
You can also use AnthropicGenerator with the Anthropic models in your pipeline.
from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack_integrations.components.generators.anthropic import AnthropicGenerator
from haystack.utils import Secret
template = """
You are an assistant giving out valuable information to language learners.
Answer this question, be brief.
Question: {{ query }}?
"""
pipe = Pipeline()
pipe.add_component("prompt_builder", PromptBuilder(template))
pipe.add_component("llm", AnthropicGenerator(Secret.from_env_var("ANTHROPIC_API_KEY")))
pipe.connect("prompt_builder", "llm")
query = "What language is spoke in Germany?"
res = pipe.run(data={"prompt_builder": {"query": {query}}})
print(res)
Updated 4 months ago