CohereChatGenerator
This component enables chat completion using Cohere's large language models (LLMs).
Name | CohereChatGenerator |
Folder Path | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere |
Most common Position in a Pipeline | After a DynamicChatPromptBuilder |
Mandatory Input variables | “messages” a List of ChatMessage instances |
Output variables | "replies": a list of ChatMessage objects”meta”: a list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
This integration supports Cohere chat
models such as command
and command-light
. Check out the most recent full list in Cohere documentation.
Parameters Overview
CohereChatGenerator
needs a Cohere API key to work. You can write this key in:
- The
api_key
init parameter - The
COHERE_API_KEY
environment variable (recommended)
Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the Co.chat
method directly to this component using the generation_kwargs
parameter, both at initialization and to run()
method. For more details on the parameters supported by the Cohere API, refer to the Cohere documentation.
Finally, the component needs a list of ChatMessage
objects to operate. ChatMessage
is a data class that contains a message, a role (who generated the message, such as user
, assistant
, system
, function
), and optional metadata.
Streaming
CohereChatGenerator
supports streaming the tokens from the LLM directly in output. To do so, pass a function to the streaming_callback
init parameter.
Usage
You need to install cohere-haystack
package to use the CohereChatGenerator
:
pip install cohere-haystack
On its own
from haystack_integrations.components.generators.cohere import CohereChatGenerator
from haystack.dataclasses import ChatMessage
generator = CohereChatGenerator(api_key="<YOUR_COHERE_KEY>")
message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
print(generator.run([message]))
In a Pipeline
You can also use CohereChatGenerator
to use cohere chat models in your pipeline.
from haystack import Pipeline
from haystack.components.builders import DynamicChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.cohere import CohereChatGenerator
from haystack.utils import Secret
pipe = Pipeline()
pipe.add_component("prompt_builder", DynamicChatPromptBuilder())
pipe.add_component("llm", CohereChatGenerator(Secret.from_token(COHERE_API_KEY)))
pipe.connect("prompt_builder", "llm")
country = "Germany"
system_message = ChatMessage.from_system("You are an assistant giving out valuable information to language learners.")
messages = [system_message, ChatMessage.from_user("What's the official language of {{ country }}?")]
res = pipe.run(data={"prompt_builder": {"template_variables": {"country": country}, "prompt_source": messages}})
print(res)
Updated 8 months ago