MistralChatGenerator
This component enables chat completion using Mistralβs text generation models.
Name | MistralChatGenerator |
Folder Path | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral |
Most common Position in a Pipeline | After a DynamicChatPromptBuilder |
Mandatory Input variables | βmessagesβ a list of ChatMessage instances |
Output variables | "replies": a list of ChatMessage objectsβmetaβ: a list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
Overview
This integration supports Mistralβs models provided through the generative endpoint. For a full list of available models, check out the Mistral documentation.
MistralChatGenerator
needs a Mistral API key to work. You can write this key in:
- The
api_key
- The
MISTRAL_API_KEY
environment variable (recommended)
Currently, available models are:
mistral-tiny
(default)mistral-small
mistral-medium
This component needs a list of ChatMessage
objects to operate. ChatMessage
is a data class that contains a message, a role (who generated the message, such as user
, assistant
, system
, function
), and optional metadata.
Refer to the Mistral API documentation for more details on the parameters supported by the Mistral API, which you can provide with generation_kwargs
when running the component.
Streaming
MistralChatGenerator
supports streaming the tokens from the LLM directly in output. To do so, pass a function to the streaming_callback
when initializing.
Usage
Install the mistral-haystack
package to use the MistralChatGenerator
:
pip install mistral-haystack
On its own
from haystack_integrations.components.generators.mistral import MistralChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
generator = MistralChatGenerator(api_key=Secret.from_token("<your-api-key>", streaming_callback=print_streaming_chunk)
message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
print(generator.run([message]))
In a Pipeline
Below is an example RAG Pipeline where we answer questions based on the URL contents. We add the contents of the URL into our messages
in the DynamidPromptBuilder
and generate an answer with the MistralChatGenerator
.
from haystack import Document
from haystack import Pipeline
from haystack.components.builders import DynamicChatPromptBuilder
from haystack.components.generators.utils import print_streaming_chunk
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.mistral import MistralChatGenerator
fetcher = LinkContentFetcher()
converter = HTMLToDocument()
prompt_builder = DynamicChatPromptBuilder(runtime_variables=["documents"])
llm = MistralChatGenerator(streaming_callback=print_streaming_chunk, model='mistral-small')
message_template = """Answer the following question based on the contents of the article: {{query}}\n
Article: {{documents[0].content}} \n
"""
messages = [ChatMessage.from_user(message_template)]
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="fetcher", instance=fetcher)
rag_pipeline.add_component(name="converter", instance=converter)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)
rag_pipeline.connect("fetcher.streams", "converter.sources")
rag_pipeline.connect("converter.documents", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
question = "What are the capabilities of Mixtral?"
result = rag_pipeline.run(
{
"fetcher": {"urls": ["https://mistral.ai/news/mixtral-of-experts"]},
"prompt_builder": {"template_variables": {"query": question}, "prompt_source": messages},
"llm": {"generation_kwargs": {"max_tokens": 165}},
},
)
Updated 8 months ago