DocumentationAPI ReferenceπŸ““ TutorialsπŸ§‘β€πŸ³ Cookbook🀝 IntegrationsπŸ’œ Discord🎨 Studio (Waitlist)
Documentation

MistralChatGenerator

This component enables chat completion using Mistral’s text generation models.

NameMistralChatGenerator
Folder Pathhttps://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral
Most common Position in a PipelineAfter a DynamicChatPromptBuilder
Mandatory Input variablesβ€œmessages” a list of ChatMessage instances
Output variables"replies": a list of ChatMessage objects

”meta”: a list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on

Overview

This integration supports Mistral’s models provided through the generative endpoint. For a full list of available models, check out the Mistral documentation.

MistralChatGenerator needs a Mistral API key to work. You can write this key in:

  • The api_key
  • The MISTRAL_API_KEY environment variable (recommended)

Currently, available models are:

  • mistral-tiny (default)
  • mistral-small
  • mistral-medium

This component needs a list of ChatMessage objects to operate. ChatMessage is a data class that contains a message, a role (who generated the message, such as user, assistant, system, function), and optional metadata.

Refer to the Mistral API documentation for more details on the parameters supported by the Mistral API, which you can provide with generation_kwargs when running the component.

Streaming

MistralChatGenerator supports streaming the tokens from the LLM directly in output. To do so, pass a function to the streaming_callback when initializing.

Usage

Install the mistral-haystack package to use the MistralChatGenerator:

pip install mistral-haystack

On its own

from haystack_integrations.components.generators.mistral import MistralChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage

generator = MistralChatGenerator(api_key=Secret.from_token("<your-api-key>", streaming_callback=print_streaming_chunk)
message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
print(generator.run([message]))

In a Pipeline

Below is an example RAG Pipeline where we answer questions based on the URL contents. We add the contents of the URL into our messages in the DynamidPromptBuilder and generate an answer with the MistralChatGenerator.

from haystack import Document
from haystack import Pipeline
from haystack.components.builders import DynamicChatPromptBuilder
from haystack.components.generators.utils import print_streaming_chunk
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.dataclasses import ChatMessage

from haystack_integrations.components.generators.mistral import MistralChatGenerator

fetcher = LinkContentFetcher()
converter = HTMLToDocument()
prompt_builder = DynamicChatPromptBuilder(runtime_variables=["documents"])
llm = MistralChatGenerator(streaming_callback=print_streaming_chunk, model='mistral-small')

message_template = """Answer the following question based on the contents of the article: {{query}}\n
               Article: {{documents[0].content}} \n 
           """
messages = [ChatMessage.from_user(message_template)]

rag_pipeline = Pipeline()
rag_pipeline.add_component(name="fetcher", instance=fetcher)
rag_pipeline.add_component(name="converter", instance=converter)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)

rag_pipeline.connect("fetcher.streams", "converter.sources")
rag_pipeline.connect("converter.documents", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")

question = "What are the capabilities of Mixtral?"

result = rag_pipeline.run(
    {
        "fetcher": {"urls": ["https://mistral.ai/news/mixtral-of-experts"]},
        "prompt_builder": {"template_variables": {"query": question}, "prompt_source": messages},
      
        "llm": {"generation_kwargs": {"max_tokens": 165}},
    },
)

Related Links

Check out the API reference in the GitHub repo or in our docs: