DocumentationAPI Reference📓 Tutorials🧑‍🍳 Cookbook🤝 Integrations💜 Discord🎨 Studio (Waitlist)
Documentation

MistralChatGenerator

Haystack is an open-source framework for building production-ready LLM applications, retrieval-augmented generative pipelines and state-of-the-art search systems that work intelligently over large document collections. Learn more about Haystack and how it works.

NameMistralChatGenerator
Sourcehttps://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral
Most common position in a pipelineAfter a ChatPromptBuilder
Mandatory input variables“messages” A list of ChatMessage objects
Output variables"replies": A list of ChatMessage objects

”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on

Overview

This integration supports Mistral’s models provided through the generative endpoint. For a full list of available models, check out the Mistral documentation.

MistralChatGenerator needs a Mistral API key to work. You can write this key in:

  • The api_key init parameter using Secret API
  • The MISTRAL_API_KEY environment variable (recommended)

Currently, available models are:

  • mistral-tiny (default)
  • mistral-small
  • mistral-medium(soon to be deprecated)
  • mistral-large-latest
  • codestral-latest
  • open-mistral-nemo

This component needs a list of ChatMessage objects to operate. ChatMessage is a data class that contains a message, a role (who generated the message, such as user, assistant, system, function), and optional metadata.

Refer to the Mistral API documentation for more details on the parameters supported by the Mistral API, which you can provide with generation_kwargs when running the component.

Streaming

MistralChatGenerator supports streaming the tokens from the LLM directly in output. To do so, pass a function to the streaming_callback when initializing.

Usage

Install the mistral-haystack package to use the MistralChatGenerator:

pip install mistral-haystack

On its own

from haystack_integrations.components.generators.mistral import MistralChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret

generator = MistralChatGenerator(api_key=Secret.from_env_var("MISTRAL_API_KEY"), streaming_callback=print_streaming_chunk)
message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
print(generator.run([message]))

In a Pipeline

Below is an example RAG Pipeline where we answer questions based on the URL contents. We add the contents of the URL into our messages in the ChatPromptBuilder and generate an answer with the MistralChatGenerator.

from haystack import Document
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.utils import print_streaming_chunk
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.dataclasses import ChatMessage

from haystack_integrations.components.generators.mistral import MistralChatGenerator

fetcher = LinkContentFetcher()
converter = HTMLToDocument()
prompt_builder = ChatPromptBuilder(variables=["documents"])
llm = MistralChatGenerator(streaming_callback=print_streaming_chunk, model='mistral-small')

message_template = """Answer the following question based on the contents of the article: {{query}}\n
               Article: {{documents[0].content}} \n 
           """
messages = [ChatMessage.from_user(message_template)]

rag_pipeline = Pipeline()
rag_pipeline.add_component(name="fetcher", instance=fetcher)
rag_pipeline.add_component(name="converter", instance=converter)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)

rag_pipeline.connect("fetcher.streams", "converter.sources")
rag_pipeline.connect("converter.documents", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")

question = "What are the capabilities of Mixtral?"

result = rag_pipeline.run(
    {
        "fetcher": {"urls": ["https://mistral.ai/news/mixtral-of-experts"]},
        "prompt_builder": {"template_variables": {"query": question}, "template": messages},
      
        "llm": {"generation_kwargs": {"max_tokens": 512}},
    },
)

Related Links

Check out the API reference in the GitHub repo or in our docs: