MariaDBKeywordRetriever

A keyword-based Retriever that fetches documents matching a query from the MariaDB Document Store.

Most common position in a pipeline	1. Before a PromptBuilder in a RAG pipeline 2. The last component in a keyword search pipeline 3. Before a TransformersExtractiveReader in an extractive QA pipeline
Mandatory init variables	document_store: An instance of a MariaDBDocumentStore
Mandatory run variables	query: A string
Output variables	documents: A list of documents matching the query
API reference	MariaDB
GitHub link	https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mariadb

Overview

The MariaDBKeywordRetriever is a keyword-based Retriever compatible with the MariaDBDocumentStore. It uses MariaDB's built-in full-text search to find Documents that match the given query.

In addition to query, the Retriever accepts optional parameters including top_k (the maximum number of Documents to retrieve) and filters to narrow the search space.

Installation

To quickly set up a MariaDB 11.7 instance, you can use Docker:

shell

docker run -d -p 3306:3306 \
  -e MARIADB_ROOT_PASSWORD=secret \
  -e MARIADB_DATABASE=haystack \
  -e MARIADB_USER=haystack \
  -e MARIADB_PASSWORD=secret \
  mariadb:11.7

Install the system library and the integration:

shell

# Ubuntu / Debian
sudo apt-get install -y libmariadb-dev

pip install mariadb-haystack

Usage

On its own

python

import os
from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore
from haystack_integrations.components.retrievers.mariadb import MariaDBKeywordRetriever

os.environ["MARIADB_USER"] = "haystack"
os.environ["MARIADB_PASSWORD"] = "secret"

document_store = MariaDBDocumentStore()
retriever = MariaDBKeywordRetriever(document_store=document_store)

retriever.run(query="my search query")

In a RAG pipeline

python

import os
from haystack import Document, Pipeline
from haystack.components.builders import AnswerBuilder, ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy

from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore
from haystack_integrations.components.retrievers.mariadb import MariaDBKeywordRetriever

os.environ["MARIADB_USER"] = "haystack"
os.environ["MARIADB_PASSWORD"] = "secret"
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"

prompt_template = [
    ChatMessage.from_user(
        """
Given these documents, answer the question.
Documents:
{% for doc in documents %}
    {{ doc.content }}
{% endfor %}

Question: {{question}}

Answer:
"""
    ),
]

document_store = MariaDBDocumentStore()

documents = [
    Document(content="There are over 7,000 languages spoken around the world today."),
    Document(
        content="Elephants have been observed to recognize themselves in mirrors."
    ),
    Document(
        content="Bioluminescent waves can be seen in the Maldives and Puerto Rico."
    ),
]

document_store.write_documents(documents=documents, policy=DuplicatePolicy.SKIP)

retriever = MariaDBKeywordRetriever(document_store=document_store)
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="retriever", instance=retriever)
rag_pipeline.add_component(
    instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
    name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")

question = "languages spoken around the world today"
result = rag_pipeline.run(
    {
        "retriever": {"query": question},
        "prompt_builder": {"question": question},
        "answer_builder": {"query": question},
    }
)
print(result["answer_builder"])