HomeGuidesAPI ReferenceTutorials
Haystack

Retriever

The Retriever performs document retrieval by sweeping through a DocumentStore and returning a set of candidate Documents that are relevant to the query. See what Retrievers are available and how to choose the best one for your use case.

The Retriever takes a query as input and checks it against the Documents contained in the DocumentStore. It scores each Document for its relevance to the query and returns the top candidates.

We have tested Haystack Retrievers on various modalities, including:

  • Text
  • Tables
  • Images

The Retriever is tightly coupled with the DocumentStore. You must specify a DocumentStore when initializing the Retriever.

When used in combination with a Reader, the Retriever can quickly sift out irrelevant Documents, saving the Reader from doing more work than it needs to, and speeding up the querying process.

Position in a PipelineAt the beginning of a querying Pipeline
InputQuery
OutputDocuments
ClassesBM25Retriever
ElasticsearchRetriever
DensePassageRetriever
TableTextRetriever
EmbeddingRetriever
TfidfRetriever
ElasticsearchFilterOnlyRetriever
MultiModalRetriever

If you're unsure which Retriever to use, see the sections below explaining each Retriever type. Our starting recommendations are to use an EmbeddingRetriever if you can use GPU acceleration. If you can't use GPU, we recommend the BM25Retriever.

Usage

To initialize a Retriever, pass a DocumentStore as its argument:

from haystack.nodes import BM25Retriever

retriever = BM25Retriever(document_store)

To run a Retriever on its own, use the retrieve() method. It returns a list of Document objects:

candidate_documents = retriever.retrieve(
    query="international climate conferences",
    top_k=10,
    filters={"year": ["2015", "2016", "2017"]}
)

To run a Retriever within a ready-made pipeline:

from haystack.pipelines import DocumentSearchPipeline

pipeline = DocumentSearchPipeline(retriever=retriever)

result = pipeline.run(
    query="international climate conferences",
    params={
        "Retriever": {
            "top_k": 10,
            "filters": {"year": ["2015", "2016", "2017"]}
        }
    }
)

DocumentStore Compatibility

Note that not all Retrievers can be paired with every DocumentStore. Here are the combinations which are supported:

InMemoryElasticsearchOpenSearchOpenDistroElasticsearchSQLFAISSMilvusWeaviatePineconeDeepsetCloud
BM25NYYYNNNYNY
TF-IDFYYYYYNNYYY
EmbeddingYYYYNYYYYY
MultihopYYYYNYYYYY
DPRYYYYNYYYYY
FilterYYYYYYYYYY
MultiModalYYYYYYYYYY

See Optimization for suggestions on how to choose top-k values.

Table Retrieval

The TableTextRetriever is designed to perform document retrieval on both text and tabular documents. It is a tri-encoder model with a separate encoder for the query, text passage, and table.

To learn more about how to use this component in Haystack, have a look at our Table Question Answering guide.

MultiModal Retrieval

Use the MultiModalRetriever to embed and search for data of different modalities such as text, table, and image. The MultiModalRetriever can handle any modality as long as there is a SentenceTransformers model that supports it. For example, you can perform the following types of search:

  • Text to image
  • Image similarity
  • Text to table
  • Table similarity

To prepare your data for the MultiModalRetriever, cast them into Document objects. For example, to iterate over a directory of images and turn them into a list of Documents, run:

docs = [
    Document(content=f"./examples/images/{filename}", content_type="image")
    for filename in os.listdir("./examples/images")
]

To initialize a MultiModalRetriever that performs text to image retrieval using the [sentence-transformers/clip-ViT-B-32](https://huggingface.co/sentence-transformers/clip-ViT-B-32) model, run:

retriever_mm = MultiModalRetriever(
    document_store=document_store,
    query_embedding_model = "sentence-transformers/clip-ViT-B-32",
    query_type="text",
    document_embedding_models = {
      "image": "sentence-transformers/clip-ViT-B-32"
    }
)
document_store.update_embeddings(retriever_mm)

To initialize a Retriever for text to table retrieval using the [deepset/all-mpnet-base-v2-table](https://huggingface.co/deepset/all-mpnet-base-v2-table) model, run:

retriever_mm = MultiModalRetriever(
    document_store=document_store,
    query_embedding_model = "deepset/all-mpnet-base-v2-table",
    document_embedding_models = {
        "table": "deepset/all-mpnet-base-v2-table"
    }
)
document_store.update_embeddings(retriever_mm)

For more information about the class, see MultiModalRetriever API.

Text Retrieval

BM25 (Recommended)

Use BM25 if you are looking for a retrieval method that doesn't need a neural network for indexing. BM25 is a variant of TF-IDF. It improves upon its predecessor in two main aspects:

  • It saturates tf after a set number of occurrences of the given term in the document
  • It normalises by document length so that short documents are favoured over long documents if they have the same amount of word overlap with the query
from haystack.document_stores import ElasticsearchDocumentStore
from haystack.nodes import BM25Retriever
from haystack.pipelines import ExtractiveQAPipeline

document_store = ElasticsearchDocumentStore()
... retriever = BM25Retriever(document_store)
... p = ExtractiveQAPipeline(reader, retriever)

For more information about the algorithm, see BM25 algorithm.

Embedding Retrieval (Recommended)

In Haystack, you have the option of using a transformer model to encode document and query. Haystack loads models directly from Hugging Face. If you're new to NLP, choosing the right model may be a difficult task. To make it easier, we suggest searching for a model on Hugging Face:

  1. Go to Hugging Face and click Models in the top menu.
  2. From the Tasks on the left, select Sentence Similarity and filter the models by Most Downloads. You get a list of most popular models. It's best to start with one of them.

To use a private model hosted on Hugging Face, enter your Hugging Face access token in the use_auth_token parameter. For more information about models, see Language Models.

One style of model that is suited to this kind of retrieval is Sentence Transformers. These models are trained in Siamese Networks and use triplet loss such that they learn to embed similar sentences near to each other in a shared embedding space.

Some models have been fine-tuned on massive information retrieval datasets and can be used to retrieve documents based on a short query (for example, multi-qa-mpnet-base-dot-v1). There are others that are more suited to semantic similarity tasks where you are trying to find the most similar documents to a given document (for example, all-mpnet-base-v2). There are even models that are multilingual (for example, paraphrase-multilingual-mpnet-base-v2). For a good overview of different models with their evaluation metrics, see Pretrained Models in the Sentence Transformers documentation.

from haystack.document_stores import ElasticsearchDocumentStore
from haystack.nodes import EmbeddingRetriever
from haystack.pipelines import ExtractiveQAPipeline

document_store = ElasticsearchDocumentStore(
    similarity="dot_product",
    embedding_dim=768
)
retriever = EmbeddingRetriever(
    document_store=document_store,
   embedding_model="sentence-transformers/multi-qa-mpnet-base-dot-v1",
   model_format="sentence_transformers"
)
document_store.update_embeddings(retriever)
... p = ExtractiveQAPipeline(reader, retriever)

You can also use OpenAI or Cohere embeddings with the EmbeddingRetriever. You need to provide your API key and specify an embedding_model when initializing the EmbeddingRetriever.

from haystack.nodes import EmbeddingRetriever

# OpenAI EmbeddingRetriever
retriever = EmbeddingRetriever(
   document_store=document_store,
   batch_size=8,
   embedding_model="ada",
   api_key="<your_openai_api_key_goes_here>",
   max_seq_len=1024
)

# Cohere EmbeddingRetriever
retriever = EmbeddingRetriever(
   document_store=document_store,
   batch_size=8,
   embedding_model="small",
   api_key="<your_cohere_api_key_goes_here>",
   max_seq_len=1024
)

When working with OpenAI embeddings, supply one of the following names as the model_name:

NameEmbedding Dimensions
ada (recommended)1024
babbage2048
curie4096
davinci12288

The OpenAI Embeddings API is subject to rate limits. However, we have added a built-in exponential back-off algorithm that saves you from needing to implement any rate-limit handling.

With Cohere, you can set model_name to small, medium or large. See Cohere's Embed endpoint for more information about available models.

Multihop Embedding Retriever

MultihopEmbeddingRetriever is an extension of EmbeddingRetriever that works iteratively. In the first iteration, it retrieves a set of Documents that best match the query. It then concatenates and embeds the query and the top-ranked Documents from the first iteration to provide context for the query. Then, it uses this embedding as input in the second iteration to retrieve Documents that best match the query and its context. You can set the number of iterations you want it to go through. By default, it's set to two iterations but you can experiment with it to find out what works best for your case. This type of retrieval is useful for fact checking, where a chain of evidence pieces leads to the final answer.

MultihopEmbeddingRetriever uses one encoder for the query and the Documents.

For more information, see Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval.

Dense Passage Retrieval

Dense Passage Retrieval is a highly performant retrieval method that calculates relevance using dense representations. Key features:

  • One BERT base model to encode documents
  • One BERT base model to encode queries
  • Ranking of Documents done by dot product similarity between query and document embeddings

Indexing using DPR is comparatively expensive in terms of required computation since all documents in the database need to be processed through the transformer. In order to keep query times low, you should store these embeddings in a vector optimized database such as FAISS or Milvus.

There are two design decisions that have made DPR particularly performant.

  • Separate encoders for document and query helps since queries are much shorter than documents

  • Training with ‘In-batch negatives’ (gold labels are treated as negative examples for other samples in same batch) is highly efficient

In Haystack, you can simply download the pretrained encoders needed to start using DPR. For DPR, you need to provide two models - one for the query and one for the documents, however the models must be trained on the same data. The easiest way to start is to go to Hugging Face and search for dpr. You'll get a list of DPR models sorted by Most Downloads, which means that the models at the top of the list are the most popular ones. Choose a ctx_encoder and a question_encoder model.

To use a private model hosted on Hugging Face, enter your Hugging Face access token in the use_auth_token parameter. For more information about models, see Language Models.

👍

Tip

When using DPR, it is recommended that you use the dot product similarity function since that is how it is trained. To do so, simply provide similarity='dot_product' when initializing the DocumentStore as in the code example below.

from haystack.document_stores import FAISSDocumentStore
from haystack.nodes import DensePassageRetriever
from haystack.pipelines import ExtractiveQAPipeline

document_store = FAISSDocumentStore(similarity="dot_product")
... retriever = DensePassageRetriever(
    document_store=document_store,
    query_embedding_model="facebook/dpr-question_encoder-single-nq-base",
    passage_embedding_model="facebook/dpr-ctx_encoder-single-nq-base"
)
... pipeline = ExtractiveQAPipeline(reader, retriever)

👍

Training DPR

Haystack supports training of your own DPR model! Check out the Training a Dense Passage Retrieval model tutorial to see how this is done!

TF-IDF

TF-IDF is a commonly used baseline for information retrieval that exploits two key intuitions:

  • Documents that have more lexical overlap with the query are more likely to be relevant.
  • Words that occur in fewer documents are more significant than words that occur in many documents.

Given a query, a TF-IDF score is computed for each document as follows:

score = tf * idf

Where:

  • tf is how many times words in the query occur in that document.
  • idf is the inverse of the fraction of documents containing the word.

In practice, both terms are usually log normalised.

from haystack.document_stores import InMemoryDocumentStore
from haystack.nodes import TfidfRetriever
from haystack.pipelines import ExtractiveQAPipeline

document_store = InMemoryDocumentStore()
... retriever = TfidfRetriever(document_store)
... p = ExtractiveQAPipeline(reader, retriever)

Deeper Dive: Dense vs Sparse

Broadly speaking, there are two categories of retrieval methods: dense and sparse.

Sparse methods, like TF-IDF and BM25, operate by looking for shared keywords between the document and the query. These methods:

  • Are simple but effective.
  • Don’t need to be trained.
  • Work on any language.

More recently, dense approaches such as Dense Passage Retrieval (DPR) have shown even better performance than their sparse counter parts. These methods embed both Document and query into a shared embedding space using deep neural networks and the top candidates are the nearest neighbour Documents to the query. They are:

  • Powerful but computationally more expensive, especially during indexing.
  • Trained using labelled datasets.
  • Language specific.

Qualitative Differences

Between these two types, there are also some qualitative differences. For example, sparse methods treat text as a bag-of-words meaning that they do not take word order and syntax into account, while the latest generation of dense methods use transformer based encoders which are designed to be sensitive to these factors.

Also dense methods are very capable of building strong semantic representations of text, but they struggle when encountering out-of-vocabulary words such as new names. By contrast, sparse methods don’t need to learn representations of words, they only care about whether they are present or absent in the text. As such, they handle out-of-vocabulary words with no problem.

Indexing

Dense methods perform indexing by processing all the Documents through a neural network and storing the resulting vectors. This is a much more expensive operation than creating the inverted-index used in sparse methods and requires significant computational power and time.

Terminology

The terms dense and sparse refer to the representations that the algorithms build for each Document and query. Sparse methods characterize texts using vectors with one dimension corresponding to each word in the vocabulary. Dimensions are zero if the word is absent and non-zero if it is present. Since most Documents contain only a small subset of the full vocabulary, these vectors are considered sparse since non-zero values are few and far between.

Dense methods, by contrast, pass text as input into neural network encoders and represent text in a vector of a manually defined size (usually 768). Though individual dimensions are not mapped to any corresponding vocabulary or linguistic feature, each dimension encodes some information about the text. There are rarely 0s in these vectors hence their relative density.


Related Links