HierarchicalDocumentSplitter

Use this component to create a multi-level document structure based on parent-children relationships between text segments.

Most common position in a pipeline	In indexing pipelines after Converters and DocumentCleaner
Mandatory init variables	block_sizes: Set of block sizes to split the document into. The blocks are split in descending order.
Mandatory run variables	documents: A list of documents to split into hierarchical blocks
Output variables	documents: A list of hierarchical documents
API reference	PreProcessors
GitHub link	https://github.com/deepset-ai/haystack/blob/main/haystack/components/preprocessors/hierarchical_document_splitter.py
Package name	haystack-ai

Overview

The HierarchicalDocumentSplitter divides documents into blocks of different sizes, creating a tree-like structure.

A block is one of the chunks of text that the splitter produces. It is similar to cutting a long piece of text into smaller pieces: each piece is a block. Blocks form a tree structure where your full document is the root block, and as you split it into smaller and smaller pieces you get child-blocks and leaf-blocks, down to whatever smallest size specified.

The AutoMergingRetriever component then leverages this hierarchical structure to improve document retrieval.

To initialize the component, you need to specify the block_size, which is the “maximum length” of each of the blocks, measured in the specific unit (see split_by parameter). Pass a set of sizes (for example, {20, 5}), and it will:

• First, split the document into blocks of up to 20 units each (the “parent” blocks).
• Then, it will split each of those into blocks of up to 5 units each (the “child” blocks).

This descending order of sizes builds the hierarchy.

These additional parameters can be set when the component is initialized:

• split_by can be "word" (default), "sentence", "passage", "page", or "token". With "token", block sizes are counted in LLM tokens using tiktoken, so every block fits a known context budget. This requires pip install tiktoken.
• split_overlap is an integer indicating the number of overlapping words, sentences, passages, or tokens between chunks, 0 being the default.
• tokenizer_encoding is the tiktoken encoding used when split_by="token". It defaults to "o200k_base", used by current OpenAI models. Set it to match your model, for example "cl100k_base" for OpenAI's text-embedding-3 embedding models.

Usage

On its own

python

from haystack import Document
from haystack.components.preprocessors import HierarchicalDocumentSplitter

doc = Document(content="This is a simple test document")
splitter = HierarchicalDocumentSplitter(
    block_sizes={3, 2}, split_overlap=0, split_by="word"
)
splitter.run([doc])
# >> {'documents': [Document(id=3f7..., content: 'This is a simple test document', meta: {'__block_size': 0, '__parent_id': None, '__children_ids': ['80a..', 'f0e..'], '__level': 0}),
# >> Document(id=80a.., content: 'This is a ', meta: {'__block_size': 3, '__parent_id': '3f7..', '__children_ids': ['e39..', 'fbf..'], '__level': 1, 'source_id': '3f7..', 'page_number': 1, 'split_id': 0, 'split_idx_start': 0}),
# >> Document(id=f0e.., content: 'simple test document', meta: {'__block_size': 3, '__parent_id': '3f7..', '__children_ids': ['5d1..', '181..'], '__level': 1, 'source_id': '3f7..', 'page_number': 1, 'split_id': 1, 'split_idx_start': 10}),
# >> Document(id=e39.., content: 'This is ', meta: {'__block_size': 2, '__parent_id': '80a..', '__children_ids': [], '__level': 2, 'source_id': '80a..', 'page_number': 1, 'split_id': 0, 'split_idx_start': 0}),
# >> Document(id=fbf.., content: 'a ', meta: {'__block_size': 2, '__parent_id': '80a..', '__children_ids': [], '__level': 2, 'source_id': '80a..', 'page_number': 1, 'split_id': 1, 'split_idx_start': 8}),
# >> Document(id=5d1.., content: 'simple test ', meta: {'__block_size': 2, '__parent_id': 'f0e..', '__children_ids': [], '__level': 2, 'source_id': 'f0e..', 'page_number': 1, 'split_id': 0, 'split_idx_start': 10}),
# >> Document(id=181.., content: 'document', meta: {'__block_size': 2, '__parent_id': 'f0e..', '__children_ids': [], '__level': 2, 'source_id': 'f0e..', 'page_number': 1, 'split_id': 1, 'split_idx_start': 22})]}

Splitting by tokens

With split_by="token", block sizes are counted in LLM tokens, so you can size parent blocks to fit your model's context. This requires pip install tiktoken.

python

from haystack import Document
from haystack.components.preprocessors import HierarchicalDocumentSplitter

doc = Document(
    content="Haystack is an open-source framework for building LLM applications."
)
splitter = HierarchicalDocumentSplitter(block_sizes={8, 4}, split_by="token")
result = splitter.run([doc])

for d in result["documents"]:
    print(d.meta["__level"], repr(d.content))
# >> 0 'Haystack is an open-source framework for building LLM applications.'
# >> 1 'Haystack is an open-source framework for'
# >> 1 ' building LLM applications.'
# >> 2 'Haystack is an'
# >> 2 ' open-source framework for'
# >> 2 ' building LLM applications'
# >> 2 '.'

In practice, use larger block sizes, such as block_sizes={512, 128}.

In a pipeline

This Haystack pipeline processes .md files by converting them to documents, cleaning the text, splitting it into sentence-based chunks, and storing the results in an In-Memory Document Store.

python

from pathlib import Path

from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.converters.txt import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner
from haystack.components.preprocessors import HierarchicalDocumentSplitter
from haystack.components.writers import DocumentWriter

document_store = InMemoryDocumentStore()

pipeline = Pipeline()
pipeline.add_component(instance=TextFileToDocument(), name="text_file_converter")
pipeline.add_component(instance=DocumentCleaner(), name="cleaner")
pipeline.add_component(
    instance=HierarchicalDocumentSplitter(
        block_sizes={10, 6, 3}, split_overlap=0, split_by="sentence"
    ),
    name="splitter",
)
pipeline.add_component(
    instance=DocumentWriter(document_store=document_store), name="writer"
)
pipeline.connect("text_file_converter.documents", "cleaner.documents")
pipeline.connect("cleaner.documents", "splitter.documents")
pipeline.connect("splitter.documents", "writer.documents")

path = "path/to/your/files"
files = list(Path(path).glob("*.md"))
pipeline.run({"text_file_converter": {"sources": files}})