Token Counters

approximate_counter

ApproximateTokenCounter

Bases: TokenCounter

Estimates tokens from text length using a flat ratio of characters to tokens.

Usage Example:

python

from haystack.dataclasses import ChatMessage
from haystack.token_counters import ApproximateTokenCounter

counter = ApproximateTokenCounter(chars_per_token=4.0)
messages = [
    ChatMessage.from_user("Hello, how are you?"),
    ChatMessage.from_assistant("I'm good, thank you! How can I assist you today?")
]
token_count = counter.count(messages)
print(f"Estimated token count: {token_count}")

init

python

__init__(
    chars_per_token: float = 4.0,
    tokens_per_image: int = 85,
    tokens_per_file: int = 1000,
) -> None

Initialize the counter.

Parameters:

• chars_per_token (float) – How many characters to treat as one token.
• tokens_per_image (int) – Tokens to charge per image, which has no text to measure. The default is what OpenAI charges for a small image; raise it if you send large ones.
• tokens_per_file (int) – Tokens to charge per file. A rough stand-in for a short document, since the real cost depends on the page count; raise it if you send long ones.

Raises:

• ValueError – If chars_per_token is not positive.

count

python

count(messages: list[ChatMessage], tools: ToolsType | None = None) -> int

Return the estimated number of tokens the given messages occupy.

Parameters:

• messages (list[ChatMessage]) – The messages to measure.
• tools (ToolsType | None) – Tools whose schemas are sent alongside the messages, and so consume tokens too.

Returns:

• int – The estimated token count, or 0 when there is nothing to measure.

to_dict

python

to_dict() -> dict[str, Any]

Serialize the counter.

Returns:

• dict[str, Any] – A dictionary representation of the counter.

tiktoken_counter

TiktokenCounter

Bases: TokenCounter

Counts tokens locally with tiktoken, OpenAI's byte-pair encoder.

Counting is an estimate, and two limits are worth knowing before relying on it:

• It is text-only, so images and files get the flat tokens_per_image / tokens_per_file estimate rather than a real count.
• It is OpenAI's encoder. Other providers tokenize differently, so expect the count to drift on them.

Usage Example:

python

from haystack.dataclasses import ChatMessage
from haystack.token_counters import TiktokenCounter

counter = TiktokenCounter(encoding="o200k_base")
messages = [
    ChatMessage.from_user("Hello, how are you?"),
    ChatMessage.from_assistant("I'm good, thank you! How can I assist you today?")
]
token_count = counter.count(messages)
print(f"Token count: {token_count}")

init

python

__init__(
    encoding: str = "o200k_base",
    tokens_per_image: int = 85,
    tokens_per_file: int = 1000,
) -> None

Initialize the counter.

Parameters:

• encoding (str) – The tiktoken encoding to count with. The default, o200k_base, is what current OpenAI models use.
• tokens_per_image (int) – Tokens to charge per image, which the tokenizer cannot measure. The default is what OpenAI charges for a small image; raise it if you send large ones.
• tokens_per_file (int) – Tokens to charge per file. A rough stand-in for a short document, since the real cost depends on the page count; raise it if you send long ones.

Raises:

• ImportError – If tiktoken is not installed.

warm_up

python

warm_up() -> None

Load the encoder, downloading its vocabulary if it is not already cached.

count

python

count(messages: list[ChatMessage], tools: ToolsType | None = None) -> int

Return the estimated number of tokens used by the given messages.

Parameters:

• messages (list[ChatMessage]) – The messages to measure.
• tools (ToolsType | None) – Tools whose schemas are sent alongside the messages, and so consume tokens too.

Returns:

• int – The estimated token count, or 0 when there is nothing to measure.

to_dict

python

to_dict() -> dict[str, Any]

Serialize the counter.

Returns:

• dict[str, Any] – A dictionary representation of the counter.

types/protocol

TokenCounter

Bases: Protocol

Estimates the number tokens used by a list of messages.

Implement to_dict so the counter's settings survive serialization. The default from_dict passes them straight back to the constructor, which is enough for plain values; override it when to_dict emitted something that has to be rebuilt first, such as a Secret or a nested component.

count

python

count(messages: list[ChatMessage], tools: ToolsType | None = None) -> int

Return the estimated number of tokens in the given messages.

Parameters:

• messages (list[ChatMessage]) – The messages to measure.
• tools (ToolsType | None) – Tools whose schemas are sent alongside the messages, and so consume tokens too. Pass them to have them counted; leave as None to measure the messages alone.

Returns:

• int – The estimated token count.

to_dict

python

to_dict() -> dict[str, Any]

Serialize the counter to a dictionary.

from_dict

python

from_dict(data: dict[str, Any]) -> TokenCounter

Deserialize the counter from a dictionary.