Skip to main content
Version: 3.1-unstable

Token Counters

approximate_counter​

ApproximateTokenCounter​

Bases: TokenCounter

Estimates tokens from text length using a flat ratio of characters to tokens.

Usage Example:​

python
from haystack.dataclasses import ChatMessage
from haystack.token_counters import ApproximateTokenCounter

counter = ApproximateTokenCounter(chars_per_token=4.0)
messages = [
    ChatMessage.from_user("Hello, how are you?"),
    ChatMessage.from_assistant("I'm good, thank you! How can I assist you today?")
]
token_count = counter.count(messages)
print(f"Estimated token count: {token_count}")

init​

python
__init__(
    chars_per_token: float = 4.0,
    tokens_per_image: int = 85,
    tokens_per_file: int = 1000,
) -> None

Initialize the counter.

Parameters:

  • chars_per_token (float) – How many characters to treat as one token.
  • tokens_per_image (int) – Tokens to charge per image, which has no text to measure. The default is what OpenAI charges for a small image; raise it if you send large ones.
  • tokens_per_file (int) – Tokens to charge per file. A rough stand-in for a short document, since the real cost depends on the page count; raise it if you send long ones.

Raises:

  • ValueError – If chars_per_token is not positive.

count​

python
count(messages: list[ChatMessage], tools: ToolsType | None = None) -> int

Return the estimated number of tokens the given messages occupy.

Parameters:

  • messages (list[ChatMessage]) – The messages to measure.
  • tools (ToolsType | None) – Tools whose schemas are sent alongside the messages, and so consume tokens too.

Returns:

  • int – The estimated token count, or 0 when there is nothing to measure.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize the counter.

Returns:

  • dict[str, Any] – A dictionary representation of the counter.

openai_counter​

OpenAITokenCounter​

Bases: TokenCounter

Counts tokens with OpenAI's input token counting API.

Unlike local token counters, this counter sends the input to OpenAI's POST /v1/responses/input_tokens endpoint. The returned count includes the model-specific formatting used for messages and tool schemas, as well as supported non-text content such as images and files.

Usage Example:​

python
from haystack.dataclasses import ChatMessage
from haystack.token_counters import OpenAITokenCounter

counter = OpenAITokenCounter("gpt-5-mini")
messages = [ChatMessage.from_user("Hello, how are you?")]
token_count = counter.count(messages)
print(f"Token count: {token_count}")

init​

python
__init__(
    model: str,
    *,
    api_key: Secret = Secret.from_env_var("OPENAI_API_KEY"),
    api_base_url: str | None = None,
    organization: str | None = None,
    timeout: float | None = None,
    max_retries: int | None = None,
    http_client_kwargs: dict[str, Any] | None = None
) -> None

Initialize the counter.

Parameters:

  • model (str) – The model whose tokenization should be used.
  • api_key (Secret) – The OpenAI API key. You can set it with the OPENAI_API_KEY environment variable or pass it explicitly.
  • api_base_url (str | None) – An optional base URL for the OpenAI API.
  • organization (str | None) – Your OpenAI organization ID.
  • timeout (float | None) – Timeout for OpenAI client calls. If unset, uses OPENAI_TIMEOUT or 30 seconds.
  • max_retries (int | None) – Maximum retries for OpenAI client calls. If unset, uses OPENAI_MAX_RETRIES or 5.
  • http_client_kwargs (dict[str, Any] | None) – Keyword arguments used to configure the underlying HTTPX client.

warm_up​

python
warm_up() -> None

Initialize the OpenAI client.

count​

python
count(messages: list[ChatMessage], tools: ToolsType | None = None) -> int

Return the exact number of input tokens OpenAI will use for the given messages and tools.

Parameters:

  • messages (list[ChatMessage]) – The messages to measure.
  • tools (ToolsType | None) – Tools whose schemas are sent alongside the messages, and so consume tokens too.

Returns:

  • int – The token count, or 0 when there is nothing to measure.

close​

python
close() -> None

Close the OpenAI client and its underlying HTTP resources.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize the counter.

Returns:

  • dict[str, Any] – A dictionary representation of the counter.

tiktoken_counter​

TiktokenCounter​

Bases: TokenCounter

Counts tokens locally with tiktoken, OpenAI's byte-pair encoder.

Counting is an estimate, and two limits are worth knowing before relying on it:

  • It is text-only, so images and files get the flat tokens_per_image / tokens_per_file estimate rather than a real count.
  • It is OpenAI's encoder. Other providers tokenize differently, so expect the count to drift on them.

Usage Example:​

python
from haystack.dataclasses import ChatMessage
from haystack.token_counters import TiktokenCounter

counter = TiktokenCounter(encoding="o200k_base")
messages = [
    ChatMessage.from_user("Hello, how are you?"),
    ChatMessage.from_assistant("I'm good, thank you! How can I assist you today?")
]
token_count = counter.count(messages)
print(f"Token count: {token_count}")

init​

python
__init__(
    encoding: str = "o200k_base",
    tokens_per_image: int = 85,
    tokens_per_file: int = 1000,
) -> None

Initialize the counter.

Parameters:

  • encoding (str) – The tiktoken encoding to count with. The default, o200k_base, is what current OpenAI models use.
  • tokens_per_image (int) – Tokens to charge per image, which the tokenizer cannot measure. The default is what OpenAI charges for a small image; raise it if you send large ones.
  • tokens_per_file (int) – Tokens to charge per file. A rough stand-in for a short document, since the real cost depends on the page count; raise it if you send long ones.

Raises:

  • ImportError – If tiktoken is not installed.

warm_up​

python
warm_up() -> None

Load the encoder, downloading its vocabulary if it is not already cached.

count​

python
count(messages: list[ChatMessage], tools: ToolsType | None = None) -> int

Return the estimated number of tokens used by the given messages.

Parameters:

  • messages (list[ChatMessage]) – The messages to measure.
  • tools (ToolsType | None) – Tools whose schemas are sent alongside the messages, and so consume tokens too.

Returns:

  • int – The estimated token count, or 0 when there is nothing to measure.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize the counter.

Returns:

  • dict[str, Any] – A dictionary representation of the counter.

types/protocol​

TokenCounter​

Bases: Protocol

Estimates the number tokens used by a list of messages.

Implement to_dict so the counter's settings survive serialization. The default from_dict passes them straight back to the constructor, which is enough for plain values; override it when to_dict emitted something that has to be rebuilt first, such as a Secret or a nested component.

count​

python
count(messages: list[ChatMessage], tools: ToolsType | None = None) -> int

Return the estimated number of tokens in the given messages.

Parameters:

  • messages (list[ChatMessage]) – The messages to measure.
  • tools (ToolsType | None) – Tools whose schemas are sent alongside the messages, and so consume tokens too. Pass them to have them counted; leave as None to measure the messages alone.

Returns:

  • int – The estimated token count.

to_dict​

python
to_dict() -> dict[str, Any]

Serialize the counter to a dictionary.

from_dict​

python
from_dict(data: dict[str, Any]) -> TokenCounter

Deserialize the counter from a dictionary.