Module haystack_integrations.components.generators.amazon_bedrock.generator

AmazonBedrockGenerator

AmazonBedrockGenerator enables text generation via Amazon Bedrock hosted LLMs.

For example, to use the Anthropic Claude model, simply initialize the AmazonBedrockGenerator with the 'anthropic.claude-v2' model name. Provide AWS credentials either via local AWS profile or directly via aws_access_key_id, aws_secret_access_key, aws_session_token, and aws_region_name parameters.

Usage example:

from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockGenerator

generator = AmazonBedrockGenerator(
        model="anthropic.claude-v2",
        max_length=99
)

print(generator.run("Who is the best American actor?"))

AmazonBedrockGenerator.init

def __init__(
        model: str,
        aws_access_key_id: Optional[Secret] = Secret.from_env_var(
            "AWS_ACCESS_KEY_ID", strict=False),
        aws_secret_access_key: Optional[Secret] = Secret.
    from_env_var(  # noqa: B008
        "AWS_SECRET_ACCESS_KEY", strict=False),
        aws_session_token: Optional[Secret] = Secret.from_env_var(
            "AWS_SESSION_TOKEN", strict=False),
        aws_region_name: Optional[Secret] = Secret.from_env_var(
            "AWS_DEFAULT_REGION", strict=False),
        aws_profile_name: Optional[Secret] = Secret.from_env_var("AWS_PROFILE",
                                                                 strict=False),
        max_length: Optional[int] = 100,
        **kwargs)

Create a new AmazonBedrockGenerator instance.

Arguments:

model: The name of the model to use.
aws_access_key_id: The AWS access key ID.
aws_secret_access_key: The AWS secret access key.
aws_session_token: The AWS session token.
aws_region_name: The AWS region name.
aws_profile_name: The AWS profile name.
max_length: The maximum length of the generated text.
kwargs: Additional keyword arguments to be passed to the model.

Raises:

ValueError: If the model name is empty or None.
AmazonBedrockConfigurationError: If the AWS environment is not configured correctly or the model is not supported.

AmazonBedrockGenerator.invoke

def invoke(*args, **kwargs)

Invokes the model with the given prompt.

Arguments:

args: Additional positional arguments passed to the generator.
kwargs: Additional keyword arguments passed to the generator.

Returns:

A list of generated responses (strings).

AmazonBedrockGenerator.run

@component.output_types(replies=List[str])
def run(prompt: str, generation_kwargs: Optional[Dict[str, Any]] = None)

Generates a list of string response to the given prompt.

Arguments:

prompt: The prompt to generate a response for.
generation_kwargs: Additional keyword arguments passed to the generator.

Raises:

ValueError: If the prompt is empty or None.
AmazonBedrockInferenceError: If the model cannot be invoked.

Returns:

A dictionary with the following keys:

replies: A list of generated responses.

AmazonBedrockGenerator.get_model_adapter

@classmethod
def get_model_adapter(cls, model: str) -> Optional[Type[BedrockModelAdapter]]

Gets the model adapter for the given model.

Arguments:

model: The model name.

Returns:

The model adapter class, or None if no adapter is found.

AmazonBedrockGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

AmazonBedrockGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "AmazonBedrockGenerator"

Deserializes the component from a dictionary.

Arguments:

data: Dictionary to deserialize from.

Returns:

Deserialized component.

Module haystack_integrations.components.generators.amazon_bedrock.adapters

BedrockModelAdapter

Base class for Amazon Bedrock model adapters.

Each subclass of this class is designed to address the unique specificities of a particular LLM it adapts, focusing on preparing the requests and extracting the responses from the Amazon Bedrock hosted LLMs.

BedrockModelAdapter.prepare_body

@abstractmethod
def prepare_body(prompt: str, **inference_kwargs) -> Dict[str, Any]

Prepares the body for the Amazon Bedrock request.

Each subclass should implement this method to prepare the request body for the specific model.

Arguments:

prompt: The prompt to be sent to the model.
inference_kwargs: Additional keyword arguments passed to the handler.

Returns:

A dictionary containing the body for the request.

BedrockModelAdapter.get_responses

def get_responses(response_body: Dict[str, Any]) -> List[str]

Extracts the responses from the Amazon Bedrock response.

Arguments:

response_body: The response body from the Amazon Bedrock request.

Returns:

A list of responses.

BedrockModelAdapter.get_stream_responses

def get_stream_responses(stream,
                         stream_handler: TokenStreamingHandler) -> List[str]

Extracts the responses from the Amazon Bedrock streaming response.

Arguments:

stream: The streaming response from the Amazon Bedrock request.
stream_handler: The handler for the streaming response.

Returns:

A list of string responses.

AnthropicClaudeAdapter

Adapter for the Anthropic Claude models.

AnthropicClaudeAdapter.prepare_body

def prepare_body(prompt: str, **inference_kwargs) -> Dict[str, Any]

Prepares the body for the Claude model

Arguments:

prompt: The prompt to be sent to the model.
inference_kwargs: Additional keyword arguments passed to the handler.

Returns:

A dictionary with the following keys:

prompt: The prompt to be sent to the model.
specified inference parameters.

MistralAdapter

Adapter for the Mistral models.

MistralAdapter.prepare_body

def prepare_body(prompt: str, **inference_kwargs) -> Dict[str, Any]

Prepares the body for the Mistral model

Arguments:

prompt: The prompt to be sent to the model.
inference_kwargs: Additional keyword arguments passed to the handler.

Returns:

A dictionary with the following keys:

prompt: The prompt to be sent to the model.
specified inference parameters.

CohereCommandAdapter

Adapter for the Cohere Command model.

CohereCommandAdapter.prepare_body

def prepare_body(prompt: str, **inference_kwargs) -> Dict[str, Any]

Prepares the body for the Command model

Arguments:

prompt: The prompt to be sent to the model.
inference_kwargs: Additional keyword arguments passed to the handler.

Returns:

A dictionary with the following keys:

prompt: The prompt to be sent to the model.
specified inference parameters.

AI21LabsJurassic2Adapter

Model adapter for AI21 Labs' Jurassic 2 models.

AI21LabsJurassic2Adapter.prepare_body

def prepare_body(prompt: str, **inference_kwargs) -> Dict[str, Any]

Prepares the body for the Jurassic 2 model.

Arguments:

prompt: The prompt to be sent to the model.
inference_kwargs: Additional keyword arguments passed to the handler.

Returns:

A dictionary with the following keys:

prompt: The prompt to be sent to the model.
specified inference parameters.

AmazonTitanAdapter

Adapter for Amazon's Titan models.

AmazonTitanAdapter.prepare_body

def prepare_body(prompt: str, **inference_kwargs) -> Dict[str, Any]

Prepares the body for the Titan model

Arguments:

prompt: The prompt to be sent to the model.
inference_kwargs: Additional keyword arguments passed to the handler.

Returns:

A dictionary with the following keys

inputText: The prompt to be sent to the model.
specified inference parameters.

MetaLlama2ChatAdapter

Adapter for Meta's Llama2 models.

MetaLlama2ChatAdapter.prepare_body

def prepare_body(prompt: str, **inference_kwargs) -> Dict[str, Any]

Prepares the body for the Llama2 model

Arguments:

prompt: The prompt to be sent to the model.
inference_kwargs: Additional keyword arguments passed to the handler.

Returns:

A dictionary with the following keys:

prompt: The prompt to be sent to the model.
specified inference parameters.

Module haystack_integrations.common.amazon_bedrock.errors

AmazonBedrockError

Any error generated by the Amazon Bedrock integration.

This error wraps its source transparently in such a way that its attributes can be accessed directly: for example, if the original error has a message attribute, AmazonBedrockError.message will exist and have the expected content.

AWSConfigurationError

Exception raised when AWS is not configured correctly

AmazonBedrockConfigurationError

Exception raised when AmazonBedrock node is not configured correctly

AmazonBedrockInferenceError

Exception for issues that occur in the Bedrock inference node

Module haystack_integrations.components.generators.amazon_bedrock.handlers

DefaultPromptHandler

DefaultPromptHandler resizes the prompt to ensure that the prompt and answer token lengths together are within the model_max_length.

DefaultPromptHandler.init

def __init__(tokenizer: Union[str, PreTrainedTokenizerBase],
             model_max_length: int,
             max_length: int = 100)

Arguments:

tokenizer: The tokenizer to be used to tokenize the prompt.
model_max_length: The maximum length of the prompt and answer tokens combined.
max_length: The maximum length of the answer tokens.

Raises:

ValueError: If the tokenizer is not a string or a PreTrainedTokenizer or PreTrainedTokenizerFast instance.

DefaultPromptHandler.call

def __call__(prompt: str, **kwargs) -> Dict[str, Union[str, int]]

Resizes the prompt to ensure that the prompt and answer is within the model_max_length

Arguments:

prompt: the prompt to be sent to the model.
kwargs: Additional keyword arguments passed to the handler.

Returns:

A dictionary containing the resized prompt and additional information.

TokenStreamingHandler

TokenStreamingHandler implementations handle the streaming of tokens from the stream.

TokenStreamingHandler.call

@abstractmethod
def __call__(token_received: str, **kwargs) -> str

This callback method is called when a new token is received from the stream.

Arguments:

token_received: The token received from the stream.
kwargs: Additional keyword arguments passed to the handler.

Returns:

The token to be sent to the stream.

DefaultTokenStreamingHandler

DefaultTokenStreamingHandler.call

def __call__(token_received, **kwargs) -> str

This callback method is called when a new token is received from the stream.

Arguments:

token_received: The token received from the stream.
kwargs: Additional keyword arguments passed to the handler.

Returns:

The token to be sent to the stream.

Module haystack_integrations.components.generators.amazon_bedrock.chat.chat_generator

AmazonBedrockChatGenerator

AmazonBedrockChatGenerator enables text generation via Amazon Bedrock hosted chat LLMs.

For example, to use the Anthropic Claude 3 Sonnet model, simply initialize the AmazonBedrockChatGenerator with the 'anthropic.claude-3-sonnet-20240229-v1:0' model name.

from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.components.generators.utils import print_streaming_chunk

messages = [ChatMessage.from_system("\nYou are a helpful, respectful and honest assistant, answer in German only"),
            ChatMessage.from_user("What's Natural Language Processing?")]


client = AmazonBedrockChatGenerator(model="anthropic.claude-3-sonnet-20240229-v1:0",
                                    streaming_callback=print_streaming_chunk)
client.run(messages, generation_kwargs={"max_tokens": 512})

If you prefer non-streaming mode, simply remove the streaming_callback parameter, capture the return value of the component's run method and the AmazonBedrockChatGenerator will return the response in a non-streaming mode.

AmazonBedrockChatGenerator.init

def __init__(
        model: str,
        aws_access_key_id: Optional[Secret] = Secret.from_env_var(
            ["AWS_ACCESS_KEY_ID"], strict=False),
        aws_secret_access_key: Optional[Secret] = Secret.
    from_env_var(  # noqa: B008
        ["AWS_SECRET_ACCESS_KEY"], strict=False),
        aws_session_token: Optional[Secret] = Secret.from_env_var(
            ["AWS_SESSION_TOKEN"], strict=False),
        aws_region_name: Optional[Secret] = Secret.from_env_var(
            ["AWS_DEFAULT_REGION"], strict=False),
        aws_profile_name: Optional[Secret] = Secret.from_env_var(
            ["AWS_PROFILE"], strict=False),
        generation_kwargs: Optional[Dict[str, Any]] = None,
        stop_words: Optional[List[str]] = None,
        streaming_callback: Optional[Callable[[StreamingChunk], None]] = None)

Initializes the AmazonBedrockChatGenerator with the provided parameters. The parameters are passed to the

Amazon Bedrock client.

Note that the AWS credentials are not required if the AWS environment is configured correctly. These are loaded automatically from the environment or the AWS configuration file and do not need to be provided explicitly via the constructor. If the AWS environment is not configured users need to provide the AWS credentials via the constructor. Aside from model, three required parameters are aws_access_key_id, aws_secret_access_key, and aws_region_name.

Arguments:

model: The model to use for generation. The model must be available in Amazon Bedrock. The model has to be specified in the format outlined in the Amazon Bedrock documentation.
aws_access_key_id: AWS access key ID.
aws_secret_access_key: AWS secret access key.
aws_session_token: AWS session token.
aws_region_name: AWS region name.
aws_profile_name: AWS profile name.
generation_kwargs: Additional generation keyword arguments passed to the model. The defined keyword parameters are specific to a specific model and can be found in the model's documentation. For example, the Anthropic Claude generation parameters can be found here.
stop_words: A list of stop words that stop model generation when encountered. They can be provided via this parameter or via models generation_kwargs under a model's specific key for stop words. For example, the Anthropic Claude stop words are provided via the stop_sequences key.
streaming_callback: A callback function that is called when a new chunk is received from the stream. By default, the model is not set up for streaming. To enable streaming simply set this parameter to a callback function that will handle the streaming chunks. The callback function will receive a StreamingChunk object and switch the streaming mode on.

AmazonBedrockChatGenerator.invoke

def invoke(*args, **kwargs)

Invokes the Amazon Bedrock LLM with the given parameters. The parameters are passed to the Amazon Bedrock

client.

Arguments:

args: The positional arguments passed to the generator.
kwargs: The keyword arguments passed to the generator.

Returns:

List of ChatMessage generated by LLM.

AmazonBedrockChatGenerator.run

@component.output_types(replies=List[ChatMessage])
def run(messages: List[ChatMessage],
        generation_kwargs: Optional[Dict[str, Any]] = None)

Generates a list of ChatMessage response to the given messages using the Amazon Bedrock LLM.

Arguments:

messages: The messages to generate a response to.
generation_kwargs: Additional generation keyword arguments passed to the model.

Returns:

A dictionary with the following keys:

replies: The generated List of ChatMessage objects.

AmazonBedrockChatGenerator.get_model_adapter

@classmethod
def get_model_adapter(cls,
                      model: str) -> Optional[Type[BedrockModelChatAdapter]]

Returns the model adapter for the given model.

Arguments:

model: The model to get the adapter for.

Returns:

The model adapter for the given model, or None if the model is not supported.

AmazonBedrockChatGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

AmazonBedrockChatGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "AmazonBedrockChatGenerator"

Deserializes the component from a dictionary.

Arguments:

data: Dictionary to deserialize from.

Returns:

Deserialized component.

Module haystack_integrations.components.embedders.amazon_bedrock.text_embedder

AmazonBedrockTextEmbedder

A component for embedding strings using Amazon Bedrock.

Usage example:

import os
from haystack_integrations.components.embedders.amazon_bedrock import AmazonBedrockTextEmbedder

os.environ["AWS_ACCESS_KEY_ID"] = "..."
os.environ["AWS_SECRET_ACCESS_KEY_ID"] = "..."
os.environ["AWS_REGION_NAME"] = "..."

embedder = AmazonBedrockTextEmbedder(
    model="cohere.embed-english-v3",
    input_type="search_query",
)

print(text_embedder.run("I love Paris in the summer."))

# {'embedding': [0.002, 0.032, 0.504, ...]}

AmazonBedrockTextEmbedder.init

def __init__(
        model: Literal["amazon.titan-embed-text-v1", "cohere.embed-english-v3",
                       "cohere.embed-multilingual-v3"],
        aws_access_key_id: Optional[Secret] = Secret.from_env_var(
            "AWS_ACCESS_KEY_ID", strict=False),
        aws_secret_access_key: Optional[Secret] = Secret.
    from_env_var(  # noqa: B008
        "AWS_SECRET_ACCESS_KEY", strict=False),
        aws_session_token: Optional[Secret] = Secret.from_env_var(
            "AWS_SESSION_TOKEN", strict=False),
        aws_region_name: Optional[Secret] = Secret.from_env_var(
            "AWS_DEFAULT_REGION", strict=False),
        aws_profile_name: Optional[Secret] = Secret.from_env_var("AWS_PROFILE",
                                                                 strict=False),
        **kwargs)

Initializes the AmazonBedrockTextEmbedder with the provided parameters. The parameters are passed to the

Amazon Bedrock client.

Arguments:

model: The embedding model to use. The model has to be specified in the format outlined in the Amazon Bedrock documentation.
aws_access_key_id: AWS access key ID.
aws_secret_access_key: AWS secret access key.
aws_session_token: AWS session token.
aws_region_name: AWS region name.
aws_profile_name: AWS profile name.
kwargs: Additional parameters to pass for model inference. For example, input_type and truncate for Cohere models.

Raises:

ValueError: If the model is not supported.
AmazonBedrockConfigurationError: If the AWS environment is not configured correctly.

AmazonBedrockTextEmbedder.run

@component.output_types(embedding=List[float])
def run(text: str)

Embeds the input text using the Amazon Bedrock model.

Arguments:

text: The input text to embed.

Raises:

TypeError: If the input text is not a string.
AmazonBedrockInferenceError: If the model inference fails.

Returns:

A dictionary with the following keys:

embedding: The embedding of the input text.

AmazonBedrockTextEmbedder.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

AmazonBedrockTextEmbedder.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "AmazonBedrockTextEmbedder"

Deserializes the component from a dictionary.

Arguments:

data: Dictionary to deserialize from.

Returns:

Deserialized component.

Module haystack_integrations.components.embedders.amazon_bedrock.document_embedder

AmazonBedrockDocumentEmbedder

A component for computing Document embeddings using Amazon Bedrock. The embedding of each Document is stored in the embedding field of the Document.

Usage example:

import os
from haystack.dataclasses import Document
from haystack_integrations.components.embedders.amazon_bedrock import AmazonBedrockDocumentEmbedder

os.environ["AWS_ACCESS_KEY_ID"] = "..."
os.environ["AWS_SECRET_ACCESS_KEY_ID"] = "..."
os.environ["AWS_REGION_NAME"] = "..."

embedder = AmazonBedrockDocumentEmbedder(
    model="cohere.embed-english-v3",
    input_type="search_document",
)

doc = Document(content="I love Paris in the winter.", meta={"name": "doc1"})

result = embedder.run([doc])
print(result['documents'][0].embedding)

# [0.002, 0.032, 0.504, ...]

AmazonBedrockDocumentEmbedder.init

def __init__(
        model: Literal["amazon.titan-embed-text-v1", "cohere.embed-english-v3",
                       "cohere.embed-multilingual-v3"],
        aws_access_key_id: Optional[Secret] = Secret.from_env_var(
            "AWS_ACCESS_KEY_ID", strict=False),
        aws_secret_access_key: Optional[Secret] = Secret.
    from_env_var(  # noqa: B008
        "AWS_SECRET_ACCESS_KEY", strict=False),
        aws_session_token: Optional[Secret] = Secret.from_env_var(
            "AWS_SESSION_TOKEN", strict=False),
        aws_region_name: Optional[Secret] = Secret.from_env_var(
            "AWS_DEFAULT_REGION", strict=False),
        aws_profile_name: Optional[Secret] = Secret.from_env_var("AWS_PROFILE",
                                                                 strict=False),
        batch_size: int = 32,
        progress_bar: bool = True,
        meta_fields_to_embed: Optional[List[str]] = None,
        embedding_separator: str = "\n",
        **kwargs)

Initializes the AmazonBedrockDocumentEmbedder with the provided parameters. The parameters are passed to the

Amazon Bedrock client.

Arguments:

model: The embedding model to use. The model has to be specified in the format outlined in the Amazon Bedrock documentation.
aws_access_key_id: AWS access key ID.
aws_secret_access_key: AWS secret access key.
aws_session_token: AWS session token.
aws_region_name: AWS region name.
aws_profile_name: AWS profile name.
batch_size: Number of Documents to encode at once. Only Cohere models support batch inference. This parameter is ignored for Amazon Titan models.
progress_bar: Whether to show a progress bar or not. Can be helpful to disable in production deployments to keep the logs clean.
meta_fields_to_embed: List of meta fields that should be embedded along with the Document text.
embedding_separator: Separator used to concatenate the meta fields to the Document text.
kwargs: Additional parameters to pass for model inference. For example, input_type and truncate for Cohere models.

Raises:

ValueError: If the model is not supported.
AmazonBedrockConfigurationError: If the AWS environment is not configured correctly.

AmazonBedrockDocumentEmbedder.run

@component.output_types(documents=List[Document])
def run(documents: List[Document])

Embed the provided Documents using the specified model.

Arguments:

documents: The Documents to embed.

Raises:

AmazonBedrockInferenceError: If the inference fails.

Returns:

A dictionary with the following keys:

documents: The Documents with the embedding field populated.

AmazonBedrockDocumentEmbedder.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

AmazonBedrockDocumentEmbedder.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "AmazonBedrockDocumentEmbedder"

Deserializes the component from a dictionary.

Arguments:

data: Dictionary to deserialize from.

Returns:

Deserialized component.

Module haystack_integrations.components.generators.amazon_bedrock.generator

AmazonBedrockGenerator

AmazonBedrockGenerator.__init__

AmazonBedrockGenerator.invoke

AmazonBedrockGenerator.run

AmazonBedrockGenerator.get_model_adapter

AmazonBedrockGenerator.to_dict

AmazonBedrockGenerator.from_dict

Module haystack_integrations.components.generators.amazon_bedrock.adapters

BedrockModelAdapter

BedrockModelAdapter.prepare_body

BedrockModelAdapter.get_responses

BedrockModelAdapter.get_stream_responses

AnthropicClaudeAdapter

AnthropicClaudeAdapter.prepare_body

MistralAdapter

MistralAdapter.prepare_body

CohereCommandAdapter

CohereCommandAdapter.prepare_body

AI21LabsJurassic2Adapter

AI21LabsJurassic2Adapter.prepare_body

AmazonTitanAdapter

AmazonTitanAdapter.prepare_body

MetaLlama2ChatAdapter

MetaLlama2ChatAdapter.prepare_body

Module haystack_integrations.common.amazon_bedrock.errors

AmazonBedrockError

AWSConfigurationError

AmazonBedrockConfigurationError

AmazonBedrockInferenceError

Module haystack_integrations.components.generators.amazon_bedrock.handlers

DefaultPromptHandler

DefaultPromptHandler.__init__

DefaultPromptHandler.__call__

TokenStreamingHandler

TokenStreamingHandler.__call__

DefaultTokenStreamingHandler

DefaultTokenStreamingHandler.__call__

Module haystack_integrations.components.generators.amazon_bedrock.chat.chat_generator

AmazonBedrockChatGenerator

AmazonBedrockChatGenerator.__init__

AmazonBedrockChatGenerator.invoke

AmazonBedrockChatGenerator.run

AmazonBedrockChatGenerator.get_model_adapter

AmazonBedrockChatGenerator.to_dict

AmazonBedrockChatGenerator.from_dict

Module haystack_integrations.components.embedders.amazon_bedrock.text_embedder

AmazonBedrockTextEmbedder

AmazonBedrockTextEmbedder.__init__

AmazonBedrockTextEmbedder.run

AmazonBedrockTextEmbedder.to_dict

AmazonBedrockTextEmbedder.from_dict

Module haystack_integrations.components.embedders.amazon_bedrock.document_embedder

AmazonBedrockDocumentEmbedder

AmazonBedrockDocumentEmbedder.__init__

AmazonBedrockDocumentEmbedder.run

AmazonBedrockDocumentEmbedder.to_dict

AmazonBedrockDocumentEmbedder.from_dict

AmazonBedrockGenerator.init

DefaultPromptHandler.init

DefaultPromptHandler.call

TokenStreamingHandler.call

DefaultTokenStreamingHandler.call

AmazonBedrockChatGenerator.init

AmazonBedrockTextEmbedder.init

AmazonBedrockDocumentEmbedder.init