Module haystack_integrations.components.embedders.nvidia.document_embedder

NvidiaDocumentEmbedder

A component for embedding documents using embedding models provided by NVIDIA NIMs.

Usage example:

from haystack_integrations.components.embedders.nvidia import NvidiaDocumentEmbedder

doc = Document(content="I love pizza!")

text_embedder = NvidiaDocumentEmbedder(model="NV-Embed-QA", api_url="https://ai.api.nvidia.com/v1/retrieval/nvidia")
text_embedder.warm_up()

result = document_embedder.run([doc])
print(result["documents"][0].embedding)

NvidiaDocumentEmbedder.__init__

def __init__(model: str = "NV-Embed-QA",
             api_key: Optional[Secret] = Secret.from_env_var("NVIDIA_API_KEY"),
             api_url: str = "https://ai.api.nvidia.com/v1/retrieval/nvidia",
             prefix: str = "",
             suffix: str = "",
             batch_size: int = 32,
             progress_bar: bool = True,
             meta_fields_to_embed: Optional[List[str]] = None,
             embedding_separator: str = "\n",
             truncate: Optional[Union[EmbeddingTruncateMode, str]] = None)

Create a NvidiaTextEmbedder component.

Arguments:

• model: Embedding model to use.
• api_key: API key for the NVIDIA NIM.
• api_url: Custom API URL for the NVIDIA NIM.
• prefix: A string to add to the beginning of each text.
• suffix: A string to add to the end of each text.
• batch_size: Number of Documents to encode at once. Cannot be greater than 50.
• progress_bar: Whether to show a progress bar or not.
• meta_fields_to_embed: List of meta fields that should be embedded along with the Document text.
• embedding_separator: Separator used to concatenate the meta fields to the Document text.
• truncate: Specifies how inputs longer that the maximum token length should be truncated. If None the behavior is model-dependent, see the official documentation for more information.

NvidiaDocumentEmbedder.warm_up

def warm_up()

Initializes the component.

NvidiaDocumentEmbedder.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

NvidiaDocumentEmbedder.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "NvidiaDocumentEmbedder"

Deserializes the component from a dictionary.

Arguments:

• data: The dictionary to deserialize from.

Returns:

The deserialized component.

NvidiaDocumentEmbedder.run

@component.output_types(documents=List[Document], meta=Dict[str, Any])
def run(documents: List[Document])

Embed a list of Documents.

The embedding of each Document is stored in the embedding field of the Document.

Arguments:

• documents: A list of Documents to embed.

Raises:

• RuntimeError: If the component was not initialized.
• TypeError: If the input is not a string.

Returns:

A dictionary with the following keys and values:

• documents - List of processed Documents with embeddings.
• meta - Metadata on usage statistics, etc.

Module haystack_integrations.components.embedders.nvidia.text_embedder

NvidiaTextEmbedder

A component for embedding strings using embedding models provided by NVIDIA NIMs.

For models that differentiate between query and document inputs, this component embeds the input string as a query.

Usage example:

from haystack_integrations.components.embedders.nvidia import NvidiaTextEmbedder

text_to_embed = "I love pizza!"

text_embedder = NvidiaTextEmbedder(model="NV-Embed-QA", api_url="https://ai.api.nvidia.com/v1/retrieval/nvidia")
text_embedder.warm_up()

print(text_embedder.run(text_to_embed))

NvidiaTextEmbedder.__init__

def __init__(model: str = "NV-Embed-QA",
             api_key: Optional[Secret] = Secret.from_env_var("NVIDIA_API_KEY"),
             api_url: str = "https://ai.api.nvidia.com/v1/retrieval/nvidia",
             prefix: str = "",
             suffix: str = "",
             truncate: Optional[Union[EmbeddingTruncateMode, str]] = None)

Create a NvidiaTextEmbedder component.

Arguments:

• model: Embedding model to use.
• api_key: API key for the NVIDIA NIM.
• api_url: Custom API URL for the NVIDIA NIM.
• prefix: A string to add to the beginning of each text.
• suffix: A string to add to the end of each text.
• truncate: Specifies how inputs longer that the maximum token length should be truncated. If None the behavior is model-dependent, see the official documentation for more information.

NvidiaTextEmbedder.warm_up

def warm_up()

Initializes the component.

NvidiaTextEmbedder.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

NvidiaTextEmbedder.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "NvidiaTextEmbedder"

Deserializes the component from a dictionary.

Arguments:

• data: The dictionary to deserialize from.

Returns:

The deserialized component.

NvidiaTextEmbedder.run

@component.output_types(embedding=List[float], meta=Dict[str, Any])
def run(text: str)

Embed a string.

Arguments:

• text: The text to embed.

Raises:

• RuntimeError: If the component was not initialized.
• TypeError: If the input is not a string.

Returns:

A dictionary with the following keys and values:

• embedding - Embeddng of the text.
• meta - Metadata on usage statistics, etc.

Module haystack_integrations.components.generators.nvidia.generator

NvidiaGenerator

A component for generating text using generative models provided by NVIDIA NIMs.

Usage example:

from haystack_integrations.components.generators.nvidia import NvidiaGenerator

generator = NvidiaGenerator(
    model="meta/llama3-70b-instruct",
    model_arguments={
        "temperature": 0.2,
        "top_p": 0.7,
        "max_tokens": 1024,
    },
)
generator.warm_up()

result = generator.run(prompt="What is the answer?")
print(result["replies"])
print(result["meta"])
print(result["usage"])

NvidiaGenerator.__init__

def __init__(model: str,
             api_url: str = _DEFAULT_API_URL,
             api_key: Optional[Secret] = Secret.from_env_var("NVIDIA_API_KEY"),
             model_arguments: Optional[Dict[str, Any]] = None)

Create a NvidiaGenerator component.

Arguments:

• model: Name of the model to use for text generation. See the NVIDIA NIMs for more information on the supported models.
• api_key: API key for the NVIDIA NIM.
• api_url: Custom API URL for the NVIDIA NIM.
• model_arguments: Additional arguments to pass to the model provider. Different models accept different arguments. Search your model in the NVIDIA NIMs to know the supported arguments.

NvidiaGenerator.warm_up

def warm_up()

Initializes the component.

NvidiaGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

NvidiaGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "NvidiaGenerator"

Deserializes the component from a dictionary.

Arguments:

• data: Dictionary to deserialize from.

Returns:

Deserialized component.

NvidiaGenerator.run

@component.output_types(replies=List[str], meta=List[Dict[str, Any]])
def run(prompt: str)

Queries the model with the provided prompt.

Arguments:

• prompt: Text to be sent to the generative model.

Returns:

A dictionary with the following keys:

• replies - Replies generated by the model.
• meta - Metadata for each reply.