Module haystack_integrations.components.embedders.nvidia.document_embedder

NvidiaDocumentEmbedder

A component for embedding documents using embedding models provided by NVIDIA AI Foundation Endpoints and NVIDIA Inference Microservices.

Usage example:

from haystack_integrations.components.embedders.nvidia import NvidiaDocumentEmbedder

doc = Document(content="I love pizza!")

text_embedder = NvidiaDocumentEmbedder(model="nvolveqa_40k")
text_embedder.warm_up()

result = document_embedder.run([doc])
print(result["documents"][0].embedding)

NvidiaDocumentEmbedder.__init__

def __init__(model: str,
             api_key: Optional[Secret] = Secret.from_env_var("NVIDIA_API_KEY"),
             api_url: Optional[str] = None,
             prefix: str = "",
             suffix: str = "",
             batch_size: int = 32,
             progress_bar: bool = True,
             meta_fields_to_embed: Optional[List[str]] = None,
             embedding_separator: str = "\n")

Create a NvidiaTextEmbedder component.

Arguments:

• model: Embedding model to use.
• api_key: API key for the NVIDIA AI Foundation Endpoints.
• api_url: Custom API URL for the NVIDIA Inference Microservices.
• prefix: A string to add to the beginning of each text.
• suffix: A string to add to the end of each text.
• batch_size: Number of Documents to encode at once. Cannot be greater than 50.
• progress_bar: Whether to show a progress bar or not.
• meta_fields_to_embed: List of meta fields that should be embedded along with the Document text.
• embedding_separator: Separator used to concatenate the meta fields to the Document text.

NvidiaDocumentEmbedder.warm_up

def warm_up()

Initializes the component.

NvidiaDocumentEmbedder.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

NvidiaDocumentEmbedder.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "NvidiaDocumentEmbedder"

Deserializes the component from a dictionary.

Arguments:

• data: The dictionary to deserialize from.

Returns:

The deserialized component.

NvidiaDocumentEmbedder.run

@component.output_types(documents=List[Document], meta=Dict[str, Any])
def run(documents: List[Document])

Embed a list of Documents.

The embedding of each Document is stored in the embedding field of the Document.

Arguments:

• documents: A list of Documents to embed.

Raises:

• RuntimeError: If the component was not initialized.
• TypeError: If the input is not a string.

Returns:

A dictionary with the following keys and values:

• documents - List of processed Documents with embeddings.
• meta - Metadata on usage statistics, etc.

Module haystack_integrations.components.embedders.nvidia.text_embedder

NvidiaTextEmbedder

A component for embedding strings using embedding models provided by NVIDIA AI Foundation Endpoints and NVIDIA Inference Microservices.

For models that differentiate between query and document inputs, this component embeds the input string as a query.

Usage example:

from haystack_integrations.components.embedders.nvidia import NvidiaTextEmbedder

text_to_embed = "I love pizza!"

text_embedder = NvidiaTextEmbedder(model="nvolveqa_40k")
text_embedder.warm_up()

print(text_embedder.run(text_to_embed))

NvidiaTextEmbedder.__init__

def __init__(model: str,
             api_key: Optional[Secret] = Secret.from_env_var("NVIDIA_API_KEY"),
             api_url: Optional[str] = None,
             prefix: str = "",
             suffix: str = "")

Create a NvidiaTextEmbedder component.

Arguments:

• model: Embedding model to use.
• api_key: API key for the NVIDIA AI Foundation Endpoints.
• api_url: Custom API URL for the NVIDIA Inference Microservices.
• prefix: A string to add to the beginning of each text.
• suffix: A string to add to the end of each text.

NvidiaTextEmbedder.warm_up

def warm_up()

Initializes the component.

NvidiaTextEmbedder.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

NvidiaTextEmbedder.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "NvidiaTextEmbedder"

Deserializes the component from a dictionary.

Arguments:

• data: The dictionary to deserialize from.

Returns:

The deserialized component.

NvidiaTextEmbedder.run

@component.output_types(embedding=List[float], meta=Dict[str, Any])
def run(text: str)

Embed a string.

Arguments:

• text: The text to embed.

Raises:

• RuntimeError: If the component was not initialized.
• TypeError: If the input is not a string.

Returns:

A dictionary with the following keys and values:

• embedding - Embeddng of the text.
• meta - Metadata on usage statistics, etc.

Module haystack_integrations.components.generators.nvidia.generator

NvidiaGenerator

A component for generating text using generative models provided by NVIDIA AI Foundation Endpoints and NVIDIA Inference Microservices.

Usage example:

from haystack_integrations.components.generators.nvidia import NvidiaGenerator

generator = NvidiaGenerator(
    model="nv_llama2_rlhf_70b",
    model_arguments={
        "temperature": 0.2,
        "top_p": 0.7,
        "max_tokens": 1024,
    },
)
generator.warm_up()

result = generator.run(prompt="What is the answer?")
print(result["replies"])
print(result["meta"])
print(result["usage"])

NvidiaGenerator.__init__

def __init__(model: str,
             api_url: Optional[str] = None,
             api_key: Optional[Secret] = Secret.from_env_var("NVIDIA_API_KEY"),
             model_arguments: Optional[Dict[str, Any]] = None)

Create a NvidiaGenerator component.

Arguments:

• model: Name of the model to use for text generation. See the Nvidia catalog for more information on the supported models.
• api_key: API key for the NVIDIA AI Foundation Endpoints.
• api_url: Custom API URL for the NVIDIA Inference Microservices.
• model_arguments: Additional arguments to pass to the model provider. Different models accept different arguments. Search your model in the Nvidia catalog to know the supported arguments.

NvidiaGenerator.warm_up

def warm_up()

Initializes the component.

NvidiaGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

NvidiaGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "NvidiaGenerator"

Deserializes the component from a dictionary.

Arguments:

• data: Dictionary to deserialize from.

Returns:

Deserialized component.

NvidiaGenerator.run

@component.output_types(replies=List[str], meta=List[Dict[str, Any]])
def run(prompt: str)

Queries the model with the provided prompt.

Arguments:

• prompt: Text to be sent to the generative model.

Returns:

A dictionary with the following keys:

• replies - Replies generated by the model.
• meta - Metadata for each reply.