ParallelWebSearch

Search the web using the Parallel Search API.

Most common position in a pipeline	Before a ChatPromptBuilder or at the beginning of an indexing pipeline
Mandatory init variables	api_key: A Parallel API key. Can be set with PARALLEL_API_KEY env var.
Mandatory run variables	query: A string with your search query.
Output variables	documents: A list of Haystack Documents containing search result excerpts and metadata. links: A list of strings of resulting URLs. session_id: A string identifying the search session.
API reference	Integrations
GitHub link	https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/parallel/src/haystack_integrations/components/websearch/parallel/parallel_websearch.py
Package name	parallel-haystack

Overview

When you give ParallelWebSearch a query, it uses the Parallel Search API to search the web and return LLM-optimized excerpts as Haystack Document objects. It also returns a list of the source URLs and the identifier of the search session.

Each returned Document contains the result's excerpts joined into its content and a meta dictionary with title and url fields, excerpts when nonempty, and publish_date where the API provides one.

ParallelWebSearch requires a Parallel API key to work. By default, it reads from the PARALLEL_API_KEY environment variable. You can also pass an api_key directly during initialization.

The top_k parameter controls the maximum number of results returned (default is 10). It maps to the advanced_settings.max_results API parameter.

You can refine search results using search_params, which supports keys such as mode, objective, max_chars_total, session_id, and advanced_settings (with nested source_policy domain and date filters, fetch_policy, excerpt_settings, location, and max_results). These can be set at initialization or per run() call. Passing search_params to run() replaces the entire initialization-time dictionary; it does not merge individual keys. An explicit advanced_settings.max_results takes precedence over top_k. The Search API offers four modes — turbo, fast, basic, and advanced — in increasing order of latency and quality. See the Parallel Search API reference for the full list of parameters.

Searches that belong to the same task can share a session. The session_id output holds the identifier the API used, whether you sent one in search_params or the API generated it. Pass it into follow-up searches to get better contextual results.

ParallelWebSearch supports both synchronous (run()) and asynchronous (run_async()) operation.

Installation

Install the integration and set your Parallel API key before running the examples:

bash

pip install parallel-haystack
export PARALLEL_API_KEY="YOUR_PARALLEL_API_KEY"

Usage

On its own

python

from haystack.utils import Secret
from haystack_integrations.components.websearch.parallel import ParallelWebSearch

web_search = ParallelWebSearch(
    api_key=Secret.from_env_var("PARALLEL_API_KEY"),
    top_k=5,
)
result = web_search.run(query="What is Haystack by deepset?")

for doc in result["documents"]:
    print(doc.content)
    print(doc.meta["url"])

With a faster search mode and a domain filter:

python

from haystack.utils import Secret
from haystack_integrations.components.websearch.parallel import ParallelWebSearch

web_search = ParallelWebSearch(
    api_key=Secret.from_env_var("PARALLEL_API_KEY"),
    top_k=5,
    search_params={
        "mode": "turbo",
        "advanced_settings": {"source_policy": {"include_domains": ["arxiv.org"]}},
    },
)
result = web_search.run(query="Latest retrieval-augmented generation research")

for doc in result["documents"]:
    print(doc.meta["title"], doc.meta["url"])

Reusing a session across related searches:

python

from haystack.utils import Secret
from haystack_integrations.components.websearch.parallel import ParallelWebSearch

web_search = ParallelWebSearch(api_key=Secret.from_env_var("PARALLEL_API_KEY"))

first = web_search.run(query="What is Haystack by deepset?")
second = web_search.run(
    query="Who maintains Haystack?",
    search_params={"session_id": first["session_id"]},
)

print(second["links"])

In a pipeline

This pipeline passes search excerpts to a prompt and generates an answer. ParallelChatGenerator also performs its own web research, so its answer is not restricted to the supplied excerpts.

python

from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.parallel import ParallelChatGenerator
from haystack_integrations.components.websearch.parallel import ParallelWebSearch

web_search = ParallelWebSearch(
    api_key=Secret.from_env_var("PARALLEL_API_KEY"),
    top_k=3,
)

prompt_template = [
    ChatMessage.from_system("You are a helpful assistant."),
    ChatMessage.from_user(
        "Given the information below:\n"
        "{% for document in documents %}{{ document.content }}\n{% endfor %}\n"
        "Answer the following question: {{ query }}.\nAnswer:",
    ),
]

prompt_builder = ChatPromptBuilder(
    template=prompt_template,
    required_variables=["query", "documents"],
)

llm = ParallelChatGenerator(
    api_key=Secret.from_env_var("PARALLEL_API_KEY"),
    generation_kwargs={"reasoning": {"effort": "low"}},
)

pipe = Pipeline()
pipe.add_component("search", web_search)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)

pipe.connect("search.documents", "prompt_builder.documents")
pipe.connect("prompt_builder.prompt", "llm.messages")

query = "What is Haystack by deepset?"
result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})
print(result["llm"]["replies"][0].text)