Skip to main content
Version: 3.5

OpenSearchDocumentStore

A Document Store for storing and retrieving documents in OpenSearch.

OpenSearch is a scalable, open-source search and analytics suite for full-text search, application monitoring, log analytics, and vector search. For more information, see the OpenSearch documentation.

OpenSearchDocumentStore supports keyword and vector similarity retrieval, including approximate nearest neighbor search. You can use it to compare sparse and dense retrieval approaches. It is compatible with Amazon OpenSearch Service.

Initialization​

Choose the instructions for your deployment: a local Docker instance or an Amazon OpenSearch Service domain. A local instance does not require an AWS account or IAM credentials.

Local Docker Instance​

Install and run OpenSearch. If you have Docker available, you can use the following commands.

OpenSearch 2.12 and later require a custom administrator password for the demo security configuration. Replace SecureHaystack!2026 with your own strong password in both the Docker command and the Python example below. The username remains admin. See Setting a custom admin password.

shell
docker pull opensearchproject/opensearch:3.5.0
docker run \
-p 9200:9200 \
-p 9600:9600 \
-e "discovery.type=single-node" \
-e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" \
-e 'OPENSEARCH_INITIAL_ADMIN_PASSWORD=SecureHaystack!2026' \
opensearchproject/opensearch:3.5.0

Alternatively, use the docker-compose.yml provided in the OpenSearch integration repository. Use the same administrator password in that file and in http_auth in the Python example below. Run this command from the integration directory:

shell
docker compose up

Once OpenSearch is running, install the integration:

shell
pip install opensearch-haystack

Initialize the Document Store and write two documents. The local demo uses self-signed certificates, so this example disables certificate verification with verify_certs=False.

python
from haystack import Document
from haystack_integrations.document_stores.opensearch import OpenSearchDocumentStore

document_store = OpenSearchDocumentStore(
hosts="https://localhost:9200",
use_ssl=True,
verify_certs=False,
http_auth=("admin", "SecureHaystack!2026"),
)
document_store.write_documents(
[Document(content="This is first"), Document(content="This is second")],
)
print(document_store.count_documents())
# >> 2

For a production cluster, enable certificate verification and configure ca_certs if your cluster requires a custom CA certificate. See the OpenSearch Python client documentation.

Amazon OpenSearch Service​

To use Amazon OpenSearch Service, create an OpenSearch domain or use an existing one. A domain is an AWS-managed OpenSearch cluster. Once it is ready, find its endpoint in the Amazon OpenSearch Service console and follow the instructions below for the authentication method configured for your domain.

IAM authentication​

Install Boto3 alongside the integration:

shell
pip install opensearch-haystack boto3

Configure AWS credentials through the Boto3 credential provider chain, for example with an IAM role, an AWS profile, or the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. When configuring temporary credentials through environment variables, also set AWS_SESSION_TOKEN.

Use the integration's AWSAuth wrapper to sign requests. Replace the example hostname with your domain endpoint without https://, and set aws_region_name to the domain's region:

python
from haystack.utils import Secret
from haystack_integrations.document_stores.opensearch import OpenSearchDocumentStore
from haystack_integrations.document_stores.opensearch.auth import AWSAuth

document_store = OpenSearchDocumentStore(
hosts=[{"host": "search-my-domain.us-east-1.es.amazonaws.com", "port": 443}],
http_auth=AWSAuth(aws_region_name=Secret.from_token("us-east-1")),
use_ssl=True,
verify_certs=True,
)

AWSAuth uses the es signing service by default for Amazon OpenSearch Service domains.

The IAM principal must have the required domain permissions and, when fine-grained access control is enabled, the appropriate OpenSearch role mapping. See IAM authentication and fine-grained access control.

Username and password​

If your domain uses the internal user database, install the integration:

shell
pip install opensearch-haystack

In the AWS example above, replace the http_auth=AWSAuth(...) argument with:

python
http_auth = ((username, password),)

Boto3 and the AWSAuth import are not required for this authentication method. Use the domain's OpenSearch username and password, not your AWS console sign-in credentials.

When using HTTP basic authentication, ensure that the domain access policy does not require IAM-signed requests.

Supported Retrievers​

OpenSearchBM25Retriever: A keyword-based Retriever that fetches documents matching a query from the Document Store.

OpenSearchEmbeddingRetriever: Compares the query and document embeddings and fetches the documents most relevant to the query.

Additional References​

🧑‍🍳 Cookbook: PDF-Based Question Answering with Amazon Bedrock and Haystack