Installation
You can install Haystack in a couple of ways - basic using pip, full, and custom. You can also install REST API. Choose your installation method and follow the instructions.
Haystack Repos
All the core Haystack components live in the haystack repo. But there's also the haystack-extras repo which contains components that are not as widely used, and you need to install them separately.
Installing Haystack Core
These are the instructions for installing the components from the haystack repo.
Basic Installation
Use pip to install a basic version of Haystack's latest release:
pip install farm-haystack
This command installs everything you need for basic Pipelines that use an Elasticsearch DocumentStore.
Full Installation
To use more advanced features, like certain DocumentStores, FileConverters, OCR, or Ray, install further dependencies. The following command installs the latest release of Haystack and all its dependencies:
pip install --upgrade pip
pip install 'farm-haystack[all]' ## or 'all-gpu' for the GPU-enabled dependencies
Installing from main
main
If you want to try out the newest features that are not in an official release yet, you can install Haystack from the main
branch. The following command installs from main
with dev
dependencies:
pip install git+https://github.com/deepset-ai/haystack.git@main#egg=farm-haystack[dev]
To be able to make changes to Haystack code, install with the following commands:
# Clone the repo
git clone https://github.com/deepset-ai/haystack.git
# Move into the cloned folder
cd haystack
# Upgrade pip
pip install --upgrade pip
# Install Haystack in editable mode
pip install -e '.[all]'
Additionally, if you want to contribute to the Haystack, check our Contributor Guidelines first.
Custom Installation
You can choose the dependencies you want to install. To do so, specify them in the pip install
command:
pip install 'farm-haystack[DEPENDENCY_OPTION]'
You can find a full list of dependency options at haystack/pyproject.toml. Common options are:
Dependency Option | Description |
---|---|
all | Install Haystack and all optional dependencies, including developer tools. |
all-gpu | Install Haystack and all optional dependencies, including developer tools with GPU support. |
colab | Install Haystack and all dependencies needed to run Haystack in Google Colab. |
crawler | Install Haystack and all web crawling tools. |
dev | Install Haystack and all development tools needed by contributors. |
ocr | Install Haystack and all OCR tools. |
docstores | Install Haystack and all DocumentStores. |
docstores-gpu | Install Haystack and all DocumentStores with GPU support. |
faiss | Install Haystack and FAISS support for the FAISSDocumentStore. |
faiss-gpu | Install Haystack and FAISS support for the FAISSDocumentStore with GPU. |
opensearch | Install Haystack and OpenSearch support for the OpenSearchDocumentStore. |
elasticsearch | Install Haystack and Elasticsearch support for the ElasticsearchDocumentStore |
pdf | Install Haystack and PyMuPDF for PDFToTextConverter. If you don't want to use PyMuPDF for licensing issues, do _not _install this extra and use PDFToTextConverter instead. |
pinecone | Install Haystack and Pinecone support for the PineconeDocumentStore. |
preprocessing | Install Haystack and the basic preprocessing tools, such as langdetect for language identification and nltk for precise document splitting. |
file-conversion | Install Haystack and all the dependencies for file conversion and parsing, like python-docx , tika , markdown . |
ray | Install Haystack and Ray support for the RayPipeline. |
weaviate | Install Haystack and Weaviate support for the WeaviateDocumentStore. |
If you are running pip<21.3
, you can't install dependency groups that reference other groups. Instead, you can only specify groups that contain direct package references.
# instead of [all]
pip install farm-haystack[sql,only-faiss,weaviate,pinecone,opensearch,graphdb,inmemorygraph,crawler,preprocessing,ocr,onnx,ray,dev]
# instead of [all-gpu]
pip install farm-haystack[sql,only-faiss-gpu,weaviate,pinecone,opensearch,graphdb,inmemorygraph,crawler,preprocessing,ocr,onnx-gpu,ray,dev]
Installing the REST API
Haystack can be deployed as a service exposing a REST API after installing an additional package called rest_api
. The package is not available on PyPI so you have to run the following command to install it:
pip install "git+https://github.com/deepset-ai/haystack.git#egg=rest_api&subdirectory=rest_api"
Other Operating Systems
Windows
We recommend installing WSL to use Haystack on Windows:
pip install farm-haystack -f https://download.pytorch.org/whl/torch_stable.html
Apple Silicon (M1)
Macs with an M1 processor require some extra dependencies to install Haystack:
# some additional dependencies needed on m1 mac
brew install postgresql
brew install cmake
brew install rust
# haystack installation
GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install farm-haystack
Installing Extras
You need to install the nodes from the haystack-extras repo one by one. For detailed instructions, see the documentation of each node:
Updated over 1 year ago