DocumentationAPI ReferenceTutorialsGitHub Code ExamplesDiscord Community

Enabling GPU Acceleration

Speed up your Haystack application by engaging the GPU.

The Transformer models used in Haystack are designed to be run on GPU-accelerated hardware. The steps for GPU acceleration setup depend on the environment that you're working in.

Enabling the GPU in Linux

  1. Ensure that you have a fitting version of NVIDIA CUDA installed. To learn how to install CUDA, see the NVIDIA CUDA Guide for Linux.

  2. Run the nvidia-smiin the command line to check if the GPU is enabled. If the GPU is enabled, the output shows a list of available GPUs and their memory usage:
    A screenshot of the command output with the name of the GPU device and its memory usage highlighted.

Enabling the GPU in Colab

  1. In your Colab environment, select Runtime>Change Runtime type.

The Runtime dropdown menu in Colab

  1. Choose Hardware accelerator>GPU.
  2. To check if the GPU is enabled, run:


The output should show the GPUs available and their usage.