HomeGuidesAPI ReferenceTutorials
Haystack

Enabling GPU Acceleration

Speed up your Haystack application by engaging the GPU.

The Transformer models used in Haystack are designed to be run on GPU accelerated hardware. The steps for GPU acceleration setup depend on the environment that you're working in.

Enabling the GPU in Linux

  1. Ensure that a fitting version of NVIDIA CUDA is installed. To learn how to install CUDA, see the NVIDIA CUDA Guide for Linux.

  2. Check if the GPU is being used by running the nvidia-smiin the command line. If the GPU is enabled, you should see output resembling this:

Expected output when GPU is enabled.Expected output when GPU is enabled.

Expected output when GPU is enabled. Look out for the list of available GPUs which is shown here in the line that reads 0 NVIDIA GeForce .... This shows that device number 0 is an NVIDIA GeForce card. You can also see the memory-usage in the center cell which in this case reads 927MiB / 8116MiB.

Enabling the GPU in Colab

  1. In your Colab environment, select Runtime -> Change Runtime type.
  2. Choose Hardware accelerator -> GPU.
The Runtime dropdown menu in ColabThe Runtime dropdown menu in Colab

The Runtime dropdown menu in Colab

To check if the GPU is enabled, run:

%%bash

nvidia-smi

The output should be similar to what is shown in the image above.