The Transformer models used in Haystack are designed to be run on GPU-accelerated hardware. The steps for GPU acceleration setup depend on the environment that you're working in.
Ensure that you have a fitting version of NVIDIA CUDA installed. To learn how to install CUDA, see the NVIDIA CUDA Guide for Linux.
nvidia-smiin the command line to check if the GPU is enabled. If the GPU is enabled, the output shows a list of available GPUs and their memory usage:
- In your Colab environment, select Runtime>Change Runtime type.
- Choose Hardware accelerator>GPU.
- To check if the GPU is enabled, run:
The output should show the GPUs available and their usage.
Updated 11 months ago