How to get NVIDIA GPU working on self-hosted runner for GitHub Actions

[saltz]

Why are you starting this discussion?

Question

What GitHub Actions topic or product is this about?

Actions Runner

Discussion Details

I'm setting up a self-hosted runner for GitHub Actions and have installed an NVIDIA GPU on the machine, but the GPU is not being utilized by my workflow jobs.

OS: Ubuntu 24.04.3 LTS
ARCH: x86_64
GPU: NVIDIA GeForce RTX 3060
NVIDIA-SMI: 580.126.09
Driver Version: 580.126.09
CUDA Version: 13.0

What I have already done

I have installed the NVIDIA drivers and verified with nvidia-smi, registered the self-hosted runner with the gpu label, created a workflow that targets the gpu label. This workflow runs some dotnet tests which could utilize the gpu.

Questions

1. Do I need to configure specific environment variables for GPU access?
2. Are there additional CUDA/GPU libraries or dependencies needed in the runner environment?
3. Are there any known issues with NVIDIA GPU support on self-hosted runners?

Any guidance or documentation would be helpful!

[andreas-agouridis]

Hi @saltz,

Getting a self-hosted runner to actually utilize an NVIDIA GPU requires more than just installing drivers—it’s mostly about making sure your workflows can see and access the GPU. Here’s a breakdown:

1. Verify GPU access from the runner host
   Run nvidia-smi from the runner machine. If it shows your GPU, drivers are installed correctly.
   Make sure the runner process itself has permission to access the GPU.
2. Install CUDA and necessary libraries
   Having the NVIDIA driver is enough for nvidia-smi, but most workloads (including .NET ML or GPU-accelerated libraries) require the CUDA toolkit and runtime.
   For Ubuntu 24.04, you can install CUDA 13.0 as you already have, but confirm that the relevant libraries are on the PATH and LD_LIBRARY_PATH. Example:
   export PATH=/usr/local/cuda-13.0/bin:$PATH
   export LD_LIBRARY_PATH=/usr/local/cuda-13.0/lib64:$LD_LIBRARY_PATH
   Some .NET GPU libraries might also need cuDNN or NCCL depending on your workload.
3. Make sure the runner sees the GPU
   Self-hosted runners just run commands—they don’t automatically pass GPU access to Docker or other isolated environments.
   If you’re using Docker or dind, you need NVIDIA Container Toolkit:
   sudo apt install -y nvidia-container-toolkit
   sudo systemctl restart docker
   Then your workflows can use:
   runs-on: [self-hosted, gpu]
   container:
   image: nvidia/cuda:13.0-base
   options: --gpus all
4. Environment variables
   Many GPU libraries rely on these:
   export CUDA_VISIBLE_DEVICES=0 # which GPU(s) to use
   export NVIDIA_VISIBLE_DEVICES=all
   Dotnet ML libraries may automatically detect the GPU if these are set and CUDA is installed.
5. Workflow configuration tips
   Use the gpu label in runs-on:
   jobs:
   gpu-job:
   runs-on: [self-hosted, gpu]
   steps:
   • uses: actions/checkout@v3
   • name: Test GPU access
     run: nvidia-smi
     This ensures your job lands on a machine with GPU hardware.
6. Known limitations
   GitHub Actions itself doesn’t provide GPU drivers—everything is dependent on your host setup.
   If using Docker, GPU access won’t work unless the container is run with --gpus and the NVIDIA Container Toolkit is installed.
   Some .NET GPU libraries may require additional native dependencies (CUDA/cuDNN).