GPU Compute on Shani OS — NVIDIA CUDA, AMD ROCm, and Intel oneAPI

Shani OS ships with GPU drivers pre-installed — Mesa's open-source stack for AMD and Intel, and the proprietary nvidia-open stack for NVIDIA. The GPU compute toolchains (ROCm, CUDA development libraries, oneAPI) are not pre-installed on the host; the recommended approach is to run them inside Distrobox containers using the vendor's official images. This keeps the host OS clean and immutable while giving you fully-configured compute environments that survive every OS update.

This post covers how to set up and use GPU compute — from verifying hardware, to running PyTorch or TensorFlow, to writing portable GPU code that runs on any vendor.

For GPU passthrough to VMs (dedicating a GPU to a Windows VM for gaming or isolated ML), see Virtual Machines on Shani OS. For using GPU compute inside Apptainer containers for HPC clusters, see Apptainer on Shani OS.

Full reference: docs.shani.dev — Containers.

---

What Ships on the Host

Shani OS includes GPU drivers, not compute toolchains. Here is what is pre-installed:

| Vendor | What's included on the host | |---|---| | AMD | vulkan-radeon (Mesa RADV), AMDGPU kernel driver | | NVIDIA | nvidia-open, nvidia-utils, nvidia-prime, libva-nvidia-driver | | Intel | vulkan-intel, libva-intel-driver, intel-media-driver, vpl-gpu-rt | | All | mesa, vulkan-mesa-layers, libvdpau-va-gl |

For compute workloads (ML training, GPGPU, scientific simulation), you need the vendor compute stack — ROCm for AMD, CUDA libraries for NVIDIA, oneAPI for Intel. These live inside Distrobox containers, not on the host.

---

Choosing Your GPU Compute Stack

| GPU | Driver (host) | Compute API | Recommended path | |---|---|---|---| | NVIDIA (RTX / Tesla / H-series) | nvidia-open | CUDA | Distrobox with nvidia/cuda image | | AMD (RX 6000 / RX 7000 / Instinct) | vulkan-radeon / AMDGPU | ROCm / HIP | Distrobox with rocm/dev-ubuntu image | | Intel (Arc, Xe, integrated) | vulkan-intel | oneAPI / SYCL | Distrobox with intel/oneapi-basekit | | Any (portability) | any above | OpenCL | Available in all three vendor images |

---

Verify Your GPU

Before setting up a compute container, confirm your GPU is visible to the host.

# All vendors — check kernel-detected GPUs
lspci | grep -E "VGA|3D|Display"

# NVIDIA — check driver and GPU visibility
nvidia-smi

# AMD — check kernel driver is loaded
lsmod | grep amdgpu
ls /dev/dri/

# Intel — check device nodes
ls /dev/dri/

---

AMD ROCm via Distrobox

ROCm is AMD's open-source GPU compute stack — the equivalent of CUDA for AMD hardware. The host has the AMDGPU kernel driver and Vulkan driver installed; the ROCm userspace compute libraries live inside a Distrobox container.

# Create a ROCm-capable container using AMD's official image
distrobox create --name rocm-dev \
  --image rocm/dev-ubuntu-22.04:latest \
  --additional-flags "--device=/dev/kfd --device=/dev/dri --group-add video --group-add render"

distrobox enter rocm-dev

# Verify ROCm can see your GPU
rocminfo
rocm-smi

# Install PyTorch for ROCm
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.0

# Verify GPU access
python3 -c "import torch; print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"

# Install TensorFlow for ROCm
pip install tensorflow-rocm

The --device=/dev/kfd --device=/dev/dri flags give the container access to the GPU compute and render devices from the host. The --group-add video --group-add render flags ensure the container user has the right group permissions.

The container lives in the @containers Btrfs subvolume and survives every OS update. Your entire ROCm ML environment is independent of the host OS.

ROCm Environment Variables

# Target a specific GPU by index (multi-GPU systems)
export ROCR_VISIBLE_DEVICES=0

# Force ROCm to use a specific architecture (e.g. gfx1100 for RX 7900 XT)
export HSA_OVERRIDE_GFX_VERSION=11.0.0

# Enable HIP kernel timing / verbose logging
export AMD_LOG_LEVEL=3

ROCm via Apptainer (HPC clusters)

If you are running workloads on a cluster, Apptainer's --rocm flag injects the host ROCm runtime into the container automatically:

apptainer exec --rocm rocm_pytorch.sif python3 train.py

# Build a ROCm-capable SIF from AMD's Docker image
apptainer build --fakeroot rocm_pytorch.sif docker://rocm/pytorch:latest

See Apptainer on Shani OS for full cluster workflow details.

---

NVIDIA CUDA via Distrobox

The host has nvidia-open and nvidia-utils installed. The CUDA development toolchain (nvcc, cuDNN, etc.) lives inside a Distrobox container using NVIDIA's official CUDA base images.

# Verify the host driver is working
nvidia-smi

# If using raw Podman (not Distrobox) for NVIDIA, you must first generate the CDI spec:
# sudo pacman -S nvidia-container-toolkit
# sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
# Then use: podman run --device nvidia.com/gpu=all --security-opt=label=disable ...
# Distrobox handles this automatically — no CDI setup required.

# Create a CUDA-capable container
distrobox create --name cuda-dev --image nvidia/cuda:12.3.0-devel-ubuntu22.04
distrobox enter cuda-dev

# Install PyTorch with CUDA 12.1
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# Install common ML stack
pip install tensorflow transformers datasets accelerate wandb

# Verify GPU access
python3 -c "import torch; print(torch.cuda.device_count(), torch.cuda.get_device_name(0))"

CUDA via Apptainer

apptainer exec --nv pytorch_latest.sif python3 train.py

With --nv, Apptainer injects the host NVIDIA runtime into the container — no driver bundling required in the SIF image.

---

Intel oneAPI via Distrobox

Intel Arc discrete GPUs and Intel integrated graphics (Xe architecture) support GPU compute via Intel's oneAPI toolkit. The host has vulkan-intel and the Intel media drivers; oneAPI itself runs in a container.

# Create an Intel compute container
distrobox create --name intel-dev \
  --image intel/oneapi-basekit:latest \
  --additional-flags "--device=/dev/dri"

distrobox enter intel-dev

# Verify Intel GPU visibility
sycl-ls

# Run a SYCL sample
cd /opt/intel/oneapi/samples && icpx -fsycl vector_add.cpp -o vector_add && ./vector_add

---

HIP — Write Once, Run on AMD and NVIDIA

ROCm's HIP layer provides a CUDA-compatible programming model that compiles for both AMD and NVIDIA hardware. If you are writing new GPU code or porting existing CUDA code, HIP gives you cross-vendor portability.

# Inside your ROCm Distrobox container:

# Convert a CUDA source file to HIP
hipify-perl my_kernel.cu > my_kernel.hip.cpp

# Compile for AMD GPU
hipcc my_kernel.hip.cpp -o my_kernel

# The same source compiles for NVIDIA with nvcc after hipify

HIP code uses hip:: namespaces and hipMalloc/hipMemcpy APIs which map directly to the underlying vendor runtime at compile time.

---

PyTorch / JAX / TensorFlow via Nix

For pure-Nix ML workflows without containers — useful for quick CPU-based experimentation or when you want everything managed declaratively. Note that GPU-accelerated Nix ML paths require matching the Nix-packaged ROCm or CUDA version against your hardware and host driver, which is more involved than the Distrobox approach. For most GPU work, Distrobox containers are more reliable.

# NVIDIA path — uses pre-built CPU binaries (GPU requires matching CUDA version)
nix-shell -p python311 python311Packages.torch-bin

# Project-specific environment — create a shell.nix

# shell.nix for a CPU-based PyTorch project
{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
  buildInputs = with pkgs; [
    python311
    python311Packages.torch-bin
    python311Packages.numpy
    python311Packages.pandas
    python311Packages.matplotlib
    jupyter
  ];
}

nix-shell  # enter the environment
jupyter lab

For GPU-accelerated Nix environments, the Distrobox container approach using vendor images is recommended — it handles library versioning against the installed driver automatically.

---

Jupyter with GPU Access

Via Distrobox (recommended for GPU-accelerated notebooks):

# CUDA notebooks
distrobox enter cuda-dev
pip install jupyter
jupyter lab --ip=0.0.0.0 --no-browser
# Access at http://localhost:8888

# ROCm notebooks — same pattern
distrobox enter rocm-dev
pip install jupyter
jupyter lab --ip=0.0.0.0 --no-browser

Via Nix (CPU / no GPU acceleration without additional setup):

nix-shell -p jupyter python311 python311Packages.numpy python311Packages.matplotlib
jupyter lab

Via Flatpak:

flatpak install flathub org.jupyter.JupyterLab

---

Monitoring GPU Utilisation

# AMD — requires entering the ROCm container, or using host kernel tools
radeontop                   # real-time AMD GPU stats (host)
# rocm-smi is available inside the rocm-dev Distrobox container

# NVIDIA
nvidia-smi                  # NVIDIA GPU monitor (host)
nvidia-smi dmon             # continuous monitoring

# Intel
sudo intel_gpu_top          # Intel GPU utilisation (host)

# MangoHud (gaming + compute) — overlays GPU stats on screen
MANGOHUD=1 python3 train.py

---

Resources

Shani OS Troubleshooting Guide — when things go wrong
The Shani OS Software Ecosystem — what to use for each type of software
Shani OS for Researchers and HPC — reproducibility stack: Apptainer, Nix, DVC, cluster workflows
Apptainer on Shani OS — HPC containers with --nv and --rocm GPU passthrough
Virtual Machines on Shani OS — GPU passthrough via VFIO for dedicated VM compute
Gaming on Shani OS — GPU setup, drivers, and gaming-oriented GPU configuration
ROCm documentation — AMD's official compute stack reference
HIP porting guide — migrating CUDA code to HIP/ROCm
Telegram community

---

Built in India 🇮🇳 · Immutable · Atomic · Zero Telemetry

Latest Posts