phoebus_os/docs/adr/0012-nvidia-driver-policy.md
Phoebus b4a6b5df80
Some checks are pending
ci / test (push) Waiting to run
feat(nvidia): hardware-driven driver policy, installed integration and live payload
- resolve the NVIDIA branch and kernel-module flavor from PCI device IDs
  (Turing+ open modules; Maxwell/Pascal/Volta proprietary, masked below 595;
  Fermi/Kepler fall back to nouveau, reported; unknown IDs reported)
- capture pci_id/pci_address in the probe; drive VIDEO_CARDS, package.use,
  package.accept_keywords, package.mask, modprobe.d and dracut from the plan,
  preserving hybrid Intel/AMD
- add the nvidia_drivers install stage (after the kernel, before boot) that
  emerges the driver, regenerates the initramfs, records /etc/phoebus/nvidia and
  fails the install on failure; sys update rebuilds modules and the initramfs
- refuse AI changes to x11-drivers/nvidia-drivers
- live ISO: signed non-free components, stable-kernel pin (testing 7.2 breaks
  the 550 driver), DKMS build for the exact kernel, gated pre-greetd load,
  fail-safe entry, gpu-report, and a --verify-live-nvidia build gate that runs
  on cached rootfs builds and fails on any missing component
- docs: ADR 0012, STATUS/CHANGELOG/HANDOFF, live README
- also carries the pending OpenCode model-variant/streaming changes and fixes
  the GUI bridge install thread join that could hang
2026-09-22 21:53:28 -07:00

6.9 KiB

ADR 0012: NVIDIA accelerated-driver policy and payload

  • Status: Accepted
  • Date: 2026-09-23

Context

Phoebus must run NVIDIA's official accelerated driver stack on both targets:

  • the installed Gentoo system, through Portage's x11-drivers/nvidia-drivers;
  • the Debian-based live ISO, without downloading or compiling a driver at boot.

NVIDIA's Linux driver has three relevant regimes (verified 2026-09 against NVIDIA/Gentoo and the PCI ID database):

  • Turing (GTX 16xx / RTX 20xx) and newer can use NVIDIA's open kernel modules (kernel-open). The open modules are recommended from 560 and mandatory from 595; Blackwell (RTX 50xx) requires them.
  • Maxwell, Pascal and Volta cannot use the open modules and need the proprietary modules from a branch that still supports them.
  • Fermi, Kepler and older have no security-supported proprietary branch; the in-tree nouveau driver is the honest choice.

The prior code classified GPUs only by vendor name and set VIDEO_CARDS="nvidia" for any NVIDIA device, with no branch/flavor decision and no live payload. This is unsafe: it can install a driver that cannot drive the GPU, or claim success while the GPU is unserved.

Decision

1. Classify from the PCI device ID, not marketing names

A generated table (phoebus_core::nvidia_devices, produced by scripts/generate-nvidia-devices.py from the public PCI ID database) maps an NVIDIA display device ID to its chip generation via the device's NVIDIA codename. The probe records vendor:device and the PCI address for each VGA/3D/Display controller. Unknown IDs are reported as unknown and never claimed as supported.

2. Deterministic branch/flavor selection

One driver install covers every adapter, so the branch must serve the oldest supported GPU present:

Generation Branch Module flavor Portage
Turing / Ampere / Ada / Hopper / Blackwell current open (kernel-open) x11-drivers/nvidia-drivers ~amd64
Maxwell / Pascal / Volta proprietary proprietary (-kernel-open) mask >=x11-drivers/nvidia-drivers-595
Fermi / Kepler and older none (nouveau) — no NVIDIA package

Hybrid Intel/AMD GPUs keep their VIDEO_CARDS entries; the NVIDIA decision only adds nvidia, or nouveau when the card is unserved.

3. The AI cannot change it

The driver branch and module flavor are compatibility/security decisions. There is no AI action for them, set_package_use changes touching x11-drivers/nvidia-drivers are refused by the fixed-policy check, and the advisor prompt states the rule. The model may explain the choice, never override it.

4. Installed Gentoo integration

  • Scoped package.use (dist-kernel, kernel-open/-kernel-open), a scoped ~amd64 keyword for the driver atom, and a scoped package.mask for the legacy branch.
  • A modprobe.d policy (nouveau blacklist, nvidia_drm.modeset=1, nvidia-uvm softdep) and a dracut force_drivers fragment, both written before the kernel is built.
  • A dedicated nvidia_drivers install stage runs after the kernel and before boot: it emerges the driver and regenerates the initramfs so the modules and firmware are embedded. Its failure fails the install; the resolved choice is recorded in /etc/phoebus/nvidia and shown in the plan.
  • sys update system runs @module-rebuild and kernel-install add-all on NVIDIA systems, so a kernel update rebuilds the modules and regenerates the initramfs before the next boot.

5. Live ISO payload

  • The signed Debian contrib/non-free/non-free-firmware components supply the driver. Debian 13/testing ships branch 550.163.01, whose proprietary modules cover Maxwell through Ada/Hopper. Blackwell is unsupported by that branch; on live it falls back to nouveau, reported, never silently claimed.
  • The live kernel is pinned to Debian stable (trixie, 6.12 LTS). Testing's current kernel (7.2) removed the strncpy declaration the 550 driver still uses, so nvidia-kernel-dkms cannot build against it; the rolling kernel metas are filtered out and the exact stable kernel/headers/kbuild packages are resolved from trixie and pinned in an apt setup hook.
  • livefs-hook.sh builds the DKMS modules for that exact kernel and fails if any module is missing. Debian renames them nvidia-current*; the hook and the verifier accept either that or the upstream names.
  • build.sh runs --verify-live-nvidia on every build (including PHOEBUS_REUSE_ROOTFS=1): it checks modules, userspace libraries, nvidia-smi, EGL external-platform manifests, firmware, the gated load script and the nouveau blacklist, and records the bundled driver version, module flavor and kernel in /etc/phoebus/nvidia. A missing or mismatched component fails the build.
  • Nouveau blacklisting and module options come from Debian's own nvidia-kernel-common. Its unconditional modules-load entry is removed in favour of a gated service that loads the driver before greetd/Mango only when an NVIDIA display device is present, so Intel/AMD/virtual GPUs keep their normal boot and no failed unit is recorded. The documented fail-safe entry disables modesetting and blacklists the NVIDIA modules.
  • Debian live packages and configuration never enter the Gentoo target.

A minimal live rootfs was built and its NVIDIA payload verified this session (driver 550.163.01, kernel 6.12.107+deb13-amd64, modules nvidia-current*).

Consequences

  • The live image grows by the NVIDIA driver, firmware and userspace payload.
  • The live branch (550) cannot accelerate Blackwell; the installed Gentoo system can (open modules from the current branch). This asymmetry is documented.
  • The live kernel is held at Debian stable rather than testing. This is deliberate: it is both the project's stability preference and the only way the Debian 550 driver builds. It must be revisited when Debian ships a driver whose kernel support matches its testing kernel.
  • The installed system may compile the driver from source; the dist-kernel flag plus @module-rebuild keep it coherent across kernel updates.
  • The generated device table must be regenerated when NVIDIA releases new hardware; unknown IDs degrade safely to "unknown/nouveau" rather than a wrong claim.
  • No NVIDIA-accelerated boot has been observed: this host has no NVIDIA GPU and QEMU's virtual GPU cannot establish it. A minimal live rootfs with the driver was built and its payload verified, but the full ISO and the live/installed NVIDIA gates are unverified here. This ADR records the design, not a claim of working hardware acceleration.

Non-goals

  • NVIDIA's standalone .run installer (never; packages only).
  • Bundling multiple driver branches in one live rootfs (they conflict).
  • Forcing closed modules onto hardware that needs open modules, or vice versa.
  • A global ~amd64 or a global dist-kernel switch.