phoebus_os/docs/adr/0012-nvidia-driver-policy.md
Phoebus 3b1b54ffb9
Some checks are pending
ci / test (push) Waiting to run
docs: reconcile NVIDIA, verification and harness status
Record the reliability pass across STATUS, HANDOFF, CHANGELOG, ADR 0012,
the live README and the boot-verification log: the deterministic live GPU
decision with a real nouveau fallback, the 580 legacy cap and mixed
open/proprietary incompatibility, the compatibility verifier, the
installed lifecycle fix, the repaired GUI harness, the automated but
unrun rollback gate, and the scoped live sudoers. Withdraw the stale
handoff claim that the gui_bridge tests hang (the workspace suite
passes) and record the installed-target missing-sys gap.
2026-09-23 06:28:30 -07:00

164 lines
8.8 KiB
Markdown

# ADR 0012: NVIDIA accelerated-driver policy and payload
- **Status:** Accepted
- **Date:** 2026-09-23
## Context
Phoebus must run NVIDIA's official accelerated driver stack on both targets:
- the **installed Gentoo system**, through Portage's `x11-drivers/nvidia-drivers`;
- the **Debian-based live ISO**, without downloading or compiling a driver at
boot.
NVIDIA's Linux driver has three relevant regimes (verified 2026-09 against
NVIDIA/Gentoo and the PCI ID database):
- **Turing (GTX 16xx / RTX 20xx) through Hopper** can use NVIDIA's **open
kernel modules** (`kernel-open`). The open modules are *recommended* from 560
and *mandatory* from 595.
- **Blackwell (RTX 50xx)** *requires* the open kernel modules; the proprietary
kernel modules cannot drive it.
- **Maxwell, Pascal and Volta** cannot use the open modules and need the
proprietary modules. NVIDIA's **580 series is the last** to support them; the
next series drops those architectures.
- **Fermi, Kepler and older** have no security-supported proprietary branch;
the in-tree `nouveau` driver is the honest choice.
The prior code classified GPUs only by vendor *name* and set
`VIDEO_CARDS="nvidia"` for any NVIDIA device, with no branch/flavor decision and
no live payload. This is unsafe: it can install a driver that cannot drive the
GPU, or claim success while the GPU is unserved.
## Decision
### 1. Classify from the PCI device ID, not marketing names
A generated table (`phoebus_core::nvidia_devices`, produced by
`scripts/generate-nvidia-devices.py` from the public PCI ID database) maps an
NVIDIA display device ID to its chip generation via the device's NVIDIA
codename. The probe records `vendor:device` and the PCI address for each
`VGA`/`3D`/`Display` controller. Unknown IDs are **reported as unknown** and
never claimed as supported.
### 2. Deterministic branch/flavor selection
One driver install covers every adapter, so the branch is computed **across the
whole device set** rather than assuming the oldest card's branch serves the
newer ones:
| Generation | Branch | Module flavor | Portage |
| --- | --- | --- | --- |
| Turing / Ampere / Ada / Hopper | current | open (`kernel-open`) | `x11-drivers/nvidia-drivers ~amd64` |
| Blackwell | current | open required (`kernel-open`) | `x11-drivers/nvidia-drivers ~amd64` |
| Maxwell / Pascal / Volta | proprietary (580 is the last series) | proprietary (`-kernel-open`) | mask `>=x11-drivers/nvidia-drivers-581` |
| Fermi / Kepler and older | none (nouveau) | — | no NVIDIA package |
A Blackwell GPU (open-only) together with a Maxwell/Pascal/Volta GPU
(proprietary-only) has **no single compatible branch**. The plan reports that
incompatibility explicitly and leaves the adapters on `nouveau` instead of
installing one driver that cannot accelerate one of the cards. Pascal plus a
Turing-class GPU stays proprietary and supported, because the newer generation
accepts either flavor.
Hybrid Intel/AMD GPUs keep their `VIDEO_CARDS` entries; the NVIDIA decision
only adds `nvidia`, or `nouveau` when the card is unserved.
### 3. The AI cannot change it
The driver branch and module flavor are **compatibility/security** decisions.
There is no AI action for them, `set_package_use` changes touching
`x11-drivers/nvidia-drivers` are refused by the fixed-policy check, and the
advisor prompt states the rule. The model may *explain* the choice, never
override it.
### 4. Installed Gentoo integration
- Scoped `package.use` (`dist-kernel`, `kernel-open`/`-kernel-open`), a scoped
`~amd64` keyword for the driver atom, and a scoped `package.mask` for the
legacy branch.
- A `modprobe.d` policy (nouveau blacklist, `nvidia_drm.modeset=1`,
`nvidia-uvm` softdep) and a dracut `force_drivers` fragment, both written
**before** the kernel is built.
- A dedicated `nvidia_drivers` install stage runs **after** the kernel and
**before** boot: it emerges the driver and regenerates the initramfs so the
modules and firmware are embedded. Its failure **fails the install**; the
resolved choice is recorded in `/etc/phoebus/nvidia` and shown in the plan.
- `sys update system` runs `@module-rebuild` and `kernel-install add-all` on
NVIDIA systems, so a kernel update rebuilds the modules and regenerates the
initramfs before the next boot.
### 5. Live ISO payload
- The signed Debian `contrib`/`non-free`/`non-free-firmware` components supply
the driver. Debian 13/testing ships branch **550.163.01**, whose proprietary
modules cover Maxwell through Hopper. **Blackwell is unsupported by that
branch**; on live it falls back to nouveau, reported, never silently claimed.
- The live driver choice is **deterministic and device-based**:
`phoebus-installer --live-gpu-plan` enumerates the PCI display controllers
(class `0x03`, vendor `0x10de`), classifies them against the generated
device table, and decides `load-nvidia`, `nouveau-fallback`, `no-device` or
`disabled`. It never matches on a vendor name alone.
- `nouveau` is blacklisted so it cannot *auto-bind* a supported GPU, but an
unsupported GPU (or a failed accelerated load) explicitly `modprobe`s
`nouveau`, so the documented fallback is real. The fail-safe GRUB entry sets
`nomodeset phoebus.nvidia=off`; the load script and the unit both stand down
on that command line, so recovery intent is authoritative.
- The live policy file `etc/modprobe.d/phoebus-nvidia.conf` is created by
`build.sh` (not only on the installed Gentoo target) and replaces Debian's
unconditional nouveau configuration.
- The **live kernel is pinned to Debian stable (trixie, 6.12 LTS)**. Testing's
current kernel (7.2) removed the `strncpy` declaration the 550 driver still
uses, so `nvidia-kernel-dkms` cannot build against it; the rolling kernel
metas are filtered out and the exact stable kernel/headers/kbuild packages are
resolved from `trixie` and pinned in an apt setup hook.
- `livefs-hook.sh` builds the DKMS modules for that exact kernel and fails if
any module is missing. Debian renames them `nvidia-current*`; the hook and the
verifier accept either that or the upstream names.
- `build.sh` runs `--verify-live-nvidia` on **every** build (including
`PHOEBUS_REUSE_ROOTFS=1`). It is a compatibility gate, not a presence list:
it rejects placeholder modules/libraries, requires every module to be
registered in the live kernel's `modules.dep`, normalises the dpkg revision
and requires the userspace `libnvidia-glcore` and firmware-directory versions
to match the bundled driver, infers and checks the module flavor, requires a
non-empty GSP firmware and the exact `etc/modprobe.d/phoebus-nvidia.conf`,
and keeps every read rooted inside the build root. It records the bundled
driver version, module flavor and kernel in `/etc/phoebus/nvidia`. Any
missing or mismatched component fails the build.
- Nouveau blacklisting and module options come from Debian's own
`nvidia-kernel-common`. Its unconditional modules-load entry is removed in
favour of a gated service that loads the driver before greetd/Mango **only
when an NVIDIA display device is present**, so Intel/AMD/virtual GPUs keep
their normal boot and no failed unit is recorded. The documented fail-safe
entry disables modesetting and blacklists the NVIDIA modules.
- Debian live packages and configuration never enter the Gentoo target.
A minimal live rootfs was built and its NVIDIA payload verified this session
(driver 550.163.01, kernel 6.12.107+deb13-amd64, modules `nvidia-current*`).
## Consequences
- The live image grows by the NVIDIA driver, firmware and userspace payload.
- The live branch (550) cannot accelerate Blackwell; the installed Gentoo system
can (open modules from the current branch). This asymmetry is documented.
- The live kernel is held at Debian stable rather than testing. This is
deliberate: it is both the project's stability preference and the only way the
Debian 550 driver builds. It must be revisited when Debian ships a driver
whose kernel support matches its testing kernel.
- The installed system may compile the driver from source; the `dist-kernel`
flag plus `@module-rebuild` keep it coherent across kernel updates.
- The generated device table must be regenerated when NVIDIA releases new
hardware; unknown IDs degrade safely to "unknown/nouveau" rather than a wrong
claim.
- **No NVIDIA-accelerated boot has been observed**: this host has no NVIDIA GPU
and QEMU's virtual GPU cannot establish it. A minimal live rootfs with the
driver was built and its payload verified, but the full ISO and the
live/installed NVIDIA gates are unverified here. This ADR records the design,
not a claim of working hardware acceleration.
## Non-goals
- NVIDIA's standalone `.run` installer (never; packages only).
- Bundling multiple driver branches in one live rootfs (they conflict).
- Forcing closed modules onto hardware that needs open modules, or vice versa.
- A global `~amd64` or a global `dist-kernel` switch.