phoebus_os/docs/adr/0006-package-lanes.md
Phoebus 297f3201ff
Some checks are pending
ci / test (push) Waiting to run
feat(sys): hardware-aware lane selection and new lanes
Replace the fixed category routing (ADR 0006) with a shared, inspectable,
hardware- and preference-aware selection policy over resolved candidates from
Portage (source/binary/vendor), Nix, Flatpak, Snap, AppImage and the Phoebus
overlay. The policy keeps base components under Portage, separates hard
constraints from preferences, honours and persists explicit choices, errors
instead of silently switching lanes, and reuses an already-installed lane.

- phoebus-core::selection: the pure policy (16 tests: base-system, low-memory
  vs high-core, sandbox requirement, GPU/arch/storage rejection, explicit
  preference, unavailable backend, reproducibility, render).
- phoebus-executor::chooser: resolves real metadata and hardware facts.
- phoebus-executor::{snap,appimage,ebuild}: new lanes. Snap detects daemon and
  confinement and installs app-containers/snapd through Portage; AppImage
  adopts images with a pinned SHA-256, checks the ELF architecture without
  executing, integrates a desktop entry and preserves user data on removal; the
  custom ebuild overlay binds approval to the ebuild content digest and builds
  through Portage's sandbox.
- Typed actions choose_package / set_package_preference / snap_* / appimage_* /
  custom_ebuild_* with strict validators; the assistant prompt and schema are
  reconciled (no category routing; custom ebuilds are untrusted).
- Installed sys: the installer locates the trusted static binary, copies it to
  /usr/local/bin/sys and proves it runs in the target chroot before the
  recovery snapshot, failing closed if absent; the VM harness fetches sys.
- docs/SYS.md: command/action reference and acceptance matrix; ADR 0006 revised.

Host gates pass: fmt, clippy (no new warnings), workspace tests, make test,
release and static sys builds. VM lane journeys remain pending (see HANDOFF).
2026-09-23 18:35:26 -07:00

12 KiB

ADR 0006: Package lanes and hardware-aware software selection

  • Status: Accepted, revised 2026-09-24
  • Date: 2026-09-20 (revised 2026-09-24)

Context

Phoebus is built on Gentoo/Portage and must stay a thin layer over it. There is a real need for binary and sandboxed options for non-core software without maintaining a private Gentoo binhost or letting a second package manager claim ownership of the system.

The original decision (2026-09-20) adopted three lanes and a fixed category routing: system/compiler/driver → Portage; GUI → Flatpak; per-project/dev → Nix. In practice that routing is wrong. A given application can be a good fit for different lanes on different machines and for different users:

  • a GUI application available both as a cheap, well-integrated Gentoo source build and as a heavier Flatpak might be better from source on a high-core workstation with a matching USE set, and better as a Flatpak on a low-RAM laptop or when a sandbox is required;
  • a developer tool may be best from Nix for a pinned, reproducible, per-project version, and best from Portage when it is already installed or cheap to compile;
  • some software has no maintained candidate at all and needs a local ebuild.

Category is therefore a hint, not a decision. This ADR revises the decision to a shared, inspectable, hardware- and preference-aware selection policy, and extends the lane set.

Options considered for the selection problem: fixed category routing (rejected), a fully scripted per-application table (rejected: opaque, unmaintainable, still wrong on other hardware), and a shared deterministic policy consuming resolved backend metadata (adopted).

Relevant facts established:

  • The official Gentoo binhost is a single non-systemd (hardened) baseline, so systemd/desktop packages mostly compile from source. We will not maintain a private binhost.
  • nix is not in the Gentoo main tree, so Phoebus ships and manages it itself.
  • Nix is strong for reproducible, multi-version developer environments and poor for GPU/Wayland desktop apps on a non-NixOS host (nixGL, Electron sandbox).
  • Flatpak is already designed into Phoebus and is the right fit for many sandboxed GUI apps.
  • Snap can be added through Gentoo's app-containers/snapd package (systemd is mandatory, AppArmor recommended; verified against the Gentoo wiki). It brings channel/refresh semantics and its own confinement model.
  • AppImages are single executable files with no built-in sandbox; they are useful when no maintained package exists, but must be managed honestly.
  • A local ebuild is occasionally the only correct answer; it is executable untrusted code and needs an explicit, isolated build path.

Decision

Adopt lanes with strict ownership plus a single shared selection policy used by both the deterministic sys front-end and the assistant.

Lane Owns / is suitable for Availability
Portage source (authoritative) kernel, drivers, systemd, Mesa, PipeWire, core libraries, boot stack, system services, compilers/toolchain; cheap, well-integrated source builds always
Portage binary (official binhost) packages whose USE matches the signed official binhost and whose local compile cost is high with little practical benefit when a binhost is configured
Portage vendor (-bin/-launcher) vendor blobs (google-chrome, steam-launcher, discord-bin, …) vendor ebuild present
Flatpak sandboxed graphical applications; multi-distro binaries always (default desktop)
Nix developer tools, alternate versions, reproducible per-project environments always, user-level use
Snap applications distributed as snaps, when the user wants the Snap Store and its channel/refresh model opt-in; app-containers/snapd through Portage
AppImage a single-file application with no maintained package, adopted with pinned integrity opt-in, managed inventory
Custom ebuild a maintained package that is unsuitable or absent and cannot be met otherwise Phoebus overlay, approval-gated

Portage's ownership is unchanged: Snap, AppImage and the custom overlay may never provide kernel, drivers, Mesa, PipeWire, systemd, core libraries or boot. The custom overlay is compiled by Portage; Snap and AppImage are explicitly non-authoritative.

The selection policy

The policy lives in phoebus-core::selection and is pure: it consumes already-resolved candidates (real backend metadata gathered by phoebus-executor::chooser) plus hardware facts and user preferences, and returns a reproducible decision with the reasons alternatives were rejected. Priority order:

  1. Hard compatibility constraints beat preferences. Architecture mismatch, a required sandbox a lane cannot provide, insufficient storage, a required proprietary artifact, GPU incompatibility, or an unavailable backend remove a candidate regardless of preference.
  2. Base components stay Portage. A base-system request may only resolve to a Portage lane. This is the ownership rule above.
  3. Explicit compatible user choices win, and are persisted. A per-app or global preference is honoured when a compatible candidate exists; when it does not, selection fails with an explanation and never silently switches lane.
  4. No automatic category routing. Suitability is decided from candidate evidence: measured/estimated compile time, download and install size, shared runtime cost, sandbox strength, Wayland and desktop integration, provenance, update reliability, permissions, and the actual hardware (cores, RAM, free storage, GPU, Wayland session).
  5. No silent migration or duplication. A lane that already owns an installed copy is preferred, so an application is not installed twice or moved between lanes behind the user's back.
  6. Honest estimates. Compile times are labelled measured (from the emerge.log history) or estimated; compilation is not assumed to be faster or worth its cost. Unknowns are stated as unknowns.

The review shows the selected package identity, lane, version, provenance, integrity status, sandbox level, sizes, update behaviour and the rejected alternatives with their reasons. The same facts, metadata and preferences always produce the same decision.

Acquisition and integrity

  • Portage source/binary: the existing signed Gentoo verification (ADR 0004).
  • Flatpak: flathub GPG; Snap: store signatures; both are sandboxed.
  • AppImage: a Phoebus-managed record stores the pinned SHA-256, source URL, architecture and size; acquisition is refused without a real pinned hash.
  • Custom ebuild: staged in the Phoebus overlay with a recorded rationale and a SHA-256 of the exact ebuild content; approval binds to that digest and any change invalidates it.

Nix is baked in, but "baked in" describes availability, not authority

  • the Nix toolchain and daemon are shipped and configured by the installer;
  • Portage still installs and updates Nix itself, so the daemon is system-managed rather than a stray curl | sh artifact;
  • no core component, service, library or application depends on Nix being used;
  • ignoring Nix has no effect on the system.

How the pinned, verified Nix bootstrap works without surrendering ownership is recorded in ADR 0008.

Rules

  1. Portage is the only authority over the running system. Nix, Flatpak, Snap, AppImage and the custom overlay never provide kernel, drivers, Mesa, PipeWire, systemd, core libraries or boot.
  2. Base components are Portage-only. The selector enforces this as a hard constraint.
  3. No silent backend switching. A failed install is reported; any replacement plan is resolved and approved through the normal path.
  4. Explicit choices are respected and persisted; incompatible ones error.
  5. Snap and AppImage are honest about their guarantees. Snap confinement is detected, not assumed; classic confinement is a security downgrade and is explicit; AppImage is never described as sandboxed and is never executed just to inspect it.
  6. Removal preserves user data by default. Each lane's removal removes only what Phoebus manages; a Btrfs snapshot does not automatically reverse another lane's state or external side effects, and this is stated.
  7. No arbitrary remote evaluation by default. Nix commands run by the assistant are bounded (no arbitrary flakes or URLs). Snap/Flatpak/AppImage operations use bounded identifiers/URLs and pinned hashes.
  8. The custom overlay is untrusted code. The model may propose an ebuild; staging never runs it. Building happens only through the deterministic broker after approval of the exact digest, inside the target with Portage's build sandbox.

AI integration

The assistant and the deterministic CLI emit the same typed schema. Software selection is a typed, read-only action:

ChoosePackage { name, kind?, backend?, gui?, requires_sandbox? }

The selector reports the lane and the rejected alternatives. Persisting a choice is a separate, audited action:

SetPackagePreference { name, backend? }   // backend unset clears it

Lane-specific mutations (snap_*, appimage_*, custom_ebuild_*) are bounded and validated, and a custom ebuild is staged for review before it can build. The model never gets a shell, and approval binds to the reviewed plan/artifact.

Explicit non-goals

  • Fixed category routing. Rejected: it is wrong on other hardware and for other users; suitability is decided from candidate evidence.
  • A universal pkg install meta-manager that silently picks a source. ChoosePackage is an inspectable decision, not a hidden installer, and it never executes.
  • NixOS-style system configuration.
  • A second full distro (Distrobox, pacman) — rejected: a second rootfs to maintain with weaker isolation than Flatpak.
  • A third-party/private Gentoo binhost — rejected: trust, signature and availability.
  • Homebrew — rejected: not packaged for Gentoo, FHS prefix shadowing.
  • Treating AppImage as a sandbox — rejected: it is not one.

Consequences and risks

  • Every install carries the Nix toolchain and /nix; Snap and AppImage add opt-in stores/inventories with their own CVEs and update cadences. The selector surfaces update reliability per lane.
  • The policy is a transparent heuristic, not a benchmark. It labels measured vs estimated data and reports unknowns; it must not be presented as measured performance.
  • Two trust/update pipelines (gentoo + nixpkgs) exist on every system; the daemon being Portage-managed keeps ownership clear.
  • The custom overlay increases the support surface; it is approval-gated and records why each entry exists.

Binary strategy (reconciled)

The sustainable strategy remains:

  • Portage source or the official signed Gentoo binhost, whichever matches the package's USE. Signature verification stays on (ADR 0004); no private binhost is maintained.
  • Flatpak / Snap for sandboxed applications the official binhost cannot serve.
  • Nix for developer tools, alternate versions and reproducible per-project environments (ADR 0008).
  • AppImage / custom ebuild only when no maintained candidate is suitable, with explicit integrity and an approval gate.

Packages the official OpenRC/hardened binhost cannot satisfy (systemd, polkit, NetworkManager, a desktop USE set) compile from source. That cost is accepted and measured (docs/STATUS.md records the compile times), not hidden behind a new binhost.

Next steps

  1. Wire ChoosePackage into the installed assistant and GUI so a user can see the lane decision and override it without a model.
  2. VM-verify each new lane end to end: Snap install/refresh/remove, AppImage acquire/launch/remove, and one custom-ebuild build.
  3. Add pkgcheck-based QA to the custom-ebuild stage gate.