phoebus_os/docs/VM-BOOT-VERIFICATION.md
Phoebus 25f7381b4a
Some checks are pending
ci / test (push) Waiting to run
feat(desktop): add Mango + DankMaterialShell profile with rollback, assistant and release foundations
- Mango + DankMaterialShell desktop option (desktop.profile = "mango") through
  the GURU overlay with scoped ~amd64 keywords, a sealed session entry point and
  a dwl fallback session (ADR 0009).
- Coherent rollback/recovery: coordinated @ + @var snapshots, fstab and BLS
  root-subvolume rewriting, matching-kernel selection, snapshot recovery entry,
  retention/space gates and sys undo (ADR 0007).
- Installed assistant (sys ask) with an OS-level systemd sandbox boundary and
  fail-closed typed plans; resolved package effects bound to approval.
- Nix build isolation enforced (sandbox, no fallback) plus lifecycle/cleanup.
- Desktop helpers (menu/lock/idle), transaction log, release gates with atomic
  publish and signing helpers, and a Debian-Live appliance VM harness.
2026-09-21 03:46:52 -07:00

402 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Disposable UEFI VM boot verification
This records the first successful end-to-end install and boot of Phoebus OS
against a disposable qcow2 disk. No physical disk was ever touched.
## Exact tested path
Host (Ubuntu, no root required):
- QEMU `qemu-system-x86_64` (extracted locally under `/tmp/opencode/qemudeb`),
OVMF (`OVMF_CODE_4M.fd` + `OVMF_VARS_4M.fd`).
- Target disk: `/tmp/opencode/vm/work/target.qcow2`, 30 GiB, virtio-blk
(`/dev/vda` in the guest).
- Live boot: the reference ISO's own `boot/gentoo` + `boot/gentoo.igz`, with
`-append "root=live:CDLABEL=gentoo-amd64-livegui rd.live.dir=/
rd.live.squashimg=image.squashfs cdroot console=ttyS0,115200"`.
- A host `python3 -m http.server` serves the installer tarball, the pinned
stage3 and the test config to the guest over QEMU user networking.
Phase 1 (install):
1. Boot the live ISO headless; get a root shell on the serial console.
2. Download the installer and test config from the host.
3. Write secrets to `0600` files and run the installer binary (the original
run used the Bash prototype; the harness now serves `phoebus-installer`):
`phoebus-installer --config test.conf --install --yes
--luks-passphrase-file FILE --user-password-file FILE`
4. Stages executed in order: `preflight, storage, stage3, fstab,
portage, base packages, kernel, bootloader, users, services, desktop,
finalize`.
Phase 2 (boot the disk alone, ISO detached):
5. Power off, then start a new QEMU with **no ISO and no `-kernel`**, boot
`order=c`, same OVMF NVRAM.
6. Firmware runs `\EFI\systemd\systemd-bootx64.efi`; systemd-boot boots the
"Gentoo Linux" BLS entry from the ESP.
7. `systemd-cryptsetup` prompts for the LUKS2 passphrase, unlocks
`/dev/vda2`, mounts the Btrfs `@` subvolume as `/`.
8. systemd (PID 1) starts, serial getty appears, user login succeeds.
## Verified results
- Firmware booted the disk's own systemd-boot (Boot000C "Linux Boot Manager").
- LUKS2 passphrase prompt and successful unlock.
- Btrfs subvolumes mounted: `/`, `/home`, `/var`, `/.snapshots`, `/swap`
(later fix), ESP at `/efi`.
- `cat /proc/1/comm` -> `systemd`.
- `systemd-analyze`: 1.442s kernel + 5.755s initrd + 2.942s userspace.
- Login as `phoebus` with the configured password; groups:
`wheel audio video render input plugdev power network libinput`.
- Networking up (`enp0s2 10.0.2.15/24`).
- `systemctl --failed`: exactly one unit, `swap-swapfile.swap` (fixed after
this run; see below).
## Bugs found and fixed during this work
1. **Subvolumes created after mounting `@`** - `storage_prepare` mounted
`subvol=@` before `@` existed (mount exit 32). Added `btrfs_create_layout`
(mount top-level, create subvols, unmount, then mount layout) and a
regression test (`tests/test_btrfs_order.sh`).
2. **Generated fstab overwritten by stage3** - fstab/crypttab were generated
during storage, then the stage3 tarball replaced `/etc/fstab`. Moved
generation to a new `fstab_generate` stage after extraction.
3. **`/var/tmp` absent in the target** - systemd's `kernel-install` needs a
temporary directory; `/var/tmp` was missing. The fstab stage now creates
`/var/tmp`, `/tmp`, `/root`, `/home`, and chroot commands set
`TMPDIR=/tmp`.
4. **installkernel needs `uki_generator` even for BLS** - with an explicit
`/etc/kernel/install.conf`, `05-check-config.install` always requires a
`uki_generator`. BLS now writes `uki_generator=none`.
5. **dracut could not build the `crypt` module** - the target lacked
`sys-fs/cryptsetup`. Added `sys-fs/cryptsetup` and `sys-fs/btrfs-progs` to
the base package set.
6. **No `systemd-cryptsetup`** - `sys-apps/systemd` was built without the
`cryptsetup` USE flag, so the encrypted root could not be unlocked. Added
`cryptsetup` (and `policykit`) to the systemd package.use entry.
7. **Kernel command line used unnamed LUKS** - `rd.luks.uuid` yields
`luks-<uuid>`, not the configured name. Switched to
`rd.luks.name=<uuid>=<name>`.
8. **Dracut UKI did not embed the command line** - the dracut `uki_generator`
produced a UKI with no `.cmdline`, so it booted with no `root=`. Deferred
UKI in favour of a reliable BLS entry carrying `options`. UKI remains
future work.
9. **fstab did not mount `@swap`** - the swapfile unit failed because `/swap`
was never the `@swap` subvolume. Added the `subvol=@swap` fstab entry.
10. **`local root="$1" dir="$root/..."` class bug** - reintroduced in
`boot_write_dracut_conf`; split into two `local` statements. This class of
bug is now covered by the regression suite.
11. **Wrong package atoms** - `gui-apps/fuzzel` (not in Gentoo) -> `gui-apps/wofi`,
`app-misc/brightnessctl` (not packaged) removed,
`app-crypt/gnome-keyring` -> `gnome-base/gnome-keyring`,
`gui-apps/polkit-gnome` -> `gnome-extra/polkit-gnome`.
12. **Package policy compiled too much** - unknown packages defaulted to
source. `balanced` now compiles cheap builds and uses signed Gentoo
binaries for everything else, which cut installer time substantially.
## Rust installer parity (same VM path)
The Rust installer (`installer-rs`) reproduced the entire path on a fresh
disposable qcow2 disk using the static binary
`phoebus-installer` (`x86_64-unknown-linux-gnu`, crt-static):
- Same config (legacy flat format), same pinned stage3 URL + SHA-256.
- `INSTALL_RC=0`; stages: storage, stage3, fstab, portage, base packages,
kernel, bootloader, users, services, desktop (none), finalize.
- Policy decisions matched Bash: `sudo`/`networkmanager`/`polkit` binary,
`cryptsetup`/`btrfs-progs` source.
- The installer recorded **81 then 97 measured compile times** from Portage's
`emerge.log` (real data, not estimates).
- `NO_SECRET_LEAK` from the log secret scan.
Booted with the ISO detached:
- UEFI firmware ran systemd-boot; BLS entry booted.
- LUKS2 unlock, Btrfs `@` root, `/home`, `/var`, `/swap`, ESP at `/efi`.
- `cat /proc/1/comm` -> `systemd`.
- `systemctl is-system-running` -> **running** (not degraded).
- `systemctl --failed` -> **0 loaded units listed**.
- `systemd-analyze`: 1.378s kernel + 5.238s initrd + 4.114s userspace =
10.730s.
- Login as `phoebus` succeeded; networking up (`10.0.2.15/24`).
This is the point at which the Rust installer satisfies the parity criteria and
may become the primary installer. The Bash implementation remains the
reference/fallback until the desktop milestone is also verified.
## Desktop and Flatpak (Rust installer, verified)
The Rust installer, running the desktop profile, installed the full Wayland
desktop and Flatpak successfully (`INSTALL_RC=0`; 326 measured compile times
recorded). Key change: a desktop install now selects the supported stable
Gentoo profile `default/linux/amd64/23.0/desktop/systemd` instead of
reconstructing X/Wayland/dbus USE by hand from the minimal systemd profile.
This resolved the earlier `cairo[X]`/`libglvnd[X]`/`libxkbcommon[X]`
conflicts, and `sys-fs/{btrfs-progs,cryptsetup} -man` avoided a
Sphinx/Pillow circular dependency.
Booted with the ISO detached and logged in:
- `systemctl is-system-running` -> `running`; `systemctl --failed` -> 0 units.
- `flatpak --version` -> **Flatpak 1.18.1**; `flatpak remotes` -> **flathub
configured**.
- Binaries present and runnable: `dwl 0.7`, `Waybar v0.15.0`, `wofi 1.4.1`,
`mako`, `swaybg`, `pipewire`, `wireplumber`, `grim`, `wl-clipboard`.
Session verification. The VM's serial session has **no logind seat**, so
wlroots' DRM backend correctly refuses (`Failed to start a DRM session`). This
is a headless-harness limitation, not an installer defect. Using
`WLR_BACKENDS=headless WLR_RENDERER=pixman`:
- `dwl` started (pid observed), created the `wayland-0` socket.
- `mako`, `waybar`, and `swaybg` started and connected (`swaybg` found output
`HEADLESS-1`).
- `wl-copy`/`wl-paste` round-tripped `phoebus-clip`.
- `grim /tmp/shot.png` produced a **14401-byte PNG**, proving the compositor
renders and captures.
- `wpctl status` reported the PipeWire graph; `systemctl --user --failed` -> 0.
- `waybar` logged a non-fatal warning about a missing power-menu file and
power-profiles-daemon.
The installer writes `/usr/local/bin/phoebus-session`, which starts waybar,
mako and swaybg and then execs dwl, with a software-render fallback when no
render node is present.
## Official harness on the static binary (Bash migration, Phase 2)
The committed harness (`installer/tests/vm/serial-driver.py` +
`installer/tests/vm/install.sh`) now drives the **static Rust binary** and no
longer runs the Bash installer. Re-verified end to end on a fresh disposable
qcow2 (24 GiB, virtio-blk `/dev/vda`):
```sh
# Build:
cd installer-rs && RUSTFLAGS="-C target-feature=+crt-static" \
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
# Install in the disposable VM, then boot the disk detached:
python3 installer/tests/vm/serial-driver.py install
python3 installer/tests/vm/serial-driver.py boot
```
Install (`INSTALL-RC=0`):
- The guest fetched `/tmp/phoebus-installer` and printed
`phoebus-installer 0.1.0` (the Rust binary, not Bash).
- Stages completed in proven order: `storage_prepare`, `stage_install`,
`fstab_generate`, `portage_configure`, `packages_base`, `kernel_install`,
`boot_install`, `users_configure`, `services_configure`, `desktop_install`
(none), `finalize`.
- Policy matched Bash: `crypto`/`btrfs-progs` source build, `sudo`/
`networkmanager`/`polkit` binary.
- `NO_SECRET_LEAK`; the run powered off cleanly.
Boot with the ISO detached (`serial-driver.py boot`):
- UEFI firmware ran `\EFI\systemd\systemd-bootx64.efi` (Boot0002 "Linux Boot
Manager"); the BLS "Gentoo Linux" entry booted.
- `systemd-cryptsetup` prompted for the LUKS2 passphrase and unlocked
`phoebus-root (cryptroot)`.
- Login as `phoebus` succeeded.
- `cat /proc/1/comm` -> `systemd`.
- `systemctl is-system-running` -> **running**.
- `systemctl --failed` -> **0 loaded units listed**.
- Networking: `enp0s2 UP 10.0.2.15/24`.
- Mounts: `/` (btrfs `@`), `/home`, `/var`, `/.snapshots`, `/swap`, ESP
`/efi`; `vda2` is `crypto_LUKS`.
This satisfies the Phase 2 exit gate of `docs/BASH-MIGRATION.md`: the only
proven boot path no longer depends on the retired Bash installer.
## Snapshots and safe updates (four-workstream session)
Verified on the installed Btrfs system from the static-binary harness, driving
the new `sys` binary fetched into the guest:
- `sys history` before -> `no transaction history`.
- `sys -y --action '{"action":"create_snapshot","operation":"vm test"}'` ->
`protective snapshot: @-20260920T114917-vm-test`.
- `sys history` after -> `@-20260920T114917-vm-test Committed vm test`.
- `btrfs subvolume list /` shows `@snapshots/@-20260920T114917-vm-test` as a
real subvolume (read-only snapshot).
- `sys -y prepare-updates` fetched distfiles successfully; `--buildpkgonly` was
refused by Portage with `--buildpkgonly requires all dependencies to be
merged`, which the installer now records as `status: Ready` with the reason
(`distfiles fetched; binary prebuild skipped: ...`) instead of a hard failure.
- `sys inspect-updates` reports the state and timestamp.
Unit regression tests cover the failure paths that a VM cannot easily force:
corrupt snapshot history is an error, and a high-risk action aborts when the
protective snapshot fails.
## Desktop session on an installed VM
A full dwl desktop install was built with the static Rust binary
(`DESKTOP_PROFILE=dwl`, `FLATPAK=no`) and booted with the ISO detached:
`INSTALL_RC=0`, all stages, `NO_SECRET_LEAK`, `systemctl is-system-running` ->
`running`, `0 failed units`.
Running the installed `/usr/local/bin/phoebus-session` under the headless
wlroots backend (the serial VM has no logind seat):
- the `wayland-0` socket is created before any helper starts;
- after startup the process tree is `dwl`, `waybar`, `mako`,
`swaybg -c #1e1e2e` and `/usr/libexec/polkit-gnome-authentication-agent-1`
(the polkit agent, which the previous script never started);
- dwl reports the `HEADLESS-1` output;
- after `dwl` exits, **all session processes are reaped**.
Installed and on `PATH`: `dwl`, `waybar`, `mako`, `swaybg`, `kitty`, `wofi`,
`Xwayland`. The polkit agent lives at
`/usr/libexec/polkit-gnome-authentication-agent-1`; the session script's
fallback finds it because it is not on `PATH`.
This is headless verification. A physical VT / real logind seat remains to be
exercised interactively (`docs/DEVELOPMENT.md` -> "Physical VT / real-seat
check").
## Signed binary reuse analysis
The desktop transaction's installer log (`emerge` grouping) shows the policy
already requests signed binaries for the expensive set:
- source group: `sys-fs/cryptsetup sys-fs/btrfs-progs`;
- binary groups: base tools, `gentoo-kernel-bin`/firmware, and the dwl/waybar/
wofi/mako/sway*/gtk-layer-shell/xwayland set.
The desktop install in this session took ~1h49m wall time (12:01–13:50 UTC) on
8 vCPUs. Most of that is Portage falling back to source because the OpenRC/
hardened binhost cannot satisfy the systemd/desktop USE set, even though the
policy requested binaries. Two comparable data points are now recorded: the
base-only install took ~44 min and the desktop install ~1h49m. The code changes
here remove forced flags but cannot raise reuse without a matching binhost, so
no new time saving is claimed.
The remaining source builds are caused by the **official Gentoo binhost being
built for the OpenRC/hardened profile** (`PROFILE: default/linux/amd64/17.0/
hardened`), which cannot satisfy a systemd/desktop USE set. The safe changes
made here are: stop forcing the unused `installkernel uki` flag, remove the
duplicate PipeWire and empty WirePlumber `package.use` lines, and derive the
binhost URI from the selected profile when it is not set. Signature
verification remains enabled. Publishing a matching systemd/desktop binhost is
the remaining step; no time estimate is claimed beyond the recorded ~26 min.
## Debian-Live installer appliance (actual ISO)
An appliance ISO was built **without root** by `installer/live/build.sh`
(Debian-Live 13.7 standard base + `/phoebus` overlay). The full-install
evidence below was recorded with build
`c0105bc9a282fddcbefab42baa577ea99843060f00fa45a75bcbc8358186bfd5`.
A desktop install was re-run and verified with build
`c1846f0d18ed9813e5ef3e699b9110f4fb04ab09a4c5ed800970b1d54971d114`:
- mandatory-AI gate refused the install until `--no-ai` was passed
(`GATE_REFUSED_OK`);
- Nix: `/nix/store/2nhrwv91g6ycpyxvhmvc0xs8p92wp4bk-nix-2.24.9` present,
`nix (Nix) 2.24.9`, `nix-daemon.service` **enabled and active**, and
`nix shell nixpkgs#hello` downloaded from `cache.nixos.org` and printed
`Hello, world!`;
- desktop portals: `xdg-desktop-portal 1.20.4`, `-gtk 1.15.3`, `-wlr 0.7.1`;
- desktop binaries present: `dwl`, `waybar`, `kitty`, `wofi`, `Xwayland`;
- hostname `phoebus`, networking `enp0s2 UP 10.0.2.15/24`, systemd PID 1,
`running`, `0 loaded units listed`.
The current build is
`3ba913ba660fb7436e26e587e9777f1f65ed10dcc9b50d17458ebb3d3a739b5c`: Nix is now
on by default, and `finalize` flushes the target (`sync`) before unmounting.
That build was **not** re-run end to end because the host root filesystem
reached ~98% and the base emerge became pathologically slow. Treat `c1846f0d`
as the verified artifact and `3ba913ba` as the current one.
Firmware boot (UEFI/OVMF, no `-kernel`; the ISO's own grub chain):
- auto-boots the live system on the serial console;
- login `user` / `live` (passwordless sudo);
- the overlaid tools run: `cryptsetup 2.7.5`, `sgdisk 1.0.10`,
`partprobe 3.6`, `mkfs.btrfs 6.14`, `mkfs.fat`.
Full install to a fresh 24 GiB qcow2 driven from the appliance:
- `INSTALL_RC=0`, all stages, `NO_SECRET_LEAK`.
Detached boot of the installed disk (ISO removed):
- systemd-boot -> LUKS2 unlock -> `phoebus` login;
- `systemctl is-system-running` -> `running`; `0 loaded units listed`;
- **Gentoo-only**: `Gentoo Base System release 2.18`, Portage 3.0.82.2,
`no-apt`; systemd PID 1;
- `cryptroot` Btrfs mounted at `/`, `/home`, `/var`, `/.snapshots`, `/swap`,
ESP `/efi`.
This is Bash-free and Gentoo-target-only: the appliance contributes tooling,
never configuration.
### Open items found in this run
- **Resolved:** the earlier `gentoo` hostname anomaly did not recur; the target
boots as `phoebus` with `/etc/hostname` set, and DHCP comes up
(`enp0s2 UP 10.0.2.15/24`) once awaited.
- **Implemented, not re-verified:** `/usr/local/bin/phoebus-session` (and the
exported installer log) were absent after boot in the earlier runs. The
installer wrote and existence-checked them, and the root unmounted cleanly,
but comparison with a boot-time-created file showed the **finalize writes were
not on disk**: Btrfs commits lazily and the harness used `poweroff -f`. The
fix is a `sync` in `finalize` before unmounting (build `3ba913ba`). It has not
been re-run end to end (host disk pressure). Until it is, treat the session
script as **unverified** in the shipped ISO.
## Mango + DankMaterialShell acceptance (NOT RUN)
The `mango` profile (ADR 0009) is implemented and source-tested but has not been
installed or booted. These gates must pass in a **graphical, seat-providing**
disposable VM before the profile is promoted to default. Headless wlroots tests
do not count.
Prerequisites: the `mango` install needs the GURU overlay to sync (git) and
Quickshell/DMS/scenefx to compile; the base install alone is tens of minutes.
1. Fresh install with `DESKTOP_PROFILE='mango'` reaches `systemctl
is-system-running = running` with 0 failed units; `emerge --pretend` shows
only the reviewed `~amd64` atoms.
2. Detached encrypted boot; LUKS unlock; login on a real seat.
3. `phoebus-mango.desktop` starts the session; exactly one compositor (mango),
one shell (dms), one lock/idle (DMS) and one polkit agent are running; no
waybar/mako/swaybg/swayidle/swaylock processes.
4. Panel, launcher (Mod+Space), notifications, tray, and the DMS settings panel
work; the shortcut guide matches the packaged config.
5. Audio output/input selection, volume, mic mute (WirePlumber); network and
Bluetooth controls from the panel.
6. Lock (Mod+Alt+L) and unlock; idle lock/suspend; repeated suspend/resume.
7. Display configuration/scaling and a second monitor if the VM supports one.
8. Clipboard history (Mod+V), screenshots, kitty terminal, file manager, and
`phoebus-assistant` opening `sys ask` (no model required for deterministic
`sys`).
9. An actual Flatpak app: launch, file chooser, and portal screen sharing.
10. Logout reaps DMS and session processes without touching the dwl fallback;
reboot; then boot the `phoebus-dwl.desktop` session and confirm it still
works.
11. Shell-failure handling: kill `dms` and confirm the compositor keeps running
and locking still locks; restart the shell without losing windows.
12. Upgrade from an existing dwl install keeps user configuration
(`~/.config/mango/config.conf`, DMS fragments, `~/.config/waybar`) intact.
13. Idle memory/CPU and startup time measured against the dwl session under
comparable conditions.
Until this list is recorded here with results, the Mango profile is selectable
but not default, and no desktop stability is claimed.
## Known remaining work
- The graphical session was verified with the **headless** wlroots backend
(no logind seat in the serial harness). A physical VT / seat path is
architected (`phoebus-session`) but was not exercised on real hardware.
- NVIDIA proprietary support is architecturally allowed but **NOT VERIFIED**.
- UKI is deferred (BLS is used).
- The installer rebuilds `sys-apps/systemd` from source because
boot/policykit/cryptsetup USE differ from the binhost. Shipping a matching
systemd binpkg is a future optimization.
- Desktop USE still does not match the signed binhost, so the desktop
transaction compiles a large amount from source (~26 minutes for the desktop
package set on 8 vCPUs in the test VM).