phoebus_os/installer/doc/VM-BOOT-VERIFICATION.md
Phoebus 38214ea9e7 chore: import Phoebus OS installer and Rust migration
Initial public import of the proven Bash installer (installer/) and the production Rust workspace (installer-rs/). The reference ISO, build output and local agent transcripts are gitignored.

Verified: installer/tests/run_tests.sh and cargo test --workspace both pass.
2026-09-20 02:07:33 -07:00

177 lines
8.8 KiB
Bash

# Disposable UEFI VM boot verification
This records the first successful end-to-end install and boot of Phoebus OS
against a disposable qcow2 disk. No physical disk was ever touched.
## Exact tested path
Host (Ubuntu, no root required):
- QEMU `qemu-system-x86_64` (extracted locally under `/tmp/opencode/qemudeb`),
OVMF (`OVMF_CODE_4M.fd` + `OVMF_VARS_4M.fd`).
- Target disk: `/tmp/opencode/vm/work/target.qcow2`, 30 GiB, virtio-blk
(`/dev/vda` in the guest).
- Live boot: the reference ISO's own `boot/gentoo` + `boot/gentoo.igz`, with
`-append "root=live:CDLABEL=gentoo-amd64-livegui rd.live.dir=/
rd.live.squashimg=image.squashfs cdroot console=ttyS0,115200"`.
- A host `python3 -m http.server` serves the installer tarball, the pinned
stage3 and the test config to the guest over QEMU user networking.
Phase 1 (install):
1. Boot the live ISO headless; get a root shell on the serial console.
2. Download the installer and test config from the host.
3. Write secrets to `0600` files and run:
`installer/bin/ai-installer --config test.conf --install --yes
--luks-passphrase-file FILE --user-password-file FILE`
4. Stages executed in order: `preflight, storage, stage3, fstab,
portage, base packages, kernel, bootloader, users, services, desktop,
finalize`.
Phase 2 (boot the disk alone, ISO detached):
5. Power off, then start a new QEMU with **no ISO and no `-kernel`**, boot
`order=c`, same OVMF NVRAM.
6. Firmware runs `\EFI\systemd\systemd-bootx64.efi`; systemd-boot boots the
"Gentoo Linux" BLS entry from the ESP.
7. `systemd-cryptsetup` prompts for the LUKS2 passphrase, unlocks
`/dev/vda2`, mounts the Btrfs `@` subvolume as `/`.
8. systemd (PID 1) starts, serial getty appears, user login succeeds.
## Verified results
- Firmware booted the disk's own systemd-boot (Boot000C "Linux Boot Manager").
- LUKS2 passphrase prompt and successful unlock.
- Btrfs subvolumes mounted: `/`, `/home`, `/var`, `/.snapshots`, `/swap`
(later fix), ESP at `/efi`.
- `cat /proc/1/comm` -> `systemd`.
- `systemd-analyze`: 1.442s kernel + 5.755s initrd + 2.942s userspace.
- Login as `phoebus` with the configured password; groups:
`wheel audio video render input plugdev power network libinput`.
- Networking up (`enp0s2 10.0.2.15/24`).
- `systemctl --failed`: exactly one unit, `swap-swapfile.swap` (fixed after
this run; see below).
## Bugs found and fixed during this work
1. **Subvolumes created after mounting `@`** - `storage_prepare` mounted
`subvol=@` before `@` existed (mount exit 32). Added `btrfs_create_layout`
(mount top-level, create subvols, unmount, then mount layout) and a
regression test (`tests/test_btrfs_order.sh`).
2. **Generated fstab overwritten by stage3** - fstab/crypttab were generated
during storage, then the stage3 tarball replaced `/etc/fstab`. Moved
generation to a new `fstab_generate` stage after extraction.
3. **`/var/tmp` absent in the target** - systemd's `kernel-install` needs a
temporary directory; `/var/tmp` was missing. The fstab stage now creates
`/var/tmp`, `/tmp`, `/root`, `/home`, and chroot commands set
`TMPDIR=/tmp`.
4. **installkernel needs `uki_generator` even for BLS** - with an explicit
`/etc/kernel/install.conf`, `05-check-config.install` always requires a
`uki_generator`. BLS now writes `uki_generator=none`.
5. **dracut could not build the `crypt` module** - the target lacked
`sys-fs/cryptsetup`. Added `sys-fs/cryptsetup` and `sys-fs/btrfs-progs` to
the base package set.
6. **No `systemd-cryptsetup`** - `sys-apps/systemd` was built without the
`cryptsetup` USE flag, so the encrypted root could not be unlocked. Added
`cryptsetup` (and `policykit`) to the systemd package.use entry.
7. **Kernel command line used unnamed LUKS** - `rd.luks.uuid` yields
`luks-<uuid>`, not the configured name. Switched to
`rd.luks.name=<uuid>=<name>`.
8. **Dracut UKI did not embed the command line** - the dracut `uki_generator`
produced a UKI with no `.cmdline`, so it booted with no `root=`. Deferred
UKI in favour of a reliable BLS entry carrying `options`. UKI remains
future work.
9. **fstab did not mount `@swap`** - the swapfile unit failed because `/swap`
was never the `@swap` subvolume. Added the `subvol=@swap` fstab entry.
10. **`local root="$1" dir="$root/..."` class bug** - reintroduced in
`boot_write_dracut_conf`; split into two `local` statements. This class of
bug is now covered by the regression suite.
11. **Wrong package atoms** - `gui-apps/fuzzel` (not in Gentoo) -> `gui-apps/wofi`,
`app-misc/brightnessctl` (not packaged) removed,
`app-crypt/gnome-keyring` -> `gnome-base/gnome-keyring`,
`gui-apps/polkit-gnome` -> `gnome-extra/polkit-gnome`.
12. **Package policy compiled too much** - unknown packages defaulted to
source. `balanced` now compiles cheap builds and uses signed Gentoo
binaries for everything else, which cut installer time substantially.
## Rust installer parity (same VM path)
The Rust installer (`installer-rs`) reproduced the entire path on a fresh
disposable qcow2 disk using the static binary
`phoebus-installer` (`x86_64-unknown-linux-gnu`, crt-static):
- Same config (legacy flat format), same pinned stage3 URL + SHA-256.
- `INSTALL_RC=0`; stages: storage, stage3, fstab, portage, base packages,
kernel, bootloader, users, services, desktop (none), finalize.
- Policy decisions matched Bash: `sudo`/`networkmanager`/`polkit` binary,
`cryptsetup`/`btrfs-progs` source.
- The installer recorded **81 then 97 measured compile times** from Portage's
`emerge.log` (real data, not estimates).
- `NO_SECRET_LEAK` from the log secret scan.
Booted with the ISO detached:
- UEFI firmware ran systemd-boot; BLS entry booted.
- LUKS2 unlock, Btrfs `@` root, `/home`, `/var`, `/swap`, ESP at `/efi`.
- `cat /proc/1/comm` -> `systemd`.
- `systemctl is-system-running` -> **running** (not degraded).
- `systemctl --failed` -> **0 loaded units listed**.
- `systemd-analyze`: 1.378s kernel + 5.238s initrd + 4.114s userspace =
10.730s.
- Login as `phoebus` succeeded; networking up (`10.0.2.15/24`).
This is the point at which the Rust installer satisfies the parity criteria and
may become the primary installer. The Bash implementation remains the
reference/fallback until the desktop milestone is also verified.
## Desktop and Flatpak (Rust installer, verified)
The Rust installer, running the desktop profile, installed the full Wayland
desktop and Flatpak successfully (`INSTALL_RC=0`; 326 measured compile times
recorded). Key change: a desktop install now selects the supported stable
Gentoo profile `default/linux/amd64/23.0/desktop/systemd` instead of
reconstructing X/Wayland/dbus USE by hand from the minimal systemd profile.
This resolved the earlier `cairo[X]`/`libglvnd[X]`/`libxkbcommon[X]`
conflicts, and `sys-fs/{btrfs-progs,cryptsetup} -man` avoided a
Sphinx/Pillow circular dependency.
Booted with the ISO detached and logged in:
- `systemctl is-system-running` -> `running`; `systemctl --failed` -> 0 units.
- `flatpak --version` -> **Flatpak 1.18.1**; `flatpak remotes` -> **flathub
configured**.
- Binaries present and runnable: `dwl 0.7`, `Waybar v0.15.0`, `wofi 1.4.1`,
`mako`, `swaybg`, `pipewire`, `wireplumber`, `grim`, `wl-clipboard`.
Session verification. The VM's serial session has **no logind seat**, so
wlroots' DRM backend correctly refuses (`Failed to start a DRM session`). This
is a headless-harness limitation, not an installer defect. Using
`WLR_BACKENDS=headless WLR_RENDERER=pixman`:
- `dwl` started (pid observed), created the `wayland-0` socket.
- `mako`, `waybar`, and `swaybg` started and connected (`swaybg` found output
`HEADLESS-1`).
- `wl-copy`/`wl-paste` round-tripped `phoebus-clip`.
- `grim /tmp/shot.png` produced a **14401-byte PNG**, proving the compositor
renders and captures.
- `wpctl status` reported the PipeWire graph; `systemctl --user --failed` -> 0.
- `waybar` logged a non-fatal warning about a missing power-menu file and
power-profiles-daemon.
The installer writes `/usr/local/bin/phoebus-session`, which starts waybar,
mako and swaybg and then execs dwl, with a software-render fallback when no
render node is present.
## Known remaining work
- The graphical session was verified with the **headless** wlroots backend
(no logind seat in the serial harness). A physical VT / seat path is
architected (`phoebus-session`) but was not exercised on real hardware.
- NVIDIA proprietary support is architecturally allowed but **NOT VERIFIED**.
- UKI is deferred (BLS is used).
- The installer rebuilds `sys-apps/systemd` from source because
boot/policykit/cryptsetup USE differ from the binhost. Shipping a matching
systemd binpkg is a future optimization.
- Desktop USE still does not match the signed binhost, so the desktop
transaction compiles a large amount from source (~26 minutes for the desktop
package set on 8 vCPUs in the test VM).