Delete installer/bin, lib, tui, config, tests/*.sh and README.md now that the Rust installer is the only proven path (docs/BASH-MIGRATION.md Phase 4). Move FINDINGS.md and VM-BOOT-VERIFICATION.md to docs/ and delete the Bash-only TESTING.md. The disposable VM harness under installer/tests/vm/ is retained.
224 lines
11 KiB
Markdown
224 lines
11 KiB
Markdown
# Disposable UEFI VM boot verification
|
|
|
|
This records the first successful end-to-end install and boot of Phoebus OS
|
|
against a disposable qcow2 disk. No physical disk was ever touched.
|
|
|
|
## Exact tested path
|
|
|
|
Host (Ubuntu, no root required):
|
|
|
|
- QEMU `qemu-system-x86_64` (extracted locally under `/tmp/opencode/qemudeb`),
|
|
OVMF (`OVMF_CODE_4M.fd` + `OVMF_VARS_4M.fd`).
|
|
- Target disk: `/tmp/opencode/vm/work/target.qcow2`, 30 GiB, virtio-blk
|
|
(`/dev/vda` in the guest).
|
|
- Live boot: the reference ISO's own `boot/gentoo` + `boot/gentoo.igz`, with
|
|
`-append "root=live:CDLABEL=gentoo-amd64-livegui rd.live.dir=/
|
|
rd.live.squashimg=image.squashfs cdroot console=ttyS0,115200"`.
|
|
- A host `python3 -m http.server` serves the installer tarball, the pinned
|
|
stage3 and the test config to the guest over QEMU user networking.
|
|
|
|
Phase 1 (install):
|
|
|
|
1. Boot the live ISO headless; get a root shell on the serial console.
|
|
2. Download the installer and test config from the host.
|
|
3. Write secrets to `0600` files and run the installer binary (the original
|
|
run used the Bash prototype; the harness now serves `phoebus-installer`):
|
|
`phoebus-installer --config test.conf --install --yes
|
|
--luks-passphrase-file FILE --user-password-file FILE`
|
|
4. Stages executed in order: `preflight, storage, stage3, fstab,
|
|
portage, base packages, kernel, bootloader, users, services, desktop,
|
|
finalize`.
|
|
|
|
Phase 2 (boot the disk alone, ISO detached):
|
|
|
|
5. Power off, then start a new QEMU with **no ISO and no `-kernel`**, boot
|
|
`order=c`, same OVMF NVRAM.
|
|
6. Firmware runs `\EFI\systemd\systemd-bootx64.efi`; systemd-boot boots the
|
|
"Gentoo Linux" BLS entry from the ESP.
|
|
7. `systemd-cryptsetup` prompts for the LUKS2 passphrase, unlocks
|
|
`/dev/vda2`, mounts the Btrfs `@` subvolume as `/`.
|
|
8. systemd (PID 1) starts, serial getty appears, user login succeeds.
|
|
|
|
## Verified results
|
|
|
|
- Firmware booted the disk's own systemd-boot (Boot000C "Linux Boot Manager").
|
|
- LUKS2 passphrase prompt and successful unlock.
|
|
- Btrfs subvolumes mounted: `/`, `/home`, `/var`, `/.snapshots`, `/swap`
|
|
(later fix), ESP at `/efi`.
|
|
- `cat /proc/1/comm` -> `systemd`.
|
|
- `systemd-analyze`: 1.442s kernel + 5.755s initrd + 2.942s userspace.
|
|
- Login as `phoebus` with the configured password; groups:
|
|
`wheel audio video render input plugdev power network libinput`.
|
|
- Networking up (`enp0s2 10.0.2.15/24`).
|
|
- `systemctl --failed`: exactly one unit, `swap-swapfile.swap` (fixed after
|
|
this run; see below).
|
|
|
|
## Bugs found and fixed during this work
|
|
|
|
1. **Subvolumes created after mounting `@`** - `storage_prepare` mounted
|
|
`subvol=@` before `@` existed (mount exit 32). Added `btrfs_create_layout`
|
|
(mount top-level, create subvols, unmount, then mount layout) and a
|
|
regression test (`tests/test_btrfs_order.sh`).
|
|
2. **Generated fstab overwritten by stage3** - fstab/crypttab were generated
|
|
during storage, then the stage3 tarball replaced `/etc/fstab`. Moved
|
|
generation to a new `fstab_generate` stage after extraction.
|
|
3. **`/var/tmp` absent in the target** - systemd's `kernel-install` needs a
|
|
temporary directory; `/var/tmp` was missing. The fstab stage now creates
|
|
`/var/tmp`, `/tmp`, `/root`, `/home`, and chroot commands set
|
|
`TMPDIR=/tmp`.
|
|
4. **installkernel needs `uki_generator` even for BLS** - with an explicit
|
|
`/etc/kernel/install.conf`, `05-check-config.install` always requires a
|
|
`uki_generator`. BLS now writes `uki_generator=none`.
|
|
5. **dracut could not build the `crypt` module** - the target lacked
|
|
`sys-fs/cryptsetup`. Added `sys-fs/cryptsetup` and `sys-fs/btrfs-progs` to
|
|
the base package set.
|
|
6. **No `systemd-cryptsetup`** - `sys-apps/systemd` was built without the
|
|
`cryptsetup` USE flag, so the encrypted root could not be unlocked. Added
|
|
`cryptsetup` (and `policykit`) to the systemd package.use entry.
|
|
7. **Kernel command line used unnamed LUKS** - `rd.luks.uuid` yields
|
|
`luks-<uuid>`, not the configured name. Switched to
|
|
`rd.luks.name=<uuid>=<name>`.
|
|
8. **Dracut UKI did not embed the command line** - the dracut `uki_generator`
|
|
produced a UKI with no `.cmdline`, so it booted with no `root=`. Deferred
|
|
UKI in favour of a reliable BLS entry carrying `options`. UKI remains
|
|
future work.
|
|
9. **fstab did not mount `@swap`** - the swapfile unit failed because `/swap`
|
|
was never the `@swap` subvolume. Added the `subvol=@swap` fstab entry.
|
|
10. **`local root="$1" dir="$root/..."` class bug** - reintroduced in
|
|
`boot_write_dracut_conf`; split into two `local` statements. This class of
|
|
bug is now covered by the regression suite.
|
|
11. **Wrong package atoms** - `gui-apps/fuzzel` (not in Gentoo) -> `gui-apps/wofi`,
|
|
`app-misc/brightnessctl` (not packaged) removed,
|
|
`app-crypt/gnome-keyring` -> `gnome-base/gnome-keyring`,
|
|
`gui-apps/polkit-gnome` -> `gnome-extra/polkit-gnome`.
|
|
12. **Package policy compiled too much** - unknown packages defaulted to
|
|
source. `balanced` now compiles cheap builds and uses signed Gentoo
|
|
binaries for everything else, which cut installer time substantially.
|
|
|
|
## Rust installer parity (same VM path)
|
|
|
|
The Rust installer (`installer-rs`) reproduced the entire path on a fresh
|
|
disposable qcow2 disk using the static binary
|
|
`phoebus-installer` (`x86_64-unknown-linux-gnu`, crt-static):
|
|
|
|
- Same config (legacy flat format), same pinned stage3 URL + SHA-256.
|
|
- `INSTALL_RC=0`; stages: storage, stage3, fstab, portage, base packages,
|
|
kernel, bootloader, users, services, desktop (none), finalize.
|
|
- Policy decisions matched Bash: `sudo`/`networkmanager`/`polkit` binary,
|
|
`cryptsetup`/`btrfs-progs` source.
|
|
- The installer recorded **81 then 97 measured compile times** from Portage's
|
|
`emerge.log` (real data, not estimates).
|
|
- `NO_SECRET_LEAK` from the log secret scan.
|
|
|
|
Booted with the ISO detached:
|
|
|
|
- UEFI firmware ran systemd-boot; BLS entry booted.
|
|
- LUKS2 unlock, Btrfs `@` root, `/home`, `/var`, `/swap`, ESP at `/efi`.
|
|
- `cat /proc/1/comm` -> `systemd`.
|
|
- `systemctl is-system-running` -> **running** (not degraded).
|
|
- `systemctl --failed` -> **0 loaded units listed**.
|
|
- `systemd-analyze`: 1.378s kernel + 5.238s initrd + 4.114s userspace =
|
|
10.730s.
|
|
- Login as `phoebus` succeeded; networking up (`10.0.2.15/24`).
|
|
|
|
This is the point at which the Rust installer satisfies the parity criteria and
|
|
may become the primary installer. The Bash implementation remains the
|
|
reference/fallback until the desktop milestone is also verified.
|
|
|
|
## Desktop and Flatpak (Rust installer, verified)
|
|
|
|
The Rust installer, running the desktop profile, installed the full Wayland
|
|
desktop and Flatpak successfully (`INSTALL_RC=0`; 326 measured compile times
|
|
recorded). Key change: a desktop install now selects the supported stable
|
|
Gentoo profile `default/linux/amd64/23.0/desktop/systemd` instead of
|
|
reconstructing X/Wayland/dbus USE by hand from the minimal systemd profile.
|
|
This resolved the earlier `cairo[X]`/`libglvnd[X]`/`libxkbcommon[X]`
|
|
conflicts, and `sys-fs/{btrfs-progs,cryptsetup} -man` avoided a
|
|
Sphinx/Pillow circular dependency.
|
|
|
|
Booted with the ISO detached and logged in:
|
|
|
|
- `systemctl is-system-running` -> `running`; `systemctl --failed` -> 0 units.
|
|
- `flatpak --version` -> **Flatpak 1.18.1**; `flatpak remotes` -> **flathub
|
|
configured**.
|
|
- Binaries present and runnable: `dwl 0.7`, `Waybar v0.15.0`, `wofi 1.4.1`,
|
|
`mako`, `swaybg`, `pipewire`, `wireplumber`, `grim`, `wl-clipboard`.
|
|
|
|
Session verification. The VM's serial session has **no logind seat**, so
|
|
wlroots' DRM backend correctly refuses (`Failed to start a DRM session`). This
|
|
is a headless-harness limitation, not an installer defect. Using
|
|
`WLR_BACKENDS=headless WLR_RENDERER=pixman`:
|
|
|
|
- `dwl` started (pid observed), created the `wayland-0` socket.
|
|
- `mako`, `waybar`, and `swaybg` started and connected (`swaybg` found output
|
|
`HEADLESS-1`).
|
|
- `wl-copy`/`wl-paste` round-tripped `phoebus-clip`.
|
|
- `grim /tmp/shot.png` produced a **14401-byte PNG**, proving the compositor
|
|
renders and captures.
|
|
- `wpctl status` reported the PipeWire graph; `systemctl --user --failed` -> 0.
|
|
- `waybar` logged a non-fatal warning about a missing power-menu file and
|
|
power-profiles-daemon.
|
|
|
|
The installer writes `/usr/local/bin/phoebus-session`, which starts waybar,
|
|
mako and swaybg and then execs dwl, with a software-render fallback when no
|
|
render node is present.
|
|
|
|
## Official harness on the static binary (Bash migration, Phase 2)
|
|
|
|
The committed harness (`installer/tests/vm/serial-driver.py` +
|
|
`installer/tests/vm/install.sh`) now drives the **static Rust binary** and no
|
|
longer runs the Bash installer. Re-verified end to end on a fresh disposable
|
|
qcow2 (24 GiB, virtio-blk `/dev/vda`):
|
|
|
|
```sh
|
|
# Build:
|
|
cd installer-rs && RUSTFLAGS="-C target-feature=+crt-static" \
|
|
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
|
|
|
|
# Install in the disposable VM, then boot the disk detached:
|
|
python3 installer/tests/vm/serial-driver.py install
|
|
python3 installer/tests/vm/serial-driver.py boot
|
|
```
|
|
|
|
Install (`INSTALL-RC=0`):
|
|
|
|
- The guest fetched `/tmp/phoebus-installer` and printed
|
|
`phoebus-installer 0.1.0` (the Rust binary, not Bash).
|
|
- Stages completed in proven order: `storage_prepare`, `stage_install`,
|
|
`fstab_generate`, `portage_configure`, `packages_base`, `kernel_install`,
|
|
`boot_install`, `users_configure`, `services_configure`, `desktop_install`
|
|
(none), `finalize`.
|
|
- Policy matched Bash: `crypto`/`btrfs-progs` source build, `sudo`/
|
|
`networkmanager`/`polkit` binary.
|
|
- `NO_SECRET_LEAK`; the run powered off cleanly.
|
|
|
|
Boot with the ISO detached (`serial-driver.py boot`):
|
|
|
|
- UEFI firmware ran `\EFI\systemd\systemd-bootx64.efi` (Boot0002 "Linux Boot
|
|
Manager"); the BLS "Gentoo Linux" entry booted.
|
|
- `systemd-cryptsetup` prompted for the LUKS2 passphrase and unlocked
|
|
`phoebus-root (cryptroot)`.
|
|
- Login as `phoebus` succeeded.
|
|
- `cat /proc/1/comm` -> `systemd`.
|
|
- `systemctl is-system-running` -> **running**.
|
|
- `systemctl --failed` -> **0 loaded units listed**.
|
|
- Networking: `enp0s2 UP 10.0.2.15/24`.
|
|
- Mounts: `/` (btrfs `@`), `/home`, `/var`, `/.snapshots`, `/swap`, ESP
|
|
`/efi`; `vda2` is `crypto_LUKS`.
|
|
|
|
This satisfies the Phase 2 exit gate of `docs/BASH-MIGRATION.md`: the only
|
|
proven boot path no longer depends on the retired Bash installer.
|
|
|
|
## Known remaining work
|
|
|
|
- The graphical session was verified with the **headless** wlroots backend
|
|
(no logind seat in the serial harness). A physical VT / seat path is
|
|
architected (`phoebus-session`) but was not exercised on real hardware.
|
|
- NVIDIA proprietary support is architecturally allowed but **NOT VERIFIED**.
|
|
- UKI is deferred (BLS is used).
|
|
- The installer rebuilds `sys-apps/systemd` from source because
|
|
boot/policykit/cryptsetup USE differ from the binhost. Shipping a matching
|
|
systemd binpkg is a future optimization.
|
|
- Desktop USE still does not match the signed binhost, so the desktop
|
|
transaction compiles a large amount from source (~26 minutes for the desktop
|
|
package set on 8 vCPUs in the test VM).
|