The committed harness now installs with the static Rust binary and boots the disk detached: systemd PID1, running, 0 failed units, LUKS2 unlock, login and networking. Also fixes the boot check marker so expect() does not match the shell's echoed command.
223 lines
11 KiB
Bash
223 lines
11 KiB
Bash
# Disposable UEFI VM boot verification
|
|
|
|
This records the first successful end-to-end install and boot of Phoebus OS
|
|
against a disposable qcow2 disk. No physical disk was ever touched.
|
|
|
|
## Exact tested path
|
|
|
|
Host (Ubuntu, no root required):
|
|
|
|
- QEMU `qemu-system-x86_64` (extracted locally under `/tmp/opencode/qemudeb`),
|
|
OVMF (`OVMF_CODE_4M.fd` + `OVMF_VARS_4M.fd`).
|
|
- Target disk: `/tmp/opencode/vm/work/target.qcow2`, 30 GiB, virtio-blk
|
|
(`/dev/vda` in the guest).
|
|
- Live boot: the reference ISO's own `boot/gentoo` + `boot/gentoo.igz`, with
|
|
`-append "root=live:CDLABEL=gentoo-amd64-livegui rd.live.dir=/
|
|
rd.live.squashimg=image.squashfs cdroot console=ttyS0,115200"`.
|
|
- A host `python3 -m http.server` serves the installer tarball, the pinned
|
|
stage3 and the test config to the guest over QEMU user networking.
|
|
|
|
Phase 1 (install):
|
|
|
|
1. Boot the live ISO headless; get a root shell on the serial console.
|
|
2. Download the installer and test config from the host.
|
|
3. Write secrets to `0600` files and run:
|
|
`installer/bin/ai-installer --config test.conf --install --yes
|
|
--luks-passphrase-file FILE --user-password-file FILE`
|
|
4. Stages executed in order: `preflight, storage, stage3, fstab,
|
|
portage, base packages, kernel, bootloader, users, services, desktop,
|
|
finalize`.
|
|
|
|
Phase 2 (boot the disk alone, ISO detached):
|
|
|
|
5. Power off, then start a new QEMU with **no ISO and no `-kernel`**, boot
|
|
`order=c`, same OVMF NVRAM.
|
|
6. Firmware runs `\EFI\systemd\systemd-bootx64.efi`; systemd-boot boots the
|
|
"Gentoo Linux" BLS entry from the ESP.
|
|
7. `systemd-cryptsetup` prompts for the LUKS2 passphrase, unlocks
|
|
`/dev/vda2`, mounts the Btrfs `@` subvolume as `/`.
|
|
8. systemd (PID 1) starts, serial getty appears, user login succeeds.
|
|
|
|
## Verified results
|
|
|
|
- Firmware booted the disk's own systemd-boot (Boot000C "Linux Boot Manager").
|
|
- LUKS2 passphrase prompt and successful unlock.
|
|
- Btrfs subvolumes mounted: `/`, `/home`, `/var`, `/.snapshots`, `/swap`
|
|
(later fix), ESP at `/efi`.
|
|
- `cat /proc/1/comm` -> `systemd`.
|
|
- `systemd-analyze`: 1.442s kernel + 5.755s initrd + 2.942s userspace.
|
|
- Login as `phoebus` with the configured password; groups:
|
|
`wheel audio video render input plugdev power network libinput`.
|
|
- Networking up (`enp0s2 10.0.2.15/24`).
|
|
- `systemctl --failed`: exactly one unit, `swap-swapfile.swap` (fixed after
|
|
this run; see below).
|
|
|
|
## Bugs found and fixed during this work
|
|
|
|
1. **Subvolumes created after mounting `@`** - `storage_prepare` mounted
|
|
`subvol=@` before `@` existed (mount exit 32). Added `btrfs_create_layout`
|
|
(mount top-level, create subvols, unmount, then mount layout) and a
|
|
regression test (`tests/test_btrfs_order.sh`).
|
|
2. **Generated fstab overwritten by stage3** - fstab/crypttab were generated
|
|
during storage, then the stage3 tarball replaced `/etc/fstab`. Moved
|
|
generation to a new `fstab_generate` stage after extraction.
|
|
3. **`/var/tmp` absent in the target** - systemd's `kernel-install` needs a
|
|
temporary directory; `/var/tmp` was missing. The fstab stage now creates
|
|
`/var/tmp`, `/tmp`, `/root`, `/home`, and chroot commands set
|
|
`TMPDIR=/tmp`.
|
|
4. **installkernel needs `uki_generator` even for BLS** - with an explicit
|
|
`/etc/kernel/install.conf`, `05-check-config.install` always requires a
|
|
`uki_generator`. BLS now writes `uki_generator=none`.
|
|
5. **dracut could not build the `crypt` module** - the target lacked
|
|
`sys-fs/cryptsetup`. Added `sys-fs/cryptsetup` and `sys-fs/btrfs-progs` to
|
|
the base package set.
|
|
6. **No `systemd-cryptsetup`** - `sys-apps/systemd` was built without the
|
|
`cryptsetup` USE flag, so the encrypted root could not be unlocked. Added
|
|
`cryptsetup` (and `policykit`) to the systemd package.use entry.
|
|
7. **Kernel command line used unnamed LUKS** - `rd.luks.uuid` yields
|
|
`luks-<uuid>`, not the configured name. Switched to
|
|
`rd.luks.name=<uuid>=<name>`.
|
|
8. **Dracut UKI did not embed the command line** - the dracut `uki_generator`
|
|
produced a UKI with no `.cmdline`, so it booted with no `root=`. Deferred
|
|
UKI in favour of a reliable BLS entry carrying `options`. UKI remains
|
|
future work.
|
|
9. **fstab did not mount `@swap`** - the swapfile unit failed because `/swap`
|
|
was never the `@swap` subvolume. Added the `subvol=@swap` fstab entry.
|
|
10. **`local root="$1" dir="$root/..."` class bug** - reintroduced in
|
|
`boot_write_dracut_conf`; split into two `local` statements. This class of
|
|
bug is now covered by the regression suite.
|
|
11. **Wrong package atoms** - `gui-apps/fuzzel` (not in Gentoo) -> `gui-apps/wofi`,
|
|
`app-misc/brightnessctl` (not packaged) removed,
|
|
`app-crypt/gnome-keyring` -> `gnome-base/gnome-keyring`,
|
|
`gui-apps/polkit-gnome` -> `gnome-extra/polkit-gnome`.
|
|
12. **Package policy compiled too much** - unknown packages defaulted to
|
|
source. `balanced` now compiles cheap builds and uses signed Gentoo
|
|
binaries for everything else, which cut installer time substantially.
|
|
|
|
## Rust installer parity (same VM path)
|
|
|
|
The Rust installer (`installer-rs`) reproduced the entire path on a fresh
|
|
disposable qcow2 disk using the static binary
|
|
`phoebus-installer` (`x86_64-unknown-linux-gnu`, crt-static):
|
|
|
|
- Same config (legacy flat format), same pinned stage3 URL + SHA-256.
|
|
- `INSTALL_RC=0`; stages: storage, stage3, fstab, portage, base packages,
|
|
kernel, bootloader, users, services, desktop (none), finalize.
|
|
- Policy decisions matched Bash: `sudo`/`networkmanager`/`polkit` binary,
|
|
`cryptsetup`/`btrfs-progs` source.
|
|
- The installer recorded **81 then 97 measured compile times** from Portage's
|
|
`emerge.log` (real data, not estimates).
|
|
- `NO_SECRET_LEAK` from the log secret scan.
|
|
|
|
Booted with the ISO detached:
|
|
|
|
- UEFI firmware ran systemd-boot; BLS entry booted.
|
|
- LUKS2 unlock, Btrfs `@` root, `/home`, `/var`, `/swap`, ESP at `/efi`.
|
|
- `cat /proc/1/comm` -> `systemd`.
|
|
- `systemctl is-system-running` -> **running** (not degraded).
|
|
- `systemctl --failed` -> **0 loaded units listed**.
|
|
- `systemd-analyze`: 1.378s kernel + 5.238s initrd + 4.114s userspace =
|
|
10.730s.
|
|
- Login as `phoebus` succeeded; networking up (`10.0.2.15/24`).
|
|
|
|
This is the point at which the Rust installer satisfies the parity criteria and
|
|
may become the primary installer. The Bash implementation remains the
|
|
reference/fallback until the desktop milestone is also verified.
|
|
|
|
## Desktop and Flatpak (Rust installer, verified)
|
|
|
|
The Rust installer, running the desktop profile, installed the full Wayland
|
|
desktop and Flatpak successfully (`INSTALL_RC=0`; 326 measured compile times
|
|
recorded). Key change: a desktop install now selects the supported stable
|
|
Gentoo profile `default/linux/amd64/23.0/desktop/systemd` instead of
|
|
reconstructing X/Wayland/dbus USE by hand from the minimal systemd profile.
|
|
This resolved the earlier `cairo[X]`/`libglvnd[X]`/`libxkbcommon[X]`
|
|
conflicts, and `sys-fs/{btrfs-progs,cryptsetup} -man` avoided a
|
|
Sphinx/Pillow circular dependency.
|
|
|
|
Booted with the ISO detached and logged in:
|
|
|
|
- `systemctl is-system-running` -> `running`; `systemctl --failed` -> 0 units.
|
|
- `flatpak --version` -> **Flatpak 1.18.1**; `flatpak remotes` -> **flathub
|
|
configured**.
|
|
- Binaries present and runnable: `dwl 0.7`, `Waybar v0.15.0`, `wofi 1.4.1`,
|
|
`mako`, `swaybg`, `pipewire`, `wireplumber`, `grim`, `wl-clipboard`.
|
|
|
|
Session verification. The VM's serial session has **no logind seat**, so
|
|
wlroots' DRM backend correctly refuses (`Failed to start a DRM session`). This
|
|
is a headless-harness limitation, not an installer defect. Using
|
|
`WLR_BACKENDS=headless WLR_RENDERER=pixman`:
|
|
|
|
- `dwl` started (pid observed), created the `wayland-0` socket.
|
|
- `mako`, `waybar`, and `swaybg` started and connected (`swaybg` found output
|
|
`HEADLESS-1`).
|
|
- `wl-copy`/`wl-paste` round-tripped `phoebus-clip`.
|
|
- `grim /tmp/shot.png` produced a **14401-byte PNG**, proving the compositor
|
|
renders and captures.
|
|
- `wpctl status` reported the PipeWire graph; `systemctl --user --failed` -> 0.
|
|
- `waybar` logged a non-fatal warning about a missing power-menu file and
|
|
power-profiles-daemon.
|
|
|
|
The installer writes `/usr/local/bin/phoebus-session`, which starts waybar,
|
|
mako and swaybg and then execs dwl, with a software-render fallback when no
|
|
render node is present.
|
|
|
|
## Official harness on the static binary (Bash migration, Phase 2)
|
|
|
|
The committed harness (`installer/tests/vm/serial-driver.py` +
|
|
`installer/tests/vm/install.sh`) now drives the **static Rust binary** and no
|
|
longer runs the Bash installer. Re-verified end to end on a fresh disposable
|
|
qcow2 (24 GiB, virtio-blk `/dev/vda`):
|
|
|
|
```sh
|
|
# Build:
|
|
cd installer-rs && RUSTFLAGS="-C target-feature=+crt-static" \
|
|
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
|
|
|
|
# Install in the disposable VM, then boot the disk detached:
|
|
python3 installer/tests/vm/serial-driver.py install
|
|
python3 installer/tests/vm/serial-driver.py boot
|
|
```
|
|
|
|
Install (`INSTALL-RC=0`):
|
|
|
|
- The guest fetched `/tmp/phoebus-installer` and printed
|
|
`phoebus-installer 0.1.0` (the Rust binary, not Bash).
|
|
- Stages completed in proven order: `storage_prepare`, `stage_install`,
|
|
`fstab_generate`, `portage_configure`, `packages_base`, `kernel_install`,
|
|
`boot_install`, `users_configure`, `services_configure`, `desktop_install`
|
|
(none), `finalize`.
|
|
- Policy matched Bash: `crypto`/`btrfs-progs` source build, `sudo`/
|
|
`networkmanager`/`polkit` binary.
|
|
- `NO_SECRET_LEAK`; the run powered off cleanly.
|
|
|
|
Boot with the ISO detached (`serial-driver.py boot`):
|
|
|
|
- UEFI firmware ran `\EFI\systemd\systemd-bootx64.efi` (Boot0002 "Linux Boot
|
|
Manager"); the BLS "Gentoo Linux" entry booted.
|
|
- `systemd-cryptsetup` prompted for the LUKS2 passphrase and unlocked
|
|
`phoebus-root (cryptroot)`.
|
|
- Login as `phoebus` succeeded.
|
|
- `cat /proc/1/comm` -> `systemd`.
|
|
- `systemctl is-system-running` -> **running**.
|
|
- `systemctl --failed` -> **0 loaded units listed**.
|
|
- Networking: `enp0s2 UP 10.0.2.15/24`.
|
|
- Mounts: `/` (btrfs `@`), `/home`, `/var`, `/.snapshots`, `/swap`, ESP
|
|
`/efi`; `vda2` is `crypto_LUKS`.
|
|
|
|
This satisfies the Phase 2 exit gate of `docs/BASH-MIGRATION.md`: the only
|
|
proven boot path no longer depends on `installer/bin/ai-installer`.
|
|
|
|
## Known remaining work
|
|
|
|
- The graphical session was verified with the **headless** wlroots backend
|
|
(no logind seat in the serial harness). A physical VT / seat path is
|
|
architected (`phoebus-session`) but was not exercised on real hardware.
|
|
- NVIDIA proprietary support is architecturally allowed but **NOT VERIFIED**.
|
|
- UKI is deferred (BLS is used).
|
|
- The installer rebuilds `sys-apps/systemd` from source because
|
|
boot/policykit/cryptsetup USE differ from the binhost. Shipping a matching
|
|
systemd binpkg is a future optimization.
|
|
- Desktop USE still does not match the signed binhost, so the desktop
|
|
transaction compiles a large amount from source (~26 minutes for the desktop
|
|
package set on 8 vCPUs in the test VM).
|