phoebus_os/installer/tests/vm/README.md
Phoebus b8152054b6
Some checks are pending
ci / test (push) Waiting to run
fix(installer): harden boot recovery and desktop reliability
2026-10-01 07:03:42 -07:00

7.4 KiB

Disposable UEFI VM harness

Proves that the installer produces a bootable encrypted system without ever touching a physical disk. The target is always a qcow2 file.

Files

  • run-vm.sh interactive QEMU launcher (manual testing).
  • install.sh guest-side, non-interactive installation (run in the VM).
  • serial-driver.py host-side pexpect driver (install, install-repeat, boot, boot-assessed, recovery, fallback, rollback, interrupted-update modes).
  • rollback-guest.sh, rollback-verify.sh guest-side rollback acceptance (fetch sys, snapshot, break, undo, verify).
  • interrupted-update.sh kills an active sys/Portage package transaction, recovers its lock, undoes its snapshot, and verifies after reboot.
  • snap-guest.sh guest-side Snap setup/install/refresh/remove acceptance.
  • test.conf.example installer configuration used by the driver.

Host prerequisites

  • qemu-system-x86_64, qemu-img, OVMF firmware, Python pexpect; the graphical gate also needs Pillow.
  • The current Phoebus Debian-live ISO (the installed target remains Gentoo).
  • The static Rust installer (the guest never runs the Bash tree):
cd installer-rs
RUSTFLAGS="-C target-feature=+crt-static" \
  cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu

Procedure

Serve the pinned stage3 and test config to the guest. The installer and sys run from the candidate ISO itself:

cd /path/to/serve
cp /path/to/phoebus_os/installer/tests/vm/install.sh .
cp /path/to/phoebus_os/installer/tests/vm/test.conf.example test-mango.conf  # set the real stage3 digest
sha256sum stage3.tar.xz                               # set STAGE3_SHA256
python3 -m http.server 8000 --bind 0.0.0.0

Run the installation (driver boots the ISO kernel/initrd directly and drives the serial console):

QEMU_BIN=/path/to/qemu-system-x86_64 \
QEMU_LIBDIR=/path/to/qemu/libs \
QEMU_DATADIR=/path/to/qemu/share \
OVMF_CODE=/path/to/OVMF_CODE_4M.fd \
OVMF_VARS=/path/to/OVMF_VARS.fd \
DISK=/tmp/phoebus/target.qcow2 \
ISO=/path/to/phoebus-installer-amd64.iso \
KERNEL=/path/to/live/vmlinuz INITRD=/path/to/live/initrd.img \
INSTALL_URL=http://10.0.2.2:8000/install.sh \
python3 installer/tests/vm/serial-driver.py install

The install gate defaults to test-mango.conf (Mango, Flatpak and Nix). Set VM_INSTALL_CONFIG=test.conf only for a separate minimal-profile exercise.

Then boot the disk alone (ISO detached) and verify:

DISK=/tmp/phoebus/target.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/serial-driver.py boot

To patch a console into an existing entry for observation, the BLS entry in /efi/loader/entries/*.conf takes an options line; the installer can also embed a console via KERNEL_CONSOLE (GENTOO_KERNEL_CONSOLE equivalent).

Rollback acceptance (serial-driver.py rollback)

rollback drives the real rollback path end to end on an already-installed disposable disk:

  1. boot the installed disk, unlock LUKS and log in;
  2. fetch sys and rollback-guest.sh from the host (the installed system carries /usr/local/bin/sys; the host copy is a fallback), authenticate sudo (the password is sent only at sudo's own no-echo prompt, never on a command line), record the /var/db/pkg fingerprint and a /home marker, create a recovery point with sys snapshot, install a small package (a controlled, reversible update) and run sys undo — all as root, because unprivileged runs would be refused and prove nothing;
  3. power off, boot the disk detached, and run rollback-verify.sh (also as root), comparing the package fingerprint and the /home marker, checking kernel modules, systemd state, and reading the snapshot recovery entry and sys rollback status.

Both transcripts are kept (serial-phase1.log, serial.log). The guest scripts refuse to run unprivileged (ROLLBACK-NEEDS-ROOT / AFTER-NEEDS-ROOT) so a lost sudo timestamp fails loudly instead of recording vacuous passes.

Serve the fixtures as well as the installer:

cp installer/tests/vm/rollback-guest.sh installer/tests/vm/rollback-verify.sh /path/to/serve/
cp installer-rs/target/x86_64-unknown-linux-gnu/release/sys /path/to/serve/
DISK=/tmp/phoebus/installed-clone.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/serial-driver.py rollback

Point DISK at a disposable clone of the evidence image so the original is preserved. serial-driver.py recovery selects the actual systemd-boot recovery entry with networking disabled and checks the read-only snapshot and sys. fallback and interrupted-update create their own qcow2 overlays beneath PHOEBUS_VM_DISKS, leaving the installed evidence image unchanged. The install-repeat mode requires a distinct DISK_SECOND and checks its resolved path before starting QEMU.

Snap acceptance on an installed disposable image

Serve the current static sys binary and snap-guest.sh from a small local directory. Boot an installed qcow2 overlay with serial-driver.py boot, setting BOOT_AUTHENTICATE=1, BOOT_POWEROFF=1, CHECKS_TIMEOUT=7200, and the following checks. BOOT_POWEROFF cleanly shuts down the guest after mutating checks so a later boot observes committed state:

BOOT_REQUIRE_CASES=snap-sys,snap-portage,snap-socket,snap-confinement,snap-install,snap-refresh,snap-remove
BOOT_CHECKS='echo ===CHECKS-$(printf START)===; echo SYSTEMD_STATUS:$(systemctl is-system-running); curl -fsSL http://10.0.2.2:8000/snap-guest.sh -o /tmp/snap-guest.sh; sudo bash /tmp/snap-guest.sh; echo ===CHECKS-$(printf END)==='

The guest script adds only the scoped sys-fs/squashfs-tools lzma lzo USE setting to the disposable image, then uses sys for the reviewed Portage install, socket activation and Snap actions. The boot driver logs only guest output, not the test passwords it sends for LUKS, login and sudo.

Safety (enforced, not just documented)

vmsafety.py validates every writable drive before QEMU starts:

  • DISK/DISKS must name qcow2 files inside the disposable directory PHOEBUS_VM_DISKS (default $PHOEBUS_VM_STATE/disks).
  • /dev/* paths, block and character devices, non-regular files, raw images, symlinks that resolve outside the directory, and disks whose backing chain reaches outside it are refused with a nonzero exit.
  • Each run gets a private state directory (mode 0700), its own writable OVMF variable file and a unique QMP monitor socket.
  • Screenshots use structured QMP and argument arrays; there is no shell interpolation (os.system) in the harness.
  • Secrets are passed through 0600 files and checked for leakage in the installer log by the guest script. Host serial transcripts and structured result details redact registered test credentials, including output split across pexpect reads.

Results

Every driver writes results.json (and a screenshot-manifest.json for the GUI driver) under its run directory and exits nonzero on any FAIL (and, with PHOEBUS_VM_STRICT=1, on BLOCKED/NOT RUN). Statuses are PASS, FAIL, BLOCKED, SKIPPED and NOT RUN; a timeout is always a FAIL, never a pass.

make test-vm          # unit-test the safety and result logic (no VM needed)
PHOEBUS_VM_DISKS=/var/tmp/phoebus-vm/disks \
OVMF_CODE=/usr/share/OVMF/OVMF_CODE_4M.fd \
OVMF_VARS=/usr/share/OVMF/OVMF_VARS_4M.fd \
DISK=/var/tmp/phoebus-vm/disks/target.qcow2 \
ISO=installer/live/phoebus-installer-amd64.iso \
python3 installer/tests/vm/gui-driver.py gui