phoebus_os/installer/tests/vm/README.md
Phoebus 7c5d4c5ca6
Some checks are pending
ci / test (push) Waiting to run
fix(sys): make Snap first-use setup reviewable and VM-tested
2026-09-25 18:07:13 -07:00

145 lines
6.1 KiB
Bash

# Disposable UEFI VM harness
Proves that the installer produces a bootable encrypted system without ever
touching a physical disk. The target is always a qcow2 file.
## Files
- `run-vm.sh` interactive QEMU launcher (manual testing).
- `install.sh` guest-side, non-interactive installation (run in the VM).
- `serial-driver.py` host-side pexpect driver (`install`, `boot` and
`rollback` modes).
- `rollback-guest.sh`, `rollback-verify.sh` guest-side rollback acceptance
(fetch `sys`, snapshot, break, undo, verify).
- `snap-guest.sh` guest-side Snap setup/install/refresh/remove acceptance.
- `test.conf.example` installer configuration used by the driver.
## Host prerequisites
- `qemu-system-x86_64`, `qemu-img`, OVMF firmware, Python `pexpect`.
- The reference Gentoo ISO.
- The static Rust installer (the guest never runs the Bash tree):
```sh
cd installer-rs
RUSTFLAGS="-C target-feature=+crt-static" \
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
```
## Procedure
Serve the static installer, the stage3 and the config to the guest:
```sh
cd /path/to/serve
cp /path/to/phoebus_os/installer-rs/target/x86_64-unknown-linux-gnu/release/phoebus-installer .
cp /path/to/phoebus_os/installer/tests/vm/install.sh .
cp /path/to/phoebus_os/installer/tests/vm/test.conf.example test.conf # edit TARGET_DISK etc.
sha256sum stage3.tar.xz # set STAGE3_SHA256
python3 -m http.server 8000 --bind 127.0.0.1
```
Run the installation (driver boots the ISO kernel/initrd directly and drives
the serial console):
```sh
QEMU_BIN=/path/to/qemu-system-x86_64 \
QEMU_LIBDIR=/path/to/qemu/libs \
QEMU_DATADIR=/path/to/qemu/share \
OVMF_CODE=/path/to/OVMF_CODE_4M.fd \
OVMF_VARS=/path/to/OVMF_VARS.fd \
DISK=/tmp/phoebus/target.qcow2 \
ISO=/path/to/livegui-amd64.iso \
KERNEL=/tmp/live/boot/gentoo INITRD=/tmp/live/boot/gentoo.igz \
INSTALL_URL=http://10.0.2.2:8000/install.sh \
python3 installer/tests/vm/serial-driver.py install
```
Then boot the disk alone (ISO detached) and verify:
```sh
DISK=/tmp/phoebus/target.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/serial-driver.py boot
```
To patch a console into an existing entry for observation, the BLS entry in
`/efi/loader/entries/*.conf` takes an `options` line; the installer can also
embed a console via `KERNEL_CONSOLE` (`GENTOO_KERNEL_CONSOLE` equivalent).
## Rollback acceptance (`serial-driver.py rollback`)
`rollback` drives the real rollback path end to end on an
**already-installed** disposable disk:
1. boot the installed disk, unlock LUKS and log in;
2. fetch `sys` and `rollback-guest.sh` from the host (the target does not yet
install `sys`; see `docs/HANDOFF.md`), record the `/var/db/pkg` fingerprint
and a `/home` marker, create a recovery point with `sys snapshot`, install a
small package (a controlled, reversible update) and run `sys undo`;
3. power off, boot the disk detached, and run `rollback-verify.sh`, comparing
the package fingerprint and the `/home` marker, checking kernel modules, and
reading the snapshot recovery entry and `sys rollback status`.
Serve the fixtures as well as the installer:
```sh
cp installer/tests/vm/rollback-guest.sh installer/tests/vm/rollback-verify.sh /path/to/serve/
cp installer-rs/target/x86_64-unknown-linux-gnu/release/sys /path/to/serve/
DISK=/tmp/phoebus/installed-clone.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/serial-driver.py rollback
```
Point `DISK` at a **disposable clone** of the evidence image so the original is
preserved. Booting the recovery entry itself with the network removed remains a
manual gate; the harness records the entry but does not select it.
## Snap acceptance on an installed disposable image
Serve the current static `sys` binary and `snap-guest.sh` from a small local
directory. Boot an installed **qcow2 overlay** with `serial-driver.py boot`,
setting `BOOT_AUTHENTICATE=1`, `BOOT_POWEROFF=1`, `CHECKS_TIMEOUT=7200`,
and the following checks. `BOOT_POWEROFF` cleanly shuts down the guest after
mutating checks so a later boot observes committed state:
```sh
BOOT_REQUIRE_CASES=snap-sys,snap-portage,snap-socket,snap-confinement,snap-install,snap-refresh,snap-remove
BOOT_CHECKS='echo ===CHECKS-$(printf START)===; echo SYSTEMD_STATUS:$(systemctl is-system-running); curl -fsSL http://10.0.2.2:8000/snap-guest.sh -o /tmp/snap-guest.sh; sudo bash /tmp/snap-guest.sh; echo ===CHECKS-$(printf END)==='
```
The guest script adds only the scoped `sys-fs/squashfs-tools lzma lzo` USE
setting to the disposable image, then uses `sys` for the reviewed Portage
install, socket activation and Snap actions. The boot driver logs only guest
output, not the test passwords it sends for LUKS, login and `sudo`.
## Safety (enforced, not just documented)
`vmsafety.py` validates every writable drive before QEMU starts:
- `DISK`/`DISKS` must name **qcow2** files inside the disposable directory
`PHOEBUS_VM_DISKS` (default `$PHOEBUS_VM_STATE/disks`).
- `/dev/*` paths, block and character devices, non-regular files, raw images,
symlinks that resolve outside the directory, and disks whose backing chain
reaches outside it are **refused** with a nonzero exit.
- Each run gets a private state directory (mode 0700), its own writable OVMF
variable file and a unique QMP monitor socket.
- Screenshots use structured QMP and argument arrays; there is no shell
interpolation (`os.system`) in the harness.
- Secrets are passed through 0600 files and checked for leakage in the
installer log by the guest script.
## Results
Every driver writes `results.json` (and a `screenshot-manifest.json` for the
GUI driver) under its run directory and exits nonzero on any `FAIL` (and, with
`PHOEBUS_VM_STRICT=1`, on `BLOCKED`/`NOT RUN`). Statuses are `PASS`, `FAIL`,
`BLOCKED`, `SKIPPED` and `NOT RUN`; a timeout is always a `FAIL`, never a pass.
```sh
make test-vm # unit-test the safety and result logic (no VM needed)
PHOEBUS_VM_DISKS=/var/tmp/phoebus-vm/disks \
OVMF_CODE=/usr/share/OVMF/OVMF_CODE_4M.fd \
OVMF_VARS=/usr/share/OVMF/OVMF_VARS_4M.fd \
DISK=/var/tmp/phoebus-vm/disks/target.qcow2 \
ISO=installer/live/phoebus-installer-amd64.iso \
python3 installer/tests/vm/gui-driver.py gui
```