phoebus_os/installer/tests/vm/README.md
Phoebus c759477c7f test(vm): automate rollback acceptance on a disposable clone
Add a serial-driver rollback mode and guest scripts that exercise the real
rollback path: record the /var/db/pkg fingerprint and a /home marker,
create a recovery point with sys snapshot, install a small package as a
controlled reversible update, run sys undo, power off, boot detached and
compare the restored package state and /home against the pre-rollback
values. Checks kernel modules, the snapshot recovery entry and sys
rollback status. sys is fetched from the host, matching the existing
harness evidence procedure; booting the recovery entry offline remains a
documented manual gate. bash -n guards for every guest script are added
to the no-VM unit tests.
2026-09-23 06:19:17 -07:00

126 lines
5 KiB
Bash

# Disposable UEFI VM harness
Proves that the installer produces a bootable encrypted system without ever
touching a physical disk. The target is always a qcow2 file.
## Files
- `run-vm.sh` interactive QEMU launcher (manual testing).
- `install.sh` guest-side, non-interactive installation (run in the VM).
- `serial-driver.py` host-side pexpect driver (`install`, `boot` and
`rollback` modes).
- `rollback-guest.sh`, `rollback-verify.sh` guest-side rollback acceptance
(fetch `sys`, snapshot, break, undo, verify).
- `test.conf.example` installer configuration used by the driver.
## Host prerequisites
- `qemu-system-x86_64`, `qemu-img`, OVMF firmware, Python `pexpect`.
- The reference Gentoo ISO.
- The static Rust installer (the guest never runs the Bash tree):
```sh
cd installer-rs
RUSTFLAGS="-C target-feature=+crt-static" \
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
```
## Procedure
Serve the static installer, the stage3 and the config to the guest:
```sh
cd /path/to/serve
cp /path/to/phoebus_os/installer-rs/target/x86_64-unknown-linux-gnu/release/phoebus-installer .
cp /path/to/phoebus_os/installer/tests/vm/install.sh .
cp /path/to/phoebus_os/installer/tests/vm/test.conf.example test.conf # edit TARGET_DISK etc.
sha256sum stage3.tar.xz # set STAGE3_SHA256
python3 -m http.server 8000 --bind 127.0.0.1
```
Run the installation (driver boots the ISO kernel/initrd directly and drives
the serial console):
```sh
QEMU_BIN=/path/to/qemu-system-x86_64 \
QEMU_LIBDIR=/path/to/qemu/libs \
QEMU_DATADIR=/path/to/qemu/share \
OVMF_CODE=/path/to/OVMF_CODE_4M.fd \
OVMF_VARS=/path/to/OVMF_VARS.fd \
DISK=/tmp/phoebus/target.qcow2 \
ISO=/path/to/livegui-amd64.iso \
KERNEL=/tmp/live/boot/gentoo INITRD=/tmp/live/boot/gentoo.igz \
INSTALL_URL=http://10.0.2.2:8000/install.sh \
python3 installer/tests/vm/serial-driver.py install
```
Then boot the disk alone (ISO detached) and verify:
```sh
DISK=/tmp/phoebus/target.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/serial-driver.py boot
```
To patch a console into an existing entry for observation, the BLS entry in
`/efi/loader/entries/*.conf` takes an `options` line; the installer can also
embed a console via `KERNEL_CONSOLE` (`GENTOO_KERNEL_CONSOLE` equivalent).
## Rollback acceptance (`serial-driver.py rollback`)
`rollback` drives the real rollback path end to end on an
**already-installed** disposable disk:
1. boot the installed disk, unlock LUKS and log in;
2. fetch `sys` and `rollback-guest.sh` from the host (the target does not yet
install `sys`; see `docs/HANDOFF.md`), record the `/var/db/pkg` fingerprint
and a `/home` marker, create a recovery point with `sys snapshot`, install a
small package (a controlled, reversible update) and run `sys undo`;
3. power off, boot the disk detached, and run `rollback-verify.sh`, comparing
the package fingerprint and the `/home` marker, checking kernel modules, and
reading the snapshot recovery entry and `sys rollback status`.
Serve the fixtures as well as the installer:
```sh
cp installer/tests/vm/rollback-guest.sh installer/tests/vm/rollback-verify.sh /path/to/serve/
cp installer-rs/target/x86_64-unknown-linux-gnu/release/sys /path/to/serve/
DISK=/tmp/phoebus/installed-clone.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/serial-driver.py rollback
```
Point `DISK` at a **disposable clone** of the evidence image so the original is
preserved. Booting the recovery entry itself with the network removed remains a
manual gate; the harness records the entry but does not select it.
## Safety (enforced, not just documented)
`vmsafety.py` validates every writable drive before QEMU starts:
- `DISK`/`DISKS` must name **qcow2** files inside the disposable directory
`PHOEBUS_VM_DISKS` (default `$PHOEBUS_VM_STATE/disks`).
- `/dev/*` paths, block and character devices, non-regular files, raw images,
symlinks that resolve outside the directory, and disks whose backing chain
reaches outside it are **refused** with a nonzero exit.
- Each run gets a private state directory (mode 0700), its own writable OVMF
variable file and a unique QMP monitor socket.
- Screenshots use structured QMP and argument arrays; there is no shell
interpolation (`os.system`) in the harness.
- Secrets are passed through 0600 files and checked for leakage in the
installer log by the guest script.
## Results
Every driver writes `results.json` (and a `screenshot-manifest.json` for the
GUI driver) under its run directory and exits nonzero on any `FAIL` (and, with
`PHOEBUS_VM_STRICT=1`, on `BLOCKED`/`NOT RUN`). Statuses are `PASS`, `FAIL`,
`BLOCKED`, `SKIPPED` and `NOT RUN`; a timeout is always a `FAIL`, never a pass.
```sh
make test-vm # unit-test the safety and result logic (no VM needed)
PHOEBUS_VM_DISKS=/var/tmp/phoebus-vm/disks \
OVMF_CODE=/usr/share/OVMF/OVMF_CODE_4M.fd \
OVMF_VARS=/usr/share/OVMF/OVMF_VARS_4M.fd \
DISK=/var/tmp/phoebus-vm/disks/target.qcow2 \
ISO=installer/live/phoebus-installer-amd64.iso \
python3 installer/tests/vm/gui-driver.py gui
```