phoebus_os/installer/tests/vm/README.md
Phoebus cc23df69de test(vm): enforce harness disk safety and structured results
Validate every writable drive (qcow2, inside PHOEBUS_VM_DISKS, no /dev/*,
block/char devices, raw images, escaping symlinks or outside backing
chains), give each run its own OVMF variable file, QMP socket and state
directory, and use QMP plus argument arrays instead of shell interpolation.
Return nonzero on any failed assertion, timeout, EOF or missing result and
record PASS/FAIL/BLOCKED/SKIPPED/NOT RUN. Add 16 unit tests, a report
generator and a findings register; make test runs the harness suite.
2026-09-21 20:01:50 -07:00

96 lines
3.6 KiB
Bash

# Disposable UEFI VM harness
Proves that the installer produces a bootable encrypted system without ever
touching a physical disk. The target is always a qcow2 file.
## Files
- `run-vm.sh` interactive QEMU launcher (manual testing).
- `install.sh` guest-side, non-interactive installation (run in the VM).
- `serial-driver.py` host-side pexpect driver (`install` and `boot` modes).
- `test.conf.example` installer configuration used by the driver.
## Host prerequisites
- `qemu-system-x86_64`, `qemu-img`, OVMF firmware, Python `pexpect`.
- The reference Gentoo ISO.
- The static Rust installer (the guest never runs the Bash tree):
```sh
cd installer-rs
RUSTFLAGS="-C target-feature=+crt-static" \
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
```
## Procedure
Serve the static installer, the stage3 and the config to the guest:
```sh
cd /path/to/serve
cp /path/to/phoebus_os/installer-rs/target/x86_64-unknown-linux-gnu/release/phoebus-installer .
cp /path/to/phoebus_os/installer/tests/vm/install.sh .
cp /path/to/phoebus_os/installer/tests/vm/test.conf.example test.conf # edit TARGET_DISK etc.
sha256sum stage3.tar.xz # set STAGE3_SHA256
python3 -m http.server 8000 --bind 127.0.0.1
```
Run the installation (driver boots the ISO kernel/initrd directly and drives
the serial console):
```sh
QEMU_BIN=/path/to/qemu-system-x86_64 \
QEMU_LIBDIR=/path/to/qemu/libs \
QEMU_DATADIR=/path/to/qemu/share \
OVMF_CODE=/path/to/OVMF_CODE_4M.fd \
OVMF_VARS=/path/to/OVMF_VARS.fd \
DISK=/tmp/phoebus/target.qcow2 \
ISO=/path/to/livegui-amd64.iso \
KERNEL=/tmp/live/boot/gentoo INITRD=/tmp/live/boot/gentoo.igz \
INSTALL_URL=http://10.0.2.2:8000/install.sh \
python3 installer/tests/vm/serial-driver.py install
```
Then boot the disk alone (ISO detached) and verify:
```sh
DISK=/tmp/phoebus/target.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/serial-driver.py boot
```
To patch a console into an existing entry for observation, the BLS entry in
`/efi/loader/entries/*.conf` takes an `options` line; the installer can also
embed a console via `KERNEL_CONSOLE` (`GENTOO_KERNEL_CONSOLE` equivalent).
## Safety (enforced, not just documented)
`vmsafety.py` validates every writable drive before QEMU starts:
- `DISK`/`DISKS` must name **qcow2** files inside the disposable directory
`PHOEBUS_VM_DISKS` (default `$PHOEBUS_VM_STATE/disks`).
- `/dev/*` paths, block and character devices, non-regular files, raw images,
symlinks that resolve outside the directory, and disks whose backing chain
reaches outside it are **refused** with a nonzero exit.
- Each run gets a private state directory (mode 0700), its own writable OVMF
variable file and a unique QMP monitor socket.
- Screenshots use structured QMP and argument arrays; there is no shell
interpolation (`os.system`) in the harness.
- Secrets are passed through 0600 files and checked for leakage in the
installer log by the guest script.
## Results
Every driver writes `results.json` (and a `screenshot-manifest.json` for the
GUI driver) under its run directory and exits nonzero on any `FAIL` (and, with
`PHOEBUS_VM_STRICT=1`, on `BLOCKED`/`NOT RUN`). Statuses are `PASS`, `FAIL`,
`BLOCKED`, `SKIPPED` and `NOT RUN`; a timeout is always a `FAIL`, never a pass.
```sh
make test-vm # unit-test the safety and result logic (no VM needed)
PHOEBUS_VM_DISKS=/var/tmp/phoebus-vm/disks \
OVMF_CODE=/usr/share/OVMF/OVMF_CODE_4M.fd \
OVMF_VARS=/usr/share/OVMF/OVMF_VARS_4M.fd \
DISK=/var/tmp/phoebus-vm/disks/target.qcow2 \
ISO=installer/live/phoebus-installer-amd64.iso \
python3 installer/tests/vm/gui-driver.py gui
```