phoebus_os/installer/tests/vm/README.md
2026-10-12 04:27:15 -07:00

184 lines
8.4 KiB
Bash

# Disposable UEFI VM harness
Proves that the installer produces a bootable encrypted system without ever
touching a physical disk. The target is always a qcow2 file.
## Files
- `run-vm.sh` interactive QEMU launcher (manual testing).
- `install.sh` guest-side, non-interactive installation (run in the VM).
- `serial-driver.py` host-side pexpect driver (`install`, `install-repeat`,
`boot`, `boot-assessed`, `recovery`, `fallback`, `rollback`,
`interrupted-update` modes).
- `rollback-guest.sh`, `rollback-verify.sh` guest-side rollback acceptance
(fetch `sys`, snapshot, break, undo, verify).
- `interrupted-update.sh` kills an active `sys`/Portage package transaction,
recovers its lock, undoes its snapshot, and verifies after reboot.
- `snap-guest.sh` guest-side Snap setup/install/refresh/remove acceptance.
- `test.conf.example` installer configuration used by the driver.
## Host prerequisites
- `qemu-system-x86_64`, `qemu-img`, OVMF firmware, Python `pexpect`; the
graphical gate also needs Pillow.
- The current Phoebus Debian-live ISO (the installed target remains Gentoo).
- The static Rust installer (the guest never runs the Bash tree):
```sh
cd installer-rs
RUSTFLAGS="-C target-feature=+crt-static" \
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
```
## Procedure
Serve the pinned stage3 and test config to the guest. The installer and `sys`
run from the candidate ISO itself:
```sh
cd /path/to/serve
cp /path/to/phoebus_os/installer/tests/vm/install.sh .
cp /path/to/phoebus_os/installer/tests/vm/test.conf.example test-mango.conf # set the real stage3 digest
sha256sum stage3.tar.xz # set STAGE3_SHA256
python3 -m http.server 8000 --bind 0.0.0.0
```
Run the installation (driver boots the ISO kernel/initrd directly and drives
the serial console):
```sh
QEMU_BIN=/path/to/qemu-system-x86_64 \
QEMU_LIBDIR=/path/to/qemu/libs \
QEMU_DATADIR=/path/to/qemu/share \
OVMF_CODE=/path/to/OVMF_CODE_4M.fd \
OVMF_VARS=/path/to/OVMF_VARS.fd \
DISK=/tmp/phoebus/target.qcow2 \
ISO=/path/to/phoebus-installer-amd64.iso \
KERNEL=/path/to/live/vmlinuz INITRD=/path/to/live/initrd.img \
INSTALL_URL=http://10.0.2.2:8000/install.sh \
python3 installer/tests/vm/serial-driver.py install
```
The install gate defaults to `test-mango.conf` (Mango, Flatpak and Nix). Set
`VM_INSTALL_CONFIG=test.conf` only for a separate minimal-profile exercise.
Then boot the disk alone (ISO detached) and verify:
```sh
DISK=/tmp/phoebus/target.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/serial-driver.py boot
```
To patch a console into an existing entry for observation, the BLS entry in
`/efi/loader/entries/*.conf` takes an `options` line; the installer can also
embed a console via `KERNEL_CONSOLE` (`GENTOO_KERNEL_CONSOLE` equivalent).
## Rollback acceptance (`serial-driver.py rollback`)
`rollback` drives the real rollback path end to end on an
**already-installed** disposable disk:
1. boot the installed disk, unlock LUKS and log in;
2. fetch `sys` and `rollback-guest.sh` from the host (the installed system
carries `/usr/local/bin/sys`; the host copy is a fallback), authenticate
`sudo` (the password is sent only at sudo's own no-echo prompt, never on a
command line), record the `/var/db/pkg` fingerprint and a `/home` marker,
create a recovery point with `sys snapshot`, install a small package (a
controlled, reversible update) and run `sys undo` — all as root, because
unprivileged runs would be refused and prove nothing;
3. power off, boot the disk detached, and run `rollback-verify.sh` (also as
root), comparing the package fingerprint and the `/home` marker, checking
kernel modules, systemd state, and reading the snapshot recovery entry and
`sys rollback status`.
Both transcripts are kept (`serial-phase1.log`, `serial.log`). The guest
scripts refuse to run unprivileged (`ROLLBACK-NEEDS-ROOT` /
`AFTER-NEEDS-ROOT`) so a lost `sudo` timestamp fails loudly instead of
recording vacuous passes.
Serve the fixtures as well as the installer:
```sh
cp installer/tests/vm/rollback-guest.sh installer/tests/vm/rollback-verify.sh /path/to/serve/
cp installer-rs/target/x86_64-unknown-linux-gnu/release/sys /path/to/serve/
DISK=/tmp/phoebus/installed-clone.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/serial-driver.py rollback
```
Point `DISK` at a **disposable clone** of the evidence image so the original is
preserved. `serial-driver.py recovery` selects the actual systemd-boot recovery
entry with networking disabled and checks the read-only snapshot and `sys`.
`fallback` and `interrupted-update` create their own qcow2 overlays beneath
`PHOEBUS_VM_DISKS`, leaving the installed evidence image unchanged. The
`install-repeat` mode requires a distinct `DISK_SECOND` and checks its resolved
path before starting QEMU.
## Snap acceptance on an installed disposable image
Serve the current static `sys` binary and `snap-guest.sh` from a small local
directory. Boot an installed **qcow2 overlay** with `serial-driver.py boot`,
setting `BOOT_AUTHENTICATE=1`, `BOOT_POWEROFF=1`, `CHECKS_TIMEOUT=7200`,
and the following checks. `BOOT_POWEROFF` cleanly shuts down the guest after
mutating checks so a later boot observes committed state:
```sh
BOOT_REQUIRE_CASES=snap-sys,snap-portage,snap-socket,snap-confinement,snap-install,snap-refresh,snap-remove
BOOT_CHECKS='echo ===CHECKS-$(printf START)===; echo SYSTEMD_STATUS:$(systemctl is-system-running); curl -fsSL http://10.0.2.2:8000/snap-guest.sh -o /tmp/snap-guest.sh; sudo bash /tmp/snap-guest.sh; echo ===CHECKS-$(printf END)==='
```
The guest script adds only the scoped `sys-fs/squashfs-tools lzma lzo` USE
setting to the disposable image, then uses `sys` for the reviewed Portage
install, socket activation and Snap actions. The boot driver logs only guest
output, not the test passwords it sends for LUKS, login and `sudo`.
## Safety (enforced, not just documented)
`vmsafety.py` validates every writable drive before QEMU starts:
- `DISK`/`DISKS` must name **qcow2** files inside the disposable directory
`PHOEBUS_VM_DISKS` (default `$PHOEBUS_VM_STATE/disks`).
- `/dev/*` paths, block and character devices, non-regular files, raw images,
symlinks that resolve outside the directory, and disks whose backing chain
reaches outside it are **refused** with a nonzero exit.
- Each run gets a private state directory (mode 0700), its own writable OVMF
variable file and a unique QMP monitor socket.
- Screenshots use structured QMP and argument arrays; there is no shell
interpolation (`os.system`) in the harness.
- Secrets are passed through 0600 files and checked for leakage in the
installer log by the guest script. Host serial transcripts and structured
result details redact registered test credentials, including output split
across pexpect reads.
## Results
Every driver writes `results.json` (and a `screenshot-manifest.json` for the
GUI driver) under its run directory and exits nonzero on any `FAIL` (and, with
`PHOEBUS_VM_STRICT=1`, on `BLOCKED`/`NOT RUN`). Statuses are `PASS`, `FAIL`,
`BLOCKED`, `SKIPPED` and `NOT RUN`; a timeout is always a `FAIL`, never a pass.
```sh
make test-vm # unit-test the safety and result logic (no VM needed)
PHOEBUS_VM_DISKS=/var/tmp/phoebus-vm/disks \
OVMF_CODE=/usr/share/OVMF/OVMF_CODE_4M.fd \
OVMF_VARS=/usr/share/OVMF/OVMF_VARS_4M.fd \
DISK=/var/tmp/phoebus-vm/disks/target.qcow2 \
ISO=installer/live/phoebus-installer-amd64.iso \
python3 installer/tests/vm/gui-driver.py gui
```
## Agentic hardening acceptance
`agentic-hardening-guest.sh` requires QEMU/KVM and root, and is intended only
for an installed disposable overlay. Supply the current static binary as
`/tmp/phoebus-hardening-sys` and the static `isolation_acceptance` test executable
as `/tmp/phoebus-hardening-isolation`. Run the two user probes as the normal
account with `PHOEBUS_ISOLATION_FIXTURE_ROOT=$HOME` and
`--ignored --skip model_system_unit`; the root fixture runs the system-manager
probe separately. It also exercises deterministic history, descriptor-backed
snapshots, exact Portage installation/removal, persistent lock permissions,
Flatpak commit installation/removal, and the active Nix store-path pin.
Serve the files through the existing serial boot harness, authenticate sudo,
require every reported case, and request a clean poweroff. Never reuse the
sole source-build evidence image as a writable target: create a checked qcow2
backing overlay first. Preserve failed diagnostics and distinguish fixture
errors from product failures.