Validate every writable drive (qcow2, inside PHOEBUS_VM_DISKS, no /dev/*, block/char devices, raw images, escaping symlinks or outside backing chains), give each run its own OVMF variable file, QMP socket and state directory, and use QMP plus argument arrays instead of shell interpolation. Return nonzero on any failed assertion, timeout, EOF or missing result and record PASS/FAIL/BLOCKED/SKIPPED/NOT RUN. Add 16 unit tests, a report generator and a findings register; make test runs the harness suite.
66 lines
5.6 KiB
Markdown
66 lines
5.6 KiB
Markdown
# Verification and findings register
|
|
|
|
Companion to `docs/STATUS.md`. The machine-readable per-run results live outside
|
|
git under `/var/tmp/phoebus-vm/<run-id>/results.json`; this file is the
|
|
human-readable register of what was found, fixed and still unverified.
|
|
|
|
## Run matrix (this session, 2026-09-22)
|
|
|
|
```json
|
|
{
|
|
"source_revision": "7e9fa9c",
|
|
"iso_sha256": "428dbd7547db8690596bf5c3a0d7a6fa8aee4c9f9eca1e926c216712dd050b36",
|
|
"static_binary_sha256": "2f9cd4b36a12e469d2b6187166acaf8b11f7f382cc19f7ad08bb2a809c855591",
|
|
"runs": [
|
|
{"scenario": "gui", "result": "PASS 23/23", "exit_code": 0,
|
|
"artifacts": "/var/tmp/phoebus-vm/gui-gui-20260922T025537Z-69f754cf"},
|
|
{"scenario": "make test (rust + vmsafety)", "result": "PASS", "exit_code": 0},
|
|
{"scenario": "cargo fmt --check", "result": "PASS", "exit_code": 0},
|
|
{"scenario": "cargo clippy --workspace --all-targets", "result": "PASS (warnings only)", "exit_code": 0},
|
|
{"scenario": "opencode probe (live 1.18.32)", "result": "ready", "exit_code": 0},
|
|
{"scenario": "opencode probe (unauth LAN bind)", "result": "authentication_required", "exit_code": 1},
|
|
{"scenario": "noninteractive install + detached boot", "result": "NOT RUN on the final ISO", "exit_code": null},
|
|
{"scenario": "full graphical install past AI", "result": "BLOCKED", "exit_code": null},
|
|
{"scenario": "rollback VM acceptance", "result": "NOT RUN", "exit_code": null},
|
|
{"scenario": "AI sandbox canary VM test", "result": "NOT RUN", "exit_code": null},
|
|
{"scenario": "second physical computer OpenCode transport", "result": "BLOCKED", "exit_code": null}
|
|
]
|
|
}
|
|
```
|
|
|
|
## Confirmed defects fixed
|
|
|
|
| ID | Defect | Symptom | Fix | Regression test |
|
|
|----|--------|---------|-----|-----------------|
|
|
| D1 | Disk safety was not revalidated before destroying the target | A disk that disappeared, was mounted, became the live medium, or was replaced under the same name between selection and install would be partitioned on the strength of a stale review | `Installer::revalidate_target_disk` re-probes and compares identity in `storage_prepare` | `orchestrate::tests::{revalidation_refuses_a_disappeared_disk,revalidation_refuses_a_now_protected_disk,revalidation_refuses_reused_device_name_with_a_new_size,...}` |
|
|
| D2 | `probe_disks` parsed human-readable `lsblk` sizes as bytes | Every disk displayed `0 B`; the size-identity check was blind | pass `-b` to `lsblk` | `probe::tests::probe_disks_classifies_optical_as_protected` asserts the byte size |
|
|
| D3 | The bridge could replay or overlap a destructive install | A second `start_install` (retry, second frontend, reconnect) could re-run storage preparation | approval is single-use, consumed before work; `installing`/`installed` guards | `start_install_cannot_be_replayed_after_success` |
|
|
| D4 | The welcome's primary action was not keyboard-focused | No focus ring; Return did not open the installer | request activation + `forceActiveFocus()`; regression test also rejects the invalid `onActiveChanged` signal | `welcome_primary_action_is_keyboard_reachable` |
|
|
| D5 | Escape during installation silently closed the frontend | A destructive operation could be abandoned with no explanation | Escape is refused during progress and the UI says so | `progress_cannot_be_abandoned_silently` |
|
|
| D6 | Provider transport accepted unsafe URL schemes / unbounded responses / followed no protocol allowlist | A config-supplied endpoint could target `file:`, inject curl config, or return a huge body | `validate_endpoint`, `proto = "=http,https"`, `max-filesize`, redirects disabled | `endpoint_validation_rejects_unsafe_schemes_and_injection`, `curl_transport_refuses_an_unsafe_endpoint`, `curl_transport_bounds_the_response_and_disables_redirects` |
|
|
| D7 | `TargetRoot` followed symlinks out of the target | A planted symlink could redirect an installer write to the host | `reject_symlink_components` in `write`/`ensure_dir` | `target::tests::{refuses_to_write_through_an_escaping_symlink,refuses_a_symlinked_leaf}` |
|
|
| D8 | The VM harness documented disk safety but did not enforce it, and reported observations rather than results | A mistyped `DISK` could point at a real device; a timeout could look like success; shell interpolation in screenshot conversion | `vmsafety.py`; per-run OVMF/QMP; structured results; argument arrays | `test_vmsafety.py` (16 tests), run by `make test` |
|
|
|
|
## Suspected risks (reproduced or bounded)
|
|
|
|
- **Prompt injection in device labels / model output.** Typed proposals pass
|
|
`deny_unknown_fields` and a closed action set; device labels are display-only.
|
|
Bounded by tests (`phoebus-ai` parse/allowlist). No VM canary run.
|
|
- **Live sudoers is `NOPASSWD: ALL` (live-only).** It does not reach the
|
|
installed system, but the GUI bridge uses it. Narrowing is a recorded follow-up.
|
|
- **OpenCode `/config/providers` returns provider credentials in cleartext.**
|
|
Upstream behaviour; the probe keeps only counts and docs mandate loopback +
|
|
tunnel. Not a Phoebus defect, but a deployment hazard.
|
|
|
|
## Missing verification (blocked / not run)
|
|
|
|
- Full graphical installation past the AI step and two fresh-disk graphical
|
|
installs: **BLOCKED** — no provider key and no local model; normal install
|
|
requires a connected model.
|
|
- `install`/`boot` modes on the final ISO: **NOT RUN** (the earlier appliance
|
|
build passed `INSTALL-RC=0` + detached boot; the final ISO's binary changed,
|
|
so this is not automatically re-verified).
|
|
- Rollback acceptance, AI sandbox canary, installed Mango/dwl, physical seat,
|
|
NVIDIA: **NOT RUN / unverified**.
|
|
- Second physical computer OpenCode transport: **BLOCKED**; host-local forwarded
|
|
hop and unauthenticated-LAN refusal were tested instead.
|