docs(evidence): record the iterative VM sys verification loop
Some checks are pending
ci / test (push) Waiting to run
Some checks are pending
ci / test (push) Waiting to run
This commit is contained in:
parent
1d982a3760
commit
685321f85b
4 changed files with 62 additions and 4 deletions
|
|
@ -23,6 +23,13 @@ this file is the curated, human-readable summary.
|
||||||
`AI_ROOT_RC=0` (system sandbox), `AI_USER_RC=0` (`--user` sandbox, was
|
`AI_ROOT_RC=0` (system sandbox), `AI_USER_RC=0` (`--user` sandbox, was
|
||||||
RC=124 timeout), `AI_NOSANDBOX_RC=0`, hostile `run_shell` reply `AI_SHELL_RC=1`;
|
RC=124 timeout), `AI_NOSANDBOX_RC=0`, hostile `run_shell` reply `AI_SHELL_RC=1`;
|
||||||
the expanded `sys-acceptance.sh` matrix reports `SUMMARY PASS=49 FAIL=0`.
|
the expanded `sys-acceptance.sh` matrix reports `SUMMARY PASS=49 FAIL=0`.
|
||||||
|
Additional loops on the same golden overlay pass: mutating acceptance as root
|
||||||
|
`SUMMARY PASS=42 FAIL=0` (`mutate-install`/`mutate-remove`/`mutate-history`),
|
||||||
|
rollback acceptance `PASS=11` (snapshot → change → `sys undo` → detached
|
||||||
|
verify), and the Snap journey `PASS=11`
|
||||||
|
(`serial-boot-20260929T093329Z-4ef893cf`,
|
||||||
|
`serial-rollback-20260929T094640Z-256c2ba1`,
|
||||||
|
`serial-boot-20260929T094957Z-3d4a9211`).
|
||||||
Also observed: a read-only `flatpak search` can trigger Flatpak's interactive
|
Also observed: a read-only `flatpak search` can trigger Flatpak's interactive
|
||||||
`appstream-update` polkit prompt when the session is not recognized as active
|
`appstream-update` polkit prompt when the session is not recognized as active
|
||||||
(headless serial), which can block an unattended run; on a real active seat
|
(headless serial), which can block an unattended run; on a real active seat
|
||||||
|
|
|
||||||
|
|
@ -40,8 +40,13 @@ commit it. It should be short and current; use `git log` for detail.
|
||||||
`sys-acceptance.sh` `SUMMARY PASS=49 FAIL=0`. Hypothesis recorded (not
|
`sys-acceptance.sh` `SUMMARY PASS=49 FAIL=0`. Hypothesis recorded (not
|
||||||
changed): headless/inactive-session `flatpak search` can trigger Flatpak's
|
changed): headless/inactive-session `flatpak search` can trigger Flatpak's
|
||||||
interactive `appstream-update` polkit prompt; the unattended acceptance runs
|
interactive `appstream-update` polkit prompt; the unattended acceptance runs
|
||||||
detached to avoid the hang. Next: mutate acceptance, rollback, Snap, source
|
detached to avoid the hang. Additional loops on the same golden overlay with
|
||||||
conversion / tuned kernel on the same golden overlay.
|
the fixed binary: mutating acceptance as root `SUMMARY PASS=42 FAIL=0`
|
||||||
|
(`serial-boot-20260929T093329Z-4ef893cf`), rollback `PASS=11`
|
||||||
|
(`serial-rollback-20260929T094640Z-256c2ba1`), Snap journey `PASS=11`
|
||||||
|
(`serial-boot-20260929T094957Z-3d4a9211`). Next: source conversion / tuned
|
||||||
|
kernel (multi-hour compiles) and a live direct-provider model in the VM remain
|
||||||
|
NotRun.
|
||||||
- **Boundary-hardening pass (2026-09-28, IMPLEMENTED + UNIT-TESTED, VM NOT
|
- **Boundary-hardening pass (2026-09-28, IMPLEMENTED + UNIT-TESTED, VM NOT
|
||||||
RUN):** six host-provable defects fixed with regression tests. (1) Removed
|
RUN):** six host-provable defects fixed with regression tests. (1) Removed
|
||||||
the `PHOEBUS_ALLOW_UNPRIVILEGED_EXEC=1` env backdoor: it made
|
the `PHOEBUS_ALLOW_UNPRIVILEGED_EXEC=1` env backdoor: it made
|
||||||
|
|
|
||||||
|
|
@ -112,11 +112,11 @@ Booting that recovery entry offline remains a manual gate.
|
||||||
| Personal preferences + alert state (user-owned, system-default precedence) | yes | yes (save/reload/isolation/precedence/failure) | yes (installed VM: saved to user config, exit 0) | no |
|
| Personal preferences + alert state (user-owned, system-default precedence) | yes | yes (save/reload/isolation/precedence/failure) | yes (installed VM: saved to user config, exit 0) | no |
|
||||||
| Personal write ownership (elevated chown, symlink refusal, cross-uid refusal) | yes | yes (ownership-fixer seam + CLI refusal) | no | no |
|
| Personal write ownership (elevated chown, symlink refusal, cross-uid refusal) | yes | yes (ownership-fixer seam + CLI refusal) | no | no |
|
||||||
| Personal preference does not need the root-owned system transaction log | yes | yes (CLI regression: read-only transaction dir) | yes (installed VM acceptance) | n/a |
|
| Personal preference does not need the root-owned system transaction log | yes | yes (CLI regression: read-only transaction dir) | yes (installed VM acceptance) | n/a |
|
||||||
| Installed `sys` acceptance matrix (read-only/failure) | yes | yes | yes (installed VM: `PASS=40 FAIL=0`, catalogue + regression group; **plus service-lifecycle, versioned-atom and path-normalization cases added 2026-09-28 — host-level `tests/sys_flaws.rs` 9/9 PASS, VM rerun pending**) | no |
|
| Installed `sys` acceptance matrix (read-only/failure) | yes | yes | yes (installed VM, HEAD binary on a golden overlay `serial-boot-20260929T092147Z-6f88907a`: read-only `SUMMARY PASS=49 FAIL=0`; root mutate `serial-boot-20260929T093329Z-4ef893cf`: `SUMMARY PASS=42 FAIL=0` incl. `mutate-install`/`mutate-remove`/`mutate-history`) | no |
|
||||||
| `sys` adversarial input battery (injection/traversal/boundaries) | yes | yes (54-case `tests/sys_adversarial.rs`; five validation gaps fixed: file-path metacharacters + `..`, application-name traversal, ebuild `..` category (real overlay escape), Flatpak empty element, Nix option prefix) | no | no |
|
| `sys` adversarial input battery (injection/traversal/boundaries) | yes | yes (54-case `tests/sys_adversarial.rs`; five validation gaps fixed: file-path metacharacters + `..`, application-name traversal, ebuild `..` category (real overlay escape), Flatpak empty element, Nix option prefix) | no | no |
|
||||||
| Portage config readable for non-root queries (`normalize_config_permissions`) | yes | yes (modes + keyring exclusion) | yes (fresh VM install: 0755/0644 out of the box; non-root `search` returns 3 results, `updates` reports 10) | no |
|
| Portage config readable for non-root queries (`normalize_config_permissions`) | yes | yes (modes + keyring exclusion) | yes (fresh VM install: 0755/0644 out of the box; non-root `search` returns 3 results, `updates` reports 10) | no |
|
||||||
| Privileged catalogue mutations (install/remove/depclean/cache/apply updates, alternate-root) | yes | yes (unit) | yes (fresh installed VM via root: 7/7 PASS) | no |
|
| Privileged catalogue mutations (install/remove/depclean/cache/apply updates, alternate-root) | yes | yes (unit) | yes (fresh installed VM via root: 7/7 PASS) | no |
|
||||||
| Rollback acceptance (snapshot -> change -> `sys undo` -> detached verify) | yes | yes | yes (`serial-rollback-20260926T030237Z-912f6386` PASS=11 FAIL=0 on a fresh disposable clone: package DB changed then restored, root marker gone, `/home` survives, boot running, no secret leak) | no |
|
| Rollback acceptance (snapshot -> change -> `sys undo` -> detached verify) | yes | yes | yes (`serial-rollback-20260926T030237Z-912f6386` PASS=11 FAIL=0; re-verified with the HEAD binary `serial-rollback-20260929T094640Z-256c2ba1` PASS=11 on a fresh disposable overlay: package DB changed then restored, root marker gone, `/home` survives, systemd running, recovery entry present) | no |
|
||||||
| Privileged `--broker` plan execution + replay rejection in a VM (pkexec path) | yes | yes (privileged entry point) | yes (non-root `pkexec` with a terminal polkit agent: cancelled auth, broker JSON success, replay rejected; required the shipped policy + wheel-admin rule) | n/a |
|
| Privileged `--broker` plan execution + replay rejection in a VM (pkexec path) | yes | yes (privileged entry point) | yes (non-root `pkexec` with a terminal polkit agent: cancelled auth, broker JSON success, replay rejected; required the shipped policy + wheel-admin rule) | n/a |
|
||||||
| Cache-cleanup preview is non-destructive (real `eclean`/`nix-store`) | yes | yes (preview uses `--pretend --deep` / `--print-dead`; sorted bound items) | yes (real `eclean`: upstream preview, cancelled approval, approved run, drift refusal, failed cleaner; user data preserved) | n/a |
|
| Cache-cleanup preview is non-destructive (real `eclean`/`nix-store`) | yes | yes (preview uses `--pretend --deep` / `--print-dead`; sorted bound items) | yes (real `eclean`: upstream preview, cancelled approval, approved run, drift refusal, failed cleaner; user data preserved) | n/a |
|
||||||
| Reverse-dependency inspection includes installed targets | yes | yes (self-reference compares the dependent, not the target) | yes (real `sys-libs/glibc`, `sys-libs/zlib` dependents) | n/a |
|
| Reverse-dependency inspection includes installed targets | yes | yes (self-reference compares the dependent, not the target) | yes (real `sys-libs/glibc`, `sys-libs/zlib` dependents) | n/a |
|
||||||
|
|
|
||||||
|
|
@ -3,6 +3,52 @@
|
||||||
This records the first successful end-to-end install and boot of Phoebus OS
|
This records the first successful end-to-end install and boot of Phoebus OS
|
||||||
against a disposable qcow2 disk. No physical disk was ever touched.
|
against a disposable qcow2 disk. No physical disk was ever touched.
|
||||||
|
|
||||||
|
## 2026-09-29 Iterative `sys` VM loop on the golden overlay (AI sandbox + journeys)
|
||||||
|
|
||||||
|
Fresh qcow2 overlays were created on the golden installed image
|
||||||
|
(`/var/tmp/phoebus-vm/disks/phoebus-headless-80g.qcow2`) and booted detached with
|
||||||
|
the static `sys` built from HEAD; the base image was never modified.
|
||||||
|
|
||||||
|
- **Loop 1 (`serial-boot-20260929T074650Z-2f8fff0b`) found two real defects:**
|
||||||
|
(a) unprivileged `sys ask` hung on a polkit
|
||||||
|
`org.freedesktop.systemd1.manage-units` password prompt because the model
|
||||||
|
sandbox used the system manager (`RC=124` after a 200 s timeout); (b) an
|
||||||
|
unprivileged read-only AI plan printed the plan then exited 1 because the
|
||||||
|
service layer required the root-owned `/var/lib/phoebus/transactions.toml`.
|
||||||
|
- **Loop 2 (`serial-boot-20260929T075731Z-e4696546`) hit Flatpak's interactive
|
||||||
|
`appstream-update` polkit prompt** during a read-only `flatpak search` on the
|
||||||
|
headless serial session (the session is not recognized as polkit-active).
|
||||||
|
Polkit's upstream default is `allow_active=yes`, so a real active seat is not
|
||||||
|
affected; the unattended acceptance now runs detached so it cannot hang.
|
||||||
|
- **Fixes:** sandbox uses `systemd-run --user` with the user-manager property
|
||||||
|
subset for unprivileged callers (full system sandbox for root); transaction
|
||||||
|
recording is best-effort for non-root plans and mandatory for privileged
|
||||||
|
mutations.
|
||||||
|
- **Loop 3 (`serial-boot-20260929T084043Z-cfa7f7a5`):** read-only acceptance
|
||||||
|
reached `SUMMARY PASS=49 FAIL=0`; the root sandbox test timed out only because
|
||||||
|
the cached `sudo` timestamp expired during the long matrix (harness ordering).
|
||||||
|
- **Loop 4 (`serial-boot-20260929T092147Z-6f88907a`, driver PASS=8):**
|
||||||
|
system sandbox `AI_ROOT_RC=0`; user sandbox `AI_USER_RC=0` (was 124);
|
||||||
|
no-sandbox `AI_NOSANDBOX_RC=0`; hostile model `run_shell` reply
|
||||||
|
`AI_SHELL_RC=1`; read-only acceptance `SUMMARY PASS=49 FAIL=0`; 0 failed
|
||||||
|
units; clean poweroff.
|
||||||
|
- **Loop 5 (`serial-boot-20260929T093329Z-4ef893cf`, driver PASS=7):**
|
||||||
|
mutating acceptance as root `SUMMARY PASS=42 FAIL=0`, including
|
||||||
|
`mutate-install`, `mutate-remove`, `mutate-history` on `app-misc/hello`.
|
||||||
|
- **Loop 6 (`serial-rollback-20260929T094640Z-256c2ba1`, PASS=11):**
|
||||||
|
snapshot → change → `sys undo` → detached verify with the new binary
|
||||||
|
(`rollback-package-state-restored`, `rollback-home-survives`,
|
||||||
|
`rollback-root-restored`, `rollback-recovery-entry`, `rollback-status-recorded`).
|
||||||
|
- **Loop 7 (`serial-boot-20260929T094957Z-3d4a9211`, PASS=11):** Snap journey
|
||||||
|
(`snap-sys`, `snap-portage`, `snap-socket`, `snap-confinement`, `snap-install`,
|
||||||
|
`snap-refresh`, `snap-remove`); confinement remains `partial`, and a
|
||||||
|
revision-changing refresh / strict AppArmor confinement still needs an
|
||||||
|
external Store revision and an AppArmor-ready guest kernel.
|
||||||
|
- **NotRun in this loop:** source conversion and tuned-kernel build/boot/fallback
|
||||||
|
(multi-hour compiles on the disposable image); a live direct-provider model in
|
||||||
|
the VM (no credential/GPU); recovery-entry BLS selection (manual gate);
|
||||||
|
NVIDIA.
|
||||||
|
|
||||||
## 2026-09-28 Headless dual-target golden install (Run 13) & detached boot proven
|
## 2026-09-28 Headless dual-target golden install (Run 13) & detached boot proven
|
||||||
|
|
||||||
- **Golden install completed (Run 13, `serial-install-20260928T002557Z-e39d1934`):**
|
- **Golden install completed (Run 13, `serial-install-20260928T002557Z-e39d1934`):**
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue