phoebus_os/docs/HANDOFF.md

275 lines
17 KiB
Markdown

# Handoff
Rolling note between work sessions. **Update this file before you finish**, then
commit it. It should be short and current; use `git log` for detail.
- **ISO rebuild + interactive VM (2026-09-26, in progress):** new ISO
`installer/live/phoebus-installer-amd64-20260926.iso` (1.6 GiB, SHA-256
`250dbe9a…`, build `20260926T041500Z`, from HEAD `11c0ee0`) built via
`PHOEBUS_REUSE_ROOTFS=1`; embedded `phoebus-installer`/`sys` hash-match
the fresh static build, Gentoo key has both fingerprints, UI payload
present. Interactive VM RUNNING (KVM/GTK, fresh
`phoebus-interactive-20260926.qcow2` + private vars in
`/var/tmp/phoebus-interactive/`, screenshot at welcome). Unpinned
signed-DIGESTS full install RUNNING (`gui-install-20260926T040728Z-f3ae68f4`
on `disks/unpinned-accept-1.qcow2`, guest past partitioning/LUKS/Btrfs;
`--no-ai` is test-only). `run-vm.sh` gained `--disk-name`/`--display` +
TCG `-cpu max` fallback. `gates --execute`: 0/0/6 NotRun, truthful, no
publish. Quality gates green (717/717, clippy -D warnings, vmsafety
18/18). Old Sep-24 ISO kept alongside. Next: finish unpinned
install+detached boot, tamper refusals, recovery-entry boot, seat
desktop, live AI, snap revision/strict, nix GC recheck — then commit docs
evidence.
- **Security/readiness follow-up (2026-09-26):** trust anchor + executable
gates + strengthened rollback PASS.
1. **Stage3 trust anchor (fixed):** `release_key_path()` ignores
`PHOEBUS_GENTOO_RELEASE_KEY`; production always uses the provisioned
key, test injection only via explicit `verify_digests_with_key`.
Host-level Good-signature check of the real DIGESTS passes (not VM
evidence). VM proof blocked: needs a rebuilt live ISO + disposable
install (exact prerequisites in `docs/VM-BOOT-VERIFICATION.md`).
2. **Release gates executable:** invalid `checks`/`desktop` modes fixed
(real `boot` modes), `--execute` runs each argv and reports
Passed/Failed/NotRun from evidence (unconfigured host: 0/0/6, publish
refuses). Found and fixed a `gui-driver.py` SyntaxError. 6 new tests.
3. **Strengthened rollback rerun PASSED**
(`serial-rollback-20260926T034218Z-7fab51ef`, PASS=11 FAIL=0, exit 0,
no leak, overlay `rollback-accept-8.qcow2`): mandatory
`===ROLLBACK-MARKER present===` before undo, DB `ca398d71…`→`fda3780a…`
→restored, marker absent after detached reboot, `/home` survives,
systemd running. `SERVE` shellcheck fix included.
4. **Recovery-entry boot** is now a distinct manual gate with its own
procedure (NOT RUN). Roadmap no longer lists passed `sys undo` as
unfinished; Secure Boot filed as a separate opt-in milestone.
5. Quality gates: `cargo fmt --check`, `clippy --workspace --all-targets
-- -D warnings`, `cargo test --workspace` 717/717 (one
`assistant_bridge` flake under parallel load, green in isolation and
on rerun), release + static-pie builds, `test_vmsafety.py` 18/18,
`shellcheck` clean. Seat desktop, live AI, Snap revision/strict, and
NVIDIA stay open with exact prerequisites (STATUS).
- **Interface contract (unchanged):** ordinary requests use AI; only explicit
`--legacy` selects deterministic handling; source flags constrain the AI and
never bypass it; a missing model stops the workflow and offers the exact
`--legacy` command without executing it.
- **Hardening pass (2026-09-26):** stage3 fail-closed + truthful boot
security + rollback acceptance PASS.
1. **Stage3 (security defect, fixed):** the unpinned path warned and
continued on DIGESTS-download failure, missing SHA-512 entry, missing
gpg/key, failed key import, and gpg launch errors. All stop the install
before extraction now; bad signature/hash also stop it. The pinned path
requires a valid 64-hex pin (placeholder refused) before any network
use. `stage_install` checks for `gpg`/`sha256sum`/`sha512sum` first.
16 new regression tests (every failure branch, nonzero-exit and
cannot-start doubles, both positive paths). The live image ships `gnupg`
plus the Gentoo release key, fingerprint-checked at build time against
Gentoo's published fingerprints (`13EB…2910` weekly, `D99E…2058`
signing). Commits `2162670`, `744059b`.
2. **Boot security (fixed):** `uki=true`/`secureboot=true` are rejected at
`validate_config` (shared by CLI/TUI/GUI bridge); the plan preview
always reports unsigned BLS. Parsing/validation/rendering regressions +
golden updated. Commit `95e0d05`. Authenticated boot stays an open
limitation (STATUS).
3. **Rollback acceptance PASSED for real**
(`serial-rollback-20260926T030237Z-912f6386`, PASS=11 FAIL=0, exit 0, no
secret leak): fresh overlay `rollback-accept-7.qcow2` on
`fresh-install-1.qcow2` — snapshot → `app-misc/hello` install (DB
`ca39…`→`fda3…`) → `sys undo <id>` rc=0 → detached reboot (DB restored,
root marker absent, `/home` survives, systemd running, recovery entry
present). The 2026-09-25 pass claim had no transcript and did not
reproduce; fixed harness defects: sudo auth (stale-prompt match bug
found and fixed with unique markers), root phases, explicit `--legacy`,
verified break (`nano` was pre-installed), targeted undo, marker-absent
assertion, preserved phase-1 log. Commits `744059b`, `5f69996`.
4. **Release candidate (local, isolated):** manifest (2 artifacts), verify,
tamper refusal, and publish refusal against the NotRun report all behave;
`gates --execute` is 0 passed / 6 NotRun (every gate needs VM/seat/model;
`run_gates` never executes gate commands itself). Nothing published.
5. Quality gates: `cargo fmt --check`, `cargo clippy --workspace
--all-targets -- -D warnings`, `cargo test --workspace` 712/712,
release + static-pie builds, `test_vmsafety.py` 18/18, `make doctor`
19 ok / 1 disk-space warning.
- **Interface contract (unchanged):** ordinary requests use AI; only explicit
`--legacy` selects deterministic handling; source flags constrain the AI and
never bypass it; a missing model stops the workflow and offers the exact
`--legacy` command without executing it.
- **Quality pass (2026-09-26):** boxed `ExecError::Command` details
(`CommandFailure`, 128 → 96 bytes, measured with a path-dependent probe crate;
constructor + all match sites updated, error text/redaction/stderr-tail
preserved, 4 regression tests), cleared `cargo clippy --workspace --all-targets
-- -D warnings` with no blanket allows (type alias, `CommandRecord` params
struct, `DiskPhase` renames), and split `service.rs` (4218 lines →
`service/mod,plan,validation,execution,tests`) and `gui_bridge.rs`
(2763 lines → `gui_bridge/mod,protocol,bridge,tests`) as structural refactors
with explicit re-exports and `pub(super)` seams — no `service.rs` left
alongside `service/mod.rs`. `make test` green (692/692, unchanged across the
splits). Release + static-pie builds pass. The disposable VM re-ran the
current 38-case `sys` acceptance matrix against the new static `sys`
(`70f0a72b…`): guest `PASS=38 FAIL=0`, driver `PASS=41 FAIL=0`
(`serial-boot-20260926T014746Z-3089a820`, overlay
`sys-acceptance-refactor.qcow2`). Commits `e69bfb9`, `1bf31c5`, `2dd48d1`.
- **Interface contract (unchanged):** ordinary requests use AI; only explicit
`--legacy` selects deterministic handling; source flags constrain the AI and
never bypass it; a missing model stops the workflow and offers the exact
`--legacy` command without executing it.
- **Snap first-use follow-up (2026-09-25):** the previous "no snapd" note was a
real integration gap. The normal `sys install … --legacy --snap` resolver needs
snapd to confirm a Store candidate before approval, while a raw typed
`snap_install` silently added it during execution. First-use setup is now a
separately reviewed Portage install plus socket enable/start, with exact
guidance when absent. Fresh installs set scoped
`sys-fs/squashfs-tools lzma lzo`, which Gentoo's snapd ebuild requires.
A disposable VM exposed a second bug: Portage candidate resolution picked
the first dependency (`dev-lang/go-bootstrap`) instead of the requested
`app-containers/snapd`; it now matches the requested atom. Snap confinement
is reported from the daemon rather than assumed strict. The disposable VM
completed setup, Store install, refresh and remove (`PASS=10 FAIL=0`), with
partial confinement. A fresh overlay and a clean reboot repeat with the final
static binary each passed `PASS=11 FAIL=0`, including clean poweroff. Refresh
found no newer revision.
- **Previous session:** second-round `sys` review fixes.
Follow-up to `d3f676a`. Three further confirmed defects are fixed with
regression tests: unsafe cleanup previews, non-exact Flatpak/Snap update
execution, and reverse-dependency self-reference. `make test` is green
(661 → 683 tests). This corrects the earlier blanket claim that every review
finding was already fixed.
- **Interface contract (unchanged):** ordinary requests use AI; only explicit
`--legacy` selects deterministic handling; source flags constrain the AI and
never bypass it; a missing model stops the workflow and offers the exact
`--legacy` command without executing it.
## Snap first-use changes
- `phoebus-core::system` generates scoped SquashFS USE flags for new installs.
- Portage candidate resolution selects the requested atom from the pretend
plan, even when dependencies precede it. A multi-package regression covers
the bug found by the VM.
- The Snap install path gives exact setup commands when snapd is absent; the
typed action no longer adds an unreviewed Portage dependency. Candidate
sandbox levels and missing `snap info` fields are treated conservatively.
- `installer/tests/vm/snap-guest.sh` drives the installed-system journey. The
serial driver can authenticate for boot checks and logs only guest output,
keeping sent test passwords out of its transcript.
## Previous sys review work
1. **Cleanup previews are non-destructive (finding 1).** `eclean-dist`/
`eclean-pkg` delete by default, so `portage_cleaners` now previews with
`--pretend --deep` (same selection policy as execution) and executes with
`--deep`; the Nix preview uses the documented `nix-store --gc --print-dead`.
Items are read from stdout+stderr, sorted and bound. Tests prove a cleaner
never runs destructively during planning, revalidation or cancelled approval.
2. **Exact Flatpak/Snap update transactions (finding 2).**
`ResolvedPlan.updates` carries `UpdateTransaction`s: full Flatpak ref + full
installed/remote commits (via `flatpak info`, since the `commit` column is
abbreviated), or snap name + old/new revisions + channel. Execution consumes
the reviewed transaction (`add_executable`… `flatpak update --commit=<c>`),
revalidates each source immediately before its mutation, verifies the
resulting commit/revision, and refuses non-exact transactions. A source whose
resolution failed is never re-discovered. `PlanStore` aggregation includes
`updates`.
3. **Reverse dependencies (finding 3).** The reverse branch no longer drops
valid matches; it excludes only a genuine self-reference (the *dependent* is
the queried package).
4. **Determinism.** `All selected packages:` removals are sorted (depclean
previews are now order-independent); `emerge --depclean --ask=n`.
## Verified in disposable VMs
- **Cache cleanup + depclean (real tools):** `eclean` upstream preview is
non-destructive; `sys clean cache` preview/cancelled/approved/drift/failed all
behave correctly; depclean cancel/approve with transaction history and
preserved `/home`.
- **Reverse dependencies:** `sys-libs/glibc` and `sys-libs/zlib` report real
dependents (conditional/versioned included).
- **Flatpak:** installed via Portage (`sys-apps/flatpak`, enabling
`media-libs/freetype[harfbuzz]`), flathub added, an app installed and
downgraded, then `sys apply updates --flatpak` resolved full ref+commits,
deployed the pinned commit, and verified it.
- **Custom ebuild:** stage → build/install (VDB entry + installed file) →
remove.
- **Managed AppImage:** acquire with pinned SHA-256, list, run the managed ELF,
remove; a placeholder SHA is refused.
- **Nix:** a `NIX_ENABLED=yes` install bootstraps Nix; `nix profile install
nixpkgs#nodejs-slim` into `/nix/var/nix/profiles/profile` is enumerated by
`sys list --nix` and found by `sys list nodejs --nix`, with store paths and
generations reported separately.
- **Non-root `pkexec sys --broker`:** with the shipped polkit action and the
standard wheel-admin rule installed on the disposable image, a wheel user
authenticated through a terminal agent; cancellations changed nothing, the
approved removal returned the broker JSON outcome, and the replay was
rejected. The shipped privilege policy is unchanged.
- **Read-only acceptance:** `installer/tests/vm/sys-acceptance.sh` `PASS=38
FAIL=0` re-run against the final static `sys` (`dd61c610…`), including the new
`rdeps-legacy` reverse-dependency regression.
- **Snap first use:** the disposable VM installed `app-containers/snapd` through
the reviewed Portage plan (six effects), enabled and started `snapd.socket`,
installed Snap Store `hello-world` 6.4 revision 29, ran refresh, then removed
it. The daemon reported `partial` confinement and the install review agreed.
The harness reported `PASS=10 FAIL=0`
(`serial-boot-20260926T001830Z-107996de`).
Final-binary runs on a fresh overlay passed `PASS=11 FAIL=0` on first boot
(`serial-boot-20260926T005327Z-c024cc43`) and after a clean reboot
(`serial-boot-20260926T010605Z-e6193c3a`); both shut down cleanly.
## Not completed / blocked
- **Stage3 VM proof with the new binary:** unit-tested (19 stage3 tests) and
the pinned path is unchanged for real pins, but no full install has run
with the fail-closed binary yet; the next disposable install is the proof.
The unpinned (signed-DIGESTS) VM proof additionally needs a rebuilt live
ISO carrying the provisioned key (`installer/live/build.sh` now provisions
it; no ISO was rebuilt in this session).
- **Installed Mango + DMS session under a seat:** still needs a
seat-providing graphical VM (or physical VT): login, kitty, DMS
spotlight, lock/idle, polkit agent, logout, reboot
(`docs/DEVELOPMENT.md` physical-VT procedure).
- **Snap update to a different revision / strict confinement:** refresh with no
newer revision and partial confinement were VM-verified. A revision-changing
refresh and AppArmor-backed strict confinement were not (need an
AppArmor-ready guest kernel with loaded policy, and a Store snap that
actually has a newer revision than installed).
- **Live AI:** a local OpenAI-compatible model (`llama-server` + a GGUF model)
was served on the host; the AI workflow reached the model, but the sandboxed
call exceeded the 120 s transport timeout on CPU (`curl` exit 28). Needs a
faster model/GPU or a remote provider credential. Fixture-backed routing and
failure tests pass (`ask-offline`, missing-model offer, source constraint).
- **Recovery boot (distinct from rollback):** the recovery BLS entry is present
and asserted in the rollback acceptance, but booting it interactively was not
automated. Method: select "Phoebus Recovery (known-good snapshot)" in the
systemd-boot menu with no network.
- **Nix re-run flakiness:** a later `sys list --nix` re-run reported the profile
link dangling (the profile store path was gone) and correctly listed no
packages; the first run with a real profile is the positive evidence.
## Known blockers / risks
- VM serial login is occasionally flaky; retry the boot when `detached-boot-login`
fails.
- `make doctor` passes (KVM accessible via an ACL). Disk is shared by build and
VM; watch free space.
- The pre-existing `test-disk.qcow2` cannot be unlocked (unknown passphrase); do
not guess or brute-force it.
## Next actions (in order)
1. **Stage3 install proof:** next disposable install with the fail-closed
binary (pinned path), then a rebuilt live ISO (with the provisioned
Gentoo key) proving the signed-DIGESTS path in the VM.
2. **Recovery boot** by selecting the recovery BLS entry with the network down.
3. **Live AI** with a faster local model/GPU or an authorized provider
credential (natural-language discovery, source constraints, reviewed install).
4. **Installed desktop under a seat** (Mango + DMS login through reboot).
5. **Nix profile GC stability** if the dangling re-run is reproducible.
6. **Snap strict confinement and changed-revision update** in a disposable VM
when an AppArmor-ready kernel and a revision-changing Store test are available.
## Definition of a good stopping point
- `make test` passes; `docs/STATUS.md`, `docs/CHANGELOG.md` and this file
updated; committed (`scripts/agent-commit.sh`) and pushed
(`scripts/publish.sh`).