phoebus_os/docs/HANDOFF.md
2026-09-26 10:08:47 -07:00

19 KiB
Raw Blame History

Handoff

Rolling note between work sessions. Update this file before you finish, then commit it. It should be short and current; use git log for detail.

  • Workstation experience session (2026-09-26, IMPLEMENTED + UNIT-TESTED, VM BLOCKED): Mango workstation atoms (22 apps, exact, no substitute; Kitty kept, Nemo default, CUPS local-only, sshd installed/disabled), Brave stable scoped overlay (pin pre-network fail-closed; no pin recorded yet), Zsh-by-default

    • packaged plugins, Vanilla-DMZ 24 cursors, default ccache 2G, AI reuse handoff (per-user 0600 key, truthful status, no Approve&Execute on errors), startup-check model (blur optional, PPD+D-Bus, matugen, cava) with truthful summary, assistant dark-default + light + responsive + token-drift guard, 60/20 GiB disk gates. Gates: fmt, clippy -D, 736/736 tests, release build, vmsafety 18/18, shellcheck info-only. NO new ISO (8 GiB free, needs ≥20; target needs 60–80), NO install/boot this session — all VM gates NotRun with exact unblock in STATUS. Next: free ≥80 GiB, pin Brave commit, build ISO, disposable 80 GiB install + detached boot, seat-VM compositor/cursor/ DMS/theme checks, fixture + live-AI handoff, local CUPS job, sshd-disabled
    • PPD D-Bus recheck after reboot.
  • Interface contract (unchanged): ordinary requests use AI; only explicit --legacy selects deterministic handling; source flags constrain the AI and never bypass it; a missing model stops the workflow and offers the exact --legacy command without executing it.

  • ISO rebuild + verification session (2026-09-26, DONE except seat-compositor): new ISO installer/live/phoebus-installer-amd64-20260926.iso (1.6 GiB, 250dbe9a…, build 20260926T041500Z, HEAD 11c0ee0 + 0b41781) built via PHOEBUS_REUSE_ROOTFS=1, content-verified (embedded hashes match, Gentoo key both fingerprints, UI payload present); old Sep-24 ISO kept. Interactive VM RUNNING (KVM/GTK, phoebus-interactive-20260926.qcow2 + private vars in /var/tmp/phoebus-interactive/, screenshot at welcome, undisturbed). Unpinned signed-DIGESTS full install PASSED (gui-install-20260926T040728Z-f3ae68f4, INSTALL-RC=0, NO_SECRET_LEAK, ~165 min) + detached boot (running, 0 failed units, sys --legacy + snapshot). Tamper refusals PASSED (placeholder pin pre-network, hash mismatch post-download, both RC=1 + no leak). Recovery-entry boot PASSED (snapshot cmdline/findmnt, login, sys --legacy, truthful ask failure, systemd degraded). Seat desktop PARTIAL: seat0/DRM/greeter/auth verified, Mango dies in <1s (libseat, no seatd) — the blocker. Snap: no Store revision, still partial confinement. Nix: GC-dangling NOT reproduced; new gap: modern per-user Nix profiles invisible to sys (follow-up). Live AI: still no credential/model (NotRun). gui-driver.py boot sys-check fixed (--legacy). gates --execute: 0/0/6 NotRun, no publish. Quality gates green (717/717, clippy -D, vmsafety 18/18). Evidence disk disks/unpinned-accept-1.qcow2 KEPT; scratch overlays removed. Next: seatd-or-logind fix for Mango (design+rebuild+retest), per-user Nix enumeration, live AI credential, snap revision/strict, NVIDIA.

  • Security/readiness follow-up (2026-09-26): trust anchor + executable gates + strengthened rollback PASS.

    1. Stage3 trust anchor (fixed): release_key_path() ignores PHOEBUS_GENTOO_RELEASE_KEY; production always uses the provisioned key, test injection only via explicit verify_digests_with_key. Host-level Good-signature check of the real DIGESTS passes (not VM evidence). VM proof blocked: needs a rebuilt live ISO + disposable install (exact prerequisites in docs/VM-BOOT-VERIFICATION.md).
    2. Release gates executable: invalid checks/desktop modes fixed (real boot modes), --execute runs each argv and reports Passed/Failed/NotRun from evidence (unconfigured host: 0/0/6, publish refuses). Found and fixed a gui-driver.py SyntaxError. 6 new tests.
    3. Strengthened rollback rerun PASSED (serial-rollback-20260926T034218Z-7fab51ef, PASS=11 FAIL=0, exit 0, no leak, overlay rollback-accept-8.qcow2): mandatory ===ROLLBACK-MARKER present=== before undo, DB ca398d71…→fda3780a… →restored, marker absent after detached reboot, /home survives, systemd running. SERVE shellcheck fix included.
    4. Recovery-entry boot is now a distinct manual gate with its own procedure (NOT RUN). Roadmap no longer lists passed sys undo as unfinished; Secure Boot filed as a separate opt-in milestone.
    5. Quality gates: cargo fmt --check, clippy --workspace --all-targets -- -D warnings, cargo test --workspace 717/717 (one assistant_bridge flake under parallel load, green in isolation and on rerun), release + static-pie builds, test_vmsafety.py 18/18, shellcheck clean. Seat desktop, live AI, Snap revision/strict, and NVIDIA stay open with exact prerequisites (STATUS).
  • Interface contract (unchanged): ordinary requests use AI; only explicit --legacy selects deterministic handling; source flags constrain the AI and never bypass it; a missing model stops the workflow and offers the exact --legacy command without executing it.

  • Hardening pass (2026-09-26): stage3 fail-closed + truthful boot security + rollback acceptance PASS.

    1. Stage3 (security defect, fixed): the unpinned path warned and continued on DIGESTS-download failure, missing SHA-512 entry, missing gpg/key, failed key import, and gpg launch errors. All stop the install before extraction now; bad signature/hash also stop it. The pinned path requires a valid 64-hex pin (placeholder refused) before any network use. stage_install checks for gpg/sha256sum/sha512sum first. 16 new regression tests (every failure branch, nonzero-exit and cannot-start doubles, both positive paths). The live image ships gnupg plus the Gentoo release key, fingerprint-checked at build time against Gentoo's published fingerprints (13EB…2910 weekly, D99E…2058 signing). Commits 2162670, 744059b.
    2. Boot security (fixed): uki=true/secureboot=true are rejected at validate_config (shared by CLI/TUI/GUI bridge); the plan preview always reports unsigned BLS. Parsing/validation/rendering regressions + golden updated. Commit 95e0d05. Authenticated boot stays an open limitation (STATUS).
    3. Rollback acceptance PASSED for real (serial-rollback-20260926T030237Z-912f6386, PASS=11 FAIL=0, exit 0, no secret leak): fresh overlay rollback-accept-7.qcow2 on fresh-install-1.qcow2 — snapshot → app-misc/hello install (DB ca39…→fda3…) → sys undo <id> rc=0 → detached reboot (DB restored, root marker absent, /home survives, systemd running, recovery entry present). The 2026-09-25 pass claim had no transcript and did not reproduce; fixed harness defects: sudo auth (stale-prompt match bug found and fixed with unique markers), root phases, explicit --legacy, verified break (nano was pre-installed), targeted undo, marker-absent assertion, preserved phase-1 log. Commits 744059b, 5f69996.
    4. Release candidate (local, isolated): manifest (2 artifacts), verify, tamper refusal, and publish refusal against the NotRun report all behave; gates --execute is 0 passed / 6 NotRun (every gate needs VM/seat/model; run_gates never executes gate commands itself). Nothing published.
    5. Quality gates: cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace 712/712, release + static-pie builds, test_vmsafety.py 18/18, make doctor 19 ok / 1 disk-space warning.
  • Interface contract (unchanged): ordinary requests use AI; only explicit --legacy selects deterministic handling; source flags constrain the AI and never bypass it; a missing model stops the workflow and offers the exact --legacy command without executing it.

  • Quality pass (2026-09-26): boxed ExecError::Command details (CommandFailure, 128 → 96 bytes, measured with a path-dependent probe crate; constructor + all match sites updated, error text/redaction/stderr-tail preserved, 4 regression tests), cleared cargo clippy --workspace --all-targets -- -D warnings with no blanket allows (type alias, CommandRecord params struct, DiskPhase renames), and split service.rs (4218 lines → service/mod,plan,validation,execution,tests) and gui_bridge.rs (2763 lines → gui_bridge/mod,protocol,bridge,tests) as structural refactors with explicit re-exports and pub(super) seams — no service.rs left alongside service/mod.rs. make test green (692/692, unchanged across the splits). Release + static-pie builds pass. The disposable VM re-ran the current 38-case sys acceptance matrix against the new static sys (70f0a72b…): guest PASS=38 FAIL=0, driver PASS=41 FAIL=0 (serial-boot-20260926T014746Z-3089a820, overlay sys-acceptance-refactor.qcow2). Commits e69bfb9, 1bf31c5, 2dd48d1.

  • Interface contract (unchanged): ordinary requests use AI; only explicit --legacy selects deterministic handling; source flags constrain the AI and never bypass it; a missing model stops the workflow and offers the exact --legacy command without executing it.

  • Snap first-use follow-up (2026-09-25): the previous "no snapd" note was a real integration gap. The normal sys install … --legacy --snap resolver needs snapd to confirm a Store candidate before approval, while a raw typed snap_install silently added it during execution. First-use setup is now a separately reviewed Portage install plus socket enable/start, with exact guidance when absent. Fresh installs set scoped sys-fs/squashfs-tools lzma lzo, which Gentoo's snapd ebuild requires. A disposable VM exposed a second bug: Portage candidate resolution picked the first dependency (dev-lang/go-bootstrap) instead of the requested app-containers/snapd; it now matches the requested atom. Snap confinement is reported from the daemon rather than assumed strict. The disposable VM completed setup, Store install, refresh and remove (PASS=10 FAIL=0), with partial confinement. A fresh overlay and a clean reboot repeat with the final static binary each passed PASS=11 FAIL=0, including clean poweroff. Refresh found no newer revision.

  • Previous session: second-round sys review fixes. Follow-up to d3f676a. Three further confirmed defects are fixed with regression tests: unsafe cleanup previews, non-exact Flatpak/Snap update execution, and reverse-dependency self-reference. make test is green (661 → 683 tests). This corrects the earlier blanket claim that every review finding was already fixed.

  • Interface contract (unchanged): ordinary requests use AI; only explicit --legacy selects deterministic handling; source flags constrain the AI and never bypass it; a missing model stops the workflow and offers the exact --legacy command without executing it.

Snap first-use changes

  • phoebus-core::system generates scoped SquashFS USE flags for new installs.
  • Portage candidate resolution selects the requested atom from the pretend plan, even when dependencies precede it. A multi-package regression covers the bug found by the VM.
  • The Snap install path gives exact setup commands when snapd is absent; the typed action no longer adds an unreviewed Portage dependency. Candidate sandbox levels and missing snap info fields are treated conservatively.
  • installer/tests/vm/snap-guest.sh drives the installed-system journey. The serial driver can authenticate for boot checks and logs only guest output, keeping sent test passwords out of its transcript.

Previous sys review work

  1. Cleanup previews are non-destructive (finding 1). eclean-dist/ eclean-pkg delete by default, so portage_cleaners now previews with --pretend --deep (same selection policy as execution) and executes with --deep; the Nix preview uses the documented nix-store --gc --print-dead. Items are read from stdout+stderr, sorted and bound. Tests prove a cleaner never runs destructively during planning, revalidation or cancelled approval.
  2. Exact Flatpak/Snap update transactions (finding 2). ResolvedPlan.updates carries UpdateTransactions: full Flatpak ref + full installed/remote commits (via flatpak info, since the commit column is abbreviated), or snap name + old/new revisions + channel. Execution consumes the reviewed transaction (add_executable… flatpak update --commit=<c>), revalidates each source immediately before its mutation, verifies the resulting commit/revision, and refuses non-exact transactions. A source whose resolution failed is never re-discovered. PlanStore aggregation includes updates.
  3. Reverse dependencies (finding 3). The reverse branch no longer drops valid matches; it excludes only a genuine self-reference (the dependent is the queried package).
  4. Determinism. All selected packages: removals are sorted (depclean previews are now order-independent); emerge --depclean --ask=n.

Verified in disposable VMs

  • Cache cleanup + depclean (real tools): eclean upstream preview is non-destructive; sys clean cache preview/cancelled/approved/drift/failed all behave correctly; depclean cancel/approve with transaction history and preserved /home.
  • Reverse dependencies: sys-libs/glibc and sys-libs/zlib report real dependents (conditional/versioned included).
  • Flatpak: installed via Portage (sys-apps/flatpak, enabling media-libs/freetype[harfbuzz]), flathub added, an app installed and downgraded, then sys apply updates --flatpak resolved full ref+commits, deployed the pinned commit, and verified it.
  • Custom ebuild: stage → build/install (VDB entry + installed file) → remove.
  • Managed AppImage: acquire with pinned SHA-256, list, run the managed ELF, remove; a placeholder SHA is refused.
  • Nix: a NIX_ENABLED=yes install bootstraps Nix; nix profile install nixpkgs#nodejs-slim into /nix/var/nix/profiles/profile is enumerated by sys list --nix and found by sys list nodejs --nix, with store paths and generations reported separately.
  • Non-root pkexec sys --broker: with the shipped polkit action and the standard wheel-admin rule installed on the disposable image, a wheel user authenticated through a terminal agent; cancellations changed nothing, the approved removal returned the broker JSON outcome, and the replay was rejected. The shipped privilege policy is unchanged.
  • Read-only acceptance: installer/tests/vm/sys-acceptance.sh PASS=38 FAIL=0 re-run against the final static sys (dd61c610…), including the new rdeps-legacy reverse-dependency regression.
  • Snap first use: the disposable VM installed app-containers/snapd through the reviewed Portage plan (six effects), enabled and started snapd.socket, installed Snap Store hello-world 6.4 revision 29, ran refresh, then removed it. The daemon reported partial confinement and the install review agreed. The harness reported PASS=10 FAIL=0 (serial-boot-20260926T001830Z-107996de). Final-binary runs on a fresh overlay passed PASS=11 FAIL=0 on first boot (serial-boot-20260926T005327Z-c024cc43) and after a clean reboot (serial-boot-20260926T010605Z-e6193c3a); both shut down cleanly.

Not completed / blocked

  • Stage3 VM proof with the new binary: unit-tested (19 stage3 tests) and the pinned path is unchanged for real pins, but no full install has run with the fail-closed binary yet; the next disposable install is the proof. The unpinned (signed-DIGESTS) VM proof additionally needs a rebuilt live ISO carrying the provisioned key (installer/live/build.sh now provisions it; no ISO was rebuilt in this session).
  • Installed Mango + DMS session under a seat: still needs a seat-providing graphical VM (or physical VT): login, kitty, DMS spotlight, lock/idle, polkit agent, logout, reboot (docs/DEVELOPMENT.md physical-VT procedure).
  • Snap update to a different revision / strict confinement: refresh with no newer revision and partial confinement were VM-verified. A revision-changing refresh and AppArmor-backed strict confinement were not (need an AppArmor-ready guest kernel with loaded policy, and a Store snap that actually has a newer revision than installed).
  • Live AI: a local OpenAI-compatible model (llama-server + a GGUF model) was served on the host; the AI workflow reached the model, but the sandboxed call exceeded the 120 s transport timeout on CPU (curl exit 28). Needs a faster model/GPU or a remote provider credential. Fixture-backed routing and failure tests pass (ask-offline, missing-model offer, source constraint).
  • Recovery boot (distinct from rollback): the recovery BLS entry is present and asserted in the rollback acceptance, but booting it interactively was not automated. Method: select "Phoebus Recovery (known-good snapshot)" in the systemd-boot menu with no network.
  • Nix re-run flakiness: a later sys list --nix re-run reported the profile link dangling (the profile store path was gone) and correctly listed no packages; the first run with a real profile is the positive evidence.

Known blockers / risks

  • VM serial login is occasionally flaky; retry the boot when detached-boot-login fails.
  • make doctor passes (KVM accessible via an ACL). Disk is shared by build and VM; watch free space.
  • The pre-existing test-disk.qcow2 cannot be unlocked (unknown passphrase); do not guess or brute-force it.

Next actions (in order)

  1. Stage3 install proof: next disposable install with the fail-closed binary (pinned path), then a rebuilt live ISO (with the provisioned Gentoo key) proving the signed-DIGESTS path in the VM.
  2. Recovery boot by selecting the recovery BLS entry with the network down.
  3. Live AI with a faster local model/GPU or an authorized provider credential (natural-language discovery, source constraints, reviewed install).
  4. Installed desktop under a seat (Mango + DMS login through reboot).
  5. Nix profile GC stability if the dangling re-run is reproducible.
  6. Snap strict confinement and changed-revision update in a disposable VM when an AppArmor-ready kernel and a revision-changing Store test are available.

Definition of a good stopping point

  • make test passes; docs/STATUS.md, docs/CHANGELOG.md and this file updated; committed (scripts/agent-commit.sh) and pushed (scripts/publish.sh).