From b120c1e5e4a7b80d452f7f7108e2306a5f3b478e Mon Sep 17 00:00:00 2001 From: Phoebus Date: Sun, 20 Sep 2026 05:03:37 -0700 Subject: [PATCH] docs: record AI guidance, desktop reliability and snapshot verification Update the status matrix, roadmap, changelog and handoff; add the WS4/WS3 VM evidence and a concise real-VT desktop test procedure. --- docs/CHANGELOG.md | 32 +++++++++++++++++++++++++++++ docs/DEVELOPMENT.md | 30 ++++++++++++++++++++++++++- docs/HANDOFF.md | 37 +++++++++++++++++++++------------- docs/ROADMAP.md | 24 +++++++++++++--------- docs/STATUS.md | 21 ++++++++++++++----- docs/VM-BOOT-VERIFICATION.md | 39 ++++++++++++++++++++++++++++++++++++ 6 files changed, 153 insertions(+), 30 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index e69b9e1..e093d72 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -15,6 +15,38 @@ Format: Keep a Changelog. Dates are UTC. `CHANGELOG.md`, and the top-level `AGENTS.md`. - `.gitignore` covering the ISO, build output, transcripts and secrets. +## 2026-09-20 — AI guidance, desktop reliability, snapshots + +### Added +- AI-assisted install guidance: an OpenAI-compatible provider layer for + **DeepSeek** and **OpenAI** plus a test transport + (`phoebus-ai::provider`), credential loading and a curl transport that keeps + the API key out of argv, config and logs (`phoebus-executor::ai`), and CLI + flags `--ai`, `--ai-provider`, `--ai-model`, `--ai-api-key-file`, + `--ai-proposal-file`, `--ai-facts`. The TUI offers + DeepSeek/ChatGPT/without-AI. +- Typed proposal extensions: scoped `package.use`, explicit build jobs and + binary-package preference. Every proposal now requires a non-empty reason and + duplicate/conflicting changes are rejected before anything is applied. +- `sys` history and update preparation are VM-verified on a Btrfs install. + +### Changed +- `phoebus-session` now starts `dwl` first, waits for the Wayland socket before + launching waybar/mako/swaybg, establishes `XDG_RUNTIME_DIR` and a session bus, + starts the polkit authentication agent, and reaps helpers on logout. +- Portage policy: the unused `installkernel uki` flag is no longer forced, the + duplicate/empty PipeWire/WirePlumber lines are gone, and the signed binhost is + derived from the selected profile when unset. +- Update preparation records a successful distfile fetch as ready even when + `--buildpkgonly` is skipped because dependencies must be merged first, and the + state note now includes the redacted Portage reason. + +### Fixed +- A high-risk `sys` action now creates its protective snapshot inside + `actions::execute` and **fails closed** if the `.snapshots` subvolume exists + but the snapshot cannot be created; corrupt snapshot history is surfaced + instead of being hidden as empty. + ## 2026-09-20 — Retire the Bash installer (migration Phases 0–4) ### Added diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index ba1a4ef..24b4bfa 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -88,7 +88,35 @@ grim /tmp/shot.png # proves the compositor renders ``` On a real machine the installer writes `/usr/local/bin/phoebus-session` -(waybar + mako + swaybg + dwl). +(dwl + waybar + mako + swaybg + the polkit agent). + +### Physical VT / real-seat check (needs a person) + +The serial harness has no logind seat, so this must be done on real hardware or +a seat-providing VM. From a text VT (e.g. `Ctrl+Alt+F3`), log in and run: + +```sh +/usr/local/bin/phoebus-session +``` + +Check, in order: + +1. `dwl` starts and the screen shows the compositor (no `Failed to start a DRM + session`). +2. `waybar` and `mako` appear only after the compositor is up (they must not + log a failed Wayland connection). +3. Open a terminal: `kitty`. Run an X11 app to exercise XWayland (e.g. + `xterm` or `env -u WAYLAND_DISPLAY kitty` if no X app is installed). +4. Trigger a privilege prompt (e.g. `pkexec true` or a polkit-gnome dialog) and + confirm the authentication agent appears. +5. Log out (`Mod+Shift+e` for dwl). Confirm `waybar`, `mako`, `swaybg` and the + polkit agent are gone (`pgrep -a 'waybar|mako|swaybg|polkit-gnome'` prints + nothing) and that the session returns to the VT login. +6. Reboot and confirm the same result. + +Record the result (date, kernel, GPU, pass/fail per step) in +`docs/VM-BOOT-VERIFICATION.md`. A headless wlroots session is not physical-seat +verification. ## Inspecting the reference ISO without booting diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md index 5233006..6e6561e 100644 --- a/docs/HANDOFF.md +++ b/docs/HANDOFF.md @@ -3,31 +3,39 @@ Rolling note between work sessions. **Update this file before you finish**, then commit it. It should be short and current; use `git log` for detail. -- **Last updated:** commit `fc64793` (2026-09-20) -- **Current focus:** the Bash installer is retired (migration Phases 0–4, see - `docs/BASH-MIGRATION.md`); `installer-rs/` is the only installer. -- **Last verified:** static Rust binary, full encrypted UEFI VM install + - detached boot (`docs/VM-BOOT-VERIFICATION.md`); desktop stack and Flatpak - verified headless. +- **Last updated:** commit `TBD_HEAD` (2026-09-20) +- **Current focus:** four workstreams — AI-assisted install guidance, desktop + session reliability, signed-binary reuse, and snapshots/safe updates. +- **Last verified:** static Rust binary install + detached boot + (`docs/VM-BOOT-VERIFICATION.md`); in this session additionally: `sys` + snapshot/history and update preparation on an installed Btrfs VM, and the + generated `phoebus-session` script against stub binaries. - **Public repo:** (remote `origin`, SSH port 2222). ## Next actions (in order) -1. **Physical-seat desktop**: run `phoebus-session`/dwl on a real VT and verify - kitty, wofi, XWayland, the polkit agent, logout and reboot interactively. -2. **Desktop binary strategy**: match desktop USE to a signed binhost (or - publish one) to stop compiling ~26 minutes of desktop packages. -3. **Snapshots + safe updates end-to-end**: prove `sys` snapshot and - `prepare-updates` on an installed system, then implement and test rollback - before wiring `sys undo`. +1. **Physical-seat desktop**: run `/usr/local/bin/phoebus-session` on a real VT + and check kitty, XWayland, the polkit agent, logout/reap and reboot + (`docs/DEVELOPMENT.md` -> "Physical VT / real-seat check"). +2. **Matching signed binhost**: the official Gentoo binhost is OpenRC/hardened, + so systemd/desktop packages compile from source. Publish (or point at) a + systemd/desktop binhost to raise binary reuse; verify with `emerge -pv`. +3. **Rollback gate**: implement Btrfs default-subvolume + ESP handling and pass + a detached-boot test before exposing `sys undo`. +4. **Live AI provider call**: exercise DeepSeek/OpenAI with a real 0600 key + file; the path is currently fixture-verified only. +5. **Desktop graphical login**: exercise a real logind seat. ## Known blockers / risks +- No live DeepSeek/OpenAI call has been made (no key in this environment). - No Forgejo Actions runner is known to exist; `.forgejo/workflows/ci.yml` only runs once one is registered. -- Desktop graphical login has not been exercised with a real logind seat. +- Desktop graphical login / physical seat has not been exercised; the serial + VM has no logind seat. - NVIDIA is not verified on hardware. +- Rollback is not implemented (by design, gated). ## Environment notes @@ -36,6 +44,7 @@ commit it. It should be short and current; use `git log` for detail. - Forgejo token for API tasks: `~/.config/phoebus/forgejo.token` (0600). - The VM harness disks are disposable qcow2 files on `/` (not `/tmp`, which is a small tmpfs on the dev host); never point `DISK` at `/dev/*`. +- The VM harness honours `CHECKS_TIMEOUT` for long guest checks. ## Definition of a good stopping point diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index d9ce327..984aaba 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -10,29 +10,33 @@ more interesting. Status vocabulary: **Done**, **Partial**, **Next**, **Later**. | 3 | Port the deterministic core | Done | `phoebus-core` (config, plan, policy, validation, models) | | 4 | Port the destructive backend | Done | `phoebus-executor` (storage, LUKS2, Btrfs, Portage, boot, users, services) | | 5 | **Rust VM boot parity** | Done | Full encrypted UEFI install + boot, zero failed units, 10.7 s | -| 6 | Finish the desktop | Partial | dwl/waybar/mako/swaybg/pipewire verified headless; physical seat + interactive login pending | +| 6 | Finish the desktop | Partial | session script race-free + polkit + helper cleanup (functional test); headless compositor verified; physical seat + interactive login pending | | 7 | Finish Flatpak | Partial | `flatpak 1.18.1` installed, flathub configured; no app installed yet | | 8 | Instrument source/binary policy | Done | measured compile times harvested from `emerge.log` (326 entries) | -| 9 | Background update preparation | Partial | `--fetchonly` + `--buildpkgonly` implemented and unit-tested; not VM-tested | -| 10 | Snapshots / rollback | Partial | snapshot create + history done; rollback not implemented | -| 11 | AI privilege and privacy architecture | Partial | typed actions + enforced privacy boundary; OS sandbox not implemented | -| 12 | `sys` operations | Partial | history, enable/disable service, inspect, install, prepare-updates | +| 9 | Background update preparation | Partial | distfile fetch VM-verified; `--buildpkgonly` skipped while deps need merging (recorded Ready); activation explicit | +| 10 | Snapshots / rollback | Partial | snapshot create + history VM-verified on Btrfs; rollback not implemented | +| 11 | AI privilege and privacy architecture | Partial | typed actions + enforced privacy boundary + guided DeepSeek/OpenAI proposal review; OS sandbox not implemented | +| 12 | `sys` operations | Partial | history, enable/disable, inspect, install, prepare-updates VM-verified | | 13 | Kernel optimization policy | Later | conservative fragments only; SAFE/OPTIMIZED/AGGRESSIVE selection exists | | 14 | ISO / release pipeline | Partial | manifest/verify/assemble-plan tooling; no scheduling/publish/smoke test | | 15 | Recovery environment | Later | recovery BLS entry generated; full recovery image not built | -| — | Rust TUI | Partial | ratatui UI with plan/ERASE; not yet the default on all paths | +| — | Rust TUI | Partial | ratatui UI with plan/ERASE and AI provider selection; not yet the default on all paths | +| — | AI-guided install optimization | Partial | DeepSeek/OpenAI providers, evidence-gated typed review, TUI opt-in; fixture-verified, no live call | | — | Git workflow + agent guide | Done | this `docs/` set, `.githooks/`, `scripts/` | ## Immediate next steps 1. **Physical-seat desktop**: run `phoebus-session`/dwl on a real VT (or a `seatd`-backed session) and verify kitty, wofi, XWayland, the polkit agent, - logout and reboot interactively. + logout and reboot interactively (`docs/DEVELOPMENT.md`). 2. **Desktop binary strategy**: match the desktop USE set to a signed binhost (or publish one) so desktop installs stop compiling ~26 minutes of packages. -3. **Snapshots and safe updates end-to-end**: prove `sys` snapshot + - `prepare-updates` on an installed system, then implement and test rollback - before wiring `sys undo`. + The official binhost is OpenRC/hardened; a systemd/desktop host is needed. +3. **Rollback as a safety gate**: implement Btrfs default-subvolume + ESP + handling and pass a detached-boot test before exposing `sys undo` + (snapshot/history/prepare-updates are now VM-verified). +4. **Live AI provider call**: exercise DeepSeek/OpenAI with a real 0600 key + file; the guided path is currently fixture-verified only. ## Later diff --git a/docs/STATUS.md b/docs/STATUS.md index 2277b52..e3fe6ac 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -56,17 +56,19 @@ on the install path (Bash migration Phase 2, see | Rust VM boot parity | yes | yes | yes | no | | Rust TUI | yes | yes | smoke test | no | | Desktop profile (dwl/waybar/mako/swaybg) | yes | partial | headless only | no | +| Desktop session script (socket wait, polkit, cleanup) | yes | yes | functional test (stubs) | no | | PipeWire/WirePlumber | yes | no | headless (`wpctl`) | no | | Flatpak + flathub | yes | no | installed + remote | no | | XWayland / polkit agent / kitty / wofi | yes | no | installed, not launched | no | | Package policy + measured history | yes | yes | yes (326 entries) | no | -| Background update preparation | yes | yes | no | no | -| Btrfs snapshots + history | yes | yes | no | no | +| Background update preparation | yes | yes | yes (fetch; buildpkgonly skipped) | no | +| Btrfs snapshots + history | yes | yes | yes | no | | Btrfs rollback | no | no | no | no | -| AI typed proposals | yes | yes | no | n/a | -| AI privacy boundary | yes | yes | no | n/a | +| AI typed proposals + evidence review | yes | yes | fixture end-to-end | n/a | +| AI guided install (DeepSeek/OpenAI) | yes | yes | fixture path only (no live call) | n/a | +| AI privacy boundary (facts only) | yes | yes | no | n/a | | AI OS sandbox | no | no | no | no | -| `sys` verbs | yes | yes | no | no | +| `sys` verbs | yes | yes | yes | no | | OpenCode pinned bootstrap | yes | yes | no | n/a | | Release manifest/verify | yes | yes | no | n/a | | NVIDIA proprietary | architecture only | no | no | no | @@ -89,5 +91,14 @@ on the install path (Bash migration Phase 2, see signing or atomic publish. - The desktop transaction compiles heavily because desktop USE does not match the signed binhost (~26 min for the desktop set on 8 vCPUs in the test VM). + The official Gentoo binhost is OpenRC/hardened, so a systemd system can only + reuse binaries for packages whose USE matches; the policy still requests + signed binaries where it can. Publishing a matching binhost is future work. +- **Live AI provider calls are not verified.** DeepSeek/OpenAI requests are + built and reviewed by fixtures (`phoebus-ai::provider::TestTransport`) and the + curl transport is covered by a recording runner; no call was made with a real + key. - The installer rebuilds `sys-apps/systemd` from source because boot/policykit/cryptsetup USE differ from the binhost. +- `sys undo` is intentionally not implemented: rollback needs default-subvolume + and ESP handling plus a detached-boot test, which is a separate safety gate. diff --git a/docs/VM-BOOT-VERIFICATION.md b/docs/VM-BOOT-VERIFICATION.md index 5096743..a12b21c 100644 --- a/docs/VM-BOOT-VERIFICATION.md +++ b/docs/VM-BOOT-VERIFICATION.md @@ -209,6 +209,45 @@ Boot with the ISO detached (`serial-driver.py boot`): This satisfies the Phase 2 exit gate of `docs/BASH-MIGRATION.md`: the only proven boot path no longer depends on the retired Bash installer. +## Snapshots and safe updates (four-workstream session) + +Verified on the installed Btrfs system from the static-binary harness, driving +the new `sys` binary fetched into the guest: + +- `sys history` before -> `no transaction history`. +- `sys -y --action '{"action":"create_snapshot","operation":"vm test"}'` -> + `protective snapshot: @-20260920T114917-vm-test`. +- `sys history` after -> `@-20260920T114917-vm-test Committed vm test`. +- `btrfs subvolume list /` shows `@snapshots/@-20260920T114917-vm-test` as a + real subvolume (read-only snapshot). +- `sys -y prepare-updates` fetched distfiles successfully; `--buildpkgonly` was + refused by Portage with `--buildpkgonly requires all dependencies to be + merged`, which the installer now records as `status: Ready` with the reason + (`distfiles fetched; binary prebuild skipped: ...`) instead of a hard failure. +- `sys inspect-updates` reports the state and timestamp. + +Unit regression tests cover the failure paths that a VM cannot easily force: +corrupt snapshot history is an error, and a high-risk action aborts when the +protective snapshot fails. + +## Signed binary reuse analysis + +The desktop transaction's installer log (`emerge` grouping) shows the policy +already requests signed binaries for the expensive set: + +- source group: `sys-fs/cryptsetup sys-fs/btrfs-progs`; +- binary groups: base tools, `gentoo-kernel-bin`/firmware, and the dwl/waybar/ + wofi/mako/sway*/gtk-layer-shell/xwayland set. + +The remaining source builds are caused by the **official Gentoo binhost being +built for the OpenRC/hardened profile** (`PROFILE: default/linux/amd64/17.0/ +hardened`), which cannot satisfy a systemd/desktop USE set. The safe changes +made here are: stop forcing the unused `installkernel uki` flag, remove the +duplicate PipeWire and empty WirePlumber `package.use` lines, and derive the +binhost URI from the selected profile when it is not set. Signature +verification remains enabled. Publishing a matching systemd/desktop binhost is +the remaining step; no time estimate is claimed beyond the recorded ~26 min. + ## Known remaining work - The graphical session was verified with the **headless** wlroots backend