# Verification and findings register Companion to `docs/STATUS.md`. The machine-readable per-run results live outside git under `/var/tmp/phoebus-vm//results.json`; this file is the human-readable register of what was found, fixed and still unverified. ## OpenCode adapter (2026-09-22, second pass) Implemented a real OpenCode adapter (not just a probe), integrated through canonical configuration (`ai.backend = "opencode"`), the GUI bridge, the installer CLI and `sys ask`. Defects found while implementing and testing: | ID | Defect | Fix | Test | |----|--------|-----|------| | O1 | No OpenCode proposal adapter; the GUI used the OpenAI-compatible transport | `phoebus-executor::opencode_client` implements the real session/message protocol as a `ModelTransport` | live + bridge end-to-end | | O2 | `POST /session` was sent a `model` field per the OpenAPI schema | the shipped 1.18.32 rejects it (BadRequest); the model is selected on the message | real-HTTP integration | | O3 | The probe had no credential input | `ProbeRequest.auth` + basic-auth header via stdin | auth integration test | | O4 | The probe did not enforce the connection destination | `Endpoint::checked_target` resolves and requires loopback for plain http; the connection is pinned | `transport_policy_enforces_loopback_for_plain_http`, insecure-destination test | | O5 | Handwritten HTTP read had no total deadline and a UTF-8-unsafe chunk decoder that fell back on malformed input | removed the handwritten parser; the established curl transport is used and bounds time, protocol, redirects and size | real-HTTP chunked/oversize/redirect/disconnect tests | | O6 | The catalog was parsed without checking the HTTP status; a providers-shaped 500 could look like a catalog | status is checked before parsing; non-2xx is `incompatible` | `a_misleading_success_shaped_500_does_not_produce_a_catalog` | | O7 | Catalog presence counted as readiness | `ready` is reserved for a validated response; the probe returns `model_catalog`/`reachable` | `an_empty_catalog_is_reachable_not_ready` | | O8 | `probe_live_server_when_configured` passed when its env var was absent and only asserted reachability | replaced by env-gated `opencode_live` tests that assert a real inference; `PHOEBUS_OPENCODE_REQUIRE_LIVE=1` fails when the live run cannot happen | `make opencode-acceptance` | | O9 | The credential test built an already-sanitised struct | tests feed credential-bearing server bodies and assert errors/catalogs never contain them | `one_shot_rejects_a_non_assistant_or_errored_reply`, `catalog_never_reads_provider_keys` | | O10 | A provider name satisfied the mandatory-AI gate | both the bridge and the CLI require a validated interaction bound to the current configuration; config/secret changes invalidate it | `a_provider_name_alone_does_not_satisfy_the_ai_requirement`, `an_unreachable_provider_does_not_satisfy_the_ai_gate` | | O11 | The proposal prompt allowed the model to key the change by action name | prompt now requires exactly `action` + `value` with explicit examples | live bridge `ai_check` | | O12 | The GUI duplicate-session assertion accepted two pre-existing sessions | requires exactly one session before and after | harness | | O13 | GUI navigation was inferred from a surviving process/screenshot | the frontend emits secret-free `phoebus-state` lines; the harness asserts the current step and AI state | harness | | O14 | A model could return a malformed proposal with no second chance | one bounded corrective retry in `complete_proposal_at`/`complete_plan`, with the strict parser still final | `a_rejected_proposal_is_retried_once_with_a_correction`, `a_retry_that_still_fails_is_still_rejected` | ### Graphical OpenCode acceptance (VM verified) `gui-driver.py opencode` on the built ISO (real model, no simulation): - welcome → installer → AI step; selected **OpenCode agent** through the UI; - typed the server URL, provider and model through the UI; - **Test connection** obtained a real model response (green "Model responded"); - **Connect and validate** obtained and validated a proposal (green "Validated", frontend state `aiConnected=true`); - run `gui-opencode-20260922T192457Z-01714c92`: **18/18 PASS**, exit 0; the full GUI regression `gui-gui-20260922T191954Z-876d5e91` was **28/28 PASS** (both on the final ISO `sha256 0c386a1b…`). - The model is reached through a guest-loopback relay to the host OpenCode server (the SSH-tunnel shape), so the transport policy is satisfied. - Authenticated server: the same harness with a password-protected server got through authentication and a real Test connection; a flaky malformed proposal was correctly rejected. Use `deepseek-flash` (reliable) over `deepseek-v4-flash` (occasionally malformed). ### Graphical journey: interaction defects fixed (2026-09-22) | ID | Defect | Fix | Test | |----|--------|-----|------| | G1 | Network **Re-check** ran the handshake init path and reset the step, clearing the selected disk | a read-only `readiness` bridge op refreshes the probe without touching the form; only the first handshake initialises | harness `readiness-refresh-preserves-wizard`, `disk-survives-backtrack`; Rust `readiness_reports_link_and_service_reachability_separately` | | G2 | Model-catalog requests used stale configuration/credentials, and a stale reply could overwrite a newer one | the catalog syncs edited config + secrets first, replies carry a stable `error_kind` (auth required/rejected, insecure transport, model unavailable, unreachable), and a request sequence ignores superseded replies | Rust `ai_catalog_errors_are_machine_readable`, `connection_error_kind_classifies_common_failures`; harness exercises the auth error | | G3 | The green AI indicators survived edits to provider/model/endpoint/credentials | every connection-field edit invalidates the validated state; account/disk edits preserve it | harness `ai-invalidated-on-model-edit`; Rust `a_config_change_invalidates_the_validated_interaction` | | G4 | Suggestion lists were mouse-only | Up/Down move a visible highlight, Enter selects, Escape dismisses, `Accessible` roles; Enter in an open list is consumed | harness `keyboard-suggestion-select` | | G5 | Selecting a keyboard layout implied it applied to the live session | the step states the live session keeps its layout and the choice applies to the installed system | manual review of the language step | | G6 | Readiness only detected a default route | `link` and verified `services` reachability are reported separately | Rust `readiness_reports_link_and_service_reachability_separately` | | G7 | A bridge crash left requests/busy indicators hanging | the bridge fails every outstanding request on exit; single-flight guards and cleared busy states with retry | Rust bridge exit path; harness `frontend-no-bridge-errors` | | G8 | `PhoebusToggle` Return/Enter called a signal handler as a function | a single `Keys.onPressed` handler toggles on Space/Return/Enter | harness `gui-install-erase-confirmed` | ### Full graphical install `gui-driver.py gui-install` drives the real controls on a fresh 30 GiB qcow2 (run `gui-gui-install-20260922T151232Z-0ef19969`): **24 PASS / 1 FAIL**. Every journey assertion passed (OpenCode connect/review/accept, disk, account, ERASE, confirmation toggle) and the backend reported the real stage sequence with elapsed time and no fabricated percentage. The one FAIL is `gui-install-completes`: the four-hour harness timeout fired during `kernel_install` (base packages ~116 min; the AI-selected optimized kernel compiled for >100 min on this host). The completed install + detached boot gate is **BLOCKED on host build time**; the deterministic noninteractive install + detached boot remains the proven boot path. ### User trial findings (interactive VM, 2026-09-22) Observed while a user drove the live ISO by hand (run `/var/tmp/phoebus-vm/user-session*`): | ID | Finding | Status | |----|---------|--------| | U1 | Timezone (and locale, keyboard) were **free-text**: the user was expected to know `America/New_York` etc. Only ~5 keymap presets were offered, then manual entry. | **Fixed**: searchable pickers populated from the live system (418 timezones from `zone.tab`, 513 locales, 82 keymaps), type-to-filter and click-to-choose; a custom value is still accepted. | | U2 | The AI **provider/model** fields were free-text; the user had to know the exact model id. | **Fixed**: `ai_catalog` bridge op returns the OpenCode catalog (223 providers, real model ids) or an OpenAI-compatible `/models` list; the GUI shows selectable Provider/Model combos with a "Load list" action. | | U3 | An invalid model produced only `OpenCode message failed: HTTP 500` — opaque. | **Fixed in source**: the error now names `provider/model` and says to check the model exists and the credentials are valid. (Needs the next ISO.) | | U4 | At first boot the DMS app-launcher/overview overlay was open on top of the welcome window, hiding the primary action until dismissed. | **Open**: reproduced once; investigating whether it is a DMS first-run behaviour or a focus race. | | U5 | The OpenCode model default `deepseek-chat` is not present on every server, so the first attempt failed. | **Mitigated**: the picker now lists only real models. Considering defaulting the model to unset and requiring a pick. | | U6 | **Showstopper.** A fully-validated OpenCode install was refused at the final step with *"installing Phoebus requires a successful, validated AI interaction"*. The validated state was cleared by **unrelated** configuration and secret changes (selecting a disk, typing the account password, setting the LUKS passphrase). | **Fixed**: the AI validation is now bound to a digest of only the AI connection settings; non-AI config changes preserve it, and only an AI provider/model/endpoint/server change or an AI credential change invalidates it. Regression tests: `a_non_ai_config_change_keeps_the_validated_interaction`, `an_account_secret_does_not_invalidate_but_an_ai_key_does`. | | U7 | The account/encryption step required several attempts; the password/passphrase minimum was only revealed as a red error after pressing Next, and the offending fields could be scrolled off-screen. | **Mitigated**: password/passphrase fields now carry an "At least 8 characters" / "Repeat …" placeholder hint before submission. | U6 blocked a real install; U1–U3 and U6–U7 are implemented, U4 remains open. The user's session reached the AI step, obtained and applied a validated OpenCode proposal, selected the disk, completed the account step and the review — and only then hit U6. Evidence: `/var/tmp/phoebus-vm/user-session3/` (`shots/0008`–`0016`, `observations.log`). OpenCode protocol/transport tests: 26 executor unit tests + 7 real-HTTP integration tests + 2 live tests (skipped offline, real inference when configured). Bridge gate tests: 5. Full suite: 336 Rust tests. ## Run matrix (this session, 2026-09-22) ```json { "source_revision": "7e9fa9c", "iso_sha256": "428dbd7547db8690596bf5c3a0d7a6fa8aee4c9f9eca1e926c216712dd050b36", "static_binary_sha256": "2f9cd4b36a12e469d2b6187166acaf8b11f7f382cc19f7ad08bb2a809c855591", "runs": [ {"scenario": "gui", "result": "PASS 23/23", "exit_code": 0, "artifacts": "/var/tmp/phoebus-vm/gui-gui-20260922T025537Z-69f754cf"}, {"scenario": "make test (rust + vmsafety)", "result": "PASS", "exit_code": 0}, {"scenario": "cargo fmt --check", "result": "PASS", "exit_code": 0}, {"scenario": "cargo clippy --workspace --all-targets", "result": "PASS (warnings only)", "exit_code": 0}, {"scenario": "opencode probe (live 1.18.32)", "result": "ready", "exit_code": 0}, {"scenario": "opencode probe (unauth LAN bind)", "result": "authentication_required", "exit_code": 1}, {"scenario": "noninteractive install + detached boot", "result": "NOT RUN on the final ISO", "exit_code": null}, {"scenario": "full graphical install past AI", "result": "BLOCKED", "exit_code": null}, {"scenario": "rollback VM acceptance", "result": "NOT RUN", "exit_code": null}, {"scenario": "AI sandbox canary VM test", "result": "NOT RUN", "exit_code": null}, {"scenario": "second physical computer OpenCode transport", "result": "BLOCKED", "exit_code": null} ] } ``` ## Confirmed defects fixed | ID | Defect | Symptom | Fix | Regression test | |----|--------|---------|-----|-----------------| | D1 | Disk safety was not revalidated before destroying the target | A disk that disappeared, was mounted, became the live medium, or was replaced under the same name between selection and install would be partitioned on the strength of a stale review | `Installer::revalidate_target_disk` re-probes and compares identity in `storage_prepare` | `orchestrate::tests::{revalidation_refuses_a_disappeared_disk,revalidation_refuses_a_now_protected_disk,revalidation_refuses_reused_device_name_with_a_new_size,...}` | | D2 | `probe_disks` parsed human-readable `lsblk` sizes as bytes | Every disk displayed `0 B`; the size-identity check was blind | pass `-b` to `lsblk` | `probe::tests::probe_disks_classifies_optical_as_protected` asserts the byte size | | D3 | The bridge could replay or overlap a destructive install | A second `start_install` (retry, second frontend, reconnect) could re-run storage preparation | approval is single-use, consumed before work; `installing`/`installed` guards | `start_install_cannot_be_replayed_after_success` | | D4 | The welcome's primary action was not keyboard-focused | No focus ring; Return did not open the installer | request activation + `forceActiveFocus()`; regression test also rejects the invalid `onActiveChanged` signal | `welcome_primary_action_is_keyboard_reachable` | | D5 | Escape during installation silently closed the frontend | A destructive operation could be abandoned with no explanation | Escape is refused during progress and the UI says so | `progress_cannot_be_abandoned_silently` | | D6 | Provider transport accepted unsafe URL schemes / unbounded responses / followed no protocol allowlist | A config-supplied endpoint could target `file:`, inject curl config, or return a huge body | `validate_endpoint`, `proto = "=http,https"`, `max-filesize`, redirects disabled | `endpoint_validation_rejects_unsafe_schemes_and_injection`, `curl_transport_refuses_an_unsafe_endpoint`, `curl_transport_bounds_the_response_and_disables_redirects` | | D7 | `TargetRoot` followed symlinks out of the target | A planted symlink could redirect an installer write to the host | `reject_symlink_components` in `write`/`ensure_dir` | `target::tests::{refuses_to_write_through_an_escaping_symlink,refuses_a_symlinked_leaf}` | | D8 | The VM harness documented disk safety but did not enforce it, and reported observations rather than results | A mistyped `DISK` could point at a real device; a timeout could look like success; shell interpolation in screenshot conversion | `vmsafety.py`; per-run OVMF/QMP; structured results; argument arrays | `test_vmsafety.py` (16 tests), run by `make test` | ## Suspected risks (reproduced or bounded) - **Prompt injection in device labels / model output.** Typed proposals pass `deny_unknown_fields` and a closed action set; device labels are display-only. Bounded by tests (`phoebus-ai` parse/allowlist). No VM canary run. - **Live sudoers is `NOPASSWD: ALL` (live-only).** It does not reach the installed system, but the GUI bridge uses it. Narrowing is a recorded follow-up. - **OpenCode `/config/providers` returns provider credentials in cleartext.** Upstream behaviour; the probe keeps only counts and docs mandate loopback + tunnel. Not a Phoebus defect, but a deployment hazard. ## Missing verification (blocked / not run) - Full graphical installation past the AI step and two fresh-disk graphical installs: **BLOCKED** — no provider key and no local model; normal install requires a connected model. - `install`/`boot` modes on the final ISO: **NOT RUN** (the earlier appliance build passed `INSTALL-RC=0` + detached boot; the final ISO's binary changed, so this is not automatically re-verified). - Rollback acceptance, AI sandbox canary, installed Mango/dwl, physical seat, NVIDIA: **NOT RUN / unverified**. - Second physical computer OpenCode transport: **BLOCKED**; host-local forwarded hop and unauthenticated-LAN refusal were tested instead.