Add an 'OpenCode agent' choice with server/password/provider/model fields, a Test connection action, and a secret-free frontend state line the harness asserts. Require exactly one installer session and assert the current step and AI state instead of inferring from a screenshot. Add a live-OpenCode acceptance make target and document the adapter, transport policy, states and workflows (ADR 0011 updated).
92 lines
8.9 KiB
Markdown
92 lines
8.9 KiB
Markdown
# Verification and findings register
|
|
|
|
Companion to `docs/STATUS.md`. The machine-readable per-run results live outside
|
|
git under `/var/tmp/phoebus-vm/<run-id>/results.json`; this file is the
|
|
human-readable register of what was found, fixed and still unverified.
|
|
|
|
## OpenCode adapter (2026-09-22, second pass)
|
|
|
|
Implemented a real OpenCode adapter (not just a probe), integrated through
|
|
canonical configuration (`ai.backend = "opencode"`), the GUI bridge, the
|
|
installer CLI and `sys ask`. Defects found while implementing and testing:
|
|
|
|
| ID | Defect | Fix | Test |
|
|
|----|--------|-----|------|
|
|
| O1 | No OpenCode proposal adapter; the GUI used the OpenAI-compatible transport | `phoebus-executor::opencode_client` implements the real session/message protocol as a `ModelTransport` | live + bridge end-to-end |
|
|
| O2 | `POST /session` was sent a `model` field per the OpenAPI schema | the shipped 1.18.32 rejects it (BadRequest); the model is selected on the message | real-HTTP integration |
|
|
| O3 | The probe had no credential input | `ProbeRequest.auth` + basic-auth header via stdin | auth integration test |
|
|
| O4 | The probe did not enforce the connection destination | `Endpoint::checked_target` resolves and requires loopback for plain http; the connection is pinned | `transport_policy_enforces_loopback_for_plain_http`, insecure-destination test |
|
|
| O5 | Handwritten HTTP read had no total deadline and a UTF-8-unsafe chunk decoder that fell back on malformed input | removed the handwritten parser; the established curl transport is used and bounds time, protocol, redirects and size | real-HTTP chunked/oversize/redirect/disconnect tests |
|
|
| O6 | The catalog was parsed without checking the HTTP status; a providers-shaped 500 could look like a catalog | status is checked before parsing; non-2xx is `incompatible` | `a_misleading_success_shaped_500_does_not_produce_a_catalog` |
|
|
| O7 | Catalog presence counted as readiness | `ready` is reserved for a validated response; the probe returns `model_catalog`/`reachable` | `an_empty_catalog_is_reachable_not_ready` |
|
|
| O8 | `probe_live_server_when_configured` passed when its env var was absent and only asserted reachability | replaced by env-gated `opencode_live` tests that assert a real inference; `PHOEBUS_OPENCODE_REQUIRE_LIVE=1` fails when the live run cannot happen | `make opencode-acceptance` |
|
|
| O9 | The credential test built an already-sanitised struct | tests feed credential-bearing server bodies and assert errors/catalogs never contain them | `one_shot_rejects_a_non_assistant_or_errored_reply`, `catalog_never_reads_provider_keys` |
|
|
| O10 | A provider name satisfied the mandatory-AI gate | both the bridge and the CLI require a validated interaction bound to the current configuration; config/secret changes invalidate it | `a_provider_name_alone_does_not_satisfy_the_ai_requirement`, `an_unreachable_provider_does_not_satisfy_the_ai_gate` |
|
|
| O11 | The proposal prompt allowed the model to key the change by action name | prompt now requires exactly `action` + `value` with explicit examples | live bridge `ai_check` |
|
|
| O12 | The GUI duplicate-session assertion accepted two pre-existing sessions | requires exactly one session before and after | harness |
|
|
| O13 | GUI navigation was inferred from a surviving process/screenshot | the frontend emits secret-free `phoebus-state` lines; the harness asserts the current step and AI state | harness |
|
|
|
|
OpenCode protocol/transport tests: 26 executor unit tests + 7 real-HTTP
|
|
integration tests + 2 live tests (skipped offline, real inference when
|
|
configured). Bridge gate tests: 5. Full suite: 336 Rust tests.
|
|
|
|
## Run matrix (this session, 2026-09-22)
|
|
|
|
```json
|
|
{
|
|
"source_revision": "7e9fa9c",
|
|
"iso_sha256": "428dbd7547db8690596bf5c3a0d7a6fa8aee4c9f9eca1e926c216712dd050b36",
|
|
"static_binary_sha256": "2f9cd4b36a12e469d2b6187166acaf8b11f7f382cc19f7ad08bb2a809c855591",
|
|
"runs": [
|
|
{"scenario": "gui", "result": "PASS 23/23", "exit_code": 0,
|
|
"artifacts": "/var/tmp/phoebus-vm/gui-gui-20260922T025537Z-69f754cf"},
|
|
{"scenario": "make test (rust + vmsafety)", "result": "PASS", "exit_code": 0},
|
|
{"scenario": "cargo fmt --check", "result": "PASS", "exit_code": 0},
|
|
{"scenario": "cargo clippy --workspace --all-targets", "result": "PASS (warnings only)", "exit_code": 0},
|
|
{"scenario": "opencode probe (live 1.18.32)", "result": "ready", "exit_code": 0},
|
|
{"scenario": "opencode probe (unauth LAN bind)", "result": "authentication_required", "exit_code": 1},
|
|
{"scenario": "noninteractive install + detached boot", "result": "NOT RUN on the final ISO", "exit_code": null},
|
|
{"scenario": "full graphical install past AI", "result": "BLOCKED", "exit_code": null},
|
|
{"scenario": "rollback VM acceptance", "result": "NOT RUN", "exit_code": null},
|
|
{"scenario": "AI sandbox canary VM test", "result": "NOT RUN", "exit_code": null},
|
|
{"scenario": "second physical computer OpenCode transport", "result": "BLOCKED", "exit_code": null}
|
|
]
|
|
}
|
|
```
|
|
|
|
## Confirmed defects fixed
|
|
|
|
| ID | Defect | Symptom | Fix | Regression test |
|
|
|----|--------|---------|-----|-----------------|
|
|
| D1 | Disk safety was not revalidated before destroying the target | A disk that disappeared, was mounted, became the live medium, or was replaced under the same name between selection and install would be partitioned on the strength of a stale review | `Installer::revalidate_target_disk` re-probes and compares identity in `storage_prepare` | `orchestrate::tests::{revalidation_refuses_a_disappeared_disk,revalidation_refuses_a_now_protected_disk,revalidation_refuses_reused_device_name_with_a_new_size,...}` |
|
|
| D2 | `probe_disks` parsed human-readable `lsblk` sizes as bytes | Every disk displayed `0 B`; the size-identity check was blind | pass `-b` to `lsblk` | `probe::tests::probe_disks_classifies_optical_as_protected` asserts the byte size |
|
|
| D3 | The bridge could replay or overlap a destructive install | A second `start_install` (retry, second frontend, reconnect) could re-run storage preparation | approval is single-use, consumed before work; `installing`/`installed` guards | `start_install_cannot_be_replayed_after_success` |
|
|
| D4 | The welcome's primary action was not keyboard-focused | No focus ring; Return did not open the installer | request activation + `forceActiveFocus()`; regression test also rejects the invalid `onActiveChanged` signal | `welcome_primary_action_is_keyboard_reachable` |
|
|
| D5 | Escape during installation silently closed the frontend | A destructive operation could be abandoned with no explanation | Escape is refused during progress and the UI says so | `progress_cannot_be_abandoned_silently` |
|
|
| D6 | Provider transport accepted unsafe URL schemes / unbounded responses / followed no protocol allowlist | A config-supplied endpoint could target `file:`, inject curl config, or return a huge body | `validate_endpoint`, `proto = "=http,https"`, `max-filesize`, redirects disabled | `endpoint_validation_rejects_unsafe_schemes_and_injection`, `curl_transport_refuses_an_unsafe_endpoint`, `curl_transport_bounds_the_response_and_disables_redirects` |
|
|
| D7 | `TargetRoot` followed symlinks out of the target | A planted symlink could redirect an installer write to the host | `reject_symlink_components` in `write`/`ensure_dir` | `target::tests::{refuses_to_write_through_an_escaping_symlink,refuses_a_symlinked_leaf}` |
|
|
| D8 | The VM harness documented disk safety but did not enforce it, and reported observations rather than results | A mistyped `DISK` could point at a real device; a timeout could look like success; shell interpolation in screenshot conversion | `vmsafety.py`; per-run OVMF/QMP; structured results; argument arrays | `test_vmsafety.py` (16 tests), run by `make test` |
|
|
|
|
## Suspected risks (reproduced or bounded)
|
|
|
|
- **Prompt injection in device labels / model output.** Typed proposals pass
|
|
`deny_unknown_fields` and a closed action set; device labels are display-only.
|
|
Bounded by tests (`phoebus-ai` parse/allowlist). No VM canary run.
|
|
- **Live sudoers is `NOPASSWD: ALL` (live-only).** It does not reach the
|
|
installed system, but the GUI bridge uses it. Narrowing is a recorded follow-up.
|
|
- **OpenCode `/config/providers` returns provider credentials in cleartext.**
|
|
Upstream behaviour; the probe keeps only counts and docs mandate loopback +
|
|
tunnel. Not a Phoebus defect, but a deployment hazard.
|
|
|
|
## Missing verification (blocked / not run)
|
|
|
|
- Full graphical installation past the AI step and two fresh-disk graphical
|
|
installs: **BLOCKED** — no provider key and no local model; normal install
|
|
requires a connected model.
|
|
- `install`/`boot` modes on the final ISO: **NOT RUN** (the earlier appliance
|
|
build passed `INSTALL-RC=0` + detached boot; the final ISO's binary changed,
|
|
so this is not automatically re-verified).
|
|
- Rollback acceptance, AI sandbox canary, installed Mango/dwl, physical seat,
|
|
NVIDIA: **NOT RUN / unverified**.
|
|
- Second physical computer OpenCode transport: **BLOCKED**; host-local forwarded
|
|
hop and unauthenticated-LAN refusal were tested instead.
|