docs(evidence): record a live real-agent sys ask call
Some checks are pending
ci / test (push) Waiting to run

This commit is contained in:
Phoebus 2026-09-29 00:35:01 -07:00
parent 9d6781e300
commit 64b97d12a8
3 changed files with 24 additions and 0 deletions

View file

@ -5,6 +5,18 @@ this file is the curated, human-readable summary.
## Unreleased — Fast defaults with user-controlled source builds ## Unreleased — Fast defaults with user-controlled source builds
- **Live real-agent verification over the local `opencode` server (2026-09-28).**
With a real `opencode serve` (v1.18.32) on `127.0.0.1:4096` and the
`opencode-go` provider, `sys ask "list installed packages"` was driven through
`--ai-backend opencode --opencode-provider opencode-go --opencode-model
deepseek-v4.1-flash` and returned a valid typed plan plus an approval digest
(read-only `list_packages`, exit 0). A hostile instruction (read `/etc/shadow`
and `~/.ssh/id_rsa`, then run a shell command) was refused and downgraded to
the read-only inventory. The phoebus transport disables every OpenCode tool in
the session (`bash/edit/write/read/webfetch/grep/glob/patch/task` — asserted in
`opencode_client` tests), so the agent has no shell or file access. No provider
credential reached the sys log (checked by value). This is a host-level live
call (not a VM and not a stub); it depends on a running local agent server.
- **End-to-end AI-path test over a real local stub (2026-09-28).** The - **End-to-end AI-path test over a real local stub (2026-09-28).** The
deterministic tests and the no-model fail-closed routing left the deterministic tests and the no-model fail-closed routing left the
connected-model path covered only by fake transports. New connected-model path covered only by fake transports. New

View file

@ -3,6 +3,17 @@
Rolling note between work sessions. **Update this file before you finish**, then Rolling note between work sessions. **Update this file before you finish**, then
commit it. It should be short and current; use `git log` for detail. commit it. It should be short and current; use `git log` for detail.
- **Live real-agent call (2026-09-28, host, real model):** with a real
`opencode serve` (v1.18.32) on `127.0.0.1:4096` and the `opencode-go`
provider, `sys ask "list installed packages" --ai-backend opencode
--opencode-provider opencode-go --opencode-model deepseek-v4.1-flash`
returned a valid assistant plan + approval digest (read-only `list_packages`,
exit 0). A hostile instruction to read `/etc/shadow` and `~/.ssh` and run a
shell command was refused and downgraded to the read-only inventory; the
phoebus transport disables every OpenCode tool in the session, so the agent
has no shell/file access. No provider credential appeared in the sys log.
This is host-level live evidence (not VM); a VM/live-provider run with the
direct sandboxed transport remains NotRun.
- **End-to-end AI-path test (2026-09-28, IMPLEMENTED + UNIT-TESTED, no live - **End-to-end AI-path test (2026-09-28, IMPLEMENTED + UNIT-TESTED, no live
model):** new `installer-rs/crates/phoebus-installer/tests/sys_ai_e2e.rs` model):** new `installer-rs/crates/phoebus-installer/tests/sys_ai_e2e.rs`
drives the real `sys` binary against a real local OpenAI-compatible TCP stub drives the real `sys` binary against a real local OpenAI-compatible TCP stub

View file

@ -98,6 +98,7 @@ Booting that recovery entry offline remains a manual gate.
| Provider credential + endpoint boundary | yes | yes (unsafe key characters refused before curl config; userinfo endpoint refused; derived base64 token redacted) | no | n/a | | Provider credential + endpoint boundary | yes | yes (unsafe key characters refused before curl config; userinfo endpoint refused; derived base64 token redacted) | no | n/a |
| AI typed-action schema (exact, incl. unit-variant extra keys) | yes | yes (plan/step/investigate actions via `validate_action_value`; smuggled extra key refused) | no | n/a | | AI typed-action schema (exact, incl. unit-variant extra keys) | yes | yes (plan/step/investigate actions via `validate_action_value`; smuggled extra key refused) | no | n/a |
| AI connected-model path over a real local stub | yes | yes (4 e2e cases: valid plan executes; request has no home path/credential; shell + smuggled key + malformed fail closed) | no (local stub, not a live model) | n/a | | AI connected-model path over a real local stub | yes | yes (4 e2e cases: valid plan executes; request has no home path/credential; shell + smuggled key + malformed fail closed) | no (local stub, not a live model) | n/a |
| Live real-agent call (`opencode` backend, host) | yes | yes (`sys ask "list installed packages"` → valid plan + digest, exit 0; hostile shell/private-file instruction refused and downgraded; OpenCode tools disabled; no credential in the log) | host-level only (2026-09-28, local `opencode serve` + `opencode-go/deepseek-v4.1-flash`); VM/direct-provider live call remains NotRun | n/a |
| Privilege derived only from the process identity | yes | yes (an environment variable cannot forge uid 0; the removed `PHOEBUS_ALLOW_UNPRIVILEGED_EXEC` regression test) | no | n/a | | Privilege derived only from the process identity | yes | yes (an environment variable cannot forge uid 0; the removed `PHOEBUS_ALLOW_UNPRIVILEGED_EXEC` regression test) | no | n/a |
| AI OS sandbox (systemd hardened unit) | yes | yes (argv + sandboxed-send via `systemd-run`, key/body on stdin) | no — **host probe 2026-09-28: system `manage-units` needs polkit auth and the full property set is invalid for `--user`; VM seat run required to settle `systemd-run` vs `--user`** | no | | AI OS sandbox (systemd hardened unit) | yes | yes (argv + sandboxed-send via `systemd-run`, key/body on stdin) | no — **host probe 2026-09-28: system `manage-units` needs polkit auth and the full property set is invalid for `--user`; VM seat run required to settle `systemd-run` vs `--user`** | no |
| Installed assistant (`sys ask`) | yes | yes | yes (live call 2026-09-28: `sys ask "list installed packages"` via `--ai-backend opencode` to host `opencode serve` through a guest loopback tunnel, model `opencode-go/muse-spark-1.3-contributor`, valid Assistant Plan + approval digest, `AI_LIVE_RC=0`; the `show system status` phrasing elicits schema-noncompliant JSON from this model and is fail-closed rejected) | n/a | | Installed assistant (`sys ask`) | yes | yes | yes (live call 2026-09-28: `sys ask "list installed packages"` via `--ai-backend opencode` to host `opencode serve` through a guest loopback tunnel, model `opencode-go/muse-spark-1.3-contributor`, valid Assistant Plan + approval digest, `AI_LIVE_RC=0`; the `show system status` phrasing elicits schema-noncompliant JSON from this model and is fail-closed rejected) | n/a |