feat(ai): show a concrete action example; record VM live-agent and local-model evidence
Some checks are pending
ci / test (push) Waiting to run

This commit is contained in:
Phoebus 2026-09-29 03:29:46 -07:00
parent 685321f85b
commit d96aae1cfa
5 changed files with 53 additions and 8 deletions

View file

@ -5,6 +5,23 @@ this file is the curated, human-readable summary.
## Unreleased — Fast defaults with user-controlled source builds
- **VM live AI: real capable agent, and a local-model prompt fix (2026-09-29).**
Booted the golden overlay and, from inside the guest, reached the host's real
`opencode` agent through the loopback tunnel: `sys ask "list installed
packages" --ai-backend opencode --opencode-provider opencode-go
--opencode-model deepseek-v4.1-flash` returned a valid Assistant Plan + digest
(`CASE live-agent PASS`), and a hostile shell/private-file instruction was
refused/downgraded (`CASE live-agent-hostile-rejected PASS`)
(`serial-boot-20260929T102703Z-ffd1ec25`). A real local model (Qwen2.5-1.5B
GGUF served by `llama-server`) was also driven through the direct sandbox in
both scopes; it reached the model, but the 1.5B model could not satisfy the
exact typed schema (bare-string action, then a missing `rationale` reason) and
the correction prompt exceeded its 4096-token context, so `sys` failed closed
— model capacity, not a `sys` defect. The assistant prompt now shows the
concrete action-object shape (`{"action":"list_packages"}`) instead of only
`[...]` placeholders, which made the small model emit action objects. Gates:
`cargo fmt`, `clippy --workspace --all-targets -- -D warnings`,
`cargo test --workspace` 843 passed / 0 failed.
- **VM-verified AI sandbox and unprivileged AI path (2026-09-28).** The first
disposable-VM run of the newly built static `sys` (overlay on the golden
installed image, `serial-boot-20260929T092147Z-6f88907a`) found and fixed two

View file

@ -25,6 +25,18 @@ commit it. It should be short and current; use `git log` for detail.
call (`sys ask` with a real provider is still VM/live NotRun) but it closes
the "connected-model path only tested with fake transports" gap. Gates:
`cargo test --workspace` 841 passed / 0 failed.
- **VM live AI (2026-09-29, VM-VERIFIED):** from inside the installed guest,
the host's real `opencode` agent was reached through the loopback tunnel and
`sys ask` produced a valid plan (`live-agent`), with a hostile instruction
refused/downgraded (`live-agent-hostile-rejected`)
(`serial-boot-20260929T102703Z-ffd1ec25`). A real local model (Qwen2.5-1.5B
GGUF via `llama-server`) was also driven through the direct sandbox in both
scopes; it reached the model but failed closed on schema-noncompliant output.
The assistant prompt now shows a concrete action-object example
(`{"action":"list_packages"}`), which made the small model emit action
objects; a capable model is still required, and a 4096-token local context
can be exceeded by the one-shot correction turn. Source conversion and
tuned-kernel builds remain NotRun (multi-hour compiles).
- **First VM test loop on the golden image (2026-09-28, VM-VERIFIED):** booted a
fresh overlay of the installed `phoebus-headless-80g.qcow2` with the static
`sys` from HEAD and found two real defects, both fixed. (1) Unprivileged

View file

@ -98,7 +98,8 @@ Booting that recovery entry offline remains a manual gate.
| Provider credential + endpoint boundary | yes | yes (unsafe key characters refused before curl config; userinfo endpoint refused; derived base64 token redacted) | no | n/a |
| AI typed-action schema (exact, incl. unit-variant extra keys) | yes | yes (plan/step/investigate actions via `validate_action_value`; smuggled extra key refused) | no | n/a |
| AI connected-model path over a real local stub | yes | yes (4 e2e cases: valid plan executes; request has no home path/credential; shell + smuggled key + malformed fail closed) | no (local stub, not a live model) | n/a |
| Live real-agent call (`opencode` backend, host) | yes | yes (`sys ask "list installed packages"` → valid plan + digest, exit 0; hostile shell/private-file instruction refused and downgraded; OpenCode tools disabled; no credential in the log) | host-level only (2026-09-28, local `opencode serve` + `opencode-go/deepseek-v4.1-flash`); VM/direct-provider live call remains NotRun | n/a |
| Live real-agent call (`opencode` backend) | yes | yes (`sys ask "list installed packages"` → valid plan + digest, exit 0; hostile shell/private-file instruction refused and downgraded; OpenCode tools disabled; no credential in the log) | yes — host direct (2026-09-28) and **from inside the installed VM through the loopback tunnel** (`serial-boot-20260929T102703Z-ffd1ec25`, `live-agent` + `live-agent-hostile-rejected`) with `opencode-go/deepseek-v4.1-flash` | n/a |
| Local model through the direct sandbox (VM) | yes | yes (real Qwen2.5-1.5B GGUF via `llama-server`; both sandbox scopes reached the model; schema-noncompliant output failed closed) | VM-run 2026-09-29: reached and failed closed; a 1.5B model is too weak for exact typed plans and its 4096-token context was exceeded by the correction turn — a capable model is required (a capable agent produces valid plans) | n/a |
| Privilege derived only from the process identity | yes | yes (an environment variable cannot forge uid 0; the removed `PHOEBUS_ALLOW_UNPRIVILEGED_EXEC` regression test) | no | n/a |
| AI OS sandbox (systemd hardened unit) | yes | yes (argv + sandboxed-send; system properties for root, user-manager subset for non-root) | yes (`serial-boot-20260929T092147Z-6f88907a`: `AI_ROOT_RC=0` system sandbox, `AI_USER_RC=0` `--user` sandbox; previously the user path hung on a polkit `manage-units` prompt) | no |
| Unprivileged read-only AI plan completes | yes | yes (transaction recording best-effort for non-root; mandatory for privileged mutations) | yes (`serial-boot-20260929T092147Z-6f88907a`: `AI_NOSANDBOX_RC=0` as `phoebus`) | n/a |

View file

@ -45,9 +45,21 @@ the static `sys` built from HEAD; the base image was never modified.
revision-changing refresh / strict AppArmor confinement still needs an
external Store revision and an AppArmor-ready guest kernel.
- **NotRun in this loop:** source conversion and tuned-kernel build/boot/fallback
(multi-hour compiles on the disposable image); a live direct-provider model in
the VM (no credential/GPU); recovery-entry BLS selection (manual gate);
NVIDIA.
(multi-hour compiles on the disposable image); recovery-entry BLS selection
(manual gate); NVIDIA.
- **Live AI in the VM (`serial-boot-20260929T102703Z-ffd1ec25`, driver PASS=6):**
from inside the guest, the host's real `opencode` agent was reached through the
loopback tunnel (`fwd.py`). `sys ask "list installed packages" --ai-backend
opencode --opencode-provider opencode-go --opencode-model deepseek-v4.1-flash`
returned a valid Assistant Plan + digest (`live-agent` PASS); a hostile
shell/private-file instruction was refused/downgraded
(`live-agent-hostile-rejected` PASS). Separately, a real local model
(Qwen2.5-1.5B GGUF via `llama-server`) was driven through the direct sandbox in
both scopes (`serial-boot-20260929T101322Z-b3d002ed`); it reached the model and
failed closed on schema-noncompliant output. The assistant prompt now shows a
concrete action-object example, which made the small model emit action objects;
a capable model is still required for valid plans, and a very small local
context can be exceeded by the one-shot correction turn.
## 2026-09-28 Headless dual-target golden install (Run 13) & detached boot proven

View file

@ -81,10 +81,12 @@ inspect_candidate, check_backend, inspect_installation, list_packages, \
package_info, package_owners, package_dependencies, depclean, clean_cache, \
check_updates, apply_updates.
Each reply is ONE JSON object with version 1 and a `kind`:
Each reply is ONE JSON object with version 1 and a `kind`. An action is always \
an object with an `action` field — for example {\"action\":\"list_packages\"} or \
{\"action\":\"enable_service\",\"name\":\"sshd\"} — never a bare string.
1. Investigate (read-only evidence): \
{\"version\":1,\"kind\":\"investigate\",\"requests\":[<read-only action>,...]}. \
{\"version\":1,\"kind\":\"investigate\",\"requests\":[{\"action\":\"check_backend\",\"backend\":\"flatpak\"}]}. \
Use this to answer search, inventory, information and update-check requests from \
real evidence. Useful tools: search_packages (query, optional backend), \
discover_candidates, inspect_candidate, check_backend, inspect_installation, \
@ -92,8 +94,9 @@ list_packages (optional backend/filter), package_info, check_updates \
(optional backend), package_owners, package_dependencies. Never request a \
mutating action as evidence.
2. Plan or answer: {\"version\":1,\"summary\":\"...\",\"actions\":[...],\
\"rationale\":[{\"action\":\"...\",\"reason\":\"...\"}]}. Give every action a \
2. Plan or answer: {\"version\":1,\"summary\":\"...\",\"actions\":\
[{\"action\":\"list_packages\"}],\
\"rationale\":[{\"action\":\"list_packages\",\"reason\":\"...\"}]}. Give every action a \
non-empty reason. For a pure question or a search summary, return an empty \
`actions` list and put the answer in `summary`, grounded ONLY in the evidence \
you were given: never invent package names, versions, descriptions, sizes, \