feat(ai): show a concrete action example; record VM live-agent and local-model evidence
Some checks are pending
ci / test (push) Waiting to run
Some checks are pending
ci / test (push) Waiting to run
This commit is contained in:
parent
685321f85b
commit
d96aae1cfa
5 changed files with 53 additions and 8 deletions
|
|
@ -5,6 +5,23 @@ this file is the curated, human-readable summary.
|
|||
|
||||
## Unreleased — Fast defaults with user-controlled source builds
|
||||
|
||||
- **VM live AI: real capable agent, and a local-model prompt fix (2026-09-29).**
|
||||
Booted the golden overlay and, from inside the guest, reached the host's real
|
||||
`opencode` agent through the loopback tunnel: `sys ask "list installed
|
||||
packages" --ai-backend opencode --opencode-provider opencode-go
|
||||
--opencode-model deepseek-v4.1-flash` returned a valid Assistant Plan + digest
|
||||
(`CASE live-agent PASS`), and a hostile shell/private-file instruction was
|
||||
refused/downgraded (`CASE live-agent-hostile-rejected PASS`)
|
||||
(`serial-boot-20260929T102703Z-ffd1ec25`). A real local model (Qwen2.5-1.5B
|
||||
GGUF served by `llama-server`) was also driven through the direct sandbox in
|
||||
both scopes; it reached the model, but the 1.5B model could not satisfy the
|
||||
exact typed schema (bare-string action, then a missing `rationale` reason) and
|
||||
the correction prompt exceeded its 4096-token context, so `sys` failed closed
|
||||
— model capacity, not a `sys` defect. The assistant prompt now shows the
|
||||
concrete action-object shape (`{"action":"list_packages"}`) instead of only
|
||||
`[...]` placeholders, which made the small model emit action objects. Gates:
|
||||
`cargo fmt`, `clippy --workspace --all-targets -- -D warnings`,
|
||||
`cargo test --workspace` 843 passed / 0 failed.
|
||||
- **VM-verified AI sandbox and unprivileged AI path (2026-09-28).** The first
|
||||
disposable-VM run of the newly built static `sys` (overlay on the golden
|
||||
installed image, `serial-boot-20260929T092147Z-6f88907a`) found and fixed two
|
||||
|
|
|
|||
|
|
@ -25,6 +25,18 @@ commit it. It should be short and current; use `git log` for detail.
|
|||
call (`sys ask` with a real provider is still VM/live NotRun) but it closes
|
||||
the "connected-model path only tested with fake transports" gap. Gates:
|
||||
`cargo test --workspace` 841 passed / 0 failed.
|
||||
- **VM live AI (2026-09-29, VM-VERIFIED):** from inside the installed guest,
|
||||
the host's real `opencode` agent was reached through the loopback tunnel and
|
||||
`sys ask` produced a valid plan (`live-agent`), with a hostile instruction
|
||||
refused/downgraded (`live-agent-hostile-rejected`)
|
||||
(`serial-boot-20260929T102703Z-ffd1ec25`). A real local model (Qwen2.5-1.5B
|
||||
GGUF via `llama-server`) was also driven through the direct sandbox in both
|
||||
scopes; it reached the model but failed closed on schema-noncompliant output.
|
||||
The assistant prompt now shows a concrete action-object example
|
||||
(`{"action":"list_packages"}`), which made the small model emit action
|
||||
objects; a capable model is still required, and a 4096-token local context
|
||||
can be exceeded by the one-shot correction turn. Source conversion and
|
||||
tuned-kernel builds remain NotRun (multi-hour compiles).
|
||||
- **First VM test loop on the golden image (2026-09-28, VM-VERIFIED):** booted a
|
||||
fresh overlay of the installed `phoebus-headless-80g.qcow2` with the static
|
||||
`sys` from HEAD and found two real defects, both fixed. (1) Unprivileged
|
||||
|
|
|
|||
|
|
@ -98,7 +98,8 @@ Booting that recovery entry offline remains a manual gate.
|
|||
| Provider credential + endpoint boundary | yes | yes (unsafe key characters refused before curl config; userinfo endpoint refused; derived base64 token redacted) | no | n/a |
|
||||
| AI typed-action schema (exact, incl. unit-variant extra keys) | yes | yes (plan/step/investigate actions via `validate_action_value`; smuggled extra key refused) | no | n/a |
|
||||
| AI connected-model path over a real local stub | yes | yes (4 e2e cases: valid plan executes; request has no home path/credential; shell + smuggled key + malformed fail closed) | no (local stub, not a live model) | n/a |
|
||||
| Live real-agent call (`opencode` backend, host) | yes | yes (`sys ask "list installed packages"` → valid plan + digest, exit 0; hostile shell/private-file instruction refused and downgraded; OpenCode tools disabled; no credential in the log) | host-level only (2026-09-28, local `opencode serve` + `opencode-go/deepseek-v4.1-flash`); VM/direct-provider live call remains NotRun | n/a |
|
||||
| Live real-agent call (`opencode` backend) | yes | yes (`sys ask "list installed packages"` → valid plan + digest, exit 0; hostile shell/private-file instruction refused and downgraded; OpenCode tools disabled; no credential in the log) | yes — host direct (2026-09-28) and **from inside the installed VM through the loopback tunnel** (`serial-boot-20260929T102703Z-ffd1ec25`, `live-agent` + `live-agent-hostile-rejected`) with `opencode-go/deepseek-v4.1-flash` | n/a |
|
||||
| Local model through the direct sandbox (VM) | yes | yes (real Qwen2.5-1.5B GGUF via `llama-server`; both sandbox scopes reached the model; schema-noncompliant output failed closed) | VM-run 2026-09-29: reached and failed closed; a 1.5B model is too weak for exact typed plans and its 4096-token context was exceeded by the correction turn — a capable model is required (a capable agent produces valid plans) | n/a |
|
||||
| Privilege derived only from the process identity | yes | yes (an environment variable cannot forge uid 0; the removed `PHOEBUS_ALLOW_UNPRIVILEGED_EXEC` regression test) | no | n/a |
|
||||
| AI OS sandbox (systemd hardened unit) | yes | yes (argv + sandboxed-send; system properties for root, user-manager subset for non-root) | yes (`serial-boot-20260929T092147Z-6f88907a`: `AI_ROOT_RC=0` system sandbox, `AI_USER_RC=0` `--user` sandbox; previously the user path hung on a polkit `manage-units` prompt) | no |
|
||||
| Unprivileged read-only AI plan completes | yes | yes (transaction recording best-effort for non-root; mandatory for privileged mutations) | yes (`serial-boot-20260929T092147Z-6f88907a`: `AI_NOSANDBOX_RC=0` as `phoebus`) | n/a |
|
||||
|
|
|
|||
|
|
@ -45,9 +45,21 @@ the static `sys` built from HEAD; the base image was never modified.
|
|||
revision-changing refresh / strict AppArmor confinement still needs an
|
||||
external Store revision and an AppArmor-ready guest kernel.
|
||||
- **NotRun in this loop:** source conversion and tuned-kernel build/boot/fallback
|
||||
(multi-hour compiles on the disposable image); a live direct-provider model in
|
||||
the VM (no credential/GPU); recovery-entry BLS selection (manual gate);
|
||||
NVIDIA.
|
||||
(multi-hour compiles on the disposable image); recovery-entry BLS selection
|
||||
(manual gate); NVIDIA.
|
||||
- **Live AI in the VM (`serial-boot-20260929T102703Z-ffd1ec25`, driver PASS=6):**
|
||||
from inside the guest, the host's real `opencode` agent was reached through the
|
||||
loopback tunnel (`fwd.py`). `sys ask "list installed packages" --ai-backend
|
||||
opencode --opencode-provider opencode-go --opencode-model deepseek-v4.1-flash`
|
||||
returned a valid Assistant Plan + digest (`live-agent` PASS); a hostile
|
||||
shell/private-file instruction was refused/downgraded
|
||||
(`live-agent-hostile-rejected` PASS). Separately, a real local model
|
||||
(Qwen2.5-1.5B GGUF via `llama-server`) was driven through the direct sandbox in
|
||||
both scopes (`serial-boot-20260929T101322Z-b3d002ed`); it reached the model and
|
||||
failed closed on schema-noncompliant output. The assistant prompt now shows a
|
||||
concrete action-object example, which made the small model emit action objects;
|
||||
a capable model is still required for valid plans, and a very small local
|
||||
context can be exceeded by the one-shot correction turn.
|
||||
|
||||
## 2026-09-28 Headless dual-target golden install (Run 13) & detached boot proven
|
||||
|
||||
|
|
|
|||
|
|
@ -81,10 +81,12 @@ inspect_candidate, check_backend, inspect_installation, list_packages, \
|
|||
package_info, package_owners, package_dependencies, depclean, clean_cache, \
|
||||
check_updates, apply_updates.
|
||||
|
||||
Each reply is ONE JSON object with version 1 and a `kind`:
|
||||
Each reply is ONE JSON object with version 1 and a `kind`. An action is always \
|
||||
an object with an `action` field — for example {\"action\":\"list_packages\"} or \
|
||||
{\"action\":\"enable_service\",\"name\":\"sshd\"} — never a bare string.
|
||||
|
||||
1. Investigate (read-only evidence): \
|
||||
{\"version\":1,\"kind\":\"investigate\",\"requests\":[<read-only action>,...]}. \
|
||||
{\"version\":1,\"kind\":\"investigate\",\"requests\":[{\"action\":\"check_backend\",\"backend\":\"flatpak\"}]}. \
|
||||
Use this to answer search, inventory, information and update-check requests from \
|
||||
real evidence. Useful tools: search_packages (query, optional backend), \
|
||||
discover_candidates, inspect_candidate, check_backend, inspect_installation, \
|
||||
|
|
@ -92,8 +94,9 @@ list_packages (optional backend/filter), package_info, check_updates \
|
|||
(optional backend), package_owners, package_dependencies. Never request a \
|
||||
mutating action as evidence.
|
||||
|
||||
2. Plan or answer: {\"version\":1,\"summary\":\"...\",\"actions\":[...],\
|
||||
\"rationale\":[{\"action\":\"...\",\"reason\":\"...\"}]}. Give every action a \
|
||||
2. Plan or answer: {\"version\":1,\"summary\":\"...\",\"actions\":\
|
||||
[{\"action\":\"list_packages\"}],\
|
||||
\"rationale\":[{\"action\":\"list_packages\",\"reason\":\"...\"}]}. Give every action a \
|
||||
non-empty reason. For a pure question or a search summary, return an empty \
|
||||
`actions` list and put the answer in `summary`, grounded ONLY in the evidence \
|
||||
you were given: never invent package names, versions, descriptions, sizes, \
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue