fix(sys): cover every AI package lane in source review
Some checks are pending
ci / test (push) Waiting to run

This commit is contained in:
Phoebus 2026-10-03 12:35:16 -07:00
parent c1aee13d3e
commit be423597f2
12 changed files with 232 additions and 97 deletions

View file

@ -5,6 +5,19 @@ this file is the curated, human-readable summary.
## Unreleased — Fast defaults with user-controlled source builds
- **Source review coverage correction (2026-10-03).** Candidate inspection is
now required for every AI package recommendation, including Portage source
and binary packages; a vendor binary ebuild need not appear in the separate
vendor lane. Plans and execution revalidate inspected candidate details for
every lane. A direct typed Flatpak install now refuses an unverified remote,
and the general assistant also refuses direct AppImage updates and Nix
preparation. Portage and Nix source labels no longer claim an official tree,
signed binhost, or pinned registry without establishing that fact. Rust
format, strict clippy, workspace tests, `make check`, and dynamic/static
release builds pass. The running VM's read-only Portage inspection reported
`review_required` and an unknown exact repository origin; no package was
installed.
- **Third-party package source review (2026-10-03).** AI installation now
requires `inspect_candidate` evidence for the exact third-party backend and
package ID before recommending it. The inspection reports source,

View file

@ -3,6 +3,24 @@
Rolling note between work sessions. **Update this file before you finish**, then
commit it. It should be short and current; use `git log` for detail.
- **Source review coverage correction (2026-10-03):** The prior note's claim
that vendor Portage packages were covered by a separate lane was incomplete:
vendor binary ebuilds can resolve as Portage source. AI install proposals now
require exact `inspect_candidate` evidence for every lane. The broker
re-inspects every selected candidate at plan and execution, and direct typed
Flatpak installs refuse unverified remotes. Portage and Nix labels now state
when source origin or registry revision is unknown. Direct general AI
AppImage updates and Nix preparation cannot bypass the install workflow.
Rust format, strict clippy, workspace tests, `make check`, and
dynamic/static release builds pass. The running VM showed a read-only
`app-editors/vim` inspection with `review_required` and unknown exact
repository origin (`/var/tmp/phoebus-vm/all-source-portage.png`). The updated
static binary was installed in the VM user's `~/.local/bin/sys` (SHA-256
`d687107f68d5559b13eafd75c2de4977759a7240e91768aa0c85436b38753a29`)
and the Assistant reconnected. No package was installed. A fresh VM install
remains unverified because `make doctor` reports 8 GiB free against its
20 GiB gate.
- **Third-party source health review (2026-10-03):** AI installation now must
inspect the exact candidate for third-party lanes before making a proposal.
The read-only report separates backend evidence from publisher trust and

View file

@ -48,6 +48,25 @@ disposable VM; `serial-driver.py recovery` now automates the same check.
## Matrix
Package source review coverage correction (2026-10-03): AI install proposals
from every lane now require `inspect_candidate`, including Portage source and
binary candidates that may encapsulate vendor software. The broker uses fresh
candidate inspection for the reviewed plan and revalidates it before execution;
direct typed Flatpak installs also reject an unverified remote. Portage
repository origin, binhost policy, Nix registry revision and independent
upstream advisory status are not proven by this check and are explicitly
described as unknown. Unit tests cover Portage inspection and direct Flatpak
remote rejection. Rust format, strict clippy, workspace tests, `make check`,
and dynamic/static release builds pass. The running VM's read-only inspection
of `app-editors/vim` returned `review_required` and the explicit unknown
repository origin (`/var/tmp/phoebus-vm/all-source-portage.png`). The updated
static binary was installed in the VM user's `~/.local/bin/sys` (SHA-256
`d687107f68d5559b13eafd75c2de4977759a7240e91768aa0c85436b38753a29`)
and the Assistant reconnected. No package was installed. A fresh VM install
under this correction remains unverified; `make doctor` reports 8 GiB free
against its 20 GiB VM gate.
Third-party source health review (2026-10-03): `inspect_candidate` now returns
structured `health` evidence. AI install proposals from Portage vendor, Nix,
Flatpak, Snap, AppImage and custom ebuild lanes require that exact inspection;

View file

@ -120,9 +120,10 @@ identify -> evidence -> AI investigation -> proposed selection
specified backend, ensuring the model evaluates and explains candidates for
that source while preserving the full investigation, explanation, and
approval cycle.
- **Third-party source review** requires the model to inspect the exact
candidate before recommending a Portage vendor, Nix, Flatpak, Snap,
AppImage, or custom ebuild source. `inspect_candidate` returns a structured
- **Package source review** requires the model to inspect the exact candidate
before recommending a package from any lane, including ordinary Portage
source and binary packages. A vendor binary ebuild can appear in the
Portage source lane. `inspect_candidate` returns a structured
health report: source, integrity evidence, reported version and freshness
limits, isolation, and a `hold` or `review_required` status. The model must
explain uncertainty rather than declare a package safe or current. For
@ -130,7 +131,12 @@ identify -> evidence -> AI investigation -> proposed selection
`flathub` install remote, that the remote uses HTTPS, and that GPG
verification is enabled. It refuses an unverified remote and repeats the
check before execution; a changed version, remote, or verification state
invalidates approval. The plan shows these source health findings. A hash or
invalidates approval. This Flatpak check also applies to direct typed
administrator actions. For all reviewed installs, the selected candidate
is inspected again and a changed version, source, or verification state
invalidates approval. Portage repository origin and Nix registry revision
are not currently established by this inspection and are labeled as such.
The plan shows these source health findings. A hash or
catalog entry alone does not authenticate an AppImage publisher, and this
review cannot prove that an upstream release is uncompromised.
- **General assistant plans** cannot use direct package install actions to

View file

@ -106,12 +106,13 @@ installation status or availability.
Rules: prefer inspection before mutation. Where an action takes a `kind`, it is \
exactly one of base-system, desktop-app, developer-tool, cli-tool, unclassified; a \
GUI application is kind desktop-app plus a separate `gui` boolean. \
For third-party software, request inspect_candidate for the exact backend and id \
For every package recommendation, request inspect_candidate for the exact backend and id \
before recommending an install. Describe the source verification, reported \
version, isolation and remaining uncertainty; do not claim a package is safe or \
current from a catalog listing, checksum or model knowledge alone. Use the \
dedicated sys install workflow for package installation; direct install_package, \
flatpak_install, snap_install, nix_install, appimage_import, appimage_acquire and custom_ebuild_install actions \
flatpak_install, snap_install, nix_prepare, nix_install, appimage_import, \
appimage_acquire, appimage_update and custom_ebuild_install actions \
are not accepted in a general assistant plan. \
Do not choose a software lane by \
category alone: use choose_package to compare the available lanes against the \

View file

@ -212,8 +212,8 @@ Portage owns the kernel, drivers, systemd, the host graphics/audio stack and boo
Prefer an existing installation and the user's saved preference; never migrate or duplicate \
silently. If the user asked for a specific backend, investigate only that lane and honour it. \
AppImage catalog leads are discovery links and cannot be proposed as verified install candidates. \
Before proposing a candidate from a source outside Gentoo's source tree or signed binhost, \
request inspect_candidate for that exact backend and id. Read its health report: check \
Before proposing any package, request inspect_candidate for that exact backend and id. \
Vendor binaries can appear as ordinary Portage source candidates. Read the health report: check \
source identity, integrity, version/freshness, isolation and the stated limits. A reported \
version is not proof of the latest upstream release; a hash is not proof of publisher identity. \
If the inspection is unavailable or on hold, do not claim the package is healthy. \
@ -528,9 +528,7 @@ pub fn investigate_install(
}
InstallStep::Propose { proposal, .. } => {
proposal.validate_structural().map_err(AiError::Rejected)?;
if proposal.backend.requires_source_review()
&& !inspected.contains(&(proposal.backend, proposal.candidate.clone()))
{
if !inspected.contains(&(proposal.backend, proposal.candidate.clone())) {
chat.messages.push(ChatMessage {
role: "assistant".into(),
content: content.clone(),
@ -609,8 +607,14 @@ mod tests {
.iter()
.map(|r| Evidence {
request: r.describe(),
output: if matches!(r, SystemAction::InspectCandidate { .. }) {
r#"{"backend":"flatpak","id":"org.mozilla.firefox","available":true,"health":{"status":"review_required"}}"#.into()
output: if let SystemAction::InspectCandidate { backend, id } = r {
serde_json::json!({
"backend": backend,
"id": id,
"available": true,
"health": {"status": "review_required"}
})
.to_string()
} else {
"[{\"backend\":\"flatpak\",\"id\":\"org.mozilla.firefox\"}]".into()
},
@ -743,6 +747,36 @@ mod tests {
assert_eq!(session.evidence.borrow().len(), 1);
}
#[test]
fn a_portage_candidate_also_requires_inspection() {
let proposal = r#"{"version":2,"kind":"propose","proposal":{"application":"chrome","backend":"portage-source","candidate":"www-client/google-chrome","reason":"requested"}}"#;
let inspect = r#"{"version":2,"kind":"investigate","requests":[{"action":"inspect_candidate","backend":"portage-source","id":"www-client/google-chrome"}]}"#;
let transport = crate::provider::ScriptedTransport::new(vec![
chat_response(proposal),
chat_response(inspect),
chat_response(proposal),
]);
let session = Sessions {
evidence: std::cell::RefCell::new(Vec::new()),
answer: None,
};
let actual = investigate_install(
ProviderKind::Local,
"http://127.0.0.1:11434/v1/chat/completions",
"m",
&facts(),
"install Chrome",
&transport,
&session,
&|_| Ok(()),
&Limits::default(),
)
.unwrap();
assert_eq!(actual.candidate, "www-client/google-chrome");
assert_eq!(session.evidence.borrow().len(), 1);
assert!(transport.requests()[1].contains("inspect_candidate"));
}
#[test]
fn unavailable_requires_candidate_discovery_and_stops_without_a_proposal() {
let unavailable =

View file

@ -105,20 +105,6 @@ impl Backend {
matches!(self, Backend::Flatpak | Backend::Snap)
}
/// Sources outside Gentoo's ordinary source tree and signed binhost need
/// an explicit inspection before an AI installation recommendation.
pub fn requires_source_review(self) -> bool {
matches!(
self,
Backend::PortageVendor
| Backend::Nix
| Backend::Flatpak
| Backend::Snap
| Backend::AppImage
| Backend::CustomEbuild
)
}
pub fn all() -> &'static [Backend] {
&[
Backend::PortageSource,

View file

@ -1410,6 +1410,17 @@ pub fn execute_inner(
crate::query::search(exec, root, query, *backend).render()
}
SystemAction::FlatpakInstall { app_id } => {
let source = chooser::candidate_detail(exec, root, Backend::Flatpak, app_id);
if source.health.status == "hold" {
return Err(ExecError::operation(
"sys",
"verify Flatpak source",
format!(
"{app_id} cannot be installed from an unverified Flatpak remote: {}",
source.candidate.notes.join("; ")
),
));
}
run_in_target(
exec,
root,

View file

@ -285,9 +285,9 @@ fn portage_candidate(
},
id,
if binary {
"official signed Gentoo binhost"
"configured Portage binhost (origin and signature policy not established here)"
} else {
"Gentoo source tree (/var/db/repos)"
"configured Portage repositories (exact repository not established here)"
},
);
c.verified = true;
@ -371,7 +371,11 @@ pub fn resolve_candidates(exec: &Exec, root: &Path, need: &Need) -> Vec<Candidat
// --- Nix ----------------------------------------------------------
if crate::nix::installed(root) {
let mut c = Candidate::new(Backend::Nix, name, "nixpkgs (pinned)");
let mut c = Candidate::new(
Backend::Nix,
name,
"configured nixpkgs registry (revision not established)",
);
c.verified = true;
c.update_reliability = UpdateReliability::Manual;
c.sandbox = SandboxLevel::None;
@ -618,11 +622,15 @@ pub fn candidate_health(candidate: &Candidate) -> CandidateHealth {
"review_required"
};
let integrity = match candidate.backend {
Backend::PortageSource => "Portage resolved this ebuild; it may fetch upstream source or a vendor binary, whose publisher is not authenticated by this inspection".into(),
Backend::PortageBinary => "Portage resolved a binary package; its configured binhost and package signature policy must be checked during installation".into(),
Backend::PortageVendor => "A vendor binary ebuild may fetch third-party executable code; a recorded digest does not establish publisher trust".into(),
Backend::Nix => "nixpkgs resolved this attribute; Nix must enforce trusted store signatures when substituting binaries, and this inspection has not independently audited the package".into(),
Backend::Snap => "The Snap Store reports this snap; snapd verifies store assertions during installation, but the name alone does not establish publisher trust".into(),
Backend::AppImage => "A managed SHA-256 records the bytes, but a hash alone does not authenticate the publisher".into(),
Backend::CustomEbuild => "The staged ebuild is untrusted executable build code; its digest only detects changes after review".into(),
Backend::Flatpak if candidate.verified => "The selected HTTPS Flatpak remote has GPG verification enabled; the artifact itself is verified during installation".into(),
_ if candidate.verified => "The configured package backend reports this candidate; this inspection has not independently verified the artifact".into(),
_ => "No cryptographic publisher or artifact verification is established".into(),
Backend::Flatpak => "No cryptographic publisher or artifact verification is established".into(),
};
let freshness = match candidate.version.as_deref() {
Some(version) if !version.trim().is_empty() => format!(
@ -1042,7 +1050,11 @@ pub fn resolve_for_backend(
&& phoebus_core::action::valid_nix_attribute(application)
&& crate::nix::verify_attribute(exec, root, application)
{
let mut c = Candidate::new(Backend::Nix, application, "nixpkgs (pinned)");
let mut c = Candidate::new(
Backend::Nix,
application,
"configured nixpkgs registry (revision not established)",
);
c.verified = true;
c.update_reliability = UpdateReliability::Manual;
c.notes.push(
@ -1405,6 +1417,16 @@ mod tests {
let report = candidate_health(&staged);
assert_eq!(report.status, "hold");
assert!(report.integrity.contains("untrusted executable build code"));
let mut portage = Candidate::new(
Backend::PortageSource,
"www-client/google-chrome",
"Gentoo tree",
);
portage.verified = true;
let report = candidate_health(&portage);
assert_eq!(report.status, "review_required");
assert!(report.integrity.contains("vendor binary"));
}
fn ok(stdout: &str) -> CommandOutput {

View file

@ -349,20 +349,22 @@ pub fn revalidate(exec: &Exec, root: &Path, resolved: &ResolvedInstall) -> Resul
"the reviewed candidate is no longer available from real metadata",
)
})?;
let checked;
let current = if backend == Backend::Flatpak {
checked = chooser::candidate_detail(exec, root, backend, &current.id);
if checked.health.status == "hold" {
return Err(ExecError::operation(
"install",
"revalidate",
"the Flatpak source no longer passes the HTTPS and GPG checks",
));
}
&checked.candidate
} else {
current
};
let checked = chooser::candidate_detail(exec, root, backend, &current.id);
if !checked.candidate.available {
return Err(ExecError::operation(
"install",
"revalidate",
"the inspected candidate is no longer available",
));
}
if backend == Backend::Flatpak && checked.health.status == "hold" {
return Err(ExecError::operation(
"install",
"revalidate",
"the Flatpak source no longer passes the HTTPS and GPG checks",
));
}
let current = &checked.candidate;
if current.version != resolved.candidate.version {
return Err(ExecError::operation(
"install",
@ -597,46 +599,50 @@ fn finalize(
source: InstallSource,
allow_duplicate: bool,
) -> Result<ResolvedInstall> {
let candidate = if candidate.backend == Backend::Flatpak {
let detail = chooser::candidate_detail(exec, root, candidate.backend, &candidate.id);
if detail.health.status == "hold" {
return Err(ExecError::operation(
"install",
"source health review",
format!(
"{} cannot be installed from an unverified Flatpak remote: {}",
candidate.id,
detail.candidate.notes.join("; ")
),
));
}
detail.candidate
} else {
candidate
};
let detail = chooser::candidate_detail(exec, root, candidate.backend, &candidate.id);
if !detail.candidate.available {
return Err(ExecError::operation(
"install",
"source health review",
format!(
"{} was no longer available during source inspection",
candidate.id
),
));
}
if candidate.backend == Backend::Flatpak && detail.health.status == "hold" {
return Err(ExecError::operation(
"install",
"source health review",
format!(
"{} cannot be installed from an unverified Flatpak remote: {}",
candidate.id,
detail.candidate.notes.join("; ")
),
));
}
let candidate = detail.candidate;
let inspection = chooser::installed_copies(exec, root, &request.application);
let mut warnings = Vec::new();
if candidate.backend.requires_source_review() {
let health = chooser::candidate_health(&candidate);
if source == InstallSource::AiRecommendation && health.status == "hold" {
return Err(ExecError::operation(
"install",
"source health review",
format!(
"{} cannot be recommended from {}: {}. Use a source with verifiable \
provenance or inspect and select an explicit legacy source yourself",
candidate.id,
candidate.backend.as_str(),
health.integrity
),
));
}
warnings.push(format!(
"Source health: {}. {}",
health.integrity, health.freshness
let health = chooser::candidate_health(&candidate);
if source == InstallSource::AiRecommendation && health.status == "hold" {
return Err(ExecError::operation(
"install",
"source health review",
format!(
"{} cannot be recommended from {}: {}. Use a source with verifiable \
provenance or inspect and select an explicit legacy source yourself",
candidate.id,
candidate.backend.as_str(),
health.integrity
),
));
warnings.push(health.limitation.into());
}
warnings.push(format!(
"Source health: {}. {}",
health.integrity, health.freshness
));
warnings.push(health.limitation.into());
let candidate_is_portage = candidate.backend.is_portage();
if !inspection.copies.is_empty() {
let summary = inspection
@ -1375,6 +1381,20 @@ mod tests {
err.to_string().contains("unverified Flatpak remote"),
"{err}"
);
let auth = crate::auth::authorize(&crate::auth::AutoConfirm, "review", "confirm").unwrap();
let err = actions::execute(
&exec,
&SystemAction::FlatpakInstall {
app_id: "org.mozilla.firefox".into(),
},
Some(&auth),
root,
)
.unwrap_err();
assert!(
err.to_string().contains("unverified Flatpak remote"),
"{err}"
);
verified.store(true, Ordering::SeqCst);
let resolved = resolve_explicit(&exec, root, &request).unwrap();

View file

@ -824,9 +824,11 @@ pub fn plan_general(
SystemAction::InstallPackage { .. }
| SystemAction::FlatpakInstall { .. }
| SystemAction::SnapInstall { .. }
| SystemAction::NixPrepare { .. }
| SystemAction::NixInstall { .. }
| SystemAction::AppImageImport { .. }
| SystemAction::AppImageAcquire { .. }
| SystemAction::AppImageUpdate { .. }
| SystemAction::CustomEbuildInstall { .. }
)
}) {

View file

@ -2010,22 +2010,25 @@ fn plan_general_refuses_a_foreign_lane_under_a_constraint() {
#[test]
fn plan_general_refuses_direct_third_party_install_actions() {
use phoebus_ai::provider::{chat_response, ScriptedTransport};
let transport = ScriptedTransport::new(vec![chat_response(
r#"{"version":1,"summary":"install it","actions":[{"action":"flatpak_install","app_id":"org.mozilla.firefox"}],"rationale":[{"action":"flatpak_install","reason":"requested"}]}"#,
)]);
let exec = make_test_exec(Path::new("/"));
let store = PlanStore::with_dir(tempdir().unwrap().keep());
let err = plan_general(
&store,
&exec,
Path::new("/"),
"install Firefox",
&SystemFacts::default(),
&general_ai(&transport),
None,
)
.unwrap_err();
assert!(err.to_string().contains("dedicated `sys install`"), "{err}");
for plan in [
r#"{"version":1,"summary":"install it","actions":[{"action":"flatpak_install","app_id":"org.mozilla.firefox"}],"rationale":[{"action":"flatpak_install","reason":"requested"}]}"#,
r#"{"version":1,"summary":"prepare it","actions":[{"action":"nix_prepare","packages":["hello"]}],"rationale":[{"action":"nix_prepare","reason":"requested"}]}"#,
] {
let transport = ScriptedTransport::new(vec![chat_response(plan)]);
let err = plan_general(
&store,
&exec,
Path::new("/"),
"install software",
&SystemFacts::default(),
&general_ai(&transport),
None,
)
.unwrap_err();
assert!(err.to_string().contains("dedicated `sys install`"), "{err}");
}
}
#[test]