feat(sys): discover external Portage overlay leads
Some checks are pending
ci / test (push) Waiting to run
Some checks are pending
ci / test (push) Waiting to run
This commit is contained in:
parent
c9c17cc16f
commit
d461721152
7 changed files with 284 additions and 1 deletions
|
|
@ -5,6 +5,13 @@ this file is the curated, human-readable summary.
|
|||
|
||||
## Unreleased — Fast defaults with user-controlled source builds
|
||||
|
||||
- **External Portage overlay discovery (2026-10-02).** `sys search <term>
|
||||
--legacy --overlays` adds read-only GPO leads to the configured package
|
||||
catalogue. Leads are visibly unverified and never become install candidates
|
||||
merely because a remote index lists them. The network lookup is opt-in and
|
||||
bounded; the existing offline search remains local. Host and running-VM
|
||||
lookups found `app-misc/chatgpt-desktop` for `chatgpt desktop`.
|
||||
|
||||
- **Natural install lane parsing (2026-10-02).** `sys install chatgpt desktop
|
||||
through nix` now separates the Nix lane from the descriptive application
|
||||
name. If an AI plan fails, `sys` searches that lane and suggests a verified
|
||||
|
|
|
|||
|
|
@ -3,6 +3,18 @@
|
|||
Rolling note between work sessions. **Update this file before you finish**, then
|
||||
commit it. It should be short and current; use `git log` for detail.
|
||||
|
||||
- **External overlay leads (2026-10-02):** GPO lists
|
||||
`app-misc/chatgpt-desktop` in Gentoo community overlays, but `sys search`
|
||||
previously queried only Portage repositories enabled locally. The new
|
||||
`sys search chatgpt --legacy --overlays --portage` keeps those local results
|
||||
and adds a bounded read-only GPO lookup with clearly unverified leads. A
|
||||
host lookup returned the ChatGPT desktop ebuild. The updated static binary
|
||||
is installed at `~/.local/bin/sys` in the running playground; a VM query
|
||||
displayed the external lead beneath an empty configured-Portage result.
|
||||
No overlay is enabled by this search, and the ebuild is not treated as an
|
||||
install candidate until a reviewed overlay is configured and Portage
|
||||
resolves it. Host checks, release and static builds passed.
|
||||
|
||||
- **`sys` natural install error (2026-10-02):** The phrase `sys install chatgpt
|
||||
desktop through nix` used to pass the entire phrase as a package name and
|
||||
print invalid legacy examples. The parser now extracts the requested lane;
|
||||
|
|
|
|||
|
|
@ -48,6 +48,17 @@ disposable VM; `serial-driver.py recovery` now automates the same check.
|
|||
|
||||
## Matrix
|
||||
|
||||
External overlay discovery (2026-10-02): local Portage search now labels its
|
||||
scope as configured repositories. `sys search <term> --legacy --overlays`
|
||||
adds a bounded, opt-in read-only lookup of GPO and labels its results as
|
||||
unverified external leads. No repository is enabled or package installed by
|
||||
search. Parser, offline failure, CLI guard, and live GPO lookup checks pass;
|
||||
the live query found `app-misc/chatgpt-desktop` for `chatgpt`. The updated
|
||||
static binary was installed as the running VM user's `~/.local/bin/sys` and
|
||||
`sys search chatgpt desktop --legacy --overlays --portage` displayed the
|
||||
external lead after an empty configured-Portage result. No overlay or package
|
||||
was installed.
|
||||
|
||||
Natural install lane parsing (2026-10-02): `sys` recognizes trailing
|
||||
`through/via/using/with <lane>` as a backend constraint. A model failure for a
|
||||
descriptive package name searches the chosen lane and offers an exact
|
||||
|
|
|
|||
10
docs/SYS.md
10
docs/SYS.md
|
|
@ -333,6 +333,7 @@ separate updater widget.
|
|||
```sh
|
||||
sys search firefox --legacy # every source
|
||||
sys search firefox --legacy --flatpak # one source
|
||||
sys search chatgpt --legacy --overlays --portage # configured Portage + external GPO leads
|
||||
sys list --legacy # installed inventory, all lanes
|
||||
sys list vim --legacy --portage # inventory, source + text filter
|
||||
sys info firefox --legacy # version/source/status/deps/size
|
||||
|
|
@ -345,6 +346,15 @@ sys deps vim --legacy # forward dependencies
|
|||
sys rdeps vim --legacy # reverse dependencies (equery)
|
||||
```
|
||||
|
||||
Portage searches cover repositories configured on this machine. The optional
|
||||
`--overlays` flag sends the search term to the unofficial GPO overlay index and
|
||||
lists matching ebuild names as **external leads** below the local catalogue.
|
||||
Those leads are not validated install candidates: review and sync a repository
|
||||
first, and accept `~amd64` only for a specific package if needed. The external
|
||||
lookup has an eight-second limit; a network failure leaves local results
|
||||
available. `--overlays` is restricted to deterministic `search --legacy` and
|
||||
cannot be combined with a non-Portage source flag.
|
||||
|
||||
During a terminal mutation, `sys` shows the package manager's output as it
|
||||
arrives: Portage build and merge logs, Flatpak transaction progress, Snap
|
||||
operations, Nix realisation, custom ebuild builds, and AppImage downloads.
|
||||
|
|
|
|||
|
|
@ -155,7 +155,11 @@ fn search_portage(exec: &Exec, root: &Path, query: &str, backend: Backend) -> So
|
|||
entry
|
||||
})
|
||||
.collect();
|
||||
SourceReport::ok(backend, "Gentoo tree (emerge --search)", entries)
|
||||
SourceReport::ok(
|
||||
backend,
|
||||
"configured Portage repositories (emerge --search)",
|
||||
entries,
|
||||
)
|
||||
}
|
||||
Ok(out) => SourceReport::failed(
|
||||
backend,
|
||||
|
|
@ -171,6 +175,119 @@ fn search_portage(exec: &Exec, root: &Path, query: &str, backend: Backend) -> So
|
|||
}
|
||||
}
|
||||
|
||||
/// An external overlay index result is a lead for review, never an install
|
||||
/// candidate. GPO is an unofficial index and the named repository may not be
|
||||
/// enabled, synced, trusted, or compatible with this system.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ExternalOverlayLead {
|
||||
pub atom: String,
|
||||
}
|
||||
|
||||
fn encode_query(query: &str) -> String {
|
||||
let mut encoded = String::new();
|
||||
for byte in query.bytes() {
|
||||
if byte.is_ascii_alphanumeric() || b"-._~".contains(&byte) {
|
||||
encoded.push(char::from(byte));
|
||||
} else {
|
||||
encoded.push_str(&format!("%{byte:02X}"));
|
||||
}
|
||||
}
|
||||
encoded
|
||||
}
|
||||
|
||||
fn valid_gpo_atom(atom: &str) -> bool {
|
||||
let Some((category, package)) = atom.split_once('/') else {
|
||||
return false;
|
||||
};
|
||||
!category.is_empty()
|
||||
&& !package.is_empty()
|
||||
&& !package.contains('/')
|
||||
&& [category, package].iter().all(|part| {
|
||||
part.bytes().all(|byte| {
|
||||
byte.is_ascii_lowercase() || byte.is_ascii_digit() || b"+_.-".contains(&byte)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
fn parse_gpo_search(html: &str, query: &str) -> Vec<ExternalOverlayLead> {
|
||||
let terms: Vec<String> = query
|
||||
.split_whitespace()
|
||||
.map(str::to_ascii_lowercase)
|
||||
.collect();
|
||||
let mut atoms: Vec<String> = html
|
||||
.split("href=\"/")
|
||||
.skip(1)
|
||||
.filter_map(|fragment| fragment.split_once('"').map(|(path, _)| path))
|
||||
.filter(|atom| valid_gpo_atom(atom))
|
||||
.filter(|atom| {
|
||||
terms
|
||||
.iter()
|
||||
.all(|term| atom.to_ascii_lowercase().contains(term))
|
||||
})
|
||||
.map(str::to_string)
|
||||
.collect();
|
||||
let normalized_query = terms.join("-");
|
||||
atoms.sort_by_key(|atom| {
|
||||
let package = atom.rsplit('/').next().unwrap_or(atom);
|
||||
let rank = if package == normalized_query {
|
||||
0
|
||||
} else if package == format!("{normalized_query}-desktop") {
|
||||
1
|
||||
} else if package.starts_with(&format!("{normalized_query}-")) {
|
||||
2
|
||||
} else {
|
||||
3
|
||||
};
|
||||
(rank, atom.len(), atom.clone())
|
||||
});
|
||||
atoms.dedup();
|
||||
atoms.truncate(12);
|
||||
atoms
|
||||
.into_iter()
|
||||
.map(|atom| ExternalOverlayLead { atom })
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Opt-in, read-only discovery of packages indexed on GPO. Results never enter
|
||||
/// the verified local catalogue or the installation candidate set.
|
||||
pub fn search_external_overlays(
|
||||
exec: &Exec,
|
||||
root: &Path,
|
||||
query: &str,
|
||||
) -> std::result::Result<Vec<ExternalOverlayLead>, String> {
|
||||
if !phoebus_core::action::valid_search_query(query) {
|
||||
return Err("invalid overlay search query".into());
|
||||
}
|
||||
let url = format!(
|
||||
"https://gpo.zugaina.org/Search?search={}",
|
||||
encode_query(query)
|
||||
);
|
||||
let out = capture(
|
||||
exec,
|
||||
root,
|
||||
"curl",
|
||||
&[
|
||||
"--fail",
|
||||
"--silent",
|
||||
"--show-error",
|
||||
"--proto",
|
||||
"=https",
|
||||
"--connect-timeout",
|
||||
"3",
|
||||
"--max-time",
|
||||
"8",
|
||||
"--max-filesize",
|
||||
"524288",
|
||||
&url,
|
||||
],
|
||||
)
|
||||
.map_err(|_| "GPO lookup unavailable (curl or network error)".to_string())?;
|
||||
if !out.success {
|
||||
return Err("GPO lookup unavailable (service or network error)".into());
|
||||
}
|
||||
Ok(parse_gpo_search(&out.stdout_lossy(), query))
|
||||
}
|
||||
|
||||
fn parse_flatpak_lines(text: &str) -> Vec<Vec<String>> {
|
||||
text.lines()
|
||||
.filter(|l| !l.trim().is_empty())
|
||||
|
|
@ -1665,6 +1782,40 @@ mod tests {
|
|||
assert_eq!(source.entries[0].version.as_deref(), Some("128.0"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn external_overlay_search_reports_leads_without_install_candidates() {
|
||||
let runner = RecordingRunner::new();
|
||||
runner.set_response(
|
||||
"curl",
|
||||
ok(r#"<a href="/Overlays/gentoo-zh">nav</a>
|
||||
<a href="/app-misc/chatgpt-desktop"><div>app-misc/chatgpt-desktop
|
||||
<div>Desktop application for ChatGPT and Codex</div></div></a>
|
||||
<a href="/app-misc/chatgpt-desktop">duplicate</a>
|
||||
<a href="/app-misc/../../bad">unsafe</a>"#),
|
||||
);
|
||||
let exec = make_exec(runner.clone());
|
||||
let leads = search_external_overlays(&exec, Path::new("/"), "chatgpt desktop")
|
||||
.expect("external search response");
|
||||
assert_eq!(leads.len(), 1);
|
||||
assert_eq!(leads[0].atom, "app-misc/chatgpt-desktop");
|
||||
let calls = runner.recorded();
|
||||
assert_eq!(calls.len(), 1);
|
||||
assert_eq!(calls[0].program, "curl");
|
||||
assert!(calls[0].args.iter().any(|arg| arg == "--max-time"));
|
||||
assert!(calls[0]
|
||||
.args
|
||||
.iter()
|
||||
.any(|arg| arg.ends_with("Search?search=chatgpt%20desktop")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn external_overlay_lookup_fails_closed_when_unavailable() {
|
||||
let runner = RecordingRunner::new();
|
||||
runner.fail("curl", "offline");
|
||||
let exec = make_exec(runner);
|
||||
assert!(search_external_overlays(&exec, Path::new("/"), "chatgpt").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_failed_search_is_not_reported_as_empty() {
|
||||
let runner = RecordingRunner::new();
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@ mod progress;
|
|||
administration that never contacts a model.\n\n\
|
||||
Examples (AI default -> deterministic equivalent):\n \
|
||||
sys search chrome -> sys search chrome --legacy\n \
|
||||
sys search chatgpt --legacy --overlays -> include external overlay leads\n \
|
||||
sys list -> sys list --legacy\n \
|
||||
sys info firefox -> sys info firefox --legacy\n \
|
||||
sys updates -> sys updates --legacy\n \
|
||||
|
|
@ -61,6 +62,12 @@ struct Cli {
|
|||
#[arg(long)]
|
||||
legacy: bool,
|
||||
|
||||
/// For `search --legacy`, also query GPO for external Portage overlay leads.
|
||||
/// Sends the search terms to gpo.zugaina.org; results are not installable
|
||||
/// until a reviewed repository is enabled and synced.
|
||||
#[arg(long)]
|
||||
overlays: bool,
|
||||
|
||||
/// Internal broker invocation by Phoebus OS application layer.
|
||||
#[arg(long)]
|
||||
broker: bool,
|
||||
|
|
@ -248,6 +255,16 @@ fn main() -> ExitCode {
|
|||
|
||||
fn run() -> std::result::Result<(), Box<dyn std::error::Error>> {
|
||||
let cli = Cli::parse();
|
||||
if cli.overlays
|
||||
&& (cli.broker
|
||||
|| cli.assistant_bridge
|
||||
|| cli.action.is_some()
|
||||
|| cli.action_file.is_some()
|
||||
|| cli.execute_plan.is_some()
|
||||
|| cli.propose)
|
||||
{
|
||||
return Err("--overlays applies only to an interactive `sys search --legacy`".into());
|
||||
}
|
||||
|
||||
// Broker stdout is a JSON protocol. The assistant bridge is also a stdio
|
||||
// protocol, so neither entry point gets human progress messages.
|
||||
|
|
@ -371,6 +388,9 @@ fn run() -> std::result::Result<(), Box<dyn std::error::Error>> {
|
|||
|
||||
// 4. Explicit legacy mode (--legacy)
|
||||
if cli.legacy {
|
||||
if cli.overlays {
|
||||
return run_overlay_search(&cli, &exec, &full_phrase, backend);
|
||||
}
|
||||
if let Some((application, source)) = parse_install_text(&full_phrase) {
|
||||
let backend = combine_install_backend(backend, source)?;
|
||||
if let Some(b) = backend {
|
||||
|
|
@ -473,6 +493,10 @@ fn run() -> std::result::Result<(), Box<dyn std::error::Error>> {
|
|||
};
|
||||
}
|
||||
|
||||
if cli.overlays {
|
||||
return Err("--overlays requires `sys search <term> --legacy --overlays`".into());
|
||||
}
|
||||
|
||||
// 5. Default AI mode (natural language)
|
||||
if cli.text.is_empty() {
|
||||
eprintln!("sys: no instruction provided.");
|
||||
|
|
@ -514,6 +538,48 @@ fn run() -> std::result::Result<(), Box<dyn std::error::Error>> {
|
|||
run_ask(&cli, &exec, &plans, &full_phrase, backend)
|
||||
}
|
||||
|
||||
fn run_overlay_search(
|
||||
cli: &Cli,
|
||||
exec: &Exec,
|
||||
instruction: &str,
|
||||
backend: Option<Backend>,
|
||||
) -> std::result::Result<(), Box<dyn std::error::Error>> {
|
||||
let Some(SystemAction::SearchPackages { query, .. }) = parse_natural(instruction) else {
|
||||
return Err("--overlays applies only to `sys search <term> --legacy`".into());
|
||||
};
|
||||
let action = SystemAction::SearchPackages {
|
||||
query: query.clone(),
|
||||
backend,
|
||||
};
|
||||
action.validate()?;
|
||||
if backend.is_some_and(|b| !b.is_portage()) {
|
||||
return Err("--overlays can only be combined with --portage".into());
|
||||
}
|
||||
|
||||
let phase = exec.progress_phase("Searching configured package sources");
|
||||
let local = query::search(exec, &cli.root, &query, backend);
|
||||
phase.complete();
|
||||
println!("{}", local.render());
|
||||
|
||||
let phase = exec.progress_phase("Checking the external Portage overlay index");
|
||||
let external = query::search_external_overlays(exec, &cli.root, &query);
|
||||
phase.complete();
|
||||
match external {
|
||||
Ok(leads) if leads.is_empty() => {
|
||||
println!("External Portage overlays: no matching leads on GPO.")
|
||||
}
|
||||
Ok(leads) => {
|
||||
println!("External Portage overlay leads (not enabled or verified locally):");
|
||||
for lead in leads {
|
||||
println!(" {} — https://gpo.zugaina.org/{}", lead.atom, lead.atom);
|
||||
}
|
||||
println!("Review and sync an overlay before installing any of these packages.");
|
||||
}
|
||||
Err(reason) => println!("External Portage overlays: {reason}."),
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Overlay an explicit source flag onto a deterministic legacy action. Only
|
||||
/// commands whose source can be selected accept a flag; anything else is
|
||||
/// refused, so a flag can never silently change an unrelated command.
|
||||
|
|
|
|||
|
|
@ -430,6 +430,32 @@ fn natural_install_lane_is_not_part_of_the_package_name() {
|
|||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn external_overlay_search_requires_explicit_legacy_search() {
|
||||
let root = tempfile::tempdir().expect("temp root");
|
||||
for args in [
|
||||
vec!["search", "chatgpt", "--overlays"],
|
||||
vec!["install", "chatgpt", "--legacy", "--overlays"],
|
||||
vec!["search", "chatgpt", "--legacy", "--overlays", "--nix"],
|
||||
vec![
|
||||
"--legacy",
|
||||
"--action",
|
||||
r#"{"action":"history"}"#,
|
||||
"--overlays",
|
||||
],
|
||||
] {
|
||||
let out = sys()
|
||||
.arg("--root")
|
||||
.arg(root.path())
|
||||
.args(&args)
|
||||
.output()
|
||||
.expect("run sys");
|
||||
assert!(!out.status.success(), "{args:?}");
|
||||
let stderr = String::from_utf8_lossy(&out.stderr);
|
||||
assert!(stderr.contains("--overlays"), "{args:?}: {stderr}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_explicit_backend_never_falls_back_to_another_lane() {
|
||||
if effective_uid_is_root() {
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue