Make a passing CI/release result mean the checks actually ran. - .forgejo/workflows/ci.yml is fail-closed: it provisions the pinned toolchain or fails, then runs one 'make ci' (fmt-check, clippy, workspace tests, VM-harness safety tests, release build, static installer build). - Pin stable Rust 1.93.1 in installer-rs/rust-toolchain.toml; run cargo gates with --locked for reproducible dependencies. - phoebus-release gates now records passed/failed/blocked/skipped/not_run (only passed is a pass), executes real host gate commands, and imports the existing VM-harness results.json instead of a competing harness. Gates with no automation are always blocked. - Bind gate reports and harness evidence to a candidate name+SHA-256 (--candidate; PHOEBUS_VM_ARTIFACT[_SHA256]); stale or unbound evidence is failed, never passed. - publish refuses unless the report is candidate-bound and all-passed, the manifest verifies the artifact and a detached signature verifies; atomic publish preserves and restores the previous release on failure. - vmsafety.Results records the verified artifact digest; 3 new Python tests. - 20 new Rust regression tests; ADR 0012 and docs updated. Verified locally: make ci (364 Rust tests, 19 harness tests, release and static builds) exit 0; static binary is static-pie linked. CI runner, VM and model gates remain unexecuted here.
4.2 KiB
4.2 KiB
ADR 0012: Release verification evidence and publication authorization
- Status: Accepted
- Date: 2026-09-22
Context
Two failure modes made a passing result meaningless:
- CI skipped required checks.
.forgejo/workflows/ci.ymlwrapped every Rust step inif command -v cargo; then ...; else echo "skipping"; fi. A runner without Cargo reported green while running nothing. - Release gates did not execute and were not bound to an artifact.
phoebus-release gates --executeran no command (there were no host gates, and every non-host gate was recordednot_run).publishaccepted any report whose results were allpassed, without checking that the report described the artifact being published, that the artifact matched its manifest, or that a signature had been verified. A stale report for an old build could therefore authorize a different one.
The project already has a disposable VM harness (installer/tests/vm/*.py)
that writes structured results.json. A second, competing harness is not
wanted.
Decision
- CI is fail-closed. The workflow provisions the pinned toolchain or fails
with a clear message; it never treats a missing prerequisite as a pass. The
whole required set is one command,
make ci, so CI and local development run the same checks (fmt, clippy, workspace tests, VM-harness safety tests, release build, static installer build). The toolchain is pinned ininstaller-rs/rust-toolchain.tomland dependencies use the committedCargo.lock(--locked). - Gate outcomes are distinct. A gate is
passed,failed,blocked,skippedornot_run; onlypassedis a pass.blockedmeans a prerequisite (VM, graphical seat, model, CI runner, or an automated command) is missing;skippedmeans an operator excluded it;not_runmeans execution was not requested. - Gates perform their advertised check. Host gates run their real command
through the command layer and record the exit status. VM/seat/model gates
name the real harness scenario and are resolved from that harness's
results.json(reusing the existing harness); with no evidence they areblocked. Gates whose automation does not exist yet have an empty command and can only ever beblocked. - Evidence is bound to the artifact.
phoebus-release gates --candidate <file>records the candidate's name and SHA-256 in the report; the harness records the digest it verified fromPHOEBUS_VM_ARTIFACT[_SHA256]. Imported evidence whose digest differs from the candidate is reported asfailed("stale evidence"), never passed. - Publication is authorized, not assumed.
phoebus-release publishrefuses unless the report is bound to the exact candidate digest, every gate passed, the candidate matches its manifest entry, and a detached signature verifies. Check failures happen before the destination is touched.atomic_publishkeepslatest.iso.prevand restores it if the final rename fails, so a failed or interrupted publication never removes the working release.
Consequences
- A green CI job means the listed checks actually ran. There is no "skipping" path for a required check.
make release-gateswrites a report that reflects reality. Becauseno-model-recovery,updates-and-rollbackanddesktop-sessionhave no automated harness yet, they are alwaysblocked, so a full release cannot be authorized until those checks are automated (or given reviewed manual evidence). This is deliberate: it is preferable to be unable to publish than to publish on a false pass.- The release report and per-gate logs are diagnostics, not source; they are gitignored, redacted with the environment redactor, and bounded in size.
- The remaining external prerequisite is a registered Forgejo Actions runner; until one exists the workflow is unexecuted and is recorded as such.
Status
Accepted and implemented. The gate engine, harness-evidence import, artifact
binding and publication refusals are covered by unit tests; make ci passes
locally. CI execution and VM/model gates remain unverified here (no runner, no
disposable VM or model in this environment).