phoebus_os/docs/adr/0012-release-verification-and-publication.md
Phoebus 1d37fbc1b8 ci(release): fail closed and bind gates to the candidate artifact
Make a passing CI/release result mean the checks actually ran.

- .forgejo/workflows/ci.yml is fail-closed: it provisions the pinned
  toolchain or fails, then runs one 'make ci' (fmt-check, clippy, workspace
  tests, VM-harness safety tests, release build, static installer build).
- Pin stable Rust 1.93.1 in installer-rs/rust-toolchain.toml; run cargo gates
  with --locked for reproducible dependencies.
- phoebus-release gates now records passed/failed/blocked/skipped/not_run
  (only passed is a pass), executes real host gate commands, and imports the
  existing VM-harness results.json instead of a competing harness. Gates with
  no automation are always blocked.
- Bind gate reports and harness evidence to a candidate name+SHA-256
  (--candidate; PHOEBUS_VM_ARTIFACT[_SHA256]); stale or unbound evidence is
  failed, never passed.
- publish refuses unless the report is candidate-bound and all-passed, the
  manifest verifies the artifact and a detached signature verifies; atomic
  publish preserves and restores the previous release on failure.
- vmsafety.Results records the verified artifact digest; 3 new Python tests.
- 20 new Rust regression tests; ADR 0012 and docs updated.

Verified locally: make ci (364 Rust tests, 19 harness tests, release and
static builds) exit 0; static binary is static-pie linked. CI runner, VM and
model gates remain unexecuted here.
2026-09-22 07:34:27 -07:00

4.2 KiB

ADR 0012: Release verification evidence and publication authorization

  • Status: Accepted
  • Date: 2026-09-22

Context

Two failure modes made a passing result meaningless:

  1. CI skipped required checks. .forgejo/workflows/ci.yml wrapped every Rust step in if command -v cargo; then ...; else echo "skipping"; fi. A runner without Cargo reported green while running nothing.
  2. Release gates did not execute and were not bound to an artifact. phoebus-release gates --execute ran no command (there were no host gates, and every non-host gate was recorded not_run). publish accepted any report whose results were all passed, without checking that the report described the artifact being published, that the artifact matched its manifest, or that a signature had been verified. A stale report for an old build could therefore authorize a different one.

The project already has a disposable VM harness (installer/tests/vm/*.py) that writes structured results.json. A second, competing harness is not wanted.

Decision

  1. CI is fail-closed. The workflow provisions the pinned toolchain or fails with a clear message; it never treats a missing prerequisite as a pass. The whole required set is one command, make ci, so CI and local development run the same checks (fmt, clippy, workspace tests, VM-harness safety tests, release build, static installer build). The toolchain is pinned in installer-rs/rust-toolchain.toml and dependencies use the committed Cargo.lock (--locked).
  2. Gate outcomes are distinct. A gate is passed, failed, blocked, skipped or not_run; only passed is a pass. blocked means a prerequisite (VM, graphical seat, model, CI runner, or an automated command) is missing; skipped means an operator excluded it; not_run means execution was not requested.
  3. Gates perform their advertised check. Host gates run their real command through the command layer and record the exit status. VM/seat/model gates name the real harness scenario and are resolved from that harness's results.json (reusing the existing harness); with no evidence they are blocked. Gates whose automation does not exist yet have an empty command and can only ever be blocked.
  4. Evidence is bound to the artifact. phoebus-release gates --candidate <file> records the candidate's name and SHA-256 in the report; the harness records the digest it verified from PHOEBUS_VM_ARTIFACT[_SHA256]. Imported evidence whose digest differs from the candidate is reported as failed ("stale evidence"), never passed.
  5. Publication is authorized, not assumed. phoebus-release publish refuses unless the report is bound to the exact candidate digest, every gate passed, the candidate matches its manifest entry, and a detached signature verifies. Check failures happen before the destination is touched. atomic_publish keeps latest.iso.prev and restores it if the final rename fails, so a failed or interrupted publication never removes the working release.

Consequences

  • A green CI job means the listed checks actually ran. There is no "skipping" path for a required check.
  • make release-gates writes a report that reflects reality. Because no-model-recovery, updates-and-rollback and desktop-session have no automated harness yet, they are always blocked, so a full release cannot be authorized until those checks are automated (or given reviewed manual evidence). This is deliberate: it is preferable to be unable to publish than to publish on a false pass.
  • The release report and per-gate logs are diagnostics, not source; they are gitignored, redacted with the environment redactor, and bounded in size.
  • The remaining external prerequisite is a registered Forgejo Actions runner; until one exists the workflow is unexecuted and is recorded as such.

Status

Accepted and implemented. The gate engine, harness-evidence import, artifact binding and publication refusals are covered by unit tests; make ci passes locally. CI execution and VM/model gates remain unverified here (no runner, no disposable VM or model in this environment).