phoebus_os/docs/adr/0012-release-verification-and-publication.md
Phoebus 1d37fbc1b8 ci(release): fail closed and bind gates to the candidate artifact
Make a passing CI/release result mean the checks actually ran.

- .forgejo/workflows/ci.yml is fail-closed: it provisions the pinned
  toolchain or fails, then runs one 'make ci' (fmt-check, clippy, workspace
  tests, VM-harness safety tests, release build, static installer build).
- Pin stable Rust 1.93.1 in installer-rs/rust-toolchain.toml; run cargo gates
  with --locked for reproducible dependencies.
- phoebus-release gates now records passed/failed/blocked/skipped/not_run
  (only passed is a pass), executes real host gate commands, and imports the
  existing VM-harness results.json instead of a competing harness. Gates with
  no automation are always blocked.
- Bind gate reports and harness evidence to a candidate name+SHA-256
  (--candidate; PHOEBUS_VM_ARTIFACT[_SHA256]); stale or unbound evidence is
  failed, never passed.
- publish refuses unless the report is candidate-bound and all-passed, the
  manifest verifies the artifact and a detached signature verifies; atomic
  publish preserves and restores the previous release on failure.
- vmsafety.Results records the verified artifact digest; 3 new Python tests.
- 20 new Rust regression tests; ADR 0012 and docs updated.

Verified locally: make ci (364 Rust tests, 19 harness tests, release and
static builds) exit 0; static binary is static-pie linked. CI runner, VM and
model gates remain unexecuted here.
2026-09-22 07:34:27 -07:00

78 lines
4.2 KiB
Markdown

# ADR 0012: Release verification evidence and publication authorization
- **Status:** Accepted
- **Date:** 2026-09-22
## Context
Two failure modes made a passing result meaningless:
1. **CI skipped required checks.** `.forgejo/workflows/ci.yml` wrapped every
Rust step in `if command -v cargo; then ...; else echo "skipping"; fi`. A
runner without Cargo reported green while running nothing.
2. **Release gates did not execute and were not bound to an artifact.**
`phoebus-release gates --execute` ran no command (there were no host gates,
and every non-host gate was recorded `not_run`). `publish` accepted any
report whose results were all `passed`, without checking that the report
described the artifact being published, that the artifact matched its
manifest, or that a signature had been verified. A stale report for an old
build could therefore authorize a different one.
The project already has a disposable VM harness (`installer/tests/vm/*.py`)
that writes structured `results.json`. A second, competing harness is not
wanted.
## Decision
1. **CI is fail-closed.** The workflow provisions the pinned toolchain or fails
with a clear message; it never treats a missing prerequisite as a pass. The
whole required set is one command, `make ci`, so CI and local development run
the same checks (fmt, clippy, workspace tests, VM-harness safety tests,
release build, static installer build). The toolchain is pinned in
`installer-rs/rust-toolchain.toml` and dependencies use the committed
`Cargo.lock` (`--locked`).
2. **Gate outcomes are distinct.** A gate is `passed`, `failed`, `blocked`,
`skipped` or `not_run`; **only `passed` is a pass**. `blocked` means a
prerequisite (VM, graphical seat, model, CI runner, or an automated command)
is missing; `skipped` means an operator excluded it; `not_run` means
execution was not requested.
3. **Gates perform their advertised check.** Host gates run their real command
through the command layer and record the exit status. VM/seat/model gates
name the real harness scenario and are resolved from that harness's
`results.json` (reusing the existing harness); with no evidence they are
`blocked`. Gates whose automation does not exist yet have an empty command
and can only ever be `blocked`.
4. **Evidence is bound to the artifact.** `phoebus-release gates --candidate
<file>` records the candidate's name and SHA-256 in the report; the harness
records the digest it verified from `PHOEBUS_VM_ARTIFACT[_SHA256]`. Imported
evidence whose digest differs from the candidate is reported as `failed`
("stale evidence"), never passed.
5. **Publication is authorized, not assumed.** `phoebus-release publish`
refuses unless the report is bound to the exact candidate digest, every gate
passed, the candidate matches its manifest entry, and a detached signature
verifies. Check failures happen before the destination is touched.
`atomic_publish` keeps `latest.iso.prev` and restores it if the final rename
fails, so a failed or interrupted publication never removes the working
release.
## Consequences
- A green CI job means the listed checks actually ran. There is no "skipping"
path for a required check.
- `make release-gates` writes a report that reflects reality. Because
`no-model-recovery`, `updates-and-rollback` and `desktop-session` have no
automated harness yet, they are always `blocked`, so a full release cannot be
authorized until those checks are automated (or given reviewed manual
evidence). This is deliberate: it is preferable to be unable to publish than
to publish on a false pass.
- The release report and per-gate logs are diagnostics, not source; they are
gitignored, redacted with the environment redactor, and bounded in size.
- The remaining external prerequisite is a registered Forgejo Actions runner;
until one exists the workflow is unexecuted and is recorded as such.
## Status
Accepted and implemented. The gate engine, harness-evidence import, artifact
binding and publication refusals are covered by unit tests; `make ci` passes
locally. CI execution and VM/model gates remain unverified here (no runner, no
disposable VM or model in this environment).