phoebus_os/docs/DEVELOPMENT.md
Phoebus 1d37fbc1b8 ci(release): fail closed and bind gates to the candidate artifact
Make a passing CI/release result mean the checks actually ran.

- .forgejo/workflows/ci.yml is fail-closed: it provisions the pinned
  toolchain or fails, then runs one 'make ci' (fmt-check, clippy, workspace
  tests, VM-harness safety tests, release build, static installer build).
- Pin stable Rust 1.93.1 in installer-rs/rust-toolchain.toml; run cargo gates
  with --locked for reproducible dependencies.
- phoebus-release gates now records passed/failed/blocked/skipped/not_run
  (only passed is a pass), executes real host gate commands, and imports the
  existing VM-harness results.json instead of a competing harness. Gates with
  no automation are always blocked.
- Bind gate reports and harness evidence to a candidate name+SHA-256
  (--candidate; PHOEBUS_VM_ARTIFACT[_SHA256]); stale or unbound evidence is
  failed, never passed.
- publish refuses unless the report is candidate-bound and all-passed, the
  manifest verifies the artifact and a detached signature verifies; atomic
  publish preserves and restores the previous release on failure.
- vmsafety.Results records the verified artifact digest; 3 new Python tests.
- 20 new Rust regression tests; ADR 0012 and docs updated.

Verified locally: make ci (364 Rust tests, 19 harness tests, release and
static builds) exit 0; static binary is static-pie linked. CI runner, VM and
model gates remain unexecuted here.
2026-09-22 07:34:27 -07:00

290 lines
11 KiB
Markdown

# Development
## Host requirements
Ubuntu/Debian host names (the development machine is Ubuntu 26.04):
```sh
sudo apt install qemu-system-x86 qemu-utils ovmf-generic seabios ipxe-qemu \
xorriso mtools dosfstools squashfs-tools cpio zstd python3-pexpect
# Rust quality gates (names may be versioned; check `apt search rustfmt`)
sudo apt install rust-clippy rustfmt
# or, if cargo is rustup-managed:
# rustup component add rustfmt clippy
```
Optional: `expect`, `cryptsetup`, `btrfs-progs`, `parted`, `kpartx` for offline
inspection of VM images. `7z`/`p7zip` is useful for reading the reference ISO.
## Reference media
The Gentoo LiveGUI ISO is a **reference artifact**, not source. It is ignored by
`.gitignore`. Keep it outside git; point the VM harness at it by path.
## Rust installer
The Bash prototype has been retired (see `docs/BASH-MIGRATION.md`); the Rust
workspace is the only installer.
```sh
cd installer-rs
cargo fmt --all -- --check
cargo clippy --workspace --all-targets
cargo test --workspace
cargo build --release -p phoebus-installer
# Static binary for the live environment:
RUSTFLAGS="-C target-feature=+crt-static" \
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
```
Useful CLI invocations:
```sh
target/release/phoebus-installer --config FILE --plan
target/release/phoebus-installer --config FILE --dry-run --install
target/release/phoebus-installer --config FILE --install --yes \
--luks-passphrase-file /secure/luks.pw --user-password-file /secure/user.pw
```
## Disposable VM testing
Never point the harness at a physical disk. The target is always a qcow2 file.
`installer/tests/vm/serial-driver.py` drives QEMU over the serial console:
```sh
cd /path/to/serve
cp /path/to/phoebus_os/installer-rs/target/x86_64-unknown-linux-gnu/release/phoebus-installer .
cp /path/to/phoebus_os/installer/tests/vm/install.sh .
python3 -m http.server 8000 --bind 127.0.0.1
QEMU_BIN=qemu-system-x86_64 \
OVMF_CODE=/usr/share/OVMF/OVMF_CODE_4M.fd \
OVMF_VARS=/usr/share/OVMF/OVMF_VARS_4M.fd \
DISK=/tmp/phoebus/target.qcow2 \
ISO=/path/to/livegui-*.iso \
KERNEL=/tmp/live/boot/gentoo INITRD=/tmp/live/boot/gentoo.igz \
INSTALL_URL=http://10.0.2.2:8000/install.sh \
python3 installer/tests/vm/serial-driver.py install
# Then boot the disk alone (ISO detached):
DISK=/tmp/phoebus/target.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/serial-driver.py boot
```
`installer/tests/vm/run-vm.sh` launches an interactive VM instead.
### Debian-Live appliance harness
The appliance ISO (`installer/live/build.sh`) boots its own GRUB, so it has its
own driver. It is the fastest path to a full install on hosts where the Gentoo
LiveGUI ISO is unavailable:
```sh
cd installer-rs
RUSTFLAGS="-C target-feature=+crt-static" \
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
cd ..
BASE_ISO=/path/to/debian-live-standard.iso installer/live/build.sh
# Serve the stage3, config and 0600 secrets (host 10.0.2.2:8000):
mkdir -p /path/serve && cd /path/serve
cp /path/to/stage3-amd64-systemd-*.tar.xz stage3.tar.xz && sha256sum stage3.tar.xz
cp /path/to/installer/tests/vm/appliance-guest.sh .
cp /path/to/installer/tests/vm/appliance-test.conf.example test.conf # set STAGE3_SHA256
printf 'phoebus-luks-test' > luks.pw && chmod 600 luks.pw
printf 'phoebus-user-test' > user.pw && chmod 600 user.pw
python3 -m http.server 8000 --bind 127.0.0.1
qemu-img create -f qcow2 /path/target.qcow2 24G
cp /usr/share/OVMF/OVMF_VARS_4M.fd /path/OVMF_VARS.fd
OVMF_CODE=/usr/share/OVMF/OVMF_CODE_4M.fd \
OVMF_VARS=/path/OVMF_VARS.fd \
DISK=/path/target.qcow2 \
ISO=/path/to/installer/live/phoebus-installer-amd64.iso \
python3 installer/tests/vm/appliance-driver.py install
# Detached boot (no ISO):
DISK=/path/target.qcow2 OVMF_CODE=... OVMF_VARS=... \
python3 installer/tests/vm/appliance-driver.py boot
```
`INSTALL_TIMEOUT` and `CHECKS_TIMEOUT` bound the long steps. A full base install
still fetches and merges the Gentoo stage3 and base packages, so it takes tens
of minutes.
### Verifying the graphical stack
The serial-only VM has no logind seat, so wlroots' DRM backend cannot start
there. For headless verification use:
```sh
export WLR_BACKENDS=headless WLR_RENDERER=pixman WLR_RENDERER_ALLOW_SOFTWARE=1
dwl & # creates wayland-0
grim /tmp/shot.png # proves the compositor renders
```
On a real machine the installer writes `/usr/local/bin/phoebus-session`
(dwl + waybar + mako + swaybg + swayidle + the polkit agent) and the helpers
`phoebus-menu` (wofi drun launcher / power menu), `phoebus-lock` (swaylock) and
`phoebus-idle` (swayidle).
### Physical VT / real-seat check (needs a person)
The serial harness has no logind seat, so this must be done on real hardware or
a seat-providing VM. From a text VT (e.g. `Ctrl+Alt+F3`), log in and run:
```sh
/usr/local/bin/phoebus-session
```
Check, in order:
1. `dwl` starts and the screen shows the compositor (no `Failed to start a DRM
session`).
2. `waybar` and `mako` appear only after the compositor is up (they must not
log a failed Wayland connection).
3. Open a terminal: `kitty`. Run an X11 app to exercise XWayland (e.g.
`xterm` or `env -u WAYLAND_DISPLAY kitty` if no X app is installed).
4. Run `phoebus-menu` and confirm the application launcher (wofi) appears; run
`phoebus-menu --power` and confirm lock/suspend/log out/reboot/shut down are
offered.
5. Check Waybar shows network and audio controls; click the power module.
6. Run `phoebus-lock` and confirm the session locks (and unlocks).
7. Trigger a privilege prompt (e.g. `pkexec true` or a polkit-gnome dialog) and
confirm the authentication agent appears.
8. Log out (`Mod+Shift+e` for dwl). Confirm `waybar`, `mako`, `swaybg`,
`swayidle` and the polkit agent are gone
(`pgrep -a 'waybar|mako|swaybg|swayidle|polkit-gnome'` prints nothing) and
that the session returns to the VT login.
9. Reboot and confirm the same result.
Record the result (date, kernel, GPU, pass/fail per step) in
`docs/VM-BOOT-VERIFICATION.md`. A headless wlroots session is not physical-seat
verification.
### Mango + DankMaterialShell (default)
Mango + DankMaterialShell is the default desktop: omitting `desktop.profile`
selects `mango`. The installer writes GURU's `repos.conf`, installs
`dev-vcs/git`, syncs the overlay with `emaint sync -r guru`, and installs Mango,
DMS, Quickshell and scenefx with **scoped** `~amd64` keywords. It also installs
and enables `greetd` (with `tuigreet`) for authenticated login; the default
session is Mango and `phoebus-dwl.desktop` remains selectable.
To choose another profile (including `none` or the dwl fallback), set
`DESKTOP_PROFILE='dwl'` (or `[desktop] profile = "dwl"`); the login manager
still lets you pick between the installed sessions.
```sh
/usr/local/bin/phoebus-session-mango # Mango + DMS (default)
/usr/local/bin/phoebus-session # dwl fallback
```
To inspect the resolved effect of the transition before installing, use
`emerge --pretend gui-wm/mangowm gui-apps/dankmaterialshell` and review the
`~amd64` list; add only reviewed atoms to
`/etc/portage/package.accept_keywords/phoebus`. Never run a global
`~amd64` switch.
**Acceptance is a graphical VM with a real seat.** Headless compositor tests do
not qualify. The gates are listed in `docs/VM-BOOT-VERIFICATION.md`; the default
was an explicit owner decision and no desktop stability is claimed until they
pass.
### Building the live desktop ISO
```sh
cd installer-rs
RUSTFLAGS="-C target-feature=+crt-static" \
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
cd ..
installer/live/build.sh
```
This builds a rootless Debian testing rootfs with Mango (source build) +
DankMaterialShell (pinned release), live autologin, the "Install Phoebus"
launcher and a serial/VT recovery console, then splices it into the base
Debian-Live ISO. See `installer/live/README.md` for the full description,
including the world-traversable `WORKDIR` requirement of the unprivileged user
namespace.
## Inspecting the reference ISO without booting
```sh
7z l livegui-*.iso
7z x livegui-*.iso image.squashfs boot/gentoo boot/gentoo.igz
unsquashfs -l image.squashfs # file listing
unsquashfs -d /tmp/root image.squashfs etc/portage/make.conf
```
## Make targets
`make help` lists the common tasks. `make context` prints a session briefing
(state, roadmap next steps, handoff), `make doctor` checks the host
prerequisites, and `make test`, `make check`, `make build`, `make static`,
`make hooks`, `make commit MSG="..."`, `make publish`, `make backup`,
`make forgejo-init`, `make public` and `make release TAG=...` wrap the rest.
## Continuous integration
`.forgejo/workflows/ci.yml` runs the same checks as `make ci`:
```sh
make ci # fmt-check, clippy, test-rust, test-vm, build, static
```
The workflow is **fail-closed**: it provisions the pinned toolchain
(`installer-rs/rust-toolchain.toml`) or fails with a clear message, so a missing
Cargo/rustfmt/clippy is never reported as a pass. Dependencies are checked
against the committed `Cargo.lock` (`--locked`).
It runs only once a Forgejo Actions runner is registered for the repository; see
Forgejo's documentation for `act_runner`. Until then, run `make ci` before
pushing. A runner that uses different labels needs its `runs-on` value updated.
## Release verification
`phoebus-release` writes an evidence-bound gate report and refuses to publish
without one (ADR 0012):
```sh
# Host gates run here; VM/seat/model gates are imported from existing harness
# results and are `blocked` without evidence.
make release-gates
installer-rs/target/release/phoebus-release gates --execute \
--candidate /path/phoebus-installer-amd64.iso \
--harness-results /var/tmp/phoebus-vm
# Publication requires: a candidate-bound all-passed report, a matching
# manifest, and a verified detached signature.
installer-rs/target/release/phoebus-release publish \
/path/phoebus-installer-amd64.iso /srv/releases/latest.iso \
--gate-report gate-report.json --manifest /path/phoebus-release.json \
--signature /path/phoebus-installer-amd64.iso.minisig
```
Gate outcomes are `passed`, `failed`, `blocked`, `skipped` and `not_run`; only
`passed` is a pass. A report for one artifact digest cannot authorize another
(`--candidate` binds the report; the harness records the digest it verified from
`PHOEBUS_VM_ARTIFACT[_SHA256]`). `no-model-recovery`, `updates-and-rollback` and
`desktop-session` have no automated harness yet, so they stay explicitly
`blocked` until one exists.
## Backups
The only remotes are self-hosted, so keep off-box copies:
```sh
make backup # writes ~/backups/phoebus/phoebus-<stamp>.bundle
```
Restore with `git clone /path/to/phoebus-<stamp>.bundle phoebus_os`.
## Quality gates before commit
1. `make ci` (the complete gate set the CI workflow runs)
2. Update `docs/CHANGELOG.md` and `docs/STATUS.md` when behavior changes.
3. Commit (see `docs/GIT-WORKFLOW.md`).