The committed harness now installs with the static Rust binary and boots the disk detached: systemd PID1, running, 0 failed units, LUKS2 unlock, login and networking. Also fixes the boot check marker so expect() does not match the shell's echoed command.
428 lines
17 KiB
Markdown
428 lines
17 KiB
Markdown
# Bash migration — persistent execution prompt and evidence ledger
|
||
|
||
This file is both the prompt and the progress record for removing the Bash
|
||
installer from Phoebus OS. It is authoritative: read it at the start of every
|
||
session that touches the migration, and update it before you stop.
|
||
|
||
Do not treat this as a design document. Do the work, prove it, record it.
|
||
|
||
---
|
||
|
||
## 0. How to use this file
|
||
|
||
1. Read `AGENTS.md`, then this file, then `docs/ROADMAP.md` and
|
||
`docs/HANDOFF.md`.
|
||
2. Run `make context` and `git log --oneline -10`.
|
||
3. Find the lowest-numbered phase whose exit gate is not yet `VERIFIED`.
|
||
4. Execute only that phase's tasks. Do not start a later phase.
|
||
5. After each task, run its **Verification** command and paste the observed
|
||
result into the **Evidence** column. Then commit.
|
||
6. Before you stop, update `docs/HANDOFF.md` and the **Session log** at the
|
||
bottom of this file.
|
||
|
||
An agent that skips the verification step has not done the task. There is no
|
||
partial credit and no "looks correct".
|
||
|
||
---
|
||
|
||
## 1. Mission
|
||
|
||
The production installer is the Rust workspace in `installer-rs/`. It has
|
||
already reproduced the proven encrypted UEFI VM boot, so the Bash tree in
|
||
`installer/` is now a reference/fallback that must be retired.
|
||
|
||
Goal: **zero Bash in the shipping installer and its test/verification path**,
|
||
with every deletion backed by a passing automated check or recorded VM
|
||
evidence.
|
||
|
||
Out of scope unless Phase 5 is explicitly requested: developer tooling under
|
||
`scripts/`, `.githooks/` and the `Makefile`. Those are host tooling, not the
|
||
product, and may remain Bash.
|
||
|
||
---
|
||
|
||
## 2. Hard rules (the evidence gate)
|
||
|
||
These rules override any instruction to move fast.
|
||
|
||
1. **No task advances to `VERIFIED` without executed evidence.** The Evidence
|
||
column must contain: the exact command, the observed result (the key output
|
||
lines, not a summary), and the commit hash it was run at. "Unit tests pass"
|
||
is not evidence; the command and its output are.
|
||
2. **Code inspection, compilation, or reasoning never count as verification.**
|
||
A task whose Verification is a test must have that test actually run.
|
||
3. **A test that cannot run in this environment is `BLOCKED`, never
|
||
`VERIFIED`.** Name the blocker and the exact command that would unblock it.
|
||
4. **VM tasks require the disposable qcow2 VM.** Never point `DISK` at a
|
||
`/dev/*` device. If the host cannot run the VM, the task stays `BLOCKED`.
|
||
5. **Do not weaken, delete, or skip a check to make a task pass.** If a
|
||
verification fails, fix the cause or mark the task `BLOCKED` with the
|
||
failure recorded.
|
||
6. **Do not delete anything from `installer/` until Phase 2's exit gate is
|
||
`VERIFIED`.** Preserve the proven boot path (AGENTS.md rule 9).
|
||
7. **One logical change per commit**, conventional-commit message, via
|
||
`scripts/agent-commit.sh`. Never commit the ISO, build output, transcripts
|
||
or secrets.
|
||
8. **Never send or log secrets.** Use the existing redactor and 0600 secret
|
||
files; the VM evidence must include the `NO_SECRET_LEAK` check.
|
||
|
||
### Status vocabulary
|
||
|
||
| Status | Meaning |
|
||
| ------ | ------- |
|
||
| `TODO` | Not started. |
|
||
| `DOING` | Started, not finished. A blocker must be written next to it. |
|
||
| `BLOCKED` | Cannot proceed. The blocker and the unblock command are recorded. |
|
||
| `VERIFIED` | The Verification command was run and its raw result is in Evidence. |
|
||
|
||
There is deliberately no `DONE`. `VERIFIED` is the only terminal state.
|
||
|
||
---
|
||
|
||
## 3. Preconditions
|
||
|
||
Record these once under Phase 0 before any migration work.
|
||
|
||
- `make doctor` reports the host prerequisites.
|
||
- `make test` passes (Bash suite + `cargo test --workspace`).
|
||
- `cd installer-rs && cargo fmt --all -- --check` and
|
||
`cargo clippy --workspace --all-targets` are clean.
|
||
- The reference ISO path and the VM harness variables are known (see
|
||
`installer/tests/vm/README.md`).
|
||
|
||
---
|
||
|
||
## 4. Phase 0 — Preserve and instrument
|
||
|
||
**Tasks**
|
||
|
||
- [ ] Tag the current tree as the Bash reference:
|
||
`git tag -a v0.1.0-bash-reference -m "Proven Bash installer reference"`.
|
||
- [ ] Confirm `installer-rs/crates/phoebus-installer/tests/parity.rs` passes
|
||
today, so the Rust plan is known to match the Bash plan before anything
|
||
is removed.
|
||
- [ ] Write a CLI-coverage test (Rust) that asserts every mode in
|
||
`installer/bin/ai-installer:53-73` maps to a Rust CLI path. It should
|
||
fail today for `--bootstrap-opencode`.
|
||
- [ ] Copy `installer/config/opencode.manifest` into the Rust crate's
|
||
resources (e.g. `installer-rs/crates/phoebus-installer/resources/`) and
|
||
reference it from the new coverage test.
|
||
|
||
**Verification**
|
||
|
||
```sh
|
||
git tag --list 'v0.1.0-bash-reference'
|
||
cd installer-rs && cargo test --workspace --test parity
|
||
cargo test --workspace cli_coverage
|
||
```
|
||
|
||
**Evidence**
|
||
|
||
| Task | Command | Result | Commit |
|
||
| ---- | ------- | ------ | ------ |
|
||
| tag | | | |
|
||
| parity baseline | | | |
|
||
| CLI coverage test (expected fail on opencode) | | | |
|
||
| manifest relocated | | | |
|
||
|
||
**Exit gate:** the reference tag exists, the parity test passes, and the new
|
||
coverage test fails *only* on `--bootstrap-opencode`.
|
||
|
||
---
|
||
|
||
## 5. Phase 1 — Close the product gaps
|
||
|
||
**Tasks**
|
||
|
||
- [ ] Add `phoebus-executor/src/opencode.rs` implementing the behavior of
|
||
`installer/lib/opencode.sh`: architecture detection, manifest parsing,
|
||
fetch through `Exec`, SHA-256 verification, fail-closed on a missing or
|
||
placeholder (`^0+$`) checksum, extraction, symlink, and the config
|
||
writer. No `sh -c`; every command through `Exec`.
|
||
- [ ] Unit-test it with a fixture manifest and the recording/never runner:
|
||
(a) good checksum extracts, (b) placeholder checksum is refused,
|
||
(c) mismatch is refused and the download is discarded, (d) unsupported
|
||
arch is refused.
|
||
- [ ] Add `--bootstrap-opencode` to `installer-rs/crates/phoebus-installer/src/main.rs`
|
||
so the CLI-coverage test from Phase 0 passes.
|
||
- [ ] Port any remaining Bash-only CLI surface (`--backend`, `--ui` if still
|
||
meaningful) or record why it is intentionally dropped.
|
||
- [ ] Confirm the `sys` binary covers the Bash `ai.sh` action set, or record
|
||
the gap as a separate task.
|
||
|
||
**Verification**
|
||
|
||
```sh
|
||
cd installer-rs && cargo test --workspace
|
||
cargo test --workspace cli_coverage
|
||
cargo clippy --workspace --all-targets
|
||
```
|
||
|
||
Plus a manual smoke test of the new flag against a local fixture with a known
|
||
SHA-256:
|
||
|
||
```sh
|
||
phoebus-installer --bootstrap-opencode # with the manifest pointed at the fixture
|
||
```
|
||
|
||
**Evidence**
|
||
|
||
| Task | Command | Result | Commit |
|
||
| ---- | ------- | ------ | ------ |
|
||
| opencode executor + tests | | | |
|
||
| CLI flag + coverage green | | | |
|
||
| dropped-surface rationale | | | |
|
||
| sys coverage vs ai.sh | | | |
|
||
|
||
**Exit gate:** `cargo test --workspace` passes and `cli_coverage` is green.
|
||
|
||
---
|
||
|
||
## 6. Phase 2 — Move the VM harness to the Rust binary (the critical gate)
|
||
|
||
The harness currently installs with Bash (`installer/tests/vm/install.sh:41`).
|
||
Until it installs with the static Rust binary, deleting Bash would regress the
|
||
only proven boot path.
|
||
|
||
**Tasks**
|
||
|
||
- [ ] Build the static binary:
|
||
`RUSTFLAGS="-C target-feature=+crt-static" cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu`.
|
||
- [ ] Rewrite the guest-side script to fetch and run the static
|
||
`phoebus-installer` binary instead of the Bash installer, keeping the
|
||
secret-file handling and the `NO_SECRET_LEAK` scan.
|
||
- [ ] Point `installer/tests/vm/README.md` and the serial driver's serve
|
||
instructions at the Rust binary.
|
||
- [ ] Run the full encrypted UEFI install with the static binary, then boot
|
||
the disk with the ISO detached and unlock LUKS.
|
||
- [ ] Record the serial log, boot timing, `systemctl is-system-running`, and
|
||
the failed-unit count in `installer/doc/VM-BOOT-VERIFICATION.md`.
|
||
- [ ] Update `docs/STATUS.md` so the "Rust VM boot parity" row reflects the
|
||
static-binary harness.
|
||
|
||
**Verification**
|
||
|
||
```sh
|
||
# Build:
|
||
cd installer-rs && RUSTFLAGS="-C target-feature=+crt-static" \
|
||
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
|
||
|
||
# Install in the disposable VM (install mode), then boot it detached:
|
||
python3 installer/tests/vm/serial-driver.py install
|
||
python3 installer/tests/vm/serial-driver.py boot
|
||
```
|
||
|
||
The boot evidence must show: LUKS unlock, login, networking,
|
||
`systemctl is-system-running = running`, `0 failed units`, and
|
||
`NO_SECRET_LEAK`.
|
||
|
||
**Evidence**
|
||
|
||
| Task | Command | Result | Commit |
|
||
| ---- | ------- | ------ | ------ |
|
||
| static build | | | |
|
||
| guest script uses Rust binary | | | |
|
||
| VM install (serial log path) | | | |
|
||
| VM detached boot | | | |
|
||
| docs updated | | | |
|
||
|
||
**Exit gate:** the detached boot evidence above exists and is recorded. Only
|
||
then may Phase 3 begin.
|
||
|
||
---
|
||
|
||
## 7. Phase 3 — Retire the tests
|
||
|
||
**Tasks**
|
||
|
||
- [ ] Map each file in `installer/tests/*.sh` to existing Rust coverage. For
|
||
each, either cite the Rust test that covers the same behavior or add one.
|
||
- [ ] Replace `installer-rs/crates/phoebus-installer/tests/parity.rs` with
|
||
golden-file tests that pin the Rust plan output; then delete
|
||
`parity.rs`.
|
||
- [ ] Remove the Bash suite and `shellcheck` step from
|
||
`.forgejo/workflows/ci.yml`.
|
||
- [ ] Remove the Bash-suite invocation from `.githooks/pre-commit` (keep the
|
||
secret scan and image-size checks).
|
||
- [ ] Remove `test-bash` from the `Makefile` and make `test` run the Rust
|
||
suite.
|
||
|
||
**Verification**
|
||
|
||
```sh
|
||
cd installer-rs && cargo test --workspace
|
||
make test
|
||
git grep -n "run_tests.sh\|ai-installer\|installer/lib" -- . ':!docs' ':!installer'
|
||
```
|
||
|
||
The `git grep` must return nothing outside `docs/` and the not-yet-deleted
|
||
`installer/` tree.
|
||
|
||
**Evidence**
|
||
|
||
| Task | Command | Result | Commit |
|
||
| ---- | ------- | ------ | ------ |
|
||
| Bash-test coverage mapping | | | |
|
||
| golden plan tests | | | |
|
||
| parity.rs removed | | | |
|
||
| CI + hook updated | | | |
|
||
|
||
**Exit gate:** `make test` is green and `git grep` shows no code reference to
|
||
the Bash installer.
|
||
|
||
---
|
||
|
||
## 8. Phase 4 — Delete the Bash tree
|
||
|
||
**Tasks**
|
||
|
||
- [ ] Move `installer/doc/FINDINGS.md` and
|
||
`installer/doc/VM-BOOT-VERIFICATION.md` under `docs/` and fix links.
|
||
- [ ] Delete `installer/bin/`, `installer/lib/`, `installer/tui/`,
|
||
`installer/config/`, `installer/tests/*.sh`, and `installer/README.md`.
|
||
- [ ] Update `AGENTS.md`, `README.md`, `docs/ARCHITECTURE.md`,
|
||
`docs/DEVELOPMENT.md`, `docs/ROADMAP.md` and `docs/CONTRIBUTING.md` to
|
||
remove Bash-installer instructions and shellcheck gates.
|
||
- [ ] Update `docs/CHANGELOG.md` and `docs/STATUS.md`.
|
||
|
||
**Verification**
|
||
|
||
```sh
|
||
git grep -n "installer/bin\|installer/lib\|ai-installer\|run_tests.sh" -- . ':!docs/BASH-MIGRATION.md' ':!.git'
|
||
make test
|
||
```
|
||
|
||
The `git grep` must return nothing. `make test` must pass.
|
||
|
||
**Evidence**
|
||
|
||
| Task | Command | Result | Commit |
|
||
| ---- | ------- | ------ | ------ |
|
||
| docs relocated | | | |
|
||
| Bash tree deleted | | | |
|
||
| references updated | | | |
|
||
| no dangling references | | | |
|
||
|
||
**Exit gate:** no tracked file references the Bash installer and `make test` is
|
||
green.
|
||
|
||
---
|
||
|
||
## 9. Phase 5 — Developer tooling (optional, only if requested)
|
||
|
||
`scripts/*.sh`, `.githooks/*` and the `Makefile` recipes are host tooling.
|
||
Port them to a `cargo xtask` only if the goal is literally zero Bash in the
|
||
repository. If done, apply the same evidence gate: the replacement command's
|
||
output is the evidence, and the old script may only be deleted once the
|
||
replacement is `VERIFIED`.
|
||
|
||
---
|
||
|
||
## 10. Ledger
|
||
|
||
Phase status is the minimum of its tasks. A phase is `VERIFIED` only when its
|
||
exit gate has been run and recorded.
|
||
|
||
| Phase | Scope | Status | Exit-gate evidence |
|
||
| ----- | ----- | ------ | ------------------ |
|
||
| 0 | Preserve and instrument | VERIFIED | tag `v0.1.0-bash-reference`; parity 1 passed; `cli_coverage` failed only on `--bootstrap-opencode` (`fb8bdfd`) |
|
||
| 1 | Close product gaps (opencode) | VERIFIED | `cargo test --workspace`: opencode 7 passed, `cli_coverage` 2 passed, clippy rc 0 (`4071c7d`) |
|
||
| 2 | VM harness on the Rust binary | VERIFIED | static-binary install `INSTALL-RC=0` + detached boot (systemd PID1, running, 0 failed units, LUKS, login) |
|
||
| 3 | Retire the tests | TODO | |
|
||
| 4 | Delete the Bash tree | TODO | |
|
||
| 5 | Developer tooling (optional) | TODO | |
|
||
|
||
### Task ledger
|
||
|
||
| ID | Task | Status | Blocker (if any) | Evidence |
|
||
| -- | ---- | ------ | ---------------- | -------- |
|
||
| 0.1 | Tag `v0.1.0-bash-reference` | VERIFIED | | `git tag --list 'v0.1.0-bash-reference'` -> the tag (`ec88d65`) |
|
||
| 0.2 | Parity test baseline | VERIFIED | | `cargo test -p phoebus-installer --test parity` -> 1 passed (`fb8bdfd`) |
|
||
| 0.3 | Rust CLI-coverage test | VERIFIED | | before: failed only `--bootstrap-opencode` (assert at `cli_coverage.rs:57`); after 1.3: 2 passed (`4071c7d`) |
|
||
| 0.4 | Relocate `opencode.manifest` | VERIFIED | | `crates/phoebus-installer/resources/opencode.manifest`; `relocated_opencode_manifest... ok` (`fb8bdfd`) |
|
||
| 1.1 | `phoebus-executor::opencode` | VERIFIED | | `crates/phoebus-executor/src/opencode.rs` (`4071c7d`) |
|
||
| 1.2 | Opencode unit tests | VERIFIED | | `cargo test -p phoebus-executor opencode` -> 7 passed (`4071c7d`) |
|
||
| 1.3 | `--bootstrap-opencode` flag | VERIFIED | | `cli_coverage` 2 passed; CLI smoke: `OpenCode ready at .../opencode` (`4071c7d`) |
|
||
| 1.4 | Port/justify remaining CLI surface | VERIFIED | | `--backend`/`--ui` recorded as intentionally dropped in `tests/cli_coverage.rs` (`fb8bdfd`) |
|
||
| 1.5 | Confirm `sys` covers `ai.sh` | VERIFIED | | Gap recorded (below) (`4071c7d`) |
|
||
| 2.1 | Static binary build | VERIFIED | | `target/x86_64-unknown-linux-gnu/release/phoebus-installer` 3.4M, `statically linked` (`f984b01`) |
|
||
| 2.2 | Guest script uses Rust binary | VERIFIED | | `installer/tests/vm/install.sh` fetches `phoebus-installer` (`f984b01`) |
|
||
| 2.3 | Harness docs updated | VERIFIED | | `installer/tests/vm/README.md`, `run-vm.sh` (`f984b01`) |
|
||
| 2.4 | VM install with Rust binary | VERIFIED | | serial log: `phoebus-installer 0.1.0`, `===INSTALL-RC=0===`, `NO_SECRET_LEAK`, `===POWEROFF===` (commit `TBD2`) |
|
||
| 2.5 | VM detached boot | VERIFIED | | `systemd-boot` -> LUKS unlock -> login -> `running`, `0 loaded units listed`, `enp0s2 10.0.2.15/24` (commit `TBD2`) |
|
||
| 2.6 | Boot evidence recorded | VERIFIED | | `installer/doc/VM-BOOT-VERIFICATION.md` "Official harness on the static binary" (commit `TBD2`) |
|
||
| 3.1 | Bash-test coverage mapping | TODO | | |
|
||
| 3.2 | Golden plan tests; remove `parity.rs` | TODO | | |
|
||
| 3.3 | CI updated | TODO | | |
|
||
| 3.4 | Pre-commit hook updated | TODO | | |
|
||
| 3.5 | `Makefile` updated | TODO | | |
|
||
| 4.1 | Relocate `installer/doc` | TODO | | |
|
||
| 4.2 | Delete Bash tree | TODO | | |
|
||
| 4.3 | Update all references | TODO | | |
|
||
| 4.4 | No dangling references | TODO | | |
|
||
|
||
### Recorded gaps (from task 1.5)
|
||
|
||
`sys` is the runtime typed-action broker (`enable/disable` service, `inspect`,
|
||
`install`, `history`, updates); the Bash `ai.sh` was installer-time AI
|
||
assistance. The overlap is covered: proposal parsing/validation lives in
|
||
`phoebus-ai` (fail-closed, `deny_unknown_fields`) and OpenCode provisioning in
|
||
`phoebus-executor::opencode`. **Not ported:** interactive provider onboarding
|
||
and 0600 API-credential storage (`ai_store_api_key`), and the
|
||
keyword-based offline explainer. These belong to the installer/TUI surface, not
|
||
`sys`; record as separate future work rather than silently dropping them.
|
||
|
||
---
|
||
|
||
## 11. Session protocol
|
||
|
||
**Start:** read this file, confirm the ledger is current, pick the lowest
|
||
incomplete phase, run its Verification before changing anything so you have a
|
||
baseline.
|
||
|
||
**Stop:** run `make test`; update this ledger's rows and the Session log;
|
||
update `docs/HANDOFF.md`, `docs/CHANGELOG.md` and `docs/STATUS.md` if behavior
|
||
or verification changed; commit with `scripts/agent-commit.sh`; push with
|
||
`scripts/publish.sh`.
|
||
|
||
Never edit a row to `VERIFIED` in the same commit that changes the code unless
|
||
that commit also contains the executed evidence.
|
||
|
||
---
|
||
|
||
## 12. Verification command reference
|
||
|
||
```sh
|
||
# Full quality gate
|
||
make test
|
||
cd installer-rs && cargo fmt --all -- --check
|
||
cd installer-rs && cargo clippy --workspace --all-targets
|
||
cd installer-rs && cargo test --workspace
|
||
|
||
# Static binary for the live installer environment
|
||
cd installer-rs && RUSTFLAGS="-C target-feature=+crt-static" \
|
||
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
|
||
|
||
# VM harness (never point DISK at /dev/*)
|
||
python3 installer/tests/vm/serial-driver.py install
|
||
python3 installer/tests/vm/serial-driver.py boot
|
||
|
||
# Reference check (should shrink to zero by Phase 4)
|
||
git grep -n "installer/bin\|installer/lib\|ai-installer\|run_tests.sh"
|
||
```
|
||
|
||
---
|
||
|
||
## 13. Session log
|
||
|
||
Append one entry per session, newest last. Keep it short; `git log` has the
|
||
detail.
|
||
|
||
- `YYYY-MM-DD` — (agent/session) phase worked, tasks moved, verification run,
|
||
blocker, commit hash.
|
||
- `2026-09-20` — Phases 0–2. Tagged `v0.1.0-bash-reference`; baseline cleanup
|
||
(`ec88d65`); CLI-coverage gate (`fb8bdfd`); OpenCode executor + 7 tests and
|
||
`--bootstrap-opencode` (`4071c7d`); harness moved to the static binary
|
||
(`f984b01`); full VM install `INSTALL-RC=0` + detached boot (systemd PID1,
|
||
`running`, 0 failed units, LUKS, login) recorded in
|
||
`installer/doc/VM-BOOT-VERIFICATION.md`. No blockers.
|