phoebus_os/docs/BASH-MIGRATION.md

411 lines
14 KiB
Markdown

# Bash migration — persistent execution prompt and evidence ledger
This file is both the prompt and the progress record for removing the Bash
installer from Phoebus OS. It is authoritative: read it at the start of every
session that touches the migration, and update it before you stop.
Do not treat this as a design document. Do the work, prove it, record it.
---
## 0. How to use this file
1. Read `AGENTS.md`, then this file, then `docs/ROADMAP.md` and
`docs/HANDOFF.md`.
2. Run `make context` and `git log --oneline -10`.
3. Find the lowest-numbered phase whose exit gate is not yet `VERIFIED`.
4. Execute only that phase's tasks. Do not start a later phase.
5. After each task, run its **Verification** command and paste the observed
result into the **Evidence** column. Then commit.
6. Before you stop, update `docs/HANDOFF.md` and the **Session log** at the
bottom of this file.
An agent that skips the verification step has not done the task. There is no
partial credit and no "looks correct".
---
## 1. Mission
The production installer is the Rust workspace in `installer-rs/`. It has
already reproduced the proven encrypted UEFI VM boot, so the Bash tree in
`installer/` is now a reference/fallback that must be retired.
Goal: **zero Bash in the shipping installer and its test/verification path**,
with every deletion backed by a passing automated check or recorded VM
evidence.
Out of scope unless Phase 5 is explicitly requested: developer tooling under
`scripts/`, `.githooks/` and the `Makefile`. Those are host tooling, not the
product, and may remain Bash.
---
## 2. Hard rules (the evidence gate)
These rules override any instruction to move fast.
1. **No task advances to `VERIFIED` without executed evidence.** The Evidence
column must contain: the exact command, the observed result (the key output
lines, not a summary), and the commit hash it was run at. "Unit tests pass"
is not evidence; the command and its output are.
2. **Code inspection, compilation, or reasoning never count as verification.**
A task whose Verification is a test must have that test actually run.
3. **A test that cannot run in this environment is `BLOCKED`, never
`VERIFIED`.** Name the blocker and the exact command that would unblock it.
4. **VM tasks require the disposable qcow2 VM.** Never point `DISK` at a
`/dev/*` device. If the host cannot run the VM, the task stays `BLOCKED`.
5. **Do not weaken, delete, or skip a check to make a task pass.** If a
verification fails, fix the cause or mark the task `BLOCKED` with the
failure recorded.
6. **Do not delete anything from `installer/` until Phase 2's exit gate is
`VERIFIED`.** Preserve the proven boot path (AGENTS.md rule 9).
7. **One logical change per commit**, conventional-commit message, via
`scripts/agent-commit.sh`. Never commit the ISO, build output, transcripts
or secrets.
8. **Never send or log secrets.** Use the existing redactor and 0600 secret
files; the VM evidence must include the `NO_SECRET_LEAK` check.
### Status vocabulary
| Status | Meaning |
| ------ | ------- |
| `TODO` | Not started. |
| `DOING` | Started, not finished. A blocker must be written next to it. |
| `BLOCKED` | Cannot proceed. The blocker and the unblock command are recorded. |
| `VERIFIED` | The Verification command was run and its raw result is in Evidence. |
There is deliberately no `DONE`. `VERIFIED` is the only terminal state.
---
## 3. Preconditions
Record these once under Phase 0 before any migration work.
- `make doctor` reports the host prerequisites.
- `make test` passes (Bash suite + `cargo test --workspace`).
- `cd installer-rs && cargo fmt --all -- --check` and
`cargo clippy --workspace --all-targets` are clean.
- The reference ISO path and the VM harness variables are known (see
`installer/tests/vm/README.md`).
---
## 4. Phase 0 — Preserve and instrument
**Tasks**
- [ ] Tag the current tree as the Bash reference:
`git tag -a v0.1.0-bash-reference -m "Proven Bash installer reference"`.
- [ ] Confirm `installer-rs/crates/phoebus-installer/tests/parity.rs` passes
today, so the Rust plan is known to match the Bash plan before anything
is removed.
- [ ] Write a CLI-coverage test (Rust) that asserts every mode in
`installer/bin/ai-installer:53-73` maps to a Rust CLI path. It should
fail today for `--bootstrap-opencode`.
- [ ] Copy `installer/config/opencode.manifest` into the Rust crate's
resources (e.g. `installer-rs/crates/phoebus-installer/resources/`) and
reference it from the new coverage test.
**Verification**
```sh
git tag --list 'v0.1.0-bash-reference'
cd installer-rs && cargo test --workspace --test parity
cargo test --workspace cli_coverage
```
**Evidence**
| Task | Command | Result | Commit |
| ---- | ------- | ------ | ------ |
| tag | | | |
| parity baseline | | | |
| CLI coverage test (expected fail on opencode) | | | |
| manifest relocated | | | |
**Exit gate:** the reference tag exists, the parity test passes, and the new
coverage test fails *only* on `--bootstrap-opencode`.
---
## 5. Phase 1 — Close the product gaps
**Tasks**
- [ ] Add `phoebus-executor/src/opencode.rs` implementing the behavior of
`installer/lib/opencode.sh`: architecture detection, manifest parsing,
fetch through `Exec`, SHA-256 verification, fail-closed on a missing or
placeholder (`^0+$`) checksum, extraction, symlink, and the config
writer. No `sh -c`; every command through `Exec`.
- [ ] Unit-test it with a fixture manifest and the recording/never runner:
(a) good checksum extracts, (b) placeholder checksum is refused,
(c) mismatch is refused and the download is discarded, (d) unsupported
arch is refused.
- [ ] Add `--bootstrap-opencode` to `installer-rs/crates/phoebus-installer/src/main.rs`
so the CLI-coverage test from Phase 0 passes.
- [ ] Port any remaining Bash-only CLI surface (`--backend`, `--ui` if still
meaningful) or record why it is intentionally dropped.
- [ ] Confirm the `sys` binary covers the Bash `ai.sh` action set, or record
the gap as a separate task.
**Verification**
```sh
cd installer-rs && cargo test --workspace
cargo test --workspace cli_coverage
cargo clippy --workspace --all-targets
```
Plus a manual smoke test of the new flag against a local fixture with a known
SHA-256:
```sh
phoebus-installer --bootstrap-opencode # with the manifest pointed at the fixture
```
**Evidence**
| Task | Command | Result | Commit |
| ---- | ------- | ------ | ------ |
| opencode executor + tests | | | |
| CLI flag + coverage green | | | |
| dropped-surface rationale | | | |
| sys coverage vs ai.sh | | | |
**Exit gate:** `cargo test --workspace` passes and `cli_coverage` is green.
---
## 6. Phase 2 — Move the VM harness to the Rust binary (the critical gate)
The harness currently installs with Bash (`installer/tests/vm/install.sh:41`).
Until it installs with the static Rust binary, deleting Bash would regress the
only proven boot path.
**Tasks**
- [ ] Build the static binary:
`RUSTFLAGS="-C target-feature=+crt-static" cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu`.
- [ ] Rewrite the guest-side script to fetch and run the static
`phoebus-installer` binary instead of the Bash installer, keeping the
secret-file handling and the `NO_SECRET_LEAK` scan.
- [ ] Point `installer/tests/vm/README.md` and the serial driver's serve
instructions at the Rust binary.
- [ ] Run the full encrypted UEFI install with the static binary, then boot
the disk with the ISO detached and unlock LUKS.
- [ ] Record the serial log, boot timing, `systemctl is-system-running`, and
the failed-unit count in `installer/doc/VM-BOOT-VERIFICATION.md`.
- [ ] Update `docs/STATUS.md` so the "Rust VM boot parity" row reflects the
static-binary harness.
**Verification**
```sh
# Build:
cd installer-rs && RUSTFLAGS="-C target-feature=+crt-static" \
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
# Install in the disposable VM (install mode), then boot it detached:
python3 installer/tests/vm/serial-driver.py install
python3 installer/tests/vm/serial-driver.py boot
```
The boot evidence must show: LUKS unlock, login, networking,
`systemctl is-system-running = running`, `0 failed units`, and
`NO_SECRET_LEAK`.
**Evidence**
| Task | Command | Result | Commit |
| ---- | ------- | ------ | ------ |
| static build | | | |
| guest script uses Rust binary | | | |
| VM install (serial log path) | | | |
| VM detached boot | | | |
| docs updated | | | |
**Exit gate:** the detached boot evidence above exists and is recorded. Only
then may Phase 3 begin.
---
## 7. Phase 3 — Retire the tests
**Tasks**
- [ ] Map each file in `installer/tests/*.sh` to existing Rust coverage. For
each, either cite the Rust test that covers the same behavior or add one.
- [ ] Replace `installer-rs/crates/phoebus-installer/tests/parity.rs` with
golden-file tests that pin the Rust plan output; then delete
`parity.rs`.
- [ ] Remove the Bash suite and `shellcheck` step from
`.forgejo/workflows/ci.yml`.
- [ ] Remove the Bash-suite invocation from `.githooks/pre-commit` (keep the
secret scan and image-size checks).
- [ ] Remove `test-bash` from the `Makefile` and make `test` run the Rust
suite.
**Verification**
```sh
cd installer-rs && cargo test --workspace
make test
git grep -n "run_tests.sh\|ai-installer\|installer/lib" -- . ':!docs' ':!installer'
```
The `git grep` must return nothing outside `docs/` and the not-yet-deleted
`installer/` tree.
**Evidence**
| Task | Command | Result | Commit |
| ---- | ------- | ------ | ------ |
| Bash-test coverage mapping | | | |
| golden plan tests | | | |
| parity.rs removed | | | |
| CI + hook updated | | | |
**Exit gate:** `make test` is green and `git grep` shows no code reference to
the Bash installer.
---
## 8. Phase 4 — Delete the Bash tree
**Tasks**
- [ ] Move `installer/doc/FINDINGS.md` and
`installer/doc/VM-BOOT-VERIFICATION.md` under `docs/` and fix links.
- [ ] Delete `installer/bin/`, `installer/lib/`, `installer/tui/`,
`installer/config/`, `installer/tests/*.sh`, and `installer/README.md`.
- [ ] Update `AGENTS.md`, `README.md`, `docs/ARCHITECTURE.md`,
`docs/DEVELOPMENT.md`, `docs/ROADMAP.md` and `docs/CONTRIBUTING.md` to
remove Bash-installer instructions and shellcheck gates.
- [ ] Update `docs/CHANGELOG.md` and `docs/STATUS.md`.
**Verification**
```sh
git grep -n "installer/bin\|installer/lib\|ai-installer\|run_tests.sh" -- . ':!docs/BASH-MIGRATION.md' ':!.git'
make test
```
The `git grep` must return nothing. `make test` must pass.
**Evidence**
| Task | Command | Result | Commit |
| ---- | ------- | ------ | ------ |
| docs relocated | | | |
| Bash tree deleted | | | |
| references updated | | | |
| no dangling references | | | |
**Exit gate:** no tracked file references the Bash installer and `make test` is
green.
---
## 9. Phase 5 — Developer tooling (optional, only if requested)
`scripts/*.sh`, `.githooks/*` and the `Makefile` recipes are host tooling.
Port them to a `cargo xtask` only if the goal is literally zero Bash in the
repository. If done, apply the same evidence gate: the replacement command's
output is the evidence, and the old script may only be deleted once the
replacement is `VERIFIED`.
---
## 10. Ledger
Phase status is the minimum of its tasks. A phase is `VERIFIED` only when its
exit gate has been run and recorded.
| Phase | Scope | Status | Exit-gate evidence |
| ----- | ----- | ------ | ------------------ |
| 0 | Preserve and instrument | TODO | |
| 1 | Close product gaps (opencode) | TODO | |
| 2 | VM harness on the Rust binary | TODO | |
| 3 | Retire the tests | TODO | |
| 4 | Delete the Bash tree | TODO | |
| 5 | Developer tooling (optional) | TODO | |
### Task ledger
| ID | Task | Status | Blocker (if any) | Evidence |
| -- | ---- | ------ | ---------------- | -------- |
| 0.1 | Tag `v0.1.0-bash-reference` | TODO | | |
| 0.2 | Parity test baseline | TODO | | |
| 0.3 | Rust CLI-coverage test | TODO | | |
| 0.4 | Relocate `opencode.manifest` | TODO | | |
| 1.1 | `phoebus-executor::opencode` | TODO | | |
| 1.2 | Opencode unit tests | TODO | | |
| 1.3 | `--bootstrap-opencode` flag | TODO | | |
| 1.4 | Port/justify remaining CLI surface | TODO | | |
| 1.5 | Confirm `sys` covers `ai.sh` | TODO | | |
| 2.1 | Static binary build | TODO | | |
| 2.2 | Guest script uses Rust binary | TODO | | |
| 2.3 | Harness docs updated | TODO | | |
| 2.4 | VM install with Rust binary | TODO | | |
| 2.5 | VM detached boot | TODO | | |
| 2.6 | Boot evidence recorded | TODO | | |
| 3.1 | Bash-test coverage mapping | TODO | | |
| 3.2 | Golden plan tests; remove `parity.rs` | TODO | | |
| 3.3 | CI updated | TODO | | |
| 3.4 | Pre-commit hook updated | TODO | | |
| 3.5 | `Makefile` updated | TODO | | |
| 4.1 | Relocate `installer/doc` | TODO | | |
| 4.2 | Delete Bash tree | TODO | | |
| 4.3 | Update all references | TODO | | |
| 4.4 | No dangling references | TODO | | |
---
## 11. Session protocol
**Start:** read this file, confirm the ledger is current, pick the lowest
incomplete phase, run its Verification before changing anything so you have a
baseline.
**Stop:** run `make test`; update this ledger's rows and the Session log;
update `docs/HANDOFF.md`, `docs/CHANGELOG.md` and `docs/STATUS.md` if behavior
or verification changed; commit with `scripts/agent-commit.sh`; push with
`scripts/publish.sh`.
Never edit a row to `VERIFIED` in the same commit that changes the code unless
that commit also contains the executed evidence.
---
## 12. Verification command reference
```sh
# Full quality gate
make test
cd installer-rs && cargo fmt --all -- --check
cd installer-rs && cargo clippy --workspace --all-targets
cd installer-rs && cargo test --workspace
# Static binary for the live installer environment
cd installer-rs && RUSTFLAGS="-C target-feature=+crt-static" \
cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu
# VM harness (never point DISK at /dev/*)
python3 installer/tests/vm/serial-driver.py install
python3 installer/tests/vm/serial-driver.py boot
# Reference check (should shrink to zero by Phase 4)
git grep -n "installer/bin\|installer/lib\|ai-installer\|run_tests.sh"
```
---
## 13. Session log
Append one entry per session, newest last. Keep it short; `git log` has the
detail.
- `YYYY-MM-DD` — (agent/session) phase worked, tasks moved, verification run,
blocker, commit hash.