phoebus_os/docs/BASH-MIGRATION.md
Phoebus fc64793aec docs: update references after retiring the Bash installer
Point AGENTS.md, README.md and the docs at the Rust-only toolchain and the
moved VM-BOOT-VERIFICATION.md, and drop the Bash installer from DEVELOPMENT.md.
2026-09-20 04:00:19 -07:00

21 KiB
Raw Blame History

Bash migration — persistent execution prompt and evidence ledger

This file is both the prompt and the progress record for removing the Bash installer from Phoebus OS. It is authoritative: read it at the start of every session that touches the migration, and update it before you stop.

Do not treat this as a design document. Do the work, prove it, record it.


0. How to use this file

  1. Read AGENTS.md, then this file, then docs/ROADMAP.md and docs/HANDOFF.md.
  2. Run make context and git log --oneline -10.
  3. Find the lowest-numbered phase whose exit gate is not yet VERIFIED.
  4. Execute only that phase's tasks. Do not start a later phase.
  5. After each task, run its Verification command and paste the observed result into the Evidence column. Then commit.
  6. Before you stop, update docs/HANDOFF.md and the Session log at the bottom of this file.

An agent that skips the verification step has not done the task. There is no partial credit and no "looks correct".


1. Mission

The production installer is the Rust workspace in installer-rs/. It has already reproduced the proven encrypted UEFI VM boot, so the Bash tree in installer/ is now a reference/fallback that must be retired.

Goal: zero Bash in the shipping installer and its test/verification path, with every deletion backed by a passing automated check or recorded VM evidence.

Out of scope unless Phase 5 is explicitly requested: developer tooling under scripts/, .githooks/ and the Makefile. Those are host tooling, not the product, and may remain Bash.


2. Hard rules (the evidence gate)

These rules override any instruction to move fast.

  1. No task advances to VERIFIED without executed evidence. The Evidence column must contain: the exact command, the observed result (the key output lines, not a summary), and the commit hash it was run at. "Unit tests pass" is not evidence; the command and its output are.
  2. Code inspection, compilation, or reasoning never count as verification. A task whose Verification is a test must have that test actually run.
  3. A test that cannot run in this environment is BLOCKED, never VERIFIED. Name the blocker and the exact command that would unblock it.
  4. VM tasks require the disposable qcow2 VM. Never point DISK at a /dev/* device. If the host cannot run the VM, the task stays BLOCKED.
  5. Do not weaken, delete, or skip a check to make a task pass. If a verification fails, fix the cause or mark the task BLOCKED with the failure recorded.
  6. Do not delete anything from installer/ until Phase 2's exit gate is VERIFIED. Preserve the proven boot path (AGENTS.md rule 9).
  7. One logical change per commit, conventional-commit message, via scripts/agent-commit.sh. Never commit the ISO, build output, transcripts or secrets.
  8. Never send or log secrets. Use the existing redactor and 0600 secret files; the VM evidence must include the NO_SECRET_LEAK check.

Status vocabulary

Status Meaning
TODO Not started.
DOING Started, not finished. A blocker must be written next to it.
BLOCKED Cannot proceed. The blocker and the unblock command are recorded.
VERIFIED The Verification command was run and its raw result is in Evidence.

There is deliberately no DONE. VERIFIED is the only terminal state.


3. Preconditions

Record these once under Phase 0 before any migration work.

  • make doctor reports the host prerequisites.
  • make test passes (Bash suite + cargo test --workspace).
  • cd installer-rs && cargo fmt --all -- --check and cargo clippy --workspace --all-targets are clean.
  • The reference ISO path and the VM harness variables are known (see installer/tests/vm/README.md).

4. Phase 0 — Preserve and instrument

Tasks

  • Tag the current tree as the Bash reference: git tag -a v0.1.0-bash-reference -m "Proven Bash installer reference".
  • Confirm installer-rs/crates/phoebus-installer/tests/parity.rs passes today, so the Rust plan is known to match the Bash plan before anything is removed.
  • Write a CLI-coverage test (Rust) that asserts every mode in installer/bin/ai-installer:53-73 maps to a Rust CLI path. It should fail today for --bootstrap-opencode.
  • Copy installer/config/opencode.manifest into the Rust crate's resources (e.g. installer-rs/crates/phoebus-installer/resources/) and reference it from the new coverage test.

Verification

git tag --list 'v0.1.0-bash-reference'
cd installer-rs && cargo test --workspace --test parity
cargo test --workspace cli_coverage

Evidence

Task Command Result Commit
tag
parity baseline
CLI coverage test (expected fail on opencode)
manifest relocated

Exit gate: the reference tag exists, the parity test passes, and the new coverage test fails only on --bootstrap-opencode.


5. Phase 1 — Close the product gaps

Tasks

  • Add phoebus-executor/src/opencode.rs implementing the behavior of installer/lib/opencode.sh: architecture detection, manifest parsing, fetch through Exec, SHA-256 verification, fail-closed on a missing or placeholder (^0+$) checksum, extraction, symlink, and the config writer. No sh -c; every command through Exec.
  • Unit-test it with a fixture manifest and the recording/never runner: (a) good checksum extracts, (b) placeholder checksum is refused, (c) mismatch is refused and the download is discarded, (d) unsupported arch is refused.
  • Add --bootstrap-opencode to installer-rs/crates/phoebus-installer/src/main.rs so the CLI-coverage test from Phase 0 passes.
  • Port any remaining Bash-only CLI surface (--backend, --ui if still meaningful) or record why it is intentionally dropped.
  • Confirm the sys binary covers the Bash ai.sh action set, or record the gap as a separate task.

Verification

cd installer-rs && cargo test --workspace
cargo test --workspace cli_coverage
cargo clippy --workspace --all-targets

Plus a manual smoke test of the new flag against a local fixture with a known SHA-256:

phoebus-installer --bootstrap-opencode   # with the manifest pointed at the fixture

Evidence

Task Command Result Commit
opencode executor + tests
CLI flag + coverage green
dropped-surface rationale
sys coverage vs ai.sh

Exit gate: cargo test --workspace passes and cli_coverage is green.


6. Phase 2 — Move the VM harness to the Rust binary (the critical gate)

The harness currently installs with Bash (installer/tests/vm/install.sh:41). Until it installs with the static Rust binary, deleting Bash would regress the only proven boot path.

Tasks

  • Build the static binary: RUSTFLAGS="-C target-feature=+crt-static" cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu.
  • Rewrite the guest-side script to fetch and run the static phoebus-installer binary instead of the Bash installer, keeping the secret-file handling and the NO_SECRET_LEAK scan.
  • Point installer/tests/vm/README.md and the serial driver's serve instructions at the Rust binary.
  • Run the full encrypted UEFI install with the static binary, then boot the disk with the ISO detached and unlock LUKS.
  • Record the serial log, boot timing, systemctl is-system-running, and the failed-unit count in docs/VM-BOOT-VERIFICATION.md.
  • Update docs/STATUS.md so the "Rust VM boot parity" row reflects the static-binary harness.

Verification

# Build:
cd installer-rs && RUSTFLAGS="-C target-feature=+crt-static" \
  cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu

# Install in the disposable VM (install mode), then boot it detached:
python3 installer/tests/vm/serial-driver.py install
python3 installer/tests/vm/serial-driver.py boot

The boot evidence must show: LUKS unlock, login, networking, systemctl is-system-running = running, 0 failed units, and NO_SECRET_LEAK.

Evidence

Task Command Result Commit
static build
guest script uses Rust binary
VM install (serial log path)
VM detached boot
docs updated

Exit gate: the detached boot evidence above exists and is recorded. Only then may Phase 3 begin.


7. Phase 3 — Retire the tests

Tasks

  • Map each file in installer/tests/*.sh to existing Rust coverage. For each, either cite the Rust test that covers the same behavior or add one.
  • Replace installer-rs/crates/phoebus-installer/tests/parity.rs with golden-file tests that pin the Rust plan output; then delete parity.rs.
  • Remove the Bash suite and shellcheck step from .forgejo/workflows/ci.yml.
  • Remove the Bash-suite invocation from .githooks/pre-commit (keep the secret scan and image-size checks).
  • Remove test-bash from the Makefile and make test run the Rust suite.

Verification

cd installer-rs && cargo test --workspace
make test
git grep -n "run_tests.sh\|ai-installer\|installer/lib" -- . ':!docs' ':!installer'

The git grep must return nothing outside docs/ and the not-yet-deleted installer/ tree.

Evidence

Task Command Result Commit
Bash-test coverage mapping
golden plan tests
parity.rs removed
CI + hook updated

Exit gate: make test is green and git grep shows no code reference to the Bash installer.


8. Phase 4 — Delete the Bash tree

Tasks

  • Move installer/doc/FINDINGS.md and installer/doc/VM-BOOT-VERIFICATION.md under docs/ and fix links.
  • Delete installer/bin/, installer/lib/, installer/tui/, installer/config/, installer/tests/*.sh, and installer/README.md.
  • Update AGENTS.md, README.md, docs/ARCHITECTURE.md, docs/DEVELOPMENT.md, docs/ROADMAP.md and docs/CONTRIBUTING.md to remove Bash-installer instructions and shellcheck gates.
  • Update docs/CHANGELOG.md and docs/STATUS.md.

Verification

git grep -n "installer/bin\|installer/lib\|ai-installer\|run_tests.sh" -- . ':!docs/BASH-MIGRATION.md' ':!.git'
make test

The git grep must return nothing. make test must pass.

Evidence

Task Command Result Commit
docs relocated
Bash tree deleted
references updated
no dangling references

Exit gate: no tracked file references the Bash installer and make test is green.


9. Phase 5 — Developer tooling (optional, only if requested)

scripts/*.sh, .githooks/* and the Makefile recipes are host tooling. Port them to a cargo xtask only if the goal is literally zero Bash in the repository. If done, apply the same evidence gate: the replacement command's output is the evidence, and the old script may only be deleted once the replacement is VERIFIED.


10. Ledger

Phase status is the minimum of its tasks. A phase is VERIFIED only when its exit gate has been run and recorded.

Phase Scope Status Exit-gate evidence
0 Preserve and instrument VERIFIED tag v0.1.0-bash-reference; parity 1 passed; cli_coverage failed only on --bootstrap-opencode (fb8bdfd)
1 Close product gaps (opencode) VERIFIED cargo test --workspace: opencode 7 passed, cli_coverage 2 passed, clippy rc 0 (4071c7d)
2 VM harness on the Rust binary VERIFIED static-binary install INSTALL-RC=0 + detached boot (systemd PID1, running, 0 failed units, LUKS, login)
3 Retire the tests VERIFIED make test green; git grep "run_tests.sh|ai-installer|installer/lib" outside docs/installer returns nothing (875278b)
4 Delete the Bash tree VERIFIED no tracked file references the Bash installer (git grep empty); make test green (TBD4)
5 Developer tooling (optional) TODO

Task ledger

ID Task Status Blocker (if any) Evidence
0.1 Tag v0.1.0-bash-reference VERIFIED git tag --list 'v0.1.0-bash-reference' -> the tag (ec88d65)
0.2 Parity test baseline VERIFIED cargo test -p phoebus-installer --test parity -> 1 passed (fb8bdfd)
0.3 Rust CLI-coverage test VERIFIED before: failed only --bootstrap-opencode (assert at cli_coverage.rs:57); after 1.3: 2 passed (4071c7d)
0.4 Relocate opencode.manifest VERIFIED crates/phoebus-installer/resources/opencode.manifest; relocated_opencode_manifest... ok (fb8bdfd)
1.1 phoebus-executor::opencode VERIFIED crates/phoebus-executor/src/opencode.rs (4071c7d)
1.2 Opencode unit tests VERIFIED cargo test -p phoebus-executor opencode -> 7 passed (4071c7d)
1.3 --bootstrap-opencode flag VERIFIED cli_coverage 2 passed; CLI smoke: OpenCode ready at .../opencode (4071c7d)
1.4 Port/justify remaining CLI surface VERIFIED --backend/--ui recorded as intentionally dropped in tests/cli_coverage.rs (fb8bdfd)
1.5 Confirm sys covers ai.sh VERIFIED Gap recorded (below) (4071c7d)
2.1 Static binary build VERIFIED target/x86_64-unknown-linux-gnu/release/phoebus-installer 3.4M, statically linked (f984b01)
2.2 Guest script uses Rust binary VERIFIED installer/tests/vm/install.sh fetches phoebus-installer (f984b01)
2.3 Harness docs updated VERIFIED installer/tests/vm/README.md, run-vm.sh (f984b01)
2.4 VM install with Rust binary VERIFIED serial log: phoebus-installer 0.1.0, ===INSTALL-RC=0===, NO_SECRET_LEAK, ===POWEROFF=== (commit 54701bd)
2.5 VM detached boot VERIFIED systemd-boot -> LUKS unlock -> login -> running, 0 loaded units listed, enp0s2 10.0.2.15/24 (commit 54701bd)
2.6 Boot evidence recorded VERIFIED docs/VM-BOOT-VERIFICATION.md "Official harness on the static binary" (commit 54701bd)
3.1 Bash-test coverage mapping VERIFIED map below; no Bash test lacks a Rust equivalent (4c59a1f)
3.2 Golden plan tests; remove parity.rs VERIFIED cargo test -p phoebus-installer --test plan_golden -> 3 passed; parity.rs deleted (4c59a1f)
3.3 CI updated VERIFIED .forgejo/workflows/ci.yml -> fmt + clippy + test (875278b)
3.4 Pre-commit hook updated VERIFIED Bash-suite block removed; image-size + secret scan kept (875278b)
3.5 Makefile updated VERIFIED test: test-rust; test-bash removed (875278b)
4.1 Relocate installer/doc VERIFIED FINDINGS.md, VM-BOOT-VERIFICATION.md -> docs/; Bash-only TESTING.md deleted (TBD4)
4.2 Delete Bash tree VERIFIED installer/{bin,lib,tui,config,tests/*.sh,README.md} removed; installer/tests/vm/ kept (TBD4)
4.3 Update all references VERIFIED AGENTS.md, README.md, docs/{ARCHITECTURE,DEVELOPMENT,CONTRIBUTING,ROADMAP,STATUS,GIT-WORKFLOW}.md (TBD4)
4.4 No dangling references VERIFIED git grep "installer/bin|installer/lib|ai-installer|run_tests.sh" -> nothing (TBD4)

Recorded gaps (from task 1.5)

sys is the runtime typed-action broker (enable/disable service, inspect, install, history, updates); the Bash ai.sh was installer-time AI assistance. The overlap is covered: proposal parsing/validation lives in phoebus-ai (fail-closed, deny_unknown_fields) and OpenCode provisioning in phoebus-executor::opencode. Not ported: interactive provider onboarding and 0600 API-credential storage (ai_store_api_key), and the keyword-based offline explainer. These belong to the installer/TUI surface, not sys; record as separate future work rather than silently dropping them.

Bash test → Rust coverage map (task 3.1)

Every installer/tests/test_*.sh maps to committed Rust coverage:

Bash test Behavior Rust coverage
test_plan.sh plan facts, step order, subvolumes, preview phoebus-core::plan::tests::{sequence_matches_proven_order, render_contains_key_facts, destructive_preview_warns_loudly}; phoebus-installer/tests/plan_golden.rs
test_validation.sh hostname/user/tz/disk rules, protected disks, password strength phoebus-core::validate::tests::{hostname_rules, password_strength_levels, ...}; phoebus-core::disk::tests::{optical_live_media_is_protected, loop_and_dm_are_protected, mounted_is_protected}
test_pkgpolicy.sh compile-time lookup, policy/history/binhost decision phoebus-core::policy::tests::{balanced_compiles_cheap_and_binaries_expensive, source_first_always_compiles, binary_first_prefers_binary_but_falls_back, history_overrides_estimate}
test_state.sh config round-trip, secret redaction phoebus-core::config::tests::{legacy_config_parses_flat_keys, load_from_path_detects_legacy_format}; phoebus-core::secret + Redactor::tests::{redactor_scrubs_registered_values, redactor_handles_multiple_and_dedup}
test_ai.sh proposal parse/allowlist/validate; offline translator phoebus-ai::tests::{parses_and_applies_a_valid_proposal, unknown_action_fails_closed, invalid_value_rejects_whole_proposal}; phoebus-core::action::tests::{parses_typed_actions, rejects_shell_action, rejects_unknown_fields_and_bad_values}
test_opencode.sh manifest + checksum policy phoebus-executor::opencode::tests (7 tests)
test_safety.sh destructive gate, dry-run sandbox root phoebus-executor::auth tests; phoebus-executor::exec::tests::dry_run_does_not_execute; phoebus-core::target tests
test_install_dryrun.sh dry-run never writes the host root phoebus-executor::exec::tests::dry_run_does_not_execute; phoebus-installer dry-run path
test_btrfs_order.sh subvolumes created before subvol=@ mount phoebus-executor::storage::tests::create_layout_creates_subvolumes_before_mounting_root
test_regressions.sh cmdline/install.conf/dracut/fstab order, TMPDIR, base pkgs, package.use, log scrub phoebus-executor::{kernel, boot, portage, storage} tests (cmdline_uses_rd_luks_name, install_conf_has_uki_generator_none, base_packages_include_storage_tools); phoebus-core::system::tests::fstab_mounts_all_subvolumes_including_swap; Redactor log-scrub tests
test_syntax.sh bash -n over the Bash tree obsolete once the Bash tree is deleted; cargo fmt/clippy/compile replace it

No Bash test exercises behavior that lacks a Rust equivalent. test_syntax.sh is deleted with the Bash tree (Phase 4).


11. Session protocol

Start: read this file, confirm the ledger is current, pick the lowest incomplete phase, run its Verification before changing anything so you have a baseline.

Stop: run make test; update this ledger's rows and the Session log; update docs/HANDOFF.md, docs/CHANGELOG.md and docs/STATUS.md if behavior or verification changed; commit with scripts/agent-commit.sh; push with scripts/publish.sh.

Never edit a row to VERIFIED in the same commit that changes the code unless that commit also contains the executed evidence.


12. Verification command reference

# Full quality gate
make test
cd installer-rs && cargo fmt --all -- --check
cd installer-rs && cargo clippy --workspace --all-targets
cd installer-rs && cargo test --workspace

# Static binary for the live installer environment
cd installer-rs && RUSTFLAGS="-C target-feature=+crt-static" \
  cargo build --release -p phoebus-installer --target x86_64-unknown-linux-gnu

# VM harness (never point DISK at /dev/*)
python3 installer/tests/vm/serial-driver.py install
python3 installer/tests/vm/serial-driver.py boot

# Reference check (should shrink to zero by Phase 4)
git grep -n "installer/bin\|installer/lib\|ai-installer\|run_tests.sh"

13. Session log

Append one entry per session, newest last. Keep it short; git log has the detail.

  • YYYY-MM-DD — (agent/session) phase worked, tasks moved, verification run, blocker, commit hash.
  • 2026-09-20 — Phases 0–3. Tagged v0.1.0-bash-reference; baseline cleanup (ec88d65); CLI-coverage gate (fb8bdfd); OpenCode executor + 7 tests and --bootstrap-opencode (4071c7d); harness moved to the static binary (f984b01); full VM install INSTALL-RC=0 + detached boot (systemd PID1, running, 0 failed units, LUKS, login) recorded in docs/VM-BOOT-VERIFICATION.md; golden plan tests replace parity.rs (4c59a1f); CI/hook/Makefile/scripts now gate on Rust only and git grep has no dangling Bash reference (875278b). No blockers. Phase 4 next.