Some checks are pending
ci / test (push) Waiting to run
- Mango + DankMaterialShell desktop option (desktop.profile = "mango") through the GURU overlay with scoped ~amd64 keywords, a sealed session entry point and a dwl fallback session (ADR 0009). - Coherent rollback/recovery: coordinated @ + @var snapshots, fstab and BLS root-subvolume rewriting, matching-kernel selection, snapshot recovery entry, retention/space gates and sys undo (ADR 0007). - Installed assistant (sys ask) with an OS-level systemd sandbox boundary and fail-closed typed plans; resolved package effects bound to approval. - Nix build isolation enforced (sandbox, no fallback) plus lifecycle/cleanup. - Desktop helpers (menu/lock/idle), transaction log, release gates with atomic publish and signing helpers, and a Debian-Live appliance VM harness.
101 lines
4.6 KiB
Markdown
101 lines
4.6 KiB
Markdown
# ADR 0007: Coherent rollback and recovery
|
|
|
|
- **Status:** Accepted
|
|
- **Date:** 2026-09-21
|
|
|
|
## Context
|
|
|
|
Btrfs snapshots and transaction history existed, but rollback was deliberately
|
|
unimplemented. The proven layout separates the filesystems that must move
|
|
together:
|
|
|
|
```
|
|
GPT: 1 GiB FAT32 ESP (mounted /efi)
|
|
LUKS2 -> Btrfs: @ / @home /home @var /var @snapshots /.snapshots @swap /swap
|
|
```
|
|
|
|
Three facts make a naive rollback wrong:
|
|
|
|
1. **`/var` is a separate subvolume.** The Portage package database
|
|
(`/var/db/pkg`), Portage state (`/var/lib/portage`), the update state and the
|
|
snapshot/transaction history all live in `@var`. Restoring `@` alone leaves
|
|
the package manager describing packages that do not exist in the restored
|
|
root.
|
|
2. **The kernel command line carries an explicit root subvolume.**
|
|
`rootflags=subvol=@` overrides the Btrfs default subvolume, so changing the
|
|
default alone does not change what boots.
|
|
3. **The kernel and initramfs live on the ESP, which is not snapshotted.**
|
|
A restored older root has older `/lib/modules`; it must boot a kernel whose
|
|
modules match, or it will fail to load drivers.
|
|
|
|
The ESP also holds the BLS entries, so a recovery entry that reuses the live
|
|
root's kernel/options still depends on the root being healthy.
|
|
|
|
## Decision
|
|
|
|
Rollback is a **coordinated** operation over four things, implemented in
|
|
`phoebus-executor::rollback` and planned by `phoebus-core::rollback`:
|
|
|
|
1. **Snapshot set.** A recovery point captures `@` *and* the separately mounted
|
|
`@var` under one timestamp/operation name. `/home` is intentionally excluded
|
|
so rollback never discards user documents. `/nix` is inside `@`, so the Nix
|
|
store rolls back with the OS package state (consistent by construction).
|
|
|
|
2. **Root generation.** Rollback creates a *writable* clone of the chosen
|
|
snapshot (`@restore-<utc>`) and, when present, `@varrestore-<utc>`. The
|
|
running root is preserved by the pre-rollback snapshot; the previous root is
|
|
also offered as a boot entry.
|
|
|
|
3. **Boot arguments, in both places.** The restored root's `/etc/fstab` is
|
|
rewritten to mount `subvol=<new root>` and `subvol=<new var>`, *and* the BLS
|
|
entries are rewritten (`rootflags=subvol=<new root>`). The Btrfs default
|
|
subvolume is set to match, as a fallback for a mount without an argument.
|
|
|
|
4. **Kernel/initramfs pairing.** The restored root's `/lib/modules` versions are
|
|
used to select the newest BLS entry with matching modules. A "Phoebus OS
|
|
(previous)" entry and a "Phoebus Recovery" entry that boots the chosen
|
|
snapshot directly into `rescue.target` are written, so recovery does not
|
|
depend on the live root.
|
|
|
|
### State machine and interruption safety
|
|
|
|
`var/lib/phoebus/rollback-state.toml` records `Idle | Planned | Activated |
|
|
Verified | Failed`. The safety snapshot is taken first; boot entries are
|
|
switched last; a `sync` runs before completion. An interruption therefore leaves
|
|
the system booting a known root, and re-running resumes (an already-staged
|
|
restored root is detected and reused). `sys undo status` promotes `Activated`
|
|
to `Verified` once the restored subvolume is the running root.
|
|
|
|
### Retention and space
|
|
|
|
Retention keeps the newest `DEFAULT_KEEP_SNAPSHOTS` (8) committed snapshots and
|
|
never prunes the newest, an in-flight rollback target, or `Created`/`Failed`
|
|
records. Rollback refuses to start unless at least 1 GiB is free on the target.
|
|
|
|
### Exposure
|
|
|
|
`sys undo` is the only acceptance test for a rollback. `sys undo` restores the
|
|
newest committed snapshot; `sys undo <id>` selects one; `sys undo status`
|
|
reports state. It requires root and a reviewed typed action, like every other
|
|
high-risk mutation.
|
|
|
|
## Consequences
|
|
|
|
- More subvolumes accumulate (`@restore-*`, `@varrestore-*`); retention bounds
|
|
them and the operator can delete them explicitly.
|
|
- A rollback is a reboot-level operation; it changes the *next* boot, not the
|
|
running system. This is deliberate: swapping the mounted root in place is not
|
|
safe.
|
|
- The path is unit-tested for the command sequence, fstab/boot-entry rewriting,
|
|
kernel selection, interruption resume and retention. End-to-end VM
|
|
verification (broken update, rollback, detached boot, package-state check) is
|
|
the acceptance gate recorded in `docs/STATUS.md`.
|
|
|
|
## Alternatives rejected
|
|
|
|
- **Changing only the Btrfs default subvolume.** Rejected: the explicit
|
|
`rootflags=subvol=` wins, so this does not boot the restored root.
|
|
- **Renaming the mounted root subvolume.** Rejected: unsafe while it is the live
|
|
root.
|
|
- **Snapshotting the ESP.** Impossible (vfat, not Btrfs); handled instead by
|
|
selecting a matching kernel entry and keeping a known-good entry.
|