phoebus_os/docs/adr/0007-coherent-rollback.md
Phoebus 25f7381b4a
Some checks are pending
ci / test (push) Waiting to run
feat(desktop): add Mango + DankMaterialShell profile with rollback, assistant and release foundations
- Mango + DankMaterialShell desktop option (desktop.profile = "mango") through
  the GURU overlay with scoped ~amd64 keywords, a sealed session entry point and
  a dwl fallback session (ADR 0009).
- Coherent rollback/recovery: coordinated @ + @var snapshots, fstab and BLS
  root-subvolume rewriting, matching-kernel selection, snapshot recovery entry,
  retention/space gates and sys undo (ADR 0007).
- Installed assistant (sys ask) with an OS-level systemd sandbox boundary and
  fail-closed typed plans; resolved package effects bound to approval.
- Nix build isolation enforced (sandbox, no fallback) plus lifecycle/cleanup.
- Desktop helpers (menu/lock/idle), transaction log, release gates with atomic
  publish and signing helpers, and a Debian-Live appliance VM harness.
2026-09-21 03:46:52 -07:00

4.6 KiB

ADR 0007: Coherent rollback and recovery

  • Status: Accepted
  • Date: 2026-09-21

Context

Btrfs snapshots and transaction history existed, but rollback was deliberately unimplemented. The proven layout separates the filesystems that must move together:

GPT: 1 GiB FAT32 ESP (mounted /efi)
LUKS2 -> Btrfs: @ /  @home /home  @var /var  @snapshots /.snapshots  @swap /swap

Three facts make a naive rollback wrong:

  1. /var is a separate subvolume. The Portage package database (/var/db/pkg), Portage state (/var/lib/portage), the update state and the snapshot/transaction history all live in @var. Restoring @ alone leaves the package manager describing packages that do not exist in the restored root.
  2. The kernel command line carries an explicit root subvolume. rootflags=subvol=@ overrides the Btrfs default subvolume, so changing the default alone does not change what boots.
  3. The kernel and initramfs live on the ESP, which is not snapshotted. A restored older root has older /lib/modules; it must boot a kernel whose modules match, or it will fail to load drivers.

The ESP also holds the BLS entries, so a recovery entry that reuses the live root's kernel/options still depends on the root being healthy.

Decision

Rollback is a coordinated operation over four things, implemented in phoebus-executor::rollback and planned by phoebus-core::rollback:

  1. Snapshot set. A recovery point captures @ and the separately mounted @var under one timestamp/operation name. /home is intentionally excluded so rollback never discards user documents. /nix is inside @, so the Nix store rolls back with the OS package state (consistent by construction).

  2. Root generation. Rollback creates a writable clone of the chosen snapshot (@restore-<utc>) and, when present, @varrestore-<utc>. The running root is preserved by the pre-rollback snapshot; the previous root is also offered as a boot entry.

  3. Boot arguments, in both places. The restored root's /etc/fstab is rewritten to mount subvol=<new root> and subvol=<new var>, and the BLS entries are rewritten (rootflags=subvol=<new root>). The Btrfs default subvolume is set to match, as a fallback for a mount without an argument.

  4. Kernel/initramfs pairing. The restored root's /lib/modules versions are used to select the newest BLS entry with matching modules. A "Phoebus OS (previous)" entry and a "Phoebus Recovery" entry that boots the chosen snapshot directly into rescue.target are written, so recovery does not depend on the live root.

State machine and interruption safety

var/lib/phoebus/rollback-state.toml records Idle | Planned | Activated | Verified | Failed. The safety snapshot is taken first; boot entries are switched last; a sync runs before completion. An interruption therefore leaves the system booting a known root, and re-running resumes (an already-staged restored root is detected and reused). sys undo status promotes Activated to Verified once the restored subvolume is the running root.

Retention and space

Retention keeps the newest DEFAULT_KEEP_SNAPSHOTS (8) committed snapshots and never prunes the newest, an in-flight rollback target, or Created/Failed records. Rollback refuses to start unless at least 1 GiB is free on the target.

Exposure

sys undo is the only acceptance test for a rollback. sys undo restores the newest committed snapshot; sys undo <id> selects one; sys undo status reports state. It requires root and a reviewed typed action, like every other high-risk mutation.

Consequences

  • More subvolumes accumulate (@restore-*, @varrestore-*); retention bounds them and the operator can delete them explicitly.
  • A rollback is a reboot-level operation; it changes the next boot, not the running system. This is deliberate: swapping the mounted root in place is not safe.
  • The path is unit-tested for the command sequence, fstab/boot-entry rewriting, kernel selection, interruption resume and retention. End-to-end VM verification (broken update, rollback, detached boot, package-state check) is the acceptance gate recorded in docs/STATUS.md.

Alternatives rejected

  • Changing only the Btrfs default subvolume. Rejected: the explicit rootflags=subvol= wins, so this does not boot the restored root.
  • Renaming the mounted root subvolume. Rejected: unsafe while it is the live root.
  • Snapshotting the ESP. Impossible (vfat, not Btrfs); handled instead by selecting a matching kernel entry and keeping a known-good entry.