--- title: Keep data you need across rollbacks on a dataset outside the BE area: zfs impact: HIGH tags: [zfs, datasets, properties, quota, compression, boot-environments] verified: 2026-11-04 --- # Keep data you need across rollbacks on a dataset outside the BE Keep data on its own dataset outside `/ROOT`, with limits and compression set as dataset properties. A generation is the BE `/ROOT/solnix-N`: data in the BE root reverts on a rollback, and data written after `boot` or `switch` cloned the BE stays behind in the old one. **Antipattern** ```sh # do not run these mkdir /srv/db && cp -r dump/* /srv/db # lands in the running BE: reverted by solnix-rebuild rollback pfexec zfs create rpool/ROOT/solnix-3/data # a child of the BE: beadm list -d shows it as part of that BE ``` **The Solnix way** ```sh zfs list -H -o name / # the running BE, e.g. rpool/ROOT/solnix-3 (images default to spool) zfs list -o name,mountpoint,canmount -r rpool # ROOT/solnix-N = generations; nix, home, var/* = shared pfexec zfs create -o mountpoint=/srv/db -o quota=100G rpool/db # /srv/db absent or empty pfexec zfs set compression=on rpool/db # lz4 if the pool has lz4_compress, else lzjb; new writes only zfs get -r -s local -o name,property,value all rpool/db # only values set here (others are default, inherited, temporary, or - for read-only) pfexec zfs inherit compression rpool/db # drop the local value, take the parent's again pfexec zfs set reservation=20G rpool/db # quota and reservation are not inherited; they cover descendants pfexec zfs set refquota=80G rpool/db # refquota/refreservation count only this dataset's own data ``` Shared by every BE: `/nix` (store, Nix database, profiles). On systems `solnix-install` made, also `/home` and, by design (a real install with it is not yet run: see Unverified), `/var/` (adm cores crash cron fm lib log logadm mail saf spool tmp svc/log); images from `lib/make-be.nix` create only the pool, `/ROOT`, the BE and `/nix`, so there `/home` and `/var` are in the BE. Always in the BE: `/etc`, `/var/svc/manifest`, `/var/svc/profile`. Leave the `canmount` values these datasets were created with (`noauto` on `nix` and the `var/` leaves, `off` on the `var` and `var/svc` containers). **The illumos way** (OpenIndiana, OmniOS, SmartOS) ```sh pfexec zfs create -o mountpoint=/export/data rpool/data # outside rpool/ROOT: one copy for every BE pfexec zfs set quota=100G rpool/data zfs get compression,quota,reservation rpool/data ``` Same commands. BEs live under `rpool/ROOT/` and `/export` and homes outside them (OpenIndiana also has a shared `rpool/VARSHARE`); beadm recognises shared datasets when it clones a BE. **The difference.** The ZFS commands are the same as illumos; the layout differs. Solnix also keeps the Nix store out of the BE, and the solnix-install layout puts most of `/var` outside it too, so that a rollback would leave logs, `/var/tmp` and spools alone (that layout is not yet proven on a real install: see Unverified). For ordinary pools and datasets there is no Nix module (pgfarm.nix: "solnix has no filesystems/zpool module yet"), so use `zfs create`; modules that make their own are `solnix.ephemeralPool` (a pool and its datasets on blank instance-store disks) and the builder zone (its zonepath dataset). Do not write files into the BE under a shared dataset that `zfs mount -a` mounts (`/home`, `/export`): it will not mount over a non-empty directory, and install-be then fails the `boot` or `switch`: after `switch` has already reconciled SMF live, and leaving the new, unactivated BE `solnix-N` behind (check `beadm list`). The BE's own `/nix` and `/var/` under the `-O` mounts are expected (the early store set lives there); leave them alone. **Check.** `zfs list -H -o name /srv/db` prints a dataset that does not start with `/ROOT/`. **Unverified** - The `/var` layout is proven on scratch pools; a real install and boot with it is "PLANNED (not yet run)". **Where sources disagree.** design-generations.md §0, illumos-domain.md and product.md pillar 5 say `/nix/store` lives inside the BE; lib/make-be.nix and install-be.sh (the code) use one shared `/nix`. This rule follows the code. The handbook's ZFS chapter lists quota among the properties that are "inherited down the tree"; the ZFS Administration Guide (5.4.2) excepts quotas and reservations, and this rule follows it. **Sources** - [zfs(8)](https://solnix.io/docs/man/zfs.8) (illumos, CDDL), DESCRIPTION; Native Properties (canmount, compression, quota, refquota, reservation, refreservation); zfs create, set, get, inherit, list, mount -O; Examples 1, 4, 6, 7, 11 - [ZFS Administration Guide: Managing ZFS File Systems](https://illumos.org/books/zfs-admin/gavwq.html) (PDL), 5.4.2 Inheriting ZFS Properties; 5.6 ZFS Quotas and Reservations - [Handbook: ZFS storage](https://solnix.io/docs/handbook/zfs#the-model-pools-and-datasets) (site 307b67f), The model: pools and datasets; `zfs` — manage datasets - [docs/design-var-state.md](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/docs/design-var-state.md) (solnix 5a8e5ce), The problem; The layout; Mount order (`zfs mount` refuses a non-empty directory); Who does what (make-be has no `/var`); Proof (PLANNED) - [docs/design-nix-managed-base.md](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/docs/design-nix-managed-base.md) (solnix 5a8e5ce), Decision (one shared `/nix`) - [docs/installer-contract.md](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/docs/installer-contract.md) (solnix 5a8e5ce), row 6 Partition (`/nix` and `/home` outside the BE) - [lib/make-be.nix](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/lib/make-be.nix) (solnix 5a8e5ce), `pool ? "spool"`; `/nix` created canmount=noauto - [modules/system/activation/install-be.sh](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/modules/system/activation/install-be.sh) (solnix 5a8e5ce), steps 4/4a (inittab mounts), 7c (files under a shared mountpoint) - [modules/system/activation/default.nix](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/modules/system/activation/default.nix) (solnix 5a8e5ce), `_rootds=$(zfs list -H -o name /)`; `do_boot` (install-be lays /etc, SMF and /boot into the BE) - [docs/design-generations.md](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/docs/design-generations.md) (solnix 5a8e5ce), §0 (store inside the BE: stale) - [.kiro/steering/product.md](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/.kiro/steering/product.md) (solnix 5a8e5ce), pillar 5 (stale on the store's location) - [modules/config/community/builder-zone.nix](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/modules/config/community/builder-zone.nix) (solnix 5a8e5ce), option `dataset` (the zonepath) - [modules/config/ephemeral-pool.nix](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/modules/config/ephemeral-pool.nix) (solnix 5a8e5ce), `solnix.ephemeralPool` (name, disks, datasets) - [modules/config/pgfarm.nix](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/modules/config/pgfarm.nix) (solnix 5a8e5ce), limitation 2 "THE BUILD POOL IS IMPERATIVE" - [.kiro/steering/illumos-domain.md](https://codeberg.org/solnix/solnix/src/commit/5a8e5ce75a33086d7be4410149847bdf7501b8fc/.kiro/steering/illumos-domain.md) (solnix 5a8e5ce), ZFS root + boot environments (shared datasets outside the BE) - [beadm(8)](https://solnix.io/docs/man/beadm.8) (illumos, CDDL), beadm list -d (subordinate file systems of a BE); Example 16 - [OpenIndiana handbook: Managing Boot Environments](https://docs.openindiana.org/handbook/getting-started/#managing-boot-environments), Features of the beadm utility (shared datasets)