homehub/DESIGN.md

# homehub — lean home-drive sync across 3–4 machines via Synology

Status: **design, no code yet**. Open decisions in §13.

## 0. Ground truth this design was built against

| Fact | Value | Consequence |
|---|---|---|
| Hub | `nas.lue.dk`, DSM, Linux 4.4.302+ x86_64, `/bin/sh` | rsync-over-ssh, no daemon, no Syncthing package |
| Hub target | `/volume1/homes/bl` (4.9 TB free) | **cannot be the sync root — see next row** |
| Hub root | `/volume1/homes/bl` already holds **178 G of unrelated legacy data** (`Documents`, `Photos`, `OneDrive`, `CloudStation`, `home_backup`, `x1c_backup`, `wg-bo.conf`, `certs`, `@eaDir`) | sync root is a dedicated subtree `$HUB_BASE/homehub`; `check_hub_stray()` refuses to sync into a root that is not ours |
| NAS rsync | **3.1.2 (protocol 31)** | **no `--ignore-older`** → use `--update` both ways |
| Client rsync | 3.5.1 | protocol downgrade fine; do not use ≥3.2-only flags |
| Hub perms | siblings `admin, cl, fk, ida, …` on same share | `--no-perms --no-owner --no-group --chmod=Du=rwx,Dgo=,Fu=rw,Fgo=` |
| Laptop offline | weeks | no `--delete`; NAT-friendly health check (hub-side markers only) |
| Client ssh | `id_rsa` has a passphrase; `SSH_AUTH_SOCK` comes from a running agent | a **passphrase-free dedicated key** is mandatory (a user unit has no agent) and rsync gets the same identity via `-e` |
| NixOS | not started (`/nix` absent) | design must be identical pre/post migration → §8 |

Size reality check: `~` ≈ 250 GB, of which file-sync-worthy personal content is
**~0.5 GB** (`Documents` 71 M, `Pictures` 323 M, `bin` 45 M, `journal` 72 K,
dotfiles, `~/.pi/agent/sessions` 11 M). `~/git` 139 G + `~/src` 21 G are *repos*,
not files, and 7.2 G of `~/src` alone is `build/`+`target/`+`dist/`.

## 1. Four data classes, four mechanisms

| Class | What | Mechanism | Direction |
|---|---|---|---|
| A | personal files (docs, pictures, bin, dotfiles, tool *state*) | rsync via hub, mtime-arbitrated | two-way |
| B | git repos (`~/git/*`, repo-like dirs in `~/src/*`) | **manifest + clone** (`repos.tsv`) | two-way, metadata only |
| C | Chromium bookmark bar | per-host export + **union-by-URL merge** | many-to-one-to-many |
| D | everything else (caches, toolchains, builds, `tmp`, `Downloads`) | **default deny** | never |

Class B is the load-bearing idea. "Copy only the directory and the remote; each
machine has its own state" is not a file-sync requirement — it is a manifest. A
`repos.tsv` line is ~120 bytes where the repo is 4 GB, dirty state and local
branches never leave the machine, and 130 GB of vendored upstream clones
(`esp-idf`, `poky`, `bitbake`, `zephyr-sdk`, `llama.cpp`, `ollama`) become one
line each instead of a 139 GB transfer.

## 2. Hub layout

```
/volume1/homes/bl/homehub/        # dedicated sync root — NOT the Synology home
  .hub/
    meta/
      markers/<host>        # epoch of last SUCCESS (hub clock: no client skew)
      alive/<host>          # epoch of last ATTEMPT (written even on failure)
      alerted/<host>        # epoch of last alert (alert de-duplication)
    log.tsv                 # one line per attempt: epoch, host, rc, note
    tombstones/<host>       # epoch \t path  — paths that host deleted
    gitman/repos.<host>     # class B manifest
    bookmarks/<host>.json   # exported bookmark bar per host
    prev/YYYY-MM/           # overwritten older versions; prev/reaped/ for reaped
  <mirrored home layout>    # Documents/, Pictures/, bin/, .ssh/, .pi/, ...
```

**The sync root must be a dedicated subtree.** `/volume1/homes/bl` is 178 G of
pre-existing, unrelated data. Pointing the tool at it would have (a) merged the
NAS's `Documents/` into `~/Documents` on the very first pull, (b) written our
dotfiles over the NAS's own `.config`, and (c) let the tombstone reaper move
unrelated legacy files into `.hub/prev/` whenever a local delete happened to match
a legacy path. `check_hub_stray()` now refuses to `sync --exec` into a hub root
containing anything that is not ours.

Nothing on the hub is ever edited by hand except `.hub/bookmarks/*` if you must.
`.hub/` is excluded from every pull and every push, so it can never leak into a
home tree or be deleted by one.

## 3. Sync algorithm (one run, on one machine, ~10 min cadence)

```
1  touch  .hub/alive/<host>            # "I am up" — even if the sync then fails
2  PULL  tombstones from all hosts      # apply locally: rm -rf within active sets
3  PULL  rsync -rltDl -z --update       # hub -> local, mtime-arbitrated, additive
4  PUSH  rsync --update --backup        # local -> hub, older hub files parked in .hub/prev/
5  write .hub/markers/<host>            # ONLY on rc=0
```

Ordering note: pull-before-push means a file you edited locally while the hub had
a newer version is preserved by `--update` (skipped) *and* the hub copy is kept in
`.hub/prev/` — you never silently lose the other machine's work.

Command shape (pass 4, local → hub):

```
rsync -rltDl -z --update \
      --no-perms --no-owner --no-group \
      --chmod=Du=rwx,Dgo=,Fu=rw,Fgo= \
      --backup --backup-dir="$HUB/.hub/prev/$(date +%Y-%m)" \
      --filter="merge $STATE/effective.filter" \
      ~/  nas.lue.dk:/volume1/homes/bl/
```

Semantics, stated plainly so it is not a surprise later:
**edits always win; deletions only win if nobody else touched the file.**

## 4. Which files: sets + host membership (default deny)

`sets.d/<name>.include` holds rsync filter rules (`+ /Documents/***`, …).
`hosts/<hostname>` lists which sets that machine participates in.
`effective.filter` is generated per host per run:

```
- /.hub/***
- <§9 denylist rules, emitted first so they beat set includes>
+ /<prefix>/***     # union of this host's sets, in stable order
- *                 # default deny: anything not explicitly included never moves
```

First-match-wins, so the `-` global denylist must be emitted **before** `+` lines
that would otherwise match (e.g. `+ /src/***` vs `- /src/**/build/***`).

Proposed initial sets (tick/adjust in §13):

| Set | Members | Desktops | Laptop |
|---|---|---|---|
| `core` | `.ssh/`, `.gitconfig`, `.bashrc`, `.profile`, `.p10k.zsh`, `bin/`, `Documents/`, `Pictures/`, `journal/`, `.local/state/homehub/`, `.pi/agent/{sessions,extensions,settings.json,models.json,AGENTS.md}`, `.gtk-bookmarks` | yes | yes |
| `repos` | `.hub`-side `gitman/repos.tsv` (Class B metadata only) | yes | yes |
| `work` | non-repo remainder of `~/src` (see §13 D2) | yes | opt-in |
| `bulk` | non-repo leftovers in `rustlings/`, `rustql/` | yes | no |

`~/Downloads` (35 G) and `~/tmp` are out, as decided. `~/.config/**` is **not** in
any set — see §8.

## 5. Deletion policy: tombstones + reaper, never `--delete`

`--delete` is incompatible with a weeks-offline peer: machine A, back from a
3-week trip, would delete from the hub files that another machine created or
edited while it was away, and the offline machine would then pull that deletion.

Instead:

- Push pass 1 is a `--dry-run --delete --itemize-changes` scan; `deleting` lines
  are appended to `.hub/tombstones/<host>` (cost: one extra metadata-only scan).
- Real push is additive. Tombstones are applied locally on every pull (step 2).
- A **reaper** (runs only when a tombstone is older than the grace window, §13)
  moves the hub copy to `.hub/prev/reaped/` and prunes the tombstone line, so a
  file deleted in spring does not resurrect on a laptop that returns in autumn.
- `homehub forget <path>` deletes tombstones when you want a file back for good.

Why not `restic`? `--backup --backup-dir=.hub/prev/YYYY-MM` gives one prior
generation for free on every overwrite, needs no new tool on DSM 7/kernel 4.4,
and 4.9 TB free makes "keep the last version" affordable. Check whether Snapshot
Replication covers the `homes` share — `/volume1/@snapshots` was not readable as
`bl`; if it does, you get coarse history for free and should enable it.

## 6. Class B — git manifest spec

`gitman/inventory` walks `$HOME` over `git/` and `src/`, **stops descending at the
first `.git`** (so a repo's own submodules/modules are not double-counted), and
writes `.hub/gitman/repos.<host>.tsv`:

```
relpath   remote_url   head_branch   head_sha   dirty   unpushed
src/rustql           https://…       main   9f2c1ab   0   3
git/ev_fw            git@…:…/ev_fw   main   41aa0c2   12  0
```

- `remote_url` is `git remote get-url origin`; that plus the path is the only thing
  that crosses machines, exactly as asked.
- `dirty` / `unpushed` are **advisory only** — read by `homehub status`, never
  synced as content. They answer "why does `ev_fw` look different on the laptop"
  without turning rsync into a bad git.
- `homehub git missing` = union(all hosts' tsv) − repos present locally.
  `homehub git clone-missing` prints commands; `--exec` runs them;
  `--lazy` adds `--filter=blob:none` for the multi-GB vendored clones.
- Never file-sync any path that the inventory recognises as a repo. No `.git`
  exclusion weirdness, no `build/` 7.2 GB problem: repo trees are cloned, not rsynced.
- Caveat to accept: unpushed commits do **not** survive disk death (you declined a
  bare mirror on the hub — fine for personal use, just know it is the trade).
- `~/src/zephyrproject` is a `west` workspace: inventory records the top repo and
  its sub-repos get recreated by `west update`, not by us.

## 7. Class C — bookmark bar

Live file: `~/.config/chromium/Profile 5/Bookmarks` (mtime 2026-10-24 — this is the
live profile; `~/.config/google-chrome/Default/Bookmarks` is dead, from 2021).

- **Read** the live file, copy to `.hub/bookmarks/<host>.json` (safe: read-only).
- **Merge** all hosts' files: union of `bookmark_bar` children keyed by URL,
  preserving each host's folder nesting, dedupe identical URL+title. Idempotent,
  so no conflict concept is needed for bookmarks.
- Write the merge back to the live path **only when the browser is not running**
  (Chromium holds the file in memory and re-serialises it constantly; writing while
  it runs is how bookmarks silently vanish). If it is running, defer to next run.
- Keep the last N exports on the hub, so a bad merge is always reversible.
- Boring alternative, if you ever stop caring about "only the bar": Chromium Sync.

## 8. Class D and the NixOS migration (the conflict in your answer 5)

Stop treating `~/.config` as one thing. Split by ownership, not by path:

| Kind | Owner | Where it physically lives | Synced? |
|---|---|---|---|
| **declarative config** | Nix / home-manager (later: a git repo) | nix store, symlinked into `~/.config` | **no** |
| **state** (session history, caches-of-record, per-app data you want everywhere) | you | `~/.local/state/homehub/<name>/` | **yes** |
| **cache / binaries** | tools | `~/.cache`, `~/.local/share/…`, `.pi/agent/install` | **no** |

The sync set is defined **only by paths under the shared state root** plus an explicit
top-level list. Two roots, and the difference between them is not cosmetic:

| Root | Contents | Synced? |
|---|---|---|
| `~/.local/state/shared/<name>/` | **your** app state, symlinked to wherever each app expects it | **yes** |
| `~/.local/state/homehub/` | homehub's own scratch: generated filters, markers, `pull.err`, tombstone cache, `broken.*` | **no** (denied) |

Conflating them is a real bug I hit while testing: with the state root itself in the
`base` set, g9 pushed `effective.filter`, the laptop overwrote it with its own, and
they would have swapped generated filters forever. `~/.local/state/homehub/` is now
in the denylist so it cannot come back by accident.

On Debian you create the symlink by hand (or stow); on NixOS home-manager's `file.` /
`xdg.config."…".source` creates the *same* symlink into the *same* synced path. So
the migration rewrites the symlink layer and touches zero sync rules — that is the
resolution of "I like dotfiles locally, but I also want pi history synced": **the
sync tree owns state, Nix owns config, and neither writes into `~/.config` directly.**

Concrete for pi agent, from this box: sync `sessions/` (11 M, 96 `.jsonl` files —
append-only, so rsync-safe) and `settings.json`, `models.json`, `AGENTS.md`,
`extensions/`, hand-written `skills/*` trees; exclude `install/` (154 M),
`bin/` (9 M), `npm/`, and any `node_modules` under `skills/`. No sqlite/`-wal`
files exist in `~/.pi`, which is why plain rsync is safe here — if a tool ever
stores state in sqlite, it goes in the never-sync list or gets a dump-first hook.

## 9. Never-sync denylist (derived from actual `du`, not guessed)

`tmp/` (11 G) · `Downloads/` (35 G, decided) · `git/` and repo trees (Class B) ·
`.cache/` 12 G · `.steam/` 25 G · `.wine/` 14 G, `.wine_games/` 8.5 G,
`.wine_firefly/` 2.2 G · `zephyr-sdk-0.17.4/` 12 G · `go/` 8.5 G ·
`.espressif/` 6 G · `.npm/` 4.8 G · `.rustup/` 4.5 G · `.cargo/` 982 M ·
`.local/share/Trash/` · `.vscode/` 1.7 G, `.config/{google-chrome,discord,chromium,Code,sublime-text,VSCodium,Atom}` 11 G total (local by
default; only the bookmark file crosses machines, via §7) · `**/{build,target,dist,node_modules,__pycache__,.venv,.pixi,.zig-cache}/` ·
`.git/` (belt-and-braces alongside §6) · `.bash_history-*.tmp` (four 0-byte junk
files from a broken history append — also worth fixing at the source) ·
`.gksu.lock`, `*.tmp`, `*.bak`, `*.backup`.

## 10. "Only tell me when it has been broken for a day"

This is a **freshness** problem, not an error-notification problem, so build a
freshness monitor, not an error notifier.

- Sync units are silent: `homehub-sync.service` prints nothing on success,
  `LogLevelMax=warning`. Failures produce no output and no popup; they simply fail
  to update `markers/<host>`. No popups, no retry storms.
- Health unit: `homehub-fresh.timer`, `OnUnitActiveSec=30m`, runs `homehub fresh`,
  which reads hub markers over one ssh call — **no inbound connectivity needed**,
  so it works for the NATted laptop.
- Alert rule: `alive/<host>` is fresh (machine is on) **and** `markers/<host>` is
  older than **24 h** **and** the last attempt's rc is *not* a network-class rc.
  Network-class rc (5, 10, 11, 12, 30, 35, 255 — unreachable/timeout/connection)
  is the weeks-offline laptop and stays **silent forever**; it only shows in
  `homehub status`. That distinction is what keeps your requirement #2 (offline
  for weeks) true without ever popping a notification.
- De-duplication: after alerting, stamp `alerted/<host>`; re-alert at most weekly
  while still broken, clear on success. So one mail per incident, not 48.
- Transport: `sendmail` → your existing `mail.lue.dk`. Works with no GUI session,
  which `notify-send` does not.

## 11. Files to be created (this repo = the policy; ~200 lines of shell total)

```
~/homehub/                     # small git repo, hub-backed via a bare repo on the NAS
  DESIGN.md  README.md
  bin/homehub                  # subcommands: sync, status, fresh, forget, git, bookmarks, filter, setup, install
  lib/merge_bookmarks.py       # class C union-merge
  config                       # per-machine: hub path, key, thresholds, bookmark source
  rules/denylist               # §9 — DEFAULT DENY, emitted first
  rules/sets.d/base.list       # every machine (laptop included)
  rules/sets.d/docs.list  media.list  misc.list
  rules/hosts/g9               # -> sets: base docs media misc
  rules/hosts/_laptop-template # -> sets: base docs
  systemd/homehub-sync.{service,timer}
  systemd/homehub-fresh.{service,timer}
```

Unit details that matter: `OnUnitActiveSec=10min RandomizedDelaySec=3min
Persistent=true` (catches up after suspend/off), `After=network-online.target
Conflicts=sleep.target`, `LogLevelMax=warning` + zero stdout on success, and a
**passphrase-less dedicated ed25519 deploy key** (`~/.ssh/homehub_ed25519`,
`IdentitiesOnly yes`) rather than `id_rsa` — a user unit has no `SSH_AUTH_SOCK`
(yours is a socket at `~/.ssh/agent/…` from a running agent), so depending on the
agent makes the background sync fail in exactly the way you would notice 26 hours
later. Optionally restrict the hub key with `from=` and a forced
`command="rsync --server…"`.

## 12. Rollout order

1. Hub: `mkdir -p .hub/{meta/{markers,alive,alerted},tombstones,bookmarks,prev,gitman}`, deploy key.
2. Write rules + `homehub sync --dry` on `g9`; inspect the file list by hand (this is the only real review step).
3. First real `core` sync from `g9`; verify `700/600` perms on the hub and that `.hub/` is absent locally.
4. `gitman/inventory` from `g9`; on machine #2 run `homehub git clone-missing --lazy`.
5. Machine #2 file sync, then watch convergence both ways.
6. Bookmark export+merge.
7. **Health monitor last** — it is only trustworthy once the sync semantics above have survived a week.
8. NixOS migration later: re-point the symlink layer only (§8); rules unchanged.

## 13. Decisions taken (was: open decisions)

| # | Decision | Outcome |
|---|---|---|
| D1 | Secrets in the sync set | **yes** — `.ssh/` incl. private keys, and `auth.json` if you add it. The hub now holds every key you own: it must stay LAN-only, and the hub copy is forced `700/600`. |
| D2 | `~/src` | **manifest only.** No file sync of repo trees, so the 7.2 G of `build/`+`target/`+`dist/` and every `.git` ambiguity disappeared. |
| D3 | Reaper grace | **30 days.** |
| D4 | Laptop membership | **`base` + `docs`.** (`core` was renamed `base`; `Documents`/`Pictures` were split into `docs`/`media` so the question was even well-posed — my original set list had `Documents` inside `core` *and* asked whether to include it.) |
| D5 | Alert transport | no local MTA yet → auto-detect `msmtp`/`sendmail`, else spool to `~/.local/state/homehub/alerts.out`. msmtp config for `mail.lue.dk:587` is in README. |

### Found while implementing (each one changed the design)

1. **`/volume1/homes/bl` is not empty** — 178 G of unrelated legacy data. Sync root
   is now the dedicated subtree `$HUB_BASE/homehub`, plus a stray-content guard (§2).
2. **A live GitLab PAT sat inside a repo remote URL** (`https://user:glpat-…@gitlab.com/…`,
   3 repos). The manifest would have published it to the NAS. Remotes are now scrubbed
   of userinfo and the row is flagged `stripped`, which `git clone` reports.
   *Separately: rotate that token, it has been sitting in `.git/config` on every
   machine that cloned those repos.*
3. **`id_rsa` is passphrase-protected** and `SSH_AUTH_SOCK` comes from a live agent, so
   a systemd user unit cannot authenticate. Fixed by a dedicated passphrase-free
   ed25519 key — and, more importantly, by splitting *auth failure* (loud, after 24 h)
   from *network failure* (silent forever) in `hub_probe()`. Before that split, a dead
   key produced no marker, no heartbeat, and therefore no alert, ever.
4. **Synology writes `@eaDir` into every directory** — now denied, or a pull litters
   an `@eaDir` next to every synced directory.
5. **`~/debian` is a dpkg work area** (150 M, 3598 files = 86 % of the first dry-run
   push list). Denied as build output, like `~/tmp` and `~/Downloads`.
6. **rsync was not being told which key to use** (no `-e`), so it would have
   authenticated as a different identity from the rest of the tool.

### Still open

* Rotate the GitLab PAT mentioned above; consider `id_rsa` (2016, 4096-bit RSA) too.
* Reconcile the legacy `home_backup` / `x1c_backup` trees on the NAS — they look like
  earlier attempts at exactly this problem.
* Whether `~/bin` should sync at all: it is your scripts *plus* vendored binaries
  (`protoc` 9 M, five gRPC plugins), 45 M of mostly machine-local bytes.
* `~/public_html` symlinks into `~/src`, `~/git` and `/tmp`; they will dangle on a
  machine that has not cloned the target.