From 75a7f74576a182396073ddd938dcaf8e78cf8c42 Mon Sep 17 00:00:00 2001 From: Bo Lorentsen Date: Sat, 24 Oct 2026 16:41:30 +0200 Subject: [PATCH] Add .rules: invariants and gotchas for future development Most entries are a data-loss bug that was found in testing and must not come back: filter ordering, harvest-before-pull, seen-gated deletion attribution, edits-beat-deletes, POSIX sh on DSM, and the never-2>/dev/null-an-rsync rule. Symlinked as AGENTS.md so a future agent session loads it automatically. --- .rules | 99 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ AGENTS.md | 1 + README.md | 1 + 3 files changed, 101 insertions(+) create mode 100644 .rules create mode 120000 AGENTS.md diff --git a/.rules b/.rules new file mode 100644 index 0000000..32d4c3c --- /dev/null +++ b/.rules @@ -0,0 +1,99 @@ +# homehub — rules for anyone (including future me) changing this code + +Invariants and gotchas, most of them paid for with a data-loss bug found in testing. +Rationale for design choices: `DESIGN.md`. Operations: `README.md`. This file is only +the "do not break this" list. + +## Hard invariants + +1. **The sync root is a dedicated subtree, never a home directory.** `HUB_PATH` must + point at something like `/volume1/homes/bl/homehub`. `/volume1/homes/bl` already + holds 178 G of unrelated data (`Documents`, `Photos`, `OneDrive`, `home_backup`, + `wg-bo.conf`); syncing from it would merge that into `~/` on the first pull and let + a tombstone reap unrelated files by name collision. `check_hub_stray()` enforces + this — do not weaken it, and keep it running *after* `gen_filter` (it compares + against the generated include list and must not block when that list is missing). +2. **Default deny, and filter order is load-bearing.** Emitted order is: + `- /.hub/***` → denylist → tombstone excludes → set includes (+ ancestors) → `- *`. + rsync is first-match-wins, so anything that belongs in the first group and lands + later simply never applies. Verify with `homehub filter | less` after any rule change. +3. **Deletions are harvested BEFORE the pull.** The pull re-downloads anything the hub + still has; harvesting afterwards means deletions never propagate at all. +4. **Never propagate a deletion this machine never saw.** Attribution comes from + `$STATE/seen` (paths actually pushed/pulled). Without it a newly added machine + harvests every hub file as its own deletion and tombstones the whole hub on join. + The failure mode must stay "fails to delete", never "deletes wrongly". +5. **Edits beat deletes.** A hub copy whose mtime is newer than our last successful + sync is somebody's edit, not our deletion (`EDITED-ELSEWHERE` in harvest); a local + file touched after the tombstone epoch is kept (`apply_tombstones`). +6. **Repos are never file-synced.** No git repo path may appear in a set — class B + (`homehub git inventory`) publishes path + remote only. Adding a repo to a set + resurrects the 7.2 G of `build/`+`target/` and every `.git` ambiguity. +7. **`~/.config` is denied wholesale.** Shared state lives in `~/.local/state/shared/` + and is symlinked to by apps; homehub's own scratch (`~/.local/state/homehub`) is + denied too — syncing it made machines overwrite each other's generated filters. +8. **Scrub credentials before publishing anything derived from git config.** Repo + remotes carried a live GitLab PAT (`https://user:glpat-…@host/…`); `git_row` strips + userinfo and flags `stripped`. Same rule for any new manifest/export. +9. **Never `2>/dev/null` an rsync/ssh invocation here.** A transport error looks + exactly like "nothing to do" and silently disables deletion propagation. Capture + stderr to `$STATE/*.err` and report the rc. + +## Portability traps + +10. **NAS rsync is 3.1.2 (protocol 31); client is 3.5.1.** No `--ignore-older` + (needs ≥3.2.3 both ends). Arbitration is `--update` in both directions only. +11. **Hub scripts run under POSIX `sh` on DSM.** No bashisms, no arrays, and each + script must end with `exit 0` — a trailing `[ -f "$f" ] || continue` in an empty + directory makes the whole call report failure. Use `hub_script "args" <<'EOS'`; + a heredoc cannot share stdin with a pipe, so pass data via a spool file on the hub. +12. **rsync needs `-e "$RSH"`.** It must authenticate with the same deploy key as the + rest of the tool; otherwise it silently uses a different identity/agent. +13. **Paths contain spaces and non-ASCII** (`Profile 5`, `陈田花园写真`). Quote + everything, never `awk` a path out of a column, and note that `--list-only` + ignores `--out-format` (that is why `seen` is built from itemized transfer logs). +14. **Never `-L`.** Symlinks travel as symlinks (`-l`); some point into `~/git`/`~/src` + and must not be dereferenced into the hub. + +## Alert semantics (§10 of DESIGN.md — do not blur these) + +15. Network-class rc (5/10/11/12/30/35/255) ⇒ **silent, indefinitely**: that is a laptop + being off the LAN for three weeks. Auth/config failure ⇒ **loud** after + `FRESH_MAX_AGE`, via `$STATE/broken.*` markers readable without the hub. A machine + with a dead key produces no marker and no heartbeat, so without the local markers it + would never alert about anything, ever. +16. Sync units stay silent on success (`HH_QUIET=1`). Alerts are deduped by + `.hub/meta/alerted/` and `$STATE/alerted-local.`. + +## Testing before touching sync/filter/deletion code + +17. Re-run the two-fake-home scenario against a scratch hub root. It is the regression + suite for the semantics that destroy data: + ```sh + HUB=/volume1/homes/bl/hht-test + printf 'base\ndocs\n' > rules/hosts/mach-m; cp rules/hosts/mach-m rules/hosts/mach-n + # two HOMEs under /tmp, each needs .ssh/known_hosts copied in + HOME=/tmp/x/a HH_HOST=mach-m HUB_PATH=$HUB HH_KEY=/dev/null bin/homehub sync --exec + ``` + Assert: new machine receives and creates **no** tombstones; delete propagates and + does not resurrect; a later delete loses to a peer's edit; `GRACE_DAYS=0` reaps into + `.hub/prev/reaped/`; `forget ` lets the path travel again. +18. Test hooks exist so nothing has to touch real data: `HH_HOST`, `HOME`, `HUB_PATH`, + `GRACE_DAYS`/`FRESH_MAX_AGE`/`TOMB_LIMIT` (env-overridable), `HH_KEY=/dev/null` + (fall back to the agent), `HH_FORCE=1`, `sync` being dry unless `--exec`. +19. Clean up afterwards: `rm rules/hosts/mach-*`, `ssh nas.lue.dk rm -rf `, + `rm -rf /tmp/`. + +## Not done yet / likely next + +20. Bookmark write-back stops short of overwriting the live file; a v2 should merge into + the live file only when the browser is closed, with a timestamped backup. +21. Untested: 3+ machine convergence, suspend/resume with `Persistent=true`, mtime + conflict resolution with actually skewed clocks, msmtp delivery. +22. Ideas: `homehub doctor` (perms on hub, NTP synced, key installed, hub quota), + an occasional `rsync -mc` checksum audit, `loginctl enable-linger` guidance for + headless boxes, and the NixOS migration only needs the symlink layer re-pointed. +23. The Gitea repo is currently **public** (anonymous `upload-pack` returns 200). + Keep secrets out of it and make it private; `config` already exposes NAS paths and + mail addresses there. Also still outstanding: rotate the GitLab PAT, which sits in + `~/.gitconfig`'s `insteadOf` rewrite and therefore in every clone's `.git/config`. diff --git a/AGENTS.md b/AGENTS.md new file mode 120000 index 0000000..8a63b64 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1 @@ +.rules \ No newline at end of file diff --git a/README.md b/README.md index 30644eb..af2ee02 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ is only the operational half. lib/merge_bookmarks.py class C union-merge (jq + python3 only) systemd/homehub-*.{service,timer} DESIGN.md rationale for everything below + .rules -> AGENTS.md invariants + gotchas for future changes (auto-loaded by agents) ``` ## Install (per machine)