Add .rules: invariants and gotchas for future development
Most entries are a data-loss bug that was found in testing and must not come back: filter ordering, harvest-before-pull, seen-gated deletion attribution, edits-beat-deletes, POSIX sh on DSM, and the never-2>/dev/null-an-rsync rule. Symlinked as AGENTS.md so a future agent session loads it automatically.
This commit is contained in:
1 parent
c445d83006
commit
75a7f74576
3 files changed
+101
No files matched your search
@@ -0,0 +1,99 @@
|
||||
# homehub — rules for anyone (including future me) changing this code
|
||||
|
||||
Invariants and gotchas, most of them paid for with a data-loss bug found in testing.
|
||||
Rationale for design choices: `DESIGN.md`. Operations: `README.md`. This file is only
|
||||
the "do not break this" list.
|
||||
|
||||
## Hard invariants
|
||||
|
||||
1. **The sync root is a dedicated subtree, never a home directory.** `HUB_PATH` must
|
||||
point at something like `/volume1/homes/bl/homehub`. `/volume1/homes/bl` already
|
||||
holds 178 G of unrelated data (`Documents`, `Photos`, `OneDrive`, `home_backup`,
|
||||
`wg-bo.conf`); syncing from it would merge that into `~/` on the first pull and let
|
||||
a tombstone reap unrelated files by name collision. `check_hub_stray()` enforces
|
||||
this — do not weaken it, and keep it running *after* `gen_filter` (it compares
|
||||
against the generated include list and must not block when that list is missing).
|
||||
2. **Default deny, and filter order is load-bearing.** Emitted order is:
|
||||
`- /.hub/***` → denylist → tombstone excludes → set includes (+ ancestors) → `- *`.
|
||||
rsync is first-match-wins, so anything that belongs in the first group and lands
|
||||
later simply never applies. Verify with `homehub filter | less` after any rule change.
|
||||
3. **Deletions are harvested BEFORE the pull.** The pull re-downloads anything the hub
|
||||
still has; harvesting afterwards means deletions never propagate at all.
|
||||
4. **Never propagate a deletion this machine never saw.** Attribution comes from
|
||||
`$STATE/seen` (paths actually pushed/pulled). Without it a newly added machine
|
||||
harvests every hub file as its own deletion and tombstones the whole hub on join.
|
||||
The failure mode must stay "fails to delete", never "deletes wrongly".
|
||||
5. **Edits beat deletes.** A hub copy whose mtime is newer than our last successful
|
||||
sync is somebody's edit, not our deletion (`EDITED-ELSEWHERE` in harvest); a local
|
||||
file touched after the tombstone epoch is kept (`apply_tombstones`).
|
||||
6. **Repos are never file-synced.** No git repo path may appear in a set — class B
|
||||
(`homehub git inventory`) publishes path + remote only. Adding a repo to a set
|
||||
resurrects the 7.2 G of `build/`+`target/` and every `.git` ambiguity.
|
||||
7. **`~/.config` is denied wholesale.** Shared state lives in `~/.local/state/shared/`
|
||||
and is symlinked to by apps; homehub's own scratch (`~/.local/state/homehub`) is
|
||||
denied too — syncing it made machines overwrite each other's generated filters.
|
||||
8. **Scrub credentials before publishing anything derived from git config.** Repo
|
||||
remotes carried a live GitLab PAT (`https://user:glpat-…@host/…`); `git_row` strips
|
||||
userinfo and flags `stripped`. Same rule for any new manifest/export.
|
||||
9. **Never `2>/dev/null` an rsync/ssh invocation here.** A transport error looks
|
||||
exactly like "nothing to do" and silently disables deletion propagation. Capture
|
||||
stderr to `$STATE/*.err` and report the rc.
|
||||
|
||||
## Portability traps
|
||||
|
||||
10. **NAS rsync is 3.1.2 (protocol 31); client is 3.5.1.** No `--ignore-older`
|
||||
(needs ≥3.2.3 both ends). Arbitration is `--update` in both directions only.
|
||||
11. **Hub scripts run under POSIX `sh` on DSM.** No bashisms, no arrays, and each
|
||||
script must end with `exit 0` — a trailing `[ -f "$f" ] || continue` in an empty
|
||||
directory makes the whole call report failure. Use `hub_script "args" <<'EOS'`;
|
||||
a heredoc cannot share stdin with a pipe, so pass data via a spool file on the hub.
|
||||
12. **rsync needs `-e "$RSH"`.** It must authenticate with the same deploy key as the
|
||||
rest of the tool; otherwise it silently uses a different identity/agent.
|
||||
13. **Paths contain spaces and non-ASCII** (`Profile 5`, `陈田花园写真`). Quote
|
||||
everything, never `awk` a path out of a column, and note that `--list-only`
|
||||
ignores `--out-format` (that is why `seen` is built from itemized transfer logs).
|
||||
14. **Never `-L`.** Symlinks travel as symlinks (`-l`); some point into `~/git`/`~/src`
|
||||
and must not be dereferenced into the hub.
|
||||
|
||||
## Alert semantics (§10 of DESIGN.md — do not blur these)
|
||||
|
||||
15. Network-class rc (5/10/11/12/30/35/255) ⇒ **silent, indefinitely**: that is a laptop
|
||||
being off the LAN for three weeks. Auth/config failure ⇒ **loud** after
|
||||
`FRESH_MAX_AGE`, via `$STATE/broken.*` markers readable without the hub. A machine
|
||||
with a dead key produces no marker and no heartbeat, so without the local markers it
|
||||
would never alert about anything, ever.
|
||||
16. Sync units stay silent on success (`HH_QUIET=1`). Alerts are deduped by
|
||||
`.hub/meta/alerted/<host>` and `$STATE/alerted-local.<kind>`.
|
||||
|
||||
## Testing before touching sync/filter/deletion code
|
||||
|
||||
17. Re-run the two-fake-home scenario against a scratch hub root. It is the regression
|
||||
suite for the semantics that destroy data:
|
||||
```sh
|
||||
HUB=/volume1/homes/bl/hht-test
|
||||
printf 'base\ndocs\n' > rules/hosts/mach-m; cp rules/hosts/mach-m rules/hosts/mach-n
|
||||
# two HOMEs under /tmp, each needs .ssh/known_hosts copied in
|
||||
HOME=/tmp/x/a HH_HOST=mach-m HUB_PATH=$HUB HH_KEY=/dev/null bin/homehub sync --exec
|
||||
```
|
||||
Assert: new machine receives and creates **no** tombstones; delete propagates and
|
||||
does not resurrect; a later delete loses to a peer's edit; `GRACE_DAYS=0` reaps into
|
||||
`.hub/prev/reaped/`; `forget <path>` lets the path travel again.
|
||||
18. Test hooks exist so nothing has to touch real data: `HH_HOST`, `HOME`, `HUB_PATH`,
|
||||
`GRACE_DAYS`/`FRESH_MAX_AGE`/`TOMB_LIMIT` (env-overridable), `HH_KEY=/dev/null`
|
||||
(fall back to the agent), `HH_FORCE=1`, `sync` being dry unless `--exec`.
|
||||
19. Clean up afterwards: `rm rules/hosts/mach-*`, `ssh nas.lue.dk rm -rf <scratch root>`,
|
||||
`rm -rf /tmp/<fake homes>`.
|
||||
|
||||
## Not done yet / likely next
|
||||
|
||||
20. Bookmark write-back stops short of overwriting the live file; a v2 should merge into
|
||||
the live file only when the browser is closed, with a timestamped backup.
|
||||
21. Untested: 3+ machine convergence, suspend/resume with `Persistent=true`, mtime
|
||||
conflict resolution with actually skewed clocks, msmtp delivery.
|
||||
22. Ideas: `homehub doctor` (perms on hub, NTP synced, key installed, hub quota),
|
||||
an occasional `rsync -mc` checksum audit, `loginctl enable-linger` guidance for
|
||||
headless boxes, and the NixOS migration only needs the symlink layer re-pointed.
|
||||
23. The Gitea repo is currently **public** (anonymous `upload-pack` returns 200).
|
||||
Keep secrets out of it and make it private; `config` already exposes NAS paths and
|
||||
mail addresses there. Also still outstanding: rotate the GitLab PAT, which sits in
|
||||
`~/.gitconfig`'s `insteadOf` rewrite and therefore in every clone's `.git/config`.
|
||||
@@ -13,6 +13,7 @@ is only the operational half.
|
||||
lib/merge_bookmarks.py class C union-merge (jq + python3 only)
|
||||
systemd/homehub-*.{service,timer}
|
||||
DESIGN.md rationale for everything below
|
||||
.rules -> AGENTS.md invariants + gotchas for future changes (auto-loaded by agents)
|
||||
```
|
||||
|
||||
## Install (per machine)
|
||||
|
||||
Reference in new issue
Block a user