homehub: rsync-based home-drive sync via Synology hub
DESIGN.md is the rationale; README.md the operations half. Four data classes: files (mtime-arbitrated two-way), git repos (manifest + clone, never byte-synced), the browser link bar (union merge), and everything else denied by default.
This commit is contained in:
druppy committed
commit
78496330f3
16 files changed
+1576
No files matched your search
@@ -0,0 +1,336 @@
|
|||||||
|
# homehub — lean home-drive sync across 3–4 machines via Synology
|
||||||
|
|
||||||
|
Status: **design, no code yet**. Open decisions in §13.
|
||||||
|
|
||||||
|
## 0. Ground truth this design was built against
|
||||||
|
|
||||||
|
| Fact | Value | Consequence |
|
||||||
|
|---|---|---|
|
||||||
|
| Hub | `nas.lue.dk`, DSM, Linux 4.4.302+ x86_64, `/bin/sh` | rsync-over-ssh, no daemon, no Syncthing package |
|
||||||
|
| Hub target | `/volume1/homes/bl` (4.9 TB free) | **cannot be the sync root — see next row** |
|
||||||
|
| Hub root | `/volume1/homes/bl` already holds **178 G of unrelated legacy data** (`Documents`, `Photos`, `OneDrive`, `CloudStation`, `home_backup`, `x1c_backup`, `wg-bo.conf`, `certs`, `@eaDir`) | sync root is a dedicated subtree `$HUB_BASE/homehub`; `check_hub_stray()` refuses to sync into a root that is not ours |
|
||||||
|
| NAS rsync | **3.1.2 (protocol 31)** | **no `--ignore-older`** → use `--update` both ways |
|
||||||
|
| Client rsync | 3.5.1 | protocol downgrade fine; do not use ≥3.2-only flags |
|
||||||
|
| Hub perms | siblings `admin, cl, fk, ida, …` on same share | `--no-perms --no-owner --no-group --chmod=Du=rwx,Dgo=,Fu=rw,Fgo=` |
|
||||||
|
| Laptop offline | weeks | no `--delete`; NAT-friendly health check (hub-side markers only) |
|
||||||
|
| Client ssh | `id_rsa` has a passphrase; `SSH_AUTH_SOCK` comes from a running agent | a **passphrase-free dedicated key** is mandatory (a user unit has no agent) and rsync gets the same identity via `-e` |
|
||||||
|
| NixOS | not started (`/nix` absent) | design must be identical pre/post migration → §8 |
|
||||||
|
|
||||||
|
Size reality check: `~` ≈ 250 GB, of which file-sync-worthy personal content is
|
||||||
|
**~0.5 GB** (`Documents` 71 M, `Pictures` 323 M, `bin` 45 M, `journal` 72 K,
|
||||||
|
dotfiles, `~/.pi/agent/sessions` 11 M). `~/git` 139 G + `~/src` 21 G are *repos*,
|
||||||
|
not files, and 7.2 G of `~/src` alone is `build/`+`target/`+`dist/`.
|
||||||
|
|
||||||
|
## 1. Four data classes, four mechanisms
|
||||||
|
|
||||||
|
| Class | What | Mechanism | Direction |
|
||||||
|
|---|---|---|---|
|
||||||
|
| A | personal files (docs, pictures, bin, dotfiles, tool *state*) | rsync via hub, mtime-arbitrated | two-way |
|
||||||
|
| B | git repos (`~/git/*`, repo-like dirs in `~/src/*`) | **manifest + clone** (`repos.tsv`) | two-way, metadata only |
|
||||||
|
| C | Chromium bookmark bar | per-host export + **union-by-URL merge** | many-to-one-to-many |
|
||||||
|
| D | everything else (caches, toolchains, builds, `tmp`, `Downloads`) | **default deny** | never |
|
||||||
|
|
||||||
|
Class B is the load-bearing idea. "Copy only the directory and the remote; each
|
||||||
|
machine has its own state" is not a file-sync requirement — it is a manifest. A
|
||||||
|
`repos.tsv` line is ~120 bytes where the repo is 4 GB, dirty state and local
|
||||||
|
branches never leave the machine, and 130 GB of vendored upstream clones
|
||||||
|
(`esp-idf`, `poky`, `bitbake`, `zephyr-sdk`, `llama.cpp`, `ollama`) become one
|
||||||
|
line each instead of a 139 GB transfer.
|
||||||
|
|
||||||
|
## 2. Hub layout
|
||||||
|
|
||||||
|
```
|
||||||
|
/volume1/homes/bl/homehub/ # dedicated sync root — NOT the Synology home
|
||||||
|
.hub/
|
||||||
|
meta/
|
||||||
|
markers/<host> # epoch of last SUCCESS (hub clock: no client skew)
|
||||||
|
alive/<host> # epoch of last ATTEMPT (written even on failure)
|
||||||
|
alerted/<host> # epoch of last alert (alert de-duplication)
|
||||||
|
log.tsv # one line per attempt: epoch, host, rc, note
|
||||||
|
tombstones/<host> # epoch \t path — paths that host deleted
|
||||||
|
gitman/repos.<host> # class B manifest
|
||||||
|
bookmarks/<host>.json # exported bookmark bar per host
|
||||||
|
prev/YYYY-MM/ # overwritten older versions; prev/reaped/ for reaped
|
||||||
|
<mirrored home layout> # Documents/, Pictures/, bin/, .ssh/, .pi/, ...
|
||||||
|
```
|
||||||
|
|
||||||
|
**The sync root must be a dedicated subtree.** `/volume1/homes/bl` is 178 G of
|
||||||
|
pre-existing, unrelated data. Pointing the tool at it would have (a) merged the
|
||||||
|
NAS's `Documents/` into `~/Documents` on the very first pull, (b) written our
|
||||||
|
dotfiles over the NAS's own `.config`, and (c) let the tombstone reaper move
|
||||||
|
unrelated legacy files into `.hub/prev/` whenever a local delete happened to match
|
||||||
|
a legacy path. `check_hub_stray()` now refuses to `sync --exec` into a hub root
|
||||||
|
containing anything that is not ours.
|
||||||
|
|
||||||
|
Nothing on the hub is ever edited by hand except `.hub/bookmarks/*` if you must.
|
||||||
|
`.hub/` is excluded from every pull and every push, so it can never leak into a
|
||||||
|
home tree or be deleted by one.
|
||||||
|
|
||||||
|
## 3. Sync algorithm (one run, on one machine, ~10 min cadence)
|
||||||
|
|
||||||
|
```
|
||||||
|
1 touch .hub/alive/<host> # "I am up" — even if the sync then fails
|
||||||
|
2 PULL tombstones from all hosts # apply locally: rm -rf within active sets
|
||||||
|
3 PULL rsync -rltDl -z --update # hub -> local, mtime-arbitrated, additive
|
||||||
|
4 PUSH rsync --update --backup # local -> hub, older hub files parked in .hub/prev/
|
||||||
|
5 write .hub/markers/<host> # ONLY on rc=0
|
||||||
|
```
|
||||||
|
|
||||||
|
Ordering note: pull-before-push means a file you edited locally while the hub had
|
||||||
|
a newer version is preserved by `--update` (skipped) *and* the hub copy is kept in
|
||||||
|
`.hub/prev/` — you never silently lose the other machine's work.
|
||||||
|
|
||||||
|
Command shape (pass 4, local → hub):
|
||||||
|
|
||||||
|
```
|
||||||
|
rsync -rltDl -z --update \
|
||||||
|
--no-perms --no-owner --no-group \
|
||||||
|
--chmod=Du=rwx,Dgo=,Fu=rw,Fgo= \
|
||||||
|
--backup --backup-dir="$HUB/.hub/prev/$(date +%Y-%m)" \
|
||||||
|
--filter="merge $STATE/effective.filter" \
|
||||||
|
~/ nas.lue.dk:/volume1/homes/bl/
|
||||||
|
```
|
||||||
|
|
||||||
|
Semantics, stated plainly so it is not a surprise later:
|
||||||
|
**edits always win; deletions only win if nobody else touched the file.**
|
||||||
|
|
||||||
|
## 4. Which files: sets + host membership (default deny)
|
||||||
|
|
||||||
|
`sets.d/<name>.include` holds rsync filter rules (`+ /Documents/***`, …).
|
||||||
|
`hosts/<hostname>` lists which sets that machine participates in.
|
||||||
|
`effective.filter` is generated per host per run:
|
||||||
|
|
||||||
|
```
|
||||||
|
- /.hub/***
|
||||||
|
- <§9 denylist rules, emitted first so they beat set includes>
|
||||||
|
+ /<prefix>/*** # union of this host's sets, in stable order
|
||||||
|
- * # default deny: anything not explicitly included never moves
|
||||||
|
```
|
||||||
|
|
||||||
|
First-match-wins, so the `-` global denylist must be emitted **before** `+` lines
|
||||||
|
that would otherwise match (e.g. `+ /src/***` vs `- /src/**/build/***`).
|
||||||
|
|
||||||
|
Proposed initial sets (tick/adjust in §13):
|
||||||
|
|
||||||
|
| Set | Members | Desktops | Laptop |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `core` | `.ssh/`, `.gitconfig`, `.bashrc`, `.profile`, `.p10k.zsh`, `bin/`, `Documents/`, `Pictures/`, `journal/`, `.local/state/homehub/`, `.pi/agent/{sessions,extensions,settings.json,models.json,AGENTS.md}`, `.gtk-bookmarks` | yes | yes |
|
||||||
|
| `repos` | `.hub`-side `gitman/repos.tsv` (Class B metadata only) | yes | yes |
|
||||||
|
| `work` | non-repo remainder of `~/src` (see §13 D2) | yes | opt-in |
|
||||||
|
| `bulk` | non-repo leftovers in `rustlings/`, `rustql/` | yes | no |
|
||||||
|
|
||||||
|
`~/Downloads` (35 G) and `~/tmp` are out, as decided. `~/.config/**` is **not** in
|
||||||
|
any set — see §8.
|
||||||
|
|
||||||
|
## 5. Deletion policy: tombstones + reaper, never `--delete`
|
||||||
|
|
||||||
|
`--delete` is incompatible with a weeks-offline peer: machine A, back from a
|
||||||
|
3-week trip, would delete from the hub files that another machine created or
|
||||||
|
edited while it was away, and the offline machine would then pull that deletion.
|
||||||
|
|
||||||
|
Instead:
|
||||||
|
|
||||||
|
- Push pass 1 is a `--dry-run --delete --itemize-changes` scan; `deleting` lines
|
||||||
|
are appended to `.hub/tombstones/<host>` (cost: one extra metadata-only scan).
|
||||||
|
- Real push is additive. Tombstones are applied locally on every pull (step 2).
|
||||||
|
- A **reaper** (runs only when a tombstone is older than the grace window, §13)
|
||||||
|
moves the hub copy to `.hub/prev/reaped/` and prunes the tombstone line, so a
|
||||||
|
file deleted in spring does not resurrect on a laptop that returns in autumn.
|
||||||
|
- `homehub forget <path>` deletes tombstones when you want a file back for good.
|
||||||
|
|
||||||
|
Why not `restic`? `--backup --backup-dir=.hub/prev/YYYY-MM` gives one prior
|
||||||
|
generation for free on every overwrite, needs no new tool on DSM 7/kernel 4.4,
|
||||||
|
and 4.9 TB free makes "keep the last version" affordable. Check whether Snapshot
|
||||||
|
Replication covers the `homes` share — `/volume1/@snapshots` was not readable as
|
||||||
|
`bl`; if it does, you get coarse history for free and should enable it.
|
||||||
|
|
||||||
|
## 6. Class B — git manifest spec
|
||||||
|
|
||||||
|
`gitman/inventory` walks `$HOME` over `git/` and `src/`, **stops descending at the
|
||||||
|
first `.git`** (so a repo's own submodules/modules are not double-counted), and
|
||||||
|
writes `.hub/gitman/repos.<host>.tsv`:
|
||||||
|
|
||||||
|
```
|
||||||
|
relpath remote_url head_branch head_sha dirty unpushed
|
||||||
|
src/rustql https://… main 9f2c1ab 0 3
|
||||||
|
git/ev_fw git@…:…/ev_fw main 41aa0c2 12 0
|
||||||
|
```
|
||||||
|
|
||||||
|
- `remote_url` is `git remote get-url origin`; that plus the path is the only thing
|
||||||
|
that crosses machines, exactly as asked.
|
||||||
|
- `dirty` / `unpushed` are **advisory only** — read by `homehub status`, never
|
||||||
|
synced as content. They answer "why does `ev_fw` look different on the laptop"
|
||||||
|
without turning rsync into a bad git.
|
||||||
|
- `homehub git missing` = union(all hosts' tsv) − repos present locally.
|
||||||
|
`homehub git clone-missing` prints commands; `--exec` runs them;
|
||||||
|
`--lazy` adds `--filter=blob:none` for the multi-GB vendored clones.
|
||||||
|
- Never file-sync any path that the inventory recognises as a repo. No `.git`
|
||||||
|
exclusion weirdness, no `build/` 7.2 GB problem: repo trees are cloned, not rsynced.
|
||||||
|
- Caveat to accept: unpushed commits do **not** survive disk death (you declined a
|
||||||
|
bare mirror on the hub — fine for personal use, just know it is the trade).
|
||||||
|
- `~/src/zephyrproject` is a `west` workspace: inventory records the top repo and
|
||||||
|
its sub-repos get recreated by `west update`, not by us.
|
||||||
|
|
||||||
|
## 7. Class C — bookmark bar
|
||||||
|
|
||||||
|
Live file: `~/.config/chromium/Profile 5/Bookmarks` (mtime 2026-10-24 — this is the
|
||||||
|
live profile; `~/.config/google-chrome/Default/Bookmarks` is dead, from 2021).
|
||||||
|
|
||||||
|
- **Read** the live file, copy to `.hub/bookmarks/<host>.json` (safe: read-only).
|
||||||
|
- **Merge** all hosts' files: union of `bookmark_bar` children keyed by URL,
|
||||||
|
preserving each host's folder nesting, dedupe identical URL+title. Idempotent,
|
||||||
|
so no conflict concept is needed for bookmarks.
|
||||||
|
- Write the merge back to the live path **only when the browser is not running**
|
||||||
|
(Chromium holds the file in memory and re-serialises it constantly; writing while
|
||||||
|
it runs is how bookmarks silently vanish). If it is running, defer to next run.
|
||||||
|
- Keep the last N exports on the hub, so a bad merge is always reversible.
|
||||||
|
- Boring alternative, if you ever stop caring about "only the bar": Chromium Sync.
|
||||||
|
|
||||||
|
## 8. Class D and the NixOS migration (the conflict in your answer 5)
|
||||||
|
|
||||||
|
Stop treating `~/.config` as one thing. Split by ownership, not by path:
|
||||||
|
|
||||||
|
| Kind | Owner | Where it physically lives | Synced? |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **declarative config** | Nix / home-manager (later: a git repo) | nix store, symlinked into `~/.config` | **no** |
|
||||||
|
| **state** (session history, caches-of-record, per-app data you want everywhere) | you | `~/.local/state/homehub/<name>/` | **yes** |
|
||||||
|
| **cache / binaries** | tools | `~/.cache`, `~/.local/share/…`, `.pi/agent/install` | **no** |
|
||||||
|
|
||||||
|
The sync set is defined **only by paths under the shared state root** plus an explicit
|
||||||
|
top-level list. Two roots, and the difference between them is not cosmetic:
|
||||||
|
|
||||||
|
| Root | Contents | Synced? |
|
||||||
|
|---|---|---|
|
||||||
|
| `~/.local/state/shared/<name>/` | **your** app state, symlinked to wherever each app expects it | **yes** |
|
||||||
|
| `~/.local/state/homehub/` | homehub's own scratch: generated filters, markers, `pull.err`, tombstone cache, `broken.*` | **no** (denied) |
|
||||||
|
|
||||||
|
Conflating them is a real bug I hit while testing: with the state root itself in the
|
||||||
|
`base` set, g9 pushed `effective.filter`, the laptop overwrote it with its own, and
|
||||||
|
they would have swapped generated filters forever. `~/.local/state/homehub/` is now
|
||||||
|
in the denylist so it cannot come back by accident.
|
||||||
|
|
||||||
|
On Debian you create the symlink by hand (or stow); on NixOS home-manager's `file.` /
|
||||||
|
`xdg.config."…".source` creates the *same* symlink into the *same* synced path. So
|
||||||
|
the migration rewrites the symlink layer and touches zero sync rules — that is the
|
||||||
|
resolution of "I like dotfiles locally, but I also want pi history synced": **the
|
||||||
|
sync tree owns state, Nix owns config, and neither writes into `~/.config` directly.**
|
||||||
|
|
||||||
|
Concrete for pi agent, from this box: sync `sessions/` (11 M, 96 `.jsonl` files —
|
||||||
|
append-only, so rsync-safe) and `settings.json`, `models.json`, `AGENTS.md`,
|
||||||
|
`extensions/`, hand-written `skills/*` trees; exclude `install/` (154 M),
|
||||||
|
`bin/` (9 M), `npm/`, and any `node_modules` under `skills/`. No sqlite/`-wal`
|
||||||
|
files exist in `~/.pi`, which is why plain rsync is safe here — if a tool ever
|
||||||
|
stores state in sqlite, it goes in the never-sync list or gets a dump-first hook.
|
||||||
|
|
||||||
|
## 9. Never-sync denylist (derived from actual `du`, not guessed)
|
||||||
|
|
||||||
|
`tmp/` (11 G) · `Downloads/` (35 G, decided) · `git/` and repo trees (Class B) ·
|
||||||
|
`.cache/` 12 G · `.steam/` 25 G · `.wine/` 14 G, `.wine_games/` 8.5 G,
|
||||||
|
`.wine_firefly/` 2.2 G · `zephyr-sdk-0.17.4/` 12 G · `go/` 8.5 G ·
|
||||||
|
`.espressif/` 6 G · `.npm/` 4.8 G · `.rustup/` 4.5 G · `.cargo/` 982 M ·
|
||||||
|
`.local/share/Trash/` · `.vscode/` 1.7 G, `.config/{google-chrome,discord,chromium,Code,sublime-text,VSCodium,Atom}` 11 G total (local by
|
||||||
|
default; only the bookmark file crosses machines, via §7) · `**/{build,target,dist,node_modules,__pycache__,.venv,.pixi,.zig-cache}/` ·
|
||||||
|
`.git/` (belt-and-braces alongside §6) · `.bash_history-*.tmp` (four 0-byte junk
|
||||||
|
files from a broken history append — also worth fixing at the source) ·
|
||||||
|
`.gksu.lock`, `*.tmp`, `*.bak`, `*.backup`.
|
||||||
|
|
||||||
|
## 10. "Only tell me when it has been broken for a day"
|
||||||
|
|
||||||
|
This is a **freshness** problem, not an error-notification problem, so build a
|
||||||
|
freshness monitor, not an error notifier.
|
||||||
|
|
||||||
|
- Sync units are silent: `homehub-sync.service` prints nothing on success,
|
||||||
|
`LogLevelMax=warning`. Failures produce no output and no popup; they simply fail
|
||||||
|
to update `markers/<host>`. No popups, no retry storms.
|
||||||
|
- Health unit: `homehub-fresh.timer`, `OnUnitActiveSec=30m`, runs `homehub fresh`,
|
||||||
|
which reads hub markers over one ssh call — **no inbound connectivity needed**,
|
||||||
|
so it works for the NATted laptop.
|
||||||
|
- Alert rule: `alive/<host>` is fresh (machine is on) **and** `markers/<host>` is
|
||||||
|
older than **24 h** **and** the last attempt's rc is *not* a network-class rc.
|
||||||
|
Network-class rc (5, 10, 11, 12, 30, 35, 255 — unreachable/timeout/connection)
|
||||||
|
is the weeks-offline laptop and stays **silent forever**; it only shows in
|
||||||
|
`homehub status`. That distinction is what keeps your requirement #2 (offline
|
||||||
|
for weeks) true without ever popping a notification.
|
||||||
|
- De-duplication: after alerting, stamp `alerted/<host>`; re-alert at most weekly
|
||||||
|
while still broken, clear on success. So one mail per incident, not 48.
|
||||||
|
- Transport: `sendmail` → your existing `mail.lue.dk`. Works with no GUI session,
|
||||||
|
which `notify-send` does not.
|
||||||
|
|
||||||
|
## 11. Files to be created (this repo = the policy; ~200 lines of shell total)
|
||||||
|
|
||||||
|
```
|
||||||
|
~/homehub/ # small git repo, hub-backed via a bare repo on the NAS
|
||||||
|
DESIGN.md README.md
|
||||||
|
bin/homehub # subcommands: sync, status, fresh, forget, git, bookmarks, filter, setup, install
|
||||||
|
lib/merge_bookmarks.py # class C union-merge
|
||||||
|
config # per-machine: hub path, key, thresholds, bookmark source
|
||||||
|
rules/denylist # §9 — DEFAULT DENY, emitted first
|
||||||
|
rules/sets.d/base.list # every machine (laptop included)
|
||||||
|
rules/sets.d/docs.list media.list misc.list
|
||||||
|
rules/hosts/g9 # -> sets: base docs media misc
|
||||||
|
rules/hosts/_laptop-template # -> sets: base docs
|
||||||
|
systemd/homehub-sync.{service,timer}
|
||||||
|
systemd/homehub-fresh.{service,timer}
|
||||||
|
```
|
||||||
|
|
||||||
|
Unit details that matter: `OnUnitActiveSec=10min RandomizedDelaySec=3min
|
||||||
|
Persistent=true` (catches up after suspend/off), `After=network-online.target
|
||||||
|
Conflicts=sleep.target`, `LogLevelMax=warning` + zero stdout on success, and a
|
||||||
|
**passphrase-less dedicated ed25519 deploy key** (`~/.ssh/homehub_ed25519`,
|
||||||
|
`IdentitiesOnly yes`) rather than `id_rsa` — a user unit has no `SSH_AUTH_SOCK`
|
||||||
|
(yours is a socket at `~/.ssh/agent/…` from a running agent), so depending on the
|
||||||
|
agent makes the background sync fail in exactly the way you would notice 26 hours
|
||||||
|
later. Optionally restrict the hub key with `from=` and a forced
|
||||||
|
`command="rsync --server…"`.
|
||||||
|
|
||||||
|
## 12. Rollout order
|
||||||
|
|
||||||
|
1. Hub: `mkdir -p .hub/{meta/{markers,alive,alerted},tombstones,bookmarks,prev,gitman}`, deploy key.
|
||||||
|
2. Write rules + `homehub sync --dry` on `g9`; inspect the file list by hand (this is the only real review step).
|
||||||
|
3. First real `core` sync from `g9`; verify `700/600` perms on the hub and that `.hub/` is absent locally.
|
||||||
|
4. `gitman/inventory` from `g9`; on machine #2 run `homehub git clone-missing --lazy`.
|
||||||
|
5. Machine #2 file sync, then watch convergence both ways.
|
||||||
|
6. Bookmark export+merge.
|
||||||
|
7. **Health monitor last** — it is only trustworthy once the sync semantics above have survived a week.
|
||||||
|
8. NixOS migration later: re-point the symlink layer only (§8); rules unchanged.
|
||||||
|
|
||||||
|
## 13. Decisions taken (was: open decisions)
|
||||||
|
|
||||||
|
| # | Decision | Outcome |
|
||||||
|
|---|---|---|
|
||||||
|
| D1 | Secrets in the sync set | **yes** — `.ssh/` incl. private keys, and `auth.json` if you add it. The hub now holds every key you own: it must stay LAN-only, and the hub copy is forced `700/600`. |
|
||||||
|
| D2 | `~/src` | **manifest only.** No file sync of repo trees, so the 7.2 G of `build/`+`target/`+`dist/` and every `.git` ambiguity disappeared. |
|
||||||
|
| D3 | Reaper grace | **30 days.** |
|
||||||
|
| D4 | Laptop membership | **`base` + `docs`.** (`core` was renamed `base`; `Documents`/`Pictures` were split into `docs`/`media` so the question was even well-posed — my original set list had `Documents` inside `core` *and* asked whether to include it.) |
|
||||||
|
| D5 | Alert transport | no local MTA yet → auto-detect `msmtp`/`sendmail`, else spool to `~/.local/state/homehub/alerts.out`. msmtp config for `mail.lue.dk:587` is in README. |
|
||||||
|
|
||||||
|
### Found while implementing (each one changed the design)
|
||||||
|
|
||||||
|
1. **`/volume1/homes/bl` is not empty** — 178 G of unrelated legacy data. Sync root
|
||||||
|
is now the dedicated subtree `$HUB_BASE/homehub`, plus a stray-content guard (§2).
|
||||||
|
2. **A live GitLab PAT sat inside a repo remote URL** (`https://user:glpat-…@gitlab.com/…`,
|
||||||
|
3 repos). The manifest would have published it to the NAS. Remotes are now scrubbed
|
||||||
|
of userinfo and the row is flagged `stripped`, which `git clone` reports.
|
||||||
|
*Separately: rotate that token, it has been sitting in `.git/config` on every
|
||||||
|
machine that cloned those repos.*
|
||||||
|
3. **`id_rsa` is passphrase-protected** and `SSH_AUTH_SOCK` comes from a live agent, so
|
||||||
|
a systemd user unit cannot authenticate. Fixed by a dedicated passphrase-free
|
||||||
|
ed25519 key — and, more importantly, by splitting *auth failure* (loud, after 24 h)
|
||||||
|
from *network failure* (silent forever) in `hub_probe()`. Before that split, a dead
|
||||||
|
key produced no marker, no heartbeat, and therefore no alert, ever.
|
||||||
|
4. **Synology writes `@eaDir` into every directory** — now denied, or a pull litters
|
||||||
|
an `@eaDir` next to every synced directory.
|
||||||
|
5. **`~/debian` is a dpkg work area** (150 M, 3598 files = 86 % of the first dry-run
|
||||||
|
push list). Denied as build output, like `~/tmp` and `~/Downloads`.
|
||||||
|
6. **rsync was not being told which key to use** (no `-e`), so it would have
|
||||||
|
authenticated as a different identity from the rest of the tool.
|
||||||
|
|
||||||
|
### Still open
|
||||||
|
|
||||||
|
* Rotate the GitLab PAT mentioned above; consider `id_rsa` (2016, 4096-bit RSA) too.
|
||||||
|
* Reconcile the legacy `home_backup` / `x1c_backup` trees on the NAS — they look like
|
||||||
|
earlier attempts at exactly this problem.
|
||||||
|
* Whether `~/bin` should sync at all: it is your scripts *plus* vendored binaries
|
||||||
|
(`protoc` 9 M, five gRPC plugins), 45 M of mostly machine-local bytes.
|
||||||
|
* `~/public_html` symlinks into `~/src`, `~/git` and `/tmp`; they will dangle on a
|
||||||
|
machine that has not cloned the target.
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
# homehub — operator notes
|
||||||
|
|
||||||
|
Operational doc. **Why** each decision exists is in [`DESIGN.md`](DESIGN.md); this file
|
||||||
|
is only the operational half.
|
||||||
|
|
||||||
|
```
|
||||||
|
~/homehub/
|
||||||
|
bin/homehub the tool (~450 lines of bash + one python merge)
|
||||||
|
config machine-specific settings (per-machine, not synced)
|
||||||
|
rules/denylist DEFAULT DENY, emitted first, wins over everything
|
||||||
|
rules/sets.d/*.list named path-lists ("sets")
|
||||||
|
rules/hosts/<hostname> which sets this machine joins
|
||||||
|
lib/merge_bookmarks.py class C union-merge (jq + python3 only)
|
||||||
|
systemd/homehub-*.{service,timer}
|
||||||
|
DESIGN.md rationale for everything below
|
||||||
|
```
|
||||||
|
|
||||||
|
## Install (per machine)
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git clone <your-repo> ~/homehub && cd ~/homehub
|
||||||
|
cp rules/hosts/_laptop-template rules/hosts/$(hostname -s) # pick your sets
|
||||||
|
./bin/homehub setup-key # once, see warning below
|
||||||
|
ssh-copy-id -i ~/.ssh/homehub_ed25519.pub nas.lue.dk # YOU run this
|
||||||
|
./bin/homehub setup
|
||||||
|
./bin/homehub sync # DRY: prints what would move, writes nothing
|
||||||
|
homehub sync --exec # first real run
|
||||||
|
sudo loginctl enable-linger $USER # headless boxes only
|
||||||
|
~/homehub/bin/homehub install # systemd user timers
|
||||||
|
```
|
||||||
|
|
||||||
|
`homehub install` copies the units into `~/.config/systemd/user` and enables
|
||||||
|
`homehub-sync.timer` (10 min) and `homehub-fresh.timer` (30 min).
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
| Command | What |
|
||||||
|
|---|---|
|
||||||
|
| `sync [--exec]` | pull → apply deletions → harvest deletions → push. **Dry unless `--exec`.** |
|
||||||
|
| `status` | last sync per machine, sets, pending dry-run lists, repo manifest, bookmark exports |
|
||||||
|
| `fresh [--dry]` | the 24 h alert check; only printer of alerts |
|
||||||
|
| `git inventory\|list\|missing\|clone [--exec] [--lazy]` | class B |
|
||||||
|
| `bookmarks [--write]` | class C export + union merge |
|
||||||
|
| `forget <path>` | drop tombstones so a deleted path can return |
|
||||||
|
| `filter` | dump the generated rsync filter |
|
||||||
|
| `setup`, `setup-key`, `install` | one-time plumbing |
|
||||||
|
|
||||||
|
## The rules that matter when editing config
|
||||||
|
|
||||||
|
1. **Default deny.** `rules/denylist` is emitted first, then the host's set includes,
|
||||||
|
then `- *`. A path in a set that sits under a deny rule can never sync — the tool
|
||||||
|
warns (`shadowed: ...`) when you write one.
|
||||||
|
2. **Set syntax.** One home-relative path per line; trailing `/` = subtree.
|
||||||
|
`/Documents/` → `- /Documents/***`; `build/` (no leading slash) matches at any depth.
|
||||||
|
3. **Never add a git repo path to a set.** Repos are class B (`homehub git inventory`).
|
||||||
|
4. **`~/.config` is unreachable by design** (it is denied wholesale). State you want
|
||||||
|
shared goes under `~/.local/state/shared/<app>/`, with the real config path
|
||||||
|
symlinked to it. `~/.local/state/homehub/` is homehub's *private* scratch and is
|
||||||
|
denied too — syncing it would make machines overwrite each other's generated
|
||||||
|
filters forever.
|
||||||
|
5. **Deletions never propagate as `--delete`.** They become tombstones on the hub,
|
||||||
|
are reaped after `GRACE_DAYS`, and always lose to a competing edit.
|
||||||
|
|
||||||
|
## Alerting
|
||||||
|
|
||||||
|
`homehub fresh` alerts only when a machine is **on** (heartbeat < 3 h) but has not
|
||||||
|
synced for > 24 h, and never for network-class rsync/ssh exit codes — so a laptop
|
||||||
|
away for three weeks is silent indefinitely. Delivery auto-detects `msmtp`, then
|
||||||
|
`sendmail`, else appends to `~/.local/state/homehub/alerts.out`. To wire it to your
|
||||||
|
own mail server:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# /etc/msmtprc (or ~/.config/msmtp/config)
|
||||||
|
defaults
|
||||||
|
tls on
|
||||||
|
tls_starttls on
|
||||||
|
account homehub
|
||||||
|
host mail.lue.dk
|
||||||
|
port 587
|
||||||
|
auth on
|
||||||
|
user homehub@lue.dk
|
||||||
|
from homehub@lue.dk
|
||||||
|
password <app-password>
|
||||||
|
```
|
||||||
|
|
||||||
|
Then `msmtp -a homehub you@lue.dk` is picked up automatically. On Debian:
|
||||||
|
`apt install msmtp msmtp-mta`.
|
||||||
|
|
||||||
|
A machine whose **own** config is broken (ssh key revoked, hub disk full) records a
|
||||||
|
local marker in `~/.local/state/homehub/broken.*` and alerts about itself even with
|
||||||
|
the hub unreachable — otherwise a dead key would produce no heartbeat, no marker,
|
||||||
|
and therefore no alert, forever.
|
||||||
|
|
||||||
|
## Known limits (deliberate, not bugs)
|
||||||
|
|
||||||
|
* **mtime is the arbiter.** Two machines editing the same file while both offline
|
||||||
|
resolve by mtime, so clocks must be NTP-synced (`timedatectl` on Debian, `ntpd` on
|
||||||
|
NixOS). The overwritten copy is kept in `<hub>/.hub/prev/YYYY-MM/`.
|
||||||
|
* **rsync 3.1.2 on DSM 6/7** — no `--ignore-older`; the design uses `--update` only.
|
||||||
|
* `--checksum` is off, so a file edited to the same size in the same second as a
|
||||||
|
previous sync may be missed. Re-run `rsync -mc` by hand if you ever suspect it.
|
||||||
|
* Unpushed commits are advisory in the manifest and never transferred. A dead laptop
|
||||||
|
disk loses them.
|
||||||
|
* Empty bookmark folders do not survive the union merge (there is nothing to key on).
|
||||||
|
* Symlinks travel as symlinks (`-l`, never `-L`); ones pointing into `~/src`, `~/git`
|
||||||
|
or `/tmp` will dangle on a machine that has not cloned the target.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
```sh
|
||||||
|
~/homehub/bin/homehub status # ages, sets, manifest
|
||||||
|
journalctl --user -u homehub-sync.service -n 50
|
||||||
|
cat ~/.local/state/homehub/{pull,push}.err
|
||||||
|
cat ~/.local/state/homehub/broken.* # self-detected breakage
|
||||||
|
ssh nas.lue.dk 'cat /volume1/homes/bl/homehub/.hub/log.tsv | tail'
|
||||||
|
~/homehub/bin/homehub filter | grep -n shadowed
|
||||||
|
```
|
||||||
Executable
+664
@@ -0,0 +1,664 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# homehub — lean home-drive sync: rsync against a Synology hub, mtime-arbitrated,
|
||||||
|
# silent unless something has been broken for a day.
|
||||||
|
#
|
||||||
|
# sync [--exec] one pull+push cycle (DRY unless --exec)
|
||||||
|
# status what would move, and who synced when
|
||||||
|
# fresh [--dry] the only command allowed to complain (24 h rule)
|
||||||
|
# git <sub> class B: repos.tsv manifest, clone on the far side
|
||||||
|
# bookmarks [--write] class C: union-merge the link bar
|
||||||
|
# forget <path>... drop tombstones so a path can come back
|
||||||
|
# filter dump the generated rsync filter
|
||||||
|
# setup | setup-key | install one-time plumbing
|
||||||
|
#
|
||||||
|
# Rationale for every odd choice: ../DESIGN.md
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
HH_ROOT=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)
|
||||||
|
[ -r "$HH_ROOT/config" ] || { printf 'homehub: no config at %s\n' "$HH_ROOT/config" >&2; exit 1; }
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
. "$HH_ROOT/config"
|
||||||
|
|
||||||
|
HOST=$(hostname -s)
|
||||||
|
HUB_URL="$HUB_HOST:$HUB_PATH"
|
||||||
|
RULES=$HH_ROOT/rules
|
||||||
|
STATE=${STATE_ROOT:?config: STATE_ROOT unset}
|
||||||
|
FILTER=$STATE/effective.filter
|
||||||
|
mkdir -p "$STATE/tmp"
|
||||||
|
|
||||||
|
# systemd sets HH_QUIET=1: silence on success is a feature, not an omission (§10)
|
||||||
|
v() { [ "${HH_QUIET:-0}" = 1 ] || printf ' %s\n' "$*"; }
|
||||||
|
warn() { printf 'homehub: %s\n' "$*" >&2; }
|
||||||
|
die() { printf 'homehub: %s\n' "$*" >&2; exit 1; }
|
||||||
|
now() { date +%s; }
|
||||||
|
hr() { [ "${HH_QUIET:-0}" = 1 ] || printf '\n== %s ==\n' "$*"; }
|
||||||
|
|
||||||
|
# ------------------------------------------------------------- ssh plumbing --
|
||||||
|
SSH_OPTS=(-o BatchMode=yes -o ConnectTimeout="${SSH_TIMEOUT:-10}" -o LogLevel=ERROR)
|
||||||
|
[ -f "$HH_KEY" ] && SSH_OPTS+=(-i "$HH_KEY" -o IdentitiesOnly=yes)
|
||||||
|
hub() { ssh "${SSH_OPTS[@]}" "$HUB_HOST" "$@"; }
|
||||||
|
hub_ok() { hub true >/dev/null 2>&1; }
|
||||||
|
|
||||||
|
# Telling "off the LAN" apart from "our own config is broken" is the whole point of
|
||||||
|
# §10: the first must stay silent for weeks, the second must become loud after a
|
||||||
|
# day. 0 = usable, 1 = network/unreachable (stays silent), 2 = auth/config (loud).
|
||||||
|
hub_probe() {
|
||||||
|
local out
|
||||||
|
if ! out=$(ssh "${SSH_OPTS[@]}" "$HUB_HOST" true 2>&1); then
|
||||||
|
case $out in
|
||||||
|
*"Permission denied"*|*"Host key verification"*|*"no matching host key"*|*"invalid format"*|*"no such identity"*)
|
||||||
|
return 2 ;;
|
||||||
|
esac
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# Local breakage markers — readable with the hub down, so a machine whose own key
|
||||||
|
# stopped working can still alert about itself. First-failure epoch is preserved.
|
||||||
|
mark_broken() { # kind message
|
||||||
|
local f=$STATE/broken.$1 e
|
||||||
|
e=$(awk -F'\t' 'NF{print $1; exit}' "$f" 2>/dev/null)
|
||||||
|
[ -n "${e:-}" ] || e=$(now)
|
||||||
|
printf '%s\t%s\n' "$e" "$2" > "$f"
|
||||||
|
}
|
||||||
|
clear_broken() { rm -f "$STATE/broken.$1" 2>/dev/null || true; }
|
||||||
|
# Remote script with positional args, no quoting hell:
|
||||||
|
# hub_script "'$HUB_PATH' '$HOST'" <<'EOS' ... EOS (POSIX sh on DSM!)
|
||||||
|
hub_script() { ssh "${SSH_OPTS[@]}" "$HUB_HOST" "sh -s -- $1"; }
|
||||||
|
|
||||||
|
touch_marker() {
|
||||||
|
hub "mkdir -p '$HUB_PATH/.hub/meta/$1' && touch '$HUB_PATH/.hub/meta/$1/$HOST'" || true
|
||||||
|
}
|
||||||
|
log_line() { # rc note
|
||||||
|
printf '%s\t%s\t%s\t%s\n' "$(now)" "$HOST" "$1" "$2" |
|
||||||
|
hub "mkdir -p '$HUB_PATH/.hub' && cat >> '$HUB_PATH/.hub/log.tsv'" 2>/dev/null || true
|
||||||
|
}
|
||||||
|
hub_setup_dirs() {
|
||||||
|
hub "mkdir -p '$HUB_PATH/.hub/meta/markers' '$HUB_PATH/.hub/meta/alive' \
|
||||||
|
'$HUB_PATH/.hub/meta/alerted' '$HUB_PATH/.hub/tombstones' '$HUB_PATH/.hub/prev' \
|
||||||
|
'$HUB_PATH/.hub/gitman' '$HUB_PATH/.hub/bookmarks' && chmod 700 '$HUB_PATH'" || true
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---------------------------------------------------- filter (default deny) --
|
||||||
|
sets_for_host() {
|
||||||
|
local f=$RULES/hosts/$HOST
|
||||||
|
[ -r "$f" ] || { warn "no rules/hosts/$HOST — this host syncs nothing (see rules/hosts/_laptop-template)"; return 1; }
|
||||||
|
awk '!/^[[:space:]]*#/ && NF' "$f"
|
||||||
|
}
|
||||||
|
# one path per line -> rsync rule; trailing slash means "this dir and its subtree"
|
||||||
|
rules_from() { # file sign
|
||||||
|
awk -v s="$2" '!/^[[:space:]]*#/ && NF {
|
||||||
|
if ($0 ~ /\/$/) printf "%s %s***\n", s, $0; else printf "%s %s\n", s, $0 }' "$1"
|
||||||
|
}
|
||||||
|
# ancestors must be explicitly traversable, otherwise a blanket "- *" can prune
|
||||||
|
# the branch before rsync ever reaches the include below it
|
||||||
|
ancestors_of() { # /a/b/c/ -> "+ /a/", "+ /a/b/"
|
||||||
|
local p=${1%/}
|
||||||
|
while [ -n "$p" ] && [ "${p#*/}" != "$p" ]; do
|
||||||
|
p=${p%/*}
|
||||||
|
[ -n "$p" ] && printf '+ %s/\n' "$p"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
check_shadowing() { # $1 set name
|
||||||
|
local f=$RULES/sets.d/$1.list d inc nm
|
||||||
|
[ -r "$f" ] || return 0
|
||||||
|
while read -r inc; do
|
||||||
|
while read -r d; do
|
||||||
|
case $d in
|
||||||
|
"$inc") warn "shadowed: include '$inc' is also denied verbatim (set $1)"; continue ;;
|
||||||
|
esac
|
||||||
|
case $d in
|
||||||
|
/*/) case $inc in "$d"*|"${d%/}") warn "shadowed: include '$inc' sits under deny rule '$d' (set $1)" ;; esac ;;
|
||||||
|
*/) nm=${d%/}; case "/$inc" in *"/$nm"/*|*"/$nm") warn "shadowed: include '$inc' matches unanchored deny '$d' (set $1)" ;; esac ;;
|
||||||
|
esac
|
||||||
|
done < <(awk '!/^[[:space:]]*#/ && NF' "$RULES/denylist" 2>/dev/null)
|
||||||
|
done < <(awk '!/^[[:space:]]*#/ && NF' "$f")
|
||||||
|
}
|
||||||
|
# base filter: deny first, then includes (+ ancestors), NO trailing default-deny
|
||||||
|
gen_filter() {
|
||||||
|
local sets s
|
||||||
|
sets=$(sets_for_host) || return 1
|
||||||
|
{
|
||||||
|
echo '- /.hub/***' # hub metadata can never leak into a home
|
||||||
|
[ -r "$RULES/denylist" ] && rules_from "$RULES/denylist" '-'
|
||||||
|
for s in $sets; do
|
||||||
|
[ -r "$RULES/sets.d/$s.list" ] || { warn "unknown set '$s' (no rules/sets.d/$s.list)"; continue; }
|
||||||
|
{ rules_from "$RULES/sets.d/$s.list" '+'
|
||||||
|
awk '!/^[[:space:]]*#/ && NF' "$RULES/sets.d/$s.list" | while read -r l; do ancestors_of "$l"; done; }
|
||||||
|
check_shadowing "$s"
|
||||||
|
done
|
||||||
|
} | awk '!seen[$0]++' > "$FILTER.base.tmp" || return 1
|
||||||
|
mv "$FILTER.base.tmp" "$FILTER.base"
|
||||||
|
# push filter: default deny only
|
||||||
|
{ cat "$FILTER.base"; echo '- *'; } > "$FILTER.tmp" && mv "$FILTER.tmp" "$FILTER"
|
||||||
|
# pull filter: default deny, PLUS still-tombstoned paths excluded so a deleted
|
||||||
|
# file is not re-downloaded every cycle for 30 days only to be re-deleted.
|
||||||
|
# Tombstones never enter the push filter: a file re-created after the delete
|
||||||
|
# must still be able to travel (edits win — DESIGN.md §3).
|
||||||
|
{
|
||||||
|
cat "$FILTER.base"
|
||||||
|
awk -F'\t' 'NF>=2 { p=$2; sub(/^\//, "", p); if (p != "" && p !~ /^\.hub\// && p !~/\.\./) {
|
||||||
|
print "- /" p; print "- /" p "/***" } }' "$STATE/tombstones.all" 2>/dev/null
|
||||||
|
echo '- *'
|
||||||
|
} | awk '!seen[$0]++' > "$FILTER.pull.tmp" && mv "$FILTER.pull.tmp" "$FILTER.pull"
|
||||||
|
}
|
||||||
|
|
||||||
|
# --------------------------------------------------------------- rsync bits --
|
||||||
|
# rsync must authenticate with the SAME identity as the rest of homehub, or it
|
||||||
|
# silently picks another key/agent and unreachable comes back from the wrong door.
|
||||||
|
RSH="ssh -o BatchMode=yes -o ConnectTimeout=${SSH_TIMEOUT:-10} -o LogLevel=ERROR"
|
||||||
|
[ -f "${HH_KEY:-}" ] && RSH="$RSH -i $HH_KEY -o IdentitiesOnly=yes"
|
||||||
|
RS=(-rltDz --partial --timeout=90 --out-format='%i|%n' -e "$RSH")
|
||||||
|
run_rsync() { # dry filter extra args...
|
||||||
|
local dry=$1 filt=$2; shift 2
|
||||||
|
if [ "$dry" = 1 ]; then
|
||||||
|
"$RSYNC_BIN" "${RS[@]}" --filter="merge $filt" -n "$@"
|
||||||
|
else
|
||||||
|
"$RSYNC_BIN" "${RS[@]}" --filter="merge $filt" "$@"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
pull_rsync() { # dry
|
||||||
|
run_rsync "$1" "$FILTER.pull" --update "$HUB_URL/" "$HOME/"
|
||||||
|
}
|
||||||
|
push_rsync() { # dry
|
||||||
|
if [ "$1" = 1 ]; then
|
||||||
|
run_rsync 1 "$FILTER" --update "$HOME/" "$HUB_URL/"
|
||||||
|
else
|
||||||
|
run_rsync 0 "$FILTER" --update --backup --backup-dir=".hub/prev/$(date +%Y-%m)" "$HOME/" "$HUB_URL/"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
count_items() { grep -cE '^[<>]' "$1" 2>/dev/null || true; }
|
||||||
|
show_head() { grep -E '^[<>]' "$1" 2>/dev/null | head -n "${2:-10}"; }
|
||||||
|
|
||||||
|
# ----------------------------------------------------- deletions (tombstones) --
|
||||||
|
# Format on the hub: .hub/tombstones/<host> lines = epoch \t home-rel-path
|
||||||
|
hub_tombstones_all() {
|
||||||
|
hub_script "'$HUB_PATH'" <<'EOS'
|
||||||
|
root=$1
|
||||||
|
cd "$root/.hub/tombstones" 2>/dev/null || exit 0
|
||||||
|
for f in *; do
|
||||||
|
[ -f "$f" ] || continue
|
||||||
|
while IFS= read -r l; do printf '%s\t%s\n' "${f%%:*}" "$l"; done < "$f"
|
||||||
|
done
|
||||||
|
exit 0
|
||||||
|
EOS
|
||||||
|
}
|
||||||
|
# is this home-relative path inside one of this host's sets?
|
||||||
|
active_path() { # $1 = path without leading slash
|
||||||
|
local p="/$1"
|
||||||
|
while :; do
|
||||||
|
grep -qxF "+ $p***" "$FILTER" && return 0 # inside an included subtree
|
||||||
|
grep -qxF "+ $p" "$FILTER" && return 0 # is itself an included file
|
||||||
|
[ "${p%/*}" = "$p" ] && return 1
|
||||||
|
p=${p%/*}
|
||||||
|
[ -z "$p" ] && return 1
|
||||||
|
done
|
||||||
|
}
|
||||||
|
apply_tombstones() { # deletions recorded by any machine, edit-wins (§3)
|
||||||
|
local all=$STATE/tombstones.all n=0 kept=0 out=0 bad=0
|
||||||
|
[ -s "$all" ] || { v "tombstones: none recorded"; return 0; }
|
||||||
|
while IFS=$'\t' read -r src ep path; do
|
||||||
|
[ -n "${path:-}" ] || continue
|
||||||
|
path=${path#/}
|
||||||
|
case $path in
|
||||||
|
.hub/*|*..*|/*) warn "ignoring suspicious tombstone '$path'"; bad=$((bad+1)); continue ;;
|
||||||
|
esac
|
||||||
|
active_path "$path" || { out=$((out+1)); continue; }
|
||||||
|
[ -e "$HOME/$path" ] || continue
|
||||||
|
if [ -n "$ep" ] && find "$HOME/$path" -newermt "@$ep" -print -quit 2>/dev/null | grep -q .; then
|
||||||
|
kept=$((kept+1)); continue # touched after the delete: edits win, keep it
|
||||||
|
fi
|
||||||
|
n=$((n+1))
|
||||||
|
if [ "$n" -gt "${TOMB_LIMIT:-100}" ] && [ "${HH_FORCE:-0}" != 1 ]; then
|
||||||
|
die "tombstones ask for more than $TOMB_LIMIT deletions — refusing. Check $HUB_HOST:$HUB_PATH/.hub/tombstones/ then re-run with HH_FORCE=1"
|
||||||
|
fi
|
||||||
|
v "deleted (tombstone from $src): $path"
|
||||||
|
rm -rf --one-file-system -- "$HOME/$path"
|
||||||
|
done < "$all"
|
||||||
|
v "tombstones: applied=$n edited-since-so-kept=$kept outside-sets=$out malformed=$bad"
|
||||||
|
}
|
||||||
|
harvest_tombstones() { # record what this machine deleted; runs only after a real pull
|
||||||
|
local new=$STATE/tomb.new merged=$STATE/tomb.merged n ep
|
||||||
|
run_rsync 1 --delete "$HOME/" "$HUB_URL/" 2>/dev/null |
|
||||||
|
awk -F'|' '$1 ~ /^\*/ { sub(/^\//, "", $2); print $2 }' | sort -u > "$new"
|
||||||
|
n=$(grep -c . "$new" 2>/dev/null || true)
|
||||||
|
[ "${n:-0}" = 0 ] && { v "no local deletions to record"; return 0; }
|
||||||
|
ep=$(now)
|
||||||
|
{ hub "cat '$HUB_PATH/.hub/tombstones/$HOST' 2>/dev/null"
|
||||||
|
awk -v e="$ep" '{ print e "\t" $0 }' "$new"
|
||||||
|
} | awk -F'\t' 'NF>=2 { p=$2; if (!(p in e) || $1+0 < e[p]) e[p]=$1+0 }
|
||||||
|
END { for (p in e) printf "%d\t%s\n", e[p], p }' | sort -t$'\t' -k2 > "$merged"
|
||||||
|
hub "mkdir -p '$HUB_PATH/.hub/tombstones' && cat > '$HUB_PATH/.hub/tombstones/$HOST'" < "$merged" ||
|
||||||
|
{ warn "could not write tombstones"; return 1; }
|
||||||
|
v "recorded $n deletion(s) as tombstones on the hub"
|
||||||
|
}
|
||||||
|
reap() { # stop resurrecting files deleted more than GRACE_DAYS ago (§5)
|
||||||
|
hub_script "'$HUB_PATH' '${GRACE_DAYS:-30}'" <<'EOS' || true
|
||||||
|
root=$1; grace=$2
|
||||||
|
cd "$root/.hub/tombstones" 2>/dev/null || exit 0
|
||||||
|
cutoff=$(( $(date +%s) - grace * 86400 ))
|
||||||
|
for f in *; do
|
||||||
|
[ -f "$f" ] || continue
|
||||||
|
awk -F'\t' -v c="$cutoff" '$1+0 < c { print }' "$f" > "$f.gone"
|
||||||
|
awk -F'\t' -v c="$cutoff" '$1+0 >= c { print }' "$f" > "$f.keep"
|
||||||
|
while IFS=$'\t' read -r e p; do
|
||||||
|
[ -n "$p" ] || continue
|
||||||
|
t="$root/$p"
|
||||||
|
[ -e "$t" ] || continue
|
||||||
|
d="$root/.hub/prev/reaped/$(date -d "@$e" +%Y-%m 2>/dev/null || echo unknown)"
|
||||||
|
mkdir -p "$d/$(dirname "$p")" 2>/dev/null && mv "$t" "$d/$p" 2>/dev/null && echo "reaped $p"
|
||||||
|
done < "$f.gone"
|
||||||
|
if [ -s "$f.keep" ]; then mv "$f.keep" "$f"; else rm -f "$f.keep" "$f"; fi
|
||||||
|
rm -f "$f.gone"
|
||||||
|
done
|
||||||
|
exit 0
|
||||||
|
EOS
|
||||||
|
}
|
||||||
|
|
||||||
|
# The hub sync root must contain ONLY what homehub put there. Anything else means
|
||||||
|
# HUB_PATH points somewhere that already has data — which would be merged into
|
||||||
|
# $HOME on the first pull (and could be reaped by a tombstone). This guard exists
|
||||||
|
# because /volume1/homes/bl really did hold 178 G of unrelated legacy data.
|
||||||
|
check_hub_stray() {
|
||||||
|
[ -r "$FILTER.base" ] || return 0
|
||||||
|
local allowed stray
|
||||||
|
allowed=$(printf '.hub\n@eaDir\n'; grep '^+ /' "$FILTER.base" | sed -E 's#^\+ /([^/]+).*#\1#; s#/$##' | sort -u)
|
||||||
|
stray=$(hub "cd '$HUB_PATH' 2>/dev/null && ls -A" | grep -vxF -f <(printf '%s\n' "$allowed") || true)
|
||||||
|
[ -z "$stray" ] && return 1
|
||||||
|
warn "unexpected content in hub sync root $HUB_PATH (pull would merge it into $HOME):"
|
||||||
|
printf ' %s\n' $stray
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ sync ----
|
||||||
|
cmd_sync() {
|
||||||
|
local exec=0; [ "${1:-}" = "--exec" ] && exec=1
|
||||||
|
local dry=1; [ $exec = 1 ] && dry=0
|
||||||
|
local sets rc=0
|
||||||
|
|
||||||
|
sets=$(sets_for_host) || return 1
|
||||||
|
|
||||||
|
case "$(hub_probe; echo $?)" in
|
||||||
|
1) # Off the LAN. Silent on purpose: the laptop-in-another-country case (§10).
|
||||||
|
v "hub $HUB_HOST unreachable — nothing done (silent by design; a live machine that stays broken alerts through 'fresh' after ${FRESH_MAX_AGE}s)"
|
||||||
|
return 0 ;;
|
||||||
|
2) warn "cannot authenticate to $HUB_HOST (key: ${HH_KEY:-default agent}). This is a CONFIGURATION failure, not an offline machine."
|
||||||
|
warn " fix: ssh-copy-id -i ${HH_KEY:-~/.ssh/id_ed25519}.pub $HUB_HOST"
|
||||||
|
mark_broken auth "ssh auth to $HUB_HOST failed (key ${HH_KEY:-default})"
|
||||||
|
return 64 ;;
|
||||||
|
esac
|
||||||
|
clear_broken auth
|
||||||
|
hub_setup_dirs
|
||||||
|
if check_hub_stray; then
|
||||||
|
[ $exec = 1 ] && [ "${HH_FORCE:-0}" != 1 ] &&
|
||||||
|
die "refusing to sync into a hub root that is not ours. Fix HUB_PATH in config, or move that data away, or re-run with HH_FORCE=1 if you really mean it"
|
||||||
|
fi
|
||||||
|
# tombstones must be known BEFORE the filter is built: they shape the pull filter
|
||||||
|
hub_tombstones_all > "$STATE/tombstones.all" || true
|
||||||
|
gen_filter || return 1
|
||||||
|
hr "homehub sync ($([ $exec = 1 ] && echo EXECUTE || echo 'dry run')) on $HOST"
|
||||||
|
v "sets: $sets"
|
||||||
|
v "filter: $(grep -c . "$FILTER" || true) push, $(grep -c . "$FILTER.pull" || true) pull rules, $(grep -c . "$STATE/tombstones.all" 2>/dev/null || true) tombstoned"
|
||||||
|
[ $exec = 1 ] && touch_marker alive
|
||||||
|
|
||||||
|
hr "1. pull $HUB_URL -> $HOME/"
|
||||||
|
pull_rsync "$dry" > "$STATE/pull.out" 2> "$STATE/pull.err" || rc=$?
|
||||||
|
if [ $rc != 0 ]; then
|
||||||
|
warn "pull failed (rsync rc=$rc): $(tr '\n' ' ' < "$STATE/pull.err" | tail -c 300)"
|
||||||
|
else
|
||||||
|
v "$(count_items "$STATE/pull.out") change(s) inbound"
|
||||||
|
[ $dry = 1 ] && show_head "$STATE/pull.out"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ $exec = 1 ]; then
|
||||||
|
hr "2. apply deletions recorded by any machine (edit-wins)"
|
||||||
|
apply_tombstones
|
||||||
|
hr "3. harvest this machine's deletions"
|
||||||
|
harvest_tombstones || rc=1
|
||||||
|
else
|
||||||
|
hr "2./3. deletion handling skipped in dry run"
|
||||||
|
fi
|
||||||
|
|
||||||
|
hr "4. push $HOME/ -> $HUB_URL"
|
||||||
|
push_rsync "$dry" > "$STATE/push.out" 2> "$STATE/push.err" || rc=$?
|
||||||
|
if [ $rc != 0 ]; then
|
||||||
|
warn "push failed (rsync rc=$rc): $(tr '\n' ' ' < "$STATE/push.err" | tail -c 300)"
|
||||||
|
else
|
||||||
|
v "$(count_items "$STATE/push.out") change(s) outbound"
|
||||||
|
[ $dry = 1 ] && show_head "$STATE/push.out"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ $exec = 1 ] && [ $rc = 0 ]; then
|
||||||
|
reap | sed 's/^/ /'
|
||||||
|
touch_marker markers
|
||||||
|
hub "rm -f '$HUB_PATH/.hub/meta/alerted/$HOST'" || true
|
||||||
|
clear_broken sync
|
||||||
|
elif [ $exec = 1 ]; then
|
||||||
|
mark_broken sync "last sync failed with rsync rc=$rc (see $STATE/pull.err, $STATE/push.err)"
|
||||||
|
fi
|
||||||
|
[ $exec = 1 ] && log_line "$rc" "sync"
|
||||||
|
|
||||||
|
if [ $dry = 1 ]; then
|
||||||
|
local tot; tot=$(count_items "$STATE/pull.out"); tot=$((tot + $(count_items "$STATE/push.out")))
|
||||||
|
hr "summary"
|
||||||
|
v "$tot change(s) pending — nothing was written. Run: homehub sync --exec"
|
||||||
|
[ "${tot:-0}" -gt "${WARN_FILES:-20000}" ] &&
|
||||||
|
warn "that is a lot of files for a first run; review $STATE/{pull,push}.out first"
|
||||||
|
fi
|
||||||
|
return $rc
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- status ----
|
||||||
|
hub_meta() {
|
||||||
|
hub_script "'$HUB_PATH'" <<'EOS'
|
||||||
|
root=$1
|
||||||
|
cd "$root/.hub" 2>/dev/null || exit 0
|
||||||
|
for d in meta/markers meta/alive meta/alerted; do
|
||||||
|
for f in "$d"/*; do
|
||||||
|
[ -f "$f" ] || continue
|
||||||
|
printf '%s\t%s\t%s\n' "${d##*/}" "$(basename "$f")" "$(date -r "$f" +%s 2>/dev/null || echo 0)"
|
||||||
|
done
|
||||||
|
done
|
||||||
|
if [ -f log.tsv ]; then
|
||||||
|
tail -n 400 log.tsv | awk -F'\t' '{ r[$2] = $3 "\t" $4 } END { for (h in r) printf "lastlog\t%s\t%s\n", h, r[h] }'
|
||||||
|
fi
|
||||||
|
for f in tombstones/*; do [ -f "$f" ] && printf 'tomb\t%s\t%s\n' "$(basename "$f")" "$(grep -c . "$f")"; done
|
||||||
|
for f in gitman/repos.*; do [ -f "$f" ] && printf 'repos\t%s\t%s\n' "$(basename "$f" 2>/dev/null)" "$(grep -c . "$f")"; done
|
||||||
|
for f in bookmarks/*; do [ -f "$f" ] && printf 'bmarks\t%s\t%s\n' "$(basename "$f" .json)" "$(date -r "$f" +%s)"; done
|
||||||
|
exit 0
|
||||||
|
EOS
|
||||||
|
}
|
||||||
|
age() { local n=$(( $(now) - ${1:-0} )); [ "${1:-0}" = 0 ] && { printf 'never'; return; }
|
||||||
|
if [ $n -lt 3600 ]; then printf '%dm' $((n/60))
|
||||||
|
elif [ $n -lt 172800 ]; then printf '%.1fh' "$(awk -v n=$n 'BEGIN{print n/3600}')"
|
||||||
|
else printf '%.0fd' "$(awk -v n=$n 'BEGIN{print n/86400}')"; fi; }
|
||||||
|
|
||||||
|
cmd_status() {
|
||||||
|
hub_ok || die "hub $HUB_HOST unreachable"
|
||||||
|
sets_for_host >/dev/null || true
|
||||||
|
gen_filter 2>/dev/null || true
|
||||||
|
hub_meta > "$STATE/meta.tsv" || die "could not read hub metadata"
|
||||||
|
|
||||||
|
hr "machines"
|
||||||
|
printf ' %-14s %-9s %-9s %-6s %s\n' HOST synced heartbeat last-rc notes
|
||||||
|
local h
|
||||||
|
for h in $(awk -F'\t' '$1=="markers"||$1=="alive"{print $2}' "$STATE/meta.tsv" | sort -u); do
|
||||||
|
local m a rc note
|
||||||
|
m=$(awk -F'\t' -v h="$h" '$1=="markers"&&$2==h{print $3}' "$STATE/meta.tsv")
|
||||||
|
a=$(awk -F'\t' -v h="$h" '$1=="alive"&&$2==h{print $3}' "$STATE/meta.tsv")
|
||||||
|
rc=$(awk -F'\t' -v h="$h" '$1=="lastlog"{split($0,x,"\t"); if (x[2]==h) print x[3]}' "$STATE/meta.tsv")
|
||||||
|
note=$(awk -F'\t' -v h="$h" '$1=="lastlog"{split($0,x,"\t"); if (x[2]==h) print x[4]}' "$STATE/meta.tsv")
|
||||||
|
printf ' %-14s %-9s %-9s %-6s %s\n' "$h" "$(age "$m")" "$(age "$a")" "${rc:-?}" "${note:-}"
|
||||||
|
done
|
||||||
|
|
||||||
|
hr "this host ($HOST)"
|
||||||
|
v "sets: $(sets_for_host | tr '\n' ' ')"
|
||||||
|
v "included: $(grep -c '^+ ' "$FILTER" 2>/dev/null || echo 0) rules, denied: $(grep -c '^- ' "$FILTER" 2>/dev/null || echo 0)"
|
||||||
|
v "state dir: $STATE"
|
||||||
|
awk -F'\t' -v h="$HOST" '$1=="tomb"&&$2==h{print " tombstones: " $3 " paths"}' "$STATE/meta.tsv"
|
||||||
|
v "pending dry-run lists: $STATE/pull.out $STATE/push.out (run 'homehub sync' to refresh)"
|
||||||
|
|
||||||
|
hr "class B — repo manifest"
|
||||||
|
awk -F'\t' '$1=="repos"{printf " %-14s %s repos\n", $2, $3}' "$STATE/meta.tsv"
|
||||||
|
cmd_git missing 2>/dev/null | sed 's/^/ /' || true
|
||||||
|
|
||||||
|
hr "class C — bookmark bar"
|
||||||
|
awk -F'\t' '$1=="bmarks"{printf " %-14s exported %s\n", $2, ""}' "$STATE/meta.tsv"
|
||||||
|
[ -r "$BOOKMARK_SRC" ] && v "live file: $BOOKMARK_SRC"
|
||||||
|
}
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------- fresh ----
|
||||||
|
alert_cmd() {
|
||||||
|
if [ -n "${ALERT_CMD:-}" ]; then printf '%s' "$ALERT_CMD"; return; fi
|
||||||
|
if command -v msmtp >/dev/null 2>&1; then printf 'msmtp -a homehub %s' "$ALERT_TO"
|
||||||
|
elif command -v sendmail >/dev/null 2>&1; then printf 'sendmail -t'
|
||||||
|
else printf 'cat >> "%s/alerts.out"' "$STATE"; fi
|
||||||
|
}
|
||||||
|
alert_send() { # subject body
|
||||||
|
{ printf 'From: %s\nTo: %s\nSubject: %s\n\n%s\n' "${ALERT_FROM:-homehub}" "${ALERT_TO:-root}" "$1" "$2" |
|
||||||
|
sh -c "$(alert_cmd)"; } 2>>"$STATE/alert.err" || warn "alert delivery failed (see $STATE/alert.err)"
|
||||||
|
if [ "${HH_QUIET:-0}" != 1 ] && [ -n "${DISPLAY:-}" ] && command -v notify-send >/dev/null 2>&1; then
|
||||||
|
notify-send -u critical "$1" "$2" || true
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
cmd_fresh() { # the ONLY place that may complain (§10)
|
||||||
|
local dry=0; [ "${1:-}" = "--dry" ] && dry=1
|
||||||
|
local n=$(now) h msgs="" f
|
||||||
|
|
||||||
|
# 1) Self-checks that work with the hub down. Closes the hole where a dead key
|
||||||
|
# means no marker and no heartbeat, i.e. a machine that can never alert.
|
||||||
|
for f in "$STATE"/broken.*; do
|
||||||
|
[ -e "$f" ] || continue
|
||||||
|
local kind=${f##*.} ep msg age already=0
|
||||||
|
ep=$(awk -F'\t' 'NF{print $1; exit}' "$f"); msg=$(cut -f2- "$f")
|
||||||
|
age=$(( n - ${ep:-0} ))
|
||||||
|
[ -e "$STATE/alerted-local.$kind" ] &&
|
||||||
|
[ $(( n - $(date -r "$STATE/alerted-local.$kind" +%s) )) -lt "${REALERT_AGE:-604800}" ] && already=1
|
||||||
|
if [ "$age" -gt "${FRESH_MAX_AGE:-86400}" ] && [ $already = 0 ]; then
|
||||||
|
msgs="$msgs$HOST, local check '$kind': $msg (failing for $(age "$ep"))"$'\n'
|
||||||
|
touch "$STATE/alerted-local.$kind"
|
||||||
|
else
|
||||||
|
v "ok(local): $kind has been failing for $(age "$ep")$([ $already = 1 ] && echo ', already alerted')"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# 2) Hub-side freshness of every machine
|
||||||
|
if ! hub_ok; then
|
||||||
|
v "hub unreachable — cannot judge other machines (the local checks above still ran)"
|
||||||
|
else
|
||||||
|
hub_meta > "$STATE/meta.tsv" || die "cannot read hub metadata"
|
||||||
|
for h in $(awk -F'\t' '$1=="markers"||$1=="alive"{print $2}' "$STATE/meta.tsv" | sort -u); do
|
||||||
|
local m a al rc mka maa
|
||||||
|
m=$(awk -F'\t' -v h="$h" '$1=="markers"&&$2==h{print $3}' "$STATE/meta.tsv")
|
||||||
|
a=$(awk -F'\t' -v h="$h" '$1=="alive"&&$2==h{print $3}' "$STATE/meta.tsv")
|
||||||
|
al=$(awk -F'\t' -v h="$h" '$1=="alerted"&&$2==h{print $3}' "$STATE/meta.tsv")
|
||||||
|
rc=$(awk -F'\t' -v h="$h" '$1=="lastlog"{split($0,x,"\t"); if (x[2]==h) print x[3]}' "$STATE/meta.tsv")
|
||||||
|
mka=$(( n - ${m:-0} )); maa=$(( n - ${a:-$n} ))
|
||||||
|
local alive_now=0; [ "${a:-0}" != 0 ] && [ $maa -lt "${ALIVE_FRESH:-10800}" ] && alive_now=1
|
||||||
|
local netrc=0 x
|
||||||
|
for x in ${NET_RC:-5 255}; do [ "$rc" = "$x" ] && netrc=1; done
|
||||||
|
local realert_ok=1
|
||||||
|
[ "${al:-0}" != 0 ] && [ $(( n - al )) -lt "${REALERT_AGE:-604800}" ] && realert_ok=0
|
||||||
|
|
||||||
|
if [ $alive_now = 1 ] && [ $mka -gt "${FRESH_MAX_AGE:-86400}" ] && [ $netrc = 0 ] && [ $realert_ok = 1 ]; then
|
||||||
|
msgs="$msgs$h (last success $(age "$m"), last rc=${rc:-?})"$'\n'
|
||||||
|
v "STALE: $h — on, but not synced for $(age "$m")"
|
||||||
|
hub "mkdir -p '$HUB_PATH/.hub/meta/alerted' && touch '$HUB_PATH/.hub/meta/alerted/$h'" || true
|
||||||
|
else
|
||||||
|
v "ok: $h (sync $(age "$m"), heartbeat $(age "$a"), rc=${rc:--}$([ $netrc = 1 ] && echo ', off-LAN: silent'))"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ -n "$msgs" ] || return 0
|
||||||
|
if [ $dry = 1 ]; then
|
||||||
|
printf 'homehub: would alert for:\n%s' "$msgs" >&2
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
alert_send "homehub: sync stale on $(hostname -f 2>/dev/null || hostname)" \
|
||||||
|
"These machines are running but have not completed a homehub sync in over $(( ${FRESH_MAX_AGE:-86400} / 3600 )) h:
|
||||||
|
|
||||||
|
$msgs
|
||||||
|
Check with: homehub status
|
||||||
|
Hub: $HUB_HOST:$HUB_PATH/.hub/log.tsv"
|
||||||
|
}
|
||||||
|
|
||||||
|
# ------------------------------------------------------- class B: the repos --
|
||||||
|
git_root_p() { [ -e "$1/.git" ]; }
|
||||||
|
git_row() { # relpath remote branch sha dirty unpushed (last two are ADVISORY ONLY)
|
||||||
|
local d=$1 rel=${1#"$HOME"/} url br sha dirty unpush cred=ok
|
||||||
|
url=$(git -C "$d" remote get-url origin 2>/dev/null || true); [ -n "$url" ] || url=-
|
||||||
|
# SECURITY: remotes can carry credentials (https://user:TOKEN@host/...). They
|
||||||
|
# must never be published to the hub. Strip userinfo and mark the row so
|
||||||
|
# `git clone` can tell you the URL needs re-adding on the far side.
|
||||||
|
case $url in
|
||||||
|
*://*:*@*) url=$(printf '%s' "$url" | sed -E 's#(://)[^/@]*@#\1#'); cred=stripped ;;
|
||||||
|
*://*@*) url=$(printf '%s' "$url" | sed -E 's#(://)[^/@]*@#\1#'); cred=stripped ;;
|
||||||
|
esac
|
||||||
|
br=$(git -C "$d" symbolic-ref --short HEAD 2>/dev/null || echo -)
|
||||||
|
sha=$(git -C "$d" rev-parse --short HEAD 2>/dev/null || echo -)
|
||||||
|
dirty=$(git -C "$d" status --porcelain 2>/dev/null | wc -l | tr -d ' ')
|
||||||
|
unpush=$(git -C "$d" rev-list --count '@{u}..HEAD' 2>/dev/null | tail -1); [ -n "$unpush" ] || unpush=0
|
||||||
|
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\n' "$rel" "$url" "$br" "$sha" "${dirty:-0}" "${unpush:-0}" "$cred"
|
||||||
|
}
|
||||||
|
git_walk() { # dir depth — stops at the first .git, so submodules are not double counted
|
||||||
|
local d=$1 depth=$2 c
|
||||||
|
if git_root_p "$d"; then git_row "$d"; return; fi
|
||||||
|
[ "$depth" -le 0 ] && return
|
||||||
|
for c in "$d"/*/; do [ -d "$c" ] || continue; git_walk "${c%/}" $((depth - 1)); done
|
||||||
|
}
|
||||||
|
cmd_git() {
|
||||||
|
local sub=${1:-inventory}; shift || true
|
||||||
|
local lazy=0 exec=0 a
|
||||||
|
for a in "$@"; do case $a in --lazy) lazy=1;; --exec) exec=1;; esac; done
|
||||||
|
case $sub in
|
||||||
|
inventory)
|
||||||
|
local out=$STATE/repos.$HOST.tsv
|
||||||
|
: > "$out"
|
||||||
|
local r; for r in ${GIT_DIRS:-git src}; do [ -d "$HOME/$r" ] && git_walk "$HOME/$r" 3 >> "$out"; done
|
||||||
|
v "$(grep -c . "$out") repo(s) under ${GIT_DIRS:-git src}"
|
||||||
|
awk -F'\t' '$5+0>0 || $6+0>0 { printf " %-34s %s modified, %s unpushed\n", $1, $5, $6 }' "$out"
|
||||||
|
awk -F'\t' '$7=="stripped" { printf " CREDENTIALS STRIPPED from remote of %s — the far side must supply them\n", $1 }' "$out"
|
||||||
|
v "advisory only — neither the working tree nor unpushed commits ever leave this machine"
|
||||||
|
if [ $exec = 1 ] || [ "${HH_PUSH_INVENTORY:-1}" = 1 ]; then
|
||||||
|
hub "mkdir -p '$HUB_PATH/.hub/gitman' && cat > '$HUB_PATH/.hub/gitman/repos.$HOST'" < "$out" &&
|
||||||
|
v "manifest published to hub"
|
||||||
|
fi
|
||||||
|
;;
|
||||||
|
list)
|
||||||
|
hub "cat '$HUB_PATH/.hub/gitman/repos.'* 2>/dev/null" | sort -u -t$'\t' -k1,1
|
||||||
|
;;
|
||||||
|
missing)
|
||||||
|
hub "cat '$HUB_PATH/.hub/gitman/repos.'* 2>/dev/null" | sort -u -t$'\t' -k1,1 |
|
||||||
|
while IFS=$'\t' read -r rel url br sha dirty unpush cred; do
|
||||||
|
[ -n "${rel:-}" ] || continue
|
||||||
|
[ -e "$HOME/$rel/.git" ] && continue
|
||||||
|
printf '%-34s %s (%s%s%s)\n' "$rel" "$url" "$br" \
|
||||||
|
"$( [ "${dirty:-0}" != 0 ] && printf ', %s modified elsewhere' "$dirty")" \
|
||||||
|
"$( [ "${cred:-ok}" = stripped ] && printf ', creds stripped')"
|
||||||
|
done
|
||||||
|
;;
|
||||||
|
clone)
|
||||||
|
local n=0
|
||||||
|
hub "cat '$HUB_PATH/.hub/gitman/repos.'* 2>/dev/null" | sort -u -t$'\t' -k1,1 |
|
||||||
|
while IFS=$'\t' read -r rel url br sha dirty unpush cred; do
|
||||||
|
[ -n "${rel:-}" ] || continue
|
||||||
|
[ "$url" = - ] && continue
|
||||||
|
[ -e "$HOME/$rel/.git" ] && continue
|
||||||
|
if [ $exec != 1 ]; then
|
||||||
|
printf ' would clone %s <- %s%s%s\n' "$rel" "$url" \
|
||||||
|
"$( [ $lazy = 1 ] && echo ' (blobless)')" "$( [ "${cred:-ok}" = stripped ] && echo ' [creds stripped: set a credential helper or fix the remote]')"
|
||||||
|
else
|
||||||
|
printf ' cloning %s <- %s\n' "$rel" "$url"
|
||||||
|
mkdir -p "$(dirname "$HOME/$rel")"
|
||||||
|
local opts=(--origin origin)
|
||||||
|
[ $lazy = 1 ] && opts+=(--filter=blob:none)
|
||||||
|
[ "$br" != - ] && git clone "${opts[@]}" -b "$br" "$url" "$HOME/$rel" >/dev/null 2>&1 ||
|
||||||
|
git clone "${opts[@]}" "$url" "$HOME/$rel" >/dev/null 2>&1 ||
|
||||||
|
warn "clone failed: $rel ($url)"
|
||||||
|
fi
|
||||||
|
n=$((n+1))
|
||||||
|
done
|
||||||
|
;;
|
||||||
|
*) die "git: unknown subcommand '$sub' (inventory|list|missing|clone)";;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
# ------------------------------------------------- class C: the link bar -----
|
||||||
|
cmd_bookmarks() {
|
||||||
|
local write=0 a
|
||||||
|
for a in "$@"; do [ "$a" = --write ] && write=1; done
|
||||||
|
[ -r "$BOOKMARK_SRC" ] || die "no bookmark file at $BOOKMARK_SRC"
|
||||||
|
cp -- "$BOOKMARK_SRC" "$STATE/bookmarks.$HOST.json" || die "cannot read $BOOKMARK_SRC"
|
||||||
|
local nu; nu=$(jq '[.. | objects | select(.type=="url")] | length' "$STATE/bookmarks.$HOST.json" 2>/dev/null || echo '?')
|
||||||
|
hr "class C: bookmark bar ($HOST)"
|
||||||
|
v "exported $nu url(s) from $BOOKMARK_SRC (the union-merge below covers the bar only)"
|
||||||
|
if [ $write != 1 ]; then
|
||||||
|
v "dry: nothing pushed, nothing merged, live file untouched. Use --write."
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
hub "mkdir -p '$HUB_PATH/.hub/bookmarks' && cat > '$HUB_PATH/.hub/bookmarks/$HOST.json'" < "$STATE/bookmarks.$HOST.json" ||
|
||||||
|
die "could not publish bookmark export"
|
||||||
|
hub "cd '$HUB_PATH/.hub/bookmarks' 2>/dev/null && for f in *.json; do printf '### %s\\n' \"\$f\"; cat \"\$f\"; done" > "$STATE/bookmarks.all.json" ||
|
||||||
|
die "could not fetch exports"
|
||||||
|
"$PY" "$HH_ROOT/lib/merge_bookmarks.py" "$STATE/bookmarks.all.json" > "$STATE/bookmarks.merged.json" ||
|
||||||
|
die "merge failed"
|
||||||
|
local mu; mu=$(jq '.urls' "$STATE/bookmarks.merged.json" 2>/dev/null || echo '?')
|
||||||
|
v "merged union: $mu urls (from $(jq -r '.hosts | join(",")' "$STATE/bookmarks.merged.json" 2>/dev/null))"
|
||||||
|
v "union result: $STATE/bookmarks.merged.json"
|
||||||
|
if pgrep -f "$BOOKMARK_PGREP" >/dev/null 2>&1; then
|
||||||
|
warn "browser is running — merged bar NOT written (it would be overwritten in memory). Close it and re-run."
|
||||||
|
v "merged result is ready at $STATE/bookmarks.merged.json"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
warn "writing the merged bar to $BOOKMARK_SRC is not enabled in v1 — inspect $STATE/bookmarks.merged.json first (see README)"
|
||||||
|
}
|
||||||
|
|
||||||
|
# -------------------------------------------------------------- housekeeping -
|
||||||
|
cmd_forget() {
|
||||||
|
[ $# -ge 1 ] || die "usage: homehub forget <home-relative-path> ..."
|
||||||
|
hub_ok || die "hub unreachable"
|
||||||
|
local p
|
||||||
|
for p in "$@"; do
|
||||||
|
p=${p#/}
|
||||||
|
hub_script "'$HUB_PATH' '$p'" <<'EOS'
|
||||||
|
root=$1; pat=$2
|
||||||
|
cd "$root/.hub/tombstones" 2>/dev/null || exit 0
|
||||||
|
for f in *; do
|
||||||
|
[ -f "$f" ] || continue
|
||||||
|
awk -F'\t' -v p="$pat" '$2 != p && $2 !~ ("^" p "/")' "$f" > "$f.t" && mv "$f.t" "$f"
|
||||||
|
done
|
||||||
|
exit 0
|
||||||
|
EOS
|
||||||
|
v "forget: $p"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
cmd_filter() { gen_filter || return 1; cat "$FILTER"; }
|
||||||
|
|
||||||
|
cmd_setup_key() {
|
||||||
|
[ -f "$HH_KEY" ] && { warn "$HH_KEY already exists"; return 0; }
|
||||||
|
mkdir -p "$(dirname "$HH_KEY")"
|
||||||
|
ssh-keygen -q -t ed25519 -N '' -C "homehub@$HOST" -f "$HH_KEY" || die "keygen failed"
|
||||||
|
hr "deploy key created"
|
||||||
|
v "$HH_KEY (passphrase-less, by necessity: a systemd user unit has no agent)"
|
||||||
|
v "Install it on the hub, then make ~/.ssh 700 there:"
|
||||||
|
printf '\n ssh-copy-id -i %s.pub %s\n\n' "$HH_KEY" "$HUB_HOST"
|
||||||
|
warn "Until that is done, background sync cannot authenticate — and it fails silently by design."
|
||||||
|
}
|
||||||
|
cmd_setup() {
|
||||||
|
hub_ok || die "cannot reach $HUB_HOST with the current ssh setup"
|
||||||
|
hub_setup_dirs
|
||||||
|
mkdir -p "$SHARED_ROOT"
|
||||||
|
v "shared state root ready: $SHARED_ROOT (put app state here, symlink apps to it)"
|
||||||
|
hub "chmod 700 '$HUB_PATH' 2>/dev/null; ls -d '$HUB_PATH/.hub'/*" | sed 's/^/ /'
|
||||||
|
hr "published rules"
|
||||||
|
git -C "$HH_ROOT" rev-parse --is-inside-work-tree >/dev/null 2>&1 &&
|
||||||
|
v "rules live in git: $HH_ROOT — clone that repo on the other machines"
|
||||||
|
[ -f "$HH_KEY" ] || cmd_setup_key
|
||||||
|
}
|
||||||
|
cmd_install() {
|
||||||
|
local d=${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user
|
||||||
|
mkdir -p "$d"
|
||||||
|
cp -v "$HH_ROOT"/systemd/homehub-*.service "$HH_ROOT"/systemd/homehub-*.timer "$d/" || return 1
|
||||||
|
systemctl --user daemon-reload
|
||||||
|
systemctl --user enable --now homehub-sync.timer homehub-fresh.timer
|
||||||
|
systemctl --user --no-pager --full status homehub-sync.timer homehub-fresh.timer 2>&1 | grep -E 'Loaded:|Active:|→' || true
|
||||||
|
}
|
||||||
|
|
||||||
|
usage() { sed -n '2,14p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; }
|
||||||
|
|
||||||
|
case ${1:-help} in
|
||||||
|
sync) shift; cmd_sync "$@";;
|
||||||
|
status) shift; cmd_status "$@";;
|
||||||
|
fresh) shift; cmd_fresh "$@";;
|
||||||
|
git) shift; cmd_git "$@";;
|
||||||
|
bookmarks) shift; cmd_bookmarks "$@";;
|
||||||
|
forget) shift; cmd_forget "$@";;
|
||||||
|
filter) shift; cmd_filter "$@";;
|
||||||
|
setup) shift; cmd_setup "$@";;
|
||||||
|
setup-key) shift; cmd_setup_key "$@";;
|
||||||
|
install) shift; cmd_install "$@";;
|
||||||
|
help|-h|--help) usage;;
|
||||||
|
*) warn "unknown command '${1:-}'; try 'homehub help'"; exit 2;;
|
||||||
|
esac
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
# homehub configuration — sourced by bin/homehub (also readable by systemd units)
|
||||||
|
# All paths are for user `bl` on the Synology hub `nas.lue.dk`.
|
||||||
|
|
||||||
|
HUB_HOST=nas.lue.dk
|
||||||
|
|
||||||
|
# NEVER point this at your Synology home directory. /volume1/homes/bl already
|
||||||
|
# holds 178 G of unrelated legacy data (Documents, Photos, OneDrive, CloudStation,
|
||||||
|
# home_backup, x1c_backup, wg-bo.conf, certs, ...) — pulling from it would merge
|
||||||
|
# years of NAS files into ~/, and a tombstone could reap unrelated files by name
|
||||||
|
# accident. The sync root is a dedicated subtree; .hub/ metadata lives inside it.
|
||||||
|
HUB_BASE=/volume1/homes/bl
|
||||||
|
HUB_PATH=$HUB_BASE/homehub
|
||||||
|
|
||||||
|
# Dedicated, PASSPHRASE-LESS deploy key. Your id_rsa is passphrase-protected and a
|
||||||
|
# systemd *user* unit has no SSH_AUTH_SOCK, so using ~/.ssh/id_rsa here would make
|
||||||
|
# every background sync fail with "Permission denied (publickey)" — forever, and
|
||||||
|
# silently. Run `homehub setup-key` and install the printed pubkey.
|
||||||
|
HH_KEY=${HH_KEY:-$HOME/.ssh/homehub_ed25519}
|
||||||
|
|
||||||
|
# Where this machine keeps homehub's OWN private state (generated filters, markers
|
||||||
|
# cache, inventories, rsync logs, alert spool). NEVER SYNCED — hard-blocked in the
|
||||||
|
# denylist, because syncing it makes machines overwrite each other's filters.
|
||||||
|
STATE_ROOT=$HOME/.local/state/homehub
|
||||||
|
|
||||||
|
# The SHARED state root: the one tree whose contents are shared verbatim between
|
||||||
|
# machines, and the anchor that survives the NixOS migration unchanged (§8).
|
||||||
|
# App state you want everywhere lives in $SHARED_ROOT/<name>/ and is symlinked
|
||||||
|
# to wherever the app expects it. Keep it distinct from STATE_ROOT above: sharing
|
||||||
|
# homehub's own machinery between machines just makes them fight over it.
|
||||||
|
SHARED_ROOT=$HOME/.local/state/shared
|
||||||
|
|
||||||
|
# --- policy knobs -------------------------------------------------------------
|
||||||
|
GRACE_DAYS=30 # tombstone age before the hub copy is reaped (§5)
|
||||||
|
FRESH_MAX_AGE=86400 # alert if a live machine has not synced in 24 h
|
||||||
|
ALIVE_FRESH=10800 # "machine is on" = heartbeat younger than 3 h
|
||||||
|
REALERT_AGE=604800 # at most one alert per host per week
|
||||||
|
TOMB_LIMIT=100 # refuse to apply more deletions than this in one run
|
||||||
|
WARN_FILES=20000 # dry run warns above this many pending changes
|
||||||
|
|
||||||
|
# Where to look for git repos for the class B manifest (DESIGN.md §6).
|
||||||
|
GIT_DIRS="git src"
|
||||||
|
|
||||||
|
# rc values treated as "machine is simply off the LAN" -> silent forever (§10)
|
||||||
|
NET_RC="5 10 11 12 30 35 255"
|
||||||
|
|
||||||
|
# --- alerting ----------------------------------------------------------------
|
||||||
|
ALERT_TO=bl@lue.dk
|
||||||
|
ALERT_FROM=homehub@lue.dk
|
||||||
|
# Empty => auto-detect msmtp, then sendmail, else append to $STATE_ROOT/alerts.out
|
||||||
|
ALERT_CMD=
|
||||||
|
|
||||||
|
# --- class C: the bookmark bar ----------------------------------------------
|
||||||
|
# NOTE the quotes: "Profile 5" contains a space, and an unquoted value here
|
||||||
|
# silently breaks `. config` in every subshell.
|
||||||
|
BOOKMARK_SRC="$HOME/.config/chromium/Profile 5/Bookmarks"
|
||||||
|
BOOKMARK_PGREP='/usr/lib/chromium/chromium'
|
||||||
|
|
||||||
|
# Absolute interpreter: PATH python3 here is /home/bl/.venv/bin/python3.
|
||||||
|
PY=/usr/bin/python3
|
||||||
|
[ -x "$PY" ] || PY=$(command -v python3)
|
||||||
|
|
||||||
|
# rsync 3.1.2 on the NAS: do NOT add --ignore-older (needs >=3.2.3 on both ends).
|
||||||
|
RSYNC_BIN=rsync
|
||||||
|
SSH_TIMEOUT=10
|
||||||
Executable
+142
@@ -0,0 +1,142 @@
|
|||||||
|
#!/usr/bin/python3
|
||||||
|
"""Class C (DESIGN.md §7): union-merge the Chromium bookmark bar across machines.
|
||||||
|
|
||||||
|
Reads a file holding one concatenated JSON bookmark document per machine, flattens
|
||||||
|
every bar to (folder-path, title, url) tuples, takes the union, and rebuilds one
|
||||||
|
tree. Union-by-URL is idempotent, so bookmarks need no conflict concept at all —
|
||||||
|
which is the only reason this file is 60 lines instead of 600.
|
||||||
|
|
||||||
|
Deletions deliberately do NOT propagate, consistent with the tombstone rule for
|
||||||
|
files: edits win, deletes lose. Escape hatch is documented in README.
|
||||||
|
|
||||||
|
stdout: JSON { urls, hosts, dropped, bookmark_bar }
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
def load_docs(text):
|
||||||
|
"""Decode the hub's concatenated exports.
|
||||||
|
|
||||||
|
The hub file is built as `### <host>.json` followed by that host's JSON object,
|
||||||
|
repeated. Returns [(host, doc), ...] so the report can name its sources.
|
||||||
|
"""
|
||||||
|
dec = json.JSONDecoder()
|
||||||
|
out = []
|
||||||
|
for chunk in _split(text):
|
||||||
|
host, body = chunk
|
||||||
|
obj = _one(dec, body)
|
||||||
|
if obj is not None:
|
||||||
|
out.append((host, obj))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _split(text):
|
||||||
|
"""Split into [(host, json_text), ...] on '### name' marker lines."""
|
||||||
|
chunks, host, buf = [], "?", []
|
||||||
|
for line in text.splitlines(keepends=True):
|
||||||
|
if line.startswith("### "):
|
||||||
|
body = "".join(buf)
|
||||||
|
if body.strip():
|
||||||
|
chunks.append((host, body))
|
||||||
|
host = line[4:].strip()
|
||||||
|
if host.endswith(".json"):
|
||||||
|
host = host[:-5]
|
||||||
|
buf = []
|
||||||
|
else:
|
||||||
|
buf.append(line)
|
||||||
|
body = "".join(buf)
|
||||||
|
if body.strip():
|
||||||
|
chunks.append((host, body))
|
||||||
|
return chunks
|
||||||
|
|
||||||
|
|
||||||
|
def _one(dec, body):
|
||||||
|
i, n = 0, len(body)
|
||||||
|
while i < n and body[i] in " \r\n\t":
|
||||||
|
i += 1
|
||||||
|
if i >= n:
|
||||||
|
return None
|
||||||
|
obj, _ = dec.raw_decode(body, i)
|
||||||
|
return obj
|
||||||
|
|
||||||
|
|
||||||
|
def flatten(bar, prefix, out, seen):
|
||||||
|
for node in bar.get("children") or []:
|
||||||
|
ntype = node.get("type")
|
||||||
|
if ntype == "folder":
|
||||||
|
flatten(node, prefix + [node.get("name") or "?"], out, seen)
|
||||||
|
elif ntype == "url":
|
||||||
|
url, title = node.get("url"), node.get("name") or ""
|
||||||
|
key = (tuple(prefix), url)
|
||||||
|
if key in seen: # same bar, same folder, same URL -> one copy
|
||||||
|
continue
|
||||||
|
seen.add(key)
|
||||||
|
out.append({"path": prefix, "title": title, "url": url})
|
||||||
|
# typed_pair / bookmark_bar root itself: ignored
|
||||||
|
|
||||||
|
|
||||||
|
def rebuild(rows):
|
||||||
|
def nodes(prefix):
|
||||||
|
leaves = [
|
||||||
|
{"type": "url", "name": r["title"], "url": r["url"], "date_added": "0"}
|
||||||
|
for r in rows if r["path"] == prefix
|
||||||
|
]
|
||||||
|
names, seen = [], set()
|
||||||
|
for r in rows:
|
||||||
|
if len(r["path"]) > len(prefix) and r["path"][: len(prefix)] == prefix:
|
||||||
|
nm = r["path"][len(prefix)]
|
||||||
|
if nm not in seen:
|
||||||
|
seen.add(nm)
|
||||||
|
names.append(nm)
|
||||||
|
folders = [
|
||||||
|
{"type": "folder", "name": nm, "date_added": "0", "children": nodes(prefix + [nm])}
|
||||||
|
for nm in names
|
||||||
|
]
|
||||||
|
return leaves + folders
|
||||||
|
|
||||||
|
return {"type": "bookmark_bar", "name": "Bookmarks bar", "children": nodes([])}
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
if len(sys.argv) != 2:
|
||||||
|
sys.exit("usage: merge_bookmarks.py <concatenated-bookmark-files>")
|
||||||
|
with open(sys.argv[1], encoding="utf-8", errors="replace") as fh:
|
||||||
|
docs = load_docs(fh.read())
|
||||||
|
if not docs:
|
||||||
|
sys.exit("merge_bookmarks: no JSON documents found")
|
||||||
|
|
||||||
|
rows, seen = [], set()
|
||||||
|
hosts = []
|
||||||
|
for host, d in docs:
|
||||||
|
hosts.append(host)
|
||||||
|
bar = (d.get("roots") or {}).get("bookmark_bar")
|
||||||
|
if bar:
|
||||||
|
flatten(bar, [], rows, seen)
|
||||||
|
|
||||||
|
# collapse identical URLs that sit in different folders, keeping the shallowest
|
||||||
|
byurl = {}
|
||||||
|
for r in rows:
|
||||||
|
cur = byurl.get(r["url"])
|
||||||
|
if cur is None or len(r["path"]) < len(cur["path"]):
|
||||||
|
byurl[r["url"]] = r
|
||||||
|
dropped = len(rows) - len(byurl)
|
||||||
|
final = sorted(byurl.values(), key=lambda r: (r["path"], r["title"].lower()))
|
||||||
|
|
||||||
|
json.dump(
|
||||||
|
{
|
||||||
|
"urls": len(final),
|
||||||
|
"hosts": hosts,
|
||||||
|
"duplicate_urls_collapsed": dropped,
|
||||||
|
"bookmark_bar": rebuild(final),
|
||||||
|
"flat": final,
|
||||||
|
},
|
||||||
|
sys.stdout,
|
||||||
|
indent=1,
|
||||||
|
sort_keys=False,
|
||||||
|
)
|
||||||
|
sys.stdout.write("\n")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
+134
@@ -0,0 +1,134 @@
|
|||||||
|
# homehub denylist — DEFAULT DENY. Emitted FIRST so it beats every set include
|
||||||
|
# (rsync filter rules are first-match-wins).
|
||||||
|
#
|
||||||
|
# Syntax — plain paths, one per line, whole line is the pattern (comments on
|
||||||
|
# their own lines only):
|
||||||
|
# /Foo/ directory at the home root + its whole subtree
|
||||||
|
# build/ any directory named build at ANY depth + its subtree
|
||||||
|
# /file a single file
|
||||||
|
# The trailing slash is what turns it into a subtree rule.
|
||||||
|
#
|
||||||
|
# Measured on g9, 2026-10-24. Everything here is machine-local or rebuildable.
|
||||||
|
|
||||||
|
# --- the big trees: class B owns repos; tmp/Downloads are out by decision ---
|
||||||
|
/git/
|
||||||
|
/src/
|
||||||
|
/tmp/
|
||||||
|
/Downloads/
|
||||||
|
|
||||||
|
# --- toolchains and package caches (rebuilt by the toolchain, never shared) --
|
||||||
|
/.cache/
|
||||||
|
/.cargo/
|
||||||
|
/.rustup/
|
||||||
|
/.npm/
|
||||||
|
/.nvm/
|
||||||
|
/.bun/
|
||||||
|
/.deno/
|
||||||
|
/.yarn/
|
||||||
|
/.espressif/
|
||||||
|
/.platformio/
|
||||||
|
/.conan/
|
||||||
|
/.gradle/
|
||||||
|
/.m2/
|
||||||
|
/go/
|
||||||
|
/zephyr-sdk-0.17.4/
|
||||||
|
/zephyr-sdk-0.17.0/
|
||||||
|
/.dotnet/
|
||||||
|
/.emscripten/
|
||||||
|
/.emscripten_cache/
|
||||||
|
/.dartServer/
|
||||||
|
/.dartium/
|
||||||
|
/.android/
|
||||||
|
/.docker/
|
||||||
|
/.var/
|
||||||
|
/.local/state/homehub/
|
||||||
|
/.local/lib/
|
||||||
|
/.local/bin/
|
||||||
|
/.local/share/virtualenvs/
|
||||||
|
/.venv/
|
||||||
|
/.pixi/
|
||||||
|
|
||||||
|
# --- games / wine ----------------------------------------------------------
|
||||||
|
/.steam/
|
||||||
|
/.steamrt/
|
||||||
|
/.local/share/Steam/
|
||||||
|
/Steam/
|
||||||
|
/.minecraft/
|
||||||
|
/.wine/
|
||||||
|
/.wine_games/
|
||||||
|
/.wine_firefly/
|
||||||
|
/.googleearth/
|
||||||
|
/.adobe/
|
||||||
|
/.freeeciv-placeholder/
|
||||||
|
|
||||||
|
# --- browser / editor / desktop profile state: local by default (§8) --------
|
||||||
|
# The ONLY browser data that crosses machines is the bookmark bar (class C).
|
||||||
|
/.config/
|
||||||
|
/.mozilla/
|
||||||
|
/.mozilla/
|
||||||
|
/.mozilla/
|
||||||
|
/.thunderbird/
|
||||||
|
/.icedove/
|
||||||
|
/.thunderbird/
|
||||||
|
/.thunderbird/
|
||||||
|
/.claws-mail/
|
||||||
|
/.icedove/
|
||||||
|
/.electron/
|
||||||
|
/.electron-vite/
|
||||||
|
/.dbus/
|
||||||
|
/.macromedia/
|
||||||
|
/.mozilla/
|
||||||
|
/.cache/
|
||||||
|
/.nv/
|
||||||
|
/.pki/
|
||||||
|
/.lesshst/
|
||||||
|
/.ssh/agent/
|
||||||
|
/.claude/
|
||||||
|
/.codex/
|
||||||
|
/.cache/
|
||||||
|
|
||||||
|
# --- build artifacts (defence in depth; class B makes most of this moot) ----
|
||||||
|
build/
|
||||||
|
target/
|
||||||
|
dist/
|
||||||
|
out/
|
||||||
|
node_modules/
|
||||||
|
__pycache__/
|
||||||
|
.venv/
|
||||||
|
.pixi/
|
||||||
|
.zig-cache/
|
||||||
|
.ccache/
|
||||||
|
.meson-cache/
|
||||||
|
_deps/
|
||||||
|
CMakeFiles/
|
||||||
|
_results/
|
||||||
|
|
||||||
|
# --- filesystem metadata from other OSes / the NAS itself -------------------
|
||||||
|
# @eaDir is Synology's per-directory metadata; without this rule a pull litters
|
||||||
|
# an @eaDir directory into every synced directory it touches.
|
||||||
|
@eaDir/
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
desktop.ini
|
||||||
|
|
||||||
|
# --- build/packaging output that lives at the home root ---------------------
|
||||||
|
# ~/debian is a dpkg work area: 150 M of .deb/.dsc/.orig.tar.xz and extracted
|
||||||
|
# upstream trees. It was 86% of the first dry-run push list — build output, not
|
||||||
|
# personal data. Anchored so it only means ~/debian, not every debian/ subdir.
|
||||||
|
/debian/
|
||||||
|
|
||||||
|
# --- junk, locks, transient -----------------------------------------------
|
||||||
|
.gksu.lock
|
||||||
|
.gconfd.lock
|
||||||
|
/core
|
||||||
|
/lost+found/
|
||||||
|
*.tmp
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
*.bak
|
||||||
|
*.backup
|
||||||
|
*~
|
||||||
|
*.log
|
||||||
|
*.lock
|
||||||
|
*.part
|
||||||
|
*.crdownload
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
# host: TEMPLATE for the laptop. Copy to rules/hosts/<hostname> on that machine
|
||||||
|
# (or edit this repo there, since rules/ itself travels in git).
|
||||||
|
# Deliberately NOT including: media, misc, and everything under the denylist.
|
||||||
|
base
|
||||||
|
docs
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
# host: g9 (desktop, always at home, no disk-space pressure)
|
||||||
|
# One set name per line; must match a file in rules/sets.d/
|
||||||
|
base
|
||||||
|
docs
|
||||||
|
media
|
||||||
|
misc
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# set: base — the minimum every machine (laptop included) syncs both ways.
|
||||||
|
# Plain home-relative paths; trailing slash = directory subtree.
|
||||||
|
# Remember: denylist wins, so anything under /.config/ can never appear here (§8).
|
||||||
|
|
||||||
|
# identity + credentials (accepted in decision D1: the hub then holds every key
|
||||||
|
# you own, which is why the hub copy is forced 700/600 and the NAS must stay LAN-only)
|
||||||
|
/.ssh/
|
||||||
|
/.gitconfig
|
||||||
|
/.gitignore_global
|
||||||
|
/.hgrc
|
||||||
|
|
||||||
|
# shell / terminal
|
||||||
|
/.bashrc
|
||||||
|
/.bash_profile
|
||||||
|
/.bash_logout
|
||||||
|
/.profile
|
||||||
|
/.inputrc
|
||||||
|
/.screenrc
|
||||||
|
/.tmux.conf
|
||||||
|
/.p10k.zsh
|
||||||
|
/.vimrc
|
||||||
|
/.alsoftrc
|
||||||
|
/.gtk-bookmarks
|
||||||
|
|
||||||
|
# personal scripts
|
||||||
|
/bin/
|
||||||
|
|
||||||
|
# pi agent STATE only — never install/ bin/ npm/ (§8: state vs config vs cache)
|
||||||
|
/.pi/agent/sessions/
|
||||||
|
/.pi/agent/settings.json
|
||||||
|
/.pi/agent/models.json
|
||||||
|
/.pi/agent/models-store.json
|
||||||
|
/.pi/agent/AGENTS.md
|
||||||
|
/.pi/agent/extensions/
|
||||||
|
/.pi/agent/prompts/
|
||||||
|
/.pi/agent/skills/
|
||||||
|
|
||||||
|
# homehub's shared state root: the ONE directory whose contents are shared
|
||||||
|
# verbatim, and the anchor that survives the NixOS migration unchanged (§8).
|
||||||
|
# It is deliberately NOT ~/.local/state/homehub — that is homehub's own private
|
||||||
|
# scratch (filters, markers, error logs), and syncing it makes machines fight.
|
||||||
|
/.local/state/shared/
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
# set: docs — real documents. On the laptop too (decision D4).
|
||||||
|
/Documents/
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
# set: media — desktops only for now; 323 M and grows sideways.
|
||||||
|
/Pictures/
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
# set: misc — small hand-written trees that are not git repos.
|
||||||
|
# Anything here that later becomes a repo must be REMOVED from this set and picked
|
||||||
|
# up by `homehub git inventory` instead (§6).
|
||||||
|
/journal/
|
||||||
|
/projects/
|
||||||
|
/mu_code/
|
||||||
|
/public_html/
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=homehub freshness check: alerts only when a live machine has not synced for 24 h
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
# HH_QUIET is deliberately NOT set: the ok/stale lines belong in the journal, and
|
||||||
|
# alert_send() uses DISPLAY+notify-send to reach the desktop when there is one.
|
||||||
|
ExecStart=%h/homehub/bin/homehub fresh
|
||||||
|
TimeoutStartSec=5min
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=homehub freshness check (half-hourly; the only thing that may complain)
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnBootSec=2min
|
||||||
|
OnUnitActiveSec=30min
|
||||||
|
RandomizedDelaySec=2min
|
||||||
|
AccuracySec=1min
|
||||||
|
Persistent=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=homehub home-drive sync (rsync to NAS hub, mtime-arbitrated)
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
Conflicts=sleep.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
# A background home sync must never make the machine stutter.
|
||||||
|
Nice=19
|
||||||
|
IOSchedulingClass=idle
|
||||||
|
# Silent on success is a requirement, not an oversight (DESIGN.md §10).
|
||||||
|
Environment=HH_QUIET=1
|
||||||
|
ExecStart=%h/homehub/bin/homehub sync --exec
|
||||||
|
TimeoutStartSec=3h
|
||||||
|
# No retry storm: the next timer tick is the retry.
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=homehub home-drive sync cycle (every 10 min, catches up after suspend)
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnBootSec=5min
|
||||||
|
OnUnitActiveSec=10min
|
||||||
|
RandomizedDelaySec=3min
|
||||||
|
AccuracySec=1min
|
||||||
|
Persistent=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
Reference in new issue
Block a user