cli: day-1 bootstrap helpers — config check, agent cert, node pair, plaintext crypt#171
Merged
Merged
Conversation
Crypt `password`/`password2` are now plaintext by default (a literal or
{ env = "VAR" }); squirrel obscures them into rclone's on-disk form at
load time, retiring the manual `rclone obscure` step. The obscure helper
matches rclone's format (AES-CTR under rclone's public key, base64url)
but derives the IV deterministically from the plaintext so the rendered
rclone.conf stays byte-stable — the "unexpected rewrite is a signal"
invariant WriteRcloneConfig relies on. Determinism costs nothing here:
obscure is obfuscation, not encryption (the key is public).
Breaking change: a config whose values are already obscured must add
`obscured = true` to the crypt block to keep them verbatim, or reveal
them back to plaintext. Docs and the existing crypt tests carry the new
marker.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
`squirrel agent cert` generates an ECDSA self-signed cert+key at the configured [agent.tls] paths and prints the sha256: pin ready to paste into peers' [nodes.X.tls] cert_fingerprint. It refuses to clobber an existing cert without --force, since regenerating changes the fingerprint and breaks every peer that pinned it. The agent also logs its own cert fingerprint on the startup line, so an operator can read the pin straight from the log. The pin is computed the same way the peer-sync client verifies it (SHA-256 over the leaf DER), via a shared agent.FingerprintCertFile helper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
`squirrel config check` parses and resolves the whole config (env vars included), stats every volume path and node byte-path, validates offload_requires against target capabilities, and prints the affirmative "N volumes, M destinations, K nodes — all resolvable" summary. It is strictly read-only (a question, per ux-principle 2): it never opens the store or creates a path. Missing/empty volume paths (F4/F8), missing node byte-paths (F34), and structurally-unsatisfiable offload targets — a mirror-layout crypt destination (F21) — are surfaced; problems exit non-zero, advisory warnings do not. `squirrel node pair <peer>` generates the two bearer tokens a bidirectional peer relationship needs and emits the matching config halves for both machines, with each token already placed in the two slots that must agree — killing the F3 four-binding token-matrix error class. It fills this node's endpoint/fingerprint where known and leaves clearly-marked placeholders for the peer-side facts it cannot know. It prints only; it never edits either config. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Follow the friction log's own convention of striking closed entries with a PR reference: F1 (agent cert + startup fingerprint), F2 (plaintext crypt), F3 (node pair), F4 (config check). F34 is only partially closed — config check surfaces a missing node byte-path, but load-time validation is still absent — so it is annotated, not struck. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
There was a problem hiding this comment.
Pull request overview
Adds “day-1 bootstrap” CLI helpers and config ergonomics to reduce setup friction around config validation, agent TLS pinning, node pairing, and crypt secret handling.
Changes:
- Add
squirrel config checkfor read-only, full config resolution + path/policy validation with an affirmative summary. - Add
squirrel agent certand log the agent TLS fingerprint at startup for easy peer pinning. - Add
squirrel node pair <peer>to emit both sides of a bidirectional peer relationship (tokens/endpoints/fingerprints) and switch cryptpassword/password2to plaintext-by-default with anobscured = truemigration marker.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| sync/rclone_test.go | Updates test config fixture to include crypt.obscured = true. |
| README.md | Documents plaintext-by-default crypt secrets and the obscured migration marker. |
| docs/src/content/docs/reference/configuration.md | Updates reference docs for crypt secrets + obscured flag. |
| docs/src/content/docs/layouts/encrypted.md | Updates encrypted layout docs and adds migration note for already-obscured configs. |
| config/obscure.go | Implements rclone-compatible “obscure” encoding with deterministic IV derivation. |
| config/obscure_test.go | Adds round-trip/determinism tests and an optional cross-check against rclone reveal. |
| config/destinations.go | Resolves crypt secrets (plaintext→obscured) and supports crypt.obscured passthrough. |
| config/crypt_plaintext_test.go | Tests plaintext→obscured behavior and obscured = true migration behavior. |
| config/config.go | Updates Crypt struct docs to reflect plaintext-by-default + obscured behavior. |
| config/config_test.go | Updates config tests/fixtures to include obscured = true where needed. |
| cmd/squirrel/root.go | Registers new top-level config and node command groups. |
| cmd/squirrel/node.go | Adds squirrel node parent command. |
| cmd/squirrel/node_pair.go | Implements squirrel node pair <peer> which prints both config halves with tokens/placeholders. |
| cmd/squirrel/node_pair_test.go | Tests node pairing output shape, token cross-match, and fingerprint embedding. |
| cmd/squirrel/config.go | Adds squirrel config parent command. |
| cmd/squirrel/config_check.go | Implements squirrel config check validation/statting + summarized outcome. |
| cmd/squirrel/config_check_test.go | Tests config check success/warn/fail paths and “read-only” invariant. |
| cmd/squirrel/agent.go | Adds agent cert subcommand and logs cert fingerprint on startup when TLS is enabled. |
| cmd/squirrel/agent_cert.go | Implements squirrel agent cert to generate cert/key and print pasteable pin snippet. |
| cmd/squirrel/agent_cert_test.go | Tests cert generation, refusal to overwrite, --force, and missing TLS config behavior. |
| agent/cert.go | Adds self-signed cert generation + fingerprinting helpers used by CLI and server logging. |
| agent/cert_test.go | Tests cert/key validity, fingerprint correctness/shape, uniqueness, and file fingerprinting. |
| agent/agent.go | Adds Server.CertFingerprint() accessor used for startup logging. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- config/obscure_test.go: the IV-uniqueness check now base64-decodes the obscured values and compares the actual leading IV bytes (via a new obscuredIV helper) instead of a base64 string prefix, and checks the obscureRclone errors so a regression can't hide behind zero values. - agent/cert_test.go: TestGenerateSelfSignedCertUnique now checks the errors from GenerateSelfSignedCert rather than comparing zero-value fingerprints on a silent failure. - config/destinations.go: cryptObscuredFlag's error is wrapped with the "crypt:" prefix at the call site, matching the other resolveCrypt errors (unknown field, secret resolution); the test asserts the prefixed message. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Make the HTTP listener optional. When `[agent] listen` is absent/empty the agent runs its background schedulers (index/sync/audit cadences) WITHOUT binding an HTTP server — the mode for cadence-only machines (a roaming laptop that pushes to a hub and never receives peer syncs). When `listen` is set, behaviour is unchanged: scheduler + HTTP server, token required. - config: `listen` no longer required; a bearer token is required only when a listener is configured (nothing to authenticate otherwise). - agent: validateConfig relaxed accordingly; new Server.RunSchedulers runs the scan + cadence loops with the same shutdown discipline as Serve but no listener. - cmd: runAgent dispatches to RunSchedulers when listen is empty, and fails fast if a listener-less agent has no cadence and no scan (silent idling is degradation, not a valid config); the marker/init boundary is untouched — the agent still never escalates. - docs: reference-setup laptop drops the dummy listen + token; agent guide and configuration reference describe listener-less operation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Strike the listener-less friction entry now that the cadence-only agent mode lands, per the friction log's PR-reference convention. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Add ResetDestination: an operator-driven verb that forgets a destination's recorded remote state (remote_objects, remote_packs, the durability vector, and push-freshness) so the next sync treats it as fresh and re-uploads. It is the supported recovery for a wrecked or repointed destination, replacing hand SQL or a cross-machine rename. The reset is audit-preserving: the runs table and the append-only destination_run_ids_history advance log are untouched, and the content/files rows are never modified. The reset itself is recorded as a kind='audit' run (reusing the destination-scoped audit shape, so no schema change) with a new free-text 'reset-destination' runs_audit transition carrying the cleared counts. Everything runs in one transaction — a reset is all-or-nothing. CountDestinationRecordedState backs the CLI's dry-run preview. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
The packed and content-addressed layout guards keyed on run history by destination name: a recorded success whose manifest segment / pack map is absent was refused outright, so following the guard's own advice (a fresh root) still refused. Now, when the marker is missing, the guard probes the configured root; an empty root — a wiped or repointed destination, or one cleared by `squirrel destination reset` — is treated as a fresh start and re-uploaded. A non-empty root, or any error probing it, keeps the refusal (fail-closed). The refusal message now also points at `destination reset`. Adds Rclone.remoteRootEmpty (recursive lsf) and a shared freshStartOnEmptyRoot helper. The fake-rclone test shim learns `lsf`; the mirror-era guard tests now seed a remote file so they exercise a genuine layout conflict, and new tests cover the empty-root fresh start on both layouts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
`squirrel destination reset <name>` is a weighty change command: it prints
what it will clear, refuses without --yes, and offers --dry-run — mirroring
ux-principle 2. Parent `destination` command and the `reset` subcommand live
in separate files. A configured-but-unrecorded destination reports "nothing
to reset" rather than minting an empty run.
restore's node dead-end ("restore from node destinations is not supported")
now points at the machine-replacement runbook — rebuilding an edge machine
from its hub is a reverse peer push driven from the hub, not a restore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
New "Recovery & disaster runbooks" guide: resetting a wrecked destination, rebuilding a dead machine from its hub via a reverse peer push (re-pairing with `node pair`), and the manual DR that already works and stays — mirror restore, ride-along index-snapshot recovery, and packed/content-addressed recovery. Adds `squirrel destination reset` to the CLI reference and the guide to the sidebar. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
- config/agent.go: the "token required" error now states the requirement is conditional on a listener — "auth.token is required when [agent] listen is set (... omit listen for a listener-less, scheduler-only agent)" — instead of the unconditional "no agent without authentication", which read as wrong now that listener-less mode exists. - cmd/squirrel/agent.go: the listener-less "nothing to run" fail-fast now lists hook.interval alongside sync_every/index_every/scan_interval, so the remediation matches the real scheduling rules (agentHasWork and the scheduler both count an interval hook as work). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
FIX 1 (fail-closed): remoteRootEmpty no longer reads any lsf error with empty stdout as an empty root — runPlain carries stderr only in the error, so an auth/network failure was being mistaken for a wiped destination and letting the layout guard proceed. Now only rclone's canonical "directory not found" (or "object not found") counts as a fresh root; every other error surfaces so the guard fails closed. New test injects an auth-style lsf error (guard refuses) vs a not-found (fresh root). FIX 2: the restore node-restore signpost pointed at a repo source path (docs/guides/recovery) that doesn't exist for an installed binary; it now names the published docs route (guides/recovery). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
F35 made Listen optional and Token conditional on Listen, but three doc comments on the public Config/Agent structs still declared them unconditionally required, contradicting the relaxed validateConfig / resolveAgentAuth. Correct the comments so the documented contract matches the validated one. Doc-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Textual conflicts:
- cmd/squirrel/root.go: keep this branch's config/node/destination
commands alongside main's `squirrel version`.
- cmd/squirrel/agent.go: keep the startup cert-fingerprint attributes (F1)
on top of main's `version` var, which replaced this branch's local
`agentVersion` const.
- sync/{content_addressed,packed}_test.go: keep both sides' tests — this
branch's F20 fresh-start cases and main's dry-run previews (which
replaced the dry-run-refused cases), plus both sets of fake-rclone env
resets.
Semantic conflicts the merge exposed:
- Duplicate rclone-obscure helpers: main's config/rclone_obscure.go
(#153) and this branch's config/obscure.go declared the same symbol.
Consolidated on main's rcloneObscure — it is the merged, golden-pinned
implementation — and repointed resolveCryptSecret at it (it can no
longer fail, so the error path went away). Ported this branch's unique
coverage, the cross-check against a real `rclone reveal`, into main's
test file; the IV-distinctness assertion could not come along, as main
deliberately fixes the IV at zero and documents that trade-off.
- The remote-marker gate (#150/#170) writes .squirrel-volume under the
destination root, which made this branch's "empty root means fresh
start" probe (F20) unreachable: a wiped root is re-bootstrapped with a
marker before any layout guard runs, so it never looks empty again.
remoteRootEmpty now skips volume markers — they are squirrel's own
bookkeeping, not content. Anything else still reads as non-empty and
keeps the guard's refusal. Main's two dry-run layout-flip tests now
seed mirror-era bytes, the same adaptation this branch already applied
to their non-dry-run siblings, so they still describe a genuine layout
conflict rather than a wiped destination.
- The fake rclone shim's `lsf` returned underlying names through a crypt
overlay; real rclone reports decrypted ones. It now strips the data
suffix, so a name-matching caller sees what rclone would.
- F21 is implemented on both sides: main rejects an unsatisfiable
offload_requires at config load, ahead of `config check`'s own
classification. Its test now pins the surviving contract — name the
target, explain it can never gate, exit non-zero.
go vet, go test ./..., and gofmt are clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQUzRmQHqbP2kV9kFrbBHL
mbertschler
pushed a commit
that referenced
this pull request
Jul 24, 2026
main now carries #171, so its bootstrap-helper and fresh-start work meets this branch's refusal/audit work. - sync/content_addressed.go, sync/packed.go: the layout guards had both changed — main added the fresh-start-on-empty-root recognition and the `destination reset` hint, this branch wrapped the refusal in ErrRefused so it records a terminal 'refused' run. Composed: an empty root is still a fresh start, and a genuine layout conflict refuses with both the reset hint and the sentinel. - store/runs_audit.go, cmd/squirrel/agent.go: both sides appended at the same spot — keep this branch's abort/alarm/contested transitions and the startup reaper alongside main's reset-destination transition and its listener-less helpers. The reaper still runs in runAgent ahead of serveAgent, so it covers both the HTTP and the new listener-less path. go vet, go test ./..., and gofmt are clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQUzRmQHqbP2kV9kFrbBHL
mbertschler
pushed a commit
that referenced
this pull request
Jul 24, 2026
main now carries #171 and the listener-less agent mode (#175), which is where this branch's two new cadences had to be reconciled. Textual conflicts: - store/runs_audit.go, agent/agent.go: both sides appended at the same spot — keep this branch's pull-durability transition and its VerifyRunner/DurabilityPuller types alongside main's reset-destination transition; main rewrote the Config doc comment for listener-less mode, so its version of that comment stands. - docs reference: both sides added to the [agent] section. Kept main's listener-less paragraph and this branch's key tables, and corrected the table to match — `listen` is no longer required, and `auth.token` is required only alongside it. Left as-is the two would have contradicted each other in adjacent paragraphs. Semantic conflict: main's listener-less start gate (cmd's agentHasWork) duplicated the scheduler's volume-cadence logic and predated this branch's verify_every / pull_durability_every, so it would have refused to start a listener-less agent whose only work is one of those cadences — exactly the receive-only htpc in design/reference-setup.md that F33 exists to serve. Replaced the duplicate with an exported agent.ScheduledWorkInConfig, built on the same resolvers the scheduler's own anyScheduledWork gate uses, so the two cannot drift again; the refusal message now names every cadence. Covered by a table test including the receive-only pull-only config. go vet, go test ./..., and gofmt are clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQUzRmQHqbP2kV9kFrbBHL
This was referenced Jul 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bootstrap-day helpers from #162 — the friction-densest stretch of the reference-setup walk (F1–F4, F34). Four of the five scoped items land here; listener-less agent mode (F35) is deferred to a small follow-up PR so this one stays reviewable.
Changes
squirrel config check(F4, F8, F34, F21) — parses and resolves the whole config (env vars included), stats every volume path and node byte-path, validatesoffload_requiresagainst target capabilities, and prints the affirmativeN volumes, M destinations, K nodes — all resolvable. Strictly read-only (a question per ux-principle 2): it never opens the store or creates a path. Missing/empty volume paths, missing node byte-paths, and structurally-unsatisfiable offload targets are surfaced; problems exit non-zero, advisory warnings do not.squirrel agent cert(F1) — generates an ECDSA self-signed cert+key at the configured[agent.tls]paths and prints thesha256:pin ready to paste into peers'[nodes.X.tls]. Refuses to clobber an existing cert without--force(regenerating changes the fingerprint and breaks every peer that pinned it). The agent now also logs its own fingerprint on the startup line.squirrel node pair <peer>(F3) — see decision below.node pairshapeThe issue left "exact shape to be designed in the PR". I implemented the simplest defensible shape:
squirrel node pair <peer>emits the matching config halves for both sides of the relationship. It generates the two bearer tokens a bidirectional pair needs and places each in the two slots that must agree ([nodes.X].auth.beareron one side ≡[agent.auth.peers.X].beareron the other), so the four-binding token matrix is correct by construction — killing the F3 error class. This node's endpoint/fingerprint are filled where known; peer-side facts it cannot know (peer endpoint/path/fingerprint, and where the peer mounts this node) are clearly-marked<...>placeholders, fillable via--peer-endpoint/--peer-path/--peer-fingerprint/--local-endpoint. It prints only — it never edits either config, so the operator reviews and pastes.Alternatives not taken (heavier): writing directly into config files, or a stateful interactive handshake between two running agents. Please confirm this print-both-halves shape before it sets precedent.
Crypt
password/password2are now plaintext by default (a literal or{ env = "VAR" }); squirrel obscures them into rclone's on-disk form at load time, retiring the manualrclone obscurestep. A config that already holds obscured values will double-obscure and fail to decrypt. Migration, documented indocs/.../encrypted.mdand the README:obscured = trueto thecryptblock (a marker key) to render them verbatim.The obscure helper matches rclone's format (AES-CTR under rclone's public key, base64url) but derives the IV deterministically from the plaintext, so the rendered
rclone.confstays byte-stable and the "unexpected rewrite is a signal" invariant inWriteRcloneConfigholds. Determinism costs nothing: obscure is obfuscation, not encryption (the key is public). Cross-checked against realrclone revealin a (rclone-gated) test. This reconciles with the parallel #153 obscure work — expect a merge reconciliation if both land.Testing
go vet ./...,go test ./...,golangci-lint run(v2.12.2) — all clean.obscuredcrypt resolution; cert generation + fingerprint identity with the peer-sync verifier;config checkaffirmative/empty/missing/offload-unsatisfiable/read-only;agent certgenerate/refuse-overwrite/force/no-tls;node pairboth-halves cross-match + placeholders + fingerprint embedding.Scope
Addresses #162 (F1–F4, F34). Listener-less agent mode (F35) follows in a separate PR referencing #162, so this PR intentionally does not close the issue.
🤖 Generated with Claude Code
https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Generated by Claude Code