fix(memory-sync): only queue failures we attribute to the remote#68
Merged
Merged
Conversation
…short shas Three follow-ups from the activation run (PR #63). AGENT_MEMORY_SYNC_REACHABILITY_CHECK_COMMAND had two silent failure modes. `false` parsed as a JSON boolean, then fell through a falsy check and returned null, so the default ssh probe ran while the operator believed they had disabled it. A genuinely malformed value threw an uncaught SyntaxError that took the whole CLI down. Both now print one visible warning naming the offending value and the expected shape, and fall back to the default probe. `restore <short-sha>` failed with "could not fetch ref", because git only serves full object names to remotes. Restore now resolves locally first via git rev-parse; the working copy already fetches full branch history, so the common case costs no extra round trip. When a short-looking sha stays unresolved both locally and remotely, the error says to use the full 40-character sha. The watch offline behaviour needed no production change: since PR #65 watch goes through performPush, which already commits locally and skips the push with one log line when the remote is unreachable, while genuine config and data errors still crash loud for the supervisor. Only the fail-loud half was untested and the watch plist template still documented the opposite. Both fixed. Note for both machines: rebuild after merge, and restart the watch and sync launchd services so they pick up the loader change. Refs: 1b63070d-9ea1-4a38-bba0-e58a4678b596
Review found the supervisor contract was asserted, not implemented.
performPush's catch had no classification and no rethrow, so every error
raised inside the try block became "remote unavailable; queued" with exit 0:
ENOSPC and EACCES while writing the working copy, a failing commit, a broken
git config, prepareWorkingCopy and StateStore failures alike. Under launchd
that is the worst possible shape. A machine with a full disk logs a benign
queued line every tick, exits 0, is never respawned by KeepAlive, and never
syncs again. Invisible by construction.
Adds RemoteUnavailableError, thrown from lookupRemoteHead and push.
performPush rethrows anything else. Discriminating on exitCode was not viable:
the generic git fallback also throws exitCode 4 for any failing subcommand, so
the code does not carry the information. Matching on message text was rejected
as revert-sensitive.
Note precisely what the boundary is, since it is a choice and not a discovery.
prepareWorkingCopy also fetches, and that call is equally remote-attributable
in principle. It is deliberately NOT attributed, because a fetch failure can
just as easily be local, a full disk writing the packfile. So a transient drop
in the window after ls-remote succeeds now crashes rather than queues. That is
bounded: a sustained outage never reaches the fetch, because the reachability
precheck queues first, and schemes the precheck waves through are still caught
by ls-remote. It fails loudly and restartably, which is the safe direction.
The boundary test uses a stub git binary that forwards everything to real git
except commit, which exits 17, so a non-network failure is raised inside the
try window after the working copy is already prepared against a reachable
remote. Confirmed red before this change: exit 0 with the false "remote
unavailable" note.
Also from review: the docs named a class ("a config/data error") that was only
true of one example, so README, machine-setup and the watch plist header now
name the mechanism instead. restore captures the resolved full sha rather than
discarding it, so --output json reports the commit actually restored from
instead of the abbreviation typed. resolveLocalCommit no longer advertises
branch and tag input it was never safe for; the hex precondition enforced in
restore is now stated at the method. A test title that contradicted its body
is fixed, with the element-shape case it hinted at added, plus a test pinning
empty-as-unset. The README documents that convention and, answering the
original incident, how to actually disable the probe.
Note for both machines: rebuild and restart the watch and sync services after
merge.
Refs: 1b63070d-9ea1-4a38-bba0-e58a4678b596
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes task
1b63070d.The finding that mattered
The task's AC1 said non-network errors must still crash loud, so launchd
respawns. The code did not guarantee that.
performPush's catch had noclassification and no rethrow, so everything raised inside the try block
became
"remote unavailable; queued"with exit 0: ENOSPC and EACCES whilewriting the working copy, a failing commit, a broken git config,
prepareWorkingCopyand StateStore failures alike.On a real machine that is the worst possible shape. A full disk logs a benign
queued line every tick, exits 0, is never respawned by KeepAlive, and never
syncs again. Invisible by construction.
The reviewer offered a waiver. I declined it — this runs on both machines
against the real memory remote, and shipping AC1 as met would have been a false
claim about a production safety property.
The fix
RemoteUnavailableError, thrown fromlookupRemoteHeadandpush.performPushrethrows anything else. Discriminating on exitCode was notviable: the generic git fallback also throws exitCode 4 for any failing
subcommand, so the code does not carry the information. Message matching was
rejected as revert-sensitive.
Be precise about the boundary, because it is a choice, not a discovery.
prepareWorkingCopyalso fetches, and that is equally remote-attributable inprinciple. It is deliberately not attributed, because a fetch failure can
just as easily be local (a full disk writing the packfile). So a transient drop
in the window after
ls-remotesucceeds now crashes rather than queues.Measured with a stub git: status 0 before, status 4 after.
That is bounded and safe: a sustained outage never reaches the fetch, because
the reachability precheck queues first, and schemes the precheck waves through
are still caught by
ls-remote(verified with an unreachable https remotequeueing cleanly at exit 0). It fails loudly and restartably.
Verification
The boundary test uses a stub git binary forwarding everything to real git
except
commit, forced to exit 17, raising a non-network failure inside thetry window after the working copy is prepared against a reachable remote.
Confirmed red before the fix — exit 0 with the false "remote unavailable"
note — and the reviewer re-confirmed by stripping the guard from a scratch copy
and getting exactly that failure back.
107 tests, 106 pass. Coverage 94.66 / 75.10 / 93.33 against the 86/65/86 gate.
Also in this PR
example. README, machine-setup and the watch plist header now name the
mechanism.
restorecaptures the resolved full sha, so--output jsonreports thecommit actually restored from rather than the abbreviation typed.
resolveLocalCommitno longer advertises branch/tag input it was never safefor; the hex precondition enforced elsewhere is now stated at the method.
hinted at is added, and empty-as-unset is pinned and documented — answering
the original incident, where an operator tried to disable the probe and there
was no documented way to do it.
Accepted waiver
AC4's "suite green" is not met and is not restated as satisfied anywhere.
The pre-existing
cross-machine-profilestest fails on a hardcoded/Users/lannguyensipath from a committed profile; confirmed byte-identical onmaster and untouched here. Tracked as
112a0864.After merge
Rebuild and restart the watch and sync launchd services on both machines.
Note this checkout is what the live
com.agent-memory-sync.watch.macbookservice runs from, and its
dist/currently holds this branch's build.Follow-up filed
11424b5e— the mirror image of the fixed bug still exists: a permanentlymisconfigured remote is correctly classified and therefore queues silently
forever at exit 0, indistinguishable from a laptop on a plane. Also pins the
deliberate fetch boundary, which no test currently guards.