Skip to content

Finding #10: bulk post-sync completion snapshot (vendored blob rebuild) - #22

Merged
bcourbage merged 6 commits into
mainfrom
fix/finding10-sync-snapshot
Jul 20, 2026
Merged

Finding #10: bulk post-sync completion snapshot (vendored blob rebuild)#22
bcourbage merged 6 commits into
mainfrom
fix/finding10-sync-snapshot

Conversation

@bcourbage

@bcourbage bcourbage commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Draft — for review, do not merge. Finding #10 (fast, safe sync completion), integrated on top of the cumulative main (PRs #16#21) at 968f4ae. Includes the authorized vendored-blob bump a57f5c4e → 2f345306 and OCaml patch 0005.

Problem

finalizeSyncUI made O(n) per-row unison_bridge_ri_get_details bridge calls on the main thread at sync completion (a completion freeze on large syncs); per-row progress used an O(n) node walk.

Fix

  • OCaml patches/0005syncComplete carries the final post-sync stateItem array (!theState after do_unisonSynchronize). No new OCaml allocation; C reads final values via the already-registered accessors while the array is GC-rooted.
  • C (transactional, shared deliver_sync_snapshot) — resolve accessors and marshal ONE complete snapshot (progress + details + bytes) with the exn-safe wrappers; any accessor raise / OOM / bad shape aborts before publishing → ok=false, never partial, never "no failures". ABI guards: Is_block/tag-0 before Wosize_val, size_t→int bound; the Swift trampoline rejects ok+count>0+null-rows.
  • CoordinatorSyncResults .available/.unavailable → effects .presentSyncResults(SessionID, snapshot) / .presentSyncUnavailable(SessionID, reason:), bound to the exact pending (SessionID, OperationID), lease released once, and NEITHER entering restartRequired (engine is quiescent post-commit).
  • SwiftfinalizeSyncUI(snapshot:) applies once on main via the pure SyncCompletionModel (identical details-based attribution, zero bridge calls); a count mismatch routes to finalizeSyncUnavailable. The per-row getter loop is gone; an O(1) rowToNode index is rebuilt atomically with items/tree.
  • Results-unavailable gate — a first-class ReconcileActionGate state blocks every engine-reaching action (Direction/Revert, Ignore incl. its phase-independent path, Diff incl. canDiff, Details, Go/Sync) at both validation and method boundaries; only Rescan/Profiles/Quit remain. finalizeSyncUnavailable shows a dedicated orange warning (not the green success check).

Cumulative integration (PRs #16#21 preserved)

Merged current green main via a normal merge. The two conflicts were resolved preserving both sides:

Validation (at merge 2ab70c7; final head 0aef318 adds a README-only fix)

  • make build (Debug) + make build CONFIG=Release succeed.
  • Full cumulative suite 674 tests, 0 failures (1 pre-existing env-gated skip).
  • Real local synchronization (test_h_realSync_propagatesFiles_andDeliversSnapshot): the REAL public path — unison_bridge_synchronize()do_unisonSynchronizesyncComplete !theState (patch 0005's real caller, not the Debug seam) → C marshaller → Swift handler — over conflict-free one-sided files: one ok=true snapshot, count == scanned, plausible fields, files actually propagated on disk. ✅
  • PR L1: app-global diff-request broker (owner + drain) + genuine run_show_diffs fault test #21's real diff-fault/recovery (test_r_runShowDiffs_realBridgeRaise_thenEngineUsable): real unison_bridge_run_show_diffs forced through bridge_call2_exn's raise path → false return, no callback published, broker cleared, subsequent real diff succeeds. ✅
  • Release-hook scan (nm): the UNISON_DEBUG_HOOKS-gated seams (run_sync_snapshot, fail_snapshot_accessor_at, fail_strdup_at, force_next_callbacks_raise, …) are absent from the Release binary; they are present/functional in the Debug test host (the accessor-raise and alloc-failure tests exercise them and pass). unison_bridge_test_status is an ungated, always-compiled synthetic-status helper (pre-existing), not one of the gated seams.

Vendor provenance

Base 91421d0, OCaml 5.5.0, patches 0002, 0003, 0004, 0005 (no 0006). Blob SHA-256 2f345306314305d8e921fea587d913b628b86872553bc3776047ef58fe1dfc89. vendor/README.md provenance updated for internal consistency (the Reproducibility line now lists the full 0002+0003+0004+0005 set).

Limitations

  • os_log-style runtime redaction is N/A here; the controller UI wiring (finalizeSyncUI/finalizeSyncUnavailable) is not driven by an AppKit UI harness — the gate, coordinator, SyncCompletionModel, index builder, and the real C marshaller (Debug seam + the real unison_bridge_synchronize path) are all tested.
  • A full live remote (ssh) sync is still not driven by an automated harness; the local-replica real path and the marshaller are covered.

bcourbage and others added 3 commits July 20, 2026 08:19
…4e → 2f345306)

Sync completion made an O(n) series of per-row `unison_bridge_ri_get_details`
bridge calls on the main thread (freeze on large reconciliations), and row
progress walked all nodes O(n) per update (quadratic).

Vendor: new patch 0005 changes `external syncComplete` to carry the final
post-sync `stateItem array` (`!theState`, after do_unisonSynchronize finalizes
every row's whatHappened). Blob rebuilt at the same base 91421d0 + OCaml 5.5.0;
REQUIRED_PATCHES + vendor/README provenance (checksum/size/patch accounting)
updated. Only `uimacbridge.ml` changed vs the prior blob inputs.

C `syncComplete(value state)`: resolves accessors, marshals ONE complete
snapshot (final progress + details + bytes) while the array is CAMLparam-rooted,
using the exn-safe wrappers; any accessor raise / OOM aborts before publishing
and delivers `ok=false` (no partial rows, never "no failures"). Kept separate
from g_ri_roots (read-only completion result, not scan-state).

Coordinator: `SyncResults .available/.unavailable` → `.presentSyncResults(s,
snapshot)` / `.presentSyncUnavailable(s, reason)`, both bound to the exact
pending (SessionID, OperationID), releasing the lease once. An unavailable
snapshot does NOT enter restartRequired — the engine is quiescent post-commit.

Swift: `finalizeSyncUI(snapshot:)` applies the cached snapshot once on the main
actor via the pure `SyncCompletionModel` (identical details-based failure
attribution, zero bridge calls); a count mismatch routes to
`finalizeSyncUnavailable` (clear terminal message; only Rescan/Profiles/Quit).
The completion-time per-row getter loop is deleted. New O(1) `rowToNode` index,
rebuilt atomically with items/tree in replaceItems/beginScanning, replaces the
O(n) `leafNode(forRow:)` walk.

Tests: SyncCompletionModel (row kinds, count mismatch, skipped≠failure, large
5000-row set, row-index builder, and ZERO per-row getter calls proven by a new
Debug-only C counter `unison_bridge_test_ri_get_details_count` — absent from
Release, verified via nm); coordinator available/unavailable/abandoned/
non-interactive-close/duplicate-reject. Full suite 561, Debug + Release green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review corrections on #22 (blob unchanged at 2f345306):

1. Results-unavailable action gate. Added `resultsUnavailable` to
   ReconcileActionGate; it now blocks EVERY engine-reaching action at both
   validation and method boundaries — Direction/Revert, Ignore (its
   phase-independent path), Diff incl. canDiff, Details, Go/Sync — while
   allowing Rescan, Profiles/close, Quit. Fixes the leak where `.done` still
   permitted `.details` (→ ri_get_details) and `.ignore`.

2. Presentation. `finalizeSyncUnavailable` no longer renders the green success
   check: `completionEmphasis(resultsUnavailable:)` returns an orange warning
   symbol + a dedicated accessibility label. CompletionEmphasis gained an
   accessibilityLabel used for the status icon.

3. Real bridge-boundary tests. Factored the C marshalling into a shared
   `deliver_sync_snapshot`; a Debug seam `unison_bridge_test_run_sync_snapshot`
   runs the REAL accessor→strdup→handler path over the just-scanned rooted rows
   on the OCaml thread. New tests: real boundary (ok=true, exact row count,
   populated details); accessor-raise at row 1 and allocation failure at row 1
   (each → exactly one unavailable completion, zero partial rows, runtime still
   usable); a direct wrong-OperationID coordinator rejection; a gate matrix; and
   the presentation descriptor. Debug seams are absent from Release (nm-verified).

4. ABI hardening. `syncComplete` validates `Is_block`/tag-0 before `Wosize_val`
   and bounds `size_t`→int; the Swift trampoline rejects the malformed
   `ok=true, count>0, rows==NULL` shape as unavailable rather than empty success.

Full suite 569, Debug + Release green; blob and patches untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…/guard tests

Verification-only (no production behavior change; blob unchanged at 2f345306):

1. Real end-to-end sync test (test_h_realSync): drives the actual public path
   unison_bridge_synchronize() → vendored unisonSynchronize → do_unisonSynchronize
   → syncComplete !theState (patch 0005) → C marshaller → Swift handler, over a
   local IntegrationFixture with conflict-free one-sided files. Asserts dispatch
   == UNISON_BRIDGE_OK, exactly one ok=true completion, snapshot count == scanned
   rows, plausible terminal fields (a "done" row + populated details), and that
   the one-sided files were actually propagated between the two roots. Uses the
   real OCaml caller, not the test seam.

2. Accessor-failure now covers all three fields (progress/details/bytes) at
   row 1 — each: exactly one ok=false, zero partial rows, and a subsequent clean
   snapshot.

3. Direct Swift malformed-success guard test: extracted an internal
   UnisonBridge.convertSyncCompletion(ok:count:rows:) helper (the trampoline now
   calls it) and assert ok=true+count>0+null-rows and negative count both map to
   unavailable, while ok=true+count 0+null-rows is a genuine empty success.

4. Wrong-OperationID test strengthened: after the bogus completion, the real
   sync phase/lease is asserted still pending (.syncing(s, op)); the genuine
   completion then lands and transitions to .ready.

Full suite 571, Debug + Release green; Debug test symbols absent from Release
(nm); blob and patch set unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bcourbage

Copy link
Copy Markdown
Owner Author

Verification pass (final) — 050894d7, CI green

Test evidence updated (verification-only; production code unchanged aside from extracting the internal convertSyncCompletion test seam):

  • Real end-to-end sync (test_h_realSync_propagatesFiles_andDeliversSnapshot): drives the actual unison_bridge_synchronize() → vendored unisonSynchronizedo_unisonSynchronizesyncComplete !theState (patch 0005) → C marshaller → Swift handler over a local fixture with conflict-free one-sided files. Asserts dispatch UNISON_BRIDGE_OK, exactly one ok=true completion, snapshot count == scanned rows, plausible terminal fields, and that both one-sided files were actually propagated between roots. Uses the real OCaml caller, not the test seam.
  • Accessor failure across all three fields (progress/details/bytes) at row 1 — each → one ok=false, zero partial rows, then a clean snapshot.
  • Direct Swift malformed-success guard via UnisonBridge.convertSyncCompletion: ok=true+count>0+null-rows and negative count → unavailable; ok=true+count 0+null-rows → genuine empty success.
  • Wrong-OperationID: after the bogus completion the real op is asserted still pending (.syncing(s, op)), then the genuine completion transitions to .ready.

Totals: full suite 571, 0 failures; Debug + Release build green; Debug test symbols absent from Release (nm); blob unchanged at 2f345306314305d8e921fea587d913b628b86872553bc3776047ef58fe1dfc89, patches/ unchanged.

bcourbage and others added 3 commits July 20, 2026 13:51
…apshot

# Conflicts:
#	TODO.md
#	Tests/BridgeTests.swift
The 'Patches applied' row and checksum note already list patch 0005; the
Reproducibility line's parenthetical still said (0002+0003+0004). Corrected
to (0002+0003+0004+0005) so the provenance is internally consistent. No blob
or patch change (blob stays 2f345306).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
leafNode(forRow:) already does an O(1) rowToNode lookup (the index is
rebuilt synchronously with items/tree). The doc still described it as an
O(items) tree walk and proposed future caching. Corrected to state the
O(1) index lookup. Comment only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bcourbage
bcourbage marked this pull request as ready for review July 20, 2026 21:03
@bcourbage
bcourbage merged commit e328322 into main Jul 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant