Skip to content

feat(config): add per-remote sync interval#836

Merged
wesm merged 7 commits into
kenn-io:mainfrom
rodboev:feat/config-per-remote-sync-interval
Jun 24, 2026
Merged

feat(config): add per-remote sync interval#836
wesm merged 7 commits into
kenn-io:mainfrom
rodboev:feat/config-per-remote-sync-interval

Conversation

@rodboev

@rodboev rodboev commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

The serve daemon already runs a local sync every fifteen minutes, but the [[remote_hosts]] entries introduced in #600 are only ever pulled by the one-shot agentsview sync command, so a long-running daemon never refreshes data from configured remotes on its own. Users running the dashboard as a persistent service have to drive remote pulls out-of-band with cron or a wrapper script, which is exactly the kind of scheduling the daemon is otherwise responsible for.

This adds an optional interval field to each [[remote_hosts]] entry, expressed as a duration string such as "5m" or "1h". The field is decoded natively the same way events_coalesce_interval is, and it defaults to zero, which means "do not periodically sync this host." Configs written before this change behave exactly as they do today, and nothing starts pulling from a remote unless the user explicitly sets an interval.

On the daemon side, the periodic sync routine now launches one lightweight ticker goroutine per remote host that has a positive interval, each calling the same single-host sync path the CLI uses, in incremental mode, logging any failure without disturbing the others or the local periodic sync. After a successful remote sync pass, the goroutine emits a data_changed SSE event through the serve broadcaster so connected UI clients pick up the new data immediately rather than waiting for the next local periodic sync. The emitter is threaded as a sync.Emitter interface to keep the dependency direction clean; the CLI path continues to run without one. The remote host list is validated before these goroutines start; if validation fails the daemon logs a warning and keeps serving with local sync intact rather than refusing to start. The existing fifteen-minute local sync loop is left untouched, and because per-host remote sync reuses the already-tested runRemoteSyncOnce path, the new surface area is the configuration field, its validation, the scheduling glue, and the emit-on-success behavior.

Tests cover loading the new field from a config file, validating the negative-interval case, and the emit-on-success/no-emit-on-error/nil-emitter behavior of the sync loop.

Closes #412

@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (433932e)

Verdict: One medium issue remains; no high or critical findings were reported.

Medium

  • cmd/agentsview/main.go:679 - Periodic remote syncs call runRemoteSyncOnce, which builds a remote sync engine without the serve broadcaster/emitter. When scheduled remote sync writes new or updated sessions, connected UI clients do not receive a data_changed SSE event and can stay stale until manual refresh or an unrelated local sync event.
    • Suggested fix: Thread the serve broadcaster/emitter into scheduled remote syncs, for example by adding an optional emitter to ssh.RemoteSync/runRemoteSyncOnce and emitting when remote sync reports synced sessions.

Panel: ci_default_security | Synthesis: codex, 8s | Members: codex_default (codex/default, done, 4m47s), codex_security (codex/security, done, 1m44s) | Total: 6m39s

@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (983654f)

Summary verdict: One medium issue remains; no high or critical findings were reported.

Medium

  • cmd/agentsview/main.go:679 — Scheduled remote sync writes through runRemoteSyncOnce, which creates a separate remote sync engine without the server broadcaster. Sessions ingested by this interval path will not emit data_changed SSE events, so the live UI can stay stale until polling or manual refresh even though local periodic syncs refresh immediately.
    • Fix: Thread the server emitter/broadcaster into the remote sync path, or emit after a successful remote run that synced sessions, and add a focused test for the scheduled path.

Panel: ci_default_security | Synthesis: codex, 7s | Members: codex_default (codex/default, done, 4m49s), codex_security (codex/security, done, 59s) | Total: 5m55s

@rodboev

rodboev commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Addressed. startRemoteHostSync now takes a sync.Emitter and calls Emit("remote") after each successful remote sync pass. The broadcaster is threaded from the serve setup through startPeriodicSync into the per-host goroutines. Emit is skipped on sync failure and when the emitter is nil (CLI path). Three tests cover the success, error, and nil-emitter cases via an extracted runRemoteHostSyncLoop helper.

@rodboev rodboev changed the title feat(config): add per-remote sync interval - Jun 24, 2026
@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (2ebbdd3)

Scheduled remote sync changes need one medium-severity test cleanup; no critical or high issues reported.

Medium

  • cmd/agentsview/main_test.go:303 - The new tests start runRemoteHostSyncLoop, but the loop has no cancellation path and the done channel is never read. Each test leaks a ticker goroutine; the error case keeps logging every 10ms after the test returns, which can contaminate later tests and make shuffled or captured-log runs flaky.
    • Fix: Add a cancellable context/done channel to runRemoteHostSyncLoop or inject a tick channel for tests, then cancel and wait for the goroutine to exit in each test.

Panel: ci_default_security | Synthesis: codex, 7s | Members: codex_default (codex/default, done, 4m26s), codex_security (codex/security, done, 27s) | Total: 5m0s

@rodboev rodboev changed the title - feat(config): add per-remote sync interval Jun 24, 2026
@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (f56bbaa)

Scheduled remote sync has one medium-severity behavioral regression; no high or critical findings were reported.

Medium

  • cmd/agentsview/main.go:696 - Scheduled remote sync emits data_changed after every successful run, even when RemoteSync.Run synced zero sessions. Existing sync emitters only notify when session data changed, so unchanged remotes can still cause frontend refreshes and “new data” signals every interval.
    • Fix: Preserve the remote sync stats from rs.Run and emit only when SessionsSynced > 0 or another user-visible session change occurred.

Panel: ci_default_security | Synthesis: codex, 7s | Members: codex_default (codex/default, done, 3m51s), codex_security (codex/security, done, 1m13s) | Total: 5m11s

@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (33e7f6e)

No Medium, High, or Critical findings were reported.

The only reported issues were Low severity, so they are omitted per review-combination rules.


Panel: ci_default_security | Synthesis: codex, 5s | Members: codex_default (codex/default, done, 4m36s), codex_security (codex/security, done, 2m7s) | Total: 6m48s

@wesm wesm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I will merge (and rebase if necessary) after landing #826

@wesm

wesm commented Jun 24, 2026

Copy link
Copy Markdown
Member

rebasing

- fix(config): skip periodic remote sync when host validation fails
- chore: retrigger CI after flaky e2e
- feat(config): emit SSE event after scheduled remote sync
- fix(config): add done channel to remote sync loop to prevent goroutine leaks in tests
- fix(config): gate remote sync SSE emit on sessions actually synced
@wesm wesm force-pushed the feat/config-per-remote-sync-interval branch from 33e7f6e to 16e75a8 Compare June 24, 2026 16:41
Remote host intervals configure daemon scheduling and should not become part of the manual remote sync authorization contract. Non-local daemon requests should continue to match configured hosts by the SSH target identity only, so clients do not need to echo scheduling-only config back to the server.\n\nScheduled remote sync also needs to emit the same sessions scope as manual remote sync so SSE consumers refresh consistently.
@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (16e75a8)

Medium: scheduled remote sync bypasses the main sync serialization path.

Medium

  • cmd/agentsview/main.go:841 - Scheduled remote sync calls rs.Run directly from its own goroutine instead of using the main sync engine’s exclusive lock. This can overlap with local sync/resync or /sync/remotes, interleaving DB writes and remote skip-cache load/replace; a full resync could also swap the DB while a remote import is writing. Route scheduled remote sync through the same serialization path as manual remote sync, for example by executing rs.Run inside engine.RunExclusive with daemon work tracking as appropriate.

Panel: ci_default_security | Synthesis: codex, 7s | Members: codex_default (codex/default, done, 4m32s), codex_security (codex/security, done, 2m4s) | Total: 6m43s

The scheduled remote sync scope test only needs to capture the first emitted scope. Leaving later sends blocking can strand the sync loop inside the test emitter before it observes the done channel, which makes the test vulnerable to scheduler timing in CI.
@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (7a11ed6)

High-risk concurrency issue found; no security-specific vulnerability was identified.

High

  • cmd/agentsview/main.go:841 - Scheduled remote sync runs rs.Run(context.Background()) outside the main sync engine’s RunExclusive lock. Manual remote sync uses that lock because RemoteSync.Run builds its own engine and writes to the same SQLite archive and remote skip cache. As written, an interval sync can overlap a local sync/resync or another interval sync. During a resync swap, sessions may be written to the old DB while the remote skip cache is later written to the reopened new DB, causing unchanged remote files to be skipped even though their sessions were not copied into the new archive.

    Suggested fix: Pass the daemon sync engine into the scheduled remote sync path and run the full RemoteSync.Run under engine.RunExclusive, while preserving existing idle-work tracking.


Panel: ci_default_security | Synthesis: codex, 8s | Members: codex_default (codex/default, done, 3m21s), codex_security (codex/security, done, 1m56s) | Total: 5m25s

Scheduled remote sync writes to the same SQLite archive and remote skip cache as manual remote sync. Running it outside the daemon sync engine lock allowed interval jobs to overlap local sync or resync work, which could leave skip-cache state ahead of the archive after a DB swap.\n\nRun scheduled remote pulls under the engine exclusive lock so they share the same single-writer boundary as daemon-owned manual remote sync.
@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (bb9ac1b)

Summary verdict: One medium lifecycle bug needs fixing; no security findings were reported.

Medium

  • cmd/agentsview/main.go:895 - Scheduled remote sync work runs outside idleTracker.Do, unlike the existing local periodic sync. In background daemon mode, the idle timeout can fire while a remote SSH sync is still downloading or writing to SQLite, which can shut down and close the DB while that goroutine is active.

    Fix: Pass the idle tracker into the remote sync loop and wrap each syncFn invocation with idleTracker.Do or BeginWork, skipping new work when the daemon is draining.


Panel: ci_default_security | Synthesis: codex, 8s | Members: codex_default (codex/default, done, 4m24s), codex_security (codex/security, done, 53s) | Total: 5m25s

Background daemons use the idle tracker to avoid shutting down while internal write work is active. The scheduled remote sync loop was outside that boundary, so an idle timeout could begin shutdown while SSH sync was still writing to SQLite under the exclusive sync lock.\n\nWrap each scheduled remote sync tick in idle-tracked work so active remote pulls postpone idle shutdown and new ticks are skipped once the daemon is draining.
@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (dc277cc)

Medium finding remains; no Critical or High issues were reported.

Medium

  • cmd/agentsview/main.go:873 - Scheduled remote sync always sets Full: false. If the initial data-version resync aborts and falls back to incremental sync, DB.NeedsResync() remains true and the old remote skip cache remains, so scheduled remote sync can skip unchanged remote files and leave remote sessions parsed with stale data.

    Fix: Set Full based on database.NeedsResync() or propagate a force-full flag after an attempted data-version resync, and add a focused test for that case.


Panel: ci_default_security | Synthesis: codex, 9s | Members: codex_default (codex/default, done, 6m36s), codex_security (codex/security, done, 1m13s) | Total: 7m58s

Scheduled remote sync uses the remote skip cache, so it cannot safely run incrementally while the local archive still needs a parser data-version resync. If the initial resync aborts and leaves NeedsResync true, unchanged remote files must be parsed again instead of skipped against stale cache state.\n\nUse the DB stale-data marker to force full scheduled remote pulls until the archive has been rebuilt successfully.
@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (9297bcc)

Summary verdict: one Medium issue needs attention; no High or Critical findings were reported.

Medium

  • cmd/agentsview/main.go:878 - Scheduled remote syncs run with context.Background(), so an in-flight SSH/tar operation will not be canceled when the serve context is canceled by SIGTERM/SIGINT or daemon shutdown. Since the SSH helpers use exec.CommandContext, this loses the intended cancellation path and can leave child processes running after agentsview exits.

    Fix: Thread the serve context through startPeriodicSync, startRemoteHostSync, and remoteHostSyncFunc, select on ctx.Done() in the loop, and pass that context to rs.Run.


Panel: ci_default_security | Synthesis: codex, 7s | Members: codex_default (codex/default, done, 5m20s), codex_security (codex/security, done, 2m20s) | Total: 7m47s

Scheduled remote sync shells out through SSH helpers that honor command contexts, but the scheduled path was still passing context.Background. That let remote child processes outlive serve cancellation and daemon shutdown.\n\nThread the serve context through the scheduled sync loop and remote runner so ticker goroutines exit on cancellation and in-flight remote sync commands receive the same cancellation signal.
@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (372018c)

Verdict: No Medium, High, or Critical findings to report.

The only reported issue was Low severity, so it is omitted per the review-combination rules.


Panel: ci_default_security | Synthesis: codex, 6s | Members: codex_default (codex/default, done, 5m47s), codex_security (codex/security, done, 2m20s) | Total: 8m13s

@wesm

wesm commented Jun 24, 2026

Copy link
Copy Markdown
Member

good, merging. thank you!

@wesm wesm merged commit 86e1fa9 into kenn-io:main Jun 24, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Feature: periodic SSH remote sync from serve (config-driven [[remotes]] list)

2 participants