feat: add converge package for push-based metric stabilization

[davidbirdsong]

Pull Request Template

Description

Adds converge/, a pure state machine engine for push-based metric stabilization. It watches a stream of timestamped observations, detects when values stop changing, and emits push-ready samples with correct source timestamps.

This is the core building block for migrating zone traffic metrics from Prometheus scrape (with 4+ minute scrape_delay) to Victoria Metrics push with accurate timestamps.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Code refactoring
• Other (please describe):

Testing

• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Code Quality

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation

Additional Notes

The package separates into a functional core and an imperative shell:

• Engine (pure, no I/O): Ingest([]Observation) → []Sample, Expire(now) → []Sample, Flush() → []Sample
• Runner (thin loop): wires a Fetcher and Sink to the engine on a tick, handles live/backfill priority and graceful shutdown

  Fetcher ──► Runner ──► Engine ──► Runner ──► Sink
  (CF API)    (loop)     (state)    (loop)     (VM push)

Internally the engine manages windows (one per source time bucket), each containing trackers (one per metric series). A tracker counts consecutive identical uint64 values and signals when the configured threshold is crossed.

Key decisions

• Eager push, correct later. Threshold=1 pushes on first observation. Threshold=3 waits for convergence. Config knob, not a code change.
• Backfill rate limiting. Backfill is capped at N Fetch calls per tick (default: 1). Live lane always runs first with no cap. Bounds API consumption without starving freshness.
• No special backfill code path. Old data flows through the same Ingest path. It stabilizes faster because the source has already finished aggregating.
• WindowTTL forced flush. If a window exceeds its TTL without stabilizing, the engine pushes the best known values so nothing is silently dropped.

What's NOT included

• Fetcher implementation (CF GraphQL adapter)
• Sink implementation (VM push client)
• Integration with main.go / mode flag

[davidbirdsong]

This change is part of the following stack:

• feat: add converge package for push-based metric stabilization #25 ◀

_{Change managed by git-spice.}

[Copilot]

Pull request overview

Adds a new converge/ Go package that implements a push-oriented metric stabilization engine (with a thin runner loop) intended to support migrating time-bucketed Cloudflare analytics metrics from scrape-based Prometheus collection to timestamp-correct push-based ingestion.

Changes:

• Introduces a pure(ish) state-machine-style Engine that ingests Observations and emits stabilized Samples, with TTL-based window expiry and flush semantics.
• Adds a Run() convenience loop to drive the engine with a Fetcher and Sink, supporting live polling plus rate-limited backfill.
• Updates module dependencies (go.mod / go.sum) and ignore rules for .gograph/.

Reviewed changes

Copilot reviewed 10 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
converge/engine.go	New core engine + config/defaults + ingest/expire/flush/stats.
converge/window.go	Per-bucket window state and forced-flush sampling.
converge/tracker.go	Per-series stabilization tracker (consecutive-identical threshold).
converge/runner.go	Loop wiring Fetcher→Engine→Sink with live/backfill lanes.
converge/types.go	Public types and interfaces (Observation/Sample/Fetcher/Sink/Stats).
converge/engine_test.go	Unit tests for engine behavior (stabilization, TTL expiry, flush).
converge/tracker_test.go	Unit tests for tracker state transitions and edge cases.
go.mod	Adds new indirect requirements (notably github.com/prometheus/prometheus).
go.sum	Corresponding checksum additions for new/updated transitive modules.
.gitignore	Ignores .gograph/ at repo root.
converge/.gitignore	Ignores .gograph/ under converge/.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[davidbirdsong]

will fix this up in the next PR

[mraerino]

why are these two different calls? any downsides from doing both in a single call so we only ever need to process a single set of samples?

[davidbirdsong]

The separation make sense semantically to me.

• expire check runs even when fetch fails, otherwise we'd have to pass in a nil set of samples or something
• testability is easier since expire can be tested without having to construct observations so we can make targeted assertions with less setup