Skip to content

feat(routing): CockroachDB support, payment-audit trace fix, and cost-aware routing - #350

Merged
jagan-jaya merged 7 commits into
mainfrom
feat/cockroach-audit-multiobjective
Aug 11, 2026
Merged

feat(routing): CockroachDB support, payment-audit trace fix, and cost-aware routing#350
jagan-jaya merged 7 commits into
mainfrom
feat/cockroach-audit-multiobjective

Conversation

@jagan-jaya

Copy link
Copy Markdown
Collaborator

Bundles three independent changes made together this session. Each is its own commit, so they can be reviewed (or reverted) separately.

1. CockroachDB support (feat(db))

CockroachDB is PostgreSQL wire-compatible, so it runs on the existing postgres feature and migrations_pg unchanged.

  • Optional pg_sslmode / pg_ssl_root_cert on PgDatabase, applied through a single shared connection-URL builder in storage.rs (secure / CockroachDB Cloud clusters).
  • docker-compose.cockroach.yml overlay: swaps the postgresql service for a single-node CockroachDB on host 26257 (coexists with a local Postgres on 5432), creates the role/DB, and sets default_int_size=4 + serial_normalization='sql_sequence' so INTEGER/SERIAL typing matches schema_pg.rs.
  • diesel_pg_cockroach.toml (no [print_schema]) so diesel migration run doesn't fail on CockroachDB's information_schema introspection.
  • oneclick.sh --cockroach flow + README section.

Verified: all migrations_pg DDL applies on CockroachDB v24.1.5; column types come out Int4/Int8 correctly; end-to-end oneclick.sh --cockroach brings up the stack and serves live traffic.

2. Payment-audit trace fix (fix(analytics))

The audit timeline and the raw summary path reused the list-level dimension filters (status, gateway, routing_approach, …), which fragmented a single transaction's trace — e.g. status=failure dropped the decide_gateway events and left only the update_gateway_score failure event, and each match card under-counted events.

  • Timeline uses a scope-only filter set, so a selected txn shows both decide_gateway and update_gateway_score.
  • raw_summary_fragment / load_exact use scope+routing filters, so the per-card event count reflects the full trace (consistent with the materialized path).

Verified against live ClickHouse data + unit tests.

3. Cost-aware tie-break + EV-ranked candidates (feat(routing))

  • Multi-objective now adopts its EV-best pick on AUTH_WON too (not only COST_WON), so tied SR scores resolve deterministically to the cheaper PSP instead of an arbitrary same-score tie-break. decided_gateway no longer disagrees with multi_objective_info.
  • Replaced srHead/chosen with a ranked candidate list (auth, cost, EV per PSP, best-EV first) flagged isSrHead/isChosen — a decision now explains why the runner-up lost (its cost is visible even on AUTH_WON). UI panels + the simulator's authGap read the flagged rows; AUTH_WON collapses to one card instead of two duplicates.
  • Config: dlocal seed-cost tiers (LATAM/APAC/MEA).

Breaking API change: multi_objective_info.srHead / .chosen are removed; consumers should read the flagged ranked rows. All in-repo consumers (backend A/B cost attribution, both UI panels, simulator authGap) are updated.

Verified: cargo build + cargo test (decider + analytics suites) pass; frontend tsc --noEmit clean.

CockroachDB is PostgreSQL wire-compatible, so it runs on the existing
`postgres` feature and migrations_pg unchanged. Adds:

- optional pg_sslmode / pg_ssl_root_cert on PgDatabase for secure clusters,
  applied through a single shared connection-URL builder in storage.rs
- docker-compose.cockroach.yml overlay that swaps the `postgresql` service for
  a single-node CockroachDB on host 26257 (coexists with a local Postgres),
  creates the role/DB, and sets default_int_size=4 + serial_normalization so
  integer typing matches schema_pg.rs
- diesel_pg_cockroach.toml (no [print_schema]) so `diesel migration run` does
  not fail on CockroachDB's information_schema introspection
- an `oneclick.sh --cockroach` flow and a README section
The audit timeline and the raw summary path reused the list-level dimension
filters (status, gateway, routing_approach, ...), which fragmented a single
transaction's trace and undercounted its events (e.g. a status=failure filter
dropped the decide_gateway events and showed only the update_gateway_score
failure event). Separate "which transactions match" (outer has() filters) from
"the transaction's own aggregates":

- timeline uses a scope-only filter set so a selected txn shows both the
  decide_gateway and update_gateway_score events
- raw_summary_fragment / load_exact use scope+routing filters so the per-card
  event count reflects the full trace, matching the materialized path
- multi-objective now adopts its EV-best pick on AUTH_WON too (not only
  COST_WON), so tied SR scores resolve deterministically to the cheaper PSP
  instead of an arbitrary same-score tie-break; decided_gateway no longer
  disagrees with multi_objective_info

- replace srHead/chosen with a `ranked` candidate list (auth, cost, EV per PSP,
  best-EV first) flagged isSrHead/isChosen, so a decision explains why the
  runner-up lost (its cost is now visible even on AUTH_WON); the UI panels and
  the simulator's authGap read the flagged rows, and AUTH_WON collapses to a
  single card instead of two duplicates

- config: add dlocal seed-cost tiers (LATAM/APAC/MEA)
Copilot AI lite review requested due to automatic review settings August 11, 2026 13:04
@jagan-jaya jagan-jaya self-assigned this Aug 11, 2026
@jagan-jaya jagan-jaya changed the title CockroachDB support, payment-audit trace fix, and cost-aware routing feat(routing): CockroachDB support, payment-audit trace fix, and cost-aware routing Aug 11, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR bundles three independent improvements across the backend and dashboard: (1) CockroachDB (PostgreSQL-wire) support in local/dev workflows, (2) a ClickHouse payment-audit filtering fix to prevent per-transaction trace fragmentation, and (3) cost-aware/deterministic multi-objective routing with a richer “ranked candidates” explanation surface for consumers.

Changes:

  • Add CockroachDB local support via compose overlay + oneclick.sh flag, plus optional Postgres SSL query parameters in config/URL building.
  • Fix payment-audit timeline vs summary filtering so a selected transaction returns its full curated trace rather than being truncated by list-level dimension filters.
  • Replace srHead/chosen with an EV-ranked candidate list (ranked) and make AUTH-wins adopt the EV-best head deterministically to avoid “decided gateway” vs “info” mismatches.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
website/src/types/api.ts Introduces RankedPsp and updates MultiObjectiveInfo to use ranked instead of srHead/chosen.
website/src/components/pages/DecisionSimulatorPage.tsx Updates simulator panels and derived metrics to read SR-head/chosen from flagged ranked rows and renders all ranked candidates.
website/src/components/pages/DecisionExplorerPage.tsx Same UI update as simulator: renders ranked candidates and consumes flags instead of removed fields.
src/storage.rs Centralizes Postgres URL construction and appends optional sslmode/sslrootcert (for secure Cockroach/Postgres clusters).
src/decider/gatewaydecider/multi_objective/mod.rs Extends multi-objective API model with ranked: Vec<RankedPsp> and updates docs/serialization.
src/decider/gatewaydecider/multi_objective/algorithm.rs Implements EV-ranked candidate list, deterministic AUTH-won decision adoption, and adds tests for tie determinism + ranked output.
src/decider/gatewaydecider/flow_new.rs Ensures decided gateway follows multi-objective output when present (including AUTH_WON) and adjusts cost attribution extraction.
src/config.rs Adds pg_sslmode and pg_ssl_root_cert to PgDatabase config.
src/analytics/clickhouse/metrics/audit_timeline.rs Switches timeline queries to the new scope-only filter set to avoid fragmenting a selected transaction’s trace.
src/analytics/clickhouse/metrics/audit_summaries.rs Adjusts raw/exact summary paths to use scope-appropriate filters so event counts reflect full traces.
src/analytics/clickhouse/filters.rs Adds payment_audit_timeline_filters and payment_audit_summary_scope_filters plus unit tests.
README.md Documents CockroachDB usage and the oneclick.sh --cockroach workflow.
oneclick.sh Adds --cockroach/--postgres selection, Cockroach init flow, and Cockroach-specific Diesel migration invocation.
docker-compose.cockroach.yml New compose overlay to replace Postgres with single-node CockroachDB and run init/migrations against it.
diesel_pg_cockroach.toml New Diesel CLI config to avoid CockroachDB information_schema schema-regeneration failure.
config/development.toml Adds seed-cost tiers (notably for dlocal) to support cost-aware routing experiments locally.
Suppressed comments (1)

src/decider/gatewaydecider/multi_objective/algorithm.rs:273

  • build_ranked sorts only by ev. When two candidates have equal (or non-comparable) EV, the resulting order falls back to the underlying HashMap iteration order, which is intentionally non-deterministic. Since this list is presented as "ranked (best first)", add a deterministic tie-break (e.g. PSP name) to keep ordering stable.
    let mut ranked: Vec<RankedPsp> = score_map
        .iter()
        .filter(|(gw, _)| cost_bps(gw).is_finite())
        .map(|(gw, &score)| {
            let c = cost_bps(gw);
            RankedPsp {
                summary: make_summary(gw.clone(), score, Some(c), costs),
                ev: ev(score, c),
                is_sr_head: head_psp == Some(gw.as_str()),
                is_chosen: chosen_psp == Some(gw.as_str()),
            }
        })
        .collect();
    ranked.sort_by(|a, b| b.ev.partial_cmp(&a.ev).unwrap_or(std::cmp::Ordering::Equal));
    ranked

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/decider/gatewaydecider/multi_objective/algorithm.rs Outdated
Comment thread src/storage.rs
Comment thread src/decider/gatewaydecider/multi_objective/mod.rs Outdated
Comment thread website/src/types/api.ts
…inistic

Addresses PR review:
- auth_won early-return paths (fewer than two PSPs, non-finite head, or head
  without cost data) perform no EV ranking, so report qualified_count = 0
  instead of score_map.len() — the previous value rendered a misleading
  "N PSPs ranked on EV" message
- break EV ties in the ranked candidate list by PSP name, so equal-EV
  candidates order deterministically rather than by HashMap iteration order
Addresses remaining PR review comments:
- percent-encode pg_sslmode / pg_ssl_root_cert before appending them to the
  connection URL, so an sslrootcert path containing a space or reserved
  character can't break the query string (libpq rejects a raw space)
- correct the `ranked` doc (backend model + frontend type): it is empty
  whenever EV ranking was not performed (fewer than two PSPs, or the SR head
  had no finite score / no cost data), not only when no PSP had cost data
@jagan-jaya
jagan-jaya merged commit 3b47c78 into main Aug 11, 2026
17 checks passed
@jagan-jaya
jagan-jaya deleted the feat/cockroach-audit-multiobjective branch August 11, 2026 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants