feat(agent-003): implement full Market Monitor workflow mirroring AGENT-002#80
Conversation
Mirrors the AGENT-002 Governance Analyst structure to give AGENT-003 Market Monitor the same full standalone TypeScript implementation that AGENT-002 has: - `agent-003-market-monitor/` — standalone Node.js process - `src/index.ts` — banner, cycle runner, polling vs single-run mode - `src/config.ts` — config + thresholds mirrored from the character - `src/types.ts` — ecocredit marketplace types + per-workflow shapes - `src/ledger.ts` — LCD client for ecocredit + marketplace endpoints - `src/store.ts` — SQLite state (trade obs, anomaly dedupe, liquidity snapshots, retirement summaries, workflow executions) - `src/ooda.ts` — generic OODA executor (same shape as agent-002) - `src/monitor.ts` — Claude narrative layer, one function per workflow - `src/output.ts` — console + optional Discord webhook dispatcher - `src/workflows/price-anomaly-detection.ts` — WF-MM-01 - `src/workflows/liquidity-monitor.ts` — WF-MM-02 - `src/workflows/retirement-tracking.ts` — WF-MM-03 Design decisions documented in the agent README: 1. Deterministic numbers, narrative-only LLM calls. All severity classification, z-score computation, liquidity health scoring, and demand index derivation happen locally. Claude only writes the report from those numbers. Keeps the agent cheap, reproducible, and auditable. 2. Sell-order-as-trade proxy for the MVP until Ledger MCP exposes filled-trade events. The code path swaps cleanly once real trade events are available. 3. Batch-supply delta as the retirement source for the MVP, capped at 100 most recent batches per cycle to protect the LCD. A follow-up PR will plug in a real MsgRetire tx-stream client. 4. Alert dedupe by (trade_id, severity) — a trade that escalates from WARNING to CRITICAL still fires a new alert; a same-severity re-alert does not. Thresholds (`config.market.*`) mirror the character definition at `agents/packages/agents/src/characters/market-monitor.ts` so downstream tooling has a single source of truth. Anomaly severity boundaries (WARNING >= 2.0, CRITICAL >= 3.5 z-score) and the liquidity depth floor live in `config.ts` and are referenced from the system prompt. CI: the new agent is added to the `agents` job so `npx tsc --noEmit` runs against it on every PR, matching how agent-002-governance-analyst is wired. - Lands in: `agent-003-market-monitor/`, `.github/workflows/ci.yml` - Changes: new standalone AGENT-003 process with 3 workflows (WF-MM-01/02/03) - Validate: `cd agent-003-market-monitor && npm ci && npx tsc --noEmit` Refs phase-2/2.2-agentic-workflows.md §WF-MM-01, §WF-MM-02, §WF-MM-03 Refs agents/packages/agents/src/characters/market-monitor.ts (regen-network#64)
There was a problem hiding this comment.
Code Review
This pull request introduces AGENT-003, a market monitor for the Regen ecocredit marketplace. It implements three OODA-based workflows: price anomaly detection, liquidity monitoring, and retirement tracking, using a standalone Node.js process with SQLite for state and Claude for narrative reporting. Feedback focused on correcting statistical inconsistencies in Z-score calculations, improving the accuracy of retirement tracking by using deltas instead of cumulative totals, handling non-USD denominations, and addressing potential issues with pagination, concurrency, and overlapping execution cycles.
| let totalRetired = 0; | ||
| let batchesWithRetirement = 0; | ||
| for (const row of rows) { | ||
| const retired = Number(row.supply.retired_amount); | ||
| if (Number.isFinite(retired) && retired > 0) { | ||
| totalRetired += retired; | ||
| batchesWithRetirement++; | ||
| } | ||
| } | ||
| if (totalRetired === 0) continue; |
There was a problem hiding this comment.
The current implementation aggregates the absolute retired_amount from the top 100 batches. Since retired_amount is a cumulative total for each batch, the resulting totalRetired and demandIndex reflect the static state of those batches rather than recent retirement activity. Furthermore, as batches fall out of the "top 100" list, the aggregated total will fluctuate based on batch creation frequency rather than actual demand. To accurately track "patterns" and "demand signals" as intended by WF-MM-03, the agent should compute the delta between the current and previously observed retirement amounts.
| const classMedian = median(classSamples); | ||
| const classMean = classSamples.reduce((a, b) => a + b, 0) / classSamples.length; | ||
| const classStd = stddev(classSamples, classMean); | ||
| const zClass = classStd > 0 ? (trade.pricePerCredit - classMedian) / classStd : 0; |
There was a problem hiding this comment.
The Z-score calculation is statistically inconsistent because it uses the classMedian as the center but divides by the classStd (which is calculated relative to the classMean). For a standard Z-score, you should use (value - mean) / stddev. Alternatively, for a more robust "Modified Z-score", you could use the Median Absolute Deviation (MAD). This inconsistency also applies to the zBatch calculation on line 177.
| const zClass = classStd > 0 ? (trade.pricePerCredit - classMedian) / classStd : 0; | |
| const zClass = classStd > 0 ? (trade.pricePerCredit - classMean) / classStd : 0; |
| ? batchSamples.reduce((a, b) => a + b, 0) / batchSamples.length | ||
| : classMean; | ||
| const batchStd = batchSamples.length > 1 ? stddev(batchSamples, batchMean) : classStd; | ||
| const zBatch = batchStd > 0 ? (trade.pricePerCredit - batchMedian) / batchStd : 0; |
There was a problem hiding this comment.
Similar to the zClass calculation, the zBatch calculation is statistically inconsistent as it uses batchMedian with batchStd (derived from batchMean).
| const zBatch = batchStd > 0 ? (trade.pricePerCredit - batchMedian) / batchStd : 0; | |
| const zBatch = batchStd > 0 ? (trade.pricePerCredit - batchMean) / batchStd : 0; |
| const usdPerAskUnit = isUsdStableDenom(order.ask_denom) ? 1 : 1; | ||
| const pricePerCredit = (ask * usdPerAskUnit) / qty; |
There was a problem hiding this comment.
The usdPerAskUnit is hardcoded to 1 for all denominations. This will result in incorrect price calculations and false anomaly alerts for any sell orders listed in non-USD denominations (e.g., REGEN tokens). While the PR notes this as an MVP limitation, it is recommended to at least skip orders with unknown denominations to prevent polluting the baseline with incorrect data.
| const usdPerAskUnit = isUsdStableDenom(order.ask_denom) ? 1 : 1; | |
| const pricePerCredit = (ask * usdPerAskUnit) / qty; | |
| const usdPerAskUnit = isUsdStableDenom(order.ask_denom) ? 1 : null; | |
| if (usdPerAskUnit === null) return null; | |
| const pricePerCredit = (ask * usdPerAskUnit) / qty; |
| const interval = setInterval(() => { | ||
| runCycle(ledger).catch((err) => | ||
| console.error(`Cycle failed:`, err) | ||
| ); | ||
| }, config.pollIntervalMs); | ||
|
|
||
| const shutdown = () => { | ||
| console.log("\nShutting down gracefully..."); | ||
| clearInterval(interval); | ||
| store.close(); | ||
| process.exit(0); | ||
| }; |
There was a problem hiding this comment.
Using setInterval can lead to overlapping execution cycles if a cycle takes longer than the polling interval (e.g., due to network latency or slow LLM responses). It is safer to use a recursive setTimeout pattern to ensure that the next cycle only starts after the previous one has finished.
| const interval = setInterval(() => { | |
| runCycle(ledger).catch((err) => | |
| console.error(`Cycle failed:`, err) | |
| ); | |
| }, config.pollIntervalMs); | |
| const shutdown = () => { | |
| console.log("\nShutting down gracefully..."); | |
| clearInterval(interval); | |
| store.close(); | |
| process.exit(0); | |
| }; | |
| let timeoutId: NodeJS.Timeout; | |
| const runNext = () => { | |
| runCycle(ledger) | |
| .catch((err) => console.error(`Cycle failed:`, err)) | |
| .finally(() => { | |
| timeoutId = setTimeout(runNext, config.pollIntervalMs); | |
| }); | |
| }; | |
| runNext(); | |
| const shutdown = () => { | |
| console.log("\nShutting down gracefully..."); | |
| clearTimeout(timeoutId); | |
| store.close(); | |
| process.exit(0); | |
| }; |
| async getCreditClasses(): Promise<CreditClass[]> { | ||
| const params = new URLSearchParams(); | ||
| params.set("pagination.limit", "200"); | ||
|
|
||
| const data = await this.get( | ||
| `/regen/ecocredit/v1/classes?${params.toString()}` | ||
| ); | ||
| return (data.classes || []) as CreditClass[]; | ||
| } |
There was a problem hiding this comment.
| await Promise.all( | ||
| cappedBatches.map(async (batch) => { | ||
| const supply = await ledger.getBatchSupply(batch.denom); | ||
| if (supply) { | ||
| batchesWithSupply.push({ | ||
| batch, | ||
| supply, | ||
| classId: classIdFromBatchDenom(batch.denom), | ||
| }); | ||
| } | ||
| }) | ||
| ); |
The AGENT-001 Registry Reviewer standalone TypeScript implementation has been in the repo since PR regen-network#64 but was never wired into the CI `Agents` job. Only `agents/` (the ElizaOS character pack) and `agent-002-governance-analyst/` run `npx tsc --noEmit` on every PR. `agent-001-registry-reviewer/` has its own `package.json`, `tsconfig.json`, and a full workflow implementation (reviewer, ooda, 3 workflows, ledger client, store, types). It compiles cleanly on Node 22 locally — this PR adds it to the CI matrix so regressions are caught on every PR. After PRs regen-network#80 and regen-network#81 (which added agent-003 and agent-004 to CI), this PR closes the gap for the only remaining unwired standalone agent: agent-001. The full CI Agents job now typechecks four standalone agent processes: agents/ (ElizaOS character pack) agent-001-registry-reviewer/ (this PR) agent-002-governance-analyst/ (already wired) agent-003-market-monitor/ (added in PR regen-network#80) agent-004-validator-monitor/ (added in PR regen-network#81) ## Validation $ cd agent-001-registry-reviewer && npm ci && npx tsc --noEmit (exit 0) Same commands as the CI step. - Lands in: `.github/workflows/ci.yml` - Changes: add agent-001-registry-reviewer npm ci + typecheck steps - Validate: `cd agent-001-registry-reviewer && npm ci && npx tsc --noEmit`
Follow-up to PR regen-network#80 — promised in the PR description as "Unit tests for median, stddev, classifyAnomaly, scoreHealth, computeDemandIndex, etc., planned as a separate test-only PR so the original PR stays a single-concern 'add the agent' change." Adds 52 unit tests across 3 test files covering every deterministic helper function in the AGENT-003 workflows. The helpers compute severity, liquidity health, and demand signals — they're the parts most likely to silently drift in a future refactor, so pinning their exact behavior prevents regressions that a typecheck cannot catch. ## Changes ### Helper exports Eight previously module-private helpers are now exported so the test file can import them: price-anomaly-detection.ts: median, stddev, classifyAnomaly, classIdFromBatchDenom, isUsdStableDenom liquidity-monitor.ts: askUsd, scoreHealth retirement-tracking.ts: computeDemandIndex The export is the only production-code change — no behavior change, no API rename. Module consumers unchanged. ### Test files src/workflows/price-anomaly-detection.test.ts (30 tests) median — empty, one-element, odd length, even length, non-mutation, negatives, floats stddev — empty, single, two-element, known five-element, n-1 denominator (sample, not population) classifyAnomaly — INFO / WARNING / CRITICAL boundaries, max of two inputs, symmetry for negative z-scores classIdFromBatchDenom — canonical regen format, single-letter, no dash, empty, degenerate leading dash isUsdStableDenom — USDC/USDT/DAI case-insensitive, ibc/* wrappers, non-stable rejects, empty string src/workflows/liquidity-monitor.test.ts (13 tests) askUsd — normal, zero quantity, negative quantity, non-finite ask, non-finite quantity, fractional scoreHealth — CRITICAL at sub-half depth, DEGRADED between half and full, HEALTHY at 3x floor with strong count, score-cap when depth saturates, count-cap at 20, zero case src/workflows/retirement-tracking.test.ts (9 tests) computeDemandIndex — zero, volume cap (60), count cap (20 at 10 retirements), breadth cap (20 at 5 retirees), total cap (100), moderate activity, log10 scaling, sub-1 floor, rounding ### Vitest setup vitest.config.ts (8 lines) — standard config, node env package.json — adds "test" and "test:watch" scripts + vitest ^2.1.0 devDep tsconfig.json — excludes *.test.ts from the production typecheck path (tests still typecheck via vitest itself) .gitignore — adds *.db-shm and *.db-wal (SQLite WAL side files from running tests locally; the base *.db pattern doesn't catch them) ## Validation $ cd agent-003-market-monitor && npm test Test Files 3 passed (3) Tests 52 passed (52) $ cd agent-003-market-monitor && npx tsc --noEmit (exit 0) Both CI checks continue to pass after this PR. ## Scope Does NOT touch the workflow OODA loops themselves, the Claude narrative layer, the LCD client, the SQLite store, or any output formatting. The tests cover the pure functions only — the surrounding integration machinery is tested implicitly when AGENT-003 runs a live cycle against the Regen LCD. - Lands in: `agent-003-market-monitor/` - Changes: 52 unit tests + vitest setup + 8 helper exports - Validate: `cd agent-003-market-monitor && npm test` ## PR relationship This PR is based on PR regen-network#80's branch (the AGENT-003 implementation) because the helpers don't exist on upstream/main. If regen-network#80 merges first, this PR rebases cleanly.
Replaces the batch-supply-delta MVP proxy in WF-MM-03 with a real tx-search client that reads recent MsgRetire events from the Cosmos LCD. This closes the follow-up documented in PR regen-network#80's design-decision regen-network#3: "Batch supply delta as retirement source (MVP). A follow-up PR will plug in a real tx-stream client for MsgRetire once the LCD event endpoint is available." ## What changes in the workflow The observe phase no longer walks batches → supplies. Instead: const retirements = await ledger.getRecentRetirementTxs(200); The orient phase no longer synthesizes retirement records from supply deltas. Instead it calls a new pure function `aggregateRetirementsByClass` that groups a list of real Retirement records by class id, computes the top retiree by cumulative quantity (not count), counts unique retirees via a Set, and measures the jurisdiction-metadata coverage percentage. The act phase is unchanged — it still persists, outputs, and alerts on demand-index jumps. ## What changes in the ledger client Adds two new methods to `LedgerClient`: - `getRecentRetirementTxs(limit)` — queries the LCD tx-search endpoint with `events=message.action='/regen.ecocredit.v1.MsgRetire'`, reverse-ordered, limited. Parses each tx response into zero or more Retirement records. Returns [] on error (transient LCD failures degrade to "no recent retirements" rather than crashing the cycle). - `parseRetirementsFromTx(tx)` — public parser exposed so unit tests can feed synthetic tx responses. Walks `logs[].events[]` AND top-level `events[]` for cross-SDK compatibility, filters on the Regen v1 and v1beta1 EventRetire type URLs, and extracts the owner / batch_denom / amount / jurisdiction / reason attributes. Skips malformed events (missing batch_denom, non-finite amount, non-positive amount). The parser is a pure function — no side effects, no LCD calls — so it is fully unit-testable with synthetic inputs. ## New tests (16 total across 2 files) **src/ledger.test.ts** (9 tests) - empty tx - single EventRetire with all attributes - v1beta1 fallback event type - non-retirement events ignored - missing batch_denom ignored - non-finite, zero, and negative amounts ignored - multiple retirements in one tx (batched) - events read from tx.events[] as well as logs[].events[] - tx hash and timestamp carried through to Retirement **src/workflows/retirement-tracking.test.ts** (7 new tests for aggregateRetirementsByClass, on top of the existing 9 for computeDemandIndex) - empty retirements → empty map - single retirement → one-entry summary - multiple retirements grouped by class id - top retiree identified by cumulative quantity, not count - jurisdiction metadata percentage - classes with zero total quantity skipped - empty retiree string handled without crashing Full test suite: 68 passed across 4 files (up from 52 in PR regen-network#99). ## What this unlocks The old MVP proxy produced RetirementSummary records with empty `topRetiree` / `topRetireeQuantity` / `pctWithJurisdiction` fields — the supply-delta source couldn't carry retiree identity or compliance metadata. The new implementation populates every field on the Retirement type. Downstream: - Demand index calculations now distinguish "5 unique retirees each retiring 200" from "1 whale retiring 1000". - Jurisdiction-metadata coverage surfaces compliance-driven demand (per the SPEC note: ">50% jurisdiction coverage typically implies compliance-driven demand"). - Top-retiree identification feeds the narrative layer with concrete context instead of a placeholder. - Each Retirement carries its real tx hash and timestamp, so downstream auditing can link back to the exact on-chain event. ## Scope Does NOT touch WF-MM-01 (price anomaly detection) or WF-MM-02 (liquidity monitor). The price oracle integration noted elsewhere in the README is still future work. The LCD-level error handling returns empty arrays on failure — a future follow-up might distinguish transient failures from genuine "no recent retirements". - Lands in: `agent-003-market-monitor/` - Changes: new tx-search client methods + rewritten WF-MM-03 + 16 new tests - Validate: `cd agent-003-market-monitor && npm test && npx tsc --noEmit` ## PR relationship Based on PR regen-network#99 (AGENT-003 unit tests) which is based on PR regen-network#80 (AGENT-003 initial implementation). If both land first, this PR rebases cleanly. Sibling PR to a forthcoming AGENT-004 real delegation tx-stream follow-up. Refs phase-2/2.2-agentic-workflows.md §WF-MM-03 Refs PR regen-network#80's design decision regen-network#3 (MVP proxy → real tx-stream)
…gination, concurrency Addresses Gemini review feedback on PR regen-network#80: WF-MM-01 (price-anomaly-detection): * Z-score now uses (value - mean) / stddev so the numerator and denominator are consistent. Median is still surfaced in the alert payload as a separate context signal. * sellOrderToTrade now skips non-USD-stablecoin asks instead of silently treating them as 1:1 USD. Non-USD orders would otherwise pollute class/batch baselines with arbitrary-priced noise. WF-MM-02 (liquidity-monitor): * top-10 depth sort pre-computes askUsd once per order instead of recomputing it inside every comparator call (sort can call the comparator O(n log n) times, so this is a real hot-path win on large classes). WF-MM-03 (retirement-tracking): * Switched from aggregating cumulative retired_amount to computing per-batch deltas against the prior cycle's observation. The cumulative total was misreporting demand and fluctuated based on which batches fell into the top 100 slice. First-observation reads are recorded as the baseline and do not count toward the delta. * Retirement volume is now valued in USD using the most recent median ask price from the WF-MM-02 liquidity snapshot for the class, falling back to 1:1 when no snapshot exists yet. * Per-cycle batch supply fan-out is now chunked at concurrency 5 instead of firing 100 Promise.all requests at the LCD at once, which would trip rate limits on public endpoints. LedgerClient: * getCreditClasses walks every pagination page (up to a safety cap of 25 pages at 200 per page) instead of silently truncating to the oldest 200 classes. New classes created after the ceiling were being invisible to every workflow. Main loop: * setInterval → recursive setTimeout so a slow cycle can never overlap with the next tick — no more race conditions or database contention when the LLM narrative or LCD is slow. Store: * New batch_retirement_state table + getPreviousRetiredAmount / upsertRetiredAmount helpers back the per-batch delta logic. * New getLatestMedianAsk helper reads the most recent liquidity snapshot so WF-MM-03 can value retirement volume without pulling a separate query. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…k#80 onto regen-network#103 The real MsgRetire tx-stream branch (this PR) was cut from agent-003's pre-fixup state, so it was missing the four non-z-score fixes from regen-network#80's gemini-review commit (5d04ae6). Bringing them forward so regen-network#80 can be closed as fully superseded: - `LedgerClient.getCreditClasses` — walks every pagination page up to a 25-page safety cap (was silently truncating to oldest 200 classes; new classes were invisible to every workflow). - `liquidity-monitor` — top-10 depth sort pre-computes askUsd once per order instead of recomputing it in each comparator call. Also filters non-USD-stablecoin asks so arbitrary-denominated orders don't pollute class/batch baselines. - `index.ts` main loop — `setInterval` → recursive `setTimeout` so a slow cycle can't overlap with the next tick (no race conditions or database contention when the LCD/LLM is slow). - Z-score comment refreshed; the `-- mean not median --` fix from 8c75610 is kept as the authoritative version. The regen-network#80 retirement-delta work (batch-supply-delta + per-batch cumulative state) is intentionally **not** ported — this PR's tx-stream approach supersedes it wholesale and is exact where the delta method was approximate. The now-unused `batch_retirement_state` table and its `getPreviousRetiredAmount`/`upsertRetiredAmount` helpers are removed in the same commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Superseded by #103 — the real MsgRetire tx-stream implementation fully replaces the batch-supply-delta approach here, and all four non-delta fixes from this PR's gemini fixup commit ( |
#96) The AGENT-001 Registry Reviewer standalone TypeScript implementation has been in the repo since PR #64 but was never wired into the CI `Agents` job. Only `agents/` (the ElizaOS character pack) and `agent-002-governance-analyst/` run `npx tsc --noEmit` on every PR. `agent-001-registry-reviewer/` has its own `package.json`, `tsconfig.json`, and a full workflow implementation (reviewer, ooda, 3 workflows, ledger client, store, types). It compiles cleanly on Node 22 locally — this PR adds it to the CI matrix so regressions are caught on every PR. After PRs #80 and #81 (which added agent-003 and agent-004 to CI), this PR closes the gap for the only remaining unwired standalone agent: agent-001. The full CI Agents job now typechecks four standalone agent processes: agents/ (ElizaOS character pack) agent-001-registry-reviewer/ (this PR) agent-002-governance-analyst/ (already wired) agent-003-market-monitor/ (added in PR #80) agent-004-validator-monitor/ (added in PR #81) ## Validation $ cd agent-001-registry-reviewer && npm ci && npx tsc --noEmit (exit 0) Same commands as the CI step. - Lands in: `.github/workflows/ci.yml` - Changes: add agent-001-registry-reviewer npm ci + typecheck steps - Validate: `cd agent-001-registry-reviewer && npm ci && npx tsc --noEmit`
Summary
Gives AGENT-003 Market Monitor the same full standalone TypeScript implementation that AGENT-002 Governance Analyst already has. AGENT-003's character definition was merged in #64; this PR adds the workflow code that runs against Regen Ledger.
Structure — mirrors AGENT-002
Design decisions
Deterministic numbers, narrative-only LLM calls. All severity classification, z-score computation, liquidity health scoring, and demand index derivation happen locally in plain TypeScript. Claude is only invoked to write the report from those numbers. Keeps the agent cheap, reproducible, and auditable — and makes the hard parts trivially unit-testable in a follow-up PR.
Sell-order-as-trade proxy for the MVP. Until Ledger MCP exposes filled-trade events, the agent treats open sell orders as trade observations for the z-score baseline. An unusually high or low ask price is still a market structure signal worth surfacing, and the code path swaps cleanly once real trade events are available.
Batch-supply delta as retirement source for the MVP. WF-MM-03 reads `retired_amount` from each batch supply and aggregates per class. Capped at the 100 most recent batches per cycle so the agent doesn't hammer the LCD on a mainnet with thousands of historical batches. A follow-up PR will plug in a real `MsgRetire` tx-stream client.
Alert dedupe by (trade_id, severity). WF-MM-01 only alerts once per `(trade_id, severity)` tuple. A trade that later escalates from WARNING to CRITICAL still fires a new alert; a same-severity re-alert does not.
Thresholds single-sourced from the character. `config.market.*` mirrors the threshold constants exported from `agents/packages/agents/src/characters/market-monitor.ts`. The system prompt is interpolated from the same config so the deterministic pipeline and the narrative layer can never drift.
Layer boundary
This agent operates at Layer 1 only — read-only, informational output, cannot submit proposals, create/modify/cancel sell orders, or execute transactions. Matches the framework principle of starting with the lowest-risk, highest-value capability. Raising the automation layer is a separate governance decision.
Test plan
Scope not included (deliberate follow-ups)
Refs `phase-2/2.2-agentic-workflows.md` §WF-MM-01, §WF-MM-02, §WF-MM-03
Refs `agents/packages/agents/src/characters/market-monitor.ts` (#64)