chore: snapshot local worktree changes#631
Conversation
There was a problem hiding this comment.
Pull request overview
This snapshot PR bundles several unrelated worktree changes across the web UI, API/runtime wiring, automation (Alpaca reconciliation), discovery strategy deployment deduping, and assorted tests/migrations.
Changes:
- Adds Alpaca reconciliation as an automation job + admin API/CLI endpoints, plus metrics and extensive unit tests.
- Introduces a migration to deduplicate discovery-generated paper strategies and enforces uniqueness via a partial unique index; bumps required schema version.
- Updates the web dashboard/activity UX (historical events + recent runs), plus various robustness tweaks and new component tests.
Reviewed changes
Copilot reviewed 53 out of 53 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| web/src/pages/strategies-page.tsx | Removes strategy list filter UI/state; simplifies query to fetch all strategies. |
| web/src/pages/pipeline-run-page.tsx | Adds safer guards for phase_timings and config_snapshot rendering. |
| web/src/pages/order-detail-page.tsx | Refactors “options info” rendering to use typed access helpers instead of repeated casts. |
| web/src/pages/dashboard-page.tsx | Adds the new “Recent runs” widget to the dashboard layout. |
| web/src/pages/dashboard-page.test.tsx | Updates dashboard test to stub runs/events endpoints and assert recent runs + activity content. |
| web/src/components/universe/watchlist-table.tsx | Changes score rendering to rely solely on score field (see PR comment re API mismatch). |
| web/src/components/universe/watchlist-table.test.tsx | Adds basic rendering/navigation tests for the watchlist table. |
| web/src/components/pipeline/decision-inspector.tsx | Adjusts structured output rendering condition to handle falsy-but-present values. |
| web/src/components/pipeline/decision-inspector.test.tsx | Adds a unit test covering structured output rendering. |
| web/src/components/dashboard/recent-runs.tsx | New dashboard widget fetching/displaying recent pipeline runs. |
| web/src/components/dashboard/recent-runs.test.tsx | Tests recent runs widget (happy path + empty state). |
| web/src/components/dashboard/activity-feed.tsx | Adds historical event fetch via REST and merges with live websocket feed; improves item rendering. |
| web/src/components/dashboard/activity-feed.test.tsx | Updates tests to account for REST-seeded events and react-query wrapper. |
| web/src/components/backtests/backtest-equity-chart.tsx | Makes tooltip formatters resilient to non-number values/labels. |
| web/src/components/backtests/backtest-equity-chart.test.tsx | Adds tests for non-empty and empty chart states (with recharts mocked). |
| migrations/discovery_strategy_dedup_migration_test.go | Adds SQL-fragment assertions + integration test for dedup migration behavior. |
| migrations/000031_discovery_strategy_dedup.up.sql | Deduplicates discovery/options paper strategies and adds partial unique index. |
| migrations/000031_discovery_strategy_dedup.down.sql | Drops the partial unique index. |
| internal/repository/postgres/strategy_test.go | Extends test schema to include options market_type and adds unique-index rejection test. |
| internal/repository/postgres/schema_version_test.go | Makes “ahead” test relative to RequiredSchemaVersion. |
| internal/repository/postgres/schema_version.go | Bumps RequiredSchemaVersion to 31. |
| internal/metrics/metrics_test.go | Adds test coverage for new Alpaca reconcile metrics counter. |
| internal/metrics/metrics.go | Adds AlpacaReconcileRunsTotal metric and RecordAlpacaReconcileRun. |
| internal/discovery/orchestrator.go | Uses CreateOrReusePaperStrategy to avoid duplicate paper strategy creation. |
| internal/discovery/options/orchestrator.go | Same reuse logic for options discovery deploy path. |
| internal/discovery/options/generator_test.go | Adds retry/empty-response tests and verifies prompt schema token changes. |
| internal/discovery/options/generator.go | Treats empty parsed rules as an error; updates schema docs from operator→op. |
| internal/discovery/deploy_test.go | Adds tests for create-or-reuse strategy behavior + unique-violation detection. |
| internal/discovery/deploy.go | Implements CreateOrReusePaperStrategy with “list then create” + unique-violation retry. |
| internal/data/polygon/universe_test.go | Adds a rate-limit spacing test that currently requires real-time waiting (see PR comment). |
| internal/data/polygon/universe.go | Increases inter-page delay to 12s to respect Polygon free-tier limits. |
| internal/cli/root.go | Adds automation alpaca-reconcile CLI command that hits the admin endpoint. |
| internal/cli/automation_test.go | Tests the new CLI command against a stub HTTP server. |
| internal/automation/orchestrator_test.go | Adds tests for job last summary cloning and Alpaca reconcile metrics/summary behavior. |
| internal/automation/orchestrator.go | Adds LastSummary plumbing, cloning helpers, and registers broker reconciliation jobs. |
| internal/automation/jobs_broker_reconciliation.go | New scheduled job registration + implementation wrapper for Alpaca reconciliation. |
| internal/automation/alpaca_reconciliation_test.go | Large unit test suite for reconciliation + verification behavior and orchestrator job presence. |
| internal/automation/alpaca_reconciliation.go | Implements Alpaca reconcile/verify services + audit logging and snapshot mapping. |
| internal/automation/alpaca_client_adapter_test.go | Adds adapter tests; pagination test currently doesn’t exercise pagination (see PR comment). |
| internal/automation/alpaca_client_adapter.go | Implements Alpaca API adapter for orders/fills/positions snapshots. |
| internal/api/server.go | Wires Alpaca admin endpoints and injects reconciler dependency into the server. |
| internal/api/request_logger_test.go | Adds tests for request logging suppression behavior. |
| internal/api/middleware.go | Suppresses request logs for probe/high-volume paths. |
| internal/api/automation_handlers_test.go | Adds test ensuring automation status includes Alpaca last summary. |
| internal/api/automation_alpaca_handlers_test.go | Tests the new Alpaca admin reconcile endpoint behavior. |
| internal/api/automation_alpaca_handlers.go | New handlers/interfaces for /automation/alpaca/verify and /automation/alpaca/reconcile. |
| internal/agent/runner_test.go | Adds a panic-in-phase test to ensure run is marked failed and emits PipelineError. |
| internal/agent/runner.go | Adds panic recovery to mark runs failed + persist terminal events; introduces unconditional DEBUG logs (see PR comment). |
| cmd/tradingagent/runtime_test.go | Updates schema version expectations; adds test ensuring Alpaca reconcile job is wired. |
| cmd/tradingagent/runtime.go | Wires Alpaca client adapter + reconciler into API deps and automation orchestrator. |
| .task/checksum/build | New local task checksum artifact. |
| .hermes/plans/2026-04-22-alpaca-reconciliation.md | Adds local plan doc for Alpaca reconciliation work. |
| .hermes/plans/2026-04-20_165747-hermes-integration.md | Removes older Hermes integration plan doc. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| <Badge variant={scoreBadgeVariant(t.score)}> | ||
| {t.score.toFixed(2)} | ||
| </Badge> |
There was a problem hiding this comment.
/api/v1/universe/watchlist returns universe.TrackedTicker JSON with a watch_score field (see internal/api/universe_handlers.go), not score. With this change, t.score can be undefined at runtime and toFixed() will throw. Please restore compatibility (e.g., read watch_score when score is absent) or update the API to return score consistently before relying on it here.
| func TestListActiveTickersRespectsFreeTierRateLimit(t *testing.T) { | ||
| var firstRequestAt time.Time | ||
| var serverURL string | ||
|
|
||
| server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | ||
| w.Header().Set("Content-Type", "application/json") | ||
|
|
||
| switch r.URL.Query().Get("cursor") { | ||
| case "": | ||
| firstRequestAt = time.Now() | ||
| _, _ = fmt.Fprintf(w, `{"results":[{"ticker":"AAA","name":"Alpha","primary_exchange":"XNAS","type":"CS","active":true}],"next_url":"%s/v3/reference/tickers?cursor=page-2"}`, | ||
| serverURL, | ||
| ) | ||
| case "page-2": | ||
| if time.Since(firstRequestAt) < 11*time.Second { | ||
| w.WriteHeader(http.StatusTooManyRequests) | ||
| _, _ = w.Write([]byte(`{"status":"ERROR","request_id":"req-rate","error":"rate limit exceeded"}`)) | ||
| return | ||
| } | ||
| _, _ = w.Write([]byte(`{"results":[{"ticker":"BBB","name":"Beta","primary_exchange":"XNYS","type":"CS","active":true}]}`)) |
There was a problem hiding this comment.
This test enforces a real-time 11s+ delay to validate rate limiting, which will slow the unit test suite significantly and may cause CI timeouts/flakiness. Consider injecting a clock/sleeper into the Polygon client so the production code can wait 12s while tests can advance time instantly (or assert that the client requested the second page only after the injected delay).
| // Rate limit pause: Polygon free tier allows 5 req/min, so paginated | ||
| // reference-ticker requests need roughly 12s spacing to avoid 429s. | ||
| select { | ||
| case <-ctx.Done(): | ||
| return tickers, ctx.Err() | ||
| case <-time.After(250 * time.Millisecond): | ||
| case <-time.After(12 * time.Second): | ||
| } |
There was a problem hiding this comment.
The hard-coded 12 * time.Second sleep will make ListActiveTickers extremely slow for multi-page universes (12s per page) and also makes the behavior hard to test. Consider centralizing this behind a configurable rate limiter / injected sleep function (defaulting to 12s for free-tier) and/or switching to a 429-aware backoff so paid-tier or higher limits aren’t unnecessarily throttled.
| func TestAlpacaClientAdapterListFills_PaginatesActivities(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| var requestCount int | ||
| server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | ||
| if r.Method != http.MethodGet { | ||
| t.Fatalf("request method = %s, want %s", r.Method, http.MethodGet) | ||
| } | ||
| if r.URL.Path != "/v2/account/activities/FILL" { | ||
| t.Fatalf("request path = %s, want %s", r.URL.Path, "/v2/account/activities/FILL") | ||
| } | ||
| requestCount++ | ||
| w.Header().Set("Content-Type", "application/json") | ||
| switch requestCount { | ||
| case 1: | ||
| if got := r.URL.Query().Get("page_token"); got != "" { | ||
| t.Fatalf("first page_token = %q, want empty", got) | ||
| } | ||
| _, _ = w.Write([]byte(`[ | ||
| { | ||
| "activity_type": "FILL", | ||
| "id": "fill-2", | ||
| "order_id": "order-1", | ||
| "symbol": "SNAL", | ||
| "side": "buy", | ||
| "qty": "14", | ||
| "price": "0.92", | ||
| "transaction_time": "2026-04-15T19:20:04.943982Z", | ||
| "order_status": "filled" | ||
| }, | ||
| { | ||
| "activity_type": "FILL", | ||
| "id": "fill-1", | ||
| "order_id": "order-1", | ||
| "symbol": "SNAL", | ||
| "side": "buy", | ||
| "qty": "186", | ||
| "price": "0.92", | ||
| "transaction_time": "2026-04-15T19:20:02.66268Z", | ||
| "order_status": "partially_filled" | ||
| } | ||
| ]`)) | ||
| case 2: | ||
| if got := r.URL.Query().Get("page_token"); got != "fill-1" { | ||
| t.Fatalf("second page_token = %q, want fill-1", got) | ||
| } | ||
| _, _ = w.Write([]byte(`[]`)) | ||
| default: | ||
| t.Fatalf("unexpected request count %d", requestCount) | ||
| } | ||
| })) | ||
| defer server.Close() | ||
|
|
||
| client := alpacaexec.NewClient("test-key", "test-secret", true, slog.New(slog.NewTextHandler(testDiscardWriter{}, nil))) | ||
| client.SetBaseURL(server.URL) | ||
|
|
||
| adapter := NewAlpacaClientAdapter(client) | ||
| fills, err := adapter.ListFills(context.Background()) | ||
| if err != nil { | ||
| t.Fatalf("ListFills() error = %v", err) | ||
| } | ||
| if requestCount != 1 { | ||
| t.Fatalf("requestCount = %d, want 1", requestCount) | ||
| } |
There was a problem hiding this comment.
TestAlpacaClientAdapterListFills_PaginatesActivities doesn’t actually exercise pagination: the first response has 2 items (< page_size=100), so ListFills should stop after 1 request. The test name + server handler’s case 2 suggest you intended to validate the page_token follow-up call. Either rename the test to reflect single-page behavior, or make the first response length equal to alpacaActivitiesPageSize so the adapter must request the second page and you can assert page_token usage.
| return changed | ||
| } | ||
|
|
||
| func fillDedupeKey(fill BrokerFillSnapshot) string { |
There was a problem hiding this comment.
fillDedupeKey ignores the broker-provided ActivityID and dedupes purely on (ticker, side, qty, price, executed_at). If Alpaca emits multiple fills with identical execution fields (common for small partial fills), this will collapse distinct fills into one trade and/or make reconcile/verify non-idempotent. Consider incorporating ActivityID into the dedupe key and persisting it (e.g., a dedicated column / unique index) so repeated reconciliations can deterministically avoid duplicates without risking dropped fills.
| func fillDedupeKey(fill BrokerFillSnapshot) string { | |
| func fillDedupeKey(fill BrokerFillSnapshot) string { | |
| if activityID := strings.TrimSpace(fill.ActivityID); activityID != "" { | |
| return strings.Join([]string{"activity", activityID}, "|") | |
| } |
| // Run executes one prepared run and returns the canonical result. | ||
| func (r *Runner) Run(ctx context.Context, prepared PreparedRun) (*RunResult, error) { | ||
| func (r *Runner) Run(ctx context.Context, prepared PreparedRun) (result *RunResult, runErr error) { | ||
| slog.Info("DEBUG: runner.Run entered", slog.String("run_id", prepared.RunID.String())) |
There was a problem hiding this comment.
There are unconditional slog.Info("DEBUG: ...") statements using the package-level logger. These will ship as noisy production logs and bypass the runner’s configured logger/level. Please remove them or switch to r.logger.Debug(...) (or equivalent) behind a debug flag so normal runs aren’t spammed.
| slog.Info("DEBUG: runner.Run entered", slog.String("run_id", prepared.RunID.String())) |
|
@copilot apply changes based on the comments in this thread |
|
@copilot apply changes based on the comments in this thread |
1 similar comment
|
@copilot apply changes based on the comments in this thread |
Summary
Notes
Validation