Skip to content

Cross-run memory: org/repo profile that accumulates across runs #75

Description

@andthattoo

Summary

Each run is currently a blank slate. The state manager persists ExploitRecord, FixRecord, and RunRecord per run, but nothing accumulates across runs. An org/repo profile that grows over time would let the agent leverage historical context instead of starting from scratch.

Goals

  • Build an OrgProfile model that aggregates data across runs for each org/repo
  • Feed the profile into root agent context so findings are informed by history
  • Make the agent aware of the team's tech stack, common patterns, and past vulnerabilities

Proposed Changes

1. OrgProfile model

A persistent model that grows across runs:

OrgProfile
+-- tech_stack: ["Solidity 0.8.x", "Foundry", "OpenZeppelin"]
+-- vulnerability_history: [{class, count, avg_severity, fp_rate}]
+-- suppression_rules: ["skip gas optimizations", ...]
+-- codebase_patterns: ["proxy/upgradeable", "governor/timelock"]

2. Profile accumulation

After each run completes, update the OrgProfile with:

  • Detected tech stack and patterns
  • New vulnerability findings (class, severity)
  • FP rates from user feedback (depends on feedback loop feature)
  • Which findings were acted on vs dismissed

3. Profile injection into run_exploit() context

The root prompt already accepts prior_findings and instructions. Add an org_context field populated from the profile:

  • Root prompt: "This org typically uses OpenZeppelin's ReentrancyGuard, flag cases where they forgot it"
  • Analyzer prompt: "Skip these known patterns, focus on deviations"
  • Verifier prompt: "Severity calibration based on historical data for this org"

Implementation Notes

  • Mostly a state schema change + prompt injection at run_exploit() time
  • The LLM does the heavy lifting of using the context
  • Depends on the feedback loop feature for FP rates and suppression rules, but tech stack / vulnerability history can be built independently

Context

  • State infrastructure: LocalStateManager, ExploitRecord, RunRecord already exist
  • See docs/product-learning.md for full design rationale

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions