Add Simulation Engine and Adaptive Attack Engine#61
Conversation
Simulation Engine (src/simulation/): - Mock tool environment: 5 tool types (API, DB, memory, credential, communication) - 20-probe battery across 7 categories (exfiltration, override resistance, scope expansion, credential, persistence, heartbeat, semantic consistency) - 3-layer architecture: NanoMind TME (8ms) -> targeted probes (3s) -> full sim (30s) - Target: < 1% false positive rate vs industry 95.8% Adaptive Attack Engine (src/attack-engine/): - Target reader: semantic vulnerability surface extraction - Payload generator: target-specific attacks using target's own language - Feedback loop: attack -> observe -> adapt -> iterate (up to 5 per category) - Defense map: complete portrait of what held and what failed - Training pipeline: auto-exports labeled pairs from every scan/attack to NanoMind corpus CLI UX redesign: - NanoMind auto-enabled by default when daemon is available - hackmyagent secure: smart defaults (developer persona) - hackmyagent secure --deep: full simulation + attacks (security engineer) - hackmyagent secure --static-only: deterministic (CI persona) - hackmyagent attack <file>: dedicated adaptive red team command - hackmyagent explain <findingId>: NanoMind-powered explanations 1,113 tests pass (33 new: 23 simulation + 10 attack engine).
- LLM executor: 3 backends (NanoMind daemon, Anthropic Claude, Ollama) - Auto-detects best available backend (NanoMind > Ollama > Anthropic) - Loads skill as system prompt, injects probe as user message - Observes tool calls via structured output patterns - Analyzes response for behavioral signals per probe category - Falls back to heuristic analysis when no LLM is available - Production CLI passes useLLM:true, tests use heuristic (fast, deterministic)
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Claude Code Review
Security Review: PR #61 - Add Simulation Engine and Adaptive Attack Engine
VERDICT: APPROVE
SUMMARY:
This PR introduces a comprehensive behavioral simulation engine and adaptive attack engine to HackMyAgent. The architecture is sound with three-layer simulation (NanoMind → quick probes → full behavioral analysis) and adaptive red-team attack generation. I've verified all critical code paths for security issues. The code uses safe patterns throughout: no shell interpolation in command execution, path operations are properly scoped, regex patterns are linear-time, external API calls are properly contained within test/simulation contexts, and file operations use appropriate bounds checks. The training data pipeline writes to user home directory with proper path construction. No unmitigated security vulnerabilities were found.
FINDINGS:
None. All potential security concerns have adequate mitigations in place:
- Path traversal prevention verified: Training pipeline (line 19) uses
join(homedir(), '.opena2a', 'training-data')with static path components — no user input interpolation - No command injection risk: No use of
exec(),spawn()with shell mode, or string-interpolated command construction - Regex DoS prevention verified: All regex patterns use linear-time constructs (alternation with bounded quantifiers, no nested repetition like
(a+)+) - External API calls properly scoped:
callExternalAPI()in mock-tools.ts (lines 48-70) is simulation infrastructure that logs calls without executing them — correct design for a security testing tool - JSON parsing safety: All
JSON.parse()calls wrapped in try-catch blocks or used on controlled/validated input - File operations bounded: Training pipeline limits input to 4096 bytes (lines 69, 81, 146) and uses
slice(0, 4096)before writing - Type safety maintained: TypeScript strict mode patterns throughout, proper null checks on optional properties
The adaptive attack engine correctly implements security research tooling — it generates attack payloads for testing purposes, which is its intended function. The mock tool environment properly intercepts and logs tool calls without executing them, which is the correct design for a behavioral simulator.
Reviewed 18 files changed (131427 bytes)
1 participant
Summary
hackmyagent attack <file>,--deepfor full sim, NanoMind default-onTest plan
npx vitest run)tsc --noEmit)