Skip to content

Llm guard#545

Closed
Ashraf-Ali-aa wants to merge 290 commits intoRunMaestro:0.16.0-RCfrom
Ashraf-Ali-aa:llm-guard
Closed

Llm guard#545
Ashraf-Ali-aa wants to merge 290 commits intoRunMaestro:0.16.0-RCfrom
Ashraf-Ali-aa:llm-guard

Conversation

@Ashraf-Ali-aa
Copy link
Copy Markdown

@Ashraf-Ali-aa Ashraf-Ali-aa commented Mar 9, 2026

Summary

This PR introduces LLM Guard, a comprehensive security scanning pipeline that protects Maestro users from sensitive data exposure and malicious content when interacting with AI agents.

Core Security Features

  • PII Detection: Identifies emails, phone numbers, SSNs, IP addresses, and credit card numbers
  • Secrets Detection: Catches API keys, tokens, passwords, and connection strings before they're sent to AI providers
  • Prompt Injection Detection: Analyzes input structure to detect potential prompt injection attacks
  • Invisible Character Detection: Identifies hidden Unicode characters and encoding-based attacks
  • Malicious URL Detection: Scans for dangerous URLs and phishing attempts
  • Dangerous Code Pattern Detection: Identifies potentially harmful code constructs
  • Output Injection Detection: Monitors AI responses for injected malicious content

User Interface

  • LlmGuardIndicator: Real-time security status display in the main interface
  • SecurityBadge: Visual indicators in the session list showing security state
  • SecurityEventsPanel: New Right Bar tab for viewing security events and findings
  • ScanProgressIndicator: Shows scanning progress during input processing
  • SanitizedContentDiff: Visual diff view showing what content was sanitized
  • SensitiveContentOverlay: Highlights sensitive content directly in the input area
  • SessionSecurityModal: Per-session security policy configuration

Settings & Configuration

  • Dedicated LLM Guard tab in Settings Modal
  • Three action modes: Warn, Sanitize, or Block
  • Custom regex patterns support for organization-specific rules
  • Configuration import/export for team sharing
  • Per-session security policy overrides
  • Security recommendations system

Integration Points

  • Input scanning with real-time preview via useSensitiveContentDetection hook
  • Toast notifications via useSecurityToasts hook
  • Group Chat inter-agent message protection
  • Keyboard shortcuts for security features
  • Security event logging system

Additional Changes

  • Spellcheck IPC handlers: Added Electron spell-check API support (get system locale, add words to dictionary, manage spell-checker languages)

Test Plan

  • Enable LLM Guard in Settings → LLM Guard tab
  • Test PII detection by typing an email address or phone number
  • Verify security badge appears on session when content is detected
  • Test sanitization mode - confirm sensitive content is replaced
  • Test block mode - confirm submission is prevented
  • Check Security Events panel in Right Bar shows findings
  • Verify per-session policies work via session context menu
  • Test custom regex patterns in settings
  • Test config export/import functionality

Files Changed

Security Core (src/main/security/llm-guard/)

  • Main scanning pipeline and detection logic
  • Pattern definitions and code scanner
  • URL scanner and recommendations system
  • Security logger and config export

UI Components (src/renderer/components/)

  • LlmGuardIndicator, SecurityBadge, SecurityEventsPanel
  • SanitizedContentDiff, ScanProgressIndicator
  • SensitiveContentOverlay, SessionSecurityModal
  • FindingDetails, SecurityRecommendationsPanel

Settings (src/renderer/components/Settings/tabs/)

  • LlmGuardTab for comprehensive configuration

Hooks (src/renderer/hooks/)

  • useSensitiveContentDetection, useSecurityToasts

Tests (src/__tests__/)

  • Comprehensive test coverage for all security features

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • LLM Guard: real-time input/output scanning (secrets, PII, prompt injection, code, URLs), per-session policies, inter-agent protection, sanitization/blocking, recommendations, toasts, Security tab with live events, filtering, diffs, and export.
    • UI: sensitive-content overlay, scanning progress indicator, session security badge, LLM Guard status indicator, spellcheck APIs and native context-menu suggestions.
  • Documentation

    • Comprehensive LLM Guard docs: quick start, configuration, patterns, examples, troubleshooting, and export/import guidance.
  • Tests

    • Extensive unit/integration tests covering security logger, recommendations, guards, UI components, and IPC handlers.

pedramamini and others added 30 commits March 1, 2026 11:25
Add inline comment and eslint-disable to the search-close effect that
intentionally omits searchOpen/onSearchClose from the dependency array.
The effect should fire only on tab changes, not prop changes — the
omission is deliberate and now documented for future readers.
- terminalTabHelpers: replace 'does not set exitCode when not provided'
  with 'clears exitCode when not provided (stale value reset)' to verify
  that prior exitCode values are cleared on state transitions

- useDebouncedPersistence: add three tests for activeTerminalTabId
  normalization — valid ID is preserved, stale ID falls back to first
  tab, and orphaned ID with empty terminalTabs falls back to null

- tabStore: add closeTerminalTab describe block with two tests covering
  the kill-after-validation ordering — PTY kill fires only after helper
  confirms the close, and is suppressed entirely when the last tab
  cannot be closed
- Resize wizard modal from max-w-5xl (1024px) to 1200px wide / 85vh tall,
  matching Director's Notes dimensions
- Add runAllDocuments toggle to PhaseReviewScreen (Step 5), visible when
  there are 2+ generated documents
- When enabled, all documents with tasks are included in the batch config
  instead of only the first
- Extract reusable ToggleSwitch component to ui/ and refactor
  SettingCheckbox to use it
- Add test for run-all-documents batch behavior

Closes RunMaestro#490
…e activity indicators

Broadcast new history entries via IPC when they are added, subscribe in
the UnifiedHistoryTab with RAF batching and deduplication, and extend
the HistoryStatsBar with spinning Active agent count and Queued message
count indicators derived from the Zustand session store.
Introduce maestro:// URL scheme support for navigating to agents, tabs,
and groups from external apps and OS notification clicks.

- Add deep-links module with URL parsing, protocol registration,
  single-instance locking, and cross-platform event handling
- Wire notification click handlers to navigate to the originating
  agent/tab via deep link dispatch
- Thread sessionId/tabId context through notification preload bridge
- Add onDeepLink listener in renderer with routing to existing
  navigation handlers
- Register maestro:// protocol in electron-builder config
- Add 18 tests covering URL parsing and notification click wiring
- Add explicit type="button" to ToggleSwitch to prevent unintended form
  submissions
- Pass ariaLabel={title} in SettingCheckbox for screen reader accessibility
…orrect types

- Replace unstable sessionNameMap Zustand selector (new Map per render) with
  a stable ref + subscribe pattern to avoid streaming effect re-subscription
- Dedupe within batch before merging; compute setTotalEntries and
  setHistoryStats from deduplicated entries only (not raw batch)
- Clear pendingEntriesRef on cleanup to prevent stale replay after resubscribe
- Use HistoryEntry (not UnifiedHistoryEntry) in preload callback type since
  the wire payload lacks sourceSessionId
- Use canonical UsageStats interface in global.d.ts (fixes pre-existing
  cacheReadTokens/cacheWriteTokens field name mismatch)
- URI-encode sessionId/tabId when constructing deep link URLs in
  notification click handler to prevent malformed URLs with special chars
- Add process.exit(0) after app.quit() so secondary instances exit
  immediately without running further module-level setup
- Use useRef for sessions in deep link effect to avoid tearing down
  and re-registering the IPC listener on every sessions change
- Guard against navigating to non-existent session IDs in deep link
  handler to prevent invalid UI state
- Add cross-reference comment in global.d.ts linking to canonical
  ParsedDeepLink type (can't import in ambient declaration file)
- Add test for URI-encoding round-trip in notification click handler
- Add shared deep-link-urls.ts with buildSessionDeepLink(),
  buildGroupDeepLink(), and buildFocusDeepLink() utilities
- Add {{AGENT_DEEP_LINK}}, {{TAB_DEEP_LINK}}, {{GROUP_DEEP_LINK}}
  template variables available in system prompts, custom AI commands,
  and Auto Run documents
- Wire activeTabId and groupId into TemplateContext at all call sites
  (agentStore, useInputProcessing, useRemoteHandlers,
  useDocumentProcessor, useMergeTransferHandlers, batch-processor)
- Refactor notifications.ts to use shared buildSessionDeepLink()
- Add sessionId/tabId to notifyToast callers where context is available
  (merge, transfer, summarize, PR creation)
- Add docs/deep-links.md documentation page with URL format, usage
  examples, template variables, and platform behavior
- Add 8 tests for URL builders, 6 tests for template variable
  substitution including URI encoding
…tions

- shell:trashItem: catch "Operation was aborted" when user cancels (MAESTRO-A4)
- shell:openPath: log warning instead of throwing for missing paths (MAESTRO-B3)
- groupChat:startModerator: catch errors when group chat not found (MAESTRO-B2)
…URL registry support

- Add `symphony: boolean` (default true) to EncoreFeatureFlags
- Gate Symphony modal, menu item, keyboard shortcut (⇧⌘Y), and command palette entry
- Add `symphonyRegistryUrls` setting for user-configured additional registry URLs
- Replace single `fetchRegistry()` with `fetchRegistries()` that fetches default + custom URLs in parallel
- Merge repositories by slug (default registry wins on conflicts), isolated per-URL error handling
- Add Symphony toggle + Registry Sources UI in Settings > Encore tab
- Update tests for new symphony flag across all encore feature assertions
Add github.pull_request and github.issue event types to CueEventType union.
Add repo and poll_minutes fields to CueSubscription interface.
Add cue_github_seen SQLite table with 5 CRUD functions for tracking
seen GitHub items (isGitHubItemSeen, markGitHubItemSeen, hasAnyGitHubSeen,
pruneGitHubSeen, clearGitHubSeenForSubscription).
Create cue-github-poller.ts module that polls GitHub CLI for new PRs/issues,
seeds existing items on first run, and fires CueEvents for new items.
Comprehensive test suite with 17 test cases covering all polling behaviors.
All 264 Cue tests pass, lint clean.
Add GitHub Pull Request and GitHub Issue event type blocks with descriptions,
YAML configuration examples, and seven new GitHub template variables (CUE_GH_*)
to the Cue Help Modal documentation.
…ed component structure

- Resolved 3 App.tsx conflicts: adapted to new useSessionListProps hook pattern,
  removed usageDashboardOpen prop (now store-sourced), kept Symphony encore gating
- Resolved 4 SettingsModal.test.tsx conflicts: adopted mockUseSettingsOverrides pattern,
  added symphony/usageStats to default encoreFeatures mock
- SessionList.tsx (modify/delete): removed old monolith, applied encore feature gating
  to new HamburgerMenuContent.tsx (Symphony, Usage Dashboard, Director's Notes)
- SettingsModal.tsx (modify/delete): removed old file, ported Symphony toggle + registry
  URL management and Usage & Stats toggle to new EncoreTab.tsx, gated stats section
  in GeneralTab.tsx behind encoreFeatures.usageStats
The custom Electron menu (added to fix Cmd+Shift+{/} tab cycling) removed
the View menu role, which provided native Cmd+=/- zoom and Cmd+0 reset.
This adds font size shortcuts directly in the renderer keyboard handler
using the existing fontSize setting (2px steps, range 10-24). Cmd+0 now
resets font size instead of goToLastTab since zoom reset is standard behavior.
- Add "Create Worktree" action to QuickActionsModal (Cmd+K) for git repo sessions
- Resolve to parent session when invoked from a worktree child
- Auto-focus newly created worktree sessions after creation
Format symphony-registry.json to fix CI prettier check failure.
Add fontSizeIncrease/Decrease/Reset to FIXED_SHORTCUTS registry
per CodeRabbit review feedback.
- Redact registry URLs before logging to prevent credential leakage
- Skip registry cache when custom source URLs are configured (stale cache fix)
- Runtime-validate symphonyRegistryUrls from settings store
- Reset modal-open flags when Encore Feature toggles are disabled
- Normalize registry URLs before duplicate/default checks
- Add aria-label to icon-only registry URL remove button
- Expose setSymphonyRegistryUrls in getSettingsActions()
- Validate persisted symphonyRegistryUrls with Array.isArray guard
Address all 20 PR RunMaestro#488 review comments:
- Fix concurrency tracking leak in cue-engine stopRun
- Fix SSH silent fallback in cue-executor (error when sshStore unavailable)
- Fix SIGKILL escalation using exitCode/signalCode check
- Add NaN guards in cue-filter numeric comparisons
- Wire real executor in onCueRun handler (was stub)
- Fix shortcut conflict (maestroCue: Meta+Shift+Q)
- Add projectRoot to CueSessionStatus in preload
- Add timeout_minutes > 0 validation in YAML loader
- Add stale response guards in CueYamlEditor and useCue
- Fix file.changed payload key in CueModal
- Add stopped status display in CueModal
- Add immediate validation on YAML load
- Tighten source_session/fan_out element typing
- Add missing cleanup in github-poller test
- Add NaN test cases for cue-filter

Port Cue integrations to refactored component structure:
- Add Cue menu item to HamburgerMenuContent
- Add Cue session tracking to SessionList
- Add Maestro Cue Encore toggle to EncoreTab

Fix .prettierignore to exclude itself from formatting.
Batch processor tracked tokens/cost/time internally but never
communicated them to Symphony. Added real-time updateStatus calls
during execution and pass accumulated stats to symphony.complete()
on finalization.
Symphony auto-finalization only sent a toast notification when the PR
was marked ready. Now also records a history entry so the PR result
appears in the session's History tab.
Default directory was ~/Maestro-Symphony/{owner}-{repo} which collides
when contributing multiple issues to the same repo. Now uses
{owner}-{repo}-{issueNumber} for unique directories per contribution.
…gnment test

Remove 3 unused store subscriptions in App.tsx that became dead after
the main merge, and update TerminalOutput test to match the corrected
user message alignment logic from PR RunMaestro#493.
pedramamini and others added 22 commits March 17, 2026 23:26
…rrors (RunMaestro#527)

OpenCode's yargs parser misinterprets leading '---' (YAML frontmatter in
slash command prompts) as flags, causing exit code 1 when resuming sessions.
Adding '--' separator before the prompt safely terminates flag parsing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…cation

Replace per-agent detect/spawn/command functions with generic implementations:
- detectAgent() replaces detectClaude/detectCodex/detectOpenCode/detectDroid
- spawnJsonLineAgent() replaces spawnCodexAgent/spawnOpenCodeAgent/spawnDroidAgent
- getAgentCommand() replaces getClaudeCommand/getCodexCommand/etc
- findCommandInPath() replaces findClaudeInPath/findCodexInPath
- Backward-compatible wrappers retained for existing consumers

Simplify run-playbook.ts and send.ts by replacing if/else chains with
single detectAgent() call driven by agent definitions.

Net -317 lines. All 114 tests pass.
…in file explorer

macOS (APFS/HFS+) returns NFD-encoded filenames from fs.readdir(), while
most systems use NFC. Two entries that look identical can have different
byte representations (e.g., é as U+00E9 NFC vs e+U+0301 NFD), causing
the existing string-comparison dedup guards to miss them.

Apply NFC normalization at three layers:
- IPC handler (fs:readDir) for both local and SSH remote branches
- loadFileTreeRecursive dedup guard (defense-in-depth)
- flattenedTree render-time dedup in FileExplorerPanel
Concurrent tree loads (initial load, auto-refresh timer, manual refresh)
could cause React to surface stale or doubled data when overlapping
async loads both call setSessions. A useRef counter now tracks load
sequence — each load captures a sequence number at start and discards
its result if a newer load has started by the time the async work
completes. Applied to refreshFileTree, refreshGitFileState, and the
initial-load effect.
…ention

Add regression tests across three layers:
- loadFileTree: NFD/NFC dedup at root and nested directory levels
- FileExplorerPanel: rendering dedup for NFD/NFC sibling entries
- IPC fs:readDir handler: NFC normalization for local and SSH entries
…ntity, catch guard

- Replace shared loadSequenceRef with per-session Map<string, number> so
  loads for different sessions don't cancel each other
- Use raw entry.name for path construction in fs:readDir IPC handler to
  preserve filesystem identity for remote operations (name still NFC)
- Use node.name (not normalizedName) for fullPath in flattenedTree to
  stay consistent with tree manipulation functions
- Add staleness re-checks after statsPromise awaits in refreshFileTree
  and initial load to prevent stale overwrites
- Add staleness guard in initial load .catch() to avoid clearing tree
  state for superseded loads
- Reset fileTreeLoading on all stale discard paths (both .then and .catch)
Parse JSON once in StdoutHandler.processLine() and pass the pre-parsed
object downstream via new parseJsonObject() and detectErrorFromParsed()
methods on AgentOutputParser. Previously each NDJSON line was parsed 3x:
once in detectErrorFromLine, once at the outer level, and once inside
parseJsonLine via handleParsedEvent. All 5 parsers now implement the
new interface methods, with existing string-based methods delegating to
them. GeminiOutputParser also caches error patterns at construction time.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The detectErrorFromParsed methods were setting raw: { parsedJson }
which violates the AgentError.raw type (only allows exitCode, stderr,
stdout, errorLine). The parsedJson data is already on the top-level
parsedJson field — no need to duplicate it in raw.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…field (TASK-S05)

- Fix skipBatchForReadOnly in buildAgentArgs() to skip batch args whenever
  readOnlyMode is true, regardless of readOnlyArgs length (empty array was
  falsy, allowing -y through for Gemini CLI)
- Add readOnlyCliEnforced field to AgentConfig for explicit CLI enforcement
  tracking (true for Claude/Codex/OpenCode/Droid, false for Gemini CLI)
- Centralize YOLO flag filtering via filterYoloArgs() utility, replacing
  hardcoded flag lists in 3 renderer spawn locations
- Log warning when readOnlyMode requested but agent has no CLI enforcement
- Replace readOnlyArgs?.length checks in group-chat-router with
  readOnlyCliEnforced for correct --no-sandbox gating
- Add 16 new tests: 7 readOnly+batchModeArgs interaction tests,
  9 filterYoloArgs utility tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add vi mock for logger module (not available in test context on main)
- Remove --skip-git from expected opencode args (not present on main)
- Fix logger context format: 'AgentArgs' → '[AgentArgs]' to match main

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove debug console.log statements from StdoutHandler.ts
- Fix spellcheck locale validation to filter against availableSpellCheckerLanguages
- Add inter_agent_scan to SecurityEventData and SecurityEvent types
- Fix empty-string replacement handling in SanitizedContentDiff.tsx (use explicit undefined check)
- Fix warn-only scans logged as input_scan (now properly uses 'warning' eventType)
- Add .catch() handler to logSecurityEvent promise in process.ts
- Forward promptLlmGuardEnabled prop to AppUtilityModals
- Don't log raw spellcheck words (log length instead)
- Fix forEach callback lint issues in recommendations.test.ts
- Replace PAT-shaped fixtures with concatenated pieces in test files
- Fix defensive selectors in MainPanel.tsx for llmGuardSettings

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The security-logger.ts module imports uuid (v4 as uuidv4) for generating
unique security event IDs, but the package was not declared in dependencies.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
StdoutHandler.ts:
- Move synopsis logging after sanitization to avoid logging raw sensitive content
- Use guardedText preview instead of raw event.text/streamedText

RightPanel.tsx:
- Extract loadSecurityEventIds as useCallback for reuse
- Update refreshSecurityEvents to also refresh badge by calling loadSecurityEventIds

group-chat-router.ts:
- Add post-loop check when all mentioned participants are blocked
- Reset state to idle and remove power block when no participants respond
- Emit system warning message when round is cancelled due to all blocks
- Fix historyContext sanitization bypass by rebuilding context per-participant
- Use sanitized message in historyContext to prevent raw message leakage

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Enhanced `docs.json` to include LLM Guard in the Encore Features section.
- Updated `encore-features.md` to reflect the addition of LLM Guard with a description and quick start guide.
- Improved `llm-guard.md` with detailed configuration options and detection types for better user guidance.
Security improvements:
- Redact sensitive values in security event logs (findings now store
  redactedPreview instead of raw values to prevent audit log from
  becoming a secondary store of secrets)
- Add zero-length regex match protection in collectMatches to prevent
  infinite loops
- Suppress raw text preview in StdoutHandler debug logs when LLM Guard
  is enabled
- Add catch handler to logSecurityEvent in StdoutHandler

UI improvements:
- Normalize overlapping findings in SanitizedContentDiff to prevent
  rendering issues
- Apply per-type highlight colors in diff views (using getHighlightColor
  instead of hardcoded red/green)
- Gate unread security event count on settings hydration to prevent
  false "all unread" state on startup
- Add race protection to loadSecurityEventIds using sequence counter

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Skip logging prompt preview on Windows when LLM Guard detected findings,
even in warn mode where effectivePrompt may still contain sensitive
content.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use activeSession.id instead of activeTab?.agentSessionId to properly
scope security events to the current session. The Maestro session ID
matches what's used in logSecurityEvent, so events are correctly
filtered to the current session.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Ashraf-Ali-aa Ashraf-Ali-aa changed the base branch from main to 0.16.0-RC March 17, 2026 23:29
@Ashraf-Ali-aa
Copy link
Copy Markdown
Author

Ashraf-Ali-aa commented Mar 17, 2026

@pedramamini I have rebased the branch, the other issues are already resolved, I used @jeffscottward Prompt to fix the CR outstanding issues , i.e the last 3 commits

@pedramamini
Copy link
Copy Markdown
Collaborator

Hey! We're consolidating onto the rc branch — 0.16.0-RC is being retired since rc already contains all its commits. Could you retarget this PR to rc? Thanks!

@pedramamini pedramamini deleted the branch RunMaestro:0.16.0-RC March 18, 2026 20:47
@Ashraf-Ali-aa
Copy link
Copy Markdown
Author

@pedramamini do you want me to open a new PR and point to rc since this one is now closed?

@Ashraf-Ali-aa
Copy link
Copy Markdown
Author

#605 @pedramamini

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants