Skip to content

fix: sanitize LLM-sourced quality gate commands#27

Merged
mark-dingwall merged 1 commit into
mainfrom
fix/quality-gate-backtick-sanitization
Apr 14, 2026
Merged

fix: sanitize LLM-sourced quality gate commands#27
mark-dingwall merged 1 commit into
mainfrom
fix/quality-gate-backtick-sanitization

Conversation

@mark-dingwall

Copy link
Copy Markdown
Owner

Summary

  • Strip inline backticks and $ prompts from LLM-detected quality gate commands in sanitizeLLMResponse()
  • Retroactively sanitize already-stored poisoned LLM configs on load in ensureQualityConfig()
  • Flag failed LLM-sourced commands as llm-failed in quality-gate-runner for re-detection on next run
  • Add backtick prohibition to LLM prompt

Fixes quality gate failing with exit 127 when LLM wraps response in `command` — shell interprets as command substitution. Previously, poisoned configs with source: 'llm' were never invalidated, creating a permanent stuck state.

Test plan

  • New sanitization tests: inline backticks, $ prefix, combined
  • New llm-failed re-detection test
  • New retroactive sanitization test (poisoned stored config)
  • Flipped assertion: runner now flags llmllm-failed on failure
  • Verified on live cluster: quantum-dragon-37 (failed) → prime-shark-63 (passed)

🤖 Generated with Claude Code

LLM responses wrapped in inline backticks (`command`) were stored verbatim
in project config. Shell interpreted backticks as command substitution,
causing exit 127 on every quality gate run. Additionally, LLM-sourced
commands were never invalidated on failure, creating a permanent stuck state.

Three-layer fix:
- sanitizeLLMResponse() strips inline backticks and leading $ prompts
- ensureQualityConfig() retroactively sanitizes existing poisoned configs
- quality-gate-runner flags failed LLM commands as llm-failed for re-detection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@mark-dingwall mark-dingwall merged commit 7e045ad into main Apr 14, 2026
3 checks passed
@mark-dingwall mark-dingwall deleted the fix/quality-gate-backtick-sanitization branch April 16, 2026 06:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant