-
Notifications
You must be signed in to change notification settings - Fork 7
Fix editorial workflow adding code fences to files #676
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
LLMs often wrap their markdown output in ```mdx or ```markdown code fences, even when instructed to return only the content. This caused the editorial review workflow to commit files with spurious code fence markers at the beginning and end. Added stripCodeFences() helper that removes these markers from only the start and end of responses, preserving any legitimate code blocks within the content. Fixes issue seen in PR #673. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| return null; | ||
| } | ||
| const revisedContent = stripCodeFences(rawContent); | ||
| if (revisedContent === "NO_CHANGES_NEEDED") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing empty content check after stripping code fences
Medium Severity
The getEditorialFromOpenAI function lost an empty content check during the refactor. The old code checked !revisedContent which caught empty strings after trimming. The new code only checks !rawContent before calling stripCodeFences, but doesn't verify if revisedContent is empty afterward. If an LLM returns whitespace-only content or content that becomes empty after stripping code fences, the function returns a suggestion with empty revisedContent, which would overwrite the documentation file with empty content in createEditorialPR.
| result = result.replace(CODE_FENCE_OPEN_REGEX, ""); | ||
| result = result.replace(CODE_FENCE_CLOSE_REGEX, ""); | ||
| return result; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing final trim allows whitespace to break comparison
Low Severity
The stripCodeFences function trims whitespace before stripping code fences but not after. If an LLM wraps its response in code fences and includes leading/trailing whitespace inside (e.g., "```mdx\n NO_CHANGES_NEEDED \n```"), the result will be " NO_CHANGES_NEEDED " with preserved internal whitespace. This causes the "NO_CHANGES_NEEDED" equality check to fail, potentially creating an editorial PR that overwrites documentation with the literal text " NO_CHANGES_NEEDED ".
Summary
stripCodeFences()helper to remove markdown code fence markers (\``mdx,```md,```markdown, or````) from the beginning and end of LLM responsesfixValeIssues(),getEditorialFromAnthropic(), andgetEditorialFromOpenAI()Problem
The editorial review workflow was creating PRs with
\``mdxat the beginning and```` at the end of documentation files. This happened because LLMs often wrap their code/markdown output in code fences, even when explicitly instructed to return only the content.Example: PR #673 shows this issue.
Solution
The fix strips code fence markers from only the start and end of LLM responses, preserving any legitimate code blocks within the actual content.
Test plan
pnpm vale:editorial --pr <number>on a recent merged PR🤖 Generated with Claude Code
Note
Prevents LLM-generated editorial changes from being wrapped in code fences.
stripCodeFences()and regexes to remove starting/ending ```mdx/markdown fencesfixValeIssues(),getEditorialFromAnthropic(), andgetEditorialFromOpenAI()Written by Cursor Bugbot for commit 246d2f1. This will update automatically on new commits. Configure here.