Skip to content

Fix editorial workflow adding code fences to files #676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nearestnabors merged 1 commit into from
Jan 21, 2026
Merged

Fix editorial workflow adding code fences to files #676

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 23 additions & 5 deletions scripts/vale-editorial.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ const MAX_AI_TOKENS = 8192;
const OWNER = "ArcadeAI";
const REPO = "docs";
const EDITORIAL_COMMENT_REGEX = /<!-- Editorial: (.+?) -->/;
const CODE_FENCE_OPEN_REGEX = /^```(?:mdx?|markdown)?\n/;
const CODE_FENCE_CLOSE_REGEX = /\n```$/;
const HTTP_UNPROCESSABLE_ENTITY = 422;
const MEGABYTE = KILOBYTE * KILOBYTE;
const MAX_BUFFER_MB = 10;
Expand Down Expand Up @@ -84,6 +86,15 @@ type ValeIssue = {

type ValeOutput = Record<string, ValeIssue[]>;

// Strip markdown code fences from the beginning and end of LLM responses
// LLMs often wrap their output in ```mdx or ```markdown blocks
function stripCodeFences(content: string): string {
let result = content.trim();
result = result.replace(CODE_FENCE_OPEN_REGEX, "");
result = result.replace(CODE_FENCE_CLOSE_REGEX, "");
return result;
}
Copy link

@cursor cursor bot Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing final trim allows whitespace to break comparison

Low Severity

The stripCodeFences function trims whitespace before stripping code fences but not after. If an LLM wraps its response in code fences and includes leading/trailing whitespace inside (e.g., "```mdx\n NO_CHANGES_NEEDED \n```"), the result will be " NO_CHANGES_NEEDED " with preserved internal whitespace. This causes the "NO_CHANGES_NEEDED" equality check to fail, potentially creating an editorial PR that overwrites documentation with the literal text " NO_CHANGES_NEEDED ".

Fix in Cursor Fix in Web


// Run Vale on content and return issues
function runValeOnContent(filename: string, content: string): ValeIssue[] {
// Create temp directory and file with correct extension
Expand Down Expand Up @@ -155,15 +166,18 @@ async function fixValeIssues(
messages: [{ role: "user", content: prompt }],
});
const textBlock = response.content.find((b) => b.type === "text");
return textBlock?.type === "text" ? textBlock.text : content;
return textBlock?.type === "text"
? stripCodeFences(textBlock.text)
: content;
}

const response = await ai.client.chat.completions.create({
model: "gpt-4-turbo",
max_tokens: MAX_AI_TOKENS,
messages: [{ role: "user", content: prompt }],
});
return response.choices[0]?.message?.content ?? content;
const result = response.choices[0]?.message?.content;
return result ? stripCodeFences(result) : content;
} catch (error) {
console.error("Error fixing Vale issues:", error);
return content;
Expand Down Expand Up @@ -266,7 +280,7 @@ async function getEditorialFromAnthropic(
return null;
}

const revisedContent = textBlock.text.trim();
const revisedContent = stripCodeFences(textBlock.text);

if (revisedContent === "NO_CHANGES_NEEDED") {
return null;
Expand Down Expand Up @@ -298,8 +312,12 @@ async function getEditorialFromOpenAI(
messages: [{ role: "user", content: prompt }],
});

const revisedContent = response.choices[0]?.message?.content?.trim();
if (!revisedContent || revisedContent === "NO_CHANGES_NEEDED") {
const rawContent = response.choices[0]?.message?.content;
if (!rawContent) {
return null;
}
const revisedContent = stripCodeFences(rawContent);
if (revisedContent === "NO_CHANGES_NEEDED") {
Copy link

@cursor cursor bot Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing empty content check after stripping code fences

Medium Severity

The getEditorialFromOpenAI function lost an empty content check during the refactor. The old code checked !revisedContent which caught empty strings after trimming. The new code only checks !rawContent before calling stripCodeFences, but doesn't verify if revisedContent is empty afterward. If an LLM returns whitespace-only content or content that becomes empty after stripping code fences, the function returns a suggestion with empty revisedContent, which would overwrite the documentation file with empty content in createEditorialPR.

Fix in Cursor Fix in Web

return null;
}

Expand Down