feat: ✨ Add resilient schema parsing#9
Conversation
Separate generation schemas (strict, for AI SDK structured output)
from parse schemas (tolerant, with coercion preprocessing) to handle
common model output drift:
- String values where numbers expected (confidence, line numbers)
- Percentage confidence ("90%" → 0.9)
- Severity aliases ("Warning" → "medium", "Major" → "high")
- Side aliases ("right" → "RIGHT", "added" → "RIGHT")
- Non-array fields coerced to arrays
- Invalid objects filtered from object arrays
Add fallback from structured output to plain text JSON generation
when providers do not support structured output, with clear error
reporting when both paths fail.
Update AI SDK dependencies to latest versions.
AI 代码审查汇总PR: #9 (feat: ✨ Add resilient schema parsing) 总体评价共发现 26 条可执行问题，建议优先处理 CRITICAL/HIGH。 主要问题（按严重级别）
可执行建议
潜在风险
测试建议
文件级覆盖说明
无法 inline 的已处理项
覆盖状态
未覆盖文件清单:
无 patch 文件覆盖清单:
轮次与预算
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| if (['false', '0', 'no', 'n', 'pending'].includes(text)) { | ||
| return false; | ||
| } | ||
| return false; |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| return parsed.data; | ||
| } | ||
|
|
||
| function shouldFallbackToTextJson(error) { |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| if (!item || typeof item !== 'object' || Array.isArray(item)) { | ||
| continue; | ||
| } | ||
| const parsed = itemSchema.safeParse(item); |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| return null; | ||
| } | ||
|
|
||
| const normalized = text.includes('%') || numberValue > 1 ? numberValue / 100 : numberValue; |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| if (['NONE', 'GENERAL', 'OVERALL'].includes(text)) { | ||
| return 'FILE'; | ||
| } | ||
| return 'RIGHT'; |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| if (text.startsWith('crit') || text === 'blocker' || text === 'blocking') { | ||
| return 'critical'; | ||
| } | ||
| if (text.startsWith('hi') || text === 'major') { |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| }); | ||
| assert.equal(parsedPercent.findings[0].confidence, 0.9); | ||
|
|
||
| const parsedOutOfRange = schema.parse({ |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| /Expected number, received string/ | ||
| ); | ||
|
|
||
| assert.throws( |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
|
||
| runtime.configureRuntime({ model: createFakeModel() }); | ||
| const agent = createAgent(); | ||
| const result = await runtime.runStructuredWithRepair(agent, 'review', { allowRepair: false }); |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| } | ||
|
|
||
| const text = String(value).trim(); | ||
| const normalized = text.replace(/^[LR#:\s]+/i, ''); |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
- Change "hi" severity match from startsWith to exact match to prevent "hint" from being incorrectly classified as "high" severity - Add "hint" as a valid alias for "low" severity - Remove numberValue > 1 condition from confidence normalization so only percentage values (with % sign) are divided by 100 - Add comprehensive test coverage for confidence bounds and hint severity
| const SEVERITIES = ['critical', 'high', 'medium', 'low']; | ||
| const SIDES = ['LEFT', 'RIGHT', 'FILE']; | ||
|
|
||
| function coerceString(value) { |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| model, | ||
| instructions, | ||
| schema: plannerOutputSchema, | ||
| schema: plannerGenerationSchema, |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| return Math.min(1, Math.max(0, normalized)); | ||
| } | ||
|
|
||
| function coerceObjectArray(value, itemSchema) { |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| if (['false', '0', 'no', 'n', 'pending'].includes(text)) { | ||
| return false; | ||
| } | ||
| return false; |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| return parsed.data; | ||
| } | ||
|
|
||
| function shouldFallbackToTextJson(error) { |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| } | ||
|
|
||
| const parsed = agent.schema.safeParse(parsedObject); | ||
| const schema = agent.parseSchema || agent.schema; |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| const userPrompt = buildUserInput(agent, input, repairContext); | ||
| const model = agent.modelInstance || runtimeState.model; | ||
|
|
||
| try { |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| category: null, | ||
| path: 'src/a.js', | ||
| side: 'right', | ||
| line: 'R42', |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| severity: 'Warning', | ||
| category: null, | ||
| path: 'src/a.js', | ||
| side: 'right', |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| reason: 'keep one path' | ||
| } | ||
| ], | ||
| done: 'true', |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| const SEVERITIES = ['critical', 'high', 'medium', 'low']; | ||
| const SIDES = ['LEFT', 'RIGHT', 'FILE']; | ||
|
|
||
| function coerceString(value) { |
There was a problem hiding this comment.
[HIGH] 8个新增coerce函数无对应单元测试
src/agents.js新增的8个coerce函数承担了模型输出的核心容错解析逻辑，但目前没有任何单元测试覆盖其边界条件（如null/undefined输入、极端字符串、百分比置信度、带前缀行号等），回归风险高。
建议: 在test/agents.test.js中为每个coerce函数添加边界值测试：null/undefined/空字符串、超出范围的数值、各种别名映射（如'blocker'->critical）、百分比和0-1置信度、带L/R前缀行号等。
风险: 任何coerce逻辑的静默变更都可能导致模型输出被错误解析，直接影响review质量且难以察觉
置信度: 0.92
| return 'RIGHT'; | ||
| } | ||
|
|
||
| function coercePositiveIntegerOrNull(value) { |
There was a problem hiding this comment.
[MEDIUM] coercePositiveIntegerOrNull 边界条件缺少测试验证
coercePositiveIntegerOrNull 的多条边界路径（0、负数、前缀剥离后空串、非数字字符串）行为需要显式测试用例确认。
建议: 添加测试：输入 0→null, -5→null, 'L0'→null, '#abc'→null, 'R3'→3, ' 42 '→42, ''→null, true→null 等。
风险: 边界条件错误可能导致 line 被设为 null（丢失定位）或错误值，影响 inline comment 发布。
置信度: 0.88
| if (['false', '0', 'no', 'n', 'pending'].includes(text)) { | ||
| return false; | ||
| } | ||
| return false; |
There was a problem hiding this comment.
[MEDIUM] coerceBoolean 对无法识别的值静默返回 false，可能导致 planner done 被误判
coerceBoolean 在无法匹配时硬编码返回 false，对于 planner 的 done 字段，任何非预期值都会阻止规划器标记为完成
建议: 考虑对无法识别的值返回默认值参数（如 coerceBoolean(value, defaultValue)），或在 done 字段上使用更宽松的匹配列表（如加入 'finished'/'complete'/'done'）
风险: 规划器可能永远无法标记 done=true，导致多余轮次或超时
置信度: 0.85
| if (text.startsWith('lo') || text === 'minor' || text === 'info' || text === 'informational' || text === 'hint') { | ||
| return 'low'; | ||
| } | ||
| return 'medium'; |
There was a problem hiding this comment.
[MEDIUM] coerceSeverity 对未识别输入静默降级为 'medium'，缺少可观测性
未识别的 severity 值被静默降级为 'medium'，无日志或指标，导致输入质量问题不可追踪。
建议: 在 coerceSeverity 无法识别输入时，添加 console.warn 或 debug 日志记录原始值，或在返回对象中附加警告标记，使运营可观测。
风险: 模型输出中的 severity 拼写错误或新变体会被静默降级，掩盖数据质量问题。
置信度: 0.85
| return Number.isInteger(numberValue) && numberValue > 0 ? numberValue : null; | ||
| } | ||
|
|
||
| function coerceConfidence(value) { |
There was a problem hiding this comment.
[MEDIUM] coerceConfidence 百分比/小数混合解析边界缺少测试
coerceConfidence 涉及百分比/小数双模式、夹紧逻辑和 NaN 处理，边界条件复杂但缺少测试。
建议: 添加测试：'85%'→0.85, '0.85'→0.85, '150%'→1, '1.5'→1, '0'→0, '100%'→1, NaN→null, '-50%'→0, Infinity→null。
风险: confidence 解析错误可能导致 finding 被错误过滤或排序，影响评审优先级。
置信度: 0.85
| if (['NONE', 'GENERAL', 'OVERALL'].includes(text)) { | ||
| return 'FILE'; | ||
| } | ||
| return 'RIGHT'; |
There was a problem hiding this comment.
[MEDIUM] coerceSide 对未识别输入默认为 'RIGHT'，可能导致行号定位偏差
coerceSide 默认返回 'RIGHT' 可能导致原本属于 LEFT（旧代码）的行号被错误标记为新代码侧，影响评审定位。
建议: 对无法识别的 side 值记录警告，或在 coerceSide 返回时标记是否为 fallback 值，便于下游（如 diff-map.js）做额外校验。
风险: 行号定位偏差会导致 inline comment 发布到错误的 diff 侧，GitHub API 可能直接拒绝或误导开发者。
置信度: 0.82
| const parsed = itemSchema.safeParse(item); | ||
| if (parsed.success) { | ||
| out.push(parsed.data); | ||
| } |
There was a problem hiding this comment.
[MEDIUM] coerceObjectArray 静默丢弃未通过 safeParse 的对象，发现项可能无声丢失
coerceObjectArray 在 safeParse 失败时静默跳过，可能导致 LLM 生成的发现项被无声丢弃，用户无法得知审查结果不完整
建议: 添加计数或日志记录被丢弃的项数，至少在 debug 级别输出；或返回附带错误信息的结果以便上层处理
风险: 重要安全发现可能因字段缺失被静默丢弃，审查结果不完整且无感知
置信度: 0.82
| model, | ||
| instructions, | ||
| schema: plannerOutputSchema, | ||
| schema: plannerGenerationSchema, |
There was a problem hiding this comment.
[MEDIUM] generation/parse schema 双层架构缺少集成测试
generation 和 parse schema 的协作逻辑未被测试覆盖，特别是 coerceObjectArray 在 parseSchema 中对未能通过 itemSchema.safeParse 的项被静默丢弃（R133-R136）。
建议: 添加集成测试：构造一个包含部分无效项的模型输出，验证 parseSchema 正确保留有效项、丢弃无效项，且最终结果符合 reviewOutputSchema。
风险: 如果 safeParse 过于严格丢弃了有价值但格式微偏差的 finding，会导致评审结果静默丢失。
置信度: 0.80
| return parsed.data; | ||
| } | ||
|
|
||
| function shouldFallbackToTextJson(error) { |
There was a problem hiding this comment.
[MEDIUM] shouldFallbackToTextJson启发式匹配缺乏测试且脆弱
shouldFallbackToTextJson依赖消息字符串的启发式匹配来决定是否回退到文本JSON模式，但没有测试验证匹配逻辑，且匹配条件脆弱——provider SDK更新可能改变错误消息措辞。
建议: 为shouldFallbackToTextJson添加测试覆盖各种已知和未知的错误消息模式，并考虑添加更稳健的匹配策略（如错误类型/代码检查优先于消息匹配）。
风险: 回退逻辑失效会导致原本可恢复的structured output失败变成硬失败，降低系统可用性
置信度: 0.88
| return parseStructuredText(agent, result.text || ''); | ||
| } | ||
|
|
||
| function parseStructuredObject(agent, outputObject) { |
There was a problem hiding this comment.
[MEDIUM] parseStructuredObject和parseStructuredText双路径解析缺少集成测试
新的双路径解析中，parseStructuredObject和parseStructuredText都会优先使用agent.parseSchema进行二次解析（容错coerce），但缺少测试验证当generation schema解析成功而parse schema产生不同结果时的行为。
建议: 添加测试：构造一个generation schema接受但parse schema会进一步coerce的对象，验证最终输出符合parse schema的约束。
风险: 如果parseSchema的coerce逻辑与generation schema行为不一致，可能产生静默的数据变形
置信度: 0.85
1 participant
Summary
No object generated.Test Plan
node --test test/agents.test.js test/model-runtime.test.jsnpm run checknpm testnpm run test:schema-supportwith API credentials🤖 Generated with TiyCode