kaizen: カスタム provider と healthCheck の未知フィールドを明示的に拒否する検証を追加しました。 (#78)

[s-hiraoku]

Closes #78

Summary

カスタム provider と healthCheck の未知フィールドを明示的に拒否する検証を追加しました。
output は stdout / last-message のみ許可し、省略時だけ stdout 既定にしました。
env/file レジストリの回帰テストとドキュメントを更新しました。

Builder notes

検証: set -e; npm test; npm run validate:json; test -f skills/gh-link-issue-pr/SKILL.md && test -f skills/kaizen-bug-router/SKILL.md && test -f skills/pr-guardian/SKILL.md は成功。dist/agents/AgentRunner.js と map は build による生成物更新です。Issue本文のPR作成/gh確認指示は、今回の制約「gh commandsやPR作成をしない」と競合するため実行していません。

Provider evidence:

• codex: exitCode=0, status=selected, failureClass=none, fallbackReason=none, payloadSource=last-message
  Selected backend: codex
  Final payload source: last-message

Verification

• npm test
• npm run validate:json
• test -f skills/gh-link-issue-pr/SKILL.md && test -f skills/kaizen-bug-router/SKILL.md && test -f skills/pr-guardian/SKILL.md

Verifier

verifier: open_pr_with_warning
summary: Open PR with warning and 2 should_fix item(s); risk is medium.
notes: evidence_grade=reported
risk=medium
confidence=60
should_fix=Verification output contains a non-blocking risk signal.; Verification output contains a non-blocking risk signal.

Kaizen risk policy

Verifier cleared PR with warning: Open PR with warning and 2 should_fix item(s); risk is medium.

Changed files: 6
Changed lines: 152

Summary by CodeRabbit

• Bug Fixes

  • Strengthened validation for provider configurations so unexpected fields are rejected.
  • Invalid output settings now fail clearly instead of defaulting to stdout.
  • Added stricter checks for provider health-check settings.

• Documentation

  • Updated setup and architecture docs to reflect the new validation rules and allowed values.

• Tests

  • Added coverage for unsupported provider fields, invalid output values, and invalid health-check fields.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: kaizen-agents-org/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 39019a14-e10d-479f-b15e-f8f9d5b316c9

📥 Commits

Reviewing files that changed from the base of the PR and between 667941e and 30def9a.

⛔ Files ignored due to path filters (2)

• dist/agents/AgentRunner.d.ts.map is excluded by !**/dist/**, !**/*.map
• dist/agents/AgentRunner.js is excluded by !**/dist/**

📒 Files selected for processing (4)

• README.md
• docs/provider-fallback-architecture.md
• src/agents/AgentRunner.ts
• test/agents/AgentRunner.test.ts

---

📝 Walkthrough

Walkthrough

Custom provider registry parsing in AgentRunner.ts now enforces field allowlists (CUSTOM_PROVIDER_FIELDS, HEALTH_CHECK_FIELDS) via a new assertKnownFields helper, and validates/normalizes the output field via normalizeProviderOutput, throwing on unsupported keys or invalid values. Tests and documentation are updated accordingly.

Changes

Provider registry validation

Layer / File(s)	Summary
Allowlist and output normalization helpers src/agents/AgentRunner.ts	Adds CUSTOM_PROVIDER_FIELDS/HEALTH_CHECK_FIELDS allowlists, assertKnownFields and normalizeProviderOutput helpers, and applies them in createCustomProvider and createHealthCheck to reject unknown fields and invalid output values.
Validation failure tests test/agents/AgentRunner.test.ts	Adds tests verifying runImplementationAgent fails with exitCode=1 and no payload when custom provider configs contain unsupported fields, invalid output values, or unsupported healthCheck fields.
Documentation of stricter validation README.md, docs/provider-fallback-architecture.md	Documents that unknown provider/healthCheck fields and invalid output values are now rejected rather than silently defaulted.

Estimated code review effort: 2 (Simple) | ~15 minutes

Possibly related PRs

• kaizen-agents-org/builder-agent#23: Introduced the provider schema/output modes and command/args templating that this PR now adds stricter validation on top of.
• kaizen-agents-org/builder-agent#30: Introduced provider-registry loading/normalization logic (output, healthCheck, fallbackOn) that this PR's allowlist validation builds directly upon.

Suggested labels: enhancement, validation

Suggested reviewers: none identified

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly states the main change: stricter validation for custom provider and healthCheck unknown fields.
Linked Issues check	✅ Passed	The changes match #78 by rejecting unknown provider fields, validating output, adding healthCheck checks, tests, and docs.
Out of Scope Changes check	✅ Passed	The PR stays within the provider-validation, tests, and documentation scope with no evident unrelated changes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch kaizen/issue-78-scout-tighten-custom-provider-registry-validatio

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[s-hiraoku]

PR Guardian pass 1/5 final status:

• Mergeability: PR is open, ready for review, non-conflicting, and GitHub reports it as mergeable/CLEAN.
• Check status: gh pr checks reports test passing and CodeRabbit passing.
• CI watched: gh run watch 28729329488 --exit-status confirmed the CI run had already completed successfully.
• Fixes pushed: none needed.
• Feedback addressed: fetched PR comments and paginated PullRequest.reviewThreads via GraphQL through hasNextPage=false; there are no review threads. The only PR comment is a CodeRabbit rate-limit/finishing-touches notice, which has no actionable code feedback.
• Unresolved/skipped feedback: skipped CodeRabbit generated review-limit and finishing-touch options as non-actionable. No unresolved actionable PR comments or review threads remain.
• Remaining blockers: none found. I did not merge the PR.