chore(harness): complete agentic transformation#178
Conversation
WalkthroughThis PR introduces an AI harness infrastructure system for GenUI SDK enabling orchestrated agent-based repository development. It adds comprehensive workflow documentation, feature validation tracking, environment initialization scripts, and updates repository entry points with unified documentation comments. ChangesAI Harness Infrastructure
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint skipped: no ESLint configuration detected in root package.json. To enable, add Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (4)
scripts/ai-init.sh (2)
55-55: ⚡ Quick winAvoid suppressing test failure diagnostics.
Redirecting stderr to
/dev/null(2>/dev/null) hides test failure details that would help developers diagnose issues. When tests fail, line 58 only prints a generic warning without showing the underlying error.🔍 Proposed fix to preserve error output
- if pnpm --filter `@opentiny/genui-sdk-core` test -- --run 2>/dev/null; then + if pnpm --filter `@opentiny/genui-sdk-core` test -- --run; then ok "`@opentiny/genui-sdk-core` 单测通过" else echo "[ai-init] WARN: core 单测未通过或脚本不可用（可设 SKIP_AI_TESTS=1 仅检查工件）"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ai-init.sh` at line 55, The test invocation suppresses stderr by appending `2>/dev/null` to the `pnpm --filter `@opentiny/genui-sdk-core` test -- --run` command, which hides failure diagnostics; remove the `2>/dev/null` redirection (or redirect stderr to a logfile) so that test error output is preserved while keeping the existing `if` conditional behavior that checks the command's exit status.
58-61: ⚡ Quick winClarify exit behavior when tests fail.
The comment on line 59 says "工件检查已通过时仍允许开工" (still allow starting work when artifact checks pass), but line 60 exits with code 1, which will halt pipelines and prevent continuing. This contradicts the intent expressed in the comment.
If the intent is to allow work to proceed when artifacts are valid but tests fail, consider:
- Exiting with 0 and relying on the WARNING to alert developers, or
- Documenting that exit code 1 is intentional to enforce test-passing before starting work.
Option 1: Allow proceeding with warning
echo "[ai-init] WARN: core 单测未通过或脚本不可用（可设 SKIP_AI_TESTS=1 仅检查工件）" - # 工件检查已通过时仍允许开工，但返回非 0 提醒修复 - exit 1 + # 工件检查已通过，允许开工但需修复测试 + ok "环境检查完成（但需修复测试），可开始 Agent 会话" + exit 0Option 2: Clarify that tests must pass
echo "[ai-init] WARN: core 单测未通过或脚本不可用（可设 SKIP_AI_TESTS=1 仅检查工件）" - # 工件检查已通过时仍允许开工，但返回非 0 提醒修复 + # 核心测试必须通过才能开始（或设置 SKIP_AI_TESTS=1 跳过） exit 1🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ai-init.sh` around lines 58 - 61, The comment and behavior conflict: the WARN echo ("[ai-init] WARN: core 单测未通过或脚本不可用（可设 SKIP_AI_TESTS=1 仅检查工件）") says work may proceed when artifact checks pass but the script currently calls "exit 1", which halts pipelines. Decide and implement one of two fixes: (A) to allow proceeding, change the exit code after the WARN from "exit 1" to "exit 0" (keep the WARN and optionally add a note about SKIP_AI_TESTS) so pipelines continue; or (B) to enforce tests, update the comment/message to explicitly state that failure will abort the run and keep "exit 1" so the behavior is documented. Locate the lines with the WARN echo and the "exit 1" in scripts/ai-init.sh to apply the chosen change.scripts/ai-verify.sh (2)
138-142: Document ai-init.sh failure behavior.When
RUN_INIT=1, line 140 executesbash scripts/ai-init.shwithout explicit error handling. Due toset -euo pipefailon line 7, ifai-init.shexits with a non-zero code (which it does when core tests fail per line 60 of ai-init.sh), this script will halt immediately.This may be intentional (ensuring init succeeds before proceeding), but should be documented for clarity.
📝 Optional: Add comment documenting behavior
# --- 可选：运行 ai-init --- if [[ "$RUN_INIT" == "1" ]]; then info "执行 scripts/ai-init.sh ..." + # 注意: 若 ai-init.sh 失败（如核心测试未通过），本脚本会立即退出 bash scripts/ai-init.sh echo "" fi🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ai-verify.sh` around lines 138 - 142, The block that runs ai-init.sh when RUN_INIT=1 can cause the whole script to exit if ai-init.sh returns non-zero because scripts/ai-verify.sh uses set -euo pipefail; update scripts/ai-verify.sh by adding a clear comment above the RUN_INIT block stating that when RUN_INIT=1, invoking bash scripts/ai-init.sh will abort the verifier on any non-zero exit (per set -euo pipefail) and that this is intentional to fail-fast on init errors (or mention alternative handling if desired), referencing RUN_INIT and ai-init.sh so future readers understand the behavior.
111-124: Consider simplifying section printing logic.The
print_sectionfunction uses a complex chain ofsed | tail | sed | whileto extract and format sections from the Node.js report. While functional, this approach is fragile if the marker format changes and harder to maintain.♻️ Optional: More robust section extraction
Consider using awk or a simpler approach:
print_section() { local title="$1" local marker="$2" echo -e "${CYAN}## ${title}${NC}" awk -v marker="$marker" ' /^---'"$marker"'---$/ { in_section=1; next } /^---/ && in_section { exit } in_section && NF { if (/^commands:/) { print " " $0 } else { print " - " $0 } } ' <<< "$REPORT" echo "" }This is more readable and handles edge cases more predictably.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ai-verify.sh` around lines 111 - 124, The print_section function's sed/tail/sed/while pipeline is fragile; replace it with a single awk-based extraction that uses the marker variable to set an in_section flag, prints lines until the next --- header, skips blank lines, prefixes lines starting with "commands:" with four spaces and other lines with " - ", and preserves the existing color/heading echo; update the body of print_section to call awk with marker="$marker" and redirect REPORT into awk to make the logic simpler and more robust.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@AGENTS.md`:
- Around line 70-73: The fenced code block containing "pwd → 读本文件 →
ai-progress.md → ai-features.json → git log --oneline -20 → bash
scripts/ai-init.sh → 选特性 ID → 声明本会话目标" is missing a language tag and triggers
MD040; update that fenced block by adding a language identifier (e.g., text)
after the opening backticks so it becomes ```text ... ``` to satisfy the
markdown linter.
In `@ai-features.json`:
- Around line 25-27: Update ai-features.json so every entry with "passes": true
also has the corresponding "commit" field populated with the exact commit SHA
(or PR merge commit) that was used to verify the pass; specifically edit the
entries shown (the block with "passes": true, "evidence": "bash
scripts/ai-init.sh 退出码 0（含 core vitest 45 passed）", currently with "commit": "")
and likewise the other mentioned blocks (lines covering entries 44-47, 64-67,
85-88, 103-106, 122-125, 141-144, 160-163, 177-180, 196-199) to add the
verifier-approved commit id, ensuring L2/L3 features only get their "passes",
"evidence", and "commit" updated after Evaluator approval.
In `@ai-review.md`:
- Line 27: The review marks `bash scripts/ai-init.sh` and `pnpm ai:verify` as
pending but already shows PASS on line 7; either finish running and recording
the harness verification results for `ai-init.sh` and `ai:verify` and update
line 27 and the `evidence` field in ai-features.json, then keep the PASS in the
conclusion and set `passes: true`, or change the conclusion to a conditional
state (e.g., "CONDITIONAL PASS pending harness script validation") and do not
set `passes: true` or update `evidence` in ai-features.json until the harness
verification for those commands is complete.
In `@scripts/ai-init.sh`:
- Around line 33-40: Update the REQUIRED array in scripts/ai-init.sh to match
the files validated by scripts/ai-verify.sh: add ai-review.md,
scripts/ai-verify.sh, docs/inner-docs/ai-harness-prompts.md, and
.cursor/rules/genui-harness.mdc so both scripts validate the same artifacts;
locate the REQUIRED declaration in ai-init.sh and append these missing filenames
(preserving array syntax and ordering if needed) so ai-init.sh cannot succeed
while ai-verify.sh later fails.
---
Nitpick comments:
In `@scripts/ai-init.sh`:
- Line 55: The test invocation suppresses stderr by appending `2>/dev/null` to
the `pnpm --filter `@opentiny/genui-sdk-core` test -- --run` command, which hides
failure diagnostics; remove the `2>/dev/null` redirection (or redirect stderr to
a logfile) so that test error output is preserved while keeping the existing
`if` conditional behavior that checks the command's exit status.
- Around line 58-61: The comment and behavior conflict: the WARN echo
("[ai-init] WARN: core 单测未通过或脚本不可用（可设 SKIP_AI_TESTS=1 仅检查工件）") says work may
proceed when artifact checks pass but the script currently calls "exit 1", which
halts pipelines. Decide and implement one of two fixes: (A) to allow proceeding,
change the exit code after the WARN from "exit 1" to "exit 0" (keep the WARN and
optionally add a note about SKIP_AI_TESTS) so pipelines continue; or (B) to
enforce tests, update the comment/message to explicitly state that failure will
abort the run and keep "exit 1" so the behavior is documented. Locate the lines
with the WARN echo and the "exit 1" in scripts/ai-init.sh to apply the chosen
change.
In `@scripts/ai-verify.sh`:
- Around line 138-142: The block that runs ai-init.sh when RUN_INIT=1 can cause
the whole script to exit if ai-init.sh returns non-zero because
scripts/ai-verify.sh uses set -euo pipefail; update scripts/ai-verify.sh by
adding a clear comment above the RUN_INIT block stating that when RUN_INIT=1,
invoking bash scripts/ai-init.sh will abort the verifier on any non-zero exit
(per set -euo pipefail) and that this is intentional to fail-fast on init errors
(or mention alternative handling if desired), referencing RUN_INIT and
ai-init.sh so future readers understand the behavior.
- Around line 111-124: The print_section function's sed/tail/sed/while pipeline
is fragile; replace it with a single awk-based extraction that uses the marker
variable to set an in_section flag, prints lines until the next --- header,
skips blank lines, prefixes lines starting with "commands:" with four spaces and
other lines with " - ", and preserves the existing color/heading echo; update
the body of print_section to call awk with marker="$marker" and redirect REPORT
into awk to make the logic simpler and more robust.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 88648432-d778-4388-9f9e-c08ac0ee8696
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (18)
.cursor/rules/genui-harness.mdcAGENTS.mdCONTRIBUTING.zh-CN.mdREADME.zh-CN.mdai-features.jsonai-progress.mdai-review.mdai-spec.mddocs/inner-docs/AI_HARNESS.mddocs/inner-docs/USAGE.mddocs/inner-docs/ai-harness-prompts.mdpackage.jsonpackages/core/src/index.tspackages/frameworks/vue/src/index.tspackages/server/README.mdpackages/server/src/index.tsscripts/ai-init.shscripts/ai-verify.sh
| ``` | ||
| pwd → 读本文件 → ai-progress.md → ai-features.json → git log --oneline -20 | ||
| → bash scripts/ai-init.sh → 选特性 ID → 声明本会话目标 | ||
| ``` |
There was a problem hiding this comment.
Add a language identifier to the fenced code block.
The code fence at Line 70 is missing a language tag, which triggers markdown lint (MD040).
Suggested fix
-```
+```text
pwd → 读本文件 → ai-progress.md → ai-features.json → git log --oneline -20
→ bash scripts/ai-init.sh → 选特性 ID → 声明本会话目标</details>
<!-- suggestion_start -->
<details>
<summary>📝 Committable suggestion</summary>
> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
```suggestion
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 70-70: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@AGENTS.md` around lines 70 - 73, The fenced code block containing "pwd → 读本文件
→ ai-progress.md → ai-features.json → git log --oneline -20 → bash
scripts/ai-init.sh → 选特性 ID → 声明本会话目标" is missing a language tag and triggers
MD040; update that fenced block by adding a language identifier (e.g., text)
after the opening backticks so it becomes ```text ... ``` to satisfy the
markdown linter.
| "passes": true, | ||
| "evidence": "bash scripts/ai-init.sh 退出码 0（含 core vitest 45 passed）", | ||
| "commit": "" |
There was a problem hiding this comment.
Populate commit for each passes: true feature to keep validation traceable.
All completed entries are marked passes: true, but commit is empty. That weakens reproducibility and makes it hard to audit which change actually satisfied each acceptance set.
Based on learnings: "Do not set 'passes: true' without verification; L2/L3 features require Evaluator PASS" and "For L2/L3 ... obtain Evaluator approval before updating passes, evidence, and commit fields in ai-features.json".
Also applies to: 44-47, 64-67, 85-88, 103-106, 122-125, 141-144, 160-163, 177-180, 196-199
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ai-features.json` around lines 25 - 27, Update ai-features.json so every
entry with "passes": true also has the corresponding "commit" field populated
with the exact commit SHA (or PR merge commit) that was used to verify the pass;
specifically edit the entries shown (the block with "passes": true, "evidence":
"bash scripts/ai-init.sh 退出码 0（含 core vitest 45 passed）", currently with
"commit": "") and likewise the other mentioned blocks (lines covering entries
44-47, 64-67, 85-88, 103-106, 122-125, 141-144, 160-163, 177-180, 196-199) to
add the verifier-approved commit id, ensuring L2/L3 features only get their
"passes", "evidence", and "commit" updated after Evaluator approval.
| - `pnpm --filter @opentiny/tiny-schema-renderer build` → exit 0 | ||
| - `pnpm --filter genui-sdk-docs build` → exit 0 | ||
| - `genui-sdk-playground-web` dev + `curl http://localhost:5173/` → 200 | ||
| - `bash scripts/ai-init.sh` / `pnpm ai:verify` → 待最终确认 |
There was a problem hiding this comment.
Verify completion of harness scripts before finalizing PASS.
Line 27 marks bash scripts/ai-init.sh and pnpm ai:verify as "待最终确认" (pending final confirmation), but the overall conclusion on Line 7 is already PASS. For an L2 harness evaluation, the harness verification tooling itself should be validated before authorizing all features to passes: true.
Based on learnings, L2 features require Evaluator PASS with complete verification before updating passes and evidence fields in ai-features.json.
✅ Suggested clarification
Either:
- Complete the verification of
ai-init.shandai:verify, update line 27 with results, then confirm PASS, or - Change the conclusion to "CONDITIONAL PASS pending harness script validation" until verification is complete.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ai-review.md` at line 27, The review marks `bash scripts/ai-init.sh` and
`pnpm ai:verify` as pending but already shows PASS on line 7; either finish
running and recording the harness verification results for `ai-init.sh` and
`ai:verify` and update line 27 and the `evidence` field in ai-features.json,
then keep the PASS in the conclusion and set `passes: true`, or change the
conclusion to a conditional state (e.g., "CONDITIONAL PASS pending harness
script validation") and do not set `passes: true` or update `evidence` in
ai-features.json until the harness verification for those commands is complete.
| REQUIRED=( | ||
| AGENTS.md | ||
| ai-spec.md | ||
| ai-features.json | ||
| ai-progress.md | ||
| scripts/ai-init.sh | ||
| docs/inner-docs/AI_HARNESS.md | ||
| ) |
There was a problem hiding this comment.
Synchronize required artifact list with ai-verify.sh.
The REQUIRED array lists only 6 harness files, but scripts/ai-verify.sh (lines 30-41) validates 10 files, including ai-review.md, scripts/ai-verify.sh itself, docs/inner-docs/ai-harness-prompts.md, and .cursor/rules/genui-harness.mdc. This inconsistency means ai-init.sh may succeed while ai-verify.sh later fails due to missing artifacts.
🔧 Proposed fix to align artifact lists
REQUIRED=(
AGENTS.md
ai-spec.md
ai-features.json
ai-progress.md
+ ai-review.md
scripts/ai-init.sh
+ scripts/ai-verify.sh
docs/inner-docs/AI_HARNESS.md
+ docs/inner-docs/ai-harness-prompts.md
+ .cursor/rules/genui-harness.mdc
)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| REQUIRED=( | |
| AGENTS.md | |
| ai-spec.md | |
| ai-features.json | |
| ai-progress.md | |
| scripts/ai-init.sh | |
| docs/inner-docs/AI_HARNESS.md | |
| ) | |
| REQUIRED=( | |
| AGENTS.md | |
| ai-spec.md | |
| ai-features.json | |
| ai-progress.md | |
| ai-review.md | |
| scripts/ai-init.sh | |
| scripts/ai-verify.sh | |
| docs/inner-docs/AI_HARNESS.md | |
| docs/inner-docs/ai-harness-prompts.md | |
| .cursor/rules/genui-harness.mdc | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/ai-init.sh` around lines 33 - 40, Update the REQUIRED array in
scripts/ai-init.sh to match the files validated by scripts/ai-verify.sh: add
ai-review.md, scripts/ai-verify.sh, docs/inner-docs/ai-harness-prompts.md, and
.cursor/rules/genui-harness.mdc so both scripts validate the same artifacts;
locate the REQUIRED declaration in ai-init.sh and append these missing filenames
(preserving array syntax and ordering if needed) so ai-init.sh cannot succeed
while ai-verify.sh later fails.
| 在仓库根目录： | ||
|
|
| * - renderer：Schema 渲染（GenuiRenderer） | ||
| * - config-provider：主题与全局配置 | ||
| * 子路径导出见 package.json exports（./chat、./renderer、./config-provider）。 | ||
| */ |
2 participants
chore(harness): complete agentic transformation
Summary by CodeRabbit
Documentation
Chores