test(uipath-troubleshoot): add classic Click Simulate-on-Java silent no-op scenario

[Stefan-Virgil]

What

e2e diagnose scenario exercising the classic Click silent no-op playbook (added in #2079).

A classic Click (UiPath.Core.Activities.Click) with SimulateClick=True in a Java desktop automation reports Successful but never actuates the Submit button — job Successful, zero Error logs, record still in Draft. The agent must route via the no-signature table to the classic click-silent-no-op.md playbook, confirm the no-op from runtime logs, and identify the input-method/target-tech cause + Default (hardware events) fix.

Fixtures

• Manifest-driven uip mock (data/m/r/): folders, jobs list/get (Successful), Error logs (empty), Info logs (status still Draft); docsai ask is passthrough.
• Frozen classic project source (process/): classic ui:Click with SimulateClick="True", Java full selector, no post-action check.
• RESOLUTION.md reference for the judge.

Grading — suite two-criteria convention

skill_triggered + llm_judge against RESOLUTION.md (threshold 0.7). No command_executed (source-required scenario; the agent can diagnose from Main.xaml alone).

Hint hygiene

The agent-visible surface (process/ + data/) and the task description: field carry no diagnosis hints — only the symptom and grading. The input-method/target-tech knowledge lives in the skill's playbook; the reference answer lives only in RESOLUTION.md.

Validation

3/3 SUCCESS over 3 parallel reps (experiments/default.yaml, --repeats 3 -j 3): skill_triggered=1.0 and llm_judge=1.0 in every rep (score 1.000). Transcript confirms each rep routed via the no-signature table → Classic Activities → click-silent-no-op.md.

Merge order

Depends on #2079 for correct CI grading — merge #2079 first.

🤖 Generated with Claude Code

[github-actions]

Claude finished @Stefan-Virgil's task in 3m 20s —— View job

---

PR Review: test(uipath-troubleshoot) — classic Click Simulate-on-Java silent no-op scenario

• Read review criteria and project conventions
• Read full PR diff
• Read existing PR conversation
• Read all changed files in full
• Validate task YAML structure and tags
• Check for cross-skill references, secrets, link integrity
• Post final review

---

Summary

Adds an e2e diagnose test scenario for the classic Click silent no-op playbook (from #2079). The scenario exercises the case where SimulateClick=True on a Java target silently does nothing — job Successful, zero errors, claim still in Draft. Fixtures include a manifest-driven mock, a frozen classic project with the misconfigured Click, and a reference RESOLUTION.md for the LLM judge.

Change-by-Change Review

1. task.yaml — Task definition

Severity: OK

Well-structured. Follows all suite conventions:

• task_id matches skill-troubleshoot-classic-click-noop-simulate pattern ✓
• Tags [uipath-troubleshoot, rpa, e2e, mode:diagnose] — correct skill tag first, rpa domain tag for activity-package scenario, e2e tier (not smoke), valid mode:diagnose ✓
• run_limits uses canonical values (task_timeout: 5400, max_turns: 60, turn_timeout: 3600) ✓
• sandbox correctly sets mock_path_dirs: ["m"] and overlays _shared/mock_template, process/, data/ ✓
• success_criteria follows the mandatory two-criteria convention: skill_triggered + llm_judge only — no forbidden command_executed/file_exists/file_contains ✓
• llm_judge prompt is the exact canonical block from the suite CLAUDE.md ✓
• simulation block present with appropriate persona/constraints ✓
• allowed_tools includes AskUserQuestion and TodoWrite alongside standard tools, matching sibling tasks ✓
• reference: file: RESOLUTION.md correctly wired ✓

2. RESOLUTION.md — Ground truth for LLM judge

Severity: OK

Thorough and well-structured. Identifies the root cause (SimulateClick on Java = silent no-op), the contributing factor (no Verify Execution on classic Click), provides runtime evidence (Info logs showing status still Draft), source evidence (Main.xaml selector targeting java.exe with SimulateClick=True), and a clear immediate + preventive fix. The investigation summary table with hypothesis confidence/status is a nice touch. No secrets or PII.

3. README.md — Scenario documentation

Severity: Low

Clear explanation of what the test covers, how it reproduces the issue, and success criteria. One note:

• Line 10: References skills/uipath-troubleshoot/references/activity-packages/classic-activities/playbooks/click-silent-no-op.md which doesn't exist on this branch. The PR description correctly notes the dependency on feat(uipath-troubleshoot): add classic Click silent no-op playbook #2079 — this is expected and acceptable given the stated merge order.

4. data/m/r/manifest.json — Mock dispatch rules

Severity: OK

6 rules, well-ordered:

• Rule precedence is correct — the more specific or jobs logs ... --level Error (→ 40errl.json, empty errors) comes before the generic or jobs logs ... (→ 50infl.json, Info logs) ✓
• docsai ask correctly set to passthrough: true ✓
• unmocked_default returns empty JSON array with exit 0 — safe fallback ✓
• No shadow issues detected in rule ordering ✓

5. data/m/r/10fold.json — Folders fixture

Severity: OK

Returns Finance and Shared folders. The Finance folder key matches the job's FolderKey in 30jobg.json ✓

6. data/m/r/20jobs.json — Jobs list fixture

Severity: OK

Two Successful jobs for ClassicClaimsSubmit, most recent matching the job key used throughout. Machine is MOCK-HOST (properly scrubbed) ✓

7. data/m/r/30jobg.json — Job detail fixture

Severity: OK

Detailed job record: State: Successful, EntryPointPath: Main.xaml, HostMachineName: MOCK-HOST, email is rpa-service@contoso.com (fictional domain). No PII ✓

8. data/m/r/40errl.json — Error logs fixture (empty)

Severity: OK

Empty Data array — correctly models the "zero Error logs" symptom ✓

9. data/m/r/50infl.json — Info logs fixture

Severity: OK

6 log entries showing the execution flow: process start → Click start/end → Get Text end → "Claim status after submit: Draft" → process end. The Draft status is the key runtime evidence. Timestamps are coherent and sequential ✓

10. process/Main.xaml — Frozen classic project

Severity: OK

Clean classic XAML with the exact fault condition: ui:Click with SimulateClick="True" targeting app='java.exe' with a <java role='push button' .../> selector. Followed by GetText and LogMessage — no post-click assertion. Activity names are neutral business names (Click 'Submit Claim', not diagnostic hints) ✓

11. process/project.json — Project manifest

Severity: OK

Dependencies: UiPath.System.Activities [23.10.5], UiPath.UIAutomation.Activities [23.10.7]. Classic design profile, CSharp expression language, Windows target framework. Standard structure ✓

12. process/project.uiproj — Project descriptor

Severity: OK

Minimal 6-line file with Name, ProjectType, Description, MainFile. Clean ✓

What's Missing

Nothing identified. The scenario is complete and follows all suite conventions. The dependency on #2079 for the playbook file is documented and the merge order is specified in the PR body.

Area Ratings

Area	Status	Notes
Frontmatter	N/A	Test task, not a skill SKILL.md
E2E Tests	OK	Proper e2e tag, two-criteria convention, canonical judge block
Skill Body	N/A	No skill changes in this PR
References & Assets	OK	All fixture files consistent; README reference to #2079 playbook is expected
Repo Hygiene	OK	CODEOWNERS covered by wildcard; no secrets/PII; MOCK-HOST + contoso.com; neutral naming

Issues for Manual Review

• Merge ordering: PR depends on feat(uipath-troubleshoot): add classic Click silent no-op playbook #2079 for the click-silent-no-op.md playbook. The README reference at line 10 will be a dangling link until feat(uipath-troubleshoot): add classic Click silent no-op playbook #2079 merges. Ensure feat(uipath-troubleshoot): add classic Click silent no-op playbook #2079 merges first as stated.
• Validation claim: PR description states 3/3 SUCCESS with score 1.000 across 3 parallel reps. This cannot be independently verified from the PR diff alone.

Conclusion

Clean, well-crafted test scenario that follows all troubleshoot suite conventions exactly. Task YAML uses the canonical two-criteria shape, manifest rules are correctly ordered with no shadow issues, fixtures are internally consistent, process files carry no diagnostic hints in names, and PII is properly scrubbed. No issues found that would block merge. Ship it after #2079.

---

[github-actions]

Claude finished @Stefan-Virgil's task in 2m 26s —— View job

---

Coder-eval task lint (advisory)

1 task YAML changed; verdicts: 0 Critical, 0 High, 0 Medium, 0 Low, 1 OK.

Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge.

Evidence of passing run

✅ Author confirms passing run: "3/3 SUCCESS over 3 parallel reps (experiments/default.yaml, --repeats 3 -j 3): skill_triggered=1.0 and llm_judge=1.0 in every rep (score 1.000)."

Per-task lint

tests/tasks/uipath-troubleshoot/activity-packages/classic-click-noop-simulate/task.yaml — verdict: OK

No issues. Task follows the canonical troubleshoot suite two-criteria pattern (skill_triggered + llm_judge against RESOLUTION.md). Prompt is symptom-only with no procedure leak — mentions "Java claims app" (user context, not diagnosis) but does not reference SimulateClick, input methods, or investigation steps. RESOLUTION.md is not in the sandbox template_sources, preventing the agent from reading the answer. No near-duplicates among classic-* siblings (MoveFile/source-not-found tests a different activity and failure mode; OpenApp/wrong-scope-type likewise). run_limits correctly at top level. No env_packages.

Within-PR duplicates

No duplicate clusters detected (single task added).

Conclusion

✅ All changed tasks pass the rubric. Evidence of passing run confirmed.

---