Skip to content

feat(troubleshoot): add artifact-first investigation path for traces and solution files#2088

Open
MariaVimer wants to merge 2 commits into
UiPath:mainfrom
MariaVimer:feat/troubleshoot-artifact-investigation
Open

feat(troubleshoot): add artifact-first investigation path for traces and solution files#2088
MariaVimer wants to merge 2 commits into
UiPath:mainfrom
MariaVimer:feat/troubleshoot-artifact-investigation

Conversation

@MariaVimer

Copy link
Copy Markdown

What

Adds an artifact-first investigation path to the uipath-troubleshoot skill — guidance on extracting evidence from customer-provided files (.uis/.uipx, trace JSON, HAR, screenshots, eval/debug run outputs) before issuing uip CLI commands. Also adds two previously undocumented agent failure classes.

Why

Production case (Bilfinger SE) exposed a gap: the skill had no way to direct an investigator to unzip and inspect a .uis solution file or correlate cross-agent schemas. Both failure modes in that case were only diagnosable from artifacts that were already in hand — not from CLI queries:

  • Failure Mode 1 (CHAT_MODEL.UNSUCCESSFUL_STOP_REASON, 0 completion tokens, 4m22s): visible in the trace's completion span, but the contributing factor — a schema naming mismatch where the upstream agent outputs snake_case and the downstream input schema defines PascalCase — was only visible after unzipping both .uis files and comparing outputSchema vs inputSchema.
  • Failure Mode 2 (17,808 completion tokens, all-null agentOutput, job status = success): a silent failure not surfaced by Orchestrator. Triggered by inputSkontoBetrag: "AmountMatched" — an intentional sentinel value emitted by the extraction agent on a specific data condition — that the matching agent's prompt had no instruction for. Only diagnosable by reading both agents' prompts side by side.

Changes

SKILL.md

  • Adds a step 0 artifact inventory table before the uip traces anchor step — investigators triage what files they have and what each one yields before reaching for the CLI
  • Extends the anchored branch (step 2) to include .uis/.uipx/.flow/.bpmn as primary evidence sources
  • Strengthens the silent-failure routing in §4 to name "all-null agentOutput" explicitly and state that these failures are not detectable from Orchestrator job status alone

references/investigation_guide.md

  • Adds a new Customer-Provided Artifacts section (before Data Correlation) with:
    • Artifact-type → what to extract table covering solution files, trace JSON, HAR, screenshots, and eval/debug run outputs
    • Solution file unzip + cross-agent schema comparison instructions
    • Per-SpanType trace extraction table (completion, agentRun, agentOutput, llmCall)

references/products/agents/summary.md

  • Mandates checking the agentOutput span even when the job reports success
  • Documents two new failure classes:
    • Schema naming mismatch — upstream outputs snake_case, downstream input schema defines PascalCase; runtime populates both, producing null PascalCase alongside actual values in the rendered prompt
    • Semantic contract mismatch — upstream outputs a controlled-vocabulary value (sentinel, status marker, code) the downstream agent's prompt has no instruction for; results in undefined model behavior on a specific input path, detectable only by reading both prompts against actual trace input values

Scope

Changes are additive and backward-compatible. No existing playbooks, schemas, or agent files are modified. No hooks touched.


Created with Claude. Claude can make mistakes. Please double-check responses.

…and solution files

Addresses a coverage gap exposed by a production agent failure case
(Bilfinger SE, two intermittent failure modes on Gemini 2.5 Flash PTU):
the skill had no guidance on extracting evidence from customer-provided
artifacts (.uis/.uipx, trace JSON, HAR, screenshots) before issuing CLI
commands, and no documented failure class for silent all-null agentOutput
or for cross-agent schema/semantic contract mismatches.

Changes:
- SKILL.md: add §2 step 0 artifact inventory table; extend anchored
  branch to include .uis/.uipx/.flow/.bpmn; strengthen §4 silent-failure
  routing to name all-null agentOutput explicitly
- references/investigation_guide.md: add Customer-Provided Artifacts
  section with artifact→insight table, solution file unzip instructions,
  and per-SpanType trace extraction table
- references/products/agents/summary.md: mandate agentOutput span check
  even on succeeded jobs; add schema naming mismatch and semantic
  contract mismatch as documented failure classes with detection guidance
…s summary

Schema naming mismatch and semantic contract mismatch were derived from
one production case (Bilfinger). Insufficient evidence to document them
as general failure classes. Retain only the agentOutput null-check
mandate, which reflects documented agent runtime behavior.
@JosephMar
JosephMar requested a review from saksharthakkar July 16, 2026 17:17

When the user provides files directly, extract evidence from them before running CLI commands. Artifact types and what to extract:

**Solution files (`.uis`, `.uipx`, `.flow`, `.bpmn`, `caseplan.json`)** — zipped archives. Unzip and read:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only uis, uipx are zipped files? Can we generalize this to zip files?
something like, ", zipped archives — unzip, read the content"

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only uis, uipx are zipped files? Can we generalize this to zip files? something like, ", zipped archives — unzip, read the content"

.uis and .uipx are what we call a 'solution' in our product - when you build any kind of automation, you can export it as a .uis or the entire package as a .uipx.

.uis/.uipx aren't archived files and definitely not a .zip.
They are bundles of related components users build in cloud (RPA workflows, AI agents, Apps, assets, and queues) into a single deployable package.

When a user wants to debug, often you need the solution for a better root-cause analysis (you need to see what is actually running on our platform). That's the .uis/.uipx solution. If Claude/Codex can see that solution it will have better context for root cause analysis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants