feat: add dashboard-aware presentation tools to cve-impact and fleet-inventory skills by TomerFi · Pull Request #131 · RHEcosystemAppEng/agentic-plugins

TomerFi · 2026-07-02T19:57:52Z

Summary

Add load_cve_dashboard to the cve-impact skill workflow, prerequisites, and dependencies
Add load_inventory_dashboard to the fleet-inventory skill workflow, prerequisites, and dependencies
Both skills instruct the client to call the respective dashboard presentation tool with collected data after fetching results

These presentation tools are implemented in insights-mcp #389. They detect UI support at runtime — rendering a dashboard on UI-capable clients or falling back to text on others.

For full context on the approach investigation and decision, see APPENG-5287 and the MCP UI Integration Strategy ADR.

Validation

validate_skills_tier1.py (cve-impact)          ✅ passed, 0 warnings
validate_skills_tier1.py (fleet-inventory)     ✅ passed, 0 warnings
validate_skills_tier2.py (cve-impact)          ⚠️ passed with 4 warnings (pre-existing, same on main)
validate_skills_tier2.py (fleet-inventory)     ⚠️ passed with 5 warnings (pre-existing, same on main)
validate_skill_doc_links.py (cve-impact)       ✅ 14 links checked, 0 errors
validate_skill_doc_links.py (fleet-inventory)  ✅ 6 links checked, 0 errors
validate_docs_tree_links.py (cve-impact)       ✅ 104 files scanned, 0 errors
validate_docs_tree_links.py (fleet-inventory)  ✅ 104 files scanned, 0 errors

Recordings

CVE Dashboard (cve-impact):

Fleet Inventory Dashboard (fleet-inventory):

Related tickets

APPENG-4742 — SRE Application (parent)
APPENG-4782 — Inventory dashboard
APPENG-4783 — Vulnerability dashboard
APPENG-5287 — Dashboard integration approach investigation

…inventory skills Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

dmartinol · 2026-07-08T07:20:47Z

[Critical] Text fallback path removed from 01-account-cves.md

The old Step 4 contained the complete text-based result handling:

Parse response using references/01-cve-response-parser.py (not jq or inline Python)
Sort by CVSS score (highest first) or by affected system count
Provide summary table: CVE ID, severity, affected systems count, remediation availability

These instructions are removed entirely by this PR. The new Step 4 instructs the agent to call load_cve_dashboard before parsing or printing anything, and the tool description says "falls back to text if unavailable" — but there is now no guidance on what that text fallback looks like. If load_cve_dashboard fails or the client does not support UI, the agent has nothing to follow.

The removed instructions should be preserved as a fallback branch (e.g. "If dashboard unavailable / falls back to text: …").

dmartinol · 2026-07-08T07:20:54Z

[Moderate] Asymmetric dashboard coverage across CVE flows

The load_cve_dashboard call is added only to flow 01-account-cves.md (account-level). Flows 02-system-all-cves.md and 03-system-remediatable-cves.md collect the same kind of CVE data but receive no dashboard rendering. The user experience varies depending on which flow the agent enters, with no clear justification for the asymmetry.

dmartinol · 2026-07-08T07:21:02Z

[Architectural] Consider a dedicated presentation skill

Both skills now call dashboard MCP tools directly. This goes against the repo's core principle: "Skills encapsulate tools — never call MCP tools directly; always invoke skills."

A dedicated render-cve-dashboard / render-inventory-dashboard skill (or a single render-dashboard skill parameterized by type) would:

Restore the encapsulation invariant
Resolve the asymmetric flow coverage naturally — any flow that collects CVE or host data invokes the skill, coverage is automatic
Keep the fallback text logic (parse, sort, summary table) in one place rather than per-flow
Allow the remediation skill and any future skill collecting CVE/host data to reuse the same presentation layer

The trade-off is one additional skill file per dashboard type.

TomerFi · 2026-07-13T15:35:17Z

[Critical] Text fallback path removed from 01-account-cves.md

The old Step 4 contained the complete text-based result handling:

Parse response using references/01-cve-response-parser.py (not jq or inline Python)

Sort by CVSS score (highest first) or by affected system count

Provide summary table: CVE ID, severity, affected systems count, remediation availability

These instructions are removed entirely by this PR. The new Step 4 instructs the agent to call load_cve_dashboard before parsing or printing anything, and the tool description says "falls back to text if unavailable" — but there is now no guidance on what that text fallback looks like. If load_cve_dashboard fails or the client does not support UI, the agent has nothing to follow.

The removed instructions should be preserved as a fallback branch (e.g. "If dashboard unavailable / falls back to text: …").

@dmartinol The fallback is handled server-side — load_cve_dashboard detects whether the client supports MCP Apps. When UI is supported, the tool returns content instructing the model not to print a table. When it's not, the tool returns content instructing the model to print the full CVE table from its prior get_cves response. The model already has the CVE data in context at that point, so it has everything it needs.

TomerFi · 2026-07-13T15:42:13Z

[Moderate] Asymmetric dashboard coverage across CVE flows

The load_cve_dashboard call is added only to flow 01-account-cves.md (account-level). Flows 02-system-all-cves.md and 03-system-remediatable-cves.md collect the same kind of CVE data but receive no dashboard rendering. The user experience varies depending on which flow the agent enters, with no clear justification for the asymmetry.

@dmartinol Good catch — flows 02 and 03 use a different tool (get_system_cves) with pagination and client-side filtering, so adding the dashboard call there isn't as straightforward as flow 01. The data shape and accumulation pattern need to be verified against what the dashboard expects. I'll look into it.

TomerFi · 2026-07-13T15:45:17Z

[Architectural] Consider a dedicated presentation skill

Both skills now call dashboard MCP tools directly. This goes against the repo's core principle: "Skills encapsulate tools — never call MCP tools directly; always invoke skills."

A dedicated render-cve-dashboard / render-inventory-dashboard skill (or a single render-dashboard skill parameterized by type) would:

Restore the encapsulation invariant

Resolve the asymmetric flow coverage naturally — any flow that collects CVE or host data invokes the skill, coverage is automatic

Keep the fallback text logic (parse, sort, summary table) in one place rather than per-flow

Allow the remediation skill and any future skill collecting CVE/host data to reuse the same presentation layer

The trade-off is one additional skill file per dashboard type.

@dmartinol The cve-impact skill already calls MCP tools directly — get_cves, get_cve_details, etc. That's what skills do: they orchestrate tool calls. The "never call MCP tools directly" rule means the agent shouldn't bypass skills to call tools on its own, not that skills can't call tools. load_cve_dashboard is just another tool call within the skill's workflow.

…ashboard support to flows 02 and 03 Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

TomerFi · 2026-07-17T21:50:01Z

Pushed a new commit: feat: update skills for client-side dashboard data fetching and add dashboard support to flows 02 and 03

All three CVE flows and fleet inventory now use the client-side dashboard approach
Dashboard is called before the data fetch so the model receives the dashboard's instructions before having data in context
Flows 02 (system-level) and 03 (remediatable) now have dashboard support
For context on why we moved to client-side fetching, see the insights-mcp team discussion

Recordings with skills on Cursor:

Flow 01 — account-level CVEs: Show me all CVEs affecting my account
Flow 02 — system-level CVEs: What CVEs affect system XXX?
Flow 03 — remediatable CVEs: Show me remediatable CVEs on system XXX
Fleet inventory: Show me my fleet

dmartinol · 2026-07-20T10:05:43Z

 **Scope**: devices=all (account-wide) | severities=all, most critical, or selected | remediation=available or does not matter

-**Tool**: `vulnerability__get_cves` (single request; no offset pagination in API)
+**Tools**: `vulnerability__get_cves` fetcing the data for analysis and `vulnerability__load_cve_dashboard` displaying an interactive dashboard


typo in "fetcing"

dmartinol · 2026-07-20T10:06:25Z

 - `limit`: From HITL (default 20)

+
+**Before** continuing further, invoke the dashbaord tool to initaite the dashbaord, and examin the instruction returning from the tool.


typo in "initaite"

dmartinol · 2026-07-20T10:25:49Z

Code review

Found 3 issues:

Misspelled tool name in code example — the second example block uses vulnerability__load_cve_dashbaord (letters transposed) instead of vulnerability__load_cve_dashboard. An AI agent following this example verbatim will invoke a non-existent MCP tool and fail at runtime.

agentic-plugins/rh-sre/skills/cve-impact/references/flows/01-account-cves.md

Lines 69 to 78 in 8c3519a

    
           **Example** (all severities, remediation doesn't matter): 
        
           ``` 
        
           vulnerability__load_cve_dashbaord( 
        
             impact="7,6,5,4", 
        
             sort="-cvss_score", 
        
             limit=20, 
        
             advisory_available="true,false" 
        
           ) 
        
           ```

Same typo propagated into the CRITICAL constraint note in flow 03 — the CRITICAL block was extended with the sentence load_cve_dashbaord takes a string advisory_available, misspelling the tool name in the highest-attention guidance section of the file.

agentic-plugins/rh-sre/skills/cve-impact/references/flows/03-system-remediatable-cves.md

Lines 14 to 16 in 8c3519a

    
           **CRITICAL**: `get_system_cves` does NOT support `advisory_available` as a request parameter. We must paginate through ALL CVEs and filter client-side for `attributes.advisory_available === true`. `load_cve_dashbaord` takes a string `advisory_available`. Do NOT use `get_cves` + `get_cve_systems` per CVE—does not scale.

advisory_available placed in the shared parameters table, contradicting the CRITICAL note in the same file — the PR adds advisory_available: "true" - Only exist in load_cve_dashboard to the Step 4 parameters block shared by both vulnerability__load_cve_dashboard and vulnerability__get_system_cves. The existing CRITICAL note explicitly states "get_system_cves does NOT support advisory_available as a request parameter." Listing it in the shared block — even with an inline qualifier — creates the exact ambiguity the CRITICAL note was written to prevent; a model scanning the parameters list may still pass it to get_system_cves, silently returning unfiltered CVEs.

agentic-plugins/rh-sre/skills/cve-impact/references/flows/03-system-remediatable-cves.md

Lines 57 to 70 in 8c3519a

    
           **Tools**: `vulnerability__load_cve_dashboard` and `vulnerability__get_system_cves` (from lightspeed-mcp) 
        
           **Parameters**: 
        
           - `system_uuid`: Required (from Step 2) 
        
           - `limit`: `100` 
        
           - `offset`: `0`, `100`, `200`, ... per pagination strategy 
        
           - `advisory_available`: `"true"` - Only exist in `load_cve_dashboard` 
        
           **Before** continuing further, invoke the dashbaord tool to initaite the dashbaord, and examin the instruction returning from the tool: 
        
           ``` 
        
           vulnerability__load_cve_dashboard(system_uuid="<resolved-uuid>", limit=100, advisory_available="true") 
        
           ``` 
        
           **After** the dashbaord is loaded, get the raw data for further analysis.

Please run make validate (or at minimum make validate-mcp-tools) before merging — the validate_mcp_tools.py script checks that every tool name referenced in allowed-tools and flow files resolves against the live MCP server, which would catch the typos above automatically.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

Fix spelling errors in dashboard-related flow files and fleet-inventory SKILL.md: dashbaord→dashboard, fetcing→fetching, initaite→initiate, examin→examine, succesful→successful, and load_cve_dashbaord tool name. Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

TomerFi · 2026-07-20T19:46:14Z

Hey @dmartinol (or dmartinol's claude ;-) ).

Regarding issues 1 and 2 (typos):

An AI agent following this example verbatim will invoke a non-existent MCP tool and fail at runtime.

First, let me ease your mind — there's not a single LLM that won't be able to interpret "dashboard" from "dashbaord", especially when the tool itself is correctly named in allowed-tools and in the tool spec as exposed by the MCP server. That said, thank you — that was a great catch! Maybe it's worth adding a spelling check as a tool configured in this project and/or as a CI step. This project is mostly markdown — typos are bound to happen.

All typos were fixed in 04bbce5.

Regarding issue 3 (advisory_available in the parameters table):

a model scanning the parameters list may still pass it to get_system_cves, silently returning unfiltered CVEs.

I don't necessarily agree. As you noted yourself, I've added:

advisory_available: "true" - Only exist in load_cve_dashboard

Which clearly states advisory_available is only applicable to load_cve_dashboard. The CRITICAL note you're referring to explains to the agent that it needs to filter advisory_available CVEs client-side after fetching data from get_system_cves. On top of that, the tool spec as exposed by the MCP server doesn't include advisory_available as an argument for get_system_cves — so I sincerely doubt that adding advisory_available: "true" - Only exist in load_cve_dashboard to a section that exemplifies both get_system_cves and load_cve_dashboard will cause the agent to push advisory_available to get_system_cves.

That said, if it would make you feel comfortable, we can beef the instruction up, something like:

advisory_available: "true" - ONLY exist in load_cve_dashboard, DO NOT use with get_system_cves

What do you think?

Regarding "Please run make validate":

Please run make validate (or at minimum make validate-mcp-tools) before merging — the validate_mcp_tools.py script checks that every tool name referenced in allowed-tools and flow files resolves against the live MCP server, which would catch the typos above automatically.

A few things here:

The implication that I pushed a PR without running make validate doesn't make a lot of sense, since I stated the validation results in the PR description. I'd appreciate a bit of credit, thanks.
You are wrong — make validate does not catch these typos. The validate_mcp_tools.py script only checks tool names declared in the allowed-tools frontmatter of SKILL.md files. It does not scan flow files, markdown body content, or code examples. The allowed-tools entries are all correctly spelled, so the validator passes with flying colors. You can verify this yourself by reading the script — specifically parse_frontmatter() which only extracts the allowed-tools: line from YAML frontmatter.
Furthermore, validate_mcp_tools.py quietly reports most of its issues as warnings and still exits 0. Similarly, make validate exits 0 for many errors, reporting them as warnings in a very long output that requires analysis. This cannot be considered a gate for PRs in its current form.

Generated by @TomerFi with the help of AI tools to make sure this message is grammatically correct and delivers the idea.

dmartinol

:Lgtm BUT before merging it we should approve changes to insights-mcp, rebuild the image and update mcps.json to use it. ATM we have:
insights-mcp/red-hat-lightspeed-mcp@sha256:98ee33fa7cb690d02d33f57900ab95ad831b8e7985710e0ccb87f53ce1b09865

r2dedios

I only saw a few typos.

r2dedios · 2026-07-21T08:10:31Z

+)
+```
+
+**After** the dashboard is loaded, get the raw data for further analysis.


Aren't these examples repeated? Is it useful at this point to include the example twice?

Check lines 58-75

Hey @r2dedios , the repeated examples are intentional. The flow has two distinct steps: first call load_cve_dashboard (the "Before" step), then call get_cves (the "After" step). Even though the parameters are identical, I want the agent to see explicit examples for each tool at each step. We could trim it down to one example per tool instead of two, but I'm not sure it's worth the risk — I'd rather be verbose and clear than concise and ambiguous for the agent. What do you think?

r2dedios · 2026-07-21T08:12:10Z


 **Parameters**: `per_page=10` on first call, then `page` for pagination. Optional filters: `display_name`, `tags`, `staleness`, `hostname_or_id`. See [references/01-parameter-reference.md](references/01-parameter-reference.md).

+**Before** continuing further, call `inventory__load_inventory_dashboard(...)` with the same query parameters used in the `list_hosts` call, and examin the instruction returning from the tool.


typo examin -> examine

r2dedios · 2026-07-21T08:13:01Z

 - `system_uuid`: Required (from Step 2)
 - `limit`: `100`
 - `offset`: `0`, `100`, `200`, ... per pagination strategy
+- `advisory_available`: `"true"` - Only exist in `load_cve_dashboard`


typo Only exist -> Only exists

r2dedios · 2026-07-21T08:14:16Z


-**Tool**: `vulnerability__get_system_cves` (paginated; filter client-side for `advisory_available === true`)
+**Tool**: `vulnerability__get_system_cves` fetching the data for analysis and `vulnerability__load_cve_dashboard` displaying an interactive dashboard


Should it be Tools?

Fix examin→examine in fleet-inventory SKILL.md, Only exist→Only exists and Tool→Tools in flow 03. Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

TomerFi · 2026-07-21T14:15:38Z

Issue for spell checking tool: #145.

r2dedios

LGTM!! 🚀

feat: add dashboard-aware presentation tools to cve-impact and fleet-…

932ef7b

…inventory skills Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

TomerFi requested a review from dmartinol as a code owner July 2, 2026 19:57

TomerFi mentioned this pull request Jul 2, 2026

feat: add dashboard-aware presentation tools to cve-impact and fleet-inventory skills RHEcosystemAppEng/agentic-collections-skills#3

Closed

TomerFi marked this pull request as draft July 2, 2026 20:00

feat: update skills for client-side dashboard data fetching and add d…

8c3519a

…ashboard support to flows 02 and 03 Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

TomerFi marked this pull request as ready for review July 17, 2026 21:50

TomerFi requested a review from r2dedios as a code owner July 17, 2026 21:50

dmartinol reviewed Jul 20, 2026

View reviewed changes

dmartinol reviewed Jul 21, 2026

View reviewed changes

r2dedios requested changes Jul 21, 2026

View reviewed changes

fix(rh-sre): fix additional typos found in PR review

7991dcb

Fix examin→examine in fleet-inventory SKILL.md, Only exist→Only exists and Tool→Tools in flow 03. Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

r2dedios approved these changes Jul 21, 2026

View reviewed changes

		- `limit`: From HITL (default 20)


		Before continuing further, invoke the dashbaord tool to initaite the dashbaord, and examin the instruction returning from the tool.


		Parameters: `per_page=10` on first call, then `page` for pagination. Optional filters: `display_name`, `tags`, `staleness`, `hostname_or_id`. See [references/01-parameter-reference.md](references/01-parameter-reference.md).

		Before continuing further, call `inventory__load_inventory_dashboard(...)` with the same query parameters used in the `list_hosts` call, and examin the instruction returning from the tool.


		Tool: `vulnerability__get_system_cves` (paginated; filter client-side for `advisory_available === true`)
		Tool: `vulnerability__get_system_cves` fetching the data for analysis and `vulnerability__load_cve_dashboard` displaying an interactive dashboard

Uh oh!

Conversation

TomerFi commented Jul 2, 2026

Summary

Validation

Recordings

Related tickets

Uh oh!

dmartinol commented Jul 8, 2026

Uh oh!

dmartinol commented Jul 8, 2026

Uh oh!

dmartinol commented Jul 8, 2026

Uh oh!

TomerFi commented Jul 13, 2026

Uh oh!

TomerFi commented Jul 13, 2026

Uh oh!

TomerFi commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TomerFi commented Jul 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmartinol commented Jul 20, 2026

Code review

Uh oh!

TomerFi commented Jul 20, 2026

Uh oh!

dmartinol left a comment

Choose a reason for hiding this comment

Uh oh!

r2dedios left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TomerFi commented Jul 21, 2026

Uh oh!

r2dedios left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

TomerFi commented Jul 13, 2026 •

edited

Loading