Fixes issue #96

[iswaryaalex]

Root cause

The flakiness is not due to the skill itself but due to the skill not actually getting invoked in the first place.
The agent was never invoking the skill - claude just answered narrow prompts ("write a lemond launcher", "re-point the OpenAI client") entirely from training knowledge.

Changes Summary

skills/local-ai-app-integration/evals/evals.py

• test_launcher_module_written — changed to a description-trigger test: natural prompt using "embeddable lemonade /
  local AI / local backend", single logs_contains("local-ai-app-integration") check only
• test_launcher_implementation — new test: explicit skill invocation, checks secrets, socket, subprocess
• All other tests — added "Use the local-ai-app-integration skill." to prompts and
  logs_contains("local-ai-app-integration") as first assertion

skills/local-ai-app-integration/SKILL.md

• Frontmatter description — restored to clean generic form, removed hacky per-prompt enumeration
• Step 4 launcher bullets — added exact stdlib tokens inline: secrets.token_urlsafe(32), socket,
  subprocess.Popen(...), GET /api/v1/health
• Step 5 — removed duplicated timeout emphasis; kept timeout=120 in code example and client table only
• Step 7 — fixed /v1/health typo → /api/v1/health
• Synopsis and checklist — simplified timeout references to plain "120 seconds"
• Verification checklist — restored concise timeout item

skills/local-ai-app-integration/reference.md

• Fixed four /v1/ shorthand instances → /api/v1/ (system-info, models, install ×2)

[danielholanda]

A key part of our behavioral tests is understanding whether or not the description provided by the skill actually makes the LLM trigger it. That said, I understand some of your tests are small "unit" tests of the actual skill instead of only its desctiption.

My recommendation here is the following: Instead of changing the harness:

• Have at least one test where we check whether the agent triggers your skill when it should. Example: You can add the following sentence to the beginning of your test_launcher_module_written test: "Help me turn this python app into an app that targets local llms".
• Add Make sure to use the "local-ai-app-integration" skillto your other tests directly.

[iswaryaalex]

• Have at least one test where we check whether the agent triggers your skill when it should. Example: You can add the following sentence to the beginning of your test_launcher_module_written test: "Help me turn this python app into an app that targets local llms".

https://github.com/amd/skills/pull/100/changes#diff-6e9abf55c55ec3e10d31c015524528d2eab52e678f9f8ad0653264c07ac6e075R33 We are doing a unint test but that alone may not capture. Looking at 2nd suggestion

[iswaryaalex]

• Add Make sure to use the "local-ai-app-integration" skillto your other tests directly.

@danielholanda This feels a bit like cheating but guarantees to work.
Added one test to explictly check for skill invocation https://github.com/amd/skills/pull/100/changes#diff-6e9abf55c55ec3e10d31c015524528d2eab52e678f9f8ad0653264c07ac6e075R28