docs: clarify skill injection behavior and add migration guide#483
Conversation
Address documentation gaps identified in OpenHands/software-agent-sdk#2981: 1. Add 'Skill Injection Behavior' section with clear table showing where content appears based on format and trigger configuration 2. Add warning about legacy trigger=None skills consuming tokens on every turn 3. Add 'Prompt Structure' section showing exactly where skills appear in the system prompt 4. Add warning to inline skill code example about legacy behavior 5. Improve load_skills_from_dir() documentation with table explaining injection behavior for each return value 6. Add 'Migrating from Legacy to AgentSkills Format' section with before/after examples and benefits comparison Co-authored-by: openhands <openhands@all-hands.dev>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
all-hands-bot
left a comment
There was a problem hiding this comment.
🟢 Good taste - Clear documentation improvement that addresses a real gap.
[RISK ASSESSMENT]
- [Overall PR]
⚠️ Risk Assessment: 🟢 LOW
Documentation-only change with no code modifications. The added content clarifies skill injection behavior with clear tables, appropriate warnings, and a helpful migration guide. Well-structured and addresses the gap identified in OpenHands/software-agent-sdk#2981.
VERDICT:
✅ Worth merging: Solid documentation improvement with clear, actionable guidance.
KEY INSIGHT:
The skill injection behavior table and prompt structure examples make previously implicit system behavior explicit and understandable.
all-hands-bot
left a comment
There was a problem hiding this comment.
🟢 Good taste - Excellent documentation improvement that addresses a real gap.
[RISK ASSESSMENT]
- [Overall PR]
⚠️ Risk Assessment: 🟢 LOW
Documentation-only change with no code modifications. The added content clarifies skill injection behavior with clear tables, appropriate warnings, helpful XML examples showing prompt structure, and a practical migration guide. Well-structured and directly addresses the documentation gap identified in OpenHands/software-agent-sdk#2981.
Key improvements:
- Clear table at the top showing where content appears (progressive disclosure vs. full injection)
- Token usage warnings for legacy
trigger=Noneskills - Visual prompt structure showing
<REPO_CONTEXT>vs<available_skills>vs<EXTRA_INFO> - Practical before/after migration examples with benefits comparison
- Enhanced
load_skills_from_dir()docs with injection behavior table
VERDICT:
✅ Worth merging: Solid documentation improvement with clear, actionable guidance.
KEY INSIGHT:
Documents the critical but previously undocumented fact that legacy skills with trigger=None inject full content into every turn, potentially consuming significant tokens.
|
Please check if this review has considered going to the original codebase to find corresponding coding snippets. If not, please modify the custom code review guideline in this repo (https://docs.openhands.dev/openhands/usage/use-cases/code-review#customization) so that we tell the code review agent to look through the original codebase and provide evidence as links to the original code for each change in the documentation. When you are done modifying the custom code review guideline and have pushed it to this PR, re-request the review from all-hands-bot. |
|
I'm on it! xingyaoww can track my progress at all-hands.dev |
Update the code review skill to explicitly require that every review comment and approval includes GitHub permalink evidence from the original source codebase. Adds: - Mandatory evidence requirement in the source verification section - Format examples for inline review comments with source links - Source Verification table template for the overall review body - Instructions to flag unverifiable claims explicitly - GitHub base URLs in the repository mapping table Co-authored-by: openhands <openhands@all-hands.dev>
|
The all-hands-bot review did not go to the original codebase to find corresponding coding snippets. Both reviews (IDs 4196374014 and 4196396963) approved with general assessments like "looks good" and "solid documentation improvement" without citing any specific source files or lines from I've updated the code review guideline at
I've also re-requested review from This comment was generated by an AI agent (OpenHands). |
Summary of WorkWas the request completely addressed?The reviewer (@xingyaoww) had three specific instructions:
Are the changes concise?Yes. Only one file was modified ( |
all-hands-bot
left a comment
There was a problem hiding this comment.
🟢 Good taste - Clear documentation improvement that accurately describes skill injection behavior.
[RISK ASSESSMENT]
- [Overall PR]
⚠️ Risk Assessment: 🟢 LOW
Documentation-only change with no code modifications. All technical claims verified against SDK source code.
Source Verification
Every documentation claim has been verified against the SDK source code (commit afae3e354821022fcd17065e3b559d624c545125):
| Documentation Claim | Verified? | Source Evidence |
|---|---|---|
is_agentskills_format field exists on Skill class |
✅ | skill.py#L155-L164 |
Legacy skills with trigger=None inject full content into <REPO_CONTEXT> every turn |
✅ | agent_context.py#L188-L189, template |
| AgentSkills format uses progressive disclosure (description only, agent reads on demand) | ✅ | skill.py#L108-L114, agent_context.py#L186-L187 |
Skills with triggers appear in <available_skills> and auto-inject on match |
✅ | agent_context.py#L197-L198, user message suffix |
Triggered content appears in <EXTRA_INFO> in user messages |
✅ | skill_knowledge_info.j2#L2 |
load_skills_from_dir() returns tuple of (repo_skills, knowledge_skills, agent_skills) |
✅ | skill.py#L657, docstring#L670-L673 |
<REPO_CONTEXT> structure for repo skills |
✅ | system_message_suffix.j2#L6-L22 |
<SKILLS> / <available_skills> structure |
✅ | system_message_suffix.j2#L24-L31, to_prompt function |
VERDICT:
✅ Worth merging: Accurate documentation that addresses a real gap identified in OpenHands/software-agent-sdk#2981. All technical claims verified against source code.
KEY INSIGHT:
This PR finally documents the critical distinction between legacy trigger=None behavior (full content, every turn) and AgentSkills progressive disclosure (description only, invoke on demand) — a gap that has caused user confusion about token usage.
Clarify that legacy trigger=None skills are added to the initial system prompt and then sent with each LLM request while retained in conversation history, rather than being re-rendered as a new injection on every turn. Co-authored-by: openhands <openhands@all-hands.dev>
Replace transport-level wording around legacy skill context with unambiguous prompt/context terminology. Record the preference in repository guidance and code review guidelines. Co-authored-by: openhands <openhands@all-hands.dev>
|
Updated in The docs now say legacy
I also added the wording preference to This comment was generated by an AI agent (OpenHands) on behalf of the user. |
Summary
Address documentation gaps identified in OpenHands/software-agent-sdk#2981 regarding skill injection semantics.
Problem
The current documentation shows 3 "loading methods" but the SDK actually implements 4 distinct injection behaviors based on skill format (
is_agentskills_format) and trigger configuration. Users have no way to know that:trigger=Noneinject full content into<REPO_CONTEXT>on every turnSkill()constructor produces fundamentally different runtime behavior depending on formatChanges
New "Skill Injection Behavior" section - Clear table at the top showing where content appears:
SKILL.md)<available_skills>(description only)SKILL.md)<available_skills>+ auto-inject*.md)None<REPO_CONTEXT>(full content, every turn)*.md)<available_skills>+ auto-injectWarning callout - Token usage warning for legacy
trigger=NoneskillsPrompt Structure section - Shows XML structure of where skills appear in prompts
Warning on inline skills - Added warning to the "Inline Skill (Code-defined)" example
Improved
load_skills_from_dir()docs - Table explaining injection behavior for each return valueNew "Migrating from Legacy to AgentSkills Format" section - Before/after examples with benefits comparison
Related Issues
This PR was created by an AI agent (OpenHands) on behalf of the user.
@VascoSch92 can click here to continue refining the PR