Skip to content

feat: add Claude Code Plugin with skill-bench testing framework#55

Merged
sonesuke merged 1 commit intomainfrom
feat/claude-code-plugin
Mar 3, 2026
Merged

feat: add Claude Code Plugin with skill-bench testing framework#55
sonesuke merged 1 commit intomainfrom
feat/claude-code-plugin

Conversation

@sonesuke
Copy link
Copy Markdown
Owner

@sonesuke sonesuke commented Mar 2, 2026

Summary

  • Add Claude Code Plugin structure (.claude-plugin/, claude-plugin/)
  • Implement two skills: arxiv-search, arxiv-fetch
  • Add skill-bench testing framework with test cases for each skill
  • Add check scripts for MCP server, skill invocation, and parameter validation

Plugin Structure

.claude-plugin/
  marketplace.json  # Marketplace configuration
claude-plugin/
  .claude-plugin/
    plugin.json     # Plugin definition with MCP servers
  skills/
    arxiv-search/SKILL.md
    arxiv-fetch/SKILL.md

Skills

  1. arxiv-search: Search arXiv for papers with query and limit parameters
  2. arxiv-fetch: Fetch paper details by arXiv ID

Skill-Bench Testing Framework

Located in agents/skill-bench/:

  • runner.sh: Test runner that executes trials and evaluates results
  • cases/: Test case definitions (TOML format)
  • tools/: Check scripts for validating test results

Test Cases

Each skill has multiple test variations:

  • triggering.toml: Verify skill is triggered appropriately
  • functional.toml: Basic functional test
  • functional-with-limit.toml: Functional tests with specific parameters

Notes

  • MCP server currently fails in tests because arxiv-cli binary is not installed in test environment
  • This will be addressed in a follow-up by ensuring the binary is available during tests

🤖 Generated with Claude Code

- Add Claude Code Plugin structure (.claude-plugin/, claude-plugin/)
- Implement two skills: arxiv-search, arxiv-fetch
- Add skill-bench testing framework with test cases for each skill
- Add check scripts for MCP server, skill invocation, and parameter validation

## Plugin Structure

.claude-plugin/
  marketplace.json  # Marketplace configuration
claude-plugin/
  .claude-plugin/
    plugin.json     # Plugin definition with MCP servers
  skills/
    arxiv-search/SKILL.md
    arxiv-fetch/SKILL.md

## Skills

1. arxiv-search: Search arXiv for papers with query and limit
2. arxiv-fetch: Fetch paper details by arXiv ID

## Skill-Bench Testing Framework

Located in agents/skill-bench/:
- runner.sh: Test runner that executes trials and evaluates results
- cases/: Test case definitions (TOML format)
- tools/: Check scripts for validating test results

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@sonesuke sonesuke merged commit 2997d52 into main Mar 3, 2026
4 checks passed
@sonesuke sonesuke deleted the feat/claude-code-plugin branch March 3, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants