From c741a559cbce980c763f78a23b568d2c56d61b73 Mon Sep 17 00:00:00 2001 From: Aaron Stannard Date: Sat, 18 Jul 2026 22:05:12 +0000 Subject: [PATCH] docs(skills): add Native Manifest and Bundled Resources concept pages (#96, #97) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Group 1 of the skill-server 0.4.0 epic (#95) — the two foundational concept pages the rest depend on. Written from the skill-server 0.4.0 source (Endpoints.cs, docs/specs/native-manifest.md, docs/specs/skills.md) and the netclaw 0.25.0 consumption side. - skills/native-manifest.md: the versioned HATEOAS /manifest.json sidecar vs the Cloudflare RFC feed, the skills/subagents collection trees, API version negotiation, and what netclaw pulls from it (subagents + archives), fail-soft / prune-after-confirm. Documents the real /api/v1/ content routes (the spec shorthand drops the prefix). - skills/bundled-resources.md: skill-md vs archive artifact shapes, deterministic archives with preserved exec bits, the automatic backfill, and how netclaw downloads/verifies/extracts (zip-slip hardening) into ~/.netclaw/skills/.server-feeds/, plus skill_read_resource and shell access. - Sidebar: both under Skills (the section rename to "Skills & Agents" lands with the custom-subagents move, #103). Forward-links to not-yet-written pages (skillserver-cli, subagent-distribution) point at existing pages for now; the later epic PRs repoint them. Closes #96, #97. --- astro.config.mjs | 2 + social-images-manifest.json | 8 ++ src/content/docs/skills/bundled-resources.md | 70 +++++++++++++++ src/content/docs/skills/native-manifest.md | 91 ++++++++++++++++++++ 4 files changed, 171 insertions(+) create mode 100644 src/content/docs/skills/bundled-resources.md create mode 100644 src/content/docs/skills/native-manifest.md diff --git a/astro.config.mjs b/astro.config.mjs index fa6e63c..b887ac8 100644 --- a/astro.config.mjs +++ b/astro.config.mjs @@ -68,6 +68,8 @@ export default defineConfig({ label: 'Skills', items: [ { label: 'Overview', slug: 'skills/overview' }, + { label: 'Native Manifest & Sync', slug: 'skills/native-manifest' }, + { label: 'Bundling Resources with Skills', slug: 'skills/bundled-resources' }, { label: 'Skill Server', slug: 'skills/skill-server' }, { label: 'External Skills', slug: 'skills/external-skills' }, { label: 'Skill Feeds', slug: 'skills/skill-feeds' }, diff --git a/social-images-manifest.json b/social-images-manifest.json index 001d31b..9dab948 100644 --- a/social-images-manifest.json +++ b/social-images-manifest.json @@ -247,6 +247,14 @@ "architecture/context-structure": { "hash": "f28ea5f10f62df6db39a6a2ad5159083bc75eae24fa8bfe56f9a0a803414370c", "imageUrl": "https://hcti.io/v1/image/8e1125b30cb2852b87279f8adfd0e4e820f434222b55ae6596dc5571483f2e8f" + }, + "skills/bundled-resources": { + "hash": "f457960f1866b08f19e213d2d78c47c50ea1a228142a198a8739a318389ab31b", + "imageUrl": "https://hcti.io/v1/image/3fffc186335b9cd7165d55c0626e17d01d233a59cefcc1eda61f3729dc665a82" + }, + "skills/native-manifest": { + "hash": "d420501a6c5bbcc2593deb9fc57fba6fd3df695281aeb585bd7e6e29e83ed8f0", + "imageUrl": "https://hcti.io/v1/image/eb22b7288dcc2e2e997fcd389ba65f0032ee0830fa553481f3648eebdc3bd0bb" } } } diff --git a/src/content/docs/skills/bundled-resources.md b/src/content/docs/skills/bundled-resources.md new file mode 100644 index 0000000..8d171e8 --- /dev/null +++ b/src/content/docs/skills/bundled-resources.md @@ -0,0 +1,70 @@ +--- +title: "Bundling Resources with Skills" +description: "How skills ship supporting files — references, scripts, assets — as deterministic archives, and how netclaw downloads and extracts them." +--- + +A skill is more than its `SKILL.md`. Many carry supporting files: reference docs the agent can read, scripts it can run, assets it needs. A skill server bundles those into a single deterministic archive so netclaw can pull the whole skill in one verified download instead of fetching files one at a time. + +## Two artifact shapes + +A skill server projects every published skill into one of two shapes, based on whether it has supporting files: + +| Shape | When | What's served | +|-------|------|---------------| +| `skill-md` | `SKILL.md` only | the single markdown file | +| `archive` | `SKILL.md` + supporting files | a `.zip` at `/api/v1/skills/{name}/{version}/archive.zip` | + +An archive holds `SKILL.md` at its root and the supporting files under their relative paths — the same layout the skill author used: + +``` +my-skill/ +├── SKILL.md +├── references/ +│ └── api-notes.md +├── scripts/ +│ └── check.sh # executable bit preserved +└── assets/ + └── template.json +``` + +Archives are **deterministic**: entries are ordered, timestamps are fixed, and Unix permission bits are preserved (masked to standard bits), so an executable script stays executable after extraction and the same skill version always produces a byte-identical archive with the same digest. + +## What the author does + +Nothing special. Lay the skill out as a directory — `SKILL.md` plus optional `references/`, `scripts/`, and `assets/` — and publish it. The server decides the shape: a skill with supporting files becomes an archive automatically. See [Skill Server](/skills/skill-server/) for publishing. + +Skills published before archives existed aren't left behind. The server **backfills** them on startup — it scans versioned skills that have resources but no archive, builds the deterministic archive from the stored files, and records it. The backfill is additive and idempotent: the original `SKILL.md` bytes, digest, and per-file routes stay exactly as they were, so nothing you already depend on changes, and a version that already has an archive is skipped. + +## How netclaw consumes an archive + +When a synced feed offers a skill as an archive, netclaw takes the bundle instead of individual files: + +1. **Download** the `.zip` from the archive URL. +2. **Verify** its SHA-256 digest against the manifest before touching the contents. A mismatch is rejected and logged — nothing gets extracted. +3. **Extract** with hardening: zip-slip path traversal, symlink entries, and duplicate names are all refused, and each file is content-scanned for prompt-injection the same way a single `SKILL.md` is. +4. **Write** the tree under `~/.netclaw/skills/.server-feeds///`, preserving the `references/` and `scripts/` subpaths and the Unix modes. + +The agent reads bundled files through the `skill_read_resource` tool, by logical name — it doesn't need to know the on-disk path. And because executable bits survive the round trip, a synced `scripts/check.sh` can be run through the shell tool. + +:::caution +Bundled resources are code and content from wherever the feed points. They ride the same trust boundary as any synced skill: digest-verified on the way in, content-scanned, and extracted under hardening — but a script you sync from a feed is a script you're trusting. Only subscribe to [skill feeds](/skills/skill-feeds/) you control or trust, and remember that a `Personal`-audience session is the only one that can run a synced script through the shell. +::: + +## Limitations + +- Archives are `.zip` only. +- Bundling is the server's call, driven by whether a skill has supporting files — there's no separate "make this an archive" switch. +- Per-file resource routes remain for compatibility, but archives are how netclaw pulls a resourceful skill; a client that only understands `skill-md` still gets the markdown and misses the extras. +- Digest verification covers the archive as a whole. Netclaw extracts only after the whole bundle verifies, so a partial or corrupted download never lands on disk. + +## Related pages + +- [Native Manifest & Sync](/skills/native-manifest/) — how the archive URL and digest reach netclaw +- [Skill Feeds](/skills/skill-feeds/) — subscribing to the feeds that carry these skills +- [Skill Server](/skills/skill-server/) — publishing skills with resources +- [Images, Audio, and Other File Types](/guides/multimodal-files/) — how the agent reads the files it pulls + +## Resources + +- [AgentSkills.io](https://agentskills.io) — the open skill format these bundles follow +- [Zip Slip](https://security.snyk.io/research/zip-slip-vulnerability) — the path-traversal class netclaw hardens extraction against diff --git a/src/content/docs/skills/native-manifest.md b/src/content/docs/skills/native-manifest.md new file mode 100644 index 0000000..6794979 --- /dev/null +++ b/src/content/docs/skills/native-manifest.md @@ -0,0 +1,91 @@ +--- +title: "Native Manifest & Sync" +description: "How a skill server exposes skills and subagents to netclaw through the versioned native manifest, alongside the Cloudflare RFC feed." +--- + +A skill server publishes two discovery feeds. The [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc) feed at `/.well-known/agent-skills/index.json` is the vendor-neutral one — any tool that speaks the RFC can read it. The **native manifest** at `/manifest.json` is a richer, versioned feed for skill-server-aware clients like netclaw: it adds version negotiation, version history, subagents, and archive metadata the RFC feed doesn't carry. + +The native manifest is a **sidecar, not a replacement**. It never supersedes the RFC feed — a client that only understands the RFC keeps working, and netclaw reads both. What the native manifest buys netclaw is the extra material: subagent definitions (which never appear in the RFC feed) and bundled-resource archives. + +## The two feeds + +| | RFC feed | Native manifest | +|---|----------|-----------------| +| Entry point | `/.well-known/agent-skills/index.json` | `/manifest.json` | +| Audience | any RFC-compatible tool | skill-server-aware clients | +| Carries | skills | skills **+ subagents + archive metadata** | +| Versioned | no | yes (API version negotiation) | +| netclaw uses it for | skill discovery, digests | subagents, archives, version history | + +## How the manifest is shaped + +The manifest is [HATEOAS](https://en.wikipedia.org/wiki/HATEOAS) — clients follow links rather than building URLs by hand. The root at `/manifest.json` lists the available API versions, each with a set of links: + +```json +{ + "apiVersion": "v1", + "versions": { + "v1": { + "self": { "href": "/manifest.json" }, + "skills": { "href": "/skills/v1/index.json" }, + "subagents": { "href": "/subagents/v1/index.json" }, + "skillSearch": { "href": "/api/v1/skills" }, + "subagentSearch": { "href": "/api/v1/subagents" } + } + } +} +``` + +From there, a client follows `href` values down two collection trees — one for skills, one for subagents — each with its own index, pages, and per-item version history: + +``` +/skills/v1/index.json # skill collection +/skills/v1/{skill}/index.json # one skill +/skills/v1/{skill}/versions/{version}.json # a specific version +/subagents/v1/index.json # subagent collection +/subagents/v1/{agent}/versions/{version}.json +``` + +The manifest links point at the REST API under `/api/v1/` for the actual bytes — `/api/v1/skills/{name}/{version}/archive.zip` for a bundled skill, `/api/v1/subagents/{name}/{version}/agent.md` for a subagent. Follow the links; the paths are readable but the server owns pagination boundaries, so don't hard-code them. + +:::note +The manifest declares the routes, but the wire paths that matter are the ones the running server serves. If you're scripting against a server directly, read them off `/manifest.json` rather than a spec — the REST content lives under `/api/v1/`, which is easy to miss from the shorthand in the collection trees. +::: + +## Version negotiation + +`apiVersion` names the version the server recommends; `versions` lists every version it offers. A client picks the newest version it *also* supports: + +1. The client knows its own supported versions (netclaw today: `v1`). +2. It reads the server's `versions`. +3. It uses the most recent version in the intersection. +4. If there's no overlap, it falls back to the oldest the server offers. + +The payoff is forward compatibility in both directions: when a server ships `v2`, an older netclaw keeps using `v1`, and a newer netclaw automatically prefers `v2` — no coordinated upgrade. + +## What netclaw does with it + +On each sync, the daemon fetches the RFC index first, then — for feeds served by a skill server — follows the native manifest to pull what the RFC feed can't describe: + +- **Subagents.** It walks the subagent collection, downloads each `agent.md`, verifies its digest, checks the frontmatter `name` matches, and writes it under `~/.netclaw/agents/.server-feeds//`. See [Custom Subagents](/guides/custom-subagents/). +- **Archives.** When a skill is published as an archive, netclaw pulls the bundled `.zip` instead of individual files. See [Bundling Resources with Skills](/skills/bundled-resources/). + +Sync is **fail-soft and prune-after-confirm**: a feed that's unreachable or serves a bad digest doesn't take down the others or wipe your last-good copy, and stale entries are pruned only after a successful sync replaces them. A server that speaks only the RFC feed still works — netclaw just doesn't get subagents or archives from it. + +## Limitations + +- The native manifest is skill-server-specific. A generic RFC feed won't expose subagents or archive metadata, so those features need a [skill server](/skills/skill-server/). +- Version negotiation covers the manifest API, not skill or subagent content versions — those are tracked per item in the collection trees. +- There's no transitive dependency resolution. A skill's `metadata.subagent` route is a runtime pointer, not a package dependency the manifest resolves for you. + +## Related pages + +- [Skill Feeds](/skills/skill-feeds/) — configuring a feed and the sync lifecycle +- [Bundling Resources with Skills](/skills/bundled-resources/) — the archive artifacts the manifest points to +- [Custom Subagents](/guides/custom-subagents/) — defining the subagents the manifest distributes +- [Skill Server](/skills/skill-server/) — running the server that publishes it + +## Resources + +- [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc) — the vendor-neutral feed the manifest sits beside +- [HATEOAS](https://en.wikipedia.org/wiki/HATEOAS) — the follow-the-links hypermedia style the manifest uses