diff --git a/CLAUDE.md b/CLAUDE.md index f47d588afd..d7a318de8a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,24 +4,52 @@ Open-source video rendering framework: write HTML, render video. ## Skills -This repo ships AI agent skills via [vercel-labs/skills](https://github.com/vercel-labs/skills). Install them before writing compositions — they encode framework-specific patterns that generic docs don't cover. +This repo ships 19 AI agent skills via [vercel-labs/skills](https://github.com/vercel-labs/skills). Install them before writing compositions — they encode framework-specific patterns that generic docs don't cover. ```bash -npx skills add heygen-com/hyperframes +npx skills add heygen-com/hyperframes # interactive picker +npx skills add heygen-com/hyperframes --all # install all 19 (skips picker) +npx skills add heygen-com/hyperframes --skill # just one (bare name, no leading slash) ``` -**Video-creation workflows** route through one entry skill — read `/hyperframes` first: it orients you to the whole surface and maps "make me a video" intent to a concrete workflow. Consult it before invoking a specific workflow: +**`/hyperframes` is the entry skill — read it first.** It's the capability map for the domain skills below AND the intent router for the creation workflows. The full README skills section mirrors this list; keep them in sync (see "Skill catalog maintenance" below). + +### Creation workflows - `/product-launch-video` — a **product** URL (or a pre-written script / text brief in no-capture mode) → product launch / promo video, up to ~3 min (sweet spot ~30-90s). - `/website-to-video` — a **general** website / URL → a video _of_ the site (tour / showcase / social clip from captured screenshots + assets); for a product **launch / promo**, use `/product-launch-video`. - `/faceless-explainer` — arbitrary text, **no URL and no website capture** → faceless explainer, up to ~3 min (sweet spot ~30-90s); every visual is LLM-invented (typography / abstract graphics / diagram / data-viz). -- `/embedded-captions` — an existing talking-head video (MP4) → the same footage with captions / subtitles added (verbatim rail + embedded climax, or pure-cinematic embed); the footage itself is untouched (no NLE-style editing). -- `/talking-head-recut` — an existing talking-head / interview / podcast video (MP4) → the same footage packaged with designed **graphic overlays** (kinetic titles, lower-thirds, data callouts, pull-quotes, side panels, pip) synced to the transcript; the clip plays unchanged underneath, footage untouched. Replaces the removed `/footage-recut`. For plain captions/subtitles → `/embedded-captions`. - `/pr-to-video` — a GitHub PR (URL / `owner/repo#N` / "this PR") → code-change explainer, up to ~3 min (changelog / feature reveal / fix / refactor). A PR link, not a product website. +- `/embedded-captions` — an existing talking-head video (MP4) → the same footage with captions / subtitles added (verbatim rail + embedded climax, or pure-cinematic embed); the footage itself is untouched (no NLE-style editing). +- `/talking-head-recut` — an existing talking-head / interview / podcast video (MP4) → the same footage packaged with designed **graphic overlays** (kinetic titles, lower-thirds, data callouts, pull-quotes, side panels, PiP) synced to the transcript; the clip plays unchanged underneath, footage untouched. For plain captions/subtitles → `/embedded-captions`. - `/motion-graphics` — a short (typically under 10s) design-led **motion graphic**, motion-is-the-message, no narration: kinetic type, a stat / number count-up, a chart, a logo sting, a lower-third / overlay, or an animated tweet / headline / captured-page highlight; rendered to MP4 or a transparent overlay. Longer / narrated / custom → `/general-video`. +- `/music-to-video` — a **music track** (audio file, or video to pull audio from) → beat-synced video (lyric / slideshow / kinetic promo). Music drives pacing; user-supplied images / videos are cut onto the same beat grid. +- `/slideshow` — a **presentation / pitch deck / interactive deck** — discrete slides, fragment reveals, branching, hotspot navigation, presenter mode. Output is a navigable deck, not a rendered video. - `/general-video` — fallback for any other video creation (title card, longer brand / sizzle reel, multi-scene montage, static loop, custom composition); the original hyperframes flow — design → plan → layout → build → validate, any length. +- `/remotion-to-hyperframes` — port an existing Remotion (React) composition to HyperFrames HTML. One-way migration, not creation. + +### Domain skills (loaded on demand) + +Atomic capabilities the creation workflows compose against — pull one when you need that specific layer: + +- `/hyperframes-core` — the composition contract: `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, determinism rules. Read before writing composition HTML. +- `/hyperframes-animation` — all animation knowledge: atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP default, plus Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). +- `/hyperframes-creative` — non-animation creative direction: `frame.md` / `design.md` handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. +- `/hyperframes-media` — audio + media: TTS voiceover, background music, sound effects, Whisper transcription, background removal, caption authoring (one shared `scripts/audio.mjs` engine, multi-provider). +- `/media-use` — resolve any media need (BGM, SFX, image, icon) into a frozen local file + ledger record. One verb (`resolve`) over the HeyGen catalog with manifest tracking; keeps search noise on disk. +- `/hyperframes-cli` — CLI dev loop: `init`, `add`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, `lambda` (AWS Lambda cloud rendering). +- `/hyperframes-registry` — install and wire registry blocks and components into compositions via `hyperframes add`. Covers authoring a new block or component to contribute upstream. + +## Skill catalog maintenance + +When adding a new skill, or substantially renaming / repurposing an existing one, update all three agent-facing discoverability surfaces in lockstep: + +1. The skill list above (CLAUDE.md) AND the `## Skills` section in `README.md` AND `docs/guides/skills.mdx` (rendered at [hyperframes.heygen.com/guides/skills](https://hyperframes.heygen.com/guides/skills)). Out-of-date entries silently kill discovery. +2. If the skill changes the routing surface for "make a video" requests, also update the capability map and intent router in `skills/hyperframes/SKILL.md` — that's the canonical router agents read first. +3. Mirror the Router / Creation workflows / Domain skills grouping across all three surfaces so a skill always lives in the same column. +4. Skill count appears in the README and CLAUDE.md intro lines ("19 AI agent skills…") — update on add/remove. The `docs/guides/skills.mdx` page deliberately omits a count to avoid drift; keep it count-free. -**Porting an existing composition?** `/remotion-to-hyperframes` translates a Remotion (React) video composition into HyperFrames HTML — a source migration, separate from the creation workflows above. +The skill's own `SKILL.md` frontmatter `description:` is the source of truth for the one-line "use when" blurb; copy from there into the catalog rather than paraphrasing. ## Build & Test diff --git a/README.md b/README.md index 7c35c60cf2..54cc2427bf 100644 --- a/README.md +++ b/README.md @@ -47,6 +47,48 @@ Try a prompt like: The skills teach agents the HyperFrames production loop: plan the video, write valid HTML, wire seekable animations, add media, lint, preview, and render. They work with Claude Code, Cursor, Gemini CLI, Codex, and other coding agents that support skills. +## Skills + +HyperFrames ships 19 skills agents load on demand. Read `/hyperframes` first — it's the router and capability map; it picks a workflow for any "make me a video" request and points to the domain skills below. + +Run `npx skills add heygen-com/hyperframes` for the interactive picker, `npx skills add heygen-com/hyperframes --all` to install all 19 at once (skips the picker), or `npx skills add heygen-com/hyperframes --skill ` for just one (bare name, no leading `/`). + +### Router + +| Skill | Use when | +| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `/hyperframes` | **Read first** for any request to make / create / edit / animate / render a video, animation, or motion graphic. Capability map for the domain skills and intent router for the creation workflows below. | + +### Creation workflows + +| Skill | Use when | +| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `/product-launch-video` | Marketing / launching / promoting a **product** — from its URL, a brief, or a script (even if the site is only named). Up to ~3 min (sweet spot 30-90s). | +| `/website-to-video` | Turning a **general website** into a video — site tour, portfolio / landing-page showcase, social clip from the site's own visuals. | +| `/faceless-explainer` | **Explaining a topic / concept** from arbitrary text — no product, no URL, no website capture; every visual is LLM-invented (typography / abstract / diagram / data-viz). | +| `/pr-to-video` | A **GitHub pull request** (PR URL, `owner/repo#N` ref, or "this PR") → changelog / feature-reveal / fix / refactor explainer, read via the `gh` CLI. | +| `/embedded-captions` | Adding **captions / subtitles** to an existing talking-head video (footage untouched) — verbatim rail, embedded climax behind the subject, or pure-cinematic embed. | +| `/talking-head-recut` | Packaging an existing talking-head / interview / podcast video with **designed graphic overlays** — lower-thirds, data callouts, kinetic titles, pull-quotes, side panels, PiP. | +| `/motion-graphics` | A short, **unnarrated, design-led motion graphic** (~under 10s) — kinetic type, stat / chart hit, logo sting, lower-third, animated tweet / headline. MP4 or transparent overlay. | +| `/music-to-video` | A **music track** (audio file, or video to pull audio from) → a **beat-synced** video — lyric, slideshow, or kinetic promo; music drives pacing. | +| `/slideshow` | A **presentation / pitch deck / interactive deck** — discrete slides, fragment reveals, branching, hotspot navigation, presenter mode. Output is a navigable deck, not a rendered video. | +| `/general-video` | **Anything else** — longer or multi-scene pieces, brand / sizzle reel, title card, static loop, freeform composition. Input- and length-agnostic fallback. | +| `/remotion-to-hyperframes` | **Porting an existing Remotion** (React) composition's source to HyperFrames HTML. One-way migration, not creation. | + +### Domain skills (loaded on demand) + +Atomic capabilities the creation workflows compose against — pull one when you need that specific layer. + +| Skill | Covers | +| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `/hyperframes-core` | The composition contract — `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, determinism rules. | +| `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). | +| `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. | +| `/hyperframes-media` | Audio + media — TTS voiceover, background music, sound effects, Whisper transcription, background removal, caption authoring (one shared audio engine). | +| `/media-use` | Resolve any media need (BGM, SFX, image, icon) into a frozen local file + ledger record. One verb (`resolve`) over the HeyGen catalog with manifest tracking. | +| `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress`). | +| `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. | + For visual design handoff workflows, see the [Claude Design guide](https://hyperframes.heygen.com/guides/claude-design) and [Open Design guide](https://hyperframes.heygen.com/guides/open-design). ### Manually with the CLI diff --git a/docs/docs.json b/docs/docs.json index 75a3ac0d26..ba2833d27b 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -75,6 +75,7 @@ "group": "Guides", "pages": [ "guides/pipeline", + "guides/skills", "guides/authentication", "guides/video-components", "guides/html-in-canvas", diff --git a/docs/guides/skills.mdx b/docs/guides/skills.mdx new file mode 100644 index 0000000000..5ac8423573 --- /dev/null +++ b/docs/guides/skills.mdx @@ -0,0 +1,106 @@ +--- +title: Skills catalog +description: "Every HyperFrames skill agents load on demand — what each one is for, and how to install them." +--- + +HyperFrames ships AI agent skills via [vercel-labs/skills](https://github.com/vercel-labs/skills). They teach your agent the framework-specific patterns — `data-*` attributes, GSAP timeline registration, adapter registries, runtime-owned media playback — that generic web docs don't cover. **Install them before writing compositions** and your agent produces valid HyperFrames HTML on the first try. + +The skills split into three groups: + +- **Router** — the entry skill that picks a workflow for any "make me a video" request. +- **Creation workflows** — one per input shape (URL, PR, music track, captions, etc.). Read by the router; you can also invoke directly if you already know which one fits. +- **Domain skills** — atomic capabilities (animation, media, CLI, registry) the workflows compose against. Load one when you need that specific layer. + +## Install + + + + ```bash + npx skills add heygen-com/hyperframes + ``` + + Opens a picker so you can choose which skills to add. Works with [Claude Code](https://claude.ai/claude-code), [Cursor](https://cursor.sh), [Gemini CLI](https://github.com/google-gemini/gemini-cli), [Codex CLI](https://github.com/openai/codex), [GitHub Copilot CLI](/guides/copilot-cli), and [Google Antigravity](/guides/antigravity). + + + ```bash + npx skills add heygen-com/hyperframes --all + ``` + + Writes every skill to your project in one shot. Recommended when you want the full set without selecting from the picker. + + + ```bash + npx skills add heygen-com/hyperframes --skill + ``` + + Pass the bare skill name (no leading `/`) — e.g. `--skill hyperframes-animation`. Useful when you want a single capability without the full set. + + + Once installed, invoke `/hyperframes` in your agent. It's the capability map for everything below and routes "make me a video" intent to the right creation workflow based on your input. + + + + + **Don't see a slash command after install?** Open a new agent session, or run `/skills` (Copilot CLI) / restart your agent's skill loader. Most agents pick up new skills on the next prompt. + + +## Router + +The single entry point. Read `/hyperframes` before invoking anything else — it knows what's available and which workflow fits your request. + +| Skill | Use when | +| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `/hyperframes` | **Read first** for any request to make / create / edit / animate / render a video, animation, or motion graphic. Capability map for the domain skills and intent router for the creation workflows below. | + +## Creation workflows + +One workflow per input shape. The router (`/hyperframes`) picks one of these for you — but you can invoke them directly when you already know which fits. + +| Skill | Use when | +| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `/product-launch-video` | Marketing / launching / promoting a **product** — from its URL, a brief, or a script (even if the site is only named). Up to ~3 min (sweet spot 30-90s). | +| `/website-to-video` | Turning a **general website** into a video — site tour, portfolio / landing-page showcase, social clip from the site's own visuals. | +| `/faceless-explainer` | **Explaining a topic / concept** from arbitrary text — no product, no URL, no website capture; every visual is LLM-invented (typography / abstract / diagram / data-viz). | +| `/pr-to-video` | A **GitHub pull request** (PR URL, `owner/repo#N` ref, or "this PR") → changelog / feature-reveal / fix / refactor explainer, read via the `gh` CLI. | +| `/embedded-captions` | Adding **captions / subtitles** to an existing talking-head video (footage untouched) — verbatim rail, embedded climax behind the subject, or pure-cinematic embed. | +| `/talking-head-recut` | Packaging an existing talking-head / interview / podcast video with **designed graphic overlays** — lower-thirds, data callouts, kinetic titles, pull-quotes, side panels, PiP. | +| `/motion-graphics` | A short, **unnarrated, design-led motion graphic** (~under 10s) — kinetic type, stat / chart hit, logo sting, lower-third, animated tweet / headline. MP4 or transparent overlay. | +| `/music-to-video` | A **music track** (audio file, or video to pull audio from) → a **beat-synced** video — lyric, slideshow, or kinetic promo; music drives pacing. | +| `/slideshow` | A **presentation / pitch deck / interactive deck** — discrete slides, fragment reveals, branching, hotspot navigation, presenter mode. Output is a navigable deck, not a rendered video. | +| `/general-video` | **Anything else** — longer or multi-scene pieces, brand / sizzle reel, title card, static loop, freeform composition. Input- and length-agnostic fallback. | +| `/remotion-to-hyperframes` | **Porting an existing Remotion** (React) composition's source to HyperFrames HTML. One-way migration, not creation. | + +## Domain skills (loaded on demand) + +Atomic capabilities the creation workflows compose against — pull one when you need that specific layer. + +| Skill | Covers | +| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `/hyperframes-core` | The composition contract — `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, determinism rules. | +| `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). | +| `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. | +| `/hyperframes-media` | Audio + media — TTS voiceover, background music, sound effects, Whisper transcription, background removal, caption authoring (one shared audio engine). | +| `/media-use` | Resolve any media need (BGM, SFX, image, icon) into a frozen local file + ledger record. One verb (`resolve`) over the HeyGen catalog with manifest tracking. | +| `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress`). | +| `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. | + +## Source of truth + +The one-line "use when" for each skill comes from its own `SKILL.md` frontmatter `description:` field in the [hyperframes repo](https://github.com/heygen-com/hyperframes/tree/main/skills). The same catalog lives in the [README's `## Skills` section](https://github.com/heygen-com/hyperframes#skills) and the repo `CLAUDE.md`; all three surfaces are kept in sync when a skill is added or renamed. + +## Next steps + + + + Install skills, scaffold a project, and render your first video. + + + Produce a valid first draft in Claude Design, then refine in any AI coding agent. + + + Install and invoke HyperFrames skills from Copilot CLI in your terminal. + + + Use HyperFrames skills in Google Antigravity. + + diff --git a/docs/introduction.mdx b/docs/introduction.mdx index 671310dd51..c10e106874 100644 --- a/docs/introduction.mdx +++ b/docs/introduction.mdx @@ -39,6 +39,9 @@ Run `npx hyperframes render --output demo.mp4` and this produces an MP4 with det Go from zero to rendered video in under 5 minutes. + + Every HyperFrames AI agent skill — router, creation workflows, and domain skills. Install with `npx skills add heygen-com/hyperframes`. + ## Why Hyperframes? @@ -117,6 +120,9 @@ Run `npx hyperframes render --output demo.mp4` and this produces an MP4 with det Build and render your first video in 60 seconds + + Every HyperFrames AI agent skill — router, creation workflows, and domain skills + Understand the HTML-based data model behind every video diff --git a/docs/quickstart.mdx b/docs/quickstart.mdx index 5a1b756fa2..ce8e8ada59 100644 --- a/docs/quickstart.mdx +++ b/docs/quickstart.mdx @@ -26,7 +26,7 @@ The installer shows a picker. Select the **core set** every project needs — th | `/hyperframes-registry` | Install catalog blocks and components | | `/general-video` | The general authoring workflow — the fallback for any video that doesn't match a specific workflow | -The **workflow skills** are optional — add the ones that match your inputs, and `/hyperframes` routes to whichever you've installed: `/product-launch-video`, `/website-to-video`, `/faceless-explainer`, `/pr-to-video`, `/embedded-captions`, `/talking-head-recut`, `/motion-graphics`, `/remotion-to-hyperframes`. +The **workflow skills** are optional — add the ones that match your inputs, and `/hyperframes` routes to whichever you've installed: `/product-launch-video`, `/website-to-video`, `/faceless-explainer`, `/pr-to-video`, `/embedded-captions`, `/talking-head-recut`, `/motion-graphics`, `/music-to-video`, `/slideshow`, `/general-video`, `/remotion-to-hyperframes`. See the [skills catalog](/guides/skills) for the full list with one-line "use when" descriptions. To skip the picker and install everything (core + every workflow) in one shot, run `npx skills add heygen-com/hyperframes --all`.