diff --git a/README.md b/README.md index 59d60c0..ca34a7e 100644 --- a/README.md +++ b/README.md @@ -195,6 +195,8 @@ Read [the agent guide](docs/AGENT_GUIDE.md) before automating the CLI and [the p - Record the real workflow in Cursorful at 1080p, then caption/render it here. Cursorful remains an operator tool, not a project dependency. - Generate a local ElevenLabs narration file with `npm run voiceover:elevenlabs -- --script narration.txt --voice-id VOICE_ID`. +- Build a reusable local ElevenLabs phrase library with `npm run voiceover:library -- --budget 36000`; use `--resume` to continue safely. The library is indexed by voice and phrase category, and its manifests contain no API key. +- Restore the generated library from the [voice library guide](docs/VOICE_LIBRARY.md) when working from a fresh clone. - Create opt-in, human-reviewed fal assets with `npm run fal:image-edit -- ... --approved-for-generated-marketing` or `npm run fal:reference-video -- ... --approved-for-generated-marketing`. - Generated images/video are never eBay source-of-truth/main listing photos or evidence of condition. Full setup and QA details: [AI provider workflows](docs/AI_PROVIDERS.md). diff --git a/assets/clipcaptionai-self-ad/plan.svg b/assets/clipcaptionai-self-ad/plan.svg new file mode 100644 index 0000000..79f4962 --- /dev/null +++ b/assets/clipcaptionai-self-ad/plan.svg @@ -0,0 +1,13 @@ + + + + + + $ npm run clipkit -- video plan + --brief-file brief.txt + --assets-dir approved-assets + ✓ planned 8 shots + ✓ inputs hashed + outputs/video-runs/self-ad/run.json + Plan before you render. + diff --git a/assets/clipcaptionai-self-ad/qa.svg b/assets/clipcaptionai-self-ad/qa.svg new file mode 100644 index 0000000..66f869f --- /dev/null +++ b/assets/clipcaptionai-self-ad/qa.svg @@ -0,0 +1,13 @@ + + + + + + $ npm run clipkit -- video qa + ✓ file-exists + ✓ h264-video + ✓ audio-not-silent + ✓ manifest + hashes + QA PASSED + Ship with proof. + diff --git a/assets/clipcaptionai-self-ad/render.svg b/assets/clipcaptionai-self-ad/render.svg new file mode 100644 index 0000000..64b6767 --- /dev/null +++ b/assets/clipcaptionai-self-ad/render.svg @@ -0,0 +1,13 @@ + + + + + + $ npm run clipkit -- video render + --run outputs/video-runs/self-ad + Composition PromptVideo + Codec h264 + Output final/self-ad.mp4 + + Render the cut. + diff --git a/docs/AI_PROVIDERS.md b/docs/AI_PROVIDERS.md index abc89ab..ac6b916 100644 --- a/docs/AI_PROVIDERS.md +++ b/docs/AI_PROVIDERS.md @@ -17,6 +17,14 @@ npm run voiceover:elevenlabs -- \ The command writes MP3 audio and a sibling generation manifest containing the voice/model IDs, text hash, response request ID, and audio hash. It never writes the key or narration text into that manifest. +To spend a bounded character budget building reusable local assets across the configured cloned voice and available premade voices: + +```bash +npm run voiceover:library -- --budget 36000 --resume +``` + +This writes `outputs/voiceover/elevenlabs-library/library.json`, one MP3 plus one non-secret manifest per phrase, and retries only safe provider failures. Use `--dry-run` before a large batch; the command checks the live subscription balance and leaves a safety reserve. The checked-in phrase catalog covers hooks, workflow, features, captions, B-roll, quality, and calls to action. Generated audio remains subject to human review for pronunciation, tone, and licensing suitability. + For the demo, review the exact narration before generation and make sure it clearly explains how Codex and GPT-5.6 were used. ## fal reviewed marketing assets diff --git a/docs/ASSET_RECOVERY.md b/docs/ASSET_RECOVERY.md new file mode 100644 index 0000000..edd29c4 --- /dev/null +++ b/docs/ASSET_RECOVERY.md @@ -0,0 +1,36 @@ +# Asset Recovery Bundle + +The `asset-recovery-2026-07-23` GitHub Release preserves the verified self-created ClipCaptionAI B-roll cards and the local public-source SFX library. + +The B-roll bundle contains: + +- `plan.svg` +- `render.svg` +- `qa.svg` + +The separate `clipcaptionai-sfx-library.tar.gz` release asset contains all 256 local SFX files plus `sfx-library/index.json`. + +Restore them from a fresh checkout with: + +```bash +mkdir -p /tmp/clipcaptionai-asset-recovery +gh release download asset-recovery-2026-07-23 \ + --repo jongan69/ClipCaptionAI \ + --pattern 'clipcaptionai-cleared-assets.tar.gz' \ + --dir /tmp/clipcaptionai-asset-recovery +tar -xzf /tmp/clipcaptionai-asset-recovery/clipcaptionai-cleared-assets.tar.gz \ + -C . +``` + +To restore the SFX library as well: + +```bash +gh release download asset-recovery-2026-07-23 \ + --repo jongan69/ClipCaptionAI \ + --pattern 'clipcaptionai-sfx-library.tar.gz' \ + --dir /tmp/clipcaptionai-asset-recovery +tar -xzf /tmp/clipcaptionai-asset-recovery/clipcaptionai-sfx-library.tar.gz \ + -C . +``` + +The SFX files are preserved because they are publicly available local assets, but public availability is not the same as verified commercial-use clearance. Review source terms before publishing a video commercially. The `music-library/` manifest explicitly marks its tracks `review_before_commercial_use`, so music remains excluded. The downloaded YouTube/movie `scene-library/` is intentionally excluded until its rights are reviewed. diff --git a/docs/VOICE_LIBRARY.md b/docs/VOICE_LIBRARY.md new file mode 100644 index 0000000..ed7b20a --- /dev/null +++ b/docs/VOICE_LIBRARY.md @@ -0,0 +1,29 @@ +# ElevenLabs Voice Library + +The repository contains the generator and manifests for the local ElevenLabs phrase library. The generated MP3 files are distributed as a GitHub Release asset so normal clones stay small. + +## Restore the generated library + +From a fresh checkout: + +```bash +mkdir -p /tmp/clipcaptionai-voice-library +gh release download voice-library-2026-07-23 \ + --repo jongan69/ClipCaptionAI \ + --pattern 'clipcaptionai-elevenlabs-library.tar.gz' \ + --dir /tmp/clipcaptionai-voice-library +tar -xzf /tmp/clipcaptionai-voice-library/clipcaptionai-elevenlabs-library.tar.gz \ + -C outputs/voiceover +``` + +The archive restores `outputs/voiceover/elevenlabs-library/`, including 672 MP3 clips, per-clip generation manifests, and `library.json`. Verify the download before extracting it: + +```text +SHA-256: 3fb5e58e7a6acde17ac81c4c78ddb5e7294dd5a80fc89d4a80b142adc81f3d29 +``` + +The generated audio is reusable production material, but still requires human review for pronunciation, tone, and suitability. The source generator is resumable and checks the live ElevenLabs balance: + +```bash +npm run voiceover:library -- --resume --budget 36000 +``` diff --git a/examples/clipcaptionai-self-ad-brief.txt b/examples/clipcaptionai-self-ad-brief.txt new file mode 100644 index 0000000..fe35d59 --- /dev/null +++ b/examples/clipcaptionai-self-ad-brief.txt @@ -0,0 +1,8 @@ +Meet ClipCaptionAI. +Turn a creative brief into a real video run. +Use your footage, images, captions, B-roll, narration, and AI providers. +Let your coding model direct the workflow through a reproducible CLI. +Plan the shots and inspect the run before spending provider credits. +Render a polished cut from the command line. +Every render gets a manifest, hashes, output metadata, and technical QA. +ClipCaptionAI. Prompt it. Render it. Ship the cut. diff --git a/examples/clipcaptionai-self-ad-voiceover.txt b/examples/clipcaptionai-self-ad-voiceover.txt new file mode 100644 index 0000000..520d403 --- /dev/null +++ b/examples/clipcaptionai-self-ad-voiceover.txt @@ -0,0 +1 @@ +Meet ClipCaptionAI. It turns a creative brief into a real video run. Use your own footage and images, add captions, B-roll, narration, music, or AI-generated assets. Your coding model can direct the workflow through a reproducible command line. Every render records its plan, input hashes, output metadata, and technical quality checks. ClipCaptionAI: prompt it, render it, and ship the cut. diff --git a/package.json b/package.json index f4e1bab..fe129f8 100644 --- a/package.json +++ b/package.json @@ -99,6 +99,7 @@ "sample:props": "node scripts/make-sample-props.mjs", "video": "node scripts/video.mjs", "voiceover:elevenlabs": "node scripts/generate-elevenlabs-voiceover.mjs", + "voiceover:library": "node scripts/generate-elevenlabs-library.mjs", "fal:image-edit": "node scripts/fal-image-edit.mjs", "fal:reference-video": "node scripts/fal-reference-video.mjs", "typecheck": "tsc --noEmit", diff --git a/scripts/generate-elevenlabs-library.mjs b/scripts/generate-elevenlabs-library.mjs new file mode 100644 index 0000000..2e14891 --- /dev/null +++ b/scripts/generate-elevenlabs-library.mjs @@ -0,0 +1,187 @@ +#!/usr/bin/env node +import {createHash} from 'node:crypto'; +import fs from 'node:fs'; +import path from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {ensureDir, loadEnv, outputsRoot, parseArgs} from './lib.mjs'; + +const scriptName = path.basename(fileURLToPath(import.meta.url)); +const args = parseArgs(process.argv.slice(2)); +loadEnv(); +const usage = ` +Usage: + npm run voiceover:library -- --budget 36000 + npm run voiceover:library -- --resume --budget 36000 + +Options: + --budget N Maximum planned characters. Default: 36000. + --reserve N Safety reserve below the account limit. Default: 2000. + --out-dir DIR Default: outputs/voiceover/elevenlabs-library. + --model ID Default: eleven_multilingual_v2. + --output-format FORMAT Default: mp3_44100_128. + --resume Skip clips whose audio and manifest already exist. + --dry-run Show the planned library and estimated character cost. + --max-clips N Generate at most N clips after planning. + +Requires ELEVENLABS_API_KEY in .env or the environment. The key is never +written to output files or printed. +`; + +if (args.help || args.h) { + console.log(usage); + process.exit(0); +} + +const clean = (value) => String(value ?? '').replace(/\s+/g, ' ').trim(); +const sha256 = (value) => createHash('sha256').update(value).digest('hex'); +const numeric = (value, fallback) => { + const parsed = Number(value ?? fallback); + if (!Number.isFinite(parsed) || parsed < 0) throw new Error(`Invalid numeric value: ${value}`); + return parsed; +}; +const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + +const voices = [ + ['jon', process.env.ELEVENLABS_VOICE_ID, 'configured cloned voice'], + ['bella', 'hpp4J3VqNfWAUOO0d1Us', 'professional bright warm'], + ['roger', 'CwhRBWXzGAHq8TQ4Fs17', 'laid-back casual resonant'], + ['sarah', 'EXAVITQu4vr4xnSDxMaL', 'mature reassuring confident'], + ['laura', 'FGY2WhTYpPnrIDTdsKH5', 'enthusiastic social creator'], + ['charlie', 'IKne3meq5aSn9XLyUdCD', 'deep confident energetic'], + ['liam', 'TX3LPaxmHKxFdv7VOQHJ', 'energetic social creator'], + ['alice', 'Xb7hH8MSUJpSbSDYk0k2', 'clear engaging educator'], + ['eric', 'cjVigY5qzO86Huf0OWal', 'smooth trustworthy'], + ['george', 'JBFqnCBsd6RMkjVDRZzb', 'warm captivating storyteller'], + ['callum', 'N2lVS1w4EtoT3dr4eOWO', 'husky character voice'], + ['river', 'SAz9YHcvj6GT2YYXdXww', 'relaxed neutral informative'], + ['harry', 'SOYHLrjzK2X1ezoPC6cr', 'fierce character voice'], + ['matilda', 'XrExE9yKIg1WjnnlVkGX', 'knowledgeable professional'], + ['will', 'bIHbv24MWmeRgasZH58o', 'relaxed optimist'], + ['jessica', 'cgSgspJ2msm6clMCkdW9', 'playful bright warm'], + ['brian', 'nPczCjzI2devNBz1zQrb', 'deep resonant comforting'], + ['daniel', 'onwK4e9ZLuTAKqWW03F9', 'steady broadcaster'], + ['lily', 'pFZP5JQG7iQjIQuC4Bku', 'velvety actress'], + ['adam', 'pNInz6obpgDQGcFmaJgB', 'dominant firm'], + ['bill', 'pqHfZKP75CvOlQylNhV4', 'wise mature balanced'], +].filter(([, voiceId]) => clean(voiceId)); + +// These are intentionally complete, reusable spoken assets rather than one long ad. +// Each phrase can be cut into a hook, explainer, transition, or CTA in a future edit. +const phrases = [ + ['hook', 'Meet ClipCaptionAI, the command-line video editor built for creative teams and AI models.'], + ['hook', 'Start with a brief, a folder of approved assets, and a clear outcome. ClipCaptionAI turns that direction into a video run.'], + ['hook', 'Your next product video should not begin with a blank timeline. It should begin with a plan you can inspect.'], + ['hook', 'From idea to finished cut, ClipCaptionAI keeps the creative brief, assets, render, and quality checks connected.'], + ['workflow', 'Plan a run before spending provider credits. Review the shots, sources, prompts, framing, and export settings first.'], + ['workflow', 'The model can direct the workflow, while the CLI keeps every decision explicit, reproducible, and easy to resume.'], + ['workflow', 'A run manifest records what was requested, what was rendered, which files were used, and what passed technical QA.'], + ['workflow', 'Use dry-run mode to validate paths, providers, and output settings before an external generation call begins.'], + ['workflow', 'Resume an existing run instead of starting over. The manifest is the handoff point between planning, rendering, and review.'], + ['feature', 'Bring your own footage, product photos, logos, captions, music, sound effects, and approved B-roll into one composition.'], + ['feature', 'Generate clean vertical, horizontal, or contained layouts from versioned configuration instead of editing source code.'], + ['feature', 'Choose shot recipes, caption styles, audio presets, and export settings that match the channel you are publishing to.'], + ['feature', 'Add narration as a real audio input, mix it above a music bed, and verify that the final file is not silently broken.'], + ['feature', 'Use local assets for reliable demos, then add OpenAI, ElevenLabs, or fal generation when the brief calls for it.'], + ['feature', 'The renderer stays deterministic, so a model can make creative choices without losing control of the final export.'], + ['feature', 'Every successful run produces a final artifact, a manifest, hashes, media metadata, and a machine-readable QA result.'], + ['feature', 'The desktop app is optional and thin. The CLI is the production surface that works for people, scripts, and coding agents.'], + ['captions', 'Captions are part of the composition, not an afterthought. Keep the message readable, paced, and safe inside the frame.'], + ['captions', 'Use a clear headline for the hook, supporting copy for the proof, and a concise call to action at the end.'], + ['captions', 'A good caption survives muted playback. A good voiceover adds rhythm, context, and confidence without fighting the visuals.'], + ['broll', 'Show the work as it happens: the brief becomes a plan, the plan becomes a render, and the render becomes a checked deliverable.'], + ['broll', 'Use interface captures, product details, source footage, and workflow cards to make the benefit visible in seconds.'], + ['broll', 'B-roll should prove the product promise. Show inputs, decisions, transformations, and the final result instead of decorative noise.'], + ['quality', 'Before you ship, check that the file exists, the duration is valid, the dimensions are correct, the codec is supported, and the audio is present.'], + ['quality', 'Technical QA catches black screens, missing audio, wrong framing, broken paths, and incomplete renders before your audience does.'], + ['quality', 'A passing manifest is evidence about this artifact and this run. It does not pretend that an unverified provider completed work remotely.'], + ['quality', 'Keep secrets in the environment. Keep prompts, model IDs, request IDs, hashes, and QA state in the non-secret manifest.'], + ['cta', 'ClipCaptionAI. Prompt it, render it, inspect it, and ship the cut.'], + ['cta', 'Turn the next creative brief into a video you can actually review. Try ClipCaptionAI today.'], + ['cta', 'Stop losing the story between the prompt and the export. Keep the whole run in one place with ClipCaptionAI.'], + ['cta', 'Build once, review clearly, and reuse the assets that work. ClipCaptionAI is your model-facing video production CLI.'], + ['cta', 'When the brief is ready, the next step is simple: plan the run, render the cut, and let QA tell you what shipped.'], +]; + +const apiKey = clean(process.env.ELEVENLABS_API_KEY); +const budget = numeric(args.budget, 36000); +const reserve = numeric(args.reserve, 2000); +const modelId = clean(args.model ?? 'eleven_multilingual_v2'); +const outputFormat = clean(args['output-format'] ?? 'mp3_44100_128'); +const outDir = path.resolve(args['out-dir'] ?? path.join(outputsRoot, 'voiceover', 'elevenlabs-library')); +const indexPath = path.join(outDir, 'library.json'); +const resume = args.resume === true; + +if (!voices.length) throw new Error('No ElevenLabs voices configured. Set ELEVENLABS_VOICE_ID in .env.'); +const planned = []; +for (const [voiceKey, voiceId, voiceDescription] of voices) { + for (const [index, [category, text]] of phrases.entries()) { + const id = `${String(index + 1).padStart(2, '0')}-${category}-${voiceKey}`; + const audio = path.join(outDir, voiceKey, `${id}.mp3`); + planned.push({id, category, voice_key: voiceKey, voice_id: voiceId, voice_description: voiceDescription, text, text_characters: text.length, audio}); + } +} +const maxClips = args['max-clips'] === undefined ? planned.length : Math.floor(numeric(args['max-clips'], planned.length)); +const selected = planned.slice(0, maxClips); +const pending = resume + ? selected.filter((item) => { + const manifestPath = item.audio.replace(/\\.mp3$/i, '.generation.json'); + return !(fs.existsSync(item.audio) && fs.existsSync(manifestPath)); + }) + : selected; +// eleven_multilingual_v2 currently reports a character cost near 0.5x raw text +// for this account. Keep a conservative ceil per clip, while recording the +// authoritative provider cost in each generation manifest. +const estimated = pending.reduce((sum, item) => sum + Math.ceil(item.text_characters * 0.5), 0); +if (estimated > budget) throw new Error(`Planned text costs ${estimated} characters, above --budget ${budget}. Reduce --max-clips or increase the budget.`); + +if (args['dry-run'] === true) { + console.log(JSON.stringify({provider: 'elevenlabs', model_id: modelId, output_format: outputFormat, voices: voices.map(([key, id, description]) => ({key, voice_id: id, description})), clips: selected.length, pending_clips: pending.length, estimated_characters: estimated, budget, reserve, output_directory: outDir, dry_run: true}, null, 2)); + process.exit(0); +} +if (!apiKey) throw new Error('ELEVENLABS_API_KEY is required in .env or the environment.'); +ensureDir(outDir); + +const subscriptionResponse = await fetch('https://api.elevenlabs.io/v1/user/subscription', {headers: {'xi-api-key': apiKey}}); +if (!subscriptionResponse.ok) throw new Error(`Could not read ElevenLabs subscription (${subscriptionResponse.status}).`); +const subscription = await subscriptionResponse.json(); +const remaining = Math.max(0, Number(subscription.character_limit ?? 0) - Number(subscription.character_count ?? 0)); +if (estimated > Math.max(0, remaining - reserve)) { + throw new Error(`Planned text costs ${estimated} characters, but only ${remaining} remain after the ${reserve}-character safety reserve.`); +} + +const existing = fs.existsSync(indexPath) ? JSON.parse(fs.readFileSync(indexPath, 'utf8')) : null; +const entries = new Map((existing?.entries ?? []).map((entry) => [entry.id, entry])); +const failures = []; +for (const [position, item] of selected.entries()) { + const manifestPath = item.audio.replace(/\.mp3$/i, '.generation.json'); + if (resume && fs.existsSync(item.audio) && fs.existsSync(manifestPath)) continue; + const endpoint = `https://api.elevenlabs.io/v1/text-to-speech/${encodeURIComponent(item.voice_id)}?output_format=${encodeURIComponent(outputFormat)}`; + let response; + for (let attempt = 1; attempt <= 3; attempt += 1) { + response = await fetch(endpoint, {method: 'POST', headers: {'content-type': 'application/json', 'xi-api-key': apiKey}, body: JSON.stringify({text: item.text, model_id: modelId})}); + if (response.ok || ![408, 409, 429, 500, 502, 503, 504].includes(response.status) || attempt === 3) break; + await sleep(1500 * attempt); + } + if (!response.ok) { + const detail = (await response.text()).slice(0, 500); + failures.push({id: item.id, status: response.status, detail}); + console.error(`Failed ${position + 1}/${selected.length}: ${item.id} (${response.status})`); + continue; + } + const audioBuffer = Buffer.from(await response.arrayBuffer()); + if (!audioBuffer.length) throw new Error(`ElevenLabs returned empty audio for ${item.id}.`); + ensureDir(path.dirname(item.audio)); + fs.writeFileSync(item.audio, audioBuffer); + const entry = {...item, model_id: modelId, output_format: outputFormat, text_sha256: sha256(item.text), audio_sha256: sha256(audioBuffer), audio_bytes: audioBuffer.length, character_cost: Number(response.headers.get('character-cost') ?? item.text_characters), request_id: response.headers.get('request-id'), created_at: new Date().toISOString(), manifest: manifestPath}; + fs.writeFileSync(manifestPath, `${JSON.stringify({provider: 'elevenlabs', script: scriptName, ...entry}, null, 2)}\n`); + entries.set(item.id, entry); + fs.writeFileSync(indexPath, `${JSON.stringify({provider: 'elevenlabs', model_id: modelId, output_format: outputFormat, generated_at: new Date().toISOString(), entries: [...entries.values()], failures}, null, 2)}\n`); + console.error(`Generated ${position + 1}/${selected.length}: ${item.id}`); + await sleep(250); +} + +const finalEntries = [...entries.values()]; +const totalTextCharacters = selected.reduce((sum, item) => sum + item.text_characters, 0); +const totalBilledCharacters = finalEntries.reduce((sum, entry) => sum + Number(entry.character_cost ?? entry.text_characters), 0); +fs.writeFileSync(indexPath, `${JSON.stringify({provider: 'elevenlabs', model_id: modelId, output_format: outputFormat, generated_at: new Date().toISOString(), planned_clips: selected.length, planned_text_characters: totalTextCharacters, estimated_billable_characters: estimated, generated_clips: finalEntries.length, generated_billable_characters: totalBilledCharacters, failures, entries: finalEntries}, null, 2)}\n`); +console.log(JSON.stringify({provider: 'elevenlabs', output_directory: outDir, index: indexPath, planned_clips: selected.length, generated_clips: finalEntries.length, planned_text_characters: totalTextCharacters, generated_billable_characters: totalBilledCharacters, failures: failures.length, remaining_before_run: remaining}, null, 2));