Skip to content

feat(slice-3): admin Create-pack wizard + POST /api/packs/scaffold #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lopadova merged 4 commits into from
May 18, 2026
Merged

feat(slice-3): admin Create-pack wizard + POST /api/packs/scaffold #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
34f48f4
feat(slice-3): admin Create-pack wizard + POST /api/packs/scaffold
lopadova May 18, 2026
1a886ce
fix(slice-3): structured error code + strict type validation + tmp cl…
lopadova May 18, 2026
57b4551
fix(slice-3): trim API inputs, drop license auto-default, document si…
lopadova May 18, 2026
58a1990
fix(slice-3): empty optional strings → undefined; result-state modal …
lopadova May 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions bun.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions docs/PROGRESS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@

## 2026-05-18

- **v1.7 slices 1+2 shipped — pack authoring tutorial + `aqa pack new` CLI.** PR #25 merged (`6cc0013`), prerelease tag `v1.7.0-rc.1` published. **19 review iterations** with Copilot + Codex; the convergence pattern hit a sharp tail (5→1→4→2→1→2→0 real items per round) after Copilot started re-flagging the same ~13 already-addressed comments. Real issues caught and fixed before merge: slug-length validation against derived-ID schema cap (52-char limit), in-memory schema validation of generated Scenario/RiskMap/PackManifest before writing, symlink rejection at both packs/ parent and packDir, non-directory parent rejection, atomic backup-rename `--force` (failed scaffolds restore the original pack), `package.json#files` matching reality, scoped publish guidance, schema-valid profile snippet, integration test asserts `scn-pack-demo-starter` actually executed (rejects false-positives via bundled packs), honest NO_NETWORK_PROBE documentation. 54 tests in `@aqa/kit` (12 pack-new + 42 run-cmd). **Still pending in v1.7:** slice 3 (admin Create-pack wizard) and slice 4 (audit + wire/implement 81 silent admin placeholder buttons, plan in `docs/internal/admin-placeholder-audit.md`). Final `v1.7.0` tag after those slices ship.
- **v1.6 shipped — `aqa run` + bundled packs + ecosystem foundation.** PR #24 merged (`21d7b10`), tag `v1.6.0` pushed, GitHub release published. The CLI now has the missing `aqa run` command that closes the loop between `aqa init` and a real audit trail. **21 review iterations** with Copilot + Codex, every one surfacing a real bug or coverage gap (zero false alarms). 42 TDD tests in `packages/kit/test/run-cmd.test.ts` cover every behavior. Highlights: SUT-aware init pack selection, three-tier pack discovery (project / node_modules / kit-bundled — all 5 baseline packs now ship inside `@aqa/kit`'s tarball via `bundle-packs.mjs`), atomic run-dir creation (TOCTOU-safe for concurrent seeded runs), path-traversal + symlink-escape rejection, `applies_when` filtering, manifest-name dedup with priority, legacy bare-slug aliasing, agent-mode rejection until that driver lands, unrelated-broken-pack tolerance with structured `warnings`, capped error strings (`MAX_DETAIL_PER_KIND`), detail samples in `run_finished` audit event for auditors. **Known scoped follow-ups:** real HTTP probe runner (current is no-network stub → release-gate strict semantics deferred), `EventChainWriter` ↔ `verifyEventChain` canonical-form reconciliation, browser-driven ecosystem smoke.
- **Next macro task — v1.7 pack-authoring story.** Per user confirmation: (a) `docs/PACK-AUTHORING.md` community tutorial, (b) `aqa pack new <slug>` CLI scaffolding, (c) Admin "Create pack" wizard wired over the new CLI. PLUS: a full audit pass on every placeholder button/interaction in the admin panel — no `onClick={() => {}}` or no-op silent clicks. Each placeholder either gets wired to a real endpoint, gets a client-side implementation, or gets an explicit "decorative" doc note.
- **v1.5 admin design integration shipped.** PR #23 merged (`f7b879f`), tag `v1.5.0` pushed, GitHub release created. The 30-screen hi-fi prototype from Claude Design is now the official admin web panel: bundled into `packages/admin/src/app.tsx` (8.9k LOC, `@ts-nocheck`), token-driven CSS, Vite production build. New `E2E (Playwright, admin UI)` CI job runs the full Playwright suite (`*.e2e.ts`) — per-screen smoke for all 19 nav routes + audit-chain verify (OK/tampered) + Findings views (Clusters/List/Kanban) + Replay tabs + risk-map matrix + theme + palette. Total 36 Playwright tests green in 1m27s. **Known scoped tradeoffs (deferred):** in-memory routing only (not URL-driven), live-mode still reads in-file mocks (no real fetch layer wired). Both intentional for the design port; will be picked up in v1.6.
Expand Down
2 changes: 1 addition & 1 deletion docs/internal/admin-placeholder-audit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ The line numbers below were captured against commit `~v1.6.0` (post-merge of PR

### Packs (lines ~6850-7040)

- `6850`, `6853` — top-bar "Import manifest" / "Install pack" (slice 3 wires "Install pack" via the new wizard)
- ~~`6850`, `6853` — top-bar "Import manifest" / "Install pack" (slice 3 wires "Install pack" via the new wizard)~~ **DONE (slice 3, PR #26):** the "Install pack" placeholder was renamed to "Create pack" and wired to the new `<CreatePackWizard>` component that POSTs to `/api/packs/scaffold`. "Import manifest" remains a placeholder for slice 4b.
- `6910`, `6914`, `6925` — pack row actions (toggle / inspect)
- `6986` — pack detail actions
- `7028`, `7033`, `7038` — scenario picker inside pack detail
Expand Down
326 changes: 324 additions & 2 deletions packages/admin/src/app.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4153,6 +4153,322 @@ function FindingsKanban({ findings: initialFindings, onConfirmTerminal }) {

Object.assign(window, { AuditChainViewer, LiveTerminal, ReplayCommandPanel, FindingsKanban });

// =============================================================
// Create-pack wizard (v1.7 slice 3)
// =============================================================
// Front-end for POST /api/packs/scaffold. The endpoint delegates to
// `runPackNew` from @aqa/kit, the same code path as the `aqa pack new`
// CLI — so the form validation here is a usability layer; authoritative
// validation lives server-side. We deliberately keep the form thin: a
// slug + sut-type covers the minimum viable pack, and the optional
// fields (description/author/license/force) are collapsed behind an
// "Advanced" disclosure.
function CreatePackWizard({ open, onClose }) {
const [slug, setSlug] = React.useState('');
const [sutType, setSutType] = React.useState('api');
const [description, setDescription] = React.useState('');
const [author, setAuthor] = React.useState('');
// license is intentionally empty by default — only forwarded when the
// user explicitly types something in Advanced. Otherwise we'd silently
// bake "Apache-2.0" into every pack scaffolded from the admin, which
// overrides whatever default `runPackNew` would otherwise pick and
// surprises users in orgs with a different default license.
const [license, setLicense] = React.useState('');
const [force, setForce] = React.useState(false);
const [advancedOpen, setAdvancedOpen] = React.useState(false);
const [submitting, setSubmitting] = React.useState(false);
const [error, setError] = React.useState(null);
const [result, setResult] = React.useState(null);
const toast = useToast();

// Mirror the Slug schema regex from @aqa/schemas so the user gets
// immediate feedback rather than waiting for the server round-trip.
// The server is still the source of truth (this is a usability check,
// not a security boundary).
const SLUG_PATTERN = /^[a-z0-9](?:-?[a-z0-9])*$/;
const MAX_SLUG_LEN = 52;
const slugTrimmed = slug.trim();
const slugError = (() => {
if (slugTrimmed === '') return null; // empty is "not yet entered", not an error
if (slugTrimmed.length > MAX_SLUG_LEN) {
return `${slugTrimmed.length} chars — max ${MAX_SLUG_LEN}`;
}
if (!SLUG_PATTERN.test(slugTrimmed)) {
return 'lowercase a-z, 0-9, single dashes only';
}
return null;
})();
const canSubmit = slugTrimmed !== '' && slugError === null && !submitting;

function reset() {
setSlug('');
setSutType('api');
setDescription('');
setAuthor('');
setLicense('');
setForce(false);
setAdvancedOpen(false);
setError(null);
setResult(null);
setSubmitting(false);
}

function handleClose() {
if (submitting) return; // don't abandon a request mid-flight
reset();
onClose?.();
}

async function handleSubmit() {
if (!canSubmit) return;
setSubmitting(true);
setError(null);
try {
const body = {
slug: slugTrimmed,
sut_type: sutType,
...(description.trim() ? { description: description.trim() } : {}),
...(author.trim() ? { author: author.trim() } : {}),
...(license.trim() ? { license: license.trim() } : {}),
...(force ? { force: true } : {}),
};
const res = await fetch('/api/packs/scaffold', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
const text = await res.text();
let parsed = null;
try {
parsed = text ? JSON.parse(text) : null;
} catch {
parsed = null;
}
if (!res.ok) {
const msg = parsed?.error ?? `HTTP ${res.status}`;
setError(msg);
toast.push({ kind: 'error', title: 'Create pack failed', body: msg });
return;
}
setResult(parsed);
toast.push({
kind: 'success',
title: 'Pack scaffolded',
body: parsed?.pack_dir ?? slugTrimmed,
});
} catch (e) {
// Network error / no server / CORS. In mock-data dev (admin running
// without @aqa/server), we surface a clear message instead of a
// generic failure so the user knows it's an environment thing.
// Surface BOTH the inline alert and a toast for parity with the
// HTTP-failure branch above — otherwise the user gets different
// feedback for "server returned 500" vs "couldn't reach server"
// even though both mean "your request didn't succeed".
const msg = e instanceof Error ? e.message : String(e);
const full = `Could not reach /api/packs/scaffold (${msg}). The admin is in mock-data mode or the server is down — no files were written.`;
setError(full);
// Toast body uses `full` (with the mock-mode hint) rather than the
// raw exception `msg` — that hint is the most useful piece of
// context when a fetch fails, and the toast disappears on its
// own so the user might miss it if it only contains the cryptic
// exception message.
toast.push({ kind: 'error', title: 'Create pack failed', body: full });
} finally {
setSubmitting(false);
}
}

return (
<Modal
open={open}
onClose={handleClose}
title={result ? 'Pack created' : 'Create pack'}
sub={
result
? 'Your pack is on disk and ready to edit. The wizard wrote the manifest + a starter scenario + a placeholder risk.'
: 'Scaffolds a runnable pack under <project>/packs/<slug>/. Same code path as `aqa pack new` on the CLI.'
}
size="md"
footer={
result ? (
<>
<button className="btn" onClick={handleClose} data-testid="create-pack-done">
Done
</button>
</>
) : (
<>
<button className="btn" onClick={handleClose} disabled={submitting}>
Cancel
</button>
<button
className="btn primary"
data-testid="create-pack-submit"
onClick={handleSubmit}
disabled={!canSubmit}
>
{submitting ? 'Scaffolding…' : (<>
<I.Plus size={12} />
Create pack
</>)}
</button>
</>
)
}
>
{result ? (
<div className="col gap-12" data-testid="create-pack-result">
<Alert kind="success" title="Pack scaffolded successfully">
<div className="col gap-4">
<div>
<strong>Location:</strong>{' '}
<span className="mono" style={{ fontSize: 12 }}>
{result.pack_dir}
</span>
</div>
<div>
<strong>Files:</strong>{' '}
<span className="mono" style={{ fontSize: 11 }}>
{(result.files ?? []).join(', ')}
</span>
</div>
</div>
</Alert>
<div style={{ fontSize: 12, color: 'var(--text-secondary)' }}>
Next: open <code>pack.yaml</code> and <code>scenarios/starter.yaml</code> in your editor,
replace the placeholder probe URL and oracle with real ones, then wire this pack into a
profile in <code>.aqa/profiles.yaml</code>.
</div>
</div>
) : (
<div className="col gap-12">
{error && (
<Alert kind="error" title="Scaffold failed">
{error}
</Alert>
)}
<div className="field-row">
<label className="field-label" htmlFor="cp-slug">
Slug *
</label>
<input
id="cp-slug"
className="input mono"
data-testid="create-pack-slug"
placeholder="pack-myapp-smoke"
value={slug}
onChange={(e) => setSlug(e.target.value)}
autoFocus
/>
<div
className="field-hint"
style={slugError ? { color: 'var(--accent-danger)' } : undefined}
>
{slugError
? slugError
: 'lowercase a-z and 0-9 with single dashes; up to 52 chars. Used as both the manifest `name:` and the on-disk directory name.'}
</div>
</div>
<div className="field-row">
<label className="field-label" htmlFor="cp-sut">
SUT type *
</label>
<select
id="cp-sut"
className="select"
data-testid="create-pack-sut"
value={sutType}
onChange={(e) => setSutType(e.target.value)}
>
<option value="api">api — HTTP service</option>
<option value="web">web — browser UI</option>
<option value="cli">cli — command-line tool</option>
<option value="lib">lib — library / SDK</option>
<option value="agent">agent — LLM / autonomous agent</option>
<option value="pipeline">pipeline — data / build / CI pipeline</option>
</select>
<div className="field-hint">
Controls <code>applies_when.sut_type</code> in the generated manifest. The pack will
only run against projects whose detected SUT type matches.
</div>
</div>
<button
type="button"
className="btn xs ghost"
onClick={() => setAdvancedOpen((v) => !v)}
style={{ alignSelf: 'flex-start' }}
data-testid="create-pack-advanced-toggle"
>
{advancedOpen ? <I.ChevronDown size={11} /> : <I.ChevronRight size={11} />}
Advanced
</button>
{advancedOpen && (
<>
<div className="field-row">
<label className="field-label" htmlFor="cp-desc">
Description
</label>
<input
id="cp-desc"
className="input"
placeholder="One-line summary of what this pack proves."
value={description}
onChange={(e) => setDescription(e.target.value)}
/>
</div>
<div className="field-row">
<label className="field-label" htmlFor="cp-author">
Author
</label>
<input
id="cp-author"
className="input"
placeholder="Your name or team"
value={author}
onChange={(e) => setAuthor(e.target.value)}
/>
</div>
<div className="field-row">
<label className="field-label" htmlFor="cp-license">
License (SPDX)
</label>
<input
id="cp-license"
className="input mono"
placeholder="Apache-2.0 (default if left blank)"
value={license}
onChange={(e) => setLicense(e.target.value)}
/>
<div className="field-hint">
Leave blank to use the kit's default (Apache-2.0). The CLI flag is{' '}
<code>--license &lt;spdx&gt;</code>.
</div>
</div>
<label
className="row gap-8"
style={{ alignItems: 'center', cursor: 'pointer', fontSize: 12 }}
>
<input
type="checkbox"
data-testid="create-pack-force"
checked={force}
onChange={(e) => setForce(e.target.checked)}
/>
<span>
<strong>Force overwrite</strong> — replace the existing pack directory if one is
already at <code>packs/{slugTrimmed || '<slug>'}/</code>. The existing pack is
backed up to a sibling directory and restored on failure.
</span>
</label>
</>
)}
</div>
)}
</Modal>
);
}
Object.assign(window, { CreatePackWizard });

// ============ shell.jsx ============
// =============================================================
// agentic-qa-kit · admin panel — Shell (Sidebar + TopBar + Palette)
Expand Down Expand Up @@ -6900,6 +7216,7 @@ Object.assign(window, { PageFindings, PageFindingDetail, PageRiskMap, PageRiskEd

// ---------------- Packs ----------------
function PagePacks({ onNavigate, onOpenPack }) {
const [createOpen, setCreateOpen] = React.useState(false);
return (
<div className="page" data-screen-label="09 Packs">
<PageHeader
Expand All @@ -6911,13 +7228,18 @@ function PagePacks({ onNavigate, onOpenPack }) {
<I.Upload size={12} />
Import manifest
</button>
<button className="btn sm primary">
<button
className="btn sm primary"
data-testid="packs-create-btn"
onClick={() => setCreateOpen(true)}
>
<I.Plus size={12} />
Install pack
Create pack
</button>
</>
}
/>
<CreatePackWizard open={createOpen} onClose={() => setCreateOpen(false)} />

<Alert kind="warning" title="One pack is unsigned">
<span className="mono">community-stripe@0.3.1</span> is installed without signature
Expand Down
Loading
Loading