Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,9 +182,16 @@ npm run check:technique-template
# and no identical rule/checkpoint body authored inline at multiple sites.
# Near-duplicates of a fragment are reported as warnings. Hard-zero.
npm run check:fragments

# Audience convention: every output declared audience: agent that also carries an
# #### artifact filename must name a JSON artifact (an agent-audience artifact is
# serialized as JSON on disk). Fails only on NEW violations beyond
# scripts/audience-baseline.json (re-snapshot intentional changes with
# --update-baseline).
npm run check:audience
```

The binding-fidelity, technique-template, and fragments guards also run as Vitest tests (`tests/binding-fidelity.test.ts`, `tests/technique-template.test.ts`, `tests/fragments-guard.test.ts`), so `npm test` fails on new binding drift, a template deviation, or fragment drift.
The binding-fidelity, technique-template, fragments, and audience guards also run as Vitest tests (`tests/binding-fidelity.test.ts`, `tests/technique-template.test.ts`, `tests/fragments-guard.test.ts`, `tests/audience-guard.test.ts`), so `npm test` fails on new binding drift, a template deviation, fragment drift, or a non-JSON agent-audience artifact.

## Branch Structure

Expand Down
28 changes: 28 additions & 0 deletions docs/technique-protocol-specification.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ metadata:
<member description>
#### artifact (optional: the persistence filename)
`<filename-or-{token}-template>`
#### audience (optional: the intended reader — human | agent)
`agent`

## Protocol (present when the technique does work)
### <N>. <Title>
Expand All @@ -93,12 +95,38 @@ loader rejects the singular `## Input` / `## Output` (and `## Output(s)`) varian
- `#### <member>` is a named component of the entry (`components[member]`).
- `#### artifact` (Outputs) is the persistence filename — a literal (`code-review.md`) or a
`{token}`-template the worker interpolates at runtime (`{package_name}-plan.md`, the token being a snake_case symbol).
- `#### audience` (Outputs) is the intended reader of the output/artifact — `human` or `agent`.
Absent means `human`. An `agent`-audience artifact is serialized as **JSON** on disk (named under
the same `artifactPrefix` rule as any artifact); a `human`-audience artifact is prose markdown.
- `#### default` (Inputs) is the input's default value.
- An entry whose description opens with `optional` (e.g. `*(optional)*`) is `required: false`.

`parseEntrySubsections` splits an entry's lead description from its `####` members for both inputs
and outputs.

##### Choosing the audience

`audience` makes format follow function, so pick it from who reads the artifact:

- **`agent`** — an artifact written *only* for the next agent to consume as state: ID-bearing
tables, routing / reconciliation / index state, anything a later step reads back mechanically.
- **`human`** — an artifact a person reads linearly: design write-ups, summaries, READMEs.
- **absent ⇒ `human`** — the default when a declaration omits `audience`.

Anti-patterns, per side:

- **`agent`**: no prose narrative and no restating another artifact — reference it by ID or link and
keep the content structured JSON.
- **`human`**: keep the existing state-once, single-source-and-link, and exception-only reporting
discipline.
- **cross-cutting**: don't serialize agent state as prose, and don't dress a human document up as a
data dump.

*Out of scope:* the per-artifact JSON *field schemas* for specific agent artifacts (assumptions-log,
findings-classification, etc.) are a later increment (V5), not part of the `audience` attribute
itself. This attribute states *who reads* an artifact and *that* an agent artifact is JSON; it does
not fix the shape of any particular JSON payload.

#### The symbol model

A technique has one namespace of **mutable symbols**. Direction is **structural — carried by the
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@
"check:variable-model": "tsx scripts/check-variable-model.ts",
"check:fragments": "tsx scripts/check-fragments.ts",
"check:review-mode": "tsx scripts/check-review-mode-gating.ts",
"check:stealth": "tsx scripts/check-stealth-isolation.ts"
"check:stealth": "tsx scripts/check-stealth-isolation.ts",
"check:audience": "tsx scripts/check-audience.ts"
},
"license": "MIT",
"devDependencies": {
Expand Down
1 change: 1 addition & 0 deletions schemas/technique.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@
"description": { "type": "string", "description": "Human-readable description of this output" },
"components": { "$ref": "#/definitions/outputComponentsDefinition" },
"artifact": { "$ref": "#/definitions/outputArtifact", "description": "Optional. When populated, specifies the artifact name to create when persisting this output." },
"audience": { "type": "string", "enum": ["human", "agent"], "description": "Optional. The intended reader of this output/artifact — `human` (a person reads it linearly) or `agent` (the next agent consumes it as state). Absent means `human`. An `agent`-audience artifact is serialized as JSON on disk under the artifactPrefix rule." },
"destination": { "type": "string", "description": "Delivery-only, server-populated on a step-bound get_technique: the session-bag name this output lands under when the step binding remaps it. Absent otherwise — an unremapped output lands under its own id. Never authored in technique markdown." }
},
"required": [ "id" ],
Expand Down
1 change: 1 addition & 0 deletions scripts/audience-baseline.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[]
153 changes: 153 additions & 0 deletions scripts/check-audience.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
/**
* check-audience — agent-audience artifact JSON-format convention guard (#224 V4).
*
* An output declared with `#### audience` = `agent` is written for the next agent to consume as
* state, and by convention (docs/technique-protocol-specification.md §3.2) an agent-audience
* artifact is serialized as JSON on disk. This guard walks every technique `.md` in the corpus
* through the real loader and, for each output that is BOTH `audience: agent` AND carries an
* `#### artifact` filename, asserts the artifact name follows the JSON-format convention: it (or,
* for a `{token}`-template name, its literal suffix) ends in `.json`.
*
* This is a distinct concern from check-binding-fidelity.ts — that guard checks input/output
* binding conformance and treats `#### artifact` as opaque presence. The audience/JSON-format
* convention is a separate one-guard-per-concern check, so it lives in its own script. Enum
* *validity* (`human`|`agent`) is already enforced by the Zod `.strict()` schema at load; this
* guard checks the on-disk *format* convention the schema cannot express.
*
* The current corpus carries no agent-audience adoption, so the expected baseline is empty. Any
* violation is snapshotted in scripts/audience-baseline.json; the guard fails ONLY on violations
* absent from that baseline (i.e. a NEW agent-audience artifact whose name is not JSON). Regenerate
* the baseline after an intentional, reviewed change with:
* npx tsx scripts/check-audience.ts --update-baseline
*
* Run: npx tsx scripts/check-audience.ts [--root <workflows-dir>]
* To check a dedicated worktree's workflows instead of the repo's own ../workflows, pass
* `--root <path>` (or set WORKFLOWS_DIR).
*/
import { readFileSync, readdirSync, existsSync, statSync, writeFileSync } from 'node:fs';
import { join, relative, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { tryLoadMarkdownTechnique, tryLoadNestedTechnique } from '../src/loaders/markdown-technique-loader.js';
import { resolveWorkflowsRoot } from './workflows-root.js';

const DIR = fileURLToPath(new URL('.', import.meta.url));
const ROOT = resolveWorkflowsRoot(resolve(join(DIR, '..', 'workflows')));
const BASELINE = join(DIR, 'audience-baseline.json');

const GROUPED_INDEX = 'TECHNIQUE.md';

export interface AudienceViolation {
/** `<workflow>::<technique-id>::<output-id>` — stable key for the baseline. */
key: string;
detail: string;
}

/**
* An agent-audience artifact must be JSON on disk. Accept a name whose literal suffix is `.json`
* — including a `{token}`-templated name (`{package_name}-state.json`) whose fixed tail is `.json`.
*/
function isJsonArtifactName(name: string): boolean {
return /\.json$/i.test(name.trim());
}

type LoadedTechnique = NonNullable<Awaited<ReturnType<typeof tryLoadMarkdownTechnique>>>;

/**
* Load a technique file, tolerating non-technique `.md` files that share the `techniques/` tree.
* The loader THROWS a parse error on a file with no `## Capability` / no `metadata.version` (e.g. a
* `README.md`); such a file is not a technique, so the walker skips it rather than crashing the
* guard. A file that parses but fails schema validation (a genuinely malformed technique) returns
* null from the loader and is likewise skipped — the schema-validation test owns that failure.
*/
async function tryLoad(load: () => Promise<LoadedTechnique | null>): Promise<LoadedTechnique | null> {
try { return await load(); } catch { return null; }
}

/** Load every technique in a workflow's `techniques/` tree through the real loader. */
async function loadWorkflowTechniques(techniquesDir: string): Promise<Array<{ id: string; technique: LoadedTechnique }>> {
const out: Array<{ id: string; technique: LoadedTechnique }> = [];
if (!existsSync(techniquesDir)) return out;
for (const entry of readdirSync(techniquesDir).sort()) {
const full = join(techniquesDir, entry);
if (statSync(full).isDirectory()) {
// Grouped technique: `<group>/TECHNIQUE.md` index + `<group>/<op>.md` children.
if (existsSync(join(full, GROUPED_INDEX))) {
const index = await tryLoad(() => tryLoadMarkdownTechnique(techniquesDir, entry));
if (index) out.push({ id: entry, technique: index });
}
for (const child of readdirSync(full).sort()) {
if (!child.endsWith('.md') || child === GROUPED_INDEX) continue;
const opName = child.slice(0, -'.md'.length);
const op = await tryLoad(() => tryLoadNestedTechnique(techniquesDir, entry, opName));
if (op) out.push({ id: `${entry}::${opName}`, technique: op });
}
} else if (entry.endsWith('.md') && entry !== GROUPED_INDEX) {
// Flat standalone technique `<id>.md`.
const id = entry.slice(0, -'.md'.length);
const t = await tryLoad(() => tryLoadMarkdownTechnique(techniquesDir, id));
if (t) out.push({ id, technique: t });
}
}
return out;
}

export async function collectAudienceViolations(root: string = ROOT): Promise<AudienceViolation[]> {
const out: AudienceViolation[] = [];
if (!existsSync(root)) return out;
for (const workflow of readdirSync(root).sort()) {
const techniquesDir = join(root, workflow, 'techniques');
if (!existsSync(techniquesDir) || !statSync(techniquesDir).isDirectory()) continue;
for (const { id, technique } of await loadWorkflowTechniques(techniquesDir)) {
for (const o of technique.outputs ?? []) {
// Only agent-audience outputs that also declare an artifact filename are in scope: those
// are the artifacts written to disk that the convention says must be JSON.
if (o.audience !== 'agent') continue;
const name = o.artifact?.name;
if (!name) continue;
if (!isJsonArtifactName(name)) {
out.push({
key: `${workflow}::${technique.id}::${o.id}`,
detail: `output '${o.id}' in technique '${id}' is audience: agent but its artifact name '${name}' is not JSON — an agent-audience artifact is serialized as JSON on disk (rename to a .json filename)`,
});
}
}
}
}
return out.sort((a, b) => a.key.localeCompare(b.key));
}

function loadBaseline(): Set<string> {
if (!existsSync(BASELINE)) return new Set();
try { return new Set(JSON.parse(readFileSync(BASELINE, 'utf-8')) as string[]); } catch { return new Set(); }
}

/** Violations not in the committed baseline (`added`) and baselined keys no longer present (`fixed`). */
export async function diffBaseline(root: string = ROOT): Promise<{ added: AudienceViolation[]; fixed: string[] }> {
const all = await collectAudienceViolations(root);
const baseline = loadBaseline();
return {
added: all.filter(v => !baseline.has(v.key)),
fixed: [...baseline].filter(k => !all.some(v => v.key === k)),
};
}

const isMain = !!process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href;
if (isMain) {
const all = await collectAudienceViolations();
if (process.argv.includes('--update-baseline')) {
writeFileSync(BASELINE, JSON.stringify(all.map(v => v.key), null, 2) + '\n');
process.stdout.write(`audience: baseline updated with ${all.length} entr(ies) at ${relative(process.cwd(), BASELINE)}\n`);
process.exit(0);
}
const baseline = loadBaseline();
const fresh = all.filter(v => !baseline.has(v.key));
const fixed = [...baseline].filter(k => !all.some(v => v.key === k));
if (fresh.length === 0) {
process.stdout.write(`audience: OK — ${all.length} total, ${baseline.size} baselined, 0 NEW${fixed.length ? `, ${fixed.length} fixed` : ''}\n`);
if (fixed.length) process.stdout.write(` ${fixed.length} baselined entr(ies) no longer present — run --update-baseline to shrink the baseline\n`);
process.exit(0);
}
process.stdout.write(`audience: ${fresh.length} NEW violation(s) — an agent-audience artifact is not JSON:\n`);
for (const v of fresh) process.stdout.write(` ${v.key} — ${v.detail}\n`);
process.exit(1);
}
44 changes: 28 additions & 16 deletions src/loaders/markdown-technique-loader.ts
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,9 @@ interface IndexParse {
capability: string;
inputs: Array<{ id: string; description?: string; required?: boolean; components?: Record<string, string>; default?: string }> | undefined;
protocol: ProtocolBlock[] | undefined;
outputs: Array<{ id: string; description?: string; artifact?: { name: string }; components?: Record<string, string> }> | undefined;
// `audience` is carried as an unrefined string so a mistyped value reaches OutputItemDefinitionSchema's
// `human`/`agent` enum and is rejected loudly at load, rather than being narrowed away here.
outputs: Array<{ id: string; description?: string; artifact?: { name: string }; audience?: string; components?: Record<string, string> }> | undefined;
rules: Record<string, string | string[]> | undefined;
}

Expand Down Expand Up @@ -285,31 +287,36 @@ function parseTechniqueIndex(raw: string, sourcePath: string, id: string): Index
};
}

/** Reserved `####` sub-section keys per entry kind: `default` on inputs, `artifact`/`audience` on
* outputs. A sub-section whose title matches one of these is entry metadata, not a component. */
type ReservedKey = 'artifact' | 'default' | 'audience';

/**
* Split an Inputs/Output entry body into its lead description and `####` sub-sections.
* Each sub-section names a component member of the entry; a sub-section whose title equals
* `reserved` (case-insensitive) is pulled out as entry metadata instead of a component —
* `artifact` for outputs (the persistence filename), `default` for inputs (the default value).
* Returns the lead description, the component map (excluding the reserved member), and the
* reserved member's value when present.
* Each sub-section names a component member of the entry; a sub-section whose title matches one of
* the `reserved` keys (case-insensitive) is pulled out as entry metadata instead of a component —
* `artifact`/`audience` for outputs (the persistence filename and intended reader), `default` for
* inputs (the default value). Returns the lead description, the component map (excluding reserved
* members), and a map of each matched reserved member's value.
*/
function parseEntrySubsections(
body: string,
reserved: 'artifact' | 'default',
): { description?: string; components?: Record<string, string>; reserved?: string } {
reserved: readonly ReservedKey[],
): { description?: string; components?: Record<string, string>; reserved: Partial<Record<ReservedKey, string>> } {
const subs = splitSections(body, 4);
// Lead description is everything before the first `#### ` heading.
const firstHeading = body.search(/^####\s/m);
const lead = firstHeading === -1 ? body : body.slice(0, firstHeading);
const description = bodyParagraphs(lead) || undefined;
const out: { description?: string; components?: Record<string, string>; reserved?: string } = {};
const out: { description?: string; components?: Record<string, string>; reserved: Partial<Record<ReservedKey, string>> } = { reserved: {} };
if (description) out.description = description;
const components: Record<string, string> = {};
for (const s of subs) {
const value = bodyParagraphs(s.body);
if (s.title.toLowerCase() === reserved) {
// Strip surrounding inline-code backticks from a filename/default literal.
out.reserved = value.replace(/^`+|`+$/g, '').trim();
const key = reserved.find((r) => r === s.title.toLowerCase());
if (key) {
// Strip surrounding inline-code backticks from a filename/default/enum literal.
out.reserved[key] = value.replace(/^`+|`+$/g, '').trim();
} else {
components[s.title] = value;
}
Expand All @@ -324,13 +331,13 @@ function parseInputsSection(section: Section | undefined): IndexParse['inputs']
if (items.length === 0) return undefined;
const result: NonNullable<IndexParse['inputs']> = [];
for (const item of items) {
const { description, components, reserved } = parseEntrySubsections(item.body, 'default');
const { description, components, reserved } = parseEntrySubsections(item.body, ['default']);
const entry: { id: string; description?: string; components?: Record<string, string>; default?: string } = { id: item.title };
// A leading "(optional)" stays in the description prose — optionality is conveyed at the
// point of use, not synthesized into a flag (the retired `required` field was never enforced).
if (description) entry.description = description;
if (components) entry.components = components;
if (reserved !== undefined) entry.default = reserved;
if (reserved.default !== undefined) entry.default = reserved.default;
result.push(entry);
}
return result;
Expand Down Expand Up @@ -382,16 +389,21 @@ function parseOutputsSection(section: Section | undefined): IndexParse['outputs'
if (items.length === 0) return undefined;
const result: NonNullable<IndexParse['outputs']> = [];
for (const item of items) {
const { description, components, reserved } = parseEntrySubsections(item.body, 'artifact');
const { description, components, reserved } = parseEntrySubsections(item.body, ['artifact', 'audience']);
const out: {
id: string;
description?: string;
artifact?: { name: string };
audience?: string;
components?: Record<string, string>;
} = { id: item.title };
if (description) out.description = description;
if (components) out.components = components;
if (reserved !== undefined) out.artifact = { name: reserved };
if (reserved.artifact !== undefined) out.artifact = { name: reserved.artifact };
// Pass the authored value through verbatim; the `human`/`agent` enum on OutputItemDefinitionSchema
// is the single validator, so a mistyped audience fails loudly at load (technique dropped with a
// logged warning) rather than being silently discarded here.
if (reserved.audience !== undefined) out.audience = reserved.audience;
result.push(out);
}
return result;
Expand Down
Loading
Loading