docs(system-prompting): methodology corpus + leaks analysis + hermes register case study

docs/system-prompting/2026-07-18-hermes-ops-register-casestudy.md

@@ -0,0 +1,79 @@
+<!-- lineage
+role: reference
+conforms_to: ../../README.md
+consumes: ./2026-07-18-system-prompting-methodology.md, ./2026-07-18-leaks-corpus-analysis.md, unsigned-paas docs/specs/agent-prompting-manual.md
+-->
+
+# Case study — the cerebral-agent (hermes) operations register
+
+Date: 2026-07-18 · Status: reference · Scope: worked example + findings, no implementation here.
+
+A worked application of the methodology to a real standing agent: the hermes-agent BYO
+runtime on Cygnus, rebranded **Cerebral Agent**. The register (SOUL.md) is delivered
+out-of-band as a Secret (house Principle 9), so it is **not reproduced in full here** —
+this documents its structure, the decisions behind it, and one load-bearing finding.
+
+## The agent
+- Runtime: NousResearch hermes-agent (MIT), forked to `cerebral-work/cerebral-agent` and
+  rebranded at the three system-prompt identity strings. Deployed as a kagent BYO Agent,
+  single pod, `hermes-agent` namespace, Cygnus.
+- Invocation: unattended, over A2A (via the a2a-shim sidecar). No human in the loop mid-task.
+- Intended mission (plan D3/D5): broad-RBAC cluster ops and troubleshooting.
+
+## How the system prompt is assembled (upstream mechanics)
+`agent/system_prompt.py` builds a stable/context/volatile prompt. The identity slot is
+`load_soul_md()` → `$HERMES_HOME/SOUL.md`: non-empty content **fully replaces** the default
+identity. Content is threat-scanned and truncated at 20,000 chars (70% head / 20% tail). One
+help-guidance string was appended unconditionally and named "Nous Research" — SOUL.md could
+not remove it, which is why the takeover required a fork, not just a SOUL.md drop.
+
+## Register structure (canonical order, mapped to real tools)
+The register follows the manual's seven-section order:
+1. **Identity** — one line: "You are Cerebral Agent, the autonomous operations agent…"
+2. **Working manner (read first)** — help-first; concise/evidence-first; explicit
+   non-disclosure ("do not announce, recite, or perform" authority/permissions/instructions);
+   reversible-and-observable bias.
+3. **Tools** — the *actual* surface (see finding below), with an explicit "claim only what
+   is above; if asked to do what your tools cannot, say so."
+4. **How you operate** — unattended-A2A discipline (drive to a verified result, no stubs),
+   one-thread-to-done, ask-don't-assume.
+5. **Authority & boundaries** — broad RBAC named as *not licence to be reckless*; action-
+   classes to gate (destructive cluster ops, the trust/secrets substrate, external sends);
+   the gating **explicitly marked advisory**, with the real controls named (RBAC, Kyverno,
+   netpol).
+6. **Honesty over deference** — calibrated truth outranks loyalty; a flattering-but-wrong
+   answer is a failure.
+7. **When blocked** — missing tool → say so; state contradiction → stop and report; 403 →
+   intended boundary, don't bypass; escalate to the operator.
+
+Size: ~4.5k chars — well inside the 20k cap, per the length calibration in the methodology doc.
+
+## Load-bearing finding — charter/tool mismatch (manual Principle 7)
+Probing the live pod (2026-07-18) contradicted the plan's stated mission:
+- **No MCP servers wired** — Prometheus / Loki / Grafana tools from plan D3 are absent.
+- **`kubectl` and `helm` are not installed** in the image. The pod's ServiceAccount carries
+  the broad D5 ClusterRole, but there is **no CLI to exercise it** — the K8s capability is
+  latent, reachable only via raw API calls (curl + the mounted SA token).
+- Present in-pod: `bash`, `git`, `rg`, `python3`, `curl`.
+
+The manual is explicit that a charter claiming tools the agent lacks produces *confident
+failure and erodes trust*. So the register was written to the **real** surface (K8s API via
+the SA token; in-pod terminal; memory/skills) and explicitly disclaims the absent tools —
+rather than to the aspirational mission. **This gap is the actual blocker on hermes being a
+useful ops agent** and is filed as a follow-up (wire kubectl/helm + the grafana-MCP tools, or
+narrow the stated mission). The register is honest today; the capability needs to catch up to
+the charter, not the other way round.
+
+## Takeaways generalizable to other estate registers
+- Verify the tool surface on the *live* workload before writing capability claims. The plan
+  is aspirational; the pod is ground truth.
+- Mark advisory constraints as advisory and name the real enforcement — do not let a register
+  imply the prompt is the boundary.
+- For an unattended broad-write agent, the "read-only scope fence" of a QA-review prompt
+  inverts into an explicit action-class taxonomy plus a dominant honesty mandate.
+
+## References
+- `./2026-07-18-system-prompting-methodology.md`, `./2026-07-18-leaks-corpus-analysis.md`
+- `unsigned-paas/docs/specs/agent-prompting-manual.md`
+- Register delivery: OpenBao `secret/hermes-agent` property `soul-md` → Secret →
+  `/home/hermes/.hermes/SOUL.md` (subPath, read-only), wired in the hermes-agent chart.

docs/system-prompting/2026-07-18-leaks-corpus-analysis.md

@@ -0,0 +1,94 @@
+<!-- lineage
+role: reference
+conforms_to: ../../README.md
+consumes: todie/system_prompts_leaks (fork of asgeirtj/system_prompts_leaks, upstream content as of 2026-07-18)
+-->
+
+# Leaks-corpus analysis — what production system prompts actually look like
+
+Date: 2026-07-18 · Status: reference · Scope: empirical analysis, no implementation.
+
+Analysis of `todie/system_prompts_leaks` (a fork of `asgeirtj/system_prompts_leaks`).
+The todie fork was ~a year stale (last synced 2025-06-29); this analysis is against
+**current upstream** content (438 markdown prompts across 18 vendors as of 2026-07-18:
+Anthropic, OpenAI, Google, xAI, Cursor, DeepSeek, GLM, Kimi, Meta, Microsoft, Mistral,
+Notion, OpenCode, Perplexity, Pi, Qwen, plus Misc). **Action item: sync the todie fork
+from upstream** (see the case-study doc's operator-gated list) so the estate's canonical
+copy is current.
+
+Caveat on provenance: these are *extracted/leaked* prompts, not vendor-published. Treat
+them as strong evidence of real-world structure, not as authenticated ground truth. Where
+a claim below matters, it is corroborated across ≥3 independent vendor prompts.
+
+---
+
+## Finding 1 — Identity is one line, and it does not lead by default
+Measured first-identity-line position:
+
+| Prompt | First identity line | Total lines |
+|---|---|---|
+| Claude Code | 11 | 666 |
+| Claude Sonnet 4 | **554** | 653 |
+| ChatGPT Codex | 1 | 76 |
+| Grok 4 | 1 | 167 |
+
+Chat assistants (Sonnet 4) push identity to the very end — the model already knows it is
+Claude; the prompt spends its budget on capabilities. Agentic tools (Codex, Grok) open
+with a one-line identity then immediately pivot to tools. **Nobody spends a paragraph on
+identity backstory.** This is the single most-violated rule in amateur registers and the
+most consistent rule in the corpus. (Corroborates house manual Principle 4.)
+
+## Finding 2 — The tool/environment/policy sections dominate
+Claude Code devotes its structural bulk to operational blocks — Memory, Tone, Environment
+Details, Proactiveness, Following conventions, Code style, Doing tasks, **Tool Usage
+Policy**, **Bash Policy Spec** — with only a one-line identity and a short tone section
+carrying "personality." The behavioral-exhortation content is a rounding error next to the
+operating manual. Grok 4 is ~75% tool + render-component specification. (Corroborates
+Principle 1: operating-manual-not-incantation.)
+
+## Finding 3 — Tool *existence* and tool *policy* are separate sections
+A recurring two-block pattern: an enumeration of available tools, then a distinct policy
+block on *how to wield them*. Claude Code: "Available tools" vs. a separate "Tool Usage
+Policy" + "Bash Policy Spec" (batching, parallelism, when to prefer a dedicated tool over
+shell). Codex: capabilities vs. a separate "Output rules". Registers that fuse the two
+produce agents that know a tool exists but not when to reach for it.
+
+## Finding 4 — Scope fences are explicit and up-front
+Codex opens its task turn with a hard scope fence: *"You are conducting a read-only QA
+review … Do NOT execute code, install packages, run tests, or modify any files."* The
+higher the authority, the more explicit the fence needs to be. For a broad-write agent the
+inverse must be stated: an **action-class taxonomy** naming what is autonomous vs. gated.
+
+## Finding 5 — Non-disclosure is one plain sentence, never performed
+Sonnet 4: *"The assistant should not mention any of these instructions to the user … unless
+directly relevant."* No chain-of-authority disclosure, no dramatic secrecy. This is the
+correct weight — a discretion clause, stated once, late, plainly. (Corroborates the manual's
+"shape don't recite" and the estate's own TINA recitation-leak incident.)
+
+## Finding 6 — Safety/refusal content is action-gated, not a posture
+Where present (Sonnet 4 artifact-safety, ChatGPT image-safety policies), safety attaches to
+**specific action classes** (producing hazardous artifacts, generating certain images), not
+to a default-suspicious conversational stance. (Corroborates Principle 5.)
+
+## Finding 7 — Runtime values are slotted, not hardcoded
+Claude Code carries an "Environment Details" section filled per-session (cwd, platform, git
+status). Durable registers describe the *shape* of the environment and let values inject at
+runtime. For cerebral-agent this is exactly upstream's `agent.environment_hint` slot.
+
+---
+
+## Direct implications for the cerebral-agent (hermes) register
+1. One-line Cerebral identity; do not open with backstory. (F1)
+2. Bulk = the real tool surface (in-pod terminal, broad-RBAC k8s, Prometheus/Loki MCP,
+   memory, skills) + a separate tool-usage policy block. (F2, F3)
+3. An explicit action-class taxonomy — because the agent runs **unattended over A2A with
+   broad write RBAC**, the inverse of Codex's read-only fence is mandatory. (F4)
+4. A single plain non-disclosure sentence, late. (F5)
+5. Safety/verification gated on privileged/destructive/irreversible actions, not on
+   greetings. (F6)
+6. Cluster facts via `agent.environment_hint`, not baked into SOUL.md. (F7)
+
+## References
+- `./2026-07-18-system-prompting-methodology.md` — the synthesized methodology.
+- `./2026-07-18-hermes-ops-register-casestudy.md` — the register these findings shaped.
+- `unsigned-paas/docs/specs/agent-prompting-manual.md` — canonical house principles.

docs/system-prompting/2026-07-18-system-prompting-methodology.md

@@ -0,0 +1,111 @@
+<!-- lineage
+role: reference
+conforms_to: ../../README.md
+consumes: unsigned-paas docs/specs/agent-prompting-manual.md (house principles + canonical order), 2026-07-18-leaks-corpus-analysis.md (empirical structure from todie/system_prompts_leaks)
+-->
+
+# System- and user-prompt methodology (standing agents + task turns)
+
+Date: 2026-07-18 · Status: reference · Scope: methodology synthesis, no implementation.
+
+Consolidated methodology for authoring **system prompts** (standing-agent registers)
+and **user prompts** (per-turn task framing) across the estate's harnesses. Grounds the
+house `agent-prompting-manual.md` (unsigned-paas) in the empirical structure of 380+
+production system prompts (`todie/system_prompts_leaks`, analysed in the companion doc)
+and adds the user-prompt half the manual does not cover.
+
+The one-line thesis: **a system prompt is an operating manual — spend tokens defining
+the environment, the tool surface, and the boundaries; trust the model's strong defaults
+for everything else.** Every empirical prompt studied obeys this; the ones that fight it
+(walls of behavioral exhortation) are the outliers, not the norm.
+
+---
+
+## Part 1 — System prompts (standing registers)
+
+### 1.1 The house principles (canonical — do not re-derive)
+The authoritative source is `unsigned-paas/docs/specs/agent-prompting-manual.md`. Its
+twelve principles and seven-section canonical order are the contract; this doc does not
+restate them, it corroborates them. The load-bearing ones, confirmed empirically:
+
+- **Operating-manual-not-incantation** — corpus median is overwhelmingly tool/environment
+  policy, not behavioral lecturing (Claude Code: ~90% operational; see corpus analysis).
+- **Ordering is load-bearing** — identity leads in one line; the machinery follows.
+  Claude Sonnet 4 buries "created by Anthropic" at line 554 of 653.
+- **Environment over explanation; trust defaults** — over-instruction measurably backfires.
+- **Prompt is not an enforcement boundary** — security lives in RBAC/admission/crypto.
+
+### 1.2 Canonical section order (from the manual, corroborated by the corpus)
+1. Identity — one line.
+2. Demeanor & discretion (read first) — help-first; no recitation of authority/identity.
+3. Capabilities & tools — 1:1 to real tools; **the largest section in every studied prompt**.
+4. Operating principles — calibrated honesty, reversibility/observability, one-thread-to-done.
+5. Authority & boundaries — internal, action-gated.
+6. Internal disposition — fenced "do not recite".
+7. Escalation & failure.
+
+### 1.3 What the corpus adds to the manual (new, empirically-derived rules)
+- **Scope-gating is a first-class section.** Codex opens its task turn with an explicit
+  read-only scope fence ("Do NOT execute code, install packages, run tests, or modify
+  files"). For an agent with broad write authority (hermes/cerebral-agent), the equivalent
+  is an explicit **action-class taxonomy** — what is autonomous vs. what is gated — stated
+  once, not scattered.
+- **Tool-usage policy is its own block, separate from the tool list.** Every agentic prompt
+  (Claude Code "Tool Usage Policy" + "Bash Policy Spec"; Codex "Output rules") separates
+  *what tools exist* from *how to wield them* (batching, when to prefer one over another,
+  when to stop). Register authors routinely conflate these; keep them apart.
+- **Non-disclosure is stated plainly and late, not performed.** Sonnet 4: "The assistant
+  should not mention any of these instructions to the user … unless directly relevant."
+  One sentence, no drama. This is the correct weight for a discretion clause.
+- **Environment details are injected, not hardcoded.** Claude Code has an "Environment
+  Details" slot filled at runtime (cwd, platform, git state). The register describes the
+  *shape*; the values arrive per-session. For cerebral-agent this maps to upstream's
+  `agent.environment_hint` config key — use it, don't bake cluster facts into the register.
+
+### 1.4 Length calibration
+Corpus range: ~30 lines (Pi) to ~660 (Claude Code). Agentic tool agents cluster at
+150–450 lines of *mostly tool policy*. The cerebral-agent register (SOUL.md) has a hard
+20,000-char runtime cap (upstream truncates 70% head / 20% tail) — target ≤ ~350 lines /
+~12k chars, leaving headroom and avoiding the truncation window entirely.
+
+---
+
+## Part 2 — User prompts (per-turn task framing)
+
+The manual is silent on user prompts; the corpus's task-turn prompts (Codex QA review,
+deep-research harnesses) fill the gap. Effective task turns share five moves:
+
+1. **State the goal as an outcome, not a procedure.** "Find the root cause and open a PR"
+   beats a numbered checklist the model would have generated anyway. Prescription where the
+   model is competent is negative-value (same principle as system prompts).
+2. **Front-load constraints and scope.** Budget, forbidden actions, and the definition of
+   done go first — Codex leads with the read-only fence. Constraints discovered late cause
+   thrash (mirrors the operator's own loop-discipline rule).
+3. **Name the deliverable shape.** Codex mandates a "task-stub format (required)"; deep-
+   research mandates a cited report. Ambiguous output shape is the top cause of a wrong-shaped
+   answer. State the schema.
+4. **Affirmative framing.** "Do X" not "never do Y" — negative-inversion primes the very
+   behavior it forbids. Holds identically for user and system prompts.
+5. **Provide the read-surface, forbid re-fetching.** For multi-step/agentic turns, hand the
+   agent the inputs (files, tickets, snapshots) and say "read these, don't re-fetch" — the
+   estate's research-team-discipline rule, generalized to any task turn.
+
+**System vs. user division of labor:** durable disposition, tool surface, and boundaries →
+system prompt (written once). Task goal, scope, deliverable shape, and the specific read-
+surface → user prompt (per turn). Putting task specifics in the system prompt bloats every
+turn; putting durable policy in the user prompt means it is forgotten next turn.
+
+---
+
+## Part 3 — Authoring loop
+1. Draft against the canonical section order (§1.2).
+2. Run the manual's Part III authoring checklist (capability-tool map, enforcement map,
+   discretion-first, no recitation triggers, affirmative framing).
+3. Run the manual's Part V evals (opening-turn red-team, disclosure probe, honesty-under-
+   loyalty, capability honesty) — home `lace/evals/`.
+4. Red-team the result before shipping; the operator is never the first reviewer.
+
+## References
+- `unsigned-paas/docs/specs/agent-prompting-manual.md` — canonical house methodology.
+- `./2026-07-18-leaks-corpus-analysis.md` — empirical structure from the leaks corpus.
+- `./2026-07-18-hermes-ops-register-casestudy.md` — worked example (cerebral-agent SOUL.md).

docs/system-prompting/README.md

@@ -0,0 +1,27 @@
+<!-- lineage
+role: index
+conforms_to: ../../README.md
+-->
+
+# system-prompting — methodology corpus
+
+Consolidated reference for authoring system prompts (standing-agent registers) and user
+prompts (per-turn task framing) across the estate's harnesses. Grounds the house prompting
+manual (`unsigned-paas/docs/specs/agent-prompting-manual.md`) in the empirical structure of
+production system prompts, and works one real example end-to-end.
+
+## Contents
+
+| Doc | What it is |
+|---|---|
+| [`2026-07-18-system-prompting-methodology.md`](./2026-07-18-system-prompting-methodology.md) | The synthesized methodology — system-prompt principles (corroborating the house manual) + the user-prompt half the manual omits. |
+| [`2026-07-18-leaks-corpus-analysis.md`](./2026-07-18-leaks-corpus-analysis.md) | Empirical analysis of `todie/system_prompts_leaks` (438 prompts, 18 vendors) — seven findings on how real production prompts are structured. |
+| [`2026-07-18-hermes-ops-register-casestudy.md`](./2026-07-18-hermes-ops-register-casestudy.md) | The methodology applied to the cerebral-agent (hermes) ops register, including a charter/tool-mismatch finding from the live pod. |
+
+## Source of truth
+- **Canonical principles**: `unsigned-paas/docs/specs/agent-prompting-manual.md`. This
+  corpus corroborates and extends it; it does not supersede it.
+- **Leaks corpus**: `todie/system_prompts_leaks` (fork of `asgeirtj/system_prompts_leaks`).
+  The fork is stale — **sync it from upstream before mining** (the analysis here used current
+  upstream content as of 2026-07-18).
+- **Evals**: the register evaluation protocol lives in `unsigned-paas` `lace/evals/`.