From 2035eb307cc7c2e7771976cecab560afb169efe0 Mon Sep 17 00:00:00 2001 From: Segun Olumbe <11536849+nugehs@users.noreply.github.com> Date: Mon, 17 Aug 2026 09:44:17 +0100 Subject: [PATCH] docs: lead with independent trust harness Stop competing with native agent loops. Position Otito as the independent merge-evidence layer beside Codex, Claude Code, Gemini, and Cursor. --- CHANGELOG.md | 4 + README.md | 11 +- .../README.md | 6 +- docs/07-harness-thesis/README.md | 24 ++- docs/07-harness-thesis/ax-score-spec.md | 4 +- docs/08-tutorials-integration/README.md | 2 + docs/09-convergence-thesis/README.md | 3 +- docs/11-determinism-thesis/README.md | 14 +- docs/12-dual-mode-thesis/README.md | 32 ++-- docs/13-prompt-determinism-thesis/README.md | 14 +- docs/14-trust-harness-thesis/README.md | 181 ++++++++++++++++++ docs/EXECUTIVE-SUMMARY.md | 20 +- docs/GLOSSARY.md | 2 + docs/index.md | 19 +- mkdocs.yml | 2 + src/lib/ax.js | 10 +- 16 files changed, 284 insertions(+), 64 deletions(-) create mode 100644 docs/14-trust-harness-thesis/README.md diff --git a/CHANGELOG.md b/CHANGELOG.md index f8e4caa..6046dd0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ This project follows SemVer. ## [Unreleased] +### Docs + +- **Trust harness positioning.** Lead with independent merge evidence rather than a cheaper or smarter model loop. New thesis page: [docs/14-trust-harness-thesis/README.md](docs/14-trust-harness-thesis/README.md). + ## [1.6.2] - 2026-08-15 ### Changed diff --git a/README.md b/README.md index 7573305..bebad9b 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Òtítọ́ -**Local-first context and merge evidence for coding agents: turn AI-generated changes into trusted commits.** +**Models generate the change. Otito proves whether it is safe to merge.** [![CI](https://img.shields.io/github/actions/workflow/status/BASHBOP/otito/otito-ci.yml?style=flat-square&label=CI)](https://github.com/BASHBOP/otito/actions/workflows/otito-ci.yml) [![npm](https://img.shields.io/npm/v/@bashbop/otito?style=flat-square)](https://www.npmjs.com/package/@bashbop/otito) [![license: MIT](https://img.shields.io/badge/license-MIT-blue?style=flat-square)](LICENSE) [![node](https://img.shields.io/badge/node-%E2%89%A518.18-339933?style=flat-square)](https://nodejs.org/) @@ -16,9 +16,11 @@ \___/ |_| |___| |_| \___/ ``` -An agent's output is bounded by two things: the **model** and the **harness** around it. The harness includes the prompts, the context it's given, the codebase it works in, and the gates it must pass before code merges. You don't control the model. You control the harness, and a tighter harness lets a cheaper model do the same work with fewer wasted tokens. +The generic agent loop (prompting, retries, tool routing, file editing) is becoming infrastructure. Frontier models already plan, search repositories, use tools, and recover from mistakes. Model vendors are packaging native harnesses with file tools and sandbox execution. Competing there is a losing bet. -**Òtítọ́** is that harness layer. It is local-first, deterministic, and model-agnostic: it discovers repositories, builds local indexes, generates task-aware context before an agent edits, scores how much a change actually touches, and gates merge readiness. It works the same way every time, with no server, no account, and no code leaving the machine. Because it depends on software fundamentals rather than any one model, the harness you build today keeps working as models change underneath it. +What remains strategically important is the **trust harness**: accurate repository context, permission and sandbox boundaries, exact validation against the changed code, risk-sensitive policies, CODEOWNERS and human approval, reproducible receipts, and independent evidence the model cannot award itself. Stronger models increase that need, because teams will let them change more code with less supervision. + +**Òtítọ́** is that independent trust layer. It is local-first, deterministic, and model-agnostic: it discovers repositories, builds local indexes, generates task-aware context before an agent edits, scores how much a change actually touches, and gates merge readiness. It works the same way every time, with no server, no account, and no code leaving the machine. It does not compete with Codex, Claude Code, Gemini, Cursor, or future native harnesses. It integrates with them, and keeps working as models change underneath it. ## Trusted agent workflow @@ -30,7 +32,7 @@ Request -> context -> scoped change -> exact validation -> review evidence -> hu Otito helps the agent understand and bound a task before editing, then gives the maintainer evidence to decide whether to trust the result. A passing local gate is never an automatic merge approval: hosted CI, GitHub review, CODEOWNERS, and the human release decision remain separate authorities. -It does not try to replace `opensrc`, `code-structure`, Daytona, or Harnss. It gives developers and coding agents a single CLI that can: +It does not try to replace native agent harnesses, `opensrc`, `code-structure`, Daytona, or Harnss. It gives developers and coding agents a single CLI that can: Deterministic merge gates are the differentiated core. This is the human-in-the-loop checkpoint that a model cannot grade for itself: @@ -76,6 +78,7 @@ The published MkDocs site is a practical discovery and delivery guide: - [Determinism Thesis & Harness Boundary](docs/11-determinism-thesis/README.md) - [Dual-Mode Thesis & Complementary Stack](docs/12-dual-mode-thesis/README.md) - [Prompt Determinism Thesis & Settings Trap](docs/13-prompt-determinism-thesis/README.md) +- [Trust Harness Thesis & Commodity Loop](docs/14-trust-harness-thesis/README.md) - [Glossary](docs/GLOSSARY.md) Read it at: diff --git a/docs/06-builder-founder-operating-loop/README.md b/docs/06-builder-founder-operating-loop/README.md index a529206..73a49ca 100644 --- a/docs/06-builder-founder-operating-loop/README.md +++ b/docs/06-builder-founder-operating-loop/README.md @@ -22,7 +22,7 @@ durable evidence ## Tool Boundaries -Òtítọ́ is the product surface. Its context, impact, and gate commands provide repository context, PR review context, workspace reports, agent-facing evidence, and merge readiness. +Òtítọ́ is the independent trust-harness surface. Its context, impact, and gate commands provide repository context, PR review context, workspace reports, agent-facing evidence, and merge readiness without competing with the coding agent's own loop. `otito impact` is the canonical change-impact analyzer. Use it when scope is unclear, when import-neighbor evidence matters, or when a diff needs validation against the original change request. The standalone `impact-map` work has been absorbed into otito for normal product and agent workflows. @@ -34,7 +34,7 @@ The rule is simple: | Is this PR ready to merge under the repo's governance rules? | Òtítọ́ gate | | What files, import neighbors, tests, or missed diff areas might this change affect? | `otito impact` | -This keeps the public story clear: one product, with context and impact analysis before its final merge-safety gate. +This keeps the public story clear: model-native agents generate changes; one model-agnostic product provides context and impact analysis before its final merge-safety evidence. --- @@ -151,7 +151,7 @@ The ledger can live in a PR description, merge note, release note, company pilot | Track | Question | Artifact | | --- | --- | --- | -| Product | Is otito still the context foundation? | README, docs, CLI/MCP behavior, tests | +| Product | Is otito still the independent context and merge-evidence layer? | README, docs, CLI/MCP behavior, tests | | Gate | Is Òtítọ́ still the merge-safety signal? | Òtítọ́ workflow, local report, PR check, policy mode | | Release | Can another maintainer understand what shipped? | SemVer impact, changelog, tag, GitHub release | | Governance | Can a company see who was accountable? | CODEOWNERS, review policy, branch protection, decision record | diff --git a/docs/07-harness-thesis/README.md b/docs/07-harness-thesis/README.md index bd30ed4..b7088fe 100644 --- a/docs/07-harness-thesis/README.md +++ b/docs/07-harness-thesis/README.md @@ -1,6 +1,6 @@ # Harness Thesis & Agent Experience -> _Why otito bets on the harness, not the model — and the roadmap that follows from it._ +> _Why otito still bets on a harness, which harness is durable, and the AX roadmap that follows._ This document maps a widely-shared argument about agentic engineering onto otito's existing surface, then turns it into a concrete, prioritised roadmap. The source is a @@ -11,12 +11,14 @@ matters most, and naming it sharpens the product. ## The thesis in one line -> Everyone obsesses over the model — the engine of the F1 car. The rest of the car is the -> **harness**: prompts, skills, the codebase, the gates. You control the harness far more -> than you control the model, and a tighter harness lets a cheaper model do the same work. +> Everyone obsesses over the model. You still control a harness, but the generic +> agent loop is becoming vendor infrastructure. Otito owns the **trust harness**: +> context, gates, receipts, and independent merge evidence. A tighter repo still +> lets a cheaper model do more; that is now the AX score, not the lead claim. -otito **is** a harness tool. It is deterministic, local-first, and model-agnostic — the -part of an AI stack that keeps working as models churn underneath it. +otito **is** that trust layer. It is deterministic, local-first, and model-agnostic. +The generic loop belongs to Codex, Claude Code, Gemini, Cursor, and future native +harnesses. See [the trust harness thesis](../14-trust-harness-thesis/README.md). ## Eight lessons, mapped to otito @@ -24,8 +26,11 @@ part of an AI stack that keeps working as models churn underneath it. The video reframes "code context tool" as something bigger: the environment an agent runs in. otito already generates harnesses (`otito harness`, `repo_harness`) and gates -(`otito gate`, `review_gate`). The README now leads with this. The strategic claim — -*the harness survives model churn* — is both true of otito and currently going viral. +(`otito gate`, `review_gate`). Keep that vocabulary, then split it. The generic loop +(prompts, retries, file tools) is becoming infrastructure. The **trust harness** +(context, validation, owners, receipts) is the durable product. The README now leads +with merge evidence, not cheaper models. See +[the trust harness thesis](../14-trust-harness-thesis/README.md). ### 2. "A cheaper model works if the codebase is easy to change" → make it measurable @@ -112,7 +117,8 @@ model cannot do for itself**, framed as the durable half of the stack. - Matt Pocock & David Ondrej, _Agentic Engineering Workflow_ — - otito source referenced above: `src/lib/tokens.js`, `src/lib/impact.js`, `src/lib/review.js`, `src/lib/risk-paths.js`, `src/lib/harness.js` -- Companion: [Determinism Thesis](../11-determinism-thesis/README.md) (why the harness must be non-model), +- Companion: [Trust Harness Thesis](../14-trust-harness-thesis/README.md) (which harness is durable), + [Determinism Thesis](../11-determinism-thesis/README.md) (why the harness must be non-model), [Dual-Mode Thesis](../12-dual-mode-thesis/README.md) (probabilistic generation + deterministic verification), [Prompt Determinism Thesis](../13-prompt-determinism-thesis/README.md) (prompt settings are not a gate), [Convergence Thesis](../09-convergence-thesis/README.md) diff --git a/docs/07-harness-thesis/ax-score-spec.md b/docs/07-harness-thesis/ax-score-spec.md index 7129e29..bfd793a 100644 --- a/docs/07-harness-thesis/ax-score-spec.md +++ b/docs/07-harness-thesis/ax-score-spec.md @@ -23,7 +23,9 @@ _"how cheap and safe is it for an agent to make a change here?"_ for a given tas It is the measurable form of the harness thesis (see [README.md](./README.md), lesson 2): a better codebase needs **fewer tokens** and has -**better guardrails**, so a cheaper model can do the same work. The AX score rises as the +**better guardrails**, so a cheaper model can do the same work. That remains a cost +property. The product lead is independent merge evidence (see +[the trust harness thesis](../14-trust-harness-thesis/README.md)). The AX score rises as the harness improves and falls when the codebase is hard to change. No competing tool ships this. Two framings, same engine: diff --git a/docs/08-tutorials-integration/README.md b/docs/08-tutorials-integration/README.md index 515f015..faddfba 100644 --- a/docs/08-tutorials-integration/README.md +++ b/docs/08-tutorials-integration/README.md @@ -83,3 +83,5 @@ merge-ready — the same trust-layer discipline otito uses on itself. agents plus deterministic merge evidence. - [Prompt Determinism Thesis & Settings Trap](../13-prompt-determinism-thesis/README.md) — why "tell it not to randomize" is not a merge gate. +- [Trust Harness Thesis & Commodity Loop](../14-trust-harness-thesis/README.md) — why independent + merge evidence outlasts generic agent orchestration. diff --git a/docs/09-convergence-thesis/README.md b/docs/09-convergence-thesis/README.md index 8ac93e9..02a3571 100644 --- a/docs/09-convergence-thesis/README.md +++ b/docs/09-convergence-thesis/README.md @@ -136,7 +136,8 @@ execution — the half of verification a model structurally cannot do for itself - otito source referenced above: `src/lib/impact.js` (`generateImpact`, `validateAgainstDiff`), `src/lib/review.js`, `src/lib/risk-paths.js`, `src/lib/codeowners.js`, `src/lib/ax.js` -- Companions: [Dual-Mode Thesis](../12-dual-mode-thesis/README.md) (probabilistic vs deterministic 101), +- Companions: [Trust Harness Thesis](../14-trust-harness-thesis/README.md) (which harness is durable), + [Dual-Mode Thesis](../12-dual-mode-thesis/README.md) (probabilistic vs deterministic 101), [Prompt Determinism Thesis](../13-prompt-determinism-thesis/README.md) (prompt settings are not a gate), [Determinism Thesis](../11-determinism-thesis/README.md) (why model output varies), [Harness Thesis](../07-harness-thesis/README.md) diff --git a/docs/11-determinism-thesis/README.md b/docs/11-determinism-thesis/README.md index 187e87a..0630608 100644 --- a/docs/11-determinism-thesis/README.md +++ b/docs/11-determinism-thesis/README.md @@ -133,11 +133,12 @@ variance with prompt settings, read this doc for why LLM variance is structural, not a settings bug. ```text -Dual-mode (docs/12) → two modes, complementary roles -Prompt determinism (docs/13) → you cannot collapse modes via prompting -Determinism (this doc) → why the model cannot be the trust layer -Harness (docs/07) → what you control instead of the model -Convergence (docs/09) → how you measure intent vs. execution deterministically +Trust harness (docs/14) -> which harness is durable; integrate with native loops +Dual-mode (docs/12) -> two modes, complementary roles +Prompt determinism (docs/13) -> you cannot collapse modes via prompting +Determinism (this doc) -> why the model cannot be the trust layer +Harness (docs/07) -> what you still control; AX as a cost property +Convergence (docs/09) -> how you measure intent vs. execution deterministically ``` ## Priorities @@ -164,7 +165,8 @@ variance, ship deterministic verification, and keep the merge decision human.** - otito source referenced above: `src/lib/context-engine.js`, `src/lib/converge.js`, `src/lib/review.js`, `src/lib/pass-local.js`, `src/lib/impact.js`, `tests/telemetry.test.js`, `tests/context-engine.test.js` -- Companions: [Harness Thesis](../07-harness-thesis/README.md), +- Companions: [Trust Harness Thesis](../14-trust-harness-thesis/README.md), + [Harness Thesis](../07-harness-thesis/README.md), [Convergence Thesis](../09-convergence-thesis/README.md), [Dual-Mode Thesis](../12-dual-mode-thesis/README.md), [Prompt Determinism Thesis](../13-prompt-determinism-thesis/README.md) diff --git a/docs/12-dual-mode-thesis/README.md b/docs/12-dual-mode-thesis/README.md index ecbf6a2..6fbe11a 100644 --- a/docs/12-dual-mode-thesis/README.md +++ b/docs/12-dual-mode-thesis/README.md @@ -22,13 +22,15 @@ the onboarding explainer for why otito sits beside an LLM instead of replacing o ## The thesis in one line -> Coding agents are **probabilistic** — they interpret, generate, and adapt. Merge -> readiness is **deterministic** — same repo state, same rules, same verdict. A +> Coding agents are **probabilistic**: they interpret, generate, and adapt. Merge +> readiness is **deterministic**: same repo state, same rules, same verdict. A > trustworthy agent stack uses **both modes on purpose**, with a bright line between -> generation and verification. +> generation and verification. Native model harnesses own generation. Otito owns +> verification. otito **is** the deterministic mode for repository work: rules, gates, receipts, and -git facts — not token sampling. +git facts, not token sampling. It integrates with Codex, Claude Code, Gemini, and +Cursor rather than competing with their agent loops. ## What lines up — otito is already the deterministic mode @@ -53,10 +55,14 @@ LLM outputs vary at scale. This doc explains *what to do about it architecturall The determinism doc owns "models vary; harnesses don't." This doc owns the **split itself**: *probabilistic generation, deterministic verification.* +[The trust harness thesis](../14-trust-harness-thesis/README.md) then names **which +harness** sits on the deterministic side: independent merge evidence, not the generic +agent loop. That reframes otito from "anti-AI" to "complementary." Teams already accept that chatbots are probabilistic. otito makes the same acceptance explicit for coding agents: -let the model judge and draft; let the harness rule and measure. +let the model judge and draft; let the harness rule and measure. Native hosts generate. +Otito attests. ### 2. Place deterministic work at guarantees @@ -129,14 +135,17 @@ agents** — the if-this-then-that layer for repository trust. ## How the thesis docs fit together ```text -Dual-mode (this doc) → two modes, complementary roles -Prompt determinism (docs/13) → you cannot collapse modes via prompting -Determinism (docs/11) → why the probabilistic mode cannot self-verify -Harness (docs/07) → what you control instead of the model -Convergence (docs/09) → how you measure intent vs. execution deterministically +Trust harness (docs/14) -> which harness is durable; integrate with native loops +Dual-mode (this doc) -> two modes, complementary roles +Prompt determinism (docs/13) -> you cannot collapse modes via prompting +Determinism (docs/11) -> why the probabilistic mode cannot self-verify +Harness (docs/07) -> what you still control; AX as a cost property +Convergence (docs/09) -> how you measure intent vs. execution deterministically ``` Read this doc first for onboarding. Read +[the trust harness thesis](../14-trust-harness-thesis/README.md) when the question is +whether generic agent loops still matter. Read [the prompt determinism thesis](../13-prompt-determinism-thesis/README.md) when someone proposes "just tell it not to randomize." Read determinism next for the engineering depth on LLM variance. Read harness and convergence when designing workflows and gates. @@ -165,7 +174,8 @@ in a stack that still needs probabilistic generation. - otito source referenced above: `src/lib/pass-local.js`, `src/lib/risk-paths.js`, `src/lib/converge.js`, `src/lib/review.js`, `src/lib/context-engine.js`, `codex/skills/otito-context/`, `codex/skills/otito-review/` -- Companions: [Determinism Thesis](../11-determinism-thesis/README.md), +- Companions: [Trust Harness Thesis](../14-trust-harness-thesis/README.md), + [Determinism Thesis](../11-determinism-thesis/README.md), [Harness Thesis](../07-harness-thesis/README.md), [Convergence Thesis](../09-convergence-thesis/README.md), [Prompt Determinism Thesis](../13-prompt-determinism-thesis/README.md) diff --git a/docs/13-prompt-determinism-thesis/README.md b/docs/13-prompt-determinism-thesis/README.md index 4406443..b3289c2 100644 --- a/docs/13-prompt-determinism-thesis/README.md +++ b/docs/13-prompt-determinism-thesis/README.md @@ -123,11 +123,12 @@ merge evidence** — because the chat is not the artifact that ships. ## How the thesis docs fit together ```text -Dual-mode (docs/12) → two modes, complementary roles -Prompt determinism (this doc) → you cannot collapse modes via prompting -Determinism (docs/11) → why generation settings still fail at scale -Harness (docs/07) → what you control instead of the model -Convergence (docs/09) → how you measure intent vs. execution deterministically +Trust harness (docs/14) -> which harness is durable; integrate with native loops +Dual-mode (docs/12) -> two modes, complementary roles +Prompt determinism (this doc) -> you cannot collapse modes via prompting +Determinism (docs/11) -> why generation settings still fail at scale +Harness (docs/07) -> what you still control; AX as a cost property +Convergence (docs/09) -> how you measure intent vs. execution deterministically ``` Read dual-mode first, then this doc when someone proposes "just tell it not to @@ -156,7 +157,8 @@ scope probabilistic work; otito verifies deterministic facts about what actually - otito source referenced above: `src/lib/context-engine.js`, `src/lib/converge.js`, `src/lib/pass-local.js`, `src/lib/code-map/generate.js`, `src/lib/review.js` -- Companions: [Dual-Mode Thesis](../12-dual-mode-thesis/README.md), +- Companions: [Trust Harness Thesis](../14-trust-harness-thesis/README.md), + [Dual-Mode Thesis](../12-dual-mode-thesis/README.md), [Determinism Thesis](../11-determinism-thesis/README.md), [Harness Thesis](../07-harness-thesis/README.md), [Convergence Thesis](../09-convergence-thesis/README.md) diff --git a/docs/14-trust-harness-thesis/README.md b/docs/14-trust-harness-thesis/README.md new file mode 100644 index 0000000..ec8d478 --- /dev/null +++ b/docs/14-trust-harness-thesis/README.md @@ -0,0 +1,181 @@ +# Trust Harness Thesis & the Commodity Loop + +> _Why the generic model harness is becoming infrastructure, and why independent +> merge evidence is the durable product._ + +This document maps a now-public shift in agent engineering onto otito's existing +surface, then turns it into a concrete, prioritised roadmap. It mirrors +[the harness thesis](../07-harness-thesis/README.md), +[the dual-mode thesis](../12-dual-mode-thesis/README.md), and +[the determinism thesis](../11-determinism-thesis/README.md): take the sources, +name what otito has quietly already built, and let the naming sharpen the product. + +The sources are OpenAI's Agents SDK evolution +([post](https://openai.com/index/the-next-evolution-of-the-agents-sdk/)), +OpenAI's harness-engineering write-up from its agent-first experiment +([post](https://openai.com/index/harness-engineering/)), and Anthropic's +agent-eval guidance +([post](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents)). +The shared argument is not "harnesses no longer matter." It is that **two +different harnesses are being collapsed into one word**, and only one of them +is becoming a commodity. + +Frontier models are already good at planning, searching repositories, using +tools, and recovering from mistakes. That makes the basic agent loop +(prompting, retries, tool routing, file editing) less differentiated and +increasingly built into products and SDKs. OpenAI is packaging a model-native +harness with file tools and sandbox execution. Anthropic treats an evaluated +"agent" as the **combination of the model and its harness**, not the model +alone. Those are signals that the generic loop is becoming infrastructure. + +What remains strategically important is the **trust harness**: independent +evidence that a generated change is safe to merge. + +## The thesis in one line + +> Models generate the change. Otito proves whether the change is safe to merge. +> The generic model harness is becoming a commodity. The independent trust and +> governance harness is becoming more important. + +otito **owns the second category** and integrates with Codex, Claude Code, +Gemini, Cursor, and future native harnesses rather than competing with them. + +## What lines up: otito is already the trust harness + +| Source claim | otito's existing answer | +| --- | --- | +| Native SDKs absorb prompting, retries, file tools, and sandbox execution | otito does not ship an agent loop. Hosts call `context_pack`, `review_gate`, and `convergence_score` as procedures. | +| Constraints, repository knowledge, tools, tests, and feedback loops are prerequisites for scaling | Context packs, risk paths, validation plans, receipts, and the operating loop are the shipped form of those prerequisites. | +| An evaluated "agent" is model plus harness | Otito is the **independent** half of that pair: deterministic facts a host cannot grade for itself. | +| Stronger models increase unsupervised change volume | Merge gates, CODEOWNERS, branch-protection checks, and human decision remain separate authorities. | +| Graders should verify outcomes in the environment, not the transcript | `review_gate` and validation attestation run against the **exact staged tree**, not the chat. | +| Code-based graders are objective, reproducible, and easy to debug | Gates, convergence receipts, and policy profiles are recomputable from repo state. | + +The useful observation: the original harness thesis is still true as a **cost +property** (a tighter repo lets a cheaper model do more). It is no longer the +**lead claim**. Token savings narrow as models get better. Independent merge +evidence does not. + +## Four layers, mapped to otito + +| Layer | Trajectory | Otito's job | +| --- | --- | --- | +| Context generation | Still useful, easier for model vendors to absorb | Keep shipping `context_pack` and code maps as grounding, not as the product identity | +| Generic orchestration | Rapidly commoditising | Do not compete with native loops, sandboxes, or file tools | +| Deterministic merge evidence | Durable, valuable core | Own `review_gate`, validation receipts, convergence, CODEOWNERS, and policy | +| Organisation-level governance | Strongest commercial opportunity | Bind those receipts to owners, required checks, and a human merge decision | + +The workflow already encodes this split: + +```text +Request -> context -> scoped change -> exact validation -> review evidence -> human decision +``` + +Context is still in the loop. It is no longer the claim that differentiates +otito from a native harness. + +## Six lessons, mapped to otito + +### 1. Split the word "harness" (positioning) + +[The harness thesis](../07-harness-thesis/README.md) taught "you control the +harness, not the model." This doc teaches **which harness**. The generic loop +is what vendors will ship. The trust harness is what organisations still have +to own: current repository context, permission boundaries, exact validation, +risk-sensitive policy, CODEOWNERS, receipts, and independent evidence. + +Lead with that split. Do not lead with "a better harness makes the model +smarter or cheaper." + +### 2. Integrate with native harnesses instead of replacing them + +Codex, Claude Code, Gemini, Cursor, and future model-native scaffolds will +keep getting better at editing files. otito's MCP and CLI skills already sit +beside those hosts. The product bet is **complementarity**: they generate; otito +attests. + +### 3. Keep verification outside the model loop + +OpenAI's own experiment found constraints, repository knowledge, tools, tests, +and feedback loops were prerequisites for scaling, not optional extras. +Anthropic's eval guidance is the same idea from the measurement side: grade +the **outcome in the environment**, not the model's self-description. + +otito already refuses to let a model award itself a merge. `review_gate`, +`review_verdict`, and `convergence_score` recompute from git facts. + +### 4. Treat stronger models as more governance demand, not less + +A weaker model needed a tighter loop to look competent. A stronger model needs +a tighter **trust** loop because teams will let it run longer, touch more +files, and ask for less supervision. AX scoring still measures cheap-and-safe +changeability. Governance demand scales with capability, not against it. + +### 5. Make organisation-level policy the commercial edge + +Local merge evidence is the durable core. The strongest commercial surface is +what sits on top of it: CODEOWNERS, required checks, risk profiles, workspace +receipts, and a human decision record that survives chat memory. That is +governance a vendor harness cannot honestly provide for a customer's repo +without becoming the customer's control plane. + +### 6. Hold the line against competing with infrastructure (what this is NOT) + +otito is not: + +- a better agent loop than Codex or Claude Code +- a sandbox or file-tool runtime +- a claim that context packs will stay uniquely hard +- a promise that cheaper models remain the main buyer reason + +The differentiated bet: **be the independent proof layer beside whoever +generates the patch.** + +## How the thesis docs fit together + +```text +Trust harness (this doc) -> which harness is durable, and who to integrate with +Dual-mode (docs/12) -> two modes, complementary roles +Prompt determinism (docs/13) -> you cannot collapse modes via prompting +Determinism (docs/11) -> why the probabilistic mode cannot self-verify +Harness (docs/07) -> what you still control; AX as a cost property +Convergence (docs/09) -> how you measure intent vs. execution deterministically +``` + +Read this doc first when the question is "do harnesses still matter?" Read +dual-mode next for the generation/verification split. Read harness and +convergence when designing AX scores and gates. + +## Priorities + +| Priority | Work | Why first | Effort | +| --- | --- | --- | --- | +| **P0** | Trust-harness positioning (lessons 1, 2, 6) | Stops otito competing with vendor loops in its own README | Low: this doc plus README | +| **P0** | Keep merge evidence load-bearing in CI | Makes "prove the change" the default path, not a docs claim | Medium: ties to docs/09 | +| **P1** | Host integration snippets for native harnesses | Codex, Claude Code, Gemini, Cursor, Zed should all call the same procedures | Low | +| **P1** | Organisation-level governance packet | CODEOWNERS, required checks, policy profiles, and decision records as the commercial surface | Medium | +| **P2** | Context as grounding, not identity | Keep `context_pack` excellent without leading the product story with it | Low | + +## What this is NOT + +Not a claim that agent loops are worthless. Not a claim that context generation +is finished. Not a replacement for Codex, Claude Code, Gemini, or Cursor. The +differentiated bet is **independence**: models generate the change; otito +proves whether it is safe to merge. + +## Sources + +- OpenAI, _The next evolution of the Agents SDK_: + +- OpenAI, _Harness engineering_: + +- Anthropic, _Demystifying evals for AI agents_: + +- otito source referenced above: `src/lib/pass-local.js`, `src/lib/review.js`, + `src/lib/converge.js`, `src/lib/policy.js`, `src/lib/codeowners.js`, + `src/lib/context-engine.js`, `codex/skills/otito-review/` +- Companions: [Harness Thesis](../07-harness-thesis/README.md), + [Dual-Mode Thesis](../12-dual-mode-thesis/README.md), + [Determinism Thesis](../11-determinism-thesis/README.md), + [Convergence Thesis](../09-convergence-thesis/README.md), + [Prompt Determinism Thesis](../13-prompt-determinism-thesis/README.md) diff --git a/docs/EXECUTIVE-SUMMARY.md b/docs/EXECUTIVE-SUMMARY.md index 79ca963..5ac97c5 100644 --- a/docs/EXECUTIVE-SUMMARY.md +++ b/docs/EXECUTIVE-SUMMARY.md @@ -1,6 +1,6 @@ # otito - Executive Summary -**Purpose:** Context foundation for AI-assisted software teams
+**Purpose:** Independent trust and merge evidence for AI-assisted software teams
**Maintainer:** Oluwasegun Olumbe
**Version:** See `package.json` in the repository root for the canonical version; `server.json` is kept in sync for the MCP manifest. The surface includes multi-domain discoverability, eval + data-access subcommands, code maps for C#, Python, Java, Ruby, and Rust, and the v1.0 absorption (`impact`, `pass`, `pass-pr`, `review`). @@ -11,10 +11,10 @@ otito answers one practical question: ```text -What should an agent or reviewer know before changing this repository? +Is this generated change safe to merge, and can a human recompute the evidence? ``` -It is a Node.js CLI and MCP server that produces deterministic, local-first repository context for coding agents, maintainers, and reviewers. +It is a Node.js CLI and MCP server that produces deterministic, local-first repository context and merge evidence for coding agents, maintainers, and reviewers. Native agent harnesses generate the change. Otito proves whether it is safe to merge. --- @@ -33,7 +33,7 @@ It is a Node.js CLI and MCP server that produces deterministic, local-first repo | Governance | CI gates, Òtítọ́ readiness, CODEOWNERS, SemVer guidance, security reporting, templates, and review policy | | Demo packet | Company-facing packet that links the executive summary, case study, proof run, launch note, and pilot checklist | | Pilot runbook | Step-by-step first repository and pull request pilot with roles, evidence, stop conditions, and triage | -| Thesis pack | Public explainer pages for harness, tutorials, convergence, usage, determinism, dual-mode, and prompt determinism | +| Thesis pack | Public explainer pages for harness, tutorials, convergence, usage, determinism, dual-mode, prompt determinism, and the trust harness | | Proof index | Sanitized public evidence map plus private/internal evidence boundaries for company reviewers | | Review policy | Branch protection, required checks, CODEOWNERS, conversation-resolution, and admin-decision snapshot | | Feedback loop | Structured company pilot intake that turns reviewer concerns into docs, gates, proof, or roadmap work | @@ -45,15 +45,17 @@ It is a Node.js CLI and MCP server that produces deterministic, local-first repo ## Product Position -otito is part of a larger trust layer: +otito is the independent trust layer beside native agent harnesses: ```text -otito -> context before change -Òtítọ́ -> validation before merge -Humans -> accountability before release +Models generate the change. Otito proves whether it is safe to merge. + +native hosts -> generation (Codex, Claude Code, Gemini, Cursor) +otito -> context before change, evidence before merge +Humans -> accountability before release ``` -This makes otito useful for maintainers who want AI-assisted development without losing the shape of the repository, test expectations, ownership boundaries, and review discipline. +This makes otito useful for maintainers who want AI-assisted development without losing the shape of the repository, test expectations, ownership boundaries, and review discipline. It does not compete with generic orchestration. Deterministic merge evidence is the differentiated core; organisation-level governance is the commercial edge. --- diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md index fa56da4..27a029e 100644 --- a/docs/GLOSSARY.md +++ b/docs/GLOSSARY.md @@ -5,9 +5,11 @@ | AI governance | Rules, checks, review processes, and release practices that keep AI-assisted work accountable | | Catalog | A local index of repositories that otito can search across | | Code map | A JSON-first map of source files, domains, imports, exports, symbols, and routes | +| Commodity loop | The generic agent harness (prompting, retries, tool routing, file editing) now shipping inside model products and SDKs | | Context pack | A task-aware packet of likely files, related files, validation commands, and patterns | | MCP | Model Context Protocol, a standard way for agents to call tools | | PR review context | A diff-aware report with changed files, risk prompts, review targets, and optional GitHub comments | +| Trust harness | Independent context, validation, policy, CODEOWNERS, receipts, and merge evidence that a model cannot award itself | | Òtítọ́ merge gate | The product's merge-safety check for PR readiness before merge | | `otito` | The canonical CLI and MCP server command for this project | | `.otito/` | Local artifact directory for generated reports, indexes, harnesses, and context packets | diff --git a/docs/index.md b/docs/index.md index aa59799..2259543 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,17 +1,17 @@ # :material-source-branch: Òtítọ́ -## Local repo intelligence for agents and reviewers +## Independent trust infrastructure for agents and reviewers **Prepared by:** Oluwasegun Olumbe
-**Status:** v1.0.0 — clean Òtítọ́ product cutover, ready for Bashbop publication
+**Status:** v1.6.2 in this repository; verify npm, GitHub Release, and MCP Registry publication separately
**Category:** Practical AI governance for developers -> A Bashbop Ltd product for teams that want context before code changes, review prompts before merge, and less guessing in agent workflows. +> A Bashbop Ltd product for teams that want any coding agent to produce evidence a human can trust before merge. --- !!! info "About Òtítọ́" - Òtítọ́ is a local-first context system. It inspects repositories, builds code maps, creates task-aware context packs, prepares PR review harnesses, and exposes the same workflow through an MCP server. + Òtítọ́ is a local-first trust harness for coding agents. It maps repository context before a change, then produces deterministic impact, validation, ownership, and review evidence before merge. It complements model-native agent loops instead of replacing them. Its command-line and package identity is `otito`. @@ -21,10 +21,10 @@ ## :material-sparkles: What's New -!!! tip "v1.0.0 — Òtítọ́ clean cutover (2026-07-29)" - - Start from a local checkout while npm publication is pending. - - Run the deterministic CLI with **`otito`**. - - Configure MCP with **`node /path/to/otito/src/cli.js mcp`**. +!!! tip "v1.6.2 in this repository (2026-08-15)" + - Lightweight CLI commands use command-specific module loading. + - Repeatable benchmarks cover version, help, and full context execution. + - Publication remains a separate proof chain across npm, GitHub Release, and MCP Registry. See [CHANGELOG.md](https://github.com/BASHBOP/otito/blob/main/CHANGELOG.md) for the full history. @@ -40,13 +40,14 @@ See [CHANGELOG.md](https://github.com/BASHBOP/otito/blob/main/CHANGELOG.md) for | 04 | [:material-tag-check: Release Readiness](./04-release-readiness/README.md) | SemVer, changelog discipline, CI, and release gates | :material-check-circle: Active | | 05 | [:material-play-circle: Trust-Layer Demo](./05-trust-layer-demo/README.md) | Òtítọ́ as a repeatable review workflow | :material-check-circle: Active | | 06 | [:material-repeat: Builder-Founder Loop](./06-builder-founder-operating-loop/README.md) | Session rhythm, evidence ledger, governance ladder, and next-action rule | :material-check-circle: Active | -| 07 | [Harness Thesis & AX](./07-harness-thesis/README.md) | Why the harness matters more than the model, plus AX scoring | :material-check-circle: Active | +| 07 | [Harness Thesis & AX](./07-harness-thesis/README.md) | The original cost and agent-experience thesis, now subordinate to trust evidence | :material-check-circle: Active | | 08 | [Tutorials Integration](./08-tutorials-integration/README.md) | Codespaces setup and MCP onboarding for tutorials | :material-check-circle: Active | | 09 | [Convergence Thesis](./09-convergence-thesis/README.md) | Intent-vs-diff convergence scoring and receipts | :material-check-circle: Active | | 10 | [Usage Dashboard](./10-usage-dashboard/README.md) | Local usage logging and performance trends | :material-check-circle: Active | | 11 | [Determinism Thesis](./11-determinism-thesis/README.md) | Why model variance is structural and the harness is separate | :material-check-circle: Active | | 12 | [Dual-Mode Thesis](./12-dual-mode-thesis/README.md) | Probabilistic generation beside deterministic verification | :material-check-circle: Active | | 13 | [Prompt Determinism Thesis](./13-prompt-determinism-thesis/README.md) | Why prompt settings do not turn a model into a gate | :material-check-circle: Active | +| 14 | [Trust Harness Thesis](./14-trust-harness-thesis/README.md) | Why independent merge evidence outlasts generic agent orchestration | :material-check-circle: Active | --- diff --git a/mkdocs.yml b/mkdocs.yml index 627d3ab..7629bad 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -106,6 +106,8 @@ nav: - Overview: 12-dual-mode-thesis/README.md - Prompt Determinism Thesis: - Overview: 13-prompt-determinism-thesis/README.md + - Trust Harness Thesis: + - Overview: 14-trust-harness-thesis/README.md - Evaluation Guide: EVALS.md - Glossary: GLOSSARY.md diff --git a/src/lib/ax.js b/src/lib/ax.js index 9943358..354e3ca 100644 --- a/src/lib/ax.js +++ b/src/lib/ax.js @@ -1,8 +1,8 @@ -// Agent Experience (AX) score: a single 0–100 number answering "how cheap and -// safe is it for an agent to make a change here?". It is the measurable form of -// the harness thesis — a better codebase needs fewer tokens and has better -// guardrails, so a cheaper model can do the same work. See -// docs/07-harness-thesis/ax-score-spec.md. +// Agent Experience (AX) score: a single 0-100 number answering "how cheap and +// safe is it for an agent to make a change here?". It remains a cost and +// guardrail diagnostic. The product lead is independent merge evidence, not a +// cheaper model. See docs/07-harness-thesis/ax-score-spec.md and +// docs/14-trust-harness-thesis/README.md. // // AX is a composition layer, not new analysis: Changeability/Containment/Clarity // come from generateImpact (token estimate + blast radius + concepts/risk), and