Skip to content

feat: add hardened scout implementation contract#628

Open
elixlabssolutions wants to merge 3 commits into
kunchenguid:mainfrom
elixlabssolutions:fm/scout-contract-ship-v3
Open

feat: add hardened scout implementation contract#628
elixlabssolutions wants to merge 3 commits into
kunchenguid:mainfrom
elixlabssolutions:fm/scout-contract-ship-v3

Conversation

@elixlabssolutions

Copy link
Copy Markdown

Intent

Implement the Firstmate hardened scout-to-implementation contract as a three-file, instruction-only change. Add one internal skill as the single full owner of proportional snapshot evidence, classified claims, readiness scoring, scope and behavior contracts, tests, delivery choreography, independent implementation packets, hard stops, quality gates, model-fit limits, and claim-keyed feedback. Add only the exact one-line section 13 trigger to AGENTS.md and one executable static contract test using tests/lib.sh. Preserve existing generic scout lifecycle and scratch teardown, do not add runtime enforcement or invent a work-order mechanism, do not touch any fourth tracked path, and preserve captain-only merge authority. The implementation was serialized after overlapping PR 619 merged, rebased onto exact refreshed origin/main fcb9e8a, and must retain exact three-path parity through all pipeline fixes.

What Changed

  • Added the internal scout-to-implementation contract, defining proportional evidence, claim classification, readiness gates, self-contained implementation packets, hard stops, quality checks, and claim-keyed feedback.
  • Registered the contract’s precise agent-only trigger in AGENTS.md for implementation-shaping scout work and subsequent dispatch or promotion.
  • Added an executable static contract test that protects the trigger, report/packet schema, safety boundaries, lifecycle compatibility, and captain-only merge authority.

Risk Assessment

✅ Low: The change is confined to the required three instruction/test paths, preserves captain-only merge authority and existing scout lifecycle language, and adds no runtime enforcement or new work-order mechanism.

Testing

The supplied full-suite baseline had succeeded; I reran the focused static contract test and manually exercised the Markdown instruction surface from AGENTS trigger through the self-contained packet, capturing a CLI transcript because this is an instruction-only change with no rendered UI. All checks passed and the worktree remains clean.

Evidence: End-to-end instruction-surface verification transcript
Firstmate hardened scout-to-implementation contract: end-to-end instruction-surface verification
Target: b5403660fd7a0e2c5d7ea1bab43773745dc59ddb
Required base: fcb9e8a5f2668e77fea86be5013bc3f218a01008
Base ancestry: verified

1. Captain-facing AGENTS trigger (the loader entry point):
767:- `scout-implementation-contract` - load before briefing a scout whose output may scope implementation, and before dispatching or promoting implementation from such a report.

2. Loaded skill ownership and preserved authority boundary:
---
name: scout-implementation-contract
description: >-
  Agent-only contract for implementation-shaping scout output.
  Use before briefing a scout whose output may scope implementation, and before dispatching or promoting implementation from such a report.
  Owns proportional evidence, readiness, report, implementation-packet, safety, quality, and feedback requirements.
user-invocable: false
metadata:
  internal: true
---

# scout-implementation-contract

Use before briefing a scout whose output may scope implementation, and before dispatching or promoting implementation from such a report.
This skill is the single owner of Firstmate's hardened scout report and implementation-packet contract.
It applies only when a scout may scope, sequence, authorize, or directly brief implementation.
Existing generic scouts remain backward compatible, retain their current lifecycle, and remain eligible for scratch-scout teardown.

## Authority and safety boundary

Firstmate owns intake, proportionality-tier selection, report acceptance, implementation dispatch or promotion, and local feedback capture.
The captain's existing merge authority remains unchanged.
A scout report is evidence and a recommendation, not authority for unapproved implementation or external action.
A precise scout does not make a weak model safe for high-risk work.
Model fit must reflect ambiguity, risk, blast radius, and the selected tier rather than report precision alone.
The inputs and outputs are Markdown instruction contracts, and enforcement in this version is a static shell test rather than a runtime parser or new CLI schema.
Readiness outcomes are the only new state transitions, so existing scout lifecycle, promotion, and teardown mechanics remain unchanged.
Instruction-level idempotency and concurrency rely on evidence refresh, exact-head checks, and overlap detection before dispatch.

## Proportionality tiers

Select exactly one tier before gathering evidence, and record why it fits.
The tier controls permitted omissions but never relaxes a hard stop or authority boundary.
An item may be marked N/A only when the selected tier permits it and the report cites evidence proving why it does not apply.


3. Implementation worker receives this self-contained packet surface:
### Packet template

`` `markdown
packet_version: firstmate.scout-implementation.v1
source_scout_id: <stable scout ID>
source_base: <remote and exact SHA>
source_freshness_utc: <UTC timestamp>
tier: <LEAN, STANDARD, or CRITICAL>
verdict_at_handoff: <GO or GO WITH EXPLICIT ASSUMPTIONS>

# Authority and model-fit boundary

State implementation authorization, merge authority, selected model or reviewer fit, and this sentence: A precise scout does not make a weak model safe for high-risk work.

# Objective

State the exact implementation objective.

# Complete requirements

Copy every required behavior and acceptance criterion.

# Exact write scope and reasons

List every approved path with one reason.

# File and symbol or region map

List every action, verified reference pattern, and do-not-change note.

# Behavioral contract

Copy all input and output, state, authority, failure, retry, concurrency, security, observability, compatibility, and migration requirements.

# Dependencies and ordering

State the dependency graph, serialization order, and parallel-safe work.

# Tests with signals

Copy every setup or input, exact command, expected success signal, and expected failure signal.

# Exclusions and non-goals

Copy every forbidden or intentionally unchanged behavior and path.

# Explicit assumptions

List every permitted assumption with its claim ID and invalidation stop.

# Hard stops

Copy every applicable hard stop, including stale evidence, scope widening, invented semantics or authority, invalid assumptions, incomplete tests, unapproved high-risk actions, failed gates, and model mismatch.

# Delivery and done conditions

State work-order applicability, exact base and branch, commit order, local validation, pipeline ownership, exact-head invalidation, PR-ready criteria, merge authority, teardown, and done reporting.

# Feedback owed

Require claim-ID-keyed feedback with status, evidence, impact, and general lesson.
`` `

Do not issue an implementation packet for NO-GO.
Do not use an implementation packet with unresolved template markers.

## Post-implementation feedback

Firstmate records feedback at local `data/<scout-id>/feedback.md` after implementation evidence is available.
Key every feedback row to a stable scout claim ID.
Each row records the claim ID, one status from confirmed or wrong or incomplete, implementation evidence, implementation impact, and a general lesson.
Keep project-specific facts in that local feedback file or the project's existing durable knowledge home.
Shared instructions may receive only generalized lessons through a separate reviewed pull request.

## Quality gate and self-audit

Fail the report or packet quality gate when any required section is missing.
Fail when any unresolved template marker or the literal placeholders `TBD`, `TODO`, or `???` remains in a completed artifact.
Fail when evidence is stale, approved scope intersects an exclusion, or an assumption is unclassified.
Fail when any test command lacks an expected success or expected failure signal.
Fail when the packet depends on another narrative or omits a report requirement needed for implementation.
Fail when any hard gate fails or the selected model or reviewer does not fit the tier.
Fail when the report declares more than one readiness outcome or no readiness outcome.
Accept the report only after the self-audit records every check as passing and the score supports the declared outcome.

4. Change-boundary and contract exercise:
Changed tracked paths:
.agents/skills/scout-implementation-contract/SKILL.md
AGENTS.md
tests/fm-scout-implementation-contract.test.sh
ok - internal owner metadata and the unique precise trigger are protected
ok - tiers, readiness scoring, and classified claim ledgers are protected
ok - snapshot, approved scope, file mapping, and behavioral dimensions are protected
ok - test classes, delivery choreography, hard stops, and safety are protected
ok - report schema, independent packet, quality gate, and feedback are protected
Patch whitespace: clean

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"
  • Baseline already provided as successful: command -v tmux &gt;/dev/null || { echo &#34;tmux is required for e2e tests&#34; &gt;&amp;2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo &#34;== $t ==&#34;; bash &#34;$t&#34; || rc=1; done; exit &#34;$rc&#34;
  • bash tests/fm-scout-implementation-contract.test.sh
  • Manual end-to-end instruction-surface walkthrough recorded in scout-implementation-contract-e2e-transcript.txt: verified base ancestry, exact AGENTS loader trigger, loaded ownership/authority boundary, rendered packet template, exact three changed paths, contract exercise, and patch whitespace.
  • git diff --check fcb9e8a5f2668e77fea86be5013bc3f218a01008 b5403660fd7a0e2c5d7ea1bab43773745dc59ddb
🔧 **Document** - 1 issue found → auto-fixed ✅
  • ⚠️ tests/fm-scout-implementation-contract.test.sh:121 - The required static contract test does not protect the no-runtime-enforcement, captain-only merge, or full lifecycle/scratch-teardown guarantees; expanding it would require a test edit outside the documentation-only scope.

🔧 Fix: Protect scout contract safety boundaries
✅ Re-checked - no issues remain.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant