feat(qa): feature-engagement feedback loop — manifest + coverage scorer + forcing gate (WS0, all-WARN) - #1018
Merged
Merged
Conversation
…er + forcing gate (WS0, all-WARN)
WS0 closes the keystone QA blind spot: today an entire authored story subsystem
(companion approval, camp downtime, faction questlines, the companion agenda,
decisions) can be 100% INERT across a 5-persona sweep and the RRI still scores
10/10 — no gate is engagement-coverage. A frozen-relationship run that narrated
the companion but never moved a gauge passed; a run that seeded factions and never
joined one passed.
WHAT THIS ADDS (all strictly additive; every system ships severity='warn' — ZERO
new fatals, so a currently-green pipeline stays green):
* qa/feature_engagement.py — a pure module: a SystemSpec manifest of the 10 authored
story systems + engagement_coverage(state, tool_counts, session_beats) returning
{coverage, engaged[], na[], inert[{id,why,severity}]}. ENGAGED if the detector is
true; N/A if the precondition is false/unknown; INERT iff the precondition is TRUE
and the detector FALSE (owed-but-dead — the signal). PURE-READ over engine-mutated
snapshot state (attitude_value, last_long_rest_day, faction.joined/standing,
narrative_arc.act, consequence.fired/trigger_day, the arc/agenda fired flags,
campaign.decisions/quests/factions/*_arcs) or DM tool-counts — never fiction.
Reuses story_readout.structural_coverage_from_state / felt_shape_from_state so the
shared buckets never drift; old snapshots round-trip (null-guards throughout).
* servers/engine/tests/test_feature_engagement_manifest.py — the forcing meta-test
(mirrors test_tool_schema_budget.py): asserts {s.id for s in SYSTEMS} ==
REVIEWED_SYSTEM_IDS, every severity in {fatal,warn}, ids unique+snake_case,
precondition/detector callable — so changing the manifest is a visible, reviewed diff.
* qa/test_feature_engagement.py — table-driven behavior tests: INERT-in-scope (the
keystone), COMPLETE→engaged, and the conditional N/A logic (short run, solo party,
factionless world, session_beats=None, combat-sprint), consequences due-vs-future
(strict < final day), narrative_arc absent/None null-guard.
WIRING (additive at every callsite, defensive import → degrade-to-no-op):
* qa/assert_behavioral.py — emits one engagement_<id> WARN per inert system after
structural_completeness. All-WARN ⇒ adds ZERO fatals.
* qa/inject_structural_coverage.py — merges score['engagement_coverage'] beside
structural_coverage (session_beats=None → beats-keyed systems N/A — safe).
* qa/scores_db.py — adds engagement_pct (REAL) + engagement_inert (TEXT); auto-ALTER
backfills NULL on a legacy db.
* qa/release_readiness.py — adds ONE deterministic gate 'story_engagement'. Cross-persona
roll-up: owed-by-≥1 AND engaged-by-none ⇒ inert for the sweep; FAILS only on a FATAL
inert system (all-WARN ⇒ always passes when evidence is present). EVIDENCE-GAP SKIP
when no persona block carries engagement_coverage, so a legacy corpus's RRI is
byte-identical (mirrors the latency-gate skip). Prints an ENGAGEMENT section naming
each inert system + a fix hint.
* qa/SCORING.md — documents the axis, the forcing meta-test, the gate, and the
WARN-first→FATAL graduation discipline (graduation is a FUTURE post-sweep PR).
* .github/workflows/ci.yml — adds test_feature_engagement.py to the qa CI allowlist.
DEFERRED (by design): FATAL graduation is a future post-sweep PR after one real sweep
calibrates the inert/owed classification. Two systems (faction_arc, companion_quest_arc)
are BLOCKED and stay WARN regardless — a snapshot-only precondition can't tell
seeded-but-locked from never-seeded (a known open spike).
Tests: 299 targeted (qa engagement + structural + RRI + scores_db + behavioral) +
3000 engine tests green, single-process (-p no:xdist). Two pre-existing exact
skipped_gates assertions updated to admit the additive story_engagement evidence-gap
skip; gates_total/RRI invariants (11 gates, 10.0) unchanged.
|
Warning Review limit reached
More reviews will be available in 43 minutes and 52 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughIntroduces a WS0 "dead-system tracker" ( ChangesWS0 Feature Engagement Coverage System
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
qa/inject_structural_coverage.py (1)
106-125: 💤 Low valueDuplicate snapshot/transcript loading logic in
compute_engagement().This function duplicates lines 91-103 from
compute(). While acceptable for clarity and defensive isolation, a small refactor could extract the shared snapshot+transcript resolution into a helper to reduce duplication.Not blocking since both functions are small and the duplication is intentional for defensive isolation (a bug in one doesn't affect the other).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@qa/inject_structural_coverage.py` around lines 106 - 125, The compute_engagement() function contains duplicate snapshot and transcript loading logic that is also present in the compute() function (lines 91-103). Extract the shared snapshot resolution and transcript path handling logic into a separate helper function (such as _load_snapshot_and_transcript() or _resolve_snapshot_and_transcript()) that both compute() and compute_engagement() can call. This reduces code duplication while maintaining the defensive isolation between the two functions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@qa/scores_db.py`:
- Around line 423-424: The schema columns engagement_pct and engagement_inert
are defined but the write path does not populate them. Locate where add_run() is
called in the code that processes score.json (likely in
inject_structural_coverage.py or the function that calls it), and extract the
engagement_coverage data from the scores. Calculate engagement_pct as the ratio
of engaged systems to total systems (engaged divided by engaged plus inert),
extract the inert system IDs and create engagement_inert as a comma-separated
string of those IDs, then pass both computed values as parameters to the
add_run() function call.
In `@qa/test_feature_engagement.py`:
- Around line 130-131: The variable `eng` in the coverage parsing statement is
assigned from unpacking block["coverage"].split("/") but never used, which
triggers the RUF059 lint warning. Replace `eng` with an underscore (_) to
indicate the variable is intentionally discarded and not needed for the
subsequent assertion that only uses the `exp` value.
---
Nitpick comments:
In `@qa/inject_structural_coverage.py`:
- Around line 106-125: The compute_engagement() function contains duplicate
snapshot and transcript loading logic that is also present in the compute()
function (lines 91-103). Extract the shared snapshot resolution and transcript
path handling logic into a separate helper function (such as
_load_snapshot_and_transcript() or _resolve_snapshot_and_transcript()) that both
compute() and compute_engagement() can call. This reduces code duplication while
maintaining the defensive isolation between the two functions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 30de4822-75b1-4cbf-90eb-18e8dedfee15
📒 Files selected for processing (11)
.github/workflows/ci.ymlqa/SCORING.mdqa/assert_behavioral.pyqa/feature_engagement.pyqa/inject_structural_coverage.pyqa/release_readiness.pyqa/scores_db.pyqa/test_deterministic_rri_gate.pyqa/test_feature_engagement.pyqa/test_release_readiness.pyservers/engine/tests/test_feature_engagement_manifest.py
Comment on lines
+423
to
+424
| ("engagement_pct", "Engagement"), | ||
| ("engagement_inert", "Inert systems"), |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check if there's a write path that populates engagement_pct and engagement_inert
rg -n "engagement_pct|engagement_inert" qa/scores_db.py -B 5 -A 5
echo "---"
# Check if add_run or similar writes these fields
ast-grep --pattern 'def add_run($$$)'Repository: electricsheephq/WorldOS
Length of output: 9137
🏁 Script executed:
# Search for calls to add_run with engagement fields
rg -n "add_run\s*\(" qa/ -A 20 | grep -E "(add_run|engagement_pct|engagement_inert)" | head -50Repository: electricsheephq/WorldOS
Length of output: 5774
🏁 Script executed:
# Check if feature_engagement module exists and computes these values
find . -name "*feature_engagement*" -type fRepository: electricsheephq/WorldOS
Length of output: 185
🏁 Script executed:
# Search for any assignment to engagement_pct or engagement_inert
rg -n "engagement_pct\s*=" --type py
rg -n "engagement_inert\s*=" --type pyRepository: electricsheephq/WorldOS
Length of output: 305
🏁 Script executed:
# Check feature_engagement.py for engagement_coverage function
head -100 qa/feature_engagement.pyRepository: electricsheephq/WorldOS
Length of output: 4815
🏁 Script executed:
# Search for calls to engagement_coverage function
rg -n "engagement_coverage" --type py -B 2 -A 2Repository: electricsheephq/WorldOS
Length of output: 19906
🏁 Script executed:
# Check if engagement_coverage is being passed to add_run anywhere
rg -n "engagement_coverage.*add_run|add_run.*engagement" --type py -B 3 -A 3Repository: electricsheephq/WorldOS
Length of output: 49
🏁 Script executed:
# Check if engagement_pct/engagement_inert are read from scores_db anywhere
rg -n "engagement_pct|engagement_inert" --type py -B 2 -A 2 | grep -v "scores_db.py"Repository: electricsheephq/WorldOS
Length of output: 2243
🏁 Script executed:
# Check if there's any backfill/migration that might populate these fields
rg -n "backfill|migrate" qa/ --type py | head -20Repository: electricsheephq/WorldOS
Length of output: 629
🏁 Script executed:
# Look at how engagement_coverage is actually used in release_readiness
sed -n '1144,1180p' qa/release_readiness.pyRepository: electricsheephq/WorldOS
Length of output: 2523
The schema columns engagement_pct and engagement_inert are defined but not actually populated by the write path.
The engagement_coverage block is computed by qa/feature_engagement.engagement_coverage() and stored in score.json by inject_structural_coverage.py, but the data is never transformed and written to the database via add_run(). The values would need to be extracted as engagement_pct = len(engaged)/(len(engaged)+len(inert)) and engagement_inert = comma-joined inert system ids, then passed to add_run(). Currently, no code performs this transformation or population step.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@qa/scores_db.py` around lines 423 - 424, The schema columns engagement_pct
and engagement_inert are defined but the write path does not populate them.
Locate where add_run() is called in the code that processes score.json (likely
in inject_structural_coverage.py or the function that calls it), and extract the
engagement_coverage data from the scores. Calculate engagement_pct as the ratio
of engaged systems to total systems (engaged divided by engaged plus inert),
extract the inert system IDs and create engagement_inert as a comma-separated
string of those IDs, then pass both computed values as parameters to the
add_run() function call.
Comment on lines
+130
to
+131
| eng, exp = block["coverage"].split("/") | ||
| assert int(exp) == len(block["engaged"]) + len(block["inert"]) |
There was a problem hiding this comment.
Remove the unused unpacked variable in coverage parsing.
On Line 130, eng is assigned but never used (RUF059), which can trip lint-strict CI.
Suggested patch
- eng, exp = block["coverage"].split("/")
+ _eng, exp = block["coverage"].split("/")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| eng, exp = block["coverage"].split("/") | |
| assert int(exp) == len(block["engaged"]) + len(block["inert"]) | |
| _eng, exp = block["coverage"].split("/") | |
| assert int(exp) == len(block["engaged"]) + len(block["inert"]) |
🧰 Tools
🪛 Ruff (0.15.17)
[warning] 130-130: Unpacked variable eng is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@qa/test_feature_engagement.py` around lines 130 - 131, The variable `eng` in
the coverage parsing statement is assigned from unpacking
block["coverage"].split("/") but never used, which triggers the RUF059 lint
warning. Replace `eng` with an underscore (_) to indicate the variable is
intentionally discarded and not needed for the subsequent assertion that only
uses the `exp` value.
Source: Linters/SAST tools
… thread filter Adversarial pre-merge review found 2 verified one-line majors (files already in the diff): 1. REVIEWED_SYSTEM_IDS was `frozenset(s.id for s in SYSTEMS)` — DERIVED from SYSTEMS, so the forcing meta-test compared SYSTEMS against itself, a tautology that could never fail, defeating the manifest-drift guard the keystone exists for. Now an INDEPENDENT hardcoded literal of the 10 ids; editing SYSTEMS without updating it is a real failure. (Stronger "auto-detect a brand-new untracked player-facing tool" forcing is a noted follow-up — needs a tool->system classification.) 2. consequences_fired false-INERTed on re-armed worldsim standing-thread consequences (worldsim.tick rolls trigger_day forward in place, fired=False forever) — _owed_consequences now excludes thread_id-tagged ones, matching the engine's own due/overdue contract (consequences.due `not c.thread_id`; scene_debt thread skip). +regression test. Nits: guard the inject callsite in try/except (mirror assert_behavioral); tighten the "byte-identical RRI" doc to "RRI math byte-identical; rri.json gains additive engagement_* keys". Pre-graduation calibration (factions joinable, 'failed' stage status, acts has_arc, session_beats from transcript) deferred to the WARN->FATAL graduation PR per the review. 114 WS0 tests green.
100yenadmin
added a commit
that referenced
this pull request
Jun 19, 2026
…mparable across rulers (#1034) The 2026-06 cycle materially tightened the scoring ruler (feature-engagement coverage scorer #1018, acts felt-shape #1001/#1002, betrayal un-inversion #999, romance gate #997, dm_advanced_time unmask #1024, gate-severity accuracy #1030), so a run scores LOWER under sc_d4b93982763a/lc_d7fcfddd5bf7 than under the v1.0.4 rulers — BY DESIGN (the scorer is a tightening feedback loop). Document the ruler-version mechanism + history in SCORING.md §0 and annotate it in the v1.0.5-rc1 CHANGELOG, so current numbers are never mis-compared to historic ones (every scores_db row is fenced by scoring_config_version/lens_config_version). Stable-checkpoint hygiene per owner. Co-authored-by: Eva <arncalso@gmail.com>
100yenadmin
pushed a commit
that referenced
this pull request
Jul 2, 2026
Add 5 progression/closure obligations to _compute_beat_obligations (the SOLE every-beat cue seam ridden by persist_beat + scene_context.durable), keeping the HARD mechanical loop from quietly stalling — the relationship cues (#1017/#1018/ - party_stuck_one_location (med): 8+ act-local beats, <2 visited locations, no in-place-progression (byte-identical to assert_behavioral's party_traveled exception) -> travel_to / add_location. - combat_left_hanging (med): combat active but no living hostile (mirrors end_combat's order-based detection) -> end_combat. Owns the beat over xp_unawarded while combat is active. - xp_unawarded (med): xp-mode, NON-combat, living party member, a defeated monster still carrying xp_value>0 (proactive twin of the xp_not_orphaned FATAL) -> end_combat / award_xp. - clock_dm_frozen (low): substantial beats, day==1 & morning, not in combat, HONEST snapshot proxy; fires only when visited>=2 (party_stuck owns the clock otherwise) -> advance_time / long_rest / downtime. - quest_unresolved_late (med): substantial beats, a quest exists, zero quests completed AND no objective ever recorded done, anti-spam vs quest_resolvable/ quest_stalled -> complete_objective / complete_quest. All CUE-ONLY (Option A — no engine auto-action); pure reads of engine-mutated gauges with defensive getattr (older/partial snapshot degrades a cue to skipped); precedence gates collapse the worst case to ~2-3 cues. ADDITIVE: a fully-progressed snapshot still yields [] (no obligations key). Pin _PARTY_STUCK_BEATS=8 to assert_behavioral's SINGLE_SCENE_MIN_BEATS. Tests (TDD): per-kind FIRE+CLEAR + a fully-progressed-snapshot empty-digest case in test_beat_obligations.py; new deterministic qa/test_ws3a_progression_invariants.py (NO LLM) proves each named verb MOVES its gauge on a real persisted campaign + the cue fires-then-clears, wired into qa/fast_gate.sh. Skill docs (SKILL.md step-6b + AGENT.md closure obligations) name the 5 cues. fast_gate: GREEN (253 passed). focused suite: 83 passed.
100yenadmin
added a commit
that referenced
this pull request
Jul 2, 2026
…#1160) * feat(engine): WS3a — DM-unavoidable per-beat progression/closure cues Add 5 progression/closure obligations to _compute_beat_obligations (the SOLE every-beat cue seam ridden by persist_beat + scene_context.durable), keeping the HARD mechanical loop from quietly stalling — the relationship cues (#1017/#1018/ - party_stuck_one_location (med): 8+ act-local beats, <2 visited locations, no in-place-progression (byte-identical to assert_behavioral's party_traveled exception) -> travel_to / add_location. - combat_left_hanging (med): combat active but no living hostile (mirrors end_combat's order-based detection) -> end_combat. Owns the beat over xp_unawarded while combat is active. - xp_unawarded (med): xp-mode, NON-combat, living party member, a defeated monster still carrying xp_value>0 (proactive twin of the xp_not_orphaned FATAL) -> end_combat / award_xp. - clock_dm_frozen (low): substantial beats, day==1 & morning, not in combat, HONEST snapshot proxy; fires only when visited>=2 (party_stuck owns the clock otherwise) -> advance_time / long_rest / downtime. - quest_unresolved_late (med): substantial beats, a quest exists, zero quests completed AND no objective ever recorded done, anti-spam vs quest_resolvable/ quest_stalled -> complete_objective / complete_quest. All CUE-ONLY (Option A — no engine auto-action); pure reads of engine-mutated gauges with defensive getattr (older/partial snapshot degrades a cue to skipped); precedence gates collapse the worst case to ~2-3 cues. ADDITIVE: a fully-progressed snapshot still yields [] (no obligations key). Pin _PARTY_STUCK_BEATS=8 to assert_behavioral's SINGLE_SCENE_MIN_BEATS. Tests (TDD): per-kind FIRE+CLEAR + a fully-progressed-snapshot empty-digest case in test_beat_obligations.py; new deterministic qa/test_ws3a_progression_invariants.py (NO LLM) proves each named verb MOVES its gauge on a real persisted campaign + the cue fires-then-clears, wired into qa/fast_gate.sh. Skill docs (SKILL.md step-6b + AGENT.md closure obligations) name the 5 cues. fast_gate: GREEN (253 passed). focused suite: 83 passed. * fix(ws3a): xp_unawarded cue points only at the ledger-consuming path (end_combat) — bare award_xp invites duplicate awards (CodeRabbit Major, verified vs _award_kill_xp idempotent-zero at server.py:6919); tighten AGENT.md quest-stall wording * docs+lint(ws3a): SKILL.md quest_unresolved_late guard precision (CodeRabbit); drop unused imports (code-quality) * fix(ws3a): xp_unawarded points at award_xp (fires out-of-combat → end_combat inapplicable, evaos P2); fix combat_left_hanging mirror-site citation (evaos P3, 6467→~7009); SKILL.md 6b match; follow-up #1312 for engine reconcile of stranded kill-XP --------- Co-authored-by: Eva <arncalso@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WS0 — the feature-engagement feedback loop (keystone of the sprint)
The blind spot this closes
Today an entire authored story subsystem (companion approval, camp downtime, faction questlines, the companion agenda, recorded decisions) can be 100% inert across a 5-persona sweep and the RRI still scores 10/10 — because no gate is engagement-coverage. A frozen-relationship run that narrated the companion but never moved a gauge passed; a run that seeded factions and never joined one passed; a run that never camped passed. This PR makes those "dead system" shapes a visible, queryable, gated signal.
All-WARN-first design (why this can't break the green pipeline)
Every system ships
severity='warn', so the axis is strictly additive:assert_behavioral.pyemitsengagement_<id>checks withfatal=False→ zero new fatals; every currently-green run stays green.release_readiness.pystory_engagementgate FAILS only on a FATAL inert system → all-WARN ⇒ it always passes when evidence is present (inert systems are reported, not gated).score.jsoncarriesengagement_coverage(a legacy corpus), the gate is an evidence-gap SKIP — excluded frompassed/total, so RRI math is byte-identical (mirrors the latency-gate skip).gates_totalstays 11 / RRI 10.0 on the existing fixtures.It would have caught the approval / camp deaths
The two proven failure shapes — a companion stuck at
attitude_value 0all run, and a multi-day run where nobody everlast_long_rest_day'd — are nowcompanion_approval/camp_downtimeinert-in-scope (precondition true, detector false), surfaced per-run in the behavioral gate and rolled up in theENGAGEMENTRRI section with a fix hint.The conditional N/A logic (keeps the loop from ever false-RED-ing)
A system is N/A (no occasion to engage) — never inert — when its precondition is false or unknown:
session_beatslives in the transcript, not the snapshot, so the signature takes it explicitly and every beats-keyed precondition defaults to N/A when it isNone(the inject callsite passesNone→ safe under-detect, never a false-RED).consequences_firedis owed only when a consequence'strigger_dayis strictly < the final day;trigger_day == final_dayis WARN-not-owed (may fire on the unseen last beat) ⇒ N/A; future-dated ⇒ N/A.WORLDOS_GATE_COMBAT_SPRINT, all FATAL systems are skipped (mirrorsassert_behavioral.py).Deterministic, snapshot-derived — engine state only, never fiction
Every predicate reads only engine-mutated snapshot fields (
attitude_value,last_long_rest_day,faction.joined/standing,narrative_arc.act,consequence.fired/trigger_day, the arc/agendafiredflags,campaign.decisions/quests/factions/*_arcs) or DM tool-counts — never prose (engine invariant #3). It reusesstory_readout.structural_coverage_from_state/felt_shape_from_stateso the shared buckets never drift; old snapshots round-trip (null-guards throughout, incl. absent/Nonenarrative_arc).The forcing meta-test
servers/engine/tests/test_feature_engagement_manifest.pymirrorstest_tool_schema_budget.py: it asserts{s.id for s in SYSTEMS} == REVIEWED_SYSTEM_IDS(and severities/ids), so changing the tracked-system set is a deliberate, visible diff — the manifest can never silently drift out of coverage (the exact failure WS0 exists to prevent).Deferred FATAL graduation
Graduation to FATAL is a FUTURE, post-sweep PR — after one real 5-persona sweep calibrates the inert/owed classification (the same discipline as
flat_arc/caster_has_spellbook). The gate logic already enforces FATAL-inert ⇒ fail (forward-compat test included), so graduation is a one-line manifest change, not a gate rewrite.2 BLOCKED systems (stay WARN regardless)
faction_arc+companion_quest_arc: a snapshot-only precondition can't tell seeded-but-locked from never-seeded beyond thejoinedlatch — a known open spike. They are pinned WARN byBLOCKED_SYSTEM_IDSand the manifest test, and must never graduate until the spike is resolved.The 10 tracked systems
companion_approval · camp_downtime · quests_objectives · acts_advance · consequences_fired · factions_membership · faction_arc* · companion_quest_arc* · companion_agenda · decisions_recorded (*BLOCKED)
Files
qa/feature_engagement.py— pure module (SystemSpec manifest +engagement_coverage).servers/engine/tests/test_feature_engagement_manifest.py— forcing meta-test.qa/test_feature_engagement.py— table-driven behavior tests.qa/assert_behavioral.py— additiveengagement_<id>WARNs afterstructural_completeness.qa/inject_structural_coverage.py— mergeengagement_coverageintoscore.json(beats=None).qa/scores_db.py—engagement_pct(REAL) +engagement_inert(TEXT); auto-ALTER backfills NULL.qa/release_readiness.py— thestory_engagementdeterministic gate +ENGAGEMENTreport section + signals.qa/SCORING.md— documents the axis + WARN-first→FATAL discipline..github/workflows/ci.yml— addstest_feature_engagement.pyto the qa CI allowlist.qa/test_deterministic_rri_gate.py/qa/test_release_readiness.py— admit the additive evidence-gap skip; add WS0 gate behavior tests (absent⇒skip, all-WARN-inert⇒pass+report, cross-persona roll-up, FATAL⇒fail).gates_total/RRI invariants unchanged.Tests (single-process,
-p no:xdist)qa/test_feature_engagement.py+servers/engine/tests/test_feature_engagement_manifest.py— 33 pass.servers/engine/tests/— 3000 pass.Do not merge — for review.
Summary by CodeRabbit
Release Notes
New Features
Tests