Skip to content

feat: semantic-health baseline watermark (v1.38.0)#148

Merged
solaitken merged 6 commits into
mainfrom
feat/health-baseline-watermark
Jul 21, 2026
Merged

feat: semantic-health baseline watermark (v1.38.0)#148
solaitken merged 6 commits into
mainfrom
feat/health-baseline-watermark

Conversation

@solaitken

@solaitken solaitken commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Operators can now acknowledge vault history up to a chosen date so that one-time bulk seeds stop pinning the semantic-health verdict at watch forever. A new health.silence_before watermark in _brain.yaml (managed by o2b brain health-baseline set/get/clear) filters advisory findings whose underlying entries are entirely older than the watermark out of the surfaced report, while detection, storage, and every finding tied to newer entries stay exactly as they are.

Closes #147.

Why this matters

flowchart LR
    A[Bulk seed of preferences] --> B[concept-gap and batch-inflation fire]
    B --> C{Watermark set?}
    C -- no --> D[Verdict pinned at watch forever]
    C -- yes --> E[Findings entirely before baseline suppressed]
    E --> F[Verdict reflects current state]
    E --> G[Report prints suppressed count and baseline date]
    B -.entries after baseline.-> F
Loading

Before this change the only way to clear a permanently firing advisory was deleting or editing real memory. Now acknowledgement is a first-class, explicit, reversible operation that never touches stored data.

Concrete benefits

  • A one-time seed burst no longer keeps o2b brain health at watch; the verdict is computed from the surfaced findings only.
  • Suppression is never silent: the report prints suppressed: N finding(s) older than baseline <date> and carries an additive suppressed object (mirrored snake_case on MCP brain_health).
  • Conservative by construction: an entry with no parseable timestamp counts as newer than any watermark, so the filter can only hide what is provably old; a concept-gap term with even one post-baseline mention surfaces with its full frequency.
  • Byte-identical when unset: no watermark means no new output, fields, or behavior anywhere (CLI text, CLI JSON, MCP), proven by regression tests.
  • Invalid watermark values fail loudly at config load; the CLI setter validates input, exits 2 on usage errors, and emits {ok:false} envelopes on --json operational failures.
  • The _brain.yaml upsert follows the repository convention (file lock plus atomic temp-file rename) and preserves unrelated config content byte-for-byte.
  • Consolidation bonus: four duplicate detector-local ISO timestamp parsers collapsed into one shared iso-time module.

What ships

  • src/core/brain/health/iso-time.ts - shared ISO parsing helpers (single source, previously four copies).
  • src/core/brain/health-baseline.ts - watermark read/write with a pure YAML upsert transform.
  • src/cli/brain/verbs/health-baseline.ts - o2b brain health-baseline set <date>|now / get / clear.
  • src/core/brain/health/reconcile.ts - surfacing-only baseline filter, additive suppressed report field, verdict over surfaced findings.
  • src/core/brain/policy.ts, types.ts - health.silence_before config parsing and validation.
  • src/cli/brain/verbs/health.ts, src/mcp/brain/health-tools.ts - explicit suppression line in text output and additive MCP mirror.
  • Tests beside every change, including exact-boundary cases (equal-instant, straddling burst) and MCP presence/absence of suppressed.

Test plan

  • bun run fmt - no changes
  • bun run lint - exactly 134 warnings, 0 errors (baseline held)
  • bun run typecheck - clean
  • bun test - 6803 pass, 0 fail
  • bun run scripts/sync-version.ts --check - manifests in sync
  • Temp-vault smoke: set 2026-01-01 suppresses a pre-baseline burst and prints the suppressed line; clear restores the full report; unset vault output byte-identical to v1.37.0

Summary by CodeRabbit

  • New Features

    • Added semantic-health baselining via o2b brain health-baseline set|get|clear (watermark stored in _brain.yaml).
    • Health reports now suppress applicable advisory findings older than the configured watermark and include a breakdown of how many were suppressed plus the baseline date.
    • The brain_health MCP tool output now includes suppression details when baselining hides findings.
  • Documentation

    • Updated README, CLI reference, MCP safety notes, and changelog for version 1.38.0.
    • Added help/usage text for the new health-baseline command.
  • Release

    • Bumped package and plugin versions to 1.38.0.

solaitken and others added 4 commits July 21, 2026 15:32
Add an optional acknowledge-before watermark so an operator can declare
"state up to this date is seen and accepted", clearing the permanent
`watch` verdict a bulk-seeded vault gets from time-less `concept-gap` and
`confirmed_at`-anchored `batch-concept-inflation` findings, without
deleting or editing real memory.

- Config: new `health.silence_before` key in `_brain.yaml` (date-only
  `YYYY-MM-DD` or full ISO-8601 timestamp), parsed/validated alongside
  the existing health thresholds. Invalid value throws a BrainConfigError
  (no silent fallback); absent means the feature is off and behavior is
  byte-identical to before.
- CLI: `o2b brain health-baseline set <date>|now | get | clear` records
  the watermark by upserting only the `silence_before` line in
  `_brain.yaml`. Usage errors exit 2 via usageError; --json operational
  failures use the {ok:false,message} envelope, matching state.ts.
- Surfacing filter only in reconcileSemanticHealth: a batch burst is
  hidden when its windowEnd predates the watermark; a concept gap is
  hidden only when every corpus entry mentioning the term is older than
  it (an undated/unparseable entry counts as newer, so the filter can
  only ever hide what is provably old). The verdict folds the surfaced
  findings. Detection and stored files are untouched.
- Transparency: SemanticHealthReport gains an additive
  `suppressed: {conceptGaps, batchInflation, baseline}` present only when
  a watermark hid something; `o2b brain health` prints a
  "suppressed: N finding(s) older than baseline <iso>" line, and the
  brain_health MCP tool passes the field through additively (no new
  tool). When unset, text and JSON output stay byte-identical.
- Extract the duplicated parseIsoUtc/ISO_DATE_ONLY_RE from stale-claim.ts
  and batch-inflation.ts into shared health/iso-time.ts (one source), now
  also used by the config validator and the reconcile filter.

Deviations from the brief:
- Verb named `health-baseline` (sibling to the existing `health` verb)
  with the date as a POSITIONAL argument (`set 2026-01-01`) rather than a
  `--value` flag as in `brain state`; a single required value reads
  cleaner positionally. Subcommands are set|get|clear (no separate
  list, since there is exactly one value).
- SemanticHealthInput threads dates as an optional parallel
  `corpusPrincipleDates` array aligned with `corpusPrinciples` (rather
  than restructuring entries to {text,timestamp}), keeping all existing
  callers and tests byte-identical when the watermark is unset.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add missing watermark boundary tests (windowEnd == watermark, burst
straddling the watermark, concept-gap newest mention == watermark);
dedupe the fourth parseIsoUtc copy in health/thesis.ts onto the shared
health/iso-time.ts source; and write _brain.yaml atomically
(atomicWriteText, temp-file + rename) in writeHealthBaseline instead of
a plain writeFileSync, matching schema-mutate.ts so a crash mid-write
cannot truncate the vault config.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
writeHealthBaseline now takes the same withFileLock + atomicWriteText
lock schema-mutate.ts uses for _brain.yaml, so a concurrent health-baseline
set and another _brain.yaml writer can no longer lose an update. The CLI
verb's set/clear subcommands await the now-async writer. Also adds
explicit MCP coverage for the brain_health suppressed mirror: present
(snake_case) when silence_before hid a finding, absent entirely otherwise.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CHANGELOG 1.38.0 entry with compare link, README what-is-new rotation,
cli-reference section for o2b brain health-baseline and the
health.silence_before watermark, mcp.md note for the additive
brain_health suppressed mirror, package.json bumped to 1.38.0 with
sync-version propagation to all seven manifests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@solaitken
solaitken enabled auto-merge (squash) July 21, 2026 16:04
@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@solaitken, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 25 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a3d2884c-6851-4c34-9526-840e70297997

📥 Commits

Reviewing files that changed from the base of the PR and between d5b9cbc and 88f3c12.

📒 Files selected for processing (2)
  • src/core/brain/health-baseline.ts
  • tests/core/brain/health-baseline.test.ts
📝 Walkthrough

Walkthrough

Adds semantic-health baselining through health.silence_before, CLI-managed _brain.yaml updates, timestamp-aware suppression of historical findings, and conditional suppression details in CLI and MCP health outputs. Documentation, tests, and version metadata are updated for release 1.38.0.

Changes

Semantic health baseline

Layer / File(s) Summary
Health configuration and timestamp contracts
src/core/brain/types.ts, src/core/brain/policy.ts, src/core/brain/health/*, tests/core/brain/health/*
Adds the optional watermark configuration, shared ISO parsing, authored-date tracking, and validation coverage.
Baseline storage and CLI management
src/core/brain/health-baseline.ts, src/cli/brain*, src/cli/brain/verbs/*, tests/cli/brain-health-baseline.test.ts, tests/core/brain/health-baseline.test.ts
Adds locked atomic YAML updates and get, set, and clear operations with human-readable and JSON output.
Semantic-health suppression
src/core/brain/doctor.ts, src/core/brain/health/reconcile.ts, tests/core/brain/health/reconcile.test.ts
Filters fully historical concept-gap and batch-inflation findings, preserves boundary and undated findings, recalculates verdicts, and reports hidden counts.
CLI, MCP, documentation, and release surfaces
src/cli/brain/verbs/health.ts, src/mcp/brain/health-tools.ts, tests/mcp/brain-health.test.ts, README.md, docs/*, CHANGELOG.md, package.json, pyproject.toml, *.plugin.*
Surfaces suppression details, documents the behavior, adds release notes, and bumps version metadata to 1.38.0.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Operator
  participant HealthBaselineCLI
  participant BrainYaml
  participant BrainHealth
  Operator->>HealthBaselineCLI: set baseline
  HealthBaselineCLI->>BrainYaml: locked atomic update
  BrainHealth->>BrainYaml: resolve health.silence_before
  BrainHealth-->>Operator: surface findings and suppressed counts
Loading

Possibly related PRs

Suggested reviewers: itechmeat

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 78.26% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding a semantic-health baseline watermark for v1.38.0.
Linked Issues check ✅ Passed The PR implements the requested optional acknowledge-before watermark and suppresses only findings fully older than it, matching issue #147.
Out of Scope Changes check ✅ Passed The changes shown are all tied to the baseline watermark feature, supporting docs/tests, or the version bump.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/health-baseline-watermark

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/core/brain/health-baseline.ts`:
- Around line 78-109: The applyHealthSilenceBeforeToYaml function must support
CRLF content and preserve the existing health block’s indentation. Normalize or
account for trailing carriage returns when locating health: and its child keys,
and derive the inserted or replaced silence_before indentation from the existing
block’s sibling indentation rather than always using two spaces.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 52306efa-f1c4-4239-b680-0f55e2bafefc

📥 Commits

Reviewing files that changed from the base of the PR and between b0c3797 and 55d5991.

📒 Files selected for processing (34)
  • .claude-plugin/plugin.json
  • .codex-plugin/plugin.json
  • CHANGELOG.md
  • README.md
  • docs/cli-reference.md
  • docs/mcp.md
  • openclaw.plugin.json
  • package.json
  • plugin.yaml
  • plugins/codex/.codex-plugin/plugin.json
  • plugins/hermes/plugin.yaml
  • pyproject.toml
  • src/cli/brain.ts
  • src/cli/brain/help-text.ts
  • src/cli/brain/verbs/health-baseline.ts
  • src/cli/brain/verbs/health.ts
  • src/cli/brain/verbs/index.ts
  • src/core/brain/doctor.ts
  • src/core/brain/health-baseline.ts
  • src/core/brain/health/batch-inflation.ts
  • src/core/brain/health/concept-gap.ts
  • src/core/brain/health/iso-time.ts
  • src/core/brain/health/reconcile.ts
  • src/core/brain/health/stale-claim.ts
  • src/core/brain/health/thesis.ts
  • src/core/brain/policy.ts
  • src/core/brain/types.ts
  • src/mcp/brain/health-tools.ts
  • tests/cli/brain-health-baseline.test.ts
  • tests/core/brain/health-baseline.test.ts
  • tests/core/brain/health/concept-gap.test.ts
  • tests/core/brain/health/reconcile.test.ts
  • tests/core/brain/policy-health.test.ts
  • tests/mcp/brain-health.test.ts

Comment thread src/core/brain/health-baseline.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/core/brain/health-baseline.ts (1)

92-95: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Prevent a leading blank line when writing to an empty file.

If the original file is completely empty (!hadContent), base currently evaluates to "\n" (since "".replace(...) + "\n" yields "\n"). This results in the newly generated YAML unnecessarily starting with a blank line (\nhealth:\n...).

Only append the newline to base if the file originally had content.

✨ Proposed fix
     if (headerIdx < 0) {
-      const base = normalized.replace(/[\r\n]+$/, "") + eol;
+      const base = hadContent ? normalized.replace(/[\r\n]+$/, "") + eol : "";
       const separator = hadContent ? eol : "";
       return `${base}${separator}health:${eol}  silence_before: ${quoted}${eol}`;
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/core/brain/health-baseline.ts` around lines 92 - 95, Update the headerIdx
< 0 branch in the health baseline writer so base appends eol only when
hadContent is true; preserve the separator and generated health YAML structure
while ensuring an originally empty file starts directly with health:.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/core/brain/health-baseline.ts`:
- Around line 140-142: Update the sibling-indentation detection loop around
lines and the `line` variable to ignore comment-only lines before evaluating
indentation or ending the scan. Continue past trimmed lines beginning with the
YAML comment marker, then determine indentation only from the first actual child
key so inserted `silence_before` uses the correct spacing.

---

Nitpick comments:
In `@src/core/brain/health-baseline.ts`:
- Around line 92-95: Update the headerIdx < 0 branch in the health baseline
writer so base appends eol only when hadContent is true; preserve the separator
and generated health YAML structure while ensuring an originally empty file
starts directly with health:.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e16b70d6-a478-401d-8b25-4bf1052a3c67

📥 Commits

Reviewing files that changed from the base of the PR and between 55d5991 and d5b9cbc.

📒 Files selected for processing (2)
  • src/core/brain/health-baseline.ts
  • tests/core/brain/health-baseline.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/core/brain/health-baseline.test.ts

Comment thread src/core/brain/health-baseline.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@solaitken
solaitken merged commit c31a257 into main Jul 21, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Change request: Support baselining for semantic health advisory warnings

2 participants