Skip to content

feat: knowledge intake and consolidation (v1.36.0)#145

Merged
solaitken merged 14 commits into
mainfrom
feat/knowledge-intake-and-consolidation
Jul 19, 2026
Merged

feat: knowledge intake and consolidation (v1.36.0)#145
solaitken merged 14 commits into
mainfrom
feat/knowledge-intake-and-consolidation

Conversation

@solaitken

@solaitken solaitken commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

feat: knowledge intake and consolidation (v1.36.0)

Summary

The vault now captures from the operator's pocket and consolidates what accumulates: an inbound Telegram bot lands phone captures in a staged area, a report-then-apply drain pass classifies and routes each capture, keyed Brave and Tavily providers plus a full-page extract feed the citation-constrained research pipeline, and on the consolidation side facts roll up into higher tiers on pure counters, entities get diarized profiles with a stated-vs-evidenced gap, synthesis findings carry causal context and decomposed confidence behind an evidence-identity gate, the link graph gains a capped idempotent repair lane with efficacy holdouts, and skill proposals pass a deterministic verifier before ever reaching the pending queue.

Why this matters

flowchart LR
    subgraph Intake["Intake: content reaches the vault"]
        TG["Telegram bot<br/>(allowlist, /catchup)"] -->|"capture contract"| ST["Staged captures"]
        ST -->|"classify + route<br/>report then apply"| RT["source page / note / obligation"]
        WEB["Brave / Tavily / page extract<br/>(env-gated, keyed)"] --> RP["Research pipeline<br/>(no uncited claims)"]
    end
    subgraph Consolidation["Consolidation: accumulation becomes insight"]
        F["Facts"] -->|"count trigger"| RL["Rollup ladder<br/>facts to identity tier"]
        E["Entity documents"] --> DZ["Diarized profile<br/>stated vs evidenced"]
        SY["Synthesis findings"] -->|"evidence-identity gate"| OK["Emitted + excluded counted"]
        LG["Link graph"] -->|"dry-run, capped, idempotent"| RE["Repair lane + holdout gate"]
        SP["Skill proposals"] -->|"verifier before pending"| PQ["Versioned, merged, human-approved"]
    end
    Intake --> Consolidation
Loading

Concrete benefits

  • Phone capture closes the biggest input-path gap: a message to the bot becomes a staged vault capture with provenance, and /catchup replays what was missed; nothing runs without the token and an explicit runner.
  • The inbox stops being a graveyard: every staged capture is classified structurally, routed to a source page, note, or obligation, archived idempotently, and named in a per-item report.
  • Research gains real sources without weakening the citation contract: providers join only when their keys are set, a keyless pool stays byte-identical, and page extract feeds full text, never invented content.
  • The identity tier stays small and current: rollups fire on counters, below threshold the dream pass is byte-identical.
  • One command answers "what do we actually know about this entity" with claims separated from evidence, every line carrying a proof identity.
  • Synthesis loss is visible: identity-less findings are excluded and counted, never silently dropped.
  • The link graph can be densified safely: dry-run default, exact confirmation, hard write cap, reruns converge to zero, and the holdout gate fails on dangling or unhydrated targets.
  • Thin skill drafts never reach the human queue; accepted skills evolve version-over-version and same-name collisions merge instead of forking.

What ships

Unit Kanban Artifact
Inbound Telegram capture bot + /catchup (capture contract) t_f8f5ef6a src/core/brain/capture/, o2b brain telegram-capture
Inbox-drain classify-and-route pass t_b0bba8cb src/core/brain/capture/inbox-drain.ts, o2b brain inbox-drain
Brave/Tavily providers + page extract (keyed fetch helper) t_1dcbf352 src/core/brain/research/external-fetch.ts, providers/, extract.ts
Synthesis causal context, decomposed confidence, evidence-identity gate t_40fa4e8d src/core/brain/deep-synthesis.ts
Subject diarization with stated-vs-evidenced gap t_28ba3fc4 src/core/brain/diarization.ts, o2b brain diarize, MCP brain_diarize
Count-triggered fact rollup ladder t_c5263e27 src/core/brain/rollup-ladder.ts, dream synthesize phase
Memory-graph repair lane + efficacy holdouts t_6832aac6 src/core/brain/link-graph/repair-lane.ts, graph-holdout.ts, o2b brain repair-lane
Skill-proposal verifier gate, versioning, same-name merge t_6fc8663c src/core/brain/skill-proposal-verifier.ts, skill-proposals.ts

MCP surface: 106 -> 107 tools (brain_diarize). Every new surface is opt-in (env, config, or explicit verb) and byte-identical when unused, each with a regression test.

Test plan

  • bun run fmt clean
  • bun run lint exactly 134 warnings / 0 errors (baseline)
  • bun run typecheck clean
  • bun test full foreground suite green (6605+ pass / 0 fail)
  • bun run scripts/sync-version.ts --check passes
  • Phase-4 QA smoke of all eight features through the real CLI in an isolated scratch vault, zero real network calls
  • Self-review of the full branch diff versus main: verdict CLEAN; one spec-alignment follow-up applied (holdout gate fails on unhydrated targets)

Summary by CodeRabbit

  • New Features
    • Added o2b brain telegram-capture for allowlisted Telegram intake with staged capture and /catchup.
    • Added o2b brain inbox-drain for dry-run-first classify-and-route with optional apply.
    • Added deterministic o2b brain diarize with stated-vs-evidenced output.
    • Expanded o2b brain deep-synthesis JSON/MCP reports with causal context, decomposed confidence, and excluded-findings ledger.
    • Added count-triggered rollup ladder (persisted) and guarded o2b brain repair-lane.
    • Added brain_diarize MCP tool and expanded MCP/JSON surfaces.
  • Documentation
    • Updated README, CLI reference, MCP docs, and knowledge-intake design/plan notes for v1.36.0.
  • Chores
    • Bumped version to 1.36.0 across plugin/package manifests.

solaitken and others added 12 commits July 19, 2026 15:40
Phase-0 artifacts for the knowledge intake and consolidation wave:
design doc (consultant Variant 2, two shared seams), implementation
plan (8 tasks in two tracks), and the variant audit trail with the
raw consultant output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-note contract (I1, t_f8f5ef6a)

Carry seam 1 of the knowledge-intake-and-consolidation wave: the
capture-note contract module that owns the inbound staging vocabulary,
plus the long-poll Telegram capture bot that writes only through it.

Seam 1 (src/core/brain/capture/capture-note.ts): frontmatter kind
(brain-capture), provenance (source, sender, capture timestamp),
staging and archive path helpers, and read/write/list/archive
functions. The layout mirrors the inbox-versus-processed distinction in
governance/forget-plan.ts but lives in its own Brain/captures subtree so
the signal inbox and the dream pass stay byte-identical for vaults that
never capture.

Bot (src/core/brain/capture/telegram-capture.ts): an explicit CLI runner
verb (o2b brain telegram-capture run|catchup) long-polling getUpdates via
fetch with no new dependencies and no webhook framework. It reuses the
MarkdownV2 escaping from src/core/discipline/telegram.ts and a new bot
token config key. A chat-id allowlist from config gates every update;
each accepted text becomes one capture note through the contract; every
rejected or malformed update is one explicit logged decision in a JSONL
ledger; /catchup replies with captures since the last acknowledged one;
a missing token is a typed error at startup. Nothing runs from hooks.

The update-handling core is unit-tested with an injected transport (no
real network); the runner verb is tested for the missing-token error and
the disk-only catchup path.

Deviations from plan.md (recorded per the codebase-over-plan rule):
- No pre-existing "bot token config" existed (discipline delivery is
  handled externally by Hermes cron), so seam 1 introduces
  resolveTelegramBotToken / resolveTelegramCaptureAllowlist in
  src/core/config.ts following the existing config-resolver conventions;
  the token key is redacted by redactMapping.
- Captures live in Brain/captures (+ /processed) rather than sharing the
  signal inbox directory, to guarantee the byte-identical opt-out for the
  dream and signal machinery.
- The verb lives under "o2b brain" (where the knowledge features live),
  matching the curated command-manifest subset that omits recent verbs.
- Rejected/malformed decisions are logged to a dedicated
  Brain/log/capture-decisions.jsonl sidecar instead of the validated
  brain-log event-kind enum, keeping the opt-out byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s (I2, t_b0bba8cb)

Add the report-then-apply pass that drains staged captures written by the
I1 Telegram bot. It walks the staging area through the seam-1 capture-note
contract and classifies each capture from STRUCTURAL signals only, never a
natural-language word list:

  - source-reference: a url-shaped body (a single http(s):// token) ->
    routed through ingestSource (src/core/brain/ingest/ingest.ts).
  - obligation: a leading @obligation marker, optionally carrying a cadence
    from the obligation vocabulary -> routed through the existing obligation
    machinery (addObligation), with a named-constant default cadence.
  - idea: everything else is an atomic idea -> create-or-merge note under
    captured/, merging into an existing same-slug note instead of forking.

Dry-run is the default and writes nothing (regression-tested); --apply
executes the route and archives each processed capture through the
contract, so a rerun finds no staged captures and is a no-op (the
processed marker is the idempotency key, regression-tested). Each item is
reported with its action and reason; unroutable items (an obligation
marker without a title, a same-title collision) are reported and left in
place. Adds obligationExists to obligations.ts for the structural
collision check.

Deviations from plan.md (recorded per the codebase-over-plan rule):
- "explicit frontmatter" source detection reduces to url-shape here because
  every capture carries the same brain-capture kind; url-shape and the
  obligation marker are the active structural signals.
- Idea notes land in a vault-relative captured/ directory via the
  create-note write envelope, which refuses the Brain machinery root; the
  merge path appends to the existing note.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tity gate (S1, t_40fa4e8d)

Give every deep-synthesis finding an additive causal-context field and a
decomposed, deterministic confidence, and gate emission on a shared
evidence-identity type so identity-less findings are a visible loss
rather than a silent drop.

deep-synthesis.ts and its types:
  - Export EvidenceIdentity (path + kind + content hash) and the
    hasEvidenceIdentity predicate. Identity is tied to retrievable
    content: the hash is a sha256 of the evidence bytes. S2 (diarization)
    consumes this shared type for stated-vs-evidenced lines.
  - Each matched, readable note becomes one SynthesisFinding carrying its
    evidence identity, a SynthesisCausalContext (typed relations,
    superseded_by pointer, dangling-citation count), and a
    SynthesisConfidence decomposed into support (incident positive
    relations), opposition (incident contradictions), freshness (recency
    in [0,1] from note age against the stale-age horizon), and coverage
    (resolvable citations). Every component is computed from data already
    flowing through the pass; no model call, no language analysis.
  - The read-failure path that used to `continue` silently now records an
    excludedFindings entry with a structural reason; the report carries
    excludedFindingCount so the loss is always visible.

Additive only: the prior report fields, the `checked` dimensions, and the
steelman seed selection (buildStrongestObjection) are untouched, so
existing consumers and the CLI/MCP surfaces stay byte-identical.
Regression-tested on the prior output shape and the objection ordering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, t_28ba3fc4)

Add subject diarization: given a registry entity, assemble its document
set and emit a profile skeleton plus one needs-llm-step envelope for the
prose, with a deterministic stated-vs-evidenced section. No model is ever
called inline.

Core (src/core/brain/diarization.ts):
  - Resolves the entity through the registry; an unknown entity is a typed
    DiarizationError.
  - Document set = the entity registry file plus every ingested source
    page the subject appears in (anchored through the existing atomic-fact
    machinery, not a word list).
  - STATED claims are atomic assertions decomposed from the entity's own
    registered body that anchor the subject. EVIDENCED signals are the
    frequency and recency with which the subject appears across ingested
    sources - pure counts and timestamps, never language analysis of the
    claim text. Each gap line is classified stated_corroborated,
    stated_unevidenced, or evidenced_unstated.
  - Every line carries the shared S1 evidence-identity type
    (EvidenceIdentity from deep-synthesis) and is gated on it; an
    identity-less line is reported in excludedLineCount, never dropped
    silently.
  - Emits the profile skeleton (frontmatter + sections + a prose marker)
    and exactly one needs-llm-step envelope mirroring the write-session
    grammar (status, step, prompt, schema hints, target path) as plain
    read-only data.

Surfaces: new CLI verb `o2b brain diarize <entity>` and, per the
deep-synthesis precedent for a read analysis, an MCP twin brain_diarize
(full tier, read-only). Registry baselines updated (parity frozen list,
mcp.test.ts and removed-tools.ts tool counts 106 -> 107); tool
description within the 300-char cap.

Deviations from plan.md (recorded per the codebase-over-plan rule):
- Profile notes target Brain/profiles/<entity-id>.md; diarization emits
  the skeleton and envelope only and never writes, so the write happens
  through the caller's own submit path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…se (S3, t_c5263e27)

Add a deterministic, count-only rollup ladder to the dream synthesize
phase. It never reads a fact's content - only how many facts exist at
each rung since that rung last rolled up. When the count of new facts at
a rung reaches its threshold, the ladder emits one needs-llm-step rollup
envelope (the LLM writes the summary note; core never does) and records
the counter reset in the dream report.

rollup-ladder.ts (new): pure planRollupLadder over a persisted per-tier
ledger. The rungs compose - a fired fact rollup increments the rollup
rung's source count, so enough fact rollups cascade into one identity
rollup in the same pass. The top rung is `identity`, literally the
highest existing frontmatter tier weight. Named-constant thresholds
(DEFAULT_FACT_ROLLUP_THRESHOLD, DEFAULT_ROLLUP_IDENTITY_THRESHOLD),
config-overridable through a new optional `rollup:` block in _brain.yaml
(resolveRollupThresholds). Ledger read/write helpers plus the
Brain/rollup-ladder.json path.

dream.ts: the ladder runs before the `changed` gate over the current
fact count (preference artifacts); a fired rung counts as a state change.
The DreamRunSummary gains an additive `rollups` field, and the dream
summary log event gains a `rollups` key - both only when a rung fires.
The ledger is persisted only on a firing, non-dry-run pass, so a
below-threshold run writes no ledger, adds no report or log entry, and
stays byte-identical (regression-tested). Triggers are pure counters,
never language analysis.

Config plumbing: BrainRollupConfig type, the optional `rollup:` block in
the loadBrainConfig parser (hard-error on a mistyped threshold), and the
resolve helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…R1, t_1dcbf352)

Carry seam 2 of the knowledge-intake-and-consolidation wave: the keyed
external-fetch helper, the Brave and Tavily providers, a full-page extract
step, and the provider-pool wiring in research.ts. Every test mocks at the
fetch boundary; no test path reaches the network.

Seam 2 (src/core/brain/research/external-fetch.ts): env-gated HTTP with an
injected transport, Bearer auth by default (or a named header), typed
ExternalFetchError kinds (disabled, network, auth, http, payload), and a
shared response cache keyed by the normalized request. The cache key is
built only from method, url, sorted query, and body, never from the key, so
no credential can leak into a cache key. Composed error messages are run
through the shared redactor with the key as a literal, so an upstream error
string embedding the key is scrubbed before it surfaces.

Providers (research/providers/brave.ts, tavily.ts): each maps its response
verbatim into ProviderSearchResult, inventing nothing. A provider joins the
pool only when its key env (BRAVE_API_KEY, TAVILY_API_KEY) is set; a keyless
pool reports itself empty explicitly (isEmpty) and leaves writeResearchReport
untouched, so a vault that never configures a key behaves byte-identically to
before this feature (regression-tested). runResearchPool aggregates results
and carries provider failures as typed errors, never as content.

Extract step (research/extract.ts): fetches full page text through the helper
and reduces it mechanically to bounded plain text (no natural-language word
list). findingFromExtract hands a verbatim prefix of that text to the
existing citation-constrained pipeline, citing the fetched URL, so the report
never carries invented content and the citation contract still holds.

Deviations from plan.md (recorded per the codebase-over-plan rule):
- Seam 2 honors "Bearer auth" as the helper default; Brave really uses the
  X-Subscription-Token header, so the helper supports a named-header auth
  scheme alongside Bearer rather than forcing every provider onto Bearer.
- The provider contract (ResearchProvider, ProviderSearchResult) lives in
  research/providers/provider.ts so brave/tavily and research.ts share it
  without an import cycle; research.ts re-exports the key envs and pool API.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…y holdouts (G1, t_6832aac6)

Add the CLI-first memory-graph repair lane and its graph-efficacy holdout
harness. Both are deterministic and read only link structure; there is no
natural-language word list anywhere.

Repair lane (src/core/brain/link-graph/repair-lane.ts): candidate edges are
ordered by identity strength (explicit references, session continuity,
same-topic evidence, then opt-in inferred), with confidence and endpoints
breaking ties. A confidence threshold and a hard per-run write cap are named
constants. Dry-run is the default and writes nothing; apply requires the exact
confirmation phrase (RepairConfirmationError otherwise). The lane never creates
a dangling edge - a candidate whose endpoint does not resolve to a durable note
is skipped - so the holdout gate stays satisfiable. A forward scan seeded from
each note's current wikilinks and grown as the run writes makes a rerun after
apply converge to zero writes (idempotence regression-tested).

collectRepairCandidates draws the default candidates from structural signals
only: explicit textual references (a note names another note's title without
linking it), session continuity (notes co-referenced in one continuity event),
and same-topic evidence (co-occurrence over shared references). Inferred
candidates are not collected here; they are opt-in at the lane.

Holdout harness (src/core/brain/link-graph/graph-holdout.ts): graph-neighbor
holdouts measure graph lift (targets reachable only through the graph) SEPARATELY
from direct recall (targets that are already a direct neighbor). Every target
must resolve to durable memory and hydrate into bounded evidence (capped by a
named constant); a dangling edge fails the gate. Both the resolve-and-hydrate
pass and the dangling-fails case are tested.

CLI: o2b brain repair-lane (dry-run default; --apply --confirm "apply repair";
--include-inferred; --json), wired through the brain verb dispatch and help text.

Deviations from plan.md (recorded per the codebase-over-plan rule):
- The lane takes caller-supplied candidates and a separate collectRepairCandidates
  derives them from vault structure, so the ordering/gating/idempotence core is
  unit-testable in isolation from candidate generation.
- The holdout harness builds a canonical-key adjacency from note wikilinks
  rather than the Store snapshot, so it runs without a built search index; graph
  lift is a 2-hop-not-1-hop reachability measure over that adjacency.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… (K1, t_6fc8663c)

Add a deterministic pre-promotion verifier gate to skill-proposal learning,
plus version tracking and same-name support merging, without changing the
human accept/reject flow (regression-tested).

Verifier (src/core/brain/skill-proposal-verifier.ts): a pure gate that runs
before a draft reaches pending, validating the candidate against its OWN
supporting records - a present pattern key, distinct evidence count, count of
records that structurally support the pattern, and a confidence floor (named
constants). Every check is a fixed structural predicate; there is no natural-
language word list. A rejection records a concise reason naming the failed
checks.

learnSkillProposals integration:
- Each candidate passes the verifier before any write; rejections are appended
  to a JSONL ledger (Brain/skill-proposals/verifier-rejections.jsonl) and
  reported in the new verifierRejected result field. Support is re-derived per
  record (recordSupportsCandidate) reusing the existing action/shape/hour
  derivation, so the gate validates real support, not just a count.
- Same-name identity is the (pattern_kind, key) name_key, independent of the
  specific records. A candidate colliding with an ACCEPTED proposal evolves
  that skill (increments its version and its procedure's version, merges
  support) instead of forking; a collision with a PENDING draft accumulates
  support into it; a collision with a human-REJECTED name stays suppressed.
- New drafts carry name_key and version 1; acceptSkillProposal stamps the
  version onto the accepted proposal and the generated procedure. Reading a
  legacy proposal without a version field defaults to version 1, so existing
  proposals are untouched by the field (byte-identical opt-out).

Deviations from plan.md (recorded per the codebase-over-plan rule):
- The verifier takes a projected VerifierCandidate (evidence as {id,
  supportsPattern}) rather than raw continuity records, so it stays a pure,
  independently-tested predicate while learnSkillProposals supplies the
  structural support derivation it already owns.
- The verifier's minimum evidence tracks the run's effective minSupport, so a
  legitimate minSupport-2 run is not rejected by a fixed floor while the
  default (minSupport 3) still gates thin candidates.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The graph-efficacy holdout gate passed whenever no edge was dangling, so a
target that resolved to durable memory but hydrated into an empty body still
passed. The repair-lane design requires every target to both resolve AND
hydrate into bounded evidence, so a resolved-but-empty target must fail.

Pass the gate only when danglingCount and unhydratedCount are both zero, and
surface unhydratedCount on the gate result so a failed gate names its reason
alongside danglingCount, keeping the per-resolution hydrated/evidenceChars
fields intact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nd exclusions on CLI and MCP surfaces

The S1 finding pass (t_40fa4e8d) added per-finding causal context,
decomposed confidence (support/opposition/freshness/coverage), and the
excludedFindings/excludedFindingCount evidence-loss ledger to the core
DeepSynthesisReport, but neither the `brain deep-synthesis --json` verb
nor the brain_deep_synthesis MCP handler emitted them, so the operator
saw the pre-S1 dossier shape and the evidence loss stayed invisible.

Add a shared synthesisFindingsJson serializer in core so both surfaces
emit byte-identical snake_case findings, excluded_findings, and
excluded_finding_count. Prior fields are unchanged. Refresh the stale
t_04e94382 docstring reference on the verb and note the new fields in
the MCP tool description within the 300-char registry cap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
….36.0

CHANGELOG entry and link reference for 1.36.0, README What-is-new
rewrite with a one-line pointer to the previous release, cli-reference
section for the capture runner, inbox drain, diarize, repair lane, and
synthesis JSON additions, and mcp.md bullets for brain_diarize and the
additive deep-synthesis fields. Version propagated with
scripts/sync-version.ts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@solaitken
solaitken enabled auto-merge (squash) July 19, 2026 18:32
@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9d880768-f33c-4f29-b609-2128a39825ef

📥 Commits

Reviewing files that changed from the base of the PR and between 9ca0257 and f673a46.

📒 Files selected for processing (2)
  • src/core/brain/capture/telegram-capture.ts
  • tests/core/brain/capture/telegram-capture.test.ts

📝 Walkthrough

Walkthrough

Version 1.36.0 adds Telegram capture and inbox draining, keyed web research and extraction, deterministic rollups, diarization, enriched synthesis findings, graph repair, skill verification/evolution, and corresponding CLI, MCP, documentation, and test updates.

Changes

Knowledge intake and consolidation

Layer / File(s) Summary
Release metadata and architecture
.claude-plugin/plugin.json, package.json, docs/brainstorm/knowledge-intake-and-consolidation/*, CHANGELOG.md, README.md
Bumps release metadata to 1.36.0 and documents the intake and consolidation workflows.
Capture staging and inbox routing
src/core/brain/capture/*, src/core/config.ts, src/cli/brain/verbs/*, tests/core/brain/capture/*, tests/cli/brain-*
Adds Telegram polling, catchup watermarks, capture-note persistence, structural inbox classification, dry-run/apply routing, and archival.
Keyed research and extraction
src/core/brain/research/*, tests/core/brain/research/*
Adds typed external fetching, caching, Brave/Tavily provider pooling, bounded page extraction, and citation-constrained findings.
Consolidation outputs
src/core/brain/deep-synthesis.ts, src/core/brain/diarization.ts, src/core/brain/rollup-ladder.ts, src/core/brain/dream.ts, src/mcp/brain/knowledge-tools.ts
Adds evidence-identity-gated findings, deterministic subject profiles, persisted count-triggered rollups, and additive CLI/MCP synthesis fields.
Graph repair lane
src/core/brain/link-graph/*, src/cli/brain/verbs/repair-lane.ts, tests/core/brain/link-graph/*
Adds graph holdout evaluation, ordered candidate collection, dry-run repair, exact confirmation for writes, caps, and idempotency.
Skill verification and evolution
src/core/brain/skill-proposal-verifier.ts, src/core/brain/skill-proposals.ts, tests/core/brain/skill-proposal*.test.ts
Adds deterministic verification, rejection logging, same-name merging, versioning, and accepted-skill evolution.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

Suggested reviewers: itechmeat

Sequence Diagram(s)

sequenceDiagram
  participant Operator
  participant BrainCLI
  participant Telegram
  participant CaptureStore
  participant InboxDrain
  participant BrainVault
  Operator->>BrainCLI: telegram-capture run
  BrainCLI->>Telegram: poll updates
  Telegram-->>BrainCLI: text update
  BrainCLI->>CaptureStore: write staged capture
  Operator->>BrainCLI: inbox-drain
  BrainCLI->>InboxDrain: classify staged captures
  InboxDrain->>BrainVault: route and archive captures
  BrainVault-->>BrainCLI: drain report
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 57.24% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately summarizes the main v1.36.0 knowledge intake and consolidation changes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/knowledge-intake-and-consolidation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

🧹 Nitpick comments (5)
src/core/brain/skill-proposals.ts (1)

573-581: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

Verifier-rejection ledger grows without bound and swallows no write errors.

appendVerifierRejection appends one line per rejection on every run with no rotation/cap, and a failed appendFileSync would surface as an unhandled throw that aborts the whole learnSkillProposals pass. Consider a size/age cap on the ledger and/or isolating the append so a ledger-write failure doesn't fail intake. Low priority given intake volume.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/core/brain/skill-proposals.ts` around lines 573 - 581, Update
appendVerifierRejection to bound the verifier-rejection ledger by applying an
appropriate size or age rotation/cap before appending, and isolate filesystem
failures so append errors are handled without aborting the surrounding
learnSkillProposals pass. Preserve the existing JSONL entry format and vault
path safety.
src/core/brain/link-graph/repair-lane.ts (1)

365-380: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Explicit-reference matching has no safeguard against generic/short titles at fixed high confidence.

mentionsTerm only filters titles shorter than 2 characters (Line 291); any title of length ≥2 (e.g. a common word) is matched at word boundaries across every other note's body and, if matched, always gets EXPLICIT_REFERENCE_CONFIDENCE = 0.9 — well above the 0.5 write threshold and the strongest tier in orderCandidates. A vault with a generically-titled note could generate a large number of high-confidence spurious edges that get auto-applied ahead of the write cap. Consider raising the minimum title length or scaling confidence by title specificity (e.g. length, word count).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/core/brain/link-graph/repair-lane.ts` around lines 365 - 380, Harden the
explicit-reference matching in the repair flow around `mentionsTerm` and the
`IDENTITY_STRENGTH.explicitReference` candidate by rejecting or down-weighting
generic, short titles before assigning `EXPLICIT_REFERENCE_CONFIDENCE`. Ensure
common two-character or otherwise nonspecific titles cannot automatically
produce high-confidence edges above the write threshold, while preserving strong
confidence for sufficiently specific titles.
src/core/brain/capture/telegram-capture.ts (1)

220-256: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Harden the long-poll loop against network hiccups: no fetch timeouts, no retry/backoff.

createFetchTelegramTransport's fetch() calls have no timeout, so a stalled connection can block the poll loop indefinitely. And runTelegramCapture has no try/catch around getUpdates/sendMessage; any thrown error (including an HTTP error from either call) propagates out and terminates the whole long-running bot process, with no retry/backoff for what's meant to be a resilient poller.

♻️ Suggested direction
     async getUpdates(offset: number): Promise<TelegramUpdate[]> {
       const url = `${base}/getUpdates?offset=${offset}&timeout=${LONG_POLL_TIMEOUT_SECONDS}`;
-      const res = await fetch(url);
+      const res = await fetch(url, { signal: AbortSignal.timeout((LONG_POLL_TIMEOUT_SECONDS + 10) * 1000) });

And in the poll loop, wrap getUpdates/sendMessage so a transient failure logs and continues instead of crashing the process:

-    const updates = await opts.transport.getUpdates(offset);
+    let updates: TelegramUpdate[];
+    try {
+      updates = await opts.transport.getUpdates(offset);
+    } catch {
+      cycles += 1;
+      continue; // transient failure - retry next cycle
+    }

Also applies to: 263-289

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/core/brain/capture/telegram-capture.ts` around lines 220 - 256, Harden
createFetchTelegramTransport and runTelegramCapture against transient network
failures: add a bounded timeout to each fetch call, and wrap
getUpdates/sendMessage in the runTelegramCapture loop so failures are logged and
the poller continues with retry/backoff rather than propagating and terminating
the process. Preserve offset handling and decision processing for successful
updates.
src/core/brain/rollup-ladder.ts (1)

144-186: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

source computation hardcodes ROLLUP_TIER.fact instead of "the tier below this rung".

Works correctly today because there are exactly two rungs, but the ternary at line 159 will silently break the composition contract described in the module docstring if a third rung is ever appended (its source would keep reading produced["fact"] instead of the rung immediately below it). Tracking the previous rung's tier explicitly (e.g. index-based) instead of hardcoding ROLLUP_TIER.fact would make the composition correct-by-construction.

♻️ Sketch: derive source from the previous rung instead of a hardcoded tier
-  const entries: RollupLadderEntry[] = [];
-  for (const rung of rungs) {
-    const source = rung.tier === ROLLUP_TIER.fact ? factCount : (produced[ROLLUP_TIER.fact] ?? 0);
+  const entries: RollupLadderEntry[] = [];
+  for (let i = 0; i < rungs.length; i++) {
+    const rung = rungs[i]!;
+    const source = i === 0 ? factCount : (produced[rungs[i - 1]!.tier] ?? 0);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/core/brain/rollup-ladder.ts` around lines 144 - 186, Update source
computation in planRollupLadder to derive each non-base rung’s source from the
immediately preceding rung’s tier rather than hardcoding ROLLUP_TIER.fact. Track
the prior rung by index or equivalent while preserving factCount for the base
rung and the existing produced-update ordering.
src/core/brain/deep-synthesis.ts (1)

532-574: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

hasEvidenceIdentity gate is currently unreachable at both call sites. In both files, the EvidenceIdentity fed into hasEvidenceIdentity is always built from a non-empty path, a non-empty constant kind, and a sha256Hex(...) digest — which is never an empty string. So the identity check can never return false here, and the corresponding exclusion path (no_evidence_identity in deep-synthesis, the identity-gated drop in diarization's push) is dead code today, despite being documented as a live "visible loss" gate.

  • src/core/brain/deep-synthesis.ts#L532-L574: the no_evidence_identity branch (lines 546-549) can't currently fire; either drop the redundant check for this construction path or add a code comment noting it's defensive for a future evidence-construction change.
  • src/core/brain/diarization.ts#L186-L222: same — the hasEvidenceIdentity branch inside push (190-193) can't currently reject a line built from entityRel/source-page path + sha256Hex; same call: drop or document as forward-defensive.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/core/brain/deep-synthesis.ts` around lines 532 - 574, The
hasEvidenceIdentity checks are unreachable for the currently constructed
identities. In src/core/brain/deep-synthesis.ts lines 532-574, remove the
redundant no_evidence_identity gate or document it as defensive for future
evidence-construction changes; apply the same choice to the hasEvidenceIdentity
branch in push in src/core/brain/diarization.ts lines 186-222, preserving the
remaining finding and diarization behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.codex-plugin/plugin.json:
- Line 3: Revert the manual version change in the plugin manifest and restore
its prior version value. Treat package.json as the sole version source, and use
scripts/sync-version.ts to propagate any intended version update to mirrored
files.

In `@src/cli/brain/verbs/diarize.ts`:
- Around line 62-65: Update the catch block in src/cli/brain/verbs/diarize.ts
(lines 62-65) to check flags["json"] and emit the { ok: false, error: ... } JSON
envelope for both DiarizationError and other operational errors, while
preserving plain fail() behavior otherwise. Apply the same conditional handling
in the RepairConfirmationError catch block in src/cli/brain/verbs/repair-lane.ts
(lines 60-65), using the existing error message for the envelope.
- Around line 10-21: Update the missing-query validation in cmdBrainDiarize to
call usageError instead of fail, preserving the existing usage message. Import
usageError from ../helpers.ts and ensure this usage-error path exits with the
helper’s code-2 plain-text stderr behavior.

In `@src/core/brain/capture/inbox-drain.ts`:
- Around line 237-265: Separate plan.execute() from archiveCapture() handling so
a successful route is never reported as unroutable when archiving fails.
Preserve the routed result and surface archive-after-success failure distinctly,
ensuring the item is not marked safe to retry or re-executed on the next --apply
run; update the item counters and reason fields around the existing routing loop
accordingly.

In `@src/core/brain/capture/telegram-capture.ts`:
- Around line 121-136: Refactor renderCatchup into a planCatchup flow that
builds the reply and exposes the newest capture watermark without advancing it.
Update handleCaptureUpdate and runTelegramCapture so the Telegram path calls
writeCatchupWatermark only after transport.sendMessage resolves successfully,
while preserving CLI behavior by committing the watermark when stdout delivery
completes.

In `@src/core/brain/dream.ts`:
- Around line 346-358: Rebuild the rollup ladder plan after the mutation path
finalizes runId via nextAvailableDreamRunId, so every fired entry’s
envelope.target_path uses the unique final identifier. Keep the initial plan for
pre-mutation threshold evaluation if needed, but ensure the plan consumed for
rollup execution, ledger updates, and returned artifacts is recomputed with the
corrected runId; add coverage for two non-dry-run dream() calls sharing now with
a firing rung.

In `@src/core/brain/link-graph/graph-holdout.ts`:
- Around line 103-118: Guard the ensureInsideVault call in hydrateEvidence so
targets that escape the vault return { resolved: false, text: "" } instead of
propagating an exception. Keep valid-target parsing and fallback reading
unchanged, ensuring evaluateGraphHoldouts can continue processing remaining
holdouts.

In `@src/core/brain/link-graph/repair-lane.ts`:
- Around line 226-230: Update runRepairLane’s existing-edge check and linked
tracking to handle candidates whose endpointKey returns null, ensuring a
successfully applied candidate is recorded and skipped on subsequent runs.
Preserve the documented convergence behavior for arbitrary RepairCandidate
inputs without changing normal non-null key handling.
- Around line 134-136: Update noteAbsPath and the candidate handling in
runRepairLane so ensureInsideVault failures are caught and converted into the
existing skip-* decision flow rather than escaping and aborting the lane. Apply
this before processing candidate.source or candidate.target, preserve normal
existence checks for valid paths, and ensure invalid endpoints cannot create
edges.

In `@src/core/brain/research/external-fetch.ts`:
- Around line 129-139: Update normalizeRequestKey to include the request’s
accept value in the deterministic cache key, using the same normalization used
by request handling and a stable representation when it is absent. Preserve
exclusion of credentials and headers unrelated to response shape.

In `@src/core/brain/skill-proposals.ts`:
- Around line 129-152: Update the verifier input in the candidate gate around
verifySkillProposalCandidate so it uses the full support count from the
candidate rather than the capped candidate.records sample. Preserve the existing
record-level evidence checks, and ensure supportingCount(candidate) reflects all
records so minSupport values greater than six are evaluated correctly.

In `@src/core/config.ts`:
- Around line 854-858: Update resolveTelegramBotToken and the related
config-value handling around the additionally affected lines to verify
telegram_bot_token is a string before calling trim. Ignore numeric, boolean, and
other non-string values so they resolve to the existing null fallback rather
than throwing.

---

Nitpick comments:
In `@src/core/brain/capture/telegram-capture.ts`:
- Around line 220-256: Harden createFetchTelegramTransport and
runTelegramCapture against transient network failures: add a bounded timeout to
each fetch call, and wrap getUpdates/sendMessage in the runTelegramCapture loop
so failures are logged and the poller continues with retry/backoff rather than
propagating and terminating the process. Preserve offset handling and decision
processing for successful updates.

In `@src/core/brain/deep-synthesis.ts`:
- Around line 532-574: The hasEvidenceIdentity checks are unreachable for the
currently constructed identities. In src/core/brain/deep-synthesis.ts lines
532-574, remove the redundant no_evidence_identity gate or document it as
defensive for future evidence-construction changes; apply the same choice to the
hasEvidenceIdentity branch in push in src/core/brain/diarization.ts lines
186-222, preserving the remaining finding and diarization behavior.

In `@src/core/brain/link-graph/repair-lane.ts`:
- Around line 365-380: Harden the explicit-reference matching in the repair flow
around `mentionsTerm` and the `IDENTITY_STRENGTH.explicitReference` candidate by
rejecting or down-weighting generic, short titles before assigning
`EXPLICIT_REFERENCE_CONFIDENCE`. Ensure common two-character or otherwise
nonspecific titles cannot automatically produce high-confidence edges above the
write threshold, while preserving strong confidence for sufficiently specific
titles.

In `@src/core/brain/rollup-ladder.ts`:
- Around line 144-186: Update source computation in planRollupLadder to derive
each non-base rung’s source from the immediately preceding rung’s tier rather
than hardcoding ROLLUP_TIER.fact. Track the prior rung by index or equivalent
while preserving factCount for the base rung and the existing produced-update
ordering.

In `@src/core/brain/skill-proposals.ts`:
- Around line 573-581: Update appendVerifierRejection to bound the
verifier-rejection ledger by applying an appropriate size or age rotation/cap
before appending, and isolate filesystem failures so append errors are handled
without aborting the surrounding learnSkillProposals pass. Preserve the existing
JSONL entry format and vault path safety.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8c8c6ed3-398b-4653-ad3b-2b80c9fd8e96

📥 Commits

Reviewing files that changed from the base of the PR and between 95dc857 and dfeaeb3.

📒 Files selected for processing (73)
  • .claude-plugin/plugin.json
  • .codex-plugin/plugin.json
  • CHANGELOG.md
  • README.md
  • docs/brainstorm/knowledge-intake-and-consolidation/cli-output/claude.md
  • docs/brainstorm/knowledge-intake-and-consolidation/cli-output/prompt.md
  • docs/brainstorm/knowledge-intake-and-consolidation/design.md
  • docs/brainstorm/knowledge-intake-and-consolidation/plan.md
  • docs/brainstorm/knowledge-intake-and-consolidation/variants.md
  • docs/cli-reference.md
  • docs/mcp.md
  • openclaw.plugin.json
  • package.json
  • plugin.yaml
  • plugins/codex/.codex-plugin/plugin.json
  • plugins/hermes/plugin.yaml
  • pyproject.toml
  • src/cli/brain.ts
  • src/cli/brain/help-text.ts
  • src/cli/brain/verbs/deep-synthesis.ts
  • src/cli/brain/verbs/diarize.ts
  • src/cli/brain/verbs/inbox-drain.ts
  • src/cli/brain/verbs/index.ts
  • src/cli/brain/verbs/repair-lane.ts
  • src/cli/brain/verbs/telegram-capture.ts
  • src/core/brain/capture/capture-note.ts
  • src/core/brain/capture/inbox-drain.ts
  • src/core/brain/capture/telegram-capture.ts
  • src/core/brain/deep-synthesis.ts
  • src/core/brain/diarization.ts
  • src/core/brain/dream.ts
  • src/core/brain/link-graph/graph-holdout.ts
  • src/core/brain/link-graph/repair-lane.ts
  • src/core/brain/obligations.ts
  • src/core/brain/paths.ts
  • src/core/brain/policy.ts
  • src/core/brain/research/external-fetch.ts
  • src/core/brain/research/extract.ts
  • src/core/brain/research/providers/brave.ts
  • src/core/brain/research/providers/provider.ts
  • src/core/brain/research/providers/tavily.ts
  • src/core/brain/research/research.ts
  • src/core/brain/rollup-ladder.ts
  • src/core/brain/skill-proposal-verifier.ts
  • src/core/brain/skill-proposals.ts
  • src/core/brain/types.ts
  • src/core/config.ts
  • src/mcp/brain/knowledge-tools.ts
  • tests/cli/brain-deep-synthesis.test.ts
  • tests/cli/brain-inbox-drain.test.ts
  • tests/cli/brain-repair-lane.test.ts
  • tests/cli/brain-telegram-capture.test.ts
  • tests/core/brain.dream.rollup.test.ts
  • tests/core/brain/capture/capture-note.test.ts
  • tests/core/brain/capture/inbox-drain.test.ts
  • tests/core/brain/capture/telegram-capture.test.ts
  • tests/core/brain/deep-synthesis.test.ts
  • tests/core/brain/diarization.test.ts
  • tests/core/brain/link-graph/graph-holdout.test.ts
  • tests/core/brain/link-graph/repair-collect.test.ts
  • tests/core/brain/link-graph/repair-lane.test.ts
  • tests/core/brain/research/external-fetch.test.ts
  • tests/core/brain/research/extract.test.ts
  • tests/core/brain/research/providers.test.ts
  • tests/core/brain/rollup-ladder.test.ts
  • tests/core/brain/skill-proposal-lifecycle.test.ts
  • tests/core/brain/skill-proposal-verifier.test.ts
  • tests/core/config-telegram-capture.test.ts
  • tests/mcp/brain-tools-parity.test.ts
  • tests/mcp/deep-synthesis-tool.test.ts
  • tests/mcp/diarize-tool.test.ts
  • tests/mcp/mcp.test.ts
  • tests/mcp/removed-tools.test.ts

Comment thread .codex-plugin/plugin.json
Comment thread src/cli/brain/verbs/diarize.ts Outdated
Comment on lines +62 to +65
} catch (err) {
if (err instanceof DiarizationError) return fail(err.message);
return fail((err as Error).message ?? String(err));
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Emit JSON failure envelopes on operational errors when --json is requested.

As per learnings, when the user requests --json output, the CLI should emit a { ok: false, error: ... } JSON envelope for operational failures rather than printing plain text via fail().

  • src/cli/brain/verbs/diarize.ts#L62-L65: Update the catch block to check flags["json"] and emit a JSON envelope if true.
  • src/cli/brain/verbs/repair-lane.ts#L60-L65: Update the RepairConfirmationError catch block to emit a JSON envelope when flags["json"] is true.
📍 Affects 2 files
  • src/cli/brain/verbs/diarize.ts#L62-L65 (this comment)
  • src/cli/brain/verbs/repair-lane.ts#L60-L65
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/cli/brain/verbs/diarize.ts` around lines 62 - 65, Update the catch block
in src/cli/brain/verbs/diarize.ts (lines 62-65) to check flags["json"] and emit
the { ok: false, error: ... } JSON envelope for both DiarizationError and other
operational errors, while preserving plain fail() behavior otherwise. Apply the
same conditional handling in the RepairConfirmationError catch block in
src/cli/brain/verbs/repair-lane.ts (lines 60-65), using the existing error
message for the envelope.

Source: Learnings

Comment thread src/core/brain/capture/inbox-drain.ts
Comment thread src/core/brain/capture/telegram-capture.ts Outdated
Comment thread src/core/brain/link-graph/repair-lane.ts Outdated
Comment thread src/core/brain/link-graph/repair-lane.ts Outdated
Comment thread src/core/brain/research/external-fetch.ts
Comment thread src/core/brain/skill-proposals.ts
Comment thread src/core/config.ts
Comment on lines +854 to +858
export function resolveTelegramBotToken(configPath?: string): string | null {
const env = process.env["TELEGRAM_BOT_TOKEN"]?.trim();
const raw = env || discoverConfig(configPath).data["telegram_bot_token"]?.trim();
return raw !== undefined && raw.length > 0 ? raw : null;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Guard against non-string config values before .trim().

?.trim() only guards null/undefined; a YAML value like telegram_bot_token: 123456 (parsed as a number) or a boolean would throw a TypeError here instead of degrading to the intended null/[] fallback.

🛡️ Proposed fix
 export function resolveTelegramBotToken(configPath?: string): string | null {
   const env = process.env["TELEGRAM_BOT_TOKEN"]?.trim();
-  const raw = env || discoverConfig(configPath).data["telegram_bot_token"]?.trim();
+  const cfgValue = discoverConfig(configPath).data["telegram_bot_token"];
+  const raw = env || (typeof cfgValue === "string" ? cfgValue.trim() : undefined);
   return raw !== undefined && raw.length > 0 ? raw : null;
 }

 export function resolveTelegramCaptureAllowlist(configPath?: string): string[] {
   const env = process.env["TELEGRAM_CHAT_ALLOWLIST"]?.trim();
-  const raw = env || discoverConfig(configPath).data["telegram_chat_allowlist"]?.trim();
+  const cfgValue = discoverConfig(configPath).data["telegram_chat_allowlist"];
+  const raw = env || (typeof cfgValue === "string" ? cfgValue.trim() : undefined);
   if (!raw) return [];

Also applies to: 867-877

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/core/config.ts` around lines 854 - 858, Update resolveTelegramBotToken
and the related config-value handling around the additionally affected lines to
verify telegram_bot_token is a string before calling trim. Ignore numeric,
boolean, and other non-string values so they resolve to the existing null
fallback rather than throwing.

@solaitken

Copy link
Copy Markdown
Collaborator Author

CodeRabbit triage: 14 findings are being fixed in a follow-up commit (usage-error exit code, JSON failure envelopes, distinct archive-after-route state in inbox-drain, catchup watermark advancing only after a delivered send, rollup envelope rebuilt with the final run id, guarded vault-escape handling in the holdout harness and repair lane, null endpoint-key idempotence, accept-aware cache key, full support count for the verifier gate, non-string config guards, long-poll timeout and retry, index-derived rollup rungs, minimum title length for explicit references, defensive-gate comments). Two deliberate skips: the mirrored-version finding is answered inline (bump was done via sync-version.ts, CI sync check green, same as the withdrawn finding on PR #144), and the verifier-rejection ledger growth note is declined because a ledger append failure aborting intake is this repo's fail-loud convention; rotation is deferred.

- diarize: usageError (exit 2) for missing entity; JSON error envelope under --json
- repair-lane CLI: JSON error envelope for confirmation failures under --json
- inbox-drain: split route/archive handling into a distinct archive-failed state; idempotent idea-note merge so a re-run never duplicates
- telegram-capture: advance the catchup watermark only after a successful send; long-poll fetch timeout and resilient poll loop that logs transport errors instead of crashing
- dream: rebuild the rollup plan after a run-id collision so target_path matches the corrected run_id
- graph-holdout: treat a vault-escaping target as dangling instead of throwing
- repair-lane core: return null on vault escape (skip-unsafe-path); refuse null-endpoint-key candidates (skip-null-endpoint-key); raise the explicit-reference minimum title length to a named constant of 4
- external-fetch: include the accept type in the cache key
- skill-proposals: carry the full supporting record set to the verifier gate
- rollup-ladder: derive each rung's source from the previous rung by index
- deep-synthesis / diarization: document the defensive evidence-identity gates

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@solaitken

Copy link
Copy Markdown
Collaborator Author

Review follow-up landed as 9ca0257: 14 findings fixed (usage-error exit code, JSON failure envelopes, distinct archive-failed state with idempotent idea merge, catchup watermark committed only after a delivered send, rollup plan rebuilt with the final run id, guarded vault-escape handling in both the holdout harness and the repair lane, explicit skip for null endpoint keys, accept-aware cache key, uncapped support count for the verifier gate, long-poll timeouts with a continue-on-transport-error loop, index-derived rollup rungs, a named minimum title length for explicit references, and defensive-gate comments). The non-string config guard turned out already safe: the YAML parser coerces scalars to strings, now locked by a regression test. Full suite 6624 pass.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/core/brain/capture/telegram-capture.ts (1)

293-355: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

No backoff on repeated getUpdates transport errors.

The deferred-commit fix here correctly resolves the prior critical finding (watermark now advances only after sendMessage succeeds). However, on a getUpdates failure the loop just continues straight into the next cycle with no delay. Since this runs unbounded in production (runTelegramCapture is invoked with no maxCycles, only a shouldStop signal handler in src/cli/brain/verbs/telegram-capture.ts), a persistent failure (bad token, DNS outage, Telegram API downtime) turns into a hot loop hammering the API as fast as fetch can fail, risking additional rate-limiting/bans and pinning CPU.

🛠️ Suggested direction: exponential backoff on consecutive transport errors
+  let consecutiveGetUpdatesFailures = 0;
   while (opts.maxCycles === undefined || cycles < opts.maxCycles) {
     if (opts.shouldStop?.() === true) break;
     cycles += 1;
     let updates: TelegramUpdate[];
     try {
       updates = await opts.transport.getUpdates(offset);
+      consecutiveGetUpdatesFailures = 0;
     } catch (err) {
       decisions.push(
         logTransportError(vault, -1, null, `getUpdates failed: ${messageOf(err)}`, opts.now),
       );
+      consecutiveGetUpdatesFailures += 1;
+      const backoffMs = Math.min(30_000, 1_000 * 2 ** (consecutiveGetUpdatesFailures - 1));
+      await new Promise((resolve) => setTimeout(resolve, backoffMs));
       continue;
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/core/brain/capture/telegram-capture.ts` around lines 293 - 355,
Single-line getUpdates failures currently retry without delay, causing a hot
loop during persistent transport outages. Update runTelegramCapture to track
consecutive getUpdates transport failures and await an exponential backoff
before retrying, resetting the failure count after a successful getUpdates call;
preserve decision logging, stop handling, and normal polling behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/core/brain/capture/telegram-capture.ts`:
- Around line 293-355: Single-line getUpdates failures currently retry without
delay, causing a hot loop during persistent transport outages. Update
runTelegramCapture to track consecutive getUpdates transport failures and await
an exponential backoff before retrying, resetting the failure count after a
successful getUpdates call; preserve decision logging, stop handling, and normal
polling behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 825d10fd-8854-4c8f-9f97-6fee79bacf4c

📥 Commits

Reviewing files that changed from the base of the PR and between dfeaeb3 and 9ca0257.

📒 Files selected for processing (25)
  • src/cli/brain/verbs/diarize.ts
  • src/cli/brain/verbs/inbox-drain.ts
  • src/cli/brain/verbs/repair-lane.ts
  • src/cli/brain/verbs/telegram-capture.ts
  • src/core/brain/capture/inbox-drain.ts
  • src/core/brain/capture/telegram-capture.ts
  • src/core/brain/deep-synthesis.ts
  • src/core/brain/diarization.ts
  • src/core/brain/dream.ts
  • src/core/brain/link-graph/graph-holdout.ts
  • src/core/brain/link-graph/repair-lane.ts
  • src/core/brain/research/external-fetch.ts
  • src/core/brain/rollup-ladder.ts
  • src/core/brain/skill-proposals.ts
  • tests/cli/brain-diarize.test.ts
  • tests/cli/brain-repair-lane.test.ts
  • tests/core/brain.dream.rollup.test.ts
  • tests/core/brain/capture/inbox-drain.test.ts
  • tests/core/brain/capture/telegram-capture.test.ts
  • tests/core/brain/link-graph/graph-holdout.test.ts
  • tests/core/brain/link-graph/repair-collect.test.ts
  • tests/core/brain/link-graph/repair-lane.test.ts
  • tests/core/brain/research/external-fetch.test.ts
  • tests/core/brain/skill-proposals.test.ts
  • tests/core/config-telegram-capture.test.ts
🚧 Files skipped from review as they are similar to previous changes (16)
  • src/cli/brain/verbs/diarize.ts
  • tests/core/brain/link-graph/repair-collect.test.ts
  • tests/core/brain/capture/inbox-drain.test.ts
  • tests/core/brain/research/external-fetch.test.ts
  • src/cli/brain/verbs/telegram-capture.ts
  • src/cli/brain/verbs/inbox-drain.ts
  • src/core/brain/diarization.ts
  • src/core/brain/research/external-fetch.ts
  • src/core/brain/rollup-ladder.ts
  • src/core/brain/link-graph/repair-lane.ts
  • src/core/brain/capture/inbox-drain.ts
  • src/cli/brain/verbs/repair-lane.ts
  • src/core/brain/deep-synthesis.ts
  • src/core/brain/dream.ts
  • src/core/brain/link-graph/graph-holdout.ts
  • src/core/brain/skill-proposals.ts

…lures

A failed getUpdates retried immediately, so a persistent transport
outage (bad runtime token, DNS failure, API downtime) turned the
long-poll loop into a hot loop that hammered the API and pinned the
CPU, since production runs with no maxCycles.

Track consecutive getUpdates transport failures and await an
exponential backoff (base 1000 ms doubling, capped at 30000 ms)
before retrying, resetting the counter after any clean poll. The
sleep is injectable so tests assert the growing waits without
spending wall-clock time, and a stop requested during a backoff
ends the loop promptly instead of issuing another poll.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@solaitken

Copy link
Copy Markdown
Collaborator Author

Incremental review finding addressed in f673a46: repeated getUpdates transport failures now back off exponentially (base 1000 ms doubling, capped at 30000 ms, named constants), the counter resets after a clean poll, and a stop request during backoff ends the loop promptly. Full suite 6627 pass.

@solaitken
solaitken merged commit 842d690 into main Jul 19, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants