feat: knowledge intake and consolidation (v1.36.0)

.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "open-second-brain",
-  "version": "1.35.0",
+  "version": "1.36.0",
   "description": "Plugin-first second brain package for AI agents and humans.",
   "author": {
     "name": "Open Second Brain contributors"

.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "open-second-brain",
-  "version": "1.35.0",
+  "version": "1.36.0",
   "description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
   "skills": "./skills",
   "hooks": "./hooks/hooks.json",

CHANGELOG.md

@@ -5,6 +5,26 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.36.0] - 2026-07-19
+
+A knowledge intake and consolidation wave: eight units that carry content from the operator's pocket into the vault and turn what accumulates into trustworthy insight. Phone captures arrive through an inbound Telegram bot and drain through a report-then-apply routing pass, keyed web-research providers feed the citation-constrained pipeline, facts roll up into higher tiers on pure counters, entities gain diarized profiles with a stated-vs-evidenced gap, synthesis findings carry causal context and decomposed confidence behind an evidence-identity gate, the link graph gets a capped idempotent repair lane with efficacy holdouts, and skill proposals pass a deterministic verifier before reaching the pending queue. Two shared seams carry the wave: a capture-note contract and a keyed external-fetch helper. The kernel still calls no LLM, and every new surface is byte-identical when unused.
+
+### Added
+
+- Inbound Telegram capture bot (`o2b brain telegram-capture run | catchup`): an explicit long-polling runner (fetch-based getUpdates, no new dependencies) gated by `telegram_bot_token` (env `TELEGRAM_BOT_TOKEN`) and a chat-id allowlist `telegram_chat_allowlist` (env `TELEGRAM_CHAT_ALLOWLIST`, empty accepts nothing); each accepted message becomes one staged capture note through a new capture-note contract (kind, provenance, capture timestamp) under `Brain/captures/`; rejected or malformed updates log one decision each to a capture-decisions sidecar; `catchup` replays captures since the last acknowledged one; a missing token is a typed startup error.
+- Inbox-drain pass (`o2b brain inbox-drain [--apply]`): walks staged captures through the contract, classifies each structurally (URL-shaped body as source reference, explicit obligation marker as task, otherwise atomic idea), routes on apply (source ingest, note create in `captured/`, obligation open), archives processed captures, and reports every item with action and reason; dry-run is the default and writes nothing; rerun after apply is a no-op.
+- Keyed web-research providers plus full-page extract: Brave and Tavily join the research pool only when `BRAVE_API_KEY` or `TAVILY_API_KEY` is set, through a shared external-fetch helper (env-gated, Bearer or named-header auth, typed network/auth/http/payload errors, response cache keyed by normalized request, keys never in cache keys or error text); a keyless pool reports itself empty and stays byte-identical; the extract step feeds verbatim page text into the existing citation-constrained pipeline.
+- Count-triggered fact rollup ladder in the dream synthesize phase: when new facts at a tier reach the threshold (optional `rollup:` config block with `fact_threshold` and `identity_threshold`), one needs-llm-step rollup envelope is emitted and the counter reset is recorded in the dream report over a persisted ledger (`Brain/rollup-ladder.json`); rungs compose from facts to rollups to the identity tier; below threshold the dream output is byte-identical.
+- Subject diarization (`o2b brain diarize`, MCP `brain_diarize`): assembles an entity's document set from the registry and ingested source pages, computes a stated-vs-evidenced section deterministically (stated claims versus evidence frequency and recency, each line carrying an evidence identity and classified stated_corroborated, stated_unevidenced, or evidenced_unstated), and emits a profile skeleton plus one needs-llm-step envelope; unknown entities are a typed error.
+- Synthesis findings hardening: every deep-synthesis finding carries an additive causal-context field and decomposed confidence components (support, opposition, freshness, coverage, all deterministic); findings without an evidence identity are excluded and counted with reasons; the exported evidence-identity type and predicate are shared with diarization; the steelman seed selection is unchanged; both the CLI `--json` payload and the MCP structured content expose the new fields additively.
+- Memory-graph repair lane (`o2b brain repair-lane [--apply --confirm]`): candidate edges ordered by identity strength (explicit references, session continuity, same-topic evidence; inferred candidates opt-in), gated by a confidence threshold and a hard per-run write cap, dry-run by default, apply requires the exact confirmation phrase, and idempotent forward-scan batching makes reruns converge to zero writes; the paired holdout harness measures graph lift separately from direct recall and fails the gate on any dangling or unhydrated target.
+- Skill-proposal verification and evolution: a deterministic pre-promotion verifier validates each draft against its own supporting records before it reaches pending (rejections recorded with reasons in a ledger), accepted skills carry a version that increments on evolution, and a same-name collision merges support instead of forking; the human accept and reject flow is unchanged and legacy proposals default to version 1.
+
+### Changed
+
+- The MCP tool surface grows from 106 to 107 tools (`brain_diarize`).
+- Deep-synthesis reports gain additive fields (causal context, decomposed confidence, excluded-finding counters) on the core report, the CLI JSON payload, and the MCP structured content; prior fields and consumers are unchanged.
+
 ## [1.35.0] - 2026-07-19
 
 A trusted recall and memory write surface wave: ten units that carry vault knowledge to the agent at the moment it matters and make every step accountable. An opt-in bounded recall brief lands on each prompt, retrieval zero-ranks quarantined material and hands back receipts naming every exclusion, a supersede relation on a successor fades the superseded note without editing it, and MCP agents gain the full atomic note lifecycle: update, append, and all-or-nothing mixed batches. Two shared kernels carry the wave: a deterministic retrieval rank-adjustment stage and an atomic multi-operation write core. The kernel still calls no LLM, and every new surface is byte-identical when its flag or param is omitted.
@@ -6582,6 +6602,7 @@ plugin config (vault field)`, and exits with a clear
 - Sandbox vault and plugin manifest fixtures for tests.
 - GitHub release workflow for tag-based and manually dispatched releases.
 
+[1.36.0]: https://github.com/itechmeat/open-second-brain/compare/v1.35.0...v1.36.0
 [1.35.0]: https://github.com/itechmeat/open-second-brain/compare/v1.34.0...v1.35.0
 [1.34.0]: https://github.com/itechmeat/open-second-brain/compare/v1.33.0...v1.34.0
 [1.33.0]: https://github.com/itechmeat/open-second-brain/compare/v1.32.0...v1.33.0

README.md

@@ -8,7 +8,9 @@ Open Second Brain plugs into [Hermes Agent](https://github.com/NousResearch/herm
 
 ## What is new
 
-Open Second Brain 1.35.0 is a trusted recall and memory write surface release. Vault knowledge now reaches the agent at the moment it matters: an opt-in hook injects a bounded, fenced recall brief on every prompt (hard caps, confidence floor, one audit line per decision, nothing injected on any error), the morning brief renders a typed age-labeled activity timeline, and recurring recall gaps stop evaporating - they become durable task notes, surface as a session-start agenda, and close themselves once the gap is filled. What retrieval returns can be trusted: an opt-in gate zero-ranks quarantined material out of both semantic and lexical search and attaches two receipts naming every exclusion, a `supersedes` relation on a successor fades the outdated note without editing it, session recall can inline the raw capture beside each derived record with an `extracted` discriminator, and budget clipping can no longer strip `session_id` or `agent_id` from a pack. MCP agents gain the full write lifecycle - `brain_update_note`, `brain_append_note`, and the all-or-nothing `brain_write_batch` - and `o2b doctor --readiness` answers the Day-0 wiring question with three functional probes and a real exit code. The kernel still calls no LLM.
+Open Second Brain 1.36.0 is a knowledge intake and consolidation release. Content now reaches the vault from the operator's pocket: an inbound Telegram bot (explicit long-poll runner, chat allowlist, `/catchup`) lands each message as a staged capture with provenance, and `o2b brain inbox-drain` classifies every capture structurally and routes it to a source page, a note, or an obligation with a per-item report, dry-run first. Keyed Brave and Tavily providers plus a full-page extract step feed the citation-constrained research pipeline and stay byte-identical without keys. What accumulates now consolidates: facts roll up into higher tiers on pure counters inside the dream pass, `o2b brain diarize` distills everything known about an entity into a profile with a stated-vs-evidenced gap, synthesis findings carry causal context and decomposed confidence behind an evidence-identity gate that counts every exclusion, `o2b brain repair-lane` densifies the link graph with a capped, idempotent, dry-run-first lane whose holdout harness proves graph lift and fails on dangling or unhydrated targets, and skill proposals pass a deterministic verifier, carry versions, and merge same-name collisions before a human ever reviews them. The kernel still calls no LLM.
+
+Previous release, 1.35.0, was a trusted recall and memory write surface wave: a bounded audited recall brief on every prompt, a retrieval trust gate with per-pack receipts, relation-only supersede fade, the full MCP note write lifecycle, and fail-fast doctor readiness probes. Details live in the [CHANGELOG](CHANGELOG.md).
 
 ## Why
 

docs/brainstorm/knowledge-intake-and-consolidation/cli-output/claude.md

@@ -0,0 +1,39 @@
+### Variant 1: Convention-first, zero extraction
+
+- **Approach**: Extract no shared modules. Each of the eight tasks lands self-contained in kanban order (1→8); the capture note shape, env-gating pattern, and evidence-identity fields are documented conventions (in code comments plus a short docs page), with each consumer implementing its own copy. Task 2 pattern-matches whatever frontmatter task 1 happened to write; tasks 5 and 6 each define their own evidence-identity fields; task 3 builds its own gated fetch-plus-cache inline.
+- **Trade-offs**:
+  - Pro: lowest upfront cost; all eight tasks are parallelizable with no blocking dependency chain.
+  - Pro: no speculative abstraction; every line shipped has exactly one consumer today.
+  - Con: the capture contract is defined implicitly by task 1's output and re-derived by task 2's parser - a drift bug factory, and the drain pass's structural classification (frontmatter, markers) depends on that shape being exact.
+  - Con: evidence-identity duplicated between the finding gate (6) and diarization gap section (5) means two vocabularies that will diverge, and "excluded for missing proof identity" will mean different things in two reports.
+  - Con: keyed env gating plus typed network errors plus cache implemented at least twice (Brave/Tavily, page extract), likely three times (Telegram polling).
+- **Complexity**: small
+- **Risk**: medium (drift between paired tasks 1/2 and 5/6 is near-certain rework inside the same PR)
+
+### Variant 2: Two hard seams, two conventions
+
+- **Approach**: Extract exactly the two seams that have concrete dual consumers with real coupling: (a) a capture-note contract module (frontmatter `kind`, provenance, timestamps, staging/archive paths) that the Telegram bot writes through and the inbox drain reads through, anchored on the existing inbox-vs-processed distinction in forget-plan.ts; (b) a small keyed-env-gated HTTP fetch helper (Bearer auth, typed errors, shared cache) used by Brave, Tavily, and full-page extract - Telegram may adopt it but keeps its existing token plumbing. Evidence-identity (seam 3) stays a shared exported type plus predicate in the deep-synthesis types, not a module; dream extension (seam 4) stays "the rollup ladder is ordinary code in the synthesize phase" - no plugin registry. Each seam lands inside the commit of its first consumer (contract with task 1, fetch helper with task 3), preserving one-atomic-commit-per-task. Ordering: 1→2 (intake track), 6→5 then 4 (consolidation track), 3 anywhere before or parallel, 7 and 8 fully independent.
+- **Trade-offs**:
+  - Pro: the two extractions are exactly where silent drift would break a same-PR sibling task; the two non-extractions avoid frameworks with one consumer.
+  - Pro: type-level evidence-identity gives 5 and 6 one vocabulary at zero runtime cost, and the compiler enforces it - fits the deterministic-kernel and no-new-deps constraints.
+  - Pro: ordering gives each track a contract-defining task first (1 defines captures, 6 defines evidence identity), so downstream tasks consume rather than guess; the two tracks plus 3/7/8 can proceed in parallel.
+  - Con: seam-inside-first-consumer-commit means task 2's author must wait for task 1's commit (and 5 waits on 6); mild serialization within tracks.
+  - Con: if a fourth network consumer appears later (e.g. more providers), the minimal fetch helper may need a second pass; accepted as cheap.
+- **Complexity**: medium
+- **Risk**: low
+
+### Variant 3: Platform-first, four subsystems
+
+- **Approach**: Treat all four candidate seams as first-class subsystems landed before any feature task: an intake staging bus (contract, writer API, reader iterator, archive semantics), a `net/` gateway (provider registry, env gating, cache, budgets) that all network callers must route through including Telegram, a synthesis evidence-identity module with decomposed-confidence types, and a formal dream-phase extension registry into which the rollup ladder plugs and which the repair lane explicitly declines. Feature tasks then become thin consumers, in any order.
+- **Trade-offs**:
+  - Pro: cleanest long-term layering; future intake sources and providers slot in without touching consumers.
+  - Pro: eliminates all cross-task drift by construction; every seam has a single owner file.
+  - Con: the dream-phase registry is a framework whose only new client is the rollup ladder - and the wave's own spec says the repair lane is NOT a dream phase, so half the registry's motivation is explicitly out of scope. Speculative generality against this repo's "no stubs, no do-nothing surfaces" ethos.
+  - Con: forcing Telegram long-polling through a generic gateway fights its existing, working token/config plumbing in telegram.ts, contradicting the task's "reuse existing plumbing" instruction.
+  - Con: subsystem-first commits are not attributable to any one kanban task, straining the one-atomic-commit-per-task convention, and the whole wave serializes behind platform work.
+- **Complexity**: large
+- **Risk**: high
+
+### Recommended: Variant 2
+
+**Rationale**: The rework risk in this wave is concentrated in exactly two places - the task-1/task-2 capture contract and the task-5/task-6 evidence vocabulary - and Variant 2 hardens the first as a module and the second as a shared type, which is proportionate to two consumers each. Variant 1 guarantees same-PR drift on those pairs, while Variant 3 builds a dream-extension framework the spec itself undercuts and violates the repo's aversion to speculative surfaces and its commit-per-task convention. Variant 2 also yields the best ordering story: each track opens with its contract-defining task, keeping the eight commits atomic and the two tracks parallel.