T3 — logbook pipeline: writer/reader, tier routing, salient push

.github/workflows/logbook-ci.yml

@@ -0,0 +1,42 @@
+name: logbook-ci
+
+# Automated acceptance for the Python logbook pipeline (the record layer's
+# writer/reader): schema validation, tier routing, the redaction gate, and the
+# transport-not-brain salient push. Round-trips the synthetic sample and asserts
+# the pipeline creates NO real entries in the repo (AGENTS.md rule 2 / ADR-0005).
+#
+# Runs on EVERY pull request as a signal (no paths filter). Like harness-ci it is
+# NOT a hard ruleset-required check — cc waits for green before self-merging, and
+# CI checks are signals not gates here (adr-0001, matching homelab-ops). pip is
+# cached; the only dependency is PyYAML.
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: logbook-ci-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  logbook-ci:
+    name: logbook-ci
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: logbook/pipeline
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+          cache: pip
+          cache-dependency-path: logbook/pipeline/pyproject.toml
+      - run: python -m pip install --upgrade pip
+      - run: pip install -e ".[dev]"
+      - name: test + coverage gate (100% pipeline surface)
+        run: pytest --cov --cov-report=term-missing

.gitignore

@@ -47,6 +47,7 @@ __pycache__/
 *.pyo
 dist/
 build/
+*.egg-info/
 .venv/
 venv/
 target/

logbook/pipeline/README.md

@@ -0,0 +1,83 @@
+# logbook pipeline (T3)
+
+Writer/reader for logbook entries — the **mechanism** of the record's capture
+layer. It validates against [`../schema/entry.schema.yaml`](../schema/entry.schema.yaml),
+routes by visibility tier, enforces the redaction gate, and offers a
+transport-not-brain salient-push hook.
+
+It provides the mechanism only. *What* to write and *what is salient* are the
+drive layer's judgment (T4/T5), never this package's.
+
+## Language
+
+Python. The pipeline touches the **private record**, so it runs on a local model
+host at runtime (AGENTS.md rule 6) and co-locates with the Python L3 cognition
+layer (ADR-0006). The TypeScript harness (L2) stays on the world-facing side of
+the JSON bridge seam; this is the record-facing side.
+
+## Discipline this enforces
+
+- **Visibility tiers** (AGENTS.md rule 3): every entry defaults to `private`.
+  The writer **refuses** to emit a `shareable`/`narrative` entry without
+  `redaction_checked: true` — a publication gate, raised as
+  `RedactionRequiredError`. Validation and the gate are separate: a `shareable`
+  entry with the flag unset is *well formed* but *not emittable*.
+- **The record never enters this repo** (AGENTS.md rule 2 / ADR-0005). The store
+  roots **outside** the working tree by default (`~/.commonplace/logbook`), so a
+  stray entry can never be committed. The test suite asserts the repo tree holds
+  no `*.entry.md`.
+- **Transport, not brain** (`exploration/gateway-selection.md`): the salient-push
+  sinks carry an already-chosen entry to the operator's channel; they contain no
+  salience logic, LLM call, or filtering.
+
+## Layout
+
+```
+src/commonplace_logbook/
+  schema.py    parse entry.schema.yaml (a small DSL) into a typed SchemaSpec
+  entry.py     Entry model + markdown frontmatter parse/serialize
+  validate.py  structural validation; applies schema defaults
+  store.py     visibility-tier -> on-disk path (env-overridable, out-of-repo)
+  writer.py    validate -> redaction gate -> tier-routed write
+  reader.py    read/loads an entry, validating against the schema
+  salient.py   transport-not-brain push sinks (AstrBot / Discord / Null)
+  cli.py       validate / route / emit
+```
+
+## Usage
+
+```python
+from commonplace_logbook import read_entry, write_entry, Entry, salient_push
+
+entry = read_entry("note.entry.md")        # parse + validate
+result = write_entry(entry)                  # validate, gate, route by tier
+salient_push(entry, note="felt worth saying")  # only if the agent judged it salient
+```
+
+CLI:
+
+```sh
+commonplace-logbook validate note.entry.md   # schema check
+commonplace-logbook route    note.entry.md   # show tier + target path (no write)
+commonplace-logbook emit     note.entry.md   # validate, gate, write to the store
+```
+
+## Configuration (env)
+
+| Variable | Purpose | Default |
+|---|---|---|
+| `COMMONPLACE_LOGBOOK_HOME` | store root for all tiers | `~/.commonplace/logbook` |
+| `COMMONPLACE_OBSIDIAN_VAULT` | if set, `private` entries route here | unset |
+| `COMMONPLACE_LOGBOOK_SCHEMA` | override the schema path | in-tree `../schema` |
+| `COMMONPLACE_ASTRBOT_ENDPOINT` / `_TARGET` / `_TOKEN` | AstrBot push channel | unset |
+| `COMMONPLACE_DISCORD_WEBHOOK` | thin Discord-webhook fallback channel | unset |
+
+With no channel configured, the push hook resolves to a `NullSink` (no network),
+so it is safe to call unconditionally.
+
+## Develop
+
+```sh
+pip install -e ".[dev]"
+pytest --cov --cov-report=term-missing   # 100% pipeline surface, mirrors logbook-ci
+```

logbook/pipeline/pyproject.toml

@@ -0,0 +1,45 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "commonplace-logbook"
+version = "0.1.0"
+description = "Writer/reader for commonplace logbook entries — schema validation, visibility-tier routing, transport-not-brain salient push."
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "AGPL-3.0-only" }
+authors = [{ name = "commonplace dyad" }]
+dependencies = [
+  "PyYAML>=6.0",
+]
+
+[project.optional-dependencies]
+dev = [
+  "pytest>=8",
+  "pytest-cov>=5",
+]
+
+[project.scripts]
+commonplace-logbook = "commonplace_logbook.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+testpaths = ["tests"]
+addopts = "-ra"
+
+[tool.coverage.run]
+branch = true
+source = ["commonplace_logbook"]
+
+[tool.coverage.report]
+show_missing = true
+fail_under = 100
+exclude_lines = [
+  "pragma: no cover",
+  "if TYPE_CHECKING:",
+  "raise NotImplementedError",
+]

logbook/pipeline/src/commonplace_logbook/__init__.py

@@ -0,0 +1,66 @@
+"""commonplace logbook pipeline — writer/reader, tier routing, salient push.
+
+Validates against ``logbook/schema/entry.schema.yaml``; defaults every entry to
+the ``private`` tier and refuses to emit ``shareable``/``narrative`` without
+``redaction_checked: true`` (AGENTS.md rule 3). It provides the *mechanism* for
+writing and pushing entries — never the judgment of what to write or what is
+salient (that is the drive layer, T4/T5).
+"""
+
+from __future__ import annotations
+
+from .entry import Entry, parse_markdown, to_markdown
+from .errors import (
+    EntryFormatError,
+    LogbookError,
+    RedactionRequiredError,
+    SchemaError,
+    ValidationError,
+)
+from .reader import loads, read_entry
+from .salient import (
+    AstrBotSink,
+    DiscordWebhookSink,
+    NullSink,
+    SalientSink,
+    format_message,
+    salient_push,
+    sink_from_env,
+)
+from .schema import SchemaSpec, load_schema
+from .store import NARRATIVE, PRIVATE, SHAREABLE, StoreConfig, entry_filename
+from .validate import validate_entry, validate_payload
+from .writer import WriteResult, write_entry
+
+__all__ = [
+    "Entry",
+    "parse_markdown",
+    "to_markdown",
+    "LogbookError",
+    "SchemaError",
+    "ValidationError",
+    "RedactionRequiredError",
+    "EntryFormatError",
+    "loads",
+    "read_entry",
+    "load_schema",
+    "SchemaSpec",
+    "validate_entry",
+    "validate_payload",
+    "write_entry",
+    "WriteResult",
+    "StoreConfig",
+    "entry_filename",
+    "PRIVATE",
+    "SHAREABLE",
+    "NARRATIVE",
+    "salient_push",
+    "sink_from_env",
+    "format_message",
+    "SalientSink",
+    "NullSink",
+    "AstrBotSink",
+    "DiscordWebhookSink",
+]
+
+__version__ = "0.1.0"

logbook/pipeline/src/commonplace_logbook/cli.py

@@ -0,0 +1,62 @@
+"""Thin CLI over the pipeline: validate / route / emit.
+
+Reads an entry's markdown from a path or stdin (``-``). Intended for local use;
+``emit`` routes to the (out-of-repo) private store by default.
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+from typing import List, Optional
+
+from .errors import LogbookError
+from .reader import loads
+from .writer import write_entry
+
+
+def _read_source(src: str) -> str:
+    if src == "-":
+        return sys.stdin.read()
+    return Path(src).read_text(encoding="utf-8")
+
+
+def main(argv: Optional[List[str]] = None) -> int:
+    parser = argparse.ArgumentParser(prog="commonplace-logbook", description=__doc__)
+    sub = parser.add_subparsers(dest="cmd", required=True)
+
+    p_val = sub.add_parser("validate", help="validate an entry against the schema")
+    p_val.add_argument("source", help="path to an entry .md, or '-' for stdin")
+
+    p_route = sub.add_parser("route", help="show the tier + target path (no write)")
+    p_route.add_argument("source", help="path to an entry .md, or '-' for stdin")
+
+    p_emit = sub.add_parser("emit", help="validate, gate, and write to the routed store")
+    p_emit.add_argument("source", help="path to an entry .md, or '-' for stdin")
+    p_emit.add_argument("--dry-run", action="store_true", help="route but do not write")
+
+    args = parser.parse_args(argv)
+
+    try:
+        entry = loads(_read_source(args.source))
+        if args.cmd == "validate":
+            print(f"ok: valid {entry.visibility} {entry.type} entry")
+            return 0
+        if args.cmd == "route":
+            result = write_entry(entry, dry_run=True)
+            print(f"{result.tier}\t{result.path}")
+            return 0
+        if args.cmd == "emit":
+            result = write_entry(entry, dry_run=args.dry_run)
+            verb = "would write" if args.dry_run else "wrote"
+            print(f"{verb} [{result.tier}] {result.path}")
+            return 0
+    except LogbookError as exc:
+        print(f"error: {exc}", file=sys.stderr)
+        return 1
+    return 2  # pragma: no cover - argparse enforces a subcommand
+
+
+if __name__ == "__main__":  # pragma: no cover
+    raise SystemExit(main())