feat(custom-profiles): opt-in domain profiles — Hacker News chronicle + Hermes situational tracker - #21
Merged
Merged
Conversation
Add a custom-profiles mechanism with zero impact when no profile is active: one CUSTOM_PROFILE switch names the active profile(s); each lives in its own custom-profiles/<name>/ folder. Prompt fragments (<target>.add.md / <target>.replace.md) shape the wiki maintainer/writer; an optional ingestor.py feeds data/sources/ via the existing watcher, supervised by a new profile_runner sidecar that sleeps when no profile is set. - braindb/custom_profile.py: brace-safe add/replace prompt composition (multi-profile) - braindb/profile_runner.py: dormant ingestor supervisor (restart-on-exit) - braindb/routers/wiki.py: wrap maintainer + writer assembly with profile add/replace - docker-compose.yml: CUSTOM_PROFILE env + profile_runner service - custom-profiles/SKILL.md + README + example/: profile authoring guide - custom-profiles/hackernews/: keyless worked example (HN -> tech-entity wikis) - custom-profiles/gdrive/: Google Drive folder follower, incremental + diff-aware - tests/test_custom_profile.py
Append a 'Current date (UTC)' line to both the maintainer and writer prompts (additive, in wiki.py) so temporal reasoning (recency, 'as of', dated/bucketed sections) has an anchor. Ground the hackernews chronicle-aging rule to that date and add a consistent dated-line format.
Scheduled ask to an external Hermes web-search agent -> a dated report dropped as a datasource -> a central 'Hormuz status' situational wiki with retrospective now/recent-days/recent-weeks/forecast/history sections. Adds query.md, ingestor.py, wiki_writer.add.md, wiki_maintainer.add.md, sample_answer.md, README, .env.example. .gitignore whitelists the profile but keeps .env/.state/__pycache__ out (the Hermes bearer token stays local).
A weak writer model sometimes emits a body that is serialization garbage
(`""`, `body=""`, `{}`) rather than a real page. The previous gate only
tested `.strip()` (whitespace), so quote/brace junk slipped through, got
written verbatim as the wiki content, and bumped the revision -- which then
self-perpetuated into empty, high-revision pages (a marker-less page can no
longer be section-edited, so every later cycle re-emits junk and re-bumps).
Add a single `_is_blank_body()` definition (matches ONLY pure
punctuation/whitespace, optionally wrapped as `body=...`; any real body has
letters/markup and never matches), use it for the existing empty-body gate,
and add one backstop guard on the final body right before persist -- routing
blank bodies to the existing release_or_fail_jobs path instead of writing
them. The legitimate empty-body cases (attach no-op, attach after section
edits) return earlier and are untouched, so this only rejects genuinely
unpersistable writes. Healing-positive: a rejected entity stays a healable
orphan and is retried next cycle; good pages can no longer be overwritten by
junk.
The scheduler's HTTP read-timeout on /wiki/maintain and /wiki/write (knob WIKI_AGENT_TIMEOUT, default 1200s / 20 min) is too tight for smaller/slower quantised local models (e.g. Gemma 12B QAT), whose full-body writes can exceed 20 min. At 1200s the scheduler abandoned slow-but-completing writes and forced needless retries. Bump the default to 2400s (40 min) and sync the .env.example docs. This only controls the scheduler client's patience -- the API is not bounded by it and a write past the deadline still commits.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds custom profiles — opt-in, self-contained overlays that reshape what BrainDB ingests and how its
auto-maintained wikis read, with zero effect on defaults when inactive — plus two ready-to-run profiles
and a pair of wiki-writer robustness fixes. Bumps the version to 0.9.0.
Why / what's exciting
The wiki pipeline was one-size-fits-all. Profiles let a deployment teach the maintainer/writer how to
name, structure, and source pages for a domain — from one folder, one env switch — without forking core:
hackernews(keyless) — polls Hacker News and turns the firehose into company/project/tech wikiswith a dated
current-developmentschronicle that ages gracefully intobackground.hermes— on a schedule, asks an external web-search Hermes agent a dated, stateless question andfolds each answer into a live, dashboard-shaped situational page (status · confidence · now ·
recent-days · recent-weeks · forecast · history — every line dated +
[[ref:UUID]]-cited), while theother entities each report mentions get their own cross-linked wikis. Dormant by default; a dry-run
sample exercises the whole pipeline offline.
How
braindb/custom_profile.py,braindb/profile_runner.py):.add.md/.replace.mdprompt fragments + an optional supervised
ingestor.pyper profile. Core API, schema, and base promptsare untouched; everything is additive and gated by
CUSTOM_PROFILE.custom-profiles/:example/template,hackernews/,hermes/,gdrive/. Each profile's real.env/state is gitignored; only.env.exampletemplates are tracked.Also in this release