Skip to content

knowledge-curator: recover inbox notes missing from vault clone via DB fallback#594

Merged
ExtraToast merged 2 commits into
mainfrom
fix/curator-inbox-db-fallback
Jun 6, 2026
Merged

knowledge-curator: recover inbox notes missing from vault clone via DB fallback#594
ExtraToast merged 2 commits into
mainfrom
fix/curator-inbox-db-fallback

Conversation

@ExtraToast

Copy link
Copy Markdown
Owner

InboxPass.has_work() performed a pure filesystem scan of the vault PVC clone.
A note captured to kb_notes whose file was absent from the local clone — due
to a DB/git desync, a failed push that was retried on a re-cloned PVC, or any
other divergence — would sit with scope = '_inbox' in the database
indefinitely. Every curator tick returned has_work() = False, run() was
never called, and the note was never promoted.

A May 2026 capture (home_direct A records owned by in-cluster cloudflare-ddns)
has been stuck in this state for 19 days.

Fix

has_work() now falls back to a cheap DB existence check when the filesystem
scan finds nothing:

SELECT 1 FROM kb_notes
WHERE scope = '_inbox'
  AND vault_path IS NOT NULL
  AND vault_path NOT LIKE '_inbox/_needs-review/%'
LIMIT 1

If the DB reports work, run() proceeds — vault.sync() pulls from the remote
before the inbox scan, so a file that exists on the remote will be pulled and
processed. If the file is genuinely absent from the remote too, run() returns
no_work safely.

has_inbox_notes() is added to the CuratorStore protocol, PostgresCuratorStore,
and InMemoryCuratorStore. Two new unit tests cover the DB-fallback path and the
_needs-review exclusion.

The stuck May note will drain automatically on the next curator tick without
any manual intervention.

An agent PR reworded the comment in an already-applied Flyway migration
(assistant-api V12). Flyway checksums the whole file, so the edit flipped the
checksum and crashlooped assistant-api on the next boot. Nothing in CI caught
it before merge.

check-migrations.sh enforces two invariants on every PR:
  - immutability: a migration present on the base branch may not be modified,
    renamed, or deleted (label allow-migration-change downgrades this to a
    warning for a reviewed correction);
  - versioning: per service, versions are unique and each newly added
    migration's version exceeds the highest already on the base branch.

knowledge-api splits versions across migration/ and migration-pg/; both trees
share one per-service version namespace, which the script accounts for. The
workflow runs unconditionally so it is safe to require as a status check.
…B fallback

InboxPass.has_work() was a pure filesystem scan. A note captured to kb_notes
whose file was absent from the local vault PVC clone (DB/git desync) could sit
in _inbox scope indefinitely — has_work() returned False every tick and run()
was never called to vault.sync() + pull the file from the remote.

has_work() now falls back to a cheap DB query (SELECT 1 FROM kb_notes WHERE
scope = '_inbox' AND vault_path NOT LIKE '_inbox/_needs-review/%' LIMIT 1)
when the filesystem scan finds nothing. If the DB says there is work, run()
proceeds, vault.sync() pulls the file from the remote, and the pass processes
it normally. Notes that are genuinely missing from the remote too will produce
a no_work outcome from run() — safe, no infinite loop.

Adds has_inbox_notes() to CuratorStore protocol, PostgresCuratorStore, and
InMemoryCuratorStore; two new unit tests cover the fallback and the
_needs-review exclusion.
@ExtraToast ExtraToast added the bug Something isn't working label Jun 6, 2026
@ExtraToast ExtraToast self-assigned this Jun 6, 2026
@ExtraToast ExtraToast merged commit 745e089 into main Jun 6, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant