ingest-worker DLQ for malformed payloads + stale skeleton docs + MCP API gaps for council (link_note, project vocab)

[ExtraToast]

Filed from the agent-kit council integration review (read-only research council, 2026-07-03) — these repos are tightly coupled with agent-kit's fanout/supervise runner and knowledge loop.

ingest-worker ACKs malformed payloads (lost, not DLQ'd) + stale docs

consumer.py ACKs malformed payloads (comment claims broker redelivery/DLQ handles it) while handler failures basic_nack(requeue=false). Schema drift from council/agent captures is silently dropped. Residue: ingest-worker/README.md and knowledge_worker/__main__.py docstrings still say "ACK/log skeleton" though __main__.py now wires VaultHandler and handlers.py writes vault pointers.
Action: basic_nack(requeue=false) parse failures with error metadata (or persist poison payloads) + metric/test; update the stale skeleton docs.

MCP API gaps for the council loop

agent-kit's council captures decisions/lessons and wants provenance links. Add: link_note/link_notes for kb_relations writes (supersedes/derived_from/mentions/run+task provenance); project-vocabulary tools mirroring topic tools (list_projects/add_project/merge_projects) so project:<org>/<repo> scoping is first-class.

[jorisjonkers-dev-agents]

Ready to pick up

Three related defects.

---

1. Malformed payloads ACK'd and silently dropped (no DLQ)

Current state (verified in ingest-worker/src/knowledge_worker/consumer.py lines 107-120):
Parse failures (ValidationError, json.JSONDecodeError, UnicodeDecodeError) are caught and basic_ack'd with a log. The comment says "a future change can route these to the dead-letter queue instead." Handler failures at line 131 do basic_nack(requeue=False) — those are lost too. No metrics or persistent record of dropped messages exist.

Expected state:

• Parse failures route to a dead-letter queue: basic_nack(delivery_tag=..., requeue=False) (and configure a DLX on the queue) so poison payloads are visible and recoverable.
• A consumer.parse_failed.count metric (or structured log counter) allows alerting on schema drift.
• A test that publishes a malformed payload and asserts the DLQ receives it (not silence).

Files: ingest-worker/src/knowledge_worker/consumer.py lines 108-120; RabbitMQ queue config (DLX binding).

---

2. Stale skeleton docs

Current state: ingest-worker/README.md and knowledge_worker/__main__.py docstrings still describe the "ACK/log skeleton" pattern, but __main__.py now wires VaultHandler and handlers.py writes vault pointers.

Expected state: Update docs to match the actual wired behavior. No functional change needed.

Files: ingest-worker/README.md, knowledge_worker/__main__.py docstrings.

---

3. MCP API gaps: link_note/link_notes and project-vocabulary tools

Current state: The MCP server has no tool for writing kb_relations (note provenance, supersedes/derived_from/mentions links). It also has no project-vocabulary tools (list_projects, add_project, merge_projects), so project:<org>/<repo> scoping is unmanageable via the API.

Expected state:

• link_note(source_id, target_id, relation_type) / link_notes(relations: list) MCP tools that write kb_relations rows.
• list_projects(), add_project(slug, display_name), merge_projects(from, to) MCP tools mirroring the existing topic tools.

Files: mcp-server/src/ (or wherever MCP tool handlers live); knowledge_api/ route handlers; test fixtures.

[jorisjonkers-dev-agents]

PR #10 (merged) addresses defects 1 and 2 from the pickup comment:

• Parse failures now basic_nack(requeue=False) → routed to knowledge.ingest.dlq via the DLX
• Missing Binding bean added in IngestQueueConfig so the DLX actually delivers to the DLQ
• Integration test added that asserts malformed payloads land on the DLQ
• Stale skeleton docs updated in README.md and __main__.py

Defect 3 (MCP API gaps — link_note/link_notes, list_projects/add_project/merge_projects) is out of scope for this bug fix: it requires schema migrations, new route handlers in knowledge-api, and MCP tool registration — none of which are well-bounded by the existing code. Tracking as a follow-up feature on this issue.