What you're trying to do
The read-side kb.list_* methods (kb.list_pages, kb.list_claims,
kb.list_entities, kb.list_relations, kb.list_sources, kb.list_pending)
return the entire collection in one shot. Under the hood storage.list_*
reads and parses every YAML file in the directory, and the JSONL/MCP handlers
hand back the full list with no limit/offset/cursor. On a KB with tens of
thousands of claims this is unbounded memory on the server and an unbounded
payload over MCP/JSONL — the exact scale the perf-benchmark issues (1k/10k/100k)
anticipate. There's no way for a client to page through results.
Suggested shape
- Add
limit: int and cursor: str | None params to every kb.list_* method
(CLI --limit / --cursor too). Default limit from
retrieval.default_limit.
- Response envelope:
{"items": [...], "_meta": {"next_cursor": "<opaque>" | null, "total": <int>}}. The cursor is an opaque, stable token (e.g. the last id in a
sorted-by-id page), so paging is deterministic and resumable.
storage.list_* grows a paged variant that stops reading after limit files
in id order rather than slurping the whole directory.
Acceptance
kb.list_claims --limit 50 returns 50 items + a next_cursor; passing that
cursor returns the next 50 with no overlap and no gaps.
- The last page returns
next_cursor: null.
- A flat-list client gets a clear one-release deprecation note in
_meta (mirrors
how other envelope changes are staged).
- A fuzz test over a 1k-artifact KB asserts full coverage and zero duplicates
across paged iteration.
Out of scope
- Filtering/sorting predicates beyond id-order paging (separate query-language
ask).
- Pagination of
kb.search/kb.context (already limit-bounded by ranking).
What you're trying to do
The read-side
kb.list_*methods (kb.list_pages,kb.list_claims,kb.list_entities,kb.list_relations,kb.list_sources,kb.list_pending)return the entire collection in one shot. Under the hood
storage.list_*reads and parses every YAML file in the directory, and the JSONL/MCP handlers
hand back the full list with no
limit/offset/cursor. On a KB with tens ofthousands of claims this is unbounded memory on the server and an unbounded
payload over MCP/JSONL — the exact scale the perf-benchmark issues (1k/10k/100k)
anticipate. There's no way for a client to page through results.
Suggested shape
limit: intandcursor: str | Noneparams to everykb.list_*method(CLI
--limit/--cursortoo). Defaultlimitfromretrieval.default_limit.{"items": [...], "_meta": {"next_cursor": "<opaque>" | null, "total": <int>}}. The cursor is an opaque, stable token (e.g. the last id in asorted-by-id page), so paging is deterministic and resumable.
storage.list_*grows a paged variant that stops reading afterlimitfilesin id order rather than slurping the whole directory.
Acceptance
kb.list_claims --limit 50returns 50 items + anext_cursor; passing thatcursor returns the next 50 with no overlap and no gaps.
next_cursor: null._meta(mirrorshow other envelope changes are staged).
across paged iteration.
Out of scope
ask).
kb.search/kb.context(alreadylimit-bounded by ranking).