You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+37-17Lines changed: 37 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,6 +24,16 @@ Traditional RAG rediscovers knowledge from scratch on every query. Nothing accum
24
24
25
25
OpenKB has two layers: a **wiki foundation** that compiles and maintains your knowledge, and **generators** (query / chat / Skill Factory) that turn it into useful output. See [Usage](#️-usage) for the full command list.
26
26
27
+
### Features
28
+
29
+
-**Broad format support** — PDF, Word, Markdown, PowerPoint, HTML, Excel, CSV, text, URLs, and more
30
+
-**Scale to long documents** — Long and complex documents are handled via [PageIndex](https://github.com/VectifyAI/PageIndex) tree indexing, enabling accurate, vectorless context-aware retrieval
31
+
-**Native multi-modality** — Retrieves and understands figures, tables, and images, not just text
32
+
-**Compiled Wiki** — LLM compiles your documents into summaries, concept pages, entity pages, and cross-links, all kept in sync
33
+
-**Query & Chat** — One-off questions or multi-turn conversations over your wiki, with persisted sessions you can resume
34
+
-**Skill Factory** — Distill redistributable agent skills from your wiki
35
+
-**Obsidian compatible** — Wiki is plain `.md` files with `[[wikilinks]]`. Open in Obsidian for graph view and browsing
36
+
27
37
# 🚀 Getting Started
28
38
29
39
### Install
@@ -71,8 +81,8 @@ openkb query "What are the main findings?"
71
81
# 5. Or chat interactively
72
82
openkb chat
73
83
74
-
#6. Or distill your wiki into a redistributable skill
75
-
openkb skill new my-expert "Reason like an expert on <topic-from-your-docs>"
84
+
#(Optional) Distill a redistributable agent skill from your wiki
85
+
openkb skill new my-expert "Reason like an expert on <your-topic>"
76
86
```
77
87
78
88
### Set up your LLM
@@ -108,17 +118,16 @@ wiki/ │ ← the foundation
108
118
├── AGENTS.md Wiki schema (LLM instructions)
109
119
├── sources/ Full-text conversions
110
120
├── summaries/ Per-document summaries
111
-
├── concepts/ Cross-document synthesis ← the good stuff
121
+
├── concepts/ Cross-document synthesis
112
122
├── entities/ Specific named things (people, orgs, places, products)
113
123
├── explorations/ Saved query results
114
124
└── reports/ Lint reports
115
125
│
116
126
┌──────────────────────┼──────────────────────┐
117
127
▼ ▼ ▼
118
128
query / chat Skill Factory (future)
119
-
(LLM answers from openkb skill new ppt / podcast /
120
-
the wiki) → output/skills/ report / …
121
-
+ marketplace.json
129
+
(LLM answers from (redistributable ppt / podcast /
130
+
the wiki) agent skills) report / …
122
131
```
123
132
124
133
### Short vs. Long Document Handling
@@ -155,7 +164,7 @@ OpenKB commands fall into two layers: the **wiki foundation** (compile + manage
155
164
|`openkb init`| Initialize a new knowledge base (interactive) |
156
165
| <code>openkb add <file_or_dir_or_URL></code> | Add documents and compile to wiki. URL ingest auto-detects PDF (saved as `.pdf` → PageIndex / markitdown) vs HTML (trafilatura main-content extract → `.md`) |
157
166
| <code>openkb remove <doc></code> | Remove a document and clean up its wiki pages, images, registry, and PageIndex state (use `--dry-run` to preview, `--keep-raw` / `--keep-empty` to retain artifacts) |
158
-
| <code>openkb recompile [<doc>] [--all]</code> | Re-run the current compile pipeline on already-indexed docs (e.g. to backfill the `entities/` layer) without re-indexing. Regenerates summaries and rewrites concept pages — manual edits are overwritten. Use `--dry-run` to preview, `--refresh-schema` to also update `wiki/AGENTS.md`|
167
+
| <code>openkb recompile [<doc>] [--all]</code> | Re-run the current compile pipeline on already-indexed docs (e.g. to backfill the `entities/` layer) without re-indexing. Regenerates summaries and rewrites concept pages; manual edits are overwritten. Use `--dry-run` to preview, `--refresh-schema` to also update `wiki/AGENTS.md`|
159
168
|`openkb watch`| Watch `raw/` and auto-compile new files |
160
169
|`openkb lint`| Run structural + knowledge health checks |
161
170
|`openkb list`| List indexed documents and concepts |
@@ -174,8 +183,8 @@ A "generator" reads from the compiled wiki and produces something usable: an ans
174
183
|`openkb chat`| Interactive multi-turn session over the wiki (use `--resume`, `--list`, `--delete` to manage sessions) |
175
184
| <code>openkb skill new <name> "<intent>"</code> | A redistributable Anthropic Skill at `<kb>/output/skills/<name>/` + auto-updated `marketplace.json`|
176
185
| <code>openkb skill validate [name]</code> | Structural lint of compiled skills (frontmatter, file sizes, wikilinks, scripts/ stdlib check with `--strict`). Auto-runs at end of `skill new`|
177
-
| <code>openkb skill eval <name></code> | Trigger-accuracy evaluation — does the `description:` field actually fire? LLM generates eval prompts; grader LLM scores activation. `--save` persists the eval set |
178
-
| <code>openkb skill history <name></code> / <code>openkb skill rollback <name></code> | Iteration workspace — every overwrite saves the previous version to `output/skills/<name>-workspace/iteration-N/` with a structural diff. Rollback restores any iteration |
186
+
| <code>openkb skill eval <name></code> | Trigger-accuracy evaluation: does the `description:` field actually fire? LLM generates eval prompts; grader LLM scores activation. `--save` persists the eval set |
187
+
| <code>openkb skill history <name></code> / <code>openkb skill rollback <name></code> | Iteration workspace: every overwrite saves the previous version to `output/skills/<name>-workspace/iteration-N/` with a structural diff. Rollback restores any iteration |
179
188
180
189
### Query & Chat — ask the wiki
181
190
@@ -205,7 +214,7 @@ Inside a chat, type `/` to access slash commands (Tab to complete):
205
214
206
215
### 🛠 Skill Factory — *Drop in a book. Out comes a digital expert.*
207
216
208
-
The newest generator. `openkb skill new` distills any subset of your wiki into an [Anthropic Skill](https://docs.claude.com/en/docs/build-with-claude/skills)— a portable folder that **Claude Code, Codex CLI, Gemini CLI, and Cursor** all install and load natively. Drop in a book's worth of papers; out comes a specialist that other agents can call on.
217
+
The newest generator. `openkb skill new` distills an [agent skill](https://docs.claude.com/en/docs/build-with-claude/skills)from any subset of your wiki, a portable folder that major agents (Claude Code, Codex, etc.) install and load natively. Drop in a book's worth of papers; out comes a specialist that other agents can call on.
209
218
210
219
```bash
211
220
openkb skill new karpathy-thinking \
@@ -281,7 +290,7 @@ extra_headers:
281
290
Copilot-Integration-Id: vscode-chat
282
291
```
283
292
284
-
Subscription-based providers that authenticate via OAuth device flow (e.g. `chatgpt/*`, `github_copilot/*`) need no API key — OpenKB skips the missing-key warning for them.
293
+
Subscription-based providers that authenticate via OAuth device flow (e.g. `chatgpt/*`, `github_copilot/*`) need no API key; OpenKB skips the missing-key warning for them.
285
294
286
295
Model names use `provider/model` LiteLLM [format](https://docs.litellm.ai/docs/providers) (OpenAI models can omit the prefix):
287
296
@@ -294,7 +303,7 @@ Model names use `provider/model` LiteLLM [format](https://docs.litellm.ai/docs/p
294
303
### PageIndex Integration
295
304
296
305
Long documents are challenging for LLMs due to context limits, context rot, and summarization loss.
297
-
[PageIndex](https://github.com/VectifyAI/PageIndex) solves this with vectorless, reasoning-based retrieval — building a hierarchical tree index that lets LLMs reason over the index for context-aware retrieval.
306
+
[PageIndex](https://github.com/VectifyAI/PageIndex) solves this with vectorless, reasoning-based retrieval, building a hierarchical tree index that lets LLMs reason over the index for context-aware retrieval.
298
307
299
308
PageIndex runs locally by default using the [open-source version](https://github.com/VectifyAI/PageIndex), with no external dependencies required.
300
309
@@ -329,7 +338,7 @@ OpenKB's wiki is a directory of Markdown files with `[[wikilinks]]`. Obsidian re
329
338
330
339
### Using with Claude Code / Codex / Gemini CLI
331
340
332
-
OpenKB ships a `SKILL.md` so any agent CLI can read your compiled wiki — no extra runtime, no MCP setup, just install the skill once.
341
+
OpenKB ships a `SKILL.md` so any agent CLI can read your compiled wiki. No extra runtime, no MCP setup, just install the skill once.
333
342
334
343
**Claude Code**:
335
344
@@ -344,15 +353,15 @@ OpenKB ships a `SKILL.md` so any agent CLI can read your compiled wiki — no ex
The skill is read-only — it won't run `openkb add`, `remove`, or `lint --fix` without you asking. See [`skills/openkb/SKILL.md`](skills/openkb/SKILL.md) for the full instruction set.
364
+
The skill is read-only; it won't run `openkb add`, `remove`, or `lint --fix` without you asking. See [`skills/openkb/SKILL.md`](skills/openkb/SKILL.md) for the full instruction set.
356
365
357
366
# 🧭 Learn More
358
367
@@ -362,9 +371,11 @@ The skill is read-only — it won't run `openkb add`, `remove`, or `lint --fix`
0 commit comments