DevRAG

A local RAG system for developer teams that ingests code, GitHub PRs, GitHub issues, Jira Cloud tickets, Slite pages, Slack conversations, documents, and Claude Code session logs — surfaced through Claude Code slash commands and a standalone CLI. Designed for zero-friction adoption: index your repo, search immediately.

Why DevRAG?

Most code search tools treat source files as plain text. DevRAG uses tree-sitter AST parsing to chunk code along semantic boundaries (functions, classes, methods), so searches return meaningful units instead of fragments split mid-function.

It combines vector search (semantic) with BM25 (keyword) via Qdrant's server-side Reciprocal Rank Fusion, which is critical for code where exact identifiers like refreshToken matter as much as the concept of "token refresh." Both signals live in Qdrant as named vectors on the same points (dense + bm25), so search is a single round trip per collection and metadata filters apply equally to both legs.

PR history, issues, Jira tickets, and Slite wiki pages are first-class data sources — encoding why code changed, what bugs and features are tracked, what project management context exists, and what internal knowledge base content says.

Quick Start

Install

Install directly from GitHub (no clone required):

# As a global CLI tool (recommended)
uv tool install git+https://github.com/tomharris/dev-rag.git

# Or run without installing
uvx --from git+https://github.com/tomharris/dev-rag.git devrag --help

# Or add as a project dependency
uv add git+https://github.com/tomharris/dev-rag.git
# pip install git+https://github.com/tomharris/dev-rag.git

For development (from source):

git clone https://github.com/tomharris/dev-rag.git
cd dev-rag
uv sync
uv run devrag --help

Index and Search

# Index the current repository
devrag index repo .

# Search across code, PRs, issues, and docs
devrag search "how does authentication work"

# Check what's indexed
devrag status

Requirements

Python 3.12+
Ollama running locally with nomic-embed-text pulled:
```
ollama pull nomic-embed-text
```

Models on a network that blocks Hugging Face

DevRAG's reranker and BM25 models come from Hugging Face. On first use it auto-downloads a self-hosted bundle of both models from a dev-rag GitHub release (reachable even when huggingface.co is blocked) and caches them locally; later runs load offline. To fetch them explicitly:

devrag download-models          # fetch/refresh the model bundle
devrag download-models --force  # re-download even if cached

Overrides (in ~/.config/devrag/devrag.yaml or .devrag.yaml):

network:
  auto_download_models: true            # set false to disable auto-fetch (CI/air-gapped)
  model_bundle_url: ""                  # internal/air-gapped mirror of the bundle
  model_bundle_sha256: ""               # expected checksum for a custom model_bundle_url
sparse_embedding:
  cache_dir: ""                         # FastEmbed cache; "" = ~/.cache/devrag/fastembed

Note: the bundle carries the default model names. If you override retrieval.reranker_model or sparse_embedding.model, you need Hugging Face (or a faithful mirror) access for your chosen model.

CLI Reference

Search

devrag search "query"                      # Search all sources
devrag search "query" --scope code         # Code only
devrag search "query" --scope prs          # PR history only
devrag search "query" --scope issues       # Issues only
devrag search "query" --scope slite        # Slite pages only
devrag search "query" --scope slack        # Slack conversations only
devrag search "query" --scope docs         # Documents only
devrag search "query" --scope sessions     # Claude Code session logs only
devrag search "query" --top-k 10           # More results

# Narrow with metadata filters (all optional, AND-combined)
devrag search "auth" --repo myrepo --chunk-type diff
devrag search "bug" --pr-number 1234
devrag search "refund flow" --ticket-key PROJ-123
devrag search "deploy" --session-id <uuid>
devrag search "incident" --channel C0123ABCD   # Slack channel id

The query router automatically classifies intent and targets relevant collections. "Why did we switch to Redis?" routes to PR history; "is there a bug with login?" routes to issues and Jira tickets; "what sprint is auth in?" routes to Jira; "what does our wiki say about deploys?" routes to Slite pages and documents; "how does the auth middleware work?" routes to code.

Indexing

# Code (AST-aware, incremental by default; also indexes the repo's docs)
devrag index repo .                        # Current directory (code + docs)
devrag index repo /path/to/repo            # Specific path
devrag index repo . --full                 # Force full re-index
devrag index repo . --name my-repo         # Name for multi-repo support
devrag index repo . --no-docs              # Code only, skip the repo's docs
devrag index remove-repo my-repo           # Remove a repo from the index (code + docs)
devrag index refresh                       # Re-index ALL registered repos (incremental; code + docs)
devrag index refresh --full                # Full rebuild of every registered repo (external sources untouched)

# `index repo` also indexes the repo's own docs (md/mdx/txt/rst/html/adoc) into
# the documents collection, tagged with the repo name (so `search --repo <name>`
# narrows to them). Use `index docs` below only for a standalone docs tree.

# Documents (section-aware splitting; standalone directory, not tied to a repo)
devrag index docs ./specs                  # Default: **/*.md
devrag index docs ./docs --glob "**/*.txt,**/*.rst"

# GitHub PRs (incremental with cursor tracking)
export GITHUB_TOKEN=ghp_xxx
devrag index prs owner/repo                # Incremental from cursor (90d on first run)
devrag index prs owner/repo --since 180d   # Override cursor for backfill

# GitHub Issues (incremental, skips PRs)
devrag index issues owner/repo             # Last 90 days
devrag index issues owner/repo --since 30d # Custom lookback

# Jira Cloud tickets (incremental via JQL)
export JIRA_EMAIL=you@company.com
export JIRA_TOKEN=your-api-token
devrag index jira                          # Uses JQL from .devrag.yaml
devrag index jira --since 180d             # Custom lookback

# Slite pages (incremental, section-aware chunking)
export SLITE_TOKEN=your-api-token
devrag index slite                         # Last 90 days, all channels
devrag index slite --since 30d             # Custom lookback

# Slack conversations — NO Slack App required (uses your browser session)
export SLACK_XOXC_TOKEN=xoxc-...           # web-client token (see "Slack without an app")
export SLACK_XOXD_COOKIE=xoxd-...          # the `d` cookie value
devrag index slack                         # Public channels you're in, last 90 days
devrag index slack --since 30d             # Custom first-sync lookback

# Claude Code session logs (local JSONL, incremental via file mtime)
devrag index sessions                      # Incremental from cursor (60d first run)
devrag index sessions --since 30d          # Override cursor
devrag index sessions --logs-dir /custom/path

Slack without an app

Most Slack integrations require creating a Slack App and having a workspace admin install it with OAuth scopes. DevRAG skips that entirely: it authenticates as you, the logged-in user, using the same credentials your browser already holds — so no app, no admin approval, and it indexes exactly what you can already see.

Get the credentials automatically (recommended; you just need to be logged in to the Slack desktop app or a browser):

eval "$(devrag auth slack --workspace mycorp)"      # mycorp = the <x> in https://<x>.slack.com
devrag index slack

devrag auth slack reads both credentials from the Slack desktop app by default — the d cookie from its cookie store and the xoxc token from its localStorage (where Slack now keeps the token; it's no longer in any web page) — validates the pair, and prints the two export lines, so eval "$(…)" sets SLACK_XOXC_TOKEN / SLACK_XOXD_COOKIE in your shell. The desktop app is a Chromium app, so its cookie store decrypts with the same machinery used for browsers (no extra setup) and its localStorage is read with a bundled pure-python LevelDB reader — both work even while Slack is running (the stores are copied first; nothing is written to disk by devrag). Set slack.workspace in .devrag.yaml to skip --workspace on re-runs.

--browser selects the source explicitly: slack (the desktop app) or chrome/firefox/brave/edge/chromium. With no --browser, devrag tries the Slack desktop app first, then sweeps every supported browser, skipping any whose profile it can't read — so an unsupported/missing app or browser doesn't abort the search. The cookie must live on the machine running the command — if you're logged in only on a different machine, or your app/browser stores its profile in a non-standard location, use --browser or the manual fallback below.

Manual fallback — if cookie decryption fails (locked keyring / Chrome app-bound encryption)

Extract the two credentials from your browser (while logged in to Slack on the web):

Open Slack in your browser → DevTools (F12) → Console.
Token (xoxc-…): run JSON.parse(localStorage.localConfig_v2).teams[Object.keys(JSON.parse(localStorage.localConfig_v2).teams)[0]].token (or find xoxc- under Application → Local Storage).
Cookie (xoxd-…): DevTools → Application → Cookies → https://app.slack.com → copy the value of the cookie named d (URL-decode %2F→/ etc. if needed).
export SLACK_XOXC_TOKEN=xoxc-… and export SLACK_XOXD_COOKIE=xoxd-…, then devrag index slack.

Scope: public channels you belong to by default. To index only specific channels (of any type), set an allowlist in .devrag.yaml under slack.channel_ids. Threads are indexed as single chunks; loose channel chatter is grouped into time-window chunks.

⚠️ Caveats. These are personal session credentials, not an official API token, and using Slack's internal web API is against a strict reading of Slack's Terms of Service — use at your own risk on workspaces where you have permission to read the content. The token/cookie expire when your browser session rotates or you log out; when that happens devrag index slack fails with a clear "re-extract from browser" error rather than silently indexing nothing. Credentials are read from env vars only and never stored in config files.

Code indexing is incremental — file content hashes are tracked in SQLite, so unchanged files are skipped on re-index.

Other Commands

devrag status                              # Show index stats
devrag serve                               # Start MCP server for Claude Code
devrag reindex --all                       # Clear all indexes and re-embed code repos + their docs (re-sync PRs/issues/Jira/Slite/Slack manually)
devrag reindex --name my_project           # Re-index a single repo's code + docs (preserves other repos and external sources)
devrag config set embedding.model nomic-embed-text
devrag config get vector_store.backend

Evaluation

Test retrieval quality with a JSONL file of queries and expected results:

devrag eval run test_queries.jsonl --output results.jsonl --top-k 5
devrag eval compare results_v1.jsonl results_v2.jsonl

Query file format:

{"query": "How does rate limiting work?", "expected_files": ["src/middleware/rate_limit.ts"]}
{"query": "Why did we migrate to Redis?", "expected_prs": [1234, 1301]}

Metrics: precision@k, recall@k, and MRR (Mean Reciprocal Rank).

Claude Code Integration

DevRAG ships as a FastMCP server that integrates directly with Claude Code.

Setup

# If installed globally via `uv tool install`:
claude mcp add devrag -- devrag serve

# Or run on demand without installing:
claude mcp add devrag -- uvx --from git+https://github.com/tomharris/dev-rag.git devrag serve

Skills

The repo includes Claude Code skills in .claude/skills/:

Skill	Description
`/rag-search <query>`	Search code, PRs, and docs with results grouped by source type
`/rag-index [path]`	Index a repo, docs directory, or sync PRs
`/rag-pr <query>`	Search PR history for why code changed
`rag-first` (auto)	Auto-triggers on codebase questions to search DevRAG before Grep/Glob/Explore

The rag-first skill fires automatically when Claude detects a codebase question — no slash command needed.

For a hard safety net, the repo ships .claude/hooks/rag_first_gate.py — a per-turn block gate that denies the first Grep, Glob, or Agent(Explore/general-purpose) call of a turn until mcp__devrag__search has run, then releases for the rest of the turn. The script is tracked, but hook registration is per-user. To enable it, add this hooks block to your .claude/settings.local.json (alongside permissions):

"hooks": {
  "PreToolUse": [
    {
      "matcher": "Grep|Glob|Agent",
      "hooks": [{
        "type": "command",
        "command": "python3 ${CLAUDE_PROJECT_DIR}/.claude/hooks/rag_first_gate.py",
        "timeout": 5
      }]
    }
  ],
  "PostToolUse": [
    {
      "matcher": "mcp__devrag__search",
      "hooks": [{
        "type": "command",
        "command": "python3 ${CLAUDE_PROJECT_DIR}/.claude/hooks/rag_first_gate.py",
        "timeout": 5
      }]
    }
  ],
  "UserPromptSubmit": [
    {
      "matcher": "",
      "hooks": [{
        "type": "command",
        "command": "python3 ${CLAUDE_PROJECT_DIR}/.claude/hooks/rag_first_gate.py",
        "timeout": 5
      }]
    }
  ]
}

Restart your Claude Code session after editing so the hooks register.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                     DEVELOPER INTERFACES                            │
│  CLI (Typer)                          MCP Server (FastMCP)         │
│  devrag search|index|status           Claude Code slash commands    │
└────────────┬──────────────────────────────────┬─────────────────────┘
             │                                  │
             ▼                                  ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     RETRIEVAL LAYER                                 │
│  Query Router ──→ Hybrid Search (server-side dense+BM25 RRF) ──→ Reranker│
│  (intent → collections)                          (cross-encoder)   │
│       └─→ repo-preference boost ──→ dedup per source ──→ final_k    │
└────────────┬────────────────────────────────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     STORAGE LAYER                                   │
│  QdrantStore (HTTP or embedded local path)                          │
│    points carry named vectors: {dense, bm25 sparse}                 │
│  SQLite: file hashes, chunk mappings, sync cursors, metrics         │
└────────────┬────────────────────────────────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     INGESTION LAYER                                 │
│  Code     PR       Issue    Jira     Slite     Doc                   │
│  Indexer  Indexer  Indexer  Indexer  Indexer   Indexer                │
│  (AST)   (GitHub) (GitHub) (Cloud)  (REST)   (sections)             │
│                                                                     │
│  Ollama Embedder (nomic-embed-text, 768d)                          │
└─────────────────────────────────────────────────────────────────────┘

Collections

Collection	Source	Contents
`code_chunks`	Code indexer	AST-extracted functions, classes, methods
`pr_diffs`	PR indexer	PR descriptions and file-level diff hunks
`pr_discussions`	PR indexer	Review comments and threads
`issue_descriptions`	Issue indexer	Issue titles and bodies
`issue_discussions`	Issue indexer	Issue comments
`jira_descriptions`	Jira indexer	Jira ticket summaries and descriptions
`jira_discussions`	Jira indexer	Jira ticket comments
`slite_pages`	Slite indexer	Slite page sections (markdown)
`slack_messages`	Slack indexer	Slack threads and time-window conversations
`documents`	Doc indexer	Markdown/text sections
`session_logs`	Sessions indexer	Claude Code user↔assistant exchanges from local JSONL

Configuration

DevRAG uses YAML configuration with two layers (project overrides user):

User config: ~/.config/devrag/devrag.yaml
Project config: .devrag.yaml (commit this to share with your team)

embedding:
  model: nomic-embed-text        # Embedding model name
  provider: ollama               # ollama or sentence-transformers
  ollama_url: http://localhost:11434
  batch_size: 64
  max_tokens: 8192                   # Truncation limit for embedding context

vector_store:
  qdrant_url: http://localhost:6333  # Qdrant HTTP endpoint
  qdrant_path: ""                    # optional: local filesystem path for embedded mode (no server needed)
  embedding_dim: 768                 # Vector dimensionality

retrieval:
  top_k: 20                     # Candidates before reranking
  final_k: 5                    # Results after reranking (dedup runs on the full pool first)
  rerank: true
  reranker_model: cross-encoder/ms-marco-MiniLM-L-6-v2
  reranker_max_length: 512      # Cross-encoder token limit; raise for longer chunks
  max_per_source: 2             # Max results kept per file/PR/issue/etc.
  repo_boost: 0.15              # Soft preference for the cwd repo (fraction of score spread; 0 = off)

code:
  chunk_max_tokens: 512
  respect_gitignore: true
  exclude_patterns:
    - "*.min.js"
    - "vendor/**"
    - "node_modules/**"
    - "*.lock"
    - "*.generated.*"

prs:
  github_token_env: GITHUB_TOKEN  # Env var name (token never stored in config)
  backfill_days: 90
  include_draft: false
  chunk_max_tokens: 512              # Max tokens per PR chunk

issues:
  github_token_env: GITHUB_TOKEN  # Reuses same token as PRs
  backfill_days: 90
  chunk_max_tokens: 512
  include_labels: []              # Only index issues with these labels (empty = all)
  exclude_labels: ["wontfix"]     # Skip issues with these labels

jira:
  jira_token_env: JIRA_TOKEN      # Env var for API token
  jira_email_env: JIRA_EMAIL      # Env var for email (basic auth)
  instance_url: https://yoursite.atlassian.net
  jql: "project = DEV"            # JQL filter for scoping tickets
  backfill_days: 90
  chunk_max_tokens: 512

slite:
  slite_token_env: SLITE_TOKEN    # Env var for API token
  channel_ids: []                 # Filter to specific channels (empty = all)
  backfill_days: 90
  chunk_max_tokens: 512
  chunk_overlap_tokens: 50

slack:
  slack_token_env: SLACK_XOXC_TOKEN   # Env var for the xoxc web-client token
  slack_cookie_env: SLACK_XOXD_COOKIE # Env var for the xoxd `d` cookie
  channel_ids: []                     # Allowlist of channel ids (empty = all public channels you're in)
  gap_minutes: 30                     # Quiet gap that starts a new conversation window
  backfill_days: 90
  chunk_max_tokens: 512
  chunk_overlap_tokens: 50

sessions:
  logs_dir: ~/.claude/projects    # Claude Code JSONL log directory
  backfill_days: 60
  chunk_max_tokens: 512

documents:
  glob_patterns: ["**/*.md", "**/*.mdx", "**/*.txt", "**/*.rst", "**/*.html", "**/*.adoc"]
  chunk_max_tokens: 512
  chunk_overlap_tokens: 50

network:
  ca_bundle: ""                   # PEM CA bundle for all outbound HTTPS (Slack/Jira/Slite/GitHub)

Corporate proxies (TLS interception)

If you're behind a TLS-intercepting corporate proxy, devrag auth slack, index prs/issues/jira/slite/slack, and their MCP equivalents may fail with:

[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain

httpx verifies against the bundled certifi CA list and does not consult the macOS keychain or honor SSL_CERT_FILE on its own, so the proxy's injected root CA is unknown to it. Point DevRAG at a PEM bundle containing that root CA, either via config:

network:
  ca_bundle: ~/corp-ca.pem

or via the standard env vars (honored as a fallback when ca_bundle is empty):

export REQUESTS_CA_BUNDLE=~/corp-ca.pem   # or SSL_CERT_FILE

To export your proxy's root CA from the macOS keychain to a PEM:

security find-certificate -a -p -c "<your proxy CA name>" \
  /Library/Keychains/System.keychain > ~/corp-ca.pem

(or in Keychain Access → select the root CA → File → Export Items → .pem). The Ollama embedder talks to localhost and is unaffected.

File Exclusion

DevRAG respects .gitignore by default. For additional exclusions specific to the index, create a .devragignore file. It uses full .gitignore syntax — directory patterns (docs/internal/), anchoring, **, and ! negation all work — and applies to both code and docs indexing. Unlike .gitignore, it can also exclude files that are tracked by git.

Deployment Modes

DevRAG uses Qdrant as its vector store. You can run it either embedded (local filesystem, no server needed) or against a Qdrant server.

Embedded mode (single-user, no server):

vector_store:
  qdrant_path: ~/.local/share/devrag/qdrant

Server mode (shared, scales horizontally):

docker run -d -p 6333:6333 -v qdrant_data:/qdrant/storage qdrant/qdrant

vector_store:
  qdrant_url: http://localhost:6333

Switching modes is a config change; re-index with devrag reindex --all after switching.

Development

git clone https://github.com/tomharris/dev-rag.git
cd dev-rag
uv sync --extra dev

# Run tests
uv run pytest tests/

# Run a single test
uv run pytest tests/test_config.py::test_load_config -v

# Integration tests (requires Ollama running with nomic-embed-text)
SKIP_INTEGRATION=0 uv run pytest tests/test_integration.py -v

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 152 Commits
.claude		.claude
devrag		devrag
docs		docs
scripts		scripts
tests		tests
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
README.md		README.md
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

DevRAG

Why DevRAG?

Quick Start

Install

Index and Search

Requirements

Models on a network that blocks Hugging Face

CLI Reference

Search

Indexing

Slack without an app

Other Commands

Evaluation

Claude Code Integration

Setup

Skills

Architecture

Collections

Configuration

Corporate proxies (TLS interception)

File Exclusion

Deployment Modes

Development

License

About

Uh oh!

Releases 1

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

DevRAG

Why DevRAG?

Quick Start

Install

Index and Search

Requirements

Models on a network that blocks Hugging Face

CLI Reference

Search

Indexing

Slack without an app

Other Commands

Evaluation

Claude Code Integration

Setup

Skills

Architecture

Collections

Configuration

Corporate proxies (TLS interception)

File Exclusion

Deployment Modes

Development

License

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Releases 1

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages