Skip to content

feat: codebase embedding index for semantic search #25

Description

@alpibrupa

Problem

All navigation is grep + glob — pattern matching on names. On a large codebase, explore/plan quality is capped because the agent cannot find functions by semantic intent (e.g. "functions that validate A2A envelopes").

Design

New tool: semantic_search

semantic_search(query: Str, top_k: Int) -> List[SearchResult]

Each result: file path, function name, SigId, relevance score, one-line summary.

Implementation requires:

  1. Index builder: walk all .lex files, extract function signatures + docstrings + examples {} blocks, embed via a local embedding model (or the configured LLM provider's embedding endpoint)
  2. Store embeddings in .lex/index.db (sqlite-vec or similar)
  3. On query: embed query, ANN search, return top_k

The index should be lazily rebuilt when source files change (compare mtime or SigId against stored hash).

Dependencies

  • Check if lex-llm exposes an embedding API (prov.embed(text) -> Vec[Float])
  • If not, file a companion ticket in lex-llm

Acceptance

  • semantic_search "validate A2A envelope" returns relevant functions
  • Available to explore, plan, review agents
  • Index rebuild is incremental (unchanged files not re-embedded)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions