feat: add lat.md support to the codebase (#279)

* feat: add lat.md support to the codebase

* Fix ty errors for optional Rust backend and test typing

* fix: tests

* fix: python 3.15 alpha 7

* fix: make version latest

Author: vinitkumar

.coverage

@@ -37,10 +37,11 @@ jobs:
         uses: actions/setup-python@v6
         with:
           python-version: '3.13'
-      - name: Install uv
-        uses: astral-sh/setup-uv@v6
+      - name: Install the latest version of uv
+        uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
         with:
           enable-cache: true
+          version: "latest"
       - name: Install dependencies
         run: |
           uv venv

.github/workflows/lint.yml

@@ -28,7 +28,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [pypy-3.10, pypy-3.11, '3.10', '3.11', '3.12', '3.13', '3.14', '3.14t', '3.15.0-alpha.3']
+        python-version: [pypy-3.10, pypy-3.11, '3.10', '3.11', '3.12', '3.13', '3.14', '3.14t', '3.15.0-alpha.7']
         os: [
           ubuntu-latest,
           windows-latest,

.github/workflows/pythonpackage.yml

@@ -135,3 +135,6 @@ rust/target/
 Cargo.lock
 *.rlib
 *.rmeta
+.codex
+.pi
+.cursor

.gitignore

@@ -0,0 +1,105 @@
+# Before starting work
+
+- Run `lat search` to find sections relevant to your task. Read them to understand the design intent before writing code.
+- Run `lat expand` on user prompts to expand any `[[refs]]` — this resolves section names to file locations and provides context.
+
+# Post-task checklist (REQUIRED — do not skip)
+
+After EVERY task, before responding to the user:
+
+- [ ] Update `lat.md/` if you added or changed any functionality, architecture, tests, or behavior
+- [ ] Run `lat check` — all wiki links and code refs must pass
+- [ ] Do not skip these steps. Do not consider your task done until both are complete.
+
+---
+
+# What is lat.md?
+
+This project uses [lat.md](https://www.npmjs.com/package/lat.md) to maintain a structured knowledge graph of its architecture, design decisions, and test specs in the `lat.md/` directory. It is a set of cross-linked markdown files that describe **what** this project does and **why** — the domain concepts, key design decisions, business logic, and test specifications. Use it to ground your work in the actual architecture rather than guessing.
+
+# Commands
+
+```bash
+lat locate "Section Name"      # find a section by name (exact, fuzzy)
+lat refs "file#Section"        # find what references a section
+lat search "natural language"  # semantic search across all sections
+lat expand "user prompt text"  # expand [[refs]] to resolved locations
+lat check                      # validate all links and code refs
+```
+
+Run `lat --help` when in doubt about available commands or options.
+
+If `lat search` fails because no API key is configured, explain to the user that semantic search requires a key provided via `LAT_LLM_KEY` (direct value), `LAT_LLM_KEY_FILE` (path to key file), or `LAT_LLM_KEY_HELPER` (command that prints the key). Supported key prefixes: `sk-...` (OpenAI) or `vck_...` (Vercel). If the user doesn't want to set it up, use `lat locate` for direct lookups instead.
+
+# Syntax primer
+
+- **Section ids**: `lat.md/path/to/file#Heading#SubHeading` — full form uses project-root-relative path (e.g. `lat.md/tests/search#RAG Replay Tests`). Short form uses bare file name when unique (e.g. `search#RAG Replay Tests`, `cli#search#Indexing`).
+- **Wiki links**: `[[target]]` or `[[target|alias]]` — cross-references between sections. Can also reference source code: `[[src/foo.ts#myFunction]]`.
+- **Source code links**: Wiki links in `lat.md/` files can reference functions, classes, constants, and methods in TypeScript/JavaScript/Python/Rust/Go/C files. Use the full path: `[[src/config.ts#getConfigDir]]`, `[[src/server.ts#App#listen]]` (class method), `[[lib/utils.py#parse_args]]`, `[[src/lib.rs#Greeter#greet]]` (Rust impl method), `[[src/app.go#Greeter#Greet]]` (Go method), `[[src/app.h#Greeter]]` (C struct). `lat check` validates these exist.
+- **Code refs**: `// @lat: [[section-id]]` (JS/TS/Rust/Go/C) or `# @lat: [[section-id]]` (Python) — ties source code to concepts
+
+# Test specs
+
+Key tests can be described as sections in `lat.md/` files (e.g. `tests.md`). Add frontmatter to require that every leaf section is referenced by a `// @lat:` or `# @lat:` comment in test code:
+
+```markdown
+---
+lat:
+  require-code-mention: true
+---
+# Tests
+
+Authentication and authorization test specifications.
+
+## User login
+
+Verify credential validation and error handling for the login endpoint.
+
+### Rejects expired tokens
+Tokens past their expiry timestamp are rejected with 401, even if otherwise valid.
+
+### Handles missing password
+Login request without a password field returns 400 with a descriptive error.
+```
+
+Every section MUST have a description — at least one sentence explaining what the test verifies and why. Empty sections with just a heading are not acceptable. (This is a specific case of the general leading paragraph rule below.)
+
+Each test in code should reference its spec with exactly one comment placed next to the relevant test — not at the top of the file:
+
+```python
+# @lat: [[tests#User login#Rejects expired tokens]]
+def test_rejects_expired_tokens():
+    ...
+
+# @lat: [[tests#User login#Handles missing password]]
+def test_handles_missing_password():
+    ...
+```
+
+Do not duplicate refs. One `@lat:` comment per spec section, placed at the test that covers it. `lat check` will flag any spec section not covered by a code reference, and any code reference pointing to a nonexistent section.
+
+# Section structure
+
+Every section in `lat.md/` **must** have a leading paragraph — at least one sentence immediately after the heading, before any child headings or other block content. The first paragraph must be ≤250 characters (excluding `[[wiki link]]` content). This paragraph serves as the section's overview and is used in search results, command output, and RAG context — keeping it concise guarantees the section's essence is always captured.
+
+```markdown
+# Good Section
+
+Brief overview of what this section documents and why it matters.
+
+More detail can go in subsequent paragraphs, code blocks, or lists.
+
+## Child heading
+
+Details about this child topic.
+```
+
+```markdown
+# Bad Section
+
+## Child heading
+
+Details about this child topic.
+```
+
+The second example is invalid because `Bad Section` has no leading paragraph. `lat check` validates this rule and reports errors for missing or overly long leading paragraphs.

AGENTS.md

@@ -60,6 +60,7 @@
 EMAIL = "mail@vinitkumar.me"
 
 
+# @lat: [[architecture#CLI entrypoint]]
 def create_parser() -> argparse.ArgumentParser:
     """Create and configure the argument parser."""
     parser = argparse.ArgumentParser(
@@ -228,6 +229,7 @@ def create_parser() -> argparse.ArgumentParser:
     return parser
 
 
+# @lat: [[behavior#Input readers]]
 def read_input(args: argparse.Namespace) -> dict[str, Any] | list[Any]:
     """
     Read JSON input from the specified source.

json2xml/cli.py

@@ -72,7 +72,7 @@ def get_unique_id(element: str) -> str:
 ]
 
 
-def get_xml_type(val: ELEMENT) -> str:
+def get_xml_type(val: Any) -> str:
     """
     Get the XML type of a given value.
 
@@ -227,6 +227,7 @@ def get_xpath31_tag_name(val: Any) -> str:
     return "string"
 
 
+# @lat: [[behavior#XPath 3.1 format]]
 def convert_to_xpath31(obj: Any, parent_key: str | None = None) -> str:
     """
     Convert a Python object to XPath 3.1 json-to-xml format.
@@ -267,7 +268,7 @@ def convert_to_xpath31(obj: Any, parent_key: str | None = None) -> str:
 
 
 def convert(
-    obj: ELEMENT,
+    obj: Any,
     ids: Any,
     attr_type: bool,
     item_func: Callable[[str], str],
@@ -631,6 +632,7 @@ def convert_none(
     return f"<{key}{attr_string}></{key}>"
 
 
+# @lat: [[architecture#Conversion engine]]
 def dicttoxml(
     obj: ELEMENT,
     root: bool = True,

json2xml/dicttoxml.py

@@ -17,22 +17,24 @@
 from collections.abc import Callable
 from typing import Any
 
+RustStringTransform = Callable[[str], str]
+
 LOG = logging.getLogger("dicttoxml_fast")
 
 # Try to import the Rust implementation
 _USE_RUST = False
-_rust_dicttoxml = None
+_rust_dicttoxml: Callable[..., bytes] | None = None
+rust_escape_xml: RustStringTransform | None = None
+rust_wrap_cdata: RustStringTransform | None = None
 
 try:
-    from json2xml_rs import dicttoxml as _rust_dicttoxml  # type: ignore[import-not-found]  # pragma: no cover
-    from json2xml_rs import escape_xml_py as rust_escape_xml  # type: ignore[import-not-found]  # pragma: no cover
-    from json2xml_rs import wrap_cdata_py as rust_wrap_cdata  # type: ignore[import-not-found]  # pragma: no cover
+    from json2xml_rs import dicttoxml as _rust_dicttoxml  # pragma: no cover
+    from json2xml_rs import escape_xml_py as rust_escape_xml  # pragma: no cover
+    from json2xml_rs import wrap_cdata_py as rust_wrap_cdata  # pragma: no cover
     _USE_RUST = True  # pragma: no cover
     LOG.debug("Using Rust backend for dicttoxml")  # pragma: no cover
 except ImportError:  # pragma: no cover
     LOG.debug("Rust backend not available, using pure Python")
-    rust_escape_xml = None
-    rust_wrap_cdata = None
 
 # Import the pure Python implementation as fallback
 from json2xml import dicttoxml as _py_dicttoxml  # noqa: E402
@@ -48,6 +50,7 @@ def get_backend() -> str:
     return "rust" if _USE_RUST else "python"
 
 
+# @lat: [[architecture#Backend selection]]
 def dicttoxml(
     obj: Any,
     root: bool = True,

json2xml/dicttoxml_fast.py

@@ -8,6 +8,7 @@
 from .utils import InvalidDataError
 
 
+# @lat: [[architecture#Core pipeline]]
 class Json2xml:
     """
     Wrapper class to convert the data to xml
@@ -34,6 +35,8 @@ def __init__(
         self.cdata = cdata
         self.list_headers = list_headers
 
+    # @lat: [[behavior#Conversion output]]
+    # @lat: [[behavior#Invalid XML payloads]]
     def to_xml(self) -> Any | None:
         """
         Convert to xml using dicttoxml.dicttoxml and then pretty print it.

json2xml/json2xml.py

@@ -24,6 +24,7 @@ class StringReadError(Exception):
     pass
 
 
+# @lat: [[behavior#Input readers]]
 def readfromjson(filename: str) -> dict[str, str]:
     """Reads a JSON file and returns a dictionary."""
     try:
@@ -44,7 +45,7 @@ def readfromurl(url: str, params: dict[str, str] | None = None) -> dict[str, str
     raise URLReadError("URL is not returning correct response")
 
 
-def readfromstring(jsondata: str) -> dict[str, str]:
+def readfromstring(jsondata: object) -> dict[str, str]:
     """Loads JSON data from a string and returns a dictionary."""
     if not isinstance(jsondata, str):
         raise StringReadError("Input is not a proper JSON string")