feat(languages): YAML node-type registry — stepping stone for .scm migration (closes #43)#185
Merged
Merged
Conversation
…gration (closes #43) Issue #43 — Approach 2 (YAML node-type registry) stepping stone. Externalises hardcoded tree-sitter node-type lookups to a YAML config. The Python walker stays, but reads node types from config instead of hardcoding them. Adding a language = adding YAML lines, not Python. This is the low-risk stepping stone recommended by issue #43 before any .scm migration. It does NOT change existing parser behaviour — it adds a new config-driven path (find_nodes_by_category) that future parsers and the eventual .scm engine (Phase B) can use. New files: - scripts/languages/__init__.py — package marker, re-exports API - scripts/languages/node_types.yaml — config for all 7 tree-sitter languages (python, rust, javascript, typescript, tsx, css, html) - scripts/languages/loader.py — cached YAML loader with frozenset caching + NodeTypeError on unknown language/category - tests/test_node_types.py — 32 tests (28 pass, 4 skip without tree-sitter) - docs/design/0005-node-type-registry.md — design doc Modified: - scripts/base_parser.py — added find_nodes_by_category(root, language, category) method (additive, backward compatible). Existing find_nodes_by_type / find_nodes_by_types unchanged. Trigger conditions for issue #43 (all met): 1. P0 bugs #31, #32 — CLOSED 2. Phase 2 #22 (Rust core) — CLOSED as 'premature optimization, deferred' 3. 5+ languages commitment — BOS instructed to proceed 4. Pilot parity — this IS the pilot (Phase A) Phase B (Rust .scm pilot) documented as next step in design doc. Not implemented in this PR because tree-sitter Query API is not available in all environments; the YAML registry (Phase A) works without tree-sitter at runtime — it's pure config. Verified: - tests/test_node_types.py: 28 passed, 4 skipped (BaseParser tests need tree-sitter, which is optional in minimal environments) - sync_command_count.py --check: clean (78 commands, no new command) - check_design_doc.py: PASS - All 7 tree-sitter languages covered in YAML config
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Closes #43
Ringkasan
Implementasi Approach 2 (YAML node-type registry) dari issue #43 — low-risk stepping stone yang externalise hardcoded tree-sitter node-type lookups ke YAML config. Issue #43 merekomendasikan ini sebagai "DO" action sebelum full .scm migration. Existing parsers tidak berubah — ini purely additive.
Trigger Conditions (all met)
Perubahan
New files
scripts/languages/__init__.pyscripts/languages/node_types.yamlscripts/languages/loader.pyget_node_types(lang, cat)→ frozenset.NodeTypeErroron unknown language/category (fail loud, not silent).tests/test_node_types.pydocs/design/0005-node-type-registry.mdModified
scripts/base_parser.py— addedfind_nodes_by_category(root, language, category)method (additive, backward compatible). Existingfind_nodes_by_type/find_nodes_by_typesunchanged.Design
YAML maps semantic categories (stable across languages) to concrete tree-sitter node types (language-specific):
Category names (
function_def,call,import,class_def) are the same across all languages. This is the design property that the future .scm engine (Phase B) will exploit: a .scm query file for one language maps 1:1 to a category here.Phase B: .scm Pilot (Next Step, NOT in this PR)
Phase B — Rust .scm pilot — documented in design doc as the next step. Not implemented in this PR karena tree-sitter Query API tidak available di semua environments. YAML registry (Phase A) works without tree-sitter at runtime — it's pure config.
Verifikasi
tests/test_node_types.py: 28 passed, 4 skipped (BaseParser tests need tree-sitter, which is optional in minimal environments per CONTEXT.md)sync_command_count.py --check: clean (78 commands, no new command added)check_design_doc.py: PASSTrade-offs (from design doc)