Skip to content

feat(languages): YAML node-type registry — stepping stone for .scm migration (closes #43)#185

Merged
Wolfvin merged 1 commit into
mainfrom
feat/issue-43-node-types-registry
Jul 3, 2026
Merged

feat(languages): YAML node-type registry — stepping stone for .scm migration (closes #43)#185
Wolfvin merged 1 commit into
mainfrom
feat/issue-43-node-types-registry

Conversation

@Wolfvin

@Wolfvin Wolfvin commented Jul 3, 2026

Copy link
Copy Markdown
Owner

Closes #43

Ringkasan

Implementasi Approach 2 (YAML node-type registry) dari issue #43 — low-risk stepping stone yang externalise hardcoded tree-sitter node-type lookups ke YAML config. Issue #43 merekomendasikan ini sebagai "DO" action sebelum full .scm migration. Existing parsers tidak berubah — ini purely additive.

Trigger Conditions (all met)

  1. P0 bugs [BUG-01] scan: pr.store_scan_result(result) calls non-existent method — SQLite analysis_cache never populated #31, [BUG-02] --deep flag runs HybridEngine twice — LSP findings double-counted #32 — CLOSED ✓
  2. Phase 2 [ROADMAP] Phase 2 — Speed (Rust core via PyO3) #22 (Rust core) — CLOSED as 'premature optimization, deferred' ✓ (status clarified)
  3. 5+ languages commitment — BOS instructed to proceed ✓
  4. Pilot parity — this IS the pilot (Phase A) ✓

Perubahan

New files

File Purpose
scripts/languages/__init__.py Package marker, re-exports public API
scripts/languages/node_types.yaml Config for all 7 tree-sitter languages (python, rust, javascript, typescript, tsx, css, html). Maps semantic categories (function_def, call, import, class_def) to concrete node types.
scripts/languages/loader.py Cached YAML loader with frozenset caching. get_node_types(lang, cat) → frozenset. NodeTypeError on unknown language/category (fail loud, not silent).
tests/test_node_types.py 32 tests — loader, YAML validity, cache, error handling, BaseParser integration
docs/design/0005-node-type-registry.md Design doc with trade-offs, Phase B plan

Modified

  • scripts/base_parser.py — added find_nodes_by_category(root, language, category) method (additive, backward compatible). Existing find_nodes_by_type / find_nodes_by_types unchanged.

Design

YAML maps semantic categories (stable across languages) to concrete tree-sitter node types (language-specific):

python:
  function_def: [function_definition]
  call: [call]
  import: [import_statement]

rust:
  function_def: [function_item]
  call: [call_expression]
  import: [use_declaration]

Category names (function_def, call, import, class_def) are the same across all languages. This is the design property that the future .scm engine (Phase B) will exploit: a .scm query file for one language maps 1:1 to a category here.

Phase B: .scm Pilot (Next Step, NOT in this PR)

Phase B — Rust .scm pilot — documented in design doc as the next step. Not implemented in this PR karena tree-sitter Query API tidak available di semua environments. YAML registry (Phase A) works without tree-sitter at runtime — it's pure config.

Verifikasi

  • tests/test_node_types.py: 28 passed, 4 skipped (BaseParser tests need tree-sitter, which is optional in minimal environments per CONTEXT.md)
  • sync_command_count.py --check: clean (78 commands, no new command added)
  • check_design_doc.py: PASS
  • All 7 tree-sitter languages covered in YAML config

Trade-offs (from design doc)

…gration (closes #43)

Issue #43 — Approach 2 (YAML node-type registry) stepping stone.

Externalises hardcoded tree-sitter node-type lookups to a YAML config.
The Python walker stays, but reads node types from config instead of
hardcoding them. Adding a language = adding YAML lines, not Python.

This is the low-risk stepping stone recommended by issue #43 before any
.scm migration. It does NOT change existing parser behaviour — it adds
a new config-driven path (find_nodes_by_category) that future parsers
and the eventual .scm engine (Phase B) can use.

New files:
- scripts/languages/__init__.py     — package marker, re-exports API
- scripts/languages/node_types.yaml — config for all 7 tree-sitter
                                       languages (python, rust, javascript,
                                       typescript, tsx, css, html)
- scripts/languages/loader.py       — cached YAML loader with frozenset
                                       caching + NodeTypeError on unknown
                                       language/category
- tests/test_node_types.py          — 32 tests (28 pass, 4 skip without
                                       tree-sitter)
- docs/design/0005-node-type-registry.md — design doc

Modified:
- scripts/base_parser.py            — added find_nodes_by_category(root,
                                       language, category) method (additive,
                                       backward compatible). Existing
                                       find_nodes_by_type / find_nodes_by_types
                                       unchanged.

Trigger conditions for issue #43 (all met):
1. P0 bugs #31, #32 — CLOSED
2. Phase 2 #22 (Rust core) — CLOSED as 'premature optimization, deferred'
3. 5+ languages commitment — BOS instructed to proceed
4. Pilot parity — this IS the pilot (Phase A)

Phase B (Rust .scm pilot) documented as next step in design doc. Not
implemented in this PR because tree-sitter Query API is not available
in all environments; the YAML registry (Phase A) works without
tree-sitter at runtime — it's pure config.

Verified:
- tests/test_node_types.py: 28 passed, 4 skipped (BaseParser tests need
  tree-sitter, which is optional in minimal environments)
- sync_command_count.py --check: clean (78 commands, no new command)
- check_design_doc.py: PASS
- All 7 tree-sitter languages covered in YAML config
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@sonarqubecloud

sonarqubecloud Bot commented Jul 3, 2026

Copy link
Copy Markdown

@Wolfvin Wolfvin merged commit d57d03f into main Jul 3, 2026
2 of 8 checks passed
@Wolfvin Wolfvin deleted the feat/issue-43-node-types-registry branch July 3, 2026 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[PROPOSAL] Adopt tree-sitter query files (.scm) for node discovery — phased rollout, defer full migration until triggers met

1 participant