Skip to content

Embedded Python methods in ObjectScript classes are indexed but semantically empty #1262

Description

@isc-tdyar

Summary

IRIS lets a class method be written in Python rather than ObjectScript via the
Language = python keyword. This is common in modern IRIS code and is the main
mechanism for using Python libraries from ObjectScript classes.

CBM creates a Method node for these, but the body contributes nothing: no call
edges, no import edges, no body tokens for FTS/BM25 search, and complexity
metrics of zero. The method is a name and a signature with an opaque blob
attached.

The grammar already keeps ObjectScript call patterns from firing on a Python
body, so this produces no false edges. The problem is blindness.

What the grammar already gives us

The vendored grammar handles the keyword and switches production on its value:

  • objectscript_udl/parser.c:1585 defines sym_method_keyword_language
  • Language = cache / objectscript yields method_keywords plus parsed
    statement nodes
  • Language = python / tsql yields external_method_keywords plus a single
    opaque external_method_body_content leaf

AST for ClassMethod Hello(name As %String) As %String [ Language = python ]:

method_definition
  method_name 'Hello'
  arguments '(name As %String)'
  return_type 'As %String'
  external_method_keywords '[ Language = python ]'
    method_keyword_language 'Language = python'
      typename 'python'                 <-- the discriminator
  external_method_body_content          <-- entire body, one leaf token

So the language is detectable and the body's byte range is available. Nothing in
CBM consults either. Grepping internal/ src/ tests/ (excluding vendored/)
for external_method_keywords, method_keyword_language, or
external_method_body_content returns nothing.

Observed impact

Same if / for logic written both ways, in one class:

Method complexity cognitive body_tokens
ObjectScript 2 3 Os a %Integer %Status i x
[ Language = python ] 0 0 Py a (signature only)

body_tokens comes from extract_body_ident_tokens
(internal/cbm/extract_defs.c:84-125), which collects only leaf nodes whose
kind is in a fixed identifier list (:98-107).
external_method_body_content is a leaf, but its kind is not in that list, so
it is skipped. That costs FTS5/BM25 body indexing (searching for a symbol that
appears only in a Python body returns nothing), the MinHash fingerprint, and
complexity, which stays 0.

Call resolution asymmetry for the identical line of Python:

Location iris.cls('My.Other').DoThing() -> edge?
plain .py file yes
[ Language = python ] method in .cls no
ObjectScript method in .cls yes (native syntax)

Also: Language = python routines emit garbage nodes

.mac / .int routines accept a Language keyword too, but the routine
grammar has no support for it (objectscript_routine/parser.c has no
method_keyword_language). With ROUTINE MyRoutine [Type=INC,Language=python],
error recovery combined with func_types = {"tag"}
(internal/cbm/lang_specs.c:1631) emits Function nodes named import and
def.

That one is graph pollution rather than blindness, so it may belong in its own
bug. Happy to file separately if you'd prefer.

Proposed scope

Following the existing Trigger-sidecar precedent in extract_defs.c:5923-5945:

  1. Detect the language: read external_method_keywords ->
    method_keyword_language -> typename.
  2. Tag the node, for example "language":"python" in the properties JSON. No
    schema change, same mechanism trigger_body / sql_map_globals already use.
    Or a CBMDefinition field if you'd rather have it first-class.
  3. Populate body tokens from the opaque leaf's text by whitespace and
    punctuation split. Cheap, and restores search_code over Python bodies.
  4. Re-parse the external_method_body_content byte range with
    tree_sitter_python and run the existing Python call/import extractors at a
    byte offset, so iris.cls(...), import, and iris.execute(...) behave as
    they do in .py files.

Step 4 carries the real value, and it reuses machinery that already exists and
is tested.

Suggested deferrals

To keep this reviewable I'd propose covering call and method dispatch only,
including ObjectScript strings inside iris.execute(...) since that needs the
same nested-language handling, and deferring:

  • iris.gref("^Global") reads and writes, which need a Global node concept
  • DB-API, iris.sql.exec(...), and %SQL.Statement SQL strings, which need
    table and query modeling

Those belong in a separate "IRIS data-access edges" discussion rather than call
resolution. Value-marshaling types (IRISList, Vector, ByRef) and the
sitecustomize bootstrap have no graph meaning, so I'd skip them entirely.

Dependency

This should land after #1261 (the Language = python body lexer drops
following methods) and #1260 (iris.cls resolves class-blind). Re-parsing a
Python body is only correct once the body's byte range is right, and it would
otherwise inherit the #1260 mis-resolution, turning a blindness bug into a false-edge
bug at greater volume.

Tests

New cases in the objectscript_udl_* group in tests/test_extraction.c:
keyword detection and node tagging, non-empty body_tokens, an iris.cls
CALLS edge originating from a Language = python method, and the routine
import/def garbage-node regression.

Confirming scope before I start, per the contribution guidance on new extraction
behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestparsing/qualityGraph extraction bugs, false positives, missing edgespriority/normalStandard review queue; useful PR with ordinary maintainer urgency.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions