fix(standards/cpp): support functor operator() and type conversion operators in func_start regex - #1759
Conversation
…erators in func_start Closes squid-protocol#1752
squid-protocol
left a comment
There was a problem hiding this comment.
Thanks for taking this on, Yunus — operator() and type-conversion operators being invisible to func_start was a real gap, and I appreciate you tracking down the actual root cause in issue #1752 rather than just patching the symptom.
I tested the new alternative against a few cases and wanted to flag one before merging. The three examples from #1752 all work correctly:
operator()→ capturesMyClass::operator()✅operator bool()→ capturesMyClass::operator bool✅operator ::AABB()→ capturesMyClass::operator ::AABB✅
But for the "namespace-qualified conversions" case the PR description also calls out, the captured name is wrong:
>>> rx.search("MyClass::operator std::string() const {\n").group(1)
'std::string' # expected: 'MyClass::operator std::string'
>>> rx.search("MyClass::operator Foo() const {\n").group(1)
'Foo' # expected: 'MyClass::operator Foo'It still matches (so recall goes up, which is the point of the issue), but the earlier "return type" loop in the pattern (a few lines above the identifier capture) is greedy and doesn't recognize operator as special — for any conversion target that isn't one of the hardcoded primitive keywords (bool, int, void, etc.), that loop consumes ClassName::operator as if it were an ordinary return-type token, and the new alternative never gets a chance to fire. What lands in group 1 is just the trailing type name, silently dropped of both the class qualifier and the operator keyword — so it ends up misattributed as an unrelated function/identifier (e.g. std::string) rather than the conversion operator. bool and the leading-:: case both happen to dodge this because they're either in that keyword exclusion list or can't be parsed as a bare qualified identifier, so they force the loop to back off — but std::string, or a plain custom class name like Foo, don't get that protection.
Could you extend the "not a function" shield (or add a similar guard) so the return-type loop doesn't eat the operator token itself? Also, since this touches func_start for a widely-used language, it'd be good to add a regression case to tests/extraction/languages/test_cpp.py covering at least a non-primitive conversion type (operator std::string() or similar) so this doesn't regress silently.
One more small thing: the PR says "Closes #1752", but that issue also lists two other bugs (C++ digit-separator string-shielding, and #else macro-shielding) that this PR doesn't address — might be worth switching to "part of #1752" or similar so the issue doesn't auto-close before those are fixed too.
Happy to help iterate on the regex if useful — this is close, and the recall improvement is a nice find either way. Thanks again for digging into this!
…_start regex When parsing non-primitive conversion operators like MyClass::operator std::string(), the return type loop previously consumed MyClass::operator as a return type prefix, leaving only std::string captured as the function name. Add a negative lookahead to prevent the return type loop from consuming operator identifiers. Addresses part of squid-protocol#1752
…tors Addresses part of squid-protocol#1752
|
@squid-protocol Thanks for the detailed feedback! I've updated the regex and pushed the fix:
|
squid-protocol
left a comment
There was a problem hiding this comment.
Confirmed — I tested the negative lookahead against all the cases from before, including the ones that were broken:
>>> rx.search("MyClass::operator std::string() const {\n").group(1)
'MyClass::operator std::string' # was 'std::string'
>>> rx.search("MyClass::operator Foo() const {\n").group(1)
'MyClass::operator Foo' # was 'Foo'All correctly capture the full qualified name now, and the ordinary cases (plain functions, constructors, operator+, operator new) are still unaffected. No ReDoS regression from the new lookahead either (linear up to n=50k). Nice fix.
I also approved the pending workflow runs on this branch — as a first-time contributor's PR, GitHub was holding CI at "action_required" and nothing had actually run yet. It's executing now; I'll merge once it's green.
Thanks again for the thorough follow-through, Yunus!
|
Looked at the three failures — they need two different responses, not one.
and add a short note to the PR description on why the fixtures changed (see
(The "MISSING CLASSES IN godot/variant.h" line right above that is unrelated noise, for what it's worth — I checked, those are The two numeric regressions are real and specific to this PR — I diffed the branch against |
|
@emre155 Thank you so much for your hard work on this, and congratulations on your first contribution! I went ahead and pushed the final fixes for the remaining CI issues (the tree-sitter regressions and the downstream ruff/mypy checks) directly to your branch, since the instructions for resolving the tree-sitter accuracy audit regressions weren't very clear initially. I really appreciate your time and effort getting this started! This is now fully green and ready for merge. |
7a4784f to
c3d8bec
Compare
Addresses part of #1752
Summary of Changes
(?!(?:[a-zA-Z_]\w*::)*operator\b)to the preceding return-type loop infunc_start. This prevents the return-type loop from greedily consumingMyClass::operatorwhen encountering non-primitive conversion operators (such asMyClass::operator std::string()orMyClass::operator Foo()).operator(), symbol operators, memory operators, and type conversion operators (including primitive, leading-::, and namespace-qualified targets).tests/extraction/languages/test_cpp.pycoveringoperator(),operator bool(),operator std::string(), andoperator Foo().Golden Crucible Updates:
The golden master fixtures were regenerated because the new C++ operator regex now correctly identifies conversion operators like
Variant::operator ::AABBandVariant::operator Stringwhich were previously missed. This caused expected cascading shifts in the topological coordinates and mass metrics for the C++ corpus.Golden Crucible Updates:
The golden master fixtures were regenerated because the new C++ operator regex now correctly identifies conversion operators like
Variant::operator ::AABBandVariant::operator Stringwhich were previously missed. This caused expected cascading shifts in the topological coordinates and mass metrics for the C++ corpus.