unified: Stop using the tree-sitter-swift grammar

With the Swift front-end fully switched to swift-syntax, nothing links
the tree-sitter Swift grammar any more. Remove every reference to it;
the vendored crate itself is deleted in the following commit, so that
this one shows only what actually changed.

- Drop `unified/extractor/tree-sitter-swift` as a workspace member, path
  dependency and BUILD.bazel dependency.
- Drop the now-unused `tree-sitter` and `tree-sitter-embedded-template`
  direct dependencies from the extractor (neither is referenced any
  longer; the tree-sitter runtime is still pulled in transitively where
  the shared extractor needs it).
- Rewrite the "Swift Parser" section of `AGENTS.md`, which still pointed
  at `grammar.js` and `node-types.yml`, to describe `swift-syntax-parse`
  and the hand-maintained `swift_node_types.yml`, and note that the
  tests need the parser binary.

The mapping's comments also explained many rules by how the tree-sitter
path had behaved. That is now of historical interest only, so each is
restated in terms of swift-syntax and the target AST alone — no rule
changes, and the corpus is unaffected. Two were more than stylistic:

- The `subscriptCallExpr` rule and its corpus case said the parser
  reports `xs[0]` and `xs(0)` identically. swift-syntax distinguishes
  them, so the collapse to `call_expr` is now purely ours, and a
  dedicated `subscript_expr` would need only a schema addition and a
  remap.
- `discardAssignmentExpr` mapped to `name_expr` "because tree-sitter
  treated `_` as a name". The standing reason is that the target AST has
  no expression-level discard — only `ignore_pattern`, which is a
  pattern.

References to tree-sitter's *node model* are kept: yeast is built on it,
so `adapter.rs` still explains named/anonymous nodes, `extra` tokens and
byte-offset conventions in those terms.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: tausbn

Cargo.lock

@@ -9,7 +9,6 @@ members = [
     "shared/yeast-schema",
     "ruby/extractor",
     "unified/extractor",
-    "unified/extractor/tree-sitter-swift",
     "unified/swift-syntax-rs",
     "rust/extractor",
     "rust/extractor/macros",

Cargo.toml

@@ -1,23 +1,32 @@
 # Agent instructions
 
-This is a CodeQL extractor based on tree-sitter.
+This is a CodeQL extractor that maps a language's parse tree onto a shared AST
+using the `yeast` desugaring engine. Swift, the only language so far, is parsed
+by Apple's swift-syntax rather than by tree-sitter.
 
 ## Building
 - To build the extractor, run `scripts/create-extractor-pack.sh`
 
 ## Swift Parser
-- The Swift parser is defined by `extractor/tree-sitter-swift/grammar.js` and can be edited if needed.
+- Swift source is parsed by `swift-syntax-parse`, a small Swift/Rust binary in
+  `swift-syntax-rs` that wraps Apple's swift-syntax and emits the parse tree as
+  JSON. There is no grammar in this repository to edit.
 
-- After editing the grammar, always run `scripts/regenerate-grammar.sh`.
+- `extractor/src/languages/swift/adapter.rs` converts that JSON into a yeast AST.
 
-- The raw parse tree is described by `extractor/tree-sitter-swift/node-types.yml` and should be reviewed after grammar changes.
+- The raw parse tree's shape is described by `extractor/swift_node_types.yml`,
+  which is maintained by hand.
 
 ## AST Mapping
 - The target AST shape is described by `extractor/ast_types.yml`.
 
 - The mapping from the parse tree to the target AST is found in `extractor/src/languages/swift/swift.rs`
 
-- To run tests for the parser and mapping, run `cargo test` in the `extractor` directory.
+- To run tests for the parser and mapping, run `cargo test` in the `extractor`
+  directory. The tests need the `swift-syntax-parse` binary: point
+  `CODEQL_EXTRACTOR_UNIFIED_SWIFT_SYNTAX_PARSE` at it, or put it on `PATH`.
+  Corpus tests skip themselves when it cannot be found, so check for skips
+  before concluding a change is clean.
 
 - Extractor test cases are located at `extractor/tests/corpus/swift/*/*.swift`.
 

unified/AGENTS.md

@@ -20,6 +20,5 @@ codeql_rust_binary(
     ) + [
         "//shared/tree-sitter-extractor",
         "//shared/yeast",
-        "//unified/extractor/tree-sitter-swift",
     ],
 )

unified/extractor/BUILD.bazel

@@ -7,9 +7,6 @@ edition = "2024"
 
 # When updating these dependencies, run `misc/bazel/3rdparty/update_cargo_deps.sh`
 [dependencies]
-tree-sitter = ">= 0.23.0"
-tree-sitter-embedded-template = "0.25.0"
-tree-sitter-swift = { path = "tree-sitter-swift" }
 clap = { version = "4.5", features = ["derive"] }
 tracing = "0.1"
 tracing-subscriber = { version = "0.3.20", features = ["env-filter"] }

unified/extractor/Cargo.toml

@@ -56,9 +56,6 @@ supertypes:
   # A statement is anything that can appear in a block.
   # This type contains all of 'expr' and has partial overlap with 'member'.
   # For example, type_alias_declaration can appear either as a stmt or member.
-  # constructor_declaration and destructor_declaration appear here because
-  # tree-sitter-swift's error recovery for #if/#endif in class bodies can place
-  # init/deinit declarations at the wrong (statement) level.
   stmt:
     - expr
     - variable_declaration

unified/extractor/ast_types.yml

@@ -17,9 +17,8 @@
 //! * Collection nodes are already elided to JSON arrays upstream, so a
 //!   list-valued field maps directly to that field holding several children.
 //!
-//! Note: this preserves swift-syntax's own kind/field names. Aligning those
-//! names with the tree-sitter-swift schema (so the rewrite rules in
-//! [`super::swift`] fire) is done incrementally in the rules.
+//! Note: this preserves swift-syntax's own kind/field names; the rewrite rules
+//! in [`super::swift`] match those names directly.
 
 use std::collections::BTreeMap;
 

unified/extractor/src/languages/swift/adapter.rs

@@ -77,8 +77,8 @@ fn chained_modifier(ctx: &mut yeast::build::BuildCtx<'_, SwiftContext>) -> Optio
 
 /// Combine a list of boolean sub-conditions into a single expression by
 /// left-folding with the infix `&&` operator. Used by control-flow
-/// rules (`if`, `guard`, `while`, `repeat-while`) whose tree-sitter
-/// nodes carry one or more comma-separated conditions that the target
+/// rules (`if`, `guard`, `while`, `repeat-while`), which carry one or
+/// more comma-separated conditions that the target
 /// AST represents as a single `condition:` field. Panics on an empty
 /// input because every caller's grammar guarantees at least one
 /// condition.
@@ -144,7 +144,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
         // ---- Literals ----
         // swift-syntax does not distinguish the lexical integer/string forms
         // (hex/binary/octal, single- vs multi-line, raw): each is a single
-        // `*LiteralExpr` kind, so the tree-sitter variants collapse to one rule.
+        // `*LiteralExpr` kind, so one rule per literal type suffices.
         rule!((integerLiteralExpr) => (int_literal)),
         rule!((floatLiteralExpr) => (float_literal)),
         rule!((booleanLiteralExpr) => (boolean_literal)),
@@ -169,8 +169,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
         rule!((declReferenceExpr baseName: @name) => (name_expr identifier: (identifier #{name}))),
         // A discard `_` used as an expression — e.g. the target of a discarding
         // assignment `_ = x`. swift-syntax models it as a `discardAssignmentExpr`;
-        // the tree-sitter path treated the bare `_` as a name, so map it to a
-        // `name_expr` too.
+        // the target AST has no expression-level discard (only `ignore_pattern`,
+        // which is a pattern), so it becomes a `name_expr` over the `_` token.
         rule!((discardAssignmentExpr wildcard: @@w) => (name_expr identifier: (identifier #{w}))),
         // ---- Operators ----
         // The parser front-end folds operator chains into nested
@@ -244,10 +244,9 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
                 accessor_kind: (accessor_kind "get")
                 body: (block stmt: {body}))
         ),
-        // A property with an explicit accessor block. The two shapes differ only
-        // by the presence of an initializer (tree-sitter split them into distinct
-        // `willset_didset_block` vs computed-accessor node types; swift-syntax
-        // makes both plain `accessorDecl`s):
+        // A property with an explicit accessor block. swift-syntax makes both
+        // shapes plain `accessorDecl`s, so they are told apart by the presence
+        // of an initializer:
         //
         //  * With an initializer (`var x: T = e { willSet {…} didSet {…} }`) it is
         //    a *stored* property with observers: emit the backing
@@ -398,8 +397,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
         // payload parameters; an element with a raw value (`case a = 1`) or a
         // plain element (`case north`) becomes a `variable_declaration`. All
         // carry the shared case modifiers / chained tag from `ctx` (set by the
-        // `enumCaseDecl` rule below) and are tagged `enum_case` (after any
-        // `chained_declaration` tag, matching the tree-sitter modifier order).
+        // `enumCaseDecl` rule below) and are tagged `enum_case`, after any
+        // `chained_declaration` tag.
         rule!(
             (enumCaseElement name: @name parameterClause: (enumCaseParameterClause parameters: _* @params))
             =>
@@ -432,8 +431,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
         // Enum cases. A single `case` declaration may carry modifiers
         // (e.g. `indirect`) and list several comma-separated elements; each
         // becomes its own declaration carrying those shared modifiers, and
-        // non-first ones are tagged `chained_declaration` (mirroring the
-        // tree-sitter `enum_entry` rule). The modifiers are published into `ctx`
+        // non-first ones are tagged `chained_declaration`. The modifiers are
+        // published into `ctx`
         // for the element rules above, which build the actual declaration.
         rule!(
             (enumCaseDecl modifiers: _* @mods elements: _* @@cases)
@@ -530,7 +529,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
         // A function declaration (parameters/return type/body optional). The
         // parameters and return type nest under `signature`; the body is a
         // `codeBlock`. A bodyless function (a protocol requirement) still emits
-        // an empty `block`, matching the tree-sitter path.
+        // an empty `block`.
         rule!(
             (functionDecl
                 name: @name
@@ -723,8 +722,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
                 condition: {and_chain(&mut ctx, cond)}
                 else: {else_stmts})
         ),
-        // Ternary (`c ? a : b`) desugars to an `if_expr`, as in the tree-sitter
-        // path.
+        // Ternary (`c ? a : b`) desugars to an `if_expr`.
         rule!(
             (ternaryExpr condition: @cond thenExpression: @then_val elseExpression: @else_val)
             =>
@@ -769,8 +767,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
             (pattern_guard_expr pattern: {pat} value: {val})
         ),
         // Optional binding (`if let x = foo`, or shorthand `if let x`) desugars
-        // to a `pattern_guard_expr` matching `Optional.some(x)`, exactly as the
-        // tree-sitter path does. The initialized form is matched first.
+        // to a `pattern_guard_expr` matching `Optional.some(x)`. The initialized
+        // form is matched first.
         rule!(
             (optionalBindingCondition
                 pattern: (identifierPattern identifier: @name)
@@ -845,10 +843,12 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
         ),
         rule!((arrayElement expression: @e) => expr { e }),
         // A dictionary literal (`["a": 1]`) is kept as an opaque `map_literal`
-        // leaf (its source span), matching the tree-sitter path.
+        // leaf (its source span).
         rule!((dictionaryExpr) => (map_literal)),
-        // A subscript access (`xs[0]`) is modelled as a call, exactly as the
-        // tree-sitter grammar does (it parses `xs[0]` like `xs(0)`).
+        // A subscript access (`xs[0]`) is modelled as a call. swift-syntax does
+        // report a distinct `subscriptCallExpr`, so giving
+        // subscripts their own shape needs only a `subscript_expr` node in
+        // ast_types.yml and a remap here.
         rule!(
             (subscriptCallExpr calledExpression: @callee arguments: _* @args)
             =>
@@ -898,9 +898,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
         rule!((isExpr expression: @val type: @ty) => (type_test_expr expr: {val} operator: (infix_operator "is") type: {ty})),
         // Await expression → unary_expr with operator "await"
         rule!((awaitExpr expression: @val) => (unary_expr operator: (prefix_operator "await") operand: {val})),
-        // Force-unwrap (`x!`) → postfix unary_expr. swift-syntax has a dedicated
-        // `forceUnwrapExpr` node (the tree-sitter path used the generic postfix
-        // operator rule instead).
+        // Force-unwrap (`x!`) → postfix unary_expr, via swift-syntax's dedicated
+        // `forceUnwrapExpr` node.
         rule!((forceUnwrapExpr expression: @e) => (unary_expr operator: (postfix_operator "!") operand: {e})),
         // ---- Imports ----
         // An import declaration. The dotted path (a list of
@@ -961,8 +960,8 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
         // A named type (`Int`). `identifierType.name` is the type-name token.
         rule!((identifierType name: @@n) => (named_type_expr name: (identifier #{n}))),
         // A qualified type (`Outer.Inner`, `NSString.CompareOptions`). swift-syntax
-        // nests these as `memberType` nodes; like the old tree-sitter `user_type`
-        // rule, we keep the whole dotted path as the opaque `named_type_expr` name.
+        // nests these as `memberType` nodes; we keep the whole dotted path as the
+        // opaque `named_type_expr` name.
         rule!((memberType) @ty => (named_type_expr name: (identifier #{ty}))),
         // Sugared types desugar to `generic_type_expr`: `T?` -> Optional<T>,
         // `[T]` -> Array<T>, `[K: V]` -> Dictionary<K, V>.
@@ -1029,9 +1028,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
         // expansions uniformly as a `macroExpansionExpr`).
         rule!((macroExpansionExpr) => (unsupported_node)),
         // A nominal type's `inheritanceClause` (`: Base, Proto`) becomes a list
-        // of `base_type`s, one per inherited type. The tree-sitter path dropped
-        // it (no corpus target had a `base_type`) and the mapping matched that
-        // for parity; swift-syntax exposes it cleanly. Each declaration keyword
+        // of `base_type`s, one per inherited type. Each declaration keyword
         // gets its own rule; the bodies are identical but for the keyword.
         // Class declaration with body containing members
         rule!(
@@ -1100,8 +1097,7 @@ fn translation_rules() -> Vec<Rule<SwiftContext>> {
         // An `extension Foo { … }` is likewise a `class_like_declaration`, named
         // by the extended type. The extended type is captured opaquely (as its
         // source text) so that qualified names (`extension String.Interpolation`,
-        // a `memberType`) name the declaration just like simple ones, matching the
-        // old tree-sitter `user_type` behaviour.
+        // a `memberType`) name the declaration just like simple ones.
         rule!(
             (extensionDecl
                 extensionKeyword: @kind

unified/extractor/src/languages/swift/swift.rs

@@ -1,6 +1,6 @@
-// TODO: tree-sitter-swift parses `xs[0]` as a call_expression (same shape
-// as `xs(0)`), so the mapping currently produces a call_expr. Update the
-// parser / add a separate subscript_expr node and remap when fixed.
+// TODO: `xs[0]` is mapped to a call_expr, even though swift-syntax reports a
+// distinct subscriptCallExpr. Giving subscripts their own shape needs only a
+// subscript_expr node in ast_types.yml and a remap.
 let first = xs[0]
 
 ---

unified/extractor/tests/corpus/swift/collections/subscript-access.output

@@ -1,4 +1,4 @@
-// TODO: tree-sitter-swift parses `xs[0]` as a call_expression (same shape
-// as `xs(0)`), so the mapping currently produces a call_expr. Update the
-// parser / add a separate subscript_expr node and remap when fixed.
+// TODO: `xs[0]` is mapped to a call_expr, even though swift-syntax reports a
+// distinct subscriptCallExpr. Giving subscripts their own shape needs only a
+// subscript_expr node in ast_types.yml and a remap.
 let first = xs[0]