Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# TODO

- Revisit bare metavariable kind inference. A bare name used in both an
identifier-only position and an expression position is currently inferred as
`id` for every occurrence. For example, `$obj = $obj + $_` looks as though
`$obj` captures an arbitrary expression, but the assignment target makes it
an identifier capture. This inference may be too broad. Consider requiring
an explicit kind for cross-position reuse, improving the diagnostic, or
adding a storage-path metavariable kind.
11 changes: 8 additions & 3 deletions cli/cli_args.mbt
Original file line number Diff line number Diff line change
Expand Up @@ -291,13 +291,18 @@ fn parse_scan_pattern_specs(
exit_code=2,
)
}
patterns.push({ shape: argv[index + 1], guards: Map([]) })
patterns.push({
shape: argv[index + 1],
guards: Map([]),
match_mode: Default,
})
has_guard.push(false)
index += 2
} else if arg.has_prefix("--pattern=") {
patterns.push({
shape: arg["--pattern=".length():].to_owned(),
guards: Map([]),
match_mode: Default,
})
has_guard.push(false)
index += 1
Expand Down Expand Up @@ -336,7 +341,7 @@ fn attach_scan_guard(
index -= 1
if !has_guard[index] {
let shape = patterns[index].shape
patterns[index] = { shape, guards }
patterns[index] = { shape, guards, match_mode: Default }
has_guard[index] = true
return
}
Expand All @@ -354,7 +359,7 @@ fn parse_cli_guard_map(source : String) -> Map[String, String] raise {
let docs = @yaml.Yaml::load_from_string(source) catch {
err =>
raise CliError::Usage(
message="invalid --guard YAML: \{to_repr(err)}",
message="invalid --guard YAML: \{Repr(err)}",
exit_code=2,
)
}
Expand Down
4 changes: 2 additions & 2 deletions cli/dump_command.mbt
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ pub fn render_dump_command(

///|
fn render_dump_impl_debug(source : String) -> String raise {
"\{to_repr(@untyped_ast.from_impl(parse_dump_impl(source)))}"
"\{Repr(@untyped_ast.from_impl(parse_dump_impl(source)))}"
}

///|
fn render_dump_expr_debug(source : String) -> String raise {
"\{to_repr(@untyped_ast.from_expr(parse_dump_expr(source)))}"
"\{Repr(@untyped_ast.from_expr(parse_dump_expr(source)))}"
}

///|
Expand Down
4 changes: 2 additions & 2 deletions cli/scan.mbt
Original file line number Diff line number Diff line change
Expand Up @@ -336,8 +336,8 @@ fn anonymous_pattern_rule(
rule_id: pattern.shape,
description: "Anonymous CLI pattern.",
definition: Structural({
inside_expr: None,
inside_toplevel: None,
inside_expr: [],
inside_toplevel: [],
patterns: [pattern],
patterns_not: [],
patterns_not_mode: PruneOnNegative,
Expand Down
1 change: 1 addition & 0 deletions cli/scan_wbtest.mbt
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ fn guarded_cli_pattern_scan_options() -> CliOptions {
{
shape: "$(callee:id)($(value:const))",
guards: { "$callee": "^@html\\.render$", "$value": "raw" },
match_mode: Default,
},
],
scan_root: "testdata/guard",
Expand Down
174 changes: 119 additions & 55 deletions docs/RuleSpec.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,11 +66,12 @@ Only these top-level keys are accepted:
- `patterns` (optional for structural rules): non-empty YAML array
- `patterns-not` (optional for structural rules): non-empty YAML array using
the same object schema as `patterns`
- `inside-expr` (optional for structural rules): YAML mapping using the same
`shape` and optional `guard` schema as `patterns`, used as an outer context
- `inside-toplevel` (optional for structural rules): YAML mapping using the
same `shape` and optional `guard` schema as `inside-expr`; its shape is parsed
as one MoonBit top-level item
- `inside-expr` (optional for structural rules): non-empty YAML array using the
same `shape` and optional `guard` object schema as `patterns`; entries are
ordered alternative outer expression contexts
- `inside-toplevel` (optional for structural rules): non-empty YAML array using
the same `shape` and optional `guard` keys as `inside-expr`, plus optional
`match-mode`; each shape is parsed as one MoonBit top-level item
- `taint` (required for taint rules): YAML mapping

Unknown top-level keys are rejected.
Expand All @@ -94,33 +95,37 @@ including trailing newlines produced by block scalars.

### Pattern Objects

Structural entries in `patterns`, `patterns-not`, `inside-expr`, and
`inside-toplevel` use this object schema.
Structural entries in `patterns`, `patterns-not`, and `inside-expr` use this
object schema. `inside-toplevel` adds the `match-mode` key described below.

Only these keys are accepted:

- `shape` (required): YAML string containing one MoonBit expression snippet
- `shape` (required): YAML string containing one MoonBit expression snippet,
or one top-level item for `inside-toplevel`
- `guard` (optional): YAML mapping from `$`-prefixed capture name to regex string
- `match-mode` (optional, `inside-toplevel` only): `exact` or `partial`

Unknown keys inside a pattern object are rejected.
`match-mode` is rejected in `patterns`, `patterns-not`, `inside-expr`, and
taint clauses.

Taint `sources`, `sinks`, and `sanitizers` use the same `shape` key. A `guard`
is invalid in these entries.

## Shapes

An ordinary `shape` must be a single MoonBit expression snippet.
`inside-toplevel.shape` must be exactly one MoonBit top-level item.
An ordinary `shape` must be a single MoonBit expression snippet. Each
`inside-toplevel` entry's `shape` must be exactly one MoonBit top-level item.

Valid expression shapes include calls, method calls, field accesses,
operators, blocks, conditionals, loops, matches, lambdas, collection literals,
record expressions, and other expression-sized MoonBit syntax. Ordinary
`patterns`, `patterns-not`, and `inside-expr` shapes are not a whole file,
top-level declaration, package fragment, or import list.

`inside-toplevel.shape` is parsed as one top-level item, such as a function,
top-level `let`, `test`, method `impl`, view, or top-level expression. It is
one item and cannot represent a whole file or import list.
Each `inside-toplevel` shape is parsed as one top-level item, such as a
function, top-level `let`, `test`, method `impl`, view, or top-level expression.
It is one item and cannot represent a whole file or import list.

Shapes are structural:

Expand All @@ -129,7 +134,9 @@ Shapes are structural:
- operators match literally
- call and method-call argument kinds, labels, order, and arity must match
- type annotations and type names in matched syntax must match where present
- source locations, formatting, and comments do not participate in matching
- source locations and formatting do not participate in matching; top-level
documentation is an AST field and follows the `inside-toplevel` matching
mode

The scanner does not type-check shapes and does not resolve names semantically.
For example, two imported names that refer to the same definition compare as
Expand Down Expand Up @@ -714,47 +721,56 @@ context. It may be used with `patterns`, with `patterns-not`, or with both.
```yaml
id: wrapped-target
description: |
Match a target call only inside wrapper(...).
Match a target call inside either supported context.
inside-expr:
shape: wrapper($(prefix:exp), __TARGET__)
- shape: wrapper($(prefix:exp), __TARGET__)
- shape: container($(prefix:exp), __TARGET__)
patterns:
- shape: target.call($(prefix:exp))
```

`inside-expr` is a YAML mapping. Its `shape` is parsed as one MoonBit
expression snippet, and its optional `guard` filters `id` and `const` captures
declared by that outer shape.
`inside-expr` is a non-empty YAML array of pattern objects. Each `shape` is
parsed as one MoonBit expression snippet, and its optional `guard` filters `id`
and `const` captures declared by that outer shape. Entries are ordered
alternatives.

Additional rules:

- `inside-expr` must contain exactly one `__TARGET__` occurrence in a
binding-capable position.
- Every `inside-expr` entry must contain exactly one `__TARGET__` occurrence in
a binding-capable position.
- `__TARGET__` must occupy a whole expression position, such as a whole call
argument, receiver, or block expression. If it appears only as a label or
other non-expression value, no target subtree can be searched.
- `__TARGET__` is reserved and must not be used as an metavar name.
- Entries in `patterns` and `patterns-not` must not contain `__TARGET__` in a
binding-capable position.
- Metavars declared by `inside-expr` remain visible when matching the
inner pattern entries; inner shapes reference them by repeating the same
metavar form.
- Inner `patterns` and `patterns-not` must not use a visible `inside-expr`
metavar name with a different kind.
- Captures declared by the selected `inside-expr` entry remain visible when
matching the inner pattern entries; inner shapes reference them by repeating
the same metavar form.
- Any capture reused by an inner `patterns` or `patterns-not` entry must be
declared by every `inside-expr` alternative with the same kind. This includes
named ellipsis captures and their ellipsis kinds. Outer captures that are not
referenced by an inner entry may differ between alternatives.

Runtime behavior:

- the current expression is first matched against `inside-expr`
- if it matches, the subtree captured by `__TARGET__` is searched
- the current expression tries eligible `inside-expr` entries in YAML order
- an entry whose shape does not match, or whose guard fails, falls through to
the next entry
- the first entry whose shape and guard both match selects the captured
`__TARGET__` subtree and bindings
- once an entry is selected, later alternatives are not tried even if inner
matching produces no finding
- when `patterns` is present, each expression in the captured subtree first
tries the ordered positive patterns using the bindings established by
`inside-expr`; a positive hit is recorded and its matched subtree covers any
nested negative matches
the selected outer entry; a positive hit is recorded and its matched subtree
covers any nested negative matches
- when `patterns` and `patterns-not` are both present, a candidate that fails
all positive patterns is then checked against `patterns-not` using the
`inside-expr` bindings; a negative match outside a positive-hit subtree
the selected outer bindings; a negative match outside a positive-hit subtree
rejects the whole outer match
- when `patterns` is absent, every expression in the captured subtree is
checked against `patterns-not` using the `inside-expr` bindings; if none of
checked against `patterns-not` using the selected outer bindings; if none of
them match, the outer expression produces one hit
- if an inner positive or negative pattern references an inherited `id` capture
with the same inline `$(name:id)` form, that candidate is skipped when the
Expand All @@ -769,39 +785,79 @@ outer expression do not produce additional findings. With only `patterns-not`,

### `inside-toplevel`

`inside-toplevel` restricts a structural rule to matches inside one MoonBit
top-level item. It uses the same object schema and target-subtree semantics as
`inside-expr`. Its `shape` is parsed as exactly one top-level item, not as an
expression.
`inside-toplevel` restricts a structural rule to matches inside selected
MoonBit top-level items. It is a non-empty ordered array using the same object
schema and target-subtree semantics as `inside-expr`. Each entry's `shape` is
parsed as exactly one top-level item, not as an expression.

```yaml
id: safe-function-target
description: |
Match calls only in selected top-level functions.
inside-toplevel:
shape: |
fn $(name:id)($(param:id) : Int) -> Int { __TARGET__ }
guard:
$name: "^safe_"
- shape: |
fn $(name:id)($(param:id) : Int) -> Int { __TARGET__ }
guard:
$name: "^safe_"
patterns:
- shape: call($(param:id))
```

`match-mode` is resolved independently for every ordered alternative:

| `match-mode` | Shape item | Effective matching |
| --- | --- | --- |
| omitted | function definition | `partial` |
| omitted | any other top-level item | `exact` |
| `exact` | any top-level item | `exact` |
| `partial` | function definition | `partial` |
| `partial` | any other top-level item | compile error |

Exact matching compares the complete parsed top-level AST, preserving the
behavior used before function shapes became partial by default:

```yaml
inside-toplevel:
- shape: |
fn $(name:id) { __TARGET__ }
match-mode: exact
```

Partial matching is limited to function definitions. It always matches the
function name, body, and `__TARGET__` exactly. The following function-header
fields are ignored only when the shape leaves them in their default form:

- type qualifier, `async`, parameter list, type parameters, return type, error
type, visibility, attributes, and documentation
- the top-level `where` clause

Writing any such field keeps it exact. For example, `fn f()` requires an
explicit empty parameter list, `pub fn` requires public visibility, and
`async fn`, a return type, `noraise`, type parameters, documentation,
attributes, or a `where` clause constrain the candidate exactly.

Migration note: an older rule that depended on an omitted function-header
field being absent must add `match-mode: exact`. Broad function-context rules
can remain unmarked and use the new partial default.

Additional rules:

- `inside-toplevel` and `inside-expr` are mutually exclusive.
- `inside-toplevel` must contain exactly one `__TARGET__` occurrence in a
binding-capable expression position within the top-level item.
- Every `inside-toplevel` entry must contain exactly one `__TARGET__`
occurrence in a binding-capable expression position within the top-level
item.
- The top-level item itself may declare `id` and `const` captures, and its
optional `guard` may filter those captures.
- Metavars declared by `inside-toplevel` remain visible to inner `patterns` and
`patterns-not`, using the same inherited binding and kind-consistency rules as
`inside-expr`.
- Captures declared by the selected `inside-toplevel` entry remain visible to
inner `patterns` and `patterns-not`, using the same all-alternatives
declaration and kind-consistency rules as `inside-expr`.
- `inside-toplevel` is not supported on taint rules.

The candidate top-level item is first matched against `inside-toplevel`; if it
matches, the expression subtree captured by `__TARGET__` is searched with the
same inherited-binding and negative-coverage behavior as `inside-expr`.
The candidate top-level item tries eligible `inside-toplevel` entries in YAML
order. The first entry whose shape and guard both match selects the target and
bindings; later alternatives are not tried after selection. The expression
subtree captured by `__TARGET__` is searched with the same inherited-binding
and negative-coverage behavior as `inside-expr`.
Reporting differs: with `patterns`, every inner positive hit produces a
finding whose `loc` is the inner match location. With only `patterns-not`, one
finding is produced at the matched top-level item location.
Expand Down Expand Up @@ -933,8 +989,11 @@ A rule set or rule file is rejected when any of these conditions occurs:
- `inside-expr` appears on a taint rule
- `inside-toplevel` appears on a taint rule
- both `inside-expr` and `inside-toplevel` appear
- `inside-expr` has a non-mapping value
- `inside-toplevel` has a non-mapping value
- `inside-expr` or `inside-toplevel` is not an array or is empty
- an `inside-expr` or `inside-toplevel` entry is not a mapping
- `match-mode` appears outside an `inside-toplevel` entry
- `match-mode` is not `exact` or `partial`
- `match-mode: partial` is used with a non-function top-level shape
- `inside-expr` is present without `patterns` or `patterns-not`
- `inside-toplevel` is present without `patterns` or `patterns-not`
- `patterns` is not an array or is empty
Expand All @@ -953,7 +1012,8 @@ A rule set or rule file is rejected when any of these conditions occurs:
- `guard` appears in any taint clause
- `metavars` appears in any pattern object
- `shape` is not valid as one MoonBit expression
- `inside-toplevel.shape` is not exactly one valid MoonBit top-level item
- an `inside-toplevel` entry's `shape` is not exactly one valid MoonBit
top-level item
- a shape uses an unsupported metavar kind
- a shape uses the same metavar name across multiple metavar kinds
- a bare `$name` cannot be inferred to one compatible kind
Expand All @@ -970,12 +1030,16 @@ A rule set or rule file is rejected when any of these conditions occurs:
- an ellipsis kind is incompatible with its list position, conflicts with
another typed occurrence, or shares a name with a normal metavar
- a guard regex is invalid
- `inside-expr` does not contain exactly one binding-capable `__TARGET__`
- `inside-toplevel` does not contain exactly one binding-capable `__TARGET__`
- an `inside-expr` entry does not contain exactly one binding-capable
`__TARGET__`
- an `inside-toplevel` entry does not contain exactly one binding-capable
`__TARGET__`
- a structural `patterns` or `patterns-not` entry contains binding-capable
`__TARGET__`
- a structural `patterns` or `patterns-not` entry uses an inherited
`inside-expr` or `inside-toplevel` metavar name with a different kind
- a capture reused by `patterns` or `patterns-not` is missing from any outer
alternative, or a named ellipsis is declared with a different ellipsis kind
- a taint source contains binding-capable `__SOURCE__`
- a taint sink or sanitizer does not contain exactly one binding-capable
`__SOURCE__`
Expand Down Expand Up @@ -1022,7 +1086,7 @@ id: unsafe-wrapper
description: |
Match a sink only under an unsafe wrapper.
inside-expr:
shape: unsafe(__TARGET__)
- shape: unsafe(__TARGET__)
patterns:
- shape: sink($_)
```
Expand Down Expand Up @@ -1056,7 +1120,7 @@ id: wrapper-without-danger
description: |
Match wrappers whose payload contains no danger call.
inside-expr:
shape: wrapper(__TARGET__)
- shape: wrapper(__TARGET__)
patterns-not:
- shape: danger()
```
Expand Down
Loading
Loading