-
Notifications
You must be signed in to change notification settings - Fork 0
T-0162: triage the CodeQL alerts by measuring them #23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| { | ||
| "$comment": [ | ||
| "Alerts that stay open on purpose, in the shape the strict and doctor", | ||
| "baselines already use: a committed list a check compares against, so", | ||
| "debt is visible in the diff and only an unaccounted alert breaks the", | ||
| "build.", | ||
| "", | ||
| "Only for alerts that are real and not yet fixed. An alert that is a", | ||
| "false positive belongs in GitHub's own dismissal, with its reason,", | ||
| "where it stops being open at all — three were dismissed that way on", | ||
| "2026-08-05 after being measured linear. This file is the other case:", | ||
| "known, real, and waiting on a decision.", | ||
| "", | ||
| "Keyed by rule and path rather than by alert number, because a number", | ||
| "belongs to an instance and changes when the line moves — which is how", | ||
| "the same acceptance.ts pattern arrived as a `new` alert on PR #22", | ||
| "after the file was edited around it.", | ||
| "", | ||
| "An entry is a promise that somebody measured it and opened a card." | ||
| ], | ||
| "accepted": [ | ||
| { | ||
| "rule": "js/regex-injection", | ||
| "path": "packages/workfile/src/modules/search/search.ts", | ||
| "card": "T-0190", | ||
| "reason": "Not a false positive. `/(a+)+$/` — six characters, inside the 256-character pattern cap and the imsu flag subset — takes 57s against a 32-character body and does not finish against the 20,000-character one the search path allows. Open because the fix needs a design decision the card lays out, not because it is noise.", | ||
| "measured": "2026-08-05" | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| name: "workfile" | ||
|
|
||
| # Fixtures are inputs, not code. `test/fixtures/legacy-v1/` is a snapshot of the | ||
| # pre-workfile board this package migrates *away* from — it is checked in so the | ||
| # migration has something to read, is never imported and never runs, and its one | ||
| # alert (`js/stack-trace-exposure`) is a faithful record of how that old server | ||
| # behaved. Analysing it reports the bug we keep on purpose. | ||
| # | ||
| # The demo bundle is generated output; the analysis belongs on its source. | ||
| paths-ignore: | ||
| - packages/workfile/test/fixtures | ||
| - packages/workfile/dist | ||
| - packages/workfile/ui/dist | ||
| - apps/demo/public |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| name: CodeQL baseline | ||
|
|
||
| # An alert on a pull request is commented on inline and gets looked at within a | ||
| # minute. An alert on code that is already merged has no such messenger, so | ||
| # twelve of them sat open on `main` for a week and the `codeql` job passed on | ||
| # every single run — it uploads results, it does not read them (T-0162). | ||
| # | ||
| # This reads them. Daily rather than on push: the alerts API serves the previous | ||
| # analysis until the new one finishes processing, so a check that races the | ||
| # upload reports the state it was called to replace. A day is the honest | ||
| # latency, and it is bounded — which "nothing ever looks" was not. | ||
| on: | ||
| schedule: | ||
| - cron: "17 6 * * *" | ||
| workflow_dispatch: | ||
| # Deliberately not on `pull_request`. The first version of this ran on changes | ||
| # to the accepted list, on the theory that the PR editing the list should be | ||
| # the one to check it — and it failed on its own first run, correctly. The | ||
| # alerts API answers for `main`, so a pull request that *fixes* three alerts | ||
| # sees them still open and reports the state it exists to change. There is no | ||
| # query that would help: the alerts a merge will close do not close until the | ||
| # merge is analysed. Daily is the honest cadence for a question only `main` can | ||
| # answer. | ||
|
|
||
| jobs: | ||
| baseline: | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| security-events: read | ||
| contents: read | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
| - name: Every open alert is accounted for | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| run: | | ||
| set -euo pipefail | ||
| gh api "repos/${{ github.repository }}/code-scanning/alerts?state=open&per_page=100" \ | ||
| --jq '[.[] | {rule: .rule.id, path: .most_recent_instance.location.path, severity: .rule.security_severity_level, number: .number}]' \ | ||
| > open.json | ||
|
|
||
| jq -r '.accepted[] | "\(.rule)\t\(.path)"' .github/codeql/accepted-alerts.json | sort -u > accepted.tsv | ||
| jq -r '.[] | "\(.rule)\t\(.path)"' open.json | sort -u > open.tsv | ||
|
|
||
| { | ||
| echo "## CodeQL alerts open on \`main\`" | ||
| echo | ||
| echo "| severity | rule | path |" | ||
| echo "|---|---|---|" | ||
| jq -r '.[] | "| \(.severity) | \(.rule) | \(.path) |"' open.json | ||
| } >> "$GITHUB_STEP_SUMMARY" | ||
|
|
||
| # Unaccepted: open but not in the committed list. | ||
| if comm -23 open.tsv accepted.tsv | grep -q .; then | ||
| echo "::error::CodeQL alerts are open that the accepted list does not cover" | ||
| comm -23 open.tsv accepted.tsv | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Stale: accepted but no longer open. Not a failure — the debt | ||
| # went away, which is the direction this is supposed to move — | ||
| # but the entry has to go, or the list stops meaning anything. | ||
| if comm -13 open.tsv accepted.tsv | grep -q .; then | ||
| echo "::warning::accepted entries no longer match an open alert; remove them" | ||
| comm -13 open.tsv accepted.tsv | ||
| fi | ||
|
|
||
| echo "every open alert is accounted for" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
41 changes: 41 additions & 0 deletions
41
.project/cards/T-0190-a-regex-search-query-can-hang-the-process-and-the-.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| --- | ||
| id: T-0190 | ||
| title: A regex search query can hang the process, and the caps do not stop it | ||
| status: backlog | ||
| type: bug | ||
| priority: high | ||
| area: core | ||
| created: 2026-08-05 | ||
| updated: 2026-08-05 | ||
| --- | ||
|
|
||
| `search` accepts a `/pattern/flags` query ([[CHG-0021]]) and compiles it with `new RegExp`. Three guards already bound it — the pattern is capped at 256 characters, flags are a subset of `imsu`, and each record body is truncated to 20,000 characters before matching. None of them bounds *backtracking*. | ||
|
|
||
| Measured with `/(a+)+$/`, six characters, well inside every cap: | ||
|
|
||
| | body | time | | ||
| |---|---| | ||
| | 20 chars | 104ms | | ||
| | 24 chars | 232ms | | ||
| | 28 chars | 3,717ms | | ||
| | 30 chars | 14,342ms | | ||
| | 32 chars | 57,113ms | | ||
|
|
||
| Two more characters is roughly four times the work. Against the 20,000-character body cap it does not finish. | ||
|
|
||
| On the CLI this is self-inflicted: your own query, your own terminal, Ctrl-C. What makes it a bug rather than a footgun is the HTTP surface. `/api/v2/search?q=` takes the same query, the rebinding guard covers cross-origin but not a request that is allowed to arrive, and `workfile ui --host 0.0.0.0` puts it on the network — the configuration the CI workflow itself describes as "unauthenticated read and write access to a repository". | ||
|
|
||
| This is CodeQL alerts #19 and #20 (`js/regex-injection`). They are **not** false positives, which is why they are a card and not a dismissal. | ||
|
|
||
| No obvious cheap fix, which is the other reason this is a card: | ||
|
|
||
| - **Reject nested quantifiers.** A heuristic, never complete, and it would refuse legitimate patterns. | ||
| - **Run the match under a deadline.** Node has no regex timeout; it needs a worker thread, and the match loop is currently synchronous inside an async function. | ||
| - **Use a linear-time engine.** RE2 is the correct answer and a native dependency, which this package has avoided everywhere else. | ||
| - **Drop user-supplied regex.** Honest, and removes a documented feature. | ||
|
|
||
| ## Acceptance criteria | ||
|
|
||
| - [ ] A catastrophic pattern cannot hold the process past a bounded time | ||
| - [ ] The HTTP search path is bounded whether or not the CLI one is | ||
| - [ ] Whichever route is taken says what it costs the regex feature |
22 changes: 22 additions & 0 deletions
22
...HG-0129-a-record-body-can-no-longer-stall-the-doctor-or-choose-a-link-scheme.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| --- | ||
| id: CHG-0129 | ||
| title: A record body can no longer stall the doctor or choose a link scheme | ||
| type: fixed | ||
| area: core | ||
| visibility: public | ||
| created: 2026-08-05 | ||
| updated: 2026-08-05 | ||
| --- | ||
|
|
||
| Two scans over document bodies were quadratic. A body of unclosed Markdown | ||
| links took 43.6 seconds at 128KB in the link check the doctor runs for every | ||
| document, and a folder of path separators took 189ms at 16,000 characters in | ||
| `doc create --folder`. Both are bounded and linear now, and a test holds each | ||
| to a measured budget. | ||
|
|
||
| The UI also decides for itself which URL schemes a record body may put in a | ||
| link. `javascript:`, `data:`, `file:` and protocol-relative targets render as | ||
| their own text instead of becoming a link. React 19 already blocked | ||
| `javascript:`, so nothing here was exploitable — but a record body is written | ||
| by whichever agent held the card, and that defence belongs to this package | ||
| rather than to a dependency's minor version. |
42 changes: 42 additions & 0 deletions
42
...-0024-a-static-analyser-reports-ambiguity-only-a-measurement-tells-you-which.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| --- | ||
| id: LRN-0024 | ||
| title: A static analyser reports ambiguity; only a measurement tells you which half is real | ||
| status: active | ||
| confidence: high | ||
| created: 2026-08-05 | ||
| updated: 2026-08-05 | ||
| --- | ||
|
|
||
| Ten CodeQL alerts sat open on main, every one rated `high` by its rule. The | ||
| temptation in both directions is the same mistake: fix them all to clear the | ||
| tab, or dismiss them all because static analysis is noisy. [[T-0162]] said the | ||
| work was "saying which", and that turned out to be literal — the answer came | ||
| from a benchmark, not from reading the code. | ||
|
|
||
| Same rule, `js/polynomial-redos`, four sites, at 16× the input: | ||
|
|
||
| | site | growth | | ||
| |---|---| | ||
| | `acceptance.ts` ITEM | 1× | | ||
| | `records/index.ts` QUERY_TOKEN | 1× | | ||
| | `docs/validation.ts` link scan | **217×** | | ||
| | `docs/docs.ts` trailing slash | **230×** | | ||
|
|
||
| Two noise, two quadratic — and the quadratic ones were bad in a way reading had | ||
| not suggested: 43.6 seconds on a 128KB document body, in the scan the doctor | ||
| runs over every document in the workspace. | ||
|
|
||
| The reverse error was there too. `js/regex-injection` in `search.ts` looked | ||
| handled: a 256-character pattern cap, a flag allowlist, a 20,000-character body | ||
| cap. Three guards, none of which bounds backtracking. `/(a+)+$/` is six | ||
| characters and takes 57 seconds against a 32-character body ([[T-0190]]). | ||
|
|
||
| **A static analyser reports that a shape *can* be exploited. Whether it is | ||
| depends on the engine, the caps around it and the input that reaches it — three | ||
| things the analyser does not know and a benchmark answers in minutes.** The | ||
| measurement is also what makes a dismissal honest: "false positive" alone is an | ||
| opinion, and the same sentence with numbers in it is a claim someone can check. | ||
|
|
||
| Cheap enough that there is no excuse: each of these was a five-line script. | ||
| Related to [[LRN-0018]] — the gap is always narrower than it looks, and here it | ||
| was narrower in both directions at once. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.