Skip to content

fix(validate): trailing comments read as requests; waivers covered whole files - #111

Merged
pitimon merged 1 commit into
mainfrom
fix/110-comment-and-pinned-waivers
Jul 25, 2026
Merged

fix(validate): trailing comments read as requests; waivers covered whole files#111
pitimon merged 1 commit into
mainfrom
fix/110-comment-and-pinned-waivers

Conversation

@pitimon

@pitimon pitimon commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Summary

Closes items 4 and 3 of #110, in the order the issue set: both protect the control itself rather than extending its reach.

  • Item 4 — a URL in a trailing comment counted as a request. const x = 5; // see https://github.com/nodejs/node/issues/123 failed the check. Reference links after code are common, and the issue named this the likeliest source of pressure to switch the whole thing off. Comments are now positional ranges scanned with string state tracked — the token being looked for (//) is the one every URL contains, so fetch("https://x") must not read as a comment.
  • Item 3 — link_from/data_from waived a whole file, permanently. Waivers are now pinned to the URL literal ({"file": ..., "url": ...}). A bare filename is rejected; a pin matching nothing is reported as stale.

What this catches that main does not

The scenario #110 describes, which is the shape of #100: adding an owner-avatar <img src> to SkillDetailPanel.jsx — a plausible "show skill authors" change — to a file that already holds a github.com link waiver and already interpolates owner names. On main it passes silently. Here it is a finding, and closing it is a visible diff in outbound-hosts.json that a reviewer has to approve.

Two defects surfaced while doing it

  • The request map was keyed by file, so a second request to the same host from one file overwrote the first — only the last line was ever reported. That is why src/lib/skills-manager.js:371 was invisible; the real count is 9 records, not 8.
  • A pre-existing over-exemption in the dangerous direction: /* note */ fetch("https://x") starts with /*, so COMMENT_LINE_RE exempted the whole line, live call included. That regex now covers only the JSDoc * continuation line.

Design note — why line-local

No block-comment state is carried across lines. A regex literal like /[/*]/ opens a block comment as far as this scanner is concerned; carrying that forward would silence the rest of the file, which is a miss. Confined to one line the same mistake costs at most one false finding — visible and fixable. There is a test for exactly this.

Test plan

  • ci:local exit 0 — 891 root tests (+8), 256 dashboard
  • validate:outbound green: 24 declared hosts
  • Behaviour-preserving on the real tree: old and new scanners return the identical 78 mention/request entries — nothing lost, nothing gained. Differences appear only against fixtures.
  • checkOutbound was 113 lines (over the 50 rule); split into checkRequestPermission (45) and checkReadmeTable (12), leaving it at 41
  • New tests cover: trailing comment is a mention · // inside a string is not a comment · code after a closed block comment is still a request · an unterminated /* stops at its own line · a waiver covers the URL not the file · every request per file is reported · a stale pin is reported · a bare filename is rejected

Not in this PR

#110 items 1, 2, 5, 6, 7 remain open and are unchanged.

…ole files

Closes the two items #110 put first, for the reason it gave: both protect the
control itself rather than extending it.

ITEM 4 — a URL in a trailing comment was a request.

  const x = 5; // see nodejs/node#123

COMMENT_LINE_RE only matched a comment that STARTS the line. Reference links
after code are common, and a check that flags ordinary code is a check someone
switches off — which would take the rest of this control with it.

Comments are now positional ranges, scanned with string state tracked, because
the token being looked for (`//`) is the one every URL contains. Deliberately
line-local: a regex literal like /[/*]/ opens a block comment as far as this
scanner is concerned, and carrying that state across lines would silence the
rest of the file — a miss, the direction this check must never fail in.

That also fixed a pre-existing over-exemption in the opposite direction:
`/* note */ fetch("https://x")` began with `/*`, so the whole line — live call
included — was exempt. COMMENT_LINE_RE now covers only the JSDoc `*`
continuation line, the one shape ranges cannot see.

ITEM 3 — link_from/data_from waived the whole file, forever.

Once a file was listed, any later request to that host from it passed. The
concrete scenario is the shape of #100: adding an owner-avatar <img src> to
SkillDetailPanel.jsx — a plausible "show skill authors" change — to a file that
already holds a github.com link waiver and already interpolates owner names.

Waivers are now pinned to the URL literal: {"file": ..., "url": ...}. Pinned to
the literal rather than a line number because line numbers move under every edit
above them, and a waiver that churns is one people rubber-stamp. A bare filename
is rejected outright, and a pin matching nothing is reported as stale.

Two defects surfaced while doing it:
- The request map was keyed by file, so a second call to the same host from one
  file overwrote the first. Only the last line was ever reported — which is why
  skills-manager.js:371 was invisible. Now every occurrence is a record.
- checkOutbound reached 113 lines; split into checkRequestPermission (45) and
  checkReadmeTable (12), leaving it at 41.

Verified behaviour-preserving on the real tree: the old and new scanners return
the identical 78 mention/request entries — nothing lost, nothing gained. The
differences appear only against the fixtures.

8 new tests, including that `//` inside a string is not a comment and that code
after a CLOSED block comment on the same line is still a request.

ci:local exit 0: 891 root tests, 256 dashboard.
@pitimon
pitimon merged commit e5dc56c into main Jul 25, 2026
1 check passed
@pitimon
pitimon deleted the fix/110-comment-and-pinned-waivers branch July 25, 2026 21:53
pitimon added a commit that referenced this pull request Jul 25, 2026
…ose (#114)

Closes #110 items 2, 6 and 1. Stacked on #111.

ITEM 2 — the walker saw two directories and one extension set.

Added: dashboard/*.html (the served shell — a preconnect, font CDN or script
src here is a browser request), .css (skipped by EXTENSION while the walker was
already standing in that directory), i18n .json, dashboard/vite.config.js, and
scripts/. Maintainer-run is a caveat for the purpose field, not a reason to be
invisible.

That surfaced eight things nothing had ever mentioned, each read in source
before being classified:

  scripts/build-pricing-seed.cjs   really fetches the LiteLLM list -> request_from
  seed-snapshot.json               records that URL as a value      -> data_from
  dashboard/vite.config.js         proxies /proxy/ipcheck           -> request_from
  dashboard/index.html             footer links to the upstream repo -> link_from
  www.tokentracker.cc              JSON-LD metadata, not fetched     -> new entry
  www.npmjs.com                    same block                        -> new entry
  schema.org                       JSON-LD @context identifier       -> ignored
  example.invalid                  reserved never-resolving TLD      -> ignored

ITEM 6 — the inventory's prose was unchecked.

`user_data: true` with `readme: false` passed; so did a wrong `from` and purpose
text describing something the code does not do. The fields a reader trusts most
were the ones nothing verified. Types and required values are now checked, and
`from` is checked against WHERE THE HOST IS ACTUALLY REACHED — a host only ever
requested from src/ cannot honestly be `browser`.

ITEM 1 — unresolvable authorities were dropped.

The issue's three examples (bracketed IPv6, a Cyrillic lookalike, a single-label
intranet host) all resolve now; they predate deferring to new URL(). What is
left is interpolation that pins nothing, which is ordinary code — which is why
the flag-flip version had to be reverted inside #109.

So each is declared with a why, matched on file + authority text, and a new one
fails. 15 declared across 8 files: the local bind and Host-header parsing, git
remote normalisation into project_ref, the user's own HTTPS_PROXY, dev-server
proxying, and this scanner's own probe strings.

Also: checkOutbound and collectHosts were both over the 50-line rule; split into
scanLine (39), checkUnresolved (30) and checkInventoryMetadata (29), leaving
them at 46 and 21. File is 640 lines.

19 new tests. The metadata check was proved non-vacuous against five
deliberately-wrong entries before being committed, with a valid control.

ci:local exit 0: 903 root tests, 256 dashboard.

Co-authored-by: itarun.p <itarun.p@somapait.com>
pitimon added a commit that referenced this pull request Jul 26, 2026
15 commits since v0.39.43. Tracker: #125.

User-facing:
- The Projects panel honours the date range. It read no query parameter at
  all, so picking "24h" narrowed every other card while Projects kept showing
  all-time totals with nothing on screen saying so (#118).
- Per-repo cost, with rows written before the change named as unpriced rather
  than shown as a confident $0 (#121).
- Sources with no per-repo attribution are named, instead of their absence
  reading as "that tool cost nothing here" (#118).
- A plan-value card answering what README:32 already promised, labelled as
  list-price-equivalent rather than a saving (#122).
- The Codex quota chip no longer vanishes on a 401 (#119).
- `sync --compact`, which reclaimed 34,924 lines to 5,636 on a real install
  (#117).

Behind the scenes: outbound-privacy validator hardening (#111, #114), avatar
proxy per-hop revalidation, port-aware allowlisting and a real download cap
(#109, #112), a parser conformance ratchet (#116, #120), scripts/graph out of
the product gate (#115), and a usage-limits fixture capture tool (#124).

Version bumped in all four lockstep locations by the `version` hook:
package.json, package-lock.json, both TokenTrackerBar targets, and the Windows
csproj. validate:version-lockstep passes.

WATCH AFTER SHIPPING: #121 migrates cursors.json on every user's first sync
after upgrading, re-keying project buckets from project|source|hour to
project|source|model|hour. Without that the old bucket strands and its usage is
counted twice. Verified read-only against a real 2,015-bucket state — every key
migrated, total_tokens preserved exactly at 6,281,653,062, and all 2,015
queuedKey markers survived, the loss of which would re-append every row. That
is one machine; a differently-shaped state is the residual risk.

ci:local exit 0: 972 root tests, 302 dashboard.

Co-authored-by: itarun.p <itarun.p@somapait.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant