Skip to content

feat(extractor): resolve const namespace identifiers and NS.KEY member access#202

Merged
bdshadow merged 6 commits into
mainfrom
bdshadow/resolve-const-namespace
Jun 25, 2026
Merged

feat(extractor): resolve const namespace identifiers and NS.KEY member access#202
bdshadow merged 6 commits into
mainfrom
bdshadow/resolve-const-namespace

Conversation

@bdshadow

@bdshadow bdshadow commented May 21, 2026

Copy link
Copy Markdown
Member

Problem

The extractor warns W_DYNAMIC_NAMESPACE whenever a namespace is
referenced through a constant instead of a literal:

const namespaceName = 'myNamespace';
const { t } = useTranslate(namespaceName); // ← W_DYNAMIC_NAMESPACE

const NS = { AUTH: 'auth' } as const;
const { t } = useTranslate(NS.AUTH);       // ← W_DYNAMIC_NAMESPACE

This is exactly the shape customers reach for to avoid namespace typos
(a typed constants object on top of which TypeScript catches typos).
Today, choosing safety means losing extraction. The only workarounds
are bare literals everywhere or @tolgee-key magic comments at every
call site.

Solution

A small per-file constant pre-pass runs before parsing and builds a
flat Map<string, string>. It walks the already-merged token stream
(not regex over raw source, and not the TS AST), using a small state
machine with a funcDepth counter so only module-top-level
declarations are captured:

  • const NAME = 'literal' (as const)?NAME → 'literal'
  • const NS = { K1: 'l1', K2: 'l2' } as constNS.K1 → 'l1', NS.K2 → 'l2'
  • Object-literal form supports the non-as const variant too. Nested
    objects are intentionally skipped (existing dynamic warning still
    fires).
  • Duplicate keys within an object literal resolve last-write-wins,
    matching JS object semantics.

The map is threaded through ParserContext.constants. getValue.ts
resolves bare variable tokens via the map, and a one-step lookahead
catches the variable + dot + variable (member-access) pattern.

A few additional TextMate scopes (variable.other.constant.object.ts,
variable.other.constant.property.ts, variable.other.property.ts)
are mapped to the variable customType — without that, tokens inside
useTranslate(NS.K) arrive with customType: undefined and bypass
parsing entirely.

Scope and limitations

  • Same-file only. Cross-file import { NS } from './namespaces'
    is still warned. Following imports is a fair chunk more work (module
    resolution, alias paths, ts-morph or equivalent) and explicitly out
    of scope for this PR.
  • Member access is limited to one level (NS.KEY). Deeper paths still
    warn.
  • Only module-top-level const declarations are captured; a
    const X = '...' nested inside a function body or branch is invisible
    to the pre-pass, matching the visibility the call site would have.
  • Nested object literals are skipped (only flat string-valued objects
    are flattened into the map).

Verification

  • npm run test:unit — 646 passed.
  • Specifically:
    • test/unit/extractor/react/useTranslate.test.ts — 140 tests.
    • test/unit/extractor/react/tComponent.test.ts — 42 tests.
  • Vue / Svelte / Ngx parser suites all remain green (shared parser
    files were modified).
  • End-to-end on the companion tolgee-js testapp Namespaces.tsx:
    3 keys extracted, namespace namespaced, zero warnings.

Test plan

  • Customer pattern reproduces and warning is gone for the same-file
    shape.
  • No regression for the existing dynamic-key/dynamic-default-value
    paths.
  • Cross-file case is documented as a known limitation (PR
    description and the new extractConstants.ts doc comment).

Summary by CodeRabbit

  • New Features

    • Extractor now resolves module-level constant declarations when analyzing component props and function arguments, supporting plain constants, const-asserted declarations, and nested object properties for improved namespace and key extraction.
  • Tests

    • Added comprehensive test coverage for constant resolution scenarios, including various declaration patterns, duplicate key handling, and negative cases.

@coderabbitai

coderabbitai Bot commented May 21, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

A new token-stream pre-pass (extractConstants) builds a Map<string, string> of module-top-level const bindings using a depth-aware state machine. This map is stored in ParserContext.constants and consulted in getValue to resolve variable and NS.MEMBER member-access tokens to string primitives instead of opaque expr placeholders.

Changes

Top-level constant namespace resolution

Layer / File(s) Summary
ParserContext constants field and token type mapping
src/extractor/parser/types.ts, src/extractor/parser/generalMapper.ts
ParserContext gains a constants: Map<string, string> field; generalMapper adds fall-through cases for variable.other.constant.object.ts, variable.other.constant.property.ts, and variable.other.property.ts to map to the 'variable' category.
extractConstants token-stream state machine
src/extractor/parser/extractConstants.ts
New module with inline docs, internal State/Context types, createContext, capture lifecycle helpers, depth helpers, stepInObjectBody (object-literal key/value capture with nested-block abandon), stepAtTopLevel (state machine across const stages), and the exported extractConstants entry point that returns the populated Map.
Parser wires extractConstants into context
src/extractor/parser/parser.ts
Imports extractConstants and assigns context.constants from the filtered post-merge token array during parse.
Variable and member-access resolution in getValue
src/extractor/parser/tree/getValue.ts
New case 'variable' peeks for acessor.dot + variable to detect NAME.PROP, consumes matched tokens, and resolves via context.constants; returns primitive on a hit or expr on a miss.
Unit tests for namespace resolution
test/unit/extractor/react/tComponent.test.ts, test/unit/extractor/react/useTranslate.test.ts
New describe('top-level const namespace identifiers') suites for both T component and useTranslate cover plain consts, export … as const, const-object member access, last-write-wins for duplicate object keys, an array-destructure regression, and negative cases for function-local consts and unknown identifiers/properties.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐇 Hop, hop through the token stream I go,
Plucking constants from the module's flow!
const NS = { KEY: 'value' } as const — found!
Now namespaces resolve safe and sound.
No more expr when a primitive will do,
The rabbit mapped your consts — the key came through! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 36.36% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: resolving const namespace identifiers and NS.KEY member access in the extractor, matching the core feature addition across multiple files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch bdshadow/resolve-const-namespace

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 34965941-3241-4bf0-bf2e-cb2cc2dfca4b

📥 Commits

Reviewing files that changed from the base of the PR and between c1652fc and 0328422.

📒 Files selected for processing (8)
  • src/extractor/extractor.ts
  • src/extractor/parser/extractConstants.ts
  • src/extractor/parser/generalMapper.ts
  • src/extractor/parser/parser.ts
  • src/extractor/parser/tree/getValue.ts
  • src/extractor/parser/types.ts
  • test/unit/extractor/react/tComponent.test.ts
  • test/unit/extractor/react/useTranslate.test.ts

Comment thread src/extractor/parser/extractConstants.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/extractor/parser/extractConstants.ts`:
- Around line 89-95: The current loop in the non-abandoned branch uses
ctx.result.has(composed) to avoid overwriting existing entries, which preserves
the first-write instead of JavaScript's last-write-wins semantics; change the
behavior in the block that iterates ctx.captureProps so that for each { key,
value } you always assign into ctx.result (e.g., call ctx.result.set(composed,
value) unconditionally) so later properties on the same captured object
(identified by ctx.captureName + '.' + key) overwrite earlier ones; keep the
check for ctx.abandonedObject as-is so this change only affects the
non-abandoned path.
- Around line 166-170: The code incorrectly treats a list.begin token like a
block by setting ctx.funcDepth = 1 (in the branch handling "c === 'block.begin'
|| c === 'list.begin'"), which relies on stepInFunctionBody() to consume the
matching end token but stepInFunctionBody() never consumes list.end; change the
logic so that list.begin is handled separately from block.begin: do not set
ctx.funcDepth when c === 'list.begin'—instead skip the array/destructure pattern
until you see the corresponding 'list.end' token (or use a short local skip
loop) and then resume with ctx.state = 'Idle'; keep the existing funcDepth
behavior only for block.begin so top-level array destructures (e.g. const [x] =
...) do not leave ctx.funcDepth > 0 and block extraction continues correctly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 935005c8-8f92-4862-a3dd-b7f94fb1b6ba

📥 Commits

Reviewing files that changed from the base of the PR and between 4f5c425 and 880dfe1.

📒 Files selected for processing (1)
  • src/extractor/parser/extractConstants.ts

Comment thread src/extractor/parser/extractConstants.ts
Comment thread src/extractor/parser/extractConstants.ts Outdated
@bdshadow
bdshadow marked this pull request as ready for review May 25, 2026 08:54
@bdshadow
bdshadow requested review from JanCizmar and dkrizan May 25, 2026 08:55
bdshadow added 6 commits June 22, 2026 14:06
When useTranslate(NS), <T ns={NS}>, or t('key', { ns: NS }) refers to
a top-level `const NS = '...'` (optionally annotated as const), the
extractor now substitutes the literal value instead of emitting
W_DYNAMIC_NAMESPACE. Lets users keep namespaces in a shared constants
file without forcing them to inline string literals everywhere.
Builds on top of the bare-identifier resolution: extractConstants now
also flattens `const NS = { K: 'literal' } as const` (and the plain
non-`as const` form) into `NS.K -> 'literal'` entries, and
getValue consumes `variable + dot + variable` sequences to look them
up.

Also maps the TextMate scopes used for member access in argument
position (`variable.other.constant.object.ts`,
`variable.other.constant.property.ts`, `variable.other.property.ts`)
to the `variable` customType so they reach the parser at all.

Cross-file imports of constants files still warn — Path A is
same-file only.
Walks the already-merged TextMate token stream instead of regex-scanning
the raw source. Strings, comments and template literals are no longer
ambiguous (TextMate has classified them upstream), and a funcDepth
counter ensures we only capture declarations at module top level.

This addresses the scope-shadowing concern raised in PR review: a
`const NS` declared inside a function body no longer leaks into the
top-level constants map. Added a regression test pinning the behaviour.

No behavioural change for the documented happy paths; existing 249
react tests plus the new case pass.
Each sub-state of the walker now has its own function:
- stepInFunctionBody  (skip-and-track block depth)
- stepInObjectBody    (object literal property capture)
- stepAtTopLevel      (the main const-declaration FSM)
- finalizeObjectCapture / resetCapture / createContext

extractConstants() is now a small dispatch loop. No behaviour change;
all existing tests pass.
The finalize loop was guarding with `!result.has(composed)`, which
preserved the first-encountered value for duplicate keys within a
single `const NS = { ... }` literal. JS object-literal semantics are
last-write-wins, so the extractor's output diverged from what the
runtime actually sees. Always assign and let later properties win.

The cross-declaration guard in stepAtTopLevel (`const X = '...'`
followed by another `const X = '...'`) is unrelated and stays — a
re-declaration is a TS error anyway, and first-write is safer.
The AfterConst branch reused funcDepth for array-destructure
patterns (`const [a, b] = ...`), but stepInFunctionBody only
consumes block.begin/end. The matching list.end was therefore
ignored, leaving funcDepth permanently > 0 and silently skipping
every subsequent top-level const declaration.

Track list-bracket depth in its own listDepth counter with a
matching stepInListSkip dispatcher. Object destructure
(`const { x } = ...`) keeps the existing funcDepth path.
Regression test added.
@bdshadow
bdshadow force-pushed the bdshadow/resolve-const-namespace branch from b23a585 to e140b82 Compare June 22, 2026 12:09
@bdshadow
bdshadow merged commit 51e16fc into main Jun 25, 2026
16 checks passed
@bdshadow
bdshadow deleted the bdshadow/resolve-const-namespace branch June 25, 2026 14:46
TolgeeMachine added a commit that referenced this pull request Jun 25, 2026
# [2.20.0](v2.19.0...v2.20.0) (2026-06-25)

### Features

* **extractor:** resolve const namespace identifiers and NS.KEY member access ([#202](#202)) ([51e16fc](51e16fc))
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 2.20.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants