Skip to content

feat(gonx): add static analysis dependency detection#99

Open
chadxz wants to merge 2 commits intonaxodev:mainfrom
chadxz:feat/gonx-static-analysis-dependency-detection
Open

feat(gonx): add static analysis dependency detection#99
chadxz wants to merge 2 commits intonaxodev:mainfrom
chadxz:feat/gonx-static-analysis-dependency-detection

Conversation

@chadxz
Copy link

@chadxz chadxz commented Jan 27, 2026

Why?

The existing go-runtime dependency detection requires Go to be
installed and a go.work file to be configured. This is problematic
for teams where not everyone has Go installed (e.g., frontend devs)
or CI environments without Go.

This adds a new static-analysis strategy that uses tree-sitter
WASM to parse Go source files directly, requiring no Go toolchain.

Addresses #98

How?

  • Added dependencyStrategy option: 'go-runtime' | 'static-analysis'
    | 'auto'
  • Implemented tree-sitter based import extraction (parser-init.ts,
    extract-imports.ts)
  • Implemented go.mod parsing for module paths and replace directives
    (parse-go-mod.ts)
  • Built import map with per-project replace directive scoping
    (build-import-map.ts)
  • Used longest-prefix matching for import resolution with caching
    (resolve-import.ts)
  • Added 'auto' strategy that tries go-runtime first, falls back to
    static-analysis with warning logging
  • 206 unit tests covering all components
  • E2E tests for full integration
  • Documentation in docs/static-analysis.md

@nx-cloud
Copy link

nx-cloud bot commented Jan 27, 2026

View your CI Pipeline Execution ↗ for commit 2d2dff5

Command Status Duration Result
nx affected -t lint test ❌ Failed 11s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 4s View ↗

☁️ Nx Cloud last updated this comment at 2026-02-09 18:02:09 UTC

@NachoVazquez
Copy link
Contributor

Thanks @chadxz ! I really appreciate it. I will take a deeper look in the morning.

@NachoVazquez
Copy link
Contributor

For now, there are some issues in the pipeline. Let me know if you need anything from me there.

@chadxz
Copy link
Author

chadxz commented Jan 27, 2026

I don’t know what’s wrong with the tests, they’re running fine for me locally, I’ll have to take a look tomorrow.

@NachoVazquez
Copy link
Contributor

It can be a Windows thing.

Copy link
Contributor

@NachoVazquez NachoVazquez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Praise: This is a great PR. I think most of my observations are around the docs and making sure we don't create confusion about the need for a go work with the default dependency resolution technique, but the rest looks good!

The static analysis strategy is useful when:

- Go is not installed (CI environments, frontend-only developers)
- You don't use `go.work` files
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: I don't think this assertion is true. This plugin was designed to work without the go.work by default. You can see it here https://youtu.be/_5aXT0-xCyA?t=85

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will have to investigate. Initial investigation made me think it was required.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested go list -m -json in a monorepo without go.work - it returns {"Path": "command-line-arguments"} with exit code 0. go-runtime is silently detecting zero dependencies rather than failing.

I saw in a previous commit that some work was done to remove dependence on go.work for build and serve executors, but perhaps still dependency detection still relies on go.work at the root of the monorepo?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh interesting, I might have missed this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, you are absolutely right. go list -m -json only works when there is a go.work or a root go.mod.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this does it. Let's use your strategy as the default, then.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's nuke the old way of doing things, since I'm moving away from needing a go.work on this project.

Thank you @chadxz , this is a great discovery.


- Go is not installed (CI environments, frontend-only developers)
- You don't use `go.work` files
- You want faster dependency detection
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: I also beleive this is true, but it would be awesome to benchmark both approaches so we can give some solid numbers on how fast it is.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be most notable in monorepos with many go projects as the overhead would be in spawning the processes. I'll see if i can come up with something.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preliminary benchmarks show that go list is surprisingly fast but file io dominates at larger scale which doesn't change for either solution. So performance-wise this is a lateral move overall. Probably should just omit the performance discussion from the docs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! Thank you for your research!

*
* This is an alternative to the go-runtime strategy that:
* - Does NOT require Go to be installed
* - Does NOT require go.work file
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: let's skip this one, since the go-runtime doesn't need go.work either.

* Cache for sorted module paths to avoid re-sorting on every resolve call.
* Key is the Map reference (via WeakMap), value is the sorted array.
*/
const sortedPathsCache = new WeakMap<Map<string, unknown>, string[]>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: nice optimization

Comment on lines 28 to 35
/**
* Strategy for detecting dependencies between Go projects.
*
* - `go-runtime` (default): Uses `go list -m -json` command (requires Go installed)
* - `static-analysis`: Uses tree-sitter WASM parsing (no Go required)
* - `auto`: Tries go-runtime first, falls back to static-analysis if Go unavailable
*/
dependencyStrategy?: DependencyStrategy;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Praise: Nice, I don't rule out moving to static analysis in the next major release if it is stable for everyone.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are actually making it the default now! Good work.

@NachoVazquez
Copy link
Contributor

Hey @chadxz, just to take it from where we left. In my mind, the missing piece here is to replace the existing dependency detection with yours; we don't need to keep the old strategy.

@chadxz
Copy link
Author

chadxz commented Feb 2, 2026

Understood. Also need to address ci failure

@NachoVazquez
Copy link
Contributor

Understood. Also need to address ci failure

Yeah, hopefully it is only a missing dependency that we need to declare explicitly.

@chadxz chadxz force-pushed the feat/gonx-static-analysis-dependency-detection branch 2 times, most recently from c251c09 to 77d7601 Compare February 5, 2026 23:35
Why?
====

The existing go-runtime dependency detection requires Go to be
installed and a go.work file to be configured. This is
problematic for teams where not everyone has Go installed
(e.g., frontend devs) or CI environments without Go.

This adds a static-analysis strategy that uses tree-sitter WASM
to parse Go source files directly, requiring no Go toolchain.

How?
====

- Chose web-tree-sitter (WASM) over tree-sitter (native) to
  avoid native compilation requirements — the WASM build runs
  everywhere Node.js does with no platform-specific binaries.
- Investigated web-tree-sitter's module export shape across
  environments (native Node.js vs ts-jest). Found that ts-jest
  returns the raw Emscripten Module instead of the expected
  named exports. Solved with a static import in production
  code and a jest.mock normalizing the module shape in tests,
  keeping type assertions out of production code entirely.
- Validated go.mod parsing against the Go module reference
  spec, covering quoted/unquoted module paths, single-line
  and block replace directives, versioned replacements, and
  inline/multi-line comments.
- Verified import extraction handles all Go import patterns:
  single, grouped, aliased, dot, blank, raw string literals,
  and cgo pseudo-import filtering.
- Used longest-prefix matching for import resolution with
  per-project replace directive scoping and caching.
- Adopted jest.mock('fs/promises') patterns matching the rest
  of the gonx test suite instead of introducing memfs as a
  new dependency.
- 206 unit tests and E2E tests pass locally; format and build
  checks clean.

----

Addresses naxodev#98.
@chadxz chadxz force-pushed the feat/gonx-static-analysis-dependency-detection branch from 77d7601 to dcd08b6 Compare February 6, 2026 00:15
@chadxz
Copy link
Author

chadxz commented Feb 7, 2026

@NachoVazquez I've pushed a few commits that I think may address the ci failure. I've got the refactor to remove the existing implementation in favor of this new one but I ran across an issue with the web-tree-sitter types that I have submitted a fix for to clean up. Not a blocker, though.

@NachoVazquez
Copy link
Contributor

Thanks, @chadxz. I was out for the weekend, so I didn't reply before. Taking a look at the new changes.

@NachoVazquez
Copy link
Contributor

I see we have new issues now, that's progress, also I think we are still mentioning the old go mod list method as the default, let's change that, actually, my idea is to completly remove the old dependency calculation strategy and use yours as the only one.

@chadxz
Copy link
Author

chadxz commented Feb 9, 2026

@NachoVazquez I pushed the commit replacing the dependency resolution with the tree-sitter implementation. This should also address the current CI failure.

Why?
====

The go-runtime strategy required Go to be installed and a go.work file
to function, making it unusable in CI environments or for frontend
developers. The static-analysis strategy (tree-sitter) has no such
requirements and produces equivalent results. Maintaining three
strategies (go-runtime, static-analysis, auto) added complexity
without meaningful benefit.

How?
====

- Removed go-runtime strategy, auto strategy, and the
  dependencyStrategy option entirely
- Made static analysis the only code path in createDependencies
- Deleted all go-runtime utility files and associated types
- Added unit tests for the static-analysis index module
- Simplified e2e tests to remove strategy-specific describe blocks
- Updated docs and README to reflect the single approach
@chadxz chadxz force-pushed the feat/gonx-static-analysis-dependency-detection branch from ffd2999 to 2d2dff5 Compare February 9, 2026 14:45
@NachoVazquez
Copy link
Contributor

@chadxz, the tests are failing again. As a test, comment out the Windows runner from the matrix. Let's see if the issue is reproducible outside of Windows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants