feat(gonx): add static analysis dependency detection#99
feat(gonx): add static analysis dependency detection#99chadxz wants to merge 2 commits intonaxodev:mainfrom
Conversation
|
View your CI Pipeline Execution ↗ for commit 2d2dff5
☁️ Nx Cloud last updated this comment at |
|
Thanks @chadxz ! I really appreciate it. I will take a deeper look in the morning. |
|
For now, there are some issues in the pipeline. Let me know if you need anything from me there. |
|
I don’t know what’s wrong with the tests, they’re running fine for me locally, I’ll have to take a look tomorrow. |
|
It can be a Windows thing. |
NachoVazquez
left a comment
There was a problem hiding this comment.
Praise: This is a great PR. I think most of my observations are around the docs and making sure we don't create confusion about the need for a go work with the default dependency resolution technique, but the rest looks good!
| The static analysis strategy is useful when: | ||
|
|
||
| - Go is not installed (CI environments, frontend-only developers) | ||
| - You don't use `go.work` files |
There was a problem hiding this comment.
suggestion: I don't think this assertion is true. This plugin was designed to work without the go.work by default. You can see it here https://youtu.be/_5aXT0-xCyA?t=85
There was a problem hiding this comment.
I will have to investigate. Initial investigation made me think it was required.
There was a problem hiding this comment.
I tested go list -m -json in a monorepo without go.work - it returns {"Path": "command-line-arguments"} with exit code 0. go-runtime is silently detecting zero dependencies rather than failing.
I saw in a previous commit that some work was done to remove dependence on go.work for build and serve executors, but perhaps still dependency detection still relies on go.work at the root of the monorepo?
There was a problem hiding this comment.
Oh interesting, I might have missed this.
There was a problem hiding this comment.
Oh, you are absolutely right. go list -m -json only works when there is a go.work or a root go.mod.
There was a problem hiding this comment.
Well, this does it. Let's use your strategy as the default, then.
There was a problem hiding this comment.
Let's nuke the old way of doing things, since I'm moving away from needing a go.work on this project.
Thank you @chadxz , this is a great discovery.
|
|
||
| - Go is not installed (CI environments, frontend-only developers) | ||
| - You don't use `go.work` files | ||
| - You want faster dependency detection |
There was a problem hiding this comment.
note: I also beleive this is true, but it would be awesome to benchmark both approaches so we can give some solid numbers on how fast it is.
There was a problem hiding this comment.
I think this would be most notable in monorepos with many go projects as the overhead would be in spawning the processes. I'll see if i can come up with something.
There was a problem hiding this comment.
preliminary benchmarks show that go list is surprisingly fast but file io dominates at larger scale which doesn't change for either solution. So performance-wise this is a lateral move overall. Probably should just omit the performance discussion from the docs.
There was a problem hiding this comment.
Sounds good! Thank you for your research!
| * | ||
| * This is an alternative to the go-runtime strategy that: | ||
| * - Does NOT require Go to be installed | ||
| * - Does NOT require go.work file |
There was a problem hiding this comment.
suggestion: let's skip this one, since the go-runtime doesn't need go.work either.
| * Cache for sorted module paths to avoid re-sorting on every resolve call. | ||
| * Key is the Map reference (via WeakMap), value is the sorted array. | ||
| */ | ||
| const sortedPathsCache = new WeakMap<Map<string, unknown>, string[]>(); |
There was a problem hiding this comment.
praise: nice optimization
| /** | ||
| * Strategy for detecting dependencies between Go projects. | ||
| * | ||
| * - `go-runtime` (default): Uses `go list -m -json` command (requires Go installed) | ||
| * - `static-analysis`: Uses tree-sitter WASM parsing (no Go required) | ||
| * - `auto`: Tries go-runtime first, falls back to static-analysis if Go unavailable | ||
| */ | ||
| dependencyStrategy?: DependencyStrategy; |
There was a problem hiding this comment.
Praise: Nice, I don't rule out moving to static analysis in the next major release if it is stable for everyone.
There was a problem hiding this comment.
We are actually making it the default now! Good work.
|
Hey @chadxz, just to take it from where we left. In my mind, the missing piece here is to replace the existing dependency detection with yours; we don't need to keep the old strategy. |
|
Understood. Also need to address ci failure |
Yeah, hopefully it is only a missing dependency that we need to declare explicitly. |
c251c09 to
77d7601
Compare
Why?
====
The existing go-runtime dependency detection requires Go to be
installed and a go.work file to be configured. This is
problematic for teams where not everyone has Go installed
(e.g., frontend devs) or CI environments without Go.
This adds a static-analysis strategy that uses tree-sitter WASM
to parse Go source files directly, requiring no Go toolchain.
How?
====
- Chose web-tree-sitter (WASM) over tree-sitter (native) to
avoid native compilation requirements — the WASM build runs
everywhere Node.js does with no platform-specific binaries.
- Investigated web-tree-sitter's module export shape across
environments (native Node.js vs ts-jest). Found that ts-jest
returns the raw Emscripten Module instead of the expected
named exports. Solved with a static import in production
code and a jest.mock normalizing the module shape in tests,
keeping type assertions out of production code entirely.
- Validated go.mod parsing against the Go module reference
spec, covering quoted/unquoted module paths, single-line
and block replace directives, versioned replacements, and
inline/multi-line comments.
- Verified import extraction handles all Go import patterns:
single, grouped, aliased, dot, blank, raw string literals,
and cgo pseudo-import filtering.
- Used longest-prefix matching for import resolution with
per-project replace directive scoping and caching.
- Adopted jest.mock('fs/promises') patterns matching the rest
of the gonx test suite instead of introducing memfs as a
new dependency.
- 206 unit tests and E2E tests pass locally; format and build
checks clean.
----
Addresses naxodev#98.
77d7601 to
dcd08b6
Compare
|
@NachoVazquez I've pushed a few commits that I think may address the ci failure. I've got the refactor to remove the existing implementation in favor of this new one but I ran across an issue with the web-tree-sitter types that I have submitted a fix for to clean up. Not a blocker, though. |
|
Thanks, @chadxz. I was out for the weekend, so I didn't reply before. Taking a look at the new changes. |
|
I see we have new issues now, that's progress, also I think we are still mentioning the old |
|
@NachoVazquez I pushed the commit replacing the dependency resolution with the tree-sitter implementation. This should also address the current CI failure. |
Why? ==== The go-runtime strategy required Go to be installed and a go.work file to function, making it unusable in CI environments or for frontend developers. The static-analysis strategy (tree-sitter) has no such requirements and produces equivalent results. Maintaining three strategies (go-runtime, static-analysis, auto) added complexity without meaningful benefit. How? ==== - Removed go-runtime strategy, auto strategy, and the dependencyStrategy option entirely - Made static analysis the only code path in createDependencies - Deleted all go-runtime utility files and associated types - Added unit tests for the static-analysis index module - Simplified e2e tests to remove strategy-specific describe blocks - Updated docs and README to reflect the single approach
ffd2999 to
2d2dff5
Compare
|
@chadxz, the tests are failing again. As a test, comment out the Windows runner from the matrix. Let's see if the issue is reproducible outside of Windows. |
Why?
The existing go-runtime dependency detection requires Go to be
installed and a go.work file to be configured. This is problematic
for teams where not everyone has Go installed (e.g., frontend devs)
or CI environments without Go.
This adds a new
static-analysisstrategy that uses tree-sitterWASM to parse Go source files directly, requiring no Go toolchain.
Addresses #98
How?
dependencyStrategyoption: 'go-runtime' | 'static-analysis'| 'auto'
extract-imports.ts)
(parse-go-mod.ts)
(build-import-map.ts)
(resolve-import.ts)
static-analysis with warning logging