Use cjs/es-module-lexer instead of acorn

[tjallingt]

Given that @guybedford is involved in nft I don't expect this to be a novel idea but I'd like to discuss it anyway.
I think it would be possible to use these two modules to detect all requires instead of depending on acorn and walking the AST. The main advantage would be that it's likely faster and would make nft smaller/simpler.

https://github.com/guybedford/cjs-module-lexer
https://github.com/guybedford/es-module-lexer

Unfortunately cjs-module-lexer doesn't output required files but this PR adds support for it: nodejs/cjs-module-lexer#10

The main blocker for that PR appears to be that detecting requires is somewhat of an unbounded problem, however clear boundaries in what can/cannot be detected might help both cjs-module-lexer and nft.

[styfle]

@tjallingt Sounds good to me those two packages can pass all of our existing tests 👍

@guybedford Any pros/cons to these two packages vs acorn?

[guybedford]

@tjallingt are you finding performance limitations with this project? The obvious first step if there are perf bottlenecks might be exploring threading models since analysis is per-file anyway.

Using the lexers could certainly improve performance, but most of the performance cost of the analysis is actually the asset expression analysis. Moving this layer into a lower level language would be a great effort too. If I could get an intern I'd put them on it :P

[tjallingt]

Sorry for not replying earlier. I have no issues with nft performance but on a glance it appeared to me the lexers and nft where closely related in how they function (finding imports).

However I clearly didn't think it through very carefully though as I did see nft pickup on other files (fs.readFile).

Indeed while a custom lexer might help perf a little, the static evaluation is much more tricky and core to what nft does so my suggestion is somewhat non-sensical.