Can you please add support for dynamically loading (any) grammar libraries at runtime? The current approach of vendoring every existing grammar and linking them in build-time is not scalable and not compatible with linux distributions.
This issue has been already solved e.g. in diffsitter (afnanenayet/diffsitter#177) or helix (helix-editor/helix#432).
You can either load a grammar library for the specified language from the system library path (this is typically /lib:/usr/lib:/usr/local/lib, but the path may vary between linux distros; the point is that you don’t depend on exact path, this is handled by libc) as any other system library – grammar libraries are expected to be named libtree-sitter-<lang>.so (diffsitter’s approach).
Or you can look up grammar libraries directly in some predefined directory (e.g. /usr/lib/tree-sitter or some app-specific dir) named as <lang>.so (helix’s approach).
The former is better, but both are acceptable for Linux distros – in the latter case, we can just symlink <your-directory-for-grammars> to e.g. /usr/lib/tree-sitter where system-wide grammars are installed by the system’s package manager (and used by all programs that use tree-sitter). These approaches are not mutually exclusive – you can support both.
Also, you can continue to provide some grammars with tree-grepper (as both diffsitter and helix do, they use git submodules) for the convenience of the users that build tree-grepper themselves and their distro don’t provide tree-sitter grammars yet (for example, Alpine Linux and Arch Linux (AUR) already provide many grammars). The aim is to allow package maintainers to package grammars separately and give users the freedom to choose what grammars they will install.
(I opened the same issue also in Wilfred/difftastic#123)
Can you please add support for dynamically loading (any) grammar libraries at runtime? The current approach of vendoring every existing grammar and linking them in build-time is not scalable and not compatible with linux distributions.
This issue has been already solved e.g. in diffsitter (afnanenayet/diffsitter#177) or helix (helix-editor/helix#432).
You can either load a grammar library for the specified language from the system library path (this is typically
/lib:/usr/lib:/usr/local/lib, but the path may vary between linux distros; the point is that you don’t depend on exact path, this is handled by libc) as any other system library – grammar libraries are expected to be namedlibtree-sitter-<lang>.so(diffsitter’s approach).Or you can look up grammar libraries directly in some predefined directory (e.g.
/usr/lib/tree-sitteror some app-specific dir) named as<lang>.so(helix’s approach).The former is better, but both are acceptable for Linux distros – in the latter case, we can just symlink
<your-directory-for-grammars>to e.g./usr/lib/tree-sitterwhere system-wide grammars are installed by the system’s package manager (and used by all programs that use tree-sitter). These approaches are not mutually exclusive – you can support both.Also, you can continue to provide some grammars with tree-grepper (as both diffsitter and helix do, they use git submodules) for the convenience of the users that build tree-grepper themselves and their distro don’t provide tree-sitter grammars yet (for example, Alpine Linux and Arch Linux (AUR) already provide many grammars). The aim is to allow package maintainers to package grammars separately and give users the freedom to choose what grammars they will install.
(I opened the same issue also in Wilfred/difftastic#123)