Automatically build sources

[Th3Whit3Wolf]

This pull request does a couple things.

1. Any package inside ./pkgs (with the exception of ./pkgs/bud, ./pkgs/_sources and ./pkgs/default.nix will be called (as in callPackage) and can use inherit (source) pname version src so long as the name of parent folder for the default.nix or the name of the nix file stripped of .nix is the same name used in sources.toml
2. Start work on automatically building vimPlugins, vscode extension, and other development tools from prefixes in sources.toml Currently only vimPlugins are implemented. Addresses 321.

I recognize that flake.nix is not a great place for an overlay. I was unable to find a way to make an overlay that lets you import inputs, which means defining rakeTree and flattenTree before writing what's currently in the flake. It would also have had to be used as (import ./path/filename.nix).overlay.

[blaggacao]

So you do think an importPackages would be suitable addition to digga? Could be, but we'd have to make and discuss the use case over there.

[Th3Whit3Wolf]

Yes I do, I use the rakePkgs function exclusively to build the localPackages overlay. Which builds all packages in the ./pkgs directory so I never have to touch ./pkgs/default.nix when I add a new package. I do not know how to move it into digga and function properly.

[blaggacao]

we could make pkgs complely default.nix-less and add add nvfetcher stuff separately.

We could also consider to teach rakeLeaves to not consider basenames starting with an underscore.

[blaggacao]

Maybe you can have a quick look at the digga impl. With a little help, it shouldn't be too hard.
https://github.com/divnix/digga/blob/main/src/importers.nix

[Th3Whit3Wolf]

I tried actually, rakePkgs is easy but I could not figure out how to make the localPackages overlay fit in there.

[blaggacao]

I'd model it after

digga/src/importers.nix

Lines 120 to 127 in fb3497d

	importOverlays = dir:
	{
	# Meant to output a module that sets the overlays option
	# overlays order matters. mkAfter ensures those in-house
	# overlays are loaded later (after external ones), so the latter
	# can be modified via internal overlays
	overlays = lib.mkAfter (builtins.attrValues (flattenTree (rakeLeaves dir)));
	};

returning a module that can be importsed:

{
  overlays = {};
}

[blaggacao]

I think you can use rakeLeaves as is. the only (orthogonal) addition could be to skip on _ prefixes.

[blaggacao]

In theory, you could also reuse flattenTree and derive the appropriate package name from that string, and finally callPackage on its values.

[blaggacao]

now, I got a clearer picture what this is all about 😄

@danielphan2003 Can vs-ext generalize well enough over <package set> so that we could actually inaugurate a pkgsSet lib? Or merge anything +- viable back into digga?

[danielphan2003]

Yes, it can. Head over to vs-ext for usage, assuming this PR is merged.

[blaggacao]

I kind of see a divnix/vim-ext repo not too far on the horizon. @Th3Whit3Wolf what do you think?

It would make more sense to be able to iterate independently of the devos release cycle. 🚀

Relatively soon (v1.0) devos will become very boring and slow motion.

[Th3Whit3Wolf]

I kind of see a divnix/vim-ext repo not too far on the horizon. @Th3Whit3Wolf what do you think?

We could do that, would it be better to have a repo for each <package set> or one repo that has all of them?

[blaggacao]

would it be better to have a repo for each  or one repo that has all of them?

That depends probably on two things:

How much design overlap there is and how much does it hurt if release cycles have to be coupled.

Do you have any idea about what we could expect?

[Th3Whit3Wolf]

I think multiples repos are best for now.

I think there could also be a future where one larger repo or maybe even digga creates the overlay itself as to not clutter the users' flake inputs.

[blaggacao]

https://github.com/divnix/vim-ext the project is yours 🚀

[Th3Whit3Wolf]

@blaggacao sorry it's taken me so long to respond, I've been quite busy lately.

I've been looking at vs-ext and I'm not sure it's best to make vim-ext right now. Unless I'm mistaken the goal of vs-ext updates is to take use bud to take a vscode extension unique id, query openvsx, and then update that extensions metadata.

This could be done for vim plugins however it would be a lot more work. Vim plugins aren't collectively stored in one location. You can find them on github,gitlab,(any code hosting service), vimawesome, and nvim-awesome So there would be need to create a method to query each one. If vim-ext is being described as

A kick ass library to dominate your Vim Extensions (with DevOS)

I would expect it to build vim plugins that have build requirements (could be node, rust, clojure, or really anything).

[Th3Whit3Wolf]

Follow up question, how much reliant is devos on nvfetcher?

I've started playing around with something similliar to nvfetcher with a few key differences

1. you don't have to declare fetch (just src)
2. My intentions are to generate overlays, with metadata, using platform apis (github, gitlab, openvsx, vscode).

[Pacman99]

Follow up question, how much reliant is devos on nvfetcher?

We made a general decision to use nvfetcher over flakes for updating srcs because it was a cleaner solution that was built for the use case. But if another solution is better, I'm sure minds can be changed :). We would have to run through a similar discussion/debate again.

[blaggacao]

@Th3Whit3Wolf this time sorry for me chiming in late for the same reason as yours.

Well, one day, if subflakes get really advanced, we might even want to switch something flake compatible, since there is a benefit in creating a SBOM easily from the flake.lock.

Ideally flakes would gain the capability to make use of a library (such as nvchecker that already has wide platform support and an upstream community momentum).

vim-ext can be also really small. The point in having it in a repo also is a bit not having it in devos or digga (for now). The reason is that all those editors might a) either have their own dynamic and benefit from a separate release schedule for the users (nix flake lock --update-input vim-ext). Another reason is that beyond some initial preferences of Tim, we would probably try to avoid being opinionated with anything that doesn't directly relate to flakes/nix. So by making it a separate repo, we essentially avoid "pissing of" those who don't use X or Y. A third reason is maintenance and support. If the project grows, then it will be helpful to forward vim-ext issues to the repo in order to create "tighter" feedback cycles between authors and users and convert users more easily into co-authors / co-maintainers once they find a tiny and easy to understand codebase. vs-ext / vim-ext can well be the 'entry drug' heheheh. 😄

[Th3Whit3Wolf]

@blaggacao I have been selected for a program that will likely eat up all of my free time for the next 7 months. I do not how soon I will be able to commit to vim-ext.

[blaggacao]

@Th3Whit3Wolf Thanks for letting us know. I think you already did a good start to kick off vim ergonomics with devos. Now the strategically most important task is maybe to broaden the user base and get morenusers of various ecosystems onvoarx.