support package dependencies via paths relative to the build root

[TUSF]

zig build should be able to fetch from a relative path such as:

    .url = "../myproject/",

or a file URI like:

    .url = "file:///C:/Code/myproject/"

I think this would mostly be useful for when operating on local forks of code that aren't published anywhere else, without needing to explicitly vendor those dependencies into the same folder. (Although perhaps this might make vendored dependencies also easier to work with using the build system?)

[nektro]

this is a rabbit hole of un-reproducibile builds and an anti-pattern

I think this would mostly be useful for when operating on local forks of code that aren't published anywhere else

this should be possible once the git, hg, svn, etc backends are supported

[MoAlyousef]

Could you elaborate on how this is an antipattern?

This is supported by CMake (FetchContent and ExternalProject_Add) and Cargo (path) for example.

[andrewrk]

There is no need to debate whether the file:/// scheme will work; it is planned. The discussion is about how it should work. Some potential topics for discussion:

• How to handle relative paths (relative to build.zig probably)
• Should environment variables be expanded? If so, what syntax for this?
• Potential footguns, and ideas to guide the user away from triggering them.

As for implementation, here is where it parses the URI and then is hard-coded to use the http client:

zig/src/Package.zig

Lines 334 to 355 in 37424fd

	const uri = try std.Uri.parse(url);
	const rand_int = std.crypto.random.int(u64);
	const tmp_dir_sub_path = "tmp" ++ s ++ hex64(rand_int);
	const actual_hash = a: {
	var tmp_directory: Compilation.Directory = d: {
	const path = try global_cache_directory.join(gpa, &.{tmp_dir_sub_path});
	errdefer gpa.free(path);
	const iterable_dir = try global_cache_directory.handle.makeOpenPathIterable(tmp_dir_sub_path, .{});
	errdefer iterable_dir.close();
	break :d .{
	.path = path,
	.handle = iterable_dir.dir,
	};
	};
	defer tmp_directory.closeAndFree(gpa);
	var req = try http_client.request(uri, .{}, .{});
	defer req.deinit();

Instead, it should check the scheme of the URI after parsing it and decide what to do.

[AdamGoertz]

If no one else is working on this, I'd like to take a stab at it. I think there's plenty of room to get started on this without having to answer all questions about relative paths and environment variables.

The main trade-off I see with the questions that Andrew has raised is how much we prioritize making it easy to use file:/// URIs versus making it portable and easy to depend on. A user who is writing their own build.zig.zon can easily choose the a URI that works for their system, but dependencies buried several layers deep in the dependency tree are the main concern.

My take on the questions Andrew raised:

• Relative paths are relative to build.zig
• I would say 'no' to expanding environment variables, simply because they aren't portable across operating systems. If we did choose to support this, we'd need a syntax for expanding them that can't appear in a valid URI. Probably something like {VAR}, since afaik {} can't appear in a URI and it's a pretty common choice of syntax.

[mitchellh]

I'm just sharing my use case here: monorepos. I sometimes will use a monorepo for my projects, particularly when I'm still "baking" a package I don't feel is ready to fully extract. I cleanly separate my packages out into a separate dir like pkg with its own dedicated build.zig so that its ready to be extracted one day if I want to.

It'd be nice to represent these in the build.zig.zon file somehow. Right now I just import their build.zig directly.

Here is the current folder layout of a project I'm working on. The folders in pkg/ are extractable standalone Zig packages but I just haven't extracted them yet. They may depend on each other (adjacent folders).

.
├── pkg
│   ├── fontconfig
│   ├── freetype
│   ├── harfbuzz
│   ├── <etc...>
├── src
│   ├── <main app zig files>
└── vendor
    ├── <git submodules>

[praschke]

absolute paths do not make sense across machines, let alone across operating systems. additionally, file:// only supports absolute paths; relative paths are not URIs.

1. why is this being implemented as .url = "../myproject/", and not .path = "../myproject/",? at least for relative paths?

2. what good reason is there for absolute paths to be supported?

3. .url = "file:///etc/shadow",

this can easily become a supply chain attack, targeted or shotgun, and you cannot make a list of every user's secret files. please do not tell me that packages will have to have zig in them, or the package manager won't be a replacement for git submodules. #14488

[tauoverpi]

It would be best if paths were all relative to build.zig used in the zig build invocation without the ability to reference directories above it. This can later be enforced for build.zig aswell with #14286

[jiacai2050]

It would be best if paths were all relative to build.zig used in the zig build invocation without the ability to reference directories above it.

This sounds counter-intuitive, the path should be relative to its package root, not where zig build is invoked.

This is how go mod's replace works

• https://go.dev/doc/modules/gomod-ref#replace

[AdamGoertz]

It would be best if paths were all relative to build.zig used in the zig build invocation without the ability to reference directories above it.

This sounds counter-intuitive, the path should be relative to its package root, not where zig build is invoked.

This is how go mod's replace works

• https://go.dev/doc/modules/gomod-ref#replace

I agree with jiacai2050 that relative paths should be relative to the package root, which is currently how it is implemented in my open PR.

As for whether or not absolute paths should be allowed, I’m torn. I agree with the sentiment that in most cases it’s a bad practice to use absolute paths, but I’m not convinced that’s sufficient justification to disallow them.

I’m curious if anyone has a scenario in mind where absolute paths would be especially useful. I haven’t yet come up with a compelling use-case myself (just vague notions), but that doesn’t mean no such use-case exists.