Skip to content

Automatically add dependency on manifest file.#190

Closed
de-vri-es wants to merge 1 commit intodiondokter:masterfrom
de-vri-es:track-manifest-as-dependency
Closed

Automatically add dependency on manifest file.#190
de-vri-es wants to merge 1 commit intodiondokter:masterfrom
de-vri-es:track-manifest-as-dependency

Conversation

@de-vri-es
Copy link
Copy Markdown

@de-vri-es de-vri-es commented Jan 16, 2026

This PR adds a const _: &[u8] = include_bytes($manifest_path) to automatically trigger recompilation if the manifest changes. This way, no build script is needed for that anymore.

Tested with a simple example and cargo expand, showing this line ending up at the top of the the expanded source:

const _: &[u8] = b"/// This is the testing device\ndevice Device {\n    byte-order LE\n    register-address-type u8\n\n    register Foo {\n        address 0\n\n        fields size-bits=24 {\n            (uint)value @23:0\n        }\n    }\n}\n";

The dependency tracking is only added when the input is a manifest and not when using the CLI.

Comment thread compiler/src/lib.rs Outdated
Comment on lines +32 to +37
if let Some(path) = file_path.to_str() {
output +=
&quote::quote! { const _: &[u8] = ::core::include_bytes!(#path); }.to_string();
} else {
diagnostics.add_warning("Manifest path contains invalid Unicode, automatic dependency tracking will not work. Consider renaming the manifest file.");
}
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that I don't think you can actually provide a path with invalid Unicode, as the path has to be passed as a macro argument from a Rust source file.

So it might be fine to just unwrap() and remove the warning.

@diondokter
Copy link
Copy Markdown
Owner

Hmmm, you've expressed this desire before: #146

I'll consider it again...

@de-vri-es
Copy link
Copy Markdown
Author

If you want the adding of the dependency tracking can be moved from the compiler to the macro. Only downside is that then the macro crate should also depend on quote, but I don't think that's a big problem.

And then the compiler lib doesn't have to know anything about dependency tracking.

@de-vri-es de-vri-es force-pushed the track-manifest-as-dependency branch from 5052a0c to d377b37 Compare January 16, 2026 18:09
@de-vri-es
Copy link
Copy Markdown
Author

If you want the adding of the dependency tracking can be moved from the compiler to the macro. Only downside is that then the macro crate should also depend on quote, but I don't think that's a big problem.

And then the compiler lib doesn't have to know anything about dependency tracking.

Moved it to the macro crate. Much smaller diff in total and arguably a cleaner place to implement this.

But now the macro crate depends on quote too. I think that's fine though, since it's in the dep tree either way.

@diondokter
Copy link
Copy Markdown
Owner

Ok, I've decided. I don't want this. For no other reason really than 'it feels wrong to me'. Sorry :)

@diondokter diondokter closed this Feb 22, 2026
@de-vri-es
Copy link
Copy Markdown
Author

Eh? This is the standard way that proc-macros do this kind of thing. It's much more fool-proof than a build script, and it saves compiling the build script too. I'm struggling to see a downside 😮

@diondokter
Copy link
Copy Markdown
Owner

The largest driver written with device-driver I know about is in a private repo and clocks in at 354 KB. That's large, but not as large as imaginable. For example, the STM32H753 SVD file is 3.84 MB.

The thing that feels bad about it is that we'd be asking the compiler to include a string of potentially multiple megabytes only to then ignore it.

That driver of 354 KB I mentioned already starts lagging Rust Analyzer and I don't want to add to that.
I do agree that having to add the build.rs is annoying. But you build it once and that's it.

There is a real solution: rust-lang/rust#99515
But it's nightly and it needs further design.

Also, I use macro expansion a lot to view the source of a driver and I don't want to see the original source code there again. That's not great debugging.

So, it is just how it is until Rust provides a better way.

(I am thinking about adding an additional way to build the drivers. Namely something you can add as a build-dependency instead of a macro)

@de-vri-es
Copy link
Copy Markdown
Author

The largest driver written with device-driver I know about is in a private repo and clocks in at 354 KB. That's large, but not as large as imaginable. For example, the STM32H753 SVD file is 3.84 MB.

The thing that feels bad about it is that we'd be asking the compiler to include a string of potentially multiple megabytes only to then ignore it.

Well, it's up to you of course, but I don't think include_bytes!() will lag rust analyzer. To test I just made a project including 4 files of 10 MiB, each being written with random data every 0.5 seconds, and I noticed no lag from rust-analyzer at all. I think proc-macro execution is what creates lag, not an include_bytes!() in the macro output.

And the input file will just have been read by the proc-macro, so on the bright side it's probably in the OS disk cache already ^_^

Also, I use macro expansion a lot to view the source of a driver and I don't want to see the original source code there again. That's not great debugging.

I do understand the pain of debugging proc macros. Even cargo expand is annoying, because it also completely expands the macro output, even things like write!(), resulting in code that doesn't even compile.

But since this project already has the ddc tool, isn't that a much nicer alternative than running cargo expand? It won't expand macros in the compiler output, giving you the actual macro output rather than the fully expanded output.

rust-lang/rust#99515 doesn't seem likely to produce anything usable any time soon, as it doesn't seem the discussion around the public API is progressing much. In the mean time there's a pretty workable solution, I think.

But again, up to you ^_^

@de-vri-es
Copy link
Copy Markdown
Author

Oh, and if you let rust-analyze inline the macro for you, you won't see the file contents. You will see the include_bytes!() statement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants