Skip to content

Linux and mac compatibility#1029

Open
aimir wants to merge 40 commits intoknurling-rs:mainfrom
gen-bc:linux-and-mac-compatibility-branched
Open

Linux and mac compatibility#1029
aimir wants to merge 40 commits intoknurling-rs:mainfrom
gen-bc:linux-and-mac-compatibility-branched

Conversation

@aimir
Copy link
Copy Markdown
Contributor

@aimir aimir commented Jan 21, 2026

Currently without correct locations in symbol tables on mac, but everything else should work as expected, with no impact on embedded systems

aimir and others added 29 commits October 20, 2025 17:23
…one is not enough to make sure symbols are actually exported on mac
…either avoid it by prefixing with or strip this prefix explicitly
… on linux and mac, to offset addresses accordingly, since the linker script won't work on them (there's probably a more normal way to do this only once)
…s-calculations

Return to no-std, use linker symbols to calculate the base address
On macOS, each defmt log statement created a unique section in the .defmt
segment. With 200+ logs, this exceeded Mach-O's 255 section limit.

Fix: Group logs by severity level instead of unique hash per log.
fix(macos): avoid 255 section-per-segment limit
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Linux and mac compatibility fixes, part 2
This reverts commit ba1f9ae.
@jonathanpallant
Copy link
Copy Markdown
Contributor

Nice - I'm excited to look through this in detail soon.

@mhatzl mhatzl self-requested a review January 21, 2026 15:46
@aimir aimir force-pushed the linux-and-mac-compatibility-branched branch from 84a3cea to 6b12bee Compare January 21, 2026 16:20
Comment thread macros/src/construct.rs

if for_macos {
sub_section = format!(",{:x}", hash(&sub_section));
// Use a single section per severity level instead of unique section per log.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this has an effect on the linker's ability to garbage collect defmt strings in functions that are not called?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it do much harm if it didn't?
There are some very flash-constrained devices running Linux, but I'm not aware of tiny embedded Mac systems and non-mac targets should be unaffected.

Comment thread defmt/src/export/mod.rs
@jonathanpallant
Copy link
Copy Markdown
Contributor

I had a trial run with this, but defmt-print cannot open Mach-O files, so this isn't usable on macOS.

How have you been testing it?

@tomasbarton-stemcell
Copy link
Copy Markdown

tomasbarton-stemcell commented Feb 17, 2026

I tested this on Linux and it works well 🥳

Although I still needed to have a custom linker file that collects all interned strings into a single section:

Error: defmt version found, but no `.defmt` section - check your linker configuration

Ideally the elf parser could deal with it without the linker file, because the symbols are in the elf file when I print them out.

Alternatively modify the provided defmt.x so that it does INSERT AFTER ... (somehow, maybe behind a feature flag?) otherwise it cannot be used as by default it replaces the default linker file instead of extending it.

@jonathanpallant
Copy link
Copy Markdown
Contributor

I think the best solution is to only put items into the .defmt section when running bare-metal (when cfg(target_os = "none")). Otherwise let linker just put them wherever. And then fix the decoder to be more flexible about where it looks for these symbols.

@aimir
Copy link
Copy Markdown
Contributor Author

aimir commented Feb 26, 2026

I had a trial run with this, but defmt-print cannot open Mach-O files, so this isn't usable on macOS.

How have you been testing it?

I just used the decoder directly, but let me go ahead and change this - added a commit to support Mach-O files in defmt-print 😁

@jonathanpallant
Copy link
Copy Markdown
Contributor

@knoellle is going to take a look at this one, right after his Robocup competition.

@knoellle
Copy link
Copy Markdown
Member

Thank you for your work so far.
I was able to craft a native Linux program and a linker script that worked but don't have a Mac to test with.

What is your plan with the linker scripts? As @tomasbarton-stemcell mentioned already, I think we should either include linker scripts for Linux/Mac or adjust the elf parser to work without linker scripts.
The defmt/build.rs already generates/modifies linker scripts depending enabled features. Using a different linker script as a base depending on the platform seems reasonable to me, though not needing one would be preferable I think.

@mhatzl
Copy link
Copy Markdown
Contributor

mhatzl commented Mar 24, 2026

Thanks for the PR!
Tested it on macOS and it worked without the need for a linker script.
The encoded defmt logs are output on stdout and piping it to defmt-print given the binary correctly decoded and displayed the logs.

As mentioned in the PR description, the location information currently does not work for macOS.
@aimir do you have an idea why or maybe even time to fix that?

@aimir
Copy link
Copy Markdown
Contributor Author

aimir commented Apr 5, 2026

Thank you for your work so far. I was able to craft a native Linux program and a linker script that worked but don't have a Mac to test with.

What is your plan with the linker scripts? As @tomasbarton-stemcell mentioned already, I think we should either include linker scripts for Linux/Mac or adjust the elf parser to work without linker scripts. The defmt/build.rs already generates/modifies linker scripts depending enabled features. Using a different linker script as a base depending on the platform seems reasonable to me, though not needing one would be preferable I think.

Mac doesn't have linker scripts, and like @mhatzl said it should work without the need for one.

It does mean that things like the default panic handler that's auto-configured by defmt/build.rs. I didn't find an elegant workaround yet, and I really want to merge this - is this a must for you?

@aimir
Copy link
Copy Markdown
Contributor Author

aimir commented Apr 5, 2026

Thanks for the PR! Tested it on macOS and it worked without the need for a linker script. The encoded defmt logs are output on stdout and piping it to defmt-print given the binary correctly decoded and displayed the logs.

As mentioned in the PR description, the location information currently does not work for macOS. @aimir do you have an idea why or maybe even time to fix that?

The issue is that by default mac location information is scattered across files, and the table parsing doesn't handle this well.

Running without split debug info (RUSTFLAGS="-C split-debuginfo=off) seems to resolve the issue - @mhatzl can you check if that works for you?

@mhatzl
Copy link
Copy Markdown
Contributor

mhatzl commented Apr 15, 2026

Hi @aimir,
thanks for your help!
Unfortunately I am not able to get correct location infos on macOS with split-debuginfo=off.

looking at the docs for split-debuginfo and the cargo profile docs, the default for macOS is packed except for Cargo profiles that have debug infos enabled it is unpacked.

Since the dev profile has debug infos enabled, split-debuginfo is set to unpacked by default when run via Cargo. Setting it to packed creates this dSym folder containing DWARF data, but the location info is still missing if this DWARF file is passed to defmt-print.

@knoellle
Copy link
Copy Markdown
Member

It does mean that things like the default panic handler that's auto-configured by defmt/build.rs. I didn't find an elegant workaround yet, and I really want to merge this - is this a must for you?

I spent some time attempted making the default panic stuff work without linker scripts but didn't find a way without breaking user code. See #1050.

Again, I don't have a mac to test with, but if I understand correctly then attempting to use defmt::panic on macOS shouldn't successfully link or run (still not entirely sure how that works over there) unless the user defines a custom defmt panic handler.
defmt::panic calls _defmt_panic, which is weakly linked to __defmt_default_panic but can be overridden using the defmt::panic_handler macro.
Without the linker script and no custom implementation, _defmt_panic would be an undefined symbol.

What are your thoughts on requiring a custom panic handler for building for macOS specifically?

@knoellle
Copy link
Copy Markdown
Member

knoellle commented Apr 20, 2026

I'm curious how you tested the linux compatibility.
I have been able to get it to work, but I haven't found a good way of passing the linker script to the linker. My workaround so far was setting RUSTFLAGS="-C link-arg=-T<path to my repo>/.cargo/defmt.x"

For embedded platforms, we use flip-link which does a bunch of magic to find linker scripts inside library search paths which other linkers don't do.
So far I have always had to pass an absolute path to my linker script to compile my linux binaries.

  • defmt build.rs generates the default linker script but doesn't know if a linker script is going to be used at all.
  • end user .cargo/config.toml shouldn't contain an absolute path to a file in the repo or worse to the one generated by defmt build.rs
  • Requiring a build.rs in the end user project to find either the defmt-generated linker script (somewhere in target/, maybe requires the defmt build.rs to somehow pass it's own OUT_DIR to the end user build.rs) or one provided by the user project and adding the rustflag that way seems very heavy handed.

How do you build your linux binaries?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants