fix(fused-ffi): complete ABI v9 context_size + bump stale sizeof assert (80→88)#30
Open
lalalune wants to merge 1 commit into
Open
fix(fused-ffi): complete ABI v9 context_size + bump stale sizeof assert (80→88)#30lalalune wants to merge 1 commit into
lalalune wants to merge 1 commit into
Conversation
…rt (80→88) The streaming-LLM config struct gained `context_size` (int32 @ offset 80) for ABI v9 — `_llm_stream_open` now honors `cfg->context_size` (>0) instead of only the ELIZA_LLM_N_CTX env default. The TS marshaller (ffi-bindings.ts) was already emitting 88 bytes with context_size at offset 80, but the C-side ABI-guard `static_assert(sizeof(eliza_llm_stream_config_t) == 80)` was never bumped, so the fused `elizainference` target failed to compile. Bump it to 88 to match the real layout (8×int32 + 5×ptr, pointer-aligned). Validated: a host Metal build of `libelizainference` (ABI v12) loads + generates on the real google/gemma-4-E2B with context_size=4096 honored (83 tok/s, M4 Max). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The streaming-LLM config struct
eliza_llm_stream_config_tgainedcontext_size(int32 @ offset 80) for ABI v9 —eliza_inference_llm_stream_opennow honorscfg->context_size(>0) instead of only theELIZA_LLM_N_CTXenv default. The TS marshaller (ffi-bindings.ts) was already emitting 88 bytes withcontext_sizeat offset 80, but the C-side ABI-guardstatic_assert(sizeof(...) == 80)was never bumped — so the fusedelizainferencetarget fails to compile. This bumps it to 88 to match the real layout.Validated: a host Metal build of
libelizainference(ABI v12) loads + generates on the realgoogle/gemma-4-E2Bwithcontext_size=4096honored — "The capital of France is Paris.", 83 tok/s on an Apple M4 Max.🤖 Generated with Claude Code