UPSTREAM PR #18526: model: fix SWA per-layer freq_base

[loci-dev]

Mirrored from ggml-org/llama.cpp#18526

I think the if constexpr (iswa) can be wrong here, so pushing this PR for discussion.

At least for gemma3, SWA layers are expected to use different freq_base than full attention layer. Using if constexpr (iswa) will make it a fixed value cross all layers.

Here is the original code of gemma3 (PR ggml-org/llama.cpp#12343) for reference:

        for (int il = 0; il < n_layer; ++il) {
            const bool is_sliding          = (il + 1) % sliding_window_pattern;
            const float freq_base_l        = is_sliding ? 10000.0f    : freq_base;
            const float freq_scale_l       = is_sliding ? 1.0f        : freq_scale;
            struct ggml_tensor * KQ_mask_l = is_sliding ? KQ_mask_swa : KQ_mask;

[loci-review]

Explore the complete analysis inside the Version Insights

Perfect! I've retrieved the summary report for your project. Here are the key findings:

Summary Report for llama.cpp PR #776

Project Information:

• Repository: llama.cpp (owner: auroralabs-loci)
• Pull Request: UPSTREAM PR #18526: model: fix SWA per-layer freq_base #776
• Comparing versions to identify performance changes

Key Results:
✅ No Significant Performance Regressions Detected

The analysis compared the base version against the target version and found:

• No modified functions showed performance changes greater than 2%
• Both Response Time and Throughput Time metrics remained stable
• All performance variations are within the acceptable ±2% threshold

Conclusion:
This pull request maintains stable performance characteristics and does not introduce any concerning performance degradations. The changes are safe to proceed from a performance perspective.