Segmentation fault when testing composite mean functions (ProdMean, SumMean) with Julia 1.12.1

[orebas]

Commentary from human author:
When testing in 1.12.1, there was a consistent segfault in some of the tests. It is moderately easy to workaround, but Claude suspects it is a julia compiler bug, and at that point I am out of my depth. I attach Claude's comprehensive report below. It was not able to create a package-independent replication to pass to the Julia compiler devs, and I think that would be a good thing to have.

To be clear, I have not submitted workarounds (that are described below as "in-place". Let me know if that would be helpful.

@@@@@@@@ Claude's report @@@@@@@@@

Segfault Investigation Report for GaussianProcesses.jl

Executive Summary

A segmentation fault occurs in GaussianProcesses.jl when testing composite mean functions (ProdMean, SumMean) with Julia 1.12.1. The crash is caused by using view() in recursive set_params! calls and has the characteristics of a Julia compiler optimization bug.

The Bug

Location

• File: src/common.jl
• Function: set_params!(obj::CompositeMeanOrKernel, hyp::AbstractVector)
• Line: 115 (in the recursive call with view())

Symptoms

1. ✅ Segfault with default optimizations (julia test.jl)
2. ✅ Works with --optimize=0 (julia --optimize=0 test.jl)
3. ✅ Works with println debug statements (classic Heisenbug)
4. ✅ Still crashes with --check-bounds=yes (not an OOB issue)
5. ✅ Fixed by materializing slice (hyp[i:(j-1)] instead of view(hyp, i:(j-1)))

Crash Pattern

[463655] signal 11 (128): Segmentation fault
in expression starting at test/means.jl:7
set_params! at src/common.jl:113

Occurs during:

• Test file: test/means.jl
• When: Testing ProdMean and SumMean with Calculus.gradient
• Why: Repeated calls to set_params! with slightly perturbed parameters

Evidence This is a Julia Compiler Bug

1. Optimization-Dependent Behavior

# CRASHES
$ julia test_segfault.jl
[signal 11 (128): Segmentation fault]

# WORKS
$ julia --optimize=0 test_segfault.jl
✓ ProdMean works

2. Debug Statement Side Effects

Adding println() calls inside the loop prevents the crash - classic compiler fence behavior.

3. View vs. Slice Behavior

# CRASHES
set_params!(c, view(hyp, i:(j - 1)))

# WORKS
params_slice = hyp[i:(j - 1)]
set_params!(c, params_slice)

4. No Unsafe Code

• All leaf set_params! methods use safe operations (copyto!, scalar assignment)
• No @inbounds in the crash path
• No ccall, pointer(), or other unsafe operations
• No threading involved

Expert Consultation

Consulted GPT-5 and Gemini-2.5-Pro. Both agreed:

GPT-5: "The pattern is consistent with either an OOB write under @inbounds or a compiler/GC-liveness bug with SubArray under recursion"

Gemini-2.5-Pro: "Overwhelmingly likely that this is a bug in the Julia compiler or garbage collector (GC)... The compiler might be making an incorrect assumption about the lifetime or aliasing of the view object"

Related Julia Issues

Web search found historical issues with similar patterns:

• #19792: LLVM miss-compiles inlined broadcast for Array & SubArray
• #46986: Segmentation fault in v1.8.2 with SubArray
• #29224: SubArray × Matrix with recursive concerns

None match exactly, suggesting this may be a new manifestation in Julia 1.12.1.

The Fix

Current Workaround (in place)

function set_params!(obj::CompositeMeanOrKernel, hyp::AbstractVector)
    length(hyp) == num_params(obj) ||
        throw(ArgumentError("$(typeof(obj)) object requires $(num_params(obj)) hyperparameters"))
    i = 1
    for c in components(obj)
        np = num_params(c)
        j = i + np
        # Materialize the view to avoid potential compiler bug
        params_slice = hyp[i:(j - 1)]
        set_params!(c, params_slice)
        i = j
    end
end

Recommended Long-Term Fix

Pass indices instead of views (avoids both the bug AND allocations):

function set_params!(obj::CompositeMeanOrKernel, hyp::AbstractVector, offset::Int=1)
    for c in components(obj)
        offset = set_params!(c, hyp, offset)
    end
    return offset
end

function set_params!(obj::LeafType, hyp::AbstractVector, offset::Int)
    np = num_params(obj)
    # Use indices directly
    for k in 1:np
        internal_params[k] = hyp[offset + k - 1]
    end
    return offset + np
end

Test Results

Before Fix

• Segfault in test/means.jl during ProdMean and SumMean tests
• Crash point: Calculus.gradient calling set_params! repeatedly

After Fix (Materialized Slices)

• ✅ All tests run to completion
• ✅ 40 tests pass
• ⚠️ 2 tests have minor numerical precision differences (MeanPeriodic, likely unrelated)

Recommendations for Maintainers

1. Keep the current fix (materializing slices) - it's safe and works
2. Consider the index-passing approach for better performance
3. Monitor Julia issues for similar reports and potential upstream fixes
4. Consider filing a Julia issue if you can create a minimal reproducer

System Information

• Julia Version: 1.12.1
• Platform: Linux 6.6.87.2-microsoft-standard-WSL2
• GaussianProcesses.jl: v0.12.6 (with compat updates)

Files Modified

• src/common.jl lines 108-120 (set_params!)
• src/common.jl lines 143-155 (set_priors!, same pattern)

Questions?

Contact information or create an issue at the GaussianProcesses.jl repository.

---

Investigation conducted with assistance from Claude (Anthropic) and expert AI models (GPT-5, Gemini-2.5-Pro)