Skip to content

Always expect a Libtask trace to be present#2757

Merged
penelopeysm merged 2 commits into
mainfrom
py/always-expect-trace
Jan 26, 2026
Merged

Always expect a Libtask trace to be present#2757
penelopeysm merged 2 commits into
mainfrom
py/always-expect-trace

Conversation

@penelopeysm

Copy link
Copy Markdown
Member

Closes #2754: with this patch, I get the following results

julia> while true
           score = AdvancedPS.advance!(trace)
           score === nothing && break
           acc_val = trace.model.f.varinfo.accs[:LogLikelihood].logp

           print("score: $(round(score, digits=3))")
           println("\taccumulator: $(round(acc_val, digits=3))")
       end
score: -3.418	accumulator: 0.0
score: -3.417	accumulator: -3.418
score: -3.417	accumulator: -6.835
score: -3.417	accumulator: -10.252
score: -3.418	accumulator: -13.669
score: -3.417	accumulator: -17.087
score: -3.419	accumulator: -20.505
score: -3.418	accumulator: -23.923
score: -3.417	accumulator: -27.341
score: -3.417	accumulator: -30.758

@github-actions

Copy link
Copy Markdown
Contributor

Turing.jl documentation for PR #2757 is available at:
https://TuringLang.github.io/Turing.jl/previews/PR2757/

Comment thread src/mcmc/particle_mcmc.jl
Comment on lines -426 to -431
arg_vi_id = objectid(vi)
vi = get_trace_local_varinfo_maybe(vi)
using_local_vi = objectid(vi) == arg_vi_id

trng = get_trace_local_rng_maybe(ctx.rng)
resample = get_trace_local_resampled_maybe(true)

@penelopeysm penelopeysm Jan 25, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some explanation is necessary, mostly for the unknown future person (which might well be me) who comes across this, and wonders why I got rid of these checks.

Generally, these functions used to attempt to check whether the function was being executed from inside a Libtask taped task. If not, then we attempted to use a "normal" execution path, i.e., using the VarInfo that is passed to tilde_assume!!, etc. -- but if we were, then we had to use the VarInfo that was stored as a 'taped global' in Libtask.

The problem with this is that the check objectid(vi) == arg_vi_id used to sometimes return true, even if there was a trace present, i.e., even if get_trace_local_varinfo_maybe returned the VarInfo stored in the taped globals. I actually completely don't know why. The fact that this was true meant that several lines below this, we never set the VarInfo back in the task's taped globals, which was responsible for the behaviour in #2754.

Why can we remove these checks...? Well, there is no way to actually use ParticleMCMCContext without going through sample(model, ::Union{SMC,PG}, ...). That function (technically, the functions that it calls) is what sets the leaf context of a model to be ParticleMCMCContext:

function TracedModel(
model::Model, varinfo::AbstractVarInfo, rng::Random.AbstractRNG, resample::Bool
)
model = DynamicPPL.setleafcontext(model, ParticleMCMCContext(rng))
args, kwargs = DynamicPPL.make_evaluate_args_and_kwargs(model, varinfo)
fargs = (model.f, args...)
return TracedModel(model, varinfo, resample, fargs, kwargs)
end

It is also what wraps the model in a Libtask trace. (That code is in AdvancedPSLibtaskExt.)

Furthermore, it is meaningless to use ParticleMCMCContext outside of a Libtask trace (it is pretty much impossible to run SMC on a Turing model without the functionality that Libtask provides -- trust me, I've tried). So there is a one-to-one correspondence: if you are using ParticleMCMCContext, you are within a Libtask trace, and vice versa.

So there's really no point in checking whether it is local or not. My current hypothesis is that the check is a vestigial bit of code from the old implementation, which used SamplingContext{<:SMC}, and in those cases maybe it was easier to accidentally run into this situation where the model was being executed with that context, but outside of a Libtask trace. Personally, I still don't know how that could ever happen; but anyway, here we are.

Technically, the code that I'm changing here was written in #2585; but the whole logic of 'checking whether we're in a trace or not' predates that PR, so goes back quite a while.

@codecov

codecov Bot commented Jan 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 70.00000% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.05%. Comparing base (751ae1d) to head (026d07c).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/mcmc/particle_mcmc.jl 70.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2757      +/-   ##
==========================================
+ Coverage   84.96%   85.05%   +0.08%     
==========================================
  Files          20       20              
  Lines        1264     1258       -6     
==========================================
- Hits         1074     1070       -4     
+ Misses        190      188       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@penelopeysm

penelopeysm commented Jan 25, 2026

Copy link
Copy Markdown
Member Author

CI failures are in the optimisation interface, which is completely unexpected, but also unrelated to this PR, and will be fixed in #2747. It appears to be random since not all runners are failing.

@penelopeysm penelopeysm merged commit c79b60d into main Jan 26, 2026
25 of 30 checks passed
@penelopeysm penelopeysm deleted the py/always-expect-trace branch January 26, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ProduceLogLikelihoodAccumulator does not properly accumulate

1 participant