Skip to content

Implement the Stop-Gradient DPF#202

Closed
DanWaxman wants to merge 0 commit intostate-space-models:mainfrom
DanWaxman:dw-make-pf-diff
Closed

Implement the Stop-Gradient DPF#202
DanWaxman wants to merge 0 commit intostate-space-models:mainfrom
DanWaxman:dw-make-pf-diff

Conversation

@DanWaxman
Copy link
Contributor

Currently, the PF defined in the SMC classes is not differentiable, in the sense that estimators of the score (i.e., grad(marginal log likelihood)) will be biased. In fact, automatic differentiation at all is broken now on CPU, but bias persists even after fixing that. This is unfortunate, because it limits what one can do for system identification -- for example, SGMCMC -- which rely on these score estimates being unbiased.

There are several ways to ameliorate this (c.f. the PyDPF paper), but one of the simplest is the stop-gradient trick, which recovers some classical gradient estimators by a simple + term - term trick.

This PR implements the stop-gradient version of the DPF, and compared the resulting MLL and score estimates of a bootstrap PF to the analytically-known ones in a linear-Gaussian SSM. These show the expected results, i.e., that both the PF as-is and DPF have good MLL estimates, but the PF is highly biased: image

This bias persists under many Monte Carlo draws, even at the true parameters: image

The implementation adds minimal overhead (which is also in line with the paper):

PF (standard): 15.80 ± 1.80 ms (median ± std over 1000 evals)
PF (diff-resampling): 15.97 ± 1.14 ms (median ± std over 1000 evals)

See the discussion in #201 for design discussion.

n.b., currently there is a notebook illustrating the results in docs/examples. This should probably move or otherwise be deleted, but seemed useful for review/discussion purposes.

@DanWaxman
Copy link
Contributor Author

This will currently fail the noop tests in tests/cuthbert/smc/test_particle_filters.py, because the DPF algorithm will normalize the weights. This seems okay to me? I.e., maybe the noop test should be checking up to a constant factor in the weights or something.

@AdrienCorenflos
Copy link
Contributor

#207 is merged @DanWaxman so you can proceed with implementing this as a decorator I guess!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants