Implement the Stop-Gradient DPF#202
Closed
DanWaxman wants to merge 0 commit intostate-space-models:mainfrom
Closed
Implement the Stop-Gradient DPF#202DanWaxman wants to merge 0 commit intostate-space-models:mainfrom
DanWaxman wants to merge 0 commit intostate-space-models:mainfrom
Conversation
Contributor
Author
|
This will currently fail the noop tests in |
Contributor
|
#207 is merged @DanWaxman so you can proceed with implementing this as a decorator I guess! |
2bbe624 to
7b1a120
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently, the PF defined in the SMC classes is not differentiable, in the sense that estimators of the score (i.e., grad(marginal log likelihood)) will be biased. In fact, automatic differentiation at all is broken now on CPU, but bias persists even after fixing that. This is unfortunate, because it limits what one can do for system identification -- for example, SGMCMC -- which rely on these score estimates being unbiased.
There are several ways to ameliorate this (c.f. the PyDPF paper), but one of the simplest is the stop-gradient trick, which recovers some classical gradient estimators by a simple + term - term trick.
This PR implements the stop-gradient version of the DPF, and compared the resulting MLL and score estimates of a bootstrap PF to the analytically-known ones in a linear-Gaussian SSM. These show the expected results, i.e., that both the PF as-is and DPF have good MLL estimates, but the PF is highly biased:
This bias persists under many Monte Carlo draws, even at the true parameters:
The implementation adds minimal overhead (which is also in line with the paper):
See the discussion in #201 for design discussion.
n.b., currently there is a notebook illustrating the results in
docs/examples. This should probably move or otherwise be deleted, but seemed useful for review/discussion purposes.