Skip to content

Switch to a more sensible normalization strategy for loss metrics#70

Open
allevitan wants to merge 9 commits intomasterfrom
newpoisson
Open

Switch to a more sensible normalization strategy for loss metrics#70
allevitan wants to merge 9 commits intomasterfrom
newpoisson

Conversation

@allevitan
Copy link
Collaborator

This makes loss metrics a bit more interpretable. I may put some more work into writing better normalizations if I have a similar internet-free train ride in the future, where I can do some cdtools work but don't have a stable connection to any GPUS...

Key notes:

  • This affects all reported loss metrics, so after merging this PR, expect the loss metrics for typical reconstructions to change.

Changes:

  • Adds a more flexible "normalizer" class to enable different kinds of normalization for the calculated loss metrics when used for reconstructions.
  • Removes the intensity-based normalization for the amplitude MSE and switches to reporting a "straight up" amplitude MSE. I have come to believe this is more interpretable in most cases.
  • Switches to use a more reasonable normalization and offset correction for the Poisson NLL results, so that they stop being insane values and never go negative anymore. This will hopefully help encourage adoption of the Poisson NLL loss metric.
  • Added initializations for all three loss metrics to all reconstruction models.
  • Adjusted the tests to account for the new loss values, and ensured that the different losses all see test coverage
  • Update the docs to match, including the tutorial section.
  • Update the appropriate example script (near_field_ptycho.py, data taken with a photon counting detector) to use the Poisson NLL error metric.

allevitan and others added 9 commits March 19, 2026 11:15
…y which allows for a nice normalization of the Poisson NLL. The slow tests will now likely fail
…malizations, and add the new normalization to all models
Adds 'intensity_mse' as a selectable loss function (alongside the
existing 'amplitude_mse' and 'poisson_nll') to FancyPtycho,
MultislicePtycho, Bragg2DPtycho, Multislice2DPtycho, and RPI.
Models that previously had a hardcoded amplitude_mse assignment now
use the same configurable pattern as FancyPtycho, with the loss
parameter threaded through from_dataset and from_calibration as well.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Updates tutorial_simple_ptycho.py to match simple_ptycho.py: removes
the def loss() method and instead assigns self.loss and
self.loss_normalizer as instance attributes in __init__.

Updates tutorial.rst accordingly: adds the loss assignment to the
__init__ code block with an explanation, and removes def loss() from
the forward model section.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
poisson_nll now returns a sum rather than a mean, consistent with the
normalizer pattern. Remove the per-pixel divisions from the numpy
reference calculations accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Use poisson_nll in test_near_field_ptycho and intensity_mse in
test_Adam_gold_balls to exercise these loss paths end-to-end.
Thresholds left as-is pending re-running on a GPU machine.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Revert test_Adam_gold_balls to default amplitude_mse
- Add new test_intensity_MSE test (gold balls + AdamReconstructor)
- Update near_field threshold to 17 (poisson_nll scale)
Thresholds for test_intensity_MSE to be tuned after GPU run.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant