df/f plots producing huge peaks toward end of recording session

[aatamian]

I was wondering if there was a way to remove these huge peaks that are appearing in my df/f plots after running lizard wizard. Can there be an upper threshold setting on wizards staff that allows me to remove them after having it run through lizard wizard?

[mrjholt]

The most relevant Lizard Wizard parameters --f_baseline_perc (default 8) and --win_sz (default 500). If your baseline percentile is too low or the window too narrow, you can get unstable F0 estimates that produce artificially large ΔF/F values. Bump --f_baseline_perc up slightly will stabilize the baseline and reduce extreme ratio spikes. You could also increase --win_sz to smooth the baseline estimate over a longer window. Wizards Staff doesnt have an upper threshold

[aatamian]

I changed the f_baseline_perc value up to 15 and caiman ignored all of the 7 wells (even though it generated a df/f plot for most of the wells I passed through it (5 out of 7 wells)). Of the 5 it generated, the wells still had the same large peak as was seen when the f_baseline_perc value was at default 8. I dropped it down from 15 to 10 and it did not address 3 out of the 4 images that has a huge peak (the 4th well with a great peak did not generate a df/f plot at all with this increase in the f_baseline_perc value). I'll try to change the --win_sz and see what can change from modulating that.

[mrjholt]

Worth trying --win_sz to see if it helps but given what you're seeing I'd also suggest looking at the CaImAn quality control parameters as well. Tightening --min_corr (try 0.85–0.9) and --min_pnr (try 6–8) would be more selective about which components get accepted in the first place so this would filter out the ones that tend to produce those extreme traces. You could also try bumping --min_SNR from 3 to 4. Too strict is better than not strict enough filtering

[aatamian]

I've bumped up the SNR to 5 and it cleaned up the other traces but the 4 of the wells that had that huge peak still remained. I increased the min_corr and min_pnr value to 0.95 and 10 respectively and it changed some things, but It still introduced a massive peak into a well that didn't have it to begin with. I'll be starting to combine the alterations of these parameters now that I've gone through and changed each one singularly and see what the combined one would look like.

[mrjholt]

The fact that cranking min_corr to 0.95 and min_pnr to 10 actually introduced a huge peak into a well that was previously clean makes sense. At those extreme values, you're rejecting so many components that the remaining few may be poorly isolated or have weird spatial profiles that produce artifact traces you are seeingf

As you start combining parameters, it's worth keeping in mind what each one is actually doing mechanistically:

• min_SNR controls how confident CaImAn needs to be that a detected transient is real signal vs. noise
• min_corr is about how spatially coherent a component looks in the correlation image. If you push it too high and you bias toward only "perfect" looking ROIs you can paradoxically favor artifacts
• min_pnr sets how much a component's peak signal needs to stand out above background noise

They all filter the same pool of candidate components, just from different angles, so understanding which problem each one addresses should guide the combinations rather than doing large individually sweeping setting of values.

That said, the persistent huge peaks surviving even aggressive upstream filtering tells us this likely needs a post-hoc solution rather than trying to solve it purely at the CaImAn detection stage. I'm going to look into adding an upper threshold feature in Wizards Staff so we can handle this downstream without compromising detection

[aatamian]

Sounds good. I'll keep trying out different combinations and take a look at the outputs. I agree that some sort of post-hoc upper threshold feature would be great in Wizards Staff. What would help is if I could somehow get the values for the y-axis on these df/f plots to get an idea of what the height is of these "huge" peak and see how variable it is from plot to plot and understand what that minimum upper threshold value would have to be.

[mrjholt]

closing this out since its being handled in wizards staff