Add scale_global_freshwater_to_zero scheme

[anton-seaice]

Scale global freshwater to zero, per ACCESS-NRI/access-om3-configs#920

[anton-seaice]

This scaling in non-deterministic, at some points, on some processors, at some times

[anton-seaice]

This calculation of "global_fw_sum" is not deterministic

e.g. in the first timestep across three runs:

-2.221565603340084461E-06
-2.221565603340084885E-06
-2.221565603340084038E-06

Which sometimes has the impact of changing precip_fact calculated below:

e.g. 5th timestep in two runs:

(med_phases_prep_ocn_balance_freshwater): global_precip_sum   2.445005571174440969E-05
(med_phases_prep_ocn_balance_freshwater): global_fw_sum  -9.422571120482979517E-07
(med_phases_prep_ocn_balance_freshwater): Scaling rain & snow by non-unity precip_fact   1.038538035379432234E+00
(med_phases_prep_ocn_balance_freshwater): global_fw_sum  -3.194984688258079762E-20

(med_phases_prep_ocn_balance_freshwater): global_precip_sum   2.445005571174440969E-05
(med_phases_prep_ocn_balance_freshwater): global_fw_sum  -9.422571120482981635E-07
(med_phases_prep_ocn_balance_freshwater): Scaling rain & snow by non-unity precip_fact   1.038538035379432456E+00
(med_phases_prep_ocn_balance_freshwater): global_fw_sum  -3.033213311637417572E-20

@manodeep (& @dougiesquire ) - I am trying to calculate a scaling factor based on some global sums, and then apply the scaling factor to all values of an array. I've found the scaling factor is not deterministic - appears to be impacted by machine precision ?

This is where the problem occurs:
(precip_fact = 1 - (global_fw_sum(1)/global_precip_sum(1)))

Is clipping precip fact at 16 sig figures the best approach, or is there a better way ( by reordering operations or something? )

[manodeep]

Reductions of floats are not required to be deterministic by the MPI standards, and, consequently, frequently aren’t unless you take extra steps. If you don’t need the extra precision, you could chop off some of those extra digits and hopefully, the truncated numbers will be deterministic.

These are 64-but floats right?

[anton-seaice]

thanks !