One more checksum issue

[DennisHeimbigner]

In discussion #6, I noted a difference in how checksums were computed.

The essence of this problem is as follows:

The difference then is that the netcdf checksum includes the counts for variable length data and Hyrax excludes the counts.

Including the counts allows checksum computation at a lower level of the stack independent (more or less) of the dmr. I have implemented both and including the counts is very slightly faster, but not by much.

So, as a stake in the ground, I am proposing that we include the count when computing the checksum.

[DennisHeimbigner]

A clarification. The counts to which I refer are the counts for variable length variables.
It does not refer to chunk level counts.

[DennisHeimbigner]

One more point, checksum computing using the Hyrax method requres
recursion since for a complex type, the count may occur in a variety of
places in the variable's data depending on the occurrence of strings and
sequences in the type.

[ndp-opendap]

@DennisHeimbigner -

Can you provide an example dataset that would demonstrate this when serialized as a DAP4 Data Response?

Thanks - @ndp-opendap

[DennisHeimbigner]

It has been a long time since I tested this issue; the original discussion #6 was created In February 2023.

So I did the tests again using this dataset: dap4://test.opendap.org/opendap/nc4_test_files/nc4_strings.nc?dap4.checksum=true.

Apparently you guys changed the checksum computation -- at least for arrays of string -- to include the counts
(or possibly it always worked for strings and I only ran my test on Sequences).
So now it works "correctly" for strings (from my point of view). Any idea when/if you changed this?

I will also try to test it with a sequence to see if the inclusion of counts also works for Sequence types.
Can you point me to a simple sequence test on test.opendap.org?

[Mikejmnez]

I will also try to test it with a sequence to see if the inclusion of counts also works for Sequence types. Can you point me to a simple sequence test on test.opendap.org?

@DennisHeimbigner Sorry about the delayed response! I found this very simple sequence (4 rows): temperature.csv