Checksums have issues...

[ndp-opendap]

And we need to fix them!

Should checksums really be optional?

How should they be requested if they continue to be optional? (The TDS currently accepts a query parameter, dap4.checksum=true, which will cause it to provide checksums for variables.

The narrative about checksums says:

As an option, checksums will be computed for the values of all the “top-level” variables present in the DMR of a returned response from a server. The term “top-level” means that the variable is not a field of a Structure (or Sequence) typed variable.

Which could be restated as:

As an option, checksums will be computed for the all variables that are present in the DMR of a returned response from a server, with the exception of variable members of a Structure or a Sequence.

[DennisHeimbigner]

The problem is that since Hyrax provides checksums by default and TDS does not, we will be forced
to break one of the two server implementations. I have always been in favor of making the default be
no-checksums. This is because the checksum is explicitly not cryptographic but rather is used to
detect changes in the returned data between calls. I guess is that this change detection capability is
very rarely used.

[jgallagher59701]

The good thing about optional checksums is that the server only needs to compute them when they will be needed, saving time. And it's significant. The time spent computing the checksum was about equal to the time DAP2 spent turning little endian bytes to big endian.

The bad thing about optional checksums is that optional behaviors (seem to) make interoperability less likely.

Can we amend the spec so that it's not an optional behavior and, at the same time. not a constant burden.

Maybe an additional response type? That is, you make a checksum request and get a checksum response - it's neither part of the DMR nor the data response.

I'm wondering how to write clients that can read from both existing servers.

[DennisHeimbigner]

Can we amend the spec so that it's not an optional behavior and, at the same time. not a constant burden.

As far as I can see, this can only happen if no-checksum is the default and some variant of dap4.checksum=true|false is supported.

[ndp-opendap]

I am updating Hyrax to use optional dap4 checksums and include them only when a client includes the query parameter dap4.checksum=true

There is also a wrinkle with the response that I have opened an additional ticket to address.