Finalise API for (de)compression methods

[Cadair]

As mentioned in #15 (comment) and various other places we need to decide and pin down the API for the input and output to all the encode and decode methods.

Many of the methods require knowing the tile size and the dtype of the buffer (maybe other things?).

We should probably take a lot of inspiration or even directly steal what numcodecs does, but at the very least we need our API to be compatible (i.e. a valid subset) of theirs.

The numcodecs helper function ensure_contiguious_ndarray looks like it would do what we want?

https://github.com/zarr-developers/numcodecs/blob/be267852afaba9f67c534f106bbc7586a41cbc60/numcodecs/compat.py#L126

I guess we need to decide if we want to depend on numcodecs or vendor / reimplement that helper function?

[Cadair]

@astrofrog Your thoughts?

[astrofrog]

Ok so now I have implemented all compression types, and we definitely need to be able to pass in the dtype of the array when compressing values, as well as the tile size (for PLIO_1 and RICE_1) or shape (for HCOMPRESS).

When decompressing values, we need to return not just bytes but information about the dtype in which the values are represented since that might not match ZBITPIX which is the final depth (e.g. for HCOMPRESS, either int32 or int64 is returned but we might need to later cast to ZBITPIX=16).

I think this means we can't ever accept pure bytes and return pure bytes - we need to accept/return either buffers with the dtype and shape set or just restrict ourselves to Numpy arrays.

One thing that isn't immediately obvious to me is whether the buffer API allows one to actually get the dtype or just the itemsize - if the latter it will be problematic as we won't be able to check that the inputs are integers, or distinguish between int32 and uint32 for example, or between different endians (important with FITS files). In this scenario I think we would need to accept/return Numpy arrays only.

[astrofrog]

Buffers allow the dtype to be encoded in 'format' so in fact we can just accept/return buffer objects.

Note that np.frombuffer doesn't actually properly interpret buffer objects, one should use np.asarray instead:

In [56]: x = np.arange(16).reshape((4, 4))

Access the memoryview object (which is buffer-API-compliant):

In [57]: x.data
Out[57]: <memory at 0x7f8ded81bb90>

In [58]: x.data.format
Out[58]: 'l'

In [59]: x.data.strides
Out[59]: (32, 8)

In [60]: np.frombuffer(x.data)
Out[60]: 
array([0.0e+000, 4.9e-324, 9.9e-324, 1.5e-323, 2.0e-323, 2.5e-323,
       3.0e-323, 3.5e-323, 4.0e-323, 4.4e-323, 4.9e-323, 5.4e-323,
       5.9e-323, 6.4e-323, 6.9e-323, 7.4e-323])

In [61]: np.asarray(x.data)
Out[61]: 
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11],
       [12, 13, 14, 15]])

In [62]: np.frombuffer(x[::2].data)
---------------------------------------------------------------------------
BufferError                               Traceback (most recent call last)
Input In [62], in <cell line: 1>()
----> 1 np.frombuffer(x[::2].data)

BufferError: memoryview: underlying buffer is not C-contiguous

In [63]: np.asarray(x[::2].data)
Out[63]: 
array([[ 0,  1,  2,  3],
       [ 8,  9, 10, 11]])

[astrofrog]

In #19 I've made it so we properly accept/return 'real' buffer objects (not byte strings or numpy arrays)

[astrofrog]

I'm milestoning this as not being critical for the first PR since our plan to have no public API initially