Single `rs.read` function that detects file extension

[dennisbrookner]

Summary

There are currently a number of file-format-specific read_ functions:

• read_mtz
• read_cif
• read_crystfel
• read_precognition
• read_csv
• read_pickle

These should obviously all continue to exist, for use cases where one and only one file format is an acceptable input. However, there are many cases where any of these inputs are acceptable, and this ends up requiring a bunch of control logic to check what file format the user has supplied. I think a more mature API should have a single rs.read (or perhaps rs.read_data or something) function which would have the call signature

rs.read(filename, fileformat='auto')

When fileformat is left as "auto", the function would check for an expected set of file extensions (which we would want to carefully curate) and if none of those are found, raise an error and tell the user to supply the file format.

Example implementation elsewhere

A good example of this type of API is mdtraj's mdtraj.load, which wraps around a number of format-specific loading functions.

Design choices

• What file formats are accepted, and what do we call these formats?
• I believe that both rs.read_crystfel and rs.read_precognition accept.hkl files. Do we pick one or the other to be the default? Require specification? Are there any other similar duplicates? Is this an unsurmountable issue for implementing this function?
• Are there situations where the downstream code needs to know which file format was read? I tend to think not; code may need to check for things like "are there phases?", "is this merged?" etc., but in principle none of those are format-specific. But, in theory, the function could take an optional argument like return_extension and then return a tuple of (rs.DataSet, filetype). That feels overcomplicated to me; a developer can always just chose to keep using the format-specific load functions if needed.
• Are there any other weird edge cases to consider?

I'm happy to take a stab at this and make a PR if that's of interest.

[kmdalton]

I have no objection to this. I'd be happy to review a PR. I will also point out that we have a hidden method, rs.io.read_dials_stills which hasn't made it to the top level yet. Maybe we should consider integrating that if feasible.

[dennisbrookner]

Great! I'll take a stab at this when I get a chance.

Any thoughts on the default behavior for .hkl files? I don't know how heavily used the read_precognition and read_crystfel functions are at this point. I also don't know the architecture of an .hkl file; what happens if you try to mix and match those file formats? Will it throw an error, or just read the file in a foolish way? If it reliably errors out, then it's easy enough to try both with a try/except situtation.

It maybe doesn't matter too too much, as long as we clarify that using this rs.read function is accepting the risks of auto-detection.

[DHekstra]

right, there are multiple hkl formats out there. HKL2000 got one too, I think.
the doc string could encourage the user to supply a CSV file instead with a single row with column headers.

[dennisbrookner]

If you have any handy, could you slack me an example file in some of these formats?

[DHekstra]

see also https://github.com/project-gemmi/gemmi/blob/master/docs/hkl.rst -- GEMMI seems to already handle XDS hkl files. I don't have an HKL2000 sample at hand.

[DHekstra]

also,

from careless.io import xds
ds=xds.read_hkl(hkl_name)

[DHekstra]

either way, it seems straightforward to provide XDS HKL file support in rs