Some questions

[h-2]

I hope this is the right place to ask! First of all, this looks really nice, and I would love to replace the implementations I have with this!

Scanning through the source briefly, I came to the following conclusions, are they correct?

• the code cannot be used to wrap other istreams/ostreams, only regular files (from filenames)
• the strength of the compression cannot be configured for ostreams
• the implementation for BGZF is single-threaded

Have you considered these things before and are they things that

• you do not want to change (to avoid feature creep)
• you want to change but haven't done so yet
• you don't want to change but might accept contributions for

?

Thanks for providing this library in any case!

[jonathonl]

• the code cannot be used to wrap other istreams/ostreams, only regular files (from filenames)

Yeah, I have personally only had a need for file streams, but I can see the advantage of implementing something analogous to std::stringbuf and std::stringstream. If implemented, I would want to rename (or typedef) the existing classes to be more consistent to std lib naming convention (gz::ofstream, gz::filebuf, etc.). This is a low priority for me, but would accept a PR if you found it useful.

• the strength of the compression cannot be configured for ostreams

This has only been implemented for zstd::obuf (https://github.com/jonathonl/shrinkwrap/blob/master/include/shrinkwrap/zstd.hpp#L193), but I was planning to add support for the gzip buffer and probably the stream classes as well. I will probably do this in the short term

• the implementation for BGZF is single-threaded

This has been in the back of my mind. BGZF would be the easiest since the block sizes are predictably small. One could just buffer multiple blocks worth of uncompressed data and then compress the blocks in parallel. I'll eventually get around to this.

[h-2]

Thanks for the quick reply!

Yeah, I have personally only had a need for file streams, but I can see the advantage of implementing something analogous to std::stringbuf and std::stringstream

To us, the ability to read from std::cin and write to std::cout is quite important. I will see if I can make a PR for this at some point!

This has only been implemented for zstd::obuf

Ah, I missed that one!

but I was planning to add support for the gzip buffer and probably the stream classes as well. I will probably do this in the short term

That's great to hear!

This has been in the back of my mind. BGZF would be the easiest since the block sizes are predictably small. One could just buffer multiple blocks worth of uncompressed data and then compress the blocks in parallel. I'll eventually get around to this.

That's also great to hear. We have been using parallel queues to implement this, but I have been wondering if a simpler solution might not work equally well (or better).

In case you are interested, here is a link to the implementation:
https://github.com/seqan/seqan/blob/master/include/seqan/stream/iostream_bgzf.h
And here's a more modern version of the code:
https://github.com/seqan/seqan3/blob/master/include/seqan3/contrib/stream/bgzf_istream.hpp

[jonathonl]

To us, the ability to read from std::cin and write to std::cout is quite important. I will see if I can make a PR for this at some point!

I see. I tend to use "/dev/stdin" and "/dev/stdout", though this will not work on Windows. For cross compatible, I suppose you would have to use FILE* stdin:

shrinkwrap::gz::ibuf buf(stdin);
std::istream is(&buf);

[h-2]

I made a small benchmark you might be interested in: https://github.com/h-2/bgzf_bench
(only reading for now)