-
Notifications
You must be signed in to change notification settings - Fork 4
True random access support for LZ4 lazy decompression #90
Copy link
Copy link
Open
Labels
enhancementNew feature or requestNew feature or requestrefactoringChanges to code structure or APIsChanges to code structure or APIs
Description
Follow-up to #88
Should split this to a separate task since it will take a little longer to get right, and isn't of the highest priority since lazy decompression is experimental anyways.
Lazy decompression does still work with LZ4 but only sequentially. But since we store the LZ4 compressed data in a serious of individual blocks, from which we can easily build an index of which block contains which chunk of the decompressed data, it is possible to decompress only individual blocks.
I have in mind how to do this but it does require a fair bit more bookkeeping overhead regarding building the index, tracking which blocks have been fully read in, so on.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestrefactoringChanges to code structure or APIsChanges to code structure or APIs