fix: read directory entries in name order so packs are reproducible - #4
Merged
Boy0000 merged 1 commit intoJul 31, 2026
Conversation
duranaaron
force-pushed
the
nexo/deterministic-directory-read-order
branch
from
July 29, 2026 15:44
5486c08 to
c103b50
Compare
|
looks good, thanks |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DirectoryFileTreeReader.hasNext()callsfolder.listFiles()and reads the array as it comes back. That order is whatever the file system hands over, and it is not the same on every machine: ext4 returns entries in hash order that depends on when each file was created, APFS and NTFS have their own orders again. The reader's output order becomes the zip entry order of the pack that gets written, so the same folder of assets serializes to a different byte stream, and a different SHA-1, on two different hosts.This shows up on any setup where more than one machine builds a pack from the same sources and the results are expected to match. In our case several backend servers each generate a pack from an identical asset folder and a proxy has to serve one of them to every player. Today they disagree, and which host built the pack decides what a player downloads.
The change sorts each folder's children by name before reading them:
Traversal order is otherwise untouched: files still come before the subdirectories they sit next to, and subdirectories are still visited breadth-first in the order they were found. Only the order within a single folder becomes fixed. Reading is already the slow part next to a
listFiles()syscall, so sorting a few dozen names per folder does not register.The added test in
DirectoryFileTreeReaderTestwritesz.txt,m.txt,a.txtand asubfolder in non-alphabetical order and asserts the reader returnsa.txt, m.txt, z.txt, sub/b.txt, sub/y.txt. It fails onmasterwithout the change (verified on APFS, where creation order is preserved) and passes with it../gradlew buildis green on this branch.The same fix is open upstream as unnamed#93, where it has an lgtm from @FixedDev. Opening it here as well because this fork is what Nexo actually builds against, and a merge here reaches installs without waiting on the upstream chain.