Skip to content

fix: read directory entries in name order so packs are reproducible - #4

Merged
Boy0000 merged 1 commit into
Nexo-MC:masterfrom
duranaaron:nexo/deterministic-directory-read-order
Jul 31, 2026
Merged

fix: read directory entries in name order so packs are reproducible#4
Boy0000 merged 1 commit into
Nexo-MC:masterfrom
duranaaron:nexo/deterministic-directory-read-order

Conversation

@duranaaron

Copy link
Copy Markdown

DirectoryFileTreeReader.hasNext() calls folder.listFiles() and reads the array as it comes back. That order is whatever the file system hands over, and it is not the same on every machine: ext4 returns entries in hash order that depends on when each file was created, APFS and NTFS have their own orders again. The reader's output order becomes the zip entry order of the pack that gets written, so the same folder of assets serializes to a different byte stream, and a different SHA-1, on two different hosts.

This shows up on any setup where more than one machine builds a pack from the same sources and the results are expected to match. In our case several backend servers each generate a pack from an identical asset folder and a proxy has to serve one of them to every player. Today they disagree, and which host built the pack decides what a player downloads.

The change sorts each folder's children by name before reading them:

Arrays.sort(children, Comparator.comparing(File::getName));

Traversal order is otherwise untouched: files still come before the subdirectories they sit next to, and subdirectories are still visited breadth-first in the order they were found. Only the order within a single folder becomes fixed. Reading is already the slow part next to a listFiles() syscall, so sorting a few dozen names per folder does not register.

The added test in DirectoryFileTreeReaderTest writes z.txt, m.txt, a.txt and a sub folder in non-alphabetical order and asserts the reader returns a.txt, m.txt, z.txt, sub/b.txt, sub/y.txt. It fails on master without the change (verified on APFS, where creation order is preserved) and passes with it. ./gradlew build is green on this branch.

The same fix is open upstream as unnamed#93, where it has an lgtm from @FixedDev. Opening it here as well because this fork is what Nexo actually builds against, and a merge here reaches installs without waiting on the upstream chain.

@duranaaron
duranaaron force-pushed the nexo/deterministic-directory-read-order branch from 5486c08 to c103b50 Compare July 29, 2026 15:44
@Boy0000

Boy0000 commented Jul 31, 2026

Copy link
Copy Markdown

looks good, thanks

@Boy0000
Boy0000 merged commit d15b269 into Nexo-MC:master Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants