Match pynndescent params and fix diffusion map reproducibility#6
Merged
Conversation
- pynndescent now accepts random_state and n_jobs, and uses the same UMAP/scanpy-matching NNDescent parameters as scib-metrics (n_trees, n_iters, max_candidates, low_memory, compressed). - Use deterministic v0 in eigsh for reproducible diffusion embeddings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Eigsh can return eigenvectors with arbitrary signs, causing non-deterministic diffusion embeddings across processes. Canonicalize by making the largest absolute element of each eigenvector positive. Note: this does not fully resolve kbet_per_label diffs for small clusters with degenerate eigenvalues (repeated eigenvalues produce a non-unique eigenspace that sign-flipping alone cannot canonicalize). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pynndescentnow acceptsrandom_stateandn_jobs, and uses the same UMAP/scanpy-matching NNDescent parameters as scib-metrics (n_trees,n_iters,max_candidates,low_memory,compressed)v0ineigshfor reproducible diffusion embeddingsImpact
These changes fix neighbor graph discrepancies between scib-metrics and scib-rapids that caused ~1-2% differences in
kbet_per_labeland exact matches for all neighbor-dependent metrics (leiden, kbet, lisi).Note: small diffs remain for
kbet_per_labelon clusters with degenerate eigenvalues (repeated eigenvalues in the transition matrix produce a non-unique eigenspace). This is a fundamental limitation ofeigshon small structured graphs, not an implementation bug.🤖 Generated with Claude Code