Skip to content

Support obsm embeddings in the Loader#252

Open
moinfar wants to merge 1 commit into
scverse:mainfrom
moinfar:support_obsm
Open

Support obsm embeddings in the Loader#252
moinfar wants to merge 1 commit into
scverse:mainfrom
moinfar:support_obsm

Conversation

@moinfar

@moinfar moinfar commented Jul 13, 2026

Copy link
Copy Markdown

Summary

Adds support for loading dense obsm arrays (e.g. a scVI embedding "X_emb") through the mini-batch Loader, yielded row-aligned with X.

  • New obsm_keys argument on Loader.add_adata / add_adatas (and an obsm argument on add_dataset / add_datasets) selects which obsm keys to load — only the requested keys are read, keeping I/O minimal.
  • Backward compatible: on-disk store format is untouched; existing calls are unchanged. The only observable difference is the extra obsm=None key in each yielded batch.
  • Validation raises clear errors for: non-dense obsm, mismatched n_obs, and inconsistent keys / feature-shape / dtype across datasets. obsm dtype/shape is required to be homogeneous across datasets (kept simple; can be relaxed later).

Use of AI agents

This change was developed with AI assistance (Claude Code). I briefly reviewed all changes (the implementation and tests were authored via the tool and reviewed by me). The correctness of row alignment, concurrency, and edge-case handling was additionally checked by a second run of an AI agent.

@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.02985% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.60%. Comparing base (6269ade) to head (dac0578).

Files with missing lines Patch % Lines
src/annbatch/loader.py 93.93% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #252      +/-   ##
==========================================
- Coverage   93.29%   91.60%   -1.70%     
==========================================
  Files          15       15              
  Lines        1432     1489      +57     
==========================================
+ Hits         1336     1364      +28     
- Misses         96      125      +29     
Files with missing lines Coverage Δ
src/annbatch/types.py 100.00% <100.00%> (ø)
src/annbatch/loader.py 88.53% <93.93%> (-2.40%) ⬇️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@moinfar

moinfar commented Jul 13, 2026

Copy link
Copy Markdown
Author

Is related to #91 and #102

@selmanozleyen

Copy link
Copy Markdown
Member

Thanks! this will be done in the following PR's after #251, because we need to warn the users about the changing default behaviour first.

@moinfar

moinfar commented Jul 13, 2026

Copy link
Copy Markdown
Author

Thanks a lot.

Comment thread src/annbatch/loader.py
self,
adatas: list[ad.AnnData],
*,
obsm_keys: list[str] | None = None,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the goal is not to have to special-case this but to use https://anndata.scverse.org/en/stable/accessors.html

i.e.,

Loader(...).add_adatas(adatas, acc={A.obsm["pca"], A.layers["counts"]})

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This accessor concept is really interesting.
Looking forward to seeing it in the annbatch interface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants