__`. The corresponding p-value is saved under the same column name pattern with suffix `p`
+Once calculated, the correlation values against pseudotime are saved in the feature attribute/metadata table (`I__RNA_pseudotime__r`, here). The column name follows this pattern: `____r`. The corresponding p-value is saved under the same pattern with the suffix `p` (`I__RNA_pseudotime__p`)
```{code-cell} ipython3
ds.RNA.feats.head()
```
---
-### 4) Visualize pseudotime correlated features
+## 4) Visualize pseudotime correlated features
In this section will do deeper on how to use the pseudotime correlation values for further exploratory analysis.
@@ -152,7 +152,7 @@ ds.plot_layout(
```
---
-### 5) Identify feature modules based on pseudotime
+## 5) Identify feature modules based on pseudotime
`run_pseudotime_marker_search` is excellent to find the genes are linearly correlated with the pseudotime. This function provides us informative statistical metrics to identify genes that are most strongly correlated with the pseudotime. However, with these methods we do not recover all the dynamic patterns of expression along the pseudotime. For example, there might be certain genes that express only in the middle of the trajectory or in one branch of the trajectory.
@@ -172,9 +172,11 @@ There are two primary results of `run_pseudotime_aggregation`:
1) The binned matrix is saved under `aggregate___`
2) Feature clusters are saved under feature attributes table
+Features with mean expression below `min_exp` or with no variation along the ordering are treated as invalid. They are excluded from the clustering and from the heatmap below, and they receive the unassigned cluster value (`-1`) in the feature table.
+
```{code-cell} ipython3
# The binned data matrix. Here we print the shape of the matrix indicating the number of features and numner of bins respectively
-ds.z.RNA.aggregated_I_I_RNA_pseudotime.data.shape
+ds.RNA.z['aggregated_I_I_RNA_pseudotime/data'].shape
```
```{code-cell} ipython3
@@ -198,7 +200,7 @@ ds.plot_pseudotime_heatmap(
)
```
-The heatmap above shows the gene expression dynamics as the cells progress throught the pseudotime. Here, cluster 1 captures the genes that have highest expression in early pseudotime while cluster 15 captures genes whose expression peak in the late pseudotime.
+The heatmap above shows the gene expression dynamics as the cells progress through the pseudotime. Each block of rows is one feature module. Some modules capture genes that peak early in the pseudotime while others peak later. The module numbers are assigned by the clustering step and do not follow the pseudotime order.
We can visualize the expression of the above selected genes on UMAP to check whether their cluster identity corroborates their expression pattern.
@@ -214,7 +216,7 @@ ds.plot_layout(
```
---
-### 6) Merging pseudotime-based feature modules into a new assay
+## 6) Merging pseudotime-based feature modules into a new assay
The pseudotime based clusters of features can be used create a new assay. `add_grouped_assay` will take each cluster and take the mean expression of genes from that cluster and add it to a new assay. The motivation behind this approach is that we do not have to add many columns to our cell metadata table and have the mean cluster values readily available for analysis.
@@ -259,7 +261,7 @@ This figure complements the heatmap we generated earlier very nicely. Using this
+++
---
-### 7) Comparing pseudotime based feature modules with cluster markers
+## 7) Comparing pseudotime based feature modules with cluster markers
Here we will compare the pseudotime based feature module extraction approach with classical cluster marker approach.
@@ -268,44 +270,61 @@ Here we will compare the pseudotime based feature module extraction approach wit
ds.run_marker_search(group_key='RNA_cluster')
```
-Here we extract features from pseudotime-based cluster/group 13. These genes are the ones that show high expressio in Beta cells.
+First we extract the marker genes for cell cluster 8, which predominantly contains the Beta cells.
+
+```{code-cell} ipython3
+cell_cluster_markers = ds.get_markers(
+ group_key='RNA_cluster',
+ group_id='8',
+).feature_name
+
+cell_cluster_markers.head()
+```
+
+Next we pick the pseudotime feature module that shares the most genes with these Beta cell markers. The module numbering is assigned by clustering and can change between runs, so we select the module in a data-driven way instead of hard-coding a cluster id.
```{code-cell} ipython3
ptime_feat_clusts = ds.RNA.feats.to_pandas_dataframe(
columns=['names', 'pseudotime_clusters']
)
-ptime_based_markers = ptime_feat_clusts.names[ptime_feat_clusts.pseudotime_clusters == 13]
-ptime_based_markers.head()
+
+beta_marker_names = set(cell_cluster_markers)
+assigned = ptime_feat_clusts[ptime_feat_clusts.pseudotime_clusters != -1]
+module_overlap = assigned.groupby('pseudotime_clusters')['names'].apply(
+ lambda names: len(set(names) & beta_marker_names)
+)
+beta_module = int(module_overlap.idxmax())
+beta_module
```
-Now we extract all the marker genes for cell cluster 8, this cluster predominantly contains the Beta cells.
+The genes belonging to this Beta associated module are:
```{code-cell} ipython3
-cell_cluster_markers = ds.get_markers(
- group_key='RNA_cluster',
- group_id='8'
-).feature_name
-
-cell_cluster_markers.head()
+ptime_based_markers = ptime_feat_clusts.names[
+ ptime_feat_clusts.pseudotime_clusters == beta_module
+]
+ptime_based_markers.head()
```
```{code-cell} ipython3
-# let's checkout the number of Beta cell associated genes from both methods
+# Number of genes captured by each approach
ptime_based_markers.shape, cell_cluster_markers.shape
```
```{code-cell} ipython3
-# let's checkout the number of common Beta cell associated genes from both methods
-len(set(cell_cluster_markers.index).intersection(ptime_based_markers.index))
+# Number of genes shared by both approaches, compared by gene name
+len(set(ptime_based_markers) & set(cell_cluster_markers))
```
Let's visualize the cumulative expression of genes that are present only in cluster marker based approach
```{code-cell} ipython3
-temp = list(set(cell_cluster_markers.index).difference(ptime_based_markers.index))
+name_to_idx = dict(zip(ptime_feat_clusts.names, ptime_feat_clusts.index))
+cell_only = sorted((set(cell_cluster_markers) - set(ptime_based_markers)) & set(name_to_idx))
+cell_only_idx = sorted(name_to_idx[name] for name in cell_only)
ds.cells.insert(
column_name='Cell cluster based markers',
- values=ds.RNA.normed(feat_idx=sorted(temp)).mean(axis=1).compute(),
+ values=ds.RNA.normed(feat_idx=cell_only_idx).mean(axis=1).compute(),
overwrite=True)
ds.plot_layout(
@@ -318,10 +337,11 @@ ds.plot_layout(
Let's now do this the other way and visualize the cumulative expression of genes that are present only in pseudotime-based approach
```{code-cell} ipython3
-temp = list(set(ptime_based_markers.index).difference(cell_cluster_markers.index))
+ptime_only = sorted((set(ptime_based_markers) - set(cell_cluster_markers)) & set(name_to_idx))
+ptime_only_idx = sorted(name_to_idx[name] for name in ptime_only)
ds.cells.insert(
column_name='Pseudotime based markers',
- values=ds.RNA.normed(feat_idx=sorted(temp)).mean(axis=1).compute(),
+ values=ds.RNA.normed(feat_idx=ptime_only_idx).mean(axis=1).compute(),
overwrite=True)
ds.plot_layout(
diff --git a/docs/source/vignettes/reference_atlas_mapping.md b/docs/source/vignettes/reference_atlas_mapping.md
new file mode 100644
index 00000000..a293935d
--- /dev/null
+++ b/docs/source/vignettes/reference_atlas_mapping.md
@@ -0,0 +1,132 @@
+---
+jupytext:
+ cell_metadata_filter: -all
+ text_representation:
+ extension: .md
+ format_name: myst
+ format_version: 0.13
+kernelspec:
+ display_name: Python 3 (ipykernel)
+ language: python
+ name: python3
+---
+
+(reference_atlas_mapping)=
+
+# Build and map a reusable reference atlas
+
+This workflow builds or refreshes a harmonized RNA/PCA graph, stores a versioned mapping reference, and then maps query cells without moving reference coordinates. It uses the Kang PBMC datasets from {ref}`the projection vignette `.
+
+Scarf implements a Symphony-style fixed-reference correction. The `symphonyStyleV1` variant uses fixed soft assignments and a scalar ridge term, so it should not be presented as a complete reimplementation of the Symphony R model. Its shared PCA, soft-assignment, and correction contracts are checked against a static fixture generated by Symphony R 0.1.3, including a nonzero correction case. Technical batch labels should describe preparation, donor, or sequencing batches, not biological cell types.
+
+```{code-cell} ipython3
+import numpy as np
+import pandas as pd
+import scarf
+
+scarf.fetch_dataset(
+ dataset_name='kang_15K_pbmc_rnaseq',
+ save_path='scarf_datasets',
+ as_zarr=True
+)
+scarf.fetch_dataset(
+ dataset_name='kang_14K_ifnb-pbmc_rnaseq',
+ save_path='scarf_datasets',
+ as_zarr=True
+)
+
+ds_ctrl = scarf.DataStore('scarf_datasets/kang_15K_pbmc_rnaseq/data.zarr')
+ds_stim = scarf.DataStore('scarf_datasets/kang_14K_ifnb-pbmc_rnaseq/data.zarr')
+```
+
+The reference must already have a selected RNA feature set. For a production atlas, `reference_batch` should contain known technical batches. This compact example uses one recorded control batch to demonstrate the artifact API.
+
+```{code-cell} ipython3
+ds_ctrl.cells.insert(
+ 'reference_batch',
+ np.repeat('control', ds_ctrl.cells.N),
+ overwrite=True
+)
+
+reference = ds_ctrl.build_mapping_reference(
+ feat_key='hvgs',
+ batch_columns=['reference_batch']
+)
+```
+
+Reload the artifact before mapping when the build and mapping occur in different sessions. Scarf validates its feature set, active cells, normalization, PCA loadings, corrected coordinates, ANN contract, and batch metadata before loading it. Rebuilding creates a content-addressed artifact and leaves prior complete artifacts intact.
+
+```{code-cell} ipython3
+reference = ds_ctrl.get_mapping_reference(feat_key='hvgs')
+```
+
+Map the query in two streaming passes. The first pass estimates batch and soft-cluster statistics. The second pass corrects query coordinates and queries the immutable reference ANN index. Missing reference features default to `reference_mean`, which contributes zero after reference scaling. In `symphonyStyleV1`, `query_batches` has one row per active query cell and its columns are combined into query-specific technical batch groups rather than matched to reference category names.
+
+```{code-cell} ipython3
+result = reference.map_query(
+ target_assay=ds_stim.RNA,
+ target_name='stim_symphony',
+ target_feat_key='hvgs_symphony',
+ save_k=5,
+ query_batches=pd.DataFrame(
+ {
+ 'reference_batch': np.repeat(
+ 'stimulated',
+ len(ds_stim.cells.fetch('ids', key='I'))
+ )
+ }
+ )
+)
+result
+```
+
+Inspect stored provenance, feature coverage, and the uncorrected and corrected latent coordinates. The two coordinate arrays support diagnostics; nearest neighbors are calculated from the corrected coordinates.
+
+```{code-cell} ipython3
+projection = ds_ctrl.z['RNA']['projections']['stim_symphony']
+dict(projection.attrs)
+
+uncorrected = projection['uncorrectedLatent'][:]
+corrected = projection['correctedLatent'][:]
+np.mean(np.abs(corrected - uncorrected))
+```
+
+Use held-out donors when selecting an abstention threshold. The example below only inspects evidence for the mapped query; its vote fractions are not calibrated probabilities. Rows that fail the threshold receive `NA` by default. `referenceDistancePercentile` is evaluated against the reference self-neighbor distribution, so it does not change with query composition.
+
+```{code-cell} ipython3
+evidence = ds_ctrl.get_target_label_evidence(
+ target_name='stim_symphony',
+ reference_class_group='cluster_labels',
+ threshold_fraction=0.6
+)
+evidence.head()
+```
+
+Split-conformal prediction sets are optional and require calibration examples that are exchangeable with future queries.
+
+```{code-cell} ipython3
+calibrated = ds_ctrl.get_target_label_evidence(
+ target_name='stim_symphony',
+ reference_class_group='cluster_labels',
+ calibration_nonconformity=np.array([0.08, 0.12, 0.2, 0.25]),
+ conformal_alpha=0.1
+)
+calibrated[['label', 'predictionSet']].head()
+```
+
+Opening the reference read-only returns arrays in memory by default. Large queries can supply a writable Zarr group through `result_store` so neighbors and latent coordinates remain out of core.
+
+The same reference can map an unshifted control query. Its corrected coordinates should remain close to its uncorrected coordinates, while the reference itself remains unchanged.
+
+```{code-cell} ipython3
+control_result = reference.map_query(
+ target_assay=ds_ctrl.RNA,
+ target_name='control_symphony',
+ target_feat_key='hvgs_control_symphony',
+ save_k=5,
+ query_batches=pd.DataFrame(
+ {'reference_batch': ds_ctrl.cells.fetch('reference_batch', key='I')}
+ )
+)
+control_result
+```
diff --git a/docs/source/vignettes/zarr_explanation.md b/docs/source/vignettes/zarr_explanation.md
index 62240877..d5f59192 100644
--- a/docs/source/vignettes/zarr_explanation.md
+++ b/docs/source/vignettes/zarr_explanation.md
@@ -63,7 +63,7 @@ Scarf uses [Zarr format](https://zarr.readthedocs.io/en/stable/) to store raw co
- Availability of compression algorithms like [LZ4](https://github.com/lz4/lz4) that provides very high compression and decompression speeds.
- Automatic storage of intermediate and processed data
-In Scarf, the data is always synchronized between hard disk and RAM and as such there is no need to save data manually or synchronize the
+In Scarf, the data is always synchronized between hard disk and RAM and as such there is no need to save data manually or synchronize the store by hand.
We can inspect how the data is organized within the Zarr file using `show_zarr_tree` method of the `DataStore`.
@@ -150,6 +150,8 @@ ds.cells.insert(
If we try to add values for a column that already exists then Scarf will throw an error. For example, if we simply rerun the command above, we should get an error
```{code-cell} ipython3
+:tags: [raises-exception]
+
ds.cells.insert(
column_name='is_clust1',
values=is_clust_1
@@ -175,7 +177,7 @@ Please checkout the API of the `Metadata` class to get information on how to per
+++
-Scarf uses Zarr format so that data can be stored in rectangular chunks. The raw data is saved in the `counts` level within each assay level in the Zarr hierarchy. It can easily be accessed as a [Dask](https://dask.org/) array using the `rawData` attribute of the assay. Note that for a standard analysis one would not interact with the raw data directly. Scarf internally optimizes the use of this Dask array to minimize the memory requirement of all operations.
+Scarf uses Zarr format so that data can be stored in rectangular chunks. The raw data is saved in the `counts` level within each assay level in the Zarr hierarchy. It can easily be accessed as a chunked array using the `rawData` attribute of the assay. This chunked array exposes a NumPy-like interface (indexing, reductions, ufuncs) but evaluates lazily by streaming over row chunks, which keeps the memory requirement bounded regardless of dataset size. Note that for a standard analysis one would not interact with the raw data directly. Scarf internally optimizes the use of this chunked array to minimize the memory requirement of all operations.
```{code-cell} ipython3
ds.RNA.rawData
@@ -293,5 +295,39 @@ ds.export_markers_to_csv(
)
```
+---
+### 6) Zarr v3 and sharded count arrays
+
+Scarf 0.33+ writes new datasets as Zarr v3. Existing v2 stores remain readable; new metadata written into a v2 store still uses v2-compatible codecs.
+
+Count matrices created by the writers are finalized as sharded arrays (default profile `fast_local`: chunks 512×512, shards 4096×4096). Inspect layout with `show_zarr_tree`, which prints chunk and shard info for arrays at each node.
+
+Set the storage profile with the `SCARF_ZARR_PROFILE` environment variable (`fast_local` or `cloud`) or the `zarrProfile` argument when opening a `DataStore`. To convert an older store to v3 with sharding:
+
+```bash
+uv run python -m scarf.tools.repack_zarr input.zarr output.zarr --profile fast_local
+```
+
+### Memory budget and remote stores
+
+Bound streaming memory when opening a store:
+
+```python
+ds = scarf.DataStore('path/to/data.zarr', mem_budget='8G', nthreads=4)
+```
+
+For object-storage backed Zarr, use `zarrProfile='cloud'` or set `SCARF_ZARR_PROFILE=cloud`. Pass `local_cache=True` to `make_graph` to stage normalized data locally before PCA and KNN, which reduces repeated remote reads.
+
+After Harmony batch correction, corrected embeddings are stored under the PCA reduction group as `harmonizedData` with attribute `isHarmonized=True`.
+
+### Metadata query helpers
+
+Subset cells programmatically without exporting full tables:
+
+```python
+idx = ds.cells.get_index_by('RNA_cluster', 3)
+ds.cells.sift(idx)
+```
+
---
That is all for this vignette.
diff --git a/profiling/.env.example b/profiling/.env.example
new file mode 100644
index 00000000..c77abd61
--- /dev/null
+++ b/profiling/.env.example
@@ -0,0 +1,6 @@
+# Local credentials for profiling s3:// access
+# cp profiling/.env.example profiling/.env
+
+R2_ENDPOINT=https://.r2.cloudflarestorage.com
+R2_ACCESS_KEY_ID=key
+R2_SECRET_ACCESS_KEY=secret
diff --git a/profiling/LEARNINGS.md b/profiling/LEARNINGS.md
new file mode 100644
index 00000000..e1ed407c
--- /dev/null
+++ b/profiling/LEARNINGS.md
@@ -0,0 +1,241 @@
+# Scarf cloud profiling learnings (100k / 250k)
+
+Date range: 2026-07-14 to 2026-07-15
+Environment: Modal `scarf_profiling`, app `scarf-profiling`, region `eu` (was `eu-west-1`; broadened for capacity), secret `scarf-r2`
+Data: `s3://scarf-tests/scarf-profiling/` (datasets / stores / results)
+Dataset source: nested CELLxGENE samples already prepared on R2
+
+This note is the baseline for quantifying later changes (500k+, code defaults, orchestrator CPU/mem tables). Times are stage wall seconds from result JSON. Peaks are `peakCgroupBytes`.
+
+## Objective
+
+Minimize wall time without pointless overprovisioning. Prefer using memory and CPU in ways that actually cut stage time. Do not treat `workingCopies` as a speed dial.
+
+## Code changes already wired
+
+| Change | Location | Effect |
+|--------|----------|--------|
+| Cloud default `targetChunkBytes` = 128 MiB when remote and unset | `scarf/storage/zarr_store.py` (`DEFAULT_CLOUD_TARGET_CHUNK_BYTES`) | Matches layout-sweep winner |
+| Auto marker batch when `gene_batch_size is None` | `scarf/markers.py` `resolve_marker_gene_batch_size` | `min(col_chunk, n_features, budgetCap)` with `budgetCap = (memoryBytes // workingCopies) // (n_cells * 32)` |
+| Optional profiling override | `profiling` `markerGeneBatchSize` | Layout sweep forced `50`; later runs leave unset for auto |
+
+## Machine sizes used (Modal hard limit vs Scarf budget)
+
+Two different numbers matter:
+
+| Concept | Meaning |
+|---------|---------|
+| Modal memory | Hard cgroup limit on the container (OOM kill if exceeded) |
+| Scarf `memoryBytes` / `scarfMemoryBudget` | Software budget used for chunk geometry at write time and auto marker batch size. Usually ~75% of Modal in our configs |
+
+| Run tag | Cells | Modal CPU | Modal RAM | Scarf budget | Notes |
+|---------|------:|----------:|----------:|-------------:|-------|
+| layout sweep (`baseline`, `chunk*`) | 100k | 4 | 32 GiB | ~24 GiB | Forced marker batch 50 |
+| `auto_markers_c4_m32` | 100k | 4 | 32 GiB | ~24 GiB | Auto markers; UMAP parallel off |
+| `auto_markers_c4_m32_scarf16` | 100k | 4 | 32 GiB | **16 GiB** | Done; markers 219s (faster than c4_m32) |
+| `auto_markers_c4_m16` | 100k | 4 | 16 GiB | ~12 GiB | Both Modal and Scarf cut together |
+| `auto_markers_c8_m32` | 100k | 8 | 32 GiB | ~24 GiB | Speed pack (UMAP/ANN parallel) |
+| `auto_markers_c8_m32_250k` | 250k | 8 | 32 GiB | ~24 GiB | Same speed pack |
+| `auto_markers_c8_m48_500k` | 500k | 8 | 48 GiB | 36 GiB | Done; 4339s, max peak ~10 GiB |
+| `auto_markers_c8_m64_1m` | 1M | 8 | 64 GiB | 48 GiB | In progress |
+
+Local layout TOMLs under `profiling/layouts/` are gitignored. Treat `LEARNINGS.md` as the durable record; recreate TOMLs from these rows when needed.
+
+## Constants that must stay stable when comparing
+
+| Knob | Value used in successful speed runs | Rule |
+|------|-------------------------------------|------|
+| `workingCopies` | 4 in profiling configs (library default is 8) | Model of concurrent in-memory copies. Change only if peaks/OOM show the model is wrong |
+| Assay / workflow seeds | graph 4466, umap/leiden 4444, `topN=2000`, `k=11`, `dims=50` | Keep fixed across A/B |
+| Modal vs Scarf coupling | Usually Scarf ≈ 75% of Modal | Decouple on purpose only for budget-mismatch experiments (below) |
+
+## How to quantify a future change
+
+1. Pick a fixed reference run tag (see tables below). Prefer `auto_markers_c8_m32` for 100k and `auto_markers_c8_m32_250k` for 250k.
+2. Change one variable (size, CPU, mem, parallel flags, or code). Keep `workingCopies` and workflow seeds fixed unless the experiment is about the copy model.
+3. Compare stage seconds and peak GiB. Also report total seconds and max peak.
+4. Result URIs: `{resultsUri}/results/{runTag}/{nRows}/{stage}.json`
+5. Spawn via deployed app bare `run_all_jobs.spawn(...)` (avoids stuck `with_options` queues from local `modal run`). Stage workers still apply config resources inside the deployed orchestrator.
+
+Useful call IDs from this campaign (`/tmp/scarf_calib_calls.txt`):
+
+```
+auto_markers_c4_m32=fc-01KXJX3H7988EH0TZM6EGP7AQC
+auto_markers_c4_m16=fc-01KXJZ3VFQKMZ0T77ET87AD7KD
+auto_markers_c8_m32=fc-01KXK3JTNTC44X44K3NFQRTDQ5
+auto_markers_c8_m32_250k=fc-01KXK8BRWK9P50XW0NY4FWE378
+auto_markers_c8_m48_500k=fc-01KXKE1ZCG0FG7KFEQ1H0QX35K
+auto_markers_c4_m32_scarf16=fc-01KXKEX6N0N44661M6ZX26NX9C
+```
+
+## Layout sweep @ 100k (4 CPU / 32 GiB)
+
+All layouts: forced `markerGeneBatchSize=50`, `annParallel=false`, `umapParallel=false`, workers=4, workingCopies=4.
+
+| Tag | Nominal chunk target | Total s | Max peak GiB | Markers s | Markers peak |
+|-----|---------------------|--------:|-------------:|----------:|-------------:|
+| baseline | memory-first / default | 2467 | 7.24 | 1742 | 3.51 |
+| chunk64m | 64 MiB | 1189 | 6.85 | 366 | 1.13 |
+| **chunk128m** | **128 MiB** | **1147** | **6.50** | **353** | **1.01** |
+| chunk256m | 256 MiB | 1301 | 6.43 | 536 | 1.20 |
+| chunk512m | 512 MiB | 1188 | 6.23 | 521 | 1.37 |
+
+**Learning:** ~128 MiB cloud count chunks win on total time. Larger nominal targets did not help markers and often hurt them. Baseline markers were catastrophic (1742s) under the old layout.
+
+Configs: `profiling/layouts/100k_{baseline,chunk64m,chunk128m,chunk256m,chunk512m}.toml`
+
+### Stage detail: layout winner (`chunk128m`)
+
+| Stage | Seconds | Peak GiB |
+|-------|--------:|---------:|
+| createStore | 91.3 | 3.44 |
+| initializeStore | 122.9 | 6.50 |
+| reopenStore | 5.8 | 0.48 |
+| filterCells | 15.9 | 0.48 |
+| markHvgs | 221.6 | 1.06 |
+| makeGraph | 159.7 | 6.43 |
+| runUmap | 154.8 | 0.73 |
+| runLeiden | 22.4 | 0.71 |
+| findMarkers | 352.7 | 1.01 |
+| **total** | **1147.1** | **6.50** |
+
+## Auto markers and machine experiments @ 100k
+
+Cloud 128 MiB default (no forced batch 50). Scarf budget ~24 GiB on 32 GiB Modal boxes (~12 GiB on 16 GiB).
+
+| Tag | CPU | Modal GiB | Parallel UMAP/ANN | Total s | Max peak | Markers s | Markers peak |
+|-----|----:|----------:|-------------------|--------:|---------:|----------:|-------------:|
+| auto_markers_c4_m32 | 4 | 32 | off | 1215 | 6.50 | 269 | 4.54 |
+| auto_markers_c4_m16 | 4 | 16 | off | 1177 | 6.48 | 316 | 4.95 |
+| **auto_markers_c8_m32** | **8** | **32** | **on** | **1095** | **7.11** | **331** | **7.11** |
+
+### Stage detail vs control
+
+| Stage | c4_m32 (control) | c4_m16 | c8_m32 (speed pack) |
+|-------|-----------------:|-------:|--------------------:|
+| createStore | 107s / 3.5G | 111s / 3.4G | 104s / 3.5G |
+| initializeStore | 196s / 6.5G | 137s / 6.5G | 141s / 6.4G |
+| reopenStore | 9s / 0.5G | 8s / 0.5G | 7s / 0.5G |
+| filterCells | 23s / 0.5G | 22s / 0.5G | 20s / 0.5G |
+| markHvgs | 255s / 1.1G | 225s / 1.4G | 241s / 1.1G |
+| makeGraph | 183s / 6.3G | 175s / 5.9G | 168s / 6.3G |
+| runUmap | 146s / 0.7G | 155s / 0.7G | **58s / 0.7G** |
+| runLeiden | 27s / 0.7G | 27s / 0.7G | 25s / 0.7G |
+| findMarkers | **269s / 4.5G** | 316s / 5.0G | 331s / 7.1G |
+| **total** | **1215s** | **1177s** | **1095s** |
+
+### Learnings (100k resources)
+
+1. **Auto marker batches beat forced batch 50** on markers (353s → 269s at 4CPU/32GiB) and raise marker peak (1.0 → 4.5 GiB). That is productive use of Scarf budget.
+2. **Shrinking Modal RAM alone is the wrong optimization for speed.** 16 GiB vs 32 GiB: totals similar; markers got slower (269 → 316s). Peaks stayed ~6.5 GiB on non-marker heavy stages.
+3. **Extra Modal RAM does nothing unless Scarf `memoryBytes` grows** so auto batches can grow under fixed `workingCopies`.
+4. **Parallel UMAP is the clearest speed win** (146s → 58s). Almost no RAM change.
+5. **8 workers did not speed markers**; they slowed them (269 → 331s) even with higher peak (7.1 GiB). Treat marker thread/worker scaling as unresolved.
+6. **HVG and markers dominate** after UMAP is fixed (~22% each of the 4CPU control total).
+
+## Scale: 100k → 250k (same speed pack)
+
+Config family: 8 CPU / 32 GiB, workers=8, workingCopies=4, `annParallel=true`, `umapParallel=true`, auto markers, cloud 128 MiB chunks.
+
+| Stage | 100k (`c8_m32`) | 250k (`c8_m32_250k`) | Time scale | Peak 100k | Peak 250k |
+|-------|----------------:|---------------------:|-----------:|----------:|----------:|
+| createStore | 104s | 264s | 2.53× | 3.5G | 3.8G |
+| initializeStore | 141s | 320s | 2.27× | 6.4G | 6.6G |
+| reopenStore | 7s | 8s | 1.07× | 0.5G | 0.5G |
+| filterCells | 20s | 21s | 1.06× | 0.5G | 0.5G |
+| markHvgs | 241s | 585s | 2.42× | 1.1G | 1.3G |
+| makeGraph | 168s | 307s | 1.83× | 6.3G | **14.6G** |
+| runUmap | 58s | 101s | 1.74× | 0.7G | 0.9G |
+| runLeiden | 25s | 51s | 2.04× | 0.7G | 1.1G |
+| findMarkers | 331s | 688s | 2.08× | 7.1G | 8.8G |
+| **total** | **1095s** | **2346s** | **2.14×** | **7.1G** | **14.6G** |
+
+Cells ×2.5 → wall ×2.14 (slightly better than linear).
+**makeGraph peak roughly doubled** (6.3 → 14.6 GiB); still under 32 GiB. Markers remain the slowest stage at both sizes; HVG second.
+
+Configs:
+
+- `profiling/layouts/100k_auto_markers_c8_m32.toml`
+- `profiling/layouts/250k_auto_markers_c8_m32.toml`
+
+## What actually moves speed
+
+| Lever | Helps? | Evidence |
+|-------|--------|----------|
+| Cloud ~128 MiB count chunks | Yes | Layout sweep totals |
+| Auto marker batch (larger Scarf budget) | Yes for markers | 353s → 269s @ 4c/32G |
+| `umapParallel` / more CPU for UMAP | Yes | 146s → 58s |
+| More Modal RAM with same Scarf budget | No | Peaks unchanged |
+| Cutting Modal RAM for its own sake | No for speed | 16G markers slower |
+| Raising `workers` for markers | Not shown; can hurt | 4→8 workers, markers +62s |
+| Tuning `workingCopies` for speed | Do not | It is a copy-count model, not a perf knob |
+
+## Ops notes (Modal)
+
+- Prefer bare `Function.from_name(...).spawn` on the deployed app for `run_all_jobs`.
+- Prefer broad Modal region `eu` over narrow `eu-west` / `eu-west-1` for capacity ([region selection](https://modal.com/docs/guide/region-selection); broad multiplier 1.5x vs narrow 1.75x).
+- Tight region + high memory often sat in queue; capacity messages mentioned 16.8 / 28.8 / 48.8 GiB under `eu-west-1`.
+- Never `modal deploy` from the agent; user deploys after code changes.
+- Use `uv` for local Python / Modal CLI.
+
+## Current best reference profiles
+
+| Size | Recommended reference tag | Machine | Notes |
+|------|---------------------------|---------|-------|
+| 100k | `auto_markers_c8_m32` | 8 CPU / 32 GiB | Fastest complete funnel so far (1095s) |
+| 250k | `auto_markers_c8_m32_250k` | 8 CPU / 32 GiB | Same settings; 2346s, max peak 14.6 GiB |
+
+For a pure markers comparison at 100k without parallel UMAP noise, use `auto_markers_c4_m32` (269s markers).
+
+## Budget mismatch @ 100k: Modal 32 GiB, Scarf 16 GiB (done)
+
+Tag `auto_markers_c4_m32_scarf16`. Matched to `c4_m32` (4 CPU, workers=4, workingCopies=4, UMAP/ANN off). Only Scarf budget cut to 16 GiB; Modal stayed 32 GiB. Store built under that budget.
+
+| Stage | c4_m32 (~24G Scarf) | scarf16 (16G Scarf) | Δ |
+|-------|--------------------:|--------------------:|--:|
+| createStore | 107s / 3.5G | 115s / 3.3G | +9s |
+| initializeStore | 196s / 6.5G | 115s / 6.6G | −81s |
+| markHvgs | 255s / 1.1G | 226s / 1.2G | −30s |
+| makeGraph | 183s / 6.3G | 165s / 6.1G | −18s |
+| runUmap | 146s / 0.7G | 164s / 0.7G | +19s |
+| findMarkers | **269s / 4.5G** | **219s / 6.1G** | **−51s** |
+| **total** | **1215s** | **1052s** | **−163s** |
+
+**Reading:** Lower Scarf budget did **not** slow markers here. Markers were faster with a higher peak (6.1 vs 4.5 GiB). Contrast `c4_m16` (Modal 16 + Scarf ~12): markers 316s / 5.0G. So cutting Modal+budget together hurt markers; cutting only Scarf budget did not. Do not treat this single A/B as proof that budget is unbound; geometry/noise may dominate. Call `fc-01KXKMXS06BTJTBE7JVR5N8BWZ`.
+
+## Open questions
+
+1. Why scarf16 markers faster than c4_m32 despite smaller software budget?
+2. Can HVG use memory/CPU more productively (still ~1–1.4 GiB peak while taking 20%+ of wall)?
+3. Why did 8 workers slow markers vs 4 at 100k?
+4. makeGraph cgroup peak 250k→500k drop: re-measure with clean A/B before trusting.
+
+## Scale: 500k (8 CPU / 48 GiB, region `eu`)
+
+Tag `auto_markers_c8_m48_500k`. Same speed-pack knobs as 100k/250k (workers=8, workingCopies=4, UMAP/ANN parallel, auto markers). Modal 48 GiB / Scarf 36 GiB. Respawned late stages onto broad region `eu` after `eu-west-1` capacity stalls.
+
+| Stage | 100k | 250k | 500k | 250→500 time |
+|-------|-----:|-----:|-----:|-------------:|
+| createStore | 104s / 3.5G | 264s / 3.8G | 474s / 4.5G | 1.80× |
+| initializeStore | 141s / 6.4G | 320s / 6.6G | 592s / 7.0G | 1.85× |
+| markHvgs | 241s / 1.1G | 585s / 1.3G | 1095s / 1.1G | 1.87× |
+| makeGraph | 168s / 6.3G | 307s / 14.6G | 436s / 9.3G | 1.42× |
+| runUmap | 58s / 0.7G | 101s / 0.9G | 143s / 1.2G | 1.42× |
+| runLeiden | 25s / 0.7G | 51s / 1.1G | 100s / 1.4G | 1.96× |
+| findMarkers | 331s / 7.1G | 688s / 8.8G | 1473s / 10.1G | 2.14× |
+| **total** | **1095s** | **2346s** | **4339s** | **1.85×** |
+
+Cells 250k→500k is 2×; wall time 1.85× (slightly better than linear). Max peak at 500k ~10.1 GiB (markers), so 48 GiB was generous. makeGraph cgroup peak 9.3 GiB is *below* 250k's 14.6 GiB; treat that drop cautiously (cgroup vs RSS gap on 250k, restart mid-run). See discussion in chat / prefer RSS for graph sizing.
+
+Call ids: original `fc-01KXKE1ZCG0FG7KFEQ1H0QX35K` (cancelled); eu resume `fc-01KXKH7Z9V88NXFG2DJ204C8C8`.
+
+## In progress: 1M (8 CPU / 64 GiB, region `eu`)
+
+| Field | Value |
+|-------|-------|
+| Tag | `auto_markers_c8_m64_1m` |
+| Config | `profiling/layouts/1m_auto_markers_c8_m64.toml` |
+| Machine | 8 CPU / 64 GiB Modal, Scarf budget 48 GiB (75%) |
+| Knobs | workers=8, workingCopies=4, UMAP/ANN parallel on, auto markers |
+| Call | see `/tmp/scarf_calib_calls.txt` key `auto_markers_c8_m64_1m` |
+| Compare to | `auto_markers_c8_m32` (100k), `_250k`, `c8_m48_500k` |
diff --git a/profiling/__init__.py b/profiling/__init__.py
new file mode 100644
index 00000000..ad95557b
--- /dev/null
+++ b/profiling/__init__.py
@@ -0,0 +1 @@
+"""Scarf Modal/R2 profiling harness."""
diff --git a/profiling/config.example.toml b/profiling/config.example.toml
new file mode 100644
index 00000000..4713e6a8
--- /dev/null
+++ b/profiling/config.example.toml
@@ -0,0 +1,173 @@
+# Copy to profiling/config.toml and replace secret, region, endpoint, bucket, and prefixes.
+
+modalEnvironmentName = "scarf_profiling"
+modalAppName = "scarf-profiling"
+modalSecretName = "replace-with-modal-secret"
+modalCloud = "aws"
+modalRegion = "eu"
+r2EndpointUrl = "https://replace-with-account-id.r2.cloudflarestorage.com"
+datasetPrefixUri = "s3://example-bucket/scarf-profiling/datasets"
+resultsUri = "s3://example-bucket/scarf-profiling"
+# Optional: isolate stores/results under stores// and results//.
+# runTag = "chunk256m"
+# [storageLayout]
+# targetChunkBytes = 268435456
+# minFeatureChunk = 500
+# maxFeatureChunk = 10000
+targetSizes = [
+ 10000,
+ 25000,
+ 50000,
+ 100000,
+ 250000,
+ 500000,
+ 1000000,
+ 2500000,
+ 5000000,
+ 10000000,
+]
+samplingSeed = 0
+
+# Full Cellxgene prepare loads CSR into RAM (indices downcast to int32).
+[prepareResources]
+modalMemoryRequestMb = 196608
+modalMemoryLimitMb = 212992
+modalCpuRequest = 8.0
+modalCpuLimit = 16.0
+timeoutSeconds = 86400
+ephemeralDiskMb = 524288
+
+[workflow]
+assayName = "RNA"
+cellKey = "I"
+filterAttrs = [
+ "RNA_nCounts",
+ "RNA_nFeatures",
+ "RNA_percentMito",
+ "RNA_percentRibo",
+]
+filterMinQuantile = 0.01
+filterMaxQuantile = 0.99
+minFeaturesPerCell = 10
+minCellsPerFeature = 20
+h5adBatchSize = 1000
+topN = 2000
+hvgMinCells = 20
+hvgKey = "hvgs"
+k = 11
+dims = 50
+nCentroids = 1000
+graphSeed = 4466
+annParallel = false
+umapEpochs = 300
+umapSeed = 4444
+umapParallel = false
+umapLabel = "UMAP"
+leidenResolution = 1.0
+leidenSeed = 4444
+leidenLabel = "leiden_cluster"
+markerFeatureKey = "I"
+# When omitted or null, Scarf chooses a budget-safe batch from on-disk feature chunks.
+# markerGeneBatchSize = 50
+graphLocalCache = "auto"
+
+# Tune memory before large runs. Values are examples.
+
+[stageResources.createStore]
+modalMemoryRequestMb = 65536
+modalMemoryLimitMb = 73728
+modalCpuRequest = 8.0
+modalCpuLimit = 8.0
+scarfMemoryBudget = 51539607552
+workers = 8
+workingCopies = 4
+timeoutSeconds = 82800
+ephemeralDiskMb = 524288
+
+[stageResources.initializeStore]
+modalMemoryRequestMb = 65536
+modalMemoryLimitMb = 73728
+modalCpuRequest = 8.0
+modalCpuLimit = 8.0
+scarfMemoryBudget = 51539607552
+workers = 8
+workingCopies = 4
+timeoutSeconds = 82800
+ephemeralDiskMb = 524288
+
+[stageResources.reopenStore]
+modalMemoryRequestMb = 16384
+modalMemoryLimitMb = 20480
+modalCpuRequest = 4.0
+modalCpuLimit = 4.0
+scarfMemoryBudget = 12884901888
+workers = 4
+workingCopies = 2
+timeoutSeconds = 3600
+ephemeralDiskMb = 524288
+
+[stageResources.filterCells]
+modalMemoryRequestMb = 32768
+modalMemoryLimitMb = 40960
+modalCpuRequest = 8.0
+modalCpuLimit = 8.0
+scarfMemoryBudget = 25769803776
+workers = 8
+workingCopies = 4
+timeoutSeconds = 14400
+ephemeralDiskMb = 524288
+
+[stageResources.markHvgs]
+modalMemoryRequestMb = 65536
+modalMemoryLimitMb = 73728
+modalCpuRequest = 8.0
+modalCpuLimit = 8.0
+scarfMemoryBudget = 51539607552
+workers = 8
+workingCopies = 4
+timeoutSeconds = 43200
+ephemeralDiskMb = 524288
+
+[stageResources.makeGraph]
+modalMemoryRequestMb = 131072
+modalMemoryLimitMb = 147456
+modalCpuRequest = 16.0
+modalCpuLimit = 16.0
+scarfMemoryBudget = 120259084288
+workers = 16
+workingCopies = 4
+timeoutSeconds = 82800
+ephemeralDiskMb = 524288
+
+[stageResources.runUmap]
+modalMemoryRequestMb = 65536
+modalMemoryLimitMb = 73728
+modalCpuRequest = 8.0
+modalCpuLimit = 8.0
+scarfMemoryBudget = 51539607552
+workers = 8
+workingCopies = 4
+timeoutSeconds = 82800
+ephemeralDiskMb = 524288
+
+[stageResources.runLeiden]
+modalMemoryRequestMb = 65536
+modalMemoryLimitMb = 73728
+modalCpuRequest = 8.0
+modalCpuLimit = 8.0
+scarfMemoryBudget = 51539607552
+workers = 8
+workingCopies = 4
+timeoutSeconds = 82800
+ephemeralDiskMb = 524288
+
+[stageResources.findMarkers]
+modalMemoryRequestMb = 131072
+modalMemoryLimitMb = 147456
+modalCpuRequest = 16.0
+modalCpuLimit = 16.0
+scarfMemoryBudget = 120259084288
+workers = 16
+workingCopies = 4
+timeoutSeconds = 82800
+ephemeralDiskMb = 524288
diff --git a/profiling/config.py b/profiling/config.py
new file mode 100644
index 00000000..66b34e22
--- /dev/null
+++ b/profiling/config.py
@@ -0,0 +1,241 @@
+import tomllib
+from pathlib import Path
+from typing import Any, Literal, Self
+
+from pydantic import BaseModel, ConfigDict, Field, model_validator
+
+from profiling.datasets import DEFAULT_TARGET_SIZES
+
+StageName = Literal[
+ "createStore",
+ "initializeStore",
+ "reopenStore",
+ "filterCells",
+ "markHvgs",
+ "makeGraph",
+ "runUmap",
+ "runLeiden",
+ "findMarkers",
+]
+
+STAGE_ORDER: tuple[StageName, ...] = (
+ "createStore",
+ "initializeStore",
+ "reopenStore",
+ "filterCells",
+ "markHvgs",
+ "makeGraph",
+ "runUmap",
+ "runLeiden",
+ "findMarkers",
+)
+
+MAX_TIMEOUT_SECONDS = 86_400
+
+
+class WorkflowParameters(BaseModel):
+ model_config = ConfigDict(extra="forbid", frozen=True)
+
+ assayName: str = "RNA"
+ cellKey: str = "I"
+ filterAttrs: tuple[str, ...] = (
+ "RNA_nCounts",
+ "RNA_nFeatures",
+ "RNA_percentMito",
+ "RNA_percentRibo",
+ )
+ filterMinQuantile: float = 0.01
+ filterMaxQuantile: float = 0.99
+ minFeaturesPerCell: int = 10
+ minCellsPerFeature: int = 20
+ h5adBatchSize: int = 1000
+ topN: int = 2000
+ hvgMinCells: int = 20
+ hvgKey: str = "hvgs"
+ k: int = 11
+ dims: int = 50
+ nCentroids: int = 1000
+ graphSeed: int = 4466
+ annParallel: bool = False
+ umapEpochs: int = 300
+ umapSeed: int = 4444
+ umapParallel: bool = False
+ umapLabel: str = "UMAP"
+ leidenResolution: float = 1.0
+ leidenSeed: int = 4444
+ leidenLabel: str = "leiden_cluster"
+ markerFeatureKey: str = "I"
+ markerGeneBatchSize: int | None = None
+ graphLocalCache: bool | str = "auto"
+
+ @property
+ def resolvedHvgKey(self) -> str:
+ return f"{self.cellKey}__{self.hvgKey}"
+
+ @property
+ def resolvedMarkerGroupKey(self) -> str:
+ if self.cellKey == "I":
+ return f"{self.assayName}_{self.leidenLabel}"
+ return f"{self.assayName}_{self.cellKey}_{self.leidenLabel}"
+
+
+class StageResources(BaseModel):
+ model_config = ConfigDict(extra="forbid", frozen=True)
+
+ modalMemoryRequestMb: int
+ modalMemoryLimitMb: int
+ modalCpuRequest: float
+ modalCpuLimit: float
+ scarfMemoryBudget: int
+ workers: int
+ workingCopies: int
+ timeoutSeconds: int
+ ephemeralDiskMb: int
+
+ @model_validator(mode="after")
+ def _check_bounds(self) -> Self:
+ if self.modalMemoryRequestMb <= 0 or self.modalMemoryLimitMb <= 0:
+ raise ValueError("Modal memory values must be positive")
+ if self.modalMemoryLimitMb < self.modalMemoryRequestMb:
+ raise ValueError("modalMemoryLimitMb must be >= modalMemoryRequestMb")
+ if self.modalCpuRequest <= 0 or self.modalCpuLimit <= 0:
+ raise ValueError("Modal CPU values must be positive")
+ if self.modalCpuLimit < self.modalCpuRequest:
+ raise ValueError("modalCpuLimit must be >= modalCpuRequest")
+ if self.scarfMemoryBudget <= 0:
+ raise ValueError("scarfMemoryBudget must be positive")
+ if self.workers <= 0 or self.workingCopies <= 0:
+ raise ValueError("workers and workingCopies must be positive")
+ if not (0 < self.timeoutSeconds <= MAX_TIMEOUT_SECONDS):
+ raise ValueError(f"timeoutSeconds must be in 1..{MAX_TIMEOUT_SECONDS}")
+ if self.ephemeralDiskMb <= 0:
+ raise ValueError("ephemeralDiskMb must be positive")
+ return self
+
+
+class PrepareResources(BaseModel):
+ """Modal sizing for dataset prepare (full CSR resident in memory)."""
+
+ model_config = ConfigDict(extra="forbid", frozen=True)
+
+ modalMemoryRequestMb: int = 196_608
+ modalMemoryLimitMb: int = 212_992
+ modalCpuRequest: float = 8.0
+ modalCpuLimit: float = 16.0
+ timeoutSeconds: int = 86_400
+ ephemeralDiskMb: int = 524_288
+
+ @model_validator(mode="after")
+ def _check_bounds(self) -> Self:
+ if self.modalMemoryRequestMb <= 0 or self.modalMemoryLimitMb <= 0:
+ raise ValueError("Modal memory values must be positive")
+ if self.modalMemoryLimitMb < self.modalMemoryRequestMb:
+ raise ValueError("modalMemoryLimitMb must be >= modalMemoryRequestMb")
+ if self.modalCpuRequest <= 0 or self.modalCpuLimit <= 0:
+ raise ValueError("Modal CPU values must be positive")
+ if self.modalCpuLimit < self.modalCpuRequest:
+ raise ValueError("modalCpuLimit must be >= modalCpuRequest")
+ if not (0 < self.timeoutSeconds <= MAX_TIMEOUT_SECONDS):
+ raise ValueError(f"timeoutSeconds must be in 1..{MAX_TIMEOUT_SECONDS}")
+ if self.ephemeralDiskMb <= 0:
+ raise ValueError("ephemeralDiskMb must be positive")
+ return self
+
+
+class StorageLayout(BaseModel):
+ model_config = ConfigDict(extra="forbid", frozen=True)
+
+ targetChunkBytes: int | None = None
+ minFeatureChunk: int = 500
+ maxFeatureChunk: int = 10_000
+
+ @model_validator(mode="after")
+ def _check_bounds(self) -> Self:
+ if self.targetChunkBytes is not None and self.targetChunkBytes <= 0:
+ raise ValueError("targetChunkBytes must be positive when set")
+ if self.minFeatureChunk <= 0 or self.maxFeatureChunk <= 0:
+ raise ValueError("feature chunk bounds must be positive")
+ if self.minFeatureChunk > self.maxFeatureChunk:
+ raise ValueError("minFeatureChunk must be <= maxFeatureChunk")
+ return self
+
+
+class ProfilingConfig(BaseModel):
+ model_config = ConfigDict(extra="forbid", frozen=True)
+
+ modalEnvironmentName: str = "scarf_profiling"
+ modalAppName: str = "scarf-profiling"
+ modalSecretName: str
+ modalCloud: str = "aws"
+ modalRegion: str
+ r2EndpointUrl: str
+ datasetPrefixUri: str
+ resultsUri: str
+ runTag: str = ""
+ storageLayout: StorageLayout = Field(default_factory=StorageLayout)
+ targetSizes: tuple[int, ...] = Field(default_factory=lambda: DEFAULT_TARGET_SIZES)
+ samplingSeed: int = 0
+ workflow: WorkflowParameters = Field(default_factory=WorkflowParameters)
+ prepareResources: PrepareResources = Field(default_factory=PrepareResources)
+ stageResources: dict[StageName, StageResources]
+
+ @model_validator(mode="after")
+ def _check_config(self) -> Self:
+ if self.modalEnvironmentName != "scarf_profiling":
+ raise ValueError("modalEnvironmentName must be scarf_profiling")
+ if not self.datasetPrefixUri.startswith("s3://"):
+ raise ValueError("datasetPrefixUri must be an s3:// URI")
+ if not self.resultsUri.startswith("s3://"):
+ raise ValueError("resultsUri must be an s3:// URI")
+ if "/" in self.runTag or "\\" in self.runTag or self.runTag in {".", ".."}:
+ raise ValueError("runTag must be a single path segment")
+ if not self.targetSizes:
+ raise ValueError("targetSizes must not be empty")
+ if any(size <= 0 for size in self.targetSizes):
+ raise ValueError("targetSizes must be positive")
+ if tuple(sorted(self.targetSizes)) != self.targetSizes:
+ raise ValueError("targetSizes must be strictly increasing")
+ if len(set(self.targetSizes)) != len(self.targetSizes):
+ raise ValueError("targetSizes must be unique")
+ missing = [stage for stage in STAGE_ORDER if stage not in self.stageResources]
+ if missing:
+ raise ValueError(f"Missing stageResources for: {', '.join(missing)}")
+ return self
+
+ def datasetUri(self, nRows: int) -> str:
+ return f"{self.datasetPrefixUri.rstrip('/')}/{nRows}.h5ad"
+
+ def sourceUri(self) -> str:
+ return f"{self.datasetPrefixUri.rstrip('/')}/source.h5ad"
+
+ def _tagged_prefix(self, kind: str) -> str:
+ base = f"{self.resultsUri.rstrip('/')}/{kind}"
+ if self.runTag:
+ return f"{base}/{self.runTag}"
+ return base
+
+ def storeUri(self, nRows: int) -> str:
+ return f"{self._tagged_prefix('stores')}/{nRows}.zarr"
+
+ def resultUri(self, nRows: int, stage: StageName) -> str:
+ return f"{self._tagged_prefix('results')}/{nRows}/{stage}.json"
+
+ def resourcesFor(self, stage: StageName) -> StageResources:
+ return self.stageResources[stage]
+
+
+def load_profiling_config(path: str | Path) -> ProfilingConfig:
+ config_path = Path(path)
+ raw = tomllib.loads(config_path.read_text())
+ return ProfilingConfig.model_validate(_normalize_raw_config(raw))
+
+
+def _normalize_raw_config(raw: dict[str, Any]) -> dict[str, Any]:
+ payload = dict(raw)
+ fixed = payload.pop("fixedResources", None)
+ if fixed is not None and "stageResources" not in payload:
+ payload["stageResources"] = fixed
+ payload.pop("sourceProvenance", None)
+ payload.pop("capacityCases", None)
+ payload.pop("blockSeeds", None)
+ return payload
diff --git a/profiling/datasets.py b/profiling/datasets.py
new file mode 100644
index 00000000..9ef45d5a
--- /dev/null
+++ b/profiling/datasets.py
@@ -0,0 +1,1660 @@
+import hashlib
+import os
+import uuid
+from collections.abc import Callable, Sequence
+from contextlib import closing
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+from urllib.request import urlopen
+
+import h5py
+import numpy as np
+
+DEFAULT_SAMPLING_SEED = 0
+DEFAULT_TARGET_SIZES = (
+ 10_000,
+ 25_000,
+ 50_000,
+ 100_000,
+ 250_000,
+ 500_000,
+ 1_000_000,
+ 2_500_000,
+ 5_000_000,
+ 10_000_000,
+)
+
+_MASK_64 = (1 << 64) - 1
+_SPLITMIX_INCREMENT = 0x9E3779B97F4A7C15
+_SPLITMIX_MULTIPLIER_1 = 0xBF58476D1CE4E5B9
+_SPLITMIX_MULTIPLIER_2 = 0x94D049BB133111EB
+_SAMPLING_DOMAIN = b"scarf-cellxgene-row-sampling-v1"
+_SOURCE_ROWS_DIGEST_DOMAIN = b"scarf-ordered-source-rows-v1\0"
+_DEFAULT_IO_CHUNK_BYTES = 16 * 1024 * 1024
+_DEFAULT_ROW_BATCH_SIZE = 1024
+_DEFAULT_COPY_BUFFER_BYTES = 64 * 1024 * 1024
+_DEFAULT_INDPTR_CHUNK_ROWS = 1_000_000
+_DEFAULT_LOAD_CHUNK_ELEMENTS = 8_388_608
+_H5_DATA_CHUNK_BYTES = 4 * 1024 * 1024
+
+
+def sha256_file(path: str | Path, *, chunkBytes: int = 16 * 1024 * 1024) -> str:
+ if chunkBytes <= 0:
+ raise ValueError("chunkBytes must be positive")
+ digest = hashlib.sha256()
+ with Path(path).open("rb") as handle:
+ while True:
+ chunk = handle.read(chunkBytes)
+ if not chunk:
+ break
+ digest.update(chunk)
+ return digest.hexdigest()
+
+
+@dataclass(frozen=True, slots=True)
+class SourceSpec:
+ datasetId: str
+ versionId: str
+ url: str
+ nRows: int
+ nColumns: int
+ nnz: int
+ sourceBytes: int
+ matrixKey: str = "X"
+ matrixEncoding: str = "csr_matrix"
+ rawCounts: bool = True
+ cellIdsKey: str = "obs/_index"
+ featureIdsKey: str = "var/_index"
+ featureNameKey: str = "var/feature_name"
+ dataDtype: str = "float32"
+ indicesDtype: str = "int64"
+ indptrDtype: str = "int64"
+
+
+@dataclass(frozen=True, slots=True)
+class DownloadResult:
+ filePath: Path
+ fileBytes: int
+ sha256: str
+
+
+@dataclass(frozen=True, slots=True)
+class SourceValidation:
+ nRows: int
+ nColumns: int
+ nnz: int
+ finalRowNnz: int
+ dataDtype: str
+ indicesDtype: str
+ indptrDtype: str
+
+
+@dataclass(frozen=True, slots=True)
+class H5adWriteResult:
+ filePath: Path
+ nRows: int
+ nColumns: int
+ nnz: int
+ sourceRowsSha256: str
+ finalSourceRow: int
+ dataDtype: str
+ indicesDtype: str
+ indptrDtype: str
+
+
+@dataclass(frozen=True, slots=True)
+class InMemoryCsrSource:
+ """CSR matrix plus labels held in process memory for fast row gathers."""
+
+ data: np.ndarray
+ indices: np.ndarray
+ indptr: np.ndarray
+ nRows: int
+ nColumns: int
+ dataDtype: np.dtype
+ indicesDtype: np.dtype
+ indptrDtype: np.dtype
+ cellIds: np.ndarray
+ featureIds: np.ndarray
+ featureNames: np.ndarray
+
+ @property
+ def residentBytes(self) -> int:
+ return int(
+ self.data.nbytes
+ + self.indices.nbytes
+ + self.indptr.nbytes
+ + self.cellIds.nbytes
+ + self.featureIds.nbytes
+ + self.featureNames.nbytes
+ )
+
+
+@dataclass(frozen=True, slots=True)
+class PreparedValidation:
+ fileBytes: int
+ sha256: str
+ sourceRowsSha256: str
+ nRows: int
+ nColumns: int
+ nnz: int
+ finalSourceRow: int
+
+
+@dataclass(frozen=True, slots=True)
+class PreparedArtifact:
+ localPath: Path
+ targetRows: int
+ nColumns: int
+ nnz: int
+ fileBytes: int
+ sha256: str
+ sourceRowsSha256: str
+ finalSourceRow: int
+ dataDtype: str
+ indicesDtype: str
+ indptrDtype: str
+
+
+@dataclass(frozen=True, slots=True)
+class PreparationResult:
+ sourcePath: Path
+ sourceSha256: str
+ sourceSpec: SourceSpec
+ seed: int
+ artifacts: tuple[PreparedArtifact, ...]
+
+
+SOURCE_SPEC = SourceSpec(
+ datasetId="dcfd4feb-18a3-4b30-81d7-1b0c544a8ab3",
+ versionId="1bc30289-9565-4099-abf9-3326328c11ac",
+ url=(
+ "https://datasets.cellxgene.cziscience.com/"
+ "1bc30289-9565-4099-abf9-3326328c11ac.h5ad"
+ ),
+ nRows=11_441_407,
+ nColumns=45_525,
+ nnz=19_516_755_155,
+ sourceBytes=46_292_192_475,
+)
+
+
+def splitmix64(value: int) -> int:
+ """Return SplitMix64 for one unsigned 64-bit input."""
+ mixed = (value + _SPLITMIX_INCREMENT) & _MASK_64
+ mixed = ((mixed ^ (mixed >> 30)) * _SPLITMIX_MULTIPLIER_1) & _MASK_64
+ mixed = ((mixed ^ (mixed >> 27)) * _SPLITMIX_MULTIPLIER_2) & _MASK_64
+ return mixed ^ (mixed >> 31)
+
+
+def sampling_salt(
+ seed: int,
+ sourceVersion: str = SOURCE_SPEC.versionId,
+) -> int:
+ material = (
+ _SAMPLING_DOMAIN + b"\0" + sourceVersion.encode() + b"\0" + str(seed).encode()
+ )
+ return int.from_bytes(hashlib.sha256(material).digest()[:8], "little")
+
+
+def row_priorities(
+ nRows: int,
+ *,
+ seed: int = DEFAULT_SAMPLING_SEED,
+ sourceVersion: str = SOURCE_SPEC.versionId,
+) -> np.ndarray:
+ if nRows < 0:
+ raise ValueError("nRows must be nonnegative")
+
+ # For source row i:
+ # priority(i) = SplitMix64((i + salt(sourceVersion, seed)) mod 2**64).
+ # SplitMix64 is a fixed integer permutation, so this is reproducible across
+ # Python and NumPy versions. Selection ranks (priority, source row index),
+ # which makes the row index the explicit collision tie-break.
+ values = np.arange(nRows, dtype=np.uint64)
+ values += np.uint64(sampling_salt(seed, sourceVersion))
+ values += np.uint64(_SPLITMIX_INCREMENT)
+ values = (values ^ (values >> np.uint64(30))) * np.uint64(_SPLITMIX_MULTIPLIER_1)
+ values = (values ^ (values >> np.uint64(27))) * np.uint64(_SPLITMIX_MULTIPLIER_2)
+ return values ^ (values >> np.uint64(31))
+
+
+def _normalize_targets(targetRows: Sequence[int], nRows: int) -> tuple[int, ...]:
+ targets = tuple(int(value) for value in targetRows)
+ if not targets:
+ raise ValueError("At least one target row count is required")
+ if any(value <= 0 for value in targets):
+ raise ValueError("Target row counts must be positive")
+ if tuple(sorted(set(targets))) != targets:
+ raise ValueError("Target row counts must be unique and increasing")
+ if targets[-1] > nRows:
+ raise ValueError(f"Largest target has {targets[-1]} rows, source has {nRows}")
+ return targets
+
+
+def select_rows_from_priorities(
+ priorities: np.ndarray,
+ targetRows: Sequence[int],
+) -> dict[int, np.ndarray]:
+ priorities = np.asarray(priorities)
+ if priorities.ndim != 1:
+ raise ValueError("priorities must be one-dimensional")
+ if not np.issubdtype(priorities.dtype, np.integer):
+ raise TypeError("priorities must use an integer dtype")
+ targets = _normalize_targets(targetRows, len(priorities))
+ source_rows = np.arange(len(priorities), dtype=np.int64)
+ ranked = np.lexsort((source_rows, priorities.astype(np.uint64, copy=False)))
+ return {
+ target: np.sort(ranked[:target].astype(np.int64, copy=False), kind="stable")
+ for target in targets
+ }
+
+
+def select_nested_rows(
+ nRows: int,
+ targetRows: Sequence[int] = DEFAULT_TARGET_SIZES,
+ *,
+ seed: int = DEFAULT_SAMPLING_SEED,
+ sourceVersion: str = SOURCE_SPEC.versionId,
+) -> dict[int, np.ndarray]:
+ priorities = row_priorities(
+ nRows,
+ seed=seed,
+ sourceVersion=sourceVersion,
+ )
+ return select_rows_from_priorities(priorities, targetRows)
+
+
+def ordered_source_row_digest(
+ sourceRows: np.ndarray | Sequence[int],
+ *,
+ chunkRows: int = 1_000_000,
+) -> str:
+ rows = np.asarray(sourceRows)
+ if rows.ndim != 1:
+ raise ValueError("sourceRows must be one-dimensional")
+ if not np.issubdtype(rows.dtype, np.integer):
+ raise TypeError("sourceRows must use an integer dtype")
+ if chunkRows <= 0:
+ raise ValueError("chunkRows must be positive")
+ if len(rows) and int(rows.min()) < 0:
+ raise ValueError("sourceRows cannot contain negative values")
+ if len(rows) and int(rows.max()) > np.iinfo(np.uint64).max:
+ raise ValueError("sourceRows exceed uint64")
+
+ digest = hashlib.sha256()
+ digest.update(_SOURCE_ROWS_DIGEST_DOMAIN)
+ digest.update(len(rows).to_bytes(8, "little"))
+ for start in range(0, len(rows), chunkRows):
+ normalized = np.asarray(rows[start : start + chunkRows], dtype=" DownloadResult:
+ if expectedBytes < 0:
+ raise ValueError("expectedBytes must be nonnegative")
+ if chunkBytes <= 0:
+ raise ValueError("chunkBytes must be positive")
+
+ destination = Path(destination)
+ destination.parent.mkdir(parents=True, exist_ok=True)
+ temporary = destination.with_name(f".{destination.name}.{uuid.uuid4().hex}.part")
+ digest = hashlib.sha256()
+ file_bytes = 0
+ try:
+ with closing(opener(url)) as response:
+ headers = getattr(response, "headers", {})
+ declared = headers.get("Content-Length") or headers.get("content-length")
+ if declared is not None and int(declared) != expectedBytes:
+ raise ValueError(
+ f"Source declares {declared} bytes, expected {expectedBytes}"
+ )
+ with temporary.open("xb") as handle:
+ while chunk := response.read(chunkBytes):
+ handle.write(chunk)
+ digest.update(chunk)
+ file_bytes += len(chunk)
+ if file_bytes != expectedBytes:
+ raise ValueError(f"Downloaded {file_bytes} bytes, expected {expectedBytes}")
+ os.replace(temporary, destination)
+ except BaseException:
+ temporary.unlink(missing_ok=True)
+ raise
+
+ return DownloadResult(
+ filePath=destination,
+ fileBytes=file_bytes,
+ sha256=digest.hexdigest(),
+ )
+
+
+def _decode_text(value: Any) -> str:
+ if isinstance(value, bytes):
+ return value.decode()
+ return str(value)
+
+
+def _require_encoding(
+ value: h5py.File | h5py.Group | h5py.Dataset,
+ path: str,
+ encodingType: str,
+ encodingVersion: str,
+) -> None:
+ actual_type = value.attrs.get("encoding-type")
+ actual_version = value.attrs.get("encoding-version")
+ if actual_type is None or _decode_text(actual_type) != encodingType:
+ raise ValueError(
+ f"{path} encoding-type is {actual_type!r}, expected {encodingType!r}"
+ )
+ if actual_version is None or _decode_text(actual_version) != encodingVersion:
+ raise ValueError(
+ f"{path} encoding-version is {actual_version!r}, "
+ f"expected {encodingVersion!r}"
+ )
+
+
+def _require_group(h5: h5py.File, path: str) -> h5py.Group:
+ if path not in h5 or not isinstance(h5[path], h5py.Group):
+ raise ValueError(f"Required HDF5 group is missing: {path}")
+ return h5[path]
+
+
+def _require_dataset(h5: h5py.File, path: str) -> h5py.Dataset:
+ if path not in h5 or not isinstance(h5[path], h5py.Dataset):
+ raise ValueError(f"Required HDF5 dataset is missing: {path}")
+ return h5[path]
+
+
+def _validate_string_dataset(
+ dataset: h5py.Dataset,
+ path: str,
+ expectedLength: int,
+ *,
+ requireEncoding: bool,
+) -> None:
+ if dataset.shape != (expectedLength,):
+ raise ValueError(
+ f"{path} shape is {dataset.shape}, expected {(expectedLength,)}"
+ )
+ if h5py.check_string_dtype(dataset.dtype) is None:
+ raise ValueError(f"{path} must contain strings, got {dataset.dtype}")
+ if requireEncoding:
+ _require_encoding(dataset, path, "string-array", "0.2.0")
+
+
+def _decode_string_array(values: np.ndarray) -> np.ndarray:
+ flat = np.asarray(values).reshape(-1)
+ if flat.dtype.kind in {"U", "S", "O"}:
+ decoded = [
+ item.decode() if isinstance(item, bytes | np.bytes_) else str(item)
+ for item in flat.tolist()
+ ]
+ return np.asarray(decoded, dtype=object)
+ raise ValueError(f"Unsupported string array dtype: {flat.dtype}")
+
+
+def _load_string_column(
+ h5: h5py.File,
+ path: str,
+ *,
+ expectedLength: int,
+) -> np.ndarray:
+ """Load a string AnnData column from a dataset or categorical group."""
+ if path not in h5:
+ raise ValueError(f"Required HDF5 path is missing: {path}")
+ value = h5[path]
+ if isinstance(value, h5py.Dataset):
+ if value.shape != (expectedLength,):
+ raise ValueError(
+ f"{path} shape is {value.shape}, expected {(expectedLength,)}"
+ )
+ return _decode_string_array(np.asarray(value[:]))
+ if isinstance(value, h5py.Group):
+ encoding = value.attrs.get("encoding-type")
+ if encoding is None or _decode_text(encoding) != "categorical":
+ raise ValueError(f"{path} is a group but not a categorical string column")
+ categories = _decode_string_array(
+ np.asarray(_require_dataset(h5, f"{path}/categories")[:])
+ )
+ codes = np.asarray(_require_dataset(h5, f"{path}/codes")[:])
+ if codes.shape != (expectedLength,):
+ raise ValueError(
+ f"{path}/codes shape is {codes.shape}, expected {(expectedLength,)}"
+ )
+ if codes.min(initial=0) < -1 or codes.max(initial=-1) >= len(categories):
+ raise ValueError(f"{path}/codes contains out-of-range category ids")
+ # AnnData uses -1 for missing; map those to empty strings.
+ output = np.empty(expectedLength, dtype=object)
+ missing = codes < 0
+ output[missing] = ""
+ output[~missing] = categories[codes[~missing]]
+ return output
+ raise ValueError(f"{path} must be a string dataset or categorical group")
+
+
+def _validate_indptr(
+ indptr: h5py.Dataset,
+ *,
+ nRows: int,
+ nnz: int,
+ expectedDtype: str,
+ chunkRows: int,
+) -> int:
+ if chunkRows <= 0:
+ raise ValueError("chunkRows must be positive")
+ if indptr.shape != (nRows + 1,):
+ raise ValueError(f"CSR indptr shape is {indptr.shape}, expected {(nRows + 1,)}")
+ if indptr.dtype != np.dtype(expectedDtype):
+ raise ValueError(
+ f"CSR indptr dtype is {indptr.dtype}, expected {expectedDtype}"
+ )
+ if int(indptr[0]) != 0:
+ raise ValueError("CSR indptr must start at zero")
+ if int(indptr[-1]) != nnz:
+ raise ValueError(f"CSR indptr terminates at {int(indptr[-1])}, expected {nnz}")
+
+ previous: int | None = None
+ for start in range(0, nRows + 1, chunkRows):
+ values = np.asarray(indptr[start : min(start + chunkRows, nRows + 1)])
+ if previous is not None and len(values) and int(values[0]) < previous:
+ raise ValueError(f"CSR indptr decreases at position {start}")
+ if len(values) > 1 and np.any(values[1:] < values[:-1]):
+ offset = int(np.flatnonzero(values[1:] < values[:-1])[0]) + start + 1
+ raise ValueError(f"CSR indptr decreases at position {offset}")
+ if len(values):
+ previous = int(values[-1])
+
+ if nRows == 0:
+ return 0
+ return int(indptr[-1]) - int(indptr[-2])
+
+
+def validate_source_h5ad(
+ path: str | Path,
+ *,
+ spec: SourceSpec = SOURCE_SPEC,
+ indptrChunkRows: int = _DEFAULT_INDPTR_CHUNK_ROWS,
+) -> SourceValidation:
+ path = Path(path)
+ file_bytes = path.stat().st_size
+ if file_bytes != spec.sourceBytes:
+ raise ValueError(f"Source has {file_bytes} bytes, expected {spec.sourceBytes}")
+
+ with h5py.File(path, mode="r") as h5:
+ _require_encoding(h5, "/", "anndata", "0.1.0")
+ matrix = _require_group(h5, spec.matrixKey)
+ _require_encoding(
+ matrix,
+ spec.matrixKey,
+ spec.matrixEncoding,
+ "0.1.0",
+ )
+ shape = tuple(int(value) for value in matrix.attrs.get("shape", ()))
+ expected_shape = (spec.nRows, spec.nColumns)
+ if shape != expected_shape:
+ raise ValueError(
+ f"{spec.matrixKey} shape is {shape}, expected {expected_shape}"
+ )
+
+ data = _require_dataset(h5, f"{spec.matrixKey}/data")
+ indices = _require_dataset(h5, f"{spec.matrixKey}/indices")
+ indptr = _require_dataset(h5, f"{spec.matrixKey}/indptr")
+ if data.shape != (spec.nnz,):
+ raise ValueError(f"CSR data shape is {data.shape}, expected {(spec.nnz,)}")
+ if indices.shape != (spec.nnz,):
+ raise ValueError(
+ f"CSR indices shape is {indices.shape}, expected {(spec.nnz,)}"
+ )
+ if data.dtype != np.dtype(spec.dataDtype):
+ raise ValueError(
+ f"CSR data dtype is {data.dtype}, expected {spec.dataDtype}"
+ )
+ if indices.dtype != np.dtype(spec.indicesDtype):
+ raise ValueError(
+ f"CSR indices dtype is {indices.dtype}, expected {spec.indicesDtype}"
+ )
+ final_row_nnz = _validate_indptr(
+ indptr,
+ nRows=spec.nRows,
+ nnz=spec.nnz,
+ expectedDtype=spec.indptrDtype,
+ chunkRows=indptrChunkRows,
+ )
+
+ cell_ids = _require_dataset(h5, spec.cellIdsKey)
+ feature_ids = _require_dataset(h5, spec.featureIdsKey)
+ _validate_string_dataset(
+ cell_ids,
+ spec.cellIdsKey,
+ spec.nRows,
+ requireEncoding=True,
+ )
+ _validate_string_dataset(
+ feature_ids,
+ spec.featureIdsKey,
+ spec.nColumns,
+ requireEncoding=True,
+ )
+ feature_names = _load_string_column(
+ h5,
+ spec.featureNameKey,
+ expectedLength=spec.nColumns,
+ )
+ if feature_names.shape != (spec.nColumns,):
+ raise ValueError(
+ f"{spec.featureNameKey} resolved length is {feature_names.shape}, "
+ f"expected {(spec.nColumns,)}"
+ )
+
+ return SourceValidation(
+ nRows=spec.nRows,
+ nColumns=spec.nColumns,
+ nnz=spec.nnz,
+ finalRowNnz=final_row_nnz,
+ dataDtype=spec.dataDtype,
+ indicesDtype=spec.indicesDtype,
+ indptrDtype=spec.indptrDtype,
+ )
+
+
+def _validate_selected_rows(
+ sourceRows: np.ndarray | Sequence[int],
+ nRows: int,
+) -> np.ndarray:
+ rows = np.asarray(sourceRows)
+ if rows.ndim != 1:
+ raise ValueError("sourceRows must be one-dimensional")
+ if not np.issubdtype(rows.dtype, np.integer):
+ raise TypeError("sourceRows must use an integer dtype")
+ if len(rows) == 0:
+ raise ValueError("sourceRows cannot be empty")
+ if int(rows.min()) < 0 or int(rows.max()) >= nRows:
+ raise ValueError(f"sourceRows must be between 0 and {nRows - 1}")
+ rows = rows.astype(np.int64, copy=False)
+ if np.any(rows[1:] <= rows[:-1]):
+ raise ValueError("sourceRows must be unique and in source order")
+ return rows
+
+
+def _row_boundaries(
+ indptr: h5py.Dataset,
+ rows: np.ndarray,
+) -> tuple[np.ndarray, np.ndarray]:
+ span_rows = int(rows[-1] - rows[0] + 1)
+ if span_rows <= len(rows) * 4:
+ pointers = np.asarray(indptr[int(rows[0]) : int(rows[-1]) + 2])
+ local_rows = rows - rows[0]
+ return pointers[local_rows], pointers[local_rows + 1]
+ return np.asarray(indptr[rows]), np.asarray(indptr[rows + 1])
+
+
+def _selected_nnz(
+ indptr: h5py.Dataset,
+ rows: np.ndarray,
+ rowBatchSize: int,
+) -> int:
+ total = 0
+ for start in range(0, len(rows), rowBatchSize):
+ batch = rows[start : start + rowBatchSize]
+ row_starts, row_ends = _row_boundaries(indptr, batch)
+ counts = row_ends - row_starts
+ if np.any(counts < 0):
+ raise ValueError("Source CSR indptr is not monotonic")
+ total += int(counts.sum(dtype=np.int64))
+ return total
+
+
+def _chunk_elements(length: int, dtype: np.dtype[Any]) -> int:
+ desired = max(1, _H5_DATA_CHUNK_BYTES // max(1, dtype.itemsize))
+ return max(1, min(max(1, length), desired))
+
+
+def _create_numeric_dataset(
+ group: h5py.Group,
+ name: str,
+ length: int,
+ dtype: np.dtype[Any],
+) -> h5py.Dataset:
+ return group.create_dataset(
+ name,
+ shape=(length,),
+ maxshape=(None,),
+ dtype=dtype,
+ chunks=(_chunk_elements(length, dtype),),
+ compression="gzip",
+ compression_opts=4,
+ )
+
+
+def _create_string_dataset(
+ group: h5py.Group,
+ name: str,
+ length: int,
+) -> h5py.Dataset:
+ dataset = group.create_dataset(
+ name,
+ shape=(length,),
+ maxshape=(None,),
+ dtype=h5py.string_dtype(encoding="utf-8"),
+ chunks=(max(1, min(max(1, length), 65_536)),),
+ compression="gzip",
+ compression_opts=4,
+ )
+ dataset.attrs["encoding-type"] = "string-array"
+ dataset.attrs["encoding-version"] = "0.2.0"
+ return dataset
+
+
+def _create_dataframe_group(
+ h5: h5py.File,
+ name: str,
+ columns: Sequence[str],
+) -> h5py.Group:
+ group = h5.create_group(name)
+ group.attrs["encoding-type"] = "dataframe"
+ group.attrs["encoding-version"] = "0.2.0"
+ group.attrs["_index"] = "_index"
+ group.attrs["column-order"] = np.asarray(
+ columns,
+ dtype=h5py.string_dtype(encoding="utf-8"),
+ )
+ return group
+
+
+def _copy_pair_slice(
+ sourceData: h5py.Dataset,
+ sourceIndices: h5py.Dataset,
+ destinationData: h5py.Dataset,
+ destinationIndices: h5py.Dataset,
+ sourceStart: int,
+ sourceEnd: int,
+ destinationStart: int,
+ copyBufferBytes: int,
+) -> int:
+ item_bytes = sourceData.dtype.itemsize + sourceIndices.dtype.itemsize
+ chunk_elements = max(1, copyBufferBytes // max(1, item_bytes))
+ output_position = destinationStart
+ for start in range(sourceStart, sourceEnd, chunk_elements):
+ end = min(start + chunk_elements, sourceEnd)
+ data_values = np.asarray(sourceData[start:end])
+ index_values = np.asarray(sourceIndices[start:end])
+ output_end = output_position + len(data_values)
+ destinationData[output_position:output_end] = data_values
+ destinationIndices[output_position:output_end] = index_values
+ output_position = output_end
+ return output_position
+
+
+def _copy_masked_csr_window(
+ sourceData: h5py.Dataset,
+ sourceIndices: h5py.Dataset,
+ destinationData: h5py.Dataset,
+ destinationIndices: h5py.Dataset,
+ rowStarts: np.ndarray,
+ rowEnds: np.ndarray,
+ destinationStart: int,
+) -> int:
+ span_start = int(rowStarts[0])
+ span_end = int(rowEnds[-1])
+ span_nnz = span_end - span_start
+ selected_nnz = int((rowEnds - rowStarts).sum(dtype=np.int64))
+ if selected_nnz == 0:
+ return destinationStart
+
+ data_values = np.asarray(sourceData[span_start:span_end])
+ index_values = np.asarray(sourceIndices[span_start:span_end])
+ local_starts = rowStarts.astype(np.int64, copy=False) - span_start
+ local_ends = rowEnds.astype(np.int64, copy=False) - span_start
+ deltas = np.zeros(span_nnz + 1, dtype=np.int16)
+ np.add.at(deltas, local_starts, 1)
+ np.add.at(deltas, local_ends, -1)
+ mask = np.cumsum(deltas[:-1], dtype=np.int32) > 0
+ if int(mask.sum()) != selected_nnz:
+ raise ValueError("Selected CSR row mask has an unexpected size")
+
+ destination_end = destinationStart + selected_nnz
+ destinationData[destinationStart:destination_end] = data_values[mask]
+ destinationIndices[destinationStart:destination_end] = index_values[mask]
+ return destination_end
+
+
+def _copy_csr_batch(
+ sourceData: h5py.Dataset,
+ sourceIndices: h5py.Dataset,
+ destinationData: h5py.Dataset,
+ destinationIndices: h5py.Dataset,
+ rows: np.ndarray,
+ rowStarts: np.ndarray,
+ rowEnds: np.ndarray,
+ destinationStart: int,
+ copyBufferBytes: int,
+) -> int:
+ counts = rowEnds - rowStarts
+ selected_nnz = int(counts.sum(dtype=np.int64))
+ if selected_nnz == 0:
+ return destinationStart
+
+ span_start = int(rowStarts[0])
+ span_end = int(rowEnds[-1])
+ span_nnz = span_end - span_start
+ item_bytes = sourceData.dtype.itemsize + sourceIndices.dtype.itemsize
+ estimated_bytes = span_nnz * (2 * item_bytes + 7)
+ if span_nnz <= selected_nnz * 2 and estimated_bytes <= copyBufferBytes:
+ return _copy_masked_csr_window(
+ sourceData,
+ sourceIndices,
+ destinationData,
+ destinationIndices,
+ rowStarts,
+ rowEnds,
+ destinationStart,
+ )
+
+ output_position = destinationStart
+ max_span_nnz = max(1, copyBufferBytes // max(1, 2 * item_bytes + 7))
+ window_start = 0
+ while window_start < len(rows):
+ window_end = window_start + 1
+ while (
+ window_end < len(rows)
+ and int(rowEnds[window_end] - rowStarts[window_start]) <= max_span_nnz
+ ):
+ window_end += 1
+
+ window_starts = rowStarts[window_start:window_end]
+ window_ends = rowEnds[window_start:window_end]
+ if int(window_ends[-1] - window_starts[0]) > max_span_nnz:
+ output_position = _copy_pair_slice(
+ sourceData,
+ sourceIndices,
+ destinationData,
+ destinationIndices,
+ int(window_starts[0]),
+ int(window_ends[0]),
+ output_position,
+ copyBufferBytes,
+ )
+ else:
+ output_position = _copy_masked_csr_window(
+ sourceData,
+ sourceIndices,
+ destinationData,
+ destinationIndices,
+ window_starts,
+ window_ends,
+ output_position,
+ )
+ window_start = window_end
+ if output_position != destinationStart + selected_nnz:
+ raise ValueError("Copied CSR data length does not match selected rows")
+ return output_position
+
+
+def _copy_selected_strings(
+ source: h5py.Dataset,
+ destination: h5py.Dataset,
+ rows: np.ndarray,
+ rowBatchSize: int,
+) -> None:
+ output_start = 0
+ for start in range(0, len(rows), rowBatchSize):
+ batch = rows[start : start + rowBatchSize]
+ span_rows = int(batch[-1] - batch[0] + 1)
+ if span_rows <= len(batch) * 2:
+ values = np.asarray(source[int(batch[0]) : int(batch[-1]) + 1])
+ values = values[batch - batch[0]]
+ else:
+ values = np.asarray(source[batch])
+ output_end = output_start + len(batch)
+ destination[output_start:output_end] = values
+ output_start = output_end
+
+
+def _check_integer_capacity(dtype: np.dtype[Any], maximum: int) -> None:
+ if not np.issubdtype(dtype, np.integer):
+ raise ValueError(f"CSR index dtype must be integer, got {dtype}")
+ if maximum > int(np.iinfo(dtype).max):
+ raise OverflowError(f"{maximum} does not fit in {dtype}")
+
+
+def _load_numeric_dataset(
+ dataset: h5py.Dataset,
+ *,
+ dtype: np.dtype[Any] | None = None,
+ chunkElements: int = _DEFAULT_LOAD_CHUNK_ELEMENTS,
+) -> np.ndarray:
+ if dataset.ndim != 1:
+ raise ValueError("Expected a 1-D HDF5 dataset")
+ if chunkElements <= 0:
+ raise ValueError("chunkElements must be positive")
+ length = int(dataset.shape[0])
+ out_dtype = np.dtype(dataset.dtype if dtype is None else dtype)
+ output = np.empty(length, dtype=out_dtype)
+ for start in range(0, length, chunkElements):
+ end = min(start + chunkElements, length)
+ output[start:end] = np.asarray(dataset[start:end], dtype=out_dtype)
+ return output
+
+
+def load_csr_source_into_memory(
+ path: str | Path,
+ *,
+ spec: SourceSpec = SOURCE_SPEC,
+ chunkElements: int = _DEFAULT_LOAD_CHUNK_ELEMENTS,
+) -> InMemoryCsrSource:
+ """Load the source CSR into RAM for repeated nested subset writes.
+
+ Column indices are stored as int32 when they fit, which roughly halves
+ resident memory versus the on-disk int64 index array.
+ """
+ path = Path(path)
+ with h5py.File(path, mode="r") as h5:
+ data_dataset = _require_dataset(h5, f"{spec.matrixKey}/data")
+ indices_dataset = _require_dataset(h5, f"{spec.matrixKey}/indices")
+ indptr_dataset = _require_dataset(h5, f"{spec.matrixKey}/indptr")
+ if data_dataset.shape != (spec.nnz,):
+ raise ValueError(
+ f"CSR data shape is {data_dataset.shape}, expected {(spec.nnz,)}"
+ )
+ if indices_dataset.shape != (spec.nnz,):
+ raise ValueError(
+ f"CSR indices shape is {indices_dataset.shape}, expected {(spec.nnz,)}"
+ )
+ if indptr_dataset.shape != (spec.nRows + 1,):
+ raise ValueError(
+ f"CSR indptr shape is {indptr_dataset.shape}, "
+ f"expected {(spec.nRows + 1,)}"
+ )
+
+ data = _load_numeric_dataset(data_dataset, chunkElements=chunkElements)
+ max_column = spec.nColumns - 1
+ if max_column <= int(np.iinfo(np.int32).max):
+ indices = _load_numeric_dataset(
+ indices_dataset,
+ dtype=np.dtype(np.int32),
+ chunkElements=chunkElements,
+ )
+ else:
+ indices = _load_numeric_dataset(
+ indices_dataset,
+ chunkElements=chunkElements,
+ )
+ indptr = _load_numeric_dataset(indptr_dataset, chunkElements=chunkElements)
+ if int(indptr[0]) != 0 or int(indptr[-1]) != spec.nnz:
+ raise ValueError("Loaded CSR indptr does not match source nnz")
+
+ cell_ids = _decode_string_array(_require_dataset(h5, spec.cellIdsKey)[:])
+ feature_ids = _decode_string_array(_require_dataset(h5, spec.featureIdsKey)[:])
+ feature_names = _load_string_column(
+ h5,
+ spec.featureNameKey,
+ expectedLength=spec.nColumns,
+ )
+
+ if cell_ids.shape != (spec.nRows,):
+ raise ValueError(
+ f"Loaded cell ids shape is {cell_ids.shape}, expected {(spec.nRows,)}"
+ )
+ if feature_ids.shape != (spec.nColumns,):
+ raise ValueError(
+ f"Loaded feature ids shape is {feature_ids.shape}, "
+ f"expected {(spec.nColumns,)}"
+ )
+ if feature_names.shape != (spec.nColumns,):
+ raise ValueError(
+ f"Loaded feature names shape is {feature_names.shape}, "
+ f"expected {(spec.nColumns,)}"
+ )
+
+ return InMemoryCsrSource(
+ data=data,
+ indices=indices,
+ indptr=indptr,
+ nRows=spec.nRows,
+ nColumns=spec.nColumns,
+ dataDtype=np.dtype(spec.dataDtype),
+ indicesDtype=np.dtype(spec.indicesDtype),
+ indptrDtype=np.dtype(spec.indptrDtype),
+ cellIds=cell_ids,
+ featureIds=feature_ids,
+ featureNames=feature_names,
+ )
+
+
+def _copy_memory_csr_batch(
+ source: InMemoryCsrSource,
+ destinationData: h5py.Dataset,
+ destinationIndices: h5py.Dataset,
+ rows: np.ndarray,
+ destinationStart: int,
+) -> int:
+ starts = source.indptr[rows]
+ ends = source.indptr[rows + 1]
+ counts = ends - starts
+ if np.any(counts < 0):
+ raise ValueError("Source CSR indptr is not monotonic")
+ selected_nnz = int(counts.sum(dtype=np.int64))
+ if selected_nnz == 0:
+ return destinationStart
+
+ buffer_data = np.empty(selected_nnz, dtype=source.data.dtype)
+ buffer_indices = np.empty(selected_nnz, dtype=source.indices.dtype)
+ position = 0
+ for start, end in zip(starts.tolist(), ends.tolist(), strict=True):
+ width = end - start
+ buffer_data[position : position + width] = source.data[start:end]
+ buffer_indices[position : position + width] = source.indices[start:end]
+ position += width
+ if position != selected_nnz:
+ raise ValueError("Copied CSR batch length does not match selected nnz")
+
+ destination_end = destinationStart + selected_nnz
+ destinationData[destinationStart:destination_end] = buffer_data
+ if buffer_indices.dtype == source.indicesDtype:
+ destinationIndices[destinationStart:destination_end] = buffer_indices
+ else:
+ destinationIndices[destinationStart:destination_end] = buffer_indices.astype(
+ source.indicesDtype,
+ copy=False,
+ )
+ return destination_end
+
+
+def _copy_selected_strings_array(
+ source: np.ndarray,
+ destination: h5py.Dataset,
+ rows: np.ndarray,
+ rowBatchSize: int,
+) -> None:
+ output_start = 0
+ for start in range(0, len(rows), rowBatchSize):
+ batch = rows[start : start + rowBatchSize]
+ output_end = output_start + len(batch)
+ destination[output_start:output_end] = source[batch]
+ output_start = output_end
+
+
+def write_h5ad_sample_from_memory(
+ source: InMemoryCsrSource,
+ destinationPath: str | Path,
+ sourceRows: np.ndarray | Sequence[int],
+ *,
+ rowBatchSize: int = _DEFAULT_ROW_BATCH_SIZE,
+) -> H5adWriteResult:
+ if rowBatchSize <= 0:
+ raise ValueError("rowBatchSize must be positive")
+
+ destination_path = Path(destinationPath)
+ destination_path.parent.mkdir(parents=True, exist_ok=True)
+ if destination_path.exists():
+ raise FileExistsError(f"Destination already exists: {destination_path}")
+ temporary_path = destination_path.with_name(
+ f".{destination_path.name}.{uuid.uuid4().hex}.part"
+ )
+ rows = _validate_selected_rows(sourceRows, source.nRows)
+ nnz = int((source.indptr[rows + 1] - source.indptr[rows]).sum(dtype=np.int64))
+ _check_integer_capacity(source.indptrDtype, nnz)
+ _check_integer_capacity(source.indicesDtype, source.nColumns - 1)
+
+ try:
+ with h5py.File(temporary_path, mode="w") as destination:
+ destination.attrs["encoding-type"] = "anndata"
+ destination.attrs["encoding-version"] = "0.1.0"
+ matrix = destination.create_group("X")
+ matrix.attrs["encoding-type"] = "csr_matrix"
+ matrix.attrs["encoding-version"] = "0.1.0"
+ matrix.attrs["shape"] = np.asarray(
+ [len(rows), source.nColumns],
+ dtype=np.int64,
+ )
+ destination_data = _create_numeric_dataset(
+ matrix,
+ "data",
+ nnz,
+ source.dataDtype,
+ )
+ destination_indices = _create_numeric_dataset(
+ matrix,
+ "indices",
+ nnz,
+ source.indicesDtype,
+ )
+ destination_indptr = _create_numeric_dataset(
+ matrix,
+ "indptr",
+ len(rows) + 1,
+ source.indptrDtype,
+ )
+ destination_indptr[0] = 0
+
+ output_data_position = 0
+ output_row_position = 0
+ for start in range(0, len(rows), rowBatchSize):
+ batch = rows[start : start + rowBatchSize]
+ batch_starts = source.indptr[batch]
+ batch_ends = source.indptr[batch + 1]
+ counts = batch_ends - batch_starts
+ cumulative = np.cumsum(counts, dtype=np.int64) + output_data_position
+ output_row_end = output_row_position + len(batch)
+ destination_indptr[output_row_position + 1 : output_row_end + 1] = (
+ cumulative.astype(source.indptrDtype, copy=False)
+ )
+ output_data_position = _copy_memory_csr_batch(
+ source,
+ destination_data,
+ destination_indices,
+ batch,
+ output_data_position,
+ )
+ output_row_position = output_row_end
+ if output_data_position != nnz:
+ raise ValueError(
+ f"Copied {output_data_position} values, expected {nnz}"
+ )
+
+ obs = _create_dataframe_group(destination, "obs", ())
+ output_cell_ids = _create_string_dataset(obs, "_index", len(rows))
+ _copy_selected_strings_array(
+ source.cellIds,
+ output_cell_ids,
+ rows,
+ rowBatchSize,
+ )
+
+ var = _create_dataframe_group(
+ destination,
+ "var",
+ ("feature_name",),
+ )
+ output_feature_ids = _create_string_dataset(
+ var,
+ "_index",
+ source.nColumns,
+ )
+ output_feature_names = _create_string_dataset(
+ var,
+ "feature_name",
+ source.nColumns,
+ )
+ output_feature_ids[:] = source.featureIds
+ output_feature_names[:] = source.featureNames
+
+ os.replace(temporary_path, destination_path)
+ except BaseException:
+ temporary_path.unlink(missing_ok=True)
+ raise
+
+ return H5adWriteResult(
+ filePath=destination_path,
+ nRows=len(rows),
+ nColumns=source.nColumns,
+ nnz=nnz,
+ sourceRowsSha256=ordered_source_row_digest(rows),
+ finalSourceRow=int(rows[-1]),
+ dataDtype=str(source.dataDtype),
+ indicesDtype=str(source.indicesDtype),
+ indptrDtype=str(source.indptrDtype),
+ )
+
+
+def write_h5ad_sample(
+ sourcePath: str | Path,
+ destinationPath: str | Path,
+ sourceRows: np.ndarray | Sequence[int],
+ *,
+ spec: SourceSpec = SOURCE_SPEC,
+ rowBatchSize: int = _DEFAULT_ROW_BATCH_SIZE,
+ copyBufferBytes: int = _DEFAULT_COPY_BUFFER_BYTES,
+) -> H5adWriteResult:
+ if rowBatchSize <= 0:
+ raise ValueError("rowBatchSize must be positive")
+ if copyBufferBytes <= 0:
+ raise ValueError("copyBufferBytes must be positive")
+
+ source_path = Path(sourcePath)
+ destination_path = Path(destinationPath)
+ destination_path.parent.mkdir(parents=True, exist_ok=True)
+ if destination_path.exists():
+ raise FileExistsError(f"Destination already exists: {destination_path}")
+ temporary_path = destination_path.with_name(
+ f".{destination_path.name}.{uuid.uuid4().hex}.part"
+ )
+
+ try:
+ with (
+ h5py.File(source_path, mode="r") as source,
+ h5py.File(temporary_path, mode="w") as destination,
+ ):
+ source_matrix = _require_group(source, spec.matrixKey)
+ shape = tuple(int(value) for value in source_matrix.attrs.get("shape", ()))
+ if len(shape) != 2:
+ raise ValueError(f"{spec.matrixKey} has invalid shape metadata")
+ n_source_rows, n_columns = shape
+ rows = _validate_selected_rows(sourceRows, n_source_rows)
+
+ source_data = _require_dataset(source, f"{spec.matrixKey}/data")
+ source_indices = _require_dataset(
+ source,
+ f"{spec.matrixKey}/indices",
+ )
+ source_indptr = _require_dataset(
+ source,
+ f"{spec.matrixKey}/indptr",
+ )
+ if source_indptr.shape != (n_source_rows + 1,):
+ raise ValueError("Source CSR indptr length does not match shape")
+ if not np.issubdtype(source_indices.dtype, np.integer):
+ raise ValueError("Source CSR indices must use an integer dtype")
+ if not np.issubdtype(source_indptr.dtype, np.integer):
+ raise ValueError("Source CSR indptr must use an integer dtype")
+ data_dtype = str(source_data.dtype)
+ indices_dtype = str(source_indices.dtype)
+ indptr_dtype = str(source_indptr.dtype)
+
+ nnz = _selected_nnz(source_indptr, rows, rowBatchSize)
+ _check_integer_capacity(source_indptr.dtype, nnz)
+ _check_integer_capacity(source_indices.dtype, n_columns - 1)
+
+ destination.attrs["encoding-type"] = "anndata"
+ destination.attrs["encoding-version"] = "0.1.0"
+ matrix = destination.create_group("X")
+ matrix.attrs["encoding-type"] = "csr_matrix"
+ matrix.attrs["encoding-version"] = "0.1.0"
+ matrix.attrs["shape"] = np.asarray(
+ [len(rows), n_columns],
+ dtype=np.int64,
+ )
+ destination_data = _create_numeric_dataset(
+ matrix,
+ "data",
+ nnz,
+ source_data.dtype,
+ )
+ destination_indices = _create_numeric_dataset(
+ matrix,
+ "indices",
+ nnz,
+ source_indices.dtype,
+ )
+ destination_indptr = _create_numeric_dataset(
+ matrix,
+ "indptr",
+ len(rows) + 1,
+ source_indptr.dtype,
+ )
+ destination_indptr[0] = 0
+
+ output_data_position = 0
+ output_row_position = 0
+ for start in range(0, len(rows), rowBatchSize):
+ batch = rows[start : start + rowBatchSize]
+ row_starts, row_ends = _row_boundaries(source_indptr, batch)
+ counts = row_ends - row_starts
+ cumulative = np.cumsum(counts, dtype=np.int64) + output_data_position
+ output_row_end = output_row_position + len(batch)
+ destination_indptr[output_row_position + 1 : output_row_end + 1] = (
+ cumulative.astype(source_indptr.dtype, copy=False)
+ )
+ output_data_position = _copy_csr_batch(
+ source_data,
+ source_indices,
+ destination_data,
+ destination_indices,
+ batch,
+ row_starts,
+ row_ends,
+ output_data_position,
+ copyBufferBytes,
+ )
+ output_row_position = output_row_end
+ if output_data_position != nnz:
+ raise ValueError(
+ f"Copied {output_data_position} values, expected {nnz}"
+ )
+
+ obs = _create_dataframe_group(destination, "obs", ())
+ output_cell_ids = _create_string_dataset(obs, "_index", len(rows))
+ source_cell_ids = _require_dataset(source, spec.cellIdsKey)
+ _copy_selected_strings(
+ source_cell_ids,
+ output_cell_ids,
+ rows,
+ rowBatchSize,
+ )
+
+ var = _create_dataframe_group(
+ destination,
+ "var",
+ ("feature_name",),
+ )
+ output_feature_ids = _create_string_dataset(
+ var,
+ "_index",
+ n_columns,
+ )
+ output_feature_names = _create_string_dataset(
+ var,
+ "feature_name",
+ n_columns,
+ )
+ source_feature_ids = _require_dataset(source, spec.featureIdsKey)
+ source_feature_names = _load_string_column(
+ source,
+ spec.featureNameKey,
+ expectedLength=n_columns,
+ )
+ output_feature_ids[:] = source_feature_ids[:]
+ output_feature_names[:] = source_feature_names[:]
+
+ os.replace(temporary_path, destination_path)
+ except BaseException:
+ temporary_path.unlink(missing_ok=True)
+ raise
+
+ return H5adWriteResult(
+ filePath=destination_path,
+ nRows=len(rows),
+ nColumns=n_columns,
+ nnz=nnz,
+ sourceRowsSha256=ordered_source_row_digest(rows),
+ finalSourceRow=int(rows[-1]),
+ dataDtype=data_dtype,
+ indicesDtype=indices_dtype,
+ indptrDtype=indptr_dtype,
+ )
+
+
+def read_csr_row(
+ path: str | Path,
+ row: int,
+ *,
+ matrixKey: str = SOURCE_SPEC.matrixKey,
+) -> tuple[np.ndarray, np.ndarray]:
+ with h5py.File(path, mode="r") as h5:
+ matrix = _require_group(h5, matrixKey)
+ shape = tuple(int(value) for value in matrix.attrs.get("shape", ()))
+ if len(shape) != 2 or row < 0 or row >= shape[0]:
+ raise IndexError(f"CSR row {row} is outside matrix shape {shape}")
+ indptr = _require_dataset(h5, f"{matrixKey}/indptr")
+ start, end = (int(value) for value in indptr[row : row + 2])
+ indices = np.asarray(_require_dataset(h5, f"{matrixKey}/indices")[start:end])
+ data = np.asarray(_require_dataset(h5, f"{matrixKey}/data")[start:end])
+ return indices, data
+
+
+def _attribute_strings(value: Any) -> list[str]:
+ return [_decode_text(item) for item in np.asarray(value).reshape(-1)]
+
+
+def _validate_output_dataframe(
+ h5: h5py.File,
+ name: str,
+ expectedLength: int,
+ columns: Sequence[str],
+) -> None:
+ group = _require_group(h5, name)
+ _require_encoding(group, name, "dataframe", "0.2.0")
+ if _decode_text(group.attrs.get("_index")) != "_index":
+ raise ValueError(f"{name} dataframe index must be _index")
+ if _attribute_strings(group.attrs.get("column-order", ())) != list(columns):
+ raise ValueError(f"{name} dataframe columns do not match {list(columns)}")
+ expected_keys = {"_index", *columns}
+ if set(group.keys()) != expected_keys:
+ raise ValueError(
+ f"{name} contains {sorted(group.keys())}, expected {sorted(expected_keys)}"
+ )
+ _validate_string_dataset(
+ _require_dataset(h5, f"{name}/_index"),
+ f"{name}/_index",
+ expectedLength,
+ requireEncoding=True,
+ )
+ for column in columns:
+ _validate_string_dataset(
+ _require_dataset(h5, f"{name}/{column}"),
+ f"{name}/{column}",
+ expectedLength,
+ requireEncoding=True,
+ )
+
+
+def validate_prepared_h5ad(
+ path: str | Path,
+ *,
+ expectedRows: int,
+ expectedColumns: int,
+ expectedNnz: int,
+ expectedSourceRows: np.ndarray | Sequence[int],
+ expectedFinalIndices: np.ndarray,
+ expectedFinalData: np.ndarray,
+ expectedDataDtype: str,
+ expectedIndicesDtype: str,
+ expectedIndptrDtype: str,
+ expectedSha256: str | None = None,
+ indptrChunkRows: int = _DEFAULT_INDPTR_CHUNK_ROWS,
+) -> PreparedValidation:
+ path = Path(path)
+ source_rows = _validate_selected_rows(expectedSourceRows, max(1, 1 << 63))
+ if len(source_rows) != expectedRows:
+ raise ValueError(
+ f"Expected source row list has {len(source_rows)} rows, "
+ f"expected {expectedRows}"
+ )
+
+ with h5py.File(path, mode="r") as h5:
+ if set(h5.keys()) != {"X", "obs", "var"}:
+ raise ValueError(
+ f"Prepared H5AD root contains unexpected keys: {sorted(h5.keys())}"
+ )
+ _require_encoding(h5, "/", "anndata", "0.1.0")
+ matrix = _require_group(h5, "X")
+ _require_encoding(matrix, "X", "csr_matrix", "0.1.0")
+ shape = tuple(int(value) for value in matrix.attrs.get("shape", ()))
+ if shape != (expectedRows, expectedColumns):
+ raise ValueError(
+ f"Prepared matrix shape is {shape}, expected "
+ f"{(expectedRows, expectedColumns)}"
+ )
+
+ data = _require_dataset(h5, "X/data")
+ indices = _require_dataset(h5, "X/indices")
+ indptr = _require_dataset(h5, "X/indptr")
+ if data.shape != (expectedNnz,) or indices.shape != (expectedNnz,):
+ raise ValueError("Prepared CSR data lengths do not match expected nnz")
+ if data.dtype != np.dtype(expectedDataDtype):
+ raise ValueError(
+ f"Prepared data dtype is {data.dtype}, expected {expectedDataDtype}"
+ )
+ if indices.dtype != np.dtype(expectedIndicesDtype):
+ raise ValueError(
+ f"Prepared indices dtype is {indices.dtype}, "
+ f"expected {expectedIndicesDtype}"
+ )
+ _validate_indptr(
+ indptr,
+ nRows=expectedRows,
+ nnz=expectedNnz,
+ expectedDtype=expectedIndptrDtype,
+ chunkRows=indptrChunkRows,
+ )
+
+ final_start, final_end = (
+ int(value) for value in indptr[expectedRows - 1 : expectedRows + 1]
+ )
+ final_indices = np.asarray(indices[final_start:final_end])
+ final_data = np.asarray(data[final_start:final_end])
+ if not np.array_equal(final_indices, expectedFinalIndices):
+ raise ValueError("Prepared final-row indices do not match source")
+ if not np.array_equal(final_data, expectedFinalData):
+ raise ValueError("Prepared final-row data do not match source")
+
+ _validate_output_dataframe(h5, "obs", expectedRows, ())
+ _validate_output_dataframe(
+ h5,
+ "var",
+ expectedColumns,
+ ("feature_name",),
+ )
+
+ import anndata
+
+ backed = anndata.read_h5ad(path, backed="r")
+ try:
+ if backed.shape != (expectedRows, expectedColumns):
+ raise ValueError(
+ f"AnnData sees shape {backed.shape}, expected "
+ f"{(expectedRows, expectedColumns)}"
+ )
+ finally:
+ backed.file.close()
+
+ from scarf.readers import H5adReader
+
+ reader = H5adReader(
+ str(path),
+ matrix_key="X",
+ cell_attrs_key="obs",
+ cell_ids_key="_index",
+ feature_attrs_key="var",
+ feature_ids_key="_index",
+ feature_name_key="feature_name",
+ )
+ try:
+ if (reader.nCells, reader.nFeatures) != (
+ expectedRows,
+ expectedColumns,
+ ):
+ raise ValueError(
+ "Scarf H5adReader dimensions do not match prepared artifact"
+ )
+ if np.dtype(reader.matrixDtype) != np.dtype(expectedDataDtype):
+ raise ValueError("Scarf H5adReader sees an unexpected data dtype")
+ finally:
+ reader.h5.close()
+
+ actual_sha256 = sha256_file(path)
+ if expectedSha256 is not None and actual_sha256 != expectedSha256.lower():
+ raise ValueError(
+ f"Prepared SHA256 is {actual_sha256}, expected {expectedSha256}"
+ )
+ return PreparedValidation(
+ fileBytes=path.stat().st_size,
+ sha256=actual_sha256,
+ sourceRowsSha256=ordered_source_row_digest(source_rows),
+ nRows=expectedRows,
+ nColumns=expectedColumns,
+ nnz=expectedNnz,
+ finalSourceRow=int(source_rows[-1]),
+ )
+
+
+def prepare_local_datasets(
+ sourcePath: str | Path,
+ outputDirectory: str | Path,
+ *,
+ targetRows: Sequence[int] = DEFAULT_TARGET_SIZES,
+ seed: int = DEFAULT_SAMPLING_SEED,
+ spec: SourceSpec = SOURCE_SPEC,
+ rowBatchSize: int = _DEFAULT_ROW_BATCH_SIZE,
+ copyBufferBytes: int = _DEFAULT_COPY_BUFFER_BYTES,
+ onArtifact: Callable[[PreparedArtifact], None] | None = None,
+) -> PreparationResult:
+ del copyBufferBytes # retained for call-site compatibility; unused in-memory path
+ source_path = Path(sourcePath)
+ output_directory = Path(outputDirectory)
+ output_directory.mkdir(parents=True, exist_ok=True)
+ source_validation = validate_source_h5ad(source_path, spec=spec)
+ source_sha256 = sha256_file(source_path)
+ selections = select_nested_rows(
+ spec.nRows,
+ targetRows,
+ seed=seed,
+ sourceVersion=spec.versionId,
+ )
+ memory_source = load_csr_source_into_memory(source_path, spec=spec)
+
+ artifacts: list[PreparedArtifact] = []
+ for target_rows, source_rows in selections.items():
+ output_path = output_directory / f"{target_rows}.h5ad"
+ if output_path.exists():
+ output_path.unlink()
+ written = write_h5ad_sample_from_memory(
+ memory_source,
+ output_path,
+ source_rows,
+ rowBatchSize=rowBatchSize,
+ )
+ final_start = int(memory_source.indptr[written.finalSourceRow])
+ final_end = int(memory_source.indptr[written.finalSourceRow + 1])
+ final_indices = np.asarray(
+ memory_source.indices[final_start:final_end],
+ dtype=memory_source.indicesDtype,
+ )
+ final_data = np.asarray(memory_source.data[final_start:final_end])
+ validated = validate_prepared_h5ad(
+ output_path,
+ expectedRows=target_rows,
+ expectedColumns=spec.nColumns,
+ expectedNnz=written.nnz,
+ expectedSourceRows=source_rows,
+ expectedFinalIndices=final_indices,
+ expectedFinalData=final_data,
+ expectedDataDtype=source_validation.dataDtype,
+ expectedIndicesDtype=source_validation.indicesDtype,
+ expectedIndptrDtype=source_validation.indptrDtype,
+ )
+ if validated.sourceRowsSha256 != written.sourceRowsSha256:
+ raise ValueError("Source-row digest changed during validation")
+ artifact = PreparedArtifact(
+ localPath=output_path,
+ targetRows=target_rows,
+ nColumns=spec.nColumns,
+ nnz=written.nnz,
+ fileBytes=validated.fileBytes,
+ sha256=validated.sha256,
+ sourceRowsSha256=validated.sourceRowsSha256,
+ finalSourceRow=written.finalSourceRow,
+ dataDtype=written.dataDtype,
+ indicesDtype=written.indicesDtype,
+ indptrDtype=written.indptrDtype,
+ )
+ artifacts.append(artifact)
+ if onArtifact is not None:
+ onArtifact(artifact)
+
+ return PreparationResult(
+ sourcePath=source_path,
+ sourceSha256=source_sha256,
+ sourceSpec=spec,
+ seed=seed,
+ artifacts=tuple(artifacts),
+ )
+
+
+def write_fixture_h5ad(
+ destinationPath: str | Path,
+ *,
+ nRows: int,
+ nColumns: int = 500,
+ seed: int = 0,
+ avgNnzPerRow: int = 40,
+) -> PreparedArtifact:
+ """Write a small synthetic CSR H5AD for downstream stage smoke tests."""
+ if nRows <= 0 or nColumns <= 0:
+ raise ValueError("nRows and nColumns must be positive")
+ if avgNnzPerRow <= 0:
+ raise ValueError("avgNnzPerRow must be positive")
+
+ destination_path = Path(destinationPath)
+ destination_path.parent.mkdir(parents=True, exist_ok=True)
+ if destination_path.exists():
+ raise FileExistsError(destination_path)
+
+ rng = np.random.default_rng(seed)
+ nnz_per_row = np.clip(
+ rng.poisson(avgNnzPerRow, size=nRows),
+ 1,
+ nColumns,
+ ).astype(np.int64)
+ indptr = np.empty(nRows + 1, dtype=np.int64)
+ indptr[0] = 0
+ np.cumsum(nnz_per_row, out=indptr[1:])
+ nnz = int(indptr[-1])
+ indices = np.empty(nnz, dtype=np.int64)
+ data = rng.integers(1, 20, size=nnz, dtype=np.int32).astype(np.float32)
+ for row, start, end in zip(
+ range(nRows),
+ indptr[:-1],
+ indptr[1:],
+ strict=True,
+ ):
+ chosen = rng.choice(nColumns, size=int(end - start), replace=False)
+ chosen.sort()
+ indices[start:end] = chosen
+
+ feature_ids = np.asarray(
+ [f"ENSG{i:011d}" for i in range(nColumns)],
+ dtype=object,
+ )
+ feature_names = np.asarray(
+ ["MT-ND1" if i % 50 == 0 else f"GENE{i}" for i in range(nColumns)],
+ dtype=object,
+ )
+ cell_ids = np.asarray([f"cell-{i}" for i in range(nRows)], dtype=object)
+
+ temporary_path = destination_path.with_name(
+ f".{destination_path.name}.{uuid.uuid4().hex}.part"
+ )
+ try:
+ with h5py.File(temporary_path, mode="w") as h5:
+ h5.attrs["encoding-type"] = "anndata"
+ h5.attrs["encoding-version"] = "0.1.0"
+ matrix = h5.create_group("X")
+ matrix.attrs["encoding-type"] = "csr_matrix"
+ matrix.attrs["encoding-version"] = "0.1.0"
+ matrix.attrs["shape"] = np.asarray([nRows, nColumns], dtype=np.int64)
+ _create_numeric_dataset(matrix, "data", nnz, data.dtype)[:] = data
+ _create_numeric_dataset(matrix, "indices", nnz, indices.dtype)[:] = indices
+ _create_numeric_dataset(
+ matrix,
+ "indptr",
+ nRows + 1,
+ indptr.dtype,
+ )[:] = indptr
+
+ obs = _create_dataframe_group(h5, "obs", ())
+ _create_string_dataset(obs, "_index", nRows)[:] = cell_ids
+ var = _create_dataframe_group(h5, "var", ("feature_name",))
+ _create_string_dataset(var, "_index", nColumns)[:] = feature_ids
+ _create_string_dataset(var, "feature_name", nColumns)[:] = feature_names
+ os.replace(temporary_path, destination_path)
+ except BaseException:
+ temporary_path.unlink(missing_ok=True)
+ raise
+
+ return PreparedArtifact(
+ localPath=destination_path,
+ targetRows=nRows,
+ nColumns=nColumns,
+ nnz=nnz,
+ fileBytes=destination_path.stat().st_size,
+ sha256=sha256_file(destination_path),
+ sourceRowsSha256=ordered_source_row_digest(np.arange(nRows, dtype=np.int64)),
+ finalSourceRow=nRows - 1,
+ dataDtype=str(data.dtype),
+ indicesDtype=str(indices.dtype),
+ indptrDtype=str(indptr.dtype),
+ )
+
+
+def prepare_fixture_datasets(
+ outputDirectory: str | Path,
+ *,
+ targetRows: Sequence[int],
+ nColumns: int = 500,
+ seed: int = 0,
+ onArtifact: Callable[[PreparedArtifact], None] | None = None,
+) -> tuple[PreparedArtifact, ...]:
+ output_directory = Path(outputDirectory)
+ output_directory.mkdir(parents=True, exist_ok=True)
+ artifacts: list[PreparedArtifact] = []
+ for n_rows in targetRows:
+ artifact = write_fixture_h5ad(
+ output_directory / f"{n_rows}.h5ad",
+ nRows=int(n_rows),
+ nColumns=nColumns,
+ seed=seed + int(n_rows),
+ )
+ artifacts.append(artifact)
+ if onArtifact is not None:
+ onArtifact(artifact)
+ return tuple(artifacts)
diff --git a/profiling/metrics.py b/profiling/metrics.py
new file mode 100644
index 00000000..4b7460d1
--- /dev/null
+++ b/profiling/metrics.py
@@ -0,0 +1,1109 @@
+import math
+import os
+import shutil
+import threading
+import time
+from collections.abc import Callable, Iterable, Iterator
+from contextlib import contextmanager
+from dataclasses import dataclass
+from io import TextIOBase
+from pathlib import Path
+from types import TracebackType
+from typing import Literal, Self
+
+type PeakScope = Literal["operation", "containerLifetime", "unavailable"]
+type LimitValue = int | Literal["max"] | None
+type _ReadText = Callable[[Path], str]
+type _WriteText = Callable[[Path, str], None]
+type _ListPids = Callable[[Path], Iterable[int]]
+type _AffinityReader = Callable[[int], Iterable[int]]
+type _DiskUsageReader = Callable[[Path], object]
+
+
+@dataclass(frozen=True, slots=True)
+class DiskUsage:
+ totalBytes: int
+ freeBytes: int
+ usedBytes: int
+
+
+@dataclass(frozen=True, slots=True)
+class ResourceMeasurement:
+ sampleCount: int
+ sampleIntervalSeconds: float
+ operationBaselineBytes: int | None = None
+ operationPeakBytes: int | None = None
+ operationIncrementalPeakBytes: int | None = None
+ operationPeakSource: str | None = None
+ processTreeRssBaselineBytes: int | None = None
+ processTreeRssPeakBytes: int | None = None
+ processTreeRssIncrementalPeakBytes: int | None = None
+ processTreeRssAfterBytes: int | None = None
+ cgroupPath: str | None = None
+ cgroupMemoryCurrentBaselineBytes: int | None = None
+ cgroupMemoryCurrentPeakBytes: int | None = None
+ cgroupMemoryCurrentAfterBytes: int | None = None
+ cgroupMemoryPeakBytes: int | None = None
+ cgroupMemoryPeakScope: PeakScope = "unavailable"
+ memoryMaxBytes: LimitValue = None
+ memorySwapCurrentBeforeBytes: int | None = None
+ memorySwapCurrentAfterBytes: int | None = None
+ memorySwapCurrentPeakBytes: int | None = None
+ memorySwapMaxBytes: LimitValue = None
+ memoryEventsBefore: dict[str, int] | None = None
+ memoryEventsAfter: dict[str, int] | None = None
+ memoryEventsDelta: dict[str, int] | None = None
+ cpuQuotaMicros: LimitValue = None
+ cpuPeriodMicros: int | None = None
+ cpuQuotaCores: float | None = None
+ cpuAffinityCpus: tuple[int, ...] | None = None
+ cpuAffinityCount: int | None = None
+ cpuAffinitySource: str | None = None
+ ephemeralDiskPath: str | None = None
+ ephemeralDiskBefore: DiskUsage | None = None
+ ephemeralDiskAfter: DiskUsage | None = None
+ ephemeralDiskPeak: DiskUsage | None = None
+ samplingErrorCount: int = 0
+
+
+@dataclass(frozen=True, slots=True)
+class StageTimings:
+ inputSetupSeconds: float | None = None
+ measuredOperationSeconds: float | None = None
+ validationPersistenceSeconds: float | None = None
+ wholeFunctionSeconds: float | None = None
+
+
+@dataclass(frozen=True, slots=True)
+class _Sample:
+ processTreeRssBytes: int | None
+ cgroupMemoryCurrentBytes: int | None
+ memorySwapCurrentBytes: int | None
+ ephemeralDisk: DiskUsage | None
+
+
+@dataclass(frozen=True, slots=True)
+class _CpuSnapshot:
+ quotaMicros: LimitValue = None
+ periodMicros: int | None = None
+ quotaCores: float | None = None
+ affinityCpus: tuple[int, ...] | None = None
+ affinitySource: str | None = None
+
+
+def _default_read_text(path: Path) -> str:
+ return path.read_text(encoding="utf-8")
+
+
+def _default_write_text(path: Path, value: str) -> None:
+ path.write_text(value, encoding="utf-8")
+
+
+def _default_list_pids(procRoot: Path) -> list[int]:
+ return [
+ int(entry.name)
+ for entry in procRoot.iterdir()
+ if entry.name.isdigit() and entry.is_dir()
+ ]
+
+
+def _default_affinity_reader(pid: int) -> Iterable[int]:
+ reader = getattr(os, "sched_getaffinity", None)
+ if reader is None:
+ raise OSError("CPU affinity is unavailable")
+ return reader(pid)
+
+
+def _default_disk_usage_reader(path: Path) -> object:
+ return shutil.disk_usage(path)
+
+
+def _optional_read(path: Path, readText: _ReadText) -> str | None:
+ try:
+ value = readText(path)
+ except Exception:
+ return None
+ return value if isinstance(value, str) else None
+
+
+def _optional_write(path: Path, value: str, writeText: _WriteText) -> bool:
+ try:
+ writeText(path, value)
+ except Exception:
+ return False
+ return True
+
+
+def _nonnegative_int(value: str | None) -> int | None:
+ if value is None:
+ return None
+ try:
+ parsed = int(value.strip())
+ except (TypeError, ValueError):
+ return None
+ return parsed if parsed >= 0 else None
+
+
+def _limit_value(value: str | None) -> LimitValue:
+ if value is None:
+ return None
+ value = value.strip()
+ if value == "max":
+ return "max"
+ return _nonnegative_int(value)
+
+
+def _camel_case(value: str) -> str:
+ first, *remaining = value.split("_")
+ return first + "".join(part[:1].upper() + part[1:] for part in remaining)
+
+
+def parse_memory_events(value: str) -> dict[str, int]:
+ events: dict[str, int] = {}
+ for line in value.splitlines():
+ parts = line.split()
+ if len(parts) != 2:
+ continue
+ count = _nonnegative_int(parts[1])
+ if count is not None:
+ events[_camel_case(parts[0])] = count
+ return events
+
+
+def _memory_event_delta(
+ before: dict[str, int] | None,
+ after: dict[str, int] | None,
+) -> dict[str, int] | None:
+ if before is None or after is None:
+ return None
+ return {
+ key: after.get(key, 0) - before.get(key, 0)
+ for key in sorted(before.keys() | after.keys())
+ }
+
+
+def _parse_proc_status(value: str) -> tuple[int | None, int | None]:
+ parent_pid: int | None = None
+ rss_bytes: int | None = None
+ for line in value.splitlines():
+ key, separator, raw_value = line.partition(":")
+ if not separator:
+ continue
+ parts = raw_value.split()
+ if not parts:
+ continue
+ if key == "PPid":
+ parent_pid = _nonnegative_int(parts[0])
+ elif key == "VmRSS":
+ amount = _nonnegative_int(parts[0])
+ if amount is None:
+ continue
+ unit = parts[1].lower() if len(parts) > 1 else "b"
+ scale = {"b": 1, "kb": 1024, "mb": 1024**2}.get(unit)
+ if scale is not None:
+ rss_bytes = amount * scale
+ return parent_pid, rss_bytes
+
+
+def read_process_tree_rss_bytes(
+ rootPid: int,
+ *,
+ procRoot: str | Path = "/proc",
+ readText: _ReadText | None = None,
+ listPids: _ListPids | None = None,
+) -> int | None:
+ if isinstance(rootPid, bool) or rootPid <= 0:
+ return None
+ proc_root = Path(procRoot)
+ text_reader = _default_read_text if readText is None else readText
+ pid_reader = _default_list_pids if listPids is None else listPids
+ try:
+ pids = {int(pid) for pid in pid_reader(proc_root)}
+ except Exception:
+ return None
+ pids.add(rootPid)
+
+ records: dict[int, tuple[int | None, int | None]] = {}
+ children: dict[int, set[int]] = {}
+ for pid in pids:
+ status = _optional_read(proc_root / str(pid) / "status", text_reader)
+ if status is None:
+ continue
+ parent_pid, rss_bytes = _parse_proc_status(status)
+ records[pid] = (parent_pid, rss_bytes)
+ if parent_pid is not None:
+ children.setdefault(parent_pid, set()).add(pid)
+
+ tree_pids: set[int] = set()
+ pending = [rootPid]
+ while pending:
+ pid = pending.pop()
+ if pid in tree_pids:
+ continue
+ tree_pids.add(pid)
+ pending.extend(children.get(pid, ()))
+
+ rss_values = [
+ records[pid][1]
+ for pid in tree_pids
+ if pid in records and records[pid][1] is not None
+ ]
+ if not rss_values:
+ return None
+ return sum(rss_values)
+
+
+def _coerce_disk_usage(value: object) -> DiskUsage | None:
+ if isinstance(value, DiskUsage):
+ return value
+ try:
+ if all(hasattr(value, name) for name in ("total", "used", "free")):
+ total = getattr(value, "total")
+ used = getattr(value, "used")
+ free = getattr(value, "free")
+ else:
+ total, used, free = value
+ except (AttributeError, TypeError, ValueError):
+ return None
+ values = (total, free, used)
+ if any(isinstance(item, bool) or not isinstance(item, int) for item in values):
+ return None
+ if any(item < 0 for item in values):
+ return None
+ return DiskUsage(totalBytes=total, freeBytes=free, usedBytes=used)
+
+
+def _parse_cpuset(value: str | None) -> tuple[int, ...] | None:
+ if value is None:
+ return None
+ cpus: set[int] = set()
+ try:
+ for item in value.strip().split(","):
+ if not item:
+ continue
+ if "-" not in item:
+ cpus.add(int(item))
+ continue
+ start, end = (int(part) for part in item.split("-", 1))
+ if start > end:
+ return None
+ cpus.update(range(start, end + 1))
+ except ValueError:
+ return None
+ return tuple(sorted(cpus)) if cpus else None
+
+
+def _relative_cgroup_path(value: str) -> tuple[str, ...] | None:
+ for line in value.splitlines():
+ parts = line.split(":", 2)
+ if len(parts) != 3 or parts[0] != "0" or parts[1]:
+ continue
+ path_parts = tuple(part for part in parts[2].split("/") if part)
+ if any(part in {".", ".."} for part in path_parts):
+ return None
+ return path_parts
+ return None
+
+
+def _relative_cgroup_v1_memory_path(value: str) -> tuple[str, ...] | None:
+ for line in value.splitlines():
+ parts = line.split(":", 2)
+ if len(parts) != 3:
+ continue
+ controllers = {item.strip() for item in parts[1].split(",") if item.strip()}
+ if "memory" not in controllers:
+ continue
+ path_parts = tuple(part for part in parts[2].split("/") if part)
+ if any(part in {".", ".."} for part in path_parts):
+ return None
+ return ("memory",) + path_parts
+ return None
+
+
+_V1_MEMORY_FILES = {
+ "memory.current": "memory.usage_in_bytes",
+ "memory.peak": "memory.max_usage_in_bytes",
+ "memory.max": "memory.limit_in_bytes",
+ "memory.swap.current": "memory.memsw.usage_in_bytes",
+ "memory.swap.max": "memory.memsw.limit_in_bytes",
+}
+
+
+class ResourceSampler:
+ def __init__(
+ self,
+ *,
+ sampleIntervalSeconds: float = 0.1,
+ rootPid: int | None = None,
+ procRoot: str | Path = "/proc",
+ cgroupRoot: str | Path = "/sys/fs/cgroup",
+ cgroupPath: str | Path | None = None,
+ ephemeralDiskPath: str | Path | None = "/tmp",
+ readText: _ReadText | None = None,
+ writeText: _WriteText | None = None,
+ listPids: _ListPids | None = None,
+ affinityReader: _AffinityReader | None = None,
+ diskUsageReader: _DiskUsageReader | None = None,
+ ) -> None:
+ if isinstance(sampleIntervalSeconds, bool):
+ raise ValueError("sampleIntervalSeconds must be a positive finite number")
+ interval = float(sampleIntervalSeconds)
+ if not math.isfinite(interval) or interval <= 0:
+ raise ValueError("sampleIntervalSeconds must be a positive finite number")
+ selected_pid = os.getpid() if rootPid is None else rootPid
+ if isinstance(selected_pid, bool) or selected_pid <= 0:
+ raise ValueError("rootPid must be a positive integer")
+
+ self.sampleIntervalSeconds = interval
+ self.rootPid = selected_pid
+ self.procRoot = Path(procRoot)
+ self.cgroupRoot = Path(cgroupRoot)
+ self._explicitCgroupPath = None if cgroupPath is None else Path(cgroupPath)
+ self.ephemeralDiskPath = (
+ None if ephemeralDiskPath is None else Path(ephemeralDiskPath)
+ )
+ self._readText = _default_read_text if readText is None else readText
+ self._writeText = _default_write_text if writeText is None else writeText
+ self._listPids = _default_list_pids if listPids is None else listPids
+ self._affinityReader = (
+ _default_affinity_reader if affinityReader is None else affinityReader
+ )
+ self._diskUsageReader = (
+ _default_disk_usage_reader if diskUsageReader is None else diskUsageReader
+ )
+ self._usesDefaultTextIo = readText is None and writeText is None
+
+ self._stateLock = threading.Lock()
+ self._lifecycleLock = threading.Lock()
+ self._running = False
+ self._generation = 0
+ self._stopEvent: threading.Event | None = None
+ self._thread: threading.Thread | None = None
+ self._peakHandle: TextIOBase | None = None
+ self._reset_values()
+
+ def _reset_values(self) -> None:
+ self._result: ResourceMeasurement | None = None
+ self._sampleCount = 0
+ self._samplingErrorCount = 0
+ self._cgroupPath: Path | None = None
+ self._cgroupVersion: Literal[1, 2] | None = None
+ self._processTreeBaselineRssBytes: int | None = None
+ self._processTreePeakRssBytes: int | None = None
+ self._cgroupMemoryCurrentBaselineBytes: int | None = None
+ self._cgroupMemoryCurrentPeakBytes: int | None = None
+ self._memorySwapCurrentBeforeBytes: int | None = None
+ self._memorySwapCurrentPeakBytes: int | None = None
+ self._memoryEventsBefore: dict[str, int] | None = None
+ self._memoryMaxBytes: LimitValue = None
+ self._memorySwapMaxBytes: LimitValue = None
+ self._cgroupMemoryPeakInitialBytes: int | None = None
+ self._cgroupMemoryPeakScope: PeakScope = "unavailable"
+ self._cpuSnapshot = _CpuSnapshot()
+ self._ephemeralDiskBefore: DiskUsage | None = None
+ self._ephemeralDiskPeak: DiskUsage | None = None
+
+ @property
+ def isRunning(self) -> bool:
+ with self._stateLock:
+ return self._running
+
+ @property
+ def sampleCount(self) -> int:
+ with self._stateLock:
+ return self._sampleCount
+
+ @property
+ def result(self) -> ResourceMeasurement | None:
+ with self._stateLock:
+ return self._result
+
+ def _note_error(self) -> None:
+ with self._stateLock:
+ self._samplingErrorCount += 1
+
+ def _cgroup_file_name(self, name: str) -> str:
+ if self._cgroupVersion == 1:
+ return _V1_MEMORY_FILES.get(name, name)
+ return name
+
+ def _resolve_cgroup_path(self) -> Path | None:
+ if self._explicitCgroupPath is not None:
+ # Explicit paths are treated as cgroup v2 unless v1 memory files exist.
+ if (
+ _optional_read(
+ self._explicitCgroupPath / "memory.usage_in_bytes",
+ self._readText,
+ )
+ is not None
+ ):
+ self._cgroupVersion = 1
+ else:
+ self._cgroupVersion = 2
+ return self._explicitCgroupPath
+ candidates = (
+ self.procRoot / str(self.rootPid) / "cgroup",
+ self.procRoot / "self" / "cgroup",
+ )
+ for candidate in candidates:
+ value = _optional_read(candidate, self._readText)
+ if value is None:
+ continue
+ relative = _relative_cgroup_path(value)
+ if relative is not None:
+ path = self.cgroupRoot.joinpath(*relative)
+ if (
+ _optional_read(path / "memory.current", self._readText) is not None
+ or _optional_read(path / "cgroup.controllers", self._readText)
+ is not None
+ ):
+ self._cgroupVersion = 2
+ return path
+ relative_v1 = _relative_cgroup_v1_memory_path(value)
+ if relative_v1 is not None:
+ path = self.cgroupRoot.joinpath(*relative_v1)
+ if (
+ _optional_read(path / "memory.usage_in_bytes", self._readText)
+ is not None
+ ):
+ self._cgroupVersion = 1
+ return path
+ flat = self.cgroupRoot / "memory"
+ if (
+ _optional_read(flat / "memory.usage_in_bytes", self._readText)
+ is not None
+ ):
+ self._cgroupVersion = 1
+ return flat
+ probes_v2 = ("cgroup.controllers", "memory.current", "cpu.max")
+ if any(
+ _optional_read(self.cgroupRoot / name, self._readText) is not None
+ for name in probes_v2
+ ):
+ self._cgroupVersion = 2
+ return self.cgroupRoot
+ if (
+ _optional_read(
+ self.cgroupRoot / "memory" / "memory.usage_in_bytes",
+ self._readText,
+ )
+ is not None
+ ):
+ self._cgroupVersion = 1
+ return self.cgroupRoot / "memory"
+ return None
+
+ def _read_cgroup_int(self, name: str) -> int | None:
+ if self._cgroupPath is None:
+ return None
+ return _nonnegative_int(
+ _optional_read(
+ self._cgroupPath / self._cgroup_file_name(name),
+ self._readText,
+ )
+ )
+
+ def _read_cgroup_limit(self, name: str) -> LimitValue:
+ if self._cgroupPath is None:
+ return None
+ return _limit_value(
+ _optional_read(
+ self._cgroupPath / self._cgroup_file_name(name),
+ self._readText,
+ )
+ )
+
+ def _read_memory_events(self) -> dict[str, int] | None:
+ if self._cgroupPath is None or self._cgroupVersion == 1:
+ return None
+ value = _optional_read(self._cgroupPath / "memory.events", self._readText)
+ return None if value is None else parse_memory_events(value)
+
+ def _read_disk_usage(self) -> DiskUsage | None:
+ if self.ephemeralDiskPath is None:
+ return None
+ try:
+ value = self._diskUsageReader(self.ephemeralDiskPath)
+ except Exception:
+ return None
+ return _coerce_disk_usage(value)
+
+ def _read_cpu_snapshot(self) -> _CpuSnapshot:
+ quota: LimitValue = None
+ period: int | None = None
+ if self._cgroupPath is not None:
+ value = _optional_read(self._cgroupPath / "cpu.max", self._readText)
+ if value is not None:
+ parts = value.split()
+ if len(parts) >= 2:
+ quota = _limit_value(parts[0])
+ period = _nonnegative_int(parts[1])
+ if period == 0:
+ period = None
+ quota_cores = (
+ quota / period if isinstance(quota, int) and period is not None else None
+ )
+
+ affinity: tuple[int, ...] | None = None
+ source: str | None = None
+ try:
+ values = self._affinityReader(self.rootPid)
+ affinity = tuple(sorted({int(value) for value in values}))
+ source = "schedGetaffinity"
+ except Exception:
+ pass
+ if affinity is None and self._cgroupPath is not None:
+ for name in ("cpuset.cpus.effective", "cpuset.cpus"):
+ affinity = _parse_cpuset(
+ _optional_read(self._cgroupPath / name, self._readText)
+ )
+ if affinity is not None:
+ source = "cgroupCpuset"
+ break
+ return _CpuSnapshot(
+ quotaMicros=quota,
+ periodMicros=period,
+ quotaCores=quota_cores,
+ affinityCpus=affinity,
+ affinitySource=source,
+ )
+
+ @staticmethod
+ def _verified_peak_reset(
+ initialPeak: int,
+ resetPeak: int | None,
+ currentBefore: int | None,
+ currentAfter: int | None,
+ ) -> bool:
+ if resetPeak is None or resetPeak >= initialPeak:
+ return False
+ if resetPeak == 0:
+ return True
+ current = currentAfter if currentAfter is not None else currentBefore
+ return current is None or resetPeak >= current
+
+ @staticmethod
+ def _read_peak_handle(handle: TextIOBase) -> int | None:
+ try:
+ handle.seek(0)
+ return _nonnegative_int(handle.read())
+ except Exception:
+ return None
+
+ @staticmethod
+ def _write_peak_handle(handle: TextIOBase, value: str) -> bool:
+ try:
+ handle.seek(0)
+ handle.write(value)
+ handle.flush()
+ try:
+ handle.truncate()
+ except OSError:
+ pass
+ except Exception:
+ return False
+ return True
+
+ def _peak_reset_values(self, current: int | None) -> tuple[str, ...]:
+ if current in (None, 0):
+ return ("0",)
+ return ("0", str(current))
+
+ def _prepare_cgroup_peak(self) -> tuple[PeakScope, int | None]:
+ if self._cgroupPath is None:
+ return "unavailable", None
+ path = self._cgroupPath / self._cgroup_file_name("memory.peak")
+ initial_peak = _nonnegative_int(_optional_read(path, self._readText))
+ if initial_peak is None:
+ return "unavailable", None
+ # cgroup v1 max_usage_in_bytes is usually not resettable mid-run.
+ if self._cgroupVersion == 1:
+ return "containerLifetime", initial_peak
+ current_before = self._read_cgroup_int("memory.current")
+
+ if self._usesDefaultTextIo:
+ try:
+ handle = path.open("r+", encoding="utf-8")
+ except Exception:
+ return "containerLifetime", initial_peak
+ handle_initial = self._read_peak_handle(handle)
+ if handle_initial is not None:
+ initial_peak = handle_initial
+ for reset_value in self._peak_reset_values(current_before):
+ if not self._write_peak_handle(handle, reset_value):
+ continue
+ current_after = self._read_cgroup_int("memory.current")
+ reset_peak = self._read_peak_handle(handle)
+ if self._verified_peak_reset(
+ initial_peak,
+ reset_peak,
+ current_before,
+ current_after,
+ ):
+ self._peakHandle = handle
+ return "operation", initial_peak
+ try:
+ handle.close()
+ except Exception:
+ pass
+ return "containerLifetime", initial_peak
+
+ for reset_value in self._peak_reset_values(current_before):
+ if not _optional_write(path, reset_value, self._writeText):
+ continue
+ current_after = self._read_cgroup_int("memory.current")
+ reset_peak = _nonnegative_int(_optional_read(path, self._readText))
+ if self._verified_peak_reset(
+ initial_peak,
+ reset_peak,
+ current_before,
+ current_after,
+ ):
+ return "operation", initial_peak
+ return "containerLifetime", initial_peak
+
+ def _read_final_cgroup_peak(self) -> int | None:
+ if self._cgroupMemoryPeakScope == "unavailable":
+ return None
+ peak_name = self._cgroup_file_name("memory.peak")
+ if self._cgroupMemoryPeakScope == "operation":
+ if self._peakHandle is not None:
+ return self._read_peak_handle(self._peakHandle)
+ if self._cgroupPath is None:
+ return None
+ return _nonnegative_int(
+ _optional_read(
+ self._cgroupPath / peak_name,
+ self._readText,
+ )
+ )
+ if self._cgroupPath is None:
+ return self._cgroupMemoryPeakInitialBytes
+ final_peak = _nonnegative_int(
+ _optional_read(self._cgroupPath / peak_name, self._readText)
+ )
+ if final_peak is None:
+ return self._cgroupMemoryPeakInitialBytes
+ if self._cgroupMemoryPeakInitialBytes is None:
+ return final_peak
+ return max(final_peak, self._cgroupMemoryPeakInitialBytes)
+
+ def _close_peak_handle(self) -> None:
+ handle, self._peakHandle = self._peakHandle, None
+ if handle is None:
+ return
+ try:
+ handle.close()
+ except Exception:
+ pass
+
+ def _collect_sample(self) -> _Sample:
+ process_rss = read_process_tree_rss_bytes(
+ self.rootPid,
+ procRoot=self.procRoot,
+ readText=self._readText,
+ listPids=self._listPids,
+ )
+ return _Sample(
+ processTreeRssBytes=process_rss,
+ cgroupMemoryCurrentBytes=self._read_cgroup_int("memory.current"),
+ memorySwapCurrentBytes=self._read_cgroup_int("memory.swap.current"),
+ ephemeralDisk=self._read_disk_usage(),
+ )
+
+ def _sample_and_record(
+ self,
+ *,
+ isBaseline: bool,
+ force: bool,
+ generation: int,
+ ) -> _Sample:
+ try:
+ sample = self._collect_sample()
+ except Exception:
+ self._note_error()
+ sample = _Sample(None, None, None, None)
+ with self._stateLock:
+ if generation != self._generation:
+ return sample
+ if not force and not self._running:
+ return sample
+ self._sampleCount += 1
+ if isBaseline:
+ self._processTreeBaselineRssBytes = sample.processTreeRssBytes
+ self._cgroupMemoryCurrentBaselineBytes = sample.cgroupMemoryCurrentBytes
+ self._memorySwapCurrentBeforeBytes = sample.memorySwapCurrentBytes
+ self._ephemeralDiskBefore = sample.ephemeralDisk
+ if sample.processTreeRssBytes is not None:
+ current = self._processTreePeakRssBytes
+ self._processTreePeakRssBytes = max(
+ sample.processTreeRssBytes,
+ current if current is not None else sample.processTreeRssBytes,
+ )
+ if sample.cgroupMemoryCurrentBytes is not None:
+ current = self._cgroupMemoryCurrentPeakBytes
+ self._cgroupMemoryCurrentPeakBytes = max(
+ sample.cgroupMemoryCurrentBytes,
+ (
+ current
+ if current is not None
+ else sample.cgroupMemoryCurrentBytes
+ ),
+ )
+ if sample.memorySwapCurrentBytes is not None:
+ current = self._memorySwapCurrentPeakBytes
+ self._memorySwapCurrentPeakBytes = max(
+ sample.memorySwapCurrentBytes,
+ (current if current is not None else sample.memorySwapCurrentBytes),
+ )
+ if sample.ephemeralDisk is not None:
+ if (
+ self._ephemeralDiskPeak is None
+ or sample.ephemeralDisk.usedBytes
+ > self._ephemeralDiskPeak.usedBytes
+ ):
+ self._ephemeralDiskPeak = sample.ephemeralDisk
+ return sample
+
+ def _thread_main(
+ self,
+ stopEvent: threading.Event,
+ readyEvent: threading.Event,
+ generation: int,
+ ) -> None:
+ readyEvent.wait()
+ if stopEvent.is_set():
+ return
+ while not stopEvent.wait(self.sampleIntervalSeconds):
+ self._sample_and_record(
+ isBaseline=False,
+ force=False,
+ generation=generation,
+ )
+
+ def start(self) -> Self:
+ with self._lifecycleLock:
+ with self._stateLock:
+ if self._running:
+ return self
+ self._close_peak_handle()
+ with self._stateLock:
+ self._reset_values()
+ self._generation += 1
+ generation = self._generation
+ self._running = True
+ stop_event = threading.Event()
+ self._stopEvent = stop_event
+ self._thread = None
+
+ self._cgroupPath = self._resolve_cgroup_path()
+ self._memoryEventsBefore = self._read_memory_events()
+ self._memoryMaxBytes = self._read_cgroup_limit("memory.max")
+ self._memorySwapMaxBytes = self._read_cgroup_limit("memory.swap.max")
+ self._cpuSnapshot = self._read_cpu_snapshot()
+
+ ready_event = threading.Event()
+ thread = threading.Thread(
+ target=self._thread_main,
+ args=(stop_event, ready_event, generation),
+ name=f"resource-sampler-{self.rootPid}",
+ daemon=True,
+ )
+ try:
+ thread.start()
+ except Exception:
+ self._note_error()
+ else:
+ with self._stateLock:
+ self._thread = thread
+
+ peak_scope, initial_peak = self._prepare_cgroup_peak()
+ self._cgroupMemoryPeakScope = peak_scope
+ self._cgroupMemoryPeakInitialBytes = initial_peak
+ self._sample_and_record(
+ isBaseline=True,
+ force=True,
+ generation=generation,
+ )
+ ready_event.set()
+ return self
+
+ def sample(self) -> None:
+ with self._stateLock:
+ generation = self._generation
+ running = self._running
+ if not running:
+ return
+ self._sample_and_record(
+ isBaseline=False,
+ force=False,
+ generation=generation,
+ )
+
+ def _build_result(
+ self,
+ finalSample: _Sample,
+ memoryEventsAfter: dict[str, int] | None,
+ cgroupMemoryPeakBytes: int | None,
+ ) -> ResourceMeasurement:
+ with self._stateLock:
+ sample_count = self._sampleCount
+ error_count = self._samplingErrorCount
+ process_baseline = self._processTreeBaselineRssBytes
+ process_peak = self._processTreePeakRssBytes
+ cgroup_baseline = self._cgroupMemoryCurrentBaselineBytes
+ cgroup_current_peak = self._cgroupMemoryCurrentPeakBytes
+ swap_before = self._memorySwapCurrentBeforeBytes
+ swap_peak = self._memorySwapCurrentPeakBytes
+ events_before = (
+ None
+ if self._memoryEventsBefore is None
+ else dict(self._memoryEventsBefore)
+ )
+ disk_before = self._ephemeralDiskBefore
+ disk_peak = self._ephemeralDiskPeak
+ cpu = self._cpuSnapshot
+
+ operation_baseline = cgroup_baseline
+ operation_peak = cgroup_current_peak
+ operation_source: str | None = (
+ "cgroupMemoryCurrent" if cgroup_baseline is not None else None
+ )
+ if (
+ cgroup_baseline is not None
+ and self._cgroupMemoryPeakScope == "operation"
+ and cgroupMemoryPeakBytes is not None
+ and (operation_peak is None or cgroupMemoryPeakBytes >= operation_peak)
+ ):
+ operation_peak = cgroupMemoryPeakBytes
+ operation_source = "cgroupMemoryPeak"
+ if cgroup_baseline is None:
+ operation_baseline = process_baseline
+ operation_peak = process_peak
+ operation_source = (
+ "processTreeRss" if process_baseline is not None else None
+ )
+ incremental_peak = (
+ max(0, operation_peak - operation_baseline)
+ if operation_peak is not None and operation_baseline is not None
+ else None
+ )
+ process_incremental_peak = (
+ max(0, process_peak - process_baseline)
+ if process_peak is not None and process_baseline is not None
+ else None
+ )
+
+ events_after = None if memoryEventsAfter is None else dict(memoryEventsAfter)
+ affinity_count = None if cpu.affinityCpus is None else len(cpu.affinityCpus)
+ return ResourceMeasurement(
+ sampleCount=sample_count,
+ sampleIntervalSeconds=self.sampleIntervalSeconds,
+ operationBaselineBytes=operation_baseline,
+ operationPeakBytes=operation_peak,
+ operationIncrementalPeakBytes=incremental_peak,
+ operationPeakSource=operation_source,
+ processTreeRssBaselineBytes=process_baseline,
+ processTreeRssPeakBytes=process_peak,
+ processTreeRssIncrementalPeakBytes=process_incremental_peak,
+ processTreeRssAfterBytes=finalSample.processTreeRssBytes,
+ cgroupPath=(None if self._cgroupPath is None else str(self._cgroupPath)),
+ cgroupMemoryCurrentBaselineBytes=cgroup_baseline,
+ cgroupMemoryCurrentPeakBytes=cgroup_current_peak,
+ cgroupMemoryCurrentAfterBytes=(finalSample.cgroupMemoryCurrentBytes),
+ cgroupMemoryPeakBytes=cgroupMemoryPeakBytes,
+ cgroupMemoryPeakScope=self._cgroupMemoryPeakScope,
+ memoryMaxBytes=self._memoryMaxBytes,
+ memorySwapCurrentBeforeBytes=swap_before,
+ memorySwapCurrentAfterBytes=finalSample.memorySwapCurrentBytes,
+ memorySwapCurrentPeakBytes=swap_peak,
+ memorySwapMaxBytes=self._memorySwapMaxBytes,
+ memoryEventsBefore=events_before,
+ memoryEventsAfter=events_after,
+ memoryEventsDelta=_memory_event_delta(
+ events_before,
+ events_after,
+ ),
+ cpuQuotaMicros=cpu.quotaMicros,
+ cpuPeriodMicros=cpu.periodMicros,
+ cpuQuotaCores=cpu.quotaCores,
+ cpuAffinityCpus=cpu.affinityCpus,
+ cpuAffinityCount=affinity_count,
+ cpuAffinitySource=cpu.affinitySource,
+ ephemeralDiskPath=(
+ None if self.ephemeralDiskPath is None else str(self.ephemeralDiskPath)
+ ),
+ ephemeralDiskBefore=disk_before,
+ ephemeralDiskAfter=finalSample.ephemeralDisk,
+ ephemeralDiskPeak=disk_peak,
+ samplingErrorCount=error_count,
+ )
+
+ def stop(self) -> ResourceMeasurement:
+ with self._lifecycleLock:
+ with self._stateLock:
+ if not self._running:
+ if self._result is not None:
+ return self._result
+ result = ResourceMeasurement(
+ sampleCount=0,
+ sampleIntervalSeconds=self.sampleIntervalSeconds,
+ ephemeralDiskPath=(
+ None
+ if self.ephemeralDiskPath is None
+ else str(self.ephemeralDiskPath)
+ ),
+ )
+ self._result = result
+ return result
+ self._running = False
+ generation = self._generation
+ stop_event = self._stopEvent
+ thread = self._thread
+
+ if stop_event is not None:
+ stop_event.set()
+ if thread is not None and thread is not threading.current_thread():
+ try:
+ thread.join(
+ timeout=max(
+ 0.25,
+ min(2.0, self.sampleIntervalSeconds * 2),
+ )
+ )
+ except RuntimeError:
+ self._note_error()
+
+ final_sample = self._sample_and_record(
+ isBaseline=False,
+ force=True,
+ generation=generation,
+ )
+ memory_events_after = self._read_memory_events()
+ cgroup_peak = self._read_final_cgroup_peak()
+ self._close_peak_handle()
+ result = self._build_result(
+ final_sample,
+ memory_events_after,
+ cgroup_peak,
+ )
+ with self._stateLock:
+ self._result = result
+ self._stopEvent = None
+ self._thread = None
+ return result
+
+ def __enter__(self) -> Self:
+ return self.start()
+
+ def __exit__(
+ self,
+ excType: type[BaseException] | None,
+ exc: BaseException | None,
+ traceback: TracebackType | None,
+ ) -> bool:
+ try:
+ self.stop()
+ except Exception:
+ pass
+ return False
+
+
+class StageTimer:
+ def __init__(self, *, clock: Callable[[], float] = time.perf_counter) -> None:
+ self._clock = clock
+ self._active = False
+ self._wholeStarted: float | None = None
+ self._wholeElapsed: float | None = None
+ self._inputSetupElapsed = 0.0
+ self._operationElapsed = 0.0
+ self._validationPersistenceElapsed = 0.0
+ self._hasInputSetup = False
+ self._hasOperation = False
+ self._hasValidationPersistence = False
+
+ def __enter__(self) -> Self:
+ if self._active:
+ raise RuntimeError("StageTimer is already active")
+ self._active = True
+ self._wholeStarted = self._clock()
+ self._wholeElapsed = None
+ self._inputSetupElapsed = 0.0
+ self._operationElapsed = 0.0
+ self._validationPersistenceElapsed = 0.0
+ self._hasInputSetup = False
+ self._hasOperation = False
+ self._hasValidationPersistence = False
+ return self
+
+ def __exit__(
+ self,
+ excType: type[BaseException] | None,
+ exc: BaseException | None,
+ traceback: TracebackType | None,
+ ) -> bool:
+ if self._wholeStarted is not None:
+ self._wholeElapsed = max(0.0, self._clock() - self._wholeStarted)
+ self._wholeStarted = None
+ self._active = False
+ return False
+
+ @contextmanager
+ def inputSetup(self) -> Iterator[None]:
+ started = self._clock()
+ try:
+ yield
+ finally:
+ self._inputSetupElapsed += max(0.0, self._clock() - started)
+ self._hasInputSetup = True
+
+ @contextmanager
+ def operation(self) -> Iterator[None]:
+ started = self._clock()
+ try:
+ yield
+ finally:
+ self._operationElapsed += max(0.0, self._clock() - started)
+ self._hasOperation = True
+
+ @contextmanager
+ def validationPersistence(self) -> Iterator[None]:
+ started = self._clock()
+ try:
+ yield
+ finally:
+ self._validationPersistenceElapsed += max(
+ 0.0,
+ self._clock() - started,
+ )
+ self._hasValidationPersistence = True
+
+ @property
+ def result(self) -> StageTimings:
+ return StageTimings(
+ inputSetupSeconds=(
+ self._inputSetupElapsed if self._hasInputSetup else None
+ ),
+ measuredOperationSeconds=(
+ self._operationElapsed if self._hasOperation else None
+ ),
+ validationPersistenceSeconds=(
+ self._validationPersistenceElapsed
+ if self._hasValidationPersistence
+ else None
+ ),
+ wholeFunctionSeconds=self._wholeElapsed,
+ )
+
+
+__all__ = [
+ "DiskUsage",
+ "LimitValue",
+ "PeakScope",
+ "ResourceMeasurement",
+ "ResourceSampler",
+ "StageTimer",
+ "StageTimings",
+ "parse_memory_events",
+ "read_process_tree_rss_bytes",
+]
diff --git a/profiling/modal_app.py b/profiling/modal_app.py
new file mode 100644
index 00000000..fbf390d1
--- /dev/null
+++ b/profiling/modal_app.py
@@ -0,0 +1,494 @@
+"""Modal entrypoints for one-shot Scarf profiling.
+
+Deploy once (you run this), then trigger jobs that keep running if your laptop
+disconnects:
+
+ modal deploy --env scarf_profiling -m profiling.modal_app
+
+ modal run --env scarf_profiling -m profiling.modal_app -- prepare-fixture --config profiling/config.toml --sizes 10000
+ modal run --env scarf_profiling -m profiling.modal_app -- prepare --config profiling/config.toml
+ modal run --env scarf_profiling -m profiling.modal_app -- run --config profiling/config.toml --size 10000 --stage createStore
+ modal run --env scarf_profiling -m profiling.modal_app -- run-all --config profiling/config.toml --sizes 10000
+
+prepare / run / run-all spawn on the deployed app and return immediately.
+run-all fans out one size pipeline per container (stages stay sequential).
+Watch progress with: modal app logs scarf-profiling --env scarf_profiling
+"""
+
+import argparse
+import os
+from pathlib import Path
+from typing import Any
+
+import modal
+
+from profiling.config import (
+ STAGE_ORDER,
+ ProfilingConfig,
+ StageName,
+ load_profiling_config,
+)
+from profiling.datasets import (
+ SOURCE_SPEC,
+ download_source,
+ prepare_fixture_datasets,
+ prepare_local_datasets,
+ sha256_file,
+)
+from profiling.modal_image import COMMON_FUNCTION_OPTIONS, app
+from profiling.modal_resources import (
+ BASE_EPHEMERAL_DISK_MB,
+ modal_function_options,
+ validate_modal_environment,
+)
+from profiling.r2 import download_file, object_exists, object_size, upload_file
+from profiling.results import result_exists, write_result
+from profiling.stages import run_stage
+
+_WORK = Path("/tmp/scarf-profiling")
+
+
+@app.function(
+ **COMMON_FUNCTION_OPTIONS,
+ timeout=86_400,
+ memory=196_608,
+ cpu=8.0,
+ ephemeral_disk=BASE_EPHEMERAL_DISK_MB,
+)
+def prepare_datasets(configDict: dict[str, Any]) -> dict[str, Any]:
+ config = ProfilingConfig.model_validate(configDict)
+ os.environ.setdefault("R2_ENDPOINT", config.r2EndpointUrl)
+ work = _WORK / "prepare"
+ work.mkdir(parents=True, exist_ok=True)
+ source_path = work / "source.h5ad"
+ source_uri = config.sourceUri()
+ source_origin = "local-cache"
+ if not source_path.is_file():
+ if object_exists(source_uri):
+ download_file(source_uri, source_path)
+ source_origin = "r2-cache"
+ else:
+ download_source(
+ source_path,
+ url=SOURCE_SPEC.url,
+ expectedBytes=SOURCE_SPEC.sourceBytes,
+ )
+ upload_file(source_path, source_uri)
+ source_origin = "cellxgene+r2-upload"
+
+ uploaded: list[dict[str, Any]] = []
+ skipped: list[dict[str, Any]] = []
+ # Fixture uploads are tiny; real nested samples are much larger.
+ minimum_real_bytes = 5_000_000
+
+ pending_sizes: list[int] = []
+ for n_rows in config.targetSizes:
+ uri = config.datasetUri(n_rows)
+ existing = object_size(uri)
+ if existing is not None and existing >= minimum_real_bytes:
+ skipped.append(
+ {
+ "nRows": n_rows,
+ "uri": uri,
+ "fileBytes": existing,
+ "status": "skipped-existing",
+ }
+ )
+ continue
+ pending_sizes.append(n_rows)
+
+ def _upload_artifact(artifact: Any) -> None:
+ uri = config.datasetUri(artifact.targetRows)
+ upload_file(artifact.localPath, uri)
+ uploaded.append(
+ {
+ "nRows": artifact.targetRows,
+ "uri": uri,
+ "fileBytes": artifact.fileBytes,
+ "nnz": artifact.nnz,
+ "status": "uploaded",
+ }
+ )
+
+ source_sha256 = None
+ if pending_sizes:
+ prepared = prepare_local_datasets(
+ source_path,
+ work / "subsets",
+ targetRows=tuple(pending_sizes),
+ seed=config.samplingSeed,
+ spec=SOURCE_SPEC,
+ onArtifact=_upload_artifact,
+ )
+ source_sha256 = prepared.sourceSha256
+ elif source_path.is_file():
+ source_sha256 = sha256_file(source_path)
+
+ return {
+ "uploaded": uploaded,
+ "skipped": skipped,
+ "sourceSha256": source_sha256,
+ "sourceOrigin": source_origin,
+ "sourceUri": source_uri,
+ }
+
+
+@app.function(
+ **COMMON_FUNCTION_OPTIONS,
+ timeout=3_600,
+ memory=8_192,
+ cpu=2.0,
+ ephemeral_disk=BASE_EPHEMERAL_DISK_MB,
+)
+def prepare_fixture_datasets_job(
+ configDict: dict[str, Any],
+ sizes: list[int] | None = None,
+ nColumns: int = 500,
+) -> dict[str, Any]:
+ """Upload tiny synthetic H5ADs so stage jobs can be tested without Cellxgene."""
+ config = ProfilingConfig.model_validate(configDict)
+ os.environ.setdefault("R2_ENDPOINT", config.r2EndpointUrl)
+ selected = tuple(sizes) if sizes else (10_000,)
+ for size in selected:
+ if size not in config.targetSizes:
+ raise ValueError(
+ f"fixture size {size} is not in config.targetSizes; "
+ "add it to config or choose an existing size"
+ )
+ work = _WORK / "fixture"
+ if work.exists():
+ for path in sorted(work.rglob("*"), reverse=True):
+ if path.is_file():
+ path.unlink()
+ elif path.is_dir():
+ path.rmdir()
+ work.mkdir(parents=True, exist_ok=True)
+ uploaded: list[dict[str, Any]] = []
+
+ def _upload_artifact(artifact: Any) -> None:
+ uri = config.datasetUri(artifact.targetRows)
+ upload_file(artifact.localPath, uri)
+ uploaded.append(
+ {
+ "nRows": artifact.targetRows,
+ "uri": uri,
+ "fileBytes": artifact.fileBytes,
+ "nnz": artifact.nnz,
+ "nColumns": artifact.nColumns,
+ }
+ )
+
+ prepare_fixture_datasets(
+ work,
+ targetRows=selected,
+ nColumns=nColumns,
+ seed=config.samplingSeed,
+ onArtifact=_upload_artifact,
+ )
+ return {"uploaded": uploaded, "kind": "fixture"}
+
+
+@app.function(
+ **COMMON_FUNCTION_OPTIONS,
+ timeout=86_400,
+ memory=65_536,
+ cpu=8.0,
+ ephemeral_disk=BASE_EPHEMERAL_DISK_MB,
+)
+def run_stage_job(
+ configDict: dict[str, Any],
+ nRows: int,
+ stage: StageName,
+) -> dict[str, Any]:
+ config = ProfilingConfig.model_validate(configDict)
+ resources = config.resourcesFor(stage)
+ os.environ.setdefault("R2_ENDPOINT", config.r2EndpointUrl)
+
+ work = _WORK / f"{nRows}-{stage}"
+ if work.exists():
+ for path in sorted(work.rglob("*"), reverse=True):
+ if path.is_file():
+ path.unlink()
+ elif path.is_dir():
+ path.rmdir()
+ work.mkdir(parents=True, exist_ok=True)
+
+ local_h5ad: Path | None = None
+ if stage == "createStore":
+ local_h5ad = work / f"{nRows}.h5ad"
+ download_file(config.datasetUri(nRows), local_h5ad)
+
+ result = run_stage(
+ stage,
+ nRows=nRows,
+ storeUri=config.storeUri(nRows),
+ workflow=config.workflow,
+ resources=resources,
+ localH5adPath=local_h5ad,
+ storageLayout=config.storageLayout,
+ )
+ write_result(config, result)
+ return result.to_json()
+
+
+@app.function(
+ **COMMON_FUNCTION_OPTIONS,
+ timeout=86_400,
+ memory=2048,
+ cpu=1.0,
+ ephemeral_disk=BASE_EPHEMERAL_DISK_MB,
+)
+def run_size_jobs(
+ configDict: dict[str, Any],
+ nRows: int,
+ stages: list[StageName] | None = None,
+) -> dict[str, Any]:
+ """Run stages sequentially for one size (skip if result JSON exists)."""
+ config = ProfilingConfig.model_validate(configDict)
+ os.environ.setdefault("R2_ENDPOINT", config.r2EndpointUrl)
+ if nRows not in config.targetSizes:
+ raise ValueError(f"size {nRows} is not in config.targetSizes")
+ selected_stages = tuple(stages) if stages else STAGE_ORDER
+ parallel_sizes = max(1, len(config.targetSizes))
+ outcomes: list[dict[str, Any]] = []
+
+ for stage in selected_stages:
+ if result_exists(config, nRows, stage):
+ outcomes.append(
+ {
+ "nRows": nRows,
+ "stage": stage,
+ "status": "skipped",
+ "resultUri": config.resultUri(nRows, stage),
+ }
+ )
+ continue
+ resources = config.resourcesFor(stage)
+ options = modal_function_options(
+ config,
+ resources,
+ maxContainers=parallel_sizes,
+ )
+ result = run_stage_job.with_options(**options).remote(
+ configDict,
+ nRows,
+ stage,
+ )
+ outcomes.append(result)
+ if result.get("status") == "error":
+ return {
+ "nRows": nRows,
+ "stopped": True,
+ "failed": result,
+ "outcomes": outcomes,
+ }
+
+ return {"nRows": nRows, "stopped": False, "outcomes": outcomes}
+
+
+@app.function(
+ **COMMON_FUNCTION_OPTIONS,
+ timeout=86_400,
+ memory=2048,
+ cpu=1.0,
+ ephemeral_disk=BASE_EPHEMERAL_DISK_MB,
+)
+def run_all_jobs(
+ configDict: dict[str, Any],
+ sizes: list[int] | None = None,
+ stages: list[StageName] | None = None,
+) -> dict[str, Any]:
+ """Run sizes in parallel; stages within each size stay sequential."""
+ config = ProfilingConfig.model_validate(configDict)
+ os.environ.setdefault("R2_ENDPOINT", config.r2EndpointUrl)
+ selected_sizes = tuple(sizes) if sizes else config.targetSizes
+ selected_stages = tuple(stages) if stages else STAGE_ORDER
+ for n_rows in selected_sizes:
+ if n_rows not in config.targetSizes:
+ raise ValueError(f"size {n_rows} is not in config.targetSizes")
+
+ parallel_sizes = max(1, len(selected_sizes))
+ orchestrator_options = modal_function_options(
+ config,
+ config.resourcesFor("reopenStore"),
+ maxContainers=parallel_sizes,
+ )
+ orchestrator_options["memory"] = (2048, 4096)
+ orchestrator_options["cpu"] = (1.0, 1.0)
+ orchestrator_options["timeout"] = 86_400
+
+ stage_list = list(selected_stages)
+ handles = [
+ run_size_jobs.with_options(**orchestrator_options).spawn(
+ configDict,
+ n_rows,
+ stage_list,
+ )
+ for n_rows in selected_sizes
+ ]
+ size_results = [handle.get() for handle in handles]
+ failed = [item for item in size_results if item.get("stopped")]
+ return {
+ "stopped": bool(failed),
+ "failed": failed[0] if failed else None,
+ "sizes": size_results,
+ }
+
+
+@app.function(
+ **COMMON_FUNCTION_OPTIONS,
+ timeout=300,
+ memory=1024,
+ cpu=1.0,
+ ephemeral_disk=BASE_EPHEMERAL_DISK_MB,
+)
+def smoke_check(configDict: dict[str, Any]) -> dict[str, Any]:
+ config = ProfilingConfig.model_validate(configDict)
+ os.environ.setdefault("R2_ENDPOINT", config.r2EndpointUrl)
+ probe = f"{config.resultsUri.rstrip('/')}/smoke/ok.json"
+ from profiling.r2 import put_json
+
+ put_json(probe, {"ok": True})
+ return {"probeUri": probe, "exists": object_exists(probe)}
+
+
+def _load_config(path: str) -> ProfilingConfig:
+ config = load_profiling_config(path)
+ validate_modal_environment(config)
+ return config
+
+
+def _deployed_function(config: ProfilingConfig, name: str) -> modal.Function:
+ try:
+ return modal.Function.from_name(
+ config.modalAppName,
+ name,
+ environment_name=config.modalEnvironmentName,
+ )
+ except Exception as exc:
+ raise SystemExit(
+ f"Could not find deployed function {config.modalAppName}/{name}. "
+ "Deploy first with:\n"
+ f" modal deploy --env {config.modalEnvironmentName} -m profiling.modal_app\n"
+ f"Original error: {exc}"
+ ) from exc
+
+
+def _print_spawned(label: str, call: Any) -> None:
+ call_id = (
+ getattr(call, "object_id", None) or getattr(call, "call_id", None) or str(call)
+ )
+ print(f"spawned {label}: {call_id}")
+ print("disconnect is safe; watch with:")
+ print(" modal app logs scarf-profiling --env scarf_profiling")
+
+
+@app.local_entrypoint()
+def main(*arg_list: str) -> None:
+ parser = argparse.ArgumentParser(prog="profiling.modal_app")
+ sub = parser.add_subparsers(dest="command", required=True)
+
+ smoke_parser = sub.add_parser("smoke")
+ smoke_parser.add_argument("--config", required=True)
+
+ prepare_parser = sub.add_parser("prepare")
+ prepare_parser.add_argument("--config", required=True)
+
+ fixture_parser = sub.add_parser("prepare-fixture")
+ fixture_parser.add_argument("--config", required=True)
+ fixture_parser.add_argument("--sizes", nargs="*", type=int, default=[10_000])
+ fixture_parser.add_argument("--n-columns", type=int, default=500)
+
+ run_parser = sub.add_parser("run")
+ run_parser.add_argument("--config", required=True)
+ run_parser.add_argument("--size", type=int, required=True)
+ run_parser.add_argument("--stage", choices=STAGE_ORDER, required=True)
+
+ all_parser = sub.add_parser("run-all")
+ all_parser.add_argument("--config", required=True)
+ all_parser.add_argument("--sizes", nargs="*", type=int, default=None)
+ all_parser.add_argument("--stages", nargs="*", choices=STAGE_ORDER, default=None)
+
+ args = parser.parse_args(list(arg_list))
+ config = _load_config(args.config)
+ payload = config.model_dump(mode="python")
+
+ if args.command == "smoke":
+ smoke_options = modal_function_options(
+ config,
+ config.resourcesFor("reopenStore"),
+ )
+ print(smoke_check.with_options(**smoke_options).remote(payload))
+ return
+
+ if args.command == "prepare":
+ prepare_options = modal_function_options(
+ config,
+ config.prepareResources,
+ )
+ call = (
+ _deployed_function(config, "prepare_datasets")
+ .with_options(**prepare_options)
+ .spawn(payload)
+ )
+ _print_spawned("prepare_datasets", call)
+ return
+
+ if args.command == "prepare-fixture":
+ sizes = list(args.sizes) if args.sizes else [10_000]
+ for size in sizes:
+ if size not in config.targetSizes:
+ raise SystemExit(f"size {size} is not in config.targetSizes")
+ fixture_options = modal_function_options(
+ config,
+ config.resourcesFor("reopenStore"),
+ )
+ call = (
+ _deployed_function(config, "prepare_fixture_datasets_job")
+ .with_options(**fixture_options)
+ .spawn(payload, sizes, args.n_columns)
+ )
+ _print_spawned("prepare_fixture_datasets_job", call)
+ return
+
+ if args.command == "run":
+ if args.size not in config.targetSizes:
+ raise SystemExit(f"size {args.size} is not in config.targetSizes")
+ if result_exists(config, args.size, args.stage):
+ print(
+ {
+ "nRows": args.size,
+ "stage": args.stage,
+ "status": "skipped",
+ "resultUri": config.resultUri(args.size, args.stage),
+ }
+ )
+ return
+ resources = config.resourcesFor(args.stage)
+ options = modal_function_options(config, resources)
+ call = (
+ _deployed_function(config, "run_stage_job")
+ .with_options(**options)
+ .spawn(payload, args.size, args.stage)
+ )
+ _print_spawned(f"run_stage_job {args.size}/{args.stage}", call)
+ return
+
+ if args.command == "run-all":
+ sizes = list(args.sizes) if args.sizes else None
+ stages = list(args.stages) if args.stages else None
+ if sizes:
+ for size in sizes:
+ if size not in config.targetSizes:
+ raise SystemExit(f"size {size} is not in config.targetSizes")
+ coordinator_options = modal_function_options(
+ config,
+ config.resourcesFor("reopenStore"),
+ )
+ call = (
+ _deployed_function(config, "run_all_jobs")
+ .with_options(**coordinator_options)
+ .spawn(payload, sizes, stages)
+ )
+ _print_spawned("run_all_jobs", call)
+ return
diff --git a/profiling/modal_image.py b/profiling/modal_image.py
new file mode 100644
index 00000000..d43f7cec
--- /dev/null
+++ b/profiling/modal_image.py
@@ -0,0 +1,32 @@
+import modal
+
+
+MODAL_APP_NAME = "scarf-profiling"
+
+app = modal.App(MODAL_APP_NAME)
+
+image = (
+ modal.Image.debian_slim(python_version="3.12")
+ .apt_install(
+ "build-essential",
+ "git",
+ "libfftw3-dev",
+ "libmetis-dev",
+ "libtbb-dev",
+ )
+ .uv_sync(
+ groups=["profiling"],
+ frozen=True,
+ extra_options="--no-default-groups",
+ )
+ .add_local_python_source("scarf", "profiling", copy=True)
+ .add_local_file("uv.lock", "/root/uv.lock", copy=True)
+ .add_local_file("VERSION", "/root/VERSION", copy=True)
+)
+
+
+COMMON_FUNCTION_OPTIONS = {
+ "image": image,
+ "retries": 0,
+ "single_use_containers": True,
+}
diff --git a/profiling/modal_resources.py b/profiling/modal_resources.py
new file mode 100644
index 00000000..91fbe352
--- /dev/null
+++ b/profiling/modal_resources.py
@@ -0,0 +1,61 @@
+from typing import Any
+
+import modal
+
+from profiling.config import PrepareResources, ProfilingConfig, StageResources
+
+# Modal currently requires ephemeral disk in this range (MiB).
+MIN_EPHEMERAL_DISK_MB = 524_288
+MAX_EPHEMERAL_DISK_MB = 3_145_728
+BASE_EPHEMERAL_DISK_MB = MIN_EPHEMERAL_DISK_MB
+
+
+def resolve_ephemeral_disk_mb(requestedMb: int) -> int:
+ if requestedMb > MAX_EPHEMERAL_DISK_MB:
+ raise ValueError(
+ "Requested ephemeral disk is "
+ f"{requestedMb} MiB, but Modal allows at most "
+ f"{MAX_EPHEMERAL_DISK_MB} MiB"
+ )
+ if requestedMb < MIN_EPHEMERAL_DISK_MB:
+ return MIN_EPHEMERAL_DISK_MB
+ return requestedMb
+
+
+def validate_modal_environment(config: ProfilingConfig) -> None:
+ if config.modalEnvironmentName != "scarf_profiling":
+ raise ValueError("Modal environment must be scarf_profiling")
+ environment = modal.Environment.from_name(
+ config.modalEnvironmentName,
+ create_if_missing=False,
+ )
+ environment.hydrate()
+
+
+def modal_function_options(
+ config: ProfilingConfig,
+ resources: StageResources | PrepareResources,
+ *,
+ maxContainers: int = 1,
+) -> dict[str, Any]:
+ if maxContainers <= 0:
+ raise ValueError("maxContainers must be positive")
+ secret = modal.Secret.from_name(
+ config.modalSecretName,
+ environment_name=config.modalEnvironmentName,
+ required_keys=["R2_ACCESS_KEY_ID", "R2_SECRET_ACCESS_KEY"],
+ )
+ # ephemeral_disk is fixed on @app.function; with_options does not accept it.
+ _ = resolve_ephemeral_disk_mb(resources.ephemeralDiskMb)
+ return {
+ "cpu": (resources.modalCpuRequest, resources.modalCpuLimit),
+ "memory": (resources.modalMemoryRequestMb, resources.modalMemoryLimitMb),
+ "env": {"R2_ENDPOINT": config.r2EndpointUrl},
+ "secrets": [secret],
+ "retries": 0,
+ "max_containers": maxContainers,
+ "buffer_containers": 0,
+ "timeout": resources.timeoutSeconds,
+ "cloud": config.modalCloud,
+ "region": config.modalRegion,
+ }
diff --git a/profiling/r2.py b/profiling/r2.py
new file mode 100644
index 00000000..481b3089
--- /dev/null
+++ b/profiling/r2.py
@@ -0,0 +1,194 @@
+import json
+import os
+import time
+from dataclasses import dataclass
+from datetime import timedelta
+from pathlib import Path
+from typing import Any
+from urllib.parse import urlsplit
+
+from obstore.store import from_url
+
+_ENV_PATH = Path(__file__).resolve().parent / ".env"
+_DEFAULT_TRANSFER_CHUNK_BYTES = 16 * 1024 * 1024
+_CREDENTIAL_KEYS = (
+ "R2_ENDPOINT",
+ "R2_ACCESS_KEY_ID",
+ "R2_SECRET_ACCESS_KEY",
+)
+# Large objects (Cellxgene source ~46 GiB) exceed obstore's default 30s request timeout.
+_CLIENT_OPTIONS = {
+ "timeout": "12h",
+ "connect_timeout": "120s",
+ "read_timeout": "30m",
+}
+_RETRY_CONFIG = {
+ "max_retries": 20,
+ "retry_timeout": timedelta(minutes=30),
+}
+
+
+@dataclass(frozen=True, slots=True)
+class ObjectDownload:
+ fileBytes: int
+ eTag: str | None
+
+
+@dataclass(frozen=True, slots=True)
+class ObjectUpload:
+ fileBytes: int
+ eTag: str | None
+
+
+def _load_local_env(path: Path = _ENV_PATH) -> None:
+ if not path.is_file():
+ return
+ for line in path.read_text().splitlines():
+ line = line.strip()
+ if not line or line.startswith("#") or "=" not in line:
+ continue
+ key, value = line.split("=", 1)
+ key = key.strip()
+ value = value.strip().strip('"').strip("'")
+ if key and key not in os.environ:
+ os.environ[key] = value
+
+
+def storage_options(uri: str) -> dict[str, str] | None:
+ if not uri.startswith("s3://"):
+ return None
+ _load_local_env()
+ missing = [name for name in _CREDENTIAL_KEYS if not os.environ.get(name)]
+ if missing:
+ raise RuntimeError(f"Missing R2 environment settings: {', '.join(missing)}")
+ endpoint = os.environ["R2_ENDPOINT"]
+ access_key = os.environ["R2_ACCESS_KEY_ID"]
+ secret_key = os.environ["R2_SECRET_ACCESS_KEY"]
+ assert endpoint and access_key and secret_key
+ return {
+ "access_key_id": access_key,
+ "secret_access_key": secret_key,
+ "endpoint": endpoint.rstrip("/"),
+ }
+
+
+def open_r2_object(uri: str) -> tuple[Any, str]:
+ parsed = urlsplit(uri)
+ if parsed.scheme != "s3" or not parsed.netloc:
+ raise ValueError(f"Expected an s3:// object URI, got: {uri}")
+ key = parsed.path.lstrip("/")
+ if not key:
+ raise ValueError("R2 object URI must include an object key")
+ options = storage_options(uri)
+ assert options is not None
+ store = from_url(
+ f"s3://{parsed.netloc}",
+ client_options=_CLIENT_OPTIONS,
+ retry_config=_RETRY_CONFIG,
+ **options,
+ )
+ return store, key
+
+
+def join_uri(prefix: str, *parts: str) -> str:
+ base = prefix.rstrip("/")
+ suffix = "/".join(part.strip("/") for part in parts if part)
+ return f"{base}/{suffix}" if suffix else base
+
+
+def object_exists(uri: str) -> bool:
+ store, key = open_r2_object(uri)
+ try:
+ store.head(key)
+ except FileNotFoundError:
+ return False
+ return True
+
+
+def object_size(uri: str) -> int | None:
+ store, key = open_r2_object(uri)
+ try:
+ meta = store.head(key)
+ except FileNotFoundError:
+ return None
+ return int(meta["size"])
+
+
+def get_json(uri: str) -> dict[str, Any]:
+ store, key = open_r2_object(uri)
+ result = store.get(key)
+ body = bytes(result.bytes())
+ payload = json.loads(body.decode("utf-8"))
+ if not isinstance(payload, dict):
+ raise ValueError(f"Expected JSON object at {uri}")
+ return payload
+
+
+def put_json(uri: str, value: dict[str, Any]) -> None:
+ store, key = open_r2_object(uri)
+ body = (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode(
+ "utf-8"
+ )
+ store.put(key, body)
+
+
+def download_file(
+ uri: str,
+ destination: str | Path,
+ *,
+ chunkBytes: int = _DEFAULT_TRANSFER_CHUNK_BYTES,
+ maxAttempts: int = 8,
+) -> ObjectDownload:
+ """Download with ranged GETs so large objects can resume after timeouts."""
+ if chunkBytes <= 0:
+ raise ValueError("chunkBytes must be positive")
+ if maxAttempts <= 0:
+ raise ValueError("maxAttempts must be positive")
+
+ store, key = open_r2_object(uri)
+ destination_path = Path(destination)
+ destination_path.parent.mkdir(parents=True, exist_ok=True)
+ meta = store.head(key)
+ total = int(meta["size"])
+ part_path = destination_path.with_name(f".{destination_path.name}.part")
+ offset = part_path.stat().st_size if part_path.is_file() else 0
+ if offset > total:
+ part_path.unlink()
+ offset = 0
+
+ attempts = 0
+ while offset < total:
+ end = min(offset + chunkBytes, total)
+ try:
+ chunk = bytes(store.get_range(key, start=offset, end=end))
+ except Exception:
+ attempts += 1
+ if attempts >= maxAttempts:
+ raise
+ time.sleep(min(60.0, 2.0**attempts))
+ store, key = open_r2_object(uri)
+ continue
+ if not chunk:
+ raise RuntimeError(f"Empty range response for {uri} at offset {offset}")
+ with part_path.open("ab" if offset else "wb") as handle:
+ handle.write(chunk)
+ offset += len(chunk)
+ attempts = 0
+
+ if offset != total:
+ raise RuntimeError(f"Downloaded {offset} bytes from {uri}, expected {total}")
+ os.replace(part_path, destination_path)
+ e_tag = meta.get("e_tag")
+ return ObjectDownload(fileBytes=total, eTag=str(e_tag) if e_tag else None)
+
+
+def upload_file(source: str | Path, uri: str) -> ObjectUpload:
+ source_path = Path(source)
+ if not source_path.is_file():
+ raise FileNotFoundError(source_path)
+ store, key = open_r2_object(uri)
+ file_bytes = source_path.stat().st_size
+ store.put(key, source_path, use_multipart=True)
+ meta = store.head(key)
+ e_tag = meta.get("e_tag")
+ return ObjectUpload(fileBytes=file_bytes, eTag=str(e_tag) if e_tag else None)
diff --git a/profiling/results.py b/profiling/results.py
new file mode 100644
index 00000000..7edf57b8
--- /dev/null
+++ b/profiling/results.py
@@ -0,0 +1,13 @@
+from profiling.config import ProfilingConfig, StageName
+from profiling.r2 import object_exists, put_json
+from profiling.stages import StageRunResult
+
+
+def result_exists(config: ProfilingConfig, nRows: int, stage: StageName) -> bool:
+ return object_exists(config.resultUri(nRows, stage))
+
+
+def write_result(config: ProfilingConfig, result: StageRunResult) -> str:
+ uri = config.resultUri(result.nRows, result.stage)
+ put_json(uri, result.to_json())
+ return uri
diff --git a/profiling/stages.py b/profiling/stages.py
new file mode 100644
index 00000000..9b5d91e4
--- /dev/null
+++ b/profiling/stages.py
@@ -0,0 +1,254 @@
+from dataclasses import asdict, dataclass
+from pathlib import Path
+from typing import Any
+
+from scarf import DataStore, H5adReader, H5adToZarr
+
+from profiling.config import (
+ StageName,
+ StageResources,
+ StorageLayout,
+ WorkflowParameters,
+)
+from profiling.metrics import ResourceMeasurement, ResourceSampler, StageTimer
+from profiling.r2 import storage_options
+
+
+@dataclass(frozen=True, slots=True)
+class StageRunResult:
+ stage: StageName
+ nRows: int
+ status: str
+ seconds: float | None
+ peakRssBytes: int | None
+ peakCgroupBytes: int | None
+ modalMemoryMb: int
+ scarfMemoryBudget: int
+ storeUri: str
+ error: str | None = None
+
+ def to_json(self) -> dict[str, Any]:
+ return asdict(self)
+
+
+def _open_datastore(
+ storeUri: str,
+ workflow: WorkflowParameters,
+ resources: StageResources,
+ *,
+ initialize: bool,
+) -> DataStore:
+ options = storage_options(storeUri)
+ arguments: dict[str, Any] = {
+ "nthreads": resources.workers,
+ "zarr_mode": "r+",
+ "zarrProfile": "cloud" if storeUri.startswith("s3://") else None,
+ "storage_options": options,
+ "mem_budget": resources.scarfMemoryBudget,
+ "working_copies": resources.workingCopies,
+ }
+ if initialize:
+ arguments.update(
+ {
+ "assay_types": {workflow.assayName: "RNA"},
+ "default_assay": workflow.assayName,
+ "min_features_per_cell": workflow.minFeaturesPerCell,
+ "min_cells_per_feature": workflow.minCellsPerFeature,
+ }
+ )
+ return DataStore(storeUri, **arguments)
+
+
+def run_create_store(
+ *,
+ localH5adPath: Path,
+ storeUri: str,
+ workflow: WorkflowParameters,
+ resources: StageResources,
+ storageLayout: StorageLayout | None = None,
+) -> None:
+ options = storage_options(storeUri)
+ reader = H5adReader(
+ str(localH5adPath),
+ matrix_key="X",
+ cell_attrs_key="obs",
+ cell_ids_key="_index",
+ feature_attrs_key="var",
+ feature_ids_key="_index",
+ feature_name_key="feature_name",
+ )
+ layout_kwargs: dict[str, Any] = {}
+ if storageLayout is not None and storageLayout.targetChunkBytes is not None:
+ layout_kwargs = {
+ "targetChunkBytes": storageLayout.targetChunkBytes,
+ "minFeatureChunk": storageLayout.minFeatureChunk,
+ "maxFeatureChunk": storageLayout.maxFeatureChunk,
+ }
+ try:
+ writer = H5adToZarr(
+ reader,
+ storeUri,
+ assay_name=workflow.assayName,
+ storage_options=options,
+ mem_budget=resources.scarfMemoryBudget,
+ nthreads=resources.workers,
+ working_copies=resources.workingCopies,
+ **layout_kwargs,
+ )
+ writer.dump(batch_size=workflow.h5adBatchSize)
+ finally:
+ if hasattr(reader, "h5") and hasattr(reader.h5, "close"):
+ reader.h5.close()
+
+
+def run_stage(
+ stage: StageName,
+ *,
+ nRows: int,
+ storeUri: str,
+ workflow: WorkflowParameters,
+ resources: StageResources,
+ localH5adPath: Path | None = None,
+ storageLayout: StorageLayout | None = None,
+ sampleIntervalSeconds: float = 0.25,
+) -> StageRunResult:
+ timer = StageTimer()
+ sampler = ResourceSampler(sampleIntervalSeconds=sampleIntervalSeconds)
+ error: str | None = None
+ status = "ok"
+ measurement: ResourceMeasurement | None = None
+
+ with timer:
+ sampler.start()
+ try:
+ with timer.operation():
+ if stage == "createStore":
+ if localH5adPath is None:
+ raise ValueError("createStore requires localH5adPath")
+ run_create_store(
+ localH5adPath=localH5adPath,
+ storeUri=storeUri,
+ workflow=workflow,
+ resources=resources,
+ storageLayout=storageLayout,
+ )
+ elif stage == "initializeStore":
+ store = _open_datastore(
+ storeUri, workflow, resources, initialize=True
+ )
+ del store
+ elif stage == "reopenStore":
+ store = _open_datastore(
+ storeUri, workflow, resources, initialize=False
+ )
+ del store
+ else:
+ store = _open_datastore(
+ storeUri, workflow, resources, initialize=False
+ )
+ _run_analysis(stage, store, workflow, resources)
+ del store
+ except Exception as exc:
+ status = "error"
+ error = f"{type(exc).__name__}: {exc}"
+ finally:
+ measurement = sampler.stop()
+
+ seconds = timer.result.measuredOperationSeconds
+ peak_cgroup = None
+ if measurement is not None:
+ if measurement.operationPeakSource in {
+ "cgroupMemoryCurrent",
+ "cgroupMemoryPeak",
+ }:
+ peak_cgroup = measurement.operationPeakBytes
+ else:
+ peak_cgroup = measurement.cgroupMemoryCurrentPeakBytes
+ return StageRunResult(
+ stage=stage,
+ nRows=nRows,
+ status=status,
+ seconds=seconds,
+ peakRssBytes=measurement.processTreeRssPeakBytes if measurement else None,
+ peakCgroupBytes=peak_cgroup,
+ modalMemoryMb=resources.modalMemoryLimitMb,
+ scarfMemoryBudget=resources.scarfMemoryBudget,
+ storeUri=storeUri,
+ error=error,
+ )
+
+
+def _run_analysis(
+ stage: StageName,
+ store: DataStore,
+ workflow: WorkflowParameters,
+ resources: StageResources,
+) -> None:
+ if stage == "filterCells":
+ store.auto_filter_cells(
+ attrs=workflow.filterAttrs,
+ min_p=workflow.filterMinQuantile,
+ max_p=workflow.filterMaxQuantile,
+ show_qc_plots=False,
+ )
+ return
+ if stage == "markHvgs":
+ store.mark_hvgs(
+ from_assay=workflow.assayName,
+ cell_key=workflow.cellKey,
+ min_cells=workflow.hvgMinCells,
+ top_n=workflow.topN,
+ show_plot=False,
+ hvg_key_name=workflow.hvgKey,
+ )
+ return
+ if stage == "makeGraph":
+ store.make_graph(
+ from_assay=workflow.assayName,
+ cell_key=workflow.cellKey,
+ feat_key=workflow.hvgKey,
+ dims=workflow.dims,
+ k=workflow.k,
+ ann_parallel=workflow.annParallel,
+ rand_state=workflow.graphSeed,
+ n_centroids=workflow.nCentroids,
+ show_elbow_plot=False,
+ local_cache=workflow.graphLocalCache,
+ )
+ return
+ if stage == "runUmap":
+ store.run_umap(
+ from_assay=workflow.assayName,
+ cell_key=workflow.cellKey,
+ feat_key=workflow.hvgKey,
+ n_epochs=workflow.umapEpochs,
+ random_seed=workflow.umapSeed,
+ label=workflow.umapLabel,
+ parallel=workflow.umapParallel,
+ nthreads=resources.workers,
+ )
+ return
+ if stage == "runLeiden":
+ store.run_leiden_clustering(
+ from_assay=workflow.assayName,
+ cell_key=workflow.cellKey,
+ feat_key=workflow.hvgKey,
+ resolution=workflow.leidenResolution,
+ label=workflow.leidenLabel,
+ random_seed=workflow.leidenSeed,
+ )
+ return
+ if stage == "findMarkers":
+ store.run_marker_search(
+ from_assay=workflow.assayName,
+ group_key=workflow.resolvedMarkerGroupKey,
+ cell_key=workflow.cellKey,
+ feat_key=workflow.markerFeatureKey,
+ gene_batch_size=workflow.markerGeneBatchSize,
+ use_prenormed=False,
+ prenormed_store=None,
+ n_threads=resources.workers,
+ skip_save=False,
+ )
+ return
+ raise ValueError(f"No analysis operation for {stage}")
diff --git a/pypi_README.rst b/pypi_README.rst
deleted file mode 100644
index ae51fb2f..00000000
--- a/pypi_README.rst
+++ /dev/null
@@ -1,30 +0,0 @@
-|PyPI| |Docs| |Tests| |Coverage| |Downloads| |Black| |Slack|
-
-`Installation`_ | `Documentation`_ | `Tutorials`_ | `Research-Article`_
-
-|IMG1|
-
-.. |PyPI| image:: https://img.shields.io/pypi/v/scarf.svg
- :target: https://pypi.org/project/scarf
-.. |Docs| image:: https://readthedocs.org/projects/scarf/badge/?version=latest
- :target: https://scarf.readthedocs.io
-.. |Tests| image:: https://github.com/parashardhapola/scarf/actions/workflows/pytest.yml/badge.svg
- :target: https://github.com/parashardhapola/scarf/actions/workflows/pytest.yml
-.. |Coverage| image:: https://codecov.io/gh/parashardhapola/scarf/branch/master/graph/badge.svg?token=ZvJXuYq3pd
- :target: https://codecov.io/gh/parashardhapola/scarf
-.. |Downloads| image:: https://pepy.tech/badge/scarf
- :target: https://pepy.tech/project/scarf
-.. |Black| image:: https://img.shields.io/badge/code%20style-black-000000.svg
- :target: https://github.com/psf/black
-.. |Slack| image:: https://img.shields.io/badge/Slack-4A154B?style=for-the-badge&logo=slack&logoColor=white
- :target: https://scarf-group.slack.com/archives/C0418C7RXU4
-
-.. |IMG1| image:: https://raw.githubusercontent.com/parashardhapola/scarf/master/docs/source/logo_wide.png
- :target: https://github.com/parashardhapola/scarf
-
-Analyze large-scale (aka atlas-scale) single-cell genomics datasets with very low memory requirement.
-
-.. _Installation: https://scarf.readthedocs.io/en/latest/install.html
-.. _Documentation: http://scarf.rtfd.io
-.. _Tutorials: https://scarf.readthedocs.io/en/latest/vignettes/basic_tutorial_scRNAseq.html
-.. _Research-Article: https://www.nature.com/articles/s41467-022-32097-3
diff --git a/pyproject.toml b/pyproject.toml
index 65c8f21f..6d2d11ca 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,20 +1,123 @@
-[build.system]
-requires = [
- "setuptools",
- "wheel"
-]
+[build-system]
+requires = ["setuptools>=61", "wheel"]
build-backend = "setuptools.build_meta"
+[project]
+name = "scarf"
+dynamic = ["version"]
+description = "Out-of-core, graph-first workflows for million-cell RNA-seq and CITE-seq"
+readme = {file = "README.md", content-type = "text/markdown"}
+requires-python = ">=3.12"
+license = "BSD-3-Clause"
+authors = [
+ {name = "Parashar Dhapola", email = "parashar.dhapola@gmail.com"},
+]
+keywords = [
+ "single-cell",
+ "scRNA-seq",
+ "scATAC-seq",
+ "CITE-seq",
+ "Harmony",
+ "WNN",
+ "batch-correction",
+ "Zarr",
+ "memory-efficient",
+]
+classifiers = [
+ "Natural Language :: English",
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.12",
+ "Programming Language :: Python :: 3.13",
+ "Programming Language :: Python :: 3.14",
+]
+dependencies = [
+ "h5py>=3.11",
+ "hnswlib>=0.8",
+ "leidenalg>=0.12",
+ "loguru>=0.7",
+ "networkx>=3",
+ "numba>=0.60",
+ "numcodecs>=0.16",
+ "numpy>=2",
+ "packaging",
+ "pandas>=2.2.2",
+ "scikit-learn>=1.5",
+ "scikit-network>=0.33.1",
+ "scipy>=1.13,!=1.17.0",
+ "setuptools>=61",
+ "statsmodels>=0.14.2",
+ "threadpoolctl",
+ "tqdm>=4.60",
+ "umap-learn>=0.5.6",
+ "zarr>=3.1.2",
+ "obstore",
+ "sgtsnepi",
+]
+
+[project.optional-dependencies]
+extra = [
+ "anndata>=0.12",
+ "requests>=2.28",
+ "kneed>=0.8",
+ "matplotlib>=3.9",
+ "seaborn>=0.13",
+ "datashader>=0.18",
+ "ipython-autotime>=0.3",
+ "ipywidgets>=8",
+]
+test = [
+ "pytest>=7.0",
+ "pytest-cov>=4",
+ "pytest-xdist>=3",
+ "topacedo>=0.2.10",
+ "anndata>=0.12",
+]
+docs = [
+ "jinja2",
+ "jedi",
+ "myst-parser",
+ "myst-nb",
+ "sphinx",
+ "sphinx_autodoc_typehints",
+ "sphinx-book-theme",
+ "sphinx-copybutton",
+ "sphinx-external-toc",
+ "sphinx-tabs",
+ "topacedo",
+ "nbclient",
+]
+
+[project.urls]
+Homepage = "https://github.com/parashardhapola/scarf"
+
+[tool.setuptools]
+include-package-data = false
+
+[tool.setuptools.dynamic]
+version = {file = "VERSION"}
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["scarf*"]
+exclude = ["profiling*", "scripts*", "tests*", "docs*", "bin*"]
+
[tool.pytest.ini_options]
minversion = "6.0"
-addopts = "-ra -v"
+addopts = "-ra -v -m 'not integration' -n auto --dist loadfile"
+markers = [
+ "integration: live network tests",
+]
testpaths = [
- "scarf/tests",
+ "tests",
]
norecursedirs = [
- "scarf/tests"
+ "datasets",
]
+[tool.coverage.run]
+source = ["scarf"]
+parallel = true
+
[tool.interrogate]
ignore-init-method = true
ignore-init-module = false
@@ -30,7 +133,7 @@ fail-under = 50
exclude = [
"setup.py",
"docs",
- "scarf/tests"
+ "tests"
]
ignore-regex = [
"^get$",
@@ -43,3 +146,65 @@ whitelist-regex = []
color = true
generate-badge = '.'
badge-format = "svg"
+
+[dependency-groups]
+profiling = [
+ "anndata>=0.12.18",
+ "igraph>=1.0.0",
+ "modal>=1.5.2",
+ "pydantic>=2.13.4",
+]
+dev = [
+ "mypy>=2.1.0",
+ "ruff>=0.15.20",
+]
+
+[tool.uv]
+no-binary-package = ["pcst-fast", "hnswlib"]
+extra-build-variables = { hnswlib = { HNSWLIB_NO_NATIVE = "1" } }
+
+[tool.uv.pip]
+no-binary = ["hnswlib"]
+extra-build-variables = { hnswlib = { HNSWLIB_NO_NATIVE = "1" } }
+
+[tool.uv.extra-build-dependencies]
+pcst-fast = ["pybind11>=2.11"]
+
+[tool.mypy]
+strict = false
+
+python_version = "3.12"
+exclude = ["^docs/", "^setup\\.py$"]
+
+disable_error_code = ["import-untyped"]
+
+disallow_untyped_defs = true
+disallow_incomplete_defs = true
+
+check_untyped_defs = true
+disallow_untyped_calls = true
+no_implicit_optional = true
+
+warn_return_any = true
+warn_unused_configs = true
+warn_redundant_casts = true
+warn_unused_ignores = false
+
+ignore_missing_imports = false
+
+[[tool.mypy.overrides]]
+module = [
+ "scarf.plots",
+ "scarf.plotting",
+ "scarf.plotting.*",
+]
+ignore_missing_imports = true
+
+[[tool.mypy.overrides]]
+module = ["profiling.*", "tests.*"]
+disallow_untyped_defs = false
+disallow_incomplete_defs = false
+disallow_untyped_calls = false
+check_untyped_defs = false
+warn_return_any = false
+ignore_errors = true
diff --git a/requirements.txt b/requirements.txt
deleted file mode 100644
index d5866a5d..00000000
--- a/requirements.txt
+++ /dev/null
@@ -1,24 +0,0 @@
-dask[array]<=2024.7.1
-h5py>=3.3.0
-hnswlib==0.8.0
-importlib_metadata
-joblib
-leidenalg
-loguru
-networkx==3.4.2
-numba<=0.61.2
-numcodecs<0.16
-numpy==1.26.4
-packaging
-pandas
-polars<=1.25
-pyarrow
-scikit-learn==1.6.1
-scikit-network<=0.33.2
-scipy==1.15.2
-setuptools
-statsmodels
-threadpoolctl
-tqdm
-umap-learn
-zarr<=2.16.0
diff --git a/requirements_extra.txt b/requirements_extra.txt
deleted file mode 100644
index 1bffaace..00000000
--- a/requirements_extra.txt
+++ /dev/null
@@ -1,9 +0,0 @@
-requests
-gensim
-kneed
-matplotlib
-seaborn
-cmocean
-datashader
-ipython-autotime
-ipywidgets
diff --git a/scarf/__init__.py b/scarf/__init__.py
index 54525001..30fa5157 100644
--- a/scarf/__init__.py
+++ b/scarf/__init__.py
@@ -34,21 +34,113 @@
import warnings
-from dask.array import PerformanceWarning
-from importlib_metadata import version
+from importlib.metadata import PackageNotFoundError, version
+from pathlib import Path
+from zarr.errors import UnstableSpecificationWarning
-warnings.filterwarnings("ignore", category=DeprecationWarning)
-warnings.filterwarnings("ignore", category=PerformanceWarning)
+from .datastore.datastore import DataStore
+from .downloader import fetch_dataset, show_available_datasets
+from .mapping_reference import MappingReference, MappingResult
+from .meld_assay import GffReader, coordinate_melding
+from .merge import AssayMerge, DatasetMerge, ZarrMerge
+from .readers import (
+ CSVReader,
+ CrDirReader,
+ CrH5Reader,
+ CrReader,
+ H5adReader,
+ LoomReader,
+ NaboH5Reader,
+)
+from .utils import (
+ clean_array,
+ controlled_compute,
+ get_log_level,
+ load_zarr,
+ logger,
+ permute_into_chunks,
+ prefetch_blocks,
+ rescale_array,
+ rolling_window,
+ set_verbosity,
+ show_dask_progress,
+ system_call,
+ tqdmbar,
+ tqdm_params,
+)
+from .writers import (
+ CSVtoZarr,
+ CrToZarr,
+ H5adToZarr,
+ LoomToZarr,
+ NaboH5ToZarr,
+ SparseToZarr,
+ SubsetZarr,
+ create_zarr_count_assay,
+ create_zarr_dataset,
+ create_zarr_obj_array,
+ dask_to_zarr,
+ subset_assay_zarr,
+ to_h5ad,
+ to_mtx,
+ write_renorm_subset_to_zarr,
+)
+
+warnings.filterwarnings("ignore", category=UnstableSpecificationWarning)
try:
__version__ = version("scarf")
-except ImportError:
- print("Scarf is not installed", flush=True)
+except PackageNotFoundError:
+ version_path = Path(__file__).resolve().parents[1] / "VERSION"
+ __version__ = (
+ version_path.read_text().strip() if version_path.is_file() else "unavailable"
+ )
-from .datastore.datastore import DataStore
-from .readers import *
-from .writers import *
-from .meld_assay import *
-from .utils import *
-from .downloader import *
-from .merge import *
\ No newline at end of file
+__all__ = [
+ "AssayMerge",
+ "CSVReader",
+ "CSVtoZarr",
+ "CrDirReader",
+ "CrH5Reader",
+ "CrReader",
+ "CrToZarr",
+ "DataStore",
+ "DatasetMerge",
+ "GffReader",
+ "H5adReader",
+ "H5adToZarr",
+ "LoomReader",
+ "LoomToZarr",
+ "MappingReference",
+ "MappingResult",
+ "NaboH5Reader",
+ "NaboH5ToZarr",
+ "SparseToZarr",
+ "SubsetZarr",
+ "ZarrMerge",
+ "clean_array",
+ "controlled_compute",
+ "coordinate_melding",
+ "create_zarr_count_assay",
+ "create_zarr_dataset",
+ "create_zarr_obj_array",
+ "dask_to_zarr",
+ "fetch_dataset",
+ "get_log_level",
+ "load_zarr",
+ "logger",
+ "permute_into_chunks",
+ "prefetch_blocks",
+ "rescale_array",
+ "rolling_window",
+ "set_verbosity",
+ "show_available_datasets",
+ "show_dask_progress",
+ "subset_assay_zarr",
+ "system_call",
+ "to_h5ad",
+ "to_mtx",
+ "tqdmbar",
+ "tqdm_params",
+ "write_renorm_subset_to_zarr",
+]
diff --git a/scarf/_types.py b/scarf/_types.py
new file mode 100644
index 00000000..9566aa68
--- /dev/null
+++ b/scarf/_types.py
@@ -0,0 +1,23 @@
+from typing import Literal
+
+import zarr
+
+type ZarrMode = Literal["r", "r+", "a", "w", "w-"]
+
+
+def as_zarr_array(node: zarr.Array | zarr.Group, *, name: str = "") -> zarr.Array:
+ if isinstance(node, zarr.Array):
+ return node
+ label = f" at {name!r}" if name else ""
+ raise TypeError(f"Expected Zarr array{label}, got {type(node).__name__}")
+
+
+def as_zarr_group(node: zarr.Array | zarr.Group, *, name: str = "") -> zarr.Group:
+ if isinstance(node, zarr.Group):
+ return node
+ label = f" at {name!r}" if name else ""
+ raise TypeError(f"Expected Zarr group{label}, got {type(node).__name__}")
+
+
+def array_metadata_shards(array: zarr.Array) -> tuple[int, ...] | None:
+ return getattr(array.metadata, "shards", None)
diff --git a/scarf/ann.py b/scarf/ann.py
index 845435e7..af0e0702 100644
--- a/scarf/ann.py
+++ b/scarf/ann.py
@@ -1,19 +1,44 @@
-from typing import Optional
+from collections.abc import Callable, Generator
+from typing import Any
-import dask.array as da
import numpy as np
import pandas as pd
from threadpoolctl import threadpool_limits
-from .harmony import run_harmony
+from .chunked import ChunkedArray
+from .harmony import HarmonyResult, fit_harmony
from .utils import controlled_compute, logger, tqdmbar
__all__ = ["AnnStream", "instantiate_knn_index", "fix_knn_query"]
+EMBEDDING_CACHE_MAX_BYTES = 256 * 1024 * 1024
+
def instantiate_knn_index(
- space, dim, max_elements, ef_construction, M, random_seed, ef, num_threads
-):
+ space: str,
+ dim: int,
+ max_elements: int,
+ ef_construction: int,
+ M: int,
+ random_seed: int,
+ ef: int,
+ num_threads: int,
+) -> Any:
+ """Create and configure an hnswlib KNN index.
+
+ Args:
+ space: Distance metric name accepted by hnswlib (e.g. ``'l2'``, ``'cosine'``).
+ dim: Embedding dimensionality.
+ max_elements: Maximum number of vectors the index can hold.
+ ef_construction: ``ef_construction`` parameter for index building.
+ M: ``M`` parameter controlling graph connectivity.
+ random_seed: Random seed for index construction.
+ ef: ``ef`` search parameter set after index creation.
+ num_threads: Number of threads for hnswlib queries.
+
+ Returns:
+ Configured hnswlib Index instance.
+ """
import hnswlib
ann_idx = hnswlib.Index(space=space, dim=dim)
@@ -28,11 +53,26 @@ def instantiate_knn_index(
return ann_idx
-def fix_knn_query(indices: np.ndarray, distances: np.ndarray, ref_idx: np.ndarray):
+def fix_knn_query(
+ indices: np.ndarray,
+ distances: np.ndarray,
+ ref_idx: np.ndarray,
+) -> tuple[np.ndarray, np.ndarray, int]:
+ """Remove self-neighbor entries from KNN query results when recall is imperfect.
+
+ Args:
+ indices: Neighbor indices from ``knn_query``, shape (n_queries, k).
+ distances: Neighbor distances matching ``indices``.
+ ref_idx: Global index of each query row (used to detect missing self loops).
+
+ Returns:
+ Tuple of (fixed_indices, fixed_distances, n_mis) where ``n_mis`` counts
+ queries whose first neighbor was not a self match.
+ """
fixed_ind, fixed_dist = indices.copy()[:, 1:], distances.copy()[:, 1:]
# Identify positions where first index is not a self loop
mis_idx = indices[:, 0].reshape(1, -1)[0] != ref_idx
- n_mis = mis_idx.sum()
+ n_mis = int(mis_idx.sum())
if n_mis > 0:
for n, i, j, k in zip(
np.where(mis_idx)[0], ref_idx[mis_idx], indices[mis_idx], distances[mis_idx]
@@ -53,14 +93,45 @@ def fix_knn_query(indices: np.ndarray, distances: np.ndarray, ref_idx: np.ndarra
class AnnStream:
+ """Stream row blocks through dimensionality reduction and fit ANN / k-means.
+
+ Args:
+ data: ChunkedArray of cell-by-feature counts.
+ k: Number of nearest neighbors to query.
+ n_cluster: Number of k-means clusters for seed partitions.
+ reduction_method: One of ``'pca'``, ``'lsi'``, or ``'custom'``.
+ dims: Number of reduced dimensions (or loadings columns) to use.
+ loadings: Precomputed loading matrix; if None, loadings are fit from data.
+ use_for_pca: Boolean mask of cells used when fitting PCA.
+ mu: Feature means for scaling (PCA).
+ sigma: Feature std devs for scaling (PCA).
+ ann_metric: hnswlib distance metric.
+ ann_efc: hnswlib ``ef_construction``.
+ ann_ef: hnswlib ``ef`` search parameter.
+ ann_m: hnswlib ``M`` connectivity parameter.
+ nthreads: Thread limit for sklearn / block compute.
+ ann_parallel: If True, use ``nthreads`` for hnswlib as well.
+ rand_state: Random seed.
+ do_kmeans_fit: Whether to fit MiniBatchKMeans seed partitions.
+ disable_scaling: Skip z-scaling before PCA projection.
+ ann_idx: Existing hnswlib index to reuse instead of fitting.
+ lsi_skip_first: Drop first LSI component (depth) when using LSI.
+ lsi_params: Extra kwargs forwarded to sklearn TruncatedSVD (minus reserved keys).
+ harmonize: Whether to run Harmony batch correction before ANN fitting.
+ harmonized_data: Precomputed harmonized ChunkedArray (optional).
+ batches: Batch metadata DataFrame for Harmony.
+ """
+
+ reducer: Callable[[np.ndarray], np.ndarray]
+
def __init__(
self,
- data,
+ data: ChunkedArray,
k: int,
n_cluster: int,
reduction_method: str,
- dims: int,
- loadings: np.ndarray,
+ dims: int | None,
+ loadings: np.ndarray | None,
use_for_pca: np.ndarray,
mu: np.ndarray,
sigma: np.ndarray,
@@ -73,14 +144,17 @@ def __init__(
rand_state: int,
do_kmeans_fit: bool,
disable_scaling: bool,
- ann_idx,
+ ann_idx: Any | None,
lsi_skip_first: bool,
- lsi_params: dict,
+ lsi_params: dict[str, Any],
harmonize: bool,
- harmonized_data: Optional[da.Array] = None,
- batches: Optional[pd.DataFrame] = None,
- ):
+ harmonized_data: ChunkedArray | None = None,
+ batches: pd.DataFrame | None = None,
+ cache_embeddings: bool = True,
+ harmony_params: dict[str, Any] | None = None,
+ ) -> None:
self.data = data
+ self._embeddings: np.ndarray | None = None
self.k = k
if self.k >= self.data.shape[0]:
self.k = self.data.shape[0] - 1
@@ -95,6 +169,7 @@ def __init__(
self.annEfc = ann_efc
self.annEf = ann_ef
self.annM = ann_m
+ self.annPath: str | None = None
self.nthreads = nthreads
if ann_parallel:
self.annThreads = self.nthreads
@@ -107,9 +182,12 @@ def __init__(
self.clusterLabels: np.ndarray = np.repeat(-1, self.nCells)
self.harmonize = harmonize
self.harmonizedData = harmonized_data
+ self.harmonyResult: HarmonyResult | None = None
self.batches = batches
+ self.harmonyParams = dict(harmony_params or {})
+ self.featureScaling = not disable_scaling
disable_reduction = False
- if self.dims < 1:
+ if self.dims is not None and self.dims < 1:
disable_reduction = True
with threadpool_limits(limits=self.nthreads):
if self.method == "pca":
@@ -126,16 +204,31 @@ def __init__(
# AnnStream, it could still be overwritten by _handle_batch_size. Hence need to hard set it here.
self.dims = self.loadings.shape[1]
# it is okay for dimensions to be larger than batch size here because we will not fit the PCA
+ loadings = self.loadings
if disable_scaling:
if disable_reduction:
- self.reducer = lambda x: x
+
+ def reducer(x: np.ndarray) -> np.ndarray:
+ return x
else:
- self.reducer = lambda x: x.dot(self.loadings)
+ assert loadings is not None
+ loadings_mat = loadings
+
+ def reducer(x: np.ndarray) -> np.ndarray:
+ return np.asarray(x.dot(loadings_mat))
else:
if disable_reduction:
- self.reducer = lambda x: self.transform_z(x)
+
+ def reducer(x: np.ndarray) -> np.ndarray:
+ return self.transform_z(x)
else:
- self.reducer = lambda x: self.transform_z(x).dot(self.loadings)
+ assert loadings is not None
+ loadings_mat = loadings
+
+ def reducer(x: np.ndarray) -> np.ndarray:
+ return np.asarray(self.transform_z(x).dot(loadings_mat))
+
+ self.reducer = reducer
elif self.method == "lsi":
if self.loadings is None or len(self.loadings) == 0:
if disable_reduction is False:
@@ -145,10 +238,19 @@ def __init__(
if lsi_skip_first:
self.loadings = self.loadings[:, 1:]
self.dims = self.loadings.shape[1]
+ loadings = self.loadings
if disable_reduction:
- self.reducer = lambda x: x
+
+ def reducer(x: np.ndarray) -> np.ndarray:
+ return x
else:
- self.reducer = lambda x: x.dot(self.loadings)
+ assert loadings is not None
+ loadings_mat = loadings
+
+ def reducer(x: np.ndarray) -> np.ndarray:
+ return np.asarray(x.dot(loadings_mat))
+
+ self.reducer = reducer
elif self.method == "custom":
if self.loadings is None or len(self.loadings) == 0:
logger.warning(
@@ -156,12 +258,23 @@ def __init__(
)
else:
self.dims = self.loadings.shape[1]
+ loadings = self.loadings
if disable_reduction:
- self.reducer = lambda x: x
+
+ def reducer(x: np.ndarray) -> np.ndarray:
+ return x
else:
- self.reducer = lambda x: x.dot(self.loadings)
+ assert loadings is not None
+ loadings_mat = loadings
+
+ def reducer(x: np.ndarray) -> np.ndarray:
+ return np.asarray(x.dot(loadings_mat))
+
+ self.reducer = reducer
else:
raise ValueError(f"ERROR: Unknown reduction method: {self.method}")
+ if cache_embeddings:
+ self._maybe_build_embeddings()
if ann_idx is None:
self.annIdx = self._fit_ann()
else:
@@ -170,11 +283,38 @@ def __init__(
self.annIdx.set_num_threads(self.annThreads)
self.kmeans = self._fit_kmeans(do_kmeans_fit)
- def _handle_batch_size(self):
- if self.dims > self.data.shape[0]:
+ @property
+ def embeddings(self) -> np.ndarray | None:
+ """Reduced cell embeddings when cached in memory, else None."""
+ return self._embeddings
+
+ def _embedding_bytes(self) -> int:
+ if self.dims is None or self.dims < 1:
+ cols = self.nFeats
+ else:
+ cols = self.dims
+ return self.nCells * cols * 8
+
+ def _reduced_blocks(self, msg: str) -> list[np.ndarray]:
+ """Apply the reducer to every row block in parallel, preserving order."""
+ return self.data.map_blocks(
+ lambda _i, s, e: self.reducer(self.data._materialize_range(s, e)),
+ nthreads=self.nthreads,
+ msg=msg,
+ )
+
+ def _maybe_build_embeddings(self) -> None:
+ if self.harmonize or self._embedding_bytes() > EMBEDDING_CACHE_MAX_BYTES:
+ return
+ self._embeddings = np.vstack(
+ self._reduced_blocks(msg="Building cell embeddings")
+ )
+
+ def _handle_batch_size(self) -> int:
+ if self.dims is not None and self.dims > self.data.shape[0]:
self.dims = self.data.shape[0]
batch_size = self.data.chunksize[0] # Assuming all chunks are same size
- if self.dims >= batch_size:
+ if self.dims is not None and self.dims >= batch_size:
self.dims = batch_size - 1 # -1 because we will do PCA +1
logger.info(
f"Number of PCA/LSI components reduced to batch size of {batch_size}"
@@ -184,43 +324,77 @@ def _handle_batch_size(self):
logger.info(f"Cluster number reduced to batch size of {batch_size}")
return batch_size
- def iter_blocks(self, msg: str = "") -> np.ndarray:
- for i in tqdmbar(self.data.blocks, desc=msg, total=self.data.numblocks[0]):
- yield controlled_compute(i, self.nthreads)
+ def iter_blocks(self, msg: str = "") -> Generator[np.ndarray, None, None]:
+ """Yield row blocks of raw data as NumPy arrays with optional progress bar."""
+ yield from self.data.stream_blocks(nthreads=self.nthreads, msg=msg)
def transform_z(self, a: np.ndarray) -> np.ndarray:
- return (a - self.mu) / self.sigma
+ """Z-score a block using fitted ``mu`` and ``sigma``."""
+ return np.asarray((a - self.mu) / self.sigma)
+
+ def transform_query(self, a: np.ndarray) -> np.ndarray:
+ """Project query feature rows into this index's native latent space."""
+ values = np.asarray(a)
+ if values.ndim != 2:
+ raise ValueError("Query data must be a two-dimensional array")
+ if values.shape[1] != self.nFeats:
+ raise ValueError(
+ f"Query has {values.shape[1]} features but reference expects {self.nFeats}"
+ )
+ result = np.asarray(self.reducer(values))
+ expected_dims = (
+ self.dims if self.dims is not None and self.dims > 0 else self.nFeats
+ )
+ if result.ndim != 2 or result.shape[1] != expected_dims:
+ raise ValueError("Query transform did not produce the ANN index dimensions")
+ if not np.all(np.isfinite(result)):
+ raise ValueError("Query transform produced non-finite values")
+ return result
def transform_ann(
- self, a: np.ndarray, k: int = None, self_indices: np.ndarray = None
- ) -> tuple:
+ self,
+ a: np.ndarray,
+ k: int | None = None,
+ self_indices: np.ndarray | None = None,
+ ) -> tuple[np.ndarray, np.ndarray] | tuple[np.ndarray, np.ndarray, int]:
+ """Query the ANN index for neighbors of transformed rows in ``a``.
+
+ Args:
+ a: Transformed embedding block, shape (n_rows, n_dims).
+ k: Number of neighbors; defaults to ``self.k``.
+ self_indices: Global row indices for self-neighbor correction.
+
+ Returns:
+ Tuple of (indices, distances) from hnswlib, optionally corrected
+ when ``self_indices`` is provided.
+ """
if k is None:
k = self.k
# Adding +1 to k because first neighbour will be the query itself
if self_indices is None:
i, d = self.annIdx.knn_query(a, k=k)
- return i, d
- else:
- i, d = self.annIdx.knn_query(a, k=k + 1)
- return fix_knn_query(i, d, self_indices)
+ return np.asarray(i), np.asarray(d)
+ i, d = self.annIdx.knn_query(a, k=k + 1)
+ return fix_knn_query(i, d, self_indices)
- def _fit_pca(self, disable_scaling, use_for_pca) -> None:
+ def _fit_pca(self, disable_scaling: bool, use_for_pca: np.ndarray) -> None:
from sklearn.decomposition import IncrementalPCA
from numpy.linalg import LinAlgError
+ assert self.dims is not None
+ dims = self.dims
+
# We fit 1 extra PC dim than specified and then ignore the last PC.
- self._pca = IncrementalPCA(
- n_components=self.dims + 1, batch_size=self.batchSize
- )
+ self._pca = IncrementalPCA(n_components=dims + 1, batch_size=self.batchSize)
do_sample_subset = False if use_for_pca.sum() == self.nCells else True
s, e = 0, 0
# We store the first block of values here. if such a case arises that we are left with less dims+1 cells to fit
# then those cells can be added to end_reservoir for fitting. if there are no such cells then end reservoir is
# just by itself after fitting rest of the cells. If may be the case that the first batch itself has less than
# dims+1 cells. in that we keep adding cells to carry_over pile until it is big enough.
- end_reservoir = []
+ end_reservoir: np.ndarray | None = None
# carry_over store cells that can yet not be added to end_reservoir ot be used for fitting pca directly.
- carry_over = []
+ carry_over: np.ndarray | None = None
for i in self.iter_blocks(msg="Fitting PCA"):
if do_sample_subset:
e = s + i.shape[0]
@@ -228,13 +402,13 @@ def _fit_pca(self, disable_scaling, use_for_pca) -> None:
s = e
if disable_scaling is False:
i = self.transform_z(i)
- if len(carry_over) > 0:
+ if carry_over is not None:
i = np.vstack((carry_over, i))
- carry_over = []
- if len(i) < (self.dims + 1):
+ carry_over = None
+ if len(i) < (dims + 1):
carry_over = i
continue
- if len(end_reservoir) == 0:
+ if end_reservoir is None:
end_reservoir = i
continue
try:
@@ -242,12 +416,16 @@ def _fit_pca(self, disable_scaling, use_for_pca) -> None:
except LinAlgError:
# Add retry counter to make memory consumption doesn't escalate
carry_over = i
- if len(carry_over) > 0:
- i = np.vstack((end_reservoir, carry_over))
+ if carry_over is not None:
+ if end_reservoir is not None:
+ fit_batch = np.vstack((end_reservoir, carry_over))
+ else:
+ fit_batch = carry_over
else:
- i = end_reservoir
+ assert end_reservoir is not None
+ fit_batch = end_reservoir
try:
- self._pca.partial_fit(i, check_input=False)
+ self._pca.partial_fit(fit_batch, check_input=False)
except LinAlgError:
logger.warning(
"{i.shape[0]} samples were not used in PCA fitting due to LinAlgError",
@@ -255,52 +433,48 @@ def _fit_pca(self, disable_scaling, use_for_pca) -> None:
)
self.loadings = self._pca.components_[:-1, :].T
- def _fit_lsi(self, lsi_skip_first, lsi_params) -> None:
- import warnings
+ def _fit_lsi(self, lsi_skip_first: bool, lsi_params: dict[str, Any]) -> None:
+ from sklearn.decomposition import TruncatedSVD
- with warnings.catch_warnings():
- warnings.simplefilter("ignore", UserWarning)
- from gensim.models import LsiModel
- from gensim.matutils import Dense2Corpus
+ assert self.dims is not None
+ dims = self.dims
- for i in ["corpus", "num_topics", "id2word", "chunksize", "dtype"]:
- if i in lsi_params:
- del lsi_params[i]
+ reserved = {"n_components", "random_state"}
+ for key in list(lsi_params):
+ if key in reserved:
+ del lsi_params[key]
logger.warning(
- f"Provided parameter, {i}, for LSI model will not be used"
+ f"Provided parameter, {key}, for LSI model will not be used"
)
- self._lsiModel = LsiModel(
- corpus=Dense2Corpus(
- controlled_compute(self.data.blocks[0], self.nthreads).T
- ),
- num_topics=self.dims + 1, # +1 because first dim will be discarded
- chunksize=self.data.chunksize[0],
- id2word={x: x for x in range(self.data.shape[1])},
+
+ mat = np.vstack(list(self.iter_blocks(msg="Fitting LSI model")))
+ svd = TruncatedSVD(
+ n_components=dims + 1,
+ random_state=self.randState,
**lsi_params,
)
- for n, i in enumerate(self.iter_blocks(msg="Fitting LSI model")):
- if n == 0:
- continue
- self._lsiModel.add_documents(Dense2Corpus(i.T))
+ svd.fit(mat)
+ components = svd.components_.T
if lsi_skip_first:
- self.loadings = self._lsiModel.get_topics().T[:, 1:]
+ self.loadings = components[:, 1:]
else:
- self.loadings = self._lsiModel.get_topics().T
+ self.loadings = components
- def _fit_ann(self):
- def _transform_values():
- pca_array = []
- for _i in self.iter_blocks(msg="Calculating uncorrected latent dimensions"):
- pca_array.append(self.reducer(_i))
- return np.vstack(pca_array).T
+ def _fit_ann(self) -> Any:
+ def _transform_values() -> np.ndarray:
+ return np.vstack(
+ self._reduced_blocks(msg="Calculating uncorrected latent dimensions")
+ ).T
- dims = self.dims
- if dims < 1:
- dims = self.data.shape[1]
+ ann_dims = (
+ self.dims
+ if self.dims is not None and self.dims >= 1
+ else self.data.shape[1]
+ )
ann_idx = instantiate_knn_index(
self.annMetric,
- dims,
+ ann_dims,
self.nCells,
self.annEfc,
self.annM,
@@ -310,9 +484,15 @@ def _transform_values():
)
if self.harmonize:
if self.harmonizedData is None:
- self.harmonizedData = da.from_array(
- run_harmony(_transform_values(), self.batches).T,
- chunks=self.data.chunksize,
+ if self.batches is None:
+ raise ValueError("Harmony requires batch metadata")
+ self.harmonyResult = fit_harmony(
+ _transform_values(), self.batches, **self.harmonyParams
+ )
+ self.harmonizedData = ChunkedArray.from_numpy(
+ self.harmonyResult.corrected.T,
+ block_size=self.data.chunksize[0],
+ nthreads=self.nthreads,
)
for i in tqdmbar(
self.harmonizedData.blocks,
@@ -320,12 +500,22 @@ def _transform_values():
total=self.harmonizedData.numblocks[0],
):
ann_idx.add_items(controlled_compute(i, self.nthreads))
+ elif self._embeddings is not None:
+ bs = self.batchSize
+ n_blocks = int(np.ceil(self.nCells / bs))
+ for start in tqdmbar(
+ range(0, self.nCells, bs),
+ desc="Fitting ANN",
+ total=n_blocks,
+ ):
+ end = min(start + bs, self.nCells)
+ ann_idx.add_items(self._embeddings[start:end])
else:
for i in self.iter_blocks(msg="Fitting ANN"):
ann_idx.add_items(self.reducer(i))
return ann_idx
- def _fit_kmeans(self, do_ann_fit):
+ def _fit_kmeans(self, do_ann_fit: bool) -> Any | None:
from sklearn.cluster import MiniBatchKMeans
if do_ann_fit is False:
@@ -336,11 +526,39 @@ def _fit_kmeans(self, do_ann_fit):
batch_size=self.batchSize,
n_init=3,
)
- temp = []
+ temp: list[int] = []
with threadpool_limits(limits=self.nthreads):
- for i in self.iter_blocks(msg="Fitting kmeans"):
- kmeans.partial_fit(self.reducer(i))
- for i in self.iter_blocks(msg="Estimating seed partitions"):
- temp.extend(kmeans.predict(self.reducer(i)))
+ if self._embeddings is not None:
+ bs = self.batchSize
+ n_blocks = int(np.ceil(self.nCells / bs))
+ # Two phases: predict must run on the fully fitted model, so it
+ # cannot be interleaved with partial_fit. Both passes read the
+ # already-materialized embeddings, so no extra data reads occur.
+ for start in tqdmbar(
+ range(0, self.nCells, bs),
+ desc="Fitting kmeans",
+ total=n_blocks,
+ ):
+ end = min(start + bs, self.nCells)
+ kmeans.partial_fit(self._embeddings[start:end])
+ for start in tqdmbar(
+ range(0, self.nCells, bs),
+ desc="Estimating seed partitions",
+ total=n_blocks,
+ ):
+ end = min(start + bs, self.nCells)
+ temp.extend(kmeans.predict(self._embeddings[start:end]))
+ else:
+ for i in self.iter_blocks(msg="Fitting kmeans"):
+ kmeans.partial_fit(self.reducer(i))
+ predicted = self.data.map_blocks(
+ lambda _i, s, e: np.asarray(
+ kmeans.predict(self.reducer(self.data._materialize_range(s, e)))
+ ),
+ nthreads=self.nthreads,
+ msg="Estimating seed partitions",
+ )
+ for part in predicted:
+ temp.extend(part)
self.clusterLabels = np.array(temp)
return kmeans
diff --git a/scarf/assay.py b/scarf/assay.py
index 6d5da626..a4ec14c6 100644
--- a/scarf/assay.py
+++ b/scarf/assay.py
@@ -9,85 +9,164 @@
method for feature selection.
"""
-from typing import Generator, List, Optional, Tuple, Union
+from collections.abc import Callable, Generator
+from typing import Any, cast
import numpy as np
import pandas as pd
import zarr
-from dask.array.core import Array as daskArrayType
-from dask.array.core import from_zarr
+from numpy.typing import NDArray
from scipy.sparse import csr_matrix, vstack
-from zarr import hierarchy as z_hierarchy
+from ._types import as_zarr_array, as_zarr_group
+from .chunked import ChunkedArray
from .metadata import MetaData
-from .utils import controlled_compute, logger, show_dask_progress
+from .utils import array_digest, controlled_compute, logger, show_dask_progress
__all__ = ["Assay", "RNAassay", "ATACassay", "ADTassay"]
+PSEUDOTIME_AGGREGATION_SCHEMA_VERSION = 2
-def norm_dummy(_, counts: daskArrayType) -> daskArrayType:
+type NormMethod = Callable[["Assay", ChunkedArray], ChunkedArray]
+type PercentFeatures = dict[str, str]
+
+
+def _read_block(
+ zarr_arr: zarr.Array,
+ row_idx: np.ndarray,
+ col_idx: np.ndarray,
+) -> np.ndarray:
+ """Read ``zarr_arr[row_idx, col_idx]`` returning rows/cols in index order.
+
+ A basic slice is used only for an index run that is provably contiguous
+ (consecutive ascending integers), so it selects exactly the requested
+ positions and never includes neighbouring rows or columns. Any other
+ selection falls back to orthogonal (fancy) indexing, which preserves the
+ order of the index arrays. This centralizes the read path so callers never
+ hand-roll ``slice(idx[0], idx[-1] + 1)``.
+ """
+ from .chunked import _is_contiguous
+
+ def axis_sel(idx: np.ndarray) -> slice | np.ndarray:
+ idx = np.asarray(idx)
+ if idx.size > 0 and _is_contiguous(idx):
+ return slice(int(idx[0]), int(idx[-1]) + 1)
+ return idx
+
+ row_sel = axis_sel(row_idx)
+ col_sel = axis_sel(col_idx)
+ if isinstance(row_sel, slice) and isinstance(col_sel, slice):
+ return np.asarray(zarr_arr[row_sel, col_sel])
+ return np.asarray(zarr_arr.get_orthogonal_selection((row_sel, col_sel)))
+
+
+def _feature_stats_tile_shape(
+ n_cells: int,
+ n_features: int,
+ *,
+ row_chunk: int,
+ col_chunk: int,
+ budget: Any | None = None,
+ target_bytes: int | None = None,
+) -> tuple[int, int]:
+ """Choose a dense stats tile that fits the active memory budget.
+
+ Full-width feature chunks (common on small matrices) would otherwise
+ materialize ``n_cells x n_features`` uint32+float64 temporaries at once.
+ """
+ from .storage.budget import ResourceBudget, get_resource_budget
+
+ if budget is None:
+ resolved = get_resource_budget()
+ elif isinstance(budget, ResourceBudget):
+ resolved = budget
+ else:
+ raise TypeError("budget must be a ResourceBudget or None")
+ work = resolved.memoryBytes // max(1, resolved.workingCopies)
+ if target_bytes is None:
+ target_bytes = min(work // 4, 256 * 1024 * 1024)
+ target_bytes = max(8 * 1024 * 1024, int(target_bytes))
+ rows = max(1, min(int(n_cells), max(1, int(row_chunk))))
+ cols = max(1, min(int(n_features), max(1, int(col_chunk))))
+ # One uint32 source band plus one float64 scaled band.
+ bytes_per_element = 12
+ while rows > 1 and rows * cols * bytes_per_element > target_bytes:
+ rows = max(1, rows // 2)
+ while cols > 1 and rows * cols * bytes_per_element > target_bytes:
+ cols = max(1, cols // 2)
+ return rows, cols
+
+
+def norm_dummy(_: "Assay", counts: ChunkedArray) -> ChunkedArray:
"""A dummy normalizer. Doesn't perform any normalization. This is useful
when the 'raw data' is already normalized.
Args:
_:
- counts: A dask array with 'raw' counts data
+ counts: A chunked array with 'raw' counts data
- Returns: Dask array
+ Returns: A chunked array
"""
return counts
-def norm_lib_size(assay, counts: daskArrayType) -> daskArrayType:
+def norm_lib_size(assay: "Assay", counts: ChunkedArray) -> ChunkedArray:
"""Performs library size normalization on the data. This is the default
method for RNA assays.
Args:
assay: An instance of the assay object
- counts: A dask array with raw counts data
+ counts: A chunked array with raw counts data
- Returns: A dask array (delayed matrix) containing normalized data.
+ Returns: A chunked array (delayed matrix) containing normalized data.
"""
+ assert assay.sf is not None and assay.scalar is not None
return assay.sf * counts / assay.scalar.reshape(-1, 1)
-def norm_lib_size_log(assay, counts: daskArrayType) -> daskArrayType:
+def norm_lib_size_log(assay: "Assay", counts: ChunkedArray) -> ChunkedArray:
"""Performs library size normalization and then transforms the values into
log scale.
Args:
assay: An instance of the assay object
- counts: A dask array with raw counts data
+ counts: A chunked array with raw counts data
- Returns: A dask array (delayed matrix) containing normalized data.
+ Returns: A chunked array (delayed matrix) containing normalized data.
"""
- return np.log1p(assay.sf * counts / assay.scalar.reshape(-1, 1))
+ assert assay.sf is not None and assay.scalar is not None
+ return cast(ChunkedArray, np.log1p(assay.sf * counts / assay.scalar.reshape(-1, 1)))
-def norm_clr(_, counts: daskArrayType) -> daskArrayType:
+def norm_clr(_: "Assay", counts: ChunkedArray) -> ChunkedArray:
"""Performs centered log-ratio normalization (ADT). This is the default
method for ADT assays.
Args:
_:
- counts: A dask array with raw counts data
+ counts: A chunked array with raw counts data
- Returns: A dask array (delayed matrix) containing normalized data.
+ Returns: A chunked array (delayed matrix) containing normalized data.
"""
- f = np.exp(np.log1p(counts).sum(axis=0) / len(counts))
- return np.log1p(counts / f)
+ f = np.exp(cast(NDArray[Any], np.log1p(counts).sum(axis=0)) / len(counts))
+ return cast(ChunkedArray, np.log1p(counts / f))
-def norm_tf_idf(assay, counts: daskArrayType) -> daskArrayType:
+def norm_tf_idf(assay: "Assay", counts: ChunkedArray) -> ChunkedArray:
"""Performs TF-IDF normalization This is the default method for ATAC
assays.
Args:
assay: An instance of the assay object
- counts: A dask array with raw counts data
+ counts: A chunked array with raw counts data
- Returns: A dask array (delayed matrix) containing normalized data.
+ Returns: A chunked array (delayed matrix) containing normalized data.
"""
+ assert (
+ assay.n_term_per_doc is not None
+ and assay.n_docs is not None
+ and assay.n_docs_per_term is not None
+ )
t_f = counts / assay.n_term_per_doc.reshape(-1, 1)
# TODO: Split TF and IDF functionality to make it similar to norml_lib and zscaling
idf = np.log2(1 + (assay.n_docs / (assay.n_docs_per_term + 1)))
@@ -100,18 +179,19 @@ class Assay:
for later KNN graph construction.
Args:
- z (z_hierarchy.Group): Zarr hierarchy where raw data is located
+ z (zarr.Group): Zarr hierarchy where raw data is located
+ workspace: Workspace name when assays live under ``matrices/`` (None for legacy layout)
name (str): A label/name for assay.
cell_data: Metadata class object for the cell attributes.
- nthreads: number for threads to use for dask parallel computations
- min_cells_per_feature:
+ nthreads: number of threads to use for parallel computations
+ min_cells_per_feature: Minimum cells expressing a feature for it to be kept
Attributes:
name: A label for the assay instance
z: Zarr group that contains the assay
cells: A Metadata class object for cell attributes
nthreads: number of threads to use for computations
- rawData: dask array containing the raw data
+ rawData: chunked array containing the raw data
feats: a MetaData class object for feature attributes
attrs: Zarr attributes for the zarr group of the assay
normMethod: normalization method to use.
@@ -120,38 +200,56 @@ class Assay:
def __init__(
self,
- z: z_hierarchy.Group,
- workspace: Union[str, None],
+ z: zarr.Group,
+ workspace: str | None,
name: str, # FIXME change to assay_name
cell_data: MetaData,
nthreads: int,
min_cells_per_feature: int = 10,
- ):
+ ) -> None:
self.name = name
self.cells = cell_data
self.nthreads = nthreads
if workspace is None:
- self.rawData = from_zarr(z[f"{name}/counts"], inline_array=True)
+ self.rawData = ChunkedArray(
+ as_zarr_array(z[f"{name}/counts"], name=f"{name}/counts"),
+ nthreads=nthreads,
+ )
self.feats = MetaData(z[f"{name}/featureData"]) # type: ignore
- self.z: zarr.Group = z[self.name] # type: ignore
+ self.z = as_zarr_group(z[self.name], name=self.name)
else:
- self.rawData = from_zarr(z[f"matrices/{name}/counts"], inline_array=True)
+ self.rawData = ChunkedArray(
+ as_zarr_array(
+ z[f"matrices/{name}/counts"], name=f"matrices/{name}/counts"
+ ),
+ nthreads=nthreads,
+ )
self.feats = MetaData(z[f"{workspace}/{name}/featureData"]) # type: ignore
- self.z = z[f"{workspace}/{name}"]
+ self.z = as_zarr_group(z[f"{workspace}/{name}"], name=f"{workspace}/{name}")
self.attrs = self.z.attrs
if "percentFeatures" not in self.attrs:
self.attrs["percentFeatures"] = {}
- self.normMethod = norm_dummy
- self.sf = None
+ self.normMethod: NormMethod = norm_dummy
+ self.sf: int | None = None
+ self.scalar: np.ndarray | None = None
+ self.n_term_per_doc: np.ndarray | None = None
+ self.n_docs: int | None = None
+ self.n_docs_per_term: np.ndarray | None = None
self._ini_feature_props(min_cells_per_feature)
+ def _percent_features(self) -> PercentFeatures:
+ raw = self.attrs.get("percentFeatures", {})
+ if not isinstance(raw, dict):
+ return {}
+ return {str(k): str(v) for k, v in raw.items()}
+
def normed(
self,
- cell_idx: Optional[np.ndarray] = None,
- feat_idx: Optional[np.ndarray] = None,
- **kwargs,
- ) -> daskArrayType:
- """This function normalizes the raw and returns a delayed dask array of
+ cell_idx: np.ndarray | None = None,
+ feat_idx: np.ndarray | None = None,
+ **kwargs: Any,
+ ) -> ChunkedArray:
+ """This function normalizes the raw and returns a delayed chunked array of
the normalized data.
Args:
@@ -163,7 +261,7 @@ def normed(
feature attribute table)
**kwargs:
- Returns: A dask array (delayed matrix) containing normalized data.
+ Returns: A chunked array (delayed matrix) containing normalized data.
"""
if cell_idx is None:
cell_idx = self.cells.active_index("I")
@@ -172,7 +270,7 @@ def normed(
counts = self.rawData[:, feat_idx][cell_idx, :]
return self.normMethod(self, counts)
- def to_raw_sparse(self, cell_key) -> csr_matrix:
+ def to_raw_sparse(self, cell_key: str) -> csr_matrix:
"""
Args:
@@ -235,13 +333,14 @@ def add_percent_feature(self, feat_pattern: str, name: str) -> None:
Returns:
"""
- if name in self.attrs["percentFeatures"]:
- if self.attrs["percentFeatures"][name] == feat_pattern:
+ if name in self._percent_features():
+ if self._percent_features()[name] == feat_pattern:
return None
else:
logger.info(f"Pattern for percentage feature {name} updated.")
+ percent_features = self._percent_features()
self.attrs["percentFeatures"] = {
- **{k: v for k, v in self.attrs["percentFeatures"].items()},
+ **percent_features,
**{name: feat_pattern},
}
feat_idx = sorted(
@@ -296,7 +395,7 @@ def _verify_keys(self, cell_key: str, feat_key: str) -> None:
def _get_cell_feat_idx(
self, cell_key: str, feat_key: str
- ) -> Tuple[np.ndarray, np.ndarray]:
+ ) -> tuple[np.ndarray, np.ndarray]:
"""Verifies the provided key by calling _verify_keys and fetches the
indices of rows that have True value in respective column.
@@ -329,7 +428,7 @@ def _create_subset_hash(cell_idx: np.ndarray, feat_idx: np.ndarray) -> int:
return hash(tuple([hash(tuple(cell_idx)), hash(tuple(feat_idx))]))
@staticmethod
- def _get_summary_stats_loc(cell_key: str) -> Tuple[str, str]:
+ def _get_summary_stats_loc(cell_key: str) -> tuple[str, str]:
"""A convenience method that returns the location of feature-wise
summary statistics Currently summaries are stored under pattern:
summary_stats_{cell_key}
@@ -392,11 +491,26 @@ def _load_stats_loc(self, cell_key: str) -> str:
f"Summary statistics have not been calculated for cell key: {cell_key}"
)
if identifier not in self.feats.locations:
- self.feats.mount_location(self.z[stats_loc], identifier)
+ self.feats.mount_location(
+ as_zarr_group(self.z[stats_loc], name=stats_loc), identifier
+ )
else:
logger.debug(f"Location ({stats_loc}) already mounted")
return identifier
+ @staticmethod
+ def _finalize_staged_mirror(
+ mirror: zarr.Array | None,
+ subset_hash: int,
+ subset_params: dict[str, Any],
+ ) -> None:
+ """Mark a mirrored staging array complete so staging reuses it as-is."""
+ if mirror is None:
+ return
+ mirror.attrs["staged_subset_hash"] = subset_hash
+ mirror.attrs["staged_subset_params"] = subset_params
+ mirror.attrs["staged_complete"] = True
+
def save_normalized_data(
self,
cell_key: str,
@@ -406,7 +520,8 @@ def save_normalized_data(
log_transform: bool,
renormalize_subset: bool,
update_keys: bool,
- ) -> daskArrayType:
+ mirror: zarr.Array | None = None,
+ ) -> ChunkedArray:
"""Create a new zarr group and saves the normalized data in the group
for the selected features only.
@@ -424,12 +539,12 @@ def save_normalized_data(
to the documentation of the 'normed' method of the RNAassay for
further description of this parameter.
update_keys: Whether to update the keys. If True then the 'latest_feat_key' and
- 'latest_feat_key' attributes of the assay will be updated. It can be useful
+ 'latest_cell_key' attributes of the assay will be updated. It can be useful
to set False in case where you only need to save the normalized data but
don't intend to use it directly. For example, when mapping onto a different
dataset and aligning features to that dataset.
- Returns: Dask array containing the normalized data
+ Returns: A chunked array containing the normalized data
"""
from .writers import dask_to_zarr
@@ -445,9 +560,10 @@ def save_normalized_data(
"renormalize_subset": renormalize_subset,
}
if location in self.z:
+ attrs = self.z[location].attrs
if (
- subset_hash == self.z[location].attrs["subset_hash"]
- and subset_params == self.z[location].attrs["subset_params"]
+ attrs.get("subset_hash") == subset_hash
+ and attrs.get("subset_params") == subset_params
):
logger.info(
f"Using existing normalized data with cell key {cell_key} and feat key {feat_key}"
@@ -457,7 +573,10 @@ def save_normalized_data(
feat_key.split("__", 1)[1] if feat_key != "I" else "I"
)
self.attrs["latest_cell_key"] = cell_key
- return from_zarr(self.z[location + "/data"], inline_array=True)
+ return ChunkedArray(
+ as_zarr_array(self.z[location + "/data"], name=location + "/data"),
+ nthreads=self.nthreads,
+ )
else:
# Creating group here to overwrite all children
self.z.create_group(location, overwrite=True)
@@ -467,25 +586,36 @@ def save_normalized_data(
log_transform=log_transform,
renormalize_subset=renormalize_subset,
)
- dask_to_zarr(vals, self.z, location + "/data", batch_size, self.nthreads)
+ dask_to_zarr(
+ vals,
+ self.z,
+ location + "/data",
+ vals.chunksize,
+ self.nthreads,
+ mirror=mirror,
+ )
self.z[location].attrs["subset_hash"] = subset_hash
self.z[location].attrs["subset_params"] = subset_params
+ self._finalize_staged_mirror(mirror, subset_hash, subset_params)
if update_keys:
self.attrs["latest_feat_key"] = (
feat_key.split("__", 1)[1] if feat_key != "I" else "I"
)
self.attrs["latest_cell_key"] = cell_key
- return from_zarr(self.z[location + "/data"], inline_array=True)
+ return ChunkedArray(
+ as_zarr_array(self.z[location + "/data"], name=location + "/data"),
+ nthreads=self.nthreads,
+ )
def iter_normed_feature_wise(
self,
- cell_key: Optional[str],
- feat_key: Optional[str],
+ cell_key: str | None,
+ feat_key: str | None,
batch_size: int,
- msg: Optional[str],
+ msg: str | None,
as_dataframe: bool = True,
- **norm_params,
- ) -> Generator[Union[pd.DataFrame, Tuple[np.ndarray, np.ndarray]], None, None]:
+ **norm_params: Any,
+ ) -> Generator[pd.DataFrame | tuple[np.ndarray, np.ndarray], None, None]:
"""This generator iterates over all the features marked by `feat_key`
in batches.
@@ -498,8 +628,10 @@ def iter_normed_feature_wise(
batch_size: Number of genes to be loaded in the memory at a time.
msg: Message to be displayed in the progress bar
as_dataframe: If true (default) then the yielded matrices are pandas dataframe
+ **norm_params: Extra keyword arguments forwarded to ``normed``.
Returns:
+ Generator yielding DataFrames or (matrix, feature index) tuples.
"""
from .utils import tqdmbar
@@ -515,7 +647,7 @@ def iter_normed_feature_wise(
if msg is None:
msg = ""
- data: daskArrayType = self.normed(
+ data: ChunkedArray = self.normed(
cell_idx=cell_idx,
feat_idx=feat_idx,
**norm_params,
@@ -537,7 +669,7 @@ def iter_normed_feature_wise(
)
def save_normed_for_query(
- self, feat_key: Optional[str], batch_size: int, overwrite: bool = True
+ self, feat_key: str | None, batch_size: int, overwrite: bool = True
) -> None:
"""This methods dumps normalized values for features (as marked by
`feat_key`) onto disk in the 'prenormed' slot under the assay's own
@@ -554,7 +686,7 @@ def save_normed_for_query(
Returns:
None
"""
- from joblib import Parallel, delayed
+ from concurrent.futures import ThreadPoolExecutor
from .writers import create_zarr_obj_array
@@ -569,10 +701,13 @@ def write_wrapper(idx: str, v: np.ndarray) -> None:
for mat, inds in self.iter_normed_feature_wise(
None, feat_key, batch_size, "Saving features", False
):
- Parallel(n_jobs=self.nthreads)(
- delayed(write_wrapper)(inds[i], mat[i])
- for i in range(len(inds)) # type: ignore
- )
+ write_args = ((inds[i], mat[i]) for i in range(len(inds))) # type: ignore
+ if self.nthreads > 1:
+ with ThreadPoolExecutor(max_workers=self.nthreads) as ex:
+ list(ex.map(lambda args: write_wrapper(*args), write_args))
+ else:
+ for args in write_args:
+ write_wrapper(*args)
def save_aggregated_ordering(
self,
@@ -585,36 +720,68 @@ def save_aggregated_ordering(
smoothen: bool = True,
z_scale: bool = True,
batch_size: int = 100,
- **norm_params,
- ):
- """
+ **norm_params: Any,
+ ) -> tuple[ChunkedArray, NDArray[Any]]:
+ """Bin normalized expression along a cell ordering and cache the result.
Args:
- cell_key: Name of the key (column) from cell attribute table. The data will be fetched for only those cells.
- feat_key: Name of the key (column) from feature attribute table.
- ordering_key:
- min_exp:
- window_size:
- chunk_size:
- smoothen:
- z_scale:
- batch_size:
- **norm_params:
+ cell_key: Boolean column in cell metadata selecting cells.
+ feat_key: Boolean column in feature metadata selecting features.
+ ordering_key: Cell metadata column with pseudotime or ordering values.
+ min_exp: Minimum mean expression to retain a feature.
+ window_size: Rolling window size for smoothing along ordering.
+ chunk_size: Number of ordering bins stored per feature row.
+ smoothen: Whether to apply rolling-window smoothing.
+ z_scale: Whether to z-scale values within each feature.
+ batch_size: Feature batch size for iteration.
+ **norm_params: Extra keyword arguments forwarded to ``normed``.
Returns:
-
+ None
"""
from .utils import rolling_window
from .writers import create_zarr_dataset
- cell_ordering = self.cells.fetch(ordering_key, key=cell_key)
+ cell_ordering = np.asarray(
+ self.cells.fetch(ordering_key, key=cell_key),
+ dtype=float,
+ )
cell_idx, feat_idx = self._get_cell_feat_idx(cell_key, feat_key)
- hashes = [hash(tuple(x)) for x in (cell_idx, feat_idx, cell_ordering)]
+ n_cells = cell_ordering.shape[0]
+ if cell_ordering.ndim != 1 or n_cells == 0:
+ raise ValueError("Cell ordering must be a non-empty one-dimensional array")
+ if not np.isfinite(cell_ordering).all():
+ raise ValueError("Cell ordering must contain only finite values")
+ if not isinstance(window_size, int) or isinstance(window_size, bool):
+ raise TypeError("window_size must be an integer")
+ if not isinstance(chunk_size, int) or isinstance(chunk_size, bool):
+ raise TypeError("chunk_size must be an integer")
+ if window_size <= 0:
+ raise ValueError("window_size must be greater than zero")
+ if chunk_size <= 0:
+ raise ValueError("chunk_size must be greater than zero")
+
+ effective_window = min(window_size, n_cells)
+ effective_bins = min(chunk_size, n_cells)
+ if effective_window != window_size:
+ logger.warning(
+ f"Reducing window_size from {window_size} to {effective_window} "
+ "for the selected cell count"
+ )
+ if effective_bins != chunk_size:
+ logger.warning(
+ f"Reducing chunk_size from {chunk_size} to {effective_bins} "
+ "for the selected cell count"
+ )
+
+ hashes = [array_digest(x) for x in (cell_idx, feat_idx, cell_ordering)]
params = {
"min_exp": min_exp,
"window_size": window_size,
+ "effective_window": effective_window,
"chunk_size": chunk_size,
+ "effective_bins": effective_bins,
"smoothen": smoothen,
"z_scale": z_scale,
"norm_params": norm_params,
@@ -626,6 +793,8 @@ def save_aggregated_ordering(
and hashes == self.z[location].attrs["hashes"]
and "params" in self.z[location].attrs
and params == self.z[location].attrs["params"]
+ and self.z[location].attrs.get("schema_version")
+ == PSEUDOTIME_AGGREGATION_SCHEMA_VERSION
):
logger.info(f"Using existing aggregated data from {location}")
else:
@@ -638,13 +807,13 @@ def save_aggregated_ordering(
location + "/data",
(batch_size,),
"float64",
- (feat_idx.shape[0], chunk_size),
+ (feat_idx.shape[0], effective_bins),
)
- ordering_idx = np.argsort(cell_ordering)
- feat_idx = []
- valid_feats = []
+ ordering_idx = np.argsort(cell_ordering, kind="stable")
+ stored_feat_idx: list[int] = []
+ valid_feat_flags: list[bool] = []
s = 0
- for df in self.iter_normed_feature_wise(
+ for item in self.iter_normed_feature_wise(
cell_key,
feat_key,
batch_size,
@@ -652,44 +821,102 @@ def save_aggregated_ordering(
True,
**norm_params,
):
- valid_feats.extend(list((df.mean() > min_exp).values))
- feat_idx.extend(list(df.columns))
+ df = cast(pd.DataFrame, item)
+ stored_feat_idx.extend(list(df.columns))
+ ordered = df.iloc[ordering_idx].to_numpy(dtype=float)
+ if not np.isfinite(ordered).all():
+ invalid_columns = np.asarray(df.columns)[
+ ~np.isfinite(ordered).all(axis=0)
+ ]
+ raise ValueError(
+ f"Normalized features contain non-finite values: "
+ f"{invalid_columns.tolist()}"
+ )
if smoothen:
- df = rolling_window(df.reindex(ordering_idx).values, window_size)
+ ordered = rolling_window(ordered, effective_window)
+ if not np.isfinite(ordered).all():
+ raise ValueError(
+ "Smoothed feature profiles contain non-finite values"
+ )
+
+ mean_expression = df.mean(axis=0).to_numpy(dtype=float)
+ standard_deviation = ordered.std(axis=0)
+ valid_features = (
+ (mean_expression > min_exp)
+ & np.isfinite(standard_deviation)
+ & (standard_deviation > np.finfo(float).eps)
+ )
+ valid_feat_flags.extend(valid_features.tolist())
if z_scale:
- df = (df - df.mean(axis=0)) / df.std(axis=0)
- df_mean = np.array(
- [x.mean(axis=0) for x in np.array_split(df, chunk_size)]
- ).T
+ processed = np.zeros_like(ordered, dtype=float)
+ processed[:, valid_features] = (
+ ordered[:, valid_features]
+ - ordered[:, valid_features].mean(axis=0)
+ ) / standard_deviation[valid_features]
+ else:
+ processed = ordered.copy()
+ processed[:, ~valid_features] = 0.0
+ df_mean = np.stack(
+ [
+ values.mean(axis=0)
+ for values in np.array_split(
+ processed,
+ effective_bins,
+ axis=0,
+ )
+ ],
+ axis=1,
+ )
+ if not np.isfinite(df_mean).all():
+ raise ValueError(
+ "Binned feature profiles contain non-finite values"
+ )
g[s : s + df_mean.shape[0]] = df_mean
s += df_mean.shape[0]
g = create_zarr_dataset(
self.z,
location + "/feature_indices",
- (len(feat_idx),),
+ (len(stored_feat_idx),),
"uint64",
- (len(feat_idx),),
+ (len(stored_feat_idx),),
)
- g[:] = np.array(feat_idx).astype(int)
+ g[:] = np.array(stored_feat_idx).astype(int)
g = create_zarr_dataset(
self.z,
location + "/valid_features",
- (len(feat_idx),),
+ (len(stored_feat_idx),),
"bool",
- (len(feat_idx),),
+ (len(stored_feat_idx),),
)
- g[:] = np.array(valid_feats).astype(int)
+ g[:] = np.array(valid_feat_flags).astype(int)
self.z[location].attrs["hashes"] = hashes
- self.z[location].attrs["params"] = params
+ self.z[location].attrs["params"] = cast(Any, params)
+ self.z[location].attrs["schema_version"] = (
+ PSEUDOTIME_AGGREGATION_SCHEMA_VERSION
+ )
- ret_val1 = from_zarr(self.z[location + "/data"], inline_array=True)
- ret_val2 = self.z[location + "/feature_indices"][:]
+ ret_val1 = ChunkedArray(
+ as_zarr_array(self.z[location + "/data"], name=location + "/data"),
+ nthreads=self.nthreads,
+ )
+ ret_val2 = np.asarray(
+ as_zarr_array(
+ self.z[location + "/feature_indices"],
+ name=location + "/feature_indices",
+ )[:]
+ )
if location + "/valid_features" in self.z:
- valid_feats = self.z[location + "/valid_features"][:]
+ valid_feats = np.asarray(
+ as_zarr_array(
+ self.z[location + "/valid_features"],
+ name=location + "/valid_features",
+ )[:],
+ dtype=bool,
+ )
ret_val1 = ret_val1[valid_feats]
ret_val2 = ret_val2[valid_feats]
@@ -697,7 +924,7 @@ def save_aggregated_ordering(
def score_features(
self,
- feature_names: List[str],
+ feature_names: list[str],
cell_key: str,
ctrl_size: int,
n_bins: int,
@@ -720,12 +947,19 @@ def score_features(
from .feat_utils import binned_sampling
- def _names_to_idx(i):
+ def _names_to_idx(i: list[str]) -> np.ndarray:
return self.feats.get_index_by(i, "names", None)
- def _calc_mean(i):
+ def _calc_mean(i: np.ndarray | list[str] | list[int]) -> np.ndarray:
+ if isinstance(i, list):
+ if not i or isinstance(i[0], str):
+ feat_selection = self.feats.get_index_by(i, "names", None)
+ else:
+ feat_selection = np.asarray(i, dtype=int)
+ else:
+ feat_selection = i
return (
- self.normed(cell_idx=cell_idx, feat_idx=np.array(sorted(i)))
+ self.normed(cell_idx=cell_idx, feat_idx=np.sort(feat_selection))
.mean(axis=1)
.compute()
)
@@ -742,9 +976,18 @@ def _calc_mean(i):
obs_avg, list(feature_idx), ctrl_size, n_bins, rand_seed
)
cell_idx, _ = self._get_cell_feat_idx(cell_key, "I")
- return _calc_mean(feature_idx) - _calc_mean(control_idx)
+ if isinstance(self, RNAassay) and self.normMethod is norm_lib_size:
+ means = self._mean_normed_feature_groups(
+ cell_idx,
+ {
+ "target": np.asarray(feature_idx, dtype=int),
+ "control": np.asarray(control_idx, dtype=int),
+ },
+ )
+ return np.asarray(means["target"] - means["control"])
+ return np.asarray(_calc_mean(feature_idx) - _calc_mean(control_idx))
- def __repr__(self):
+ def __repr__(self) -> str:
f = self.feats.fetch_all("I")
assay_name = str(self.__class__).split(".")[-1][:-2]
return f"{assay_name} {self.name} with {f.sum()}({len(f)}) features"
@@ -755,7 +998,7 @@ class RNAassay(Assay):
normalization of scRNA-Seq data.
Args:
- z (z_hierarchy.Group): Zarr hierarchy where raw data is located
+ z (zarr.Group): Zarr hierarchy where raw data is located
name (str): A label/name for assay.
cell_data: Metadata class object for the cell attributes.
**kwargs: kwargs to be passed to the Assay class
@@ -767,25 +1010,119 @@ class RNAassay(Assay):
It is set to None until normed method is called.
"""
- def __init__(self, z: z_hierarchy.Group, name: str, cell_data: MetaData, **kwargs):
- super().__init__(z=z, name=name, cell_data=cell_data, **kwargs)
+ def __init__(
+ self,
+ z: zarr.Group,
+ name: str,
+ cell_data: MetaData,
+ *,
+ workspace: str | None = None,
+ nthreads: int = 1,
+ min_cells_per_feature: int = 10,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(
+ z=z,
+ workspace=workspace,
+ name=name,
+ cell_data=cell_data,
+ nthreads=nthreads,
+ min_cells_per_feature=min_cells_per_feature,
+ **kwargs,
+ )
self.normMethod = norm_lib_size
if "size_factor" in self.attrs:
- self.sf = int(self.attrs["size_factor"])
+ self.sf = int(cast(int, self.attrs["size_factor"]))
else:
self.sf = 1000
self.attrs["size_factor"] = self.sf
- self.scalar = None
+ self.scalar: np.ndarray | None = None
+
+ def save_normalized_data(
+ self,
+ cell_key: str,
+ feat_key: str,
+ batch_size: int,
+ location: str,
+ log_transform: bool,
+ renormalize_subset: bool,
+ update_keys: bool,
+ mirror: zarr.Array | None = None,
+ ) -> ChunkedArray:
+ if not renormalize_subset:
+ return super().save_normalized_data(
+ cell_key,
+ feat_key,
+ batch_size,
+ location,
+ log_transform,
+ renormalize_subset,
+ update_keys,
+ mirror=mirror,
+ )
+
+ from .writers import write_renorm_subset_to_zarr
+
+ if feat_key != "I":
+ feat_key = cell_key + "__" + feat_key
+ cell_idx, feat_idx = self._get_cell_feat_idx(cell_key, feat_key)
+ subset_hash = self._create_subset_hash(cell_idx, feat_idx)
+ subset_params = {
+ "log_transform": log_transform,
+ "renormalize_subset": renormalize_subset,
+ }
+ if location in self.z:
+ attrs = self.z[location].attrs
+ if (
+ attrs.get("subset_hash") == subset_hash
+ and attrs.get("subset_params") == subset_params
+ ):
+ logger.info(
+ f"Using existing normalized data with cell key {cell_key} and feat key {feat_key}"
+ )
+ if update_keys:
+ self.attrs["latest_feat_key"] = (
+ feat_key.split("__", 1)[1] if feat_key != "I" else "I"
+ )
+ self.attrs["latest_cell_key"] = cell_key
+ return ChunkedArray(
+ as_zarr_array(self.z[location + "/data"], name=location + "/data"),
+ nthreads=self.nthreads,
+ )
+ self.z.create_group(location, overwrite=True)
+
+ write_renorm_subset_to_zarr(
+ self,
+ cell_idx,
+ feat_idx,
+ self.z,
+ location + "/data",
+ self.nthreads,
+ log_transform=log_transform,
+ mirror=mirror,
+ )
+ self.z[location].attrs["subset_hash"] = subset_hash
+ self.z[location].attrs["subset_params"] = subset_params
+ self._finalize_staged_mirror(mirror, subset_hash, subset_params)
+ if update_keys:
+ self.attrs["latest_feat_key"] = (
+ feat_key.split("__", 1)[1] if feat_key != "I" else "I"
+ )
+ self.attrs["latest_cell_key"] = cell_key
+ return ChunkedArray(
+ as_zarr_array(self.z[location + "/data"], name=location + "/data"),
+ nthreads=self.nthreads,
+ )
def normed(
self,
- cell_idx: Optional[np.ndarray] = None,
- feat_idx: Optional[np.ndarray] = None,
+ cell_idx: np.ndarray | None = None,
+ feat_idx: np.ndarray | None = None,
renormalize_subset: bool = False,
log_transform: bool = False,
- **kwargs,
- ) -> daskArrayType:
- """This function normalizes the raw and returns a delayed dask array of
+ **kwargs: Any,
+ ) -> ChunkedArray:
+ """This function normalizes the raw and returns a delayed chunked array of
the normalized data. Unlike the `normed` method in the generic Assay
class this method is optimized for scRNA-Seq data and takes additional
parameters that will be used by `norm_lib_size` (default normalization
@@ -805,7 +1142,7 @@ class this method is optimized for scRNA-Seq data and takes additional
**kwargs: kwargs have no effect here.
Returns:
- A dask array (delayed matrix) containing normalized data.
+ A chunked array (delayed matrix) containing normalized data.
"""
if cell_idx is None:
cell_idx = self.cells.active_index("I")
@@ -827,6 +1164,292 @@ class this method is optimized for scRNA-Seq data and takes additional
self.normMethod = norm_method_cache
return val
+ def iter_raw_column_blocks(
+ self,
+ cell_idx: np.ndarray,
+ feat_idx: np.ndarray,
+ batch_size: int,
+ msg: str | None = None,
+ ) -> Generator[tuple[int, np.ndarray, np.ndarray, float, str], None, None]:
+ """Read raw count column batches with remote-aware staging.
+
+ Yields ``(block_idx, raw, feat_cols, read_sec, source)`` where ``raw`` has
+ shape ``(len(cell_idx), len(feat_cols))``.
+ """
+ from .storage.zarr_store import is_remote_datastore
+ from .utils import iter_column_blocks
+
+ zarr_arr = cast(zarr.Array, self.rawData._backing)
+ cell_idx = np.asarray(cell_idx)
+ feat_idx = np.asarray(feat_idx)
+ batch_size = max(1, batch_size)
+ batches = [
+ feat_idx[s : s + batch_size] for s in range(0, len(feat_idx), batch_size)
+ ]
+ n_blocks = len(batches)
+ if msg:
+ logger.debug(
+ f"({self.name}) {msg}: {len(feat_idx)} features in "
+ f"{n_blocks} batches (width {batch_size})"
+ )
+
+ def read_block(block_idx: int) -> np.ndarray:
+ return _read_block(zarr_arr, cell_idx, batches[block_idx])
+
+ remote = is_remote_datastore("", self.z)
+ for block_idx, raw, read_sec, source in iter_column_blocks(
+ n_blocks,
+ read_block,
+ remote=remote,
+ msg=msg,
+ ):
+ yield block_idx, raw, batches[block_idx], read_sec, source
+
+ def iter_raw_feature_columns(
+ self,
+ cell_idx: np.ndarray,
+ feat_idx: np.ndarray,
+ batch_size: int,
+ scalar: np.ndarray,
+ sf: float,
+ log_transform: bool = False,
+ prefetch_depth: int = 1,
+ msg: str | None = None,
+ ) -> Generator[tuple[np.ndarray, np.ndarray], None, None]:
+ """Iterate library-size normalized feature columns without streaming
+ the full normalized matrix.
+
+ Raw count columns are read directly from the backing Zarr array in
+ chunk-aligned batches and normalized in memory using a precomputed
+ per-cell scalar (library size). Reads are prefetched in parallel.
+
+ Args:
+ cell_idx: Integer indices of cells to include (in output order).
+ feat_idx: Integer indices of features to iterate over.
+ batch_size: Number of feature columns per batch.
+ scalar: Per-cell normalization factor aligned to ``cell_idx``.
+ sf: Size factor multiplier applied before dividing by ``scalar``.
+ log_transform: If True, apply ``log1p`` after normalization.
+ prefetch_depth: Number of batches to read ahead in parallel.
+ msg: Progress bar description.
+
+ Yields:
+ Tuples of ``(normed_batch, feat_index_batch)`` where ``normed_batch``
+ has shape ``(len(cell_idx), batch_columns)``.
+ """
+ import time
+
+ from .utils import process_rss_mb
+
+ cell_idx = np.asarray(cell_idx)
+ scalar_col = np.asarray(scalar, dtype=np.float32).reshape(-1, 1)
+ scalar_col[scalar_col == 0] = 1
+ feat_idx = np.asarray(feat_idx)
+ n_batches = max(
+ 1, (len(feat_idx) + max(1, batch_size) - 1) // max(1, batch_size)
+ )
+
+ for block_idx, raw, cols, read_sec, source in self.iter_raw_column_blocks(
+ cell_idx=cell_idx,
+ feat_idx=feat_idx,
+ batch_size=batch_size,
+ msg=msg,
+ ):
+ t0 = time.perf_counter()
+ normed = (sf * raw.astype(np.float32)) / scalar_col
+ if log_transform:
+ normed = np.log1p(normed)
+ if msg:
+ logger.debug(
+ f"({self.name}) {msg} batch {block_idx + 1}/{n_batches}: "
+ f"cols={len(cols)} read {read_sec:.1f}s ({source}) "
+ f"norm {time.perf_counter() - t0:.1f}s "
+ f"rss {process_rss_mb():.0f} MiB"
+ )
+ yield normed, cols
+
+ def _mean_normed_feature_groups(
+ self,
+ cell_idx: np.ndarray,
+ feature_groups: dict[str, np.ndarray],
+ block_rows: int | None = None,
+ ) -> dict[str, np.ndarray]:
+ """Per-cell mean of library-size normalized counts for each feature group.
+
+ Reads the union of all requested feature columns once and streams over
+ row blocks aligned to the array's on-disk row chunk. This avoids the
+ full ChunkedArray normalization path (and
+ its repeated wide-chunk reads) used by ``normed`` when scoring small,
+ scattered gene sets such as cell cycle markers. Values are computed in
+ float64 to match ``norm_lib_size``. Row blocks are read ahead in
+ parallel and accumulated as they arrive (each writes a disjoint row
+ slice, so order does not matter).
+ """
+ from .storage.budget import worker_prefetch_depth
+ from .utils import prefetch_blocks
+
+ zarr_arr = cast(zarr.Array, self.rawData._backing)
+ cell_idx = np.asarray(cell_idx)
+ if self.normMethod is norm_lib_size and self.sf is None:
+ raise ValueError(
+ "RNA library-size normalization requires a size factor (sf), got None"
+ )
+ sf = float(self.sf) if self.sf is not None else 1.0
+ scalar = np.asarray(
+ self.cells.fetch_all(self.name + "_nCounts")[cell_idx], dtype=np.float64
+ )
+ scalar[scalar == 0] = 1
+
+ union = np.unique(
+ np.concatenate([np.asarray(v, dtype=int) for v in feature_groups.values()])
+ )
+ local_pos = {
+ key: np.searchsorted(union, np.asarray(idx, dtype=int))
+ for key, idx in feature_groups.items()
+ }
+
+ n_cells = len(cell_idx)
+ out = {key: np.empty(n_cells, dtype=np.float64) for key in feature_groups}
+ if n_cells == 0:
+ return out
+
+ if block_rows is None:
+ chunks = getattr(zarr_arr, "chunks", None)
+ block_rows = int(chunks[0]) if chunks else n_cells
+ block_rows = max(1, int(block_rows))
+
+ starts = range(0, n_cells, block_rows)
+
+ def read(start: int) -> tuple[int, np.ndarray]:
+ rows = cell_idx[start : start + block_rows]
+ return start, _read_block(zarr_arr, rows, union)
+
+ max_ahead = worker_prefetch_depth()
+ for start, raw in prefetch_blocks(starts, read, max_ahead=max_ahead):
+ end = start + raw.shape[0]
+ normed = (sf * raw.astype(np.float64)) / scalar[start:end, None]
+ for key, pos in local_pos.items():
+ out[key][start:end] = normed[:, pos].mean(axis=1)
+ return out
+
+ def _streaming_feature_stats(
+ self,
+ cell_idx: np.ndarray,
+ feat_idx: np.ndarray,
+ ) -> dict[str, np.ndarray]:
+ """Per-feature library-size normalized stats in one streaming pass.
+
+ Decodes each physical Zarr chunk at most once for the selected cells and
+ features, normalizes dense sub-tiles in float64 in place, and accumulates
+ per-feature nonzero count, sum, and sum of squares. Remote stores may
+ prefetch the next physical chunk while compute continues. Values match
+ ``norm_lib_size``. Returns ``normed_tot`` (sum), ``normed_n`` (nonzero
+ count), and ``sigmas`` (population variance).
+ """
+ import time
+
+ from .storage.zarr_store import is_remote_datastore
+ from .utils import iter_column_blocks, process_rss_mb
+
+ zarr_arr = cast(zarr.Array, self.rawData._backing)
+ cell_idx = np.asarray(cell_idx)
+ feat_idx = np.asarray(feat_idx)
+ if self.normMethod is norm_lib_size and self.sf is None:
+ raise ValueError(
+ "RNA library-size normalization requires a size factor (sf), got None"
+ )
+ sf = float(self.sf) if self.sf is not None else 1.0
+ scalar = np.asarray(
+ self.cells.fetch_all(self.name + "_nCounts")[cell_idx], dtype=np.float64
+ )
+ scalar[scalar == 0] = 1
+ inv_scalar = 1.0 / scalar
+
+ n_features = len(feat_idx)
+ n_cells = len(cell_idx)
+ nz = np.zeros(n_features, dtype=np.float64)
+ s1 = np.zeros(n_features, dtype=np.float64)
+ s2 = np.zeros(n_features, dtype=np.float64)
+ if n_cells == 0 or n_features == 0:
+ return {"normed_tot": s1, "normed_n": nz, "sigmas": s2}
+
+ chunks = getattr(zarr_arr, "chunks", None)
+ row_chunk = int(chunks[0]) if chunks and len(chunks) > 0 else n_cells
+ col_chunk = int(chunks[1]) if chunks and len(chunks) > 1 else n_features
+ row_chunk = max(1, row_chunk)
+ col_chunk = max(1, col_chunk)
+
+ cell_pos = np.arange(n_cells, dtype=np.intp)
+ feat_pos = np.arange(n_features, dtype=np.intp)
+ cell_bins = np.asarray(cell_idx // row_chunk, dtype=np.intp)
+ feat_bins = np.asarray(feat_idx // col_chunk, dtype=np.intp)
+
+ tiles: list[tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]] = []
+ for row_bin in np.unique(cell_bins):
+ cell_mask = cell_bins == row_bin
+ local_cells = cell_pos[cell_mask]
+ rows = cell_idx[cell_mask]
+ for col_bin in np.unique(feat_bins):
+ feat_mask = feat_bins == col_bin
+ local_feats = feat_pos[feat_mask]
+ cols = feat_idx[feat_mask]
+ tiles.append((local_cells, rows, local_feats, cols))
+
+ n_blocks = len(tiles)
+ remote = is_remote_datastore("", self.z)
+ sub_rows, sub_cols = _feature_stats_tile_shape(
+ max((len(rows) for _, rows, _, _ in tiles), default=1),
+ max((len(cols) for _, _, _, cols in tiles), default=1),
+ row_chunk=row_chunk,
+ col_chunk=col_chunk,
+ )
+
+ def read_block(block_idx: int) -> np.ndarray:
+ _, rows, _, cols = tiles[block_idx]
+ return _read_block(zarr_arr, rows, cols)
+
+ def accumulate_block(
+ raw: np.ndarray,
+ local_cells: np.ndarray,
+ local_feats: np.ndarray,
+ ) -> None:
+ height = raw.shape[0]
+ width = raw.shape[1]
+ for row_start in range(0, height, sub_rows):
+ row_end = min(height, row_start + sub_rows)
+ local_inv = inv_scalar[local_cells[row_start:row_end]]
+ for col_start in range(0, width, sub_cols):
+ col_end = min(width, col_start + sub_cols)
+ band = raw[row_start:row_end, col_start:col_end]
+ feat_slice = local_feats[col_start:col_end]
+ nz[feat_slice] += (band > 0).sum(axis=0)
+ scaled = band.astype(np.float64, copy=True)
+ scaled *= sf
+ np.multiply(scaled, local_inv[:, None], out=scaled)
+ s1[feat_slice] += scaled.sum(axis=0)
+ s2[feat_slice] += np.einsum("ij,ij->j", scaled, scaled)
+
+ for block_idx, raw, read_sec, source in iter_column_blocks(
+ n_blocks,
+ read_block,
+ remote=remote,
+ msg=f"({self.name}) Computing feature stats",
+ ):
+ local_cells, _, local_feats, _ = tiles[block_idx]
+ t_compute = time.perf_counter()
+ accumulate_block(raw, local_cells, local_feats)
+ compute_sec = time.perf_counter() - t_compute
+ logger.info(
+ f"({self.name}) feature stats block {block_idx + 1}/{n_blocks}: "
+ f"read {read_sec:.1f}s ({source}) compute {compute_sec:.1f}s "
+ f"rss {process_rss_mb():.0f} MiB"
+ )
+ del raw
+
+ mean = s1 / n_cells
+ sigmas = s2 / n_cells - np.square(mean)
+ return {"normed_tot": s1, "normed_n": nz, "sigmas": sigmas}
+
def set_feature_stats(self, cell_key: str) -> None:
"""Calculates summary statistics for the features of the assay using
only cells that are marked True by the 'cell_key' parameter.
@@ -845,33 +1468,49 @@ def set_feature_stats(self, cell_key: str) -> None:
else:
if identifier in self.feats.locations:
del self.feats.locations[identifier]
- n_cells = show_dask_progress(
- (self.normed(cell_idx, feat_idx) > 0).sum(axis=0),
- f"({self.name}) Computing nCells",
- self.nthreads,
- )
- tot = show_dask_progress(
- self.normed(cell_idx, feat_idx).sum(axis=0),
- f"({self.name}) Computing normed_tot",
- self.nthreads,
- )
- sigmas = show_dask_progress(
- self.normed(cell_idx, feat_idx).var(axis=0),
- f"({self.name}) Computing sigmas",
- self.nthreads,
- )
+ n_used = int(len(cell_idx))
+ # The single-pass streaming path only implements the library-size
+ # normalization formula (sf * raw / nCounts). Any other norm method
+ # (e.g. log-transformed or renormalized variants) falls back to the
+ # generic ChunkedArray reductions, which honour self.normMethod.
+ if self.normMethod is norm_lib_size:
+ stats = self._streaming_feature_stats(cell_idx, feat_idx)
+ n_cells = stats["normed_n"]
+ tot = stats["normed_tot"]
+ sigmas = stats["sigmas"]
+ else:
+ n_cells = show_dask_progress(
+ (self.normed(cell_idx, feat_idx) > 0).sum(axis=0),
+ f"({self.name}) Computing nCells",
+ self.nthreads,
+ )
+ tot = show_dask_progress(
+ self.normed(cell_idx, feat_idx).sum(axis=0),
+ f"({self.name}) Computing normed_tot",
+ self.nthreads,
+ )
+ sigmas = show_dask_progress(
+ self.normed(cell_idx, feat_idx).var(axis=0),
+ f"({self.name}) Computing sigmas",
+ self.nthreads,
+ )
# idx = n_cells > min_cells
# self.feats.update_key(idx, key=feat_key)
# n_cells, tot, sigmas = n_cells[idx], tot[idx], sigmas[idx]
self.z.create_group(stats_loc, overwrite=True)
- self.feats.mount_location(self.z[stats_loc], identifier)
+ self.feats.mount_location(
+ as_zarr_group(self.z[stats_loc], name=stats_loc), identifier
+ )
self.feats.insert(
"normed_tot", tot.astype(float), overwrite=True, location=identifier
)
+ # Mean over the cells actually used (cell_key subset), matching the
+ # denominator of the variance computed above. self.cells.N counts all
+ # primary cells, including those filtered out, so it is not used here.
self.feats.insert(
"avg",
- (tot / self.cells.N).astype(float),
+ (tot / max(1, n_used)).astype(float),
overwrite=True,
location=identifier,
)
@@ -897,8 +1536,8 @@ def set_feature_stats(self, cell_key: str) -> None:
return None
def set_summary_stats(
- self, cell_key: str = None, n_bins: int = 200, lowess_frac: float = 0.1
- ) -> Tuple[str, str]:
+ self, cell_key: str | None = None, n_bins: int = 200, lowess_frac: float = 0.1
+ ) -> tuple[str, str]:
"""Calculates summary statistics for the features of the assay using only cells that are marked True by the 'cell_key' parameter.
Args:
@@ -913,7 +1552,7 @@ def set_summary_stats(
c_var_col: The name of the column in the feature attribute table that contains the corrected variance values.
"""
- def col_renamer(x):
+ def col_renamer(x: str) -> str:
return f"{identifier}_{x}"
if cell_key is None:
@@ -957,8 +1596,8 @@ def mark_hvgs(
hvg_key_name: str,
keep_bounds: bool,
show_plot: bool,
- max_cells: int,
- **plot_kwargs,
+ max_cells: int | float,
+ **plot_kwargs: Any,
) -> None:
"""Identifies highly variable genes in the dataset.
@@ -1004,7 +1643,7 @@ def mark_hvgs(
**plot_kwargs: Keyword arguments for matplotlib.pyplot.scatter function
"""
- def col_renamer(x):
+ def col_renamer(x: str) -> str:
return f"{identifier}_{x}"
logger.info("Calculating summary statistics")
@@ -1039,11 +1678,17 @@ def col_renamer(x):
keep_bounds=keep_bounds,
)
idx = idx & self.feats.fetch_all("I") & bl
- n_valid_feats = idx.sum()
- if top_n > n_valid_feats:
+ n_valid_feats = int(idx.sum())
+ if n_valid_feats == 0:
+ raise ValueError(
+ "No features passed HVG candidate filters "
+ f"(min_cells={min_cells}, max_cells={max_cells}, "
+ f"min_mean={min_mean}, max_mean={max_mean})."
+ )
+ if top_n >= n_valid_feats:
logger.warning(
f"WARNING: Number of valid features are less then value "
- f"of parameter `top_n`: {top_n}. Resetting `top_n` to {n_valid_feats}"
+ f"of parameter `top_n`: {top_n}. Resetting `top_n` to {n_valid_feats - 1}"
)
top_n = n_valid_feats - 1
min_var = (
@@ -1078,12 +1723,22 @@ class ATACassay(Assay):
"""This subclass of Assay is designed for feature selection and
normalization of scATAC-Seq data."""
- def __init__(self, z: z_hierarchy.Group, name: str, cell_data: MetaData, **kwargs):
+ def __init__(
+ self,
+ z: zarr.Group,
+ name: str,
+ cell_data: MetaData,
+ *,
+ workspace: str | None = None,
+ nthreads: int = 1,
+ min_cells_per_feature: int = 10,
+ **kwargs: Any,
+ ) -> None:
"""This Assay subclass is designed for feature selection and
normalization of scATAC-Seq data.
Args:
- z (z_hierarchy.Group): Zarr hierarchy where raw data is located
+ z (zarr.Group): Zarr hierarchy where raw data is located
name (str): A label/name for assay.
cell_data: Metadata class object for the cell attributes.
**kwargs:
@@ -1094,22 +1749,30 @@ def __init__(self, z: z_hierarchy.Group, name: str, cell_data: MetaData, **kwarg
n_docs: Number of cells. Used for TF-IDF normalization
n_docs_per_term: Number of cells per feature. Used for TF-IDF normalization
"""
- super().__init__(z=z, name=name, cell_data=cell_data, **kwargs)
+ super().__init__(
+ z=z,
+ workspace=workspace,
+ name=name,
+ cell_data=cell_data,
+ nthreads=nthreads,
+ min_cells_per_feature=min_cells_per_feature,
+ **kwargs,
+ )
self.normMethod = norm_tf_idf
- self.n_term_per_doc = None
- self.n_docs = None
- self.n_docs_per_term = None
+ self.n_term_per_doc: np.ndarray | None = None
+ self.n_docs: int | None = None
+ self.n_docs_per_term: np.ndarray | None = None
def normed(
self,
- cell_idx: Optional[np.ndarray] = None,
- feat_idx: Optional[np.ndarray] = None,
- **kwargs,
- ) -> daskArrayType:
- """This function normalizes the raw and returns a delayed dask array of
+ cell_idx: np.ndarray | None = None,
+ feat_idx: np.ndarray | None = None,
+ **kwargs: Any,
+ ) -> ChunkedArray:
+ """This function normalizes the raw and returns a delayed chunked array of
the normalized data. Unlike the `normed` method in the generic Assay
class this method is optimized for scATAC-Seq data. This method uses
- the the normalization indicated by attribute self.normMethod which by
+ the normalization indicated by attribute self.normMethod which by
default is set to `norm_tf_idf`. The TF-IDF normalization is performed
using only the cells and features indicated by the 'cell_idx' and
'feat_idx' parameters.
@@ -1123,13 +1786,13 @@ class this method is optimized for scATAC-Seq data. This method uses
feature attribute table)
**kwargs:
- Returns: A dask array (delayed matrix) containing normalized data.
+ Returns: A chunked array (delayed matrix) containing normalized data.
"""
if cell_idx is None:
cell_idx = self.cells.active_index("I")
if feat_idx is None:
feat_idx = self.feats.active_index("I")
- counts: daskArrayType = self.rawData[:, feat_idx][cell_idx, :]
+ counts: ChunkedArray = self.rawData[:, feat_idx][cell_idx, :]
self.n_term_per_doc = self.cells.fetch_all(self.name + "_nFeatures")[cell_idx]
self.n_docs = len(cell_idx)
self.n_docs_per_term = self.feats.fetch_all("nCells")[feat_idx]
@@ -1157,7 +1820,9 @@ def set_feature_stats(self, cell_key: str) -> None:
self.nthreads,
)
self.z.create_group(stats_loc, overwrite=True)
- self.feats.mount_location(self.z[stats_loc], identifier)
+ self.feats.mount_location(
+ as_zarr_group(self.z[stats_loc], name=stats_loc), identifier
+ )
self.feats.insert(
"prevalence", prevalence.astype(float), overwrite=True, location=identifier
)
@@ -1175,8 +1840,7 @@ def mark_prevalent_peaks(
Args:
cell_key: Cells to use for selection of most prevalent peaks. The provided value for `cell_key` should be a
column in cell attributes table with boolean values.
- top_n: Number of top prevalent peaks to be selected. This value is ignored if a value is provided
- for `min_var` parameter.
+ top_n: Number of top prevalent peaks to be selected. (Default: 500)
prevalence_key_name: Base label for marking prevalent peaks in the features attributes column. The value for
'cell_key' parameter is prepended to this value.
@@ -1210,7 +1874,7 @@ class ADTassay(Assay):
(feature-barcodes library) data from CITE-Seq experiments.
Args:
- z (z_hierarchy.Group): Zarr hierarchy where raw data is located
+ z (zarr.Group): Zarr hierarchy where raw data is located
name (str): A label/name for assay.
cell_data: Metadata class object for the cell attributes.
**kwargs:
@@ -1219,20 +1883,44 @@ class ADTassay(Assay):
normMethod: Pointer to the function to be used for normalization of the raw data
"""
- def __init__(self, z: z_hierarchy.Group, name: str, cell_data: MetaData, **kwargs):
- """This subclass of Assay is designed for normalization of ADT/HTO
- (feature-barcodes library) data from CITE-Seq experiments."""
- super().__init__(z=z, name=name, cell_data=cell_data, **kwargs)
+ def __init__(
+ self,
+ z: zarr.Group,
+ name: str,
+ cell_data: MetaData,
+ *,
+ workspace: str | None = None,
+ nthreads: int = 1,
+ min_cells_per_feature: int = 10,
+ **kwargs: Any,
+ ) -> None:
+ """Initialize ADTassay with CLR normalization.
+
+ Args:
+ z: Zarr hierarchy where raw data is located.
+ name: Assay label.
+ cell_data: Cell metadata object.
+ **kwargs: Forwarded to ``Assay.__init__`` (workspace, nthreads, etc.).
+ """
+ super().__init__(
+ z=z,
+ workspace=workspace,
+ name=name,
+ cell_data=cell_data,
+ nthreads=nthreads,
+ min_cells_per_feature=min_cells_per_feature,
+ **kwargs,
+ )
self.normMethod = norm_clr
def normed(
self,
- cell_idx: Optional[np.ndarray] = None,
- feat_idx: Optional[np.ndarray] = None,
- **kwargs,
- ) -> daskArrayType:
- """This function normalizes the raw and returns a delayed dask array of
- the normalized data. This method uses the the normalization indicated
+ cell_idx: np.ndarray | None = None,
+ feat_idx: np.ndarray | None = None,
+ **kwargs: Any,
+ ) -> ChunkedArray:
+ """This function normalizes the raw and returns a delayed chunked array of
+ the normalized data. This method uses the normalization indicated
by attribute self.normMethod which by default is set to `norm_clr`. The
centered log-ratio normalization is performed using only the cells and
features indicated by the 'cell_idx' and 'feat_idx' parameters.
@@ -1246,7 +1934,7 @@ def normed(
feature attribute table)
**kwargs:
- Returns: A dask array (delayed matrix) containing normalized data.
+ Returns: A chunked array (delayed matrix) containing normalized data.
"""
if cell_idx is None:
cell_idx = self.cells.active_index("I")
diff --git a/scarf/bio_data.py b/scarf/bio_data.py
index b6b75952..9653b7cc 100644
--- a/scarf/bio_data.py
+++ b/scarf/bio_data.py
@@ -3,11 +3,9 @@
g. cell cycle genes).
"""
-from typing import List
-
__all__ = ["s_phase_genes", "g2m_phase_genes"]
-s_phase_genes: List[str] = [
+s_phase_genes: list[str] = [
"MCM5",
"PCNA",
"TYMS",
@@ -53,7 +51,7 @@
"E2F8",
]
-g2m_phase_genes: List[str] = [
+g2m_phase_genes: list[str] = [
"HMGB2",
"CDK1",
"NUSAP1",
diff --git a/scarf/chunked.py b/scarf/chunked.py
new file mode 100644
index 00000000..6cc461fa
--- /dev/null
+++ b/scarf/chunked.py
@@ -0,0 +1,787 @@
+"""A minimal, Zarr-backed chunked array used in place of Dask.
+
+This module provides ``ChunkedArray``, a lazy, row-chunked array over a Zarr
+(or in-memory NumPy) 2D matrix. It mirrors the small slice of the Dask Array
+API that Scarf relies on:
+
+- block iteration via ``.blocks`` (each block is one row-chunk)
+- lazy element-wise ufuncs and broadcast arithmetic
+- ``.dot`` against a small loadings matrix
+- streaming, thread-parallel reductions (``sum``/``mean``/``var``/
+ ``count_nonzero``/``argmax``) that run eagerly but lazily, returning a
+ deferred result so a progress bar with a message can still be attached.
+
+Element-wise operations stay lazy and are applied per row-block at compute
+time. Reductions are evaluated by streaming over row-blocks and combining the
+partial results, which keeps peak memory bounded and independent of the task
+graph topology.
+"""
+
+from collections.abc import Callable, Iterator
+from typing import Any, Literal, cast
+
+import numpy as np
+import zarr
+from numpy.typing import NDArray
+
+__all__ = ["ChunkedArray", "Block"]
+
+type OpKind = Literal["unary", "binary", "matmul"]
+type BType = Literal["scalar", "col", "row", "full"]
+type Backing = np.ndarray | zarr.Array
+type ReductionOp = Literal["sum", "mean", "var", "std", "count_nonzero", "argmax"]
+type UfuncSide = Literal["left", "right"]
+type BlockFn = Callable[[int, int, int], NDArray[Any]]
+
+
+def _is_contiguous(idx: np.ndarray) -> bool:
+ """True if idx is a strictly increasing run of consecutive integers."""
+ if idx.size == 0:
+ return True
+ return bool(
+ idx[0] >= 0 and np.array_equal(idx, np.arange(idx[0], idx[0] + idx.size))
+ )
+
+
+class _Op:
+ """A structured element-wise (or matmul) operation on a ChunkedArray.
+
+ Operations are recorded lazily and replayed per row-block at compute time.
+ Storing them structurally (rather than as opaque closures) lets indexing
+ re-subset per-feature and per-row operands so column/row selection can be
+ pushed through already-applied normalization.
+ """
+
+ __slots__ = ("kind", "func", "operand", "side", "btype")
+
+ def __init__(
+ self,
+ kind: OpKind,
+ func: Callable[..., NDArray[Any]] | None = None,
+ operand: object | None = None,
+ side: UfuncSide = "left",
+ btype: BType | None = None,
+ ) -> None:
+ self.kind = kind # 'unary' | 'binary' | 'matmul'
+ self.func = func
+ self.operand = operand
+ self.side = side
+ self.btype = btype # for 'binary': 'scalar' | 'col' | 'row' | 'full'
+
+ def apply(self, a: NDArray[Any], start: int, end: int) -> NDArray[Any]:
+ if self.kind == "unary":
+ assert self.func is not None
+ return np.asarray(self.func(a))
+ if self.kind == "matmul":
+ return np.asarray(a @ self.operand)
+ assert self.func is not None
+ o = self.operand
+ if self.btype == "col":
+ o = np.asarray(o)[start:end]
+ if np.asarray(o).ndim == 1:
+ o = np.asarray(o).reshape(-1, 1)
+ elif self.btype == "full":
+ o = np.asarray(o)[start:end]
+ return (
+ np.asarray(self.func(a, o))
+ if self.side == "left"
+ else np.asarray(self.func(o, a))
+ )
+
+ def subset_cols(self, col_idx: np.ndarray) -> "_Op":
+ if self.kind == "matmul":
+ raise NotImplementedError("Column-indexing after .dot is not supported")
+ if self.kind == "binary" and self.btype == "row":
+ return _Op(
+ "binary", self.func, np.asarray(self.operand)[col_idx], self.side, "row"
+ )
+ if self.kind == "binary" and self.btype == "full":
+ return _Op(
+ "binary",
+ self.func,
+ np.asarray(self.operand)[:, col_idx],
+ self.side,
+ "full",
+ )
+ return self
+
+ def subset_rows(self, row_idx: np.ndarray) -> "_Op":
+ if self.kind == "binary" and self.btype == "col":
+ return _Op(
+ "binary", self.func, np.asarray(self.operand)[row_idx], self.side, "col"
+ )
+ if self.kind == "binary" and self.btype == "full":
+ return _Op(
+ "binary",
+ self.func,
+ np.asarray(self.operand)[row_idx],
+ self.side,
+ "full",
+ )
+ return self
+
+
+def _unary_op(func: Callable[..., NDArray[Any]]) -> _Op:
+ return _Op("unary", func=func)
+
+
+def _matmul_op(b: np.ndarray) -> _Op:
+ return _Op("matmul", operand=b)
+
+
+def _classify_operand(other: object, n_rows: int, n_cols: int) -> tuple[BType, object]:
+ """Classify a binary operand for per-block broadcasting.
+
+ Returns one of 'scalar', 'col' (per-row vector), 'row' (per-feature
+ vector) or 'full' (per-element matrix), along with the coerced operand.
+ """
+ if np.isscalar(other):
+ return "scalar", other
+ arr = np.asarray(other)
+ if arr.ndim == 0:
+ return "scalar", arr
+ shape = arr.shape
+ if (arr.ndim == 1 and shape[0] == n_rows) or (
+ arr.ndim == 2 and shape == (n_rows, 1)
+ ):
+ return "col", arr
+ if (arr.ndim == 1 and shape[0] == n_cols) or (
+ arr.ndim == 2 and shape == (1, n_cols)
+ ):
+ return "row", arr
+ if arr.ndim == 2 and shape[0] == n_rows:
+ return "full", arr
+ if arr.size == 1:
+ return "scalar", arr
+ return "row", arr
+
+
+def _binary_op(
+ func: Callable[..., NDArray[Any]], other: object, side: str, kind: BType
+) -> _Op:
+ return _Op(
+ "binary", func=func, operand=other, side=cast(UfuncSide, side), btype=kind
+ )
+
+
+class Block:
+ """A single row-chunk view of a ChunkedArray."""
+
+ __slots__ = ("_parent", "_start", "_end", "_row_perm")
+
+ def __init__(
+ self,
+ parent: "ChunkedArray",
+ start: int,
+ end: int,
+ row_perm: np.ndarray | None = None,
+ ) -> None:
+ self._parent = parent
+ self._start = start
+ self._end = end
+ self._row_perm = row_perm
+
+ @property
+ def shape(self) -> tuple[int, int]:
+ n = (self._end - self._start) if self._row_perm is None else len(self._row_perm)
+ return (n, self._parent.out_cols)
+
+ @property
+ def dtype(self) -> np.dtype[Any]:
+ return self._parent.dtype
+
+ def compute(
+ self, nthreads: int | None = None, msg: str | None = None
+ ) -> np.ndarray:
+ a = self._parent._materialize_range(self._start, self._end)
+ if self._row_perm is not None:
+ a = a[self._row_perm]
+ return a
+
+ def __array__(self, dtype: np.dtype[Any] | None = None) -> np.ndarray:
+ a = self.compute()
+ return a.astype(dtype) if dtype is not None else a
+
+ def __getitem__(self, key: object) -> "Block":
+ # Supports block[row_index, :] used during merge row permutation.
+ if isinstance(key, tuple):
+ row_key = key[0]
+ else:
+ row_key = key
+ return Block(self._parent, self._start, self._end, row_perm=np.asarray(row_key))
+
+
+class _Reduction:
+ """A deferred reduction result that behaves like a NumPy array.
+
+ It computes lazily and caches the result. ``compute`` accepts an optional
+ progress message; any other array-like use (arithmetic, ufuncs, attribute
+ access) forces evaluation transparently without a progress bar.
+ """
+
+ __slots__ = ("_parent", "_op", "_axis", "_cached")
+
+ def __init__(
+ self, parent: "ChunkedArray", op: ReductionOp, axis: int | None
+ ) -> None:
+ self._parent = parent
+ self._op = op
+ self._axis = axis
+ self._cached: np.ndarray | None = None
+
+ def compute(
+ self, nthreads: int | None = None, msg: str | None = None
+ ) -> np.ndarray:
+ if self._cached is None:
+ self._cached = self._parent._reduce(self._op, self._axis, nthreads, msg)
+ return self._cached
+
+ @property
+ def _arr(self) -> np.ndarray:
+ return self.compute()
+
+ def __array__(self, dtype: np.dtype[Any] | None = None) -> np.ndarray:
+ a = self._arr
+ return a.astype(dtype) if dtype is not None else a
+
+ def __array_ufunc__(
+ self, ufunc: Any, method: str, *inputs: Any, **kwargs: Any
+ ) -> Any:
+ if method != "__call__":
+ return NotImplemented
+ resolved = tuple(i._arr if isinstance(i, _Reduction) else i for i in inputs)
+ return ufunc(*resolved, **kwargs)
+
+ def __getattr__(self, item: str) -> Any:
+ return getattr(self._arr, item)
+
+ def __len__(self) -> int:
+ return len(self._arr)
+
+ def __iter__(self) -> Iterator[Any]:
+ return iter(self._arr)
+
+ def __getitem__(self, key: object) -> Any:
+ return self._arr[cast(Any, key)]
+
+ def _bin(
+ self,
+ other: object,
+ func: Callable[[NDArray[Any], NDArray[Any]], NDArray[Any]],
+ side: UfuncSide,
+ ) -> NDArray[Any]:
+ o_arr = other._arr if isinstance(other, _Reduction) else np.asarray(other)
+ return func(self._arr, o_arr) if side == "left" else func(o_arr, self._arr)
+
+ def __mul__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.multiply, "left")
+
+ def __rmul__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.multiply, "right")
+
+ def __truediv__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.true_divide, "left")
+
+ def __rtruediv__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.true_divide, "right")
+
+ def __add__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.add, "left")
+
+ def __radd__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.add, "right")
+
+ def __sub__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.subtract, "left")
+
+ def __rsub__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.subtract, "right")
+
+ def __gt__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.greater, "left")
+
+ def __lt__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.less, "left")
+
+ def __ge__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.greater_equal, "left")
+
+ def __le__(self, o: object) -> NDArray[Any]:
+ return self._bin(o, np.less_equal, "left")
+
+ def __repr__(self) -> str:
+ return f""
+
+
+class ChunkedArray:
+ """A lazy, row-chunked 2D array backed by Zarr or NumPy."""
+
+ __array_priority__ = 1000.0
+
+ def __init__(
+ self,
+ backing: Backing,
+ rows: np.ndarray | None = None,
+ cols: np.ndarray | None = None,
+ ops: list[_Op] | None = None,
+ out_cols: int | None = None,
+ block_size: int | None = None,
+ nthreads: int = 1,
+ is_numpy: bool | None = None,
+ ) -> None:
+ self._backing = backing
+ self._rows = None if rows is None else np.asarray(rows)
+ self._cols = None if cols is None else np.asarray(cols)
+ self._ops: list[_Op] = list(ops) if ops else []
+ self._nthreads = nthreads
+ if is_numpy is None:
+ is_numpy = isinstance(backing, np.ndarray)
+ self._is_numpy = is_numpy
+ backing_rows, backing_cols = backing.shape
+ self._n_rows = backing_rows if self._rows is None else int(self._rows.size)
+ base_cols = backing_cols if self._cols is None else int(self._cols.size)
+ self._out_cols = base_cols if out_cols is None else int(out_cols)
+ if block_size is None:
+ if self._is_numpy:
+ block_size = self._n_rows if self._n_rows > 0 else 1
+ else:
+ from .storage.zarr_store import array_shard_rows
+
+ block_size = array_shard_rows(cast(zarr.Array, self._backing))
+ self._block_size = max(int(block_size), 1)
+
+ @classmethod
+ def from_numpy(
+ cls, arr: np.ndarray, block_size: int | None = None, nthreads: int = 1
+ ) -> "ChunkedArray":
+ arr = np.asarray(arr)
+ return cls(arr, block_size=block_size, nthreads=nthreads, is_numpy=True)
+
+ @property
+ def shape(self) -> tuple[int, int]:
+ return (self._n_rows, self._out_cols)
+
+ @property
+ def out_cols(self) -> int:
+ return self._out_cols
+
+ @property
+ def dtype(self) -> np.dtype[Any]:
+ if not self._ops:
+ return self._backing.dtype
+ return np.dtype(np.float64)
+
+ @property
+ def chunksize(self) -> tuple[int, int]:
+ return (
+ min(self._block_size, self._n_rows) if self._n_rows else self._block_size,
+ self._out_cols,
+ )
+
+ @property
+ def numblocks(self) -> tuple[int, int]:
+ return (self._n_block_count(), 1)
+
+ @property
+ def chunks(self) -> tuple[tuple[int, ...], tuple[int, ...]]:
+ sizes = tuple(e - s for s, e in self._ranges())
+ return (sizes if sizes else (0,), (self._out_cols,))
+
+ @property
+ def nthreads(self) -> int:
+ return self._nthreads
+
+ def __len__(self) -> int:
+ return self._n_rows
+
+ def _n_block_count(self) -> int:
+ if self._n_rows == 0:
+ return 0
+ return int(np.ceil(self._n_rows / self._block_size))
+
+ def _ranges(self) -> list[tuple[int, int]]:
+ return [
+ (i, min(i + self._block_size, self._n_rows))
+ for i in range(0, self._n_rows, self._block_size)
+ ]
+
+ # -- materialization -------------------------------------------------
+ def _read(self, start: int, end: int) -> np.ndarray:
+ if self._rows is None:
+ row_sel: slice | np.ndarray = slice(start, end)
+ rows_contiguous = True
+ else:
+ row_sel = self._rows[start:end]
+ rows_contiguous = _is_contiguous(row_sel)
+ if self._is_numpy:
+ numpy_backing = cast(np.ndarray, self._backing)
+ if self._cols is None:
+ return np.asarray(numpy_backing[row_sel])
+ return np.asarray(numpy_backing[row_sel][:, self._cols])
+ # Zarr backing
+ zarr_backing = cast(zarr.Array, self._backing)
+ if self._cols is None:
+ if isinstance(row_sel, slice):
+ return np.asarray(zarr_backing[row_sel, :])
+ if rows_contiguous:
+ return np.asarray(
+ zarr_backing[int(row_sel[0]) : int(row_sel[-1]) + 1, :]
+ )
+ return np.asarray(
+ zarr_backing.get_orthogonal_selection((row_sel, slice(None)))
+ )
+ if isinstance(row_sel, slice):
+ return np.asarray(
+ zarr_backing.get_orthogonal_selection((row_sel, self._cols))
+ )
+ if rows_contiguous:
+ return np.asarray(
+ zarr_backing.get_orthogonal_selection(
+ (slice(int(row_sel[0]), int(row_sel[-1]) + 1), self._cols)
+ )
+ )
+ return np.asarray(zarr_backing.get_orthogonal_selection((row_sel, self._cols)))
+
+ def _materialize_range(self, start: int, end: int) -> np.ndarray:
+ a = self._read(start, end)
+ for op in self._ops:
+ a = op.apply(a, start, end)
+ return a
+
+ # -- block parallelism ----------------------------------------------
+ def _map_blocks(
+ self,
+ fn: BlockFn,
+ nthreads: int | None,
+ msg: str | None,
+ ) -> list[NDArray[Any]]:
+ from .parallel import map_shards
+
+ ranges = self._ranges()
+ nthreads = self._nthreads if nthreads is None else nthreads
+ results = map_shards(ranges, fn, workers=nthreads, msg=msg)
+ return [np.asarray(r) for r in results]
+
+ def stream_blocks(
+ self,
+ nthreads: int | None = None,
+ msg: str | None = None,
+ prefetch: int | None = None,
+ ) -> Iterator[np.ndarray]:
+ """Yield materialized row blocks in order with bounded read-ahead.
+
+ For sequential consumers (PCA/k-means/ANN fitting) that must see blocks
+ in order while the next few are fetched in the background.
+ """
+ from .storage.budget import worker_prefetch_depth
+ from .parallel import stream_shards
+
+ threads = self._nthreads if nthreads is None else nthreads
+ depth = worker_prefetch_depth(prefetch)
+ ranges = self._ranges()
+ yield from stream_shards(
+ ranges,
+ lambda r: self._materialize_range(r[0], r[1]),
+ workers=depth,
+ within_block_threads=threads if threads and threads > 1 else None,
+ msg=msg,
+ total=len(ranges),
+ )
+
+ def map_blocks(
+ self,
+ fn: BlockFn,
+ nthreads: int | None = None,
+ msg: str | None = None,
+ ) -> list[NDArray[Any]]:
+ """Public ordered map of ``fn(block_idx, start, end)`` over row blocks."""
+ return self._map_blocks(fn, nthreads, msg)
+
+ def compute(
+ self, nthreads: int | None = None, msg: str | None = None
+ ) -> np.ndarray:
+ if self._n_rows == 0:
+ return np.empty((0, self._out_cols), dtype=self.dtype)
+
+ def fn(i: int, s: int, e: int) -> NDArray[Any]:
+ return self._materialize_range(s, e)
+
+ parts = self._map_blocks(fn, nthreads, msg)
+ return np.vstack(parts) if len(parts) > 1 else parts[0]
+
+ def __array__(self, dtype: np.dtype[Any] | None = None) -> np.ndarray:
+ a = self.compute()
+ return a.astype(dtype) if dtype is not None else a
+
+ # -- blocks ----------------------------------------------------------
+ @property
+ def blocks(self) -> Iterator[Block]:
+ for s, e in self._ranges():
+ yield Block(self, s, e)
+
+ # -- lazy elementwise ------------------------------------------------
+ def _with_op(self, op: _Op, out_cols: int | None = None) -> "ChunkedArray":
+ return ChunkedArray(
+ self._backing,
+ rows=self._rows,
+ cols=self._cols,
+ ops=self._ops + [op],
+ out_cols=self._out_cols if out_cols is None else out_cols,
+ block_size=self._block_size,
+ nthreads=self._nthreads,
+ is_numpy=self._is_numpy,
+ )
+
+ def _unary(self, func: Callable[..., NDArray[Any]]) -> "ChunkedArray":
+ return self._with_op(_unary_op(func))
+
+ def _binary(
+ self, func: Callable[..., NDArray[Any]], other: object, side: str
+ ) -> "ChunkedArray":
+ if isinstance(other, _Reduction):
+ other = other._arr
+ kind, operand = _classify_operand(other, self._n_rows, self._out_cols)
+ return self._with_op(_binary_op(func, operand, side, kind))
+
+ def __array_ufunc__(
+ self, ufunc: Any, method: str, *inputs: Any, **kwargs: Any
+ ) -> Any:
+ if method != "__call__" or kwargs.get("out") is not None:
+ return NotImplemented
+ if len(inputs) == 1:
+ return self._unary(ufunc)
+ if len(inputs) == 2:
+ a, b = inputs
+ if a is self:
+ return self._binary(ufunc, b, "left")
+ return self._binary(ufunc, a, "right")
+ return NotImplemented
+
+ def __mul__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.multiply, o, "left")
+
+ def __rmul__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.multiply, o, "right")
+
+ def __truediv__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.true_divide, o, "left")
+
+ def __rtruediv__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.true_divide, o, "right")
+
+ def __add__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.add, o, "left")
+
+ def __radd__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.add, o, "right")
+
+ def __sub__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.subtract, o, "left")
+
+ def __rsub__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.subtract, o, "right")
+
+ def __gt__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.greater, o, "left")
+
+ def __lt__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.less, o, "left")
+
+ def __ge__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.greater_equal, o, "left")
+
+ def __le__(self, o: object) -> "ChunkedArray":
+ return self._binary(np.less_equal, o, "left")
+
+ def dot(self, b: np.ndarray | NDArray[Any]) -> "ChunkedArray":
+ b_arr = np.asarray(b)
+ return self._with_op(_matmul_op(b_arr), out_cols=b_arr.shape[1])
+
+ # -- indexing --------------------------------------------------------
+ @staticmethod
+ def _local_positions(key: object, length: int) -> np.ndarray | None:
+ """Resolve an index key into integer positions within ``length``.
+
+ Returns None for a full slice (no-op).
+ """
+ if isinstance(key, slice):
+ if key == slice(None):
+ return None
+ return np.asarray(np.arange(length)[key])
+ key_arr = np.asarray(key)
+ if key_arr.dtype == bool:
+ return np.asarray(np.arange(length)[key_arr])
+ return np.asarray(key_arr.astype(int))
+
+ def __getitem__(self, key: object) -> "ChunkedArray":
+ if isinstance(key, tuple):
+ if len(key) != 2:
+ raise IndexError("ChunkedArray supports at most 2D indexing")
+ row_key, col_key = key
+ else:
+ row_key, col_key = key, slice(None)
+
+ rows = self._rows
+ cols = self._cols
+ ops = self._ops
+ out_cols = self._out_cols
+ n_rows = self._n_rows
+
+ col_pos = self._local_positions(col_key, out_cols)
+ if col_pos is not None:
+ ops = [op.subset_cols(col_pos) for op in ops]
+ base_cols = cols if cols is not None else np.arange(self._backing.shape[1])
+ cols = base_cols[col_pos]
+ out_cols = int(col_pos.size)
+
+ row_pos = self._local_positions(row_key, n_rows)
+ if row_pos is not None:
+ ops = [op.subset_rows(row_pos) for op in ops]
+ base_rows = rows if rows is not None else np.arange(self._backing.shape[0])
+ rows = base_rows[row_pos]
+ n_rows = int(row_pos.size)
+
+ block_size = n_rows if (self._is_numpy and n_rows > 0) else self._block_size
+ new = ChunkedArray(
+ self._backing,
+ rows=rows,
+ cols=cols,
+ ops=ops,
+ out_cols=out_cols,
+ block_size=block_size,
+ nthreads=self._nthreads,
+ is_numpy=self._is_numpy,
+ )
+ return new
+
+ # -- reductions (deferred, evaluated by streaming over blocks) -------
+ def sum(self, axis: int | None = None) -> _Reduction:
+ return _Reduction(self, "sum", axis)
+
+ def mean(self, axis: int | None = None) -> _Reduction:
+ return _Reduction(self, "mean", axis)
+
+ def var(self, axis: int | None = None) -> _Reduction:
+ return _Reduction(self, "var", axis)
+
+ def std(self, axis: int | None = None) -> _Reduction:
+ return _Reduction(self, "std", axis)
+
+ def mean_and_std(
+ self,
+ axis: int = 0,
+ nthreads: int | None = None,
+ msg: str | None = None,
+ ) -> tuple[NDArray[Any], NDArray[Any]]:
+ """Compute column mean and std in a single pass over row blocks."""
+ if axis != 0:
+ raise NotImplementedError("mean_and_std only supports axis=0")
+
+ def fn(i: int, s: int, e: int) -> NDArray[Any]:
+ a = self._materialize_range(s, e).astype(np.float64, copy=False)
+ return np.array([a.sum(axis=0), np.square(a).sum(axis=0)])
+
+ parts = self._map_blocks(fn, nthreads, msg)
+ stacked = np.sum(parts, axis=0)
+ s1, s2 = stacked[0], stacked[1]
+ n = self._n_rows
+ mean = s1 / n
+ variance = s2 / n - np.square(mean)
+ return np.asarray(mean), np.asarray(np.sqrt(np.clip(variance, 0, None)))
+
+ def count_nonzero(self, axis: int | None = None) -> _Reduction:
+ return _Reduction(self, "count_nonzero", axis)
+
+ def argmax(self, axis: int | None = None) -> _Reduction:
+ return _Reduction(self, "argmax", axis)
+
+ def _reduce(
+ self,
+ op: ReductionOp,
+ axis: int | None,
+ nthreads: int | None,
+ msg: str | None,
+ ) -> np.ndarray:
+ if axis == 1 or axis is None:
+ return self._reduce_axis1(op, axis, nthreads, msg)
+ return self._reduce_axis0(op, nthreads, msg)
+
+ def _reduce_axis1(
+ self,
+ op: ReductionOp,
+ axis: int | None,
+ nthreads: int | None,
+ msg: str | None,
+ ) -> np.ndarray:
+ def fn(i: int, s: int, e: int) -> NDArray[Any]:
+ a = self._materialize_range(s, e)
+ if op == "sum":
+ return np.asarray(a.sum(axis=axis))
+ if op == "mean":
+ return np.asarray(a.mean(axis=axis))
+ if op == "var":
+ return np.asarray(a.var(axis=axis))
+ if op == "std":
+ return np.asarray(a.std(axis=axis))
+ if op == "count_nonzero":
+ return np.asarray(np.count_nonzero(a, axis=axis))
+ if op == "argmax":
+ return np.asarray(a.argmax(axis=axis))
+ raise ValueError(f"Unknown reduction {op}")
+
+ parts = self._map_blocks(fn, nthreads, msg)
+ if axis is None:
+ arr = np.asarray(parts)
+ if op == "sum":
+ return np.asarray(arr.sum())
+ if op == "mean":
+ return np.asarray(arr.mean())
+ raise ValueError(f"Reduction {op} with axis=None is not supported")
+ return np.concatenate(parts)
+
+ def _reduce_axis0(
+ self, op: ReductionOp, nthreads: int | None, msg: str | None
+ ) -> np.ndarray:
+ if op in ("sum", "mean"):
+
+ def fn(i: int, s: int, e: int) -> NDArray[Any]:
+ a = self._materialize_range(s, e)
+ return np.asarray(a.sum(axis=0))
+
+ parts = self._map_blocks(fn, nthreads, msg)
+ total = np.sum(parts, axis=0)
+ if op == "mean":
+ return np.asarray(total / self._n_rows)
+ return np.asarray(total)
+ if op == "count_nonzero":
+
+ def fn(i: int, s: int, e: int) -> NDArray[Any]:
+ a = self._materialize_range(s, e)
+ return np.asarray(np.count_nonzero(a, axis=0))
+
+ parts = self._map_blocks(fn, nthreads, msg)
+ return np.asarray(np.sum(parts, axis=0))
+ if op in ("var", "std"):
+
+ def fn(i: int, s: int, e: int) -> NDArray[Any]:
+ a = self._materialize_range(s, e).astype(np.float64, copy=False)
+ return np.array([a.sum(axis=0), np.square(a).sum(axis=0)])
+
+ parts = self._map_blocks(fn, nthreads, msg)
+ stacked = np.sum(parts, axis=0)
+ s1, s2 = stacked[0], stacked[1]
+ n = self._n_rows
+ mean = s1 / n
+ variance = s2 / n - np.square(mean)
+ if op == "std":
+ return np.asarray(np.sqrt(np.clip(variance, 0, None)))
+ return np.asarray(variance)
+ if op == "argmax":
+ raise NotImplementedError("argmax(axis=0) is not supported")
+ raise ValueError(f"Unknown reduction {op}")
+
+ def __repr__(self) -> str:
+ return (
+ f"ChunkedArray(shape={self.shape}, dtype={self.dtype}, "
+ f"chunksize={self.chunksize}, numblocks={self.numblocks[0]})"
+ )
diff --git a/scarf/datastore/base_datastore.py b/scarf/datastore/base_datastore.py
index a1088b6f..d583e5df 100644
--- a/scarf/datastore/base_datastore.py
+++ b/scarf/datastore/base_datastore.py
@@ -1,22 +1,23 @@
-from typing import List, Union, Optional
-
import numpy as np
import zarr
+from collections.abc import Iterable
+from typing import Any, cast
+
from loguru import logger
+from .._types import ZarrMode, as_zarr_array, as_zarr_group
from ..assay import RNAassay, ATACassay, ADTassay, Assay
from ..metadata import MetaData
from ..utils import show_dask_progress, controlled_compute, load_zarr, ZARRLOC
-def sanitize_hierarchy(
- z: zarr.Group, assay_name: str, workspace: Union[str, None]
-) -> bool:
+def sanitize_hierarchy(z: zarr.Group, assay_name: str, workspace: str | None) -> bool:
"""Test if an assay node in zarr object was created properly.
Args:
z: Zarr hierarchy object
assay_name: String value with name of assay.
+ workspace: Workspace name (None for legacy layout without ``matrices/``).
Returns:
True if assay_name is present in z and contains `counts` and `featureData` child nodes else raises error
@@ -24,21 +25,23 @@ def sanitize_hierarchy(
if workspace is None:
zw = z
else:
- zw = z[workspace]
- if assay_name in zw:
- if "featureData" not in zw[assay_name]:
- raise KeyError(f"ERROR: 'featureData' not found in {assay_name}")
- else:
+ zw = as_zarr_group(z[workspace], name=workspace)
+ if assay_name not in zw:
raise KeyError(f"ERROR: {assay_name} not found in zarr file")
+ assay_zw = as_zarr_group(zw[assay_name], name=assay_name)
+ if "featureData" not in assay_zw:
+ raise KeyError(f"ERROR: 'featureData' not found in {assay_name}")
if workspace is None:
- if "counts" not in z[assay_name]:
+ if "counts" not in assay_zw:
raise KeyError(f"ERROR: 'counts' not found in {assay_name}")
else:
if "matrices" not in z:
- raise KeyError(f"ERROR: Workspace defined but no 'matrices' slot found")
- if assay_name not in z["matrices"]:
+ raise KeyError("ERROR: Workspace defined but no 'matrices' slot found")
+ matrices = as_zarr_group(z["matrices"], name="matrices")
+ if assay_name not in matrices:
raise KeyError(f"ERROR: {assay_name} not found in workspace matrices slot")
- if "counts" not in z["matrices"][assay_name]:
+ matrix_assay = as_zarr_group(matrices[assay_name], name=assay_name)
+ if "counts" not in matrix_assay:
raise KeyError(
f"ERROR: 'counts' not found in {assay_name} in workspace matrices slot"
)
@@ -67,6 +70,7 @@ class BaseDataStore:
synchronizer: Used as `synchronizer` parameter when opening the Zarr file. Please refer to this page for
more details: https://zarr.readthedocs.io/en/stable/api/sync.html. By default
ThreadSynchronizer will be used.
+ workspace: Workspace name within the Zarr store (None for legacy single-workspace layout).
Attributes:
cells: MetaData object with cells and info about each cell (e. g. RNA_nCounts ids).
@@ -77,18 +81,26 @@ class BaseDataStore:
def __init__(
self,
zarr_loc: ZARRLOC,
- assay_types: dict,
+ assay_types: dict[str, str],
default_assay: str,
min_features_per_cell: int,
min_cells_per_feature: int,
mito_pattern: str,
ribo_pattern: str,
nthreads: int,
- zarr_mode: str,
- workspace: Union[str, None],
- synchronizer,
+ zarr_mode: ZarrMode,
+ workspace: str | None,
+ synchronizer: Any,
+ storage_options: dict[str, Any] | None = None,
):
- self.z = load_zarr(zarr_loc=zarr_loc, mode=zarr_mode, synchronizer=synchronizer)
+ self.zarr_mode = zarr_mode
+ self.zarr_loc = zarr_loc
+ self.z = load_zarr(
+ zarr_loc=zarr_loc,
+ mode=zarr_mode,
+ synchronizer=synchronizer,
+ storage_options=storage_options,
+ )
self.workspace = workspace
self.nthreads = nthreads
# The order is critical here:
@@ -119,13 +131,13 @@ def _load_cells(self) -> MetaData:
Metadata object
"""
try:
- cell_data: zarr.Group = self.zw["cellData"] # type: ignore
+ cell_data = as_zarr_group(self.zw["cellData"], name="cellData")
except KeyError as e:
raise KeyError(f"cellData not found in zarr file at {self.z.path}") from e
return MetaData(cell_data)
@property
- def assay_names(self) -> List[str]:
+ def assay_names(self) -> list[str]:
"""Load all assay names present in the Zarr file. Zarr writers create
an 'is_assay' attribute in the assay level and this function looks for
presence of those attributes to load assay names.
@@ -140,7 +152,7 @@ def assay_names(self) -> List[str]:
assays.append(i)
return assays
- def _load_default_assay(self, assay_name: Optional[str] = None) -> str:
+ def _load_default_assay(self, assay_name: str | None = None) -> str:
"""This function sets a given assay name as defaultAssay attribute. If
`assay_name` value is None then the top-level directory attributes in
the Zarr file are looked up for presence of previously used default
@@ -154,11 +166,12 @@ def _load_default_assay(self, assay_name: Optional[str] = None) -> str:
"""
if assay_name is None:
if "defaultAssay" in self.zw.attrs:
- assay_name = self.zw.attrs["defaultAssay"]
+ assay_name = cast(str, self.zw.attrs["defaultAssay"])
else:
if len(self.assay_names) == 1:
assay_name = self.assay_names[0]
- self.zw.attrs["defaultAssay"] = assay_name
+ if self.zarr_mode == "r+":
+ self.zw.attrs["defaultAssay"] = assay_name
else:
raise ValueError(
"ERROR: You have more than one assay data. "
@@ -172,17 +185,19 @@ def _load_default_assay(self, assay_name: Optional[str] = None) -> str:
logger.info(
f"Default assay changed from {self.zw.attrs['defaultAssay']} to {assay_name}"
)
- self.zw.attrs["defaultAssay"] = assay_name
+ if self.zarr_mode == "r+":
+ self.zw.attrs["defaultAssay"] = assay_name
else:
raise ValueError(
f"ERROR: The provided default assay name: {assay_name} was not found. "
f"Please Choose one from: {' '.join(self.assay_names)}\n"
"Please note that the names are case-sensitive."
)
+ assert assay_name is not None
return assay_name
def _load_assays(
- self, min_cells: int, custom_assay_types: Optional[dict] = None
+ self, min_cells: int, custom_assay_types: dict | None = None
) -> None:
"""This function loads all the assay names present in attribute
`assayNames` as Assay objects. An attempt is made to automatically
@@ -226,7 +241,12 @@ def _load_assays(
)
if "assayTypes" not in self.zw.attrs:
self.zw.attrs["assayTypes"] = {}
- z_attrs = dict(self.zw.attrs["assayTypes"])
+ raw_types = self.zw.attrs["assayTypes"]
+ z_attrs: dict[str, str] = (
+ {str(k): str(v) for k, v in raw_types.items()}
+ if isinstance(raw_types, dict)
+ else {}
+ )
if custom_assay_types is None:
custom_assay_types = {}
for i in self.assay_names:
@@ -281,8 +301,8 @@ def _load_assays(
def _get_assay(
self,
- from_assay: Union[str, None],
- ) -> Union[Assay, RNAassay, ADTassay, ATACassay]:
+ from_assay: str | None,
+ ) -> Assay | RNAassay | ADTassay | ATACassay:
"""This is a convenience function used internally to quickly obtain the
assay object that is linked to an assay name.
@@ -293,7 +313,9 @@ def _get_assay(
"""
if from_assay is None or from_assay == "":
from_assay = self._defaultAssay
- return self.__getattribute__(from_assay)
+ return cast(
+ Assay | RNAassay | ADTassay | ATACassay, self.__getattribute__(from_assay)
+ )
def _get_latest_feat_key(self, from_assay: str) -> str:
"""Looks up the value in assay level attributes for key
@@ -306,7 +328,7 @@ def _get_latest_feat_key(self, from_assay: str) -> str:
Name of the latest feature that was used to run `save_normalized_data`
"""
assay = self._get_assay(from_assay)
- return assay.attrs["latest_feat_key"]
+ return cast(str, assay.attrs["latest_feat_key"])
def _get_latest_cell_key(self, from_assay: str) -> str:
"""Looks up the value in assay level attributes for key
@@ -319,13 +341,13 @@ def _get_latest_cell_key(self, from_assay: str) -> str:
Name of the latest feature that was used to run `save_normalized_data`
"""
assay = self._get_assay(from_assay)
- return assay.attrs["latest_cell_key"]
+ return cast(str, assay.attrs.get("latest_cell_key", "I"))
def _ini_cell_props(
self,
min_features: int,
- mito_pattern: Optional[str],
- ribo_pattern: Optional[str],
+ mito_pattern: str | None,
+ ribo_pattern: str | None,
) -> None:
"""This function is called on class initialization. For each assay, it
calculates per-cell statistics i.e. nCounts, nFeatures, percentMito and
@@ -350,7 +372,7 @@ def _ini_cell_props(
self.nthreads,
)
self.cells.insert(var_name, n_c.astype(np.float64), overwrite=True)
- if type(assay) == RNAassay:
+ if isinstance(assay, RNAassay):
min_nc = min(n_c)
if min(n_c) < assay.sf:
logger.warning(
@@ -366,7 +388,7 @@ def _ini_cell_props(
)
self.cells.insert(var_name, n_f.astype(np.float64), overwrite=True)
- if type(assay) == RNAassay:
+ if isinstance(assay, RNAassay):
if mito_pattern == "":
pass
else:
@@ -432,7 +454,9 @@ def set_default_assay(self, assay_name: str) -> None:
"""
if assay_name not in self.assay_names:
available = ", ".join(self.assay_names)
- raise ValueError(f"Assay '{assay_name}' not found. Available assays: {available}")
+ raise ValueError(
+ f"Assay '{assay_name}' not found. Available assays: {available}"
+ )
self._defaultAssay = assay_name
self.zw.attrs["defaultAssay"] = assay_name
@@ -451,14 +475,12 @@ def get_cell_vals(
given feature from normalized matrix.
Args:
- from_assay: Name of assay to be used. If no value is provided then the default assay will be used.
- cell_key: One of the columns from cell metadata table that indicates the cells to be used. The values in
- the chosen column should be boolean (Default value: 'I')
- k: A cell metadata column or name of a feature.
- clip_fraction: This value is multiplied by 100 and the percentiles are soft-clipped from either end.
- (Default value: 0)
- use_precached: Whether to use pre-calculated values from 'prenormed' slot. Used only if 'prenormed' is
- present (Default value: True)
+ from_assay: Name of assay to be used.
+ cell_key: Boolean column in cell metadata selecting cells (default: ``'I'``).
+ k: Cell metadata column name or feature name whose values are fetched.
+ clip_fraction: Fraction (0-1) for soft percentile clipping of numeric values.
+ use_precached: Use values from the ``prenormed`` slot when available.
+ cache_key: Zarr group name for precached feature values (default: ``'prenormed'``).
Returns:
The requested values
@@ -476,12 +498,17 @@ def get_cell_vals(
)
vals = None
if use_precached and cache_key in assay.z:
- g = assay.z[cache_key]
+ g = as_zarr_group(assay.z[cache_key], name=cache_key)
vals = np.zeros(assay.cells.N)
n_feats = 0
for i in feat_idx:
- if i in g:
- vals += assay.z[cache_key][i][:]
+ feat_key = str(i)
+ if feat_key in g:
+ feat_vals = np.asarray(
+ as_zarr_array(g[feat_key], name=feat_key)[:],
+ dtype=np.float64,
+ )
+ vals += feat_vals
n_feats += 1
if n_feats == 0:
logger.debug(f"Could not find prenormed values for feat: {k}")
@@ -510,8 +537,8 @@ def get_cell_vals(
vals[vals > max_v] = max_v
return vals
- def __repr__(self):
- def formatter(label, iter_vals):
+ def __repr__(self) -> str:
+ def formatter(label: str | None, iter_vals: Iterable[str]) -> str:
if label is None:
line = ""
else:
@@ -545,11 +572,15 @@ def formatter(label, iter_vals):
f"features and following metadata:"
)
res += formatter(None, assay.feats.columns)
- if "projections" in self.zw[i]:
- targets = []
- layouts = []
- for j in self.zw[i]["projections"]:
- if isinstance(self.zw[i]["projections"][j], zarr.Group): # type: ignore
+ assay_group = as_zarr_group(self.zw[i], name=i)
+ if "projections" in assay_group:
+ targets: list[str] = []
+ layouts: list[str] = []
+ projections = as_zarr_group(
+ assay_group["projections"], name="projections"
+ )
+ for j in projections:
+ if isinstance(projections[j], zarr.Group):
targets.append(j)
else:
layouts.append(j)
diff --git a/scarf/datastore/datastore.py b/scarf/datastore/datastore.py
index c0b4b55d..0d3ca362 100644
--- a/scarf/datastore/datastore.py
+++ b/scarf/datastore/datastore.py
@@ -1,18 +1,174 @@
-from typing import Iterable, List, Literal, Optional, Tuple, Union
+import time
+from collections.abc import Iterable, Sequence
+from typing import Any, Literal, cast
import numpy as np
import pandas as pd
-from dask import array as daskarr
+import zarr
from loguru import logger
-
-from ..assay import Assay, ATACassay, RNAassay
+from numpy.typing import NDArray
+
+from .._types import ZarrMode, as_zarr_array, as_zarr_group
+from ..assay import (
+ PSEUDOTIME_AGGREGATION_SCHEMA_VERSION,
+ Assay,
+ ATACassay,
+ RNAassay,
+)
+from ..chunked import ChunkedArray
from ..feat_utils import hto_demux
-from ..utils import ZARRLOC, controlled_compute, tqdmbar
-from ..writers import create_zarr_dataset, create_zarr_obj_array
+from ..markers import resolve_marker_gene_batch_size, sort_marker_results
+from ..utils import ZARRLOC, array_digest, controlled_compute, tqdmbar
+from ..writers import create_zarr_dataset
from .mapping_datastore import MappingDatastore
__all__ = ["DataStore"]
+_MARKER_LAYOUT_V2 = "compact_v2"
+_MARKER_STAT_COLUMNS = (
+ "score",
+ "mean",
+ "mean_rest",
+ "frac_exp",
+ "frac_exp_rest",
+ "fold_change",
+ "p_value",
+)
+_MARKER_OUT_COLUMNS = ("feature_index", *_MARKER_STAT_COLUMNS)
+
+
+def _feature_column_chunk(assay: Assay, n_features: int) -> int:
+ backing = getattr(assay.rawData, "_backing", None)
+ chunks = getattr(backing, "chunks", None)
+ if chunks and len(chunks) > 1:
+ return max(1, int(chunks[1]))
+ return max(1, int(n_features))
+
+
+def _shared_marker_feature_index(markers: dict[Any, pd.DataFrame]) -> np.ndarray:
+ for vals in markers.values():
+ if len(vals) != 0:
+ return np.sort(np.asarray(vals.index.values, dtype=np.int32))
+ raise ValueError("Cannot save empty marker results")
+
+
+def _marker_stats_matrix(vals: pd.DataFrame, feature_index: np.ndarray) -> np.ndarray:
+ aligned = vals.reindex(feature_index)
+ return np.asarray(
+ aligned.loc[:, list(_MARKER_STAT_COLUMNS)].to_numpy(dtype=np.float64)
+ )
+
+
+def _write_compact_marker_stats(
+ cluster_group: zarr.Group,
+ stats: np.ndarray,
+) -> None:
+ from ..storage.zarr_store import ZarrArraySpec, create_numeric_array
+
+ n_features = int(stats.shape[0])
+ spec = ZarrArraySpec(
+ shape=(n_features, len(_MARKER_STAT_COLUMNS)),
+ chunks=(n_features, len(_MARKER_STAT_COLUMNS)),
+ dtype="float64",
+ overwrite=True,
+ )
+ arr = create_numeric_array(cluster_group, "stats", spec)
+ arr[:] = stats
+
+
+def _load_marker_cluster_frame(
+ slot_group: zarr.Group,
+ cluster_group: zarr.Group,
+ feature_names: np.ndarray,
+ *,
+ group_id: Any,
+) -> pd.DataFrame:
+ out_cols = list(_MARKER_OUT_COLUMNS)
+ if slot_group.attrs.get("layout") == _MARKER_LAYOUT_V2 and "stats" in cluster_group:
+ feature_index = np.asarray(
+ as_zarr_array(slot_group["feature_index"], name="feature_index")[:]
+ )
+ stats = np.asarray(as_zarr_array(cluster_group["stats"], name="stats")[:])
+ df = pd.DataFrame(stats, columns=list(_MARKER_STAT_COLUMNS))
+ df["feature_index"] = feature_index
+ df["feature_name"] = feature_names[feature_index.astype(int)]
+ df["group_id"] = group_id
+ return sort_marker_results(df[["group_id", "feature_name", *out_cols[1:]]])
+
+ available_cols = [col for col in out_cols if col in cluster_group]
+ if not available_cols:
+ return pd.DataFrame([[] for _ in out_cols], index=out_cols).T
+ cols = [
+ np.asarray(as_zarr_array(cluster_group[x], name=x)[:]) for x in available_cols
+ ]
+ df = pd.DataFrame(cols, index=available_cols).T
+ df["group_id"] = group_id
+ df["feature_name"] = feature_names[df.feature_index.astype("int")]
+ return df[["group_id", "feature_name", *available_cols[1:]]]
+
+
+def _validated_pseudotime_regressor(
+ assay: Assay,
+ cell_key: str,
+ pseudotime_key: str,
+) -> np.ndarray:
+ try:
+ pseudotime = np.asarray(
+ assay.cells.fetch(pseudotime_key, key=cell_key),
+ dtype=float,
+ )
+ except (TypeError, ValueError) as exc:
+ raise TypeError(
+ f"Pseudotime column '{pseudotime_key}' must be numeric"
+ ) from exc
+
+ if pseudotime.ndim != 1:
+ raise ValueError(
+ f"Pseudotime column '{pseudotime_key}' must be one-dimensional"
+ )
+ expected_size = assay.cells.active_index(cell_key).shape[0]
+ if pseudotime.shape[0] != expected_size:
+ raise ValueError(
+ f"Pseudotime column '{pseudotime_key}' has {pseudotime.shape[0]} values, "
+ f"but cell_key '{cell_key}' selects {expected_size} cells"
+ )
+ if not np.isfinite(pseudotime).all():
+ validity_key = f"{pseudotime_key}__valid"
+ if validity_key in assay.cells.columns:
+ raise ValueError(
+ f"Pseudotime column '{pseudotime_key}' contains unscored cells. "
+ f"Use cell_key='{validity_key}' for downstream analysis"
+ )
+ raise ValueError(
+ f"Pseudotime column '{pseudotime_key}' contains non-finite values"
+ )
+ if pseudotime.size < 2 or np.unique(pseudotime).size < 2:
+ raise ValueError(
+ f"Pseudotime column '{pseudotime_key}' must contain at least two distinct values"
+ )
+ return pseudotime
+
+
+def _group_assignment_digest(values: np.ndarray) -> str:
+ return array_digest(np.asarray(values).astype(str))
+
+
+def _scatter_feature_clusters(
+ n_features: int,
+ feature_indices: np.ndarray,
+ clusters: np.ndarray,
+ unassigned_value: int,
+) -> np.ndarray:
+ feature_indices = np.asarray(feature_indices, dtype=int)
+ clusters = np.asarray(clusters, dtype=int)
+ if feature_indices.shape != clusters.shape:
+ raise ValueError("Feature indices and cluster assignments are misaligned")
+ if unassigned_value in clusters:
+ raise ValueError("unassigned_value conflicts with an assigned feature cluster")
+ values = np.full(n_features, unassigned_value, dtype=int)
+ values[feature_indices] = clusters
+ return values
+
class DataStore(MappingDatastore):
"""This class extends MappingDatastore and consequently inherits methods of
@@ -41,22 +197,45 @@ class DataStore(MappingDatastore):
synchronizer: Used as `synchronizer` parameter when opening the Zarr file. Please refer to this page for
more details: https://zarr.readthedocs.io/en/stable/api/sync.html. By default
ThreadSynchronizer will be used.
+ mem_budget: Memory budget bounding streaming and concurrency. Accepts bytes, a suffixed size
+ (e.g. '8G'), or a fraction of total system memory (e.g. '0.6'). When None, it is
+ auto-detected (SCARF_MEM_BUDGET env var, else total system memory). Override it to
+ simulate reading on a machine with a different memory size than the writer.
+ working_copies: Number of concurrent in-memory working copies the memory budget is divided
+ across. When None, uses SCARF_WORKING_COPIES env var or the default.
"""
def __init__(
self,
zarr_loc: ZARRLOC,
- assay_types: Optional[dict] = None,
- default_assay: Optional[str] = None,
+ assay_types: dict[str, str] | None = None,
+ default_assay: str | None = None,
min_features_per_cell: int = 10,
min_cells_per_feature: int = 20,
- mito_pattern: Optional[str] = None,
- ribo_pattern: Optional[str] = None,
+ mito_pattern: str | None = None,
+ ribo_pattern: str | None = None,
nthreads: int = 2,
- zarr_mode: str = "r+",
- workspace: Union[str, None] = None,
- synchronizer=None,
- ):
+ zarr_mode: ZarrMode = "r+",
+ workspace: str | None = None,
+ synchronizer: Any = None,
+ zarrProfile: Literal["fast_local", "cloud"] | None = None,
+ storage_options: dict[str, Any] | None = None,
+ mem_budget: int | str | None = None,
+ working_copies: int | None = None,
+ ) -> None:
+ from ..storage.budget import resolve_budget, set_resource_budget
+ from ..storage.zarr_store import (
+ configure_zarr_io_for_profile,
+ set_storage_profile,
+ )
+
+ set_storage_profile(zarrProfile)
+ set_resource_budget(
+ resolve_budget(
+ memory=mem_budget, workers=nthreads, working_copies=working_copies
+ )
+ )
+ configure_zarr_io_for_profile()
if zarr_mode not in ["r", "r+"]:
raise ValueError(
"ERROR: Zarr file can only be accessed using either 'r' or 'r+' mode"
@@ -73,6 +252,7 @@ def __init__(
zarr_mode=zarr_mode,
workspace=workspace,
synchronizer=synchronizer,
+ storage_options=storage_options,
)
def get_assay(self, assay_name: str) -> Assay:
@@ -87,7 +267,7 @@ def get_assay(self, assay_name: str) -> Assay:
if assay_name not in self.assay_names:
raise ValueError(f"ERROR: Assay {assay_name} not found in the Zarr file")
else:
- return getattr(self, assay_name)
+ return cast(Assay, getattr(self, assay_name))
def filter_cells(
self,
@@ -139,7 +319,7 @@ def filter_cells(
def auto_filter_cells(
self,
- attrs: Optional[Iterable[str]] = None,
+ attrs: Iterable[str] | None = None,
min_p: float = 0.01,
max_p: float = 0.99,
show_qc_plots: bool = True,
@@ -198,10 +378,20 @@ def auto_filter_cells(
def mark_hto_identities(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
label: str = "Hashtag_identity",
) -> None:
+ """Assign HTO hashtag identities to cells using demultiplexing.
+
+ Args:
+ from_assay: HTO assay name (default: ``'HTO'``).
+ cell_key: Boolean cell metadata column selecting cells (default: latest for assay).
+ label: Column name to store identities in cell metadata.
+
+ Returns:
+ None
+ """
if from_assay is None:
from_assay = "HTO"
if cell_key is None:
@@ -220,11 +410,196 @@ def mark_hto_identities(
key=cell_key,
)
+ def run_doublet_detection(
+ self,
+ cluster_key: str,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ cluster_sample_fraction: float = 0.05,
+ max_cells_per_cluster: int = 100,
+ simulation_ratio: float = 1.0,
+ heterotypic_fraction: float = 0.8,
+ save_k: int = 5,
+ smoothing_t: int = 2,
+ normalize_scores: bool = True,
+ label: str = "doublet_score",
+ batch_size: int = 1000,
+ random_seed: int = 4444,
+ ) -> None:
+ """Flag potential doublets by simulating and mapping synthetic doublets.
+
+ Synthetic doublets are simulated by summing the raw counts of pairs of
+ observed cells drawn from a per-cluster subsample, with a tunable bias
+ toward cross-cluster (heterotypic) pairs. The simulated profiles are
+ projected onto the existing reference graph with `run_mapping`, and each
+ reference cell is scored by how frequently it appears among the nearest
+ neighbours of the simulated doublets (`get_mapping_score`). The score is
+ then diffused over the KNN graph using the same operator as
+ `get_imputed`. The final per-cell score is written to cell metadata so
+ that users can threshold and filter doublets themselves. This is a
+ graph-native adaptation of the Scrublet and DoubletFinder approach.
+
+ Args:
+ cluster_key: Cell metadata column with cluster or group labels used
+ to stratify the candidate pool (for example ``'RNA_cluster'``).
+ from_assay: Assay to use. Defaults to the latest used assay. Only
+ RNAassay type assays are supported.
+ cell_key: Cell key matching the desired graph (default: ``'I'``).
+ feat_key: Feature key matching the desired graph. Defaults to the
+ latest used feature key for the assay.
+ cluster_sample_fraction: Fraction of cells sampled from each cluster
+ to build the candidate pool. (Default value: 0.05)
+ max_cells_per_cluster: Cap on the number of cells sampled per cluster.
+ (Default value: 100)
+ simulation_ratio: Number of simulated doublets expressed as a
+ multiple of the number of reference cells. (Default value: 1.0)
+ heterotypic_fraction: Fraction of simulated doublets forced to be
+ cross-cluster. Set to 0 to disable the bias. (Default value: 0.8)
+ save_k: Number of reference neighbours stored per simulated doublet.
+ (Default value: 5)
+ smoothing_t: Diffusion power used to smoothen scores over the graph,
+ same as the ``t`` parameter of `get_imputed`. (Default value: 2)
+ normalize_scores: If True, the final score is min-max scaled to the
+ 0-1 range for interpretability. (Default value: True)
+ label: Base name for the score column in cell metadata. The assay
+ name (and cell key when not ``'I'``) is prepended.
+ (Default value: 'doublet_score')
+ batch_size: Number of simulated doublets written per batch.
+ (Default value: 1000)
+ random_seed: Seed for reproducible sampling. (Default value: 4444)
+
+ Returns:
+ None
+ """
+ import shutil
+ import tempfile
+
+ from scipy.sparse import csr_matrix
+
+ from ..doublet_utils import (
+ sample_cluster_pool,
+ simulate_doublet_pairs,
+ write_doublet_target_zarr,
+ )
+
+ from_assay, cell_key, feat_key = self._get_latest_keys(
+ from_assay, cell_key, feat_key
+ )
+ source_assay = self._get_assay(from_assay)
+ if type(source_assay) != RNAassay: # noqa: E721
+ raise TypeError(
+ "ERROR: Doublet detection is only supported for RNAassay type assays. "
+ f"The provided assay is {type(source_assay)} type"
+ )
+ if cluster_key not in self.cells.columns:
+ raise ValueError(
+ f"ERROR: `cluster_key` {cluster_key} not found in cell metadata. Provide a column "
+ f"with cluster or group labels, for example '{from_assay}_cluster'"
+ )
+
+ rng = np.random.default_rng(random_seed)
+ active_idx = self.cells.active_index(cell_key)
+ n_active = len(active_idx)
+ clusters = self.cells.fetch(cluster_key, key=cell_key)
+
+ pool_positions = sample_cluster_pool(
+ clusters, cluster_sample_fraction, max_cells_per_cluster, rng
+ )
+ pool_clusters = np.asarray(clusters)[pool_positions]
+ pool_raw_rows = np.asarray(active_idx)[pool_positions]
+ logger.info(
+ f"Sampled {len(pool_positions)} cells across "
+ f"{len(np.unique(pool_clusters))} clusters to seed doublet simulation"
+ )
+
+ pool_counts = controlled_compute(
+ source_assay.rawData[pool_raw_rows, :], self.nthreads
+ )
+ pool_csr = csr_matrix(pool_counts)
+
+ n_sim = max(1, int(round(simulation_ratio * n_active)))
+ left, right = simulate_doublet_pairs(
+ pool_clusters, n_sim, heterotypic_fraction, rng
+ )
+ sim_counts = (pool_csr[left] + pool_csr[right]).tocsr()
+ logger.info(f"Simulated {n_sim} synthetic doublets")
+
+ temp_dir = tempfile.mkdtemp(prefix="scarf_doublet_")
+ target_name = f"_doublet_sim_{from_assay}"
+ target_feat_key = f"{feat_key}_doublet"
+ try:
+ write_doublet_target_zarr(
+ zarr_loc=temp_dir,
+ assay_name=from_assay,
+ sim_counts=sim_counts,
+ feat_ids=source_assay.feats.fetch_all("ids"),
+ feat_names=source_assay.feats.fetch_all("names"),
+ dtype=str(source_assay.rawData.dtype),
+ batch_size=batch_size,
+ )
+ target_ds = DataStore(
+ temp_dir,
+ default_assay=from_assay,
+ assay_types={from_assay: "RNA"},
+ nthreads=self.nthreads,
+ )
+ self.run_mapping(
+ target_assay=target_ds._get_assay(from_assay),
+ target_name=target_name,
+ target_feat_key=target_feat_key,
+ from_assay=from_assay,
+ cell_key=cell_key,
+ feat_key=feat_key,
+ save_k=save_k,
+ batch_size=batch_size,
+ )
+
+ raw_scores = None
+ for _, score in self.get_mapping_score(
+ target_name=target_name,
+ from_assay=from_assay,
+ cell_key=cell_key,
+ log_transform=True,
+ ):
+ raw_scores = score
+ if raw_scores is None:
+ raise RuntimeError(
+ "ERROR: Mapping scores could not be computed for simulated doublets"
+ )
+
+ temp_col = self._col_renamer(from_assay, cell_key, f"{label}__raw")
+ self.cells.insert(temp_col, raw_scores, key=cell_key, overwrite=True)
+ smoothed = self.get_imputed(
+ from_assay=from_assay,
+ cell_key=cell_key,
+ feature_name=temp_col,
+ feat_key=feat_key,
+ t=smoothing_t,
+ )
+ self.cells.drop(temp_col)
+
+ scores = np.asarray(smoothed, dtype=float)
+ if normalize_scores:
+ lo, hi = scores.min(), scores.max()
+ scores = (scores - lo) / (hi - lo) if hi > lo else np.zeros_like(scores)
+ final_col = self._col_renamer(from_assay, cell_key, label)
+ self.cells.insert(final_col, scores, key=cell_key, overwrite=True)
+ logger.info(f"Doublet scores stored in cell metadata column '{final_col}'")
+ finally:
+ store_loc = f"{from_assay}/projections/{target_name}"
+ try:
+ if store_loc in self.zw:
+ del self.zw[store_loc]
+ except Exception as e: # noqa: BLE001
+ logger.debug(f"Could not remove temporary projection group: {e}")
+ shutil.rmtree(temp_dir, ignore_errors=True)
+
def mark_hvgs(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- min_cells: Optional[int] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ min_cells: int | None = None,
top_n: int = 500,
min_var: float = -np.inf,
max_var: float = np.inf,
@@ -236,8 +611,8 @@ def mark_hvgs(
keep_bounds: bool = False,
show_plot: bool = True,
hvg_key_name: str = "hvgs",
- max_cells: int | None = np.inf,
- **plot_kwargs,
+ max_cells: float | None = None,
+ **plot_kwargs: Any,
) -> None:
"""Identify and mark genes as highly variable genes (HVGs). This is a
critical and required feature selection step and is only applicable to
@@ -252,7 +627,7 @@ def mark_hvgs(
to identify rare populations of cells. Very small values might lead to a higher signal-to-noise
ratio in the selected features. By default, a value is set assuming smallest population has no
less than 1% of all cells. So for example, if you have 1000 cells (as per cell_key parameter)
- then `min-cells` will be set to 10.
+ then `min_cells` will be set to 10.
max_cells: Maximum number of cells where a gene should have non-zero expression values for it to be
considered a candidate for HVG selection. This can be useful to filter out genes that are
expressed in too many cells. Default value is infinity, meaning no upper limit.
@@ -293,12 +668,14 @@ def mark_hvgs(
f"Setting `min_cells` to {min_cells}. Only those genes that are present in atleast this number "
f"of cells will be considered HVGs."
)
- if max_cells is None:
- max_cells = np.inf
+ if max_cells is None or max_cells == np.inf:
+ max_cells_int: int | float = np.inf
+ else:
+ max_cells_int = int(max_cells)
assay.mark_hvgs(
cell_key=cell_key,
min_cells=min_cells,
- max_cells=max_cells,
+ max_cells=max_cells_int,
top_n=top_n,
min_var=min_var,
max_var=max_var,
@@ -315,8 +692,8 @@ def mark_hvgs(
def mark_prevalent_peaks(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
top_n: int = 10000,
prevalence_key_name: str = "prevalent_peaks",
) -> None:
@@ -330,8 +707,7 @@ def mark_prevalent_peaks(
cell_key: Cells to use for selection of most prevalent peaks. By default, all cells with True value in
'I' will be used. The provided value for `cell_key` should be a column in cell metadata table
with boolean values.
- top_n: Number of top prevalent peaks to be selected. This value is ignored if a value is provided
- for `min_var` parameter. (Default: 500)
+ top_n: Number of top prevalent peaks to be selected. (Default: 10000)
prevalence_key_name: Base label for marking prevalent peaks in the features metadata column. The value for
'cell_key' parameter is prepended to this value. (Default value: 'prevalent_peaks')
@@ -350,17 +726,17 @@ def mark_prevalent_peaks(
def run_marker_search(
self,
- from_assay: Optional[str] = None,
- group_key: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
- gene_batch_size: int = 50,
+ from_assay: str | None = None,
+ group_key: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ gene_batch_size: int | None = None,
use_prenormed: bool = False,
- prenormed_store: Optional[str] = None,
- n_threads: Optional[int] = None,
+ prenormed_store: zarr.Group | str | None = None,
+ n_threads: int | None = None,
skip_save: bool = False,
- **norm_params,
- ) -> Optional[dict]:
+ **norm_params: Any,
+ ) -> dict[str, Any] | None:
"""Identifies group specific features for a given assay.
Please check out the ``find_markers_by_rank`` function for further details of how marker features for groups
@@ -372,18 +748,19 @@ def run_marker_search(
how the cells will be grouped. Usually this would be a column denoting cell clusters.
cell_key: To run the test on specific subset of cells, provide the name of a boolean column in
the cell metadata table. (Default value: 'I')
- feat_key:
- gene_batch_size: Number of genes to be loaded in memory at a time. All cells (from ell_key) are loaded for
- these number of cells at a time.
- use_prenormed: If True then prenormalized cache generated using Assay.save_normed_for_query is used.
- This can speed up the results. (Default value: True)
- prenormed_store: If prenormalized values were computed in a custom manner then, the Zarr group's location
- can be provided here. (Default value: None)
- n_threads: Number of threads to use to run the marker search. Only used if use_prenormed is True.
- skip_save: If True then the results are not saved to the Zarr hierarchy.
+ feat_key: Boolean feature metadata column selecting features (default: ``'I'``).
+ gene_batch_size: Number of genes loaded per batch; all selected cells are loaded for each batch.
+ When None (default), the batch size is the minimum of the on-disk feature chunk
+ width and a budget-safe cap derived from the active memory budget.
+ use_prenormed: If True, use prenormalized cache from ``Assay.save_normed_for_query``.
+ (Default value: False)
+ prenormed_store: Custom Zarr group with prenormalized values (default: None).
+ n_threads: Threads for marker search when ``use_prenormed`` is True.
+ skip_save: If True, return results without writing to Zarr.
+ **norm_params: Extra keyword arguments forwarded to ``normed``.
Returns:
- None
+ Marker dict if ``skip_save`` is True, else None.
"""
from ..markers import find_markers_by_rank
@@ -392,19 +769,38 @@ def run_marker_search(
"ERROR: Please provide a value for `group_key`. This should be the name of a column from "
"cell metadata object that has information on how cells should be grouped."
)
- if cell_key is None:
- cell_key = "I"
+ from_assay, cell_key, _ = self._get_latest_keys(from_assay, cell_key, None)
if feat_key is None:
feat_key = "I"
if n_threads is None:
n_threads = self.nthreads
assay = self._get_assay(from_assay)
+ n_features = len(assay.feats.active_index(feat_key))
+ if gene_batch_size is None:
+ gene_batch_size = resolve_marker_gene_batch_size(
+ n_features=n_features,
+ n_cells=len(assay.cells.active_index(cell_key)),
+ column_chunk=_feature_column_chunk(assay, n_features),
+ )
+
slot_name = f"{cell_key}__{group_key}"
- z = self.zw[assay.name]
- if "markers" not in z:
- z.create_group("markers")
- group = z["markers"].create_group(slot_name, overwrite=True)
+ logger.debug(
+ f"Running marker search for {from_assay}/{slot_name} "
+ f"(feat_key={feat_key}, batch_size={gene_batch_size})"
+ )
+ assay_grp = as_zarr_group(self.zw[assay.name], name=assay.name)
+ if "markers" not in assay_grp:
+ assay_grp.create_group("markers")
+ markers_grp = as_zarr_group(assay_grp["markers"], name="markers")
+
+ prenormed_group: zarr.Group | None
+ if isinstance(prenormed_store, str):
+ prenormed_group = as_zarr_group(
+ self.zw[prenormed_store], name=prenormed_store
+ )
+ else:
+ prenormed_group = prenormed_store
markers = find_markers_by_rank(
assay=assay,
@@ -413,31 +809,82 @@ def run_marker_search(
feat_key=feat_key,
batch_size=gene_batch_size,
use_prenormed=use_prenormed,
- prenormed_store=prenormed_store,
+ prenormed_store=prenormed_group,
n_threads=n_threads,
**norm_params,
)
if skip_save:
return markers
+
+ from ..storage.zarr_store import is_remote_datastore
+
+ remote = is_remote_datastore(self.zarr_loc, self.z)
+ t_save = time.perf_counter()
+ remote_slot = markers_grp.create_group(slot_name, overwrite=True)
+ workers = max(1, int(n_threads or self.nthreads))
+ self._write_marker_slot(
+ remote_slot,
+ markers,
+ workers=workers if remote else 1,
+ )
+ logger.info(
+ f"Saved marker results to {assay.name}/markers/{slot_name} "
+ f"in {time.perf_counter() - t_save:.1f}s "
+ f"({len(markers)} clusters, layout={_MARKER_LAYOUT_V2})"
+ )
+ return None
+
+ @staticmethod
+ def _write_marker_slot(
+ group: zarr.Group,
+ markers: dict[Any, pd.DataFrame],
+ *,
+ workers: int = 1,
+ ) -> None:
+ from ..storage.zarr_store import create_metadata_column
+
+ feature_index = _shared_marker_feature_index(markers)
+ group.attrs["layout"] = _MARKER_LAYOUT_V2
+ group.attrs["statColumns"] = list(_MARKER_STAT_COLUMNS)
+ create_metadata_column(
+ group,
+ "feature_index",
+ data=feature_index,
+ dtype=np.int32,
+ overwrite=True,
+ )
+
+ def write_cluster(item: tuple[Any, pd.DataFrame]) -> None:
+ cluster_id, vals = item
+ if len(vals) == 0:
+ return
+ cluster_group = group.create_group(str(cluster_id))
+ stats = _marker_stats_matrix(vals, feature_index)
+ _write_compact_marker_stats(
+ cluster_group,
+ stats,
+ )
+
+ items = list(markers.items())
+ if workers <= 1:
+ for item in items:
+ write_cluster(item)
else:
- for i in markers:
- g = group.create_group(i)
- vals = markers[i]
- if len(vals) != 0:
- for j in vals.columns:
- create_zarr_obj_array(g, j, vals[j].values, dtype=vals[j].dtype)
- return None
+ from concurrent.futures import ThreadPoolExecutor
+
+ with ThreadPoolExecutor(max_workers=workers) as ex:
+ list(ex.map(write_cluster, items))
def run_pseudotime_marker_search(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
- pseudotime_key: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ pseudotime_key: str | None = None,
min_cells: int = 10,
gene_batch_size: int = 50,
- **norm_params,
+ **norm_params: Any,
) -> None:
"""Identify genes that a correlated with a given pseudotime ordering of
cells. The results are saved in feature attribute tables. For example,
@@ -449,12 +896,13 @@ def run_pseudotime_marker_search(
from_assay: Name of the assay to be used. If no value is provided then the default assay will be used.
cell_key: To run the test on specific subset of cells, provide the name of a boolean column in
the cell metadata table. (Default value: 'I')
- feat_key:
+ feat_key: Boolean feature metadata column selecting features (default: ``'I'``).
pseudotime_key: Required parameter. This has to be a column name from cell metadata table. This column
contains values for pseudotime ordering of the cells.
min_cells: Minimum number of cells where a gene should have non-zero value to be considered for test.
(Default: 10)
gene_batch_size: Number of genes to be loaded in memory at a time. (Default value: 50).
+ **norm_params: Extra keyword arguments forwarded to ``normed``.
Returns: None
"""
@@ -472,7 +920,7 @@ def run_pseudotime_marker_search(
if feat_key is None:
feat_key = "I"
assay = self._get_assay(from_assay)
- ptime = assay.cells.fetch(pseudotime_key, key=cell_key)
+ ptime = _validated_pseudotime_regressor(assay, cell_key, pseudotime_key)
markers = find_markers_by_regression(
assay=assay,
cell_key=cell_key,
@@ -482,24 +930,30 @@ def run_pseudotime_marker_search(
batch_size=gene_batch_size,
**norm_params,
)
+ feature_index = assay.feats.active_index(feat_key)
+ markers = markers.reindex(feature_index)
+ if markers.isna().any(axis=None):
+ raise ValueError("Pseudotime marker results are not aligned to feat_key")
assay.feats.insert(
f"{cell_key}__{pseudotime_key}__r",
np.array(markers["r_value"].values),
+ key=feat_key,
overwrite=True,
)
assay.feats.insert(
f"{cell_key}__{pseudotime_key}__p",
np.array(markers["p_value"].values),
+ key=feat_key,
overwrite=True,
)
def run_pseudotime_aggregation(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
- pseudotime_key: Optional[str] = None,
- cluster_label: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ pseudotime_key: str | None = None,
+ cluster_label: str | None = None,
min_exp: float = 1e-3,
window_size: int = 200,
chunk_size: int = 50,
@@ -508,8 +962,9 @@ def run_pseudotime_aggregation(
n_neighbours: int = 11,
n_clusters: int = 10,
batch_size: int = 100,
- ann_params: Optional[dict] = None,
- nan_cluster_value: Union[int, str] = -1,
+ ann_params: dict | None = None,
+ nan_cluster_value: int = -1,
+ **norm_params: Any,
) -> None:
"""This method performs clustering of features based on pseudotime
ordered cells. The values from the pseudotime ordered cells are
@@ -523,13 +978,13 @@ def run_pseudotime_aggregation(
cell_key: To run the test on specific subset of cells, provide the name of a boolean column in
the cell metadata table. (Default value: The cell key that was used to generate the latest graph)
feat_key: To use only a subset of features, provide the name of a boolean column in the feature
- metadata/attribute table. Default value: The cell key that was used to generate the latest graph)
+ metadata/attribute table. (Default value: 'I')
pseudotime_key: Required parameter. This has to be a column name from cell attribute table. This
column contains values for pseudotime ordering of the cells.
cluster_label: Required parameter. Name of the column under which the feature cluster identity will be
saved in the feature attribute table.
- min_exp: Features with mean normalized expression than this value are dropped and hence not assigned
- a cluster identity (Default value: 10)
+ min_exp: Features with mean normalized expression below this value are dropped and not assigned
+ a cluster identity. (Default value: 1e-3)
window_size: The window for calculating rolling mean of feature values along pseudotime ordering. Larger
values will slow down processing but produce more smoothened. The choice of value here depends
on the number of cells in the analysis. Larger value will be useful to produce smooth profiles
@@ -546,6 +1001,7 @@ def run_pseudotime_aggregation(
ann_params: The parameter to forward to HNSWlib index instantiation step. (Default value: {})
nan_cluster_value: The value to use for features that are not assigned a cluster identity.
(Default value: -1)
+ **norm_params: Extra keyword arguments forwarded to normalized expression calculation.
Returns: None
"""
@@ -568,6 +1024,12 @@ def run_pseudotime_aggregation(
"of each feature will be saved under this column name. If this column already exists "
"then it will be overwritten."
)
+ if not isinstance(nan_cluster_value, (int, np.integer)) or isinstance(
+ nan_cluster_value, (bool, np.bool_)
+ ):
+ raise TypeError("nan_cluster_value must be an integer")
+ nan_cluster_value = int(nan_cluster_value)
+ _validated_pseudotime_regressor(assay, cell_key, pseudotime_key)
df, feat_ids = assay.save_aggregated_ordering(
cell_key=cell_key,
@@ -579,6 +1041,7 @@ def run_pseudotime_aggregation(
smoothen=smoothen,
z_scale=z_scale,
batch_size=batch_size,
+ **norm_params,
)
if ann_params is None:
ann_params = {}
@@ -589,19 +1052,45 @@ def run_pseudotime_aggregation(
n_threads=self.nthreads,
ann_params=ann_params,
)
- temp = np.ones(assay.feats.N) * -1
- temp[feat_ids] = clusts
+ temp = _scatter_feature_clusters(
+ assay.feats.N,
+ feat_ids,
+ clusts,
+ nan_cluster_value,
+ )
assay.feats.insert(
- cluster_label, temp.astype(int), fill_value=-1, overwrite=True
+ cluster_label,
+ temp,
+ fill_value=nan_cluster_value,
+ overwrite=True,
)
+
+ location = f"aggregated_{cell_key}_{feat_key}_{pseudotime_key}"
+ aggregation_group = as_zarr_group(assay.z[location], name=location)
+ cluster_digest = _group_assignment_digest(temp)
+ aggregation_group.attrs["cluster_label"] = cluster_label
+ aggregation_group.attrs["cluster_digest"] = cluster_digest
+ aggregation_group.attrs["nan_cluster_value"] = nan_cluster_value
+
+ for assay_name in self.assay_names:
+ grouped_assay = self._get_assay(assay_name)
+ if (
+ grouped_assay.attrs.get("grouped_from_assay") == assay.name
+ and grouped_assay.attrs.get("grouped_group_key") == cluster_label
+ and grouped_assay.attrs.get("grouped_group_digest") != cluster_digest
+ ):
+ logger.warning(
+ f"Grouped assay '{assay_name}' is stale after updating "
+ f"feature groups in '{cluster_label}'. Rerun add_grouped_assay"
+ )
return None
def get_markers(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- group_key: Optional[str] = None,
- group_id: Optional[Union[str, int]] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ group_key: str | None = None,
+ group_id: str | int | None = None,
min_score: float = 0.25,
min_frac_exp: float = 0.2,
) -> pd.DataFrame:
@@ -630,7 +1119,7 @@ def get_markers(
"""
if cell_key is None:
- cell_key = "I"
+ from_assay, cell_key, _ = self._get_latest_keys(from_assay, cell_key, None)
if group_key is None:
raise ValueError(
"ERROR: Please provide a value for group_key. "
@@ -638,44 +1127,39 @@ def get_markers(
)
assay = self._get_assay(from_assay)
try:
- g = assay.z["markers"][f"{cell_key}__{group_key}"] # type: ignore
+ markers_grp = as_zarr_group(assay.z["markers"], name="markers")
+ g = as_zarr_group(
+ markers_grp[f"{cell_key}__{group_key}"],
+ name=f"{cell_key}__{group_key}",
+ )
except KeyError:
raise KeyError(
"ERROR: Couldn't find the location of markers. Please make sure that you have already called "
"`run_marker_search` method with same value of `cell_key` and `group_key`"
)
- out_cols = [
- "feature_index",
- "score",
- "mean",
- "mean_rest",
- "frac_exp",
- "frac_exp_rest",
- "fold_change",
- "p_value",
- ]
+ out_cols = list(_MARKER_OUT_COLUMNS)
gids = sorted(set(assay.cells.fetch(group_key, key=cell_key)))
if group_id is not None:
gids = [group_id]
+ feature_names = assay.feats.fetch_all("names")
dfs = []
for gid in gids:
- if gid in g:
- cols = [g[gid][x][:] for x in out_cols] # type: ignore
- df = pd.DataFrame(
- cols,
- index=out_cols,
- ).T
- df["group_id"] = gid
- df["feature_name"] = assay.feats.fetch_all("names")[
- df.feature_index.astype("int")
- ]
+ group_name = str(gid)
+ if group_name in g:
+ marker_grp = as_zarr_group(g[group_name], name=group_name)
+ df = _load_marker_cluster_frame(
+ g,
+ marker_grp,
+ feature_names,
+ group_id=gid,
+ )
else:
logger.debug(f"No markers found for {gid} returning empty dataframe")
df = pd.DataFrame([[] for _ in out_cols], index=out_cols).T
df["group_id"] = []
df["feature_name"] = []
- df = df[["group_id", "feature_name"] + out_cols]
+ df = df[["group_id", "feature_name"] + list(out_cols[1:])]
dfs.append(df)
dfs = pd.concat(dfs)
return dfs[
@@ -684,10 +1168,10 @@ def get_markers(
def export_markers_to_csv(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- group_key: Optional[str] = None,
- csv_filename: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ group_key: str | None = None,
+ csv_filename: str | None = None,
min_score: float = 0.25,
min_frac_exp: float = 0.2,
) -> None:
@@ -743,10 +1227,10 @@ def export_markers_to_csv(
def run_cell_cycle_scoring(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- s_genes: Optional[List[str]] = None,
- g2m_genes: Optional[List[str]] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ s_genes: list[str] | None = None,
+ g2m_genes: list[str] | None = None,
n_bins: int = 50,
rand_seed: int = 4466,
s_score_label: str = "S_score",
@@ -823,10 +1307,10 @@ def run_cell_cycle_scoring(
def add_grouped_assay(
self,
- from_assay: Optional[str] = None,
- group_key: Optional[str] = None,
- assay_label: Optional[str] = None,
- exclude_values: Optional[list] = None,
+ from_assay: str | None = None,
+ group_key: str | None = None,
+ assay_label: str | None = None,
+ exclude_values: list | None = None,
) -> None:
"""Add a new assay to the DataStore by grouping together multiple
features and taking their means. This method requires that the features
@@ -845,6 +1329,8 @@ def add_grouped_assay(
Returns: None
"""
+ from ..storage.zarr_store import write_dense_in_shard_rows
+
from ..writers import create_zarr_count_assay
if assay_label is None:
@@ -867,7 +1353,7 @@ def add_grouped_assay(
module_ids = [f"group_{x}" for x in group_set]
g = create_zarr_count_assay(
- z=assay.z["/"], # type: ignore
+ z=self.zw,
assay_name=assay_label,
workspace=self.workspace,
chunk_size=assay.rawData.chunksize, # type: ignore
@@ -878,26 +1364,35 @@ def add_grouped_assay(
)
cell_idx = np.array(list(range(assay.cells.N)))
+ n_groups = len(group_set)
+ matrix = np.zeros((assay.cells.N, n_groups), dtype=np.float64)
for n, i in tqdmbar(
- enumerate(group_set), desc="Writing to Zarr", total=len(group_set)
+ enumerate(group_set), desc="Computing grouped means", total=len(group_set)
):
feat_idx = np.where(groups == i)[0]
- temp = np.zeros(assay.cells.N)
- temp[cell_idx] = (
+ matrix[:, n] = (
assay.normed(cell_idx=cell_idx, feat_idx=feat_idx)
.mean(axis=1)
.compute()
)
- g[:, n] = temp
+ write_dense_in_shard_rows(
+ g,
+ lambda start, end: matrix[start:end, :],
+ msg="Writing grouped assay",
+ )
self._load_assays(min_cells=0, custom_assay_types={assay_label: "Assay"})
self._ini_cell_props(min_features=0, mito_pattern="", ribo_pattern="")
+ grouped_assay = self._get_assay(assay_label)
+ grouped_assay.attrs["grouped_from_assay"] = assay.name
+ grouped_assay.attrs["grouped_group_key"] = group_key
+ grouped_assay.attrs["grouped_group_digest"] = _group_assignment_digest(groups)
def add_melded_assay(
self,
- from_assay: Optional[str] = None,
- external_bed_fn: Optional[str] = None,
- assay_label: Optional[str] = None,
+ from_assay: str | None = None,
+ external_bed_fn: str | None = None,
+ assay_label: str | None = None,
peaks_col: str = "ids",
scalar_coeff: float = 1e5,
renormalization: bool = True,
@@ -912,6 +1407,8 @@ def add_melded_assay(
This method has been designed for snATAC-Seq data and can be used to quantify accessibility of specific
genomic loci such as gene bodies, promoters, enhancers, motifs, etc.
+ Features from the BED file are retained even when they do not overlap any peak; those zero-count features
+ are marked invalid during assay initialization.
Args:
from_assay: Name of assay to be used. If no value is provided then the default assay will be used.
@@ -952,15 +1449,22 @@ def add_melded_assay(
)
peaks_coords = assay.feats.fetch_all(peaks_col)
- for n, i in enumerate(peaks_coords):
- error_msg = (
- f"ERROR: Coordinate format check failed for element: {i} (position {n}). The format should "
- f"be chr:start-end. Please note the colon and hyphen position"
+ coords_ser = pd.Series(peaks_coords, dtype="object")
+ string_mask = coords_ser.map(lambda x: isinstance(x, str))
+ colon_counts = coords_ser.str.count(":")
+ hyphen_counts = coords_ser.str.split(":").str[-1].str.count("-")
+ invalid_mask = (
+ ~string_mask
+ | colon_counts.ne(1).fillna(True)
+ | hyphen_counts.ne(1).fillna(True)
+ )
+ invalid_coords = invalid_mask.to_numpy(dtype=bool)
+ if invalid_coords.any():
+ n = int(np.flatnonzero(invalid_coords)[0])
+ raise ValueError(
+ f"ERROR: Coordinate format check failed for element: {peaks_coords[n]} (position {n}). "
+ f"The format should be chr:start-end. Please note the colon and hyphen position"
)
- if len(i.split(":")) != 2:
- raise ValueError(error_msg)
- if len(i.split(":")[1].split("-")) != 2:
- raise ValueError(error_msg)
coordinate_melding(
assay,
@@ -970,6 +1474,7 @@ def add_melded_assay(
peaks_col=peaks_col,
scalar_coeff=scalar_coeff,
renormalization=renormalization,
+ peaks_coords=peaks_coords,
)
self._load_assays(min_cells=10, custom_assay_types={assay_label: assay_type})
@@ -977,19 +1482,19 @@ def add_melded_assay(
def make_bulk(
self,
- from_assay: Optional[str] = None,
+ from_assay: str | None = None,
cell_key: str = "I",
- group_key: Optional[str] = None,
- secondary_group_key: Optional[str] = None,
+ group_key: str | None = None,
+ secondary_group_key: str | None = None,
aggr_type: Literal["mean", "sum"] = "mean",
return_fraction: bool = False,
feature_label: Literal["index", "id", "name"] = "index",
remove_empty_features: bool = True,
pseudo_reps: int = 1,
- null_vals: Optional[list] = None,
- secondary_null_vals: Optional[list] = None,
+ null_vals: list[Any] | None = None,
+ secondary_null_vals: list[Any] | None = None,
random_seed: int = 4466,
- ) -> Union[pd.DataFrame, Tuple[pd.DataFrame, pd.DataFrame]]:
+ ) -> pd.DataFrame | tuple[pd.DataFrame, pd.DataFrame]:
"""Merge data from cells to create a bulk profile.
Args:
@@ -1013,10 +1518,10 @@ def make_bulk(
is returned. The second dataframe contains the fraction of cells expressing each feature in each group.
"""
- def make_reps(v, n_reps: int, seed: int) -> List[np.ndarray]:
- v = list(v)
+ def make_reps(v: NDArray[Any], n_reps: int, seed: int) -> list[NDArray[Any]]:
+ v_list = list(v)
random_state = np.random.RandomState(seed)
- shuffled_idx = random_state.choice(v, len(v), replace=False)
+ shuffled_idx = random_state.choice(v_list, len(v_list), replace=False)
rep_idx = np.array_split(shuffled_idx, n_reps)
return [np.array(sorted(x)) for x in rep_idx]
@@ -1030,21 +1535,22 @@ def make_reps(v, n_reps: int, seed: int) -> List[np.ndarray]:
raise ValueError("ERROR: Please provide a value for `group_key` parameter")
else:
groups = self.cells.fetch_all(group_key)
- groups_set = sorted(set(self.cells.fetch(group_key, key=cell_key)))
+ active_idx = self.cells.active_index(cell_key)
+ groups_set = sorted(set(groups[active_idx]))
if secondary_group_key is None:
- sec_groups = [None]
- sec_groups_set = [None]
+ sec_groups: NDArray[Any] = np.array([None], dtype=object)
+ sec_groups_set: list[Any] = [None]
else:
sec_groups = self.cells.fetch_all(secondary_group_key)
- sec_groups_set = sorted(
- set(self.cells.fetch(secondary_group_key, key=cell_key))
- )
+ sec_groups_set = sorted(set(sec_groups[active_idx]))
assay = self._get_assay(from_assay)
- vals = {}
- fracs = {}
+ vals: dict[str, NDArray[Any]] = {}
+ fracs: dict[str, NDArray[Any]] = {}
all_feat_idx = np.arange(assay.feats.N)
+ active_mask = np.zeros(self.cells.N, dtype=bool)
+ active_mask[active_idx] = True
for g in tqdmbar(groups_set):
if g in null_vals:
continue
@@ -1052,9 +1558,11 @@ def make_reps(v, n_reps: int, seed: int) -> List[np.ndarray]:
if sg in secondary_null_vals:
continue
if sg is None and len(sec_groups) == 1:
- g_idx = np.where(groups == g)[0]
+ g_idx = np.where((groups == g) & active_mask)[0]
else:
- g_idx = np.where((groups == g) & (sec_groups == sg))[0]
+ g_idx = np.where((groups == g) & (sec_groups == sg) & active_mask)[
+ 0
+ ]
rep_indices = make_reps(g_idx, pseudo_reps, random_seed)
for n, idx in enumerate(rep_indices):
if sg is None and len(sec_groups) == 1:
@@ -1086,41 +1594,40 @@ def make_reps(v, n_reps: int, seed: int) -> List[np.ndarray]:
(assay.rawData[idx] > 0).mean(axis=0).compute()
)
- vals = pd.DataFrame(vals).fillna(0)
+ vals_df = pd.DataFrame(vals).fillna(0)
empty_idx = None
if remove_empty_features:
- empty_idx = vals.sum(axis=1) != 0
- vals = vals.loc[empty_idx]
+ empty_idx = vals_df.sum(axis=1) != 0
+ vals_df = vals_df.loc[empty_idx]
if feature_label == "id":
- vals.set_index(
- pd.Series(assay.feats.fetch_all("ids")).reindex(vals.index).values,
+ vals_df.set_index(
+ pd.Series(assay.feats.fetch_all("ids")).reindex(vals_df.index).values,
inplace=True,
drop=True,
)
elif feature_label == "name":
- vals.set_index(
- pd.Series(assay.feats.fetch_all("names")).reindex(vals.index).values,
+ vals_df.set_index(
+ pd.Series(assay.feats.fetch_all("names")).reindex(vals_df.index).values,
inplace=True,
drop=True,
)
if return_fraction:
- fracs = pd.DataFrame(fracs).fillna(0)
+ fracs_df = pd.DataFrame(fracs).fillna(0)
if empty_idx is not None:
- fracs = fracs[empty_idx]
- fracs.set_index(vals.index, inplace=True, drop=True)
- return vals, fracs
- else:
- return vals
+ fracs_df = fracs_df[empty_idx]
+ fracs_df.set_index(vals_df.index, inplace=True, drop=True)
+ return vals_df, fracs_df
+ return vals_df
def to_anndata(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- layers: Optional[dict] = None,
- ):
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ layers: dict[str, str] | None = None,
+ ) -> Any:
"""Writes an assay of the Zarr hierarchy to AnnData file format.
Args:
@@ -1166,17 +1673,41 @@ def show_zarr_tree(self, start: str = "/", depth: int = 2) -> None:
Returns:
None
"""
- print(self.zw[start].tree(expand=True, level=depth))
+ from ..storage.zarr_store import array_info
+
+ root = start.strip("/")
+ node: zarr.Group = (
+ self.zw if root == "" else as_zarr_group(self.zw[root], name=root)
+ )
+ print(node.tree(level=depth))
+ for key in node.array_keys():
+ print(f" {key}: {array_info(as_zarr_array(node[key], name=key))}")
def calc_membership_strength(
self, from_assay: str, cell_key: str, feat_key: str, clust_key: str
) -> None:
+ """Store per-cell cluster membership strength from the latest KNN graph.
+
+ For each cell, computes the fraction of KNN neighbors sharing the most
+ common cluster label and saves it in cell metadata.
+
+ Args:
+ from_assay: Assay used to locate the KNN graph.
+ cell_key: Boolean column selecting cells.
+ feat_key: Feature key used when the graph was built.
+ clust_key: Cell metadata column with cluster assignments.
+
+ Returns:
+ None
+ """
loc = self._get_latest_graph_loc(
from_assay=from_assay, cell_key=cell_key, feat_key=feat_key
)
n_cells, k = self._get_graph_ncells_k(graph_loc=loc)
clusts = self.cells.fetch(clust_key, key=cell_key)
- v = pd.DataFrame(clusts[self.zw[loc]["edges"][:, 1].reshape(k, n_cells)])
+ graph_grp = as_zarr_group(self.zw[loc], name=loc)
+ edges = np.asarray(as_zarr_array(graph_grp["edges"], name="edges")[:])
+ v = pd.DataFrame(clusts[edges[:, 1].reshape(k, n_cells)])
x = np.array([v[x].value_counts().index[0] for x in v])
self.cells.insert(
f"{from_assay}_{cell_key}_cluster_membership_strength",
@@ -1191,8 +1722,8 @@ def smart_label(
to_relabel: str,
base_label: str,
cell_key: str = "I",
- new_col_name: Optional[str] = None,
- ) -> Union[None, List[str]]:
+ new_col_name: str | None = None,
+ ) -> None | list[str]:
"""A convenience function to relabel the values in a cell attribute
column (A) based on the values in another cell attribute column (B).
For each unique value in A, the most frequently occurring value in B is
@@ -1237,19 +1768,20 @@ def smart_label(
return ret_val
else:
self.cells.insert(new_col_name, ret_val, overwrite=True)
+ return None
def plot_cells_dists(
self,
- from_assay: Optional[str] = None,
- cols: Optional[List[str]] = None,
- cell_key: Optional[str] = None,
- group_key: Optional[str] = None,
+ from_assay: str | None = None,
+ cols: list[str] | None = None,
+ cell_key: str | None = None,
+ group_key: str | None = None,
color: str = "steelblue",
cmap: str = "tab20",
- fig_size: Optional[tuple] = None,
+ fig_size: tuple | None = None,
label_size: float = 10.0,
title_size: float = 10.0,
- sup_title: Optional[str] = None,
+ sup_title: str | None = None,
sup_title_size: float = 12.0,
scatter_size: float = 1.0,
max_points: int = 10000,
@@ -1345,19 +1877,19 @@ def plot_cells_dists(
def plot_layout(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- layout_key: Optional[str] = None,
- color_by: Optional[str] = None,
- subselection_key: Optional[str] = None,
- size_vals: Union[np.ndarray, List[float], None] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ layout_key: str | list[str] | None = None,
+ color_by: str | list[str] | None = None,
+ subselection_key: str | None = None,
+ size_vals: np.ndarray | list[float] | None = None,
clip_fraction: float = 0.01,
width: float = 6,
height: float = 6,
default_color: str = "steelblue",
- cmap: Optional[str] = None,
- color_key: Optional[dict] = None,
- mask_values: Optional[list] = None,
+ cmap: str | None = None,
+ color_key: dict[str, str] | None = None,
+ mask_values: list[Any] | None = None,
mask_name: str = "NA",
mask_color: str = "k",
point_size: float = 10,
@@ -1370,12 +1902,12 @@ def plot_layout(
frame_offset: float = 0.05,
spine_width: float = 0.5,
spine_color: str = "k",
- displayed_sides: tuple = ("bottom", "left"),
+ displayed_sides: tuple[str, ...] = ("bottom", "left"),
legend_ondata: bool = True,
legend_onside: bool = True,
legend_size: float = 12,
legends_per_col: int = 20,
- title: Union[str, List[str], None] = None,
+ title: str | list[str] | None = None,
title_size: int = 12,
hide_title: bool = False,
cbar_shrink: float = 0.6,
@@ -1384,16 +1916,17 @@ def plot_layout(
cspacing: float = 1,
shuffle_df: bool = False,
sort_values: bool = False,
- savename: Optional[str] = None,
+ savename: str | None = None,
save_dpi: int = 300,
- ax=None,
+ ax: Any = None,
force_ints_as_cats: bool = True,
n_columns: int = 4,
w_pad: float = 1,
h_pad: float = 1,
show_fig: bool = True,
- scatter_kwargs: Optional[dict] = None,
- ):
+ scatter_kwargs: dict[str, Any] | None = None,
+ use_plotting: bool = False,
+ ) -> Any:
"""Create a scatter plot with a chosen layout. The method fetches the
coordinates based from the cell metadata columns with `layout_key`
prefix. DataShader library is used to draw fast rasterized image is
@@ -1424,7 +1957,7 @@ def plot_layout(
height: Figure height (Default value: 6)
default_color: A default color for the cells. (Default value: steelblue)
cmap: A matplotlib colourmap to be used to colour categorical or continuous values plotted on the cells.
- (Default value: tab20 for categorical variables and cmocean.deep for continuous variables)
+ (Default value: tab20 for categorical variables and viridis for continuous variables)
color_key: A custom colour map for cells. These can be used for categorical variables only. The keys in this
dictionary should be the category label as present in the `color_by` column and values should be
valid matplotlib colour names or hex codes of colours. (Default value: None)
@@ -1497,39 +2030,91 @@ def plot_layout(
Ignored if only plotting one scatterplot.
show_fig: Whether to render the figure and display it using plt.show() (Default value: True)
scatter_kwargs: Keyword argument to be passed to matplotlib's scatter command
+ use_plotting: If True, try ``scarf.plotting.embedding`` when the call is
+ compatible (no shading, no masks, single layout). Otherwise fall back
+ to the legacy renderer with a warning. Default False keeps legacy
+ behavior and return type.
Returns:
- None
+ None when ``show_fig`` is True. Otherwise axes (legacy) or a
+ ``PlotResult`` when ``use_plotting`` successfully bridges.
"""
# TODO: add support for providing a list of subselections, from_assay and cell_keys
# TODO: add support for different kinds of point markers
from ..plots import plot_scatter, shade_scatter
+ from ..plotting._legacy import copy_plot_mutables, try_bridge_plot_layout
+
+ color_key, mask_values, scatter_kwargs = copy_plot_mutables(
+ color_key=color_key,
+ mask_values=mask_values,
+ scatter_kwargs=scatter_kwargs,
+ )
if from_assay is None:
from_assay = self._defaultAssay
if cell_key is None:
- cell_key = "I"
+ cell_key = self._get_latest_cell_key(from_assay)
if layout_key is None:
raise ValueError("Please provide a value for `layout_key` parameter.")
if clip_fraction >= 0.5:
raise ValueError(
"ERROR: clip_fraction cannot be larger than or equal to 0.5"
)
- if isinstance(layout_key, str):
- layout_key: List[str] = [layout_key]
+
+ handled, bridged = try_bridge_plot_layout(
+ self,
+ use_plotting=use_plotting,
+ layout_key=layout_key,
+ color_by=color_by,
+ do_shading=do_shading,
+ mask_values=mask_values,
+ subselection_key=subselection_key,
+ shuffle_df=shuffle_df,
+ legend_ondata=legend_ondata,
+ legend_onside=legend_onside,
+ force_ints_as_cats=force_ints_as_cats,
+ clip_fraction=clip_fraction,
+ ax=ax,
+ cell_key=cell_key,
+ from_assay=from_assay,
+ point_size=point_size,
+ size_vals=size_vals,
+ sort_values=sort_values,
+ cmap=cmap,
+ default_color=default_color,
+ mask_color=mask_color,
+ width=width,
+ height=height,
+ n_columns=n_columns,
+ show_fig=show_fig,
+ savename=savename,
+ save_dpi=save_dpi,
+ color_key=color_key,
+ title=title,
+ scatter_kwargs=scatter_kwargs,
+ )
+ if handled:
+ return bridged
+
+ if isinstance(layout_key, list):
+ layout_keys = layout_key
+ else:
+ layout_keys = [layout_key]
# If a list of layout keys and color_by (e.g. layout_key=['UMAP', 'tSNE'], color_by=['gene1', 'gene2'] the
# grid layout will be: plot1: UMAP + gene1, plot2: UMAP + gene2, plot3: tSNE + gene1, plot4: tSNE + gene2
dfs = []
- for lk in layout_key:
+ for lk in layout_keys:
x = self.cells.fetch(f"{lk}1", key=cell_key)
y = self.cells.fetch(f"{lk}2", key=cell_key)
if color_by is None:
- color_by = "vc"
- if isinstance(color_by, str):
- color_by: List[str] = [color_by]
- for c in color_by:
+ color_cols = ["vc"]
+ elif isinstance(color_by, str):
+ color_cols = [color_by]
+ else:
+ color_cols = color_by
+ for c in color_cols:
if c == "vc":
v = np.ones(len(x)).astype(int)
else:
@@ -1642,11 +2227,11 @@ def plot_layout(
def plot_cluster_tree(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
- cluster_key: Optional[str] = None,
- fill_by_value: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ cluster_key: str | None = None,
+ fill_by_value: str | None = None,
force_ints_as_cats: bool = True,
width: float = 1,
lvr_factor: float = 0.5,
@@ -1660,17 +2245,17 @@ def plot_cluster_tree(
fontsize: float = 10,
root_color: str = "#C0C0C0",
non_leaf_color: str = "k",
- cmap="tab20",
- color_key: Optional[dict] = None,
+ cmap: str = "tab20",
+ color_key: dict[str, str] | None = None,
edgecolors: str = "k",
edgewidth: float = 1,
alpha: float = 0.7,
- figsize=(5, 5),
- ax=None,
+ figsize: tuple[float, float] = (5, 5),
+ ax: Any = None,
show_fig: bool = True,
- savename: Optional[str] = None,
+ savename: str | None = None,
save_dpi: int = 300,
- ):
+ ) -> None:
"""Plots a hierarchical layout of the clusters detected using
`run_clustering` in a binary tree form. This helps evaluate the
relationships between the clusters. This figure can complement
@@ -1752,25 +2337,45 @@ def plot_cluster_tree(
)
clusts = self.cells.fetch(cluster_key, key=cell_key)
graph_loc = self._get_latest_graph_loc(from_assay, cell_key, feat_key)
- dendrogram_loc = self.zw[graph_loc].attrs["latest_dendrogram"]
+ graph_grp = as_zarr_group(self.zw[graph_loc], name=graph_loc)
+ dendrogram_loc = cast(str, graph_grp.attrs["latest_dendrogram"])
n_clusts = len(set(clusts))
coalesced_loc = dendrogram_loc + f"_coalesced_{n_clusts}"
if coalesced_loc in self.zw:
subgraph = DiGraph()
- subgraph.add_edges_from(self.zw[coalesced_loc + "/edgelist"][:])
- for i, j in zip(
- self.zw[coalesced_loc + "/nodelist"][:],
- self.zw[coalesced_loc + "/partition_id"][:],
- ):
+ subgraph.add_edges_from(
+ np.asarray(
+ as_zarr_array(
+ self.zw[coalesced_loc + "/edgelist"],
+ name=f"{coalesced_loc}/edgelist",
+ )[:]
+ )
+ )
+ nodelist = np.asarray(
+ as_zarr_array(
+ self.zw[coalesced_loc + "/nodelist"],
+ name=f"{coalesced_loc}/nodelist",
+ )[:]
+ )
+ partition_ids = np.asarray(
+ as_zarr_array(
+ self.zw[coalesced_loc + "/partition_id"],
+ name=f"{coalesced_loc}/partition_id",
+ )[:]
+ )
+ for i, j in zip(nodelist, partition_ids):
node = int(i[0])
subgraph.nodes[node]["nleaves"] = int(i[1])
if j != "-1":
subgraph.nodes[node]["partition_id"] = j
else:
- subgraph = CoalesceTree(make_digraph(self.zw[dendrogram_loc][:]), clusts)
+ dendrogram = np.asarray(
+ as_zarr_array(self.zw[dendrogram_loc], name=dendrogram_loc)[:]
+ )
+ subgraph = CoalesceTree(make_digraph(dendrogram), clusts)
edge_list = to_pandas_edgelist(subgraph).values
store = create_zarr_dataset(
- self.zw, coalesced_loc + "/edgelist", (100000,), "u8", edge_list.shape
+ self.zw, f"{coalesced_loc}/edgelist", (100000,), "u8", edge_list.shape
)
store[:] = edge_list
node_list = []
@@ -1781,19 +2386,19 @@ def plot_cluster_tree(
node_list.append((i, d["nleaves"]))
partition_id_list.append(str(p))
- node_list = np.array(node_list)
+ node_list_arr = np.array(node_list)
store = create_zarr_dataset(
self.zw,
- coalesced_loc + "/nodelist",
+ f"{coalesced_loc}/nodelist",
(100000,),
- node_list.dtype,
- node_list.shape,
+ node_list_arr.dtype,
+ node_list_arr.shape,
)
- store[:] = node_list
+ store[:] = node_list_arr
store = create_zarr_dataset(
self.zw,
- coalesced_loc + "/partition_id",
+ f"{coalesced_loc}/partition_id",
(100000,),
str,
(len(partition_id_list),),
@@ -1834,21 +2439,22 @@ def plot_cluster_tree(
savename=savename,
save_dpi=save_dpi,
)
+ return None
def plot_marker_heatmap(
self,
- from_assay: Optional[str] = None,
- group_key: Optional[str] = None,
- cell_key: Optional[str] = None,
+ from_assay: str | None = None,
+ group_key: str | None = None,
+ cell_key: str | None = None,
topn: int = 5,
log_transform: bool = True,
vmin: float = -1,
vmax: float = 2,
- savename: Optional[str] = None,
+ savename: str | None = None,
save_dpi: int = 300,
show_fig: bool = True,
- **heatmap_kwargs,
- ):
+ **heatmap_kwargs: Any,
+ ) -> None:
"""Displays a heatmap of top marker gene expression for the chosen
groups (usually cell clusters).
@@ -1884,38 +2490,83 @@ def plot_marker_heatmap(
raise ValueError("ERROR: Please provide a value for `group_key`")
if cell_key is None:
cell_key = "I"
- if "markers" not in self.zw[assay.name]:
+ assay_grp = as_zarr_group(self.zw[assay.name], name=assay.name)
+ if "markers" not in assay_grp:
raise KeyError("ERROR: Please run `run_marker_search` first")
slot_name = f"{cell_key}__{group_key}"
- if slot_name not in self.zw[assay.name]["markers"]:
+ markers_grp = as_zarr_group(assay_grp["markers"], name="markers")
+ if slot_name not in markers_grp:
raise KeyError(
f"ERROR: Please run `run_marker_search` first with {group_key} as `group_key` and "
f"{cell_key} as `cell_key`"
)
- g = self.zw[assay.name]["markers"][slot_name]
- feat_idx = []
- for i in g.keys():
- if "feature_index" in g[i]:
- feat_idx.extend(g[i]["feature_index"][:][:topn])
+ g = as_zarr_group(markers_grp[slot_name], name=slot_name)
+ feat_idx: list[Any] = []
+ if g.attrs.get("layout") == _MARKER_LAYOUT_V2 and "feature_index" in g:
+ shared_index = np.asarray(
+ as_zarr_array(g["feature_index"], name="feature_index")[:]
+ )
+ for cluster_name in g.group_keys():
+ marker_grp = as_zarr_group(g[cluster_name], name=cluster_name)
+ if "stats" not in marker_grp:
+ continue
+ stats = np.asarray(as_zarr_array(marker_grp["stats"], name="stats")[:])
+ top = np.argsort(-stats[:, 0])[:topn]
+ feat_idx.extend(shared_index[top])
+ else:
+ for i in g.group_keys():
+ marker_grp = as_zarr_group(g[i], name=i)
+ if "feature_index" in marker_grp:
+ feat_idx.extend(
+ np.asarray(
+ as_zarr_array(
+ marker_grp["feature_index"], name="feature_index"
+ )[:topn]
+ )
+ )
if len(feat_idx) == 0:
raise ValueError("ERROR: Marker list is empty for all the groups")
- feat_idx = np.array(sorted(set(feat_idx))).astype(int)
+ feat_idx_arr = np.array(sorted(set(feat_idx))).astype(int)
cell_idx = np.array(assay.cells.active_index(cell_key))
normed_data = assay.normed(
cell_idx=cell_idx,
- feat_idx=feat_idx,
+ feat_idx=feat_idx_arr,
log_transform=log_transform,
)
- nc = normed_data.chunks[0]
- # FIXME: avoid conversion to dask dataframe here
- # Unfortunately doing this dask array in a loop is 10x slower
- normed_data = normed_data.to_dask_dataframe()
- groups = daskarr.from_array(
- assay.cells.fetch(group_key, cell_key), chunks=nc
- ).to_dask_dataframe()
- df = controlled_compute(normed_data.groupby(groups).mean(), self.nthreads)
+ groups = assay.cells.fetch(group_key, cell_key)
+ # Streaming per-group mean: accumulate per-block group sums and counts,
+ # then divide. Groups are ordered like a pandas groupby (sorted labels).
+ group_sums: dict = {}
+ group_counts: dict = {}
+ row_start = 0
+ for block in tqdmbar(
+ normed_data.blocks,
+ total=normed_data.numblocks[0],
+ desc="Aggregating marker values per group",
+ ):
+ a = controlled_compute(block, self.nthreads)
+ block_groups = groups[row_start : row_start + a.shape[0]]
+ row_start += a.shape[0]
+ bdf = pd.DataFrame(a)
+ bdf["__group__"] = block_groups
+ grouped = bdf.groupby("__group__")
+ block_sum = grouped.sum()
+ block_count = grouped.size()
+ for label in block_sum.index:
+ vals = block_sum.loc[label].to_numpy(dtype=np.float64)
+ if label not in group_sums:
+ group_sums[label] = vals
+ group_counts[label] = int(block_count.loc[label])
+ else:
+ group_sums[label] += vals
+ group_counts[label] += int(block_count.loc[label])
+ labels = sorted(group_sums.keys())
+ df = pd.DataFrame(
+ np.vstack([group_sums[label] / group_counts[label] for label in labels]),
+ index=labels,
+ )
df = df.apply(lambda x: (x - x.mean()) / x.std(), axis=0)
- df.columns = assay.feats.fetch_all("names")[feat_idx]
+ df.columns = assay.feats.fetch_all("names")[feat_idx_arr]
df = df.T
# noinspection PyTypeChecker
df[df < vmin] = vmin
@@ -1931,23 +2582,23 @@ def plot_marker_heatmap(
def plot_pseudotime_heatmap(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
- feature_cluster_key: Optional[str] = None,
- pseudotime_key: Optional[str] = None,
- show_features: Optional[list] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ feature_cluster_key: str | None = None,
+ pseudotime_key: str | None = None,
+ show_features: list[str] | None = None,
width: int = 5,
height: int = 10,
vmin: float = -2.0,
vmax: float = 2.0,
- heatmap_cmap: Optional[str] = None,
- pseudotime_cmap: Optional[str] = None,
- clusterbar_cmap: Optional[str] = None,
+ heatmap_cmap: str | None = None,
+ pseudotime_cmap: str | None = None,
+ clusterbar_cmap: str | None = None,
tick_fontsize: int = 10,
axis_fontsize: int = 12,
feature_label_fontsize: int = 12,
- savename: Optional[str] = None,
+ savename: str | None = None,
save_dpi: int = 300,
show_fig: bool = True,
) -> None:
@@ -1983,7 +2634,7 @@ def plot_pseudotime_heatmap(
value. (Default value: 2.0)
heatmap_cmap: Colormap for the heatmap (Default value: coolwarm)
pseudotime_cmap: Colormap for the pseudotime bar. It should be some kind of continuous colormap.
- (Default value: cmocean.deep)
+ (Default value: viridis)
clusterbar_cmap: Colormap for the cluster bar showing the span of each feature cluster.
(Default value: tab20)
tick_fontsize: Font size for cbar ticks (Default value: 10)
@@ -2000,17 +2651,28 @@ def plot_pseudotime_heatmap(
from ..plots import plot_annotated_heatmap
assay = self._get_assay(from_assay)
- for i in [cell_key, feat_key, feature_cluster_key, feature_cluster_key]:
- if i is None:
- var_name = list(dict(i=i).keys())[0] # Trick to get variables own name
- raise ValueError(
- f"ERROR: Please provide a value for parameter `{var_name}`"
- )
+ if cell_key is None:
+ raise ValueError("ERROR: Please provide a value for parameter `cell_key`")
+ if feat_key is None:
+ raise ValueError("ERROR: Please provide a value for parameter `feat_key`")
+ if feature_cluster_key is None:
+ raise ValueError(
+ "ERROR: Please provide a value for parameter `feature_cluster_key`"
+ )
+ if pseudotime_key is None:
+ raise ValueError(
+ "ERROR: Please provide a value for parameter `pseudotime_key`"
+ )
- cell_ordering = assay.cells.fetch(pseudotime_key, key=cell_key)
+ cell_ordering = np.asarray(
+ assay.cells.fetch(pseudotime_key, key=cell_key),
+ dtype=float,
+ )
# noinspection PyProtectedMember
cell_idx, feat_idx = assay._get_cell_feat_idx(cell_key, feat_key)
- hashes = [hash(tuple(x)) for x in (cell_idx, feat_idx, cell_ordering)]
+ hashes = [
+ array_digest(np.asarray(x)) for x in (cell_idx, feat_idx, cell_ordering)
+ ]
location = f"aggregated_{cell_key}_{feat_key}_{pseudotime_key}"
if location not in assay.z:
raise KeyError(
@@ -2018,29 +2680,79 @@ def plot_pseudotime_heatmap(
f"Please make sure that you have run `run_pseudotime_aggregation` with the same values for "
f"parameters: `cell_key`, `feat_key` and `pseudotime_key`"
)
- else:
- if hashes != assay.z[location].attrs["hashes"]:
- raise ValueError(
- "ERROR: The values under one or more of these columns: `cell_key`, `feat_key` or/and "
- "`pseudotime_key have been updated after running `run_pseudotime_aggregation`"
- )
+ agg_grp = as_zarr_group(assay.z[location], name=location)
+ if agg_grp.attrs.get("schema_version") != PSEUDOTIME_AGGREGATION_SCHEMA_VERSION:
+ raise ValueError(
+ f"Aggregated data at '{location}' uses an old cache schema. "
+ "Rerun run_pseudotime_aggregation before plotting"
+ )
+ if hashes != cast(list[str], agg_grp.attrs["hashes"]):
+ raise ValueError(
+ "ERROR: The values under one or more of these columns: `cell_key`, `feat_key` or/and "
+ "`pseudotime_key have been updated after running `run_pseudotime_aggregation`"
+ )
- da = daskarr.from_zarr(assay.z[location + "/data"], inline_array=True)
- feature_indices = assay.z[location + "/feature_indices"][:]
- da = da[: feature_indices.shape[0]]
+ da = ChunkedArray(
+ as_zarr_array(agg_grp["data"], name="data"), nthreads=self.nthreads
+ )
+ feature_indices = np.asarray(
+ as_zarr_array(agg_grp["feature_indices"], name="feature_indices")[:]
+ )
+ if "valid_features" not in agg_grp:
+ raise ValueError(
+ f"Aggregated data at '{location}' has no valid_features mask. "
+ "Rerun run_pseudotime_aggregation"
+ )
+ valid_features = np.asarray(
+ as_zarr_array(agg_grp["valid_features"], name="valid_features")[:],
+ dtype=bool,
+ )
+ if valid_features.shape[0] != feature_indices.shape[0]:
+ raise ValueError(
+ "Aggregated feature indices and validity mask are misaligned"
+ )
+ da_arr = np.asarray(da[: feature_indices.shape[0]])
+ if da_arr.shape[0] != feature_indices.shape[0]:
+ raise ValueError(
+ "Aggregated feature matrix and feature indices are misaligned"
+ )
+ da_arr = da_arr[valid_features]
+ feature_indices = feature_indices[valid_features]
+ if not np.isfinite(da_arr).all():
+ raise ValueError("Aggregated feature matrix contains non-finite values")
+
+ all_feature_clusters = assay.feats.fetch_all(feature_cluster_key)
+ cached_cluster_label = agg_grp.attrs.get("cluster_label")
+ cached_cluster_digest = agg_grp.attrs.get("cluster_digest")
+ current_cluster_digest = _group_assignment_digest(all_feature_clusters)
+ if cached_cluster_label is None or cached_cluster_digest is None:
+ raise ValueError(
+ "Aggregated data has no completed feature-clustering provenance. "
+ "Rerun run_pseudotime_aggregation"
+ )
+ if cached_cluster_label != feature_cluster_key:
+ logger.warning(
+ f"Heatmap requested feature clusters '{feature_cluster_key}', but "
+ f"the aggregation cache was clustered as '{cached_cluster_label}'"
+ )
+ if cached_cluster_digest != current_cluster_digest:
+ logger.warning(
+ f"Feature cluster column '{feature_cluster_key}' changed after "
+ "aggregation and may be stale"
+ )
- feature_clusters = assay.feats.fetch_all(feature_cluster_key)[feature_indices]
+ feature_clusters = all_feature_clusters[feature_indices]
feature_labels = assay.feats.fetch_all("names")[feature_indices]
idx = np.argsort(feature_clusters)
feature_clusters = feature_clusters[idx]
feature_labels = feature_labels[idx]
- da = da.compute()[idx]
+ da_arr = da_arr[idx]
ordering = assay.cells.fetch(pseudotime_key, key=cell_key)
plot_annotated_heatmap(
- df=da,
+ df=da_arr,
xbar_values=ordering,
ybar_values=feature_clusters,
display_row_labels=show_features,
@@ -2064,11 +2776,12 @@ def metric_lisi(
self,
label_colnames: Iterable[str],
use_latest_knn: bool = True,
- from_assay: Optional[str] = None,
- knn_loc: Optional[str] = None,
+ from_assay: str | None = None,
+ knn_loc: str | None = None,
save_result: bool = False,
return_lisi: bool = True,
- ) -> Optional[List[Tuple[str, np.ndarray]]]:
+ perplexity: float = 30,
+ ) -> list[tuple[str, np.ndarray]] | None:
"""Calculate Local Inverse Simpson Index (LISI) scores for cell populations.
LISI measures how well mixed different cell populations are in the local neighborhood
@@ -2079,17 +2792,22 @@ def metric_lisi(
use_latest_knn: Whether to use the most recent KNN graph (default: True)
from_assay: Name of assay to use if not using latest KNN
knn_loc: Location of KNN graph if not using latest (default: None)
- save_result: Whether to save LISI scores to cell metadata (default: True)
- return_lisi: Whether to return LISI scores (default: False)
+ save_result: Whether to save LISI scores to cell metadata (default: False)
+ return_lisi: Whether to return LISI scores (default: True)
+ perplexity: Effective neighborhood size used by LISI. It is reduced
+ with a warning when the graph has fewer than three times this
+ many neighbors.
Returns:
If return_lisi is True, returns list of tuples containing:
+
- Label column name
- numpy array of LISI scores for that label
+
If return_lisi is False, returns None
Raises:
- ValueError: If using custom KNN graph but required parameters are missing
+ ValueError: If KNN inputs, perplexity, or labels are invalid
KeyError: If label columns not found in cell metadata
Notes:
@@ -2098,9 +2816,12 @@ def metric_lisi(
Higher scores indicate more mixing between different labels.
"""
+ label_cols = list(label_colnames)
+ if from_assay is None:
+ from_assay = self._load_default_assay()
+
if use_latest_knn and knn_loc is None:
knn_loc = self._get_latest_knn_loc(from_assay)
- cell_key = self.zw[self._load_default_assay()].attrs["latest_cell_key"]
logger.info(f"Using the latest knn graph at location: {knn_loc}")
else:
@@ -2111,32 +2832,40 @@ def metric_lisi(
logger.info(f"Using the knn graph at location: {knn_loc}")
- knn = self.zw[knn_loc]
+ normed_part = knn_loc.split("/")[1]
+ _, cell_key, _ = normed_part.split("__")
+ knn_grp = as_zarr_group(self.zw[knn_loc], name=knn_loc)
- distances = knn["distances"]
- indices = knn["indices"]
+ distances = as_zarr_array(knn_grp["distances"], name="distances")
+ indices = as_zarr_array(knn_grp["indices"], name="indices")
try:
- metadata = self.cells.to_pandas_dataframe(
- columns=label_colnames + [cell_key]
- )
+ metadata = self.cells.to_pandas_dataframe(columns=label_cols + [cell_key])
metadata = metadata[metadata[cell_key]]
except KeyError:
raise KeyError(
- f"Could not find the column(s) {label_colnames} in the cell metadata table."
+ f"Could not find the column(s) {label_cols} in the cell metadata table."
)
from ..metrics import compute_lisi
- lisi_scores = compute_lisi(distances, indices, metadata, label_colnames)
+ lisi_scores = compute_lisi(
+ distances,
+ indices,
+ metadata,
+ label_cols,
+ perplexity=perplexity,
+ )
# lisi_scores Shape -> (n_cells, n_labels)
if save_result:
- for col, vals in zip(label_colnames, lisi_scores.T):
+ for col, vals in zip(label_cols, lisi_scores.T):
col_name = f"lisi__{col}__{knn_loc.split('/')[-1]}"
- self.cells.insert(column_name=col_name, values=vals, overwrite=True)
+ self.cells.insert(
+ column_name=col_name, values=vals, overwrite=True, key=cell_key
+ )
if return_lisi:
- return list(zip(label_colnames, lisi_scores.T))
+ return list(zip(label_cols, lisi_scores.T))
else:
return None
@@ -2144,9 +2873,11 @@ def metric_silhouette(
self,
use_latest_knn: bool = True,
res_label: str = "leiden_cluster",
- from_assay: Optional[str] = None,
- knn_loc: Optional[str] = None,
- ) -> Optional[np.ndarray]:
+ from_assay: str | None = None,
+ knn_loc: str | None = None,
+ random_seed: int = 4444,
+ sample_size: int = 11,
+ ) -> np.ndarray | None:
"""Calculate modified silhouette scores for evaluating cluster separation.
This implements a graph-based silhouette score that measures how similar cells
@@ -2154,15 +2885,18 @@ def metric_silhouette(
Args:
use_latest_knn: Whether to use most recent KNN graph (default: True)
- res_label: Column name containing cluster labels (default: "leiden_cluster")
+ res_label: Base or full column name containing cluster labels
+ (default: "leiden_cluster")
from_assay: Name of assay to use if not using latest KNN (default: None)
knn_loc: Location of KNN graph if not using latest (default: None)
+ random_seed: Seed used for cluster sampling.
+ sample_size: Maximum size of each sampled cluster group.
Returns:
numpy array of silhouette scores for each cluster, or None if computation fails
Raises:
- ValueError: If using custom KNN graph but required parameters are missing
+ ValueError: If graph, labels, sampling, or embedding data are invalid
Notes:
Scores range from -1 to 1:
@@ -2174,18 +2908,11 @@ def metric_silhouette(
NaN values indicate clusters that couldn't be scored due to size constraints.
"""
- def compute_graph_feats(knn_loc: str):
- k = knn_loc.rsplit("/", 1)[-1].split("__")[-1]
- dims = knn_loc.rsplit("/", 2)[0].split("__")[-2]
- feat_key = knn_loc.split("/")[1].split("__")[-1]
- return k, dims, feat_key
-
if from_assay is None:
from_assay = self._load_default_assay()
if use_latest_knn and knn_loc is None:
knn_loc = self._get_latest_knn_loc(from_assay)
- k, dims, feat_key = compute_graph_feats(knn_loc)
logger.info(
f"Using the latest knn graph at location: {knn_loc} for assay: {from_assay}"
)
@@ -2195,66 +2922,150 @@ def compute_graph_feats(knn_loc: str):
raise ValueError("Please provide values for the KNN graph location.")
if knn_loc not in self.zw:
raise ValueError(f"Could not find the knn graph at location: {knn_loc}")
- k, dims, feat_key, from_assay = compute_graph_feats(knn_loc)
logger.info(f"Using the knn graph at location: {knn_loc}")
- from ..metrics import knn_to_csr_matrix, silhouette_scoring
+ from ..metrics import silhouette_scoring
- isHarmonized = self.zw[knn_loc.rsplit("/", 1)[0]].attrs["isHarmonized"]
-
- batches = None
- if isHarmonized:
- batches = self.zw[knn_loc.rsplit("/", 2)[0] + "/harmonizedData"].attrs[
- "batches"
- ]
-
- ann_obj = self.make_graph(
- feat_key=feat_key,
- dims=dims,
- k=k,
- return_ann_object=True,
- harmonize=isHarmonized,
- batch_columns=batches,
+ normed_part = knn_loc.split("/")[1]
+ _, cell_key, feat_key_parsed = normed_part.split("__")
+ ann_obj = self._load_ann_stream(
+ from_assay=from_assay,
+ cell_key=cell_key,
+ feat_key=feat_key_parsed,
+ knn_loc=knn_loc,
)
- graph = knn_to_csr_matrix(self.z[knn_loc].indices, self.z[knn_loc].distances)
- hvg_data = self.z[knn_loc.rsplit("/", 3)[0] + "/data"]
+ knn_grp = as_zarr_group(self.z[knn_loc], name=knn_loc)
+ neighbor_indices = as_zarr_array(knn_grp["indices"], name="indices")
+ neighbor_distances = as_zarr_array(knn_grp["distances"], name="distances")
+ if ann_obj.harmonizedData is not None:
+ metric_data = ann_obj.harmonizedData
+ data_is_reduced = True
+ else:
+ if ann_obj.harmonize:
+ raise ValueError("Harmony coordinates are missing for this KNN graph")
+ metric_data = ann_obj.data
+ data_is_reduced = False
scores = silhouette_scoring(
- self, ann_obj, graph, hvg_data, from_assay, res_label
+ self,
+ ann_obj,
+ None,
+ metric_data,
+ from_assay,
+ res_label,
+ cell_key=cell_key,
+ random_seed=random_seed,
+ sample_size=sample_size,
+ data_is_reduced=data_is_reduced,
+ distance_metric=cast(Any, ann_obj.annMetric),
+ neighbor_indices=neighbor_indices,
+ neighbor_distances=neighbor_distances,
)
return scores
- def metric_integration(
- self, batch_labels: List[str], metric: Literal["ari", "nmi"] = "ari"
- ) -> Optional[float]:
- """Calculate integration score between different batch labels.
+ def metric_label_concordance(
+ self,
+ label_columns: Sequence[str],
+ metric: Literal["ari", "nmi"] = "ari",
+ ) -> float:
+ """Compare two metadata label partitions using ARI or NMI.
- Measures how well aligned different batches are after integration by comparing
- their cluster assignments using standard clustering metrics.
+ This measures whether two labelings of the same cells agree, for
+ example predicted clusters against imported reference annotations. It
+ does not measure batch mixing; use :meth:`metric_batch_mixing` or
+ :meth:`metric_lisi` for that.
Args:
- batch_labels: List of column names containing batch labels to compare
- metric: Metric to use for comparison (default: "ari")
- - "ari": Adjusted Rand Index
- - "nmi": Normalized Mutual Information
+ label_columns: Exactly two cell metadata column names to compare.
+ metric: ``"ari"`` for the adjusted Rand index or ``"nmi"`` for
+ normalized mutual information.
Returns:
- Integration score between 0 and 1, or None if metric is not recognized
- Higher scores indicate better alignment between batches
+ Agreement between the two partitions. ARI ranges from -1 to 1 and
+ NMI from 0 to 1, with higher values meaning stronger agreement.
- Notes:
- ARI and NMI measure the agreement between different batch labelings:
- - Score near 1: Batches are well integrated
- - Score near 0: Batches show poor integration
+ Raises:
+ ValueError: If the number of columns or the metric name is invalid.
+ """
+ from ..metrics import label_concordance_score
- ARI is adjusted for chance and generally more stringent than NMI.
+ label_values = [
+ np.asarray(self.cells.fetch_all(column)) for column in label_columns
+ ]
+ return label_concordance_score(label_values, metric)
+
+ def metric_integration(
+ self,
+ batch_labels: list[str],
+ metric: Literal["ari", "nmi"] = "ari",
+ ) -> float:
+ """Backward-compatible alias for :meth:`metric_label_concordance`.
+
+ This method compares label agreement and does not measure neighborhood
+ mixing. Use :meth:`metric_batch_mixing` to evaluate batch integration.
"""
- from ..metrics import integration_score
+ logger.warning(
+ "`metric_integration` measures label concordance. Use "
+ "`metric_label_concordance` for ARI/NMI or `metric_batch_mixing` "
+ "for neighborhood integration quality."
+ )
+ return self.metric_label_concordance(batch_labels, metric)
- batch_labels_vals = []
- for batch in batch_labels:
- vals = np.array(self.cells.fetch_all(batch))
- batch_labels_vals.append(vals)
+ def metric_batch_mixing(
+ self,
+ label_colname: str,
+ use_latest_knn: bool = True,
+ from_assay: str | None = None,
+ knn_loc: str | None = None,
+ perplexity: float = 30,
+ ) -> float:
+ """Summarize batch LISI as a normalized neighborhood-mixing score.
- scores = integration_score(batch_labels_vals, metric)
- return scores
+ This computes batch LISI on the current KNN graph and rescales its mean
+ against the mixing that perfectly integrated data would reach given the
+ dataset's batch sizes. Unlike raw LISI, the result is bounded in
+ ``[0, 1]``, which makes it easier to compare across graphs and datasets.
+
+ Args:
+ label_colname: Cell metadata column holding the batch assignment.
+ use_latest_knn: Whether to use the most recent KNN graph
+ (default: True).
+ from_assay: Name of assay to use if not using the latest KNN.
+ knn_loc: Location of the KNN graph if not using the latest.
+ perplexity: Effective neighborhood size passed to LISI.
+
+ Returns:
+ A value in ``[0, 1]``. Scores near 1 indicate that neighborhoods mix
+ batches as well as the global composition allows, and scores near 0
+ indicate poorly mixed batches.
+
+ Raises:
+ ValueError: If KNN inputs are invalid or the column has fewer than
+ two batches.
+ """
+ from ..metrics import lisi_batch_mixing_score
+
+ if from_assay is None:
+ from_assay = self._load_default_assay()
+ resolved_knn_loc = knn_loc
+ if use_latest_knn and resolved_knn_loc is None:
+ resolved_knn_loc = self._get_latest_knn_loc(from_assay)
+ if resolved_knn_loc is None:
+ raise ValueError("Please provide values for the KNN graph location.")
+
+ lisi_result = self.metric_lisi(
+ label_colnames=[label_colname],
+ use_latest_knn=use_latest_knn,
+ from_assay=from_assay,
+ knn_loc=resolved_knn_loc,
+ save_result=False,
+ return_lisi=True,
+ perplexity=perplexity,
+ )
+ if lisi_result is None:
+ raise RuntimeError("LISI computation did not return scores")
+
+ normed_part = resolved_knn_loc.split("/")[1]
+ _, cell_key, _ = normed_part.split("__")
+ batch_labels = self.cells.fetch(label_colname, key=cell_key)
+ return lisi_batch_mixing_score(lisi_result[0][1], batch_labels)
diff --git a/scarf/datastore/graph_datastore.py b/scarf/datastore/graph_datastore.py
index 7ba0b1df..2f628171 100644
--- a/scarf/datastore/graph_datastore.py
+++ b/scarf/datastore/graph_datastore.py
@@ -1,17 +1,244 @@
+import hashlib
+import json
import os
-from typing import Callable, List, Optional, Tuple, Union
+import shutil
+import tempfile
+import time
+from collections.abc import Callable, Iterator
+from contextlib import contextmanager
+from typing import Any, Literal, cast
import numpy as np
import pandas as pd
-from dask.array import from_zarr # type: ignore
+import zarr
from loguru import logger
+from numpy.typing import NDArray
from scipy.sparse import coo_matrix, csr_matrix
-
-from ..assay import Assay
-from ..utils import clean_array, show_dask_progress, system_call, tqdmbar
+from scipy.sparse.csgraph import connected_components
+from scipy.sparse.linalg import ArpackNoConvergence, svds
+
+from .._types import as_zarr_array, as_zarr_group
+from ..ann import AnnStream
+from ..assay import Assay, RNAassay
+from ..chunked import ChunkedArray
+from ..mapping_reference import MAPPING_REFERENCE_SCHEMA_VERSION, MappingReference
+from ..storage.zarr_store import (
+ copy_zarr_array,
+ create_or_open_staged_normed_array,
+ has_ann_index,
+ is_remote_datastore,
+ legacy_ann_index_path,
+ load_ann_index,
+ load_ann_index_from_path,
+ open_or_create_staged_normed_array,
+ save_ann_index,
+ zarr_root_path,
+)
+from ..utils import clean_array, show_dask_progress
from ..writers import create_zarr_dataset
from .base_datastore import BaseDataStore
+_HARMONY_ANN_CONTRACT_VERSION = 1
+
+
+def _validate_source_sink_labels(
+ labels: pd.Series,
+ sources: list[Any],
+ sinks: list[Any],
+ context: str,
+) -> None:
+ overlap = sorted(set(sources) & set(sinks))
+ if overlap:
+ raise ValueError(f"Source and sink labels overlap in {context}: {overlap}")
+
+ present = set(pd.unique(labels))
+ missing_sources = [source for source in sources if source not in present]
+ missing_sinks = [sink for sink in sinks if sink not in present]
+ if missing_sources or missing_sinks:
+ raise ValueError(
+ f"Source/sink labels were not found in {context}. "
+ f"Missing sources: {missing_sources}; missing sinks: {missing_sinks}"
+ )
+
+
+def _make_source_sink_vector(
+ labels: pd.Series,
+ sources: list[Any],
+ sinks: list[Any],
+) -> np.ndarray:
+ sink_mask = labels.isin(sinks).to_numpy(dtype=bool)
+ source_mask = labels.isin(sources).to_numpy(dtype=bool)
+ labelled = sink_mask | source_mask
+ if labelled.all():
+ raise ValueError(
+ "All selected cells are labelled as sources or sinks, so the "
+ "source/sink vector cannot be balanced over unlabelled cells"
+ )
+
+ vector = np.zeros(labels.shape[0], dtype=float)
+ vector[sink_mask] = 1.0
+ vector[source_mask] = -1.0
+ vector[~labelled] = -vector.sum() / int((~labelled).sum())
+ return vector
+
+
+def _validate_source_sink_vector(
+ values: np.ndarray,
+ n_cells: int,
+ context: str,
+) -> np.ndarray:
+ try:
+ vector = np.asarray(values, dtype=float)
+ except (TypeError, ValueError) as exc:
+ raise TypeError(f"{context} must contain numeric values") from exc
+
+ if vector.ndim == 2 and vector.shape == (n_cells, 1):
+ vector = vector[:, 0]
+ elif vector.ndim != 1:
+ raise ValueError(
+ f"{context} must be one-dimensional or have shape ({n_cells}, 1)"
+ )
+
+ if vector.shape[0] != n_cells:
+ raise ValueError(
+ f"Size mismatch between {context} ({vector.shape[0]}) and graph ({n_cells})"
+ )
+ if not np.isfinite(vector).all():
+ raise ValueError(f"{context} must contain only finite values")
+
+ tolerance = 1e-10 * max(1.0, float(np.abs(vector).sum()))
+ if not np.isclose(vector.sum(), 0.0, atol=tolerance, rtol=0.0):
+ raise ValueError(f"The values in {context} must sum to zero")
+ return vector
+
+
+def _select_pseudotime_component(
+ graph: csr_matrix,
+ selected_cell_indices: np.ndarray,
+ component_policy: Literal["largest", "error"],
+) -> tuple[np.ndarray, list[int]]:
+ if component_policy not in {"largest", "error"}:
+ raise ValueError("component_policy must be either 'largest' or 'error'")
+
+ n_components, component_labels = connected_components(
+ graph, directed=False, return_labels=True
+ )
+ sizes = np.bincount(component_labels, minlength=n_components).astype(int).tolist()
+ if n_components == 1:
+ return np.ones(graph.shape[0], dtype=bool), sizes
+ if component_policy == "error":
+ raise ValueError(
+ f"The selected graph has {n_components} connected components with sizes {sizes}"
+ )
+
+ retained_component = min(
+ range(n_components),
+ key=lambda component_id: (
+ -sizes[component_id],
+ int(selected_cell_indices[component_labels == component_id].min()),
+ ),
+ )
+ return np.asarray(component_labels == retained_component, dtype=bool), sizes
+
+
+def _random_walk_laplacian_transpose(graph: csr_matrix) -> csr_matrix:
+ degree = np.asarray(graph.sum(axis=1), dtype=float).ravel()
+ if np.any(degree <= 0):
+ raise ValueError("The retained graph contains isolated cells")
+ n_cells = graph.shape[0]
+ inverse_degree = csr_matrix(
+ (1.0 / degree, (range(n_cells), range(n_cells))),
+ shape=(n_cells, n_cells),
+ )
+ identity = csr_matrix(
+ (np.ones(n_cells), (range(n_cells), range(n_cells))),
+ shape=(n_cells, n_cells),
+ )
+ # For symmetric A, I - A @ D^-1 is the transpose of PBA's I - D^-1 @ A.
+ return identity - graph.dot(inverse_degree)
+
+
+def _truncated_pba_potential(
+ laplacian_transpose: csr_matrix,
+ n_singular_vals: int,
+ random_seed: int,
+ source_sink: np.ndarray,
+) -> np.ndarray:
+ random_state = np.random.RandomState(random_seed)
+ initial_vector = random_state.rand(laplacian_transpose.shape[0])
+ logger.info("Calculating SVD of graph laplacian. This might take a while...")
+ try:
+ left_vectors, singular_values, right_vectors_t = svds(
+ laplacian_transpose,
+ k=n_singular_vals,
+ which="SM",
+ v0=initial_vector,
+ )
+ except ArpackNoConvergence as exc:
+ raise RuntimeError(
+ "Pseudotime SVD did not converge. Try a smaller n_singular_vals value"
+ ) from exc
+
+ order = np.argsort(singular_values)
+ singular_values = singular_values[order]
+ left_vectors = left_vectors[:, order]
+ right_vectors = right_vectors_t[order, :].T
+
+ rank_tolerance = (
+ max(laplacian_transpose.shape)
+ * np.finfo(singular_values.dtype).eps
+ * max(1.0, float(singular_values.max()))
+ )
+ if singular_values[0] > max(1e-8, rank_tolerance):
+ raise ValueError("The graph Laplacian does not contain the expected null mode")
+ if np.any(singular_values[1:] <= rank_tolerance):
+ raise ValueError(
+ "The graph Laplacian contains additional near-zero singular modes"
+ )
+
+ inverse_singular_values = 1.0 / singular_values[1:]
+ left_vectors = left_vectors[:, 1:]
+ right_vectors = right_vectors[:, 1:]
+ # The input is L.T, so U @ inv(S) @ V.T applies the PBA operator pinv(L).
+ return np.asarray(
+ left_vectors
+ @ (inverse_singular_values * (right_vectors.T @ source_sink).ravel())
+ )
+
+
+class _GraphBuildProgress:
+ """Step-wise progress logging and timing for ``make_graph``."""
+
+ def __init__(self, total: int) -> None:
+ self._total = total
+ self._step = 0
+ self._started = time.perf_counter()
+ self._records: list[tuple[int, str, float, str]] = []
+
+ @contextmanager
+ def step(self, name: str, *, cached: bool = False) -> Iterator[None]:
+ self._step += 1
+ mode = "reusing cached" if cached else "computing"
+ label = f"{self._step}/{self._total}"
+ logger.info(f"make_graph step {label}: {name} ({mode})")
+ t0 = time.perf_counter()
+ try:
+ yield
+ finally:
+ elapsed = time.perf_counter() - t0
+ self._records.append((self._step, name, elapsed, mode))
+ logger.info(
+ f"make_graph step {label}: {name} finished in {elapsed:.1f}s ({mode})"
+ )
+
+ def finish(self) -> None:
+ total_elapsed = time.perf_counter() - self._started
+ accounted = sum(r[2] for r in self._records)
+ logger.info(
+ f"make_graph finished in {total_elapsed:.1f}s "
+ f"({self._step}/{self._total} steps, {accounted:.1f}s in logged steps)"
+ )
+
class GraphDataStore(BaseDataStore):
"""This class extends BaseDataStore by providing methods required to
@@ -26,9 +253,274 @@ class GraphDataStore(BaseDataStore):
z: The Zarr file (directory) used for this datastore instance.
"""
- def __init__(self, **kwargs):
+ def __init__(self, **kwargs: Any) -> None:
super().__init__(**kwargs)
+ def _mapping_reference_metadata(
+ self,
+ assay: Assay,
+ from_assay: str,
+ cell_key: str,
+ feat_key: str,
+ reduction_loc: str,
+ ann_loc: str,
+ batch_columns: list[str],
+ ) -> dict[str, Any]:
+ from ..mapping_utils import array_hash, array_store_hash
+
+ normed_loc = f"{from_assay}/normed__{cell_key}__{feat_key}"
+ normed_group = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ reduction_group = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ feature_key = f"{cell_key}__{feat_key}" if feat_key != "I" else "I"
+ feature_ids = assay.feats.fetch("ids", key=feature_key)
+ batch_values = (
+ np.column_stack(
+ [self.cells.fetch(column, key=cell_key) for column in batch_columns]
+ )
+ if batch_columns
+ else np.empty((len(self.cells.active_index(cell_key)), 0), dtype=str)
+ )
+ loadings = np.asarray(
+ as_zarr_array(reduction_group["reduction"], name="reduction")[:]
+ )
+ corrected_hash = ""
+ if "harmonizedData" in reduction_group:
+ corrected_hash = array_store_hash(
+ as_zarr_array(reduction_group["harmonizedData"], name="harmonizedData")
+ )
+ ann_group = as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ ann_parts = ann_loc.rsplit("/", 1)[-1].split("__")
+ return {
+ "assay": from_assay,
+ "cellKey": cell_key,
+ "featureKey": feat_key,
+ "reductionPath": reduction_loc,
+ "annPath": ann_loc,
+ "featureHash": array_hash(feature_ids),
+ "cellHash": array_hash(self.cells.fetch("ids", key=cell_key)),
+ "batchValueHash": array_hash(batch_values),
+ "batchColumns": batch_columns,
+ "subsetHash": normed_group.attrs["subset_hash"],
+ "subsetParams": normed_group.attrs["subset_params"],
+ "loadingsHash": array_hash(loadings),
+ "correctedCoordinatesHash": corrected_hash,
+ "reductionMethod": "pca",
+ "annContract": {
+ "metric": ann_parts[1],
+ "efConstruction": int(ann_parts[2]),
+ "ef": int(ann_parts[3]),
+ "m": int(ann_parts[4]),
+ "randomState": int(ann_parts[5]),
+ "featureScaling": bool(ann_group.attrs.get("featureScaling", True)),
+ },
+ }
+
+ def _persist_mapping_reference(
+ self,
+ assay: Assay,
+ from_assay: str,
+ cell_key: str,
+ feat_key: str,
+ reduction_loc: str,
+ ann_loc: str,
+ knn_loc: str,
+ batch_columns: list[str],
+ ann_obj: AnnStream,
+ ) -> None:
+ from ..mapping_reference import persist_mapping_reference
+ from ..mapping_utils import _distance_quantile_summary
+ from ..symphony import SymphonyReferenceModel, weighted_centroids
+
+ if ann_obj.harmonyResult is None:
+ return
+ if ann_obj.loadings is None:
+ raise RuntimeError(
+ "Cannot persist a mapping reference without PCA loadings"
+ )
+ harmony = ann_obj.harmonyResult
+ try:
+ cluster_mass, raw_centroids = weighted_centroids(
+ harmony.original.T, harmony.assignments
+ )
+ _, corrected_centroids = weighted_centroids(
+ harmony.corrected.T, harmony.assignments
+ )
+ except ValueError as exc:
+ if "empty cluster" not in str(exc):
+ raise
+ raise ValueError(
+ "Harmony produced an empty reference cluster. Rebuild with a "
+ "smaller harmony_params['nclust'] value."
+ ) from exc
+ ridge_values = np.diag(harmony.ridge)[1:]
+ correction_ridge = (
+ float(np.mean(ridge_values[ridge_values > 0]))
+ if np.any(ridge_values > 0)
+ else 1.0
+ )
+ model = SymphonyReferenceModel(
+ feature_means=ann_obj.mu,
+ feature_scales=ann_obj.sigma,
+ loadings=ann_obj.loadings,
+ centroids=harmony.centroids.T,
+ raw_centroids=raw_centroids,
+ corrected_centroids=corrected_centroids,
+ cluster_mass=cluster_mass,
+ sigma=harmony.sigma,
+ correction_ridge=correction_ridge,
+ )
+ reduction_group = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ feature_key = f"{cell_key}__{feat_key}" if feat_key != "I" else "I"
+ feature_ids = assay.feats.fetch("ids", key=feature_key)
+ metadata = self._mapping_reference_metadata(
+ assay,
+ from_assay,
+ cell_key,
+ feat_key,
+ reduction_loc,
+ ann_loc,
+ batch_columns,
+ )
+ metadata["harmonyParameters"] = harmony.parameters
+ metadata["batchLevels"] = [list(levels) for levels in harmony.batch_levels]
+ knn_group = as_zarr_group(self.zw[knn_loc], name=knn_loc)
+ distance_quantiles, distance_values = _distance_quantile_summary(
+ as_zarr_array(knn_group["distances"], name="distances")
+ )
+ persist_mapping_reference(
+ reduction_group,
+ model,
+ feature_ids,
+ metadata,
+ reference_distance_quantiles=distance_quantiles,
+ reference_distance_values=distance_values,
+ )
+
+ def get_mapping_reference(
+ self,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ ) -> MappingReference:
+ """Load a validated immutable RNA/PCA Symphony-style mapping reference."""
+ from ..mapping_reference import (
+ load_mapping_reference,
+ resolve_mapping_reference_group,
+ )
+
+ from_assay, cell_key, feat_key = self._get_latest_keys(
+ from_assay, cell_key, feat_key
+ )
+ assay = self._get_assay(from_assay)
+ if not isinstance(assay, RNAassay):
+ raise TypeError("Mapping references currently support RNA assays only")
+ normed_loc = f"{from_assay}/normed__{cell_key}__{feat_key}"
+ if normed_loc not in self.zw:
+ raise KeyError("No normalized reference data exists for the requested keys")
+ normed_group = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ reduction_loc = cast(str, normed_group.attrs.get("latest_reduction"))
+ if not reduction_loc or reduction_loc not in self.zw:
+ raise KeyError("No reduction exists for the requested reference")
+ reduction_group = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ ann_loc = cast(str, reduction_group.attrs.get("latest_ann"))
+ if not ann_loc or ann_loc not in self.zw:
+ raise KeyError("No ANN index exists for the requested reference")
+ try:
+ artifact, _, is_legacy = resolve_mapping_reference_group(reduction_group)
+ except KeyError:
+ raise ValueError(
+ "This harmonized graph predates the mapping-reference artifact. "
+ "Rebuild it with build_mapping_reference."
+ ) from None
+ artifact_ann_path = artifact.attrs.get("annPath")
+ if isinstance(artifact_ann_path, str):
+ ann_loc = artifact_ann_path
+ if ann_loc not in self.zw:
+ raise ValueError(
+ "The mapping-reference ANN index is missing. Rebuild the reference."
+ )
+ ann_group = as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ if not bool(ann_group.attrs.get("isHarmonized", False)):
+ raise ValueError(
+ "The requested graph is not harmonized. Build a harmonized mapping reference first."
+ )
+ batch_columns = [
+ str(column) for column in cast(list[Any], artifact.attrs["batchColumns"])
+ ]
+ expected = self._mapping_reference_metadata(
+ assay,
+ from_assay,
+ cell_key,
+ feat_key,
+ reduction_loc,
+ ann_loc,
+ batch_columns,
+ )
+ legacy_omissions = {"correctedCoordinatesHash", "annContract"}
+ for key, value in expected.items():
+ if is_legacy and key in legacy_omissions:
+ continue
+ if artifact.attrs.get(key) != value:
+ raise ValueError(
+ f"Mapping reference is stale because {key} no longer matches. "
+ "Rebuild it with build_mapping_reference."
+ )
+ return load_mapping_reference(
+ self, from_assay, cell_key, feat_key, reduction_loc, ann_loc
+ )
+
+ def build_mapping_reference(
+ self,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ batch_columns: list[str] | None = None,
+ **graph_kwargs: Any,
+ ) -> MappingReference:
+ """Build and return a versioned RNA/PCA Symphony-style mapping reference."""
+ if self.zarr_mode != "r+":
+ raise ValueError("Building a mapping reference requires a read-write store")
+ if batch_columns is None:
+ raise ValueError("batch_columns is required to build a mapping reference")
+ if graph_kwargs.get("feat_scaling", True) is False:
+ raise ValueError(
+ "Mapping references require feat_scaling=True because query "
+ "projection uses the stored reference mean and scale."
+ )
+ force_harmony_refit = bool(graph_kwargs)
+ if not force_harmony_refit:
+ try:
+ current = self.get_mapping_reference(from_assay, cell_key, feat_key)
+ except (KeyError, ValueError):
+ force_harmony_refit = True
+ else:
+ current_columns = [
+ str(column)
+ for column in cast(
+ list[Any], current.metadata.get("batchColumns", [])
+ )
+ ]
+ force_harmony_refit = (
+ current_columns != batch_columns
+ or current.metadata.get("schemaVersion")
+ != MAPPING_REFERENCE_SCHEMA_VERSION
+ )
+ self.make_graph(
+ from_assay=from_assay,
+ cell_key=cell_key,
+ feat_key=feat_key,
+ harmonize=True,
+ batch_columns=batch_columns,
+ _force_harmony_refit=force_harmony_refit,
+ **graph_kwargs,
+ )
+ reference = self.get_mapping_reference(from_assay, cell_key, feat_key)
+ if not bool(reference.metadata.get("complete", False)):
+ raise RuntimeError(
+ "Mapping reference build did not produce a complete artifact"
+ )
+ return reference
+
@staticmethod
def _choose_reduction_method(assay: Assay, reduction_method: str) -> str:
"""This is a convenience function to determine the linear dimension
@@ -62,24 +554,39 @@ def _choose_reduction_method(assay: Assay, reduction_method: str) -> str:
def _set_graph_params(
self,
- from_assay,
- cell_key,
- feat_key,
- log_transform=None,
- renormalize_subset=None,
- reduction_method="auto",
- dims=None,
- pca_cell_key=None,
- ann_metric=None,
- ann_efc=None,
- ann_ef=None,
- ann_m=None,
- rand_state=None,
- k=None,
- n_centroids=None,
- local_connectivity=None,
- bandwidth=None,
- ) -> tuple:
+ from_assay: str,
+ cell_key: str,
+ feat_key: str,
+ log_transform: bool | None = None,
+ renormalize_subset: bool | None = None,
+ reduction_method: str = "auto",
+ dims: int | None = None,
+ pca_cell_key: str | None = None,
+ ann_metric: str | None = None,
+ ann_efc: int | None = None,
+ ann_ef: int | None = None,
+ ann_m: int | None = None,
+ rand_state: int | None = None,
+ k: int | None = None,
+ n_centroids: int | None = None,
+ local_connectivity: float | None = None,
+ bandwidth: float | None = None,
+ ) -> tuple[
+ bool,
+ bool,
+ str,
+ int,
+ str,
+ str,
+ int,
+ int,
+ int,
+ int,
+ int,
+ int,
+ float,
+ float,
+ ]:
"""This function allows determination of values for the parameters of
`make_graph` function. This function harbours the default values for
each parameter. If parameter value is None, then before choosing the
@@ -110,7 +617,12 @@ def _set_graph_params(
Finalized values for the all the optional parameters in the same order
"""
- def log_message(category, name, value, custom_msg=None):
+ def log_message(
+ category: str,
+ name: str,
+ value: Any,
+ custom_msg: str | None = None,
+ ) -> bool:
"""Convenience function to log variable usage messages for
make_graph.
@@ -136,7 +648,7 @@ def log_message(category, name, value, custom_msg=None):
logger.info(custom_msg)
return True
- default_values = {
+ default_values: dict[str, Any] = {
"log_transform": True,
"renormalize_subset": True,
"dims": 11,
@@ -150,13 +662,19 @@ def log_message(category, name, value, custom_msg=None):
normed_loc = f"{from_assay}/normed__{cell_key}__{feat_key}"
if log_transform is None or renormalize_subset is None:
- if normed_loc in self.zw and "subset_params" in self.zw[normed_loc].attrs:
- # This works in coordination with save_normalized_data
- subset_params = self.zw[normed_loc].attrs["subset_params"]
- c_log_transform, c_renormalize_subset = (
- subset_params["log_transform"],
- subset_params["renormalize_subset"],
- )
+ if normed_loc in self.zw:
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ if "subset_params" in normed_grp.attrs:
+ # This works in coordination with save_normalized_data
+ subset_params = cast(
+ dict[str, Any], normed_grp.attrs["subset_params"]
+ )
+ c_log_transform, c_renormalize_subset = (
+ subset_params["log_transform"],
+ subset_params["renormalize_subset"],
+ )
+ else:
+ c_log_transform, c_renormalize_subset = None, None
else:
c_log_transform, c_renormalize_subset = None, None
if log_transform is None:
@@ -177,12 +695,13 @@ def log_message(category, name, value, custom_msg=None):
renormalize_subset = bool(renormalize_subset)
if dims is None or pca_cell_key is None:
- if (
- normed_loc in self.zw
- and "latest_reduction" in self.zw[normed_loc].attrs
- ):
- reduction_loc = self.zw[normed_loc].attrs["latest_reduction"]
- c_dims, c_pca_cell_key = reduction_loc.rsplit("__", 2)[1:]
+ if normed_loc in self.zw:
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ if "latest_reduction" in normed_grp.attrs:
+ reduction_loc_attr = cast(str, normed_grp.attrs["latest_reduction"])
+ c_dims, c_pca_cell_key = reduction_loc_attr.rsplit("__", 2)[1:]
+ else:
+ c_dims, c_pca_cell_key = None, None
else:
c_dims, c_pca_cell_key = None, None
if dims is None:
@@ -204,7 +723,7 @@ def log_message(category, name, value, custom_msg=None):
raise ValueError(
f"ERROR: `pca_use_cell_key` {pca_cell_key} does not exist in cell metadata"
)
- if self.cells.get_dtype(pca_cell_key) != bool:
+ if self.cells.get_dtype(pca_cell_key) != bool: # noqa: E721
raise TypeError(
"ERROR: Type of `pca_use_cell_key` column in cell metadata should be `bool`"
)
@@ -223,18 +742,28 @@ def log_message(category, name, value, custom_msg=None):
or ann_m is None
or rand_state is None
):
- if (
- reduction_loc in self.zw
- and "latest_ann" in self.zw[reduction_loc].attrs
- ):
- ann_loc = self.zw[reduction_loc].attrs["latest_ann"]
- (
- c_ann_metric,
- c_ann_efc,
- c_ann_ef,
- c_ann_m,
- c_rand_state,
- ) = ann_loc.rsplit("/", 1)[1].split("__")[1:]
+ if reduction_loc in self.zw:
+ reduction_grp = as_zarr_group(
+ self.zw[reduction_loc], name=reduction_loc
+ )
+ if "latest_ann" in reduction_grp.attrs:
+ ann_loc_attr = cast(str, reduction_grp.attrs["latest_ann"])
+ ann_parts = ann_loc_attr.rsplit("/", 1)[1].split("__")
+ (
+ c_ann_metric,
+ c_ann_efc,
+ c_ann_ef,
+ c_ann_m,
+ c_rand_state,
+ ) = ann_parts[1:6]
+ else:
+ c_ann_metric, c_ann_efc, c_ann_ef, c_ann_m, c_rand_state = (
+ None,
+ None,
+ None,
+ None,
+ None,
+ )
else:
c_ann_metric, c_ann_efc, c_ann_ef, c_ann_m, c_rand_state = (
None,
@@ -256,14 +785,14 @@ def log_message(category, name, value, custom_msg=None):
log_message("cached", "ann_efc", ann_efc)
else:
ann_efc = None # Will be set after value for k is determined
- log_message("default", "ann_efc", f"min(100, max(k * 3, 50))")
+ log_message("default", "ann_efc", "min(100, max(k * 3, 50))")
if ann_ef is None:
if c_ann_ef is not None:
ann_ef = int(c_ann_ef)
log_message("cached", "ann_ef", ann_ef)
else:
ann_ef = None # Will be set after value for k is determined
- log_message("default", "ann_ef", f"min(100, max(k * 3, 50))")
+ log_message("default", "ann_ef", "min(100, max(k * 3, 50))")
if ann_m is None:
if c_ann_m is not None:
ann_m = int(c_ann_m)
@@ -283,14 +812,19 @@ def log_message(category, name, value, custom_msg=None):
rand_state = int(rand_state)
if k is None:
- if (
- reduction_loc in self.zw
- and "latest_ann" in self.zw[reduction_loc].attrs
- ):
- ann_loc = self.zw[reduction_loc].attrs["latest_ann"]
- knn_loc = self.zw[ann_loc].attrs["latest_knn"]
- k = int(knn_loc.rsplit("__", 1)[1])
- log_message("cached", "k", k)
+ if reduction_loc in self.zw:
+ reduction_grp = as_zarr_group(
+ self.zw[reduction_loc], name=reduction_loc
+ )
+ if "latest_ann" in reduction_grp.attrs:
+ ann_loc_attr = cast(str, reduction_grp.attrs["latest_ann"])
+ ann_grp = as_zarr_group(self.zw[ann_loc_attr], name=ann_loc_attr)
+ knn_loc_attr = cast(str, ann_grp.attrs["latest_knn"])
+ k = int(knn_loc_attr.rsplit("__", 1)[1])
+ log_message("cached", "k", k)
+ else:
+ k = default_values["k"]
+ log_message("default", "k", k)
else:
k = default_values["k"]
log_message("default", "k", k)
@@ -301,31 +835,41 @@ def log_message(category, name, value, custom_msg=None):
if ann_efc is None:
ann_efc = min(100, max(k * 3, 50))
ann_efc = int(ann_efc)
- ann_loc = f"{reduction_loc}/ann__{ann_metric}__{ann_efc}__{ann_ef}__{ann_m}__{rand_state}"
+ ann_loc = (
+ f"{reduction_loc}/ann__{ann_metric}__{ann_efc}__{ann_ef}__"
+ f"{ann_m}__{rand_state}"
+ )
knn_loc = f"{ann_loc}/knn__{k}"
if n_centroids is None:
- if (
- reduction_loc in self.zw
- and "latest_kmeans" in self.zw[reduction_loc].attrs
- ):
- kmeans_loc = self.zw[reduction_loc].attrs["latest_kmeans"]
- n_centroids = int(
- kmeans_loc.split("/")[-1].split("__")[1]
- ) # depends on param_joiner
- log_message("default", "n_centroids", n_centroids)
+ if reduction_loc in self.zw:
+ reduction_grp = as_zarr_group(
+ self.zw[reduction_loc], name=reduction_loc
+ )
+ if "latest_kmeans" in reduction_grp.attrs:
+ kmeans_loc_attr = cast(str, reduction_grp.attrs["latest_kmeans"])
+ n_centroids = int(
+ kmeans_loc_attr.split("/")[-1].split("__")[1]
+ ) # depends on param_joiner
+ log_message("default", "n_centroids", n_centroids)
+ else:
+ n_centroids = default_values["n_centroids"]
+ log_message("default", "n_centroids", n_centroids)
else:
- # n_centroids = min(data.shape[0]/10, max(500, data.shape[0]/100))
n_centroids = default_values["n_centroids"]
log_message("default", "n_centroids", n_centroids)
n_centroids = int(n_centroids)
if local_connectivity is None or bandwidth is None:
- if knn_loc in self.zw and "latest_graph" in self.zw[knn_loc].attrs:
- graph_loc = self.zw[knn_loc].attrs["latest_graph"]
- c_local_connectivity, c_bandwidth = map(
- float, graph_loc.rsplit("/")[-1].split("__")[1:]
- )
+ if knn_loc in self.zw:
+ knn_grp = as_zarr_group(self.zw[knn_loc], name=knn_loc)
+ if "latest_graph" in knn_grp.attrs:
+ graph_loc_attr = cast(str, knn_grp.attrs["latest_graph"])
+ c_local_connectivity, c_bandwidth = map(
+ float, graph_loc_attr.rsplit("/")[-1].split("__")[1:]
+ )
+ else:
+ c_local_connectivity, c_bandwidth = None, None
else:
c_local_connectivity, c_bandwidth = None, None
if local_connectivity is None:
@@ -364,10 +908,10 @@ def log_message(category, name, value, custom_msg=None):
def _get_latest_keys(
self,
- from_assay: Optional[str],
- cell_key: Optional[str],
- feat_key: Optional[str],
- ) -> Tuple[str, str, str]:
+ from_assay: str | None,
+ cell_key: str | None,
+ feat_key: str | None,
+ ) -> tuple[str, str, str]:
if from_assay is None:
from_assay = self._defaultAssay
if cell_key is None:
@@ -391,12 +935,16 @@ def _get_latest_graph_loc(
Path of graph in the Zarr hierarchy
"""
normed_loc = f"{from_assay}/normed__{cell_key}__{feat_key}"
- reduction_loc = self.zw[normed_loc].attrs["latest_reduction"]
- ann_loc = self.zw[reduction_loc].attrs["latest_ann"]
- knn_loc = self.zw[ann_loc].attrs["latest_knn"]
- return self.zw[knn_loc].attrs["latest_graph"]
-
- def _get_latest_knn_loc(self, from_assay: str = None) -> str:
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ reduction_loc = cast(str, normed_grp.attrs["latest_reduction"])
+ reduction_grp = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ ann_loc = cast(str, reduction_grp.attrs["latest_ann"])
+ ann_grp = as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ knn_loc = cast(str, ann_grp.attrs["latest_knn"])
+ knn_grp = as_zarr_group(self.zw[knn_loc], name=knn_loc)
+ return cast(str, knn_grp.attrs["latest_graph"])
+
+ def _get_latest_knn_loc(self, from_assay: str | None = None) -> str:
"""Convenience function to identify location of the latest KNN graph in
the Zarr hierarchy.
@@ -413,17 +961,384 @@ def _get_latest_knn_loc(self, from_assay: str = None) -> str:
if from_assay not in self.assay_names:
raise ValueError(f"ERROR: Assay {from_assay} does not exist")
- latest_cell_key = self.zw[from_assay].attrs["latest_cell_key"]
- latest_feat_key = self.zw[from_assay].attrs["latest_feat_key"]
+ latest_cell_key = cast(
+ str,
+ as_zarr_group(self.zw[from_assay], name=from_assay).attrs[
+ "latest_cell_key"
+ ],
+ )
+ latest_feat_key = cast(
+ str,
+ as_zarr_group(self.zw[from_assay], name=from_assay).attrs[
+ "latest_feat_key"
+ ],
+ )
normed_loc = f"{from_assay}/normed__{latest_cell_key}__{latest_feat_key}"
- reduction_loc = self.zw[normed_loc].attrs["latest_reduction"]
- if "reduction" not in self.zw[reduction_loc]:
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ reduction_loc = cast(str, normed_grp.attrs["latest_reduction"])
+ reduction_grp = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ if "reduction" not in reduction_grp:
raise ValueError(f"ERROR: PCA Reduction not found in {reduction_loc}")
- latest_ann = self.zw[reduction_loc].attrs["latest_ann"]
- ann_loc = self.zw[latest_ann]
- latest_knn = ann_loc.attrs["latest_knn"]
+ latest_ann = cast(str, reduction_grp.attrs["latest_ann"])
+ ann_grp = as_zarr_group(self.zw[latest_ann], name=latest_ann)
+ latest_knn = cast(str, ann_grp.attrs["latest_knn"])
return latest_knn
+ def _ann_stream_recoverable(
+ self,
+ ann_loc: str,
+ reduction_loc: str,
+ normed_loc: str,
+ ) -> bool:
+ """Return True when an ANN stream can be loaded or rebuilt without a full make_graph."""
+ if ann_loc in self.zw and has_ann_index(
+ as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ ):
+ return True
+ legacy = legacy_ann_index_path(zarr_root_path(self.zw), ann_loc)
+ if legacy is not None and os.path.exists(legacy):
+ return True
+ if reduction_loc in self.zw:
+ reduction_grp = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ if "reduction" in reduction_grp:
+ if normed_loc in self.zw:
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ if "data" in normed_grp:
+ return True
+ return False
+
+ def _resolve_ann_index(
+ self,
+ ann_loc: str,
+ ann_metric: str,
+ dim: int,
+ ann_index_fetcher: Callable | None = None,
+ persist: bool = True,
+ ) -> Any:
+ """Load ANN index from zarr, legacy file, custom fetcher, or return None to rebuild."""
+ ann_group: zarr.Group | None = (
+ as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ if ann_loc in self.zw
+ else None
+ )
+
+ if ann_index_fetcher is not None:
+ try:
+ ann_index_fn = ann_index_fetcher(ann_loc)
+ except Exception:
+ ann_index_fn = None
+ logger.warning("Custom `ann_index_fetcher` failed")
+ if ann_index_fn is not None and os.path.exists(ann_index_fn):
+ return load_ann_index_from_path(ann_index_fn, ann_metric, dim)
+
+ if ann_group is not None and has_ann_index(ann_group):
+ return load_ann_index(ann_group, ann_metric, dim)
+
+ legacy = legacy_ann_index_path(zarr_root_path(self.zw), ann_loc)
+ if legacy is not None and os.path.exists(legacy):
+ idx = load_ann_index_from_path(legacy, ann_metric, dim)
+ if persist and self.zarr_mode == "r+" and ann_group is not None:
+ save_ann_index(ann_group, idx)
+ return idx
+
+ logger.info(
+ "ANN index not found in store; will rebuild from normalized data and loadings"
+ )
+ return None
+
+ def _persist_ann_index(
+ self,
+ ann_loc: str,
+ ann_idx: Any,
+ ann_index_saver: Callable | None = None,
+ ) -> None:
+ """Save an hnswlib index into the zarr hierarchy or via a custom saver."""
+ if ann_index_saver is not None:
+ try:
+ ann_index_saver(ann_idx, ann_loc)
+ return
+ except Exception:
+ logger.warning("Custom `ann_index_saver` failed")
+ if ann_loc not in self.zw:
+ self.zw.create_group(ann_loc, overwrite=True)
+ if self.zarr_mode != "r+":
+ logger.debug("Skipping ANN index persistence on read-only store")
+ return
+ save_ann_index(as_zarr_group(self.zw[ann_loc], name=ann_loc), ann_idx)
+
+ def _has_ann_stream_cache(
+ self,
+ from_assay: str,
+ cell_key: str,
+ feat_key: str,
+ knn_loc: str | None = None,
+ feat_scaling: bool | None = None,
+ ) -> bool:
+ try:
+ if knn_loc is None:
+ normed_loc = f"{from_assay}/normed__{cell_key}__{feat_key}"
+ if normed_loc not in self.zw:
+ return False
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ reduction_loc = cast(str, normed_grp.attrs["latest_reduction"])
+ reduction_grp = as_zarr_group(
+ self.zw[reduction_loc], name=reduction_loc
+ )
+ ann_loc = cast(str, reduction_grp.attrs["latest_ann"])
+ ann_grp = as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ knn_loc = cast(str, ann_grp.attrs["latest_knn"])
+ else:
+ ann_loc = knn_loc.rsplit("/", 1)[0]
+
+ if knn_loc not in self.zw:
+ return False
+ if ann_loc not in self.zw:
+ return False
+ if feat_scaling is not None:
+ ann_grp = as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ cached_scaling = bool(ann_grp.attrs.get("featureScaling", True))
+ if cached_scaling != feat_scaling:
+ return False
+ reduction_loc = ann_loc.rsplit("/ann__", 1)[0]
+ normed_loc = reduction_loc.rsplit("/reduction__", 1)[0]
+ return self._ann_stream_recoverable(ann_loc, reduction_loc, normed_loc)
+ except KeyError:
+ return False
+
+ def _load_or_compute_norm_stats(
+ self,
+ normed_loc: str,
+ data: ChunkedArray,
+ reduction_method: str,
+ ) -> tuple[np.ndarray, np.ndarray]:
+ mu, sigma = np.ndarray([]), np.ndarray([])
+ if reduction_method not in ["pca", "manual"]:
+ return mu, sigma
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ need_mu = "mu" not in normed_grp
+ need_sigma = "sigma" not in normed_grp
+ if not need_mu:
+ mu = np.asarray(as_zarr_array(normed_grp["mu"], name="mu")[:])
+ if not need_sigma:
+ sigma = np.asarray(as_zarr_array(normed_grp["sigma"], name="sigma")[:])
+ if need_mu and need_sigma:
+ mu_raw, sigma_raw = data.mean_and_std(
+ nthreads=self.nthreads,
+ msg="Calculating mean and std. dev. of norm. data",
+ )
+ mu = clean_array(mu_raw)
+ sigma = clean_array(sigma_raw, 1)
+ if self.zarr_mode == "r+":
+ g = create_zarr_dataset(normed_grp, "mu", (100000,), "f8", mu.shape)
+ g[:] = mu
+ g = create_zarr_dataset(
+ normed_grp, "sigma", (100000,), "f8", sigma.shape
+ )
+ g[:] = sigma
+ else:
+ logger.debug("Skipping mu/sigma persistence on read-only store")
+ elif need_mu:
+ mu = clean_array(
+ show_dask_progress(
+ data.mean(axis=0),
+ "Calculating mean of norm. data",
+ self.nthreads,
+ )
+ )
+ if self.zarr_mode == "r+":
+ g = create_zarr_dataset(normed_grp, "mu", (100000,), "f8", mu.shape)
+ g[:] = mu
+ else:
+ logger.debug("Skipping mu persistence on read-only store")
+ elif need_sigma:
+ sigma = clean_array(
+ show_dask_progress(
+ data.std(axis=0),
+ "Calculating std. dev. of norm. data",
+ self.nthreads,
+ ),
+ 1,
+ )
+ if self.zarr_mode == "r+":
+ g = create_zarr_dataset(
+ normed_grp, "sigma", (100000,), "f8", sigma.shape
+ )
+ g[:] = sigma
+ else:
+ logger.debug("Skipping sigma persistence on read-only store")
+ return mu, sigma
+
+ @staticmethod
+ def _normed_data_cached(
+ assay: Assay,
+ cell_key: str,
+ feat_key: str,
+ location: str,
+ log_transform: bool,
+ renormalize_subset: bool,
+ ) -> bool:
+ resolved_feat = feat_key if feat_key == "I" else f"{cell_key}__{feat_key}"
+ if location not in assay.z:
+ return False
+ try:
+ cell_idx, feat_idx = assay._get_cell_feat_idx(cell_key, resolved_feat)
+ subset_hash = assay._create_subset_hash(cell_idx, feat_idx)
+ subset_params = {
+ "log_transform": log_transform,
+ "renormalize_subset": renormalize_subset,
+ }
+ grp = assay.z[location]
+ return (
+ subset_hash == grp.attrs["subset_hash"]
+ and subset_params == grp.attrs["subset_params"]
+ )
+ except (KeyError, ValueError, AttributeError):
+ return False
+
+ @staticmethod
+ def _staged_normed_cached(
+ cache_path: str,
+ subset_hash: int,
+ subset_params: dict[str, Any],
+ ) -> bool:
+ if not os.path.isfile(os.path.join(cache_path, "zarr.json")):
+ return False
+ try:
+ root = zarr.open_group(cache_path, mode="r")
+ if "data" not in root:
+ return False
+ staged = root["data"]
+ return (
+ staged.attrs.get("staged_subset_hash") == subset_hash
+ and staged.attrs.get("staged_subset_params") == subset_params
+ and bool(staged.attrs.get("staged_complete"))
+ )
+ except Exception:
+ return False
+
+ def _load_ann_stream(
+ self,
+ from_assay: str,
+ cell_key: str,
+ feat_key: str,
+ feat_scaling: bool = True,
+ knn_loc: str | None = None,
+ ) -> AnnStream:
+ """Load an AnnStream from an existing graph without recomputing KNN."""
+
+ if knn_loc is None:
+ normed_loc = f"{from_assay}/normed__{cell_key}__{feat_key}"
+ if normed_loc not in self.zw:
+ raise KeyError(f"No normalized data at {normed_loc}")
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ reduction_loc = cast(str, normed_grp.attrs["latest_reduction"])
+ reduction_grp = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ ann_loc = cast(str, reduction_grp.attrs["latest_ann"])
+ ann_grp = as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ knn_loc = cast(str, ann_grp.attrs["latest_knn"])
+ else:
+ ann_loc = knn_loc.rsplit("/", 1)[0]
+ reduction_loc = ann_loc.rsplit("/ann__", 1)[0]
+ normed_loc = reduction_loc.rsplit("/reduction__", 1)[0]
+
+ if knn_loc not in self.zw:
+ raise KeyError(f"KNN graph not found at {knn_loc}")
+
+ ann_parts = ann_loc.rsplit("/", 1)[-1].split("__")
+ ann_metric = ann_parts[1]
+ ann_efc, ann_ef, ann_m, rand_state = map(int, ann_parts[2:6])
+ red_parts = reduction_loc.rsplit("/", 1)[-1].split("__")
+ reduction_method, dims, pca_cell_key = (
+ red_parts[1],
+ int(red_parts[2]),
+ red_parts[3],
+ )
+ k = int(knn_loc.rsplit("/", 1)[-1].split("__")[-1])
+
+ data = ChunkedArray(
+ as_zarr_array(
+ as_zarr_group(self.zw[normed_loc], name=normed_loc)["data"],
+ name="data",
+ ),
+ nthreads=self.nthreads,
+ )
+ mu, sigma = self._load_or_compute_norm_stats(normed_loc, data, reduction_method)
+
+ loadings: NDArray[Any] | None = None
+ reduction_grp = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ if "reduction" in reduction_grp:
+ loadings = np.asarray(
+ as_zarr_array(reduction_grp["reduction"], name="reduction")[:]
+ )
+
+ ann_grp = as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ cached_scaling = bool(ann_grp.attrs.get("featureScaling", True))
+ if cached_scaling != feat_scaling:
+ raise ValueError(
+ f"ANN index at {ann_loc} was built with featureScaling="
+ f"{cached_scaling}, not {feat_scaling}. Rebuild the graph."
+ )
+ harmonize = cast(bool, ann_grp.attrs.get("isHarmonized", False))
+ harmonized_data = None
+ batches = None
+ if harmonize and "harmonizedData" in reduction_grp:
+ harmonized_arr = as_zarr_array(
+ reduction_grp["harmonizedData"], name="harmonizedData"
+ )
+ harmonized_data = ChunkedArray(harmonized_arr, nthreads=self.nthreads)
+ batch_columns = cast(list[str] | None, harmonized_arr.attrs.get("batches"))
+ if batch_columns:
+ batches = pd.DataFrame(
+ {
+ x: self.cells.fetch(x, key=cell_key).astype(object)
+ for x in batch_columns
+ }
+ )
+
+ temp_dim = dims if dims > 0 else data.shape[1]
+ ann_idx = self._resolve_ann_index(
+ ann_loc,
+ ann_metric,
+ temp_dim,
+ ann_index_fetcher=None,
+ persist=(self.zarr_mode == "r+"),
+ )
+ rebuilt_ann = ann_idx is None
+
+ use_for_pca = self.cells.fetch(pca_cell_key, key=cell_key)
+ logger.info(f"Loaded existing ANN stream from {ann_loc}")
+ ann_obj = AnnStream(
+ data=data,
+ k=k,
+ n_cluster=2,
+ reduction_method=reduction_method,
+ dims=dims,
+ loadings=loadings,
+ use_for_pca=use_for_pca,
+ mu=mu,
+ sigma=sigma,
+ ann_metric=ann_metric,
+ ann_efc=ann_efc,
+ ann_ef=ann_ef,
+ ann_m=ann_m,
+ nthreads=self.nthreads,
+ ann_parallel=False,
+ rand_state=rand_state,
+ do_kmeans_fit=False,
+ disable_scaling=not feat_scaling,
+ ann_idx=ann_idx,
+ lsi_skip_first=True,
+ lsi_params={},
+ harmonize=harmonize,
+ harmonized_data=harmonized_data,
+ batches=batches,
+ cache_embeddings=False,
+ )
+ ann_obj.annPath = ann_loc
+ if rebuilt_ann and self.zarr_mode == "r+":
+ self._persist_ann_index(ann_loc, ann_obj.annIdx)
+ return ann_obj
+
def _get_ini_embed(
self, from_assay: str, cell_key: str, feat_key: str, n_comps: int
) -> np.ndarray:
@@ -446,17 +1361,24 @@ def _get_ini_embed(
from ..utils import rescale_array
normed_loc = f"{from_assay}/normed__{cell_key}__{feat_key}"
- reduction_loc = self.zw[normed_loc].attrs["latest_reduction"]
- kmeans_loc = self.zw[reduction_loc].attrs["latest_kmeans"]
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ reduction_loc = cast(str, normed_grp.attrs["latest_reduction"])
+ reduction_grp = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ kmeans_loc = cast(str, reduction_grp.attrs["latest_kmeans"])
+ kmeans_grp = as_zarr_group(self.zw[kmeans_loc], name=kmeans_loc)
pc = PCA(n_components=n_comps).fit_transform(
- self.zw[kmeans_loc]["cluster_centers"][:]
+ np.asarray(
+ as_zarr_array(kmeans_grp["cluster_centers"], name="cluster_centers")[:]
+ )
)
for i in range(n_comps):
pc[:, i] = rescale_array(pc[:, i])
- clusters = self.zw[kmeans_loc]["cluster_labels"][:].astype(np.uint32)
+ clusters = np.asarray(
+ as_zarr_array(kmeans_grp["cluster_labels"], name="cluster_labels")[:]
+ ).astype(np.uint32)
return np.array([pc[x] for x in clusters]).astype(np.float32, order="C")
- def _get_graph_ncells_k(self, graph_loc: str) -> Tuple[int, int]:
+ def _get_graph_ncells_k(self, graph_loc: str) -> tuple[int, int]:
"""
Args:
@@ -466,14 +1388,18 @@ def _get_graph_ncells_k(self, graph_loc: str) -> Tuple[int, int]:
"""
if graph_loc.startswith(self._integratedGraphsLoc):
- attrs = self.zw[graph_loc].attrs
- return attrs["n_cells"], attrs["n_neighbors"]
- knn_loc = self.zw[graph_loc.rsplit("/", 1)[0]]
- return knn_loc["indices"].shape
+ graph_grp = as_zarr_group(self.zw[graph_loc], name=graph_loc)
+ attrs = dict(graph_grp.attrs)
+ return cast(int, attrs["n_cells"]), cast(int, attrs["n_neighbors"])
+ knn_grp = as_zarr_group(
+ self.zw[graph_loc.rsplit("/", 1)[0]], name=graph_loc.rsplit("/", 1)[0]
+ )
+ indices = as_zarr_array(knn_grp["indices"], name="indices")
+ return indices.shape[0], indices.shape[1]
def _store_to_sparse(
- self, graph_loc: str, sparse_format: str = "csr", use_k: Optional[int] = None
- ) -> tuple:
+ self, graph_loc: str, sparse_format: str = "csr", use_k: int | None = None
+ ) -> tuple[int, csr_matrix | coo_matrix]:
"""
Args:
@@ -485,7 +1411,7 @@ def _store_to_sparse(
"""
logger.debug(f"Loading graph from location: {graph_loc}")
- store = self.zw[graph_loc]
+ store = as_zarr_group(self.zw[graph_loc], name=graph_loc)
n_cells, k = self._get_graph_ncells_k(graph_loc)
# TODO: can we have a progress bar for graph loading. Append to coo matrix?
if use_k is None:
@@ -498,7 +1424,8 @@ def _store_to_sparse(
indexer = np.tile([True] * use_k + [False] * (k - use_k), n_cells)
else:
indexer = None
- w, e = store["weights"][:], store["edges"][:]
+ w = np.asarray(as_zarr_array(store["weights"], name="weights")[:])
+ e = np.asarray(as_zarr_array(store["edges"], name="edges")[:])
if indexer is not None:
w, e = w[indexer], e[indexer]
if sparse_format == "csr":
@@ -510,38 +1437,99 @@ def _store_to_sparse(
(w, (e[:, 0], e[:, 1])), shape=(n_cells, n_cells)
)
+ @staticmethod
+ def _resolve_local_cache_plan(
+ zarr_loc: Any,
+ group: zarr.Group,
+ local_cache: bool | str,
+ ) -> tuple[bool, str | None, bool]:
+ """Return whether to stage, cache base directory, and remove-on-success flag."""
+ if local_cache is False or not is_remote_datastore(zarr_loc, group):
+ return False, None, False
+ if local_cache is True or local_cache == "auto":
+ return True, tempfile.mkdtemp(prefix="scarf_local_cache_"), True
+ if isinstance(local_cache, str):
+ os.makedirs(local_cache, exist_ok=True)
+ return True, local_cache, False
+ raise TypeError(
+ f"local_cache must be 'auto', True, False, or a path string, got {local_cache!r}"
+ )
+
+ def _normed_cache_key(self, subset_hash: int, subset_params: dict[str, Any]) -> str:
+ import hashlib
+ import json
+
+ payload = json.dumps(
+ {"subset_hash": subset_hash, "subset_params": subset_params},
+ sort_keys=True,
+ )
+ return hashlib.sha256(payload.encode()).hexdigest()[:32]
+
+ def _stage_normed_data(
+ self,
+ remote_array: zarr.Array,
+ subset_hash: int,
+ subset_params: dict[str, Any],
+ cache_base: str,
+ ) -> ChunkedArray:
+ """Copy normalized data to a local scratch Zarr for multi-pass graph building."""
+ cache_key = self._normed_cache_key(subset_hash, subset_params)
+ cache_path = os.path.join(cache_base, cache_key, "normed.zarr")
+ os.makedirs(os.path.dirname(cache_path), exist_ok=True)
+ staged = open_or_create_staged_normed_array(cache_path, remote_array)
+ if (
+ staged.attrs.get("staged_subset_hash") == subset_hash
+ and staged.attrs.get("staged_subset_params") == subset_params
+ and staged.attrs.get("staged_complete")
+ ):
+ logger.info(f"Reusing staged normalized data at {cache_path}")
+ else:
+ logger.info(f"Staging normalized data locally at {cache_path}")
+ copy_zarr_array(
+ remote_array,
+ staged,
+ msg="Staging normalized data locally",
+ )
+ staged.attrs["staged_subset_hash"] = subset_hash
+ staged.attrs["staged_subset_params"] = subset_params
+ staged.attrs["staged_complete"] = True
+ return ChunkedArray(staged, nthreads=self.nthreads)
+
def make_graph(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
- pca_cell_key: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ pca_cell_key: str | None = None,
reduction_method: str = "auto",
- dims: Optional[int] = None,
- k: Optional[int] = None,
- ann_metric: Optional[str] = None,
- ann_efc: Optional[int] = None,
- ann_ef: Optional[int] = None,
- ann_m: Optional[int] = None,
+ dims: int | None = None,
+ k: int | None = None,
+ ann_metric: str | None = None,
+ ann_efc: int | None = None,
+ ann_ef: int | None = None,
+ ann_m: int | None = None,
ann_parallel: bool = False,
- rand_state: Optional[int] = None,
- n_centroids: Optional[int] = None,
- batch_size: Optional[int] = None,
- log_transform: Optional[bool] = None,
- renormalize_subset: Optional[bool] = None,
- local_connectivity: Optional[float] = None,
- bandwidth: Optional[float] = None,
+ rand_state: int | None = None,
+ n_centroids: int | None = None,
+ batch_size: int | None = None,
+ log_transform: bool | None = None,
+ renormalize_subset: bool | None = None,
+ local_connectivity: float | None = None,
+ bandwidth: float | None = None,
update_keys: bool = True,
return_ann_object: bool = False,
- custom_loadings: Optional[np.ndarray] = None,
+ custom_loadings: np.ndarray | None = None,
feat_scaling: bool = True,
lsi_skip_first: bool = True,
harmonize: bool = False,
- batch_columns: Optional[List[str]] = None,
+ batch_columns: list[str] | None = None,
show_elbow_plot: bool = False,
- ann_index_fetcher: Optional[Callable] = None,
- ann_index_saver: Optional[Callable] = None,
- ):
+ ann_index_fetcher: Callable | None = None,
+ ann_index_saver: Callable | None = None,
+ local_cache: bool | str = "auto",
+ harmony_params: dict[str, Any] | None = None,
+ _force_harmony_refit: bool = False,
+ ) -> AnnStream | None:
"""Creates a cell neighbourhood graph. Performs following steps in the
process:
@@ -637,22 +1625,26 @@ def make_graph(
reduced dimensions. `dims` parameter is ignored when this is provided.
(Default value: None)
feat_scaling: If True (default) then the feature will be z-scaled otherwise not. It is highly recommended
- that this is kept as True unless you know what you are doing. `feat_scaling` is internally
- turned off when during cross sample mapping using CORAL normalized values are being used.
- Read more about this in `run_mapping` method.
+ that this is kept as True unless you know what you are doing.
lsi_skip_first: Whether to remove the first LSI dimension when using ATAC-Seq data.
- harmonize: Run Harmony to perform batch integration
- batch_columns: Columns in cell metadata table to use for batch integration
+ harmonize: If True, run Harmony batch correction on the PCA embedding before
+ building the KNN graph. Requires ``batch_columns``.
+ batch_columns: Cell metadata columns defining batch variables for Harmony.
+ harmony_params: Optional keyword arguments forwarded to ``fit_harmony``.
show_elbow_plot: If True, then an elbow plot is shown when PCA is fitted to the data. Not shown when using
existing PCA loadings or custom loadings. (Default value: False)
- ann_index_fetcher:
- ann_index_saver:
+ ann_index_fetcher: Optional callable to load a pre-built ANN index instead of fitting one.
+ ann_index_saver: Optional callable to persist a fitted ANN index for reuse.
+ local_cache: When ``'auto'`` or ``True``, remote stores copy the normalized
+ matrix to a local scratch Zarr before PCA/ANN/kmeans/KNN so
+ multi-pass reads hit local disk instead of object storage.
+ A string value is treated as a persistent scratch base path
+ keyed by ``subset_hash`` (~8 GB for 1M cells x 2000 HVGs in
+ float32). ``False`` disables staging.
Returns:
Either None or `AnnStream` object
"""
- from ..ann import AnnStream
-
if from_assay is None:
from_assay = self._defaultAssay
assay = self._get_assay(from_assay)
@@ -661,19 +1653,18 @@ def make_graph(
if cell_key is None:
cell_key = "I"
if feat_key is None:
- bool_cols = [
+ bool_col_parts = [
x.split("__", 1)
for x in assay.feats.columns
- if assay.feats.get_dtype(x) == bool and x != "I"
+ if assay.feats.get_dtype(x) == bool and x != "I" # noqa: E721
]
- bool_cols = [f"{x[1]}({x[0]})" for x in bool_cols]
- bool_cols = " ".join(map(str, bool_cols))
+ bool_cols_msg = " ".join(f"{part[1]}({part[0]})" for part in bool_col_parts)
raise ValueError(
"ERROR: You have to choose which features that should be used for graph construction. "
"Ideally you should have performed a feature selection step before making this graph. "
"Feature selection step adds a column to your feature table. \n"
"You have following boolean columns in the feature "
- f"metadata of assay {from_assay} which you can choose from: {bool_cols}\n The values in "
+ f"metadata of assay {from_assay} which you can choose from: {bool_cols_msg}\n The values in "
f"brackets indicate the cell_key for which the feat_key is available. Choosing 'I' "
f"as `feat_key` means that you will use all the genes for graph creation."
)
@@ -722,11 +1713,11 @@ def make_graph(
batches = None
if harmonize:
if batch_columns is None:
- raise ValueError(f"Harmonization requested but no batches provided")
+ raise ValueError("Harmonization requested but no batches provided")
else:
if isinstance(batch_columns, list) is False:
raise ValueError(
- f"batches must be a list of columns in cell metadata that represent batches"
+ "batches must be a list of columns in cell metadata that represent batches"
)
batches = pd.DataFrame(
{
@@ -739,78 +1730,264 @@ def make_graph(
reduction_loc = (
f"{normed_loc}/reduction__{reduction_method}__{dims}__{pca_cell_key}"
)
- ann_loc = f"{reduction_loc}/ann__{ann_metric}__{ann_efc}__{ann_ef}__{ann_m}__{rand_state}"
+ ann_loc = (
+ f"{reduction_loc}/ann__{ann_metric}__{ann_efc}__{ann_ef}__"
+ f"{ann_m}__{rand_state}"
+ )
+ if not feat_scaling:
+ ann_loc = f"{ann_loc}__unscaled"
+ if harmonize:
+ if batches is None:
+ raise ValueError("Harmony requires batch metadata")
+ from ..mapping_utils import array_hash
+
+ harmony_contract = {
+ "version": _HARMONY_ANN_CONTRACT_VERSION,
+ "batchColumns": batch_columns,
+ "batchValueHash": array_hash(
+ batches.astype(str).to_numpy().reshape(-1)
+ ),
+ "parameters": harmony_params or {},
+ }
+ contract_hash = hashlib.sha256(
+ json.dumps(
+ harmony_contract,
+ sort_keys=True,
+ separators=(",", ":"),
+ default=str,
+ ).encode()
+ ).hexdigest()[:16]
+ ann_loc = f"{ann_loc}__harmony_{contract_hash}"
knn_loc = f"{ann_loc}/knn__{k}"
kmeans_loc = f"{reduction_loc}/kmeans__{n_centroids}__{rand_state}"
graph_loc = f"{knn_loc}/graph__{local_connectivity}__{bandwidth}"
- data = assay.save_normalized_data(
+ normed_short = normed_loc.split("/")[-1]
+ cached_normed = self._normed_data_cached(
+ assay,
cell_key,
feat_key,
- batch_size,
- normed_loc.split("/")[-1],
+ normed_short,
log_transform,
renormalize_subset,
- update_keys,
)
+ cache_enabled, cache_base, remove_on_success = self._resolve_local_cache_plan(
+ self.zarr_loc, self.z, local_cache
+ )
+ planned = 7
+ if cache_enabled:
+ planned += 1
+ progress = _GraphBuildProgress(planned)
+
+ # When caching locally and the normalized matrix must be recomputed,
+ # mirror each normalized band straight into the local staging cache
+ # during the write pass so it never has to be downloaded back from the
+ # remote store afterwards.
+ staged_mirror = None
+ if cache_enabled and not cached_normed and cache_base is not None:
+ resolved_feat = feat_key if feat_key == "I" else f"{cell_key}__{feat_key}"
+ pre_cell_idx, pre_feat_idx = assay._get_cell_feat_idx(
+ cell_key, resolved_feat
+ )
+ pre_hash = assay._create_subset_hash(pre_cell_idx, pre_feat_idx)
+ pre_params = {
+ "log_transform": log_transform,
+ "renormalize_subset": renormalize_subset,
+ }
+ pre_cache_key = self._normed_cache_key(pre_hash, pre_params)
+ pre_cache_path = os.path.join(cache_base, pre_cache_key, "normed.zarr")
+ os.makedirs(os.path.dirname(pre_cache_path), exist_ok=True)
+ staged_mirror = create_or_open_staged_normed_array(
+ pre_cache_path, (len(pre_cell_idx), len(pre_feat_idx))
+ )
+
+ with progress.step("normalize expression matrix", cached=cached_normed):
+ data = assay.save_normalized_data(
+ cell_key,
+ feat_key,
+ batch_size,
+ normed_short,
+ log_transform,
+ renormalize_subset,
+ update_keys,
+ mirror=staged_mirror,
+ )
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ subset_hash = normed_grp.attrs.get("subset_hash")
+ subset_params = normed_grp.attrs.get("subset_params")
+ if subset_hash is None or subset_params is None:
+ raise RuntimeError(
+ f"Normalized matrix at {normed_loc} is missing subset metadata; "
+ "delete the partial group and retry make_graph"
+ )
+ subset_hash = cast(int, subset_hash)
+ subset_params = cast(dict[str, Any], subset_params)
+ graph_succeeded = False
+ try:
+ if cache_enabled:
+ if cache_base is None:
+ raise RuntimeError(
+ "cache_base must be set when cache_enabled is True"
+ )
+ cache_key = self._normed_cache_key(subset_hash, subset_params)
+ cache_path = os.path.join(cache_base, cache_key, "normed.zarr")
+ staged_cached = self._staged_normed_cached(
+ cache_path, subset_hash, subset_params
+ )
+ with progress.step(
+ "stage normalized matrix locally", cached=staged_cached
+ ):
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ data = self._stage_normed_data(
+ as_zarr_array(normed_grp["data"], name="data"),
+ subset_hash,
+ subset_params,
+ cache_base,
+ )
+ result = self._run_graph_from_normed_data(
+ data=data,
+ assay=assay,
+ from_assay=from_assay,
+ cell_key=cell_key,
+ feat_key=feat_key,
+ normed_loc=normed_loc,
+ reduction_loc=reduction_loc,
+ ann_loc=ann_loc,
+ knn_loc=knn_loc,
+ kmeans_loc=kmeans_loc,
+ graph_loc=graph_loc,
+ batch_size=batch_size,
+ custom_loadings=custom_loadings,
+ reduction_method=reduction_method,
+ dims=dims,
+ pca_cell_key=pca_cell_key,
+ harmonize=harmonize,
+ batch_columns=batch_columns,
+ batches=batches,
+ harmony_params=harmony_params,
+ ann_metric=ann_metric,
+ ann_efc=ann_efc,
+ ann_ef=ann_ef,
+ ann_m=ann_m,
+ ann_parallel=ann_parallel,
+ rand_state=rand_state,
+ k=k,
+ n_centroids=n_centroids,
+ local_connectivity=local_connectivity,
+ bandwidth=bandwidth,
+ feat_scaling=feat_scaling,
+ lsi_skip_first=lsi_skip_first,
+ ann_index_fetcher=ann_index_fetcher,
+ ann_index_saver=ann_index_saver,
+ return_ann_object=return_ann_object,
+ show_elbow_plot=show_elbow_plot,
+ progress=progress,
+ force_harmony_refit=_force_harmony_refit,
+ )
+ graph_succeeded = True
+ return result
+ finally:
+ progress.finish()
+ if cache_enabled and cache_base is not None:
+ if remove_on_success:
+ if graph_succeeded and not return_ann_object:
+ shutil.rmtree(cache_base, ignore_errors=True)
+ elif not graph_succeeded:
+ logger.warning(
+ f"Graph build failed; local cache scratch retained at {cache_base}"
+ )
+ elif not graph_succeeded:
+ logger.warning(
+ f"Graph build failed; local cache retained at {cache_base}"
+ )
+
+ def _run_graph_from_normed_data(
+ self,
+ *,
+ data: ChunkedArray,
+ assay: Assay,
+ from_assay: str,
+ cell_key: str,
+ feat_key: str,
+ normed_loc: str,
+ reduction_loc: str,
+ ann_loc: str,
+ knn_loc: str,
+ kmeans_loc: str,
+ graph_loc: str,
+ batch_size: int,
+ custom_loadings: NDArray[Any] | None,
+ reduction_method: str,
+ dims: int,
+ pca_cell_key: str,
+ harmonize: bool,
+ batch_columns: list[str] | None,
+ batches: pd.DataFrame | None,
+ harmony_params: dict[str, Any] | None,
+ ann_metric: str,
+ ann_efc: int,
+ ann_ef: int,
+ ann_m: int,
+ ann_parallel: bool,
+ rand_state: int,
+ k: int,
+ n_centroids: int,
+ local_connectivity: float,
+ bandwidth: float,
+ feat_scaling: bool,
+ lsi_skip_first: bool,
+ ann_index_fetcher: Callable | None,
+ ann_index_saver: Callable | None,
+ return_ann_object: bool,
+ show_elbow_plot: bool,
+ progress: _GraphBuildProgress,
+ force_harmony_refit: bool,
+ ) -> AnnStream | None:
+
if custom_loadings is not None and data.shape[1] != custom_loadings.shape[0]:
raise ValueError(
f"Provided custom loadings has {custom_loadings.shape[0]} features while the data "
f"has {data.shape[1]} features."
)
- loadings = None
+ loadings: NDArray[Any] | None = None
fit_kmeans = True
- mu, sigma = np.ndarray([]), np.ndarray([])
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ cached_stats = (
+ reduction_method in ("pca", "manual")
+ and "mu" in normed_grp
+ and "sigma" in normed_grp
+ )
+ with progress.step("normalization statistics", cached=cached_stats):
+ mu, sigma = self._load_or_compute_norm_stats(
+ normed_loc, data, reduction_method
+ )
use_for_pca = self.cells.fetch(pca_cell_key, key=cell_key)
harmonized_data = None
-
- if reduction_method in ["pca", "manual"]:
- if "mu" in self.zw[normed_loc]:
- mu = self.zw[normed_loc]["mu"][:]
- else:
- mu = clean_array(
- show_dask_progress(
- data.mean(axis=0),
- "Calculating mean of norm. data",
- self.nthreads,
- )
- )
- g = create_zarr_dataset(
- self.zw[normed_loc], "mu", (100000,), "f8", mu.shape
- )
- g[:] = mu
- if "sigma" in self.zw[normed_loc]:
- sigma = self.zw[normed_loc]["sigma"][:]
- else:
- sigma = clean_array(
- show_dask_progress(
- data.std(axis=0),
- "Calculating std. dev. of norm. data",
- self.nthreads,
- ),
- 1,
- )
- g = create_zarr_dataset(
- self.zw[normed_loc], "sigma", (100000,), "f8", sigma.shape
- )
- g[:] = sigma
if reduction_loc in self.zw:
- if "reduction" in self.zw[reduction_loc]:
- loadings = self.zw[reduction_loc]["reduction"][:]
+ reduction_grp = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ if "reduction" in reduction_grp:
+ loadings = np.asarray(
+ as_zarr_array(reduction_grp["reduction"], name="reduction")[:]
+ )
if data.shape[1] != loadings.shape[0]:
logger.warning(
"Consistency breached in loading pre-cached loadings. Will perform fresh reduction."
)
loadings = None
del self.zw[reduction_loc]
- if harmonize and "harmonizedData" in self.zw[reduction_loc]:
- if "batches" in self.zw[reduction_loc]["harmonizedData"].attrs:
- if (
- self.zw[reduction_loc]["harmonizedData"].attrs["batches"]
- == batch_columns
- ):
- harmonized_data = from_zarr(
- self.zw[reduction_loc]["harmonizedData"], inline_array=True
+ if (
+ harmonize
+ and not force_harmony_refit
+ and "harmonizedData" in reduction_grp
+ ):
+ harmonized_arr = as_zarr_array(
+ reduction_grp["harmonizedData"], name="harmonizedData"
+ )
+ if "batches" in harmonized_arr.attrs:
+ if harmonized_arr.attrs["batches"] == batch_columns:
+ harmonized_data = ChunkedArray(
+ harmonized_arr,
+ nthreads=self.nthreads,
)
if custom_loadings is None:
@@ -830,14 +2007,21 @@ def make_graph(
if reduction_loc in self.zw:
del self.zw[reduction_loc]
if harmonized_data is not None:
- # Moved this statement here to print after the status of dim loadings
logger.info(f"Using existing harmonized data with {dims} dims")
ann_idx = None
+ had_cached_ann_idx = False
+ replace_ann_after_fit = False
if ann_loc in self.zw:
+ ann_grp = as_zarr_group(self.zw[ann_loc], name=ann_loc)
reset_ann = False
- if "isHarmonized" in self.zw[ann_loc].attrs:
- if self.zw[ann_loc].attrs["isHarmonized"]:
+ cached_scaling = ann_grp.attrs.get("featureScaling")
+ if cached_scaling is not None and bool(cached_scaling) != feat_scaling:
+ reset_ann = True
+ elif cached_scaling is None and feat_scaling is False:
+ reset_ann = True
+ if "isHarmonized" in ann_grp.attrs:
+ if cast(bool, ann_grp.attrs["isHarmonized"]):
if harmonize is False:
reset_ann = True
if harmonized_data is None:
@@ -848,164 +2032,248 @@ def make_graph(
else:
if harmonize: # Mostly for backward compatibility
reset_ann = True
+ if force_harmony_refit:
+ reset_ann = True
if reset_ann:
- del self.zw[ann_loc]
+ replace_ann_after_fit = True
else:
- if ann_index_fetcher is None:
- if hasattr(self.zw.chunk_store, "path"):
- ann_index_fn = os.path.join(
- self.zw.chunk_store.path, ann_loc, "ann_idx"
- )
- else:
- ann_index_fn = None
- logger.warning(
- f"No custom `ann_index_fetcher` provided and zarr path is not local"
- )
- else:
- # noinspection PyBroadException
- try:
- ann_index_fn = ann_index_fetcher(ann_loc)
- except:
- ann_index_fn = None
- logger.warning(f"Custom `ann_index_fetcher` failed")
- if ann_index_fn is None or os.path.exists(ann_index_fn) is False:
- logger.warning(f"Ann index file expected but could not be found")
- else:
- import hnswlib
-
- temp = dims if dims > 0 else data.shape[1]
- ann_idx = hnswlib.Index(space=ann_metric, dim=temp)
- ann_idx.load_index(ann_index_fn)
- # TODO: check if ANN is index is trained with expected number of cells.
- logger.info(f"Using existing ANN index")
+ temp = dims if dims > 0 else data.shape[1]
+ ann_idx = self._resolve_ann_index(
+ ann_loc,
+ ann_metric,
+ temp,
+ ann_index_fetcher=ann_index_fetcher,
+ persist=(self.zarr_mode == "r+"),
+ )
+ if ann_idx is not None:
+ had_cached_ann_idx = True
+ logger.info("Using existing ANN index")
if kmeans_loc in self.zw:
fit_kmeans = False
- logger.info(f"using existing kmeans cluster centers")
+ logger.info("using existing kmeans cluster centers")
disable_scaling = True if feat_scaling is False else False
- # TODO: expose LSImodel parameters
- ann_obj = AnnStream(
- data=data,
- k=k,
- n_cluster=n_centroids,
- reduction_method=reduction_method,
- dims=dims,
- loadings=loadings,
- use_for_pca=use_for_pca,
- mu=mu,
- sigma=sigma,
- ann_metric=ann_metric,
- ann_efc=ann_efc,
- ann_ef=ann_ef,
- ann_m=ann_m,
- nthreads=self.nthreads,
- ann_parallel=ann_parallel,
- rand_state=rand_state,
- do_kmeans_fit=fit_kmeans,
- disable_scaling=disable_scaling,
- ann_idx=ann_idx,
- lsi_skip_first=lsi_skip_first,
- lsi_params={},
- harmonize=harmonize,
- harmonized_data=harmonized_data,
- batches=batches,
+ cached_ann_stream = (
+ loadings is not None and had_cached_ann_idx and not fit_kmeans
+ )
+ need_embeddings = (
+ ann_idx is None
+ or fit_kmeans
+ or knn_loc not in self.zw
+ or graph_loc not in self.zw
)
- if reduction_loc not in self.zw:
- logger.debug(f"Saving loadings to {reduction_loc}")
- self.zw.create_group(reduction_loc, overwrite=True)
- if ann_obj.loadings is not None:
- # can be None when no dimred is performed
- g = create_zarr_dataset(
- self.zw[reduction_loc],
- "reduction",
- data.chunksize,
- "f8",
- ann_obj.loadings.shape,
- )
- g[:, :] = ann_obj.loadings
- if harmonize and harmonized_data is None and ann_obj.harmonizedData is not None:
- g = create_zarr_dataset(
- self.zw[reduction_loc],
- "harmonizedData",
- data.chunksize,
- "f8",
- ann_obj.harmonizedData.shape,
+ # TODO: expose LSImodel parameters
+ with progress.step(
+ "dimension reduction, ANN index, and kmeans",
+ cached=cached_ann_stream,
+ ):
+ ann_obj = AnnStream(
+ data=data,
+ k=k,
+ n_cluster=n_centroids,
+ reduction_method=reduction_method,
+ dims=dims,
+ loadings=loadings,
+ use_for_pca=use_for_pca,
+ mu=mu,
+ sigma=sigma,
+ ann_metric=ann_metric,
+ ann_efc=ann_efc,
+ ann_ef=ann_ef,
+ ann_m=ann_m,
+ nthreads=self.nthreads,
+ ann_parallel=ann_parallel,
+ rand_state=rand_state,
+ do_kmeans_fit=fit_kmeans,
+ disable_scaling=disable_scaling,
+ ann_idx=ann_idx,
+ lsi_skip_first=lsi_skip_first,
+ lsi_params={},
+ harmonize=harmonize,
+ harmonized_data=harmonized_data,
+ batches=batches,
+ harmony_params=harmony_params,
+ cache_embeddings=need_embeddings,
)
- g[:] = ann_obj.harmonizedData
- g.attrs["batches"] = batch_columns
- if ann_loc not in self.zw:
- logger.debug(f"Saving ANN index to {ann_loc}")
- self.zw.create_group(ann_loc, overwrite=True)
- if ann_idx is None:
- if ann_index_saver is None:
- if hasattr(self.zw.chunk_store, "path"):
- ann_obj.annIdx.save_index(
- os.path.join(self.zw.chunk_store.path, ann_loc, "ann_idx")
+ ann_obj.annPath = ann_loc
+ if (
+ harmonize
+ and reduction_method == "pca"
+ and ann_obj.featureScaling
+ and ann_obj.harmonyResult is not None
+ ):
+ from ..symphony import weighted_centroids
+
+ try:
+ weighted_centroids(
+ ann_obj.harmonyResult.original.T,
+ ann_obj.harmonyResult.assignments,
+ )
+ weighted_centroids(
+ ann_obj.harmonyResult.corrected.T,
+ ann_obj.harmonyResult.assignments,
+ )
+ except ValueError as exc:
+ if "empty cluster" not in str(exc):
+ raise
+ raise ValueError(
+ "Harmony produced an empty reference cluster. Rebuild with a "
+ "smaller harmony_params['nclust'] value."
+ ) from exc
+
+ if replace_ann_after_fit and ann_loc in self.zw:
+ del self.zw[ann_loc]
+ save_loadings = reduction_loc not in self.zw
+ save_harmonized = (
+ harmonize and harmonized_data is None and ann_obj.harmonizedData is not None
+ )
+ save_ann = ann_idx is None
+ save_kmeans = fit_kmeans
+ cached_persist = not (
+ save_loadings or save_harmonized or save_ann or save_kmeans
+ )
+ with progress.step("persist graph artifacts to Zarr", cached=cached_persist):
+ if save_loadings:
+ logger.debug(f"Saving loadings to {reduction_loc}")
+ self.zw.create_group(reduction_loc, overwrite=True)
+ reduction_grp = as_zarr_group(
+ self.zw[reduction_loc], name=reduction_loc
+ )
+ if ann_obj.loadings is not None:
+ g = create_zarr_dataset(
+ reduction_grp,
+ "reduction",
+ data.chunksize,
+ "f8",
+ ann_obj.loadings.shape,
)
- else:
- logger.warning(
- "No custom `ann_index_saver` provided and local path is unknown"
+ g[:, :] = ann_obj.loadings
+ if save_harmonized:
+ reduction_grp = as_zarr_group(
+ self.zw[reduction_loc], name=reduction_loc
+ )
+ if ann_obj.harmonizedData is not None:
+ harmonized = ann_obj.harmonizedData
+ g = create_zarr_dataset(
+ reduction_grp,
+ "harmonizedData",
+ harmonized.chunksize,
+ "f8",
+ harmonized.shape,
)
- if ann_index_saver is not None:
- try:
- ann_index_saver(ann_obj.annIdx, ann_loc)
- except:
- logger.warning("Custom `ann_index_saver` failed")
-
- if fit_kmeans:
- logger.debug(f"Saving kmeans clusters to {kmeans_loc}")
- self.zw.create_group(kmeans_loc, overwrite=True)
- g = create_zarr_dataset(
- self.zw[kmeans_loc],
- "cluster_centers",
- (1000, 1000),
- "f8",
- ann_obj.kmeans.cluster_centers_.shape,
- )
- g[:, :] = ann_obj.kmeans.cluster_centers_
- g = create_zarr_dataset(
- self.zw[kmeans_loc],
- "cluster_labels",
- (100000,),
- "f8",
- ann_obj.clusterLabels.shape,
- )
- g[:] = ann_obj.clusterLabels
- if knn_loc in self.zw and graph_loc in self.zw:
- logger.info(f"KNN graph already exists will not recompute.")
+ start = 0
+ for block in harmonized.blocks:
+ values = np.asarray(block.compute())
+ stop = start + values.shape[0]
+ g[start:stop, :] = values
+ start = stop
+ g.attrs["batches"] = batch_columns
+ if save_ann:
+ if ann_loc not in self.zw:
+ logger.debug(f"Saving ANN index to {ann_loc}")
+ self.zw.create_group(ann_loc, overwrite=True)
+ self._persist_ann_index(
+ ann_loc,
+ ann_obj.annIdx,
+ ann_index_saver=ann_index_saver,
+ )
+ if save_kmeans:
+ if ann_obj.kmeans is None:
+ raise RuntimeError("kmeans model missing despite fit_kmeans=True")
+ logger.debug(f"Saving kmeans clusters to {kmeans_loc}")
+ self.zw.create_group(kmeans_loc, overwrite=True)
+ kmeans_grp = as_zarr_group(self.zw[kmeans_loc], name=kmeans_loc)
+ g = create_zarr_dataset(
+ kmeans_grp,
+ "cluster_centers",
+ (1000, 1000),
+ "f8",
+ ann_obj.kmeans.cluster_centers_.shape,
+ )
+ g[:, :] = ann_obj.kmeans.cluster_centers_
+ g = create_zarr_dataset(
+ kmeans_grp,
+ "cluster_labels",
+ (100000,),
+ "f8",
+ ann_obj.clusterLabels.shape,
+ )
+ g[:] = ann_obj.clusterLabels
+
+ cached_knn = knn_loc in self.zw
+ cached_graph = knn_loc in self.zw and graph_loc in self.zw
+ if cached_knn and cached_graph:
+ with progress.step("KNN neighbor search", cached=True):
+ pass
+ with progress.step("smooth KNN distances into graph", cached=True):
+ logger.info("KNN graph already exists will not recompute.")
else:
from ..knn_utils import self_query_knn, smoothen_dists
- recall = None
+ recall: str | None = None
if knn_loc not in self.zw:
- recall = self_query_knn(
- ann_obj=ann_obj,
- store=self.zw.create_group(knn_loc, overwrite=True),
- chunk_size=batch_size,
- nthreads=self.nthreads,
+ with progress.step("KNN neighbor search", cached=False):
+ recall_val = self_query_knn(
+ ann_obj=ann_obj,
+ store=self.zw.create_group(knn_loc, overwrite=True),
+ chunk_size=batch_size,
+ nthreads=self.nthreads,
+ )
+ recall = "%.2f" % recall_val
+ else:
+ with progress.step("KNN neighbor search", cached=True):
+ pass
+
+ knn_grp = as_zarr_group(self.zw[knn_loc], name=knn_loc)
+ with progress.step("smooth KNN distances into graph", cached=cached_graph):
+ smoothen_dists(
+ self.zw.create_group(graph_loc, overwrite=True),
+ as_zarr_array(knn_grp["indices"], name="indices"),
+ as_zarr_array(knn_grp["distances"], name="distances"),
+ local_connectivity,
+ bandwidth,
+ batch_size,
)
- recall = "%.2f" % recall
-
- smoothen_dists(
- self.zw.create_group(graph_loc, overwrite=True),
- self.zw[knn_loc]["indices"],
- self.zw[knn_loc]["distances"],
- local_connectivity,
- bandwidth,
- batch_size,
- )
if recall is not None:
logger.info(f"ANN recall: {recall}%")
- self.zw[normed_loc].attrs["latest_reduction"] = reduction_loc
- self.zw[reduction_loc].attrs["latest_ann"] = ann_loc
- self.zw[reduction_loc].attrs["latest_kmeans"] = kmeans_loc
- self.zw[ann_loc].attrs["isHarmonized"] = harmonize
- self.zw[ann_loc].attrs["latest_knn"] = knn_loc
- self.zw[knn_loc].attrs["latest_graph"] = graph_loc
+ with progress.step("finalize graph metadata", cached=False):
+ normed_grp = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ reduction_grp = as_zarr_group(self.zw[reduction_loc], name=reduction_loc)
+ ann_grp = as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ knn_grp = as_zarr_group(self.zw[knn_loc], name=knn_loc)
+ normed_grp.attrs["latest_reduction"] = reduction_loc
+ reduction_grp.attrs["latest_ann"] = ann_loc
+ reduction_grp.attrs["latest_kmeans"] = kmeans_loc
+ ann_grp.attrs["isHarmonized"] = harmonize
+ ann_grp.attrs["featureScaling"] = feat_scaling
+ ann_grp.attrs["latest_knn"] = knn_loc
+ knn_grp.attrs["latest_graph"] = graph_loc
+ if (
+ harmonize
+ and reduction_method == "pca"
+ and ann_obj.harmonyResult is not None
+ ):
+ if ann_obj.featureScaling:
+ self._persist_mapping_reference(
+ assay,
+ from_assay,
+ cell_key,
+ feat_key,
+ reduction_loc,
+ ann_loc,
+ knn_loc,
+ batch_columns or [],
+ ann_obj,
+ )
+ else:
+ logger.warning(
+ "Skipping mapping-reference persistence because "
+ "feat_scaling=False is incompatible with query projection."
+ )
if return_ann_object:
return ann_obj
if show_elbow_plot:
@@ -1021,13 +2289,13 @@ def make_graph(
def load_graph(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
- symmetric: Optional[bool] = None,
- upper_only: Optional[bool] = None,
- use_k: Optional[int] = None,
- graph_loc: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ symmetric: bool | None = None,
+ upper_only: bool | None = None,
+ use_k: int | None = None,
+ graph_loc: str | None = None,
) -> csr_matrix:
"""Load the cell neighbourhood as a scipy sparse matrix.
@@ -1049,7 +2317,7 @@ def load_graph(
A scipy sparse matrix representing cell neighbourhood graph.
"""
- def symmetrize(g):
+ def symmetrize(g: csr_matrix) -> csr_matrix:
t = g + g.T
t = t - g.multiply(g.T)
return t
@@ -1087,12 +2355,12 @@ def symmetrize(g):
def run_tsne(
self,
- from_assay: str = None,
- cell_key: str = None,
- feat_key: str = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
symmetric_graph: bool = False,
graph_upper_only: bool = False,
- ini_embed: np.ndarray = None,
+ ini_embed: np.ndarray | None = None,
tsne_dims: int = 2,
lambda_scale: float = 1.0,
max_iter: int = 500,
@@ -1103,7 +2371,7 @@ def run_tsne(
label: str = "tSNE",
verbose: bool = True,
parallel: bool = False,
- nthreads: int = None,
+ nthreads: int | None = None,
) -> None:
"""Run SGtSNE-pi (Read more here:
https://github.com/fcdimitr/sgtsnepi/tree/v1.0.1). This is an
@@ -1141,9 +2409,7 @@ def run_tsne(
Returns:
"""
- from uuid import uuid4
- from ..knn_utils import export_knn_to_mtx
- from pathlib import Path
+ from ..knn_utils import run_sgtsne
import sys
if sys.platform not in ["posix", "linux"]:
@@ -1154,8 +2420,6 @@ def run_tsne(
from_assay, cell_key, feat_key
)
- uid = str(uuid4())
- knn_mtx_fn = Path(temp_file_loc, f"{uid}.mtx").resolve()
graph = self.load_graph(
from_assay=from_assay,
cell_key=cell_key,
@@ -1163,41 +2427,36 @@ def run_tsne(
symmetric=symmetric_graph,
upper_only=graph_upper_only,
)
- export_knn_to_mtx(str(knn_mtx_fn), graph)
-
- ini_emb_fn = Path(temp_file_loc, f"{uid}.txt").resolve()
- with open(ini_emb_fn, "w") as h:
- if ini_embed is None:
- ini_embed = self._get_ini_embed(
- from_assay, cell_key, feat_key, tsne_dims
- ).flatten()
- else:
- if ini_embed.shape != (graph.shape[0], tsne_dims):
- raise ValueError(
- "ERROR: Provided initial embedding does not shape required shape: "
- f"{(graph.shape[0], tsne_dims)}"
- )
- h.write("\n".join(map(str, ini_embed)))
- out_fn = Path(temp_file_loc, f"{uid}_output.txt").resolve()
+ if ini_embed is None:
+ ini_embed = self._get_ini_embed(from_assay, cell_key, feat_key, tsne_dims)
+ else:
+ if ini_embed.shape != (graph.shape[0], tsne_dims):
+ raise ValueError(
+ "ERROR: Provided initial embedding does not shape required shape: "
+ f"{(graph.shape[0], tsne_dims)}"
+ )
if parallel:
if nthreads is None:
nthreads = self.nthreads
else:
- assert type(nthreads) == int
+ assert isinstance(nthreads, int)
else:
nthreads = 1
- cmd = (
- f"sgtsne -m {max_iter} -l {lambda_scale} -d {tsne_dims} -e {early_iter} -p {nthreads} -a {alpha}"
- f" -h {box_h} -i {ini_emb_fn} -o {out_fn} {knn_mtx_fn}"
- )
- if verbose:
- system_call(cmd)
- else:
- os.system(cmd)
try:
- emb = pd.read_csv(out_fn, header=None, sep=" ")[
- list(range(tsne_dims))
- ].values.T
+ emb = run_sgtsne(
+ graph,
+ ini_embed,
+ tsne_dims=tsne_dims,
+ max_iter=max_iter,
+ early_iter=early_iter,
+ alpha=alpha,
+ lambda_scale=lambda_scale,
+ box_h=box_h,
+ temp_file_loc=temp_file_loc,
+ verbose=verbose,
+ parallel=parallel,
+ nthreads=nthreads,
+ )
for i in range(tsne_dims):
self.cells.insert(
self._col_renamer(from_assay, cell_key, f"{label}{i + 1}"),
@@ -1205,24 +2464,20 @@ def run_tsne(
key=cell_key,
overwrite=True,
)
- for fn in [out_fn, knn_mtx_fn, ini_emb_fn]:
- Path.unlink(fn)
- except FileNotFoundError:
+ except (FileNotFoundError, ImportError) as exc:
logger.error(
- "SG-tSNE failed, possibly due to missing libraries or file permissions. SG-tSNE currently "
- "fails on readthedocs"
+ "SG-tSNE failed, possibly due to missing sgtsne executable or "
+ f"sgtsnepi package: {exc}"
)
- for fn in [knn_mtx_fn, ini_emb_fn]:
- Path.unlink(fn)
def run_umap(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
- symmetric_graph: Optional[bool] = False,
- graph_upper_only: Optional[bool] = False,
- ini_embed: Optional[np.ndarray] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ symmetric_graph: bool | None = False,
+ graph_upper_only: bool | None = False,
+ ini_embed: np.ndarray | None = None,
umap_dims: int = 2,
spread: float = 2.0,
min_dist: float = 1,
@@ -1236,9 +2491,9 @@ def run_umap(
dens_var_shift: float = 0.1,
random_seed: int = 4444,
label: str = "UMAP",
- integrated_graph: Optional[str] = None,
+ integrated_graph: str | None = None,
parallel: bool = False,
- nthreads: Optional[int] = None,
+ nthreads: int | None = None,
) -> None:
"""Runs UMAP algorithm using the precomputed cell-neighbourhood graph.
The calculated UMAP coordinates are saved in the cell metadata table.
@@ -1273,10 +2528,10 @@ def run_umap(
select per positive sample in the optimization process. Increasing this value will
result in greater repulsive force being applied, greater optimization cost, but
slightly more accuracy. (Default value: 5)
- use_density_map:
- dens_lambda:
- dens_frac:
- dens_var_shift:
+ use_density_map: If True, run densMAP (density-preserving UMAP) instead of standard UMAP.
+ dens_lambda: densMAP density preservation strength (Default value: 2.0).
+ dens_frac: Fraction of nearest neighbors used for local density estimation (Default value: 0.3).
+ dens_var_shift: Variance shift for density correction (Default value: 0.1).
random_seed: (Default value: 4444)
label: base label for UMAP dimensions in the cell metadata column (Default value: 'UMAP')
integrated_graph:
@@ -1325,8 +2580,11 @@ def run_umap(
graph_loc = self._get_latest_graph_loc(from_assay, cell_key, feat_key)
knn_loc = graph_loc.rsplit("/", 1)[0]
logger.trace(f"Loading KNN dists and indices from {knn_loc}")
- dists = self.zw[knn_loc].distances[:]
- indices = self.zw[knn_loc].indices[:]
+ knn_group = as_zarr_group(self.zw[knn_loc], name=knn_loc)
+ dists = np.asarray(
+ as_zarr_array(knn_group["distances"], name="distances")[:]
+ )
+ indices = np.asarray(as_zarr_array(knn_group["indices"], name="indices")[:])
dmat = csr_matrix(
(
dists.flatten(),
@@ -1338,7 +2596,7 @@ def run_umap(
shape=(indices.shape[0], indices.shape[0]),
)
# dmat = dmat.maximum(dmat.transpose()).todok()
- logger.trace(f"Created sparse KNN dists and indices")
+ logger.trace("Created sparse KNN dists and indices")
densmap_kwds = {
"lambda": dens_lambda,
"frac": dens_frac,
@@ -1378,11 +2636,11 @@ def run_umap(
def run_leiden_clustering(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
resolution: float = 1.0,
- integrated_graph: Optional[str] = None,
+ integrated_graph: str | None = None,
symmetric_graph: bool = False,
graph_upper_only: bool = False,
label: str = "leiden_cluster",
@@ -1449,9 +2707,10 @@ def run_leiden_clustering(
if integrated_graph is not None:
from_assay = integrated_graph
+ membership = np.array(part.membership) + 1
self.cells.insert(
self._col_renamer(from_assay, cell_key, label),
- np.array(part.membership) + 1,
+ membership,
fill_value=-1,
key=cell_key,
overwrite=True,
@@ -1460,16 +2719,16 @@ def run_leiden_clustering(
def run_clustering(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
- n_clusters: Optional[int] = None,
- integrated_graph: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ n_clusters: int | None = None,
+ integrated_graph: str | None = None,
symmetric_graph: bool = False,
graph_upper_only: bool = False,
balanced_cut: bool = False,
- max_size: Optional[int] = None,
- min_size: Optional[int] = None,
+ max_size: int | None = None,
+ min_size: int | None = None,
max_distance_fc: float = 2,
force_recalc: bool = False,
label: str = "cluster",
@@ -1539,7 +2798,9 @@ def run_clustering(
dendrogram_loc = f"{graph_loc}/dendrogram"
# tuple are changed to list when saved as zarr attrs
if dendrogram_loc in self.zw and force_recalc is False:
- dendrogram = self.zw[dendrogram_loc][:]
+ dendrogram = np.asarray(
+ as_zarr_array(self.zw[dendrogram_loc], name=dendrogram_loc)[:]
+ )
logger.info("Using existing dendrogram")
else:
paris = skn.hierarchy.Paris(reorder=False)
@@ -1553,19 +2814,23 @@ def run_clustering(
)
dendrogram = paris.fit_transform(graph)
dendrogram[dendrogram == np.inf] = 0
+ graph_grp = as_zarr_group(self.zw[graph_loc], name=graph_loc)
g = create_zarr_dataset(
- self.zw[graph_loc],
+ graph_grp,
dendrogram_loc.rsplit("/", 1)[1],
(5000,),
"f8",
(graph.shape[0] - 1, 4),
)
g[:] = dendrogram
- self.zw[graph_loc].attrs["latest_dendrogram"] = dendrogram_loc
+ as_zarr_group(self.zw[graph_loc], name=graph_loc).attrs["latest_dendrogram"] = (
+ dendrogram_loc
+ )
if balanced_cut:
from ..dendrogram import BalancedCut
+ assert max_size is not None and min_size is not None
labels = BalancedCut(
dendrogram, max_size, min_size, max_distance_fc
).get_clusters()
@@ -1582,14 +2847,15 @@ def run_clustering(
key=cell_key,
overwrite=True,
)
+ return None
def run_topacedo_sampler(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
- cluster_key: Optional[str] = None,
- use_k: Optional[int] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ cluster_key: str | None = None,
+ use_k: int | None = None,
density_depth: int = 2,
density_bandwidth: float = 5.0,
max_sampling_rate: float = 0.05,
@@ -1606,7 +2872,7 @@ def run_topacedo_sampler(
save_seeds_key: str = "sketch_seeds",
rand_state: int = 4466,
return_edges: bool = False,
- ) -> Union[None, List]:
+ ) -> None | list[Any]:
"""Perform sub-sampling (aka sketching) of cells using TopACeDo
algorithm. Sub-sampling required that cells are partitioned in cluster
already. Since, sub-sampling is dependent on cluster information,
@@ -1674,7 +2940,12 @@ def run_topacedo_sampler(
)
graph_loc = self._get_latest_graph_loc(from_assay, cell_key, feat_key)
try:
- dendrogram = self.zw[f"{graph_loc}/dendrogram"][:]
+ dendrogram = np.asarray(
+ as_zarr_array(
+ self.zw[f"{graph_loc}/dendrogram"],
+ name=f"{graph_loc}/dendrogram",
+ )[:]
+ )
except KeyError:
raise KeyError(
"ERROR: Couldn't find the dendrogram for clustering. Please note that "
@@ -1724,18 +2995,19 @@ def run_topacedo_sampler(
logger.debug(f"Seed cells saved under column: '{key}'")
if return_edges:
- return edges
+ return cast(list[Any], edges)
+ return None
def get_imputed(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- feature_name: Optional[str] = None,
- feat_key: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feature_name: str | None = None,
+ feat_key: str | None = None,
t: int = 2,
cache_operator: bool = True,
) -> np.ndarray:
- """
+ """Impute feature values by diffusing along the KNN graph (MAGIC-style).
Args:
from_assay: Name of assay to be used. If no value is provided then the default assay will be used.
@@ -1778,17 +3050,23 @@ def calc_diff_operator(g: csr_matrix, to_power: int) -> coo_matrix:
if magic_loc in self.zw:
logger.info("Using existing MAGIC diffusion operator")
if self._cachedMagicOperatorLoc == magic_loc:
- diff_op = self._cachedMagicOperator
+ diff_op = cast(coo_matrix, self._cachedMagicOperator)
else:
n_cells, _ = self._get_graph_ncells_k(graph_loc)
- store = self.zw[magic_loc]
+ store = as_zarr_group(self.zw[magic_loc], name=magic_loc)
diff_op = coo_matrix(
- (store["data"][:], (store["row"][:], store["col"][:])),
+ (
+ np.asarray(as_zarr_array(store["data"], name="data")[:]),
+ (
+ np.asarray(as_zarr_array(store["row"], name="row")[:]),
+ np.asarray(as_zarr_array(store["col"], name="col")[:]),
+ ),
+ ),
shape=(n_cells, n_cells),
)
if cache_operator:
self._cachedMagicOperator = diff_op
- self._cachedMagicOperatorLoc = magic_loc
+ self._cachedMagicOperatorLoc = magic_loc # type: ignore[assignment]
else:
self._cachedMagicOperator = None
self._cachedMagicOperatorLoc = None
@@ -1805,30 +3083,33 @@ def calc_diff_operator(g: csr_matrix, to_power: int) -> coo_matrix:
store = self.zw.create_group(magic_loc, overwrite=True)
for i, j in zip(["row", "col", "data"], ["uint32", "uint32", "float32"]):
zg = create_zarr_dataset(store, i, (1000000,), j, shape)
- zg[:] = diff_op.__getattribute__(i)
- self.zw[graph_loc].attrs["latest_magic"] = magic_loc
+ zg[:] = getattr(diff_op, i)
+ as_zarr_group(self.zw[graph_loc], name=graph_loc).attrs["latest_magic"] = (
+ magic_loc
+ )
if cache_operator:
self._cachedMagicOperator = diff_op
- self._cachedMagicOperatorLoc = magic_loc
+ self._cachedMagicOperatorLoc = magic_loc # type: ignore[assignment]
else:
self._cachedMagicOperator = None
self._cachedMagicOperatorLoc = None
- return diff_op.dot(data)
+ return cast(np.ndarray, diff_op.dot(data))
def run_pseudotime_scoring(
self,
- from_assay: Optional[str] = None,
- cell_key: Optional[str] = None,
- subset_cell_key: Optional[str] = None,
- feat_key: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ subset_cell_key: str | None = None,
+ feat_key: str | None = None,
n_singular_vals: int = 30,
- source_sink_key: Optional[str] = None,
- sources: Optional[List] = None,
- sinks: Optional[List] = None,
- ss_vec: Optional[np.ndarray] = None,
+ source_sink_key: str | None = None,
+ sources: list[Any] | None = None,
+ sinks: list[Any] | None = None,
+ ss_vec: np.ndarray | None = None,
min_max_norm_ptime: bool = True,
random_seed: int = 4444,
label: str = "pseudotime",
+ component_policy: Literal["largest", "error"] = "largest",
) -> None:
"""
Calculate differentiation potential of cells. This function is a reimplementation of population balance
@@ -1856,60 +3137,13 @@ def run_pseudotime_scoring(
are in 0 to 1 range. (Default: True)
random_seed: A random seed for svds (Default: 4444)
label: label: Base label for pseudotime in the cell metadata column (Default value: 'pseudotime')
+ component_policy: How to handle a disconnected selected graph. ``'largest'`` scores the largest connected
+ component and marks other selected cells as unscored. ``'error'`` raises instead.
Returns:
"""
- from scipy.sparse.linalg import svds
-
- def inverse_degree(g):
- d = np.ravel(g.sum(axis=1))
- n = g.shape[0]
- d[d != 0] = 1 / d[d != 0]
- return csr_matrix((d, (range(n), range(n))), shape=[n, n])
-
- def laplacian(g, inv_deg):
- n = g.shape[0]
- identity = csr_matrix((np.ones(n), (range(n), range(n))), shape=[n, n])
- return identity - g.dot(inv_deg)
-
- def make_source_sink_vector(c, source, sink):
- ss = list(source) + list(sink)
-
- r = np.zeros(c.shape[0])
- r[c.isin(sink)] = 1
- r[c.isin(source)] = -1
-
- n = c.isin(ss).sum()
- v = (0 - r.sum()) / (r.shape[0] - n)
- r[~c.isin(ss)] = v
- return r
-
- def pseudo_inverse(lap, k, rseed, r):
- random_state = np.random.RandomState(rseed)
- # noinspection PyArgumentList
- v0 = random_state.rand(lap.shape[0])
- # TODO: add thread management here
- logger.info(
- "Calculating SVD of graph laplacian. This might take a while...",
- )
- u, s, vt = svds(lap, k=k, which="SM", v0=v0)
- # Because the order of singular values is not guaranteed
- idx = np.argsort(s)
- # Extracting the second smallest values
- s = s[idx][1:].T
- s = 1 / s
- u = u[:, idx][:, 1:]
- vt = vt[idx, :][1:, :].T
- # Computing matmul in an iterative way to save memory
- n = u.shape[0]
- # TODO: Use numba for this part
- ilap = np.zeros(n)
- for i in tqdmbar(range(n), desc="Calculating pseudotime"):
- ilap[i] = (vt * u[i, :] * s * r).sum()
- return ilap
-
from_assay, cell_key, feat_key = self._get_latest_keys(
from_assay, cell_key, feat_key
)
@@ -1931,80 +3165,147 @@ def pseudo_inverse(lap, k, rseed, r):
if cell_idx.shape[0] != cell_idx.sum():
graph = graph[cell_idx][:, cell_idx]
- if source_sink_key is None:
+ if graph.shape[0] == 0:
+ raise ValueError("No cells were selected for pseudotime scoring")
+ parent_cell_indices = self.cells.active_index(cell_key)
+ selected_cell_indices = parent_cell_indices[np.asarray(cell_idx, dtype=bool)]
+ retained_mask, component_sizes = _select_pseudotime_component(
+ graph,
+ selected_cell_indices,
+ component_policy,
+ )
+ retained_graph = graph[retained_mask][:, retained_mask].tocsr()
+ if len(component_sizes) > 1:
+ logger.warning(
+ f"Selected graph components have sizes {component_sizes}. "
+ f"Scoring the largest component with {retained_graph.shape[0]} cells"
+ )
+
+ retained_n_cells = retained_graph.shape[0]
+ if not isinstance(n_singular_vals, int) or isinstance(n_singular_vals, bool):
+ raise TypeError("n_singular_vals must be an integer")
+ if n_singular_vals < 2:
+ raise ValueError("n_singular_vals must be at least 2")
+ if retained_n_cells < 4:
+ raise ValueError(
+ "The retained graph must contain at least 4 cells for pseudotime scoring"
+ )
+ effective_k = min(n_singular_vals, retained_n_cells - 2)
+ if effective_k != n_singular_vals:
+ logger.warning(
+ f"Reducing n_singular_vals from {n_singular_vals} to {effective_k} "
+ "for the retained graph size"
+ )
+
+ label_arguments_supplied = (
+ source_sink_key is not None or sources is not None or sinks is not None
+ )
+ if ss_vec is not None and label_arguments_supplied:
+ raise ValueError(
+ "Provide either ss_vec or source_sink_key with source/sink labels, not both"
+ )
+ if ss_vec is None and source_sink_key is None:
if sources is not None or sinks is not None:
- logger.warning(
- "Provide `sources` and `sinks` will not be used because `source_sink_key` has not been "
- "provided"
- )
- if ss_vec is None:
- if source_sink_key is None:
- logger.warning(
- "No source/sink info or custom source sink vector provided. The results might not be "
- "reflect true pseudotime."
- )
- ss_vec = np.ones(graph.shape[0])
- else:
- clusts = pd.Series(
- self.cells.fetch(source_sink_key, key=cell_key)[cell_idx]
- )
- if sources is None:
- sources = []
- else:
- if isinstance(sources, list) is False:
- raise ValueError(
- "ERROR: Parameter `sources` should be of 'list' type"
- )
- if sinks is None:
- sinks = []
- else:
- if isinstance(sinks, list) is False:
- raise ValueError(
- "ERROR: Parameter `sinks` should be of 'list' type"
- )
- ss_vec = make_source_sink_vector(clusts, sources, sinks)
- else:
- if source_sink_key is not None:
- logger.warning(
- "Sources/sinks from `source_sink_key` will not be because custom vector `ss_vec` is "
- "provided"
- )
- ss_vec = np.array(ss_vec)
- if ss_vec.shape[0] != graph.shape[0]:
- raise ValueError(
- f"ERROR: Size mismatch between `ss_vec` ({ss_vec.shape[0]}) and "
- f"graph ({graph.shape[0]:})"
- )
- if ss_vec.sum() > 1e-10:
raise ValueError(
- f"ERROR: The sum of all the values in `ss_vec` should be zero. Here we test if the sum is less"
- f" 1e-10"
+ "source_sink_key is required when sources or sinks are provided"
)
+ raise ValueError("Provide source/sink labels or a custom zero-sum ss_vec")
- ss_vec = ss_vec.reshape(-1, 1)
+ if ss_vec is not None:
+ full_source_sink = _validate_source_sink_vector(
+ ss_vec,
+ graph.shape[0],
+ "ss_vec",
+ )
+ retained_source_sink = _validate_source_sink_vector(
+ full_source_sink[retained_mask],
+ retained_n_cells,
+ "ss_vec restricted to the retained component",
+ )
+ else:
+ if sources is not None and not isinstance(sources, list):
+ raise TypeError("sources must be a list")
+ if sinks is not None and not isinstance(sinks, list):
+ raise TypeError("sinks must be a list")
+ source_labels = [] if sources is None else sources
+ sink_labels = [] if sinks is None else sinks
+ if not source_labels and not sink_labels:
+ raise ValueError("At least one source or sink label must be provided")
+
+ selected_labels = pd.Series(
+ self.cells.fetch(cast(str, source_sink_key), key=cell_key)[cell_idx]
+ )
+ _validate_source_sink_labels(
+ selected_labels,
+ source_labels,
+ sink_labels,
+ "the selected cells",
+ )
+ retained_labels = selected_labels.iloc[
+ np.flatnonzero(retained_mask)
+ ].reset_index(drop=True)
+ _validate_source_sink_labels(
+ retained_labels,
+ source_labels,
+ sink_labels,
+ "the retained connected component",
+ )
+ retained_source_sink = _validate_source_sink_vector(
+ _make_source_sink_vector(
+ retained_labels,
+ source_labels,
+ sink_labels,
+ ),
+ retained_n_cells,
+ "generated source/sink vector",
+ )
- ptime = pseudo_inverse(
- laplacian(graph, inverse_degree(graph)),
- n_singular_vals,
+ retained_ptime = _truncated_pba_potential(
+ _random_walk_laplacian_transpose(retained_graph),
+ effective_k,
random_seed,
- ss_vec,
+ retained_source_sink,
)
+ if not np.isfinite(retained_ptime).all():
+ raise ValueError("Pseudotime calculation produced non-finite values")
+ value_range = float(np.ptp(retained_ptime))
+ potential_scale = max(1.0, float(np.abs(retained_ptime).max()))
+ if value_range <= np.finfo(float).eps * potential_scale:
+ raise ValueError("Pseudotime calculation produced a constant potential")
if min_max_norm_ptime:
- # noinspection PyArgumentList
- ptime = ptime - ptime.min()
- ptime = ptime / ptime.max()
+ retained_ptime = (retained_ptime - retained_ptime.min()) / value_range
+ retained_ptime = np.clip(retained_ptime, 0.0, 1.0)
+ if not np.isfinite(retained_ptime).all():
+ raise ValueError("Pseudotime normalization produced non-finite values")
+
+ ptime = np.full(graph.shape[0], np.nan, dtype=float)
+ ptime[retained_mask] = retained_ptime
+ output_column = self._col_renamer(from_assay, subset_cell_key, label)
+ validity_column = f"{output_column}__valid"
self.cells.insert(
- self._col_renamer(from_assay, subset_cell_key, label),
+ output_column,
ptime,
key=subset_cell_key,
overwrite=True,
)
+ self.cells.insert(
+ validity_column,
+ retained_mask,
+ fill_value=False,
+ key=subset_cell_key,
+ overwrite=True,
+ )
+ if not retained_mask.all():
+ logger.warning(
+ f"Unscored cells contain NaN pseudotime. Use cell key "
+ f"'{validity_column}' for downstream analysis"
+ )
return None
def integrate_assays(
self,
- assays: List[str],
+ assays: list[str],
label: str,
method: str = "snn",
chunk_size: int = 10000,
@@ -2026,7 +3327,7 @@ def integrate_assays(
"""
from ..knn_utils import merge_graphs, wnn_integration
- def load_pca_knn(assay_name):
+ def load_pca_knn(assay_name: str) -> tuple[csr_matrix, NDArray[Any]]:
g = self.load_graph(
from_assay=assay_name,
cell_key=None,
@@ -2040,14 +3341,17 @@ def load_pca_knn(assay_name):
return_ann_object=True,
update_keys=False,
)
+ if ao is None:
+ raise RuntimeError(
+ f"make_graph did not return AnnStream for {assay_name}"
+ )
if ao.loadings is None:
logger.warning(
f"No dimension reduction was user for {assay_name} data. "
f"Memory consumption will be high."
)
return g, ao.data.compute()
- else:
- return g, ao.data.dot(ao.loadings).compute()
+ return g, ao.data.dot(ao.loadings).compute()
if method == "snn":
merged_graph = []
@@ -2085,21 +3389,23 @@ def load_pca_knn(assay_name):
ig_loc = self._integratedGraphsLoc
if ig_loc not in self.zw:
self.zw.create_group(ig_loc)
- if label in self.zw[ig_loc]:
+ ig_grp = as_zarr_group(self.zw[ig_loc], name=ig_loc)
+ if label in ig_grp:
del self.zw[f"{ig_loc}/{label}"]
store = self.zw.create_group(f"{ig_loc}/{label}")
store.attrs["n_cells"] = n_cells
store.attrs["n_neighbors"] = n_neighbors
+ edge_chunk = chunk_size * n_neighbors
zge = create_zarr_dataset(
store,
- f"edges",
- (chunk_size,),
- (np.uint32, np.uint32),
+ "edges",
+ (edge_chunk,),
+ ("u8", "u8"),
(n_cells * n_neighbors, 2),
)
zgw = create_zarr_dataset(
- store, f"weights", (chunk_size,), np.float64, (n_cells * n_neighbors)
+ store, "weights", (edge_chunk,), "f8", (n_cells * n_neighbors,)
)
zge[:, 0] = merged_graph.row
diff --git a/scarf/datastore/mapping_datastore.py b/scarf/datastore/mapping_datastore.py
index 5aef6d42..7223ff1a 100644
--- a/scarf/datastore/mapping_datastore.py
+++ b/scarf/datastore/mapping_datastore.py
@@ -1,20 +1,25 @@
+from collections.abc import Callable, Generator
+from typing import Any, cast
import os
-from typing import Generator, Tuple, List, Dict, Union, Callable, Optional
+import warnings
import numpy as np
import pandas as pd
-from dask import array as daskarr
+import zarr
from loguru import logger
+from numpy.typing import NDArray
from scipy.sparse import csr_matrix
-from .graph_datastore import GraphDataStore
+from .._types import as_zarr_array, as_zarr_group
+from ..ann import AnnStream
from ..assay import Assay, RNAassay
+from ..chunked import ChunkedArray
+from ..mapping_reference import MappingReference, MappingResult
+from ..symphony import SYMPHONY_STYLE_VARIANT
+from .graph_datastore import GraphDataStore
from ..utils import (
- show_dask_progress,
- clean_array,
tqdmbar,
controlled_compute,
- system_call,
)
from ..writers import create_zarr_dataset
@@ -25,8 +30,564 @@ class MappingDatastore(GraphDataStore):
required for label transfer, mapping score generation and co-embedding.
"""
- def __init__(self, **kwargs):
- super().__init__(**kwargs)
+ _PROJECTION_SCHEMA_VERSION = 2
+ _LEGACY_PROJECTION_SCHEMA_VERSIONS = {1}
+
+ @staticmethod
+ def _validate_projection_arrays(store: zarr.Group, target_name: str) -> None:
+ if "indices" not in store or "distances" not in store:
+ raise ValueError(
+ f"Projection {target_name!r} is missing indices or distances."
+ )
+ indices = as_zarr_array(store["indices"], name="indices")
+ distances = as_zarr_array(store["distances"], name="distances")
+ if (
+ len(indices.shape) != 2
+ or len(distances.shape) != 2
+ or indices.shape != distances.shape
+ ):
+ raise ValueError(
+ f"Projection {target_name!r} has incompatible neighbor arrays."
+ )
+ if indices.shape[1] < 1:
+ raise ValueError(
+ f"Projection {target_name!r} does not contain any neighbors."
+ )
+
+ def _load_complete_projection(
+ self,
+ target_name: str,
+ from_assay: str,
+ cell_key: str,
+ feat_key: str | None = None,
+ ) -> zarr.Group:
+ from ..mapping_utils import array_hash
+
+ store_loc = f"{from_assay}/projections/{target_name}"
+ if store_loc not in self.zw:
+ raise KeyError(
+ f"Projections have not been computed for {target_name}. Run run_mapping first."
+ )
+ store = as_zarr_group(self.zw[store_loc], name=store_loc)
+ attrs = store.attrs
+ self._validate_projection_arrays(store, target_name)
+ if "schemaVersion" not in attrs:
+ if "complete" in attrs and not bool(attrs["complete"]):
+ raise ValueError(
+ f"Projection {target_name!r} is incomplete. Run run_mapping again."
+ )
+ warnings.warn(
+ f"Projection {target_name!r} predates projection provenance. "
+ "Re-run run_mapping to remove this compatibility warning.",
+ DeprecationWarning,
+ stacklevel=3,
+ )
+ return store
+ if not bool(attrs.get("complete", False)):
+ raise ValueError(
+ f"Projection {target_name!r} is incomplete. Run run_mapping again."
+ )
+ schema_version = attrs.get("schemaVersion")
+ if schema_version in self._LEGACY_PROJECTION_SCHEMA_VERSIONS:
+ warnings.warn(
+ f"Projection {target_name!r} uses legacy projection schema "
+ f"{schema_version}. Re-run run_mapping to upgrade its provenance.",
+ DeprecationWarning,
+ stacklevel=3,
+ )
+ elif schema_version != self._PROJECTION_SCHEMA_VERSION:
+ raise ValueError(
+ f"Projection {target_name!r} uses an incompatible schema. Run run_mapping again."
+ )
+ if attrs.get("assay") != from_assay or attrs.get("cellKey") != cell_key:
+ raise ValueError(
+ f"Projection {target_name!r} does not match the selected reference assay or cells."
+ )
+ stored_feat_key = attrs.get("featureKey")
+ if feat_key is not None and stored_feat_key != feat_key:
+ logger.warning(
+ f"Projection {target_name!r} uses feature key {stored_feat_key!r}, "
+ f"not the current key {feat_key!r}; validating its stored provenance."
+ )
+ reference_cells = self.cells.fetch("ids", key=cast(str, attrs["cellKey"]))
+ if attrs.get("referenceCellHash") != array_hash(reference_cells):
+ raise ValueError(
+ f"Projection {target_name!r} was built from a different reference cell set."
+ )
+ if schema_version == self._PROJECTION_SCHEMA_VERSION:
+ self._validate_projection_provenance(store, target_name)
+ return store
+
+ def _validate_projection_provenance(
+ self, store: zarr.Group, target_name: str
+ ) -> None:
+ from ..mapping_utils import array_hash
+
+ attrs = store.attrs
+ save_k = attrs.get("saveK")
+ if (
+ isinstance(save_k, bool)
+ or not isinstance(save_k, int | np.integer)
+ or int(save_k)
+ != int(as_zarr_array(store["indices"], name="indices").shape[1])
+ ):
+ raise ValueError(
+ f"Projection {target_name!r} has inconsistent saved-neighbor provenance."
+ )
+ assay_name = cast(str, attrs["assay"])
+ cell_key = cast(str, attrs["cellKey"])
+ feature_key = cast(str, attrs["featureKey"])
+ source_assay = self._get_assay(assay_name)
+ feature_column = (
+ feature_key if feature_key == "I" else f"{cell_key}__{feature_key}"
+ )
+ reference_features = source_assay.feats.fetch("ids", key=feature_column)
+ if attrs.get("referenceFeatureHash") != array_hash(reference_features):
+ raise ValueError(
+ f"Projection {target_name!r} was built from a different reference feature set."
+ )
+
+ reference_path = cast(str, attrs.get("referencePath", ""))
+ if reference_path not in self.zw:
+ raise ValueError(
+ f"Projection {target_name!r} references missing normalized data."
+ )
+ normed = as_zarr_group(self.zw[reference_path], name=reference_path)
+ if attrs.get("referenceSubsetHash") != normed.attrs.get("subset_hash"):
+ raise ValueError(
+ f"Projection {target_name!r} references changed normalized data."
+ )
+
+ reduction_path = cast(str, attrs.get("reductionPath", ""))
+ if reduction_path not in self.zw:
+ raise ValueError(
+ f"Projection {target_name!r} references a missing reduction."
+ )
+ reduction = as_zarr_group(self.zw[reduction_path], name=reduction_path)
+ if "reduction" not in reduction:
+ raise ValueError(
+ f"Projection {target_name!r} references a reduction without loadings."
+ )
+ loadings = np.asarray(
+ as_zarr_array(reduction["reduction"], name="reduction")[:]
+ )
+ if attrs.get("reductionHash") != array_hash(loadings):
+ raise ValueError(
+ f"Projection {target_name!r} references changed reduction loadings."
+ )
+
+ ann_path = cast(str, attrs.get("annPath", ""))
+ if ann_path not in self.zw:
+ raise ValueError(
+ f"Projection {target_name!r} references a missing ANN index."
+ )
+ ann = as_zarr_group(self.zw[ann_path], name=ann_path)
+ expected_scaling = bool(attrs.get("annFeatureScaling"))
+ if bool(ann.attrs.get("featureScaling", True)) != expected_scaling:
+ raise ValueError(
+ f"Projection {target_name!r} references an ANN index with changed scaling."
+ )
+ if bool(ann.attrs.get("isHarmonized", False)) != bool(
+ attrs.get("annIsHarmonized")
+ ):
+ raise ValueError(
+ f"Projection {target_name!r} references a changed ANN coordinate space."
+ )
+ ann_parts = ann_path.rsplit("/", 1)[-1].split("__")
+ if len(ann_parts) < 6:
+ raise ValueError(
+ f"Projection {target_name!r} has an invalid ANN provenance path."
+ )
+ stored_ann_values = (
+ attrs.get("annEfc"),
+ attrs.get("annEf"),
+ attrs.get("annM"),
+ attrs.get("annRandomState"),
+ )
+ if not all(isinstance(value, int | float) for value in stored_ann_values):
+ raise ValueError(
+ f"Projection {target_name!r} is missing numeric ANN settings."
+ )
+ ann_efc, ann_ef, ann_m, ann_random_state = cast(
+ tuple[int | float, ...], stored_ann_values
+ )
+ if (
+ attrs.get("annMetric") != ann_parts[1]
+ or int(ann_efc) != int(ann_parts[2])
+ or int(ann_ef) != int(ann_parts[3])
+ or int(ann_m) != int(ann_parts[4])
+ or int(ann_random_state) != int(ann_parts[5])
+ ):
+ raise ValueError(
+ f"Projection {target_name!r} references incompatible ANN settings."
+ )
+
+ if "referenceFeatureIndices" not in store:
+ raise ValueError(
+ f"Projection {target_name!r} is missing selected-feature provenance."
+ )
+ feature_indices = np.asarray(
+ as_zarr_array(
+ store["referenceFeatureIndices"], name="referenceFeatureIndices"
+ )[:],
+ dtype=np.int64,
+ )
+ all_feature_ids = source_assay.feats.fetch_all("ids")
+ if np.any(feature_indices < 0) or np.any(
+ feature_indices >= len(all_feature_ids)
+ ):
+ raise ValueError(
+ f"Projection {target_name!r} contains invalid reference feature indices."
+ )
+ if attrs.get("selectedFeatureHash") != array_hash(
+ all_feature_ids[feature_indices]
+ ):
+ raise ValueError(
+ f"Projection {target_name!r} references a changed selected feature set."
+ )
+ if "__intersection_" in ann_path and ann.attrs.get(
+ "selectedFeatureHash"
+ ) != attrs.get("selectedFeatureHash"):
+ raise ValueError(
+ f"Projection {target_name!r} references a changed intersection ANN index."
+ )
+ if "__intersection_" in ann_path:
+ source_ann_path = attrs.get("annSourcePath")
+ if (
+ not isinstance(source_ann_path, str)
+ or not source_ann_path
+ or ann.attrs.get("sourceAnnPath") != source_ann_path
+ or source_ann_path not in self.zw
+ ):
+ raise ValueError(
+ f"Projection {target_name!r} has invalid intersection ANN provenance."
+ )
+ source_ann = as_zarr_group(self.zw[source_ann_path], name=source_ann_path)
+ if bool(source_ann.attrs.get("featureScaling", True)) != expected_scaling:
+ raise ValueError(
+ f"Projection {target_name!r} references a changed source ANN space."
+ )
+ if attrs.get("correctionMethod") == "symphony":
+ artifact_path = attrs.get("mappingReferencePath")
+ if not isinstance(artifact_path, str) or artifact_path not in self.zw:
+ raise ValueError(
+ f"Projection {target_name!r} references a missing mapping artifact."
+ )
+ from ..mapping_reference import validate_mapping_reference_artifact
+
+ validate_mapping_reference_artifact(
+ as_zarr_group(self.zw[artifact_path], name=artifact_path)
+ )
+
+ @staticmethod
+ def _same_assay_store(source_assay: Assay, target_assay: Assay) -> bool:
+ if source_assay is target_assay:
+ return True
+ source_group = source_assay.z
+ target_group = target_assay.z
+
+ def normalized_store_path(group: zarr.Group) -> str:
+ value = str(getattr(group, "store_path", "")).rstrip("/")
+ if value.startswith("file://"):
+ return os.path.realpath(value[7:])
+ return value
+
+ source_store_path = normalized_store_path(source_group)
+ target_store_path = normalized_store_path(target_group)
+ if source_store_path and source_store_path == target_store_path:
+ return True
+ return getattr(source_group, "store", None) is getattr(
+ target_group, "store", None
+ ) and getattr(source_group, "path", None) == getattr(target_group, "path", None)
+
+ def _guard_mapping_target_path(
+ self,
+ source_assay: Assay,
+ target_assay: Assay,
+ source_cell_key: str,
+ source_feat_key: str,
+ target_cell_key: str,
+ target_feat_key: str,
+ ) -> None:
+ if not self._same_assay_store(source_assay, target_assay):
+ return
+ source_path = f"normed__{source_cell_key}__{source_feat_key}"
+ target_path = f"normed__{target_cell_key}__{target_feat_key}"
+ if source_path == target_path:
+ raise ValueError(
+ "The mapping target normalization path matches the reference path. "
+ "Choose a distinct target_feat_key so reference data cannot be overwritten."
+ )
+
+ @staticmethod
+ def _projection_block_size(indices: Any) -> int:
+ chunks = getattr(indices, "chunks", None)
+ if chunks is not None and len(chunks) > 0:
+ return int(chunks[0])
+ return min(max(int(indices.shape[0]), 1), 10_000)
+
+ def _iter_projection_neighbor_rows(
+ self, store: zarr.Group
+ ) -> Generator[tuple[int, np.ndarray, np.ndarray, np.ndarray, bool], None, None]:
+ from ..mapping_utils import distance_weights
+
+ indices = as_zarr_array(store["indices"], name="indices")
+ distances = as_zarr_array(store["distances"], name="distances")
+ uninformative = (
+ as_zarr_array(store["uninformative"], name="uninformative")
+ if "uninformative" in store
+ else None
+ )
+ block_size = self._projection_block_size(indices)
+ for start in range(0, indices.shape[0], block_size):
+ stop = min(start + block_size, indices.shape[0])
+ block_indices = np.asarray(indices[start:stop])
+ block_distances = np.asarray(distances[start:stop])
+ block_weights = distance_weights(block_distances)
+ block_uninformative = (
+ np.asarray(uninformative[start:stop], dtype=bool)
+ if uninformative is not None
+ else np.zeros(stop - start, dtype=bool)
+ )
+ rows = zip(
+ block_indices,
+ block_weights,
+ block_distances,
+ block_uninformative,
+ strict=True,
+ )
+ for offset, row in enumerate(rows):
+ neighbors, weights, row_distances, force_unknown = row
+ yield (
+ start + offset,
+ neighbors,
+ weights,
+ row_distances,
+ bool(force_unknown),
+ )
+
+ def _projection_provenance(
+ self,
+ source_assay: Assay,
+ target_assay: Assay,
+ source_assay_name: str,
+ target_name: str,
+ cell_key: str,
+ feat_key: str,
+ target_cell_key: str,
+ target_feat_key: str,
+ feature_indices: np.ndarray,
+ correction_method: str,
+ ann_obj: AnnStream,
+ ) -> dict[str, Any]:
+ from ..mapping_utils import array_hash
+
+ reference_features = source_assay.feats.fetch(
+ "ids", key=f"{cell_key}__{feat_key}" if feat_key != "I" else "I"
+ )
+ selected_feature_ids = source_assay.feats.fetch_all("ids")[feature_indices]
+ if correction_method == "intersection":
+ feature_coverage = float(len(feature_indices) / len(reference_features))
+ else:
+ target_feature_ids = target_assay.feats.fetch_all("ids")
+ feature_coverage = float(
+ np.isin(reference_features, target_feature_ids).sum()
+ / len(reference_features)
+ )
+ reference_path = f"{source_assay_name}/normed__{cell_key}__{feat_key}"
+ reduction_path: str | None = None
+ ann_path: str | None = ann_obj.annPath
+ if ann_path is not None:
+ reduction_path = ann_path.rsplit("/ann__", 1)[0]
+ if reference_path in self.zw:
+ normed = as_zarr_group(self.zw[reference_path], name=reference_path)
+ if reduction_path is None:
+ reduction_path = cast(str | None, normed.attrs.get("latest_reduction"))
+ if reduction_path is not None and reduction_path in self.zw:
+ reduction = as_zarr_group(self.zw[reduction_path], name=reduction_path)
+ if ann_path is None:
+ ann_path = cast(str | None, reduction.attrs.get("latest_ann"))
+ reduction_hash = ""
+ reference_subset_hash: int | str = ""
+ ann_feature_scaling = ann_obj.featureScaling
+ ann_is_harmonized = ann_obj.harmonize
+ ann_source_path = ""
+ if ann_path is not None and ann_path in self.zw:
+ ann_group = as_zarr_group(self.zw[ann_path], name=ann_path)
+ stored_source_path = ann_group.attrs.get("sourceAnnPath")
+ if isinstance(stored_source_path, str):
+ ann_source_path = stored_source_path
+ if reference_path in self.zw:
+ normed = as_zarr_group(self.zw[reference_path], name=reference_path)
+ reference_subset_hash = cast(int | str, normed.attrs.get("subset_hash", ""))
+ if reduction_path is not None and reduction_path in self.zw:
+ reduction = as_zarr_group(self.zw[reduction_path], name=reduction_path)
+ if "reduction" in reduction:
+ reduction_hash = array_hash(
+ np.asarray(
+ as_zarr_array(reduction["reduction"], name="reduction")[:]
+ )
+ )
+ return {
+ "schemaVersion": self._PROJECTION_SCHEMA_VERSION,
+ "complete": False,
+ "assay": source_assay_name,
+ "targetName": target_name,
+ "cellKey": cell_key,
+ "featureKey": feat_key,
+ "targetCellKey": target_cell_key,
+ "targetFeatureKey": target_feat_key,
+ "referencePath": reference_path,
+ "reductionPath": reduction_path or "",
+ "annPath": ann_path or "",
+ "referenceCellHash": array_hash(self.cells.fetch("ids", key=cell_key)),
+ "targetCellHash": array_hash(
+ target_assay.cells.fetch("ids", key=target_cell_key)
+ ),
+ "referenceFeatureHash": array_hash(reference_features),
+ "selectedFeatureHash": array_hash(selected_feature_ids),
+ "referenceSubsetHash": reference_subset_hash,
+ "reductionHash": reduction_hash,
+ "featureCoverage": feature_coverage,
+ "reductionMethod": ann_obj.method,
+ "reductionDimensions": int(
+ ann_obj.dims if ann_obj.dims is not None else ann_obj.nFeats
+ ),
+ "annMetric": ann_obj.annMetric,
+ "annEfc": int(ann_obj.annEfc),
+ "annEf": int(ann_obj.annEf),
+ "annM": int(ann_obj.annM),
+ "annRandomState": int(ann_obj.randState),
+ "annFeatureScaling": ann_feature_scaling,
+ "annIsHarmonized": ann_is_harmonized,
+ "annSourcePath": ann_source_path,
+ "correctionMethod": correction_method,
+ "algorithmVariant": (
+ SYMPHONY_STYLE_VARIANT
+ if correction_method == "symphony"
+ else correction_method
+ ),
+ }
+
+ def _build_intersection_ann(
+ self,
+ ann_obj: AnnStream,
+ source_assay: Assay,
+ cell_key: str,
+ feat_key: str,
+ feature_indices: np.ndarray,
+ ann_index_saver: Callable | None,
+ ) -> AnnStream:
+ from ..mapping_utils import _distance_quantile_summary, array_hash
+
+ if ann_obj.method != "pca" or ann_obj.loadings is None:
+ raise ValueError(
+ "missing_feature_policy='intersection' only supports PCA references"
+ )
+ if ann_obj.harmonize:
+ raise ValueError(
+ "missing_feature_policy='intersection' is not supported for harmonized references"
+ )
+ active_indices = source_assay.feats.active_index(
+ f"{cell_key}__{feat_key}" if feat_key != "I" else "I"
+ )
+ positions = np.searchsorted(active_indices, feature_indices)
+ if len(positions) != len(feature_indices) or not np.array_equal(
+ active_indices[positions], feature_indices
+ ):
+ raise ValueError("Failed to align selected reference feature positions")
+ intersection_ann = AnnStream(
+ data=ann_obj.data[:, positions],
+ k=ann_obj.k,
+ n_cluster=2,
+ reduction_method="pca",
+ dims=ann_obj.loadings.shape[1],
+ loadings=ann_obj.loadings[positions, :],
+ use_for_pca=np.ones(ann_obj.nCells, dtype=bool),
+ mu=ann_obj.mu[positions],
+ sigma=ann_obj.sigma[positions],
+ ann_metric=ann_obj.annMetric,
+ ann_efc=ann_obj.annEfc,
+ ann_ef=ann_obj.annEf,
+ ann_m=ann_obj.annM,
+ nthreads=self.nthreads,
+ ann_parallel=False,
+ rand_state=ann_obj.randState,
+ do_kmeans_fit=False,
+ disable_scaling=not ann_obj.featureScaling,
+ ann_idx=None,
+ lsi_skip_first=True,
+ lsi_params={},
+ harmonize=False,
+ cache_embeddings=False,
+ )
+ if ann_obj.annPath is None:
+ raise ValueError("The reference ANN path is unavailable")
+ selected_feature_hash = array_hash(
+ source_assay.feats.fetch_all("ids")[feature_indices]
+ )
+ intersection_path = (
+ f"{ann_obj.annPath}__intersection_{selected_feature_hash[:16]}"
+ )
+ if intersection_path not in self.zw:
+ self.zw.create_group(intersection_path)
+ intersection_group = as_zarr_group(
+ self.zw[intersection_path], name=intersection_path
+ )
+ intersection_group.attrs.update(
+ {
+ "featureScaling": intersection_ann.featureScaling,
+ "isHarmonized": False,
+ "selectedFeatureHash": selected_feature_hash,
+ "sourceAnnPath": ann_obj.annPath,
+ }
+ )
+ self._persist_ann_index(
+ intersection_path,
+ intersection_ann.annIdx,
+ ann_index_saver,
+ )
+ sample_stride = max(
+ int(np.ceil(intersection_ann.nCells / 100_000)),
+ 1,
+ )
+ sampled_distances: list[np.ndarray] = []
+ entry_start = 0
+ for block in intersection_ann.iter_blocks():
+ entry_end = entry_start + len(block)
+ transformed = intersection_ann.transform_query(block)
+ _, distances, _ = cast(
+ tuple[np.ndarray, np.ndarray, int],
+ intersection_ann.transform_ann(
+ transformed,
+ self_indices=np.arange(entry_start, entry_end),
+ ),
+ )
+ sample_mask = (
+ np.arange(entry_start, entry_end, dtype=np.int64) % sample_stride == 0
+ )
+ sampled_distances.append(
+ np.asarray(distances[sample_mask, 0], dtype=np.float64)
+ )
+ entry_start = entry_end
+ nearest_distances = np.concatenate(sampled_distances)
+ distance_quantiles, distance_values = _distance_quantile_summary(
+ nearest_distances
+ )
+ for name, values in (
+ ("referenceDistanceQuantiles", distance_quantiles),
+ ("referenceDistanceValues", distance_values),
+ ):
+ output = create_zarr_dataset(
+ intersection_group,
+ name,
+ (min(len(values), 1_001),),
+ "f8",
+ values.shape,
+ )
+ output[:] = values
+ intersection_ann.annPath = intersection_path
+ return intersection_ann
def run_mapping(
self,
@@ -34,9 +595,9 @@ def run_mapping(
target_name: str,
target_feat_key: str,
target_cell_key: str = "I",
- from_assay: Optional[str] = None,
- cell_key: str = "I",
- feat_key: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
save_k: int = 3,
batch_size: int = 1000,
ref_mu: bool = True,
@@ -45,8 +606,10 @@ def run_mapping(
exclude_missing: bool = False,
filter_null: bool = False,
feat_scaling: bool = True,
- ann_index_fetcher: Optional[Callable] = None,
- ann_index_saver: Optional[Callable] = None,
+ ann_index_fetcher: Callable | None = None,
+ ann_index_saver: Callable | None = None,
+ missing_feature_policy: str | None = None,
+ query_batches: pd.DataFrame | None = None,
) -> None:
"""Projects cells from external assays into the cell-neighbourhood
graph using existing PCA loadings and ANN index. For each external cell
@@ -66,24 +629,31 @@ def run_mapping(
save_k: Number of the nearest neighbours to identify for each target cell (Default value: 3)
batch_size: Number of cells that will be projected as a batch. This used to decide the chunk size when
normalized data for the target cells is saved to disk.
- ref_mu: If True (default), Then mean values of features as in the reference are used,
- otherwise mean is calculated using target cells. Turning this to False is not recommended.
- ref_sigma: If True (default), Then standard deviation values of features as present in the reference are
- used, otherwise std. dev. is calculated using target cells. Turning this to False is not
- recommended.
- run_coral: If True then CORAL feature rescaling algorithm is used to correct for domain shift in target
- cells. Read more about CORAL algorithm in function ``coral``. This algorithm creates a m by m
- matrix where m is the number of features being used for mapping; so it is not advised to use this
- in a case where a large number of features are being used (>10k for example).
- (Default value: False)
+ ref_mu: Deprecated compatibility flag. Reference means are always used.
+ ref_sigma: Deprecated compatibility flag. Reference scales are always used.
+ run_coral: Deprecated experimental feature-space correction. If True,
+ CORAL aligns target features to the reference distribution.
+ It creates an m by m matrix where m is the number of
+ features, so it is not suitable for very large feature
+ sets. Build a Symphony-style mapping reference for new
+ harmonized atlas workflows. (Default value: False)
exclude_missing: If set to True then only those features that are present in both reference and
target are used. If not all reference features from `feat_key` are present in target data
- then a new graph will be created for reference and mapping will be done onto that graph.
- (Default value: False)
+ then a compatibility graph key is retained while mapping uses an isolated intersection
+ index. Deprecated; use ``missing_feature_policy='intersection'``.
filter_null: If True then those features that have a total sum of 0 in the target cells are removed.
This has an affect only when `exclude_missing` is True. (Default value: False)
- feat_scaling: If False then features from target cells are not scaled. This is automatically set to False
- if `run_coral` is True (Default value: True). Setting this to False is not recommended.
+ feat_scaling: If False then features from target cells are not scaled.
+ Setting this to False is not recommended.
+ missing_feature_policy: Handling for reference features absent from the
+ target. ``'zero'`` fills them with zero values, ``'intersection'``
+ constructs an isolated overlap-only ANN index, and ``'error'``
+ rejects incomplete overlap. Harmonized mapping references also
+ support ``'reference_mean'``, their neutral default.
+ ``exclude_missing=True`` is retained as a deprecated alias for
+ ``'intersection'``.
+ query_batches: Optional query batch metadata for Symphony-style correction
+ when mapping into a reusable harmonized reference.
ann_index_fetcher:
ann_index_saver:
@@ -97,22 +667,136 @@ def run_mapping(
)
source_assay = self._get_assay(from_assay)
- if type(target_assay) != type(source_assay):
+ if type(target_assay) is not type(source_assay):
raise TypeError(
f"ERROR: Source assay ({type(source_assay)}) and target assay "
f"({type(target_assay)}) are of different types. "
f"Mapping can only be performed between same assay types"
)
- if type(target_assay) == RNAassay:
+ if isinstance(target_assay, RNAassay):
if target_assay.sf != source_assay.sf:
logger.info(
f"Resetting target assay's size factor from {target_assay.sf} to {source_assay.sf}"
)
target_assay.sf = source_assay.sf
+ if not ref_mu or not ref_sigma:
+ warnings.warn(
+ "ref_mu and ref_sigma are deprecated and ignored. Mapping always "
+ "uses the reference graph's mean and scale.",
+ DeprecationWarning,
+ stacklevel=2,
+ )
+ self._guard_mapping_target_path(
+ source_assay,
+ target_assay,
+ cell_key,
+ feat_key,
+ target_cell_key,
+ target_feat_key,
+ )
- if target_feat_key == feat_key:
+ normed_loc = f"{from_assay}/normed__{cell_key}__{feat_key}"
+ if normed_loc in self.zw:
+ normed_group = as_zarr_group(self.zw[normed_loc], name=normed_loc)
+ reduction_loc = cast(str | None, normed_group.attrs.get("latest_reduction"))
+ if reduction_loc is not None and reduction_loc in self.zw:
+ reduction_group = as_zarr_group(
+ self.zw[reduction_loc], name=reduction_loc
+ )
+ ann_loc = cast(str | None, reduction_group.attrs.get("latest_ann"))
+ if ann_loc is not None and ann_loc in self.zw:
+ ann_group = as_zarr_group(self.zw[ann_loc], name=ann_loc)
+ if bool(ann_group.attrs.get("isHarmonized", False)):
+ if run_coral:
+ raise ValueError(
+ "CORAL cannot be combined with a harmonized mapping reference"
+ )
+ if exclude_missing or missing_feature_policy == "intersection":
+ raise ValueError(
+ "Harmonized mapping references do not support intersection-only "
+ "feature mapping. Use reference_mean, zero, or error handling."
+ )
+ try:
+ reference = self.get_mapping_reference(
+ from_assay, cell_key, feat_key
+ )
+ except ValueError as exc:
+ if "predates the mapping-reference artifact" not in str(
+ exc
+ ):
+ raise
+ if self.zarr_mode != "r+":
+ raise ValueError(
+ "This read-only harmonized reference needs a one-time "
+ "upgrade. Reopen it with zarr_mode='r+' and call "
+ "build_mapping_reference(..., batch_columns=[...])."
+ ) from exc
+ harmonized = as_zarr_array(
+ reduction_group["harmonizedData"],
+ name="harmonizedData",
+ )
+ batch_columns = cast(
+ list[str] | None, harmonized.attrs.get("batches")
+ )
+ if not batch_columns:
+ raise ValueError(
+ "The legacy harmonized graph does not record its batch "
+ "columns. Call build_mapping_reference with batch_columns."
+ ) from exc
+ warnings.warn(
+ "This harmonized graph predates portable mapping references. "
+ "It will be rebuilt once before mapping.",
+ DeprecationWarning,
+ stacklevel=2,
+ )
+ reference = self.build_mapping_reference(
+ from_assay,
+ cell_key,
+ feat_key,
+ batch_columns=batch_columns,
+ )
+ reference.map_query(
+ target_assay,
+ target_name,
+ target_feat_key,
+ target_cell_key=target_cell_key,
+ save_k=save_k,
+ query_batches=query_batches,
+ missing_feature_policy=(
+ missing_feature_policy or "reference_mean"
+ ),
+ )
+ return None
+
+ if run_coral:
+ warnings.warn(
+ "CORAL mapping is deprecated and will be removed in a future release. "
+ "Build a Symphony-style mapping reference instead.",
+ DeprecationWarning,
+ stacklevel=2,
+ )
+
+ if missing_feature_policy is None:
+ missing_feature_policy = "intersection" if exclude_missing else "zero"
+ if exclude_missing:
+ warnings.warn(
+ "exclude_missing is deprecated; use "
+ "missing_feature_policy='intersection' instead.",
+ DeprecationWarning,
+ stacklevel=2,
+ )
+ if missing_feature_policy not in {"zero", "intersection", "error"}:
raise ValueError(
- f"ERROR: `target_feat_key` cannot be sample as `feat_key`: {feat_key}"
+ "missing_feature_policy must be one of 'zero', 'intersection', or 'error'"
+ )
+ if exclude_missing and missing_feature_policy != "intersection":
+ raise ValueError(
+ "exclude_missing=True is only compatible with missing_feature_policy='intersection'"
+ )
+ if run_coral and missing_feature_policy == "intersection":
+ raise ValueError(
+ "CORAL does not support intersection-only feature mapping. "
+ "Use zero or error feature handling."
)
feat_idx = align_features(
@@ -125,37 +809,132 @@ def run_mapping(
filter_null,
exclude_missing,
self.nthreads,
+ missing_feature_policy,
)
logger.debug(f"{len(feat_idx)} features being used for mapping")
- if np.all(
- source_assay.feats.active_index(cell_key + "__" + feat_key) == feat_idx
+ source_feature_indices = source_assay.feats.active_index(
+ f"{cell_key}__{feat_key}" if feat_key != "I" else "I"
+ )
+ full_feature_overlap = len(source_feature_indices) == len(
+ feat_idx
+ ) and np.array_equal(source_feature_indices, feat_idx)
+ ann_feat_key = feat_key
+ if (
+ ann_feat_key == feat_key
+ and ann_index_fetcher is None
+ and ann_index_saver is None
+ and self._has_ann_stream_cache(
+ from_assay,
+ cell_key,
+ ann_feat_key,
+ feat_scaling=feat_scaling,
+ )
):
- ann_feat_key = feat_key
+ ann_obj: AnnStream | None = self._load_ann_stream(
+ from_assay,
+ cell_key,
+ ann_feat_key,
+ feat_scaling=feat_scaling,
+ )
else:
- ann_feat_key = f"{feat_key}_common_{target_name}"
- a = np.zeros(source_assay.feats.N).astype(bool)
- a[feat_idx] = True
- source_assay.feats.insert(
- cell_key + "__" + ann_feat_key, a, fill_value=False, overwrite=True
+ previous_ann_path: str | None = None
+ reference_normed_path = f"{from_assay}/normed__{cell_key}__{ann_feat_key}"
+ reference_reduction_path: str | None = None
+ if reference_normed_path in self.zw:
+ reference_normed_group = as_zarr_group(
+ self.zw[reference_normed_path], name=reference_normed_path
+ )
+ reference_reduction_path = cast(
+ str | None,
+ reference_normed_group.attrs.get("latest_reduction"),
+ )
+ if (
+ reference_reduction_path is not None
+ and reference_reduction_path in self.zw
+ ):
+ reference_reduction_group = as_zarr_group(
+ self.zw[reference_reduction_path],
+ name=reference_reduction_path,
+ )
+ previous_ann_path = cast(
+ str | None,
+ reference_reduction_group.attrs.get("latest_ann"),
+ )
+ ann_obj = self.make_graph(
+ from_assay=from_assay,
+ cell_key=cell_key,
+ feat_key=ann_feat_key,
+ return_ann_object=True,
+ update_keys=False,
+ feat_scaling=feat_scaling,
+ ann_index_fetcher=ann_index_fetcher,
+ ann_index_saver=ann_index_saver,
)
- if run_coral:
- feat_scaling = False
- ann_obj = self.make_graph(
- from_assay=from_assay,
- cell_key=cell_key,
- feat_key=ann_feat_key,
- return_ann_object=True,
- update_keys=False,
- feat_scaling=feat_scaling,
- ann_index_fetcher=ann_index_fetcher,
- ann_index_saver=ann_index_saver,
- )
+ if (
+ previous_ann_path is not None
+ and reference_reduction_path is not None
+ and ann_obj is not None
+ and ann_obj.annPath != previous_ann_path
+ ):
+ as_zarr_group(
+ self.zw[reference_reduction_path],
+ name=reference_reduction_path,
+ ).attrs["latest_ann"] = previous_ann_path
+ if ann_obj is None:
+ raise ValueError("ERROR: AnnStream could not be created for mapping")
+ if ann_obj.harmonize:
+ raise ValueError(
+ "This harmonized reference predates the Symphony-style mapping artifact. "
+ "Rebuild it with build_mapping_reference before mapping a query."
+ )
+ if missing_feature_policy == "intersection":
+ if not full_feature_overlap:
+ if exclude_missing:
+ compatibility_feat_key = f"{feat_key}_common_{target_name}"
+ feat_mask = np.zeros(source_assay.feats.N, dtype=bool)
+ feat_mask[feat_idx] = True
+ source_assay.feats.insert(
+ f"{cell_key}__{compatibility_feat_key}",
+ feat_mask,
+ fill_value=False,
+ overwrite=True,
+ )
+ self.make_graph(
+ from_assay=from_assay,
+ cell_key=cell_key,
+ feat_key=compatibility_feat_key,
+ dims=min(
+ int(
+ ann_obj.dims
+ if ann_obj.dims is not None
+ else len(feat_idx)
+ ),
+ max(len(feat_idx) - 1, 1),
+ ),
+ k=ann_obj.k,
+ return_ann_object=False,
+ update_keys=False,
+ feat_scaling=feat_scaling,
+ ann_index_fetcher=ann_index_fetcher,
+ ann_index_saver=ann_index_saver,
+ )
+ ann_obj = self._build_intersection_ann(
+ ann_obj,
+ source_assay,
+ cell_key,
+ feat_key,
+ feat_idx,
+ ann_index_saver,
+ )
if save_k > ann_obj.k:
logger.warning(f"`save_k` was decreased to {ann_obj.k}")
save_k = ann_obj.k
- target_data = daskarr.from_zarr(
- target_assay.z[f"normed__{target_cell_key}__{target_feat_key}/data"],
- inline_array=True,
+ target_data = ChunkedArray(
+ as_zarr_array(
+ target_assay.z[f"normed__{target_cell_key}__{target_feat_key}/data"],
+ name=f"normed__{target_cell_key}__{target_feat_key}/data",
+ ),
+ nthreads=self.nthreads,
)
if run_coral is True:
# Reversing coral here to correct target data
@@ -167,58 +946,433 @@ def run_mapping(
target_cell_key,
self.nthreads,
)
- target_data = daskarr.from_zarr(
- target_assay.z[
- f"normed__{target_cell_key}__{target_feat_key}/data_coral"
- ],
- inline_array=True,
- )
- if ann_obj.method == "pca" and run_coral is False:
- if ref_mu is False:
- mu = show_dask_progress(
- target_data.mean(axis=0),
- "Calculating mean of target norm. data",
- self.nthreads,
- )
- ann_obj.mu = clean_array(mu)
- if ref_sigma is False:
- sigma = show_dask_progress(
- target_data.std(axis=0),
- "Calculating std. dev. of target norm. data",
- self.nthreads,
- )
- ann_obj.sigma = clean_array(sigma, 1)
+ target_data = ChunkedArray(
+ as_zarr_array(
+ target_assay.z[
+ f"normed__{target_cell_key}__{target_feat_key}/data_coral"
+ ],
+ name=(f"normed__{target_cell_key}__{target_feat_key}/data_coral"),
+ ),
+ nthreads=self.nthreads,
+ )
if "projections" not in source_assay.z:
source_assay.z.create_group("projections")
- store = source_assay.z["projections"].create_group(target_name, overwrite=True)
- nc, nk = target_assay.cells.fetch_all("I").sum(), save_k
+ projections = as_zarr_group(source_assay.z["projections"], name="projections")
+ store = projections.create_group(target_name, overwrite=True)
+ store.attrs["schemaVersion"] = self._PROJECTION_SCHEMA_VERSION
+ store.attrs["complete"] = False
+ nc = target_assay.cells.active_index(target_cell_key).shape[0]
+ nk = save_k
zi = create_zarr_dataset(store, "indices", (batch_size,), "u8", (nc, nk))
zd = create_zarr_dataset(store, "distances", (batch_size,), "f8", (nc, nk))
+ feature_index_store = create_zarr_dataset(
+ store,
+ "referenceFeatureIndices",
+ (min(max(len(feat_idx), 1), 100_000),),
+ "i8",
+ feat_idx.shape,
+ )
+ feature_index_store[:] = feat_idx
+ correction_method = "coral" if run_coral else "none"
+ if missing_feature_policy == "intersection":
+ correction_method = "intersection"
+ for key, value in self._projection_provenance(
+ source_assay,
+ target_assay,
+ from_assay,
+ target_name,
+ cell_key,
+ feat_key,
+ target_cell_key,
+ target_feat_key,
+ feat_idx,
+ correction_method,
+ ann_obj,
+ ).items():
+ store.attrs[key] = value
+ store.attrs["saveK"] = int(save_k)
entry_start = 0
- for i in tqdmbar(
- target_data.blocks,
- desc=f"Mapping cells from {target_name}",
- total=target_data.numblocks[0],
- ):
- a: np.ndarray = controlled_compute(i, self.nthreads)
- ki, kd = ann_obj.transform_ann(ann_obj.reducer(a), k=save_k)
- entry_end = entry_start + len(ki)
- zi[entry_start:entry_end, :] = ki
- zd[entry_start:entry_end, :] = kd
- entry_start = entry_end
+ try:
+ for i in tqdmbar(
+ target_data.blocks,
+ desc=f"Mapping cells from {target_name}",
+ total=target_data.numblocks[0],
+ ):
+ block: NDArray[Any] = controlled_compute(i, self.nthreads)
+ knn_query = ann_obj.transform_ann(
+ ann_obj.transform_query(block), k=save_k
+ )
+ ki, kd = knn_query[0], knn_query[1]
+ entry_end = entry_start + len(ki)
+ zi[entry_start:entry_end, :] = ki
+ zd[entry_start:entry_end, :] = kd
+ entry_start = entry_end
+ if entry_start != nc:
+ raise RuntimeError(
+ f"Mapped {entry_start} target cells but expected {nc}"
+ )
+ store.attrs["complete"] = True
+ except Exception:
+ store.attrs["complete"] = False
+ raise
return None
+ def _map_with_mapping_reference(
+ self,
+ reference: MappingReference,
+ target_assay: Assay,
+ target_name: str,
+ target_feat_key: str,
+ target_cell_key: str,
+ save_k: int,
+ query_batches: pd.DataFrame | None,
+ correction_method: str,
+ missing_feature_policy: str,
+ result_store: zarr.Group | None = None,
+ ) -> MappingResult:
+ from ..mapping_utils import align_features, array_hash
+ from ..symphony import (
+ accumulate_sufficient_statistics,
+ apply_query_correction,
+ initialize_sufficient_statistics,
+ project_pca,
+ soft_cluster_assignments,
+ solve_query_correction,
+ zero_norm_rows,
+ )
+
+ if type(target_assay) is not type(self._get_assay(reference.assay_name)):
+ raise TypeError("Reference and query assays must have the same type")
+ if correction_method != "symphony":
+ raise ValueError(
+ "Harmonized mapping references require correction_method='symphony'"
+ )
+ if missing_feature_policy not in {"reference_mean", "zero", "error"}:
+ raise ValueError(
+ "Mapping references support reference_mean, zero, or error feature handling"
+ )
+ source_assay = self._get_assay(reference.assay_name)
+ if target_assay.sf != source_assay.sf and isinstance(target_assay, RNAassay):
+ logger.info(
+ f"Resetting target assay's size factor from {target_assay.sf} to {source_assay.sf}"
+ )
+ target_assay.sf = source_assay.sf
+ self._guard_mapping_target_path(
+ source_assay,
+ target_assay,
+ reference.cell_key,
+ reference.feature_key,
+ target_cell_key,
+ target_feat_key,
+ )
+ feature_indices = align_features(
+ source_assay,
+ target_assay,
+ reference.cell_key,
+ reference.feature_key,
+ target_feat_key,
+ target_cell_key,
+ filter_null=False,
+ exclude_missing=False,
+ nthreads=self.nthreads,
+ missing_feature_policy=missing_feature_policy,
+ missing_feature_values=reference.model.feature_means,
+ )
+ source_feature_indices = source_assay.feats.active_index(
+ f"{reference.cell_key}__{reference.feature_key}"
+ if reference.feature_key != "I"
+ else "I"
+ )
+ if not np.array_equal(feature_indices, source_feature_indices):
+ raise ValueError(
+ "The mapping reference requires its complete reference feature set"
+ )
+ source_feature_ids = source_assay.feats.fetch(
+ "ids",
+ key=(
+ f"{reference.cell_key}__{reference.feature_key}"
+ if reference.feature_key != "I"
+ else "I"
+ ),
+ )
+ if not np.array_equal(source_feature_ids, reference.feature_ids):
+ raise ValueError(
+ "Reference feature identifiers no longer match the immutable artifact"
+ )
+ target_data = ChunkedArray(
+ as_zarr_array(
+ target_assay.z[f"normed__{target_cell_key}__{target_feat_key}/data"],
+ name=f"normed__{target_cell_key}__{target_feat_key}/data",
+ ),
+ nthreads=self.nthreads,
+ )
+ n_cells = target_data.shape[0]
+ batch_codes, n_batches = self._query_batch_codes(query_batches, n_cells)
+ query_batch_columns = (
+ [str(column) for column in query_batches.columns]
+ if query_batches is not None
+ else []
+ )
+ query_batch_hash = array_hash(batch_codes)
+ counts, sums = initialize_sufficient_statistics(n_batches, reference.model)
+ entry_start = 0
+ zero_norm_count = 0
+ for block in target_data.blocks:
+ values = controlled_compute(block, self.nthreads)
+ coordinates = project_pca(values, reference.model)
+ assignments = soft_cluster_assignments(coordinates, reference.model)
+ uninformative_rows = zero_norm_rows(coordinates)
+ zero_norm_count += int(uninformative_rows.sum())
+ entry_end = entry_start + len(values)
+ if not np.all(uninformative_rows):
+ informative_rows = ~uninformative_rows
+ accumulate_sufficient_statistics(
+ counts,
+ sums,
+ coordinates[informative_rows],
+ assignments[informative_rows],
+ batch_codes[entry_start:entry_end][informative_rows],
+ )
+ entry_start = entry_end
+ if entry_start != n_cells:
+ raise RuntimeError(
+ f"Read {entry_start} query cells but expected {n_cells} during correction"
+ )
+ correction = solve_query_correction(counts, sums, reference.model)
+ if reference.ann_path not in self.zw:
+ raise ValueError(
+ "The mapping reference ANN index is missing. Rebuild the reference."
+ )
+ reference_ann_group = as_zarr_group(
+ self.zw[reference.ann_path], name=reference.ann_path
+ )
+ if not bool(reference_ann_group.attrs.get("featureScaling", True)):
+ raise ValueError(
+ "The mapping reference ANN was built without feature scaling "
+ "and cannot use reference-scaled query projection."
+ )
+ reference_knn_path = cast(str, reference_ann_group.attrs.get("latest_knn"))
+ if not reference_knn_path or reference_knn_path not in self.zw:
+ raise ValueError(
+ "The mapping reference KNN metadata is missing. Rebuild the reference."
+ )
+ ann_obj = self._load_ann_stream(
+ reference.assay_name,
+ reference.cell_key,
+ reference.feature_key,
+ feat_scaling=True,
+ knn_loc=reference_knn_path,
+ )
+ if not ann_obj.harmonize:
+ raise RuntimeError("Mapping reference ANN index is not harmonized")
+ if save_k > ann_obj.k:
+ logger.warning(f"`save_k` was decreased to {ann_obj.k}")
+ save_k = ann_obj.k
+ write_projection = self.zarr_mode == "r+" or result_store is not None
+ store: zarr.Group | None = result_store
+ projection_path = ""
+ feature_coverage = float(
+ np.isin(
+ reference.feature_ids,
+ target_assay.feats.fetch_all("ids"),
+ ).sum()
+ / len(reference.feature_ids)
+ )
+ if write_projection:
+ if store is None:
+ if "projections" not in source_assay.z:
+ source_assay.z.create_group("projections")
+ projections = as_zarr_group(
+ source_assay.z["projections"], name="projections"
+ )
+ store = projections.create_group(target_name, overwrite=True)
+ projection_path = f"{reference.assay_name}/projections/{target_name}"
+ else:
+ projection_path = getattr(store, "path", "")
+ store.attrs["schemaVersion"] = self._PROJECTION_SCHEMA_VERSION
+ store.attrs["complete"] = False
+ for key, value in self._projection_provenance(
+ source_assay,
+ target_assay,
+ reference.assay_name,
+ target_name,
+ reference.cell_key,
+ reference.feature_key,
+ target_cell_key,
+ target_feat_key,
+ feature_indices,
+ "symphony",
+ ann_obj,
+ ).items():
+ store.attrs[key] = value
+ store.attrs["saveK"] = int(save_k)
+ store.attrs["mappingReferencePath"] = reference.artifact_path
+ store.attrs["queryBatchCount"] = int(n_batches)
+ store.attrs["queryBatchColumns"] = query_batch_columns
+ store.attrs["queryBatchHash"] = query_batch_hash
+ feature_index_store = create_zarr_dataset(
+ store,
+ "referenceFeatureIndices",
+ (min(max(len(feature_indices), 1), 100_000),),
+ "i8",
+ feature_indices.shape,
+ )
+ feature_index_store[:] = feature_indices
+ row_chunk = min(max(int(target_data.chunksize[0]), 1), max(n_cells, 1))
+ indices: Any = create_zarr_dataset(
+ store, "indices", (row_chunk, save_k), "u8", (n_cells, save_k)
+ )
+ distances: Any = create_zarr_dataset(
+ store, "distances", (row_chunk, save_k), "f8", (n_cells, save_k)
+ )
+ uncorrected: Any = create_zarr_dataset(
+ store,
+ "uncorrectedLatent",
+ (row_chunk, reference.model.n_dims),
+ "f8",
+ (n_cells, reference.model.n_dims),
+ )
+ corrected: Any = create_zarr_dataset(
+ store,
+ "correctedLatent",
+ (row_chunk, reference.model.n_dims),
+ "f8",
+ (n_cells, reference.model.n_dims),
+ )
+ uninformative: Any = create_zarr_dataset(
+ store,
+ "uninformative",
+ (row_chunk,),
+ "bool",
+ (n_cells,),
+ )
+ else:
+ indices = np.empty((n_cells, save_k), dtype=np.uint64)
+ distances = np.empty((n_cells, save_k), dtype=np.float64)
+ uncorrected = np.empty((n_cells, reference.model.n_dims), dtype=np.float64)
+ corrected = np.empty((n_cells, reference.model.n_dims), dtype=np.float64)
+ uninformative = np.empty(n_cells, dtype=bool)
+ entry_start = 0
+ try:
+ for block in tqdmbar(
+ target_data.blocks,
+ desc=f"Mapping cells from {target_name} with Symphony-style correction",
+ total=target_data.numblocks[0],
+ ):
+ values = controlled_compute(block, self.nthreads)
+ coordinates = project_pca(values, reference.model)
+ assignments = soft_cluster_assignments(coordinates, reference.model)
+ uninformative_rows = zero_norm_rows(coordinates)
+ entry_end = entry_start + len(values)
+ corrected_coordinates = apply_query_correction(
+ coordinates,
+ assignments,
+ batch_codes[entry_start:entry_end],
+ reference.model,
+ correction,
+ )
+ corrected_coordinates[uninformative_rows] = coordinates[
+ uninformative_rows
+ ]
+ knn_query = ann_obj.transform_ann(corrected_coordinates, k=save_k)
+ neighbor_indices, neighbor_distances = cast(
+ tuple[np.ndarray, np.ndarray], knn_query
+ )
+ indices[entry_start:entry_end] = neighbor_indices
+ distances[entry_start:entry_end] = neighbor_distances
+ uncorrected[entry_start:entry_end] = coordinates
+ corrected[entry_start:entry_end] = corrected_coordinates
+ uninformative[entry_start:entry_end] = uninformative_rows
+ entry_start = entry_end
+ if entry_start != n_cells:
+ raise RuntimeError(
+ f"Mapped {entry_start} query cells but expected {n_cells}"
+ )
+ if store is not None:
+ store.attrs["complete"] = True
+ except Exception:
+ if store is not None:
+ store.attrs["complete"] = False
+ raise
+ return MappingResult(
+ projection_path=projection_path,
+ n_cells=n_cells,
+ correction_method="symphony",
+ diagnostics={
+ "featureCoverage": feature_coverage,
+ "queryBatchCount": float(n_batches),
+ "zeroNormCellCount": float(zero_norm_count),
+ "algorithmVariant": SYMPHONY_STYLE_VARIANT,
+ },
+ indices=None if store is not None else indices,
+ distances=None if store is not None else distances,
+ uncorrected_latent=None if store is not None else uncorrected,
+ corrected_latent=None if store is not None else corrected,
+ uninformative=None if store is not None else uninformative,
+ )
+
+ @staticmethod
+ def _query_batch_codes(
+ query_batches: pd.DataFrame | None, n_cells: int
+ ) -> tuple[np.ndarray, int]:
+ if query_batches is None:
+ return np.zeros(n_cells, dtype=np.int64), 1
+ if len(query_batches) != n_cells:
+ raise ValueError("query_batches must have one row per target cell")
+ if query_batches.shape[1] == 0:
+ raise ValueError("query_batches must include at least one column")
+ if query_batches.columns.duplicated().any():
+ raise ValueError("query_batches column names must be unique")
+ if query_batches.isna().any().any():
+ raise ValueError("query_batches cannot contain missing values")
+ labels = query_batches.astype(str).agg("\x1f".join, axis=1)
+ codes, _ = pd.factorize(labels, sort=True)
+ return np.asarray(codes, dtype=np.int64), int(codes.max()) + 1
+
+ @staticmethod
+ def _label_vote_decision(
+ reference_labels: np.ndarray,
+ neighbors: np.ndarray,
+ weights: np.ndarray,
+ threshold_fraction: float,
+ na_val: str,
+ force_unknown: bool = False,
+ ) -> tuple[Any, float, float, float, bool, dict[Any, float]]:
+ votes: dict[Any, float] = {}
+ for neighbor, weight in zip(neighbors, weights):
+ label = reference_labels[neighbor]
+ votes[label] = votes.get(label, 0.0) + float(weight)
+ total = float(sum(votes.values()))
+ if total <= 0 or not votes:
+ return na_val, 0.0, 0.0, 0.0, True, {}
+ votes = {label: value / total for label, value in votes.items()}
+ ordered = sorted(votes.items(), key=lambda item: item[1], reverse=True)
+ top_vote = ordered[0][1]
+ second_vote = ordered[1][1] if len(ordered) > 1 else 0.0
+ winners = [label for label, vote in ordered if np.isclose(vote, top_vote)]
+ is_unknown = force_unknown or top_vote < threshold_fraction or len(winners) != 1
+ entropy = -sum(value * np.log(value) for _, value in ordered if value > 0)
+ prediction = na_val if is_unknown else winners[0]
+ return (
+ prediction,
+ top_vote,
+ float(entropy),
+ top_vote - second_vote,
+ is_unknown,
+ votes,
+ )
+
def get_mapping_score(
self,
target_name: str,
- target_groups: Optional[np.ndarray] = None,
- from_assay: Optional[str] = None,
- cell_key: str = "I",
+ target_groups: np.ndarray | None = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
log_transform: bool = True,
multiplier: float = 1000,
weighted: bool = True,
fixed_weight: float = 0.1,
- ) -> Generator[Tuple[str, np.ndarray], None, None]:
+ ) -> Generator[tuple[str, np.ndarray], None, None]:
"""Yields the mapping scores that were a result of a mapping.
Mapping scores are an indication of degree of similarity of reference cells in the graph to the target cells.
@@ -242,20 +1396,14 @@ def get_mapping_score(
Yields:
A tuple of group name and mapping score of reference cells for that target group.
"""
- if from_assay is None:
- from_assay = self._defaultAssay
- store_loc = f"{from_assay}/projections/{target_name}"
- if store_loc not in self.zw:
- raise KeyError(
- f"ERROR: Projections have not been computed for {target_name} in th latest graph. Please"
- f" run `run_mapping` or update latest_graph by running `make_graph` with desired parameters"
- )
- store = self.zw[store_loc]
-
- indices = store["indices"][:]
- dists = store["distances"][:]
- # TODO: add more robust options for distance calculation here
- dists = 1 / (np.log1p(dists) + 1)
+ from_assay, cell_key, feat_key = self._get_latest_keys(
+ from_assay, cell_key, None
+ )
+ store = self._load_complete_projection(
+ target_name, from_assay, cell_key, feat_key
+ )
+ indices = as_zarr_array(store["indices"], name="indices")
+ distances = as_zarr_array(store["distances"], name="distances")
n_cells, n_k = indices.shape
if target_groups is not None:
@@ -268,18 +1416,33 @@ def get_mapping_score(
else:
groups = pd.Series(np.zeros(n_cells))
- ref_n_cells = self.cells.fetch_all(cell_key).sum()
+ ref_n_cells = self.cells.active_index(cell_key).shape[0]
for group in sorted(groups.unique()):
- coi = {x: None for x in groups[groups == group].index.values}
ms = np.zeros(ref_n_cells)
- for n, i, j in zip(range(len(indices)), indices, dists):
- if n in coi:
- for x, y in zip(i, j):
- if weighted:
- ms[x] += y
- else:
- ms[x] += fixed_weight
- ms = multiplier * ms / (len(coi) * n_k)
+ group_count = 0
+ block_size = self._projection_block_size(indices)
+ for start in range(0, n_cells, block_size):
+ stop = min(start + block_size, n_cells)
+ block_groups = groups.iloc[start:stop].to_numpy()
+ mask = block_groups == group
+ if not mask.any():
+ continue
+ block_indices = np.asarray(indices[start:stop])[mask]
+ if weighted:
+ from ..mapping_utils import distance_weights
+
+ block_weights = distance_weights(
+ np.asarray(distances[start:stop])[mask]
+ )
+ else:
+ block_weights = np.full(
+ block_indices.shape, fixed_weight, dtype=np.float64
+ )
+ np.add.at(ms, block_indices.reshape(-1), block_weights.reshape(-1))
+ group_count += int(mask.sum())
+ if group_count == 0:
+ continue
+ ms = multiplier * ms / (group_count * n_k)
if log_transform:
ms = np.log1p(ms)
yield group, ms
@@ -287,11 +1450,11 @@ def get_mapping_score(
def get_target_classes(
self,
target_name: str,
- from_assay: Optional[str] = None,
- cell_key: str = "I",
- reference_class_group: Optional[str] = None,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ reference_class_group: str | None = None,
threshold_fraction: float = 0.5,
- target_subset: Optional[List[int]] = None,
+ target_subset: list[int] | None = None,
na_val: str = "NA",
) -> pd.Series:
"""Perform classification of target cells using a reference group.
@@ -312,14 +1475,9 @@ def get_target_classes(
Returns: A pandas Series containing predicted class for each cell in the projected sample (`target_name`).
"""
- if from_assay is None:
- from_assay = self._defaultAssay
- store_loc = f"{from_assay}/projections/{target_name}"
- if store_loc not in self.zw:
- raise KeyError(
- f"ERROR: Projections have not been computed for {target_name} in th latest graph. Please"
- f" run `run_mapping` or update latest_graph by running `make_graph` with desired parameters"
- )
+ from_assay, cell_key, feat_key = self._get_latest_keys(
+ from_assay, cell_key, None
+ )
if reference_class_group is None:
raise ValueError(
"ERROR: A value is required for the parameter `reference_class_group`. "
@@ -331,47 +1489,307 @@ def get_target_classes(
raise ValueError(
"ERROR: `threshold_fraction` should have a value between 0 and 1"
)
+ target_subset_set: dict[int, None] | None = None
if target_subset is not None:
- if type(target_subset) != list:
+ if not isinstance(target_subset, list):
raise TypeError("ERROR: `target_subset` should be type")
- target_subset = {x: None for x in target_subset}
-
- store = self.zw[store_loc]
- indices = store["indices"][:]
- dists = store["distances"][:]
+ target_subset_set = {x: None for x in target_subset}
- preds = []
- weights = 1 - (dists / dists.max(axis=1).reshape(-1, 1))
- for n in range(indices.shape[0]):
- if target_subset is not None and n not in target_subset:
+ store = self._load_complete_projection(
+ target_name, from_assay, cell_key, feat_key
+ )
+ preds: list[Any] = []
+ prediction_indices: list[int] = []
+ for row in self._iter_projection_neighbor_rows(store):
+ n, neighbors, weights, _, force_unknown = row
+ if target_subset_set is not None and n not in target_subset_set:
continue
- wd = {}
- for i, j in zip(indices[n, :-1], weights[n, :-1]):
- k = ref_groups[i]
- if k not in wd:
- wd[k] = 0
- wd[k] += j
- temp = na_val
- s = weights[n, :-1].sum()
- for i, j in wd.items():
- if j / s > threshold_fraction:
- if temp == na_val:
- temp = i
- else:
- temp = na_val
- break
- preds.append(temp)
- return pd.Series(preds)
+ prediction, _, _, _, _, _ = self._label_vote_decision(
+ ref_groups,
+ neighbors,
+ weights,
+ threshold_fraction,
+ na_val,
+ force_unknown=force_unknown,
+ )
+ preds.append(prediction)
+ prediction_indices.append(n)
+ return pd.Series(preds, index=prediction_indices)
- def load_unified_graph(
+ def get_target_label_evidence(
self,
+ target_name: str,
+ reference_class_group: str,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ threshold_fraction: float = 0.5,
+ na_val: str = "NA",
+ max_distance: float | None = None,
+ calibration_nonconformity: np.ndarray | None = None,
+ conformal_alpha: float = 0.1,
+ ) -> pd.DataFrame:
+ """Return neighbor-vote evidence, novelty context, and unknown assignments.
+
+ ``calibration_nonconformity`` optionally adds split-conformal prediction
+ sets. Its calibration rows must be exchangeable with future queries.
+ """
+ if not 0 <= threshold_fraction <= 1:
+ raise ValueError("threshold_fraction must be between zero and one")
+ from_assay, cell_key, feat_key = self._get_latest_keys(
+ from_assay, cell_key, None
+ )
+ if reference_class_group not in self.cells.columns:
+ raise KeyError(
+ f"Reference label column {reference_class_group!r} was not found"
+ )
+ store = self._load_complete_projection(
+ target_name, from_assay, cell_key, feat_key
+ )
+ reference_labels = self.cells.fetch(reference_class_group, key=cell_key)
+ class_labels = np.asarray(pd.unique(reference_labels), dtype=object)
+ class_positions = {
+ label: position for position, label in enumerate(class_labels)
+ }
+
+ predictions: list[Any] = []
+ vote_fraction: list[float] = []
+ vote_entropy: list[float] = []
+ top_two_margin: list[float] = []
+ best_distances: list[float] = []
+ label_scores: list[np.ndarray] = []
+ for row in self._iter_projection_neighbor_rows(store):
+ _, neighbors, weights, row_distances, force_unknown = row
+ (
+ prediction,
+ top_vote,
+ entropy,
+ margin,
+ _,
+ votes,
+ ) = self._label_vote_decision(
+ reference_labels,
+ neighbors,
+ weights,
+ threshold_fraction,
+ na_val,
+ force_unknown=force_unknown,
+ )
+ if max_distance is not None and row_distances[0] > max_distance:
+ prediction = na_val
+ predictions.append(prediction)
+ vote_fraction.append(top_vote)
+ vote_entropy.append(entropy)
+ top_two_margin.append(margin)
+ best_distances.append(float(row_distances[0]))
+ score_row = np.zeros(len(class_labels), dtype=np.float64)
+ for label, score in votes.items():
+ score_row[class_positions[label]] = score
+ label_scores.append(score_row)
+
+ best = np.asarray(best_distances)
+ distance_quantiles, distance_values = self._reference_distance_summary(
+ store, from_assay, cell_key, feat_key
+ )
+ unique_distance_values = np.unique(distance_values)
+ right_indices = (
+ np.searchsorted(distance_values, unique_distance_values, side="right") - 1
+ )
+ unique_distance_quantiles = distance_quantiles[right_indices]
+ distance_percentile = np.interp(
+ best,
+ unique_distance_values,
+ unique_distance_quantiles,
+ left=0.0,
+ right=1.0,
+ )
+ feature_coverage_value = store.attrs.get("featureCoverage", 1.0)
+ if not isinstance(feature_coverage_value, int | float):
+ raise RuntimeError(
+ "Projection provenance is missing numeric feature coverage"
+ )
+ feature_coverage = float(feature_coverage_value)
+ result = pd.DataFrame(
+ {
+ "label": predictions,
+ "voteFraction": vote_fraction,
+ "voteEntropy": vote_entropy,
+ "topTwoMargin": top_two_margin,
+ "featureCoverage": feature_coverage,
+ "referenceDistancePercentile": distance_percentile,
+ "isUnknown": np.asarray(predictions) == na_val,
+ }
+ )
+ if calibration_nonconformity is not None:
+ from ..mapping_utils import conformal_prediction_sets
+
+ prediction_masks = conformal_prediction_sets(
+ np.vstack(label_scores),
+ calibration_nonconformity,
+ alpha=conformal_alpha,
+ )
+ result["predictionSet"] = [
+ tuple(class_labels[mask].tolist()) for mask in prediction_masks
+ ]
+ return result
+
+ def _reference_distance_summary(
+ self,
+ projection: zarr.Group,
from_assay: str,
cell_key: str,
feat_key: str,
- target_names: List[str],
+ ) -> tuple[np.ndarray, np.ndarray]:
+ from ..mapping_utils import _distance_quantile_summary
+
+ artifact_path = projection.attrs.get("mappingReferencePath")
+ if isinstance(artifact_path, str) and artifact_path in self.zw:
+ artifact = as_zarr_group(self.zw[artifact_path], name=artifact_path)
+ if (
+ "referenceDistanceQuantiles" in artifact
+ and "referenceDistanceValues" in artifact
+ ):
+ return (
+ np.asarray(
+ as_zarr_array(
+ artifact["referenceDistanceQuantiles"],
+ name="referenceDistanceQuantiles",
+ )[:]
+ ),
+ np.asarray(
+ as_zarr_array(
+ artifact["referenceDistanceValues"],
+ name="referenceDistanceValues",
+ )[:]
+ ),
+ )
+
+ ann_path = projection.attrs.get("annPath")
+ if not isinstance(ann_path, str) or ann_path not in self.zw:
+ stored_assay = projection.attrs.get("assay")
+ stored_cell_key = projection.attrs.get("cellKey")
+ stored_feature_key = projection.attrs.get("featureKey")
+ if isinstance(stored_assay, str):
+ from_assay = stored_assay
+ if isinstance(stored_cell_key, str):
+ cell_key = stored_cell_key
+ if isinstance(stored_feature_key, str):
+ feat_key = stored_feature_key
+ normed_path = f"{from_assay}/normed__{cell_key}__{feat_key}"
+ normed = as_zarr_group(self.zw[normed_path], name=normed_path)
+ reduction_path = cast(str, normed.attrs["latest_reduction"])
+ reduction = as_zarr_group(self.zw[reduction_path], name=reduction_path)
+ ann_path = cast(str, reduction.attrs["latest_ann"])
+ ann = as_zarr_group(self.zw[ann_path], name=ann_path)
+ if "referenceDistanceQuantiles" in ann and "referenceDistanceValues" in ann:
+ return (
+ np.asarray(
+ as_zarr_array(
+ ann["referenceDistanceQuantiles"],
+ name="referenceDistanceQuantiles",
+ )[:]
+ ),
+ np.asarray(
+ as_zarr_array(
+ ann["referenceDistanceValues"],
+ name="referenceDistanceValues",
+ )[:]
+ ),
+ )
+ knn_path = cast(str, ann.attrs["latest_knn"])
+ knn = as_zarr_group(self.zw[knn_path], name=knn_path)
+ reference_distances = as_zarr_array(knn["distances"], name="referenceDistances")
+ return _distance_quantile_summary(reference_distances)
+
+ @staticmethod
+ def calibrate_label_transfer_threshold(
+ vote_fractions: np.ndarray,
+ correct: np.ndarray,
+ target_coverage: float = 0.9,
+ ) -> dict[str, float]:
+ """Choose a vote threshold on held-out, donor-level validation data."""
+ fractions = np.asarray(vote_fractions, dtype=np.float64)
+ correct = np.asarray(correct, dtype=bool)
+ if fractions.ndim != 1 or correct.shape != fractions.shape:
+ raise ValueError("vote_fractions and correct must be matching vectors")
+ if not 0 < target_coverage <= 1:
+ raise ValueError("target_coverage must be in (0, 1]")
+ valid = fractions[correct]
+ if valid.size == 0:
+ raise ValueError("At least one correct held-out prediction is required")
+ threshold = float(np.quantile(valid, 1 - target_coverage))
+ selected = fractions >= threshold
+ accuracy = float(correct[selected].mean()) if selected.any() else 0.0
+ return {
+ "voteThreshold": threshold,
+ "validationCoverage": float(selected.mean()),
+ "validationAccuracy": accuracy,
+ }
+
+ def project_mapping_layout(
+ self,
+ target_name: str,
+ reference_layout_key: str,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ label: str | None = None,
+ ) -> str | np.ndarray:
+ """Place query cells into an unchanged reference layout by neighbor weighting.
+
+ Writable stores return the saved Zarr path. Read-only stores return the
+ coordinates as a NumPy array.
+ """
+ from_assay, cell_key, feat_key = self._get_latest_keys(
+ from_assay, cell_key, None
+ )
+ store = self._load_complete_projection(
+ target_name, from_assay, cell_key, feat_key
+ )
+ if label is None:
+ label = f"fixed_{reference_layout_key}"
+ reference_layout = np.column_stack(
+ (
+ self.cells.fetch(f"{reference_layout_key}1", key=cell_key),
+ self.cells.fetch(f"{reference_layout_key}2", key=cell_key),
+ )
+ )
+ indices = as_zarr_array(store["indices"], name="indices")
+ distances = as_zarr_array(store["distances"], name="distances")
+ persist_layout = self.zarr_mode == "r+"
+ if persist_layout:
+ layout: Any = create_zarr_dataset(
+ store,
+ label,
+ (self._projection_block_size(indices), 2),
+ "f8",
+ (indices.shape[0], 2),
+ )
+ else:
+ layout = np.empty((indices.shape[0], 2), dtype=np.float64)
+ from ..mapping_utils import distance_weights
+
+ for start in range(0, indices.shape[0], self._projection_block_size(indices)):
+ stop = min(start + self._projection_block_size(indices), indices.shape[0])
+ block_indices = np.asarray(indices[start:stop])
+ block_weights = distance_weights(np.asarray(distances[start:stop]))
+ layout[start:stop] = np.einsum(
+ "nk,nkd->nd", block_weights, reference_layout[block_indices]
+ )
+ if persist_layout:
+ layout.attrs["complete"] = True
+ layout.attrs["referenceLayoutKey"] = reference_layout_key
+ layout.attrs["projectionSchemaVersion"] = self._PROJECTION_SCHEMA_VERSION
+ return f"{from_assay}/projections/{target_name}/{label}"
+ return np.asarray(layout)
+
+ def load_unified_graph(
+ self,
+ from_assay: str | None,
+ cell_key: str | None,
+ feat_key: str | None,
+ target_names: list[str],
use_k: int,
target_weight: float,
- ) -> Tuple[List[int], csr_matrix]:
+ ) -> tuple[list[int], csr_matrix]:
"""This is similar to ``load_graph`` but includes projected cells and
their edges.
@@ -392,15 +1810,37 @@ def load_unified_graph(
if from_assay is None:
from_assay = self._defaultAssay
+ if cell_key is None:
+ cell_key = self._get_latest_cell_key(from_assay)
if feat_key is None:
feat_key = self._get_latest_feat_key(from_assay)
graph_loc = self._get_latest_graph_loc(from_assay, cell_key, feat_key)
- edges = self.zw[graph_loc].edges[:]
- weights = self.zw[graph_loc].weights[:]
- ref_n_cells = self.cells.fetch_all(cell_key).sum()
- store = self.zw[from_assay].projections
- pidx = np.vstack([store[x].indices[:, :use_k] for x in target_names])
- n_cells = [ref_n_cells] + [store[x].indices.shape[0] for x in target_names]
+ graph_group = as_zarr_group(self.zw[graph_loc], name=graph_loc)
+ edges = np.asarray(as_zarr_array(graph_group["edges"], name="edges")[:])
+ weights = np.asarray(as_zarr_array(graph_group["weights"], name="weights")[:])
+ ref_n_cells = self.cells.active_index(cell_key).shape[0]
+ projection_stores = [
+ self._load_complete_projection(target_name, from_assay, cell_key, feat_key)
+ for target_name in target_names
+ ]
+ pidx = np.vstack(
+ [
+ np.asarray(
+ as_zarr_array(
+ projection_store["indices"],
+ name="indices",
+ )[:, :use_k]
+ )
+ for projection_store in projection_stores
+ ]
+ )
+ n_cells = [ref_n_cells] + [
+ as_zarr_array(
+ projection_store["indices"],
+ name="indices",
+ ).shape[0]
+ for projection_store in projection_stores
+ ]
ne = []
nw = []
for n, i in enumerate(pidx):
@@ -440,11 +1880,14 @@ def _save_embedding(
cell_key: str,
label: str,
embedding: np.ndarray,
- n_cells: List[int],
- target_names: List[str],
+ n_cells: list[int],
+ target_names: list[str],
) -> None:
g = create_zarr_dataset(
- self.zw[from_assay].projections,
+ as_zarr_group(
+ as_zarr_group(self.zw[from_assay], name=from_assay)["projections"],
+ name="projections",
+ ),
label,
(1000, 2),
"float64",
@@ -466,10 +1909,10 @@ def _save_embedding(
def run_unified_umap(
self,
- target_names: List[str],
- from_assay: Optional[str] = None,
- cell_key: str = "I",
- feat_key: Optional[str] = None,
+ target_names: list[str],
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
use_k: int = 3,
target_weight: float = 0.1,
spread: float = 2.0,
@@ -482,7 +1925,7 @@ def run_unified_umap(
ini_embed_with: str = "kmeans",
label: str = "unified_UMAP",
parallel: bool = False,
- nthreads: Optional[int] = None,
+ nthreads: int | None = None,
) -> None:
"""Calculates the UMAP embedding for graph obtained using
``load_unified_graph``.
@@ -532,10 +1975,9 @@ def run_unified_umap(
from ..umap import fit_transform
from ..utils import get_log_level
- if from_assay is None:
- from_assay = self._defaultAssay
- if feat_key is None:
- feat_key = self._get_latest_feat_key(from_assay)
+ from_assay, cell_key, feat_key = self._get_latest_keys(
+ from_assay, cell_key, feat_key
+ )
n_cells, graph = self.load_unified_graph(
from_assay=from_assay,
cell_key=cell_key,
@@ -572,10 +2014,10 @@ def run_unified_umap(
def run_unified_tsne(
self,
- target_names: List[str],
- from_assay: Optional[str] = None,
- cell_key: str = "I",
- feat_key: Optional[str] = None,
+ target_names: list[str],
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
use_k: int = 3,
target_weight: float = 0.5,
lambda_scale: float = 1.0,
@@ -618,14 +2060,11 @@ def run_unified_tsne(
Returns:
"""
- from uuid import uuid4
- from ..knn_utils import export_knn_to_mtx
- from pathlib import Path
+ from ..knn_utils import run_sgtsne
- if from_assay is None:
- from_assay = self._defaultAssay
- if feat_key is None:
- feat_key = self._get_latest_feat_key(from_assay)
+ from_assay, cell_key, feat_key = self._get_latest_keys(
+ from_assay, cell_key, feat_key
+ )
n_cells, graph = self.load_unified_graph(
from_assay=from_assay,
cell_key=cell_key,
@@ -637,69 +2076,61 @@ def run_unified_tsne(
ini_embed = self._get_uni_ini_embed(
from_assay, cell_key, feat_key, graph, ini_embed_with, n_cells[0]
)
-
- uid = str(uuid4())
- ini_emb_fn = Path(temp_file_loc, f"{uid}.txt").resolve()
- with open(ini_emb_fn, "w") as h:
- h.write("\n".join(map(str, ini_embed.flatten())))
- del ini_embed
- knn_mtx_fn = Path(temp_file_loc, f"{uid}.mtx").resolve()
- export_knn_to_mtx(str(knn_mtx_fn), graph)
- out_fn = Path(temp_file_loc, f"{uid}_output.txt").resolve()
- cmd = (
- f"sgtsne -m {max_iter} -l {lambda_scale} -d {2} -e {early_iter} -p 1 -a {alpha}"
- f" -h {box_h} -i {ini_emb_fn} -o {out_fn} {knn_mtx_fn}"
- )
- if verbose:
- system_call(cmd)
- else:
- os.system(cmd)
- t = pd.read_csv(out_fn, header=None, sep=" ")[[0, 1]].values
- self._save_embedding(from_assay, cell_key, label, t, n_cells, target_names)
- for fn in [out_fn, knn_mtx_fn, ini_emb_fn]:
- Path.unlink(fn)
+ emb = run_sgtsne(
+ graph,
+ ini_embed,
+ tsne_dims=2,
+ max_iter=max_iter,
+ early_iter=early_iter,
+ alpha=alpha,
+ lambda_scale=lambda_scale,
+ box_h=box_h,
+ temp_file_loc=temp_file_loc,
+ verbose=verbose,
+ )
+ self._save_embedding(from_assay, cell_key, label, emb.T, n_cells, target_names)
return None
def plot_unified_layout(
self,
- from_assay: Optional[str] = None,
- layout_key: Optional[str] = None,
+ from_assay: str | None = None,
+ layout_key: str | None = None,
show_target_only: bool = False,
ref_name: str = "reference",
- target_groups: Optional[List[str]] = None,
+ target_groups: list[str] | None = None,
width: float = 6,
height: float = 6,
- cmap: Optional[str] = None,
- color_key: Optional[Dict] = None,
+ cmap: str | None = None,
+ color_key: dict[Any, Any] | None = None,
mask_color: str = "k",
point_size: float = 10,
ax_label_size: float = 12,
frame_offset: float = 0.05,
spine_width: float = 0.5,
spine_color: str = "k",
- displayed_sides: tuple = ("bottom", "left"),
+ displayed_sides: tuple[str, ...] = ("bottom", "left"),
legend_ondata: bool = False,
legend_onside: bool = True,
legend_size: float = 12,
legends_per_col: int = 20,
- title: Union[str, List[str], None] = None,
+ title: str | list[str] | None = None,
title_size: int = 12,
hide_title: bool = False,
cbar_shrink: float = 0.6,
marker_scale: float = 70,
lspacing: float = 0.1,
cspacing: float = 1,
- savename: Optional[str] = None,
+ savename: str | None = None,
save_dpi: int = 300,
- ax=None,
+ ax: Any = None,
force_ints_as_cats: bool = True,
n_columns: int = 1,
w_pad: float = 1,
h_pad: float = 1,
- scatter_kwargs: Optional[Dict] = None,
+ scatter_kwargs: dict[Any, Any] | None = None,
shuffle_zorder: bool = True,
show_fig: bool = True,
- ):
+ ) -> Any:
"""Plots the reference and target cells in their unified space.
This function helps to plot the reference and target cells, the coordinates for which were obtained from
@@ -718,7 +2149,7 @@ def plot_unified_layout(
width: Figure width (Default value: 6)
height: Figure height (Default value: 6)
cmap: A matplotlib colourmap to be used to colour categorical or continuous values plotted on the cells.
- (Default value: tab20 for categorical variables and cmocean.deep for continuous variables)
+ (Default value: tab20 for categorical variables and viridis for continuous variables)
color_key: A custom colour map for cells. These can be used for categorical variables only. The keys in this
dictionary should be the category label as present in the `color_by` column and values should be
valid matplotlib colour names or hex codes of colours. (Default value: None)
@@ -781,21 +2212,31 @@ def plot_unified_layout(
"that for either `run_unified_umap` or `run_unified_tsne`. Please see the default values "
"for `label` parameter in those functions if unsure."
)
- t = self.zw[from_assay].projections[layout_key][:]
- attrs = dict(self.zw[from_assay].projections[layout_key].attrs)
- t_names = attrs["target_names"]
- ref_n_cells = attrs["n_cells"][0]
- t_n_cells = attrs["n_cells"][1:]
+ projections = as_zarr_group(
+ as_zarr_group(self.zw[from_assay], name=from_assay)["projections"],
+ name="projections",
+ )
+ layout = as_zarr_array(projections[layout_key], name=layout_key)
+ t = np.asarray(layout[:])
+ attrs = dict(layout.attrs)
+ t_names = cast(list[str], attrs["target_names"])
+ n_cells_attr = cast(list[int], attrs["n_cells"])
+ ref_n_cells = n_cells_attr[0]
+ t_n_cells = n_cells_attr[1:]
x = t[:, 0]
y = t[:, 1]
df = pd.DataFrame({f"{layout_key}1": x, f"{layout_key}2": y})
+ plot_color_key = color_key
+ plot_mask_values: list[Any] | None
+ mask_name: str
+ group_col: NDArray[Any]
if target_groups is None:
- if color_key is not None:
+ if plot_color_key is not None:
temp_raise_error = False
- if ref_name not in color_key:
+ if ref_name not in plot_color_key:
temp_raise_error = True
for i in t_names:
- if i not in color_key:
+ if i not in plot_color_key:
temp_raise_error = True
if temp_raise_error:
temp = " ".join(t_names)
@@ -806,72 +2247,73 @@ def plot_unified_layout(
import seaborn as sns
temp_cmap = sns.color_palette("hls", n_colors=len(t_names) + 1).as_hex()
- color_key = {k: v for k, v in zip(t_names, temp_cmap[1:])}
- color_key[ref_name] = temp_cmap[0]
- target_groups = []
+ plot_color_key = {k: v for k, v in zip(t_names, temp_cmap[1:])}
+ plot_color_key[ref_name] = temp_cmap[0]
+ group_labels: list[str] = []
for i, j in zip(t_names, t_n_cells):
- target_groups.extend([i for _ in range(j)])
- target_groups = np.array(target_groups).astype(object)
- mask_values = None
+ group_labels.extend([i for _ in range(j)])
+ group_col = np.array(group_labels, dtype=object)
+ plot_mask_values = None
mask_name = "NA"
else:
- if len(target_groups) == len(t_names):
- temp = []
- for i in target_groups:
- temp.extend(list(i))
- target_groups = list(temp)
- color_key = None
- mask_values = [ref_name]
+ group_labels = list(target_groups)
+ if len(group_labels) == len(t_names):
+ flattened: list[str] = []
+ for entry in group_labels:
+ flattened.extend(list(entry))
+ group_labels = flattened
+ plot_mask_values = [ref_name]
mask_name = ref_name
- target_groups = np.array(target_groups).astype(object)
- if len(target_groups) != sum(t_n_cells):
+ group_col = np.array(group_labels, dtype=object)
+ if len(group_col) != sum(t_n_cells):
raise ValueError(
"ERROR: Number of values in `target_groups` should be same as no. of target cells"
)
# Turning array to object forces np.NaN to 'nan'
- if any(target_groups == "nan"):
+ if np.any(group_col == "nan"):
raise ValueError("ERROR: `target_groups` cannot contain nan values")
df["vc"] = np.hstack(
- [[ref_name for _ in range(ref_n_cells)], target_groups]
+ [[ref_name for _ in range(ref_n_cells)], group_col]
).astype(object)
if show_target_only:
df = df[ref_n_cells:]
if shuffle_zorder:
df = df.sample(frac=1)
+ plot_title = title if isinstance(title, str) else None
return plot_scatter(
[df],
- ax,
- width,
- height,
- mask_color,
- cmap,
- color_key,
- mask_values,
- mask_name,
- mask_color,
- point_size,
- ax_label_size,
- frame_offset,
- spine_width,
- spine_color,
- displayed_sides,
- legend_ondata,
- legend_onside,
- legend_size,
- legends_per_col,
- title,
- title_size,
- hide_title,
- cbar_shrink,
- marker_scale,
- lspacing,
- cspacing,
- savename,
- save_dpi,
- force_ints_as_cats,
- n_columns,
- w_pad,
- h_pad,
- show_fig,
- scatter_kwargs,
+ in_ax=ax,
+ width=width,
+ height=height,
+ default_color="steelblue",
+ color_map=cmap,
+ color_key=plot_color_key,
+ mask_values=plot_mask_values or [],
+ mask_name=mask_name,
+ mask_color=mask_color,
+ point_size=point_size,
+ ax_label_size=ax_label_size,
+ frame_offset=frame_offset,
+ spine_width=spine_width,
+ spine_color=spine_color,
+ displayed_sides=displayed_sides,
+ legend_ondata=legend_ondata,
+ legend_onside=legend_onside,
+ legend_size=legend_size,
+ legends_per_col=legends_per_col,
+ titles=plot_title,
+ title_size=title_size,
+ hide_title=hide_title,
+ cbar_shrink=cbar_shrink,
+ marker_scale=marker_scale,
+ lspacing=lspacing,
+ cspacing=cspacing,
+ savename=savename or "",
+ dpi=save_dpi,
+ force_ints_as_cats=force_ints_as_cats,
+ n_columns=n_columns,
+ w_pad=w_pad,
+ h_pad=h_pad,
+ show_fig=show_fig,
+ scatter_kwargs=scatter_kwargs or {},
)
diff --git a/scarf/dendrogram.py b/scarf/dendrogram.py
index 1c70ca6a..28d58d14 100644
--- a/scarf/dendrogram.py
+++ b/scarf/dendrogram.py
@@ -1,16 +1,24 @@
-from typing import List, Dict
+from collections.abc import Iterator
import networkx as nx
import numpy as np
+import pandas as pd
from .utils import logger, tqdmbar
__all__ = ["BalancedCut", "CoalesceTree", "make_digraph"]
-def make_digraph(d: np.ndarray, clust_info=None) -> nx.DiGraph:
- """Convert dendrogram into directed graph."""
+def make_digraph(d: np.ndarray, clust_info: np.ndarray | None = None) -> nx.DiGraph:
+ """Convert a scipy linkage matrix into a directed tree graph.
+ Args:
+ d: Linkage matrix from hierarchical clustering.
+ clust_info: Optional cluster label per leaf (-1 if unknown).
+
+ Returns:
+ Directed graph with merge nodes and leaf cluster annotations.
+ """
g = nx.DiGraph()
n = d.shape[0] + 1 # Dendrogram contains one less sample
if clust_info is not None:
@@ -22,14 +30,14 @@ def make_digraph(d: np.ndarray, clust_info=None) -> nx.DiGraph:
clust_info = np.ones(d.shape[0] + 1) * -1
for i in tqdmbar(d, desc="Constructing graph from dendrogram"):
v = i[2] # Distance between clusters
- i = i.astype(int)
- g.add_node(n, nleaves=i[3], dist=v)
- if i[0] <= d.shape[0]:
- g.add_node(i[0], nleaves=0, dist=v, cluster=clust_info[i[0]])
- if i[1] <= d.shape[0]:
- g.add_node(i[1], nleaves=0, dist=v, cluster=clust_info[i[1]])
- g.add_edge(n, i[0])
- g.add_edge(n, i[1])
+ row = i.astype(int)
+ g.add_node(n, nleaves=row[3], dist=v)
+ if row[0] <= d.shape[0]:
+ g.add_node(row[0], nleaves=0, dist=v, cluster=clust_info[row[0]])
+ if row[1] <= d.shape[0]:
+ g.add_node(row[1], nleaves=0, dist=v, cluster=clust_info[row[1]])
+ g.add_edge(n, row[0])
+ g.add_edge(n, row[1])
n += 1
if g.number_of_edges() != d.shape[0] * 2:
logger.warning(
@@ -39,10 +47,10 @@ def make_digraph(d: np.ndarray, clust_info=None) -> nx.DiGraph:
def CoalesceTree(graph: nx.DiGraph, clusters: np.ndarray) -> nx.DiGraph:
- def calc_steps_to_top(g: nx.DiGraph, c: np.ndarray):
- import pandas as pd
+ """Coalesce a hierarchy graph to the nodes holding each cluster partition."""
- s = {}
+ def calc_steps_to_top(g: nx.DiGraph, c: np.ndarray) -> pd.Series:
+ s: dict[int, int] = {}
for i in range(len(c)):
s[i] = 0
q = [i]
@@ -52,46 +60,46 @@ def calc_steps_to_top(g: nx.DiGraph, c: np.ndarray):
q.append(j)
return pd.Series(s).sort_values()
- def iter_predecessors(g: nx.DiGraph, v):
+ def iter_predecessors(g: nx.DiGraph, v: int) -> Iterator[int]:
q = [v]
while len(q) > 0:
for i in g.predecessors(q.pop(0)):
yield i
q.append(i)
- def aggregate_leaves(g: nx.DiGraph, v):
+ def aggregate_leaves(g: nx.DiGraph, v: int) -> list[int]:
q = [v]
- l = []
+ leaves: list[int] = []
while len(q) > 0:
for i in g.successors(q.pop(0)):
if g.nodes[i]["nleaves"] == 0:
- l.append(i)
+ leaves.append(i)
else:
q.append(i)
- return l
+ return leaves
- def get_holding_nodes(g: nx.DiGraph, c):
- hn = {}
+ def get_holding_nodes(g: nx.DiGraph, c: np.ndarray) -> dict[int, int]:
+ hn: dict[int, int] = {}
s = calc_steps_to_top(g, c)
for i in tqdmbar(set(c), desc="Identifying the top node for cluster"):
- l = set(np.where(c == i)[0])
- nl = len(l)
- for j in iter_predecessors(g, s.reindex(l).idxmin()):
+ cluster_nodes = set(np.where(c == i)[0])
+ nl = len(cluster_nodes)
+ for j in iter_predecessors(g, s.reindex(cluster_nodes).idxmin()):
if g.nodes[j]["nleaves"] >= nl:
l2 = aggregate_leaves(g, j)
- if len(l.intersection(l2)) == nl:
+ if len(cluster_nodes.intersection(l2)) == nl:
hn[j] = i
break
return hn
- def aggregate_predecessors(g: nx.DiGraph, v):
- p = []
+ def aggregate_predecessors(g: nx.DiGraph, v: int) -> list[int]:
+ p: list[int] = []
for i in iter_predecessors(g, v):
p.append(i)
return p
- def make_subgraph(g: nx.DiGraph, vs):
- sn = list(vs.keys())
+ def make_subgraph(g: nx.DiGraph, vs: dict[int, int]) -> nx.DiGraph:
+ sn: list[int] = list(vs.keys())
for i in vs:
sn.extend(aggregate_predecessors(g, i))
sn = list(set(sn))
@@ -104,6 +112,18 @@ def make_subgraph(g: nx.DiGraph, vs):
class BalancedCut:
+ """Identify balanced cluster splits from a hierarchical clustering dendrogram.
+
+ Args:
+ dendrogram: Linkage matrix from hierarchical clustering.
+ max_size: Maximum leaves allowed in a cluster before splitting.
+ min_size: Minimum leaves required before merging subtrees.
+ max_distance_fc: Max fold-change in merge distance for subtree merging.
+ """
+
+ graph: nx.DiGraph
+ branchpoints: dict[int, list[int]]
+
def __init__(
self,
dendrogram: np.ndarray,
@@ -118,10 +138,10 @@ def __init__(
self.maxDistFc = max_distance_fc
self.branchpoints = self._get_branchpoints()
- def _successors(self, start: int, min_leaves: int) -> List[int]:
+ def _successors(self, start: int, min_leaves: int) -> list[int]:
"""Get tree downstream of a node."""
q = [start]
- d = []
+ d: list[int] = []
while len(q) > 0:
i = q.pop(0)
if self.graph.nodes[i]["nleaves"] > min_leaves:
@@ -132,7 +152,7 @@ def _successors(self, start: int, min_leaves: int) -> List[int]:
def _get_mean_dist(self, start_node: int) -> float:
"""Get mean distances in downstream tree of a node."""
s_nodes = self._successors(start_node, -1)
- return np.array([self.graph.nodes[x]["dist"] for x in s_nodes]).mean()
+ return float(np.array([self.graph.nodes[x]["dist"] for x in s_nodes]).mean())
def _are_subtrees_mergeable(self, s1: int, s2: int) -> bool:
n1, n2 = self.graph.nodes[s1]["nleaves"], self.graph.nodes[s2]["nleaves"]
@@ -150,11 +170,11 @@ def _are_subtrees_mergeable(self, s1: int, s2: int) -> bool:
return False
return True
- def _get_branchpoints(self) -> Dict[int, List[int]]:
+ def _get_branchpoints(self) -> dict[int, list[int]]:
"""Aggregate leaves bottom up until target size is reached."""
n_leaves = int((self.graph.number_of_nodes() + 1) / 2)
- leaves = {x: None for x in range(n_leaves)}
- bps = {}
+ leaves: dict[int, None] = {x: None for x in range(n_leaves)}
+ bps: dict[int, list[int]] = {}
pbar = tqdmbar(total=len(leaves), desc="Identifying nodes to split")
while len(leaves) > 0:
leaf, _ = leaves.popitem()
@@ -195,7 +215,7 @@ def _get_branchpoints(self) -> Dict[int, List[int]]:
return bps
def _valid_names_in_branchpoints(self) -> None:
- leaves = []
+ leaves: list[int] = []
for i in self.branchpoints:
leaves.extend(self.branchpoints[i])
n_leaves = len(leaves)
@@ -223,17 +243,3 @@ def get_clusters(self) -> np.ndarray:
logger.warning(f"{(c == 0).sum()} samples were not assigned a cluster")
c[c == 0] = -1
return c
-
- # def test(self):
- # n = 30
- # np.random.seed(154)
- # randz = ward(gamma.rvs(0.3, size=n * 4).reshape((n, 4)))
- #
- # graph = z_to_g(randz)
- # branchpoints = get_branchpoints(graph, max_leaf=10, min_leaves=2, fc=1.5)
- # clusters = bps_to_clusts(branchpoints)
- # dend = dendrogram(randz)
- # clusters[dend['leaves']]
- #
- # res = [9, 5, 5, 1, 1, 1, 1, 7, 7, 7, 3, 3, 3, 3, 3, 4, 4, 4, 2, 2, 2, 8,
- # 8, 8, 6, 6, 6, 6, 6, 6]
diff --git a/scarf/doublet_utils.py b/scarf/doublet_utils.py
new file mode 100644
index 00000000..8bb97589
--- /dev/null
+++ b/scarf/doublet_utils.py
@@ -0,0 +1,152 @@
+"""Utilities for synthetic doublet detection.
+
+This implements a Scrublet/DoubletFinder style strategy adapted to Scarf's
+out-of-core graph and projection framework. Synthetic doublets are simulated by
+summing pairs of observed transcriptomes, projected onto the existing reference
+KNN graph and scored by how often each reference cell is found among the
+nearest neighbours of the simulated doublets. The scores are subsequently
+diffused over the graph using the same operator that powers `get_imputed`.
+"""
+
+import numpy as np
+import zarr
+from numpy.typing import NDArray
+from scipy.sparse import csr_matrix
+
+from .utils import logger
+
+__all__ = [
+ "sample_cluster_pool",
+ "simulate_doublet_pairs",
+ "write_doublet_target_zarr",
+]
+
+
+def sample_cluster_pool(
+ clusters: NDArray,
+ fraction: float,
+ max_per_cluster: int,
+ rng: np.random.Generator,
+) -> NDArray[np.int64]:
+ """Draw a per-cluster subsample of cell positions to seed doublet simulation.
+
+ For each cluster a fraction of its cells is sampled, capped at
+ `max_per_cluster`. The returned positions index into the array of clusters
+ that was passed in (i.e. positions among the active reference cells).
+
+ Args:
+ clusters: Cluster label for each active reference cell.
+ fraction: Fraction of each cluster to sample.
+ max_per_cluster: Hard cap on the number of cells sampled per cluster.
+ rng: Random number generator.
+
+ Returns:
+ Sorted array of sampled cell positions.
+ """
+ pool = []
+ for c in np.unique(clusters):
+ idx = np.where(clusters == c)[0]
+ n = min(int(np.ceil(len(idx) * fraction)), max_per_cluster, len(idx))
+ if n <= 0:
+ continue
+ pool.append(rng.choice(idx, size=n, replace=False))
+ if len(pool) == 0:
+ raise ValueError("ERROR: No cells could be sampled to simulate doublets")
+ return np.sort(np.concatenate(pool))
+
+
+def simulate_doublet_pairs(
+ pool_clusters: NDArray,
+ n_sim: int,
+ heterotypic_fraction: float,
+ rng: np.random.Generator,
+ max_tries: int = 20,
+) -> tuple[NDArray[np.int64], NDArray[np.int64]]:
+ """Generate index pairs into the candidate pool for simulated doublets.
+
+ A configurable fraction of the pairs are biased to be heterotypic (the two
+ cells coming from different clusters), because homotypic doublets are
+ largely indistinguishable from singlets and carry little detection signal.
+
+ Args:
+ pool_clusters: Cluster label for each cell in the candidate pool.
+ n_sim: Number of doublet pairs to generate.
+ heterotypic_fraction: Fraction of pairs forced to be cross-cluster.
+ rng: Random number generator.
+ max_tries: Maximum resampling rounds used to satisfy the heterotypic
+ constraint before falling back to whatever was drawn.
+
+ Returns:
+ A tuple of two arrays holding the left and right pool indices.
+ """
+ pool_size = len(pool_clusters)
+ left = rng.integers(0, pool_size, size=n_sim)
+ right = rng.integers(0, pool_size, size=n_sim)
+ if heterotypic_fraction > 0 and len(np.unique(pool_clusters)) > 1:
+ want_hetero = rng.random(n_sim) < heterotypic_fraction
+ for _ in range(max_tries):
+ clash = want_hetero & (pool_clusters[left] == pool_clusters[right])
+ if not clash.any():
+ break
+ right[clash] = rng.integers(0, pool_size, size=int(clash.sum()))
+ return left, right
+
+
+def write_doublet_target_zarr(
+ zarr_loc: str,
+ assay_name: str,
+ sim_counts: csr_matrix,
+ feat_ids: NDArray,
+ feat_names: NDArray,
+ dtype: str = "uint32",
+ batch_size: int = 1000,
+) -> zarr.Group:
+ """Materialise simulated doublet counts as a minimal Scarf Zarr hierarchy.
+
+ The resulting store mirrors the feature universe of the reference assay so
+ that `run_mapping` can align features by id without rebuilding the graph.
+
+ Args:
+ zarr_loc: Destination path (or store) for the temporary Zarr hierarchy.
+ assay_name: Name to give the simulated assay (matched to the reference).
+ sim_counts: Sparse matrix of simulated doublet counts (doublets x genes).
+ feat_ids: Feature ids in the same order as the reference raw matrix.
+ feat_names: Feature names in the same order as the reference raw matrix.
+ dtype: Storage dtype for the count matrix.
+ batch_size: Number of rows written per batch.
+
+ Returns:
+ The root Zarr group of the written store.
+ """
+ from .storage.zarr_store import write_dense_in_shard_rows
+ from .utils import load_zarr
+ from .writers import (
+ create_cell_data,
+ create_zarr_count_assay,
+ finalize_writer_counts,
+ load_count_store,
+ )
+
+ n_sim = sim_counts.shape[0]
+ z = load_zarr(zarr_loc=zarr_loc, mode="w")
+ ids = np.array([f"doublet_{i}" for i in range(n_sim)])
+ create_cell_data(z, workspace=None, ids=ids, names=ids)
+ create_zarr_count_assay(
+ z=z,
+ assay_name=assay_name,
+ workspace=None,
+ chunk_size=(batch_size, 1000),
+ n_cells=n_sim,
+ feat_ids=np.asarray(feat_ids),
+ feat_names=np.asarray(feat_names),
+ dtype=dtype,
+ )
+ store = load_count_store(z, assay_name, None)
+ write_dense_in_shard_rows(
+ store,
+ lambda s, e: sim_counts[s:e].toarray().astype(dtype),
+ msg="Writing simulated doublets",
+ )
+ finalize_writer_counts(z, assay_name, None)
+ logger.debug(f"Wrote {n_sim} simulated doublets to {zarr_loc}")
+ return z
diff --git a/scarf/downloader.py b/scarf/downloader.py
index 14147f61..c1b34309 100644
--- a/scarf/downloader.py
+++ b/scarf/downloader.py
@@ -15,6 +15,7 @@
import tarfile
import time
from json import JSONDecodeError
+from typing import Any
import pandas as pd
@@ -22,34 +23,32 @@
__all__ = ["show_available_datasets", "fetch_dataset"]
+type JsonDict = dict[str, Any]
+type DatasetEntry = tuple[str, str]
+type DatasetsMap = dict[str, DatasetEntry]
+type FileMap = dict[str, str]
+
class OSFdownloader:
- """
- A class for downloading datasets from OSF.
+ """Download datasets from an OSF project via the OSF API.
Attributes:
- projectId:
- storages:
- url:
- datasets:
- sourceFile:
- sources:
-
- Methods:
- get_json:
- get_all_pages:
- show_datasets:
- get_dataset_file_ids:
+ projectId: OSF node identifier.
+ storages: Storage providers to search.
+ url: OSF files API endpoint.
+ datasets: Parsed dataset metadata table.
+ sourceFile: Raw OSF file listing JSON.
+ sources: Mapping of dataset names to download URLs.
"""
- def __init__(self, project_id):
+ def __init__(self, project_id: str) -> None:
self.projectId = project_id
self.storages = ["osfstorage", "figshare"]
self.url = f"https://api.osf.io/v2/nodes/{self.projectId}/files/"
self.datasets, self.sourceFile = self._populate_datasets()
self.sources = self._populate_sources()
- def get_json(self, storage, endpoint, url):
+ def get_json(self, storage: str, endpoint: str, url: str | None) -> Any:
import requests
if endpoint != "":
@@ -58,7 +57,7 @@ def get_json(self, storage, endpoint, url):
url = self.url + f"{storage}/{endpoint}"
return requests.get(url).json()
- def get_all_pages(self, storage, endpoint=""):
+ def get_all_pages(self, storage: str, endpoint: str = "") -> list[JsonDict]:
data = []
url = None
n_attempts = 0
@@ -80,10 +79,10 @@ def get_all_pages(self, storage, endpoint=""):
return data
@staticmethod
- def _process_path(node):
- return node["attributes"]["path"].rstrip("/").lstrip("/")
+ def _process_path(node: JsonDict) -> str:
+ return str(node["attributes"]["path"]).rstrip("/").lstrip("/")
- def _populate_datasets(self):
+ def _populate_datasets(self) -> tuple[DatasetsMap, str]:
datasets = {}
source_fn = ""
for storage in self.storages:
@@ -95,7 +94,7 @@ def _populate_datasets(self):
datasets[i["attributes"]["name"]] = (path, storage)
return datasets, source_fn
- def _populate_sources(self):
+ def _populate_sources(self) -> dict[str, Any]:
import requests
n_attempts = 0
@@ -104,7 +103,7 @@ def _populate_sources(self):
source_fn = self._get_files_for_node("osfstorage", self.sourceFile)[
"sources.csv"
]
- return (
+ return dict(
pd.read_csv(io.StringIO(requests.get(source_fn).text))
.set_index("id")
.to_dict()
@@ -112,11 +111,13 @@ def _populate_sources(self):
except KeyError:
time.sleep(1)
n_attempts += 1
+ msg = "Failed to load dataset sources after 5 attempts"
+ raise KeyError(msg)
- def show_datasets(self):
+ def show_datasets(self) -> None:
print("\n".join(sorted(self.datasets.keys())))
- def _get_files_for_node(self, storage, file_id):
+ def _get_files_for_node(self, storage: str, file_id: str) -> FileMap:
base_url = f"https://files.de-1.osf.io/v1/resources/{self.projectId}/providers/"
ret_val = {}
for i in self.get_all_pages(storage, file_id):
@@ -124,17 +125,18 @@ def _get_files_for_node(self, storage, file_id):
ret_val[i["attributes"]["name"]] = base_url + f"{storage}/{path}"
return ret_val
- def get_dataset_file_ids(self, dataset_name):
+ def get_dataset_file_ids(self, dataset_name: str) -> FileMap:
if dataset_name not in self.datasets:
+ available = "\n".join(sorted(self.datasets.keys()))
raise KeyError(
f"ERROR: {dataset_name} was not found. "
- f"Please choose one of the following:\n{self.show_datasets()}"
+ f"Please choose one of the following:\n{available}"
)
file_id, storage = self.datasets[dataset_name]
return self._get_files_for_node(storage, file_id)
-osfd = None
+osfd: OSFdownloader | None = None
def handle_download(url: str, out_fn: str, seq_counter: str = "") -> None:
@@ -206,17 +208,18 @@ def fetch_dataset(
zarr_ext = ".zarr.tar.gz"
- def has_zarr(entry):
+ def has_zarr(entry: FileMap) -> bool:
for e in entry:
if e.endswith(zarr_ext):
return True
return False
- def get_zarr_entry(entry):
+ def get_zarr_entry(entry: FileMap) -> tuple[str, str]:
for e in entry:
if e.endswith(zarr_ext):
return e, entry[e]
- return False, False
+ msg = "No zarr entry found in dataset files"
+ raise KeyError(msg)
global osfd
if osfd is None:
diff --git a/scarf/feat_utils.py b/scarf/feat_utils.py
index 0e53287e..5cf8d661 100644
--- a/scarf/feat_utils.py
+++ b/scarf/feat_utils.py
@@ -1,6 +1,6 @@
"""Utility functions for features."""
-from typing import List, Tuple
+from typing import Any
import numpy as np
import pandas as pd
@@ -8,37 +8,40 @@
__all__ = ["fit_lowess", "binned_sampling", "hto_demux"]
-def fit_lowess(a, b, n_bins: int, lowess_frac: float) -> np.ndarray:
+def fit_lowess(
+ a: np.ndarray, b: np.ndarray, n_bins: int, lowess_frac: float
+) -> np.ndarray:
"""Fits a LOWESS (Locally Weighted Scatterplot Smoothing) curve.
Args:
- a:
- b:
- n_bins:
- lowess_frac:
+ a: Log-binned x values (e.g. log mean expression).
+ b: Log y values to regress against (e.g. log variance).
+ n_bins: Number of histogram bins for grouping genes.
+ lowess_frac: LOWESS smoothing fraction.
Returns:
+ Per-feature corrected variance estimates.
"""
from statsmodels.nonparametric.smoothers_lowess import lowess
stats = pd.DataFrame({"a": a, "b": b}).apply(np.log)
bin_edges = np.histogram(stats.a, bins=n_bins)[1]
bin_edges[-1] += 0.1 # For including last gene
- bin_idx = []
+ bin_idx: list[list[Any]] = []
for i in range(n_bins):
idx = pd.Series((stats.a >= bin_edges[i]) & (stats.a < bin_edges[i + 1]))
if sum(idx) > 0:
bin_idx.append(list(idx[idx].index))
- bin_vals = []
+ bin_vals: list[list[float]] = []
for idx in bin_idx:
temp_stat = stats.reindex(idx)
temp_gene = temp_stat.idxmin().b
bin_vals.append([temp_stat.b[temp_gene], temp_stat.a[temp_gene]])
- bin_vals = np.array(bin_vals).T
+ bin_array = np.array(bin_vals).T
bin_cor_fac = lowess(
- bin_vals[0], bin_vals[1], return_sorted=False, frac=lowess_frac, it=100
+ bin_array[0], bin_array[1], return_sorted=False, frac=lowess_frac, it=100
).T
- fixed_var = {}
+ fixed_var: dict[Any, float] = {}
for bcf, indices in zip(bin_cor_fac, bin_idx):
for idx in indices:
fixed_var[idx] = np.e ** (stats.b[idx] - bcf)
@@ -47,11 +50,11 @@ def fit_lowess(a, b, n_bins: int, lowess_frac: float) -> np.ndarray:
def binned_sampling(
values: pd.Series,
- feature_list: List[str],
+ feature_list: list[str],
ctrl_size: int,
n_bins: int,
rand_seed: int,
-) -> List[str]:
+) -> list[str]:
"""Score a set of genes [Satija15]_. The score is the average expression of
a set of genes subtracted with the average expression of a reference set of
genes. The reference set is randomly sampled from the `gene_pool` for each
@@ -73,13 +76,11 @@ def binned_sampling(
A list of sampled features.
"""
n_items = int(np.round(len(values) / (n_bins - 1)))
- feature_list = set(feature_list)
- # Made following more linter friendly
- # obs_cut = obs_avg.rank(method='min') // n_items
+ feature_set = set(feature_list)
obs_cut: pd.Series = values.fillna(0).rank(method="min").divide(n_items).astype(int)
- control_genes = set()
- for cut in np.unique(obs_cut[list(feature_list)]):
+ control_genes: set[Any] = set()
+ for cut in np.unique(obs_cut[list(feature_set)]):
sub_obs = obs_cut[obs_cut == cut]
if len(sub_obs) == 0:
continue
@@ -89,7 +90,7 @@ def binned_sampling(
sample_size = ctrl_size
r_genes = sub_obs.sample(n=sample_size, random_state=rand_seed).index
control_genes.update(set(r_genes))
- return list(control_genes - feature_list)
+ return list(control_genes - feature_set)
def hto_demux(hto_counts: pd.DataFrame) -> pd.Series:
@@ -112,14 +113,14 @@ def clr_normalize(df: pd.DataFrame) -> pd.DataFrame:
def calc_cluster_labels(
df: pd.DataFrame, n_centers: int | None = None, n_starts: int = 100
- ):
+ ) -> np.ndarray:
if n_centers is None:
n_centers = df.shape[1] + 1
kmeans = KMeans(n_clusters=n_centers, init="random", n_init=n_starts)
kmeans.fit(df)
- return kmeans.labels_
+ return np.asarray(kmeans.labels_)
- def calc_cluster_avg_exp(df: pd.DataFrame) -> Tuple[pd.Series, pd.DataFrame]:
+ def calc_cluster_avg_exp(df: pd.DataFrame) -> tuple[pd.Series, pd.DataFrame]:
df["cluster"] = calc_cluster_labels(df)
return df["cluster"], df.groupby("cluster").mean()
@@ -131,20 +132,20 @@ def get_background_cutoff(vals: np.ndarray, quantile: float = 0.99) -> int:
p = 1 / (1 + np.exp(fit.params[0]) * fit.params[1])
n = np.exp(fit.params[0]) * p / (1 - p)
dist = nbinom(n=n, p=p, loc=mu)
- return round(dist.ppf(quantile))
+ return int(round(dist.ppf(quantile)))
def discretize_counts(
df: pd.DataFrame, clust_labels: pd.Series, clust_exp: pd.DataFrame
) -> pd.DataFrame:
min_clust = clust_exp.idxmin()
- cutoffs = {}
+ cutoffs: dict[Any, int] = {}
for hto in df:
bg_values = df[hto][clust_labels == min_clust[hto]].values
cutoffs[hto] = get_background_cutoff(bg_values)
- cutoffs = pd.Series(cutoffs)
- return df > cutoffs
+ cutoff_series = pd.Series(cutoffs)
+ return df > cutoff_series
- def identity_renamer(x: int):
+ def identity_renamer(x: int) -> str:
if x == 0:
return "Negative"
elif x == 1:
diff --git a/scarf/harmony.py b/scarf/harmony.py
index 291bdf1b..8bfa41a5 100644
--- a/scarf/harmony.py
+++ b/scarf/harmony.py
@@ -1,3 +1,5 @@
+from collections.abc import Callable
+from dataclasses import dataclass
from functools import partial
import numpy as np
@@ -6,52 +8,153 @@
from .utils import tqdmbar, logger
+type ClusterFn = str | Callable[[np.ndarray, int], np.ndarray]
+
+
+@dataclass(frozen=True)
+class HarmonyResult:
+ """Converged Harmony state required for portable reference mapping."""
+
+ original: np.ndarray
+ corrected: np.ndarray
+ assignments: np.ndarray
+ centroids: np.ndarray
+ sigma: np.ndarray
+ ridge: np.ndarray
+ batch_columns: tuple[str, ...]
+ batch_levels: tuple[tuple[str, ...], ...]
+ parameters: dict[str, object]
+
def run_harmony(
data_mat: np.ndarray,
meta_data: pd.DataFrame,
- theta=None,
- lamb=None,
- sigma=0.1,
- nclust=None,
- tau=0,
- block_size=0.05,
- max_iter_harmony=50,
- max_iter_kmeans=20,
- epsilon_cluster=1e-5,
- epsilon_harmony=1e-4,
- random_state=0,
- cluster_fn="kmeans",
-):
- """Run Harmony."""
+ theta: float | int | np.ndarray | list[float] | None = None,
+ lamb: float | int | np.ndarray | list[float] | None = None,
+ sigma: float | np.ndarray = 0.1,
+ nclust: int | None = None,
+ tau: float = 0,
+ block_size: float = 0.05,
+ max_iter_harmony: int = 50,
+ max_iter_kmeans: int = 20,
+ epsilon_cluster: float = 1e-5,
+ epsilon_harmony: float = 1e-4,
+ random_state: int = 0,
+ cluster_fn: ClusterFn = "kmeans",
+) -> np.ndarray:
+ """Run Harmony batch correction on a PCA embedding.
+
+ Args:
+ data_mat: Embedding matrix, shape (n_dims, n_cells).
+ meta_data: Batch metadata DataFrame (one column per batch variable).
+ theta: Cluster diversity penalty per batch level (default: 1 per level).
+ lamb: Ridge penalty per batch level (default: 1 per level).
+ sigma: Kernel width(s) for soft k-means clustering.
+ nclust: Number of Harmony clusters (default: min(round(N/30), 100)).
+ tau: Protects small batches when > 0.
+ block_size: Fraction of cells per update block.
+ max_iter_harmony: Maximum Harmony iterations.
+ max_iter_kmeans: Maximum k-means iterations per round.
+ epsilon_cluster: Convergence threshold for clustering.
+ epsilon_harmony: Convergence threshold for Harmony.
+ random_state: Random seed.
+ cluster_fn: Clustering backend (``'kmeans'``).
+
+ Returns:
+ Batch-corrected embedding matrix, shape (n_dims, n_cells).
+ """
+
+ return fit_harmony(
+ data_mat,
+ meta_data,
+ theta=theta,
+ lamb=lamb,
+ sigma=sigma,
+ nclust=nclust,
+ tau=tau,
+ block_size=block_size,
+ max_iter_harmony=max_iter_harmony,
+ max_iter_kmeans=max_iter_kmeans,
+ epsilon_cluster=epsilon_cluster,
+ epsilon_harmony=epsilon_harmony,
+ random_state=random_state,
+ cluster_fn=cluster_fn,
+ ).corrected
+
+
+def fit_harmony(
+ data_mat: np.ndarray,
+ meta_data: pd.DataFrame,
+ theta: float | int | np.ndarray | list[float] | None = None,
+ lamb: float | int | np.ndarray | list[float] | None = None,
+ sigma: float | np.ndarray = 0.1,
+ nclust: int | None = None,
+ tau: float = 0,
+ block_size: float = 0.05,
+ max_iter_harmony: int = 50,
+ max_iter_kmeans: int = 20,
+ epsilon_cluster: float = 1e-5,
+ epsilon_harmony: float = 1e-4,
+ random_state: int = 0,
+ cluster_fn: ClusterFn = "kmeans",
+) -> HarmonyResult:
+ """Fit Harmony and return both corrected coordinates and portable state."""
+ if data_mat.ndim != 2:
+ raise ValueError("Harmony data_mat must be two-dimensional")
+ if data_mat.shape[1] != len(meta_data):
+ raise ValueError(
+ "Harmony metadata rows must match the number of embedding columns"
+ )
+ if data_mat.shape[1] < 2:
+ raise ValueError("Harmony requires at least two cells")
+ if meta_data.empty:
+ raise ValueError("Harmony requires at least one batch metadata column")
+ if meta_data.columns.duplicated().any():
+ raise ValueError("Harmony batch metadata column names must be unique")
+ if meta_data.isna().any().any():
+ raise ValueError("Harmony batch metadata cannot contain missing values")
+ if not np.all(np.isfinite(data_mat)):
+ raise ValueError("Harmony input contains non-finite values")
N = data_mat.shape[1]
if nclust is None:
- nclust = np.min([np.round(N / 30.0), 100]).astype(int)
-
- if type(sigma) is float and nclust > 1:
- sigma = np.repeat(sigma, nclust)
-
- phi = pd.get_dummies(meta_data).to_numpy().T
+ nclust = max(1, int(np.min([np.round(N / 30.0), 100])))
+ elif nclust < 1 or nclust > N:
+ raise ValueError("Harmony nclust must be between one and the cell count")
+
+ sigma_arr = np.asarray(sigma, dtype=np.float64)
+ if sigma_arr.ndim == 0:
+ sigma_arr = np.full(nclust, float(sigma_arr.item()), dtype=np.float64)
+ elif sigma_arr.shape != (nclust,):
+ raise ValueError("Harmony sigma must be scalar or have one value per cluster")
+ if not np.all(np.isfinite(sigma_arr)) or np.any(sigma_arr <= 0):
+ raise ValueError("Harmony sigma values must be finite and positive")
+
+ phi_frame = pd.get_dummies(meta_data)
+ phi = phi_frame.to_numpy().T
phi_n = meta_data.describe().loc["unique"].to_numpy().astype(int)
- if theta is None:
- theta = np.repeat([1] * len(phi_n), phi_n)
- elif isinstance(theta, float) or isinstance(theta, int):
- theta = np.repeat([theta] * len(phi_n), phi_n)
- elif len(theta) == len(phi_n):
- theta = np.repeat([theta], phi_n)
-
- assert len(theta) == np.sum(phi_n), "each batch variable must have a theta"
-
- if lamb is None:
- lamb = np.repeat([1] * len(phi_n), phi_n)
- elif isinstance(lamb, float) or isinstance(lamb, int):
- lamb = np.repeat([lamb] * len(phi_n), phi_n)
- elif len(lamb) == len(phi_n):
- lamb = np.repeat([lamb], phi_n)
-
- assert len(lamb) == np.sum(phi_n), "each batch variable must have a lambda"
+ def _expand_parameter(
+ values: float | int | np.ndarray | list[float] | None,
+ name: str,
+ ) -> np.ndarray:
+ if values is None:
+ return np.ones(int(np.sum(phi_n)), dtype=np.float64)
+ array = np.asarray(values, dtype=np.float64)
+ if array.ndim == 0:
+ return np.full(int(np.sum(phi_n)), float(array.item()))
+ if array.shape == (len(phi_n),):
+ return np.repeat(array, phi_n)
+ if array.shape != (int(np.sum(phi_n)),):
+ raise ValueError(f"Each Harmony batch level must have a {name}")
+ return array
+
+ theta_arr = _expand_parameter(theta, "theta")
+ lamb_arr = _expand_parameter(lamb, "lambda")
+ if not np.all(np.isfinite(theta_arr)) or np.any(theta_arr < 0):
+ raise ValueError("Harmony theta values must be finite and non-negative")
+ if not np.all(np.isfinite(lamb_arr)) or np.any(lamb_arr < 0):
+ raise ValueError("Harmony lambda values must be finite and non-negative")
# Number of items in each category.
N_b = phi.sum(axis=1)
@@ -59,9 +162,9 @@ def run_harmony(
Pr_b = N_b / N
if tau > 0:
- theta = theta * (1 - np.exp(-((N_b / (nclust * tau)) ** 2)))
+ theta_arr = theta_arr * (1 - np.exp(-((N_b / (nclust * tau)) ** 2)))
- lamb_mat = np.diag(np.insert(lamb, 0, 0))
+ lamb_mat = np.diag(np.insert(lamb_arr, 0, 0))
phi_moe = np.vstack((np.repeat(1, N), phi))
@@ -72,8 +175,8 @@ def run_harmony(
phi,
phi_moe,
Pr_b,
- sigma,
- theta,
+ sigma_arr,
+ theta_arr,
max_iter_harmony,
max_iter_kmeans,
epsilon_cluster,
@@ -85,33 +188,70 @@ def run_harmony(
cluster_fn,
)
- return ho.result()
+ batch_columns = tuple(str(column) for column in meta_data.columns)
+ batch_levels = tuple(
+ tuple(str(value) for value in pd.unique(meta_data[column]))
+ for column in meta_data.columns
+ )
+ cluster_backend = (
+ cluster_fn
+ if isinstance(cluster_fn, str)
+ else (
+ f"{getattr(cluster_fn, '__module__', type(cluster_fn).__module__)}."
+ f"{getattr(cluster_fn, '__qualname__', type(cluster_fn).__qualname__)}"
+ )
+ )
+ parameters: dict[str, object] = {
+ "nclust": int(nclust),
+ "sigma": sigma_arr.tolist(),
+ "theta": theta_arr.tolist(),
+ "lambda": lamb_arr.tolist(),
+ "tau": float(tau),
+ "blockSize": float(block_size),
+ "maxIterHarmony": int(max_iter_harmony),
+ "maxIterKmeans": int(max_iter_kmeans),
+ "epsilonCluster": float(epsilon_cluster),
+ "epsilonHarmony": float(epsilon_harmony),
+ "randomState": int(random_state),
+ "clusterBackend": cluster_backend,
+ "phiColumns": [str(column) for column in phi_frame.columns],
+ }
+ return HarmonyResult(
+ original=np.asarray(data_mat, dtype=np.float64).copy(),
+ corrected=ho.result(),
+ assignments=ho.R.copy(),
+ centroids=ho.Y.copy(),
+ sigma=sigma_arr.copy(),
+ ridge=lamb_mat.copy(),
+ batch_columns=batch_columns,
+ batch_levels=batch_levels,
+ parameters=parameters,
+ )
-class Harmony(object):
+class Harmony:
def __init__(
self,
- Z,
- Phi,
- Phi_moe,
- Pr_b,
- sigma,
- theta,
- max_iter_harmony,
- max_iter_kmeans,
- epsilon_kmeans,
- epsilon_harmony,
- K,
- block_size,
- lamb,
- random_state=None,
- cluster_fn="kmeans",
- ):
+ Z: np.ndarray,
+ Phi: np.ndarray,
+ Phi_moe: np.ndarray,
+ Pr_b: np.ndarray,
+ sigma: np.ndarray,
+ theta: np.ndarray,
+ max_iter_harmony: int,
+ max_iter_kmeans: int,
+ epsilon_kmeans: float,
+ epsilon_harmony: float,
+ K: int,
+ block_size: float,
+ lamb: np.ndarray,
+ random_state: int | None = None,
+ cluster_fn: ClusterFn = "kmeans",
+ ) -> None:
self.Z_corr = np.array(Z)
self.Z_orig = np.array(Z)
- self.Z_cos = self.Z_orig / self.Z_orig.max(axis=0)
- self.Z_cos = self.Z_cos / np.linalg.norm(self.Z_cos, ord=2, axis=0)
+ self.Z_cos = _normalize_columns(self.Z_orig)
self.Phi = Phi
self.Phi_moe = Phi_moe
@@ -132,23 +272,28 @@ def __init__(
self.max_iter_kmeans = max_iter_kmeans
self.theta = theta
- self.objective_harmony = []
- self.objective_kmeans = []
- self.objective_kmeans_dist = []
- self.objective_kmeans_entropy = []
- self.objective_kmeans_cross = []
- self.kmeans_rounds = []
+ self.objective_harmony: list[float] = []
+ self.objective_kmeans: list[float] = []
+ self.objective_kmeans_dist: list[float] = []
+ self.objective_kmeans_entropy: list[float] = []
+ self.objective_kmeans_cross: list[float] = []
+ self.kmeans_rounds: list[int] = []
self.allocate_buffers()
- if cluster_fn == "kmeans":
- cluster_fn = partial(Harmony._cluster_kmeans, random_state=random_state)
- self.init_cluster(cluster_fn)
+ resolved_cluster_fn: Callable[[np.ndarray, int], np.ndarray]
+ if isinstance(cluster_fn, str):
+ resolved_cluster_fn = partial(
+ Harmony._cluster_kmeans, random_state=random_state
+ )
+ else:
+ resolved_cluster_fn = cluster_fn
+ self.init_cluster(resolved_cluster_fn)
self.harmonize(self.max_iter_harmony)
- def result(self):
+ def result(self) -> np.ndarray:
return self.Z_corr
- def allocate_buffers(self):
+ def allocate_buffers(self) -> None:
self._scale_dist = np.zeros((self.K, self.N))
self.dist_mat = np.zeros((self.K, self.N))
self.O = np.zeros((self.K, self.B))
@@ -157,9 +302,11 @@ def allocate_buffers(self):
self.Phi_Rk = np.zeros((self.B + 1, self.N))
@staticmethod
- def _cluster_kmeans(data, K, random_state):
+ def _cluster_kmeans(
+ data: np.ndarray, K: int, random_state: int | None
+ ) -> np.ndarray:
# Start with cluster centroids
- return (
+ centers = (
KMeans(
n_clusters=K,
init="k-means++",
@@ -170,11 +317,12 @@ def _cluster_kmeans(data, K, random_state):
.fit(data)
.cluster_centers_
)
+ return np.asarray(centers)
- def init_cluster(self, cluster_fn):
+ def init_cluster(self, cluster_fn: Callable[[np.ndarray, int], np.ndarray]) -> None:
self.Y = cluster_fn(self.Z_cos.T, self.K).T
# (1) Normalize
- self.Y = self.Y / np.linalg.norm(self.Y, ord=2, axis=0)
+ self.Y = _normalize_columns(self.Y)
# (2) Assign cluster probabilities
self.dist_mat = 2 * (1 - np.dot(self.Y.T, self.Z_cos))
self.R = -self.dist_mat
@@ -189,7 +337,7 @@ def init_cluster(self, cluster_fn):
# Save results
self.objective_harmony.append(self.objective_kmeans[-1])
- def compute_objective(self):
+ def compute_objective(self) -> None:
kmeans_error = np.sum(np.multiply(self.R, self.dist_mat))
# Entropy
_entropy = np.sum(safe_entropy(self.R) * self.sigma[:, np.newaxis])
@@ -205,7 +353,7 @@ def compute_objective(self):
self.objective_kmeans_entropy.append(_entropy)
self.objective_kmeans_cross.append(_cross_entropy)
- def harmonize(self, iter_harmony=10):
+ def harmonize(self, iter_harmony: int = 10) -> int:
converged = False
for i in tqdmbar(range(1, iter_harmony + 1), desc="Harmonizing batches"):
# STEP 1: Clustering
@@ -228,7 +376,7 @@ def harmonize(self, iter_harmony=10):
logger.warning("Stopped before convergence")
return 0
- def cluster(self):
+ def cluster(self) -> int:
# Z_cos has changed
# R is assumed to not have changed
# Update Y to match new integrated data
@@ -252,7 +400,7 @@ def cluster(self):
self.objective_harmony.append(self.objective_kmeans[-1])
return 0
- def update_R(self):
+ def update_R(self) -> int:
self._scale_dist = -self.dist_mat
self._scale_dist = self._scale_dist / self.sigma[:, None]
self._scale_dist -= np.max(self._scale_dist, axis=0)
@@ -280,7 +428,7 @@ def update_R(self):
self.O += np.dot(self.R[:, b], self.Phi[:, b].T)
return 0
- def check_convergence(self, i_type):
+ def check_convergence(self, i_type: int) -> bool:
obj_old = 0.0
obj_new = 0.0
# Clustering, compute new window mean
@@ -304,19 +452,38 @@ def check_convergence(self, i_type):
return True
-def safe_entropy(x: np.array):
+def safe_entropy(x: np.ndarray) -> np.ndarray:
y = np.multiply(x, np.log(x))
y[~np.isfinite(y)] = 0.0
- return y
-
-
-def moe_correct_ridge(Z_orig, R, W, K, Phi_Rk, Phi_moe, lamb):
+ return np.asarray(y)
+
+
+def moe_correct_ridge(
+ Z_orig: np.ndarray,
+ R: np.ndarray,
+ W: np.ndarray,
+ K: int,
+ Phi_Rk: np.ndarray,
+ Phi_moe: np.ndarray,
+ lamb: np.ndarray,
+) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
Z_corr = Z_orig.copy()
for i in range(K):
Phi_Rk = np.multiply(Phi_moe, R[i, :])
x = np.dot(Phi_Rk, Phi_moe.T) + lamb
- W = np.dot(np.dot(np.linalg.inv(x), Phi_Rk), Z_orig.T)
+ try:
+ W = np.linalg.solve(x, np.dot(Phi_Rk, Z_orig.T))
+ except np.linalg.LinAlgError as exc:
+ raise ValueError("Harmony ridge system is singular") from exc
W[0, :] = 0 # do not remove the intercept
Z_corr -= np.dot(W.T, Phi_Rk)
- Z_cos = Z_corr / np.linalg.norm(Z_corr, ord=2, axis=0)
+ Z_cos = _normalize_columns(Z_corr)
return Z_cos, Z_corr, W, Phi_Rk
+
+
+def _normalize_columns(values: np.ndarray) -> np.ndarray:
+ norms = np.linalg.norm(values, ord=2, axis=0)
+ normalized = np.zeros_like(values, dtype=np.float64)
+ nonzero = norms > 0
+ normalized[:, nonzero] = values[:, nonzero] / norms[nonzero]
+ return normalized
diff --git a/scarf/knn_utils.py b/scarf/knn_utils.py
index 92606541..2b7b2a35 100644
--- a/scarf/knn_utils.py
+++ b/scarf/knn_utils.py
@@ -1,52 +1,74 @@
"""Utility functions for running the KNN algorithm."""
-from typing import List, Tuple
+from collections.abc import Generator
+from typing import cast
import numpy as np
import pandas as pd
from numba import jit
from scipy.sparse import csr_matrix, coo_matrix
+import zarr
from .ann import AnnStream
-from .utils import tqdmbar, controlled_compute
+from .storage.budget import worker_prefetch_depth
+from .utils import controlled_compute, logger, prefetch_blocks, tqdmbar
from .writers import create_zarr_dataset
__all__ = [
"self_query_knn",
"smoothen_dists",
"export_knn_to_mtx",
+ "run_sgtsne",
"merge_graphs",
"wnn_integration",
]
-def self_query_knn(ann_obj: AnnStream, store, chunk_size: int, nthreads: int) -> float:
+def self_query_knn(
+ ann_obj: AnnStream, store: zarr.Group, chunk_size: int, nthreads: int
+) -> float:
"""Constructs KNN graph.
Args:
- ann_obj ():
- store ():
- chunk_size ():
- nthreads (): Number of threads to use.
+ ann_obj: Fitted AnnStream with reducer and ANN index.
+ store: Zarr group where ``indices`` and ``distances`` arrays are written.
+ chunk_size: Row chunk size for output Zarr arrays.
+ nthreads: Number of threads to use.
Returns:
- None
+ Approximate recall percentage (fraction of queries with a self neighbor).
"""
- def get_transformed_data():
+ def get_transformed_data() -> Generator[np.ndarray, None, None]:
msg = "Identifying neighbors"
- if ann_obj.harmonizedData is None:
- for _i in tqdmbar(
- ann_obj.data.blocks, desc=msg, total=ann_obj.data.numblocks[0]
- ):
- yield ann_obj.reducer(controlled_compute(_i, nthreads))
- else:
- for _i in tqdmbar(
- ann_obj.harmonizedData.blocks,
+ if ann_obj.embeddings is not None:
+ bs = ann_obj.batchSize
+ n_blocks = int(np.ceil(ann_obj.nCells / bs))
+ for start in tqdmbar(
+ range(0, ann_obj.nCells, bs),
desc=msg,
- total=ann_obj.harmonizedData.numblocks[0],
+ total=n_blocks,
):
- yield controlled_compute(_i, nthreads)
+ end = min(start + bs, ann_obj.nCells)
+ yield ann_obj.embeddings[start:end]
+ return
+ if ann_obj.harmonizedData is None:
+ source = ann_obj.data
+
+ def transform(block: np.ndarray) -> np.ndarray:
+ return ann_obj.reducer(controlled_compute(block, nthreads))
+ else:
+ source = ann_obj.harmonizedData
+
+ def transform(block: np.ndarray) -> np.ndarray:
+ return controlled_compute(block, nthreads)
+
+ blocks = prefetch_blocks(
+ source.blocks,
+ transform,
+ max_ahead=worker_prefetch_depth(),
+ )
+ yield from tqdmbar(blocks, desc=msg, total=source.numblocks[0])
from threadpoolctl import threadpool_limits
@@ -62,21 +84,24 @@ def get_transformed_data():
with threadpool_limits(limits=nthreads):
for i in get_transformed_data():
nsample_end = nsample_start + i.shape[0]
- ki, kv, nm = ann_obj.transform_ann(
- i,
- k=n_neighbors,
- self_indices=np.arange(nsample_start, nsample_end),
+ ki, kv, nm = cast(
+ tuple[np.ndarray, np.ndarray, int],
+ ann_obj.transform_ann(
+ i,
+ k=n_neighbors,
+ self_indices=np.arange(nsample_start, nsample_end),
+ ),
)
z_knn[nsample_start:nsample_end, :] = ki
z_dist[nsample_start:nsample_end, :] = kv
nsample_start = nsample_end
tnm += nm
recall = ann_obj.data.shape[0] - tnm
- recall = 100 * recall / ann_obj.data.shape[0]
- return recall
+ recall_pct = 100.0 * recall / ann_obj.data.shape[0]
+ return recall_pct
-def _is_umap_version_new():
+def _is_umap_version_new() -> bool:
import umap
from packaging import version
@@ -86,16 +111,48 @@ def _is_umap_version_new():
return False
-def smoothen_dists(store, z_idx, z_dist, lc: float, bw: float, chunk_size: int):
+def _patch_null_weights(
+ zgw: zarr.Array,
+ null_positions: list[int],
+ fill_value: float,
+ patch_chunk: int,
+) -> None:
+ """Patch zero edge weights without loading the full weights array."""
+ if not null_positions:
+ return
+ null_positions_arr = np.asarray(null_positions, dtype=np.int64)
+ n_weights = zgw.shape[0]
+ for chunk_start in range(0, n_weights, patch_chunk):
+ chunk_end = min(chunk_start + patch_chunk, n_weights)
+ in_chunk = (null_positions_arr >= chunk_start) & (
+ null_positions_arr < chunk_end
+ )
+ if not np.any(in_chunk):
+ continue
+ local_idx = null_positions_arr[in_chunk] - chunk_start
+ block = np.asarray(zgw[chunk_start:chunk_end], dtype=np.float64)
+ block[local_idx] = fill_value
+ zgw[chunk_start:chunk_end] = block
+ return
+
+
+def smoothen_dists(
+ store: zarr.Group,
+ z_idx: zarr.Array,
+ z_dist: zarr.Array,
+ lc: float,
+ bw: float,
+ chunk_size: int,
+) -> None:
"""Smoothens KNN distances.
Args:
- store ():
- z_idx ():
- z_dist ():
- lc ():
- bw ():
- chunk_size ():
+ store: Zarr group for edge arrays.
+ z_idx: Zarr array of KNN neighbor indices.
+ z_dist: Zarr array of KNN distances.
+ lc: UMAP local connectivity.
+ bw: UMAP bandwidth.
+ chunk_size: Row batch size for streaming smooth knn.
Returns:
None
@@ -107,31 +164,38 @@ def smoothen_dists(store, z_idx, z_dist, lc: float, bw: float, chunk_size: int):
n_cells, n_neighbors = z_idx.shape
zge = create_zarr_dataset(
store,
- f"edges",
+ "edges",
(chunk_size * n_neighbors,),
("u8", "u8"),
(n_cells * n_neighbors, 2),
)
zgw = create_zarr_dataset(
- store, f"weights", (chunk_size * n_neighbors,), "f8", (n_cells * n_neighbors,)
+ store, "weights", (chunk_size * n_neighbors,), "f8", (n_cells * n_neighbors,)
)
last_row = 0
val_counts = 0
- null_idx = []
+ null_positions: list[int] = []
global_min = 1
for i in tqdmbar(range(0, n_cells, chunk_size), desc="Smoothening KNN distances"):
if i + chunk_size > n_cells:
ki, kv = z_idx[i:n_cells, :], z_dist[i:n_cells, :]
else:
ki, kv = z_idx[i : i + chunk_size, :], z_dist[i : i + chunk_size, :]
- kv = kv.astype(np.float32, order="C")
+ ki_arr = np.asarray(ki)
+ kv_arr = np.asarray(kv, dtype=np.float32)
+ if kv_arr.flags.c_contiguous is False:
+ kv_arr = np.ascontiguousarray(kv_arr)
sigmas, rhos = smooth_knn_dist(
- kv, k=n_neighbors, local_connectivity=lc, bandwidth=bw
+ kv_arr, k=n_neighbors, local_connectivity=lc, bandwidth=bw
)
if umap_is_latest:
- rows, cols, vals, _ = compute_membership_strengths(ki, kv, sigmas, rhos)
+ rows, cols, vals, _ = compute_membership_strengths(
+ ki_arr, kv_arr, sigmas, rhos
+ )
else:
- rows, cols, vals = compute_membership_strengths(ki, kv, sigmas, rhos)
+ rows, cols, vals = compute_membership_strengths(
+ ki_arr, kv_arr, sigmas, rhos
+ )
rows = rows + last_row
start = val_counts
end = val_counts + len(rows)
@@ -141,25 +205,20 @@ def smoothen_dists(store, z_idx, z_dist, lc: float, bw: float, chunk_size: int):
zge[start:end, 1] = cols
zgw[start:end] = vals
- # Fixing edges with 0 weights
- # We are doing these steps here to have minimum operations outside
- # the scope of a progress bar
- nidx = vals == 0
- if nidx.sum() > 0:
- min_val = vals[~nidx].min()
- if min_val < global_min:
- global_min = min_val
- null_idx.extend(nidx)
-
- # The whole zarr array needs to copied, modified and written back.
- # Or is this assumption wrong?
- w = zgw[:]
- w[null_idx] = global_min
- zgw[:] = w
+ local_null = np.flatnonzero(vals == 0)
+ if local_null.size > 0:
+ nz_vals = vals[vals != 0]
+ if nz_vals.size > 0:
+ min_val = nz_vals.min()
+ if min_val < global_min:
+ global_min = min_val
+ null_positions.extend((start + local_null).tolist())
+
+ _patch_null_weights(zgw, null_positions, global_min, chunk_size * n_neighbors)
return None
-def export_knn_to_mtx(mtx: str, csr_graph, batch_size: int = 1000) -> None:
+def export_knn_to_mtx(mtx: str, csr_graph: csr_matrix, batch_size: int = 1000) -> None:
"""Exports KNN matrix in Matrix Market format.
Args:
@@ -192,6 +251,118 @@ def export_knn_to_mtx(mtx: str, csr_graph, batch_size: int = 1000) -> None:
return None
+def run_sgtsne(
+ graph: csr_matrix | coo_matrix,
+ ini_embed: np.ndarray,
+ *,
+ tsne_dims: int = 2,
+ max_iter: int = 500,
+ early_iter: int = 200,
+ alpha: int = 10,
+ lambda_scale: float = 1.0,
+ box_h: float = 0.7,
+ temp_file_loc: str = ".",
+ verbose: bool = True,
+ parallel: bool = False,
+ nthreads: int = 1,
+) -> np.ndarray:
+ """Run SG-t-SNE on a sparse graph.
+
+ Uses the ``sgtsne`` executable when available, otherwise falls back to the
+ ``sgtsnepi`` Python package.
+
+ Args:
+ graph: Sparse cell-neighbourhood graph.
+ ini_embed: Initial embedding with shape (n_cells, tsne_dims).
+ tsne_dims: Number of tSNE dimensions.
+ max_iter: Maximum number of iterations.
+ early_iter: Number of early exaggeration iterations.
+ alpha: Early exaggeration multiplier.
+ lambda_scale: Lambda rescaling parameter.
+ box_h: Grid side length (accuracy control).
+ temp_file_loc: Directory for temporary files used by the CLI backend.
+ verbose: Whether to print SG-t-SNE logs.
+ parallel: Whether to run tSNE in parallel mode (CLI backend only).
+ nthreads: Number of threads for parallel CLI runs.
+
+ Returns:
+ Embedding array with shape (tsne_dims, n_cells).
+ """
+ import os
+ import shutil
+ from pathlib import Path
+ from uuid import uuid4
+
+ from loguru import logger
+
+ from .utils import system_call
+
+ n_cells = graph.shape[0]
+ ini_embed = np.asarray(ini_embed)
+ if ini_embed.shape == (n_cells * tsne_dims,):
+ ini_embed = ini_embed.reshape(n_cells, tsne_dims)
+ if ini_embed.shape != (n_cells, tsne_dims):
+ raise ValueError(
+ f"ini_embed must have shape ({n_cells}, {tsne_dims}), got {ini_embed.shape}"
+ )
+
+ if shutil.which("sgtsne") is not None:
+ uid = str(uuid4())
+ knn_mtx_fn = Path(temp_file_loc, f"{uid}.mtx").resolve()
+ export_knn_to_mtx(str(knn_mtx_fn), graph)
+ ini_emb_fn = Path(temp_file_loc, f"{uid}.txt").resolve()
+ with open(ini_emb_fn, "w") as h:
+ h.write("\n".join(map(str, ini_embed.flatten())))
+ out_fn = Path(temp_file_loc, f"{uid}_output.txt").resolve()
+ threads = nthreads if parallel else 1
+ cmd = (
+ f"sgtsne -m {max_iter} -l {lambda_scale} -d {tsne_dims} -e {early_iter} "
+ f"-p {threads} -a {alpha} -h {box_h} -i {ini_emb_fn} -o {out_fn} {knn_mtx_fn}"
+ )
+ if verbose:
+ system_call(cmd)
+ else:
+ os.system(cmd)
+ try:
+ emb = np.asarray(
+ pd.read_csv(out_fn, header=None, sep=" ")[
+ list(range(tsne_dims))
+ ].values.T
+ )
+ finally:
+ for fn in (out_fn, knn_mtx_fn, ini_emb_fn):
+ if fn.exists():
+ fn.unlink()
+ return emb
+
+ try:
+ from sgtsnepi import sgtsnepi
+ except ImportError as exc:
+ raise ImportError(
+ "SG-t-SNE requires the sgtsne executable on PATH or the sgtsnepi package."
+ ) from exc
+
+ if parallel:
+ logger.warning(
+ "parallel=True is not supported by the sgtsnepi Python backend; "
+ "running single-threaded"
+ )
+
+ return np.asarray(
+ sgtsnepi(
+ graph,
+ y0=ini_embed.T,
+ d=tsne_dims,
+ max_iter=max_iter,
+ early_exag=early_iter,
+ lambda_par=lambda_scale,
+ h=box_h,
+ alpha=alpha,
+ silent=not verbose,
+ )
+ )
+
+
@jit(nopython=True)
def calc_snn(indices: np.ndarray) -> np.ndarray:
"""Calculates shared nearest neighbour between each node and its neighbour.
@@ -207,12 +378,12 @@ def calc_snn(indices: np.ndarray) -> np.ndarray:
for j in range(nk):
k = indices[i][j]
snn[i][j] = len(set(indices[i]).intersection(set(indices[k])))
- return snn / (nk - 1)
+ return np.asarray(snn / (nk - 1))
def weight_sort_indices(
i: np.ndarray, w: np.ndarray, wn: np.ndarray, n: int
-) -> Tuple[np.ndarray, np.ndarray]:
+) -> tuple[np.ndarray, np.ndarray]:
"""Sort the array i and w based on values of wn. Only keep the top n
values.
@@ -230,12 +401,12 @@ def weight_sort_indices(
i = i[idx]
w = w[idx]
# Removing duplicate neighbours
- _, idx = np.unique(i, return_index=True)
- idx = sorted(idx)
- return i[idx][:n], w[idx][:n]
+ _, unique_idx = np.unique(i, return_index=True)
+ unique_idx_arr = np.asarray(sorted(unique_idx))
+ return i[unique_idx_arr][:n], w[unique_idx_arr][:n]
-def merge_graphs(csr_mats: List[csr_matrix]) -> coo_matrix:
+def merge_graphs(csr_mats: list[csr_matrix]) -> coo_matrix:
"""Merge multiple graphs of same size and shape such that the merged graph
have the same size and shape. Edge values are sorted based on their weight
and the shared neighbours.
@@ -243,8 +414,8 @@ def merge_graphs(csr_mats: List[csr_matrix]) -> coo_matrix:
Args:
csr_mats: A list of two or more CSR matrices representing the graphs to be merged.
- Returns: A merged graph in CSR matrix form.
- The merged graph has the same number of edges as each input graph
+ Returns:
+ coo_matrix: Merged graph with the same shape and edge count as inputs.
"""
try:
assert len(set([x.shape for x in csr_mats])) == 1
@@ -261,7 +432,8 @@ def merge_graphs(csr_mats: List[csr_matrix]) -> coo_matrix:
snns.append(
calc_snn(mat.indices.reshape((mat.shape[0], mat[0].indices.shape[0])))
)
- col, data = [], []
+ col: list[int] = []
+ data: list[float] = []
for i in tqdmbar(range(csr_mats[0].shape[0]), desc="Merging graph edges"):
mi = np.hstack([mat[i].indices for mat in csr_mats])
mwn = np.hstack([mat[i].data + snns[n][i] for n, mat in enumerate(csr_mats)])
@@ -275,80 +447,154 @@ def merge_graphs(csr_mats: List[csr_matrix]) -> coo_matrix:
def wnn_integration(
- name1: str, g1: csr_matrix, ld1, name2: str, g2: csr_matrix, ld2, n_threads: int
-):
- """
+ name1: str,
+ g1: csr_matrix,
+ ld1: np.ndarray,
+ name2: str,
+ g2: csr_matrix,
+ ld2: np.ndarray,
+ n_threads: int,
+) -> coo_matrix:
+ """Build a weighted nearest-neighbor graph from two modality-specific KNN graphs.
Args:
- name1:
- g1:
- ld1:
- name2:
- g2:
- ld2:
- n_threads:
+ name1: Label for the first modality (used in log messages).
+ g1: CSR KNN graph for modality 1.
+ ld1: Latent embedding matrix for modality 1, shape (n_cells, n_dims).
+ name2: Label for the second modality.
+ g2: CSR KNN graph for modality 2.
+ ld2: Latent embedding matrix for modality 2.
+ n_threads: Thread limit for affinity calculations.
Returns:
-
+ coo_matrix: WNN graph combining both modalities.
"""
- def make_estimates(g, ld, msg=""):
- return np.array(
- [
- ld[g[i].indices].mean(axis=0)
- for i in tqdmbar(range(g.shape[0]), desc=msg)
- ]
- )
-
- def get_kth_l(g, ld, k):
- return ld[[g[x].indices[k] for x in range(g.shape[0])]]
-
- def calc_theta(ld, le, b, c):
- a = np.sqrt(((ld - le) ** 2).sum(axis=1))
- d = a - b
- d[d < 0] = 0
- return np.exp(((-1 * d) / (c - d)).astype(np.float128))
+ g1 = g1.tocsr(copy=False)
+ g2 = g2.tocsr(copy=False)
+ ld1 = np.asarray(ld1)
+ ld2 = np.asarray(ld2)
+
+ if g1.shape != g2.shape:
+ raise ValueError("WNN graphs must have the same shape")
+ if g1.shape[0] != g1.shape[1] or g1.shape[0] == 0:
+ raise ValueError("WNN graphs must be non-empty square matrices")
+
+ n_cells = g1.shape[0]
+ for name, graph in ((name1, g1), (name2, g2)):
+ row_degrees = np.diff(graph.indptr)
+ if np.unique(row_degrees).size != 1:
+ raise ValueError(f"WNN graph for {name} must have a regular row degree")
+ if row_degrees[0] < 2:
+ raise ValueError(
+ f"WNN graph for {name} must have at least two neighbors per cell"
+ )
- def calc_affinity_ratios(
- g_self, g_other, ld, sigma: int = -2, epsilon: float = 10e-4, name=""
- ):
- l_self = make_estimates(
- g_self, ld, msg=f"({name}) Predicting within modality profile"
- )
- l_cross = make_estimates(
- g_other, ld, msg=f"({name}) Predicting cross modality profile"
+ k1 = int(np.diff(g1.indptr)[0])
+ k2 = int(np.diff(g2.indptr)[0])
+ nk = min(k1, k2)
+ if k1 != k2:
+ logger.warning(
+ f"WNN graphs have different neighbor counts ({name1}: {k1}, "
+ f"{name2}: {k2}). The integrated graph will retain {nk} "
+ "neighbors per cell."
)
- b = np.sqrt(((ld - get_kth_l(g_self, ld, 0)) ** 2).sum(axis=1))
- c = np.sqrt(((ld - get_kth_l(g_self, ld, sigma)) ** 2).sum(axis=1))
-
- theta_self = calc_theta(ld, l_self, b, c)
- theta_cross = calc_theta(ld, l_cross, b, c)
+ for name, ld in ((name1, ld1), (name2, ld2)):
+ if ld.ndim != 2 or ld.shape[1] == 0:
+ raise ValueError(f"WNN embedding for {name} must be a non-empty matrix")
+ if ld.shape[0] != n_cells:
+ raise ValueError(
+ f"WNN embedding for {name} must have one row per graph cell"
+ )
+ if not np.all(np.isfinite(ld)):
+ raise ValueError(f"WNN embedding for {name} contains non-finite values")
+
+ def calc_theta(
+ point: np.ndarray,
+ estimate: np.ndarray,
+ nearest_distance: float,
+ anchor_distance: float,
+ ) -> float:
+ estimate_distance = float(np.sqrt(((point - estimate) ** 2).sum(axis=0)))
+ adjusted_distance = max(estimate_distance - nearest_distance, 0)
+ bandwidth = max(anchor_distance - nearest_distance, np.finfo(np.float64).eps)
+ return float(np.exp(-adjusted_distance / bandwidth))
+
+ def calc_affinity_ratio(
+ point: np.ndarray,
+ self_estimate: np.ndarray,
+ cross_estimate: np.ndarray,
+ nearest_distance: float,
+ anchor_distance: float,
+ epsilon: float = 1e-4,
+ ) -> float:
+ theta_self = calc_theta(point, self_estimate, nearest_distance, anchor_distance)
+ theta_cross = calc_theta(
+ point, cross_estimate, nearest_distance, anchor_distance
+ )
+ return float(np.clip(theta_self / (theta_cross + epsilon), 0, 200))
- return (theta_self / (theta_cross + epsilon)).astype(np.float128)
+ neighbor_indices1 = g1.indices.reshape(n_cells, k1)
+ neighbor_indices2 = g2.indices.reshape(n_cells, k2)
from threadpoolctl import threadpool_limits
+ col_parts: list[np.ndarray] = []
+ data_parts: list[np.ndarray] = []
with threadpool_limits(limits=n_threads):
- sr = calc_affinity_ratios(g1, g2, ld1, name=name1)
- sp = calc_affinity_ratios(g2, g1, ld2, name=name2)
-
- wr = np.exp(sr) / (np.exp(sr) + np.exp(sp))
- wp = np.exp(sp) / (np.exp(sr) + np.exp(sp))
-
- nk = g1[0].indices.shape[0]
- col, data = [], []
- for n in tqdmbar(range(g1.shape[0]), desc="Building WNN graph"):
- mixed_k = np.array(sorted(set(g1[n].indices).union(g2[n].indices)))
- dr = np.sqrt(((ld1[n] - ld1[mixed_k]) ** 2).sum(axis=1)) * wr[n]
- dp = np.sqrt(((ld2[n] - ld2[mixed_k]) ** 2).sum(axis=1)) * wp[n]
- w_d = dr + dp
- idx = np.argsort(w_d)[:nk]
- col.append(mixed_k[idx])
- v = w_d[idx]
- data.append(np.exp(-((v - v[0]) / (v - v[0]).mean())))
-
- data = np.hstack(data)
- row = np.repeat(range(g1.shape[0]), nk)
- col = np.hstack(col)
- return coo_matrix((data, (row, col)), shape=g1.shape)
+ for n in tqdmbar(range(n_cells), desc="Building WNN graph"):
+ neighbors1 = neighbor_indices1[n]
+ neighbors2 = neighbor_indices2[n]
+ mixed_k = np.union1d(neighbors1, neighbors2)
+
+ distances1 = np.sqrt(((ld1[n] - ld1[mixed_k]) ** 2).sum(axis=1))
+ distances2 = np.sqrt(((ld2[n] - ld2[mixed_k]) ** 2).sum(axis=1))
+
+ self_distances1 = distances1[np.searchsorted(mixed_k, neighbors1)]
+ self_distances2 = distances2[np.searchsorted(mixed_k, neighbors2)]
+ ranked_distances1 = np.sort(self_distances1)
+ ranked_distances2 = np.sort(self_distances2)
+
+ score1 = calc_affinity_ratio(
+ ld1[n],
+ ld1[neighbors1].mean(axis=0),
+ ld1[neighbors2].mean(axis=0),
+ float(ranked_distances1[0]),
+ float(ranked_distances1[-2]),
+ )
+ score2 = calc_affinity_ratio(
+ ld2[n],
+ ld2[neighbors2].mean(axis=0),
+ ld2[neighbors1].mean(axis=0),
+ float(ranked_distances2[0]),
+ float(ranked_distances2[-2]),
+ )
+
+ max_score = max(score1, score2)
+ exp1 = np.exp(score1 - max_score)
+ exp2 = np.exp(score2 - max_score)
+ normalizer = exp1 + exp2
+ weighted_distances = distances1 * (exp1 / normalizer) + distances2 * (
+ exp2 / normalizer
+ )
+
+ idx = np.argsort(weighted_distances)[:nk]
+ col_parts.append(mixed_k[idx])
+ selected_distances = weighted_distances[idx]
+ offsets = selected_distances - selected_distances[0]
+ scale = offsets.mean()
+ if scale <= np.finfo(np.float64).eps:
+ edge_weights = np.ones(selected_distances.shape, dtype=np.float64)
+ else:
+ edge_weights = np.maximum(
+ np.exp(-(offsets / scale)), np.finfo(np.float64).tiny
+ )
+ data_parts.append(edge_weights)
+
+ merged_data = np.hstack(data_parts)
+ row = np.repeat(np.arange(n_cells), nk)
+ merged_col = np.hstack(col_parts)
+ if not np.all(np.isfinite(merged_data)):
+ raise FloatingPointError("WNN integration produced non-finite graph weights")
+ return coo_matrix((merged_data, (row, merged_col)), shape=g1.shape)
diff --git a/scarf/mapping_reference.py b/scarf/mapping_reference.py
new file mode 100644
index 00000000..7e05351d
--- /dev/null
+++ b/scarf/mapping_reference.py
@@ -0,0 +1,417 @@
+"""Persistent handles for immutable Symphony-style mapping references."""
+
+from dataclasses import dataclass
+import hashlib
+import json
+from typing import TYPE_CHECKING, Any, cast
+import warnings
+
+import numpy as np
+import zarr
+
+from ._types import as_zarr_array, as_zarr_group
+from .symphony import SYMPHONY_STYLE_VARIANT, SymphonyReferenceModel
+from .writers import create_zarr_dataset, create_zarr_obj_array
+
+if TYPE_CHECKING:
+ import pandas as pd
+
+ from .assay import Assay
+
+
+MAPPING_REFERENCE_SCHEMA_VERSION = 2
+MAPPING_REFERENCE_GROUP = "mappingReference"
+MAPPING_REFERENCES_GROUP = "mappingReferences"
+LATEST_MAPPING_REFERENCE_ATTRIBUTE = "latestMappingReference"
+
+
+@dataclass(frozen=True)
+class MappingResult:
+ projection_path: str
+ n_cells: int
+ correction_method: str
+ diagnostics: dict[str, float | str]
+ indices: np.ndarray | None = None
+ distances: np.ndarray | None = None
+ uncorrected_latent: np.ndarray | None = None
+ corrected_latent: np.ndarray | None = None
+ uninformative: np.ndarray | None = None
+
+
+@dataclass(frozen=True)
+class MappingReference:
+ """An immutable Symphony-style reference loaded from a SCARF Zarr store."""
+
+ datastore: Any
+ assay_name: str
+ cell_key: str
+ feature_key: str
+ reduction_path: str
+ ann_path: str
+ artifact_path: str
+ model: SymphonyReferenceModel
+ feature_ids: np.ndarray
+ metadata: dict[str, Any]
+ reference_distance_quantiles: np.ndarray | None = None
+ reference_distance_values: np.ndarray | None = None
+
+ def map_query(
+ self,
+ target_assay: "Assay",
+ target_name: str,
+ target_feat_key: str,
+ target_cell_key: str = "I",
+ save_k: int = 3,
+ query_batches: "pd.DataFrame | None" = None,
+ correction_method: str = "symphony",
+ missing_feature_policy: str = "reference_mean",
+ result_store: zarr.Group | None = None,
+ ) -> MappingResult:
+ """Map a query into the fixed reference coordinate system.
+
+ A writable reference stores results under its projection group. A
+ read-only reference returns arrays in memory unless ``result_store`` is
+ supplied.
+ """
+ return cast(
+ MappingResult,
+ self.datastore._map_with_mapping_reference(
+ self,
+ target_assay=target_assay,
+ target_name=target_name,
+ target_feat_key=target_feat_key,
+ target_cell_key=target_cell_key,
+ save_k=save_k,
+ query_batches=query_batches,
+ correction_method=correction_method,
+ missing_feature_policy=missing_feature_policy,
+ result_store=result_store,
+ ),
+ )
+
+
+def persist_mapping_reference(
+ reduction_group: zarr.Group,
+ model: SymphonyReferenceModel,
+ feature_ids: np.ndarray,
+ metadata: dict[str, Any],
+ reference_distance_quantiles: np.ndarray | None = None,
+ reference_distance_values: np.ndarray | None = None,
+) -> str:
+ """Write a complete immutable reference artifact into a reduction group."""
+ model = SymphonyReferenceModel(
+ feature_means=np.asarray(model.feature_means, dtype=np.float64),
+ feature_scales=np.asarray(model.feature_scales, dtype=np.float64),
+ loadings=np.asarray(model.loadings, dtype=np.float64),
+ centroids=np.asarray(model.centroids, dtype=np.float64),
+ raw_centroids=np.asarray(model.raw_centroids, dtype=np.float64),
+ corrected_centroids=np.asarray(model.corrected_centroids, dtype=np.float64),
+ cluster_mass=np.asarray(model.cluster_mass, dtype=np.float64),
+ sigma=np.asarray(model.sigma, dtype=np.float64),
+ correction_ridge=float(model.correction_ridge),
+ )
+ if reference_distance_quantiles is not None:
+ reference_distance_quantiles = np.asarray(
+ reference_distance_quantiles, dtype=np.float64
+ )
+ if reference_distance_values is not None:
+ reference_distance_values = np.asarray(
+ reference_distance_values, dtype=np.float64
+ )
+ metadata = {**metadata, "algorithmVariant": SYMPHONY_STYLE_VARIANT}
+ artifact_hash = mapping_reference_hash(
+ model,
+ feature_ids,
+ metadata,
+ reference_distance_quantiles,
+ reference_distance_values,
+ )
+ if MAPPING_REFERENCES_GROUP not in reduction_group:
+ reduction_group.create_group(MAPPING_REFERENCES_GROUP)
+ references = as_zarr_group(
+ reduction_group[MAPPING_REFERENCES_GROUP], name=MAPPING_REFERENCES_GROUP
+ )
+ if artifact_hash in references:
+ existing = as_zarr_group(references[artifact_hash], name=artifact_hash)
+ if bool(existing.attrs.get("complete", False)):
+ _validate_mapping_reference_hash(existing, artifact_hash)
+ reduction_group.attrs[LATEST_MAPPING_REFERENCE_ATTRIBUTE] = artifact_hash
+ return f"{MAPPING_REFERENCES_GROUP}/{artifact_hash}"
+ del references[artifact_hash]
+ group = references.create_group(artifact_hash)
+ group.attrs["schemaVersion"] = MAPPING_REFERENCE_SCHEMA_VERSION
+ group.attrs["complete"] = False
+ group.attrs["artifactHash"] = artifact_hash
+ for key, value in metadata.items():
+ group.attrs[key] = value
+ try:
+ create_zarr_obj_array(group, "featureIds", np.asarray(feature_ids))
+ _write_array(group, "featureMeans", model.feature_means)
+ _write_array(group, "featureScales", model.feature_scales)
+ _write_array(group, "loadings", model.loadings)
+ _write_array(group, "centroids", model.centroids)
+ _write_array(group, "rawCentroids", model.raw_centroids)
+ _write_array(group, "correctedCentroids", model.corrected_centroids)
+ _write_array(group, "clusterMass", model.cluster_mass)
+ _write_array(group, "sigma", model.sigma)
+ if reference_distance_quantiles is not None:
+ _write_array(
+ group,
+ "referenceDistanceQuantiles",
+ reference_distance_quantiles,
+ )
+ if reference_distance_values is not None:
+ _write_array(
+ group,
+ "referenceDistanceValues",
+ reference_distance_values,
+ )
+ group.attrs["correctionRidge"] = float(model.correction_ridge)
+ group.attrs["complete"] = True
+ reduction_group.attrs[LATEST_MAPPING_REFERENCE_ATTRIBUTE] = artifact_hash
+ except Exception:
+ group.attrs["complete"] = False
+ raise
+ return f"{MAPPING_REFERENCES_GROUP}/{artifact_hash}"
+
+
+def _load_symphony_model_from_group(group: zarr.Group) -> SymphonyReferenceModel:
+ return SymphonyReferenceModel(
+ feature_means=np.asarray(
+ as_zarr_array(group["featureMeans"], name="featureMeans")[:]
+ ),
+ feature_scales=np.asarray(
+ as_zarr_array(group["featureScales"], name="featureScales")[:]
+ ),
+ loadings=np.asarray(as_zarr_array(group["loadings"], name="loadings")[:]),
+ centroids=np.asarray(as_zarr_array(group["centroids"], name="centroids")[:]),
+ raw_centroids=np.asarray(
+ as_zarr_array(group["rawCentroids"], name="rawCentroids")[:]
+ ),
+ corrected_centroids=np.asarray(
+ as_zarr_array(group["correctedCentroids"], name="correctedCentroids")[:]
+ ),
+ cluster_mass=np.asarray(
+ as_zarr_array(group["clusterMass"], name="clusterMass")[:]
+ ),
+ sigma=np.asarray(as_zarr_array(group["sigma"], name="sigma")[:]),
+ correction_ridge=_number_attribute(group, "correctionRidge"),
+ )
+
+
+def load_mapping_reference(
+ datastore: Any,
+ assay_name: str,
+ cell_key: str,
+ feature_key: str,
+ reduction_path: str,
+ ann_path: str,
+) -> MappingReference:
+ """Load a persisted reference after the caller has validated its provenance."""
+ reduction_group = as_zarr_group(datastore.zw[reduction_path], name=reduction_path)
+ group, relative_path, is_legacy = resolve_mapping_reference_group(reduction_group)
+ group_path = f"{reduction_path}/{relative_path}"
+ group = as_zarr_group(datastore.zw[group_path], name=group_path)
+ if not bool(group.attrs.get("complete", False)):
+ raise ValueError(
+ "Mapping reference is incomplete. Rebuild the harmonized reference."
+ )
+ schema_version = group.attrs.get("schemaVersion")
+ if is_legacy:
+ if schema_version != 1:
+ raise ValueError(
+ "Legacy mapping reference schema is incompatible. Rebuild the reference."
+ )
+ warnings.warn(
+ "This mapping reference uses the legacy overwrite-in-place layout. "
+ "Rebuild it to create a content-addressed artifact.",
+ DeprecationWarning,
+ stacklevel=3,
+ )
+ elif schema_version != MAPPING_REFERENCE_SCHEMA_VERSION:
+ raise ValueError(
+ "Mapping reference schema is incompatible. Rebuild the harmonized reference."
+ )
+ if not is_legacy:
+ validate_mapping_reference_artifact(group)
+ model = _load_symphony_model_from_group(group)
+ return MappingReference(
+ datastore=datastore,
+ assay_name=assay_name,
+ cell_key=cell_key,
+ feature_key=feature_key,
+ reduction_path=reduction_path,
+ ann_path=ann_path,
+ artifact_path=group_path,
+ model=model,
+ feature_ids=np.asarray(
+ as_zarr_array(group["featureIds"], name="featureIds")[:]
+ ),
+ metadata=dict(group.attrs),
+ reference_distance_quantiles=(
+ np.asarray(
+ as_zarr_array(
+ group["referenceDistanceQuantiles"],
+ name="referenceDistanceQuantiles",
+ )[:]
+ )
+ if "referenceDistanceQuantiles" in group
+ else None
+ ),
+ reference_distance_values=(
+ np.asarray(
+ as_zarr_array(
+ group["referenceDistanceValues"],
+ name="referenceDistanceValues",
+ )[:]
+ )
+ if "referenceDistanceValues" in group
+ else None
+ ),
+ )
+
+
+def validate_mapping_reference_artifact(group: zarr.Group) -> None:
+ """Validate a content-addressed mapping-reference artifact."""
+ if not bool(group.attrs.get("complete", False)):
+ raise ValueError(
+ "Mapping reference is incomplete. Rebuild the harmonized reference."
+ )
+ if group.attrs.get("schemaVersion") != MAPPING_REFERENCE_SCHEMA_VERSION:
+ raise ValueError(
+ "Mapping reference schema is incompatible. Rebuild the harmonized reference."
+ )
+ artifact_hash = group.attrs.get("artifactHash")
+ if not isinstance(artifact_hash, str):
+ raise ValueError("Mapping reference is missing its artifact hash.")
+ group_name = str(getattr(group, "path", "")).rstrip("/").rsplit("/", 1)[-1]
+ if group_name and group_name != artifact_hash:
+ raise ValueError("Mapping reference path does not match its artifact hash.")
+ _validate_mapping_reference_hash(group, artifact_hash)
+
+
+def resolve_mapping_reference_group(
+ reduction_group: zarr.Group,
+) -> tuple[zarr.Group, str, bool]:
+ artifact_hash = reduction_group.attrs.get(LATEST_MAPPING_REFERENCE_ATTRIBUTE)
+ if artifact_hash is not None and MAPPING_REFERENCES_GROUP in reduction_group:
+ relative_path = f"{MAPPING_REFERENCES_GROUP}/{artifact_hash}"
+ if relative_path in reduction_group:
+ return (
+ as_zarr_group(reduction_group[relative_path], name=relative_path),
+ relative_path,
+ False,
+ )
+ if MAPPING_REFERENCE_GROUP in reduction_group:
+ return (
+ as_zarr_group(
+ reduction_group[MAPPING_REFERENCE_GROUP],
+ name=MAPPING_REFERENCE_GROUP,
+ ),
+ MAPPING_REFERENCE_GROUP,
+ True,
+ )
+ raise KeyError("No mapping-reference artifact exists in this reduction")
+
+
+def mapping_reference_hash(
+ model: SymphonyReferenceModel,
+ feature_ids: np.ndarray,
+ metadata: dict[str, Any],
+ reference_distance_quantiles: np.ndarray | None = None,
+ reference_distance_values: np.ndarray | None = None,
+) -> str:
+ from .mapping_utils import array_hash
+
+ digest = hashlib.sha256()
+ for values in (
+ feature_ids,
+ model.feature_means,
+ model.feature_scales,
+ model.loadings,
+ model.centroids,
+ model.raw_centroids,
+ model.corrected_centroids,
+ model.cluster_mass,
+ model.sigma,
+ ):
+ digest.update(array_hash(np.asarray(values)).encode())
+ for optional_values in (
+ reference_distance_quantiles,
+ reference_distance_values,
+ ):
+ if optional_values is not None:
+ digest.update(array_hash(np.asarray(optional_values)).encode())
+ digest.update(repr(float(model.correction_ridge)).encode())
+ digest.update(
+ json.dumps(
+ metadata, sort_keys=True, separators=(",", ":"), default=str
+ ).encode()
+ )
+ return digest.hexdigest()
+
+
+def _validate_mapping_reference_hash(group: zarr.Group, expected_hash: str) -> None:
+ model = _load_symphony_model_from_group(group)
+ feature_ids = np.asarray(as_zarr_array(group["featureIds"], name="featureIds")[:])
+ distance_quantiles = (
+ np.asarray(
+ as_zarr_array(
+ group["referenceDistanceQuantiles"],
+ name="referenceDistanceQuantiles",
+ )[:]
+ )
+ if "referenceDistanceQuantiles" in group
+ else None
+ )
+ distance_values = (
+ np.asarray(
+ as_zarr_array(
+ group["referenceDistanceValues"],
+ name="referenceDistanceValues",
+ )[:]
+ )
+ if "referenceDistanceValues" in group
+ else None
+ )
+ metadata = {
+ key: value
+ for key, value in dict(group.attrs).items()
+ if key
+ not in {
+ "artifactHash",
+ "complete",
+ "correctionRidge",
+ "schemaVersion",
+ }
+ }
+ actual_hash = mapping_reference_hash(
+ model,
+ feature_ids,
+ metadata,
+ distance_quantiles,
+ distance_values,
+ )
+ if actual_hash != expected_hash:
+ raise ValueError(
+ "Mapping reference content does not match its artifact hash. "
+ "Rebuild the reference."
+ )
+
+
+def _write_array(group: zarr.Group, name: str, values: np.ndarray) -> None:
+ values = np.asarray(values, dtype=np.float64)
+ chunks = (
+ (min(max(values.shape[0], 1), 10_000),)
+ if values.ndim == 1
+ else (min(max(values.shape[0], 1), 1_000), values.shape[1])
+ )
+ array = create_zarr_dataset(group, name, chunks, "f8", values.shape)
+ array[...] = values
+
+
+def _number_attribute(group: zarr.Group, name: str) -> float:
+ value = group.attrs[name]
+ if not isinstance(value, int | float):
+ raise ValueError(f"Mapping reference attribute {name!r} must be numeric")
+ return float(value)
diff --git a/scarf/mapping_utils.py b/scarf/mapping_utils.py
index cd10f5cb..577bf8bd 100644
--- a/scarf/mapping_utils.py
+++ b/scarf/mapping_utils.py
@@ -1,53 +1,241 @@
-"""Utility functions for the mapping."""
+"""Numerical and feature-alignment utilities for reference mapping."""
-from typing import Tuple
+import hashlib
+from typing import Any, cast
-import dask.array as daskarr
import numpy as np
import pandas as pd
from .assay import Assay
-from .utils import controlled_compute, show_dask_progress, logger, tqdmbar
+from .chunked import ChunkedArray
+from .utils import controlled_compute, show_dask_progress, logger
-__all__ = ["align_features", "coral"]
+__all__ = [
+ "align_features",
+ "array_hash",
+ "array_store_hash",
+ "conformal_prediction_sets",
+ "coral",
+ "distance_weights",
+]
-def _cov_diaged(da: daskarr) -> daskarr:
- a = daskarr.cov(da, rowvar=0)
- a[a == np.inf] = 0
- a[a == np.nan] = 0
- return a + np.eye(a.shape[0])
-
+def array_hash(values: np.ndarray | list[Any]) -> str:
+ """Return a stable content hash for numeric or identifier arrays."""
+ arr = np.asarray(values)
+ digest = hashlib.sha256()
+ digest.update(str(arr.shape).encode())
+ digest.update(arr.dtype.str.encode())
+ if arr.dtype.kind in {"O", "S", "U"}:
+ digest.update(
+ "\x1f".join(str(value) for value in arr.reshape(-1)).encode("utf-8")
+ )
+ else:
+ digest.update(np.ascontiguousarray(arr).tobytes())
+ return digest.hexdigest()
-def _correlation_alignment(s: daskarr, t: daskarr, nthreads: int) -> daskarr:
- from scipy.linalg import fractional_matrix_power as fmp
- from threadpoolctl import threadpool_limits
- s_cov = show_dask_progress(
- _cov_diaged(s), f"CORAL: Computing source covariance", nthreads
+def array_store_hash(values: Any) -> str:
+ """Hash a row-addressable array without materializing it in memory."""
+ shape = tuple(int(value) for value in values.shape)
+ dtype = np.dtype(values.dtype)
+ digest = hashlib.sha256()
+ digest.update(str(shape).encode())
+ digest.update(dtype.str.encode())
+ if not shape:
+ digest.update(np.asarray(values[...]).tobytes())
+ return digest.hexdigest()
+ chunks = getattr(values, "chunks", None)
+ row_chunk = (
+ int(chunks[0])
+ if chunks is not None and len(chunks) > 0
+ else min(max(shape[0], 1), 10_000)
)
- t_cov = show_dask_progress(
- _cov_diaged(t), f"CORAL: Computing target covariance", nthreads
+ for start in range(0, shape[0], row_chunk):
+ stop = min(start + row_chunk, shape[0])
+ block = np.asarray(values[start:stop])
+ if dtype.kind in {"O", "S", "U"}:
+ digest.update(
+ "\x1f".join(str(value) for value in block.reshape(-1)).encode("utf-8")
+ )
+ else:
+ digest.update(np.ascontiguousarray(block).tobytes())
+ return digest.hexdigest()
+
+
+def _distance_quantile_summary(
+ distances: Any,
+ max_samples: int = 100_000,
+ n_quantiles: int = 1_001,
+) -> tuple[np.ndarray, np.ndarray]:
+ """Summarize first-neighbor distances with deterministic row sampling."""
+ shape = tuple(int(value) for value in distances.shape)
+ if len(shape) not in {1, 2}:
+ raise ValueError("Neighbor distances must be one- or two-dimensional")
+ n_rows = shape[0]
+ if n_rows < 1:
+ raise ValueError("Neighbor distances are empty")
+ if len(shape) == 2 and shape[1] < 1:
+ raise ValueError("Neighbor distances do not contain any neighbors")
+ if max_samples < 1 or n_quantiles < 1:
+ raise ValueError("Sampling and quantile counts must be positive")
+
+ stride = max(int(np.ceil(n_rows / max_samples)), 1)
+ chunks = getattr(distances, "chunks", None)
+ block_size = (
+ int(chunks[0])
+ if chunks is not None and len(chunks) > 0
+ else min(n_rows, 10_000)
)
+ sampled: list[np.ndarray] = []
+ for start in range(0, n_rows, block_size):
+ stop = min(start + block_size, n_rows)
+ block = np.asarray(distances[start:stop])
+ if block.ndim == 2:
+ block = block[:, 0]
+ mask = np.arange(start, stop, dtype=np.int64) % stride == 0
+ sampled.append(np.asarray(block[mask], dtype=np.float64))
+ values = np.concatenate(sampled)
+ quantiles = np.linspace(0.0, 1.0, min(n_quantiles, len(values)))
+ return quantiles, np.quantile(values, quantiles)
+
+
+def distance_weights(distances: np.ndarray) -> np.ndarray:
+ """Convert HNSW squared-L2 distances into normalized inverse-L2 weights."""
+ values = np.asarray(distances, dtype=np.float64)
+ if values.ndim != 2:
+ raise ValueError("Expected a two-dimensional distance array")
+ if not np.all(np.isfinite(values)):
+ raise ValueError("Neighbor distances must be finite")
+ if np.any(values < 0):
+ raise ValueError("Neighbor distances must be non-negative")
+
+ l2_distances = np.sqrt(values)
+ weights = np.zeros_like(l2_distances)
+ zero_mask = l2_distances == 0
+ zero_count = zero_mask.sum(axis=1)
+ rows_with_zero = zero_count > 0
+ if rows_with_zero.any():
+ weights[rows_with_zero] = (
+ zero_mask[rows_with_zero] / zero_count[rows_with_zero, np.newaxis]
+ )
+ rows_without_zero = ~rows_with_zero
+ if rows_without_zero.any():
+ inverse = 1.0 / l2_distances[rows_without_zero]
+ weights[rows_without_zero] = inverse / inverse.sum(axis=1, keepdims=True)
+ return weights
+
+
+def conformal_prediction_sets(
+ label_scores: np.ndarray,
+ calibration_nonconformity: np.ndarray,
+ alpha: float = 0.1,
+) -> np.ndarray:
+ """Return class-membership masks from split-conformal p-values."""
+ scores = np.asarray(label_scores, dtype=np.float64)
+ calibration = np.asarray(calibration_nonconformity, dtype=np.float64)
+ if scores.ndim != 2:
+ raise ValueError("label_scores must be a two-dimensional array")
+ if calibration.ndim != 1 or calibration.size == 0:
+ raise ValueError("calibration_nonconformity must be a non-empty vector")
+ if not 0 < alpha < 1:
+ raise ValueError("alpha must be strictly between zero and one")
+ if not np.all(np.isfinite(scores)) or not np.all(np.isfinite(calibration)):
+ raise ValueError("Conformal inputs must be finite")
+ nonconformity = 1.0 - scores
+ p_values = (
+ (calibration[np.newaxis, np.newaxis, :] >= nonconformity[:, :, np.newaxis]).sum(
+ axis=2
+ )
+ + 1
+ ) / (len(calibration) + 1)
+ return cast(np.ndarray, p_values > alpha)
+
+
+def _streaming_covariance(data: ChunkedArray, nthreads: int, msg: str) -> np.ndarray:
+ """Computes the (features x features) covariance by streaming row-blocks.
+
+ Uses the identity cov = (XtX - n * mean (x) mean) / (n - 1), accumulating
+ the cross-product XtX and the column sums over row-blocks so peak memory
+ stays bounded by a single block plus the small (features x features) matrix.
+ """
+ n_cols = data.shape[1]
+ xtx = np.zeros((n_cols, n_cols), dtype=np.float64)
+ col_sum = np.zeros(n_cols, dtype=np.float64)
+ n_rows = 0
+ for block in data.stream_blocks(nthreads=nthreads, msg=msg):
+ a = block.astype(np.float64, copy=False)
+ xtx += a.T @ a
+ col_sum += a.sum(axis=0)
+ n_rows += a.shape[0]
+ if n_rows < 2:
+ raise ValueError("CORAL requires at least two cells in each dataset")
+ mean = col_sum / n_rows
+ cov = (xtx - n_rows * np.outer(mean, mean)) / (n_rows - 1)
+ if not np.all(np.isfinite(cov)):
+ raise ValueError("CORAL covariance contains non-finite values")
+ return cov
+
+
+def _cov_diaged(data: ChunkedArray, nthreads: int, msg: str) -> np.ndarray:
+ a = _streaming_covariance(data, nthreads, msg)
+ a = (a + a.T) / 2
+ if not np.all(np.isfinite(a)):
+ raise ValueError("CORAL covariance contains non-finite values")
+ return cast(np.ndarray, a + np.eye(a.shape[0], dtype=a.dtype))
+
+
+def _symmetric_matrix_power(matrix: np.ndarray, power: float) -> np.ndarray:
+ eigenvalues, eigenvectors = np.linalg.eigh(matrix)
+ if not np.all(np.isfinite(eigenvalues)):
+ raise ValueError("CORAL covariance decomposition failed")
+ if np.any(eigenvalues <= 0):
+ raise ValueError("CORAL covariance must be positive definite")
+ result = (eigenvectors * np.power(eigenvalues, power)) @ eigenvectors.T
+ if not np.all(np.isfinite(result)):
+ raise ValueError("CORAL covariance transform contains non-finite values")
+ return cast(np.ndarray, result)
+
+
+def _correlation_alignment(
+ s: ChunkedArray, t: ChunkedArray, nthreads: int
+) -> ChunkedArray:
+ from threadpoolctl import threadpool_limits
+
+ if s.shape[1] != t.shape[1]:
+ raise ValueError("CORAL source and target must have the same feature count")
+ s_cov = _cov_diaged(s, nthreads, "CORAL: Computing source covariance")
+ t_cov = _cov_diaged(t, nthreads, "CORAL: Computing target covariance")
logger.info(
"Calculating fractional power of covariance matrices. This might take a while... "
)
with threadpool_limits(limits=nthreads):
- a_coral = np.dot(fmp(s_cov, -0.5), fmp(t_cov, 0.5))
+ a_coral = _symmetric_matrix_power(s_cov, -0.5) @ _symmetric_matrix_power(
+ t_cov, 0.5
+ )
+ if not np.all(np.isfinite(a_coral)):
+ raise ValueError("CORAL transform contains non-finite values")
logger.info("Fractional power calculation complete")
- return daskarr.dot(s, a_coral)
+ return s.dot(a_coral)
-def coral(source_data, target_data, assay, feat_key: str, cell_key: str, nthreads: int):
- """Applies CORAL error correction to the input data.
+def coral(
+ source_data: ChunkedArray,
+ target_data: ChunkedArray,
+ assay: Assay,
+ feat_key: str,
+ cell_key: str,
+ nthreads: int,
+) -> None:
+ """Apply CORAL batch correction and write corrected data to Zarr.
Args:
- source_data ():
- target_data ():
- assay ():
- feat_key ():
- cell_key ():
- nthreads ():
+ source_data: Query ChunkedArray to align to the reference.
+ target_data: Reference ChunkedArray defining the target distribution.
+ assay: Target Assay whose Zarr group receives corrected data.
+ feat_key: Feature selection key used in output path.
+ cell_key: Cell selection key used in output path.
+ nthreads: Threads for streaming statistics and writes.
"""
from .writers import dask_to_zarr
from .utils import clean_array
@@ -82,39 +270,55 @@ def coral(source_data, target_data, assay, feat_key: str, cell_key: str, nthread
),
1,
)
- data = _correlation_alignment(
+ standardized = _correlation_alignment(
(source_data - sm) / sd, (target_data - tm) / td, nthreads
)
+ # The alignment is computed in standardized coordinates. Restore the
+ # reference feature scale so the result can use the reference ANN
+ # transform exactly as an uncorrected query would.
+ data = standardized * td + tm
dask_to_zarr(
data,
- assay.z["/"],
- f"{assay.z.name}/normed__{cell_key}__{feat_key}/data_coral",
- 1000,
+ assay.z,
+ f"normed__{cell_key}__{feat_key}/data_coral",
+ data.chunksize,
nthreads,
msg="Writing out coral corrected data",
)
def _order_features(
- s_assay,
- t_assay,
+ s_assay: Assay,
+ t_assay: Assay,
s_feat_ids: np.ndarray,
filter_null: bool,
- exclude_missing: bool,
+ missing_feature_policy: str,
nthreads: int,
target_cell_key: str = "I",
-) -> Tuple[np.ndarray, np.ndarray]:
+) -> tuple[np.ndarray, np.ndarray]:
s_ids = pd.Series(s_assay.feats.fetch_all("ids"))
t_ids = pd.Series(t_assay.feats.fetch_all("ids"))
+ if s_ids.duplicated().any():
+ duplicates = s_ids[s_ids.duplicated()].iloc[:5].tolist()
+ raise ValueError(f"Reference feature identifiers must be unique: {duplicates}")
+ if t_ids.duplicated().any():
+ duplicates = t_ids[t_ids.duplicated()].iloc[:5].tolist()
+ raise ValueError(f"Target feature identifiers must be unique: {duplicates}")
+ selected_ids = pd.Series(s_feat_ids)
+ if selected_ids.duplicated().any():
+ duplicates = selected_ids[selected_ids.duplicated()].iloc[:5].tolist()
+ raise ValueError(
+ f"Selected reference feature identifiers must be unique: {duplicates}"
+ )
t_idx = t_ids.isin(s_feat_ids)
if t_idx.sum() == 0:
raise ValueError(
"ERROR: None of the features from reference were found in the target data"
)
if filter_null:
- if exclude_missing is False:
+ if missing_feature_policy != "intersection":
logger.warning(
- "`filter_null` has not effect because `exclude_missing` is False"
+ "`filter_null` has no effect unless missing_feature_policy is 'intersection'"
)
else:
t_idx[t_idx] = (
@@ -127,7 +331,9 @@ def _order_features(
!= 0
)
t_idx = t_idx[t_idx].index
- if exclude_missing:
+ if len(t_idx) == 0:
+ raise ValueError("No target features remain after applying the feature policy")
+ if missing_feature_policy == "intersection":
s_idx = s_ids.isin(t_ids.values[t_idx])
else:
s_idx = s_ids.isin(s_feat_ids)
@@ -155,60 +361,116 @@ def align_features(
filter_null: bool,
exclude_missing: bool,
nthreads: int,
+ missing_feature_policy: str | None = None,
+ missing_feature_values: np.ndarray | None = None,
) -> np.ndarray:
"""Aligns target features to source features.
Args:
- source_assay ():
- target_assay ():
- source_cell_key ():
- source_feat_key ():
- target_feat_key ():
- filter_null ():
- exclude_missing ():
- nthreads ():
+ source_assay: Reference assay with features to align to.
+ target_assay: Target assay whose features are reordered and saved.
+ source_cell_key: Cell key for source normalization params.
+ source_feat_key: Feature key on the source assay.
+ target_feat_key: Feature key label for saved target data.
+ target_cell_key: Cell key on the target assay.
+ filter_null: Drop target features with zero counts in selected cells.
+ exclude_missing: Deprecated alias for ``missing_feature_policy='intersection'``.
+ nthreads: Threads for streaming alignment.
+ missing_feature_policy: One of ``'zero'``, ``'intersection'``, ``'error'``,
+ or ``'reference_mean'``.
+ ``'zero'`` fills absent target features with zero;
+ ``'intersection'`` retains only shared features; ``'error'``
+ rejects incomplete feature overlap; and ``'reference_mean'`` fills
+ missing values from the stored reference mean.
+ missing_feature_values: Per-reference-feature values used for absent
+ target features. Required when ``missing_feature_policy`` is
+ ``'reference_mean'`` and target features are absent.
Returns:
+ Target feature index array aligned to source order.
"""
from .writers import create_zarr_dataset
- source_feat_ids = source_assay.feats.fetch(
- "ids", key=source_cell_key + "__" + source_feat_key
+ if missing_feature_policy is None:
+ missing_feature_policy = "intersection" if exclude_missing else "zero"
+ if missing_feature_policy not in {
+ "zero",
+ "intersection",
+ "error",
+ "reference_mean",
+ }:
+ raise ValueError(
+ "missing_feature_policy must be one of 'zero', 'intersection', 'error', "
+ "or 'reference_mean'"
+ )
+ if exclude_missing and missing_feature_policy != "intersection":
+ raise ValueError(
+ "exclude_missing=True is only compatible with missing_feature_policy='intersection'"
+ )
+
+ source_feature_key = (
+ source_feat_key
+ if source_feat_key == "I"
+ else f"{source_cell_key}__{source_feat_key}"
)
+ source_feat_ids = source_assay.feats.fetch("ids", key=source_feature_key)
s_idx, t_idx = _order_features(
source_assay,
target_assay,
source_feat_ids,
filter_null,
- exclude_missing,
+ missing_feature_policy,
nthreads,
target_cell_key,
)
- logger.info(f"{(t_idx == -1).sum()} features missing in target data")
+ n_missing = int((t_idx == -1).sum())
+ if missing_feature_policy == "error" and n_missing:
+ raise ValueError(
+ f"Target data is missing {n_missing} required reference features"
+ )
+ logger.info(f"{n_missing} features missing in target data")
+ if missing_feature_values is not None:
+ missing_feature_values = np.asarray(missing_feature_values)
+ if missing_feature_values.shape != (len(t_idx),):
+ raise ValueError(
+ "missing_feature_values must have one value per aligned reference feature"
+ )
+ if missing_feature_policy == "reference_mean" and n_missing:
+ if missing_feature_values is None:
+ raise ValueError(
+ "reference_mean feature handling requires missing_feature_values"
+ )
+ if not np.all(np.isfinite(missing_feature_values)):
+ raise ValueError("missing_feature_values must be finite")
normed_loc = f"normed__{source_cell_key}__{source_feat_key}"
- norm_params = source_assay.z[normed_loc].attrs["subset_params"]
+ norm_params = cast(
+ dict[str, Any], source_assay.z[normed_loc].attrs["subset_params"]
+ )
sorted_t_idx = np.array(sorted(t_idx[t_idx != -1]))
normed_data = target_assay.normed(
target_assay.cells.active_index(target_cell_key), sorted_t_idx, **norm_params
)
- loc = f"{target_assay.z.name}/normed__{target_cell_key}__{target_feat_key}/data"
-
+ normed_loc = f"normed__{target_cell_key}__{target_feat_key}"
og = create_zarr_dataset(
- target_assay.z["/"], loc, (1000,), "float64", (normed_data.shape[0], len(t_idx))
+ target_assay.z,
+ f"{normed_loc}/data",
+ (1000, len(t_idx)),
+ "float64",
+ (normed_data.shape[0], len(t_idx)),
)
pos_start, pos_end = 0, 0
unsorter_idx = np.argsort(np.argsort(t_idx[t_idx != -1]))
- for i in tqdmbar(
- normed_data.blocks,
- total=normed_data.numblocks[0],
- desc=f"({target_assay.name}) Writing aligned data to {loc.split('/')[1]}",
+ for i in normed_data.stream_blocks(
+ nthreads=nthreads,
+ msg=f"({target_assay.name}) Writing aligned data to {normed_loc}",
):
pos_end += i.shape[0]
- a = np.ones((i.shape[0], len(t_idx)))
- a[:, np.where(t_idx != -1)[0]] = controlled_compute(i, nthreads)[
- :, unsorter_idx
- ]
+ if missing_feature_values is None:
+ a = np.zeros((i.shape[0], len(t_idx)), dtype=i.dtype)
+ else:
+ a = np.broadcast_to(missing_feature_values, (i.shape[0], len(t_idx))).copy()
+ a[:, np.where(t_idx != -1)[0]] = i[:, unsorter_idx]
og[pos_start:pos_end, :] = a
pos_start = pos_end
return s_idx
diff --git a/scarf/markers.py b/scarf/markers.py
index e9c5573a..fb35c8b7 100644
--- a/scarf/markers.py
+++ b/scarf/markers.py
@@ -1,21 +1,80 @@
"""Module to find biomarkers."""
-from typing import Optional
+from collections.abc import Generator
+from typing import Any
import numpy as np
import pandas as pd
-from numba import jit
+import numba
+from numba import jit, njit, prange, set_num_threads
from scipy.stats import linregress, norm
from scipy.stats import rankdata
+import zarr
-from scarf.assay import Assay
+from scarf._types import as_zarr_array, as_zarr_group
+from scarf.assay import Assay, RNAassay, norm_lib_size
+from scarf.chunked import ChunkedArray
from scarf.utils import logger, tqdmbar
+_MARKER_SORT_BY = ("score", "p_value")
+_MARKER_SORT_ASCENDING = (False, True)
+# Dense marker batches keep raw + float32 normed + prefetch slack.
+_MARKER_BYTES_PER_CELL_FEATURE = 32
-def read_prenormed_batches(store, cell_idx: np.ndarray, batch_size: int, desc: str):
- batch = {}
+
+def resolve_marker_gene_batch_size(
+ *,
+ n_features: int,
+ n_cells: int,
+ column_chunk: int,
+ memory_bytes: int | None = None,
+ working_copies: int | None = None,
+) -> int:
+ """Choose an automatic marker batch that fits the active memory budget.
+
+ Explicit caller-provided batch sizes are unchanged. This helper only applies
+ when ``gene_batch_size`` is left as ``None``.
+ """
+ from scarf.storage.budget import get_resource_budget
+
+ n_features = max(1, int(n_features))
+ n_cells = max(1, int(n_cells))
+ column_chunk = max(1, int(column_chunk))
+ if memory_bytes is None or working_copies is None:
+ budget = get_resource_budget()
+ memory_bytes = budget.memoryBytes if memory_bytes is None else memory_bytes
+ working_copies = (
+ budget.workingCopies if working_copies is None else working_copies
+ )
+ work = max(1, int(memory_bytes)) // max(1, int(working_copies))
+ budget_cap = max(1, work // (n_cells * _MARKER_BYTES_PER_CELL_FEATURE))
+ return max(1, min(column_chunk, n_features, budget_cap))
+
+
+def sort_marker_results(df: pd.DataFrame) -> pd.DataFrame:
+ frame = df.copy()
+ if "feature_index" not in frame.columns:
+ frame["feature_index"] = frame.index
+ sort_by = list(_MARKER_SORT_BY)
+ ascending = list(_MARKER_SORT_ASCENDING)
+ if "feature_name" in frame.columns:
+ sort_by.append("feature_name")
+ ascending.append(True)
+ else:
+ sort_by.append("feature_index")
+ ascending.append(True)
+ return frame.sort_values(by=sort_by, ascending=ascending)
+
+
+def read_prenormed_batches(
+ store: zarr.Group,
+ cell_idx: np.ndarray,
+ batch_size: int,
+ desc: str,
+) -> Generator[pd.DataFrame, None, None]:
+ batch: dict[int, np.ndarray] = {}
for i in tqdmbar(store.keys(), desc=desc):
- batch[int(i)] = store[i][:][cell_idx]
+ batch[int(i)] = np.asarray(as_zarr_array(store[str(i)])[cell_idx])
if len(batch) == batch_size:
yield pd.DataFrame(batch)
batch = {}
@@ -23,13 +82,17 @@ def read_prenormed_batches(store, cell_idx: np.ndarray, batch_size: int, desc: s
yield pd.DataFrame(batch)
-def mannwhitneyu_from_ranks(ranked_df, groups, group_set):
+def mannwhitneyu_from_ranks(
+ ranked_df: pd.DataFrame,
+ groups: np.ndarray,
+ group_set: np.ndarray,
+) -> pd.DataFrame:
"""
Vectorized Mann-Whitney U test using pre-computed ranks with tie correction.
- This function calculates two-sided p-values by reusing rank data that has
- already been computed, avoiding redundant ranking operations. Includes tie
- correction which is critical for zero-inflated data (e.g., scRNA-seq) where
+ This function calculates two-sided p-values by reusing rank data that has
+ already been computed, avoiding redundant ranking operations. Includes tie
+ correction which is critical for zero-inflated data (e.g., scRNA-seq) where
many values are identical.
Args:
@@ -52,7 +115,7 @@ def mannwhitneyu_from_ranks(ranked_df, groups, group_set):
# This is critical for zero-inflated data where many cells have the same value
# T = Σ(t³ - t) / (n * (n - 1)) where t is the size of each tied group
tie_corrections = np.zeros(ranked_df.shape[1])
-
+
for col_idx in range(ranked_df.shape[1]):
ranks = ranked_df.iloc[:, col_idx].values
# Count occurrences of each unique rank
@@ -61,7 +124,7 @@ def mannwhitneyu_from_ranks(ranked_df, groups, group_set):
tied_counts = counts[counts > 1]
if len(tied_counts) > 0:
# Σ(t³ - t) for all tied groups
- tie_sum = np.sum(tied_counts ** 3 - tied_counts)
+ tie_sum = np.sum(tied_counts**3 - tied_counts)
# Normalize by n*(n-1) to get the correction term
tie_corrections[col_idx] = tie_sum / (n_total * (n_total - 1))
@@ -98,6 +161,125 @@ def mannwhitneyu_from_ranks(ranked_df, groups, group_set):
return pd.DataFrame(pvals, index=ranked_df.columns).T
+@njit(parallel=True, cache=True)
+def _marker_stats_batch(
+ data: np.ndarray,
+ int_indices: np.ndarray,
+ group_counts: np.ndarray,
+ n_total: float,
+) -> np.ndarray:
+ """Compute per-gene, per-group marker statistics for a batch.
+
+ For each feature column a single argsort yields average ranks (for the
+ Mann-Whitney U rank sums), tie sums for tie correction, and dense ranks
+ (for the score). Per-group sum, count, and nonzero counts are accumulated
+ in the same pass. Returns an array of shape ``(n_genes, n_groups, 7)`` with
+ the last axis ordered as ``[score, mean, mean_rest, frac_exp,
+ frac_exp_rest, fold_change, z]``; the z statistic is converted to a p-value
+ by the caller.
+ """
+ n_cells = data.shape[0]
+ n_genes = data.shape[1]
+ n_groups = group_counts.shape[0]
+ out = np.zeros((n_genes, n_groups, 7))
+ for g in prange(n_genes):
+ v = data[:, g]
+ order = np.argsort(v)
+ ar = np.empty(n_cells)
+ dr = np.empty(n_cells)
+ tie_sum = 0.0
+ i = 0
+ rank = 0.0
+ while i < n_cells:
+ j = i
+ vi = v[order[i]]
+ while j + 1 < n_cells and v[order[j + 1]] == vi:
+ j += 1
+ avg = (i + j + 2) / 2.0
+ rank += 1.0
+ t = j - i + 1
+ for k in range(i, j + 1):
+ ar[order[k]] = avg
+ dr[order[k]] = rank
+ if t > 1:
+ tie_sum += t * t * t - t
+ i = j + 1
+ sum_g = np.zeros(n_groups)
+ nz_g = np.zeros(n_groups)
+ rank_g = np.zeros(n_groups)
+ drank_g = np.zeros(n_groups)
+ for c in range(n_cells):
+ grp = int_indices[c]
+ val = v[c]
+ sum_g[grp] += val
+ if val > 0:
+ nz_g[grp] += 1.0
+ rank_g[grp] += ar[c]
+ drank_g[grp] += dr[c]
+ total_sum = 0.0
+ total_nz = 0.0
+ for x in range(n_groups):
+ total_sum += sum_g[x]
+ total_nz += nz_g[x]
+ r_sum = 0.0
+ r_vals = np.empty(n_groups)
+ for x in range(n_groups):
+ cnt = group_counts[x]
+ r_vals[x] = drank_g[x] / cnt if cnt > 0 else 0.0
+ r_sum += r_vals[x]
+ if n_total > 1:
+ tie_corr = tie_sum / (n_total * (n_total - 1))
+ else:
+ tie_corr = 0.0
+ for x in range(n_groups):
+ cnt = group_counts[x]
+ rest = n_total - cnt
+ m = sum_g[x] / cnt if cnt > 0 else 0.0
+ m_o = (total_sum - sum_g[x]) / rest if rest > 0 else 0.0
+ e = nz_g[x] / cnt if cnt > 0 else 0.0
+ e_o = (total_nz - nz_g[x]) / rest if rest > 0 else 0.0
+ if m_o == 0.0:
+ fc = 0.0 if m == 0.0 else 100.1
+ else:
+ fc = m / m_o
+ r = r_vals[x] / r_sum if r_sum > 0 else 0.0
+ n1 = cnt
+ n2 = rest
+ r1 = rank_g[x]
+ u1 = r1 - (n1 * (n1 + 1.0)) / 2.0
+ mu = (n1 * n2) / 2.0
+ var = (n1 * n2 / 12.0) * ((n_total + 1.0) - tie_corr)
+ if var > 0.0:
+ z = (u1 - mu - 0.5) / np.sqrt(var)
+ else:
+ z = 0.0
+ out[g, x, 0] = r
+ out[g, x, 1] = m
+ out[g, x, 2] = m_o
+ out[g, x, 3] = e
+ out[g, x, 4] = e_o
+ out[g, x, 5] = fc
+ out[g, x, 6] = z
+ return out
+
+
+def _batch_stats(
+ data: np.ndarray,
+ int_indices: np.ndarray,
+ group_counts: np.ndarray,
+ n_total: int,
+) -> np.ndarray:
+ """Run the numba marker kernel and convert z statistics to p-values."""
+ out = _marker_stats_batch(
+ np.ascontiguousarray(data, dtype=np.float32),
+ int_indices,
+ group_counts.astype(np.float32),
+ np.float32(n_total),
+ )
+ out[:, :, 6] = 2.0 * norm.sf(np.abs(out[:, :, 6]))
+ return np.asarray(out)
+
+
def find_markers_by_rank(
assay: Assay,
group_key: str,
@@ -105,10 +287,11 @@ def find_markers_by_rank(
feat_key: str,
batch_size: int,
use_prenormed: bool,
- prenormed_store: Optional[str],
+ prenormed_store: zarr.Group | None,
n_threads: int,
- **norm_params,
-) -> dict:
+ prefetch_depth: int = 1,
+ **norm_params: Any,
+) -> dict[Any, pd.DataFrame]:
"""Identify marker genes/features for given groups using a rank-based approach.
Uses a two-sided Mann-Whitney U test with tie correction to identify genes
@@ -124,6 +307,7 @@ def find_markers_by_rank(
use_prenormed: Whether to use pre-normalized data if available
prenormed_store: Name of the store containing pre-normalized data
n_threads: Number of threads to use for parallel processing
+ prefetch_depth: Number of feature batches to read ahead in parallel
**norm_params: Additional parameters to pass to normalization functions
Returns:
@@ -131,55 +315,20 @@ def find_markers_by_rank(
like fold changes, two-sided p-values, and effect sizes
"""
- from joblib import Parallel, delayed
-
- def calc(vdf):
- # Rank data once for all subsequent calculations
- ranked_vdf = vdf.rank(method="dense")
- ranked_vdf_average = vdf.rank(method="average")
-
- # Calculate normalized mean ranks
- r = ranked_vdf.groupby(groups).mean().reindex(group_set)
- r = r / r.sum()
-
- g = np.array([pd.Series(groups).value_counts().reindex(group_set).values]).T
- g_o = len(groups) - g
-
- s = vdf.groupby(groups).sum().reindex(group_set)
- m = s / g
- m_o = (s.sum() - s) / g_o
-
- s = (vdf > 0).groupby(groups).sum().reindex(group_set)
- e = s / g
- e_o = (s.sum() - s) / g_o
-
- fc = (m / m_o).fillna(0)
-
- # Vectorized Mann-Whitney U test using pre-computed ranks
- pvals = mannwhitneyu_from_ranks(ranked_vdf_average, groups, group_set)
-
- return np.array(
- [
- r.values,
- m.values,
- m_o.values,
- e.values,
- e_o.values,
- fc.values,
- pvals.values,
- ]
- ).T
+ from concurrent.futures import ThreadPoolExecutor
@jit(nopython=True)
- def calc_rank_mean(v):
+ def calc_rank_mean(v: np.ndarray) -> np.ndarray:
"""Calculates the mean rank of the data."""
r = np.ones(n_groups)
for x in range(n_groups):
r[x] = v[int_indices == x].mean()
- return r / r.sum()
+ return np.asarray(r / r.sum())
@jit(nopython=True)
- def calc_frac_fc(v):
+ def calc_frac_fc(
+ v: np.ndarray,
+ ) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
"""Calculates the mean rank of the data."""
m = np.zeros(n_groups)
m_o = np.zeros(n_groups)
@@ -198,8 +347,10 @@ def calc_frac_fc(v):
fc[x] = m[x] / (m_o[x])
return m, m_o, e, e_o, fc
- def prenormed_mean_rank_wrapper(gene_idx):
- d = prenormed_store[gene_idx][:][cell_idx]
+ def prenormed_mean_rank_wrapper(
+ item: tuple[int | str, np.ndarray],
+ ) -> tuple[int | str, np.ndarray]:
+ gene_idx, d = item
r = calc_rank_mean(rankdata(d, method="dense"))
m, m_o, e, e_o, fc = calc_frac_fc(d)
# Calculate p-values for this single feature
@@ -224,54 +375,144 @@ def prenormed_mean_rank_wrapper(gene_idx):
"fold_change",
"p_value",
]
+ results: dict[Any, pd.DataFrame] = {}
if use_prenormed:
if prenormed_store is None:
if "prenormed" in assay.z:
- prenormed_store = assay.z["prenormed"]
+ prenormed_store = as_zarr_group(assay.z["prenormed"], name="prenormed")
else:
raise ValueError(
"Could not find prenormed values. Run with use_prenormed=False or create pre-normed values."
)
- results = {x: [] for x in group_set}
+ prenormed_rows: dict[Any, list[list[Any]]] = {x: [] for x in group_set}
cell_idx = assay.cells.active_index(cell_key)
- batch_iterator = tqdmbar(prenormed_store.keys(), desc="Finding markers")
- temp = Parallel(n_jobs=n_threads)(
- delayed(prenormed_mean_rank_wrapper)(i) for i in batch_iterator
- )
- for i in temp:
- for j, k in zip(group_set, i[1].T):
- results[j].append([i[0]] + list(k))
- for i in results:
- results[i] = (
- pd.DataFrame(results[i], columns=out_cols)
- .sort_values(by="score", ascending=False)
- .round(5)
- )
+ gene_keys = list(prenormed_store.keys())
+ n_batches = (len(gene_keys) + batch_size - 1) // batch_size
+
+ def read_gene(gene_idx: int | str) -> tuple[int | str, np.ndarray]:
+ values = np.asarray(as_zarr_array(prenormed_store[str(gene_idx)])[cell_idx])
+ return gene_idx, values
+
+ def process_batches(executor: ThreadPoolExecutor | None) -> None:
+ batch_starts = range(0, len(gene_keys), batch_size)
+ for start in tqdmbar(batch_starts, desc="Finding markers", total=n_batches):
+ keys = gene_keys[start : start + batch_size]
+ if executor is None:
+ values = [read_gene(key) for key in keys]
+ else:
+ values = list(executor.map(read_gene, keys))
+ if executor is None:
+ batch_results = [
+ prenormed_mean_rank_wrapper(value) for value in values
+ ]
+ else:
+ batch_results = list(
+ executor.map(prenormed_mean_rank_wrapper, values)
+ )
+ for gene_idx, stats in batch_results:
+ for group_id, group_stats in zip(group_set, stats.T, strict=True):
+ prenormed_rows[group_id].append([gene_idx] + list(group_stats))
+
+ if n_threads > 1:
+ with ThreadPoolExecutor(max_workers=n_threads) as executor:
+ process_batches(executor)
+ else:
+ process_batches(None)
+ for group_id, rows in prenormed_rows.items():
+ frame = pd.DataFrame(rows, columns=out_cols)
+ cols_to_round = [col for col in frame.columns if col != "p_value"]
+ frame.loc[:, cols_to_round] = frame.loc[:, cols_to_round].round(5)
+ results[group_id] = sort_marker_results(frame)
return results
else:
- batch_iterator = assay.iter_normed_feature_wise(
- cell_key=cell_key,
- feat_key=feat_key,
- batch_size=batch_size,
- msg="Finding markers",
- **norm_params,
+ set_num_threads(min(max(1, n_threads), numba.config.NUMBA_NUM_THREADS))
+ group_counts = pd.Series(groups).value_counts().reindex(group_set).values
+ n_total = len(groups)
+
+ renormalize_subset = bool(norm_params.get("renormalize_subset", False))
+ log_transform = bool(norm_params.get("log_transform", False))
+ use_fast = (
+ assay.normMethod is norm_lib_size
+ and not renormalize_subset
+ and assay.sf is not None
)
- temp = np.vstack([calc(x) for x in batch_iterator])
- results = {}
+
+ if use_fast:
+ if not isinstance(assay, RNAassay):
+ raise TypeError(
+ "Fast raw-count marker search requires an RNAassay instance"
+ )
+ cell_idx = assay.cells.active_index(cell_key)
+ feat_idx = assay.feats.active_index(feat_key)
+ n_batches = max(1, (len(feat_idx) + batch_size - 1) // batch_size)
+ logger.debug(
+ f"Marker search (fast): {len(feat_idx)} features, {n_groups} groups, "
+ f"{n_batches} batches of {batch_size}"
+ )
+ scalar = assay.cells.fetch_all(assay.name + "_nCounts")[cell_idx]
+ sf = assay.sf
+ if sf is None:
+ raise ValueError(
+ "RNA library-size normalization requires a size factor"
+ )
+ scalar_col = np.asarray(scalar, dtype=np.float32).reshape(-1, 1)
+ scalar_col[scalar_col == 0] = 1
+ batch_stats: list[np.ndarray] = []
+ for (
+ _block_idx,
+ raw,
+ _cols,
+ _read_sec,
+ _source,
+ ) in assay.iter_raw_column_blocks(
+ cell_idx=cell_idx,
+ feat_idx=feat_idx,
+ batch_size=batch_size,
+ msg="Finding markers",
+ ):
+ normed = (float(sf) * raw.astype(np.float32)) / scalar_col
+ if log_transform:
+ normed = np.log1p(normed)
+ stats = _batch_stats(normed, int_indices, group_counts, n_total)
+ batch_stats.append(stats)
+ stats_matrix = np.vstack(batch_stats)
+ else:
+ feat_index = assay.feats.active_index(feat_key)
+ batch_stats = []
+ iterator = iter(
+ assay.iter_normed_feature_wise(
+ cell_key=cell_key,
+ feat_key=feat_key,
+ batch_size=batch_size,
+ msg="Finding markers",
+ **norm_params,
+ )
+ )
+ while True:
+ try:
+ mat = next(iterator)
+ except StopIteration:
+ break
+ values = mat.to_numpy() if isinstance(mat, pd.DataFrame) else mat[0]
+ stats = _batch_stats(
+ values,
+ int_indices,
+ group_counts,
+ n_total,
+ )
+ batch_stats.append(stats)
+ stats_matrix = np.vstack(batch_stats)
feat_index = assay.feats.active_index(feat_key)
pval_col = "p_value"
for n, i in enumerate(group_set):
df = pd.DataFrame(
- temp[:, n, :], columns=out_cols[1:], index=feat_index
- ).sort_values(by="score", ascending=False)
-
+ stats_matrix[:, n, :], columns=out_cols[1:], index=feat_index
+ )
cols_to_round = [col for col in df.columns if col != pval_col]
df.loc[:, cols_to_round] = df.loc[:, cols_to_round].round(5)
- results[i] = df
-
- results[i]["feature_index"] = results[i].index
- results[i] = results[i][out_cols]
+ df["feature_index"] = df.index
+ results[i] = sort_marker_results(df)[out_cols]
return results
@@ -282,7 +523,7 @@ def find_markers_by_regression(
regressor: np.ndarray,
min_cells: int,
batch_size: int = 50,
- **norm_params,
+ **norm_params: Any,
) -> pd.DataFrame:
"""Find features that correlate with a continuous variable using linear regression.
@@ -301,27 +542,49 @@ def find_markers_by_regression(
- p_value: Statistical significance of correlation
"""
- res = {}
- for df in assay.iter_normed_feature_wise(
+ regressor = np.asarray(regressor, dtype=float)
+ if regressor.ndim != 1:
+ raise ValueError("regressor must be one-dimensional")
+ if not np.isfinite(regressor).all():
+ raise ValueError("regressor must contain only finite values")
+ if regressor.size < 2 or np.unique(regressor).size < 2:
+ raise ValueError("regressor must contain at least two distinct values")
+ if min_cells < 1:
+ raise ValueError("min_cells must be at least 1")
+
+ res: dict[Any, tuple[float, float]] = {}
+ for feature_batch in assay.iter_normed_feature_wise(
cell_key=cell_key,
feat_key=feat_key,
batch_size=batch_size,
msg="Finding correlated features",
**norm_params,
):
- for i in df:
- v = df[i].values
- if (v > 0).sum() > min_cells:
+ if not isinstance(feature_batch, pd.DataFrame):
+ raise TypeError("Expected normalized feature batches as DataFrames.")
+ if feature_batch.shape[0] != regressor.shape[0]:
+ raise ValueError(
+ "Regressor length does not match the number of selected cells"
+ )
+ for i in feature_batch:
+ v = np.asarray(feature_batch[i].values, dtype=float)
+ if not np.isfinite(v).all():
+ raise ValueError(f"Feature {i!r} contains non-finite normalized values")
+ if (v > 0).sum() >= min_cells and np.ptp(v) > np.finfo(float).eps:
lin_obj = linregress(regressor, v)
- res[i] = (lin_obj.rvalue, lin_obj.pvalue)
+ res[i] = (float(lin_obj.rvalue), float(lin_obj.pvalue))
else:
- res[i] = (0, 1)
+ res[i] = (0.0, 1.0)
res = pd.DataFrame(res, index=["r_value", "p_value"]).T
return res
def knn_clustering(
- d_array, n_neighbours: int, n_clusters: int, n_threads: int, ann_params: dict = None
+ d_array: ChunkedArray,
+ n_neighbours: int,
+ n_clusters: int,
+ n_threads: int,
+ ann_params: dict[str, Any] | None = None,
) -> np.ndarray:
"""
@@ -337,11 +600,25 @@ def knn_clustering(
np.ndarray: 1D array of cluster assignments (integers from 1 to n_clusters)
"""
+ if len(d_array.shape) != 2:
+ raise ValueError("d_array must be two-dimensional")
+ n_genes = int(d_array.shape[0])
+ if n_genes < 2:
+ raise ValueError("At least two retained genes are required for clustering")
+ if not isinstance(n_neighbours, int) or isinstance(n_neighbours, bool):
+ raise TypeError("n_neighbours must be an integer")
+ if not 1 <= n_neighbours < n_genes:
+ raise ValueError(f"n_neighbours must satisfy 1 <= n_neighbours < {n_genes}")
+ if not isinstance(n_clusters, int) or isinstance(n_clusters, bool):
+ raise TypeError("n_clusters must be an integer")
+ if not 1 <= n_clusters <= n_genes:
+ raise ValueError(f"n_clusters must satisfy 1 <= n_clusters <= {n_genes}")
+
from .ann import instantiate_knn_index, fix_knn_query
- from .utils import controlled_compute, tqdmbar, show_dask_progress
+ from .utils import show_dask_progress
from scipy.sparse import csr_matrix
- def make_knn_mat(data, k, t):
+ def make_knn_mat(data: ChunkedArray, k: int, t: int) -> csr_matrix:
"""Create a sparse KNN adjacency matrix from the input data.
Args:
@@ -353,35 +630,33 @@ def make_knn_mat(data, k, t):
scipy.sparse.csr_matrix: Sparse adjacency matrix representing the KNN graph
"""
- for i in tqdmbar(data.blocks, desc="Fitting KNNs", total=data.numblocks[0]):
- i = controlled_compute(i, t)
+ for i in data.stream_blocks(nthreads=t, msg="Fitting KNNs"):
+ if not np.isfinite(i).all():
+ raise ValueError("Feature profiles must contain only finite values")
ann_idx.add_items(i)
s, e = 0, 0
- indices = []
- for i in tqdmbar(
- data.blocks, desc="Identifying feature KNNs", total=data.numblocks[0]
- ):
+ neighbor_indices: list[np.ndarray] = []
+ for i in data.stream_blocks(nthreads=t, msg="Identifying feature KNNs"):
e += i.shape[0]
- i = controlled_compute(i, t)
inds, d = ann_idx.knn_query(i, k=k + 1)
inds, _, _ = fix_knn_query(inds, d, np.arange(s, e))
- indices.append(inds)
+ neighbor_indices.append(inds)
s = e
- indices = np.vstack(indices)
- assert indices.shape[0] == data.shape[0]
+ indices_mat = np.vstack(neighbor_indices)
+ assert indices_mat.shape[0] == data.shape[0]
return csr_matrix(
(
- np.ones(indices.shape[0] * indices.shape[1]),
+ np.ones(indices_mat.shape[0] * indices_mat.shape[1]),
(
- np.repeat(range(indices.shape[0]), indices.shape[1]),
- indices.flatten(),
+ np.repeat(np.arange(indices_mat.shape[0]), indices_mat.shape[1]),
+ indices_mat.flatten(),
),
),
- shape=(indices.shape[0], indices.shape[0]),
+ shape=(indices_mat.shape[0], indices_mat.shape[0]),
)
- def make_clusters(mat, nc):
+ def make_clusters(mat: csr_matrix, nc: int) -> np.ndarray:
"""Generate clusters from a KNN adjacency matrix using hierarchical clustering.
Args:
@@ -396,9 +671,11 @@ def make_clusters(mat, nc):
paris = skn.hierarchy.Paris(reorder=False)
logger.info("Performing clustering, this might take a while...")
dendrogram = paris.fit_transform(mat)
- return skn.hierarchy.cut_straight(dendrogram, n_clusters=nc)
+ return np.asarray(skn.hierarchy.cut_straight(dendrogram, n_clusters=nc))
- def fix_cluster_order(data, clusters, t):
+ def fix_cluster_order(
+ data: ChunkedArray, clusters: np.ndarray, t: int
+ ) -> np.ndarray:
"""Reorder cluster labels based on feature expression patterns.
Args:
@@ -412,13 +689,13 @@ def fix_cluster_order(data, clusters, t):
idxmax = show_dask_progress(data.argmax(axis=1), "Sorting clusters", t)
cmm = pd.DataFrame([idxmax, clusters]).T.groupby(1).median()[0].sort_values()
- return (
+ return np.asarray(
pd.Series(clusters)
- .replace(dict(zip(cmm.index, range(1, 1 + len(cmm)))))
+ .replace(dict(zip(cmm.index, range(1, 1 + len(cmm)), strict=True)))
.values
)
- default_ann_params = {
+ default_ann_params: dict[str, Any] = {
"space": "l2",
"dim": d_array.shape[1],
"max_elements": d_array.shape[0],
@@ -431,7 +708,16 @@ def fix_cluster_order(data, clusters, t):
if ann_params is None:
ann_params = {}
default_ann_params.update(ann_params)
- ann_idx = instantiate_knn_index(**default_ann_params)
+ ann_idx = instantiate_knn_index(
+ space=str(default_ann_params["space"]),
+ dim=int(default_ann_params["dim"]),
+ max_elements=int(default_ann_params["max_elements"]),
+ ef_construction=int(default_ann_params["ef_construction"]),
+ M=int(default_ann_params["M"]),
+ random_seed=int(default_ann_params["random_seed"]),
+ ef=int(default_ann_params["ef"]),
+ num_threads=int(default_ann_params["num_threads"]),
+ )
return fix_cluster_order(
d_array,
make_clusters(make_knn_mat(d_array, n_neighbours, n_threads), n_clusters),
diff --git a/scarf/meld_assay.py b/scarf/meld_assay.py
index 16a45d32..5f7003cd 100644
--- a/scarf/meld_assay.py
+++ b/scarf/meld_assay.py
@@ -12,15 +12,18 @@
import gzip
import logging
-from typing import Tuple, List, Union
+from collections.abc import Callable, Iterable, Iterator
+from typing import Any
import numpy as np
import pandas as pd
from numba import jit
-from scipy.sparse import coo_matrix
-from zarr import hierarchy
+from scipy.sparse import coo_matrix, csc_matrix, csr_matrix, diags
+import zarr
-from .utils import controlled_compute, logger, tqdmbar
+from .assay import Assay
+from .storage.zarr_store import accumulate_sparse_to_shards
+from .utils import logger, tqdmbar
from .writers import create_zarr_count_assay
__all__ = ["GffReader", "coordinate_melding"]
@@ -34,7 +37,7 @@ def __init__(
gff_fn: str,
up_offset: int = 1000,
down_offset: int = 500,
- chunk_size=100000,
+ chunk_size: int = 100000,
):
"""
@@ -53,7 +56,7 @@ def __init__(
self.down = down_offset
self.chunksize = chunk_size
- def fetch_header_lines(self) -> List[str]:
+ def fetch_header_lines(self) -> list[str]:
"""Fetch header lines (starting with '#') from GFF file.
Returns: A list of all the header lines
@@ -87,7 +90,7 @@ def stream(self) -> pd.DataFrame:
for df in stream:
yield df
- def get_promoter(self, v: pd.Series) -> Tuple[int, int]:
+ def get_promoter(self, v: pd.Series) -> tuple[int, int]:
"""Create strand-aware promoter coordinates using gene start and end
coordinates.
@@ -104,7 +107,7 @@ def get_promoter(self, v: pd.Series) -> Tuple[int, int]:
else:
raise ValueError(f"ERROR: Unknown symbol for strand: {v[6]}")
- def get_body(self, v: pd.Series) -> Tuple[int, int]:
+ def get_body(self, v: pd.Series) -> tuple[int, int]:
"""Create strand-aware gene body + promoter coordinates using gene
start and end coordinates.
@@ -122,7 +125,7 @@ def get_body(self, v: pd.Series) -> Tuple[int, int]:
raise ValueError(f"ERROR: Unknown symbol for strand: {v[6]}")
@staticmethod
- def get_ids_names(v: pd.Series) -> Tuple[str, str]:
+ def get_ids_names(v: pd.Series) -> tuple[str | None, str | None]:
"""Extracts gene_id and gene_name values from last (9th) column of GFF
file record.
@@ -142,7 +145,7 @@ def get_ids_names(v: pd.Series) -> Tuple[str, str]:
return gid, name
@staticmethod
- def d_apply(d: pd.DataFrame, func) -> np.ndarray:
+ def d_apply(d: pd.DataFrame, func: Callable[[pd.Series], Any]) -> np.ndarray:
"""A convenience method to apply arbitrary functions over a dataframe.
Args:
@@ -201,7 +204,7 @@ def to_bed(
return None
-def create_bed_from_coord_ids(ids: list) -> pd.DataFrame:
+def create_bed_from_coord_ids(ids: Iterable[str]) -> pd.DataFrame:
"""Creates a 3 column BED file from list of strings in format:
@@ -210,14 +213,21 @@ def create_bed_from_coord_ids(ids: list) -> pd.DataFrame:
Returns:
A 3 column Pandas dataframe sorted by chromosome and start position
+ while preserving input indices so rows still align with assay feature
+ positions.
"""
- out = []
- for i in ids:
- j = i.split(":")
- o = [j[0], int(j[1].split("-")[0]), int(j[1].split("-")[1])]
- out.append(o)
- return pd.DataFrame(out).sort_values(by=[0, 1])
+ ser = pd.Series(list(ids), dtype="object")
+ chrom_rest = ser.str.split(":", expand=True)
+ start_end = chrom_rest[1].str.split("-", expand=True)
+ df = pd.DataFrame(
+ {
+ 0: chrom_rest[0].to_numpy(),
+ 1: start_end[0].astype(np.int64).to_numpy(),
+ 2: start_end[1].astype(np.int64).to_numpy(),
+ }
+ )
+ return df.sort_values(by=[0, 1])
@jit(nopython=True)
@@ -293,12 +303,12 @@ def get_ranges(df: pd.DataFrame, idx: np.ndarray) -> np.ndarray:
A numpy array with start and end positions
"""
- return df[[1, 2]][idx].values.astype(int)
+ return np.asarray(df[[1, 2]][idx].values, dtype=int)
def get_feature_mappings(
peaks_bed_df: pd.DataFrame, features_bed_df: pd.DataFrame
-) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+) -> tuple[np.ndarray, np.ndarray, csc_matrix]:
"""Identify which intervals from `features_bed_df` overlap with those from
`peaks_bed_df`.
@@ -307,29 +317,35 @@ def get_feature_mappings(
'chrom', 'start', 'end'. It should be sorted by chromosome and start
features_bed_df: DataFrame containing reference intervals. Must have at least 5 columns:
'chrom', 'start', 'end', 'ids', 'names'.
- It should be sorted by chromosome and start
+ It should be sorted by chromosome and start. Features are retained even
+ when their chromosome has no peaks.
Returns:
- A tuple containing three numpy arrays, the first two are simply the 'ids' and 'names'
- columns from `features_bed_df`. These are returned to ensure that they are in same order
- as the indices in the third array. The third array has shape (features_bed_df.shape[0], 2).
- The values are indices of the overlapping intervals from peaks_bed_df. If no overlap is found
- then that row has value [-1, -1].
+ A tuple containing the 'ids' and 'names' columns from `features_bed_df` (as numpy arrays)
+ and a sparse mapping matrix. The mapping has shape (n_peaks, n_features) where n_peaks is
+ `peaks_bed_df.shape[0]` and n_features is the number of features. A nonzero at position
+ (p, f) means peak `p` (indexed by its position in the assay) overlaps feature `f`. Peak
+ row indices use `peaks_bed_df.index` so they align with the columns of the assay count
+ matrix. Features are kept in the sorted feature BED order of the returned id/name arrays.
"""
- cross_indices = []
- feats_ids, feats_names = [], []
- id_counter = {}
+ feats_ids: list[str] = []
+ feats_names: list[str] = []
+ id_counter: dict[str, int] = {}
+ map_peak_rows: list[int] = []
+ map_feat_cols: list[int] = []
n_no_match = 0
- for chrom in peaks_bed_df[0].unique():
- feats_chrom_idx = np.array(features_bed_df[0] == chrom)
- if feats_chrom_idx.shape[0] == 0:
- logger.warning(f"Chromosome {chrom} not in the input feature BED")
- continue
+ feat_col = 0
+ peak_chroms = set(peaks_bed_df[0].unique())
+
+ for chrom in features_bed_df[0].unique():
+ feats_chrom_idx = (features_bed_df[0] == chrom).values
+ chrom_names = features_bed_df[4][feats_chrom_idx].values
+ chrom_ids = features_bed_df[3][feats_chrom_idx].values
- feats_names.extend(features_bed_df[4][feats_chrom_idx].values)
+ feats_names.extend(chrom_names)
# Making IDs unique
- for i in features_bed_df[3][feats_chrom_idx].values:
+ for i in chrom_ids:
if i not in id_counter:
id_counter[i] = 0
id_counter[i] += 1
@@ -337,63 +353,83 @@ def get_feature_mappings(
i = i + f"_{id_counter[i]}"
feats_ids.append(i)
- peaks_chrom_idx = np.array(peaks_bed_df[0] == chrom)
+ if chrom not in peak_chroms:
+ # Features on this chromosome are kept but overlap nothing, so no peak is present.
+ logger.warning(f"Chromosome {chrom} not in the input peak coordinates")
+ n_no_match += len(chrom_ids)
+ feat_col += len(chrom_ids)
+ continue
+
+ peaks_chrom_idx = (peaks_bed_df[0] == chrom).values
match_indices = binary_search(
get_ranges(peaks_bed_df, peaks_chrom_idx),
get_ranges(features_bed_df, feats_chrom_idx),
).astype(int)
- # Now this is the main trick. Since the peak_bed_df is a sorted dataframe.
- # The dataframe index might itself not be in sorted order.
+ # The peaks dataframe is sorted, so its positional index might not be in sorted order.
peak_idx = np.array(peaks_bed_df.index[peaks_chrom_idx])
- for i in match_indices:
- if i[0] == -1:
- assert i[1] == -1
- cross_indices.append(None)
+ for m in match_indices:
+ if m[0] == -1:
+ assert m[1] == -1
n_no_match += 1
else:
- cross_indices.append(list(peak_idx[i[0] : i[1]]))
+ peaks_for_feat = peak_idx[m[0] : m[1]]
+ map_peak_rows.extend(peaks_for_feat.tolist())
+ map_feat_cols.extend([feat_col] * peaks_for_feat.shape[0])
+ feat_col += 1
if len(feats_ids) == 0:
raise ValueError(
"ERROR: None of the features were found in the assay. Melding failed"
)
- feats_ids = np.array(feats_ids)
- feats_names = np.array(feats_names)
- cross_indices = np.array(cross_indices, dtype=object)
- if n_no_match == len(cross_indices):
+ feats_ids_arr = np.array(feats_ids)
+ feats_names_arr = np.array(feats_names)
+ n_features = feats_ids_arr.shape[0]
+ if n_no_match == n_features:
logging.critical(
"None of the provided features overlap with the peak coordinates. "
"Melding has possibly failed."
)
else:
- logger.info(
- f"{n_no_match}/{feats_ids.shape[0]} features did not overlap with any peak"
- )
- if len(set(feats_ids)) != feats_ids.shape[0]:
+ logger.info(f"{n_no_match}/{n_features} features did not overlap with any peak")
+ if len(set(feats_ids_arr)) != n_features:
raise ValueError(
"ERROR: encountered an unexpected error. Somehow the feature ids are not unique "
"despite our attempt to make them unique by appending a suffix. Please report this "
"bug on Github"
)
- assert feats_ids.shape[0] == feats_names.shape[0] == cross_indices.shape[0]
- return feats_ids, feats_names, cross_indices
+ assert feats_ids_arr.shape[0] == feats_names_arr.shape[0]
+ mapping = coo_matrix(
+ (
+ np.ones(len(map_peak_rows), dtype=np.float64),
+ (
+ np.asarray(map_peak_rows, dtype=np.int64),
+ np.asarray(map_feat_cols, dtype=np.int64),
+ ),
+ ),
+ shape=(peaks_bed_df.shape[0], n_features),
+ ).tocsc()
+ return feats_ids_arr, feats_names_arr, mapping
def create_counts_mat(
- assay,
- store: hierarchy,
- cross_map: np.ndarray,
+ assay: Assay,
+ store: zarr.Array,
+ mapping: csc_matrix,
scalar_coeff: float,
renormalization: bool,
) -> None:
"""Populate the count matrix in the Zarr store.
+ The melded value of a feature for a given cell is the sum of the TF-IDF normalized values of
+ all peaks overlapping that feature. This is computed per block as a sparse matrix product
+ between the TF-IDF matrix and the peak-to-feature `mapping` matrix.
+
Args:
- assay: Scarf Assay object which contains the rawData attribute representing Dask array of count matrix
+ assay: Scarf Assay object which contains the rawData attribute representing a chunked array of count matrix
store: Output Zarr Dataset
- cross_map: Mapping of indices. as obtained from get_feature_mappings function
+ mapping: Sparse peak-to-feature mapping of shape (n_peaks, n_features) from get_feature_mappings
scalar_coeff: An arbitrary scalar multiplier. Only used when renormalization is True.
renormalization: Whether to rescale the sum of feature values for each cell to `scalar_coeff`
@@ -401,46 +437,42 @@ def create_counts_mat(
None
"""
- idx = np.where(cross_map)[0]
- feat_idx = np.repeat(idx, list(map(len, cross_map[idx])))
- peak_idx = np.array(
- sum(list(cross_map[idx]), [])
- ) # There is no guarantee that these are in sorted order
- assert feat_idx.shape == peak_idx.shape
-
n_term_per_doc = assay.cells.fetch_all(assay.name + "_nFeatures")
n_docs = n_term_per_doc.shape[0]
n_docs_per_term = assay.feats.fetch_all("nCells")
-
- s = 0
- for a in tqdmbar(assay.rawData.blocks, total=assay.rawData.numblocks[0]):
- a = controlled_compute(a, assay.nthreads)
- tf = a / n_term_per_doc[s : s + a.shape[0]].reshape(-1, 1)
- idf = np.log2(1 + (n_docs / (n_docs_per_term + 1)))
- a = tf * idf
-
- df = pd.DataFrame(a[:, peak_idx]).T
- df["fidx"] = feat_idx
- df = df.groupby("fidx").sum().T
- if renormalization:
- df = (scalar_coeff * df) / df.sum(axis=1).values.reshape(-1, 1)
- assert df.shape[1] == idx.shape[0]
-
- coord_renamer = dict(enumerate(df.columns))
- coo = coo_matrix(df.values)
- coo.col = np.array([coord_renamer[x] for x in coo.col])
- store.set_coordinate_selection((s + coo.row, coo.col), coo.data)
- s += a.shape[0]
+ idf = np.log2(1 + (n_docs / (n_docs_per_term + 1)))
+
+ def block_stream() -> Iterator[coo_matrix]:
+ s = 0
+ # stream_blocks yields materialized, in-order row blocks with bounded
+ # read-ahead so the next block is fetched while the current one is
+ # normalized and written. This overlaps read latency with compute and
+ # writes, which matters most for remote (cloud) stores.
+ for a in assay.rawData.stream_blocks(msg="Melding assay"):
+ tf = a / n_term_per_doc[s : s + a.shape[0]].reshape(-1, 1)
+ tfidf = tf * idf
+ block = (csr_matrix(tfidf) @ mapping).tocsr()
+ if renormalization:
+ row_sums = np.asarray(block.sum(axis=1)).reshape(-1)
+ scale = np.zeros(row_sums.shape[0], dtype=np.float64)
+ nz = row_sums != 0
+ scale[nz] = scalar_coeff / row_sums[nz]
+ block = diags(scale) @ block
+ yield block.tocoo()
+ s += a.shape[0]
+
+ accumulate_sparse_to_shards(store, block_stream())
def coordinate_melding(
- assay,
- workspace: Union[str, None],
+ assay: Assay,
+ workspace: str | None,
feature_bed: pd.DataFrame,
new_assay_name: str,
peaks_col: str = "ids",
scalar_coeff: float = 1e5,
renormalization: bool = True,
+ peaks_coords: np.ndarray | None = None,
) -> None:
"""This function the coordinates of the features of the given assay and
overlaps (genomics intersection) them with a user provided set of external
@@ -450,13 +482,15 @@ def coordinate_melding(
from original feature. Similarly, a single original feature may overlap
with multiple new features and hence its value will used for multiple new
features. The values are TF-IDF normalized before they are saved into the
- new assay.
+ new assay. Features that do not overlap any peak are retained with zero
+ counts; DataStore reload marks those zero-count features invalid.
Args:
- assay: Scarf Assay object which contains the rawData attribute representing Dask array of count matrix.
+ assay: Scarf Assay object which contains the rawData attribute representing a chunked array of count matrix.
workspace:
feature_bed: DataFrame containing reference intervals. Must have at least 5 columns representing
- 'chrom', 'start', 'end', 'ids', 'names'. But the column names should 0,1,2,3,4
+ 'chrom', 'start', 'end', 'ids', 'names'. But the column names should 0,1,2,3,4.
+ Output feature order follows this dataframe.
new_assay_name: Name of the new melded assay
peaks_col: The name of the column in the feature metadata that contains the coordinate information. This
column should have coordinates in this format . The values from this
@@ -464,16 +498,23 @@ def coordinate_melding(
scalar_coeff: An arbitrary scalar multiplier. Only used when renormalization is True (Default value: 1e5)
renormalization: Whether to rescale the sum of feature values for each cell to `scalar_coeff`
(Default value: True)
+ peaks_coords: Optional pre-fetched peak coordinate strings for `peaks_col`. When provided,
+ the coordinates are not fetched again from the assay.
Returns:
None
"""
- peaks_bed = create_bed_from_coord_ids(assay.feats.fetch_all(peaks_col))
- feat_ids, feat_names, mappings = get_feature_mappings(peaks_bed, feature_bed)
+ if peaks_coords is None:
+ peaks_coords = assay.feats.fetch_all(peaks_col)
+ peaks_bed = create_bed_from_coord_ids(peaks_coords)
+ feat_ids, feat_names, mapping = get_feature_mappings(peaks_bed, feature_bed)
+
+ from .storage.zarr_store import zarr_group_root
+ store_root = zarr_group_root(assay.z, mode="r+")
g = create_zarr_count_assay(
- z=assay.z["/"],
+ z=store_root,
assay_name=new_assay_name,
workspace=workspace,
chunk_size=assay.rawData.chunksize,
@@ -486,7 +527,7 @@ def coordinate_melding(
create_counts_mat(
assay=assay,
store=g,
- cross_map=mappings,
+ mapping=mapping,
scalar_coeff=scalar_coeff,
renormalization=renormalization,
)
diff --git a/scarf/merge.py b/scarf/merge.py
index c5d75fec..7410022e 100644
--- a/scarf/merge.py
+++ b/scarf/merge.py
@@ -6,16 +6,16 @@
import os
import re
from collections import Counter
-from typing import Dict, List, Optional, Tuple, Union
+from collections.abc import Iterator
+from typing import Any
import numpy as np
import pandas as pd
-import polars as pl
import zarr
-from dask.array import from_array
-from dask.array.core import Array as daskArrayType
from scipy.sparse import coo_matrix
+from ._types import as_zarr_array, as_zarr_group
+from .chunked import Block, ChunkedArray
from .assay import Assay
from .datastore.datastore import DataStore
from .metadata import MetaData
@@ -25,9 +25,17 @@
load_zarr,
logger,
permute_into_chunks,
+ prefetch_blocks,
tqdmbar,
)
-from .writers import create_zarr_count_assay, create_zarr_obj_array
+from .storage.budget import worker_prefetch_depth
+from .storage.zarr_store import accumulate_sparse_to_shards, is_local_zarr_path
+from .writers import (
+ create_zarr_count_assay,
+ create_zarr_dataset,
+ create_zarr_obj_array,
+ finalize_writer_counts,
+)
__all__ = [
"DatasetMerge",
@@ -45,7 +53,7 @@ class DummyAssay:
def __init__(
self,
ds: DataStore,
- counts: daskArrayType,
+ counts: ChunkedArray,
feats: MetaData,
name: str,
):
@@ -55,6 +63,15 @@ def __init__(
self.name = name
+type MergeAssay = Assay | DummyAssay
+
+type _RowPlan = tuple[
+ dict[int, dict[int, np.ndarray]],
+ dict[int, dict[int, np.ndarray]],
+ np.ndarray,
+]
+
+
class AssayMerge:
# class ZarrMerge is renamed to AssayMerge for better understanding
"""Merge multiple Zarr files into a single Zarr file.
@@ -66,6 +83,8 @@ class AssayMerge:
`assays` parameter.
merge_assay_name: Name of assay in the merged Zarr file. For example, for scRNA-Seq it could be simply,
'RNA'.
+ in_workspaces: Source workspace per assay (None uses each assay's default layout).
+ out_workspace: Target workspace name in the merged Zarr file.
chunk_size: Tuple of cell and feature chunk size. (Default value: (1000, 1000)).
dtype: Dtype of the raw values in the assay. Dtype is automatically inferred from the provided assays. If
assays have different dtypes then a float type is used.
@@ -75,7 +94,6 @@ class AssayMerge:
are set as True as in the 'I' column. To keep the filtering information set the value for
this parameter to False. (Default value: True)
seed: Seed for randomization of rows in the assays.
- feat_name_ids_same: If True, then feature names and feature ids are same in the assays. (Default value: False)
Attributes:
assays: List of assay objects to be merged. For example, [ds1.RNA, ds2.RNA].
@@ -93,58 +111,76 @@ class AssayMerge:
def __init__(
self,
zarr_path: ZARRLOC,
- assays: List[Assay],
- names: List[str],
+ assays: list[MergeAssay],
+ names: list[str],
merge_assay_name: str,
- in_workspaces: Union[list[str], None] = None,
- out_workspace: Union[str, None] = None,
- chunk_size=(1000, 1000),
- dtype: Optional[str] = None,
+ in_workspaces: list[str] | None = None,
+ out_workspace: str | None = None,
+ chunk_size: tuple[int, int] = (1000, 1000),
+ dtype: str | None = None,
overwrite: bool = False,
- prepend_text: Optional[str] = "orig",
+ prepend_text: str | None = "orig",
reset_cell_filter: bool = True,
- seed: Optional[int] = 42,
- ):
+ seed: int | None = 42,
+ storage_options: dict[str, Any] | None = None,
+ _row_plan: _RowPlan | None = None,
+ _row_chunk_sizes: list[int] | None = None,
+ ) -> None:
self.assays = assays
self.names = names
self.inWorkspaces = in_workspaces
self.outWorkspace = out_workspace
self.merge_assay_name = merge_assay_name
self.chunk_size = chunk_size
+ self.storage_options = storage_options
+ self._usesSharedRowPlan = _row_plan is not None or _row_chunk_sizes is not None
+ row_plan = (
+ self.perform_randomization_rows(seed, _row_chunk_sizes)
+ if _row_plan is None
+ else _row_plan
+ )
(
self.permutations_rows,
self.permutations_rows_offset,
self.coordinates_permutations,
- ) = self.perform_randomization_rows(seed)
- self.mergedCells: pl.DataFrame = self._merge_cell_table(
+ ) = row_plan
+ for assay_idx, assay in enumerate(self.assays):
+ planned_rows = sum(
+ rows.size for rows in self.permutations_rows[assay_idx].values()
+ )
+ if planned_rows != assay.rawData.shape[0]:
+ raise ValueError(
+ "All assays from a datastore must contain the same number of cells"
+ )
+ self.mergedCells: pd.DataFrame = self._merge_cell_table(
reset_cell_filter, prepend_text
)
self.nCells: int = self.mergedCells.shape[0]
- self.featCollection: List[Dict[str, str]] = self._get_feat_ids(assays)
+ self.featCollection: list[dict[str, str]] = self._get_feat_ids(assays)
self.feat_name_ids_same: bool = self.check_feat_ids(self.featCollection)
+ self.featCollection_map: list[dict[str, str]]
if self.feat_name_ids_same is True:
- self.feat_suffix: Dict[int, int] = self.get_feat_suffix()
+ self.feat_suffix: dict[int, int] = self.get_feat_suffix()
self.featCollection = self.update_feat_ids()
- self.featCollection_map: List[Dict[str, str]] = (
- self.update_feat_ids_for_map()
- )
+ self.featCollection_map = self.update_feat_ids_for_map()
else:
- self.featCollection_map: List[Dict[str, str]] = self.featCollection.copy()
+ self.featCollection_map = self.featCollection.copy()
- self.mergedFeats: pl.DataFrame = self._merge_order_feats(self.featCollection)
- self.mergedFeats_map: pl.DataFrame = self._merge_order_feats(
+ self.mergedFeats: pd.DataFrame = self._merge_order_feats(self.featCollection)
+ self.mergedFeats_map: pd.DataFrame = self._merge_order_feats(
self.featCollection_map
)
self.nFeats: int = self.mergedFeats_map.shape[0]
- self.featOrder: List[np.ndarray] = self._ref_order_feat_idx()
+ self.featOrder: list[np.ndarray] = self._ref_order_feat_idx()
+ self.featOrder_map: list[np.ndarray]
if self.feat_name_ids_same is True:
- self.featOrder_map: List[np.ndarray] = self._ref_order_feat_idx_map()
+ self.featOrder_map = self._ref_order_feat_idx_map()
else:
- self.featOrder_map: List[np.ndarray] = self.featOrder.copy()
+ self.featOrder_map = self.featOrder.copy()
- self.cellOrder: Dict[int, Dict[int, np.ndarray]] = self._ref_order_cell_idx()
+ self.cellOrder: dict[int, dict[int, np.ndarray]] = self._ref_order_cell_idx()
self.z: zarr.Group = self._use_existing_zarr(
zarr_path, merge_assay_name, overwrite
)
@@ -167,9 +203,11 @@ def __init__(
)
def perform_randomization_rows(
- self, seed: Optional[int] = 42
- ) -> Tuple[
- Dict[int, Dict[int, np.ndarray]], Dict[int, Dict[int, np.ndarray]], np.ndarray
+ self,
+ seed: int | None = 42,
+ row_chunk_sizes: list[int] | None = None,
+ ) -> tuple[
+ dict[int, dict[int, np.ndarray]], dict[int, dict[int, np.ndarray]], np.ndarray
]:
"""
Perform randomization of rows in the assays.
@@ -178,7 +216,14 @@ def perform_randomization_rows(
Returns:
"""
rng = np.random.default_rng(seed=seed)
- chunkSize = np.array([x.rawData.chunksize[0] for x in self.assays])
+ if row_chunk_sizes is None:
+ chunkSize = np.array([x.rawData.chunksize[0] for x in self.assays])
+ else:
+ if len(row_chunk_sizes) != len(self.assays):
+ raise ValueError("Row chunk sizes must match the number of assays")
+ chunkSize = np.asarray(row_chunk_sizes, dtype=int)
+ if np.any(chunkSize <= 0):
+ raise ValueError("Row chunk sizes must be positive")
nCells = np.array([x.rawData.shape[0] for x in self.assays])
permutations = {
i: permute_into_chunks(nCells[i], chunkSize[i])
@@ -253,7 +298,7 @@ def perform_randomization_rows(
)
return permutations_rows, permutations_rows_offset, coordinates_permutations
- def _ref_order_cell_idx(self) -> Dict[int, Dict[int, np.ndarray]]:
+ def _ref_order_cell_idx(self) -> dict[int, dict[int, np.ndarray]]:
"""
Calculate the order of the cells in the merged assay.
"""
@@ -277,8 +322,8 @@ def _ref_order_cell_idx(self) -> Dict[int, Dict[int, np.ndarray]]:
return new_cells
def _merge_cell_table(
- self, reset: bool, prepend_text: Optional[str] = None
- ) -> pl.DataFrame:
+ self, reset: bool, prepend_text: str | None = None
+ ) -> pd.DataFrame:
"""Merges the cell metadata table for each sample.
Args:
@@ -295,30 +340,25 @@ def _merge_cell_table(
prepend_text = None
ret_val = []
for assay, name in zip(self.assays, self.names):
- a = assay.cells.to_polars_dataframe(assay.cells.columns)
- a = a.with_columns(
- [pl.Series("ids", np.array([f"{name}__{x}" for x in a["ids"]]))]
- )
- for i in a.columns:
+ a = assay.cells.to_pandas_dataframe(assay.cells.columns)
+ a["ids"] = np.array([f"{name}__{x}" for x in a["ids"]])
+ for i in list(a.columns):
if i not in ["ids", "I", "names"] and prepend_text is not None:
- a = a.with_columns(
- [pl.Series(f"{prepend_text}_{i}", assay.cells.fetch_all(i))]
- )
- a = a.drop([i])
+ a[f"{prepend_text}_{i}"] = assay.cells.fetch_all(i)
+ a = a.drop(columns=[i])
if reset:
- a = a.with_columns(
- [pl.Series("I", np.ones(len(a["ids"])).astype(bool))]
- )
- ret_val.append(a.to_pandas())
+ a["I"] = np.ones(len(a["ids"])).astype(bool)
+ ret_val.append(a)
# Here we merge the cell metadata tables for each sample. We simply concatenate the tables and reset the index.
ret_val_df = pd.concat(ret_val, axis=0).reset_index(drop=True)
# Now we use the offsets stored in permutations_rows_offset along with the coordinates_permutations to reorder the cells in the merged assay. The offsets are used to bring the cells in the same order as the rows in the merged assay.
- compiled_idx = [
- self.permutations_rows_offset[i][j]
- for i, j in self.coordinates_permutations
- ]
- compiled_idx = np.concatenate(compiled_idx)
+ compiled_idx = np.concatenate(
+ [
+ self.permutations_rows_offset[i][j]
+ for i, j in self.coordinates_permutations
+ ]
+ )
# Index the merged cell metadata table with the compiled_idx to get the final randomized merged cell metadata table.
ret_val_df = ret_val_df.iloc[compiled_idx]
if sum([x.cells.N for x in self.assays]) != ret_val_df.shape[0]:
@@ -329,7 +369,7 @@ def _merge_cell_table(
return ret_val_df
@staticmethod
- def _get_feat_ids(assays) -> List[Dict[str, str]]:
+ def _get_feat_ids(assays: list[MergeAssay]) -> list[dict[str, str]]:
"""Fetches ID->names mapping of features from each assay.
Args:
@@ -341,11 +381,11 @@ def _get_feat_ids(assays) -> List[Dict[str, str]]:
"""
ret_val = []
for i in assays:
- df = i.feats.to_polars_dataframe(["names", "ids"])
+ df = i.feats.to_pandas_dataframe(["names", "ids"])
ret_val.append(dict(zip(df["ids"].to_numpy(), df["names"].to_numpy())))
return ret_val
- def check_feat_ids(self, featCollection: List[Dict[str, str]]) -> bool:
+ def check_feat_ids(self, featCollection: list[dict[str, str]]) -> bool:
"""
Check if feature names and feature ids are different in the assays.
"""
@@ -361,7 +401,7 @@ def check_feat_ids(self, featCollection: List[Dict[str, str]]) -> bool:
break
return isSame
- def get_feat_suffix(self) -> Dict[int, int]:
+ def get_feat_suffix(self) -> dict[int, int]:
"""
Get the suffix of the feature ids.
"""
@@ -385,12 +425,12 @@ def get_feat_suffix(self) -> Dict[int, int]:
feat_suffix[i] = -1
return feat_suffix
- def update_feat_ids(self) -> List[Dict[str, str]]:
+ def update_feat_ids(self) -> list[dict[str, str]]:
"""
Update the feature ids in case of same feature names and ids.
Returns:
- `List[Dict[str, str]]`: List of dictionaries containing the updated feature ids for the merged assay.
+ `list[dict[str, str]]`: List of dictionaries containing the updated feature ids for the merged assay.
This function updates the feature ids for the merged assay in case the feature names and ids are the same in the assays.
This function will generate a new feature id and name for the duplicate feature names and ids.
@@ -412,7 +452,7 @@ def update_feat_ids(self) -> List[Dict[str, str]]:
if counter[val] == 1: # Unique value
in_dict[val] = val
else: # Multiple values -- update
- updated_val = f"{val}_{min_val+sum_counter[val]}"
+ updated_val = f"{val}_{min_val + sum_counter[val]}"
in_dict[updated_val] = updated_val
sum_counter[val] += 1
else:
@@ -422,7 +462,7 @@ def update_feat_ids(self) -> List[Dict[str, str]]:
num = int(val.split("_")[-1])
# replace the number with min_val
updated_val = pattern.sub(
- f"_{min_val-self.feat_suffix[i]+num}", val
+ f"_{min_val - self.feat_suffix[i] + num}", val
)
in_dict[updated_val] = updated_val
else:
@@ -431,12 +471,12 @@ def update_feat_ids(self) -> List[Dict[str, str]]:
new_featCollection.append(in_dict)
return new_featCollection
- def update_feat_ids_for_map(self) -> List[Dict[str, str]]:
+ def update_feat_ids_for_map(self) -> list[dict[str, str]]:
"""
Get the updated feature ids mapping for the merged assay in case of same feature names and ids.
Returns:
- `List[Dict[str, str]]`: List of dictionaries containing the updated feature ids for the merged assay.
+ `list[dict[str, str]]`: List of dictionaries containing the updated feature ids for the merged assay.
This function updates the feature ids for the merged assay in case the feature names and ids are the same in the assays.
This function will remove the numeric suffix from the feature ids and update them with the feature names.
@@ -444,32 +484,30 @@ def update_feat_ids_for_map(self) -> List[Dict[str, str]]:
pattern = re.compile(r"_\d+$")
new_featCollection = []
for dict_ in self.featCollection:
- in_dict = {}
- for x in dict_.values():
- # check if the value ends with a number
- if pattern.search(x):
- val = x.split("_")[:-1]
- val = "_".join(val)
- if val not in in_dict:
- in_dict[val] = val
+ in_dict: dict[str, str] = {}
+ for feat_val in dict_.values():
+ if pattern.search(feat_val):
+ base_val = "_".join(feat_val.split("_")[:-1])
+ if base_val not in in_dict:
+ in_dict[base_val] = base_val
else:
- in_dict[x] = x
+ in_dict[feat_val] = feat_val
new_featCollection.append(in_dict)
return new_featCollection
- def _merge_order_feats(self, FeatCollection) -> pl.DataFrame:
+ def _merge_order_feats(self, feat_collection: list[dict[str, str]]) -> pd.DataFrame:
"""Merge features from all the assays and determine their order.
Returns:
"""
- union_set = {}
- for ids in FeatCollection:
+ union_set: dict[str, str] = {}
+ for ids in feat_collection:
for i in ids:
if i not in union_set:
union_set[i] = ids[i]
- ret_val = pl.DataFrame(
+ ret_val = pd.DataFrame(
{
- "idx": range(len(union_set)),
+ "idx": list(range(len(union_set))),
"names": list(union_set.values()),
"ids": list(union_set.keys()),
}
@@ -485,31 +523,29 @@ def _merge_order_feats(self, FeatCollection) -> pl.DataFrame:
logger.warning("The number overlapping features is very low.")
return ret_val
- def _ref_order_feat_idx(self) -> List[np.ndarray]:
+ def _ref_order_feat_idx(self) -> list[np.ndarray]:
ret_val = []
for ids in self.featCollection:
- ordered_ids = pl.DataFrame({"ids": list(ids.keys())})
- # vals = self.mergedFeats.filter(pl.col("ids").is_in(list(ids.keys())))["idx"]
- vals = ordered_ids.join(self.mergedFeats, on="ids", how="left")[
+ ordered_ids = pd.DataFrame({"ids": list(ids.keys())})
+ vals = ordered_ids.merge(self.mergedFeats, on="ids", how="left")[
"idx"
].to_numpy()
ret_val.append(np.array(vals))
return ret_val
- def _ref_order_feat_idx_map(self) -> List[np.ndarray]:
+ def _ref_order_feat_idx_map(self) -> list[np.ndarray]:
"""
Get the order of the features in the merged assay.
Returns:
- `List[np.ndarray]`: List of numpy arrays containing the order of the features in the merged assay.
+ `list[np.ndarray]`: List of numpy arrays containing the order of the features in the merged assay.
This function returns the order of the features in the merged assay. The order is determined by the feature
"""
featorder = []
- names_to_idx = self.mergedFeats_map.select(["names", "idx"]).to_dict(
- as_series=False
+ name_to_idx_dict = dict(
+ zip(self.mergedFeats_map["names"], self.mergedFeats_map["idx"])
)
- name_to_idx_dict = dict(zip(names_to_idx["names"], names_to_idx["idx"]))
pattern = re.compile(r"_\d+$")
for dict_ in self.featCollection:
vals = []
@@ -523,17 +559,17 @@ def _ref_order_feat_idx_map(self) -> List[np.ndarray]:
return featorder
def _use_existing_zarr(
- self, zarr_loc: ZARRLOC, merge_assay_name, overwrite
+ self, zarr_loc: ZARRLOC, merge_assay_name: str, overwrite: bool
) -> zarr.Group:
if self.outWorkspace is None:
cell_slot = "cellData"
assay_slot = merge_assay_name
else:
cell_slot = f"{self.outWorkspace}/cellData"
- assay_slot = f"{self.outWorkspace}/merge_assay_name"
+ assay_slot = f"{self.outWorkspace}/{merge_assay_name}"
try:
- z = load_zarr(zarr_loc, mode="r")
+ z = load_zarr(zarr_loc, mode="r", storage_options=self.storage_options)
if cell_slot not in z:
raise ValueError(
f"ERROR: Zarr file exists but seems corrupted. Either delete the " # noqa: F541
@@ -546,9 +582,10 @@ def _use_existing_zarr(
"a different zarr path or a different assay name. Otherwise set overwrite to True"
)
try:
+ cell_data = as_zarr_group(z[cell_slot], name=cell_slot)
+ cell_ids = as_zarr_array(cell_data["ids"], name="ids")
if not all(
- z[cell_slot]["ids"][:]
- == np.array(np.array(self.mergedCells["ids"])) # type: ignore
+ np.asarray(cell_ids[:]) == np.array(self.mergedCells["ids"])
):
raise ValueError(
f"ERROR: order of cells does not match the one in existing file" # noqa: F541
@@ -558,18 +595,22 @@ def _use_existing_zarr(
f"ERROR: 'cell data seems corrupted. Either delete the " # noqa: F541
"existing file or choose another path"
)
- return load_zarr(zarr_loc, mode="r+")
- except ValueError:
+ return load_zarr(zarr_loc, mode="r+", storage_options=self.storage_options)
+ except FileNotFoundError:
# So no zarr file with same name exists. Check if a non zarr folder with the same name exists
- if isinstance(zarr_loc, str) and os.path.exists(zarr_loc):
+ if (
+ is_local_zarr_path(zarr_loc)
+ and isinstance(zarr_loc, str)
+ and os.path.exists(zarr_loc)
+ ):
raise ValueError(
f"ERROR: Directory/file with name `{zarr_loc}`exists. "
f"Either delete it or use another name"
)
# creating a new zarr file
- return load_zarr(zarr_loc, mode="w")
+ return load_zarr(zarr_loc, mode="w", storage_options=self.storage_options)
- def _ini_cell_data(self, overwrite) -> None:
+ def _ini_cell_data(self, overwrite: bool) -> None:
"""Save cell attributes to Zarr.
Returns:
@@ -591,37 +632,43 @@ def _ini_cell_data(self, overwrite) -> None:
)
def _dask_to_coo(
- self, d_arr, order: np.ndarray, order_map: np.ndarray, n_threads: int
+ self,
+ d_arr: Any,
+ order: np.ndarray,
+ order_map: np.ndarray,
+ n_threads: int,
) -> coo_matrix:
"""
- Convert a Dask array to a sparse COO matrix.
+ Convert a chunked array block to a sparse COO matrix.
Args:
- d_arr: Dask array to be converted
+ d_arr: Chunked array block to be converted
order: Original feature indices
order_map: Consolidated feature indices
n_threads: Number of threads to use for computation
Returns:
Sparse COO matrix
- This function takes a Dask array and converts it to a sparse COO matrix.
- The `order` is the original feature indices and `order_map` is the consolidated feature indices
- i.e. the indices of the features in the merged assay. If the `order` and `order_map` are the same,
- then the function will directly convert the Dask array to a COO matrix. If they are different,
- then the function will consolidate the data from the Dask array to the COO matrix using the `order_map`.
- For multiple indices mapping to the same consolidated index, the data is summed up.
+ Each source column is remapped through `order_map` to its merged feature
+ index. If multiple source columns map to the same merged feature, their
+ values are summed.
"""
- mat = np.zeros((d_arr.shape[0], self.nFeats))
computed_data = controlled_compute(d_arr, n_threads)
- # Create a mapping from original feature indices to their consolidated indices
- consolidation_map = {orig: cons for orig, cons in zip(order, order_map)}
- # Iterate through the columns of the computed data
- for i, col_data in enumerate(computed_data.T):
- consolidated_idx = consolidation_map[order[i]]
- mat[:, consolidated_idx] += col_data
+ if (
+ order.shape != order_map.shape
+ or order_map.shape[0] != computed_data.shape[1]
+ ):
+ raise ValueError("Feature order does not match the source matrix width")
- return coo_matrix(mat)
+ source = coo_matrix(computed_data)
+ mapped = coo_matrix(
+ (source.data, (source.row, order_map[source.col])),
+ shape=(computed_data.shape[0], self.nFeats),
+ )
+ if not np.array_equal(order, order_map):
+ mapped.sum_duplicates()
+ return mapped
- def dump(self, nthreads=4):
+ def dump(self, nthreads: int = 4) -> None:
"""Copy the values from individual assays to the merged assay.
Args:
@@ -629,32 +676,73 @@ def dump(self, nthreads=4):
Returns:
"""
- counter = 0
- for i, (assay, feat_order, feat_order_map) in enumerate(
- zip(self.assays, self.featOrder, self.featOrder_map)
- ):
- for j, block in tqdmbar(
- enumerate(assay.rawData.blocks),
- total=assay.rawData.numblocks[0],
- desc=f"Writing data from assay {i+1}/{len(self.assays)} to merged file",
- ):
- # Perform the inter-chunk permutation of the rows
- perm_order = self.permutations_rows[i][j]
- perm_order = perm_order - perm_order.min()
- block = block[perm_order, :]
- a = self._dask_to_coo(block, feat_order, feat_order_map, nthreads)
- # Here we use the one-to-one mapping of the chunks in the assays to the chunks in the merged assay to bring the data in the same order.
- row_idx = self.cellOrder[i][j]
- self.assayGroup.set_coordinate_selection(
- (a.row + row_idx.min(), a.col), a.data.astype(self.assayGroup.dtype)
+ assay_blocks = (
+ []
+ if self._usesSharedRowPlan
+ else [list(assay.rawData.blocks) for assay in self.assays]
+ )
+ coordinates = [
+ (int(assay_idx), int(block_idx))
+ for assay_idx, block_idx in self.coordinates_permutations
+ ]
+
+ expected_start = 0
+ for assay_idx, block_idx in coordinates:
+ row_idx = self.cellOrder[assay_idx][block_idx]
+ if row_idx.size == 0 or int(row_idx[0]) != expected_start:
+ raise AssertionError(
+ "ERROR: Merged block order does not match the cell metadata order."
)
- counter += a.shape[0]
- try:
- assert counter == self.nCells
- except AssertionError:
+ expected_start += row_idx.size
+
+ def convert_block(coordinate: tuple[int, int]) -> coo_matrix:
+ assay_idx, block_idx = coordinate
+ perm_order = self.permutations_rows[assay_idx][block_idx]
+ if self._usesSharedRowPlan:
+ row_start = int(perm_order.min())
+ row_end = int(perm_order.max()) + 1
+ block = Block(
+ self.assays[assay_idx].rawData,
+ row_start,
+ row_end,
+ row_perm=perm_order - row_start,
+ )
+ else:
+ local_order = perm_order - perm_order.min()
+ block = assay_blocks[assay_idx][block_idx][local_order, :]
+ return self._dask_to_coo(
+ block,
+ self.featOrder[assay_idx],
+ self.featOrder_map[assay_idx],
+ nthreads,
+ )
+
+ def block_stream() -> Iterator[coo_matrix]:
+ blocks = prefetch_blocks(
+ coordinates,
+ convert_block,
+ max_ahead=worker_prefetch_depth(),
+ )
+ yield from tqdmbar(
+ blocks,
+ total=len(coordinates),
+ desc="Writing merged assay",
+ )
+
+ counter = accumulate_sparse_to_shards(
+ self.assayGroup,
+ block_stream(),
+ dtype=self.assayGroup.dtype,
+ )
+ if counter != self.nCells or expected_start != self.nCells:
raise AssertionError(
"ERROR: Mismatch in number of cells in the merged assay. Please report this issue."
)
+ self.assayGroup = finalize_writer_counts(
+ self.z,
+ self.merge_assay_name,
+ self.outWorkspace,
+ )
# Alias for ZarrMerge
@@ -663,7 +751,7 @@ class ZarrMerge(AssayMerge):
Alias for AssayMerge for backward compatibility.
"""
- def __init__(self, *args, **kwargs):
+ def __init__(self, *args: Any, **kwargs: Any) -> None:
logger.warning(
"The 'ZarrMerge' class is deprecated and will be removed in a future release. Please use 'AssayMerge' instead."
)
@@ -707,18 +795,19 @@ class DatasetMerge:
def __init__(
self,
- datasets: List[DataStore],
+ datasets: list[DataStore],
zarr_path: ZARRLOC,
- names: List[str],
- in_workspaces: Union[list[str], None] = None,
- out_workspace: Union[str, None] = None,
- chunk_size=(1000, 1000),
- dtype: Optional[str] = None,
+ names: list[str],
+ in_workspaces: list[str] | None = None,
+ out_workspace: str | None = None,
+ chunk_size: tuple[int, int] = (1000, 1000),
+ dtype: str | None = None,
overwrite: bool = False,
- prepend_text: Optional[str] = "orig",
+ prepend_text: str | None = "orig",
reset_cell_filter: bool = True,
- seed: Optional[int] = 42,
- ):
+ seed: int | None = 42,
+ storage_options: dict[str, Any] | None = None,
+ ) -> None:
self.datasets = datasets
self.names = names
self.zarr_path = zarr_path
@@ -730,49 +819,68 @@ def __init__(
self.prepend_text = prepend_text
self.reset_cell_filter = reset_cell_filter
self.seed = seed
+ self.storage_options = storage_options
self.unique_assays = self.get_unique_assays()
self.n_unique_assays = len(self.unique_assays)
self.merge_generators = self.create_merge_generators()
- def get_unique_assays(self) -> List[str]:
+ def get_unique_assays(self) -> list[str]:
"""
Get unique assays from both datasets
"""
- unique_assays = set()
+ unique_assays = []
+ seen = set()
for ds in self.datasets:
- unique_assays.update(ds.assay_names)
- return list(unique_assays)
+ for assay_name in ds.assay_names:
+ if assay_name not in seen:
+ seen.add(assay_name)
+ unique_assays.append(assay_name)
+ return unique_assays
- def create_merge_generators(self) -> List[AssayMerge]:
+ def create_merge_generators(self) -> list[AssayMerge]:
"""
Create AssayMerge objects for each unique assay
"""
- gens = []
+ gens: list[AssayMerge] = []
+ shared_row_plan: _RowPlan | None = None
+ row_chunk_sizes = [
+ min(
+ int(ds.get_assay(assay_name).rawData.chunksize[0])
+ for assay_name in ds.assay_names
+ )
+ for ds in self.datasets
+ ]
for assay in self.unique_assays:
- assay_list = []
+ assay_list: list[MergeAssay] = []
for ds in self.datasets:
if assay in ds.assay_names:
assay_list.append(ds.get_assay(assay))
else:
- # Generate a dummy assay on the fly
- dummy_assay: DummyAssay = self.generate_dummy_assay(ds, assay)
- assay_list.append(dummy_assay)
- gens.append(
- AssayMerge(
- zarr_path=self.zarr_path,
- assays=assay_list,
- names=self.names,
- merge_assay_name=assay,
- in_workspaces=self.in_workspaces,
- out_workspace=self.out_workspace,
- chunk_size=self.chunk_size,
- dtype=self.dtype,
- overwrite=self.overwrite,
- prepend_text=self.prepend_text,
- reset_cell_filter=self.reset_cell_filter,
- seed=self.seed,
- )
+ assay_list.append(self.generate_dummy_assay(ds, assay))
+ generator = AssayMerge(
+ zarr_path=self.zarr_path,
+ assays=assay_list,
+ names=self.names,
+ merge_assay_name=assay,
+ in_workspaces=self.in_workspaces,
+ out_workspace=self.out_workspace,
+ chunk_size=self.chunk_size,
+ dtype=self.dtype,
+ overwrite=self.overwrite,
+ prepend_text=self.prepend_text,
+ reset_cell_filter=self.reset_cell_filter,
+ seed=self.seed,
+ storage_options=self.storage_options,
+ _row_plan=shared_row_plan,
+ _row_chunk_sizes=row_chunk_sizes,
)
+ gens.append(generator)
+ if shared_row_plan is None:
+ shared_row_plan = (
+ generator.permutations_rows,
+ generator.permutations_rows_offset,
+ generator.coordinates_permutations,
+ )
return gens
def generate_dummy_assay(self, ds: DataStore, assay_name: str) -> DummyAssay:
@@ -798,17 +906,23 @@ def generate_dummy_assay(self, ds: DataStore, assay_name: str) -> DummyAssay:
# Create a dummy assay with zero counts and matching features
dummy_shape = (ds.cells.N, reference_assay.feats.N)
- dummy_counts = zarr.zeros(
- dummy_shape, chunks=chunkShape, dtype=reference_assay.rawData.dtype
+ mem_store = zarr.storage.MemoryStore()
+ mem_group = zarr.open_group(store=mem_store, mode="w")
+ dummy_array = create_zarr_dataset(
+ mem_group,
+ "counts",
+ chunkShape,
+ reference_assay.rawData.dtype,
+ dummy_shape,
)
- dummy_counts = from_array(dummy_counts, chunks=chunkShape)
+ dummy_counts = ChunkedArray(dummy_array, nthreads=1)
dummy_assay = DummyAssay(
ds, dummy_counts, reference_assay.feats, reference_assay.name
)
logger.info(f"Generated dummy {assay_name} assay for datastore {ds}")
return dummy_assay
- def dump(self, nthreads=4) -> None:
+ def dump(self, nthreads: int = 4) -> None:
"""
Dump the merged data to the zarr file
"""
diff --git a/scarf/metadata.py b/scarf/metadata.py
index 62c28f6f..bca9e4f6 100644
--- a/scarf/metadata.py
+++ b/scarf/metadata.py
@@ -2,20 +2,32 @@
and features."""
import re
-from typing import List, Iterable, Any, Dict, Tuple, Optional, Union
+from collections.abc import Iterable, Iterator
+from dataclasses import dataclass
+from typing import Any
import numpy as np
import pandas as pd
-import polars as pl
-from zarr import hierarchy as z_hierarchy
+import zarr
+from ._types import as_zarr_array
from .feat_utils import fit_lowess
from .utils import logger
from .writers import create_zarr_obj_array
-zarrGroup = z_hierarchy.Group
+zarrGroup = zarr.Group
-__all__ = ["MetaData"]
+__all__ = ["MetaData", "MetaDataRowBlock"]
+
+
+@dataclass(frozen=True, slots=True)
+class MetaDataRowBlock:
+ """One contiguous slice of a MetaData table for blockwise scans."""
+
+ start: int
+ stop: int
+ active_global_indices: np.ndarray
+ values: dict[str, np.ndarray]
def _all_true(bools: np.ndarray) -> np.ndarray:
@@ -29,7 +41,7 @@ def _all_true(bools: np.ndarray) -> np.ndarray:
"""
a = bools.sum(axis=0)
a[a < bools.shape[0]] = 0
- return a.astype(bool)
+ return np.asarray(a, dtype=bool)
class MetaData:
@@ -49,7 +61,7 @@ def __init__(self, zgrp: zarrGroup):
Args:
zgrp: Zarr hierarchy object wherein metadata arrays are saved.
"""
- self.locations: Dict[str, zarrGroup] = {"primary": zgrp}
+ self.locations: dict[str, zarrGroup] = {"primary": zgrp}
self.N = self._get_size(self.locations["primary"], strict_mode=True)
self.index = np.array(range(self.N))
@@ -66,7 +78,9 @@ def _get_size(self, zgrp: zarrGroup, strict_mode: bool = False) -> int:
sizes = []
for i in zgrp.keys():
try:
- sizes.append(zgrp[i].shape[0])
+ child = zgrp[i]
+ if isinstance(child, zarr.Array):
+ sizes.append(child.shape[0])
except Exception:
pass
if len(sizes) > 0:
@@ -96,14 +110,14 @@ def _col_renamer(loc: str, col: str) -> str:
return f"{loc}_{col}"
return col
- def _column_map(self) -> Dict[str, Union[str, Tuple[str, str]]]:
+ def _column_map(self) -> dict[str, str | tuple[str, str]]:
"""
Returns:
"""
reserved_cols = ["I", "ids", "names"]
- col_map: Dict[str, Union[str, Tuple[str, str]]] = {
+ col_map: dict[str, str | tuple[str, str]] = {
x: "primary" for x in reserved_cols
}
for loc, zgrp in self.locations.items():
@@ -117,7 +131,7 @@ def _column_map(self) -> Dict[str, Union[str, Tuple[str, str]]]:
col_map[j] = (loc, i)
return col_map
- def _get_loc(self, column: str) -> Tuple[str, str]:
+ def _get_loc(self, column: str) -> tuple[str, str]:
"""
Args:
@@ -129,10 +143,13 @@ def _get_loc(self, column: str) -> Tuple[str, str]:
col_map = self._column_map()
if column not in col_map:
raise KeyError(f"{column} does not exist in the metadata columns.")
- loc, col = col_map[column]
+ entry = col_map[column]
+ if isinstance(entry, str):
+ return "primary", column
+ loc, col = entry
return loc, col
- def _get_array(self, column: str) -> z_hierarchy.Array:
+ def _get_array(self, column: str) -> zarr.Array:
"""
Args:
@@ -142,9 +159,9 @@ def _get_array(self, column: str) -> z_hierarchy.Array:
"""
loc, col = self._get_loc(column)
- return self.locations[loc][col] # type: ignore
+ return as_zarr_array(self.locations[loc][col], name=col)
- def get_dtype(self, column: str) -> type:
+ def get_dtype(self, column: str) -> np.dtype[Any]:
"""Returns the dtype for the given column.
Args:
@@ -161,7 +178,7 @@ def _verify_bool(self, key: str) -> bool:
Returns: None
"""
- if self.get_dtype(key) != bool:
+ if self.get_dtype(key) != bool: # noqa: E721
raise TypeError(
"ERROR: `key` should be name of a boolean type column in Metadata table"
)
@@ -190,9 +207,9 @@ def mount_location(self, zgrp: zarrGroup, identifier: str) -> None:
cols = self.columns
conflict_names = [x for x in new_cols if x in cols]
if len(conflict_names) > 0:
- conflict_names = " ".join(conflict_names)
+ conflict_str = " ".join(conflict_names)
raise ValueError(
- f"ERROR: These names in location conflict with existing names: {conflict_names}\n. "
+ f"ERROR: These names in location conflict with existing names: {conflict_str}\n. "
f"Please try with a different identifier value."
)
self.locations[identifier] = zgrp
@@ -214,7 +231,7 @@ def unmount_location(self, identifier: str) -> None:
self.locations.pop(identifier)
@property
- def columns(self) -> List[str]:
+ def columns(self) -> list[str]:
"""
Returns:
@@ -231,7 +248,7 @@ def fetch_all(self, column: str) -> np.ndarray:
Returns:
"""
- return self._get_array(column)[:]
+ return np.asarray(self._get_array(column)[:])
def active_index(self, key: str) -> np.ndarray:
"""
@@ -243,7 +260,7 @@ def active_index(self, key: str) -> np.ndarray:
"""
if self._verify_bool(key):
- return self.index[self.fetch_all(key)]
+ return np.asarray(self.index[self.fetch_all(key)])
else:
raise ValueError(
"ERROR: Unexpected error when verifying boolean key. Please report this issue"
@@ -259,7 +276,68 @@ def fetch(self, column: str, key: str = "I") -> np.ndarray:
Returns:
"""
- return self.fetch_all(column)[self.active_index(key)]
+ return np.asarray(self.fetch_all(column)[self.active_index(key)])
+
+ def default_block_rows(self, column: str = "I") -> int:
+ """Prefer the Zarr chunk length of ``column`` for row iteration."""
+ arr = self._get_array(column)
+ chunks = getattr(arr, "chunks", None)
+ if chunks and len(chunks) > 0 and int(chunks[0]) > 0:
+ return int(chunks[0])
+ return max(1, min(self.N, 100_000))
+
+ def iter_row_blocks(
+ self,
+ *,
+ cell_key: str = "I",
+ columns: Iterable[str] | None = None,
+ block_rows: int | None = None,
+ ) -> Iterator[MetaDataRowBlock]:
+ """Yield contiguous row blocks over this table.
+
+ Each block covers a half-open global index range ``[start, stop)``.
+ ``active_global_indices`` lists rows in that range that pass
+ ``cell_key``. Column arrays are aligned to those active indices only.
+
+ When ``block_rows`` is omitted, block size follows the Zarr chunk length
+ of ``cell_key`` (via ``default_block_rows``).
+
+ Concatenating ``active_global_indices`` across blocks equals
+ ``active_index(cell_key)``. Concatenating each column equals
+ ``fetch(column, cell_key)``.
+ """
+ self._verify_bool(cell_key)
+ if block_rows is None:
+ block_rows = self.default_block_rows(cell_key)
+ if block_rows < 1:
+ raise ValueError("block_rows must be >= 1")
+
+ if columns is None:
+ col_list: list[str] = []
+ else:
+ col_list = list(columns)
+ for col in col_list:
+ if col not in self.columns:
+ raise KeyError(f"{col} does not exist in the metadata columns.")
+
+ key_arr = self._get_array(cell_key)
+ col_arrays = {c: self._get_array(c) for c in col_list}
+
+ for start in range(0, self.N, block_rows):
+ stop = min(start + block_rows, self.N)
+ key_slice = np.asarray(key_arr[start:stop], dtype=bool)
+ local = np.flatnonzero(key_slice)
+ active_global = (local + start).astype(np.int64, copy=False)
+ values: dict[str, np.ndarray] = {}
+ for col, arr in col_arrays.items():
+ block = np.asarray(arr[start:stop])
+ values[col] = block[local]
+ yield MetaDataRowBlock(
+ start=start,
+ stop=stop,
+ active_global_indices=active_global,
+ values=values,
+ )
def _save(
self, column_name: str, values: np.ndarray, location: str = "primary"
@@ -287,7 +365,11 @@ def _save(
return None
def _fill_to_index(
- self, values: np.ndarray, fill_value, key: str, auto_fill_disable: bool = False
+ self,
+ values: np.ndarray,
+ fill_value: Any,
+ key: str,
+ auto_fill_disable: bool = False,
) -> np.ndarray:
"""Makes sure that the array being added to the table is of the same
shape. If the array has same shape as the table then the input array is
@@ -337,7 +419,7 @@ def _fill_to_index(
return a
def get_index_by(
- self, value_targets: List[Any], column: str, key: Optional[str] = None
+ self, value_targets: list[Any], column: str, key: str | None = None
) -> np.ndarray:
"""
@@ -353,7 +435,7 @@ def get_index_by(
values = self.fetch_all(column)
else:
values = self.fetch(column, key)
- value_map = {}
+ value_map: dict[str, list[int]] = {}
for n, x in enumerate(values):
x = x.upper()
if x not in value_map:
@@ -395,7 +477,7 @@ def index_to_bool(self, idx: np.ndarray, invert: bool = False) -> np.ndarray:
def insert(
self,
column_name: str,
- values: Union[np.ndarray, List],
+ values: np.ndarray | list,
fill_value: Any = np.nan,
key: str = "I",
overwrite: bool = False,
@@ -506,22 +588,12 @@ def sift(
def multi_sift(
self,
- columns: List[str],
+ columns: list[str],
lows: Iterable,
highs: Iterable,
keep_bounds: bool = False,
) -> np.ndarray:
- """
-
- Args:
- columns:
- lows:
- highs:
- keep_bounds:
-
- Returns:
-
- """
+ """Return boolean mask where all column filters are satisfied."""
ret_val = _all_true(
np.array(
[
@@ -533,48 +605,37 @@ def multi_sift(
return ret_val
def head(self, n: int = 5) -> pd.DataFrame:
- """
-
- Args:
- n:
-
- Returns:
-
- """
+ """Return first ``n`` rows of all metadata columns as a DataFrame."""
df = pd.DataFrame({x: self.fetch_all(x)[:n] for x in self.columns})
return df
def to_pandas_dataframe(
- self, columns: List[str], key: Optional[str] = None
+ self, columns: list[str], key: str | None = None
) -> pd.DataFrame:
- """Returns the requested columns as a Pandas dataframe, sorted on
- key."""
+ """Return requested columns as a DataFrame, optionally filtered by ``key``.
+
+ Args:
+ columns: Metadata column names to include.
+ key: If set, restrict rows to those active for this boolean key.
+
+ Returns:
+ Pandas DataFrame of the selected columns.
+ """
valid_cols = self.columns
df = pd.DataFrame({x: self.fetch_all(x) for x in columns if x in valid_cols})
if key is not None:
df = df.reindex(self.active_index(key))
return df
- def to_polars_dataframe(
- self, columns: List[str], key: Optional[str] = None
- ) -> pl.DataFrame:
- """Returns the requested columns as a Polars DataFrame, sorted on
- key."""
- valid_cols = self.columns
- df = pl.DataFrame({x: self.fetch_all(x) for x in columns if x in valid_cols})
- if key is not None:
- df = df[self.active_index(key)]
- return df
-
- def grep(self, pattern: str, only_valid=False) -> List[str]:
- """
+ def grep(self, pattern: str, only_valid: bool = False) -> list[str]:
+ """Return feature names matching a regex pattern (case insensitive).
Args:
- pattern:
- only_valid:
+ pattern: Regular expression matched against ``names`` column.
+ only_valid: Restrict to features active in ``I``.
Returns:
-
+ Sorted list of matching names.
"""
names = np.array(list(map(str.upper, self.fetch_all("names"))))
if only_valid:
@@ -591,17 +652,17 @@ def remove_trend(
lowess_frac: float = 0.1,
fill_value: float = 0,
) -> np.ndarray:
- """
+ """Remove a LOWESS trend of column ``y`` with respect to column ``x``.
Args:
- x:
- y:
- n_bins:
- lowess_frac:
- fill_value:
+ x: Column name for the independent variable.
+ y: Column name for the dependent variable.
+ n_bins: Bins for LOWESS fitting.
+ lowess_frac: LOWESS smoothing fraction.
+ fill_value: Value returned where trend cannot be estimated.
Returns:
-
+ Residual values of ``y`` after trend removal.
"""
a = self.fetch(x).astype(float)
b = self.fetch(y).astype(float)
@@ -616,5 +677,5 @@ def remove_trend(
ret_val[idx] = c
return ret_val
- def __repr__(self):
+ def __repr__(self) -> str:
return f"MetaData of {self.fetch_all('I').sum()}({self.N}) elements"
diff --git a/scarf/metrics.py b/scarf/metrics.py
index fe91a200..7bc11c52 100644
--- a/scarf/metrics.py
+++ b/scarf/metrics.py
@@ -1,26 +1,133 @@
"""
-Methods and classes for evluation
+Methods and classes for evaluation
"""
-from typing import Iterable, Optional, Sequence, Tuple, Union
+from collections.abc import Iterable, Sequence
+from typing import Literal, cast
import numpy as np
import pandas as pd
from scipy.sparse import csr_matrix
-from zarr.core import Array as zarrArrayType
+import zarr
from .ann import AnnStream
+from .chunked import ChunkedArray
from .datastore.datastore import DataStore
from .utils import (
logger,
tqdmbar,
)
+type ZarrArray = zarr.Array
+type MatrixData = np.ndarray | ZarrArray | ChunkedArray
+type NeighborMetric = Literal["l2", "cosine", "ip"]
+
+_LISI_BATCH_SIZE = 10_000
+_EDGE_BATCH_ROWS = 100_000
+
# LISI - The Local Inverse Simpson Index
+def _effective_perplexity(perplexity: float, n_neighbors: int) -> float:
+ if not np.isfinite(perplexity) or perplexity < 1:
+ raise ValueError("Perplexity must be a finite value greater than or equal to 1")
+ if n_neighbors < 3:
+ raise ValueError("LISI requires at least three neighbors per cell")
+
+ max_perplexity = n_neighbors / 3
+ if perplexity > max_perplexity:
+ logger.warning(
+ f"Perplexity {perplexity:g} requires at least "
+ f"{int(np.ceil(3 * perplexity))} neighbors, but the graph has "
+ f"{n_neighbors}. Using perplexity {max_perplexity:g}."
+ )
+ return max_perplexity
+ return perplexity
+
+
+def _neighbor_probabilities(
+ distances: np.ndarray,
+ perplexity: float,
+ tol: float,
+ max_iter: int = 50,
+) -> np.ndarray:
+ distances = np.asarray(distances, dtype=np.float64)
+ if distances.ndim != 2 or distances.shape[1] == 0:
+ raise ValueError("Distances must be a non-empty two-dimensional matrix")
+ if not np.all(np.isfinite(distances)) or np.any(distances < 0):
+ raise ValueError("Distances must contain finite, non-negative values")
+
+ centered = distances - distances.min(axis=1, keepdims=True)
+ n_points = distances.shape[0]
+ beta = np.ones(n_points, dtype=np.float64)
+ beta_min = np.full(n_points, -np.inf, dtype=np.float64)
+ beta_max = np.full(n_points, np.inf, dtype=np.float64)
+ target_entropy = np.log(perplexity)
+
+ for _ in range(max_iter):
+ with np.errstate(over="ignore", under="ignore"):
+ weights = np.exp(-centered * beta[:, None])
+ weight_sums = weights.sum(axis=1)
+ entropy = np.log(weight_sums) + (
+ beta * np.sum(centered * weights, axis=1) / weight_sums
+ )
+ entropy_diff = entropy - target_entropy
+ active = np.abs(entropy_diff) >= tol
+ if not np.any(active):
+ break
+
+ too_diffuse = active & (entropy_diff > 0)
+ too_concentrated = active & ~too_diffuse
+
+ beta_min[too_diffuse] = beta[too_diffuse]
+ bounded_above = too_diffuse & np.isfinite(beta_max)
+ beta[bounded_above] = (beta[bounded_above] + beta_max[bounded_above]) / 2
+ beta[too_diffuse & ~np.isfinite(beta_max)] *= 2
+
+ beta_max[too_concentrated] = beta[too_concentrated]
+ bounded_below = too_concentrated & np.isfinite(beta_min)
+ beta[bounded_below] = (beta[bounded_below] + beta_min[bounded_below]) / 2
+ beta[too_concentrated & ~np.isfinite(beta_min)] /= 2
+
+ with np.errstate(over="ignore", under="ignore"):
+ probabilities = np.exp(-centered * beta[:, None])
+ probabilities /= probabilities.sum(axis=1, keepdims=True)
+ return np.asarray(probabilities)
+
+
+def _simpson_from_probabilities(
+ probabilities: np.ndarray,
+ indices: np.ndarray,
+ label_codes: np.ndarray,
+ n_categories: int,
+) -> np.ndarray:
+ indices = np.asarray(indices)
+ if indices.shape != probabilities.shape:
+ raise ValueError("Neighbor indices and probabilities must have matching shapes")
+ if not np.issubdtype(indices.dtype, np.integer):
+ raise TypeError("Neighbor indices must contain integers")
+ if np.any(indices < 0) or np.any(indices >= len(label_codes)):
+ raise IndexError("Neighbor index is outside the label array")
+
+ neighbor_codes = label_codes[indices]
+ simpson = np.zeros(probabilities.shape[0], dtype=np.float64)
+ if n_categories <= probabilities.shape[1]:
+ for category in range(n_categories):
+ category_mass = np.sum(probabilities * (neighbor_codes == category), axis=1)
+ simpson += np.square(category_mass)
+ else:
+ for neighbor in range(probabilities.shape[1]):
+ same_category = neighbor_codes == neighbor_codes[:, neighbor, None]
+ category_mass = np.sum(probabilities * same_category, axis=1)
+ simpson += probabilities[:, neighbor] * category_mass
+
+ if not np.all(np.isfinite(simpson)) or np.any(simpson <= 0):
+ raise FloatingPointError("Could not compute finite Simpson indices")
+ return np.clip(simpson, np.finfo(np.float64).tiny, 1.0)
+
+
def compute_lisi(
- distances: zarrArrayType,
- indices: zarrArrayType,
+ distances: np.ndarray | ZarrArray,
+ indices: np.ndarray | ZarrArray,
metadata: pd.DataFrame,
label_colnames: Iterable[str],
perplexity: float = 30,
@@ -50,20 +157,53 @@ def compute_lisi(
Korsunsky et al. 2019 doi: 10.1038/s41592-019-0619-0
"""
- n_cells = metadata.shape[0]
- n_labels = len(label_colnames)
- # Don't count yourself
- indices = indices[:, 1:]
- distances = distances[:, 1:]
- lisi_df = np.zeros((n_cells, n_labels))
- for i, label in enumerate(label_colnames):
+ if distances.ndim != 2 or indices.ndim != 2:
+ raise ValueError("KNN distances and indices must be two-dimensional")
+ if distances.shape != indices.shape:
+ raise ValueError("KNN distances and indices must have matching shapes")
+
+ n_cells, n_neighbors = distances.shape
+ if metadata.shape[0] != n_cells:
+ raise ValueError(
+ "Metadata rows must match the number of cells in the KNN graph"
+ )
+
+ label_cols = list(label_colnames)
+ n_labels = len(label_cols)
+ if n_labels == 0:
+ return np.empty((n_cells, 0), dtype=np.float64)
+
+ effective_perplexity = _effective_perplexity(perplexity, n_neighbors)
+ categoricals: list[pd.Categorical] = []
+ for label in label_cols:
+ categorical = pd.Categorical(metadata[label])
+ if np.any(categorical.codes < 0):
+ raise ValueError(f"Label column {label!r} contains missing values")
+ categoricals.append(categorical)
logger.info(f"Computing LISI for {label}")
- labels = pd.Categorical(metadata[label])
- n_categories = len(labels.categories)
- simpson = compute_simpson(
- distances.T, indices.T, labels, n_categories, perplexity
+
+ chunk_rows = _LISI_BATCH_SIZE
+ if isinstance(distances, zarr.Array):
+ chunk_rows = min(chunk_rows, int(distances.chunks[0]))
+
+ lisi_df = np.empty((n_cells, n_labels), dtype=np.float64)
+ starts = range(0, n_cells, chunk_rows)
+ total = int(np.ceil(n_cells / chunk_rows))
+ for start in tqdmbar(starts, total=total, desc="Computing LISI"):
+ end = min(start + chunk_rows, n_cells)
+ distance_block = np.asarray(distances[start:end])
+ index_block = np.asarray(indices[start:end])
+ probabilities = _neighbor_probabilities(
+ distance_block, effective_perplexity, tol=1e-5
)
- lisi_df[:, i] = 1 / simpson
+ for label_index, labels in enumerate(categoricals):
+ simpson = _simpson_from_probabilities(
+ probabilities,
+ index_block,
+ np.asarray(labels.codes),
+ len(labels.categories),
+ )
+ lisi_df[start:end, label_index] = 1 / simpson
return lisi_df
@@ -89,91 +229,114 @@ def compute_simpson(
Returns:
np.ndarray: Array of Simpson's diversity indices, one per point
"""
- n = distances.shape[1]
- P = np.zeros(distances.shape[0])
- simpson = np.zeros(n)
- logU = np.log(perplexity)
- # Loop through each cell.
- for i in tqdmbar(range(n), desc="Computing Simpson's Diversity Index"):
- beta = 1
- betamin = -np.inf
- betamax = np.inf
- # Compute Hdiff
- P = np.exp(-distances[:, i] * beta)
- P_sum = np.sum(P)
- if P_sum == 0:
- H = 0
- P = np.zeros(distances.shape[0])
- else:
- H = np.log(P_sum) + beta * np.sum(distances[:, i] * P) / P_sum
- P = P / P_sum
- Hdiff = H - logU
- n_tries = 50
- for t in range(n_tries):
- # Stop when we reach the tolerance
- if abs(Hdiff) < tol:
- break
- # Update beta
- if Hdiff > 0:
- betamin = beta
- if not np.isfinite(betamax):
- beta *= 2
- else:
- beta = (beta + betamax) / 2
- else:
- betamax = beta
- if not np.isfinite(betamin):
- beta /= 2
- else:
- beta = (beta + betamin) / 2
- # Compute Hdiff
- P = np.exp(-distances[:, i] * beta)
- P_sum = np.sum(P)
- if P_sum == 0:
- H = 0
- P = np.zeros(distances.shape[0])
- else:
- H = np.log(P_sum) + beta * np.sum(distances[:, i] * P) / P_sum
- P = P / P_sum
- Hdiff = H - logU
- # distancesefault value
- if H == 0:
- simpson[i] = -1
- # Simpson's index
- for label_category in labels.categories:
- ix = indices[:, i]
- q = labels[ix] == label_category
- if np.any(q):
- P_sum = np.sum(P[q])
- simpson[i] += P_sum * P_sum
- return simpson
+ distances = np.asarray(distances)
+ indices = np.asarray(indices)
+ if distances.ndim != 2 or indices.ndim != 2:
+ raise ValueError("Distances and indices must be two-dimensional")
+ if distances.shape != indices.shape:
+ raise ValueError("Distances and indices must have matching shapes")
+ if np.any(labels.codes < 0):
+ raise ValueError("Labels contain missing values")
+
+ n_neighbors = distances.shape[0]
+ effective_perplexity = _effective_perplexity(perplexity, n_neighbors)
+ probabilities = _neighbor_probabilities(distances.T, effective_perplexity, tol=tol)
+ return _simpson_from_probabilities(
+ probabilities,
+ indices.T,
+ np.asarray(labels.codes),
+ len(labels.categories),
+ )
# SILHOUETTE SCORE - The Silhouette Score
def knn_to_csr_matrix(
- neighbor_indices: np.ndarray, neighbor_distances: np.ndarray
+ neighbor_indices: np.ndarray,
+ neighbor_distances: np.ndarray,
+ *,
+ use_affinities: bool = False,
) -> csr_matrix:
"""Convert k-nearest neighbors data to a Compressed Sparse Row (CSR) matrix.
- Creates a sparse adjacency matrix representation of the k-nearest neighbors graph
- where edge weights are the distances between points.
+ Creates a sparse adjacency matrix representation of a KNN graph. Distances
+ can optionally be converted to affinities.
Args:
neighbor_indices: Indices matrix from k-nearest neighbors, shape (n_samples, k)
neighbor_distances: Distances matrix from k-nearest neighbors, shape (n_samples, k)
+ use_affinities: Convert distances using ``1 / (log1p(distance) + 1)``.
Returns:
scipy.sparse.csr_matrix: Sparse adjacency matrix of shape (n_samples, n_samples)
- where non-zero entries represent distances between neighboring points
+ where non-zero entries represent neighbor weights
"""
+ neighbor_indices = np.asarray(neighbor_indices)
+ neighbor_distances = np.asarray(neighbor_distances, dtype=np.float64)
+ if neighbor_indices.ndim != 2 or neighbor_distances.ndim != 2:
+ raise ValueError("Neighbor indices and distances must be two-dimensional")
+ if neighbor_indices.shape != neighbor_distances.shape:
+ raise ValueError("Neighbor indices and distances must have matching shapes")
+ if not np.issubdtype(neighbor_indices.dtype, np.integer):
+ raise TypeError("Neighbor indices must contain integers")
+ if not np.all(np.isfinite(neighbor_distances)) or np.any(neighbor_distances < 0):
+ raise ValueError("Neighbor distances must be finite and non-negative")
+
num_samples, num_neighbors = neighbor_indices.shape
- row_indices = np.repeat(np.arange(num_samples), num_neighbors)
+ if num_samples == 0 or num_neighbors == 0:
+ raise ValueError("KNN data must contain cells and neighbors")
+ if np.any(neighbor_indices < 0) or np.any(neighbor_indices >= num_samples):
+ raise IndexError("Neighbor index is outside the graph")
+
+ weights = neighbor_distances
+ if use_affinities:
+ weights = 1 / (np.log1p(neighbor_distances) + 1)
+
+ indptr = np.arange(
+ 0,
+ num_samples * num_neighbors + 1,
+ num_neighbors,
+ dtype=np.int64,
+ )
return csr_matrix(
- (neighbor_distances[:].flatten(), (row_indices, neighbor_indices[:].flatten())),
+ (weights.reshape(-1), neighbor_indices.reshape(-1), indptr),
shape=(num_samples, num_samples),
)
+def _validated_cluster_labels(
+ cluster_labels: np.ndarray, n_nodes: int
+) -> tuple[np.ndarray, int]:
+ cluster_labels = np.asarray(cluster_labels)
+ if len(cluster_labels) != n_nodes:
+ raise ValueError("Cluster labels must have one value per graph node")
+ if not np.issubdtype(cluster_labels.dtype, np.integer):
+ raise TypeError("Cluster labels must contain integers")
+
+ unique_cluster_ids = np.unique(cluster_labels)
+ expected_cluster_ids = np.arange(0, len(unique_cluster_ids))
+ if not np.array_equal(unique_cluster_ids, expected_cluster_ids):
+ raise ValueError("Cluster labels must be contiguous integers starting at 0")
+ return cluster_labels, len(unique_cluster_ids)
+
+
+def _finalize_cluster_similarity(inter_cluster_weights: np.ndarray) -> np.ndarray:
+ inter_cluster_weights = (inter_cluster_weights + inter_cluster_weights.T) / 2
+ total_cluster_weights = inter_cluster_weights.sum(axis=1)
+ weight_union = (
+ total_cluster_weights[:, None]
+ + total_cluster_weights[None, :]
+ - inter_cluster_weights
+ )
+ similarity_matrix = np.divide(
+ inter_cluster_weights,
+ weight_union,
+ out=np.zeros_like(inter_cluster_weights),
+ where=weight_union > 0,
+ )
+ np.fill_diagonal(similarity_matrix, 1.0)
+ return np.asarray(similarity_matrix)
+
+
def calculate_weighted_cluster_similarity(
knn_graph: csr_matrix, cluster_labels: np.ndarray
) -> np.ndarray:
@@ -191,59 +354,96 @@ def calculate_weighted_cluster_similarity(
pairwise similarities between clusters
Raises:
- AssertionError: If cluster labels are not contiguous integers starting from 0
+ ValueError: If cluster labels are not contiguous integers starting from 0
"""
- unique_cluster_ids = np.unique(cluster_labels)
- expected_cluster_ids = np.arange(0, len(unique_cluster_ids))
- assert np.array_equal(unique_cluster_ids, expected_cluster_ids), (
- "Cluster labels must be contiguous integers starting at 1"
- )
-
- num_clusters = len(unique_cluster_ids)
- inter_cluster_weights = np.zeros((num_clusters, num_clusters))
-
- for cluster_id in unique_cluster_ids:
- nodes_in_cluster = np.where(cluster_labels == cluster_id)[0]
- neighbor_cluster_labels = cluster_labels[knn_graph[nodes_in_cluster].indices]
- neighbor_edge_weights = knn_graph[nodes_in_cluster].data
-
- for neighbor_cluster, edge_weight in zip(
- neighbor_cluster_labels, neighbor_edge_weights
- ):
- inter_cluster_weights[cluster_id, neighbor_cluster] += edge_weight
-
- # assert inter_cluster_weights.sum() == knn_graph.data.sum()
-
- # Ensure symmetry
- inter_cluster_weights = (inter_cluster_weights + inter_cluster_weights.T) / 2
-
- # Calculate total weights for each cluster
- total_cluster_weights = np.array(
- [inter_cluster_weights[i - 1].sum() for i in unique_cluster_ids]
+ knn_graph = knn_graph.tocsr(copy=False)
+ if knn_graph.shape[0] != knn_graph.shape[1]:
+ raise ValueError("KNN graph must be square")
+ if not np.all(np.isfinite(knn_graph.data)) or np.any(knn_graph.data < 0):
+ raise ValueError("KNN graph weights must be finite and non-negative")
+
+ cluster_labels, num_clusters = _validated_cluster_labels(
+ cluster_labels, knn_graph.shape[0]
)
+ inter_cluster_weights = np.zeros((num_clusters, num_clusters), dtype=np.float64)
+ for row_start in range(0, knn_graph.shape[0], _EDGE_BATCH_ROWS):
+ row_end = min(row_start + _EDGE_BATCH_ROWS, knn_graph.shape[0])
+ edge_start = knn_graph.indptr[row_start]
+ edge_end = knn_graph.indptr[row_end]
+ source_clusters = np.repeat(
+ cluster_labels[row_start:row_end],
+ np.diff(knn_graph.indptr[row_start : row_end + 1]),
+ )
+ target_clusters = cluster_labels[knn_graph.indices[edge_start:edge_end]]
+ pair_ids = source_clusters * num_clusters + target_clusters
+ weight_counts = np.asarray(
+ np.bincount(
+ pair_ids,
+ weights=knn_graph.data[edge_start:edge_end],
+ minlength=num_clusters * num_clusters,
+ ),
+ dtype=np.float64,
+ )
+ inter_cluster_weights += weight_counts.reshape(num_clusters, num_clusters)
- # Calculate similarity using weighted Jaccard index
- similarity_matrix = np.zeros((num_clusters, num_clusters))
+ return _finalize_cluster_similarity(inter_cluster_weights)
- for i in range(num_clusters):
- for j in range(i, num_clusters):
- weight_union = (
- total_cluster_weights[i]
- + total_cluster_weights[j]
- - inter_cluster_weights[i, j]
- )
- if weight_union > 0:
- similarity = inter_cluster_weights[i, j] / weight_union
- similarity_matrix[i, j] = similarity_matrix[j, i] = similarity
- # Set diagonal to 1 (self-similarity)
- # np.fill_diagonal(similarity_matrix, 1.0)
+def calculate_knn_cluster_similarity(
+ neighbor_indices: np.ndarray | ZarrArray,
+ neighbor_distances: np.ndarray | ZarrArray,
+ cluster_labels: np.ndarray,
+ *,
+ batch_rows: int = _EDGE_BATCH_ROWS,
+) -> np.ndarray:
+ """Stream KNN rows and calculate cluster similarity from distance affinities."""
+ if neighbor_indices.ndim != 2 or neighbor_distances.ndim != 2:
+ raise ValueError("Neighbor indices and distances must be two-dimensional")
+ if neighbor_indices.shape != neighbor_distances.shape:
+ raise ValueError("Neighbor indices and distances must have matching shapes")
+ if neighbor_indices.shape[1] == 0:
+ raise ValueError("KNN data must contain neighbors")
+ if batch_rows < 1:
+ raise ValueError("batch_rows must be greater than zero")
+
+ n_nodes, n_neighbors = neighbor_indices.shape
+ cluster_labels, num_clusters = _validated_cluster_labels(cluster_labels, n_nodes)
+ inter_cluster_weights = np.zeros((num_clusters, num_clusters), dtype=np.float64)
+ for row_start in range(0, n_nodes, batch_rows):
+ row_end = min(row_start + batch_rows, n_nodes)
+ index_block = np.asarray(neighbor_indices[row_start:row_end])
+ distance_block = np.asarray(
+ neighbor_distances[row_start:row_end], dtype=np.float64
+ )
+ if not np.issubdtype(index_block.dtype, np.integer):
+ raise TypeError("Neighbor indices must contain integers")
+ if np.any(index_block < 0) or np.any(index_block >= n_nodes):
+ raise IndexError("Neighbor index is outside the graph")
+ if not np.all(np.isfinite(distance_block)) or np.any(distance_block < 0):
+ raise ValueError("Neighbor distances must be finite and non-negative")
+
+ affinities = 1 / (np.log1p(distance_block) + 1)
+ source_clusters = np.repeat(cluster_labels[row_start:row_end], n_neighbors)
+ target_clusters = cluster_labels[index_block.reshape(-1)]
+ pair_ids = source_clusters * num_clusters + target_clusters
+ weight_counts = np.asarray(
+ np.bincount(
+ pair_ids,
+ weights=affinities.reshape(-1),
+ minlength=num_clusters * num_clusters,
+ ),
+ dtype=np.float64,
+ )
+ inter_cluster_weights += weight_counts.reshape(num_clusters, num_clusters)
- return similarity_matrix
+ return _finalize_cluster_similarity(inter_cluster_weights)
def calculate_top_k_neighbor_distances(
- matrix_a: np.ndarray, matrix_b: np.ndarray, k: int
+ matrix_a: np.ndarray,
+ matrix_b: np.ndarray,
+ k: int,
+ metric: NeighborMetric = "l2",
) -> np.ndarray:
"""Calculate distances to k nearest neighbors between two sets of points.
@@ -260,39 +460,96 @@ def calculate_top_k_neighbor_distances(
k nearest neighbors in matrix_b for each point in matrix_a
Raises:
- AssertionError: If matrices don't have the same number of features
+ ValueError: If the inputs are empty, incompatible, or k is invalid
"""
- # Check if the matrices have the same number of features (d)
- assert matrix_a.shape[1] == matrix_b.shape[1], (
- "Matrices must have the same number of features"
- )
+ matrix_a = np.asarray(matrix_a, dtype=np.float64)
+ matrix_b = np.asarray(matrix_b, dtype=np.float64)
+ if matrix_a.ndim != 2 or matrix_b.ndim != 2:
+ raise ValueError("Input matrices must be two-dimensional")
+ if matrix_a.shape[1] != matrix_b.shape[1]:
+ raise ValueError("Matrices must have the same number of features")
+ if matrix_a.shape[0] == 0 or matrix_b.shape[0] == 0:
+ raise ValueError("Input matrices must contain at least one point")
+ if not np.all(np.isfinite(matrix_a)) or not np.all(np.isfinite(matrix_b)):
+ raise ValueError("Input matrices must contain finite values")
+ if k < 1:
+ raise ValueError("k must be greater than zero")
# Ensure k is not larger than the number of points in matrix_b
k = min(k, matrix_b.shape[0])
- # Calculate squared Euclidean distances
- a_squared = np.sum(np.square(matrix_a), axis=1, keepdims=True)
- b_squared = np.sum(np.square(matrix_b), axis=1)
+ products = np.dot(matrix_a, matrix_b.T)
+ if metric == "l2":
+ a_squared = np.sum(np.square(matrix_a), axis=1, keepdims=True)
+ b_squared = np.sum(np.square(matrix_b), axis=1)
+ distances = np.sqrt(np.maximum(a_squared + b_squared - 2 * products, 0))
+ elif metric == "cosine":
+ norms = np.outer(
+ np.linalg.norm(matrix_a, axis=1),
+ np.linalg.norm(matrix_b, axis=1),
+ )
+ similarities = np.divide(
+ products,
+ norms,
+ out=np.zeros_like(products),
+ where=norms > 0,
+ )
+ distances = np.clip(1 - similarities, 0, 2)
+ elif metric == "ip":
+ distances = np.maximum(1 - products, 0)
+ else:
+ raise ValueError(f"Unsupported neighbor metric: {metric}")
- # Use broadcasting to compute pairwise distances
- distances = a_squared + b_squared - 2 * np.dot(matrix_a, matrix_b.T)
+ # Find the k smallest distances for each point in matrix_a
+ return np.partition(distances, k - 1, axis=1)[:, :k]
- # Use np.maximum to avoid small negative values due to floating point errors
- distances = np.maximum(distances, 0)
- # Find the k smallest distances for each point in matrix_a
- top_k_distances = np.partition(distances, k, axis=1)[:, :k]
+def _read_rows(data: MatrixData, row_indices: np.ndarray) -> np.ndarray:
+ row_indices = np.asarray(row_indices, dtype=np.int64)
+ if isinstance(data, ChunkedArray):
+ return data[row_indices].compute()
+ if isinstance(data, zarr.Array):
+ return np.asarray(data.get_orthogonal_selection((row_indices, slice(None))))
+ return np.asarray(data[row_indices])
- # Calculate the square root to get Euclidean distances
- return np.sqrt(top_k_distances)
+
+def _embed_rows(
+ row_indices: np.ndarray,
+ data: MatrixData,
+ ann_obj: AnnStream,
+ *,
+ data_is_reduced: bool,
+) -> np.ndarray:
+ rows = _read_rows(data, np.sort(row_indices))
+ if data_is_reduced:
+ return np.asarray(rows)
+ return np.asarray(ann_obj.reducer(rows))
+
+
+def _sample_cluster_embeddings(
+ cluster_cells: np.ndarray,
+ data: MatrixData,
+ ann_obj: AnnStream,
+ count: int,
+ rng: np.random.Generator,
+ *,
+ data_is_reduced: bool,
+) -> np.ndarray:
+ if count < 1 or count > len(cluster_cells):
+ raise ValueError("Sample count must fit within the cluster")
+ sampled = rng.choice(cluster_cells, size=count, replace=False)
+ return _embed_rows(sampled, data, ann_obj, data_is_reduced=data_is_reduced)
def process_cluster(
cluster_cells: np.ndarray,
- hvg_data: Union[np.ndarray, zarrArrayType],
+ hvg_data: MatrixData,
ann_obj: AnnStream,
k: int,
-) -> Tuple[np.ndarray, np.ndarray]:
+ *,
+ rng: np.random.Generator | None = None,
+ data_is_reduced: bool = False,
+) -> tuple[np.ndarray, np.ndarray]:
"""Process a cluster of cells to prepare data for silhouette scoring.
Randomly splits cluster cells into two groups and applies dimensionality reduction.
@@ -304,15 +561,26 @@ def process_cluster(
k: Number of cells to sample from cluster
Returns:
- Tuple[np.ndarray, np.ndarray]: Two arrays containing reduced data for
+ tuple[np.ndarray, np.ndarray]: Two arrays containing reduced data for
different subsets of cells from the cluster
"""
- np.random.shuffle(cluster_cells)
- data_cells = np.array(
- [ann_obj.reducer(hvg_data[i]) for i in sorted(cluster_cells[:k])]
+ if k < 1 or len(cluster_cells) < 2 * k:
+ raise ValueError("A cluster must contain at least 2 * k cells")
+ if rng is None:
+ rng = np.random.default_rng(4444)
+
+ selected = rng.choice(cluster_cells, size=2 * k, replace=False)
+ data_cells = _embed_rows(
+ selected[:k],
+ hvg_data,
+ ann_obj,
+ data_is_reduced=data_is_reduced,
)
- data_cells_2 = np.array(
- [ann_obj.reducer(hvg_data[i]) for i in sorted(cluster_cells[k : 2 * k])]
+ data_cells_2 = _embed_rows(
+ selected[k:],
+ hvg_data,
+ ann_obj,
+ data_is_reduced=data_is_reduced,
)
return data_cells, data_cells_2
@@ -320,11 +588,19 @@ def process_cluster(
def silhouette_scoring(
ds: DataStore,
ann_obj: AnnStream,
- graph: csr_matrix,
- hvg_data: Union[np.ndarray, zarrArrayType],
+ graph: csr_matrix | None,
+ hvg_data: MatrixData,
assay_type: str,
res_label: str,
-) -> Optional[np.ndarray]:
+ *,
+ cell_key: str = "I",
+ random_seed: int = 4444,
+ sample_size: int = 11,
+ data_is_reduced: bool = False,
+ distance_metric: NeighborMetric | None = None,
+ neighbor_indices: np.ndarray | ZarrArray | None = None,
+ neighbor_distances: np.ndarray | ZarrArray | None = None,
+) -> np.ndarray | None:
"""Compute modified silhouette scores for clusters in single-cell data.
This implementation differs from the standard silhouette score by using
@@ -333,133 +609,216 @@ def silhouette_scoring(
Args:
ds: DataStore object containing cell metadata
ann_obj: Object containing dimensionality reduction method
- graph: CSR matrix representing the KNN graph
+ graph: Optional CSR matrix representing the weighted KNN graph
hvg_data: Expression data for highly variable genes
assay_type: Type of assay (e.g., 'RNA', 'ATAC')
res_label: Label for clustering resolution
Returns:
- Optional[np.ndarray]: Array of silhouette scores for each cluster,
+ np.ndarray | None: Array of silhouette scores for each cluster,
or None if cluster labels are not found
Notes:
Scores are calculated using a sampling approach for efficiency.
NaN values indicate clusters that couldn't be scored due to size constraints.
"""
+ if sample_size < 1:
+ raise ValueError("sample_size must be greater than zero")
+
+ if res_label in ds.cells.columns:
+ cluster_column = res_label
+ else:
+ prefix = assay_type if cell_key == "I" else f"{assay_type}_{cell_key}"
+ cluster_column = f"{prefix}_{res_label}"
+
try:
- clusters = ds.cells.fetch(f"{assay_type}_{res_label}") - 1 # RNA_{res_label}
+ raw_clusters = ds.cells.fetch(cluster_column, key=cell_key)
except KeyError:
- logger.error(f"Cluster labels not found for {assay_type}_{res_label}")
+ logger.error(f"Cluster labels not found for {cluster_column}")
return None
- cluster_similarity = calculate_weighted_cluster_similarity(graph, clusters)
-
- k = 11
- score = []
-
- for n, i in enumerate(cluster_similarity):
- this_cluster_cells = np.where(clusters == n)[0]
- if len(this_cluster_cells) < 2 * k:
- k = int(len(this_cluster_cells) / 2)
- logger.warning(
- f"Warning: Cluster {n} has fewer than 22 cells. Will adjust k to {k} instead"
- )
+ categorical = pd.Categorical(raw_clusters)
+ if np.any(categorical.codes < 0):
+ raise ValueError(f"Cluster column {cluster_column!r} contains missing values")
+ clusters = np.asarray(categorical.codes, dtype=np.int64)
+ if hvg_data.shape[0] != len(clusters):
+ raise ValueError(
+ "Embedding data and cluster labels must contain the same cells"
+ )
- for n, i in tqdmbar(
+ if graph is not None:
+ if graph.shape != (len(clusters), len(clusters)):
+ raise ValueError("KNN graph and cluster labels must contain the same cells")
+ cluster_similarity = calculate_weighted_cluster_similarity(graph, clusters)
+ elif neighbor_indices is not None and neighbor_distances is not None:
+ cluster_similarity = calculate_knn_cluster_similarity(
+ neighbor_indices,
+ neighbor_distances,
+ clusters,
+ )
+ else:
+ raise ValueError("Provide a KNN graph or neighbor indices and distances")
+ num_clusters = len(categorical.categories)
+ if num_clusters < 2:
+ logger.warning("Silhouette scoring requires at least two clusters")
+ return np.full(num_clusters, np.nan)
+
+ order = np.argsort(clusters, kind="stable")
+ counts = np.bincount(clusters, minlength=num_clusters)
+ boundaries = np.cumsum(counts)
+ starts = np.concatenate(([0], boundaries[:-1]))
+ cluster_cells = [order[start:end] for start, end in zip(starts, boundaries)]
+
+ metric = distance_metric or cast(NeighborMetric, ann_obj.annMetric)
+ if metric not in {"l2", "cosine", "ip"}:
+ raise ValueError(f"Unsupported neighbor metric: {metric}")
+
+ rng = np.random.default_rng(random_seed)
+ score: list[float] = []
+ for cluster_id, similarities in tqdmbar(
enumerate(cluster_similarity),
total=len(cluster_similarity),
desc="Calculating Silhouette Scores",
):
- this_cluster_cells = np.where(clusters == n)[0]
- np.random.shuffle(this_cluster_cells)
+ this_cluster_cells = cluster_cells[cluster_id]
+ k = min(sample_size, len(this_cluster_cells) // 2)
+ if k < 1:
+ logger.warning(
+ f"Cluster {categorical.categories[cluster_id]!r} has fewer than two cells"
+ )
+ score.append(np.nan)
+ continue
+
data_this_cells, data_this_cells_2 = process_cluster(
- # n,
this_cluster_cells,
hvg_data,
ann_obj,
k,
+ rng=rng,
+ data_is_reduced=data_is_reduced,
)
- if data_this_cells.size == 0 or data_this_cells_2.size == 0:
- logger.warning(f"Warning: Reduced data for cluster {n} is empty. Skipping.")
- score.append(np.nan)
- continue
-
- k_neighbors = min(k - 1, data_this_cells_2.shape[0] - 1)
-
- if k_neighbors < 1:
- logger.warning(
- f"Warning: Not enough points in cluster {n} for comparison. Skipping."
- )
- score.append(np.nan)
- continue
-
self_dist = calculate_top_k_neighbor_distances(
- data_this_cells, data_this_cells_2, k - 1
+ data_this_cells,
+ data_this_cells_2,
+ min(k, len(data_this_cells_2)),
+ metric=metric,
).mean()
- nearest_cluster = np.argsort(i)[-1]
- nearest_cluster_cells = np.where(clusters == nearest_cluster)[0]
- np.random.shuffle(nearest_cluster_cells)
-
- if len(nearest_cluster_cells) < k:
- logger.warning(
- f"Warning: Nearest cluster {nearest_cluster} has fewer than {k} cells. Skipping."
- )
- score.append(np.nan)
- continue
-
- data_nearest_cells, _ = process_cluster(
- # nearest_cluster,
+ other_similarities = similarities.copy()
+ other_similarities[cluster_id] = -np.inf
+ nearest_cluster = int(np.argmax(other_similarities))
+ nearest_cluster_cells = cluster_cells[nearest_cluster]
+ nearest_sample_size = min(k, len(nearest_cluster_cells))
+ data_nearest_cells = _sample_cluster_embeddings(
nearest_cluster_cells,
hvg_data,
ann_obj,
- k,
+ nearest_sample_size,
+ rng,
+ data_is_reduced=data_is_reduced,
)
- if data_nearest_cells.size == 0:
- logger.warning(
- f"Warning: Reduced data for nearest cluster {nearest_cluster} is empty. Skipping."
- )
- score.append(np.nan)
- continue
-
other_dist = calculate_top_k_neighbor_distances(
- data_this_cells, data_nearest_cells, k - 1
+ data_this_cells,
+ data_nearest_cells,
+ min(k, len(data_nearest_cells)),
+ metric=metric,
).mean()
- score.append((other_dist - self_dist) / max(self_dist, other_dist))
+ denominator = max(self_dist, other_dist)
+ if denominator <= np.finfo(np.float64).eps:
+ score.append(0.0)
+ else:
+ score.append(float((other_dist - self_dist) / denominator))
- return np.array(score)
+ return np.asarray(score)
-def integration_score(
- batch_labels: Sequence[np.ndarray], metric: str = "ari"
-) -> Optional[float]:
- """Calculate integration score between two sets of batch labels.
+def label_concordance_score(
+ label_sets: Sequence[np.ndarray],
+ metric: Literal["ari", "nmi"] = "ari",
+) -> float:
+ """Compare two label partitions using ARI or NMI.
Args:
- batch_labels: Sequence containing two arrays of batch labels to compare
- metric: Metric to use for comparison, one of:
- - 'ari': Adjusted Rand Index
- - 'nmi': Normalized Mutual Information
+ label_sets: Two arrays of labels to compare.
+ metric: Either ``"ari"`` or ``"nmi"``.
Returns:
- Optional[float]: Integration score between 0 and 1, or None if metric
- is not recognized
-
- Notes:
- Higher scores indicate better agreement between batch labels,
- suggesting more effective batch integration.
+ Label agreement. ARI ranges from -1 to 1 and NMI from 0 to 1.
"""
from sklearn.metrics import adjusted_rand_score, normalized_mutual_info_score
+ if len(label_sets) != 2:
+ raise ValueError("Exactly two label arrays are required")
+
+ first = np.asarray(label_sets[0])
+ second = np.asarray(label_sets[1])
+ if first.ndim != 1 or second.ndim != 1:
+ raise ValueError("Label arrays must be one-dimensional")
+ if len(first) != len(second):
+ raise ValueError("Label arrays must have matching lengths")
+ if pd.isna(first).any() or pd.isna(second).any():
+ raise ValueError("Label arrays must not contain missing values")
+
if metric == "ari":
- return adjusted_rand_score(batch_labels[0], batch_labels[1])
- elif metric == "nmi":
- return normalized_mutual_info_score(batch_labels[0], batch_labels[1])
- else:
- logger.error(
- f"Metric {metric} not recognized. Please choose from 'ari', 'nmi', 'calinski_harabasz', or 'davies_bouldin'."
- )
- return None
+ return float(adjusted_rand_score(first, second))
+ if metric == "nmi":
+ return float(normalized_mutual_info_score(first, second))
+ raise ValueError(f"Metric {metric!r} is not one of 'ari' or 'nmi'")
+
+
+def integration_score(
+ batch_labels: Sequence[np.ndarray],
+ metric: Literal["ari", "nmi"] = "ari",
+) -> float:
+ """Backward-compatible name for :func:`label_concordance_score`."""
+ return label_concordance_score(batch_labels, metric)
+
+
+def lisi_batch_mixing_score(
+ lisi_scores: np.ndarray,
+ batch_labels: Sequence[object] | np.ndarray,
+) -> float:
+ """Normalize mean batch LISI against the dataset's batch proportions.
+
+ Raw batch LISI depends on how many batches are present and how unevenly
+ cells are split between them, so its values are hard to compare across
+ datasets. This score rescales the mean batch LISI onto a fixed range by
+ dividing the observed neighborhood mixing by the mixing that perfectly
+ integrated data would reach. The reference point is the inverse Simpson
+ index of the global batch proportions, which is the LISI a neighborhood
+ would show if it mirrored the whole dataset.
+
+ Args:
+ lisi_scores: Per-cell batch LISI values, typically the array returned
+ by :func:`compute_lisi` for a batch label.
+ batch_labels: Batch assignment for each cell, aligned with
+ ``lisi_scores``.
+
+ Returns:
+ A value in ``[0, 1]``. Scores near 1 mean neighborhoods mix batches as
+ well as the global composition allows, and scores near 0 mean batches
+ stay separated.
+
+ Raises:
+ ValueError: If the inputs are misaligned, contain non-finite scores or
+ missing labels, or describe fewer than two batches.
+ """
+ scores = np.asarray(lisi_scores, dtype=np.float64)
+ labels = pd.Categorical(batch_labels)
+ if scores.ndim != 1 or len(scores) != len(labels):
+ raise ValueError("LISI scores and batch labels must be aligned vectors")
+ if not np.all(np.isfinite(scores)):
+ raise ValueError("LISI scores must contain finite values")
+ if np.any(labels.codes < 0):
+ raise ValueError("Batch labels must not contain missing values")
+ if len(labels.categories) < 2:
+ raise ValueError("Batch mixing requires at least two batches")
+
+ counts = np.bincount(labels.codes, minlength=len(labels.categories))
+ proportions = counts / counts.sum()
+ ideal_lisi = 1 / np.square(proportions).sum()
+ normalized = (scores.mean() - 1) / (ideal_lisi - 1)
+ return float(np.clip(normalized, 0, 1))
diff --git a/scarf/parallel.py b/scarf/parallel.py
new file mode 100644
index 00000000..eef93531
--- /dev/null
+++ b/scarf/parallel.py
@@ -0,0 +1,209 @@
+"""Generic shard-parallel processing over row-banded arrays.
+
+Both primitives process shards in order with a shallow read-ahead so the next
+band downloads while the current one is worked on; the worker budget is spent
+parallelising inner-chunk IO *within* a shard (BLAS stays single-threaded for
+reproducibility) rather than fanning out over many shards at once (which only
+inflates peak memory).
+
+- ``map_shards``: budget-aware ordered map over ``(start, end)`` row ranges,
+ used by ChunkedArray reductions/compute and the write pipeline. It sets Zarr
+ ``async.concurrency`` and per-shard BLAS threads from the plan and preserves
+ result order.
+- ``stream_shards``: an ordered, bounded read-ahead generator used by
+ sequential consumers (PCA / k-means / ANN fitting) that must see blocks in
+ order while the next one is fetched in the background.
+
+Threads are the default backend: the per-shard hot paths (numpy ufuncs, BLAS
+``dot``, scipy sparse, hnswlib) release the GIL, so a thread pool already uses
+multiple cores without process pickling or memory blow-up. A ``serial`` backend
+exists for tiny inputs and tests, and as the seam for a future process backend.
+
+A thread-local ``active`` flag guards against nested fan-out: a ``map_shards``
+invoked from inside another shard worker runs serially so total in-flight
+requests stay bounded.
+"""
+
+import threading
+from collections import deque
+from collections.abc import Callable, Iterable, Iterator
+from concurrent.futures import Future, ThreadPoolExecutor
+from contextlib import contextmanager, nullcontext
+from typing import Any, Literal
+
+from threadpoolctl import threadpool_limits
+
+from .storage.budget import ShardPlan, shard_parallelism
+
+__all__ = ["map_shards", "stream_shards", "in_shard_context"]
+
+type Backend = Literal["thread", "serial"]
+type RangeProduce = Callable[[int, int, int], Any]
+
+_shard_ctx = threading.local()
+
+
+def in_shard_context() -> bool:
+ """True when the caller is already running inside a shard worker."""
+ return bool(getattr(_shard_ctx, "active", False))
+
+
+@contextmanager
+def _shard_context() -> Iterator[None]:
+ prev = getattr(_shard_ctx, "active", False)
+ _shard_ctx.active = True
+ try:
+ yield
+ finally:
+ _shard_ctx.active = prev
+
+
+@contextmanager
+def _io_concurrency(io: int | None) -> Iterator[None]:
+ if io is None:
+ yield
+ return
+ import zarr
+
+ with zarr.config.set({"async.concurrency": max(1, int(io))}):
+ yield
+
+
+def _blas_limit(within: int | None) -> Any:
+ """Process-global BLAS/OpenMP cap for the whole pool (set once, not per task).
+
+ BLAS thread counts are process-wide, so entering ``threadpool_limits`` once
+ in the driving thread bounds every worker without the per-task overhead and
+ concurrent enter/exit races of wrapping each task.
+ """
+ if within is None or within < 1:
+ return nullcontext()
+ return threadpool_limits(limits=within)
+
+
+def _imap_ordered(
+ items: Iterable[Any],
+ fn: Callable[[Any], Any],
+ *,
+ workers: int,
+ within_block_threads: int | None,
+) -> Iterator[Any]:
+ """Ordered map with a bounded rolling window of ``workers`` in-flight tasks."""
+
+ def worker(item: Any) -> Any:
+ _shard_ctx.active = True
+ return fn(item)
+
+ iterator = iter(items)
+ with (
+ _blas_limit(within_block_threads),
+ ThreadPoolExecutor(max_workers=workers) as ex,
+ ):
+ pending: deque[Future[Any]] = deque()
+
+ def enqueue() -> bool:
+ try:
+ item = next(iterator)
+ except StopIteration:
+ return False
+ pending.append(ex.submit(worker, item))
+ return True
+
+ for _ in range(workers):
+ if not enqueue():
+ break
+ while pending:
+ result = pending.popleft().result()
+ enqueue()
+ yield result
+
+
+def _resolve_plan(
+ workers: int | None, n_shards: int | None, backend: Backend
+) -> ShardPlan:
+ if backend == "serial" or in_shard_context():
+ return ShardPlan(readAhead=1, ioConcurrency=1, withinBlockThreads=1)
+ return shard_parallelism(workers=workers, n_shards=n_shards)
+
+
+def _progress(it: Iterator[Any], msg: str | None, total: int | None) -> Iterator[Any]:
+ if msg is None:
+ yield from it
+ return
+ from .utils import tqdmbar
+
+ yield from tqdmbar(it, desc=msg, total=total)
+
+
+def stream_shards(
+ items: Iterable[Any],
+ fn: Callable[[Any], Any],
+ *,
+ workers: int,
+ within_block_threads: int | None = None,
+ io_concurrency: int | None = None,
+ msg: str | None = None,
+ total: int | None = None,
+ backend: Backend = "thread",
+) -> Iterator[Any]:
+ """Yield ``fn(item)`` in order with bounded read-ahead.
+
+ Meant for ordered streaming into sequential consumers. Falls back to a
+ serial pass when ``workers <= 1``, the backend is ``serial``, or already
+ inside a shard context. ``io_concurrency`` optionally caps Zarr
+ ``async.concurrency`` for the lifetime of the stream so that
+ ``workers * io_concurrency`` in-flight requests stays bounded on a remote
+ store; leave it ``None`` to be config-neutral.
+ """
+ workers = max(1, int(workers))
+ if backend == "serial" or workers <= 1 or in_shard_context():
+ with _io_concurrency(io_concurrency), _blas_limit(within_block_threads):
+ yield from _progress((fn(item) for item in items), msg, total)
+ return
+ with _io_concurrency(io_concurrency):
+ base = _imap_ordered(
+ items, fn, workers=workers, within_block_threads=within_block_threads
+ )
+ yield from _progress(base, msg, total)
+
+
+def map_shards(
+ ranges: list[tuple[int, int]],
+ produce: RangeProduce,
+ *,
+ workers: int | None = None,
+ msg: str | None = None,
+ backend: Backend = "thread",
+) -> list[Any]:
+ """Map ``produce(block_idx, start, end)`` over row ranges, preserving order.
+
+ Budget-aware: derives a :class:`ShardPlan` and, for the duration of the
+ run, sets Zarr ``async.concurrency`` to the plan's IO concurrency and bounds
+ per-shard BLAS threads. Returns results in ``ranges`` order so downstream
+ reductions keep their current float-summation order.
+ """
+ n = len(ranges)
+ if n == 0:
+ return []
+ plan = _resolve_plan(workers, n, backend)
+ indexed = list(enumerate(ranges))
+
+ def call(item: tuple[int, tuple[int, int]]) -> Any:
+ idx, (start, end) = item
+ return produce(idx, start, end)
+
+ if plan.readAhead <= 1:
+ with _io_concurrency(plan.ioConcurrency), _blas_limit(plan.withinBlockThreads):
+ return list(_progress((call(it) for it in indexed), msg, n))
+
+ with (
+ _shard_context(),
+ _io_concurrency(plan.ioConcurrency),
+ ):
+ stream = _imap_ordered(
+ indexed,
+ call,
+ workers=plan.readAhead,
+ within_block_threads=plan.withinBlockThreads,
+ )
+ return list(_progress(stream, msg, n))
diff --git a/scarf/plots.py b/scarf/plots.py
index 352d7799..6cf1d027 100644
--- a/scarf/plots.py
+++ b/scarf/plots.py
@@ -1,194 +1,56 @@
"""Contains the code for plotting in Scarf."""
-from typing import Tuple, Optional, Union, List
+import importlib
+from collections.abc import Iterator
+from typing import Any
-import matplotlib as mpl
-import matplotlib.pyplot as plt
import numpy as np
+import numpy.typing as npt
import pandas as pd
-import seaborn as sns
-from cmocean import cm
+from .plotting._style import CUSTOM_PALETTES
from .utils import logger
-plt.rcParams["svg.fonttype"] = "none"
-
-
-# These palettes were lifted from scanpy.plotting.palettes
-custom_palettes = {
- 10: [
- "#1f77b4",
- "#ff7f0e",
- "#279e68",
- "#d62728",
- "#aa40fc",
- "#8c564b",
- "#e377c2",
- "#7f7f7f",
- "#b5bd61",
- "#17becf",
- ],
- 20: [
- "#1f77b4",
- "#aec7e8",
- "#ff7f0e",
- "#ffbb78",
- "#2ca02c",
- "#98df8a",
- "#d62728",
- "#ff9896",
- "#9467bd",
- "#c5b0d5",
- "#8c564b",
- "#c49c94",
- "#e377c2",
- "#f7b6d2",
- "#7f7f7f",
- "#c7c7c7",
- "#bcbd22",
- "#dbdb8d",
- "#17becf",
- "#9edae5",
- ],
- 28: [
- "#023fa5",
- "#7d87b9",
- "#bec1d4",
- "#d6bcc0",
- "#bb7784",
- "#8e063b",
- "#4a6fe3",
- "#8595e1",
- "#b5bbe3",
- "#e6afb9",
- "#e07b91",
- "#d33f6a",
- "#11c638",
- "#8dd593",
- "#c6dec7",
- "#ead3c6",
- "#f0b98d",
- "#ef9708",
- "#0fcfc0",
- "#9cded6",
- "#d5eae7",
- "#f3e1eb",
- "#f6c4e1",
- "#f79cd4",
- "#7f7f7f",
- "#c7c7c7",
- "#1CE6FF",
- "#336600",
- ],
- 102: [
- "#FFFF00",
- "#1CE6FF",
- "#FF34FF",
- "#FF4A46",
- "#008941",
- "#006FA6",
- "#A30059",
- "#FFDBE5",
- "#7A4900",
- "#0000A6",
- "#63FFAC",
- "#B79762",
- "#004D43",
- "#8FB0FF",
- "#997D87",
- "#5A0007",
- "#809693",
- "#6A3A4C",
- "#1B4400",
- "#4FC601",
- "#3B5DFF",
- "#4A3B53",
- "#FF2F80",
- "#61615A",
- "#BA0900",
- "#6B7900",
- "#00C2A0",
- "#FFAA92",
- "#FF90C9",
- "#B903AA",
- "#D16100",
- "#DDEFFF",
- "#000035",
- "#7B4F4B",
- "#A1C299",
- "#300018",
- "#0AA6D8",
- "#013349",
- "#00846F",
- "#372101",
- "#FFB500",
- "#C2FFED",
- "#A079BF",
- "#CC0744",
- "#C0B9B2",
- "#C2FF99",
- "#001E09",
- "#00489C",
- "#6F0062",
- "#0CBD66",
- "#EEC3FF",
- "#456D75",
- "#B77B68",
- "#7A87A1",
- "#788D66",
- "#885578",
- "#FAD09F",
- "#FF8A9A",
- "#D157A0",
- "#BEC459",
- "#456648",
- "#0086ED",
- "#886F4C",
- "#34362D",
- "#B4A8BD",
- "#00A6AA",
- "#452C2C",
- "#636375",
- "#A3C8C9",
- "#FF913F",
- "#938A81",
- "#575329",
- "#00FECF",
- "#B05B6F",
- "#8CD0FF",
- "#3B9700",
- "#04F757",
- "#C8A1A1",
- "#1E6E00",
- "#7900D7",
- "#A77500",
- "#6367A9",
- "#A05837",
- "#6B002C",
- "#772600",
- "#D790FF",
- "#9B9700",
- "#549E79",
- "#FFF69F",
- "#201625",
- "#72418F",
- "#BC23FF",
- "#99ADC0",
- "#3A2465",
- "#922329",
- "#5B4534",
- "#FDE8DC",
- "#404E55",
- "#0089A3",
- "#CB7E98",
- "#A4E804",
- "#324E72",
- ],
-}
-
-
-def clean_axis(ax, ts=11, ga=0.4):
- """Cleans a given matplotlib axis."""
+
+class _LazyPlotDependency:
+ def __init__(self, module_name: str):
+ self.module_name = module_name
+ self.module: Any | None = None
+
+ def _load(self) -> Any:
+ if self.module is None:
+ try:
+ self.module = importlib.import_module(self.module_name)
+ except ModuleNotFoundError as exc:
+ package = self.module_name.split(".", 1)[0]
+ raise ImportError(
+ f"{package} is required for plotting. Install scarf[extra]."
+ ) from exc
+ return self.module
+
+ def __getattr__(self, name: str) -> Any:
+ return getattr(self._load(), name)
+
+
+mpl = _LazyPlotDependency("matplotlib")
+plt = _LazyPlotDependency("matplotlib.pyplot")
+sns = _LazyPlotDependency("seaborn")
+
+# Back-compat alias; do not mutate process-wide rcParams on import.
+custom_palettes = CUSTOM_PALETTES
+
+
+def clean_axis(ax: Any, ts: int = 11, ga: float = 0.4) -> bool:
+ """Clean matplotlib axis spines and add a light grid.
+
+ Args:
+ ax: Matplotlib axis.
+ ts: Tick label font size.
+ ga: Grid line alpha.
+
+ Returns:
+ True
+ """
ax.xaxis.set_tick_params(labelsize=ts)
ax.yaxis.set_tick_params(labelsize=ts)
for i in ["top", "bottom", "left", "right"]:
@@ -199,8 +61,12 @@ def clean_axis(ax, ts=11, ga=0.4):
return True
-def plot_graph_qc(g):
- # TODO: add docstring description. Is this for qc of a graph, or for plotting a qc plot of a graph?
+def plot_graph_qc(g: Any) -> None:
+ """Plot KNN graph QC: node degree distribution and edge weight histogram.
+
+ Args:
+ g: Sparse adjacency matrix (CSR or COO) for the KNN graph.
+ """
_, axis = plt.subplots(1, 2, figsize=(12, 4))
ax = axis[0]
x = np.array((g != 0).sum(axis=0))[0]
@@ -231,23 +97,38 @@ def plot_qc(
data: pd.DataFrame,
color: str = "steelblue",
cmap: str = "tab20",
- fig_size: Optional[Tuple] = None,
+ fig_size: tuple | None = None,
label_size: float = 10.0,
title_size: float = 10,
- sup_title: Optional[str] = None,
+ sup_title: str | None = None,
sup_title_size: float = 12,
scatter_size: float = 1.0,
max_points: int = 10000,
show_on_single_row: bool = True,
show_fig: bool = True,
-):
- # TODO: add docstring description. Is this for qc of a plot, or for plotting a qc plot?
+) -> Any | None:
+ """Plot per-metric QC violin plots grouped by ``groups`` column.
+
+ Args:
+ data: DataFrame with a ``groups`` column and numeric metric columns.
+ color: Violin fill when a single group is shown.
+ cmap: Colormap when multiple groups are shown.
+ fig_size: Figure size tuple (auto if None).
+ label_size: Axis label font size.
+ title_size: Subplot title font size.
+ sup_title: Figure suptitle.
+ sup_title_size: Suptitle font size.
+ scatter_size: Unused (kept for API compatibility).
+ max_points: Unused (kept for API compatibility).
+ show_on_single_row: Lay out metrics in one row vs one column.
+ show_fig: Call ``plt.show()`` when True; otherwise return the figure.
+ """
n_plots = data.shape[1] - 1
n_groups = data["groups"].nunique()
if n_groups > 5 and show_on_single_row is True:
logger.info(
- f"Too many groups in the plot. If you think that plot is too wide then consider turning "
- f"`show_on_single_row` parameter to False"
+ "Too many groups in the plot. If you think that plot is too wide then consider turning "
+ "`show_on_single_row` parameter to False"
)
if show_on_single_row is True:
n_rows = 1
@@ -260,17 +141,16 @@ def plot_qc(
fig_height = 1 + 2.5 * n_rows
fig_size = (fig_width, fig_height)
fig = plt.figure(figsize=fig_size)
- grouped = data.groupby("groups")
- for i in range(n_plots):
- if data.columns[i] == "groups":
- continue
- vals = {"g": [], "v": []}
+ grouped = data.groupby("groups", observed=False)
+ metric_cols = [c for c in data.columns if c != "groups"]
+ for plot_i, metric_col in enumerate(metric_cols):
+ vals_raw: dict[str, list[Any]] = {"g": [], "v": []}
for j in sorted(data["groups"].unique()):
- val = grouped.get_group(j)[data.columns[i]].values
- vals["g"].extend([j for _ in range(len(val))])
- vals["v"].extend(list(val))
- vals = pd.DataFrame(vals)
- ax = fig.add_subplot(n_rows, n_cols, i + 1)
+ val = grouped.get_group(j)[metric_col].values
+ vals_raw["g"].extend([j for _ in range(len(val))])
+ vals_raw["v"].extend(list(val))
+ vals = pd.DataFrame(vals_raw)
+ ax = fig.add_subplot(n_rows, n_cols, plot_i + 1)
if n_groups == 1:
sns.violinplot(
y="v",
@@ -287,6 +167,7 @@ def plot_qc(
sns.violinplot(
y="v",
x="g",
+ hue="g",
data=vals,
linewidth=1,
orient="v",
@@ -294,6 +175,7 @@ def plot_qc(
inner=None,
cut=0,
palette=cmap,
+ legend=False,
)
if len(vals) > max_points:
sns.stripplot(
@@ -319,7 +201,7 @@ def plot_qc(
color="k",
alpha=0.4,
)
- ax.set_ylabel(data.columns[i], fontsize=label_size)
+ ax.set_ylabel(metric_col, fontsize=label_size)
ax.set_xlabel("")
if n_groups == 1:
ax.set_xticks([])
@@ -331,12 +213,13 @@ def plot_qc(
# clean_axis(ax)
ax.figure.patch.set_alpha(0)
ax.patch.set_alpha(0)
- fig.suptitle(sup_title, fontsize=sup_title_size)
+ if sup_title is not None:
+ fig.suptitle(sup_title, fontsize=sup_title_size)
plt.tight_layout()
if show_fig:
plt.show()
- else:
- return fig
+ return None
+ return fig
def plot_mean_var(
@@ -345,11 +228,22 @@ def plot_mean_var(
n_cells: np.ndarray,
hvg: np.ndarray,
ax_label_fs: float = 12,
- fig_size: Tuple[float, float] = (4.5, 4.0),
- ss: Tuple[float, float] = (3, 30),
- cmaps: Tuple[str, str] = ("winter", "magma_r"),
-):
- """Shows a mean-variance plot."""
+ fig_size: tuple[float, float] = (4.5, 4.0),
+ ss: tuple[float, float] = (3, 30),
+ cmaps: tuple[str, str] = ("winter", "magma_r"),
+) -> None:
+ """Show a mean-variance scatter plot with HVGs highlighted.
+
+ Args:
+ nzm: Non-zero mean expression per feature.
+ fv: Variance (or corrected variance) per feature.
+ n_cells: Number of cells expressing each feature (for coloring).
+ hvg: Boolean mask of highly variable features.
+ ax_label_fs: Axis label font size.
+ fig_size: Figure size.
+ ss: Scatter sizes for non-HVG and HVG points.
+ cmaps: Colormaps for non-HVG and HVG points.
+ """
_, ax = plt.subplots(1, 1, figsize=fig_size)
nzm = np.log2(nzm)
fv = np.log2(fv)
@@ -371,10 +265,20 @@ def plot_mean_var(
plt.show()
-def plot_elbow(var_exp, figsize: Tuple[float, float] = (None, 2)):
- from kneed import KneeLocator
+def plot_elbow(
+ var_exp: npt.NDArray[Any] | list[float],
+ figsize: tuple[float | None, float] = (None, 2),
+) -> None:
+ """Plot PCA variance explained with an automatic elbow marker.
+
+ Args:
+ var_exp: Percent variance explained per component.
+ figsize: Figure size; width auto-scales with component count if None.
+ """
+ from .plotting._deps import require_kneed
x = range(len(var_exp))
+ KneeLocator = require_kneed()
kneedle = KneeLocator(x, var_exp, S=1.0, curve="convex", direction="decreasing")
if figsize[0] is None:
figsize = (0.25 * len(var_exp), figsize[1])
@@ -391,18 +295,31 @@ def plot_elbow(var_exp, figsize: Tuple[float, float] = (None, 2)):
def plot_heatmap(
- cdf,
+ cdf: pd.DataFrame,
fontsize: float = 10,
width_factor: float = 0.03,
height_factor: float = 0.02,
- cmap=cm.matter_r,
- savename: str = None,
+ cmap: Any = "magma_r",
+ savename: str | None = None,
save_dpi: int = 300,
- figsize=None,
+ figsize: tuple[float, float] | None = None,
show_fig: bool = True,
- **heatmap_kwargs,
-):
- """Shows a heatmap plot."""
+ **heatmap_kwargs: Any,
+) -> Any:
+ """Show a clustered heatmap of the input DataFrame.
+
+ Args:
+ cdf: Data to cluster and plot.
+ fontsize: Base font size for auto figsize.
+ width_factor: Width scaling per column.
+ height_factor: Height scaling per row.
+ cmap: Colormap name or object.
+ savename: If set, save figure to this path.
+ save_dpi: DPI for saved figure.
+ figsize: Explicit figure size (auto if None).
+ show_fig: Show interactively when True.
+ **heatmap_kwargs: Extra kwargs passed to ``sns.clustermap``.
+ """
if figsize is None:
figsize = (
cdf.shape[1] * fontsize * width_factor,
@@ -430,54 +347,64 @@ def plot_heatmap(
plt.savefig(savename, dpi=save_dpi)
if show_fig:
plt.show()
- else:
- return cgx
+ return None
+ return cgx
def _scatter_fix_type(v: pd.Series, ints_as_cats: bool) -> pd.Series:
vt = v.dtype
if v.nunique() == 1:
return pd.Series(np.ones(len(v)), index=v.index).astype(np.float64)
- if vt in [np.bool_]:
+ if vt in [bool, np.bool]:
# converting first to int to handle bool
- return v.astype(np.int_).astype("category")
- if vt in [str, object] or vt.name == "category":
+ return v.astype(int).astype("category")
+ if (
+ vt in [str, object]
+ or vt.name in ("category", "string")
+ or pd.api.types.is_string_dtype(v)
+ ):
return v.astype("category")
elif np.issubdtype(vt.type, np.integer) and ints_as_cats:
if v.nunique() > 100:
logger.warning("Too many categories. set force_ints_as_cats to false")
- return v.astype(np.int_).astype("category")
+ return v.astype(int).astype("category")
else:
return v.astype(np.float64)
-def _scatter_fix_mask(v: pd.Series, mask_vals: list, mask_name: str) -> pd.Series:
- if mask_vals is None:
- mask_vals = []
- mask_vals += [np.nan]
+def _scatter_fix_mask(
+ v: pd.Series, mask_vals: list[Any] | None, mask_name: str
+) -> pd.Series:
+ # Copy so caller-provided lists are not mutated.
+ mask_list: list[Any] = list(mask_vals) if mask_vals is not None else []
+ mask_list = mask_list + [np.nan]
iscat = False
if v.dtype.name == "category":
iscat = True
v = v.astype(object)
# There is a bug in pandas which causes failure above 1M rows
# v[v.isin(mask_vals)] = mask_name
- v[np.isin(v, mask_vals)] = mask_name
+ v[np.isin(v, mask_list)] = mask_name
if iscat:
v = v.astype("category")
return v
def _scatter_make_colors(
- v: pd.Series, cmap, color_key: Optional[dict], mask_color: str, mask_name: str
-):
- from matplotlib.pyplot import get_cmap
+ v: pd.Series,
+ cmap: Any,
+ color_key: dict[Any, Any] | None,
+ mask_color: str,
+ mask_name: str,
+) -> tuple[Any | None, dict[Any, Any] | None]:
+ from matplotlib.pyplot import get_cmap # type: ignore[import-not-found]
na_idx = v == mask_name
uv = v[~na_idx].unique()
if v.dtype.name != "category":
if cmap is None:
- return cm.deep, None
+ return get_cmap("viridis"), None
else:
return get_cmap(cmap), None
else:
@@ -485,6 +412,8 @@ def _scatter_make_colors(
cmap = "custom"
if color_key is not None:
+ # Copy so caller-provided dictionaries are not mutated.
+ color_key = dict(color_key)
for i in uv:
if i not in color_key:
raise KeyError(f"ERROR: key {i} missing in `color_key`")
@@ -512,7 +441,7 @@ def _scatter_make_colors(
return None, color_key
-def _scatter_cleanup(ax, sw: float, sc: str, ds: tuple) -> None:
+def _scatter_cleanup(ax: Any, sw: float, sc: str, ds: tuple[str, ...]) -> None:
for i in ["bottom", "left", "top", "right"]:
spine = ax.spines[i]
if i in ds:
@@ -527,7 +456,7 @@ def _scatter_cleanup(ax, sw: float, sc: str, ds: tuple) -> None:
return None
-def _scatter_label_axis(df, ax, fs: float, fo: float):
+def _scatter_label_axis(df: pd.DataFrame, ax: Any, fs: float, fo: float) -> None:
x, y = df.columns[:2]
ax.set_xlabel(x, fontsize=fs)
ax.set_ylabel(y, fontsize=fs)
@@ -541,14 +470,14 @@ def _scatter_label_axis(df, ax, fs: float, fo: float):
def _scatter_legends(
- df,
- ax,
- cmap,
- ck,
+ df: pd.DataFrame,
+ ax: Any,
+ cmap: Any,
+ ck: dict[Any, Any] | None,
ondata: bool,
onside: bool,
fontsize: float,
- title: str,
+ title: str | None,
title_fontsize: float,
hide_title: bool,
n_per_col: int,
@@ -578,8 +507,8 @@ def _scatter_legends(
Returns:
"""
- from matplotlib.colors import Normalize
- from matplotlib.colorbar import ColorbarBase, make_axes_gridspec
+ from matplotlib.colors import Normalize # type: ignore[import-not-found]
+ from matplotlib.colorbar import ColorbarBase, make_axes_gridspec # type: ignore[import-not-found]
x, y, vc = df.columns[:3]
v = df[vc]
@@ -594,7 +523,7 @@ def _scatter_legends(
else:
ax.title.set_text(vc)
ax.title.set_fontsize(title_fontsize)
- centers = df[[x, y, vc]].groupby(vc).median().T
+ centers = df[[x, y, vc]].groupby(vc, observed=False).median().T
for i in centers:
if ondata:
ax.text(
@@ -606,6 +535,7 @@ def _scatter_legends(
va="center",
)
if onside:
+ assert ck is not None
ax.scatter(
[float(centers[i][x])],
[float(centers[i][y])],
@@ -629,7 +559,16 @@ def _scatter_legends(
)
cax.set_axis_off()
else:
- norm = Normalize(vmin=v.min(), vmax=v.max())
+ numeric = pd.to_numeric(v, errors="coerce")
+ finite = numeric[np.isfinite(numeric.to_numpy(dtype=np.float64))]
+ if len(finite):
+ vmin = float(finite.min())
+ vmax = float(finite.max())
+ if vmax <= vmin:
+ vmax = vmin + 1.0
+ else:
+ vmin, vmax = 0.0, 1.0
+ norm = Normalize(vmin=vmin, vmax=vmax)
cb = ColorbarBase(cax, cmap=cmap, norm=norm, orientation="horizontal")
if hide_title is False:
if title is not None:
@@ -638,11 +577,20 @@ def _scatter_legends(
cb.set_label(vc, fontsize=title_fontsize)
cb.ax.xaxis.set_label_position("bottom")
cb.ax.xaxis.set_ticks_position("top")
- cb.outline.set_visible(False)
+ outline = cb.ax.spines.get("outline")
+ if outline is not None:
+ outline.set_visible(False)
return None
-def _make_grid(width, height, w_pad, h_pad, n_panels, n_columns):
+def _make_grid(
+ width: float,
+ height: float,
+ w_pad: float | None,
+ h_pad: float | None,
+ n_panels: int,
+ n_columns: int,
+) -> tuple[Any, npt.NDArray[Any]]:
n_columns = np.minimum(n_panels, n_columns)
n_rows = np.ceil(n_panels / n_columns).astype(int)
if w_pad is None and h_pad is None:
@@ -666,53 +614,70 @@ def _make_grid(width, height, w_pad, h_pad, n_panels, n_columns):
return fig, axes
-def _create_axes(dfs, in_ax, width, height, w_pad, h_pad, n_columns):
+def _create_axes(
+ dfs: list[pd.DataFrame],
+ in_ax: npt.NDArray[Any] | Any | None,
+ width: float,
+ height: float,
+ w_pad: float | None,
+ h_pad: float | None,
+ n_columns: int,
+) -> npt.NDArray[Any]:
if len(dfs) > 1:
if in_ax is not None:
logger.warning(
- f"'in_ax' will not be used as multiple attributes will be plotted. Using internal grid"
- f"layout"
+ "'in_ax' will not be used as multiple attributes will be plotted. Using internal grid"
+ "layout"
)
_, axs = _make_grid(width, height, w_pad, h_pad, len(dfs), n_columns)
else:
if in_ax is None:
_, axs = plt.subplots(1, 1, figsize=(width, height), squeeze=False)
+ elif isinstance(in_ax, np.ndarray):
+ axs = in_ax if in_ax.ndim == 2 else in_ax.reshape(1, -1)
else:
- axs = in_ax
+ # Bare Axes from the caller
+ axs = np.array([[in_ax]], dtype=object)
return axs
-def _iter_dataframes(dfs, mask_values, mask_name, force_ints_as_cats):
+def _iter_dataframes(
+ dfs: list[pd.DataFrame],
+ mask_values: list[Any] | None,
+ mask_name: str,
+ force_ints_as_cats: bool,
+) -> Iterator[tuple[int, pd.DataFrame]]:
for n, df in enumerate(dfs):
+ panel = df.copy()
vc = df.columns[2]
v = _scatter_fix_mask(df[vc].copy(), mask_values, mask_name)
- df[vc] = _scatter_fix_type(v, force_ints_as_cats)
- yield n, df
+ panel[vc] = _scatter_fix_type(v, force_ints_as_cats)
+ yield n, panel
-def _handle_titles_type(titles, n_df):
+def _handle_titles_type(titles: str | list[str] | None, n_df: int) -> list[str] | None:
if titles is not None:
if n_df > 1:
- if len(titles) != n_df or type(titles) != list:
+ if len(titles) != n_df or not isinstance(titles, list):
logger.warning(
"Number of titles is not same as the the number of titles. Provided titles cannot be used"
)
titles = None
else:
- if type(titles) == str:
+ if isinstance(titles, str):
titles = [titles]
return titles
def plot_scatter(
- dfs,
- in_ax=None,
+ dfs: list[pd.DataFrame],
+ in_ax: npt.NDArray[Any] | Any | None = None,
width: float = 6,
height: float = 6,
default_color: str = "steelblue",
- color_map=None,
- color_key: dict = None,
- mask_values: list = None,
+ color_map: Any | None = None,
+ color_key: dict[Any, Any] | None = None,
+ mask_values: list[Any] | None = None,
mask_name: str = "NA",
mask_color: str = "k",
point_size: float = 10,
@@ -720,37 +685,72 @@ def plot_scatter(
frame_offset: float = 0.05,
spine_width: float = 0.5,
spine_color: str = "k",
- displayed_sides: tuple = ("bottom", "left"),
+ displayed_sides: tuple[str, ...] = ("bottom", "left"),
legend_ondata: bool = True,
legend_onside: bool = True,
legend_size: float = 12,
legends_per_col: int = 20,
- titles: str = None,
+ titles: str | list[str] | None = None,
title_size: int = 12,
hide_title: bool = False,
cbar_shrink: float = 0.6,
marker_scale: float = 70,
lspacing: float = 0.1,
cspacing: float = 1,
- savename: str = None,
+ savename: str | None = None,
dpi: int = 300,
force_ints_as_cats: bool = True,
n_columns: int = 4,
w_pad: float = 1,
h_pad: float = 1,
show_fig: bool = True,
- scatter_kwargs: dict = None,
-):
- """Shows scatter plots.
+ scatter_kwargs: dict[str, Any] | None = None,
+) -> npt.NDArray[Any] | None:
+ """Show one or more 2D scatter plots from annotation DataFrames.
+
+ Each DataFrame must contain x, y, and value columns. When multiple
+ DataFrames are provided, plots are arranged in a grid.
- If more then one dataframe is provided it will place the
- scatterplots in a grid.
+ Args:
+ dfs: List of DataFrames with columns [x, y, value].
+ in_ax: Existing axes array to draw into (optional).
+ width: Subplot width in inches.
+ height: Subplot height in inches.
+ default_color: Color for continuous values without a colormap.
+ color_map: Matplotlib colormap name or object.
+ color_key: Dict mapping category values to colors.
+ mask_values: Values to render with ``mask_color``.
+ mask_name: Label for masked values in legend.
+ mask_color: Color for masked points.
+ point_size: Scatter marker size.
+ ax_label_size: Axis label font size.
+ frame_offset: Axis limit padding fraction.
+ spine_width: Spine line width.
+ spine_color: Spine color.
+ displayed_sides: Tuple of spines to show.
+ legend_ondata: Draw category labels on data points.
+ legend_onside: Draw legend table beside the plot.
+ legend_size: Legend font size.
+ legends_per_col: Max legend entries per column.
+ titles: Subplot title or list of titles.
+ title_size: Title font size.
+ hide_title: Suppress titles when True.
+ cbar_shrink: Colorbar shrink factor.
+ marker_scale: Scale for on-data legend markers.
+ lspacing: Legend label spacing.
+ cspacing: Legend column spacing.
+ savename: Save path (optional).
+ dpi: Save DPI.
+ force_ints_as_cats: Treat integer columns as categorical.
+ n_columns: Grid columns when plotting multiple panels.
+ w_pad: Width padding between subplots.
+ h_pad: Height padding between subplots.
+ show_fig: Show interactively when True.
+ scatter_kwargs: Extra kwargs passed to ``ax.scatter`` (not ``c`` or ``s``).
"""
- from matplotlib.colors import to_hex
- def _handle_scatter_kwargs(sk):
- if sk is None:
- sk = {}
+ def _handle_scatter_kwargs(sk: dict[str, Any] | None) -> dict[str, Any]:
+ sk = {} if sk is None else dict(sk)
if "c" in sk:
logger.warning("scatter_kwarg value `c` will be ignored")
del sk["c"]
@@ -772,14 +772,30 @@ def _handle_scatter_kwargs(sk):
v, color_map, color_key, mask_color, mask_name
)
if v.dtype.name == "category":
+ assert col_key is not None
df["c"] = [col_key[x] for x in v]
else:
- if v.nunique() == 1:
+ if v.nunique(dropna=False) == 1:
df["c"] = [default_color for _ in v]
else:
- v = v.copy().fillna(0)
- mmv = (v - v.min()) / (v.max() - v.min())
- df["c"] = [to_hex(col_map(x)) for x in mmv]
+ from matplotlib.colors import to_hex # type: ignore[import-not-found]
+
+ v_num = pd.to_numeric(v, errors="coerce")
+ finite = np.isfinite(v_num.to_numpy(dtype=np.float64))
+ colors: list[str] = []
+ if finite.any():
+ vmin = float(v_num[finite].min())
+ vmax = float(v_num[finite].max())
+ span = vmax - vmin if vmax != vmin else 1.0
+ assert col_map is not None
+ for val, ok in zip(v_num.to_numpy(dtype=np.float64), finite):
+ if not ok:
+ colors.append(to_hex(mask_color))
+ else:
+ colors.append(to_hex(col_map((val - vmin) / span)))
+ else:
+ colors = [to_hex(mask_color) for _ in v]
+ df["c"] = colors
if "s" not in df:
df["s"] = [point_size for _ in df.index]
scatter_kwargs = _handle_scatter_kwargs(sk=scatter_kwargs)
@@ -817,62 +833,121 @@ def _handle_scatter_kwargs(sk):
)
if savename:
- plt.savefig(savename, dpi=dpi, bbox_inches="tight")
+ fig = axs.flat[0].figure
+ with mpl.rc_context({"svg.fonttype": "none", "pdf.fonttype": 42}):
+ fig.savefig(savename, dpi=dpi, bbox_inches="tight")
if show_fig:
- plt.show()
- else:
- return axs
+ axs.flat[0].figure.show()
+ return None
+ return axs
def shade_scatter(
- dfs,
- in_ax=None,
+ dfs: list[pd.DataFrame],
+ in_ax: npt.NDArray[Any] | Any | None = None,
figsize: float = 6,
pixels: int = 1000,
spread_px: int = 1,
spread_threshold: float = 0.2,
min_alpha: int = 10,
- color_map=None,
- color_key: dict = None,
- mask_values: list = None,
+ color_map: Any | None = None,
+ color_key: dict[Any, Any] | None = None,
+ mask_values: list[Any] | None = None,
mask_name: str = "NA",
mask_color: str = "k",
ax_label_size: float = 12,
frame_offset: float = 0.05,
spine_width: float = 0.5,
spine_color: str = "k",
- displayed_sides: tuple = ("bottom", "left"),
+ displayed_sides: tuple[str, ...] = ("bottom", "left"),
legend_ondata: bool = True,
legend_onside: bool = True,
legend_size: float = 12,
legends_per_col: int = 20,
- titles: Union[str, List[str]] = None,
+ titles: str | list[str] | None = None,
title_size: int = 12,
hide_title: bool = False,
cbar_shrink: float = 0.6,
marker_scale: float = 70,
lspacing: float = 0.1,
cspacing: float = 1,
- savename: str = None,
+ savename: str | None = None,
dpi: int = 300,
force_ints_as_cats: bool = True,
n_columns: int = 4,
- w_pad: float = None,
- h_pad: float = None,
+ w_pad: float | None = None,
+ h_pad: float | None = None,
show_fig: bool = True,
-):
- """Shows shaded scatter plots.
+) -> npt.NDArray[Any] | None:
+ """Show datashader-density scatter plots for large cell embeddings.
- If more then one dataframe is provided it will place the
- scatterplots in a grid.
+ Args:
+ dfs: List of DataFrames with columns [x, y, value].
+ in_ax: Existing axes to draw into.
+ figsize: Subplot width and height in inches.
+ pixels: Canvas resolution for datashader aggregation.
+ spread_px: Pixel spread for categorical shading.
+ spread_threshold: Minimum fraction of pixels to apply spread.
+ min_alpha: Minimum alpha for rendered pixels.
+ color_map: Colormap for continuous values.
+ color_key: Dict mapping categories to colors.
+ mask_values: Values to treat as masked.
+ mask_name: Label for masked category.
+ mask_color: Color for masked values.
+ ax_label_size: Axis label font size.
+ frame_offset: Axis limit padding.
+ spine_width: Spine line width.
+ spine_color: Spine color.
+ displayed_sides: Visible spines.
+ legend_ondata: Draw labels on data.
+ legend_onside: Draw side legend table.
+ legend_size: Legend font size.
+ legends_per_col: Legend entries per column.
+ titles: Subplot title(s).
+ title_size: Title font size.
+ hide_title: Suppress titles.
+ cbar_shrink: Colorbar shrink factor.
+ marker_scale: On-data legend marker scale.
+ lspacing: Legend label spacing.
+ cspacing: Legend column spacing.
+ savename: Save path (optional).
+ dpi: Save DPI.
+ force_ints_as_cats: Treat integers as categorical.
+ n_columns: Grid columns for multiple panels.
+ w_pad: Width padding between subplots.
+ h_pad: Height padding between subplots.
+ show_fig: Show interactively when True.
"""
- import datashader as dsh
- from datashader.mpl_ext import dsshow
- import datashader.transfer_functions as tf
from functools import partial
+ from .plotting._deps import require_datashader
+
+ dsh, dsshow, tf = require_datashader()
titles = _handle_titles_type(titles, len(dfs))
axs = _create_axes(dfs, in_ax, figsize, figsize, w_pad, h_pad, n_columns)
+
+ # Shared continuous limits across panels so multi-panel shading is comparable.
+ # Prefer linear / percentile limits over per-panel eq_hist (Milestone C).
+ shared_vmin: float | None = None
+ shared_vmax: float | None = None
+ cont_vals: list[Any] = []
+ for n, df in _iter_dataframes(dfs, mask_values, mask_name, force_ints_as_cats):
+ v = df[df.columns[2]]
+ if v.dtype.name != "category" and v.nunique() > 1:
+ cont_vals.append(v.to_numpy(dtype=float))
+ if cont_vals:
+ import numpy as np
+
+ all_c = np.concatenate(cont_vals)
+ finite = np.isfinite(all_c)
+ if finite.any():
+ shared_vmin = float(np.nanpercentile(all_c[finite], 1))
+ shared_vmax = float(np.nanpercentile(all_c[finite], 99))
+ if shared_vmax <= shared_vmin:
+ shared_vmin = float(np.nanmin(all_c[finite]))
+ shared_vmax = float(np.nanmax(all_c[finite]))
+
+ # Rewind iterator by rebuilding from dfs
for n, df in _iter_dataframes(dfs, mask_values, mask_name, force_ints_as_cats):
dim1, dim2, vc = df.columns[:3]
v = df[vc]
@@ -881,18 +956,26 @@ def shade_scatter(
)
if v.dtype.name == "category":
agg = dsh.count_cat(vc)
+ how = "eq_hist"
+ limits: dict[str, float] = {}
else:
if v.nunique() == 1:
agg = dsh.count(vc)
+ how = "eq_hist"
+ limits = {}
else:
agg = dsh.mean(vc)
+ how = "linear"
+ limits = {}
+ if shared_vmin is not None and shared_vmax is not None:
+ limits = {"vmin": shared_vmin, "vmax": shared_vmax}
ax = axs[int(n / n_columns), n % n_columns]
- artist = dsshow(
+ dsshow(
df,
dsh.Point(dim1, dim2),
aggregator=agg,
- norm="eq_hist",
+ norm=how,
color_key=col_key,
cmap=col_map,
alpha_range=(min_alpha, 255),
@@ -905,6 +988,7 @@ def shade_scatter(
width_scale=1,
height_scale=1,
ax=ax,
+ **limits,
)
_scatter_label_axis(df, ax, ax_label_size, frame_offset)
@@ -932,14 +1016,23 @@ def shade_scatter(
)
if savename:
- plt.savefig(savename, dpi=dpi, bbox_inches="tight")
+ fig = axs.flat[0].figure
+ with mpl.rc_context({"svg.fonttype": "none", "pdf.fonttype": 42}):
+ fig.savefig(savename, dpi=dpi, bbox_inches="tight")
if show_fig:
- plt.show()
- else:
- return axs
+ axs.flat[0].figure.show()
+ return None
+ return axs
-def _draw_pie(ax, dist, colors, xpos, ypos, size):
+def _draw_pie(
+ ax: Any,
+ dist: npt.NDArray[Any],
+ colors: list[Any],
+ xpos: float,
+ ypos: float,
+ size: float,
+) -> None:
# https://stackoverflow.com/questions/56337732/how-to-plot-scatter-pie-chart-using-matplotlib
cumsum = np.cumsum(dist)
cumsum = cumsum / cumsum[-1]
@@ -953,8 +1046,13 @@ def _draw_pie(ax, dist, colors, xpos, ypos, size):
def hierarchy_pos(
- g, root=None, width=1.0, vert_gap=0.2, vert_loc=0, leaf_vs_root_factor=0.5
-):
+ g: Any,
+ root: Any | None = None,
+ width: float = 1.0,
+ vert_gap: float = 0.2,
+ vert_loc: float = 0,
+ leaf_vs_root_factor: float = 0.5,
+) -> dict[Any, tuple[float, float]]:
"""This function was lifted from here: https://github.com/springer-
math/Mathematics-of-Epidemics-on-
Networks/blob/80c8accbe0c6b7710c0a189df17529696ac31bf9/EoN/auxiliary.py.
@@ -1011,18 +1109,18 @@ def hierarchy_pos(
root = np.random.choice(list(g.nodes))
def _hierarchy_pos(
- g,
- root,
- leftmost,
- width,
- leafdx=0.2,
- vert_gap=0.2,
- vert_loc=0,
- xcenter=0.5,
- rootpos=None,
- leafpos=None,
- parent=None,
- ):
+ g: Any,
+ root: Any,
+ leftmost: float,
+ width: float,
+ leafdx: float = 0.2,
+ vert_gap: float = 0.2,
+ vert_loc: float = 0,
+ xcenter: float = 0.5,
+ rootpos: dict[Any, tuple[float, float]] | None = None,
+ leafpos: dict[Any, tuple[float, float]] | None = None,
+ parent: Any | None = None,
+ ) -> tuple[dict[Any, tuple[float, float]], dict[Any, tuple[float, float]], int]:
"""
see hierarchy_pos docstring for most arguments
pos: a dict saying where all nodes go if they have been assigned
@@ -1106,9 +1204,9 @@ def _hierarchy_pos(
def plot_cluster_hierarchy(
- sg,
- clusts,
- color_values=None,
+ sg: Any,
+ clusts: Any,
+ color_values: Any | None = None,
force_ints_as_cats: bool = True,
width: float = 2,
lvr_factor: float = 0.5,
@@ -1119,29 +1217,56 @@ def plot_cluster_hierarchy(
root_size: float = 100,
non_leaf_size: float = 10,
show_labels: bool = False,
- fontsize=10,
+ fontsize: float = 10,
root_color: str = "#C0C0C0",
non_leaf_color: str = "k",
- cmap: str = None,
- color_key: bool = None,
+ cmap: str | None = None,
+ color_key: dict[Any, Any] | None = None,
edgecolors: str = "k",
edgewidth: float = 1,
alpha: float = 0.7,
- figsize=(5, 5),
- ax=None,
+ figsize: tuple[float, float] = (5, 5),
+ ax: Any | None = None,
show_fig: bool = True,
- savename: str = None,
- save_dpi=300,
-):
- """Shows a plot showing cluster hierarchy.
+ savename: str | None = None,
+ save_dpi: int = 300,
+) -> Any | None:
+ """Plot a cluster hierarchy tree with colored leaf nodes.
+
+ Args:
+ sg: NetworkX graph of the cluster hierarchy.
+ clusts: Cluster id per leaf cell.
+ color_values: Values for leaf coloring (default: cluster ids).
+ force_ints_as_cats: Treat integer color values as categorical.
+ width: Horizontal layout width for hierarchy positioning.
+ lvr_factor: Blend between leaf-only and root-based layout (0-1).
+ vert_gap: Vertical gap between hierarchy levels.
+ min_node_size: Minimum node marker size.
+ node_size_multiplier: Scale factor for node sizes.
+ node_power: Exponent applied to node leaf counts for sizing.
+ root_size: Marker size for root node.
+ non_leaf_size: Marker size for internal nodes.
+ show_labels: Draw cluster labels on internal nodes.
+ fontsize: Label font size.
+ root_color: Root node color.
+ non_leaf_color: Internal node color.
+ cmap: Colormap for continuous color values.
+ color_key: Dict mapping categories to colors.
+ edgecolors: Node edge color.
+ edgewidth: Node edge width.
+ alpha: Node alpha.
+ figsize: Figure size when ``ax`` is None.
+ ax: Existing axis to draw into.
+ show_fig: Show interactively when True.
+ savename: Save path (optional).
+ save_dpi: Save DPI.
Returns:
- If requested (with parameter `show_fig`) a matplotlib Axes object containing the plot
- (which is the modified `ax` parameter if given).
+ Matplotlib Axes when ``show_fig`` is False.
"""
import networkx as nx
import math
- from matplotlib.colors import to_hex
+ from matplotlib.colors import to_hex # type: ignore[import-not-found]
if color_values is None:
color_values = pd.Series(clusts)
@@ -1150,7 +1275,7 @@ def plot_cluster_hierarchy(
color_values = pd.Series(color_values)
using_clust_for_colors = False
color_values = _scatter_fix_type(color_values, force_ints_as_cats)
- cmap, color_key = _scatter_make_colors(
+ colormap, color_key = _scatter_make_colors(
color_values, cmap, color_key, "k", "longdummyvaluesofh3489hfpiqehdcbla"
)
pos = hierarchy_pos(
@@ -1163,16 +1288,19 @@ def plot_cluster_hierarchy(
if color_key is None:
cluster_values = (
pd.DataFrame({"clusters": clusts, "v": color_values})
- .groupby("clusters")
+ .groupby("clusters", observed=False)
.mean()["v"]
)
mmv: pd.Series = (cluster_values - cluster_values.min()) / (
cluster_values.max() - cluster_values.min()
)
- color_key = {k: to_hex(cmap(v)) for k, v in mmv.to_dict().items()}
+ assert colormap is not None
+ color_key = {k: to_hex(colormap(v)) for k, v in mmv.to_dict().items()}
else:
cluster_values = None
+ assert color_key is not None
+
cs = pd.Series(clusts).value_counts()
cs = (node_size_multiplier * ((cs / cs.sum()) ** node_power)).to_dict()
nc, ns = [], []
@@ -1237,31 +1365,53 @@ def plot_cluster_hierarchy(
plt.savefig(savename, dpi=save_dpi)
if show_fig:
plt.show()
- else:
- return ax
+ return None
+ return ax
def plot_annotated_heatmap(
- df: np.array,
- xbar_values: np.ndarray,
- ybar_values: np.ndarray,
- display_row_labels: list = None,
- row_labels: list = None,
+ df: npt.NDArray[Any],
+ xbar_values: npt.NDArray[Any],
+ ybar_values: npt.NDArray[Any],
+ display_row_labels: list[str] | None = None,
+ row_labels: list[str] | None = None,
width: int = 5,
height: int = 10,
vmin: float = -2.0,
vmax: float = 2.0,
- heatmap_cmap: str = None,
- xbar_cmap: str = None,
- ybar_cmap: str = None,
+ heatmap_cmap: str | None = None,
+ xbar_cmap: Any | None = None,
+ ybar_cmap: str | None = None,
tick_fontsize: int = 10,
axis_fontsize: int = 12,
row_label_fontsize: int = 12,
- savename: str = None,
+ savename: str | None = None,
save_dpi: int = 300,
show_fig: bool = True,
-):
- import matplotlib.ticker as mticker
+) -> None:
+ """Plot a heatmap with pseudotime and cluster annotation bars.
+
+ Args:
+ df: 2D expression matrix (features x ordering bins).
+ xbar_values: Values for the bottom pseudotime bar.
+ ybar_values: Values for the right cluster color bar.
+ display_row_labels: Subset of row labels to show on the heatmap.
+ row_labels: Labels for all rows (default: index strings).
+ width: Figure width in inches.
+ height: Figure height in inches.
+ vmin: Heatmap color scale minimum.
+ vmax: Heatmap color scale maximum.
+ heatmap_cmap: Colormap for the main heatmap.
+ xbar_cmap: Colormap for the pseudotime bar.
+ ybar_cmap: Colormap for the cluster bar.
+ tick_fontsize: Colorbar tick font size.
+ axis_fontsize: Title font size.
+ row_label_fontsize: Row label font size.
+ savename: Save path (optional).
+ save_dpi: Save DPI.
+ show_fig: Show interactively when True.
+ """
+ import matplotlib.ticker as mticker # type: ignore[import-not-found]
if display_row_labels is None:
display_row_labels = []
@@ -1295,9 +1445,11 @@ def plot_annotated_heatmap(
)
if len(display_row_labels) > 0:
- row_labels = {x.lower(): n for n, x in enumerate(row_labels)}
- display_row_labels = [x for x in display_row_labels if x.lower() in row_labels]
- heatmap_ax.set_yticks([row_labels[x.lower()] for x in display_row_labels])
+ row_label_index = {x.lower(): n for n, x in enumerate(row_labels)}
+ display_row_labels = [
+ x for x in display_row_labels if x.lower() in row_label_index
+ ]
+ heatmap_ax.set_yticks([row_label_index[x.lower()] for x in display_row_labels])
heatmap_ax.set_yticklabels(display_row_labels, fontsize=row_label_fontsize)
heatmap_ax.set_title(f"{df.shape[0]} features", fontsize=axis_fontsize)
@@ -1324,7 +1476,7 @@ def plot_annotated_heatmap(
binned_ptime = [x.mean() for x in np.array_split(sorted(xbar_values), df.shape[1])]
if xbar_cmap is None:
- xbar_cmap = cm.deep
+ xbar_cmap = "viridis"
ptime_ax.imshow([binned_ptime], aspect="auto", cmap=xbar_cmap)
ptime_ax.set_xticks([])
ptime_ax.set_yticks([])
diff --git a/scarf/plotting/__init__.py b/scarf/plotting/__init__.py
new file mode 100644
index 00000000..23316c47
--- /dev/null
+++ b/scarf/plotting/__init__.py
@@ -0,0 +1,51 @@
+from ._contracts import (
+ CategoricalScale,
+ CellField,
+ ColorScale,
+ FeatureRef,
+ FeatureSummary,
+ NormalizationSpec,
+ PlotProvenance,
+ SizeScale,
+ StudyDesign,
+)
+from ._figure import LegendSpec, PlotResult, collect_legends, label_panels
+from ._style import THEMES, theme_context
+from .composition import composition
+from .diagnostics import elbow, graph_qc, highly_variable_features, qc
+from .distribution import distribution
+from .embedding import embedding
+from .embedding_raster import embedding_raster
+from .heatmaps import cluster_tree, marker_heatmap, pseudotime_heatmap
+from .summary import dotplot, matrixplot
+
+__all__ = [
+ "CategoricalScale",
+ "CellField",
+ "ColorScale",
+ "FeatureRef",
+ "FeatureSummary",
+ "LegendSpec",
+ "NormalizationSpec",
+ "PlotProvenance",
+ "PlotResult",
+ "SizeScale",
+ "StudyDesign",
+ "THEMES",
+ "cluster_tree",
+ "collect_legends",
+ "composition",
+ "distribution",
+ "dotplot",
+ "elbow",
+ "embedding",
+ "embedding_raster",
+ "graph_qc",
+ "highly_variable_features",
+ "label_panels",
+ "marker_heatmap",
+ "matrixplot",
+ "pseudotime_heatmap",
+ "qc",
+ "theme_context",
+]
diff --git a/scarf/plotting/_contracts.py b/scarf/plotting/_contracts.py
new file mode 100644
index 00000000..c1bb7d48
--- /dev/null
+++ b/scarf/plotting/_contracts.py
@@ -0,0 +1,180 @@
+"""Public contracts for scarf.plotting."""
+
+from dataclasses import dataclass, field
+from typing import Any, Literal
+
+import numpy as np
+import pandas as pd
+
+LookupBy = Literal["name", "id", "index"]
+FeatureReduction = Literal["mean", "sum"]
+CellFieldKind = Literal["auto", "categorical", "continuous"]
+NormSource = Literal["assay", "raw"]
+NormTransform = Literal["none", "log1p"]
+Standardize = Literal["none", "feature"]
+
+
+@dataclass(frozen=True, slots=True)
+class FeatureRef:
+ """Reference to one assay feature.
+
+ Parameters:
+ value: Feature name, id, or physical index (see ``by``).
+ assay: Assay name. Defaults to the store default assay when omitted.
+ by: How to look up ``value``: ``name``, ``id``, or ``index``.
+ label: Optional display label.
+ reduction: Required when multiple features match; ``mean`` or ``sum``.
+ """
+
+ value: str | int
+ assay: str | None = None
+ by: LookupBy = "name"
+ label: str | None = None
+ reduction: FeatureReduction | None = None
+
+ def __post_init__(self) -> None:
+ if self.by not in ("name", "id", "index"):
+ raise ValueError("by must be 'name', 'id', or 'index'")
+ if self.reduction not in (None, "mean", "sum"):
+ raise ValueError("reduction must be 'mean', 'sum', or None")
+
+
+@dataclass(frozen=True, slots=True)
+class CellField:
+ """Reference to a cell-metadata column."""
+
+ key: str
+ kind: CellFieldKind = "auto"
+ label: str | None = None
+
+ def __post_init__(self) -> None:
+ if self.kind not in ("auto", "categorical", "continuous"):
+ raise ValueError("kind must be 'auto', 'categorical', or 'continuous'")
+
+
+@dataclass(frozen=True, slots=True)
+class StudyDesign:
+ """How cells relate to biological samples and conditions.
+
+ Supports ``sample_by`` and optional ``condition_by``. Composition pairing
+ uses ``condition_by`` with ``subject_by`` or ``pair_by``. Technical-replicate
+ collapse is deferred.
+ """
+
+ sample_by: str
+ condition_by: str | None = None
+ subject_by: str | None = None
+ pair_by: str | None = None
+ technical_replicate_by: str | None = None
+ technical_replicate_reduction: Literal["sum", "mean"] | None = None
+
+ def __post_init__(self) -> None:
+ unsupported = [
+ name
+ for name, value in (
+ ("technical_replicate_by", self.technical_replicate_by),
+ (
+ "technical_replicate_reduction",
+ self.technical_replicate_reduction,
+ ),
+ )
+ if value is not None
+ ]
+ if unsupported:
+ raise NotImplementedError(
+ "These StudyDesign fields are not supported yet: "
+ + ", ".join(unsupported)
+ + ". Collapse technical replicates into sample_by first, "
+ "or omit them."
+ )
+
+
+@dataclass(frozen=True, slots=True)
+class NormalizationSpec:
+ source: NormSource = "assay"
+ transform: NormTransform = "none"
+
+ def __post_init__(self) -> None:
+ if self.source not in ("assay", "raw"):
+ raise ValueError("source must be 'assay' or 'raw'")
+ if self.transform not in ("none", "log1p"):
+ raise ValueError("transform must be 'none' or 'log1p'")
+
+
+@dataclass(frozen=True, slots=True)
+class ColorScale:
+ cmap: str | None = None
+ vmin: float | None = None
+ vmax: float | None = None
+ vcenter: float | None = None
+ quantiles: tuple[float, float] | None = None
+ missing_color: str = "#bdbdbd"
+ scope: Literal["feature", "panel", "shared"] = "feature"
+
+ def __post_init__(self) -> None:
+ if self.quantiles is not None:
+ low, high = self.quantiles
+ if not (0.0 <= low < high <= 1.0):
+ raise ValueError("quantiles must satisfy 0 <= low < high <= 1")
+ if self.vmin is not None and self.vmax is not None and self.vmax <= self.vmin:
+ raise ValueError("vmax must be greater than vmin")
+ if (
+ self.vcenter is not None
+ and self.vmin is not None
+ and self.vmax is not None
+ and not self.vmin < self.vcenter < self.vmax
+ ):
+ raise ValueError("vcenter must be strictly between vmin and vmax")
+ if self.scope not in ("feature", "panel", "shared"):
+ raise ValueError("scope must be 'feature', 'panel', or 'shared'")
+
+
+@dataclass(frozen=True, slots=True)
+class CategoricalScale:
+ order: tuple[Any, ...] | None = None
+ palette: dict[Any, str] | None = None
+ missing_color: str = "#bdbdbd"
+ missing_label: str = "NA"
+
+
+@dataclass(frozen=True, slots=True)
+class SizeScale:
+ """Maps numeric values to marker area. Detection fraction uses domain [0, 1]."""
+
+ vmin: float = 0.0
+ vmax: float = 1.0
+ size_min: float = 10.0
+ size_max: float = 200.0
+
+ def __post_init__(self) -> None:
+ if self.size_min < 0 or self.size_max < self.size_min:
+ raise ValueError("size range must satisfy 0 <= size_min <= size_max")
+
+ def areas(self, values: np.ndarray) -> np.ndarray:
+ v = np.asarray(values, dtype=np.float64)
+ span = self.vmax - self.vmin
+ if span <= 0:
+ return np.full(v.shape, self.size_min, dtype=np.float64)
+ t = np.clip((v - self.vmin) / span, 0.0, 1.0)
+ return self.size_min + t * (self.size_max - self.size_min)
+
+
+@dataclass(frozen=True, slots=True)
+class PlotProvenance:
+ scarf_version: str
+ schema_version: str = "1"
+ assay: str | None = None
+ cell_key: str | None = None
+ n_cells: int = 0
+ n_samples: int | None = None
+ renderer: str = "matplotlib"
+ notes: tuple[str, ...] = ()
+ extras: dict[str, Any] = field(default_factory=dict)
+
+
+@dataclass(slots=True)
+class FeatureSummary:
+ """Bounded tables behind summary plots."""
+
+ aggregate: pd.DataFrame
+ per_sample: pd.DataFrame | None = None
diff --git a/scarf/plotting/_data.py b/scarf/plotting/_data.py
new file mode 100644
index 00000000..66aa1416
--- /dev/null
+++ b/scarf/plotting/_data.py
@@ -0,0 +1,346 @@
+"""Feature resolution and bounded group reducers."""
+
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from typing import Any
+
+import numpy as np
+import pandas as pd
+
+from ._contracts import (
+ FeatureRef,
+ FeatureReduction,
+ LookupBy,
+ NormalizationSpec,
+ Standardize,
+ StudyDesign,
+)
+
+
+@dataclass(frozen=True, slots=True)
+class ResolvedFeature:
+ assay: str
+ by: LookupBy
+ indices: tuple[int, ...]
+ ids: tuple[str, ...]
+ names: tuple[str, ...]
+ label: str
+ reduction: FeatureReduction | None
+ raw: FeatureRef | str
+
+ def scale_key(
+ self,
+ normalization: NormalizationSpec,
+ standardize: Standardize = "none",
+ ) -> tuple[Any, ...]:
+ return (
+ self.assay,
+ self.by,
+ self.ids,
+ self.reduction,
+ normalization.source,
+ normalization.transform,
+ standardize,
+ )
+
+
+def coerce_feature_list(
+ features: Sequence[str | FeatureRef] | Mapping[str, Sequence[str | FeatureRef]],
+) -> list[tuple[str | None, str | FeatureRef]]:
+ """Return (group_label, feature) pairs preserving order."""
+ if isinstance(features, Mapping):
+ out: list[tuple[str | None, str | FeatureRef]] = []
+ for group, items in features.items():
+ for item in items:
+ out.append((str(group), item))
+ return out
+ return [(None, item) for item in features]
+
+
+def assert_assay_supported_for_plotting(assay: Any) -> None:
+ """Reject assay transformations that the plotting adapter cannot reproduce."""
+ from ..assay import ADTassay, ATACassay
+
+ if isinstance(assay, (ADTassay, ATACassay)):
+ raise NotImplementedError(
+ f"Plotting feature values from {type(assay).__name__} is not "
+ "supported yet. Use RNA-like assays for now, or color by a "
+ "cell-metadata column."
+ )
+
+
+def resolve_feature(
+ store: Any,
+ feature: str | FeatureRef,
+ *,
+ from_assay: str | None = None,
+) -> ResolvedFeature:
+ """Resolve a feature against one assay. Case-sensitive. No silent averaging."""
+ if isinstance(feature, FeatureRef):
+ ref = feature
+ else:
+ ref = FeatureRef(value=feature, assay=from_assay)
+
+ assay_name = ref.assay or from_assay or store._defaultAssay
+ assay = store._get_assay(assay_name)
+
+ if ref.by == "index":
+ idx = int(ref.value)
+ if idx < 0 or idx >= assay.feats.N:
+ raise KeyError(
+ f"Feature index {idx} out of range for assay {assay_name!r} "
+ f"(N={assay.feats.N})"
+ )
+ indices = [idx]
+ elif ref.by == "id":
+ indices = list(assay.feats.get_index_by([str(ref.value)], "ids"))
+ else:
+ indices = list(assay.feats.get_index_by([str(ref.value)], "names"))
+
+ if len(indices) == 0:
+ raise KeyError(
+ f"Feature {ref.value!r} not found in assay {assay_name!r} by {ref.by!r}"
+ )
+ if len(indices) > 1 and ref.reduction is None:
+ raise ValueError(
+ f"Feature {ref.value!r} matches {len(indices)} entries in assay "
+ f"{assay_name!r} at indices {indices}. Pass reduction='mean' or "
+ f"reduction='sum', or look up by id/index."
+ )
+
+ idx_arr = np.asarray(indices)
+ names = tuple(str(x) for x in assay.feats.fetch_all("names")[idx_arr])
+ ids = tuple(str(x) for x in assay.feats.fetch_all("ids")[idx_arr])
+ label = ref.label or (
+ names[0] if len(names) == 1 else f"{ref.value}:{ref.reduction}"
+ )
+ return ResolvedFeature(
+ assay=assay_name,
+ by=ref.by,
+ indices=tuple(int(i) for i in indices),
+ ids=ids,
+ names=names,
+ label=label,
+ reduction=ref.reduction,
+ raw=ref if isinstance(feature, FeatureRef) else feature,
+ )
+
+
+def fetch_normalized_feature_matrix(
+ store: Any,
+ resolved: Sequence[ResolvedFeature],
+ cell_idx: np.ndarray,
+ normalization: NormalizationSpec | None = None,
+) -> np.ndarray:
+ """Return normalized values without mutating assay normalization state."""
+ from ..assay import norm_dummy, norm_lib_size, norm_lib_size_log
+ from ..utils import controlled_compute
+
+ normalization = normalization or NormalizationSpec()
+ if not resolved:
+ return np.empty((len(cell_idx), 0), dtype=np.float64)
+
+ output = np.empty((len(cell_idx), len(resolved)), dtype=np.float64)
+ assay_slots: dict[str, list[int]] = {}
+ for slot, feat in enumerate(resolved):
+ assay_slots.setdefault(feat.assay, []).append(slot)
+
+ for assay_name, slots in assay_slots.items():
+ assay = store._get_assay(assay_name)
+ if normalization.source != "raw":
+ assert_assay_supported_for_plotting(assay)
+ physical_indices = np.unique(
+ np.concatenate(
+ [np.asarray(resolved[slot].indices, dtype=np.int64) for slot in slots]
+ )
+ )
+ counts = controlled_compute(
+ assay.rawData[:, physical_indices][cell_idx, :],
+ store.nthreads,
+ ).astype(np.float64)
+ if counts.ndim == 1:
+ counts = counts.reshape(-1, 1)
+
+ if normalization.source == "raw":
+ normalized = counts
+ elif assay.normMethod is norm_dummy:
+ normalized = counts
+ elif assay.normMethod in (norm_lib_size, norm_lib_size_log):
+ if assay.sf is None:
+ raise ValueError(
+ f"Assay {assay_name!r} requires a size factor for "
+ "library-size normalization"
+ )
+ scalar = np.asarray(
+ assay.cells.fetch_all(f"{assay.name}_nCounts")[cell_idx],
+ dtype=np.float64,
+ ).copy()
+ scalar[scalar == 0] = 1.0
+ normalized = float(assay.sf) * counts / scalar[:, None]
+ else:
+ raise NotImplementedError(
+ f"Plotting does not yet support normalization method "
+ f"{assay.normMethod.__name__!r} for assay {assay_name!r}"
+ )
+ if normalization.transform == "log1p":
+ normalized = np.log1p(normalized)
+
+ for slot in slots:
+ feat = resolved[slot]
+ local = np.searchsorted(
+ physical_indices, np.asarray(feat.indices, dtype=np.int64)
+ )
+ values = normalized[:, local]
+ if values.shape[1] == 1:
+ output[:, slot] = values[:, 0]
+ elif feat.reduction == "sum":
+ output[:, slot] = values.sum(axis=1)
+ else:
+ output[:, slot] = values.mean(axis=1)
+ return output
+
+
+def summarize_features_by_group(
+ store: Any,
+ *,
+ features: Sequence[str | FeatureRef] | Mapping[str, Sequence[str | FeatureRef]],
+ group_by: str | tuple[str, ...],
+ cell_key: str = "I",
+ from_assay: str | None = None,
+ sample_by: str | None = None,
+ study_design: StudyDesign | None = None,
+ normalization: NormalizationSpec | None = None,
+ expression_cutoff: float = 0.0,
+ max_groups: int = 500,
+ max_features: int = 2000,
+ max_samples: int = 500,
+) -> tuple[pd.DataFrame, pd.DataFrame | None]:
+ """Aggregate features by group. With sample_by, samples get equal weight.
+
+ Missing group combinations are omitted (not filled with zeros).
+ """
+ condition_by: str | None = None
+ if study_design is not None:
+ sample_by = study_design.sample_by
+ condition_by = study_design.condition_by
+
+ pairs = coerce_feature_list(features)
+ if len(pairs) > max_features:
+ raise ValueError(
+ f"Too many features ({len(pairs)} > {max_features}). "
+ "Raise max_features explicitly if intentional."
+ )
+ resolved = [
+ resolve_feature(store, feat, from_assay=from_assay) for _, feat in pairs
+ ]
+ group_labels = [g for g, _ in pairs]
+ feature_labels = [r.label for r in resolved]
+
+ if isinstance(group_by, str):
+ group_keys: tuple[str, ...] = (group_by,)
+ else:
+ group_keys = tuple(group_by)
+ if len(group_keys) == 0 or len(group_keys) > 2:
+ raise ValueError("group_by must have 1 or 2 keys")
+
+ cells = store.cells
+ cell_idx = cells.active_index(cell_key)
+ group_cols = [cells.fetch(k, key=cell_key) for k in group_keys]
+ n_groups = int(
+ pd.DataFrame({k: c for k, c in zip(group_keys, group_cols)})
+ .drop_duplicates()
+ .shape[0]
+ )
+ if n_groups > max_groups:
+ raise ValueError(
+ f"Too many groups ({n_groups} > {max_groups}). "
+ "Raise max_groups explicitly if intentional."
+ )
+
+ expr = fetch_normalized_feature_matrix(
+ store,
+ resolved,
+ cell_idx,
+ normalization=normalization,
+ )
+ frac_mask = expr > expression_cutoff
+ base = pd.DataFrame({gk: col for gk, col in zip(group_keys, group_cols)})
+
+ if sample_by is not None:
+ samples = cells.fetch(sample_by, key=cell_key)
+ if condition_by is not None:
+ conditions = cells.fetch(condition_by, key=cell_key)
+ check = pd.DataFrame({"sample": samples, "condition": conditions})
+ nunique = check.groupby("sample", observed=False)["condition"].nunique()
+ bad = nunique[nunique > 1]
+ if len(bad):
+ raise ValueError(
+ "condition_by is not constant within sample(s): "
+ + ", ".join(map(str, list(bad.index[:10])))
+ )
+ valid = pd.notna(samples) & (np.asarray(samples, dtype=object) != "")
+ if int(valid.sum()) == 0:
+ raise ValueError("No cells with valid sample_by values")
+ uniq_samples = pd.unique(np.asarray(samples)[valid])
+ if len(uniq_samples) > max_samples:
+ raise ValueError(
+ f"Too many samples ({len(uniq_samples)} > {max_samples}). "
+ "Raise max_samples explicitly if intentional."
+ )
+ parts: list[pd.DataFrame] = []
+ base_v = base.loc[valid].copy()
+ base_v["sample"] = np.asarray(samples)[valid]
+ for j in range(len(resolved)):
+ part = base_v.copy()
+ part["feature"] = feature_labels[j]
+ part["feature_group"] = group_labels[j]
+ part["value"] = expr[valid, j]
+ part["detected"] = frac_mask[valid, j]
+ parts.append(part)
+ long = pd.concat(parts, ignore_index=True)
+ gb_keys = ["sample", *group_keys, "feature", "feature_group"]
+ per_sample = (
+ long.groupby(gb_keys, observed=False, dropna=False)
+ .agg(
+ mean=("value", "mean"),
+ fraction=("detected", "mean"),
+ n_cells=("value", "size"),
+ variance=("value", "var"),
+ )
+ .reset_index()
+ )
+ agg_keys = [*group_keys, "feature", "feature_group"]
+ aggregate = (
+ per_sample.groupby(agg_keys, observed=False, dropna=False)
+ .agg(
+ mean=("mean", "mean"),
+ fraction=("fraction", "mean"),
+ n_cells=("n_cells", "sum"),
+ n_samples=("sample", "nunique"),
+ variance=("variance", "mean"),
+ )
+ .reset_index()
+ )
+ return aggregate, per_sample
+
+ parts = []
+ for j in range(len(resolved)):
+ part = base.copy()
+ part["feature"] = feature_labels[j]
+ part["feature_group"] = group_labels[j]
+ part["value"] = expr[:, j]
+ part["detected"] = frac_mask[:, j]
+ parts.append(part)
+ long = pd.concat(parts, ignore_index=True)
+ agg_keys = [*group_keys, "feature", "feature_group"]
+ aggregate = (
+ long.groupby(agg_keys, observed=False, dropna=False)
+ .agg(
+ mean=("value", "mean"),
+ fraction=("detected", "mean"),
+ n_cells=("value", "size"),
+ variance=("value", "var"),
+ )
+ .reset_index()
+ )
+ return aggregate, None
diff --git a/scarf/plotting/_deps.py b/scarf/plotting/_deps.py
new file mode 100644
index 00000000..2a20028f
--- /dev/null
+++ b/scarf/plotting/_deps.py
@@ -0,0 +1,46 @@
+"""Lazy loaders for optional plotting dependencies."""
+
+from typing import Any
+
+
+def require_matplotlib() -> tuple[Any, Any]:
+ try:
+ import matplotlib as mpl
+ import matplotlib.pyplot as plt
+ except ImportError as exc:
+ raise ImportError(
+ "Scarf plotting requires matplotlib. Install with: pip install 'scarf[extra]'"
+ ) from exc
+ return plt, mpl
+
+
+def require_seaborn() -> Any:
+ try:
+ import seaborn as sns
+ except ImportError as exc:
+ raise ImportError(
+ "Scarf plotting requires seaborn. Install with: pip install 'scarf[extra]'"
+ ) from exc
+ return sns
+
+
+def require_datashader() -> tuple[Any, Any, Any]:
+ try:
+ import datashader as ds
+ import datashader.transfer_functions as tf
+ from datashader.mpl_ext import dsshow
+ except ImportError as exc:
+ raise ImportError(
+ "Scarf shading requires datashader. Install with: pip install 'scarf[extra]'"
+ ) from exc
+ return ds, dsshow, tf
+
+
+def require_kneed() -> Any:
+ try:
+ from kneed import KneeLocator
+ except ImportError as exc:
+ raise ImportError(
+ "Scarf elbow detection requires kneed. Install with: pip install 'scarf[extra]'"
+ ) from exc
+ return KneeLocator
diff --git a/scarf/plotting/_figure.py b/scarf/plotting/_figure.py
new file mode 100644
index 00000000..63968aca
--- /dev/null
+++ b/scarf/plotting/_figure.py
@@ -0,0 +1,286 @@
+"""PlotResult and figure ownership helpers."""
+
+import json
+from collections.abc import Mapping, Sequence
+from dataclasses import asdict, dataclass, field, is_dataclass
+from pathlib import Path
+from typing import Any, Hashable, cast
+
+import numpy as np
+import pandas as pd
+
+from ._contracts import PlotProvenance
+from ._deps import require_matplotlib
+from ._style import theme_context
+
+
+def _json_ready(value: Any) -> Any:
+ if not isinstance(value, type) and is_dataclass(value):
+ return _json_ready(asdict(cast(Any, value)))
+ if isinstance(value, Mapping):
+ return {str(key): _json_ready(item) for key, item in value.items()}
+ if isinstance(value, (list, tuple, set)):
+ return [_json_ready(item) for item in value]
+ if isinstance(value, np.ndarray):
+ return _json_ready(value.tolist())
+ if isinstance(value, np.generic):
+ return _json_ready(value.item())
+ if isinstance(value, Path):
+ return str(value)
+ if isinstance(value, float):
+ return value if np.isfinite(value) else None
+ if value is None or isinstance(value, (str, int, bool)):
+ return value
+ raise TypeError(f"Value of type {type(value).__name__} is not JSON serializable")
+
+
+@dataclass(slots=True)
+class LegendSpec:
+ """Description of a legend or colorbar attached to a plot."""
+
+ kind: str
+ label: str | None = None
+ scale_key: Hashable | None = None
+ extras: dict[str, Any] = field(default_factory=dict)
+
+
+@dataclass(slots=True)
+class PlotResult:
+ figure: Any
+ axes: dict[Hashable, Any]
+ tables: dict[str, pd.DataFrame]
+ legends: tuple[LegendSpec, ...]
+ scales: tuple[Any, ...]
+ provenance: PlotProvenance
+ owns_figure: bool
+ theme: str = "notebook"
+
+ def show(self) -> None:
+ self.figure.show()
+
+ def close(self) -> None:
+ if not self.owns_figure:
+ return
+ plt, _ = require_matplotlib()
+ plt.close(self.figure)
+
+ def save_provenance(
+ self,
+ path: str | Path,
+ *,
+ figure_path: str | Path | None = None,
+ dpi: float | None = None,
+ ) -> Path:
+ """Write a JSON sidecar describing this plot result."""
+ out = Path(path)
+ out.parent.mkdir(parents=True, exist_ok=True)
+ width, height = self.figure.get_size_inches()
+ payload = {
+ "provenance": self.provenance,
+ "figure": {
+ "width_inches": float(width),
+ "height_inches": float(height),
+ "dpi": float(self.figure.dpi),
+ },
+ "legends": self.legends,
+ "scales": [
+ {
+ "type": type(scale).__name__,
+ "values": scale,
+ }
+ for scale in self.scales
+ ],
+ "tables": {
+ name: {
+ "rows": len(table),
+ "columns": [str(column) for column in table.columns],
+ }
+ for name, table in self.tables.items()
+ },
+ }
+ if figure_path is not None:
+ exported = Path(figure_path)
+ payload["export"] = {
+ "filename": exported.name,
+ "format": exported.suffix.lower().lstrip("."),
+ "dpi": float(dpi if dpi is not None else self.figure.dpi),
+ }
+ out.write_text(
+ json.dumps(
+ _json_ready(payload),
+ indent=2,
+ sort_keys=True,
+ allow_nan=False,
+ )
+ + "\n",
+ encoding="utf-8",
+ )
+ return out
+
+ def save(
+ self,
+ path: str | Path,
+ *,
+ dpi: int | None = None,
+ transparent: bool = False,
+ exact_size: bool = True,
+ tiff_compression: str = "tiff_lzw",
+ provenance_sidecar: bool | str | Path = False,
+ ) -> Path:
+ out = Path(path)
+ if not out.suffix:
+ raise ValueError("Export path must include a file extension")
+ if dpi is not None and dpi <= 0:
+ raise ValueError("dpi must be positive")
+ file_type = out.suffix.lower().lstrip(".")
+ supported = self.figure.canvas.get_supported_filetypes()
+ if file_type not in supported:
+ raise ValueError(
+ f"Unsupported export format {out.suffix!r}; choose from "
+ f"{sorted(supported)}"
+ )
+ out.parent.mkdir(parents=True, exist_ok=True)
+ kwargs: dict[str, Any] = {"transparent": transparent}
+ if dpi is not None:
+ kwargs["dpi"] = dpi
+ elif file_type in ("tif", "tiff"):
+ kwargs["dpi"] = 300
+ if file_type in ("tif", "tiff"):
+ kwargs["pil_kwargs"] = {"compression": tiff_compression}
+ # Exact-size export must not crop; tight bbox changes physical size.
+ kwargs["bbox_inches"] = None if exact_size else "tight"
+ _, mpl = require_matplotlib()
+ export_rc = {"savefig.bbox": None} if exact_size else {}
+ with theme_context(self.theme), mpl.rc_context(export_rc):
+ self.figure.savefig(out, **kwargs)
+ if provenance_sidecar:
+ sidecar = (
+ out.with_suffix(out.suffix + ".json")
+ if provenance_sidecar is True
+ else Path(provenance_sidecar)
+ )
+ if sidecar.resolve() == out.resolve():
+ raise ValueError("Provenance sidecar path must differ from figure path")
+ self.save_provenance(
+ sidecar,
+ figure_path=out,
+ dpi=float(kwargs.get("dpi", self.figure.dpi)),
+ )
+ return out
+
+
+def normalize_axes_target(
+ target: Any | None,
+ *,
+ panel_keys: Sequence[Hashable],
+ figsize: tuple[float, float] | None,
+ n_columns: int | None = None,
+) -> tuple[Any, dict[Hashable, Any], bool]:
+ """Return (figure, axes_map, owns_figure)."""
+ plt, _ = require_matplotlib()
+ keys = list(panel_keys)
+ if not keys:
+ raise ValueError("panel_keys must be non-empty")
+
+ if target is None:
+ if figsize is None:
+ figsize = (4.0 * len(keys), 4.0) if len(keys) > 1 else (5.0, 5.0)
+ if len(keys) == 1:
+ fig, ax = plt.subplots(1, 1, figsize=figsize)
+ return fig, {keys[0]: ax}, True
+ ncols = n_columns if n_columns is not None else min(len(keys), 4)
+ ncols = max(1, min(ncols, len(keys)))
+ nrows = int(np.ceil(len(keys) / ncols))
+ fig, axs = plt.subplots(nrows, ncols, figsize=figsize, squeeze=False)
+ mapping: dict[Hashable, Any] = {}
+ for i, key in enumerate(keys):
+ mapping[key] = axs[i // ncols, i % ncols]
+ for j in range(len(keys), nrows * ncols):
+ fig.delaxes(axs[j // ncols, j % ncols])
+ return fig, mapping, True
+
+ if figsize is not None:
+ raise ValueError("figsize is invalid when a caller-owned target is provided")
+
+ if isinstance(target, Mapping):
+ missing = [k for k in keys if k not in target]
+ if missing:
+ raise KeyError(f"target mapping missing panel keys: {missing}")
+ mapping = {k: target[k] for k in keys}
+ figures = {id(ax.figure): ax.figure for ax in mapping.values()}
+ if len(figures) != 1:
+ raise ValueError("All target axes must belong to the same figure")
+ return next(iter(figures.values())), mapping, False
+
+ if isinstance(target, (list, tuple)) or (
+ isinstance(target, np.ndarray) and target.dtype == object
+ ):
+ arr = np.asarray(target, dtype=object).ravel()
+ if len(arr) != len(keys):
+ raise ValueError(f"Expected {len(keys)} axes for panels, got {len(arr)}")
+ figures = {id(ax.figure): ax.figure for ax in arr}
+ if len(figures) != 1:
+ raise ValueError("All target axes must belong to the same figure")
+ fig = next(iter(figures.values()))
+ return fig, {k: ax for k, ax in zip(keys, arr)}, False
+
+ if len(keys) != 1:
+ raise TypeError(
+ "A single Axes target is only valid for one-panel plots; "
+ "pass a sequence or mapping of axes for faceted plots"
+ )
+ return target.figure, {keys[0]: target}, False
+
+
+def label_panels(
+ axes: Mapping[Hashable, Any] | Sequence[Any],
+ *,
+ labels: Sequence[str] | None = None,
+ fontsize: float = 11,
+ fontweight: str = "bold",
+) -> None:
+ """Add A/B/C panel labels to axes in order."""
+ if isinstance(axes, Mapping):
+ ax_list = list(axes.values())
+ else:
+ ax_list = list(axes)
+ if labels is None:
+ labels = [chr(ord("A") + i) for i in range(len(ax_list))]
+ if len(labels) != len(ax_list):
+ raise ValueError("labels length must match number of axes")
+ for ax, lab in zip(ax_list, labels):
+ ax.text(
+ 0.02,
+ 0.98,
+ lab,
+ transform=ax.transAxes,
+ ha="left",
+ va="top",
+ fontsize=fontsize,
+ fontweight=fontweight,
+ )
+
+
+def collect_legends(
+ figure: Any,
+ results: Sequence[PlotResult],
+) -> tuple[LegendSpec, ...]:
+ """Combine legend/colorbar descriptors from several plot results."""
+ legends: list[LegendSpec] = []
+ for result in results:
+ legends.extend(result.legends)
+ out = tuple(legends)
+ figure._scarf_legends = out # type: ignore[attr-defined]
+ return out
+
+
+def as_2d_axes_array(in_ax: Any, n_columns: int = 1) -> np.ndarray:
+ """Normalize legacy in_ax inputs to a 2D object array (rows, cols)."""
+ if in_ax is None:
+ raise ValueError("in_ax is None")
+ if isinstance(in_ax, np.ndarray):
+ if in_ax.ndim == 2:
+ return in_ax
+ if in_ax.ndim == 1:
+ return in_ax.reshape(1, -1)
+ return np.array([[in_ax]], dtype=object)
diff --git a/scarf/plotting/_legacy.py b/scarf/plotting/_legacy.py
new file mode 100644
index 00000000..e7b4ec9f
--- /dev/null
+++ b/scarf/plotting/_legacy.py
@@ -0,0 +1,265 @@
+"""Helpers for legacy DataStore / scarf.plots callers.
+
+These do not change legacy public behavior by default. They provide safe
+argument handling and an explicit eligibility check for opt-in bridging.
+"""
+
+import logging
+from copy import deepcopy
+from typing import Any
+
+logger = logging.getLogger("scarf")
+
+
+def copy_plot_mutables(
+ *,
+ color_key: dict[Any, Any] | None = None,
+ mask_values: list[Any] | None = None,
+ scatter_kwargs: dict[str, Any] | None = None,
+) -> tuple[dict[Any, Any] | None, list[Any] | None, dict[str, Any] | None]:
+ """Return deep copies so legacy helpers never mutate caller inputs."""
+ return (
+ None if color_key is None else dict(color_key),
+ None if mask_values is None else list(mask_values),
+ None if scatter_kwargs is None else deepcopy(scatter_kwargs),
+ )
+
+
+def plot_layout_bridge_blockers(
+ *,
+ layout_key: str | list[str] | None,
+ color_by: str | list[str] | None,
+ do_shading: bool,
+ mask_values: list[Any] | None,
+ subset_by: str | None,
+ shuffle_df: bool,
+ legend_ondata: bool,
+ legend_onside: bool,
+ force_ints_as_cats: bool,
+ clip_fraction: float,
+ ax: Any,
+ use_plotting: bool = False,
+ title: str | list[str] | None = None,
+ scatter_kwargs: dict[str, Any] | None = None,
+) -> tuple[str, ...]:
+ """Return reasons ``plot_layout`` must stay on the legacy renderer.
+
+ With ``use_plotting=False`` (default), bridging is always blocked.
+ With ``use_plotting=True``, only hard incompatibilities block the bridge.
+ Soft differences (legacy on-data legends, integer-as-category defaults)
+ are allowed; ``scarf.plotting.embedding`` uses its own legend and type rules.
+ """
+ if not use_plotting:
+ return ("bridge_not_enabled",)
+
+ blockers: list[str] = []
+ if do_shading:
+ blockers.append("do_shading")
+ if isinstance(layout_key, list) and len(layout_key) > 1:
+ blockers.append("multiple_layout_keys")
+ if mask_values:
+ blockers.append("mask_values")
+ if shuffle_df:
+ blockers.append("shuffle_df")
+ if title is not None:
+ blockers.append("title")
+ if scatter_kwargs:
+ blockers.append("scatter_kwargs")
+ # subset_by, clip_fraction, and ax are supported by embedding.
+ _ = (subset_by, legend_ondata, legend_onside, force_ints_as_cats, clip_fraction, ax)
+ return tuple(dict.fromkeys(blockers))
+
+
+def embedding_kwargs_from_plot_layout(
+ *,
+ layout_key: str,
+ color_by: Any,
+ cell_key: str,
+ from_assay: str | None,
+ point_size: float,
+ point_sizes: Any,
+ sort_values: bool,
+ cmap: str | None,
+ show_fig: bool,
+ clip_fraction: float = 0.0,
+ subset_by: str | None = None,
+ default_color: str = "steelblue",
+ missing_color: str = "k",
+ target: Any | None = None,
+ figsize: tuple[float, float] | None = None,
+ n_columns: int | None = None,
+ color_key: dict[Any, Any] | None = None,
+) -> dict[str, Any]:
+ """Map a subset of ``plot_layout`` args to ``embedding`` kwargs."""
+ from ._contracts import CategoricalScale, ColorScale
+
+ return {
+ "layout_key": layout_key if isinstance(layout_key, str) else layout_key[0],
+ "color_by": color_by,
+ "cell_key": cell_key,
+ "from_assay": from_assay,
+ "point_size": point_size,
+ "point_sizes": point_sizes,
+ "sort_values": sort_values,
+ "color_scale": ColorScale(cmap=cmap) if cmap is not None else None,
+ "categorical_scale": (
+ CategoricalScale(palette=dict(color_key)) if color_key is not None else None
+ ),
+ "clip_fraction": clip_fraction,
+ "subset_by": subset_by,
+ "default_color": default_color,
+ "missing_color": missing_color,
+ "target": target,
+ "figsize": figsize,
+ "n_columns": n_columns,
+ "show": show_fig,
+ }
+
+
+def try_bridge_plot_layout(
+ store: Any,
+ *,
+ use_plotting: bool,
+ layout_key: str | list[str] | None,
+ color_by: str | list[str] | None,
+ do_shading: bool,
+ mask_values: list[Any] | None,
+ subselection_key: str | None,
+ shuffle_df: bool,
+ legend_ondata: bool,
+ legend_onside: bool,
+ force_ints_as_cats: bool,
+ clip_fraction: float,
+ ax: Any,
+ cell_key: str,
+ from_assay: str | None,
+ point_size: float,
+ size_vals: Any,
+ sort_values: bool,
+ cmap: str | None,
+ default_color: str,
+ mask_color: str,
+ width: float,
+ height: float,
+ n_columns: int,
+ show_fig: bool,
+ savename: str | None,
+ save_dpi: int,
+ color_key: dict[Any, Any] | None = None,
+ title: str | list[str] | None = None,
+ scatter_kwargs: dict[str, Any] | None = None,
+) -> tuple[bool, Any]:
+ """Attempt opt-in bridge to ``scarf.plotting.embedding``.
+
+ Returns ``(handled, result)``. When ``handled`` is False, callers should use
+ the legacy renderer. When True, ``result`` is a ``PlotResult`` or ``None``
+ (after ``show_fig``).
+ """
+ blockers = plot_layout_bridge_blockers(
+ layout_key=layout_key,
+ color_by=color_by,
+ do_shading=do_shading,
+ mask_values=mask_values,
+ subset_by=subselection_key,
+ shuffle_df=shuffle_df,
+ legend_ondata=legend_ondata,
+ legend_onside=legend_onside,
+ force_ints_as_cats=force_ints_as_cats,
+ clip_fraction=clip_fraction,
+ ax=ax,
+ use_plotting=use_plotting,
+ title=title,
+ scatter_kwargs=scatter_kwargs,
+ )
+ if blockers:
+ if use_plotting:
+ logger.warning(
+ "plot_layout(use_plotting=True) fell back to the legacy renderer "
+ "because: %s",
+ ", ".join(blockers),
+ )
+ return False, None
+
+ if layout_key is None:
+ raise ValueError("Please provide a value for `layout_key` parameter.")
+ if isinstance(layout_key, list):
+ layout_key = layout_key[0]
+
+ color_names = [color_by] if isinstance(color_by, str) else color_by or []
+ if cmap is not None and color_key is None:
+ categorical_cmap = False
+ for color in color_names:
+ if color not in store.cells.columns:
+ continue
+ dtype_kind = getattr(store.cells.get_dtype(color), "kind", None)
+ if dtype_kind in ("b", "O", "S", "U", "T") or (
+ dtype_kind in ("i", "u") and force_ints_as_cats
+ ):
+ categorical_cmap = True
+ break
+ if categorical_cmap:
+ logger.warning(
+ "plot_layout(use_plotting=True) fell back to the legacy renderer "
+ "because categorical cmap translation is not exact"
+ )
+ return False, None
+
+ if legend_ondata or legend_onside:
+ logger.info(
+ "plot_layout(use_plotting=True): on-data / side legends from the "
+ "legacy API are not drawn; scarf.plotting.embedding uses its own "
+ "colorbars and categorical styling."
+ )
+
+ from .embedding import embedding
+
+ from ._contracts import CellField
+
+ if isinstance(color_by, str):
+ bridged_colors: Any = [color_by]
+ unwrap_single = True
+ elif color_by is None:
+ bridged_colors = None
+ unwrap_single = False
+ else:
+ bridged_colors = list(color_by)
+ unwrap_single = False
+ if bridged_colors is not None:
+ for index, color in enumerate(bridged_colors):
+ if color not in store.cells.columns:
+ continue
+ dtype_kind = getattr(store.cells.get_dtype(color), "kind", None)
+ if dtype_kind in ("i", "u"):
+ bridged_colors[index] = CellField(
+ color,
+ kind=("categorical" if force_ints_as_cats else "continuous"),
+ )
+ if unwrap_single:
+ bridged_colors = bridged_colors[0]
+
+ kwargs = embedding_kwargs_from_plot_layout(
+ layout_key=layout_key,
+ color_by=bridged_colors,
+ cell_key=cell_key,
+ from_assay=from_assay,
+ point_size=point_size,
+ point_sizes=size_vals,
+ sort_values=sort_values,
+ cmap=cmap,
+ show_fig=False,
+ clip_fraction=clip_fraction,
+ subset_by=subselection_key,
+ default_color=default_color,
+ missing_color=mask_color,
+ target=ax,
+ figsize=None if ax is not None else (width, height),
+ n_columns=n_columns,
+ color_key=color_key,
+ )
+ result = embedding(store, **kwargs)
+ if savename is not None:
+ result.save(savename, dpi=save_dpi)
+ if show_fig:
+ result.show()
+ return True, None
+ return True, result
diff --git a/scarf/plotting/_raster.py b/scarf/plotting/_raster.py
new file mode 100644
index 00000000..38a85342
--- /dev/null
+++ b/scarf/plotting/_raster.py
@@ -0,0 +1,259 @@
+"""Blockwise embedding raster (two-pass; no full-column materialization)."""
+
+from dataclasses import dataclass
+from typing import Any
+
+import numpy as np
+
+from ._deps import require_matplotlib
+from ._style import continuous_norm
+
+
+@dataclass(frozen=True, slots=True)
+class RasterCanvas:
+ """Rasterized embedding panel."""
+
+ image: np.ndarray # float (H, W), NaN where empty
+ counts: np.ndarray # int (H, W)
+ extent: tuple[float, float, float, float] # xmin, xmax, ymin, ymax
+ vmin: float
+ vmax: float
+ n_cells: int
+ n_blocks: int
+
+
+def _finite_minmax(values: np.ndarray) -> tuple[float, float] | None:
+ v = values[np.isfinite(values)]
+ if len(v) == 0:
+ return None
+ return float(v.min()), float(v.max())
+
+
+def _priority_sample_update(
+ sample_values: np.ndarray,
+ sample_priorities: np.ndarray,
+ n_sampled: int,
+ values: np.ndarray,
+ *,
+ capacity: int,
+ rng: np.random.Generator,
+) -> tuple[np.ndarray, np.ndarray, int]:
+ """Update an exact uniform sample using independent random priorities."""
+ vals = values[np.isfinite(values)]
+ if len(vals) == 0:
+ return sample_values, sample_priorities, n_sampled
+
+ new_priorities = rng.random(len(vals))
+ combined_values = np.concatenate((sample_values[:n_sampled], vals))
+ combined_priorities = np.concatenate(
+ (sample_priorities[:n_sampled], new_priorities)
+ )
+ keep = min(capacity, len(combined_values))
+ if len(combined_values) > keep:
+ selected = np.argpartition(combined_priorities, -keep)[-keep:]
+ combined_values = combined_values[selected]
+ combined_priorities = combined_priorities[selected]
+ sample_values[:keep] = combined_values
+ sample_priorities[:keep] = combined_priorities
+ return sample_values, sample_priorities, keep
+
+
+def raster_from_metadata(
+ cells: Any,
+ *,
+ x_key: str,
+ y_key: str,
+ color_key: str | None = None,
+ cell_key: str = "I",
+ pixels: int = 400,
+ block_rows: int | None = None,
+ quantiles: tuple[float, float] | None = (0.01, 0.99),
+ seed: int = 0,
+ sample_capacity: int = 50_000,
+) -> RasterCanvas:
+ """Two-pass raster of metadata columns via ``MetaData.iter_row_blocks``.
+
+ Pass 1: bounds and color limits (exact min/max, or approximate quantiles via
+ reservoir sampling). Pass 2: accumulate mean color per pixel.
+ """
+ if pixels < 8:
+ raise ValueError("pixels must be >= 8")
+ if sample_capacity < 1:
+ raise ValueError("sample_capacity must be >= 1")
+ if quantiles is not None:
+ q0, q1 = quantiles
+ if not (0.0 <= q0 < q1 <= 1.0):
+ raise ValueError("quantiles must satisfy 0 <= low < high <= 1")
+ cols = [x_key, y_key] + ([color_key] if color_key is not None else [])
+ rng = np.random.default_rng(seed)
+
+ # --- Pass 1: bounds + color scale ---
+ xmin = ymin = np.inf
+ xmax = ymax = -np.inf
+ cmin = np.inf
+ cmax = -np.inf
+ sample_values = np.empty(sample_capacity, dtype=np.float64)
+ sample_priorities = np.empty(sample_capacity, dtype=np.float64)
+ n_sampled = 0
+ n_cells = 0
+ n_blocks = 0
+ for block in cells.iter_row_blocks(
+ cell_key=cell_key, columns=cols, block_rows=block_rows
+ ):
+ n_blocks += 1
+ if len(block.active_global_indices) == 0:
+ continue
+ x = np.asarray(block.values[x_key], dtype=np.float64)
+ y = np.asarray(block.values[y_key], dtype=np.float64)
+ finite = np.isfinite(x) & np.isfinite(y)
+ if not finite.any():
+ continue
+ n_cells += int(finite.sum())
+ xmin = min(xmin, float(x[finite].min()))
+ xmax = max(xmax, float(x[finite].max()))
+ ymin = min(ymin, float(y[finite].min()))
+ ymax = max(ymax, float(y[finite].max()))
+ if color_key is not None:
+ c = np.asarray(block.values[color_key], dtype=np.float64)[finite]
+ mm = _finite_minmax(c)
+ if mm is not None:
+ cmin = min(cmin, mm[0])
+ cmax = max(cmax, mm[1])
+ if quantiles is not None:
+ sample_values, sample_priorities, n_sampled = _priority_sample_update(
+ sample_values,
+ sample_priorities,
+ n_sampled,
+ c,
+ capacity=sample_capacity,
+ rng=rng,
+ )
+
+ if n_cells == 0 or not np.isfinite(xmin):
+ empty = np.full((pixels, pixels), np.nan, dtype=np.float64)
+ return RasterCanvas(
+ image=empty,
+ counts=np.zeros((pixels, pixels), dtype=np.int64),
+ extent=(0.0, 1.0, 0.0, 1.0),
+ vmin=0.0,
+ vmax=1.0,
+ n_cells=0,
+ n_blocks=n_blocks,
+ )
+
+ if color_key is None:
+ vmin, vmax = 0.0, 1.0
+ elif not np.isfinite(cmin):
+ vmin, vmax = 0.0, 1.0
+ elif quantiles is not None and n_sampled > 0:
+ sample = sample_values[:n_sampled]
+ q0, q1 = quantiles
+ vmin = float(np.quantile(sample, q0))
+ vmax = float(np.quantile(sample, q1))
+ if vmax <= vmin:
+ vmin, vmax = float(cmin), float(cmax if cmax > cmin else cmin + 1.0)
+ else:
+ vmin, vmax = float(cmin), float(cmax if cmax > cmin else cmin + 1.0)
+
+ # Pad extent slightly so edge points land inside bins.
+ dx = xmax - xmin
+ dy = ymax - ymin
+ pad_x = 0.01 * dx if dx > 0 else 0.5
+ pad_y = 0.01 * dy if dy > 0 else 0.5
+ xmin -= pad_x
+ xmax += pad_x
+ ymin -= pad_y
+ ymax += pad_y
+ if xmax == xmin:
+ xmax = xmin + 1.0
+ if ymax == ymin:
+ ymax = ymin + 1.0
+
+ sums = np.zeros((pixels, pixels), dtype=np.float64)
+ counts = np.zeros((pixels, pixels), dtype=np.int64)
+
+ # --- Pass 2: accumulate ---
+ for block in cells.iter_row_blocks(
+ cell_key=cell_key, columns=cols, block_rows=block_rows
+ ):
+ if len(block.active_global_indices) == 0:
+ continue
+ x = np.asarray(block.values[x_key], dtype=np.float64)
+ y = np.asarray(block.values[y_key], dtype=np.float64)
+ finite = np.isfinite(x) & np.isfinite(y)
+ if not finite.any():
+ continue
+ x = x[finite]
+ y = y[finite]
+ ix = np.clip(
+ ((x - xmin) / (xmax - xmin) * (pixels - 1e-9)).astype(np.int64),
+ 0,
+ pixels - 1,
+ )
+ iy = np.clip(
+ ((y - ymin) / (ymax - ymin) * (pixels - 1e-9)).astype(np.int64),
+ 0,
+ pixels - 1,
+ )
+ # Image row 0 is top; flip y for display extent mapping.
+ iy_img = pixels - 1 - iy
+ if color_key is None:
+ np.add.at(counts, (iy_img, ix), 1)
+ else:
+ c = np.asarray(block.values[color_key], dtype=np.float64)[finite]
+ finite_color = np.isfinite(c)
+ np.add.at(sums, (iy_img[finite_color], ix[finite_color]), c[finite_color])
+ np.add.at(counts, (iy_img[finite_color], ix[finite_color]), 1)
+
+ image = np.full((pixels, pixels), np.nan, dtype=np.float64)
+ nonzero = counts > 0
+ if color_key is None:
+ image[nonzero] = np.log1p(counts[nonzero])
+ vmin = 0.0
+ vmax = float(image[nonzero].max()) if nonzero.any() else 1.0
+ else:
+ image[nonzero] = sums[nonzero] / counts[nonzero]
+ return RasterCanvas(
+ image=image,
+ counts=counts,
+ extent=(xmin, xmax, ymin, ymax),
+ vmin=vmin,
+ vmax=vmax,
+ n_cells=n_cells,
+ n_blocks=n_blocks,
+ )
+
+
+def draw_raster_canvas(
+ ax: Any,
+ canvas: RasterCanvas,
+ *,
+ cmap: str = "viridis",
+ missing_color: str = "#f0f0f0",
+ vcenter: float | None = None,
+) -> Any:
+ """Draw a ``RasterCanvas`` onto a matplotlib axes; return the mappable."""
+ _, mpl = require_matplotlib()
+ data = canvas.image.copy()
+ cmap_obj = mpl.colormaps.get_cmap(cmap).with_extremes(bad=missing_color)
+ norm = continuous_norm(
+ mpl,
+ vmin=canvas.vmin,
+ vmax=canvas.vmax,
+ vcenter=vcenter,
+ )
+ im = ax.imshow(
+ data,
+ origin="upper",
+ extent=[
+ canvas.extent[0],
+ canvas.extent[1],
+ canvas.extent[2],
+ canvas.extent[3],
+ ],
+ cmap=cmap_obj,
+ norm=norm,
+ aspect="equal",
+ interpolation="nearest",
+ )
+ return im
diff --git a/scarf/plotting/_style.py b/scarf/plotting/_style.py
new file mode 100644
index 00000000..df77fca8
--- /dev/null
+++ b/scarf/plotting/_style.py
@@ -0,0 +1,277 @@
+"""Themes and categorical palettes for scarf.plotting."""
+
+from collections.abc import Iterator, Mapping
+from contextlib import contextmanager
+from typing import Any
+
+# Lifted from scanpy.plotting.palettes (also used by legacy scarf.plots).
+CUSTOM_PALETTES: dict[int, list[str]] = {
+ 10: [
+ "#1f77b4",
+ "#ff7f0e",
+ "#279e68",
+ "#d62728",
+ "#aa40fc",
+ "#8c564b",
+ "#e377c2",
+ "#7f7f7f",
+ "#b5bd61",
+ "#17becf",
+ ],
+ 20: [
+ "#1f77b4",
+ "#aec7e8",
+ "#ff7f0e",
+ "#ffbb78",
+ "#2ca02c",
+ "#98df8a",
+ "#d62728",
+ "#ff9896",
+ "#9467bd",
+ "#c5b0d5",
+ "#8c564b",
+ "#c49c94",
+ "#e377c2",
+ "#f7b6d2",
+ "#7f7f7f",
+ "#c7c7c7",
+ "#bcbd22",
+ "#dbdb8d",
+ "#17becf",
+ "#9edae5",
+ ],
+ 28: [
+ "#023fa5",
+ "#7d87b9",
+ "#bec1d4",
+ "#d6bcc0",
+ "#bb7784",
+ "#8e063b",
+ "#4a6fe3",
+ "#8595e1",
+ "#b5bbe3",
+ "#e6afb9",
+ "#e07b91",
+ "#d33f6a",
+ "#11c638",
+ "#8dd593",
+ "#c6dec7",
+ "#ead3c6",
+ "#f0b98d",
+ "#ef9708",
+ "#0fcfc0",
+ "#9cded6",
+ "#d5eae7",
+ "#f3e1eb",
+ "#f6c4e1",
+ "#f79cd4",
+ "#7f7f7f",
+ "#c7c7c7",
+ "#1CE6FF",
+ "#336600",
+ ],
+ 102: [
+ "#FFFF00",
+ "#1CE6FF",
+ "#FF34FF",
+ "#FF4A46",
+ "#008941",
+ "#006FA6",
+ "#A30059",
+ "#FFDBE5",
+ "#7A4900",
+ "#0000A6",
+ "#63FFAC",
+ "#B79762",
+ "#004D43",
+ "#8FB0FF",
+ "#997D87",
+ "#5A0007",
+ "#809693",
+ "#6A3A4C",
+ "#1B4400",
+ "#4FC601",
+ "#3B5DFF",
+ "#4A3B53",
+ "#FF2F80",
+ "#61615A",
+ "#BA0900",
+ "#6B7900",
+ "#00C2A0",
+ "#FFAA92",
+ "#FF90C9",
+ "#B903AA",
+ "#D16100",
+ "#DDEFFF",
+ "#000035",
+ "#7B4F4B",
+ "#A1C299",
+ "#300018",
+ "#0AA6D8",
+ "#013349",
+ "#00846F",
+ "#372101",
+ "#FFB500",
+ "#C2FFED",
+ "#A079BF",
+ "#CC0744",
+ "#C0B9B2",
+ "#C2FF99",
+ "#001E09",
+ "#00489C",
+ "#6F0062",
+ "#0CBD66",
+ "#EEC3FF",
+ "#456D75",
+ "#B77B68",
+ "#7A87A1",
+ "#788D66",
+ "#885578",
+ "#FAD09F",
+ "#FF8A9A",
+ "#D157A0",
+ "#BEC459",
+ "#456648",
+ "#0086ED",
+ "#886F4C",
+ "#34362D",
+ "#B4A8BD",
+ "#00A6AA",
+ "#452C2C",
+ "#636375",
+ "#A3C8C9",
+ "#FF913F",
+ "#938A81",
+ "#575329",
+ "#00FECF",
+ "#B05B6F",
+ "#8CD0FF",
+ "#3B9700",
+ "#04F757",
+ "#C8A1A1",
+ "#1E6E00",
+ "#7900D7",
+ "#A77500",
+ "#6367A9",
+ "#A05837",
+ "#6B002C",
+ "#772600",
+ "#D790FF",
+ "#9B9700",
+ "#549E79",
+ "#FFF69F",
+ "#201625",
+ "#72418F",
+ "#BC23FF",
+ "#99ADC0",
+ "#3A2465",
+ "#922329",
+ "#5B4534",
+ "#FDE8DC",
+ "#404E55",
+ "#0089A3",
+ "#CB7E98",
+ "#A4E804",
+ "#324E72",
+ ],
+}
+
+THEMES: dict[str, dict[str, Any]] = {
+ "paper": {
+ "font.size": 8,
+ "axes.titlesize": 9,
+ "axes.labelsize": 8,
+ "xtick.labelsize": 7,
+ "ytick.labelsize": 7,
+ "legend.fontsize": 7,
+ "axes.linewidth": 0.6,
+ "lines.linewidth": 0.8,
+ "svg.fonttype": "none",
+ "pdf.fonttype": 42,
+ "savefig.dpi": 300,
+ "figure.dpi": 150,
+ },
+ "notebook": {
+ "font.size": 10,
+ "axes.titlesize": 11,
+ "axes.labelsize": 10,
+ "xtick.labelsize": 9,
+ "ytick.labelsize": 9,
+ "legend.fontsize": 9,
+ "axes.linewidth": 0.8,
+ "svg.fonttype": "none",
+ "pdf.fonttype": 42,
+ "savefig.dpi": 150,
+ "figure.dpi": 100,
+ },
+ "minimal": {
+ "font.size": 9,
+ "axes.spines.top": False,
+ "axes.spines.right": False,
+ "svg.fonttype": "none",
+ "pdf.fonttype": 42,
+ },
+}
+
+
+def palette_for_n(n: int) -> list[str]:
+ if n <= 10:
+ return list(CUSTOM_PALETTES[10][:n])
+ if n <= 20:
+ return list(CUSTOM_PALETTES[20][:n])
+ if n <= 28:
+ return list(CUSTOM_PALETTES[28][:n])
+ if n <= 102:
+ return list(CUSTOM_PALETTES[102][:n])
+ from ._deps import require_seaborn
+
+ sns = require_seaborn()
+ return list(sns.color_palette(n_colors=n).as_hex())
+
+
+def categorical_color_map(
+ categories: list[Any],
+ *,
+ palette: Mapping[Any, str] | None = None,
+ missing_label: str | None = None,
+ missing_color: str = "#bdbdbd",
+) -> dict[Any, str]:
+ cats = list(categories)
+ if palette is not None:
+ out = dict(palette)
+ for cat in cats:
+ if cat not in out:
+ raise KeyError(f"Category {cat!r} missing from palette")
+ else:
+ colors = palette_for_n(len(cats))
+ out = dict(zip(cats, colors))
+ if missing_label is not None:
+ out[missing_label] = missing_color
+ return out
+
+
+def continuous_norm(
+ mpl: Any,
+ *,
+ vmin: float,
+ vmax: float,
+ vcenter: float | None,
+) -> Any:
+ if vmax <= vmin:
+ vmax = vmin + 1.0
+ if vcenter is None:
+ return mpl.colors.Normalize(vmin=vmin, vmax=vmax)
+ if not vmin < vcenter < vmax:
+ raise ValueError("vcenter must be strictly between the color limits")
+ return mpl.colors.TwoSlopeNorm(vmin=vmin, vcenter=vcenter, vmax=vmax)
+
+
+@contextmanager
+def theme_context(name: str = "notebook") -> Iterator[None]:
+ if name not in THEMES:
+ raise KeyError(f"Unknown theme {name!r}. Choose from: {sorted(THEMES)}")
+ from ._deps import require_matplotlib
+
+ _, mpl = require_matplotlib()
+ with mpl.rc_context(THEMES[name]):
+ yield
diff --git a/scarf/plotting/composition.py b/scarf/plotting/composition.py
new file mode 100644
index 00000000..681849ec
--- /dev/null
+++ b/scarf/plotting/composition.py
@@ -0,0 +1,438 @@
+"""Sample-level composition figures, including paired subject lines."""
+
+from typing import Any, Hashable, Literal
+
+import numpy as np
+import pandas as pd
+
+from ._contracts import CategoricalScale, PlotProvenance, StudyDesign
+from ._deps import require_matplotlib
+from ._figure import LegendSpec, PlotResult, normalize_axes_target
+from ._style import categorical_color_map, theme_context
+
+
+def _constant_within_sample(
+ samples: np.ndarray,
+ values: np.ndarray,
+ *,
+ field_name: str,
+) -> pd.Series:
+ """Return one value per sample; raise if a sample has multiple values."""
+ df = pd.DataFrame({"sample": samples, "value": values})
+ nunique = df.groupby("sample", observed=False)["value"].nunique(dropna=False)
+ bad = nunique[nunique > 1]
+ if len(bad):
+ raise ValueError(
+ f"{field_name} is not constant within sample(s): "
+ + ", ".join(map(str, list(bad.index[:10])))
+ )
+ return df.groupby("sample", observed=False)["value"].first()
+
+
+def _attach_sample_meta(
+ per_sample: pd.DataFrame,
+ *,
+ samples: np.ndarray,
+ subject_vals: np.ndarray | None,
+ pair_vals: np.ndarray | None,
+ condition_vals: np.ndarray | None,
+) -> pd.DataFrame:
+ meta_parts: list[pd.Series] = []
+ if subject_vals is not None:
+ meta_parts.append(
+ _constant_within_sample(
+ samples, subject_vals, field_name="subject_by"
+ ).rename("subject")
+ )
+ if pair_vals is not None:
+ meta_parts.append(
+ _constant_within_sample(samples, pair_vals, field_name="pair_by").rename(
+ "pair"
+ )
+ )
+ if condition_vals is not None:
+ meta_parts.append(
+ _constant_within_sample(
+ samples, condition_vals, field_name="condition_by"
+ ).rename("condition")
+ )
+ if not meta_parts:
+ return per_sample
+ meta = pd.concat(meta_parts, axis=1).reset_index()
+ return per_sample.merge(meta, on="sample", how="left")
+
+
+def _draw_pair_lines(
+ ax: Any,
+ per_sample: pd.DataFrame,
+ cat_order: list[Any],
+ condition_order: list[Any],
+ pair_col: str,
+) -> int:
+ """Draw one line per category and pair across conditions."""
+ n_lines = 0
+ positions = {
+ (cat, condition): float(index)
+ for index, (cat, condition) in enumerate(
+ (cat, condition) for cat in cat_order for condition in condition_order
+ )
+ }
+ for cat in cat_order:
+ category_rows = per_sample[per_sample["category"] == cat]
+ valid_pair = pd.notna(category_rows[pair_col]) & (
+ category_rows[pair_col].astype(str) != ""
+ )
+ category_rows = category_rows[valid_pair]
+ for _, group in category_rows.groupby(
+ pair_col,
+ observed=False,
+ dropna=True,
+ ):
+ xs: list[float] = []
+ ys: list[float] = []
+ for condition in condition_order:
+ rows = group[group["condition"] == condition]
+ values = rows["proportion"].to_numpy(dtype=np.float64)
+ finite = values[np.isfinite(values)]
+ if len(finite) == 0:
+ continue
+ xs.append(positions[(cat, condition)])
+ ys.append(float(np.mean(finite)))
+ if len(xs) >= 2:
+ ax.plot(
+ xs,
+ ys,
+ color="#757575",
+ alpha=0.45,
+ linewidth=0.9,
+ zorder=1,
+ solid_capstyle="round",
+ )
+ n_lines += 1
+ return n_lines
+
+
+def composition(
+ store: Any,
+ *,
+ category_by: str,
+ cell_key: str = "I",
+ sample_by: str | None = None,
+ subject_by: str | None = None,
+ pair_by: str | None = None,
+ condition_by: str | None = None,
+ study_design: StudyDesign | None = None,
+ kind: Literal["stacked", "per_sample"] = "stacked",
+ categorical_scale: CategoricalScale | None = None,
+ target: Any | None = None,
+ figsize: tuple[float, float] | None = None,
+ theme: str = "notebook",
+ show: bool = False,
+) -> PlotResult:
+ """Cell-type / cluster composition across samples.
+
+ - ``kind='per_sample'``: one point per sample (requires sample_by)
+ - ``kind='stacked'``: stacked bars (by sample if sample_by set, else overall)
+
+ With ``condition_by`` and ``subject_by`` or ``pair_by``, paired lines connect
+ the same subject or pair across conditions within each category. Replicate
+ samples for one pair and condition are averaged for the line.
+ """
+ require_matplotlib()
+ if kind not in ("stacked", "per_sample"):
+ raise ValueError("kind must be 'stacked' or 'per_sample'")
+ if study_design is not None:
+ sample_by = study_design.sample_by
+ subject_by = study_design.subject_by or subject_by
+ pair_by = study_design.pair_by or pair_by
+ condition_by = study_design.condition_by or condition_by
+ if (subject_by is not None or pair_by is not None) and condition_by is None:
+ raise ValueError(
+ "Paired composition requires condition_by together with "
+ "subject_by or pair_by"
+ )
+
+ cats = np.asarray(
+ store.cells.fetch(category_by, key=cell_key),
+ dtype=object,
+ ).copy()
+ if len(cats) == 0:
+ raise ValueError(f"No cells selected by cell_key {cell_key!r}")
+ missing_label = (
+ categorical_scale.missing_label if categorical_scale is not None else "NA"
+ )
+ cats[pd.isna(cats)] = missing_label
+ observed_categories = list(pd.unique(cats))
+ if categorical_scale and categorical_scale.order is not None:
+ cat_order = list(categorical_scale.order)
+ unlisted = [
+ category for category in observed_categories if category not in cat_order
+ ]
+ if unlisted:
+ raise ValueError(
+ "categorical_scale.order is missing observed values: "
+ + ", ".join(map(str, unlisted[:10]))
+ )
+ else:
+ cat_order = sorted(observed_categories, key=lambda value: str(value))
+
+ pair_col: str | None = None
+ n_pair_lines = 0
+ n_unpaired_samples = 0
+ dropped_sample_cells = 0
+
+ if sample_by is None:
+ if kind == "per_sample":
+ raise ValueError("kind='per_sample' requires sample_by or StudyDesign")
+ if subject_by is not None or pair_by is not None:
+ raise ValueError("subject_by / pair_by require sample_by or StudyDesign")
+ counts = pd.Series(cats).value_counts()
+ props = counts.reindex(cat_order).fillna(0) / max(counts.sum(), 1)
+ aggregate = pd.DataFrame(
+ {
+ "category": cat_order,
+ "proportion": [props.get(c, 0.0) for c in cat_order],
+ }
+ )
+ per_sample = None
+ props_mat = None
+ else:
+ samples = store.cells.fetch(sample_by, key=cell_key)
+ valid = pd.notna(samples) & (np.asarray(samples, dtype=object) != "")
+ if not valid.any():
+ raise ValueError(f"No cells have valid values for sample_by {sample_by!r}")
+ dropped_sample_cells = int((~valid).sum())
+ sample_vals_valid = np.asarray(samples)[valid]
+ df = pd.DataFrame(
+ {
+ "sample": sample_vals_valid,
+ "category": np.asarray(cats)[valid],
+ }
+ )
+ ct = pd.crosstab(df["sample"], df["category"])
+ ct = ct.reindex(columns=cat_order, fill_value=0)
+ props_mat = ct.div(ct.sum(axis=1).replace(0, np.nan), axis=0)
+ cell_counts = ct.stack()
+ cell_counts.index = cell_counts.index.set_names(["sample", "category"])
+ cell_counts = cell_counts.rename("n_cells").reset_index()
+ per_sample = props_mat.reset_index().melt(
+ id_vars="sample", var_name="category", value_name="proportion"
+ )
+ per_sample = per_sample.merge(
+ cell_counts, on=["sample", "category"], how="left"
+ )
+
+ subject_vals = (
+ np.asarray(store.cells.fetch(subject_by, key=cell_key))[valid]
+ if subject_by is not None
+ else None
+ )
+ pair_vals = (
+ np.asarray(store.cells.fetch(pair_by, key=cell_key))[valid]
+ if pair_by is not None
+ else None
+ )
+ condition_vals = (
+ np.asarray(store.cells.fetch(condition_by, key=cell_key))[valid]
+ if condition_by is not None
+ else None
+ )
+ per_sample = _attach_sample_meta(
+ per_sample,
+ samples=sample_vals_valid,
+ subject_vals=subject_vals,
+ pair_vals=pair_vals,
+ condition_vals=condition_vals,
+ )
+ if pair_by is not None:
+ pair_col = "pair"
+ elif subject_by is not None:
+ pair_col = "subject"
+ if pair_col is not None:
+ sample_pairs = per_sample[["sample", pair_col]].drop_duplicates()
+ missing_pair = pd.isna(sample_pairs[pair_col]) | (
+ sample_pairs[pair_col].astype(str) == ""
+ )
+ n_unpaired_samples = int(missing_pair.sum())
+
+ aggregate = (
+ per_sample.groupby("category", observed=False)["proportion"]
+ .mean()
+ .reindex(cat_order)
+ .reset_index()
+ )
+
+ palette = categorical_color_map(
+ cat_order,
+ palette=categorical_scale.palette if categorical_scale else None,
+ )
+
+ panel_key: Hashable = "composition"
+ resolved_figsize = figsize
+ if resolved_figsize is None and target is None:
+ resolved_figsize = (6.0, 4.0)
+ fig, axes, owns = normalize_axes_target(
+ target,
+ panel_keys=[panel_key],
+ figsize=resolved_figsize,
+ )
+ ax = axes[panel_key]
+
+ with theme_context(theme):
+ if kind == "stacked":
+ if sample_by is None:
+ bottom = 0.0
+ for cat in cat_order:
+ val = float(
+ aggregate.loc[aggregate["category"] == cat, "proportion"].iloc[
+ 0
+ ]
+ )
+ ax.bar(
+ 0,
+ val,
+ bottom=bottom,
+ color=palette[cat],
+ label=str(cat),
+ width=0.6,
+ )
+ bottom += val
+ ax.set_xticks([])
+ ax.set_ylabel("proportion")
+ else:
+ assert per_sample is not None and props_mat is not None
+ samples_order = list(props_mat.index)
+ bottoms = np.zeros(len(samples_order))
+ x = np.arange(len(samples_order))
+ for cat in cat_order:
+ heights = props_mat[cat].to_numpy(dtype=np.float64)
+ heights = np.nan_to_num(heights, nan=0.0)
+ ax.bar(
+ x,
+ heights,
+ bottom=bottoms,
+ color=palette[cat],
+ label=str(cat),
+ width=0.8,
+ )
+ bottoms += heights
+ ax.set_xticks(x)
+ ax.set_xticklabels(
+ [str(s) for s in samples_order], rotation=45, ha="right"
+ )
+ ax.set_ylabel("proportion")
+ ax.legend(frameon=False, bbox_to_anchor=(1.02, 1), loc="upper left")
+ else:
+ assert per_sample is not None
+ if pair_col is not None:
+ valid_conditions = per_sample["condition"].dropna()
+ valid_conditions = valid_conditions[valid_conditions.astype(str) != ""]
+ condition_order = sorted(
+ valid_conditions.unique(),
+ key=lambda value: str(value),
+ )
+ if len(condition_order) < 2:
+ raise ValueError(
+ "Paired composition requires at least two condition values"
+ )
+ n_pair_lines = _draw_pair_lines(
+ ax,
+ per_sample,
+ cat_order,
+ condition_order,
+ pair_col,
+ )
+ plot_groups = [
+ (category, condition)
+ for category in cat_order
+ for condition in condition_order
+ ]
+ for index, (category, condition) in enumerate(plot_groups):
+ rows = per_sample[
+ (per_sample["category"] == category)
+ & (per_sample["condition"] == condition)
+ ]
+ jitter = (np.arange(len(rows)) - (len(rows) - 1) / 2) * 0.02
+ ax.scatter(
+ np.full(len(rows), index) + jitter,
+ rows["proportion"],
+ c=palette[category],
+ s=40,
+ edgecolors="k",
+ linewidths=0.3,
+ zorder=2,
+ )
+ ax.set_xticks(range(len(plot_groups)))
+ ax.set_xticklabels(
+ [f"{category}\n{condition}" for category, condition in plot_groups],
+ rotation=45,
+ ha="right",
+ )
+ else:
+ for i, cat in enumerate(cat_order):
+ sub = per_sample[per_sample["category"] == cat]
+ jitter = (np.arange(len(sub)) - (len(sub) - 1) / 2) * 0.02
+ ax.scatter(
+ np.full(len(sub), i) + jitter,
+ sub["proportion"],
+ c=palette[cat],
+ s=40,
+ edgecolors="k",
+ linewidths=0.3,
+ label=str(cat),
+ zorder=2,
+ )
+ ax.set_xticks(range(len(cat_order)))
+ ax.set_xticklabels(
+ [str(c) for c in cat_order],
+ rotation=45,
+ ha="right",
+ )
+ ax.set_ylabel("proportion")
+ ax.set_ylim(-0.02, 1.02)
+
+ tables = {"aggregate": aggregate}
+ if per_sample is not None:
+ tables["per_sample"] = per_sample
+
+ from importlib.metadata import version
+
+ try:
+ scarf_version = version("scarf")
+ except Exception:
+ scarf_version = "unknown"
+
+ notes = ["composition", kind]
+ if pair_col is not None:
+ notes.append(f"paired_by={pair_col}")
+
+ result = PlotResult(
+ figure=fig,
+ axes=axes,
+ tables=tables,
+ legends=(LegendSpec(kind="categorical", label=category_by),),
+ scales=(CategoricalScale(order=tuple(cat_order), palette=palette),),
+ provenance=PlotProvenance(
+ scarf_version=scarf_version,
+ cell_key=cell_key,
+ n_cells=int(len(cats)),
+ n_samples=int(per_sample["sample"].nunique())
+ if per_sample is not None
+ else None,
+ renderer="matplotlib",
+ notes=tuple(notes),
+ extras={
+ "subject_by": subject_by,
+ "pair_by": pair_by,
+ "condition_by": condition_by,
+ "n_pair_lines": n_pair_lines,
+ "n_unpaired_samples": n_unpaired_samples,
+ "dropped_sample_cells": dropped_sample_cells,
+ },
+ ),
+ owns_figure=owns,
+ theme=theme,
+ )
+ if show:
+ result.show()
+ return result
diff --git a/scarf/plotting/diagnostics.py b/scarf/plotting/diagnostics.py
new file mode 100644
index 00000000..dfe341b9
--- /dev/null
+++ b/scarf/plotting/diagnostics.py
@@ -0,0 +1,82 @@
+"""Diagnostic plot compatibility wrappers."""
+
+from typing import Any
+
+import numpy as np
+import pandas as pd
+
+
+def qc(
+ data: pd.DataFrame,
+ color: str = "steelblue",
+ cmap: str = "tab20",
+ figsize: tuple[float, float] | None = None,
+ label_size: float = 10.0,
+ title_size: float = 10,
+ sup_title: str | None = None,
+ sup_title_size: float = 12,
+ scatter_size: float = 1.0,
+ max_points: int = 10_000,
+ show_on_single_row: bool = True,
+ show: bool = True,
+) -> Any | None:
+ """Facade for ``scarf.plots.plot_qc``."""
+ from ..plots import plot_qc
+
+ return plot_qc(
+ data=data,
+ color=color,
+ cmap=cmap,
+ fig_size=figsize,
+ label_size=label_size,
+ title_size=title_size,
+ sup_title=sup_title,
+ sup_title_size=sup_title_size,
+ scatter_size=scatter_size,
+ max_points=max_points,
+ show_on_single_row=show_on_single_row,
+ show_fig=show,
+ )
+
+
+def elbow(
+ variance_explained: np.ndarray | list[float],
+ figsize: tuple[float | None, float] = (None, 2),
+) -> None:
+ """Facade for ``scarf.plots.plot_elbow``."""
+ from ..plots import plot_elbow
+
+ return plot_elbow(variance_explained, figsize=figsize)
+
+
+def graph_qc(graph: Any) -> None:
+ """Facade for ``scarf.plots.plot_graph_qc``."""
+ from ..plots import plot_graph_qc
+
+ return plot_graph_qc(graph)
+
+
+def highly_variable_features(
+ mean_nonzero: np.ndarray,
+ corrected_variance: np.ndarray,
+ n_cells: np.ndarray,
+ selected: np.ndarray,
+ *,
+ label_size: float = 12,
+ figsize: tuple[float, float] = (4.5, 4.0),
+ point_sizes: tuple[float, float] = (3, 30),
+ colormaps: tuple[str, str] = ("winter", "magma_r"),
+) -> None:
+ """Plot diagnostics for highly variable feature selection."""
+ from ..plots import plot_mean_var
+
+ plot_mean_var(
+ nzm=mean_nonzero,
+ fv=corrected_variance,
+ n_cells=n_cells,
+ hvg=selected,
+ ax_label_fs=label_size,
+ fig_size=figsize,
+ ss=point_sizes,
+ cmaps=colormaps,
+ )
diff --git a/scarf/plotting/distribution.py b/scarf/plotting/distribution.py
new file mode 100644
index 00000000..368682f6
--- /dev/null
+++ b/scarf/plotting/distribution.py
@@ -0,0 +1,358 @@
+"""Distribution plots for metadata and feature values."""
+
+from collections.abc import Sequence
+from typing import Any, Hashable, Literal
+
+import numpy as np
+import pandas as pd
+
+from ._contracts import CellField, FeatureRef, NormalizationSpec, PlotProvenance
+from ._data import fetch_normalized_feature_matrix, resolve_feature
+from ._deps import require_matplotlib, require_seaborn
+from ._figure import LegendSpec, PlotResult, normalize_axes_target
+from ._style import theme_context
+
+DistKind = Literal["violin", "box", "hist", "ecdf"]
+
+
+def _scarf_version() -> str:
+ try:
+ from importlib.metadata import version
+
+ return version("scarf")
+ except Exception:
+ return "unknown"
+
+
+def _fetch_series(
+ store: Any,
+ key: str | CellField | FeatureRef,
+ *,
+ cell_key: str,
+ from_assay: str | None,
+ normalization: NormalizationSpec,
+) -> tuple[np.ndarray, str]:
+ if isinstance(key, CellField):
+ return np.asarray(
+ store.cells.fetch(key.key, key=cell_key)
+ ), key.label or key.key
+ if isinstance(key, FeatureRef) or (
+ isinstance(key, str) and key not in store.cells.columns
+ ):
+ resolved = resolve_feature(store, key, from_assay=from_assay)
+ cell_idx = store.cells.active_index(cell_key)
+ mat = fetch_normalized_feature_matrix(
+ store,
+ [resolved],
+ cell_idx,
+ normalization=normalization,
+ )
+ return mat[:, 0], resolved.label
+ return np.asarray(store.cells.fetch(str(key), key=cell_key)), str(key)
+
+
+def _subsample_frame(
+ df: pd.DataFrame,
+ *,
+ max_points: int,
+ rng: np.random.Generator,
+) -> tuple[pd.DataFrame, bool]:
+ """Return (frame, was_subsampled)."""
+ if max_points <= 0 or len(df) <= max_points:
+ return df, False
+ idx = rng.choice(len(df), size=max_points, replace=False)
+ return df.iloc[np.sort(idx)], True
+
+
+def _draw_violin_or_box(
+ ax: Any,
+ sns: Any,
+ df: pd.DataFrame,
+ *,
+ kind: Literal["violin", "box"],
+ color: str,
+ max_points: int,
+ point_size: float,
+ rng: np.random.Generator,
+) -> bool:
+ if kind == "violin":
+ sns.violinplot(
+ data=df,
+ x="group",
+ y="value",
+ ax=ax,
+ color=color,
+ inner=None,
+ cut=0,
+ linewidth=1,
+ )
+ else:
+ sns.boxplot(data=df, x="group", y="value", ax=ax, color=color)
+
+ subsampled = False
+ if max_points > 0 and len(df) > 0:
+ pts, subsampled = _subsample_frame(df, max_points=max_points, rng=rng)
+ sns.stripplot(
+ data=pts,
+ x="group",
+ y="value",
+ ax=ax,
+ color="k",
+ size=point_size,
+ jitter=0.35,
+ alpha=0.4,
+ )
+ return subsampled
+
+
+def _draw_hist(
+ ax: Any,
+ df: pd.DataFrame,
+ *,
+ color: str,
+ bins: int,
+ group_by: str | None,
+) -> None:
+ if group_by is None:
+ vals = df["value"].to_numpy(dtype=np.float64)
+ vals = vals[np.isfinite(vals)]
+ ax.hist(vals, bins=bins, color=color, edgecolor="white", linewidth=0.4)
+ return
+ all_values = df["value"].to_numpy(dtype=np.float64)
+ all_values = all_values[np.isfinite(all_values)]
+ shared_bins = (
+ np.histogram_bin_edges(all_values, bins=bins) if len(all_values) else bins
+ )
+ for g, sub in df.groupby("group", observed=False, sort=True):
+ vals = sub["value"].to_numpy(dtype=np.float64)
+ vals = vals[np.isfinite(vals)]
+ if len(vals) == 0:
+ continue
+ ax.hist(
+ vals,
+ bins=shared_bins,
+ alpha=0.45,
+ label=str(g),
+ edgecolor="white",
+ linewidth=0.3,
+ )
+ ax.legend(frameon=False, fontsize=8)
+
+
+def _ecdf_xy(values: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
+ v = np.sort(values[np.isfinite(values)])
+ if len(v) == 0:
+ return np.array([]), np.array([])
+ y = np.arange(1, len(v) + 1, dtype=np.float64) / len(v)
+ return v, y
+
+
+def _draw_ecdf(
+ ax: Any,
+ df: pd.DataFrame,
+ *,
+ color: str,
+ max_points: int,
+ rng: np.random.Generator,
+ group_by: str | None,
+) -> bool:
+ """Draw ECDF. May subsample values for display; returns whether subsampled."""
+ subsampled = False
+ if group_by is None:
+ vals = df["value"].to_numpy(dtype=np.float64)
+ if max_points > 0 and len(vals) > max_points:
+ vals = vals[rng.choice(len(vals), size=max_points, replace=False)]
+ subsampled = True
+ x, y = _ecdf_xy(vals)
+ if len(x):
+ ax.step(x, y, where="post", color=color, linewidth=1.5)
+ return subsampled
+
+ for g, sub in df.groupby("group", observed=False, sort=True):
+ vals = sub["value"].to_numpy(dtype=np.float64)
+ if max_points > 0 and len(vals) > max_points:
+ vals = vals[rng.choice(len(vals), size=max_points, replace=False)]
+ subsampled = True
+ x, y = _ecdf_xy(vals)
+ if len(x):
+ ax.step(x, y, where="post", linewidth=1.2, label=str(g))
+ ax.legend(frameon=False, fontsize=8)
+ ax.set_ylim(-0.02, 1.02)
+ return subsampled
+
+
+def distribution(
+ store: Any,
+ keys: str | CellField | FeatureRef | Sequence[str | CellField | FeatureRef],
+ *,
+ group_by: str | None = None,
+ cell_key: str = "I",
+ from_assay: str | None = None,
+ normalization: NormalizationSpec | None = None,
+ kind: DistKind = "violin",
+ bins: int = 40,
+ max_points: int = 10000,
+ point_size: float = 1.0,
+ seed: int = 0,
+ color: str = "steelblue",
+ target: Any | None = None,
+ figsize: tuple[float, float] | None = None,
+ theme: str = "notebook",
+ show: bool = False,
+) -> PlotResult:
+ """Violin, box, histogram, or ECDF for metadata columns and/or features.
+
+ For ``violin`` / ``box``, overlay points are a deterministic subsample when
+ ``max_points`` is below the cell count. For ``ecdf``, values may be
+ subsampled the same way for display. Both cases are recorded in provenance
+ as approximate when subsampling occurs. Histograms use all cells.
+ """
+ require_matplotlib()
+ normalization = normalization or NormalizationSpec()
+ if kind not in ("violin", "box", "hist", "ecdf"):
+ raise ValueError("kind must be 'violin', 'box', 'hist', or 'ecdf'")
+ if kind in ("violin", "box"):
+ sns = require_seaborn()
+ else:
+ sns = None
+ if bins < 1:
+ raise ValueError("bins must be >= 1")
+
+ if isinstance(keys, (str, CellField, FeatureRef)):
+ key_list: list[str | CellField | FeatureRef] = [keys]
+ else:
+ key_list = list(keys)
+ if not key_list:
+ raise ValueError("keys must be non-empty")
+ feature_assays: set[str] = set()
+ for key in key_list:
+ if not (
+ isinstance(key, FeatureRef)
+ or (isinstance(key, str) and key not in store.cells.columns)
+ ):
+ continue
+ assay_name = (
+ key.assay
+ if isinstance(key, FeatureRef) and key.assay is not None
+ else from_assay or store._defaultAssay
+ )
+ feature_assays.add(assay_name)
+
+ series_list = [
+ _fetch_series(
+ store,
+ k,
+ cell_key=cell_key,
+ from_assay=from_assay,
+ normalization=normalization,
+ )
+ for k in key_list
+ ]
+ n = len(series_list[0][0])
+ if group_by is not None:
+ groups = np.asarray(store.cells.fetch(group_by, key=cell_key))
+ if len(groups) != n:
+ raise ValueError("group_by length does not match selected cells")
+ else:
+ groups = np.zeros(n, dtype=int)
+
+ panel_keys: list[Hashable] = [label for _, label in series_list]
+ if len(set(panel_keys)) != len(panel_keys):
+ panel_keys = list(range(len(panel_keys)))
+
+ if figsize is None and target is None:
+ n_groups = int(pd.Series(groups).nunique())
+ figsize = (max(3.0, 1.2 * n_groups + 1.5) * len(panel_keys), 3.5)
+
+ fig, axes, owns = normalize_axes_target(
+ target,
+ panel_keys=panel_keys,
+ figsize=figsize,
+ n_columns=len(panel_keys),
+ )
+
+ rng = np.random.default_rng(seed)
+ any_subsampled = False
+ with theme_context(theme):
+ for (vals, label), panel_key in zip(series_list, panel_keys):
+ ax = axes[panel_key]
+ df = pd.DataFrame({"value": vals, "group": groups})
+ if kind in ("violin", "box"):
+ assert sns is not None
+ subsampled = _draw_violin_or_box(
+ ax,
+ sns,
+ df,
+ kind=kind, # type: ignore[arg-type]
+ color=color,
+ max_points=max_points,
+ point_size=point_size,
+ rng=rng,
+ )
+ any_subsampled = any_subsampled or subsampled
+ if group_by is None:
+ ax.set_xticks([])
+ elif kind == "hist":
+ _draw_hist(ax, df, color=color, bins=bins, group_by=group_by)
+ else:
+ subsampled = _draw_ecdf(
+ ax,
+ df,
+ color=color,
+ max_points=max_points,
+ rng=rng,
+ group_by=group_by,
+ )
+ any_subsampled = any_subsampled or subsampled
+ ax.set_ylabel("ECDF")
+
+ ax.set_title(label)
+ if kind in ("hist", "ecdf"):
+ ax.set_xlabel(label)
+ if kind == "hist":
+ ax.set_ylabel("count")
+ else:
+ ax.set_xlabel(group_by or "")
+ ax.set_ylabel(label)
+
+ label_counts = pd.Series([label for _, label in series_list]).value_counts()
+ tables = {}
+ for index, (vals, label) in enumerate(series_list):
+ table_name = label if label_counts[label] == 1 else f"{index}:{label}"
+ tables[table_name] = pd.DataFrame({"value": vals, "group": groups})
+ notes = ["distribution", kind]
+ if any_subsampled:
+ notes.append("subsampled_display")
+
+ result = PlotResult(
+ figure=fig,
+ axes=axes,
+ tables=tables,
+ legends=(LegendSpec(kind="distribution", label=kind),),
+ scales=(),
+ provenance=PlotProvenance(
+ scarf_version=_scarf_version(),
+ assay=(next(iter(feature_assays)) if len(feature_assays) == 1 else None),
+ cell_key=cell_key,
+ n_cells=n,
+ renderer="matplotlib",
+ notes=tuple(notes),
+ extras={
+ "max_points": max_points,
+ "seed": seed,
+ "group_by": group_by,
+ "bins": bins if kind == "hist" else None,
+ "approximate": any_subsampled,
+ "normalization": {
+ "source": normalization.source,
+ "transform": normalization.transform,
+ },
+ "assays": sorted(feature_assays),
+ },
+ ),
+ owns_figure=owns,
+ theme=theme,
+ )
+ if show:
+ result.show()
+ return result
diff --git a/scarf/plotting/embedding.py b/scarf/plotting/embedding.py
new file mode 100644
index 00000000..7cdc2736
--- /dev/null
+++ b/scarf/plotting/embedding.py
@@ -0,0 +1,699 @@
+"""Embedding scatter plots."""
+
+from collections.abc import Sequence
+from typing import Any, Hashable
+
+import numpy as np
+import pandas as pd
+
+from ._contracts import (
+ CategoricalScale,
+ CellField,
+ ColorScale,
+ FeatureRef,
+ NormalizationSpec,
+ PlotProvenance,
+)
+from ._data import fetch_normalized_feature_matrix, resolve_feature
+from ._deps import require_matplotlib
+from ._figure import LegendSpec, PlotResult, normalize_axes_target
+from ._style import categorical_color_map, continuous_norm, theme_context
+
+
+def _is_categorical(values: pd.Series, kind: str) -> bool:
+ if kind == "categorical":
+ return True
+ if kind == "continuous":
+ return False
+ if values.dtype.name in (
+ "category",
+ "object",
+ "bool",
+ "string",
+ ) or pd.api.types.is_string_dtype(values):
+ return True
+ if pd.api.types.is_integer_dtype(values) and values.nunique(dropna=True) <= 100:
+ return True
+ return False
+
+
+def _scarf_version() -> str:
+ try:
+ from importlib.metadata import version
+
+ return version("scarf")
+ except Exception:
+ return "unknown"
+
+
+def _coerce_color_items(
+ color_by: str
+ | FeatureRef
+ | CellField
+ | Sequence[str | FeatureRef | CellField]
+ | None,
+) -> list[str | FeatureRef | CellField | None]:
+ if color_by is None:
+ return [None]
+ if isinstance(color_by, (str, FeatureRef, CellField)):
+ return [color_by]
+ return list(color_by)
+
+
+def _prefetch_colors(
+ store: Any,
+ color_items: Sequence[str | FeatureRef | CellField | None],
+ *,
+ from_assay: str | None,
+ cell_key: str,
+ n_cells: int,
+ normalization: NormalizationSpec,
+) -> list[tuple[np.ndarray, str, bool, bool]]:
+ """Return list of (values, label, is_categorical, is_uniform)."""
+ out: list[tuple[np.ndarray, str, bool, bool]] = []
+
+ # Batch RNA-like feature refs / gene strings for one matrix read.
+ feature_slots: list[tuple[int, Any]] = []
+ for i, item in enumerate(color_items):
+ if item is None:
+ out.append((np.ones(n_cells), "cells", False, True))
+ continue
+ if isinstance(item, CellField):
+ vals = store.cells.fetch(item.key, key=cell_key)
+ series = pd.Series(vals)
+ out.append(
+ (
+ np.asarray(vals),
+ item.label or item.key,
+ _is_categorical(series, item.kind),
+ False,
+ )
+ )
+ continue
+ if isinstance(item, str) and item in store.cells.columns:
+ vals = store.cells.fetch(item, key=cell_key)
+ series = pd.Series(vals)
+ out.append((np.asarray(vals), item, _is_categorical(series, "auto"), False))
+ continue
+ # Feature path: placeholder, fill after batch resolve
+ out.append((np.zeros(n_cells), "", False, False))
+ feature_slots.append((i, item))
+
+ if feature_slots:
+ resolved = [
+ resolve_feature(store, item, from_assay=from_assay)
+ for _, item in feature_slots
+ ]
+ cell_idx = store.cells.active_index(cell_key)
+ mat = fetch_normalized_feature_matrix(
+ store,
+ resolved,
+ cell_idx,
+ normalization=normalization,
+ )
+ for col_i, (slot_i, _) in enumerate(feature_slots):
+ feat = resolved[col_i]
+ out[slot_i] = (mat[:, col_i], feat.label, False, False)
+ return out
+
+
+def _continuous_limits(
+ values: np.ndarray,
+ color_scale: ColorScale,
+) -> tuple[float, float]:
+ v = np.asarray(values, dtype=np.float64)
+ finite = np.isfinite(v)
+ if not finite.any():
+ return 0.0, 1.0
+ if color_scale.quantiles is not None:
+ q0, q1 = color_scale.quantiles
+ vmin = float(np.nanquantile(v[finite], q0))
+ vmax = float(np.nanquantile(v[finite], q1))
+ else:
+ vmin = float(np.nanmin(v[finite]))
+ vmax = float(np.nanmax(v[finite]))
+ if color_scale.vmin is not None:
+ vmin = color_scale.vmin
+ if color_scale.vmax is not None:
+ vmax = color_scale.vmax
+ if vmax <= vmin:
+ if color_scale.vmin is not None or color_scale.vmax is not None:
+ raise ValueError("Color limits must satisfy vmin < vmax")
+ vmax = vmin + 1.0
+ return vmin, vmax
+
+
+def _draw_categorical(
+ ax: Any,
+ xx: np.ndarray,
+ yy: np.ndarray,
+ vv: np.ndarray,
+ ss: np.ndarray,
+ *,
+ order: list[Any],
+ palette: dict[Any, str],
+ missing_color: str,
+ rasterized: bool,
+) -> None:
+ colors = [
+ missing_color if pd.isna(val) or val not in palette else palette[val]
+ for val in vv
+ ]
+ ax.scatter(
+ xx,
+ yy,
+ c=colors,
+ s=ss,
+ linewidths=0.1,
+ edgecolors="k",
+ rasterized=rasterized,
+ )
+
+
+def _add_categorical_legend(
+ ax: Any,
+ mpl: Any,
+ *,
+ order: list[Any],
+ palette: dict[Any, str],
+ label: str,
+ missing: bool,
+ missing_color: str,
+ missing_label: str,
+) -> None:
+ handles = [
+ mpl.lines.Line2D(
+ [],
+ [],
+ marker="o",
+ linestyle="",
+ markerfacecolor=palette[value],
+ markeredgecolor="k",
+ markeredgewidth=0.3,
+ markersize=5,
+ label=str(value),
+ )
+ for value in order
+ ]
+ if missing:
+ handles.append(
+ mpl.lines.Line2D(
+ [],
+ [],
+ marker="o",
+ linestyle="",
+ markerfacecolor=missing_color,
+ markeredgecolor="k",
+ markeredgewidth=0.3,
+ markersize=5,
+ label=missing_label,
+ )
+ )
+ ax.legend(
+ handles=handles,
+ title=label or None,
+ frameon=False,
+ bbox_to_anchor=(1.02, 1.0),
+ loc="upper left",
+ borderaxespad=0,
+ )
+
+
+def _draw_continuous(
+ ax: Any,
+ fig: Any,
+ xx: np.ndarray,
+ yy: np.ndarray,
+ vnum: np.ndarray,
+ ss: np.ndarray,
+ *,
+ limits: tuple[float, float],
+ cmap_name: str | None,
+ vcenter: float | None,
+ missing_color: str,
+ default_color: str,
+ label: str,
+ is_uniform: bool,
+ sort_values: bool,
+ rng: np.random.Generator | None,
+ add_colorbar: bool,
+ rasterized: bool,
+ plt: Any,
+ mpl: Any,
+) -> None:
+ finite = np.isfinite(vnum)
+ order_idx = np.arange(len(vnum))
+ if sort_values and finite.any():
+ order_idx = np.argsort(np.where(finite, vnum, -np.inf))
+ elif rng is not None:
+ order_idx = rng.permutation(len(vnum))
+ xx = xx[order_idx]
+ yy = yy[order_idx]
+ vnum = vnum[order_idx]
+ ss = ss[order_idx]
+ finite = finite[order_idx]
+
+ if is_uniform or len(xx) == 0:
+ ax.scatter(
+ xx,
+ yy,
+ c=[default_color] * len(xx),
+ s=ss if len(ss) else 10,
+ linewidths=0.1,
+ edgecolors="k",
+ rasterized=rasterized,
+ )
+ return
+
+ vmin, vmax = limits
+ if vmax == vmin:
+ vmax = vmin + 1.0
+ norm = continuous_norm(mpl, vmin=vmin, vmax=vmax, vcenter=vcenter)
+ cmap = plt.get_cmap(cmap_name or "viridis")
+ face = np.empty((len(vnum), 4))
+ face[:] = mpl.colors.to_rgba(missing_color)
+ if finite.any():
+ face[finite] = cmap(norm(vnum[finite]))
+ ax.scatter(
+ xx,
+ yy,
+ c=face,
+ s=ss,
+ linewidths=0.1,
+ edgecolors="k",
+ rasterized=rasterized,
+ )
+ if add_colorbar:
+ sm = plt.cm.ScalarMappable(cmap=cmap, norm=norm)
+ sm.set_array([])
+ cb = fig.colorbar(sm, ax=ax, shrink=0.6, fraction=0.05, pad=0.02)
+ if label:
+ cb.set_label(label)
+
+
+def _soft_clip(values: np.ndarray, clip_fraction: float) -> np.ndarray:
+ if clip_fraction <= 0:
+ return values
+ if clip_fraction >= 0.5:
+ raise ValueError("clip_fraction must be in [0, 0.5)")
+ v = np.asarray(values, dtype=np.float64).copy()
+ finite = np.isfinite(v)
+ if not finite.any():
+ return v
+ lo = float(np.percentile(v[finite], 100 * clip_fraction))
+ hi = float(np.percentile(v[finite], 100 - 100 * clip_fraction))
+ v[finite & (v < lo)] = lo
+ v[finite & (v > hi)] = hi
+ return v
+
+
+def embedding(
+ store: Any,
+ *,
+ layout_key: str,
+ color_by: str
+ | FeatureRef
+ | CellField
+ | Sequence[str | FeatureRef | CellField]
+ | None = None,
+ facet_by: str | None = None,
+ facet_order: Sequence[Any] | None = None,
+ cell_key: str = "I",
+ from_assay: str | None = None,
+ normalization: NormalizationSpec | None = None,
+ point_size: float = 10,
+ point_sizes: np.ndarray | Sequence[float] | None = None,
+ sort_values: bool = False,
+ color_scale: ColorScale | None = None,
+ categorical_scale: CategoricalScale | None = None,
+ default_color: str = "steelblue",
+ missing_color: str = "#bdbdbd",
+ clip_fraction: float = 0.0,
+ subset_by: str | None = None,
+ n_columns: int | None = None,
+ target: Any | None = None,
+ figsize: tuple[float, float] | None = None,
+ theme: str = "notebook",
+ seed: int | None = None,
+ rasterize_threshold: int = 50_000,
+ show: bool = False,
+) -> PlotResult:
+ """2D embedding scatter with shared per-feature color scales across facets.
+
+ Multi-gene × condition layouts use one row per color value and one column
+ per facet by default. Color limits and categorical palettes are computed on
+ the full selected population so panels stay comparable.
+
+ Preserves useful ``DataStore.plot_layout`` behaviors:
+ - ``sort_values=True`` draws higher continuous values on top
+ - ``point_sizes`` sets per-point marker areas (e.g. mapping confidence)
+ """
+ plt, mpl = require_matplotlib()
+ if rasterize_threshold < 0:
+ raise ValueError("rasterize_threshold must be >= 0")
+ normalization = normalization or NormalizationSpec()
+ color_scale = color_scale or ColorScale(cmap="viridis", scope="feature")
+ missing = (
+ categorical_scale.missing_color
+ if categorical_scale is not None
+ else missing_color
+ )
+
+ x = np.asarray(store.cells.fetch(f"{layout_key}1", key=cell_key), dtype=np.float64)
+ y = np.asarray(store.cells.fetch(f"{layout_key}2", key=cell_key), dtype=np.float64)
+ n = len(x)
+ if n == 0:
+ raise ValueError(f"No cells selected by cell_key {cell_key!r}")
+ finite_coordinates = np.isfinite(x) & np.isfinite(y)
+ if not finite_coordinates.any():
+ raise ValueError(f"Layout {layout_key!r} has no finite coordinates")
+ if point_sizes is not None and len(point_sizes) != n:
+ raise ValueError("point_sizes length must match number of selected cells")
+ size_arr = (
+ np.asarray(point_sizes, dtype=np.float64)
+ if point_sizes is not None
+ else np.full(n, point_size, dtype=np.float64)
+ )
+
+ color_items = _coerce_color_items(color_by)
+ color_cache = _prefetch_colors(
+ store,
+ color_items,
+ from_assay=from_assay,
+ cell_key=cell_key,
+ n_cells=n,
+ normalization=normalization,
+ )
+ if clip_fraction > 0:
+ clipped: list[tuple[np.ndarray, str, bool, bool]] = []
+ for vals, label, is_cat, is_uniform in color_cache:
+ if is_cat or is_uniform:
+ clipped.append((vals, label, is_cat, is_uniform))
+ else:
+ clipped.append(
+ (_soft_clip(vals, clip_fraction), label, is_cat, is_uniform)
+ )
+ color_cache = clipped
+
+ # Optional boolean cell subselection (legacy plot_layout behavior)
+ base_mask = finite_coordinates.copy()
+ if subset_by is not None:
+ sub = np.asarray(store.cells.fetch(subset_by, key=cell_key))
+ if sub.dtype != bool:
+ raise TypeError(f"subset_by {subset_by!r} must be boolean; got {sub.dtype}")
+ if len(sub) != n:
+ raise ValueError("subset_by length must match selected cells")
+ base_mask &= sub
+ if not base_mask.any():
+ raise ValueError("No cells remain after applying layout/subset filters")
+
+ if facet_by is not None:
+ facet_values = np.asarray(store.cells.fetch(facet_by, key=cell_key))
+ if facet_order is not None:
+ facets = list(facet_order)
+ else:
+ facets = sorted(pd.unique(facet_values), key=lambda v: (pd.isna(v), str(v)))
+ else:
+ facet_values = None
+ facets = [None]
+
+ # Stable panel keys: (color_label, facet) when faceting, else color_label
+ panel_keys: list[Hashable] = []
+ for _, label, _, _ in color_cache:
+ for fac in facets:
+ if fac is None:
+ panel_keys.append(label)
+ else:
+ panel_keys.append((label, fac))
+ if len(set(panel_keys)) != len(panel_keys):
+ panel_keys = [(i, k) for i, k in enumerate(panel_keys)]
+
+ n_colors = len(color_cache)
+ n_facets = len(facets)
+ if n_columns is None:
+ n_columns = n_facets if facet_by is not None else min(n_colors, 4)
+ n_columns = max(1, min(n_columns, len(panel_keys)))
+
+ if figsize is None and target is None:
+ nrows = int(np.ceil(len(panel_keys) / n_columns))
+ figsize = (3.6 * n_columns, 3.6 * nrows)
+
+ fig, axes, owns = normalize_axes_target(
+ target, panel_keys=panel_keys, figsize=figsize, n_columns=n_columns
+ )
+
+ selected_x = x[base_mask]
+ selected_y = y[base_mask]
+ xpad = 0.05 * (float(selected_x.max() - selected_x.min()) or 1.0)
+ ypad = 0.05 * (float(selected_y.max() - selected_y.min()) or 1.0)
+ xlim = (float(selected_x.min() - xpad), float(selected_x.max() + xpad))
+ ylim = (float(selected_y.min() - ypad), float(selected_y.max() + ypad))
+
+ labels = [label for _, label, _, _ in color_cache]
+ label_counts = pd.Series(labels).value_counts()
+
+ def display_key(index: int, label: str) -> str:
+ return label if label_counts[label] == 1 else f"{index}:{label}"
+
+ limit_map: dict[int, tuple[float, float]] = {}
+ categorical_maps: dict[int, tuple[list[Any], dict[Any, str]]] = {}
+ shared_limits: tuple[float, float] | None = None
+ if color_scale.scope == "shared":
+ shared_values = [
+ np.asarray(values, dtype=np.float64)[base_mask]
+ for values, _, is_categorical, is_uniform in color_cache
+ if not is_categorical and not is_uniform
+ ]
+ if shared_values:
+ shared_limits = _continuous_limits(
+ np.concatenate(shared_values),
+ color_scale,
+ )
+
+ for color_index, (vals, _, is_cat, is_uniform) in enumerate(color_cache):
+ if is_uniform:
+ continue
+ vals_sel = np.asarray(vals)[base_mask]
+ if is_cat:
+ observed = list(pd.Series(vals_sel).dropna().unique())
+ if categorical_scale and categorical_scale.order is not None:
+ order = list(categorical_scale.order)
+ unlisted = [value for value in observed if value not in order]
+ if unlisted:
+ raise ValueError(
+ "categorical_scale.order is missing observed values: "
+ + ", ".join(map(str, unlisted[:10]))
+ )
+ else:
+ order = sorted(observed, key=lambda v: str(v))
+ palette = categorical_color_map(
+ order,
+ palette=categorical_scale.palette if categorical_scale else None,
+ missing_label=None,
+ )
+ categorical_maps[color_index] = (order, palette)
+ elif color_scale.scope != "panel":
+ limit_map[color_index] = (
+ shared_limits
+ if shared_limits is not None
+ else _continuous_limits(vals_sel, color_scale)
+ )
+
+ legends: list[LegendSpec] = []
+ scales_out: list[Any] = [color_scale]
+ for color_index, (order, palette) in categorical_maps.items():
+ label = labels[color_index]
+ scales_out.append(CategoricalScale(order=tuple(order), palette=dict(palette)))
+ legends.append(LegendSpec(kind="categorical", label=label))
+ for color_index, (_, label, is_categorical, is_uniform) in enumerate(color_cache):
+ if is_categorical or is_uniform:
+ continue
+ limits = limit_map.get(color_index)
+ legends.append(
+ LegendSpec(
+ kind="colorbar",
+ label=label,
+ extras=(
+ {"vmin": limits[0], "vmax": limits[1]}
+ if limits is not None
+ else {"scope": "panel"}
+ ),
+ )
+ )
+
+ rng = np.random.default_rng(seed) if seed is not None else None
+ panel_limit_map: dict[str, tuple[float, float]] = {}
+
+ with theme_context(theme):
+ panel_i = 0
+ for color_index, (vals, label, is_cat, is_uniform) in enumerate(color_cache):
+ for fac_i, fac in enumerate(facets):
+ panel_key = panel_keys[panel_i]
+ ax = axes[panel_key]
+ if facet_values is None:
+ mask = base_mask.copy()
+ elif pd.isna(fac):
+ mask = base_mask & pd.isna(facet_values)
+ else:
+ mask = base_mask & (facet_values == fac)
+
+ xx = x[mask]
+ yy = y[mask]
+ vv = np.asarray(vals)[mask]
+ ss = size_arr[mask]
+ rasterized = len(xx) >= rasterize_threshold
+
+ if mask.sum() == 0:
+ ax.set_axis_off()
+ ax.set_title(
+ f"{label} | {facet_by}={fac} (empty)"
+ if fac is not None
+ else f"{label} (empty)"
+ )
+ panel_i += 1
+ continue
+
+ if is_cat:
+ order, palette = categorical_maps[color_index]
+ _draw_categorical(
+ ax,
+ xx,
+ yy,
+ vv,
+ ss,
+ order=order,
+ palette=palette,
+ missing_color=missing,
+ rasterized=rasterized,
+ )
+ if fac_i == n_facets - 1:
+ _add_categorical_legend(
+ ax,
+ mpl,
+ order=order,
+ palette=palette,
+ label=label,
+ missing=bool(pd.isna(np.asarray(vals)[base_mask]).any()),
+ missing_color=missing,
+ missing_label=(
+ categorical_scale.missing_label
+ if categorical_scale is not None
+ else "NA"
+ ),
+ )
+ else:
+ vnum = pd.to_numeric(pd.Series(vv), errors="coerce").to_numpy(
+ dtype=np.float64
+ )
+ if is_uniform:
+ limits = (0.0, 1.0)
+ elif color_scale.scope == "panel":
+ limits = _continuous_limits(vnum, color_scale)
+ panel_limit_map[str(panel_key)] = limits
+ else:
+ limits = limit_map[color_index]
+ add_cb = (not is_uniform) and (
+ color_scale.scope == "panel"
+ or facet_by is None
+ or fac_i == n_facets - 1
+ )
+ _draw_continuous(
+ ax,
+ fig,
+ xx,
+ yy,
+ vnum,
+ ss,
+ limits=limits,
+ cmap_name=color_scale.cmap,
+ vcenter=color_scale.vcenter,
+ missing_color=color_scale.missing_color,
+ default_color=default_color,
+ label=label,
+ is_uniform=is_uniform,
+ sort_values=sort_values,
+ rng=rng,
+ add_colorbar=add_cb,
+ rasterized=rasterized,
+ plt=plt,
+ mpl=mpl,
+ )
+
+ ax.set_xlim(xlim)
+ ax.set_ylim(ylim)
+ ax.set_aspect("equal", adjustable="box")
+ ax.set_xticks([])
+ ax.set_yticks([])
+ if fac is None:
+ title = label
+ else:
+ title = (
+ f"{label} | {facet_by}={fac}" if label else f"{facet_by}={fac}"
+ )
+ if title:
+ ax.set_title(title)
+ ax.set_xlabel(f"{layout_key}1")
+ ax.set_ylabel(f"{layout_key}2")
+ panel_i += 1
+
+ if color_scale.scope == "panel":
+ color_limits = panel_limit_map
+ else:
+ color_limits = {
+ display_key(index, labels[index]): limits
+ for index, limits in limit_map.items()
+ }
+ feature_assays: set[str] = set()
+ for item in color_items:
+ if not (
+ isinstance(item, FeatureRef)
+ or (isinstance(item, str) and item not in store.cells.columns)
+ ):
+ continue
+ assay_name = (
+ item.assay
+ if isinstance(item, FeatureRef) and item.assay is not None
+ else from_assay or getattr(store, "_defaultAssay", None)
+ )
+ if assay_name is not None:
+ feature_assays.add(assay_name)
+
+ result = PlotResult(
+ figure=fig,
+ axes=axes,
+ tables={},
+ legends=tuple(legends),
+ scales=tuple(scales_out),
+ provenance=PlotProvenance(
+ scarf_version=_scarf_version(),
+ assay=next(iter(feature_assays)) if len(feature_assays) == 1 else None,
+ cell_key=cell_key,
+ n_cells=int(base_mask.sum()),
+ renderer="matplotlib",
+ notes=("embedding", "materialized", f"layout={layout_key}"),
+ extras={
+ "sort_values": sort_values,
+ "color_scale_scope": color_scale.scope,
+ "color_limits": color_limits,
+ "facet_by": facet_by,
+ "n_colors": n_colors,
+ "n_facets": n_facets,
+ "panel_keys": [str(k) for k in panel_keys],
+ "clip_fraction": clip_fraction,
+ "subset_by": subset_by,
+ "input_n_cells": n,
+ "invalid_coordinate_cells": int((~finite_coordinates).sum()),
+ "rasterize_threshold": rasterize_threshold,
+ "normalization": {
+ "source": normalization.source,
+ "transform": normalization.transform,
+ },
+ "assays": sorted(feature_assays),
+ },
+ ),
+ owns_figure=owns,
+ theme=theme,
+ )
+ if show:
+ result.show()
+ return result
diff --git a/scarf/plotting/embedding_raster.py b/scarf/plotting/embedding_raster.py
new file mode 100644
index 00000000..ef3c1a29
--- /dev/null
+++ b/scarf/plotting/embedding_raster.py
@@ -0,0 +1,204 @@
+"""Blockwise raster embedding for large cell counts."""
+
+from typing import Any, Hashable
+
+import numpy as np
+
+from ._contracts import CellField, ColorScale, PlotProvenance
+from ._deps import require_matplotlib
+from ._figure import LegendSpec, PlotResult, normalize_axes_target
+from ._raster import draw_raster_canvas, raster_from_metadata
+from ._style import theme_context
+
+
+def _scarf_version() -> str:
+ try:
+ from importlib.metadata import version
+
+ return version("scarf")
+ except Exception:
+ return "unknown"
+
+
+def _is_categorical_column(
+ cells: Any,
+ *,
+ key: str,
+ cell_key: str,
+ block_rows: int | None,
+ kind: str,
+ max_categories: int = 100,
+) -> bool:
+ if kind == "categorical":
+ return True
+ if kind == "continuous":
+ return False
+ dtype = cells.get_dtype(key)
+ dtype_kind = getattr(dtype, "kind", None)
+ if dtype_kind in ("b", "O", "S", "U", "T"):
+ return True
+ if dtype_kind not in ("i", "u"):
+ return False
+
+ categories: set[Any] = set()
+ for block in cells.iter_row_blocks(
+ cell_key=cell_key,
+ columns=[key],
+ block_rows=block_rows,
+ ):
+ categories.update(np.unique(block.values[key]).tolist())
+ if len(categories) > max_categories:
+ return False
+ return True
+
+
+def embedding_raster(
+ store: Any,
+ *,
+ layout_key: str,
+ color_by: str | CellField | None = None,
+ cell_key: str = "I",
+ pixels: int = 400,
+ block_rows: int | None = None,
+ color_scale: ColorScale | None = None,
+ missing_color: str = "#f0f0f0",
+ target: Any | None = None,
+ figsize: tuple[float, float] | None = None,
+ theme: str = "notebook",
+ seed: int = 0,
+ show: bool = False,
+) -> PlotResult:
+ """Rasterize a 2D layout from cell-metadata columns via row blocks.
+
+ ``color_by`` must be a continuous cell-metadata column (not a gene).
+ Use ``CellField(key, kind="continuous")`` to explicitly treat an integer
+ measurement as continuous. Categorical blockwise raster is not supported.
+ """
+ require_matplotlib()
+ color_scale = color_scale or ColorScale(cmap="viridis", quantiles=(0.01, 0.99))
+ x_key = f"{layout_key}1"
+ y_key = f"{layout_key}2"
+ for key in (x_key, y_key):
+ if key not in store.cells.columns:
+ raise KeyError(f"Layout column {key!r} not found in cell metadata")
+
+ color_key: str | None
+ color_label: str | None
+ if isinstance(color_by, CellField):
+ color_key = color_by.key
+ color_label = color_by.label or color_by.key
+ color_kind = color_by.kind
+ else:
+ color_key = color_by
+ color_label = color_by
+ color_kind = "auto"
+ if color_key is not None and color_key not in store.cells.columns:
+ raise KeyError(
+ f"color_by {color_key!r} must be a cell-metadata column for "
+ "embedding_raster (gene coloring uses embedding() for now)"
+ )
+ if color_key is not None and _is_categorical_column(
+ store.cells,
+ key=color_key,
+ cell_key=cell_key,
+ block_rows=block_rows,
+ kind=color_kind,
+ ):
+ raise NotImplementedError(
+ "embedding_raster supports continuous color values only; use "
+ "embedding() for categorical cell metadata"
+ )
+
+ quantiles = color_scale.quantiles
+ canvas = raster_from_metadata(
+ store.cells,
+ x_key=x_key,
+ y_key=y_key,
+ color_key=color_key,
+ cell_key=cell_key,
+ pixels=pixels,
+ block_rows=block_rows,
+ quantiles=quantiles,
+ seed=seed,
+ )
+ if color_scale.vmin is not None or color_scale.vmax is not None:
+ from dataclasses import replace
+
+ canvas = replace(
+ canvas,
+ vmin=(
+ float(color_scale.vmin) if color_scale.vmin is not None else canvas.vmin
+ ),
+ vmax=(
+ float(color_scale.vmax) if color_scale.vmax is not None else canvas.vmax
+ ),
+ )
+ if canvas.vmax <= canvas.vmin:
+ raise ValueError("Color limits must satisfy vmin < vmax")
+
+ panel_key: Hashable = color_label or layout_key
+ resolved_figsize = figsize
+ if resolved_figsize is None and target is None:
+ resolved_figsize = (5.0, 5.0)
+ fig, axes, owns = normalize_axes_target(
+ target,
+ panel_keys=[panel_key],
+ figsize=resolved_figsize,
+ )
+ ax = axes[panel_key]
+ with theme_context(theme):
+ im = draw_raster_canvas(
+ ax,
+ canvas,
+ cmap=color_scale.cmap or "viridis",
+ missing_color=missing_color,
+ vcenter=color_scale.vcenter,
+ )
+ ax.set_xlabel(f"{layout_key}1")
+ ax.set_ylabel(f"{layout_key}2")
+ cb = fig.colorbar(im, ax=ax, shrink=0.7, pad=0.02)
+ cb.set_label(color_label or "log1p cell count")
+ ax.set_aspect("equal")
+
+ result = PlotResult(
+ figure=fig,
+ axes=axes,
+ tables={},
+ legends=(
+ LegendSpec(
+ kind="colorbar",
+ label=color_label or "log1p cell count",
+ ),
+ ),
+ scales=(color_scale,),
+ provenance=PlotProvenance(
+ scarf_version=_scarf_version(),
+ cell_key=cell_key,
+ n_cells=canvas.n_cells,
+ renderer="matplotlib-raster",
+ notes=(
+ "embedding_raster",
+ "two_pass",
+ f"layout={layout_key}",
+ *(
+ ("approximate_quantiles",)
+ if color_key is not None and quantiles is not None
+ else ()
+ ),
+ ),
+ extras={
+ "pixels": pixels,
+ "block_rows": block_rows,
+ "n_blocks": canvas.n_blocks,
+ "vmin": canvas.vmin,
+ "vmax": canvas.vmax,
+ "color_by": color_key,
+ "color_mode": "continuous" if color_key is not None else "density",
+ },
+ ),
+ owns_figure=owns,
+ theme=theme,
+ )
+ if show:
+ result.show()
+ return result
diff --git a/scarf/plotting/heatmaps.py b/scarf/plotting/heatmaps.py
new file mode 100644
index 00000000..dce6d57b
--- /dev/null
+++ b/scarf/plotting/heatmaps.py
@@ -0,0 +1,146 @@
+"""Heatmap, tree, and pseudotime compatibility wrappers."""
+
+from typing import Any
+
+
+def marker_heatmap(
+ store: Any,
+ *,
+ from_assay: str | None = None,
+ group_key: str | None = None,
+ cell_key: str | None = None,
+ topn: int = 5,
+ log_transform: bool = True,
+ vmin: float = -1,
+ vmax: float = 2,
+ savename: str | None = None,
+ save_dpi: int = 300,
+ show_fig: bool = True,
+ **heatmap_kwargs: Any,
+) -> None:
+ """Facade for ``DataStore.plot_marker_heatmap``."""
+ store.plot_marker_heatmap(
+ from_assay=from_assay,
+ group_key=group_key,
+ cell_key=cell_key,
+ topn=topn,
+ log_transform=log_transform,
+ vmin=vmin,
+ vmax=vmax,
+ savename=savename,
+ save_dpi=save_dpi,
+ show_fig=show_fig,
+ **heatmap_kwargs,
+ )
+
+
+def cluster_tree(
+ store: Any,
+ *,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ cluster_key: str | None = None,
+ fill_by_value: str | None = None,
+ force_ints_as_cats: bool = True,
+ width: float = 1,
+ lvr_factor: float = 0.5,
+ vert_gap: float = 0.2,
+ min_node_size: float = 10,
+ node_size_multiplier: float = 10_000.0,
+ node_power: float = 1.2,
+ root_size: float = 100,
+ non_leaf_size: float = 10,
+ show_labels: bool = True,
+ fontsize: float = 10,
+ root_color: str = "#C0C0C0",
+ non_leaf_color: str = "k",
+ cmap: str = "tab20",
+ color_key: dict[str, str] | None = None,
+ edgecolors: str = "k",
+ edgewidth: float = 1,
+ alpha: float = 0.7,
+ figsize: tuple[float, float] = (5, 5),
+ ax: Any = None,
+ show_fig: bool = True,
+ savename: str | None = None,
+ save_dpi: int = 300,
+) -> None:
+ """Facade for ``DataStore.plot_cluster_tree``."""
+ store.plot_cluster_tree(
+ from_assay=from_assay,
+ cell_key=cell_key,
+ feat_key=feat_key,
+ cluster_key=cluster_key,
+ fill_by_value=fill_by_value,
+ force_ints_as_cats=force_ints_as_cats,
+ width=width,
+ lvr_factor=lvr_factor,
+ vert_gap=vert_gap,
+ min_node_size=min_node_size,
+ node_size_multiplier=node_size_multiplier,
+ node_power=node_power,
+ root_size=root_size,
+ non_leaf_size=non_leaf_size,
+ show_labels=show_labels,
+ fontsize=fontsize,
+ root_color=root_color,
+ non_leaf_color=non_leaf_color,
+ cmap=cmap,
+ color_key=color_key,
+ edgecolors=edgecolors,
+ edgewidth=edgewidth,
+ alpha=alpha,
+ figsize=figsize,
+ ax=ax,
+ show_fig=show_fig,
+ savename=savename,
+ save_dpi=save_dpi,
+ )
+
+
+def pseudotime_heatmap(
+ store: Any,
+ *,
+ from_assay: str | None = None,
+ cell_key: str | None = None,
+ feat_key: str | None = None,
+ feature_cluster_key: str | None = None,
+ pseudotime_key: str | None = None,
+ show_features: list[str] | None = None,
+ width: int = 5,
+ height: int = 10,
+ vmin: float = -2.0,
+ vmax: float = 2.0,
+ heatmap_cmap: str | None = None,
+ pseudotime_cmap: str | None = None,
+ clusterbar_cmap: str | None = None,
+ tick_fontsize: int = 10,
+ axis_fontsize: int = 12,
+ feature_label_fontsize: int = 12,
+ savename: str | None = None,
+ save_dpi: int = 300,
+ show_fig: bool = True,
+) -> None:
+ """Facade for ``DataStore.plot_pseudotime_heatmap``."""
+ store.plot_pseudotime_heatmap(
+ from_assay=from_assay,
+ cell_key=cell_key,
+ feat_key=feat_key,
+ feature_cluster_key=feature_cluster_key,
+ pseudotime_key=pseudotime_key,
+ show_features=show_features,
+ width=width,
+ height=height,
+ vmin=vmin,
+ vmax=vmax,
+ heatmap_cmap=heatmap_cmap,
+ pseudotime_cmap=pseudotime_cmap,
+ clusterbar_cmap=clusterbar_cmap,
+ tick_fontsize=tick_fontsize,
+ axis_fontsize=axis_fontsize,
+ feature_label_fontsize=feature_label_fontsize,
+ savename=savename,
+ save_dpi=save_dpi,
+ show_fig=show_fig,
+ )
diff --git a/scarf/plotting/summary.py b/scarf/plotting/summary.py
new file mode 100644
index 00000000..f55da679
--- /dev/null
+++ b/scarf/plotting/summary.py
@@ -0,0 +1,419 @@
+"""Dotplot and matrixplot."""
+
+from collections.abc import Mapping, Sequence
+from typing import Any, Hashable
+
+import numpy as np
+import pandas as pd
+
+from ._contracts import (
+ CategoricalScale,
+ ColorScale,
+ FeatureRef,
+ NormalizationSpec,
+ PlotProvenance,
+ SizeScale,
+ StudyDesign,
+)
+from ._data import coerce_feature_list, summarize_features_by_group
+from ._deps import require_matplotlib
+from ._figure import LegendSpec, PlotResult, normalize_axes_target
+from ._style import continuous_norm, theme_context
+
+
+def _standardize_feature(df: pd.DataFrame, value_col: str = "mean") -> pd.DataFrame:
+ out = df.copy()
+ means = out.groupby("feature", observed=False)[value_col].transform("mean")
+ stds = out.groupby("feature", observed=False)[value_col].transform("std")
+ stds = stds.replace(0, np.nan)
+ out[value_col] = (out[value_col] - means) / stds
+ return out
+
+
+def _group_axis_labels(df: pd.DataFrame, group_keys: tuple[str, ...]) -> pd.Series:
+ if len(group_keys) == 1:
+ return df[group_keys[0]].astype(str)
+ return df[list(group_keys)].astype(str).agg(" | ".join, axis=1)
+
+
+def _color_limits(values: np.ndarray, scale: ColorScale) -> tuple[float, float]:
+ finite = np.isfinite(values)
+ if finite.any():
+ if scale.quantiles is not None:
+ low, high = scale.quantiles
+ vmin = float(np.nanquantile(values[finite], low))
+ vmax = float(np.nanquantile(values[finite], high))
+ else:
+ vmin = float(np.nanmin(values[finite]))
+ vmax = float(np.nanmax(values[finite]))
+ else:
+ vmin, vmax = 0.0, 1.0
+ if scale.vmin is not None:
+ vmin = scale.vmin
+ if scale.vmax is not None:
+ vmax = scale.vmax
+ if vmax <= vmin:
+ if scale.vmin is not None or scale.vmax is not None:
+ raise ValueError("Color limits must satisfy vmin < vmax")
+ vmax = vmin + 1.0
+ return vmin, vmax
+
+
+def _sample_counts(
+ store: Any,
+ *,
+ cell_key: str,
+ sample_by: str | None,
+ study_design: StudyDesign | None,
+) -> tuple[int | None, int]:
+ sample_key = study_design.sample_by if study_design is not None else sample_by
+ if sample_key is None:
+ return None, 0
+ values = np.asarray(store.cells.fetch(sample_key, key=cell_key), dtype=object)
+ valid = pd.notna(values) & (values != "")
+ return int(pd.Series(values[valid]).nunique()), int((~valid).sum())
+
+
+def _feature_assays(
+ store: Any,
+ features: Sequence[str | FeatureRef] | Mapping[str, Sequence[str | FeatureRef]],
+ from_assay: str | None,
+) -> set[str]:
+ assays: set[str] = set()
+ for _, feature in coerce_feature_list(features):
+ assay = (
+ feature.assay
+ if isinstance(feature, FeatureRef) and feature.assay is not None
+ else from_assay or store._defaultAssay
+ )
+ assays.add(assay)
+ return assays
+
+
+def dotplot(
+ store: Any,
+ *,
+ features: Sequence[str | FeatureRef] | Mapping[str, Sequence[str | FeatureRef]],
+ group_by: str | tuple[str, ...],
+ cell_key: str = "I",
+ from_assay: str | None = None,
+ sample_by: str | None = None,
+ study_design: StudyDesign | None = None,
+ normalization: NormalizationSpec | None = None,
+ expression_cutoff: float = 0.0,
+ standardize: str = "none",
+ color_scale: ColorScale | None = None,
+ size_scale: SizeScale | None = None,
+ target: Any | None = None,
+ figsize: tuple[float, float] | None = None,
+ theme: str = "notebook",
+ show: bool = False,
+) -> PlotResult:
+ """Mean expression (color) and fraction expressing (size) by group."""
+ _, mpl = require_matplotlib()
+ color_scale = color_scale or ColorScale(cmap="viridis")
+ size_scale = size_scale or SizeScale()
+ normalization = normalization or NormalizationSpec()
+
+ aggregate, per_sample = summarize_features_by_group(
+ store,
+ features=features,
+ group_by=group_by,
+ cell_key=cell_key,
+ from_assay=from_assay,
+ sample_by=sample_by,
+ study_design=study_design,
+ normalization=normalization,
+ expression_cutoff=expression_cutoff,
+ )
+ if standardize == "feature":
+ aggregate = _standardize_feature(aggregate, "mean")
+ elif standardize not in ("none", "feature"):
+ raise ValueError("standardize must be 'none' or 'feature'")
+
+ group_keys = (group_by,) if isinstance(group_by, str) else tuple(group_by)
+ plot_df = aggregate.copy()
+ plot_df["group_label"] = _group_axis_labels(plot_df, group_keys)
+ # Preserve feature order from input
+ feature_order = list(dict.fromkeys(plot_df["feature"].tolist()))
+ group_order = list(dict.fromkeys(plot_df["group_label"].tolist()))
+ plot_df["feature"] = pd.Categorical(
+ plot_df["feature"], categories=feature_order, ordered=True
+ )
+ plot_df["group_label"] = pd.Categorical(
+ plot_df["group_label"], categories=group_order, ordered=True
+ )
+
+ panel_key: Hashable = "dotplot"
+ resolved_figsize = figsize
+ if resolved_figsize is None and target is None:
+ resolved_figsize = (
+ max(4.0, 0.35 * len(group_order) + 2),
+ max(3.0, 0.3 * len(feature_order) + 1.5),
+ )
+ fig, axes, owns = normalize_axes_target(
+ target,
+ panel_keys=[panel_key],
+ figsize=resolved_figsize,
+ )
+ ax = axes[panel_key]
+
+ vals = plot_df["mean"].to_numpy(dtype=np.float64)
+ vmin, vmax = _color_limits(vals, color_scale)
+ norm = continuous_norm(
+ mpl,
+ vmin=vmin,
+ vmax=vmax,
+ vcenter=color_scale.vcenter,
+ )
+
+ areas = size_scale.areas(plot_df["fraction"].to_numpy(dtype=np.float64))
+ x = plot_df["group_label"].cat.codes.to_numpy()
+ y = plot_df["feature"].cat.codes.to_numpy()
+
+ with theme_context(theme):
+ sc = ax.scatter(
+ x,
+ y,
+ c=vals,
+ s=areas,
+ cmap=color_scale.cmap or "viridis",
+ norm=norm,
+ edgecolors="k",
+ linewidths=0.2,
+ rasterized=False,
+ )
+ ax.set_xticks(range(len(group_order)))
+ ax.set_xticklabels(group_order, rotation=45, ha="right")
+ ax.set_yticks(range(len(feature_order)))
+ ax.set_yticklabels(feature_order)
+ ax.set_xlim(-0.5, len(group_order) - 0.5)
+ ax.set_ylim(-0.5, len(feature_order) - 0.5)
+ ax.invert_yaxis()
+ cb = fig.colorbar(sc, ax=ax, shrink=0.6, pad=0.02)
+ cb.set_label("mean" if standardize == "none" else "standardized mean")
+ legend_values = np.array([0.25, 0.5, 0.75, 1.0])
+ legend_areas = size_scale.areas(legend_values)
+ handles = [
+ ax.scatter(
+ [],
+ [],
+ s=area,
+ facecolor="#bdbdbd",
+ edgecolor="k",
+ linewidth=0.2,
+ )
+ for area in legend_areas
+ ]
+ ax.legend(
+ handles,
+ [f"{value:g}" for value in legend_values],
+ title="fraction",
+ frameon=False,
+ bbox_to_anchor=(1.02, 0),
+ loc="lower left",
+ borderaxespad=0,
+ )
+ ax.set_xlabel(" / ".join(group_keys))
+ ax.set_ylabel("feature")
+
+ tables = {"aggregate": aggregate}
+ if per_sample is not None:
+ tables["per_sample"] = per_sample
+ n_samples, dropped_sample_cells = _sample_counts(
+ store,
+ cell_key=cell_key,
+ sample_by=sample_by,
+ study_design=study_design,
+ )
+ assays = _feature_assays(store, features, from_assay)
+
+ from importlib.metadata import version
+
+ try:
+ scarf_version = version("scarf")
+ except Exception:
+ scarf_version = "unknown"
+
+ result = PlotResult(
+ figure=fig,
+ axes=axes,
+ tables=tables,
+ legends=(
+ LegendSpec(kind="colorbar", label="mean"),
+ LegendSpec(kind="size", label="fraction", extras={"domain": [0.0, 1.0]}),
+ ),
+ scales=(color_scale, size_scale, CategoricalScale(order=tuple(group_order))),
+ provenance=PlotProvenance(
+ scarf_version=scarf_version,
+ assay=next(iter(assays)) if len(assays) == 1 else None,
+ cell_key=cell_key,
+ n_cells=len(store.cells.active_index(cell_key)),
+ n_samples=n_samples,
+ renderer="matplotlib",
+ notes=("dotplot",),
+ extras={
+ "group_by": list(group_keys),
+ "sample_by": (
+ study_design.sample_by if study_design is not None else sample_by
+ ),
+ "expression_cutoff": expression_cutoff,
+ "dropped_sample_cells": dropped_sample_cells,
+ "normalization": {
+ "source": normalization.source,
+ "transform": normalization.transform,
+ },
+ "assays": sorted(assays),
+ },
+ ),
+ owns_figure=owns,
+ theme=theme,
+ )
+ if show:
+ result.show()
+ return result
+
+
+def matrixplot(
+ store: Any,
+ *,
+ features: Sequence[str | FeatureRef] | Mapping[str, Sequence[str | FeatureRef]],
+ group_by: str | tuple[str, ...],
+ cell_key: str = "I",
+ from_assay: str | None = None,
+ sample_by: str | None = None,
+ study_design: StudyDesign | None = None,
+ normalization: NormalizationSpec | None = None,
+ expression_cutoff: float = 0.0,
+ value: str = "mean",
+ standardize: str = "none",
+ color_scale: ColorScale | None = None,
+ target: Any | None = None,
+ figsize: tuple[float, float] | None = None,
+ theme: str = "notebook",
+ show: bool = False,
+) -> PlotResult:
+ """Feature-by-group matrix of mean or fraction (no forced clustering)."""
+ _, mpl = require_matplotlib()
+ if value not in ("mean", "fraction"):
+ raise ValueError("value must be 'mean' or 'fraction'")
+ color_scale = color_scale or ColorScale(cmap="viridis")
+ normalization = normalization or NormalizationSpec()
+
+ aggregate, per_sample = summarize_features_by_group(
+ store,
+ features=features,
+ group_by=group_by,
+ cell_key=cell_key,
+ from_assay=from_assay,
+ sample_by=sample_by,
+ study_design=study_design,
+ normalization=normalization,
+ expression_cutoff=expression_cutoff,
+ )
+ plot_df = aggregate.copy()
+ if value == "mean" and standardize == "feature":
+ plot_df = _standardize_feature(plot_df, "mean")
+ elif standardize not in ("none", "feature"):
+ raise ValueError("standardize must be 'none' or 'feature'")
+
+ group_keys = (group_by,) if isinstance(group_by, str) else tuple(group_by)
+ plot_df["group_label"] = _group_axis_labels(plot_df, group_keys)
+ feature_order = list(dict.fromkeys(plot_df["feature"].tolist()))
+ group_order = list(dict.fromkeys(plot_df["group_label"].tolist()))
+ mat = plot_df.pivot_table(
+ index="feature",
+ columns="group_label",
+ values=value,
+ observed=False,
+ ).reindex(index=feature_order, columns=group_order)
+
+ panel_key: Hashable = "matrixplot"
+ resolved_figsize = figsize
+ if resolved_figsize is None and target is None:
+ resolved_figsize = (
+ max(4.0, 0.4 * len(group_order) + 2),
+ max(3.0, 0.35 * len(feature_order) + 1.5),
+ )
+ fig, axes, owns = normalize_axes_target(
+ target,
+ panel_keys=[panel_key],
+ figsize=resolved_figsize,
+ )
+ ax = axes[panel_key]
+ data = mat.to_numpy(dtype=np.float64)
+ vmin, vmax = _color_limits(data, color_scale)
+ norm = continuous_norm(
+ mpl,
+ vmin=vmin,
+ vmax=vmax,
+ vcenter=color_scale.vcenter,
+ )
+
+ with theme_context(theme):
+ im = ax.imshow(
+ data,
+ aspect="auto",
+ cmap=color_scale.cmap or "viridis",
+ norm=norm,
+ interpolation="nearest",
+ )
+ ax.set_xticks(range(len(group_order)))
+ ax.set_xticklabels(group_order, rotation=45, ha="right")
+ ax.set_yticks(range(len(feature_order)))
+ ax.set_yticklabels(feature_order)
+ cb = fig.colorbar(im, ax=ax, shrink=0.6, pad=0.02)
+ cb.set_label(value)
+
+ tables = {"aggregate": aggregate, "matrix": mat.reset_index()}
+ if per_sample is not None:
+ tables["per_sample"] = per_sample
+ n_samples, dropped_sample_cells = _sample_counts(
+ store,
+ cell_key=cell_key,
+ sample_by=sample_by,
+ study_design=study_design,
+ )
+ assays = _feature_assays(store, features, from_assay)
+
+ from importlib.metadata import version
+
+ try:
+ scarf_version = version("scarf")
+ except Exception:
+ scarf_version = "unknown"
+
+ result = PlotResult(
+ figure=fig,
+ axes=axes,
+ tables=tables,
+ legends=(LegendSpec(kind="colorbar", label=value),),
+ scales=(color_scale,),
+ provenance=PlotProvenance(
+ scarf_version=scarf_version,
+ assay=next(iter(assays)) if len(assays) == 1 else None,
+ cell_key=cell_key,
+ n_cells=len(store.cells.active_index(cell_key)),
+ n_samples=n_samples,
+ renderer="matplotlib",
+ notes=("matrixplot", value),
+ extras={
+ "group_by": list(group_keys),
+ "sample_by": (
+ study_design.sample_by if study_design is not None else sample_by
+ ),
+ "expression_cutoff": expression_cutoff,
+ "dropped_sample_cells": dropped_sample_cells,
+ "normalization": {
+ "source": normalization.source,
+ "transform": normalization.transform,
+ },
+ "assays": sorted(assays),
+ },
+ ),
+ owns_figure=owns,
+ theme=theme,
+ )
+ if show:
+ result.show()
+ return result
diff --git a/scarf/readers.py b/scarf/readers.py
index e1618c7f..6b50e1ff 100644
--- a/scarf/readers.py
+++ b/scarf/readers.py
@@ -12,12 +12,13 @@
import math
import os
from abc import ABC, abstractmethod
-from typing import IO, Dict, Generator, List, Optional, Tuple, Union
+from typing import IO, Any
+from collections.abc import Generator
import h5py
import numpy as np
import pandas as pd
-import polars as pl
+from numpy.typing import DTypeLike
from scipy.sparse import coo_matrix
from .utils import logger, tqdmbar
@@ -50,15 +51,18 @@ def get_file_handle(fn: str) -> IO:
raise FileNotFoundError("ERROR: FILE NOT FOUND: %s" % fn)
-def read_file(fn: str):
+def read_file(fn: str) -> Generator[str, None, None]:
"""Yields the lines from the file the given file name points to.
Args:
fn: The path to the file (file name).
"""
fh = get_file_handle(fn)
- for line in fh:
- yield line.rstrip()
+ try:
+ for line in fh:
+ yield line.rstrip()
+ finally:
+ fh.close()
class CrReader(ABC):
@@ -75,8 +79,8 @@ class CrReader(ABC):
assayFeats: A DataFrame with information about the features in the assay.
"""
- def __init__(self, grp_names):
- self.autoNames = {
+ def __init__(self, grp_names: dict[str, Any]) -> None:
+ self.autoNames: dict[str, str] = {
"Gene Expression": "RNA",
"Peaks": "ATAC",
"Antibody Capture": "ADT",
@@ -84,32 +88,42 @@ def __init__(self, grp_names):
"ADT": "ADT",
"HTO": "HTO",
}
- self.grpNames: Dict = grp_names
+ self.grpNames: dict[str, Any] = grp_names
self.nFeatures: int = len(self.feature_names())
self.nCells: int = len(self.cell_names())
self.assayFeats = self._make_feat_table()
self._auto_rename_assay_names()
@abstractmethod
- def _handle_version(self):
+ def _handle_version(self) -> dict[str, Any]:
pass
@abstractmethod
- def _read_dataset(self, key: Optional[str] = None) -> List:
+ def _read_dataset(self, key: str | None = None) -> list[Any] | None:
pass
@abstractmethod
- def consume(self, batch_size: int, lines_in_mem: int):
- """Returns a generator that yield chunks of data."""
+ def consume(
+ self, batch_size: int, lines_in_mem: int
+ ) -> Generator[coo_matrix, None, None]:
+ """Yield CSR matrix chunks of cell rows.
+
+ Args:
+ batch_size: Number of cells per yielded chunk.
+ lines_in_mem: MTX lines buffered in memory (CrDirReader only).
+
+ Yields:
+ scipy.sparse.coo_matrix chunks.
+ """
pass
- def _subset_by_assay(self, v, assay) -> List:
+ def _subset_by_assay(self, v: list[Any], assay: str | None) -> list[Any]:
if assay is None:
return v
elif assay not in self.assayFeats:
raise ValueError(f"ERROR: Assay ID {assay} is not valid")
if len(self.assayFeats[assay].shape) == 2:
- ret_val = []
+ ret_val: list[Any] = []
for i in self.assayFeats[assay].values[1:3].T:
ret_val.extend(list(v[i[0] : i[1]]))
return ret_val
@@ -123,7 +137,7 @@ def _subset_by_assay(self, v, assay) -> List:
def _make_feat_table(self) -> pd.DataFrame:
s = self.feature_types()
- span: List[Tuple] = []
+ span: list[tuple] = []
last = s[0]
last_n: int = 0
for n, i in enumerate(s[1:], 1):
@@ -138,7 +152,7 @@ def _make_feat_table(self) -> pd.DataFrame:
df["nFeatures"] = df.end - df.start
return df.T
- def _auto_rename_assay_names(self):
+ def _auto_rename_assay_names(self) -> None:
new_names = []
for k, v in self.assayFeats.T["type"].to_dict().items():
if v in self.autoNames:
@@ -147,7 +161,7 @@ def _auto_rename_assay_names(self):
new_names.append(k)
self.assayFeats.columns = new_names
- def rename_assays(self, name_map: Dict[str, str]) -> None:
+ def rename_assays(self, name_map: dict[str, str]) -> None:
"""Renames specified assays in the Reader.
Args:
@@ -155,15 +169,18 @@ def rename_assays(self, name_map: Dict[str, str]) -> None:
"""
self.assayFeats.rename(columns=name_map, inplace=True)
- def feature_ids(self, assay: str = None) -> List[str]:
+ def feature_ids(self, assay: str | None = None) -> list[str]:
"""Returns a list of feature IDs in a specified assay.
Args:
assay: Select which assay to retrieve feature IDs from.
"""
- return self._subset_by_assay(self._read_dataset("feature_ids"), assay)
+ vals = self._read_dataset("feature_ids")
+ if vals is None:
+ return []
+ return self._subset_by_assay(vals, assay)
- def feature_names(self, assay: str = None) -> List[str]:
+ def feature_names(self, assay: str | None = None) -> list[str]:
"""Returns a list of features in the dataset.
Args:
@@ -173,9 +190,11 @@ def feature_names(self, assay: str = None) -> List[str]:
if vals is None:
logger.warning("Feature names extraction failed using feature IDs")
vals = self._read_dataset("feature_ids")
+ if vals is None:
+ return []
return self._subset_by_assay(vals, assay)
- def feature_types(self) -> List[str]:
+ def feature_types(self) -> list[str]:
"""Returns a list of feature types in the dataset."""
if self.grpNames["feature_types"] is not None:
ret_val = self._read_dataset("feature_types")
@@ -184,9 +203,12 @@ def feature_types(self) -> List[str]:
default_name = list(self.autoNames.keys())[0]
return [default_name for _ in range(self.nFeatures)]
- def cell_names(self) -> List[str]:
+ def cell_names(self) -> list[str]:
"""Returns a list of names of the cells in the dataset."""
- return self._read_dataset("cell_names")
+ vals = self._read_dataset("cell_names")
+ if vals is None:
+ return []
+ return vals
class CrH5Reader(CrReader):
@@ -208,10 +230,15 @@ class CrH5Reader(CrReader):
grp: Current active group in the hierarchy.
"""
- def __init__(self, h5_fn, is_filtered: bool = True, filtering_cutoff: int = 500):
- self.h5obj = h5py.File(h5_fn, mode="r")
- self.grp = None
- self.validBarcodeIdx = None
+ def __init__(
+ self,
+ h5_fn: str,
+ is_filtered: bool = True,
+ filtering_cutoff: int = 500,
+ ) -> None:
+ self.h5obj: h5py.File = h5py.File(h5_fn, mode="r")
+ self.grp: h5py.Group
+ self.validBarcodeIdx: np.ndarray | None = None
super().__init__(self._handle_version())
if is_filtered:
self.validBarcodeIdx = np.array(range(self.nCells))
@@ -219,11 +246,11 @@ def __init__(self, h5_fn, is_filtered: bool = True, filtering_cutoff: int = 500)
self.validBarcodeIdx = self._get_valid_barcodes(filtering_cutoff)
self.nCells = len(self.validBarcodeIdx)
- def _handle_version(self):
+ def _handle_version(self) -> dict[str, str | None]:
root_key = list(self.h5obj.keys())[0]
self.grp = self.h5obj[root_key]
if root_key == "matrix":
- grps = {
+ grps: dict[str, str | None] = {
"feature_ids": "features/id",
"feature_names": "features/name",
"feature_types": "features/feature_type",
@@ -261,10 +288,13 @@ def _get_valid_barcodes(
assert len(indptr) == (s + len(idx))
return np.where(np.hstack(valid_idx))[0]
- def _read_dataset(self, key: Optional[str] = None):
- return [x.decode("UTF-8") for x in self.grp[self.grpNames[key]][:]]
+ def _read_dataset(self, key: str | None = None) -> list[str]:
+ if key is None:
+ raise ValueError("Dataset key must be provided")
+ grp_key = self.grpNames[key]
+ return [x.decode("UTF-8") for x in self.grp[grp_key][:]]
- def cell_names(self) -> List[str]:
+ def cell_names(self) -> list[str]:
"""Returns a list of names of the cells in the dataset."""
vals = np.array(self._read_dataset("cell_names"))
if self.validBarcodeIdx is not None:
@@ -273,18 +303,34 @@ def cell_names(self) -> List[str]:
# noinspection DuplicatedCode
def consume(
- self, batch_size: int, lines_in_mem: int = None
+ self, batch_size: int, lines_in_mem: int | None = None
) -> Generator[coo_matrix, None, None]:
+ """Yield CSR chunks from the Cell Ranger H5 matrix.
+
+ Args:
+ batch_size: Number of cells per chunk.
+ lines_in_mem: Unused; kept for CrReader API compatibility.
+ """
+ valid_idx = self.validBarcodeIdx
+ assert valid_idx is not None
indptr = self.grp["indptr"][:]
- for s in range(0, len(self.validBarcodeIdx), batch_size):
- v_pos = self.validBarcodeIdx[s : s + batch_size]
- idx = [np.arange(x, y) for x, y in zip(indptr[v_pos], indptr[v_pos + 1])]
- cell_idx = np.repeat(np.arange(len(idx)), [len(x) for x in idx])
- idx = np.hstack(idx)
- data = self.grp["data"][idx[0] : idx[-1] + 1]
- data = data[idx - idx[0]]
- indices = self.grp["indices"][idx[0] : idx[-1] + 1]
- indices = indices[idx - idx[0]]
+ for s in range(0, len(valid_idx), batch_size):
+ v_pos = valid_idx[s : s + batch_size]
+ row_ranges = [
+ np.arange(x, y) for x, y in zip(indptr[v_pos], indptr[v_pos + 1])
+ ]
+ cell_idx = np.repeat(
+ np.arange(len(row_ranges)), [len(x) for x in row_ranges]
+ )
+ idx = np.hstack(row_ranges) if row_ranges else np.array([], dtype=np.int64)
+ if idx.size == 0:
+ yield coo_matrix(
+ ([], ([], [])),
+ shape=(len(v_pos), self.nFeatures),
+ )
+ continue
+ data = np.asarray(self.grp["data"][idx])
+ indices = np.asarray(self.grp["indices"][idx])
yield coo_matrix(
(data, (cell_idx, indices)), shape=(len(v_pos), self.nFeatures)
)
@@ -313,17 +359,17 @@ class CrDirReader(CrReader):
def __init__(
self,
- loc,
+ loc: str,
mtx_separator: str = " ",
index_offset: int = -1,
is_filtered: bool = True,
filtering_cutoff: int = 500,
- ):
+ ) -> None:
self.loc: str = loc.rstrip("/") + "/"
- self.matFn = None
+ self.matFn: str
self.sep = mtx_separator
self.indexOffset = index_offset
- self.validBarcodeIdx = None
+ self.validBarcodeIdx: np.ndarray | None = None
super().__init__(self._handle_version())
if is_filtered:
self.validBarcodeIdx = np.array(range(self.nCells))
@@ -332,12 +378,15 @@ def __init__(
self.validBarcodeIdx = self._get_valid_barcodes(filtering_cutoff)
self.nCells = len(self.validBarcodeIdx)
- def _handle_version(self):
+ def _handle_version(self) -> dict[str, tuple[str, int] | None]:
show_error = False
+ mat_fn: str | None = None
+ feat_fn: str | None = None
+ cell_fn: str | None = None
if os.path.isfile(self.loc + "matrix.mtx.gz"):
- self.matFn = self.loc + "matrix.mtx.gz"
+ mat_fn = self.loc + "matrix.mtx.gz"
elif os.path.isfile(self.loc + "matrix.mtx"):
- self.matFn = self.loc + "matrix.mtx"
+ mat_fn = self.loc + "matrix.mtx"
else:
show_error = True
if os.path.isfile(self.loc + "features.tsv.gz"):
@@ -362,14 +411,15 @@ def _handle_version(self):
else:
cell_fn = None
show_error = True
- if show_error:
- raise IOError(
+ if show_error or mat_fn is None or feat_fn is None or cell_fn is None:
+ raise OSError(
"ERROR: Couldn't find either of these expected combinations of files:\n"
"\t- matrix.mtx, barcodes.tsv and genes.tsv\n"
"\t- matrix.mtx.gz, barcodes.tsv.gz and features.tsv.gz\n"
"Please make sure that you have not compressed or uncompressed the Cellranger output files "
"manually"
)
+ self.matFn = mat_fn
return {
"feature_ids": (feat_fn, 0),
"feature_names": (feat_fn, 1),
@@ -377,16 +427,17 @@ def _handle_version(self):
"cell_names": (cell_fn, 0),
}
- def _read_dataset(self, key: Optional[str] = None):
+ def _read_dataset(self, key: str | None = None) -> list[str] | None:
+ if key is None:
+ raise ValueError("Dataset key must be provided")
+ grp_entry = self.grpNames[key]
try:
vals = [
- x.split("\t")[self.grpNames[key][1]]
- for x in read_file(self.loc + self.grpNames[key][0])
+ x.split("\t")[grp_entry[1]] for x in read_file(self.loc + grp_entry[0])
]
except IndexError:
logger.warning(
- f"{key} extraction failed from {self.grpNames[key][0]} "
- f"in column {self.grpNames[key][1]}",
+ f"{key} extraction failed from {grp_entry[0]} in column {grp_entry[1]}",
flush=True,
)
vals = None
@@ -415,18 +466,19 @@ def read_header(self) -> pd.DataFrame:
)
return header
- def process_batch(self, dfs: List[pd.DataFrame], filtering_cutoff: int) -> np.array:
+ def process_batch(
+ self, dfs: list[pd.DataFrame], filtering_cutoff: int
+ ) -> np.ndarray:
"""Returns a list of valid barcodes after filtering out background barcodes for a given batch.
Args:
dfs: A Polar DataFrame containing a chunk of data from the MTX file.
filtering_cutoff: The cutoff value for filtering out background barcodes
"""
- pl_dfs = [pl.DataFrame(df) for df in dfs]
- pl_dfs = pl.concat(pl_dfs)
- dfs_ = pl_dfs.group_by("barcode").agg(pl.sum("count"))
- dfs_ = dfs_.filter(pl.col("count") > filtering_cutoff)
- return np.sort(dfs_["barcode"])
+ merged = pd.concat(dfs, ignore_index=True)
+ summed = merged.groupby("barcode")["count"].sum()
+ valid = summed[summed > filtering_cutoff].index.to_numpy()
+ return np.sort(valid)
def _get_valid_barcodes(
self,
@@ -497,7 +549,7 @@ def _get_valid_barcodes(
assert test_counter == header["nCounts"][0]
return np.sort(np.unique(np.hstack(valid_idx)))
- def to_sparse(self, a: np.ndarray, dtype) -> coo_matrix:
+ def to_sparse(self, a: np.ndarray, dtype: DTypeLike) -> coo_matrix:
"""Returns the input data as a sparse (COO) matrix.
Args:
@@ -517,22 +569,22 @@ def to_sparse(self, a: np.ndarray, dtype) -> coo_matrix:
dtype=dtype,
)
- def cell_names(self) -> List[str]:
+ def cell_names(self) -> list[str]:
"""Returns a list of names of the cells in the dataset."""
vals = np.array(self._read_dataset("cell_names"))
if self.validBarcodeIdx is not None:
vals = vals[(self.validBarcodeIdx + self.indexOffset)]
return list(vals)
- def rename_batches(self, collect: List[pd.DataFrame]) -> List:
- collect = [pl.DataFrame(df) for df in collect]
- df = pl.concat(collect)
- barcodes = np.array(df["barcode"])
+ def rename_batches(self, collect: list[pd.DataFrame]) -> np.ndarray:
+ df = pd.concat(collect, ignore_index=True)
+ barcodes = df["barcode"].to_numpy()
count_hash = {}
for i, x in enumerate(np.unique(barcodes)):
count_hash[x] = i
cell_idx = np.array([count_hash[x] for x in barcodes])
- df = df.with_columns([pl.Series("barcode", cell_idx)])
+ df = df.copy()
+ df["barcode"] = cell_idx
return np.array(df)
# noinspection DuplicatedCode
@@ -540,7 +592,7 @@ def consume(
self,
batch_size: int,
lines_in_mem: int = int(1e5),
- dtype=np.uint32,
+ dtype: DTypeLike = np.uint32,
) -> Generator[coo_matrix, None, None]:
"""Yields chunks of data from the MTX file.
@@ -557,12 +609,12 @@ def consume(
chunksize=lines_in_mem,
names=["gene", "barcode", "count"],
)
- unique_list = []
- collect = []
+ unique_list: list[Any] = []
+ collect: list[pd.DataFrame] = []
for chunk in matrixIO:
chunk = chunk[chunk["barcode"].isin(self.validBarcodeIdx)]
in_uniques = np.unique(chunk["barcode"].values)
- unique_list.extend(in_uniques)
+ unique_list.extend(in_uniques.tolist())
unique_list = list(set(unique_list))
if len(unique_list) > batch_size:
diff = batch_size - (len(unique_list) - len(in_uniques))
@@ -570,18 +622,17 @@ def consume(
mask_neg = in_uniques[diff:]
extra = chunk[chunk["barcode"].isin(mask_pos)]
collect.append(extra)
- collect = self.rename_batches(collect)
- mtx = self.to_sparse(np.array(collect), dtype=dtype)
+ batch_arr = self.rename_batches(collect)
+ mtx = self.to_sparse(batch_arr, dtype=dtype)
yield mtx
left_out = chunk[chunk["barcode"].isin(mask_neg)]
- collect = []
- unique_list = list(mask_neg)
- collect.append(left_out)
+ collect = [left_out]
+ unique_list = mask_neg.tolist()
else:
collect.append(chunk)
if len(collect) > 0:
- collect = self.rename_batches(collect)
- mtx = self.to_sparse(np.array(collect), dtype=dtype)
+ batch_arr = self.rename_batches(collect)
+ mtx = self.to_sparse(batch_arr, dtype=dtype)
yield mtx
@@ -629,21 +680,22 @@ def __init__(
matrix_key: str = "X",
obsm_attrs_key: str = "obsm",
category_names_key: str = "__categories",
- dtype: str = None,
- ):
- self.h5 = h5py.File(h5ad_fn, mode="r")
+ dtype: str | None = None,
+ ) -> None:
+ self.h5: h5py.File = h5py.File(h5ad_fn, mode="r")
self.matrixKey = matrix_key
self.cellAttrsKey, self.featureAttrsKey, self.obsmAttrsKey = (
cell_attrs_key,
feature_attrs_key,
obsm_attrs_key,
)
- self.groupCodes = {
+ self.groupCodes: dict[str, int] = {
self.cellAttrsKey: self._validate_group(self.cellAttrsKey),
self.featureAttrsKey: self._validate_group(self.featureAttrsKey),
self.obsmAttrsKey: self._validate_group(self.obsmAttrsKey),
self.matrixKey: self._validate_group(self.matrixKey),
}
+ self._validate_sparse_matrix()
self.nCells, self.nFeatures = (
self._get_n(self.cellAttrsKey),
self._get_n(self.featureAttrsKey),
@@ -652,7 +704,38 @@ def __init__(
self.featIdsKey = self._fix_name_key(self.featureAttrsKey, feature_ids_key)
self.featNamesKey = feature_name_key
self.catNamesKey = category_names_key
- self.matrixDtype = self._get_matrix_dtype() if dtype is None else dtype
+ self.matrixDtype: Any = self._get_matrix_dtype() if dtype is None else dtype
+
+ def _validate_sparse_matrix(self) -> None:
+ if self.groupCodes[self.matrixKey] != 2:
+ return
+
+ group = self.h5[self.matrixKey]
+ if not isinstance(group, h5py.Group):
+ return
+
+ required = {"data", "indices", "indptr"}
+ missing = required.difference(group.keys())
+ if missing:
+ raise ValueError(
+ f"ERROR: Sparse matrix group `{self.matrixKey}` is missing: "
+ f"{', '.join(sorted(missing))}"
+ )
+
+ encoding = group.attrs.get("encoding-type")
+ if isinstance(encoding, bytes):
+ encoding = encoding.decode("utf-8")
+ if encoding is None:
+ logger.warning(
+ f"Sparse matrix group `{self.matrixKey}` has no `encoding-type`; "
+ "assuming legacy CSR encoding"
+ )
+ return
+ if encoding != "csr_matrix":
+ raise ValueError(
+ f"ERROR: Sparse matrix encoding `{encoding}` is not supported. "
+ "H5adReader currently requires CSR encoding."
+ )
def _validate_group(self, group: str) -> int:
if group not in self.h5:
@@ -690,7 +773,7 @@ def _validate_group(self, group: str) -> int:
)
return ret_val
- def _get_matrix_dtype(self):
+ def _get_matrix_dtype(self) -> Any:
if self.groupCodes[self.matrixKey] == 1:
return self.h5[self.matrixKey].dtype
elif self.groupCodes[self.matrixKey] == 2:
@@ -725,18 +808,18 @@ def _fix_name_key(self, group: str, key: str) -> str:
def _get_n(self, group: str) -> int:
if self.groupCodes[group] == 0:
if self._check_exists(self.matrixKey, "shape"):
- return self.h5[self.matrixKey]["shape"][0]
+ return int(self.h5[self.matrixKey]["shape"][0])
else:
raise KeyError(
f"ERROR: `{group}` not found and `shape` key is missing in the {self.matrixKey} group. "
f"Aborting read process."
)
elif self.groupCodes[group] == 1:
- return self.h5[group].shape[0]
+ return int(self.h5[group].shape[0])
else:
for i in self.h5[group].keys():
if isinstance(self.h5[group][i], h5py.Dataset):
- return self.h5[group][i].shape[0]
+ return int(self.h5[group][i].shape[0])
raise KeyError(
f"ERROR: `{group}` key doesn't contain any child node of Dataset type."
f"Aborting because unexpected H5ad format."
@@ -746,9 +829,9 @@ def cell_ids(self) -> np.ndarray:
"""Returns a list of cell IDs."""
if self._check_exists(self.cellAttrsKey, self.cellIdsKey):
if self.groupCodes[self.cellAttrsKey] == 1:
- return self.h5[self.cellAttrsKey][self.cellIdsKey]
+ return np.asarray(self.h5[self.cellAttrsKey][self.cellIdsKey])
else:
- return self.h5[self.cellAttrsKey][self.cellIdsKey][:]
+ return np.asarray(self.h5[self.cellAttrsKey][self.cellIdsKey][:])
logger.warning(f"Could not find cells ids key: {self.cellIdsKey} in `obs`.")
return np.array([f"cell_{x}" for x in range(self.nCells)])
@@ -757,9 +840,9 @@ def feat_ids(self) -> np.ndarray:
"""Returns a list of feature IDs."""
if self._check_exists(self.featureAttrsKey, self.featIdsKey):
if self.groupCodes[self.featureAttrsKey] == 1:
- return self.h5[self.featureAttrsKey][self.featIdsKey]
+ return np.asarray(self.h5[self.featureAttrsKey][self.featIdsKey])
else:
- return self.h5[self.featureAttrsKey][self.featIdsKey][:]
+ return np.asarray(self.h5[self.featureAttrsKey][self.featIdsKey][:])
logger.warning(
f"Could not find feature ids key: {self.featIdsKey} in {self.featureAttrsKey}."
)
@@ -779,7 +862,7 @@ def feat_names(self) -> np.ndarray:
return self.feat_ids()
def _replace_category_values(
- self, v: Union[np.ndarray, h5py.Group, h5py.Dataset], key: str, group: str
+ self, v: np.ndarray | h5py.Group | h5py.Dataset, key: str, group: str
) -> np.ndarray:
# check if v is a Group with codes + categories structure
if isinstance(v, h5py.Group):
@@ -793,13 +876,15 @@ def _replace_category_values(
return np.array([f"feature_{x}" for x in range(len(codes))])
else:
# It's a Group but doesn't have the expected structure, try to read it as dataset
- logger.warning(f"{key} is a Group but missing 'codes' or 'categories', attempting to extract data")
- return v[:]
-
+ logger.warning(
+ f"{key} is a Group but missing 'codes' or 'categories', attempting to extract data"
+ )
+ return np.asarray(v[:])
+
# if v is a Dataset
if isinstance(v, h5py.Dataset):
v = v[:]
-
+
if self.catNamesKey is not None:
if self._check_exists(group, self.catNamesKey):
cat_g = self.h5[group][self.catNamesKey]
@@ -817,11 +902,11 @@ def _replace_category_values(
return np.array([c[x] for x in v])
except (IndexError, TypeError):
return v
- return v
+ return np.asarray(v)
def _get_col_data(
- self, group: str, ignore_keys: List[str]
- ) -> Generator[Tuple[str, np.ndarray], None, None]:
+ self, group: str, ignore_keys: list[str]
+ ) -> Generator[tuple[str, np.ndarray], None, None]:
if self.groupCodes[group] == 1:
for i in tqdmbar(
self.h5[group].dtype.names,
@@ -844,7 +929,7 @@ def _get_col_data(
def _get_obsm_data(
self, group: str
- ) -> Generator[Tuple[str, np.ndarray], None, None]:
+ ) -> Generator[tuple[str, np.ndarray], None, None]:
if self.groupCodes[group] == 2:
for i in tqdmbar(
self.h5[group].keys(), desc=f"Reading attributes from group {group}"
@@ -858,20 +943,20 @@ def _get_obsm_data(
continue
if isinstance(g, h5py.Dataset):
for j in range(g.shape[1]):
- yield f"{i}{j+1}", g[:, j]
+ yield f"{i}{j + 1}", g[:, j]
else:
logger.warning(
f"Reading of obsm failed because it either does not exist or is not in expected format" # noqa: F541
)
- def get_cell_columns(self) -> Generator[Tuple[str, np.ndarray], None, None]:
+ def get_cell_columns(self) -> Generator[tuple[str, np.ndarray], None, None]:
"""Creates a Generator that yields the cell columns."""
for i, j in self._get_col_data(self.cellAttrsKey, [self.cellIdsKey]):
yield i, j
for i, j in self._get_obsm_data(self.obsmAttrsKey):
yield i, j
- def get_feat_columns(self) -> Generator[Tuple[str, np.ndarray], None, None]:
+ def get_feat_columns(self) -> Generator[tuple[str, np.ndarray], None, None]:
"""Creates a Generator that yields the feature columns."""
for i, j in self._get_col_data(
self.featureAttrsKey, [self.featIdsKey, self.featNamesKey]
@@ -893,30 +978,38 @@ def consume_dataset(
def consume_group(self, batch_size: int) -> Generator[coo_matrix, None, None]:
"""Returns a generator that yield chunks of data."""
+ if batch_size <= 0:
+ raise ValueError("batch_size must be positive")
+
grp = self.h5[self.matrixKey]
- s = 0
- for ind_n in range(0, self.nCells, batch_size):
- i = grp["indptr"][ind_n : ind_n + batch_size]
- e = i[-1]
- if s != 0:
- idx = np.array([s] + list(i))
- idx = idx - idx[0]
- else:
- idx = np.array(i)
- n = idx.shape[0] - 1
- nidx = np.repeat(range(n), np.diff(idx).astype("int32"))
+ for row_start in range(0, self.nCells, batch_size):
+ row_end = min(row_start + batch_size, self.nCells)
+ indptr = np.asarray(grp["indptr"][row_start : row_end + 1])
+ data_start = int(indptr[0])
+ data_end = int(indptr[-1])
+ local_indptr = indptr - data_start
+ n_rows = row_end - row_start
+ row_indices = np.repeat(
+ np.arange(n_rows),
+ np.diff(local_indptr),
+ )
yield coo_matrix(
- (grp["data"][s:e], (nidx, grp["indices"][s:e])),
- shape=(n, self.nFeatures),
+ (
+ grp["data"][data_start:data_end],
+ (row_indices, grp["indices"][data_start:data_end]),
+ ),
+ shape=(n_rows, self.nFeatures),
)
- s = e
- def consume(self, batch_size: int):
+ def consume(self, batch_size: int) -> Generator[coo_matrix, None, None]:
"""Returns a generator that yield chunks of data."""
if self.groupCodes[self.matrixKey] == 1:
return self.consume_dataset(batch_size)
elif self.groupCodes[self.matrixKey] == 2:
return self.consume_group(batch_size)
+ raise ValueError(
+ f"ERROR: {self.matrixKey} is neither Dataset or Group type. Will not consume data"
+ )
class NaboH5Reader:
@@ -931,8 +1024,8 @@ class NaboH5Reader:
nFeatures: Number of features in dataset.
"""
- def __init__(self, h5_fn: str):
- self.h5 = h5py.File(h5_fn, mode="r")
+ def __init__(self, h5_fn: str) -> None:
+ self.h5: h5py.File = h5py.File(h5_fn, mode="r")
self._check_integrity()
self.nCells = self.h5["names"]["cells"].shape[0]
self.nFeatures = self.h5["names"]["genes"].shape[0]
@@ -943,7 +1036,7 @@ def _check_integrity(self) -> bool:
raise KeyError(f"ERROR: Expected group: {i} is missing in the H5 file")
return True
- def cell_ids(self) -> List[str]:
+ def cell_ids(self) -> list[str]:
"""Returns a list of cell IDs."""
return [x.decode("UTF-8") for x in self.h5["names"]["cells"][:]]
@@ -951,7 +1044,7 @@ def feat_ids(self) -> np.ndarray:
"""Returns a list of feature IDs."""
return np.array([f"feature_{x}" for x in range(self.nFeatures)])
- def feat_names(self) -> List[str]:
+ def feat_names(self) -> list[str]:
"""Returns a list of feature names."""
return [
x.decode("UTF-8").rsplit("_", 1)[0] for x in self.h5["names"]["genes"][:]
@@ -1009,14 +1102,14 @@ def __init__(
self,
loom_fn: str,
matrix_key: str = "matrix",
- cell_attrs_key="col_attrs",
+ cell_attrs_key: str = "col_attrs",
cell_names_key: str = "obs_names",
feature_attrs_key: str = "row_attrs",
feature_names_key: str = "var_names",
- feature_ids_key: str = None,
- dtype: str = None,
+ feature_ids_key: str | None = None,
+ dtype: str | None = None,
) -> None:
- self.h5 = h5py.File(loom_fn, mode="r")
+ self.h5: h5py.File = h5py.File(loom_fn, mode="r")
self.matrixKey = matrix_key
self.cellAttrsKey, self.featureAttrsKey = cell_attrs_key, feature_attrs_key
self.cellNamesKey, self.featureNamesKey = cell_names_key, feature_names_key
@@ -1040,7 +1133,7 @@ def _check_integrity(self) -> bool:
)
return True
- def cell_names(self) -> List[str]:
+ def cell_names(self) -> list[str]:
"""Returns a list of names of the cells in the dataset."""
if self.cellAttrsKey not in self.h5:
pass
@@ -1049,19 +1142,26 @@ def cell_names(self) -> List[str]:
f"Cell names/ids key ({self.cellNamesKey}) is missing in attributes"
)
else:
- return self.h5[self.cellAttrsKey][self.cellNamesKey][:]
+ return list(self.h5[self.cellAttrsKey][self.cellNamesKey][:])
return [f"cell_{x}" for x in range(self.nCells)]
- def cell_ids(self) -> List[str]:
+ def cell_ids(self) -> list[str]:
"""Returns a list of cell IDs."""
return self.cell_names()
def _stream_attrs(
- self, key, ignore
- ) -> Generator[Tuple[str, np.ndarray], None, None]:
+ self, key: str, ignore: str | list[str] | None
+ ) -> Generator[tuple[str, np.ndarray], None, None]:
+ ignored: set[str]
+ if isinstance(ignore, str):
+ ignored = {ignore}
+ elif ignore is None:
+ ignored = set()
+ else:
+ ignored = {name for name in ignore if name is not None}
if key in self.h5:
for i in tqdmbar(self.h5[key].keys(), desc=f"Reading {key} attributes"):
- if i in [ignore]:
+ if i in ignored:
continue
vals = self.h5[key][i][:]
if vals.dtype.names is None:
@@ -1071,11 +1171,11 @@ def _stream_attrs(
for j in vals.dtype.names:
yield i + "_" + str(j), vals[j]
- def get_cell_attrs(self) -> Generator[Tuple[str, np.ndarray], None, None]:
+ def get_cell_attrs(self) -> Generator[tuple[str, np.ndarray], None, None]:
"""Returns a Generator that yields the cells' attributes."""
return self._stream_attrs(self.cellAttrsKey, [self.cellNamesKey])
- def feature_names(self) -> List[str]:
+ def feature_names(self) -> list[str]:
"""Returns a list of feature names."""
if self.featureAttrsKey not in self.h5:
pass
@@ -1084,10 +1184,10 @@ def feature_names(self) -> List[str]:
f"Feature names key ({self.featureNamesKey}) is missing in attributes"
)
else:
- return self.h5[self.featureAttrsKey][self.featureNamesKey][:]
+ return list(self.h5[self.featureAttrsKey][self.featureNamesKey][:])
return [f"feature_{x}" for x in range(self.nFeatures)]
- def feature_ids(self) -> List[str]:
+ def feature_ids(self) -> list[str]:
"""Returns a list of feature IDs."""
if self.featureAttrsKey not in self.h5:
pass
@@ -1098,14 +1198,15 @@ def feature_ids(self) -> List[str]:
f"Feature names key ({self.featureIdsKey}) is missing in attributes"
)
else:
- return self.h5[self.featureAttrsKey][self.featureIdsKey][:]
+ return list(self.h5[self.featureAttrsKey][self.featureIdsKey][:])
return [f"feature_{x}" for x in range(self.nFeatures)]
- def get_feature_attrs(self) -> Generator[Tuple[str, np.ndarray], None, None]:
+ def get_feature_attrs(self) -> Generator[tuple[str, np.ndarray], None, None]:
"""Returns a Generator that yields the features' attributes."""
- return self._stream_attrs(
- self.featureAttrsKey, [self.featureIdsKey, self.featureNamesKey]
- )
+ ignore_keys = [
+ key for key in (self.featureIdsKey, self.featureNamesKey) if key is not None
+ ]
+ return self._stream_attrs(self.featureAttrsKey, ignore_keys)
def consume(self, batch_size: int = 1000) -> Generator[np.ndarray, None, None]:
"""Returns a generator that yield chunks of data."""
@@ -1146,15 +1247,15 @@ def __init__(
self,
csv_fn: str,
has_header: bool = True,
- id_column: Optional[int] = None,
+ id_column: int | None = None,
rows_are_cells: bool = True,
sep: str = ",",
skip_rows: int = 0,
- skip_cols: Optional[List[str]] = None,
- cell_data_cols: Optional[List[str]] = None,
- batch_size=10000,
- pandas_kwargs: Optional[dict] = None,
- ):
+ skip_cols: list[str] | None = None,
+ cell_data_cols: list[str] | None = None,
+ batch_size: int = 10000,
+ pandas_kwargs: dict[str, Any] | None = None,
+ ) -> None:
self._fn = csv_fn
if rows_are_cells is False:
raise NotImplementedError(
@@ -1165,13 +1266,14 @@ def __init__(
else:
if not isinstance(pandas_kwargs, dict):
logger.error("")
+ header_row: int | None
if has_header is False:
- has_header = None
+ header_row = None
else:
- has_header = 0
- self.pandas_kwargs = pandas_kwargs
+ header_row = 0
+ self.pandas_kwargs: dict[str, Any] = pandas_kwargs
self.pandas_kwargs["sep"] = sep
- self.pandas_kwargs["header"] = has_header
+ self.pandas_kwargs["header"] = header_row
self.pandas_kwargs["skiprows"] = skip_rows
self.pandas_kwargs["chunksize"] = batch_size
self.pandas_kwargs["index_col"] = id_column
@@ -1194,36 +1296,37 @@ def __init__(
self.cellDataIdx,
) = self._consistency_check()
- def _get_streamer(self) -> Generator:
- return pd.read_csv(self._fn, **self.pandas_kwargs)
+ def _get_streamer(self) -> Generator[pd.DataFrame, None, None]:
+ reader = pd.read_csv(self._fn, **self.pandas_kwargs)
+ yield from reader
def _consistency_check(
self,
- ) -> Tuple[
+ ) -> tuple[
int,
int,
- Optional[np.ndarray],
- Optional[np.ndarray],
- Optional[List[int]],
- Optional[List[np.dtype]],
- Optional[List[int]],
+ np.ndarray | None,
+ np.ndarray | None,
+ list[int] | None,
+ list[np.dtype] | None,
+ list[int] | None,
]:
stream = self._get_streamer()
n_cells = 0
n_features = 0
- feature_ids = None
- cell_data_dtypes = None
- cell_data_idx = None
- if self.pandas_kwargs["index_col"] is None:
- cell_ids = None
- else:
- cell_ids = []
+ feature_ids: np.ndarray | None = None
+ cell_data_dtypes: list[np.dtype] | None = None
+ cell_data_idx: list[int] | None = None
+ cell_ids: np.ndarray | None = None
+ collected_cell_ids: list[Any] | None = None
+ if self.pandas_kwargs["index_col"] is not None:
+ collected_cell_ids = []
for df in tqdmbar(stream, desc="Performing CSV file consistency check"):
n_cells += df.shape[0]
if n_features == 0:
n_features = df.shape[1]
if self.pandas_kwargs["header"] is not None:
- feature_ids = df.columns.values
+ feature_ids = np.asarray(df.columns.values)
if len(feature_ids) != n_features:
raise ValueError(
"Header length not same as number of features. This can happen if you did not"
@@ -1242,9 +1345,9 @@ def _consistency_check(
"Number of columns changed in the CSV during consistency check."
" Maybe a problem with the delimiter."
)
- if cell_ids is not None:
- cell_ids = np.ndarray(cell_ids)
- keep_cols = None
+ if collected_cell_ids is not None:
+ cell_ids = np.asarray(collected_cell_ids)
+ keep_cols: list[int] | None = None
if feature_ids is not None:
skip_names = list(set(self.skipCols).union(self.cellDataCols))
if len(skip_names) > 0:
@@ -1263,7 +1366,7 @@ def _consistency_check(
cell_data_idx,
)
- def _n_features(self):
+ def _n_features(self) -> np.ndarray:
return np.array([f"feature_{x}" for x in range(self.nFeatures)])
def cell_ids(self) -> np.ndarray:
@@ -1280,7 +1383,7 @@ def feature_ids(self) -> np.ndarray:
else:
return self.featureIds
- def consume(self) -> Generator[Tuple[np.ndarray, Optional[np.ndarray]], None, None]:
+ def consume(self) -> Generator[tuple[np.ndarray, np.ndarray | None], None, None]:
"""Returns a generator that yield chunks of data."""
stream = self._get_streamer()
if self.keepCols is None:
diff --git a/scarf/tests/test_plotting.py b/scarf/storage/__init__.py
similarity index 100%
rename from scarf/tests/test_plotting.py
rename to scarf/storage/__init__.py
diff --git a/scarf/storage/budget.py b/scarf/storage/budget.py
new file mode 100644
index 00000000..8521d654
--- /dev/null
+++ b/scarf/storage/budget.py
@@ -0,0 +1,245 @@
+"""Process-wide resource budget for predictable memory use.
+
+A single :class:`ResourceBudget` (total memory bytes, worker count, working
+copies) is the source of truth. Write-time chunk and shard geometry is derived
+from ``memoryBytes // workingCopies``; ``workers`` sets read concurrency and
+async IO parallelism. Once files are written, reads follow the on-disk chunk
+and shard geometry with no additional memory heuristics.
+"""
+
+import os
+from dataclasses import dataclass
+
+_DEFAULT_WORKING_COPIES = 8
+_FALLBACK_MEMORY_BYTES = 8 * 1024 * 1024 * 1024
+_SUFFIXES = {"K": 1024, "M": 1024**2, "G": 1024**3, "T": 1024**4}
+_MIN_RAW_BYTES = 1024 * 1024
+
+
+@dataclass(frozen=True)
+class ResourceBudget:
+ """Memory and worker budget for write-time geometry and read concurrency."""
+
+ memoryBytes: int
+ workers: int
+ workingCopies: int = _DEFAULT_WORKING_COPIES
+
+
+def detect_total_memory_bytes() -> int:
+ """Best-effort total physical system memory in bytes.
+
+ Uses total (installed) memory rather than currently free memory so the
+ derived write-time geometry is a property of the machine, not of transient
+ load at the moment the process happens to run.
+ """
+ try:
+ with open("/proc/meminfo") as handle:
+ for line in handle:
+ if line.startswith("MemTotal:"):
+ return int(line.split()[1]) * 1024
+ except OSError:
+ pass
+ try:
+ page_size = os.sysconf("SC_PAGE_SIZE")
+ total_pages = os.sysconf("SC_PHYS_PAGES")
+ if page_size > 0 and total_pages > 0:
+ return int(page_size) * int(total_pages)
+ except (ValueError, OSError, AttributeError):
+ pass
+ return _FALLBACK_MEMORY_BYTES
+
+
+def detect_workers() -> int:
+ return max(1, os.cpu_count() or 1)
+
+
+def _parse_memory_spec(spec: str | int | float) -> int:
+ guidance = (
+ "Use raw bytes, a suffixed size like '8G'/'512M', "
+ "or a fraction strictly between 0 and 1 like '0.6'."
+ )
+ if isinstance(spec, bool):
+ raise ValueError(f"Invalid memory spec: {spec!r}. {guidance}")
+ if isinstance(spec, int):
+ if spec <= 0:
+ raise ValueError(f"Memory budget must be positive, got {spec!r}.")
+ return spec
+
+ text = str(spec).strip()
+ if not text:
+ raise ValueError("Empty memory spec. " + guidance)
+ suffix = text[-1].upper()
+ if suffix in _SUFFIXES:
+ try:
+ value = float(text[:-1])
+ except ValueError as exc:
+ raise ValueError(f"Invalid memory spec: {spec!r}. {guidance}") from exc
+ if value <= 0:
+ raise ValueError(f"Memory budget must be positive, got {spec!r}.")
+ return max(1, int(value * _SUFFIXES[suffix]))
+
+ try:
+ number = float(text)
+ except ValueError as exc:
+ raise ValueError(f"Invalid memory spec: {spec!r}. {guidance}") from exc
+ if number <= 0:
+ raise ValueError(f"Memory budget must be positive, got {spec!r}.")
+ if number < 1:
+ return max(1, int(number * detect_total_memory_bytes()))
+ if number < _MIN_RAW_BYTES:
+ raise ValueError(
+ f"Ambiguous memory spec {spec!r}: a bare number >= 1 is read as bytes, "
+ f"which is implausibly small. {guidance}"
+ )
+ return int(number)
+
+
+def _resolve_working_copies(working_copies: int | None = None) -> int:
+ if working_copies is not None:
+ copies = int(working_copies)
+ if copies <= 0:
+ raise ValueError(f"workingCopies must be positive, got {copies!r}.")
+ return copies
+ env = os.environ.get("SCARF_WORKING_COPIES")
+ if env:
+ try:
+ copies = int(env)
+ except ValueError as exc:
+ raise ValueError(
+ f"Invalid SCARF_WORKING_COPIES={env!r}; expected an integer."
+ ) from exc
+ if copies <= 0:
+ raise ValueError(f"SCARF_WORKING_COPIES must be positive, got {copies!r}.")
+ return copies
+ return _DEFAULT_WORKING_COPIES
+
+
+def resolve_budget(
+ memory: int | str | None = None,
+ workers: int | None = None,
+ *,
+ working_copies: int | None = None,
+) -> ResourceBudget:
+ """Build a :class:`ResourceBudget`, auto-detecting unset fields."""
+ if memory is None:
+ env_mem = os.environ.get("SCARF_MEM_BUDGET")
+ if env_mem:
+ memory_bytes = _parse_memory_spec(env_mem)
+ else:
+ memory_bytes = detect_total_memory_bytes()
+ else:
+ memory_bytes = _parse_memory_spec(memory)
+
+ if workers is None:
+ env_workers = os.environ.get("SCARF_WORKERS")
+ if env_workers:
+ try:
+ worker_count = int(env_workers)
+ except ValueError as exc:
+ raise ValueError(
+ f"Invalid SCARF_WORKERS={env_workers!r}; expected an integer."
+ ) from exc
+ else:
+ worker_count = detect_workers()
+ else:
+ worker_count = int(workers)
+
+ return ResourceBudget(
+ memoryBytes=max(1, memory_bytes),
+ workers=max(1, worker_count),
+ workingCopies=_resolve_working_copies(working_copies),
+ )
+
+
+_activeBudget: ResourceBudget | None = None
+_cachedDefault: ResourceBudget | None = None
+
+
+def set_resource_budget(budget: ResourceBudget | None) -> None:
+ """Install the process-wide active budget (None resets to auto)."""
+ global _activeBudget, _cachedDefault
+ _activeBudget = budget
+ _cachedDefault = None
+
+
+def get_resource_budget() -> ResourceBudget:
+ """Return the active budget, lazily resolving (and caching) a default."""
+ global _cachedDefault
+ if _activeBudget is not None:
+ return _activeBudget
+ if _cachedDefault is None:
+ _cachedDefault = resolve_budget()
+ return _cachedDefault
+
+
+def worker_prefetch_depth(
+ requested: int | None = None,
+ budget: ResourceBudget | None = None,
+) -> int:
+ """Read-ahead depth for parallel block reads, capped by ``READ_AHEAD``.
+
+ Matches the shard read-ahead model: at most ``READ_AHEAD`` blocks are kept in
+ flight so IO overlaps compute without inflating peak memory. A consumer may
+ ``request`` a smaller depth; ``READ_AHEAD`` (bounded by ``workingCopies``) is
+ the ceiling.
+ """
+ budget = budget or get_resource_budget()
+ ceiling = max(1, min(READ_AHEAD, budget.workingCopies))
+ if requested is None:
+ return ceiling
+ return max(1, min(int(requested), ceiling))
+
+
+# Shards are processed in order with this shallow read-ahead so the next band
+# downloads while the current one is being processed. A deeper queue only
+# inflates peak memory (more resident bands) without improving throughput, since
+# the whole worker budget is already spent parallelising IO and compute *within*
+# each shard.
+READ_AHEAD = 2
+
+
+@dataclass(frozen=True)
+class ShardPlan:
+ """How to run a shard-parallel op.
+
+ Shards are processed in order with a shallow read-ahead: up to ``readAhead``
+ bands may be in flight so IO overlaps compute. The worker budget is spent on
+ ``ioConcurrency`` for Zarr's ``async.concurrency`` (parallel inner-chunk IO
+ within a shard); ``withinBlockThreads`` stays at 1 so BLAS/OpenMP reductions
+ stay single-threaded and bit-identical regardless of the worker count. Peak
+ resident bands is bounded by ``readAhead`` (kept at or below
+ ``workingCopies``), so memory stays within the budget the geometry was sized
+ against.
+ """
+
+ readAhead: int
+ ioConcurrency: int
+ withinBlockThreads: int
+
+
+def shard_parallelism(
+ workers: int | None = None,
+ n_shards: int | None = None,
+ *,
+ budget: ResourceBudget | None = None,
+) -> ShardPlan:
+ """Derive a :class:`ShardPlan` from the resource budget.
+
+ Spends the worker budget on parallel inner-chunk IO within a shard and
+ pipelines a shallow ``READ_AHEAD`` of shards so the next band downloads while
+ the current one is processed. BLAS threads are pinned to one so results stay
+ bit-identical across worker counts (extra BLAS threads were benchmarked to
+ add little). Read-ahead is bounded by ``workingCopies`` and the shard count
+ so peak resident memory stays within budget.
+ """
+ budget = budget or get_resource_budget()
+ workers = budget.workers if workers is None else max(1, int(workers))
+ caps = [READ_AHEAD, max(1, budget.workingCopies)]
+ if n_shards is not None:
+ caps.append(max(1, int(n_shards)))
+ read_ahead = max(1, min(caps))
+ return ShardPlan(
+ readAhead=read_ahead,
+ ioConcurrency=workers,
+ withinBlockThreads=1,
+ )
diff --git a/scarf/storage/zarr_store.py b/scarf/storage/zarr_store.py
new file mode 100644
index 00000000..bf577d41
--- /dev/null
+++ b/scarf/storage/zarr_store.py
@@ -0,0 +1,975 @@
+import math
+import os
+from collections.abc import Callable, Iterator
+from dataclasses import dataclass
+from typing import Any, Literal
+
+import numpy as np
+import zarr
+from zarr.abc.store import Store
+from zarr.codecs import BloscCodec, ZstdCodec
+
+from .._types import ZarrMode, as_zarr_array, as_zarr_group, array_metadata_shards
+from .budget import ResourceBudget, get_resource_budget
+
+StorageProfile = Literal["fast_local", "cloud"]
+
+type ZarrLocation = str | Store
+
+PROFILE_METADATA_CHUNK = 100_000
+# numcodecs compression codecs reject buffers larger than a signed 32-bit byte count.
+_CODEC_MAX_BYTES = 2_147_483_647
+DEFAULT_CLOUD_TARGET_CHUNK_BYTES = 128 * 1024 * 1024
+DEFAULT_MIN_FEATURE_CHUNK = 500
+DEFAULT_MAX_FEATURE_CHUNK = 10_000
+
+
+def _ceil_pad(n: int, chunk: int) -> int:
+ chunk = max(1, int(chunk))
+ n = max(1, int(n))
+ return math.ceil(n / chunk) * chunk
+
+
+def _fit_shard_to_byte_limit(
+ row_shard: int,
+ feature_chunk: int,
+ shard_cols: int,
+ n_features: int,
+ *,
+ itemsize: int,
+ max_bytes: int,
+) -> tuple[int, int, int]:
+ """Shrink shard geometry so ``row_shard * shard_cols * itemsize <= max_bytes``."""
+
+ def shard_bytes(rs: int, sc: int) -> int:
+ return rs * sc * itemsize
+
+ while shard_bytes(row_shard, shard_cols) > max_bytes:
+ if row_shard > 1:
+ row_shard = max(1, max_bytes // (shard_cols * itemsize))
+ while row_shard > 1 and shard_bytes(row_shard, shard_cols) > max_bytes:
+ row_shard -= 1
+ elif shard_cols > feature_chunk:
+ shard_cols = max(
+ feature_chunk,
+ (max_bytes // (row_shard * itemsize) // feature_chunk) * feature_chunk,
+ )
+ if shard_bytes(row_shard, shard_cols) > max_bytes:
+ shard_cols = max(feature_chunk, shard_cols - feature_chunk)
+ elif feature_chunk > 1:
+ feature_chunk = max(1, max_bytes // (row_shard * itemsize))
+ shard_cols = _ceil_pad(n_features, feature_chunk)
+ if shard_bytes(row_shard, shard_cols) > max_bytes:
+ shard_cols = max(
+ feature_chunk,
+ (max_bytes // (row_shard * itemsize) // feature_chunk)
+ * feature_chunk,
+ )
+ else:
+ feature_chunk = max(1, max_bytes // itemsize)
+ shard_cols = feature_chunk
+ row_shard = 1
+ break
+ return row_shard, feature_chunk, shard_cols
+
+
+def matrix_layout(
+ n_cells: int,
+ n_features: int,
+ *,
+ budget: ResourceBudget | None = None,
+ itemsize: int = 4,
+ width: int | None = None,
+ targetChunkBytes: int | None = None,
+ minFeatureChunk: int = 1,
+ maxFeatureChunk: int | None = None,
+) -> tuple[tuple[int, int], tuple[int, int] | None]:
+ """Return ``(chunks, shards)`` from the memory-first layout rule.
+
+ Geometry is driven by ``memoryBytes // workingCopies`` so a single chunk
+ (or shard) band is a budget-sized read unit, capped at the codec's maximum
+ input buffer size. When ``width`` is set (normalized/derived), returns plain
+ chunks ``(rowShard, width)`` and ``shards=None``. Otherwise returns
+ count-matrix geometry with ceil-padded feature shards.
+
+ When ``targetChunkBytes`` is set, feature-chunk width is chosen so
+ ``row_shard * feature_chunk * itemsize`` stays near that target, then
+ clamped to ``[minFeatureChunk, maxFeatureChunk]``.
+ """
+ budget = budget or get_resource_budget()
+ n_cells = max(int(n_cells), 1)
+ n_features = max(int(n_features), 1)
+ itemsize = max(1, int(itemsize))
+ work = budget.memoryBytes // max(1, budget.workingCopies)
+
+ if width is not None:
+ w = max(1, int(width))
+ row_bytes = w * itemsize
+ if row_bytes > _CODEC_MAX_BYTES:
+ raise ValueError(
+ f"One full-width row requires {row_bytes} bytes, exceeding "
+ f"the codec limit of {_CODEC_MAX_BYTES} bytes"
+ )
+ max_chunk_bytes = min(work, _CODEC_MAX_BYTES)
+ row_shard = max(1, min(n_cells, max_chunk_bytes // row_bytes))
+ chunks = (row_shard, w)
+ return chunks, None
+
+ row_shard = max(1, min(n_cells, work // (n_features * itemsize)))
+ feature_chunk = max(1, min(n_features, work // (n_cells * itemsize)))
+ if targetChunkBytes is not None:
+ target = max(itemsize, int(targetChunkBytes))
+ feature_chunk = max(1, target // (row_shard * itemsize))
+ feature_chunk = min(n_features, feature_chunk)
+ lo = max(1, int(minFeatureChunk))
+ hi = n_features if maxFeatureChunk is None else max(1, int(maxFeatureChunk))
+ if lo > hi:
+ lo, hi = hi, lo
+ feature_chunk = max(lo, min(hi, feature_chunk))
+ feature_chunk = min(n_features, feature_chunk)
+ shard_cols = _ceil_pad(n_features, feature_chunk)
+ max_shard_bytes = min(work, _CODEC_MAX_BYTES)
+ row_shard, feature_chunk, shard_cols = _fit_shard_to_byte_limit(
+ row_shard,
+ feature_chunk,
+ shard_cols,
+ n_features,
+ itemsize=itemsize,
+ max_bytes=max_shard_bytes,
+ )
+ chunks = (row_shard, feature_chunk)
+ shards = (row_shard, shard_cols)
+ return chunks, shards
+
+
+@dataclass
+class ZarrArraySpec:
+ """Specification for creating a numeric Zarr array."""
+
+ shape: tuple[int, ...]
+ chunks: tuple[int, ...]
+ dtype: Any
+ shards: tuple[int, ...] | None = None
+ zarrFormat: int = 3
+ compressors: list | None = None
+ fillValue: Any | None = None
+ overwrite: bool = True
+
+
+def get_compressors(
+ profile: StorageProfile = "fast_local", zarrFormat: int = 3
+) -> list:
+ """Return codec list for the given storage profile and Zarr format."""
+ if zarrFormat == 2:
+ from numcodecs import Blosc
+
+ return [Blosc(cname="lz4", clevel=5, shuffle=Blosc.BITSHUFFLE)]
+ if profile == "cloud":
+ return [ZstdCodec(level=3)]
+ return [BloscCodec(cname="lz4", clevel=5, shuffle="bitshuffle")]
+
+
+def _group_zarr_format(group: zarr.Group) -> int:
+ metadata = getattr(group, "metadata", None)
+ if metadata is not None and getattr(metadata, "zarr_format", None) is not None:
+ return int(metadata.zarr_format)
+ return 3
+
+
+def zarr_group_root(group: zarr.Group, mode: ZarrMode = "r+") -> zarr.Group:
+ """Open the root Zarr group sharing the same store as ``group``."""
+ return zarr.open_group(store=group.store, mode=mode)
+
+
+def zarr_root_path(group: zarr.Group) -> str | None:
+ """Return filesystem path for a Zarr group, if available."""
+ store = group.store
+ root = getattr(store, "root", None)
+ if root is not None:
+ return str(root)
+ storePath = getattr(group, "store_path", None)
+ if storePath and str(storePath).startswith("file://"):
+ return str(storePath)[7:]
+ return None
+
+
+def normalize_chunks(
+ chunks: tuple[int, ...] | int, shape: tuple[int, ...]
+) -> tuple[int, ...]:
+ """Map chunk specification to a valid per-dimension chunk tuple for ``shape``."""
+ if isinstance(chunks, int):
+ chunks = (chunks,)
+ if len(chunks) == len(shape):
+ return tuple(min(chunk, dim) for chunk, dim in zip(chunks, shape))
+ if len(chunks) == 1 and len(shape) > 1:
+ return (min(chunks[0], shape[0]),) + tuple(shape[1:])
+ if len(chunks) == 1:
+ return (min(chunks[0], shape[0]),)
+ raise ValueError(
+ f"Cannot map chunks {chunks} to array shape {shape}. "
+ "Provide one chunk size per dimension."
+ )
+
+
+_activeProfile: StorageProfile | None = None
+
+
+def set_storage_profile(profile: StorageProfile | None) -> None:
+ """Override the active Zarr storage profile for this process."""
+ global _activeProfile
+ _activeProfile = profile
+
+
+def get_storage_profile() -> StorageProfile:
+ """Return active profile from override or ``SCARF_ZARR_PROFILE`` env var."""
+ if _activeProfile is not None:
+ return _activeProfile
+ envProfile = os.environ.get("SCARF_ZARR_PROFILE", "fast_local")
+ if envProfile == "cloud":
+ return "cloud"
+ if envProfile == "fast_local":
+ return "fast_local"
+ return "fast_local"
+
+
+def configure_zarr_io_for_profile() -> None:
+ """Set zarr async IO parallelism to the budget's worker count."""
+ budget = get_resource_budget()
+ zarr.config.set({"async.concurrency": max(1, budget.workers)})
+
+
+def count_array_spec(
+ nCells: int,
+ nFeats: int,
+ dtype: Any = "uint32",
+ profile: StorageProfile | None = None,
+ *,
+ remote: bool | None = None,
+ budget: ResourceBudget | None = None,
+ targetChunkBytes: int | None = None,
+ minFeatureChunk: int | None = None,
+ maxFeatureChunk: int | None = None,
+) -> ZarrArraySpec:
+ """Build array spec for assay count matrices."""
+ profile = profile or get_storage_profile()
+ budget = budget or get_resource_budget()
+ if remote is None:
+ remote = profile == "cloud"
+ itemsize = int(np.dtype(dtype).itemsize)
+
+ layout_kwargs: dict[str, Any] = {}
+ resolved_target = targetChunkBytes
+ if resolved_target is None and remote:
+ resolved_target = DEFAULT_CLOUD_TARGET_CHUNK_BYTES
+ if resolved_target is not None:
+ layout_kwargs["targetChunkBytes"] = resolved_target
+ layout_kwargs["minFeatureChunk"] = (
+ DEFAULT_MIN_FEATURE_CHUNK if minFeatureChunk is None else minFeatureChunk
+ )
+ layout_kwargs["maxFeatureChunk"] = (
+ DEFAULT_MAX_FEATURE_CHUNK if maxFeatureChunk is None else maxFeatureChunk
+ )
+
+ chunks, shards_raw = matrix_layout(
+ nCells,
+ nFeats,
+ budget=budget,
+ itemsize=itemsize,
+ **layout_kwargs,
+ )
+ assert shards_raw is not None
+ # shards_raw's feature dimension is deliberately ceil-padded to a multiple
+ # of chunks[1] (see matrix_layout); clipping it down to nFeats here would
+ # make the shard size no longer divisible by the chunk size, which Zarr
+ # v3 rejects. matrix_layout already bounds both chunk dims by (nCells,
+ # nFeats) and the row shard dim by nCells, so no further clipping needed.
+ shards = shards_raw
+
+ return ZarrArraySpec(
+ shape=(nCells, nFeats),
+ chunks=chunks,
+ shards=shards,
+ dtype=dtype,
+ compressors=get_compressors("cloud" if remote else profile),
+ fillValue=0,
+ )
+
+
+def metadata_array_spec(
+ length: int,
+ dtype: Any,
+ profile: StorageProfile | None = None,
+) -> ZarrArraySpec:
+ """Build array spec for 1D metadata columns."""
+ chunkSize = min(PROFILE_METADATA_CHUNK, max(length, 1))
+ return ZarrArraySpec(
+ shape=(length,),
+ chunks=(chunkSize,),
+ dtype=dtype,
+ compressors=get_compressors(profile or get_storage_profile()),
+ )
+
+
+def normed_array_spec(
+ nCells: int,
+ nFeats: int,
+ profile: StorageProfile | None = None,
+ *,
+ remote: bool | None = None,
+ budget: ResourceBudget | None = None,
+) -> ZarrArraySpec:
+ """Build array spec for normalized expression matrices (graph-building slot)."""
+ profile = profile or get_storage_profile()
+ budget = budget or get_resource_budget()
+ if remote is None:
+ remote = profile == "cloud"
+ itemsize = 4
+
+ chunks, _ = matrix_layout(
+ nCells,
+ nFeats,
+ budget=budget,
+ itemsize=itemsize,
+ width=max(nFeats, 1),
+ )
+ row_chunk, n_cols = chunks
+
+ return ZarrArraySpec(
+ shape=(nCells, nFeats),
+ chunks=(row_chunk, n_cols),
+ shards=None,
+ dtype="float32",
+ compressors=get_compressors("cloud" if remote else profile),
+ fillValue=0.0,
+ )
+
+
+def array_shard_rows(array: zarr.Array) -> int:
+ """Row extent of one shard, or row chunk, or the full height."""
+ shards_meta = array_metadata_shards(array)
+ if shards_meta is not None and len(shards_meta) > 0:
+ return int(shards_meta[0])
+ chunks = getattr(array, "chunks", None)
+ if chunks is not None and len(chunks) > 0:
+ return int(chunks[0])
+ return max(int(array.shape[0]), 1)
+
+
+def iter_shard_row_slices(n_rows: int, shard_rows: int) -> Iterator[tuple[int, int]]:
+ """Yield ``(start, end)`` row slices aligned to ``shard_rows``."""
+ shard_rows = max(1, int(shard_rows))
+ n_rows = max(0, int(n_rows))
+ for start in range(0, n_rows, shard_rows):
+ yield start, min(start + shard_rows, n_rows)
+
+
+def write_dense_from_row_batches(
+ dst: zarr.Array,
+ batches: Iterator[np.ndarray],
+ *,
+ dtype: Any | None = None,
+ msg: str | None = None,
+) -> int:
+ """Write sequential dense row batches, flushing at destination row-shard edges."""
+ from ..utils import tqdmbar
+
+ shard_rows = array_shard_rows(dst)
+ pending: list[np.ndarray] = []
+ pending_rows = 0
+ out_pos = 0
+ total_rows = 0
+
+ def flush_partial(final: bool = False) -> None:
+ nonlocal pending, pending_rows, out_pos, total_rows
+ while pending_rows >= shard_rows or (final and pending_rows > 0):
+ block = np.vstack(pending) if len(pending) > 1 else pending[0]
+ take = block.shape[0] if final and pending_rows < shard_rows else shard_rows
+ piece = block[:take]
+ if dtype is not None:
+ piece = piece.astype(dtype, copy=False)
+ dst[out_pos : out_pos + piece.shape[0], :] = piece
+ out_pos += piece.shape[0]
+ total_rows += piece.shape[0]
+ if block.shape[0] > take:
+ pending = [block[take:]]
+ pending_rows = block.shape[0] - take
+ else:
+ pending = []
+ pending_rows = 0
+ if not final and pending_rows < shard_rows:
+ break
+
+ for batch in tqdmbar(batches, desc=msg or "Writing Zarr array"):
+ batch = np.asarray(batch)
+ if batch.size == 0:
+ continue
+ pending.append(batch)
+ pending_rows += batch.shape[0]
+ flush_partial()
+ flush_partial(final=True)
+ return total_rows
+
+
+def write_dense_in_shard_rows(
+ dst: zarr.Array,
+ produce: Callable[[int, int], np.ndarray],
+ *,
+ msg: str | None = None,
+ shard_rows: int | None = None,
+ also_write_to: zarr.Array | None = None,
+) -> None:
+ """Write a 2D array in row-shard bands via ``produce(start, end)``.
+
+ The produce side (read + compute) runs in order with a shallow read-ahead so
+ the next band is fetched while the current one is written; the writes stay
+ single-threaded so we never race on a shared chunk/shard file regardless of
+ how the caller's band size aligns to the on-disk geometry.
+
+ ``also_write_to`` mirrors each produced band into a second array of the same
+ shape during the same pass. This populates a local staging cache while
+ writing to a remote store, so the normalized matrix never has to be read
+ back over the network.
+ """
+ from ..utils import tqdmbar
+ from ..parallel import stream_shards
+ from .budget import shard_parallelism
+
+ n_rows = int(dst.shape[0])
+ if n_rows == 0:
+ return None
+ if shard_rows is None:
+ shard_rows = array_shard_rows(dst)
+ slices = list(iter_shard_row_slices(n_rows, shard_rows))
+ if msg is None:
+ msg = "Writing Zarr array"
+
+ plan = shard_parallelism(n_shards=len(slices))
+
+ def produce_band(bounds: tuple[int, int]) -> tuple[int, int, np.ndarray]:
+ start, end = bounds
+ return start, end, np.asarray(produce(start, end))
+
+ produced = stream_shards(
+ slices,
+ produce_band,
+ workers=plan.readAhead,
+ within_block_threads=plan.withinBlockThreads,
+ io_concurrency=plan.ioConcurrency,
+ )
+ for start, end, block in tqdmbar(produced, desc=msg, total=len(slices)):
+ dst[start:end, :] = block
+ if also_write_to is not None:
+ also_write_to[start:end, :] = block
+ return None
+
+
+def accumulate_sparse_to_shards(
+ dst: zarr.Array,
+ data_stream: Iterator[Any],
+ *,
+ shard_rows: int | None = None,
+ dtype: Any | None = None,
+) -> int:
+ """Buffer sparse COO batches and write one coordinate selection per row shard."""
+ from scipy.sparse import coo_matrix
+
+ if shard_rows is None:
+ shard_rows = array_shard_rows(dst)
+ shard_rows = max(1, int(shard_rows))
+ if dtype is None:
+ dtype = dst.dtype
+
+ s = 0
+ buf_row: list[np.ndarray] = []
+ buf_col: list[np.ndarray] = []
+ buf_data: list[np.ndarray] = []
+ next_flush = shard_rows
+
+ def _concat_or_empty(parts: list[np.ndarray]) -> np.ndarray:
+ if not parts:
+ return np.array([], dtype=np.int64)
+ return np.concatenate(parts)
+
+ def flush_through(end_row: int) -> None:
+ nonlocal next_flush, buf_row, buf_col, buf_data
+ while next_flush <= end_row:
+ band_start = next_flush - shard_rows
+ row = _concat_or_empty(buf_row)
+ col = _concat_or_empty(buf_col)
+ data = _concat_or_empty(buf_data)
+ if row.size:
+ mask = (row >= band_start) & (row < next_flush)
+ if mask.any():
+ dst.set_coordinate_selection(
+ (row[mask], col[mask]),
+ data[mask].astype(dtype, copy=False),
+ )
+ keep = row >= next_flush
+ if keep.any():
+ buf_row = [row[keep]]
+ buf_col = [col[keep]]
+ buf_data = [data[keep]]
+ else:
+ buf_row = []
+ buf_col = []
+ buf_data = []
+ next_flush += shard_rows
+
+ for batch in data_stream:
+ coo = batch if hasattr(batch, "row") else coo_matrix(batch)
+ if coo.shape[0] == 0:
+ continue
+ if coo.nnz:
+ buf_row.append(np.asarray(coo.row, dtype=np.int64) + s)
+ buf_col.append(np.asarray(coo.col, dtype=np.int64))
+ buf_data.append(np.asarray(coo.data))
+ s += coo.shape[0]
+ flush_through(s)
+
+ if buf_row:
+ row = _concat_or_empty(buf_row)
+ col = _concat_or_empty(buf_col)
+ data = _concat_or_empty(buf_data)
+ if row.size:
+ dst.set_coordinate_selection((row, col), data.astype(dtype, copy=False))
+ return s
+
+
+def is_remote_zarr_location(location: str) -> bool:
+ """Return True when ``location`` is a non-local URI (e.g. s3://, gs://)."""
+ if "://" not in location:
+ return False
+ return not location.startswith("file://")
+
+
+def is_local_zarr_path(location: ZarrLocation) -> bool:
+ """Return True when ``location`` is a plain local filesystem path string."""
+ return isinstance(location, str) and not is_remote_zarr_location(location)
+
+
+def is_remote_datastore(zarr_loc: ZarrLocation, group: zarr.Group) -> bool:
+ """Return True when the datastore primary store is a remote/object backend."""
+ if isinstance(zarr_loc, str):
+ return is_remote_zarr_location(zarr_loc)
+ if zarr_root_path(group) is not None:
+ return False
+ store_name = type(group.store).__name__
+ if store_name in ("MemoryStore", "LocalStore"):
+ return False
+ return True
+
+
+def copy_zarr_array(
+ src: zarr.Array,
+ dst: zarr.Array,
+ block_rows: int | None = None,
+ msg: str | None = None,
+) -> None:
+ """Stream-copy a 2D Zarr array in row blocks."""
+ if src.shape != dst.shape:
+ raise ValueError(f"Shape mismatch: src {src.shape} vs dst {dst.shape}")
+ if len(src.shape) != 2:
+ raise ValueError("copy_zarr_array only supports 2D arrays")
+ if block_rows is None:
+ block_rows = array_shard_rows(dst)
+ write_dense_in_shard_rows(
+ dst,
+ lambda start, end: np.asarray(src[start:end, :]),
+ msg=msg or "Copying Zarr array",
+ shard_rows=block_rows,
+ )
+ return None
+
+
+def copy_zarr_group_tree(
+ src: zarr.Group,
+ dst: zarr.Group,
+ *,
+ overwrite: bool = True,
+) -> None:
+ """Recursively copy a Zarr group tree from ``src`` into ``dst``."""
+ from ..writers import create_zarr_obj_array
+
+ for name, node in src.members():
+ if isinstance(node, zarr.Group):
+ child = dst.create_group(name, overwrite=overwrite)
+ copy_zarr_group_tree(node, child, overwrite=overwrite)
+ else:
+ arr = as_zarr_array(node, name=name)
+ create_zarr_obj_array(
+ dst,
+ name,
+ np.asarray(arr[:]),
+ dtype=arr.dtype,
+ overwrite=overwrite,
+ )
+
+
+def create_or_open_staged_normed_array(
+ cache_path: str,
+ shape: tuple[int, int],
+) -> zarr.Array:
+ """Open (reusing when shape matches) a local scratch array of ``shape``."""
+ import os
+
+ if os.path.exists(os.path.join(cache_path, "zarr.json")):
+ root = zarr.open_group(cache_path, mode="r+")
+ if "data" in root:
+ arr = as_zarr_array(root["data"], name="data")
+ if tuple(arr.shape) == tuple(shape):
+ return arr
+ root = zarr.open_group(cache_path, mode="w")
+ spec = normed_array_spec(shape[0], shape[1], profile="fast_local")
+ return create_numeric_array(root, "data", spec)
+
+
+def open_or_create_staged_normed_array(
+ cache_path: str,
+ src: zarr.Array,
+) -> zarr.Array:
+ """Open a reusable local scratch array matching ``src``'s shape."""
+ return create_or_open_staged_normed_array(
+ cache_path, (int(src.shape[0]), int(src.shape[1]))
+ )
+
+
+def _is_obstore_native_store(obj: object) -> bool:
+ return type(obj).__module__.startswith("obstore.")
+
+
+def _maybe_auto_cloud_profile(location: ZarrLocation) -> None:
+ """Select the cloud storage profile when opening a remote store without an explicit profile."""
+ if _activeProfile is not None:
+ return
+ if isinstance(location, str) and not is_remote_zarr_location(location):
+ return
+ if isinstance(location, Store):
+ return
+ set_storage_profile("cloud")
+
+
+def make_store(
+ location: ZarrLocation,
+ *,
+ storage_options: dict[str, Any] | None = None,
+ read_only: bool = False,
+) -> str | Store:
+ """Resolve a path, URI, or store object into something ``zarr.open_group`` accepts."""
+ if isinstance(location, Store):
+ return location
+
+ if _is_obstore_native_store(location):
+ from zarr.storage import ObjectStore
+
+ return ObjectStore(store=location, read_only=read_only) # type: ignore[type-var]
+
+ if isinstance(location, str):
+ if is_remote_zarr_location(location):
+ try:
+ from obstore.store import from_url as obstore_from_url
+ from zarr.storage import ObjectStore
+ except ImportError as exc:
+ raise ImportError("Remote Zarr stores require obstore.") from exc
+ _maybe_auto_cloud_profile(location)
+ obstore = obstore_from_url(location, **(storage_options or {}))
+ return ObjectStore(store=obstore, read_only=read_only) # type: ignore[type-var]
+ return location
+
+ raise TypeError(
+ f"zarr location must be a path string or zarr Store, got {type(location)!r}"
+ )
+
+
+def open_store(
+ path: ZarrLocation,
+ mode: ZarrMode = "r",
+ storage_options: dict[str, Any] | None = None,
+) -> zarr.Group:
+ """Open a Zarr group at ``path`` or from a store object."""
+ store = make_store(path, storage_options=storage_options, read_only=(mode == "r"))
+ configure_zarr_io_for_profile()
+ if isinstance(store, str):
+ return zarr.open_group(store, mode=mode)
+ return zarr.open_group(store=store, mode=mode)
+
+
+def create_numeric_array(
+ group: zarr.Group,
+ name: str,
+ spec: ZarrArraySpec,
+) -> zarr.Array:
+ """Create a numeric Zarr array from a ``ZarrArraySpec``."""
+ zarrFormat = _group_zarr_format(group)
+ chunks = normalize_chunks(spec.chunks, spec.shape)
+ profile = get_storage_profile()
+ compressors = get_compressors(profile, zarrFormat=zarrFormat)
+ if spec.compressors is not None and zarrFormat >= 3:
+ compressors = spec.compressors
+ kwargs: dict[str, Any] = {
+ "shape": spec.shape,
+ "chunks": chunks,
+ "dtype": spec.dtype,
+ "compressors": compressors,
+ "overwrite": spec.overwrite,
+ }
+ if spec.shards is not None and zarrFormat >= 3:
+ kwargs["shards"] = spec.shards
+ if spec.fillValue is not None:
+ kwargs["fill_value"] = spec.fillValue
+ return group.create_array(name, **kwargs)
+
+
+def dtype_fix(dtype: Any, data: np.ndarray) -> Any:
+ """Infer or adjust dtype for metadata arrays from sample values."""
+ if dtype is None or np.dtype(dtype).kind == "O":
+ return "U" + str(max([len(str(x)) for x in data]))
+ if np.issubdtype(data.dtype, np.dtype("S")):
+ try:
+ adata = data.astype("U")
+ except UnicodeDecodeError:
+ adata = np.array([x.decode("UTF-8") for x in data]).astype("U")
+ return adata.dtype
+ return dtype
+
+
+def create_metadata_column(
+ group: zarr.Group,
+ name: str,
+ data: np.ndarray | list | None = None,
+ dtype: Any = None,
+ overwrite: bool = True,
+ chunkSize: int | bool | None = None,
+ shape: int | None = None,
+ profile: StorageProfile | None = None,
+) -> zarr.Array:
+ """Create a 1D metadata column, optionally from provided data."""
+ if chunkSize is None or chunkSize is False:
+ chunks: tuple[int, ...] | bool = False
+ else:
+ chunks = (chunkSize,)
+
+ compressors = get_compressors(
+ profile or get_storage_profile(),
+ zarrFormat=_group_zarr_format(group),
+ )
+
+ if data is not None:
+ data = np.array(data)
+ data = np.asarray(data, dtype=dtype_fix(dtype, data))
+ if chunks is False:
+ chunks = (len(data),)
+ return group.create_array(
+ name,
+ data=data,
+ chunks=chunks,
+ overwrite=overwrite,
+ compressors=compressors,
+ )
+
+ if shape is None:
+ raise ValueError("shape is required when data is None")
+ if chunks is False:
+ chunks = (shape,)
+ return group.create_array(
+ name,
+ shape=(shape,),
+ chunks=chunks,
+ dtype=dtype,
+ overwrite=overwrite,
+ compressors=compressors,
+ )
+
+
+def repack_to_sharded(
+ srcArray: zarr.Array,
+ dstGroup: zarr.Group,
+ name: str,
+ shards: tuple[int, ...],
+ chunks: tuple[int, ...] | None = None,
+ profile: StorageProfile | None = None,
+ overwrite: bool = True,
+) -> zarr.Array:
+ """Copy ``srcArray`` into a new sharded array in ``dstGroup``."""
+ chunks = chunks or tuple(srcArray.chunks)
+ spec = ZarrArraySpec(
+ shape=srcArray.shape,
+ chunks=chunks,
+ shards=shards,
+ dtype=srcArray.dtype,
+ compressors=get_compressors(profile or get_storage_profile()),
+ overwrite=overwrite,
+ )
+ dstArray = create_numeric_array(dstGroup, name, spec)
+ shardRows = int(shards[0])
+ write_dense_in_shard_rows(
+ dstArray,
+ lambda start, end: np.asarray(srcArray[start:end, :]),
+ msg="Repacking to sharded layout",
+ shard_rows=shardRows,
+ )
+ return dstArray
+
+
+def finalize_sharded_counts(
+ store: zarr.Group,
+ assayName: str,
+ workspace: str | None = None,
+ profile: StorageProfile | None = None,
+) -> zarr.Array:
+ """Repack assay counts to sharded layout when not already sharded."""
+ profile = profile or get_storage_profile()
+ if workspace is None:
+ countsPath = f"{assayName}/counts"
+ assayGroup = as_zarr_group(store[assayName], name=assayName)
+ else:
+ countsPath = f"matrices/{assayName}/counts"
+ assayGroup = as_zarr_group(store[f"matrices/{assayName}"], name=assayName)
+
+ srcArray = as_zarr_array(store[countsPath], name=countsPath)
+ if array_metadata_shards(srcArray) is not None:
+ return srcArray
+ if _group_zarr_format(store) == 2:
+ return srcArray
+
+ spec = count_array_spec(
+ srcArray.shape[0],
+ srcArray.shape[1],
+ dtype=srcArray.dtype,
+ profile=profile,
+ )
+ shards = spec.shards
+ chunks = spec.chunks
+ assert shards is not None and chunks is not None
+
+ if shards == srcArray.shape:
+ return srcArray
+ tmpName = "counts__sharded_tmp"
+ if tmpName in assayGroup:
+ del assayGroup[tmpName]
+ repack_to_sharded(
+ srcArray,
+ assayGroup,
+ tmpName,
+ shards=shards,
+ chunks=chunks,
+ profile=profile,
+ )
+ del assayGroup["counts"]
+ repack_to_sharded(
+ as_zarr_array(assayGroup[tmpName], name=tmpName),
+ assayGroup,
+ "counts",
+ shards=shards,
+ chunks=chunks,
+ profile=profile,
+ )
+ del assayGroup[tmpName]
+
+ if workspace is None:
+ assayGroup = as_zarr_group(store[assayName], name=assayName)
+ else:
+ assayGroup = as_zarr_group(store[f"matrices/{assayName}"], name=assayName)
+ assayGroup.attrs["scarf:zarr_spec"] = {
+ "profile": profile,
+ "chunks": list(chunks),
+ "shards": list(shards),
+ "zarr_format": 3,
+ }
+ return as_zarr_array(store[countsPath], name=countsPath)
+
+
+def array_info(array: zarr.Array) -> str:
+ """Return a short summary string for a Zarr array."""
+ parts = [f"shape={array.shape}", f"dtype={array.dtype}", f"chunks={array.chunks}"]
+ shards_meta = array_metadata_shards(array)
+ if shards_meta is not None:
+ parts.append(f"shards={shards_meta}")
+ return ", ".join(parts)
+
+
+ANN_INDEX_ARRAY = "ann_idx_bytes"
+ANN_INDEX_FORMAT = "zarr-uint8-v1"
+ANN_INDEX_CHUNK_BYTES = 8 * 1024 * 1024
+
+
+def has_ann_index(group: zarr.Group, name: str = ANN_INDEX_ARRAY) -> bool:
+ """Return True when an in-zarr ANN index byte array exists."""
+ return name in group
+
+
+def legacy_ann_index_path(zw_root: str | None, ann_loc: str) -> str | None:
+ """Return filesystem path to a legacy hnswlib sibling index file, if applicable."""
+ if zw_root is None:
+ return None
+ return os.path.join(zw_root, ann_loc, "ann_idx")
+
+
+def save_ann_index(
+ group: zarr.Group,
+ ann_idx: Any,
+ name: str = ANN_INDEX_ARRAY,
+ profile: StorageProfile | None = None,
+) -> None:
+ """Persist an hnswlib index as a chunked uint8 array inside ``group``."""
+ import tempfile
+
+ with tempfile.NamedTemporaryFile(delete=False) as tmp:
+ path = tmp.name
+ try:
+ ann_idx.save_index(path)
+ data = np.fromfile(path, dtype=np.uint8)
+ finally:
+ os.unlink(path)
+
+ if name in group:
+ del group[name]
+ chunk_size = min(ANN_INDEX_CHUNK_BYTES, max(int(data.shape[0]), 1))
+ zarr_format = _group_zarr_format(group)
+ arr = group.create_array(
+ name,
+ shape=data.shape,
+ chunks=(chunk_size,),
+ dtype="uint8",
+ overwrite=True,
+ compressors=get_compressors(
+ profile or get_storage_profile(),
+ zarrFormat=zarr_format,
+ ),
+ )
+ arr[:] = data
+ group.attrs["annIndexFormat"] = ANN_INDEX_FORMAT
+ arr.attrs["byteLength"] = int(data.shape[0])
+
+
+def load_ann_index(
+ group: zarr.Group,
+ space: str,
+ dim: int,
+ name: str = ANN_INDEX_ARRAY,
+) -> Any:
+ """Load an hnswlib index from an in-zarr uint8 byte array."""
+ import tempfile
+
+ import hnswlib
+
+ if name not in group:
+ raise FileNotFoundError(f"ANN index array {name!r} not found in group")
+ data = np.asarray(as_zarr_array(group[name], name=name)[:], dtype=np.uint8)
+ with tempfile.NamedTemporaryFile(delete=False) as tmp:
+ path = tmp.name
+ try:
+ data.tofile(path)
+ idx = hnswlib.Index(space=space, dim=dim)
+ idx.load_index(path)
+ return idx
+ finally:
+ os.unlink(path)
+
+
+def load_ann_index_from_path(path: str, space: str, dim: int) -> Any:
+ """Load an hnswlib index from a legacy filesystem path."""
+ import hnswlib
+
+ idx = hnswlib.Index(space=space, dim=dim)
+ idx.load_index(path)
+ return idx
diff --git a/scarf/symphony.py b/scarf/symphony.py
new file mode 100644
index 00000000..25fdbd37
--- /dev/null
+++ b/scarf/symphony.py
@@ -0,0 +1,234 @@
+"""Portable numerical primitives for Symphony-style reference mapping."""
+
+from dataclasses import dataclass
+from typing import cast
+
+import numpy as np
+
+SYMPHONY_STYLE_VARIANT = "symphonyStyleV1"
+
+
+@dataclass(frozen=True)
+class SymphonyReferenceModel:
+ feature_means: np.ndarray
+ feature_scales: np.ndarray
+ loadings: np.ndarray
+ centroids: np.ndarray
+ raw_centroids: np.ndarray
+ corrected_centroids: np.ndarray
+ cluster_mass: np.ndarray
+ sigma: np.ndarray
+ correction_ridge: float
+
+ def __post_init__(self) -> None:
+ n_features, n_dims = self.loadings.shape
+ n_clusters = self.centroids.shape[0]
+ if self.feature_means.shape != (n_features,):
+ raise ValueError("Reference feature means have incompatible dimensions")
+ if self.feature_scales.shape != (n_features,):
+ raise ValueError("Reference feature scales have incompatible dimensions")
+ if np.any(self.feature_scales <= 0):
+ raise ValueError("Reference feature scales must be positive")
+ if self.centroids.shape != (n_clusters, n_dims):
+ raise ValueError("Reference centroids have incompatible dimensions")
+ if self.raw_centroids.shape != (n_clusters, n_dims):
+ raise ValueError("Reference raw centroids have incompatible dimensions")
+ if self.corrected_centroids.shape != (n_clusters, n_dims):
+ raise ValueError(
+ "Reference corrected centroids have incompatible dimensions"
+ )
+ if self.cluster_mass.shape != (n_clusters,) or np.any(self.cluster_mass <= 0):
+ raise ValueError("Reference cluster masses must be positive")
+ if self.sigma.shape != (n_clusters,) or np.any(self.sigma <= 0):
+ raise ValueError("Reference kernel widths must be positive")
+ if self.correction_ridge < 0:
+ raise ValueError("Reference correction ridge must be non-negative")
+ for values in (
+ self.feature_means,
+ self.feature_scales,
+ self.loadings,
+ self.centroids,
+ self.raw_centroids,
+ self.corrected_centroids,
+ self.cluster_mass,
+ self.sigma,
+ ):
+ if not np.all(np.isfinite(values)):
+ raise ValueError("Reference model contains non-finite values")
+
+ @property
+ def n_features(self) -> int:
+ return int(self.loadings.shape[0])
+
+ @property
+ def n_dims(self) -> int:
+ return int(self.loadings.shape[1])
+
+ @property
+ def n_clusters(self) -> int:
+ return int(self.centroids.shape[0])
+
+
+@dataclass(frozen=True)
+class QueryCorrection:
+ batch_offsets: np.ndarray
+ batch_counts: np.ndarray
+
+ def __post_init__(self) -> None:
+ if self.batch_offsets.ndim != 3:
+ raise ValueError(
+ "Batch offsets must have batch, cluster, and dimension axes"
+ )
+ if self.batch_counts.shape != self.batch_offsets.shape[:2]:
+ raise ValueError("Batch counts must match batch offsets")
+ if not np.all(np.isfinite(self.batch_offsets)):
+ raise ValueError("Batch offsets contain non-finite values")
+
+
+def project_pca(values: np.ndarray, model: SymphonyReferenceModel) -> np.ndarray:
+ """Project normalized expression onto immutable reference PCA loadings."""
+ values = np.asarray(values, dtype=np.float64)
+ if values.ndim != 2 or values.shape[1] != model.n_features:
+ raise ValueError(
+ f"Expected query matrix with {model.n_features} features, got {values.shape}"
+ )
+ projected = ((values - model.feature_means) / model.feature_scales) @ model.loadings
+ if not np.all(np.isfinite(projected)):
+ raise ValueError("PCA projection produced non-finite values")
+ return cast(np.ndarray, projected)
+
+
+def soft_cluster_assignments(
+ coordinates: np.ndarray, model: SymphonyReferenceModel
+) -> np.ndarray:
+ """Calculate cosine-kernel soft assignments to reference centroids."""
+ values = np.asarray(coordinates, dtype=np.float64)
+ if values.ndim != 2 or values.shape[1] != model.n_dims:
+ raise ValueError("Query coordinates have incompatible dimensions")
+ query_unit = _normalize_rows(values)
+ centroids_unit = _normalize_rows(model.centroids)
+ distances = 2.0 * (1.0 - query_unit @ centroids_unit.T)
+ logits = -distances / model.sigma[np.newaxis, :]
+ logits -= logits.max(axis=1, keepdims=True)
+ assignments = np.exp(logits)
+ assignments /= assignments.sum(axis=1, keepdims=True)
+ if not np.all(np.isfinite(assignments)):
+ raise ValueError("Soft cluster assignments contain non-finite values")
+ return cast(np.ndarray, assignments)
+
+
+def zero_norm_rows(coordinates: np.ndarray) -> np.ndarray:
+ """Return rows without directional information in reference PC space."""
+ values = np.asarray(coordinates, dtype=np.float64)
+ if values.ndim != 2:
+ raise ValueError("Query coordinates must be two-dimensional")
+ return cast(np.ndarray, np.linalg.norm(values, axis=1) == 0)
+
+
+def initialize_sufficient_statistics(
+ n_batches: int, model: SymphonyReferenceModel
+) -> tuple[np.ndarray, np.ndarray]:
+ if n_batches < 1:
+ raise ValueError("At least one query batch is required")
+ return (
+ np.zeros((n_batches, model.n_clusters), dtype=np.float64),
+ np.zeros((n_batches, model.n_clusters, model.n_dims), dtype=np.float64),
+ )
+
+
+def accumulate_sufficient_statistics(
+ counts: np.ndarray,
+ sums: np.ndarray,
+ coordinates: np.ndarray,
+ assignments: np.ndarray,
+ batch_codes: np.ndarray,
+) -> None:
+ """Accumulate per-query-batch, per-cluster weighted coordinate sums."""
+ if coordinates.shape[0] != len(batch_codes) or assignments.shape[0] != len(
+ batch_codes
+ ):
+ raise ValueError("Query batch codes must match coordinate rows")
+ if assignments.shape[1] != counts.shape[1]:
+ raise ValueError("Assignment count does not match reference clusters")
+ if np.any(batch_codes < 0) or np.any(batch_codes >= counts.shape[0]):
+ raise ValueError("Query batch codes are out of range")
+
+ for batch_code in np.unique(batch_codes):
+ batch_mask = batch_codes == batch_code
+ batch_assignments = assignments[batch_mask]
+ counts[batch_code] += batch_assignments.sum(axis=0)
+ sums[batch_code] += batch_assignments.T @ coordinates[batch_mask]
+
+
+def solve_query_correction(
+ counts: np.ndarray,
+ sums: np.ndarray,
+ model: SymphonyReferenceModel,
+) -> QueryCorrection:
+ """Fit cluster-aware batch offsets against reference raw centroids."""
+ if counts.ndim != 2 or counts.shape[1] != model.n_clusters:
+ raise ValueError("Query count statistics have incompatible dimensions")
+ if sums.shape != (counts.shape[0], model.n_clusters, model.n_dims):
+ raise ValueError("Query sum statistics have incompatible dimensions")
+ expected = counts[:, :, np.newaxis] * model.raw_centroids[np.newaxis, :, :]
+ median_mass = max(float(np.median(model.cluster_mass)), 1.0)
+ reference_regularization = model.correction_ridge * model.cluster_mass / median_mass
+ denominator = (
+ counts[:, :, np.newaxis] + reference_regularization[np.newaxis, :, np.newaxis]
+ )
+ offsets = np.divide(
+ sums - expected,
+ denominator,
+ out=np.zeros_like(sums),
+ where=denominator > 0,
+ )
+ return QueryCorrection(batch_offsets=offsets, batch_counts=counts.copy())
+
+
+def apply_query_correction(
+ coordinates: np.ndarray,
+ assignments: np.ndarray,
+ batch_codes: np.ndarray,
+ model: SymphonyReferenceModel,
+ correction: QueryCorrection,
+) -> np.ndarray:
+ """Map query PCA coordinates into the fixed corrected reference space."""
+ if coordinates.shape[0] != assignments.shape[0] or coordinates.shape[0] != len(
+ batch_codes
+ ):
+ raise ValueError("Query coordinate, assignment, and batch rows must agree")
+ if assignments.shape[1] != model.n_clusters:
+ raise ValueError("Query assignments have incompatible cluster count")
+ if correction.batch_offsets.shape[1:] != (model.n_clusters, model.n_dims):
+ raise ValueError("Query correction has incompatible reference dimensions")
+ query_offsets = np.einsum(
+ "nk,nkd->nd", assignments, correction.batch_offsets[batch_codes]
+ )
+ reference_shift = assignments @ (model.raw_centroids - model.corrected_centroids)
+ corrected = coordinates - query_offsets - reference_shift
+ if not np.all(np.isfinite(corrected)):
+ raise ValueError("Query correction produced non-finite coordinates")
+ return cast(np.ndarray, corrected)
+
+
+def weighted_centroids(
+ coordinates: np.ndarray, assignments: np.ndarray
+) -> tuple[np.ndarray, np.ndarray]:
+ """Return soft cluster masses and centroids for reference compression."""
+ values = np.asarray(coordinates, dtype=np.float64)
+ weights = np.asarray(assignments, dtype=np.float64)
+ if values.ndim != 2 or weights.ndim != 2 or values.shape[0] != weights.shape[1]:
+ raise ValueError("Coordinate and assignment dimensions do not agree")
+ mass = weights.sum(axis=1)
+ if np.any(mass <= 0):
+ raise ValueError("Reference assignments include an empty cluster")
+ centroids = (weights @ values / mass[:, np.newaxis]).astype(np.float64)
+ return mass, centroids
+
+
+def _normalize_rows(values: np.ndarray) -> np.ndarray:
+ norms = np.linalg.norm(values, axis=1, keepdims=True)
+ normalized = np.zeros_like(values, dtype=np.float64)
+ nonzero = norms[:, 0] > 0
+ normalized[nonzero] = values[nonzero] / norms[nonzero]
+ return normalized
diff --git a/scarf/tests/__init__.py b/scarf/tests/__init__.py
deleted file mode 100644
index 8794fadc..00000000
--- a/scarf/tests/__init__.py
+++ /dev/null
@@ -1,26 +0,0 @@
-import os
-import shutil
-import sys
-
-from ..utils import logger
-
-logger.remove()
-logger.add(sys.stderr, level="ERROR")
-
-__all__ = ["full_path", "remove"]
-
-
-def full_path(fn, *args):
- if fn == "" or fn is None:
- return os.path.join("scarf", "tests", "datasets")
- else:
- return os.path.join("scarf", "tests", "datasets", fn, *args)
-
-
-def remove(dir_path):
- if os.path.isdir(dir_path):
- shutil.rmtree(dir_path)
- elif os.path.exists(dir_path):
- os.unlink(dir_path)
- else:
- pass
diff --git a/scarf/tests/conftest.py b/scarf/tests/conftest.py
deleted file mode 100644
index 1895758a..00000000
--- a/scarf/tests/conftest.py
+++ /dev/null
@@ -1,4 +0,0 @@
-from .fixtures_downloader import *
-from .fixtures_readers import *
-from .fixtures_datastore import *
-
diff --git a/scarf/tests/datasets/1K_pbmc_citeseq.h5 b/scarf/tests/datasets/1K_pbmc_citeseq.h5
deleted file mode 100644
index 81315733..00000000
Binary files a/scarf/tests/datasets/1K_pbmc_citeseq.h5 and /dev/null differ
diff --git a/scarf/tests/datasets/1K_pbmc_citeseq.zarr.tar.gz b/scarf/tests/datasets/1K_pbmc_citeseq.zarr.tar.gz
deleted file mode 100644
index b8666399..00000000
Binary files a/scarf/tests/datasets/1K_pbmc_citeseq.zarr.tar.gz and /dev/null differ
diff --git a/scarf/tests/datasets/500_pbmc_atac.zarr.tar.gz b/scarf/tests/datasets/500_pbmc_atac.zarr.tar.gz
deleted file mode 100644
index eb718d46..00000000
Binary files a/scarf/tests/datasets/500_pbmc_atac.zarr.tar.gz and /dev/null differ
diff --git a/scarf/tests/datasets/aggregated_df_top_10.npy b/scarf/tests/datasets/aggregated_df_top_10.npy
deleted file mode 100644
index 6009594f..00000000
Binary files a/scarf/tests/datasets/aggregated_df_top_10.npy and /dev/null differ
diff --git a/scarf/tests/datasets/aggregated_feat_idx.npy b/scarf/tests/datasets/aggregated_feat_idx.npy
deleted file mode 100644
index 3b66bada..00000000
Binary files a/scarf/tests/datasets/aggregated_feat_idx.npy and /dev/null differ
diff --git a/scarf/tests/datasets/atac_knn_distances.npy b/scarf/tests/datasets/atac_knn_distances.npy
deleted file mode 100644
index 325bc480..00000000
Binary files a/scarf/tests/datasets/atac_knn_distances.npy and /dev/null differ
diff --git a/scarf/tests/datasets/atac_knn_indices.npy b/scarf/tests/datasets/atac_knn_indices.npy
deleted file mode 100644
index e7853d17..00000000
Binary files a/scarf/tests/datasets/atac_knn_indices.npy and /dev/null differ
diff --git a/scarf/tests/datasets/cell_attributes.csv b/scarf/tests/datasets/cell_attributes.csv
deleted file mode 100644
index 313fe368..00000000
--- a/scarf/tests/datasets/cell_attributes.csv
+++ /dev/null
@@ -1,809 +0,0 @@
-,I,ids,names,PTIME_MODULES_nCounts,PTIME_MODULES_nFeatures,RNA_G2M_score,RNA_S_score,RNA_UMAP1,RNA_UMAP2,RNA_balanced_clusters,RNA_cell_cycle_phase,RNA_cell_density,RNA_cluster,RNA_leiden_cluster,RNA_nCounts,RNA_nFeatures,RNA_percentMito,RNA_percentRibo,RNA_pseudotime,RNA_sketch_seeds,RNA_sketched,RNA_snn_value,RNA_unified_UMAP1,RNA_unified_UMAP2,assay2_nCounts,assay2_nFeatures,target_classes,mapping_scores
-0,True,AATCACGAGCAGCCCT-1,AATCACGAGCAGCCCT-1,1.9146028840632483,15.0,-0.09262390055430042,-0.019684164407220917,2.6459084,6.8622274,6,G1,1635.3163743019104,1,1,3058.0,1020.0,6.8999345977763245,41.79202092871158,0.6149374478797888,False,False,3.4,10.589231,10.660447,1651.0,9.0,1,0.8054144824310264
-1,True,AATCACGAGGAACTCG-1,AATCACGAGGAACTCG-1,1.9429760410381736,15.0,-0.025279845838156366,-0.02296731799515923,1.8557484,8.659906,27,G1,1331.7302141338587,8,1,2176.0,975.0,18.841911764705884,14.108455882352942,0.6176216869864952,False,False,3.0,10.040825,12.057594,788.0,9.0,8,0.8054144824310264
-2,True,AATCACGCACTACCGG-1,AATCACGCACTACCGG-1,1.8578043969550482,15.0,-0.08864835706481089,-0.0076830777206780415,9.69295,-11.369626,21,G1,1278.1951800137758,7,6,5536.0,2120.0,5.816473988439307,19.436416184971097,0.04650833633018682,False,False,6.799999999999999,12.0010605,-10.761246,1305.0,9.0,7,0.8054144824310264
-3,True,AATCACGCATGAATAG-1,AATCACGCATGAATAG-1,1.880634640338231,15.0,-0.11152050902681884,-0.006583769451145836,6.977848,2.8306186,8,G1,1306.5597768723965,9,9,5077.0,1462.0,4.648414417963364,45.02659050620445,0.6112025923359019,False,False,2.3,13.427173,7.700363,1007.0,9.0,9,0.9808893821763591
-4,True,AATCACGGTATAGGAT-1,AATCACGGTATAGGAT-1,1.901891912646523,15.0,-0.10505539901191005,-0.0037932092540785417,8.251231,-2.7979677,22,G1,1164.8476556241512,6,8,7386.0,2097.0,8.434876793934471,38.044949905226105,0.45369430937882754,False,False,6.8999999999999995,14.396282,0.054674238,928.0,9.0,6,1.3206616453813353
-5,True,AATCACGGTCGAATGG-1,AATCACGGTCGAATGG-1,1.9259363130711664,15.0,-0.07483857271001484,-0.012962702620913924,2.8696203,6.2859616,5,G1,1541.134561829269,1,1,3966.0,1333.0,14.246091780131115,30.963187090267272,0.6116533853041001,False,False,2.7,9.603099,8.927263,1428.0,10.0,1,0.8054144824310264
-6,True,AATCACGGTGTCTAAC-1,AATCACGGTGTCTAAC-1,1.879618727631841,15.0,-0.10944414568326681,0.0010908939374005798,2.4051206,7.9692683,5,S,1546.4653607159853,1,1,6357.0,1541.0,5.647317917256568,44.81673745477426,0.612388298156135,False,False,2.9,8.690099,10.703329,1077.0,9.0,1,1.0102850184689025
-7,True,AATCACGTCCACGTAA-1,AATCACGTCCACGTAA-1,1.8973074885383008,15.0,-0.09167494807424922,-0.00853403836255738,5.4800663,6.8413477,12,G1,1352.1825319975615,1,1,5737.0,1506.0,5.508105281506014,43.907965835802685,0.6130000705872367,False,False,4.800000000000001,9.873918,7.695063,895.0,9.0,1,0.9817706108534437
-8,True,AATCACGTCCCGTTGT-1,AATCACGTCCCGTTGT-1,1.9033433854258006,15.0,-0.0902786072188888,-0.012823941075421428,5.4946275,0.32944006,22,G1,1254.204477339983,6,8,5332.0,1742.0,9.808702175543885,33.514628657164295,0.5770699260800276,False,False,4.1000000000000005,14.220285,5.354528,1610.0,9.0,6,1.1559159954332647
-9,True,AATCACGTCGCCTTGT-1,AATCACGTCGCCTTGT-1,1.9503836684894997,15.0,-0.0797848840078176,-0.023488315476172913,-9.845075,-5.89527,10,G1,1177.667477890849,3,7,9873.0,3263.0,5.8138357135622405,12.731692494682468,0.9765974809887463,False,False,3.2,-13.166267,-6.8673778,2566.0,9.0,3,1.2323730773035082
-10,True,AATCACGTCGTTTACT-1,AATCACGTCGTTTACT-1,1.9527865874278352,15.0,-0.0910403049385046,-0.024601078042838773,-5.093038,-12.03094,24,G1,1226.4370895773172,10,7,11441.0,3275.0,11.275238178480903,17.515951402849403,0.9631948135077659,False,False,4.0,-8.788839,-10.761385,2437.0,9.0,10,1.0149610188844926
-11,True,AATCACGTCTCACTCG-1,AATCACGTCTCACTCG-1,2.0295826256267238,15.0,-0.03935599284436494,-0.03079673293265527,-0.06250891,8.262804,5,G1,1324.4120434373617,1,1,1875.0,906.0,29.493333333333332,2.506666666666667,0.6268871961484299,False,False,6.3,9.200972,12.995318,896.0,9.0,1,1.5766106259950199
-12,True,ACAGAAAAGACCATTC-1,ACAGAAAAGACCATTC-1,1.9302843479752636,15.0,-0.0854586650258603,-0.027278620576574104,-3.1279614,-13.404911,24,G1,1325.8092089295387,10,7,15316.0,3934.0,5.791329328806477,14.958213632802298,0.9594748838539864,False,False,6.499999999999999,-7.4478264,-11.790138,3837.0,9.0,10,0.9706953108584262
-13,True,ACAGAAAAGCGACATG-1,ACAGAAAAGCGACATG-1,1.9809942190057146,15.0,-0.09065688322869574,-0.03489471870873216,-3.8341582,-6.9765625,3,G1,1216.2443554103374,2,2,11838.0,3446.0,12.341611758743031,21.51545869234668,0.9546716355560309,False,False,3.4,-15.039748,-6.8850813,1651.0,9.0,2,1.0131033238105016
-14,True,ACAGAAAAGGCCTAGA-1,ACAGAAAAGGCCTAGA-1,1.894572635466383,15.0,-0.09623704331549857,-0.021230952960169847,8.448699,-12.429867,21,G1,1194.048403546214,7,6,2195.0,936.0,10.979498861047835,25.10250569476082,0.021458791239320978,False,False,5.5,11.008296,-11.105168,2073.0,9.0,7,0.9877630652298343
-15,True,ACAGAAAAGGTACAGC-1,ACAGAAAAGGTACAGC-1,1.923602176252625,15.0,-0.103005989491423,-0.0035347646575937808,5.3981943,4.3785796,12,G1,1593.3654461428523,1,1,5611.0,1428.0,6.612012119051863,48.72571734093744,0.6113470470095177,False,False,3.1,11.269537,7.7245784,1538.0,9.0,1,1.2110532970525933
-16,True,ACAGAAAAGGTTACAA-1,ACAGAAAAGGTTACAA-1,1.896852546598276,15.0,-0.0979899688715361,-0.012741612046525205,6.908522,10.887742,15,G1,1136.3607185781002,8,10,6738.0,1636.0,6.188780053428317,46.734936182843576,0.6196351581036357,True,True,4.6,15.006257,9.488196,2006.0,9.0,8,0.948632716953345
-17,True,ACAGAAACAATAGTAG-1,ACAGAAACAATAGTAG-1,1.8795965414294988,15.0,-0.10176110771026943,-0.020960593163283227,9.025863,5.55703,26,G1,1130.8305401653051,9,9,4515.0,1495.0,3.521594684385382,40.8859357696567,0.6120454900655272,False,False,4.8999999999999995,10.484926,5.837624,2010.0,9.0,9,1.2591180974451068
-18,True,ACAGAAACACAGAGAC-1,ACAGAAACACAGAGAC-1,1.879690088918888,15.0,-0.09917952087317958,-0.014060675249673244,7.934319,-3.7038977,23,G1,1180.102633073926,6,8,9486.0,2776.0,5.418511490617752,30.212945393211047,0.42960329282772103,False,True,7.1,14.962476,-0.45513263,1368.0,9.0,6,0.8054144824310264
-19,True,ACAGAAACAGACCTAT-1,ACAGAAACAGACCTAT-1,1.9694872213793844,15.0,-0.08439067125224588,-0.014616310813025158,-4.891262,-0.7927549,20,G1,1066.216839298606,4,4,8594.0,2799.0,9.750989062136375,15.289737025831975,0.9644182857009779,False,False,5.4,-17.608807,-0.9948211,4236.0,9.0,4,0.8054144824310264
-20,True,ACAGAAACAGCTACTA-1,ACAGAAACAGCTACTA-1,1.9776234695529329,15.0,-0.08198230238747423,-0.026254721246912242,-9.305638,-1.1854355,7,G1,1635.7103563845158,3,3,12756.0,3422.0,8.490122295390405,18.179680150517402,0.983336135120444,False,False,4.500000000000001,-16.334785,-1.4771385,3079.0,9.0,3,1.118423185428613
-21,True,ACAGAAACATCAGTGT-1,ACAGAAACATCAGTGT-1,1.9143367614442166,15.0,-0.09235357012593028,-0.02194865077749555,11.414043,-8.609801,11,G1,1215.8651418685913,7,6,5485.0,1562.0,8.113035551504103,37.50227894257065,0.15002312810456642,False,True,7.1000000000000005,12.914139,-7.0806255,2756.0,9.0,7,1.2066315300929993
-22,True,ACAGAAAGTCGAATGG-1,ACAGAAAGTCGAATGG-1,1.876593629845063,15.0,-0.08351020492251578,-0.022082887904243932,8.529866,-12.983943,21,G1,1119.1940445750952,7,6,3647.0,1512.0,8.225939128050452,15.958321908417878,0.016788197122685228,False,True,4.3,10.64558,-11.4172125,1972.0,10.0,7,1.1127213340393742
-23,True,ACAGAAAGTCGTATGT-1,ACAGAAAGTCGTATGT-1,1.9328029834907183,15.0,-0.12795666865434305,-0.011959013841320164,0.501181,12.729525,2,G1,850.2403893768787,5,5,5880.0,1650.0,8.503401360544217,42.993197278911566,0.7255904165010446,False,False,6.7,-0.29323268,16.415169,1136.0,9.0,5,1.3311446034738328
-24,True,ACAGAAAGTTGGACCC-1,ACAGAAAGTTGGACCC-1,1.9605230741291146,15.0,-0.06752153878920938,-0.02738148039330419,-7.8785524,0.0218554,14,G1,1255.818872153759,4,4,6151.0,2233.0,9.218013331165665,12.0955942123232,0.9495228655063732,False,False,3.6,-13.418969,-0.72872823,2761.0,9.0,4,1.0511520956857803
-25,True,ACAGAAATCGACGCGT-1,ACAGAAATCGACGCGT-1,1.8916691281157154,15.0,-0.10229398333612602,-0.027564728792835322,7.4776497,-2.934058,22,G1,1195.7869687974453,6,8,5669.0,1804.0,9.931204798024343,31.910389839477862,0.45520109551390225,False,False,7.7,14.815071,0.08478602,1417.0,9.0,6,0.8054144824310264
-26,True,ACAGAAATCTCGAGTA-1,ACAGAAATCTCGAGTA-1,1.883192495165775,15.0,-0.10148617035915784,-0.017039383860827505,7.696917,-1.9375056,22,G1,1027.250883370638,6,8,5980.0,1875.0,7.909698996655519,33.64548494983278,0.49610373452423573,False,False,3.0000000000000004,15.035936,1.5720447,1173.0,9.0,6,0.8054144824310264
-28,True,ACAGAAATCTTAGCCC-1,ACAGAAATCTTAGCCC-1,1.9256684515903995,15.0,-0.05988375713951127,-0.022115305158513822,-4.178427,-13.458362,24,G1,1180.4352160394192,10,7,13998.0,3630.0,8.48692670381483,16.959565652236034,0.9597494317807175,False,True,5.4,-7.9204836,-11.824245,3124.0,9.0,10,0.9558447829765444
-29,True,ACCCTTGAGACGTCCC-1,ACCCTTGAGACGTCCC-1,1.9266154671094529,15.0,-0.07311668226149834,-0.01455771261430732,-1.699633,-13.507107,24,G1,1307.2199267148972,10,7,14056.0,3696.0,8.708025042686398,17.551223676721683,0.9584301895311081,False,False,7.799999999999999,-6.587843,-12.211588,4017.0,9.0,10,0.9625685164062322
-30,True,ACCCTTGAGACGTCGA-1,ACCCTTGAGACGTCGA-1,1.9084909421261187,15.0,-0.10509415851892019,-0.010426675300685513,6.8856034,2.7047584,8,G1,1451.9405464082956,9,9,5813.0,1532.0,10.562532255289868,38.929984517460866,0.6108684693951845,False,False,4.0,12.857927,6.8620844,1475.0,9.0,9,1.3537176247944693
-31,True,ACCCTTGAGAGAGCGG-1,ACCCTTGAGAGAGCGG-1,1.9377355562266423,15.0,-0.09460539585616512,-0.018024137525203203,-5.6261396,-12.498869,10,G1,1201.8003843277693,3,7,16098.0,3771.0,9.995030438563797,19.673251335569635,0.9614684243398492,False,False,4.6,-9.069668,-10.986452,2305.0,9.0,3,1.1040216349363394
-33,True,ACCCTTGCACCAGCGT-1,ACCCTTGCACCAGCGT-1,1.899544289246287,15.0,-0.0642534312454616,-0.027580614813772115,-2.4103203,-12.326489,24,G1,1315.186081185937,10,7,13814.0,3541.0,6.696105400318517,14.383958303170697,0.9593821126702171,False,False,7.3999999999999995,-6.9891562,-12.658615,2127.0,9.0,10,0.9788750406701542
-34,True,ACCCTTGGTCACGCTG-1,ACCCTTGGTCACGCTG-1,1.8989244497204025,15.0,-0.06780221761292611,-0.03225494307002549,7.8903446,7.59997,27,G1,1256.2189148664474,8,10,3010.0,1070.0,11.096345514950166,33.15614617940199,0.613327551644923,False,True,4.800000000000001,13.773765,9.959809,1318.0,9.0,8,1.4094532797118868
-35,True,ACCCTTGGTCGGTGTC-1,ACCCTTGGTCGGTGTC-1,1.9922674162496778,15.0,-0.08400483963710707,-0.02735226372102239,-7.097548,-8.520551,3,G1,1306.474305421114,2,2,8960.0,3017.0,12.979910714285714,9.955357142857142,0.9559370864853503,False,False,3.1,-14.188237,-8.294819,2628.0,9.0,2,1.2088955930832395
-36,True,ACCCTTGGTGACGCCT-1,ACCCTTGGTGACGCCT-1,1.8684989813807944,15.0,-0.0907860473696177,-7.483610892146213e-05,5.740291,6.7807717,27,G1,1492.8751185759902,8,1,5560.0,1461.0,4.586330935251799,46.960431654676256,0.6127383762212906,False,False,3.1,11.156091,9.784362,2019.0,9.0,8,1.1784599979468142
-37,True,ACCCTTGGTTACACAC-1,ACCCTTGGTTACACAC-1,1.8794470496798685,15.0,-0.07666811817694673,0.0005727753746162557,2.5971267,2.7670872,12,S,1493.7553976029158,1,1,7103.0,2156.0,5.406166408559764,38.11065746867521,0.6118240648050127,False,False,3.1,10.048934,9.633621,1277.0,9.0,1,0.8054144824310264
-38,True,ACCCTTGTCCTACAAG-1,ACCCTTGTCCTACAAG-1,1.9353466112242022,15.0,-0.09056816741099151,-0.014568511852282574,-8.446106,-7.3514495,10,G1,1280.0189653784037,3,7,14712.0,3840.0,8.571234366503534,18.196030451332245,0.9675504536868252,False,False,3.3000000000000003,-13.166993,-7.5069847,3598.0,9.0,3,1.2658502293765688
-39,True,ACCCTTGTCGTCGGGT-1,ACCCTTGTCGTCGGGT-1,1.8924303247646337,15.0,-0.09846455306976869,-0.018815781254158715,4.208197,3.0059917,12,G1,1867.3819949030876,1,1,6757.0,1915.0,6.1713778303981055,41.423708746485126,0.6093151266184236,False,False,6.5,11.923405,9.056196,1347.0,9.0,1,1.63685188706804
-40,True,ACCCTTGTCTCACTCG-1,ACCCTTGTCTCACTCG-1,1.9612492536510562,15.0,-0.06978511811162805,-0.019192048864489088,-5.9530315,-5.712263,3,G1,1662.0810243785381,2,2,8265.0,2831.0,6.860254083484573,9.437386569872958,0.9637888864530519,False,False,3.3000000000000003,-15.239969,-5.843939,3303.0,9.0,2,1.059224453113974
-41,True,ACGTCCTAGATCACCT-1,ACGTCCTAGATCACCT-1,1.890923984907585,15.0,-0.07057484182913082,-0.004168466686372149,10.007795,-11.226237,21,G1,1289.3786257356405,7,6,4514.0,1841.0,13.712893221089942,16.415595923792644,0.053870042186144114,False,False,7.499999999999999,11.95194,-9.918121,865.0,9.0,7,1.212127791019733
-43,True,ACGTCCTAGCGAATGC-1,ACGTCCTAGCGAATGC-1,1.8843828604974306,15.0,-0.0963429395844203,0.008325421482058715,1.2262518,5.757637,5,S,1764.2370253801346,1,1,6847.0,1839.0,5.213962319263911,41.857747918796555,0.6140408407370542,True,True,7.4,9.921135,11.217627,1683.0,9.0,1,1.3553743959662323
-44,True,ACGTCCTAGGGCGAAG-1,ACGTCCTAGGGCGAAG-1,1.9626061492248548,15.0,-0.05250621617119254,-0.028860630781521773,-5.7333927,-3.655653,9,G1,1541.5463787019253,2,2,10241.0,3318.0,10.955961331901182,16.355824626501317,0.9504781298590307,False,False,3.6999999999999997,-15.761517,-3.6664236,3113.0,9.0,2,0.9501407005745005
-45,True,ACGTCCTAGGTTGCCC-1,ACGTCCTAGGTTGCCC-1,1.8411991286863043,15.0,-0.08153035609353625,-0.012079119397240289,9.381279,-13.654902,21,G1,1066.690640091896,7,6,3571.0,1547.0,6.048725847101652,22.5707084850182,0.0,True,True,8.1,10.070463,-12.128085,1563.0,9.0,7,0.9781724383891092
-46,True,ACGTCCTCAAACCATC-1,ACGTCCTCAAACCATC-1,2.1240603444743913,15.0,-0.06839945280437756,-0.03299701757725744,-4.0088377,3.7243028,14,G1,1197.2542121261358,4,4,765.0,512.0,20.130718954248366,3.9215686274509802,0.8777616830484557,False,False,5.0,-13.106527,2.793371,837.0,9.0,4,1.0786012599433636
-47,True,ACGTCCTCAAGAGTAT-1,ACGTCCTCAAGAGTAT-1,1.9940406175301597,15.0,-0.07262111129113716,-0.02540544507298371,-9.376629,-0.38546473,7,G1,1603.7265069186687,3,3,9683.0,3074.0,10.131157699060209,14.592584942683052,0.9806804810927897,False,False,4.7,-16.228443,-1.5504993,6456.0,9.0,3,1.3354959871622751
-49,True,ACGTCCTCATTATGCG-1,ACGTCCTCATTATGCG-1,2.034448925587455,15.0,-0.05711323462871902,-0.027609150700334782,-4.974606,-0.079187624,19,G1,1507.6808382868767,2,4,4975.0,1861.0,11.678391959798995,13.78894472361809,0.9543663654580443,False,False,2.9999999999999996,-14.459675,-0.69358426,1204.0,9.0,2,1.0799877553826405
-50,True,ACGTCCTGTATGCGTT-1,ACGTCCTGTATGCGTT-1,1.9037954077126373,15.0,-0.10632762413983757,-0.004796169795629876,0.9615184,6.489793,5,G1,1742.8540508449078,1,1,4945.0,1262.0,7.906976744186046,47.664307381193126,0.6156071771135834,False,False,6.8,9.659729,11.509306,1155.0,9.0,1,1.8163536210883573
-51,True,ACGTCCTGTCCCGTGA-1,ACGTCCTGTCCCGTGA-1,1.9482561961961429,15.0,-0.08340208602750185,-0.00889055747472582,-6.5086055,-4.7722297,13,G1,1664.770869165659,2,2,7795.0,2570.0,6.837716484926235,16.856959589480436,0.9658141163972863,False,False,3.9999999999999996,-16.368217,-5.299693,2170.0,9.0,2,1.069160195042647
-52,True,ACGTCCTGTGCACAAG-1,ACGTCCTGTGCACAAG-1,1.911524318285445,15.0,-0.12208339707426523,-0.012633811495195116,2.7054572,1.9183406,12,G1,1411.3157752752304,1,1,6876.0,1564.0,7.5770796974985455,50.174520069808025,0.6116662581149102,False,False,4.3999999999999995,11.59077,8.909735,758.0,9.0,1,1.170453247554156
-53,True,ACGTCCTTCCCAAGCG-1,ACGTCCTTCCCAAGCG-1,1.9231493720785193,15.0,-0.09072517514696551,-0.02057011127366228,6.8668537,8.437247,27,G1,1240.402009472251,8,10,5475.0,1391.0,9.077625570776256,46.10045662100457,0.6165011538181364,False,False,3.9,13.2112465,10.936267,1220.0,9.0,8,1.3014299323702554
-54,True,ACGTCCTTCGCGGTAC-1,ACGTCCTTCGCGGTAC-1,1.8911260399407708,15.0,-0.10555465517854065,-0.022654199451553374,3.8456054,1.5156982,12,G1,1318.4750853031874,1,1,5806.0,1740.0,6.4071650017223565,40.4753703065794,0.6109226755978073,False,False,3.2,12.465912,8.845622,858.0,9.0,1,0.8054144824310264
-55,True,ACGTCCTTCTGCCCTA-1,ACGTCCTTCTGCCCTA-1,1.8593612057089002,15.0,-0.08281755893925226,-0.022818658115809423,7.375444,2.2778807,8,G1,1443.4430965036154,9,9,7229.0,2373.0,5.118273620141099,31.17996956702172,0.606216996821714,False,False,3.9999999999999996,12.999833,6.5133967,1268.0,9.0,9,1.319020352364041
-56,True,ACGTCCTTCTGTGCGG-1,ACGTCCTTCTGTGCGG-1,1.9978282107380063,15.0,-0.1291482250750845,-0.016104516056073763,1.9032989,19.121885,18,G1,1221.429703116417,5,5,4309.0,1184.0,10.95381759108842,43.420747273149225,0.7231815237262016,False,False,8.8,-4.5405436,15.926964,1167.0,9.0,5,0.8054144824310264
-57,True,ACGTCCTTCTTAGCCC-1,ACGTCCTTCTTAGCCC-1,1.8795040235680138,15.0,-0.10417673935115795,-0.012292111806674916,2.491866,2.831029,12,G1,1674.4842343777418,1,1,5920.0,1719.0,5.236486486486487,41.84121621621622,0.611295414830068,False,False,4.0,10.894972,8.740703,1295.0,9.0,1,1.1922775842747282
-58,True,ACTATTCAGCAATTAG-1,ACTATTCAGCAATTAG-1,1.8716250208397454,15.0,-0.09873974141291997,-0.002366597299575949,3.032851,5.5111866,12,G1,1637.1956877335906,1,1,6095.0,1868.0,4.446267432321575,39.96718621821165,0.6112315477687476,False,False,2.9,9.682619,9.356861,1495.0,9.0,1,0.9875659855422305
-59,True,ACTATTCAGGCCTAGA-1,ACTATTCAGGCCTAGA-1,1.9822908718248735,15.0,-0.07645455319873923,-0.015195259796012724,-3.9619758,-3.8943422,17,G1,1745.468310981989,2,2,9800.0,2898.0,8.448979591836734,17.336734693877553,0.9573610689035008,False,False,5.3,-14.678455,-4.0126634,2488.0,9.0,2,2.3005542990075627
-60,True,ACTATTCAGTAATTGG-1,ACTATTCAGTAATTGG-1,1.9494896771752799,15.0,-0.10974118484265478,-0.010726607001107828,7.0587835,9.195922,15,G1,1266.4743803143501,8,10,6784.0,1656.0,13.413915094339623,44.23643867924528,0.6187331211021164,False,False,3.8,14.198982,10.02298,2064.0,9.0,8,1.1339888656653048
-61,True,ACTATTCCAATCTCGA-1,ACTATTCCAATCTCGA-1,1.989709565229192,15.0,-0.09595017547036037,-0.01685608385535281,-10.3224325,-7.364411,10,G1,1196.57967813313,3,7,9841.0,2873.0,9.236866172136978,16.380449141347423,0.9747064063278525,False,False,3.5,-12.972481,-8.096525,2289.0,9.0,3,1.3745075350529723
-62,True,ACTATTCCAATCTGCA-1,ACTATTCCAATCTGCA-1,1.8640990373110142,15.0,-0.07787698925620688,-0.013662516468510067,9.99852,-7.2948914,23,G1,1183.6172689944506,6,8,5536.0,1933.0,7.22543352601156,26.28251445086705,0.1917290687991576,False,False,3.6999999999999997,13.940358,-6.1188426,1038.0,9.0,6,0.8054144824310264
-63,True,ACTATTCCACTTCAGA-1,ACTATTCCACTTCAGA-1,1.8587843246922044,15.0,-0.08531783024404366,-0.014554262927601043,9.2084,-1.940629,23,G1,1009.1727796345949,6,8,12128.0,3367.0,5.722295514511873,28.91655672823219,0.46669974421030835,False,False,3.5000000000000004,14.163376,0.57997674,1364.0,9.0,6,0.8054144824310264
-64,True,ACTATTCCATCCTTCG-1,ACTATTCCATCCTTCG-1,1.8657433984472056,15.0,-0.06689007824535846,-0.0226436777151306,8.529536,-0.15996589,22,G1,1041.2424451112747,6,8,4649.0,1960.0,8.216820821682083,16.390621639062164,0.5387803478399062,False,False,4.3,13.964401,2.8545892,1390.0,9.0,6,1.508013832392216
-65,True,ACTATTCGTAACTTCG-1,ACTATTCGTAACTTCG-1,1.9316580492555695,15.0,-0.10560344996018134,-0.008961632216795152,6.348596,7.5691085,27,G1,1479.0055207759142,8,10,7630.0,1685.0,9.698558322411534,47.929226736566186,0.6141173864443679,False,False,3.5999999999999996,12.718884,10.227739,1071.0,9.0,8,1.4603486448954297
-66,True,ACTATTCGTATGCGTT-1,ACTATTCGTATGCGTT-1,2.033584666481111,15.0,-0.060911720310231995,-0.023422260347474202,-8.126636,-0.1539,14,G1,1594.126646950841,4,3,6794.0,2379.0,17.868707683249927,12.275537238740064,0.9730471548675709,False,False,3.5,-15.053409,-0.92805713,2106.0,9.0,4,1.3211486736114244
-67,True,ACTATTCGTCACGCTG-1,ACTATTCGTCACGCTG-1,2.0202172791313604,15.0,-0.06407273849134315,-0.023838165308345618,-4.5382743,-4.9442253,10,G1,1714.09015481174,3,2,5698.0,2233.0,13.75921375921376,11.53036153036153,0.9601567997651083,False,False,4.4,-14.511831,-4.1597834,1819.0,9.0,3,0.8054144824310264
-68,True,ACTATTCGTGCCTTCT-1,ACTATTCGTGCCTTCT-1,1.9752282667226142,15.0,-0.05783954470884885,-0.02764999751763641,-12.387285,-4.460261,7,G1,990.4759607613087,3,3,7459.0,2720.0,9.760021450596595,10.202440005362648,0.9932245030405271,False,False,3.7,-18.842728,-4.447149,2673.0,10.0,3,1.1258223601919735
-69,True,ACTATTCGTTGGGATG-1,ACTATTCGTTGGGATG-1,1.9373940662589682,15.0,-0.10287765338407372,-0.023579749884573845,8.379806,-7.64282,11,G1,958.1159641593695,7,6,15005.0,3775.0,8.250583138953681,21.552815728090636,0.2640752669995253,True,True,4.6,14.483362,-7.0831995,3889.0,9.0,7,0.9478035426099897
-70,True,ACTATTCTCACAAGAA-1,ACTATTCTCACAAGAA-1,2.000246832278422,15.0,-0.06294029999704105,-0.015643994021842748,-3.1175537,-1.7770852,19,G1,1478.637172743678,2,2,7134.0,2451.0,7.429212223156714,14.872441827866554,0.9477372583707036,False,False,3.5,-14.024736,-2.2256036,1971.0,9.0,2,1.4944681780875912
-71,True,ACTATTCTCCGGACGT-1,ACTATTCTCCGGACGT-1,1.933901768862953,15.0,-0.10583325437329355,-0.013787787504780007,0.6444152,13.119549,2,G1,1015.092177465558,5,5,14462.0,3138.0,8.207716774996543,41.76462453325958,0.7239050480111444,False,False,5.199999999999999,-0.69497895,16.233822,1006.0,9.0,5,1.0829206595588365
-72,True,ACTATTCTCGCGGTAC-1,ACTATTCTCGCGGTAC-1,1.9592700498166693,15.0,0.01474474761334097,-0.04241681591101894,0.62224597,6.7831917,5,G2M,1621.0989419817924,1,1,1648.0,807.0,23.361650485436893,4.368932038834951,0.620333207693447,False,True,5.0,9.962697,12.011572,1295.0,9.0,1,1.2352060533237517
-73,True,ACTATTCTCTCCAATT-1,ACTATTCTCTCCAATT-1,1.8863013495422911,15.0,-0.10731155866669423,-0.014151722775435874,9.522674,-0.0152956275,25,G1,995.6393796652555,6,8,9866.0,2763.0,5.98013379282384,30.356780863571863,0.5459177420541588,False,True,3.6000000000000005,13.737263,2.547135,1663.0,9.0,6,0.9413669634110602
-74,True,ACTATTCTCTCGAGTA-1,ACTATTCTCTCGAGTA-1,2.0265210746159714,15.0,-0.06351269107622752,-0.03299969503730104,-10.213072,-2.7101417,7,G1,1647.141313701868,3,3,8532.0,2692.0,11.75574308485701,10.864978902953586,0.9867050689808768,False,False,4.5,-16.388884,-3.3517702,2676.0,9.0,3,1.462783751316627
-75,True,ACTATTCTCTGCCCTA-1,ACTATTCTCTGCCCTA-1,1.879849804057742,15.0,-0.07067451416056399,-0.01706827905185314,8.046637,2.115622,8,G1,1262.433500006795,9,9,10515.0,3083.0,6.533523537803139,20.542082738944366,0.62643940436049,False,False,4.6000000000000005,13.081439,5.5957775,1724.0,9.0,9,0.9630688780490345
-77,True,ACTGTGAAGAGAGCGG-1,ACTGTGAAGAGAGCGG-1,1.9724923648909782,15.0,-0.10025597602350028,-0.020843644461494906,-4.2718983,-6.330891,9,G1,1388.5975793004036,2,2,11851.0,3092.0,10.395747194329592,27.70230360307147,0.935009614164887,False,False,3.0,-15.903538,-6.389505,1096.0,9.0,2,1.1645165933714983
-78,True,ACTGTGAAGCAGGTCA-1,ACTGTGAAGCAGGTCA-1,1.9947385828717523,15.0,-0.057442459041111574,-0.02543645849304551,-3.15415,-0.69599515,19,G1,1275.5968184620142,2,4,6296.0,2337.0,9.831639135959339,12.53176620076239,0.9446992692606417,False,False,3.8,-13.189667,-1.1440012,1180.0,9.0,2,1.0525539774112422
-81,True,ACTGTGAAGGATCACG-1,ACTGTGAAGGATCACG-1,1.9266578281674922,15.0,-0.09814519374711683,-0.016400717829896935,8.743321,4.0973234,26,G1,1305.6378846168518,9,9,7783.0,1881.0,10.073236541179494,41.68058589232944,0.6179944914865506,False,False,4.999999999999999,11.59279,6.0011744,1886.0,9.0,9,0.8054144824310264
-82,True,ACTGTGAAGGCGTTAG-1,ACTGTGAAGGCGTTAG-1,1.8881178355891592,15.0,-0.07841461162284163,-0.01923006035929569,9.20645,-14.0381365,21,G1,1090.366791844368,7,6,7287.0,2630.0,10.37463976945245,20.296418279127213,0.0006390215367841526,False,False,8.6,9.978135,-12.121716,970.0,9.0,7,1.6743943618986665
-83,True,ACTGTGAAGGCTCAAG-1,ACTGTGAAGGCTCAAG-1,1.9475653696109883,15.0,-0.08942293937292428,-0.00784013955448407,-5.9713006,-1.4599253,9,G1,1445.5336554348469,2,2,14577.0,3689.0,6.997324552377032,25.17664814433697,0.9640010456652016,False,False,2.5,-15.794286,-1.888785,3456.0,9.0,2,1.0949590491706929
-85,True,ACTGTGAAGTGGCCTC-1,ACTGTGAAGTGGCCTC-1,1.965260042299594,15.0,-0.07500620368312884,-0.01460627014926029,-4.0513477,-5.9482374,19,G1,1425.8415170162916,2,2,6207.0,2303.0,10.165941678749798,16.15917512485903,0.9557728335364338,False,False,3.5,-14.791169,-6.24024,2325.0,9.0,2,0.8054144824310264
-86,True,ACTGTGACAATTAGGA-1,ACTGTGACAATTAGGA-1,1.9327927618413883,15.0,-0.0998966036500815,-0.0156442656982727,0.9952308,14.753332,8,G1,1109.8356473222375,9,5,9303.0,2672.0,11.856390411695152,33.47307320219284,0.697008395033082,False,True,3.0,-1.5004996,14.2016325,2140.0,9.0,9,0.8054144824310264
-87,True,ACTGTGACACAAATCC-1,ACTGTGACACAAATCC-1,1.9112776409873067,15.0,-0.10435065737854013,-0.013943357410080494,6.662921,8.214513,27,G1,1393.8291646540165,8,10,6333.0,1711.0,7.468814148113059,42.74435496605084,0.6154113585046815,False,False,3.2,13.2977705,10.739697,914.0,10.0,8,1.2151900768914132
-88,True,ACTGTGACACCAGCGT-1,ACTGTGACACCAGCGT-1,1.8843854288840542,15.0,-0.031549382327760445,0.0666209967199984,9.221009,-13.117173,21,S,1133.3619707673788,7,6,3043.0,1549.0,14.919487348011831,9.924416694051923,0.005305961053718569,False,False,6.8999999999999995,10.457469,-11.725558,1411.0,9.0,7,1.3975645918688595
-89,True,ACTGTGACACTATGTG-1,ACTGTGACACTATGTG-1,1.8773673584448123,15.0,-0.08073465970932901,-0.01179629071497552,7.3555627,-1.408197,22,G1,1102.344379708171,6,8,5918.0,1991.0,8.110848259547144,30.111524163568774,0.4989222204860849,False,False,5.3999999999999995,14.456866,1.6121007,1759.0,9.0,6,1.382121204918996
-90,True,ACTGTGACAGCTGCCA-1,ACTGTGACAGCTGCCA-1,1.9733803584821725,15.0,-0.0904900306933119,-0.034761915743397646,-4.5596476,-7.0749164,19,G1,1306.0351961255074,2,2,8420.0,2891.0,10.225653206650831,15.344418052256533,0.9559341471175475,False,False,3.5999999999999996,-15.021027,-7.0312304,2326.0,9.0,2,0.9591061456682548
-91,True,ACTGTGACATCCTATT-1,ACTGTGACATCCTATT-1,1.9484008453405508,15.0,-0.09501456956603493,-0.006045972293970855,2.3803263,16.69263,2,G1,1016.6927028298378,5,5,13443.0,3371.0,7.275161794242357,36.40556423417392,0.7205631256568056,False,True,4.8999999999999995,-3.5152087,14.122371,1548.0,9.0,5,0.9575994643424515
-92,True,ACTGTGACATCCTTCG-1,ACTGTGACATCCTTCG-1,1.9115047521735686,15.0,-0.09690825422658138,-0.013000148693138524,7.1932964,4.5039196,8,G1,1397.0810600668192,9,1,6604.0,1925.0,10.902483343428225,36.2961841308298,0.6097175198073198,False,True,4.3999999999999995,12.802611,7.3782597,1581.0,9.0,9,0.8054144824310264
-93,True,ACTGTGACATTATGCG-1,ACTGTGACATTATGCG-1,1.867015604702736,15.0,-0.09069105388635482,-0.014058363906306028,10.109635,-9.319949,21,G1,1220.7012396007776,7,6,5242.0,2032.0,6.52422739412438,24.13201068294544,0.1263544505636979,False,False,6.199999999999999,13.481164,-8.171996,1071.0,9.0,7,0.8054144824310264
-94,True,ACTGTGAGTATGAGCG-1,ACTGTGAGTATGAGCG-1,1.9193074794065383,15.0,-0.0808251038682122,-0.018059408229311142,8.985912,-8.727263,11,G1,1072.5891362279654,7,6,15744.0,3906.0,9.711636178861788,18.39430894308943,0.18548127965580266,False,False,3.4,14.480761,-8.508059,3126.0,10.0,7,0.8054144824310264
-95,True,ACTGTGAGTCAAGCCC-1,ACTGTGAGTCAAGCCC-1,1.9380479095353644,15.0,-0.046582736536601534,-0.008014468924024364,0.013417046,6.093354,5,G1,1168.5311177372932,1,1,2717.0,1359.0,16.930437983069563,8.870077291129922,0.6105760287826422,False,False,1.5000000000000002,8.699713,10.604262,1577.0,9.0,1,0.8054144824310264
-96,True,ACTGTGAGTCCACACG-1,ACTGTGAGTCCACACG-1,1.9218758682230412,15.0,-0.11418510520261387,-0.005908749779647763,0.97859776,15.535449,2,G1,1104.8479398041964,5,5,5657.0,1582.0,6.487537564079901,42.425313770549764,0.719205159237297,False,False,8.5,-1.6820934,14.997838,1397.0,9.0,5,0.9586635921345452
-97,True,ACTGTGAGTGCACAAG-1,ACTGTGAGTGCACAAG-1,2.0076344269936413,15.0,-0.06553093191795308,-0.025316597127778916,-6.0604887,1.2074281,14,G1,1563.576756849885,4,4,6786.0,2329.0,13.188918361332155,13.144709696433834,0.9558605311241337,True,True,4.8999999999999995,-15.19085,0.29295868,1902.0,9.0,4,1.6224621336919873
-98,True,ACTGTGAGTTCCGTTC-1,ACTGTGAGTTCCGTTC-1,2.0513904483473366,15.0,-0.04704167494865169,-0.030341946199551377,-5.058761,2.9278603,14,G1,1476.4966849833727,4,4,6750.0,2508.0,13.6,10.592592592592593,0.9028007977447524,False,False,5.699999999999999,-13.447811,1.908975,1940.0,9.0,4,0.8054144824310264
-99,True,ACTGTGATCAATCCAG-1,ACTGTGATCAATCCAG-1,1.9677889995869646,15.0,-0.07736155571630421,-0.03320218996062414,-3.877977,-3.0950725,17,G1,1658.9715236574411,2,2,11826.0,3527.0,5.361068831388466,16.007102993404363,0.958462505027154,False,False,4.1,-14.66154,-3.169422,2539.0,9.0,2,1.5869652682386162
-100,True,ACTGTGATCTAGTCAG-1,ACTGTGATCTAGTCAG-1,1.9466893161646306,15.0,-0.09076098358083046,-0.023074522113504914,-7.0152774,-5.3941374,13,G1,1143.41269184649,2,2,12805.0,3644.0,7.254978524014057,16.876220226474032,0.9634746731867095,False,False,3.1000000000000005,-16.693508,-6.4157534,2625.0,9.0,2,0.8054144824310264
-101,True,ACTGTGATCTCATAGG-1,ACTGTGATCTCATAGG-1,2.124130229296572,15.0,-0.0554255920982218,-0.034575277293723894,-8.716258,-12.356273,16,G1,642.6370834708214,2,2,1404.0,481.0,11.253561253561253,2.92022792022792,0.9321543439904747,True,True,6.3999999999999995,-15.184017,-11.536053,271.0,9.0,2,1.234266557605285
-102,True,ACTGTGATCTCTAGGA-1,ACTGTGATCTCTAGGA-1,1.9517632949957404,15.0,-0.10409763850698597,-0.017588243472378882,-0.20712575,12.735458,2,G1,853.510957852006,5,5,10099.0,2507.0,9.456381819982177,40.568373106248146,0.7269787266345247,False,False,6.5,-0.04312111,16.949722,951.0,9.0,5,1.0581504017760734
-103,True,ACTTTCAAGAAATTGC-1,ACTTTCAAGAAATTGC-1,1.8964340325655307,15.0,-0.06780674499543277,-0.0030607945515295637,10.020077,-11.785579,21,G1,1239.8609247505665,7,6,3610.0,1547.0,11.495844875346261,18.44875346260388,0.03854061697907293,False,False,5.599999999999999,12.014874,-11.329532,637.0,9.0,7,0.8054144824310264
-104,True,ACTTTCAAGATGCTTC-1,ACTTTCAAGATGCTTC-1,1.8942135405054608,15.0,-0.10400421670057675,-0.013261345579848823,6.6327767,10.295502,15,G1,1066.0915660709143,8,10,6582.0,1911.0,6.7760559100577336,41.44636888483743,0.6307472862668468,False,False,5.7,15.171917,9.373586,857.0,9.0,8,0.8054144824310264
-105,True,ACTTTCAAGCACTCTA-1,ACTTTCAAGCACTCTA-1,1.892639797434566,15.0,-0.1008618131353074,-0.008951953980983942,9.652359,3.5414011,8,G1,1293.3827391117811,9,9,6657.0,1898.0,7.766261078563918,35.2410995944119,0.6068761448463132,False,True,5.1000000000000005,12.074815,4.759704,1853.0,9.0,9,1.3232212714969998
-106,True,ACTTTCAAGCCTCTGG-1,ACTTTCAAGCCTCTGG-1,1.8963406512652279,15.0,-0.11331042571751636,-0.010063135235411736,7.260237,6.2923594,25,G1,1345.7832995951176,6,10,6023.0,1691.0,4.51602191598871,44.59571642038851,0.6067439093096848,False,False,4.5,14.117527,8.71284,1344.0,9.0,6,0.8054144824310264
-107,True,ACTTTCAAGCGACATG-1,ACTTTCAAGCGACATG-1,1.9197130280788057,15.0,-0.09652048342705816,-0.01932593184048986,8.416509,10.292564,15,G1,1147.810196891427,8,10,3818.0,1022.0,7.778941854374017,45.52121529596648,0.6185634924820022,False,False,3.6,14.383933,10.498906,1110.0,9.0,8,0.8054144824310264
-108,True,ACTTTCAAGGTTGGAC-1,ACTTTCAAGGTTGGAC-1,1.9582891171491301,15.0,-0.07747401135661072,-0.027039160317768565,-3.6416376,-1.9445916,19,G1,1529.2768681645393,2,2,8149.0,2774.0,7.240152165909928,17.315007976438828,0.9542288044611094,False,False,2.6,-14.080324,-2.116249,1936.0,9.0,2,1.256546251028587
-109,True,ACTTTCAAGTCATCCA-1,ACTTTCAAGTCATCCA-1,2.0408762159858713,15.0,-0.062080735990089245,-0.017013267061355183,-5.266631,2.7807066,14,G1,1482.9733168929815,4,4,10549.0,3125.0,8.49369608493696,15.555976869845482,0.9356444355957817,False,False,5.3999999999999995,-14.663025,1.9532293,1672.0,9.0,4,1.1702966058509467
-110,True,ACTTTCACAAATGCGG-1,ACTTTCACAAATGCGG-1,1.901485540160068,15.0,-0.058256046977676286,-0.006145748115123761,1.2737167,7.560882,5,G1,1675.4764136373997,1,1,1497.0,666.0,19.57247828991316,8.082832331329325,0.6142047373708482,False,False,6.299999999999999,9.092782,11.775453,1395.0,9.0,1,1.1855478963496768
-111,True,ACTTTCACAAGAGAGA-1,ACTTTCACAAGAGAGA-1,1.9881571699752574,15.0,-0.07865278525264902,-0.01690831252383268,-10.430816,-3.6354675,1,G1,1231.4007664173841,3,3,10959.0,3140.0,9.289168719773702,16.379231681722786,0.9905785193245781,False,False,3.5,-14.445804,-4.201788,2687.0,9.0,3,1.397034968469201
-112,True,ACTTTCACAAGCCATT-1,ACTTTCACAAGCCATT-1,1.88816766793761,15.0,-0.09418699997783833,-0.01256697558650215,6.0996795,2.8216445,12,G1,1482.4616336226463,1,9,9256.0,2690.0,7.443820224719101,28.900172860847018,0.6157231479458831,False,False,3.9,11.829315,7.551596,1023.0,9.0,1,0.8054144824310264
-113,True,ACTTTCACAAGTAGTA-1,ACTTTCACAAGTAGTA-1,1.991366036019676,15.0,-0.07184883561454661,-0.02700686157381342,-8.155318,-8.60291,10,G1,1154.6786435991526,3,7,8787.0,2864.0,12.029133947877547,14.191419141914192,0.9645025588677265,False,False,3.6999999999999997,-12.797674,-9.291117,2492.0,9.0,3,1.0189772375382224
-114,True,ACTTTCACACTAAACC-1,ACTTTCACACTAAACC-1,1.9636512200179288,15.0,-0.05073034646695112,-0.028989124171488693,-5.852978,-4.9155946,10,G1,1677.563086733222,3,2,9092.0,3038.0,10.679718433787945,10.481742190937087,0.9600669699873128,False,False,4.500000000000001,-14.552683,-4.9078383,3471.0,9.0,3,1.0245455485087864
-115,True,ACTTTCACACTTCAGA-1,ACTTTCACACTTCAGA-1,1.8664332532387522,15.0,-0.08044743458992354,-0.01644939384433121,9.4461355,-10.887075,21,G1,1301.580148652196,7,6,4629.0,1736.0,10.153380859796933,19.356232447612875,0.05701591663650824,False,False,7.499999999999999,12.121291,-10.262825,1426.0,9.0,7,1.6106171771038273
-116,True,ACTTTCACAGGGAATC-1,ACTTTCACAGGGAATC-1,1.9010610756800377,15.0,-0.10819248165349397,-0.02128464435717434,3.5717866,6.3991256,6,G1,1656.1327080205083,1,1,7548.0,1981.0,7.180710121886593,43.693693693693696,0.6126634191717819,False,False,3.5999999999999996,9.67276,9.791462,1221.0,9.0,1,1.1916692492517824
-117,True,ACTTTCACATCAGTGT-1,ACTTTCACATCAGTGT-1,1.9663578487792606,15.0,-0.07449996334128788,-0.027096054738262126,-4.3381352,-5.009088,17,G1,1671.6289958059788,2,2,9833.0,3013.0,8.868097223634699,15.051357673141462,0.9590278667574874,False,False,4.8,-14.919808,-5.258017,1150.0,9.0,2,1.0538656653031109
-118,True,ACTTTCAGTAGTGTGG-1,ACTTTCAGTAGTGTGG-1,2.0028762645080698,15.0,-0.07890328216804067,-0.032740991478855556,-6.6934385,1.6961279,14,G1,1490.9386226385832,4,4,6864.0,2438.0,7.328088578088578,15.224358974358974,0.9579899192793372,False,False,4.6000000000000005,-16.009384,0.57214844,2694.0,9.0,4,1.1225541884034926
-119,True,ACTTTCAGTATGCGTT-1,ACTTTCAGTATGCGTT-1,1.9699978206779976,15.0,-0.0662849908772989,-0.026505389467882976,-7.35479,-2.8719203,10,G1,1413.7459429949522,3,3,7966.0,2669.0,10.369068541300527,13.670600050213407,0.9622293142616853,False,False,3.1999999999999997,-13.959063,-2.7390842,3405.0,9.0,3,1.369311926511253
-120,True,ACTTTCAGTGATTGGG-1,ACTTTCAGTGATTGGG-1,1.9241425848798015,15.0,-0.1011549593459831,-0.012651399997797734,12.66525,-9.2177105,11,G1,1136.7876921594143,7,6,11924.0,2679.0,10.45790003354579,40.77490774907749,0.13605310054438416,False,False,7.6,12.360453,-6.8366046,1724.0,9.0,7,0.9378839828532496
-121,True,ACTTTCAGTGTCTAAC-1,ACTTTCAGTGTCTAAC-1,1.9702176253231498,15.0,-0.07251054342496285,-0.02948290900829872,-7.330685,-0.59754944,10,G1,1540.2221986353397,3,4,11030.0,3523.0,10.28105167724388,13.880326382592928,0.9620596072637166,False,False,3.6999999999999997,-14.191189,-1.4582981,1724.0,9.0,3,1.282510015549939
-122,True,ACTTTCAGTTACACAC-1,ACTTTCAGTTACACAC-1,1.9002754240094941,15.0,-0.08767007353670596,-0.013301066421987593,9.55026,-4.678605,23,G1,1138.1989960074425,6,8,4887.0,1550.0,10.272150603642316,34.8475547370575,0.33490607246167725,False,False,6.8999999999999995,14.273514,-2.7547255,1639.0,9.0,6,1.15795119741862
-123,True,ACTTTCATCAAAGGTA-1,ACTTTCATCAAAGGTA-1,2.0053525793108604,15.0,-0.045531502407777744,-0.03391797980135789,-6.6274514,-5.3536544,1,G1,1493.3843783438206,3,2,8958.0,2960.0,10.069211877651261,12.51395400759098,0.9665439751153807,False,False,3.8000000000000003,-15.950292,-5.7020864,2041.0,9.0,3,1.0671095548959224
-124,True,ACTTTCATCTGGCTGG-1,ACTTTCATCTGGCTGG-1,1.9458778588056291,15.0,-0.08240510296775536,-0.03113490966173404,-9.386459,-6.8633184,10,G1,1263.0734204798937,3,7,11502.0,3637.0,7.311771865762476,14.154060163449834,0.9705164495937822,False,False,4.2,-12.398723,-7.5927157,3132.0,9.0,3,1.4321073132235262
-125,True,ACTTTCATCTGTGCGG-1,ACTTTCATCTGTGCGG-1,1.9705704044143308,15.0,-0.06566028373629774,-0.023336295207385723,-6.857171,-6.8440337,13,G1,1225.849198102951,2,2,8816.0,3031.0,10.401542649727768,11.195553539019963,0.968040228924521,False,False,3.9,-17.115747,-6.469414,1505.0,9.0,2,0.9813075068299141
-126,True,AGACACTAGACCATTC-1,AGACACTAGACCATTC-1,1.9655597333551948,15.0,-0.06527099550355364,-0.026739638390123832,-7.673124,-4.6967716,10,G1,1194.2258914858103,3,2,9620.0,3147.0,7.401247401247401,13.212058212058212,0.969023276892191,False,False,3.3,-15.258705,-5.503738,2726.0,9.0,3,1.3517423825432624
-127,True,AGACACTAGCAATTAG-1,AGACACTAGCAATTAG-1,1.9302685623358062,15.0,-0.053527832754029434,-0.0183329024064674,9.244272,-9.792043,11,G1,1273.247906446457,7,6,3122.0,1424.0,19.218449711723256,11.915438821268417,0.08989631586234813,False,False,6.0,13.009545,-9.209364,1897.0,9.0,7,1.1896827199692022
-128,True,AGACACTAGGGAGGTG-1,AGACACTAGGGAGGTG-1,1.9417296881973234,15.0,-0.08105302678946089,-0.01725997842502697,8.13858,-3.4224825,22,G1,1204.7062613368034,6,8,3375.0,1156.0,14.42962962962963,31.644444444444446,0.43964569569115763,False,False,7.1000000000000005,14.278927,-0.22619778,1320.0,9.0,6,1.5275692225903867
-129,True,AGACACTAGTATAACG-1,AGACACTAGTATAACG-1,1.8929122814307386,15.0,-0.08905174202953123,0.001448738968399449,0.30511925,5.0811133,5,S,1037.2865631952882,1,1,5074.0,1441.0,4.237288135593221,46.68900275916437,0.6116234266318952,False,False,3.5,7.9972086,9.589724,1390.0,9.0,1,0.9684021274307453
-130,True,AGACACTCAACGCATT-1,AGACACTCAACGCATT-1,1.893928517980608,15.0,-0.10179908867452041,-0.0051530811888401,5.120653,0.25168934,22,G1,1296.669214606285,6,8,6666.0,1985.0,8.04080408040804,36.108610861086106,0.576448042763361,False,False,4.2,13.659927,6.323415,1144.0,9.0,6,1.1557748892262265
-132,True,AGACACTCACATGAAA-1,AGACACTCACATGAAA-1,1.968532508658971,15.0,-0.08101819371944596,-0.02549711287575365,-10.421326,-3.6499476,1,G1,1371.0099867135286,3,3,10582.0,3281.0,10.584010584010583,13.362313362313362,0.9921049946843041,False,False,3.0,-17.10485,-4.2063203,3704.0,9.0,3,1.0146453417455004
-133,True,AGACACTCACCCTCTA-1,AGACACTCACCCTCTA-1,1.884263406559217,15.0,-0.06645754877474479,-0.00555304604059699,9.640648,-5.695478,23,G1,1161.5730865597725,6,8,4271.0,1575.0,8.054319831421212,27.604776398969797,0.2893233672292946,False,False,7.0,14.281711,-4.0700774,958.0,9.0,6,1.3433065262464856
-134,True,AGACACTCACTACGGC-1,AGACACTCACTACGGC-1,1.9759764583256292,15.0,-0.07223219903202369,-0.030847459369281738,-7.0190253,-0.9695707,14,G1,1170.822343185544,4,4,4138.0,1869.0,11.213146447559208,10.850652489125181,0.9575753820613987,False,False,3.1,-14.04022,-1.755754,2475.0,9.0,4,0.8054144824310264
-135,True,AGACACTCATCCTATT-1,AGACACTCATCCTATT-1,1.983532850304823,15.0,-0.06614508442306528,-0.021979802477902946,-5.896441,-5.383041,13,G1,1703.8182702213526,2,2,9867.0,3152.0,10.823958650045606,16.205533596837945,0.9641172484931678,False,False,4.0,-16.53302,-5.7506332,1979.0,9.0,2,1.3862740530887567
-136,True,AGACACTGTAATGCTC-1,AGACACTGTAATGCTC-1,1.8954734374426594,15.0,-0.1091875587277592,-0.011293781999864553,7.464264,3.8092983,8,G1,1386.8500839993358,9,9,7086.0,1827.0,6.209427039232289,46.401354784081285,0.6104120718317447,False,False,3.8,12.260687,6.1861205,1187.0,9.0,9,0.8054144824310264
-137,True,AGACACTGTACGGGAT-1,AGACACTGTACGGGAT-1,1.872303862166951,15.0,-0.08881781772910148,-0.022078941752193076,6.116325,-0.75833714,22,G1,1027.3127206414938,6,8,4703.0,1604.0,7.761003614714013,33.638103338294705,0.5435426938549931,False,False,4.199999999999999,14.553701,3.1371253,1258.0,9.0,6,0.8054144824310264
-138,True,AGACACTGTAGCGTAG-1,AGACACTGTAGCGTAG-1,2.068224709385442,15.0,-0.07726599635021306,-0.02468847820666064,-9.665532,-3.0413587,1,G1,1485.682854756713,3,3,5601.0,2093.0,18.425281199785754,9.891090876629173,0.9857468177817974,False,False,4.1,-15.293498,-3.3079426,2934.0,9.0,3,0.9391727281283155
-139,True,AGACACTGTATACCTG-1,AGACACTGTATACCTG-1,2.0337654921907915,15.0,-0.035614698010714776,-0.039291236786737314,0.4117912,8.699431,5,G1,1240.5807669907808,1,1,1532.0,771.0,30.026109660574413,7.506527415143603,0.6270422431854095,False,False,4.699999999999999,8.644056,12.775502,1534.0,9.0,1,0.8054144824310264
-141,True,AGACACTGTGTCTAAC-1,AGACACTGTGTCTAAC-1,1.8820001822323045,15.0,-0.07368941567341505,-0.015534136508612188,9.900773,-6.8847203,23,G1,1165.6580424457788,6,8,4744.0,1700.0,9.064080944350758,22.84991568296796,0.2225738408939001,False,False,5.800000000000001,13.998683,-5.293639,1188.0,9.0,6,1.3391534867442854
-142,True,AGACACTTCAAAGGTA-1,AGACACTTCAAAGGTA-1,1.9269118204392879,15.0,-0.08089074840278257,-0.015076211964687398,0.5829504,5.2405825,5,G1,1041.7845131754875,1,1,5667.0,1459.0,9.175930827598377,44.150344097406034,0.6105809437307363,False,False,4.199999999999999,8.668133,9.110955,690.0,9.0,1,1.1192238731153785
-143,True,AGACACTTCTTAGCAG-1,AGACACTTCTTAGCAG-1,1.9441010837991037,15.0,-0.11096538416494359,-0.004182940305674839,-0.6257613,15.682435,2,G1,1148.8032605946064,5,5,5891.0,1753.0,8.962824647767782,40.1120353080971,0.7262852959990129,False,False,5.1,-0.7066064,14.042106,1299.0,9.0,5,0.8054144824310264
-144,True,AGGGAGTAGCACCTGC-1,AGGGAGTAGCACCTGC-1,1.957833556890565,15.0,-0.1010389775011694,-0.013971709839682628,-7.7296343,-8.169029,3,G1,1251.8619155734777,2,7,10688.0,3002.0,10.086077844311378,16.88809880239521,0.9601241395770446,False,False,3.7999999999999994,-13.7501955,-8.747741,3048.0,9.0,2,1.3713055388634732
-145,True,AGGGAGTAGCACTCTA-1,AGGGAGTAGCACTCTA-1,1.906197269797348,15.0,-0.09310814853762016,-0.02396000367308873,10.165238,-5.8065104,23,G1,1141.7077158242464,6,8,4355.0,1426.0,7.715269804822044,34.489092996555684,0.25257303345337373,False,False,6.900000000000001,13.918837,-4.66509,923.0,9.0,6,1.3360037577711967
-146,True,AGGGAGTAGGATCACG-1,AGGGAGTAGGATCACG-1,1.9059903968497969,15.0,-0.10153118831231757,-0.007672518885136112,1.7252922,2.5899122,12,G1,1354.4148722738028,1,1,4929.0,1301.0,4.9502941773179145,51.57232704402516,0.6187537231918308,False,False,3.7,10.973508,9.19703,1519.0,9.0,1,1.0069201340880614
-147,True,AGGGAGTAGGGCGAAG-1,AGGGAGTAGGGCGAAG-1,1.9824154310308426,15.0,-0.06755085394336002,-0.012481272802121905,-8.289623,-1.2659498,10,G1,1503.642642363906,3,3,7649.0,2512.0,8.301738789384233,17.505556281866912,0.9666225764192599,False,False,4.199999999999999,-13.826553,-2.6719792,2334.0,9.0,3,0.8054144824310264
-148,True,AGGGAGTAGGTTACAA-1,AGGGAGTAGGTTACAA-1,1.9916368734247434,15.0,-0.08289469966228921,-0.019197495020508572,-7.44249,-3.4943151,10,G1,1436.41731454432,3,3,8188.0,2670.0,8.121641426477773,14.521250610649732,0.9715883030295386,True,True,3.6000000000000005,-15.73977,-3.5475662,1623.0,9.0,3,1.4399161372346798
-149,True,AGGGAGTAGGTTGGAC-1,AGGGAGTAGGTTGGAC-1,2.045645307965738,15.0,-0.06708119969053972,-0.00813391840403057,-7.186732,1.3808044,4,G1,1546.6160161197186,4,4,5178.0,1981.0,9.096176129779838,11.181923522595596,0.9567986740215486,False,False,5.4,-14.513669,0.23000188,1889.0,9.0,4,1.370249510524696
-150,True,AGGGAGTAGTAGTCTC-1,AGGGAGTAGTAGTCTC-1,1.9407983827117277,15.0,-0.09045136318029473,-0.017473515399601017,-12.124252,-2.9278862,7,G1,998.6132479161024,3,3,12745.0,3429.0,6.786975284425265,17.779521380933698,0.9999341963321966,True,True,4.7,-18.799204,-3.3198898,3176.0,9.0,3,0.937235816181699
-152,True,AGGGAGTGTAGCGTAG-1,AGGGAGTGTAGCGTAG-1,2.0300153716423677,15.0,-0.06741954001546069,-0.029307386527127985,-9.151705,-1.4656878,7,G1,1656.9634977132082,3,3,7288.0,2474.0,14.297475301866081,13.80351262349067,0.9844431669873921,False,False,5.199999999999999,-15.536969,-2.2467148,4014.0,9.0,3,1.7076582637188416
-154,True,AGGGAGTGTTCGGCTG-1,AGGGAGTGTTCGGCTG-1,1.94650394794739,15.0,-0.10189891699436926,-0.019138538736746657,-7.066377,-9.656598,3,G1,1184.8417666703463,2,2,9938.0,2639.0,8.200845240491045,23.032803380961965,0.949329502399926,False,False,3.1000000000000005,-14.686593,-9.002657,2290.0,9.0,2,1.110800675967786
-155,True,AGGGAGTTCAAGAAAC-1,AGGGAGTTCAAGAAAC-1,1.9470315343977591,15.0,-0.1221571468020825,-0.009736829450266707,2.057646,15.849435,2,G1,1073.5921186804771,5,5,5826.0,1741.0,6.2306900102986615,43.97528321318229,0.7194423108915827,False,True,9.3,-2.6589324,14.116144,821.0,9.0,5,0.9575994643424515
-156,True,AGGGAGTTCATGCCCT-1,AGGGAGTTCATGCCCT-1,1.8969043107268002,15.0,-0.08766494638777043,-0.020550939876704404,7.7138553,1.3971434,22,G1,1253.4215150773525,6,9,4605.0,1465.0,8.816503800217156,36.82953311617807,0.586740977593362,False,False,3.6999999999999997,13.878795,5.502858,825.0,9.0,6,0.8054144824310264
-157,True,AGGGAGTTCCCAACTC-1,AGGGAGTTCCCAACTC-1,1.9219937610651605,15.0,-0.1123819906081104,0.0019493241913706681,1.8143669,5.374422,5,S,1424.89253911376,1,1,6285.0,1448.0,8.035003977724742,51.20127287191726,0.6114980857402794,False,False,3.1999999999999997,9.387725,9.979434,1119.0,9.0,1,0.8054144824310264
-158,True,AGGGAGTTCCGCTTAC-1,AGGGAGTTCCGCTTAC-1,2.0088915619024723,15.0,-0.06757332338727688,-0.01647575850072382,-6.2522187,3.467735,14,G1,1441.8648898303509,4,4,5180.0,1926.0,9.45945945945946,13.32046332046332,0.9383917666508493,False,False,7.0,-14.71522,2.3763268,2353.0,9.0,4,1.6507563310617184
-160,True,AGGGAGTTCTCGAGTA-1,AGGGAGTTCTCGAGTA-1,1.8915453986936526,15.0,-0.10407496477364017,-0.010294432715070396,1.2022097,6.2667117,5,G1,1752.7356855422258,1,1,3485.0,1022.0,6.743185078909613,45.423242467718794,0.613937168097773,False,False,7.199999999999999,9.478535,11.157132,1007.0,9.0,1,1.2119072064054222
-161,True,AGGGAGTTCTCTAGGA-1,AGGGAGTTCTCTAGGA-1,1.957867669246695,15.0,-0.10691768231697926,-0.014312429421570091,2.7061987,18.901005,18,G1,1196.3447940945625,5,5,4406.0,1522.0,7.149341806627326,36.63186563776668,0.7231791866237086,False,False,9.1,-3.704866,15.647907,958.0,9.0,5,0.9980207379784404
-162,True,AGGGTGAAGCACCTGC-1,AGGGTGAAGCACCTGC-1,2.0397586146031923,15.0,-0.01996084022351966,-0.036177820359488175,-5.1781807,3.2959056,14,G1,1437.028485968709,4,4,6856.0,2588.0,12.689614935822638,10.064177362893815,0.927258647654456,False,False,6.800000000000001,-14.027452,2.148273,1711.0,9.0,4,2.239958061671911
-163,True,AGGGTGAAGCTTAAGA-1,AGGGTGAAGCTTAAGA-1,1.9802107172895436,15.0,-0.10999061447410213,-0.026561380306396588,2.1311727,18.807083,18,G1,1207.519254386425,5,5,3509.0,1244.0,14.44856084354517,34.65374750641208,0.7234316659834791,False,False,9.100000000000001,-3.6191807,15.690668,767.0,9.0,5,1.5744992908810693
-164,True,AGGGTGAAGGCCTAGA-1,AGGGTGAAGGCCTAGA-1,1.9990835068917752,15.0,-0.07236404544577993,-0.023745254319913447,-6.5546765,0.42638955,4,G1,1621.4113033562899,4,4,6172.0,2254.0,9.105638366817887,13.204795852235904,0.9625973898219361,False,False,4.3999999999999995,-15.174699,-0.32964537,1397.0,9.0,4,1.817077281330816
-165,True,AGGGTGAAGGCGTTAG-1,AGGGTGAAGGCGTTAG-1,1.9620204041931226,15.0,-0.12935582014355124,-0.01120823409532776,0.9282501,15.69259,2,G1,1107.087795048952,5,5,6146.0,1525.0,9.192971038073544,47.81972014318256,0.7186362275638372,False,False,9.4,-2.0567956,14.492488,993.0,9.0,5,1.352031409542768
-166,True,AGGGTGAAGGGAGGTG-1,AGGGTGAAGGGAGGTG-1,2.0138349006251075,15.0,-0.07452376690657511,-0.02074191522205611,-4.779829,1.095592,14,G1,1437.709598749876,4,4,5287.0,1941.0,13.48590883298657,13.410251560431247,0.9466798625089519,False,False,4.800000000000001,-14.562924,0.4363549,1552.0,9.0,4,0.8054144824310264
-167,True,AGGGTGAAGTCATCCA-1,AGGGTGAAGTCATCCA-1,1.8778471941564088,15.0,-0.10061880565477688,-0.020631584506698004,10.136134,-5.513826,23,G1,1140.2399310171604,6,8,4587.0,1622.0,6.780030521037715,32.962720732504906,0.2707591250544456,False,False,6.8,14.111893,-4.0814323,1602.0,9.0,6,1.4945853239801525
-168,True,AGGGTGACACTACGGC-1,AGGGTGACACTACGGC-1,1.9297264228309974,15.0,-0.0646902203118325,-0.024332676351071853,7.919957,-0.12238291,22,G1,1071.0502562969923,6,8,2793.0,1036.0,15.216612960973864,28.82205513784461,0.5444924458367872,False,False,4.699999999999999,13.967702,3.0003886,1149.0,9.0,6,0.986549540679356
-169,True,AGGGTGACAGCAGTGA-1,AGGGTGACAGCAGTGA-1,1.984137542044802,15.0,-0.0756494689965495,-0.02336710021705623,-8.917856,-2.380756,7,G1,1725.3096423447132,3,3,12338.0,3211.0,6.921705300697034,17.036796887664128,0.982287713185272,False,False,3.1,-16.2113,-3.0082824,1873.0,9.0,3,1.3383842755014106
-170,True,AGGGTGACATCAGTGT-1,AGGGTGACATCAGTGT-1,1.9097789450653049,15.0,-0.06911803679929535,-0.004740299571824414,8.035425,8.021349,27,G1,1185.869840875268,8,10,4896.0,1584.0,10.008169934640524,36.580882352941174,0.6139213389126777,False,False,5.2,13.851757,10.124836,1606.0,9.0,8,1.3006826524250614
-171,True,AGGGTGACATCCTTCG-1,AGGGTGACATCCTTCG-1,1.9009450620398822,15.0,-0.09437997948404876,-0.019304455536605078,7.2996416,-2.4540176,22,G1,1187.2551356852055,6,8,4876.0,1591.0,10.787530762920426,33.326497128794095,0.4635339802439194,False,False,7.1000000000000005,14.387135,0.54983985,1176.0,9.0,6,1.5689939908624575
-172,True,AGGGTGAGTCAAGCGA-1,AGGGTGAGTCAAGCGA-1,1.9075905945102112,15.0,-0.11464967359113308,-0.019072546692231993,2.863798,7.134179,5,G1,1250.507382042706,1,1,4785.0,1241.0,5.287356321839081,49.61337513061651,0.6118634172165451,False,False,3.1000000000000005,8.636889,9.681277,1540.0,9.0,1,1.545777420304805
-173,True,AGGGTGAGTTAAGTCC-1,AGGGTGAGTTAAGTCC-1,1.9017889465225057,15.0,-0.0794714482447183,-0.0039144269649442585,9.339613,-8.561738,11,G1,1211.5488028079271,7,6,5303.0,1857.0,11.578351876296436,26.89043937393928,0.1171388206921545,False,False,6.2,13.480804,-8.712111,1976.0,9.0,7,1.344406567033444
-174,True,AGGGTGATCATCGCTC-1,AGGGTGATCATCGCTC-1,1.8779006157448968,15.0,-0.08281205911869458,-0.011025243240351604,10.231219,-8.225378,11,G1,1215.2654928863049,7,6,6378.0,1915.0,6.945751019128253,32.659140796487925,0.14250351890000915,False,False,6.3999999999999995,13.771712,-7.422075,1455.0,9.0,7,1.3259779288380678
-175,True,AGGGTGATCCATTGCC-1,AGGGTGATCCATTGCC-1,2.018844681362616,15.0,-0.03302024198990963,-0.021773368194770347,-3.3272517,-5.6114087,3,G1,1426.1243579685688,2,2,4614.0,1955.0,18.053749458170785,11.009969657563936,0.9554010779097449,False,False,2.7000000000000006,-15.034717,-4.8673525,1900.0,10.0,2,1.2280874339147625
-176,True,AGGGTGATCCGAGAAG-1,AGGGTGATCCGAGAAG-1,1.9273932051682094,15.0,-0.07758609583544984,0.004610722989903136,8.081143,-1.1625338,22,S,1136.9777319580317,6,8,5184.0,1451.0,9.567901234567902,39.891975308641975,0.5030672911493764,False,False,4.6000000000000005,14.282672,1.9146088,1764.0,9.0,6,1.285062833113634
-177,True,AGGGTGATCCGCTTAC-1,AGGGTGATCCGCTTAC-1,1.9787561670628313,15.0,-0.08542220027141212,-0.027226092661780523,-10.037198,-8.245152,10,G1,1075.686592578888,3,7,13532.0,3308.0,10.30889742831806,18.238250073898907,0.9719571896436185,False,False,4.699999999999999,-12.228919,-8.838771,3168.0,9.0,3,0.8054144824310264
-178,True,AGGGTGATCGAACGCC-1,AGGGTGATCGAACGCC-1,1.9381192303650645,15.0,-0.07732169139685383,-0.014798374711734053,-4.8147006,-6.299513,19,G1,1351.0896033793688,2,2,12156.0,3578.0,6.5893385982231,13.984863441921684,0.9580476582948024,False,False,4.1,-15.775162,-6.420506,3278.0,9.0,2,1.23753771102208
-179,True,AGTAACCAGCTCACTA-1,AGTAACCAGCTCACTA-1,2.008151482078859,15.0,-0.0339328893908614,-0.02808084933825431,-6.2390175,-2.2200975,14,G1,1354.292534172535,4,2,7552.0,2712.0,14.009533898305085,9.229343220338983,0.9703430967145152,False,False,2.1,-16.745632,-2.1484218,2363.0,9.0,4,0.8054144824310264
-180,True,AGTAACCAGTGTTGAA-1,AGTAACCAGTGTTGAA-1,1.8992957169099052,15.0,-0.11292960516013482,-0.01974836440731568,-3.2247796,12.846995,2,G1,931.7019924521446,5,5,11717.0,3220.0,5.6499103866177345,30.152769480242384,0.7226957996154734,False,False,3.3,2.964976,13.1519785,1184.0,9.0,5,1.153800596728569
-181,True,AGTAACCCAAACCATC-1,AGTAACCCAAACCATC-1,1.9896787941171463,15.0,-0.07095069331114916,-0.02781177677856159,-8.459632,-1.586563,4,G1,1593.954465046525,4,3,15044.0,3524.0,7.577771869183728,13.659930869449614,0.9825523099230157,False,False,4.0,-14.851536,-2.3525853,2157.0,9.0,4,1.8145340755406008
-182,True,AGTAACCCACCGTGAC-1,AGTAACCCACCGTGAC-1,2.009012127312308,15.0,-0.06729689107636799,-0.04253783513799518,-7.039367,0.11341624,4,G1,1565.6348773539066,4,4,5569.0,2237.0,13.14419105764051,8.888489854551985,0.9591302073557765,False,False,3.5999999999999996,-14.823204,-0.9303894,2067.0,9.0,4,1.112774701128965
-183,True,AGTAACCCACGCGCAT-1,AGTAACCCACGCGCAT-1,1.9032648073584948,15.0,-0.1240869825472873,-0.01533688901170209,5.6774244,7.6212363,27,G1,1136.2247373312712,8,1,6264.0,1590.0,7.263729246487867,46.45593869731801,0.6142725716950314,True,True,2.0,11.633839,11.094412,1861.0,9.0,8,0.8054144824310264
-184,True,AGTAACCCACTAAACC-1,AGTAACCCACTAAACC-1,1.8797399209536272,15.0,-0.09205308541206794,-0.01619016115624707,8.812552,9.291855,15,G1,1135.8423383831978,8,10,2847.0,1053.0,3.582718651211802,41.55251141552512,0.6182550155039122,False,False,5.5,13.908773,10.854307,1920.0,9.0,8,1.333463942409766
-185,True,AGTAACCCACTACACA-1,AGTAACCCACTACACA-1,1.9129600248697474,15.0,-0.06965951546884966,-0.019162463805250857,-2.582553,-13.376397,24,G1,1323.8105005770922,10,7,17067.0,3698.0,9.222476123513212,19.558211753676687,0.9592329285962357,False,True,6.8,-6.501388,-11.774815,2446.0,9.0,10,1.07800882493837
-186,True,AGTAACCGTAGCGTAG-1,AGTAACCGTAGCGTAG-1,2.0479107710387545,15.0,-0.0637804137801157,-0.011294328907686692,-5.208605,3.8224652,14,G1,1323.0723382532597,4,4,3901.0,1714.0,15.175596001025378,15.252499359138682,0.9155700482144069,False,False,6.3,-13.824158,2.919669,1775.0,10.0,4,1.044757763005983
-187,True,AGTAACCGTCGAATGG-1,AGTAACCGTCGAATGG-1,1.966959372870528,15.0,-0.09017281089723979,-0.02652180244756926,-6.880208,-9.277692,3,G1,1202.3733336031437,2,2,8362.0,2783.0,11.95886151638364,16.443434585027504,0.9504781875501616,False,False,4.7,-14.952475,-8.613107,2918.0,9.0,2,1.1075219258728686
-188,True,AGTAACCGTCGTGCCA-1,AGTAACCGTCGTGCCA-1,1.9742828157154368,15.0,-0.06954572183639268,-0.025004564484149224,-2.6510036,-4.710017,19,G1,1416.8228048682213,2,2,6928.0,2396.0,10.825635103926096,16.35392609699769,0.9536384157550776,False,False,4.4,-14.879069,-4.730261,2968.0,9.0,2,1.4744857463590444
-189,True,AGTAACCGTCTAGGTT-1,AGTAACCGTCTAGGTT-1,2.0197237760144637,15.0,-0.0577421903198252,-0.028726369794203486,-6.947303,1.1455452,14,G1,1529.4890487343073,4,4,9901.0,3008.0,8.514291485708513,12.311887688112312,0.9548686289515712,False,False,4.2,-15.197692,0.34337413,2003.0,9.0,4,1.4081095231790977
-190,True,AGTAACCGTTGGACCC-1,AGTAACCGTTGGACCC-1,1.9735451985040848,15.0,-0.10391885525440755,-0.007243338666475434,-1.988978,15.579804,2,G1,1234.7291979640722,5,5,6421.0,1725.0,10.247624980532628,40.10278772776826,0.7321124975607457,False,False,7.1,0.26219994,14.023526,1491.0,9.0,5,0.8054144824310264
-191,True,AGTAACCTCAAGAAAC-1,AGTAACCTCAAGAAAC-1,1.944006566182653,15.0,-0.04604359597004596,-0.032834760215155585,-9.059354,-8.093079,10,G1,1133.6020619124174,3,7,7968.0,2791.0,8.170180722891565,12.625502008032129,0.9678084716665716,False,False,3.8000000000000003,-13.192566,-8.37319,3920.0,9.0,3,1.4165992585769842
-192,True,AGTAACCTCGCGGTAC-1,AGTAACCTCGCGGTAC-1,1.9684426437387044,15.0,-0.06981902319686387,-0.030591401535327117,-5.111448,-3.6757283,13,G1,985.5017040669918,2,2,10520.0,3287.0,9.923954372623575,16.378326996197718,0.9590778097497064,False,False,3.6999999999999997,-16.837934,-4.378158,4817.0,9.0,2,1.3205567218415608
-193,True,AGTAACCTCTCCAATT-1,AGTAACCTCTCCAATT-1,1.8979634203759799,15.0,-0.059929151473997605,-0.019841625535233037,8.9587145,-11.733198,21,G1,1175.9114208370447,7,6,3791.0,1521.0,11.9493537325244,20.047480875758374,0.04120311929676538,False,True,4.1,11.924159,-11.164298,1336.0,9.0,7,0.8054144824310264
-194,True,AGTAACCTCTGGCTGG-1,AGTAACCTCTGGCTGG-1,1.8859184198541095,15.0,-0.07800812547929072,-0.01609809708634754,0.5149173,7.7141066,5,G1,1611.2110636085272,1,1,3587.0,1252.0,7.081126289378311,37.106216894340676,0.620537055212367,False,False,6.3,9.579427,12.532116,1578.0,9.0,1,1.0519082176203531
-195,True,AGTAGTCAGACCAGAC-1,AGTAGTCAGACCAGAC-1,2.039567564149452,15.0,-0.04368265590547907,-0.0221483942414175,-4.107729,1.497595,14,G1,1393.0325990766287,4,4,5805.0,2179.0,10.180878552971576,11.541774332472007,0.9392906681895427,False,False,5.2,-15.035425,0.99208266,1983.0,9.0,4,1.062821352303575
-196,True,AGTAGTCAGCGAATGC-1,AGTAGTCAGCGAATGC-1,2.015483401242211,15.0,-0.06146055654566803,-0.023648989718452237,-2.581624,-4.78026,19,G1,1411.7691136449575,2,2,10670.0,3180.0,9.14714151827554,16.016869728209933,0.9556042426675108,True,True,4.2,-15.53265,-4.7096996,1886.0,9.0,2,1.1330446603053392
-197,True,AGTAGTCAGCTTGTTG-1,AGTAGTCAGCTTGTTG-1,1.912242012441276,15.0,-0.08679128439163991,-0.02344206497143412,-2.797753,-12.44052,24,G1,1319.2929827868938,10,7,13188.0,3003.0,7.089778586593873,21.40582347588717,0.959287740048452,False,True,7.599999999999999,-6.940486,-11.6441765,2032.0,9.0,10,1.5709628405879854
-198,True,AGTAGTCAGGAACTCG-1,AGTAGTCAGGAACTCG-1,1.9873961084059872,15.0,-0.07536777742113586,-0.03180078686106828,-10.436971,-2.4028747,20,G1,1467.7244146764278,4,3,10149.0,3191.0,7.350477879594049,12.602226820376393,0.9916565024382628,False,False,4.6000000000000005,-17.714714,-2.9400742,2963.0,9.0,4,1.427718777406776
-201,True,AGTAGTCCAGCTGCCA-1,AGTAGTCCAGCTGCCA-1,1.9790214285723737,15.0,-0.06411211940006553,-0.022578176937646598,-9.694992,-6.67721,10,G1,1045.421784967184,3,7,11389.0,3182.0,7.928703134603565,7.99894635174291,0.9767350652129011,True,True,3.4,-13.352134,-7.103583,2499.0,9.0,3,0.8054144824310264
-202,True,AGTAGTCGTATGCGTT-1,AGTAGTCGTATGCGTT-1,1.8886523578399372,15.0,-0.11435510093828517,-0.008910542891125414,3.207613,3.4545557,12,G1,1844.081807360053,1,1,4680.0,1333.0,4.358974358974359,47.67094017094017,0.612063794400627,False,False,5.0,11.042665,9.332835,1043.0,9.0,1,1.6482295406833074
-203,True,AGTAGTCGTGATTGGG-1,AGTAGTCGTGATTGGG-1,1.9784014444950915,15.0,-0.06917126923529704,-0.023325660312419394,-7.671123,-7.6788607,3,G1,1329.9316990971565,2,2,9153.0,2976.0,11.001857314541681,12.45493280891511,0.9609477704302996,False,False,3.3999999999999995,-14.183264,-7.749397,3278.0,9.0,2,1.204339849734597
-204,True,AGTAGTCGTGGCTTAT-1,AGTAGTCGTGGCTTAT-1,1.937370792346772,15.0,-0.0852476690319341,-0.018331858335112583,-6.3143053,-11.665253,10,G1,1225.1400522440672,3,7,13843.0,3341.0,10.879144694069204,22.849093404608826,0.9621849601356224,False,False,4.3999999999999995,-9.601259,-10.608301,1938.0,9.0,3,1.0986191307828845
-205,True,AGTAGTCGTTCGAAGG-1,AGTAGTCGTTCGAAGG-1,1.9867834125907478,15.0,-0.06619035020003189,-0.027515078422428816,-9.52604,-1.9999309,1,G1,1493.4433780908585,3,3,9858.0,3273.0,8.734023128423615,12.507608034083992,0.9818048226334961,False,False,2.5999999999999996,-16.759935,-2.970612,1612.0,9.0,3,1.0120189530495938
-206,True,AGTAGTCTCATGCCCT-1,AGTAGTCTCATGCCCT-1,1.9759440166839446,15.0,-0.08981887448122243,-0.01769604859279354,-1.9479306,-2.5904353,14,G1,1044.4829179495573,4,2,9980.0,2953.0,9.829659318637274,25.29058116232465,0.8811024647326025,False,False,1.2000000000000002,-12.60621,-0.90279055,2606.0,9.0,4,0.9657673329367392
-207,True,AGTAGTCTCCCGTTGT-1,AGTAGTCTCCCGTTGT-1,1.9566393133094453,15.0,-0.09019056175573137,-0.02395964750923653,-5.616607,-7.8448715,3,G1,987.6026282757521,2,2,14605.0,3763.0,11.235878123930162,18.062307428962683,0.9454580692173806,False,False,2.4000000000000004,-16.21682,-8.240849,2635.0,9.0,2,0.9275761967036148
-208,True,AGTAGTCTCGCGGTAC-1,AGTAGTCTCGCGGTAC-1,1.8898046552053585,15.0,-0.08483936527603522,-0.020618995842050045,6.899056,-0.1468433,22,G1,1085.7298173457384,6,8,2518.0,965.0,10.007942811755361,35.18665607625099,0.5505351534293874,False,False,4.7,14.320609,3.734429,869.0,9.0,6,1.1081635770280527
-209,True,AGTAGTCTCGTAGAGG-1,AGTAGTCTCGTAGAGG-1,1.9105795564275383,15.0,-0.1131286876545566,-0.0035259013425582297,6.028894,8.085403,27,G1,1195.5219519138336,8,1,7133.0,1710.0,5.719893452965092,46.43207626524604,0.6150576209990805,False,False,3.3,11.183523,11.403489,1545.0,9.0,8,1.1719784193997438
-211,True,AGTGACTAGCATTGAA-1,AGTGACTAGCATTGAA-1,2.0034961839355736,15.0,-0.06685575998449146,-0.020980087342572083,-5.5544257,0.60409164,20,G1,1493.7320757359266,4,4,11818.0,3387.0,7.471653410052462,15.180233542054493,0.9563478267741482,False,False,4.6000000000000005,-16.24631,0.39875418,2402.0,9.0,4,1.8390623816332334
-212,True,AGTGACTAGCGAATGC-1,AGTGACTAGCGAATGC-1,1.9231910118938516,15.0,-0.09023858606266301,-0.024295603591455953,3.8922818,3.4011533,12,G1,1893.378217563033,1,1,4135.0,1251.0,11.825876662636034,38.30713422007255,0.6092172455330276,False,False,6.3,11.309192,9.340478,1862.0,9.0,1,1.6886958137388428
-213,True,AGTGACTAGGAACTCG-1,AGTGACTAGGAACTCG-1,1.920071075335419,15.0,-0.087238866424578,-0.018162970712132895,9.093022,-11.370275,21,G1,1245.0428095906973,7,6,4513.0,1648.0,12.231331708397962,22.26900066474629,0.04398950042688977,False,False,5.599999999999999,11.672203,-10.8875475,1219.0,9.0,7,1.4585607180548472
-214,True,AGTGACTAGGATCACG-1,AGTGACTAGGATCACG-1,1.9270208770563118,15.0,-0.09104138284243529,-0.024443387068289624,7.24496,0.76392496,22,G1,1207.7217156589031,6,8,4110.0,1422.0,14.549878345498783,26.32603406326034,0.5707021834581947,True,True,3.7,13.807022,4.9449224,1256.0,9.0,6,1.626654214416418
-216,True,AGTGACTAGTAGGAAG-1,AGTGACTAGTAGGAAG-1,1.9988659218296208,15.0,-0.08538139330770252,-0.030859069005625843,-2.5866919,-1.3580492,19,G1,1293.0350689888,2,2,4908.0,1986.0,8.822330888345558,17.685411572942137,0.9307242582898891,False,False,5.000000000000001,-12.996251,-2.3639677,1886.0,9.0,2,1.174574202767167
-217,True,AGTGACTAGTGTTGAA-1,AGTGACTAGTGTTGAA-1,1.9622873449948424,15.0,-0.09720421545130667,-0.0011106592041833022,-1.1784297,15.602561,2,G1,1209.9243125095963,5,5,6452.0,1773.0,11.174829510229387,36.26782393056417,0.7355953260871654,False,True,6.3,-0.11782662,13.948348,1100.0,9.0,5,1.2269461008431268
-218,True,AGTGACTCAATCTGCA-1,AGTGACTCAATCTGCA-1,1.9880115183805345,15.0,-0.06021523059274649,-0.02290354079722359,-9.072821,-0.38701987,20,G1,1396.35428789258,4,3,8920.0,2808.0,10.448430493273543,17.286995515695068,0.9779221227610093,False,False,3.3000000000000003,-17.1895,-1.0579098,1932.0,9.0,4,1.4431469775576604
-220,True,AGTGACTCATTATGCG-1,AGTGACTCATTATGCG-1,2.040474633948175,15.0,-0.03810735375768781,-0.015159116883418108,-9.975807,-1.0048057,7,G1,1558.08698579669,3,3,5023.0,1971.0,14.433605415090584,9.655584312164045,0.9839381123530931,False,False,3.9000000000000004,-16.572664,-1.4191266,10865.0,9.0,3,1.2926364428612958
-221,True,AGTGACTGTACGGGAT-1,AGTGACTGTACGGGAT-1,1.8658606667391968,15.0,-0.08483662106648691,-0.01990163608674161,9.00566,-9.225144,11,G1,1213.5385919809341,7,6,4509.0,1841.0,6.941672211133289,25.371479263694834,0.10769841955126792,False,False,6.1,13.443702,-8.845428,1306.0,9.0,7,1.0188893454215837
-222,True,AGTGACTGTAGTGTGG-1,AGTGACTGTAGTGTGG-1,1.944628520832997,15.0,-0.08391270472692411,-0.01967614359802309,-9.82459,-9.567346,10,G1,1095.5074007064104,3,7,11807.0,3525.0,8.063013466587618,15.109680697891081,0.9663011058020665,False,False,4.9,-11.612929,-9.865841,4917.0,9.0,3,0.9932064799220534
-223,True,AGTGACTGTGGCTTAT-1,AGTGACTGTGGCTTAT-1,2.124227584016134,15.0,-0.05961756698736719,-0.026519813201065764,-7.2831745,2.9054945,14,G1,1362.1653456538916,4,4,2343.0,1042.0,24.029022620571915,6.274007682458387,0.9097585860908094,False,False,3.1000000000000005,-13.562403,1.0645906,2140.0,9.0,4,1.08477779533976
-224,True,AGTGACTTCATTTGGG-1,AGTGACTTCATTTGGG-1,1.9660326938011705,15.0,-0.08647985928774644,-0.03467858394938445,-6.658715,-2.4828436,10,G1,1413.482744127512,3,3,7671.0,2771.0,8.916699256941728,15.47386259940034,0.9711161437698143,False,False,2.9000000000000004,-15.778843,-3.0696752,2211.0,9.0,3,0.8054144824310264
-225,True,AGTGACTTCCCAAGCG-1,AGTGACTTCCCAAGCG-1,1.9718846907674594,15.0,-0.10440350520140344,0.004274095929688174,-2.1045136,15.088611,2,S,1268.0870485082269,5,5,5019.0,1396.0,10.639569635385534,39.84857541342897,0.7329628686899524,False,False,7.2,0.67419714,14.312525,1213.0,9.0,5,1.5243698758630269
-226,True,AGTGACTTCGCCCAGA-1,AGTGACTTCGCCCAGA-1,2.1548967503416705,15.0,-0.056704552548730205,-0.040784983478895365,-6.0760794,3.2444549,14,G1,1443.4711663722992,4,4,3047.0,1327.0,24.71283229405973,5.218247456514605,0.9353626808878867,False,True,5.8999999999999995,-14.593301,2.5032594,2193.0,9.0,4,1.8220154132044484
-227,True,AGTGACTTCGTTTACT-1,AGTGACTTCGTTTACT-1,2.009436536363936,15.0,-0.057015311068813475,-0.02540834467297301,-4.2355447,0.96951485,14,G1,1450.369052067399,4,4,16407.0,3999.0,7.679648930334613,14.512098494545011,0.9464330930069054,False,False,4.3,-14.66405,0.6021441,1621.0,10.0,4,1.5543627449492794
-228,True,AGTGACTTCTAGTCAG-1,AGTGACTTCTAGTCAG-1,1.9639098841540013,15.0,-0.09826708870864945,-0.023261302572870433,-10.631753,-5.9143066,10,G1,1275.491935417056,3,3,9389.0,2846.0,8.158483331558207,16.338268186175313,0.9809677971901972,False,False,2.6,-12.736212,-7.151743,2268.0,9.0,3,0.8054144824310264
-229,True,AGTGACTTCTCTAGGA-1,AGTGACTTCTCTAGGA-1,1.9670531743471937,15.0,-0.07983617822789546,-0.02857715553333101,-3.3725173,-4.60185,19,G1,1543.4820107519627,2,2,7217.0,2654.0,7.038935845919357,13.731467368712762,0.9505352015279241,False,False,4.4,-14.686379,-4.291832,2830.0,9.0,2,0.8054144824310264
-230,True,ATGAGTCAGATACCAA-1,ATGAGTCAGATACCAA-1,1.910694998021482,15.0,-0.10755965562966284,-0.013925016427268452,7.507246,10.222009,15,G1,1099.2311912626028,8,10,7178.0,1791.0,7.564781276121482,45.36082474226804,0.6203969153168982,False,False,6.0,14.726801,10.089775,829.0,9.0,8,1.2945102574612721
-231,True,ATGAGTCAGGCCTAGA-1,ATGAGTCAGGCCTAGA-1,2.033929660476258,15.0,-0.06412187179771714,-0.0274650730306219,-7.8592477,1.0067067,14,G1,1383.5188588351011,4,4,5868.0,2013.0,13.173142467620995,9.423994546693933,0.9562336110848553,False,False,3.7,-13.515028,-0.41962734,1220.0,9.0,4,1.7615150195516065
-232,True,ATGAGTCAGTCCCAAT-1,ATGAGTCAGTCCCAAT-1,1.9382044608620894,15.0,-0.09087765020396948,-0.028752725636322026,-10.60772,-9.36925,10,G1,1148.315005555749,3,7,9014.0,3051.0,6.76725094297759,13.94497448413579,0.9715380091634137,False,False,5.999999999999999,-11.760579,-9.03043,3289.0,9.0,3,0.8054144824310264
-233,True,ATGAGTCCACATGAAA-1,ATGAGTCCACATGAAA-1,1.9862839619938597,15.0,-0.07988446316382732,-0.023590422386047312,-11.081515,-4.263356,7,G1,1269.61087885499,3,3,10428.0,3251.0,9.810126582278482,13.147295742232451,0.9930996329851969,False,False,3.4999999999999996,-18.020067,-4.368959,3314.0,9.0,3,1.4214155226461038
-234,True,ATGAGTCCATGGCTAT-1,ATGAGTCCATGGCTAT-1,1.9493445477216347,15.0,-0.03149080352974349,-0.03386769022352675,8.595844,5.521287,26,G1,1239.9330451861024,9,9,1548.0,737.0,25.0,6.976744186046512,0.6120179367616124,False,False,4.3,10.468837,5.819747,788.0,9.0,9,0.8054144824310264
-235,True,ATGAGTCGTATAGGAT-1,ATGAGTCGTATAGGAT-1,1.961525541381548,15.0,-0.1014056741883011,-0.011787332641642868,-6.0236382,-5.2765417,7,G1,1391.0980592221022,3,2,15877.0,3724.0,8.99414247023997,29.615166593185112,0.9620517735004553,False,False,3.3,-13.4641695,-5.423077,3006.0,9.0,3,1.3790528332138858
-236,True,ATGAGTCGTCACTCGG-1,ATGAGTCGTCACTCGG-1,1.9709729163855587,15.0,-0.09059679680429394,-0.0049051334565988515,-1.5361401,15.654716,2,G1,1192.3622210547328,5,5,7691.0,2240.0,9.894682095956313,31.543362371603173,0.7321771078566929,False,True,6.0,-0.31161445,13.993011,2057.0,9.0,5,1.3295008470439955
-237,True,ATGAGTCGTCGTGCCA-1,ATGAGTCGTCGTGCCA-1,1.9942356383646915,15.0,-0.07587572435515191,-0.025380229263724408,-6.7467384,-5.7821975,13,G1,1359.2146485298872,2,2,7800.0,2720.0,10.884615384615385,12.23076923076923,0.9607920176334105,False,False,2.5,-16.573494,-6.039786,3063.0,9.0,2,1.1541723609248176
-238,True,ATGAGTCGTGCACAAG-1,ATGAGTCGTGCACAAG-1,1.888700455832798,15.0,-0.08005214608204642,-0.017425486144527158,10.030667,-10.439313,21,G1,1267.19097648561,7,6,4704.0,1692.0,8.60969387755102,23.724489795918366,0.07761839504007188,False,False,5.499999999999999,12.448465,-9.778137,1078.0,9.0,7,1.1840045921432583
-239,True,ATGAGTCGTTAAGCAA-1,ATGAGTCGTTAAGCAA-1,1.9532027439023074,15.0,-0.08129605292827986,-0.029628611792187492,-10.010821,-4.804679,10,G1,1558.1191532462835,3,3,9044.0,2782.0,6.037151702786378,14.849624060150376,0.9828022972400092,False,False,3.3000000000000003,-15.278864,-4.9805813,2068.0,9.0,3,1.2131002172767453
-240,True,ATGAGTCGTTTGATCG-1,ATGAGTCGTTTGATCG-1,1.9230040837798366,15.0,-0.0732228360365308,-0.01127877926514623,-3.2522728,-12.142946,24,G1,1296.7913182526827,10,7,8899.0,2551.0,10.85515226429936,19.79997752556467,0.9595654502660406,True,True,6.8,-6.96506,-11.7019615,2412.0,9.0,10,1.3913939848529155
-241,True,ATGAGTCTCAATCCAG-1,ATGAGTCTCAATCCAG-1,1.8972339045678766,15.0,-0.10934662097452795,-0.011753021461759326,2.545958,4.2550983,12,G1,1813.8059920966625,1,1,6105.0,1702.0,4.307944307944308,45.06142506142506,0.607718519717103,False,False,4.1,10.436038,9.445456,1412.0,9.0,1,1.2470856525873835
-242,True,ATGAGTCTCATCGCTC-1,ATGAGTCTCATCGCTC-1,1.9485914996478833,15.0,-0.0907139737301823,-0.016847045384850625,-6.5174108,-10.106189,3,G1,930.5267608761787,2,2,13068.0,3861.0,7.621671258034894,19.589837771655954,0.9445292153188589,False,False,4.1,-15.194768,-9.461049,1470.0,9.0,2,0.8054144824310264
-243,True,ATGAGTCTCCTTATCA-1,ATGAGTCTCCTTATCA-1,1.9185561604857253,15.0,-0.11519032569688643,-0.003575447063018964,9.837386,1.5639766,25,G1,962.2551143467426,6,9,10409.0,2277.0,6.936305120568739,48.05456816216736,0.5912641326524077,True,True,4.2,13.338084,4.0977106,571.0,9.0,6,0.9000524015178514
-244,True,ATGAGTCTCCTTCACG-1,ATGAGTCTCCTTCACG-1,1.909566683084526,15.0,-0.11364707552075888,-0.009640580438490386,5.000597,5.7478833,12,G1,1517.232741497457,1,1,5630.0,1567.0,6.980461811722913,44.95559502664298,0.6112424642399018,False,False,4.0,9.9637575,7.8894196,1088.0,9.0,1,1.6399485244002727
-245,True,ATGAGTCTCGTTTACT-1,ATGAGTCTCGTTTACT-1,1.8788403126855981,15.0,-0.07333624725693144,-0.023709955968888252,8.862596,-13.644124,21,G1,1139.287091806531,7,6,3970.0,1558.0,9.294710327455919,18.035264483627206,0.0034460559644219056,False,False,7.3,10.017577,-11.7891035,729.0,9.0,7,1.5382125678571381
-246,True,CACAGATAGCAGGTCA-1,CACAGATAGCAGGTCA-1,1.898174051312316,15.0,-0.11803585446996301,-0.021375723641095808,7.808576,9.525522,15,G1,1122.682582437992,8,10,7425.0,2019.0,5.5353535353535355,44.0,0.6201380679485924,False,False,5.2,14.875615,10.343692,1560.0,9.0,8,1.389617141286779
-247,True,CACAGATAGCCTCTGG-1,CACAGATAGCCTCTGG-1,2.0934914231895725,15.0,-0.05426606088277783,-0.04197641196463149,-7.9298472,-12.2404375,16,G1,641.9509965926409,2,2,1434.0,629.0,6.066945606694561,2.510460251046025,0.9315334324549855,False,False,6.1,-15.350565,-11.500702,1062.0,9.0,2,0.9125033401313234
-248,True,CACAGATAGCGACATG-1,CACAGATAGCGACATG-1,1.905899827817694,15.0,-0.13339191047176205,-0.008636840495927807,5.919783,4.639442,6,G1,1597.5391537323594,1,1,6568.0,1566.0,5.67904993909866,49.84774665042631,0.6096368825558802,False,False,3.1,11.957175,7.931293,1447.0,9.0,1,1.3606883983701565
-249,True,CACAGATAGTCCCAAT-1,CACAGATAGTCCCAAT-1,1.8858406616845709,15.0,-0.10830045531129179,-0.0066014400381299194,9.584839,3.9964867,8,G1,1128.0994460433722,9,9,6303.0,2069.0,6.615897191813422,36.12565445026178,0.6109226635081283,False,False,3.5000000000000004,12.084686,5.3174706,1755.0,9.0,9,0.9577368584765362
-250,True,CACAGATCAAGAGAGA-1,CACAGATCAAGAGAGA-1,1.9741766965528793,15.0,-0.08120035503877043,-0.021487682362265646,-3.2732892,-1.6592981,19,G1,1356.8133256584406,2,2,11087.0,3201.0,7.495264724452061,19.806981149093534,0.9440600790098771,False,False,3.8,-13.2692995,-2.2195203,2308.0,9.0,2,1.6533085336883095
-252,True,CACAGATCAATTAGGA-1,CACAGATCAATTAGGA-1,1.9628979902706531,15.0,-0.08293546945017305,-0.02460502592472946,-11.262711,-5.558702,1,G1,1187.7571196556091,3,3,8166.0,2577.0,12.417340191036002,14.952240999265246,0.9869834782863756,False,False,3.4000000000000004,-17.816526,-5.113067,2501.0,9.0,3,1.3445335173275872
-253,True,CACAGATCACGCGCAT-1,CACAGATCACGCGCAT-1,1.9091945917701743,15.0,-0.0866817229890888,-0.027372228265579807,-2.8760626,-13.046076,24,G1,1323.607300773263,10,7,17358.0,4060.0,6.988132273303376,20.831893075239083,0.959061183069423,False,False,7.5,-6.6627893,-11.961482,3433.0,9.0,10,1.2535270090677104
-254,True,CACAGATCACTAAACC-1,CACAGATCACTAAACC-1,1.909125968005788,15.0,-0.10948072022806724,-0.00038473302413622595,5.4973087,5.610584,12,G1,1744.458917349577,1,1,7210.0,1684.0,6.5325936199722605,50.48543689320388,0.6121234524306127,False,False,4.8,11.83121,9.444659,1164.0,9.0,1,1.2614974210295193
-255,True,CACAGATCAGCTGCCA-1,CACAGATCAGCTGCCA-1,1.9471749863806758,15.0,-0.10124475217216014,-0.014079676828782604,-2.498184,14.80123,2,G1,1252.6248314082623,5,5,8892.0,2106.0,7.658569500674764,43.71345029239766,0.735898360129958,False,False,6.8999999999999995,1.2036521,14.435313,986.0,9.0,5,1.0689573863442006
-256,True,CACAGATCAGGGAATC-1,CACAGATCAGGGAATC-1,1.8613033638757588,15.0,-0.08065933242879653,-0.0037298348822198575,10.259737,-6.2967033,23,G1,1131.5720804184675,6,8,4140.0,1652.0,8.88888888888889,22.8743961352657,0.21764991458768254,False,False,6.1000000000000005,13.817165,-5.373298,1376.0,9.0,6,0.9697530029822112
-257,True,CACAGATCATGTTCGA-1,CACAGATCATGTTCGA-1,1.9616032279032383,15.0,-0.08706609813750579,-0.01821969736669793,-6.4216986,-2.4200215,9,G1,1351.7011956125498,2,2,8734.0,2608.0,8.953514998855049,27.83375314861461,0.9653114918597391,False,False,3.1999999999999997,-16.749008,-2.6551,2857.0,9.0,2,1.0305418786578064
-258,True,CACAGATGTCAAGCCC-1,CACAGATGTCAAGCCC-1,1.9519790683451264,15.0,-0.08635927679051328,-0.022463477557814162,-4.115569,-12.60929,24,G1,1305.9845884144306,10,7,14559.0,3744.0,10.454014698811731,17.350092726148773,0.9600976374165852,False,False,5.699999999999999,-7.8798165,-11.245723,1921.0,9.0,10,1.408997248830342
-259,True,CACAGATGTCACTCGG-1,CACAGATGTCACTCGG-1,1.8868835987017194,15.0,-0.10545702514469354,-0.01035548293414312,8.120695,10.056861,15,G1,1057.6837232112885,8,10,6603.0,1843.0,6.860517946388005,42.526124488868696,0.6191842498840736,False,False,4.8999999999999995,15.10931,10.704387,1041.0,9.0,8,0.9497939589833029
-260,True,CACAGATGTCCTTAAG-1,CACAGATGTCCTTAAG-1,1.9533978814964625,15.0,-0.07155278635282526,-0.020770586335033665,-6.764694,-8.085549,3,G1,1363.4560393095016,2,2,13384.0,3768.0,9.982068141063957,15.189778840406456,0.9561299280158662,False,False,4.3,-15.0842285,-7.612093,3105.0,9.0,2,1.6488763139858706
-261,True,CACAGATGTGCACAAG-1,CACAGATGTGCACAAG-1,1.9779090884904582,15.0,-0.08287318316838081,-0.019529119772589674,-7.2426553,0.97742075,14,G1,1312.5899412930012,4,4,8075.0,2627.0,7.306501547987616,14.402476780185758,0.9550102196527467,False,False,4.4,-13.86885,-0.24455814,1853.0,9.0,4,1.204438876733728
-262,True,CACAGATGTTAAGACA-1,CACAGATGTTAAGACA-1,1.9830480234972334,15.0,-0.09527086247401946,-0.02628467528379961,-6.5058575,-9.116389,3,G1,1066.927904829383,2,2,13737.0,3766.0,10.912135109558127,14.049646938924074,0.9425982751423084,False,False,4.0,-15.082209,-9.436545,3322.0,9.0,2,1.3223600882395572
-263,True,CACAGATGTTCCACGG-1,CACAGATGTTCCACGG-1,1.8958746942479703,15.0,-0.10356223633511522,-0.03891124802105785,6.307671,6.438841,12,G1,1365.1214649826288,1,1,2545.0,970.0,9.587426326129666,28.33005893909627,0.6146709326254021,False,False,5.0,10.134251,7.1058884,1158.0,9.0,1,1.4180889356972706
-264,True,CACAGATGTTGGAGGT-1,CACAGATGTTGGAGGT-1,1.9748341599065642,15.0,-0.08506303035137558,-0.018668447231858683,-9.223729,-10.1375065,10,G1,1181.183958351612,3,7,12110.0,3456.0,12.601156069364162,17.159372419488026,0.964871546327314,False,False,5.6,-10.804631,-9.927038,3207.0,9.0,3,1.640550436902247
-265,True,CACAGATTCAAGAAAC-1,CACAGATTCAAGAAAC-1,1.9134856020367523,15.0,-0.08419378288376399,-0.0051882798562666,9.0042515,-13.854578,21,G1,1120.239810794592,7,6,4099.0,1651.0,14.174188826543059,16.467431080751403,0.0035375530519038798,False,False,6.499999999999999,10.436758,-11.754163,1036.0,9.0,7,1.2323484104496285
-266,True,CACAGATTCATGCCCT-1,CACAGATTCATGCCCT-1,1.9481054751152984,15.0,-0.08472481176051987,-0.026824819284657166,-8.722018,-10.356675,10,G1,1164.206478729844,3,7,16573.0,3997.0,10.861039039401437,19.549870270922586,0.9657253455844346,False,False,6.000000000000001,-10.887927,-10.057448,3835.0,9.0,3,1.499041005359969
-267,True,CACAGATTCCATTGCC-1,CACAGATTCCATTGCC-1,1.8581455679214434,15.0,-0.06409620076133328,-0.020523672998327427,10.495759,-8.958761,11,G1,1226.2552442848682,7,6,5154.0,1951.0,8.110205665502523,20.391928599146294,0.13442962685379053,False,False,6.6,13.345274,-7.510192,2116.0,9.0,7,0.8054144824310264
-268,True,CACAGATTCCCAACTC-1,CACAGATTCCCAACTC-1,1.9093803348390341,15.0,-0.08828690165819103,-0.015869867935006395,-2.3800576,-12.490969,24,G1,1348.863736987114,10,7,13022.0,3330.0,7.917370603593918,19.682076485946858,0.9589984515008175,False,False,8.5,-6.640193,-11.979427,4485.0,9.0,10,0.9918336898289957
-269,True,CACAGATTCCGTATGA-1,CACAGATTCCGTATGA-1,1.9632509929798163,15.0,-0.08433901556778228,-0.01315216855900474,-10.7482815,-6.2994127,1,G1,1097.8861177563667,3,7,12746.0,3692.0,8.857680841048172,14.231915895182803,0.984780307491724,False,False,3.1000000000000005,-13.469548,-7.102729,3581.0,9.0,3,0.8054144824310264
-270,True,CACAGATTCGTAGAGG-1,CACAGATTCGTAGAGG-1,1.9704229461530953,15.0,-0.05080116182891875,-0.026927952515174564,-4.3699207,-3.3322155,17,G1,1594.0050113648176,2,2,7750.0,2523.0,11.161290322580646,20.361290322580643,0.9570453162069913,False,False,4.3,-15.56773,-4.5900908,2082.0,9.0,2,1.0532599957420288
-272,True,CAGCAGCAGCTCTGTA-1,CAGCAGCAGCTCTGTA-1,1.9041296265439265,15.0,-0.09331506362679048,-0.008525065331510281,5.530037,8.750011,27,G1,1074.45114479959,8,1,7050.0,1868.0,9.21985815602837,41.98581560283688,0.617227765200647,False,False,2.9999999999999996,12.094755,11.459143,1082.0,9.0,8,0.8054144824310264
-273,True,CAGCAGCAGGCTCAAG-1,CAGCAGCAGGCTCAAG-1,1.8957838652224246,15.0,-0.09483975432631953,-0.011754775011588291,8.277005,8.820984,27,G1,1137.3976351469755,8,10,4582.0,1364.0,5.216062854648625,45.1113051069402,0.6170648577776312,False,False,5.300000000000001,14.075966,11.035069,1682.0,9.0,8,0.9446843091192515
-274,True,CAGCAGCAGGTTGCCC-1,CAGCAGCAGGTTGCCC-1,1.8935367023713838,15.0,-0.1032021445427114,-0.014970336586120484,7.4085307,8.899925,15,G1,1075.2252583503723,8,10,8329.0,2111.0,5.97910913675111,42.886300876455756,0.6181604676216871,False,False,3.4999999999999996,14.334815,11.316506,1864.0,9.0,8,0.9558057699680218
-275,True,CAGCAGCAGTCATCCA-1,CAGCAGCAGTCATCCA-1,2.072274772597774,15.0,-0.06417259968063777,-0.021043355766754472,-5.4121737,1.8840361,4,G1,1510.6264656484127,4,4,7450.0,2329.0,4.87248322147651,14.885906040268456,0.9430660231426132,False,False,4.7,-14.232456,1.0581014,1508.0,9.0,4,1.1296894359186769
-276,True,CAGCAGCCACTTCAGA-1,CAGCAGCCACTTCAGA-1,1.8454301616095674,15.0,-0.06400566788695715,-0.01830058918378252,9.411259,-12.613887,21,G1,1174.005507990718,7,6,4954.0,1998.0,7.246669358094469,16.4715381509891,0.02473113873654927,False,False,4.9,11.068652,-11.770066,1928.0,9.0,7,0.8054144824310264
-277,True,CAGCAGCCAGGGAATC-1,CAGCAGCCAGGGAATC-1,1.9643646667892538,15.0,-0.09424605911132186,-0.01275678373867263,0.71400756,18.093674,18,G1,1233.3037815615535,5,5,6161.0,1798.0,9.203051452686251,37.73738029540659,0.7248604763817115,False,False,6.999999999999999,-3.1797683,14.928721,1292.0,9.0,5,0.9949148082922232
-278,True,CAGCAGCCATCCTTCG-1,CAGCAGCCATCCTTCG-1,1.8965761332749582,15.0,-0.05928563429183141,-0.02003089588702408,-3.5908296,-12.916389,24,G1,1336.944579616189,10,7,13322.0,3500.0,8.024320672571687,14.975228944602913,0.9594495181822962,False,False,7.699999999999999,-7.3358603,-11.754679,2063.0,9.0,10,1.5971664117153936
-279,True,CAGCAGCCATGAAGGC-1,CAGCAGCCATGAAGGC-1,1.9773807084516883,15.0,-0.076204586491683,-0.023704974820893447,-5.8436213,-6.9199953,19,G1,1262.0959270745516,2,2,12835.0,3591.0,11.554343591741333,12.458122321776393,0.9605003297729222,False,False,3.5999999999999996,-15.267097,-7.049223,3452.0,9.0,2,1.3619548679656055
-280,True,CAGCAGCCATGGCTAT-1,CAGCAGCCATGGCTAT-1,1.9441397009563925,15.0,-0.09201889140150872,-0.007170106728072034,6.8749213,0.9597962,22,G1,1250.126637890935,6,8,5816.0,1652.0,16.007565337001374,35.14442916093535,0.5815675208758194,False,False,3.6,13.705687,5.19386,11101.0,9.0,6,1.1114423203132549
-281,True,CAGCAGCGTAACTTCG-1,CAGCAGCGTAACTTCG-1,1.9806747537682547,15.0,-0.07609799531464159,-0.012452881275423062,-5.6983137,-8.902907,3,G1,1088.1675548553467,2,2,9842.0,3111.0,12.121520016256857,13.04612883560252,0.9538352937924222,False,False,3.0000000000000004,-15.2246275,-8.665625,2453.0,9.0,2,0.8054144824310264
-282,True,CAGCAGCGTCCCGGTA-1,CAGCAGCGTCCCGGTA-1,1.9423026879724503,15.0,-0.13482675327219729,0.011121327107799502,0.7916824,12.804564,2,S,931.3062853962183,5,5,8443.0,1795.0,5.614118204429705,51.28508823877769,0.7242427935823973,False,False,6.0,-0.43756908,16.642206,1933.0,9.0,5,0.9267070380396117
-283,True,CAGCAGCGTCTAGGTT-1,CAGCAGCGTCTAGGTT-1,1.927168968087493,15.0,-0.081771351342875,-0.01874244640139112,8.133891,-2.9813418,22,G1,1206.0587933510542,6,8,4297.0,1440.0,13.939958110309519,29.346055387479637,0.4483497738408061,False,False,8.0,14.494095,0.5356392,797.0,9.0,6,1.4808010897496513
-284,True,CAGCAGCGTTACACAC-1,CAGCAGCGTTACACAC-1,1.9933168150936433,15.0,-0.07354621550077928,-0.024670686003292447,-12.163645,-5.6390243,7,G1,1127.6028670668602,3,3,10899.0,3290.0,10.945958344802275,19.130195430773465,0.9866743050630565,False,False,3.0,-18.30229,-5.217289,3344.0,9.0,3,1.1068635329128147
-285,True,CAGCAGCGTTAGGAGC-1,CAGCAGCGTTAGGAGC-1,1.9614114783098342,15.0,-0.08590065499104034,-0.018690333814772885,-4.679022,-4.8975286,7,G1,1455.666338995099,3,2,10253.0,3069.0,9.382619721057251,20.257485613966644,0.9639772603702849,False,False,3.0000000000000004,-13.507075,-3.893383,3385.0,9.0,3,1.0536968663288746
-286,True,CAGCAGCGTTCCGTTC-1,CAGCAGCGTTCCGTTC-1,1.8868101806629753,15.0,-0.06614894689721659,0.001800733246639475,0.6283982,6.515158,5,S,1733.8930397331715,1,1,4647.0,1472.0,7.617817947062621,38.99289864428663,0.6172750011358452,False,False,7.800000000000001,9.575956,11.817798,1410.0,9.0,1,1.7325161573380234
-288,True,CAGCAGCTCCGTATGA-1,CAGCAGCTCCGTATGA-1,2.005260149478001,15.0,-0.06572555767546823,-0.020850894637302404,-4.183463,-6.409666,19,G1,1308.2072361707687,2,2,7215.0,2461.0,14.636174636174637,14.594594594594595,0.9558262532214236,False,False,2.8000000000000003,-15.511817,-6.504503,1733.0,9.0,2,1.3352325228116744
-289,True,CAGCAGCTCGCCACTT-1,CAGCAGCTCGCCACTT-1,1.9762233244970147,15.0,-0.07864669622313997,-0.02712270646952304,-8.091279,0.44261587,14,G1,1290.582909077406,4,4,5279.0,2074.0,9.376775904527372,12.710740670581549,0.9524410896749995,False,False,3.0,-13.279324,-0.44757164,1676.0,9.0,4,0.8054144824310264
-290,True,CAGCGTGAGACCAGAC-1,CAGCGTGAGACCAGAC-1,1.902265976409908,15.0,-0.05953971100141961,-0.011441873822386466,11.453809,-8.923208,11,G1,1224.022267267108,7,6,4279.0,1543.0,11.614863285814442,23.930824959102594,0.14103469852712305,True,True,6.7,12.806714,-6.968602,1031.0,9.0,7,1.5775854309189588
-291,True,CAGCGTGAGATCGCCC-1,CAGCGTGAGATCGCCC-1,1.9575179707264967,15.0,-0.10398170452890014,-0.0272639450252243,-9.376102,-8.196126,10,G1,1201.007884055376,3,7,9435.0,2993.0,8.48966613672496,17.62586115527292,0.9691332643703283,False,False,4.5,-12.143355,-8.286798,4712.0,9.0,3,1.2880316103699065
-293,True,CAGCGTGAGGTTGGAC-1,CAGCGTGAGGTTGGAC-1,1.989439819868378,15.0,-0.0748291746597491,-0.02027836820707675,-8.823982,-7.309765,10,G1,1298.15692999959,3,7,9979.0,2823.0,10.241507165046597,16.484617697164044,0.9721469326833069,False,True,3.4000000000000004,-13.497178,-6.6546316,2570.0,9.0,3,1.3600510211168857
-294,True,CAGCGTGCACACCGCA-1,CAGCGTGCACACCGCA-1,1.922676370551629,15.0,-0.08577007694082489,-0.027942945251438287,-3.6864176,-12.391164,24,G1,1307.993331104517,10,7,12801.0,3513.0,9.358643855948754,17.38145457386142,0.9595455706767301,False,False,5.999999999999999,-7.8238006,-11.267099,3263.0,9.0,10,0.960975136721149
-296,True,CAGCGTGCATCCTATT-1,CAGCGTGCATCCTATT-1,1.951849503647278,15.0,-0.09066779002598309,-0.017390136006736452,-6.196989,-3.9396236,13,G1,1026.6741499304771,2,2,9903.0,3017.0,9.532464909623346,17.39876805008583,0.9554242747719779,False,False,3.4,-16.678116,-5.279139,5092.0,9.0,2,1.1209096610163056
-297,True,CAGCGTGCATGGCTAT-1,CAGCGTGCATGGCTAT-1,1.892774074268755,15.0,-0.09850368165593097,-0.014555628445107105,8.893151,4.6832576,8,G1,1143.2166037932038,9,9,5533.0,1808.0,6.000361467558287,35.85758178203506,0.6278893379130539,False,False,2.7999999999999994,11.622207,5.7896385,922.0,9.0,9,0.9577368584765362
-298,True,CAGCGTGGTAACTTCG-1,CAGCGTGGTAACTTCG-1,1.9977278688987814,15.0,-0.06451288872262749,-0.026235365520615003,-5.3351364,-3.4496443,9,G1,1687.5481162667274,2,2,8952.0,2942.0,7.138069705093834,14.622430741733691,0.9600106439296674,False,False,5.5,-15.958025,-3.952847,2138.0,9.0,2,1.4147295055549005
-299,True,CAGCGTGGTATAGGAT-1,CAGCGTGGTATAGGAT-1,1.9126620669578083,15.0,-0.11613802464099122,-0.0006447460591793436,4.610061,5.0987163,12,G1,1785.9470813572407,1,1,3657.0,1214.0,6.808859721082855,45.720535958435875,0.6109482903198683,False,False,3.6999999999999993,11.068561,8.518081,1004.0,9.0,1,1.4923312286500765
-300,True,CAGCGTGGTCCCGGTA-1,CAGCGTGGTCCCGGTA-1,2.03503668905526,15.0,-0.07391438206166984,-0.022177277319707055,-4.7095237,1.4924757,14,G1,1520.8275038152933,4,4,8249.0,2542.0,8.158564674506001,15.504909686022549,0.9457719391270027,False,False,5.3,-15.187228,1.0397762,3315.0,9.0,4,1.8467524754042035
-302,True,CAGCGTGGTCTTCATT-1,CAGCGTGGTCTTCATT-1,1.9378528759246565,15.0,-0.08606832575932068,-0.0220839790089908,-9.19775,-9.1499,10,G1,1082.4515704661608,3,7,12722.0,3389.0,7.506681339412042,20.853639364879736,0.9633561871043795,False,False,3.5,-12.401261,-9.858042,3063.0,9.0,3,1.2567936061319265
-303,True,CAGCGTGGTTAAGACA-1,CAGCGTGGTTAAGACA-1,2.014017201127487,15.0,-0.05613298556990644,-0.022134534567274797,-6.3368793,-0.31831452,20,G1,1345.7171936035156,4,4,9821.0,3108.0,7.881071174014866,15.711231035536096,0.9628520860093824,False,False,2.9999999999999996,-16.871943,-0.18761344,1993.0,9.0,4,0.8054144824310264
-304,True,CAGCGTGTCATCGCTC-1,CAGCGTGTCATCGCTC-1,1.9952659635229375,15.0,-0.08690162934515255,-0.029227433505201517,-6.6559978,-6.100761,3,G1,1501.5781627446413,2,2,6294.0,2405.0,13.806800127105179,13.918017159199238,0.9619277487970058,False,False,4.1,-14.033578,-5.929706,3818.0,9.0,2,1.6086021212745887
-305,True,CAGCGTGTCATGCCCT-1,CAGCGTGTCATGCCCT-1,1.8619634010403365,15.0,-0.09089002966320059,-0.040935151299475,-2.813361,13.258792,2,G1,1078.6742936745286,5,5,10753.0,3528.0,4.268576211289872,19.027248209801915,0.7172978787920485,False,False,4.1,2.3110642,13.378363,1025.0,9.0,5,0.940005096626502
-306,True,CAGCGTGTCCCGTTGT-1,CAGCGTGTCCCGTTGT-1,1.9514984517065972,15.0,-0.09573396297560784,-0.02120014336307013,-10.043522,-2.9829772,10,G1,1483.6279841512442,3,3,14949.0,3630.0,9.385243160077597,31.139206635895377,0.9878772700083635,False,False,4.0,-17.208666,-3.6935847,3503.0,9.0,3,1.0221414249343657
-307,True,CAGCGTGTCCGCTTAC-1,CAGCGTGTCCGCTTAC-1,1.9112246080232664,15.0,-0.11820340664587489,-0.009856909470963772,5.586106,6.5548515,12,G1,1275.0759824216366,1,1,6543.0,1575.0,5.639614855570839,50.16047684548372,0.6126790384106436,False,False,4.6000000000000005,9.451321,7.795113,1270.0,9.0,1,1.3337054561921877
-308,True,CAGCGTGTCGCCTTGT-1,CAGCGTGTCGCCTTGT-1,1.9220335536937052,15.0,-0.10558230889789288,-0.006182985005825939,0.06741738,7.2858453,5,G1,1575.2278412282467,1,1,4778.0,1467.0,7.262452909167015,42.50732524068648,0.6219886162432513,False,False,6.4,9.18971,12.592957,896.0,9.0,1,1.4486019550069733
-309,True,CAGCGTGTCGTCGGGT-1,CAGCGTGTCGTCGGGT-1,1.9426612801465168,15.0,-0.09233395466434968,-0.02811671006448634,-6.829375,-11.165282,3,G1,939.3805213272572,2,2,13914.0,3831.0,9.386229696708352,23.472761247664224,0.9400135920156864,False,False,2.9000000000000004,-15.631174,-10.014842,775.0,9.0,2,0.8054144824310264
-310,True,CAGCGTGTCTAGTCAG-1,CAGCGTGTCTAGTCAG-1,2.0051052040743786,15.0,-0.07092550118734878,-0.024939706140948467,-4.237031,-2.1202898,19,G1,1528.693746805191,2,2,9925.0,2842.0,13.914357682619647,18.811083123425693,0.9572126128390144,False,False,2.8,-14.317675,-2.1408563,2098.0,9.0,2,1.040309407416091
-311,True,CAGCGTGTCTGCCCTA-1,CAGCGTGTCTGCCCTA-1,1.911065684907062,15.0,-0.11789446957105151,-0.008927707456882056,7.943603,10.745313,15,G1,1204.3262564241886,8,10,7224.0,1670.0,6.589147286821706,49.18327796234773,0.6219049513748113,False,False,4.199999999999999,13.94672,10.561295,1066.0,9.0,8,0.9750606631853395
-312,True,CAGGCCAAGATCACCT-1,CAGGCCAAGATCACCT-1,1.8811423824383284,15.0,-0.09675253411856541,-0.016698010619568195,1.7498485,2.791531,12,G1,1452.9915368556976,1,1,7063.0,1863.0,3.9784793996885175,46.45334843550899,0.6107499234706271,False,False,3.3000000000000003,9.990693,8.528086,1937.0,9.0,1,1.3277792842586493
-313,True,CAGGCCAAGGTACAGC-1,CAGGCCAAGGTACAGC-1,2.005267164425277,15.0,-0.05823907393061884,-0.03150587522969367,-6.54021,-1.1740099,4,G1,1600.7668046802282,4,2,10244.0,3196.0,9.263959390862944,9.966809839906286,0.9669574089611214,False,False,2.6,-15.41965,-1.9626967,1895.0,9.0,4,1.4879203446320601
-314,True,CAGGCCAAGTGGCCTC-1,CAGGCCAAGTGGCCTC-1,1.8953359367369411,15.0,-0.11851701929787144,-0.015495060141080975,4.716559,8.064844,27,G1,1320.572511434555,8,1,6576.0,1763.0,5.428832116788321,44.95133819951338,0.6136000585475705,False,False,2.9000000000000004,10.216022,9.458859,1438.0,9.0,8,1.1884528478630074
-315,True,CAGGCCACAAGAGAGA-1,CAGGCCACAAGAGAGA-1,1.8657808204188817,15.0,-0.07057706028518514,-0.020719613018814247,10.879068,-7.935388,11,G1,1234.894474685192,7,6,4987.0,1903.0,5.694806496891919,22.05734910767997,0.14506607725757495,False,False,6.9,13.357323,-7.399864,1510.0,9.0,7,1.5646005659517708
-316,True,CAGGCCACACATGAAA-1,CAGGCCACACATGAAA-1,1.8920698811488235,15.0,-0.09906502698505831,-0.01432935114547684,4.238044,7.274736,12,G1,1253.5317683145404,1,1,7482.0,1930.0,7.805399625768511,43.58460304731355,0.6124788164865407,False,False,4.0,9.091192,8.634979,847.0,9.0,1,1.3291322357508284
-317,True,CAGGCCACATATCTGG-1,CAGGCCACATATCTGG-1,1.9210098054712659,15.0,-0.10327225517098934,-0.012701853060707672,6.263997,4.153552,12,G1,1591.1747448891401,1,9,6952.0,1728.0,6.947640966628309,48.69102416570771,0.6106789253906755,False,False,3.6999999999999997,11.4604645,7.2857785,1604.0,9.0,1,1.5244458930159492
-318,True,CAGGCCAGTACTGAGG-1,CAGGCCAGTACTGAGG-1,1.845746308970741,15.0,-0.06858948890852842,-0.020546391187058824,9.691778,-13.867936,21,G1,1147.7593779861927,7,6,4273.0,1803.0,9.735548794757781,12.941727123800609,0.00379941574657688,False,False,7.3999999999999995,9.880914,-11.997215,1443.0,9.0,7,1.6092122864742475
-319,True,CAGGCCAGTCACGCTG-1,CAGGCCAGTCACGCTG-1,1.9302089866905756,15.0,-0.05125052003304121,-0.025258912393140726,7.4866066,-7.107141,11,G1,948.3033172041178,7,6,12179.0,3937.0,7.283028163231792,12.094589046719763,0.4036574471293984,False,False,3.6000000000000005,14.8980255,-6.969359,3068.0,9.0,7,1.1208407825868383
-320,True,CAGGCCAGTCGGTGTC-1,CAGGCCAGTCGGTGTC-1,1.9309372709956314,15.0,-0.09891338355748591,-0.023404686498884154,-5.556116,-11.733703,25,G1,1131.9383014887571,6,7,12253.0,3482.0,8.389782094181017,15.02489186321717,0.9615941712576579,False,True,2.1000000000000005,-9.389994,-10.947856,3529.0,9.0,6,0.8054144824310264
-321,True,CAGGCCAGTTAAGTCC-1,CAGGCCAGTTAAGTCC-1,1.9096363724609307,15.0,-0.1078625317013818,-0.008665797694832575,7.501019,5.5559645,27,G1,1431.6936453133821,8,10,8504.0,2065.0,8.3137347130762,42.4035747883349,0.6064092113141658,False,False,3.7,13.371239,8.513163,2002.0,9.0,8,1.3225285162333158
-322,True,CATACAGAGAAATTGC-1,CATACAGAGAAATTGC-1,1.8937425165336728,15.0,-0.09812440428399043,-0.024606299212598427,7.143212,9.289116,15,G1,1401.8904049396515,8,10,8128.0,2313.0,5.167322834645669,41.22785433070866,0.6181134282600291,False,False,2.7000000000000006,14.296753,9.592731,961.0,9.0,8,0.9393306927058855
-323,True,CATACAGAGCAGGTCA-1,CATACAGAGCAGGTCA-1,1.933137208453172,15.0,-0.07702682734358052,-0.0015669925895179342,2.4695525,7.5136595,5,G1,1489.3200511932373,1,1,3983.0,1186.0,12.553351744915892,42.229475269897065,0.6127720778852997,False,False,3.7999999999999994,8.25029,11.642545,986.0,9.0,1,1.0102850184689025
-324,True,CATACAGAGCGACATG-1,CATACAGAGCGACATG-1,1.888774982085284,15.0,-0.07747857006367286,-0.025796163406969656,12.031397,-8.980037,11,G1,1198.9395132660866,7,6,2484.0,918.0,6.884057971014493,30.95813204508857,0.13526571132841314,False,False,6.499999999999999,12.723535,-7.5385222,985.0,9.0,7,0.9962154886322245
-325,True,CATACAGAGCTTAAGA-1,CATACAGAGCTTAAGA-1,1.8975941600817567,15.0,-0.10971987881362615,-0.00020798144104976907,6.5040245,4.1625996,8,G1,1520.7485368549824,9,1,8447.0,2147.0,5.789037528116491,42.39374926009234,0.6089036031296702,False,False,2.9000000000000004,12.57322,7.748602,1876.0,10.0,9,1.1982591168648236
-326,True,CATACAGAGGTTACAA-1,CATACAGAGGTTACAA-1,1.8813890824811015,15.0,-0.10194224968979472,-0.01668604452402199,7.716195,6.6074505,27,G1,1543.5538897812366,8,10,5788.0,1605.0,3.1962681409813407,46.45818935729095,0.6112718993951893,False,False,4.1,13.524274,9.127428,1957.0,9.0,8,1.237002859343705
-327,True,CATACAGCAAGAGAGA-1,CATACAGCAAGAGAGA-1,2.055712856022889,15.0,-0.08439762082682306,-0.019729103390157827,-6.140167,3.2012618,14,G1,1418.0869150608778,4,4,6320.0,2219.0,9.746835443037975,14.794303797468354,0.9329942909921487,False,False,6.100000000000001,-13.928124,2.0932927,3268.0,9.0,4,1.7527888243976026
-328,True,CATACAGCACCAGCGT-1,CATACAGCACCAGCGT-1,1.920380542338816,15.0,-0.1171352115077595,-0.00694441476083987,6.562728,3.236687,8,G1,1497.458046272397,9,9,7463.0,1859.0,8.522042074232882,44.15114565188262,0.6098702594880665,False,False,3.9000000000000004,12.360581,6.562899,964.0,9.0,9,1.5678310544261775
-330,True,CATACAGCAGCTGCCA-1,CATACAGCAGCTGCCA-1,1.9741548829208306,15.0,-0.06321792851389152,-0.010616690339016754,-8.669186,-0.65960777,7,G1,1581.8901606947184,3,3,7895.0,2818.0,8.803039898670045,12.260924635845472,0.9742986893517108,False,False,4.199999999999999,-15.230895,-2.5533588,3090.0,9.0,3,1.1385112939403332
-331,True,CATACAGCATATCTGG-1,CATACAGCATATCTGG-1,1.960414884617253,15.0,-0.10450853757498785,-0.019925577966295883,1.1727306,16.09752,2,G1,1099.15285089612,5,5,5847.0,1764.0,11.065503677099366,37.55772190867111,0.7191933505331903,False,True,9.5,-2.4869375,14.272452,1013.0,9.0,5,1.3598537505011798
-332,True,CATACAGGTCACTCGG-1,CATACAGGTCACTCGG-1,1.9888506562836559,15.0,-0.057456271119061826,-0.027679626327337836,-5.7840734,2.8080409,14,G1,1459.046968266368,4,4,4032.0,1783.0,7.192460317460317,10.66468253968254,0.9396685559347789,False,False,5.7,-14.941632,2.0607872,2474.0,9.0,4,1.3937894669484336
-333,True,CATACAGGTGTCTAAC-1,CATACAGGTGTCTAAC-1,1.8999020713830626,15.0,-0.11597629978128718,-0.009389648946489342,6.5228305,0.58743936,22,G1,1278.6056822538376,6,8,9296.0,2187.0,7.669965576592083,42.10413080895009,0.5763361859966477,False,False,4.3999999999999995,13.538184,5.8749685,2214.0,9.0,6,1.1721280544238497
-334,True,CATACAGGTTGGACCC-1,CATACAGGTTGGACCC-1,1.9939380892502347,15.0,-0.0778389144805403,-0.011263797547426308,-11.732965,-4.9690776,1,G1,1006.5759285539389,3,3,8603.0,2817.0,8.671393699872137,15.715448099500174,0.9911677128853777,False,False,3.6000000000000005,-18.748152,-5.123764,2309.0,9.0,3,1.0292877503014077
-335,True,CATACAGGTTGGGATG-1,CATACAGGTTGGGATG-1,2.0158538887071504,15.0,-0.0653749145908374,-0.020320012824726657,-3.8350995,-4.4519796,3,G1,1574.6938680708408,2,2,9119.0,2949.0,14.277881346638885,11.02094527908762,0.9571298945422917,False,False,3.5,-15.401074,-4.7322435,2011.0,9.0,2,1.2521096874567261
-337,True,CCCTCTCAGACGTCCC-1,CCCTCTCAGACGTCCC-1,1.9650613238965864,15.0,-0.08677111002692398,-0.028949691085613415,-6.724183,-10.43392,3,G1,963.5958538353443,2,2,8250.0,2664.0,8.4,18.8,0.9482690413867253,False,False,2.8,-15.596768,-9.6329365,3747.0,9.0,2,1.1647220741717665
-338,True,CCCTCTCAGCAGGTCA-1,CCCTCTCAGCAGGTCA-1,2.0727647988638918,15.0,-0.06519335245248425,-0.02680771821094998,-4.99498,4.0468736,14,G1,1355.4826240837574,4,4,7514.0,2347.0,9.901517167953154,14.559488953952622,0.9134535085992458,False,False,6.7,-13.468655,2.5225534,1774.0,9.0,4,1.282823347179086
-339,True,CCCTCTCAGGTCTACT-1,CCCTCTCAGGTCTACT-1,1.8789012904058646,15.0,-0.09957808110434618,-0.00921749936833019,3.788646,6.412645,12,G1,1684.116705223918,1,1,3837.0,1318.0,6.385196768308575,36.87776909043524,0.6121939119350903,False,False,3.8000000000000003,9.478539,9.137184,1156.0,9.0,1,1.0180081103546588
-340,True,CCCTCTCAGTCATCCA-1,CCCTCTCAGTCATCCA-1,1.9540887572581178,15.0,-0.07753709340731804,-0.022227524200471874,-6.854687,-7.053111,3,G1,1432.4177133589983,2,2,11109.0,3671.0,8.794670987487622,8.88468809073724,0.9597970431774242,False,False,3.6999999999999997,-15.808394,-6.734242,2277.0,10.0,2,1.66271512327643
-341,True,CCCTCTCCAAATGCGG-1,CCCTCTCCAAATGCGG-1,1.9916698723749264,15.0,-0.07193492331417284,-0.029654503549457034,-10.929289,-0.8748561,7,G1,1456.9272047281265,3,3,14633.0,3780.0,7.223399166268024,17.282853823549512,0.9938070510641033,False,False,4.799999999999999,-17.777576,-1.7393465,2720.0,9.0,3,1.3198912675453556
-342,True,CCCTCTCCACACCGCA-1,CCCTCTCCACACCGCA-1,1.895991602905573,15.0,-0.10983177527136297,-0.013013711488243589,5.3682623,3.131261,12,G1,1547.5188335478306,1,9,5812.0,1663.0,5.6951135581555405,41.41431520991053,0.611858401463065,False,False,3.7,11.946625,7.704663,1086.0,9.0,1,1.0097827324533222
-343,True,CCCTCTCCACAGAGAC-1,CCCTCTCCACAGAGAC-1,1.9459408824495976,15.0,-0.11883757560003565,-0.022099715125589958,-1.7543505,13.569383,2,G1,1141.1378885358572,5,5,6414.0,1680.0,8.481446835048331,41.98628001247272,0.7336333754896308,False,False,6.2,1.0240138,15.433463,1143.0,9.0,5,1.0750721142572695
-344,True,CCCTCTCCATTATGCG-1,CCCTCTCCATTATGCG-1,1.8632198530128166,15.0,-0.0820160166634955,0.0006393675329864276,9.748156,-6.4506683,23,S,1159.1340185254812,6,8,3977.0,1673.0,6.462157405079205,22.831279859190346,0.22839643503859605,False,False,5.800000000000001,14.124119,-5.0394874,1328.0,9.0,6,1.1919946538905841
-345,True,CCCTCTCGTACGGGAT-1,CCCTCTCGTACGGGAT-1,2.0487020078335343,15.0,-0.06140721209687111,-0.02636932951525803,-7.163423,3.3625908,14,G1,1403.782047227025,4,4,4178.0,1726.0,14.576352321685016,10.172331258975586,0.9415186649904357,False,False,5.1,-13.907275,1.833342,1590.0,9.0,4,1.1353175038090075
-346,True,CCCTCTCGTATACCTG-1,CCCTCTCGTATACCTG-1,1.8800198998827067,15.0,-0.06546841278065069,-0.026770819385812213,8.830024,-13.982556,21,G1,1083.0468273460865,7,6,5490.0,2287.0,12.313296903460838,10.418943533697632,0.0018762984288668262,False,False,7.8999999999999995,9.764536,-12.200286,1438.0,9.0,7,1.2260956028822898
-347,True,CCCTCTCGTCACTCGG-1,CCCTCTCGTCACTCGG-1,1.9035041147763876,15.0,-0.10452354600712949,-0.011091586979857827,7.801524,-1.9930842,22,G1,1083.1298670172691,6,8,6294.0,1859.0,8.102955195424213,37.639021290117576,0.4763128614103582,False,True,4.3999999999999995,14.436056,1.5302564,1204.0,9.0,6,1.1137790287560545
-348,True,CCCTCTCGTCTAGGTT-1,CCCTCTCGTCTAGGTT-1,1.9936208127115704,15.0,-0.07179089925946022,-0.02228598383635436,-5.979312,-2.2953367,17,G1,1536.17895257473,2,2,8466.0,3028.0,9.225135837467517,10.713442003307348,0.9614207277145181,False,False,2.8000000000000003,-15.343706,-2.7011223,2318.0,9.0,2,0.8054144824310264
-349,True,CCCTCTCGTGACGCCT-1,CCCTCTCGTGACGCCT-1,2.250942876960828,14.0,0.03210432397785415,-0.021817388458601504,-8.475535,-12.748832,16,G2M,720.7980148494244,2,2,534.0,256.0,22.846441947565545,4.119850187265918,0.9058916952231162,False,False,2.8000000000000003,-15.072312,-11.330073,963.0,9.0,2,0.8054144824310264
-350,True,CCCTCTCGTGCACAAG-1,CCCTCTCGTGCACAAG-1,1.878782848250622,15.0,-0.07834038157508054,-0.017564270309774054,9.318883,-12.701311,21,G1,1198.8450797349215,7,6,8723.0,3031.0,8.655279147082426,19.546027742749054,0.017439526960035093,False,False,5.1,10.951147,-11.4422245,1824.0,9.0,7,1.438770127615809
-351,True,CCCTCTCGTGCGGTAA-1,CCCTCTCGTGCGGTAA-1,1.9513802363668515,15.0,-0.08521237331828534,-0.03961122512999926,-10.3360615,-4.4438515,1,G1,1195.213123768568,3,3,7304.0,2535.0,7.8450164293537785,10.008214676889375,0.9862659104939607,False,False,3.4,-13.385398,-4.4662795,3395.0,9.0,3,1.0539205889133303
-352,True,CCCTCTCGTTCCACGG-1,CCCTCTCGTTCCACGG-1,1.8524357921032255,15.0,-0.0707747344177394,-0.00869661136451734,10.068282,-5.129393,23,G1,1142.5617601424456,6,8,5348.0,1969.0,6.020942408376963,25.523560209424083,0.3006275194622053,False,False,7.199999999999999,14.246238,-3.644408,1369.0,9.0,6,1.3308283858636578
-353,True,CCCTCTCGTTGGACCC-1,CCCTCTCGTTGGACCC-1,1.925535229860219,15.0,-0.04278015105758644,-0.0023465772801228857,0.66477543,7.7751274,5,G1,1629.3074842989445,1,1,2049.0,877.0,15.617374328940947,19.960956564177646,0.6201020830043741,False,False,6.4,9.281414,12.343484,1799.0,9.0,1,1.6335883792588684
-354,True,CCCTCTCTCCCAAGCG-1,CCCTCTCTCCCAAGCG-1,1.8829287457467798,15.0,-0.09685613933496715,-0.017014119937965273,1.1897258,4.18516,12,G1,1714.21916821599,1,1,5190.0,1622.0,6.377649325626204,37.39884393063584,0.6125050375925161,False,False,4.5,10.176049,10.429145,2161.0,9.0,1,1.0502393374153849
-355,True,CCCTCTCTCCGCACGA-1,CCCTCTCTCCGCACGA-1,1.9065998780812494,15.0,-0.07727762053326268,-0.020660977063113203,-3.0104413,-12.65278,24,G1,1322.6958612799644,10,7,10828.0,2798.0,8.330254894717399,21.001108237901736,0.9592728554700117,False,False,7.3,-6.539811,-11.624563,3020.0,9.0,10,0.9758455406552953
-356,True,CCCTCTCTCGAACGCC-1,CCCTCTCTCGAACGCC-1,1.9166645223951313,15.0,-0.10481363001813558,-0.0031597127915674664,4.0462704,3.5157738,12,G1,1858.6506461501122,1,1,4887.0,1456.0,7.612031921424187,42.09126253325148,0.608022887381211,False,False,5.800000000000001,11.739289,9.2189665,1459.0,9.0,1,1.302400829963068
-357,True,CCCTCTCTCTCCGAGG-1,CCCTCTCTCTCCGAGG-1,1.9033908828207595,15.0,-0.08222446710415256,-0.00648670306096745,5.1122775,1.6823483,12,G1,1631.7853568941355,1,8,4034.0,1252.0,10.436291522062469,38.795240456122954,0.591153983761569,False,False,3.1999999999999997,12.626924,6.7854333,811.0,9.0,1,1.0874029793228943
-358,True,CCCTCTCTCTCTAGGA-1,CCCTCTCTCTCTAGGA-1,1.9028863695431333,15.0,-0.09770354478510528,-0.007140147121334773,5.7871876,6.718869,27,G1,1550.562692336738,8,1,9434.0,2356.0,6.667373330506678,44.42442230231079,0.6124064567429208,False,False,3.9,11.92023,9.726447,788.0,9.0,8,1.2166313146772585
-359,True,CGGCAGTAGAGAGCGG-1,CGGCAGTAGAGAGCGG-1,1.9189803764903852,15.0,-0.0784759164468842,-0.015197015770302465,10.020934,-8.294735,11,G1,1207.5650903135538,7,6,4183.0,1452.0,12.048768826201291,30.289266076978244,0.14968138158519695,False,False,6.5,13.454297,-7.468745,1261.0,9.0,7,1.026932343957817
-360,True,CGGCAGTCACACCGCA-1,CGGCAGTCACACCGCA-1,1.91861939466199,15.0,-0.08618632591090962,-0.009278795979878313,-2.9263196,-13.675326,24,G1,1295.115891739726,10,7,14564.0,3572.0,5.8019774787146385,17.865970887118923,0.9593405261657042,False,False,6.8,-7.4710293,-12.578688,3003.0,9.0,10,0.9403405532365913
-361,True,CGGCAGTCACCAGCGT-1,CGGCAGTCACCAGCGT-1,1.953151411849151,15.0,-0.08565734879984924,-0.0266721496939675,-8.200189,-9.872453,10,G1,995.4691574722528,3,7,8587.0,2873.0,11.342727378595551,14.708279958076162,0.9587158119403458,False,False,3.6,-13.263959,-10.044881,2524.0,9.0,3,0.8054144824310264
-362,True,CGGCAGTCAGTGTGCC-1,CGGCAGTCAGTGTGCC-1,1.9193662858855807,15.0,-0.09321135289020749,-0.011265385281897343,-2.616632,-11.732062,24,G1,1269.4341300725937,10,7,10506.0,2515.0,7.766990291262136,25.785265562535695,0.9596313292574161,False,False,6.4,-7.1672173,-11.947493,2040.0,9.0,10,1.0015219091544367
-364,True,CGGCAGTGTAGAATGT-1,CGGCAGTGTAGAATGT-1,1.9514264651818216,15.0,-0.08176782883259245,-0.022586705633852473,-8.533034,-4.816835,10,G1,1122.3878692090511,3,7,9258.0,3069.0,8.425145819831497,13.728667098725426,0.9713576694284423,False,False,3.5999999999999996,-13.489421,-6.1341186,3208.0,9.0,3,1.2264156125468055
-365,True,CGGCAGTGTAGGTTTC-1,CGGCAGTGTAGGTTTC-1,1.9080313332936878,15.0,-0.11161794103824668,-0.008990922748479999,4.553734,0.99636346,22,G1,1385.6881998628378,6,1,10459.0,2333.0,6.367721579500908,46.96433693469739,0.6002382226582099,False,False,3.4000000000000004,13.426334,7.512413,1512.0,9.0,6,0.9904197242749064
-366,True,CGGCAGTGTATAGGAT-1,CGGCAGTGTATAGGAT-1,1.9269289403375431,15.0,-0.071968103287653,-0.022021385476749714,-2.2821681,-13.615139,24,G1,1283.0497619509697,10,7,10094.0,2580.0,10.768773528829007,21.983356449375865,0.959278819710406,False,False,7.2,-6.744525,-11.4052,1734.0,9.0,10,0.8054144824310264
-367,True,CGGCAGTGTATGCGTT-1,CGGCAGTGTATGCGTT-1,1.9104318405110468,15.0,-0.1266591064961649,-0.012929130369766702,2.7881358,7.0473595,5,G1,1695.3633325397968,1,1,8632.0,1982.0,7.599629286376274,48.58665430954588,0.6122969479783753,False,False,3.5999999999999996,9.068408,10.808477,919.0,9.0,1,1.112152250731469
-368,True,CGGCAGTGTCCACACG-1,CGGCAGTGTCCACACG-1,1.941869323756232,15.0,-0.08978439722216792,-0.01607590487942517,-2.08065,-13.0179405,24,G1,1287.983282983303,10,7,11259.0,2848.0,9.59232613908873,22.9061195488054,0.9593924149137347,False,False,6.8999999999999995,-6.5958343,-12.141531,2934.0,10.0,10,0.9788750406701542
-369,True,CGGCAGTGTCGAATGG-1,CGGCAGTGTCGAATGG-1,1.91850348856739,15.0,-0.11757049017796165,-0.01524024293440331,2.04788,5.8373904,12,G1,1442.5792770534754,1,1,5063.0,1374.0,5.8660873000197515,48.92356310487853,0.6322546940490276,False,True,2.6999999999999997,11.942179,11.090333,1558.0,9.0,1,1.0399810749184109
-370,True,CGGCAGTGTGCCTTCT-1,CGGCAGTGTGCCTTCT-1,1.882804736068339,15.0,-0.11011395776279331,-0.004462429786456331,11.505131,-9.56628,11,G1,1249.715460985899,7,6,5636.0,1762.0,8.481192334989354,35.87650816181689,0.1171481816461192,False,False,5.699999999999999,12.66915,-8.075925,1960.0,9.0,7,1.0690030998731175
-372,True,CGGCAGTGTTGGAGGT-1,CGGCAGTGTTGGAGGT-1,1.9614205628243575,15.0,-0.07241067824685067,-0.026341848121577267,-7.840459,-3.9509761,10,G1,1370.053646236658,3,3,12089.0,3469.0,5.782115973198776,18.810488874183143,0.966293900942943,False,False,3.1999999999999997,-13.442972,-4.2361517,1904.0,9.0,3,1.8507823626714903
-374,True,CGGCAGTTCCGGACGT-1,CGGCAGTTCCGGACGT-1,1.959377445367422,15.0,-0.09586438028343827,-0.0025033617645135627,-1.3511472,16.022533,2,G1,1241.5621903464198,5,5,4617.0,1461.0,9.075157028373402,38.05501407840589,0.7317128039812371,True,True,7.6000000000000005,0.05219317,14.3996935,1190.0,9.0,5,1.6723068237584109
-376,True,CGTCCATAGTGTTGAA-1,CGTCCATAGTGTTGAA-1,1.9551381239704457,15.0,-0.07168697944855575,-0.018093196109716485,-9.749792,-1.4847265,7,G1,1611.5776236951351,3,3,8445.0,2917.0,7.34162226169331,11.095322676139727,0.979703315802304,False,False,4.800000000000001,-15.030734,-1.3498517,3338.0,9.0,3,1.1077349154430558
-377,True,CGTCCATCACGCGCAT-1,CGTCCATCACGCGCAT-1,1.897839474835368,15.0,-0.09967999113264568,-0.02567779431363437,10.080686,-4.849407,23,G1,1145.8119096755981,6,8,5369.0,1715.0,10.29986962190352,30.974110635127584,0.31663482625370276,False,False,7.199999999999999,14.228688,-3.1542192,2116.0,9.0,6,1.1834856120867876
-378,True,CGTCCATCACTACACA-1,CGTCCATCACTACACA-1,1.9848601002011872,15.0,-0.09237949349557881,-0.0035592240437654563,-0.07276739,17.431839,18,G1,1263.0185985416174,5,5,13444.0,3134.0,12.845879202618269,34.58048199940494,0.7278801626185175,False,False,5.1,-1.8403897,14.943713,981.0,9.0,5,0.9949148082922232
-379,True,CGTCCATCAGCAGTGA-1,CGTCCATCAGCAGTGA-1,2.0148173549837534,15.0,-0.060994026555763414,-0.0379385941725445,-5.2425203,0.57739353,20,G1,1581.4556798487902,4,4,10363.0,3166.0,7.266235646048441,13.01746598475345,0.9520007705325964,False,False,4.5,-15.222205,-0.08154499,1964.0,9.0,4,1.8882570788227317
-380,True,CGTCCATCAGGGAATC-1,CGTCCATCAGGGAATC-1,1.9400373078917716,15.0,-0.08939361012202382,-0.013584944432013473,-4.3653855,-13.656964,24,G1,1135.3985050618649,10,7,14957.0,3640.0,8.892157518218895,20.45864812462392,0.9600429195778493,True,True,4.8999999999999995,-8.228996,-12.017874,3088.0,9.0,10,1.0690719696785353
-381,True,CGTCCATCATATCTGG-1,CGTCCATCATATCTGG-1,1.8738944654810383,15.0,-0.1256616595215034,0.005305883618873346,7.735076,-2.4962428,22,S,1159.927320331335,6,8,4043.0,1179.0,5.1694286420974525,47.58842443729903,0.46251057498825315,False,False,6.6000000000000005,13.938397,0.116523124,896.0,9.0,6,1.0088873921243666
-383,True,CGTCCATGTCAAGCGA-1,CGTCCATGTCAAGCGA-1,1.9604554209452123,15.0,-0.10939808854586157,-0.005156510169734916,-4.0254354,3.0926068,14,G1,1235.2945231050253,4,4,3667.0,973.0,0.436323970548132,39.29642759749114,0.8732016156226156,False,False,4.5,-13.154397,2.3658345,1195.0,9.0,4,1.2492640120368883
-384,True,CGTCCATGTGCACAAG-1,CGTCCATGTGCACAAG-1,1.9638201520581517,15.0,-0.0834095996571717,-0.01841503737227947,-5.208086,-5.389476,13,G1,1562.7849843651056,2,2,9746.0,3194.0,8.629181202544634,19.19761953621999,0.9563291590313003,False,False,3.4999999999999996,-16.275427,-5.350124,2809.0,9.0,2,1.4381038299180309
-385,True,CGTCCATGTTAGGAGC-1,CGTCCATGTTAGGAGC-1,1.968595662266925,15.0,-0.07768042787538369,-0.014567289483195842,7.766992,-7.5480494,11,G1,923.4268907010555,7,6,11219.0,3281.0,10.170246902575988,15.545057491755058,0.3366313829837774,False,False,3.5000000000000004,14.793212,-7.4846296,3174.0,9.0,7,1.2201809404287483
-386,True,CGTCCATGTTCGGCTG-1,CGTCCATGTTCGGCTG-1,1.8974652750913374,15.0,-0.10751379422265497,-0.0064373789333848335,8.290299,4.7089496,26,G1,1392.7411314621568,9,9,6320.0,1735.0,5.965189873417722,43.18037974683544,0.6113259043998333,False,False,4.3999999999999995,11.012638,6.1099777,1461.0,9.0,9,1.4851538553532957
-387,True,CGTCCATTCTGCCCTA-1,CGTCCATTCTGCCCTA-1,2.0557147611182756,15.0,-0.08016922090770714,-0.032040677634204366,-6.8965273,3.938574,14,G1,1426.5106876343489,4,4,3362.0,1622.0,12.671029149315883,8.149910767400357,0.9398242550489379,False,False,5.8999999999999995,-13.651529,2.621331,1777.0,9.0,4,1.1353175038090075
-388,True,GAGGGTAAGACCAGAC-1,GAGGGTAAGACCAGAC-1,1.961650454423746,15.0,-0.09057114498523394,-0.018522757706469654,-8.651427,-1.7656965,7,G1,1608.3845952004194,3,3,9270.0,3170.0,10.35598705501618,13.829557713052859,0.9843163147244981,False,False,5.2,-15.911959,-1.9679083,3383.0,9.0,3,1.4878660176210219
-389,True,GAGGGTAAGCACCTGC-1,GAGGGTAAGCACCTGC-1,1.8986477362807002,15.0,-0.07663836868420623,-0.01777441273712556,4.0701923,5.6515675,12,G1,1781.244106695056,1,1,2484.0,939.0,12.077294685990339,29.871175523349436,0.6114210926081723,False,False,5.2,10.004462,8.897074,1245.0,9.0,1,1.4120562125104608
-390,True,GAGGGTAAGCGACATG-1,GAGGGTAAGCGACATG-1,1.9163577732868302,15.0,-0.11475349280132699,0.0007243675014130188,5.050214,8.841273,27,S,1051.6663328632712,8,1,6893.0,1835.0,8.356303496300594,45.32134049035253,0.6292875876114878,False,False,3.3000000000000003,11.489351,11.575405,659.0,9.0,8,0.9411293383160607
-391,True,GAGGGTAAGGCCTAGA-1,GAGGGTAAGGCCTAGA-1,1.990867429296621,15.0,-0.07021314267921142,-0.028225240193827626,-9.756679,0.1068885,14,G1,1469.558833822608,4,3,7325.0,2538.0,11.890784982935154,13.924914675767917,0.982692616376982,False,False,3.9000000000000004,-17.020735,-1.1734837,1049.0,9.0,4,1.2603903856899734
-392,True,GAGGGTAAGGGAGGTG-1,GAGGGTAAGGGAGGTG-1,2.0732674181043933,15.0,-0.06703346250589129,-0.025798240559993805,-4.4784493,2.9041028,14,G1,1424.530461370945,4,4,3387.0,1439.0,13.404192500738116,12.370829642751698,0.9264838684732848,False,False,5.799999999999999,-14.152505,2.172565,1554.0,9.0,4,1.1702966058509467
-393,True,GAGGGTAAGGTCTACT-1,GAGGGTAAGGTCTACT-1,1.9823189307958076,15.0,-0.06490851249615567,-0.027769919090017622,-5.814096,-2.1105,17,G1,1264.9400390088558,2,2,12060.0,3366.0,6.865671641791045,16.558872305140962,0.9653027370680748,False,False,4.1,-17.103138,-2.7483957,3334.0,9.0,2,1.0984708838805604
-395,True,GAGGGTAAGTGTTGAA-1,GAGGGTAAGTGTTGAA-1,1.987437486753611,15.0,-0.09709189018285534,-0.037129601901632676,-8.714432,-5.3014216,3,G1,1330.4915774613619,2,2,10407.0,3056.0,10.2527145190737,14.394157778418371,0.9774933477979837,False,False,2.5,-16.38343,-5.1587996,2354.0,9.0,2,1.1941258902583858
-396,True,GAGGGTACAAATGCGG-1,GAGGGTACAAATGCGG-1,1.9245385070443923,15.0,-0.11499893922686662,-0.0010491908031318685,8.327663,9.585501,15,G1,1103.3345563858747,8,10,5464.0,1456.0,9.132503660322108,46.92532942898975,0.6207294734795556,False,False,5.299999999999999,15.300105,9.9095125,990.0,9.0,8,0.9562600999765033
-397,True,GAGGGTACAGGTTCAT-1,GAGGGTACAGGTTCAT-1,1.9629175723863408,15.0,-0.09853542184795729,-0.018727753158252806,-9.179546,-9.686361,10,G1,1123.5468605011702,3,7,9840.0,3102.0,5.813008130081301,13.729674796747968,0.9647556179414448,False,False,3.8999999999999995,-11.644603,-9.733569,1813.0,9.0,3,1.0246186277677674
-398,True,GAGGGTAGTAGTGTGG-1,GAGGGTAGTAGTGTGG-1,1.9058988445789706,15.0,-0.10856950260136691,-0.03075753357360396,-2.1277275,11.959253,2,G1,877.4017400145531,5,10,13032.0,3557.0,6.645181092694905,27.892879066912215,0.6632566775498758,False,False,3.4,3.8058274,13.140235,1321.0,9.0,5,0.8054144824310264
-399,True,GAGGGTAGTTAGGAGC-1,GAGGGTAGTTAGGAGC-1,1.8933691371161094,15.0,-0.11271625356750414,0.00490816098070174,7.0387516,6.3973045,27,S,1403.34445861727,8,10,7950.0,1978.0,4.50314465408805,46.77987421383648,0.6120070955716002,False,False,3.1,12.7371025,9.14054,1180.0,9.0,8,1.2452746909952663
-400,True,GAGGGTAGTTCGAAGG-1,GAGGGTAGTTCGAAGG-1,1.898319146576067,15.0,-0.09131883972891351,-0.007515213013949716,0.8755439,5.854018,5,G1,1738.541418850422,1,1,3931.0,1311.0,13.96591198168405,35.43627575680488,0.6138376051784263,False,False,7.1,9.105708,11.2875,970.0,9.0,1,1.2501517667997706
-401,True,GAGGGTAGTTCGGCTG-1,GAGGGTAGTTCGGCTG-1,1.9848565863362932,15.0,-0.07146221461776396,-0.025524540378327947,-6.6536975,-3.356771,10,G1,1482.3660599440336,3,3,9185.0,2907.0,9.461077844311378,12.923244420250409,0.9699474686308385,False,False,3.8000000000000003,-15.409359,-3.8414571,4520.0,9.0,3,1.1082909943197194
-402,True,GAGGGTAGTTGGACCC-1,GAGGGTAGTTGGACCC-1,1.9296106587218,15.0,-0.07105703094917791,-0.006625854381588168,6.868426,6.020367,8,G1,1367.8595857471228,9,1,2484.0,931.0,20.531400966183575,16.70692431561997,0.61463879256387,False,False,3.6,11.247502,7.6633477,1145.0,9.0,9,1.1236781032398344
-403,True,GAGGGTATCAGGGATG-1,GAGGGTATCAGGGATG-1,1.9629355320280748,15.0,-0.07350796750894377,-0.02769243867690122,-8.162191,-5.9455304,10,G1,1458.1405840069056,3,3,9481.0,3085.0,11.275181942833035,15.937137432760258,0.9690401899612058,False,False,4.3,-12.933802,-6.197296,2810.0,9.0,3,1.2974956605861356
-404,True,GAGGGTATCCGGACGT-1,GAGGGTATCCGGACGT-1,1.8898340910420204,15.0,-0.11660011022389004,-0.016778144530808867,7.796214,7.3390613,27,G1,1252.5737234055996,8,10,6018.0,1548.0,3.6556995679627784,51.41242937853107,0.6111643043872729,True,True,4.3,13.84853,9.450892,916.0,9.0,8,1.3781356827645914
-405,True,GAGGGTATCGAACGCC-1,GAGGGTATCGAACGCC-1,1.9508317339695467,15.0,-0.08935355801733412,-0.03022650797262577,-9.965252,-9.140249,10,G1,1163.1111931204796,3,7,12331.0,3685.0,9.147676587462493,13.064633849647231,0.9694202839666316,False,False,5.6000000000000005,-11.540826,-9.053819,4456.0,9.0,3,0.997428114413575
-406,True,GAGGGTATCGTTCCTG-1,GAGGGTATCGTTCCTG-1,1.9644084838018074,15.0,-0.0967348858100616,-0.0020386241074561544,0.43639222,15.343253,2,G1,1111.9533635675907,5,5,6622.0,1706.0,11.3711869525823,39.972817879794626,0.7187888891670337,False,False,7.4,-1.3966873,14.906741,750.0,9.0,5,1.072372799549762
-407,True,GAGGGTATCTCCGAGG-1,GAGGGTATCTCCGAGG-1,1.875858905758202,15.0,-0.10640138014538868,-0.02340394824606911,5.405017,8.068325,27,G1,1233.4610465466976,8,1,4978.0,1430.0,5.04218561671354,46.082764162314184,0.6144507283104254,False,False,3.6999999999999997,11.001662,11.630738,1341.0,9.0,8,0.8054144824310264
-409,True,GAGTTGTAGCCTCTTC-1,GAGTTGTAGCCTCTTC-1,1.8773614782874604,15.0,-0.10355691354429412,0.008183198519617577,10.644986,-5.539434,23,S,1123.8978370130062,6,8,4644.0,1559.0,6.2446167097329885,31.93367786391042,0.2659509262947879,False,False,7.1,14.032845,-4.4738197,1490.0,9.0,6,0.9492563449557461
-411,True,GAGTTGTCAAGTCGTT-1,GAGTTGTCAAGTCGTT-1,2.028373827987996,15.0,-0.09164821624446946,-0.0286453428308876,-9.145013,-6.962429,10,G1,1288.5011351555586,3,7,5184.0,2052.0,13.194444444444445,10.185185185185185,0.9738773330789481,False,False,3.5000000000000004,-13.448934,-7.1246886,2824.0,9.0,3,1.3786711877855171
-412,True,GAGTTGTCACTAAACC-1,GAGTTGTCACTAAACC-1,2.0154160160939285,15.0,-0.07948892051760836,-0.016113737678474408,0.922361,14.672169,2,G1,1095.8173262178898,5,5,2740.0,986.0,16.532846715328468,28.868613138686133,0.7193508806257586,False,False,7.8999999999999995,-1.2680078,15.483536,740.0,9.0,5,0.9638800466103177
-414,True,GAGTTGTCATGAATAG-1,GAGTTGTCATGAATAG-1,1.9100172876826065,15.0,-0.07600377193149989,-0.004251723874867004,9.746234,2.331165,25,G1,1147.4085248559713,6,9,5339.0,1709.0,8.203783480052444,32.10339014796779,0.5980212558898053,False,True,4.5,12.633586,4.377047,1097.0,9.0,6,1.1051376087517883
-415,True,GAGTTGTGTCGTGCCA-1,GAGTTGTGTCGTGCCA-1,1.9125066123734022,15.0,-0.09826782597365227,0.020372985589480138,8.943073,-10.758092,21,S,1282.264548331499,7,6,3295.0,1206.0,12.200303490136571,25.91805766312595,0.06510220068375193,False,False,7.1,11.97742,-10.226901,610.0,9.0,7,1.0599277571389403
-416,True,GAGTTGTGTCTAGGTT-1,GAGTTGTGTCTAGGTT-1,1.967556475422572,15.0,-0.07718401855791882,-0.031144757074560894,-11.43003,-4.152301,1,G1,1129.0501042604446,3,3,8197.0,2763.0,11.101622544833475,15.042088568988655,0.9966516297245044,False,False,3.7999999999999994,-18.00638,-4.1720934,1851.0,9.0,3,1.4881230776743295
-417,True,GAGTTGTGTGCCTTCT-1,GAGTTGTGTGCCTTCT-1,1.9677195338591296,15.0,-0.10566230511047225,-0.02343660398621728,1.8988804,18.920614,18,G1,1217.0519406348467,5,5,3894.0,1214.0,8.988186954288649,40.67796610169491,0.7234115696821902,False,False,9.3,-4.1586204,15.429235,763.0,9.0,5,1.0823755123601224
-418,True,GAGTTGTGTTCCACGG-1,GAGTTGTGTTCCACGG-1,1.9132403727830813,15.0,-0.10102529482374442,-0.004359421835149991,7.302568,6.375039,27,G1,1369.2804308831692,8,10,4995.0,1329.0,8.308308308308309,46.106106106106104,0.610111596381938,False,True,4.2,13.669567,8.868969,1196.0,9.0,8,0.9890392966214314
-419,True,GAGTTGTGTTCGAAGG-1,GAGTTGTGTTCGAAGG-1,2.0386602431705247,15.0,-0.05961259423863303,-0.023619161781378774,-9.844534,0.7698294,14,G1,1511.0530198812485,4,3,5054.0,2104.0,8.686189157103284,10.645033636723387,0.9800896687828496,False,False,4.1,-16.90356,-0.52645314,1937.0,9.0,4,1.4542779131353019
-420,True,GAGTTGTTCCGCTTAC-1,GAGTTGTTCCGCTTAC-1,1.915573832293061,15.0,-0.12527298225866573,-0.015351458169805448,1.7728759,5.2887044,5,G1,1709.9839492291212,1,1,6764.0,1604.0,8.32347723240686,48.97989355410999,0.6118866943028299,False,False,5.0,9.357945,10.472148,766.0,9.0,1,1.0031359453897972
-421,True,GCGTTTCAGATACCAA-1,GCGTTTCAGATACCAA-1,1.9935422477078797,15.0,-0.07048484157529494,-0.030364865849914845,-5.154694,-1.1153615,20,G1,1318.4492756575346,4,4,10262.0,3047.0,9.033326836873904,20.82440070161762,0.9611667235667967,False,False,3.3999999999999995,-16.523664,-0.1643319,1572.0,9.0,4,1.2626360339929414
-422,True,GCGTTTCAGCTCACTA-1,GCGTTTCAGCTCACTA-1,1.9451207673588513,15.0,-0.07359374538849026,-0.02143062591452305,-9.243341,-8.483466,10,G1,1059.0613911151886,3,7,9395.0,2989.0,7.993613624268228,13.507184672698244,0.9678401698107225,False,False,4.4,-12.7232065,-9.0594015,1899.0,9.0,3,1.0544564466971402
-423,True,GCGTTTCAGCTCTGTA-1,GCGTTTCAGCTCTGTA-1,1.8885949606387378,15.0,-0.09092886999863745,-0.024147839681820264,10.927044,-8.351941,11,G1,1233.2414600104094,7,6,4884.0,1664.0,7.186732186732187,28.74692874692875,0.1349628976429639,False,False,6.700000000000001,13.1919985,-7.434416,1179.0,9.0,7,1.5330612562039885
-424,True,GCGTTTCAGCTTGTTG-1,GCGTTTCAGCTTGTTG-1,2.0242732011430613,15.0,-0.07358062730616917,-0.030891240220429692,-9.330983,0.078763604,7,G1,1574.6309480667114,3,3,10590.0,3104.0,8.90462700661001,15.55240793201133,0.9796309612741497,False,False,3.8999999999999995,-16.248022,-1.3389395,1253.0,9.0,3,1.0793149764938728
-425,True,GCGTTTCAGGAACTCG-1,GCGTTTCAGGAACTCG-1,2.0017909000755822,15.0,-0.07444851807912907,-0.015773218114720135,-2.5234683,-4.344702,14,G1,1137.6243809461594,4,2,13348.0,3690.0,11.342523224453101,21.988312855858556,0.9206999030063117,False,False,1.6,-14.203999,-2.2558563,1924.0,9.0,4,1.0733894727636921
-426,True,GCGTTTCAGGATCACG-1,GCGTTTCAGGATCACG-1,1.967709755304794,15.0,-0.0919551651354266,-0.023417228755453354,-6.0315757,-9.812909,3,G1,1010.7253363877535,2,2,11666.0,3417.0,11.100634321961255,16.87810732041831,0.9511336335320776,False,False,3.0000000000000004,-15.330246,-9.283644,3214.0,9.0,2,1.0982061776526664
-427,True,GCGTTTCAGGTACAGC-1,GCGTTTCAGGTACAGC-1,1.8754535604837101,15.0,-0.09080379540837237,-0.003860186503558627,7.579875,9.429963,15,G1,1223.3936507850885,8,10,5641.0,1448.0,3.846835667434852,49.03385924481475,0.61855496032972,False,False,4.300000000000001,14.348682,10.16381,934.0,9.0,8,1.6166475945876415
-428,True,GCGTTTCAGGTTGCCC-1,GCGTTTCAGGTTGCCC-1,1.9499029409800286,15.0,-0.08448712077026158,-0.022439205193524662,-7.4234653,-6.185965,13,G1,1174.9671227037907,2,2,9597.0,3310.0,7.929561321246223,12.691466083150985,0.9604204617739387,False,False,3.4,-16.938093,-6.5025816,2978.0,9.0,2,0.8054144824310264
-429,True,GCGTTTCCACCGTGAC-1,GCGTTTCCACCGTGAC-1,2.081643073576501,15.0,-0.03609347463168062,-0.028820001080750043,-2.4651778,-5.6973424,19,G1,1286.2833405286074,2,2,2156.0,1020.0,27.040816326530614,4.962894248608534,0.9535516721691608,False,False,3.4999999999999996,-15.453573,-5.3314967,1249.0,9.0,2,0.8054144824310264
-430,True,GCGTTTCCAGCTGCCA-1,GCGTTTCCAGCTGCCA-1,1.9466911368327433,15.0,-0.12395255413645995,0.006007376929910137,-2.2944522,15.464206,2,S,1232.1421486735344,5,5,4302.0,1339.0,6.206415620641562,41.95722919572292,0.743193423594481,False,False,6.8999999999999995,-0.01046834,13.6125765,1294.0,9.0,5,0.8054144824310264
-431,True,GCGTTTCCATGGCTAT-1,GCGTTTCCATGGCTAT-1,1.9287115479219177,15.0,-0.09740269308868213,-0.02314803178512972,-10.3707695,-8.06818,10,G1,1157.9712169617414,3,7,15175.0,3987.0,6.9851729818780885,18.082372322899506,0.9740795629935742,False,False,4.0,-12.121913,-8.435523,2697.0,9.0,3,1.174952204829321
-433,True,GCGTTTCGTAGCGTAG-1,GCGTTTCGTAGCGTAG-1,1.9520738389292305,15.0,-0.09680622860382827,-0.0195520644068004,-6.0812874,-8.873951,25,G1,1082.416327252984,6,2,13225.0,3406.0,7.6521739130434785,17.83742911153119,0.9496429268315969,False,True,2.6,-14.478318,-8.829839,3362.0,9.0,6,1.0379934821247077
-434,True,GCGTTTCGTAGTGTGG-1,GCGTTTCGTAGTGTGG-1,1.8848002923439118,15.0,-0.09624085052171281,-0.00034100745024551876,10.313253,4.767892,26,G1,1036.6052235662937,9,9,5423.0,1564.0,4.33339479992624,45.4176654988014,0.6133969323817307,False,True,5.6,11.092388,4.607266,1174.0,9.0,9,1.115336184313886
-435,True,GCGTTTCGTATGCGTT-1,GCGTTTCGTATGCGTT-1,1.8908286834053516,15.0,-0.11207805738856084,-0.012879267882239948,1.4292498,3.4994023,12,G1,1632.7694101035595,1,1,7007.0,1801.0,5.794205794205794,45.46881689738833,0.611889558333002,False,False,4.199999999999999,10.062279,9.412356,1513.0,9.0,1,1.4750446060631406
-436,True,GCGTTTCGTCAAGCCC-1,GCGTTTCGTCAAGCCC-1,1.9793244788433022,15.0,-0.08567365803445987,-0.021554174450831583,-3.1899264,-1.9526285,19,G1,1395.3430116772652,2,2,11707.0,3289.0,10.344238489792431,15.905014094131715,0.9404946987332147,False,False,4.6,-13.329576,-2.34084,2294.0,9.0,2,1.6978102219916276
-437,True,GCGTTTCGTCGAATGG-1,GCGTTTCGTCGAATGG-1,1.9783355780166196,15.0,-0.08531803459645494,-0.030593654032190852,-8.064227,-7.8188386,10,G1,1308.6128604263067,3,2,9203.0,2953.0,11.887428012604586,12.593719439313267,0.9646991267972062,False,False,2.2,-14.2399435,-7.539477,2103.0,9.0,3,0.8054144824310264
-438,True,GCGTTTCGTCTAGGTT-1,GCGTTTCGTCTAGGTT-1,2.0039633883954897,15.0,-0.05874877798209569,-0.02572278559513492,-5.056116,-3.1967916,17,G1,1757.1379400640726,2,2,15829.0,4120.0,9.097226609387832,13.28574136079348,0.9650562207040994,False,False,4.7,-15.416959,-3.5250683,3852.0,9.0,2,1.802129890940226
-439,True,GCGTTTCGTGACAGGT-1,GCGTTTCGTGACAGGT-1,2.023416206185012,15.0,-0.06602568606362606,-0.029331534332560937,-5.504479,1.3208313,20,G1,1466.0097438246012,4,4,4965.0,2010.0,9.365558912386707,9.32527693856999,0.9522775733085661,False,False,5.7,-15.767346,0.952871,1718.0,9.0,4,1.290021396948959
-440,True,GCGTTTCGTTCGAAGG-1,GCGTTTCGTTCGAAGG-1,1.9752644734464588,15.0,-0.1363490248236485,-0.016630579698463037,1.5513514,15.252966,2,G1,1099.15285089612,5,5,4990.0,1392.0,9.39879759519038,42.28456913827655,0.7195369930497734,False,False,9.5,-2.099129,14.7687645,1387.0,9.0,5,1.4011142273600008
-441,True,GCGTTTCTCATTTGGG-1,GCGTTTCTCATTTGGG-1,1.886492977321364,15.0,-0.08472741832345376,0.005045431454759875,7.9674616,-2.7226562,22,S,1187.39734955132,6,8,3482.0,1397.0,8.271108558299828,26.3641585295807,0.46863271789780175,False,False,7.6000000000000005,14.983665,0.7728801,962.0,9.0,6,1.0169260766413897
-442,True,GCGTTTCTCCCAAGCG-1,GCGTTTCTCCCAAGCG-1,1.9015581455662387,15.0,-0.10609862603335811,-0.0028105496237303892,-2.0484564,14.427796,2,G1,1222.2123991549015,5,5,12699.0,3208.0,6.047720292936452,35.750846523348294,0.7326081582877823,False,False,6.999999999999998,0.71638346,14.672622,2130.0,9.0,5,1.076455644882097
-443,True,GCGTTTCTCCCGTTGT-1,GCGTTTCTCCCGTTGT-1,1.9506298066663137,15.0,-0.07136089107287204,-0.02116898394788601,-10.2033615,-6.3424973,10,G1,1139.548169568181,3,7,9854.0,3085.0,10.47290440430282,11.87335092348285,0.9777478723750416,False,False,3.1000000000000005,-12.08442,-7.3487835,2026.0,9.0,3,1.0611395816907838
-444,True,GCGTTTCTCTAGTCAG-1,GCGTTTCTCTAGTCAG-1,1.9964784082581721,15.0,-0.055700906815747374,-0.03759711496674763,-3.9054937,-2.2442648,19,G1,1543.690078958869,2,2,10767.0,3324.0,9.129748305006038,16.095476920219188,0.9532809659095467,False,False,2.6,-14.816743,-2.501882,1227.0,9.0,2,1.3051359729982455
-445,True,GCGTTTCTCTGCCCTA-1,GCGTTTCTCTGCCCTA-1,1.9656316325190635,15.0,-0.06069256120459873,-0.015190145644023312,-3.9933107,-3.5019958,13,G1,1099.6918728351593,2,2,9992.0,3250.0,7.7261809447558045,13.55084067253803,0.9573252455260762,False,False,2.9,-16.758545,-4.7086496,1533.0,9.0,2,1.1997971929725997
-446,True,GGTAACTAGATGCTTC-1,GGTAACTAGATGCTTC-1,1.8918720019597555,15.0,-0.09330599793876822,-0.0184775282050157,6.512094,5.2294383,12,G1,1636.7178554832935,1,9,5437.0,1542.0,4.027956593709766,45.42946477837042,0.611311013982174,False,False,3.8,10.914086,7.3337765,756.0,9.0,1,1.2396093642962331
-447,True,GGTAACTAGCCTCTGG-1,GGTAACTAGCCTCTGG-1,1.8887532887360579,15.0,-0.10111408230485952,-0.004433987388836122,2.8902223,1.7041141,12,G1,1200.6996416896582,1,1,4911.0,1342.0,4.846263490124211,46.97617593158216,0.6107310827294637,False,False,3.5,11.312934,8.372095,877.0,9.0,1,1.1652119799398735
-448,True,GGTAACTAGCTTAAGA-1,GGTAACTAGCTTAAGA-1,1.9113925680708361,15.0,-0.09511497961675365,-0.025200291393513645,-3.2451751,-14.225075,24,G1,1167.204173207283,10,7,12211.0,2947.0,8.754401768896896,23.70813201212022,0.9594379946591277,False,False,5.1,-7.8526325,-12.614308,2785.0,9.0,10,0.9400161588187672
-449,True,GGTAACTAGGGCGAAG-1,GGTAACTAGGGCGAAG-1,1.9699440082783706,15.0,-0.10878233422103392,-0.0013726262335715736,2.7056437,17.858658,18,G1,1063.50942184031,5,5,4547.0,1250.0,9.698702441170003,45.876402023312075,0.7218999254377241,False,False,5.700000000000001,-4.2892904,14.367057,649.0,9.0,5,1.2208709111950036
-450,True,GGTAACTAGTGCGTCC-1,GGTAACTAGTGCGTCC-1,1.9551212659862627,15.0,-0.09889392862430793,-0.01867137512225979,-9.528202,-9.777505,10,G1,1109.2821858525276,3,7,14708.0,3511.0,8.457982050584716,18.36415556159913,0.9687595297406814,False,False,5.2,-10.783125,-9.626292,4962.0,9.0,3,0.9702743922760952
-451,True,GGTAACTCACAAATCC-1,GGTAACTCACAAATCC-1,1.8759669373683747,15.0,-0.06802325170378722,-0.027163102640709903,5.75147,3.3351243,6,G1,1610.2430288791656,1,1,3627.0,1422.0,7.719878687620623,26.30272952853598,0.6032408273219441,True,True,4.700000000000001,12.920521,8.166678,1854.0,9.0,1,1.3799657090043111
-452,True,GGTAACTCAGACCTAT-1,GGTAACTCAGACCTAT-1,1.944626235055211,15.0,-0.09372083375991666,-0.023242899869066753,-7.742586,-9.708252,3,G1,1201.3187436759472,2,2,14031.0,3646.0,8.944480079823249,18.34509300833868,0.9535681952909236,False,False,4.1000000000000005,-13.738346,-9.0563,3166.0,9.0,2,1.0202116580598752
-453,True,GGTAACTCATCTCCCA-1,GGTAACTCATCTCCCA-1,2.033483917345862,15.0,-0.05485199446504899,-0.028545861247874088,-5.5598354,2.843624,14,G1,1399.1558936983347,4,4,4998.0,1969.0,13.505402160864346,10.504201680672269,0.9353594581749559,False,False,4.5,-15.4124,2.0603826,1233.0,9.0,4,1.0871745436712557
-454,True,GGTAACTGTCACGCTG-1,GGTAACTGTCACGCTG-1,1.9858006348937287,15.0,-0.08536124802156364,-0.02321540134089462,-7.772422,-1.0053236,9,G1,1360.5417627096176,2,2,10702.0,3188.0,9.091758549803775,16.65109325359746,0.9713656551610899,False,False,2.9,-15.644869,-1.7741287,1927.0,9.0,2,1.0390015412436735
-455,True,GGTAACTGTGACAGGT-1,GGTAACTGTGACAGGT-1,1.957304290672029,15.0,-0.11518228418070908,-0.007208030261447656,-1.4474683,14.716389,2,G1,1220.5084094703197,5,5,7530.0,1883.0,8.725099601593625,43.266932270916335,0.7371193816948719,False,False,7.799999999999999,0.42819965,13.944037,943.0,9.0,5,0.9538903929007907
-456,True,GGTAACTGTTACACAC-1,GGTAACTGTTACACAC-1,1.8857906178341237,15.0,-0.10188288408584617,-0.017670495192377116,6.6575985,4.4825907,12,G1,1251.1253600344062,1,9,9445.0,2382.0,5.897300158814187,43.23980942297512,0.6114860164741535,False,False,2.5000000000000004,10.531204,7.4353104,1691.0,9.0,1,0.9803736437095918
-457,True,GGTAACTGTTGGACCC-1,GGTAACTGTTGGACCC-1,1.9024341966254572,15.0,-0.11726949711445836,-0.013460392563490109,7.229759,10.083213,15,G1,1102.1117427647114,8,10,6993.0,1716.0,5.677105677105677,49.735449735449734,0.6190201929917489,False,False,4.4,14.629374,9.827219,1572.0,9.0,8,1.116711645410346
-458,True,GGTAACTTCCAATCCC-1,GGTAACTTCCAATCCC-1,1.99096553162365,15.0,-0.07060965099387748,-0.021836893934825553,-12.452853,-2.9943953,7,G1,1060.496095508337,3,3,9108.0,2988.0,9.541062801932368,13.790074659639878,0.999557244302952,False,False,4.7,-18.599064,-3.3331532,2449.0,9.0,3,1.134797514700434
-459,True,GGTAATCAGACGTCGA-1,GGTAATCAGACGTCGA-1,1.9408863585421237,15.0,-0.10409274709085438,-0.025884827615922645,-6.6967583,-8.433001,3,G1,1256.2316560298204,2,2,11513.0,3149.0,8.11256840093807,23.034830191956917,0.9499418506323816,False,False,5.0,-15.525059,-8.063196,2160.0,9.0,2,0.9503529382560777
-462,True,GGTAATCAGCTCTGTA-1,GGTAATCAGCTCTGTA-1,1.965317714071137,15.0,-0.08592764464281849,-0.025145371680025147,-8.6891775,-5.20088,13,G1,1294.8012250065804,2,3,12120.0,3504.0,8.556105610561056,15.825082508250825,0.9787106808347231,False,False,2.7000000000000006,-17.600815,-5.52944,3234.0,9.0,2,1.3323129786654517
-463,True,GGTAATCAGTCATCCA-1,GGTAATCAGTCATCCA-1,1.8952141853397164,15.0,-0.09300914755616117,-0.004979058477798833,1.5237654,4.588611,5,G1,1717.7054914981127,1,1,6509.0,1668.0,5.853433707174681,45.644492241511756,0.614626622547069,False,False,6.4,10.565291,11.511914,1246.0,9.0,1,0.8054144824310264
-464,True,GGTAATCAGTGCGTCC-1,GGTAATCAGTGCGTCC-1,1.9770971143775944,15.0,-0.0786451734747061,-0.023425388681303474,-10.664176,-1.3330064,7,G1,1573.1423325687647,3,3,7622.0,2759.0,8.121228024140645,13.185515612700078,0.9857140015934845,False,False,5.3,-16.834873,-2.2501268,3398.0,9.0,3,0.8054144824310264
-465,True,GGTAATCCAAGTAGTA-1,GGTAATCCAAGTAGTA-1,1.9326041014073199,15.0,-0.0775483355132815,-0.014841732257145607,-4.677714,-6.7852545,9,G1,1397.9825575202703,2,2,12974.0,3490.0,8.463080006166178,26.7303838446123,0.9449327314949466,False,False,2.4,-15.056797,-5.310502,2938.0,9.0,2,1.0123512893774396
-466,True,GGTAATCCACTTCAGA-1,GGTAATCCACTTCAGA-1,1.8828935913609999,15.0,-0.11616157770470616,-0.0029771863100975064,6.031983,3.5819085,6,G1,1624.7798773348331,1,1,7283.0,1870.0,3.7621859123987367,44.21254977344501,0.6069775310629637,False,False,4.5,12.706373,7.540646,743.0,9.0,1,1.7619741443073946
-467,True,GGTAATCGTATGAGCG-1,GGTAATCGTATGAGCG-1,1.9380398841722883,15.0,-0.09745939004839455,-0.0025259732473302335,0.72355986,6.9153223,5,G1,1656.4171461164951,1,1,4777.0,1248.0,9.776010048147374,47.079757169771824,0.6161805199712311,False,False,5.699999999999999,9.01557,11.998518,1159.0,9.0,1,1.2084786706331958
-468,True,GGTAATCGTCAAGCCC-1,GGTAATCGTCAAGCCC-1,1.9685775780255599,15.0,-0.08444370898767585,-0.015750375563740007,-2.285971,-0.46520284,14,G1,1134.3154958933592,4,4,10843.0,3073.0,10.98404500599465,23.1393525776999,0.8443196535411321,False,False,1.2,-12.340751,-1.3298858,4183.0,9.0,4,0.8054144824310264
-469,True,GGTAATCGTTCCGTTC-1,GGTAATCGTTCCGTTC-1,1.8857330027703727,15.0,-0.08848705846036635,0.008527493307551669,9.35844,-4.143062,23,S,1142.1055243462324,6,8,5942.0,1881.0,8.58296869740828,33.82699427802087,0.3696090746887515,False,False,7.1,14.362684,-2.0169702,1110.0,9.0,6,1.0050707685337341
-470,True,GGTAATCGTTCGGCTG-1,GGTAATCGTTCGGCTG-1,1.9011098495195558,15.0,-0.1102274807087149,-0.004595743901713335,7.9309607,8.98313,15,G1,1151.9602027982473,8,10,5050.0,1618.0,8.792079207920793,36.89108910891089,0.614236099290194,False,True,3.4000000000000004,14.814207,9.09316,1631.0,9.0,8,0.8054144824310264
-471,True,GGTAATCTCGTTCCTG-1,GGTAATCTCGTTCCTG-1,2.0074381757225765,15.0,-0.07160783159108604,-0.021307182482217968,-8.447385,-3.8579357,1,G1,1638.6017775088549,3,3,5444.0,2083.0,10.855988243938281,11.590742101396032,0.9794415867767325,False,True,3.0999999999999996,-16.25864,-4.5356255,1804.0,9.0,3,1.4347729404197185
-472,True,GGTAATCTCTGTGCGG-1,GGTAATCTCTGTGCGG-1,1.9822614779493652,15.0,-0.06902023507763573,-0.03355305170417033,-10.487968,-1.567702,7,G1,1599.094700023532,3,3,8970.0,2904.0,9.777034559643255,11.616499442586399,0.994213337549173,False,False,6.1,-17.235723,-2.6015809,1746.0,9.0,3,1.0861494330169923
-474,True,GTCAAGTAGATCGCCC-1,GTCAAGTAGATCGCCC-1,1.8860958970489563,15.0,-0.10069835224853681,-0.01753380282270008,10.865987,-9.377702,11,G1,1265.883071884513,7,6,3818.0,1450.0,8.486118386589837,29.36092194866422,0.11905398888001079,False,True,4.5,13.098978,-8.013861,1830.0,9.0,7,1.1589866241916746
-475,True,GTCAAGTAGATGCTTC-1,GTCAAGTAGATGCTTC-1,1.9905038578368874,15.0,-0.07783534140317036,-0.03123930066951593,-5.543269,-2.9932883,17,G1,1545.7968413829803,2,2,8002.0,2821.0,8.697825543614096,12.859285178705324,0.9658095944146652,False,True,3.6000000000000005,-16.012985,-3.5600214,2125.0,9.0,2,1.4691424885910218
-476,True,GTCAAGTAGCACCTGC-1,GTCAAGTAGCACCTGC-1,1.9624055661996864,15.0,-0.11022108785403112,-0.011743538550786092,1.856404,17.832495,18,G1,1080.1587940752506,5,5,3925.0,1210.0,8.815286624203821,41.65605095541401,0.7222010970037678,False,False,5.6,-4.436091,14.096253,954.0,9.0,5,1.2345429384636424
-477,True,GTCAAGTAGCAGCCCT-1,GTCAAGTAGCAGCCCT-1,1.9088212166387093,15.0,-0.11604828307173348,-0.010455415053863281,8.492172,9.163621,15,G1,1073.6456372737885,8,10,4780.0,1214.0,6.02510460251046,52.19665271966527,0.6185828505349793,False,False,5.4,14.44918,10.60913,1206.0,9.0,8,1.6163638299966483
-478,True,GTCAAGTAGCTCTGTA-1,GTCAAGTAGCTCTGTA-1,1.97917494630792,15.0,-0.07422637953570234,-0.023599039776816627,-7.651654,-8.341952,3,G1,1072.2969840168953,2,2,11247.0,3358.0,10.518360451676003,13.870365430781542,0.9593039965025305,False,True,2.8000000000000003,-14.225913,-8.402383,2737.0,9.0,2,0.9909435072959271
-479,True,GTCAAGTCAAACCATC-1,GTCAAGTCAAACCATC-1,2.04939635981785,15.0,-0.04240666293488034,-0.019039121246425105,-4.3301945,2.9644606,14,G1,1424.2926710993052,4,4,6855.0,2336.0,7.425237053245806,12.399708242159008,0.9252923667364973,False,False,5.4,-14.317543,1.6900156,1943.0,9.0,4,0.8054144824310264
-480,True,GTCAAGTCAAGAGAGA-1,GTCAAGTCAAGAGAGA-1,1.919275865534939,15.0,-0.11798687717667462,-0.004135494270163862,5.0957813,5.6288114,12,G1,1417.8654056414962,1,1,5735.0,1363.0,6.800348735832607,49.92153443766347,0.6122956471328568,False,False,3.8000000000000003,10.449599,7.40204,953.0,9.0,1,0.8054144824310264
-481,True,GTCAAGTCACACCGCA-1,GTCAAGTCACACCGCA-1,1.8850537697774505,15.0,-0.06765571117998781,-0.011254783574593906,5.6949844,8.3197365,27,G1,1145.0007433518767,8,1,7928.0,2207.0,7.341069626639758,40.03531786074672,0.6147984363891109,False,False,3.7,11.028669,10.825708,1945.0,9.0,8,1.2674650979792708
-482,True,GTCAAGTCACATATGC-1,GTCAAGTCACATATGC-1,1.8977190777700486,15.0,-0.1115759828288394,-0.009959368449102519,8.396104,3.1766622,8,G1,1441.383401542902,9,9,9688.0,2344.0,5.150701899256813,45.10734929810074,0.6046778460570154,False,False,4.3,12.33405,5.7299876,1363.0,9.0,9,1.3799404569406668
-483,True,GTCAAGTCACCGTGAC-1,GTCAAGTCACCGTGAC-1,1.964401215485108,15.0,-0.10967592202740256,0.013630768593817427,2.9664392,18.162455,18,S,1138.8892051428556,5,5,4355.0,1390.0,9.621125143513202,37.06084959816303,0.7228925540646978,False,False,7.899999999999999,-4.6785583,14.862218,1007.0,9.0,5,0.8054144824310264
-484,True,GTCAAGTGTAGCGTAG-1,GTCAAGTGTAGCGTAG-1,1.9139499800989541,15.0,-0.1257756163005199,-0.0102496757435221,3.0825953,3.6570723,12,G1,1668.5989705175161,1,1,5963.0,1451.0,8.049639443233271,48.51584772765386,0.6129318776376159,False,False,4.6,12.091324,10.181129,1527.0,10.0,1,1.3007900415296285
-486,True,GTCAAGTGTTAGGAGC-1,GTCAAGTGTTAGGAGC-1,1.8924152110949277,15.0,-0.12482950534273707,0.004481500966268292,4.391797,6.3245444,6,S,1582.544979646802,1,1,5220.0,1314.0,3.888888888888889,50.67049808429119,0.6133216662756666,False,False,4.0,11.274971,9.943995,1466.0,9.0,1,1.0001014312127572
-487,True,GTCAAGTGTTCCGTTC-1,GTCAAGTGTTCCGTTC-1,1.9069767154067787,15.0,-0.1052975754189846,-0.03128890701251808,-3.3508227,12.602101,2,G1,903.6031815707684,5,5,11537.0,3304.0,5.53870156886539,23.949033544248937,0.7087709872152613,False,False,3.8000000000000003,2.794634,13.691138,1068.0,9.0,5,1.134040899293891
-488,True,GTCAAGTTCAAGAAAC-1,GTCAAGTTCAAGAAAC-1,1.918895222138211,15.0,-0.08396854273225432,-0.006926246730530085,9.434865,3.3941958,8,G1,1236.7346387654543,9,9,7556.0,1952.0,8.33774483853891,42.53573319216517,0.6132792530800618,True,True,5.2,12.2731,5.1158533,902.0,9.0,9,0.8054144824310264
-489,True,GTCAAGTTCATGCCCT-1,GTCAAGTTCATGCCCT-1,1.897972502998224,15.0,-0.06448290394276891,-0.029420417769932334,3.7359707,2.811946,12,G1,1842.284537166357,1,1,2046.0,898.0,17.986314760508307,14.418377321603128,0.6122201402317655,False,False,6.1,12.005209,10.118835,954.0,9.0,1,1.398320174884873
-490,True,GTCAAGTTCTCCAATT-1,GTCAAGTTCTCCAATT-1,1.9500180174581772,15.0,-0.08085428073155704,-0.025439587224867298,-6.857682,-5.5567646,3,G1,1380.4451939761639,2,2,14326.0,3752.0,8.07622504537205,15.328772860533297,0.9662996002885605,False,False,3.1,-14.607156,-6.270057,2908.0,9.0,2,0.8054144824310264
-491,True,GTGAGCCAGATACCAA-1,GTGAGCCAGATACCAA-1,1.8864953418277608,15.0,-0.10688150428474202,-0.021090325731994403,7.9417014,0.3081697,22,G1,1145.4485174566507,6,8,5660.0,1843.0,6.07773851590106,36.64310954063604,0.5555179510166546,False,True,3.0,13.880096,3.928719,2654.0,9.0,6,0.9897589645476249
-493,True,GTGAGCCAGCCTCTGG-1,GTGAGCCAGCCTCTGG-1,2.0204990265130056,15.0,-0.0608487719228573,-0.027998946909922388,-3.11558,-2.6331613,19,G1,1558.7233407348394,2,2,5946.0,2241.0,15.943491422805247,10.99899091826438,0.9565500961035296,False,False,4.2,-13.559421,-3.2194192,2053.0,9.0,2,0.8054144824310264
-494,True,GTGAGCCAGCTTAAGA-1,GTGAGCCAGCTTAAGA-1,2.0626545595272177,15.0,-0.027137138960004872,-0.044183477704784604,-5.9080186,4.0868554,14,G1,1434.5245258659124,4,4,1670.0,918.0,12.694610778443113,11.676646706586826,0.9359884740289985,True,True,7.199999999999999,-14.530601,2.5754066,1570.0,9.0,4,1.6289097711356149
-495,True,GTGAGCCAGGCGTTAG-1,GTGAGCCAGGCGTTAG-1,1.9266850908724502,15.0,-0.09916891525734861,-0.020030571159376544,0.77162814,3.986483,5,G1,1261.3708607554436,1,1,5087.0,1515.0,12.305877727540791,39.76803617063102,0.6106950019701213,False,False,2.3000000000000003,9.002518,9.665691,2327.0,9.0,1,1.0888327821872623
-496,True,GTGAGCCAGTAATTGG-1,GTGAGCCAGTAATTGG-1,1.9786785267100906,15.0,-0.09231397536989219,-0.027625933913842642,-3.4729223,-2.5521114,19,G1,1296.7094856798649,2,2,7099.0,2368.0,7.719397098182843,12.818706860121145,0.9516894071636389,False,False,3.8,-13.104668,-3.0961804,1818.0,9.0,2,1.5037717087205094
-497,True,GTGAGCCAGTAGGAAG-1,GTGAGCCAGTAGGAAG-1,1.890445784398436,15.0,-0.11430747036121161,-0.011055639979457318,8.328998,2.7240727,8,G1,1287.1207217425108,9,9,7109.0,1882.0,4.2481361654241105,45.57603038402026,0.6044298993131737,False,False,2.9000000000000004,13.961166,6.5160375,1842.0,9.0,9,0.9808893821763591
-498,True,GTGAGCCCAAGAGTAT-1,GTGAGCCCAAGAGTAT-1,1.945627458555895,15.0,-0.11813746498517849,-0.011189775913954448,0.2500917,13.828904,2,G1,1061.5513131320477,5,5,5681.0,1544.0,8.537229361027988,44.023939447280405,0.7242434966343649,False,False,5.3,-0.6297741,15.750689,936.0,9.0,5,1.192661877989499
-499,True,GTGAGCCCACATGAAA-1,GTGAGCCCACATGAAA-1,1.9196008945223666,15.0,-0.10445829080042718,-0.014666280093300007,5.4838424,3.9967427,6,G1,1648.1316543370485,1,1,7080.0,1657.0,7.641242937853107,48.53107344632768,0.6109533780962834,False,False,4.0,12.212603,8.41571,730.0,9.0,1,1.426458888211635
-500,True,GTGAGCCGTATACCTG-1,GTGAGCCGTATACCTG-1,1.9644814176313266,15.0,-0.08252482170938098,-0.026433977582147362,-5.722054,-6.724826,13,G1,1288.336611866951,2,2,10555.0,3235.0,9.872098531501658,15.082899099952629,0.957426628017822,False,False,2.2,-16.362658,-6.6698437,5970.0,9.0,2,1.199813723524251
-501,True,GTGAGCCGTATAGGAT-1,GTGAGCCGTATAGGAT-1,2.1717708876313946,15.0,-0.05503991842504721,-0.02031723917744206,-8.440418,-12.124357,16,G1,684.5364937782288,2,2,669.0,302.0,12.406576980568012,3.7369207772795217,0.933438177065736,False,False,5.7,-15.858946,-10.6613865,736.0,9.0,2,0.9819739476362334
-502,True,GTGAGCCGTCTAGGTT-1,GTGAGCCGTCTAGGTT-1,1.9107601068615223,15.0,-0.10310638501222733,-0.01215203107053856,9.873085,4.8288255,26,G1,1018.3185731694102,9,9,4592.0,1255.0,8.362369337979095,45.55749128919861,0.6119398124069392,False,False,4.6,10.596211,4.9502316,889.0,9.0,9,1.239219353275973
-503,True,GTGAGCCGTCTTCATT-1,GTGAGCCGTCTTCATT-1,1.8942735911901045,15.0,-0.10409534957338576,-0.009337739650576762,6.560102,-0.15046738,22,G1,1021.5159673392773,6,8,6075.0,1757.0,9.82716049382716,38.91358024691358,0.5478307341893253,False,False,4.6,14.354403,3.5571644,874.0,9.0,6,1.262248996011977
-504,True,GTGAGCCGTTAAGACA-1,GTGAGCCGTTAAGACA-1,2.0044293905509303,15.0,-0.05317253293895029,-0.02665303465098171,-4.4044046,-0.82890046,20,G1,1124.474106490612,4,4,6810.0,2594.0,13.230543318649046,13.729809104258443,0.9613247874494422,False,False,5.6000000000000005,-16.811478,-0.8569928,2560.0,9.0,4,1.8367803788359192
-505,True,GTGAGCCGTTAAGTCC-1,GTGAGCCGTTAAGTCC-1,1.914767750731165,15.0,-0.10096652523816445,0.0033735242389664327,1.4475539,5.9705586,5,S,1792.288076132536,1,1,5934.0,1524.0,6.69025952140209,49.02258173238962,0.6132308121872807,False,False,6.8,9.276881,11.293825,1381.0,9.0,1,1.5813781746189925
-506,True,GTGAGCCGTTACACAC-1,GTGAGCCGTTACACAC-1,1.9605998441621446,15.0,-0.12868866280809435,-0.024523156710320453,-4.7320776,-7.8737206,3,G1,986.720964372158,2,2,15853.0,3454.0,7.613700876805652,29.268908093105406,0.9017510779184013,False,False,2.0,-15.885158,-8.332598,1453.0,9.0,2,0.8054144824310264
-507,True,GTGAGCCGTTGGAGGT-1,GTGAGCCGTTGGAGGT-1,1.8978907362289528,15.0,-0.10660163966647013,-0.005471927958434491,3.8709202,4.9065924,12,G1,1873.5644485205412,1,1,7697.0,1807.0,6.275172144991555,49.915551513576716,0.6111906024541025,False,False,5.2,10.541565,9.040163,1447.0,10.0,1,1.9461565517866377
-508,True,GTGAGCCGTTGGGATG-1,GTGAGCCGTTGGGATG-1,1.9733737451215703,15.0,-0.1105438870728522,-0.007930439457631415,2.3079946,18.245754,18,G1,1207.519254386425,5,5,4279.0,1338.0,14.839915868193502,35.68590792241178,0.7237064369412606,False,False,9.100000000000001,-4.127724,15.02495,1499.0,9.0,5,1.2445225742530512
-510,True,GTGAGCCTCCCAACTC-1,GTGAGCCTCCCAACTC-1,1.8874373387668735,15.0,-0.07495158141631046,-0.013568111921847679,9.652743,-10.709131,21,G1,1300.8082700371742,7,6,4324.0,1761.0,8.996299722479186,17.252543940795558,0.05792982617213052,False,False,8.2,12.332295,-10.453076,2493.0,9.0,7,1.479914115214732
-511,True,GTGAGCCTCCCAAGCG-1,GTGAGCCTCCCAAGCG-1,1.8870914520730193,15.0,-0.10625762921185479,0.01747026557468347,10.144069,4.105806,26,S,1006.3072529733181,9,9,8328.0,2135.0,6.015850144092219,41.63064361191162,0.6131626569486126,False,True,4.2,11.442899,4.7176294,1687.0,9.0,9,0.9716577852028643
-512,True,GTGAGCCTCCTGCTAC-1,GTGAGCCTCCTGCTAC-1,2.027946381367446,15.0,-0.06112059697144727,-0.028189268907615306,-9.917135,-0.9529053,7,G1,1653.1824572235346,3,3,9883.0,3035.0,12.182535667307498,9.410098148335525,0.9893603513057588,False,False,5.9,-16.203405,-1.6806772,2665.0,9.0,3,2.1011954019149104
-513,True,GTGAGCCTCGTCGGGT-1,GTGAGCCTCGTCGGGT-1,1.8787686266259958,15.0,-0.10079029489393893,-0.02220596211881282,8.413511,-0.5823542,23,G1,988.9731565564871,6,8,4699.0,1569.0,9.065758672057884,35.11385401149181,0.5340997388898594,False,False,3.8000000000000003,14.351809,2.8623846,1410.0,9.0,6,0.8054144824310264
-514,True,GTGAGCCTCTGAATCG-1,GTGAGCCTCTGAATCG-1,1.9389098333995145,15.0,-0.11087559964963423,-0.007738641329086898,0.11604611,12.331433,2,G1,844.5019671469927,5,5,5323.0,1566.0,6.443734736051099,41.72459139582942,0.7259674662331613,False,False,6.500000000000001,-0.070908725,16.66124,1134.0,9.0,5,1.069266113299076
-515,True,GTGGAGAAGAGAGCGG-1,GTGGAGAAGAGAGCGG-1,1.9051926906015735,15.0,-0.08338686161820022,-0.020147266981143615,9.13239,-4.3947196,23,G1,1164.9217701405287,6,8,4119.0,1441.0,10.827870842437484,31.731002670551106,0.3810027725046905,False,False,6.6000000000000005,14.4549465,-1.6591017,1395.0,9.0,6,1.1817649573553104
-516,True,GTGGAGAAGATCACCT-1,GTGGAGAAGATCACCT-1,1.9487917914529382,15.0,-0.07178846788883948,-0.014996942887340471,-6.4327035,-8.791237,3,G1,1268.0536492466927,2,2,12254.0,3594.0,8.413579239432023,16.802676677003426,0.9504671554960653,False,False,5.6,-14.978763,-8.331528,1954.0,9.0,2,1.4251293084786487
-517,True,GTGGAGAAGCACCTGC-1,GTGGAGAAGCACCTGC-1,1.8955718492896854,15.0,-0.1138256320830245,0.004793552970105145,5.8336267,5.072006,12,S,1514.9389950037003,1,1,7735.0,1769.0,4.201680672268908,51.58371040723982,0.6110450730456021,False,False,3.0999999999999996,11.265764,8.122241,1162.0,9.0,1,1.123968205432449
-518,True,GTGGAGAAGCATTGAA-1,GTGGAGAAGCATTGAA-1,1.9748978528624122,15.0,-0.06163184071320128,-0.02209429655247542,-5.0642986,-0.93358815,20,G1,954.7243008762598,4,4,6018.0,2299.0,10.784313725490197,9.92023928215354,0.9620537058419285,False,False,5.599999999999999,-17.091925,-1.2080588,1781.0,9.0,4,1.0389605241795559
-519,True,GTGGAGAAGGCTCAAG-1,GTGGAGAAGGCTCAAG-1,2.0280536439457797,15.0,-0.026460580675274025,-0.035783902893787724,0.24478208,8.634653,5,G1,1302.0978266447783,1,1,1031.0,492.0,28.51600387972842,4.364694471387003,0.6253365333800049,False,False,5.1000000000000005,9.403603,13.532088,1474.0,9.0,1,0.8054144824310264
-520,True,GTGGAGAAGGTTACAA-1,GTGGAGAAGGTTACAA-1,1.9724585499627925,15.0,-0.06316860220907766,-0.023095004999359615,-7.701872,-6.6284986,3,G1,1555.6911355406046,2,2,10988.0,3286.0,10.029122679286495,13.423734983618493,0.9672610567941193,False,False,3.7,-14.701832,-6.4220695,3242.0,9.0,2,1.263166790240676
-521,True,GTGGAGAAGTAGGAAG-1,GTGGAGAAGTAGGAAG-1,1.9184892948150047,15.0,-0.08483097292212322,-0.011536865645160429,5.630493,-0.20171976,22,G1,1150.6198026835918,6,8,4917.0,1593.0,12.34492576774456,34.49257677445597,0.5631527153179208,False,False,3.4,14.445616,4.261913,1246.0,9.0,6,1.1458554135230128
-522,True,GTGGAGAAGTGGCCTC-1,GTGGAGAAGTGGCCTC-1,1.9860356116974116,15.0,-0.116305814143599,-0.015209171830115546,-2.700012,15.363356,2,G1,1271.5897273421288,5,5,9253.0,2206.0,11.855614395331244,40.24640657084189,0.7337904098723382,False,False,7.8999999999999995,0.8834708,14.180544,1160.0,9.0,5,1.4035771029350492
-523,True,GTGGAGACAAGTCGTT-1,GTGGAGACAAGTCGTT-1,1.8805150405594686,15.0,-0.09878444559674233,-0.010786905637578444,7.5359607,4.9781437,8,G1,1399.1317790672183,9,9,5306.0,1440.0,4.730493780625706,48.17188088955899,0.6083959190058399,False,False,2.9,12.684891,6.5576515,1398.0,9.0,9,0.9967247498089045
-524,True,GTGGAGACACATATGC-1,GTGGAGACACATATGC-1,2.0066052882125085,15.0,-0.053609957457820705,-0.014353807780450457,-7.471446,-0.051164076,4,G1,1580.6797027885914,4,4,9157.0,2757.0,10.64759200611554,15.72567434749372,0.9677103027375437,False,False,4.6000000000000005,-14.6141,-0.7442251,2408.0,9.0,4,1.8819233464060614
-525,True,GTGGAGACATCTCCCA-1,GTGGAGACATCTCCCA-1,1.9852881352167047,15.0,-0.0736353704445755,-0.02417197623207743,-11.48046,-3.5127802,7,G1,1324.2157233208418,3,3,7268.0,2655.0,9.314804623004953,9.906439185470555,0.9946185058522786,False,False,3.6,-17.974112,-4.031796,2928.0,9.0,3,1.00956676208859
-526,True,GTGGAGACATGAATAG-1,GTGGAGACATGAATAG-1,1.997862569371393,15.0,-0.06730090589835278,-0.029342615380665415,-8.288498,0.0060452307,4,G1,1540.8689799308777,4,4,7941.0,2596.0,8.563153255257523,16.97519204130462,0.9733637503200613,False,False,5.6000000000000005,-14.546636,-1.1862334,2720.0,9.0,4,1.734008289080401
-527,True,GTGGAGAGTAGAATGT-1,GTGGAGAGTAGAATGT-1,1.9560694131007152,15.0,-0.06594978881131797,-0.03122968854355757,-8.0574045,-6.5586324,3,G1,1199.490616068244,2,2,8983.0,2981.0,6.801736613603473,11.699877546476678,0.9715443352494672,False,False,3.8000000000000003,-16.753887,-6.2603874,3489.0,9.0,2,1.0220909052009652
-528,True,GTGGAGAGTAGTGTGG-1,GTGGAGAGTAGTGTGG-1,1.8999575605977002,15.0,-0.09220859135698137,-0.04515692029803567,4.327864,7.6955724,27,G1,1488.3171306401491,8,1,2881.0,1237.0,15.688996876084692,18.986463033668866,0.6155038681506834,False,False,2.8000000000000003,11.787191,11.041236,1021.0,9.0,8,1.1006157928494924
-529,True,GTGGAGAGTATGCGTT-1,GTGGAGAGTATGCGTT-1,1.8804644330004883,15.0,-0.0881052091734523,-0.00470920264190363,10.160702,-4.251098,23,G1,1130.5492237210274,6,8,4516.0,1635.0,8.990256864481843,32.108060230292296,0.35541232237950837,False,False,7.3999999999999995,14.166001,-2.366508,991.0,9.0,6,1.5280308569241976
-530,True,GTGGAGAGTGACAGGT-1,GTGGAGAGTGACAGGT-1,2.047773550500772,15.0,-0.08181792310538147,-0.02665656847094947,-10.7379265,-0.41238794,7,G1,1447.9372851401567,3,3,3604.0,1615.0,11.542730299667037,10.238623751387347,0.9903124113072069,False,False,4.3,-17.57844,-1.2917508,2446.0,9.0,3,1.012522421032248
-532,True,GTGGAGATCCACGTAA-1,GTGGAGATCCACGTAA-1,1.9954623538143408,15.0,-0.05989603663214681,-0.01465790959585751,-9.261654,-3.8173409,10,G1,1401.7336284518242,3,3,8639.0,2591.0,10.429447852760736,13.022340548674615,0.9812697916904074,False,False,3.8999999999999995,-13.35116,-3.589532,2241.0,9.0,3,1.066574391796639
-533,True,GTGGAGATCCGCACGA-1,GTGGAGATCCGCACGA-1,1.9450201123214939,15.0,-0.08565160450134508,-0.005132451772211586,3.1629772,5.51316,12,G1,1803.017857760191,1,1,4621.0,1370.0,17.29062973382385,35.77147803505735,0.6124952109151263,False,False,3.8,10.8035,10.618077,1056.0,9.0,1,1.1966467743803582
-534,True,GTGGAGATCCGGACGT-1,GTGGAGATCCGGACGT-1,1.9309597455804457,15.0,-0.11911343678722212,0.004484732554154325,-1.1289706,13.687366,2,S,1070.9903070628643,5,5,5876.0,1657.0,10.704560925799864,38.49557522123894,0.7300678558589978,False,False,4.7,0.36199176,15.556907,2092.0,9.0,5,0.8054144824310264
-535,True,GTGGAGATCTTAGCCC-1,GTGGAGATCTTAGCCC-1,1.8789197958739425,15.0,-0.09872703747451962,-0.007904698459155852,-0.31033337,5.143897,5,G1,1028.9701055586338,1,1,3556.0,1155.0,6.664791901012373,41.451068616422944,0.6108977404328603,False,False,3.3,7.9720926,10.035867,1128.0,9.0,1,0.8054144824310264
-536,True,GTGTGGCAGCCTCTTC-1,GTGTGGCAGCCTCTTC-1,1.904967098237018,15.0,-0.0840913642643487,-0.0028232116808229943,8.688718,2.853584,8,G1,1368.1487836837769,9,9,5797.0,1679.0,9.315163015352768,38.53717440055201,0.6060852461586613,False,False,4.3,12.684597,5.222799,978.0,9.0,9,1.421333812828499
-537,True,GTGTGGCAGCTCACTA-1,GTGTGGCAGCTCACTA-1,1.8886636395485266,15.0,-0.10406222915004641,-0.012192039896912315,7.634615,4.164181,26,G1,1448.109961770475,9,9,14201.0,2934.0,6.098162101260475,45.37708612069572,0.6098099341195842,False,False,3.8000000000000003,11.730461,6.532735,3125.0,9.0,9,0.8054144824310264
-538,True,GTGTGGCAGGAACTCG-1,GTGTGGCAGGAACTCG-1,2.01876031145061,15.0,-0.07467242880283059,-0.032897264808150965,-7.04266,-2.6689494,10,G1,1562.22102047503,3,3,7201.0,2483.0,14.94236911540064,13.23427301763644,0.9677294751702443,False,False,3.3,-14.328333,-3.0531821,2386.0,9.0,3,1.6698757083431846
-540,True,GTGTGGCAGTAATTGG-1,GTGTGGCAGTAATTGG-1,1.920737611051871,15.0,-0.10248168876213459,-0.00952584170337291,3.3508363,5.0465765,12,G1,1762.616995356977,1,1,4732.0,1333.0,7.016060862214708,48.24598478444632,0.6113735147627062,False,False,3.8,9.96698,8.411963,1374.0,9.0,1,1.1008992261743455
-541,True,GTGTGGCAGTGCGTCC-1,GTGTGGCAGTGCGTCC-1,1.967393129967805,15.0,-0.09907354421797154,0.006165758383099809,-2.3485239,13.811834,2,S,1191.0935328900814,5,5,6246.0,1677.0,11.959654178674352,41.80275376240794,0.732807797049985,False,False,7.4,1.5323181,14.820442,758.0,9.0,5,1.4166384903767528
-542,True,GTGTGGCAGTGTTGAA-1,GTGTGGCAGTGTTGAA-1,1.8873270360965113,15.0,-0.1403465605547684,-0.016909035125758524,0.7772391,8.044735,5,G1,1485.3836473077536,1,1,5482.0,1444.0,3.6483035388544325,50.41955490696826,0.6202882594422758,False,False,3.9999999999999996,9.927231,12.392451,1257.0,9.0,1,1.228392641989862
-543,True,GTGTGGCCAAGCCATT-1,GTGTGGCCAAGCCATT-1,2.009801123880846,15.0,-0.07602417251844534,-0.028042864733199467,-7.952174,-1.8129474,10,G1,1464.262759566307,3,3,8375.0,2852.0,13.444776119402984,13.217910447761193,0.9335728120019586,False,False,2.7,-14.813199,-2.0232449,3824.0,9.0,3,0.8054144824310264
-544,True,GTGTGGCCACAGAGAC-1,GTGTGGCCACAGAGAC-1,1.9008589957140591,15.0,-0.1313728864340083,-0.00778138987355392,4.4841113,3.3045228,12,G1,1717.1841997355223,1,1,9221.0,2100.0,5.7260600802515995,48.91009651881575,0.6114296400514194,False,False,3.3,11.194004,8.22794,1592.0,9.0,1,0.8054144824310264
-545,True,GTGTGGCCACATGAAA-1,GTGTGGCCACATGAAA-1,1.8764146877219223,15.0,-0.1111497549668062,-0.02424807204054658,2.8081772,7.0896916,5,G1,1286.135250851512,1,1,4084.0,1235.0,3.5749265426052887,45.372184133202744,0.6121397593791077,False,False,3.3000000000000003,8.564429,9.542634,1834.0,9.0,1,1.4387270348037065
-546,True,GTGTGGCCACTAAACC-1,GTGTGGCCACTAAACC-1,2.014304614125087,15.0,-0.05784216087117759,-0.026449333673292868,-6.3294683,-0.306418,14,G1,1557.8175098001957,4,4,7670.0,2595.0,9.98696219035202,13.142112125162972,0.958767441528049,False,False,2.4,-14.441423,-0.8024009,1655.0,9.0,4,0.9418883927432015
-547,True,GTGTGGCCACTTCAGA-1,GTGTGGCCACTTCAGA-1,1.8888507061887982,15.0,-0.11525514814857202,-0.0168870824062931,3.973424,3.9524112,12,G1,1394.3165978342295,1,1,6409.0,1673.0,5.6639101263847715,45.779372757060386,0.6266805072490624,True,True,3.5999999999999996,12.958389,9.747328,1894.0,9.0,1,1.2098189097854344
-548,True,GTGTGGCCAGTTTCGA-1,GTGTGGCCAGTTTCGA-1,2.008340912481727,15.0,-0.06582109352721566,-0.013183690830575565,-4.7509894,-1.4714333,20,G1,1551.6061856150627,4,4,10629.0,3096.0,9.031893875246967,15.043748235958228,0.9582177526234296,False,False,2.6,-15.549908,-1.4873714,1986.0,9.0,4,1.4525959895187106
-549,True,GTGTGGCCATGGCTAT-1,GTGTGGCCATGGCTAT-1,1.8840520130588523,15.0,-0.08726258060538192,-0.02046114660062633,9.28119,1.8148487,25,G1,986.5468703061342,6,9,5764.0,1920.0,8.102012491325468,28.140180430256766,0.5938800897565121,False,False,2.7,14.540578,5.4934134,927.0,9.0,6,1.0959107728037938
-550,True,GTGTGGCGTAACTTCG-1,GTGTGGCGTAACTTCG-1,2.0461532256235286,15.0,-0.08431246607638532,-0.028595160609485422,-7.873993,-4.2421603,10,G1,1299.1468257904053,3,3,2852.0,1238.0,20.021037868162693,10.659186535764375,0.9678727529836058,False,False,3.9000000000000004,-13.141097,-4.565519,1146.0,9.0,3,1.5600077572158508
-551,True,GTGTGGCGTATACCTG-1,GTGTGGCGTATACCTG-1,1.9230146274608166,15.0,-0.10571806265852737,0.0027762165843775513,-3.05646,11.806054,2,S,859.8493028283119,5,5,15737.0,3525.0,7.599923746584483,35.432420410497556,0.7153854706216032,False,False,1.8,3.4538765,12.742705,602.0,9.0,5,0.947274088417254
-552,True,GTGTGGCGTATAGGAT-1,GTGTGGCGTATAGGAT-1,2.014939091010808,15.0,-0.07562550251156273,-0.025462812452320023,-5.2605324,-0.526607,20,G1,1147.337303698063,4,4,6442.0,2339.0,13.054951878298665,13.272275690779262,0.9627560828322599,False,False,4.5,-16.246864,-0.9355467,1005.0,9.0,4,1.2852959077809885
-553,True,GTGTGGCGTCGAATGG-1,GTGTGGCGTCGAATGG-1,1.8882892463343817,15.0,-0.11626429934828691,-0.013652043520326246,1.4797767,6.8057137,5,G1,1721.6973308771849,1,1,5652.0,1452.0,4.476291578202407,50.424628450106155,0.6124197063239225,False,False,5.5,8.66766,11.105161,1065.0,9.0,1,1.3540715914304267
-554,True,GTGTGGCGTGTCTAAC-1,GTGTGGCGTGTCTAAC-1,1.9149036336515632,15.0,-0.08855117830416426,-0.006953402057463436,11.800872,-8.146275,11,G1,1137.555390626192,7,6,4601.0,1447.0,8.128667680938927,33.62312540752011,0.16766685263510545,False,False,6.200000000000001,12.753118,-6.58073,2078.0,9.0,7,0.985777900473622
-555,True,GTGTGGCGTTAGGAGC-1,GTGTGGCGTTAGGAGC-1,1.9105927124897024,15.0,-0.0994945914022317,-0.0010649388000283332,4.870056,4.032449,12,G1,1865.770537301898,1,1,6729.0,1698.0,6.4199732501114575,46.64883340763858,0.6108701562817306,False,False,5.1,11.386353,8.931483,1255.0,9.0,1,1.5686620003807206
-556,True,GTGTGGCTCGCCACTT-1,GTGTGGCTCGCCACTT-1,1.9262843668692167,15.0,-0.10491581209580271,-0.01341335167224365,6.450916,-1.3414365,22,G1,1022.8307742029428,6,8,6728.0,1783.0,11.682520808561236,37.96076099881094,0.5130630349402919,False,False,4.6,14.868992,2.4050767,1318.0,9.0,6,1.4666392467974971
-557,True,GTGTGGCTCGCCCAGA-1,GTGTGGCTCGCCCAGA-1,1.9004333706317915,15.0,-0.07589867480913785,-0.03944408800525256,8.41546,-12.476289,21,G1,1180.560720384121,7,6,3889.0,1599.0,11.828233479043456,17.125224993571614,0.018074712648886805,False,False,5.1000000000000005,11.4219475,-11.503132,1248.0,9.0,7,1.0258942873036565
-558,True,GTGTGGCTCGCGGTAC-1,GTGTGGCTCGCGGTAC-1,1.8933313254289574,15.0,-0.09862561041729072,-0.006638174968545541,8.335781,8.421789,15,G1,1032.4547358453274,8,10,5112.0,1381.0,6.4358372456964,47.04616588419405,0.6152289422858586,False,False,4.7,14.856622,11.298916,939.0,9.0,8,0.9558057699680218
-559,True,TAACACGAGACCATTC-1,TAACACGAGACCATTC-1,1.9033489921237152,15.0,-0.07455198671300772,-0.015561541737120597,-2.8152425,-13.030462,24,G1,1348.863736987114,10,7,15835.0,3894.0,6.86454057467635,17.221345121566152,0.9590593710458568,False,True,8.0,-6.920044,-11.725984,7706.0,10.0,10,1.4736247443789987
-560,True,TAACACGAGAGAGCGG-1,TAACACGAGAGAGCGG-1,1.9519468010960916,15.0,-0.11083144011262405,-0.030922217594768354,-6.886939,-1.6452265,20,G1,997.6873728185892,4,4,8712.0,2712.0,9.630394857667586,18.296602387511477,0.9669014308476728,False,True,3.9,-17.573704,-1.1268713,4199.0,9.0,4,1.0266563462822664
-561,True,TAACACGAGCACTCTA-1,TAACACGAGCACTCTA-1,1.8800594229410257,15.0,-0.10651561863833017,-0.01115114314866432,10.218443,-8.092923,11,G1,1207.382659688592,7,6,5170.0,1867.0,7.988394584139265,28.007736943907158,0.15314046665345904,False,False,5.999999999999999,13.461852,-7.689159,1354.0,9.0,7,1.3202140573738887
-562,True,TAACACGAGGCCTAGA-1,TAACACGAGGCCTAGA-1,1.9066860947618471,15.0,-0.08276749664614409,-0.02189224227870877,-1.940731,-13.490963,24,G1,1322.960656479001,10,7,13921.0,3520.0,8.081315997413979,17.778895194310753,0.9591470330235248,True,True,7.800000000000001,-6.241779,-12.402616,2666.0,9.0,10,0.9625685164062322
-564,True,TAACACGCAAGAGTAT-1,TAACACGCAAGAGTAT-1,1.9681571149831516,15.0,-0.1309402366484819,-0.02042872298829933,-1.5339167,13.3768425,2,G1,1069.8489926308393,5,5,9075.0,2272.0,9.234159779614325,38.53443526170799,0.7323363832717652,False,False,5.6,0.8702816,15.740313,1844.0,9.0,5,0.8054144824310264
-565,True,TAACACGCAATAGTAG-1,TAACACGCAATAGTAG-1,2.1425855976425883,15.0,-0.03134326993069457,-0.017434321641441382,-8.287345,-12.267354,16,G1,664.1168532222509,2,2,2673.0,761.0,11.485222596333708,2.05761316872428,0.9315222778269401,False,False,6.3,-15.337466,-11.092378,889.0,9.0,2,1.2895372537594163
-566,True,TAACACGCACTACACA-1,TAACACGCACTACACA-1,1.9202316397211454,15.0,-0.07498771809876684,-0.01825879797603801,-3.2467015,-12.256461,24,G1,1328.758084088564,10,7,16238.0,3638.0,7.89506096809952,21.203350166276635,0.9565544187994006,False,False,7.499999999999999,-7.074737,-11.0302725,3104.0,9.0,10,0.8054144824310264
-567,True,TAACACGCAGCTGCCA-1,TAACACGCAGCTGCCA-1,1.9600644415830377,15.0,-0.07321547751492852,-0.026157096311316054,-8.797946,-6.445177,10,G1,1274.639593333006,3,7,10315.0,3226.0,6.185167232186137,15.472612699951528,0.9719181562955227,False,False,3.1000000000000005,-12.837567,-6.6821237,3020.0,10.0,3,1.0586414463989005
-568,True,TAACACGCAGGTTCAT-1,TAACACGCAGGTTCAT-1,1.9014838718921419,15.0,-0.11782215530580094,-0.013612492429579435,0.5158483,3.83589,5,G1,1213.1601306647062,1,1,6239.0,1624.0,5.465619490302934,48.1968264144895,0.6099970246009617,False,False,2.6999999999999997,9.081205,8.622112,1430.0,9.0,1,0.8054144824310264
-569,True,TAACACGCATTATGCG-1,TAACACGCATTATGCG-1,1.8909426466267778,15.0,-0.11224465863155332,-0.00932828551071655,4.847835,1.59169,12,G1,1717.5075565129519,1,1,5873.0,1454.0,5.125148986889154,48.271752085816445,0.6069933511159287,False,False,5.3,12.610448,8.843966,1247.0,9.0,1,1.0898713919160035
-570,True,TAACACGGTCAAGCCC-1,TAACACGGTCAAGCCC-1,1.93274444151844,15.0,-0.0813911012586325,-0.010181242724866955,-4.103703,-14.271429,24,G1,1070.1442987173796,10,7,16706.0,3907.0,9.631270202322519,18.711840057464386,0.9598217932824393,False,False,3.5,-8.470448,-12.166748,4555.0,9.0,10,0.9358706416958789
-571,True,TAACACGGTGATTGGG-1,TAACACGGTGATTGGG-1,1.8990788546206345,15.0,-0.10789622622727958,-0.005769823017110282,9.915226,-8.788749,11,G1,1218.828633338213,7,6,5032.0,1620.0,7.452305246422894,35.59220985691574,0.11851404160697072,False,False,6.4,13.047045,-8.476495,1750.0,9.0,7,1.3469965854545554
-573,True,TAACACGGTTCGGCTG-1,TAACACGGTTCGGCTG-1,1.953453608866699,15.0,-0.09244408803350262,-0.02760640022106376,-11.70976,-3.587752,7,G1,996.6828173696995,3,3,1740.0,1106.0,2.586206896551724,8.218390804597702,0.9908771412137711,False,False,2.3,-18.703491,-2.6982446,5385.0,9.0,3,0.8054144824310264
-575,True,TAACACGTCAGGGATG-1,TAACACGTCAGGGATG-1,1.8740479403842105,15.0,-0.08824078591520453,-0.004802221630700598,7.6794186,-1.6484816,22,G1,1083.5563035309315,6,8,5940.0,1882.0,7.878787878787879,33.686868686868685,0.4853548748010588,False,False,5.1000000000000005,14.6604185,1.7181122,2001.0,9.0,6,1.2451788854830004
-576,True,TAACACGTCTGAATCG-1,TAACACGTCTGAATCG-1,1.9261569551853854,15.0,-0.11690768121041767,-0.016804736692211393,4.802303,2.3117132,12,G1,1822.1864954680204,1,1,4245.0,1161.0,10.789163722025913,42.94464075382803,0.6063914099100421,False,False,5.999999999999999,12.159952,8.022453,1398.0,9.0,1,1.4804076122355372
-577,True,TAGGAGGAGACGTCCC-1,TAGGAGGAGACGTCCC-1,1.9936098467109646,15.0,-0.08939515979007723,-0.026621780743491974,-11.501834,-6.559144,7,G1,1071.2746607661247,3,3,8895.0,2895.0,14.592467678471051,12.906127037661607,0.9749858074559021,False,False,2.6999999999999997,-17.884352,-6.1094074,3012.0,9.0,3,0.9661959181781773
-578,True,TAGGAGGAGCACCTGC-1,TAGGAGGAGCACCTGC-1,1.9632201369032927,15.0,-0.07868706192628899,-0.018419557634556504,-6.017016,-3.182112,7,G1,1567.4161136895418,3,2,15647.0,3804.0,7.400779702179332,24.113248546047167,0.968668689195569,False,False,3.3999999999999995,-13.986904,-3.6690094,2423.0,9.0,3,1.2558742398439324
-579,True,TAGGAGGAGGGCGAAG-1,TAGGAGGAGGGCGAAG-1,2.0329549554834716,15.0,-0.07078358665253386,-0.028795454987645196,-3.5725312,-5.272411,19,G1,1556.816163316369,2,2,7485.0,2516.0,14.762859051436205,15.537742150968604,0.95561288461176,False,True,3.7,-14.98978,-5.333673,2976.0,9.0,2,1.5613533066321073
-580,True,TAGGAGGAGTAGGAAG-1,TAGGAGGAGTAGGAAG-1,1.9445393533258113,15.0,-0.09305220036230434,-0.006706019165013826,2.2003303,2.010643,12,G1,1562.44801735878,1,1,5274.0,1634.0,14.125900644671976,33.10580204778157,0.6187772805208653,False,False,3.6999999999999997,11.38515,9.805123,1235.0,9.0,1,1.0069201340880614
-581,True,TAGGAGGAGTATAACG-1,TAGGAGGAGTATAACG-1,1.971208971634281,15.0,-0.0814286077177887,-0.016238080822293572,-6.0400753,-5.4865317,3,G1,1639.4385280311108,2,2,9108.0,2886.0,10.342555994729908,16.458058849363198,0.9610227431590741,False,False,4.2,-14.379485,-5.628166,2814.0,10.0,2,1.5492031698581938
-582,True,TAGGAGGAGTCATCCA-1,TAGGAGGAGTCATCCA-1,1.958917505708069,15.0,-0.08892664863720064,-0.020712163391854548,-10.122102,-7.210983,10,G1,1112.0317127853632,3,7,9770.0,2860.0,6.366427840327534,18.075742067553737,0.9765035255073727,False,False,4.0,-12.841159,-7.9210706,2648.0,9.0,3,1.5293952261509267
-583,True,TAGGAGGCAACCAATC-1,TAGGAGGCAACCAATC-1,1.9915012917264392,15.0,-0.08112713176649827,-0.025905524979639962,-8.806339,-5.3607373,10,G1,1223.1636317968369,3,3,7251.0,2469.0,9.695214453178872,15.694386981106055,0.9715453745571917,False,False,4.6000000000000005,-12.349511,-5.6623096,2531.0,9.0,3,1.280267109300333
-584,True,TAGGAGGCAACGCATT-1,TAGGAGGCAACGCATT-1,1.951175288911056,15.0,-0.09538257082041787,-0.022832975516875437,-10.283912,-8.639964,10,G1,1165.3721543550491,3,7,14046.0,3796.0,9.917414210451375,19.78499216858892,0.9718389029432524,False,False,3.7,-12.824407,-8.980664,3240.0,9.0,3,0.9545945411403118
-585,True,TAGGAGGCAAGTCGTT-1,TAGGAGGCAAGTCGTT-1,1.898853196175605,15.0,-0.10753085614695354,-0.004224803338899363,4.8282394,6.975618,12,G1,1354.5474991127849,1,1,5953.0,1696.0,4.4851335461112045,43.80984377624727,0.6126961983460109,False,False,5.1000000000000005,9.677133,7.8861136,1426.0,9.0,1,1.3709377976051438
-586,True,TAGGAGGCAATAGTAG-1,TAGGAGGCAATAGTAG-1,1.906703705366873,15.0,-0.11098770043313871,-0.02622781748995341,8.667967,-4.3739247,23,G1,1162.123053625226,6,8,4368.0,1453.0,11.057692307692308,34.40934065934066,0.38323220618685905,False,False,5.8,14.554498,-1.511154,844.0,9.0,6,1.0077115079891399
-587,True,TAGGAGGCACTACGGC-1,TAGGAGGCACTACGGC-1,1.8945286432571844,15.0,-0.11814864150735731,-0.010970622461545834,8.968799,1.4595244,22,G1,1235.2577348947525,6,9,7910.0,1916.0,6.8773704171934265,44.424778761061944,0.584689245662746,False,True,3.0,13.288368,4.29623,1516.0,9.0,6,1.2394139250314031
-589,True,TAGGAGGCATGGCTAT-1,TAGGAGGCATGGCTAT-1,1.9957486620315232,15.0,-0.07517682883846892,-0.024963075247329908,-4.832475,-2.2758834,17,G1,1531.5107227116823,2,2,9449.0,2955.0,10.91120753518891,13.38766006984866,0.9612734498808503,False,False,4.2,-16.416534,-3.5820715,2802.0,9.0,2,1.0928298243498888
-590,True,TAGGAGGGTAACGCGA-1,TAGGAGGGTAACGCGA-1,1.8943114395229252,15.0,-0.11367024448419799,-0.007417834544337951,8.9664135,4.5975585,26,G1,1318.4765589386225,9,9,5360.0,1483.0,7.201492537313433,44.14179104477612,0.6118644776664354,True,True,4.5,11.157627,6.006687,1589.0,9.0,9,0.9762847362179835
-591,True,TAGGAGGGTACTGAGG-1,TAGGAGGGTACTGAGG-1,1.9111135075418189,15.0,-0.0975639193894592,0.003496586344868515,7.855893,2.9994626,8,S,1228.6807295084,9,9,9229.0,2243.0,7.043016578177484,46.96066746126341,0.6070583258567167,True,True,2.3000000000000003,13.324407,6.3535943,1253.0,9.0,9,1.1295934572259603
-592,True,TAGGAGGGTATAGGAT-1,TAGGAGGGTATAGGAT-1,1.9047388680279134,15.0,-0.10867092448911385,-0.009207974178863342,8.05196,6.2594175,26,G1,1264.2727644443512,9,9,5051.0,1580.0,7.463868540882993,41.39774302118392,0.6121872918068108,False,False,2.8999999999999995,10.622946,6.5859957,899.0,9.0,9,0.8054144824310264
-593,True,TAGGAGGGTTAAGCAA-1,TAGGAGGGTTAAGCAA-1,2.0047318657593602,15.0,-0.09627396671465178,-0.014319834763420546,-0.5014523,12.927036,2,G1,913.4835133701563,5,5,7558.0,1997.0,15.347975654935167,38.54194231278116,0.7272525849375067,False,False,6.0,0.30898312,16.49458,804.0,9.0,5,0.9434651752928881
-594,True,TAGGAGGGTTAAGTCC-1,TAGGAGGGTTAAGTCC-1,1.9131667171362303,15.0,-0.07055254913272051,-0.017225989924727063,12.152752,-9.678508,11,G1,1167.1029457896948,7,6,4788.0,1717.0,10.609857978279031,26.900584795321638,0.11715038552180444,False,False,4.4,12.32061,-8.3084345,597.0,9.0,7,0.8054144824310264
-595,True,TAGGAGGGTTACACAC-1,TAGGAGGGTTACACAC-1,1.9477946450344197,15.0,-0.06521006433082752,-0.015573832814642547,3.1688619,2.8093562,12,G1,1536.741395458579,1,1,2645.0,1238.0,21.134215500945178,9.338374291115311,0.6149674078587354,False,False,4.800000000000001,12.145481,10.631927,1582.0,9.0,1,1.2584565298571562
-596,True,TAGGAGGGTTCCACGG-1,TAGGAGGGTTCCACGG-1,1.9007278658713505,15.0,-0.12436534128104718,-0.010672957361665816,3.4676182,4.428808,12,G1,1885.0434751063585,1,1,5315.0,1486.0,6.942615239887112,45.90780809031044,0.6109452680758359,False,False,5.1,10.94628,9.562157,1157.0,9.0,1,1.8075590031723183
-597,True,TAGGAGGTCAAAGGTA-1,TAGGAGGTCAAAGGTA-1,1.8939617317346136,15.0,-0.12180518643418639,-0.0012523704505757322,3.2198503,5.6159115,12,G1,1813.1293862983584,1,1,6128.0,1544.0,4.29177545691906,49.05352480417755,0.61148884990741,False,False,4.6000000000000005,9.8759775,9.728944,1231.0,9.0,1,1.393284096196813
-598,True,TAGGAGGTCAAGAAAC-1,TAGGAGGTCAAGAAAC-1,1.9742953775730374,15.0,-0.07485548599031507,-0.018939021833864894,-8.661267,0.9183273,14,G1,1355.3689723163843,4,3,8683.0,2799.0,9.466774156397559,15.340320165841298,0.9689671405036082,False,False,2.2,-16.601643,0.55356663,3248.0,9.0,4,1.0072247322945997
-599,True,TAGGAGGTCACAAGAA-1,TAGGAGGTCACAAGAA-1,2.1201900482191594,15.0,-0.05359627452650709,-0.03421174294960702,-10.597267,0.3448813,14,G1,1317.2918828427792,4,3,2100.0,957.0,29.38095238095238,6.904761904761905,0.9841858844032881,False,False,4.0,-17.349266,-0.29371935,2502.0,9.0,4,0.8054144824310264
-600,True,TAGGAGGTCCCAAGCG-1,TAGGAGGTCCCAAGCG-1,1.9988233354442089,15.0,-0.08232238881495414,-0.013305361277619525,-10.023526,-0.89700824,7,G1,1636.024218633771,3,3,8857.0,2730.0,10.116292198261263,13.300214519589026,0.9905414574355751,False,False,5.8,-16.8863,-1.8074914,3159.0,9.0,3,1.8620749307639513
-601,True,TAGGAGGTCTGAATCG-1,TAGGAGGTCTGAATCG-1,1.9755390933012411,15.0,-0.05818782099568162,-0.025867726700074832,-7.6753497,-5.6118655,3,G1,1525.7555863261223,2,3,7317.0,2562.0,12.641793084597513,13.953806204728714,0.9687611809688031,False,False,3.9999999999999996,-13.435364,-5.7192535,3470.0,9.0,2,1.503104343376727
-602,True,TATTGCTAGATACCAA-1,TATTGCTAGATACCAA-1,1.9070866497110317,15.0,-0.07991389060340508,-0.004506051176124416,9.32593,-3.0909312,23,G1,1135.5415417701006,6,8,9234.0,2404.0,7.894736842105263,38.650638943036604,0.4181061881323877,False,False,5.500000000000001,13.976861,-0.9212981,1089.0,9.0,6,1.2251818304999142
-603,True,TATTGCTAGATCACCT-1,TATTGCTAGATCACCT-1,1.9266722344769016,15.0,-0.08562675880102187,-0.012331060568886415,-7.2983637,-10.608932,3,G1,951.5102065950632,2,2,15998.0,4165.0,6.5070633829228655,15.226903362920366,0.9463928509981301,False,False,3.8000000000000003,-14.682517,-10.087637,2349.0,9.0,2,0.8054144824310264
-604,True,TATTGCTAGATGCTTC-1,TATTGCTAGATGCTTC-1,1.9743303445289562,15.0,-0.09102187898879661,-0.022934394242392728,-9.331853,-2.2512615,7,G1,1617.8079160153866,3,3,8908.0,2801.0,9.508307139649753,19.21867983834755,0.9820632590015091,False,False,3.4,-14.601277,-2.2881668,2320.0,9.0,3,1.3319839568445446
-605,True,TATTGCTAGCCTCTGG-1,TATTGCTAGCCTCTGG-1,1.8800771881632905,15.0,-0.0904163885242908,0.01165367959318427,4.6101127,6.668374,6,S,1341.8668562993407,1,1,5558.0,1890.0,6.009355883411299,31.30622526088521,0.611862629929599,False,False,2.3000000000000003,8.805046,8.334858,951.0,9.0,1,0.8054144824310264
-606,True,TATTGCTAGGCGTTAG-1,TATTGCTAGGCGTTAG-1,1.9859935626688883,15.0,-0.06646771039174042,-0.018050337245951932,-8.655784,-9.622894,10,G1,1014.7960696518421,3,7,17281.0,4125.0,12.823331983102829,15.132226144320352,0.962557742606879,False,False,3.6,-13.089811,-9.804615,2224.0,9.0,3,0.9486621989407168
-607,True,TATTGCTAGGGAGGTG-1,TATTGCTAGGGAGGTG-1,1.9061415325160354,15.0,-0.11683510376426946,-0.015768572681362794,4.17502,4.838518,12,G1,1885.7940989285707,1,1,7019.0,1573.0,4.744265564895284,51.56005128935746,0.6107605971140642,False,False,6.1000000000000005,11.098937,9.626065,620.0,9.0,1,1.6478300539323947
-608,True,TATTGCTAGGGCGAAG-1,TATTGCTAGGGCGAAG-1,2.027804671298693,15.0,-0.05233977798114638,-0.020878774474114525,-2.7238657,-3.794747,19,G1,1479.1154086738825,2,2,6437.0,2274.0,16.047848376572937,15.224483455025632,0.955874405266003,False,False,3.5999999999999996,-15.67815,-4.382188,1251.0,9.0,2,0.8054144824310264
-610,True,TATTGCTCAAGAGAGA-1,TATTGCTCAAGAGAGA-1,1.891930605834647,15.0,-0.11433378879416412,-0.011763889330034735,3.0062764,5.314927,12,G1,1824.0052290707827,1,1,6681.0,1628.0,3.2031133063912587,49.453674599610835,0.6119541482747788,False,False,4.3,10.297705,9.8838005,1205.0,9.0,1,1.8122603952358871
-611,True,TATTGCTCAATCTCGA-1,TATTGCTCAATCTCGA-1,1.9150316969527776,15.0,-0.07658623405117282,-0.006658361360799661,1.161219,5.803264,5,G1,1776.2699012160301,1,1,3930.0,1266.0,8.854961832061068,35.64885496183206,0.6148259145101724,False,True,6.999999999999999,9.851053,11.085645,1190.0,9.0,1,2.0300141177107736
-612,True,TATTGCTCAATCTGCA-1,TATTGCTCAATCTGCA-1,1.9669135554056636,15.0,-0.11489039881107223,-0.025672567217626487,0.77447677,15.4919405,2,G1,1103.8704384416342,5,5,6031.0,1844.0,7.345382192007959,37.07511192173769,0.7199471630829142,False,False,9.0,-2.1658297,14.3880825,853.0,9.0,5,1.0104332441636088
-614,True,TATTGCTCACTACCGG-1,TATTGCTCACTACCGG-1,1.8718859420492193,15.0,-0.09403471657618005,-0.02402908506789862,8.236633,-3.712775,23,G1,1183.099345177412,6,8,5899.0,1942.0,8.255636548567553,28.818443804034583,0.42929088800269155,True,True,6.099999999999999,14.692297,-0.43960133,1117.0,9.0,6,0.9821697393258806
-615,True,TATTGCTCAGCTGCCA-1,TATTGCTCAGCTGCCA-1,1.8737555261088272,15.0,-0.0604862293968781,-0.052713425222367466,1.7309469,7.3114247,27,G1,1403.9662496596575,8,1,2090.0,1122.0,17.99043062200957,4.784688995215311,0.6143930943732607,False,False,2.9000000000000004,8.399804,10.903099,2308.0,9.0,8,0.8054144824310264
-616,True,TATTGCTGTAACTTCG-1,TATTGCTGTAACTTCG-1,1.8948126947674881,15.0,-0.08535820721714361,-0.011225621312497363,4.356797,8.601131,27,G1,1214.0728960633278,8,1,7302.0,2001.0,6.628321007943029,44.23445631333881,0.6163432031403224,False,False,2.4000000000000004,10.990289,10.99788,756.0,9.0,8,1.0948064196618672
-617,True,TATTGCTGTAGCGTAG-1,TATTGCTGTAGCGTAG-1,1.9797172225717925,15.0,-0.07859703572788469,-0.02484835319084938,-9.045951,-4.4229503,10,G1,1245.0829603374004,3,3,8300.0,2715.0,10.421686746987952,13.975903614457831,0.9742969554402228,False,False,3.5,-12.618501,-4.9787755,3585.0,9.0,3,1.2018442938375378
-618,True,TATTGCTTCAGGGATG-1,TATTGCTTCAGGGATG-1,1.9089545029234216,15.0,-0.10762696836472022,-0.00938552879731359,4.163219,5.40932,12,G1,1825.055550545454,1,1,5118.0,1301.0,4.962876123485737,48.57366158655725,0.6113091972989818,False,False,5.6,10.245847,9.419679,729.0,10.0,1,0.8054144824310264
-619,True,TATTGCTTCCCGTTGT-1,TATTGCTTCCCGTTGT-1,2.0015330038093033,15.0,-0.0751844801092407,-0.03339808022846287,-5.5073676,-2.3917763,17,G1,1339.216727644205,2,2,8415.0,2716.0,9.257278669043375,13.475935828877006,0.9637132842311372,False,False,3.6999999999999997,-16.223825,-3.291715,2194.0,9.0,2,1.353259659144308
-620,True,TATTGCTTCGAACGCC-1,TATTGCTTCGAACGCC-1,1.9174056017507883,15.0,-0.12154031483893259,-0.019023988110820424,0.13392748,6.919147,5,G1,1593.244992658496,1,1,7677.0,1853.0,7.19030871434154,43.376318874560376,0.6214142120706734,False,False,4.6,10.100385,12.098377,1136.0,9.0,1,0.8054144824310264
-621,True,TATTGCTTCGACGCGT-1,TATTGCTTCGACGCGT-1,1.9324273332814896,15.0,-0.08494708678375285,-0.019088442039509566,-5.406357,-12.82145,10,G1,1129.1717528998852,3,7,9889.0,2925.0,7.472949742137729,18.293052887046212,0.9623369145433637,False,False,3.3,-9.209503,-11.29541,750.0,9.0,3,1.0673286967617488
-622,True,TATTGCTTCTCACTCG-1,TATTGCTTCTCACTCG-1,1.8928693454525043,15.0,-0.07926553404055146,-0.021331143006628188,9.19427,-3.3381388,23,G1,1149.6492844969034,6,8,5802.0,2062.0,10.806618407445708,24.57773181661496,0.4018455621903199,False,False,5.800000000000001,14.405481,-1.367888,1221.0,9.0,6,0.8054144824310264
-623,True,TCACGCTAGACCATTC-1,TCACGCTAGACCATTC-1,1.9358764036019767,15.0,-0.08450114542784468,-0.0024136032772996314,-0.87585026,14.758426,2,G1,1154.8566451966763,5,5,6187.0,1753.0,7.919831905608534,37.72426054630677,0.7304825662210855,False,False,4.4,-0.17387588,14.949865,1230.0,9.0,5,1.0708122203630395
-624,True,TCACGCTAGCTTGTTG-1,TCACGCTAGCTTGTTG-1,1.9295983379944566,15.0,-0.06831844721252084,-0.023258220815318944,8.287484,-7.3376813,11,G1,977.9713559746742,7,6,8617.0,3021.0,9.156318904491123,16.21213879540443,0.2550873974521062,False,False,4.5,14.770875,-6.873608,1894.0,9.0,7,1.204322462390436
-625,True,TCACGCTAGGTTACAA-1,TCACGCTAGGTTACAA-1,1.917598370403247,15.0,-0.11697190679436241,-0.02246702580766679,8.767438,-2.447984,22,G1,1082.6714586913586,6,8,6696.0,1895.0,9.214456391875746,40.59139784946237,0.4526181719319594,False,False,4.199999999999999,14.087333,0.79129505,707.0,9.0,6,1.224782928867908
-627,True,TCACGCTAGTCTTGGT-1,TCACGCTAGTCTTGGT-1,1.8558379357477308,15.0,-0.0692642745552534,-0.012321931323616392,10.518392,-6.497881,23,G1,1139.1971760690212,6,8,5583.0,2112.0,6.591438294823572,22.246104245029553,0.21041616705454178,False,False,5.800000000000001,14.226497,-5.623344,1564.0,9.0,6,0.8054144824310264
-628,True,TCACGCTCAAGAGTAT-1,TCACGCTCAAGAGTAT-1,1.952266103973654,15.0,-0.09151796129347158,-0.019058812591293044,-6.535925,-7.4793057,13,G1,1186.745100170374,2,2,16141.0,4043.0,7.973483675113066,19.39780682733412,0.9497117703169925,False,False,3.4,-16.34276,-7.2142253,2635.0,9.0,2,1.0662710483703537
-629,True,TCACGCTCAAGCCATT-1,TCACGCTCAAGCCATT-1,1.9417782664019234,15.0,-0.10417431173726105,0.006501996768671518,-3.1189823,13.856475,2,S,1151.8440171182156,5,5,5923.0,1762.0,8.678034779672464,37.41347290224548,0.7348773420002048,False,False,7.300000000000001,1.8395907,14.770291,1230.0,9.0,5,0.8054144824310264
-630,True,TCACGCTCAATCTGCA-1,TCACGCTCAATCTGCA-1,1.8984707797511062,15.0,-0.1094967910917838,-0.018822807808201263,7.747655,3.5896783,8,G1,1443.2493682354689,9,9,9051.0,2136.0,4.364158656502044,48.558170367915146,0.6092311620882579,False,False,4.6000000000000005,12.745442,6.0176916,1453.0,9.0,9,1.3141759901229113
-631,True,TCACGCTCACGCGCAT-1,TCACGCTCACGCGCAT-1,1.8688515322456323,15.0,-0.05465622822248078,-0.0167632394739267,8.283907,1.8873179,8,G1,1200.660037189722,9,9,8158.0,2798.0,8.936013728855112,22.309389556263792,0.5865466855217216,False,False,3.8000000000000003,13.770477,4.6235995,912.0,9.0,9,0.8054144824310264
-632,True,TCACGCTCAGACCTAT-1,TCACGCTCAGACCTAT-1,1.962214653271303,15.0,-0.07175594726235997,-0.03103892953169236,-7.1814423,-3.7455363,10,G1,1551.9388055950403,3,3,9351.0,3144.0,8.352047909314512,11.014864720350765,0.9702417474092274,False,True,2.9,-14.4363365,-3.954416,2827.0,9.0,3,1.609387040090457
-633,True,TCACGCTGTAATGCTC-1,TCACGCTGTAATGCTC-1,1.8917791795721983,15.0,-0.10519531707732785,-0.011351542282766337,9.372717,0.6513048,25,G1,1017.8341751247644,6,9,14442.0,3030.0,6.141808613765407,44.17670682730924,0.5730458820512577,False,False,3.9999999999999996,13.579727,3.9031665,2490.0,9.0,6,0.8054144824310264
-634,True,TCACGCTGTAGAATGT-1,TCACGCTGTAGAATGT-1,1.8636722261231293,15.0,-0.08966961231343117,-0.023263776639935483,9.217827,-1.5559161,25,G1,1069.0836197584867,6,8,5320.0,1825.0,7.293233082706767,31.93609022556391,0.48043261311745494,False,True,3.4,13.87444,1.517037,654.0,9.0,6,0.9413669634110602
-635,True,TCACGCTGTCAAGCCC-1,TCACGCTGTCAAGCCC-1,1.9083080410775757,15.0,-0.10433090146984136,0.0005656873589959949,4.2985473,4.856627,12,S,1853.3862182945013,1,1,5394.0,1491.0,6.3589173155357805,43.65962180200223,0.6104745356767937,False,False,5.8,10.889554,8.2876005,2058.0,9.0,1,0.8054144824310264
-636,True,TCACGCTGTCACGCTG-1,TCACGCTGTCACGCTG-1,1.960140040102747,15.0,-0.08536591248924275,-0.005447677980111584,1.7747018,16.22458,2,G1,1027.7385633736849,5,5,5686.0,1848.0,10.288427717200141,35.262047133309885,0.7183090376603622,False,False,6.6000000000000005,-2.614674,14.514374,815.0,9.0,5,1.0057120305185743
-638,True,TCACGCTGTTCGGCTG-1,TCACGCTGTTCGGCTG-1,2.1258546835451124,15.0,-0.015836319610745643,-0.04032383138434816,-8.188996,2.2816377,14,G1,1291.507243514061,4,4,1252.0,657.0,18.929712460063897,3.2747603833865813,0.9368169472494425,False,False,3.7,-13.461947,0.78384954,1452.0,9.0,4,1.4186430005075945
-639,True,TCACGCTGTTTGATCG-1,TCACGCTGTTTGATCG-1,1.8436882688256773,15.0,-0.08463564517246266,-0.016615330478779243,8.283134,4.271799,26,G1,1379.3460582569242,9,9,3211.0,1327.0,4.1420118343195265,34.25724073497353,0.610683651748799,False,False,4.3999999999999995,11.175912,6.1369376,970.0,9.0,9,1.4960005175783853
-640,True,TCACGCTTCCACGTAA-1,TCACGCTTCCACGTAA-1,2.012815984319007,15.0,-0.037597914368494155,-0.03428053722499051,-0.31822318,8.333308,5,G1,1276.149196445942,1,1,2379.0,1063.0,26.944094157208912,7.608238755779739,0.6250809681324047,False,False,5.7,8.998982,13.171636,899.0,10.0,1,0.9996232605453769
-641,True,TCACGCTTCGAACGCC-1,TCACGCTTCGAACGCC-1,1.909177500977344,15.0,-0.07979266758936981,-0.02429935140142176,0.3332711,7.3696394,5,G1,1577.436096638441,1,1,4454.0,1501.0,6.668163448585541,37.49438706780422,0.619000727623201,False,False,5.2,9.313559,12.228039,1205.0,9.0,1,1.2283639736538412
-643,True,TCCAGAAAGACCAGAC-1,TCCAGAAAGACCAGAC-1,1.9671669679117965,15.0,-0.11353691683342047,-0.012261029454732683,-1.8289365,15.900776,2,G1,1238.1832092776895,5,5,6527.0,1792.0,8.043511567335683,40.32480465757622,0.7320123422731535,False,False,7.7,0.33985654,14.037793,1711.0,9.0,5,1.559218234163678
-644,True,TCCAGAAAGGGAGGTG-1,TCCAGAAAGGGAGGTG-1,1.9578764122202192,15.0,-0.07879975530602298,-0.01667196662465959,-5.8836956,-7.1240983,3,G1,1400.6363264769316,2,2,12365.0,3454.0,9.640113222806308,18.803073190456935,0.9539930412932003,False,False,4.0,-14.618614,-7.2080727,2729.0,9.0,2,1.5319548011191668
-645,True,TCCAGAAAGGTTGGAC-1,TCCAGAAAGGTTGGAC-1,1.938653331955835,15.0,-0.09216120237900936,-0.01414828275912917,2.6460013,8.381447,5,G1,1443.043089300394,1,1,6653.0,1593.0,10.43138433789268,46.309935367503385,0.6122619410711685,False,False,3.1,7.8777804,10.73781,1169.0,9.0,1,0.8054144824310264
-646,True,TCCAGAAAGTAGTCTC-1,TCCAGAAAGTAGTCTC-1,1.9916160105803282,15.0,-0.10376633590206912,-0.006064753843304488,2.3826606,18.708853,18,G1,1213.2228570729494,5,5,4955.0,1260.0,9.8284561049445,45.18668012108981,0.7230214252264942,False,False,8.6,-4.2176175,15.082658,914.0,9.0,5,0.9587819215788999
-647,True,TCCAGAAAGTGGCCTC-1,TCCAGAAAGTGGCCTC-1,1.954206815115506,15.0,-0.09373638509279127,-0.02735922612548038,-8.988244,-8.687489,10,G1,1217.1231657117605,3,7,14404.0,3767.0,7.4076645376284365,18.994723687864482,0.9672262047850572,False,False,3.3,-13.471094,-9.081588,2558.0,9.0,3,0.8054144824310264
-649,True,TCCAGAAGTATACCTG-1,TCCAGAAGTATACCTG-1,1.903186077016388,15.0,-0.097704007739599,-0.009026670163672576,8.680523,-10.458825,21,G1,1304.3006233423948,7,6,4881.0,1887.0,8.809670149559517,22.249539028887522,0.06566872208947795,False,False,7.5,12.618439,-10.119822,1036.0,9.0,7,1.4355230139629744
-650,True,TCCAGAAGTCACGCTG-1,TCCAGAAGTCACGCTG-1,1.9572352480394377,15.0,-0.06952807388472185,-0.022515713445253563,-6.2285995,-8.825859,3,G1,1223.0178946256638,2,2,10281.0,3206.0,9.678046882598968,13.578445676490613,0.9532483018940371,False,False,4.0,-15.585749,-8.358909,3435.0,9.0,2,0.8054144824310264
-651,True,TCCAGAAGTCACTCGG-1,TCCAGAAGTCACTCGG-1,1.8650512867665967,15.0,-0.07362575234719484,-0.016134839281838995,8.839259,-14.046198,21,G1,1097.0887193381786,7,6,4086.0,1839.0,10.15663240332844,14.464023494860498,0.00575559420795318,False,False,7.0,10.032454,-11.33612,1589.0,9.0,7,1.214596760873807
-652,True,TCCAGAAGTCTAGGTT-1,TCCAGAAGTCTAGGTT-1,2.018551011489666,15.0,-0.07512960332076651,-0.03440862582959197,-4.158024,-1.0393155,20,G1,1380.711725756526,4,4,7054.0,2481.0,12.673660334561951,12.999716472923165,0.9557956457188042,False,False,2.5,-14.918872,-0.76789075,2188.0,9.0,4,0.8054144824310264
-653,True,TCCAGAAGTTAAGTCC-1,TCCAGAAGTTAAGTCC-1,1.9910924971871835,15.0,-0.08917010411829576,-0.03159354519438886,-9.643506,-4.760633,13,G1,1217.815705358982,2,3,10937.0,3315.0,8.676968089969828,11.813111456523727,0.9828030493649047,False,False,3.3,-17.817225,-5.0195236,2219.0,9.0,2,1.0240089423977876
-654,True,TCCAGAATCATGCCCT-1,TCCAGAATCATGCCCT-1,1.8954514134437437,15.0,-0.0929413257637359,-0.0007940169568589686,7.6231637,2.8512356,8,G1,1364.1116490364075,9,9,7744.0,2151.0,7.541322314049586,37.93904958677686,0.6022683540136744,False,False,3.2999999999999994,12.904878,5.889982,1967.0,9.0,9,1.225133459151207
-656,True,TCCAGAATCGTAGAGG-1,TCCAGAATCGTAGAGG-1,1.962747883194215,15.0,-0.11401272039655382,-0.018218124723560457,-5.831869,-9.005712,3,G1,1136.7000368535519,2,2,17368.0,3771.0,9.269921695071396,27.17065868263473,0.9182308512756282,False,False,3.6,-15.260987,-8.353245,1571.0,9.0,2,1.072176509407569
-657,True,TCGAAGTAGCAATTAG-1,TCGAAGTAGCAATTAG-1,1.9142504941346463,15.0,-0.09306017224591881,-0.0011448702063032272,4.5495787,3.400083,6,G1,1542.6872509717941,1,1,5815.0,1698.0,8.684436801375753,38.194325021496134,0.6123457894354274,False,False,3.7,12.15024,8.964586,1077.0,9.0,1,0.8054144824310264
-658,True,TCGAAGTAGCCTCTGG-1,TCGAAGTAGCCTCTGG-1,1.9576220762778798,15.0,-0.11516109029266706,-0.00217381924561226,2.592434,19.485168,18,G1,1178.422138467431,5,5,8422.0,2142.0,7.967228686772738,41.771550700546186,0.7228105467532835,False,False,8.7,-4.423855,14.877431,1040.0,9.0,5,1.0082126749131124
-659,True,TCGAAGTAGGGAGGTG-1,TCGAAGTAGGGAGGTG-1,1.9095473483024348,15.0,-0.0661230813162079,-0.029031545509051324,12.019052,-8.318773,11,G1,1163.4477026462555,7,6,4615.0,1685.0,10.617551462621885,23.640303358613217,0.1699325442630335,False,False,7.2,13.137146,-6.6960993,1372.0,9.0,7,1.2307612572131277
-660,True,TCGAAGTCAAGAGTAT-1,TCGAAGTCAAGAGTAT-1,1.9783141262982273,15.0,-0.06533020095886305,-0.025355717379881616,-6.6145425,0.3183884,19,G1,1559.392406836152,2,4,9609.0,3060.0,10.56301384119055,13.050265376209802,0.9612092251506734,False,False,3.8999999999999995,-14.595691,-0.19346547,2444.0,9.0,2,1.3220812003669467
-661,True,TCGAAGTCAATCTCGA-1,TCGAAGTCAATCTCGA-1,2.015697292783301,15.0,-0.06056201550387597,-0.04719525350593312,9.20022,10.491923,15,G1,923.0555487126112,8,10,576.0,390.0,14.756944444444445,6.770833333333333,0.6266298216159808,False,False,4.5,15.438016,11.267436,879.0,9.0,8,0.8054144824310264
-662,True,TCGAAGTCACTACCGG-1,TCGAAGTCACTACCGG-1,2.000728791398067,15.0,-0.0724778671032713,-0.026515023578759577,-4.880422,-3.9350815,17,G1,1787.0381335318089,2,2,14003.0,3596.0,9.719345854459759,15.41812468756695,0.9632898205706526,False,False,5.2,-15.248407,-3.9588563,2795.0,9.0,2,1.8189474974340007
-663,True,TCGAAGTCATCTCCCA-1,TCGAAGTCATCTCCCA-1,1.9969715342896053,15.0,-0.06641693897611647,-0.019525290409677737,-5.168773,-1.9871955,20,G1,1194.5363852232695,4,4,12547.0,3583.0,11.74782816609548,12.4093408782976,0.9620218194993418,False,False,3.4999999999999996,-16.502924,-1.695164,4685.0,9.0,4,1.2828164428263784
-664,True,TCGAAGTCATTATGCG-1,TCGAAGTCATTATGCG-1,2.0017895851902914,15.0,-0.08718142454447704,-0.028609049996148786,-8.29914,-3.910493,10,G1,1203.6675989627838,3,3,2647.0,1185.0,19.304873441632036,10.011333585190782,0.9611119309724508,False,False,3.4,-12.756589,-3.9409451,2461.0,9.0,3,0.8054144824310264
-665,True,TCGAAGTGTCGTGCCA-1,TCGAAGTGTCGTGCCA-1,1.9002038975433235,15.0,-0.09260646131949266,-0.011750756394079144,9.132883,0.88479084,25,G1,1057.9851561188698,6,8,5697.0,1578.0,8.987186238371072,41.583289450588026,0.5724357493804056,False,False,3.7,14.52157,4.3227115,1887.0,9.0,6,0.8054144824310264
-666,True,TCGAAGTGTTAAGTCC-1,TCGAAGTGTTAAGTCC-1,1.9581467258610044,15.0,-0.11575102614331105,-0.016375943738647925,-2.6423528,14.797153,2,G1,1261.1751947253942,5,5,7408.0,1926.0,8.57181425485961,41.68466522678186,0.7318645753897091,False,False,7.7,1.2543212,14.202694,1134.0,9.0,5,1.1494006263450627
-668,True,TCGAAGTTCCATTGCC-1,TCGAAGTTCCATTGCC-1,1.9131149369327187,15.0,-0.1021042767540304,-0.0022509083619328893,7.99233,3.2500844,8,G1,1479.69571647048,9,9,9407.0,2231.0,7.345593706814075,43.371957053258214,0.6060463991473884,False,False,4.6,12.742836,6.085818,744.0,9.0,9,1.6225209498318787
-669,True,TCGAAGTTCCCAAGCG-1,TCGAAGTTCCCAAGCG-1,2.001869844770349,15.0,-0.06227077772769557,-0.013176373797655894,-3.2197552,-3.6598146,19,G1,1611.394123479724,2,2,11077.0,3103.0,10.318678342511511,16.737383768168275,0.9518225742479185,False,False,3.9999999999999996,-14.172669,-3.8595345,1749.0,9.0,2,1.5626794437614846
-670,True,TCGAAGTTCCTGCTAC-1,TCGAAGTTCCTGCTAC-1,2.0096767830436675,15.0,-0.061887638107150304,-0.03238751758761231,-5.2481937,0.2756698,4,G1,1526.6131178736687,4,4,5904.0,2342.0,16.073848238482384,10.060975609756097,0.9551937846575944,False,False,3.5999999999999996,-15.643944,0.25495446,1234.0,9.0,4,0.8054144824310264
-671,True,TCGAAGTTCTCACTCG-1,TCGAAGTTCTCACTCG-1,1.9790515183943522,15.0,-0.08357898465763129,-0.006456082864719697,-9.685635,-2.7006888,7,G1,1650.723553493619,3,3,11608.0,3236.0,6.874569262577532,14.19710544452102,0.993364325254691,False,False,5.6,-16.081768,-3.3927326,2230.0,9.0,3,1.8409411948210852
-672,True,TCGCTCAAGACGTCCC-1,TCGCTCAAGACGTCCC-1,1.9395417124089476,15.0,-0.09385538323093848,-0.015406723961333498,-4.695982,-11.585086,24,G1,1232.1851411163807,10,7,15523.0,3767.0,9.695290858725762,21.168588546028474,0.9619650459054542,False,False,4.400000000000001,-8.383047,-11.058127,2844.0,9.0,10,0.9962928677473688
-673,True,TCGCTCAAGATACCAA-1,TCGCTCAAGATACCAA-1,1.9485850268717886,15.0,-0.09436240659567915,-0.031114155875754168,-7.680683,-8.824037,3,G1,1238.53900629282,2,2,12370.0,3598.0,9.636216653193209,12.732417138237672,0.9579377055391868,False,False,3.9999999999999996,-14.338801,-8.451429,3087.0,9.0,2,1.3600913283917158
-674,True,TCGCTCAAGGATCACG-1,TCGCTCAAGGATCACG-1,1.8836556668506712,15.0,-0.10304514102279431,0.003789515169429228,4.6296215,6.688089,6,S,1551.5108725130558,1,1,5551.0,1649.0,5.5665645829580255,41.77625653035489,0.6124705495319507,False,False,3.6,10.415406,9.107108,1871.0,9.0,1,1.2197254860671147
-676,True,TCGCTCAAGTGTTGAA-1,TCGCTCAAGTGTTGAA-1,2.0199891448432687,15.0,-0.0818180911727505,-0.019644481840486373,-7.5972724,1.5781083,14,G1,1481.3613579571247,4,4,6399.0,1934.0,15.783716205657134,21.456477574621033,0.9524045432428312,False,False,4.1,-13.965091,-0.15573514,2268.0,9.0,4,1.5247417702332517
-677,True,TCGCTCACAATCTCGA-1,TCGCTCACAATCTCGA-1,1.9164413078043057,15.0,-0.10201732364444481,0.016263038282517912,6.478319,1.3585752,22,S,1310.566964611411,6,8,5623.0,1645.0,10.759381113284723,35.51484972434643,0.5826036892839831,False,True,3.3000000000000003,13.689241,6.2164187,1370.0,9.0,6,1.0197972111781748
-678,True,TCGCTCACACAAATCC-1,TCGCTCACACAAATCC-1,1.9852890712440843,15.0,-0.07646879043523469,-0.03245260567379723,-4.888874,-5.9145927,3,G1,1481.2204222232103,2,2,7180.0,2570.0,12.994428969359332,14.275766016713092,0.9579855244904801,False,False,3.8000000000000003,-14.941083,-5.9198976,2375.0,9.0,2,1.0158687321103217
-679,True,TCGCTCACACACCGCA-1,TCGCTCACACACCGCA-1,1.999204376925453,15.0,-0.058879037154677336,-0.011391860194289102,-12.018182,-3.5038552,7,G1,1007.3005505949259,3,3,6469.0,2435.0,11.284588035245015,10.61987942494976,1.0,False,False,5.0,-18.617264,-3.765828,1437.0,9.0,3,1.3173431684307069
-680,True,TCGCTCACACCCTCTA-1,TCGCTCACACCCTCTA-1,1.9462624274374065,15.0,-0.07382400620499448,-0.02416847470426486,-10.089146,-7.669301,10,G1,1105.5173516124487,3,7,13574.0,3631.0,6.954471784293502,19.640489170472964,0.9735823773914273,False,False,4.3,-12.637971,-8.288453,3433.0,9.0,3,1.6205254676928134
-681,True,TCGCTCACAGTTTCGA-1,TCGCTCACAGTTTCGA-1,1.8915648650614891,15.0,-0.09822328738044928,-0.016743998254108945,2.6616688,7.17605,5,G1,1477.4511660709977,1,1,5652.0,1535.0,5.201698513800425,46.77990092002831,0.6123430291435416,False,False,2.8000000000000003,8.828321,10.089671,970.0,9.0,1,1.1621394467763864
-683,True,TCGCTCAGTGCACAAG-1,TCGCTCAGTGCACAAG-1,2.006497646816203,15.0,-0.07879427002602445,-0.017901833013694555,-4.1339726,-4.0310006,17,G1,1740.5769593566656,2,2,8073.0,2700.0,15.384615384615385,9.537966059705191,0.9608805594016727,False,False,6.0,-14.290732,-4.621388,3458.0,9.0,2,0.8054144824310264
-684,True,TCGCTCAGTTAAGCAA-1,TCGCTCAGTTAAGCAA-1,1.9700584801713001,15.0,-0.07325021632501696,-0.026615847539000412,-7.657188,-2.0054142,9,G1,1325.4932126104832,2,2,12272.0,3472.0,11.3754889178618,21.789439374185136,0.971923813219599,False,False,2.8,-16.554491,-2.907848,2817.0,9.0,2,1.0305418786578064
-685,True,TCGCTCATCCTACAAG-1,TCGCTCATCCTACAAG-1,1.9785556257865555,15.0,-0.10125969936482485,-0.007976703808120569,1.6027129,15.568839,2,G1,1073.5921186804771,5,5,3947.0,1450.0,11.933113757284014,29.896123638206234,0.7192258740665514,False,False,9.0,-2.1932206,14.391827,923.0,9.0,5,1.1006156182254136
-686,True,TCGCTCATCCTTCACG-1,TCGCTCATCCTTCACG-1,1.973522896201295,15.0,-0.08369975095855868,-0.025903591671914604,-6.7747817,-4.5814643,13,G1,972.1545389294624,2,2,7737.0,2786.0,12.252811167119038,13.532376890267546,0.9587773208116617,False,False,2.7,-15.998331,-5.0962353,1726.0,9.0,2,1.0003902924206178
-687,True,TGCTCGTAGCACCTGC-1,TGCTCGTAGCACCTGC-1,1.8579265201413198,15.0,-0.10467112089784639,-0.035214631880570134,-2.7362394,12.5479765,2,G1,926.105884462595,5,5,13684.0,3697.0,3.7489038292896812,25.285004384682843,0.7028736497788481,False,False,3.8000000000000003,3.0671618,13.237379,1453.0,9.0,5,1.2201613949944972
-688,True,TGCTCGTAGCATTGAA-1,TGCTCGTAGCATTGAA-1,1.9046380056557495,15.0,-0.09523622334719847,-0.014615991451283568,8.05597,10.511884,15,G1,1107.310365691781,8,10,5643.0,1599.0,5.954279638490164,42.79638490164806,0.6297563560813222,False,False,6.3,15.06465,10.066391,1389.0,9.0,8,0.8054144824310264
-689,True,TGCTCGTAGCCTCTTC-1,TGCTCGTAGCCTCTTC-1,2.2012553001677158,15.0,-0.07729852691092999,-0.011064088733991647,-8.425517,-11.59289,16,G1,667.5909079015255,2,2,702.0,260.0,13.532763532763532,3.988603988603989,0.9361578595849888,False,True,5.5,-15.562806,-10.768108,502.0,9.0,2,0.9485778766191572
-690,True,TGCTCGTAGGCCTAGA-1,TGCTCGTAGGCCTAGA-1,1.8787996298189602,15.0,-0.0973596903829462,-0.008228954137414608,5.1466784,2.4985347,6,G1,1722.5439319610596,1,1,5180.0,1547.0,5.945945945945946,42.181467181467184,0.6028290183325881,False,False,4.2,12.601604,7.737241,775.0,9.0,1,1.5043685629660082
-692,True,TGCTCGTAGTATAACG-1,TGCTCGTAGTATAACG-1,1.9029590986195781,15.0,-0.10208298418587183,-0.009544096901998844,2.575842,5.2503543,12,G1,1693.3400313705206,1,1,6554.0,1710.0,7.155935306682942,45.69728410131218,0.6123075764130359,False,False,3.0999999999999996,11.253841,10.7593775,1036.0,9.0,1,0.8054144824310264
-693,True,TGCTCGTAGTGGCCTC-1,TGCTCGTAGTGGCCTC-1,1.9527954111443704,15.0,-0.0934065169378935,-0.023240545455601846,1.6277683,7.7308974,5,G1,1644.9857279360294,1,1,3342.0,998.0,15.320167564332735,36.65469778575703,0.6138171769359551,False,False,5.8999999999999995,8.81556,11.399123,1009.0,9.0,1,1.185433140001456
-694,True,TGCTCGTCAACGCATT-1,TGCTCGTCAACGCATT-1,1.9764199496084418,15.0,-0.10814717703308083,-0.018484488885105763,1.1048962,19.067942,18,G1,1236.2221630066633,5,5,5540.0,1727.0,10.667870036101084,36.35379061371841,0.7243678254301371,False,False,7.2,-4.5346007,15.811884,1484.0,9.0,5,0.8054144824310264
-696,True,TGCTCGTCACCGTGAC-1,TGCTCGTCACCGTGAC-1,2.017170651269734,15.0,-0.08169076698592634,-0.038972218980803656,-11.237457,-2.5500104,14,G1,1364.8111157864332,4,3,6763.0,2642.0,11.932574301345557,10.217359160136034,0.994136009278356,False,False,4.0,-17.435148,-2.7646334,2525.0,9.0,4,0.8054144824310264
-697,True,TGCTCGTCATGAATAG-1,TGCTCGTCATGAATAG-1,1.9623875537805024,15.0,-0.07235568731138896,-0.02553361979556653,-11.752244,-2.450853,7,G1,1144.3104189932346,3,3,10623.0,3377.0,10.11955191565471,12.830650475383601,0.998351773163012,False,True,4.0,-18.351603,-2.8916817,3126.0,9.0,3,1.1935534825687564
-698,True,TGCTCGTGTACTGAGG-1,TGCTCGTGTACTGAGG-1,1.9858896821608998,15.0,-0.07411883945372134,-0.009942708842368295,-5.8816123,-6.8266935,25,G1,849.8574360609055,6,2,7649.0,2699.0,14.537848084716957,15.07386586481893,0.9155101123985143,False,True,3.4000000000000004,-16.259775,-7.330807,2182.0,9.0,6,0.899061679436515
-699,True,TGCTCGTGTCAAGCCC-1,TGCTCGTGTCAAGCCC-1,1.8748626623162896,15.0,-0.09138556262179273,-0.014298765448484407,6.425994,-1.1642622,22,G1,1049.7662628293037,6,8,5225.0,1654.0,5.645933014354067,40.17224880382775,0.5205849077250941,False,False,4.199999999999999,14.694947,2.7314882,1632.0,9.0,6,0.9895195640099048
-700,True,TGCTCGTGTCTAGGTT-1,TGCTCGTGTCTAGGTT-1,2.054112133552775,15.0,-0.0640941049821096,-0.036757635508340034,-5.275005,3.516658,14,G1,1464.3138327896595,4,4,4582.0,1903.0,9.602793539938892,10.650371017023135,0.9385021061215706,False,False,5.699999999999999,-14.6442995,1.6346612,1597.0,9.0,4,0.8054144824310264
-701,True,TGCTCGTGTTCCACGG-1,TGCTCGTGTTCCACGG-1,2.023594136425409,15.0,-0.07564997548061698,-0.024741966661985977,-4.052011,0.07990292,14,G1,1440.4397929161787,4,4,11260.0,3313.0,8.943161634103019,17.841918294849023,0.9472986526985316,False,False,4.699999999999999,-14.706867,-0.23539524,1189.0,9.0,4,1.0506509094123777
-702,True,TGCTCGTGTTGGGATG-1,TGCTCGTGTTGGGATG-1,1.9134716533111513,15.0,-0.09999172554141952,-0.0024481513064159525,1.6725556,5.78431,5,G1,1779.306372821331,1,1,5722.0,1418.0,7.829430269136665,49.108703250611676,0.61331692087955,False,False,6.6,9.656051,10.708947,1647.0,9.0,1,1.4837678568491623
-703,True,TGCTCGTTCATGCCCT-1,TGCTCGTTCATGCCCT-1,1.881934595308163,15.0,-0.11252771860175467,-0.010632253407736582,5.493402,4.9906754,12,G1,1506.100800678134,1,1,6979.0,1763.0,4.585184123799971,45.794526436452216,0.6079674215041945,False,False,2.9,11.018494,7.689675,910.0,9.0,1,1.0613058845862255
-704,True,TGCTCGTTCCGAGAAG-1,TGCTCGTTCCGAGAAG-1,1.9029217099536027,15.0,-0.09267053623989505,-0.014833936924749738,11.21525,-6.7385354,11,G1,1160.5451955497265,7,6,6573.0,1938.0,9.736802069070439,33.91145595618439,0.22840552035639494,False,False,4.6000000000000005,13.62536,-5.5181084,1464.0,9.0,7,0.8054144824310264
-705,True,TGCTCGTTCGTCGGGT-1,TGCTCGTTCGTCGGGT-1,1.9369948068584733,15.0,-0.08932438612635764,-0.02553557796394498,8.273165,-7.599273,11,G1,992.153044089675,7,6,12315.0,3459.0,8.932196508323184,19.53714981729598,0.2539868940414648,False,False,4.2,14.435333,-7.403716,1448.0,9.0,7,1.2294035465817903
-706,True,TGCTCGTTCGTTTACT-1,TGCTCGTTCGTTTACT-1,1.9277159086446656,15.0,-0.07036862584310155,-0.0184053210417798,-3.7497492,-11.8405,24,G1,1286.8815755099058,10,7,11728.0,3299.0,8.995566166439291,17.573328785811732,0.9600926057890252,False,False,6.299999999999999,-7.61334,-11.685133,1441.0,9.0,10,1.1476447307389888
-707,True,TGGTACAAGACGTCGA-1,TGGTACAAGACGTCGA-1,2.004295999361444,15.0,-0.07562541684067492,-0.02299019299881922,-5.9322824,-2.6855078,9,G1,1719.1389792710543,2,2,9184.0,2770.0,11.770470383275262,19.52308362369338,0.964511877220525,False,False,3.9999999999999996,-15.622446,-2.7776437,2082.0,9.0,2,1.7048563686127152
-708,True,TGGTACAAGCACTCTA-1,TGGTACAAGCACTCTA-1,2.0643182157987736,15.0,-0.06740167147916143,-0.029723474391037633,-5.4121566,3.4130397,14,G1,1457.841810375452,4,4,5797.0,1973.0,12.937726410212179,14.352251164395376,0.9282918028979709,False,False,6.0,-14.124995,1.4777582,781.0,9.0,4,1.895003129573888
-709,True,TGGTACAAGCAGCCCT-1,TGGTACAAGCAGCCCT-1,1.9026842717308456,15.0,-0.10187769054590697,-0.001048259636526199,3.9137306,6.1722093,12,G1,1354.2896008491516,1,1,7233.0,1958.0,7.286050048389327,40.32904742154016,0.6119232935801731,False,False,2.5000000000000004,10.355409,9.417555,1499.0,9.0,1,0.9803736437095918
-710,True,TGGTACAAGGTCTACT-1,TGGTACAAGGTCTACT-1,1.8911326893078662,15.0,-0.07767227803005977,-0.028746873407067573,10.118851,-12.190447,21,G1,1217.8547803610563,7,6,3900.0,1627.0,10.23076923076923,17.46153846153846,0.028254068653785484,False,False,5.0,11.34842,-11.202848,1267.0,9.0,7,0.8054144824310264
-711,True,TGGTACACAAGTCGTT-1,TGGTACACAAGTCGTT-1,1.9103384377242905,15.0,-0.13589125098307445,-0.016662831306742443,3.7740633,3.0617135,12,G1,1586.2375886291265,1,1,5563.0,1304.0,4.2782671220564445,51.86050692072623,0.6135241267667881,False,False,4.7,12.616341,9.805595,1417.0,9.0,1,1.0211365684291407
-712,True,TGGTACACAATCAAGA-1,TGGTACACAATCAAGA-1,1.9062022761246855,15.0,-0.06721210176296277,-0.017700977226286645,9.301143,-10.953219,21,G1,1296.3735474646091,7,6,4192.0,1667.0,12.95324427480916,18.606870229007633,0.04739248647864181,False,False,6.6,11.804786,-10.518646,961.0,9.0,7,1.890189192948571
-713,True,TGGTACACATGGCTAT-1,TGGTACACATGGCTAT-1,1.9939949435103408,15.0,-0.06535287668070865,-0.016464587714196233,-5.233852,-1.9723355,9,G1,1663.3033603727818,2,2,15500.0,3830.0,10.019354838709678,16.36774193548387,0.9603734595332298,False,False,2.7,-14.778332,-2.5045917,4188.0,10.0,2,1.5302818959712985
-714,True,TGGTACACATGTTCGA-1,TGGTACACATGTTCGA-1,2.0816411151407244,15.0,-0.054978283577986696,0.0077873192932065086,10.477175,2.6624699,25,S,808.9151745140553,6,9,564.0,355.0,28.54609929078014,6.560283687943262,0.6328648247165117,False,True,2.0,11.687203,3.3552747,2381.0,9.0,6,0.8054144824310264
-715,True,TGGTACAGTAACTTCG-1,TGGTACAGTAACTTCG-1,1.944688244505543,15.0,-0.0718212767220713,-0.024312997577003663,-8.757403,-5.6888995,10,G1,1309.0606059283018,3,3,9521.0,2998.0,9.515807163113118,15.51307635752547,0.9688419358431407,False,False,3.8000000000000003,-12.672276,-6.2802315,2813.0,9.0,3,1.0999347589582273
-716,True,TGGTACAGTCGTGCCA-1,TGGTACAGTCGTGCCA-1,1.9743895223305992,15.0,-0.1180362110078446,-0.013022978013333922,1.1733916,15.332442,2,G1,1106.8139469772577,5,5,4505.0,1467.0,10.366259711431743,36.958934517203105,0.7195504465318351,False,False,9.200000000000001,-1.959967,14.826358,560.0,9.0,5,1.248793220324594
-717,True,TGGTACAGTTAAGTCC-1,TGGTACAGTTAAGTCC-1,2.0052163478038985,15.0,-0.08232316880676963,-0.029941410615868413,-6.121143,-0.07703727,19,G1,1549.5485635846853,2,4,6970.0,2489.0,16.499282639885223,6.829268292682927,0.9664943867705973,False,False,3.3,-14.506975,-1.1525476,2504.0,9.0,2,1.0796593453050356
-718,True,TGGTACATCACAAGAA-1,TGGTACATCACAAGAA-1,1.9087888615193525,15.0,-0.07991571616344173,-0.015331485282082164,4.5097537,3.3156636,12,G1,1812.7740164548159,1,1,5253.0,1568.0,9.308966304968589,38.14962878355226,0.6111591756128197,False,False,5.500000000000001,12.351362,9.53583,1436.0,9.0,1,1.7382680596526896
-719,True,TGGTACATCCTGCTAC-1,TGGTACATCCTGCTAC-1,1.8876826057782135,15.0,-0.10865942957313936,-0.015512719646262219,4.648127,4.322025,6,G1,1702.2992860525846,1,1,6193.0,1731.0,5.603100274503472,43.72678830938156,0.6119303235052337,False,False,2.9,11.735623,9.060129,1099.0,9.0,1,1.4065931698728518
-720,True,TGGTACATCCTTCACG-1,TGGTACATCCTTCACG-1,1.9974978974785114,15.0,-0.08004887991688274,-0.011995778393580799,-9.755837,-3.6560767,10,G1,1269.9728217571974,3,3,9373.0,2957.0,6.828123332977702,8.791208791208792,0.9850314569033785,False,False,3.8000000000000003,-13.145911,-5.1307683,2334.0,9.0,3,1.3905506255957503
-721,True,TGGTACATCGCGGTAC-1,TGGTACATCGCGGTAC-1,1.9861702677708417,15.0,-0.08086381434844428,-0.0170025602232003,-3.921139,-3.8080227,19,G1,1563.0256024301052,2,2,7706.0,2315.0,9.680768232546068,15.792888658188424,0.9503294105291532,False,False,4.0,-13.875796,-4.3224316,784.0,9.0,2,1.2764001638668243
-722,True,TGGTACATCGTAGAGG-1,TGGTACATCGTAGAGG-1,2.0379557063454006,15.0,-0.044605027775162405,-0.021193761101783587,-5.155376,-5.1119237,3,G1,1101.6976715177298,2,2,3800.0,1660.0,21.86842105263158,11.078947368421053,0.9498818601705068,False,False,2.4,-17.015408,-5.9378247,1734.0,9.0,2,0.8054144824310264
-723,True,TGGTACATCTGTGCGG-1,TGGTACATCTGTGCGG-1,1.8860542898113724,15.0,-0.1084213719553249,-0.02642675688499331,6.8260283,-2.3086681,22,G1,1128.3589890897274,6,8,10194.0,2540.0,7.318030213851285,40.955463998430446,0.4796580700076769,False,False,6.1,14.870863,1.4870076,2343.0,9.0,6,0.8054144824310264
-724,True,TGGTACATCTTAGCCC-1,TGGTACATCTTAGCCC-1,1.917364680268752,15.0,-0.07432804270267496,-0.02057432293736497,-3.411143,-12.941483,24,G1,1334.5839346796274,10,7,11723.0,2862.0,8.410816343939265,18.015866245841508,0.9594457652444184,False,True,7.0,-7.72156,-11.5654745,2027.0,9.0,10,1.357837327667586
-725,True,TGTCCTGAGACCAGAC-1,TGTCCTGAGACCAGAC-1,1.9440843478251237,15.0,-0.08817014152731359,-0.02356311469871993,-2.1106577,-4.3707504,19,G1,1251.6893403232098,2,2,12461.0,3445.0,7.655886365460236,27.975282882593692,0.9505496029097,False,True,3.4000000000000004,-14.984768,-3.716016,3853.0,9.0,2,0.8054144824310264
-726,True,TGTCCTGAGATCACCT-1,TGTCCTGAGATCACCT-1,1.9656132902217094,15.0,-0.091232798940644,-0.027955299018585695,0.45000005,15.490623,2,G1,1105.997310757637,5,5,4237.0,1416.0,9.794666037290536,34.41113995751711,0.7203067558848951,False,False,8.3,-1.930388,14.350856,607.0,9.0,5,1.4729340716123187
-727,True,TGTCCTGAGCAGCCCT-1,TGTCCTGAGCAGCCCT-1,1.9734430199328175,15.0,-0.12466789045697432,-0.012944363877971479,-4.765042,-5.4361906,9,G1,1279.3510907292366,2,2,14915.0,3096.0,10.3855179349648,29.782098558498156,0.945398541154095,False,False,2.8,-16.278988,-4.7119308,1386.0,9.0,2,0.8054144824310264
-728,True,TGTCCTGAGCTCACTA-1,TGTCCTGAGCTCACTA-1,2.0129199589867244,15.0,-0.046455346585956465,-0.027795147866941032,-3.833991,1.2910342,14,G1,1428.452069401741,4,4,15384.0,3738.0,7.722308892355694,16.978679147165888,0.9415303607827716,False,False,5.800000000000001,-14.903009,0.63063896,1937.0,9.0,4,1.5524679448293264
-731,True,TGTCCTGCAAGAGTAT-1,TGTCCTGCAAGAGTAT-1,1.880229815854178,15.0,-0.08976118113084655,-0.010429224770543453,8.716928,2.0649524,8,G1,1258.5271293222904,9,9,5949.0,1851.0,7.060010085728694,35.56900319381408,0.6026359538116105,False,True,4.6,13.277973,5.3792305,1005.0,9.0,9,1.5090271454508366
-732,True,TGTCCTGCAATCAAGA-1,TGTCCTGCAATCAAGA-1,1.9121122231911436,15.0,-0.10220777520834244,-0.00827667083883044,9.207056,5.1361117,26,G1,1121.9906817749143,9,9,4100.0,1179.0,10.439024390243903,43.4390243902439,0.6132679830460128,False,False,3.4000000000000004,11.08734,5.405723,1516.0,9.0,9,0.994405497775594
-733,True,TGTCCTGCAGCAGTGA-1,TGTCCTGCAGCAGTGA-1,1.9741747581081432,15.0,-0.09282845928220478,-0.027166726210659103,-8.237478,-2.2110348,20,G1,1318.79146258533,4,3,6557.0,2306.0,9.791062986121702,13.25301204819277,0.9759200356607102,False,True,2.5999999999999996,-17.649857,-2.1510115,2227.0,9.0,4,1.0266563462822664
-734,True,TGTCCTGCAGGTTCAT-1,TGTCCTGCAGGTTCAT-1,2.011345775358929,15.0,-0.08083217385542966,-0.02359120563004058,-6.9259295,1.5645906,4,G1,1491.2352100163698,4,4,10800.0,3106.0,8.36111111111111,13.703703703703704,0.9553014294977409,False,False,4.9,-14.776884,0.6592473,2221.0,9.0,4,1.7596138170802211
-735,True,TGTCCTGCATCAGTGT-1,TGTCCTGCATCAGTGT-1,2.0371746282503764,15.0,-0.06859238007127956,-0.020697888361983174,-6.848836,-6.4982953,3,G1,1340.3999814540148,2,2,6118.0,2264.0,16.181758744687805,11.768551814318405,0.9647265414578146,False,False,4.2,-15.285648,-6.284294,1849.0,9.0,2,1.2565453708589873
-736,True,TGTCCTGCATCCTATT-1,TGTCCTGCATCCTATT-1,1.8925517654975101,15.0,-0.07222736615686572,-0.03439407005768594,9.25675,-9.098481,11,G1,1232.3332973569632,7,6,3839.0,1548.0,14.11825996353217,13.961969262828863,0.1103943871471451,False,False,6.300000000000001,13.458703,-9.040423,1174.0,9.0,7,1.1538380923351783
-737,True,TGTCCTGCATGAAGGC-1,TGTCCTGCATGAAGGC-1,1.9236508661044558,15.0,-0.11994642799308824,-0.0072494272686586345,3.3905487,2.6621463,12,G1,1779.5438902527094,1,1,7825.0,1646.0,8.447284345047922,49.78913738019169,0.611382959772176,False,False,5.4,12.046296,9.274032,838.0,9.0,1,1.5503286204324531
-738,True,TGTCCTGGTAACTTCG-1,TGTCCTGGTAACTTCG-1,1.9935006662436916,15.0,-0.08749817673860144,-0.027152448235304504,-8.838496,-3.423599,7,G1,1656.2593463212252,3,3,8115.0,2590.0,9.661121380160196,19.71657424522489,0.9820539815447719,False,False,4.2,-14.19843,-3.4327896,2818.0,9.0,3,1.4709352270695035
-739,True,TGTCCTGGTAGAATGT-1,TGTCCTGGTAGAATGT-1,1.9252233422202916,15.0,-0.0851615540486835,-0.007476365747280431,5.4570036,8.590933,27,G1,1059.1375079900026,8,1,6932.0,1864.0,8.799769186381997,44.0132717830352,0.615222212255545,False,False,2.7,11.137352,11.336961,911.0,9.0,8,1.4194593160291944
-740,True,TGTCCTGGTAGCGTAG-1,TGTCCTGGTAGCGTAG-1,1.957688685325323,15.0,-0.11743777365381364,-0.0016742928003249218,0.29628354,12.34783,2,G1,874.7393051683903,5,5,5136.0,1768.0,9.871495327102803,33.31386292834891,0.7250754621375203,False,False,7.1000000000000005,-0.48804674,16.727488,1163.0,9.0,5,0.9362699326799151
-741,True,TGTCCTGGTATAGGAT-1,TGTCCTGGTATAGGAT-1,1.9754912911559004,15.0,-0.11724692527246106,-0.003286104085647206,2.1683464,18.888487,18,G1,1217.0519406348467,5,5,3672.0,1199.0,9.667755991285404,38.3442265795207,0.7233762333126846,False,False,9.3,-3.9285257,15.4664755,738.0,9.0,5,1.7829168942005256
-742,True,TGTCCTGGTCACTCGG-1,TGTCCTGGTCACTCGG-1,1.8877548610826922,15.0,-0.10858152879149881,-0.004433987388836122,1.9156111,3.575112,12,G1,1723.6746903955936,1,1,4911.0,1373.0,6.373447363062513,46.42638973732437,0.6109081962773073,False,False,3.8,10.680849,9.2193365,1216.0,9.0,1,1.2449347700414877
-743,True,TGTCCTGGTCCACACG-1,TGTCCTGGTCCACACG-1,1.8651994322952643,15.0,-0.05714878313462741,-0.012393644667660218,8.948169,-3.853235,23,G1,1172.0634506344795,6,8,4025.0,1751.0,6.136645962732919,20.29813664596273,0.385913429808859,False,False,6.000000000000001,14.361672,-1.8630714,1197.0,9.0,6,1.3984666828444978
-744,True,TGTCCTGGTGACAGGT-1,TGTCCTGGTGACAGGT-1,2.1044371472376504,15.0,-0.04664805145368088,-0.02866048077956508,-6.0115957,3.9681327,14,G1,1396.805547952652,4,4,2710.0,1193.0,20.66420664206642,9.59409594095941,0.932409247898045,False,False,6.8,-14.557359,2.8455603,1966.0,9.0,4,0.8054144824310264
-745,True,TGTCCTGGTGCCTTCT-1,TGTCCTGGTGCCTTCT-1,1.8599432686230368,15.0,-0.08198562118747262,-0.038779218602953164,9.035441,-13.116464,21,G1,1100.6179761737585,7,6,4944.0,2067.0,8.353559870550162,15.29126213592233,0.01138698590317793,False,False,6.0,11.052999,-11.904284,890.0,9.0,7,1.1903291071671906
-747,True,TGTCCTGTCCTGCTAC-1,TGTCCTGTCCTGCTAC-1,1.9438201957052987,15.0,-0.08914412301135688,-0.02854755328545661,-2.8329115,-2.476909,19,G1,1356.29573084414,2,2,9895.0,3083.0,8.29711975745326,13.623041940373927,0.9495302697137441,False,False,3.5000000000000004,-14.351428,-2.7654119,2573.0,9.0,2,0.8054144824310264
-748,True,TGTCCTGTCCTTATCA-1,TGTCCTGTCCTTATCA-1,1.8739426976336102,15.0,-0.09084828777012971,-0.02692042923361035,10.124462,-9.719194,21,G1,1273.2722176611423,7,6,4867.0,1887.0,7.643312101910828,18.24532566262585,0.08095612650161749,False,False,6.1,12.764945,-9.372049,1108.0,9.0,7,0.8054144824310264
-749,True,TGTCCTGTCGCCTTGT-1,TGTCCTGTCGCCTTGT-1,2.0391777688168444,15.0,-0.06475327887057554,-0.01786127512728626,-8.414596,0.8401632,4,G1,1488.4632907807827,4,4,3748.0,1612.0,14.541088580576307,10.005336179295623,0.9729194677056379,False,False,5.6,-14.48177,-0.31520638,2084.0,9.0,4,1.1399911048344968
-750,True,TGTCCTGTCGTCGGGT-1,TGTCCTGTCGTCGGGT-1,2.0455329391093113,15.0,-0.0664465838119449,-0.027784874808380176,-4.7377,-1.5307406,20,G1,1450.8899179548025,4,4,6080.0,2233.0,10.707236842105264,13.388157894736842,0.9588598092610706,False,False,2.3000000000000007,-15.264599,-2.0068157,7171.0,9.0,4,1.6103570683267916
-751,True,TGTCCTGTCTCGAGTA-1,TGTCCTGTCTCGAGTA-1,1.8962142807611084,15.0,-0.12022368531598933,-0.011267283412213851,2.4431252,7.9659977,5,G1,1650.7070639953017,1,1,7275.0,1830.0,4.769759450171821,46.87285223367697,0.6145300802073601,False,False,4.6000000000000005,9.472733,10.841999,1164.0,9.0,1,1.4207972118459975
-752,True,TGTGATGAGACCATTC-1,TGTGATGAGACCATTC-1,1.8699770888531382,15.0,-0.08138970090325326,0.002360002747190691,9.8293085,-6.079503,23,S,1153.779224023223,6,8,5015.0,1953.0,3.7088733798604188,21.136590229312063,0.2531702832926107,False,False,6.3999999999999995,14.244544,-4.742673,1210.0,9.0,6,1.2099737981140368
-753,True,TGTGATGAGACGTCCC-1,TGTGATGAGACGTCCC-1,1.8594184193400594,15.0,-0.06296257322249116,-0.0422615648201028,11.88913,-9.04346,11,G1,1224.3050207346678,7,6,1700.0,935.0,9.764705882352942,16.294117647058822,0.13286077154027318,False,False,6.9,12.769146,-7.235843,3087.0,10.0,7,1.4578952213257133
-754,True,TGTGATGAGACGTCGA-1,TGTGATGAGACGTCGA-1,2.002670043717114,15.0,-0.059429990320731874,-0.022190872163394604,-8.268753,1.2638988,4,G1,1478.0252715051174,4,4,6996.0,2529.0,14.122355631789594,11.120640365923386,0.9716872707156552,False,False,5.8999999999999995,-15.148985,0.22915505,2515.0,9.0,4,1.5463095259133282
-755,True,TGTGATGAGATCACCT-1,TGTGATGAGATCACCT-1,1.9902697802956906,15.0,-0.04358081790299998,-0.022092633763587374,-5.172722,-0.6400332,20,G1,1163.6595047563314,4,4,7115.0,2570.0,11.581166549543218,12.902319044272664,0.9607989690671437,True,True,5.000000000000001,-16.96524,-0.15379454,2676.0,9.0,4,1.0223161199091562
-756,True,TGTGATGAGCACCTGC-1,TGTGATGAGCACCTGC-1,1.9423891097703267,15.0,-0.09052049708754564,-0.01584507720106633,-6.913187,-12.242573,10,G1,1071.421182692051,3,7,10927.0,3137.0,12.656721881577743,18.586986364052347,0.9631460197112968,False,False,3.8000000000000003,-10.0325365,-10.7817745,1827.0,9.0,3,0.9637775905036999
-757,True,TGTGATGAGCCTCTTC-1,TGTGATGAGCCTCTTC-1,1.8915394344338388,15.0,-0.08891244813036746,-0.015114335502685016,3.545871,8.349058,5,G1,1360.532907217741,1,1,3692.0,1156.0,7.719393282773565,43.066088840736725,0.612930847596162,False,False,3.4999999999999996,9.195339,10.307538,1299.0,9.0,1,0.8054144824310264
-759,True,TGTGATGAGTAGTCTC-1,TGTGATGAGTAGTCTC-1,2.0258883651591555,15.0,0.022383412193512642,-0.03590509565117481,0.003162016,8.216598,5,G2M,1296.9681415706873,1,1,1352.0,710.0,24.70414201183432,4.881656804733728,0.6278250436898888,False,False,5.0,9.689995,12.692268,896.0,9.0,1,1.3328911693489227
-760,True,TGTGATGAGTATAACG-1,TGTGATGAGTATAACG-1,1.8958107075448165,15.0,-0.10549974771799459,-0.01036072880733075,3.114544,2.4391558,12,G1,1580.433781400323,1,1,5850.0,1488.0,5.965811965811966,47.21367521367522,0.6135218189775006,False,False,5.0,12.491536,10.150838,1091.0,9.0,1,1.186046434490902
-761,True,TGTGATGAGTCATCCA-1,TGTGATGAGTCATCCA-1,1.9191157579869884,15.0,-0.07850085128313056,-0.019075964964028923,5.742293,1.8336173,12,G1,1786.7683380842209,1,1,3536.0,1147.0,16.68552036199095,29.0158371040724,0.5992554581262096,False,False,6.5,12.816557,8.282912,1502.0,9.0,1,1.287099085379463
-762,True,TGTGATGAGTCCCAAT-1,TGTGATGAGTCCCAAT-1,2.0127671490312786,15.0,-0.04936384706084413,-0.030599954815243722,-5.9047256,0.8160577,20,G1,1473.191466987133,4,4,7725.0,2545.0,13.229773462783172,10.964401294498382,0.9578311293628247,False,True,4.8,-16.260683,0.6389205,2463.0,9.0,4,1.7559983261623684
-765,True,TGTGATGCAATCTGCA-1,TGTGATGCAATCTGCA-1,1.943544232085012,15.0,-0.06534355853981896,-0.018846317303197455,8.087464,-8.607146,11,G1,1169.5885694473982,7,6,12582.0,3963.0,9.871244635193133,12.056906692099826,0.1782334255817215,False,False,4.6,14.091203,-8.498599,2775.0,9.0,7,0.9362956489179584
-768,True,TGTGATGGTAACTTCG-1,TGTGATGGTAACTTCG-1,1.8896757760310963,15.0,-0.08874529609013322,-0.0074656569207062675,5.9505515,3.9363832,6,G1,1535.353196874261,1,1,6558.0,1755.0,5.85544373284538,47.04178103080207,0.6103428685909125,False,False,4.3999999999999995,12.151866,7.619268,1150.0,9.0,1,0.8054144824310264
-770,True,TGTGATGGTGCACAAG-1,TGTGATGGTGCACAAG-1,1.9746013971306806,15.0,-0.056877750106204286,-0.03754468346130253,-7.821018,-2.9351847,10,G1,1455.0421804487705,3,3,7344.0,2735.0,10.049019607843137,9.191176470588236,0.9720968309468662,False,True,2.8000000000000003,-14.23063,-3.0232525,3109.0,9.0,3,1.2811098899469326
-772,True,TGTGATGTCATCGCTC-1,TGTGATGTCATCGCTC-1,1.971919922367477,15.0,-0.11586427455100505,-0.0076923973472884325,2.838526,18.93781,18,G1,1169.9681016132236,5,5,5355.0,1481.0,8.571428571428571,40.26143790849673,0.7227137404306239,False,False,8.1,-3.9903052,14.759049,964.0,9.0,5,1.1492905295886513
-773,True,TGTGATGTCCCGTTGT-1,TGTGATGTCCCGTTGT-1,1.8979202933055017,15.0,-0.10313909647706189,0.003933925223191009,8.23399,4.512122,26,S,1440.2268758416176,9,9,6828.0,1754.0,6.5172817809021675,46.42647920328061,0.6100413467122258,False,False,4.3,11.591798,6.469926,871.0,9.0,9,1.3965182310137936
-774,True,TGTGATGTCCGCTTAC-1,TGTGATGTCCGCTTAC-1,1.8668055891437116,15.0,-0.10038819217327795,-0.021287043358448166,9.860882,-9.349485,11,G1,1226.159728512168,7,6,4057.0,1595.0,6.630515158984471,21.838797140744393,0.10715825320928203,False,False,5.2,12.834761,-8.610558,1029.0,9.0,7,1.3336802027849282
-775,True,TGTGATGTCCTACAAG-1,TGTGATGTCCTACAAG-1,1.918207204784291,15.0,-0.11055075472171733,-0.011230227963632176,3.5585155,6.8085895,6,G1,1346.4516777023673,1,1,7863.0,1933.0,9.105939208953325,45.28805799313239,0.6126006382095321,False,False,3.8000000000000003,8.998334,8.670845,1188.0,9.0,1,0.9768199251844366
-776,True,TGTGATGTCGCGGTAC-1,TGTGATGTCGCGGTAC-1,1.969157764004461,15.0,-0.13616527293340647,0.0008666349362728663,0.43775743,12.073963,2,S,874.7393051683903,5,5,8002.0,1790.0,9.585103724068983,44.23894026493377,0.7250018967120385,False,False,7.1000000000000005,-0.32569414,17.088692,991.0,9.0,5,0.9267070380396117
-777,True,TTACAGGAGCTTGTTG-1,TTACAGGAGCTTGTTG-1,1.9144210910157125,15.0,-0.12063061890787728,-0.003286641118509707,8.298939,-2.3887007,22,G1,1189.415604159236,6,8,4853.0,1439.0,8.530805687203792,41.211621677313005,0.4649355255540338,False,False,7.6,14.589276,0.6786306,895.0,9.0,6,1.26166821458594
-779,True,TTACAGGCAAGTCGTT-1,TTACAGGCAAGTCGTT-1,1.9547158435558727,15.0,-0.07583643431629818,-0.029920067662846515,-4.1658845,-3.999729,17,G1,1760.917803823948,2,2,12054.0,3468.0,7.375145180023229,18.234610917537747,0.9581938335193284,False,False,5.1,-14.5362215,-4.2232075,2724.0,9.0,2,1.6548521468330977
-781,True,TTACAGGCACCGTGAC-1,TTACAGGCACCGTGAC-1,2.0211588464387775,15.0,-0.06458810275685482,-0.03025165062459449,-2.8186924,-3.4168575,19,G1,1506.8784712404013,2,2,12226.0,3270.0,9.487976443644692,17.16015049893669,0.9560814977192231,False,False,3.8999999999999995,-15.387221,-3.4632876,1654.0,9.0,2,1.060198703168392
-782,True,TTACAGGCACGCGCAT-1,TTACAGGCACGCGCAT-1,1.880100179222348,15.0,-0.07976150557628482,-0.02639488317693825,11.414364,-8.977948,11,G1,1231.6576531976461,7,6,5351.0,1896.0,8.521771631470752,20.07101476359559,0.13161221951324903,False,False,6.1,13.517597,-7.434969,1158.0,9.0,7,0.9625823252815406
-783,True,TTACAGGCACTACGGC-1,TTACAGGCACTACGGC-1,1.863786659967178,15.0,-0.07376606373169986,-0.02068025537412085,8.456794,-14.064933,21,G1,1084.4156861752272,7,6,8209.0,3048.0,7.784139359239859,14.045559751492265,0.0023921021241353115,False,False,7.6,10.099889,-12.412533,1169.0,9.0,7,1.054449906360884
-784,True,TTACAGGCACTATGTG-1,TTACAGGCACTATGTG-1,1.8820304905342313,15.0,-0.1016387705800872,-0.014795039472315835,3.9309125,5.466453,12,G1,1817.991869442165,1,1,7184.0,1839.0,3.8557906458797326,49.05345211581292,0.6101786568235836,False,False,4.3,10.082329,8.650741,1661.0,9.0,1,1.3891576830101735
-787,True,TTACAGGGTCTAGGTT-1,TTACAGGGTCTAGGTT-1,1.869406827501072,15.0,-0.10160865458638463,0.0008477900810101996,9.676507,-11.391399,21,S,1289.3595440238714,7,6,4199.0,1732.0,9.764229578471065,18.599666587282687,0.04268725381887684,False,False,6.6,11.69186,-10.466122,2919.0,9.0,7,0.8054144824310264
-788,True,TTACAGGGTGCCTTCT-1,TTACAGGGTGCCTTCT-1,1.9140753111259359,15.0,-0.10169341340039735,-0.007594760717570141,0.9084095,4.3206205,5,G1,1440.4715542197227,1,1,9058.0,2143.0,8.423493044822257,44.53521748730404,0.6024070038649685,False,False,2.6999999999999997,9.224276,10.115904,884.0,9.0,1,0.9413450291278668
-790,True,TTACAGGTCACAAGAA-1,TTACAGGTCACAAGAA-1,1.9107201860887741,15.0,-0.09848695059892426,-0.015341848100337173,9.091842,1.4287473,25,G1,951.530439004302,6,9,5596.0,1689.0,9.88205861329521,36.02573266619014,0.5940503294929078,False,False,3.2,14.292254,5.3474603,1469.0,9.0,6,0.9746864115033027
-791,True,TTACAGGTCCAATCCC-1,TTACAGGTCCAATCCC-1,1.983734173066752,15.0,-0.08118050682909767,-0.020626071384939347,-4.5361385,-3.844927,17,G1,1724.290115788579,2,2,9033.0,2880.0,7.539023580205912,13.572456548212111,0.9587155700839172,False,False,4.4,-14.843795,-4.109962,2756.0,9.0,2,1.6863565313395337
-792,True,TTACAGGTCGTAGAGG-1,TTACAGGTCGTAGAGG-1,1.9949702387845065,15.0,-0.11751151307077841,-0.013825701204634812,-1.9741431,15.74029,2,G1,1245.386822938919,5,5,6631.0,1815.0,13.84406575177198,36.66113708339617,0.7327370943618671,False,False,8.299999999999999,0.90784925,13.792503,1115.0,9.0,5,0.8054144824310264
-793,True,TTACAGGTCGTTTACT-1,TTACAGGTCGTTTACT-1,1.9890036483227598,15.0,-0.07626212465988462,-0.02915275709703739,-5.29986,-1.3441175,20,G1,1171.1466239392757,4,4,7176.0,2486.0,12.639353400222966,14.241917502787068,0.9626546477152669,False,False,4.300000000000001,-16.491022,-1.3243436,1407.0,9.0,4,1.104877227849968
-794,True,TTCGATTAGCCTCTGG-1,TTCGATTAGCCTCTGG-1,1.9631693251041176,15.0,-0.06865708469557708,-0.01428093165622459,-7.780041,-1.2791023,14,G1,1343.2738090604544,4,4,14500.0,3792.0,7.337931034482758,23.76551724137931,0.9479123612226927,False,False,1.1,-13.474938,-0.9305661,2745.0,9.0,4,1.0926648690157201
-795,True,TTCGATTAGGCGTTAG-1,TTCGATTAGGCGTTAG-1,1.928713646918808,15.0,-0.07403403075879844,-0.018897249880283722,8.851331,-11.595025,21,G1,1249.209385380149,7,6,2515.0,1150.0,17.852882703777336,14.671968190854871,0.03526252521358326,False,False,5.8999999999999995,11.472835,-10.496805,1319.0,9.0,7,1.0149907358058574
-796,True,TTCGATTAGGTTGCCC-1,TTCGATTAGGTTGCCC-1,1.8649231107748268,15.0,-0.08106329206572681,-0.0008553770251572661,0.14592372,4.6286373,5,G1,975.1607788205147,1,1,4432.0,1552.0,4.30956678700361,36.28158844765343,0.6101413900688135,False,False,3.3000000000000003,8.256551,9.4347725,1459.0,9.0,1,1.1256431281235757
-797,True,TTCGATTAGTATAACG-1,TTCGATTAGTATAACG-1,1.9179810431212303,15.0,-0.1089560915722368,-0.008929451068204801,3.7961442,2.0741343,12,G1,1526.6916943192482,1,1,7818.0,1805.0,7.802507035047327,46.763878229726274,0.6117926384130513,False,False,4.3,11.994957,8.831274,897.0,9.0,1,0.9404045018181607
-798,True,TTCGATTCAATCTGCA-1,TTCGATTCAATCTGCA-1,1.9608337696806817,15.0,-0.08077660865412815,-0.007930530694502825,-12.441348,-4.0289507,7,G1,964.5888964086771,3,3,12079.0,3651.0,9.421309711068798,15.05919364185777,0.9952075944258093,False,False,4.8,-18.631914,-4.345091,2261.0,9.0,3,1.1682968396501654
-799,True,TTCGATTCATTATGCG-1,TTCGATTCATTATGCG-1,2.0118543827454745,15.0,-0.07824926123351353,-0.027633763608679678,-5.8079047,1.9708694,14,G1,1556.311996564269,4,4,9379.0,2777.0,10.864697728968974,17.7950741017166,0.9461969530887814,False,True,4.3,-14.733806,0.78425884,1493.0,9.0,4,0.8054144824310264
-800,True,TTCGATTGTAACGCGA-1,TTCGATTGTAACGCGA-1,2.0344376272162523,15.0,-0.06042445240137878,-0.030891970544974143,-5.219139,1.6673121,14,G1,1489.2060773521662,4,4,8297.0,2750.0,10.087983608533206,15.258527178498252,0.9447871588863059,False,False,4.699999999999999,-14.991589,1.0256742,1905.0,9.0,4,1.848549175898292
-802,True,TTCGATTGTGCGGTAA-1,TTCGATTGTGCGGTAA-1,1.9888991161998046,15.0,-0.08881710451137034,-0.02906812130343859,-6.7208543,-4.2697744,13,G1,1164.5691663473845,2,2,6311.0,2487.0,16.19394707653304,11.376960861987007,0.9562796949061462,False,False,2.2,-16.034702,-4.3789635,3529.0,9.0,2,1.0208956556010638
-803,True,TTCGATTGTGTCTAAC-1,TTCGATTGTGTCTAAC-1,1.9477074839190385,15.0,-0.10874677100114714,-0.018153062733224132,2.0897775,17.661932,18,G1,1068.0044005140662,5,5,9171.0,2237.0,7.34925308036201,41.020608439646715,0.7222859859982317,True,True,4.8,-4.106,14.503065,1622.0,9.0,5,0.9552628918617548
-804,True,TTCGATTGTTAAGACA-1,TTCGATTGTTAAGACA-1,1.9195328507802165,15.0,-0.10269830108318244,-0.0023951703948935695,0.78249884,4.268289,5,G1,1215.2527044862509,1,1,7856.0,1933.0,9.521384928716904,42.90987780040733,0.6113422542057891,False,False,2.3000000000000003,9.402512,8.975839,793.0,9.0,1,0.8054144824310264
-805,True,TTCGATTGTTAAGCAA-1,TTCGATTGTTAAGCAA-1,1.9421704727415126,15.0,-0.08238245495608054,-0.012615184329428642,-12.081141,-6.5755415,7,G1,992.9331075549126,3,3,10452.0,3310.0,9.672789896670494,16.054343666283966,0.9798407488628106,False,False,3.1999999999999997,-18.472147,-5.788819,2927.0,9.0,3,0.8054144824310264
-806,True,TTCGATTGTTCGGCTG-1,TTCGATTGTTCGGCTG-1,1.903802507693129,15.0,-0.0968745433938719,-0.009759256367627749,7.413578,-3.3116705,22,G1,1199.1119232177734,6,8,3624.0,1368.0,10.596026490066226,31.76048565121413,0.44256553865472414,False,False,8.0,14.787608,0.11021995,1569.0,9.0,6,1.0169260766413897
-807,True,TTCGATTTCACAAGAA-1,TTCGATTTCACAAGAA-1,1.9640852845711783,15.0,-0.07740569991184637,-0.027162351538518852,-6.9134884,-6.79006,3,G1,1308.4515990763903,2,2,11540.0,3527.0,9.939341421143848,14.428076256499134,0.9633406149641547,False,False,3.5,-15.62653,-7.1037297,3259.0,9.0,2,1.4677254263487154
-808,True,TTCGATTTCCAATCCC-1,TTCGATTTCCAATCCC-1,1.9125397362007335,15.0,-0.10178809368463583,-0.02359250685130612,2.3287635,7.083042,5,G1,1227.3833480924368,1,1,4609.0,1340.0,12.280321110870037,35.86461271425472,0.6121793169715376,False,False,3.5,8.380452,9.634605,870.0,9.0,1,1.340469844704727
-809,True,TTCGATTTCGAACGCC-1,TTCGATTTCGAACGCC-1,1.8854513209431758,15.0,-0.1193423568330109,-0.0059031673451434685,4.001051,7.340846,12,G1,1269.300905406475,1,1,6563.0,1834.0,5.1653207374676215,43.821423129666314,0.6127036409071476,False,False,3.5000000000000004,8.850339,9.3605,1656.0,9.0,1,1.3494763739696543
-810,True,TTGTTTGAGAAATTGC-1,TTGTTTGAGAAATTGC-1,1.9752689028906927,15.0,-0.07757841973821714,-0.00987810292818464,-10.772302,-3.7890031,10,G1,1330.34066426754,3,3,5050.0,1978.0,15.069306930693068,10.732673267326733,0.9914607104775305,False,False,3.2,-17.594032,-3.3788261,1761.0,10.0,3,1.1917359633853428
-811,True,TTGTTTGAGACCAGAC-1,TTGTTTGAGACCAGAC-1,1.9621097761570383,15.0,-0.09456534419079017,-0.02008473801119061,-6.5687246,-9.546865,3,G1,1010.1624644100666,2,2,13551.0,3755.0,8.892332669175707,15.984060216958158,0.9511266457105118,False,False,3.0,-15.020468,-9.309875,3123.0,9.0,2,1.3374992685758762
-813,True,TTGTTTGAGACGTCCC-1,TTGTTTGAGACGTCCC-1,1.8745730638202727,15.0,-0.06834019932176122,-0.030724741947241542,9.180972,-13.146715,21,G1,1132.0540813356638,7,6,4943.0,1991.0,10.277159619664172,13.857980983208577,0.005732670789113113,False,False,6.899999999999999,10.251482,-11.901026,1999.0,9.0,7,1.396173189026539
-815,True,TTGTTTGAGCAATTAG-1,TTGTTTGAGCAATTAG-1,1.9066917558300296,15.0,-0.1054992652371328,-0.018887705732656974,5.51581,3.0671074,6,G1,1684.0923893153667,1,1,9507.0,2223.0,7.100031555695803,43.78878720942463,0.6069726364709669,False,False,4.2,12.87091,7.9099436,1433.0,9.0,1,1.188516974947064
-816,True,TTGTTTGAGGCCTAGA-1,TTGTTTGAGGCCTAGA-1,1.9181357430858224,15.0,-0.07599780166017484,-0.01865549315817125,-4.6159506,-12.828959,24,G1,1297.5579494386911,10,7,11028.0,2983.0,8.07943416757345,15.034457743924555,0.9601773092099093,False,False,5.6,-7.7111163,-11.129664,1154.0,9.0,10,0.9416146684201823
-817,True,TTGTTTGCAAGTCGTT-1,TTGTTTGCAAGTCGTT-1,1.9533856827931142,15.0,-0.12885978272673784,-0.008880581728370654,-2.6169894,14.133298,2,G1,1164.3194821029902,5,5,9933.0,2589.0,7.701600724856539,34.34007852612504,0.7328779266116664,False,False,7.3,1.3515564,15.070348,1501.0,9.0,5,1.4685209615728756
-818,True,TTGTTTGCACCCTCTA-1,TTGTTTGCACCCTCTA-1,1.9217212087577784,15.0,-0.11378021075890364,-0.000840733228175742,2.281904,19.181974,18,G1,1198.0498885288835,5,5,5609.0,1868.0,4.671064360848636,37.22588696737386,0.7229715840619367,False,False,9.600000000000001,-4.176912,15.232005,737.0,9.0,5,1.5835220671435046
-819,True,TTGTTTGCACTATGTG-1,TTGTTTGCACTATGTG-1,1.9473365061266577,15.0,-0.08515531677873708,-0.028869038328033735,-8.072493,-5.4398336,13,G1,1411.8426660001278,2,2,12440.0,3714.0,7.564308681672026,12.733118971061094,0.9742945039929677,False,False,2.8000000000000003,-16.999773,-5.9771724,2409.0,9.0,2,1.3707553553494534
-820,True,TTGTTTGCACTTCAGA-1,TTGTTTGCACTTCAGA-1,1.8671083641448456,15.0,-0.05843897425594144,-0.031179197758492774,7.938827,-13.788934,21,G1,1058.2730616033077,7,6,3704.0,1693.0,8.450323974082073,15.955723542116631,0.0054462222247503205,False,True,6.7,10.610644,-12.3141575,521.0,9.0,7,0.8054144824310264
-821,True,TTGTTTGCAGCTACTA-1,TTGTTTGCAGCTACTA-1,1.9479615766758482,15.0,-0.09976606577679925,0.019993425155632117,-0.34441575,13.141485,2,S,964.2033444046974,5,5,5330.0,1752.0,8.03001876172608,36.02251407129456,0.7270697192371115,False,False,5.1000000000000005,-0.08312444,16.123432,1308.0,9.0,5,1.1779758086943841
-822,True,TTGTTTGCATATCTGG-1,TTGTTTGCATATCTGG-1,1.985089671216524,15.0,-0.05410006347740781,-0.03239042250160321,-2.3342252,-3.2354357,19,G1,1494.2676855921745,2,2,4960.0,2126.0,19.43548387096774,7.237903225806452,0.9544236710401232,False,False,4.8,-14.051211,-3.4206548,2035.0,9.0,2,1.0525581538937765
-823,True,TTGTTTGCATCCTATT-1,TTGTTTGCATCCTATT-1,1.8909349033372356,15.0,-0.07287427077146383,-0.012455177568919184,10.540299,-7.8922668,11,G1,1215.4865544587374,7,6,5063.0,1846.0,6.814141813154256,27.078807031404306,0.13658278960963938,False,False,7.5,13.823376,-7.8621225,2083.0,9.0,7,0.8054144824310264
-824,True,TTGTTTGGTACGGGAT-1,TTGTTTGGTACGGGAT-1,1.9202850900238484,15.0,-0.09669055217116364,-0.0021071062286991994,7.4346137,10.0420685,15,G1,1204.904662579298,8,10,8930.0,1953.0,7.435610302351623,48.21948488241881,0.6222434056885917,False,False,5.5,14.833162,10.197359,909.0,9.0,8,1.6211560017212299
-825,True,TTGTTTGGTATACCTG-1,TTGTTTGGTATACCTG-1,1.9343771314970037,15.0,-0.0874661068835826,-0.0235254712585281,-9.707651,-9.559587,10,G1,1172.2345714718103,3,7,15244.0,3730.0,7.340593020204671,18.80739963264235,0.9667254728563515,False,False,5.6,-11.290097,-9.605325,2939.0,9.0,3,1.6623647202889213
-826,True,TTGTTTGGTCACGCTG-1,TTGTTTGGTCACGCTG-1,1.910520151675248,15.0,-0.09986865582294008,-0.0199584924426294,9.609835,-3.6800518,23,G1,1143.5191202908754,6,8,6662.0,2023.0,9.606724707295106,37.00090063044131,0.3955592402813177,False,False,6.900000000000001,14.519898,-1.3914303,1453.0,9.0,6,1.0970140788841232
-827,True,TTGTTTGGTTTGATCG-1,TTGTTTGGTTTGATCG-1,1.915807264806834,15.0,-0.105964352439357,-0.011693154419245698,4.383486,4.548897,12,G1,1778.920767262578,1,1,8382.0,1896.0,7.217847769028872,47.51849200668099,0.6116652448150776,False,False,4.6000000000000005,10.368675,8.270589,1128.0,9.0,1,1.1008992261743455
-829,True,TTGTTTGTCCCGTTGT-1,TTGTTTGTCCCGTTGT-1,1.8904271189518058,15.0,-0.11790194059342561,-0.007929377743535616,6.647358,3.584762,6,G1,1616.9419396817684,1,1,7154.0,1761.0,6.13642717360917,46.12804025719877,0.6092344315439678,False,False,4.300000000000001,12.4094925,8.071194,1140.0,9.0,1,1.2636853133742638
-830,True,TTGTTTGTCCGCTTAC-1,TTGTTTGTCCGCTTAC-1,1.9340631091893716,15.0,-0.08316946058226787,0.007226235772029667,-0.59491754,16.386585,2,S,1120.9773846194148,5,5,7012.0,2021.0,8.214489446662864,38.861950941243585,0.7260288282457544,False,True,3.9000000000000004,-0.9130639,13.869253,712.0,9.0,5,0.8054144824310264
-831,True,TTGTTTGTCTAGTCAG-1,TTGTTTGTCTAGTCAG-1,1.914348065114423,15.0,-0.11822268268567233,-0.016390210968518143,2.1490452,4.5141597,12,G1,1790.6346159130335,1,1,5774.0,1673.0,9.005888465535158,39.729823346033946,0.6138953933954113,False,False,4.1,10.697361,10.452252,1207.0,9.0,1,1.4296175557143471
-832,True,TTGTTTGTCTCCGAGG-1,TTGTTTGTCTCCGAGG-1,1.9746648258488386,15.0,-0.03403378221692178,-0.03391155545281759,-7.931644,-12.821559,16,G1,678.1539006382227,2,2,4608.0,1253.0,5.403645833333333,1.6710069444444444,0.9320526574632143,False,False,6.000000000000001,-15.255273,-11.320599,418.0,10.0,2,1.0639007614998948
-833,True,TTTCATGAGACCATTC-1,TTTCATGAGACCATTC-1,1.9053436661977547,15.0,-0.0826095216364206,-0.009610521892234585,1.5492808,6.737165,5,G1,1748.7209868729115,1,1,7009.0,1884.0,6.9910115565701245,43.629619061207016,0.6136561007123282,False,False,7.3,9.397441,11.10729,764.0,9.0,1,1.8353995103829035
-834,True,TTTCATGAGCTCACTA-1,TTTCATGAGCTCACTA-1,1.902431747507048,15.0,-0.09963029218843172,-0.010319001386962555,5.0693865,5.910128,12,G1,1696.854793831706,1,1,6250.0,1690.0,6.96,44.032,0.6108920397404606,False,False,5.199999999999999,10.163379,8.36053,779.0,9.0,1,0.9936018591465023
-835,True,TTTCATGAGGAACTCG-1,TTTCATGAGGAACTCG-1,2.081869246235219,15.0,-0.025958231548530022,-0.02486455454979241,-12.472483,-1.6728506,7,G1,1073.500611051917,3,3,3239.0,1399.0,30.56498919419574,2.686014201914171,0.9983722905717957,False,False,3.8000000000000003,-18.925875,-2.2101033,1974.0,9.0,3,0.8054144824310264
-837,True,TTTCATGCAATAGTAG-1,TTTCATGCAATAGTAG-1,1.980117458915072,15.0,-0.06346314162452374,-0.0315566840222714,-10.975349,-2.4686308,7,G1,1359.0012575536966,3,3,12135.0,3529.0,10.045323444581788,12.632880098887515,0.9965198728365181,False,True,3.0000000000000004,-18.049793,-3.0058286,2661.0,9.0,3,1.354153753751116
-838,True,TTTCATGCACAAATCC-1,TTTCATGCACAAATCC-1,2.0865810434795247,15.0,-0.05635228353817205,-0.010165918643523786,-4.786767,2.4467435,14,G1,1192.5043985396624,4,4,1619.0,855.0,23.65657813465102,5.435453983940704,0.8818081429272452,False,False,3.3,-13.075273,1.6400137,3321.0,9.0,4,0.8054144824310264
-839,True,TTTCATGCACAGAGAC-1,TTTCATGCACAGAGAC-1,1.9827361549109466,15.0,-0.0607892369610962,-0.019737441285118423,-7.1416144,2.125142,14,G1,1416.560026422143,4,4,12117.0,3454.0,8.021787571180985,14.549806057605018,0.9444012609014811,False,False,4.2,-13.788753,0.52584934,2347.0,9.0,4,1.5214671427815627
-840,True,TTTCATGGTATGAGCG-1,TTTCATGGTATGAGCG-1,1.9848093043658221,15.0,-0.09059345451060567,-0.0014275532219360197,1.7681144,15.977026,2,G1,1099.15285089612,5,5,5376.0,1536.0,12.239583333333334,39.78794642857143,0.7192568907062931,True,True,9.700000000000001,-2.2076035,14.47884,1325.0,9.0,5,1.1657563932436377
-841,True,TTTCATGGTCTTCATT-1,TTTCATGGTCTTCATT-1,1.899735812440576,15.0,-0.08175444927533435,-0.012624052354469926,5.698633,6.0660357,12,G1,1468.4821402207017,1,1,7416.0,2046.0,4.854368932038835,43.67583603020496,0.6117084621413412,False,False,3.9,10.459781,8.147521,1262.0,8.0,1,0.9882727834584122
-842,True,TTTCATGGTGCACAAG-1,TTTCATGGTGCACAAG-1,1.9004130326272624,15.0,-0.0918545229977413,-0.010409649779120872,8.287515,-1.3116577,22,G1,1103.3079520314932,6,8,6844.0,2034.0,8.781414377556985,35.8562244301578,0.4981501824944171,False,False,5.1,14.564746,1.8483461,1747.0,9.0,6,1.0027619583455896
-843,True,TTTCATGGTGCCTTCT-1,TTTCATGGTGCCTTCT-1,1.9497846825193248,15.0,-0.07771081713371006,-0.026227772326966575,-3.0179029,-5.781449,19,G1,1474.7950672507286,2,2,9843.0,3384.0,5.61820583155542,14.62968607131972,0.9512974980485656,False,True,4.1,-14.781908,-5.7593703,1971.0,9.0,2,1.496729092924822
-844,True,TTTCATGGTTAAGACA-1,TTTCATGGTTAAGACA-1,1.900858909556776,15.0,-0.06961057296364562,-0.012668994643159244,8.709948,10.139294,15,G1,1061.6687054932117,8,10,5930.0,1996.0,6.020236087689713,32.78246205733558,0.6229006318903403,False,False,5.8999999999999995,15.312605,10.760088,1491.0,9.0,8,0.8933758631233296
-845,True,TTTCATGGTTGGGATG-1,TTTCATGGTTGGGATG-1,1.9735293007275374,15.0,-0.13416815742397137,-0.006083169087767962,1.7496735,18.55981,18,G1,1217.0519406348467,5,5,4940.0,1247.0,10.34412955465587,43.48178137651822,0.7233114173337123,False,False,9.3,-4.405904,15.23818,886.0,9.0,5,1.3995114052562618
-846,True,TTTCATGGTTTGATCG-1,TTTCATGGTTTGATCG-1,2.00550161956267,15.0,-0.07137386898171123,-0.025626175483850946,-7.2914724,0.9384664,14,G1,1536.9589839726686,4,4,4052.0,1743.0,16.33761105626851,10.069101678183612,0.9602837923338556,False,False,4.4,-14.240705,-0.54623425,2197.0,9.0,4,1.3002162541380378
-847,True,TTTCATGTCCACGTAA-1,TTTCATGTCCACGTAA-1,1.8869538571397328,15.0,-0.05614630630870724,-0.02393392858219023,9.560914,-14.147546,21,G1,1090.366791844368,7,6,4296.0,1881.0,7.937616387337058,12.965549348230912,0.0004933585999722467,False,False,8.9,9.5675535,-12.099556,729.0,9.0,7,1.0564091990052558
-848,True,TTTCATGTCTCATAGG-1,TTTCATGTCTCATAGG-1,1.8661443215062412,15.0,-0.0718229829660782,-0.00399756258204758,10.023972,-10.23923,21,G1,1293.6926913708448,7,6,3377.0,1521.0,8.380219129404797,17.145395321291087,0.06935945092627059,False,False,7.1000000000000005,12.298812,-9.924648,1332.0,9.0,7,1.3748487670888325
-849,True,TTTCATGTCTCTAGGA-1,TTTCATGTCTCTAGGA-1,1.9809075896782677,15.0,-0.06153621579901342,-0.01526941759027743,-9.512626,-4.242979,1,G1,1413.3102297782898,3,3,4036.0,1559.0,12.314172447968286,11.719524281466798,0.9856237457178274,False,False,4.4,-13.542432,-4.7362223,1807.0,9.0,3,1.537307872645059
-850,True,TTTCGATAGCAATTAG-1,TTTCGATAGCAATTAG-1,1.956032534047517,15.0,-0.10356088419049281,-0.011195574606825912,2.6174123,19.53445,18,G1,1196.3447940945625,5,5,4592.0,1577.0,9.277003484320558,36.106271777003485,0.722755055273252,False,False,9.8,-3.7965312,15.82559,630.0,9.0,5,0.9980207379784404
-851,True,TTTCGATAGCAGCCCT-1,TTTCGATAGCAGCCCT-1,1.9783649555935634,15.0,-0.07733700303184136,-0.026364638522549277,-4.890395,-4.507329,13,G1,1228.7539929002523,2,2,15498.0,4121.0,9.568976642147375,19.11859594786424,0.9545851307045427,False,False,3.1000000000000005,-16.634544,-5.424287,680.0,9.0,2,1.3227646984289116
-852,True,TTTCGATAGGCTCAAG-1,TTTCGATAGGCTCAAG-1,1.9281777510328753,15.0,-0.11685922562183546,-0.008410210926966546,0.5070734,14.122722,2,G1,1069.4703082293272,5,5,5761.0,1727.0,8.140947752126367,40.67002256552682,0.7231650662258335,False,False,6.199999999999998,-0.9211489,15.650139,1039.0,9.0,5,0.8054144824310264
-854,True,TTTCGATAGGTTACAA-1,TTTCGATAGGTTACAA-1,1.9834395145658077,15.0,-0.09702810349193229,-0.018276666472641277,-3.4579217,13.970264,14,G1,1054.8793014734983,4,5,12319.0,3260.0,11.307736017533891,27.883756798441432,0.7744462018865857,False,False,4.4,1.3054883,13.664667,2180.0,9.0,4,0.8054144824310264
-855,True,TTTCGATAGTATAACG-1,TTTCGATAGTATAACG-1,1.9778467375833202,15.0,-0.04323153209971339,-0.023362595884264925,-3.2459311,-0.27868858,19,G1,1389.2521254718304,2,4,3469.0,1579.0,11.761314499855866,11.040645719227443,0.9476087568427997,False,False,3.1999999999999997,-13.188602,-0.64472383,1996.0,9.0,2,0.8054144824310264
-856,True,TTTCGATAGTCATCCA-1,TTTCGATAGTCATCCA-1,1.9246239818181294,15.0,-0.10216006134427047,-0.008844625754924033,2.0101724,5.810622,12,G1,1779.5804100111127,1,1,5316.0,1487.0,11.963882618510159,40.331075996990215,0.6131774238886664,False,False,4.8999999999999995,9.586488,10.659369,927.0,9.0,1,1.0108431860397664
-857,True,TTTCGATAGTGGCCTC-1,TTTCGATAGTGGCCTC-1,2.0252024060360094,15.0,-0.05089159468233808,-0.024863508278896745,-8.975499,-0.09873688,20,G1,1496.6306136399508,4,3,9154.0,2894.0,9.17631636443085,15.479571771902993,0.9766590446020111,False,False,3.0,-17.47184,-1.0116367,1433.0,9.0,4,1.314136678557241
-858,True,TTTCGATCAACGCATT-1,TTTCGATCAACGCATT-1,1.9678142474000782,15.0,-0.07942279407544897,-0.02854721781866202,-3.8061025,-5.9308567,3,G1,1405.0495891273022,2,2,13092.0,3667.0,11.083104185762297,14.634891536816376,0.9544199724810613,True,True,3.5000000000000004,-15.34601,-5.980405,2282.0,9.0,2,1.0131033238105016
-859,True,TTTCGATCAATTAGGA-1,TTTCGATCAATTAGGA-1,1.9976063899980538,15.0,-0.06975534555776722,-0.018812394644858735,-5.245734,1.2630126,20,G1,1509.1067515611649,4,4,6655.0,2216.0,8.685199098422238,14.154770848985725,0.9498201497478881,False,False,5.7,-16.052538,1.3479481,1543.0,9.0,4,1.1298925429918807
-860,True,TTTCGATCACCAGCGT-1,TTTCGATCACCAGCGT-1,2.025678871959194,15.0,-0.06966074956002373,-0.03189665857404402,-11.600535,-1.1176513,7,G1,1367.1188849061728,3,3,3454.0,1633.0,16.47365373480023,5.67458019687319,0.9950585460661505,False,False,5.3,-18.401112,-1.9424376,1829.0,9.0,3,0.8054144824310264
-861,True,TTTCGATCAGGTTCAT-1,TTTCGATCAGGTTCAT-1,1.9774142722357326,15.0,-0.06653119721498757,-0.02353947072103861,-4.546644,-2.496202,17,G1,1570.1034287959337,2,2,8844.0,2834.0,11.79330619629127,13.319764812302125,0.9607875744580767,False,False,3.1000000000000005,-16.156998,-2.4776654,2580.0,9.0,2,1.298738306364051
-862,True,TTTCGATCATCAGTGT-1,TTTCGATCATCAGTGT-1,1.9358900144835494,15.0,-0.11175001430945306,-0.008746293391300711,3.559014,3.1816068,12,G1,1846.6502016484737,1,1,3938.0,1067.0,9.217877094972067,45.606907059421026,0.6044411598690631,False,False,5.1000000000000005,11.84136,8.207761,930.0,9.0,1,1.2179190971276364
-863,True,TTTCGATGTATGAGCG-1,TTTCGATGTATGAGCG-1,1.928528420475325,15.0,-0.1043877832768613,-0.014673192928056472,-2.7567112,15.052473,2,G1,1265.617069043219,5,5,12433.0,2849.0,7.4640070779377465,44.06820558191909,0.7295941395490527,False,False,7.8,1.2518011,13.926078,2014.0,9.0,5,0.8054144824310264
-864,True,TTTCGATGTCCCGTGA-1,TTTCGATGTCCCGTGA-1,1.9433387732719194,15.0,-0.08336764534356744,-0.02615581120603276,-10.345494,-7.386103,10,G1,1131.2083248347044,3,7,12730.0,3427.0,9.057344854673998,18.735271013354282,0.9755312349074768,False,False,3.9000000000000004,-12.350902,-8.013443,2445.0,9.0,3,1.5083956432650072
-866,True,TTTCGATGTTCCGTTC-1,TTTCGATGTTCCGTTC-1,2.0693157343073265,15.0,-0.05383839834697074,-0.030110677832313446,-4.050998,-4.589004,19,G1,1609.2243176698685,2,2,6028.0,2288.0,27.488387524883876,8.775713337757134,0.9566639519110978,False,False,4.1,-13.734276,-4.868857,2217.0,9.0,2,0.8054144824310264
-867,True,TTTCGATTCCCGTTGT-1,TTTCGATTCCCGTTGT-1,1.8855441123116619,15.0,-0.11588649012662205,-0.009914594012233583,4.051968,5.3594155,6,G1,1666.8162060678005,1,1,5134.0,1524.0,5.960264900662252,42.34514998052201,0.6123927158888267,False,False,4.199999999999999,10.575139,8.686182,927.0,9.0,1,1.7161832713700895
-868,True,TTTCGATTCCGAGAAG-1,TTTCGATTCCGAGAAG-1,2.008966448093938,15.0,-0.04678429160923411,-0.030891768557708723,-5.420791,-2.58762,9,G1,1625.4862473756075,2,2,7007.0,2579.0,13.686313686313687,11.859569002426145,0.959927520721118,False,False,4.0,-16.170618,-2.930263,985.0,9.0,2,1.8487134598031176
-869,True,TTTCGATTCCTGCTAC-1,TTTCGATTCCTGCTAC-1,1.9603115358658523,15.0,-0.0770408588158617,-0.01344687302879709,1.7376593,18.634333,18,G1,1216.3890205323696,5,5,5807.0,1776.0,7.938694678835888,38.7807818150508,0.7235056831280424,False,False,9.000000000000002,-3.4221091,15.927235,783.0,9.0,5,1.26054803309828
-870,True,TTTCGATTCTCTAGGA-1,TTTCGATTCTCTAGGA-1,1.9778148053057394,15.0,-0.051016845639497994,-0.02739020176319023,-5.3302994,-4.2790384,17,G1,1786.0439134985209,2,2,10914.0,3333.0,7.669048927982408,15.869525380245555,0.9624274387504391,False,False,4.700000000000001,-14.7514,-4.531816,1843.0,9.0,2,1.7006603210004854
-871,True,TTTCGATTCTGTGCGG-1,TTTCGATTCTGTGCGG-1,2.2338719272078404,15.0,-0.042323489482747646,-0.02457908320019663,-7.8920197,-12.066318,16,G1,678.5019699335098,2,2,553.0,290.0,13.200723327305607,6.509945750452079,0.9357306342565149,False,False,5.9,-15.480569,-11.224276,632.0,9.0,2,0.9485778766191572
-872,True,TTTGTTGAGACGTCCC-1,TTTGTTGAGACGTCCC-1,2.027454786020684,15.0,-0.03272929838927921,-0.009741436391826452,9.341307,9.809969,15,G1,1038.162241205573,8,10,2036.0,981.0,28.68369351669941,7.170923379174853,0.6229043973488843,False,False,4.699999999999999,15.100548,11.025036,1113.0,10.0,8,0.8937778591737116
-874,True,TTTGTTGAGCACCTGC-1,TTTGTTGAGCACCTGC-1,2.0016144480294282,15.0,-0.0571163442666586,-0.03435010721624269,-6.068311,2.2698145,14,G1,1506.447009652853,4,4,8634.0,2853.0,6.636553161917998,13.458420199212416,0.9459527397869153,False,False,4.8,-14.5372925,1.1269181,2632.0,9.0,4,1.5993908315924596
-875,True,TTTGTTGAGCTCACTA-1,TTTGTTGAGCTCACTA-1,1.9736209768879647,15.0,-0.06479071304707006,-0.03400256284493233,-5.314267,-8.256545,13,G1,1169.837094038725,2,2,10706.0,3336.0,8.910891089108912,14.272370633289745,0.9528409114681339,False,False,3.0000000000000004,-16.47763,-7.799421,3107.0,9.0,2,0.9526172425755786
-876,True,TTTGTTGAGGGAGGTG-1,TTTGTTGAGGGAGGTG-1,2.0138076763775508,15.0,-0.07509019714854476,-0.02770749938230492,-7.732722,0.25982577,4,G1,1509.3713083565235,4,4,5533.0,2242.0,9.958431230797036,11.115127417314296,0.9694846788513838,False,False,4.8,-15.318817,-0.3536881,2133.0,9.0,4,1.5553820261544036
-877,True,TTTGTTGAGTGGCCTC-1,TTTGTTGAGTGGCCTC-1,1.9104197268944285,15.0,-0.11032744758679827,0.0037365960671110303,5.007131,6.5467052,12,S,1298.165541909635,1,1,4182.0,1270.0,7.388809182209469,45.2415112386418,0.6121347732725128,False,False,4.8,9.621283,6.9951615,1172.0,9.0,1,0.8054144824310264
-878,True,TTTGTTGCAAGAGAGA-1,TTTGTTGCAAGAGAGA-1,1.9019245679658685,15.0,-0.09791754972812704,-0.015742645667559795,2.6460757,4.100134,12,G1,1842.1758461147547,1,1,3389.0,1126.0,7.170256712894659,42.01829448214813,0.6133079864408679,False,False,4.2,11.495862,10.216359,1800.0,9.0,1,0.8054144824310264
-879,True,TTTGTTGCAAGTAGTA-1,TTTGTTGCAAGTAGTA-1,1.968728554080061,15.0,-0.08221014213225711,-0.02256306442708976,-8.123048,-3.0154252,4,G1,1697.3799031078815,4,3,11872.0,3368.0,8.372641509433961,16.307277628032345,0.9745183450741869,False,False,3.2,-14.509166,-3.350533,2405.0,9.0,4,0.8054144824310264
-881,True,TTTGTTGCAGGTTCAT-1,TTTGTTGCAGGTTCAT-1,1.9157857543664985,15.0,-0.076752258059448,-0.0290014820493784,7.9043465,-6.919629,11,G1,950.9687501788139,7,6,14878.0,4223.0,6.9565801855088045,17.838419142357843,0.3030912308265342,False,True,3.6,14.842916,-6.789481,2122.0,9.0,7,1.1231825187212259
-882,True,TTTGTTGGTAACGCGA-1,TTTGTTGGTAACGCGA-1,1.943312629895224,15.0,-0.0932854322139371,-0.014591335418098445,-8.301857,-3.7428346,10,G1,1434.5371922850609,3,3,8238.0,2842.0,7.853848021364409,15.768390386016023,0.970242342942307,False,False,3.0999999999999996,-14.013627,-3.7452972,4123.0,9.0,3,1.0447717255928088
-883,True,TTTGTTGTCATCGCTC-1,TTTGTTGTCATCGCTC-1,1.957806326704775,15.0,-0.06847312759986651,-0.016089844252105388,-6.030591,-1.6460114,9,G1,1472.9912744164467,2,2,16973.0,4248.0,5.962410887880751,19.324809992340775,0.9619333826362116,False,False,2.9000000000000004,-15.946691,-1.0646045,1441.0,9.0,2,1.2441389063273753
-884,True,TTTGTTGTCCAATCCC-1,TTTGTTGTCCAATCCC-1,1.9075688473238475,15.0,-0.10661440192694789,-0.015417194682835741,8.255973,1.1994703,8,G1,1101.9708822071552,9,9,7113.0,1939.0,9.475608041613945,37.48066919724448,0.5886626988052546,False,False,2.7,13.960351,5.5284853,1020.0,9.0,9,0.8054144824310264
-885,True,TTTGTTGTCCATTGCC-1,TTTGTTGTCCATTGCC-1,1.9504314105868148,15.0,-0.06561495271036624,-0.021414982917128764,-9.2922535,-2.7916152,7,G1,1550.6583121269941,3,3,11578.0,3219.0,6.555536362065987,16.773190533770943,0.9863150086644521,False,False,4.199999999999999,-15.460365,-3.6372786,2273.0,9.0,3,1.4568611182082167
-886,True,TTTGTTGTCCTACAAG-1,TTTGTTGTCCTACAAG-1,1.9251247027646166,15.0,-0.1145327893365303,0.0015631880818967638,5.6546464,4.15667,6,S,1612.6362722963095,1,1,7246.0,1800.0,9.205078664090532,43.058239028429476,0.610292287787697,False,False,4.5,12.751787,7.777711,954.0,9.0,1,1.4332994794648768
-887,True,TTTGTTGTCCTTCACG-1,TTTGTTGTCCTTCACG-1,2.029579151840361,15.0,-0.046387429305931836,-0.04263561169921185,-0.17627867,7.842231,5,G1,1249.5995611995459,1,1,1594.0,861.0,21.64366373902133,6.900878293601004,0.6584404132656012,False,False,4.3999999999999995,8.735247,13.291577,1036.0,9.0,1,1.4515487513862504
-888,True,TTTGTTGTCGACGCGT-1,TTTGTTGTCGACGCGT-1,1.9939973652769272,15.0,-0.0635679237988607,-0.016249270797186542,-6.9197288,2.4502618,4,G1,1479.2349963188171,4,4,5164.0,2090.0,13.129357087529048,13.381099922540667,0.951128413152932,False,False,5.300000000000001,-15.084896,1.495179,1729.0,9.0,4,1.4047135774338826
-889,True,TTTGTTGTCGCCACTT-1,TTTGTTGTCGCCACTT-1,1.9445120367680255,15.0,-0.0839984809134657,-0.02202600184367002,-5.9126496,-7.3615575,3,G1,1484.7777071297169,2,2,14166.0,3808.0,6.1979387265283075,16.476069462092333,0.9518926392563122,False,False,4.6000000000000005,-15.102908,-7.110796,2616.0,9.0,2,0.8054144824310264
-890,True,TTTGTTGTCGTAGAGG-1,TTTGTTGTCGTAGAGG-1,1.9710957290948403,15.0,-0.11861214520476623,-0.018631847448560477,-2.5691123,14.208008,2,G1,1241.0094733685255,5,5,6851.0,1744.0,11.210042329586921,41.789519778134576,0.7322556311436602,False,False,7.6,1.1887627,14.585867,824.0,9.0,5,1.3272521773497132
-891,True,TTTGTTGTCTCTAGGA-1,TTTGTTGTCTCTAGGA-1,1.9628245783546727,15.0,-0.06286823249838676,-0.028605509095688937,-9.306996,-3.4692883,1,G1,1559.950357273221,3,3,11197.0,3392.0,6.001607573457176,14.584263642046977,0.9871437083080061,False,True,3.2,-16.988478,-4.3092875,2439.0,9.0,3,1.8941064638830696
diff --git a/scarf/tests/datasets/knn_distances.npy b/scarf/tests/datasets/knn_distances.npy
deleted file mode 100644
index 27278c7f..00000000
Binary files a/scarf/tests/datasets/knn_distances.npy and /dev/null differ
diff --git a/scarf/tests/datasets/knn_indices.npy b/scarf/tests/datasets/knn_indices.npy
deleted file mode 100644
index e7e30175..00000000
Binary files a/scarf/tests/datasets/knn_indices.npy and /dev/null differ
diff --git a/scarf/tests/datasets/knn_weights.npy b/scarf/tests/datasets/knn_weights.npy
deleted file mode 100644
index 8ab43758..00000000
Binary files a/scarf/tests/datasets/knn_weights.npy and /dev/null differ
diff --git a/scarf/tests/datasets/markers_cluster1.csv b/scarf/tests/datasets/markers_cluster1.csv
deleted file mode 100644
index b4db3e65..00000000
--- a/scarf/tests/datasets/markers_cluster1.csv
+++ /dev/null
@@ -1,23 +0,0 @@
-,group_id,feature_name,feature_index,score,mean,mean_rest,frac_exp,frac_exp_rest,fold_change,p_value
-0,1,LEF1,8752.0,0.33122,0.2967,0.04111,0.73571,0.12725,7.21684,1.3468492583919777e-56
-1,1,CCR7,29040.0,0.31948,0.31051,0.05481,0.71429,0.1482,5.6654,1.303081203640238e-48
-2,1,PASK,5916.0,0.3078,0.08755,0.0126,0.27857,0.06587,6.94808,2.959313972605772e-15
-3,1,FAM153CP,11107.0,0.30597,0.09362,0.01476,0.3,0.08084,6.34314,9.01971057038007e-15
-4,1,CAMK4,10283.0,0.3032,0.12619,0.02217,0.44286,0.11078,5.69199,6.432216159402198e-24
-5,1,OXNAD1,6124.0,0.29315,0.2405,0.04098,0.60714,0.17814,5.86813,4.561360581110357e-32
-6,1,PDE3B,19299.0,0.28502,0.16879,0.03557,0.51429,0.18713,4.74522,4.607808821916747e-21
-7,1,RCAN3,547.0,0.28094,0.22673,0.0463,0.64286,0.21407,4.89703,1.1497416494441125e-31
-8,1,PIK3IP1,34866.0,0.27789,0.31654,0.06082,0.72143,0.26347,5.20433,3.204752190325667e-35
-9,1,PCED1B,21585.0,0.27194,0.0985,0.01763,0.31429,0.09132,5.58611,3.962141555008794e-14
-10,1,ANK3,18064.0,0.26989,0.05188,0.01045,0.21429,0.0494,4.9669,8.042095391236527e-12
-11,1,FHIT,6642.0,0.26883,0.06146,0.01148,0.26429,0.06437,5.35291,6.378882432475336e-14
-12,1,LDLRAP1,568.0,0.26668,0.1414,0.03687,0.52857,0.17066,3.83499,4.9506042495638135e-23
-13,1,TCF7,10484.0,0.26362,0.35899,0.06701,0.75714,0.22455,5.35724,1.18335482474118e-40
-14,1,TRABD2A,4310.0,0.26222,0.11259,0.0285,0.47857,0.1003,3.95082,4.288612022936954e-28
-15,1,TRAT1,6877.0,0.2622,0.1261,0.0252,0.42143,0.11527,5.00337,5.385352981156799e-20
-16,1,CHRM3-AS2,3269.0,0.2597,0.07844,0.01529,0.3,0.05988,5.13003,3.340274876456977e-18
-17,1,TBC1D4,23357.0,0.25868,0.05019,0.01044,0.2,0.07036,4.806,1.9327088042662477e-07
-18,1,PRKCQ-AS1,17615.0,0.25857,0.36278,0.07586,0.76429,0.21108,4.78202,6.628889398177008e-42
-19,1,TMEM204,26536.0,0.25476,0.05645,0.01091,0.27857,0.06287,5.17677,2.2915637229641518e-15
-20,1,NELL2,21563.0,0.25295,0.06383,0.01107,0.3,0.05389,5.76786,1.776913554122277e-19
-21,1,BCL11B,24832.0,0.2506,0.54998,0.12834,0.88571,0.26198,4.2854,1.6507082668410988e-48
diff --git a/scarf/tests/datasets/pseudotime_clusters.npy b/scarf/tests/datasets/pseudotime_clusters.npy
deleted file mode 100644
index 71a2743b..00000000
Binary files a/scarf/tests/datasets/pseudotime_clusters.npy and /dev/null differ
diff --git a/scarf/tests/datasets/pseudotime_markers_r_values.csv b/scarf/tests/datasets/pseudotime_markers_r_values.csv
deleted file mode 100644
index e4dbec64..00000000
--- a/scarf/tests/datasets/pseudotime_markers_r_values.csv
+++ /dev/null
@@ -1,10737 +0,0 @@
-,names,I__RNA_pseudotime__r
-6,AL627309.5,0.1230549217056546
-14,LINC01409,0.02697594296065394
-16,LINC01128,-0.06488185841951406
-24,NOC2L,-0.005567253956048132
-29,HES4,0.06671517782535455
-30,ISG15,0.23636363748713451
-42,TNFRSF18,-0.21417934390153354
-43,TNFRSF4,-0.15897884988130603
-44,SDF4,-0.039933597761962
-45,B3GALT6,-0.040967828909551175
-48,UBE2J2,-0.014120672687761925
-51,ACAP3,0.13718386481836156
-52,PUSL1,0.0068323665641420415
-53,INTS11,-0.02113976481616301
-55,CPTP,-0.06969205159494302
-57,DVL1,0.023184020126135272
-59,AURKAIP1,-0.029504706616703496
-60,CCNL2,0.045257420497981184
-61,MRPL20-AS1,-0.07333460013050093
-62,MRPL20,-0.05875118335163438
-70,ATAD3B,0.007633271807662926
-71,ATAD3A,0.019501792053005115
-73,SSU72,-0.00031632196672365924
-74,AL645728.1,-0.06300234164385085
-75,FNDC10,0.05980393758077972
-77,AL691432.2,0.029970291678462907
-78,MIB2,-0.24014760143053754
-80,CDK11B,-0.05215958375284411
-82,SLC35E2B,-0.05736428867890881
-83,CDK11A,-0.11250500821854219
-84,SLC35E2A,-0.041969797992089164
-85,NADK,0.3777765390075238
-86,GNB1,0.2506066088252328
-97,FAAP20,0.012513316457367767
-99,SKI,-0.0033973857506125623
-104,RER1,-0.09063835287590319
-105,PEX10,-0.0003860865930343741
-109,PANK4,0.06402702012057145
-111,AL139246.5,-0.11380095908753837
-112,TNFRSF14-AS1,-0.02571978055231639
-113,TNFRSF14,0.09163652045622896
-115,PRXL2B,-0.039661811088409285
-136,TPRG1L,0.03398381277567639
-137,WRAP73,-0.02672456817716085
-143,LRRC47,0.007645878335275087
-145,CEP104,0.06615876347907368
-146,DFFB,-0.05457637807639354
-147,C1orf174,-0.049844654360918395
-168,KCNAB2,-0.04808576132692299
-170,RPL22,-0.236210517827778
-173,ICMT,0.034638232433495564
-177,ACOT7,-0.09698521249600359
-182,TNFRSF25,-0.15352758246249806
-184,NOL9,0.0005619370160748977
-186,ZBTB48,0.024292385599682133
-187,KLHL21,-0.019966391627239993
-188,PHF13,-0.019463482818329037
-189,THAP3,0.0019628436548927773
-190,DNAJC11,0.004571992880053765
-194,CAMTA1,0.10346889792931625
-202,VAMP3,0.21812487861858856
-204,PER3,-0.04522825714047427
-208,PARK7,-0.10012148047778027
-216,RERE,0.22735127970261543
-219,ENO1,0.1391947774208346
-227,H6PD,0.045854756434876395
-232,SLC25A33,-0.025720955285396578
-234,PIK3CD,0.06495424022804244
-237,CLSTN1,-0.10360510382397015
-238,CTNNBIP1,-0.09460321131278472
-241,LZIC,-0.005256521828738424
-243,NMNAT1,0.07758944078179397
-245,RBP7,0.3554503453866769
-246,UBE4B,0.08170302928451648
-247,KIF1B,0.17589793182073735
-250,PGD,0.4400119901074925
-254,DFFA,0.05948297214194947
-256,PEX14,-0.02054688471217005
-258,CASZ1,0.04452592790114171
-261,TARDBP,-0.053910779294433095
-263,AL109811.2,-0.08985000133016134
-264,SRM,-0.0933477759816086
-265,EXOSC10,-0.016006769020427632
-268,MTOR,0.03613929608744564
-271,UBIAD1,0.04181058517497577
-278,FBXO44,-0.143642892475068
-279,FBXO6,-0.008588845313193313
-280,MAD2L2,-0.08598847881205167
-282,AGTRAP,0.2985823142605977
-285,MTHFR,0.13992756662582434
-286,CLCN6,0.13029517829098325
-289,AL021155.5,0.2118140977687788
-290,KIAA2013,0.07206094552080566
-291,PLOD1,0.11796094290853949
-293,MFN2,0.11952713801693134
-294,MIIP,0.11659202671286495
-295,TNFRSF8,0.1435668960400651
-297,TNFRSF1B,0.47907507726847964
-298,VPS13D,0.02450165123222427
-300,DHRS3,-0.3001979867946025
-338,PRDM2,0.026335074461534404
-344,TMEM51,0.19520669774466087
-350,EFHD2,-0.04198061434469568
-354,CASP9,-0.007058581171555149
-355,DNAJC16,0.0858229787284598
-357,AGMAT,-0.07363005354763731
-359,DDI2,-0.012351059041843245
-362,PLEKHM2,0.11798320093760765
-369,AL450998.2,-0.08571108857869655
-370,SPEN,0.11381785156085569
-371,ZBTB17,0.03377566474099116
-385,FBXO42,0.08648134145750665
-386,SZRD1,-0.033362919469690396
-388,NECAP2,-0.03635147647835925
-393,NBPF1,-0.04882800173144501
-398,CROCC,-0.07694291369395005
-401,BX284668.5,0.022571240396238732
-406,ATP13A2,-0.03738901234356289
-407,SDHB,0.05017363273521305
-408,PADI2,0.1373171029292057
-413,PADI4,0.23251315570147651
-416,RCC2,0.13444352641100238
-417,ARHGEF10L,0.27667702832331054
-429,IFFO2,-0.11203573504000285
-431,UBR4,0.09485488144299467
-433,EMC1,0.015196811205233338
-434,MRTO4,-0.012466719343090384
-436,AKR7A2,-0.041727984072374484
-437,SLC66A1,0.09528782708719437
-438,CAPZB,0.08932824478362296
-439,MICOS10,-0.01386066241508532
-443,TMCO4,-0.02853236100540018
-446,OTUD3,-0.06596742216326937
-465,MUL1,0.0058776915639362996
-467,CDA,0.37783580502154923
-468,PINK1,0.13922519884111875
-470,DDOST,-0.13505339050878262
-474,HP1BP3,-0.010565498951759492
-475,EIF4G3,0.048648410701711185
-477,ECE1,0.1545263184326597
-485,USP48,0.1355832894754495
-491,LINC00339,0.044434902727545335
-492,CDC42,0.05970505965291489
-494,CDC42-IT1,0.11079686982431751
-497,ZBTB40,-0.06667110701980225
-507,KDM1A,-0.007217550325998549
-509,LUZP1,-0.09447161858203226
-511,LINC01355,-0.009535755163188747
-513,HNRNPR,0.04513786171893914
-517,TCEA3,-0.0723311604405331
-521,ID3,-0.07075408554332689
-523,MDS2,-0.0624758255795342
-524,RPL11,-0.3521296662998126
-526,ELOA,0.04975837717528601
-527,PITHD1,-0.16402276074599387
-528,LYPLA2,-0.021297180180469135
-529,GALE,0.011795680860376063
-530,HMGCL,-0.041319989705999725
-531,FUCA1,0.1365353249929212
-532,CNR2,-0.04311906877377416
-534,PNRC2,-0.03739976099394955
-535,SRSF10,-0.03870922905211816
-545,NIPAL3,-0.11299906558104254
-547,RCAN3,-0.1546415624599943
-548,NCMAP-DT,-0.07271399828589718
-550,SRRM1,-0.06357989463992358
-552,CLIC4,0.2505700865392326
-553,RUNX3,-0.443379061726743
-559,SYF2,0.013113557526173039
-562,RSRP1,0.07523694483873336
-565,TMEM50A,0.05938580537035053
-567,MACO1,-0.0977749993723366
-568,LDLRAP1,-0.13546153921067158
-570,MAN1C1,-0.06109922662660458
-571,AL031280.1,-0.05276257299899762
-572,SELENON,0.09269587918026835
-574,MTFR1L,0.065471317426969
-578,PAQR7,0.08276066475676322
-579,STMN1,-0.15678612376166642
-580,PAFAH2,-0.12707976235154347
-585,PDIK1L,0.008603173774090778
-589,ZNF593,0.0345500296356332
-592,CEP85,-0.0203318387036094
-593,SH3BGRL3,0.11805054308154371
-594,UBXN11,0.19531207547041204
-595,CD52,-0.2859481438086418
-599,DHDDS,0.046911952631789905
-601,HMGN2,0.015894416800225814
-602,RPS6KA1,0.1341334836714293
-604,ARID1A,0.19969715471983238
-605,PIGV,-0.0013645436924121595
-606,ZDHHC18,-0.04197998375283504
-608,GPN2,0.060770212078623674
-610,GPATCH3,0.0443430456718138
-611,NUDC,-0.16531331508572075
-616,SLC9A1,0.045098465474636244
-618,WDTC1,-0.018210496908345957
-619,TMEM222,-0.1396495053789517
-620,SYTL1,-0.3043601546708943
-621,MAP3K6,-0.05117118588239713
-625,WASF2,0.18987379362159923
-628,AHDC1,-0.0590461543721751
-629,FGR,0.42475906254103013
-631,IFI6,0.333186637849025
-634,FAM76A,-0.03839817624399984
-635,STX12,0.2404357866351109
-639,PPP1R8,-0.05479117674744419
-640,THEMIS2,0.4007152306320804
-641,RPA2,-0.2954480735175474
-644,XKR8,0.08750986952815397
-645,EYA3,0.012092667850428788
-646,PTAFR,0.41544314993122294
-647,DNAJC8,-0.12634769603595938
-648,ATP5IF1,-0.3213303137954154
-649,AL353622.1,-0.08396144513454108
-651,SESN2,0.0492211928820459
-652,MED18,-0.042886918170366294
-653,PHACTR4,0.05081690291893618
-654,RCC1,0.04433811240705384
-655,SNHG3,-0.0908858955753079
-656,TRNAU1AP,-0.12519955633062738
-657,SNHG12,-0.16567442822269965
-658,TAF12,-0.052789562928356244
-661,AL360012.1,0.10478296033865367
-662,GMEB1,-0.03805970938047825
-663,YTHDF2,0.030946087428810144
-666,EPB41,-0.2153544543816789
-669,SRSF4,0.055411410681447004
-670,MECR,0.0036921106172176027
-681,MATN1-AS1,-0.038909331258823815
-683,LAPTM5,0.4679250936659717
-687,SDC3,0.13638723709006054
-688,PUM1,0.21661077512036042
-690,SNRNP40,-0.10983494996911104
-691,ZCCHC17,-0.029476186162603703
-699,PEF1,-0.04820455599839965
-708,PTP4A2,0.23356626881075868
-711,KHDRBS1,0.11244545932092129
-713,TMEM39B,0.07029023545338424
-714,KPNA6,0.049332727871123895
-716,TXLNA,0.01727459129312241
-717,CCDC28B,-0.2593965388814996
-721,TMEM234,-0.03346104239695377
-722,EIF3I,-0.09509860821216531
-724,LCK,-0.5051305776916664
-725,HDAC1,-0.1273041572505737
-726,MARCKSL1,0.009874320863173346
-730,BSDC1,-0.03265118353167306
-733,ZBTB8OS,-0.09063856688000357
-734,RBBP4,0.08154028097827945
-735,SYNC,-0.11583267395200326
-738,YARS,-0.044084655044928096
-739,S100PBP,-0.11940850106959405
-744,RNF19B,0.2297778434021784
-746,AK2,0.1353866446983729
-750,ZNF362,0.09899506971809227
-754,PHC2,0.15142815141994492
-762,SMIM12,0.01369673908911601
-770,TMEM35B,-0.2428063657375646
-771,ZMYM6,0.12480569910449067
-772,ZMYM1,-0.0024400541977930602
-774,SFPQ,0.13612612993620762
-775,ZMYM4,0.12354360535253361
-777,KIAA0319L,0.16819319865842008
-778,NCDN,0.054407980509276886
-779,AC004865.2,-0.14652531514469286
-781,PSMB2,0.054687176724772045
-782,C1orf216,-0.1291107641125408
-785,AGO4,0.3096421783111556
-786,AGO1,0.11630040401184202
-789,AGO3,0.10565962888205979
-792,ADPRHL2,0.03131540761274148
-793,COL8A2,0.13310814576698693
-794,TRAPPC3,-0.042783151968809725
-795,MAP7D1,-0.01873385505899189
-796,THRAP3,-0.10663277815072107
-798,EVA1B,0.0866028233653545
-800,STK40,0.09130010089409536
-801,LSM10,0.0765447131511328
-803,MRPS15,-0.008835141781899059
-804,CSF3R,0.47582557218353894
-812,ZC3H12A,-0.025846121989861387
-813,MEAF6,-0.18295529910853137
-814,SNIP1,-0.06727665190497625
-816,GNL2,0.04362277410521528
-819,C1orf109,-0.09753718506652714
-824,YRDC,-0.09098009807422307
-825,C1orf122,-0.022112928580704076
-826,MTF1,0.2103586886500932
-828,INPP5B,0.02001845868562048
-829,SF3A3,0.014406568206325493
-830,FHL3,0.09395212917496838
-831,UTP11,0.058845182653001996
-840,RRAGC,0.11156200179882095
-842,MYCBP,0.1444173490634703
-845,AKIRIN1,-0.016466751358024133
-846,NDUFS5,-0.010276694420487004
-847,MACF1,-0.1596830646954101
-852,PABPC4,0.14099640859043872
-858,PPIE,-0.05975264866179944
-865,TRIT1,0.12568773772199582
-866,MYCL,0.25104435123173013
-868,MFSD2A,0.15427884316991602
-869,CAP1,0.29972506709527996
-870,PPT1,0.47124542163685446
-871,RLF,0.10199497539192409
-873,AL050341.2,-0.026682253912128324
-874,ZMPSTE24,0.1409732880330726
-875,COL9A2,0.15403674716452606
-876,SMAP2,0.3283344183536616
-880,ZFP69,-0.09571657298990982
-882,EXO5,-0.007230376951609572
-885,ZNF684,0.03358600257179506
-891,NFYC,0.10164193001459686
-893,CITED4,0.0671532015304515
-896,CTPS1,-0.009482371046693593
-899,SCMH1,-0.006574362503734779
-905,HIVEP3,0.030880578831552933
-911,FOXJ3,-0.11705452427200572
-915,PPCS,0.05503323930736351
-916,CCDC30,0.039944925314208936
-917,PPIH,-0.0400107740439354
-920,YBX1,0.2331984163639987
-922,P3H1,0.07453255867459331
-923,C1orf50,-0.08665257411753087
-926,SVBP,0.0515949412664668
-927,ERMAP,0.1337533285394462
-929,ZNF691,-0.05423299666950909
-930,SLC2A1,-0.1778396979504289
-934,EBNA1BP2,-0.09828858473076153
-943,ELOVL1,-0.01715667661499036
-944,MED8,-0.05799311093153786
-946,SZT2,0.08845231213928195
-948,HYI,0.021303071367521617
-951,KDM4A,0.007750325510282202
-953,ST3GAL3,-0.02473636082489289
-957,IPO13,0.06429319488868504
-959,DPH2,-0.01887336098927653
-960,ATP6V0B,0.443302897104421
-967,DMAP1,-0.04350089222311067
-968,ERI3,-0.049605732310712136
-970,RNF220,0.02523068446703389
-972,ARMH1,0.01681325468282223
-975,RPS8,-0.2885976428911654
-977,PLK3,0.18626530558304363
-980,PTCH2,0.13418766076727154
-981,EIF2B3,-0.039138482790820335
-982,HECTD3,0.06099141970403076
-983,UROD,0.054041784768249834
-987,MUTYH,-0.06434839777990788
-988,TOE1,-0.0923645041874432
-989,TESK2,0.09681289816837337
-992,PRDX1,0.12519723911666208
-993,AKR1A1,0.15040454677555024
-994,NASP,-0.025558913454575078
-996,GPBP1L1,0.062343207846760985
-997,TMEM69,0.0530467414137923
-998,IPP,-0.07626732315920767
-1000,MAST2,0.10202662452714799
-1006,POMGNT1,-0.046330196126079644
-1009,LRRC41,0.03501280236425198
-1010,UQCRH,-0.032844634720913875
-1011,NSUN4,0.10587899156409974
-1012,FAAH,0.1010766404625462
-1018,MKNK1,0.04853177818365361
-1019,MOB3C,0.08893456036103724
-1020,ATPAF1,-0.004066493768567328
-1023,EFCAB14,0.07602766451248576
-1035,CMPK1,0.0012916334326167204
-1049,SPATA6,0.1246181054072658
-1052,BEND5,-0.00955397115151429
-1064,FAF1,-0.07116391230445514
-1066,CDKN2C,0.0014144130834979418
-1069,RNF11,0.06246464199054348
-1073,EPS15,0.1410045347063142
-1076,OSBPL9,0.07742288060247088
-1077,NRDC,-0.023256606583312578
-1081,TXNDC12,-0.0009449990125919163
-1082,KTI12,-0.04527880425174545
-1085,BTF3L4,0.14844410630317828
-1087,CC2D1B,0.06256685192176688
-1091,PRPF38A,-0.11039641872673985
-1092,TUT4,-0.24507557300975935
-1094,GPX7,-0.11596291754669838
-1095,SHISAL2A,-0.1677011635017399
-1096,COA7,0.061975290665355266
-1097,ZYG11B,0.07909048850700386
-1100,ECHDC2,-0.1314668856512397
-1101,SCP2,-0.09057797001358026
-1106,CPT2,0.11366083312481434
-1108,CZIB,-0.08376016732814294
-1110,MAGOH,-0.1866368534115611
-1112,LRP8,0.0257608418375831
-1119,LINC02812,0.009279852303847261
-1122,NDC1,0.0344100920185222
-1123,YIPF1,0.09828560500743282
-1125,HSPB11,-0.1743519216965921
-1126,LRRC42,-0.013176444210681868
-1128,TMEM59,-0.10451064508961974
-1129,TCEANC2,0.09901024894936104
-1133,MRPL37,0.05143282698286113
-1134,SSBP3,-0.16879238009449665
-1144,TTC4,-0.06268326975161274
-1158,USP24,0.028682784058302036
-1177,OMA1,-0.0033358509274704923
-1180,MYSM1,0.015969381795219444
-1182,JUN,0.17302807853647978
-1194,HOOK1,-0.06492444063724286
-1200,NFIA,0.07857369488388166
-1205,TM2D1,0.02044531563015189
-1207,PATJ,-0.11912565494359523
-1210,USP1,-0.013546011283409638
-1211,DOCK7,0.03292647946419866
-1214,ATG4C,0.05427593296523082
-1225,ALG6,-0.03155681713450471
-1226,ITGB3BP,-0.0015932796199634237
-1230,PGM1,0.021121173652675938
-1237,JAK1,-0.34740017651946964
-1243,LEPROT,0.24197799687232788
-1247,PDE4B,-0.01707736008319981
-1255,MIER1,-0.020165101226946647
-1256,SLC35D1,-0.13687504872128373
-1261,SERBP1,-0.023938849952812435
-1263,GADD45A,-0.0032990416727488392
-1267,WLS,0.20800104481321738
-1282,LRRC40,-0.05493551620636063
-1283,SRSF11,-0.012560099802708335
-1285,ANKRD13C,0.06008367441392752
-1296,ZRANB2,-0.19056029390311033
-1309,FPGT,0.08818669165704399
-1317,CRYZ,-0.1321077867094611
-1318,TYW3,-0.019571042315263146
-1325,ACADM,0.01435775165835368
-1326,RABGGTB,0.040324105773069996
-1336,PIGK,-0.07806500507221757
-1338,AK5,-0.20843807640171724
-1341,AC118549.1,-0.024320200975218458
-1342,USP33,0.09864250692309813
-1343,MIGA1,-0.0455560268593408
-1345,NEXN,0.16677504094961393
-1346,FUBP1,-0.05836851139080973
-1347,DNAJB4,-0.00339277528733327
-1349,AC103591.3,0.15735806081551837
-1353,IFI44L,0.3711302764621523
-1354,IFI44,0.3176279459708511
-1363,LINC01781,0.0023198944259790436
-1380,PRKACB,-0.2939579677030131
-1384,RPF1,0.004090458107404308
-1385,GNG5,0.2096155130025133
-1387,CTBS,0.18811612213587883
-1391,SSX2IP,-0.056520620429776895
-1394,MCOLN2,-0.08071082658680018
-1398,C1orf52,-0.03074797426699041
-1399,BCL10,0.07941219693710325
-1407,ZNHIT6,-0.1365869382564246
-1410,ODF2L,-0.19670874297287622
-1416,SH3GLB1,0.08096949321535866
-1418,SELENOF,-0.007795034541650271
-1419,HS2ST1,0.0044825080046648254
-1423,LMO4,0.08979342870561986
-1427,PKN2,0.18748452500106594
-1428,GTF2B,-0.013183305851795935
-1429,KYAT3,0.034803642363612296
-1430,RBMXL1,-0.008284572822916704
-1431,GBP3,0.09720532035249377
-1432,GBP1,0.3452450852926635
-1433,GBP2,0.3842171606000256
-1436,GBP4,0.2815849589682886
-1437,AC099063.4,-0.11368677068902676
-1438,GBP5,0.2148605754384967
-1442,LRRC8B,0.13903818375912283
-1443,LRRC8C-DT,-0.031858937725204255
-1445,LRRC8C,0.12869878397355683
-1448,LRRC8D,0.17409105524279866
-1450,ZNF326,0.006280514757241575
-1461,ZNF644,-0.037404425867191525
-1464,TGFBR3,-0.3882886524779984
-1468,BTBD8,0.016618276547262802
-1471,GLMN,-0.16769656701719501
-1472,RPAP2,-0.17054093115556165
-1474,EVI5,0.3270945056160331
-1475,RPL5,-0.40871562680410606
-1476,DIPK1A,-0.09176863521795961
-1478,MTF2,-0.007298385309597535
-1479,TMED5,-0.004971397653712314
-1480,CCDC18,0.06141494126681908
-1481,CCDC18-AS1,-0.04965677191389404
-1482,DR1,0.21147914981841964
-1487,DNTTIP2,0.03625851701669107
-1488,GCLM,-0.08431685828155337
-1495,ABCD3,-0.0795500226354738
-1501,ALG14,-0.04841466236474992
-1506,RWDD3,-0.09622041185693504
-1517,PTBP2,0.0469629302625801
-1518,DPYD,0.42093917703796907
-1533,AGL,-0.08797839003289
-1535,SLC35A3,0.0008301271666048649
-1536,MFSD14A,0.006444876810803581
-1538,SASS6,-0.05279770415367116
-1539,TRMT13,-0.08198597956628349
-1541,DBT,0.007177681428351698
-1543,RTCA-AS1,-0.005616975329124829
-1544,RTCA,-0.007978571026859296
-1546,CDC14A,-0.22848396969594262
-1554,SLC30A7,0.05069320517747298
-1555,DPH5,-0.06570813626434331
-1557,AC093157.1,-0.07494748785469671
-1559,S1PR1,-0.2789643669697766
-1568,RNPC3,-0.030718090568228166
-1579,PRMT6,-0.05600066592478537
-1581,VAV3,-0.034130438428482
-1583,LINC02785,0.14030445998429134
-1584,SLC25A24,0.19941852070687732
-1590,FAM102B,0.15704029117160623
-1591,HENMT1,-0.2689955328157962
-1593,PRPF38B,-0.05953467960000291
-1596,STXBP3,0.15170639908596212
-1599,GPSM2,-0.04897909357660194
-1600,CLCC1,0.07019459452104725
-1601,WDR47,0.02561704876145906
-1602,TAF13,0.040861776965014945
-1604,TMEM167B,0.19430794902606022
-1608,SARS,-0.09259653454824265
-1610,PSRC1,0.07198337826651541
-1612,SORT1,0.39178755007903593
-1613,PSMA5,-0.09406513953471722
-1616,CYB561D1,-0.04673108425680487
-1620,GNAI3,0.12653083304636528
-1623,AMPD2,0.16160137362618554
-1625,GSTM4,0.024584267797337054
-1626,GSTM2,-0.04788791597569748
-1631,GSTM3,-0.011592687260680391
-1637,AHCYL1,0.17203308213531657
-1638,STRIP1,0.09552756211274767
-1649,RBM15,-0.007477965111677607
-1650,LAMTOR5-AS1,-0.03219690439689876
-1651,SLC16A4,-0.1216738372603351
-1652,AL355488.1,-0.0016838267738990538
-1653,LAMTOR5,0.054566764431779186
-1660,AL365361.1,-0.12803915775743088
-1661,KCNA3,-0.04688197748573514
-1662,CD53,-0.012033041593474353
-1665,LRIF1,-0.013335750290024352
-1667,DRAM2,0.13626764958487877
-1668,CEPT1,0.14355736191537305
-1670,AL355816.2,-0.07066290090154864
-1671,DENND2D,-0.2609870158109731
-1672,CHI3L2,-0.1642259535439208
-1679,WDR77,0.043476972105719616
-1680,ATP5PB,0.05076359413922439
-1681,C1orf162,0.32358069131373585
-1684,RAP1A,0.0648204341243554
-1686,INKA2,0.12976797528194542
-1689,DDX20,0.01571916120808659
-1695,CTTNBP2NL,0.11808386363050709
-1698,ST7L,-0.07055033741854601
-1699,CAPZA1,0.24801354494737946
-1700,MOV10,0.005570535761832372
-1702,RHOC,-0.09836513341351112
-1708,SLC16A1,0.01977664646162683
-1709,SLC16A1-AS1,-0.004133305562075322
-1712,LRIG2,0.02439341004747944
-1714,PHTF1,-0.008778105460234855
-1715,RSBN1,-0.10157379024789476
-1717,PTPN22,-0.2617828463659572
-1720,AP4B1,-0.046857163519514605
-1721,DCLRE1B,0.09871411248439374
-1723,HIPK1,0.12616833798293883
-1728,TRIM33,0.027473447984524054
-1729,BCAS2,-0.2249348311907223
-1732,NRAS,0.12593563696865961
-1733,CSDE1,0.25285887303437243
-1734,SIKE1,-0.029430252612593844
-1737,TSPAN2,-0.1400589886846058
-1747,SLC22A15,0.19020909409331463
-1752,ATP1A1,0.06946492102908475
-1758,CD58,0.06525207555902474
-1761,CD2,-0.5266817468161709
-1764,CD101,0.20343048492344443
-1766,TTF2,0.0005019683276468543
-1771,MAN1A2,0.034123043802422336
-1773,TENT5C,-0.014318861365760108
-1774,GDAP2,0.16932417788174703
-1775,WDR3,-0.01589971377240118
-1781,WARS2,-0.025428709984878
-1783,WARS2-AS1,-0.04212085717849835
-1791,ZNF697,0.12483226137362713
-1796,NOTCH2,0.4089965092621748
-1798,SEC22B,0.04171545223585034
-1800,NBPF26,0.15122286685324252
-1801,AC253572.2,0.2986804356320298
-1803,LINC00623,-0.2625158459883076
-1804,FCGR1B,0.3552205391869643
-1809,SRGAP2C,0.16080571026469473
-1817,LINC02591,-0.0765436503762333
-1824,AC246785.3,-0.05939496089189511
-1825,NBPF15,-0.02269229530885977
-1829,SRGAP2B,0.26392149067081144
-1832,AC245014.3,0.204468411095803
-1836,GPR89A,-0.047921651623902266
-1838,CD160,-0.4951034362314438
-1839,RNF115,-0.03081419986969479
-1840,POLR3C,0.08751987705660158
-1842,PIAS3,0.013175344665962159
-1846,PEX11B,0.004207484145735683
-1847,RBM8A,-0.12444921121517695
-1849,LIX1L,-0.044890131801122536
-1852,POLR3GL,-0.2548760056962633
-1853,TXNIP,-0.24210423614979326
-1855,AC239799.2,0.0634334418648291
-1857,NBPF10,0.12048690528121034
-1858,NOTCH2NLA,0.16453720979387315
-1861,AC245407.2,-0.11947067215730695
-1862,NBPF12,0.055632387021188175
-1864,PRKAB2,-0.0065533442302265115
-1865,AC242426.2,0.002825090664763407
-1866,FMO5,0.08214312810076006
-1869,BCL9,-0.05028112375203732
-1876,GPR89B,-0.08353380783317926
-1879,NBPF11,0.12609815162116803
-1883,LINC01138,-0.1749206619066541
-1888,NBPF14,0.2257552393934731
-1891,PDE4DIP,-0.016118141731702028
-1895,NBPF9,0.18769652811402443
-1896,AC245297.3,-0.1354711362358376
-1898,NOTCH2NLC,0.08212060214956937
-1899,NBPF19,0.12812289911578142
-1902,FCGR1A,0.43532475104214685
-1903,HIST2H2BF,-0.03729379313211941
-1913,HIST2H2BE,0.056901789297599745
-1914,HIST2H2AC,-0.05654517395153597
-1915,HIST2H2AB,-0.14449138908436177
-1916,BOLA1,-0.12978335393237952
-1918,SF3B4,0.004470151857384013
-1919,MTMR11,0.26113387315503706
-1922,VPS45,0.07332625068950144
-1923,PLEKHO1,0.5035809583724987
-1925,ANP32E,-0.1448910181400107
-1928,APH1A,0.11889163030811739
-1931,MRPS21,-0.1477068320927416
-1932,PRPF3,-0.046749354700098486
-1933,RPRD2,-0.07684441423228272
-1934,TARS2,-0.09294316925868414
-1938,ADAMTSL4,0.22561964408183047
-1939,ADAMTSL4-AS1,0.12324593615304487
-1940,MCL1,0.45410926181443145
-1941,ENSA,-0.14670010229929212
-1942,GOLPH3L,-0.11232339432405065
-1944,CTSS,0.7016822384868344
-1946,ARNT,0.12235242872920422
-1948,SETDB1,0.13972906723236872
-1949,CERS2,0.0946153830700544
-1953,MINDY1,0.07831826555655008
-1954,PRUNE1,0.015155368313324855
-1956,C1orf56,-0.20452132747565646
-1957,CDC42SE1,-0.19397441252160788
-1958,MLLT11,-0.051638691297096216
-1959,GABPB2,-0.07900349480879348
-1962,TNFAIP8L2,0.17242359258641593
-1963,SCNM1,0.14902543763491177
-1966,VPS72,-0.04640002607605422
-1967,PIP5K1A,0.035279485370556
-1968,PSMD4,0.04430310918092479
-1970,ZNF687,0.033404100282053754
-1971,PI4KB,0.10213972351438531
-1973,RFX5,0.03521461839861062
-1977,PSMB4,0.1012183517334879
-1978,POGZ,0.0973029162487224
-1982,SNX27,0.35976951818733477
-1988,MRPL9,-0.20609886985516268
-2001,THEM4,-0.18269974958141202
-2004,S100A10,0.19870528705485824
-2006,S100A11,0.536212225924074
-2059,S100A9,0.5074430864285135
-2060,S100A12,0.40200166667951426
-2061,S100A8,0.4765702106780999
-2066,S100A6,0.5399221492483436
-2068,S100A4,0.2932608217106712
-2077,CHTOP,0.05779476378190596
-2078,SNAPIN,0.019168281098890315
-2079,ILF2,0.05608711831287201
-2081,INTS3,0.06674777206914573
-2084,SLC27A3,0.12471434634770238
-2085,GATAD2B,0.004875924208183606
-2087,DENND4B,0.11132729511041824
-2088,CRTC2,0.07374641216329506
-2089,SLC39A1,0.0908701527519854
-2093,JTB,-0.07881385870807925
-2095,RAB13,0.15789865143941761
-2096,RPS27,-0.49257792739220135
-2098,TPM3,0.31019575411692885
-2100,C1orf43,0.12506035358677903
-2101,UBAP2L,0.0507413123925315
-2102,HAX1,-0.13077926931336073
-2104,ATP8B2,-0.18498202039051417
-2106,IL6R,0.3786466269332097
-2110,UBE2Q1,0.03336837878167554
-2115,ADAR,0.23997727247682285
-2118,PMVK,0.01916927035246405
-2120,PBXIP1,-0.18518987106886758
-2121,PYGO2,0.000350504509218759
-2123,SHC1,0.08051180991311827
-2124,CKS1B,-0.006346006852151962
-2125,FLAD1,-0.06676813530462025
-2127,ZBTB7B,0.1937344392317168
-2131,ADAM15,0.2698429998196673
-2132,EFNA4,0.017678175173785653
-2135,SLC50A1,-0.02381871369881321
-2136,DPM3,-0.1487236991814259
-2137,KRTCAP2,-0.34959091963485306
-2142,MTX1,0.0702802280728636
-2144,GBA,0.08548295597950745
-2145,FAM189B,0.06410095901777403
-2146,SCAMP3,-0.053420844463528516
-2147,CLK2,0.03290016710382531
-2150,FDPS,-0.18936473510429577
-2152,RUSC1,0.06808246497822917
-2153,ASH1L,0.029922077661316816
-2155,ASH1L-AS1,-0.07522566647440038
-2157,MSTO1,0.00035104913729497626
-2158,AL353807.5,0.15508956394962561
-2159,YY1AP1,0.16689286889271882
-2160,DAP3,0.01991744758431806
-2162,GON4L,-0.036525003544427885
-2163,SYT11,-0.16860317897762028
-2164,RIT1,0.10349327932433873
-2165,KHDC4,0.0422320880936085
-2167,ARHGEF2,0.1045097225971159
-2171,SSR2,-0.25070702515277415
-2172,UBQLN4,-0.06268489532515444
-2173,LAMTOR2,0.16372942043983033
-2176,LMNA,0.09418577461310396
-2177,SEMA4A,0.30737722757855546
-2178,SLC25A44,0.0847269952331866
-2179,PMF1,-0.07148415264766075
-2182,SMG5,0.1329782847509423
-2183,TMEM79,0.0288319434873534
-2184,GLMP,0.21195185373623224
-2186,CCT3,-0.19757054724937154
-2191,MEF2D,0.2022872030608336
-2195,NAXE,-0.03592580600654372
-2196,GPATCH4,-0.08265738669012052
-2208,ISG20L2,0.013252835097596833
-2209,RRNAD1,0.06116424414000072
-2210,MRPL24,0.01018196975219451
-2211,HDGF,0.1932889802417052
-2212,PRCC,0.007149244776527536
-2214,SH2D2A,-0.2695707108933275
-2219,ARHGEF11,0.26651935844426017
-2221,ETV3,0.03490235604361974
-2226,FCRL3,-0.20306752676376374
-2228,FCRL2,0.0021332422619659203
-2229,FCRL1,0.003260296669874765
-2235,CD1D,0.3880787716406388
-2238,CD1C,0.039309488838070665
-2256,MNDA,0.6103464035725646
-2257,PYHIN1,-0.46446402106841145
-2258,IFI16,0.28799312532403404
-2259,AIM2,0.11261876797872315
-2270,DUSP23,0.0255029204554246
-2271,FCRL6,-0.4813954644286315
-2272,SLAMF8,0.04624542381872483
-2279,TAGLN2,0.11058681980376629
-2284,PIGM,0.0818174715772197
-2287,IGSF8,-0.06593090157630456
-2292,PEA15,0.3091459412331339
-2293,DCAF8,0.07009558422977734
-2295,PEX19,-0.05275311018735095
-2296,COPA,0.21163562775555472
-2297,NCSTN,0.24063169801346934
-2300,SLAMF6,-0.2838325890442219
-2302,CD84,-0.020470458921117086
-2303,SLAMF1,-0.13014205753368674
-2305,CD48,-0.18166784976267028
-2306,SLAMF7,-0.10734083005559832
-2307,LY9,-0.23279420352094643
-2308,CD244,-0.0799922686481144
-2313,F11R,0.059095283046884474
-2314,TSTD1,-0.3562796528538796
-2315,USF1,0.16430394066957113
-2316,ARHGAP30,-0.008954666478600807
-2320,PFDN2,-0.06931157372193703
-2321,NIT1,0.13454206573391478
-2322,DEDD,0.09065258926315638
-2323,UFC1,-0.19065077817485168
-2325,USP21,-0.06413586595843046
-2326,PPOX,-0.09507901908891667
-2327,B4GALT3,-0.09531757609884006
-2329,NDUFS2,0.08819974195614023
-2330,FCER1G,0.2607122945415529
-2332,TOMM40L,0.058199777115089765
-2336,SDHC,0.09127013667871443
-2339,AL592295.5,-0.058755705286668417
-2342,FCGR2A,0.4593963961128281
-2343,HSPA6,0.233480061049543
-2344,FCGR3A,-0.0866751069698364
-2347,FCGR2B,0.10018697425120651
-2349,FCRLA,0.0077574604543496125
-2351,DUSP12,-0.035307786765881054
-2353,ATF6,0.17396640582183373
-2361,SH2D1B,-0.32208325088305384
-2362,UHMK1,-0.0030093162884387787
-2364,UAP1,-0.16826555439903954
-2367,HSD17B7,-0.017539997011195654
-2386,MGST3,-0.06747917179805098
-2387,ALDH9A1,-0.09526376487182348
-2389,TMCO1,-0.1686291586202359
-2391,UCK2,0.07758930624507468
-2399,POGK,0.10113262556452483
-2400,TADA1,0.05459198315985523
-2408,POU2F1,0.04452830775678509
-2410,CD247,-0.581358808773943
-2414,CREG1,0.28045813067953285
-2416,RCSD1,0.2454420969241478
-2417,MPZL1,0.17472890809005834
-2420,MPC2,-0.18897351022546624
-2421,DCAF6,0.06855878501632345
-2423,TIPRL,-0.03403461669123291
-2424,SFT2D2,0.1091208966596912
-2425,TBX19,0.08965819909234402
-2428,XCL2,-0.3814920766751192
-2437,ATP1B1,0.06073766206605843
-2438,NME7,-0.13567365333903478
-2440,BLZF1,0.035448013477034475
-2442,SLC19A2,0.004802856522455602
-2444,F5,0.23010572923522174
-2447,SELL,0.20437501705633612
-2450,METTL18,-0.17636009612853087
-2451,SCYL3,0.0441387856953231
-2452,KIFAP3,-0.1633688968564747
-2458,GORAB,-0.02749608912653107
-2470,PRRC2C,-0.04137563053559955
-2473,VAMP4,0.04812367682301079
-2474,EEF1AKNMT,0.04538029445542833
-2479,PIGC,-0.07643512093828045
-2481,SUCO,-0.021143732245071326
-2482,FASLG,-0.36174047576710305
-2488,AL645568.1,-0.08119931678700155
-2489,PRDX6,0.028381445153257006
-2494,KLHL20,-0.030795769367784637
-2496,DARS2,0.046426709212237585
-2497,GAS5,-0.22958885816893873
-2499,ZBTB37,-0.04692600130520321
-2501,RC3H1,0.18937372379989345
-2505,RABGAP1L,0.20490919963600185
-2507,RABGAP1L-IT1,0.09553714830653236
-2509,Z99127.4,-0.11581889882433677
-2510,CACYBP,-0.020410913170189598
-2511,MRPS14,-0.05151218458039087
-2513,KIAA0040,0.09733236923811879
-2520,COP1,0.2695341228865061
-2539,RALGPS2,0.0022057827131025044
-2541,FAM20B,0.13231448586876446
-2542,TOR3A,0.09441927776775924
-2543,ABL2,0.10687282542950279
-2544,SOAT1,0.19878226049958972
-2554,TOR1AIP2,0.08479940949564885
-2555,AL353708.3,-0.007045640928199204
-2556,TOR1AIP1,0.12300196671192276
-2559,CEP350,0.0596150279731648
-2561,QSOX1,0.2335794213817601
-2563,ACBD6,-0.11499880430399381
-2565,XPR1,0.0021591449509549983
-2570,STX6,0.09645612123800804
-2571,MR1,0.1073155280412313
-2572,IER5,0.22740771063023182
-2584,GLUL,0.3477604759053914
-2588,RNASEL,0.08851394613149052
-2593,NPL,0.3081697173817382
-2595,DHX9,0.010309444360711338
-2604,SMG7,0.1545805077784018
-2605,NCF2,0.5781328370976107
-2607,ARPC5,0.33842355292186055
-2608,RGL1,0.0762295581741006
-2612,TSEN15,-0.11132630796356957
-2616,C1orf21,-0.4313548556218341
-2620,EDEM3,0.2264722358386208
-2621,NIBAN1,0.23316843641784424
-2623,RNF2,0.03025746282929334
-2624,TRMT1L,0.06652849890303991
-2625,SWT1,0.01320426407401006
-2626,IVNS1ABP,-0.04476941047630166
-2634,TPR,-0.030959887806552216
-2635,ODR4,0.10583958935923386
-2640,PTGS2,0.14402180014402652
-2642,PLA2G4A,0.14873972773371713
-2663,RGS18,0.15234894685806716
-2670,RGS2,0.43624749487557524
-2671,UCHL5,-0.07152796038531664
-2672,RO60,-0.0022874226813659354
-2673,GLRX2,0.07190935755074561
-2674,CDC73,0.03706545142160386
-2694,ZBTB41,0.00935444606159036
-2697,DENND1B,0.030651288654708467
-2701,NEK7,0.0858757120853416
-2705,PTPRC,0.19103389122331113
-2707,MIR181A1HG,0.01606529288973126
-2717,ZNF281,0.11153371508439994
-2720,DDX59,-0.002074153995109934
-2722,CAMSAP2,0.09513672566895462
-2725,KIF21B,-0.10493497960439388
-2730,TMEM9,-0.03422086399314643
-2740,CSRP1,0.08653702143920076
-2742,NAV1,0.03932930323350387
-2746,IPO9,0.07186873777468501
-2750,TIMM17A,0.06741776610742044
-2751,RNPEP,0.2873980809884269
-2752,ELF3-AS1,0.028745756759732587
-2756,ARL8A,0.16128821663620085
-2757,PTPN7,-0.2997093998179809
-2760,PPP1R12B,0.054907595249520484
-2764,KDM5B,0.00207345774736278
-2767,RABIF,0.03522407119599279
-2768,KLHL12,0.00370141859174803
-2769,ADIPOR1,0.1444662116974638
-2770,CYB5R1,0.0609601928295955
-2771,TMEM183A,-0.02827917475594576
-2782,BTG2,-0.025148117354092073
-2786,ATP2B4,-0.1831404376826135
-2787,LAX1,-0.22049819120376699
-2790,SNRPE,-0.09428362807079878
-2805,PPP1R15B,0.13631971468425322
-2807,MDM4,0.09955700624706107
-2816,RBBP5,0.05015830330410572
-2817,DSTYK,-0.04341631248796663
-2820,NUAK2,0.04981101523444204
-2828,ELK4,0.01609934909433947
-2830,NUCKS1,-0.32761944783489594
-2831,RAB29,-0.16329123972693962
-2833,SLC41A1,-0.0643129797679448
-2845,FAM72A,0.07991391194108509
-2846,SRGAP2,0.2846267102548133
-2847,IKBKE,0.004501519865573392
-2850,RASSF5,-0.08693293098035197
-2851,EIF2D,0.04154642059937186
-2854,MAPKAPK2,0.045183094273143674
-2859,FCMR,-0.24049494144065728
-2863,PFKFB2,0.10955470652784356
-2864,YOD1,0.09506763257831813
-2870,CD55,0.20451648041819132
-2873,CR1,0.3213608644833712
-2877,CD46,0.18917072781270175
-2878,MIR29B2CHG,0.05425892933478768
-2901,TRAF3IP3,-0.3111210882364119
-2904,UTP25,-0.011666727107108326
-2915,RCOR3,0.1014036905826955
-2916,TRAF5,-0.22203860494991748
-2918,LINC00467,0.07988275588003411
-2921,SLC30A1,0.17601902972490063
-2928,LPGAT1,0.17781847629838943
-2930,INTS7,0.001120743578174862
-2934,PPP2R5A,0.04529788799522194
-2938,PACC1,0.03614122645537616
-2940,NENF,0.016909525415405757
-2944,AC092803.1,0.062366120790481816
-2946,ATF3,0.17499832398630055
-2950,BATF3,0.18059565493785826
-2951,NSL1,0.016941119888240267
-2952,TATDN3,0.14807611136163945
-2954,FLVCR1-DT,0.02382782024614989
-2955,FLVCR1,0.04496229960365304
-2958,ANGEL2,0.026540109672115286
-2959,RPS6KC1,0.14096581041533307
-2968,SMYD2,-0.02571246882853043
-2975,KCTD3,0.11962953277017511
-2981,GPATCH2,-0.06201082832308754
-2988,RRP15,-0.08004689528591243
-2995,LYPLAL1,0.05114835257961428
-3003,EPRS,-0.014298652679386223
-3004,BPNT1,-0.10051535832830272
-3005,IARS2,0.056864119732748374
-3006,RAB3GAP2,0.03253595098370535
-3013,MARC1,0.19299996422153068
-3018,HLX,0.2165596505459172
-3023,DUSP10,0.024548714726290437
-3032,TAF1A,-0.035873926320862
-3034,MIA3,0.013521437000894997
-3036,AIDA,-0.038880744028908364
-3037,BROX,0.09117446275608249
-3040,AL392172.1,-0.07223377721398516
-3043,TLR5,0.2826812743567339
-3049,CAPN2,0.19625122610931447
-3050,TP53BP2,0.1117240869950755
-3052,FBXO28,0.029261838631335725
-3053,DEGS1,-0.04954735404353803
-3056,NVL,-0.11204561668560711
-3057,CNIH4,0.07201930567275709
-3058,WDR26,0.1406390839401906
-3066,LBR,-0.011494977809536932
-3072,SRP9,-0.11932823312038646
-3074,AL591895.1,-0.030760960331218206
-3075,TMEM63A,-0.038264610445633444
-3077,PYCR2,-0.09656732567954394
-3080,SDE2,0.006915828785865056
-3082,H3F3A,0.31219859156186963
-3084,ACBD3,0.08676520033196737
-3088,PARP1,-0.1628218373952822
-3091,ITPKB,-0.08372264869312128
-3096,COQ8A,-0.02335154952953042
-3104,ZNF678,-0.027561789367112878
-3105,SNAP47,-0.0574076502348705
-3106,JMJD4,-0.029068982179845545
-3113,ARF1,0.14954922777535104
-3114,C1orf35,-0.1091235954529634
-3115,MRPL55,-0.07091029785252198
-3117,GUK1,-0.18044255028466294
-3120,IBA57,-0.06070782225859193
-3122,OBSCN,-0.09119137820754586
-3128,TRIM11,0.09161928789455985
-3133,HIST3H2A,-0.1302108360097372
-3135,RNF187,-0.04339836874641278
-3136,RHOU,0.12810038446785527
-3142,RAB4A,-0.023267083004019938
-3145,CCSAP,0.14871290288920863
-3147,NUP133,0.022677007038566885
-3150,ABCB10,0.04323381651073919
-3151,TAF5L,0.05844823918601125
-3152,URB2,-0.007987565975430795
-3155,GALNT2,0.1647507396125177
-3161,COG2,0.08348217018601198
-3171,TTC13,0.046932541212893406
-3172,ARV1,-0.006768021387820978
-3173,FAM89A,-0.08360351862004672
-3176,C1orf131,-0.05175518793055274
-3177,GNPAT,0.05725110530673056
-3178,EXOC8,0.054360126587478186
-3179,SPRTN,-0.06751262584704222
-3180,EGLN1,0.04293022046917649
-3181,AL445524.1,0.18177431665948565
-3182,TSNAX,0.12085659144621962
-3184,DISC1,0.17176477972372958
-3185,AL136171.2,-0.028451417960747866
-3189,SIPA1L2,0.14732949103072254
-3192,MAP10,0.05074014711500616
-3194,NTPCR,0.04755314317810244
-3195,PCNX2,-0.04052772296670836
-3208,COA6-AS1,-0.08511833648103066
-3209,COA6,-0.007196621804926549
-3210,TARBP1,0.011993347924302266
-3215,IRF2BP2,0.33161363445674846
-3231,TOMM20,-0.09433600685416231
-3232,RBM34,-0.06466649489043075
-3233,ARID4B,-0.07558317044885751
-3234,GGPS1,-0.0237949367492587
-3235,TBCE,-0.0641743227631362
-3237,TBCE,0.025111434324045025
-3238,B3GALNT2,0.010847647888431648
-3240,LYST,0.5217539007628255
-3244,NID1,0.2131189610729805
-3246,GPR137B,0.06266813067107095
-3247,ERO1B,-0.029892056221831633
-3249,LGALS8,0.1954343269941745
-3253,HEATR1,-0.051818717012853
-3255,MTR,0.00826902736187243
-3269,CHRM3-AS2,-0.1114454813986403
-3285,FH,-0.04048071042476534
-3286,KMO,0.17382615151067712
-3287,OPN3,0.15581939218254587
-3288,CHML,0.13286149532797714
-3298,CEP170,0.2537026965845072
-3301,SDCCAG8,0.1531648026754695
-3302,AKT3,-0.1787330775878146
-3307,ZBTB18,0.18130532370974803
-3314,ADSS,-0.08760146084099567
-3316,DESI2,-0.14293508710582215
-3320,COX20,-0.23755173569275495
-3321,HNRNPU,0.27180755258143374
-3323,AL356512.1,0.08812581768213701
-3324,EFCAB2,0.1358505659727487
-3329,SMYD3,-0.08818902825731513
-3334,TFB2M,-0.025947786448639782
-3335,CNST,-0.04372285549354217
-3339,SCCPDH,0.027553193021135635
-3342,AHCTF1,0.06945245138609928
-3345,ZNF669,-0.09752332652381723
-3348,ZNF124,0.030181367717348733
-3349,AL390728.6,0.074162199873565
-3351,ZNF496,0.04033314136445934
-3354,NLRP3,0.3616454098142198
-3405,SH3BP5L,0.02132343751735445
-3406,ZNF672,0.07356180964484776
-3407,ZNF692,-0.011021527287068263
-3409,PGBD2,-0.03275262341449746
-3412,SH3YL1,-0.16006899189392845
-3413,ACP1,-0.11009330127735954
-3425,TMEM18,-0.08984148556373082
-3458,EIPR1,-0.08674912590100226
-3460,TRAPPC12,0.055621680156723696
-3463,ADI1,0.13063561477880672
-3466,RNASEH1,-0.10735327116359063
-3467,RNASEH1-AS1,-0.05513901699326858
-3469,RPS7,-0.4635382002999135
-3493,CMPK2,0.25701933343604016
-3494,RSAD2,0.19895108578175624
-3497,RNF144A,-0.11249665500192177
-3503,LINC01871,-0.3620338644473577
-3513,ID2,-0.30591570271374846
-3514,KIDINS220,0.15546387253322616
-3520,ASAP2,0.12860927224878216
-3521,ITGB1BP1,-0.13752729317218254
-3522,CPSF3,0.08892898802574133
-3523,IAH1,-0.11751400990522555
-3524,ADAM17,0.1741955881862438
-3527,YWHAQ,-0.08893844508620935
-3531,TAF1B,-0.10846354861040636
-3532,AC010969.2,0.03902686217265188
-3537,AC104794.3,0.047737819642685676
-3538,KLF11,0.1623847008523021
-3544,HPCAL1,0.08202523513510757
-3545,ODC1,0.0020127810120394145
-3547,NOL10,-0.028854637975822453
-3553,PDIA6,0.017086264794617542
-3559,SLC66A3,0.14240521510206508
-3560,ROCK2,0.17940821124126907
-3565,E2F6,-0.0348011328299306
-3569,LPIN1,-0.1502631858066601
-3577,TRIB2,-0.17754096923675694
-3587,NBAS,0.050725083443742366
-3589,DDX1,0.07784556060108243
-3606,FAM49A,0.31008234711093946
-3611,SMC6,0.03199372889279875
-3612,GEN1,0.02632413281637072
-3618,RDH14,-0.1307125036857916
-3627,LINC00954,-0.03386369337683092
-3628,TTC32,0.06005370548849525
-3633,LAPTM4A,0.30339338884424283
-3637,PUM2,0.1474562523634391
-3638,RHOB,0.3287228550574142
-3641,HS1BP3,0.03239179903610253
-3646,LDAH,0.016637613353321846
-3668,ATAD2B,0.11367114720878344
-3669,UBXN2A,0.003468605385645744
-3671,WDCP,-0.08229346622688569
-3673,SF3B6,-0.06467705109316257
-3674,FAM228B,-0.002792465144689234
-3675,TP53I3,0.13478038831744965
-3680,ITSN2,0.10586386655722342
-3683,NCOA1,0.05488287230771231
-3684,PTRHD1,-0.018586892611111235
-3686,ADCY3,0.06157199148701501
-3688,DNAJC27,-0.033938193669163445
-3689,DNAJC27-AS1,-0.01230295463428027
-3692,POMC,0.014085576885204603
-3694,DNMT3A,-0.07839222307608121
-3696,DTNB,-0.03119622232392976
-3698,ASXL2,0.01983426412097881
-3700,RAB10,0.39343835375978176
-3702,HADHA,0.1673893777364338
-3703,HADHB,0.21277159118093
-3706,SELENOI,0.014179199400509072
-3713,SLC35F6,0.17557493678184938
-3719,TMEM214,0.06840104968588048
-3725,OST4,-0.18774865237272534
-3730,PREB,0.027903406645029806
-3733,SLC5A6,0.04615662270577039
-3734,ATRAID,-0.11110235701860098
-3735,CAD,-0.08375773825524688
-3740,MPV17,0.023559819125910098
-3741,GTF3C2,0.11230828330218272
-3744,EIF2B4,0.024715326922401713
-3745,SNX17,0.09184579441525452
-3746,ZNF513,0.0564950169020854
-3747,PPM1G,-0.08779081974559691
-3748,NRBP1,0.02739472440681292
-3754,ZNF512,0.021506683846262895
-3756,GPN1,-0.003501832597439905
-3757,SUPT7L,-0.017521550570058855
-3758,SLC4A1AP,0.03938627789764893
-3760,MRPL33,0.04069894028986808
-3762,BABAM2,0.003027383042324261
-3765,FOSL2,0.217738791768323
-3766,AC104695.4,0.10327841039808457
-3769,PLB1,0.11115947110580157
-3772,PPP1CB,0.26420026278680453
-3775,TRMT61B,-0.0012996156343653786
-3776,WDR43,-0.05123204172042152
-3780,CLIP4,0.08288698149146724
-3786,YPEL5,0.15701981310087892
-3787,LBH,-0.4090660759378844
-3790,LCLAT1,-0.08960387642591332
-3802,MEMO1,0.11670310138779728
-3803,DPY30,-0.11365094241936294
-3805,SPAST,0.09361190799741402
-3807,SLC30A6,0.058476413137746666
-3808,NLRC4,0.20021140935348625
-3809,YIPF4,0.19617175398979228
-3810,AL133245.1,0.07315426731555935
-3811,BIRC6,0.057851166335426
-3813,AL133243.2,0.006253101105526019
-3814,AL133243.3,0.03222639283386975
-3816,TTC27,-0.05841830236078515
-3821,RASGRP3,0.0026779429021710163
-3823,FAM98A,-0.015802962222383705
-3835,FEZ2,0.1547711184020215
-3837,STRN,-0.028778843865757287
-3838,HEATR5B,-0.030138959769695235
-3839,GPATCH11,-0.08009306309497742
-3840,EIF2AK2,0.2115477088370233
-3842,CEBPZOS,-0.056737914629396875
-3843,CEBPZ,-0.0765714095569197
-3845,NDUFAF7,0.037230688416023415
-3846,PRKD3,0.011543929523673039
-3849,QPCT,0.2642356264703035
-3851,AC006369.1,-0.22956883640888479
-3852,CDC42EP3,0.3807748695090205
-3855,RMDN2,0.06121595549122663
-3857,CYP1B1,0.290401376929399
-3858,CYP1B1-AS1,0.08410953786632651
-3862,ATL2,-0.01276119276789875
-3866,HNRNPLL,0.02760944534660607
-3868,GALM,-0.11535794539835806
-3870,SRSF7,-0.19436001750157023
-3871,GEMIN6,-0.09418006310423045
-3872,DHX57,0.1642377554152741
-3874,MORN2,0.06123886373638641
-3877,SOS1,0.05146416070065335
-3880,MAP4K3,0.027750373563033563
-3882,MAP4K3-DT,0.029279254529265646
-3884,THUMPD2,-0.0436092259533727
-3886,SLC8A1,0.44469684924270564
-3898,EML4,0.021793877286907196
-3899,COX7A2L,-0.04187021932932055
-3901,MTA3,-0.027628068839137456
-3903,HAAO,0.10665160308251519
-3909,AC010883.3,-0.052273995642736264
-3910,ZFP36L2,0.24720541853290165
-3911,LINC01126,0.054866070696253745
-3913,THADA,0.014436350907164653
-3919,LRPPRC,0.01866472128930627
-3921,PPM1B,0.08753802145019343
-3923,PREPL,-0.0789427730341284
-3924,CAMKMT,-0.07778475477112191
-3935,SRBD1,0.10504586762087802
-3936,PRKCE,0.08138696827697625
-3944,ATP6V1E2,-0.005695870471239839
-3946,RHOQ,0.3019773418275863
-3948,PIGF,0.013086211189250579
-3949,CRIPT,-0.05429220541656803
-3951,SOCS5,0.034599325834306346
-3955,MCFD2,0.0698155476124835
-3956,TTC7A,0.28515906906879596
-3961,CALM2,0.410165090685854
-3965,MSH2,-0.14335149396932065
-3968,MSH6,0.044902513205472304
-3969,FBXO11,0.2122844626724248
-3972,FOXN2,0.2466077168016932
-3974,PPP1R21,0.05702710942541635
-3994,ASB3,0.01631883848777843
-3996,ERLEC1,-0.12511380111590242
-3998,PSME4,-0.03540280329369151
-3999,ACYP2,-0.03331740638333218
-4003,SPTBN1,-0.18360997417445407
-4010,RTN4,0.33312330062204215
-4013,RPS27A,-0.46776282378813483
-4014,MTIF2,0.09559200925036916
-4016,CCDC88A,0.4263579213881338
-4017,CFAP36,-0.13242520309969005
-4018,PPP4R3B,0.010671270342786391
-4019,AC015982.2,-0.016208403181401262
-4020,PNPT1,0.08966696078321537
-4028,VRK2,0.09624863697662284
-4030,FANCL,0.04039729049566903
-4042,BCL11A,0.1423755293211823
-4044,PAPOLG,0.040791182983521745
-4046,REL,0.21799143447725058
-4048,PUS10,-0.04515966599101342
-4049,PEX13,-0.012532522126643719
-4050,KIAA1841,-0.04542126657653033
-4051,AC016747.1,-0.022554775658569757
-4052,C2orf74,-0.09107428318218894
-4054,USP34,-0.029857915793299078
-4056,AC016727.1,0.036497116993240805
-4057,XPO1,0.058771076896417675
-4060,CCT4,-0.11614678481381732
-4063,COMMD1,-0.025803905615719266
-4065,B3GNT2,0.012881785206636106
-4069,EHBP1,-0.01736133303487485
-4075,WDPCP,-0.05909754478427741
-4076,MDH1,-0.09369245857188405
-4077,UGP2,-0.05024958134494417
-4078,VPS54,0.08689395054618619
-4080,PELI1,0.30856519339411626
-4081,AC012368.1,0.1586711540566311
-4088,AC008074.2,0.08116128826815029
-4089,AFTPH,0.17219484498080917
-4091,SERTAD2,0.20446363241055152
-4095,LINC02245,0.003384222618208172
-4096,SLC1A4,0.1063937086863396
-4098,CEP68,-0.06970327483602243
-4099,RAB1A,0.22420928835497989
-4100,ACTR2,0.5026462265115831
-4121,ETAA1,-0.037012827131026604
-4126,C1D,-0.055574063948737915
-4127,WDR92,-0.020372939580815323
-4128,PNO1,-0.024620604459946845
-4129,PPP3R1,0.09618334511835591
-4134,PLEK,0.026695200262293522
-4135,FBXO48,-0.060521811904619534
-4138,ARHGAP25,-0.03061435589184341
-4146,GFPT1,-0.05526287138077426
-4147,NFU1,-0.007669675903745408
-4148,AAK1,-0.2511040941702531
-4149,ANXA4,0.21547978828581796
-4151,GMCL1,-0.0570178929269787
-4152,SNRNP27,-0.04246709358522554
-4153,MXD1,0.2299151565144025
-4155,PCBP1-AS1,0.030092742124526915
-4156,PCBP1,0.2900059724630289
-4159,C2orf42,-0.03871977551024012
-4160,TIA1,0.04108368243579689
-4161,PCYOX1,-0.039808208993005724
-4162,SNRPG,-0.05787209480278358
-4163,FAM136A,-0.008663350289473579
-4178,TEX261,0.03376052767588121
-4181,NAGK,0.5086751419051943
-4183,MCEE,-0.0457008824013247
-4185,MPHOSPH10,-0.039499003591504725
-4186,PAIP2B,-0.057965784386090506
-4187,ZNF638,0.07998297235540185
-4188,AC007878.1,0.015943322449660065
-4189,DYSF,0.3393801687525607
-4191,EXOC6B,-0.0059567709331388346
-4195,SFXN5,0.16320188695574417
-4199,SMYD5,-0.019899461782203514
-4200,PRADC1,-0.05788594655318647
-4201,CCT7,-0.12428551662985415
-4202,FBXO41,0.008261412967854173
-4206,ALMS1,-0.03707543693491771
-4210,TPRKB,0.020329619847435074
-4212,DUSP11,-0.06104778183430893
-4215,STAMBP,0.034156389769550495
-4218,DGUOK,-0.1017342771172798
-4221,TET3,0.30168003597520643
-4223,BOLA3,-0.23269411092440553
-4225,MOB1A,0.33359902600860414
-4227,MTHFD2,0.2214112014055077
-4229,DCTN1,0.0569050298067693
-4233,WDR54,-0.15135946347578744
-4235,INO80B,-0.020225032490778977
-4236,WBP1,-0.11689232557652773
-4237,MOGS,-0.007659187736146792
-4239,MRPL53,0.04099193528279888
-4240,CCDC142,0.05539280725906855
-4241,TTC31,0.00956260301473889
-4245,PCGF1,-0.06524123934293295
-4248,AUP1,0.011873417854754377
-4249,HTRA2,-0.034665336578651305
-4250,LOXL3,0.1690342403702416
-4251,DOK1,0.19480423190117604
-4257,HK2,0.19135054568212706
-4261,POLE4,0.13546112899063228
-4267,MRPL19,-0.05448638337995482
-4268,GCFC2,-0.012210371021707196
-4308,SUCLG1,-0.052361462863966035
-4310,TRABD2A,-0.12125818491843157
-4311,TMSB10,0.10511678429860147
-4313,KCMF1,0.06897375632923691
-4317,TGOLN2,0.14193972598407162
-4318,RETSAT,-0.06965441749648942
-4319,ELMOD3,-0.009605995038643598
-4321,CAPG,0.39819518401501885
-4324,PARTICL,-0.012077891107724227
-4325,MAT2A,0.12491002150096266
-4326,GGCX,0.05167848831284616
-4327,VAMP8,0.19430916195554676
-4328,VAMP5,0.37044902003129837
-4329,RNF181,0.042258896534659016
-4330,TMEM150A,0.11713627284205845
-4331,USP39,0.051563916578957356
-4332,C2orf68,0.030728681533663885
-4334,GNLY,-0.6456208330327069
-4337,ST3GAL5,0.18927816930157162
-4341,POLR1A,-0.00814619172315291
-4342,PTCD3,-0.04451824739656674
-4343,IMMT,0.05739336032240736
-4345,MRPL35,-0.015404888647798844
-4347,KDM3A,-0.0703627867451164
-4348,CHMP3,0.15758528141422729
-4350,RNF103,0.007702334174074196
-4351,RMND5A,0.0288682706510757
-4352,CD8A,-0.3927922730204935
-4353,CD8B,-0.2941808878372046
-4361,CYTOR,-0.14576467832204215
-4366,KRCC1,0.031384448399978754
-4373,EIF2AK3,0.00043979394160324525
-4376,RPIA,-0.012512995766950318
-4377,IGKC,0.004433955207262528
-4467,MAL,-0.16693436215844384
-4470,MRPS5,0.06769632386994065
-4471,ZNF514,-0.03097011689171506
-4472,ZNF2,-0.022110502950129403
-4476,FAHD2A,-0.04089313020981877
-4482,LINC00342,-0.17409399395707942
-4483,ANKRD36C,-0.17175873441492687
-4487,DUSP2,-0.23343147463350644
-4489,STARD7,0.16019061691596562
-4490,STARD7-AS1,0.00036397700774475344
-4491,TMEM127,0.24727779961684934
-4492,CIAO1,0.07088949637079486
-4493,SNRNP200,-0.07776059565631566
-4499,ARID5A,-0.06028055929905449
-4500,KANSL3,-0.0060451367022552665
-4502,LMAN2L,0.08296289653540581
-4503,CNNM4,-0.022755686298961582
-4505,CNNM3,-0.0730093236077145
-4507,ANKRD39,-0.11083294422376948
-4515,ANKRD36,-0.1802082348263284
-4526,AC092683.1,-0.12063412582080885
-4528,ANKRD36B,-0.09244314010759168
-4530,COX5B,0.11880736445233586
-4531,ACTR1B,-0.01583497833321699
-4533,ZAP70,-0.3382505419733231
-4534,TMEM131,0.18820974287003625
-4539,INPP4A,-0.07390163233919003
-4540,COA5,-0.049222852716025486
-4541,UNC50,-0.094134625796727
-4542,MGAT4A,-0.26995030936395475
-4543,LINC02611,-0.10896881605106465
-4545,TSGA10,-0.09447508385077935
-4548,LIPT1,-0.08459387584802684
-4549,MITD1,-0.0021143595665774452
-4550,MRPL30,-0.005083280960182778
-4553,TXNDC9,-0.14311279361246065
-4554,EIF5B,-0.0867957290704963
-4555,REV1,0.03786488898527051
-4557,AFF3,0.020790249715019672
-4563,PDCL3,-0.032031070334309485
-4573,RPL31,-0.2938736772344408
-4574,TBC1D8,0.20365145813327895
-4576,CNOT11,0.07241937650040461
-4577,RNF149,0.3164022093274636
-4583,MAP4K4,0.11731124094382614
-4584,LINC01127,0.1614642796196719
-4591,IL18R1,-0.19329431086952406
-4597,MFSD9,0.014349378598865713
-4624,MRPS9,-0.1768479465010364
-4629,TGFBRAP1,0.0425990529993937
-4631,AC012360.3,-0.08701997384161983
-4632,C2orf49,0.005396850188201053
-4637,NCK2,-0.0676131732856634
-4641,UXS1,-0.05709204988057144
-4665,GCC2,-0.17120874534936711
-4667,LIMS1,0.14082866296849894
-4669,RANBP2,0.1287029004447842
-4670,CCDC138,-0.09149703754870317
-4674,SEPTIN10,0.14282066916433478
-4683,MTLN,-0.09738506461705812
-4692,MIR4435-2HG,-0.05686565099142644
-4694,BCL2L11,0.16107200894757417
-4704,ANAPC1,-0.015283095252375067
-4705,MERTK,0.08473776574428828
-4707,TMEM87B,0.02753274833949147
-4710,ZC3H8,-0.0942175047030343
-4712,ZC3H6,-0.06832982545254067
-4714,TTL,0.03351373859470746
-4715,POLR1B,-0.029892551469070057
-4716,CHCHD5,-0.06706636300575922
-4719,AC079922.2,-0.039498104722798
-4720,SLC20A1,0.14775275111776467
-4725,IL1B,0.20046008864420162
-4732,IL1RN,0.2914690910372998
-4733,PSD4,-0.01806737625204614
-4735,PAX8-AS1,0.072276066613422
-4739,AC016745.2,0.11735504216447588
-4740,CBWD2,0.18815116325450226
-4746,RABL2A,0.003982795751804928
-4749,SLC35F5,0.0675069952281692
-4752,AC110769.2,0.006613956185903748
-4753,ACTR3,0.1530758077912592
-4766,DDX18,-0.13331040571831226
-4767,AC009404.1,0.03284228371523337
-4768,CCDC93,-0.06710299088746549
-4771,INSIG2,-0.04382207526407806
-4775,MARCO,0.2593486096736799
-4780,C2orf76,-0.06028071024401163
-4781,DBI,-0.25967891956495637
-4786,TMEM177,-0.08481461282592179
-4787,PTPN4,-0.4231067016779627
-4790,TMEM185B,-0.02121982914868197
-4791,RALB,0.32575359504413287
-4802,CLASP1,0.015220056526168311
-4804,NIFK-AS1,-0.08602634491339216
-4805,NIFK,-0.10891052573118402
-4806,TSN,-0.012797269157050404
-4822,GYPC,-0.4254197571930013
-4827,BIN1,-0.2959177513571566
-4830,ERCC3,-0.04262119664847299
-4832,MAP3K2,0.2117573735460183
-4833,MAP3K2-DT,-0.04908510473081728
-4836,IWS1,0.03321975511773485
-4842,WDR33,-0.1793531105757658
-4843,SFT2D3,0.005344919026924573
-4844,POLR2D,-0.0333509076333456
-4845,AMMECR1L,0.07313551661793095
-4848,SAP130,0.0751770276930104
-4849,UGGT1,0.10453795406567438
-4850,HS6ST1,-0.04878736244497769
-4861,SMPD4,0.04433160912503014
-4862,MZT2B,-0.2765311081509429
-4864,CCDC115,-0.008387022811329605
-4865,IMP4,0.03255449987951845
-4866,PTPN18,0.11299825451971336
-4880,FAM168B,0.04260967209131954
-4881,PLEKHB2,0.23669481632397926
-4887,MZT2A,-0.2574199263071514
-4892,C2orf27A,-0.0817096206264859
-4912,MGAT5,-0.07062390097911331
-4917,CCNT2,0.07234231368104103
-4919,RAB3GAP1,0.06962222379253569
-4921,R3HDM1,0.011978123978646735
-4922,UBXN4,0.01195382676636464
-4925,MCM6,-0.053444331277463675
-4926,DARS,0.027354713990457934
-4929,CXCR4,-0.08198311890682579
-4931,HNMT,0.4129075382011426
-4935,SPOPL,0.22738199377286847
-4936,AC092620.1,0.14347391152484842
-4948,KYNU,0.3205239516745874
-4949,ARHGAP15,-0.219470869284263
-4958,GTDC1,-0.019364252730579987
-4959,ZEB2,0.42941281787047997
-4961,ZEB2-AS1,0.17816364725875877
-4962,AC009951.6,0.08372700709644637
-4975,ACVR2A,0.09599326876755765
-4977,ORC4,-0.03870598844891137
-4978,MBD5,-0.06769588174230495
-4979,EPC2,-0.012423341131737748
-4986,MMADHC,-0.03755742825449173
-5000,RBM43,0.015129039185554024
-5001,NMI,0.2907077607867926
-5003,RIF1,0.024608752921740123
-5005,ARL5A,0.07144847770623049
-5006,AC097448.1,-0.1422282519186739
-5009,STAM2,0.1543183599797885
-5011,FMNL2,0.13259759711166658
-5014,PRPF40A,0.14093923750314447
-5015,ARL6IP6,-0.03649236911093291
-5029,NR4A2,0.16893717288810844
-5030,GPD2,0.1056911724118194
-5036,ERMN,-0.011614114184017189
-5037,CYTIP,-0.24640192624625856
-5040,ACVR1,0.08809779382684907
-5047,PKP4,0.04622997727267622
-5050,DAPL1,0.09485197255552884
-5053,WDSUB1,-0.03688961501766596
-5054,BAZ2B,0.24890965474457125
-5058,MARCH7,0.025531658064503866
-5059,CD302,0.5399192042869607
-5061,LY75,0.046945774163250804
-5066,RBMS1,0.25540480463919807
-5068,TANK,0.15920496004135914
-5071,PSMD14,-0.13664350681631096
-5077,DPP4,-0.11068521700992327
-5083,IFIH1,0.2620116285712006
-5084,GCA,0.489368089724997
-5090,COBLL1,-0.0025727328650093457
-5097,GALNT3,0.04113529903925252
-5101,TTC21B,-0.09082315733224029
-5112,STK39,-0.20246678620747088
-5113,CERS6,0.17178221715437197
-5119,DHRS9,0.19744685694031366
-5124,FASTKD1,0.07311124238666677
-5125,PPIG,0.0002505464931794843
-5129,SSB,-0.009679196134159615
-5130,METTL5,-0.15529330238036532
-5131,UBR3,0.07953848708522533
-5144,GORASP2,0.09201932295812312
-5145,TLK1,-0.019581937678105857
-5146,METTL8,-0.022465964280784616
-5147,DCAF17,-0.011337722471608764
-5148,CYBRD1,0.1863426317051021
-5149,DYNC1I2,0.038717953163053966
-5150,SLC25A12,-0.011993069205146223
-5151,HAT1,-0.06489950198500737
-5159,ITGA6,-0.24398284216679267
-5163,PDK1,-0.08061295954675789
-5166,MAP3K20,0.2697862173499763
-5170,SP3,0.12480435449953335
-5174,OLA1,-0.05464936512146665
-5177,CIR1,0.08720303787149489
-5178,SCRN3,-0.0025548819986496536
-5179,GPR155,0.07982413841548451
-5182,WIPF1,0.05988687629192841
-5186,ATF2,0.03642358553438445
-5188,ATP5MC3,-0.0286142603193885
-5192,LNPK,-0.017047456150680505
-5207,MTX2,0.04208625759256071
-5212,LINC01116,-0.003878895750669932
-5217,HNRNPA3,0.010693205267121414
-5218,NFE2L2,0.2131394768047896
-5224,AGPS,0.1034183100979215
-5231,RBM45,0.028838007878824966
-5233,CHROMR,0.09850750426624871
-5234,PRKRA,-0.0058557667768903456
-5237,PLEKHA3,-0.023802941948306167
-5238,TTN-AS1,0.03463388401386204
-5239,TTN,-0.0663577310186719
-5248,CCDC141,-0.07280206706223759
-5249,SESTD1,0.22718159287244136
-5252,CWC22,0.03857277205863466
-5255,UBE2E3,-0.0040030126804667795
-5258,LINC01934,-0.17232507629166566
-5260,ITGA4,0.28950650493469077
-5266,ITPRID2,0.25644543744641424
-5270,DNAJC10,0.14469632624102918
-5275,NUP35,-0.03367358962099626
-5283,AC096667.1,0.3163818370423029
-5292,ZC3H15,-0.0006381372710959301
-5293,ITGAV,0.05467440080706245
-5298,CALCRL,0.13416381858119159
-5307,WDR75,-0.03635407060162787
-5309,SLC40A1,0.1254005341731333
-5310,ASNSD1,-0.03331085597738606
-5312,ANKAR,-0.10987591391273722
-5313,OSGEPL1,-0.1723311180930337
-5316,ORMDL1,-0.1469543810961903
-5317,PMS1,-0.03234105629566249
-5320,HIBCH,-0.09736449109092435
-5321,INPP1,0.07812806173929018
-5322,MFSD6,-0.02492145186787968
-5324,NEMP2,-0.07300007705945748
-5327,NAB1,0.10344981720220653
-5330,GLS,-0.19362557608421654
-5331,STAT1,0.40935573011306764
-5334,STAT4,-0.34614794781041197
-5338,NABP1,0.1543672277215687
-5357,SLC39A10,-0.051765363597972146
-5359,STK17B,0.24042173706318834
-5360,AC114760.2,-0.007156091582770477
-5365,GTF3C3,0.05194476023637918
-5367,PGAP1,0.0837724922928979
-5368,ANKRD44,0.11112876728335262
-5369,AC013264.1,-0.09817014896814934
-5370,ANKRD44-IT1,0.0993455208801252
-5372,SF3B1,0.022214948718384076
-5373,COQ10B,-0.0960194085426279
-5374,HSPD1,-0.040237999018100726
-5375,HSPE1,-0.27152652807082167
-5376,MOB4,-0.038043880935277594
-5380,MARS2,-0.0007600662652021993
-5382,PLCL1,0.00012264653918536508
-5392,FTCDNL1,0.06444927168869853
-5394,C2orf69,-0.008251990795072823
-5395,TYW5,-0.011491863779485011
-5396,MAIP1,0.06183998734601139
-5398,SPATS2L,0.13240529575165064
-5400,KCTD18,0.004120302087460976
-5401,SGO2,-0.05104322296089025
-5405,BZW1,-0.04522940711714569
-5406,CLK1,0.030217637066243256
-5407,PPIL3,-0.08780461247344927
-5408,NIF3L1,-0.009948628700267829
-5409,ORC2,-0.032431173568487945
-5411,FAM126B,0.06880638062339724
-5412,NDUFB3,-0.015421633554819816
-5414,CFLAR,0.12607551922016647
-5417,CASP10,0.04624955456359288
-5418,CASP8,-0.15949584023182406
-5420,TRAK2,0.027474322702503582
-5421,STRADB,0.03761843879896187
-5426,ALS2,-0.011152624121986872
-5434,SUMO1,-0.14497788954020224
-5435,NOP58,-0.08997909378040309
-5439,BMPR2,0.11542812019290696
-5440,FAM117B,0.0983104192386103
-5441,ICA1L,-0.024984362459194356
-5442,WDR12,0.06383397841542364
-5443,CARF,-0.038604294146606453
-5444,NBEAL1,-0.05621868029841968
-5445,CYP20A1,-0.04950784739282875
-5446,ABI2,-0.06313512240125142
-5448,RAPH1,0.07672777777286625
-5449,CD28,-0.11116021424184325
-5451,ICOS,-0.1232642236739617
-5461,INO80D,-0.048122773970618524
-5463,NDUFS1,0.020898702980611264
-5465,EEF1B2,-0.35296911979411216
-5475,FASTKD2,0.006172083426191514
-5479,KLF7,0.16248304506695405
-5489,CREB1,0.11252366015517465
-5490,METTL21A,-0.0010038478968682114
-5491,LINC01857,0.0009700633220263451
-5492,CCNYL1,0.04752871163893172
-5495,PLEKHM3,0.06568424857879364
-5502,IDH1,0.19561861786975432
-5504,PIKFYVE,0.1759581635702402
-5511,RPE,0.016160805918103687
-5512,KANSL1L,0.07602093153687971
-5513,AC007038.2,-0.06493560434894792
-5514,AC007038.1,-0.003484132274783141
-5519,LANCL1,-0.06050587219896246
-5527,IKZF2,-0.19287257428035567
-5532,SPAG16,-0.10890761264297867
-5536,BARD1,-0.0011186205105511641
-5542,ATIC,-0.022702159896116368
-5555,PECR,-0.089135759186657
-5557,XRCC5,-0.02636907049752998
-5563,SMARCAL1,-0.030086231191846873
-5566,RPL37A,-0.31849345521420946
-5586,RUFY4,0.14810831288406728
-5587,CXCR2,-0.08070905421403834
-5589,ARPC2,0.046524424706771796
-5591,GPBAR1,0.3534707026572445
-5592,AAMP,0.13391720692976372
-5593,PNKD,-0.10462724776755428
-5594,TMBIM1,0.03178276597040644
-5598,SLC11A1,0.4591498509085583
-5599,CTDSP1,-0.007853821594101633
-5602,USP37,-0.0012569756709990508
-5603,CNOT9,0.07728536052331775
-5606,ZNF142,0.023564750730567453
-5607,BCS1L,-0.07528568616605288
-5608,RNF25,-0.014149962510668102
-5610,TTLL4,0.1421609739100783
-5611,CYP27A1,0.197752506728845
-5628,NHEJ1,-0.016751614403786072
-5630,CNPPD1,0.0109611933789947
-5631,RETREG2,0.0206309901773498
-5632,ZFAND2B,-0.13017278977177188
-5635,ATG9A,0.07816477152716568
-5636,ANKZF1,-0.10660728720717898
-5637,GLB1L,0.042533577694340075
-5638,STK16,-0.11203416684514324
-5639,TUBA4A,-0.013254021537091263
-5641,DNAJB2,-0.015064621582468525
-5645,DNPEP,0.04396448657294435
-5653,GMPPA,0.09588256320308595
-5661,STK11IP,0.10897987057376098
-5681,FARSB,0.03730172414519749
-5683,ACSL3,0.03321838441853163
-5689,AP1S3,0.046363714173252364
-5690,WDFY1,0.25566508233386326
-5691,MRPL44,0.025946357951968962
-5696,CUL3,0.05222726019541283
-5698,DOCK10,-0.11198441791369425
-5705,RHBDD1,0.12245676051210869
-5709,MFF,-0.106181150557881
-5712,AGFG1,0.2826093665430173
-5729,PID1,0.3309874846085023
-5732,TRIP12,0.20528332299509283
-5736,AC009950.1,0.0034088712445427597
-5737,SP110,0.2512184784377923
-5738,SP140,-0.13614296578582724
-5739,SP140L,-0.07127820281578406
-5740,SP100,0.045172093978010835
-5743,CAB39,0.07814788762131054
-5744,ITM2C,-0.04121955916339977
-5753,PSMD1,0.02119228228316989
-5759,NCL,-0.2310185495483895
-5765,PTMA,-0.21818948652115516
-5766,PDE6D,-0.04320163610563805
-5767,COPS7B,0.008921226119707863
-5770,DIS3L2,0.006398698981453612
-5781,TIGD1,-0.06371410118493212
-5782,EIF4E2,0.20780173296897317
-5785,GIGYF2,0.07149967908357367
-5792,INPP5D,0.07640959949574978
-5793,ATG16L1,-0.08404461534429475
-5796,DGKD,-0.0744629083518803
-5797,USP40,0.08391998732773222
-5819,ARL4C,-0.685015874503874
-5838,COPS8,-0.10398237101010757
-5849,LRRFIP1,0.35627895437561113
-5850,AC012076.1,0.024231959766023475
-5853,UBE2F,-0.010075779999750047
-5854,SCLY,-0.06305899695555076
-5858,ILKAP,-0.05275108298518827
-5862,PER2,0.019313476281270014
-5865,TRAF3IP1,-0.04616062229228058
-5866,ASB1,0.05289874843038447
-5875,HDAC4,0.10620308256180644
-5878,HDAC4-AS1,-0.036767666078768656
-5886,NDUFA10,0.057966391141766674
-5889,COPS9,-0.05482475000443644
-5896,ANKMY1,0.014693392055531284
-5898,DUSP28,-0.0435432764334454
-5899,RNPEPL1,-0.004665242092667018
-5901,CAPN10,-0.025501208098884242
-5902,GPR35,0.0842954401897453
-5915,MTERF4,-0.04096169833445607
-5916,PASK,-0.08886289503607818
-5917,PPP1R7,-0.09604944871903572
-5919,HDLBP,0.17513439014075374
-5920,SEPTIN2,0.16171059118932682
-5923,FARP2,0.07813253415131766
-5925,STK25,-0.016273960482534842
-5929,THAP4,0.033247397718683444
-5930,ATG4B,0.11095940278995874
-5931,DTYMK,-0.06836377255904803
-5932,ING5,-0.028531716194131865
-5934,D2HGDH,0.04372068576748159
-5968,TRNT1,-0.10005108890473387
-5969,CRBN,-0.0876690300362131
-5970,AC024060.2,-0.07426546374778237
-5972,SUMF1,0.1622295599641124
-5978,ITPR1,0.13911831758404106
-5982,BHLHE40,-0.0718955709791119
-5983,ARL8B,0.13601054145669433
-5986,EDEM1,0.05839403671334348
-6005,RAD18,-0.07640321764261367
-6013,THUMPD3-AS1,-0.17598276282566694
-6014,THUMPD3,-0.06455072409178854
-6015,SETD5,0.057808427670185875
-6017,MTMR14,0.16311172731283954
-6020,BRPF1,0.05178854117925912
-6021,OGG1,-0.02490625543493382
-6022,CAMK1,0.2962426389419097
-6023,TADA3,-0.1450514986046815
-6024,ARPC4,0.1356463579961666
-6025,TTLL3,0.07234096597008108
-6027,RPUSD3,-0.10562099938592777
-6029,JAGN1,-0.014869180761198376
-6031,IL17RC,0.04813439230917524
-6032,CRELD1,-0.09930804909671635
-6034,PRRT3,-0.0422897109899705
-6037,EMC3,0.06891643857756247
-6039,FANCD2,-0.06933708585522848
-6041,BRK1,-0.0029824045633320665
-6042,VHL,0.034069792683263656
-6045,TATDN2,0.010537486310325351
-6047,GHRL,0.02470874411638928
-6049,SEC13,0.08741774316885838
-6062,ATG7,0.23969180199284698
-6063,VGLL4,-0.04577224509155311
-6066,TAMM41,-0.06132021942171135
-6070,TSEN2,-0.07303196023966414
-6072,MKRN2,-0.03883971015793185
-6073,RAF1,0.132813161656117
-6077,RPL32,-0.3566993481227219
-6079,IQSEC1,0.16353178595116996
-6080,NUP210,-0.06993155324981747
-6087,CHCHD4,0.027167985469379454
-6088,TMEM43,0.03428092270166224
-6090,XPC,-0.005766611938844573
-6091,LSM3,-0.0741792959923476
-6094,SLC6A6,0.24466459979191366
-6098,CCDC174,-0.06355533806444673
-6104,FGD5-AS1,0.011896762419865294
-6105,NR2C2,-0.04584541409296224
-6106,MRPS25,0.00517342696588761
-6107,RBSN,-0.01011323413113225
-6108,CAPN7,0.044872199809795836
-6109,SH3BP5-AS1,-0.013076439789103484
-6110,SH3BP5,-0.19488257646695253
-6111,METTL6,-0.034435714054204966
-6112,EAF1,0.061106753929537225
-6116,HACL1,-0.01829169173487047
-6117,BTD,-0.084299697134102
-6118,ANKRD28,-0.03230280053824899
-6123,DPH3,0.06662549507685807
-6124,OXNAD1,-0.19535441806933646
-6125,RFTN1,-0.07147287023928901
-6135,PLCL2,-0.07284839790501497
-6138,TBC1D5,0.1495630944238689
-6142,SATB1,-0.08264098886324818
-6144,SATB1-AS1,-0.09853011881843977
-6148,RAB5A,0.06737399917522668
-6150,KAT2B,0.10879441887117611
-6159,UBE2E2,0.22116763507874734
-6161,UBE2E1,0.20362435426230005
-6163,RPL15,-0.3250755682054314
-6164,NR1D2,0.034407018830139545
-6172,TOP2B,-0.009258026198045832
-6173,NGLY1,-0.10241020941817416
-6174,OXSM,-0.047704194266270254
-6180,SLC4A7,-0.11715909962662147
-6181,LINC02084,-0.286041457210978
-6182,EOMES,-0.3280097017747177
-6186,CMC1,-0.40827925305800034
-6187,AZI2,0.20249452490344413
-6196,TGFBR2,0.04778129140034522
-6199,STT3B,0.13972137224505138
-6200,OSBPL10,0.006676115856927773
-6203,GPD1L,0.03442377886463393
-6205,CMTM8,-0.04318364031815813
-6206,CMTM7,0.10355379194134019
-6207,CMTM6,0.43680908863923756
-6208,DYNC1LI1,0.17490076203436541
-6209,CNOT10,-0.003674394693368801
-6213,GLB1,0.2411909776254347
-6215,CRTAP,0.33906209158305606
-6219,UBP1,0.07954175950628808
-6220,CLASP2,0.1697390785849842
-6222,PDCD6IP,0.20689750829113837
-6233,TRANK1,-0.051081733759387384
-6235,EPM2AIP1,-0.0011684890225714898
-6236,MLH1,0.04001890114363165
-6237,LRRFIP2,0.115934349925854
-6241,GOLGA4,0.008431351740577913
-6246,VILL,-0.04954213860370474
-6247,PLCD1,0.01607322817237409
-6249,ACAA1,0.1556104970775927
-6250,MYD88,0.3448393024322693
-6251,OXSR1,0.06015917240115966
-6257,EXOG,-0.102548562610109
-6262,WDR48,0.005564726709157084
-6263,GORASP1,0.1519471659237973
-6265,CSRNP1,0.1509874168285493
-6272,CX3CR1,-0.11800183592014461
-6275,SLC25A38,-0.17339476728892025
-6276,RPSA,-0.4582348206170158
-6283,EIF1B,-0.11362148128540735
-6284,ENTPD3-AS1,-0.04875542645395083
-6286,RPL14,-0.45959316299594466
-6287,ZNF619,0.03058260012866769
-6289,ZNF621,-0.0754574791128353
-6295,CTNNB1,0.14643043863790256
-6296,ULK4,-0.03701719738033826
-6297,TRAK1,0.1711535611311382
-6301,VIPR1,0.05554590886247903
-6303,SEC22C,-0.05104905434289899
-6304,SS18L2,-0.15862750169889037
-6305,NKTR,-0.03644883907510698
-6307,ZBTB47,0.14709148585357892
-6316,HIGD1A,-0.0021656968313763145
-6328,SNRK,-0.03990695087679992
-6330,ANO10,0.15861378766364354
-6331,ABHD5,0.24408958308957465
-6338,TCAIM,0.06200735311846255
-6340,ZNF445,0.07562141601675866
-6345,ZNF197,-0.048632287363702335
-6347,ZNF35,0.08123194920825982
-6351,KIAA1143,-0.021297684147432713
-6353,TMEM42,-0.05623502679081698
-6356,ZDHHC3,0.13069441253280012
-6357,EXOSC7,-0.0581488496176031
-6361,LARS2,-0.009513738196134143
-6363,LIMD1,0.1755194689653379
-6364,LIMD1-AS1,-0.004447227624271822
-6365,SACM1L,-0.05432091062515962
-6367,LZTFL1,-0.0025208691879609305
-6370,FYCO1,-0.018047778712513732
-6374,CCR1,0.41681256005515305
-6375,CCR2,0.35186475891786034
-6377,CCR5,-0.03161454950852866
-6378,CCRL2,0.1087903445526514
-6392,CCDC12,-0.1100679731927772
-6393,NBEAL2,0.19662623604526874
-6394,SETD2,0.051079272947896395
-6396,KIF9,-0.001676570743669881
-6397,KLHL18,0.10784476173573891
-6399,PTPN23,0.099050592122006
-6400,SCAP,0.013728050186675587
-6401,ELP6,-0.14122910552290271
-6403,SMARCC1,0.037110457489542566
-6405,DHX30,-0.030347426743956307
-6406,MAP4,-0.0029683829919831418
-6409,ZNF589,0.05292474020954821
-6410,NME6,-0.03717484051965727
-6414,CCDC51,-0.04090594241188569
-6415,TMA7,-0.3764204712162413
-6417,ATRIP,-0.07801072499967239
-6419,SHISA5,0.043163759490757216
-6420,PFKFB4,0.20918311960423117
-6423,UQCRC1,0.24557644421017674
-6425,SLC26A6,0.04949173922421481
-6429,NCKIPSD,0.02535293090001398
-6430,IP6K2,0.014418436131147239
-6431,PRKAR2A,0.1328930698026181
-6433,SLC25A20,-0.06090121350483151
-6434,ARIH2OS,-0.013194514307734329
-6435,ARIH2,-0.03880924605721326
-6439,P4HTM,-0.048958618818814685
-6440,WDR6,0.0183104626841483
-6441,DALRD3,-0.014836008850243578
-6442,NDUFAF3,-0.10550018247233973
-6443,IMPDH2,0.0030221265551924536
-6445,QRICH1,0.012136183832800943
-6446,QARS,0.15237836846531821
-6447,USP19,0.028491345614576537
-6449,CCDC71,-0.016098841904453307
-6450,KLHDC8B,0.09335835064367189
-6454,C3orf62,0.08122931455634039
-6455,USP4,0.1377986090794329
-6456,RHOA,0.39031143962745757
-6458,TCTA,-0.0023103426854125925
-6459,AMT,-0.06902839039532721
-6460,NICN1,-0.041613038410481175
-6461,DAG1,0.061265908591655135
-6465,APEH,0.014615290267910584
-6467,RNF123,0.1307628062461675
-6469,GMPPB,0.0081000888136635
-6470,IP6K1,-0.009419697657605699
-6473,UBA7,0.11949059585623165
-6479,MON1A,-0.056803715976849294
-6480,RBM6,0.05341029766312029
-6481,RBM5,0.14901615316965666
-6487,GNAI2,0.512234404096363
-6488,U73169.1,0.048130590251431904
-6489,U73166.1,0.04340274437230322
-6493,IFRD2,-0.13023003771715735
-6495,NAA80,-0.07761140273211664
-6497,HYAL2,0.06389450351106202
-6498,TUSC2,-0.07204141775346486
-6499,RASSF1,-0.16102148720142268
-6503,NPRL2,-0.19853940123691335
-6504,CYB561D2,-0.13533614577564312
-6505,TMEM115,-0.009306232863818336
-6507,CYB561D2,-0.13861554195844666
-6509,HEMK1,-0.10418858636313308
-6511,CISH,-0.11638152542145888
-6512,MAPKAPK3,0.3088702042808796
-6515,MANF,-0.2612868770619518
-6516,RBM15B,0.030104681403456353
-6517,DCAF1,0.03765411834436746
-6518,RAD54L2,0.020142063036616094
-6519,TEX264,-0.17145992612636596
-6529,RRP9,-0.13284685537188062
-6530,PARP3,0.004244643906438069
-6532,PCBP4,-0.12018051882861956
-6533,ABHD14B,-0.05256857326304062
-6534,ABHD14A,-0.22044816348631507
-6535,ACY1,0.039999747812647285
-6536,RPL29,-0.2942906330883844
-6538,DUSP7,0.12583175951312206
-6541,ALAS1,0.11869720706825349
-6543,TWF2,0.07155606125764012
-6545,PPM1M,-0.017085675214676153
-6546,WDR82,-0.05681741732447938
-6547,GLYCTK,0.01491265614575483
-6549,DNAH1,0.04806097401748904
-6550,BAP1,-0.005766887784979344
-6554,NISCH,0.0840637495429126
-6555,STAB1,0.3253981742764791
-6556,NT5DC2,0.042966851749628186
-6557,SMIM4,-0.018530050942995938
-6558,PBRM1,0.1498776473672137
-6559,GNL3,-0.10908007353643528
-6560,GLT8D1,-0.003448969946638408
-6561,SPCS1,-0.12546446360763797
-6562,NEK4,0.13233713466829142
-6565,ITIH4,0.0908174639550908
-6568,STIMATE,0.13595783128691158
-6569,SFMBT1,-0.044319552253120426
-6570,RFT1,0.043109540808546516
-6571,PRKCD,0.39220184209034
-6573,TKT,0.5766376785150495
-6574,DCP1A,-0.03391549928777609
-6580,ACTR8,0.024245989077569353
-6581,SELENOK,-0.1928214716020276
-6584,CACNA2D3,0.2060818499059535
-6599,CCDC66,-0.1290034462829639
-6600,TASOR,0.04565943280602091
-6601,ARHGEF3,-0.18288910085399243
-6607,APPL1,0.08050203055986004
-6612,PDE12,0.03563479064200176
-6613,ARF4,0.054877082068322015
-6615,DENND6A,0.07765442833085774
-6618,SLMAP,0.11110446799154736
-6619,FLNB,-0.052583966586005115
-6623,ABHD6,0.05522064288300451
-6624,RPP14,0.08665596707126812
-6627,PXK,0.11546692928335169
-6628,PDHB,-0.014450873941211726
-6631,KCTD6,0.05599606426840042
-6642,FHIT,-0.10240897045615076
-6646,C3orf14,0.08152931258074846
-6654,THOC7,-0.09410655774875987
-6656,AC104162.2,0.0040765342014775115
-6657,ATXN7,0.10786818654187925
-6659,PSMD6-AS2,0.0331558937529959
-6661,PSMD6,0.005531523745687
-6679,SLC25A26,-0.10646719238894516
-6680,LRIG1,-0.0354580281337802
-6684,KBTBD8,0.02203936763590275
-6686,SUCLG2,-0.042806813097001344
-6693,TMF1,0.05287850597532191
-6694,UBA3,0.11707481659540438
-6695,ARL6IP5,-0.04170957238325369
-6696,LMOD3,-0.07513397549067974
-6697,FRMD4B,0.21382806958859332
-6700,MITF,0.14534589677836526
-6705,FOXP1,-0.056182589478104616
-6708,FOXP1-IT1,0.10556785090695295
-6710,AC097634.1,0.05698554987326558
-6711,EIF4E3,0.18611090765841765
-6712,GPR27,0.12461737878877578
-6713,PROK2,0.014800187923346846
-6715,LINC00877,0.21527880702541577
-6723,RYBP,0.13198521548928974
-6726,SHQ1,0.021324107079007868
-6728,PPP4R2,0.015174872297331023
-6729,EBLN2,0.07026956178076388
-6739,ZNF717,-0.011947412496628371
-6754,GBE1,0.1005808812257288
-6771,CHMP2B,0.09479976909020917
-6775,CGGBP1,0.06971127581645581
-6776,ZNF654,0.039569574463366154
-6777,C3orf38,0.13561393777517397
-6782,ARL13B,-0.02857867362978895
-6784,DHFR2,0.008148123749056112
-6785,NSUN3,-0.02452256114624699
-6788,MTRNR2L12,-0.12696177158581673
-6794,CRYBG3,0.2627357829874388
-6796,RIOX2,0.06680335010895797
-6811,CLDND1,-0.11526256642300903
-6815,CPOX,0.045090541048961395
-6817,ST3GAL6,0.20330858775591976
-6826,CMSS1,-0.05904571465724358
-6829,TBC1D23,0.0545778893207297
-6830,NIT2,-0.15542051165664694
-6831,TOMM70,0.015241046247176279
-6835,TFG,0.09185326358254985
-6838,SENP7,-0.12361619453300957
-6839,TRMT10C,-0.07174346199354123
-6840,PCNP,-0.05483049654843696
-6841,ZBTB11,0.05513193866773714
-6842,ZBTB11-AS1,-0.04694357832456806
-6843,RPL24,-0.3463543721789223
-6844,AC084198.2,0.04111028945958516
-6845,CEP97,-0.013193444684471416
-6846,NXPE3,0.049999908428275376
-6848,NFKBIZ,0.3099720661379123
-6854,ALCAM,0.1466667906748037
-6855,CBLB,-0.2840950579508489
-6865,BBX,-0.05814168307003777
-6868,AC012020.1,-0.10134826576987642
-6869,CD47,-0.23392678279998744
-6870,LINC01215,-0.07822911950518098
-6871,IFT57,0.0002441525285279204
-6875,DZIP3,-0.1696729307949329
-6877,TRAT1,-0.21142478028066802
-6891,CD96,-0.3298393283710896
-6894,PLCXD2,-0.1493422793453618
-6898,ABHD10,0.032686685932377904
-6902,GCSAM,-0.12312142161664018
-6908,BTLA,-0.0626375707889975
-6909,ATG3,0.3719448554256399
-6910,SLC35A5,0.04756579942357771
-6915,CD200R1,-0.03177491668120734
-6917,GTPBP8,-0.09434297853610697
-6918,NEPRO,-0.0019256151550404912
-6926,SPICE1,-0.018792632699123835
-6927,SIDT1,-0.17618521065755607
-6929,USF3,-0.01616420228921921
-6930,NAA50,0.127257189908437
-6932,ATP6V1A,0.3912174928843707
-6936,CCDC191,-0.035381814036257994
-6939,QTRT2,0.10704448366042783
-6943,TIGIT,-0.3180023508591249
-6945,ZBTB20,-0.09694176099631982
-6970,B4GALT4,-0.09289000548023012
-6972,ARHGAP31,0.13437904835068998
-6974,TMEM39A,-0.022211393233863227
-6975,POGLUT1,0.016476828265475495
-6977,TIMMDC1,-0.010157695416537365
-6980,ADPRH,0.2038699637588854
-6983,COX17,-0.009449530397669745
-6988,GSK3B,0.07079449136527222
-6991,LRRC58,0.1855215527856485
-6995,NDUFB4,-0.1272574228660213
-6997,RABL3,0.0472606169605412
-6998,GTF2E1,0.060837993735569834
-7007,HCLS1,0.2180887919276411
-7008,GOLGB1,0.08454740170228527
-7009,IQCB1,-0.03819478515813517
-7010,EAF2,0.1172487196093727
-7011,SLC15A2,0.11315407260592256
-7013,CD86,0.3830432019936958
-7015,CSTA,0.538812369967757
-7016,CCDC58,-0.04665697257649865
-7017,FAM162A,-0.18459786622852475
-7018,WDR5B,-0.02562546426652532
-7019,AC083798.2,-0.06464831596700422
-7020,KPNA1,0.07770147808182942
-7022,PARP9,0.36068732869043574
-7023,DTX3L,0.268906341490131
-7024,PARP15,-0.24740233627320288
-7025,PARP14,0.38948928978046393
-7026,HSPBAP1,0.08190698173496547
-7027,SLC49A4,0.10081770769162576
-7028,LINC02035,0.10782595110549306
-7030,PDIA5,0.05469773302640452
-7031,SEC22A,-0.10030050365105227
-7035,HACD2,0.016655431925331865
-7040,CCDC14,-0.07755159432544767
-7044,UMPS,-0.017294083523716777
-7048,HEG1,-0.04259908306968886
-7051,ZNF148,0.050280740037581644
-7052,SNX4,0.05327770645168671
-7053,OSBPL11,0.2904304569876975
-7061,SLC41A3,-0.0777862993783193
-7074,ZXDC,0.06725086700539419
-7076,CHST13,0.14523045876492496
-7080,CHCHD6,-0.11193315818549622
-7091,TPRA1,0.023354326668765402
-7095,ABTB1,0.1497097000342315
-7096,MGLL,0.07977982079655425
-7099,SEC61A1,0.13934061791887928
-7100,RUVBL1,-0.15300058755158277
-7102,EEFSEC,0.015292689090951907
-7110,RPN1,0.268358581972632
-7111,RAB7A,0.2627714089678231
-7115,ACAD9,0.02698973443027827
-7119,RAB43,0.018262809986359006
-7121,ISY1,-0.052027911111745144
-7122,AC108673.3,0.062164884315621435
-7123,CNBP,0.16299019954350777
-7124,COPG1,0.07218597427981636
-7126,HMCES,-0.20503749393058066
-7127,H1FX,0.18987962629791935
-7130,MBD4,0.08383365282788875
-7134,PLXND1,0.06599379933874772
-7135,TMCC1,0.05521125439759841
-7136,AC083799.1,0.07576719426671351
-7146,PIK3R4,-0.019157014869791853
-7148,ATP2C1,0.006761059098519362
-7150,ASTE1,-0.11706401174699627
-7154,NUDT16,0.3692021472919239
-7156,MRPL3,-0.059426826708525754
-7159,ACPP,0.2026473574649871
-7160,DNAJC13,0.2801019219264978
-7161,ACAD11,-0.017997962946168584
-7163,UBA5,-0.07818455353057066
-7164,NPHP3,-0.02159132125339917
-7172,CDV3,0.25503406285068864
-7173,TOPBP1,0.001953806909620098
-7175,SRPRB,-0.08226353853090743
-7180,RYK,-0.026828359549367443
-7184,ANAPC13,0.04393427581776424
-7185,CEP63,0.0753219140701674
-7192,MSL2,0.0863941001373537
-7193,PCCB,0.07630791055536718
-7194,STAG1,-0.009718435316195345
-7198,NCK1-DT,-0.058110065696591114
-7200,NCK1,-0.1317089323200445
-7210,DBR1,0.09401905947904608
-7211,ARMC8,0.025269293829554305
-7215,CEP70,-0.060275143200359024
-7216,FAIM,-0.05139959462187447
-7217,PIK3CB,0.1885222515506593
-7222,MRPS22,-0.03204771361000072
-7229,COPB2,0.07011134466509587
-7244,SLC25A36,0.05495185837623647
-7248,PXYLP1,-0.1363211967428355
-7251,ZBTB38,-0.1523220663028639
-7252,RASA2,-0.10512453927847507
-7254,RNF7,-0.035274285787458684
-7257,ATP1B3,0.1746794834781296
-7259,TFDP2,-0.3476406743144561
-7260,GK5,-0.20777417328631934
-7261,XRN1,0.143239665380622
-7262,ATR,0.0021916339947794213
-7272,U2SURP,-0.11252220412472282
-7275,CHST2,-0.189454613356155
-7277,SLC9A9,0.003955268590882305
-7281,DIPK2A,0.12498751711026554
-7290,PLSCR1,0.441178468285024
-7313,GYG1,0.15048337264860054
-7314,HLTF,-0.1746662730652299
-7316,HPS3,0.09269771108107942
-7328,COMMD2,-0.03411051854727958
-7330,RNF13,0.3380430032503425
-7339,TSC22D2,0.02536664791427782
-7340,SERP1,0.3342955342003293
-7341,EIF2A,0.004813205416853589
-7342,SELENOT,0.022633600923706865
-7346,SIAH2,0.06624969897122307
-7353,GPR171,-0.21084689154953273
-7354,P2RY14,0.04807091508064669
-7357,P2RY13,0.46935510061977054
-7366,MBNL1,0.07923086005929666
-7367,MBNL1-AS1,-0.033095880963079076
-7373,RAP2B,-0.03758313229546962
-7381,DHX36,-0.15791334351065717
-7396,SLC33A1,-0.02168769476166392
-7398,GMPS,-0.06574168809186295
-7401,AC091607.2,-0.031994250823241914
-7405,SSR3,0.14848859641342418
-7406,TIPARP-AS1,0.043740681005824095
-7407,TIPARP,0.10230268950564061
-7413,CCNL1,0.16806236154649035
-7415,AC092944.1,0.014294342118282824
-7424,RSRC1,-0.00956529623881545
-7425,AC106707.1,-0.10005478162927858
-7427,GFM1,0.07094759819674977
-7432,MFSD1,0.37064646720120886
-7445,IFT80,-0.09930618231271594
-7446,SMC4,-0.025513525289153478
-7447,TRIM59,-0.15134349958193208
-7448,KPNA4,0.023644981079845123
-7451,PPM1L,-0.018441618886219122
-7453,NMD3,0.02219675874774093
-7476,PDCD10,-0.04509044977140075
-7477,SERPINI1,-0.13906059893554462
-7480,GOLIM4,0.12648975235844787
-7493,MYNN,-0.052273843498801625
-7502,SEC62,-0.08315254946338868
-7504,GPR160,0.039110049866373855
-7505,PHC3,0.011810899085349932
-7506,PRKCI,0.08498552399058511
-7508,SKIL,0.1587353200852427
-7516,RPL22L1,-0.10723863910263402
-7517,EIF5A2,-0.10575625633934109
-7520,TNIK,-0.3100952216320265
-7522,PLD1,0.17614469886365788
-7527,FNDC3B,0.3168949286189844
-7529,TNFSF10,0.47179610998837734
-7531,NCEH1,0.16516112664553556
-7533,ECT2,-0.0022270858911402147
-7546,TBL1XR1,0.007863257376923786
-7561,ZMAT3,0.09164988586400866
-7563,PIK3CA,0.0999233283556968
-7565,ZNF639,-0.08547805404941429
-7567,MFN1,0.08075141976181638
-7568,GNB4,0.3350091192656033
-7571,ACTL6A,0.022471472587136214
-7574,MRPL47,-0.08067578329788538
-7575,NDUFB5,0.0995882765962361
-7576,USP13,-0.06858065137716073
-7584,TTC14,-0.023000610088468623
-7588,FXR1,-0.04180995531997341
-7589,DNAJC19,-0.2241652869286366
-7605,ATP11B,0.008031728264086416
-7607,DCUN1D1,0.10418191817229754
-7608,MCCC1,0.01951907885490116
-7613,KLHL6,0.10458419216008188
-7615,KLHL24,0.03215380045644416
-7616,YEATS2,0.08930620113328208
-7619,PARL,0.0853590169039849
-7620,ABCC5,-0.029854798409276715
-7628,EIF2B5,0.036240597412022436
-7629,DVL3,0.11274198967317883
-7630,AP2M1,0.1432847851815708
-7631,ABCF3,0.013283741602712633
-7633,ALG3,0.062195014269337684
-7634,EEF1AKMT4,0.04130351800826052
-7637,PSMD2,-0.035525368322156664
-7638,EIF4G1,0.14740265807525735
-7639,FAM131A,0.07011632686534414
-7641,POLR2H,-0.17484804351464106
-7648,MAGEF1,-0.03299498165649885
-7653,VPS8,0.0552202255089508
-7657,MAP3K13,0.04681896374992779
-7658,TMEM41A,0.033419826992887994
-7661,SENP2,0.056195634274698134
-7662,IGF2BP2,0.18714472760217774
-7664,TRA2B,0.062273717478252395
-7667,DGKG,0.17057278321731723
-7672,TBCCD1,-0.01408936167992921
-7673,DNAJB11,-0.016529335681374487
-7680,EIF4A2,-0.21838666160304201
-7681,RFC4,-0.1415640563325324
-7686,ST6GAL1,-0.12010243549605883
-7687,RPL39L,-0.15257752425697382
-7692,RTP4,0.03331610005359277
-7697,BCL6,0.31467297420020207
-7698,AC072022.2,0.04439728161462636
-7706,LPP,0.03524967355883985
-7717,IL1RAP,0.06475082973315258
-7724,CCDC50,0.08849463317993045
-7740,OPA1,0.046994702749263643
-7760,ATP13A3,0.09246605418932524
-7763,TMEM44-AS1,-0.022859473512958695
-7767,LSG1,-0.021696762127801316
-7768,FAM43A,-0.10906360439641337
-7774,XXYLT1,0.12957730480560997
-7778,ACAP2,0.35657672563303494
-7780,PPP1R2,-0.18541240220394817
-7787,TNK2,0.1677951983049691
-7791,TFRC,0.05813489087366686
-7795,PCYT1A,0.1687166513413388
-7797,TCTEX1D2,-0.1706366556965615
-7800,UBXN7,0.1189950000570333
-7802,RNF168,-0.053978754744615866
-7805,WDR53,0.03635469567729678
-7806,FBXO45,0.020421760250475412
-7809,NRROS,0.08955175117675451
-7810,PIGX,0.06288115749472387
-7811,CEP19,-0.0043717881840739945
-7812,PAK2,0.24895929343088646
-7813,SENP5,0.015851659365943804
-7814,AC016949.1,0.027348414067564473
-7815,NCBP2,-0.04042797896839922
-7817,NCBP2AS2,-0.09789919983333517
-7822,DLG1,0.07361977016103016
-7829,BDH1,-0.10173139666045705
-7833,AC024560.5,-0.12111554610552858
-7834,RUBCN,0.017799369550588764
-7835,FYTTD1,0.05912384774628677
-7837,LRCH3,0.019895612146205822
-7839,IQCG,0.014317310199941117
-7840,RPL35A,-0.33231654081919315
-7844,ZNF595,-0.2557248339741949
-7845,ZNF718,-0.08351596941659868
-7849,ZNF141,-0.10937376370953847
-7852,ZNF721,-0.15365225799076573
-7853,PIGG,-0.037753287660822095
-7856,PDE6B,-0.039941253055822426
-7858,AC107464.3,-0.09266566351093118
-7859,ATP5ME,-0.03669247686478203
-7860,MYL5,-0.05550870404124113
-7861,SLC49A3,0.15093793253697213
-7862,PCGF3,0.09529437700554118
-7865,AC139887.2,-0.11968974816310574
-7870,GAK,0.1397398416519899
-7871,TMEM175,0.09341397416230153
-7872,DGKQ,-0.15788260740674917
-7874,IDUA,0.0710357289878999
-7880,SPON2,-0.48071704642517776
-7882,CTBP1-AS,-0.11297272035637582
-7883,CTBP1,0.0442368716626115
-7884,CTBP1-DT,-0.17329889537177706
-7885,MAEA,0.003056819445456474
-7886,UVSSA,0.01914545287789604
-7892,AC147067.1,0.030461242320274577
-7894,SLBP,0.012796455085166468
-7895,AC016773.1,-0.027146614007508493
-7896,TACC3,0.11272904131076288
-7897,TMEM129,-0.04625575880255265
-7899,LETM1,0.03702535124244077
-7900,NSD2,0.00485122031960335
-7901,NELFA,-0.06922697318408323
-7902,C4orf48,0.3639117159138969
-7905,HAUS3,-0.07652999958947583
-7907,MXD4,-0.11846387082984386
-7908,ZFYVE28,-0.21441403384746602
-7911,RNF4,-0.03607679798797039
-7913,FAM193A,0.030650047719264932
-7914,TNIP2,-0.022060997140104805
-7915,SH3BP2,0.28653210330838264
-7916,ADD1,-0.019190422594685137
-7918,MFSD10,-0.062116713187654674
-7919,NOP14-AS1,0.01600654643221263
-7920,NOP14,-0.02812139070100071
-7922,HTT,0.07484198483334817
-7925,RGS12,0.06800068352776323
-7929,LRPAP1,0.001473413198868886
-7937,TMEM128,0.04898331730805961
-7938,LYAR,-0.3620016222492085
-7939,ZBTB49,0.003709218692613713
-7941,NSG1,-0.16130576680505176
-7942,STX18,-0.03660670400655665
-7944,STX18-AS1,-0.0718758550233838
-7961,MAN2B2,-0.013058033733027507
-7962,MRFAP1,0.019292683797569415
-7963,LINC02482,-0.17992980675536882
-7964,AC093323.1,-0.1868958118635085
-7967,MRFAP1L1,-0.16493886675591127
-7968,BLOC1S4,-0.21700519336578422
-7970,KIAA0232,0.08425469832165998
-7971,TBC1D14,0.17306443201873434
-7972,AC097382.3,-0.00949998333195558
-7976,TADA2B,0.0675229275855305
-7977,GRPEL1,-0.04817375961962366
-7982,AFAP1,0.008938869944643997
-7988,SH3TC1,0.23623667147126984
-7992,ACOX3,0.04095302851528818
-7993,TRMT44,-0.0010979485475590218
-8026,SLC2A9,0.135545666940438
-8030,AC005674.2,0.013960747349936856
-8031,WDR1,0.023982692771557085
-8032,ZNF518B,0.06916300883187824
-8046,RAB28,0.04778104695260649
-8051,BOD1L1,0.17714709994149935
-8061,CPEB2,0.16731245482695875
-8067,FBXL5,0.44990816664335953
-8068,FAM200B,0.14723585150773583
-8069,BST1,0.39924380577488
-8070,CD38,-0.16314595300359014
-8072,FGFBP2,-0.5828075944632736
-8075,TAPT1,-0.027987328776005103
-8084,QDPR,-0.010949322086279752
-8086,LAP3,0.5374225873507635
-8088,MED28,0.062419290732269234
-8089,FAM184B,-0.045399955790954356
-8090,DCAF16,-0.024849766755293278
-8092,LCORL,0.08567512983780899
-8100,PACRGL,-0.06968656199460115
-8116,DHX15,0.12405447760313115
-8119,CCDC149,0.15023111009813092
-8121,SEPSECS,0.06440617053015057
-8122,SEPSECS-AS1,-0.09203944473728912
-8123,PI4K2B,0.044158047373477366
-8125,ZCCHC4,-0.08244340267408425
-8126,ANAPC4,-0.04012922658750458
-8129,SEL1L3,-0.0607127786342901
-8131,SMIM20,-0.09519634624577003
-8134,RBPJ,0.30593245632076643
-8136,TBC1D19,-0.1607849160289094
-8137,STIM2,-0.1685436268513922
-8171,ARAP2,-0.027128195333848106
-8173,DTHD1,-0.3473618260488518
-8183,RELL1,0.1566244049100894
-8184,PGM2,0.09651402297033389
-8185,TBC1D1,0.17533968949299847
-8193,KLF3-AS1,-0.07032831639009414
-8195,KLF3,0.053028555163336065
-8196,TLR10,0.018983849588030625
-8197,TLR1,0.28898819567306944
-8198,TLR6,0.2487776803244024
-8199,FAM114A1,0.11504561438630577
-8200,TMEM156,-0.14067140214282994
-8201,KLHL5,0.24417292081088934
-8202,AC079921.1,-0.03942145785822678
-8204,WDR19,-0.04419070959347006
-8205,RFC1,-0.11343782648613328
-8207,RPL9,-0.25453036243987115
-8208,LIAS,0.00543571549738609
-8209,UGDH,0.02063054094345541
-8212,SMIM14,0.241822237063139
-8215,UBE2K,0.038528061857272144
-8216,PDS5A,0.11603307855994144
-8217,N4BP2,0.01665957809616221
-8220,RHOH,-0.27630204127706415
-8224,RBM47,0.4186543275379054
-8240,TMEM33,0.22529611786195156
-8243,SLC30A9,0.06338276870761768
-8248,ATP8A1,-0.12042684234864885
-8260,GUF1,0.019491560182482763
-8261,GNPDA2,-0.11184508974863377
-8273,COMMD8,0.0380255416956829
-8276,ATP10D,0.14759877776884014
-8281,NFXL1,-0.024659045234960885
-8284,TXK,-0.3636447380010776
-8285,TEC,0.08952946055759095
-8286,SLAIN2,0.01290911627498759
-8289,FRYL,-0.046428930581832624
-8290,OCIAD1,-0.1569493108820384
-8292,OCIAD2,-0.31064925673803956
-8294,DCUN1D4,-0.002321658435658164
-8297,SGCB,0.005125976854425557
-8301,USP46,-0.07738567869629907
-8305,DANCR,-0.1968295140088441
-8309,SCFD2,0.016864328648440115
-8311,FIP1L1,-0.049764598655415326
-8318,CHIC2,-0.03791103461297669
-8334,SRD5A3,0.08992769253746191
-8337,TMEM165,0.18001868630092605
-8338,CLOCK,-0.07480267292919437
-8343,EXOC1,0.08115873406967014
-8346,CEP135,-0.013422874378893347
-8349,AASDH,-0.09681401344760757
-8351,PPAT,-0.06152330847150748
-8353,PAICS,-0.04685787786101414
-8354,SRP72,0.07181552162201517
-8357,HOPX,-0.5975595586450315
-8359,REST,-0.08192771486051624
-8361,NOA1,-0.03154078691757527
-8362,POLR2B,-0.07101917247044925
-8363,IGFBP7,-0.25958151684637115
-8403,CENPC,-0.12902539459522763
-8404,STAP1,-0.0583177440171262
-8405,UBA6,0.07509569420849879
-8406,UBA6-AS1,-0.09069596953570164
-8412,YTHDC1,-0.07399088206806383
-8426,SULT1B1,-0.019969368650050022
-8446,JCHAIN,-0.003159642791163291
-8447,UTP3,0.06686421121063885
-8449,RUFY3,0.2364354231845043
-8450,GRSF1,0.12376367667355322
-8451,MOB1B,0.14602435542136802
-8452,DCK,-0.02122930130583101
-8459,COX18,-0.00917209921010763
-8460,ANKRD17,0.11411458098680037
-8473,PPBP,0.07294720841112784
-8479,MTHFD2L,-0.006668669045757595
-8497,RCHY1,0.05342555144474422
-8498,THAP6,0.02221927382656926
-8501,G3BP2,0.042936141294763856
-8502,USO1,0.05360952946701822
-8505,NAAA,0.27781849874248143
-8506,SDAD1,-0.03384151632627905
-8510,CXCL10,0.12674831277412144
-8512,NUP54,-0.03665651136139037
-8514,SCARB2,0.28653195731484377
-8523,SEPTIN11,-0.07790059983072599
-8524,CCNI,0.18510294780340222
-8525,CCNG2,0.1345406767486242
-8528,CNOT6L,-0.22989090833013565
-8529,MRPL1,-0.12691863326844058
-8535,BMP2K,0.2913022880365029
-8536,PAQR3,-0.01673565161123878
-8544,ANTXR2,0.1319121058729463
-8552,RASGEF1B,0.12208591607867092
-8553,AC124016.2,-0.0975908720171235
-8554,HNRNPD,0.10268392599307069
-8556,HNRNPDL,-0.13837665920398942
-8557,ENOPH1,0.029989187732309817
-8562,SEC31A,0.12451693279434232
-8563,THAP9-AS1,0.06045745142830188
-8565,LIN54,-0.015763654210187304
-8566,COPS4,-0.03825017279136875
-8568,PLAC8,0.03455122065783618
-8570,COQ2,0.15894500453904692
-8571,HPSE,0.2575503191871873
-8572,HELQ,-0.03652389226971568
-8573,MRPS18C,-0.015075117340350504
-8574,ABRAXAS1,-0.1073220094379943
-8575,GPAT3,0.20936269689606937
-8581,WDFY3,0.3343081921670511
-8585,ARHGAP24,0.13072630809864705
-8592,AFF1,0.0042335881957839844
-8593,KLHL8,0.22815952328487832
-8596,HSD17B11,0.2322567550113275
-8597,NUDT9,-0.028397094797728287
-8607,PKD2,0.06504342887245046
-8609,PPM1K,-0.05471252192769501
-8613,HERC6,0.009331652102970272
-8614,HERC5,0.19300398285790835
-8615,PYURF,-0.14032502973836816
-8618,HERC3,0.13330867391367338
-8620,FAM13A-AS1,0.14552942408981412
-8621,FAM13A,0.1485194416301357
-8624,GPRIN3,-0.26813474769006307
-8626,SNCA,0.10664373989338326
-8645,SMARCAD1,-0.09890742331829755
-8648,PDLIM5,0.232200750250601
-8662,RAP1GDS1,0.011933117257256336
-8663,TSPAN5,-0.17013915250416103
-8666,EIF4E,-0.04067310606907543
-8669,METAP1,0.014116223831166973
-8671,ADH5,-0.13726438199313767
-8680,TRMT10A,-0.020108668039219423
-8684,DAPP1,0.33220644520276904
-8685,LAMTOR3,0.06647548207780622
-8686,DNAJB14,-0.04794347647529722
-8687,H2AFZ,-0.09059264560912982
-8696,PPP3CA,0.10111825730436656
-8698,BANK1,0.041520452666429175
-8701,SLC39A8,0.0028618497768396495
-8704,NFKB1,0.11593712481652886
-8705,MANBA,0.234257286808729
-8706,UBE2D3,0.27033789465591873
-8709,CISD2,0.09610095956879171
-8712,BDH2,-0.12462475284132918
-8726,AC004069.1,0.0544744403302805
-8727,TET2,0.3883786010147286
-8729,PPA2,0.02455614132444616
-8733,INTS12,0.013050218102545164
-8734,GSTCD,0.002356531391327379
-8739,TBCK,0.019635978850678342
-8740,AIMP1,-0.006074509683966785
-8746,PAPSS1,0.2080143860856814
-8747,SGMS2,0.22840844767178153
-8749,CYP2U1,0.03465759213904795
-8751,HADH,-0.012899667150280939
-8752,LEF1,-0.16497044636273214
-8753,LEF1-AS1,-0.060511953011381625
-8755,RPL34,-0.3603574988471146
-8756,OSTC,-0.15022698502057122
-8762,SEC24B,0.09615301931509411
-8763,MCUB,0.004201788943754147
-8764,CASP6,-0.06155793788705939
-8766,PLA2G12A,0.015537464354280321
-8769,GAR1,-0.06098586737068718
-8783,FAM241A,0.18953866489379934
-8785,AP1AR,0.11561256824734918
-8787,TIFA,0.08992494793817805
-8788,ALPK1,0.23847109580332967
-8792,LARP7,0.04477124052575982
-8799,CAMK2D,0.13037247235760502
-8817,SNHG8,-0.13237856001743845
-8820,METTL14,0.011503791767176518
-8821,SEC24D,0.19201159980033725
-8824,USP53,-0.10794962232557892
-8825,C4orf3,0.07668630455356111
-8842,ANXA5,0.5118032212254995
-8845,EXOSC9,-0.03942307299114636
-8847,BBS7,-0.046493583483013357
-8850,KIAA1109,0.15228055545836655
-8860,SPATA5,-0.06583745606028897
-8871,ANKRD50,0.1981390246785086
-8885,MFSD8,-0.0735236886622235
-8886,ABHD18,0.030284138612674587
-8887,LARP1B,-0.04157643373165685
-8888,PGRMC2,-0.09165253133656012
-8894,JADE1,-0.023909245909942395
-8895,SCLT1,0.22316127983351852
-8896,C4orf33,0.13145024715077228
-8950,ELF2,0.07136312258496365
-8952,NDUFC1,-0.010320841382506716
-8953,NAA15,-0.008896705709240067
-8954,AC097376.3,-0.03116642458941748
-8955,RAB33B,0.025588098475199026
-8956,SETD7,0.20315312222845858
-8960,MGST2,0.21221242965209908
-8961,MAML3,0.24493992600920225
-8964,SCOC,-0.08427909417563965
-8968,ELMOD2,-0.06012443393530113
-8971,TBC1D9,0.2798739521030415
-8973,AC096733.2,0.05532096879259489
-8975,ZNF330,0.1150279179280062
-8977,LINC02432,0.2839267407919806
-8980,IL15,0.2941714947542316
-8981,INPP4B,-0.20265006555337528
-8985,USP38,0.08469975027333947
-8987,GAB1,0.10162298846647567
-8990,SMARCA5,-0.05226610523163035
-9001,ANAPC10,-0.09657492529596612
-9003,ABCE1,0.003868165331208794
-9004,OTUD4,0.04039430056564711
-9007,SMAD1,0.13013720187568836
-9010,MMAA,-0.02627590855326912
-9013,ZNF827,-0.1827434913180039
-9017,LSM6,-0.04962830854905692
-9020,SLC10A7,0.0037503260056823014
-9030,TMEM184C,0.04343601434862572
-9031,PRMT9,-0.0947351195029817
-9032,ARHGAP10,-0.05327011173902323
-9047,LRBA,-0.14272003604438963
-9050,RPS3A,-0.3717912701357127
-9057,GATB,0.043504862687923515
-9067,LINC02273,-0.11626413936582426
-9073,FBXW7,0.12331296630635194
-9076,MIR4453HG,-0.04761949420524798
-9077,TMEM154,0.3679510078718408
-9080,ARFIP1,0.12670040501920646
-9086,TMEM131L,0.10559417145266227
-9088,TLR2,0.4833328269096885
-9096,PLRG1,-0.05109249341589774
-9107,MAP9,-0.16124294526667554
-9110,GUCY1A1,0.08853628696584581
-9115,CTSO,0.13471003442612337
-9120,PDGFC,0.1381299927056326
-9129,GASK1B,0.41440993012438154
-9130,FAM198B-AS1,0.24082348602031126
-9133,TMEM144,0.23641842576283786
-9138,ETFDH,0.07973840591399668
-9139,PPID,-0.07345662263995426
-9140,FNIP2,0.22290107247243734
-9142,RAPGEF2,0.2206835138423049
-9152,NAF1,-0.008095854158378327
-9156,TMA16,0.0591194416522419
-9157,MARCH1,0.5550058576168719
-9167,TMEM192,0.0715548558340797
-9168,KLHL2,0.06764056641025815
-9170,MSMO1,-0.20094911259961407
-9181,DDX60,0.1426493653665371
-9182,DDX60L,0.25864499814520137
-9183,PALLD,-0.03222368909733777
-9186,CBR4,0.004656681663342312
-9188,SH3RF1,0.1351772958893287
-9190,NEK1,-0.043287491963760714
-9192,CLCN3,0.0252748054490372
-9193,HPF1,-0.12065732009620599
-9213,GALNT7,0.1349321112656198
-9215,HMGB2,0.1932853024734827
-9217,SAP30,0.09799847329061172
-9231,FBXO8,-0.011594114292824224
-9232,CEP44,-0.01581313918469531
-9234,HPGD,-0.10851806030406265
-9242,GPM6A,-0.0818182445351569
-9248,SPCS3,0.08982415186411752
-9257,AGA,-0.09120042530040848
-9278,DCTD,-0.05439932850565246
-9283,WWC2,0.0906456931182519
-9287,CDKN2AIP,-0.13028452752757752
-9289,ING2,0.015889021645517473
-9291,RWDD4,-0.06677907537744683
-9292,TRAPPC11,0.09824312563969445
-9301,IRF2,0.03011608163316896
-9303,AC099343.3,-0.010922953220372131
-9307,CASP3,-0.01159279428873254
-9308,PRIMPOL,0.018009444498588396
-9310,ACSL1,0.3625625318835063
-9319,SLC25A4,-0.20443545541202757
-9320,CFAP97,-0.19575778214738573
-9324,ANKRD37,-0.07239662278183315
-9325,UFSP2,-0.07852214375096214
-9336,CYP4V2,-0.0453516840211674
-9373,FRG1,-0.061354953001137136
-9380,CCDC127,-0.11057793991517334
-9381,SDHA,-0.0565404982845407
-9383,PDCD6,-0.06212444760431276
-9386,EXOC3,0.07506368885795876
-9398,BRD9,0.025524015015550183
-9406,SLC12A7,0.22235671436591942
-9412,CLPTM1L,0.07359237432651365
-9416,LPCAT1,-0.05827830933310792
-9420,MRPL36,-0.18845251348076067
-9421,NDUFS6,0.08440409366900098
-9462,ICE1,-0.02456376492012195
-9467,MED10,-0.22059087142639694
-9471,NSUN2,0.057835095930869045
-9472,SRD5A1,0.21407466672190287
-9474,TENT4A,0.014145768227013269
-9490,MTRR,0.039652200561634356
-9491,FASTKD3,0.0016640570961655075
-9498,MIR4458HG,-0.024993874989696546
-9517,ATPSCKMT,-0.014068391563088572
-9519,CCT5,0.0018418517430147155
-9524,MARCH6,-0.05469635257319415
-9533,DAP,0.09353433793994097
-9543,TRIO,0.1923157091031804
-9545,OTULINL,0.22241007968826798
-9547,OTULIN,-0.04586570094278257
-9548,ANKH,-0.12880528874018565
-9561,ZNF622,-0.08047502148715067
-9562,RETREG1,-0.09544771489826655
-9566,BASP1,0.203975883951686
-9640,DROSHA,0.03464928739995989
-9641,C5orf22,0.023383119026536628
-9650,GOLPH3,0.13707891647436699
-9651,AC025181.2,0.08957611512538774
-9652,MTMR12,0.05372020946300318
-9653,ZFR,0.11880055301398144
-9654,SUB1,-0.234780484326541
-9664,TARS,-0.04102602394799863
-9669,AMACR,-0.09272053410389752
-9678,RAD1,-0.03869185519715522
-9679,BRIX1,-0.11444775552951975
-9680,DNAJC21,-0.03542760008076016
-9682,PRLR,0.1382044295702888
-9683,SPEF2,-0.07405165748399473
-9685,IL7R,-0.2756148018799732
-9690,LMBRD2,0.022769795622443446
-9691,SKP2,0.002500504320984449
-9692,NADK2,0.07115524637848102
-9700,NIPBL-DT,-0.015792393215115
-9701,NIPBL,0.2500225978151325
-9702,CPLANE1,6.627438886899169e-05
-9704,NUP155,0.03693227509258943
-9705,WDR70,-0.011330037703217994
-9725,RICTOR,0.06379158646921144
-9727,FYB1,0.4189755857334603
-9729,DAB2,-0.030263772984356223
-9737,TTC33,0.05952100202633664
-9738,PTGER4,0.02895233338556173
-9739,PRKAA1,-0.040420673229577146
-9740,RPL37,-0.3402963352426163
-9741,CARD6,0.22916433182626336
-9746,OXCT1,-0.19049131626112367
-9749,C5orf51,-0.018168401856678805
-9750,FBXO4,-0.10069891482515343
-9753,CCDC152,-0.1508827038554329
-9757,AC008875.3,-0.1562613384700555
-9760,AC025171.1,-0.09126610623578028
-9761,AC025171.3,0.07119746766452283
-9763,ANXA2R,-0.31574893999629333
-9764,AC025171.2,-0.0034908300719247334
-9766,ZNF131,-0.09963580622694469
-9768,HMGCS1,0.03601428669326155
-9770,CCL28,-0.10919698890558795
-9771,TMEM267,-0.025807472997087508
-9776,PAIP1,0.03081737012723098
-9777,NNT-AS1,-0.056550945841641886
-9778,NNT,-0.00011754478004528617
-9786,MRPS30,-0.051573871295513905
-9792,EMB,-0.041608314086915746
-9794,PARP8,0.01788748264105334
-9806,PELO,0.015955211197060042
-9812,MOCS2,-0.05322878788610112
-9816,NDUFS4,-0.03468097753778765
-9819,ARL15,0.08823524255386923
-9823,SNX18,0.20420181140999588
-9828,GZMK,-0.24551020504266943
-9831,GZMA,-0.7259227340063925
-9837,DHX29,0.06281993952389373
-9838,MTREX,0.025343197845177298
-9840,SLC38A9,0.0021200394672540633
-9842,IL31RA,0.11910519141279716
-9843,IL6ST,0.1468331229530067
-9847,ANKRD55,-0.04580043323158002
-9849,LINC01948,0.055738650830793046
-9854,MAP3K1,0.32352226413969093
-9857,SETD9,0.002442097699980177
-9858,MIER3,0.01189246713407343
-9861,GPBP1,-0.06046436123109149
-9871,GAPT,0.24844856892830633
-9876,PDE4D,-0.039181898600936536
-9884,ERCC8,-0.001752141811791387
-9886,NDUFAF2,-0.035189744351579816
-9887,SMIM15,0.06378219242112718
-9890,ZSWIM6,0.26910696619141317
-9895,KIF2A,-0.11506221752419098
-9896,DIMT1,-0.07081600705912144
-9897,IPO11,-0.04626997748070661
-9905,SREK1IP1,0.0010952620871878074
-9906,CWC27,0.008257848041443933
-9909,ADAMTS6,-0.06255819446778847
-9911,CENPK,-0.14720474515909618
-9912,PPWD1,-0.045127335848536156
-9913,TRIM23,0.006866768245080089
-9914,SHLD3,-0.048101993695340635
-9915,TRAPPC13,0.05892572566056909
-9916,SGTB,0.12278769942591286
-9917,NLN,0.2731696484321267
-9919,ERBIN,0.1759528399153463
-9920,AC010359.3,0.03465877507893711
-9922,SREK1,0.0778724901598099
-9928,MAST4,-0.03726860978753565
-9931,CD180,0.18729429183486862
-9940,PIK3R1,-0.20053862960898544
-9949,SLC30A5,0.04179779824698753
-9952,CENPH,-0.04885013749474409
-9953,MRPS36,-0.03196255748837771
-9954,CDK7,0.06506031715009025
-9955,CCDC125,-0.024380812704890977
-9956,AK6,-0.12493786533172602
-9957,TAF9,0.04438355229228321
-9958,RAD17,-0.10603632334877561
-9962,GTF2H2C,-0.08402060627970048
-9968,SMN1,-0.02427826062021582
-9970,NAIP,0.42644825143404175
-9971,GTF2H2,-0.0338546116637848
-9975,BDP1,0.007056255293629182
-9976,MCCC2,0.04220984391257312
-9980,MRPS27,-0.07814389137255126
-9981,PTCD2,-0.01687304073906856
-9987,TNPO1,0.03674957137227263
-9990,FCHO2,0.07068442224356453
-10002,BTF3,-0.23566633322543284
-10003,ANKRA2,-0.10243325611575906
-10004,UTP15,0.014025537213829854
-10013,ENC1,0.16104735997885497
-10014,HEXB,0.21646052630306914
-10015,GFM2,-0.005768463043974085
-10016,NSA2,-0.10549972217642359
-10024,HMGCR,0.09656357055034062
-10025,CERT1,0.2772337892254688
-10027,POLK,0.3048614422222934
-10031,POC5,-0.051655995570818306
-10035,IQGAP2,0.09496511562683138
-10039,F2R,-0.1765380548035368
-10041,S100Z,0.282542503258058
-10043,AGGF1,0.07853066538544547
-10044,ZBED3,0.06870404801081462
-10048,WDR41,0.06481501307557008
-10050,TBCA,0.004524309428383565
-10053,AP3B1,0.040693817201430775
-10054,SCAMP1-AS1,-0.1098681044678993
-10055,SCAMP1,-0.014561332578324896
-10056,LHFPL2,0.18911055378151756
-10057,ARSB,0.15101871162213837
-10061,JMY,-0.09820523650569578
-10062,HOMER1,0.07614162291971284
-10063,TENT2,0.12782909113048763
-10068,MTX3,-0.04550119947534074
-10071,SERINC5,0.09980329857322869
-10074,ZFYVE16,0.22466685464558797
-10075,AC008771.1,0.02464531525801111
-10079,DHFR,0.03702420884517438
-10081,MSH3,0.04583386119279253
-10083,RASGRF2,-0.09103788841342782
-10085,CKMT2-AS1,-0.023615010863680674
-10087,ZCCHC9,-0.03218260005083224
-10090,SSBP2,-0.03148240274215509
-10093,ATG10,-0.018513103576581653
-10096,RPS23,-0.33853312160200005
-10106,TMEM167A,0.25388544907663185
-10107,XRCC4,0.15082959017291536
-10108,VCAN,0.5447241166878943
-10123,COX7C,-0.19405324994468143
-10128,RASA1,0.040238867740552846
-10129,CCNH,-0.03648320026490081
-10135,TMEM161B,-0.02602966355180124
-10136,TMEM161B-AS1,-0.1746449084669817
-10143,MEF2C,0.31891936728419024
-10151,CETN3,0.010408444057457673
-10153,MBLAC2,0.0024780938359980896
-10155,LYSMD3,0.07754366939258296
-10157,LUCAT1,0.1612691207704238
-10164,ARRDC3,-0.03972461582723815
-10177,FAM172A,-0.15475281595658597
-10179,POU5F2,0.059923805263661124
-10182,AC114980.1,0.051592723973884334
-10186,SLF1,-0.11965822114438103
-10187,MCTP1,0.3348628271564881
-10191,TTC37,-0.023305375430933476
-10192,ARSK,0.014575085133763164
-10198,GLRX,0.30237290376973214
-10203,ELL2,0.11865461231888035
-10210,CAST,0.2665870414644574
-10212,ERAP1,0.08080912906142873
-10213,AC008906.1,-0.07495013477839331
-10215,AC009126.1,-0.017029334165875185
-10216,ERAP2,0.14748903069404756
-10217,LNPEP,-0.048686005785236366
-10220,RIOK2,-0.030224787604367642
-10231,CHD1,0.310122148569758
-10236,FAM174A,-0.03313766645550207
-10238,ST8SIA4,0.14358132712235538
-10240,SLCO4C1,-0.1453546350217272
-10247,PAM,0.008034666922983292
-10248,GIN1,-0.00693913062854687
-10249,PPIP5K2,0.08972209095322684
-10263,FBXL17,0.04658005448104118
-10266,FER,0.11435460515892379
-10269,AC008467.1,-0.13207578061271233
-10270,PJA2,0.1440904532193461
-10275,MAN2A1,0.08203822192005618
-10278,SLC25A46,-0.014819915550813327
-10282,WDR36,0.017226481038140885
-10283,CAMK4,-0.17544323380881435
-10286,STARD4,-0.03491721860977338
-10288,NREP,0.1103313122100939
-10291,EPB41L4A-AS1,-0.1736897494717603
-10297,APC,0.09434653445798703
-10298,SRP19,-0.1798331896145121
-10299,REEP5,0.157724198206602
-10300,DCP2,0.17719898083333516
-10304,YTHDC2,0.07367892601303198
-10314,PGGT1B,0.13181712940802207
-10316,CCDC112,0.10445335359422832
-10317,FEM1C,0.11075353376690045
-10318,TICAM2,0.286558893764155
-10320,TMED7,0.17311634274722112
-10324,ATG12,-0.012467185082513883
-10325,AP3S1,0.08489588526181406
-10330,AC034236.2,-0.023147117799557554
-10331,COMMD10,0.06942978929397024
-10348,DTWD2,-0.0626947895585135
-10350,DMXL1,0.14224027824037183
-10351,TNFAIP8,-0.20466119283230633
-10352,HSD17B4,0.2416710013591333
-10360,SRFBP1,0.020292559587711825
-10372,SNX2,0.33110948232597237
-10374,SNX24,-0.02468280409805274
-10379,CEP120,-0.013845748193812778
-10380,CSNK1G3,-0.03761387885566911
-10398,GRAMD2B,-0.04069633167971626
-10400,ALDH7A1,-0.06889406868849487
-10401,PHAX,-0.10017491963416397
-10404,LMNB1,0.2582016948212292
-10406,C5orf63,-0.07335698053949319
-10411,PRRC1,0.04156730327490468
-10417,LINC01184,-0.057433365551225524
-10418,SLC12A2,-0.03320845910978571
-10419,FBN2,0.183314634310229
-10422,ISOC1,-0.08331274495086644
-10430,HINT1,-0.3090507505377452
-10431,LYRM7,-0.022667304738541883
-10432,CDC42SE2,-0.17959971186437437
-10433,RAPGEF6,-0.15172693594816677
-10434,FNIP1,0.09474507924174846
-10437,ACSL6,-0.10796752468950034
-10446,SLC22A4,0.11211384950177256
-10447,MIR3936HG,-0.037402110633886966
-10448,SLC22A5,-0.008743935586233879
-10449,C5orf56,-0.23735637750329341
-10451,AC116366.1,0.07950195174097047
-10452,IRF1,0.3105527065024831
-10454,RAD50,0.017921301862891866
-10459,KIF3A,-0.10123516939596498
-10464,SHROOM1,0.17926559743063258
-10466,UQCRQ,0.0016507028249110921
-10467,LEAP2,-0.14276444417686215
-10468,AFF4,0.1028564371697716
-10470,ZCCHC10,-0.008919091431181338
-10473,HSPA4,0.05727388963114862
-10481,C5orf15,0.06861474942683912
-10482,VDAC1,0.05439266272755987
-10484,TCF7,-0.23019217203841352
-10485,SKP1,-0.28780950864862875
-10486,PPP2CA,0.08436685826841102
-10489,UBE2B,0.037609242151351585
-10491,CDKN2AIPNL,-0.10252055926300262
-10495,JADE2,-0.15773527400624657
-10496,SAR1B,0.018775394335782007
-10497,SEC24A,0.1272758030348309
-10498,CAMLG,-0.11839094562247555
-10499,DDX46,-0.09477058310302884
-10500,C5orf24,0.044220476108599714
-10501,TXNDC15,-0.12476860928109904
-10502,PCBD2,0.058095997316516025
-10511,H2AFY,0.4938109885533153
-10514,TIFAB,0.032848640568854
-10526,TGFBI,0.47068365675691337
-10528,SMAD5,0.12914420395042667
-10538,HNRNPA0,-0.043428366246590036
-10540,AC106791.1,-0.0562661167347277
-10543,FAM13B,0.05868733576851504
-10548,BRD8,-0.02208544261082086
-10550,CDC23,-0.08603856108612147
-10553,FAM53C,0.08419920280426607
-10555,KDM3B,0.09697187840167527
-10557,EGR1,0.09283376869232902
-10558,ETF1,0.08897152280741188
-10559,HSPA9,-0.01845783036423724
-10560,CTNNA1,0.3125820896743361
-10563,SIL1,0.14335374061225942
-10566,SNHG4,0.01357922659955517
-10567,MATR3,-0.038675781272228336
-10568,PAIP2,0.003605507669096111
-10571,MZB1,-0.0230421107806085
-10577,TMEM173,-0.04355430724311801
-10579,UBE2D2,-0.20401467442542368
-10581,CXXC5,-0.08759578800620378
-10591,PURA,-0.12227534507317812
-10592,IGIP,-0.09911045065156056
-10594,CYSTM1,0.15490502861275376
-10596,PFDN1,-0.04531210268368414
-10597,HBEGF,0.1229371763312422
-10601,ANKHD1,0.18525660997289728
-10602,SRA1,0.04953966180638912
-10603,EIF4EBP3,0.02372822771450144
-10604,APBB3,0.09543373634216107
-10605,SLC35A4,0.03369582918686398
-10606,CD14,0.5080679217268551
-10607,NDUFA2,0.021395690432759525
-10608,TMCO6,-0.08788081961065398
-10609,IK,-0.09376273454042168
-10610,WDR55,0.07846689450984087
-10612,HARS,-0.09741453647649226
-10613,HARS2,0.05971100537286767
-10614,ZMAT2,0.029073743287795082
-10660,TAF7,-0.14626166945815236
-10689,DIAPH1,0.0629602709082795
-10692,HDAC3,-0.03178305288728103
-10693,RELL2,0.028993950046158255
-10694,FCHSD1,-0.0405139450906083
-10695,ARAP3,0.12115913436365347
-10701,DELE1,0.01941655676678613
-10704,RNF14,-0.08590313213902627
-10706,GNPDA1,0.05818643504098866
-10707,NDFIP1,-0.08908742383487987
-10714,ARHGAP26,0.3087472287029177
-10717,NR3C1,0.11554100052263513
-10722,YIPF5,-0.003550040154877264
-10734,LARS,-0.044860057418616296
-10735,RBM27,-0.032636365093806824
-10740,TCERG1,0.07297884858403882
-10743,PPP2R2B,-0.3182173378029108
-10766,FBXO38,0.11473597914117417
-10770,ADRB2,0.00600503988037304
-10779,GRPEL2,-0.05087695502706438
-10781,PCYOX1L,0.06536723026605835
-10786,CSNK1A1,0.2586571789398263
-10789,PPARGC1B,0.09925770601341626
-10791,SLC26A2,0.019583944140373937
-10793,HMGXB3,0.029032520817934112
-10794,CSF1R,0.3926171073725566
-10800,TCOF1,-0.038952748115286644
-10801,CD74,0.4691360291137732
-10803,RPS14,-0.4259171431788925
-10805,NDST1,0.16666861359269894
-10811,RBM22,-0.030124591224404702
-10812,DCTN4,0.08446883077956441
-10813,SMIM3,0.0648143388782345
-10817,TNIP1,0.14896836051482748
-10818,ANXA6,-0.3552770461096477
-10820,CCDC69,0.18004183861358433
-10821,GM2A,0.3106246124783482
-10824,SLC36A1,0.1960702279141166
-10832,ATOX1,0.16292710296131555
-10835,G3BP1,-0.09630860188023635
-10844,FAM114A2,-0.11189161701099806
-10845,MFAP3,-0.015909000250526256
-10846,GALNT10,-0.12770582986679685
-10847,SAP30L-AS1,-0.07291185951613664
-10849,SAP30L,0.01603238120495835
-10852,LARP1,0.20996809962264076
-10854,CNOT8,0.08828414289277911
-10855,GEMIN5,0.033143881892104675
-10856,MRPL22,-0.0602866134198469
-10865,HAVCR2,-0.008527980939963362
-10867,MED7,-0.1391589991652118
-10868,ITK,-0.20612273858985003
-10872,CYFIP2,-0.28363265194049964
-10876,ADAM19,0.05803381356592184
-10881,THG1L,0.05919679076971172
-10882,LSM11,0.0015639278348555394
-10883,CLINT1,0.09929657348131447
-10893,EBF1,0.0003836930723564931
-10897,RNF145,-0.020139631819097368
-10900,UBLCP1,-0.026700349913052052
-10907,TTC1,0.0020682382048955047
-10908,PWWP2A,0.12382678019548991
-10914,SLU7,0.10127273713322668
-10915,PTTG1,-0.14740033863370716
-10928,CCNG1,0.014025561161623469
-10929,NUDCD2,-0.14408348637264842
-10932,MAT2B,0.010631853696208005
-10956,RARS,0.0933033282067228
-10958,PANK3,0.0698014333699749
-10964,SPDL1,-0.01352216741355714
-10965,DOCK2,0.3019055768285492
-10971,LCP2,0.17740700355333322
-10975,KCNMB1,0.19930417539370762
-10984,NPM1,-0.3348995917529061
-10989,FBXW11,-0.011456012510275407
-10990,STK10,-0.02353344791046729
-10992,UBTD2,0.1082536971138293
-11000,DUSP1,0.5522389153473157
-11002,ERGIC1,0.13135023200354962
-11005,RPL26L1,-0.04952263983877344
-11006,ATP6V0E1,-0.04996345362408512
-11007,CREBRF,0.06933759513315234
-11009,BNIP1,-0.07909147499752892
-11017,BOD1,-0.08871150045801167
-11023,CPEB4,0.1768676612950889
-11039,SFXN1,-0.13941198444242275
-11040,HRH2,0.35162970806166777
-11043,THOC3,-0.150659871802907
-11051,SIMC1,0.012148079701287643
-11052,KIAA1191,-0.019894080518962923
-11056,NOP16,-0.05241471176063997
-11057,HIGD2A,0.08610340523227816
-11058,CLTB,0.02416232674566593
-11059,FAF2,0.03015472060947058
-11060,RNF44,-0.01084136459038185
-11065,TSPAN17,0.1211017537303782
-11070,HK3,0.34479492301608183
-11071,UIMC1,-0.021402235014553437
-11072,ZNF346,-0.03310133714745422
-11075,NSD1,0.05612067411481097
-11077,RAB24,0.21896725869029202
-11078,MXD3,0.19704387688999844
-11079,PRELID1,0.19893588815067298
-11080,LMAN2,-0.1356186637044752
-11081,RGS14,-0.00010161673683425503
-11085,GRK6,-0.017070951216506196
-11086,PRR7-AS1,-0.05944415659203566
-11087,PRR7,-0.13155416122297853
-11089,PDLIM7,0.20327815037725858
-11091,DOK3,0.33347074805877563
-11092,DDX41,0.10635567463648758
-11093,FAM193B,0.014722310214336782
-11095,TMED9,-0.16647269201506815
-11096,B4GALT7,-0.015990567277507466
-11107,FAM153CP,-0.09333714030476857
-11109,RMND5B,0.04086143421443778
-11110,NHP2,-0.12856434515630322
-11112,HNRNPAB,0.08286914468774217
-11113,PHYKPL,0.04642654367720394
-11117,CLK4,-0.015416598810287193
-11120,ZNF354A,0.04921270136933359
-11121,ZNF354B,-0.02278787901988919
-11134,RUFY1,0.08083241978826565
-11136,HNRNPH1,0.02211514095776117
-11140,CANX,0.2913573127728438
-11141,MAML1,0.0011805506122212809
-11142,LTC4S,-0.02200845989088334
-11143,MGAT4B,0.04095115318230873
-11144,SQSTM1,0.2693533028781302
-11145,MRNIP,-0.10104395430625833
-11147,TBC1D9B,0.10674263587791102
-11148,RNF130,0.5504993325310872
-11152,MAPK9,0.014435959731860295
-11156,CNOT6,0.05118348108642177
-11157,SCGB3A1,-0.09997052670755482
-11161,MGAT1,0.2889420098831457
-11162,HEIH,0.03282800023035768
-11163,LINC00847,-0.04266834345205731
-11165,ZFP62,-0.01125515510723453
-11173,TRIM7,0.15331965196667005
-11177,TRIM41,-0.04868157919517649
-11179,RACK1,-0.25424707393903084
-11180,AC008443.1,0.019923188262109637
-11181,TRIM52,-0.15127758789595575
-11182,TRIM52-AS1,-0.11469710538982857
-11191,AL365272.1,-0.0654813693949249
-11192,DUSP22,0.12321331885962675
-11193,IRF4,0.09090946209015646
-11195,EXOC2,-0.01675358789359656
-11215,GMDS,0.0001572142582250036
-11217,GMDS-DT,0.010060280455120724
-11224,WRNIP1,0.039889899941959954
-11225,SERPINB1,0.34462483755570067
-11226,SERPINB9P1,-0.0005386572634956997
-11227,SERPINB9,0.226851796082813
-11229,SERPINB6,0.07428435617521982
-11231,NQO2,0.08937608632916019
-11234,RIPK1,-0.0026137830867676835
-11236,BPHL,-0.07452808264611017
-11241,PSMG4,-0.05015158614994918
-11249,FAM50B,-0.09233210746995017
-11253,PRPF4B,-0.043634510808353864
-11256,ECI2,-0.07656876553948243
-11267,CDYL,-0.03822979064481488
-11270,RPP40,-0.04651823478810567
-11274,LYRM4,0.04894035692019667
-11275,FARS2,-0.05046636544311181
-11280,F13A1,0.11690477401143773
-11281,LY86-AS1,-0.01061651403300217
-11283,LY86,0.4019093580645466
-11291,RREB1,0.17146913376954456
-11294,SSR1,0.23331652266747352
-11297,RIOK1,-0.07011854279984857
-11300,SNRNP48,-0.027063483209351886
-11302,TXNDC5,0.0409181695394987
-11303,BLOC1S5,-0.08787958833759497
-11304,EEF1E1,-0.2176155099817312
-11307,SLC35B3,0.019369359001502377
-11318,GCNT2,0.19807446903132908
-11322,PAK1IP1,-0.06774570463583948
-11323,TMEM14C,0.155329479540436
-11325,TMEM14B,-0.01862010860149528
-11332,SMIM13,0.04127984780050288
-11335,NEDD9,-0.02493865772826464
-11339,TMEM170B,0.3301564782971275
-11341,ADTRP,-0.07558025185631542
-11346,HIVEP1,0.09407933925287597
-11349,PHACTR1,0.019163579553288564
-11351,TBC1D7,0.022501195816621217
-11353,GFOD1,-0.1747954244471815
-11355,SIRT5,-0.018732201178285043
-11357,NOL7,-0.12976268502275415
-11358,RANBP9,0.05409152643608273
-11360,MCUR1,0.04841784400261389
-11364,CD83,0.0828113152999169
-11375,JARID2,0.16752180307460512
-11377,DTNBP1,-0.0017776054385242844
-11380,MYLIP,-0.055555657701375244
-11381,GMPR,0.07620018313620487
-11383,ATXN1,0.11227734380048357
-11394,FAM8A1,0.05996478442838732
-11395,NUP153,0.03716693716919704
-11397,KIF13A,0.2542271660897792
-11399,TPMT,0.16348417991560513
-11400,KDM1B,0.3357327439258133
-11401,DEK,0.19703650393160352
-11402,RNF144B,0.282997377002949
-11408,MBOAT1,0.20734126486854276
-11413,E2F3,0.1899350420689022
-11415,CDKAL1,-0.05550607111957562
-11423,SOX4,0.18380393671355855
-11435,MRS2,-0.03894665547602275
-11439,TDP2,0.06857083158260589
-11440,ACOT13,-0.16849085407387
-11442,C6orf62,0.2243148262296918
-11445,GMNN,-0.03656053600241357
-11447,RIPOR2,0.19729393406116283
-11452,CARMIL1,0.010375909436478467
-11461,TRIM38,0.2794770018388663
-11473,HIST1H1C,-0.08092785144245707
-11474,HFE,0.14423246825761124
-11475,HIST1H4C,-0.3158420390113054
-11477,HIST1H2BC,-0.08497181883902365
-11478,HIST1H2AC,0.04827755148475003
-11479,HIST1H1E,-0.2569550783240646
-11480,HIST1H2BD,-0.05258485219392434
-11482,HIST1H2BE,0.06403558131276957
-11484,HIST1H3D,-0.2690337830694853
-11486,HIST1H2BF,-0.12616255590165196
-11487,HIST1H4E,-0.06794785110575903
-11488,HIST1H2BG,-0.08577552687376576
-11491,HIST1H1D,-0.30055799564734365
-11498,HIST1H4H,-0.011617585712890809
-11500,BTN3A2,-0.1749371125850679
-11501,BTN2A2,0.12853650224030566
-11502,BTN3A1,-0.10836088274463125
-11503,BTN3A3,0.0010036835901295735
-11504,BTN2A1,-0.025381618943381687
-11506,HCG11,0.002067537379040825
-11507,HMGN4,-0.17359354211622924
-11509,ABT1,-0.16739434568866093
-11511,ZNF322,-0.07323635180585257
-11519,HIST1H2AG,-0.11144300401075902
-11521,HIST1H2BK,-0.083675172848063
-11522,HIST1H2AH,-0.08896185002297805
-11527,ZNF184,-0.06154893794783762
-11536,HIST1H3H,0.03193609512932894
-11541,HIST1H2BN,-0.06584442354599132
-11542,HIST1H2AK,-0.10244498776511368
-11543,HIST1H2AL,-0.018456285088178596
-11544,HIST1H1B,-0.027832640805136737
-11552,ZSCAN16-AS1,-0.027224941969419623
-11553,AL121944.1,-0.030435189215415947
-11555,ZSCAN16,-0.08861182404597831
-11557,ZKSCAN8,0.05129489808358666
-11559,ZKSCAN4,-0.016504414127732743
-11561,ZSCAN26,-0.05758380844403336
-11578,TRIM27,0.11791560724000767
-11603,GABBR1,0.17391520790220688
-11607,HLA-F,-0.22730034493794057
-11608,HLA-F-AS1,-0.06415901059510223
-11611,HLA-G,-0.005161636364033287
-11613,HLA-A,-0.12644030550309887
-11615,ZNRD1,-0.16873068163210037
-11616,PPP1R11,-0.00019745808684776047
-11623,TRIM26,0.09020224565215716
-11625,HCG18,-0.09931231065499051
-11626,TRIM39,0.023116089475021105
-11627,RPP21,-0.10244439435155561
-11629,HLA-E,-0.16384184004563826
-11631,GNL1,-0.08563310068595284
-11632,PRR3,-0.003595888217099152
-11633,ABCF1,-0.11616060314825062
-11634,PPP1R10,0.008861502806101543
-11635,MRPS18B,0.029807968196769074
-11636,ATAT1,-0.055317945029788075
-11637,C6orf136,0.058786586359915106
-11638,DHX16,0.037674748127929855
-11639,PPP1R18,-0.006734619076842505
-11640,NRM,0.08559044217011667
-11641,MDC1,-0.09768889664669746
-11643,TUBB,-0.13158489512011418
-11645,FLOT1,-0.06596722501700701
-11647,IER3,0.1592694224250493
-11649,LINC00243,-0.06874426065930547
-11653,GTF2H4,-0.027536969766216862
-11654,VARS2,-0.05808811018442122
-11665,CCHCR1,-0.018540366861316063
-11669,AL662844.4,-0.11717251044962904
-11670,HCG27,0.08931719276701422
-11672,HLA-C,-0.4122193592548705
-11673,HLA-B,-0.06697473725911002
-11677,MICA,-0.010325431524538728
-11678,HCP5,-0.06490755604771822
-11681,MICB,0.09384355344238278
-11683,DDX39B,-0.012436180062345635
-11686,NFKBIL1,0.05615364793150125
-11687,LTA,-0.04580097078591991
-11688,TNF,0.011138854860911169
-11689,LTB,-0.23433321871698196
-11690,LST1,0.5409240723457159
-11691,NCR3,-0.3006064792255714
-11692,AIF1,0.5853156448251817
-11693,PRRC2A,0.07620529444811798
-11694,BAG6,0.23865459608159734
-11695,APOM,0.019649087530381464
-11696,C6orf47,-0.06441691917839973
-11698,GPANK1,-0.056848824563403996
-11699,CSNK2B,-0.0005491540270379827
-11700,LY6G5B,0.1067211655924904
-11702,ABHD16A,0.027764321713739524
-11708,DDAH2,0.06275593285125275
-11709,CLIC1,-0.18676007367745312
-11710,MSH5,-0.0124605469545582
-11714,VARS,0.02817272060844086
-11715,LSM2,-0.2320971788676682
-11716,HSPA1L,0.051756976137314635
-11717,HSPA1A,0.3353227505968411
-11718,HSPA1B,0.10037363870299539
-11719,SNHG32,-0.17258390697226236
-11720,NEU1,0.11078431595063026
-11723,EHMT2,-0.0934966585491835
-11724,C2,0.15896800170141034
-11728,NELFE,-0.009443547986180986
-11729,SKIV2L,0.01228948519311532
-11730,DXO,0.024640579023799392
-11731,STK19,0.022557764349302636
-11739,ATF6B,0.025687609367132058
-11740,FKBPL,-0.05718865340689243
-11745,AGPAT1,-0.027264302469529555
-11746,RNF5,-0.057213310301569384
-11747,AGER,0.041992108127618444
-11749,PBX2,0.1166543411505163
-11750,GPSM3,0.03020388860588708
-11755,HLA-DRA,0.43960153410735775
-11756,HLA-DRB5,0.35522884494380425
-11757,HLA-DRB1,0.3878502119264251
-11758,HLA-DQA1,0.1852044898899332
-11759,HLA-DQB1,0.43106821055994726
-11762,HLA-DQA2,0.09938931594474905
-11764,HLA-DOB,0.02066282880927653
-11765,TAP2,0.06622759995061245
-11766,PSMB8,-0.030856897537992593
-11767,PSMB8-AS1,-0.08798621242250616
-11768,PSMB9,0.022432464122749768
-11769,TAP1,0.16600572960582125
-11770,HLA-DMB,0.4105450898279144
-11771,HLA-DMA,0.39092187576677645
-11772,BRD2,0.02869929828882225
-11775,HLA-DOA,0.040228656903720725
-11776,HLA-DPA1,0.3922130511318096
-11777,HLA-DPB1,0.31755410703491915
-11780,RXRB,0.037328445509694295
-11781,SLC39A7,0.017407026303973172
-11782,HSD17B8,-0.057113235186413584
-11783,RING1,-0.030604240032516435
-11785,HCG25,-0.11852728306783658
-11786,VPS52,0.040267420856655446
-11787,RPS18,-0.40950690558885433
-11788,B3GALT4,-0.1502206565279803
-11789,WDR46,0.018402442044937392
-11790,PFDN6,-0.04252567034922947
-11791,RGL2,0.04649997522328876
-11792,TAPBP,0.001534045348179779
-11793,ZBTB22,0.08515570064680798
-11794,DAXX,-0.06094398336947779
-11797,PHF1,-0.08567586645444292
-11798,CUTA,-0.25124936175538587
-11799,SYNGAP1,-0.0924020224458478
-11802,BAK1,-0.023396448973160098
-11804,ITPR3,-0.07233094724500506
-11805,UQCC2,-0.05866303140914682
-11807,LEMD2,4.440148520017228e-05
-11813,HMGA1,0.08129644775694203
-11814,SMIM29,-0.07178941007285593
-11815,AL354740.1,-0.07954954299227598
-11816,NUDT3,0.18814420337079474
-11818,RPS10,-0.3603488262800206
-11821,ILRUN,0.017692311330843136
-11822,AL451165.2,-0.061554845242276585
-11823,SNRPC,-0.1195153663329829
-11824,UHRF1BP1,0.060483000472832664
-11825,TAF11,0.013121396872692304
-11826,ANKS1A,0.0958289162466081
-11831,ZNF76,-0.09379261372002959
-11832,DEF6,-0.12729606264899165
-11833,PPARD,-0.02029017329485081
-11835,RPL10A,-0.39999279054932707
-11839,FKBP5,0.20424586558962743
-11847,SRPK1,-0.060867695281061764
-11849,MAPK14,0.25916111668770303
-11850,MAPK13,-0.20780752899480628
-11852,BRPF3,-0.06796062961832197
-11855,ETV7,0.17464271783819754
-11858,KCTD20,0.14231206925964707
-11859,STK38,0.07721058161214746
-11860,SRSF3,-0.034665993850797536
-11862,CDKN1A,0.15070580364802058
-11864,RAB44,0.10545155554223078
-11865,CPNE5,-0.004763406957357991
-11868,PPIL1,-0.0718445142575103
-11869,C6orf89,0.07122477462648173
-11872,MTCH1,0.10500726230022837
-11873,FGD2,0.3795543545537403
-11874,PIM1,-0.2305110727332045
-11877,TBC1D22B,0.0544739355825061
-11879,RNF8,-0.0842703441036873
-11880,CMTR1,0.10757534562479602
-11881,CCDC167,-0.20779767777684635
-11886,ZFAND3,0.1466773484806254
-11887,BTBD9,-0.024528721591463312
-11889,GLO1,-0.06939846052013916
-11894,SAYSD1,-0.005985193326146888
-11911,OARD1,-0.0245461571095206
-11914,NFYA,-0.004069581127069538
-11920,TREM1,0.25841042024412436
-11924,FOXP4,-0.011656077486653663
-11926,TFEB,0.1874169790573949
-11932,TOMM6,-0.04977447418381408
-11933,USP49,0.11768560782794761
-11936,MED20,0.0623954790939905
-11937,BYSL,-0.06041364648408359
-11938,CCND3,-0.2006442300121072
-11940,TAF8,0.04465906725577354
-11947,MRPS10,-0.040500588091265256
-11948,TRERF1,0.008252894793613078
-11949,UBR2,0.12312457267647117
-11951,TBCC,-0.1793746544442286
-11952,BICRAL,-0.10784354027938164
-11953,RPL7L1,0.020199908390761693
-11954,C6orf226,-0.080138897237308
-11957,AL035587.2,-0.06393564250255024
-11958,CNPY3,0.36830506719260525
-11961,PEX6,-0.04116800509046296
-11962,PPP2R5D,-0.037275658644011445
-11963,MEA1,-0.1505748441350925
-11964,KLHDC3,-0.015585460581387304
-11965,RRP36,-0.08296710143941888
-11967,CUL7,0.012503745447659314
-11968,KLC4,0.05793300152621204
-11969,MRPL2,-0.027709267322542336
-11972,SRF,0.06896375959446287
-11973,CUL9,0.02099311945477612
-11975,DNPH1,-0.009421400192511545
-11979,ZNF318,0.06901945196521557
-11982,ABCC10,0.0003646692927031053
-11984,TJAP1,-0.005142648396769716
-11986,POLR1C,-0.05375416492775467
-11987,YIPF3,0.14441469442288898
-11989,XPO5,0.007613556586484379
-11990,POLH,-0.06485008488460954
-11992,GTPBP2,0.1439816146319614
-11993,MAD2L1BP,-0.13934291642898103
-11995,MRPS18A,-0.05806566951424983
-11997,VEGFA,0.19038158265827732
-12006,MRPL14,-0.005205230918064725
-12007,TMEM63B,-0.037562260101882135
-12010,SLC29A1,0.10456251689343035
-12011,HSP90AB1,-0.23430476637559525
-12012,SLC35B2,-0.1063950418863676
-12013,NFKBIE,0.1277587928996387
-12016,AARS2,-0.010955800104834278
-12018,CDC5L,0.07371754425607792
-12021,SUPT3H,-0.10425075916445478
-12022,RUNX2,-0.02905881043402387
-12023,AL096865.1,-0.09649248001709412
-12027,ENPP4,-0.16295410144900016
-12035,PLA2G7,0.21838439533638263
-12045,CD2AP,0.00868338322192783
-12053,MMUT,0.08092750914899755
-12054,CENPQ,-0.03368655177766245
-12080,MCM3,-0.1392645768887951
-12081,PAQR8,0.05449344538080491
-12082,EFHC1,0.04860014288652461
-12083,TRAM2,0.05301168682128765
-12085,TMEM14A,-0.10336328625916014
-12091,ICK,-0.04467833012642645
-12092,FBXO9,0.09545344306827747
-12095,ELOVL5,0.1282246147058144
-12097,GCLC,0.09603712934781912
-12121,DST,0.21644531805002665
-12124,KIAA1586,-0.01933687885316447
-12125,ZNF451,0.06267216605090126
-12127,BAG2,-0.08814969125056128
-12129,PRIM2,-0.10611836842938192
-12134,AL021368.2,-0.07069777211345367
-12146,PHF3,0.07765717714448168
-12160,LMBRD1,-0.04412596312218934
-12161,COL19A1,-0.0009497305200056002
-12167,FAM135A,0.11388727444450071
-12168,SDHAF4,0.05128664456858182
-12170,SMAP1,-0.07055000855952664
-12171,B3GAT2,-0.12433217180350946
-12175,OGFRL1,0.44487459634023
-12193,CGAS,0.1667668957533457
-12194,MTO1,0.07449790175415633
-12196,EEF1A1,-0.13468604573941695
-12198,SLC17A5,0.09054266105637956
-12205,COX7A2,-0.21340482096503902
-12206,TMEM30A,0.17652580578450178
-12211,SENP6,0.1046426528636844
-12222,PHIP,0.193895525870499
-12224,HMGN3,-0.13130467871458038
-12239,BCKDHB,-0.04002765282179185
-12243,TENT5A,0.33148346029421627
-12250,IBTK,0.16107349799711174
-12253,UBE3D,-0.09768763080213423
-12255,DOP1A,0.03078438920008741
-12256,PGM3,0.010831349210028046
-12264,CYB5R4,0.205565767781044
-12268,CEP162,0.039008028029141495
-12274,NT5E,-0.078156098880204
-12275,SNX14,0.030513357783434544
-12276,SYNCRIP,0.17268057841252013
-12277,SNHG5,0.018565318176196395
-12282,ZNF292,-0.033285011918599075
-12284,SMIM8,-0.15639044727740628
-12287,SLC35A1,0.1293618054672034
-12288,RARS2,-0.10331262697209047
-12289,ORC3,-0.039448148646386544
-12290,AKIRIN2,0.2424139043681663
-12296,RNGTT,-0.07894527095470326
-12298,PNRC1,-0.11001955788983918
-12301,PM20D2,-0.04296371493162517
-12304,UBE2J1,0.34577344017362555
-12305,RRAGD,0.14833142107058603
-12308,LYRM2,0.014680365125971873
-12309,MDN1,-0.06764237392511098
-12311,CASP8AP2,-0.09650806935212407
-12313,BACH2,-0.09327603122348106
-12317,MAP3K7,0.03953306658770415
-12330,MANEA-DT,-0.014721755994818838
-12331,MANEA,0.04354058073080794
-12335,UFL1,-0.06113085085069638
-12339,NDUFAF4,-0.14466636092894108
-12341,MMS22L,0.025720654353611566
-12347,FBXL4,0.05866914114378326
-12350,PNISR,-0.1286565510170302
-12352,USP45,-0.0016593713971922038
-12354,CCNC,-0.12396374742724302
-12360,ASCC3,0.07739977727757169
-12367,HACE1,-0.055873844811875344
-12374,PREP,-0.010663024634884263
-12379,PRDM1,-0.23525671399364217
-12380,ATG5,-0.08799555567355744
-12383,CRYBG1,0.18927391313866462
-12385,RTN4IP1,0.03551442802946081
-12386,QRSL1,0.043203219693010386
-12389,CD24,0.001552616401447988
-12390,MTRES1,-0.08257418076290672
-12392,PDSS2,-0.0016343782659793036
-12395,SCML4,-0.18957029566981654
-12396,SEC63,0.11200148028034013
-12399,OSTM1,0.036764684581119464
-12402,SNX3,0.2081279875952875
-12407,FOXO3,0.12010471576008692
-12412,SESN1,-0.04683716355513902
-12414,CEP57L1,0.04765648725603105
-12415,CD164,-0.014906093964857271
-12418,SMPD2,-0.05656203986947075
-12419,MICAL1,0.0660783838338012
-12420,ZBTB24,-0.12106689420028495
-12422,AK9,-0.019790281346589507
-12423,FIG4,0.14241487494718064
-12428,CDC40,-0.0008581369812742201
-12434,CDK19,0.21165468032991291
-12435,AMD1,0.14303512352721756
-12436,GTF3C6,-0.050275599785319716
-12437,RPF2,-0.08821041810158126
-12440,MFSD4B,0.003747647653473639
-12442,REV3L,0.08395532266862046
-12445,TRAF3IP2-AS1,-0.04507006415693108
-12447,TRAF3IP2,0.006663300676140891
-12449,FYN,-0.2287649042396327
-12452,TUBE1,-0.09933106929362796
-12468,MARCKS,0.4748309081872568
-12471,HDAC2,0.1055641924308629
-12480,NT5DC1,-0.04709973868739114
-12483,TSPYL4,-0.08694560049554403
-12484,DSE,0.2566638928913937
-12485,TSPYL1,-0.0013594555210165134
-12487,CALHM6,0.4336534525343163
-12492,RWDD1,-0.19078071642411573
-12494,ZUP1,0.07668811570785575
-12495,KPNA5,-0.05483883503673501
-12503,DCBLD1,0.007633514349912768
-12504,GOPC,-0.05611543852231789
-12505,NUS1,0.05548648331116138
-12507,CEP85L,0.03686972632321423
-12510,MCM9,-0.010854234210688448
-12511,ASF1A,-0.08863618455415352
-12514,MAN1A1,-0.09113933750364449
-12518,TBC1D32,-0.04584081747102371
-12520,HSF2,-0.10242285019377982
-12521,SERINC1,0.1373363552493411
-12527,SMPDL3A,0.1756527264258438
-12537,RNF217,0.18704797747010413
-12539,HDDC2,-0.21202859407984442
-12545,NCOA7,-0.11156067056330013
-12547,HINT3,0.11644864395035111
-12548,TRMT11,-0.12944123278590466
-12549,CENPW,0.1602983352972755
-12552,RNF146,0.1309539846771969
-12553,ECHDC1,0.16438285587861656
-12558,THEMIS,-0.20151626970711947
-12565,ARHGAP18,0.05371064507934863
-12567,L3MBTL3,0.010271932460842846
-12569,SAMD3,-0.4880562324908462
-12573,EPB41L2,0.11115947210955159
-12574,AKAP7,-0.021463098022222352
-12576,MED23,0.05108061337318742
-12587,STX7,0.314733230009896
-12595,VNN1,0.1785492300149834
-12596,VNN3,0.17656904232706824
-12597,VNN2,0.28407707586933145
-12599,SLC18B1,0.10985739638124466
-12600,RPS12,-0.4234553996684834
-12610,TBPL1,-0.15729905330912092
-12613,SGK1,0.28966492099374685
-12624,HBS1L,-0.07732743895298481
-12629,AHI1,-0.11459731521264878
-12640,BCLAF1,0.13596644713763312
-12644,MAP3K5,0.20477774850063574
-12646,AL121933.2,0.05763045494847853
-12647,PEX7,-0.02978744231502777
-12653,IFNGR1,0.3178221529288002
-12658,WAKMAR2,-0.0013973350753450908
-12659,TNFAIP3,0.1716545830914423
-12667,HEBP2,0.19214067161304516
-12669,AL590617.2,0.13581373639574335
-12671,CCDC28A,0.028317981325394734
-12673,REPS1,0.09035918270316869
-12674,ABRACL,0.006962819177918652
-12675,HECA,0.059441001652246125
-12679,CITED2,0.23787480844796816
-12692,VTA1,0.011406749751515554
-12698,HIVEP2,-0.09647206518955533
-12703,AIG1,-0.02581316867239546
-12705,ADAT2,-0.049658999681759665
-12706,PEX3,-0.09837462905337786
-12707,FUCA2,0.2312971392763744
-12708,PHACTR2,0.11781799193457741
-12710,LTV1,-0.007514199718944303
-12712,PLAGL1,0.1725763907591368
-12713,HYMAI,0.12339114904021307
-12714,SF3B5,-0.06942382828219076
-12715,STX11,0.4550476863909979
-12716,UTRN,0.06690367635563772
-12719,EPM2A,0.031040741068740246
-12722,AL356599.1,-0.04613289597386885
-12724,FBXO30,0.06525406353742327
-12725,SHPRH,-0.01906736362810956
-12727,RAB32,0.336939313743553
-12734,STXBP5,-0.018344292428998397
-12743,SASH1,0.19017107021895702
-12748,TAB2,0.14801341723442382
-12751,SUMO4,-0.06337180209376386
-12752,ZC3H12D,-0.023491470409928453
-12753,PPIL4,-0.06553510867868588
-12755,GINM1,0.1274797536820652
-12758,KATNA1,-0.05926633025373322
-12759,LATS1,0.028293050275495536
-12761,NUP43,0.008530212704371688
-12762,PCMT1,0.09646246233058732
-12786,ZBTB2,0.015064667460426482
-12787,RMND1,-0.07931568370330895
-12788,ARMT1,0.1182054373550856
-12789,CCDC170,0.20669144427626765
-12794,SYNE1,-0.28820077550534734
-12801,FBXO5,-0.06685525636255213
-12802,AL080276.2,0.01060973880776447
-12803,MTRF1L,0.02303761167228734
-12808,IPCEF1,-0.014998211188250555
-12812,SCAF8,-0.043803355013312444
-12813,TIAM2,-0.022072059022330507
-12815,TFB1M,-0.03539282642958573
-12823,ARID1B,0.07149011048964438
-12827,TMEM242,-0.09830044410013439
-12830,ZDHHC14,-0.07366520213839733
-12831,SNX9,0.03747486182405059
-12835,SYNJ2,-0.17928397095035187
-12838,SERAC1,0.03207157497870114
-12839,GTF2H5,-0.08769267014192501
-12840,TULP4,-0.13167126629942805
-12844,TMEM181,-0.056646049108006316
-12845,DYNLT1,0.33180523118051397
-12846,SYTL3,-0.28542678017482775
-12847,EZR,-0.07874008048397793
-12851,RSPH3,0.15543487748410983
-12852,AL035530.2,-0.031359395621327994
-12853,TAGAP,-0.07024354902391727
-12863,SOD2,0.43859754649079913
-12864,WTAP,0.03975936473095453
-12866,ACAT2,-0.06232817359940024
-12867,TCP1,-0.051505702541987754
-12868,MRPL18,-0.09663811176004312
-12872,IGF2R,0.01439890502964196
-12887,MAP3K4,-0.05382735882965386
-12888,AGPAT4,0.039142591561966984
-12900,CAHM,0.036244325589924804
-12901,QKI,0.4154516194538947
-12924,SFT2D1,0.002669184311372087
-12926,MPC1,-0.023237355869822316
-12933,RNASET2,0.3154937726874482
-12936,FGFR1OP,0.016204969528314763
-12937,CCR6,-0.0315531428400254
-12985,WDR27,0.00814987450248758
-12986,C6orf120,0.051774472317391816
-12987,PHF10,0.022142878289382786
-12990,TCTE3,0.014989674646892104
-12991,ERMARD,-0.08013264721407924
-13005,FAM120B,0.00024412687895734633
-13008,PSMB1,-0.11293757176799415
-13009,TBP,-0.009934978566937761
-13010,PDCD2,-0.11003415218921925
-13021,AC093627.4,0.055556603945071475
-13025,FAM20C,0.2322615888032983
-13034,PRKAR1B,0.02493623656424336
-13036,AC147651.4,-0.05671743579211542
-13037,DNAAF5,0.03488202485098887
-13038,SUN1,0.031924444241578115
-13040,AC073957.3,0.0373281656692187
-13041,ADAP1,0.08554667386125643
-13042,COX19,0.022916832863271334
-13044,C7orf50,0.016550341315539063
-13051,ZFAND2A,-0.018971431966819847
-13058,INTS1,0.060391823842752726
-13060,MAFK,0.08332639439414509
-13062,PSMG3,-0.0317697284027638
-13069,MAD1L1,-0.14060205977739632
-13071,MRM2,0.05250564297193384
-13072,NUDT1,-0.00999738064026218
-13073,SNX8,0.1644771041532406
-13074,EIF3B,0.09832587172870273
-13075,CHST12,-0.23056320927495658
-13077,LFNG,0.19071792474134822
-13078,BRAT1,0.07134435039353659
-13079,IQCE,0.10223838486971337
-13080,TTYH3,0.33209479823542193
-13082,GNA12,0.08012815678794863
-13083,CARD11,-0.17930395908551067
-13093,FOXK1,-0.011580399117005742
-13094,AC092428.1,0.03517530622981927
-13095,AP5Z1,0.10099048684338904
-13099,RBAK,0.04595714061469482
-13101,WIPI2,0.12173575186588752
-13103,TNRC18,0.27370138159415336
-13109,FBXL18,0.04324395564083048
-13111,ACTB,0.38205674383007293
-13114,RNF216,-0.059868111988492054
-13117,CCZ1,0.045186231768016595
-13119,PMS2,-0.0594055274248284
-13120,AIMP2,0.10071852292720263
-13121,EIF2AK1,-0.010159069944090722
-13124,USP42,-0.01897702008246388
-13125,CYTH3,-0.06302706596254802
-13128,RAC1,0.45854287515051273
-13129,DAGLB,0.13996083518757726
-13130,KDELR2,-0.13639944211103633
-13132,ZDHHC4,0.0656511022075625
-13134,C7orf26,-0.02865213033952365
-13136,ZNF316,0.0928786274943815
-13138,ZNF12,0.00357701963597003
-13140,CCZ1B,0.00473937350415067
-13142,C1GALT1,0.020105652220666445
-13147,MIOS,0.0776924013337717
-13149,RPA3,-0.1789940785626688
-13150,UMAD1,0.05973604827489943
-13153,GLCCI1,-0.22721267409249565
-13166,NDUFA4,-0.20905486678005686
-13168,PHF14,-0.013689772789620844
-13172,TMEM106B,-0.046062834012108984
-13178,ARL4A,0.09154496634235505
-13199,ANKMY2,-0.03507757917531693
-13201,BZW2,0.011985075739520058
-13203,TSPAN13,0.0008167856157863705
-13206,AHR,0.3694582116275263
-13207,AC073332.1,-0.03910147083894978
-13211,SNX13,0.17247276600364728
-13213,HDAC9,0.22164518972137381
-13222,TWISTNB,-0.1314144875651408
-13237,SP4,-0.12107499641456825
-13239,CDCA7L,-0.012610837299446051
-13246,TOMM7,-0.3439058501110881
-13249,FAM126A,0.10891301756800502
-13251,KLHL7,-0.025447699427020066
-13252,NUP42,-0.0819814790851332
-13256,MALSU1,0.00651779311070209
-13257,IGF2BP3,0.08392942536585951
-13258,TRA2A,-0.014096018988383108
-13259,CCDC126,0.1141750553231946
-13260,FAM221A,-0.0986167224648322
-13267,MPP6,-0.06936673597050731
-13270,OSBPL3,-0.06913830165627287
-13271,CYCS,-0.06623433293197378
-13272,C7orf31,-0.006002265911625004
-13283,NFE2L3,0.11634317200695846
-13284,HNRNPA2B1,0.12035220041343064
-13285,CBX3,-0.01487742122278506
-13287,SNX10,0.47774073152725155
-13294,SKAP2,0.4588597141580946
-13296,HOTAIRM1,0.22042043209364914
-13319,HIBADH,0.008944644576935041
-13322,TAX1BP1,0.23114119313077708
-13323,JAZF1,0.06423682149437063
-13326,CREB5,0.363138659063299
-13331,CPVL,0.5212343486809282
-13335,CHN2,0.16355557557632427
-13341,SCRN1,0.09421849576236582
-13343,FKBP14,-0.013692038618007416
-13344,PLEKHA8,-0.024029877165050763
-13347,MTURN,0.06448572857490546
-13351,ZNRF2,0.08482362583932018
-13352,LINC01176,0.021507848657618938
-13353,NOD1,0.07002693277771016
-13355,GGCT,-0.043415686221850824
-13356,GARS-DT,0.21048851003031788
-13359,GARS,0.07157663585961666
-13374,LSM5,-0.10042645468216331
-13375,AVL9,0.0948377613785641
-13379,KBTBD2,0.0691291236042368
-13381,NT5C3A,0.08882576255975812
-13382,RP9,-0.03120151929627315
-13383,BBS9,0.0038772142885927017
-13393,DPY19L1,0.0815263143520744
-13401,HERPUD2,0.007327860358877633
-13402,AC018647.2,-0.006761125603123456
-13405,SEPTIN7,-0.38173408972323714
-13411,EEPD1,0.034678585830476324
-13417,AOAH,0.23809054463548285
-13423,ELMO1,0.0852785833851046
-13425,GPR141,0.044497046749514886
-13427,NME8,0.0020187635115578167
-13429,STARD3NL,-0.19264911904750737
-13430,TRGC2,-0.35382316635441435
-13433,TRGC1,-0.3171340194063383
-13446,TRG-AS1,-0.3632820477838271
-13454,VPS41,0.13229707832846485
-13459,YAE1,-0.10772569948034831
-13462,RALA,-0.015999362286110364
-13466,CDK13,0.01415233478641313
-13467,MPLKIP,-0.04520005560353257
-13479,C7orf25,-0.022392749901879564
-13480,PSMA2,0.00959389282547803
-13481,MRPL32,-0.10280332124807672
-13487,STK17A,-0.48025602268786344
-13488,COA1,-0.019931012968653575
-13489,BLVRA,0.35720472351173194
-13490,MRPS24,-0.035646622601907506
-13491,URGCP,-0.01696908146969466
-13492,UBE2D4,0.02463899122678616
-13494,AC004951.1,0.11369354096799017
-13496,DBNL,0.1922798795133673
-13499,POLM,-0.02901925417380457
-13501,POLD2,-0.0838999005426601
-13504,YKT6,0.03675912142208982
-13506,NUDCD3,-0.08245147972249174
-13508,DDX56,-0.036743788838676406
-13509,TMED4,-0.18317589432738562
-13511,OGDH,0.16152404138528256
-13512,ZMIZ2,0.06658269053058326
-13514,PPIA,-0.15930174280975545
-13515,H2AFV,0.017323903650532654
-13517,PURB,0.12042668708283341
-13519,AC004847.1,0.024059196336367855
-13520,MYO1G,0.1635237254292081
-13521,SNHG15,-0.0617514575991848
-13522,CCM2,-0.03864558221355197
-13524,TBRG4,0.020341639926629736
-13543,TNS3,0.3280933108171088
-13549,HUS1,-0.0035828449894639985
-13552,UPP1,0.10453774242719835
-13566,IKZF1,-0.19361835467371585
-13568,FIGNL1,-0.03663828479588089
-13594,SEC61G,-0.09223517611023255
-13599,LANCL2,-0.0057965403154514325
-13600,VOPP1,-0.03782505618723591
-13606,NIPSNAP2,0.18990042673994564
-13607,MRPS17,-0.005804350676632239
-13609,CCT6A,0.041739717112138496
-13610,SUMF2,-0.05075985347045136
-13611,PHKG1,-0.0001923464055490477
-13612,CHCHD2,-0.18648919066675823
-13647,ZNF736,-0.03288382625296454
-13649,ZNF680,-0.06171385701199001
-13652,ZNF107,-0.11654270015674756
-13653,ZNF138,-0.10318626571446785
-13656,ZNF273,-0.02025748356354717
-13659,ZNF117,0.058249738720933524
-13660,ERV3-1,-0.002918440033053542
-13662,ZNF92,-0.06645037041078392
-13665,VKORC1L1,0.009824780175880394
-13666,GUSB,0.17411755898888084
-13667,ASL,0.07238522766868205
-13668,CRCP,-0.06749792920743759
-13671,LINC00174,-0.04112848616938316
-13672,AC008267.5,-0.05681930628990915
-13673,KCTD7,-0.05573911098728714
-13675,RABGEF1,0.03135341554410758
-13676,AC027644.3,-0.03258274084301084
-13678,TMEM248,0.05824688367092614
-13679,SBDS,-0.16730407681821874
-13680,TYW1,-0.12287070678798512
-13682,AC006480.2,-0.09747313157297728
-13692,AUTS2,-0.22314510948419639
-13696,POM121,0.057371341397404296
-13703,NSUN5,-0.10137362466045961
-13707,BAZ1B,0.03472736981198916
-13708,BCL7B,-0.0516704950025438
-13709,TBL2,0.06170408347292175
-13713,DNAJC30,-0.01726633979533826
-13714,BUD23,-0.062378818389358874
-13716,ABHD11,0.07162711429005872
-13724,LIMK1,0.10083731525191442
-13725,EIF4H,0.12875945155935067
-13726,LAT2,0.33553141321080665
-13727,RFC2,0.010379208816570312
-13729,CLIP2,0.10919407736576144
-13732,GTF2I,0.19886416046633953
-13734,NCF1,0.5327004006334234
-13735,GTF2IRD2,0.03733242642054482
-13736,CASTOR2,0.0148573012776657
-13737,RCC1L,-0.0705139621943372
-13738,GTF2IRD2B,-0.05811618648755576
-13744,TRIM73,-0.14730254380261987
-13745,POM121C,-0.06453417316319929
-13747,HIP1,0.15699939248797648
-13750,RHBDD2,-0.01839990844752051
-13751,POR,0.1315505337761809
-13752,TMEM120A,0.11010895276950292
-13753,STYXL1,0.18093259466539155
-13754,MDH2,-0.009357368586367846
-13756,HSPB1,-0.11047216307320828
-13757,YWHAG,0.2766447885381468
-13760,DTX2,0.13914190534948273
-13764,POMZP3,-0.07635218163447757
-13768,CCDC146,-0.09994539837578088
-13769,FGL2,0.6408791505041104
-13771,GSAP,0.1290372255358975
-13774,PTPN12,0.010744985067530547
-13775,APTR,-0.06781869615263801
-13776,RSBN1L,-0.1596158260403237
-13777,TMEM60,0.055175808035912315
-13778,PHTF2,0.003300825573823123
-13791,CD36,0.5343755959160029
-13793,SEMA3C,0.1493745489334839
-13795,HGF,0.2312998305174824
-13800,SEMA3E,-0.11978819525169808
-13812,DMTF1,0.02191270710346686
-13813,TMEM243,-0.13648105634657154
-13814,TP53TG1,-0.20404995175556678
-13815,CROT,-0.10419945200379917
-13817,ABCB1,-0.17516320057218437
-13819,SLC25A40,0.2037877699774616
-13820,DBF4,-0.13570257697511223
-13822,SRI,-0.10048823426489123
-13825,STEAP4,0.26218955372886527
-13838,GTPBP10,-0.07688880005840153
-13840,CDK14,0.06301461930542789
-13843,FZD1,0.1788272559716132
-13847,MTERF1,-0.0008208408218580085
-13849,AKAP9,0.07111690264506179
-13850,CYP51A1,-0.03562050557711421
-13853,KRIT1,-0.04927419415348741
-13855,ANKIB1,0.08108811628988165
-13856,GATAD1,0.20792244730110035
-13857,AC007566.1,-0.011397946985350123
-13859,PEX1,-0.09296440305975309
-13860,RBM48,-0.11916206266445561
-13861,FAM133B,0.02204711831976267
-13862,CDK6,-0.1129935861017938
-13866,SAMD9,0.0652148055741235
-13867,SAMD9L,0.2493869946134059
-13869,VPS50,-0.04129098115763396
-13875,BET1,-0.0403296237956616
-13882,CASD1,-0.07333942389445777
-13889,PON2,-0.008045297159997434
-13894,PDK4,0.15650497449676518
-13896,SLC25A13,0.10741465966721418
-13897,SEM1,-0.05391049775483415
-13901,SDHAF3,-0.1411683037118089
-13904,AC079781.5,0.002288204763999821
-13906,LMTK2,0.08987146022181361
-13908,TECPR1,0.07806135687573329
-13909,BRI3,0.5929423390012232
-13914,TRRAP,0.02743785636133895
-13916,SMURF1,0.06981955090393477
-13919,ARPC1A,0.08769230095803439
-13920,ARPC1B,0.3086372795216774
-13921,PDAP1,-0.1282790592106239
-13922,BUD31,-0.08225120989600748
-13923,PTCD1,0.029807706707495753
-13924,CPSF4,-0.0547693909537511
-13925,ATP5MF,-0.04287208209624747
-13926,ZNF789,-0.0695408068426332
-13927,ZNF394,-0.03099162644240946
-13928,ZKSCAN5,0.008738243813965072
-13929,FAM200A,-0.12902858858494465
-13930,ZNF655,0.05984765273617852
-13932,ZSCAN25,-0.0441937167031541
-13939,TRIM4,-0.022661218534172634
-13945,ZKSCAN1,0.07996386510366887
-13946,ZSCAN21,-0.02260175091572192
-13947,ZNF3,0.028565101615624575
-13948,COPS6,-0.1674081870551585
-13949,MCM7,-0.0818769470243342
-13950,AP4M1,0.04948378656432484
-13951,TAF6,-0.061503161010755175
-13953,CNPY4,-0.14333556947663975
-13956,LAMTOR4,0.21805762564846207
-13957,MAP11,-0.004226141002199539
-13960,STAG3,-0.011306846022116545
-13964,PILRB,-0.16821324265236162
-13965,PILRA,0.4919103834617942
-13967,ZCWPW1,0.03613880317513852
-13968,MEPCE,0.10715944338540778
-13970,PPP1R35,-0.1990147071911632
-13973,TSC22D4,-0.020432650414041364
-13978,SAP25,0.08159396220038069
-13979,LRCH4,0.14598431837259854
-13983,MOSPD3,-0.0946569817114081
-13986,GNB2,0.30466144149555496
-13987,GIGYF1,0.029386805159284168
-13988,POP7,-0.011143089038235495
-13992,SLC12A9,0.20815620731247472
-13994,TRIP6,0.001360815656005385
-13995,SRRT,-0.003079208758314381
-14003,TRIM56,0.17687374307960887
-14005,AP1S1,0.03909386171336043
-14009,PLOD3,0.04645250025387641
-14010,ZNHIT1,-0.02143176058367997
-14011,CLDN15,0.0526722052240324
-14012,FIS1,-0.08164793301539988
-14015,IFT22,-0.10590793029129655
-14019,CUX1,0.24293822591740677
-14021,AC005072.1,0.06529591150535306
-14023,SH2B2,0.12781444406532586
-14025,PRKRIP1,-0.09475863344725634
-14027,ORAI2,0.06438537998066217
-14028,ALKBH4,-0.0499956562690127
-14029,LRWD1,-0.007353733965700329
-14030,POLR2J,-0.06408326729733639
-14034,POLR2J3,0.060870425009104216
-14035,RASA4,-0.002952021711425245
-14037,UPK3BL1,0.0196561354134744
-14039,POLR2J2,0.009991237311547193
-14041,FAM185A,-0.013415790707059738
-14046,ARMC10,-0.06103258364154545
-14047,NAPEPLD,-0.02024132266840311
-14048,PMPCB,-0.00614956727986268
-14049,DNAJC2,0.01632741409619347
-14050,PSMC2,0.01955316813909229
-14054,ORC5,-0.02129991783976482
-14058,AC007384.1,-0.01632513136310346
-14059,LINC01004,-0.02767343252790393
-14060,KMT2E-AS1,-0.0710341531922858
-14061,KMT2E,-0.024959198499096193
-14063,SRPK2,-0.19492195540335513
-14065,PUS7,-0.04898906927161266
-14066,RINT1,-0.00914931382268785
-14069,ATXN7L1,-0.05033523742551421
-14071,SYPL1,0.022521638720633664
-14072,NAMPT,0.38806139430295516
-14078,CCDC71L,0.020285162975084342
-14080,PIK3CG,0.13923076427406145
-14081,PRKAR2B,0.08028384905492163
-14083,HBP1,0.08886878539621168
-14085,COG5,0.07031452198000675
-14087,DUS4L,-0.027106601768869906
-14089,BCAP29,0.13112143907041318
-14092,AC002467.1,0.02556802325730258
-14093,CBLL1,-0.061543131910136675
-14095,DLD,0.08998098980312624
-14100,PNPLA8,0.08696200050518646
-14101,THAP5,-0.0777712933740775
-14102,DNAJB9,-0.0945451712530431
-14110,IMMP2L,-0.0503261010247636
-14112,LRRN3,-0.07736249111713876
-14116,ZNF277,0.03741577017150385
-14118,IFRD1,0.1576847321167617
-14121,TMEM168,0.014014874707286152
-14122,BMT2,0.08487201655343803
-14126,SMIM30,-0.05312816828228915
-14132,MDFIC,-0.10761289876467431
-14138,TFEC,0.420304083668713
-14139,TES,-0.08926242503920472
-14149,CAPZA2,0.3939500064162285
-14151,ST7,-0.03426944461116379
-14165,LSM8,-0.07293313794426189
-14174,ING3,0.011112445050813272
-14175,CPED1,0.21894357324654076
-14178,FAM3C,-0.05209682607331782
-14194,NDUFA5,-0.15153445962313422
-14198,WASL,0.06811772938384414
-14199,AC006333.2,0.033606446485685136
-14210,POT1,0.0021641589926755496
-14220,ZNF800,-0.018586213275143648
-14223,GCC1,-0.044692751084875595
-14224,ARF5,0.03452085948559413
-14228,SND1,0.11601726620733523
-14229,SND1-IT1,0.11153026537565867
-14230,LRRC4,0.040377203517164166
-14233,RBM28,-0.06519586984645273
-14235,IMPDH1,0.2853518213217939
-14237,METTL2B,0.024294213904573658
-14238,AC090114.2,0.04389096918162272
-14243,CALU,0.05775247251888534
-14249,ATP6V1F,0.11741899706299838
-14251,IRF5,0.33174177892935025
-14252,TNPO3,0.06887141489109071
-14254,TSPAN33,0.06042561755158105
-14257,AHCYL2,0.043967298235094314
-14261,AC078846.1,-0.014526590555504234
-14262,NRF1,-0.023180308778967127
-14265,UBE2H,-0.019470461750245305
-14267,ZC3HC1,0.015391276358088344
-14268,KLHDC10,0.07846440695843258
-14270,TMEM209,-0.029990828781915592
-14277,CEP41,-0.01362219475239565
-14282,COPG2,0.027315486268521438
-14290,LINC00513,-0.12699672856809557
-14291,AC016831.1,-0.017459692229993735
-14292,AC016831.5,0.0020827109176350406
-14293,AC058791.1,-0.16967102386058483
-14294,LINC-PINT,-0.10593307812620817
-14295,MKLN1,-0.012103680059401442
-14305,CHCHD3,0.004937235520289676
-14307,EXOC4,0.011204999071187714
-14310,SLC35B4,-0.07135203218894119
-14313,AKR1B1,-0.10428154418504275
-14316,BPGM,-0.00825639654394721
-14321,CYREN,-0.0453821119208539
-14322,TMEM140,0.07450175748230417
-14324,AC083862.3,0.059929383767026424
-14325,WDR91,0.06011754933892031
-14328,CNOT4,-0.05744915547134006
-14329,NUP205,0.04411404177493006
-14330,STMP1,0.1502721266209643
-14335,MTPN,0.17113039584047954
-14345,CREB3L2,0.025638277114908535
-14349,TRIM24,0.0600548840167011
-14355,ZC3HAV1,-0.004929727179426924
-14357,UBN2,0.1137620644118442
-14358,FMC1,-0.030290653241511194
-14359,LUC7L2,0.01999734855048235
-14363,HIPK2,-0.046808218469810536
-14364,TBXAS1,0.4057193078876284
-14365,PARP12,0.17774342149683162
-14366,KDM7A,0.27298286056295945
-14367,KDM7A-DT,-0.0050453024608292174
-14368,SLC37A3,1.7326317287833265e-05
-14370,MKRN1,0.15874006310418826
-14373,ADCK2,0.05129410688732933
-14374,NDUFB2,-0.20812068235811432
-14376,BRAF,0.023790393942656893
-14377,MRPS33,-0.11200742756758769
-14384,AGK,-0.144123919471838
-14386,DENND11,-0.07880661936272804
-14390,SSBP1,-0.2598444456130206
-14397,CLEC5A,0.1464791667331892
-14473,TRBC1,-0.34335317890229017
-14482,TRBC2,-0.4495940349105629
-14484,EPHB6,-0.006222053489366056
-14497,GSTK1,-0.08495508114630985
-14500,CASP2,0.012382925908716189
-14504,AC093673.1,-0.03298396036249652
-14505,ZYX,0.28668734909740096
-14507,EPHA1-AS1,-0.04872053713314457
-14514,TCAF1,-0.09160727754051945
-14525,AC004889.1,-0.0569615086458969
-14533,TPK1,0.0849477392364641
-14543,CUL1,0.2555377648050709
-14544,EZH2,-0.00437175729736677
-14548,PDIA4,-0.06546746023324009
-14549,ZNF786,-0.10575814142817841
-14551,ZNF398,-0.012081046105539898
-14552,ZNF282,0.01709870745701297
-14553,ZNF212,0.004421068116417172
-14554,ZNF783,0.014445830068486026
-14557,ZNF777,-0.10531129529181038
-14558,ZNF746,0.03820015926029236
-14559,KRBA1,-0.052901508833551776
-14560,ZNF467,0.30335834339183476
-14562,ZNF862,-0.005040193663076114
-14565,ATP6V0E2,-0.15269707239619426
-14573,LRRC61,-0.055573751098293815
-14577,REPIN1,0.06612968907061113
-14578,ZNF775,-0.007939905833975943
-14580,AC073111.4,-0.10494467512741888
-14582,GIMAP8,0.3266484342210795
-14583,GIMAP7,-0.4362809120034142
-14584,GIMAP4,-0.03486247236426622
-14585,GIMAP6,-0.1000483566363222
-14586,GIMAP2,-0.09391203834764296
-14587,GIMAP1,-0.15408854322725124
-14588,GIMAP5,-0.25469082135185506
-14589,TMEM176B,0.6062786219200049
-14590,TMEM176A,0.49298264155668725
-14595,ABCB8,-0.06696521311322191
-14598,CDK5,-0.0563937375529358
-14599,SLC4A2,0.09839324550426784
-14601,FASTK,0.02462534931446743
-14602,TMUB1,-0.1722229701960042
-14603,AGAP3,0.20425075938504234
-14608,ABCF2,0.009449673152185449
-14609,CHPF2,0.058284779483643856
-14610,SMARCD3,0.2739239995891697
-14613,NUB1,0.05293501451344952
-14618,RHEB,-0.02370280367904367
-14619,PRKAG2,0.26479486840716876
-14621,PRKAG2-AS1,-0.05894425888190446
-14624,GALNT11,-0.10337409484808309
-14626,KMT2C,0.1585030891962354
-14627,LINC01003,-0.014673774320867193
-14629,ACTR3B,-0.042611846951298035
-14640,PAXIP1-AS2,0.025454847596540823
-14641,PAXIP1,0.018158999318099053
-14644,PAXIP1-AS1,-0.10861944954458404
-14651,AC144652.1,0.022323968555957606
-14652,INSIG1,0.0014629544317080834
-14663,AC009403.1,-0.08597615753858319
-14664,RBM33,0.01818647717553289
-14671,LMBR1,-0.012901369461386024
-14673,NOM1,0.01075673491378051
-14679,UBE3C,0.08677833986843328
-14681,DNAJB6,0.014600386204787022
-14685,PTPRN2,-0.1253649657589
-14695,ESYT2,-0.12232962188890387
-14696,WDR60,-0.07707410223686335
-14709,FBXO25,-0.014033216868964509
-14712,ERICH1,0.14739166290820835
-14724,CLN8,0.014727549364313086
-14726,AC100810.1,0.04208675497574862
-14732,KBTBD11,0.17836623204709823
-14756,AC016065.1,-0.04499711141239748
-14757,MCPH1,-0.06360578082670582
-14758,ANGPT2,-0.07705791337066427
-14762,AGPAT5,0.08576469302973438
-14811,PRAG1,-0.08986368949422041
-14820,MFHAS1,-0.00901823880054148
-14821,ERI1,-0.0250461061579325
-14822,PPP1R3B,0.06915741805330139
-14831,TNKS,-0.08077596147024398
-14835,MSRA,0.07658149321114542
-14848,PINX1,-0.058883426560885646
-14852,AF131215.6,-0.13628812045829197
-14859,MTMR9,0.07865131699029104
-14866,BLK,-0.004261140118608898
-14873,FDFT1,-0.14056249328361375
-14875,CTSB,0.5638287195418196
-14893,AC068587.4,-0.04207906131479726
-14895,LONRF1,0.12476946017684043
-14911,MSR1,0.12037026457706172
-14915,MICU3,-0.02589664106480993
-14917,ZDHHC2,0.0071099863329955725
-14918,CNOT7,-0.012907562995147412
-14919,VPS37A,0.05461609232490086
-14932,PCM1,-0.04085704760254485
-14933,ASAH1,0.46813410115461807
-14936,NAT1,0.01994627860501322
-14947,CSGALNACT1,0.054785705724431866
-14949,INTS10,-0.0076663793374784775
-14953,ATP6V1B2,0.46890604333902786
-14967,DOK2,0.038517535479958696
-14968,XPO7,0.04222541692686593
-14972,FAM160B2,-0.04071430445790874
-14973,NUDT18,-0.0065378165264758726
-14975,REEP4,0.028156548958161064
-14982,POLR3D,-0.01870639956553032
-14986,PPP3CC,-0.13428508636468467
-14987,AC087854.1,-0.04525630352093478
-14988,SORBS3,-0.039991711567672866
-14991,PDLIM2,-0.06431656932021035
-14992,C8orf58,0.04379833728688639
-14993,CCAR2,0.031149328909388724
-14995,BIN3,0.010349993646950897
-15004,RHOBTB2,0.014804249253153231
-15005,TNFRSF10B,0.19728780361753226
-15008,TNFRSF10C,0.10526896221018568
-15009,TNFRSF10D,0.09379842351446671
-15011,TNFRSF10A,0.03053484439196692
-15012,AC100861.1,0.015225782753031455
-15014,CHMP7,-0.03026848333299535
-15015,R3HCC1,0.006386283470475656
-15018,ENTPD4,0.11927616946969877
-15022,SLC25A37,0.22230814075653663
-15029,ADAM28,0.026111147285910886
-15041,DOCK5,0.24070998207837035
-15044,KCTD9,-0.013491806170431858
-15051,PPP2R2A,0.11610351072140766
-15052,BNIP3L,0.38144564161105604
-15056,DPYSL2,0.4496348353094356
-15061,TRIM35,-0.006352772380694282
-15062,PTK2B,0.27874519650494806
-15064,EPHX2,-0.06672179019398752
-15065,CLU,0.0720999759937889
-15068,CCDC25,-0.1411873395853276
-15074,ELP3,0.019536714948370493
-15076,PNOC,0.02817161926211222
-15077,ZNF395,0.032562999957788406
-15083,EXTL3,0.07368155715686178
-15085,INTS9,0.005887385493624601
-15087,HMBOX1,0.08834881696849982
-15090,KIF13B,0.04024885886390301
-15104,SARAF,-0.19435156999490125
-15105,AC044849.1,-0.040270323763823834
-15106,LEPROTL1,-0.14345933807242886
-15108,AC026979.2,-0.05972517276349259
-15109,DCTN6,0.0700400377772202
-15116,GTF2E2,-0.07100629041220008
-15119,GSR,0.11862764773460868
-15120,UBXN8,0.0391705693106835
-15121,PPP2CB,0.18658180559145357
-15124,WRN,-0.05085849351637301
-15130,NRG1,0.23845954488385399
-15135,AC090204.1,0.08563566208880108
-15138,TTI2,-0.05497235848853798
-15139,MAK16,-0.021643534697808064
-15165,ZNF703,0.24501788075255385
-15168,ERLIN2,0.04254809078119926
-15169,PLPBP,-0.0363282728083786
-15171,BRF2,-0.11656960732130014
-15172,RAB11FIP1,0.355306997188352
-15175,EIF4EBP1,0.2678837908246338
-15179,ASH2L,0.053073802419534376
-15183,LSM1,-0.07603971418851112
-15184,BAG4,0.1422611177558962
-15186,DDHD2,-0.05338596224266712
-15187,PLPP5,0.028211997533375416
-15188,NSD3,-0.0450738590877014
-15199,TACC1,0.14705552817777787
-15202,PLEKHA2,0.14881455287747647
-15203,AC108863.2,-0.028197857780893416
-15205,TM2D2,0.01941621814521725
-15206,ADAM9,0.23273751952496333
-15210,IDO1,0.07067472108003615
-15227,GOLGA7,0.025370106991360206
-15232,GPAT4,0.03289443254728756
-15238,KAT6A,-0.0376926450177663
-15241,AP3M2,-0.12972870425970057
-15244,IKBKB,0.15167223224630075
-15245,POLB,0.10564639787808146
-15247,VDAC3,-0.10929490909334222
-15248,SLC20A2,-0.05906776534236991
-15250,SMIM19,-0.15919032537858258
-15254,THAP1,-0.02359135128141385
-15256,RNF170,0.025513638527093072
-15257,HOOK3,0.29357513350040226
-15258,FNTA,-0.05200813173997912
-15260,HGSNAT,0.15482789200482996
-15269,SPIDR,0.11940187969753385
-15271,CEBPD,0.4730481440089084
-15272,PRKDC,-0.09012880019769175
-15274,MCM4,-0.022326710088419945
-15275,UBE2V2,-0.07861526660881323
-15299,PCMTD1,0.07190565030180229
-15301,AC064807.1,-0.048497161961780805
-15310,RB1CC1,0.17366728599434753
-15319,ATP6V1H,0.07268112490823424
-15322,TCEA1,-0.0676811289364914
-15324,LYPLA1,0.21930143020934556
-15325,MRPL15,0.0002446258412600052
-15334,TMEM68,-0.07442095887650187
-15335,TGS1,-0.06363333837630952
-15336,LYN,0.5111123375735166
-15338,RPS20,-0.21250244930138604
-15342,CHCHD7,-0.031769185065381476
-15350,IMPAD1,0.12104738154267801
-15367,UBXN2B,0.053392523419757365
-15369,SDCBP,0.5458952176419276
-15370,NSMAF,0.06175427155228683
-15379,RAB2A,0.0008551887505038311
-15383,CHD7,-0.0812517991523084
-15387,AC022182.2,-0.04178619732701987
-15391,ASPH,0.054424831585775074
-15406,YTHDF3,0.19710673339517307
-15426,ARMC1,-0.005440031871671812
-15427,MTFR1,0.027812171507230952
-15429,PDE7A,-0.09749587080937559
-15438,RRS1,-0.009484300067401344
-15439,ADHFE1,-0.018931735827987576
-15441,MYBL1,-0.48179434222886086
-15442,VCPIP1,0.14223580582571127
-15443,SGK3,0.1339579347612261
-15445,SNHG6,-0.2259009472186224
-15448,COPS5,-0.07954641159101788
-15449,CSPP1,-0.004656571442109276
-15450,ARFGEF1,0.05941499373329395
-15468,NCOA2,0.16141507510424374
-15472,TRAM1,0.09340087357361417
-15474,LACTB2,0.006265391290975773
-15489,TERF1,-0.10921402067379593
-15495,RPL7,0.022542482188135503
-15501,STAU2,-0.03503103086479262
-15503,UBE2W,0.18980831318943345
-15505,ELOC,0.012457157893260481
-15506,TMEM70,0.03365881168214859
-15507,LY96,0.3104066507976824
-15512,GDAP1,0.07491480755289383
-15531,PEX2,-0.11075148927348354
-15535,PKIA,-0.0809365167635285
-15538,ZC2HC1A,0.08973615138117916
-15549,MRPS28,-0.13323533285331918
-15552,TPD52,-0.07077941222485042
-15556,ZBTB10,0.017435769375845952
-15561,PAG1,0.047475674286944554
-15567,FABP5,-0.04378814887194666
-15574,IMPA1,-0.09628515798481986
-15576,ZFAND1,-0.08115835597064987
-15579,SNX16,0.05496059444168725
-15588,LRRCC1,-0.0384273037189938
-15592,RBIS,-0.09567708082703881
-15599,CA2,0.12263140190946123
-15610,WWP1,-0.02291755971451913
-15611,RMDN1,0.07522744407076012
-15612,CPNE3,-0.07214260176804271
-15624,AF117829.1,0.06273792821733026
-15625,RIPK2,0.0999578422727975
-15626,OSGIN2,0.09185981114889528
-15627,NBN,0.131908304481789
-15628,DECR1,0.012523354850781019
-15633,TMEM64,-0.05444550293371845
-15638,PIP4P2,0.21573260441007322
-15641,OTUD6B-AS1,-0.21218813351942295
-15642,OTUD6B,0.00814545257829341
-15657,TRIQK,0.20675220026431435
-15662,FAM92A,0.026486592311893268
-15665,RBM12B,-0.055833270027567494
-15670,PDP1,0.2431645380805785
-15677,VIRMA,0.003409144576845264
-15682,DPY19L4,-0.005710400908779035
-15684,INTS8,-0.02771899727581531
-15687,NDUFAF6,-0.025913454152980485
-15688,TP53INP1,-0.00012239121662552632
-15693,PLEKHF2,0.15491223925918288
-15701,UQCRB,-0.20864943701059002
-15703,MTERF3,0.018481342406220094
-15704,PTDSS1,0.10470427554307134
-15707,CPQ,0.0887029451073689
-15713,MTDH,0.25509857444716394
-15716,RPL30,-0.38754924128844914
-15719,RIDA,-0.07971427498165501
-15720,POP1,0.00936474075841994
-15721,NIPAL2,0.10184758472264979
-15722,STK3,0.23370397391593656
-15728,VPS13B-DT,0.011130969655228052
-15730,VPS13B,0.03152850839457429
-15734,COX6C,-0.3300557156688226
-15738,POLR2K,-0.14481643843237196
-15740,RNF19A,-0.08740901937344196
-15746,ANKRD46,-0.0880452085044099
-15749,PABPC1,0.580471917136222
-15751,YWHAZ,-0.014898048538747898
-15757,ZNF706,0.043795998398307986
-15766,NCALD,-0.3694654897276612
-15768,RRM2B,0.20820676000197255
-15769,UBR5-AS1,0.11494008819671071
-15770,UBR5,0.0750859325335054
-15774,KLF10,0.24415365044287501
-15777,AZIN1,0.12324913200438335
-15782,ATP6V1C1,0.15653897529352123
-15792,SLC25A32,-0.014953950407730298
-15793,DCAF13,-0.0954737252625998
-15811,OXR1,0.2058963413065251
-15820,EIF3E,-0.007006096349347619
-15822,EMC2,-0.06651113264301926
-15827,NUDCD1,0.04105063852370665
-15829,ENY2,0.18695666909701164
-15831,EBAG9,-0.08198630590722997
-15850,TRPS1,0.2605023182271036
-15855,EIF3H,0.1291093504955743
-15856,UTP23,-0.02509881749521224
-15857,RAD21,0.0077757139540157744
-15863,MED30,-0.09931392505057442
-15864,EXT1,0.14438596293469289
-15877,TAF2,0.016658346372611346
-15884,MRPL13,0.020743723696357713
-15886,SNTB1,0.16535517047013676
-15900,ZHX2,-0.0806703247185642
-15904,DERL1,-0.09224203734398652
-15905,TBC1D31,-0.1374497175768171
-15910,C8orf76,-0.024586374819897314
-15911,ZHX1,0.07574973918507635
-15912,ATAD2,-0.007460607320211881
-15913,WDYHV1,-0.04663037717659234
-15914,FBXO32,-0.11367743053724826
-15919,FAM91A1,0.21887616832155707
-15927,TMEM65,0.051207428741783835
-15928,TRMT12,0.035362382618958556
-15930,RNF139,0.05147933020683845
-15931,TATDN1,-0.07435024811554367
-15933,NDUFB9,-0.10148970399450173
-15934,MTSS1,0.1215410269543963
-15941,SQLE,-0.03595950552741126
-15942,WASHC5,0.01672914946486995
-15944,NSMCE2,-0.032740921208839986
-15946,TRIB1,0.2102773978837797
-15950,LINC00861,-0.41647572386722814
-15968,MYC,-0.06304902816880521
-15969,PVT1,-0.02073077676880838
-15971,LINC00824,-0.08067318261748124
-15978,FAM49B,0.014718177328808132
-15982,ASAP1,0.32601074230497157
-15983,ASAP1-IT2,0.12857676377480987
-15992,EFR3A,0.17480342956553638
-15998,TMEM71,-0.009281772050564564
-16000,PHF20L1,0.2115370380341481
-16004,SLA,0.10152236715134304
-16005,AF305872.2,-0.10800030690727154
-16007,NDRG1,0.02340891835397803
-16008,ST3GAL1,-0.1072513114477305
-16009,AC103706.1,0.006059776210871094
-16015,ZFAT,0.007101128241731751
-16038,TRAPPC9,-0.07774861315514986
-16042,CHRAC1,0.025110146674775482
-16043,AGO2,0.03535543149605497
-16045,PTK2,0.08535707870190515
-16047,DENND3,0.2592345483135793
-16048,AC040970.1,0.0902962255770358
-16049,SLC45A4,-0.009042107457041388
-16058,PTP4A3,0.03362529406251483
-16072,JRK,0.02357515487295564
-16076,THEM6,-0.09191901973363914
-16090,LY6E,0.19411562315529418
-16096,GLI4,-0.0787608297163088
-16097,MINCR,0.0021024051898156644
-16098,ZNF696,0.018022243027139947
-16100,TOP1MT,0.004160720458322825
-16106,ZC3H3,0.06886954374955247
-16107,AC067930.8,0.0853396426349724
-16108,AC067930.3,0.07153998019303666
-16109,GSDMD,0.0641985488923712
-16110,MROH6,0.11553681081862265
-16112,NAPRT,0.15512935728417285
-16114,EEF1D,-0.269885856779195
-16115,TIGD5,-0.07433890412644191
-16117,TSTA3,-0.08387551195820614
-16119,ZNF623,-0.0064148378075763604
-16120,ZNF707,-0.0775369076864922
-16129,SCRIB,0.011698228440835033
-16130,PUF60,-0.05660800779439883
-16133,NRBP2,-0.005675956805606265
-16137,PLEC,0.24281573808191503
-16138,PARP10,0.029477737176169354
-16139,GRINA,0.42863813728358996
-16141,OPLAH,0.10753092689117316
-16143,EXOSC4,0.0998929911649747
-16144,GPAA1,-0.06393757948907151
-16145,CYC1,-0.03807732365863437
-16146,SHARPIN,0.010169135329407155
-16147,MAF1,-0.03777768085088901
-16149,HGH1,-0.03993595886095165
-16150,MROH1,0.09287927195326985
-16151,BOP1,0.05022968511238855
-16153,HSF1,-0.0057466007822539675
-16154,DGAT1,0.1525476113491698
-16158,SLC52A2,-0.08016774596042524
-16159,FBXL6,0.0009166667206273223
-16160,ADCK5,0.05019255965178351
-16161,CPSF1,0.03925162262138767
-16163,SLC39A4,0.010996514042876153
-16164,VPS28,-0.06521226972722952
-16167,CYHR1,-0.011846269724896427
-16169,KIFC2,-0.0026401568148306186
-16171,PPP1R16A,0.03281878101016887
-16174,MFSD3,0.020619323494558153
-16177,LRRC14,-0.038405187986184794
-16179,C8orf82,0.04440355710592018
-16185,ZNF34,-0.0494766986423985
-16186,RPL8,-0.17422767260302557
-16188,ZNF7,-0.042224094305788544
-16189,COMMD5,0.04908373532551785
-16191,ZNF250,-0.01478520530406901
-16192,ZNF16,0.06285205253198688
-16195,C8orf33,-0.1033988775656388
-16196,WASHC1,0.0573022212690693
-16203,CBWD1,0.2310039159436834
-16205,DOCK8-AS1,0.1371486793644156
-16206,DOCK8,0.2640184772090517
-16219,SMARCA2,-0.034748489276527565
-16226,PUM3,-0.06225655023301824
-16228,RFX3,0.0020182841969002777
-16239,PLPP6,-0.03159462516675009
-16241,CDC37L1,-0.12790838535739252
-16242,AK3,-0.0819014191420425
-16244,RCL1,-0.05454299622440559
-16246,JAK2,0.4814347645083829
-16253,PLGRKT,-0.19367940645147122
-16254,CD274,-0.010960684766602771
-16255,AL162253.2,-0.023167542932122035
-16258,RIC1,0.1810394582963478
-16260,ERMP1,-0.13206254493595804
-16261,KIAA2026,-0.006991134923188925
-16263,RANBP6,-0.10169634670013
-16267,UHRF2,0.11971263422903548
-16273,KDM4C,0.06766464376803026
-16278,DMAC1,-0.09826214288958908
-16308,ZDHHC21,-0.020096407671421162
-16313,TTC39B,-0.0710513271255881
-16314,SNAPC3,0.09603151622511592
-16315,PSIP1,-0.20128536128079255
-16324,CNTLN,0.24230619633013958
-16329,RRAGA,0.004344801882255528
-16330,HAUS6,-0.08428914093304345
-16331,PLIN2,0.03974895553571597
-16332,DENND4C,0.01788224963226478
-16336,RPS6,-0.37351426933592863
-16343,MLLT3,-0.10364578234818571
-16344,FOCAD,-0.023787984998558193
-16346,HACD4,0.21850395011143373
-16357,KLHL9,0.017444725531084636
-16366,MTAP,0.01659749900081322
-16394,CAAP1,-0.020123992698479722
-16395,PLAA,0.1737607368518723
-16396,IFT74,-0.02232700218647212
-16402,MOB3B,0.12844214789662317
-16407,C9orf72,0.4122227419502586
-16415,ACO1,0.08887258105640497
-16416,DDX58,0.11339329806235637
-16417,TOPORS,0.09323964556719308
-16418,SMIM27,-0.1752118094218135
-16419,NDUFB6,-0.011482745852226596
-16425,APTX,0.036974665767506694
-16426,DNAJA1,0.20076784128803843
-16427,SMU1,0.029135933618138654
-16428,B4GALT1,0.01510605280992788
-16431,BAG1,0.13000183502301116
-16432,CHMP5,0.10302924421208495
-16433,NFX1,-0.007042922274874806
-16436,AQP3,-0.1667163605314475
-16437,NOL6,-0.025769496195006448
-16457,UBE2R2,0.29174442603188194
-16458,UBAP2,-0.014971044951748861
-16460,DCAF12,0.26013394039358423
-16461,UBAP1,0.06176065746240694
-16463,NUDT2,-0.0027146505497668357
-16466,FAM219A,-0.004461559275290206
-16472,RPP25L,-0.09714609309308718
-16473,DCTN3,-0.09666969180901337
-16475,SIGMAR1,0.020629009847125394
-16476,GALT,-0.04645566868023879
-16477,IL11RA,-0.015805893986901614
-16492,VCP,0.03470796209673025
-16494,PIGO,-0.016479921050741114
-16496,STOML2,-0.10374463172282537
-16497,FAM214B,0.2337674547677745
-16502,TESK1,0.00962918541384455
-16503,CD72,-0.03325087205258261
-16506,SIT1,-0.22559288657161272
-16507,RMRP,0.019314104179592062
-16508,CCDC107,-0.3117959231251323
-16511,TPM2,-0.04518491046225067
-16512,TLN1,0.10480917158077571
-16513,CREB3,-0.0561269651482597
-16514,GBA2,0.07398688996776685
-16515,RGP1,-0.055632548132990525
-16521,HINT2,-0.14438959682965113
-16522,TMEM8B,0.05355625100895447
-16529,RECK,0.02999953362547164
-16530,GLIPR2,0.28261662283216166
-16532,CLTA,0.17552431950190647
-16533,GNE,0.05973781387099065
-16534,RNF38,0.0908373455881667
-16537,PAX5,-0.0011864672393094122
-16542,EBLN3P,-0.07371272481237595
-16544,ZCCHC7,-0.020371950268269722
-16546,GRHPR,-0.03197522192272978
-16547,ZBTB5,-0.10028476421674196
-16548,POLR1E,-0.0951658192271379
-16551,TOMM5,-0.19294736820237232
-16553,TRMT10B,-0.02483556593352457
-16554,EXOSC3,0.00822135689364488
-16555,DCAF10,0.11181040956160501
-16556,SLC25A51,-0.07483551893447792
-16616,LINC01410,0.16769864375286636
-16617,AL512625.3,0.04960059944003158
-16629,CBWD5,0.12523753194871706
-16644,CBWD3,0.08326007778946164
-16654,LINC01506,0.1650143031592395
-16655,PIP5K1B,0.12603188166555473
-16656,FAM122A,0.022299308093889397
-16659,FXN,-0.05872874173579522
-16660,TJP2,0.12152354790336688
-16667,PTAR1,0.03555220389880905
-16673,SMC5,0.0059435498552428455
-16675,KLF9,-0.019466205211129475
-16679,CEMIP2,-0.05585379743251632
-16680,ABHD17B,0.036042592138128736
-16681,C9orf85,-0.034243770175841376
-16686,LINC01504,0.1159265678498077
-16687,ZFAND5,0.42305151192705004
-16690,ALDH1A1,0.312957669512858
-16691,ANXA1,0.16129703562789677
-16699,C9orf40,-0.10975259317989935
-16701,CARNMT1,-0.1366350714635006
-16703,NMRK1,-0.17856077408773727
-16704,OSTF1,-0.18905884260246048
-16706,PCSK5,-0.047304784238119735
-16707,RFK,0.10046353620883045
-16708,GCNT1,0.09878870827491151
-16709,PRUNE2,0.08571358200378429
-16714,VPS13A,-0.11368873561975416
-16717,GNAQ,0.4019054639563037
-16718,CEP78,-0.29794667532944397
-16722,TLE4,0.07777803271468176
-16728,TLE1,-0.10910601906426252
-16741,FRMD3,0.1543350217645114
-16743,IDNK,-0.19171248556065837
-16744,UBQLN1,-0.08219382297095022
-16746,GKAP1,-0.040125239259316776
-16749,KIF27,0.003378370490274408
-16750,C9orf64,0.052794768113914536
-16751,HNRNPK,0.19183360671815375
-16752,AL354733.3,-0.07987674287082884
-16753,RMI1,0.10905001393954541
-16762,AGTPBP1,0.17417630126317596
-16765,NAA35,0.03849357967191493
-16766,GOLM1,-0.015373381271131742
-16769,ISCA1,-0.05756505535725553
-16770,TUT7,0.17004701789532076
-16779,DAPK1,0.27858644253872494
-16780,DAPK1-IT1,0.1106576844739865
-16781,CTSL,0.1486512078664596
-16788,SPIN1,0.008733322080257278
-16793,S1PR3,0.28607846157326317
-16796,CKS2,-0.0056756753369955676
-16797,SECISBP2,0.07470288356804698
-16798,SEMA4D,0.0071893527651549
-16809,SYK,0.45345014635781516
-16818,AUH,-0.11985014831713493
-16819,NFIL3,0.08064728876417362
-16822,SPTLC1,0.08219817420990258
-16824,IARS,-0.032382752284690376
-16825,NOL8,0.022765686997347025
-16833,BICD2,0.16535114391852254
-16835,ZNF484,-0.09143808414089566
-16836,FGD3,-0.0520726151424728
-16837,SUSD3,-0.11422944733985242
-16839,CARD19,0.0976227251103223
-16840,NINJ1,0.23841412455373465
-16845,FAM120AOS,-0.02256203115343713
-16846,FAM120A,0.41531042673999946
-16847,PHF2,0.14871257922196254
-16853,AL158152.1,0.04811755172201807
-16855,ZNF169,0.04523433884308847
-16858,MFSD14B,0.22090964481217795
-16862,FBP1,0.20984672679278452
-16864,AOPEP,-0.015267483165135563
-16872,PTCH1,-0.2632660310146329
-16878,LINC00476,-0.18363326252348044
-16880,ERCC6L2,0.01311042854063016
-16888,SLC35D2,0.0508384094146397
-16892,HABP4,-0.058810925127508545
-16894,PRXL2C,0.03039963194553589
-16896,ZNF510,-0.08670233519162106
-16897,ZNF782,0.009246864908597342
-16898,MFSD14C,0.10981601800024969
-16906,TDRD7,0.142260605336273
-16909,TSTD2,0.02186920112598491
-16910,NCBP1,-0.016142599609168144
-16912,XPA,-0.13487074056852574
-16916,TRMO,0.01593491884316468
-16917,AL499604.1,-0.057697210493837726
-16919,ANP32B,-0.04286385044007101
-16921,NANS,0.18488444957375316
-16922,TRIM14,0.054073917828060765
-16924,TBC1D2,0.21167133193440318
-16932,TGFBR1,-0.11394186061352608
-16934,ALG2,0.06483418568611451
-16935,SEC61B,-0.09122981878795491
-16942,STX17,0.03342289368898685
-16944,ERP44,0.14782931241250516
-16945,INVS,-0.04954284649905045
-16946,TEX10,0.023098166125344346
-16947,MSANTD3,-0.13250026321640937
-16952,MRPL50,-0.02728518664214158
-16953,ZNF189,-0.007840086846918191
-16957,RNF20,0.0361986916076985
-16966,SMC2,-0.01616936236332549
-16976,NIPSNAP3A,0.22480693370986976
-16978,ABCA1,0.12736316667813158
-16982,SLC44A1,0.19896412013448447
-16983,FSD1L,0.0382301288568355
-16985,FKTN,0.0075369331143332954
-16987,TMEM38B,-0.021439565708087464
-16996,RAD23B,0.1680718258619556
-16999,KLF4,0.41803199473962294
-17004,ELP1,-0.021855310614702077
-17005,ABITRAM,0.0149771866544706
-17007,TMEM245,0.0038898638531032798
-17011,PALM2-AKAP2,-0.12031022448229813
-17014,TXN,0.07026800832004533
-17019,LPAR1,0.18924640862437628
-17022,ECPAS,0.11217089275204661
-17026,DNAJC25,-0.004767884957281456
-17027,GNG10,0.2885673389249256
-17029,UGCG,-0.06699347478864658
-17030,AL138756.1,0.03649927353266406
-17031,SUSD1,0.14173505000319556
-17032,PTBP3,0.2386150184399167
-17033,HSDL2,0.01975670649522166
-17035,KIAA1958,0.1360447984600948
-17036,INIP,-0.017313341922795798
-17038,SNX30,0.2440991465546767
-17039,SLC46A2,0.18981932859130737
-17045,SLC31A2,0.46077018602934144
-17046,FKBP15,0.3673622642629094
-17047,SLC31A1,0.29265845718738814
-17048,CDC26,-0.187609474779088
-17049,PRPF4,0.020209913491461194
-17053,HDHD3,-0.05930962968711863
-17054,ALAD,-0.05027148815465617
-17055,POLE3,-0.03507548805607259
-17057,RGS3,-0.06322099911594879
-17061,ZNF618,0.0661063875487482
-17067,AKNA,-0.07120080633778955
-17070,ATP6V1G1,0.0632655283479321
-17071,TMEM268,0.15500213740305638
-17076,TNFSF8,0.005804833039789274
-17094,TLR4,0.4363536197732099
-17095,AL160272.1,0.12335879512286768
-17103,CDK5RAP2,-0.010168005516368536
-17104,MEGF9,0.4237204842606992
-17105,FBXW2,-0.013381850164708753
-17107,PSMD5,0.010009117539642601
-17108,PHF19,-0.023560254152269942
-17109,TRAF1,-0.10676707807155354
-17111,CNTRL,0.11079313435160905
-17112,RAB14,0.1491152760577986
-17113,GSN,0.272751204098456
-17116,STOM,-0.1329712877275919
-17123,NDUFA8,-0.0669437318694824
-17126,RBM18,-0.0800156856109139
-17127,MRRF,0.004255480485594553
-17128,PTGS1,0.07673341177914784
-17145,PDCL,-0.027551780511597578
-17147,RC3H2,0.010101680890106693
-17148,ZBTB6,0.005341507918057116
-17149,ZBTB26,0.038958007270143445
-17152,RABGAP1,0.15418632980743044
-17155,STRBP,-0.06802434654777878
-17157,DENND1A,0.17542852955666693
-17164,NEK6,0.12165729230025779
-17167,PSMB7,0.010246210017396487
-17176,RPL35,-0.42430452662372914
-17177,ARPC5L,-0.4614954470686954
-17178,GOLGA1,0.05447057025401439
-17179,SCAI,-0.03341842846163689
-17180,PPP6C,0.08924778900070039
-17181,RABEPK,0.0015786075045640675
-17182,HSPA5,-0.0026543470884354764
-17184,GAPVD1,0.13013379090671506
-17186,MAPKAP1,0.02484288187971097
-17189,PBX3,0.09042188633664154
-17192,MVB12B,-0.037905756256142335
-17199,ZBTB43,0.020098284803826326
-17200,ZBTB34,0.1333482866527246
-17201,RALGPS1,-0.010573610734245705
-17206,SLC2A8,0.07907346567114239
-17208,RPL12,-0.25253463545464994
-17209,LRSAM1,0.06324761095981357
-17210,NIBAN2,0.20089961920254254
-17213,PTRH1,-0.1891132710024512
-17215,TTC16,-0.25223360800841244
-17216,TOR2A,0.06813422715987175
-17217,SH2D3C,0.09354247736113898
-17219,CDK9,0.022209406398668474
-17220,FPGS,-0.057218156037516496
-17221,ENG,0.015710052461669413
-17224,ST6GALNAC6,-0.1160392102969456
-17225,ST6GALNAC4,-0.06999404863683151
-17228,DPM2,-0.05433136937868011
-17229,FAM102A,-0.17068344127071627
-17231,NAIF1,0.007523029830542751
-17232,SLC25A25,0.0018074757813384232
-17235,PTGES2,0.012583777832665736
-17238,C9orf16,-0.16620956792478705
-17239,CIZ1,0.019668928851991178
-17241,GOLGA2,-0.032792089801065086
-17242,SWI5,-0.030478719328281556
-17243,TRUB2,-0.03075864711868829
-17245,COQ4,-0.043956306655086035
-17246,SLC27A4,0.06934819828989487
-17247,URM1,-0.015646922767620138
-17252,ODF2,-0.030069520897109958
-17254,GLE1,0.08147433081503692
-17256,SPTAN1,-0.1536650372942373
-17261,SET,-0.0886147267999705
-17263,ZDHHC12,0.020959789377359286
-17264,AL441992.1,-0.014988377117970561
-17265,ZER1,-0.06581672181895937
-17266,TBC1D13,0.025061116109697766
-17267,ENDOG,0.025929916804332754
-17268,SPOUT1,-0.05778121264863809
-17270,LRRC8A,-0.06576669038869293
-17272,DOLK,-0.06783950597512671
-17273,NUP188,-0.08600622770337309
-17275,SH3GLB2,-0.1431116624541421
-17276,MIGA2,-0.0015822294292692329
-17278,DOLPP1,0.027757136503004563
-17279,CRAT,0.06998882338183131
-17281,PTPA,0.11924718062674537
-17283,IER5L,0.1460330857704713
-17291,LINC01503,0.2093558700535392
-17294,LINC00963,0.1873502904508959
-17298,NTMT1,-0.04837404168337894
-17300,ASB6,-0.06998771032861047
-17305,TOR1B,0.09885499933847051
-17306,TOR1A,0.03152869682678888
-17307,C9orf78,-0.08935611183400687
-17308,USP20,-0.1299724288007405
-17309,FNBP1,-0.02458964372572022
-17312,GPR107,0.07996713489251031
-17317,FUBP3,-0.043344249602362425
-17319,EXOSC2,-0.02300505342874723
-17320,ABL1,0.10709298995272831
-17326,NUP214,0.44934322043963976
-17328,AL157938.3,-0.04451578249373185
-17329,FAM78A,-0.018771502636346735
-17332,PRRC2B,-0.059560114080283305
-17334,POMT1,-0.017183883988665664
-17336,UCK1,-0.03764293694697955
-17338,RAPGEF1,0.04260546200886125
-17340,MED27,-0.010502879928343369
-17341,NTNG2,0.0732372261464684
-17342,SETX,0.17209584962197397
-17343,TTF1,-0.0016515826312056127
-17346,DDX31,-0.05035030687963848
-17347,GTF3C4,0.016608201370136123
-17351,TSC1,-0.010753250974619575
-17353,GTF3C5,0.0381141052979645
-17355,RALGDS,-0.11494204675651419
-17356,GBGT1,0.17441314393721366
-17359,SURF6,-0.051026093115419016
-17360,MED22,0.013793020705897815
-17361,RPL7A,-0.42641037484390787
-17362,SURF1,0.012672200931069301
-17363,SURF2,-0.13323087743271295
-17364,SURF4,0.09735648797977052
-17366,REXO4,-0.10518253100914689
-17369,SLC2A6,0.32446490054015475
-17377,VAV2,0.13431794837913835
-17378,BRD3OS,-0.03388965227783579
-17379,BRD3,0.16266767991973463
-17382,WDR5,-0.0038193914439589066
-17385,RXRA,0.4535930064561945
-17393,FCN1,0.6623251550744285
-17403,MRPS2,-0.053140071607945115
-17414,CAMSAP1,0.06267241445376809
-17416,UBAC1,0.1443406557689419
-17417,NACC2,0.16325327256006508
-17419,TMEM250,-0.02600658320961421
-17422,QSOX2,0.011837909716217013
-17428,CARD9,0.21145468091456213
-17429,SNAPC4,-0.02575980482172485
-17430,ENTR1,-0.022921036640764832
-17431,PMPCA,-0.0064177941572479075
-17432,INPP5E,0.026474259791812223
-17433,SEC16A,0.07943219561620703
-17435,NOTCH1,0.08010988260474733
-17441,AGPAT2,0.1900083581331242
-17443,SNHG7,-0.053321912981856186
-17448,TMEM141,-0.04403599058531108
-17451,RABL6,0.1473669765051469
-17453,PHPT1,0.010445500568069359
-17455,EDF1,-0.182204429150686
-17456,TRAF2,-0.10845930353272905
-17458,FBXW5,-0.03267706069251738
-17464,PAXX,-0.3636491115526496
-17465,CLIC3,-0.46495419593195675
-17466,ABCA2,-0.20708973916392212
-17467,C9orf139,0.18619537285280308
-17468,FUT7,0.0018966897695336389
-17469,NPDC1,-0.1229610054766528
-17475,UAP1L1,0.1628161987200344
-17476,MAN1B1-DT,0.032149026203244196
-17477,MAN1B1,0.0187090193406489
-17478,DPP7,-0.04908958204458912
-17482,ANAPC2,-0.08353428991116893
-17483,SSNA1,-0.17216597489425084
-17484,TPRN,0.054624385477444325
-17485,TMEM203,-0.06959782081658114
-17486,NDOR1,0.08798229408614168
-17487,BX255925.3,0.04797137121421901
-17492,TUBB4B,0.03190414908803812
-17496,NELFB,0.040927442576703645
-17497,TOR4A,0.18535915842614215
-17501,EXD3,-0.007825933910929917
-17504,NSMF,0.02022501232228107
-17506,MRPL41,-0.06301030776154401
-17507,DPH7,-0.10132361841073532
-17508,ZMYND19,0.05352399025031016
-17509,ARRDC1,0.17371330736344412
-17510,ARRDC1-AS1,-0.02265126461801077
-17511,EHMT1,-0.07483973360620694
-17517,ZMYND11,-0.1331557134043959
-17523,LARP4B,0.22684135386982565
-17526,GTPBP4,0.030836925230149737
-17529,IDI1,-0.144246519596361
-17530,WDR37,-0.04146446816784678
-17550,PFKP,-0.025014736996475037
-17551,PITRM1,-0.0008910640732139745
-17562,KLF6,0.41510776161555424
-17580,AKR1C3,-0.3119510496931781
-17598,ASB13,0.1259938387180018
-17599,TASOR2,0.02963532016547818
-17601,GDI2,0.40594383163638526
-17603,ANKRD16,-0.07188675560219739
-17604,FBH1,0.053032478706055174
-17606,IL15RA,0.1945041542869609
-17609,RBM17,0.06128543748202891
-17610,PFKFB3,0.16905213156491422
-17614,PRKCQ,-0.17706953426915456
-17615,PRKCQ-AS1,-0.286689481023805
-17622,SFMBT2,-0.1036938373187183
-17627,KIN,-0.005474053149997461
-17628,ATP5F1C,0.03814583728438443
-17629,TAF3,-0.011958450728279208
-17630,GATA3,-0.24578285050878979
-17647,CELF2,0.07337387597304555
-17655,USP6NL,0.14127589558210338
-17658,ECHDC3,0.1193851927345667
-17661,UPF2,-0.006851720109430953
-17662,DHTKD1,0.11679205080714881
-17663,SEC61A2,-0.10771287136245178
-17664,NUDT5,-0.12042121098395876
-17665,CDC123,-0.07518577311988882
-17667,CAMK1D,-0.04075669841904801
-17671,OPTN,-0.29803518611190144
-17674,PHYH,0.018514583978413133
-17675,SEPHS1,-0.009703460636304357
-17678,BEND7,-0.00794017247303368
-17680,PRPF18,-0.0010697495920261465
-17681,AL157392.3,0.015289138305637056
-17689,FAM107B,-0.30381419319462916
-17692,HSPA14,0.03823839450138374
-17693,HSPA14,-0.008792636777554716
-17696,DCLRE1C,0.14447249051900987
-17701,RPP38,-0.08055415242847853
-17702,NMT2,-0.15672989025330997
-17706,MINDY3,-0.06864921862236932
-17708,PTER,0.03608884960635379
-17711,RSU1,0.02541818682801264
-17716,TRDMT1,-0.009731682560438491
-17717,VIM-AS1,0.051344889735500825
-17718,VIM,0.441139503792532
-17724,STAM,-0.009501052327159643
-17735,NSUN6,-0.0717693249381971
-17736,ARL5B,0.05532066794897357
-17742,PLXDC2,0.48356234941578824
-17751,MLLT10,0.11583491806178778
-17753,DNAJC1,-0.22235212630798826
-17759,COMMD3,0.020522705618199938
-17760,BMI1,-0.05677578692935239
-17768,PIP4K2A,-0.2132315966249254
-17771,MSRB2,0.18526736413939043
-17775,OTUD1,0.1426686420449799
-17780,ARHGAP21,0.10470099956324087
-17793,APBB1IP,0.004735864568469695
-17795,PDSS1,0.08592269844509427
-17797,ABI1,0.027370211891719155
-17798,ANKRD26,0.005778675967888142
-17799,YME1L1,0.1048900897790491
-17800,MASTL,0.09754383646223705
-17801,ACBD5,0.042918740718071155
-17804,RAB18,0.02148849378402855
-17810,MPP7,0.15936615529008763
-17814,WAC-AS1,0.04296971510201533
-17815,WAC,0.2571914689562798
-17822,SVIL,0.07133611604846567
-17825,MTPAP,0.005804243042680137
-17827,MAP3K8,0.08057148636520227
-17834,ZNF438,0.09758398912503435
-17839,ZEB1,-0.10201401748513858
-17843,ARHGAP12,-0.131904706770046
-17844,KIF5B,0.21077033585230967
-17846,EPC1,-0.10066180040868875
-17851,CCDC7,-0.11040203019657013
-17853,ITGB1,-0.04212412248776227
-17867,CUL2,-0.06085864363059017
-17869,CREM,-0.1679865377259609
-17872,CCNY,0.24769785415400372
-17873,AL117336.2,-0.03832366757854332
-17888,ZNF248,-0.06669690262411529
-17891,ZNF25,0.14089731583297802
-17893,ZNF33A,-0.016197230293094216
-17894,ZNF37A,-0.039420756673049534
-17899,ZNF33B,-0.0773662156993662
-17904,BMS1,-0.04023500301944034
-17909,CSGALNACT2,0.128068776585733
-17910,RASGEF1A,-0.22361568000913185
-17914,HNRNPF,-0.04049612577497503
-17916,ZNF487,-0.01228147015098897
-17922,ZNF32,-0.20298352485940818
-17943,RASSF4,0.3258261122077328
-17946,ZNF22,-0.11134809240409911
-17949,ALOX5,0.44984820228012085
-17951,MARCH8,-0.01560988223990755
-17952,ZFAND4,-0.0050044075444715475
-17953,WASHC2C,0.19364275591273186
-17954,AGAP4,0.014570594747317029
-17955,TIMM23,0.00535457109134166
-17956,NCOA4,0.3501338288333509
-17976,AGAP9,0.009633005662362169
-17985,MAPK8,0.10723692111031974
-17986,ARHGAP22,0.09256029288678874
-17990,WDFY4,0.15393132656838365
-18000,TMEM273,-0.16196381795408593
-18005,ERCC6,-0.04535614182275999
-18010,PARG,0.011197419563643488
-18011,TIMM23B,0.04857971027410341
-18013,AL442003.1,-0.06511508860724605
-18015,WASHC2A,0.11692748652632108
-18017,SGMS1,0.1920891779820649
-18020,SGMS1-AS1,0.009219293251806312
-18030,CSTF2T,0.02130663310857633
-18047,IPMK,0.16537323431743273
-18048,CISD1,-0.09395100687804897
-18051,UBE2D1,0.48068476699338136
-18052,TFAM,-0.040382987017451695
-18062,CCDC6,0.05888483638392855
-18064,ANK3,-0.07198706473741581
-18077,ARID5B,-0.18865912946787375
-18078,RTKN2,-0.06714374516672347
-18084,ADO,0.1556785606060878
-18087,NRBF2,0.19529517627566623
-18088,JMJD1C,0.23108392367163497
-18090,REEP3,0.22165158651182335
-18104,SIRT1,-0.0025828426625690154
-18105,HERC4,0.04821538324786836
-18110,HNRNPH3,0.010137958008189916
-18111,RUFY2,-0.014408442100450747
-18113,SLC25A16,0.006164317076077996
-18116,CCAR1,0.08366495247394905
-18119,DDX50,-0.05254064575415995
-18120,DDX21,0.1991741957108964
-18121,KIF1BP,0.06612614717666722
-18122,SRGN,0.3321637744853449
-18123,VPS26A,-0.025186509889724814
-18124,SUPV3L1,-0.10183990114876369
-18128,HK1,0.2602926885610502
-18140,TYSND1,-0.017503151879502438
-18141,SAR1A,-0.09090309501885459
-18142,PPA1,0.046378260334525544
-18145,EIF4EBP2,0.19668521552582421
-18148,PRF1,-0.6303805567480577
-18151,SGPL1,0.09810589303310394
-18152,PCBD1,-0.002104819897090438
-18158,SLC29A3,0.13300856212571835
-18159,CDH23,0.22190873218122406
-18162,VSIR,0.4937203692200924
-18163,PSAP,0.6918177435832032
-18167,SPOCK2,-0.31907804226686
-18168,ASCC1,-0.027771059738999033
-18169,ANAPC16,-0.2772315530101726
-18170,DDIT4,-0.36914138483742776
-18172,DNAJB12,0.15012493656230166
-18173,MICU1,0.2220302172387926
-18176,MCU,0.05113540097892335
-18180,P4HA1,0.13615881882242187
-18185,ECD,0.05341943836447738
-18186,FAM149B1,0.0636715742220393
-18187,DNAJC9,-0.17150530128797706
-18189,MRPS16,-0.04149518427167257
-18192,ANXA7,0.13944664535410692
-18195,PPP3CB,0.062217206483549524
-18196,PPP3CB-AS1,0.06180024337474125
-18205,SEC24C,0.07661927755319518
-18206,FUT11,-0.14076037267790772
-18207,CHCHD1,-0.04899743647108823
-18208,ZSWIM8,0.051572260213969774
-18210,NDST2,0.04151033269807608
-18211,CAMK2G,-0.10792460447074911
-18216,VCL,0.11026387239728366
-18218,AP3M1,0.022344378715502655
-18219,ADK,0.0659852275597187
-18222,KAT6B,-0.1402907379091531
-18228,SAMD8,0.1696967055627371
-18229,VDAC2,-0.06613500432408419
-18230,COMTD1,-0.07887003722546913
-18233,ZNF503,0.15912095052903186
-18238,LRMDA,0.24756598184928824
-18240,AL589863.1,0.1716688654415001
-18253,POLR3A,0.0049740271901556445
-18254,RPS24,-0.2562759188284217
-18265,ZMIZ1,0.2061938426977334
-18266,PPIF,0.19296829972437027
-18267,ZCCHC24,0.1426819682177154
-18275,NUTM2B-AS1,-0.004651975151571467
-18278,AL135925.1,0.0028756392649428955
-18284,TMEM254,0.009926164249396434
-18286,ANXA11,-0.02448634994028409
-18292,PRXL2A,-0.027055126377883373
-18293,TSPAN14,0.05808320032218789
-18306,GHITM,0.09116246213889521
-18314,CCSER2,-0.2094546912232221
-18325,WAPL,0.10612758095605485
-18338,GLUD1,0.1662978768343788
-18339,SHLD2,-0.03325879804377034
-18341,NUTM2A-AS1,-0.011641294748843227
-18342,AL157893.2,0.11659195010674439
-18344,LINC00863,0.030917944320759966
-18348,MINPP1,0.04445052606803459
-18352,PAPSS2,0.11004643316010364
-18353,ATAD1,-0.08682693218373426
-18355,PTEN,0.2697662439531439
-18363,ANKRD22,0.30479748489905
-18364,STAMBPL1,-0.08240049640482404
-18366,ACTA2,0.06767875175812892
-18368,FAS,0.13906932603210517
-18371,LIPA,0.29691104898738013
-18372,IFIT2,0.2919537609063751
-18374,IFIT3,0.3368219300391523
-18376,IFIT1,0.26865301371694067
-18377,IFIT5,0.033647146012278126
-18385,KIF20B,-0.03582644232697067
-18395,RPP30,-0.10014737261804635
-18401,PCGF5,0.03710997179949014
-18405,TNKS2,0.1479696123481216
-18408,BTAF1,0.12826969313520192
-18409,CPEB3,0.14836925573096993
-18410,MARCH5,-0.005222874392288071
-18412,IDE,0.10866643489075455
-18414,HHEX,0.3933737977071099
-18415,EXOC6,0.11206357244373022
-18420,MYOF,0.42173157307111436
-18425,FRA10AC1,0.041879978285644846
-18429,SLC35G1,-0.06243599776660673
-18430,PLCE1,-0.06372395662075625
-18436,NOC3L,-0.06145354712384491
-18437,TBC1D12,0.07002199513391665
-18438,HELLS,-0.17523761929286938
-18447,PDLIM1,0.0391109698003592
-18449,ALDH18A1,-0.1049371210655948
-18450,TCTN3,-0.018768784601144214
-18451,ENTPD1,0.3821247260086059
-18452,ENTPD1-AS1,0.010144577004227065
-18455,CCNJ,0.11651135886569113
-18457,ZNF518A,0.11590531103327743
-18458,BLNK,0.014523555138250235
-18463,TM9SF3,0.06288146309057618
-18464,PIK3AP1,0.39854149833589153
-18465,LCOR,0.06256149846104511
-18469,ARHGAP19,0.08134437845102795
-18470,FRAT1,0.17756836280267463
-18471,FRAT2,0.2801437263488721
-18473,RRP12,0.16062810430440735
-18475,PGAM1,0.1495521143937348
-18476,EXOSC1,-0.048973880817327904
-18477,ZDHHC16,-0.01247457318452201
-18478,MMS19,0.05180158598446748
-18479,UBTD1,0.10241561152457976
-18484,PI4K2A,-0.05473746871587504
-18486,MARVELD1,0.17809934127478427
-18487,ZFYVE27,-0.032763535965375366
-18492,R3HCC1L,-0.0225935085793385
-18495,PYROXD2,-0.047288852666927186
-18496,HPS1,0.05118768731755814
-18500,GOT1,0.029720015236656225
-18505,SLC25A28,0.11180755878991425
-18508,ENTPD7,0.056168060072201334
-18509,CUTC,-0.07484618769993981
-18510,COX15,0.13576907226936005
-18512,DNMBP,0.09998911385157541
-18515,ERLIN1,0.08959052930845339
-18516,CHUK,0.12791085837826555
-18519,CWF19L1,0.05159289405345375
-18520,BLOC1S2,0.08521437436681563
-18526,SEC31B,0.030368952620376883
-18527,NDUFB8,-0.21896537944299
-18528,HIF1AN,0.02925709848873631
-18531,SLF2,-0.01999555176985012
-18533,MRPL43,-0.1315458937737174
-18536,TWNK,-0.05998350500014628
-18537,LZTS2,-0.011386423348763407
-18539,SFXN3,0.00328346874239551
-18550,BTRC,-0.048038441149106464
-18551,DPCD,0.03592210302130425
-18552,POLL,0.044546463654392826
-18553,FBXW4,-0.0303407091070271
-18557,NPM3,-0.09004024055674364
-18558,OGA,0.05103362747227972
-18561,ARMH3,0.07445840979239822
-18562,HPS6,-0.017840814765435323
-18563,LDB1,-0.03312536893604388
-18564,PPRC1,-0.02758925737285019
-18565,NOLC1,-0.05374197297280441
-18568,GBF1,0.033795015321117304
-18569,NFKB2,0.11336617291060999
-18571,FBXL15,-0.10411917492782194
-18572,CUEDC2,-0.05923252533784757
-18573,RPARP-AS1,-0.15085139156287927
-18574,C10orf95,-0.08421420289948686
-18575,MFSD13A,-0.012616112260765791
-18576,ACTR1A,0.10976948405778202
-18578,SUFU,0.04493502590286473
-18579,TRIM8,0.2406700493146933
-18580,AL391121.1,0.061920738492469866
-18581,ARL3,0.023101306578890715
-18583,WBP1L,0.054807601608595415
-18586,BORCS7,-0.013794449930898374
-18590,CNNM2,-0.052832835769748926
-18591,NT5C2,0.19608679436379
-18594,PCGF6,-0.040754314396366086
-18595,TAF5,-0.015740105067766606
-18596,ATP5MD,-0.11544713741697457
-18597,PDCD11,-0.00429674076859938
-18598,CALHM2,0.10458863921405098
-18604,NEURL1,0.06238065198607893
-18611,STN1,-0.07804299833925973
-18612,SLK,0.065031968185481
-18614,SFR1,0.029628834607309737
-18616,GSTO1,0.1949013065182691
-18618,ITPRIP,0.14049058857834978
-18636,XPNPEP1,0.07832587549001162
-18638,ADD3,-0.17136240224415403
-18639,MXI1,0.08885493142239931
-18641,SMNDC1,0.03496795630099661
-18643,DUSP5,0.10230429602555073
-18644,SMC3,-0.12581988155751808
-18647,PDCD4-AS1,-0.021510429981656153
-18648,PDCD4,-0.19824250129641693
-18649,BBIP1,-0.05358411405131711
-18652,SHOC2,0.0721898043036948
-18656,GPAM,-0.0033548153397215657
-18658,ACSL5,-0.007644543744852196
-18660,ZDHHC6,0.06533376519572479
-18661,VTI1A,0.09601841331162286
-18666,TCF7L2,0.3235471074008205
-18671,CASP7,0.10274593768038327
-18675,NHLRC2,0.03238817871230418
-18679,CCDC186,-0.08711494061770478
-18684,ABLIM1,-0.17926499164443746
-18686,FAM160B1,0.06555002854678038
-18687,TRUB1,-0.017228027197545237
-18701,SHTN1,0.34809318578413684
-18709,PDZD8,0.020681959734492397
-18717,RAB11FIP2,0.0051000276582719144
-18721,FAM204A,-0.017757874315977955
-18724,CACUL1,0.1666667552617453
-18728,EIF3A,0.24581817113503024
-18729,DENND10,0.2743278790552892
-18730,SFXN4,0.0009052181967413782
-18731,PRDX3,0.20820475653808468
-18732,GRK5,0.029349835947104008
-18735,RGS10,0.0515992803555402
-18736,TIAL1,-0.036101395372614135
-18737,BAG3,-0.029366304211636562
-18738,INPP5F,0.06324097727231752
-18739,MCMBP,0.2708572906656682
-18740,SEC23IP,0.11218150328349473
-18746,WDR11,0.1937901443051379
-18754,ATE1,0.03350315678352363
-18756,NSMCE4A,-0.047298302331269984
-18762,PLEKHA1,-0.2413358207658806
-18770,FAM24B,-0.10277829675290938
-18772,C10orf88,0.01671071278501519
-18773,PSTK,0.04150622148372544
-18774,IKZF5,0.018102467992657842
-18775,ACADSB,-0.013143594335543301
-18778,BUB3,-0.15089677611570196
-18788,CHST15,0.3908173121758276
-18789,OAT,0.10005191094814314
-18792,LHPP,0.03678945759717482
-18793,FAM53B,-0.06678803055963245
-18796,EEF1AKMT2,-0.048449876549804274
-18797,ABRAXAS2,0.039207337124889684
-18799,ZRANB1,0.06060632905340739
-18801,CTBP2,0.1809529404255962
-18809,EDRF1,-0.014113877017413437
-18812,UROS,0.013210038645847483
-18813,BCCIP,0.06123063628006903
-18814,DHX32,0.10762432071862264
-18829,PTPRE,0.41108564982292223
-18841,MGMT,-0.1907235525178869
-18852,GLRX3,-0.023711275385969142
-18860,PPP2R2D,0.008166542334603994
-18861,BNIP3,-0.15587741470456573
-18867,STK32C,0.12391488874310408
-18868,LRRC27,0.139017419132747
-18869,PWWP2B,0.09290135103352538
-18875,INPP5A,0.06177765131991048
-18884,UTF1,0.1386711370106563
-18885,VENTX,0.19311241999161233
-18888,ADAM8,-0.01456399859359469
-18889,TUBGCP2,0.14553834175363123
-18891,ZNF511,-0.0012008746164594325
-18895,FUOM,0.211196817822023
-18896,ECHS1,0.025830909484593854
-18898,PAOX,0.03973611195289591
-18899,MTG1,-0.07899269120264053
-18910,LINC01001,0.14804041907088428
-18912,BET1L,-0.07586722289663174
-18916,RIC8A,0.005429934796453689
-18917,SIRT3,-0.017654487905257515
-18918,PSMD13,-0.024538659441892634
-18920,AC136475.3,-0.0680152608312674
-18921,PGGHG,-0.043239334469585146
-18923,IFITM2,-0.23742325784772905
-18925,IFITM1,-0.3907499165205584
-18927,IFITM3,0.36138997314940363
-18935,SIGIRR,-0.27140473191148934
-18937,PTDSS2,0.028510575488980887
-18939,RNH1,0.1996769740467373
-18942,HRAS,-0.09793911767674232
-18946,RASSF7,-0.12361981403756715
-18948,PHRF1,-0.038506778214087906
-18949,IRF7,0.18740147189889123
-18953,DEAF1,0.022625131110136427
-18956,TMEM80,-0.07092494440437083
-18958,TALDO1,0.5038668106294175
-18959,GATD1,-0.014614700484698229
-18960,AP006621.3,-0.0022391764697869336
-18963,SLC25A22,-0.09217548476524887
-18965,PIDD1,0.00764017240883055
-18966,RPLP2,-0.3230827448971952
-18967,PNPLA2,0.12499525305968633
-18970,CD151,0.07381552344241997
-18971,POLR2L,-0.10463537529093703
-18972,TSPAN4,0.2254425203466002
-18974,CHID1,0.006907804682562459
-18975,AP2A2,-0.00746196835593728
-18983,TOLLIP,0.15401292355123042
-18989,MOB2,-0.1576646118337146
-19003,CTSD,0.39846955614576074
-19009,TNNI2,0.1727476845659177
-19010,LSP1,0.16215378331993094
-19015,MRPL23,0.07232868328942182
-19025,ASCL2,-0.11199813382328867
-19026,C11orf21,0.0689958580161797
-19027,TSPAN32,-0.09283319048225618
-19029,CD81,-0.4735847317785035
-19030,TSSC4,-0.031895002913410596
-19033,KCNQ1,0.1831059822971918
-19034,KCNQ1OT1,0.02314591463629525
-19037,CDKN1C,0.13540056413484605
-19039,SLC22A18,0.102149131687314
-19041,NAP1L4,-0.23417211222373033
-19043,CARS,0.05534658679779262
-19046,OSBPL5,-0.0108896214833014
-19053,ZNF195,-0.025198872493402438
-19061,NUP98,0.10915987489319111
-19062,PGAP2,-0.1084989858096363
-19063,RHOG,0.3191549448804016
-19065,STIM1,-0.029527430468067235
-19067,RRM1,-0.08774089660486535
-19076,TRIM21,0.10944480880164048
-19128,TRIM34,0.12306847344619529
-19129,TRIM5,0.09204491046746109
-19130,TRIM22,0.23485033379465398
-19151,FAM160A2,0.1330101107312938
-19157,SMPD1,0.06150104396777696
-19158,APBB1,-0.08620815722890156
-19161,ARFIP2,-0.09162504844200589
-19162,TIMM10B,-0.07186931589306382
-19163,DNHD1,-0.018219068212561335
-19164,RRP8,-0.030953521567409135
-19166,ILK,0.08725539496840566
-19167,TAF10,0.001253795200467858
-19169,TPP1,0.4177603649041034
-19174,MRPL17,-0.04837702259362486
-19192,PPFIBP2,0.10408381844020764
-19205,EIF3F,-0.10387027134581321
-19210,RIC3,-0.0884071433262797
-19214,TRIM66,-0.011862213565263888
-19216,RPL27A,-0.4191699965531535
-19220,AKIP1,0.015611031326309182
-19223,TMEM9B,0.04160970602266767
-19224,TMEM9B-AS1,0.006363031185635776
-19229,DENND5A,0.31629020923382134
-19231,TMEM41B,-0.0606396072446105
-19232,IPO7,0.12106573600865858
-19234,AC132192.2,0.04573410586058191
-19235,ZNF143,-0.016782573944628298
-19237,SWAP70,0.12432584344836703
-19240,SBF2,0.2563541439167
-19246,AMPD3,0.13049326806636144
-19247,MTRNR2L8,0.11453112369079015
-19248,RNF141,0.23524232254589036
-19251,MRVI1,0.14142222772599863
-19252,CTR9,0.056513837452686566
-19253,EIF4G2,0.2552654937416828
-19255,ZBED5,0.022715898011284073
-19256,ZBED5-AS1,-0.006605500662088019
-19265,USP47,0.013904753859520353
-19270,MICAL2,0.22942688041210224
-19284,ARNTL,-0.06627123863650206
-19285,BTBD10,-0.013774962666160599
-19288,FAR1,0.27443102953386866
-19296,RRAS2,-0.2219334290148006
-19297,COPB1,-0.023742975284947846
-19298,PSMA1,0.01587352892838994
-19299,PDE3B,-0.08888367926758675
-19300,CYP2R1,-0.04196535098852471
-19312,C11orf58,-0.00033711028724264455
-19314,RPS13,-0.20201370935733956
-19315,PIK3C2A,0.06165808627625503
-19316,NUCB2,-0.23327652286194214
-19328,KCNC1,-0.06767203705223826
-19329,SERGEF,-0.04943475069364794
-19331,SAAL1,0.03605587374650138
-19339,HPS5,0.10204700054460464
-19340,GTF2H1,0.08467980131675827
-19341,LDHA,-0.15371620938084282
-19345,TSG101,-0.02862630058354597
-19347,UEVLD,0.02387639670266285
-19349,SPTY2D1,-0.01357638718913906
-19358,ZDHHC13,-0.003230170837080514
-19372,HTATIP2,0.014675878724024772
-19373,PRMT3,0.04272284683024075
-19389,FANCF,-0.009820126368774659
-19391,SVIP,-0.08905958387328737
-19412,LIN7C,0.09032516884196
-19419,METTL15,-0.021451172252794574
-19433,ARL14EP,-0.14258172388529802
-19440,DNAJC24,-0.08324048785545259
-19441,IMMP1L,-0.22335542204647657
-19442,ELP4,-0.023492095431620896
-19449,RCN1,0.004250110078035782
-19454,EIF3M,0.1625123421002368
-19456,PRRG4,0.263996222403453
-19457,QSER1,0.16079090046284014
-19459,TCP11L1,0.015361824774063584
-19460,LINC00294,0.1290133445892948
-19461,CSTF3,-0.026037263617189957
-19463,HIPK3,0.30316908102883544
-19467,CD59,-0.05425924248583709
-19468,FBXO3,-0.05295970525500742
-19474,LMO2,0.4989865891914998
-19477,CAPRIN1,0.135860662546562
-19478,NAT10,0.021558955086264157
-19480,CAT,0.3217553402875283
-19485,APIP,0.03476228049049834
-19486,PDHX,-0.0006864488525333186
-19488,CD44,0.4441434647336419
-19490,AL133330.1,0.09610680754139461
-19498,TRIM44,0.1353259737187674
-19502,LDLRAD3,0.10583493582058487
-19504,COMMD9,0.1452971254406108
-19505,PRR5L,-0.24198348457355867
-19509,TRAF6,0.06365521662633239
-19512,C11orf74,0.09357449880792061
-19526,API5,0.04793897630221417
-19528,TTC17,0.05071509260101876
-19534,HSD17B12,0.02497195408062195
-19537,ALKBH3,-0.046859373144318
-19543,ACCS,0.06808232839891874
-19545,EXT2,-0.029229823725529517
-19548,CD82,0.03495922788747541
-19553,TSPAN18,-0.02836124232456483
-19554,TP53I11,0.023964803905711686
-19574,SLC35C1,0.02026635821401355
-19576,CRY2,0.028100639474035273
-19580,PEX16,-0.038610404763586216
-19582,PHF21A,0.19373290723857386
-19589,DGKZ,-0.06086487805595503
-19592,AMBRA1,-0.01314457630096873
-19595,ATG13,0.05552308004638984
-19596,ARHGAP1,0.01305768665450671
-19597,ZNF408,0.02732772440029107
-19599,CKAP5,-0.09477054109104215
-19603,C11orf49,-0.060046737681408266
-19605,ARFGAP2,0.005454908631164057
-19608,DDB2,-0.004946507972569303
-19610,ACP2,0.1391369700249091
-19611,NR1H3,0.06324175749443887
-19612,MADD,0.006929653297533514
-19615,SPI1,0.5540274766715332
-19616,AC090559.1,0.4678953088672918
-19617,SLC39A13,-0.10193922289752724
-19618,PSMC3,-0.01893037705873554
-19620,CELF1,0.2087740943949832
-19621,AC090559.2,0.061663631367116604
-19622,NDUFS3,-0.09750122568994272
-19623,PTPMT1,-0.035579123270754735
-19624,KBTBD4,-0.0703336040611831
-19628,MTCH2,0.06365779551122361
-19630,FNBP4,-0.02191415626318299
-19631,NUP160,0.005825070789321831
-19632,PTPRJ,0.1795390670223972
-19706,SSRP1,-0.014298930028497513
-19710,SLC43A3,0.25446117851181244
-19714,TIMM10,-0.0876211822710909
-19716,UBE2L6,0.3316697002476264
-19717,SERPING1,0.28952596028745303
-19720,CLP1,0.053659106022051865
-19721,ZDHHC5,0.05069167780036573
-19722,MED19,0.008799487135132948
-19723,TMX2,-0.006734204340582269
-19724,SELENOH,-0.024343687065284017
-19726,CTNND1,0.22618398665673128
-19743,LPXN,-0.04357044872395328
-19744,ZFP91,0.08518424842870324
-19754,FAM111A-DT,-0.044393830246932095
-19755,FAM111A,0.1135596137362594
-19757,MPEG1,0.6144417103619213
-19769,OSBP,0.024064112267347438
-19770,PATL1,0.15932932256544002
-19773,STX3,0.17673276212441683
-19775,MRPL16,-0.003977268446022285
-19786,MS4A6A,0.5789648145817967
-19787,MS4A4A,0.14905448661335477
-19788,MS4A4E,0.11248004914954275
-19790,MS4A7,0.33702590596470544
-19791,MS4A14,0.18879159898132908
-19793,MS4A1,0.00294563934022396
-19803,CCDC86,0.01652995411840721
-19806,PRPF19,0.10075942763127214
-19809,TMEM109,-0.1674553880764869
-19812,SLC15A3,0.3866673344228871
-19813,CD6,-0.23046737502584166
-19815,CD5,-0.15787798222306196
-19816,VPS37C,0.1548025542233081
-19821,DDB1,0.07549183888366592
-19822,TKFC,0.1135190169409826
-19823,CYB561A3,0.0856050009384049
-19824,TMEM138,-0.0038949633667783416
-19825,TMEM216,-0.04625714836535943
-19826,CPSF7,0.055424542624655834
-19827,SDHAF2,-0.15063056908840883
-19838,TMEM258,-0.05702922412317555
-19839,FEN1,-0.09746159677212642
-19841,FADS1,0.09549866506760614
-19844,BEST1,0.19392116138817195
-19845,FTH1,0.5444989229675227
-19848,INCENP,-0.02963901643785724
-19856,ASRGL1,0.19786390265719583
-19860,AHNAK,0.4131594106500716
-19863,EEF1G,-0.23753576848956984
-19864,TUT1,0.032183224473402425
-19865,MTA2,-0.05947397852686207
-19866,EML3,0.05772437062322118
-19869,B3GAT3,-0.1375436941596079
-19870,GANAB,-0.007756182433194822
-19871,INTS5,0.07037107512606458
-19874,CSKMT,-0.07867518924375701
-19875,UQCC3,0.03640047411016783
-19876,UBXN1,-0.16535694479280136
-19878,BSCL2,0.08705722190409504
-19880,HNRNPUL2,0.05286897836735127
-19881,TTC9C,-0.0581617422074711
-19882,ZBTB3,-0.04256451448010103
-19883,POLR2G,-0.22135469790825757
-19885,TAF6L,0.009856386295340806
-19886,TMEM223,-0.17883893677520915
-19888,TMEM179B,0.010543700783413709
-19889,NXF1,0.11997157564503945
-19890,STX5,0.044606188816401146
-19894,WDR74,0.14861033166273024
-19895,SNHG1,-0.049732780671883456
-19896,SLC3A2,-0.1581077400627601
-19907,LGALS12,0.09560934021901282
-19908,PLAAT4,-0.5115175979745045
-19910,PLAAT3,-0.3975792569086689
-19912,ATL3,-0.005659262783206555
-19914,RTN3,0.3542886878232023
-19917,MARK2,0.05693101057577632
-19919,NAA40,-0.1101457570440746
-19920,COX8A,0.021576902869347492
-19921,OTUB1,-0.033855456680874065
-19927,STIP1,0.0195288731639285
-19928,FERMT3,0.14261018287040295
-19929,TRPT1,0.021004756729487244
-19930,NUDT22,-0.008372503570229508
-19932,DNAJC4,0.09014453811757539
-19933,VEGFB,-0.15204151933010845
-19934,FKBP2,-0.0922395714211586
-19936,PPP1R14B,0.006578450467125189
-19939,PLCB3,0.1354732287968626
-19940,BAD,-0.08441309375693941
-19941,GPR137,0.03580261446708925
-19944,ESRRA,0.1831158060834596
-19945,TRMT112,-0.15232646081772278
-19946,PRDX5,0.15738556778518645
-19947,AP003774.2,-0.10581033507807502
-19948,CCDC88B,0.2295289941273771
-19949,RPS6KA4,0.2807656849121126
-19950,LINC02723,0.2056480907908956
-19959,RASGRP2,-0.06504568932569062
-19961,SF1,-0.0448705212315294
-19962,AP001462.1,-0.09589253809480854
-19963,MAP4K2,-0.09166736957925249
-19964,MEN1,-0.019592647465563447
-19966,EHD1,-0.061665298872491635
-19968,ATG2A,0.033184915598578724
-19969,PPP2R5B,0.06662640392559996
-19972,BATF2,0.15478366192444584
-19973,ARL2,-0.09869496740504831
-19974,SNX15,0.03725952051776163
-19975,SAC3D1,0.10242313346448026
-19976,NAALADL1,-0.07984592603446104
-19978,ZFPL1,-0.015227235881554866
-19980,VPS51,-0.11270361678688869
-19982,TM7SF2,-0.15388609135088213
-19983,ZNHIT2,-0.06681438056877803
-19985,FAU,-0.3234247014491553
-19986,SYVN1,0.026085461067443992
-19987,MRPL49,-0.038167368895736985
-19989,AP003068.2,0.04447360884758029
-19990,CAPN1,0.06705267627623027
-19992,POLA2,0.0230598372444871
-19994,CDC42EP2,0.23144502131796402
-19995,DPF2,-0.041371530288142305
-19998,SLC25A45,-0.14613925313490192
-19999,FRMD8,0.07942969554240047
-20000,NEAT1,0.5571314586244224
-20003,MALAT1,-0.5055356425709229
-20005,SCYL1,0.08766208063891465
-20006,LTBP3,-0.20066259664821354
-20008,ZNRD2,-0.05484660174285046
-20009,FAM89B,-0.06515793560976336
-20010,EHBP1L1,0.3503181785885314
-20012,MAP3K11,0.30291990908375765
-20013,PCNX3,-0.028276416015635537
-20014,SIPA1,0.0061828279807890595
-20015,RELA,0.05082182006591791
-20016,RELA-DT,-0.007030477935012477
-20017,KAT5,0.06896643686842208
-20018,RNASEH2C,-0.044902592050751304
-20020,AP5B1,0.3054173893574381
-20024,CFL1,-0.1040882681024588
-20026,MUS81,-0.04742392520615546
-20028,CTSW,-0.6916774773740407
-20029,FIBP,-0.02527153502933001
-20030,CCDC85B,-0.333943140229961
-20032,C11orf68,-0.0027258126365736417
-20033,DRAP1,-0.06997193334041597
-20035,SART1,0.046834600120971855
-20036,EIF1AD,0.031000201104731576
-20037,BANF1,-0.12565129914890916
-20039,CATSPER1,0.11289109347637902
-20042,SF3B2,0.0027957092853720966
-20044,PACS1,-0.10955806930718161
-20050,RAB1B,0.07461445328947859
-20054,YIF1A,-0.01412972541676579
-20058,RIN1,0.19053252109894434
-20060,BRMS1,0.043628878311067334
-20066,MRPL11,-0.14254576498838106
-20068,AP002748.4,-0.021039561300432955
-20069,DPP3,0.09916437019434562
-20072,ZDHHC24,-0.08762693745131411
-20077,CCS,0.009453939217286864
-20078,RBM14,0.031870048210695695
-20079,RBM4,-0.05610352556075242
-20080,RBM4B,-0.043329890787875004
-20083,RCE1,0.032548919588494186
-20089,KDM2A,0.11661937604147041
-20091,GRK2,0.28878565217242874
-20092,ANKRD13D,0.16556698564993305
-20093,SSH3,0.10046780043140055
-20095,RAD9A,-0.08335242177462203
-20096,POLD4,0.07237772244574371
-20100,PPP1CA,-0.0854893070653526
-20101,TBC1D10C,-0.3868673195089521
-20103,RPS6KB2,0.062267209870485375
-20106,CORO1B,-0.04448552108640447
-20109,TMEM134,-0.13596762606278406
-20110,AIP,-0.1687516958023577
-20111,PITPNM1,0.028837514714785448
-20112,CDK2AP2,-0.14920121081457502
-20114,GSTP1,0.27121515560227627
-20117,NDUFV1,0.04171155816857096
-20118,NUDT8,-0.11047772154772323
-20123,UNC93B1,0.38634262046797635
-20124,ALDH3B1,0.2973779529364544
-20126,NDUFS8,-0.036032761205392325
-20127,TCIRG1,0.3708308278649609
-20128,AP002807.1,0.10293890097189178
-20129,CHKA,0.1236834253458203
-20131,KMT5B,0.056687559400938524
-20132,C11orf24,0.04490551813456378
-20135,PPP6R3,0.029147691419350984
-20139,CPT1A,0.10842486292008367
-20141,MRPL21,-0.044830274402101036
-20142,IGHMBP2,0.04590139260972633
-20149,TPCN2,0.12495626964015524
-20162,LTO1,0.03707812867713271
-20175,FADD,-0.1483655346326286
-20177,PPFIA1,0.2587821086419941
-20191,AP002387.2,-0.19308681387388457
-20192,NADSYN1,0.006095754988053231
-20205,AP002495.1,-0.035240113877617656
-20207,RNF121,-0.0762808357609514
-20208,IL18BP,0.0012863370641786494
-20209,NUMA1,-0.03361368563705233
-20212,LAMTOR1,0.24057465708659773
-20214,ANAPC15,0.06459404143450564
-20215,FOLR3,0.19119229021856313
-20218,FOLR2,0.1931975896756216
-20219,INPPL1,0.3429744192187583
-20222,CLPB,0.18962970864989012
-20231,ARAP1,0.36706164877906855
-20235,STARD10,0.06878033803397725
-20236,AP002381.2,0.020903252530860773
-20237,ATG16L2,0.25111888408570426
-20238,FCHSD2,0.07600285519450177
-20241,P2RY2,0.13411977824818488
-20246,RELT,0.28553249459507196
-20248,FAM168A,0.0639541207645654
-20249,AP000763.3,0.07278408463698317
-20251,RAB6A,0.059980987554998935
-20254,MRPL48,-0.11304920822095064
-20255,COA4,-0.03148626696981227
-20256,PAAF1,-0.0583699692080336
-20259,UCP2,-0.06909169982930884
-20260,AP003717.4,0.04733886682427651
-20262,C2CD3,-0.04452123691640412
-20263,PPME1,-0.05659971366006085
-20266,PGM2L1,-0.04801722776540931
-20269,KCNE3,0.42337969727804614
-20273,POLD3,0.05944717332645988
-20275,RNF169,0.08143778227445574
-20276,XRRA1,-0.1644667929436452
-20278,SPCS2,-0.29934843599336114
-20279,NEU3,0.0591054424430676
-20285,ARRB1,0.18008187195323805
-20288,RPS3,-0.523421830545113
-20290,GDPD5,0.06363454884830647
-20299,DGAT2,0.11289859234647767
-20303,UVRAG,0.14239062621462995
-20309,THAP12,-0.0629192452858953
-20311,AP002360.1,-0.03659938444335159
-20312,EMSY,-0.020550302569906746
-20323,ACER3,0.30953382255382056
-20330,PAK1,0.4260727986607704
-20332,CLNS1A,-0.009124343727343468
-20334,RSF1,-0.008983323916190796
-20337,AAMDC,-0.025151045093802376
-20341,INTS4,0.07677963640433204
-20346,NDUFC2,-0.09317652424065098
-20347,ALG8,-0.008413677483455133
-20349,KCTD21,-0.00925988869944478
-20351,GAB2,0.22111446991393463
-20355,NARS2,-0.009808515068517633
-20371,PRCP,0.16134500963641488
-20372,DDIAS,0.08902305793871719
-20374,RAB30,-0.020203221253536283
-20375,RAB30-DT,-0.04729784803994817
-20377,PCF11,0.03938866239924769
-20379,AP000873.2,-0.040901453141206384
-20381,ANKRD42,0.09917745943859464
-20383,CCDC90B,-0.08675945409279451
-20389,TMEM126B,-0.10057875615519514
-20390,TMEM126A,-0.11876048626658083
-20391,CREBZF,-0.07676690482477437
-20393,SYTL2,-0.2649636879343972
-20396,PICALM,0.36535944097807993
-20398,EED,-0.015087757672847426
-20399,HIKESHI,-0.049544391971277746
-20405,PRSS23,-0.4629806773808759
-20410,TMEM135,-0.04735513681570954
-20417,CTSC,-0.19496866283728823
-20435,CHORDC1,-0.057229629269066315
-20445,SLC36A4,0.15923322522693864
-20448,SMCO4,0.39893011495129477
-20449,CEP295,0.027051499204989688
-20451,TAF1D,-0.08363736965960343
-20452,C11orf54,0.10093498409195552
-20453,MED17,-0.03968497448627527
-20456,PANX1,-0.016946063196655967
-20460,MRE11,0.03310084630401161
-20462,ANKRD49,-0.1437782401798437
-20464,FUT4,0.1333455484981591
-20465,PIWIL4,0.15217604645600383
-20471,CWC15,-0.10228906083865485
-20475,SRSF8,-0.0936739493208235
-20476,ENDOD1,0.02876905770216907
-20477,AP000787.1,-0.06184447735162529
-20478,SESN3,-0.016409855896251414
-20482,FAM76B,0.0017179847336905474
-20483,CEP57,-0.10686245854841372
-20484,MTMR2,-0.01172780968194301
-20485,MAML2,-0.08509172228824155
-20487,CCDC82,-0.17620280772026184
-20488,JRKL,-0.0005907785668878563
-20509,BIRC3,0.011647903935635018
-20510,BIRC2,0.039164320870201476
-20511,TMEM123,0.2814957959806534
-20525,DCUN1D5,-0.07505569367724609
-20530,PDGFD,-0.34669340029088586
-20534,CASP4,0.1002707815599281
-20535,CASP5,0.14592508571538426
-20536,CASP1,0.4860737521221709
-20537,CARD16,0.29001557262053396
-20542,MSANTD4,0.0456306905199446
-20543,KBTBD3,-0.08793854448350484
-20544,AASDHPPT,-0.09812119751986541
-20550,AP000766.1,0.06657652310204538
-20551,CWF19L2,-0.1456180562798286
-20552,ALKBH8,0.06392093525023765
-20556,RAB39A,0.16154589877122155
-20557,AP003307.1,0.015671781344196698
-20558,CUL5,0.05801884008904992
-20560,ACAT1,-0.030235112657573348
-20562,NPAT,-0.06606514721675859
-20563,ATM,-0.27644529393878675
-20565,POGLUT3,-0.06939891989582889
-20567,DDX10,-0.007574454952692451
-20573,RDX,0.11794929154843552
-20577,FDX1,-0.04298932860524714
-20587,POU2AF1,-0.03265327483975255
-20594,SIK2,0.026663377977295215
-20595,PPP2R1B,0.06222781933120025
-20597,ALG9,-0.033415517168244306
-20599,FDXACB1,-0.08247232352329142
-20600,C11orf1,-0.10468482806956358
-20606,DLAT,0.07336801909002322
-20608,NKAPD1,-0.016454721505367406
-20609,TIMM8B,0.03839623731696977
-20610,SDHD,-0.010145357149858621
-20611,IL18,0.18974168315140544
-20615,PTS,-0.02054152278836124
-20618,LINC02762,-0.0464913702152086
-20635,ZW10,0.022631132188686936
-20638,USP28,-0.1598984676624617
-20642,ZBTB16,-0.03803520133366927
-20647,C11orf71,-0.03729299356292925
-20648,RBM7,-0.009302535490006475
-20649,REXO2,-0.22733511957816857
-20668,BUD13,0.0011590580872502011
-20670,ZPR1,0.02703812316145751
-20678,SIK3,0.19562626729111704
-20680,PAFAH1B2,0.1935961242547043
-20681,SIDT2,0.24422202128400025
-20682,TAGLN,0.08833118754414952
-20683,PCSK7,-0.056831832902951934
-20684,RNF214,-0.0333102495031367
-20685,BACE1,0.1330011720768745
-20688,CEP164,0.0025378489839460055
-20695,FXYD6,0.13863312120106994
-20697,IL10RA,0.1374789563808918
-20702,JAML,0.523090009768654
-20703,MPZL3,0.03582522844673006
-20704,MPZL2,0.17948594573656587
-20705,CD3E,-0.5179951831000079
-20706,CD3D,-0.5499808404341607
-20707,CD3G,-0.5094756891565168
-20708,UBE4A,0.15824328035473922
-20710,ATP5MG,-0.03282093658565245
-20713,KMT2A,-0.1147201077979173
-20716,TMEM25,-0.11136172179094236
-20718,ARCN1,0.018048308638113946
-20724,DDX6,-0.1883428701088667
-20725,AP004609.3,0.06798721317099041
-20726,CXCR5,-0.03674818639563134
-20728,BCL9L,-0.18039329041388913
-20732,CCDC84,0.022159946413940906
-20734,RPS25,-0.40461934370482894
-20735,TRAPPC4,-0.1535264766992612
-20738,HYOU1,-0.013986684438139425
-20741,VPS11,-0.006163546835705636
-20742,HMBS,0.04004067889615311
-20743,H2AFX,0.0753092237395292
-20744,DPAGT1,0.057843208431215404
-20745,C2CD2L,0.19385629062006085
-20746,HINFP,-0.008051977765786618
-20748,NLRX1,0.13649046359007036
-20751,CBL,0.17184499709925383
-20753,RNF26,-0.010144229449672052
-20774,OAF,0.2778100347191818
-20778,ARHGEF12,-0.07493743763280578
-20781,TBCEL,0.042545505146076884
-20783,SC5D,-0.04963324479494893
-20785,SORL1,0.029836867205009734
-20793,UBASH3B,0.07707213070981836
-20796,CRTAM,-0.08884781440469197
-20799,HSPA8,-0.09221003989110811
-20804,GRAMD1B,0.1766114302182796
-20820,VWA5A,0.04744208057399015
-20834,TBRG1,0.08622707017875555
-20835,SIAE,0.06469103616574429
-20836,SPA17,0.017705693353504397
-20837,NRGN,0.0886177352943339
-20842,MSANTD2,0.007630122864252142
-20852,CCDC15,-0.03365646619143951
-20853,SLC37A2,0.18864908169132094
-20854,TMEM218,-0.08213961973945169
-20862,EI24,-0.0011898009126339259
-20863,STT3A,0.04400899417887879
-20871,HYLS1,-0.06831044589054541
-20872,PUS3,-0.00014854049636098827
-20881,RPUSD4,0.0021319042920243344
-20883,FAM118B,0.12392377076044077
-20884,SRPRA,-0.16136104639244972
-20885,FOXRED1,0.024390089489178337
-20886,TIRAP,0.011331974213742917
-20889,DCPS,-0.08801026531045306
-20891,ST3GAL4,-0.11100684382431902
-20904,ETS1,-0.4255789239920279
-20907,FLI1,0.2005768430794699
-20908,SENCR,0.07838092761514205
-20913,ARHGAP32,0.047839881743787345
-20919,NFRKB,-0.03406873347204011
-20920,PRDM10,0.05610810345510277
-20922,APLP2,0.5568059081804788
-20923,ST14,0.11421931469796098
-20924,ZBTB44,-0.03734445029191079
-20932,SNX19,0.13218983495439615
-20955,NCAPD3,-0.06631110280443879
-20957,VPS26B,0.002775067140356687
-20958,THYN1,-0.05787925237563759
-20959,ACAD8,-0.14377948067815582
-20976,AC026369.3,0.15497679633388703
-20980,SLC6A12,0.13563120241751855
-20985,AC007406.5,0.004780548977436207
-20986,KDM5A,0.039966546060556635
-20987,CCDC77,0.00047344439351108396
-20989,NINJ2,0.13029048817767344
-20992,AC021054.1,0.09570220856723811
-20994,WNK1,0.10630825642596287
-20998,ERC1,0.12339389646242917
-21004,ADIPOR2,0.08011372573125808
-21006,CACNA2D4,0.15327470237367538
-21011,DCP1B,-0.004633188409142895
-21025,FKBP4,-0.02076444538768315
-21028,ITFG2,-0.0822445860068775
-21032,RHNO1,-0.0039027817049014334
-21046,CRACR2A,-0.11260738698924062
-21048,PARP11,-0.030040813879280072
-21049,AC005842.1,-0.07612939668657463
-21052,CCND2,-0.36403308405191054
-21053,TIGAR,0.022440467183111257
-21056,C12orf4,0.0832250136627174
-21058,DYRK4,-0.0765500121713936
-21061,NDUFA9,0.06302774509795779
-21083,CD9,0.06439634033856508
-21085,TNFRSF1A,0.25541912508893094
-21087,LTBR,0.3536699661379874
-21089,CD27-AS1,0.05777121969062087
-21090,CD27,-0.23746478494746837
-21091,TAPBPL,0.010047637910648643
-21092,VAMP1,-0.06007271827829546
-21094,MRPL51,-0.10372042513508938
-21095,NCAPD2,-0.03830229312271937
-21098,GAPDH,0.43329671072455406
-21100,IFFO1,0.10497979134061865
-21102,NOP2,-0.010085765089664253
-21103,CHD4,0.1065831284936
-21105,LPAR5,-0.1252842650758512
-21106,ACRBP,0.06210974366378561
-21107,ING4,-0.0655999163625656
-21109,ZNF384,0.07111983932276403
-21112,COPS7A,0.17095150909692253
-21114,MLF2,0.0550729888502223
-21115,PTMS,-0.20352710543941835
-21116,LAG3,-0.21891129880200003
-21117,CD4,0.3894900315687765
-21118,GPR162,0.12497889610522868
-21122,USP5,0.01815515945790449
-21123,TPI1,0.25721941711617824
-21124,SPSB2,-0.10788965589649328
-21125,LRRC23,0.02621785797123883
-21128,ATN1,0.002599687020264149
-21129,C12orf57,-0.36325821168864936
-21131,PTPN6,0.2961990731962203
-21134,PHB2,-0.09096590205621256
-21135,EMG1,-0.07277006649937882
-21136,LPCAT3,0.20745998540731558
-21139,C1RL,0.052560600991145505
-21140,C1RL-AS1,0.0007225019022496364
-21146,PEX5,-0.008902622144263423
-21149,CD163,0.35833514799241606
-21158,SLC2A3,0.27671693954235227
-21161,FOXJ2,0.11033424101991832
-21162,C3AR1,0.29732733133828715
-21163,NECAP1,0.024863230056420382
-21164,CLEC4A,0.39492430480455143
-21173,LINC00937,0.17635492526835603
-21176,CLEC4D,0.22846419042587007
-21177,CLEC4E,0.39004209374662885
-21182,RIMKLB,0.09859969057762534
-21188,PHC1,-0.04236356303442087
-21189,M6PR,0.15018181764738828
-21190,KLRG1,-0.512217696971922
-21193,A2M-AS1,-0.2164540448372424
-21201,AC092821.3,-0.23566601500911094
-21202,KLRB1,-0.48295031064683497
-21204,AC010186.3,-0.03568806494170384
-21205,CLEC2D,-0.2870304135743258
-21207,CLECL1,0.04769083668941615
-21208,CD69,-0.2070519145491415
-21209,KLRF1,-0.5480128293580124
-21210,CLEC2B,-0.20965576358339227
-21216,CLEC12A,0.463070641732107
-21218,CLEC12B,0.132473723271978
-21222,CLEC7A,0.5266630193089675
-21225,GABARAPL1,0.030986557690675244
-21227,KLRD1,-0.663250730132468
-21231,KLRK1,-0.5421517990233045
-21232,KLRC4,-0.3353950048455591
-21237,LINC02446,-0.16538319196113793
-21238,MAGOHB,-0.10085356107644863
-21240,YBX3,0.3967968108548073
-21245,PRH1,-0.19282397255224962
-21250,TAS2R14,0.020166534024819764
-21259,SMIM10L1,-0.11343269804010145
-21269,ETV6,0.28666737998339037
-21274,BORCS5,-0.07304322186596207
-21275,DUSP16,-0.09904384567743277
-21277,CREBL2,0.1367773650220527
-21281,CDKN1B,-0.16226020512723052
-21282,AC008115.3,-0.04389354517046442
-21284,APOLD1,-0.07875370244522564
-21285,DDX47,-0.008081112561730995
-21290,HEBP1,0.13711478869905097
-21302,ATF7IP,-0.032531901218656116
-21303,PLBD1,0.43485634852715027
-21305,PLBD1-AS1,0.13961412932761366
-21308,AC010168.2,-0.03937096902242478
-21309,HIST4H4,0.02054524876902399
-21310,H2AFJ,0.11389885796292515
-21311,WBP11,-0.004331064046617098
-21318,ERP27,-0.03463980659226871
-21319,ARHGDIB,-0.05581388705916833
-21325,PTPRO,0.13521935133486962
-21326,EPS8,0.13300860392653763
-21328,STRAP,-0.10447178252216646
-21329,DERA,0.024522930108648797
-21331,MGST1,0.3032634459197431
-21346,PLEKHA5,-0.1257062333784477
-21348,AEBP2,-0.003190862568166542
-21362,PYROXD1,-0.06712649517165514
-21363,RECQL,-0.04041117192113058
-21364,GOLT1B,-0.06315999441761296
-21367,LDHB,-0.29581063394273366
-21374,CMAS,-0.056370085787584574
-21379,C2CD5,0.15409468632621184
-21382,ETNK1,0.14251621359022631
-21392,BCAT1,0.14461359237384766
-21396,LRMP,0.042699411329066295
-21400,ETFRF1,-0.09661855763182146
-21401,KRAS,-0.050804051339566746
-21404,AC087239.1,0.0028399121811434583
-21421,ITPR2,0.18545204765652318
-21424,INTS13,0.02088449232245551
-21425,FGFR1OP2,-0.12217660169634867
-21426,TM7SF3,0.06558255765120986
-21428,MED21,-0.13931085779507016
-21429,AC092747.4,0.06189664131428119
-21432,STK38L,0.22937531080863083
-21442,MRPS35,-0.04319932877124301
-21445,KLHL42,0.006123777225452617
-21449,CCDC91,-0.11999035425024149
-21455,FAR2,0.12455167184397843
-21461,ERGIC2,-0.1814312735192353
-21464,TMTC1,0.126373275575872
-21470,IPO8,0.066864774405687
-21472,CAPRIN2,-0.09137842916395254
-21477,DDX11,0.026732627054944887
-21479,SINHCAF,-0.10701596103849763
-21486,ETFBKMT,0.036192393120956866
-21487,AMN1,-0.05371974757803318
-21488,AC023157.3,0.06549679203935113
-21490,LINC02422,-0.007467381683618546
-21491,RESF1,-0.13364328342331544
-21492,AC016957.2,-0.08258432987850442
-21494,BICD1,-0.08936525622798047
-21498,FGD4,0.37062211943167406
-21499,DNM1L,0.08454608406581038
-21501,YARS2,-0.07593818562964473
-21517,ALG10B,-0.006762367204263286
-21519,CPNE8,0.20722070721798583
-21522,KIF21A,-0.25100591133011557
-21524,ABCD2,-0.11364056580776948
-21527,SLC2A13,0.045497949035202705
-21530,AC079630.1,0.13817884473829783
-21531,LRRK2,0.47588773778263405
-21541,GXYLT1,0.0563927050092358
-21542,YAF2,-0.07716729056555983
-21543,PPHLN1,-0.1138529184005133
-21544,ZCRB1,-0.05629101539829176
-21546,PRICKLE1,0.10675546642024002
-21556,PUS7L,-0.061943071018065746
-21557,IRAK4,-0.0012978409105380818
-21558,TWF1,0.04570886220156662
-21563,NELL2,-0.1331882083932435
-21567,ANO6,0.19248217115617314
-21570,AC008124.1,-0.1525850501898857
-21571,ARID2,0.0377190267141931
-21573,SCAF11,0.20063525274404573
-21574,SLC38A1,-0.4079176563871824
-21576,SLC38A2,0.19604088819479804
-21578,AC008014.1,-0.18667324148104422
-21584,AMIGO2,-0.10451511863818272
-21585,PCED1B,-0.15587649278349808
-21586,PCED1B-AS1,-0.5116247310983569
-21592,RPAP3,0.01937312121227107
-21594,AC004241.1,0.04964707836417914
-21598,SLC48A1,0.05485317121510071
-21600,HDAC7,-0.014786316028804806
-21605,VDR,0.18574487673099307
-21608,TMEM106C,-0.030301706236229967
-21614,SENP1,0.002431590741562832
-21616,ASB8,0.0002339652354432201
-21619,AC024257.3,0.08971775542217836
-21624,ZNF641,0.09705860666982036
-21633,KANSL2,-0.03874830198923701
-21634,CCNT1,0.008078628796799955
-21640,DDX23,0.08556824905392482
-21642,CCDC65,-0.18882279365469085
-21643,FKBP11,-0.4356197225185384
-21644,ARF3,0.21986898799963467
-21650,PRKAG1,0.0831325792677331
-21651,KMT2D,0.1040628597983048
-21655,LMBR1L,-0.010787576894686006
-21656,TUBA1B,0.10687119960217736
-21657,AC011603.2,0.02570198430706221
-21658,TUBA1A,0.35187610753100684
-21659,TUBA1C,0.060556805629965064
-21667,SPATS2,-0.05107359645630419
-21670,MCRS1,0.06503733602351223
-21674,FMNL3,-0.10298376763792177
-21675,TMBIM6,0.22375808500864927
-21676,NCKAP5L,0.16390160679456608
-21678,BCDIN3D,-0.12636239815156017
-21690,SMARCD1,-0.07124335615189634
-21692,COX14,-0.011990642888803938
-21693,AC074032.1,-0.022116889294847942
-21694,CERS5,-0.10170548339335317
-21695,LIMA1,-0.18106087958375303
-21699,LARP4,0.055435396942871064
-21700,DIP2B,0.1773302518692157
-21701,ATF1,-0.027955704232594514
-21703,METTL7A,0.3899848837729131
-21707,SLC11A2,0.14101007234153387
-21708,LETMD1,0.014849862412177962
-21709,CSRNP2,-0.06397959897190883
-21710,TFCP2,0.09093828021115129
-21713,DAZAP2,0.39237357453087324
-21714,SMAGP,-0.18539744744198303
-21715,BIN2,-0.08719778665481231
-21716,CELA1,-0.038868494061932664
-21717,GALNT6,0.07419157360100684
-21718,SLC4A8,0.04542846915430432
-21729,ACVR1B,0.08539100914165987
-21730,GRASP,-0.018106013619713878
-21731,NR4A1,0.11301923819520564
-21733,ATG101,0.10625652785396539
-21774,EIF4B,0.08331320810379668
-21778,SPRYD3,-0.10690436712623297
-21781,CSAD,0.055106861024039354
-21783,ZNF740,0.047392031268654954
-21784,ITGB7,-0.2464080845461938
-21785,RARG,-0.08507147630781434
-21787,MFSD5,0.01689151102353968
-21789,PFDN5,0.09233245295420803
-21791,C12orf10,-0.045227615264727454
-21792,AAAS,0.025620019146816615
-21794,SP1,0.28256476809294606
-21796,PRR13,0.23530356202057637
-21797,PCBP2,0.2285864562770191
-21799,MAP3K12,-0.1492607313294053
-21800,AC023509.3,-0.11412026605691367
-21801,TARBP2,-0.033040345323718856
-21803,ATF7,0.05289695282281804
-21806,ATP5MC2,0.05680989045427299
-21807,CALCOCO1,0.0838128199134062
-21829,SMUG1,-0.10747205188058746
-21834,CBX5,-0.11841949091199114
-21837,HNRNPA1,-0.11997132231330936
-21838,NFE2,0.3054217978197107
-21839,COPZ1,0.010745056787968004
-21843,ZNF385A,0.4217580215314069
-21844,ITGA5,0.1599829197048369
-21847,NCKAP1L,0.2767290512574127
-21855,TESPA1,-0.17824815227064236
-21877,BLOC1S1,0.1694064189919557
-21878,RDH5,-0.0004694093347357976
-21879,CD63,-0.08371926883663108
-21880,AC009779.2,0.029200900075630844
-21881,GDF11,-0.15966487408568747
-21882,SARNP,-0.022306631441980394
-21884,ORMDL2,-0.13034291819841148
-21885,DNAJC14,-0.02730995319932911
-21887,PYM1,-0.04996833879596995
-21888,DGKA,-0.1381898448123688
-21893,RAB5B,0.043021634235029
-21894,SUOX,0.12308720530321297
-21898,RPS26,-0.4510464020959033
-21900,PA2G4,-0.102169269474222
-21902,RPL41,-0.45088131380848384
-21903,ESYT1,-0.12651582855229632
-21904,ZC3H10,0.029954876761394595
-21908,MYL6B,-0.044487807923595114
-21909,MYL6,0.041785533012089614
-21910,SMARCC2,0.039182412314743474
-21912,RNF41,0.12329368154113904
-21913,NABP2,-0.06335535043992963
-21915,ANKRD52,-0.007362521791697226
-21916,COQ10A,-0.02313949607054571
-21918,CS,0.16788788525733295
-21919,AC073896.2,-0.09366051076579371
-21921,CNPY2,-0.08543667751536517
-21922,PAN2,0.05498392702466293
-21923,IL23A,-0.048626226199400824
-21924,STAT2,0.3963967267682941
-21927,TIMELESS,0.08225657128202253
-21929,SPRYD4,-0.007432775932653606
-21931,RBMS2,0.07902186472627473
-21932,BAZ2A,0.18980599429019734
-21933,ATP5F1B,0.18117399296861803
-21934,PTGES3,0.003736538965223948
-21935,NACA,-0.19221933761251933
-21936,PRIM1,-0.09170699079926482
-21945,NEMP1,-0.07823432445669569
-21946,NAB2,0.039547269080873
-21947,STAT6,0.2547639021301489
-21948,LRP1,0.49085561867726285
-21952,SHMT2,0.008441279107623193
-21954,STAC3,0.23892999456640313
-21955,R3HDM2,-0.00011282745306473956
-21961,ARHGAP9,0.10031089097162672
-21962,MARS,0.06399088052234177
-21963,DDIT3,0.025800039364418194
-21965,MBD6,0.09180044721829356
-21966,DCTN2,0.010130223181847812
-21968,PIP4K2C,-0.0013137739414035468
-21974,OS9,0.27218945461595906
-21976,AGAP2,-0.10395594793412462
-21978,TSPAN31,-0.03377366904343264
-21979,CDK4,-0.06252782259015469
-21980,MARCH9,-0.12245815036275408
-21982,METTL1,-0.015099604190497475
-21984,TSFM,-0.057312775454137044
-21985,AVIL,0.05576755326085151
-21988,CTDSP2,0.1717905272552526
-21992,ATP23,0.004817620836468258
-21993,GIHCG,-0.09258195546939228
-22003,SLC16A7,-0.007983596638652373
-22007,TAFA2,0.0565418115935738
-22009,USP15,0.3938832213538252
-22010,MON2,0.13414957669750496
-22013,AC048341.1,0.025331087801763682
-22021,RXYLT1,-0.05215774679653021
-22029,C12orf66,0.06861025114190822
-22031,XPOT,0.016573446643822178
-22033,TBK1,0.00047704433704692903
-22034,RASSF3,0.27083475719237926
-22039,GNS,0.4516135593028999
-22047,LEMD3,-0.05244931637168186
-22060,LLPH,0.014250412924925222
-22062,TMBIM4,0.05851000276888205
-22063,IRAK3,0.4668471282627401
-22065,HELB,-0.07949036251592125
-22068,CAND1,-0.007939814568265953
-22073,DYRK2,-0.21274961416650412
-22077,IFNG-AS1,-0.07859793429566161
-22081,MDM1,-0.02998485352810523
-22085,RAP1B,-0.15427302068149373
-22087,NUP107,-0.006792060766742977
-22088,SLC35E3,0.026084017162832842
-22089,MDM2,0.09343325412340961
-22092,CPM,0.1999659222489329
-22094,CPSF6,-0.004589331003036799
-22096,LYZ,0.6568269678477596
-22097,AC020656.1,0.530462022903405
-22098,YEATS4,-0.05610333060504444
-22100,FRS2,0.06843442992928672
-22101,CCT2,-0.11972768038960087
-22105,RAB3IP,-0.032858651709614
-22107,AC025159.1,0.044569972119298376
-22112,CNOT2,0.06844015497615183
-22125,ZFC3H1,0.07892815147410649
-22126,THAP2,-0.09797454650466504
-22127,TMEM19,0.02090649692385106
-22129,RAB21,0.07140121024239493
-22131,TBC1D15,0.0025852657886687724
-22145,ATXN7L3B,-0.11235663191921587
-22154,GLIPR1,0.36774417941598475
-22156,KRR1,-0.14102319392445345
-22164,NAP1L1,0.18792990746612834
-22166,BBS10,0.05311211065850219
-22167,OSBPL8,0.21826776654601043
-22170,ZDHHC17,0.02114089572243999
-22189,PAWR,-0.019944868409438453
-22191,PPP1R12A,0.023104894325917297
-22201,LIN7A,0.1994632108997382
-22208,CCDC59,-0.07227953535119949
-22209,METTL25,0.09520665981981918
-22214,TMTC2,0.1969874310887349
-22232,C12orf29,-0.07384267668175838
-22233,CEP290,-0.06180120993999262
-22234,TMTC3,0.03597201756841866
-22239,DUSP6,0.5393563631284451
-22240,AC024909.1,0.12433724520245486
-22244,POC1B,-0.016813867556951956
-22245,GALNT4,0.044559801851576354
-22246,POC1B-AS1,0.007042715941496941
-22248,ATP2B1,0.1500796708273719
-22249,ATP2B1-AS1,0.3119648261663
-22273,BTG1,-0.22822789690035283
-22274,AC025164.1,-0.01930726889166697
-22278,LINC02397,0.002014410998348205
-22280,EEA1,0.129437737162462
-22286,NUDT4,0.02036526484950702
-22287,UBE2N,-0.06250293697785606
-22288,MRPL42,-0.1061892130952146
-22290,SOCS2,-0.10028015293543324
-22291,CRADD,-0.078763835179653
-22296,PLXNC1,0.4002705179051726
-22301,CEP83,-0.02333008532748212
-22304,TMCC3,0.0007319838365632424
-22305,NDUFA12,-0.27163713814524076
-22306,NR2C1,0.06366718084492752
-22307,FGD6,0.1798260754335749
-22308,VEZT,-0.025400388874695302
-22310,METAP2,0.04141191661545094
-22311,USP44,-0.05272180368477049
-22315,SNRPF,-0.18255294685571039
-22318,HAL,0.14439910091156505
-22320,LTA4H,0.4717068352776794
-22323,ELK3,0.13191271460438542
-22326,CDK17,-0.053882124605952306
-22329,NEDD1,0.02209719744639508
-22338,TMPO-AS1,-0.0028361782203879305
-22339,TMPO,0.08342952842789311
-22340,SLC25A3,0.02772483140544145
-22341,IKBIP,0.00016516037400131798
-22342,APAF1,0.3014723024041789
-22348,UHRF1BP1L,0.2403548814901552
-22350,ACTR6,-0.17227690161109802
-22352,SCYL2,0.09341857227039699
-22360,UTP20,-0.0339833155299539
-22361,ARL1,-0.1113904046301516
-22367,CHPT1,0.19629214224712116
-22369,GNPTAB,-0.2963656335445928
-22370,DRAM1,0.17189687623885083
-22373,WASHC3,-0.08042553552584707
-22375,NUP37,-0.06895903106700178
-22398,HSP90B1,0.030257493853070968
-22399,C12orf73,-0.0844925999398108
-22400,TDG,-0.010465129836635166
-22402,HCFC2,-0.005727726276448376
-22403,NFYB,-0.14858737626788884
-22406,TXNRD1,0.06423778945310389
-22408,EID3,-0.0784084622979844
-22409,CHST11,0.004333469601310279
-22412,C12orf45,-0.029506614304755534
-22414,WASHC4,0.288816192084943
-22415,APPL2,-0.060759585101191685
-22416,C12orf75,-0.36609596690440976
-22424,CKAP4,0.30634033284465007
-22427,TCP11L2,0.019232738759505333
-22428,POLR3B,-0.032826502013028856
-22433,RIC8B,-0.007153667884471171
-22436,TMEM263,-0.15319310275819403
-22438,CRY1,-0.05151326872932001
-22441,BTBD11,0.01717571767487454
-22443,PWP1,-0.13504483510948817
-22444,PRDM4,-0.043162631247190844
-22449,CMKLR1,0.10340007637167456
-22451,FICD,-0.03368992510223088
-22452,SART3,0.0048895809153892975
-22454,ISCU,-0.14495256727189623
-22456,SELPLG,0.09407899120385727
-22458,CORO1C,0.3357856457355609
-22459,SSH1,0.090421890863024
-22463,USP30,0.037820831049363894
-22464,USP30-AS1,-0.10247780593699769
-22465,ALKBH2,-0.02518446395801906
-22466,UNG,-0.057102022878956216
-22473,KCTD10,-0.08532390202984376
-22474,UBE3B,-0.03058944465511286
-22475,MMAB,-0.11358823213660325
-22476,MVK,-0.053346818972692556
-22480,GLTP,0.15124752527185958
-22482,TCHP,-0.041101173815256804
-22483,GIT2,0.19247188935950169
-22485,ANKRD13A,0.1949908553312892
-22486,C12orf76,-0.08214126968575844
-22489,ATP2A2,0.13017864124012835
-22490,ANAPC7,-0.027885949491582757
-22491,ARPC3,0.31738104044958587
-22492,GPN3,0.03920886566837917
-22493,FAM216A,-0.046747860815701425
-22494,VPS29,0.1492221209024229
-22496,PPTC7,0.16844534694844251
-22497,TCTN1,0.0211118114522862
-22498,HVCN1,0.23400653244388978
-22499,PPP1CC,0.10118027466011704
-22506,PHETA1,0.12564059272269226
-22508,SH2B3,0.33200595797855154
-22509,ATXN2,0.10812227647598237
-22511,BRAP,0.06336104314107525
-22512,ACAD10,0.011702948456193223
-22513,ALDH2,0.4100538729263276
-22514,MAPKAPK5-AS1,-0.1618041923241122
-22515,MAPKAPK5,0.05128867721090215
-22516,TMEM116,-0.12367061008137088
-22517,ERP29,0.013344542927944585
-22519,NAA25,0.09887083160644658
-22521,TRAFD1,0.12228898869693061
-22522,HECTD4,0.06487120087557811
-22524,RPL6,-0.32628059949294347
-22525,PTPN11,0.03756725604115155
-22526,RPH3A,0.12986641850665462
-22527,OAS1,0.33891408270724843
-22529,OAS3,0.2569797283051684
-22530,OAS2,0.31500613397151067
-22534,DDX54,-0.009373615523114486
-22535,RITA1,-0.10822323432469373
-22538,TPCN1,0.1845249704711962
-22540,SLC8B1,-0.002325298470588354
-22541,PLBD2,0.04253983819449033
-22543,SDSL,0.14385154383371168
-22547,RBM19,-0.038410774436891046
-22572,MED13L,0.3167671667476376
-22574,AC130895.1,0.11848922275080111
-22578,LINC00173,0.1089647552842107
-22584,C12orf49,-0.06903545106326447
-22588,FBXW8,-0.07755125067374383
-22592,TESC,0.0993650291597447
-22594,FBXO21,0.010889397883407958
-22599,RFC5,-0.0369141068139143
-22601,WSB2,0.15078243830333213
-22603,VSIG10,0.052291169259747125
-22605,PEBP1,-0.24502183631813187
-22606,TAOK3,0.07503901939717428
-22608,SUDS3,-0.033628744656487596
-22627,PRKAB1,0.05040466039720388
-22631,RAB35,0.01073317449128033
-22633,GCN1,-0.015269825183901504
-22634,RPLP0,-0.14554545458359733
-22635,PXN-AS1,-0.05457176326048392
-22636,PXN,0.03775361450817629
-22643,COX6A1,-0.1644052786080898
-22644,TRIAP1,0.002019436475278672
-22645,GATC,-0.006700199163114931
-22646,SRSF9,0.04463608481253223
-22647,DYNLL1,0.0758171800646341
-22650,COQ5,-0.09132403728220322
-22651,RNF10,0.11101942039338461
-22652,POP5,-0.19694968720322273
-22655,MLEC,0.02029328513232541
-22659,UNC119B,0.014675999024567073
-22661,ACADS,-0.0414408935296935
-22663,SPPL3,0.06425991058554391
-22667,C12orf43,-0.03710816326449729
-22668,OASL,0.1824532331853922
-22669,P2RX7,0.27057448157308206
-22672,P2RX4,0.17909668659013805
-22673,CAMKK2,0.3506817293654365
-22674,ANAPC5,0.04212612149266465
-22676,RNF34,0.01818086082047082
-22677,KDM2B,-0.02201248366345357
-22679,ORAI1,-0.12028952048548257
-22680,MORN3,-0.12331369736773645
-22682,TMEM120B,-0.24574946758335614
-22683,RHOF,-0.33389607370361385
-22684,LINC01089,-0.11554908434755719
-22687,SETD1B,0.07633962078950446
-22692,PSMD9,-0.055623086562140156
-22695,BCL7A,0.004968070407209331
-22697,MLXIP,0.14097398337642061
-22703,VPS33A,0.033331736391025046
-22704,CLIP1,0.2257646472857074
-22706,ZCCHC8,0.03038157531007844
-22708,RSRC2,-0.06921936832108887
-22713,HCAR3,0.1347113100157907
-22715,DENR,-0.06106048878658337
-22717,HIP1R,-0.14110848333886528
-22718,VPS37B,-0.023781002253704386
-22721,OGFOD2,0.05375207102690644
-22722,ARL6IP4,-0.048077474480150775
-22723,PITPNM2,-0.11253254890984733
-22725,MPHOSPH9,-0.09052996480269945
-22726,C12orf65,-0.21482390173740876
-22727,CDK2AP1,0.194253187031726
-22729,SBNO1,0.004352751313517531
-22730,SBNO1-AS1,-0.08148074401726779
-22731,KMT5A,-0.031002747671030687
-22732,RILPL2,0.29633054129573133
-22733,SNRNP35,-0.047678216993655605
-22738,TMED2,0.07081217040185278
-22739,DDX55,-0.03980389331593487
-22740,EIF2B1,-0.025177054619194623
-22741,GTF2H3,0.010067202674307878
-22744,ATP6V0A2,-0.0008711828938363033
-22747,CCDC92,-0.07368348025885461
-22755,ZNF664,-0.001989591340246505
-22759,NCOR2,0.12350322450197793
-22761,SCARB1,0.1625747863990096
-22763,UBC,0.08432043257301677
-22764,DHX37,0.009428857437134553
-22765,BRI3BP,0.14456188856534186
-22767,AACS,0.0013682555703732828
-22811,SLC15A4,0.014074944528508939
-22813,GLT1D1,0.30547916824800897
-22834,STX2,-0.0448598822015389
-22837,RAN,-0.23618950802420582
-22855,SFSWAP,0.033912383733790394
-22857,MMP17,0.11690353026263209
-22859,ULK1,0.10947297237286546
-22861,PUS1,-0.03028641823351722
-22863,EP400,-0.017095619940553114
-22866,DDX51,-0.03223782759338769
-22867,NOC4L,0.0075874309496618605
-22869,LINC02361,-0.0886379678196706
-22880,FBRSL1,0.017520874583127396
-22884,POLE,0.09792517226701412
-22885,PXMP2,-0.059435881821421885
-22886,PGAM5,-0.056852527727555745
-22887,ANKLE2,0.036783716678347
-22888,GOLGA3,0.0654266753349734
-22889,CHFR,0.04810805019118071
-22892,ZNF605,-0.06808614725541182
-22893,ZNF26,-0.017219018490127545
-22895,ZNF84,-0.07022532151630452
-22896,ZNF140,0.010064155673106095
-22897,ZNF891,-0.005074548573398705
-22900,ZNF268,-0.024722482619675034
-22917,MPHOSPH8,-0.15595886630566907
-22919,PSPC1,0.07894775922028638
-22920,ZMYM5,-0.023826064057386125
-22922,AL355001.2,0.040904659628822275
-22923,ZMYM2,0.09764122161261657
-22929,CRYL1,0.07599533076642732
-22931,IFT88,-0.02493036581853062
-22934,EEF1AKMT1,-0.037502289040832214
-22936,XPO4,0.005447205152475542
-22937,LATS2,0.015657918174035515
-22939,SAP18,-0.11802694158846512
-22941,MRPL57,-0.1767163940752586
-22944,ZDHHC20,0.21026851694204965
-22946,MICU2,-0.05982214155573794
-22956,SACS,-0.12698393078823825
-22961,MIPEP,-0.03724786605077322
-22965,SPATA13,-0.06719958407240725
-22971,PARP4,0.08991935885218452
-22978,CENPJ,0.02917672899407747
-22984,MTMR6,0.1922459093466855
-22986,NUP58,0.14580457071523684
-22990,RNF6,0.004619222324426253
-22992,CDK8,0.05108787744112902
-22999,USP12,-0.007812782089077824
-23004,RPL21,-0.36728767962570874
-23007,GTF3A,-0.24791401083160844
-23008,MTIF3,-0.013014728774946865
-23009,LNX2,-0.010707571693772197
-23010,POLR1D,0.041105711841808695
-23018,FLT3,0.08859374323377055
-23020,PAN3,0.08232958753985099
-23022,POMP,0.0649677892679364
-23023,SLC46A3,-0.0359752438886336
-23027,SLC7A1,-0.006802517187793167
-23028,UBL3,0.1390217130690458
-23036,KATNAL1,-0.022677904231752876
-23038,LINC00426,-0.13275264521526042
-23042,HMGB1,-0.02118834077430362
-23044,USPL1,0.0016317340944231719
-23045,ALOX5AP,-0.08616972733706839
-23054,HSPH1,0.04972018968234379
-23059,FRY,0.2370659495476278
-23062,BRCA2,0.05499554921639215
-23063,N4BP2L1,0.09065046998747069
-23064,N4BP2L2,-0.05466055263024956
-23065,N4BP2L2-IT2,0.029078439019696842
-23066,PDS5B,0.09359829199324357
-23081,RFC3,0.039521869273225395
-23094,SPART,0.13267954671777027
-23098,RFXAP,-0.033779902900899404
-23101,ALG5,-0.14584265931455404
-23102,EXOSC8,-0.1442877344860003
-23103,SUPT20H,0.011611148619173636
-23112,UFM1,-0.04042671306365861
-23118,PROSER1,0.04880138838076566
-23119,NHLRC3,0.11859635741400162
-23122,COG6,-0.008676956252451485
-23125,FOXO1,-0.07187198766674929
-23126,MRPS31,-0.06823831316539723
-23129,ELF1,0.08916408769400332
-23130,WBP4,-0.004158411365261806
-23133,KBTBD7,0.05478450482887091
-23134,MTRF1,-0.07150086459596537
-23136,NAA16,-0.08529512915864557
-23137,RGCC,-0.1346653221469959
-23138,VWA8,-0.021860059402911546
-23140,DGKH,-0.01653467459560601
-23142,AKAP11,0.0391463641664753
-23148,EPSTI1,0.3319847904178926
-23149,DNAJC15,-0.03335127729460259
-23159,LACC1,0.12049576197802245
-23170,TSC22D1,0.0323678700183047
-23175,NUFIP1,-0.010811065479450961
-23177,GPALPP1,0.0437045354993785
-23178,GTF2F2,0.0035937779587471553
-23180,TPT1,-0.1577217390767606
-23182,TPT1-AS1,-0.04379996630146568
-23184,SLC25A30,0.07437954772013139
-23186,COG3,0.004495207982536085
-23191,ZC3H13,-0.04506265180323886
-23194,LCP1,0.40025154706565896
-23198,RUBCNL,0.05226202823517762
-23201,LRCH1,0.05870596794360781
-23203,ESD,0.09560512592948646
-23206,SUCLA2,-0.007849221292655353
-23211,NUDT15,0.04588232157132476
-23213,MED4,-0.15409340197490715
-23215,ITM2B,0.33045798493174583
-23217,RB1,0.25183231082954555
-23219,LPAR6,0.2221860349395595
-23220,RCBTB2,0.1480283049253099
-23225,CYSLTR2,0.127267081273152
-23227,FNDC3A,0.10080183863324987
-23229,CDADC1,-0.05556553581501588
-23231,SETDB2,0.03344720311669455
-23232,PHF11,-0.01036611950666961
-23233,RCBTB1,0.04128550524263277
-23234,ARL11,0.26306943061917376
-23235,EBPL,-0.10632461656837114
-23236,KPNA3,-0.013639090769669845
-23238,SPRYD7,-0.08087698186529266
-23239,DLEU2,0.24421159980781446
-23240,TRIM13,-0.0007962121772204151
-23242,DLEU1,0.017528602079513157
-23245,DLEU7,0.11258854154777677
-23247,RNASEH2B,0.04714351511104639
-23252,INTS6,0.06133030555488785
-23253,INTS6-AS1,-0.03921787669666852
-23254,WDFY2,0.2881841554423391
-23255,DHRS12,-0.012266835203142892
-23260,ALG11,-0.02809920182987339
-23264,NEK3,0.07748809888098687
-23267,VPS36,-0.07272875910624879
-23269,CKAP2,0.06307019017533026
-23273,SUGT1,-0.0008576930583994259
-23304,TDRD3,-0.03258979257579914
-23342,MZT1,-0.055202901314407804
-23343,BORA,0.024573431058617513
-23344,DIS3,-0.03521598910230758
-23345,PIBF1,-0.10623520495286609
-23349,KLF12,-0.275422491074582
-23351,LINC00402,-0.09164268445495052
-23357,TBC1D4,-0.0584872986283178
-23359,COMMD6,-0.26478422085593584
-23360,UCHL3,0.0065285031542474435
-23362,LMO7,-0.1045395037532416
-23371,KCTD12,0.5287730474559125
-23374,CLN5,0.002338548280737813
-23375,FBXL3,0.14835400947521704
-23378,MYCBP2,0.1709807041386299
-23384,SLAIN1,-0.10461923948671874
-23394,OBI1,-0.052789742488880735
-23396,RBM26,-0.027684360732798178
-23397,RBM26-AS1,-0.05747343244370152
-23467,TGDS,-0.01235925283377822
-23468,GPR180,-0.0021542155341851606
-23474,ABCC4,0.015098251619274795
-23478,DNAJC3-DT,-0.046459788231880565
-23479,DNAJC3,0.08549281578936115
-23481,UGGT2,0.17888263889890646
-23486,MBNL2,0.05576881264692654
-23489,RAP2A,-0.044031898068277425
-23495,IPO5,0.02344952679260517
-23501,STK24,0.1367443204360159
-23504,DOCK9,-0.14434611355709928
-23508,UBAC2,-0.12679306615982217
-23509,GPR18,-0.17334994854285957
-23510,GPR183,-0.12971396426043563
-23512,TM9SF2,0.3450050376442936
-23513,LINC01232,0.026176461062279586
-23518,CLYBL,-0.06187144476433837
-23527,PCCA,0.040341891862668074
-23530,GGACT,-0.006637730807733414
-23533,TMTC4,0.03353060055925764
-23545,TPP2,0.01401769600032713
-23550,TEX30,0.014357281377092383
-23552,BIVM,-0.06592914489961725
-23553,ERCC5,0.02076975757384117
-23571,ARGLU1,-0.1751502825301782
-23583,LIG4,-0.007581112983281692
-23584,ABHD13,0.12116458031746383
-23585,TNFSF13B,0.5554960368902753
-23597,IRS2,0.23987441380957214
-23609,RAB20,0.19792411164985482
-23611,NAXD,0.009111635443866282
-23612,CARS2,0.23775207825053454
-23613,ING1,0.060727887588383045
-23616,ANKRD10,-0.06005304192310985
-23617,ANKRD10-IT1,0.10322021382375064
-23622,ARHGEF7,0.04728262062811765
-23638,TUBGCP3,0.08884077421901877
-23641,AL139384.1,0.0980878826488004
-23642,ATP11A,0.3092050412615604
-23656,PCID2,-0.17387577007597838
-23657,CUL4A,0.10216308002250965
-23658,LAMP1,0.18166597184400227
-23663,DCUN1D2,0.0511790876019179
-23665,TMCO3,0.13381957374703415
-23668,TFDP1,0.04825121224806244
-23672,GAS6-AS1,-0.007973849028939526
-23673,GAS6,0.1256100792101876
-23681,RASA3,-0.16061745510072967
-23685,CDC16,0.06014004073957379
-23687,UPF3A,-0.17362891672357594
-23688,CHAMP1,0.0982028852029366
-23723,TTC5,-0.10943010822246442
-23726,CCNB1IP1,-0.09506541684816867
-23728,AL355075.4,0.14473172052506597
-23729,PARP2,-0.07378717072256276
-23731,TEP1,0.2095254536441842
-23733,OSGEP,-0.009887642174720185
-23735,APEX1,0.015537722173594157
-23736,PIP4P1,0.001312347296304576
-23737,PNP,-0.012081451077565343
-23746,RNASE4,0.1448955368911046
-23750,RNASE6,0.33247568303857705
-23759,RNASE2,0.3000753656990925
-23760,METTL17,-0.060374270872579315
-23763,NDRG2,0.0065247154384860066
-23770,ARHGEF40,0.25498177594323584
-23775,LINC00641,0.0159539691695258
-23776,HNRNPC,0.09608760347631559
-23778,SUPT16H,-0.0689748964951129
-23780,CHD8,0.0865803415549337
-23781,RAB2B,-0.041727648697638536
-23782,TOX4,-0.025430146875065333
-23783,METTL3,0.018394112644001953
-23856,TRDC,-0.4432270050942198
-23918,TRAC,-0.41779355214879693
-23921,DAD1,-0.34131032628538277
-23923,ABHD4,0.052067186339854084
-23926,OXA1L,0.24706276065851723
-23927,SLC7A7,0.5209382899485507
-23928,MRPL52,-0.019109396065519722
-23930,LRP10,-0.1276566299795672
-23932,RBM23,0.024959477245818148
-23934,PRMT5,0.026605467157413777
-23936,HAUS4,0.15892654940615852
-23940,PSMB5,-0.00548895605329621
-23943,ACIN1,0.09694403712957538
-23944,C14orf119,-0.11100546766031771
-23949,HOMEZ,-0.02819436096795833
-23950,PPP1R3E,-0.05713216517396781
-23951,BCL2L2,0.09953210109292
-23953,PABPN1,0.10414513424973768
-23962,NGDN,-0.0701355462975708
-23963,ZFHX2-AS1,0.04504668457500709
-23967,AP1G2,-0.012079312454778566
-23968,AL135999.1,0.1207600658287935
-23975,DHRS4-AS1,-0.024943259249550553
-23976,DHRS4,0.08800617365994265
-23977,DHRS4L2,0.15337394199847448
-23978,AL136419.3,-0.04223620781949071
-23982,PCK2,0.1566397098698699
-23983,DCAF11,0.056564048899384045
-23985,PSME1,-0.05238183519850339
-23986,EMC9,-0.006364582607511505
-23987,AL136295.2,0.02779406082446981
-23988,PSME2,0.21884556810496417
-23989,RNF31,0.022867907269680653
-23990,IRF9,-0.05455137289273327
-23991,REC8,0.01570316421049715
-23992,IPO4,-0.09234251179868919
-23994,TM9SF1,0.07573368949101048
-23996,AL136295.7,0.04149195264620275
-23998,CHMP4A,-0.11694504470158798
-23999,MDP1,-0.007357888186055674
-24000,NEDD8,-0.05488159422906632
-24001,GMPR2,0.0307204639933895
-24002,TINF2,-0.03782710120325198
-24004,RABGGTA,0.024218874248777812
-24005,AL096870.8,0.09431989606487816
-24007,DHRS1,-0.04310261727555816
-24008,NOP9,0.04921978442932396
-24009,CIDEB,0.10675228143134
-24011,LTB4R,0.2062196702481875
-24013,RIPK3,0.07900262510675554
-24017,KHNYN,0.06585628592986975
-24018,SDR39U1,0.0006458342697223737
-24023,GZMH,-0.5536619822167544
-24024,GZMB,-0.443754516694892
-24059,G2E3,0.04405012123423198
-24060,SCFD1,0.03788228944142783
-24064,STRN3,0.042975663209326054
-24065,AP4S1,-0.08218191189748812
-24066,HECTD1,0.04640434751622811
-24068,HEATR5A,0.15798719186122165
-24070,DTD2,-0.023792098873768423
-24072,NUBPL,-0.021579581506898284
-24079,ARHGAP5,-0.018478796545169658
-24089,SPTSSA,0.2236969301146083
-24090,EAPP,-0.0927887865127147
-24093,SNX6,0.005846249443486201
-24094,CFL2,-0.05194025467428462
-24095,BAZ1A,0.26033448101268303
-24096,AL121603.2,-0.030537098639451903
-24097,SRP54,0.003499377956158874
-24098,FAM177A1,-0.13064160500220684
-24099,PPP2R3C,0.016917404639837506
-24100,PRORP,-0.14873458093425296
-24101,PSMA6,0.04067559296744325
-24103,NFKBIA,0.14828573360451947
-24106,RALGAPA1,-0.027898588265523126
-24116,MBIP,-0.0315891961918352
-24142,SEC23A,0.09910995529021897
-24144,GEMIN2,-0.09855802332751235
-24145,TRAPPC6B,-0.036973124840162186
-24146,AL109628.2,-0.08604012141523898
-24148,PNN,-0.2163903501424646
-24149,MIA2,-0.017216719396031623
-24151,FBXO33,-0.05266369132560366
-24169,C14orf28,-0.0006024447428455648
-24171,KLHL28,-0.10919885495315615
-24172,TOGARAM1,-0.0515094315687257
-24175,PRPF39,-0.061215571196866554
-24176,FKBP3,-0.1532526320279989
-24177,FANCM,-0.02423848626489299
-24178,MIS18BP1,0.37994441900437137
-24191,RPS29,-0.4677588183229408
-24192,LRR1,0.047994518241196056
-24193,RPL36AL,-0.4264820701918506
-24194,MGAT2,0.09409094547021447
-24196,DNAAF2,-0.06271618201272719
-24199,KLHDC1,-0.10436183948512424
-24201,KLHDC2,-0.12494211137682637
-24202,NEMF,0.01565336746599139
-24203,AL627171.2,-0.1330674309029542
-24204,AL627171.1,-0.024404894263626027
-24205,ARF6,0.0481114749283949
-24206,LINC01588,-0.08062059852220588
-24207,VCPKMT,-0.09196301705141645
-24208,SOS2,0.19058191458072857
-24209,L2HGDH,-0.025018462833670315
-24210,DMAC2L,-0.0980513752185987
-24214,MAP4K5,-0.06409539149135572
-24217,SAV1,0.07683250134941223
-24219,NIN,0.21321270507923235
-24220,AL606834.2,-0.025350638479527704
-24224,PYGL,0.37561057542784254
-24231,TMX1,0.03947625842908451
-24242,GNG2,-0.32498401323320564
-24246,RTRAF,-0.0172790656177639
-24249,PTGDR,-0.4932800085432136
-24250,PTGER2,-0.22949150542548818
-24251,TXNDC16,0.049163541242125595
-24253,ERO1A,0.09950216356730868
-24254,AL133453.1,-0.11778181564402171
-24255,PSMC6,0.10443879200248346
-24256,STYX,0.05826602660390091
-24257,GNPNAT1,-0.09003589644911765
-24263,DDHD1,-0.1006331397421344
-24273,CNIH1,0.03402668508469494
-24275,GMFB,0.12231901361127269
-24276,CGRRF1,-0.06418919260861326
-24277,SAMD4A,0.2882838146052272
-24279,GCH1,0.29188713562053337
-24281,SOCS4,0.08818450499930111
-24282,MAPK1IP1L,0.11604955387915139
-24283,LGALS3,0.4902832599853588
-24287,FBXO34,0.013970256973195707
-24289,ATG14,-0.03032103196466237
-24292,KTN1,-0.003278183297914829
-24297,PELI2,0.19762428061665352
-24301,TMEM260,0.05573689892174415
-24311,EXOC5,0.07577879310995174
-24312,AP5M1,0.025118543672903886
-24315,NAA30,-0.09890282367965991
-24321,ACTR10,0.022483284223890624
-24322,PSMA3,-0.05244585534290409
-24324,PSMA3-AS1,0.051697372823453026
-24326,ARID4A,0.0027205948167370263
-24330,TIMM9,-0.010164854576848731
-24331,KIAA0586,-0.1348213849344729
-24334,DAAM1,0.047319117018554874
-24336,L3HYPDH,0.0019954337712946857
-24337,JKAMP,0.1489165863208888
-24339,RTN1,0.31846988896428263
-24343,PCNX4,-0.045981434072008984
-24344,DHRS7,-0.2197551806079215
-24346,PPM1A,0.026162842553936107
-24354,MNAT1,-0.16039195305539683
-24355,TRMT5,0.03231146949521868
-24356,SLC38A6,0.07796239915804207
-24357,PRKCH,-0.46501477191095125
-24359,AL359220.1,-0.1615944383504819
-24364,HIF1A,0.29594912182385563
-24366,SNAPC1,-0.0028165810257682975
-24376,PPP2R5E,0.06837658551326721
-24378,WDR89,-0.04555502092315619
-24380,AL136038.5,-0.09080614198654254
-24381,SGPP1,-0.03432451529836185
-24382,SYNE2,-0.4706948147746654
-24386,MTHFD1,0.0015335370544508572
-24389,ZBTB25,-0.08413483973932435
-24391,ZBTB1,-0.10109372925797455
-24396,PLEKHG3,-0.11928931299745797
-24398,CHURC1,-0.061304569098423536
-24401,FNTB,0.07805899776115822
-24403,MAX,0.06660754494530159
-24407,FUT8,-0.09681718369650931
-24415,GPHN,-0.002640979800932271
-24418,MPP5,-0.01822523267293198
-24420,ATP6V1D,0.04455376106814802
-24421,EIF2S1,-0.10724196144687363
-24423,TMEM229B,0.07854769432491258
-24425,PIGH,0.005009482201986922
-24428,VTI1B,0.10092486765925879
-24429,RDH11,0.011139754286568682
-24431,ZFYVE26,0.10897269843872842
-24432,RAD51B,-0.04123281329730689
-24439,ZFP36L1,0.2761897193142464
-24440,ACTN1,0.10802762567773079
-24442,DCAF5,-0.1152316451175439
-24448,ERH,-0.044211394509914466
-24449,SLC39A9,0.0697168567468582
-24454,SUSD6,0.1580146079006017
-24456,SRSF5,-0.28366889495198827
-24462,COX16,-0.040534027149561315
-24463,SYNJ2BP,0.010697242306934331
-24467,MED6,-0.11474965830086567
-24469,TTC9,-0.05243081345170204
-24475,PCNX1,-0.0044946596987970894
-24483,SIPA1L1,0.17942460790873052
-24492,DCAF4,-0.06741178952507909
-24493,ZFYVE1,0.03181452784717924
-24495,RBM25,0.02244228041166494
-24496,PSEN1,0.21589560326126395
-24500,NUMB,0.3884754215126958
-24501,AC005280.2,0.181565389789517
-24504,RIOX1,-0.025805316160678247
-24512,DNAL1,-0.017623242977451103
-24514,PNMA1,-0.09291989726278747
-24515,ELMSAN1,0.09796626842180327
-24520,ZNF410,0.010631161158520492
-24524,COQ6,-0.03569864144603815
-24525,ENTPD5,-0.0006777129960349319
-24527,ALDH6A1,0.023676640204071907
-24528,LIN52,-0.0798785430158817
-24530,ABCD4,0.005627243026549298
-24536,NPC2,0.45503343517912787
-24537,ISCA2,-0.11936970345983858
-24541,AREL1,0.06295478591646737
-24542,FCF1,-0.02568828067098305
-24543,YLPM1,0.046654249671624175
-24545,DLST,-0.026231362931345644
-24548,EIF2B2,-0.03132633708154661
-24550,MLH3,0.023742786697707033
-24551,ACYP1,-0.14066750603850947
-24553,NEK9,0.09326480137366087
-24555,TMED10,-0.10352288970538037
-24559,FOS,0.549191793045046
-24560,LINC01220,0.09579021966496895
-24562,JDP2,0.3089209876908251
-24563,BATF,-0.3087929618284475
-24565,FLVCR2,0.23062019420059401
-24566,TTLL5,0.042856662847958096
-24567,ERG28,-0.20032858128597708
-24569,IFT43,-0.08819279826642033
-24573,GPATCH2L,0.13600838538975404
-24579,VASH1,-0.017478452512383516
-24583,ANGEL1,0.010987282009322365
-24589,IRF2BPL,0.05484301831202878
-24590,AC007686.3,-0.09772881842456223
-24595,CIPC,0.07411224794856404
-24601,GSTZ1,0.10011912029243192
-24602,TMED8,0.10070917875256459
-24605,VIPAS39,-0.049815510268478504
-24606,AHSA1,-0.07079461110311731
-24608,SPTLC2,0.37599428560830656
-24609,ALKBH1,0.05946800749950338
-24610,SLIRP,0.08563652404935797
-24611,SNW1,-0.02969183638373312
-24613,ADCK1,0.02287344753400586
-24628,CEP128,-0.12885602097035967
-24632,GTF2A1,0.05867013728398205
-24636,SEL1L,0.15405362953228746
-24653,LINC02328,-0.06727656401253686
-24666,GALC,-0.01402169921246165
-24667,GPR65,-0.07284198692534044
-24679,ZC3H14,-0.0887604687510747
-24685,FOXN3,0.14977875692204515
-24694,TDP1,-0.06998459137922512
-24696,PSMC1,-0.02912030974031472
-24697,NRDE2,-0.003252318096075279
-24700,CALM1,-0.6217863472376204
-24712,RPS6KA5,-0.004063739706309966
-24713,DGLUCY,0.06202495164884017
-24714,GPR68,-0.2386198759636363
-24718,CCDC88C,-0.14298694469313608
-24720,PPP4R3A,0.09295066196030864
-24722,AL121839.2,-0.13675827781705774
-24723,TC2N,-0.2990860099702903
-24724,FBLN5,-0.14187232036897665
-24725,TRIP11,-0.056979595166694895
-24726,ATXN3,0.1429843883969752
-24727,NDUFB1,0.05848675786207022
-24728,CPSF2,0.22284135104794714
-24731,SLC24A4,0.3131347650849459
-24732,RIN3,0.179372006580688
-24734,GOLGA5,0.02786919216274897
-24738,ITPK1,0.09685347797586809
-24740,MOAP1,-0.15253927327305988
-24741,TMEM251,-0.02063006374356418
-24742,GON7,-0.12020447708395575
-24743,UBR7,-0.04059090403656086
-24744,BTBD7,0.09150825355196623
-24755,DDX24,-0.22594684588882158
-24756,IFI27L1,-0.021990202176041106
-24758,IFI27L2,0.0006536865170978967
-24762,SERPINA1,0.581038044877951
-24777,DICER1,0.36964056257268546
-24779,CLMN,0.26552431924276226
-24782,SYNE3,0.1135773441956131
-24784,SNHG10,-0.124279507745358
-24785,GLRX5,-0.06547205229047051
-24791,TCL1A,-0.005460200500739307
-24802,ATG2B,0.024352003229613464
-24803,GSKIP,-0.017620462731507563
-24806,PAPOLA,0.16036762932659954
-24811,VRK1,-0.046024366231067565
-24823,LINC01550,-0.1237689250696923
-24832,BCL11B,-0.3528237166713455
-24835,SETD3,-0.010585988694159021
-24836,CCNK,0.11513790614162864
-24844,EVL,-0.486154955423475
-24849,YY1,0.07469029805700765
-24852,SLC25A29,0.022423075685010747
-24856,WARS,0.4793037820785906
-24858,WDR25,0.03850603141955708
-24873,AL132709.8,0.15223582595183993
-24888,PPP2R5C,-0.34623470134053064
-24891,AL118558.3,0.04306768954840655
-24893,DYNC1H1,0.13557704259544606
-24894,HSP90AA1,-0.09521162064347231
-24895,WDR20,0.051755669373016115
-24897,ZNF839,-0.06829000833049077
-24898,CINP,0.015018772483321759
-24899,TECPR2,0.06335215575622612
-24902,RCOR1,0.25949352068966214
-24904,TRAF3,0.06104826009042491
-24906,CDC42BPB,0.1826656175444856
-24912,TNFAIP2,0.5357083197902209
-24916,EIF5,-0.0015985867785226924
-24917,MARK3,0.07280208115302365
-24918,CKB,0.09635212768494683
-24920,TRMT61A,-0.018993381796098108
-24922,BAG5,-0.05148794513952124
-24923,KLC1,0.014285874652697638
-24924,COA8,-0.03591622280150519
-24925,AL049840.2,-0.14334589306323317
-24928,AL049840.4,0.08410092819786949
-24929,AL049840.5,0.027423885805232265
-24931,ZFYVE21,0.001465979229785489
-24936,ATP5MPL,0.15466098941384754
-24951,INF2,0.05533925514473654
-24954,SIVA1,-0.09412278780943635
-24956,AKT1,0.07731147095761819
-24963,PLD4,0.08139557289161403
-24966,CDCA4,-0.08462345668540229
-24967,GPR132,0.0422285440797503
-24972,NUDT14,0.019272454824882886
-24973,BRF1,0.06718654163746812
-24974,BTBD6,-0.0570401214224055
-24975,PACS2,0.008087848174612947
-24978,MTA1,-0.008605931340664048
-24980,CRIP2,-0.06142817295386809
-24981,CRIP1,-0.1173329045429076
-24982,TEDC1,0.03283532993584841
-24990,IGHA1,0.001376310731389642
-24992,IGHG1,0.0041913495631793075
-24994,IGHD,0.0032296670779140314
-24995,IGHM,0.005060546986489158
-25032,FAM30A,0.0025524104207324816
-25215,NIPA2,0.027934432707054254
-25216,CYFIP1,0.3222511710087893
-25217,TUBGCP5,0.017728734015281983
-25234,SNRPN,-0.19075504002690963
-25237,SNHG14,-0.1239570388719663
-25241,UBE3A,0.02551487167465503
-25244,ATP10A,-0.04349063225670071
-25268,HERC2,-0.08517565722159179
-25275,APBA2,-0.17199831238104465
-25278,NSMCE3,-0.19166050809171878
-25293,AC127502.2,-0.0404864598698089
-25302,FAN1,0.014190491568181383
-25303,MTMR10,0.21061012447084468
-25308,KLF13,-0.16578920284061924
-25313,GOLGA8O,-0.08223099116495437
-25314,LINC02256,0.05339475941745256
-25324,GREM1,-0.023037896832709032
-25325,FMN1,0.11913416512244093
-25331,AVEN,-0.08705682772038617
-25335,EMC7,-0.1544096652928877
-25336,PGBD4,-0.04813454209955951
-25337,KATNBL1,-0.061717371142200654
-25338,EMC4,-0.09332504603912777
-25339,SLC12A6,0.19500926154263207
-25341,NOP10,0.01793109496657908
-25343,LPCAT4,-0.010230140912476106
-25344,GOLGA8A,-0.09786148155408259
-25345,GOLGA8B,-0.14553969689655824
-25351,AQR,0.11311511906943271
-25352,ZNF770,0.057614473989943724
-25357,DPH6,-0.02658173319175746
-25374,SPRED1,0.17904893900946445
-25375,FAM98B,0.01540378820199794
-25376,RASGRP1,-0.28464135573336297
-25377,AC116158.3,-0.038560520431531596
-25390,THBS1,0.23469365859946922
-25398,EIF2AK4,0.13745700229987406
-25399,SRP14,-0.31992003608316194
-25400,SRP14-AS1,-0.04879158841586476
-25403,BMF,0.16846648966470182
-25410,PLCB2,0.26960157492678494
-25412,AC020658.5,0.025977511665124024
-25415,INAFM2,0.09743624715710102
-25420,KNSTRN,0.023742936265745983
-25421,IVD,0.015079512947701922
-25422,BAHD1,0.014487908483166142
-25425,CHST14,-0.03883218612264312
-25427,CCDC32,0.014913390553927316
-25428,RPUSD2,-0.07696488949894478
-25431,RAD51-AS1,-0.10299959484550524
-25433,RMDN3,-0.09259728096534948
-25434,GCHFR,-0.18341057671572045
-25435,DNAJC17,-0.00869257384059681
-25437,ZFYVE19,-0.045085742123413396
-25440,SPINT1,0.1449224008317388
-25443,VPS18,0.10686111198018466
-25447,INO80,-0.03730764487312162
-25453,CHP1,0.3187333121546876
-25454,OIP5-AS1,-0.036431708820879975
-25458,NDUFAF1,-0.02214676175802048
-25459,RTF1,-0.008276065519951345
-25462,RPAP1,0.03580469530934378
-25465,MGA,0.02759058431749396
-25467,MAPKBP1,0.01144535631897271
-25469,JMJD7,-0.022325094313575974
-25470,PLA2G4B,0.049968858883198644
-25472,AC020659.1,-0.10602264510561561
-25473,EHD4,0.08286183311387428
-25481,VPS39,0.1023002538584113
-25483,TMEM87A,0.016350739763664385
-25484,GANC,0.12701897485909566
-25485,CAPN3,0.1059600702897305
-25486,ZNF106,0.385054602471033
-25487,SNAP23,0.2576641622702352
-25488,AC018362.1,0.029809226725046347
-25489,LRRC57,0.039881926810299555
-25490,HAUS2,0.010781418024107526
-25493,CDAN1,0.04669589362995065
-25496,TTBK2,0.08678647703716498
-25499,UBR1,-0.03542311829434252
-25501,TMEM62,-0.02132728466677624
-25503,CCNDBP1,0.0027926571943871423
-25509,ADAL,-0.04862848351660725
-25510,ZSCAN29,0.02701800793307928
-25511,TUBGCP4,-0.028918811196153433
-25512,TP53BP1,-0.025846387625162906
-25517,CATSPER2,-0.051481594740372456
-25520,PDIA3,-0.02751398042918921
-25522,SERF2,0.0916615051200225
-25524,HYPK,-0.0009051630066517335
-25525,MFAP1,-0.14992666742650768
-25526,WDR76,-0.15807328671809506
-25528,CASC4,0.006491123758753707
-25531,CTDSPL2,-0.04710950735907245
-25533,EIF3J-DT,-0.0899957447528574
-25534,EIF3J,0.006443977178074167
-25536,SPG11,0.25257256794931854
-25537,PATL2,-0.24214041618187768
-25538,B2M,-0.5369649636930913
-25539,TRIM69,0.020493319247339253
-25543,SORD,0.09750271511380622
-25560,SPATA5L1,0.04552409377186406
-25567,BLOC1S6,0.20250513370383122
-25568,SQOR,0.35006613386269075
-25592,DUT,-0.06332858899365397
-25598,CEP152,0.07672818161990078
-25603,EID1,0.02690826703473762
-25605,SECISBP2L,0.04718164326938551
-25607,COPS2,0.014690242075485612
-25608,GALK2,0.05384795726616768
-25613,DTWD1,0.045406827506057935
-25614,ATP8B4,0.06996144984429967
-25618,GABPB1,0.0032460634848345796
-25619,GABPB1-IT1,-0.15241532089575804
-25620,GABPB1-AS1,-0.061411300902162666
-25622,USP8,0.2004038788679323
-25626,TRPM7,-0.0014238281658055085
-25628,SPPL2A,0.214549282881397
-25631,AP4E1,0.07403695283242552
-25641,DMXL2,0.47586571599475047
-25646,LYSMD2,0.1497837175676061
-25647,TMOD2,0.15327913571059265
-25649,TMOD3,0.1498070893295176
-25653,LEO1,-0.16149200172113842
-25654,MAPK6,0.07853414653106013
-25661,GNB5,-0.046593468094457316
-25666,MYO5A,0.21181173434378467
-25667,ARPP19,-0.05322348465233663
-25669,FAM214A,-0.03111477189162768
-25681,RSL24D1,-0.11392899239216694
-25682,RAB27A,0.11455818744277363
-25684,PIGBOS1,0.025617944086838866
-25685,PIGB,0.00016309917482848697
-25686,CCPG1,0.1508790683984243
-25695,NEDD4,-0.055991395533003106
-25696,RFX7,0.011112356879408512
-25704,ZNF280D,0.043843438086042005
-25709,TCF12,0.07762663268563225
-25710,LINC00926,0.0023857535011409386
-25718,POLR2M,0.024021607732555794
-25723,AQP9,0.2179631719343011
-25730,ADAM10,0.19986360362144145
-25733,MINDY2,0.0063282890944537816
-25736,RNF111,0.10687735443781166
-25737,SLTM,0.02667933877365067
-25742,MYO1E,0.08024060350518318
-25748,GTF2A2,-0.06404966218096529
-25750,BNIP2,0.263017503807131
-25756,ANXA2,0.4282615176972279
-25757,ICE2,-0.07216600376750355
-25759,RORA,-0.4025895588690783
-25775,VPS13C,0.20039518298161219
-25790,TPM1,0.0011211601356654213
-25794,LACTB,0.34248906388506695
-25795,RPS27L,-0.11763880024758716
-25796,RAB8B,0.026282353473929904
-25797,APH1B,0.12486602948808544
-25802,USP3,0.320004025856026
-25805,HERC1,0.06021797434355575
-25809,CIAO2A,0.19104471777451026
-25810,SNX1,0.15642617899387493
-25811,SNX22,0.01623217532536695
-25812,PPIB,-0.2478568138999639
-25813,CSNK1G1,0.016251061842784256
-25815,TRIP4,-0.04284327373613202
-25816,ZNF609,-0.029326206183571873
-25818,OAZ2,0.23840218923472625
-25824,PLEKHO2,0.13142929348387103
-25828,AC103691.1,0.061277683233824085
-25829,SPG21,0.27936838848809376
-25830,MTFMT,-0.0974363541543447
-25837,PDCD7,-0.10052989796824276
-25838,CLPX,0.03778143686127174
-25840,PARP16,-0.11869249885728729
-25843,DPP8,0.07562170198787156
-25844,HACD3,-0.16629170893136444
-25845,INTS14,-0.0247590957608781
-25848,DENND4A,0.0781085504949888
-25849,RAB11A,0.03072150799749508
-25850,AC011939.2,0.08758387088846523
-25853,DIS3L,0.010777806637033859
-25855,TIPIN,-0.06917950579769504
-25856,MAP2K1,0.12433656272418275
-25858,SNAPC5,-0.07504902752703872
-25859,RPL4,-0.015811692471643872
-25860,ZWILCH,-0.027719112846320358
-25869,SMAD3,0.04588156272862513
-25871,AAGAB,-0.0470386973869125
-25874,C15orf61,-0.09380897395834203
-25876,MAP2K5,0.035625245232437076
-25880,PIAS1,0.17840295123884517
-25881,CALML4,0.20001378420585925
-25882,CLN6,0.06376404389018368
-25884,FEM1B,0.05388742145403629
-25888,ANP32A,0.103827980614209
-25896,GLCE,0.13398920055854277
-25903,RPLP1,-0.0911501595211071
-25908,TLE3,0.16452726110897173
-25916,UACA,0.08228959777006828
-25931,MYO9A,-0.1013865850641592
-25936,PKM,0.30944265670238696
-25937,PARP6,0.013677384741234814
-25940,HEXA,0.10570147564824604
-25945,ARIH1,0.08753376858900491
-25951,BBS4,-0.024868928348049543
-25952,ADPGK,0.255627787894761
-25960,NPTN,0.22693960561699264
-25970,STOML1,-0.0003294988097500927
-25971,PML,0.18578631800283502
-25988,UBL7,0.08920572657934524
-25989,UBL7-AS1,-0.07747750209535935
-25991,ARID3B,-0.007675038944442213
-25992,CLK3,-0.02990800643123474
-25995,EDC3,-0.0034969616279444432
-25998,CSK,0.12396815588228209
-26001,ULK3,-0.05433234901933931
-26002,SCAMP2,0.058705693330581295
-26003,MPI,-0.01455945555993314
-26004,FAM219B,0.038418686159268654
-26005,COX5A,0.014693309593388559
-26006,RPP25,0.05896642854297799
-26008,PPCDC,0.14169254142789986
-26009,C15orf39,0.3047362099810903
-26014,COMMD4,-0.09180774276135888
-26016,NEIL1,-0.12847189450015656
-26017,MAN2C1,0.1258768713563349
-26019,SIN3A,0.07023347590473919
-26021,PTPN9,0.11525482319742088
-26022,SNUPN,-0.08480373178043074
-26025,IMP3,-0.19592828866654874
-26036,UBE2Q2,-0.14377818387888505
-26037,FBXO22,-0.004914121694427078
-26041,ETFA,0.037989328533312994
-26046,SCAPER,0.025049913956425682
-26047,RCN2,-0.17408693680778498
-26048,PSTPIP1,0.08475086975602363
-26049,TSPAN3,-0.09711978271389377
-26052,PEAK1,0.01829479037262229
-26053,HMG20A,-0.04904934549272156
-26062,TBC1D2B,0.11753970981388273
-26066,IDH3A,0.1547979234137336
-26070,DNAJA4,-0.010211102060169393
-26071,WDR61,0.02409409333603322
-26076,IREB2,0.10828317316594512
-26079,PSMA4,0.3695954885424227
-26086,MORF4L1,0.03053538045694253
-26087,CTSH,0.4383468841234031
-26092,TMED3,-0.04255579007385585
-26096,MTHFS,0.05819558578519993
-26098,AC015871.7,0.022041249556681474
-26099,AC015871.1,0.06464208748197245
-26100,AC015871.4,0.072235849219462
-26101,ST20-AS1,0.11858051843683985
-26102,BCL2A1,0.21590440434575525
-26103,ZFAND6,-0.04330763277303298
-26104,FAH,0.08825473852441844
-26119,MESD,-0.08321713503579957
-26122,TLNRD1,0.04466067430834274
-26124,IL16,-0.13042800768679358
-26138,EFL1,0.07190623119262693
-26143,RPS17,0.0848281922187431
-26150,SNHG21,-0.04099893950596842
-26152,WHAMM,-0.03948280149928515
-26155,RAMAC,-0.03371514257238476
-26156,C15orf40,-0.007927958838703633
-26158,BTBD1,0.09245666471036927
-26163,TM6SF1,0.11217570611620169
-26180,WDR73,-0.10261529761646358
-26182,SEC11A,0.1483345807058228
-26185,ZNF592,0.07029721770595893
-26190,PDE8A,0.13154111494064347
-26191,AKAP13,0.14898660155401994
-26213,MRPL46,-0.06248585457546668
-26214,MRPS11,-0.016079301471821506
-26215,DET1,-0.025775492012712915
-26217,AEN,-0.05128588576196223
-26218,ISG20,-0.25374779014362225
-26221,HAPLN3,0.010534002349670429
-26225,ABHD2,0.19307974273346643
-26228,POLG,0.08574526717709359
-26229,AC124068.2,-0.01536183778947383
-26245,ANPEP,0.3202607699498158
-26246,AP3S2,0.10748200096800493
-26248,ZNF710,0.19811830335442854
-26249,AC087284.1,0.13770321784889575
-26251,IDH2,-0.12766544731647134
-26253,SEMA4B,0.1513872123780156
-26254,CIB1,-0.19188568439701803
-26257,NGRN,-0.10915791346905572
-26260,IQGAP1,0.37135157823589643
-26262,CRTC3,-0.002318832194335914
-26269,BLM,-0.03236295373195504
-26272,FURIN,0.13295971883130933
-26273,FES,0.3650120843889582
-26274,MAN2A2,0.10495235081772805
-26276,HDDC3,-0.08798281389619844
-26277,UNC45A,-0.005671814228487222
-26279,RCCD1,0.0602647824159408
-26282,VPS33B,-0.018178873932089105
-26291,SLCO3A1,0.19637600013585446
-26306,LINC01578,-0.044120905350981426
-26307,CHD2,-0.06303028992119396
-26320,MCTP2,-0.27273199276518834
-26359,ARRDC4,0.16536727316292393
-26370,IGF1R,-0.04308167431129211
-26382,LRRC28,-0.013358137002473515
-26388,MEF2A,0.25953412473334025
-26400,LINS1,-0.09214308779543837
-26401,ASB7,0.0588098451738566
-26408,LRRK1,0.21632433373192586
-26414,CHSY1,0.17441780073622204
-26415,SELENOS,-0.07201641306754449
-26416,SNRPA1,-0.17847528790513068
-26424,TM2D3,-0.06354123081186865
-26425,TARSL2,-0.09331209888362138
-26432,POLR3K,-0.06676775426244724
-26433,SNRNP25,-0.07488470820752127
-26435,MPG,-0.0008920081042516031
-26436,NPRL3,-0.17482757805491986
-26445,LUC7L,0.015282549943118773
-26446,FAM234A,0.048967717643651945
-26450,AXIN1,-0.1121454649116017
-26451,MRPL28,0.05710073507886904
-26452,TMEM8A,0.035928753125324876
-26453,NME4,-0.040227091139414486
-26458,CAPN15,-0.07872829879736021
-26461,PIGQ,0.004195932696578058
-26463,RAB40C,0.06113232337576417
-26465,METTL26,-0.06878881388262403
-26466,MCRIP2,0.07616763831063719
-26470,RHOT2,0.007916273535858534
-26473,STUB1,-0.13905531009133731
-26474,JMJD8,0.03285280994940569
-26475,WDR24,0.007452709400800616
-26479,METRN,-0.17997501534057184
-26480,ANTKMT,-0.11836630159542957
-26482,HAGHL,-0.08482764829390427
-26483,CIAO3,0.01083876216951363
-26486,RPUSD1,-0.025955090718147743
-26491,LMF1,0.0029984194516948157
-26496,AC009041.2,-0.05589689120309244
-26498,AC009041.1,-0.08242567756759402
-26517,UBE2I,-0.010186275827846846
-26520,TSR3,-0.0898795650429188
-26521,GNPTG,-0.01023735632020332
-26523,UNKL,0.026178040415071813
-26525,C16orf91,-0.11636499007677173
-26528,CLCN7,0.16358433009652718
-26533,TELO2,0.07645204389996005
-26536,TMEM204,-0.14908527163574206
-26539,CRAMP1,0.07016508473922502
-26541,JPT2,-0.15325915889815667
-26542,MAPK8IP3,-0.01545777465052421
-26546,NME3,-0.13105094375248752
-26547,MRPS34,-0.09319603963345797
-26548,EME2,0.1122700827393825
-26549,SPSB3,-0.07422458005851354
-26550,NUBP2,-0.11792470400472053
-26552,HAGH,-0.041709200851220596
-26553,FAHD1,0.020074703063081213
-26559,MSRB1,0.3858030726535792
-26561,NDUFB10,-0.04670761577312382
-26562,RPS2,-0.3120828307523026
-26563,SNHG9,-0.10722316161637811
-26566,TBL3,-0.044782929274875184
-26568,GFER,-0.09077026756594714
-26572,ZNF598,0.021614339713359434
-26575,NTHL1,-0.047688529952330826
-26576,TSC2,0.0895012370271144
-26577,PKD1,0.01776614326119405
-26583,SNHG19,-0.060633240018411805
-26584,TRAF7,0.11971830271879062
-26586,MLST8,0.054891898052666546
-26588,PGP,-0.013861895380423848
-26590,E4F1,-0.03811011041269553
-26593,ECI1,0.07497399182471054
-26595,RNPS1,-0.013522950474164889
-26596,AC009065.4,0.007152854536643655
-26605,TBC1D24,-0.042958962985864636
-26607,ATP6V0C,0.4250653520470912
-26608,AMDHD2,-0.007999593224881389
-26610,PDPK1,0.1482892649729168
-26614,AC093525.4,0.0874450904089583
-26618,ERVK13-1,0.012135658562587824
-26619,KCTD5,0.026685720333433267
-26622,SRRM2,0.054229352206989226
-26623,ELOB,0.048127618017754135
-26628,ZG16B,0.1291861429798555
-26632,FLYWCH2,-0.019756582227152743
-26633,FLYWCH1,-0.018300631288732837
-26645,HCFC1R1,0.07847731626134329
-26646,THOC6,0.0698816294220081
-26649,MMP25-AS1,-0.2631575182135891
-26650,MMP25,0.09039181603871817
-26651,IL32,-0.5992539811341191
-26652,AC108134.3,0.1615358425390794
-26655,ZNF213-AS1,-0.07807359999053319
-26657,ZNF213,0.05092359019460287
-26658,AC108134.2,-0.026021780181178948
-26662,ZNF200,-0.033686296433250884
-26663,MEFV,0.3799120279170299
-26664,LINC00921,-0.1587147717929219
-26665,ZNF263,-0.04868272917161979
-26666,TIGD7,0.004385518098368692
-26667,ZNF75A,-0.10702121301305495
-26672,ZSCAN32,-0.0055247293746679095
-26673,ZNF174,-0.027597446884745507
-26675,NAA60,0.1102682227991141
-26677,CLUAP1,-0.12837129440053074
-26678,NLRC3,-0.2330906642365416
-26681,SLX4,0.06236023260585784
-26682,DNASE1,0.02275852565361759
-26684,TRAP1,0.023660330134081836
-26686,CREBBP,0.14744910805188105
-26688,ADCY9,0.036972917387534006
-26693,TFAP4,-0.038624774486344035
-26696,PAM16,-0.010809719504095308
-26698,CORO7,0.05095123673416621
-26700,DNAJA3,-0.07255522242114028
-26703,NMRAL1,-0.029151670737215755
-26704,HMOX2,-0.3328433651191357
-26705,CDIP1,-0.029267611141778938
-26709,UBALD1,-0.009620371515636737
-26710,MGRN1,0.19398324017709573
-26712,NUDT16L1,-0.15979962362865083
-26713,ANKS3,-0.07956367097566586
-26715,ZNF500,-0.002360588337711178
-26719,ROGDI,0.2766780575989286
-26720,GLYR1,0.20366154068900927
-26722,UBN1,0.03518892194748477
-26726,NAGPA,0.02658536187913145
-26727,ALG1,0.018989735423042768
-26729,EEF2KMT,-0.03867785049092098
-26748,METTL22,0.14773094973825798
-26749,ABAT,0.10545816519205203
-26750,TMEM186,-0.11004248401401107
-26751,PMM2,0.028007549242462067
-26754,CARHSP1,0.006920953670035123
-26758,USP7,0.16997434551352425
-26761,C16orf72,0.0666935421889067
-26763,AC087190.1,-0.12991838733529767
-26775,ATF7IP2,-0.09196570208353415
-26783,NUBP1,0.09384273798892313
-26786,CIITA,0.3903328492129584
-26787,DEXI,-0.24678130293007844
-26788,CLEC16A,0.027337152894568378
-26794,SOCS1,-0.15871024643990203
-26804,LITAF,-0.23102400679289659
-26805,SNN,0.2070736827673547
-26806,TXNDC11,0.09816462926333296
-26807,AC007613.1,-0.03758757586267762
-26808,ZC3H7A,0.1641058083946313
-26809,AC010654.1,0.1137897479619533
-26811,RSL1D1,0.01924568444711439
-26813,GSPT1,0.036635090831496894
-26818,SNX29,0.07353281044506299
-26828,CPPED1,0.49125612227272725
-26836,ERCC4,0.0009258669276089052
-26841,MRTFB,-0.02767515523720865
-26847,PARN,0.019363533550117728
-26848,BFAR,-0.04964169320512445
-26854,NOMO1,0.04640240716593706
-26858,PDXDC1,0.02461832394134021
-26860,NTAN1,0.015382909195192173
-26861,RRN3,-0.06860756611842493
-26866,MARF1,0.08662461580034163
-26868,NDE1,0.03349279821735798
-26870,AC026401.3,0.09152487439970243
-26871,MYH11,-0.1074381922675661
-26874,FOPNL,-0.014145817918515054
-26876,ABCC1,0.10857367963206861
-26879,NOMO3,0.045931666007088624
-26886,XYLT1,0.06960485750098687
-26901,RPS15A,-0.4809170298218713
-26902,ARL6IP1,-0.18911846050802067
-26904,SMG1,0.14139000935115792
-26909,COQ7,0.027106517028787042
-26912,ITPRIPL2,0.3151937448710807
-26913,AC099518.5,0.06044685591530981
-26924,GDE1,0.0735668716232841
-26925,CCP110,0.012932847745804714
-26926,VPS35L,0.06374441866257355
-26928,KNOP1,-0.06284192116319848
-26946,THUMPD1,-0.05763705409703946
-26948,ERI2,0.1266586875958378
-26950,DCUN1D3,0.07162749487257927
-26951,LYRM1,-0.013198128538330191
-26954,TMEM159,0.023364461984510546
-26962,METTL9,0.275554298876023
-26963,AC005632.3,0.27327766150015703
-26964,IGSF6,0.553611927171271
-26967,NPIPB4,0.00452826017902411
-26969,UQCRC2,0.06072462659657088
-26972,MOSMO,0.008595852891854563
-26975,SDR42E2,0.024665376278127676
-26977,EEF2K,-0.061627118048231296
-26979,POLR3E,-0.077918973655638
-26980,AC092338.3,-0.0779391431932411
-26981,CDR2,-0.1029495957177935
-26984,NPIPB5,0.06678516433383769
-26991,USP31,0.09215444450207937
-26996,COG7,0.031856323235905135
-26999,GGA2,-0.018629095583677948
-27001,UBFD1,0.0009707882064210534
-27003,NDUFAB1,-0.0988707815880854
-27004,PALB2,-0.01923005721917182
-27006,DCTN5,0.08687220833952551
-27013,PRKCB,0.29439188538385486
-27017,RBBP6,0.10173644299236
-27018,TNRC6A,-0.018137037731782792
-27022,ARHGAP17,0.17834821472663157
-27026,LCMT1,-0.020002904947697533
-27031,ZKSCAN2-DT,0.040933392736454674
-27044,NSMCE1,-0.12586010300040873
-27047,IL4R,0.12519519940609736
-27048,IL21R,-0.14987755268005967
-27050,GTF3C1,-0.08943956885286104
-27051,KIAA0556,0.025639766845348497
-27056,XPO6,0.07415834497605289
-27058,SBK1,-0.2264098987740095
-27064,CLN3,0.10149380448182926
-27065,APOBR,0.2703050446156773
-27066,IL27,0.138904447512716
-27069,SGF29,-0.09866063213809986
-27071,SULT1A1,0.2521539450630462
-27073,EIF3C,-0.048056581025733514
-27077,AC145285.6,0.08076745509437741
-27078,ATXN2L,0.0018015215324526108
-27080,TUFM,0.020326717261169136
-27081,SH2B1,-0.016346496655757416
-27085,RABEP2,-0.0037281109533286298
-27086,CD19,0.002737580656794063
-27087,NFATC2IP,0.05997897091984059
-27090,SPNS1,0.1180126550802935
-27091,LAT,-0.3216148372783685
-27093,AC009093.2,0.10353696891381388
-27099,BOLA2,0.05693353248001038
-27103,SPN,-0.12548630452637383
-27105,C16orf54,-0.1268518966205119
-27108,KIF22,0.0036715270831419356
-27110,MAZ,-0.11342720026908526
-27112,AC009133.1,-0.022629878026831158
-27114,PAGR1,-0.0385842770220981
-27115,MVP,0.29745289538635816
-27116,CDIPT,-0.027762825545286182
-27120,KCTD13,-0.01608985479249315
-27122,TMEM219,0.10707916617667887
-27123,TAOK2,0.019349658236987817
-27124,HIRIP3,-0.09674185449542423
-27125,INO80E,-0.041659761557944375
-27130,ALDOA,0.1112685777199023
-27132,PPP4C,0.09507573164685157
-27134,YPEL3,-0.015357967886082916
-27139,MAPK3,0.13372580376097642
-27140,CORO1A,-0.1894589862833235
-27141,AC012645.3,-0.004753114076037877
-27142,BOLA2B,-0.005537531223348526
-27144,SULT1A3,0.018258998048317838
-27146,CD2BP2,-0.12877864399659208
-27148,TBC1D10B,0.087204369368159
-27150,MYLPF,-0.10258340019614348
-27151,ZNF48,-0.09343272630054196
-27152,SEPTIN1,-0.2743122238441577
-27154,DCTPP1,-0.08826959042691997
-27155,SEPHS2,0.014538549179987337
-27156,ITGAL,-0.06477847513239135
-27160,ZNF768,0.07415240447728648
-27161,ZNF747,0.03982498197470129
-27162,AC002310.1,-0.0688653766168248
-27163,ZNF764,-0.1307125914046994
-27164,ZNF688,0.004610792635558847
-27166,ZNF785,0.016201651312839245
-27168,ZNF689,0.010559769634751345
-27169,PRR14,0.0850385535900256
-27170,FBRS,0.0034774587798551563
-27172,SRCAP,0.017311956322510452
-27173,TMEM265,0.03559613042767861
-27174,PHKG2,-0.020889487672112107
-27176,RNF40,0.08522057922833999
-27177,ZNF629,0.11907686219643084
-27179,BCL7C,-0.17483132298406487
-27180,MIR762HG,-0.034632825915316724
-27184,FBXL19,0.03859163423427806
-27185,ORAI3,0.09944367802901938
-27187,SETD1A,0.025128291089612325
-27190,STX4,-0.028023637516444763
-27193,ZNF668,-0.0273223792011297
-27195,ZNF646,0.04028621741099114
-27197,VKORC1,0.010369966679856634
-27198,BCKDK,0.26517074622434905
-27199,KAT8,0.023363853176432407
-27201,AC135050.6,0.11753508253235335
-27205,FUS,-0.2046418644637964
-27208,PYCARD,0.38801408830229034
-27209,PYCARD-AS1,0.009909161505865094
-27212,ITGAM,0.24471222843639823
-27213,ITGAX,0.2660869909436584
-27220,AC026471.1,-0.0017559471333466622
-27221,ARMC5,0.05744057492825251
-27225,C16orf58,-0.004783310981821651
-27233,ZNF720,-0.20068374871786182
-27235,ZNF267,0.12311847978108505
-27300,VPS35,0.19101318161326464
-27304,C16orf87,0.05152711434475544
-27306,DNAJA2,-0.0013109693763321624
-27309,NETO2,0.19504470383834774
-27311,ITFG1,0.07580614954563736
-27314,PHKB,0.16806534673936763
-27323,LONP2,0.00618778966498193
-27324,SIAH1,0.020788227956047917
-27326,N4BP1,0.002804522538507819
-27344,CNEP1R1,0.005316351509095013
-27346,HEATR3,0.16323009296873647
-27349,TENT4B,0.11525950152336596
-27350,ADCY7,0.1922512190818673
-27351,BRD7,0.04870762676948827
-27359,SNX20,0.14068922450875684
-27360,NOD2,0.3120451976711432
-27362,AC007728.2,0.06838087325413517
-27363,CYLD,-0.05589543589633147
-27377,HNRNPA1P48,-0.024702226365985753
-27393,CHD9,0.04692989713178436
-27397,RBL2,-0.20929937133430557
-27399,AKTIP,-0.12948518096613057
-27403,FTO,0.0479170727016732
-27430,LPCAT2,0.3980496478955719
-27431,AC007336.1,0.1459468133370059
-27434,CES1,0.24899232327234105
-27443,AMFR,0.24240466387021833
-27445,NUDT21,0.009580268625292153
-27446,OGFOD1,-0.01556632853916149
-27448,BBS2,-0.02866988617461374
-27451,MT2A,0.04682989686340158
-27453,MT1E,-0.10787346053805776
-27457,MT1F,-0.17031917784078765
-27460,MT1X,-0.05106072792113603
-27462,NUP93,0.07631951922889105
-27464,HERPUD1,0.1746939600920673
-27466,AC012181.1,0.16074200492116933
-27468,NLRC5,0.036586606429117326
-27471,CPNE2,0.10372313339754277
-27472,FAM192A,0.05519392422023706
-27474,RSPRY1,0.041750268475965586
-27475,ARL2BP,-0.025299883520478322
-27482,CIAPIN1,-0.033857403346138085
-27483,COQ9,0.020777066685910655
-27484,POLR2C,-0.13454827248363335
-27487,ADGRG5,-0.24569912679618
-27488,ADGRG1,-0.5529476251190266
-27493,KATNB1,0.06917141104286284
-27499,ZNF319,-0.1333372904059314
-27500,USB1,0.07367441162241628
-27502,CFAP20,-0.09065733907530249
-27504,AC009107.2,-0.1374440064039286
-27505,CSNK2A2,0.02701142001075578
-27514,SETD6,-0.06756680971187969
-27515,CNOT1,0.08393853094715213
-27517,SLC38A7,0.11311580278710898
-27518,GOT2,-0.06856035910877033
-27551,TK2,0.22113606519299303
-27554,CKLF,-0.04699994015733551
-27559,CMTM3,0.12066905659688054
-27560,CMTM4,0.10095455221646125
-27561,DYNC1LI2,0.16721350845807761
-27567,NAE1,-0.2234142061538945
-27573,CIAO2B,-0.11463992788021762
-27574,CES2,0.07713664782484553
-27576,CES4A,0.055837692338412424
-27577,CBFB,0.04214236484889593
-27578,C16orf70,0.19230505932635047
-27580,TRADD,-0.1804003261149847
-27581,FBXL8,-0.02394730408752877
-27583,NOL3,0.10368042059924289
-27585,EXOC3L1,0.09456361753430581
-27586,E2F4,0.044073818393296535
-27589,TMEM208,-0.010777936806413726
-27590,FHOD1,0.08702208922183463
-27595,TPPP3,0.19106634161216854
-27596,ZDHHC1,0.17951024783533542
-27599,ATP6V0D1,0.35608961452967225
-27600,AC009061.2,-0.04717296036634246
-27603,RIPOR1,0.20071243562298866
-27609,CTCF,-0.018041735091442523
-27611,CARMIL2,-0.09465388055707603
-27612,ACD,-0.030394305141260583
-27613,PARD6A,-0.08726761569815772
-27615,C16orf86,0.04827805908038292
-27616,GFOD2,-0.026545587059352737
-27617,RANBP10,0.08158684968669534
-27620,CENPT,-0.023295358900151233
-27621,THAP11,-0.15596049389299022
-27622,NUTF2,-0.03453320002397389
-27623,EDC4,0.0027152274519555005
-27625,PSKH1,-0.028043878970041642
-27626,CTRL,5.28302859665958e-05
-27627,PSMB10,0.10418263289937499
-27629,SLC12A4,0.05808134973993063
-27631,DPEP2,0.14859973602357918
-27632,DUS2,0.09445086863391289
-27634,DDX28,0.01876238507125474
-27635,NFATC3,-0.10186018256966206
-27642,PLA2G15,0.10808571427146858
-27644,SLC7A6,-0.0799618955465541
-27645,SLC7A6OS,-0.07336063487146663
-27646,PRMT7,-0.08635077774290385
-27648,SMPD3,-0.005608997786833638
-27651,ZFP90,-0.06479728537312868
-27658,TANGO6,0.07422607316485175
-27664,UTP4,-0.0017049088669329666
-27665,SNTB2,-0.10156638807728792
-27667,VPS4A,-0.055205433679790444
-27668,COG8,0.0007161694061137913
-27670,NIP7,-0.07786047559178445
-27672,TERF2,0.00599771931680675
-27673,CYB5B,-0.02993058685750725
-27675,NFAT5,0.02710128005161709
-27681,NOB1,-0.08582764132903502
-27682,WWP2,0.11280547706861017
-27684,PDPR,0.03879271903263291
-27685,AC009060.1,-0.1104145094860375
-27687,EXOSC6,-0.08662455123901514
-27688,AARS,-0.04018584422731528
-27689,DDX19B,-0.0186041366414901
-27691,DDX19A,-0.0268894791331326
-27693,ST3GAL2,0.06570403208780204
-27695,FCSK,0.00507536476798478
-27696,COG4,-0.04903471100717209
-27697,SF3B3,0.0034562194166762503
-27700,VAC14,0.06708339316327185
-27707,CMTR2,0.008317219036082856
-27712,ZNF23,-0.11076355055777089
-27722,PHLPP2,-0.028691963377915696
-27726,AP1G1,0.08276843403752461
-27727,ATXN1L,0.006268609869103615
-27728,IST1,0.060995959096272895
-27729,ZNF821,-0.14374327282954014
-27732,DHODH,-0.026144367823301814
-27733,TXNL4B,0.11348555708692773
-27734,HP,0.13843401264689356
-27737,DHX38,-0.030186106170892676
-27744,ZFHX3,0.2913000746318796
-27758,PSMD7,0.08512860433189232
-27759,AC009120.2,0.01280472808703465
-27765,GLG1,0.021179547788733828
-27766,RFWD3,-0.009219027135465623
-27767,MLKL,0.2063217951669762
-27769,WDR59,-0.010410691779688508
-27770,ZNRF1,0.08877153786799732
-27774,ZFP1,-0.06337094915417402
-27778,CFDP1,-0.05952601066038639
-27782,TMEM170A,-0.07645646170901389
-27790,GABARAPL2,0.01334160273037219
-27792,ADAT1,0.06279907432602193
-27793,KARS,0.21442770000166078
-27794,TERF2IP,-0.16638326594873434
-27806,MON1B,0.08187538821851838
-27807,SYCE1L,-0.04881412482920661
-27813,NUDT7,1.446389596093469e-05
-27819,WWOX,-0.0027837029688729197
-27826,AC027279.1,-0.013449606185649837
-27835,MAF,-0.1742236507126264
-27853,CMC2,-0.0017348627754152546
-27857,ATMIN,0.06717703012624969
-27862,GCSH,-0.15088300837056365
-27868,CMIP,0.2800653761608766
-27870,PLCG2,0.058169658056884725
-27875,MPHOSPH6,-0.03992547082164812
-27887,HSBP1,0.3112332915098072
-27888,MLYCD,0.05144981062058001
-27894,MBTPS1,0.02589693005700959
-27897,HSDL1,0.028152621474498554
-27899,TAF1C,0.08851982996067409
-27909,MEAK7,0.036088235227908445
-27911,COTL1,0.6449018824252062
-27913,KLHL36,-0.08102998110360343
-27914,USP10,0.10751093680751014
-27916,CRISPLD2,0.27383946296931483
-27920,ZDHHC7,0.32123683281482407
-27921,KIAA0513,0.2872865286468898
-27925,GSE1,0.02186259269769299
-27933,C16orf74,0.019209687629395858
-27937,EMC8,0.07501546938301266
-27940,COX4I1,0.0071771550562804275
-27942,IRF8,0.15061802817365752
-27959,MTHFSD,-0.06804434135297711
-27979,FBXO31,-0.012152795478609186
-27981,MAP1LC3B,-0.012948659211568722
-27983,ZCCHC14,-0.005324309352818687
-27990,KLHDC4,-0.21451715996239595
-27994,SLC7A5,0.0006055999998700876
-27997,BANP,0.11105480858885317
-28007,ZFPM1,-0.10797333817453393
-28009,ZC3H18,-0.060364288399185374
-28011,CYBA,0.023491546699130988
-28012,MVD,-0.13492899138679967
-28014,SNAI3,-0.01924073819628172
-28015,RNF166,0.058413955014904
-28017,CTU2,-0.04268740662906544
-28019,PIEZO1,0.16598446098250957
-28025,APRT,-0.21566579017010848
-28026,GALNS,0.08532735777081674
-28027,TRAPPC2L,-0.11196700966505735
-28029,CBFA2T3,0.23282353781036982
-28035,ACSF3,0.028848461363345092
-28042,ZNF778,0.01696676361630112
-28044,ANKRD11,0.02420377281837444
-28050,SPG7,0.0043348475672417
-28052,RPL13,-0.379650387620572
-28055,CHMP1A,0.020777399741650007
-28056,SPATA33,-0.008081037786290513
-28058,CDK10,-0.04707379475204761
-28060,SPATA2L,-0.029525893265202458
-28061,VPS9D1,0.08709697999296247
-28063,ZNF276,-0.19088732636575872
-28064,FANCA,-0.005724797873586295
-28066,TCF25,-0.08381047297551046
-28070,DEF8,0.13238884375919288
-28071,CENPBD1,0.0977061715128153
-28076,FAM157C,0.08063876008690675
-28092,RFLNB,0.13324462098390583
-28094,VPS53,0.14941663958899715
-28098,GEMIN4,0.04176298178423316
-28099,GLOD4,-0.12266838241455508
-28101,MRM3,-0.034495930160341
-28107,TIMM22,-0.006881402634772371
-28108,ABR,0.09375525980493098
-28113,YWHAE,0.18617735305714717
-28114,CRK,0.20390568321977698
-28116,MYO1C,0.0016332315669936694
-28117,INPP5K,0.044846720277879025
-28118,PITPNA-AS1,-0.1578224638176211
-28119,PITPNA,0.18423491415785273
-28120,SLC43A2,0.38707774370419745
-28121,SCARF1,0.14720349509934988
-28122,RILP,0.16825120949612934
-28123,PRPF8,0.0433063331666016
-28126,MIR22HG,0.1638009632950121
-28127,WDR81,0.03090501831153099
-28131,SMYD4,-0.03993711424674722
-28132,RPA1,0.0013593009489968513
-28143,SMG6,-0.07781474373847247
-28148,SRR,0.007407728364680493
-28149,TSR1,0.01291367019042185
-28150,SGSM2,-0.010055662454699603
-28153,MNT,0.17054740271948746
-28155,METTL16,-0.006014805949925352
-28156,PAFAH1B1,-0.042842787618229884
-28159,CLUH,0.0191904333475791
-28163,RAP1GAP2,-0.1256216633228646
-28184,CTNS,0.0965024806994986
-28186,TAX1BP3,0.12456706936641156
-28187,EMC6,-0.08992146619235009
-28188,P2RX5,-0.024618116156905723
-28189,ITGAE,-0.040537534953785474
-28193,NCBP3,0.05664054685313245
-28195,P2RX1,0.13341116020123348
-28196,ATP2A3,-0.20975424401827333
-28198,ZZEF1,0.029039505512369772
-28199,CYB5D2,0.01586206092318869
-28200,ANKFY1,0.1838075882088267
-28203,UBE2G1,-0.06893954015838775
-28204,SPNS3,-0.10773119842430319
-28208,MYBBP1A,-0.021354061587053445
-28214,PELP1,-0.12225916567239442
-28217,ARRB2,0.38737697487226996
-28218,MED11,-0.07376160561576227
-28220,CXCL16,0.258948184569968
-28225,PSMB6,-0.02302179167658578
-28227,PLD2,0.08129703672163371
-28228,MINK1,-0.011213201280503618
-28232,SLC25A11,-0.035040654023612706
-28233,RNF167,-0.140693572731259
-28234,PFN1,-0.2947688120848683
-28236,SPAG7,-0.07928591359249981
-28237,CAMTA2,0.012374506767292317
-28243,KIF1C,0.07273163778674276
-28249,AC012146.1,-0.18872164721371237
-28251,ZNF594,-0.06701187638738429
-28252,AC087500.1,-0.005575167327440764
-28253,SCIMP,0.4405069174601143
-28255,RABEP1,-0.09372666883099996
-28256,NUP88,0.004925831896168433
-28258,RPAIN,-0.0054968094333777895
-28260,C1QBP,-0.22710586293236343
-28261,DHX33,0.043699926311250685
-28263,DERL2,-0.12931010289137648
-28264,MIS12,0.016306321366610924
-28265,NLRP1,0.19311562367076285
-28275,KIAA0753,0.0028302377263369673
-28277,TXNDC17,0.11514527494045172
-28278,MED31,-0.09915419888469301
-28282,XAF1,0.3680480425615637
-28285,ALOX12-AS1,-0.06613809201648807
-28287,AC040977.1,-0.06239384376406633
-28289,RNASEK,0.24310625207330328
-28290,C17orf49,-0.10098653198517729
-28295,CLEC10A,0.15564897663500313
-28296,ASGR2,0.2406599885272594
-28297,ASGR1,0.41642031921525563
-28298,DLG4,0.12393148436593161
-28299,ACADVL,0.10742288354023972
-28300,DVL2,-0.020895053449600094
-28301,PHF23,0.15604419295256722
-28302,GABARAP,0.5949705412899813
-28303,CTDNEP1,0.18198820016815964
-28304,ELP5,-0.0667950767813744
-28305,CLDN7,0.006674807577290234
-28309,EIF5A,-0.016140916743537476
-28310,GPS2,-0.09139642562706769
-28311,NEURL4,0.036573021144471626
-28312,ACAP1,-0.37340915013361553
-28313,KCTD11,0.021579609895219624
-28317,PLSCR3,-0.08483222376401998
-28318,TMEM256,-0.0695769756093746
-28323,TMEM102,-0.0139491245132376
-28327,CHRNB1,-0.05050400931200543
-28328,ZBTB4,-0.06264024763151671
-28330,POLR2A,0.06913383536121505
-28331,TNFSF12,0.013531501591389078
-28333,TNFSF13,0.44559302737974704
-28334,SENP3,0.08430011272965446
-28335,EIF4A1,0.4470983969993135
-28336,CD68,0.5790302135530349
-28337,AC016876.1,-0.008948892484521248
-28338,MPDU1,0.0794988982009983
-28340,FXR2,0.06846923409147897
-28342,SAT2,0.28456779901528123
-28344,TP53,0.07733216921328805
-28345,WRAP53,-0.09482202468867755
-28348,KDM6B,0.24917118261208396
-28350,NAA38,0.02627965580504318
-28352,CHD3,-0.1301629122914781
-28355,TRAPPC1,-0.01686528648806172
-28356,CNTROB,0.06853565398559715
-28364,PER1,0.20766673845689063
-28365,VAMP2,-0.31113326871976626
-28366,TMEM107,0.03227391230891459
-28368,BORCS6,-0.05486143225744201
-28370,LINC00324,0.016827725335529193
-28371,CTC1,-0.08490985801800388
-28372,PFAS,-0.060089228239925775
-28374,SLC25A35,0.03409234690711927
-28375,RANGRF,-0.14532802028454128
-28380,KRBA2,-0.03313176634550867
-28382,RPL26,-0.38003525680900463
-28384,NDEL1,0.031613051391641354
-28389,PIK3R6,0.06932877739285964
-28390,PIK3R5,-0.018870523262968127
-28396,STX8,-0.12675286128928107
-28412,GAS7,0.40247546496403397
-28424,SCO1,-0.0009508317948412415
-28426,ADPRM,-0.08257222061775367
-28427,TMEM220,-0.02928065270426795
-28441,MAP2K4,0.11446259103014687
-28452,ELAC2,0.00277145586841732
-28456,COX10-AS1,-0.07573539129063928
-28460,COX10,-0.009221943482897118
-28463,HS3ST3B1,-0.06916910986523961
-28465,AC005224.2,-0.17298461337090912
-28482,TVP23C,-0.11235458840335397
-28485,TRIM16,-0.03998316343770373
-28487,ZNF286A,-0.029577203919171055
-28495,ZSWIM7,0.01130851076829975
-28496,TTC19,-0.11173631914177516
-28498,NCOR1,-0.02514750035659653
-28500,PIGL,-0.03072423012764634
-28502,UBB,-0.42422002782385376
-28505,TRPV2,-0.02995777427574715
-28506,SNHG29,-0.19222336184750696
-28508,LRRC75A,-0.023975918696104822
-28510,ZNF287,-0.06764896503345559
-28512,ZNF624,-0.024484856906459903
-28514,CCDC144A,0.05570348049317722
-28519,TNFRSF13B,-0.0028541480269414785
-28525,MPRIP,-0.13452143701671232
-28529,FLCN,0.04735050634645326
-28531,COPS3,0.011224443400469996
-28533,MED9,-0.03760067373329203
-28536,PEMT,-0.09329323604036718
-28540,RAI1,0.045284336972167494
-28542,SMCR5,-0.010755995325956443
-28544,SREBF1,0.0202480705896021
-28545,TOM1L2,-0.06747830667192799
-28548,ATPAF2,0.07267651041171497
-28549,GID4,0.039593339561058406
-28550,DRG2,-0.030477088797785108
-28553,ALKBH5,0.08281131362713746
-28555,FLII,0.23182961038063354
-28556,MIEF2,-0.048445933970426736
-28558,TOP3A,0.17926573233051885
-28559,SMCR8,0.1513360995754915
-28560,SHMT1,-0.09242517055797991
-28565,FAM106A,0.08714710206275082
-28570,TVP23B,0.0012647651707896935
-28572,PRPSAP2,0.02145767376677765
-28577,GRAP,-0.06205793801654866
-28581,AC007952.4,0.32276576768009546
-28594,MAPK7,0.019094156371118724
-28604,ALDH3A2,0.17589087158794522
-28610,ULK2,0.1276670299922992
-28613,AKAP10,0.18466992120414227
-28616,SPECC1,0.24287519938053356
-28631,USP22,0.11358745169646543
-28634,DHRS7B,0.07498031012593215
-28635,TMEM11,0.012652023806330617
-28637,NATD1,0.09797905691483336
-28638,MAP2K3,0.2337812490795383
-28658,WSB1,0.4059887985898241
-28660,KSR1,0.213649306363172
-28663,LGALS9,0.40303764616356036
-28666,LYRM9,-0.07416279665961634
-28669,NLK,0.1106233526309159
-28677,IFT20,0.012342969164119347
-28678,TNFAIP1,0.03614466738363178
-28679,POLDIP2,0.05631395264979939
-28680,TMEM199,0.00464677277433587
-28683,SARM1,-0.11737113217696696
-28688,AC005726.1,0.05902538163794158
-28691,UNC119,0.17414954922857964
-28692,PIGS,0.0015386354720492012
-28698,AC005726.5,-0.01380322901504367
-28699,RSKR,0.07389182625983647
-28700,KIAA0100,0.15513938690447646
-28702,SDF2,-0.11242658343810218
-28703,SUPT6H,0.015557589681525015
-28705,PROCA1,-0.014776262701025295
-28706,RAB34,0.2932789007436222
-28707,RPL23A,-0.49248144919570147
-28710,NEK8,0.0012902981971409926
-28713,TRAF4,-0.053817498057877014
-28716,FAM222B,-0.08891620090918112
-28717,ERAL1,-0.03133427409144012
-28719,FLOT2,0.07671966579120881
-28722,DHRS13,-0.08132049610188534
-28723,PHF12,0.07583301043736344
-28730,MYO18A,0.03137754367060607
-28734,NUFIP2,0.11558972451360143
-28737,TAOK1,0.22839894629533103
-28738,ABHD15,0.001414217574478183
-28740,TP53I13,-0.13801881150939188
-28742,GIT1,0.026978257441226033
-28747,SSH2,0.20373647996579447
-28755,NSRP1,-0.0034463280185889743
-28757,AC104984.3,-0.014819621864399592
-28762,BLMH,0.055559009738255415
-28764,CPD,0.20579651860282044
-28765,GOSR1,-0.04494782780708301
-28770,AC005562.1,-0.044928070251883785
-28773,CRLF3,0.05258355800315717
-28777,ATAD5,-0.10199869320360562
-28780,TEFM,-0.03637988848986805
-28782,ADAP2,0.35979941654072356
-28786,RNF135,0.2506276472280922
-28790,NF1,0.0038404070207577966
-28793,EVI2B,0.4739086791307863
-28794,EVI2A,0.37306710781051244
-28795,RAB11FIP4,0.05178251363929947
-28801,COPRS,0.09290791305340897
-28802,UTP6,0.007217682943783794
-28804,SUZ12,0.04077170860544699
-28805,LRRC37B,0.07390331668671321
-28806,AC116407.2,-0.1363278980021331
-28808,RHOT1,0.20118810183286528
-28812,C17orf75,-0.017080348047639877
-28814,ZNF207,0.2203669235559745
-28820,PSMD11,0.018046863031604295
-28864,ZNF830,-0.09417999244107103
-28865,LIG3,0.0015254059725900693
-28866,RFFL,0.012596035252175944
-28869,RAD51D,0.023851667265106142
-28875,AC022916.1,-0.0722502753894175
-28876,SLFN5,-0.31455556286199626
-28877,AC022706.1,-0.17682813544226508
-28878,SLFN11,0.27695012874786284
-28881,SLFN12,0.2321317128414462
-28882,SLFN13,-0.2008967704546021
-28883,SLFN12L,-0.28076420899655086
-28890,SNHG30,-0.11196372821542364
-28892,AP2B1,-0.07604062457293632
-28893,TAF15,-0.014266734208516471
-28904,CCL5,-0.3895054495446396
-28916,CCL3,-0.16182435770748868
-28917,CCL4,-0.5487030308056304
-28928,ZNHIT3,-0.11693509852713946
-28929,MYO19,0.007148429036600317
-28930,PIGW,0.024338365454496583
-28931,GGNBP2,0.05582639342407516
-28940,AATF,0.00907704967144788
-28945,ACACA,-0.049887455577310426
-28948,TADA2A,0.017686256069708448
-28951,SYNRG,-0.27941768421319085
-28953,DDX52,0.04986109044801632
-28964,MRPL45,0.02696595548062086
-28966,SOCS7,0.008768393400151274
-28972,AC006449.6,-0.018371821564133247
-28974,MLLT6,-0.3188676176031505
-28976,CISD3,-0.16480613581560058
-28979,PSMB3,0.17623410958000654
-28980,PIP4K2B,0.005366200019161357
-28981,CWC25,0.02260442228644678
-28983,RPL23,0.09583059075172443
-28984,LASP1,0.16106545361663413
-28987,LINC00672,0.026597463013333357
-28996,RPL19,-0.39055179747218893
-28998,FBXL20,0.10348604054954594
-29000,MED1,0.054949687918370266
-29001,CDK12,-0.003962624656591785
-29006,STARD3,0.024130908512555047
-29011,MIEN1,-0.1657557499026066
-29013,IKZF3,-0.39011303787819784
-29016,ORMDL3,-0.10632291120371194
-29020,PSMD3,-0.021086102056174824
-29023,MED24,0.0052394227462900945
-29024,THRA,-0.16504239745115123
-29025,NR1D1,-0.024918336190212872
-29026,MSL1,0.050169279941743
-29027,CASC3,0.02772298457432426
-29029,WIPF2,0.15995911586461908
-29031,RARA,0.22634670518715747
-29033,RARA-AS1,0.17220307140567478
-29040,CCR7,-0.15267723022230426
-29041,SMARCE1,-0.09260763056045418
-29050,KRT10,-0.28427158458538965
-29051,TMEM99,-0.11962519371762298
-29113,EIF1,0.10734095833992004
-29116,JUP,0.20840079795017466
-29119,NT5C3B,-0.020588994759147713
-29122,KLHL11,-0.01212673112785834
-29123,ACLY,0.09649934987968516
-29126,CNP,0.09684915797031313
-29127,DNAJC7,0.13862856002547086
-29128,NKIRAS2,0.14880901859003748
-29131,DHX58,0.0803334825804443
-29132,KAT2A,-0.011998765942081418
-29134,RAB5C,0.26650335455999247
-29138,GHDC,0.029032924885914538
-29139,STAT5B,0.006809950416130562
-29143,STAT5A,0.2501080892025065
-29144,STAT3,0.11634399501733392
-29146,ATP6V0A1,0.32458195217481944
-29148,AC067852.3,0.14335224447985725
-29149,NAGLU,0.006028640917903033
-29151,AC067852.2,0.012133568799447292
-29152,COASY,0.04229502080269777
-29153,MLX,0.049113975311345044
-29156,RETREG3,-0.027336473518499858
-29157,TUBG1,-0.018741683433546735
-29164,EZH1,0.02506239419314696
-29168,VPS25,0.0049781638342720255
-29170,COA3,0.0662400314371293
-29172,BECN1,0.14972090406480937
-29173,PSME3,0.13122113767838278
-29178,AARSD1,-0.0017298636968969415
-29180,RUNDC1,0.10055045283883754
-29181,RPL27,-0.32574072036610596
-29182,IFI35,0.2230454414710495
-29183,VAT1,0.0448870643714995
-29185,BRCA1,0.04801308515343091
-29186,NBR2,0.049378659662991244
-29187,AC060780.1,-0.07018454822152122
-29188,NBR1,0.14922578120822516
-29189,TMEM106A,0.1799924790711993
-29194,DHX8,0.09341671235084419
-29199,DUSP3,0.3108952743309675
-29212,TMEM101,0.07111605891491253
-29213,LSM12,0.008452548924795628
-29214,G6PC3,0.04034344120674666
-29215,HDAC5,0.07325138136423937
-29219,ASB16-AS1,-0.06550179268355967
-29220,TMUB2,0.09563920696475384
-29221,ATXN7L3,0.039713064480927394
-29223,UBTF,-0.19075571274826486
-29224,AC003102.1,-0.012412798404537058
-29229,SLC25A39,-0.030085628629322556
-29231,GRN,0.5837404658907327
-29234,GPATCH8,-0.039968240052099695
-29235,AC103703.1,0.0451019817567721
-29236,FZD2,0.20870529977489521
-29239,CCDC43,0.04185193976653619
-29247,EFTUD2,0.12387665434878256
-29255,DCAKD,0.0312047384499698
-29256,NMT1,-0.003992903822258022
-29258,ACBD4,0.029746500030353774
-29259,AC142472.1,0.031244591989815485
-29260,HEXIM1,-0.10956629682594235
-29261,AC138150.1,-0.02557970443252392
-29264,AC008105.3,0.031579402805491354
-29265,FMNL1,0.23252986290746452
-29267,MAP3K14-AS1,-0.08295774912799425
-29269,MAP3K14,0.04093735133848416
-29271,ARHGAP27,0.2443953383573664
-29273,PLEKHM1,0.17319783825585824
-29287,KANSL1,-0.025001203211738657
-29289,KANSL1-AS1,-0.1269477873067899
-29290,ARL17B,-0.0014278937846124533
-29292,LRRC37A2,0.022182968027297106
-29293,ARL17A,0.021829322953823994
-29294,FAM215B,-0.06741013852821934
-29295,NSF,0.10774082708133836
-29300,GOSR2,0.032225411575979857
-29302,AC005670.3,-0.06967643241651947
-29303,CDC27,-0.002290646796638563
-29304,AC002558.3,0.025122303358124657
-29310,EFCAB13,0.010021656080757422
-29312,NPEPPS,0.016145212711839482
-29314,KPNB1,0.18924319513325505
-29317,TBX21,-0.4925665285968005
-29318,OSBPL7,-0.11903381425263998
-29319,MRPL10,-0.11333716103572195
-29321,SCRN2,-0.0006931123414162293
-29325,AC018521.5,-0.08556261433372324
-29326,SP2,-0.0001473832028796394
-29329,PNPO,0.10075820282741475
-29332,CDK5RAP3,0.13108575846385864
-29336,NFE2L1,0.0937956749037937
-29339,CBX1,0.05293088641588362
-29340,SNX11,0.09885808535644625
-29341,SKAP1,-0.48390850802955565
-29347,HOXB2,-0.18720078744765206
-29348,HOXB-AS1,-0.10506377725484611
-29352,HOXB4,-0.08897334334974168
-29367,CALCOCO2,0.11127291319929863
-29368,ATP5MC1,-0.00969278825695676
-29369,UBE2Z,0.16206722769090076
-29370,SNF8,0.1153437111991633
-29377,GNGT2,-0.05169549034599504
-29378,ABI3,-0.11380278686636996
-29381,ZNF652,-0.05664566888262394
-29385,PHB,-0.08033862355884633
-29392,SPOP,0.04828729240777117
-29393,SLC35B1,0.006045221730528744
-29395,FAM117A,-0.13110157857190158
-29396,KAT7,0.0293299692675375
-29408,PDK2,0.024091548185708217
-29412,PPP1R9B,0.22280643439933445
-29422,XYLT2,0.008935033792208148
-29423,MRPL27,0.029939199919014923
-29425,LRRC59,0.09408812283603933
-29427,ACSF2,0.09896291366315299
-29430,RSAD1,0.015740560322200225
-29435,SPATA20,0.050586881955237986
-29440,ABCC3,0.09169062777399316
-29441,ANKRD40,0.02892526256506855
-29442,AC005921.2,-0.10008682963919759
-29443,LUC7L3,-0.0716964319305759
-29448,TOB1,-0.014536213079538706
-29449,TOB1-AS1,-0.07594146225187459
-29453,SPAG9,0.15539328316794437
-29455,NME1,-0.05463363918105303
-29456,NME2,-4.7405416270140093e-05
-29457,MBTD1,0.0927008343994553
-29458,AC006141.1,-0.02131946904889507
-29459,UTP18,-0.014102798095837064
-29472,COX11,-0.12683034445594465
-29478,MMD,0.03439800855208993
-29481,PCTP,0.13664854901642978
-29484,NOG,-0.0669469861351763
-29485,C17orf67,-0.10332284248647747
-29487,DGKE,-0.030289765852386
-29488,TRIM25,0.20724332434290296
-29490,AC015912.3,0.1326973072728416
-29491,COIL,-0.0921795312291121
-29493,SCPEP1,0.44731185313663135
-29496,AKAP1,0.03521421288064543
-29500,MSI2,-0.1153250030667342
-29505,MRPS23,-0.08969170207954606
-29506,CUEDC1,0.14812831887555175
-29509,VEZF1,0.047519919604447784
-29510,AC015813.1,0.06204054467759284
-29511,SRSF1,0.07103805141789205
-29514,DYNLL2,-0.011799837376640797
-29519,MKS1,0.006260155003981651
-29524,TSPOAP1,-0.17008014723642845
-29525,TSPOAP1-AS1,-0.12864023438303224
-29526,AC004687.1,-0.19981279699859744
-29527,SUPT4H1,0.0894578274442009
-29530,MTMR4,-0.01769266358991587
-29535,RAD51C,-0.06749772902435576
-29539,TRIM37,0.05828650362486212
-29541,SKA2,-0.17174988430276175
-29542,PRR11,0.15183056480676296
-29544,SMG8,0.05648838696782295
-29545,GDPD1,-0.0706473462904482
-29546,YPEL2,0.18545617297971886
-29549,DHX40,0.16107922345236572
-29550,AC091271.1,0.06873018733457116
-29551,CLTC,0.2752454376901087
-29552,PTRH2,0.020363180913386566
-29553,VMP1,0.5114263765086708
-29554,AC040904.1,0.10294153650232675
-29555,TUBD1,-0.01660092455645779
-29556,RPS6KB1,0.03919538237146148
-29557,RNFT1,0.07562766331568159
-29560,HEATR6,0.007815110143440945
-29566,USP32,0.2349002259811325
-29568,APPBP2,0.11781936397065229
-29571,PPM1D,0.16883007258198873
-29572,BCAS3,-0.08594722882669627
-29584,NACA2,0.015602036751957905
-29586,INTS2,0.031549698718770114
-29587,MED13,0.12019785892847676
-29590,METTL2A,-0.07939884479598833
-29591,TLK2,0.04123626036755261
-29601,TANC2,-0.03912804849520525
-29608,CYB561,-0.04656549645429482
-29612,DCAF7,0.19614974549212655
-29613,TACO1,-0.056212000730661765
-29614,MAP3K3,0.23864651608239743
-29615,STRADA,0.050251818111827745
-29616,LIMD2,-0.08787897325349546
-29617,CCDC47,0.008611560535457134
-29618,DDX42,0.045151835717452384
-29619,FTSJ3,0.04409805608424497
-29620,PSMC5,-0.136845019997253
-29621,SMARCD2,0.19742590006208635
-29627,CD79B,0.0017018593067092487
-29632,ICAM2,-0.14536647692131163
-29633,ERN1,-0.11537046818711166
-29635,SNHG25,-0.03846092487670565
-29636,TEX2,0.03170291074773936
-29637,PECAM1,0.41531274612965763
-29638,MILR1,0.21435991964329368
-29639,POLG2,-0.04874965942036321
-29640,DDX5,-0.24079844935966077
-29641,CEP95,-0.11880647369232228
-29642,SMURF2,-0.028795556357833763
-29647,GNA13,0.1396747056441267
-29651,AXIN2,-0.06601406445964515
-29654,PRKCA,-0.10523483020396723
-29662,HELZ,0.146195339343326
-29666,PSMD12,-0.004437577318845547
-29667,PITPNC1,-0.18662602057170244
-29670,NOL11,-0.10007733638399913
-29671,BPTF,0.08782017259619096
-29673,C17orf58,-0.022288042310685856
-29674,KPNA2,0.02213598529886254
-29676,AC005332.6,-0.028375954291767837
-29681,AC005332.4,-0.15169377757742122
-29683,AMZ2,-0.0048767579979976505
-29684,ARSG,-0.028401816032840057
-29685,SLC16A6,0.15637123717423865
-29687,WIPI1,0.004260808710499494
-29688,PRKAR1A,0.3505707216704414
-29699,ABCA5,-0.12129523573154513
-29700,MAP2K6,-0.0038112246090537547
-29708,KCNJ2,0.22002123209954658
-29724,SLC39A11,0.15149825292205019
-29726,SSTR2,0.04855863341955264
-29727,COG1,-0.0008353542345324099
-29729,FAM104A,-0.016627078854906545
-29730,C17orf80,-0.052588618334346915
-29733,CDC42EP4,0.10358451060347354
-29746,RPL38,-0.30333048193604956
-29747,AC100786.1,0.09842062207681992
-29748,TTYH2,0.14770751167019722
-29756,CD300A,-0.004785879087058221
-29757,CD300LB,0.23581279316351667
-29758,CD300C,0.25226038052719113
-29761,AC064805.1,0.13560148274539321
-29762,CD300E,0.43023627659117575
-29764,RAB37,-0.16642638435808282
-29765,CD300LF,0.36049273728669695
-29767,SLC9A3R1,-0.3557144782136761
-29768,NAT9,0.0036490754762026987
-29769,TMEM104,-0.054050309706362584
-29779,MRPL58,-0.0808468383083359
-29780,KCTD2,0.006664594982584208
-29781,ATP5PD,0.010869051297057742
-29782,SLC16A5,0.14075783702385603
-29783,ARMC7,-0.0010378349753500248
-29784,NT5C,-0.18697405840370313
-29785,JPT1,-0.03784532133451042
-29788,SUMO2,-0.15692782405257868
-29789,NUP85,0.06489860530985478
-29790,GGA3,0.16489944185366393
-29791,MRPS7,-0.04713092394664405
-29792,MIF4GD,0.05628106425518815
-29794,SLC25A19,0.03349017938367286
-29795,GRB2,0.4002827709079115
-29797,AC011933.2,0.11668389011685044
-29799,TMEM94,-0.0032343282518143357
-29801,TSEN54,-0.15158463896006702
-29802,LLGL2,-0.3486481574110104
-29803,MYO15B,0.14893497250007648
-29804,RECQL5,0.056401443019760425
-29807,SAP30BP,-0.02664163335131818
-29811,GALK1,0.1266144143670613
-29812,H3F3B,0.19937003779855997
-29813,UNK,-0.08763567724241951
-29815,UNC13D,0.08939661231970186
-29816,WBP2,0.06088880047252751
-29819,TRIM65,-0.025357485440235042
-29821,MRPL38,-0.09615108679489845
-29823,ACOX1,0.2184041075596728
-29825,TEN1,-0.009646143070720361
-29828,SRP68,0.005385315663332447
-29831,EXOC7,0.029742726492036648
-29834,RNF157,-0.12834032273056187
-29835,UBALD2,-0.03811481028564505
-29837,PRPSAP1,-0.09118447332739436
-29838,SPHK1,0.09752823771068861
-29839,UBE2O,0.024104554305255002
-29841,RHBDF2,0.12011814542415772
-29847,SNHG16,-0.006046510789781613
-29848,AC015802.6,-0.029081595180918626
-29854,MXRA7,-0.14288645643453898
-29856,JMJD6,-0.00795379701355044
-29857,METTL23,-0.2200532817331246
-29858,SRSF2,0.0024327615319899346
-29859,MFSD11,-0.021392961473202673
-29866,SNHG20,0.007023831273817482
-29867,SEC14L1,0.23227915356336298
-29870,SEPTIN9,0.05863121086615484
-29882,TNRC6C,-0.17829438132667214
-29883,TNRC6C-AS1,-0.08545095677424935
-29884,TMC6,-0.20867855675571267
-29885,TMC8,-0.14697550855657743
-29887,SYNGR2,0.2650575707929611
-29889,AFMID,0.06723866009190946
-29895,SOCS3,0.11749761249584534
-29897,PGS1,0.07612735389456857
-29901,CYTH1,-0.06287028213029033
-29902,USP36,-0.07936481554816806
-29904,TIMP2,0.46090492740499367
-29908,LGALS3BP,0.0474185508239192
-29909,CANT1,0.09909253623141387
-29912,ENGASE,0.04869766223679429
-29919,CBX8,0.022329353920541938
-29922,CBX4,0.005408389243087992
-29931,GAA,0.32940284695212296
-29932,EIF4A3,-0.02560844865092839
-29935,AC087741.1,0.08706623070858817
-29936,SGSH,0.06367368302140418
-29937,SLC26A11,-0.08758729744188977
-29938,RNF213,0.28243262948808556
-29939,AC124319.1,0.03278657183234808
-29941,RNF213-AS1,-0.09028819098938398
-29942,ENDOV,0.027604223676588085
-29945,RPTOR,-0.05052774517704679
-29951,CHMP6,-0.05831916487672244
-29955,BAIAP2-DT,0.14174860290479036
-29956,BAIAP2,0.09342812279993751
-29960,CEP131,-0.03161072701655159
-29962,TEPSIN,0.04706537888440792
-29964,NDUFAF8,-0.1766703631222184
-29965,SLC38A10,0.192800627932345
-29969,AC027601.6,0.08514656172102691
-29977,ACTG1,-0.2601270089849833
-29980,FAAP100,0.07299634477060343
-29981,NPLOC4,0.07279022377363227
-29984,OXLD1,-0.07677931566313649
-29985,CCDC137,-0.09036417192404508
-29986,ARL16,-0.01723511098562315
-29987,HGS,0.11045259321393248
-29989,AC139530.1,-0.1116572210444537
-29990,MRPL12,0.024725165962076502
-29993,MCRIP1,-0.15015252331154813
-29995,P4HB,0.06870424544842003
-29998,ARHGDIA,0.16443992121814863
-29999,AC145207.5,-0.02528120894777612
-30001,ALYREF,0.0784392933237438
-30002,ANAPC11,-0.03529150549085427
-30004,PCYT2,-0.08922436457233603
-30005,SIRT7,0.003951690052154913
-30006,MAFG,0.20582185368643005
-30015,ASPSCR1,0.10298201041792791
-30016,CENPX,0.00953071577935663
-30017,LRRC45,0.023355602250866997
-30019,DCXR,-0.10778257778879984
-30021,RFNG,0.07403469367904214
-30022,GPS1,0.000986237171079841
-30023,DUS1L,-0.06038099341038449
-30024,FASN,0.011021622758800436
-30025,CCDC57,-0.10923150327605707
-30029,SLC16A3,0.3169609626030677
-30030,CSNK1D,0.038723418731561196
-30034,AC132872.1,0.01394614796834652
-30035,CD7,-0.5213912807999432
-30036,SECTM1,0.39040388729537323
-30040,OGFOD3,-0.01703646257672244
-30042,HEXD,-0.023885350413789978
-30044,CYBC1,0.0477823581788127
-30047,NARF,-0.05611928366575018
-30050,FOXK2,0.015538771037811622
-30053,AC124283.1,-0.055990704220251444
-30054,WDR45B,-0.01748602323458011
-30057,FN3KRP,-0.14747573724281293
-30061,TBCD,0.05687862520162272
-30065,B3GNTL1,-0.03130920546029623
-30067,METRNL,0.0055678760809782565
-30076,USP14,-0.0584164289744974
-30077,THOC1,-0.06212778733634682
-30089,ENOSF1,-0.05828419931186647
-30091,YES1,-0.2358047277868204
-30108,METTL4,-0.09174668130247367
-30111,SMCHD1,0.25296313343789434
-30113,EMILIN2,0.4637577697720384
-30114,LPIN2,-0.094806889354541
-30120,AP005329.3,-0.03514016414626985
-30121,MYL12A,-0.43069035821611684
-30123,MYL12B,-0.35404343292664675
-30127,TGIF1,0.010616111795568274
-30130,DLGAP1-AS1,-0.12111927827898333
-30144,LINC00667,-0.006157696660024606
-30145,ZBTB14,-0.12360974981348974
-30148,EPB41L3,0.37092315205485255
-30181,RAB12,0.17870328367081253
-30188,NDUFV2,-0.10400834069091713
-30190,NDUFV2-AS1,-0.07844317484316636
-30191,ANKRD12,-0.1490367032628711
-30194,TWSG1,0.0501488110283556
-30197,RALBP1,0.010338880756371214
-30199,PPP4R1,0.07745988744384306
-30203,RAB31,0.510882582097072
-30207,VAPA,0.11637866366779837
-30216,NAPG,-0.03335606997060493
-30235,CHMP1B,0.2373891342431829
-30238,MPPE1,0.04643758341491856
-30241,IMPA2,0.2965362227857429
-30251,AFG3L2,0.06901233607767542
-30255,SPIRE1,-0.030191885683153997
-30256,PSMG2,0.004661040354672355
-30261,PTPN2,0.14598410981563864
-30262,SEH1L,0.055431744205987565
-30264,CEP192,0.042711137178141005
-30267,LDLRAD4,0.0015305604060419722
-30280,FAM210A,-0.05609223497738798
-30281,RNMT,-0.08107452789669242
-30284,ZNF519,-0.030675557385139773
-30302,ROCK1,0.20084631013733178
-30307,ESCO1,-0.09226318497656723
-30308,SNRPD1,-0.149080618104254
-30309,ABHD3,0.07124603869299165
-30312,MIB1,0.024348427025118833
-30326,RBBP8,0.20389853338111597
-30331,RIOK3,0.19047072288602498
-30332,RMC1,-0.015253428273397956
-30333,NPC1,-0.02677283872825151
-30337,TTC39C,-0.18235501216327904
-30343,OSBPL1A,0.1377928165491098
-30346,IMPACT,0.13363279736035055
-30359,SS18,0.04067150275511898
-30361,TAF4B,0.007106588972560252
-30390,DSC2,0.21106027418046666
-30405,TRAPPC8,0.11056303691147773
-30409,RNF125,-0.19960751372906763
-30412,RNF138,0.10669720269647233
-30434,MAPRE2,-0.10461128920062783
-30436,ZNF397,0.01903943523880937
-30437,ZSCAN30,-0.06357331150055422
-30441,ZNF24,0.030587521553278416
-30445,INO80C,-0.00999955815797596
-30446,GALNT1,0.1376961751502604
-30450,C18orf21,0.07069677639947826
-30451,RPRD1A,0.06210485626403813
-30452,SLC39A6,0.058464253617439975
-30453,ELP2,-0.052925590107770225
-30459,TPGS2,-0.04943464774391598
-30460,KIAA1328,-0.08367108057179036
-30478,PIK3C3,0.06323047160602818
-30489,SETBP1,-0.10202501585427232
-30501,EPG5,0.10011133044825735
-30502,PSTPIP2,0.35852933435297585
-30503,ATP5F1A,0.08710627889761507
-30504,HAUS1,-0.025977593506058168
-30505,C18orf25,0.05771903033351364
-30512,PIAS2,-0.02695592773620664
-30520,HDHD2,0.02622277847557584
-30521,IER3IP1,-0.01625272812257469
-30526,SMAD2,0.044391741650018766
-30533,SMAD7,-0.1996031731613397
-30536,DYM,0.06502115223417233
-30541,C18orf32,-0.08468784093008934
-30544,RPL17,-0.43723191775761533
-30548,ACAA2,-0.11834767204457149
-30554,MBD1,-0.0003689330199229311
-30555,CXXC1,-0.0014236415897923844
-30561,ME2,0.18187841289963463
-30562,ELAC1,-0.039285348756322905
-30563,SMAD4,0.045858980473147605
-30564,MEX3C,-0.1642039374844774
-30572,MBD2,0.3149769632273726
-30574,POLI,0.09602021267763398
-30583,TCF4,0.08720219013952728
-30598,TXNL1,-0.02351574883985405
-30599,WDR7,0.05521461814256959
-30610,FECH,0.015745901316836404
-30612,NARS,0.13742078983461972
-30613,AC027097.1,0.012570986668468152
-30614,AC027097.2,0.026249347900171464
-30630,MALT1,0.05514006752937136
-30635,ZNF532,0.05962936465457614
-30637,SEC11C,-0.23983193225075378
-30641,LMAN1,-0.1373674930836892
-30649,PMAIP1,0.08073464146296416
-30663,PIGN,0.06798760994772365
-30664,RELCH,0.1694674457271096
-30669,ZCCHC2,0.07261659922467725
-30671,PHLPP1,0.01566569871205602
-30672,BCL2,-0.20810073392773012
-30675,KDSR,-0.011805649226762558
-30677,VPS4B,0.08687252052658473
-30690,SERPINB8,0.20228876038860363
-30714,TMX3,-0.03539526088523696
-30726,CD226,-0.0994483019642023
-30727,RTTN,-0.05012608721682985
-30752,TIMM21,-0.0010010026294522212
-30754,CYB5A,-0.06958694542726047
-30760,CNDP2,0.23813321095898604
-30764,LINC00909,-0.03398228600931874
-30765,ZNF407,-0.01836971961267781
-30769,ZADH2,0.13266603492506243
-30770,TSHZ1,-0.007890201340364776
-30790,ZNF516,0.280247578251059
-30800,ZNF236,0.009644516461562438
-30802,MBP,-0.23389964621359957
-30822,ATP9B,0.028170237637116676
-30827,NFATC1,0.06967618267863604
-30835,CTDP1,0.0894667863592019
-30841,SLC66A2,0.19379518098955192
-30843,HSBP1L1,-0.10425478095243834
-30844,TXNL4A,-0.07275217784203224
-30846,RBFA,-0.06875323189799067
-30848,ADNP2,0.04313852176112203
-30857,LINC01002,0.09765487977501704
-30860,MIER2,0.11201575733055241
-30870,TPGS1,-0.11391155319922065
-30871,CDC34,0.022008172200241424
-30872,GZMM,-0.7117981140891726
-30874,BSG,0.030431517145494143
-30876,POLRMT,-0.0011340424265016595
-30879,RNF126,-0.18957921357681679
-30881,FSTL3,0.048855885849198796
-30887,PTBP1,0.027454299968303293
-30892,CFD,0.4388240476417986
-30893,MED16,0.09926980930079517
-30894,R3HDM4,0.06876870642969928
-30896,ARID3A,0.2839754311737155
-30899,WDR18,0.04873943497441964
-30902,TMEM259,0.003924894600470161
-30904,CNN2,-0.153092145771622
-30905,ABCA7,0.03578808177027476
-30906,ARHGAP45,0.10412621184400268
-30907,POLR2E,0.045038999579883825
-30908,GPX4,0.14811320181792445
-30909,SBNO2,0.12163792364319431
-30910,STK11,0.17784336540314877
-30913,ATP5F1D,-0.04714941737690732
-30914,MIDN,0.19347678505008556
-30915,CIRBP,-0.2824979325570062
-30917,FAM174C,-0.06781201867183118
-30920,PWWP3A,-0.0505483970266064
-30922,NDUFS7,-0.0255369068517168
-30925,GAMT,-0.19942513730410197
-30926,DAZAP1,0.04765146715713794
-30927,RPS15,-0.4258083815799846
-30931,C19orf25,-0.13738547995822514
-30938,MBD3,0.06582693022566156
-30939,UQCR11,-0.012884278442878545
-30940,TCF3,-0.04021713084760583
-30944,REXO1,-0.03793079515517945
-30948,KLF16,0.048852239290230165
-30950,AC012615.1,-0.08830989552882407
-30952,ABHD17A,-0.42720059624502
-30953,SCAMP4,0.10843719990119619
-30955,CSNK1G2,-0.007751526753989596
-30957,BTBD2,0.06929542803647402
-30958,AC005306.1,-0.031864709789036806
-30959,MKNK2,0.0357253002481179
-30960,MOB3A,0.08020608785302234
-30961,IZUMO4,0.029381324149679314
-30962,AP3D1,0.07456803947628225
-30963,DOT1L,0.09920094885258933
-30965,PLEKHJ1,-0.06921263272215583
-30966,SF3A2,-0.14281965105877642
-30969,OAZ1,0.41709203162338826
-30970,PEAK3,0.10134278406725587
-30972,AC104530.1,0.11584412258295715
-30974,LSM7,-0.1145310342348809
-30975,SPPL2B,-0.042521430189456345
-30977,TIMM13,0.044297797469370896
-30978,LMNB2,-0.010715675168916925
-30980,GADD45B,0.05251130642970659
-30981,GNG7,-0.02057556622030177
-30989,SLC39A3,0.02131233775822227
-30990,SGTA,-0.005147196762112073
-30991,THOP1,-0.10333700394605168
-31003,TLE5,-0.4924447936489909
-31004,GNA11,0.09740600543615255
-31007,GNA15,0.22314863651478373
-31009,S1PR4,-0.19555178236839704
-31010,NCLN,0.07986175613578307
-31013,NFIC,0.2083165988852818
-31016,DOHH,-0.002742685622112337
-31017,FZR1,0.08094455239937301
-31018,MFSD12,0.17756047356647117
-31022,HMG20B,0.014734853724084026
-31026,CACTIN,-0.07528501119051573
-31027,PIP5K1C,-0.001174067540666709
-31030,APBA3,-0.02753770778571712
-31033,MRPL54,-0.071951271286789
-31035,MATK,-0.5266600887298329
-31039,DAPK3,0.05676666630088119
-31040,EEF2,0.17954580891721253
-31041,PIAS4,0.1120661566689739
-31042,ZBTB7A,0.12664244330945587
-31044,MAP2K2,-0.13695057945497432
-31046,SIRT6,-0.007589154444630507
-31049,YJU2,-0.05745396880398525
-31051,TMIGD2,-0.2032493067663352
-31055,MPND,0.0410351674751817
-31058,SH3GL1,0.030267205550108256
-31060,CHAF1A,-0.06389230874486573
-31062,UBXN6,0.1608672435058871
-31066,HDGFL2,-0.06769780869605017
-31069,LRG1,0.09907691162918633
-31071,TNFAIP8L1,-0.047757244377345076
-31072,MYDGF,-0.16818443580666476
-31074,DPP9,0.03400779689036546
-31079,FEM1A,-0.0018465175124071354
-31080,AC005523.2,-0.08860005091789469
-31081,TICAM1,0.11473791397424264
-31083,PLIN3,0.30732177453409637
-31084,ARRDC5,-0.010939254755992129
-31086,KDM4B,0.11589485341348299
-31092,SAFB2,0.04889812040716548
-31093,SAFB,0.0467386567632858
-31094,RPL36,-0.34285884727434196
-31095,MICOS13,-0.10551402910374587
-31097,LONP1,-0.026955048258564054
-31100,DUS3L,-0.009209815143457527
-31106,NDUFA11,-0.02754102420809061
-31108,VMAC,-0.07014026897572678
-31109,AC104532.2,-0.04079004482643299
-31110,CAPS,-0.03827876980276227
-31111,RANBP3,-0.023666800554247112
-31113,RFX2,-0.005539376843644482
-31117,MLLT1,0.08591475445078493
-31121,CLPP,-0.1029997496635962
-31122,ALKBH7,-0.09438006008039201
-31124,GTF2F1,-0.016532763590493153
-31126,KHSRP,0.08946835578680566
-31130,DENND1C,-0.0631514777101265
-31137,TNFSF14,-0.24026868804707424
-31140,GPR108,-0.017638684197555807
-31143,SH2D3A,-0.13981190801934704
-31144,VAV1,0.005549716611536318
-31145,ADGRE1,0.20886095612962832
-31152,ZNF557,0.015372732604838343
-31153,INSR,0.23888996734106893
-31154,AC119396.1,-0.04649224018398573
-31156,ARHGEF18,-0.040285968587946475
-31159,ZNF358,0.03189583587824733
-31161,MCOLN1,0.23469110069844779
-31162,PNPLA6,0.26665067002719206
-31164,XAB2,0.08409394405748832
-31165,PET100,-0.04944275252119753
-31168,STXBP2,0.33278214963812247
-31169,RETN,0.20389351856261365
-31170,MCEMP1,0.24183670384144526
-31171,TRAPPC5,0.266224290574946
-31172,FCER2,0.042075756339993
-31176,EVI5L,-0.005972718546825741
-31181,MAP2K7,0.09806687717047902
-31185,SNAPC2,0.09068222763639927
-31188,TIMM44,-0.08901197162693703
-31190,ELAVL1,0.12914611971295617
-31195,CERS4,-0.13146188610823537
-31197,CD320,-0.31369976016333556
-31198,NDUFA7,0.0004887957795309435
-31199,RPS28,-0.3633181786067579
-31202,RAB11B-AS1,0.016921299505551952
-31203,RAB11B,0.04494177434351722
-31204,MARCH2,0.06474673120485043
-31206,HNRNPM,-0.08329322272634641
-31207,PRAM1,0.4457871200154126
-31208,ZNF414,-0.05485206663716139
-31209,MYO1F,0.42610713490795626
-31225,ZNF317,0.0709288495562222
-31229,ZNF699,0.005289367882042942
-31231,ZNF559,-0.07608149469728523
-31234,ZNF266,-0.0028812299989037942
-31239,ZNF426,-0.07515173376751925
-31241,ZNF121,0.017104672653521356
-31242,ZNF561,-0.053177144303925006
-31243,ZNF561-AS1,0.014585786829079423
-31244,ZNF562,-0.01295386753840543
-31247,ZNF846,-0.09368642169072028
-31249,FBXL12,0.04725454704426189
-31250,UBL5,-0.11760595959681121
-31252,PIN1,-0.20772623941570076
-31253,OLFM2,-0.10120502636071091
-31257,SHFL,-0.025333972430423794
-31260,PPAN,-0.13210826071791285
-31261,P2RY11,-0.12846315083137752
-31262,EIF3G,-0.23097406622449454
-31263,DNMT1,-0.09054399538439031
-31264,S1PR2,-0.02923908571784921
-31265,MRPL4,0.0022704439733217075
-31268,ICAM1,0.19077284558637642
-31270,ICAM4,0.12060080751489448
-31272,ZGLP1,0.08412294548033067
-31273,FDX2,-0.031365170654358246
-31274,RAVER1,-0.008060609898603154
-31276,ICAM3,-0.21833652886352714
-31277,TYK2,0.17787759480227439
-31278,CDC37,-0.04100968819229878
-31279,PDE4A,0.02117703069426015
-31280,KEAP1,0.010099235773471353
-31282,S1PR5,-0.4767393934303378
-31283,ATG4D,-0.14451954378451481
-31284,KRI1,-0.011162672397646399
-31285,CDKN2D,-0.060429058821040896
-31287,SLC44A2,-0.12486136890649328
-31288,ILF3-DT,-0.09369270579525567
-31289,ILF3,0.013020920434189958
-31290,QTRT1,-0.0260989453204583
-31291,DNM2,0.1350888119999751
-31292,TMED1,0.025329003468411573
-31293,C19orf38,0.44477404120201064
-31294,CARM1,0.08494825843873853
-31295,YIPF2,0.019846922287627425
-31296,TIMM29,-0.01453524295891114
-31297,SMARCA4,0.11067970787510817
-31299,LDLR,-0.03432936410047263
-31307,AC011472.2,0.16721598308024066
-31308,RAB3D,0.3568979708938869
-31310,TMEM205,0.1295297256549926
-31311,CCDC159,0.06271841904431381
-31312,PLPPR2,0.2210082673433411
-31314,SWSAP1,0.04014088484992133
-31315,EPOR,0.07343990708177918
-31317,CCDC151,0.08538128544148937
-31318,PRKCSH,0.09546248955099337
-31321,ECSIT,0.0275401437693144
-31323,ELOF1,-0.1278501441666159
-31325,ACP5,0.008612216152143564
-31329,ZNF441,0.0012294714117935736
-31332,ZNF440,0.042935486238613194
-31334,ZNF439,-0.04970298919558966
-31336,ZNF700,0.034802486712442474
-31338,ZNF763,0.007908062256851632
-31339,ZNF433-AS1,-0.11090153135127731
-31342,ZNF844,-0.00583209002840183
-31345,ZNF136,-0.03483652947195865
-31346,ZNF44,-0.060908656140050886
-31347,ZNF563,-0.04079664763895247
-31349,ZNF799,-0.010409496934756186
-31353,ZNF564,0.044188256600740314
-31356,ZNF490,0.027363196717447504
-31357,ZNF791,0.04680702348274843
-31358,MAN2B1,0.3417859299779304
-31359,WDR83,-0.09507874147175518
-31360,WDR83OS,0.007072534570887276
-31362,DHPS,-0.030228188476042948
-31367,TNPO2,0.03416482016423421
-31369,TRIR,-0.18353052120976857
-31370,GET3,-0.06126246823339832
-31372,HOOK2,-0.001294360742541508
-31374,JUNB,0.3995613460207629
-31375,PRDX2,-0.3274434630000776
-31382,DNASE2,0.04193373565447344
-31385,GCDH,-0.00911016611469565
-31387,FARSA,0.0907883798106322
-31389,CALR,-0.18994576813056716
-31391,RAD23A,-0.045238460663472514
-31392,GADD45GIP1,-0.22566936557151734
-31394,NFIX,0.14531960895224297
-31397,LYL1,0.2498402686960637
-31398,TRMT1,0.18832790912659317
-31399,NACC1,0.059006235016781164
-31402,STX10,0.1351880175342034
-31403,IER2,0.15898067707456745
-31406,CACNA1A,0.042813000206062
-31407,CCDC130,-0.04848994368189351
-31408,MRI1,-0.027051545573511936
-31410,C19orf53,-0.22859768671760936
-31413,AC020916.1,0.38929196185665743
-31416,CC2D1A,0.030796264791063146
-31418,DCAF15,0.01835846415627249
-31419,RFX1,0.0502062709708439
-31423,IL27RA,-0.038603633264951975
-31427,SAMD1,-0.014064761891464266
-31428,PRKACA,0.3141673073894181
-31430,ASF1B,0.12562043659145508
-31431,AC022098.1,0.10015644598392774
-31438,ADGRE5,0.14273418560327716
-31440,DDX39A,-0.06874839649783668
-31441,PKN1,0.10945961159030045
-31442,AC008569.2,0.024879373038568887
-31444,GIPC1,-0.26154959217273027
-31445,DNAJB1,-0.07986900623288473
-31446,TECR,-0.21114529274896549
-31447,NDUFB7,-0.19244778624300385
-31449,ADGRE3,0.1251456142390486
-31450,ZNF333,0.06552144492361331
-31451,ADGRE2,0.3030011988963641
-31462,ILVBL,-0.019269374290022633
-31467,BRD4,0.08105032249643747
-31469,AKAP8,0.0028213172148801634
-31471,AKAP8L,-0.06985271802176311
-31472,WIZ,0.0158775271268742
-31473,RASAL3,-0.26715922139048026
-31493,TPM4,0.1493049402813677
-31495,RAB8A,0.11745412375252372
-31497,HSH2D,-0.009671066626280428
-31499,FAM32A,0.11833400217719174
-31502,AP1M1,0.0250687150933312
-31503,AC020911.2,0.03644919743283876
-31504,KLF2,-0.4097822679201057
-31506,EPS15L1,-0.022051717260823424
-31510,CHERP,-0.008758875397076085
-31512,SLC35E1,0.13703489991932097
-31515,AC008764.8,-0.01226526906665209
-31516,MED26,0.033504522226259985
-31520,SMIM7,0.012038404166505045
-31521,AC024075.2,0.0037743758630177926
-31522,AC024075.1,0.05007543731983098
-31523,TMEM38A,0.11090665607628587
-31526,SIN3B,0.17898355884285197
-31529,CPAMD8,0.1087982094068722
-31531,HAUS8,0.014981395947817652
-31533,MYO9B,0.18766963282452417
-31538,USE1,-0.05737141881545378
-31539,OCEL1,0.011660619317782691
-31540,NR2F6,-0.028797712455809546
-31542,BABAM1,0.07950485557934954
-31544,ABHD8,0.06532039447504383
-31545,MRPL34,-0.15872177808491097
-31547,DDA1,-0.043685214315626754
-31549,GTPBP3,-0.05235870468098454
-31553,BST2,0.16828229510891166
-31554,BISPR,0.06740727868975814
-31555,MVB12A,0.07037667779322243
-31561,SLC27A1,0.08857266616944441
-31562,AC010618.3,-0.0040096533699229015
-31563,PGLS,0.2636204844603219
-31565,NIBAN3,-0.0017700997423094644
-31566,COLGALT1,0.1830971763387983
-31569,MAP1S,0.0720093013155348
-31571,FCHO1,0.053858515795995175
-31574,JAK3,0.016065070703849067
-31575,RPL18A,-0.3162564983387396
-31577,CCDC124,-0.0009635317375965617
-31579,ARRDC2,0.006660863537132654
-31580,IL12RB1,-0.10101571297594338
-31581,MAST3,-0.01811978418053457
-31583,PIK3R2,-0.08039662247119884
-31584,IFI30,0.6237941191812438
-31585,MPV17L2,0.06728848394200511
-31591,JUND,0.22644734300707328
-31592,LSM4,-0.02418958292169447
-31593,PGPEP1,0.07180077077484628
-31595,LRRC25,0.5038229640086286
-31596,SSBP4,-0.09851256931158504
-31599,ELL,0.033809621239378695
-31601,FKBP8,-0.035221409065881316
-31603,KXD1,0.04437575000630813
-31604,AC005253.1,-0.09136033978565117
-31606,UBA52,-0.126403816687493
-31608,REX1BD,-0.0999793311013489
-31610,KLHL26,0.010312637590687358
-31611,CRTC1,0.06841228821130187
-31613,UPF1,0.0951631736862715
-31617,COPE,0.05055030647448062
-31618,DDX49,0.039480983114709725
-31619,HOMER3,0.21410550459433944
-31621,SUGP2,0.031973271421632754
-31622,ARMC6,0.024586289893449274
-31623,SLC25A42,-0.2020322789847506
-31624,TMEM161A,-0.06538189433547686
-31625,MEF2B,0.06409140652451283
-31626,BORCS8,0.040658591560110084
-31627,RFXANK,-0.08161961405142455
-31628,NR2C2AP,-0.20510087418007067
-31632,SUGP1,-0.050285306267945785
-31633,MAU2,0.11361780100140009
-31634,GATAD2A,-0.04762828533898853
-31637,NDUFA13,-0.11591688501905228
-31640,PBX4,-0.13668011255049697
-31642,LPAR2,0.05118411217283183
-31643,GMIP,0.14455249427076222
-31644,ATP13A1,0.1326709209663949
-31645,ZNF101,-0.14448618796292562
-31646,ZNF14,-0.035874370328821355
-31650,ZNF506,-0.07926340354986511
-31652,ZNF253,-0.14596660946513912
-31654,ZNF93,-0.029976454168199666
-31655,ZNF682,-0.01613238576387384
-31656,ZNF90,-0.03996224534238795
-31658,ZNF486,-0.06945192532893422
-31660,ZNF737,-0.16031448359429443
-31662,ZNF626,-0.17324388517577963
-31664,ZNF66,-0.12813646476324725
-31665,ZNF85,0.0005763561050066928
-31667,ZNF430,-0.15780697159450774
-31668,ZNF714,-0.008432933691523219
-31669,ZNF431,-0.07237162390174497
-31670,ZNF708,-0.15583098386056435
-31671,ZNF738,-0.03305618942015001
-31672,ZNF493,-0.09006807717487159
-31675,ZNF429,-0.06778311503642494
-31679,ZNF100,0.08320834088212788
-31681,ZNF43,-0.11987045214294696
-31708,AC092329.4,-0.08871083951258994
-31709,ZNF91,-0.22287959914946018
-31712,ZNF675,-0.10348319288884156
-31713,ZNF681,-0.06820268660503845
-31719,AC092279.1,-0.033873894398646125
-31720,ZNF254,-0.10672310083328763
-31723,LINC00662,-0.008620262804195774
-31725,AC006504.5,0.022972368767200907
-31745,UQCRFS1,0.1235476709602609
-31754,POP4,-0.006019566694016204
-31755,PLEKHF1,-0.4761473789868448
-31756,C19orf12,-0.16655901919306307
-31760,URI1,-0.1549749458773366
-31771,TSHZ3,0.18600415255852434
-31784,ZNF507,-0.10395369992926845
-31786,DPY19L3,0.007084217379098639
-31787,PDCD5,-0.173639221561781
-31788,ANKRD27,0.10934371299103203
-31792,NUDT19,-0.02574817074278208
-31797,CEP89,-0.012784926150710434
-31800,GPATCH1,-0.07935437963137386
-31809,CEBPA,0.3634370940935467
-31813,CEBPG,0.1800281524544441
-31814,PEPD,0.0476412215844334
-31817,KCTD15,0.14902446793463248
-31820,LSM14A,-0.0663323511683215
-31821,KIAA0355,-0.0804370600571544
-31823,GPI,0.07635692116929493
-31824,PDCD2L,-0.09723393885666662
-31826,UBA2,-0.04206664610935096
-31831,ZNF302,-0.0350730002418068
-31833,ZNF181,-0.09113347727602326
-31838,AC008555.4,-0.1612896287784488
-31844,GRAMD1A,-0.03413029924226517
-31853,FXYD1,-0.07185136114060132
-31854,FXYD7,-0.07916156776848313
-31855,FXYD5,-0.005895298691166301
-31857,LSR,-0.07806730993054828
-31860,USF2,0.1670804432187585
-31863,CD22,0.003610205060165965
-31871,FFAR2,0.2179821336904092
-31876,TMEM147-AS1,-0.012573931093417473
-31877,TMEM147,0.03965237033367385
-31883,HAUS5,-0.06947995307765059
-31884,RBM42,0.09733151238643391
-31886,COX6B1,0.001565525719653916
-31890,KMT2B,-0.010269634250548604
-31891,IGFLR1,0.11536194514900783
-31892,U2AF1L4,-0.08278945486564067
-31893,PSENEN,0.0008356083915174178
-31895,LIN37,-0.06974899385101709
-31898,PROSER3,-0.03601416221803384
-31905,NFKBID,0.06543714444743251
-31906,HCST,-0.618117235436427
-31907,TYROBP,0.499782930990521
-31911,SDHAF1,-0.064625951044657
-31913,ALKBH6,0.002348916311648211
-31919,POLR2I,-0.1427551326856941
-31920,TBCB,-0.18471209033767136
-31921,CAPNS1,0.20042597075132665
-31924,ZNF565,-0.07304952188759797
-31925,ZNF146,0.005755521176976411
-31928,ZFP14,-0.15168983408123665
-31931,ZFP82,-0.0733790735405141
-31932,ZNF566,-0.0349579624363996
-31934,ZNF260,-0.08954449353197029
-31936,ZNF529,-0.07093431126328682
-31937,ZNF529-AS1,-0.035888583496940414
-31941,LINC01534,-0.07037266897088554
-31942,ZNF567,-0.09676917361782726
-31946,ZNF790-AS1,-0.09774251181438844
-31947,ZNF790,-0.03802420597111008
-31948,ZNF345,-0.07941314562416371
-31950,ZNF568,-0.1475930268736371
-31951,ZNF420,-0.0783680410988408
-31953,AC010632.2,0.011666740172326497
-31954,ZNF585A,-0.1299103569707002
-31956,ZNF585B,-0.0316491744771851
-31957,ZNF383,-0.05662064198962002
-31961,ZNF875,-0.015987526209675182
-31964,ZNF527,-0.06235951873806962
-31965,ZNF569,-0.1895958010799119
-31966,ZNF570,-0.148131813734073
-31973,ZNF571,-0.05988936261161383
-31979,ZNF573,-0.06331142052712975
-31981,SIPA1L3,0.04619607385054292
-31987,SPINT2,0.24927897378477856
-31988,PPP1R14A,0.005871506473285547
-31990,YIF1B,0.057027200087653634
-31992,KCNK6,0.2974694940402477
-31995,PSMD8,-0.1395477709648004
-31999,FAM98C,-0.0119018074684586
-32000,RASGRP4,0.26674306796423425
-32004,MAP4K1,-0.10107452088330182
-32006,EIF3K,-0.16391887141911216
-32007,ACTN4,-0.022409796352164613
-32009,CAPN12,-0.19283076893676807
-32014,ECH1,-0.18624135092942307
-32016,HNRNPL,0.02841509519945201
-32017,AC008982.2,0.044520859041188694
-32018,RINL,-0.156352468445766
-32020,SIRT2,0.031444183093185196
-32021,NFKBIB,-0.05115723804212018
-32023,SARS2,-0.03252398591291689
-32024,MRPS12,-0.16821664982055756
-32031,PAK4,-0.06149385368046109
-32040,GMFG,0.10167816357434321
-32041,AC011445.2,-0.04281754735015724
-32042,SAMD4B,0.10938547144431593
-32043,PAF1,0.017255193330975135
-32044,MED29,-0.01306256611603622
-32045,ZFP36,0.44383104750015906
-32046,PLEKHG2,0.10970828448298278
-32047,RPS16,-0.32902273788469977
-32048,SUPT5H,-0.008483274468254183
-32049,TIMM50,0.015372174268973377
-32052,EID2B,-0.08503188788536152
-32054,EID2,-0.04748485207514284
-32061,DYRK1B,-0.017685700260643936
-32062,FBL,-0.04960264693598085
-32064,PSMC4,-0.06989785823298526
-32066,ZNF780B,-0.03194845160921013
-32067,ZNF780A,0.07190294182016572
-32069,MAP3K10,0.02012223148291495
-32072,AKT2,-0.01718771323185104
-32075,PLD3,0.1737921756478847
-32078,SERTAD1,-0.0029207610776385254
-32080,SERTAD3,0.1387073153094127
-32082,BLVRB,0.4008905313873433
-32084,SHKBP1,0.21454671924545904
-32085,LTBP4,-0.09195744938312614
-32087,COQ8B,0.034323975813448294
-32089,C19orf54,0.00360475851182852
-32090,SNRPA,-0.0968383825920439
-32092,RAB4B,-0.015864893506000382
-32094,EGLN2,-0.07504881316765784
-32102,CYP2S1,0.11414589939231427
-32105,HNRNPUL1,0.08369823848505878
-32106,TGFB1,0.09751890521106922
-32107,AC011462.5,0.016167519763653344
-32108,CCDC97,-0.0666800840148708
-32109,TMEM91,0.1735690347302328
-32110,B9D2,-0.038735001927438194
-32112,EXOSC5,-0.03294538510884622
-32113,BCKDHA,0.14938990925189738
-32116,B3GNT8,0.10644831801651342
-32117,DMAC2,-0.0658677007184274
-32122,AC243960.1,-0.25903784004099717
-32123,CEACAM21,-0.18036934240575425
-32124,CEACAM4,0.21082515923156997
-32133,RPS19,-0.4226967192773783
-32134,CD79A,-0.0019253042287418554
-32135,ARHGEF1,-0.11214422691698868
-32137,RABAC1,-0.21658250147707933
-32140,ZNF574,0.003543295186825496
-32141,POU2F2,0.48450014408393316
-32142,AC010247.1,0.04229366277077955
-32143,AC010247.2,-0.006061831947437014
-32144,DEDD2,0.05988488704796078
-32145,ZNF526,-0.07809215468084886
-32146,GSK3A,0.04404504913258795
-32147,ERF,0.060272369590380355
-32148,CIC,0.0010301456141940946
-32149,PAFAH1B3,-0.09416127026751828
-32154,LIPE-AS1,0.006685501725951815
-32178,PHLDB3,-0.06113654536392681
-32179,ETHE1,-0.01723729779735348
-32181,XRCC1,0.056753258903423934
-32185,IRGQ,0.10608956955378095
-32186,ZNF576,-0.015391872045781501
-32188,ZNF428,-0.2541572257845223
-32190,PLAUR,0.3268837399256171
-32192,SMG9,0.020565660050864087
-32193,KCNN4,0.10771178132336083
-32196,ZNF283,-0.08624200727938172
-32201,ZNF45,-0.03502627630580012
-32207,ZNF230,0.04223143201010498
-32210,ZNF223,-0.028611764869357213
-32212,ZNF224,-0.08708607045370112
-32214,ZNF225,0.03739096175172539
-32215,ZNF234,-0.041189027547638445
-32217,ZNF226,-0.10766386581592226
-32218,ZNF227,-0.06934576046754497
-32224,ZNF180,0.0596346647232715
-32229,PVR,0.14657136228734896
-32232,BCL3,0.14871742955352324
-32236,NECTIN2,0.1345122404433781
-32238,TOMM40,-0.03331164416265992
-32244,CLPTM1,-0.015675461263087345
-32245,RELB,0.07968476819197824
-32246,CLASRP,0.06207086595122042
-32247,ZNF296,-0.06833559890756101
-32249,GEMIN7,-0.08955648895689865
-32250,MARK4,-0.006011197516319457
-32251,PPP1R37,-0.028690641331322507
-32254,TRAPPC6A,-0.10150313634594844
-32255,BLOC1S3,0.004656632353599937
-32259,ERCC2,0.04461797951548292
-32261,CD3EAP,0.0592942972516841
-32262,ERCC1,0.10340344636912104
-32263,FOSB,0.3561430509162172
-32265,PPM1N,0.04944430238888332
-32266,VASP,0.39355325326584933
-32267,OPA3,0.004567980749931156
-32269,EML2,-0.052833915918888516
-32272,SNRPD2,-0.30146639370071154
-32274,FBXO46,-0.01821936937680887
-32280,DMWD,-0.1401948226644394
-32282,SYMPK,0.20369585622041686
-32285,IRF2BP1,-0.05312731928965722
-32286,MYPOP,0.00804985194248701
-32289,CCDC61,-0.027686931714163452
-32303,PPP5C,-0.11914719167144713
-32313,CALM3,0.07244827881392694
-32315,PTGIR,0.1220353745709846
-32319,PRKD2,-0.026898661144039212
-32320,STRN4,0.19868615469029266
-32322,FKRP,0.02995815499084765
-32323,SLC1A5,0.2101038846383286
-32325,AP2S1,0.2802905082002303
-32326,ARHGAP35,-0.05489633930964868
-32328,TMEM160,-0.1272721416370933
-32329,ZC3H4,0.12577265297695797
-32330,SAE1,-0.02992926109433244
-32331,BBC3,-0.049853954360721035
-32333,CCDC9,0.10419907536034059
-32334,INAFM1,0.021645667475411714
-32335,C5AR1,0.3942667682576428
-32336,C5AR2,0.21583150177070795
-32337,DHX34,0.034905001621365395
-32341,KPTN,0.04716889776539971
-32343,NAPA,0.19862981387298037
-32346,AC010331.1,0.013501058484520701
-32347,BICRA,-0.019132606499963293
-32350,NOP53,-0.28357810249038773
-32352,SELENOW,-0.27020370615665124
-32362,LIG1,-0.12966365138764163
-32365,ZSWIM9,-0.1007983198138403
-32366,CARD8,0.1045795422715315
-32368,CARD8-AS1,0.06068250575122434
-32372,EMP3,0.009757156294930656
-32375,KDELR1,0.21245923533516486
-32377,GRWD1,-0.05616759217788919
-32380,CYTH2,-0.018713115367737565
-32386,RPL18,-0.3547427850634799
-32388,SPHK2,-0.0499246583407313
-32389,DBP,-0.1090362552410183
-32390,CA11,-0.05064957923822667
-32398,BCAT2,-0.027416577733006674
-32402,PPP1R15A,0.22872787646632162
-32404,NUCB1,0.12089254358268575
-32408,BAX,0.05845702648789727
-32410,FTL,0.6374974310854091
-32411,GYS1,-0.02542789873816336
-32412,RUVBL2,0.06956288932132489
-32425,SNRNP70,0.009704776828545475
-32426,LIN7B,-0.08194739656337958
-32430,TRPM4,0.08595725079125315
-32433,CD37,0.10346569559217304
-32442,PIH1D1,-0.09727487331508887
-32443,ALDH16A1,0.09544554140645196
-32444,FLT3LG,-0.2035531135626205
-32445,RPL13A,-0.33253412071951116
-32446,RPS11,0.009686066671817439
-32447,FCGRT,0.5127993134407272
-32448,RCN3,0.0359454098309363
-32449,NOSIP,-0.23770639382262324
-32451,PRR12,-0.0693855342094637
-32453,RRAS,0.195470251497156
-32454,SCAF1,0.045905569571642484
-32455,IRF3,-0.15956186368605285
-32456,BCL2L12,-0.05922692992177569
-32457,PRMT1,-0.09692555968369058
-32462,AP2A1,0.18424020688471637
-32465,MED25,0.03845434815345377
-32467,PTOV1,0.06652173583571051
-32470,PNKP,-0.038121287307721276
-32471,AKT1S1,0.07610268188207998
-32472,TBC1D17,-0.005330685565194641
-32473,IL4I1,0.06792516250890812
-32474,NUP62,0.21246509873698632
-32475,ATF5,0.2781807170572091
-32477,VRK3,-0.06505752404383458
-32486,KCNC3,0.1620790319837647
-32487,NR1H2,0.07809808466066888
-32490,POLD1,0.06666410556385555
-32491,SPIB,-0.0031068452004839363
-32494,EMC10,0.02556348841420001
-32497,JOSD2,0.01578102542918136
-32504,CLEC11A,-0.00042664262395290857
-32509,C19orf48,-0.08858309518271353
-32531,CTU1,-0.050687556575005344
-32532,SIGLEC9,0.2749154663970839
-32533,SIGLEC7,0.1919727072137674
-32535,CD33,0.4043759326283114
-32543,ETFB,0.0050710691452455926
-32546,NKG7,-0.7176278152118245
-32550,SIGLEC10,0.22327360003051858
-32558,ZNF175,0.0017489961277721918
-32560,SIGLEC5,0.17886932651295256
-32561,AC018755.4,0.2736287019335654
-32562,SIGLEC14,0.28533483772082086
-32564,SPACA6,-0.03473357780931659
-32566,FPR1,0.5036563013288317
-32567,FPR2,0.30973738236965753
-32570,ZNF577,0.06592353122153803
-32573,ZNF649,-0.04336056043116191
-32576,ZNF350,-0.017462068469388175
-32577,ZNF615,-0.06905122977614009
-32578,ZNF614,-0.004206120466334758
-32579,ZNF432,-0.07623536409334875
-32582,ZNF841,-0.004454798106023448
-32583,ZNF616,-0.007054728958797083
-32585,ZNF836,-0.04816095754985506
-32587,PPP2R1A,0.06373892039716032
-32589,ZNF766,-0.08188557827044271
-32591,ZNF480,-0.09391667951986128
-32594,ZNF880,-0.12693325454682122
-32596,ZNF528,-0.03030346944433703
-32601,ZNF808,-0.028858636938363536
-32602,ZNF701,-0.12038694598091414
-32604,ZNF83,-0.06345759891889749
-32607,ZNF611,0.016634248599295187
-32608,ZNF600,-0.1966410577814949
-32609,ZNF28,-0.015599875909800732
-32610,ZNF468,-0.10970067990486805
-32614,ZNF816,-0.04636530670017045
-32618,ZNF160,0.03860286342120598
-32620,ZNF347,-0.08117101339462345
-32623,ZNF677,-0.027535321569049323
-32628,ZNF845,-0.08387057880078955
-32629,ZNF525,-0.020644799790504282
-32630,ZNF765,-0.003347773654948124
-32631,AC022137.4,-0.06615745406730145
-32632,ZNF761,-0.032253622245429575
-32633,ZNF813,-0.021659830826964536
-32634,ZNF331,-0.10071701872762803
-32639,NLRP12,0.2410521769397425
-32641,MYADM,0.34018457875654123
-32647,VSTM1,0.2576155598742958
-32649,OSCAR,0.4254590514698113
-32650,NDUFA3,-0.1929014445285984
-32651,TFPT,-0.05891838569458354
-32652,PRPF31,-0.03485026704862188
-32654,CNOT3,0.09981598477925194
-32655,LENG1,-0.1346142517599469
-32657,MBOAT7,0.3202875747877984
-32658,TSEN34,0.22412672815787904
-32660,RPS9,-0.11771619882305247
-32661,LILRB3,0.4839725897488994
-32663,LILRA6,0.28854216427751833
-32665,LILRB2,0.44513010726363805
-32666,LILRA5,0.41658450513929257
-32668,LAIR1,0.12347744368583147
-32674,LENG8-AS1,-0.033229018273330205
-32675,LENG8,0.061666947286139376
-32676,LENG9,0.05321188594090975
-32678,LAIR2,-0.5151897741300032
-32679,LILRA2,0.4131598600743399
-32681,LILRA1,0.35888601939447917
-32682,LILRB1,0.34491129444110163
-32684,LILRB4,0.2786039039256724
-32691,FCAR,0.16785327440574863
-32693,NCR1,-0.3451475818973619
-32698,RDH13,-0.11842324714297327
-32700,PPP1R12C,0.11783979257121381
-32710,TMEM86B,0.019491369612753908
-32711,PPP6R1,0.002919625945301138
-32712,HSPBP1,-0.05194909343182161
-32715,TMEM150B,0.22251638736231272
-32723,TMEM238,-0.17153448334716254
-32724,RPL28,-0.23298331593317062
-32725,UBE2S,-0.041224232616449544
-32727,ISOC2,0.025697756139371117
-32730,ZNF628,0.03109761236812665
-32735,ZNF579,-0.006841653518799298
-32736,FIZ1,0.04943386866878747
-32737,ZNF524,0.06924827186738192
-32740,ZNF784,0.075385056836982
-32741,ZNF580,0.04814713455089357
-32742,ZNF581,0.044275415097354644
-32743,CCDC106,-0.047639767435009917
-32744,U2AF2,0.049003502322394595
-32747,EPN1,0.1437364387282177
-32758,ZNF787,-0.02966762068989088
-32759,ZNF444,-0.06642035270169803
-32763,ZSCAN5A,0.039908569014639274
-32774,ZNF583,-0.05992896659331572
-32793,ZNF264,-0.08926846727833887
-32795,ZNF805,-0.056900753174720334
-32798,ZNF460,-0.03296317298757854
-32799,AC005261.1,0.05027831412872918
-32800,ZNF543,-0.03152543910870507
-32804,TRAPPC2B,-0.1086692905637788
-32805,ZNF548,-0.005805253944344559
-32806,ZNF17,-0.02957545835794791
-32807,ZNF749,-0.05058573589327389
-32812,ZNF419,-0.02296668059394845
-32813,ZNF773,-0.08221088087535426
-32814,ZNF549,-0.012902466534819603
-32815,ZNF550,-0.057020720534077696
-32819,ZNF134,-0.03916144498355634
-32821,ZNF211,-0.06436902607702083
-32823,ZNF551,-0.0062691292904613945
-32824,ZNF154,-0.025043988527475372
-32825,ZNF671,0.04460645894387295
-32826,ZNF776,-0.01033690071443204
-32827,ZNF586,-0.008896999838353644
-32828,ZNF552,-0.028946019721934373
-32829,ZNF587B,-0.024178310239804286
-32830,ZNF814,-0.0338911526924936
-32831,ZNF587,0.057928150125513826
-32833,ZNF417,0.07558106618055226
-32837,ZNF606,-0.03036796353239018
-32838,AC008969.1,-0.03145962166725035
-32842,ZSCAN18,-0.12379772892326617
-32845,ZNF274,-0.06582135241830754
-32846,ZNF544,-0.11886854575139093
-32847,AC020915.2,-0.05415108591081656
-32849,ZNF8,-0.0009094378300607214
-32850,ERVK3-1,-0.12678397681353518
-32851,AC010642.2,0.014691683919854084
-32853,A1BG,0.07777714113245605
-32854,A1BG-AS1,0.0343627058825945
-32859,RPS5,-0.3738638649396007
-32868,ZNF324,-0.06250554776426766
-32869,ZNF446,0.007245630744569584
-32871,SLC27A5,-0.052256294278881996
-32873,ZBTB45,0.006550286637606361
-32874,TRIM28,-0.008216263166316063
-32875,CHMP2A,0.14088498920545547
-32876,UBE2M,-0.0732752071900736
-32878,MZF1,-0.011145556996079475
-32888,ZCCHC3,-0.038264486680300226
-32893,TRIB3,-0.08266925702535155
-32894,RBCK1,-0.021349260325694046
-32895,TBC1D20,-0.058074063568101846
-32896,CSNK2A1,0.09977675466039354
-32898,SRXN1,0.04598986035945822
-32901,FAM110A,0.10834795461587071
-32905,PSMF1,0.029314387220824002
-32911,SDCBP2-AS1,-0.09210392560039739
-32913,FKBP1A,0.4260539132431566
-32914,NSFL1C,0.12421633223394976
-32915,SIRPB2,0.2908556877618683
-32917,SIRPB1,0.36400652611259265
-32918,SIRPG,-0.1521852198097185
-32921,AL117335.1,0.1397608704297619
-32922,SIRPA,0.39066136197060314
-32925,STK35,-0.019287377864260966
-32932,SNRPB,-0.08978490908049311
-32935,NOP56,-0.08363031618994733
-32936,IDH3B,0.03235206869105252
-32942,PCED1A,0.00020164088539985757
-32943,VPS16,0.003049998910440176
-32944,PTPRA,-0.10481631241456571
-32946,MRPS26,-0.1354512308186686
-32950,UBOX5,-0.022237388488355464
-32951,FASTKD5,0.03628732003023615
-32953,DDRGK1,-0.08642910079986042
-32954,ITPA,0.013909625520884924
-32957,C20orf194,0.1273737826026809
-32958,ATRN,0.11691559327761362
-32961,SIGLEC1,0.268028309348012
-32963,C20orf27,0.21674172263773095
-32965,CENPB,0.10280583499088493
-32966,CDC25B,-0.26665874810982376
-32968,AP5S1,-0.019817485792756884
-32969,MAVS,0.16969979591771453
-32971,PANK2,0.22506006704258077
-32973,RNF24,0.24180944232717158
-32980,PRNP,0.20263837814231186
-32984,RASSF2,0.3861532313106453
-32985,SLC23A2,0.04580065331416403
-32990,TMEM230,-0.11744139007241078
-32991,PCNA,-0.1459135753159158
-32992,CDS2,-0.01821433331460174
-33001,GPCPD1,0.3437906256926762
-33002,SHLD1,-0.04114939835932811
-33004,TRMT6,-0.09122394479584933
-33008,CRLS1,0.099186590925187
-33021,TMX4,0.0005365971411373112
-33023,PLCB1,0.05576670932792182
-33039,MKKS,-0.03188950679934006
-33042,SLX4IP,-0.00790322149584744
-33055,BTBD3,0.09939031999788872
-33068,TASP1,-0.08789192863354021
-33069,ESF1,-0.05183728517083539
-33070,NDUFAF5,-0.09313009701367857
-33081,KIF16B,-0.031541021696684886
-33085,SNRPB2,-0.06359982482240352
-33093,DSTN,-0.36392115530094743
-33094,RRBP1,0.14169551513400563
-33097,SNX5,0.11376116845990303
-33099,MGME1,0.0011846025053525028
-33101,PET117,-0.04627086440031584
-33102,KAT14,0.06051022260146624
-33104,ZNF133,-0.03550271323938985
-33109,POLR3F,-0.08995796343677091
-33110,RBBP9,0.04684825144360244
-33111,SEC23B,0.09127015503023388
-33112,SMIM26,-0.034129229698698665
-33113,DTD1,-0.1340374929132813
-33116,LINC00653,-0.09778899380347598
-33126,RIN2,0.25566897499972785
-33127,NAA20,-0.12080403901404399
-33128,CRNKL1,-0.03311319666182666
-33134,RALGAPA2,0.10180090545062374
-33138,KIZ,-0.05860980488025344
-33142,XRN2,0.15289148352414822
-33165,THBD,0.12519541614382645
-33166,CD93,0.4752463252996572
-33171,NXT1,-0.07123337779459044
-33173,GZF1,0.06007505268620964
-33174,NAPB,0.04148198784015737
-33183,CST3,0.6165501647697734
-33201,CST7,-0.6955115050508607
-33202,APMAP,-0.34440480377301197
-33204,ACSS1,0.020879348209302154
-33209,ENTPD6,0.06620089521604658
-33212,PYGB,0.12504439559573657
-33215,ABHD12,0.05677590556020591
-33219,ZNF337-AS1,-0.05307708834770062
-33220,ZNF337,-0.06225787420361267
-33246,HM13,0.27736450090569753
-33254,BCL2L1,-0.12101298356796716
-33262,PDRG1,-0.057272057831270935
-33267,HCK,0.5129525037736945
-33268,TM9SF4,0.17204919840448885
-33270,PLAGL2,0.17986025687729734
-33271,POFUT1,0.05516600801890599
-33273,KIF3B,0.04028581408164596
-33275,ASXL1,-0.0767827979081858
-33276,NOL4L,-0.11852869496293349
-33284,COMMD7,-0.059357434882394054
-33286,MAPRE1,0.07467941685863422
-33288,AL035071.1,0.04032775312790983
-33300,CDK5RAP1,0.013573962900382564
-33302,CBFA2T2,0.06503033416379261
-33304,NECAB3,0.004911519548317953
-33309,PXMP4,0.0027554785428261877
-33313,CHMP4B,0.09229319686331611
-33315,RALY,0.04774697063227365
-33317,EIF2S2,0.01086729102916085
-33321,AHCY,0.05571401178857257
-33323,ITCH,0.06670588273434178
-33327,DYNLRB1,-0.16704686873175575
-33330,NCOA6,0.09931453139459671
-33334,ACSS2,0.15763221507484862
-33335,GSS,-0.015542372282696981
-33337,TRPC4AP,0.09427000655696426
-33338,EDEM2,0.09285025363707719
-33341,MMP24OS,-0.020845788964450718
-33343,EIF6,-0.016275211635762114
-33346,UQCC1,-0.022268699899575195
-33349,CEP250,-0.16430905099319928
-33350,FO393401.1,-0.07353028109233621
-33352,ERGIC3,0.16844430688831744
-33354,CPNE1,-0.06655877109558068
-33355,RBM12,0.1563751463742801
-33356,NFS1,0.02066396761319031
-33357,ROMO1,-0.09757160428802135
-33358,RBM39,-0.05260671919096399
-33359,PHF20,-0.04654887000218286
-33360,SCAND1,0.06286933018698675
-33361,CNBD2,-0.11820831658076096
-33362,NORAD,0.14951759903456646
-33369,AAR2,0.059301063899838836
-33370,DLGAP4,0.050938196654254496
-33373,TGIF2,0.048219873090084096
-33374,RAB5IF,0.32561993325898003
-33375,SLA2,-0.04067083012808659
-33376,NDRG3,-0.09069604938582204
-33377,DSN1,-0.046149686877852224
-33378,SOGA1,0.14051500418095353
-33380,SAMHD1,0.5029263522946887
-33381,RBL1,-0.08777815628295421
-33384,RPN2,0.07728306065668428
-33386,MANBAL,0.02839641960017401
-33388,SRC,0.18450570640970837
-33389,BLCAP,0.1440561244299066
-33395,CTNNBL1,-0.009647528129932548
-33397,TTI1,0.01651320714707415
-33398,RPRD1B,0.07052607864850564
-33406,BPI,0.1418391052475024
-33411,SNHG17,0.024000950450715
-33412,SNHG11,-0.009410717679729815
-33413,RALGAPB,0.033760450402542865
-33417,ACTR5,0.003628973029260694
-33418,PPP1R16B,-0.11804940599743889
-33422,DHX35,-0.010643874978891597
-33431,MAFB,0.4540097405077347
-33434,TOP1,0.07512399338424833
-33436,PLCG1,-0.130916113812551
-33437,ZHX3,0.006314743176119382
-33440,CHD6,-0.08061366971453235
-33448,SRSF6,0.10345153935382997
-33449,L3MBTL1,-0.00857038737732093
-33453,IFT52,-0.00026295693645678116
-33460,OSER1,0.010994546840666706
-33461,OSER1-DT,-0.06803669637664572
-33470,TTPAL,-0.06659036715308726
-33471,SERINC3,0.10097096800336099
-33472,PKIG,-0.022992196060406207
-33473,ADA,-0.17842467873989812
-33481,YWHAB,0.1512086528650779
-33482,PABPC1L,-0.005007286169522669
-33483,TOMM34,-0.044043930790170084
-33485,STK4,-0.2064789196548999
-33497,SYS1,-0.07312992319445817
-33499,DBNDD2,-0.11335173637379023
-33500,PIGT,0.03481084551201677
-33515,DNTTIP1,0.18036476488349534
-33519,ACOT8,0.022742825757917133
-33524,CTSA,0.17592011404657468
-33527,AL162458.1,0.009725740823100648
-33528,PCIF1,0.028471631202786802
-33529,ZNF335,-0.017779278479471812
-33533,NCOA5,0.060558881926342636
-33534,CD40,0.11806730212721815
-33537,SLC35C2,0.014665539120493275
-33539,ELMO2,0.03215984032671182
-33544,TP53RK,-0.0582368687878638
-33549,ZMYND8,-0.042728534802699975
-33554,NCOA3,0.1992519554119081
-33555,SULF2,0.4071200273168332
-33573,PREX1,-0.010329648402502377
-33575,AL133342.1,0.005977100774265159
-33576,ARFGEF2,0.06421494028119643
-33578,CSE1L,0.08786385830413886
-33579,STAU1,0.039157323777401214
-33580,DDX27,-0.013928994794249862
-33581,ZNFX1,0.23134706775163172
-33582,ZFAS1,-0.017334809462123058
-33585,B4GALT5,0.11023113994291807
-33586,SLC9A8,0.007348246856439122
-33587,SPATA2,0.02665968524330142
-33588,RNF114,0.13325241219353715
-33591,UBE2V1,-0.11445701862311955
-33592,TMEM189,0.0704645481229059
-33597,CEBPB,0.5125736013245366
-33598,SMIM25,0.35157191236559915
-33601,PTPN1,0.045143278176158354
-33606,BCAS4,-0.14074292753966125
-33607,ADNP,-0.0314847826147417
-33609,DPM1,-0.052063143133141886
-33610,MOCS3,0.023960781420406953
-33613,NFATC2,-0.2811095641392935
-33617,ZFP64,0.020020757877190216
-33625,TSHZ2,-0.0595194646194698
-33633,ZNF217,0.22634003059967767
-33641,PFDN4,-0.06641007133183398
-33652,FAM210B,0.05552081340881139
-33654,CSTF1,0.004178037159590319
-33656,RTF2,0.004697064181192724
-33671,RAE1,0.06666009968208837
-33674,RBM38,-0.1568672671632006
-33679,ZBP1,-0.10648216829962344
-33680,PMEPA1,-0.02375603267766945
-33687,RAB22A,-0.014609982463377963
-33688,VAPB,-0.011726974220921926
-33693,STX16,-0.039211988591182974
-33694,NPEPL1,0.10265170485657485
-33700,GNAS,0.12376023819326097
-33704,NELFCD,-0.081511491099719
-33705,CTSZ,0.5859786895246021
-33707,ATP5F1E,0.20897362209781434
-33708,PRELID3B,-0.032443385641819686
-33709,ZNF831,-0.14427243100029993
-33718,SYCP2,-0.018582885305542654
-33719,FAM217B,0.019683100369238017
-33720,PPP1R3D,0.12575185422538304
-33724,MIR646HG,-0.05278908611026418
-33735,TAF4,0.06921532318016844
-33737,LSM14B,0.03509138105694297
-33738,PSMA7,-0.03800255910584888
-33739,SS18L1,0.08077019062147775
-33740,MTG2,-0.00868175588553859
-33742,OSBPL2,0.05107214818859964
-33743,ADRM1,0.0013063265087261673
-33748,RPS21,-0.4035822896009248
-33771,MRGBP,0.008907096217259556
-33773,OGFR,0.22220010552369066
-33775,TCFL5,0.030158830553638105
-33776,DIDO1,-0.07026644673674895
-33777,AL117379.1,-0.03503212994536042
-33778,GID8,0.07174157842014701
-33779,SLC17A9,-0.06577945621482548
-33790,YTHDF1,0.09961491201731776
-33795,ARFGAP1,0.09989024106638145
-33804,PPDPF,-0.27795821649951685
-33809,HELZ2,0.1905619977993025
-33810,GMEB2,-0.08369747912725113
-33811,MHENCR,-0.08824558461795633
-33812,STMN3,-0.17457935148972992
-33813,RTEL1,0.019713306100954658
-33815,ARFRP1,-0.05453783559086323
-33816,ZGPAT,-0.08309747694327564
-33818,LIME1,-0.4620159031189959
-33819,SLC2A4RG,-0.23098766797404094
-33825,TPD52L2,0.24525054062807325
-33826,DNAJC5,0.12406903585440344
-33827,UCKL1,-0.08479452127564441
-33830,SAMD10,-0.09134613294682145
-33831,PRPF6,0.019561598068707994
-33832,C20orf204,-0.08143946115801075
-33834,TCEA2,-0.01794718873605022
-33836,RGS19,0.16329361845124024
-33837,OPRL1,0.11427594523732691
-33842,PCMTD2,-0.048181027451935886
-33847,GATD3B,-0.01983436580918249
-33848,FP565260.1,-0.03426034450381565
-33849,FP565260.6,0.049976375466418775
-33864,U2AF1L5,-0.02092328765537915
-33906,HSPA13,0.06852050447013289
-33907,SAMSN1,0.08469991718310328
-33913,NRIP1,0.24010517838157275
-33919,USP25,0.20290739163411203
-33929,BTG3,-0.0465692843348301
-33933,C21orf91,0.015229766452800768
-33973,LINC01684,-0.16572847808316077
-33985,MRPL39,-0.045057902454196344
-33988,ATP5PF,-0.049847265631823394
-33989,GABPA,0.10965217186630918
-33990,APP,0.2348799667046799
-34002,ADAMTS5,0.1338566468292555
-34013,N6AMT1,-0.03690858125396815
-34014,LTN1,0.03680942195371345
-34015,RWDD2B,-0.0008333606377459166
-34017,USP16,-0.11225533976195284
-34018,CCT8,0.03288119440906685
-34021,MAP3K7CL,0.08960395463881679
-34024,BACH1,0.42510269116510724
-34026,AP000240.1,0.1070479245797848
-34070,TIAM1,0.14208135941819094
-34074,SOD1,-0.2849073658012511
-34076,SCAF4,0.06535088944949845
-34082,MIS18A,-0.06902776978808553
-34086,URB1,-0.01783209521301879
-34087,URB1-AS1,-0.0636920111017173
-34090,CFAP298,-0.06649225088379025
-34091,SYNJ1,0.07372008798158938
-34092,PAXBP1-AS1,0.10740627757638532
-34093,PAXBP1,0.07776648646394853
-34094,C21orf62-AS1,-0.05517554808974308
-34101,OLIG1,0.16900516406782592
-34105,IFNAR2,0.05452563010830844
-34106,IL10RB-DT,0.09857755423510545
-34107,IL10RB,0.2222028891771736
-34108,IFNAR1,0.19445827729166806
-34109,IFNGR2,0.46718109642303157
-34110,TMEM50B,-0.03900453458453563
-34113,GART,-0.04931417636340137
-34114,SON,-0.028840434374763656
-34116,CRYZL1,0.004609477422699713
-34117,ITSN1,0.2303435732048944
-34118,ATP5PO,-0.08858090865510501
-34119,LINC00649,-0.18854634930154518
-34121,MRPS6,-0.178681087929345
-34122,SLC5A3,-0.09615451130845445
-34132,KCNE1,0.16261078728753509
-34133,RCAN1,0.07616180911495354
-34137,RUNX1,0.1915297078175525
-34142,SETD4,-0.0586258345741801
-34145,CBR1,0.1756534574258446
-34148,CBR3-AS1,-0.004765142071064847
-34149,CBR3,-0.0639197497966682
-34150,DOP1B,0.06887384224303326
-34152,MORC3,0.03980122046371531
-34153,AP000692.1,0.032034322103322084
-34163,HLCS,-0.08637426587655608
-34167,PIGP,-0.14483727627140608
-34168,TTC3,0.011016844317695194
-34173,VPS26C,-0.002892073715047047
-34177,DYRK1A,0.05213427284143468
-34191,ETS2,0.25409822615327926
-34199,PSMG1,-0.04804196266522157
-34200,BRWD1,0.04993141901050437
-34205,HMGN1,-0.20140291245135047
-34206,GET1,0.02272666862008927
-34221,BACE2,0.005623152101491478
-34225,MX2,0.3228232246404514
-34226,MX1,0.3424892530622765
-34237,PRDM15,0.06122833500485892
-34240,C2CD2,0.13823466535368356
-34241,ZBTB21,0.05977222871708139
-34245,ABCG1,-0.10340424965882929
-34250,UBASH3A,-0.1803020690084753
-34252,SLC37A1,0.012492887803213054
-34263,WDR4,0.032727090387464845
-34264,NDUFV3,0.06504713826480317
-34267,PKNOX1,0.018464366455873495
-34283,RRP1B,-0.13213570429553714
-34284,PDXK,0.23181341365986038
-34286,CSTB,0.0043099000532800585
-34287,RRP1,-0.03114298420693273
-34288,AATBC,0.2088504657079268
-34289,AGPAT3,0.21920993907219685
-34290,TRAPPC10,-0.17927996141571462
-34293,LINC01678,0.04445819544577478
-34297,ICOSLG,0.061754530417389354
-34303,PFKL,0.11507128385606527
-34305,CFAP410,-0.007088546575787054
-34309,TRPM2,0.09651709875149192
-34336,UBE2G2,0.0436678595393529
-34338,SUMO3,0.18193316972185256
-34339,PTTG1IP,0.2691393198848392
-34340,ITGB2,0.3096056337895797
-34341,ITGB2-AS1,0.03259373099249871
-34344,AL844908.1,0.12131215717000814
-34345,FAM207A,-0.05339948178670664
-34350,ADARB1,-0.14531310682522106
-34353,POFUT2,0.03062971145596596
-34354,LINC00205,0.03300632795614516
-34361,COL18A1,-0.04736471482139425
-34364,SLC19A1,0.19140625315964221
-34380,COL6A2,-0.2783773833205781
-34383,SPATC1L,-0.03410973372175591
-34385,LSS,-0.023833373122181978
-34387,MCM3AP-AS1,-0.13316569719586957
-34388,MCM3AP,0.026374267943741137
-34391,YBEY,-0.12105245394275924
-34393,PCNT,-0.02230030532492002
-34395,DIP2A,-0.1472630929645011
-34397,S100B,-0.1260152750224746
-34398,PRMT2,-0.3296861739436104
-34429,IL17RA,0.43388077897646854
-34430,TMEM121B,0.11715975128709188
-34432,HDHD5,0.07398971798884516
-34434,ADA2,0.5010115227194754
-34440,ATP6V1E1,0.1626638635200167
-34441,BCL2L13,0.0998718268276393
-34442,BID,0.34370814189719456
-34443,LINC00528,0.012593763667371957
-34444,MICAL3,-0.0543104618491647
-34450,PEX26,0.07657236332025706
-34453,USP18,0.05553421695957406
-34461,TMEM191B,-0.09857379675736212
-34473,DGCR2,0.09052959022565664
-34478,ESS2,0.021300105165626828
-34482,SLC25A1,0.06013144228042225
-34486,HIRA,0.05306356490662157
-34487,C22orf39,-0.02459521204485379
-34488,MRPL40,-0.12587873596780275
-34490,UFD1,-0.12546044454930236
-34502,RTL10,-0.046566366304405
-34503,TXNRD2,0.17139978459610603
-34504,COMT,0.18644362497132008
-34506,TANGO2,0.09076060236938618
-34509,DGCR8,-0.09810128928327279
-34511,TRMT2A,0.03166789711869719
-34512,RANBP1,-0.04005276571280924
-34513,ZDHHC8,-0.005015417041529991
-34518,DGCR6L,-0.0842436198559175
-34527,KLHL22,-0.05170196022596826
-34528,MED15,0.09812160287141665
-34530,PI4KA,0.04406189329535103
-34532,SNAP29,0.06414940284928164
-34534,CRKL,0.04300908944074044
-34535,LINC01637,0.17078187903879505
-34536,AIFM3,0.11934011636644545
-34538,LZTR1,0.034192587789753606
-34539,THAP7,-0.05698002150295869
-34540,THAP7-AS1,-0.06171685486351674
-34554,HIC2,0.07873324000457946
-34555,TMEM191C,-0.021802846060731212
-34557,UBE2L3,-0.014937520419435177
-34558,YDJC,-0.04436589100604576
-34561,SDF2L1,-0.15709536695552986
-34563,PPIL2,-0.06961398401456685
-34564,YPEL1,-0.2891883718247456
-34567,MAPK1,0.024874830990564728
-34568,PPM1F,0.3189781710975638
-34569,AC245452.1,0.053875905102213566
-34570,TOP3B,0.009311350455389437
-34592,AC245060.6,-0.14072949097527207
-34593,AC245060.5,0.022973152855409004
-34657,IGLC1,0.0009006594513248822
-34659,IGLC2,-0.0019122625168397835
-34661,IGLC3,-0.002100552377528111
-34673,BCR,-0.03744896982895483
-34684,RGL4,-0.24493553639887755
-34686,VPREB3,-0.0004450065624121173
-34688,CHCHD10,0.1408593156086575
-34690,SMARCB1,-0.0024567883120507543
-34693,SLC2A11,-0.05012434988747945
-34695,MIF,-0.40895651616237144
-34701,DDT,-0.041335861917279525
-34703,CABIN1,-0.06147499952260878
-34707,SPECC1L,0.022023728875807596
-34708,ADORA2A,-0.11850193420064367
-34712,GUCD1,0.10565221213556876
-34713,SNRPD3,-0.08321120661156972
-34740,GRK3,0.3894594919076004
-34750,ASPHD2,0.12879406457540543
-34751,HPS4,-0.025726505569745515
-34752,SRRD,-0.02087012055790631
-34753,TFIP11,-0.08170730857198093
-34755,TPST2,-0.1906620213714818
-34759,MIAT,-0.12443060721575026
-34761,MIATNB,-0.036809148641056985
-34784,PITPNB,-0.0656383027639427
-34785,TTC28-AS1,-0.04670659393723048
-34787,CHEK2,0.0014714971185082457
-34788,HSCB,-0.07857833565166841
-34789,CCDC117,0.03547952468636783
-34790,XBP1,-0.29367903767304965
-34791,Z93930.2,-0.11104432704683281
-34797,KREMEN1,0.12592233388340898
-34800,RHBDD3,0.03850334083491867
-34802,EWSR1,0.14605952317976853
-34803,GAS2L1,0.07357936119808767
-34806,AP1B1,0.20446054601210234
-34811,THOC5,-0.03336444620181149
-34813,NIPSNAP1,-0.09860044539413709
-34814,NF2,0.0024910342270484543
-34818,ZMAT5,-0.09269154097496289
-34820,UQCR10,-0.043443171612326366
-34821,ASCC2,-0.0350610474424689
-34822,MTMR3,0.20129109729002448
-34823,AC003681.1,0.019419827215052032
-34831,CASTOR1,-0.07540311390766388
-34832,TBC1D10A,-0.046245239711307784
-34833,SF3A1,0.040162923961105886
-34835,RNF215,0.07138064512001963
-34839,MTFP1,-0.1770275105727103
-34846,PES1,0.10882882912412344
-34848,TCN2,0.1832523680518305
-34849,SLC35E4,0.11566154673590068
-34850,DUSP18,0.15179512788630165
-34852,MORC2-AS1,-0.11579561964602356
-34853,MORC2,-0.08778712568915611
-34854,TUG1,0.03508569764133396
-34860,SELENOM,-0.15188868245498682
-34863,RNF185,0.08081741871230366
-34865,LIMK2,0.06302408027638935
-34866,PIK3IP1,-0.1880593319901471
-34868,PATZ1,0.017490650027270235
-34870,DRG1,0.06837787642992106
-34871,EIF4ENIF1,0.02477937544764609
-34873,SFI1,-0.14286195205153274
-34875,PISD,0.2557163183670793
-34876,PRR14L,0.10943801027169456
-34877,DEPDC5,0.04275627648709521
-34879,YWHAH,0.18145613099705868
-34897,RTCB,0.07955451471754496
-34899,FBXO7,0.0422517357976952
-34920,HMGXB4,-0.0032853277213624288
-34922,TOM1,0.20116106223473595
-34925,HMOX1,0.30079402166788577
-34926,MCM5,-0.06301776784496195
-34931,APOL6,0.26153584676681796
-34939,APOL3,0.13016533705986208
-34940,APOL4,0.12992234367059471
-34941,APOL2,0.038810707359676455
-34942,APOL1,0.14487920014457706
-34943,MYH9,0.03501868389781158
-34944,Z82215.1,-0.07533372383781967
-34947,TXN2,-0.04295879252795731
-34949,EIF3D,0.00017899378329054558
-34955,IFT27,-0.022022720138437757
-34960,NCF4,0.3414342641207019
-34961,CSF2RB,0.3656156815967857
-34964,TST,0.17530008349436776
-34965,MPST,0.052787038547516756
-34967,KCTD17,-0.03181301498046146
-34970,IL2RB,-0.5235493378666132
-34974,RAC2,-0.37444575811586617
-34975,CYTH4,0.12118768054539698
-34980,MFNG,-0.062061279669090426
-34983,CDC42EP1,0.13650087694743587
-34984,LGALS2,0.44657564059567123
-34985,GGA1,-0.03132754377667306
-34986,SH3BP1,0.16654537115068743
-34988,PDXP,-0.1559580098117067
-34989,LGALS1,0.3125401585419577
-34990,NOL12,-0.04414852858539877
-34991,TRIOBP,0.3053932088643465
-34992,H1F0,0.13294573930601894
-34995,ANKRD54,-0.047407925857035176
-34996,EIF3L,0.18397309254292488
-34997,AL022311.1,0.029473179255736632
-34998,MICALL1,0.10053302313463222
-35001,POLR2F,-0.036320940317953625
-35005,PICK1,-0.049055893842665836
-35012,MAFF,-0.07508208256574025
-35013,TMEM184B,-0.012953263947394685
-35015,CSNK1E,-0.06948379745121323
-35022,DDX17,0.1785408468710487
-35025,CBY1,-0.03772030919487548
-35029,TOMM22,-0.05088302908168821
-35030,JOSD1,-0.03246195693818418
-35031,GTPBP1,0.01950288647298539
-35033,SUN2,-0.1748310244891453
-35039,DNAL4,-0.08324533614606679
-35041,CBX6,0.07015719198924399
-35044,APOBEC3A,0.31063989864452374
-35045,APOBEC3B,0.12342624062116413
-35047,APOBEC3C,-0.12552448348236883
-35048,APOBEC3D,-0.11045041144446999
-35049,APOBEC3F,-0.03717198657235934
-35050,APOBEC3G,-0.4997838702541362
-35052,CBX7,-0.06453437621644571
-35053,AL031846.2,-0.12121640746869401
-35057,RPL3,-0.5027895159148994
-35058,SYNGR1,0.042924541502080873
-35059,TAB1,0.006597289956135167
-35062,MIEF1,0.06551103272794925
-35064,ATF4,0.09579595677467698
-35065,RPS19BP1,-0.1377509767466812
-35068,GRAP2,-0.157443998898656
-35069,Z82206.1,-0.14250922965145452
-35072,TNRC6B,-0.02213244277317093
-35073,ADSL,0.028174939555305624
-35074,SGSM3,-0.08941394420470837
-35076,MRTFA,0.09796961841700996
-35080,SLC25A17,-0.03969252581492708
-35081,ST13,-0.09525256505575476
-35082,XPNPEP3,0.008678279648643003
-35084,RBX1,0.03994061215670347
-35085,EP300,0.232690342841137
-35088,L3MBTL2,-0.006240436065619491
-35089,AL035681.1,-0.07618351560124804
-35091,RANGAP1,0.004256296499987717
-35092,ZC3H7B,-0.012277865365205764
-35095,TOB2,0.056105207369955515
-35096,PHF5A,-0.014325019681326387
-35097,ACO2,0.1309227376579434
-35098,POLR3H,-0.0415937900417423
-35101,PMM1,-0.06460240772220857
-35102,DESI1,0.12919273472483114
-35103,XRCC6,0.14275079681270397
-35104,SNU13,-0.10487517185833437
-35105,MEI1,-0.11022277117053646
-35108,SREBF2,0.007081169454938743
-35110,TNFRSF13C,0.0005674993646916973
-35116,NAGA,0.34540604780599343
-35119,SMDT1,-0.12886934661434335
-35120,NDUFA6,-0.09250754708542933
-35126,TCF20,0.0027897052570357538
-35129,NFAM1,0.37648274241820906
-35131,RRP7A,0.08896548615371004
-35133,POLDIP3,0.15081980953370627
-35134,Z93241.1,0.11733284397498123
-35135,CYB5R3,0.15776056634814667
-35138,ARFGAP3,0.02252758992613604
-35139,PACSIN2,0.12571392362249367
-35141,TTLL1,-0.03784201549247648
-35143,MCAT,-0.028381826943949192
-35144,TSPO,0.4797375783076793
-35145,TTLL12,0.007013329555074291
-35159,SAMM50,-0.014837100576021917
-35160,PARVB,0.14676275496751592
-35162,AL031595.3,0.1356353197848045
-35163,PARVG,0.2298317747247783
-35168,RTL6,-0.006903231264990642
-35171,PRR5,-0.4247655472445546
-35175,NUP50-DT,0.045353437698136996
-35177,NUP50,0.10279694332924239
-35179,KIAA0930,0.40347372708124074
-35181,UPK3A,0.12815711015451836
-35182,FAM118A,0.01230336896131335
-35188,Z95331.1,0.02212963588674078
-35189,ATXN10,0.010679840713482892
-35197,PRR34-AS1,0.02436944929859854
-35198,MIRLET7BHG,0.06469757262578363
-35201,PPARA,0.052659454007134676
-35205,TTC38,-0.27906951829876336
-35208,TRMU,-0.12681866920087778
-35211,GRAMD4,0.1124741467344477
-35212,CERK,0.0005538290483002792
-35213,AL118516.1,0.009267474037349094
-35214,TBC1D22A,0.12113940497606132
-35236,C22orf34,0.0023688943029304107
-35241,Z97192.3,0.02811934009564145
-35244,BRD1,-0.03267653809561524
-35248,ZBED4,0.012028153740499257
-35249,ALG12,0.017929424876171326
-35251,CRELD2,-0.1140985993112582
-35253,PIM3,0.07411715076900174
-35256,MLC1,-0.09215423596234038
-35260,TRABD,0.056056850313986224
-35263,SELENOO,0.12374654945856219
-35266,TUBGCP6,-0.034983665046279214
-35267,HDAC10,0.01265463364728645
-35270,PLXNB2,0.38316648031158246
-35271,DENND6B,0.16048979787495188
-35274,PPP6R2,-0.038890768285675476
-35275,SBF1,-0.05366552458875341
-35278,LMF2,0.004530179291987092
-35279,NCAPH2,-0.049695173497394524
-35280,SCO2,0.464504217552165
-35282,TYMP,0.654644856651401
-35283,ODF3B,0.37314140775711396
-35284,U62317.4,0.14530863597270208
-35286,KLHDC7B,0.11311599989114351
-35288,CPT1B,-0.0025186762173096456
-35289,CHKB,0.02226768580231088
-35290,CHKB-DT,-0.044169942562104
-35293,ARSA,0.029197008570256904
-35299,RABL2B,-0.01912856736955762
-35300,PLCXD1,0.04739973230233509
-35301,GTPBP6,-0.06227397230940443
-35302,LINC00685,-0.009111763706615835
-35303,PPP2R3B,0.005725740992937892
-35309,CSF2RA,0.30870945606331035
-35310,IL3RA,0.03442975780504273
-35311,SLC25A6,0.2131137867036301
-35314,ASMTL,-0.003488245228472566
-35315,P2RY8,-0.00494814440889657
-35316,AKAP17A,0.0054587450930288365
-35320,DHRSX,0.025972156811990908
-35322,ZBED1,0.07102916795915164
-35324,CD99,-0.363791018620875
-35328,ARSD,0.2545245520388918
-35335,PRKX,0.10401557941533845
-35337,BX890604.2,-0.14160721163239176
-35343,PUDP,0.13917303651162222
-35344,STS,0.11931544685043015
-35346,PNPLA4,-0.07524037381320169
-35357,TBL1X,0.17819895041463085
-35361,WWC3,0.0117059012809619
-35367,HCCS,0.11305556954872607
-35371,MSL3,0.14923043392492857
-35374,PRPS2,0.03654078981035567
-35375,TLR7,0.28604757052674357
-35377,TLR8,0.38973048183690434
-35378,TMSB4X,-0.06957524206187676
-35386,TCEANC,-0.020719414649394273
-35387,RAB9A,-0.13789682605243564
-35388,TRAPPC2,-0.17942833432599664
-35389,OFD1,-0.20979861172233394
-35390,GPM6B,-0.11545519326632082
-35393,GEMIN8,-0.05875381702389909
-35395,FANCB,-0.024422204397564748
-35396,MOSPD2,0.24399327125875944
-35399,PIGA,-0.001787303375573558
-35406,CA5B,-0.052354266850231354
-35408,ZRSR2,0.01711554705060783
-35409,AP1S2,0.5429509810913761
-35415,SYAP1,-0.08174724384253611
-35416,TXLNG,-0.13629449765474078
-35417,RBBP7,-0.10702185138500248
-35418,REPS2,0.12801302416796684
-35422,SCML1,-0.0019557353990805595
-35427,CDKL5,0.059973051989471435
-35432,PHKA2,0.08270300660627995
-35435,PDHA1,-0.01698568292137743
-35437,SH3KBP1,0.09335036396922089
-35438,BCLAF3,0.028574963609682553
-35440,EIF1AX,4.06957554152503e-05
-35442,RPS6KA3,0.11575617593947234
-35447,CNKSR2,-0.04100334447045688
-35450,MBTPS2,-0.0160192577508903
-35452,SMS,0.1045348530596026
-35461,PRDX4,-0.027077379292058487
-35462,ACOT9,0.20698625495415873
-35464,SAT1,0.5103736169292116
-35465,APOO,-0.052109225597550435
-35467,KLHL15,0.03917601367462345
-35468,EIF2S3,0.17119337473886836
-35470,ZFX,-0.03394199786247643
-35471,PDK3,0.06966308179039123
-35474,POLA1,0.009933551112192147
-35496,CXorf21,0.29831980850891304
-35497,GK,0.28497433820900525
-35501,TAB3,0.14582310704538423
-35522,CYBB,0.6601776060218929
-35523,DYNLT3,-0.07747422824204775
-35530,RPGR,0.02003579565126187
-35535,MID1IP1,0.21251537878726431
-35544,BCOR,0.09105695887487353
-35546,ATP6AP2,0.2466257559484598
-35549,CXorf38,0.169532408985871
-35550,MED14,-0.019185276924918757
-35553,USP9X,0.028871558200980932
-35555,DDX3X,0.2460736487093401
-35557,CASK,-0.013626098677877039
-35559,GPR34,0.06985276048395408
-35560,GPR82,-0.08867338227832014
-35572,FUNDC1,-0.0002343093397555715
-35574,KDM6A,0.06252159632678769
-35578,MIR222HG,0.06686493384925506
-35584,KRBOX4,-0.037258160331320754
-35585,ZNF674,-0.03583075151897401
-35587,CHST7,0.015638030688596604
-35588,SLC9A7,0.006461674913111695
-35589,RP2,0.33692592206941935
-35593,NDUFB11,-0.13113301661003054
-35594,RBM10,0.022928227172891984
-35595,UBA1,0.09198466838300111
-35597,CDK16,0.024497510826011485
-35598,USP11,-0.19739746048477377
-35601,ZNF41,-0.05270501229343836
-35603,ARAF,0.11756585414703227
-35605,TIMP1,0.21373810751940797
-35606,CFP,0.5352854131375986
-35607,ELK1,0.08749124488841512
-35608,UXT,-0.26786631686118817
-35611,ZNF182,0.02748663142957317
-35624,FTSJ1,-0.030343045264646866
-35626,PORCN,-0.024153258375088693
-35627,EBP,-0.24614702099531455
-35628,TBC1D25,0.0013610714673084874
-35630,RBM3,0.1541019198037292
-35632,WDR13,0.08204394348621619
-35633,WAS,0.27937805476487076
-35638,HDAC6,0.012882395973393747
-35640,PCSK1N,-0.06520732704228259
-35641,PQBP1,-0.04521604121096473
-35642,TIMM17B,-0.028352872160821482
-35643,SLC35A2,-0.07645557014553267
-35644,PIM2,-0.1675942567303808
-35645,OTUD5,0.007005504037236396
-35647,GRIPAP1,0.12622448470300082
-35648,TFE3,0.08024507290936034
-35650,PRAF2,-0.035933587050234105
-35651,WDR45,-0.015964861336888955
-35652,GPKOW,-0.045330383190864736
-35654,PLP2,0.1155199221396374
-35655,PRICKLE3,0.10862106794282611
-35659,CCDC22,0.037521487056724805
-35662,PPP1R3F,-0.09062941613007315
-35682,CLCN5,0.15685039598846437
-35699,GSPT2,-0.10184666957179156
-35700,MAGED1,-0.06634031268378532
-35720,TSPYL2,-0.046212342333704755
-35722,KANTR,-0.003407885320216794
-35723,KDM5C,0.022714394473556734
-35725,IQSEC2,0.06313429945063183
-35726,SMC1A,0.10739923487371174
-35728,HSD17B10,-0.05251045288137271
-35730,HUWE1,0.015805562436102275
-35731,PHF8,0.034776966285901
-35732,FAM120C,-0.044626117573182585
-35734,TSR2,-0.08761855112005228
-35736,GNL3L,0.11779939453886554
-35738,MAGED2,-0.1063667476696755
-35741,APEX2,0.0878386757276833
-35745,FAM104B,-0.0643164467178141
-35749,MAGEH1,-0.13791145188816412
-35752,RRAGB,-0.026364166447073716
-35753,AL050309.1,-0.021856650967682942
-35755,UBQLN2,-0.00987352469210277
-35757,NBDY,-0.1289183736531296
-35758,SPIN3,0.032501822323812976
-35760,SPIN2B,-0.10372450049232503
-35768,LINC01278,-0.1311666968034566
-35771,ARHGEF9,-0.14202121757316624
-35779,LAS1L,-0.03172493610750527
-35780,MSN,0.22291230751390131
-35782,AL034397.3,0.3060337767602539
-35789,YIPF6,-0.03544949587444724
-35791,EFNB1,0.08438312114422793
-35792,PJA1,-0.021070127297787095
-35800,IGBP1,-0.045389126345570095
-35808,PDZD11,-0.11477123503027983
-35816,SNX12,0.040342273909205494
-35817,FOXO4,0.030404931673340647
-35819,IL2RG,-0.5077542020478681
-35820,MED12,-0.005006011637045277
-35822,AL590764.1,0.009431898118805059
-35824,ZMYM3,0.07056598940234207
-35825,NONO,0.1253530348127399
-35827,TAF1,0.030856203881058756
-35828,OGT,0.048449125897759224
-35829,GCNA,-0.11341479580639711
-35830,CXCR3,-0.12573867768282787
-35835,NHSL2,0.2609505623888201
-35837,PIN4,-0.13237513534700698
-35839,RPS4X,-0.4376301958725731
-35841,HDAC8,0.09666948948917642
-35857,CHIC1,-0.10557901227431997
-35858,TSIX,-0.07540962765801186
-35859,XIST,0.174946738088447
-35860,FTX,0.059693640723665585
-35861,JPX,-0.13573237602675284
-35864,Z83843.1,0.11636433620034156
-35867,RLIM,0.06161774328142752
-35869,ABCB7,0.019509755977693596
-35870,UPRT,0.041498282272999495
-35874,PBDC1,-0.2255659140419986
-35878,ATRX,-0.015445512176775564
-35879,MAGT1,0.10818998124882202
-35880,COX7B,-0.05028048921684737
-35881,ATP7A,-0.03803392386551635
-35882,PGK1,0.3332238142324831
-35884,TAF9B,0.030060716947849713
-35885,CYSLTR1,0.15144012989799277
-35888,P2RY10,-0.11293449921641152
-35889,GPR174,-0.3727144693705558
-35890,ITM2A,-0.30672077789057667
-35893,BRWD3,0.06321570231388744
-35896,SH3BGRL,0.331660521726671
-35901,HDX,0.08082825223085172
-35902,APOOL,0.09095632116282656
-35907,CHM,0.01941754033287935
-35921,DIAPH2,0.33562947766696766
-35930,CSTF2,0.05622232372631561
-35934,TRMT2B,-0.04937554214112421
-35940,TIMM8A,-0.03262026151701962
-35941,BTK,0.34401329286418225
-35942,RPL36A,-0.2761376645015826
-35943,GLA,0.13836071925241783
-35944,HNRNPH2,0.200119853182454
-35945,ARMCX4,-0.0073516203939472235
-35947,ARMCX6,-0.010860430555805272
-35948,ARMCX3,0.03102273190840908
-35954,ZMAT1,-0.004761958782683418
-35958,BEX5,-0.12834870858082917
-35964,ARMCX5,0.0301854412697916
-35965,ARMCX5-GPRASP2,0.038898341691745296
-35967,GPRASP1,-0.050031136784660035
-35979,BEX4,-0.15799599734855693
-35980,TCEAL8,-0.14517935604916143
-35982,BEX2,-0.1282908085447422
-35985,BEX3,0.03492436998229401
-35990,TCEAL4,0.03352949242201337
-35991,TCEAL3,-0.08250287303159938
-35993,TCEAL1,-0.037761926052411014
-35994,MORF4L2,-0.008536726738135022
-36005,SLC25A53,-0.0863914771962448
-36007,FAM199X,-0.07002712421046318
-36017,MORC4,-0.06982116096218996
-36020,RBM41,0.040183043757895665
-36025,PRPS1,-0.18768163478408906
-36026,TSC22D3,0.25326400008697164
-36031,VSIG1,-0.06652667519447146
-36032,PSMD10,0.15412583930418505
-36033,ATG4A,0.023625181133854006
-36040,NXT2,-0.031727938277939446
-36042,ACSL4,0.29406624916485036
-36043,TMEM164,0.15986708490450832
-36044,AMMECR1,-0.0910681051750612
-36053,ALG13,0.002476471548591849
-36077,WDR44,0.062352190195953804
-36078,DOCK11,0.06136977954593533
-36079,IL13RA1,0.3823073616449476
-36085,PGRMC1,0.049596349580456445
-36088,SLC25A43,0.02207063955248995
-36091,SLC25A5,0.16612043302854604
-36092,CXorf56,0.057296842772735605
-36093,UBE2A,0.06159219101781632
-36094,NKRF,-0.006874845784467984
-36095,SEPTIN6,-0.25638900479868143
-36096,SOWAHD,0.1319643260093204
-36097,RPL39,-0.27385701787901545
-36099,UPF3B,-0.13088353355680798
-36100,RNF113A,-0.09225850380948132
-36101,NDUFA1,-0.057825045454590676
-36103,NKAP,-0.04320457991071741
-36109,ZBTB33,-0.08851770053637799
-36112,LAMP2,0.32024694900053174
-36113,CUL4B,0.056464585647043965
-36114,MCTS1,-0.017514411735311097
-36115,C1GALT1C1,0.008658660622103583
-36136,THOC2,0.019688789326329923
-36137,XIAP,0.09113457794874814
-36140,STAG2,0.1305546160035711
-36141,SH2D1A,-0.3403161278438673
-36143,TENM1,-0.08746769217519429
-36153,OCRL,0.07658429817986681
-36157,SASH3,0.003189174780768219
-36161,UTP14A,-0.011488068165585437
-36162,BCORL1,0.08681316702000823
-36163,ELF4,0.14316260133938186
-36164,AIFM1,0.0031237928636960325
-36166,RAB33A,-0.19454146636271746
-36167,ZNF280C,-0.021493206865591606
-36168,SLC25A14,-0.003845846028703127
-36170,RBMX2,-0.0012407087083213565
-36172,ENOX2,0.039022792745724365
-36180,STK26,-0.1116239780819637
-36182,RAP2C,0.09417722487420792
-36183,RAP2C-AS1,0.04347425081350844
-36184,MBNL3,0.05991826385066163
-36193,PHF6,-0.016663114111778007
-36194,HPRT1,-0.07667093613078446
-36199,FAM122B,-0.04107829583340085
-36200,FAM122C,0.005850004781467156
-36201,MOSPD1,-0.09066729550396772
-36205,RTL8C,0.12550992460052116
-36207,RTL8A,0.015642948957280925
-36213,ZNF75D,0.03909597802180093
-36217,ZNF449,-0.023512611423003735
-36220,INTS6L,-0.01308934913563798
-36231,MMGT1,-0.04205020146914362
-36232,SLC9A6,0.03914484612593535
-36233,FHL1,0.013520470929521052
-36234,MAP7D3,-0.0861421641434301
-36237,HTATSF1,-0.048470512650640785
-36239,LINC00892,-0.14384545740845223
-36240,CD40LG,-0.11488578968703432
-36241,ARHGEF6,0.07775940449308251
-36242,AL683813.1,0.04230004856113724
-36243,RBMX,-0.031173288768002728
-36253,ATP11C,0.052980882315783795
-36277,SLITRK4,0.08993152092297647
-36291,FMR1,0.09891100427356428
-36298,IDS,0.03484956175376978
-36300,LINC00893,-0.10179336937252985
-36301,CXorf40A,0.01535874959460581
-36305,TMEM185A,-0.022860665732262885
-36312,CXorf40B,-0.022939904398138183
-36314,LINC00894,0.018290843775173163
-36316,MTM1,0.11509877311116452
-36317,MTMR1,0.017337791296368355
-36318,CD99L2,-0.024007168517867144
-36325,VMA21,0.0529273311055695
-36350,CETN2,-0.05046666227092573
-36351,NSDHL,-0.005385879399780633
-36352,ZNF185,0.1137903162455039
-36361,ZNF275,0.002195621867406697
-36365,HAUS7,-0.010464602860764428
-36368,CCNQ,-0.0570211054934531
-36372,BCAP31,0.001367519703077213
-36373,ABCD1,0.17498489889346436
-36377,IDH3G,0.04507735227470654
-36378,SSR4,-0.16424921207518037
-36383,ARHGAP4,0.12136591932111544
-36384,NAA10,-0.0625297386053286
-36385,RENBP,0.18343726964268253
-36386,HCFC1,0.15730041968229688
-36388,TMEM187,-0.12549968792736327
-36389,IRAK1,0.17801139437453892
-36390,MECP2,-0.1254045089825172
-36398,FLNA,0.17306855793601833
-36400,EMD,-0.11574939524629656
-36401,RPL10,-0.45345748330014896
-36402,AC245140.2,-0.021671330694357407
-36403,DNASE1L1,0.20590733741384087
-36404,TAZ,0.022704676160825468
-36405,AC244090.1,-0.018072116965695273
-36406,ATP6AP1,0.2245343975713343
-36407,GDI1,-0.007709860825332988
-36408,FAM50A,0.17247138707019877
-36409,PLXNA3,-0.0472701186202171
-36410,LAGE3,-0.10557754631661923
-36411,UBL4A,-0.07173422511071681
-36412,SLC10A3,0.010611353886525432
-36413,FAM3A,-0.10641377360493236
-36415,G6PD,0.15388172775844292
-36416,IKBKG,0.026137575590987654
-36422,GAB3,0.004893896016511426
-36423,DKC1,-0.039806249494750326
-36424,MPP1,0.17538813677227022
-36428,F8A1,-0.01250896017211151
-36429,FUNDC2,-0.05390832676648117
-36430,CMC4,-0.1268850337267403
-36432,BRCC3,-0.04396885725839461
-36433,VBP1,-0.1412414577478851
-36434,RAB39B,-0.05765490943971836
-36435,CLIC2,0.09520302947566933
-36443,TMLHE,0.11707919502954454
-36445,VAMP7,0.12476331483037964
-36559,MT-ND1,0.3683588058016804
-36560,MT-ND2,0.3157932653637021
-36561,MT-CO1,0.28779693192577344
-36562,MT-CO2,0.05264224281556177
-36563,MT-ATP8,0.12261611803814773
-36564,MT-ATP6,-0.1649120810326163
-36565,MT-CO3,0.05734669419739229
-36566,MT-ND3,0.1041293995087942
-36567,MT-ND4L,0.18812367227530583
-36568,MT-ND4,0.21306351177526348
-36569,MT-ND5,-0.07216012020140243
-36570,MT-ND6,-0.18792858240812466
-36571,MT-CYB,-0.08945658334898675
-36580,AL592183.1,0.019248635055742164
-36581,AC240274.1,-0.0007150145677195353
-36584,AC004556.3,0.0263942054157376
diff --git a/scarf/tests/datasets/ptime_modules_group_1.npy b/scarf/tests/datasets/ptime_modules_group_1.npy
deleted file mode 100644
index 39f6913e..00000000
Binary files a/scarf/tests/datasets/ptime_modules_group_1.npy and /dev/null differ
diff --git a/scarf/tests/datasets/sympathetic.loom b/scarf/tests/datasets/sympathetic.loom
deleted file mode 100644
index f1dc5119..00000000
Binary files a/scarf/tests/datasets/sympathetic.loom and /dev/null differ
diff --git a/scarf/tests/datasets/toy_cr_dir.tar.gz b/scarf/tests/datasets/toy_cr_dir.tar.gz
deleted file mode 100644
index fe82c753..00000000
Binary files a/scarf/tests/datasets/toy_cr_dir.tar.gz and /dev/null differ
diff --git a/scarf/tests/datasets/toy_cr_dir_empty.tar.gz b/scarf/tests/datasets/toy_cr_dir_empty.tar.gz
deleted file mode 100644
index 26e59c8b..00000000
Binary files a/scarf/tests/datasets/toy_cr_dir_empty.tar.gz and /dev/null differ
diff --git a/scarf/tests/datasets/unified_UMAP_coords.npy b/scarf/tests/datasets/unified_UMAP_coords.npy
deleted file mode 100644
index 7ae4ba56..00000000
Binary files a/scarf/tests/datasets/unified_UMAP_coords.npy and /dev/null differ
diff --git a/scarf/tests/fixtures_datastore.py b/scarf/tests/fixtures_datastore.py
deleted file mode 100644
index 2f4111d4..00000000
--- a/scarf/tests/fixtures_datastore.py
+++ /dev/null
@@ -1,217 +0,0 @@
-import numpy as np
-import pandas as pd
-import pytest
-
-from . import full_path, remove
-
-
-@pytest.fixture(scope="session")
-def toy_crdir_writer(toy_crdir_reader):
- from ..writers import CrToZarr
-
- out_fn = full_path("toy_crdir.zarr")
-
- writer = CrToZarr(toy_crdir_reader, out_fn)
- writer.dump()
- yield out_fn
- remove(out_fn)
-
-
-@pytest.fixture(scope="session")
-def toy_crdir_ds(toy_crdir_writer):
- from ..datastore.datastore import DataStore
-
- yield DataStore(toy_crdir_writer, default_assay="RNA")
-
-
-@pytest.fixture(scope="session")
-def datastore():
- import tarfile
-
- from ..datastore.datastore import DataStore
-
- fn = full_path("1K_pbmc_citeseq.zarr.tar.gz")
- out_fn = fn.replace(".tar.gz", "")
- remove(out_fn)
- tar = tarfile.open(fn, "r:gz")
- tar.extractall(out_fn)
- yield DataStore(out_fn, default_assay="RNA")
- remove(out_fn)
-
-
-@pytest.fixture
-def datastore_ephemeral():
- import tarfile
-
- from ..datastore.datastore import DataStore
-
- fn = full_path("1K_pbmc_citeseq.zarr.tar.gz")
- out_fn = fn.replace(".tar.gz", "")
- remove(out_fn)
- tar = tarfile.open(fn, "r:gz")
- tar.extractall(out_fn)
- yield DataStore(out_fn, default_assay="RNA")
- remove(out_fn)
-
-
-@pytest.fixture(scope="class")
-def auto_filter_cells(datastore):
- datastore.auto_filter_cells(show_qc_plots=False)
-
-
-@pytest.fixture(scope="class")
-def mark_hvgs(auto_filter_cells, datastore):
- datastore.mark_hvgs(top_n=100, show_plot=False)
-
-
-@pytest.fixture(scope="class")
-def make_graph(mark_hvgs, datastore):
- datastore.make_graph(feat_key="hvgs")
- graph_loc = datastore._get_latest_graph_loc(
- from_assay="RNA", cell_key="I", feat_key="hvgs"
- )
- yield graph_loc.rsplit("/", 1)[0]
-
-
-@pytest.fixture(scope="class")
-def leiden_clustering(make_graph, datastore):
- datastore.run_leiden_clustering()
- yield datastore.cells.fetch("RNA_leiden_cluster")
-
-
-@pytest.fixture(scope="class")
-def paris_clustering(make_graph, datastore):
- datastore.run_clustering(n_clusters=10)
- yield datastore.cells.fetch("RNA_cluster")
-
-
-@pytest.fixture(scope="class")
-def paris_clustering_balanced(make_graph, datastore):
- datastore.run_clustering(
- balanced_cut=True, max_size=100, min_size=10, label="balanced_clusters"
- )
- yield datastore.cells.fetch("RNA_balanced_clusters")
-
-
-@pytest.fixture(scope="class")
-def umap(make_graph, datastore):
- datastore.run_umap(n_epochs=50)
- yield np.array(
- [datastore.cells.fetch("RNA_UMAP1"), datastore.cells.fetch("RNA_UMAP2")]
- ).T
-
-
-@pytest.fixture(scope="class")
-def marker_search(datastore):
- # Testing this with Paris clusters rather than Leiden clusters because of reproducibility.
- datastore.run_marker_search(group_key="RNA_cluster")
-
-
-@pytest.fixture(scope="class")
-def pseudotime_scoring(datastore, leiden_clustering):
- datastore.run_pseudotime_scoring(
- source_sink_key="RNA_leiden_cluster", sources=[6], sinks=[3]
- )
- yield datastore.cells.fetch("RNA_pseudotime")
-
-
-@pytest.fixture(scope="class")
-def pseudotime_markers(datastore, pseudotime_scoring):
- datastore.run_pseudotime_marker_search(pseudotime_key="RNA_pseudotime")
- df = datastore.RNA.feats.to_pandas_dataframe(
- ["names", "I__RNA_pseudotime__r"], key="I"
- )
- yield df
-
-
-@pytest.fixture(scope="class")
-def pseudotime_aggregation(datastore, pseudotime_scoring):
- datastore.run_pseudotime_aggregation(
- pseudotime_key="RNA_pseudotime",
- cluster_label="pseudotime_clusters",
- n_clusters=15,
- window_size=50,
- chunk_size=10,
- )
-
-
-@pytest.fixture(scope="class")
-def grouped_assay(datastore, pseudotime_aggregation):
- datastore.add_grouped_assay(
- group_key="pseudotime_clusters", assay_label="PTIME_MODULES"
- )
-
-
-@pytest.fixture(scope="class")
-def run_mapping(make_graph, datastore):
- datastore.run_mapping(
- target_assay=datastore.RNA,
- target_name="selfmap",
- target_feat_key="hvgs_self",
- save_k=3,
- )
-
-
-@pytest.fixture(scope="class")
-def run_mapping_coral(make_graph, datastore):
- datastore.run_mapping(
- target_assay=datastore.RNA,
- target_name="selfmap_coral",
- target_feat_key="hvgs_self2",
- save_k=3,
- run_coral=True,
- )
-
-
-@pytest.fixture(scope="class")
-def run_unified_umap(run_mapping, datastore):
- datastore.run_unified_umap(target_names=["selfmap"])
-
-
-@pytest.fixture(scope="class")
-def cell_cycle_scoring(datastore):
- datastore.run_cell_cycle_scoring()
- return datastore.cells.fetch("RNA_cell_cycle_phase")
-
-
-@pytest.fixture(scope="class")
-def topacedo_sampler(paris_clustering, datastore):
- try:
- import topacedo
- except ImportError:
- pytest.skip("topacedo package not installed")
- datastore.run_topacedo_sampler(cluster_key="RNA_cluster")
- return datastore.cells.fetch("RNA_sketched")
-
-
-@pytest.fixture(scope="class")
-def cell_attrs():
- return pd.read_csv(full_path("cell_attributes.csv"), index_col=0)
-
-
-@pytest.fixture(scope="session")
-def atac_datastore():
- from ..datastore.datastore import DataStore
- import tarfile
-
- fn = full_path("500_pbmc_atac.zarr.tar.gz")
- out_fn = fn.replace(".tar.gz", "")
- remove(out_fn)
- tar = tarfile.open(fn, "r:gz")
- tar.extractall(out_fn)
- yield DataStore(out_fn)
- remove(out_fn)
-
-
-@pytest.fixture(scope="class")
-def mark_prevalent_peaks(atac_datastore):
- atac_datastore.mark_prevalent_peaks(top_n=5000)
-
-
-@pytest.fixture(scope="class")
-def make_atac_graph(mark_prevalent_peaks, atac_datastore):
- atac_datastore.make_graph(feat_key="prevalent_peaks")
- graph_loc = atac_datastore._get_latest_graph_loc(
- from_assay="ATAC", cell_key="I", feat_key="prevalent_peaks"
- )
- yield graph_loc.rsplit("/", 1)[0]
diff --git a/scarf/tests/fixtures_downloader.py b/scarf/tests/fixtures_downloader.py
deleted file mode 100644
index 63428b4a..00000000
--- a/scarf/tests/fixtures_downloader.py
+++ /dev/null
@@ -1,13 +0,0 @@
-import pytest
-
-from . import full_path, remove
-
-
-@pytest.fixture(scope="session")
-def bastidas_ponce_data():
- from ..downloader import fetch_dataset
-
- sample = "bastidas-ponce_4K_pancreas-d15_rnaseq"
- fetch_dataset(sample, full_path(None))
- yield full_path(sample, "data.h5ad")
- remove(full_path(sample))
diff --git a/scarf/tests/fixtures_readers.py b/scarf/tests/fixtures_readers.py
deleted file mode 100644
index db8cdc8a..00000000
--- a/scarf/tests/fixtures_readers.py
+++ /dev/null
@@ -1,79 +0,0 @@
-import pytest
-
-from . import full_path, remove
-
-
-@pytest.fixture(scope="session")
-def toy_crdir_reader():
- from ..readers import CrDirReader
- import tarfile
-
- fn = full_path("toy_cr_dir.tar.gz")
- out_fn = fn.replace(".tar.gz", "")
- remove(out_fn)
- tar = tarfile.open(fn, "r:gz")
- tar.extractall(out_fn)
- reader = CrDirReader(out_fn)
- reader.rename_assays({"ASSAY4": "HTO"})
- yield reader
- remove(out_fn)
-
-@pytest.fixture(scope="session")
-def toy_crdir_empty():
- from ..readers import CrDirReader
- import tarfile
-
- fn = full_path("toy_cr_dir_empty.tar.gz")
- out_fn = fn.replace(".tar.gz", "")
- remove(out_fn)
- tar = tarfile.open(fn, "r:gz")
- tar.extractall(out_fn)
- reader = CrDirReader(out_fn)
- yield reader
- remove(out_fn)
-
-@pytest.fixture(scope="session")
-def crh5_reader():
- from ..readers import CrH5Reader
-
- return CrH5Reader(full_path("1K_pbmc_citeseq.h5"))
-
-
-@pytest.fixture(scope="session")
-def mtx_dir(datastore):
- from ..writers import to_mtx
-
- fn = full_path("1K_pbmc_citeseq_dir")
- to_mtx(datastore.RNA, fn, compress=True)
- yield fn
- remove(fn)
-
-
-@pytest.fixture(scope="session")
-def crdir_reader(datastore, mtx_dir):
- from ..readers import CrDirReader
-
- reader = CrDirReader(mtx_dir)
- yield reader
-
-
-@pytest.fixture(scope="session")
-def h5ad_reader(bastidas_ponce_data):
- from ..readers import H5adReader
-
- reader = H5adReader(bastidas_ponce_data)
- yield reader
- reader.h5.close()
-
-
-@pytest.fixture(scope="session")
-def loom_reader():
- from ..readers import LoomReader
-
- reader = LoomReader(
- full_path("sympathetic.loom"),
- cell_names_key="Cell_id",
- feature_names_key="Gene",
- )
- yield reader
- reader.h5.close()
diff --git a/scarf/tests/requirements.txt b/scarf/tests/requirements.txt
deleted file mode 100644
index 1e824230..00000000
--- a/scarf/tests/requirements.txt
+++ /dev/null
@@ -1,6 +0,0 @@
-pytest
-pytest-cov
-topacedo
-anndata==0.11.4
-black
-scalene
diff --git a/scarf/tests/test_datastore.py b/scarf/tests/test_datastore.py
deleted file mode 100644
index 3e3d4769..00000000
--- a/scarf/tests/test_datastore.py
+++ /dev/null
@@ -1,279 +0,0 @@
-import numpy as np
-import pandas as pd
-
-from . import full_path, remove
-
-
-class TestToyDataStore:
- def test_toy_crdir_metadata(self, toy_crdir_ds):
- assert np.all(
- toy_crdir_ds.RNA.feats.fetch_all("ids") == ["g1", "g2", "g3", "g4"]
- )
- assert np.all(toy_crdir_ds.ADT.feats.fetch_all("ids") == ["a1", "a2"])
- assert np.all(toy_crdir_ds.HTO.feats.fetch_all("ids") == ["h1"])
- assert np.all(toy_crdir_ds.cells.fetch_all("ids") == ["b1", "b2", "b3"])
-
- def test_toy_crdir_rawdata(self, toy_crdir_ds):
- assert np.all(
- toy_crdir_ds.RNA.rawData.compute()
- == [[5, 0, 0, 2], [3, 3, 0, 7], [3, 3, 0, 7]]
- )
- assert np.all(
- toy_crdir_ds.ADT.rawData.compute() == [[30, 40], [30, 50], [0, 50]]
- )
- assert np.all(toy_crdir_ds.HTO.rawData.compute() == [[200], [100], [100]])
-
-
-class TestDataStore:
- def test_init_wrong_zarr_mode(self):
- import pytest
- import tarfile
-
- from ..datastore.datastore import DataStore
-
- fn = full_path("1K_pbmc_citeseq.zarr.tar.gz")
- out_fn = fn.replace(".tar.gz", "")
- remove(out_fn)
- tar = tarfile.open(fn, "r:gz")
- tar.extractall(out_fn)
- with pytest.raises(ValueError):
- ds = DataStore(out_fn, zarr_mode="wrong", default_assay="RNA")
- remove(out_fn)
-
- def test_auto_filter_cells(self, datastore_ephemeral):
- assert (
- datastore_ephemeral.auto_filter_cells(
- attrs=["nCounts", "nFeatures", "non_existing_column"],
- show_qc_plots=False,
- )
- is None
- )
- # show_qc_plots=True
- # howto test plots?
-
- def test_filter_cells(self, datastore_ephemeral):
- assert (
- datastore_ephemeral.filter_cells(
- attrs=["nCounts", "nFeatures", "non_existing_column"],
- lows=[None, None, None],
- highs=[None, None, None],
- reset_previous=True,
- )
- is None
- )
- # still doesn't access `if j is None:` cases for j and k
-
- def test_graph_indices(self, make_graph, datastore):
- a = np.load(full_path("knn_indices.npy"))
- b = datastore.z[make_graph]["indices"][:]
- assert np.array_equal(a, b)
-
- def test_graph_distances(self, make_graph, datastore):
- a = np.load(full_path("knn_distances.npy"))
- b = datastore.z[make_graph]["distances"][:]
- assert np.all((a - b) < 1e-3)
-
- def test_graph_weights(self, make_graph, datastore):
- a = np.load(full_path("knn_weights.npy"))
- b = datastore.z[make_graph]["graph__1.0__1.5"]["weights"][:]
- assert np.all((a - b) < 1e-5)
-
- def test_atac_graph_indices(self, make_atac_graph, atac_datastore):
- a = np.load(full_path("atac_knn_indices.npy"))
- b = atac_datastore.z[make_atac_graph]["indices"][:]
- assert a.shape == b.shape
-
- # TODO: activate this when this PR is merged and released in gensim
- # https://github.com/RaRe-Technologies/gensim/pull/3194
- # assert np.array_equal(a, b)
-
- def test_atac_graph_distances(self, make_atac_graph, atac_datastore):
- a = np.load(full_path("atac_knn_distances.npy"))
- b = atac_datastore.z[make_atac_graph]["distances"][:]
- assert a.shape == b.shape
-
- # TODO: activate this when this PR is merged and released in gensim
- # https://github.com/RaRe-Technologies/gensim/pull/3194
- # assert np.all((a - b) < 1e-5)
-
- def test_leiden_values(self, leiden_clustering, cell_attrs):
- assert len(set(leiden_clustering)) == 10
- # Disabled the following test because failing on CI
- # assert np.array_equal(leiden_clustering, cell_attrs['RNA_leiden_cluster'].values)
-
- def test_paris_values(self, paris_clustering, cell_attrs):
- assert np.array_equal(paris_clustering, cell_attrs["RNA_cluster"].values)
-
- def test_paris_balanced_values(self, paris_clustering_balanced, cell_attrs):
- assert np.array_equal(
- paris_clustering_balanced, cell_attrs["RNA_balanced_clusters"].values
- )
-
- def test_run_cell_cycle_scoring(self, cell_cycle_scoring, cell_attrs):
- assert np.array_equal(
- cell_cycle_scoring, cell_attrs["RNA_cell_cycle_phase"].values
- )
-
- def test_umap_values(self, umap, cell_attrs):
- precalc_umap = cell_attrs[["RNA_UMAP1", "RNA_UMAP2"]].values
- assert umap.shape == precalc_umap.shape
- # Disabled the following test because failing on CI
- # assert np.all((umap - precalc_umap) < 0.1)
-
- def test_get_markers(self, marker_search, paris_clustering, datastore):
- precalc_markers = pd.read_csv(full_path("markers_cluster1.csv"), index_col=0)
- markers = datastore.get_markers(group_key="RNA_cluster", group_id=1)
-
- # Check feature names and scores (always required)
- assert markers.feature_name.equals(precalc_markers.feature_name)
- diff = (markers.score - precalc_markers.score).values
- assert np.all(diff < 1e-3)
-
- # Check p_values only if they exist in reference data (backward compatible)
- if 'p_value' in precalc_markers.columns:
- assert 'p_value' in markers.columns, "p_value column missing in output"
- # P-values should match within reasonable tolerance
- p_diff = (markers.p_value - precalc_markers.p_value).values
- assert np.all(np.abs(p_diff) < 1e-3), "p_values differ from reference"
-
- def test_export_markers_to_csv(self, marker_search, paris_clustering, datastore):
- precalc_markers = pd.read_csv(full_path("markers_all_clusters.csv"))
- out_file = full_path("test_values_markers.csv")
- datastore.export_markers_to_csv(group_key="RNA_cluster", csv_filename=out_file)
- markers = pd.read_csv(out_file)
- assert markers.equals(precalc_markers)
- remove(out_file)
-
- def test_run_unified_umap(self, run_unified_umap, datastore):
- coords = datastore.z["RNA"].projections["unified_UMAP"][:]
- precalc_coords = np.load(full_path("unified_UMAP_coords.npy"))
- assert coords.shape == precalc_coords.shape
-
- def test_get_target_classes(
- self, run_mapping, paris_clustering, cell_attrs, datastore
- ):
- classes = datastore.get_target_classes(
- target_name="selfmap", reference_class_group="RNA_cluster"
- )
- assert np.array_equal(classes.values, cell_attrs["target_classes"].values)
-
- def test_get_mapping_score(self, run_mapping, cell_attrs, datastore):
- scores = next(datastore.get_mapping_score(target_name="selfmap"))[1]
- diff = scores - cell_attrs["mapping_scores"].values
- assert np.all(diff < 1e-2)
-
- def test_coral_mapping_score(self, run_mapping_coral, cell_attrs, datastore):
- # TODO: add test values for coral
- assert 1 == 1
-
- def test_repr(self, datastore):
- # TODO: Test if the expected values are printed
- print(datastore)
-
- def test_get_imputed(self, datastore):
- # TODO: Test the output values
- values = datastore.get_imputed(feature_name="CD4")
- assert values.shape == datastore.cells.fetch("I").shape
-
- def test_run_pseudotime_scoring(self, pseudotime_scoring, cell_attrs):
- diff = pseudotime_scoring - cell_attrs["RNA_pseudotime"].values
- assert np.all(diff < 1e-3)
-
- def test_run_pseudotime_marker_search(self, pseudotime_markers):
- precalc_markers = pd.read_csv(
- full_path("pseudotime_markers_r_values.csv"), index_col=0
- )
- assert np.all(precalc_markers.index == pseudotime_markers.index)
- assert np.all(precalc_markers.names.values == pseudotime_markers.names.values)
- assert np.allclose(
- precalc_markers.I__RNA_pseudotime__r.values,
- pseudotime_markers.I__RNA_pseudotime__r.values,
- )
-
- def test_run_pseudotime_aggregation(self, pseudotime_aggregation, datastore):
- precalc_values = np.load(full_path("aggregated_feat_idx.npy"))
- test_values = datastore.z.RNA.aggregated_I_I_RNA_pseudotime.feature_indices[:]
- assert np.array_equal(precalc_values.astype(np.int64), test_values.astype(np.int64))
-
- precalc_values = np.load(full_path("aggregated_df_top_10.npy"))
- test_values = datastore.z.RNA.aggregated_I_I_RNA_pseudotime.data[:10]
- assert np.all(precalc_values == test_values)
-
- precalc_values = np.load(full_path("pseudotime_clusters.npy"))
- test_values = datastore.RNA.feats.fetch_all("pseudotime_clusters")
- assert np.all(precalc_values == test_values)
-
- def test_add_grouped_assay(self, grouped_assay, datastore):
- precalc_values = np.load(full_path("ptime_modules_group_1.npy"))
- test_values = datastore.get_cell_vals(
- from_assay="PTIME_MODULES", cell_key="I", k="group_1"
- )
- assert np.allclose(precalc_values, test_values)
-
- def test_make_bulk(self, leiden_clustering, datastore):
- df = datastore.make_bulk(group_key="RNA_leiden_cluster")
- assert df.shape == (18850, 10)
- assert hash(tuple((df.values.flatten()))) == -1872129810056415572
-
- def test_to_anndata(self, datastore):
- # TODO: Check if all the attributes copied to anndata
- datastore.to_anndata()
-
- def test_run_topacedo_sampler(self, cell_attrs, topacedo_sampler):
- assert np.all(topacedo_sampler == cell_attrs["RNA_sketched"])
-
- def test_plot_cells_dists(self, datastore):
- datastore.plot_cells_dists(show_fig=False)
-
- def test_plot_layout(self, umap, paris_clustering, datastore):
- datastore.plot_layout(
- layout_key="RNA_UMAP", color_by="RNA_cluster", show_fig=False
- )
-
- # def test_plot_layout_shade(self, umap, paris_clustering, datastore):
- # datastore.plot_layout(
- # layout_key="RNA_UMAP",
- # color_by="RNA_cluster",
- # show_fig=False,
- # do_shading=True,
- # )
-
- def test_plot_cluster_tree(self, datastore):
- datastore.plot_cluster_tree(cluster_key="RNA_cluster", show_fig=False)
-
- def test_plot_marker_heatmap(self, marker_search, datastore):
- datastore.plot_marker_heatmap(group_key="RNA_cluster", show_fig=False)
-
- def test_plot_unified_layout(self, run_unified_umap, datastore):
- datastore.plot_unified_layout(layout_key="unified_UMAP", show_fig=False)
-
- def test_plot_pseudotime_heatmap(self, pseudotime_aggregation, datastore):
- datastore.plot_pseudotime_heatmap(
- cell_key="I",
- feat_key="I",
- feature_cluster_key="pseudotime_clusters",
- pseudotime_key="RNA_pseudotime",
- show_features=["Wsb1", "Rest"],
- show_fig=False,
- )
-
- def test_mark_hvgs_with_atac_assay(self, atac_datastore):
- import pytest
-
- with pytest.raises(TypeError):
- atac_datastore.mark_hvgs()
-
- def test_mark_prevalent_peaks_with_rna_assay(self, datastore):
- import pytest
-
- with pytest.raises(TypeError):
- datastore.mark_prevalent_peaks()
-
- def test_run_marker_search_with_no_groupkey(self, datastore):
- import pytest
-
- with pytest.raises(ValueError):
- datastore.run_marker_search(group_key=None)
-
- def test_run_marker_search_with_cellkey(self, datastore, paris_clustering):
- datastore.run_marker_search(group_key="RNA_cluster", cell_key="I")
diff --git a/scarf/tests/test_downloader.py b/scarf/tests/test_downloader.py
deleted file mode 100644
index a1192073..00000000
--- a/scarf/tests/test_downloader.py
+++ /dev/null
@@ -1,31 +0,0 @@
-from . import full_path, remove
-
-
-def test_downloader():
- from ..downloader import osfd, OSFdownloader
-
- if osfd is None:
- osfd = OSFdownloader("zeupv")
- # Making sure that there are at least 15 datasets in the list
- assert len(osfd.datasets) > 15
-
-
-def test_show_available_datasets():
- from ..downloader import show_available_datasets
-
- show_available_datasets()
-
-
-def test_fetch_dataset(bastidas_ponce_data):
- # TODO: check if correct file was downloaded.
- pass
-
-
-def test_downloader_as_zarr():
- from ..downloader import fetch_dataset
-
- # TODO: check if only the zarr file was downloaded
-
- sample = "tenx_5K_pbmc_rnaseq"
- fetch_dataset(sample, as_zarr=True, save_path=full_path(None))
- remove(full_path(sample))
diff --git a/scarf/tests/test_merger.py b/scarf/tests/test_merger.py
deleted file mode 100644
index e85ce1d3..00000000
--- a/scarf/tests/test_merger.py
+++ /dev/null
@@ -1,112 +0,0 @@
-import zarr
-
-from . import full_path, remove
-
-
-def test_assay_merge(datastore):
- # TODO: Evaluate the resulting merged file
- """
- Test the AssayMerge class by merging two assays of same type from the same dataset.
- """
- from ..merge import AssayMerge
-
- fn = full_path("merged_zarr.zarr")
- writer = AssayMerge(
- zarr_path=fn,
- assays=[datastore.RNA, datastore.RNA],
- names=["self1", "self2"],
- merge_assay_name="RNA",
- prepend_text="",
- overwrite=True,
- )
- writer.dump()
- tmp = zarr.open(fn + "/RNA/counts")
- assert tmp.shape[0] == 2 * datastore.cells.N
- assert int(tmp[...].sum()) == int(datastore.RNA.rawData.compute().sum() * 2)
- remove(fn)
-
-
-def test_dataset_merge_2(datastore):
- """
- Test the DatasetMerge class by merging two datasets. This will test the merging of all assays in the dataset.
- """
- from ..merge import DatasetMerge
-
- fn = full_path("merged_zarr.zarr")
- writer = DatasetMerge(
- zarr_path=fn,
- datasets=[datastore, datastore],
- names=["self1", "self2"],
- prepend_text="",
- overwrite=True
- )
- writer.dump()
- # Check if the merged file has the correct shape and counts
- # Load the merged counts file
- rna_count = zarr.open(fn + "/RNA/counts")
- assay2_count = zarr.open(fn + "/assay2/counts")
- # We merged only two datasets, so the shape should be 2*original_shape
- # Check the number of cells
- assert rna_count.shape[0] == 2 * datastore.cells.N
- assert assay2_count.shape[0] == 2 * datastore.cells.N
- # Check the count values through sum
- assert int(rna_count[...].sum()) == int(datastore.RNA.rawData.compute().sum() * 2)
- assert int(assay2_count[...].sum()) == int(
- datastore.assay2.rawData.compute().sum() * 2
- )
- remove(fn)
-
-
-def test_dataset_merge_3(datastore):
- """
- Test the DatasetMerge class by merging two datasets. This will test the merging of all assays in the dataset.
- If merging is successful for more than two datasets, it should work for any number of datasets.
- """
- from ..merge import DatasetMerge
-
- fn = full_path("merged_zarr.zarr")
- writer = DatasetMerge(
- zarr_path=fn,
- datasets=[datastore, datastore, datastore],
- names=["self1", "self2", "self3"],
- prepend_text="",
- overwrite=True
- )
- writer.dump()
- # Check if the merged file has the correct shape and counts
- # Load the merged counts file
- rna_count = zarr.open(fn + "/RNA/counts")
- assay2_count = zarr.open(fn + "/assay2/counts")
- # We merged only three datasets, so the shape should be 3*original_shape
- # Check the number of cells
- assert rna_count.shape[0] == 3 * datastore.cells.N
- assert assay2_count.shape[0] == 3 * datastore.cells.N
- # Check the count values through sum
- assert int(rna_count[...].sum()) == int(datastore.RNA.rawData.compute().sum() * 3)
- assert int(assay2_count[...].sum()) == int(
- datastore.assay2.rawData.compute().sum() * 3
- )
- remove(fn)
-
-def test_dataset_merge_cells(datastore):
- from ..merge import DatasetMerge
- from ..datastore.datastore import DataStore
-
- fn = full_path("merged_zarr.zarr")
- writer = DatasetMerge(
- zarr_path=fn,
- datasets=[datastore, datastore],
- names=["self1", "self2"],
- prepend_text="orig",
- overwrite=True,
- )
- writer.dump()
-
- ds = DataStore(
- fn,
- default_assay="RNA",
- )
-
- df = ds.cells.to_pandas_dataframe(ds.cells.columns)
- df_diff = df[df['orig_RNA_nCounts'] != df['RNA_nCounts']]
- assert len(df_diff) == 0
\ No newline at end of file
diff --git a/scarf/tests/test_metrics.py b/scarf/tests/test_metrics.py
deleted file mode 100644
index 6cbb277b..00000000
--- a/scarf/tests/test_metrics.py
+++ /dev/null
@@ -1,49 +0,0 @@
-import numpy as np
-
-
-def test_metric_lisi(datastore, make_graph):
- lables = np.random.randint(0, 2, datastore.cells.N)
- datastore.cells.insert(
- column_name="samples_id",
- values=lables,
- overwrite=True,
- )
- lisi = datastore.metric_lisi(
- label_colnames=["samples_id"], save_result=False, return_lisi=True
- )
- assert len(lisi[0][1]) == len(datastore.cells.active_index("I"))
-
-
-def test_metric_silhouette(datastore, make_graph, leiden_clustering):
- _ = datastore.metric_silhouette()
-
-
-def test_metric_integration_ari(datastore, make_graph, leiden_clustering):
- lables1 = np.random.randint(0, 2, datastore.cells.N)
- lables2 = np.random.randint(0, 2, datastore.cells.N)
- datastore.cells.insert(
- column_name="lables1",
- values=lables1,
- overwrite=True,
- )
- datastore.cells.insert(
- column_name="lables2",
- values=lables2,
- overwrite=True,
- )
- _ = datastore.metric_integration(batch_labels=["lables1", "lables2"], metric="ari")
-
-def test_metric_integration_nmi(datastore, make_graph, leiden_clustering):
- lables1 = np.random.randint(0, 2, datastore.cells.N)
- lables2 = np.random.randint(0, 2, datastore.cells.N)
- datastore.cells.insert(
- column_name="lables1",
- values=lables1,
- overwrite=True,
- )
- datastore.cells.insert(
- column_name="lables2",
- values=lables2,
- overwrite=True,
- )
- _ = datastore.metric_integration(batch_labels=["lables1", "lables2"], metric="nmi")
\ No newline at end of file
diff --git a/scarf/tests/test_readers.py b/scarf/tests/test_readers.py
deleted file mode 100644
index 52c2d4e7..00000000
--- a/scarf/tests/test_readers.py
+++ /dev/null
@@ -1,80 +0,0 @@
-import numpy as np
-
-
-def test_toy_crdir_assay_feats_table(toy_crdir_reader):
- assert np.all(
- toy_crdir_reader.assayFeats.columns
- == np.array(["RNA", "ADT", "RNA", "HTO", "RNA"])
- )
- assert np.all(
- toy_crdir_reader.assayFeats.values[1:]
- == [[0, 1, 3, 5, 6], [1, 3, 5, 6, 7], [1, 2, 2, 1, 1]]
- )
-
-
-def test_toy_crdir_reader_cells_feats(toy_crdir_reader):
- assert toy_crdir_reader.nCells == 3
- assert toy_crdir_reader.nFeatures == 8
- assert toy_crdir_reader.cell_names() == ["b1", "b2", "b3"]
- assert toy_crdir_reader.feature_names() == [
- "g1",
- "a1",
- "a2",
- "g2",
- "g3",
- "h1",
- "g4",
- "a3",
- ]
- assert toy_crdir_reader.feature_ids() == [
- "g1",
- "a1",
- "a2",
- "g2",
- "g3",
- "h1",
- "g4",
- "a3",
- ]
-
-def test_toy_crdir_empty(toy_crdir_empty):
- assert toy_crdir_empty.nCells == 0
- assert toy_crdir_empty.nFeatures == 4
- assert toy_crdir_empty.feature_names() == [
- "g1",
- "a1",
- "a2",
- "g2",
- ]
- assert toy_crdir_empty.feature_ids() == [
- "g1",
- "a1",
- "a2",
- "g2",
- ]
- # check for raise ValueError
- try:
- toy_crdir_empty.read_header()
- except ValueError:
- pass
-
-def test_crh5reader(crh5_reader):
- assert crh5_reader.nCells == 892
- assert crh5_reader.nFeatures == 36611
- n_assay_feats = list(crh5_reader.assayFeats.T.nFeatures.values)
- assert n_assay_feats == [36601, 10]
-
-
-def test_crdir_reader(crdir_reader):
- assert crdir_reader.nCells == 892
- assert crdir_reader.nFeatures == 36601 # Does not contain 10 ADTs
-
-
-def test_h5ad_reader(h5ad_reader):
- assert h5ad_reader.nCells == 3696 == len(h5ad_reader.cell_ids())
- assert h5ad_reader.nFeatures == 27998 == len(h5ad_reader.feat_names())
-
-
-def test_loom_reader(loom_reader):
- assert loom_reader.nCells == 298 == len(loom_reader.cell_ids())
- assert loom_reader.nFeatures == 16892 == len(loom_reader.feature_names())
diff --git a/scarf/tests/test_writers.py b/scarf/tests/test_writers.py
deleted file mode 100644
index 1be67ac2..00000000
--- a/scarf/tests/test_writers.py
+++ /dev/null
@@ -1,92 +0,0 @@
-import numpy as np
-
-from . import full_path, remove
-
-
-def test_crtozarr(crh5_reader):
- from ..writers import CrToZarr
-
- fn = full_path("dummy_1K_pbmc_citeseq.zarr")
- writer = CrToZarr(crh5_reader, zarr_loc=fn)
- writer.dump()
- remove(fn)
-
-
-def test_crtozarr_fromdir(crdir_reader):
- from ..writers import CrToZarr
-
- fn = full_path("1K_pbmc_citeseq_dir.zarr")
- writer = CrToZarr(crdir_reader, zarr_loc=fn)
- writer.dump()
- remove(fn)
-
-
-def test_h5adtozarr(h5ad_reader):
- from ..writers import H5adToZarr
-
- fn = full_path("bastidas.zarr")
- writer = H5adToZarr(h5ad_reader, zarr_loc=fn)
- writer.dump()
- remove(fn)
-
-
-def test_loomtozarr(loom_reader):
- from ..writers import LoomToZarr
-
- fn = full_path("sympathetic.zarr")
- writer = LoomToZarr(loom_reader, zarr_loc=fn)
- writer.dump()
- remove(fn)
-
-
-def test_sparsetozarr():
- from ..writers import SparseToZarr
- from scipy.sparse import csr_matrix
-
- cols = [1, 3, 8, 2, 3, 1, 2, 8, 9]
- rows = [0, 0, 0, 1, 1, 1, 2, 2, 2]
- data = [1, 10, 15, 10, 20, 2, 3, 1, 5]
- mat = (data, (rows, cols))
- mat = csr_matrix(mat, shape=(3, 10))
-
- fn = full_path("dummy_sparse.zarr")
-
- writer = SparseToZarr(
- mat,
- zarr_loc=fn,
- cell_ids=[f"cell_{x}" for x in range(3)],
- feature_ids=[f"feat_{x}" for x in range(10)],
- )
- writer.dump()
- remove(fn)
-
-
-def test_to_h5ad(datastore):
- # TODO: Evaluate the resulting H5ad file
- from ..writers import to_h5ad
-
- fn = full_path("test_1K_pbmc_citeseq.h5ad")
- to_h5ad(datastore.RNA, fn)
- remove(fn)
-
-
-def test_to_mtx(datastore):
- # TODO: Evaluate the resulting MTX directory
- from ..writers import to_mtx
-
- fn = full_path("test_1K_pbmc_citeseq_dir")
- to_mtx(datastore.RNA, fn)
- remove(fn)
-
-
-def test_zarr_subset(datastore):
- # TODO: Evaluate the resulting subsetted file
-
- from ..writers import SubsetZarr
-
- zarr_path = full_path("subset.zarr")
- writer = SubsetZarr(
- zarr_loc=zarr_path, assays=[datastore.RNA], cell_idx=np.array([1, 10, 100, 500])
- )
- writer.dump()
- remove(zarr_path)
diff --git a/scarf/tools/__init__.py b/scarf/tools/__init__.py
new file mode 100644
index 00000000..e69de29b
diff --git a/scarf/tools/repack_zarr.py b/scarf/tools/repack_zarr.py
new file mode 100644
index 00000000..ae041898
--- /dev/null
+++ b/scarf/tools/repack_zarr.py
@@ -0,0 +1,95 @@
+"""Repack a Zarr store from v2 to v3 with optional sharding."""
+
+import argparse
+from pathlib import Path
+
+import numpy as np
+import zarr
+
+from scarf.storage.zarr_store import (
+ StorageProfile,
+ array_info,
+ finalize_sharded_counts,
+ get_compressors,
+ get_storage_profile,
+ normalize_chunks,
+ open_store,
+)
+
+
+def _copy_group(
+ src: zarr.Group,
+ dst: zarr.Group,
+ profile: StorageProfile,
+) -> None:
+ for key in src.keys():
+ node = src[key]
+ if isinstance(node, zarr.Group):
+ new_group = dst.create_group(key, overwrite=True)
+ for attr_key, attr_val in node.attrs.items():
+ new_group.attrs[attr_key] = attr_val
+ _copy_group(node, new_group, profile)
+ continue
+ chunks = normalize_chunks(node.chunks, node.shape)
+ dst.create_array(
+ key,
+ data=np.asarray(node[...]),
+ chunks=chunks,
+ compressors=get_compressors(profile, zarrFormat=3),
+ overwrite=True,
+ )
+
+
+def repack_store(
+ input_path: str,
+ output_path: str,
+ profile: StorageProfile = "fast_local",
+ shard_counts: bool = True,
+ storage_options: dict | None = None,
+) -> None:
+ """Copy a Zarr store to a new path and optionally shard count arrays.
+
+ Args:
+ input_path: Source Zarr directory.
+ output_path: Destination Zarr directory (created or overwritten).
+ profile: Storage profile for compressors and shard sizes.
+ shard_counts: Repack assay count arrays to sharded layout when True.
+ """
+ src = open_store(input_path, mode="r", storage_options=storage_options)
+ dst = open_store(output_path, mode="w", storage_options=storage_options)
+ _copy_group(src, dst, profile)
+ if not shard_counts:
+ return
+ for key in dst.keys():
+ node = dst[key]
+ if not isinstance(node, zarr.Group):
+ continue
+ if node.attrs.get("is_assay") and "counts" in node:
+ counts = finalize_sharded_counts(dst, key, workspace=None, profile=profile)
+ print(f" {key}/counts: {array_info(counts)}")
+
+
+def main() -> None:
+ parser = argparse.ArgumentParser(
+ description="Repack Zarr v2 stores to v3 with sharding"
+ )
+ parser.add_argument("input", type=Path)
+ parser.add_argument("output", type=Path)
+ parser.add_argument(
+ "--profile",
+ choices=["fast_local", "cloud"],
+ default=get_storage_profile(),
+ )
+ parser.add_argument("--no-shard-counts", action="store_true")
+ args = parser.parse_args()
+ repack_store(
+ str(args.input),
+ str(args.output),
+ profile=args.profile,
+ shard_counts=not args.no_shard_counts,
+ )
+ print(f"Repacked {args.input} -> {args.output}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/scarf/umap.py b/scarf/umap.py
index 22f4bc73..4af6f79f 100644
--- a/scarf/umap.py
+++ b/scarf/umap.py
@@ -4,17 +4,21 @@
# License: BSD 3 clause
import locale
+from typing import Any
-from .utils import logger
+import numpy as np
+
+from .utils import logger, tqdm_params as default_tqdm_params
locale.setlocale(locale.LC_NUMERIC, "C")
__all__ = ["fit_transform"]
-def calc_dens_map_params(graph, dists):
- import numpy as np
-
+def calc_dens_map_params(
+ graph: Any, dists: np.ndarray
+) -> tuple[np.ndarray, np.ndarray]:
+ """Compute densMAP correction terms from graph and KNN distances."""
n_vertices = graph.shape[0]
mu_sum = np.zeros(n_vertices, dtype=np.float32)
ro = np.zeros(n_vertices, dtype=np.float32)
@@ -39,27 +43,26 @@ def calc_dens_map_params(graph, dists):
def simplicial_set_embedding(
- g,
- embedding,
- n_epochs,
- a,
- b,
- random_seed,
- gamma,
- initial_alpha,
- negative_sample_rate,
- densmap_kwds,
- parallel,
- nthreads,
- verbose,
-):
+ g: Any,
+ embedding: np.ndarray,
+ n_epochs: int,
+ a: float,
+ b: float,
+ random_seed: int,
+ gamma: float,
+ initial_alpha: float,
+ negative_sample_rate: float,
+ densmap_kwds: dict[str, Any],
+ parallel: bool,
+ nthreads: int,
+ verbose: bool,
+) -> np.ndarray:
+ """Run UMAP simplicial-set embedding with optional densMAP."""
import numba
from threadpoolctl import threadpool_limits
from umap.umap_ import make_epochs_per_sample
from umap.layouts import optimize_layout_euclidean
from sklearn.utils import check_random_state
- import numpy as np
- from .utils import tqdm_params
# g.data[g.data < (g.data.max() / float(n_epochs))] = 0.0
# g.eliminate_zeros()
@@ -85,10 +88,10 @@ def simplicial_set_embedding(
else:
densmap = False
- # tqdm will be activated if https://github.com/lmcinnes/umap/pull/739
- # is merged and when it is released
- tqdm_params = dict(tqdm_params)
+ tqdm_params = dict(default_tqdm_params)
tqdm_params["desc"] = "Training UMAP"
+ if "disable" not in tqdm_params:
+ tqdm_params["disable"] = not verbose
with threadpool_limits(limits=nthreads):
embedding = optimize_layout_euclidean(
@@ -106,16 +109,17 @@ def simplicial_set_embedding(
initial_alpha=initial_alpha,
negative_sample_rate=negative_sample_rate,
parallel=parallel,
- verbose=verbose,
+ verbose=False,
densmap=densmap,
densmap_kwds=densmap_kwds,
tqdm_kwds=tqdm_params,
move_other=True,
)
- return embedding
+ return np.asarray(embedding)
-def fuzzy_simplicial_set(g, set_op_mix_ratio):
+def fuzzy_simplicial_set(g: Any, set_op_mix_ratio: float) -> Any:
+ """Combine directed and undirected graph components for UMAP."""
tg = g.transpose()
prod = g.multiply(tg)
res = set_op_mix_ratio * (g + tg - prod) + (1.0 - set_op_mix_ratio) * prod
@@ -124,23 +128,44 @@ def fuzzy_simplicial_set(g, set_op_mix_ratio):
def fit_transform(
- graph,
- ini_embed,
- spread,
- min_dist,
- n_epochs,
- random_seed,
- repulsion_strength,
- initial_alpha,
- negative_sample_rate,
- densmap_kwds,
- parallel,
- nthreads,
- verbose,
-):
- from umap.umap_ import find_ab_params
+ graph: Any,
+ ini_embed: np.ndarray,
+ spread: float,
+ min_dist: float,
+ n_epochs: int,
+ random_seed: int,
+ repulsion_strength: float,
+ initial_alpha: float,
+ negative_sample_rate: float,
+ densmap_kwds: dict[str, Any],
+ parallel: bool,
+ nthreads: int,
+ verbose: bool,
+) -> tuple[np.ndarray, float, float]:
+ """Fit UMAP embedding from a fuzzy simplicial set graph.
+
+ Args:
+ graph: Sparse COO adjacency graph.
+ ini_embed: Initial embedding coordinates.
+ spread: UMAP spread parameter.
+ min_dist: UMAP min_dist parameter.
+ n_epochs: Number of optimization epochs.
+ random_seed: Random seed.
+ repulsion_strength: UMAP repulsion strength (gamma).
+ initial_alpha: Initial learning rate.
+ negative_sample_rate: Negative sampling rate.
+ densmap_kwds: Extra densMAP kwargs (empty dict disables densMAP).
+ parallel: Whether to run layout optimization in parallel.
+ nthreads: Thread limit for layout optimization.
+ verbose: Verbose UMAP output.
+
+ Returns:
+ Tuple of (embedding, a, b) UMAP parameters.
+ """
import warnings
+ from umap.umap_ import find_ab_params
+
with warnings.catch_warnings():
warnings.simplefilter("ignore", RuntimeWarning)
a, b = find_ab_params(spread=spread, min_dist=min_dist)
diff --git a/scarf/utils.py b/scarf/utils.py
index 3915728f..eae8d3c3 100644
--- a/scarf/utils.py
+++ b/scarf/utils.py
@@ -2,22 +2,34 @@
- Methods:
- clean_array: returns input array with nan and infinite values removed
- - controlled_compute: performs computation with Dask
+ - controlled_compute: materializes a ChunkedArray into NumPy
- rescale_array: performs edge trimming on values of the input vector
- - show_progress: performs computation with Dask and shows progress bar
+ - show_progress: materializes a ChunkedArray and shows a progress bar
- system_call: executes a command in the underlying operative system
- rolling_window: applies rolling window mean over a vector
"""
+import hashlib
+import resource
+import shutil
import sys
-from typing import List, Optional, TypeAlias, Union
+import tempfile
+import threading
+import time
+from collections.abc import Callable, Iterable, Iterator
+from concurrent.futures import Future, ThreadPoolExecutor
+from contextlib import contextmanager
+from pathlib import Path
+from typing import Any, TypeVar
import numpy as np
import zarr
-from dask.array.core import Array
+from zarr.abc.store import Store
from loguru import logger
from numba import jit
-from tqdm.dask import TqdmCallback
+from numpy.typing import NDArray
+
+from ._types import ZarrMode
__all__ = [
"logger",
@@ -32,12 +44,37 @@
"permute_into_chunks",
"show_dask_progress",
"controlled_compute",
+ "prefetch_blocks",
+ "ColumnBlockPipeline",
+ "iter_column_blocks",
+ "remote_column_disk_ahead",
+ "remote_column_ram_ahead",
+ "process_rss_mb",
+ "rss_peak_tracker",
+ "array_digest",
"rolling_window",
]
logger.remove()
+
+
+def stdout_is_interactive() -> bool:
+ """True when stdout is a TTY (interactive terminal)."""
+ return hasattr(sys.stdout, "isatty") and sys.stdout.isatty()
+
+
+def _flushing_stdout_sink(message: Any) -> None:
+ sys.stdout.write(str(message))
+ sys.stdout.flush()
+
+
logger.add(
- sys.stdout, colorize=True, format="{level}: {message}", level="INFO"
+ _flushing_stdout_sink,
+ colorize=stdout_is_interactive(),
+ format="{message}\n"
+ if not stdout_is_interactive()
+ else "{level}: {message}",
+ level="INFO",
)
tqdm_params = {
@@ -46,17 +83,19 @@
"colour": "#34abeb",
}
-ZARRLOC: TypeAlias = Union[str, zarr.LRUStoreCache]
+type ZARRLOC = str | Store
-def get_log_level():
+def get_log_level() -> int:
+ """Return the current minimum log level configured for Scarf's logger."""
# noinspection PyUnresolvedReferences
- return logger._core.min_level # type: ignore
+ return int(logger._core.min_level) # type: ignore[attr-defined]
def is_notebook() -> bool:
+ """Return True when running inside a Jupyter notebook kernel."""
try:
- shell = get_ipython().__class__.__name__ # type: ignore
+ shell = get_ipython().__class__.__name__ # type: ignore[name-defined]
if shell == "ZMQInteractiveShell":
return True
elif shell == "TerminalInteractiveShell":
@@ -67,13 +106,14 @@ def is_notebook() -> bool:
return False
-def tqdmbar(*args, **kwargs):
+def tqdmbar(*args: Any, **kwargs: Any) -> Any:
+ """Return a tqdm progress bar with Scarf defaults and log-level aware disable."""
params = dict(tqdm_params)
for i in kwargs:
if i in params:
del params[i]
if "disable" not in kwargs and "disable" not in params:
- if get_log_level() <= 20:
+ if get_log_level() <= 20 and stdout_is_interactive():
params["disable"] = False
else:
params["disable"] = True
@@ -87,7 +127,7 @@ def tqdmbar(*args, **kwargs):
return tqdm(*args, **kwargs, **params)
-def set_verbosity(level: Optional[str] = None, filepath: Optional[str] = None):
+def set_verbosity(level: str | None = None, filepath: str | None = None) -> None:
"""Set verbosity level of Scarf's output. Setting value of level='CRITICAL'
should silence all logs. Progress bars are automatically disabled for
levels above 'INFO'.
@@ -98,36 +138,322 @@ def set_verbosity(level: Optional[str] = None, filepath: Optional[str] = None):
is provided then all the logs are printed on standard output.
Returns:
+ None
"""
# noinspection PyUnresolvedReferences
- available_levels = logger._core.levels.keys() # type: ignore
+ available_levels = logger._core.levels.keys() # type: ignore[attr-defined]
if level is None or level not in available_levels:
raise ValueError(
f"Please provide a value for level: {', '.join(available_levels)}"
)
logger.remove()
+ interactive = stdout_is_interactive() and filepath is None
if filepath is None:
- filepath = sys.stdout # type: ignore
+ logger.add(
+ _flushing_stdout_sink,
+ colorize=interactive,
+ format="{message}\n"
+ if not interactive
+ else "{level}: {message}",
+ level=level,
+ )
+ return None
logger.add(
- filepath, # type: ignore
+ filepath, # type: ignore[arg-type]
colorize=True,
format="{level}: {message}",
level=level,
)
+def process_rss_mb() -> float:
+ """Return this process's resident set size in MiB."""
+ try:
+ with open("/proc/self/status") as handle:
+ for line in handle:
+ if line.startswith("VmRSS:"):
+ return int(line.split()[1]) / 1024.0
+ except OSError:
+ pass
+ return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.0
+
+
+@contextmanager
+def rss_peak_tracker(
+ interval_s: float = 0.25,
+) -> Iterator[Callable[[], float]]:
+ """Sample process RSS in the background and yield a peak reader.
+
+ The reader returns the maximum RSS observed since the tracker started,
+ including a final sample on exit.
+ """
+ peak = process_rss_mb()
+ stop = threading.Event()
+
+ def sample_loop() -> None:
+ nonlocal peak
+ while not stop.wait(interval_s):
+ peak = max(peak, process_rss_mb())
+
+ thread = threading.Thread(target=sample_loop, daemon=True)
+ thread.start()
+ try:
+ yield lambda: peak
+ finally:
+ stop.set()
+ thread.join(timeout=2.0)
+ peak = max(peak, process_rss_mb())
+
+
+_T = TypeVar("_T")
+
+
+def _wait_with_heartbeat(
+ future: Future[_T],
+ *,
+ label: str,
+ interval_s: float = 30.0,
+) -> _T:
+ """Wait on ``future``, logging every ``interval_s`` while blocked."""
+ from concurrent.futures import TimeoutError as FuturesTimeoutError
+
+ waited = 0.0
+ while True:
+ try:
+ return future.result(timeout=interval_s)
+ except FuturesTimeoutError:
+ waited += interval_s
+ logger.info(f"{label}: still waiting ({waited:.0f}s elapsed)")
+
+
+class ColumnBlockPipeline:
+ """Ordered block reads with one RAM read-ahead and optional disk staging.
+
+ The next block is prefetched into RAM while the current block is processed.
+ Further blocks (up to ``disk_ahead``) are staged on local disk one at a time
+ so at most one remote read runs for disk spillover while compute continues.
+ """
+
+ def __init__(
+ self,
+ n_blocks: int,
+ read_block: Callable[[int], np.ndarray],
+ *,
+ ram_ahead: int = 1,
+ disk_ahead: int = 5,
+ scratch_dir: str | None = None,
+ ) -> None:
+ self.n_blocks = n_blocks
+ self.read_block = read_block
+ self.disk_ahead = max(0, int(disk_ahead))
+ self._own_scratch = scratch_dir is None
+ self._scratch = scratch_dir or tempfile.mkdtemp(prefix="scarf_col_blocks_")
+ self._disk_done: set[int] = set()
+ self._disk_pending: dict[int, Future[int]] = {}
+ self._disk_future: Future[int] | None = None
+ self._ram_futures: dict[int, Future[np.ndarray]] = {}
+ self.ram_ahead = max(1, int(ram_ahead))
+ pool_workers = max(2, self.ram_ahead + (1 if self.disk_ahead > 0 else 0))
+ self._pool = ThreadPoolExecutor(max_workers=pool_workers)
+ self._lock = threading.Lock()
+
+ def close(self) -> None:
+ self._pool.shutdown(wait=False, cancel_futures=True)
+ if self._own_scratch:
+ shutil.rmtree(self._scratch, ignore_errors=True)
+
+ def __enter__(self) -> "ColumnBlockPipeline":
+ return self
+
+ def __exit__(self, *_exc: object) -> None:
+ self.close()
+
+ def _disk_path(self, block_idx: int) -> Path:
+ return Path(self._scratch) / f"block_{block_idx:04d}.npy"
+
+ def _write_disk(self, block_idx: int) -> int:
+ np.save(
+ self._disk_path(block_idx),
+ self.read_block(block_idx),
+ allow_pickle=False,
+ )
+ with self._lock:
+ self._disk_done.add(block_idx)
+ self._disk_pending.pop(block_idx, None)
+ return block_idx
+
+ def _load_disk(self, block_idx: int) -> np.ndarray:
+ with self._lock:
+ pending = self._disk_pending.pop(block_idx, None)
+ self._disk_done.add(block_idx)
+ if pending is not None:
+ pending.result()
+ path = self._disk_path(block_idx)
+ if not path.is_file():
+ return self.read_block(block_idx)
+ arr = np.load(path)
+ path.unlink(missing_ok=True)
+ return np.asarray(arr)
+
+ def _advance_disk_queue(self, anchor_idx: int) -> None:
+ """Start at most one serial disk fetch within the staging window."""
+ if self.disk_ahead <= 0:
+ return
+ with self._lock:
+ if self._disk_future is not None and not self._disk_future.done():
+ return
+ self._disk_future = None
+ window_end = min(self.n_blocks, anchor_idx + 2 + self.disk_ahead)
+ for idx in range(anchor_idx + 2, window_end):
+ if idx in self._disk_done or self._disk_path(idx).is_file():
+ continue
+ if idx in self._disk_pending:
+ continue
+ future = self._pool.submit(self._write_disk, idx)
+ self._disk_pending[idx] = future
+ self._disk_future = future
+ return
+
+ def _schedule_ram(self, block_idx: int) -> None:
+ if block_idx >= self.n_blocks:
+ return
+ with self._lock:
+ if block_idx in self._ram_futures:
+ return
+ if block_idx in self._disk_done or self._disk_path(block_idx).is_file():
+ future = self._pool.submit(self._load_disk, block_idx)
+ elif block_idx in self._disk_pending:
+
+ def wait_and_load(idx: int = block_idx) -> np.ndarray:
+ pending = self._disk_pending.get(idx)
+ if pending is not None:
+ pending.result()
+ return self._load_disk(idx)
+
+ future = self._pool.submit(wait_and_load)
+ else:
+ future = self._pool.submit(self.read_block, block_idx)
+ self._ram_futures[block_idx] = future
+
+ def schedule_ahead(self, from_block_idx: int) -> None:
+ """Start prefetching blocks after ``from_block_idx``."""
+ for offset in range(1, self.ram_ahead + 1):
+ self._schedule_ram(from_block_idx + offset)
+ self._advance_disk_queue(from_block_idx)
+
+ def take(
+ self,
+ block_idx: int,
+ *,
+ wait_label: str | None = None,
+ ) -> tuple[np.ndarray, float, str]:
+ """Return block ``block_idx``, wait time in seconds, and read source."""
+ t0 = time.perf_counter()
+ source = "r2"
+ label = wait_label or f"block {block_idx + 1}/{self.n_blocks} read"
+
+ def wait_on(future: Future[Any], src: str) -> np.ndarray:
+ nonlocal source
+ source = src
+ return _wait_with_heartbeat(future, label=label)
+
+ if block_idx == 0:
+ arr = self.read_block(0)
+ else:
+ with self._lock:
+ ram_future = self._ram_futures.pop(block_idx, None)
+ if ram_future is not None:
+ arr = wait_on(ram_future, "ram")
+ elif self._disk_path(block_idx).is_file() or block_idx in self._disk_done:
+ arr = self._load_disk(block_idx)
+ source = "disk"
+ else:
+ arr = wait_on(self._pool.submit(self.read_block, block_idx), "r2")
+ wait_sec = time.perf_counter() - t0
+ self.schedule_ahead(block_idx)
+ return arr, wait_sec, source
+
+
+REMOTE_COLUMN_DISK_AHEAD = 5
+
+
+def remote_column_disk_ahead(*, remote: bool, n_blocks: int = 1) -> int:
+ """Disk staging depth for remote column-block reads.
+
+ Disk spill only helps long pipelines with many trailing blocks. For a
+ handful of batches (e.g. marker search) it steals thread-pool workers
+ from RAM read-ahead and re-fetches the same R2 column chunks via disk.
+ """
+ if not remote or n_blocks < 5:
+ return 0
+ return REMOTE_COLUMN_DISK_AHEAD
+
+
+def remote_column_ram_ahead(*, remote: bool, n_blocks: int) -> int:
+ """In-flight RAM prefetches for remote column-block reads."""
+ if not remote:
+ return 1
+ from scarf.storage.budget import READ_AHEAD
+
+ return max(1, min(READ_AHEAD, n_blocks - 1))
+
+
+def iter_column_blocks(
+ n_blocks: int,
+ read_block: Callable[[int], np.ndarray],
+ *,
+ remote: bool = False,
+ disk_ahead: int | None = None,
+ scratch_dir: str | None = None,
+ msg: str | None = None,
+) -> Iterator[tuple[int, np.ndarray, float, str]]:
+ """Iterate column blocks with RAM read-ahead and optional disk staging.
+
+ Yields ``(block_idx, array, read_wait_sec, source)`` where ``source`` is
+ ``"r2"``, ``"ram"``, or ``"disk"``.
+ """
+ if n_blocks <= 0:
+ return
+ ahead = (
+ remote_column_disk_ahead(remote=remote, n_blocks=n_blocks)
+ if disk_ahead is None
+ else max(0, int(disk_ahead))
+ )
+ ram_ahead = remote_column_ram_ahead(remote=remote, n_blocks=n_blocks)
+ block_range = range(n_blocks)
+ with ColumnBlockPipeline(
+ n_blocks,
+ read_block,
+ ram_ahead=ram_ahead,
+ disk_ahead=ahead,
+ scratch_dir=scratch_dir,
+ ) as pipeline:
+ for block_idx in tqdmbar(block_range, desc=msg or "", total=n_blocks):
+ if msg:
+ logger.debug(
+ f"{msg}: reading block {block_idx + 1}/{n_blocks} "
+ f"(rss {process_rss_mb():.0f} MiB)"
+ )
+ wait_label = (
+ f"{msg}: block {block_idx + 1}/{n_blocks} read" if msg else None
+ )
+ arr, wait_sec, source = pipeline.take(block_idx, wait_label=wait_label)
+ yield block_idx, arr, wait_sec, source
+
+
def rescale_array(a: np.ndarray, frac: float = 0.9) -> np.ndarray:
"""Performs edge trimming on values of the input vector.
- Performs edge trimming on values of the input vector and constraints them between within frac and 1-frac density of
- normal distribution created with the sample mean and std. dev. of a.
+ Performs edge trimming on values of the input vector and constrains them
+ between frac and 1-frac density of a normal distribution created with the
+ sample mean and std. dev. of a.
Args:
a: numeric vector
frac: Value between 0 and 1.
- Return:
+ Returns:
The input array, edge trimmed and constrained.
"""
from scipy.stats import norm
@@ -140,89 +466,121 @@ def rescale_array(a: np.ndarray, frac: float = 0.9) -> np.ndarray:
return a
-def clean_array(x, fill_val: int = 0):
+def clean_array(x: NDArray[Any] | list[Any], fill_val: int | float = 0) -> NDArray[Any]:
"""Returns input array with nan and infinite values removed.
Args:
- x (np.ndarray): input array
+ x: input array
fill_val: value to fill zero values with (default: 0)
"""
- x = np.nan_to_num(x, copy=True)
- x[(x == np.inf) | (x == -np.inf)] = 0
- x[x == 0] = fill_val
- return x
+ arr = np.asarray(x, dtype=np.float64)
+ arr = np.nan_to_num(arr, copy=True)
+ arr[(arr == np.inf) | (arr == -np.inf)] = 0
+ arr[arr == 0] = fill_val
+ return arr
def load_zarr(
- zarr_loc: ZARRLOC, mode: str, synchronizer: Optional[zarr.ThreadSynchronizer] = None
+ zarr_loc: ZARRLOC,
+ mode: ZarrMode,
+ synchronizer: Any = None,
+ storage_options: dict[str, Any] | None = None,
) -> zarr.Group:
- if synchronizer is None:
- synchronizer = zarr.ThreadSynchronizer()
- if isinstance(zarr_loc, str):
- return zarr.open_group(zarr_loc, mode=mode, synchronizer=synchronizer)
- else:
- return zarr.group(zarr_loc, synchronizer=synchronizer)
+ """Open a Zarr group at the given path, URI, or store object.
+ Args:
+ zarr_loc: Path, remote URI (s3://, gs://, ...), or a zarr Store instance.
+ mode: Zarr open mode, e.g. ``'r'``, ``'r+'``, or ``'w'``.
+ synchronizer: Optional synchronizer (ignored under Zarr v3).
+ storage_options: Credentials and backend options for remote URIs (obstore S3Config keys).
+
+ Returns:
+ Root Zarr group.
+ """
+ from .storage.zarr_store import configure_zarr_io_for_profile, make_store
+
+ if synchronizer is not None:
+ logger.debug("ThreadSynchronizer is ignored under Zarr v3")
+ store = make_store(
+ zarr_loc,
+ storage_options=storage_options,
+ read_only=(mode == "r"),
+ )
+ configure_zarr_io_for_profile()
+ if isinstance(store, str):
+ return zarr.open_group(store, mode=mode)
+ return zarr.open_group(store=store, mode=mode)
-def controlled_compute(arr, nthreads):
- """Performs computation with Dask.
+
+def prefetch_blocks(
+ block_iter: Iterable[Any],
+ fn: Callable[[Any], Any],
+ max_ahead: int = 1,
+) -> Iterator[Any]:
+ """Apply ``fn`` to blocks with bounded read-ahead while preserving order.
Args:
- arr:
- nthreads: number of threads to use for computation
+ block_iter: Iterable of block objects to process.
+ fn: Callable invoked on each block; its return value is yielded.
+ max_ahead: Maximum number of blocks to compute ahead of the consumer.
- Returns:
- Result of computation.
+ Yields:
+ Results of ``fn(block)`` in the same order as ``block_iter``.
"""
- import dask
+ from .parallel import stream_shards
- try:
- # Multiprocessing may be faster, but it throws exception if SemLock is not implemented.
- # For example, multiprocessing won't work on AWS Lambda, in those scenarios we switch ThreadPoolExecutor
- from multiprocessing.pool import ThreadPool
+ yield from stream_shards(block_iter, fn, workers=max(1, max_ahead))
- pool = ThreadPool(nthreads)
- except Exception:
- from concurrent.futures import ThreadPoolExecutor
- pool = ThreadPoolExecutor(nthreads)
+def controlled_compute(arr: Any, nthreads: int) -> np.ndarray:
+ """Materializes a ChunkedArray, Block or deferred reduction into NumPy.
- with dask.config.set(schedular="threads", pool=pool): # type: ignore
- res = arr.compute()
- return res
+ Args:
+ arr: A ChunkedArray, Block, deferred reduction, or an already-evaluated
+ NumPy array.
+ nthreads: number of threads to use for computation
+
+ Returns:
+ Result of computation as a NumPy array.
+ """
+ if hasattr(arr, "compute"):
+ return np.asarray(arr.compute(nthreads))
+ return np.asarray(arr)
-def show_dask_progress(arr: Array, msg: Optional[str] = None, nthreads: int = 1):
- """Performs computation with Dask and shows progress bar.
+def show_dask_progress(
+ arr: Any, msg: str | None = None, nthreads: int = 1
+) -> np.ndarray:
+ """Materializes a ChunkedArray/reduction while showing a progress bar.
Args:
- arr: A Dask array
+ arr: A ChunkedArray, Block or deferred reduction.
msg: message to log, default None
nthreads: number of threads to use for computation, default 1
Returns:
- Result of computation.
+ Result of computation as a NumPy array.
"""
+ if hasattr(arr, "compute"):
+ return np.asarray(arr.compute(nthreads, msg))
+ return np.asarray(arr)
- params = dict(tqdm_params)
- if "disable" not in params:
- if get_log_level() <= 20:
- params["disable"] = False
- else:
- params["disable"] = True
- with TqdmCallback(desc=msg, **params):
- res = controlled_compute(arr, nthreads)
- return res
+def system_call(command: str) -> None:
+ """Executes a command in the underlying operative system.
+
+ Args:
+ command: Shell command string to run.
-def system_call(command):
- """Executes a command in the underlying operative system."""
+ Returns:
+ None
+ """
import shlex
import subprocess
process = subprocess.Popen(shlex.split(command), stdout=subprocess.PIPE)
while True:
- output = process.stdout.readline() # type: ignore
+ output = process.stdout.readline() # type: ignore[union-attr]
if process.poll() is not None:
break
if output:
@@ -231,35 +589,64 @@ def system_call(command):
return None
+def array_digest(values: np.ndarray) -> str:
+ array = np.ascontiguousarray(values)
+ if array.dtype.hasobject:
+ raise TypeError("Cannot create a deterministic digest for object arrays")
+ digest = hashlib.blake2b(digest_size=16)
+ digest.update(array.dtype.str.encode())
+ digest.update(np.asarray(array.shape, dtype=np.int64).tobytes())
+ digest.update(array.view(np.uint8).tobytes())
+ return digest.hexdigest()
+
+
@jit(nopython=True)
-def rolling_window(a, w):
- n, m = a.shape
- b = np.zeros(shape=(n, m))
- for i in range(n):
- if i < w:
- x = i
- y = w - i
- elif (n - i) < w:
- x = w - (n - i)
- y = n - i
- else:
- x = w // 2
- y = w // 2
- x = i - x
- y = i + y
- for j in range(m):
- b[i, j] = a[x:y, j].mean()
- return b
+def rolling_window(a: np.ndarray, w: int) -> np.ndarray:
+ """Apply a centered rolling mean with window size w along axis 0.
+ Args:
+ a: 2D numeric array.
+ w: Window size (number of rows).
-def permute_into_chunks(size: int, chunk_size: int, seed: int = 42) -> List[np.ndarray]:
+ Returns:
+ Array of the same shape as a with smoothed values.
+ """
+ if a.ndim != 2:
+ raise ValueError("a must be a two-dimensional array")
+ if w <= 0:
+ raise ValueError("w must be greater than zero")
+
+ n, m = a.shape
+ if n == 0:
+ raise ValueError("a must contain at least one row")
+ w = min(w, n)
+ left = (w - 1) // 2
+ right = w // 2
+ result = np.empty((n, m), dtype=np.float64)
+
+ for j in range(m):
+ cumulative = np.empty(n + 1, dtype=np.float64)
+ cumulative[0] = 0.0
+ for i in range(n):
+ cumulative[i + 1] = cumulative[i] + a[i, j]
+ for i in range(n):
+ start = max(0, i - left)
+ stop = min(n, i + right + 1)
+ result[i, j] = (cumulative[stop] - cumulative[start]) / (stop - start)
+ return result
+
+
+def permute_into_chunks(size: int, chunk_size: int, seed: int = 42) -> list[np.ndarray]:
"""
Permute the chunks of an array of the given size.
+
Args:
- size: The size of the array to be permuted
- chunk_size: The size of the chunks to permute
+ size: The size of the array to be permuted.
+ chunk_size: The size of the chunks to permute.
+ seed: Random seed for chunk permutations.
+
Returns:
- A permuted array of the given size
+ List of permuted chunk index arrays.
Examples:
>>> permute_into_chunks(10, 3)
[array([2, 1, 0]), array([3, 5, 4]), array([7, 8, 6]), array([9])]
diff --git a/scarf/writers.py b/scarf/writers.py
index be4fa1ee..4022e685 100644
--- a/scarf/writers.py
+++ b/scarf/writers.py
@@ -5,7 +5,7 @@
- create_zarr_obj_array: Creates and returns a Zarr object array.
- create_zarr_count_assay: Creates and returns a Zarr array with name 'counts'.
- subset_assay_zarr: Selects a subset of the data in an assay in the specified Zarr hierarchy.
- - dask_to_zarr: Creates a Zarr hierarchy from a Dask array.
+ - dask_to_zarr: Creates a Zarr hierarchy from a chunked array.
- to_h5ad: Convert a Zarr file to H5ad format
- to_mtx: Convert a Zarr file to MTX format
@@ -19,15 +19,31 @@
"""
import os
-from typing import Any, Tuple, List, Union, Dict, Optional
+from collections.abc import Iterator
+from typing import Any
import numpy as np
import pandas as pd
-import polars as pl
import zarr
-from scipy.sparse import csr_matrix, coo_matrix
+from scipy.sparse import coo_matrix, csr_matrix
+from ._types import as_zarr_array, as_zarr_group
from .readers import CrReader, H5adReader, NaboH5Reader, LoomReader, CSVReader
+from .storage.zarr_store import (
+ ZarrArraySpec,
+ _group_zarr_format,
+ accumulate_sparse_to_shards,
+ array_shard_rows,
+ count_array_spec,
+ create_metadata_column,
+ create_numeric_array,
+ finalize_sharded_counts,
+ get_storage_profile,
+ is_local_zarr_path,
+ normed_array_spec,
+ write_dense_from_row_batches,
+ write_dense_in_shard_rows,
+)
from .utils import (
controlled_compute,
logger,
@@ -43,6 +59,7 @@
"create_zarr_count_assay",
"subset_assay_zarr",
"dask_to_zarr",
+ "write_renorm_subset_to_zarr",
"SubsetZarr",
"CrToZarr",
"H5adToZarr",
@@ -55,12 +72,42 @@
]
+def _apply_budget_override(
+ mem_budget: int | str | None,
+ nthreads: int | None,
+ working_copies: int | None,
+) -> None:
+ """Install a process resource budget from writer-level overrides, if given.
+
+ Write-time chunk and shard geometry is derived from the active resource
+ budget. Passing any of these lets a caller simulate writing on a machine
+ with a different memory size or core count than the one running the code.
+ When all three are None the currently active budget is left untouched, so
+ callers that set it themselves keep control.
+ """
+ if mem_budget is None and nthreads is None and working_copies is None:
+ return
+ from .storage.budget import resolve_budget, set_resource_budget
+
+ set_resource_budget(
+ resolve_budget(
+ memory=mem_budget, workers=nthreads, working_copies=working_copies
+ )
+ )
+
+
+def _normalize_chunks(chunks: tuple[int, ...] | int) -> tuple[int, ...]:
+ if isinstance(chunks, int):
+ return (chunks,)
+ return chunks
+
+
def create_zarr_dataset(
g: zarr.Group,
name: str,
- chunks: tuple,
+ chunks: tuple[int, ...] | int,
dtype: Any,
- shape: Tuple,
+ shape: tuple[int, ...],
overwrite: bool = True,
) -> zarr.Array:
"""Creates and returns a Zarr array.
@@ -76,112 +123,64 @@ def create_zarr_dataset(
Returns:
A Zarr Array.
"""
- from numcodecs import Blosc
-
- compressor = Blosc(cname="lz4", clevel=5, shuffle=Blosc.BITSHUFFLE)
- return g.create_dataset(
- name,
- chunks=chunks,
- dtype=dtype,
+ spec = ZarrArraySpec(
shape=shape,
- compressor=compressor,
+ chunks=_normalize_chunks(chunks),
+ dtype=dtype,
overwrite=overwrite,
)
-
-
-def dtype_fix(dtype, data: np.ndarray):
- if dtype is None or dtype == object:
- return "U" + str(max([len(str(x)) for x in data]))
- if np.issubdtype(data.dtype, np.dtype("S")):
- try:
- adata = data.astype("U")
- except UnicodeDecodeError:
- adata = np.array([x.decode("UTF-8") for x in data]).astype("U")
- return adata.dtype
- return dtype
+ return create_numeric_array(g, name, spec)
def create_zarr_obj_array(
g: zarr.Group,
name: str,
- data,
- dtype: Union[str, Any] = None,
+ data: Any,
+ dtype: str | Any = None,
overwrite: bool = True,
chunk_size: int = 100000,
- shape: Optional[int] = None,
+ shape: int | None = None,
) -> zarr.Array:
- """Creates and returns a Zarr object array.
-
- A Zarr object array can contain any type of object.
- https://zarr.readthedocs.io/en/stable/tutorial.html#object-arrays
-
- Args:
- g (zarr.hierarchy):
- name (str):
- data ():
- dtype (Union[str, Any]):
- overwrite (bool):
- chunk_size (int):
- shape:
-
- Returns:
- A Zarr object Array.
- """
-
- from numcodecs import Blosc
-
- compressor = Blosc(cname="lz4", clevel=5, shuffle=Blosc.BITSHUFFLE)
-
- if chunk_size is None or chunk_size is False:
- chunks = False
- else:
- chunks = (chunk_size,)
-
- if data is not None:
- data = np.array(data)
- dtype = dtype_fix(dtype, data)
-
- return g.create_dataset(
- name,
- data=data,
- chunks=chunks,
- shape=len(data),
- dtype=dtype,
- overwrite=overwrite,
- compressor=compressor,
- )
- else:
- return g.create_dataset(
- name,
- chunks=chunks,
- shape=shape,
- dtype=dtype,
- overwrite=overwrite,
- compressor=compressor,
- )
+ """Creates and returns a metadata column array."""
+ return create_metadata_column(
+ g,
+ name,
+ data=data,
+ dtype=dtype,
+ overwrite=overwrite,
+ chunkSize=chunk_size,
+ shape=shape if data is None else None,
+ )
def create_zarr_count_assay(
z: zarr.Group,
assay_name: str,
- workspace: Union[str, None],
- chunk_size: Tuple[int, int],
+ workspace: str | None,
+ chunk_size: tuple[int, int],
n_cells: int,
- feat_ids: Union[np.ndarray, List[str]],
- feat_names: Union[np.ndarray, List[str]],
+ feat_ids: np.ndarray | list[str],
+ feat_names: np.ndarray | list[str],
dtype: str = "uint32",
+ *,
+ targetChunkBytes: int | None = None,
+ minFeatureChunk: int | None = None,
+ maxFeatureChunk: int | None = None,
) -> zarr.Array:
"""Creates and returns a Zarr array with name 'counts'.
Args:
z (zarr.Group):
assay_name (str):
- workspace (Union[str, None]):
- chunk_size (Tuple[int, int]):
+ workspace (str | None):
+ chunk_size (tuple[int, int]):
n_cells (int):
- feat_ids (Union[np.ndarray, List[str]]):
- feat_names (Union[np.ndarray, List[str]]):
+ feat_ids (np.ndarray | list[str]):
+ feat_names (np.ndarray | list[str]):
dtype (str = 'uint32'):
+ targetChunkBytes: Optional cloud feature-chunk byte target.
+ minFeatureChunk: Optional lower clamp for feature-chunk width.
+ maxFeatureChunk: Optional upper clamp for feature-chunk width.
Returns:
A Zarr array.
@@ -199,23 +198,48 @@ def create_zarr_count_assay(
)
if workspace is not None:
g = z.create_group(f"matrices/{assay_name}", overwrite=True)
+ n_feats = len(feat_ids)
+ if _group_zarr_format(g) >= 3:
+ spec = count_array_spec(
+ n_cells,
+ n_feats,
+ dtype=dtype,
+ remote=get_storage_profile() == "cloud",
+ targetChunkBytes=targetChunkBytes,
+ minFeatureChunk=minFeatureChunk,
+ maxFeatureChunk=maxFeatureChunk,
+ )
+ return create_numeric_array(g, "counts", spec)
return create_zarr_dataset(
- g, "counts", chunk_size, dtype, (n_cells, len(feat_ids)), overwrite=True
+ g, "counts", chunk_size, dtype, (n_cells, n_feats), overwrite=True
)
+def finalize_writer_counts(
+ store: zarr.Group,
+ assay_name: str,
+ workspace: str | None = None,
+) -> zarr.Array:
+ """Repack count arrays to sharded layout after writing completes."""
+ return finalize_sharded_counts(store, assay_name, workspace)
+
+
def load_count_store(
- z: zarr.Group, assay_name: str, workspace: Union[str, None]
+ z: zarr.Group, assay_name: str, workspace: str | None
) -> zarr.Array:
+ """Return the counts Zarr array for an assay in the given workspace."""
if workspace is None:
- return z[f"{assay_name}/counts"] # type: ignore
- else:
- return z[f"matrices/{assay_name}/counts"] # type: ignore
+ return as_zarr_array(z[f"{assay_name}/counts"], name=f"{assay_name}/counts")
+ return as_zarr_array(
+ z[f"matrices/{assay_name}/counts"],
+ name=f"matrices/{assay_name}/counts",
+ )
def create_cell_data(
- z: zarr.Group, workspace: Union[str, None], ids: np.ndarray, names: np.ndarray
+ z: zarr.Group, workspace: str | None, ids: np.ndarray, names: np.ndarray
) -> zarr.Group:
+ """Create the cellData group with ids, names, and default ``I`` filter column."""
if workspace is None:
g = z.create_group("cellData")
else:
@@ -226,20 +250,18 @@ def create_cell_data(
return g
-def sparse_writer(store: zarr.Array, data_stream, n_cells: int, batch_size: int) -> int:
- (
- s,
- e,
- ) = (
- 0,
- 0,
- )
- n_chunks = n_cells // batch_size + 1
- for a in tqdmbar(data_stream, total=n_chunks):
- e += a.shape[0]
- store.set_coordinate_selection((a.row + s, a.col), a.data)
- s = e
- return e
+def sparse_writer(
+ store: zarr.Array,
+ data_stream: Iterator[coo_matrix],
+ n_cells: int,
+ batch_size: int,
+) -> int:
+ """Write CSR batches into a Zarr count array row-wise.
+
+ Returns:
+ Number of rows written.
+ """
+ return accumulate_sparse_to_shards(store, data_stream, dtype=store.dtype)
class CrToZarr:
@@ -251,6 +273,12 @@ class CrToZarr:
zarr_loc: The file name for the Zarr hierarchy or a store
chunk_size: The requested size of chunks to load into memory and process.
dtype: the dtype of the data.
+ mem_budget: Memory budget driving write-time chunk and shard geometry. Accepts bytes, a
+ suffixed size (e.g. '8G'), or a fraction of total system memory (e.g. '0.6').
+ Set it to simulate writing on a machine with a different memory size.
+ nthreads: Worker count for write-time concurrency. When None, auto-detected.
+ working_copies: Number of concurrent in-memory working copies the memory budget is divided
+ across. When None, uses SCARF_WORKING_COPIES env var or the default.
Attributes:
cr: A CrReader object, containing the Cellranger data.
@@ -262,14 +290,20 @@ def __init__(
self,
cr: CrReader,
zarr_loc: ZARRLOC,
- chunk_size=(1000, 1000),
+ chunk_size: tuple[int, int] = (1000, 1000),
dtype: str = "uint32",
- workspace: Optional[str] = None,
- ):
+ workspace: str | None = None,
+ storage_options: dict[str, Any] | None = None,
+ mem_budget: int | str | None = None,
+ nthreads: int | None = None,
+ working_copies: int | None = None,
+ ) -> None:
+ _apply_budget_override(mem_budget, nthreads, working_copies)
self.cr = cr
self.chunkSizes = chunk_size
self.workspace = workspace
- self.z = load_zarr(zarr_loc=zarr_loc, mode="w")
+ self.storage_options = storage_options
+ self.z = load_zarr(zarr_loc=zarr_loc, mode="w", storage_options=storage_options)
create_cell_data(
z=self.z,
workspace=self.workspace,
@@ -289,7 +323,7 @@ def __init__(
)
@staticmethod
- def _prep_assay_input_ranges(af: pd.DataFrame) -> Dict[str, List[List[int]]]:
+ def _prep_assay_input_ranges(af: pd.DataFrame) -> dict[str, list[list[int]]]:
assay_order = (
af.T.nFeatures.groupby(af.columns).sum().sort_values(ascending=False).index
)
@@ -307,9 +341,9 @@ def _prep_assay_input_ranges(af: pd.DataFrame) -> Dict[str, List[List[int]]]:
@staticmethod
def _prep_feat_index_offset(
- ranges: Dict[str, List[List[int]]]
- ) -> Dict[str, List[int]]:
- feat_offset = {}
+ ranges: dict[str, list[list[int]]],
+ ) -> dict[str, list[int]]:
+ feat_offset: dict[str, list[int]] = {}
for i in ranges:
feat_offset[i] = []
lv = 0
@@ -354,7 +388,7 @@ def dump(self, batch_size: int = 1000, lines_in_mem: int = 100000) -> None:
)
else:
logger.warning(
- f"No feature captured from chunk {s} to {s+a.shape[0]} for assay: {assay}"
+ f"No feature captured from chunk {s} to {s + a.shape[0]} for assay: {assay}"
)
s += a.shape[0]
if s != self.cr.nCells:
@@ -362,6 +396,8 @@ def dump(self, batch_size: int = 1000, lines_in_mem: int = 100000) -> None:
"ERROR: This is a bug in CrToZarr. All cells might not have been successfully "
"written into the zarr file. Please report this issue"
)
+ for assay in input_ranges:
+ finalize_writer_counts(self.z, assay, self.workspace)
class H5adToZarr:
@@ -373,6 +409,12 @@ class H5adToZarr:
assay_name: the name of the assay (e. g. 'RNA')
workspace: An optional workspace id.
chunk_size: The requested size of chunks to load into memory and process.
+ mem_budget: Memory budget driving write-time chunk and shard geometry. Accepts bytes, a
+ suffixed size (e.g. '8G'), or a fraction of total system memory (e.g. '0.6').
+ Set it to simulate writing on a machine with a different memory size.
+ nthreads: Worker count for write-time concurrency. When None, auto-detected.
+ working_copies: Number of concurrent in-memory working copies the memory budget is divided
+ across. When None, uses SCARF_WORKING_COPIES env var or the default.
Attributes:
h5ad: A h5ad object (h5 file with added AnnData structure).
@@ -385,22 +427,31 @@ def __init__(
self,
h5ad: H5adReader,
zarr_loc: ZARRLOC,
- assay_name: Optional[str] = None,
- workspace: Union[str, None] = None,
- chunk_size=(1000, 1000),
- ):
+ assay_name: str | None = None,
+ workspace: str | None = None,
+ chunk_size: tuple[int, int] = (1000, 1000),
+ storage_options: dict[str, Any] | None = None,
+ mem_budget: int | str | None = None,
+ nthreads: int | None = None,
+ working_copies: int | None = None,
+ targetChunkBytes: int | None = None,
+ minFeatureChunk: int | None = None,
+ maxFeatureChunk: int | None = None,
+ ) -> None:
# TODO: support for multiple assay. One of the `var` datasets can be used to group features in separate assays
+ _apply_budget_override(mem_budget, nthreads, working_copies)
self.h5ad = h5ad
self.chunkSizes = chunk_size
self.workspace = workspace
+ self.storage_options = storage_options
if assay_name is None:
logger.info(
- f"No value provided for assay names. Will use default value: 'RNA'"
+ "No value provided for assay names. Will use default value: 'RNA'"
)
self.assayName = "RNA"
else:
self.assayName = assay_name
- self.z = load_zarr(zarr_loc=zarr_loc, mode="w")
+ self.z = load_zarr(zarr_loc=zarr_loc, mode="w", storage_options=storage_options)
self._ini_cell_data()
create_zarr_count_assay(
z=self.z,
@@ -411,10 +462,13 @@ def __init__(
feat_ids=self.h5ad.feat_ids(),
feat_names=self.h5ad.feat_names(),
dtype=self.h5ad.matrixDtype,
+ targetChunkBytes=targetChunkBytes,
+ minFeatureChunk=minFeatureChunk,
+ maxFeatureChunk=maxFeatureChunk,
)
self._ini_feature_data()
- def _ini_cell_data(self):
+ def _ini_cell_data(self) -> None:
ids = self.h5ad.cell_ids()
g = create_cell_data(
z=self.z,
@@ -425,20 +479,29 @@ def _ini_cell_data(self):
for i, j in self.h5ad.get_cell_columns():
create_zarr_obj_array(g, i, j, j.dtype)
- def _ini_feature_data(self):
+ def _ini_feature_data(self) -> None:
if self.workspace is None:
- g: zarr.Group = self.z[f"{self.assayName}/featureData"] # type: ignore
+ feat_group = as_zarr_group(
+ self.z[f"{self.assayName}/featureData"],
+ name=f"{self.assayName}/featureData",
+ )
else:
- g: zarr.Group = self.z[f"{self.workspace}/{self.assayName}/featureData"] # type: ignore
+ feat_group = as_zarr_group(
+ self.z[f"{self.workspace}/{self.assayName}/featureData"],
+ name=f"{self.workspace}/{self.assayName}/featureData",
+ )
for i, j in self.h5ad.get_feat_columns():
- if i not in g:
- create_zarr_obj_array(g, i, j, j.dtype)
+ if i not in feat_group:
+ create_zarr_obj_array(feat_group, i, j, j.dtype)
def dump(self, batch_size: int = 1000) -> None:
- # TODO: add informed description to docstring
- """
+ """Write h5ad matrix data into the Zarr counts array.
+
+ Args:
+ batch_size: Number of cells written per sparse_writer batch.
+
Raises:
- AssertionError: Catches eventual bugs in the class, if number of cells does not match after transformation.
+ AssertionError: If written cell count does not match expected nCells.
Returns:
None
@@ -455,6 +518,7 @@ def dump(self, batch_size: int = 1000) -> None:
"ERROR: This is a bug in H5adToZarr. All cells might not have been successfully "
"written into the zarr file. Please report this issue"
)
+ finalize_writer_counts(self.z, self.assayName, self.workspace)
class NaboH5ToZarr:
@@ -480,22 +544,24 @@ def __init__(
self,
h5: NaboH5Reader,
zarr_loc: ZARRLOC,
- assay_name: Optional[str] = None,
- workspace: Union[str, None] = None,
- chunk_size=(1000, 1000),
+ assay_name: str | None = None,
+ workspace: str | None = None,
+ chunk_size: tuple[int, int] = (1000, 1000),
dtype: str = "uint32",
- ):
+ storage_options: dict[str, Any] | None = None,
+ ) -> None:
self.h5 = h5
self.chunkSizes = chunk_size
self.workspace = workspace
+ self.storage_options = storage_options
if assay_name is None:
logger.info(
- f"No value provided for assay names. Will use default value: 'RNA'"
+ "No value provided for assay names. Will use default value: 'RNA'"
)
self.assayName = "RNA"
else:
self.assayName = assay_name
- self.z = load_zarr(zarr_loc, mode="w") # type: ignore
+ self.z = load_zarr(zarr_loc, mode="w", storage_options=storage_options)
self._ini_cell_data()
create_zarr_count_assay(
z=self.z,
@@ -508,7 +574,7 @@ def __init__(
dtype=dtype,
)
- def _ini_cell_data(self):
+ def _ini_cell_data(self) -> None:
ids = self.h5.cell_ids()
_ = create_cell_data(
z=self.z,
@@ -518,22 +584,21 @@ def _ini_cell_data(self):
)
def dump(self, batch_size: int = 500) -> None:
- # TODO: add informed description to docstring
- """
+ """Write Nabo H5 matrix data into the Zarr counts array.
+
+ Args:
+ batch_size: Number of cells written per batch.
+
Raises:
- AssertionError: Catches eventual bugs in the class, if number of cells does not match after transformation.
+ AssertionError: If written cell count does not match expected nCells.
Returns:
None
"""
store = load_count_store(self.z, self.assayName, self.workspace)
- (
- s,
- e,
- ) = (
- 0,
- 0,
- )
+ batch_size = array_shard_rows(store)
+ s = 0
+ e = 0
n_chunks = self.h5.nCells // batch_size + 1
for a in tqdmbar(self.h5.consume(batch_size), total=n_chunks):
e += a.shape[0]
@@ -544,6 +609,7 @@ def dump(self, batch_size: int = 500) -> None:
"ERROR: This is a bug in NaboH5ToZarr. All cells might not have been successfully "
"written into the zarr file. Please report this issue"
)
+ finalize_writer_counts(self.z, self.assayName, self.workspace)
class LoomToZarr:
@@ -568,22 +634,24 @@ def __init__(
self,
loom: LoomReader,
zarr_loc: ZARRLOC,
- assay_name: Optional[str] = None,
- workspace: Union[str, None] = None,
- chunk_size=(1000, 1000),
- ):
+ assay_name: str | None = None,
+ workspace: str | None = None,
+ chunk_size: tuple[int, int] = (1000, 1000),
+ storage_options: dict[str, Any] | None = None,
+ ) -> None:
# TODO: support for multiple assay. Data from within individual layers can be treated as separate assays
self.loom = loom
self.chunkSizes = chunk_size
self.workspace = workspace
+ self.storage_options = storage_options
if assay_name is None:
logger.info(
- f"No value provided for assay names. Will use default value: 'RNA'"
+ "No value provided for assay names. Will use default value: 'RNA'"
)
self.assayName = "RNA"
else:
self.assayName = assay_name
- self.z = load_zarr(zarr_loc, mode="w")
+ self.z = load_zarr(zarr_loc, mode="w", storage_options=storage_options)
self._ini_cell_data()
create_zarr_count_assay(
z=self.z,
@@ -597,9 +665,9 @@ def __init__(
)
self._ini_feature_data()
- def _ini_cell_data(self):
+ def _ini_cell_data(self) -> None:
ids = np.array(self.loom.cell_ids())
- g = create_cell_data(
+ cell_group = create_cell_data(
z=self.z,
workspace=self.workspace,
ids=ids,
@@ -607,23 +675,32 @@ def _ini_cell_data(self):
)
for i, j in self.loom.get_cell_attrs():
try:
- create_zarr_obj_array(g, i, j, j.dtype)
+ create_zarr_obj_array(cell_group, i, j, j.dtype)
except UnicodeDecodeError:
logger.warning(f"Could not import {i} cell(column) attribute")
- def _ini_feature_data(self):
+ def _ini_feature_data(self) -> None:
if self.workspace is None:
- g: zarr.Group = self.z[f"{self.assayName}/featureData"] # type: ignore
+ feat_group = as_zarr_group(
+ self.z[f"{self.assayName}/featureData"],
+ name=f"{self.assayName}/featureData",
+ )
else:
- g: zarr.Group = self.z[f"{self.workspace}/{self.assayName}/featureData"] # type: ignore
+ feat_group = as_zarr_group(
+ self.z[f"{self.workspace}/{self.assayName}/featureData"],
+ name=f"{self.workspace}/{self.assayName}/featureData",
+ )
for i, j in self.loom.get_feature_attrs():
- create_zarr_obj_array(g, i, j, j.dtype)
+ create_zarr_obj_array(feat_group, i, j, j.dtype)
def dump(self, batch_size: int = 1000) -> None:
- # TODO: add informed description to docstring
- """
+ """Write Loom matrix data into the Zarr counts array.
+
+ Args:
+ batch_size: Number of cells written per sparse_writer batch.
+
Raises:
- AssertionError: Catches eventual bugs in the class, if number of cells does not match after transformation.
+ AssertionError: If written cell count does not match expected nCells.
Returns:
None
@@ -640,6 +717,7 @@ def dump(self, batch_size: int = 1000) -> None:
"ERROR: This is a bug in LoomToZarr. All cells might not have been successfully "
"written into the zarr file. Please report this issue"
)
+ finalize_writer_counts(self.z, self.assayName, self.workspace)
class SparseToZarr:
@@ -669,17 +747,19 @@ def __init__(
self,
csr_mat: csr_matrix,
zarr_loc: ZARRLOC,
- cell_ids: Union[np.ndarray, List[str]],
- feature_ids: Union[np.ndarray, List[str]],
- assay_name: Optional[str] = None,
- workspace: Union[str, None] = None,
- feature_names: Union[np.ndarray, List[str], None] = None,
- chunk_size=(1000, 1000),
- matrix_dtype: Optional[np.dtype] = None,
- ):
+ cell_ids: np.ndarray | list[str],
+ feature_ids: np.ndarray | list[str],
+ assay_name: str | None = None,
+ workspace: str | None = None,
+ feature_names: np.ndarray | list[str] | None = None,
+ chunk_size: tuple[int, int] = (1000, 1000),
+ matrix_dtype: np.dtype | None = None,
+ storage_options: dict[str, Any] | None = None,
+ ) -> None:
self.mat = csr_mat
self.chunkSizes = chunk_size
self.workspace = workspace
+ self.storage_options = storage_options
cell_ids = np.array(cell_ids)
if matrix_dtype is None:
self.matrixDtype = self.mat.dtype
@@ -687,7 +767,7 @@ def __init__(
self.matrixDtype = matrix_dtype
if assay_name is None:
logger.info(
- f"No value provided for assay names. Will use default value: 'RNA'"
+ "No value provided for assay names. Will use default value: 'RNA'"
)
self.assayName = "RNA"
else:
@@ -702,7 +782,7 @@ def __init__(
"ERROR: Number of feature ids are not same as number of features in the matrix"
)
- self.z = load_zarr(zarr_loc, mode="w")
+ self.z = load_zarr(zarr_loc, mode="w", storage_options=storage_options)
_ = create_cell_data(
z=self.z,
workspace=self.workspace,
@@ -722,7 +802,7 @@ def __init__(
dtype=str(self.matrixDtype),
)
- def dump(self, batch_size: Optional[int] = None) -> None:
+ def dump(self, batch_size: int | None = None) -> None:
"""Write out the data matrix into the Zarr hierarchy.
Args:
@@ -739,37 +819,29 @@ def dump(self, batch_size: Optional[int] = None) -> None:
store = load_count_store(self.z, self.assayName, self.workspace)
if batch_size is None:
- batch_size = int(store.chunks[0])
- (
- s,
- e,
- ) = (
- 0,
- 0,
+ batch_size = array_shard_rows(store)
+
+ def row_batches() -> Iterator[coo_matrix]:
+ s = 0
+ for end in range(batch_size, self.nCells + batch_size, batch_size):
+ if s == self.nCells:
+ break
+ if end > self.nCells:
+ end = self.nCells
+ yield self.mat[s:end].tocoo()
+ s = end
+
+ e = accumulate_sparse_to_shards(
+ store,
+ row_batches(),
+ dtype=self.matrixDtype,
)
- n_chunks = self.nCells // batch_size + 1
- for e in tqdmbar(
- range(batch_size, self.nCells + batch_size, batch_size),
- total=n_chunks,
- desc="Writing data matrix",
- ):
- if s == self.nCells:
- raise ValueError(
- "Unexpected error encountered in writing to Zarr. The last iteration has failed. "
- "Please report this issue."
- )
- if e > self.nCells:
- e = self.nCells
- a = self.mat[s:e].tocoo() # type: ignore
- store.set_coordinate_selection(
- (a.row + s, a.col), a.data.astype(self.matrixDtype)
- )
- s = e
if e != self.nCells:
raise AssertionError(
"ERROR: This is a bug in SparseToZarr. All cells might not have been successfully "
"written into the zarr file. Please report this issue"
)
+ finalize_writer_counts(self.z, self.assayName, self.workspace)
class CSVtoZarr:
@@ -794,15 +866,17 @@ def __init__(
cr: CSVReader,
zarr_loc: ZARRLOC,
assay_name: str,
- chunk_size=(1000, 1000),
- workspace: Union[str, None] = None,
- dtype: Optional[np.dtype] = None,
- ):
+ chunk_size: tuple[int, int] = (1000, 1000),
+ workspace: str | None = None,
+ dtype: np.dtype | None = None,
+ storage_options: dict[str, Any] | None = None,
+ ) -> None:
self.csvr = cr
self.assayName = assay_name
self.chunkSizes = chunk_size
self.workspace = workspace
- self.z = load_zarr(zarr_loc, mode="w")
+ self.storage_options = storage_options
+ self.z = load_zarr(zarr_loc, mode="w", storage_options=storage_options)
if dtype is not None:
self.dtype = dtype
else:
@@ -838,40 +912,38 @@ def dump(self) -> None:
"""
store = load_count_store(self.z, self.assayName, self.workspace)
- cell_data_grp: zarr.Group = self.z["cellData"] # type: ignore
+ cell_data_grp = as_zarr_group(self.z["cellData"], name="cellData")
cell_data = [
create_zarr_obj_array(
cell_data_grp, name=x, data=None, dtype=y, shape=self.csvr.nCells
)
- for x, y in zip(self.csvr.cellDataCols, self.csvr.cellDataDtypes) # type: ignore
+ for x, y in zip(self.csvr.cellDataCols, self.csvr.cellDataDtypes or [])
]
- batch_size = self.csvr.pandas_kwargs["chunksize"]
- (
- s,
- e,
- ) = (
- 0,
- 0,
+
+ def count_batches() -> Iterator[np.ndarray]:
+ s = 0
+ for a, c in self.csvr.consume():
+ e = s + a.shape[0]
+ if c is not None:
+ for n, i in enumerate(c.T):
+ cell_data[n][s:e] = i
+ s = e
+ if self.dtype is not None:
+ yield a.astype(self.dtype)
+ else:
+ yield a
+
+ e = write_dense_from_row_batches(
+ store,
+ count_batches(),
+ msg="Writing CSV counts",
)
- if self.csvr.nCells % batch_size == 0:
- n_chunks = int(self.csvr.nCells / batch_size)
- else:
- n_chunks = (self.csvr.nCells // batch_size) + 1
- for a, c in tqdmbar(self.csvr.consume(), total=n_chunks):
- e += a.shape[0]
- if self.dtype is not None:
- store[s:e] = a.astype(self.dtype)
- else:
- store[s:e] = a
- if c is not None:
- for n, i in enumerate(c.T):
- cell_data[n][s:e] = i
- s = e
if e != self.csvr.nCells:
raise AssertionError(
- "ERROR: This is a bug in LoomToZarr. All cells might not have been successfully "
+ "ERROR: This is a bug in CSVtoZarr. All cells might not have been successfully "
"written into the zarr file. Please report this issue"
)
+ finalize_writer_counts(self.z, self.assayName, self.workspace)
def subset_assay_zarr(
@@ -880,8 +952,9 @@ def subset_assay_zarr(
out_grp: str,
cells_idx: np.ndarray,
feat_idx: np.ndarray,
- chunk_size: tuple,
-):
+ chunk_size: tuple[int, int],
+ storage_options: dict[str, Any] | None = None,
+) -> None:
"""Selects a subset of the data in an assay in the specified Zarr
hierarchy.
@@ -892,46 +965,103 @@ def subset_assay_zarr(
zarr_loc: The file name for the Zarr hierarchy.
in_grp: Group in Zarr hierarchy to subset.
out_grp: Group name in Zarr hierarchy to write subsetted assay to.
- cells_idx: A list of cell indices to (keep | drop ?).
- feat_idx: A list of feature indices to (keep | drop ?).
- chunk_size: The requested size of chunks to load into memory and process.
+ cells_idx: Indices of cells to keep in the subset.
+ feat_idx: Indices of features to keep in the subset.
+ chunk_size: Chunk size for the output Zarr array.
Returns:
None
"""
- z = load_zarr(zarr_loc, "r+")
- ig: zarr.Array = z[in_grp] # type: ignore
- og = create_zarr_dataset(
- z, out_grp, chunk_size, "uint32", (len(cells_idx), len(feat_idx))
+ z = load_zarr(zarr_loc, "r+", storage_options=storage_options)
+ ig = as_zarr_array(z[in_grp], name=in_grp)
+ spec = count_array_spec(len(cells_idx), len(feat_idx), dtype="uint32")
+ og = create_numeric_array(z, out_grp, spec)
+ write_dense_in_shard_rows(
+ og,
+ lambda start, end: np.asarray(
+ ig.get_orthogonal_selection((cells_idx[start:end], feat_idx))
+ ),
+ msg="Subsetting assay",
+ )
+ return None
+
+
+def write_renorm_subset_to_zarr(
+ assay: Any,
+ cell_idx: np.ndarray,
+ feat_idx: np.ndarray,
+ z: zarr.Group,
+ loc: str,
+ nthreads: int,
+ log_transform: bool = False,
+ msg: str | None = None,
+ mirror: zarr.Array | None = None,
+) -> None:
+ """Write library-size normalized subset data in a single scattered read pass.
+
+ For HVG subsets the per-cell scale factor is the row sum within each block,
+ so a separate ``counts.sum(axis=1)`` pass is not needed before writing.
+ """
+ counts = assay.rawData[:, feat_idx][cell_idx, :]
+ if msg is None:
+ msg = f"Writing data to {loc}"
+ spec = normed_array_spec(counts.shape[0], counts.shape[1])
+ og = create_numeric_array(z, loc, spec)
+ sf = assay.sf
+
+ def normalize_block(block: Any) -> np.ndarray:
+ block = np.asarray(block)
+ row_sum = block.sum(axis=1)
+ row_sum[row_sum == 0] = 1
+ out = sf * block / row_sum[:, np.newaxis]
+ if log_transform:
+ out = np.log1p(out)
+ return np.asarray(out, dtype=np.float32)
+
+ write_dense_in_shard_rows(
+ og,
+ lambda start, end: normalize_block(
+ controlled_compute(counts[start:end, :], nthreads)
+ ),
+ msg=msg,
+ also_write_to=mirror,
)
- pos_start, pos_end = 0, 0
- for i in tqdmbar(np.array_split(cells_idx, len(cells_idx) // chunk_size[0] + 1)):
- pos_end += len(i)
- og[pos_start:pos_end, :] = ig.get_orthogonal_selection((i, feat_idx))
- pos_start = pos_end
return None
-def dask_to_zarr(df, z, loc, chunk_size, nthreads: int, msg: Optional[str] = None):
- # TODO: perhaps change name of Dask array so it does not get confused with a dataframe
- """Creates a Zarr hierarchy from a Dask array.
+def dask_to_zarr(
+ df: Any,
+ z: zarr.Group,
+ loc: str,
+ chunk_size: tuple[int, int],
+ nthreads: int,
+ msg: str | None = None,
+ mirror: zarr.Array | None = None,
+) -> None:
+ """Creates a Zarr hierarchy from a chunked array.
Args:
- df (): Dask array.
- z (): Zarr hierarchy.
- loc (): Location to write data/Zarr hierarchy to.
- chunk_size (): Size of chunks to load into memory and process.
- nthreads (int): Number of threads to use.
- msg (str): Message to use with progress bar (Default: f"Writing data to {loc}").
+ df: ChunkedArray to materialize and write.
+ z: Root Zarr group.
+ loc: Array path within the group.
+ chunk_size: Legacy row chunk hint (superseded by profile spec when writing normed data).
+ nthreads: Threads for block compute.
+ msg: Progress bar message (default: ``Writing data to {loc}``).
+ mirror: Optional second array of the same shape to write each band into
+ during the same pass (local staging cache).
"""
if msg is None:
msg = f"Writing data to {loc}"
- og = create_zarr_dataset(z, loc, chunk_size, "float64", df.shape)
- pos_start, pos_end = 0, 0
- for i in tqdmbar(df.blocks, total=df.numblocks[0], desc=msg):
- pos_end += i.shape[0]
- og[pos_start:pos_end, :] = controlled_compute(i, nthreads)
- pos_start = pos_end
+ spec = normed_array_spec(df.shape[0], df.shape[1])
+ og = create_numeric_array(z, loc, spec)
+ write_dense_in_shard_rows(
+ og,
+ lambda start, end: controlled_compute(df[start:end, :], nthreads).astype(
+ np.float32, copy=False
+ ),
+ msg=msg,
+ also_write_to=mirror,
+ )
return None
@@ -941,40 +1071,45 @@ class SubsetZarr:
Args:
zarr_loc: Path for the output (subsetted) Zarr file
assays: Source assays to be subsetted. These assays must be from the same dataset
+ in_workspace: Source workspace name (None for legacy layout).
+ out_workspace: Target workspace name in the output Zarr file.
cell_key: Name of a boolean column in cell metadata. The cells with value True are included in the
subset. Only used when cell_idx is None.
- cell_idx: Indices of the cells to be included in the subsetted.
+ cell_idx: Explicit indices of cells to include in the subset.
reset_cell_filter: If True, then the cell filtering information is removed, i.e. even the filtered out cells
are set as True as in the 'I' column. To keep the filtering information set the value for
this parameter to False. (Default value: True)
overwrite_existing_file: If True, then overwrites the existing data. (Default value: False)
- overwrite_cell_data: If True, then overwrites cell data (Default value: True)
+ overwrite_cell_data: If True, then overwrites cell data (Default value: False)
"""
def __init__(
self,
zarr_loc: ZARRLOC,
- assays: list,
- in_workspace: Union[str, None] = None,
- out_workspace: Union[str, None] = None,
- cell_key: Optional[str] = None,
- cell_idx: Optional[np.ndarray] = None,
+ assays: list[Any],
+ in_workspace: str | None = None,
+ out_workspace: str | None = None,
+ cell_key: str | None = None,
+ cell_idx: np.ndarray | None = None,
reset_cell_filter: bool = True,
overwrite_existing_file: bool = False,
overwrite_cell_data: bool = False,
+ storage_options: dict[str, Any] | None = None,
) -> None:
self.resetCells = reset_cell_filter
self.overFn = overwrite_existing_file
self.overCells = overwrite_cell_data
self.inWorkspace = in_workspace
self.outWorkspace = out_workspace
+ self.storage_options = storage_options
self.z = self._check_files(zarr_loc)
self.assays = self._check_assays(assays)
self.cellIdx = self._check_idx(cell_key, cell_idx)
- def _check_files(self, zarr_loc: ZARRLOC):
+ def _check_files(self, zarr_loc: ZARRLOC) -> zarr.Group:
if (
- isinstance(zarr_loc, str)
+ is_local_zarr_path(zarr_loc)
+ and isinstance(zarr_loc, str)
and os.path.isdir(zarr_loc)
and self.overFn is False
):
@@ -983,10 +1118,12 @@ def _check_files(self, zarr_loc: ZARRLOC):
f"If you want to overwrite it then please set overwrite_existing_file to True. "
f"No subsetting was performed."
)
- return load_zarr(zarr_loc=zarr_loc, mode="w")
+ return load_zarr(
+ zarr_loc=zarr_loc, mode="w", storage_options=self.storage_options
+ )
@staticmethod
- def _check_assays(assays):
+ def _check_assays(assays: list[Any]) -> list[Any]:
# if type(assays) != list:
if isinstance(assays, list) is False:
raise TypeError(
@@ -1008,10 +1145,13 @@ def _check_assays(assays):
)
return assays
- def _check_idx(self, cell_key, cell_idx):
+ def _check_idx(
+ self, cell_key: str | None, cell_idx: np.ndarray | None
+ ) -> np.ndarray:
if cell_key is None and cell_idx is None:
raise ValueError("Both `cell_key` and `cell_idx` parameters cannot be None")
if cell_idx is None:
+ resolved: np.ndarray | None = None
for assay in self.assays:
try:
idx = assay.cells.fetch_all(cell_key)
@@ -1023,15 +1163,16 @@ def _check_idx(self, cell_key, cell_idx):
raise ValueError(
f"ERROR: {cell_key} is not of boolean type. Cannot perform subsetting"
)
- if cell_idx is None:
- cell_idx = idx
- else:
- if np.all(cell_idx == idx) is False:
- raise ValueError(
- f"ERROR: Provided cell_key {cell_key} is not consistent across the assays. "
- f"Please make sure that the assays are from the same DataStore."
- )
- cell_idx = np.where(cell_idx)[0]
+ if resolved is None:
+ resolved = idx
+ elif np.all(resolved == idx) is False:
+ raise ValueError(
+ f"ERROR: Provided cell_key {cell_key} is not consistent across the assays. "
+ f"Please make sure that the assays are from the same DataStore."
+ )
+ if resolved is None:
+ raise ValueError("No assays available for cell index resolution")
+ cell_idx = np.where(resolved)[0]
else:
cell_idx = np.array(cell_idx)
if np.issubdtype(cell_idx.dtype, np.integer) is False:
@@ -1044,32 +1185,31 @@ def _check_idx(self, cell_key, cell_idx):
)
return cell_idx
- def _prep_cell_data(self):
+ def _prep_cell_data(self) -> None:
if self.outWorkspace is None:
cell_slot = "cellData"
else:
cell_slot = f"{self.outWorkspace}/cellData"
if cell_slot in self.z:
- g: zarr.Group = self.z[cell_slot] # type: ignore
+ cell_group = as_zarr_group(self.z[cell_slot], name=cell_slot)
else:
- g: zarr.Group = self.z.create_group(cell_slot)
+ cell_group = self.z.create_group(cell_slot)
- if self.outWorkspace is None:
- cell_data = self.assays[0].z["/cellData"]
- else:
- cell_data = self.assays[0].z[f"/{self.inWorkspace}/cellData"]
+ cell_data = self.assays[0].cells.locations["primary"]
n_cells = len(self.cellIdx)
for i in cell_data.keys():
- if i in g and self.overCells is False:
+ if i in cell_group and self.overCells is False:
continue
if i in ["I"] and self.resetCells:
- create_zarr_obj_array(g, "I", [True for _ in range(n_cells)], "bool")
+ create_zarr_obj_array(
+ cell_group, "I", [True for _ in range(n_cells)], "bool"
+ )
continue
v = cell_data[i][:][self.cellIdx]
- create_zarr_obj_array(g, i, v, dtype=v.dtype)
+ create_zarr_obj_array(cell_group, i, v, dtype=v.dtype)
- def _prep_counts(self):
+ def _prep_counts(self) -> None:
n_cells = len(self.cellIdx)
for assay in self.assays:
create_zarr_count_assay(
@@ -1083,37 +1223,33 @@ def _prep_counts(self):
dtype=assay.rawData.dtype,
)
- def dump(self):
+ def dump(self) -> None:
self._prep_cell_data()
self._prep_counts()
for assay in self.assays:
raw_data = assay.rawData[self.cellIdx]
if self.outWorkspace is None:
- store = self.z[f"{assay.name}/counts"]
+ store = as_zarr_array(
+ self.z[f"{assay.name}/counts"],
+ name=f"{assay.name}/counts",
+ )
else:
- store = self.z[f"matrices/{assay.name}/counts"]
- (
- s,
- e,
- ) = (
- 0,
- 0,
+ store = as_zarr_array(
+ self.z[f"matrices/{assay.name}/counts"],
+ name=f"matrices/{assay.name}/counts",
+ )
+ write_dense_in_shard_rows(
+ store,
+ lambda start, end: raw_data[start:end, :].compute(),
+ msg=f"Subsetting assay: {assay.name}",
)
- for a in tqdmbar(
- raw_data.blocks,
- desc=f"Subsetting assay: {assay.name}",
- total=raw_data.numblocks[0],
- ):
- if a.shape[0] > 0:
- e += a.shape[0]
- store[s:e] = a.compute()
- s = e
+ finalize_writer_counts(self.z, assay.name, self.outWorkspace)
def to_h5ad(
- assay,
+ assay: Any,
h5ad_filename: str,
- embeddings_cols: Optional[List[str]] = None,
+ embeddings_cols: list[str] | None = None,
skip_recalc_nfeats: bool = True,
n_threads: int = 4,
) -> None:
@@ -1131,9 +1267,8 @@ def to_h5ad(
None
"""
import h5py
- from dask import array as da
- def save_attr(group, col, scarf_col, md):
+ def save_attr(group: str, col: str, scarf_col: str, md: Any) -> None:
d = md.fetch_all(scarf_col)
d_type = d.dtype
if np.issubdtype(d_type, np.number) or np.issubdtype(d_type, bool):
@@ -1141,7 +1276,7 @@ def save_attr(group, col, scarf_col, md):
else:
d_type = h5py.special_dtype(vlen=str)
try:
- h5[group].create_dataset(col, data=d.astype(d_type)) # type: ignore
+ h5[group].create_dataset(col, data=d.astype(d_type))
except TypeError:
logger.warning(f"Dtype issue in {col}, {d.type} ({d_type})")
@@ -1154,7 +1289,7 @@ def save_attr(group, col, scarf_col, md):
assay.cells.insert(
f"{assay.name}_nFeatures",
show_dask_progress(
- da.count_nonzero(assay.rawData, axis=1), # type: ignore
+ assay.rawData.count_nonzero(axis=1),
msg="Preflight: recalculating nFeatures",
nthreads=n_threads,
),
@@ -1171,11 +1306,11 @@ def save_attr(group, col, scarf_col, md):
mat_dtype = assay.rawData.dtype
else:
mat_dtype = int
- h5["X"].create_dataset( # type: ignore
+ h5["X"].create_dataset(
i, (s,), chunks=True, compression="gzip", dtype=mat_dtype
)
- h5["X/indptr"][:] = np.array([0] + list(n_feats_per_cell.cumsum())).astype(int) # type: ignore
+ h5["X/indptr"][:] = np.array([0] + list(n_feats_per_cell.cumsum())).astype(int)
s, e = 0, 0
for i in tqdmbar(
@@ -1185,8 +1320,8 @@ def save_attr(group, col, scarf_col, md):
):
i = csr_matrix(i.compute())
e += i.data.shape[0]
- h5["X/data"][s:e] = i.data # type: ignore
- h5["X/indices"][s:e] = i.indices # type: ignore
+ h5["X/data"][s:e] = i.data
+ h5["X/indices"][s:e] = i.indices
s = e
attrs = {
"encoding-type": "csr_matrix",
@@ -1247,11 +1382,11 @@ def save_attr(group, col, scarf_col, md):
h5["var"].attrs[i] = j
if len(emb_cols) > 0:
- emb_cols = np.array(emb_cols)
- c = pd.Series([x[:-1] for x in emb_cols])
+ emb_names = np.array(emb_cols)
+ c = pd.Series([x[:-1] for x in emb_names])
for i in c.unique():
- data = np.array([assay.cells.fetch_all(x) for x in emb_cols[c == i]]).T
- h5["obsm"].create_dataset( # type: ignore
+ data = np.array([assay.cells.fetch_all(x) for x in emb_names[c == i]]).T
+ h5["obsm"].create_dataset(
i.lower().replace(f"{assay.name.lower()}_", "X_"), data=data
)
@@ -1259,7 +1394,7 @@ def save_attr(group, col, scarf_col, md):
return None
-def to_mtx(assay, mtx_directory: str, compress: bool = False):
+def to_mtx(assay: Any, mtx_directory: str, compress: bool = False) -> None:
"""Save an assay as a Matrix Market directory.
Args:
@@ -1307,9 +1442,9 @@ def to_mtx(assay, mtx_directory: str, compress: bool = False):
def bed_to_sparse_array(
bed_fn: str,
bin_size: int,
- chrom_sizes: Dict[str, int],
+ chrom_sizes: dict[str, int],
min_counts_per_cell: int = 500,
- read_chunk_size=1e6,
+ read_chunk_size: float = 1e6,
sep: str = "\t",
chrom_col: int = 0,
start_col: int = 1,
@@ -1318,8 +1453,8 @@ def bed_to_sparse_array(
count_col: int = 4,
comments_startswith: str = "#",
disable_tqdm: bool = False,
- chrom_modifier=None,
-):
+ chrom_modifier: Any = None,
+) -> tuple[csr_matrix, pd.Series, pd.Series]:
"""
Args:
@@ -1343,18 +1478,18 @@ def bed_to_sparse_array(
"""
import gc
- def feat_mapper(x):
+ def feat_mapper(x: str) -> int:
return feat_idx.get(x, n_feats)
- def default_chrom_modifier(x):
+ def default_chrom_modifier(x: str) -> str:
return x + "_"
- feat_idx = {}
+ feat_idx: dict[str, int] = {}
for i in tqdmbar(chrom_sizes, disable=disable_tqdm, desc="Calculating bin indices"):
for j in range((chrom_sizes[i] // bin_size) + 1):
feat_idx[f"{i}_{j}"] = len(feat_idx)
- cell_idx = {}
- mat = []
+ cell_idx: dict[Any, int] = {}
+ mat_chunks: list[np.ndarray] = []
n_feats = len(feat_idx)
if chrom_modifier is None:
chrom_modifier = default_chrom_modifier
@@ -1376,7 +1511,7 @@ def default_chrom_modifier(x):
for i in df[barcode_col].unique():
if i not in cell_idx:
cell_idx[i] = len(cell_idx)
- mat.append(
+ mat_chunks.append(
np.vstack(
[
np.fromiter(map(cell_idx.get, df[barcode_col].values), dtype=int),
@@ -1385,10 +1520,11 @@ def default_chrom_modifier(x):
]
).T
)
- mat = np.vstack(mat)
+ mat_arr = np.vstack(mat_chunks)
gc.collect()
mat = csr_matrix(
- (mat[:, 2], (mat[:, 0], mat[:, 1])), shape=(len(cell_idx), n_feats + 1)
+ (mat_arr[:, 2], (mat_arr[:, 0], mat_arr[:, 1])),
+ shape=(len(cell_idx), n_feats + 1),
)
gc.collect()
idx = np.array(mat.sum(axis=1))[:, 0] > min_counts_per_cell
diff --git a/setup.py b/setup.py
index 4b86e972..43d9878b 100644
--- a/setup.py
+++ b/setup.py
@@ -1,24 +1,12 @@
-from setuptools import setup, find_packages
-from setuptools.command.install import install
-import os
import glob
+import os
-
-def read(f_name):
- with open(os.path.join(os.path.dirname(__file__), f_name)) as fp:
- return fp.read().rstrip("\n")
-
-
-def read_lines(f_name):
- return read(f_name).split("\n")
+from setuptools import setup
+from setuptools.command.install import install
class PostInstallCommand(install):
- """
- Post-installation for installation mode.
- From here:
- https://github.com/benfred/py-spy/blob/290584dde76834599d66d74b64165dfe9a357ef5/setup.py#L42
- """
+ """Copy bundled binaries from bin/ into the install scripts directory."""
def run(self):
install.run(self)
@@ -35,35 +23,4 @@ def run(self):
os.chmod(target, 0o655)
-if __name__ == "__main__":
- version = read("VERSION").rstrip("\n")
- core_requirements = read_lines("requirements.txt")
- extra_requirements = read_lines("requirements_extra.txt")
-
- setup(
- name="scarf",
- version=version,
- python_requires=">=3.11",
- description="Scarf: A scalable tool for single-cell omics data analysis",
- long_description=read("pypi_README.rst"),
- long_description_content_type="text/x-rst",
- author="Parashar Dhapola",
- author_email="parashar.dhapola@gmail.com",
- url="https://github.com/parashardhapola/scarf",
- license="BSD 3-Clause",
- classifiers=[
- "Development Status :: 4 - Beta",
- "License :: OSI Approved :: BSD License",
- "Natural Language :: English",
- "Programming Language :: Python :: 3",
- "Programming Language :: Python :: 3.12",
- ],
- keywords=["single-cell"],
- install_requires=core_requirements,
- extras_require={
- "extra": extra_requirements,
- },
- packages=find_packages(exclude=["tests*"]),
- include_package_data=False,
- cmdclass={"install": PostInstallCommand},
- )
+setup(cmdclass={"install": PostInstallCommand})
diff --git a/tests/__init__.py b/tests/__init__.py
new file mode 100644
index 00000000..1afaae52
--- /dev/null
+++ b/tests/__init__.py
@@ -0,0 +1,26 @@
+import os
+import shutil
+
+__all__ = ["full_path", "remove", "dask_total_sum"]
+
+_DATASETS_DIR = os.path.join(os.path.dirname(__file__), "datasets")
+
+
+def dask_total_sum(raw_data) -> int:
+ total = 0
+ for block in raw_data.blocks:
+ total += int(block.compute().sum())
+ return total
+
+
+def full_path(fn, *args):
+ if fn == "" or fn is None:
+ return _DATASETS_DIR
+ return os.path.join(_DATASETS_DIR, fn, *args)
+
+
+def remove(dir_path):
+ if os.path.isdir(dir_path):
+ shutil.rmtree(dir_path)
+ elif os.path.exists(dir_path):
+ os.unlink(dir_path)
diff --git a/tests/conftest.py b/tests/conftest.py
new file mode 100644
index 00000000..9474b133
--- /dev/null
+++ b/tests/conftest.py
@@ -0,0 +1,18 @@
+import sys
+
+import pytest
+
+from scarf.utils import logger
+
+pytest_plugins = [
+ "tests.fixtures_downloader",
+ "tests.fixtures_readers",
+ "tests.fixtures_datastore",
+]
+
+
+@pytest.fixture(scope="session", autouse=True)
+def _quiet_test_logs() -> None:
+ """Keep pytest output readable while standalone CLIs retain INFO logs."""
+ logger.remove()
+ logger.add(sys.stderr, level="ERROR")
diff --git a/scarf/tests/datasets/markers_all_clusters.csv b/tests/datasets/markers_all_clusters.csv
similarity index 94%
rename from scarf/tests/datasets/markers_all_clusters.csv
rename to tests/datasets/markers_all_clusters.csv
index 0f83c217..db781006 100644
--- a/scarf/tests/datasets/markers_all_clusters.csv
+++ b/tests/datasets/markers_all_clusters.csv
@@ -29,8 +29,8 @@ BCL11B,SIRPA,CD163,PYGL,PDLIM1,CST7,TRDC,PDK1,PGM2L1,CSF1R
,ACER3,HPSE,PGD,TCL1A,MAP4K1,PRSS23,NELL2,CD28,TNNI2
,CACNA2D3,FGL2,VCAN,PLPP5,CMC1,CCL5,SH3YL1,CD226,DAPK1
,PLXDC2,TLR8,PLBD1,BLNK,FKBP11,C1orf21,SNHG14,,LRRC25
-,HPSE,TMEM176A,LRRK2,CD24,SYNE2,XCL2,CAMK2G,,LILRB1
-,NIBAN2,DUSP6,RETN,CD40,BDH2,CTSW,DYRK2,,LINC02432
+,HPSE,TMEM176A,LRRK2,CD24,SYNE2,XCL2,CAMK2G,,LINC02432
+,NIBAN2,DUSP6,RETN,CD40,BDH2,CTSW,DYRK2,,LILRB1
,SORT1,NLRP3,AQP9,BCL11A,CD3D,PLEKHF1,,,CXCL10
,INPPL1,IMPA2,SLC11A1,COBLL1,PATL2,KLRB1,,,RNF144B
,RTN1,DSC2,ALDH2,CD72,CD69,MCTP2,,,CDK2AP1
@@ -56,8 +56,8 @@ BCL11B,SIRPA,CD163,PYGL,PDLIM1,CST7,TRDC,PDK1,PGM2L1,CSF1R
,RGS18,LILRA5,GM2A,EAF2,,TFDP2,,,CALHM6
,DOK3,NOD2,SMARCD3,MZB1,,USP28,,,TPPP3
,PELI2,MAFB,NAIP,CYB561A3,,HENMT1,,,LST1
-,NFAM1,LTBR,S1PR3,SWAP70,,GK5,,,DUSP5
-,ALDH1A1,SECTM1,VSTM1,FCGR2B,,GPR174,,,FCER1G
+,NFAM1,SECTM1,S1PR3,SWAP70,,GK5,,,DUSP5
+,ALDH1A1,LTBR,VSTM1,FCGR2B,,GPR174,,,FCER1G
,PLA2G7,NBPF14,RBP7,BCL7A,,CD38,,,HCAR3
,CRTAP,AGAP3,DENND6B,MICAL3,,ZAP70,,,TNFRSF8
,OAF,IFIT2,NLRP12,FCRL3,,DDIT4,,,SIDT2
@@ -126,10 +126,10 @@ BCL11B,SIRPA,CD163,PYGL,PDLIM1,CST7,TRDC,PDK1,PGM2L1,CSF1R
,FNDC3B,GRN,PTAFR,TCF3,,SYNE1,,,MAFB
,CYBB,KCNK6,SHTN1,SETBP1,,LYAR,,,IGSF6
,SGMS2,TNFSF13,AL355075.4,LINC01215,,GPRIN3,,,ADGRE1
-,RAB31,ATP6V1B2,TNFSF13,ZNF318,,F2R,,,CD86
-,,LPCAT2,ACAP3,SMARCB1,,LPCAT1,,,IFIT3
-,,C2CD2L,KLF4,MEF2C,,SH2D1A,,,APH1B
-,,HOMER3,HOMER3,RIC3,,STAT4,,,PLAGL2
+,RAB31,ATP6V1B2,TNFSF13,SMARCB1,,F2R,,,CD86
+,,LPCAT2,ACAP3,ZNF318,,LPCAT1,,,IFIT3
+,,C2CD2L,KLF4,MEF2C,,SH2D1A,,,PLAGL2
+,,HOMER3,HOMER3,RIC3,,STAT4,,,APH1B
,,C3AR1,CD101,ODC1,,EOMES,,,KCNJ2
,,MYOF,CD302,LINC01781,,KLHDC4,,,HSBP1
,,EMILIN2,INPPL1,TIFA,,ATP2A3,,,TTYH3
@@ -139,8 +139,8 @@ BCL11B,SIRPA,CD163,PYGL,PDLIM1,CST7,TRDC,PDK1,PGM2L1,CSF1R
,,NFE2,AGO4,SLC9A7,,TNIK,,,PIK3AP1
,,KCTD12,LIN7A,FUT8,,EBP,,,AP2A1
,,LRRK2,FCN1,BEND5,,MEX3C,,,TOR4A
-,,P2RX7,MAFG,AC008969.1,,SNTB2,,,SIGLEC5
-,,CLEC7A,BCL6,HLA-DRA,,MRPL10,,,LINC00467
+,,P2RX7,MAFG,AC008969.1,,SNTB2,,,LINC00467
+,,CLEC7A,BCL6,HLA-DRA,,MRPL10,,,SIGLEC5
,,IFI44L,GAS7,HLA-DMA,,PTPN7,,,HGF
,,MSRB1,APOBR,IGHG1,,CD7,,,SPI1
,,MPZL2,CARS2,IL4R,,CYTOR,,,PPM1N
@@ -182,8 +182,8 @@ BCL11B,SIRPA,CD163,PYGL,PDLIM1,CST7,TRDC,PDK1,PGM2L1,CSF1R
,,CXCL10,ZNF317,,,,,,TUBA1B
,,PER1,OSCAR,,,,,,TIMP1
,,RXRA,FOSL2,,,,,,TRIQK
-,,INPPL1,TIMP2,,,,,,SGPP1
-,,OAS3,ZNF516,,,,,,HLA-DRB5
+,,OAS3,TIMP2,,,,,,SGPP1
+,,INPPL1,ZNF516,,,,,,HLA-DRB5
,,JAK2,VENTX,,,,,,AC147067.1
,,FCN1,APLP2,,,,,,TTYH2
,,LILRA1,MXD1,,,,,,CARD9
diff --git a/tests/download_fixtures.py b/tests/download_fixtures.py
new file mode 100644
index 00000000..1d38ff95
--- /dev/null
+++ b/tests/download_fixtures.py
@@ -0,0 +1,167 @@
+"""Download bundled test datasets for CI and local development."""
+
+import argparse
+import gzip
+import sys
+import tarfile
+import tempfile
+import urllib.error
+import urllib.request
+from pathlib import Path
+
+import h5py
+import pandas as pd
+from scipy.sparse import csr_matrix
+
+_FIXTURES_BASE_URL = "https://raw.githubusercontent.com/parashardhapola/scarf/master/scarf/tests/datasets"
+
+FIXTURE_FILES = (
+ "1K_pbmc_citeseq.h5",
+ "1K_pbmc_citeseq.zarr.tar.gz",
+ "500_pbmc_atac.zarr.tar.gz",
+ "toy_cr_dir.tar.gz",
+ "toy_cr_dir_empty.tar.gz",
+ "sympathetic.loom",
+ "cell_attributes.csv",
+ "knn_indices.npy",
+ "knn_distances.npy",
+ "knn_weights.npy",
+ "atac_knn_indices.npy",
+ "atac_knn_distances.npy",
+ "markers_cluster1.csv",
+ "unified_UMAP_coords.npy",
+ "pseudotime_markers_r_values.csv",
+ "aggregated_feat_idx.npy",
+ "aggregated_df_top_10.npy",
+ "pseudotime_clusters.npy",
+ "ptime_modules_group_1.npy",
+)
+
+
+def datasets_dir() -> Path:
+ return Path(__file__).resolve().parent / "datasets"
+
+
+def download_fixtures(*, force: bool = False) -> None:
+ target = datasets_dir()
+ target.mkdir(parents=True, exist_ok=True)
+
+ missing: list[tuple[str, urllib.error.HTTPError]] = []
+ for name in FIXTURE_FILES:
+ dest = target / name
+ if dest.is_file() and not force:
+ continue
+ url = f"{_FIXTURES_BASE_URL}/{name}"
+ try:
+ urllib.request.urlretrieve(url, dest)
+ except urllib.error.HTTPError as exc:
+ missing.append((name, exc))
+
+ if missing:
+ details = "\n".join(f" {name}: HTTP {exc.code}" for name, exc in missing)
+ msg = f"Failed to download {len(missing)} fixture(s):\n{details}"
+ raise RuntimeError(msg)
+
+
+def build_mtx_dir_fixture() -> None:
+ archive = datasets_dir() / "1K_pbmc_citeseq_dir.tar.gz"
+ if archive.is_file():
+ return
+
+ h5_path = datasets_dir() / "1K_pbmc_citeseq.h5"
+ if not h5_path.is_file():
+ msg = (
+ "Cannot build 1K_pbmc_citeseq_dir.tar.gz without "
+ f"{h5_path.name}; download core fixtures first."
+ )
+ raise RuntimeError(msg)
+
+ from scarf.readers import CrDirReader, CrH5Reader
+
+ reader = CrH5Reader(str(h5_path))
+ rna = "RNA"
+ assay = reader.assayFeats[rna]
+ start, end = int(assay["start"]), int(assay["end"])
+ feat_ids = reader.feature_ids(rna)
+ feat_names = reader.feature_names(rna)
+ feat_types = reader.feature_types()
+ cells = reader.cell_names()
+
+ with tempfile.TemporaryDirectory(prefix="scarf_mtx_fixture_") as tmp:
+ mtx_dir = Path(tmp) / "1K_pbmc_citeseq_dir"
+ mtx_dir.mkdir()
+
+ with gzip.open(mtx_dir / "features.tsv.gz", "wt") as handle:
+ for idx in range(start, end):
+ handle.write(
+ f"{feat_ids[idx - start]}\t{feat_names[idx - start]}\t{feat_types[idx]}\n"
+ )
+
+ with gzip.open(mtx_dir / "barcodes.tsv.gz", "wt") as handle:
+ for cell in cells:
+ handle.write(f"{cell}\n")
+
+ with h5py.File(h5_path) as h5:
+ matrix = h5["matrix"]
+ mat = csr_matrix(
+ (matrix["data"][:], matrix["indices"][:], matrix["indptr"][:]),
+ shape=(len(cells), reader.nFeatures),
+ )
+ rna_mat = mat[:, start:end].T.tocoo()
+
+ with gzip.open(mtx_dir / "matrix.mtx.gz", "wt") as handle:
+ handle.write(
+ "%%MatrixMarket matrix coordinate integer general\n% Generated by Scarf\n"
+ )
+ handle.write(f"{rna_mat.shape[0]} {rna_mat.shape[1]} {rna_mat.nnz}\n")
+ pd.DataFrame(
+ {"row": rna_mat.row + 1, "col": rna_mat.col + 1, "d": rna_mat.data}
+ ).to_csv(
+ handle,
+ sep=" ",
+ header=False,
+ index=False,
+ mode="a",
+ lineterminator="\n",
+ )
+
+ CrDirReader(str(mtx_dir))
+
+ with tarfile.open(archive, "w:gz") as tar:
+ tar.add(mtx_dir, arcname=mtx_dir.name)
+
+
+def download_optional_h5ad() -> None:
+ sample = "bastidas-ponce_4K_pancreas-d15_rnaseq"
+ local_h5ad = datasets_dir() / sample / "data.h5ad"
+ if local_h5ad.is_file():
+ return
+
+ from scarf.downloader import fetch_dataset
+
+ fetch_dataset(sample, save_path=str(datasets_dir()))
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument(
+ "--force",
+ action="store_true",
+ help="Re-download fixtures even when files already exist.",
+ )
+ parser.add_argument(
+ "--with-h5ad",
+ action="store_true",
+ help="Also fetch bastidas-ponce h5ad from OSF (requires network).",
+ )
+ args = parser.parse_args(argv)
+
+ download_fixtures(force=args.force)
+ build_mtx_dir_fixture()
+ if args.with_h5ad:
+ download_optional_h5ad()
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/tests/fixtures_datastore.py b/tests/fixtures_datastore.py
new file mode 100644
index 00000000..c92574b9
--- /dev/null
+++ b/tests/fixtures_datastore.py
@@ -0,0 +1,290 @@
+import os
+import shutil
+import tempfile
+
+import numpy as np
+import pandas as pd
+import pytest
+
+from . import full_path, remove, dask_total_sum
+
+
+def _extract_zarr_fixture(tar_path: str, prefix: str) -> tuple[str, str]:
+ import tarfile
+
+ temp_dir = tempfile.mkdtemp(prefix=prefix)
+ with tarfile.open(tar_path, "r:gz") as tar:
+ tar.extractall(temp_dir, filter="data")
+ if os.path.isfile(os.path.join(temp_dir, ".zgroup")) or os.path.isfile(
+ os.path.join(temp_dir, "zarr.json")
+ ):
+ return temp_dir, temp_dir
+ for name in sorted(os.listdir(temp_dir)):
+ candidate = os.path.join(temp_dir, name)
+ if os.path.isdir(candidate) and (
+ os.path.isfile(os.path.join(candidate, ".zgroup"))
+ or os.path.isfile(os.path.join(candidate, "zarr.json"))
+ ):
+ return temp_dir, candidate
+ return temp_dir, temp_dir
+
+
+def _datastore_tar_path() -> str:
+ analyzed = full_path("1K_pbmc_citeseq_analyzed.zarr.tar.gz")
+ if os.path.isfile(analyzed):
+ return analyzed
+ return full_path("1K_pbmc_citeseq.zarr.tar.gz")
+
+
+def _has_graph(datastore) -> bool:
+ try:
+ datastore._get_latest_graph_loc(from_assay="RNA", cell_key="I", feat_key="hvgs")
+ return True
+ except KeyError:
+ return False
+
+
+def _cell_has(datastore, column: str) -> bool:
+ return column in datastore.cells.columns
+
+
+@pytest.fixture(scope="session")
+def toy_crdir_writer(toy_crdir_reader, tmp_path_factory):
+ from scarf.writers import CrToZarr
+
+ out_fn = tmp_path_factory.mktemp("toy_crdir") / "toy_crdir.zarr"
+ writer = CrToZarr(toy_crdir_reader, str(out_fn))
+ writer.dump()
+ yield str(out_fn)
+
+
+@pytest.fixture(scope="session")
+def toy_crdir_ds(toy_crdir_writer):
+ from scarf.datastore.datastore import DataStore
+
+ yield DataStore(toy_crdir_writer, default_assay="RNA")
+
+
+@pytest.fixture(scope="session")
+def datastore_zarr_root():
+ temp_dir, zarr_root = _extract_zarr_fixture(
+ _datastore_tar_path(), "scarf_session_1K_pbmc_"
+ )
+ yield zarr_root
+ remove(temp_dir)
+
+
+@pytest.fixture(scope="session")
+def datastore(datastore_zarr_root):
+ from scarf.datastore.datastore import DataStore
+
+ yield DataStore(datastore_zarr_root, default_assay="RNA")
+
+
+@pytest.fixture(scope="session")
+def rna_raw_total(datastore):
+ return dask_total_sum(datastore.RNA.rawData)
+
+
+@pytest.fixture(scope="session")
+def assay2_raw_total(datastore):
+ return dask_total_sum(datastore.assay2.rawData)
+
+
+@pytest.fixture
+def datastore_ephemeral(datastore_zarr_root):
+ from scarf.datastore.datastore import DataStore
+
+ temp_dir = tempfile.mkdtemp(prefix="scarf_ephemeral_1K_pbmc_")
+ shutil.copytree(datastore_zarr_root, temp_dir, dirs_exist_ok=True)
+ yield DataStore(temp_dir, default_assay="RNA")
+ remove(temp_dir)
+
+
+@pytest.fixture(scope="session")
+def auto_filter_cells(datastore):
+ if not _has_graph(datastore):
+ datastore.auto_filter_cells(show_qc_plots=False)
+
+
+@pytest.fixture(scope="session")
+def mark_hvgs(auto_filter_cells, datastore):
+ if not _has_graph(datastore):
+ datastore.mark_hvgs(top_n=100, show_plot=False)
+
+
+@pytest.fixture(scope="session")
+def make_graph(mark_hvgs, datastore):
+ if not _has_graph(datastore):
+ datastore.make_graph(feat_key="hvgs")
+ graph_loc = datastore._get_latest_graph_loc(
+ from_assay="RNA", cell_key="I", feat_key="hvgs"
+ )
+ yield graph_loc.rsplit("/", 1)[0]
+
+
+@pytest.fixture(scope="session")
+def leiden_clustering(make_graph, datastore):
+ if not _cell_has(datastore, "RNA_leiden_cluster"):
+ datastore.run_leiden_clustering()
+ yield datastore.cells.fetch("RNA_leiden_cluster")
+
+
+@pytest.fixture(scope="session")
+def paris_clustering(make_graph, datastore):
+ if not _cell_has(datastore, "RNA_cluster"):
+ datastore.run_clustering(n_clusters=10)
+ yield datastore.cells.fetch("RNA_cluster")
+
+
+@pytest.fixture(scope="session")
+def paris_clustering_balanced(make_graph, datastore):
+ if not _cell_has(datastore, "RNA_balanced_clusters"):
+ datastore.run_clustering(
+ balanced_cut=True, max_size=100, min_size=10, label="balanced_clusters"
+ )
+ yield datastore.cells.fetch("RNA_balanced_clusters")
+
+
+@pytest.fixture(scope="session")
+def umap(make_graph, datastore):
+ if not _cell_has(datastore, "RNA_UMAP1"):
+ datastore.run_umap(n_epochs=50)
+ yield np.array(
+ [datastore.cells.fetch("RNA_UMAP1"), datastore.cells.fetch("RNA_UMAP2")]
+ ).T
+
+
+@pytest.fixture(scope="session")
+def marker_search(datastore, paris_clustering):
+ if (
+ "markers" not in datastore.z["RNA"]
+ or "I__RNA_cluster" not in datastore.z["RNA"]["markers"]
+ ):
+ datastore.run_marker_search(group_key="RNA_cluster")
+
+
+@pytest.fixture(scope="session")
+def pseudotime_scoring(datastore, leiden_clustering):
+ if not _cell_has(datastore, "RNA_pseudotime"):
+ datastore.run_pseudotime_scoring(
+ source_sink_key="RNA_leiden_cluster", sources=[6], sinks=[3]
+ )
+ yield datastore.cells.fetch("RNA_pseudotime")
+
+
+@pytest.fixture(scope="session")
+def pseudotime_markers(datastore, pseudotime_scoring):
+ if "I__RNA_pseudotime__r" not in datastore.RNA.feats.columns:
+ datastore.run_pseudotime_marker_search(pseudotime_key="RNA_pseudotime")
+ df = datastore.RNA.feats.to_pandas_dataframe(
+ ["names", "I__RNA_pseudotime__r"], key="I"
+ )
+ yield df
+
+
+@pytest.fixture(scope="session")
+def pseudotime_aggregation(datastore, pseudotime_scoring):
+ from scarf.assay import PSEUDOTIME_AGGREGATION_SCHEMA_VERSION
+
+ location = "aggregated_I_I_RNA_pseudotime"
+ needs_rebuild = location not in datastore.z["RNA"]
+ if not needs_rebuild:
+ needs_rebuild = (
+ datastore.z["RNA"][location].attrs.get("schema_version")
+ != PSEUDOTIME_AGGREGATION_SCHEMA_VERSION
+ )
+ if needs_rebuild:
+ datastore.run_pseudotime_aggregation(
+ pseudotime_key="RNA_pseudotime",
+ cluster_label="pseudotime_clusters",
+ n_clusters=15,
+ window_size=50,
+ chunk_size=10,
+ )
+
+
+@pytest.fixture(scope="session")
+def grouped_assay(datastore, pseudotime_aggregation):
+ datastore.add_grouped_assay(
+ group_key="pseudotime_clusters", assay_label="PTIME_MODULES"
+ )
+
+
+@pytest.fixture(scope="session")
+def run_mapping(make_graph, datastore):
+ projections = datastore.z["RNA"].get("projections", None)
+ if projections is None or "selfmap" not in projections:
+ datastore.run_mapping(
+ target_assay=datastore.RNA,
+ target_name="selfmap",
+ target_feat_key="hvgs_self",
+ save_k=3,
+ )
+
+
+@pytest.fixture(scope="session")
+def run_mapping_coral(make_graph, datastore):
+ projections = datastore.z["RNA"].get("projections", None)
+ if projections is None or "selfmap_coral" not in projections:
+ datastore.run_mapping(
+ target_assay=datastore.RNA,
+ target_name="selfmap_coral",
+ target_feat_key="hvgs_self2",
+ save_k=3,
+ run_coral=True,
+ )
+
+
+@pytest.fixture(scope="session")
+def run_unified_umap(run_mapping, datastore):
+ projections = datastore.z["RNA"].get("projections", None)
+ if projections is None or "unified_UMAP" not in projections:
+ datastore.run_unified_umap(target_names=["selfmap"])
+
+
+@pytest.fixture(scope="session")
+def cell_cycle_scoring(datastore):
+ if not _cell_has(datastore, "RNA_cell_cycle_phase"):
+ datastore.run_cell_cycle_scoring()
+ return datastore.cells.fetch("RNA_cell_cycle_phase")
+
+
+@pytest.fixture(scope="session")
+def topacedo_sampler(paris_clustering, datastore):
+ import importlib.util
+
+ if importlib.util.find_spec("topacedo") is None:
+ pytest.skip("topacedo package not installed")
+ if not _cell_has(datastore, "RNA_sketched"):
+ datastore.run_topacedo_sampler(cluster_key="RNA_cluster")
+ return datastore.cells.fetch("RNA_sketched")
+
+
+@pytest.fixture(scope="session")
+def cell_attrs():
+ return pd.read_csv(full_path("cell_attributes.csv"), index_col=0)
+
+
+@pytest.fixture(scope="session")
+def atac_datastore():
+ from scarf.datastore.datastore import DataStore
+
+ fn = full_path("500_pbmc_atac.zarr.tar.gz")
+ temp_dir, zarr_root = _extract_zarr_fixture(fn, "scarf_session_atac_")
+ yield DataStore(zarr_root)
+ remove(temp_dir)
+
+
+@pytest.fixture(scope="session")
+def mark_prevalent_peaks(atac_datastore):
+ atac_datastore.mark_prevalent_peaks(top_n=5000)
+
+
+@pytest.fixture(scope="session")
+def make_atac_graph(mark_prevalent_peaks, atac_datastore):
+ atac_datastore.make_graph(feat_key="prevalent_peaks")
+ graph_loc = atac_datastore._get_latest_graph_loc(
+ from_assay="ATAC", cell_key="I", feat_key="prevalent_peaks"
+ )
+ yield graph_loc.rsplit("/", 1)[0]
diff --git a/tests/fixtures_downloader.py b/tests/fixtures_downloader.py
new file mode 100644
index 00000000..b6a152e9
--- /dev/null
+++ b/tests/fixtures_downloader.py
@@ -0,0 +1,17 @@
+import os
+
+import pytest
+
+from . import full_path
+
+
+@pytest.fixture(scope="session")
+def bastidas_ponce_data():
+ sample = "bastidas-ponce_4K_pancreas-d15_rnaseq"
+ local_h5ad = full_path(sample, "data.h5ad")
+ if not os.path.isfile(local_h5ad):
+ pytest.skip(
+ f"Bundled test data missing at {local_h5ad}. "
+ "Add data.h5ad under tests/datasets/ to run h5ad reader tests."
+ )
+ yield local_h5ad
diff --git a/tests/fixtures_readers.py b/tests/fixtures_readers.py
new file mode 100644
index 00000000..fd16f412
--- /dev/null
+++ b/tests/fixtures_readers.py
@@ -0,0 +1,80 @@
+import os
+import tarfile
+
+import pytest
+
+from . import full_path
+
+
+@pytest.fixture(scope="session")
+def toy_crdir_reader(tmp_path_factory):
+ from scarf.readers import CrDirReader
+
+ out_fn = tmp_path_factory.mktemp("toy_cr_dir")
+ with tarfile.open(full_path("toy_cr_dir.tar.gz"), "r:gz") as tar:
+ tar.extractall(out_fn, filter="data")
+ reader = CrDirReader(str(out_fn))
+ reader.rename_assays({"ASSAY4": "HTO"})
+ yield reader
+
+
+@pytest.fixture(scope="session")
+def toy_crdir_empty(tmp_path_factory):
+ from scarf.readers import CrDirReader
+
+ out_fn = tmp_path_factory.mktemp("toy_cr_dir_empty")
+ with tarfile.open(full_path("toy_cr_dir_empty.tar.gz"), "r:gz") as tar:
+ tar.extractall(out_fn, filter="data")
+ reader = CrDirReader(str(out_fn))
+ yield reader
+
+
+@pytest.fixture(scope="session")
+def crh5_reader():
+ from scarf.readers import CrH5Reader
+
+ return CrH5Reader(full_path("1K_pbmc_citeseq.h5"))
+
+
+@pytest.fixture(scope="session")
+def mtx_dir(tmp_path_factory):
+ fn = full_path("1K_pbmc_citeseq_dir.tar.gz")
+ if not os.path.isfile(fn):
+ pytest.skip(
+ f"Bundled MTX fixture missing at {fn}. "
+ "Add 1K_pbmc_citeseq_dir.tar.gz under tests/datasets/."
+ )
+ base = tmp_path_factory.mktemp("mtx_dir")
+ with tarfile.open(fn, "r:gz") as tar:
+ tar.extractall(base, filter="data")
+ yield str(base / "1K_pbmc_citeseq_dir")
+
+
+@pytest.fixture(scope="session")
+def crdir_reader(mtx_dir):
+ from scarf.readers import CrDirReader
+
+ reader = CrDirReader(mtx_dir)
+ yield reader
+
+
+@pytest.fixture(scope="session")
+def h5ad_reader(bastidas_ponce_data):
+ from scarf.readers import H5adReader
+
+ reader = H5adReader(bastidas_ponce_data)
+ yield reader
+ reader.h5.close()
+
+
+@pytest.fixture(scope="session")
+def loom_reader():
+ from scarf.readers import LoomReader
+
+ reader = LoomReader(
+ full_path("sympathetic.loom"),
+ cell_names_key="Cell_id",
+ feature_names_key="Gene",
+ )
+ yield reader
+ reader.h5.close()
diff --git a/tests/symphony_r_0_1_3_golden.json b/tests/symphony_r_0_1_3_golden.json
new file mode 100644
index 00000000..5203f212
--- /dev/null
+++ b/tests/symphony_r_0_1_3_golden.json
@@ -0,0 +1,80 @@
+{
+ "provenance": {
+ "package": "symphony",
+ "packageVersion": "0.1.3",
+ "sourceRepository": "https://github.com/immunogenomics/symphony",
+ "sourceCommit": "7c5905988734d9cfe6e1e97a658664717c4ba7b7",
+ "function": "mapQuery",
+ "doNormalize": false,
+ "doUmap": false,
+ "sigma": 0.1
+ },
+ "reference": {
+ "featureMeans": [0.0, 0.0],
+ "featureScales": [1.0, 1.0],
+ "loadings": [[1.0, 0.0], [0.0, 1.0]],
+ "centroids": [[-1.0, 0.0], [1.0, 0.0]],
+ "rawCentroids": [[-1.0, 0.0], [1.0, 0.0]],
+ "correctedCentroids": [[-1.0, 0.0], [1.0, 0.0]],
+ "clusterMass": [4.0, 4.0],
+ "sigma": [0.1, 0.1],
+ "correctionRidge": 1.0
+ },
+ "query": [
+ [-1.5, 0.2],
+ [-0.5, -0.2],
+ [0.5, 0.2],
+ [1.5, -0.2]
+ ],
+ "batchCodes": [0, 0, 0, 0],
+ "expectedProjected": [
+ [-1.5, 0.2],
+ [-0.5, -0.2],
+ [0.5, 0.2],
+ [1.5, -0.2]
+ ],
+ "expectedAssignments": [
+ [1.0, 6.0340367874094217e-18],
+ [1.0, 7.4251990982146727e-17],
+ [7.4251990982146727e-17, 1.0],
+ [6.0340367874094217e-18, 1.0]
+ ],
+ "expectedCorrected": [
+ [-1.5, 0.2],
+ [-0.5, -0.2],
+ [0.5, 0.2],
+ [1.5, -0.2]
+ ],
+ "nonzeroCorrection": {
+ "reference": {
+ "featureMeans": [0.0, 0.0],
+ "featureScales": [1.0, 1.0],
+ "loadings": [[1.0, 0.0], [0.0, 1.0]],
+ "centroids": [[1.0, 0.0]],
+ "rawCentroids": [[0.0, 0.0]],
+ "correctedCentroids": [[0.0, 0.0]],
+ "clusterMass": [4.0],
+ "sigma": [0.1],
+ "correctionRidge": 2.0
+ },
+ "query": [
+ [2.0, -2.0],
+ [4.0, -2.0],
+ [1.0, -1.0],
+ [5.0, -3.0]
+ ],
+ "batchCodes": [0, 0, 0, 0],
+ "expectedAssignments": [
+ [1.0],
+ [1.0],
+ [1.0],
+ [1.0]
+ ],
+ "expectedCorrected": [
+ [0.0, -0.6666666666666667],
+ [2.0, -0.6666666666666667],
+ [-1.0, 0.33333333333333326],
+ [3.0, -1.6666666666666667]
+ ]
+ }
+}
diff --git a/tests/test_ann.py b/tests/test_ann.py
new file mode 100644
index 00000000..7a11c9b1
--- /dev/null
+++ b/tests/test_ann.py
@@ -0,0 +1,67 @@
+import numpy as np
+
+from scarf.ann import fix_knn_query
+
+
+def test_fix_knn_query_removes_leading_self_matches():
+ indices = np.array([[4, 1, 2], [7, 5, 6]])
+ distances = np.array([[0.0, 0.2, 0.4], [0.0, 0.3, 0.6]])
+
+ fixed_indices, fixed_distances, missing_first = fix_knn_query(
+ indices,
+ distances,
+ ref_idx=np.array([4, 7]),
+ )
+
+ assert missing_first == 0
+ assert np.array_equal(fixed_indices, [[1, 2], [5, 6]])
+ assert np.allclose(fixed_distances, [[0.2, 0.4], [0.3, 0.6]])
+
+
+def test_fix_knn_query_handles_each_self_neighbor_case():
+ indices = np.array(
+ [
+ [0, 1, 2, 3],
+ [2, 3, 1, 4],
+ [0, 2, 3, 4],
+ ]
+ )
+ distances = np.array(
+ [
+ [0.0, 0.1, 0.2, 0.3],
+ [0.1, 0.2, 0.3, 0.4],
+ [0.1, 0.2, 0.3, 0.4],
+ ]
+ )
+ original_indices = indices.copy()
+ original_distances = distances.copy()
+
+ fixed_indices, fixed_distances, missing_first = fix_knn_query(
+ indices,
+ distances,
+ ref_idx=np.array([0, 1, 5]),
+ )
+
+ assert missing_first == 2
+ assert np.array_equal(
+ fixed_indices,
+ np.array(
+ [
+ [1, 2, 3],
+ [2, 3, 4],
+ [0, 2, 3],
+ ]
+ ),
+ )
+ assert np.allclose(
+ fixed_distances,
+ np.array(
+ [
+ [0.1, 0.2, 0.3],
+ [0.1, 0.2, 0.4],
+ [0.1, 0.2, 0.3],
+ ]
+ ),
+ )
+ assert np.array_equal(indices, original_indices)
+ assert np.array_equal(distances, original_distances)
diff --git a/tests/test_ann_mapping.py b/tests/test_ann_mapping.py
new file mode 100644
index 00000000..5117d9bd
--- /dev/null
+++ b/tests/test_ann_mapping.py
@@ -0,0 +1,54 @@
+import numpy as np
+import pytest
+
+from scarf.ann import AnnStream
+from scarf.chunked import ChunkedArray
+
+
+def _ann_stream() -> AnnStream:
+ data = ChunkedArray.from_numpy(
+ np.array([[0.0, 0.0], [1.0, 1.0], [2.0, 2.0]], dtype=np.float64),
+ block_size=2,
+ )
+ return AnnStream(
+ data=data,
+ k=1,
+ n_cluster=2,
+ reduction_method="pca",
+ dims=2,
+ loadings=np.eye(2),
+ use_for_pca=np.ones(3, dtype=bool),
+ mu=np.array([0.5, 0.5]),
+ sigma=np.array([0.5, 0.5]),
+ ann_metric="l2",
+ ann_efc=10,
+ ann_ef=10,
+ ann_m=8,
+ nthreads=1,
+ ann_parallel=False,
+ rand_state=0,
+ do_kmeans_fit=False,
+ disable_scaling=False,
+ ann_idx=None,
+ lsi_skip_first=True,
+ lsi_params={},
+ harmonize=False,
+ )
+
+
+def test_transform_query_matches_existing_reference_reducer():
+ ann = _ann_stream()
+ query = np.array([[1.5, 0.0], [0.0, 1.5]], dtype=np.float64)
+ reference_embeddings = ann.embeddings.copy()
+
+ np.testing.assert_array_equal(ann.transform_query(query), ann.reducer(query))
+ np.testing.assert_array_equal(ann.embeddings, reference_embeddings)
+
+
+def test_transform_query_validates_feature_shape_and_finite_results():
+ ann = _ann_stream()
+
+ with pytest.raises(ValueError, match="features"):
+ ann.transform_query(np.ones((2, 3)))
+ with pytest.raises(ValueError, match="non-finite"):
+ ann.transform_query(np.array([[np.nan, 0.0]]))
diff --git a/tests/test_bio_data.py b/tests/test_bio_data.py
new file mode 100644
index 00000000..87cc39e1
--- /dev/null
+++ b/tests/test_bio_data.py
@@ -0,0 +1,9 @@
+from scarf.bio_data import g2m_phase_genes, s_phase_genes
+
+
+def test_cell_cycle_gene_lists_are_nonempty_and_unique():
+ assert len(s_phase_genes) > 20
+ assert len(g2m_phase_genes) > 20
+ assert len(s_phase_genes) == len(set(s_phase_genes))
+ assert len(g2m_phase_genes) == len(set(g2m_phase_genes))
+ assert set(s_phase_genes).isdisjoint(g2m_phase_genes)
diff --git a/tests/test_budget.py b/tests/test_budget.py
new file mode 100644
index 00000000..667eb958
--- /dev/null
+++ b/tests/test_budget.py
@@ -0,0 +1,146 @@
+import builtins
+
+import pytest
+
+from scarf.storage.budget import (
+ READ_AHEAD,
+ ResourceBudget,
+ detect_total_memory_bytes,
+ resolve_budget,
+ worker_prefetch_depth,
+)
+
+
+def test_resolve_budget_parses_suffix(monkeypatch):
+ monkeypatch.setenv("SCARF_MEM_BUDGET", "8G")
+ monkeypatch.setenv("SCARF_WORKERS", "3")
+ budget = resolve_budget()
+ assert budget.memoryBytes == 8 * 1024**3
+ assert budget.workers == 3
+
+
+def test_resolve_budget_raw_bytes_and_explicit_workers(monkeypatch):
+ monkeypatch.delenv("SCARF_MEM_BUDGET", raising=False)
+ budget = resolve_budget(memory=12345678, workers=2)
+ assert budget.memoryBytes == 12345678
+ assert budget.workers == 2
+
+
+def test_resolve_budget_fraction_of_total(monkeypatch):
+ monkeypatch.delenv("SCARF_MEM_BUDGET", raising=False)
+ total = detect_total_memory_bytes()
+ budget = resolve_budget(memory="0.5", workers=1)
+ assert abs(budget.memoryBytes - int(total * 0.5)) <= total * 0.01
+
+
+def test_resolve_budget_default_is_total_memory(monkeypatch):
+ monkeypatch.delenv("SCARF_MEM_BUDGET", raising=False)
+ monkeypatch.delenv("SCARF_WORKERS", raising=False)
+ budget = resolve_budget()
+ assert budget.memoryBytes == detect_total_memory_bytes()
+ assert budget.workers >= 1
+
+
+def test_detect_memory_fallback_when_meminfo_absent(monkeypatch):
+ real_open = builtins.open
+
+ def fake_open(path, *args, **kwargs):
+ if str(path) == "/proc/meminfo":
+ raise OSError("no meminfo")
+ return real_open(path, *args, **kwargs)
+
+ monkeypatch.setattr(builtins, "open", fake_open)
+ monkeypatch.setattr("os.sysconf", lambda name: -1)
+ assert detect_total_memory_bytes() == 8 * 1024 * 1024 * 1024
+
+
+@pytest.mark.parametrize(
+ "spec, expected",
+ [
+ (8 * 1024**3, 8 * 1024**3),
+ ("8G", 8 * 1024**3),
+ ("512M", 512 * 1024**2),
+ (12_345_678, 12_345_678),
+ ],
+)
+def test_memory_spec_valid(spec, expected, monkeypatch):
+ monkeypatch.delenv("SCARF_MEM_BUDGET", raising=False)
+ assert resolve_budget(memory=spec, workers=1).memoryBytes == expected
+
+
+@pytest.mark.parametrize("spec", ["1.0", "100", "abc", "-5", "0", "8Q", ""])
+def test_memory_spec_invalid_or_ambiguous_rejected(spec, monkeypatch):
+ monkeypatch.delenv("SCARF_MEM_BUDGET", raising=False)
+ with pytest.raises(ValueError):
+ resolve_budget(memory=spec, workers=1)
+
+
+def test_memory_spec_bool_rejected(monkeypatch):
+ monkeypatch.delenv("SCARF_MEM_BUDGET", raising=False)
+ with pytest.raises(ValueError, match="Invalid memory spec"):
+ resolve_budget(memory=True, workers=1)
+
+
+def test_detect_memory_uses_sysconf_when_meminfo_unavailable(monkeypatch):
+ real_open = builtins.open
+
+ def fake_open(path, *args, **kwargs):
+ if str(path) == "/proc/meminfo":
+ raise OSError("no meminfo")
+ return real_open(path, *args, **kwargs)
+
+ def fake_sysconf(name):
+ if name == "SC_PAGE_SIZE":
+ return 4096
+ if name == "SC_PHYS_PAGES":
+ return 1024
+ return -1
+
+ monkeypatch.setattr(builtins, "open", fake_open)
+ monkeypatch.setattr("os.sysconf", fake_sysconf)
+ assert detect_total_memory_bytes() == 4096 * 1024
+
+
+def test_invalid_workers_env_rejected(monkeypatch):
+ monkeypatch.delenv("SCARF_MEM_BUDGET", raising=False)
+ monkeypatch.setenv("SCARF_WORKERS", "not-a-number")
+ with pytest.raises(ValueError):
+ resolve_budget(memory="8G")
+
+
+def test_fraction_uses_total_memory(monkeypatch):
+ monkeypatch.delenv("SCARF_MEM_BUDGET", raising=False)
+ total = detect_total_memory_bytes()
+ got = resolve_budget(memory="0.25", workers=1).memoryBytes
+ assert abs(got - int(total * 0.25)) <= total * 0.01
+
+
+def test_worker_prefetch_depth_capped_by_read_ahead():
+ budget = ResourceBudget(memoryBytes=4 * 1024**3, workers=4, workingCopies=8)
+ assert worker_prefetch_depth(budget=budget) == READ_AHEAD
+ assert worker_prefetch_depth(requested=1, budget=budget) == 1
+ assert worker_prefetch_depth(requested=10, budget=budget) == READ_AHEAD
+ assert worker_prefetch_depth(requested=0, budget=budget) == 1
+
+
+def test_worker_prefetch_depth_capped_by_working_copies():
+ budget = ResourceBudget(memoryBytes=4 * 1024**3, workers=4, workingCopies=1)
+ assert worker_prefetch_depth(budget=budget) == 1
+
+
+def test_working_copies_from_env(monkeypatch):
+ monkeypatch.delenv("SCARF_MEM_BUDGET", raising=False)
+ monkeypatch.setenv("SCARF_WORKING_COPIES", "8")
+ budget = resolve_budget(memory="4G", workers=1)
+ assert budget.workingCopies == 8
+
+
+def test_layout_geometry_independent_of_workers():
+ from scarf.storage.zarr_store import matrix_layout
+
+ one = ResourceBudget(memoryBytes=8 * 1024**3, workers=1, workingCopies=4)
+ eight = ResourceBudget(memoryBytes=8 * 1024**3, workers=8, workingCopies=4)
+ c1, s1 = matrix_layout(1_000_000, 50_000, budget=one, itemsize=4)
+ c8, s8 = matrix_layout(1_000_000, 50_000, budget=eight, itemsize=4)
+ assert c1 == c8
+ assert s1 == s8
diff --git a/tests/test_chunked.py b/tests/test_chunked.py
new file mode 100644
index 00000000..fe186693
--- /dev/null
+++ b/tests/test_chunked.py
@@ -0,0 +1,156 @@
+"""Tests for the ChunkedArray abstraction and the public rawData/normed API.
+
+These verify that ChunkedArray reproduces NumPy semantics for the operations
+Scarf relies on, and that the documented datastore access patterns from the
+vignettes keep working after Dask was removed.
+"""
+
+import numpy as np
+import pytest
+import zarr
+
+from scarf.chunked import ChunkedArray
+
+
+@pytest.fixture
+def backed_pair(tmp_path):
+ rng = np.random.default_rng(0)
+ n, m = 230, 70
+ dense = rng.integers(0, 6, size=(n, m)).astype(np.uint32)
+ root = zarr.open_group(str(tmp_path / "ca.zarr"), mode="w")
+ z = root.create_array("counts", shape=(n, m), chunks=(64, 32), dtype="uint32")
+ z[:, :] = dense
+ return ChunkedArray(root["counts"], nthreads=4), dense
+
+
+class TestChunkedArrayParity:
+ def test_shape_and_blocks(self, backed_pair):
+ ca, dense = backed_pair
+ from scarf.storage.zarr_store import array_shard_rows
+
+ assert ca.shape == dense.shape
+ stream_rows = array_shard_rows(ca._backing)
+ assert ca.numblocks[0] == int(np.ceil(dense.shape[0] / stream_rows))
+ assert sum(b.shape[0] for b in ca.blocks) == dense.shape[0]
+ recon = np.vstack([b.compute() for b in ca.blocks])
+ assert np.array_equal(recon, dense)
+
+ def test_compute(self, backed_pair):
+ ca, dense = backed_pair
+ assert np.array_equal(ca.compute(), dense)
+
+ @pytest.mark.parametrize("axis", [0, 1])
+ def test_reductions(self, backed_pair, axis):
+ ca, dense = backed_pair
+ assert np.allclose(ca.sum(axis=axis).compute(), dense.sum(axis))
+ assert np.allclose(ca.mean(axis=axis).compute(), dense.mean(axis))
+ assert np.allclose(ca.var(axis=axis).compute(), dense.var(axis))
+ assert np.allclose(ca.std(axis=axis).compute(), dense.std(axis))
+
+ def test_count_nonzero_and_argmax(self, backed_pair):
+ ca, dense = backed_pair
+ assert np.array_equal(
+ np.asarray(ca.count_nonzero(axis=1)), np.count_nonzero(dense, axis=1)
+ )
+ assert np.array_equal(np.asarray(ca.argmax(axis=1)), dense.argmax(1))
+
+ def test_boolean_comparison_reduction(self, backed_pair):
+ ca, dense = backed_pair
+ assert np.array_equal((ca > 0).sum(axis=0).compute(), (dense > 0).sum(0))
+
+ def test_fancy_subset(self, backed_pair):
+ ca, dense = backed_pair
+ rng = np.random.default_rng(1)
+ fidx = np.sort(rng.choice(dense.shape[1], 25, replace=False))
+ cidx = np.sort(rng.choice(dense.shape[0], 90, replace=False))
+ sub = ca[:, fidx][cidx, :]
+ ref = dense[:, fidx][cidx, :]
+ assert sub.shape == ref.shape
+ assert np.array_equal(sub.compute(), ref)
+ assert np.allclose(sub.sum(axis=1).compute(), ref.sum(1))
+
+ def test_lib_size_normalization(self, backed_pair):
+ ca, dense = backed_pair
+ sub = ca[:, np.arange(dense.shape[1])]
+ scalar = dense.sum(1).astype(float)
+ scalar[scalar == 0] = 1
+ normed = 1e4 * sub / scalar.reshape(-1, 1)
+ ref = 1e4 * dense / scalar.reshape(-1, 1)
+ assert np.allclose(normed.compute(), ref)
+ assert np.allclose(np.log1p(normed).compute(), np.log1p(ref))
+
+ def test_clr_with_axis0_inside_expression(self, backed_pair):
+ ca, dense = backed_pair
+ rng = np.random.default_rng(2)
+ fidx = np.sort(rng.choice(dense.shape[1], 20, replace=False))
+ cidx = np.sort(rng.choice(dense.shape[0], 80, replace=False))
+ sub = ca[:, fidx][cidx, :]
+ ref = dense[:, fidx][cidx, :]
+ f = np.exp(np.log1p(sub).sum(axis=0) / len(sub))
+ clr = np.log1p(sub / f)
+ ref_f = np.exp(np.log1p(ref).sum(0) / ref.shape[0])
+ assert np.allclose(clr.compute(), np.log1p(ref / ref_f))
+
+ def test_column_subset_after_ops(self, backed_pair):
+ ca, dense = backed_pair
+ scalar = dense.sum(1).astype(float)
+ scalar[scalar == 0] = 1
+ normed = np.log1p(1e4 * ca / scalar.reshape(-1, 1))
+ ref = np.log1p(1e4 * dense / scalar.reshape(-1, 1))
+ cols = np.array([3, 10, 25, 40])
+ assert np.allclose(normed[:, cols].compute(), ref[:, cols])
+
+ def test_dot(self, backed_pair):
+ ca, dense = backed_pair
+ rng = np.random.default_rng(3)
+ loadings = rng.standard_normal((dense.shape[1], 4))
+ assert np.allclose(ca.dot(loadings).compute(), dense @ loadings)
+
+ def test_from_numpy(self, backed_pair):
+ _, dense = backed_pair
+ ca = ChunkedArray.from_numpy(dense.astype(float), block_size=50, nthreads=2)
+ assert np.array_equal(ca.compute(), dense.astype(float))
+ assert np.allclose(ca.sum(axis=0).compute(), dense.sum(0))
+
+
+class TestPublicApiCompat:
+ """Mirror the rawData/normed usage documented in the vignettes."""
+
+ def test_rawdata_is_chunked(self, datastore):
+ raw = datastore.RNA.rawData
+ assert isinstance(raw, ChunkedArray)
+ assert len(raw.chunksize) == 2
+ assert raw.shape[0] == datastore.RNA.cells.N
+
+ def test_rawdata_mean_axis0_compute_reshape(self, datastore):
+ # Pattern from the MNIST vignette.
+ fidx = np.arange(20)
+ cidx = datastore.RNA.cells.active_index("I")[:50]
+ out = (
+ datastore.RNA.rawData[:, fidx][cidx, :]
+ .mean(axis=0)
+ .compute()
+ .reshape(1, -1)
+ )
+ assert out.shape == (1, 20)
+
+ def test_normed_mean_axis1_compute(self, datastore):
+ # Pattern from the pseudotime dynamics vignette.
+ vals = datastore.RNA.normed().mean(axis=1).compute()
+ assert vals.shape[0] == datastore.RNA.cells.active_index("I").shape[0]
+ assert np.all(np.isfinite(vals))
+
+ def test_custom_normmethod_numpy_semantics(self, datastore):
+ # User-overridable normMethod must accept NumPy-like array semantics.
+ def custom(_, counts):
+ lib = counts.sum(axis=1).reshape(-1, 1)
+ return np.log2(counts / lib * 1000 + 1)
+
+ assay = datastore.RNA
+ original = assay.normMethod
+ try:
+ assay.normMethod = custom
+ out = assay.normed().compute()
+ assert np.all(np.isfinite(out))
+ finally:
+ assay.normMethod = original
diff --git a/tests/test_datastore.py b/tests/test_datastore.py
new file mode 100644
index 00000000..bc2c72ac
--- /dev/null
+++ b/tests/test_datastore.py
@@ -0,0 +1,502 @@
+import numpy as np
+import pandas as pd
+
+from scarf.mapping_utils import array_hash
+
+from . import full_path
+
+
+class TestToyDataStore:
+ def test_toy_crdir_metadata(self, toy_crdir_ds):
+ assert np.all(
+ toy_crdir_ds.RNA.feats.fetch_all("ids") == ["g1", "g2", "g3", "g4"]
+ )
+ assert np.all(toy_crdir_ds.ADT.feats.fetch_all("ids") == ["a1", "a2"])
+ assert np.all(toy_crdir_ds.HTO.feats.fetch_all("ids") == ["h1"])
+ assert np.all(toy_crdir_ds.cells.fetch_all("ids") == ["b1", "b2", "b3"])
+
+ def test_toy_crdir_rawdata(self, toy_crdir_ds):
+ assert np.all(
+ toy_crdir_ds.RNA.rawData.compute()
+ == [[5, 0, 0, 2], [3, 3, 0, 7], [3, 3, 0, 7]]
+ )
+ assert np.all(
+ toy_crdir_ds.ADT.rawData.compute() == [[30, 40], [30, 50], [0, 50]]
+ )
+ assert np.all(toy_crdir_ds.HTO.rawData.compute() == [[200], [100], [100]])
+
+
+class TestDataStore:
+ def test_init_wrong_zarr_mode(self, tmp_path):
+ import pytest
+ import tarfile
+
+ from scarf.datastore.datastore import DataStore
+
+ fn = full_path("1K_pbmc_citeseq.zarr.tar.gz")
+ out_fn = tmp_path / "1K_pbmc_citeseq.zarr"
+ with tarfile.open(fn, "r:gz") as tar:
+ tar.extractall(out_fn, filter="data")
+ with pytest.raises(ValueError):
+ DataStore(str(out_fn), zarr_mode="wrong", default_assay="RNA")
+
+ def test_auto_filter_cells(self, datastore_ephemeral):
+ assert (
+ datastore_ephemeral.auto_filter_cells(
+ attrs=["nCounts", "nFeatures", "non_existing_column"],
+ show_qc_plots=False,
+ )
+ is None
+ )
+ # show_qc_plots=True
+ # howto test plots?
+
+ def test_filter_cells(self, datastore_ephemeral):
+ assert (
+ datastore_ephemeral.filter_cells(
+ attrs=["nCounts", "nFeatures", "non_existing_column"],
+ lows=[None, None, None],
+ highs=[None, None, None],
+ reset_previous=True,
+ )
+ is None
+ )
+ # still doesn't access `if j is None:` cases for j and k
+
+ def test_graph_indices(self, make_graph, datastore):
+ a = np.load(full_path("knn_indices.npy"))
+ b = datastore.z[make_graph]["indices"][:]
+ assert np.array_equal(a, b)
+
+ def test_graph_distances(self, make_graph, datastore):
+ a = np.load(full_path("knn_distances.npy"))
+ b = datastore.z[make_graph]["distances"][:]
+ assert np.all((a - b) < 1e-3)
+
+ def test_graph_weights(self, make_graph, datastore):
+ a = np.load(full_path("knn_weights.npy"))
+ b = datastore.z[make_graph]["graph__1.0__1.5"]["weights"][:]
+ assert np.all((a - b) < 1e-5)
+
+ def test_atac_graph_indices(self, make_atac_graph, atac_datastore):
+ a = np.load(full_path("atac_knn_indices.npy"))
+ b = atac_datastore.z[make_atac_graph]["indices"][:]
+ assert a.shape == b.shape
+
+ # TODO: activate this when this PR is merged and released in gensim
+ # https://github.com/RaRe-Technologies/gensim/pull/3194
+ # assert np.array_equal(a, b)
+
+ def test_atac_graph_distances(self, make_atac_graph, atac_datastore):
+ a = np.load(full_path("atac_knn_distances.npy"))
+ b = atac_datastore.z[make_atac_graph]["distances"][:]
+ assert a.shape == b.shape
+
+ # TODO: activate this when this PR is merged and released in gensim
+ # https://github.com/RaRe-Technologies/gensim/pull/3194
+ # assert np.all((a - b) < 1e-5)
+
+ def test_leiden_values(self, leiden_clustering, cell_attrs):
+ assert len(set(leiden_clustering)) == 10
+ # Disabled the following test because failing on CI
+ # assert np.array_equal(leiden_clustering, cell_attrs['RNA_leiden_cluster'].values)
+
+ def test_paris_values(self, paris_clustering, cell_attrs):
+ assert np.array_equal(paris_clustering, cell_attrs["RNA_cluster"].values)
+
+ def test_paris_balanced_values(self, paris_clustering_balanced, cell_attrs):
+ assert np.array_equal(
+ paris_clustering_balanced, cell_attrs["RNA_balanced_clusters"].values
+ )
+
+ def test_run_cell_cycle_scoring(self, cell_cycle_scoring, cell_attrs):
+ assert np.array_equal(
+ cell_cycle_scoring, cell_attrs["RNA_cell_cycle_phase"].values
+ )
+
+ def test_umap_values(self, umap, cell_attrs):
+ precalc_umap = cell_attrs[["RNA_UMAP1", "RNA_UMAP2"]].values
+ assert umap.shape == precalc_umap.shape
+ # Disabled the following test because failing on CI
+ # assert np.all((umap - precalc_umap) < 0.1)
+
+ def test_get_markers(self, marker_search, paris_clustering, datastore):
+ precalc_markers = pd.read_csv(full_path("markers_cluster1.csv"), index_col=0)
+ markers = datastore.get_markers(group_key="RNA_cluster", group_id=1)
+
+ # Check feature names and scores (always required)
+ assert markers.feature_name.equals(precalc_markers.feature_name)
+ diff = (markers.score - precalc_markers.score).values
+ assert np.all(diff < 1e-3)
+
+ # Check p_values only if they exist in reference data (backward compatible)
+ if "p_value" in precalc_markers.columns:
+ assert "p_value" in markers.columns, "p_value column missing in output"
+ # P-values should match within reasonable tolerance
+ p_diff = (markers.p_value - precalc_markers.p_value).values
+ assert np.all(np.abs(p_diff) < 1e-3), "p_values differ from reference"
+
+ def test_export_markers_to_csv(
+ self, marker_search, paris_clustering, datastore, tmp_path
+ ):
+ precalc_markers = pd.read_csv(full_path("markers_all_clusters.csv"))
+ out_file = str(tmp_path / "test_values_markers.csv")
+ datastore.export_markers_to_csv(group_key="RNA_cluster", csv_filename=out_file)
+ markers = pd.read_csv(out_file)
+ assert markers.equals(precalc_markers)
+
+ def test_run_unified_umap(self, run_unified_umap, datastore):
+ coords = datastore.z["RNA"]["projections"]["unified_UMAP"][:]
+ precalc_coords = np.load(full_path("unified_UMAP_coords.npy"))
+ assert coords.shape == precalc_coords.shape
+
+ def test_get_target_classes(self, run_mapping, paris_clustering, datastore):
+ classes = datastore.get_target_classes(
+ target_name="selfmap", reference_class_group="RNA_cluster"
+ )
+ assert len(classes) == len(datastore.cells.active_index("I"))
+ assert classes.notna().all()
+ assert (
+ array_hash(classes.astype(str).to_numpy())
+ == "595897c7c4b619dd367674bf66ce826e9ce2e731d59e5f05f1401bf5cc489e99"
+ )
+
+ def test_get_mapping_score(self, run_mapping, datastore):
+ scores = next(datastore.get_mapping_score(target_name="selfmap"))[1]
+ assert scores.shape == (len(datastore.cells.active_index("I")),)
+ assert np.all(np.isfinite(scores))
+ assert np.any(scores > 0)
+ assert (
+ array_hash(np.round(scores, 12))
+ == "f992bd7e48c7eda529c30194dab350fd5d6fea056f76146a2705993223d97d0e"
+ )
+
+ def test_coral_mapping_score(self, run_mapping_coral, datastore):
+ scores = next(datastore.get_mapping_score(target_name="selfmap_coral"))[1]
+ assert np.all(np.isfinite(scores))
+ assert np.any(scores > 0)
+
+ def test_repr(self, datastore):
+ # TODO: Test if the expected values are printed
+ print(datastore)
+
+ def test_get_imputed(self, datastore):
+ # TODO: Test the output values
+ values = datastore.get_imputed(feature_name="CD4")
+ assert values.shape == datastore.cells.fetch("I").shape
+
+ def test_run_doublet_detection(self, make_graph, paris_clustering, datastore):
+ datastore.run_doublet_detection(
+ cluster_key="RNA_cluster", simulation_ratio=0.5, random_seed=1
+ )
+ col = "RNA_doublet_score"
+ assert col in datastore.cells.columns
+ scores = datastore.cells.fetch(col)
+ n_active = datastore.cells.active_index("I").shape[0]
+ assert scores.shape[0] == n_active
+ assert not np.isnan(scores).any()
+ assert scores.min() >= 0.0 and scores.max() <= 1.0
+ # scratch column used for smoothing should be removed
+ assert "RNA_doublet_score__raw" not in datastore.cells.columns
+ # temporary simulated-doublet projection should be cleaned up
+ projections = datastore.z["RNA"].get("projections", None)
+ if projections is not None:
+ assert "_doublet_sim_RNA" not in projections
+
+ def test_run_doublet_detection_bad_cluster_key(self, make_graph, datastore):
+ import pytest
+
+ with pytest.raises(ValueError):
+ datastore.run_doublet_detection(cluster_key="not_a_column")
+
+ def test_run_pseudotime_scoring(self, pseudotime_scoring, cell_attrs):
+ diff = pseudotime_scoring - cell_attrs["RNA_pseudotime"].values
+ assert np.all(np.abs(diff) < 0.08)
+
+ def test_run_pseudotime_scoring_current_contract(
+ self,
+ leiden_clustering,
+ datastore_ephemeral,
+ ):
+ datastore_ephemeral.run_pseudotime_scoring(
+ source_sink_key="RNA_leiden_cluster",
+ sources=[6],
+ sinks=[3],
+ n_singular_vals=10,
+ label="reliability_test",
+ )
+
+ output_key = "RNA_reliability_test"
+ validity_key = f"{output_key}__valid"
+ assert validity_key in datastore_ephemeral.cells.columns
+ values = datastore_ephemeral.cells.fetch(output_key, key=validity_key)
+ assert np.isfinite(values).all()
+ assert values.min() >= 0.0
+ assert values.max() <= 1.0
+
+ def test_run_pseudotime_marker_search(self, pseudotime_markers):
+ precalc_markers = pd.read_csv(
+ full_path("pseudotime_markers_r_values.csv"), index_col=0
+ )
+ assert np.all(precalc_markers.index == pseudotime_markers.index)
+ assert np.all(precalc_markers.names.values == pseudotime_markers.names.values)
+ assert np.allclose(
+ precalc_markers.I__RNA_pseudotime__r.values,
+ pseudotime_markers.I__RNA_pseudotime__r.values,
+ rtol=0.15,
+ atol=0.15,
+ )
+
+ def test_run_pseudotime_aggregation(self, pseudotime_aggregation, datastore):
+ from scarf.assay import PSEUDOTIME_AGGREGATION_SCHEMA_VERSION
+
+ precalc_values = np.load(full_path("aggregated_feat_idx.npy"))
+ agg_group = datastore.z["RNA"]["aggregated_I_I_RNA_pseudotime"]
+ test_values = agg_group["feature_indices"][:]
+ assert np.array_equal(
+ precalc_values.astype(np.int64), test_values.astype(np.int64)
+ )
+
+ assert (
+ agg_group.attrs["schema_version"] == PSEUDOTIME_AGGREGATION_SCHEMA_VERSION
+ )
+ assert np.isfinite(agg_group["data"][:]).all()
+ valid_features = np.asarray(agg_group["valid_features"][:], dtype=bool)
+ clusters = datastore.RNA.feats.fetch_all("pseudotime_clusters")
+ assigned = clusters[test_values[valid_features].astype(int)]
+ assert assigned.min() >= 1
+ assert assigned.max() <= 15
+ assert len(np.unique(assigned)) == 15
+ assert np.all(clusters[test_values[~valid_features].astype(int)] == -1)
+
+ def test_add_grouped_assay(self, grouped_assay, datastore):
+ test_values = datastore.get_cell_vals(
+ from_assay="PTIME_MODULES", cell_key="I", k="group_1"
+ )
+ groups = datastore.RNA.feats.fetch_all("pseudotime_clusters")
+ feature_index = np.where(groups == 1)[0]
+ expected = (
+ datastore.RNA.normed(
+ cell_idx=datastore.cells.active_index("I"),
+ feat_idx=feature_index,
+ )
+ .mean(axis=1)
+ .compute()
+ )
+ assert np.allclose(expected, test_values)
+
+ def test_make_bulk(self, leiden_clustering, datastore):
+ df = datastore.make_bulk(group_key="RNA_leiden_cluster")
+ assert df.shape == (18850, 10)
+ assert hash(tuple((df.values.flatten()))) == -3925915741848261436
+
+ def test_to_anndata(self, datastore):
+ # TODO: Check if all the attributes copied to anndata
+ datastore.to_anndata()
+
+ def test_run_topacedo_sampler(self, cell_attrs, topacedo_sampler):
+ assert np.all(topacedo_sampler == cell_attrs["RNA_sketched"])
+
+ def test_plot_cells_dists(self, datastore):
+ datastore.plot_cells_dists(show_fig=False)
+
+ def test_plot_layout(self, umap, paris_clustering, datastore):
+ datastore.plot_layout(
+ layout_key="RNA_UMAP", color_by="RNA_cluster", show_fig=False
+ )
+
+ def test_plot_layout_shade(self, umap, paris_clustering, datastore):
+ axs = datastore.plot_layout(
+ layout_key="RNA_UMAP",
+ color_by="RNA_cluster",
+ show_fig=False,
+ do_shading=True,
+ shade_npixels=64,
+ )
+ assert axs is not None
+
+ def test_plot_cluster_tree(self, datastore):
+ datastore.plot_cluster_tree(cluster_key="RNA_cluster", show_fig=False)
+
+ def test_plot_marker_heatmap(self, marker_search, datastore):
+ datastore.plot_marker_heatmap(group_key="RNA_cluster", show_fig=False)
+
+ def test_plot_unified_layout(self, run_unified_umap, datastore):
+ datastore.plot_unified_layout(layout_key="unified_UMAP", show_fig=False)
+
+ def test_plot_unified_layout_target_groups(
+ self, run_unified_umap, paris_clustering, datastore
+ ):
+ from scarf._types import as_zarr_array, as_zarr_group
+
+ projections = as_zarr_group(
+ as_zarr_group(datastore.zw["RNA"], name="RNA")["projections"],
+ name="projections",
+ )
+ layout = as_zarr_array(projections["unified_UMAP"], name="unified_UMAP")
+ n_target_cells = int(layout.attrs["n_cells"][1])
+ target_groups = paris_clustering[:n_target_cells]
+ datastore.plot_unified_layout(
+ layout_key="unified_UMAP",
+ show_target_only=True,
+ legend_ondata=True,
+ target_groups=target_groups,
+ show_fig=False,
+ )
+
+ def test_plot_pseudotime_heatmap(self, pseudotime_aggregation, datastore):
+ datastore.plot_pseudotime_heatmap(
+ cell_key="I",
+ feat_key="I",
+ feature_cluster_key="pseudotime_clusters",
+ pseudotime_key="RNA_pseudotime",
+ show_features=["Wsb1", "Rest"],
+ show_fig=False,
+ )
+
+ def test_mark_hvgs_with_atac_assay(self, atac_datastore):
+ import pytest
+
+ with pytest.raises(TypeError):
+ atac_datastore.mark_hvgs()
+
+ def test_mark_hvgs_default_max_cells_unbounded(self, auto_filter_cells, datastore):
+ from unittest.mock import patch
+
+ with patch.object(datastore.RNA, "mark_hvgs") as mock:
+ datastore.mark_hvgs(show_plot=False, top_n=10)
+ assert mock.call_args.kwargs["max_cells"] == np.inf
+
+ def test_mark_prevalent_peaks_with_rna_assay(self, datastore):
+ import pytest
+
+ with pytest.raises(TypeError):
+ datastore.mark_prevalent_peaks()
+
+ def test_run_marker_search_with_no_groupkey(self, datastore):
+ import pytest
+
+ with pytest.raises(ValueError):
+ datastore.run_marker_search(group_key=None)
+
+ def test_run_marker_search_with_cellkey(self, datastore, paris_clustering):
+ datastore.run_marker_search(group_key="RNA_cluster", cell_key="I")
+
+ def test_streaming_feature_stats_matches_three_pass(self, datastore):
+ from scarf.utils import controlled_compute
+
+ assay = datastore.RNA
+ cell_idx, feat_idx = assay._get_cell_feat_idx("I", "I")
+
+ tiled = assay._streaming_feature_stats(cell_idx, feat_idx)
+
+ normed = assay.normed(cell_idx, feat_idx)
+ legacy_n = controlled_compute((normed > 0).sum(axis=0), assay.nthreads)
+ legacy_tot = controlled_compute(normed.sum(axis=0), assay.nthreads)
+ legacy_sigmas = controlled_compute(normed.var(axis=0), assay.nthreads)
+
+ np.testing.assert_allclose(tiled["normed_n"], legacy_n, rtol=0, atol=1e-6)
+ np.testing.assert_allclose(
+ tiled["normed_tot"], legacy_tot, rtol=1e-6, atol=1e-6
+ )
+ np.testing.assert_allclose(tiled["sigmas"], legacy_sigmas, rtol=1e-5, atol=1e-6)
+
+ def test_streaming_feature_stats_column_block_branch(self, datastore):
+ # Feature stats stream memory-bounded tiles; results must match the
+ # full-width reductions regardless of worker count / tile size.
+ from scarf.storage.budget import ResourceBudget, set_resource_budget
+ from scarf.utils import controlled_compute
+
+ assay = datastore.RNA
+ cell_idx, feat_idx = assay._get_cell_feat_idx("I", "I")
+ normed = assay.normed(cell_idx, feat_idx)
+ legacy_tot = controlled_compute(normed.sum(axis=0), assay.nthreads)
+ legacy_sigmas = controlled_compute(normed.var(axis=0), assay.nthreads)
+
+ try:
+ set_resource_budget(ResourceBudget(memoryBytes=1 * 1024 * 1024, workers=2))
+ tiled = assay._streaming_feature_stats(cell_idx, feat_idx)
+ finally:
+ set_resource_budget(None)
+
+ np.testing.assert_allclose(
+ tiled["normed_tot"], legacy_tot, rtol=1e-6, atol=1e-6
+ )
+ np.testing.assert_allclose(tiled["sigmas"], legacy_sigmas, rtol=1e-5, atol=1e-6)
+
+ def test_streaming_feature_stats_one_decode_per_physical_chunk(
+ self, datastore, monkeypatch
+ ):
+ import scarf.assay as assay_mod
+
+ assay = datastore.RNA
+ cell_idx, feat_idx = assay._get_cell_feat_idx("I", "I")
+ zarr_arr = assay.rawData._backing
+ row_chunk, col_chunk = zarr_arr.chunks[:2]
+ expected = len({int(i) // row_chunk for i in cell_idx}) * len(
+ {int(i) // col_chunk for i in feat_idx}
+ )
+ calls = {"n": 0}
+ original = assay_mod._read_block
+
+ def counted(zarr_arr_arg, rows, cols):
+ calls["n"] += 1
+ return original(zarr_arr_arg, rows, cols)
+
+ monkeypatch.setattr(assay_mod, "_read_block", counted)
+ assay._streaming_feature_stats(cell_idx, feat_idx)
+ assert calls["n"] == expected
+
+ def test_feature_stats_tile_shape_bounds_full_width_matrix(self):
+ from scarf.assay import _feature_stats_tile_shape
+ from scarf.storage.budget import ResourceBudget
+
+ budget = ResourceBudget(
+ memoryBytes=48 * 1024**3,
+ workers=8,
+ workingCopies=4,
+ )
+ rows, cols = _feature_stats_tile_shape(
+ 22_201,
+ 45_525,
+ row_chunk=11_792,
+ col_chunk=45_525,
+ budget=budget,
+ )
+ assert rows * cols * 12 <= 256 * 1024 * 1024
+ assert rows >= 1
+ assert cols >= 1
+ assert rows <= 11_792
+ assert cols <= 45_525
+
+ def test_streaming_feature_stats_requires_sf(self, datastore):
+ import pytest
+
+ assay = datastore.RNA
+ cell_idx, feat_idx = assay._get_cell_feat_idx("I", "I")
+ original = assay.sf
+ try:
+ assay.sf = None
+ with pytest.raises(ValueError):
+ assay._streaming_feature_stats(cell_idx, feat_idx)
+ finally:
+ assay.sf = original
+
+ def test_read_block_preserves_order_and_selection(self, datastore):
+ from scarf.assay import _read_block
+
+ backing = datastore.RNA.rawData._backing
+
+ rows = np.array([3, 4, 5, 6])
+ cols = np.array([0, 1, 2])
+ contiguous = _read_block(backing, rows, cols)
+ reference = np.asarray(backing.get_orthogonal_selection((rows, cols)))
+ np.testing.assert_array_equal(contiguous, reference)
+
+ scattered_rows = np.array([7, 2, 9])
+ scattered_cols = np.array([4, 0, 2])
+ scattered = _read_block(backing, scattered_rows, scattered_cols)
+ ref2 = np.asarray(
+ backing.get_orthogonal_selection((scattered_rows, scattered_cols))
+ )
+ np.testing.assert_array_equal(scattered, ref2)
diff --git a/tests/test_datastore_ephemeral.py b/tests/test_datastore_ephemeral.py
new file mode 100644
index 00000000..54d5475b
--- /dev/null
+++ b/tests/test_datastore_ephemeral.py
@@ -0,0 +1,690 @@
+import numpy as np
+import pytest
+
+from tests.fixtures_datastore import _has_graph
+
+
+def _active_cell_count(datastore) -> int:
+ return len(datastore.cells.active_index("I"))
+
+
+def _ensure_graph(datastore):
+ if not _has_graph(datastore):
+ datastore.auto_filter_cells(show_qc_plots=False)
+ datastore.mark_hvgs(top_n=100, show_plot=False)
+ datastore.make_graph(feat_key="hvgs")
+
+
+def _clear_umap_columns(datastore):
+ for column in ("RNA_UMAP1", "RNA_UMAP2"):
+ if column in datastore.cells.columns:
+ datastore.cells.drop(column)
+
+
+def _clear_projection(datastore, name: str):
+ projections = datastore.z["RNA"].get("projections")
+ if projections is not None and name in projections:
+ del projections[name]
+
+
+def test_run_umap_recomputes_coordinates(datastore_ephemeral):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ _clear_umap_columns(ds)
+
+ ds.run_umap(n_epochs=10, parallel=False)
+
+ umap1 = ds.cells.fetch("RNA_UMAP1")
+ umap2 = ds.cells.fetch("RNA_UMAP2")
+ assert len(umap1) == _active_cell_count(ds)
+ assert len(umap2) == _active_cell_count(ds)
+
+
+def test_run_leiden_writes_cluster_labels(datastore_ephemeral):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+
+ ds.run_leiden_clustering(label="ephemeral_leiden")
+
+ groups = ds.cells.fetch("RNA_ephemeral_leiden", key="I")
+ assert len(groups) == _active_cell_count(ds)
+ assert np.unique(groups).size >= 1
+
+
+def test_run_mapping_writes_projection(datastore_ephemeral):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ _clear_projection(ds, "freshmap")
+
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="freshmap",
+ target_feat_key="hvgs_freshmap",
+ save_k=3,
+ )
+
+ assert "freshmap" in ds.z["RNA"]["projections"]
+ assert ds.z["RNA"]["projections"]["freshmap"]["indices"].shape[
+ 0
+ ] == _active_cell_count(ds)
+
+
+def test_run_mapping_supports_all_features_key(datastore_ephemeral):
+ ds = datastore_ephemeral
+ ds.auto_filter_cells(show_qc_plots=False)
+ ds.make_graph(feat_key="I", dims=5, k=3, n_centroids=10)
+
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="all_features_map",
+ target_feat_key="all_features_map_target",
+ save_k=3,
+ )
+
+ assert ds.z["RNA"]["projections"]["all_features_map"].attrs["complete"]
+
+
+def test_run_mapping_with_coral_writes_corrected_data(datastore_ephemeral):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ _clear_projection(ds, "freshmap_coral")
+
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="freshmap_coral",
+ target_feat_key="hvgs_fresh_coral",
+ save_k=3,
+ run_coral=True,
+ )
+
+ normed_loc = "normed__I__hvgs_fresh_coral"
+ assert "data_coral" in ds.RNA.z[normed_loc]
+ assert "freshmap_coral" in ds.z["RNA"]["projections"]
+
+
+def test_coral_mapping_cache_and_rebuild_are_equivalent(datastore_ephemeral):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="coral_cached",
+ target_feat_key="hvgs_coral_cached",
+ save_k=3,
+ run_coral=True,
+ )
+ cached = ds.z["RNA"]["projections"]["coral_cached"]
+ cached_indices = cached["indices"][:]
+ cached_distances = cached["distances"][:]
+
+ normed = ds.z["RNA"]["normed__I__hvgs"]
+ reduction = ds.z[normed.attrs["latest_reduction"]]
+ ann = ds.z[reduction.attrs["latest_ann"]]
+ del ann["ann_idx_bytes"]
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="coral_rebuilt",
+ target_feat_key="hvgs_coral_rebuilt",
+ save_k=3,
+ run_coral=True,
+ )
+ rebuilt = ds.z["RNA"]["projections"]["coral_rebuilt"]
+
+ assert np.array_equal(cached_indices, rebuilt["indices"][:])
+ np.testing.assert_allclose(cached_distances, rebuilt["distances"][:])
+
+
+def test_run_unified_umap_after_mapping(datastore_ephemeral):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ _clear_projection(ds, "freshmap_unified_src")
+ _clear_projection(ds, "unified_UMAP")
+
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="freshmap_unified_src",
+ target_feat_key="hvgs_unified_src",
+ save_k=3,
+ )
+ ds.run_unified_umap(target_names=["freshmap_unified_src"], n_epochs=10)
+
+ coords = ds.z["RNA"]["projections"]["unified_UMAP"][:]
+ assert coords.shape[1] == 2
+ assert coords.shape[0] >= _active_cell_count(ds)
+
+
+def test_build_and_reload_symphony_mapping_reference(datastore_ephemeral, tmp_path):
+ import numpy as np
+ import pandas as pd
+
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ batches = np.where(np.arange(ds.cells.N) % 2 == 0, "a", "b")
+ ds.cells.insert("mapping_batch", batches, overwrite=True)
+ active_batches = ds.cells.fetch("mapping_batch", key="I").copy()
+
+ reference = ds.build_mapping_reference(
+ feat_key="hvgs",
+ batch_columns=["mapping_batch"],
+ k=3,
+ n_centroids=10,
+ harmony_params={"nclust": 5},
+ )
+ normed = ds.z["RNA"]["normed__I__hvgs"]
+ reduction = ds.z[normed.attrs["latest_reduction"]]
+ reference_coordinates = reduction["harmonizedData"][:].copy()
+ reference_loadings = reduction["reduction"][:].copy()
+ reference_ann = ds.z[reduction.attrs["latest_ann"]]
+ reference_ann_metadata = dict(reference_ann.attrs)
+ loaded = ds.get_mapping_reference(feat_key="hvgs")
+ with pytest.raises(ValueError, match="matches the reference path"):
+ loaded.map_query(
+ ds.RNA,
+ "unsafe_symphony_self",
+ "hvgs",
+ save_k=3,
+ )
+ result = loaded.map_query(
+ ds.RNA,
+ "symphony_self",
+ "hvgs_symphony_self",
+ save_k=3,
+ query_batches=pd.DataFrame({"mapping_batch": active_batches}),
+ )
+
+ assert reference.model.n_dims == loaded.model.n_dims
+ assert reference.metadata["harmonyParameters"]["nclust"] == 5
+ assert result.n_cells == _active_cell_count(ds)
+ projection = ds.z["RNA"]["projections"]["symphony_self"]
+ assert projection.attrs["complete"]
+ assert projection["correctedLatent"].shape[1] == loaded.model.n_dims
+ assert projection["indices"].chunks[1] == 3
+ assert projection["distances"].chunks[1] == 3
+ assert projection.attrs["queryBatchColumns"] == ["mapping_batch"]
+ assert isinstance(projection.attrs["queryBatchHash"], str)
+ original_indices = projection["indices"][:].copy()
+ original_corrected = projection["correctedLatent"][:].copy()
+ np.testing.assert_array_equal(reduction["harmonizedData"][:], reference_coordinates)
+ np.testing.assert_array_equal(reduction["reduction"][:], reference_loadings)
+ assert dict(reference_ann.attrs) == reference_ann_metadata
+ artifact = ds.z[loaded.artifact_path]
+ artifact.attrs["complete"] = False
+ try:
+ with pytest.raises(ValueError, match="incomplete"):
+ next(ds.get_mapping_score("symphony_self"))
+ finally:
+ artifact.attrs["complete"] = True
+
+ from scarf.datastore.datastore import DataStore
+
+ read_only = DataStore(ds.zarr_loc, default_assay="RNA", zarr_mode="r")
+ in_memory = read_only.get_mapping_reference(feat_key="hvgs").map_query(
+ ds.RNA,
+ "symphony_read_only",
+ "hvgs_symphony_read_only",
+ save_k=3,
+ query_batches=pd.DataFrame({"mapping_batch": active_batches}),
+ )
+ assert in_memory.projection_path == ""
+ assert in_memory.indices is not None
+ assert in_memory.corrected_latent is not None
+ persisted_indices = projection["indices"][:]
+ overlap = np.mean(
+ [
+ len(set(expected) & set(observed)) / len(expected)
+ for expected, observed in zip(persisted_indices, in_memory.indices)
+ ]
+ )
+ assert overlap >= 0.99
+ np.testing.assert_allclose(
+ projection["correctedLatent"][:],
+ in_memory.corrected_latent,
+ rtol=1e-10,
+ atol=1e-12,
+ )
+
+ read_only_single_thread = DataStore(
+ ds.zarr_loc,
+ default_assay="RNA",
+ zarr_mode="r",
+ nthreads=1,
+ )
+ single_thread = read_only_single_thread.get_mapping_reference(
+ feat_key="hvgs"
+ ).map_query(
+ ds.RNA,
+ "symphony_single_thread",
+ "hvgs_symphony_single_thread",
+ save_k=3,
+ query_batches=pd.DataFrame({"mapping_batch": active_batches}),
+ )
+ assert single_thread.corrected_latent is not None
+ np.testing.assert_allclose(
+ in_memory.corrected_latent,
+ single_thread.corrected_latent,
+ rtol=1e-10,
+ atol=1e-12,
+ )
+
+ import zarr
+
+ result_store = zarr.open_group(
+ str(tmp_path / "mapping_result.zarr"), mode="w"
+ ).create_group("projection")
+ streamed = read_only.get_mapping_reference(feat_key="hvgs").map_query(
+ ds.RNA,
+ "symphony_streamed",
+ "hvgs_symphony_streamed",
+ save_k=3,
+ query_batches=pd.DataFrame({"mapping_batch": active_batches}),
+ result_store=result_store,
+ )
+ assert streamed.indices is None
+ assert result_store["indices"].shape[0] == _active_cell_count(ds)
+
+ ds.cells.insert(
+ "mapping_batch",
+ np.repeat("changed", ds.cells.N),
+ overwrite=True,
+ )
+ with pytest.raises(ValueError, match="stale"):
+ ds.get_mapping_reference(feat_key="hvgs")
+
+ artifact_count = len(reduction["mappingReferences"])
+ replacement_batches = np.where(np.arange(ds.cells.N) % 2 == 0, "c", "d")
+ ds.cells.insert(
+ "mapping_batch",
+ replacement_batches,
+ overwrite=True,
+ )
+ rebuilt_reference = ds.build_mapping_reference(
+ feat_key="hvgs",
+ batch_columns=["mapping_batch"],
+ k=3,
+ n_centroids=10,
+ harmony_params={"nclust": 5},
+ )
+ assert rebuilt_reference.artifact_path != reference.artifact_path
+ assert rebuilt_reference.ann_path != reference.ann_path
+ assert len(reduction["mappingReferences"]) == artifact_count + 1
+ old_reference_result = reference.map_query(
+ ds.RNA,
+ "old_reference_replay",
+ "hvgs_old_reference_replay",
+ save_k=3,
+ query_batches=pd.DataFrame({"mapping_batch": active_batches}),
+ )
+ old_projection = ds.z["RNA"]["projections"]["old_reference_replay"]
+ np.testing.assert_array_equal(old_projection["indices"][:], original_indices)
+ np.testing.assert_allclose(
+ old_projection["correctedLatent"][:],
+ original_corrected,
+ rtol=1e-10,
+ atol=1e-12,
+ )
+ assert old_reference_result.projection_path
+
+
+def test_mapping_rejects_reference_normalization_overwrite(datastore_ephemeral):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ normed = ds.z["RNA"]["normed__I__hvgs"]
+ reduction = ds.z[normed.attrs["latest_reduction"]]
+ ann = ds.z[reduction.attrs["latest_ann"]]
+ reference_data = normed["data"][:].copy()
+ reference_loadings = reduction["reduction"][:].copy()
+ ann_metadata = dict(ann.attrs)
+
+ with pytest.raises(ValueError, match="matches the reference path"):
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="unsafe_self_map",
+ target_feat_key="hvgs",
+ save_k=3,
+ )
+
+ from scarf.datastore.datastore import DataStore
+
+ separately_opened = DataStore(ds.zarr_loc, default_assay="RNA")
+ with pytest.raises(ValueError, match="matches the reference path"):
+ ds.run_mapping(
+ target_assay=separately_opened.RNA,
+ target_name="unsafe_separate_self_map",
+ target_feat_key="hvgs",
+ save_k=3,
+ )
+
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="safe_self_map",
+ target_feat_key="hvgs_safe_self_map",
+ save_k=3,
+ )
+
+ np.testing.assert_array_equal(
+ ds.z["RNA"]["normed__I__hvgs"]["data"][:], reference_data
+ )
+ np.testing.assert_array_equal(reduction["reduction"][:], reference_loadings)
+ assert dict(ann.attrs) == ann_metadata
+
+
+def test_cached_harmony_can_rebuild_missing_mapping_artifact(datastore_ephemeral):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ batches = np.where(np.arange(ds.cells.N) % 2 == 0, "a", "b")
+ ds.cells.insert("mapping_batch", batches, overwrite=True)
+ ds.build_mapping_reference(
+ feat_key="hvgs",
+ batch_columns=["mapping_batch"],
+ k=3,
+ n_centroids=10,
+ )
+ normed = ds.z["RNA"]["normed__I__hvgs"]
+ reduction = ds.z[normed.attrs["latest_reduction"]]
+ assert "harmonizedData" in reduction
+ artifact_hash = reduction.attrs["latestMappingReference"]
+ artifact = reduction["mappingReferences"][artifact_hash]
+ legacy = reduction.create_group("mappingReference")
+ for key, value in dict(artifact.attrs).items():
+ if key not in {
+ "algorithmVariant",
+ "annContract",
+ "artifactHash",
+ "correctedCoordinatesHash",
+ }:
+ legacy.attrs[key] = value
+ legacy.attrs["schemaVersion"] = 1
+ for name in (
+ "featureIds",
+ "featureMeans",
+ "featureScales",
+ "loadings",
+ "centroids",
+ "rawCentroids",
+ "correctedCentroids",
+ "clusterMass",
+ "sigma",
+ ):
+ legacy.create_array(name, data=np.asarray(artifact[name][:]))
+ del reduction["mappingReferences"]
+ del reduction.attrs["latestMappingReference"]
+
+ with pytest.warns(DeprecationWarning, match="legacy"):
+ legacy_reference = ds.get_mapping_reference(feat_key="hvgs")
+ assert legacy_reference.metadata["schemaVersion"] == 1
+ del reduction["mappingReference"]
+
+ from scarf.datastore.datastore import DataStore
+
+ read_only = DataStore(ds.zarr_loc, default_assay="RNA", zarr_mode="r")
+ with pytest.raises(ValueError, match="zarr_mode='r\\+'"):
+ read_only.run_mapping(
+ target_assay=ds.RNA,
+ target_name="read_only_legacy_harmony",
+ target_feat_key="hvgs_read_only_legacy_harmony",
+ save_k=3,
+ )
+
+ cached_ann = ds.z[reduction.attrs["latest_ann"]]
+ cached_knn = ds.z[cached_ann.attrs["latest_knn"]]
+ cached_knn["distances"][0, 0] = 123456.0
+ rebuilt = ds.build_mapping_reference(
+ feat_key="hvgs",
+ batch_columns=["mapping_batch"],
+ )
+
+ assert rebuilt.metadata["complete"]
+ assert "mappingReferences" in reduction
+ rebuilt_ann = ds.z[rebuilt.ann_path]
+ rebuilt_knn = ds.z[rebuilt_ann.attrs["latest_knn"]]
+ assert np.isfinite(rebuilt_knn["distances"][0, 0])
+ assert rebuilt_knn["distances"][0, 0] != 123456.0
+
+
+def test_run_mapping_upgrades_legacy_harmonized_graph(datastore_ephemeral):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ batches = np.where(np.arange(ds.cells.N) % 2 == 0, "a", "b")
+ ds.cells.insert("mapping_batch", batches, overwrite=True)
+ ds.build_mapping_reference(
+ feat_key="hvgs",
+ batch_columns=["mapping_batch"],
+ k=3,
+ n_centroids=10,
+ )
+ normed = ds.z["RNA"]["normed__I__hvgs"]
+ reduction = ds.z[normed.attrs["latest_reduction"]]
+ del reduction["mappingReferences"]
+ del reduction.attrs["latestMappingReference"]
+
+ with pytest.warns(DeprecationWarning, match="rebuilt once"):
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="upgraded_harmonized_map",
+ target_feat_key="hvgs_upgraded_harmonized",
+ save_k=3,
+ )
+
+ assert ds.z["RNA"]["projections"]["upgraded_harmonized_map"].attrs["complete"]
+
+
+def test_legacy_harmony_upgrade_requires_recorded_batch_columns(
+ datastore_ephemeral,
+):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ batches = np.where(np.arange(ds.cells.N) % 2 == 0, "a", "b")
+ ds.cells.insert("mapping_batch", batches, overwrite=True)
+ ds.build_mapping_reference(
+ feat_key="hvgs",
+ batch_columns=["mapping_batch"],
+ k=3,
+ n_centroids=10,
+ )
+ normed = ds.z["RNA"]["normed__I__hvgs"]
+ reduction = ds.z[normed.attrs["latest_reduction"]]
+ del reduction["mappingReferences"]
+ del reduction.attrs["latestMappingReference"]
+ del reduction["harmonizedData"].attrs["batches"]
+
+ with pytest.raises(ValueError, match="does not record its batch columns"):
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="legacy_harmony_without_batches",
+ target_feat_key="hvgs_legacy_harmony_without_batches",
+ save_k=3,
+ )
+
+
+def test_deprecated_reference_scaling_flags_do_not_break_mapping(
+ datastore_ephemeral,
+):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+
+ with pytest.warns(DeprecationWarning, match="ignored"):
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="deprecated_scaling_flags",
+ target_feat_key="hvgs_deprecated_scaling",
+ ref_mu=False,
+ ref_sigma=False,
+ save_k=3,
+ )
+
+ assert ds.z["RNA"]["projections"]["deprecated_scaling_flags"].attrs["complete"]
+
+
+def test_mapping_feature_scaling_uses_isolated_ann_cache(datastore_ephemeral):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ normed = ds.z["RNA"]["normed__I__hvgs"]
+ reduction = ds.z[normed.attrs["latest_reduction"]]
+ original_ann_path = reduction.attrs["latest_ann"]
+
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="unscaled_mapping",
+ target_feat_key="hvgs_unscaled_mapping",
+ feat_scaling=False,
+ save_k=3,
+ )
+
+ projection = ds.z["RNA"]["projections"]["unscaled_mapping"]
+ mapping_ann_path = projection.attrs["annPath"]
+ assert mapping_ann_path != original_ann_path
+ assert mapping_ann_path.endswith("__unscaled")
+ assert not ds.z[mapping_ann_path].attrs["featureScaling"]
+ assert reduction.attrs["latest_ann"] == original_ann_path
+
+
+def test_projection_uses_stored_feature_key_and_rejects_stale_provenance(
+ datastore_ephemeral,
+):
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ ds.run_mapping(
+ target_assay=ds.RNA,
+ target_name="provenance_map",
+ target_feat_key="hvgs_provenance_map",
+ save_k=3,
+ )
+ projection = ds.z["RNA"]["projections"]["provenance_map"]
+ original_latest = ds.RNA.z.attrs["latest_feat_key"]
+ ds.RNA.z.attrs["latest_feat_key"] = "I"
+ try:
+ score = next(ds.get_mapping_score("provenance_map"))[1]
+ finally:
+ ds.RNA.z.attrs["latest_feat_key"] = original_latest
+ assert np.all(np.isfinite(score))
+
+ original_hash = projection.attrs["reductionHash"]
+ projection.attrs["reductionHash"] = "stale"
+ try:
+ with pytest.raises(ValueError, match="changed reduction"):
+ next(ds.get_mapping_score("provenance_map"))
+ finally:
+ projection.attrs["reductionHash"] = original_hash
+
+
+def test_mapping_missing_feature_policies_and_legacy_intersection(
+ datastore_ephemeral,
+):
+ from scarf.datastore.datastore import DataStore
+
+ ds = datastore_ephemeral
+ _ensure_graph(ds)
+ shared_ids = ds.RNA.feats.fetch("ids", key="I__hvgs")[: ds.assay2.feats.N]
+ ds.assay2.feats.insert(
+ "ids",
+ shared_ids,
+ overwrite=True,
+ force=True,
+ )
+ mapped = DataStore(
+ ds.zarr_loc,
+ assay_types={"RNA": "RNA", "assay2": "RNA"},
+ default_assay="RNA",
+ min_features_per_cell=0,
+ min_cells_per_feature=0,
+ )
+
+ with pytest.raises(ValueError, match="missing"):
+ mapped.run_mapping(
+ target_assay=mapped.assay2,
+ target_name="missing_error",
+ target_feat_key="missing_error_target",
+ missing_feature_policy="error",
+ save_k=3,
+ )
+
+ mapped.run_mapping(
+ target_assay=mapped.assay2,
+ target_name="missing_zero",
+ target_feat_key="missing_zero_target",
+ missing_feature_policy="zero",
+ save_k=3,
+ )
+ missing_zero = mapped.z["RNA"]["projections"]["missing_zero"]
+ assert missing_zero.attrs["complete"]
+ assert missing_zero.attrs["featureCoverage"] == pytest.approx(
+ len(shared_ids) / len(mapped.RNA.feats.fetch("ids", key="I__hvgs"))
+ )
+ zero_aligned = mapped.assay2.z["normed__I__missing_zero_target/data"][:]
+ np.testing.assert_array_equal(zero_aligned[:, len(shared_ids) :], 0.0)
+
+ with pytest.warns(DeprecationWarning, match="exclude_missing"):
+ mapped.run_mapping(
+ target_assay=mapped.assay2,
+ target_name="missing_intersection",
+ target_feat_key="missing_intersection_target",
+ exclude_missing=True,
+ save_k=3,
+ )
+ assert "I__hvgs_common_missing_intersection" in mapped.RNA.feats.columns
+ intersection_projection = mapped.z["RNA"]["projections"]["missing_intersection"]
+ intersection_ann_path = intersection_projection.attrs["annPath"]
+ assert "__intersection_" in intersection_ann_path
+ assert intersection_ann_path in mapped.z
+ assert (
+ mapped.z[intersection_ann_path].attrs["selectedFeatureHash"]
+ == intersection_projection.attrs["selectedFeatureHash"]
+ )
+ assert (
+ mapped.z[intersection_ann_path].attrs["sourceAnnPath"]
+ == intersection_projection.attrs["annSourcePath"]
+ )
+ assert np.all(
+ np.isfinite(next(mapped.get_mapping_score("missing_intersection"))[1])
+ )
+ intersection_evidence = mapped.get_target_label_evidence(
+ "missing_intersection",
+ reference_class_group="ids",
+ )
+ assert np.all(np.isfinite(intersection_evidence["referenceDistancePercentile"]))
+ mapped.run_mapping(
+ target_assay=mapped.assay2,
+ target_name="missing_intersection_unscaled",
+ target_feat_key="missing_intersection_unscaled_target",
+ missing_feature_policy="intersection",
+ feat_scaling=False,
+ save_k=3,
+ )
+ unscaled_intersection = mapped.z["RNA"]["projections"][
+ "missing_intersection_unscaled"
+ ]
+ assert not unscaled_intersection.attrs["annFeatureScaling"]
+ assert "__unscaled" in unscaled_intersection.attrs["annSourcePath"]
+ assert not mapped.z[unscaled_intersection.attrs["annPath"]].attrs["featureScaling"]
+ assert np.all(
+ np.isfinite(next(mapped.get_mapping_score("missing_intersection_unscaled"))[1])
+ )
+
+ batches = np.where(np.arange(mapped.cells.N) % 2 == 0, "a", "b")
+ mapped.cells.insert("mapping_batch", batches, overwrite=True)
+ reference = mapped.build_mapping_reference(
+ feat_key="hvgs",
+ batch_columns=["mapping_batch"],
+ k=3,
+ n_centroids=10,
+ )
+ result = reference.map_query(
+ mapped.assay2,
+ "missing_reference_mean",
+ "missing_reference_mean_target",
+ save_k=3,
+ missing_feature_policy="reference_mean",
+ )
+ assert result.n_cells == mapped.cells.active_index("I").shape[0]
+ assert result.diagnostics["featureCoverage"] == pytest.approx(
+ len(shared_ids) / len(reference.feature_ids)
+ )
+ mean_aligned = mapped.assay2.z["normed__I__missing_reference_mean_target/data"][:]
+ np.testing.assert_allclose(
+ mean_aligned[:, len(shared_ids) :],
+ np.broadcast_to(
+ reference.model.feature_means[np.newaxis, len(shared_ids) :],
+ mean_aligned[:, len(shared_ids) :].shape,
+ ),
+ rtol=0,
+ atol=0,
+ )
diff --git a/tests/test_dendrogram.py b/tests/test_dendrogram.py
new file mode 100644
index 00000000..5d4b8f9a
--- /dev/null
+++ b/tests/test_dendrogram.py
@@ -0,0 +1,48 @@
+import numpy as np
+import pytest
+from scipy.cluster.hierarchy import linkage
+from scipy.spatial.distance import pdist
+
+from scarf.dendrogram import BalancedCut, CoalesceTree, make_digraph
+
+
+def test_make_digraph_builds_expected_node_count():
+ rng = np.random.default_rng(0)
+ points = rng.normal(size=(6, 4))
+ dendrogram = linkage(pdist(points), method="average")
+ graph = make_digraph(dendrogram)
+ assert graph.number_of_edges() == dendrogram.shape[0] * 2
+ assert graph.number_of_nodes() >= len(points)
+
+
+def test_make_digraph_rejects_mismatched_cluster_info():
+ rng = np.random.default_rng(1)
+ points = rng.normal(size=(5, 3))
+ dendrogram = linkage(pdist(points), method="single")
+ with pytest.raises(ValueError, match="cluster information"):
+ make_digraph(dendrogram, clust_info=np.zeros(3))
+
+
+def test_coalesce_tree_reduces_hierarchy():
+ rng = np.random.default_rng(2)
+ points = rng.normal(size=(8, 3))
+ dendrogram = linkage(pdist(points), method="average")
+ clusters = np.array([0, 0, 1, 1, 2, 2, 3, 3])
+ graph = make_digraph(dendrogram, clust_info=clusters)
+ coalesced = CoalesceTree(graph, clusters)
+ assert coalesced.number_of_nodes() <= graph.number_of_nodes()
+
+
+def test_balanced_cut_assigns_all_cells():
+ rng = np.random.default_rng(3)
+ points = rng.normal(size=(8, 3))
+ dendrogram = linkage(pdist(points), method="average")
+ cutter = BalancedCut(
+ dendrogram,
+ max_size=4,
+ min_size=1,
+ max_distance_fc=2.0,
+ )
+ labels = cutter.get_clusters()
+ assert len(labels) == len(points)
+ assert labels.min() >= -1
diff --git a/tests/test_downloader.py b/tests/test_downloader.py
new file mode 100644
index 00000000..4351b068
--- /dev/null
+++ b/tests/test_downloader.py
@@ -0,0 +1,459 @@
+import io
+from json import JSONDecodeError
+import tarfile
+
+import pytest
+
+from . import full_path, remove
+
+
+@pytest.fixture
+def mock_osf_downloader(monkeypatch):
+ from scarf import downloader
+
+ class FakeOSFdownloader:
+ projectId = "zeupv"
+ datasets = {
+ **{f"dataset_{i}": (f"path/{i}", "osfstorage") for i in range(20)},
+ "tenx_5K_pbmc_rnaseq": ("tenx_5K", "osfstorage"),
+ }
+ sourceFile = "sources"
+
+ def show_datasets(self):
+ print("\n".join(sorted(self.datasets.keys())))
+
+ def get_dataset_file_ids(self, dataset_name):
+ if dataset_name not in self.datasets:
+ raise KeyError(dataset_name)
+ if dataset_name == "tenx_5K_pbmc_rnaseq":
+ return {"tenx_5K_pbmc.zarr.tar.gz": "http://example.test/zarr.tar.gz"}
+ return {"data.h5ad": "http://example.test/data.h5ad"}
+
+ fake = FakeOSFdownloader()
+ monkeypatch.setattr(downloader, "osfd", fake)
+ return fake
+
+
+def test_downloader(mock_osf_downloader):
+ assert len(mock_osf_downloader.datasets) > 15
+
+
+def test_osf_downloader_initializes_and_lists_sorted(monkeypatch, capsys):
+ from scarf.downloader import OSFdownloader
+
+ datasets = {
+ "zeta": ("zeta-id", "figshare"),
+ "alpha": ("alpha-id", "osfstorage"),
+ }
+ monkeypatch.setattr(
+ OSFdownloader,
+ "_populate_datasets",
+ lambda self: (datasets, "sources-id"),
+ )
+ monkeypatch.setattr(
+ OSFdownloader,
+ "_populate_sources",
+ lambda self: {"url": {"alpha": "https://example.test/alpha"}},
+ )
+
+ client = OSFdownloader("project-id")
+
+ assert client.projectId == "project-id"
+ assert client.storages == ["osfstorage", "figshare"]
+ assert client.datasets == datasets
+ assert client.sourceFile == "sources-id"
+ client.show_datasets()
+ assert capsys.readouterr().out == "alpha\nzeta\n"
+
+
+def test_osf_downloader_builds_api_urls(monkeypatch):
+ import requests
+
+ from scarf.downloader import OSFdownloader
+
+ requested_urls = []
+
+ class Response:
+ def json(self):
+ return {"data": []}
+
+ def fake_get(url):
+ requested_urls.append(url)
+ return Response()
+
+ monkeypatch.setattr(requests, "get", fake_get)
+ client = object.__new__(OSFdownloader)
+ client.url = "https://api.example.test/files/"
+
+ assert client.get_json("osfstorage", "folder", None) == {"data": []}
+ assert client.get_json("figshare", "", "https://next.example.test") == {"data": []}
+ assert requested_urls == [
+ "https://api.example.test/files/osfstorage/folder/",
+ "https://next.example.test",
+ ]
+
+
+def test_osf_downloader_retries_and_paginates(monkeypatch):
+ from scarf.downloader import OSFdownloader
+
+ client = object.__new__(OSFdownloader)
+ responses = iter(
+ [
+ JSONDecodeError("invalid response", "", 0),
+ {},
+ {"data": [{"id": "first"}], "links": {"next": "next-page"}},
+ {"data": [{"id": "second"}], "links": {"next": None}},
+ ]
+ )
+ calls = []
+ sleeps = []
+
+ def fake_get_json(storage, endpoint, url):
+ calls.append((storage, endpoint, url))
+ response = next(responses)
+ if isinstance(response, Exception):
+ raise response
+ return response
+
+ monkeypatch.setattr(client, "get_json", fake_get_json)
+ monkeypatch.setattr("scarf.downloader.time.sleep", sleeps.append)
+
+ assert client.get_all_pages("osfstorage", "folder") == [
+ {"id": "first"},
+ {"id": "second"},
+ ]
+ assert calls == [
+ ("osfstorage", "folder", None),
+ ("osfstorage", "folder", None),
+ ("osfstorage", "folder", None),
+ ("osfstorage", "folder", "next-page"),
+ ]
+ assert sleeps == [1, 1]
+
+
+def test_osf_downloader_discovers_datasets_and_files(monkeypatch):
+ from scarf.downloader import OSFdownloader
+
+ client = object.__new__(OSFdownloader)
+ client.projectId = "project-id"
+ client.storages = ["osfstorage", "figshare"]
+
+ def node(name, path):
+ return {"attributes": {"name": name, "path": path}}
+
+ def fake_get_all_pages(storage, endpoint=""):
+ if endpoint == "":
+ if storage == "osfstorage":
+ return [
+ node("sources", "/sources-id/"),
+ node("alpha", "/alpha-id/"),
+ ]
+ return [node("beta", "/beta-id/")]
+ assert storage == "osfstorage"
+ assert endpoint == "alpha-id"
+ return [
+ node("counts.mtx.gz", "/files/counts-id/"),
+ node("barcodes.tsv.gz", "/files/barcodes-id/"),
+ ]
+
+ monkeypatch.setattr(client, "get_all_pages", fake_get_all_pages)
+
+ datasets, source_file = client._populate_datasets()
+ assert datasets == {
+ "alpha": ("alpha-id", "osfstorage"),
+ "beta": ("beta-id", "figshare"),
+ }
+ assert source_file == "sources-id"
+
+ client.datasets = datasets
+ expected_files = {
+ "counts.mtx.gz": (
+ "https://files.de-1.osf.io/v1/resources/project-id/providers/"
+ "osfstorage/files/counts-id"
+ ),
+ "barcodes.tsv.gz": (
+ "https://files.de-1.osf.io/v1/resources/project-id/providers/"
+ "osfstorage/files/barcodes-id"
+ ),
+ }
+ assert client.get_dataset_file_ids("alpha") == expected_files
+
+ with pytest.raises(KeyError, match="missing was not found") as error:
+ client.get_dataset_file_ids("missing")
+ assert "alpha\nbeta" in error.value.args[0]
+
+
+def test_osf_downloader_populates_sources(monkeypatch):
+ import requests
+
+ from scarf.downloader import OSFdownloader
+
+ client = object.__new__(OSFdownloader)
+ client.sourceFile = "sources-id"
+ monkeypatch.setattr(
+ client,
+ "_get_files_for_node",
+ lambda storage, file_id: {"sources.csv": "https://example.test/sources.csv"},
+ )
+
+ class Response:
+ text = (
+ "id,title,url\n"
+ "alpha,Alpha dataset,https://example.test/alpha\n"
+ "beta,Beta dataset,https://example.test/beta\n"
+ )
+
+ requested_urls = []
+
+ def fake_get(url):
+ requested_urls.append(url)
+ return Response()
+
+ monkeypatch.setattr(requests, "get", fake_get)
+
+ assert client._populate_sources() == {
+ "title": {
+ "alpha": "Alpha dataset",
+ "beta": "Beta dataset",
+ },
+ "url": {
+ "alpha": "https://example.test/alpha",
+ "beta": "https://example.test/beta",
+ },
+ }
+ assert requested_urls == ["https://example.test/sources.csv"]
+
+
+def test_osf_downloader_source_retries_are_bounded(monkeypatch):
+ from scarf.downloader import OSFdownloader
+
+ client = object.__new__(OSFdownloader)
+ client.sourceFile = "sources-id"
+ attempts = []
+ sleeps = []
+
+ def missing_sources(storage, file_id):
+ attempts.append((storage, file_id))
+ return {}
+
+ monkeypatch.setattr(client, "_get_files_for_node", missing_sources)
+ monkeypatch.setattr("scarf.downloader.time.sleep", sleeps.append)
+
+ with pytest.raises(KeyError, match="after 5 attempts"):
+ client._populate_sources()
+ assert attempts == [("osfstorage", "sources-id")] * 5
+ assert sleeps == [1] * 5
+
+
+def test_show_available_datasets(mock_osf_downloader):
+ from scarf.downloader import show_available_datasets
+
+ show_available_datasets()
+
+
+def test_fetch_dataset(bastidas_ponce_data):
+ import os
+
+ assert os.path.isfile(bastidas_ponce_data)
+
+
+def test_downloader_as_zarr(mock_osf_downloader, monkeypatch, tmp_path):
+ from scarf import downloader
+
+ downloaded = []
+
+ def fake_handle_download(url, out_fn, seq_counter=""):
+ downloaded.append((url, out_fn))
+ with open(out_fn, "wb") as handle:
+ handle.write(b"test")
+
+ monkeypatch.setattr(downloader, "handle_download", fake_handle_download)
+
+ sample = "tenx_5K_pbmc_rnaseq"
+ save_root = str(tmp_path)
+ downloader.fetch_dataset(sample, as_zarr=True, save_path=save_root)
+ assert len(downloaded) == 1
+ assert downloaded[0][0] == "http://example.test/zarr.tar.gz"
+ assert downloaded[0][1].endswith("tenx_5K_pbmc.zarr.tar.gz")
+
+
+def test_fetch_dataset_downloads_each_non_zarr_file(monkeypatch, tmp_path):
+ from scarf import downloader
+
+ class MultiFileDownloader:
+ def get_dataset_file_ids(self, dataset_name):
+ assert dataset_name == "tiny_dataset"
+ return {
+ "matrix.mtx.gz": "https://example.test/matrix",
+ "features.tsv.gz": "https://example.test/features",
+ "tiny_dataset.zarr.tar.gz": "https://example.test/zarr",
+ }
+
+ downloads = []
+
+ def fake_handle_download(url, out_fn, seq_counter=""):
+ downloads.append((url, out_fn, seq_counter))
+
+ monkeypatch.setattr(downloader, "osfd", MultiFileDownloader())
+ monkeypatch.setattr(downloader, "handle_download", fake_handle_download)
+
+ downloader.fetch_dataset("tiny_dataset", save_path=str(tmp_path))
+
+ output_dir = tmp_path / "tiny_dataset"
+ assert output_dir.is_dir()
+ assert downloads == [
+ (
+ "https://example.test/matrix",
+ str((output_dir / "matrix.mtx.gz").absolute()),
+ "1/2",
+ ),
+ (
+ "https://example.test/features",
+ str((output_dir / "features.tsv.gz").absolute()),
+ "2/2",
+ ),
+ ]
+
+
+def test_handle_download_streams_nonempty_chunks(monkeypatch, tmp_path):
+ import requests
+
+ from scarf import downloader
+
+ calls = []
+ progress = {}
+
+ class HeadResponse:
+ headers = {"content-length": "20000001"}
+
+ class DownloadResponse:
+ def iter_content(self, chunk_size):
+ calls.append(("iter_content", chunk_size))
+ return iter([b"first", b"", b"-second"])
+
+ def fake_head(url, allow_redirects):
+ calls.append(("head", url, allow_redirects))
+ return HeadResponse()
+
+ def fake_get(url, stream):
+ calls.append(("get", url, stream))
+ return DownloadResponse()
+
+ def fake_tqdm(iterable, **kwargs):
+ progress.update(kwargs)
+ return iterable
+
+ monkeypatch.setattr(requests, "head", fake_head)
+ monkeypatch.setattr(requests, "get", fake_get)
+ monkeypatch.setattr(downloader, "tqdmbar", fake_tqdm)
+ output = tmp_path / "data.bin"
+
+ downloader.handle_download(
+ "https://example.test/data",
+ str(output),
+ seq_counter="2/3",
+ )
+
+ assert output.read_bytes() == b"first-second"
+ assert calls == [
+ ("head", "https://example.test/data", True),
+ ("get", "https://example.test/data", True),
+ ("iter_content", 10_000_000),
+ ]
+ assert progress == {"total": 3, "desc": "Downloading 2/3"}
+
+
+def test_handle_download_extracts_tar_archive(monkeypatch, tmp_path):
+ import requests
+
+ from scarf import downloader
+
+ archive = io.BytesIO()
+ payload = b"archive contents"
+ with tarfile.open(fileobj=archive, mode="w:gz") as tar:
+ member = tarfile.TarInfo("nested/data.txt")
+ member.size = len(payload)
+ tar.addfile(member, io.BytesIO(payload))
+ archive_bytes = archive.getvalue()
+
+ class HeadResponse:
+ headers = {"content-length": str(len(archive_bytes))}
+
+ class DownloadResponse:
+ def iter_content(self, chunk_size):
+ assert chunk_size == 10_000_000
+ return iter([archive_bytes])
+
+ monkeypatch.setattr(
+ requests,
+ "head",
+ lambda url, allow_redirects: HeadResponse(),
+ )
+ monkeypatch.setattr(
+ requests,
+ "get",
+ lambda url, stream: DownloadResponse(),
+ )
+ monkeypatch.setattr(downloader, "tqdmbar", lambda iterable, **kwargs: iterable)
+ output = tmp_path / "bundle.tar.gz"
+
+ downloader.handle_download("https://example.test/bundle", str(output))
+
+ assert output.read_bytes() == archive_bytes
+ assert (tmp_path / "nested" / "data.txt").read_bytes() == payload
+
+
+def test_handle_download_propagates_request_errors(monkeypatch, tmp_path):
+ import requests
+
+ from scarf import downloader
+
+ class HeadResponse:
+ headers = {}
+
+ monkeypatch.setattr(
+ requests,
+ "head",
+ lambda url, allow_redirects: HeadResponse(),
+ )
+
+ def fail_get(url, stream):
+ raise RuntimeError("request failed")
+
+ monkeypatch.setattr(requests, "get", fail_get)
+ output = tmp_path / "missing.bin"
+
+ with pytest.raises(RuntimeError, match="request failed"):
+ downloader.handle_download("https://example.test/missing", str(output))
+ assert not output.exists()
+
+
+def test_fetch_dataset_unknown_name_raises(mock_osf_downloader):
+ from scarf.downloader import fetch_dataset
+
+ with pytest.raises(KeyError):
+ fetch_dataset("this_dataset_does_not_exist", save_path=".")
+
+
+def test_fetch_dataset_as_zarr_missing_file(mock_osf_downloader, tmp_path):
+ from scarf.downloader import fetch_dataset
+
+ fetch_dataset("dataset_0", as_zarr=True, save_path=str(tmp_path))
+ assert not any(tmp_path.iterdir())
+
+
+@pytest.mark.integration
+def test_downloader_live_osf():
+ from scarf.downloader import OSFdownloader
+
+ osfd = OSFdownloader("zeupv")
+ assert len(osfd.datasets) > 15
+
+
+@pytest.mark.integration
+def test_fetch_dataset_live():
+ from scarf.downloader import fetch_dataset
+
+ sample = "tenx_5K_pbmc_rnaseq"
+ fetch_dataset(sample, as_zarr=True, save_path=full_path(None))
+ remove(full_path(sample))
diff --git a/tests/test_feat_utils.py b/tests/test_feat_utils.py
new file mode 100644
index 00000000..6c9a6c34
--- /dev/null
+++ b/tests/test_feat_utils.py
@@ -0,0 +1,70 @@
+import numpy as np
+import pandas as pd
+import pytest
+
+from scarf.feat_utils import binned_sampling, fit_lowess, hto_demux
+
+
+def test_fit_lowess_returns_per_feature_corrections():
+ rng = np.random.default_rng(0)
+ n_genes = 80
+ mean_expr = rng.uniform(0.5, 20.0, n_genes)
+ variance = mean_expr**1.5 + rng.normal(0, 0.05, n_genes)
+ variance = np.clip(variance, 0.1, None)
+
+ corrected = fit_lowess(mean_expr, variance, n_bins=8, lowess_frac=0.6)
+
+ assert corrected.shape == (n_genes,)
+ assert np.all(np.isfinite(corrected))
+ assert np.all(corrected > 0)
+
+
+def test_binned_sampling_excludes_query_genes():
+ rng = np.random.default_rng(1)
+ gene_names = [f"gene_{i}" for i in range(120)]
+ values = pd.Series(rng.exponential(1.0, len(gene_names)), index=gene_names)
+ query_genes = gene_names[10:25]
+
+ controls = binned_sampling(
+ values,
+ feature_list=query_genes,
+ ctrl_size=8,
+ n_bins=6,
+ rand_seed=42,
+ )
+
+ assert len(controls) > 0
+ assert set(controls).isdisjoint(query_genes)
+ assert all(name in gene_names for name in controls)
+
+
+def test_hto_demux_assigns_singlet_and_negative_labels():
+ rng = np.random.default_rng(2)
+ n_cells = 60
+ hto_names = ["HTO_A", "HTO_B", "HTO_C"]
+
+ background = rng.poisson(2, size=(n_cells, len(hto_names)))
+ counts = background.astype(float)
+ for i in range(n_cells):
+ dominant = i % len(hto_names)
+ counts[i, dominant] += rng.integers(30, 80)
+
+ hto_counts = pd.DataFrame(counts, columns=hto_names)
+ assignments = hto_demux(hto_counts)
+
+ assert len(assignments) == n_cells
+ assert assignments.index.equals(hto_counts.index)
+ allowed = {"Negative", "Singlet", "Doublet", *hto_names}
+ assert set(assignments.unique()).issubset(allowed)
+ assert set(assignments.unique()) & set(hto_names)
+
+
+def test_hto_demux_rejects_empty_cluster_means():
+ hto_counts = pd.DataFrame(
+ {
+ "HTO_A": [0, 0, 0],
+ "HTO_B": [0, 0, 0],
+ }
+ )
+ with pytest.raises(AssertionError):
+ hto_demux(hto_counts)
diff --git a/tests/test_graph_cache.py b/tests/test_graph_cache.py
new file mode 100644
index 00000000..9bba09f1
--- /dev/null
+++ b/tests/test_graph_cache.py
@@ -0,0 +1,163 @@
+import os
+
+import numpy as np
+import pytest
+import zarr
+from zarr.storage import MemoryStore
+
+from scarf.datastore.graph_datastore import GraphDataStore, _GraphBuildProgress
+from scarf.storage.zarr_store import copy_zarr_array, create_or_open_staged_normed_array
+
+
+def _memory_group():
+ return zarr.open_group(store=MemoryStore(), mode="w")
+
+
+def test_graph_progress_tracks_steps_and_propagates_errors() -> None:
+ progress = _GraphBuildProgress(2)
+ with progress.step("reuse graph", cached=True):
+ pass
+ with pytest.raises(RuntimeError, match="graph failed"):
+ with progress.step("build graph"):
+ raise RuntimeError("graph failed")
+ progress.finish()
+
+ assert progress._step == 2
+ assert [record[1] for record in progress._records] == ["reuse graph", "build graph"]
+
+
+def test_resolve_local_cache_plan(tmp_path):
+ local_root = _memory_group()
+ enabled, base, remove = GraphDataStore._resolve_local_cache_plan(
+ "/tmp/local.zarr", local_root, "auto"
+ )
+ assert enabled is False
+
+ enabled, base, remove = GraphDataStore._resolve_local_cache_plan(
+ "s3://bucket/path", local_root, False
+ )
+ assert enabled is False
+
+ enabled, base, remove = GraphDataStore._resolve_local_cache_plan(
+ "s3://bucket/path", local_root, str(tmp_path / "cache")
+ )
+ assert enabled is True
+ assert base == str(tmp_path / "cache")
+ assert remove is False
+
+
+def test_stage_normed_data_skips_repeat_copy(toy_crdir_ds, tmp_path, monkeypatch):
+ rna = toy_crdir_ds.RNA
+ cell_idx = np.arange(rna.cells.N)
+ feat_idx = np.array([0, 1, 3])
+ loc = "stage_src"
+ rna.z.create_group(loc, overwrite=True)
+ from scarf.writers import write_renorm_subset_to_zarr
+
+ write_renorm_subset_to_zarr(
+ rna, cell_idx, feat_idx, rna.z, f"{loc}/data", rna.nthreads
+ )
+ remote = rna.z[f"{loc}/data"]
+ subset_params = {"log_transform": False, "renormalize_subset": True}
+ calls = {"n": 0}
+ orig_copy = copy_zarr_array
+
+ def counting_copy(*args, **kwargs):
+ calls["n"] += 1
+ return orig_copy(*args, **kwargs)
+
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.copy_zarr_array", counting_copy
+ )
+ store = GraphDataStore.__new__(GraphDataStore)
+ store.nthreads = rna.nthreads
+ cache_base = str(tmp_path / "scratch")
+ store._stage_normed_data(remote, "hash1", subset_params, cache_base)
+ store._stage_normed_data(remote, "hash1", subset_params, cache_base)
+ assert calls["n"] == 1
+
+
+def test_stage_normed_data_recopies_when_subset_params_change(
+ toy_crdir_ds, tmp_path, monkeypatch
+):
+ rna = toy_crdir_ds.RNA
+ cell_idx = np.arange(rna.cells.N)
+ feat_idx = np.array([0, 1, 3])
+ loc = "stage_src_params"
+ rna.z.create_group(loc, overwrite=True)
+ from scarf.writers import write_renorm_subset_to_zarr
+
+ write_renorm_subset_to_zarr(
+ rna, cell_idx, feat_idx, rna.z, f"{loc}/data", rna.nthreads
+ )
+ remote = rna.z[f"{loc}/data"]
+ calls = {"n": 0}
+ orig_copy = copy_zarr_array
+
+ def counting_copy(*args, **kwargs):
+ calls["n"] += 1
+ return orig_copy(*args, **kwargs)
+
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.copy_zarr_array", counting_copy
+ )
+ store = GraphDataStore.__new__(GraphDataStore)
+ store.nthreads = rna.nthreads
+ cache_base = str(tmp_path / "scratch_params")
+ store._stage_normed_data(
+ remote,
+ "hash1",
+ {"log_transform": False, "renormalize_subset": True},
+ cache_base,
+ )
+ store._stage_normed_data(
+ remote, "hash1", {"log_transform": True, "renormalize_subset": True}, cache_base
+ )
+ assert calls["n"] == 2
+
+
+def test_mirror_write_lets_staging_skip_copy(toy_crdir_ds, tmp_path, monkeypatch):
+ from scarf.assay import Assay
+ from scarf.writers import write_renorm_subset_to_zarr
+
+ rna = toy_crdir_ds.RNA
+ cell_idx = np.arange(rna.cells.N)
+ feat_idx = np.array([0, 1, 3])
+ subset_hash = "mirror_hash"
+ subset_params = {"log_transform": False, "renormalize_subset": True}
+
+ store = GraphDataStore.__new__(GraphDataStore)
+ store.nthreads = rna.nthreads
+ cache_base = str(tmp_path / "cache")
+ cache_key = store._normed_cache_key(subset_hash, subset_params)
+ cache_path = os.path.join(cache_base, cache_key, "normed.zarr")
+ os.makedirs(os.path.dirname(cache_path), exist_ok=True)
+ mirror = create_or_open_staged_normed_array(
+ cache_path, (len(cell_idx), len(feat_idx))
+ )
+
+ loc = "mirror_src"
+ rna.z.create_group(loc, overwrite=True)
+ write_renorm_subset_to_zarr(
+ rna, cell_idx, feat_idx, rna.z, f"{loc}/data", rna.nthreads, mirror=mirror
+ )
+ remote = rna.z[f"{loc}/data"]
+ Assay._finalize_staged_mirror(mirror, subset_hash, subset_params)
+
+ assert mirror.attrs["staged_complete"] is True
+ assert mirror.attrs["staged_subset_hash"] == subset_hash
+ assert np.array_equal(np.asarray(mirror[:]), np.asarray(remote[:]))
+
+ calls = {"n": 0}
+ orig_copy = copy_zarr_array
+
+ def counting_copy(*args, **kwargs):
+ calls["n"] += 1
+ return orig_copy(*args, **kwargs)
+
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.copy_zarr_array", counting_copy
+ )
+ staged = store._stage_normed_data(remote, subset_hash, subset_params, cache_base)
+ assert calls["n"] == 0
+ assert np.array_equal(np.asarray(staged.compute()), np.asarray(remote[:]))
diff --git a/tests/test_graph_coverage.py b/tests/test_graph_coverage.py
new file mode 100644
index 00000000..b36a423a
--- /dev/null
+++ b/tests/test_graph_coverage.py
@@ -0,0 +1,1102 @@
+import shutil
+import sys
+from pathlib import Path
+from unittest.mock import Mock
+
+import numpy as np
+import pytest
+import zarr
+from scipy.sparse import csr_matrix
+from zarr.storage import MemoryStore
+
+from scarf.assay import ATACassay
+from scarf.datastore.datastore import DataStore
+from scarf.datastore.graph_datastore import GraphDataStore
+
+
+class _MemoryGraphStore(GraphDataStore):
+ @property
+ def assay_names(self) -> list[str]:
+ return self._assay_names
+
+
+@pytest.fixture
+def isolated_toy_datastore(toy_crdir_writer: str, tmp_path: Path) -> DataStore:
+ zarr_path = tmp_path / "toy.zarr"
+ shutil.copytree(toy_crdir_writer, zarr_path)
+ return DataStore(
+ str(zarr_path),
+ default_assay="RNA",
+ min_features_per_cell=0,
+ min_cells_per_feature=0,
+ nthreads=1,
+ )
+
+
+def _memory_graph_store(
+ assay_names: list[str] | None = None,
+) -> _MemoryGraphStore:
+ store = _MemoryGraphStore.__new__(_MemoryGraphStore)
+ store.z = zarr.open_group(store=MemoryStore(), mode="w")
+ store.workspace = None
+ store.zarr_mode = "r+"
+ store._defaultAssay = "RNA"
+ store._assay_names = assay_names or []
+ store._integratedGraphsLoc = "integratedGraphs"
+ store._cachedMagicOperator = None
+ store._cachedMagicOperatorLoc = None
+ store.nthreads = 1
+ return store
+
+
+def _add_test_graph(store: _MemoryGraphStore, label: str = "graph") -> str:
+ graph_loc = f"integratedGraphs/{label}"
+ graph_group = store.zw.create_group(graph_loc)
+ graph_group.attrs["n_cells"] = 3
+ graph_group.attrs["n_neighbors"] = 2
+ graph_group.create_array(
+ "edges",
+ data=np.array(
+ [
+ [0, 1],
+ [0, 2],
+ [1, 0],
+ [1, 2],
+ [2, 0],
+ [2, 1],
+ ],
+ dtype=np.uint64,
+ ),
+ )
+ graph_group.create_array(
+ "weights",
+ data=np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6]),
+ )
+ return graph_loc
+
+
+@pytest.mark.parametrize(
+ ("symmetric", "upper_only", "use_k", "expected"),
+ [
+ (
+ False,
+ False,
+ None,
+ np.array(
+ [
+ [0.0, 0.1, 0.2],
+ [0.3, 0.0, 0.4],
+ [0.5, 0.6, 0.0],
+ ]
+ ),
+ ),
+ (
+ False,
+ True,
+ 1,
+ np.array(
+ [
+ [0.0, 0.1, 0.0],
+ [0.3, 0.0, 0.0],
+ [0.5, 0.0, 0.0],
+ ]
+ ),
+ ),
+ (
+ False,
+ False,
+ 0,
+ np.array(
+ [
+ [0.0, 0.1, 0.0],
+ [0.3, 0.0, 0.0],
+ [0.5, 0.0, 0.0],
+ ]
+ ),
+ ),
+ (
+ False,
+ False,
+ 99,
+ np.array(
+ [
+ [0.0, 0.1, 0.2],
+ [0.3, 0.0, 0.4],
+ [0.5, 0.6, 0.0],
+ ]
+ ),
+ ),
+ (
+ True,
+ False,
+ None,
+ np.array(
+ [
+ [0.0, 0.37, 0.6],
+ [0.37, 0.0, 0.76],
+ [0.6, 0.76, 0.0],
+ ]
+ ),
+ ),
+ (
+ True,
+ True,
+ None,
+ np.array(
+ [
+ [0.0, 0.37, 0.6],
+ [0.0, 0.0, 0.76],
+ [0.0, 0.0, 0.0],
+ ]
+ ),
+ ),
+ ],
+)
+def test_load_graph_option_matrix(
+ symmetric: bool,
+ upper_only: bool,
+ use_k: int | None,
+ expected: np.ndarray,
+) -> None:
+ store = _memory_graph_store()
+ graph_loc = _add_test_graph(store)
+
+ graph = store.load_graph(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="I",
+ symmetric=symmetric,
+ upper_only=upper_only,
+ use_k=use_k,
+ graph_loc=graph_loc,
+ )
+
+ np.testing.assert_allclose(graph.toarray(), expected)
+
+
+def test_load_graph_latest_location_formats_and_errors() -> None:
+ store = _memory_graph_store()
+ graph_loc = _add_test_graph(store)
+ normed_loc = "RNA/normed__I__I"
+ reduction_loc = f"{normed_loc}/reduction__pca__2__I"
+ ann_loc = f"{reduction_loc}/ann__l2__50__50__48__1"
+ knn_loc = f"{ann_loc}/knn__2"
+
+ normed_group = store.zw.create_group(normed_loc)
+ reduction_group = store.zw.create_group(reduction_loc)
+ ann_group = store.zw.create_group(ann_loc)
+ knn_group = store.zw.create_group(knn_loc)
+ normed_group.attrs["latest_reduction"] = reduction_loc
+ reduction_group.attrs["latest_ann"] = ann_loc
+ ann_group.attrs["latest_knn"] = knn_loc
+ knn_group.attrs["latest_graph"] = graph_loc
+
+ store._get_latest_cell_key = Mock(return_value="I")
+ store._get_latest_feat_key = Mock(return_value="I")
+ graph = store.load_graph()
+ np.testing.assert_allclose(
+ graph.toarray(),
+ np.array(
+ [
+ [0.0, 0.1, 0.2],
+ [0.3, 0.0, 0.4],
+ [0.5, 0.6, 0.0],
+ ]
+ ),
+ )
+ store._get_latest_cell_key.assert_called_once_with("RNA")
+ store._get_latest_feat_key.assert_called_once_with("RNA")
+
+ n_cells, graph_coo = store._store_to_sparse(graph_loc, sparse_format="coo", use_k=1)
+ assert n_cells == 3
+ assert graph_coo.getformat() == "coo"
+ assert graph_coo.nnz == 3
+
+ with pytest.raises(ValueError, match="not found in zarr location"):
+ store.load_graph(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="I",
+ graph_loc="integratedGraphs/missing",
+ )
+
+ empty_store = _memory_graph_store()
+ with pytest.raises(KeyError):
+ empty_store.load_graph(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="I",
+ )
+
+
+def test_set_graph_params_defaults_and_reduction_validation(
+ isolated_toy_datastore: DataStore,
+) -> None:
+ store = isolated_toy_datastore
+
+ params = store._set_graph_params(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="coverageFresh",
+ reduction_method="AUTO",
+ )
+ assert params == (
+ True,
+ True,
+ "pca",
+ 11,
+ "I",
+ "l2",
+ 50,
+ 50,
+ 48,
+ 4466,
+ 11,
+ 1000,
+ 1.0,
+ 1.5,
+ )
+
+ assert store._choose_reduction_method(store.RNA, "PCA") == "pca"
+ atac_assay = ATACassay.__new__(ATACassay)
+ assert store._choose_reduction_method(atac_assay, "auto") == "lsi"
+ assert store._choose_reduction_method(store.RNA, "custom") == "custom"
+
+ with pytest.raises(ValueError, match="Please choose"):
+ store._choose_reduction_method(store.RNA, "invalid")
+
+ with pytest.raises(ValueError, match="does not exist"):
+ store._set_graph_params(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="coverageFresh",
+ reduction_method="pca",
+ pca_cell_key="missing",
+ )
+
+ with pytest.raises(TypeError, match="should be `bool`"):
+ store._set_graph_params(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="coverageFresh",
+ reduction_method="pca",
+ pca_cell_key="ids",
+ )
+
+
+def test_set_graph_params_reuses_cached_hierarchy(
+ isolated_toy_datastore: DataStore,
+) -> None:
+ store = isolated_toy_datastore
+ normed_loc = "RNA/normed__I__coverageCached"
+ reduction_loc = f"{normed_loc}/reduction__pca__7__I"
+ ann_loc = f"{reduction_loc}/ann__cosine__70__60__32__123"
+ knn_loc = f"{ann_loc}/knn__5"
+ kmeans_loc = f"{reduction_loc}/kmeans__9__123"
+ graph_loc = f"{knn_loc}/graph__2.0__3.0"
+
+ normed_group = store.zw.create_group(normed_loc)
+ reduction_group = store.zw.create_group(reduction_loc)
+ ann_group = store.zw.create_group(ann_loc)
+ knn_group = store.zw.create_group(knn_loc)
+ store.zw.create_group(kmeans_loc)
+ store.zw.create_group(graph_loc)
+ normed_group.attrs["subset_params"] = {
+ "log_transform": False,
+ "renormalize_subset": False,
+ }
+ normed_group.attrs["latest_reduction"] = reduction_loc
+ reduction_group.attrs["latest_ann"] = ann_loc
+ reduction_group.attrs["latest_kmeans"] = kmeans_loc
+ ann_group.attrs["latest_knn"] = knn_loc
+ knn_group.attrs["latest_graph"] = graph_loc
+
+ params = store._set_graph_params(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="coverageCached",
+ reduction_method="pca",
+ )
+
+ assert params == (
+ False,
+ False,
+ "pca",
+ 7,
+ "I",
+ "cosine",
+ 70,
+ 60,
+ 32,
+ 123,
+ 5,
+ 9,
+ 2.0,
+ 3.0,
+ )
+
+
+def test_set_graph_params_uses_missing_metadata_fallbacks(
+ isolated_toy_datastore: DataStore,
+) -> None:
+ store = isolated_toy_datastore
+ empty_normed_loc = "RNA/normed__I__coverageEmpty"
+ store.zw.create_group(empty_normed_loc)
+
+ defaults = store._set_graph_params(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="coverageEmpty",
+ reduction_method="pca",
+ )
+ assert defaults == (
+ True,
+ True,
+ "pca",
+ 11,
+ "I",
+ "l2",
+ 50,
+ 50,
+ 48,
+ 4466,
+ 11,
+ 1000,
+ 1.0,
+ 1.5,
+ )
+
+ normed_loc = "RNA/normed__I__coveragePartial"
+ reduction_loc = f"{normed_loc}/reduction__pca__6__I"
+ ann_loc = f"{reduction_loc}/ann__l2__50__50__48__4466"
+ knn_loc = f"{ann_loc}/knn__11"
+ normed_group = store.zw.create_group(normed_loc)
+ normed_group.attrs["subset_params"] = {
+ "log_transform": True,
+ "renormalize_subset": True,
+ }
+ normed_group.attrs["latest_reduction"] = reduction_loc
+ store.zw.create_group(reduction_loc)
+ store.zw.create_group(knn_loc)
+
+ fallbacks = store._set_graph_params(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="coveragePartial",
+ reduction_method="pca",
+ )
+ assert fallbacks == (
+ True,
+ True,
+ "pca",
+ 6,
+ "I",
+ "l2",
+ 50,
+ 50,
+ 48,
+ 4466,
+ 11,
+ 1000,
+ 1.0,
+ 1.5,
+ )
+
+
+def test_latest_knn_and_ann_stream_cache_paths(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ store = _memory_graph_store(["RNA"])
+ assay_group = store.zw.create_group("RNA")
+ assay_group.attrs["latest_cell_key"] = "I"
+ assay_group.attrs["latest_feat_key"] = "I"
+ normed_loc = "RNA/normed__I__I"
+ reduction_loc = f"{normed_loc}/reduction__pca__2__I"
+ ann_loc = f"{reduction_loc}/ann__l2__50__50__48__1"
+ knn_loc = f"{ann_loc}/knn__2"
+ normed_group = store.zw.create_group(normed_loc)
+ reduction_group = store.zw.create_group(reduction_loc)
+ ann_group = store.zw.create_group(ann_loc)
+ store.zw.create_group(knn_loc)
+ normed_group.attrs["latest_reduction"] = reduction_loc
+ reduction_group.attrs["latest_ann"] = ann_loc
+ ann_group.attrs["latest_knn"] = knn_loc
+ reduction_group.create_array("reduction", data=np.eye(2))
+ normed_group.create_array("data", data=np.eye(3, 2))
+ store._load_default_assay = Mock(return_value="RNA")
+
+ assert store._get_latest_knn_loc() == knn_loc
+ store._load_default_assay.assert_called_once_with()
+
+ with pytest.raises(ValueError, match="Assay missing does not exist"):
+ store._get_latest_knn_loc("missing")
+
+ del reduction_group["reduction"]
+ with pytest.raises(ValueError, match="PCA Reduction not found"):
+ store._get_latest_knn_loc("RNA")
+ reduction_group.create_array("reduction", data=np.eye(2))
+
+ assert store._has_ann_stream_cache("RNA", "I", "I") is True
+ assert store._has_ann_stream_cache("RNA", "I", "I", knn_loc=knn_loc) is True
+ assert store._has_ann_stream_cache("RNA", "I", "missing") is False
+ assert (
+ store._has_ann_stream_cache("RNA", "I", "I", knn_loc=f"{ann_loc}/knn__99")
+ is False
+ )
+
+ store.zw.create_group("RNA/normed__I__broken")
+ assert store._has_ann_stream_cache("RNA", "I", "broken") is False
+
+ legacy_path = tmp_path / "ann_idx"
+ legacy_path.write_bytes(b"index")
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.legacy_ann_index_path",
+ lambda *_: str(legacy_path),
+ )
+ assert (
+ store._ann_stream_recoverable(
+ "missing/ann", "missing/reduction", "missing/normed"
+ )
+ is True
+ )
+
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.legacy_ann_index_path",
+ lambda *_: None,
+ )
+ assert (
+ store._ann_stream_recoverable(
+ "missing/ann", "missing/reduction", "missing/normed"
+ )
+ is False
+ )
+
+
+def test_ann_index_resolution_and_persistence_paths(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ store = _memory_graph_store()
+ ann_loc = "RNA/normed/reduction/ann__l2__50__50__48__1"
+ ann_group = store.zw.create_group(ann_loc)
+ ann_group.create_array("ann_idx_bytes", data=np.array([1, 2, 3], dtype=np.uint8))
+ stored_index = object()
+ load_stored = Mock(return_value=stored_index)
+ monkeypatch.setattr("scarf.datastore.graph_datastore.load_ann_index", load_stored)
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.legacy_ann_index_path",
+ lambda *_: None,
+ )
+
+ assert store._resolve_ann_index(ann_loc, "l2", 3) is stored_index
+ load_stored.assert_called_once_with(ann_group, "l2", 3)
+
+ del ann_group["ann_idx_bytes"]
+ custom_path = tmp_path / "custom.idx"
+ legacy_path = tmp_path / "legacy.idx"
+ custom_path.write_bytes(b"custom")
+ legacy_path.write_bytes(b"legacy")
+ custom_index = object()
+ legacy_index = object()
+ load_path = Mock(
+ side_effect=lambda path, *_: (
+ custom_index if path == str(custom_path) else legacy_index
+ )
+ )
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.load_ann_index_from_path",
+ load_path,
+ )
+
+ assert (
+ store._resolve_ann_index(
+ ann_loc,
+ "cosine",
+ 4,
+ ann_index_fetcher=lambda _: str(custom_path),
+ )
+ is custom_index
+ )
+
+ failing_fetcher = Mock(side_effect=RuntimeError("fetch failed"))
+ assert (
+ store._resolve_ann_index(
+ ann_loc,
+ "l2",
+ 3,
+ ann_index_fetcher=failing_fetcher,
+ )
+ is None
+ )
+
+ save_index = Mock()
+ monkeypatch.setattr("scarf.datastore.graph_datastore.save_ann_index", save_index)
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.legacy_ann_index_path",
+ lambda *_: str(legacy_path),
+ )
+ assert store._resolve_ann_index(ann_loc, "l2", 3) is legacy_index
+ save_index.assert_called_once_with(ann_group, legacy_index)
+
+ custom_saver = Mock()
+ custom_only_loc = "custom/ann"
+ store._persist_ann_index(
+ custom_only_loc, custom_index, ann_index_saver=custom_saver
+ )
+ custom_saver.assert_called_once_with(custom_index, custom_only_loc)
+ assert custom_only_loc not in store.zw
+
+ fallback_loc = "fallback/ann"
+ failing_saver = Mock(side_effect=RuntimeError("save failed"))
+ store._persist_ann_index(fallback_loc, legacy_index, ann_index_saver=failing_saver)
+ assert fallback_loc in store.zw
+ save_index.assert_called_with(store.zw[fallback_loc], legacy_index)
+
+ store.zarr_mode = "r"
+ read_only_loc = "readonly/ann"
+ calls_before = save_index.call_count
+ store._persist_ann_index(read_only_loc, object())
+ assert read_only_loc in store.zw
+ assert save_index.call_count == calls_before
+
+
+def test_normalized_cache_validation_paths(
+ isolated_toy_datastore: DataStore,
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ assay = isolated_toy_datastore.RNA
+ location = "coverageNormedCache"
+ group = assay.z.create_group(location)
+ cell_idx, feat_idx = assay._get_cell_feat_idx("I", "I")
+ subset_hash = assay._create_subset_hash(cell_idx, feat_idx)
+ subset_params = {
+ "log_transform": False,
+ "renormalize_subset": True,
+ }
+ group.attrs["subset_hash"] = subset_hash
+ group.attrs["subset_params"] = subset_params
+
+ assert (
+ GraphDataStore._normed_data_cached(assay, "I", "I", location, False, True)
+ is True
+ )
+ group.attrs["subset_params"] = {
+ "log_transform": True,
+ "renormalize_subset": True,
+ }
+ assert (
+ GraphDataStore._normed_data_cached(assay, "I", "I", location, False, True)
+ is False
+ )
+ del group.attrs["subset_hash"]
+ assert (
+ GraphDataStore._normed_data_cached(assay, "I", "I", location, False, True)
+ is False
+ )
+
+ missing_path = tmp_path / "missing.zarr"
+ assert (
+ GraphDataStore._staged_normed_cached(
+ str(missing_path), subset_hash, subset_params
+ )
+ is False
+ )
+
+ staged_path = tmp_path / "staged.zarr"
+ staged_root = zarr.open_group(str(staged_path), mode="w")
+ assert (
+ GraphDataStore._staged_normed_cached(
+ str(staged_path), subset_hash, subset_params
+ )
+ is False
+ )
+ staged = staged_root.create_array("data", data=np.eye(2))
+ staged.attrs["staged_subset_hash"] = subset_hash
+ staged.attrs["staged_subset_params"] = subset_params
+ staged.attrs["staged_complete"] = True
+ assert (
+ GraphDataStore._staged_normed_cached(
+ str(staged_path), subset_hash, subset_params
+ )
+ is True
+ )
+ staged.attrs["staged_complete"] = False
+ assert (
+ GraphDataStore._staged_normed_cached(
+ str(staged_path), subset_hash, subset_params
+ )
+ is False
+ )
+
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.zarr.open_group",
+ Mock(side_effect=RuntimeError("broken cache")),
+ )
+ assert (
+ GraphDataStore._staged_normed_cached(
+ str(staged_path), subset_hash, subset_params
+ )
+ is False
+ )
+
+
+def test_partial_normalization_statistics_cache_paths(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ store = _memory_graph_store()
+ data = Mock()
+ data.mean.return_value = np.array([2.0, 4.0])
+ data.std.return_value = np.array([1.5, 2.5])
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.show_dask_progress",
+ lambda values, *_: values,
+ )
+
+ missing_mu = store.zw.create_group("missingMu")
+ missing_mu.create_array("sigma", data=np.array([3.0, 5.0]))
+ mu, sigma = store._load_or_compute_norm_stats("missingMu", data, "pca")
+ np.testing.assert_allclose(mu, [2.0, 4.0])
+ np.testing.assert_allclose(sigma, [3.0, 5.0])
+ np.testing.assert_allclose(missing_mu["mu"][:], [2.0, 4.0])
+
+ missing_sigma = store.zw.create_group("missingSigma")
+ missing_sigma.create_array("mu", data=np.array([6.0, 8.0]))
+ mu, sigma = store._load_or_compute_norm_stats("missingSigma", data, "pca")
+ np.testing.assert_allclose(mu, [6.0, 8.0])
+ np.testing.assert_allclose(sigma, [1.5, 2.5])
+ np.testing.assert_allclose(missing_sigma["sigma"][:], [1.5, 2.5])
+
+ store.zarr_mode = "r"
+ read_only_mu = store.zw.create_group("readOnlyMu")
+ read_only_mu.create_array("sigma", data=np.array([3.0, 5.0]))
+ mu, sigma = store._load_or_compute_norm_stats("readOnlyMu", data, "pca")
+ np.testing.assert_allclose(mu, [2.0, 4.0])
+ np.testing.assert_allclose(sigma, [3.0, 5.0])
+ assert "mu" not in read_only_mu
+
+ read_only_sigma = store.zw.create_group("readOnlySigma")
+ read_only_sigma.create_array("mu", data=np.array([6.0, 8.0]))
+ mu, sigma = store._load_or_compute_norm_stats("readOnlySigma", data, "pca")
+ np.testing.assert_allclose(mu, [6.0, 8.0])
+ np.testing.assert_allclose(sigma, [1.5, 2.5])
+ assert "sigma" not in read_only_sigma
+
+
+def test_remote_cache_plan_auto_and_invalid(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ store = _memory_graph_store()
+ monkeypatch.setattr(
+ "scarf.datastore.graph_datastore.is_remote_datastore",
+ lambda *_: True,
+ )
+
+ enabled, cache_path, remove = store._resolve_local_cache_plan(
+ "s3://bucket/store", store.z, "auto"
+ )
+ assert enabled is True
+ assert cache_path is not None
+ assert Path(cache_path).is_dir()
+ assert remove is True
+ shutil.rmtree(cache_path)
+
+ with pytest.raises(TypeError, match="local_cache must be"):
+ store._resolve_local_cache_plan("s3://bucket/store", store.z, object())
+
+
+def test_get_imputed_creates_loads_and_reuses_operator() -> None:
+ store = _memory_graph_store()
+ graph_loc = _add_test_graph(store)
+ values = np.array([1.0, 2.0, 4.0])
+ graph = csr_matrix(
+ np.array(
+ [
+ [0.0, 1.0, 1.0],
+ [1.0, 0.0, 1.0],
+ [1.0, 1.0, 0.0],
+ ]
+ )
+ )
+ store._get_latest_keys = Mock(return_value=("RNA", "I", "I"))
+ store.get_cell_vals = Mock(return_value=values)
+ store._get_latest_graph_loc = Mock(return_value=graph_loc)
+ store.load_graph = Mock(return_value=graph)
+
+ with pytest.raises(ValueError, match="name for the feature"):
+ store.get_imputed(feature_name=None)
+ store.get_cell_vals.assert_not_called()
+
+ first = store.get_imputed(feature_name="gene", t=1)
+ np.testing.assert_allclose(first, np.array([3.0, 2.5, 1.5]))
+ assert f"{graph_loc}/magic_1" in store.zw
+ assert store._cachedMagicOperatorLoc == f"{graph_loc}/magic_1"
+ assert store._cachedMagicOperator is not None
+ store.load_graph.assert_called_once_with(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="I",
+ symmetric=True,
+ upper_only=False,
+ )
+
+ cached_operator = store._cachedMagicOperator
+ second = store.get_imputed(feature_name="gene", t=1)
+ np.testing.assert_allclose(second, first)
+ assert store._cachedMagicOperator is cached_operator
+ assert store.load_graph.call_count == 1
+
+ store._cachedMagicOperator = None
+ store._cachedMagicOperatorLoc = None
+ loaded = store.get_imputed(feature_name="gene", t=1, cache_operator=True)
+ np.testing.assert_allclose(loaded, first)
+ assert store._cachedMagicOperator is not None
+ assert store._cachedMagicOperatorLoc == f"{graph_loc}/magic_1"
+
+ store._cachedMagicOperator = None
+ store._cachedMagicOperatorLoc = None
+ uncached = store.get_imputed(feature_name="gene", t=1, cache_operator=False)
+ np.testing.assert_allclose(uncached, first)
+ assert store._cachedMagicOperator is None
+ assert store._cachedMagicOperatorLoc is None
+
+ squared = store.get_imputed(feature_name="gene", t=2, cache_operator=False)
+ np.testing.assert_allclose(squared, np.array([2.0, 2.25, 2.75]))
+ assert f"{graph_loc}/magic_2" in store.zw
+ assert store.load_graph.call_count == 2
+ assert store._cachedMagicOperator is None
+ assert store._cachedMagicOperatorLoc is None
+
+
+def test_filter_cells_open_bounds_reset_and_boundaries(
+ isolated_toy_datastore: DataStore,
+) -> None:
+ store = isolated_toy_datastore
+ attr = "RNA_nCounts"
+ values = store.cells.fetch_all(attr)
+ lower = float(values.min())
+ upper = float(values.max())
+
+ store.cells.reset_key("I")
+ store.filter_cells(
+ attrs=[attr],
+ lows=[lower],
+ highs=[None],
+ reset_previous=True,
+ )
+ expected = values > lower
+ np.testing.assert_array_equal(store.cells.fetch_all("I"), expected)
+
+ store.filter_cells(
+ attrs=[attr],
+ lows=[None],
+ highs=[upper],
+ )
+ expected &= values < upper
+ np.testing.assert_array_equal(store.cells.fetch_all("I"), expected)
+
+ store.filter_cells(
+ attrs=[attr, "missing"],
+ lows=[None, None],
+ highs=[None, None],
+ reset_previous=True,
+ )
+ assert store.cells.fetch_all("I").all()
+
+ store.filter_cells(
+ attrs=[attr],
+ lows=[lower],
+ highs=[upper],
+ reset_previous=True,
+ keep_bounds=True,
+ )
+ assert store.cells.fetch_all("I").all()
+
+
+def test_run_marker_search_skip_save_and_errors(
+ isolated_toy_datastore: DataStore,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ store = isolated_toy_datastore
+ monkeypatch.setattr(store, "_get_latest_feat_key", lambda _: "I")
+ markers = {"cluster": object()}
+ finder = Mock(return_value=markers)
+ monkeypatch.setattr("scarf.markers.find_markers_by_rank", finder)
+ prenormed_group = store.zw.create_group("coveragePrenormed")
+
+ with pytest.raises(ValueError, match="group_key"):
+ store.run_marker_search(group_key=None)
+
+ result = store.run_marker_search(
+ group_key="ids",
+ prenormed_store="coveragePrenormed",
+ skip_save=True,
+ )
+ assert result is markers
+ first_call = finder.call_args.kwargs
+ assert first_call["assay"] is store.RNA
+ assert first_call["group_key"] == "ids"
+ assert first_call["cell_key"] == "I"
+ assert first_call["feat_key"] == "I"
+ assert first_call["batch_size"] >= 1
+ assert first_call["n_threads"] == store.nthreads
+ assert first_call["prenormed_store"].path == "coveragePrenormed"
+ assert "I__ids" not in store.zw["RNA/markers"]
+
+ finder.reset_mock()
+ result = store.run_marker_search(
+ group_key="ids",
+ cell_key="I",
+ feat_key="I",
+ gene_batch_size=2,
+ use_prenormed=True,
+ prenormed_store=prenormed_group,
+ n_threads=3,
+ skip_save=True,
+ log_transform=False,
+ )
+ assert result is markers
+ second_call = finder.call_args.kwargs
+ assert second_call["batch_size"] == 2
+ assert second_call["use_prenormed"] is True
+ assert second_call["prenormed_store"] is prenormed_group
+ assert second_call["n_threads"] == 3
+ assert second_call["log_transform"] is False
+
+ with pytest.raises(KeyError):
+ store.run_marker_search(
+ group_key="ids",
+ prenormed_store="missing",
+ skip_save=True,
+ )
+
+ finder.side_effect = RuntimeError("marker failure")
+ with pytest.raises(RuntimeError, match="marker failure"):
+ store.run_marker_search(group_key="ids", skip_save=True)
+
+
+def test_make_graph_harmony_input_validation(
+ isolated_toy_datastore: DataStore,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ store = isolated_toy_datastore
+ graph_params = (
+ True,
+ True,
+ "pca",
+ 2,
+ "I",
+ "l2",
+ 50,
+ 50,
+ 48,
+ 1,
+ 2,
+ 2,
+ 1.0,
+ 1.5,
+ )
+ set_params = Mock(return_value=graph_params)
+ monkeypatch.setattr(store, "_set_graph_params", set_params)
+
+ with pytest.raises(ValueError, match="no batches provided"):
+ store.make_graph(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="I",
+ harmonize=True,
+ batch_columns=None,
+ )
+
+ with pytest.raises(ValueError, match="batches must be a list"):
+ store.make_graph(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="I",
+ harmonize=True,
+ batch_columns="ids",
+ )
+
+ assert set_params.call_count == 2
+
+
+def test_run_tsne_orchestration_and_error_paths(
+ isolated_toy_datastore: DataStore,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ store = isolated_toy_datastore
+ graph = csr_matrix(
+ np.array(
+ [
+ [0.0, 1.0, 1.0],
+ [1.0, 0.0, 1.0],
+ [1.0, 1.0, 0.0],
+ ]
+ )
+ )
+ initial = np.array(
+ [
+ [0.0, 0.0],
+ [0.5, 0.5],
+ [1.0, 1.0],
+ ]
+ )
+ embedding = np.array(
+ [
+ [1.0, 2.0, 3.0],
+ [4.0, 5.0, 6.0],
+ ]
+ )
+ latest_keys = Mock(return_value=("RNA", "I", "I"))
+ load_graph = Mock(return_value=graph)
+ get_initial = Mock(return_value=initial)
+ runner = Mock(return_value=embedding)
+ monkeypatch.setattr(store, "_get_latest_keys", latest_keys)
+ monkeypatch.setattr(store, "load_graph", load_graph)
+ monkeypatch.setattr(store, "_get_ini_embed", get_initial)
+ monkeypatch.setattr("scarf.knn_utils.run_sgtsne", runner)
+
+ store.run_tsne(
+ symmetric_graph=True,
+ graph_upper_only=True,
+ parallel=True,
+ nthreads=None,
+ max_iter=20,
+ label="coverageTsne",
+ )
+ get_initial.assert_called_once_with("RNA", "I", "I", 2)
+ first_call = runner.call_args
+ assert first_call.args[0] is graph
+ np.testing.assert_array_equal(first_call.args[1], initial)
+ assert first_call.kwargs["parallel"] is True
+ assert first_call.kwargs["nthreads"] == store.nthreads
+ assert first_call.kwargs["max_iter"] == 20
+ np.testing.assert_allclose(store.cells.fetch("RNA_coverageTsne1"), embedding[0])
+ np.testing.assert_allclose(store.cells.fetch("RNA_coverageTsne2"), embedding[1])
+
+ store.run_tsne(
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="I",
+ ini_embed=initial,
+ parallel=False,
+ label="serialTsne",
+ )
+ assert runner.call_args.kwargs["nthreads"] == 1
+ assert runner.call_args.kwargs["parallel"] is False
+
+ with pytest.raises(ValueError, match="required shape"):
+ store.run_tsne(
+ ini_embed=np.zeros((2, 2)),
+ tsne_dims=2,
+ )
+
+ runner.side_effect = FileNotFoundError("sgtsne missing")
+ store.run_tsne(
+ ini_embed=initial,
+ parallel=True,
+ nthreads=2,
+ label="missingTsne",
+ )
+ assert runner.call_args.kwargs["nthreads"] == 2
+
+ latest_calls = latest_keys.call_count
+ monkeypatch.setattr(sys, "platform", "win32")
+ assert store.run_tsne() is None
+ assert latest_keys.call_count == latest_calls
+
+
+def test_integrate_assays_snn_writes_and_overwrites_graph() -> None:
+ store = _memory_graph_store(["RNA", "ADT"])
+ graphs = {
+ "RNA": csr_matrix(
+ np.array(
+ [
+ [0.0, 1.0, 2.0],
+ [3.0, 0.0, 4.0],
+ [5.0, 6.0, 0.0],
+ ]
+ )
+ ),
+ "ADT": csr_matrix(
+ np.array(
+ [
+ [0.0, 7.0, 8.0],
+ [9.0, 0.0, 10.0],
+ [11.0, 12.0, 0.0],
+ ]
+ )
+ ),
+ }
+ load_graph = Mock(side_effect=lambda **kwargs: graphs[kwargs["from_assay"]])
+ store.load_graph = load_graph
+
+ store.integrate_assays(
+ assays=["RNA", "ADT"],
+ label="joint",
+ method="snn",
+ chunk_size=2,
+ )
+ store.integrate_assays(
+ assays=["RNA", "ADT"],
+ label="joint",
+ method="snn",
+ chunk_size=2,
+ )
+
+ integrated_group = store.zw["integratedGraphs/joint"]
+ assert integrated_group.attrs["n_cells"] == 3
+ assert integrated_group.attrs["n_neighbors"] == 2
+ assert integrated_group["edges"].shape == (6, 2)
+ assert integrated_group["weights"].shape == (6,)
+ assert load_graph.call_count == 4
+
+ integrated = GraphDataStore.load_graph(
+ store,
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="I",
+ graph_loc="integratedGraphs/joint",
+ )
+ assert integrated.shape == (3, 3)
+ np.testing.assert_array_equal(np.diff(integrated.indptr), [2, 2, 2])
+ np.testing.assert_allclose(
+ np.sort(integrated.data),
+ np.array([7.0, 8.0, 9.0, 10.0, 11.0, 12.0]),
+ )
+
+
+def test_integrate_assays_validation_errors() -> None:
+ store = _memory_graph_store(["RNA", "ADT"])
+ graph = csr_matrix(
+ np.array(
+ [
+ [0.0, 1.0, 1.0],
+ [1.0, 0.0, 1.0],
+ [1.0, 1.0, 0.0],
+ ]
+ )
+ )
+ store.load_graph = Mock(return_value=graph)
+
+ with pytest.raises(ValueError, match="missing was not found"):
+ store.integrate_assays(
+ assays=["RNA", "missing"],
+ label="invalid",
+ method="snn",
+ )
+
+ with pytest.raises(ValueError, match="only two assays"):
+ store.integrate_assays(
+ assays=["RNA"],
+ label="invalid",
+ method="wnn",
+ )
+
+ with pytest.raises(ValueError, match="Method unknown not supported"):
+ store.integrate_assays(
+ assays=["RNA", "ADT"],
+ label="invalid",
+ method="unknown",
+ )
diff --git a/tests/test_knn_utils.py b/tests/test_knn_utils.py
new file mode 100644
index 00000000..f1e0b5db
--- /dev/null
+++ b/tests/test_knn_utils.py
@@ -0,0 +1,347 @@
+import warnings
+
+import numpy as np
+import pytest
+import zarr
+from scipy.sparse import coo_matrix, csr_matrix
+from zarr.storage import MemoryStore
+
+from scarf.knn_utils import (
+ _patch_null_weights,
+ calc_snn,
+ merge_graphs,
+ self_query_knn,
+ smoothen_dists,
+ weight_sort_indices,
+ wnn_integration,
+)
+from scarf.utils import logger
+from scarf.writers import create_zarr_dataset
+
+
+def _simple_knn_graph(n: int, k: int = 3) -> csr_matrix:
+ rows, cols, data = [], [], []
+ for i in range(n):
+ for j in range(1, k + 1):
+ neighbor = (i + j) % n
+ rows.append(i)
+ cols.append(neighbor)
+ data.append(float(j))
+ return csr_matrix((data, (rows, cols)), shape=(n, n))
+
+
+def _grouped_knn_graph(groups: list[list[int]]) -> csr_matrix:
+ n_cells = sum(len(group) for group in groups)
+ rows = []
+ cols = []
+ for group in groups:
+ for cell in group:
+ neighbors = [neighbor for neighbor in group if neighbor != cell]
+ rows.extend([cell] * len(neighbors))
+ cols.extend(neighbors)
+ return csr_matrix((np.ones(len(rows)), (rows, cols)), shape=(n_cells, n_cells))
+
+
+def _multimodal_wnn_inputs() -> tuple[csr_matrix, np.ndarray, csr_matrix, np.ndarray]:
+ g1 = _grouped_knn_graph([[0, 1, 2, 3], [4, 5, 6, 7]])
+ g2 = _grouped_knn_graph([[0, 2, 4, 6], [1, 3, 5, 7]])
+ ld1 = np.array(
+ [0, 101, -97, 2, 1e6, 1e6 + 101, 1e6 - 97, 1e6 + 2],
+ dtype=np.float64,
+ ).reshape(-1, 1)
+ ld2 = np.array(
+ [0, 1e6, 101, 1e6 + 101, -97, 1e6 - 97, 2, 1e6 + 2],
+ dtype=np.float64,
+ ).reshape(-1, 1)
+ return g1, ld1, g2, ld2
+
+
+class _BlockSource:
+ def __init__(self, blocks: list[np.ndarray]):
+ self.blocks = blocks
+ self.numblocks = (len(blocks), 1)
+ self.shape = (sum(len(block) for block in blocks), blocks[0].shape[1])
+
+
+class _SyntheticAnn:
+ def __init__(
+ self,
+ data: _BlockSource,
+ embeddings: np.ndarray | None,
+ harmonized_data: _BlockSource | None,
+ ):
+ self.data = data
+ self.embeddings = embeddings
+ self.harmonizedData = harmonized_data
+ self.nCells = data.shape[0]
+ self.k = 2
+ self.batchSize = 2
+ self.queries: list[tuple[np.ndarray, np.ndarray]] = []
+
+ @staticmethod
+ def reducer(values: np.ndarray) -> np.ndarray:
+ return np.asarray(values) + 10
+
+ def transform_ann(
+ self,
+ values: np.ndarray,
+ k: int,
+ self_indices: np.ndarray,
+ ) -> tuple[np.ndarray, np.ndarray, int]:
+ self.queries.append((np.asarray(values).copy(), self_indices.copy()))
+ offsets = np.arange(1, k + 1)
+ indices = (self_indices[:, None] + offsets) % self.nCells
+ distances = np.broadcast_to(offsets, indices.shape).astype(np.float64)
+ missed_self = int(np.count_nonzero(self_indices % 2))
+ return indices, distances, missed_self
+
+
+def test_self_query_knn_uses_cached_embeddings_and_writes_graph():
+ raw = np.arange(10, dtype=np.float64).reshape(5, 2)
+ data = _BlockSource([raw[:2], raw[2:4], raw[4:]])
+ embeddings = raw + 100
+ ann = _SyntheticAnn(data, embeddings=embeddings, harmonized_data=None)
+ store = zarr.open_group(store=MemoryStore(), mode="w")
+
+ recall = self_query_knn(ann, store, chunk_size=2, nthreads=1)
+
+ expected_indices = (np.arange(5)[:, None] + np.array([1, 2], dtype=np.int64)) % 5
+ assert recall == pytest.approx(60.0)
+ np.testing.assert_array_equal(store["indices"][:], expected_indices)
+ np.testing.assert_allclose(store["distances"][:], [[1.0, 2.0]] * 5)
+ np.testing.assert_array_equal(
+ np.vstack([values for values, _ in ann.queries]),
+ embeddings,
+ )
+ np.testing.assert_array_equal(
+ np.concatenate([indices for _, indices in ann.queries]),
+ np.arange(5),
+ )
+
+
+@pytest.mark.parametrize("use_harmonized", [False, True])
+def test_self_query_knn_streams_reduced_or_harmonized_blocks(use_harmonized):
+ raw = np.arange(10, dtype=np.float64).reshape(5, 2)
+ data = _BlockSource([raw[:2], raw[2:4], raw[4:]])
+ harmonized_values = raw + 100
+ harmonized_data = (
+ _BlockSource(
+ [
+ harmonized_values[:2],
+ harmonized_values[2:4],
+ harmonized_values[4:],
+ ]
+ )
+ if use_harmonized
+ else None
+ )
+ ann = _SyntheticAnn(
+ data,
+ embeddings=None,
+ harmonized_data=harmonized_data,
+ )
+ store = zarr.open_group(store=MemoryStore(), mode="w")
+
+ recall = self_query_knn(ann, store, chunk_size=3, nthreads=1)
+
+ expected_queries = harmonized_values if use_harmonized else raw + 10
+ assert recall == pytest.approx(60.0)
+ np.testing.assert_array_equal(
+ np.vstack([values for values, _ in ann.queries]),
+ expected_queries,
+ )
+ assert store["indices"].shape == (5, 2)
+ assert store["distances"].shape == (5, 2)
+
+
+def test_calc_snn_returns_normalized_overlap():
+ graph = _simple_knn_graph(6, k=3)
+ indices = graph.indices.reshape((6, 3))
+ snn = calc_snn(indices)
+ assert snn.shape == (6, 3)
+ assert np.all(snn >= 0)
+ assert np.all(snn <= 1)
+
+
+def test_weight_sort_indices_keeps_top_neighbors():
+ indices = np.array([4, 1, 2, 1, 3])
+ weights = np.array([0.2, 0.5, 0.4, 0.6, 0.1])
+ sort_weights = weights + np.array([0.0, 0.2, 0.1, 0.2, 0.0])
+ kept_idx, kept_w = weight_sort_indices(indices, weights, sort_weights, n=3)
+ assert len(kept_idx) == 3
+ assert len(kept_w) == 3
+ assert len(set(kept_idx)) == len(kept_idx)
+
+
+def test_merge_graphs_preserves_shape_and_edge_count():
+ g1 = _simple_knn_graph(8, k=3)
+ g2 = _simple_knn_graph(8, k=3)
+ merged = merge_graphs([g1, g2])
+ assert isinstance(merged, coo_matrix)
+ assert merged.shape == g1.shape
+ assert merged.nnz == g1.nnz
+
+
+def test_merge_graphs_rejects_mismatched_shapes():
+ g1 = _simple_knn_graph(6, k=3)
+ g2 = _simple_knn_graph(8, k=3)
+ with pytest.raises(ValueError, match="same shape"):
+ merge_graphs([g1, g2])
+
+
+def test_patch_null_weights_matches_full_rewrite(tmp_path):
+ weights = np.array([0.0, 0.2, 0.0, 0.5, 0.0, 0.3], dtype=np.float64)
+ null_positions = np.flatnonzero(weights == 0).tolist()
+ fill = 0.15
+
+ expected = weights.copy()
+ expected[null_positions] = fill
+
+ root = zarr.open_group(str(tmp_path / "weights.zarr"), mode="w")
+ zgw = create_zarr_dataset(root, "weights", (2,), "f8", weights.shape)
+ zgw[:] = weights
+ _patch_null_weights(zgw, null_positions, fill, patch_chunk=2)
+ np.testing.assert_allclose(zgw[:], expected)
+
+
+def test_smoothen_dists_runs(tmp_path):
+ pytest.importorskip("umap")
+ n_cells, n_neighbors = 24, 5
+ chunk_size = 8
+ rng = np.random.default_rng(0)
+ dist = rng.random((n_cells, n_neighbors)).astype(np.float64)
+ dist[:, 0] = 0.0
+ idx = np.tile(np.arange(n_cells), (n_cells, 1)) % n_cells
+
+ root = zarr.open_group(str(tmp_path / "graph.zarr"), mode="w")
+ knn = root.create_group("knn")
+ z_idx = create_zarr_dataset(knn, "indices", (chunk_size,), "u8", idx.shape)
+ z_dist = create_zarr_dataset(knn, "distances", (chunk_size,), "f8", dist.shape)
+ z_idx[:] = idx
+ z_dist[:] = dist
+ graph = root.create_group("graph")
+ smoothen_dists(graph, z_idx, z_dist, lc=1.0, bw=1.5, chunk_size=chunk_size)
+ assert graph["weights"].shape[0] == graph["edges"].shape[0]
+ assert graph["weights"].shape[0] > 0
+
+
+def test_wnn_integration_handles_extreme_affinities_without_runtime_warnings():
+ g1, ld1, g2, ld2 = _multimodal_wnn_inputs()
+
+ with warnings.catch_warnings():
+ warnings.simplefilter("error", RuntimeWarning)
+ merged = wnn_integration("RNA", g1, ld1, "ADT", g2, ld2, n_threads=1)
+
+ assert isinstance(merged, coo_matrix)
+ assert merged.shape == g1.shape
+ assert merged.nnz == g1.nnz
+ np.testing.assert_array_equal(
+ np.bincount(merged.row, minlength=g1.shape[0]),
+ np.repeat(g1.getnnz(axis=1)[0], g1.shape[0]),
+ )
+ assert np.all(np.isfinite(merged.data))
+ assert np.all(merged.data > 0)
+ assert np.all(merged.data <= 1)
+
+
+def test_wnn_integration_is_invariant_to_cell_order():
+ g1, _, g2, _ = _multimodal_wnn_inputs()
+ rng = np.random.default_rng(42)
+ ld1 = rng.normal(size=(g1.shape[0], 3))
+ ld2 = rng.normal(size=(g2.shape[0], 4))
+ expected = wnn_integration("RNA", g1, ld1, "ADT", g2, ld2, n_threads=1)
+
+ permutation = np.array([5, 0, 7, 2, 6, 1, 4, 3])
+ permuted = wnn_integration(
+ "RNA",
+ g1[permutation][:, permutation].tocsr(),
+ ld1[permutation],
+ "ADT",
+ g2[permutation][:, permutation].tocsr(),
+ ld2[permutation],
+ n_threads=1,
+ )
+ inverse = np.argsort(permutation)
+ restored = permuted.tocsr()[inverse][:, inverse]
+
+ np.testing.assert_allclose(expected.toarray(), restored.toarray())
+
+
+def test_wnn_integration_rejects_mismatched_graph_shapes():
+ g1 = _simple_knn_graph(6, k=3)
+ g2 = _simple_knn_graph(7, k=3)
+
+ with pytest.raises(ValueError, match="same shape"):
+ wnn_integration(
+ "RNA",
+ g1,
+ np.zeros((6, 2)),
+ "ADT",
+ g2,
+ np.zeros((7, 2)),
+ n_threads=1,
+ )
+
+
+def test_wnn_integration_rejects_irregular_row_degree():
+ g1 = _simple_knn_graph(6, k=3).tolil()
+ g1[0, 1] = 0
+ g1 = g1.tocsr()
+ g1.eliminate_zeros()
+ g2 = _simple_knn_graph(6, k=3)
+ embeddings = np.arange(12, dtype=np.float64).reshape(6, 2)
+
+ with pytest.raises(ValueError, match="regular row degree"):
+ wnn_integration("RNA", g1, embeddings, "ADT", g2, embeddings, n_threads=1)
+
+
+@pytest.mark.parametrize(
+ ("embedding", "match"),
+ [
+ (np.zeros((5, 2)), "one row per graph cell"),
+ (np.empty((6, 0)), "non-empty matrix"),
+ (
+ np.array(
+ [[0.0, 0.0]] * 5 + [[np.nan, 0.0]],
+ dtype=np.float64,
+ ),
+ "non-finite values",
+ ),
+ ],
+)
+def test_wnn_integration_rejects_invalid_embeddings(embedding, match):
+ graph = _simple_knn_graph(6, k=3)
+ valid_embedding = np.zeros((6, 2))
+
+ with pytest.raises(ValueError, match=match):
+ wnn_integration(
+ "RNA",
+ graph,
+ embedding,
+ "ADT",
+ graph,
+ valid_embedding,
+ n_threads=1,
+ )
+
+
+def test_wnn_integration_uses_minimum_neighbor_count_for_mismatched_graphs():
+ g1 = _simple_knn_graph(8, k=3)
+ g2 = _simple_knn_graph(8, k=2)
+ rng = np.random.default_rng(7)
+ ld1 = rng.normal(size=(8, 3))
+ ld2 = rng.normal(size=(8, 2))
+ messages = []
+ sink = logger.add(
+ lambda message: messages.append(message.record["message"]), level="WARNING"
+ )
+ try:
+ merged = wnn_integration("RNA", g1, ld1, "ADT", g2, ld2, n_threads=1)
+ swapped = wnn_integration("ADT", g2, ld2, "RNA", g1, ld1, n_threads=1)
+ finally:
+ logger.remove(sink)
+
+ assert any("different neighbor counts" in message for message in messages)
+ assert merged.nnz == g1.shape[0] * 2
+ assert np.all(np.isfinite(merged.data))
+ np.testing.assert_allclose(merged.toarray(), swapped.toarray())
diff --git a/tests/test_mapping_label_transfer.py b/tests/test_mapping_label_transfer.py
new file mode 100644
index 00000000..ff85796e
--- /dev/null
+++ b/tests/test_mapping_label_transfer.py
@@ -0,0 +1,306 @@
+import numpy as np
+import pytest
+
+from scarf.mapping_utils import array_hash
+from scarf.writers import create_zarr_dataset
+
+
+def _ensure_graph(datastore) -> None:
+ try:
+ datastore._get_latest_graph_loc("RNA", "I", "hvgs")
+ except KeyError:
+ datastore.auto_filter_cells(show_qc_plots=False)
+ datastore.mark_hvgs(top_n=100, show_plot=False)
+ datastore.make_graph(feat_key="hvgs")
+
+
+def _projection_store(datastore, name: str, indices: np.ndarray, distances: np.ndarray):
+ source = datastore.RNA
+ _ensure_graph(datastore)
+ if "projections" not in source.z:
+ source.z.create_group("projections")
+ store = source.z["projections"].create_group(name, overwrite=True)
+ zi = create_zarr_dataset(store, "indices", (len(indices),), "u8", indices.shape)
+ zd = create_zarr_dataset(store, "distances", (len(indices),), "f8", distances.shape)
+ zi[:] = indices
+ zd[:] = distances
+ store.attrs["complete"] = True
+ store.attrs["schemaVersion"] = 1
+ store.attrs["assay"] = "RNA"
+ store.attrs["cellKey"] = "I"
+ store.attrs["featureKey"] = "hvgs"
+ store.attrs["referenceCellHash"] = array_hash(datastore.cells.fetch("ids", key="I"))
+ store.attrs["featureCoverage"] = 1.0
+ return store
+
+
+def test_label_transfer_uses_every_neighbor_and_abstains_on_ties(datastore_ephemeral):
+ datastore = datastore_ephemeral
+ reference_ids = datastore.cells.fetch("ids", key="I")
+ _projection_store(
+ datastore,
+ "manual_labels",
+ np.array([[0, 1], [0, 1]], dtype=np.uint64),
+ np.array([[0.0, 1.0], [1.0, 1.0]]),
+ )
+
+ labels = datastore.get_target_classes(
+ "manual_labels",
+ reference_class_group="ids",
+ threshold_fraction=0.5,
+ )
+
+ assert labels.tolist() == [reference_ids[0], "NA"]
+
+
+def test_mapping_score_uses_all_saved_neighbors(datastore_ephemeral):
+ datastore = datastore_ephemeral
+ _projection_store(
+ datastore,
+ "manual_scores",
+ np.array([[0, 1]], dtype=np.uint64),
+ np.array([[1.0, 1.0]]),
+ )
+
+ score = next(
+ datastore.get_mapping_score(
+ "manual_scores", log_transform=False, multiplier=1.0
+ )
+ )[1]
+
+ np.testing.assert_allclose(score[:2], [0.25, 0.25])
+
+
+def test_mapping_evidence_and_fixed_layout_projection(datastore_ephemeral):
+ datastore = datastore_ephemeral
+ _projection_store(
+ datastore,
+ "manual_evidence",
+ np.array([[0, 1], [0, 1]], dtype=np.uint64),
+ np.array([[0.0, 1.0], [1.0, 1.0]]),
+ )
+ layout1 = np.arange(datastore.cells.N, dtype=float)
+ layout2 = layout1 * 2
+ datastore.cells.insert("fixed_layout1", layout1, overwrite=True)
+ datastore.cells.insert("fixed_layout2", layout2, overwrite=True)
+
+ evidence = datastore.get_target_label_evidence(
+ "manual_evidence",
+ reference_class_group="ids",
+ threshold_fraction=0.75,
+ )
+ layout_path = datastore.project_mapping_layout("manual_evidence", "fixed_layout")
+ projected = datastore.z[layout_path][:]
+
+ assert evidence["isUnknown"].tolist() == [False, True]
+ assert evidence["label"].tolist()[1] == "NA"
+ assert set(
+ [
+ "voteFraction",
+ "voteEntropy",
+ "topTwoMargin",
+ "featureCoverage",
+ "referenceDistancePercentile",
+ ]
+ ).issubset(evidence.columns)
+ np.testing.assert_allclose(projected[0], [0.0, 0.0])
+ np.testing.assert_allclose(projected[1], [0.5, 1.0])
+
+
+def test_label_threshold_boundary_and_subset_index(datastore_ephemeral):
+ datastore = datastore_ephemeral
+ labels = np.repeat("other", datastore.cells.N).astype(object)
+ labels[:2] = ["winner", "runner_up"]
+ datastore.cells.insert("boundary_labels", labels, overwrite=True)
+ _projection_store(
+ datastore,
+ "boundary_labels",
+ np.array([[0, 1], [0, 1]], dtype=np.uint64),
+ np.array([[1.0, 9.0], [9.0, 1.0]]),
+ )
+
+ predictions = datastore.get_target_classes(
+ "boundary_labels",
+ reference_class_group="boundary_labels",
+ threshold_fraction=0.75,
+ target_subset=[1],
+ na_val="unknown",
+ )
+
+ assert predictions.index.tolist() == [1]
+ assert predictions.tolist() == ["runner_up"]
+ evidence = datastore.get_target_label_evidence(
+ "boundary_labels",
+ reference_class_group="boundary_labels",
+ threshold_fraction=0.75,
+ na_val="unknown",
+ )
+ assert evidence["label"].tolist() == ["winner", "runner_up"]
+ np.testing.assert_allclose(evidence["voteFraction"], [0.75, 0.75])
+
+
+def test_schema_less_projection_remains_readable(datastore_ephemeral):
+ datastore = datastore_ephemeral
+ store = _projection_store(
+ datastore,
+ "legacy_projection",
+ np.array([[0, 1]], dtype=np.uint64),
+ np.array([[1.0, 1.0]]),
+ )
+ del store.attrs["schemaVersion"]
+ del store.attrs["complete"]
+
+ with pytest.warns(DeprecationWarning, match="predates"):
+ result = datastore.get_target_classes(
+ "legacy_projection",
+ reference_class_group="ids",
+ )
+
+ assert len(result) == 1
+ store.attrs["complete"] = False
+ with pytest.raises(ValueError, match="incomplete"):
+ datastore.get_target_classes(
+ "legacy_projection",
+ reference_class_group="ids",
+ )
+
+
+def test_incomplete_versioned_projection_is_rejected(datastore_ephemeral):
+ datastore = datastore_ephemeral
+ store = _projection_store(
+ datastore,
+ "incomplete_projection",
+ np.array([[0, 1]], dtype=np.uint64),
+ np.array([[1.0, 1.0]]),
+ )
+ store.attrs["complete"] = False
+
+ with pytest.raises(ValueError, match="incomplete"):
+ datastore.get_target_classes(
+ "incomplete_projection",
+ reference_class_group="ids",
+ )
+
+
+def test_read_only_fixed_layout_returns_array(datastore_ephemeral):
+ from scarf.datastore.datastore import DataStore
+
+ datastore = datastore_ephemeral
+ _projection_store(
+ datastore,
+ "read_only_layout",
+ np.array([[0, 1]], dtype=np.uint64),
+ np.array([[1.0, 1.0]]),
+ )
+ layout1 = np.arange(datastore.cells.N, dtype=float)
+ layout2 = layout1 * 2
+ datastore.cells.insert("readonly_layout1", layout1, overwrite=True)
+ datastore.cells.insert("readonly_layout2", layout2, overwrite=True)
+ read_only = DataStore(
+ datastore.zarr_loc,
+ default_assay="RNA",
+ zarr_mode="r",
+ )
+
+ projected = read_only.project_mapping_layout("read_only_layout", "readonly_layout")
+
+ assert isinstance(projected, np.ndarray)
+ np.testing.assert_allclose(projected[0], [0.5, 1.0])
+
+
+def test_conformal_prediction_sets_are_exposed_in_label_evidence(
+ datastore_ephemeral,
+):
+ datastore = datastore_ephemeral
+ _projection_store(
+ datastore,
+ "conformal_evidence",
+ np.array([[0, 1]], dtype=np.uint64),
+ np.array([[0.0, 1.0]]),
+ )
+
+ evidence = datastore.get_target_label_evidence(
+ "conformal_evidence",
+ reference_class_group="ids",
+ calibration_nonconformity=np.array([0.1, 0.2, 0.3]),
+ conformal_alpha=0.2,
+ )
+
+ assert "predictionSet" in evidence
+ assert datastore.cells.fetch("ids", key="I")[0] in evidence.loc[0, "predictionSet"]
+
+
+def test_uninformative_projection_rows_are_forced_unknown(datastore_ephemeral):
+ datastore = datastore_ephemeral
+ store = _projection_store(
+ datastore,
+ "uninformative_evidence",
+ np.array([[0, 1], [0, 1]], dtype=np.uint64),
+ np.array([[0.0, 1.0], [0.0, 1.0]]),
+ )
+ uninformative = create_zarr_dataset(
+ store,
+ "uninformative",
+ (2,),
+ "bool",
+ (2,),
+ )
+ uninformative[:] = [True, False]
+
+ evidence = datastore.get_target_label_evidence(
+ "uninformative_evidence",
+ reference_class_group="ids",
+ threshold_fraction=0.0,
+ )
+
+ assert evidence["isUnknown"].tolist() == [True, False]
+ assert evidence.loc[0, "label"] == "NA"
+ classes = datastore.get_target_classes(
+ "uninformative_evidence",
+ reference_class_group="ids",
+ threshold_fraction=0.0,
+ na_val="unknown",
+ )
+ assert classes.iloc[0] == "unknown"
+
+ decision = datastore._label_vote_decision(
+ datastore.cells.fetch("ids", key="I"),
+ np.array([0, 1]),
+ np.array([0.0, 0.0]),
+ 0.0,
+ "unknown",
+ )
+ assert decision[0] == "unknown"
+ assert decision[4]
+
+
+def test_reference_distance_percentile_is_query_composition_invariant(
+ datastore_ephemeral,
+):
+ datastore = datastore_ephemeral
+ _projection_store(
+ datastore,
+ "distance_single",
+ np.array([[0, 1]], dtype=np.uint64),
+ np.array([[1.0, 4.0]]),
+ )
+ _projection_store(
+ datastore,
+ "distance_composed",
+ np.array([[0, 1], [2, 3], [4, 5]], dtype=np.uint64),
+ np.array([[1.0, 4.0], [0.1, 0.2], [10.0, 20.0]]),
+ )
+
+ single = datastore.get_target_label_evidence(
+ "distance_single",
+ reference_class_group="ids",
+ )
+ composed = datastore.get_target_label_evidence(
+ "distance_composed",
+ reference_class_group="ids",
+ )
+
+ assert (
+ single.loc[0, "referenceDistancePercentile"]
+ == composed.loc[0, "referenceDistancePercentile"]
+ )
diff --git a/tests/test_mapping_reference.py b/tests/test_mapping_reference.py
new file mode 100644
index 00000000..46cffd14
--- /dev/null
+++ b/tests/test_mapping_reference.py
@@ -0,0 +1,164 @@
+from types import SimpleNamespace
+
+import numpy as np
+import pytest
+import zarr
+
+from scarf.mapping_reference import (
+ LATEST_MAPPING_REFERENCE_ATTRIBUTE,
+ MAPPING_REFERENCE_GROUP,
+ MAPPING_REFERENCES_GROUP,
+ load_mapping_reference,
+ persist_mapping_reference,
+)
+from scarf.symphony import SymphonyReferenceModel
+
+
+def _model() -> SymphonyReferenceModel:
+ return SymphonyReferenceModel(
+ feature_means=np.array([1.0, 2.0]),
+ feature_scales=np.array([0.5, 2.0]),
+ loadings=np.eye(2),
+ centroids=np.array([[1.0, 0.0], [0.0, 1.0]]),
+ raw_centroids=np.array([[1.0, 1.0], [2.0, 2.0]]),
+ corrected_centroids=np.array([[0.5, 1.0], [1.5, 2.0]]),
+ cluster_mass=np.array([5.0, 4.0]),
+ sigma=np.array([0.1, 0.2]),
+ correction_ridge=1.0,
+ )
+
+
+def test_mapping_reference_roundtrip(tmp_path):
+ root = zarr.open_group(str(tmp_path / "reference.zarr"), mode="w")
+ reduction = root.create_group("RNA/reduction")
+ metadata = {
+ "assay": "RNA",
+ "cellKey": "I",
+ "featureKey": "hvgs",
+ "reductionPath": "RNA/reduction",
+ "annPath": "RNA/reduction/ann",
+ "featureHash": "features",
+ "cellHash": "cells",
+ "batchValueHash": "batches",
+ "batchColumns": ["batch"],
+ "subsetHash": 1,
+ "subsetParams": {"log_transform": True},
+ "loadingsHash": "loadings",
+ "reductionMethod": "pca",
+ }
+ artifact_path = persist_mapping_reference(
+ reduction,
+ _model(),
+ np.array(["gene_a", "gene_b"]),
+ metadata,
+ reference_distance_quantiles=np.array([0.0, 1.0]),
+ reference_distance_values=np.array([0.1, 2.0]),
+ )
+
+ reference = load_mapping_reference(
+ SimpleNamespace(zw=root),
+ "RNA",
+ "I",
+ "hvgs",
+ "RNA/reduction",
+ "RNA/reduction/ann",
+ )
+
+ assert artifact_path.startswith(f"{MAPPING_REFERENCES_GROUP}/")
+ assert f"RNA/reduction/{artifact_path}" in root
+ assert (
+ reduction.attrs[LATEST_MAPPING_REFERENCE_ATTRIBUTE]
+ == artifact_path.rsplit("/", 1)[-1]
+ )
+ np.testing.assert_array_equal(reference.feature_ids, ["gene_a", "gene_b"])
+ np.testing.assert_allclose(
+ reference.model.corrected_centroids, _model().corrected_centroids
+ )
+ np.testing.assert_allclose(reference.reference_distance_values, [0.1, 2.0])
+ assert reference.metadata["algorithmVariant"] == "symphonyStyleV1"
+
+ repeated_path = persist_mapping_reference(
+ reduction,
+ _model(),
+ np.array(["gene_a", "gene_b"]),
+ metadata,
+ reference_distance_quantiles=np.array([0.0, 1.0]),
+ reference_distance_values=np.array([0.1, 2.0]),
+ )
+ assert repeated_path == artifact_path
+ assert len(reduction[MAPPING_REFERENCES_GROUP]) == 1
+
+ reduction[artifact_path]["loadings"][0, 0] = 2.0
+ with pytest.raises(ValueError, match="artifact hash"):
+ load_mapping_reference(
+ SimpleNamespace(zw=root),
+ "RNA",
+ "I",
+ "hvgs",
+ "RNA/reduction",
+ "RNA/reduction/ann",
+ )
+
+
+def test_legacy_mapping_reference_remains_readable(tmp_path):
+ root = zarr.open_group(str(tmp_path / "legacy_reference.zarr"), mode="w")
+ reduction = root.create_group("RNA/reduction")
+ legacy = reduction.create_group(MAPPING_REFERENCE_GROUP)
+ legacy.attrs["schemaVersion"] = 1
+ legacy.attrs["complete"] = True
+ legacy.attrs["correctionRidge"] = 1.0
+ legacy.attrs["batchColumns"] = ["batch"]
+ model = _model()
+ for name, values in {
+ "featureMeans": model.feature_means,
+ "featureScales": model.feature_scales,
+ "loadings": model.loadings,
+ "centroids": model.centroids,
+ "rawCentroids": model.raw_centroids,
+ "correctedCentroids": model.corrected_centroids,
+ "clusterMass": model.cluster_mass,
+ "sigma": model.sigma,
+ "featureIds": np.array(["gene_a", "gene_b"]),
+ }.items():
+ legacy.create_array(name, data=values)
+
+ with pytest.warns(DeprecationWarning, match="legacy"):
+ reference = load_mapping_reference(
+ SimpleNamespace(zw=root),
+ "RNA",
+ "I",
+ "hvgs",
+ "RNA/reduction",
+ "RNA/reduction/ann",
+ )
+
+ assert reference.artifact_path.endswith(MAPPING_REFERENCE_GROUP)
+
+
+def test_incomplete_content_addressed_reference_is_rejected(tmp_path):
+ root = zarr.open_group(str(tmp_path / "incomplete_reference.zarr"), mode="w")
+ reduction = root.create_group("RNA/reduction")
+ artifact_path = persist_mapping_reference(
+ reduction,
+ _model(),
+ np.array(["gene_a", "gene_b"]),
+ {
+ "assay": "RNA",
+ "cellKey": "I",
+ "featureKey": "hvgs",
+ "reductionPath": "RNA/reduction",
+ "annPath": "RNA/reduction/ann",
+ "batchColumns": ["batch"],
+ },
+ )
+ reduction[artifact_path].attrs["complete"] = False
+
+ with pytest.raises(ValueError, match="incomplete"):
+ load_mapping_reference(
+ SimpleNamespace(zw=root),
+ "RNA",
+ "I",
+ "hvgs",
+ "RNA/reduction",
+ "RNA/reduction/ann",
+ )
diff --git a/tests/test_mapping_utils.py b/tests/test_mapping_utils.py
new file mode 100644
index 00000000..18f9f0c4
--- /dev/null
+++ b/tests/test_mapping_utils.py
@@ -0,0 +1,162 @@
+import numpy as np
+import pytest
+
+from scarf.chunked import ChunkedArray
+from scarf.mapping_utils import (
+ _correlation_alignment,
+ _distance_quantile_summary,
+ _order_features,
+ conformal_prediction_sets,
+ distance_weights,
+)
+
+
+class _Features:
+ def __init__(self, ids: list[str]) -> None:
+ self._ids = ids
+
+ def fetch_all(self, column: str) -> np.ndarray:
+ assert column == "ids"
+ return np.asarray(self._ids)
+
+
+class _Assay:
+ def __init__(self, ids: list[str]) -> None:
+ self.feats = _Features(ids)
+
+
+def test_distance_weights_uses_all_neighbors_and_handles_zero_distance():
+ weights = distance_weights(np.array([[0.0, 1.0, 4.0], [1.0, 1.0, 1.0]]))
+
+ np.testing.assert_allclose(weights[0], [1.0, 0.0, 0.0])
+ np.testing.assert_allclose(weights[1], [1 / 3, 1 / 3, 1 / 3])
+ np.testing.assert_allclose(weights.sum(axis=1), 1.0)
+
+
+def test_distance_weights_rejects_invalid_hnsw_distances():
+ with pytest.raises(ValueError, match="non-negative"):
+ distance_weights(np.array([[-1.0, 1.0]]))
+ with pytest.raises(ValueError, match="finite"):
+ distance_weights(np.array([[np.nan, 1.0]]))
+
+
+def test_distance_quantile_summary_handles_vectors_and_neighbor_matrices():
+ first_neighbors = np.array([0.0, 1.0, 4.0, 9.0, 16.0])
+ neighbor_matrix = np.column_stack((first_neighbors, first_neighbors + 1))
+
+ vector_summary = _distance_quantile_summary(
+ first_neighbors,
+ max_samples=3,
+ n_quantiles=3,
+ )
+ matrix_summary = _distance_quantile_summary(
+ neighbor_matrix,
+ max_samples=3,
+ n_quantiles=3,
+ )
+
+ np.testing.assert_allclose(vector_summary[0], [0.0, 0.5, 1.0])
+ np.testing.assert_allclose(vector_summary[1], [0.0, 4.0, 16.0])
+ np.testing.assert_allclose(matrix_summary[0], vector_summary[0])
+ np.testing.assert_allclose(matrix_summary[1], vector_summary[1])
+
+
+def test_coral_rejects_one_cell_cohorts():
+ source = ChunkedArray.from_numpy(np.array([[1.0, 2.0]]))
+ target = ChunkedArray.from_numpy(np.array([[1.0, 2.0], [2.0, 3.0]]))
+
+ with pytest.raises(ValueError, match="at least two cells"):
+ _correlation_alignment(source, target, nthreads=1)
+
+
+def test_coral_alignment_is_finite_for_small_full_rank_inputs():
+ source = ChunkedArray.from_numpy(
+ np.array([[0.0, 1.0], [1.0, 2.0], [2.0, 4.0]], dtype=np.float64)
+ )
+ target = ChunkedArray.from_numpy(
+ np.array([[1.0, 0.0], [3.0, 1.0], [5.0, 3.0]], dtype=np.float64)
+ )
+
+ aligned = _correlation_alignment(source, target, nthreads=1).compute()
+
+ assert aligned.shape == source.shape
+ assert np.all(np.isfinite(aligned))
+
+
+def test_conformal_prediction_sets_include_high_score_labels():
+ sets = conformal_prediction_sets(
+ np.array([[0.95, 0.1], [0.7, 0.7]]),
+ np.array([0.05, 0.1, 0.2, 0.25]),
+ alpha=0.2,
+ )
+
+ assert sets.shape == (2, 2)
+ assert sets[0, 0]
+ assert not sets[0, 1]
+
+
+def test_feature_alignment_rejects_duplicate_identifiers():
+ with pytest.raises(ValueError, match="unique"):
+ _order_features(
+ _Assay(["gene_a", "gene_a"]),
+ _Assay(["gene_a", "gene_b"]),
+ np.array(["gene_a", "gene_a"]),
+ filter_null=False,
+ missing_feature_policy="zero",
+ nthreads=1,
+ )
+
+
+def test_feature_ordering_covers_zero_intersection_and_error_inputs():
+ source = _Assay(["gene_a", "gene_b", "gene_c"])
+ target = _Assay(["gene_b", "gene_a"])
+ selected = np.array(["gene_a", "gene_b", "gene_c"])
+
+ zero_source, zero_target = _order_features(
+ source,
+ target,
+ selected,
+ filter_null=False,
+ missing_feature_policy="zero",
+ nthreads=1,
+ )
+ intersection_source, intersection_target = _order_features(
+ source,
+ target,
+ selected,
+ filter_null=False,
+ missing_feature_policy="intersection",
+ nthreads=1,
+ )
+
+ np.testing.assert_array_equal(zero_source, [0, 1, 2])
+ np.testing.assert_array_equal(zero_target, [1, 0, -1])
+ np.testing.assert_array_equal(intersection_source, [0, 1])
+ np.testing.assert_array_equal(intersection_target, [1, 0])
+
+
+def test_coral_restored_values_use_reference_scaling_contract():
+ query = np.array(
+ [[1.0, 2.0], [2.0, 4.0], [4.0, 5.0], [7.0, 9.0]],
+ dtype=np.float64,
+ )
+ reference = np.array(
+ [[10.0, 3.0], [12.0, 6.0], [15.0, 8.0], [20.0, 12.0]],
+ dtype=np.float64,
+ )
+ query_mean = query.mean(axis=0)
+ query_scale = query.std(axis=0)
+ reference_mean = reference.mean(axis=0)
+ reference_scale = reference.std(axis=0)
+ standardized = _correlation_alignment(
+ ChunkedArray.from_numpy((query - query_mean) / query_scale),
+ ChunkedArray.from_numpy((reference - reference_mean) / reference_scale),
+ nthreads=1,
+ ).compute()
+ restored = standardized * reference_scale + reference_mean
+
+ np.testing.assert_allclose(
+ (restored - reference_mean) / reference_scale,
+ standardized,
+ atol=1e-12,
+ )
diff --git a/tests/test_markers.py b/tests/test_markers.py
new file mode 100644
index 00000000..eb21b8ba
--- /dev/null
+++ b/tests/test_markers.py
@@ -0,0 +1,551 @@
+import numpy as np
+import pandas as pd
+import pytest
+
+import scarf.markers as markers_module
+from scarf.assay import norm_lib_size
+from scarf.markers import (
+ _batch_stats,
+ _marker_stats_batch,
+ find_markers_by_rank,
+ find_markers_by_regression,
+ mannwhitneyu_from_ranks,
+ sort_marker_results,
+)
+from scarf.utils import controlled_compute
+
+
+def _reference_calc(
+ vdf: pd.DataFrame, groups: np.ndarray, group_set: np.ndarray
+) -> np.ndarray:
+ """Original pandas implementation used as the parity reference."""
+ ranked_vdf = vdf.rank(method="dense")
+ ranked_vdf_average = vdf.rank(method="average")
+ r = ranked_vdf.groupby(groups).mean().reindex(group_set)
+ r = r / r.sum()
+ g = np.array([pd.Series(groups).value_counts().reindex(group_set).values]).T
+ g_o = len(groups) - g
+ s = vdf.groupby(groups).sum().reindex(group_set)
+ m = s / g
+ m_o = (s.sum() - s) / g_o
+ s2 = (vdf > 0).groupby(groups).sum().reindex(group_set)
+ e = s2 / g
+ e_o = (s2.sum() - s2) / g_o
+ fc = (m / m_o).fillna(0)
+ pvals = mannwhitneyu_from_ranks(ranked_vdf_average, groups, group_set)
+ return np.array(
+ [r.values, m.values, m_o.values, e.values, e_o.values, fc.values, pvals.values]
+ ).T
+
+
+def test_batch_stats_matches_pandas_reference():
+ rng = np.random.default_rng(0)
+ n_cells, n_genes = 250, 16
+ # Zero-inflated counts exercise the tie correction path.
+ data = rng.poisson(0.6, size=(n_cells, n_genes)).astype(np.float64)
+ groups = rng.integers(1, 4, size=n_cells)
+ group_set = np.array(sorted(set(groups)))
+ idx_map = {v: i for i, v in enumerate(group_set)}
+ int_indices = np.array([idx_map[x] for x in groups])
+ group_counts = pd.Series(groups).value_counts().reindex(group_set).values
+
+ ref = _reference_calc(pd.DataFrame(data), groups, group_set)
+ got = _batch_stats(data, int_indices, group_counts, n_cells)
+
+ # score, mean, mean_rest, frac_exp, frac_exp_rest
+ assert np.allclose(got[:, :, :5], ref[:, :, :5], atol=1e-6)
+ # fold_change agrees where the reference is finite
+ finite = np.isfinite(ref[:, :, 5])
+ assert np.allclose(got[:, :, 5][finite], ref[:, :, 5][finite], atol=1e-6)
+ # two-sided p-values
+ assert np.allclose(got[:, :, 6], ref[:, :, 6], atol=1e-6)
+
+
+def test_batch_stats_distinguishes_zero_fold_change_from_zero_rest_sentinel():
+ data = np.array(
+ [
+ [0.0, 2.0, 1.0],
+ [0.0, 4.0, 3.0],
+ [0.0, 0.0, 2.0],
+ [0.0, 0.0, 2.0],
+ ]
+ )
+ stats = _batch_stats(
+ data,
+ int_indices=np.array([0, 0, 1, 1]),
+ group_counts=np.array([2, 2]),
+ n_total=4,
+ )
+
+ assert np.array_equal(stats[0, :, 5], [0.0, 0.0])
+ assert stats[1, 0, 5] == pytest.approx(100.1)
+ assert stats[1, 1, 5] == pytest.approx(0.0)
+ assert np.array_equal(stats[2, :, 5], [1.0, 1.0])
+ assert np.array_equal(stats[0, :, 6], [1.0, 1.0])
+
+
+def test_marker_stats_python_kernel_matches_compiled_kernel():
+ data = np.array(
+ [
+ [0.0, 5.0, 0.0, 1.0],
+ [0.0, 4.0, 1.0, 1.0],
+ [0.0, 0.0, 2.0, 2.0],
+ [0.0, 0.0, 3.0, 2.0],
+ [0.0, 0.0, 4.0, 3.0],
+ [0.0, 0.0, 5.0, 3.0],
+ ],
+ dtype=np.float32,
+ )
+ int_indices = np.array([0, 0, 1, 1, 2, 2])
+ group_counts = np.array([2, 2, 2], dtype=np.float32)
+ n_total = np.float32(len(data))
+
+ python_stats = _marker_stats_batch.py_func(
+ data,
+ int_indices,
+ group_counts,
+ n_total,
+ )
+ compiled_stats = _marker_stats_batch(
+ data,
+ int_indices,
+ group_counts,
+ n_total,
+ )
+
+ np.testing.assert_allclose(python_stats, compiled_stats)
+ assert python_stats[1, 0, 5] == pytest.approx(100.1)
+ assert np.array_equal(python_stats[0, :, 5], [0.0, 0.0, 0.0])
+
+
+def test_marker_stats_python_kernel_handles_single_cell_population():
+ stats = _marker_stats_batch.py_func(
+ np.array([[0.0, 2.0]], dtype=np.float32),
+ np.array([0]),
+ np.array([1], dtype=np.float32),
+ np.float32(1),
+ )
+
+ np.testing.assert_allclose(
+ stats[:, 0, :],
+ np.array(
+ [
+ [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
+ [1.0, 2.0, 0.0, 1.0, 0.0, 100.1, 0.0],
+ ]
+ ),
+ )
+
+
+def test_sort_marker_results_adds_deterministic_tie_breakers():
+ named = pd.DataFrame(
+ {
+ "score": [0.8, 0.8, 0.8],
+ "p_value": [0.02, 0.01, 0.01],
+ "feature_name": ["zeta", "beta", "alpha"],
+ },
+ index=[7, 8, 9],
+ )
+
+ sorted_named = sort_marker_results(named)
+ unnamed = named.drop(columns="feature_name").iloc[[0, 2]].copy()
+ unnamed["p_value"] = 0.01
+ sorted_unnamed = sort_marker_results(unnamed)
+
+ assert "feature_index" not in named
+ assert sorted_named["feature_name"].tolist() == ["alpha", "beta", "zeta"]
+ assert sorted_named["feature_index"].tolist() == [9, 8, 7]
+ assert sorted_unnamed["feature_index"].tolist() == [7, 9]
+
+
+def test_find_markers_by_regression_handles_expression_threshold():
+ class Assay:
+ @staticmethod
+ def iter_normed_feature_wise(**_kwargs):
+ yield pd.DataFrame(
+ {
+ "correlated": [0.0, 1.0, 2.0, 3.0],
+ "at_threshold": [0.0, 1.0, 2.0, 0.0],
+ "too_sparse": [0.0, 0.0, 1.0, 0.0],
+ "constant": [1.0, 1.0, 1.0, 1.0],
+ }
+ )
+
+ result = find_markers_by_regression(
+ Assay(),
+ cell_key="I",
+ feat_key="I",
+ regressor=np.arange(4),
+ min_cells=2,
+ )
+
+ assert result.loc["correlated", "r_value"] == pytest.approx(1.0)
+ assert result.loc["correlated", "p_value"] < 1e-10
+ assert result.loc["at_threshold", "r_value"] != 0.0
+ assert np.array_equal(result.loc["too_sparse"].to_numpy(), [0.0, 1.0])
+ assert np.array_equal(result.loc["constant"].to_numpy(), [0.0, 1.0])
+
+
+def test_find_markers_by_regression_rejects_non_dataframe_batches():
+ class Assay:
+ @staticmethod
+ def iter_normed_feature_wise(**_kwargs):
+ yield np.ones((3, 1))
+
+ with pytest.raises(TypeError, match="DataFrames"):
+ find_markers_by_regression(
+ Assay(),
+ cell_key="I",
+ feat_key="I",
+ regressor=np.arange(3),
+ min_cells=1,
+ )
+
+
+def test_find_markers_by_regression_identifies_nonfinite_feature():
+ class Assay:
+ @staticmethod
+ def iter_normed_feature_wise(**_kwargs):
+ yield pd.DataFrame({"bad_feature": [0.0, np.nan, 1.0]})
+
+ with pytest.raises(ValueError, match="bad_feature"):
+ find_markers_by_regression(
+ Assay(),
+ cell_key="I",
+ feat_key="I",
+ regressor=np.arange(3),
+ min_cells=1,
+ )
+
+
+def test_find_markers_by_rank_requires_requested_prenormed_values():
+ class Cells:
+ @staticmethod
+ def fetch(_group_key, _cell_key):
+ return np.array([0, 1])
+
+ class Assay:
+ cells = Cells()
+ z = {}
+
+ with pytest.raises(ValueError, match="Could not find prenormed values"):
+ find_markers_by_rank(
+ Assay(),
+ group_key="cluster",
+ cell_key="I",
+ feat_key="I",
+ batch_size=2,
+ use_prenormed=True,
+ prenormed_store=None,
+ n_threads=1,
+ )
+
+
+def test_find_markers_by_rank_rejects_fast_path_for_non_rna_assay():
+ class Cells:
+ @staticmethod
+ def fetch(_group_key, _cell_key):
+ return np.array([0, 1])
+
+ class Assay:
+ def __init__(self):
+ self.cells = Cells()
+ self.normMethod = norm_lib_size
+ self.sf = 1.0
+
+ with pytest.raises(TypeError, match="requires an RNAassay"):
+ find_markers_by_rank(
+ Assay(),
+ group_key="cluster",
+ cell_key="I",
+ feat_key="I",
+ batch_size=2,
+ use_prenormed=False,
+ prenormed_store=None,
+ n_threads=1,
+ )
+
+
+def test_find_markers_by_rank_slow_path_returns_groupwise_statistics():
+ data = np.array(
+ [
+ [2.0, 0.0, 0.0, 1.0],
+ [4.0, 0.0, 0.0, 1.0],
+ [0.0, 0.0, 5.0, 1.0],
+ [0.0, 0.0, 5.0, 1.0],
+ ]
+ )
+
+ class Cells:
+ @staticmethod
+ def fetch(_group_key, _cell_key):
+ return np.array(["a", "a", "b", "b"])
+
+ class Feats:
+ @staticmethod
+ def active_index(_feat_key):
+ return np.array([10, 11, 12, 13])
+
+ class Assay:
+ cells = Cells()
+ feats = Feats()
+ normMethod = None
+ sf = None
+
+ @staticmethod
+ def iter_normed_feature_wise(**_kwargs):
+ yield pd.DataFrame(data[:, :2])
+ yield pd.DataFrame(data[:, 2:])
+
+ results = find_markers_by_rank(
+ Assay(),
+ group_key="cluster",
+ cell_key="I",
+ feat_key="I",
+ batch_size=2,
+ use_prenormed=False,
+ prenormed_store=None,
+ n_threads=1,
+ )
+ group_a = results["a"].set_index("feature_index")
+ group_b = results["b"].set_index("feature_index")
+
+ assert group_a.loc[10, "fold_change"] == pytest.approx(100.1)
+ assert group_a.loc[11, "fold_change"] == pytest.approx(0.0)
+ assert group_a.loc[13, "fold_change"] == pytest.approx(1.0)
+ assert group_b.loc[12, "fold_change"] == pytest.approx(100.1)
+ assert group_b.loc[10, "fold_change"] == pytest.approx(0.0)
+ assert np.isfinite(group_a["p_value"]).all()
+ assert np.isfinite(group_b["p_value"]).all()
+
+
+def test_find_markers_fast_raw_path_computes_groupwise_statistics(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ data = np.array(
+ [
+ [4.0, 0.0, 1.0, 0.0],
+ [3.0, 0.0, 1.0, 0.0],
+ [0.0, 5.0, 1.0, 2.0],
+ [0.0, 6.0, 1.0, 2.0],
+ ]
+ )
+
+ class Cells:
+ @staticmethod
+ def fetch(_group_key, _cell_key):
+ return np.array(["a", "a", "b", "b"])
+
+ @staticmethod
+ def active_index(_cell_key):
+ return np.arange(4)
+
+ @staticmethod
+ def fetch_all(_key):
+ return data.sum(axis=1)
+
+ class Feats:
+ @staticmethod
+ def active_index(_feat_key):
+ return np.arange(4)
+
+ class FakeRNA:
+ def __init__(self):
+ self.cells = Cells()
+ self.feats = Feats()
+ self.normMethod = norm_lib_size
+ self.sf = 1_000.0
+ self.name = "RNA"
+
+ def iter_raw_column_blocks(self, **_kwargs):
+ for block_index, start in enumerate(range(0, data.shape[1], 2)):
+ columns = np.arange(start, min(start + 2, data.shape[1]))
+ yield block_index, data[:, columns], columns, 0.01, "memory"
+
+ monkeypatch.setattr(markers_module, "RNAassay", FakeRNA)
+ results = find_markers_by_rank(
+ FakeRNA(),
+ group_key="cluster",
+ cell_key="I",
+ feat_key="I",
+ batch_size=2,
+ use_prenormed=False,
+ prenormed_store=None,
+ n_threads=1,
+ )
+
+ assert set(results) == {"a", "b"}
+ for frame in results.values():
+ assert len(frame) == data.shape[1]
+ assert np.isfinite(frame["p_value"]).all()
+
+
+def test_find_markers_prenormalized_path_returns_groupwise_statistics() -> None:
+ import zarr
+ from zarr.storage import MemoryStore
+
+ root = zarr.open_group(store=MemoryStore(), mode="w")
+ values = (
+ [4.0, 3.0, 0.0, 0.0],
+ [0.0, 0.0, 5.0, 6.0],
+ [1.0, 1.0, 1.0, 1.0],
+ )
+ for feature_index, feature_values in enumerate(values):
+ array = root.create_array(str(feature_index), shape=(4,), dtype="f8")
+ array[:] = feature_values
+
+ class Cells:
+ @staticmethod
+ def fetch(_group_key, _cell_key):
+ return np.array(["a", "a", "b", "b"])
+
+ @staticmethod
+ def active_index(_cell_key):
+ return np.arange(4)
+
+ class Assay:
+ cells = Cells()
+ z = root
+
+ results = find_markers_by_rank(
+ Assay(),
+ group_key="cluster",
+ cell_key="I",
+ feat_key="I",
+ batch_size=2,
+ use_prenormed=True,
+ prenormed_store=root,
+ n_threads=1,
+ )
+
+ assert set(results) == {"a", "b"}
+ for frame in results.values():
+ assert len(frame) == len(values)
+
+
+def test_iter_raw_feature_columns_matches_normed(datastore):
+ assay = datastore.RNA
+ cell_idx = assay.cells.active_index("I")
+ feat_idx = assay.feats.active_index("I")
+ scalar = assay.cells.fetch_all(assay.name + "_nCounts")[cell_idx]
+
+ streamed = controlled_compute(
+ assay.normed(cell_idx=cell_idx, feat_idx=feat_idx), assay.nthreads
+ )
+
+ cols = []
+ mats = []
+ for mat, batch_cols in assay.iter_raw_feature_columns(
+ cell_idx=cell_idx,
+ feat_idx=feat_idx,
+ batch_size=37,
+ scalar=scalar,
+ sf=float(assay.sf),
+ prefetch_depth=3,
+ ):
+ mats.append(mat)
+ cols.append(batch_cols)
+
+ fast = np.hstack(mats)
+ assert np.array_equal(np.concatenate(cols), feat_idx)
+ assert fast.shape == streamed.shape
+ assert np.allclose(fast, streamed, rtol=1e-4, atol=1e-4)
+
+
+def test_read_prenormed_batches_yields_expected_chunks():
+ import zarr
+ from zarr.storage import MemoryStore
+
+ from scarf.markers import read_prenormed_batches
+
+ root = zarr.open_group(store=MemoryStore(), mode="w")
+ n_cells = 12
+ for batch_id in range(5):
+ arr = root.create_array(str(batch_id), shape=(n_cells,), dtype="f8")
+ arr[:] = float(batch_id)
+
+ cell_idx = np.arange(n_cells)
+ batches = list(
+ read_prenormed_batches(root, cell_idx, batch_size=2, desc="test batches")
+ )
+
+ assert len(batches) == 3
+ assert batches[0].shape == (n_cells, 2)
+ assert batches[1].shape == (n_cells, 2)
+ assert batches[2].shape == (n_cells, 1)
+ first_chunk = batches[0].iloc[:, 0]
+ last_chunk = batches[2].iloc[:, 0]
+ assert first_chunk.nunique() == 1
+ assert last_chunk.nunique() == 1
+ assert set(first_chunk.unique()).issubset({0.0, 1.0, 2.0, 3.0, 4.0})
+
+
+def test_compact_marker_save_roundtrip():
+ import zarr
+ from zarr.storage import MemoryStore
+
+ from scarf.datastore.datastore import DataStore, _load_marker_cluster_frame
+
+ index = np.array([10, 5, 7], dtype=np.int32)
+ source = pd.DataFrame(
+ {
+ "score": [0.9, 0.4, 0.2],
+ "mean": [1.0, 2.0, 3.0],
+ "mean_rest": [0.5, 0.5, 0.5],
+ "frac_exp": [0.8, 0.2, 0.1],
+ "frac_exp_rest": [0.3, 0.3, 0.3],
+ "fold_change": [2.0, 4.0, 6.0],
+ "p_value": [0.01, 0.02, 0.03],
+ },
+ index=index,
+ )
+ source["feature_index"] = source.index
+ source = sort_marker_results(source)
+ markers = {1: source}
+ root = zarr.open_group(store=MemoryStore(), mode="w")
+ slot = root.create_group("slot")
+ DataStore._write_marker_slot(slot, markers)
+ feature_names = np.array([f"g{i}" for i in range(11)])
+ loaded = _load_marker_cluster_frame(
+ slot,
+ slot["1"],
+ feature_names,
+ group_id=1,
+ )
+ assert slot.attrs["layout"] == "compact_v2"
+ assert list(slot.group_keys()) == ["1", "feature_index"] or "feature_index" in slot
+ assert len(loaded) == 3
+ assert loaded.iloc[0]["score"] == 0.9
+ assert loaded.iloc[0]["feature_name"] == "g10"
+
+
+def test_resolve_marker_gene_batch_size_respects_chunk_and_budget():
+ from scarf.markers import resolve_marker_gene_batch_size
+
+ batch = resolve_marker_gene_batch_size(
+ n_features=25_683,
+ n_cells=88_955,
+ column_chunk=948,
+ memory_bytes=24 * 1024**3,
+ working_copies=4,
+ )
+ assert batch == 948
+
+
+def test_resolve_marker_gene_batch_size_shrinks_with_more_cells():
+ from scarf.markers import resolve_marker_gene_batch_size
+
+ sizes = []
+ for n_cells in (100_000, 1_000_000, 10_000_000):
+ sizes.append(
+ resolve_marker_gene_batch_size(
+ n_features=45_525,
+ n_cells=n_cells,
+ column_chunk=10_000,
+ memory_bytes=24 * 1024**3,
+ working_copies=4,
+ )
+ )
+ assert sizes[0] > sizes[1] > sizes[2]
+ assert sizes[-1] >= 1
+ assert all(size <= 10_000 for size in sizes)
diff --git a/tests/test_meld_assay.py b/tests/test_meld_assay.py
new file mode 100644
index 00000000..f502c3e0
--- /dev/null
+++ b/tests/test_meld_assay.py
@@ -0,0 +1,434 @@
+import numpy as np
+import pandas as pd
+import pytest
+import zarr
+from zarr.storage import MemoryStore
+
+from scarf.chunked import ChunkedArray
+from scarf.meld_assay import (
+ GffReader,
+ binary_search,
+ create_bed_from_coord_ids,
+ create_counts_mat,
+ get_feature_mappings,
+)
+from scarf.writers import create_zarr_dataset
+
+
+def _features_bed(rows):
+ return pd.DataFrame({i: [r[i] for r in rows] for i in range(6)}).sort_values(
+ by=[0, 1]
+ )
+
+
+class _FakeMeta:
+ def __init__(self, columns):
+ self._columns = columns
+
+ def fetch_all(self, name):
+ return np.asarray(self._columns[name])
+
+
+class _FakeRawData:
+ def __init__(self, blocks):
+ self.blocks = blocks
+ self.numblocks = (len(blocks),)
+ self.shape = (sum(b.shape[0] for b in blocks), blocks[0].shape[1])
+
+ def stream_blocks(self, nthreads=None, msg=None, prefetch=None):
+ # Mirrors ChunkedArray.stream_blocks: yields materialized, in-order blocks.
+ yield from (np.asarray(b) for b in self.blocks)
+
+
+class _FakeAssay:
+ name = "ATAC"
+ nthreads = 1
+
+ def __init__(self, cells, feats, raw_data):
+ self.cells = cells
+ self.feats = feats
+ self.rawData = raw_data
+
+
+def _reference_melded(
+ raw, n_features_per_cell, n_cells_per_peak, mapping_dense, scalar_coeff, renorm
+):
+ n_docs = raw.shape[0]
+ idf = np.log2(1 + (n_docs / (n_cells_per_peak + 1)))
+ tf = raw / n_features_per_cell.reshape(-1, 1)
+ tfidf = tf * idf
+ melded = tfidf @ mapping_dense
+ if not renorm:
+ return melded
+ row_sums = melded.sum(axis=1)
+ out = np.zeros_like(melded)
+ nz = row_sums != 0
+ out[nz] = (scalar_coeff * melded[nz]) / row_sums[nz].reshape(-1, 1)
+ return out
+
+
+def _old_style_melded(
+ raw,
+ n_features_per_cell,
+ n_cells_per_peak,
+ feature_to_peaks,
+ scalar_coeff,
+ renorm,
+):
+ n_docs = raw.shape[0]
+ idf = np.log2(1 + (n_docs / (n_cells_per_peak + 1)))
+ tf = raw / n_features_per_cell.reshape(-1, 1)
+ tfidf = tf * idf
+
+ idx = np.array([i for i, peaks in enumerate(feature_to_peaks) if len(peaks)])
+ if idx.size == 0:
+ melded = np.zeros((raw.shape[0], len(feature_to_peaks)))
+ else:
+ feat_idx = np.repeat(idx, [len(feature_to_peaks[i]) for i in idx])
+ peak_idx = np.array(sum((feature_to_peaks[i] for i in idx), []))
+ df = pd.DataFrame(tfidf[:, peak_idx]).T
+ df["fidx"] = feat_idx
+ df = df.groupby("fidx").sum().T
+ melded = np.zeros((raw.shape[0], len(feature_to_peaks)))
+ melded[:, df.columns.to_numpy(dtype=int)] = df.values
+
+ if not renorm:
+ return melded
+ row_sums = melded.sum(axis=1)
+ out = np.zeros_like(melded)
+ nz = row_sums != 0
+ out[nz] = (scalar_coeff * melded[nz]) / row_sums[nz].reshape(-1, 1)
+ return out
+
+
+def _feature_to_peaks_from_intervals(peaks, features):
+ feature_to_peaks = []
+ for _, feat in features.iterrows():
+ peak_hits = []
+ for peak_pos, peak in peaks.iterrows():
+ if feat[0] == peak[0] and feat[1] < peak[2] and feat[2] > peak[1]:
+ peak_hits.append(int(peak_pos))
+ feature_to_peaks.append(peak_hits)
+ return feature_to_peaks
+
+
+def _build_melding_scenario():
+ peaks = create_bed_from_coord_ids(
+ ["chr1:100-200", "chr1:150-250", "chr1:400-500", "chr2:100-200"]
+ )
+ features = _features_bed(
+ [
+ ("chr1", 120, 160, "a", "A", "+"), # overlaps peaks 0, 1
+ ("chr1", 300, 350, "b", "B", "+"), # overlaps nothing
+ ("chr2", 150, 180, "c", "C", "+"), # overlaps peak 3
+ ]
+ )
+ _, _, mapping = get_feature_mappings(peaks, features)
+
+ # Rows are cells, columns are peaks (assay feature order).
+ raw = np.array(
+ [
+ [1.0, 0.0, 2.0, 3.0],
+ [0.0, 4.0, 0.0, 0.0],
+ [
+ 0.0,
+ 0.0,
+ 5.0,
+ 0.0,
+ ], # only peak 2, which maps to no feature -> melded row is all zero
+ ]
+ )
+ n_features_per_cell = np.array([3.0, 1.0, 1.0])
+ n_cells_per_peak = np.array([1.0, 1.0, 2.0, 1.0])
+
+ cells = _FakeMeta({"ATAC_nFeatures": n_features_per_cell})
+ feats = _FakeMeta({"nCells": n_cells_per_peak})
+ raw_data = _FakeRawData([raw[0:2], raw[2:3]])
+ assay = _FakeAssay(cells, feats, raw_data)
+ return assay, mapping, raw, n_features_per_cell, n_cells_per_peak
+
+
+def _run_create_counts_mat(assay, mapping, scalar_coeff, renormalization):
+ n_features = mapping.shape[1]
+ n_cells = assay.rawData.shape[0]
+ group = zarr.open_group(store=MemoryStore(), mode="w")
+ store = create_zarr_dataset(
+ group, "counts", (2, n_features), "float", (n_cells, n_features)
+ )
+ create_counts_mat(
+ assay=assay,
+ store=store,
+ mapping=mapping,
+ scalar_coeff=scalar_coeff,
+ renormalization=renormalization,
+ )
+ return store[:]
+
+
+GFF_CONTENT = """##gff-version 3
+# test annotation
+chr1\t.\tgene\t1000\t2000\t.\t+\t.\tgene_id=gene_a;gene_name=GENE_A
+chr1\t.\tgene\t3000\t4500\t.\t-\t.\tgene_id=gene_b;gene_name=GENE_B
+chr2\t.\tgene\t500\t1500\t.\t+\t.\tgene_id=gene_c;gene_name=GENE_C
+"""
+
+
+def test_gff_reader_parses_header_and_streams(tmp_path):
+ gff_path = tmp_path / "test.gff"
+ gff_path.write_text(GFF_CONTENT)
+
+ reader = GffReader(str(gff_path), up_offset=500, down_offset=200, chunk_size=2)
+ assert len(reader.header) == 2
+ chunks = list(reader.stream())
+ assert len(chunks) == 2
+ assert chunks[0].shape[0] == 2
+ assert set(chunks[0][2]) == {"gene"}
+
+
+def test_gff_reader_promoter_and_body_coordinates(tmp_path):
+ gff_path = tmp_path / "coords.gff"
+ gff_path.write_text(GFF_CONTENT)
+ reader = GffReader(str(gff_path), up_offset=500, down_offset=200)
+ plus_row = pd.Series([None] * 9)
+ plus_row[3] = 1000
+ plus_row[4] = 2000
+ plus_row[6] = "+"
+
+ promoter_start, promoter_end = reader.get_promoter(plus_row)
+ assert promoter_start == 500
+ assert promoter_end == 1200
+
+ body_start, body_end = reader.get_body(plus_row)
+ assert body_start == 500
+ assert body_end == 2000
+
+ minus_row = plus_row.copy()
+ minus_row[3] = 3000
+ minus_row[4] = 4500
+ minus_row[6] = "-"
+ m_start, m_end = reader.get_promoter(minus_row)
+ assert m_start == 4499 - reader.down
+ assert m_end == 4500 + reader.up
+
+
+def test_gff_reader_get_ids_names():
+ row = pd.Series([None] * 9)
+ row[8] = "gene_id=abc;gene_name=XYZ"
+ gene_id, gene_name = GffReader.get_ids_names(row)
+ assert gene_id == "abc"
+ assert gene_name == "XYZ"
+
+
+def test_gff_reader_to_bed_promoter(tmp_path):
+ gff_path = tmp_path / "genes.gff"
+ gff_path.write_text(GFF_CONTENT)
+ bed_path = tmp_path / "out.bed"
+
+ reader = GffReader(str(gff_path))
+ reader.to_bed(str(bed_path), flavour="promoter")
+
+ bed = pd.read_csv(bed_path, sep="\t", header=None)
+ assert bed.shape[0] == 3
+ assert bed.shape[1] == 6
+
+
+def test_gff_reader_to_bed_rejects_unknown_flavour(tmp_path):
+ gff_path = tmp_path / "genes.gff"
+ gff_path.write_text(GFF_CONTENT)
+ reader = GffReader(str(gff_path))
+ with pytest.raises(ValueError, match="flavour"):
+ reader.to_bed(str(tmp_path / "out.bed"), flavour="exon")
+
+
+def test_create_bed_from_coord_ids_sorts_intervals():
+ bed = create_bed_from_coord_ids(["chr2:100-200", "chr1:50-150", "chr1:300-400"])
+ assert bed.iloc[0, 0] == "chr1"
+ assert bed.iloc[0, 1] == 50
+ assert bed.iloc[-1, 0] == "chr2"
+
+
+def test_binary_search_finds_overlapping_ranges():
+ ranges = np.array([[0, 10], [10, 20], [20, 30], [30, 40]], dtype=np.int64)
+ queries = np.array([[5, 8], [15, 18], [25, 28]], dtype=np.int64)
+ hits = binary_search(ranges, queries)
+ assert hits.shape == (3, 2)
+ assert hits[0, 0] <= hits[0, 1]
+
+
+def test_get_feature_mappings_builds_sparse_overlap_matrix():
+ peaks = create_bed_from_coord_ids(
+ ["chr1:100-200", "chr1:150-250", "chr1:400-500", "chr2:100-200"]
+ )
+ features = _features_bed(
+ [
+ ("chr1", 120, 160, "a", "A", "+"),
+ ("chr1", 300, 350, "b", "B", "+"),
+ ("chr2", 150, 180, "c", "C", "+"),
+ ]
+ )
+ feat_ids, feat_names, mapping = get_feature_mappings(peaks, features)
+
+ assert list(feat_ids) == ["a", "b", "c"]
+ assert list(feat_names) == ["A", "B", "C"]
+ assert mapping.shape == (4, 3)
+
+ expected = np.zeros((4, 3))
+ expected[[0, 1], 0] = 1 # feature A overlaps peaks at original positions 0 and 1
+ expected[3, 2] = 1 # feature C overlaps peak at original position 3
+ np.testing.assert_array_equal(mapping.toarray(), expected)
+
+
+def test_get_feature_mappings_keeps_features_without_peaks():
+ peaks = create_bed_from_coord_ids(["chr1:100-200"])
+ features = _features_bed(
+ [
+ ("chr1", 120, 160, "a", "A", "+"),
+ ("chr3", 100, 200, "d", "D", "+"),
+ ]
+ )
+ feat_ids, _, mapping = get_feature_mappings(peaks, features)
+
+ # Feature on peak-less chromosome must be retained (not silently dropped)
+ assert list(feat_ids) == ["a", "d"]
+ assert mapping.shape[1] == 2
+ assert mapping.getcol(1).nnz == 0
+
+
+def test_get_feature_mappings_uniquifies_duplicate_ids():
+ peaks = create_bed_from_coord_ids(["chr1:100-200"])
+ features = _features_bed(
+ [
+ ("chr1", 120, 160, "dup", "A", "+"),
+ ("chr1", 500, 600, "dup", "B", "+"),
+ ]
+ )
+ feat_ids, _, _ = get_feature_mappings(peaks, features)
+ assert list(feat_ids) == ["dup", "dup_2"]
+
+
+def test_get_feature_mappings_follows_feature_bed_order():
+ peaks = create_bed_from_coord_ids(["chr1:100-200", "chr2:100-200"])
+ features = _features_bed(
+ [
+ ("chr2", 120, 160, "b", "B", "+"),
+ ("chr3", 100, 200, "c", "C", "+"),
+ ("chr1", 120, 160, "a", "A", "+"),
+ ]
+ )
+ feat_ids, _, mapping = get_feature_mappings(peaks, features)
+
+ assert list(feat_ids) == ["a", "b", "c"]
+ assert mapping.getcol(2).nnz == 0
+
+
+def test_mapping_matmul_sums_overlapping_peaks():
+ peaks = create_bed_from_coord_ids(
+ ["chr1:100-200", "chr1:150-250", "chr1:400-500", "chr2:100-200"]
+ )
+ features = _features_bed(
+ [
+ ("chr1", 120, 160, "a", "A", "+"),
+ ("chr1", 300, 350, "b", "B", "+"),
+ ("chr2", 150, 180, "c", "C", "+"),
+ ]
+ )
+ _, _, mapping = get_feature_mappings(peaks, features)
+
+ tfidf = np.array(
+ [
+ [1.0, 2.0, 3.0, 4.0],
+ [0.5, 0.0, 0.0, 1.5],
+ ]
+ )
+ melded = tfidf @ mapping.toarray()
+
+ # Feature A melds peaks 0 and 1, feature B nothing, feature C peak 3
+ np.testing.assert_allclose(melded[:, 0], tfidf[:, 0] + tfidf[:, 1])
+ np.testing.assert_allclose(melded[:, 1], np.zeros(2))
+ np.testing.assert_allclose(melded[:, 2], tfidf[:, 3])
+
+
+def test_create_counts_mat_matches_old_groupby_oracle():
+ peaks = create_bed_from_coord_ids(
+ ["chr1:100-200", "chr1:150-250", "chr1:210-260", "chr2:50-120"]
+ )
+ features = _features_bed(
+ [
+ ("chr1", 120, 240, "a", "A", "+"),
+ ("chr1", 500, 600, "b", "B", "+"),
+ ("chr2", 70, 100, "c", "C", "+"),
+ ]
+ )
+ _, _, mapping = get_feature_mappings(peaks, features)
+ raw = np.array(
+ [
+ [3.0, 0.0, 1.0, 0.0],
+ [0.0, 2.0, 0.0, 5.0],
+ [4.0, 1.0, 0.0, 0.0],
+ [0.0, 0.0, 7.0, 3.0],
+ ]
+ )
+ n_feat = np.array([2.0, 2.0, 2.0, 2.0])
+ n_cells_peak = np.array([2.0, 2.0, 2.0, 2.0])
+ assay = _FakeAssay(
+ _FakeMeta({"ATAC_nFeatures": n_feat}),
+ _FakeMeta({"nCells": n_cells_peak}),
+ _FakeRawData([raw[:2], raw[2:]]),
+ )
+ old_oracle = _old_style_melded(
+ raw,
+ n_feat,
+ n_cells_peak,
+ _feature_to_peaks_from_intervals(peaks, features),
+ scalar_coeff=1e4,
+ renorm=True,
+ )
+ written = _run_create_counts_mat(
+ assay, mapping, scalar_coeff=1e4, renormalization=True
+ )
+ np.testing.assert_allclose(written, old_oracle)
+
+
+def test_create_counts_mat_reads_real_chunked_array_stream_blocks():
+ assay, mapping, raw, n_feat, n_cells_peak = _build_melding_scenario()
+ group = zarr.open_group(store=MemoryStore(), mode="w")
+ counts = create_zarr_dataset(group, "raw", (2, raw.shape[1]), "float", raw.shape)
+ counts[:] = raw
+ assay.rawData = ChunkedArray(counts, nthreads=2)
+
+ written = _run_create_counts_mat(
+ assay, mapping, scalar_coeff=1e4, renormalization=True
+ )
+ expected = _reference_melded(
+ raw, n_feat, n_cells_peak, mapping.toarray(), scalar_coeff=1e4, renorm=True
+ )
+ np.testing.assert_allclose(written, expected)
+
+
+def test_create_counts_mat_without_renormalization():
+ assay, mapping, raw, n_feat, n_cells_peak = _build_melding_scenario()
+ written = _run_create_counts_mat(
+ assay, mapping, scalar_coeff=1e4, renormalization=False
+ )
+ expected = _reference_melded(
+ raw, n_feat, n_cells_peak, mapping.toarray(), scalar_coeff=1e4, renorm=False
+ )
+ np.testing.assert_allclose(written, expected)
+
+
+def test_create_counts_mat_with_renormalization_and_zero_sum_cell():
+ assay, mapping, raw, n_feat, n_cells_peak = _build_melding_scenario()
+ written = _run_create_counts_mat(
+ assay, mapping, scalar_coeff=1e4, renormalization=True
+ )
+ expected = _reference_melded(
+ raw, n_feat, n_cells_peak, mapping.toarray(), scalar_coeff=1e4, renorm=True
+ )
+ np.testing.assert_allclose(written, expected)
+
+ # The cell whose only signal maps to no feature must be all zeros, not NaN.
+ assert np.all(np.isfinite(written))
+ np.testing.assert_array_equal(written[2], np.zeros(mapping.shape[1]))
+ # Non-empty cells are rescaled to sum to scalar_coeff.
+ np.testing.assert_allclose(written[0].sum(), 1e4)
+ np.testing.assert_allclose(written[1].sum(), 1e4)
diff --git a/tests/test_merger.py b/tests/test_merger.py
new file mode 100644
index 00000000..4c480b3b
--- /dev/null
+++ b/tests/test_merger.py
@@ -0,0 +1,575 @@
+import threading
+
+import numpy as np
+import pandas as pd
+import pytest
+import zarr
+from zarr.storage import MemoryStore
+
+from scarf.chunked import ChunkedArray
+from scarf.merge import AssayMerge
+from scarf.storage.budget import (
+ ResourceBudget,
+ get_resource_budget,
+ set_resource_budget,
+)
+from scarf.storage.zarr_store import count_array_spec
+
+
+class _MergeMeta:
+ def __init__(self, **columns):
+ self._columns = {key: np.asarray(value) for key, value in columns.items()}
+ self.columns = list(self._columns)
+ self.N = len(next(iter(self._columns.values())))
+
+ def to_pandas_dataframe(self, columns):
+ return pd.DataFrame({key: self._columns[key] for key in columns})
+
+ def fetch_all(self, key):
+ return self._columns[key]
+
+
+class _MergeAssay:
+ def __init__(
+ self,
+ name,
+ counts,
+ cell_ids,
+ feature_ids,
+ feature_names,
+ block_size,
+ ):
+ self.name = name
+ self.rawData = ChunkedArray.from_numpy(
+ np.asarray(counts),
+ block_size=block_size,
+ )
+ self.cells = _MergeMeta(
+ ids=cell_ids,
+ names=cell_ids,
+ I=np.ones(len(cell_ids), dtype=bool),
+ )
+ self.feats = _MergeMeta(ids=feature_ids, names=feature_names)
+
+
+class _MergeDataStore:
+ def __init__(self, assays):
+ self._assays = {assay.name: assay for assay in assays}
+ self.assay_names = list(self._assays)
+ self.cells = assays[0].cells
+
+ def get_assay(self, name):
+ return self._assays[name]
+
+
+@pytest.fixture
+def tiny_merge_budget():
+ previous = get_resource_budget()
+ set_resource_budget(ResourceBudget(memoryBytes=96, workers=2, workingCopies=2))
+ try:
+ yield
+ finally:
+ set_resource_budget(previous)
+
+
+def test_assay_merge(datastore, rna_raw_total, tmp_path):
+ fn = str(tmp_path / "merged.zarr")
+ writer = AssayMerge(
+ zarr_path=fn,
+ assays=[datastore.RNA, datastore.RNA],
+ names=["self1", "self2"],
+ merge_assay_name="RNA",
+ prepend_text="",
+ overwrite=True,
+ )
+ writer.dump()
+ tmp = zarr.open(fn + "/RNA/counts")
+ assert tmp.shape[0] == 2 * datastore.cells.N
+ assert int(tmp[...].sum()) == rna_raw_total * 2
+
+
+def test_assay_merge_maps_features_and_preserves_row_order(
+ tmp_path,
+ tiny_merge_budget,
+):
+ cell_ids = ["c0", "c1", "c2"]
+ left = _MergeAssay(
+ "RNA",
+ [[1, 10], [2, 20], [0, 0]],
+ cell_ids,
+ ["id_a", "id_b"],
+ ["A", "B"],
+ block_size=1,
+ )
+ right = _MergeAssay(
+ "RNA",
+ [[30, 300], [40, 400], [50, 500]],
+ cell_ids,
+ ["id_b", "id_c"],
+ ["B", "C"],
+ block_size=3,
+ )
+ fn = str(tmp_path / "feature_order.zarr")
+ writer = AssayMerge(
+ zarr_path=fn,
+ assays=[left, right],
+ names=["left", "right"],
+ merge_assay_name="RNA",
+ prepend_text="",
+ seed=3,
+ )
+ writer.dump()
+
+ root = zarr.open_group(fn, mode="r")
+ counts_array = root["RNA/counts"]
+ counts = np.asarray(counts_array[:])
+ merged_cell_ids = np.asarray(root["cellData/ids"][:]).astype(str)
+ merged_feature_ids = np.asarray(root["RNA/featureData/ids"][:]).astype(str)
+
+ np.testing.assert_array_equal(merged_feature_ids, ["id_a", "id_b", "id_c"])
+ expected = {
+ "left__c0": [1, 10, 0],
+ "left__c1": [2, 20, 0],
+ "left__c2": [0, 0, 0],
+ "right__c0": [0, 30, 300],
+ "right__c1": [0, 40, 400],
+ "right__c2": [0, 50, 500],
+ }
+ for cell_id, row in zip(merged_cell_ids, counts, strict=True):
+ np.testing.assert_array_equal(row, expected[cell_id])
+
+ spec = count_array_spec(*counts_array.shape, dtype=counts_array.dtype)
+ assert spec.shards is not None
+ assert spec.shards[0] < counts_array.shape[0]
+ assert counts_array.chunks == spec.chunks
+ assert counts_array.metadata.shards == spec.shards
+ assert counts_array.fill_value == 0
+
+
+def test_assay_merge_preserves_order_when_prefetch_finishes_out_of_order(
+ monkeypatch,
+ tmp_path,
+ tiny_merge_budget,
+):
+ import scarf.merge as merge_module
+
+ cell_ids = ["c0", "c1", "c2", "c3"]
+ left = _MergeAssay(
+ "RNA",
+ [[1, 10], [2, 20], [3, 30], [4, 40]],
+ cell_ids,
+ ["id_a", "id_b"],
+ ["A", "B"],
+ block_size=1,
+ )
+ right = _MergeAssay(
+ "RNA",
+ [[50, 500], [60, 600], [70, 700], [80, 800]],
+ cell_ids,
+ ["id_b", "id_c"],
+ ["B", "C"],
+ block_size=4,
+ )
+ first_started = threading.Event()
+ second_finished = threading.Event()
+ lock = threading.Lock()
+ call_count = 0
+ completed: list[int] = []
+ original_compute = merge_module.controlled_compute
+
+ def delayed_compute(arr, nthreads):
+ nonlocal call_count
+ with lock:
+ call_index = call_count
+ call_count += 1
+ if call_index == 0:
+ first_started.set()
+ if not second_finished.wait(timeout=2):
+ raise RuntimeError("Second prefetched block did not complete")
+ elif call_index == 1:
+ if not first_started.wait(timeout=2):
+ raise RuntimeError("First prefetched block did not start")
+
+ result = original_compute(arr, nthreads)
+ with lock:
+ completed.append(call_index)
+ if call_index == 1:
+ second_finished.set()
+ return result
+
+ monkeypatch.setattr(merge_module, "controlled_compute", delayed_compute)
+ fn = str(tmp_path / "out_of_order_prefetch.zarr")
+ writer = AssayMerge(
+ zarr_path=fn,
+ assays=[left, right],
+ names=["left", "right"],
+ merge_assay_name="RNA",
+ prepend_text="",
+ seed=3,
+ )
+ writer.dump()
+
+ assert completed.index(1) < completed.index(0)
+ root = zarr.open_group(fn, mode="r")
+ counts = np.asarray(root["RNA/counts"][:])
+ merged_cell_ids = np.asarray(root["cellData/ids"][:]).astype(str)
+ expected = {
+ "left__c0": [1, 10, 0],
+ "left__c1": [2, 20, 0],
+ "left__c2": [3, 30, 0],
+ "left__c3": [4, 40, 0],
+ "right__c0": [0, 50, 500],
+ "right__c1": [0, 60, 600],
+ "right__c2": [0, 70, 700],
+ "right__c3": [0, 80, 800],
+ }
+ for cell_id, row in zip(merged_cell_ids, counts, strict=True):
+ np.testing.assert_array_equal(row, expected[cell_id])
+
+
+def test_dask_to_coo_sums_consolidated_features():
+ merge = object.__new__(AssayMerge)
+ merge.nFeats = 1
+
+ result = AssayMerge._dask_to_coo(
+ merge,
+ np.array([[2, 3], [5, 7]]),
+ np.array([0, 1]),
+ np.array([0, 0]),
+ 1,
+ )
+
+ assert result.nnz == 2
+ np.testing.assert_array_equal(result.toarray(), [[5], [12]])
+
+
+def test_dataset_merge_shares_row_order_across_assay_chunk_sizes(
+ tmp_path,
+ tiny_merge_budget,
+):
+ from scarf.merge import DatasetMerge
+
+ cell_ids = ["c0", "c1", "c2", "c3", "c4"]
+ left = _MergeDataStore(
+ [
+ _MergeAssay(
+ "RNA",
+ [[1, 10], [2, 20], [3, 30], [4, 40], [5, 50]],
+ cell_ids,
+ ["rna_a", "rna_b"],
+ ["RNA A", "RNA B"],
+ block_size=3,
+ ),
+ _MergeAssay(
+ "ADT",
+ [[101, 110], [102, 120], [103, 130], [104, 140], [105, 150]],
+ cell_ids,
+ ["adt_a", "adt_b"],
+ ["ADT A", "ADT B"],
+ block_size=2,
+ ),
+ ]
+ )
+ right = _MergeDataStore(
+ [
+ _MergeAssay(
+ "RNA",
+ [[6, 60], [7, 70], [8, 80], [9, 90], [10, 100]],
+ cell_ids,
+ ["rna_a", "rna_b"],
+ ["RNA A", "RNA B"],
+ block_size=3,
+ ),
+ _MergeAssay(
+ "ADT",
+ [[106, 160], [107, 170], [108, 180], [109, 190], [110, 200]],
+ cell_ids,
+ ["adt_a", "adt_b"],
+ ["ADT A", "ADT B"],
+ block_size=2,
+ ),
+ ]
+ )
+ fn = str(tmp_path / "mixed_assay_chunks.zarr")
+
+ writer = DatasetMerge(
+ datasets=[left, right],
+ zarr_path=fn,
+ names=["left", "right"],
+ prepend_text="",
+ seed=3,
+ )
+ writer.dump()
+
+ assert writer.unique_assays == ["RNA", "ADT"]
+ for generator in writer.merge_generators:
+ assert (
+ max(
+ rows.size
+ for blocks in generator.permutations_rows.values()
+ for rows in blocks.values()
+ )
+ <= 2
+ )
+ root = zarr.open_group(fn, mode="r")
+ merged_cell_ids = np.asarray(root["cellData/ids"][:]).astype(str)
+ rna = np.asarray(root["RNA/counts"][:])
+ adt = np.asarray(root["ADT/counts"][:])
+ expected = {
+ "left__c0": ([1, 10], [101, 110]),
+ "left__c1": ([2, 20], [102, 120]),
+ "left__c2": ([3, 30], [103, 130]),
+ "left__c3": ([4, 40], [104, 140]),
+ "left__c4": ([5, 50], [105, 150]),
+ "right__c0": ([6, 60], [106, 160]),
+ "right__c1": ([7, 70], [107, 170]),
+ "right__c2": ([8, 80], [108, 180]),
+ "right__c3": ([9, 90], [109, 190]),
+ "right__c4": ([10, 100], [110, 200]),
+ }
+ for cell_id, rna_row, adt_row in zip(
+ merged_cell_ids,
+ rna,
+ adt,
+ strict=True,
+ ):
+ expected_rna, expected_adt = expected[cell_id]
+ np.testing.assert_array_equal(rna_row, expected_rna)
+ np.testing.assert_array_equal(adt_row, expected_adt)
+
+
+def test_dataset_merge_shared_row_plan_handles_missing_assays(
+ tmp_path,
+ tiny_merge_budget,
+):
+ from scarf.merge import DatasetMerge
+
+ cell_ids = ["c0", "c1", "c2"]
+ left = _MergeDataStore(
+ [
+ _MergeAssay(
+ "RNA",
+ [[1, 10], [2, 20], [3, 30]],
+ cell_ids,
+ ["rna_a", "rna_b"],
+ ["RNA A", "RNA B"],
+ block_size=3,
+ )
+ ]
+ )
+ right = _MergeDataStore(
+ [
+ _MergeAssay(
+ "ADT",
+ [[101, 110], [102, 120], [103, 130]],
+ cell_ids,
+ ["adt_a", "adt_b"],
+ ["ADT A", "ADT B"],
+ block_size=2,
+ )
+ ]
+ )
+ fn = str(tmp_path / "missing_assays.zarr")
+
+ writer = DatasetMerge(
+ datasets=[left, right],
+ zarr_path=fn,
+ names=["left", "right"],
+ prepend_text="",
+ seed=3,
+ )
+ writer.dump()
+
+ root = zarr.open_group(fn, mode="r")
+ merged_cell_ids = np.asarray(root["cellData/ids"][:]).astype(str)
+ rna = np.asarray(root["RNA/counts"][:])
+ adt = np.asarray(root["ADT/counts"][:])
+ expected = {
+ "left__c0": ([1, 10], [0, 0]),
+ "left__c1": ([2, 20], [0, 0]),
+ "left__c2": ([3, 30], [0, 0]),
+ "right__c0": ([0, 0], [101, 110]),
+ "right__c1": ([0, 0], [102, 120]),
+ "right__c2": ([0, 0], [103, 130]),
+ }
+ for cell_id, rna_row, adt_row in zip(
+ merged_cell_ids,
+ rna,
+ adt,
+ strict=True,
+ ):
+ expected_rna, expected_adt = expected[cell_id]
+ np.testing.assert_array_equal(rna_row, expected_rna)
+ np.testing.assert_array_equal(adt_row, expected_adt)
+
+
+def test_dataset_merge_2(datastore, rna_raw_total, assay2_raw_total, tmp_path):
+ from scarf.merge import DatasetMerge
+
+ fn = str(tmp_path / "merged.zarr")
+ writer = DatasetMerge(
+ zarr_path=fn,
+ datasets=[datastore, datastore],
+ names=["self1", "self2"],
+ prepend_text="",
+ overwrite=True,
+ )
+ writer.dump()
+ rna_count = zarr.open(fn + "/RNA/counts")
+ assay2_count = zarr.open(fn + "/assay2/counts")
+ assert rna_count.shape[0] == 2 * datastore.cells.N
+ assert assay2_count.shape[0] == 2 * datastore.cells.N
+ assert int(rna_count[...].sum()) == rna_raw_total * 2
+ assert int(assay2_count[...].sum()) == assay2_raw_total * 2
+
+
+def test_dataset_merge_3(datastore, rna_raw_total, assay2_raw_total, tmp_path):
+ from scarf.merge import DatasetMerge
+
+ fn = str(tmp_path / "merged.zarr")
+ writer = DatasetMerge(
+ zarr_path=fn,
+ datasets=[datastore, datastore, datastore],
+ names=["self1", "self2", "self3"],
+ prepend_text="",
+ overwrite=True,
+ )
+ writer.dump()
+ rna_count = zarr.open(fn + "/RNA/counts")
+ assay2_count = zarr.open(fn + "/assay2/counts")
+ assert rna_count.shape[0] == 3 * datastore.cells.N
+ assert assay2_count.shape[0] == 3 * datastore.cells.N
+ assert int(rna_count[...].sum()) == rna_raw_total * 3
+ assert int(assay2_count[...].sum()) == assay2_raw_total * 3
+
+
+def test_dataset_merge_cells(datastore, tmp_path):
+ from scarf.datastore.datastore import DataStore
+ from scarf.merge import DatasetMerge
+
+ fn = str(tmp_path / "merged.zarr")
+ writer = DatasetMerge(
+ zarr_path=fn,
+ datasets=[datastore, datastore],
+ names=["self1", "self2"],
+ prepend_text="orig",
+ overwrite=True,
+ )
+ writer.dump()
+
+ ds = DataStore(
+ fn,
+ default_assay="RNA",
+ )
+
+ df = ds.cells.to_pandas_dataframe(ds.cells.columns)
+ df_diff = df[df["orig_RNA_nCounts"] != df["RNA_nCounts"]]
+ assert len(df_diff) == 0
+
+
+def test_assay_merge_rejects_duplicate_sample_names(datastore, tmp_path):
+ fn = str(tmp_path / "merged_dup_names.zarr")
+ with pytest.raises(ValueError, match="unique name"):
+ AssayMerge(
+ zarr_path=fn,
+ assays=[datastore.RNA, datastore.RNA],
+ names=["dup", "dup"],
+ merge_assay_name="RNA",
+ prepend_text="",
+ overwrite=True,
+ )
+
+
+def test_assay_merge_rejects_existing_workspace_assay(
+ datastore,
+ rna_raw_total,
+ tmp_path,
+):
+ fn = str(tmp_path / "workspace_merged.zarr")
+ writer = AssayMerge(
+ zarr_path=fn,
+ assays=[datastore.RNA, datastore.RNA],
+ names=["self1", "self2"],
+ merge_assay_name="RNA",
+ out_workspace="merged",
+ prepend_text="",
+ )
+ writer.dump()
+
+ root = zarr.open_group(fn, mode="r")
+ counts = root["matrices/RNA/counts"]
+ assert counts.shape[0] == 2 * datastore.cells.N
+ assert int(counts[...].sum()) == 2 * rna_raw_total
+
+ with pytest.raises(ValueError, match="already contains RNA assay"):
+ AssayMerge(
+ zarr_path=fn,
+ assays=[datastore.RNA, datastore.RNA],
+ names=["self1", "self2"],
+ merge_assay_name="RNA",
+ out_workspace="merged",
+ prepend_text="",
+ )
+
+
+def test_assay_merge_validation_failure_does_not_overwrite_store(monkeypatch):
+ calls = []
+ root = zarr.open_group(store=MemoryStore(), mode="w")
+ root.create_group("cellData")
+ root.create_group("RNA")
+
+ def fake_load_zarr(zarr_loc, mode, storage_options=None):
+ calls.append(mode)
+ if mode == "r":
+ return root
+ pytest.fail(f"Unexpected destructive open mode: {mode}")
+
+ monkeypatch.setattr("scarf.merge.load_zarr", fake_load_zarr)
+ merge = object.__new__(AssayMerge)
+ merge.outWorkspace = None
+ merge.storage_options = None
+
+ with pytest.raises(ValueError, match="already contains RNA assay"):
+ AssayMerge._use_existing_zarr(merge, MemoryStore(), "RNA", False)
+
+ assert calls == ["r"]
+
+
+def test_assay_merge_store_skips_local_exists_guard(monkeypatch):
+ calls = []
+
+ def fake_load_zarr(zarr_loc, mode, storage_options=None):
+ calls.append((zarr_loc, mode, storage_options))
+ if mode == "r":
+ raise FileNotFoundError("missing")
+ return zarr.open_group(store=MemoryStore(), mode="w")
+
+ monkeypatch.setattr("scarf.merge.load_zarr", fake_load_zarr)
+ merge = object.__new__(AssayMerge)
+ merge.outWorkspace = None
+ merge.storage_options = {"region": "us-east-1"}
+ root = AssayMerge._use_existing_zarr(merge, MemoryStore(), "RNA", False)
+ assert isinstance(root, zarr.Group)
+ assert calls[-1] == (calls[-1][0], "w", {"region": "us-east-1"})
+
+
+def test_dummy_assay_holds_zero_counts(datastore):
+ from scarf.merge import DummyAssay
+ from scarf.writers import create_zarr_dataset
+
+ mem = zarr.open_group(store=MemoryStore(), mode="w")
+ dummy_array = create_zarr_dataset(
+ mem,
+ "counts",
+ datastore.RNA.rawData.chunksize,
+ datastore.RNA.rawData.dtype,
+ (datastore.cells.N, datastore.RNA.feats.N),
+ )
+ dummy = DummyAssay(
+ datastore,
+ ChunkedArray(dummy_array, nthreads=1),
+ datastore.RNA.feats,
+ "RNA",
+ )
+ assert dummy.name == "RNA"
+ assert int(dummy.rawData.compute().sum()) == 0
diff --git a/scarf/tests/test_metadata.py b/tests/test_metadata.py
similarity index 70%
rename from scarf/tests/test_metadata.py
rename to tests/test_metadata.py
index c5ffc0cd..3f007179 100644
--- a/scarf/tests/test_metadata.py
+++ b/tests/test_metadata.py
@@ -1,23 +1,22 @@
import numpy as np
import pytest
-from . import full_path, remove
-
@pytest.fixture
-def dummy_metadata():
+def dummy_metadata(tmp_path):
import zarr
- from ..metadata import MetaData
- fn = full_path("dummy_metadata.zarr")
- remove(fn)
- g = zarr.open(fn)
+ from scarf.metadata import MetaData
+
+ fn = str(tmp_path / "dummy_metadata.zarr")
+ g = zarr.open_group(fn, mode="w")
data = np.array([1, 1, 1, 1, 0, 0, 1, 1, 1]).astype(bool)
- g.create_dataset(
- "I", data=data, chunks=(100000,), shape=len(data), dtype=data.dtype
+ g.create_array(
+ "I",
+ data=data,
+ chunks=(100000,),
)
yield MetaData(g)
- remove(fn)
def test_metadata_attrs(dummy_metadata):
diff --git a/tests/test_metadata_blocks.py b/tests/test_metadata_blocks.py
new file mode 100644
index 00000000..02485d82
--- /dev/null
+++ b/tests/test_metadata_blocks.py
@@ -0,0 +1,96 @@
+"""Milestone C: MetaData.iter_row_blocks parity and make_bulk cell_key semantics."""
+
+import numpy as np
+
+
+def test_iter_row_blocks_matches_active_index_and_fetch(datastore):
+ cells = datastore.cells
+ col = "ids"
+ expected_idx = cells.active_index("I")
+ expected_vals = cells.fetch(col, key="I")
+
+ for block_rows in (None, 7, 1, cells.default_block_rows("I")):
+ blocks = list(
+ cells.iter_row_blocks(cell_key="I", columns=[col], block_rows=block_rows)
+ )
+ got_idx = (
+ np.concatenate([b.active_global_indices for b in blocks])
+ if blocks
+ else np.array([], dtype=np.int64)
+ )
+ got_vals = (
+ np.concatenate([b.values[col] for b in blocks]) if blocks else np.array([])
+ )
+ np.testing.assert_array_equal(got_idx, expected_idx)
+ np.testing.assert_array_equal(got_vals, expected_vals)
+ assert all(b.start < b.stop for b in blocks)
+ assert blocks[0].start == 0
+ assert blocks[-1].stop == cells.N
+
+
+def test_iter_row_blocks_chunk_edges_cover_full_range(datastore):
+ cells = datastore.cells
+ chunk = cells.default_block_rows("I")
+ blocks = list(cells.iter_row_blocks(cell_key="I", block_rows=chunk))
+ assert sum(b.stop - b.start for b in blocks) == cells.N
+ for b in blocks:
+ assert b.stop - b.start <= chunk
+
+
+def test_iter_row_blocks_respects_subset_cell_key(datastore):
+ cells = datastore.cells
+ keep = np.zeros(cells.N, dtype=bool)
+ keep[::3] = True
+ cells.insert("block_subset", keep, overwrite=True)
+ expected = cells.active_index("block_subset")
+ got = np.concatenate(
+ [
+ b.active_global_indices
+ for b in cells.iter_row_blocks(
+ cell_key="block_subset", columns=["ids"], block_rows=11
+ )
+ ]
+ )
+ np.testing.assert_array_equal(got, expected)
+
+
+def test_make_bulk_respects_non_default_cell_key(leiden_clustering, datastore):
+ """Inactive cells that share a group label must not enter the bulk profile."""
+ ds = datastore
+ active = ds.cells.fetch_all("I").copy()
+ assert active.dtype == bool
+ active_idx = np.flatnonzero(active)
+ drop = np.zeros(ds.cells.N, dtype=bool)
+ drop[active_idx[::2]] = True
+ subset = active & ~drop
+ ds.cells.insert("bulk_subset", subset, overwrite=True)
+
+ full = ds.make_bulk(
+ group_key="RNA_leiden_cluster",
+ cell_key="I",
+ aggr_type="sum",
+ remove_empty_features=False,
+ feature_label="index",
+ )
+ sub = ds.make_bulk(
+ group_key="RNA_leiden_cluster",
+ cell_key="bulk_subset",
+ aggr_type="sum",
+ remove_empty_features=False,
+ feature_label="index",
+ )
+ shared = [c for c in sub.columns if c in full.columns]
+ assert shared
+ for c in shared:
+ assert (sub[c].to_numpy() <= full[c].to_numpy() + 1e-6).all()
+
+ from scarf.utils import controlled_compute
+
+ clusters = ds.cells.fetch_all("RNA_leiden_cluster")
+ subset_idx = ds.cells.active_index("bulk_subset")
+ g_val = clusters[subset_idx[0]]
+ col = str(g_val)
+ assert col in sub.columns
+ idx_g = subset_idx[clusters[subset_idx] == g_val]
+ expected = controlled_compute(ds.RNA.rawData[idx_g].sum(axis=0), ds.nthreads)
+ np.testing.assert_allclose(sub[col].to_numpy(), expected, rtol=1e-5, atol=1e-6)
diff --git a/tests/test_metrics.py b/tests/test_metrics.py
new file mode 100644
index 00000000..330d162c
--- /dev/null
+++ b/tests/test_metrics.py
@@ -0,0 +1,376 @@
+import numpy as np
+import pandas as pd
+import pytest
+from scipy.sparse import csr_matrix
+
+from scarf.metrics import (
+ _effective_perplexity,
+ _neighbor_probabilities,
+ calculate_knn_cluster_similarity,
+ calculate_top_k_neighbor_distances,
+ calculate_weighted_cluster_similarity,
+ compute_lisi,
+ integration_score,
+ knn_to_csr_matrix,
+ label_concordance_score,
+ lisi_batch_mixing_score,
+ silhouette_scoring,
+)
+
+
+def test_compute_lisi_uses_all_stored_neighbors():
+ metadata = pd.DataFrame({"batch": [1, 0, 0, 0]})
+ indices = np.tile(np.array([0, 1, 2]), (4, 1))
+ distances = np.ones_like(indices, dtype=np.float64)
+
+ scores = compute_lisi(
+ distances,
+ indices,
+ metadata,
+ label_colnames=["batch"],
+ perplexity=1,
+ )
+
+ assert np.allclose(scores[:, 0], 1.8)
+
+
+def test_compute_lisi_rejects_missing_labels():
+ metadata = pd.DataFrame({"batch": [0, 1, np.nan, 1]})
+ indices = np.tile(np.array([0, 1, 2]), (4, 1))
+ distances = np.ones_like(indices, dtype=np.float64)
+
+ with pytest.raises(ValueError, match="missing values"):
+ compute_lisi(distances, indices, metadata, ["batch"])
+
+
+def test_compute_lisi_returns_empty_matrix_when_no_labels_are_requested():
+ metadata = pd.DataFrame(index=np.arange(2))
+ distances = np.empty((2, 0))
+ indices = np.empty((2, 0), dtype=np.int64)
+
+ scores = compute_lisi(distances, indices, metadata, [])
+
+ assert scores.shape == (2, 0)
+ assert scores.dtype == np.float64
+
+
+def test_compute_lisi_handles_more_categories_than_neighbors():
+ metadata = pd.DataFrame({"batch": ["a", "b", "c", "d"]})
+ indices = np.array(
+ [
+ [0, 1, 2],
+ [1, 2, 3],
+ [2, 3, 0],
+ [3, 0, 1],
+ ]
+ )
+ distances = np.ones_like(indices, dtype=np.float64)
+
+ scores = compute_lisi(
+ distances,
+ indices,
+ metadata,
+ label_colnames=["batch"],
+ perplexity=1,
+ )
+
+ assert np.allclose(scores[:, 0], 3.0)
+
+
+def test_compute_lisi_caps_perplexity_to_neighbor_capacity():
+ metadata = pd.DataFrame({"batch": [0, 0, 1, 1]})
+ indices = np.tile(np.array([0, 1, 2, 3, 0, 1]), (4, 1))
+ distances = np.tile(np.arange(6, dtype=np.float64), (4, 1))
+
+ capped = compute_lisi(distances, indices, metadata, ["batch"], perplexity=2)
+ oversized = compute_lisi(distances, indices, metadata, ["batch"], perplexity=30)
+
+ assert np.allclose(oversized, capped)
+
+
+@pytest.mark.parametrize("perplexity", [0.5, np.inf, np.nan])
+def test_effective_perplexity_rejects_invalid_values(perplexity):
+ with pytest.raises(ValueError, match="finite value"):
+ _effective_perplexity(perplexity, n_neighbors=3)
+
+
+def test_effective_perplexity_requires_three_neighbors():
+ with pytest.raises(ValueError, match="at least three"):
+ _effective_perplexity(perplexity=1, n_neighbors=2)
+
+
+def test_neighbor_probabilities_calibrate_diffuse_and_concentrated_rows():
+ distances = np.array(
+ [
+ [0.0, 0.1, 0.2, 0.3],
+ [0.0, 10.0, 20.0, 30.0],
+ ]
+ )
+
+ probabilities = _neighbor_probabilities(
+ distances,
+ perplexity=2,
+ tol=1e-8,
+ max_iter=100,
+ )
+ log_probabilities = np.log(
+ probabilities,
+ out=np.zeros_like(probabilities),
+ where=probabilities > 0,
+ )
+ calibrated_perplexity = np.exp(-np.sum(probabilities * log_probabilities, axis=1))
+
+ assert np.allclose(probabilities.sum(axis=1), 1.0)
+ assert np.allclose(calibrated_perplexity, 2.0, rtol=1e-7)
+
+
+def test_compute_lisi_rejects_invalid_neighbor_distances():
+ metadata = pd.DataFrame({"batch": [0, 0, 1, 1]})
+ indices = np.tile(np.array([0, 1, 2]), (4, 1))
+ distances = np.ones_like(indices, dtype=np.float64)
+ distances[0, 0] = -1
+
+ with pytest.raises(ValueError, match="finite, non-negative"):
+ compute_lisi(distances, indices, metadata, ["batch"], perplexity=1)
+
+
+def test_metric_lisi_single_category_is_one(datastore, make_graph):
+ labels = np.zeros(datastore.cells.N, dtype=np.int8)
+ datastore.cells.insert(
+ column_name="single_batch",
+ values=labels,
+ overwrite=True,
+ )
+ lisi = datastore.metric_lisi(
+ label_colnames=(name for name in ["single_batch"]),
+ save_result=False,
+ return_lisi=True,
+ )
+
+ assert lisi is not None
+ assert np.allclose(lisi[0][1], 1)
+
+
+def test_knn_affinities_decrease_with_distance():
+ indices = np.array([[1, 2], [0, 2], [3, 0], [2, 1]])
+ distances = np.array([[0.1, 10.0], [0.1, 3.0], [0.2, 4.0], [0.2, 5.0]])
+
+ graph = knn_to_csr_matrix(indices, distances, use_affinities=True)
+
+ assert graph[0, 1] > graph[0, 2]
+
+
+def test_cluster_similarity_is_symmetric_with_unit_diagonal():
+ graph = csr_matrix(
+ np.array(
+ [
+ [0.0, 1.0, 0.2, 0.0],
+ [1.0, 0.0, 0.3, 0.0],
+ [0.2, 0.3, 0.0, 1.0],
+ [0.0, 0.0, 1.0, 0.0],
+ ]
+ )
+ )
+ labels = np.array([0, 0, 1, 1])
+
+ similarities = calculate_weighted_cluster_similarity(graph, labels)
+
+ assert np.allclose(similarities, similarities.T)
+ assert np.allclose(np.diag(similarities), 1)
+ assert similarities[0, 1] > 0
+
+
+def test_streamed_knn_similarity_matches_csr_similarity():
+ indices = np.array([[1, 2], [0, 2], [3, 0], [2, 1]])
+ distances = np.array([[0.1, 10.0], [0.1, 3.0], [0.2, 4.0], [0.2, 5.0]])
+ labels = np.array([0, 0, 1, 1])
+ graph = knn_to_csr_matrix(indices, distances, use_affinities=True)
+
+ streamed = calculate_knn_cluster_similarity(
+ indices,
+ distances,
+ labels,
+ batch_rows=2,
+ )
+ materialized = calculate_weighted_cluster_similarity(graph, labels)
+
+ assert np.allclose(streamed, materialized)
+
+
+def test_top_k_distances_accept_all_candidates():
+ distances = calculate_top_k_neighbor_distances(
+ np.array([[0.0]]),
+ np.array([[1.0], [2.0]]),
+ k=2,
+ )
+
+ assert np.allclose(np.sort(distances[0]), [1.0, 2.0])
+
+
+def test_top_k_cosine_distances_handle_opposite_and_zero_vectors():
+ distances = calculate_top_k_neighbor_distances(
+ np.array([[1.0, 0.0], [0.0, 0.0]]),
+ np.array([[1.0, 0.0], [0.0, 1.0], [-1.0, 0.0]]),
+ k=10,
+ metric="cosine",
+ )
+
+ assert np.allclose(
+ np.sort(distances, axis=1),
+ np.array(
+ [
+ [0.0, 1.0, 2.0],
+ [1.0, 1.0, 1.0],
+ ]
+ ),
+ )
+
+
+def test_top_k_inner_product_distances_are_clipped_at_zero():
+ distances = calculate_top_k_neighbor_distances(
+ np.array([[2.0, 0.0], [0.5, 0.0]]),
+ np.array([[1.0, 0.0], [0.25, 0.0], [-1.0, 0.0]]),
+ k=3,
+ metric="ip",
+ )
+
+ assert np.allclose(
+ np.sort(distances, axis=1),
+ np.array(
+ [
+ [0.0, 0.5, 3.0],
+ [0.5, 0.875, 1.5],
+ ]
+ ),
+ )
+
+
+def test_top_k_distances_reject_unsupported_metric():
+ with pytest.raises(ValueError, match="Unsupported neighbor metric"):
+ calculate_top_k_neighbor_distances(
+ np.array([[0.0]]),
+ np.array([[1.0]]),
+ k=1,
+ metric="manhattan",
+ )
+
+
+def test_metric_silhouette(datastore, make_graph, leiden_clustering):
+ scores = datastore.metric_silhouette(random_seed=42)
+ scores_from_full_label = datastore.metric_silhouette(
+ res_label="RNA_leiden_cluster",
+ random_seed=42,
+ )
+
+ assert scores is not None
+ assert scores_from_full_label is not None
+ assert np.array_equal(scores, scores_from_full_label, equal_nan=True)
+ assert np.isfinite(scores).any()
+ assert np.all(np.abs(scores[np.isfinite(scores)]) <= 1)
+
+
+def test_small_cluster_does_not_invalidate_other_silhouette_scores():
+ class Cells:
+ columns = ["RNA_subset_cluster"]
+
+ @staticmethod
+ def fetch(column, key="I"):
+ assert column == "RNA_subset_cluster"
+ assert key == "subset"
+ return np.array([1, 2, 2, 2, 2, 3, 3, 3, 3])
+
+ class Store:
+ cells = Cells()
+
+ class Ann:
+ annMetric = "l2"
+
+ @staticmethod
+ def reducer(values):
+ return values
+
+ data = np.array(
+ [
+ [20.0, 20.0],
+ [0.0, 0.0],
+ [0.0, 0.1],
+ [0.1, 0.0],
+ [0.1, 0.1],
+ [10.0, 10.0],
+ [10.0, 10.1],
+ [10.1, 10.0],
+ [10.1, 10.1],
+ ]
+ )
+ graph = csr_matrix(np.ones((len(data), len(data))) - np.eye(len(data)))
+
+ scores = silhouette_scoring(
+ Store(),
+ Ann(),
+ graph,
+ data,
+ "RNA",
+ "cluster",
+ cell_key="subset",
+ sample_size=2,
+ random_seed=42,
+ )
+
+ assert scores is not None
+ assert np.isnan(scores[0])
+ assert np.isfinite(scores[1:]).all()
+
+
+@pytest.mark.parametrize("metric", ["ari", "nmi"])
+def test_label_concordance_identical_partitions(metric):
+ labels = np.array([0, 0, 1, 1])
+
+ assert label_concordance_score([labels, labels], metric) == pytest.approx(1)
+ assert integration_score([labels, labels], metric) == pytest.approx(1)
+
+
+def test_label_concordance_requires_exactly_two_partitions():
+ with pytest.raises(ValueError, match="Exactly two"):
+ label_concordance_score([np.array([0, 1])])
+
+
+def test_lisi_batch_mixing_score():
+ labels = np.array([0, 0, 1, 1])
+
+ assert lisi_batch_mixing_score(np.ones(4), labels) == pytest.approx(0)
+ assert lisi_batch_mixing_score(np.full(4, 2.0), labels) == pytest.approx(1)
+
+
+def test_metric_label_concordance(datastore, make_graph, leiden_clustering):
+ rng = np.random.default_rng(42)
+ labels1 = rng.integers(0, 2, datastore.cells.N)
+ labels2 = labels1.copy()
+ datastore.cells.insert(
+ column_name="labels1",
+ values=labels1,
+ overwrite=True,
+ )
+ datastore.cells.insert(
+ column_name="labels2",
+ values=labels2,
+ overwrite=True,
+ )
+
+ assert datastore.metric_label_concordance(["labels1", "labels2"]) == pytest.approx(
+ 1
+ )
+ assert datastore.metric_integration(["labels1", "labels2"]) == pytest.approx(1)
+ mixing_score = datastore.metric_batch_mixing("labels1")
+ assert 0 <= mixing_score <= 1
+
+
+def test_silhouette_scoring_missing_cluster_labels(datastore):
+ result = silhouette_scoring(
+ datastore,
+ ann_obj=None,
+ graph=None,
+ hvg_data=None,
+ assay_type="RNA",
+ res_label="missing_resolution_label",
+ )
+ assert result is None
diff --git a/scarf/tests/test_misc.py b/tests/test_misc.py
similarity index 54%
rename from scarf/tests/test_misc.py
rename to tests/test_misc.py
index d20bc822..962c0507 100644
--- a/scarf/tests/test_misc.py
+++ b/tests/test_misc.py
@@ -1,8 +1,5 @@
-from . import full_path, remove
-
-
-def test_export_knn_to_mtx(datastore, make_graph):
- from ..knn_utils import export_knn_to_mtx
+def test_export_knn_to_mtx(datastore, make_graph, tmp_path):
+ from scarf.knn_utils import export_knn_to_mtx
graph = datastore.load_graph(
from_assay="RNA",
@@ -11,7 +8,6 @@ def test_export_knn_to_mtx(datastore, make_graph):
symmetric=False,
upper_only=False,
)
- fn = full_path("test_export_mtx_from_graph.mtx")
+ fn = str(tmp_path / "test_export_mtx_from_graph.mtx")
ret_val = export_knn_to_mtx(fn, graph)
assert ret_val is None
- remove(fn)
diff --git a/tests/test_parallel.py b/tests/test_parallel.py
new file mode 100644
index 00000000..16b870d7
--- /dev/null
+++ b/tests/test_parallel.py
@@ -0,0 +1,192 @@
+import threading
+import time
+
+import numpy as np
+import pytest
+import zarr
+from zarr.storage import MemoryStore
+
+from scarf.chunked import ChunkedArray
+from scarf.parallel import in_shard_context, map_shards, stream_shards
+from scarf.storage.budget import (
+ READ_AHEAD,
+ ResourceBudget,
+ set_resource_budget,
+ shard_parallelism,
+)
+
+
+@pytest.fixture
+def budget():
+ set_resource_budget(
+ ResourceBudget(memoryBytes=32 * 1024**3, workers=8, workingCopies=8)
+ )
+ yield
+ set_resource_budget(None)
+
+
+def test_shard_parallelism_spends_budget_within_shard(budget):
+ plan = shard_parallelism(workers=8, n_shards=20)
+ assert plan.readAhead == READ_AHEAD
+ assert plan.ioConcurrency == 8
+ assert plan.withinBlockThreads == 1
+
+
+def test_shard_parallelism_read_ahead_capped_by_shard_count(budget):
+ plan = shard_parallelism(workers=8, n_shards=1)
+ assert plan.readAhead == 1
+
+
+def test_shard_parallelism_read_ahead_capped_by_working_copies():
+ tight = ResourceBudget(memoryBytes=32 * 1024**3, workers=8, workingCopies=1)
+ plan = shard_parallelism(workers=8, n_shards=100, budget=tight)
+ assert plan.readAhead == 1
+ assert plan.ioConcurrency == 8
+ assert plan.withinBlockThreads == 1
+
+
+def test_map_shards_preserves_order(budget):
+ ranges = [(i * 10, i * 10 + 10) for i in range(6)]
+ out = map_shards(ranges, lambda idx, s, e: (idx, s, e), workers=8)
+ assert out == [(i, i * 10, i * 10 + 10) for i in range(6)]
+
+
+def test_map_shards_empty(budget):
+ assert map_shards([], lambda i, s, e: i, workers=8) == []
+
+
+def test_map_shards_bounds_in_flight(budget):
+ lock = threading.Lock()
+ in_flight = 0
+ max_seen = 0
+ ranges = [(i, i + 1) for i in range(16)]
+
+ def produce(idx, s, e):
+ nonlocal in_flight, max_seen
+ with lock:
+ in_flight += 1
+ max_seen = max(max_seen, in_flight)
+ time.sleep(0.01)
+ with lock:
+ in_flight -= 1
+ return idx
+
+ map_shards(ranges, produce, workers=8)
+ assert max_seen <= READ_AHEAD
+
+
+def test_map_shards_serial_backend_runs_inline(budget):
+ seen_context = []
+
+ def produce(idx, s, e):
+ seen_context.append(in_shard_context())
+ return idx
+
+ out = map_shards([(0, 1), (1, 2)], produce, workers=8, backend="serial")
+ assert out == [0, 1]
+
+
+def test_nested_map_shards_runs_serial(budget):
+ inner_context = []
+
+ def outer(idx, s, e):
+ assert in_shard_context() is True
+
+ def inner(j, a, b):
+ inner_context.append(in_shard_context())
+ return j
+
+ return map_shards([(0, 1), (1, 2), (2, 3)], inner, workers=8)
+
+ map_shards([(0, 1), (1, 2)], outer, workers=8)
+ assert inner_context and all(inner_context)
+
+
+def test_stream_shards_preserves_order(budget):
+ out = list(stream_shards(range(5), lambda x: x * 2, workers=4))
+ assert out == [0, 2, 4, 6, 8]
+
+
+def test_stream_shards_bounds_and_restores_io_concurrency():
+ with zarr.config.set({"async.concurrency": 7}):
+ seen = []
+
+ def fn(x):
+ seen.append(zarr.config.get("async.concurrency"))
+ return x
+
+ out = list(stream_shards(range(4), fn, workers=2, io_concurrency=3))
+ assert out == [0, 1, 2, 3]
+ assert seen and all(s == 3 for s in seen)
+ assert zarr.config.get("async.concurrency") == 7
+
+
+def test_stream_shards_config_neutral_without_io():
+ with zarr.config.set({"async.concurrency": 5}):
+ seen = []
+
+ def fn(x):
+ seen.append(zarr.config.get("async.concurrency"))
+ return x
+
+ list(stream_shards(range(4), fn, workers=2))
+ assert seen and all(s == 5 for s in seen)
+
+
+def test_map_shards_sets_io_concurrency_from_plan(budget):
+ with zarr.config.set({"async.concurrency": 99}):
+ seen = []
+
+ def produce(idx, s, e):
+ seen.append(zarr.config.get("async.concurrency"))
+ return idx
+
+ # budget: workers=8; the whole worker budget is spent within a shard, so
+ # async.concurrency is set to 8 for the duration of the op.
+ map_shards([(i, i + 1) for i in range(4)], produce, workers=8)
+ assert seen and all(s == 8 for s in seen)
+ assert zarr.config.get("async.concurrency") == 99
+
+
+def _toy_chunked(data, chunk_rows, nthreads):
+ root = zarr.open_group(store=MemoryStore(), mode="w")
+ arr = root.create_array(
+ "d", shape=data.shape, chunks=(chunk_rows, data.shape[1]), dtype=data.dtype
+ )
+ arr[:] = data
+ return ChunkedArray(arr, nthreads=nthreads)
+
+
+def test_stream_blocks_matches_serial_materialization():
+ rng = np.random.default_rng(0)
+ data = rng.standard_normal((35, 6)).astype(np.float32)
+ serial = np.vstack(list(_toy_chunked(data, 10, 1).stream_blocks(nthreads=1)))
+ parallel = np.vstack(list(_toy_chunked(data, 10, 4).stream_blocks(nthreads=4)))
+ assert np.array_equal(serial, data)
+ assert np.array_equal(serial, parallel)
+
+
+def test_reductions_bit_identical_across_threads():
+ rng = np.random.default_rng(1)
+ data = rng.standard_normal((37, 5)).astype(np.float32)
+ ca1 = _toy_chunked(data, 10, 1)
+ ca4 = _toy_chunked(data, 10, 4)
+ assert np.array_equal(
+ np.asarray(ca1.sum(axis=0).compute(1)),
+ np.asarray(ca4.sum(axis=0).compute(4)),
+ )
+ assert np.array_equal(
+ np.asarray(ca1.var(axis=0).compute(1)),
+ np.asarray(ca4.var(axis=0).compute(4)),
+ )
+ m1, s1 = ca1.mean_and_std(nthreads=1)
+ m4, s4 = ca4.mean_and_std(nthreads=4)
+ assert np.array_equal(m1, m4)
+ assert np.array_equal(s1, s4)
+
+
+def test_compute_matches_source_across_threads():
+ rng = np.random.default_rng(2)
+ data = rng.standard_normal((40, 4)).astype(np.float32)
+ assert np.array_equal(_toy_chunked(data, 8, 1).compute(1), data)
+ assert np.array_equal(_toy_chunked(data, 8, 5).compute(5), data)
diff --git a/tests/test_pcst_fast.py b/tests/test_pcst_fast.py
new file mode 100644
index 00000000..78f58fe6
--- /dev/null
+++ b/tests/test_pcst_fast.py
@@ -0,0 +1,11 @@
+import pcst_fast
+
+
+def test_pcst_fast_sanity_on_numpy2() -> None:
+ """PyPI wheels for pcst-fast 1.0.10 need a source build with pybind11>=2.11."""
+ edges = [[0, 1], [1, 2], [2, 3]]
+ prizes = [1.0, 1.0, 1.0, 1.0]
+ costs = [0.8, 1.8, 2.8]
+ nodes, edge_ids = pcst_fast.pcst_fast(edges, prizes, costs, -1, 1, "strong", 0)
+ assert list(nodes) == [0, 1]
+ assert list(edge_ids) == [0]
diff --git a/tests/test_plots.py b/tests/test_plots.py
new file mode 100644
index 00000000..b292cf5f
--- /dev/null
+++ b/tests/test_plots.py
@@ -0,0 +1,140 @@
+import matplotlib
+import networkx as nx
+import pytest
+
+matplotlib.use("Agg")
+
+import scarf.plots as plots # noqa: E402
+
+
+def test_shade_scatter_continuous_panels_share_linear_limits():
+ import matplotlib.pyplot as plt
+ import numpy as np
+ import pandas as pd
+
+ x = np.linspace(-1.0, 1.0, 30)
+ dfs = [
+ pd.DataFrame({"x": x, "y": x, "value": np.linspace(0.0, 1.0, 30)}),
+ pd.DataFrame({"x": x, "y": -x, "value": np.linspace(10.0, 20.0, 30)}),
+ ]
+ axes = plots.shade_scatter(
+ dfs,
+ pixels=32,
+ n_columns=2,
+ legend_ondata=False,
+ legend_onside=False,
+ show_fig=False,
+ )
+ assert axes is not None
+ artists = [
+ next(child for child in ax.get_children() if hasattr(child, "get_clim"))
+ for ax in axes.flat
+ ]
+ assert artists[0].get_clim() == pytest.approx(artists[1].get_clim())
+ assert artists[0].norm.__class__.__name__ == "Normalize"
+ plt.close(axes.flat[0].figure)
+
+
+def _assert_expected_tree_layout(
+ positions: dict[str, tuple[float, float]],
+) -> None:
+ expected = {
+ "root": (1.2, 1.0),
+ "left": (0.4, 0.5),
+ "leaf": (0.4, 0.0),
+ "right": (2.0, 0.5),
+ }
+ assert positions.keys() == expected.keys()
+ for node, coordinates in expected.items():
+ assert positions[node] == pytest.approx(coordinates)
+
+
+def test_hierarchy_pos_directed_tree_uses_source_as_root():
+ graph = nx.DiGraph(
+ [
+ ("root", "left"),
+ ("root", "right"),
+ ("left", "leaf"),
+ ]
+ )
+
+ positions = plots.hierarchy_pos(
+ graph,
+ width=2.0,
+ vert_gap=0.5,
+ vert_loc=1.0,
+ )
+
+ _assert_expected_tree_layout(positions)
+
+ branch_positions = plots.hierarchy_pos(graph, root="left")
+ assert branch_positions.keys() == {"left", "leaf"}
+ assert branch_positions["left"][1] == 0
+ assert branch_positions["leaf"][1] == pytest.approx(-0.2)
+
+
+def test_hierarchy_pos_undirected_tree_selects_root_deterministically(monkeypatch):
+ graph = nx.Graph(
+ [
+ ("root", "left"),
+ ("root", "right"),
+ ("left", "leaf"),
+ ]
+ )
+ choices = []
+
+ def choose_root(nodes):
+ choices.append(nodes)
+ return "root"
+
+ monkeypatch.setattr(plots.np.random, "choice", choose_root)
+
+ positions = plots.hierarchy_pos(
+ graph,
+ width=2.0,
+ vert_gap=0.5,
+ vert_loc=1.0,
+ )
+
+ assert len(choices) == 1
+ assert set(choices[0]) == set(graph)
+ _assert_expected_tree_layout(positions)
+
+
+@pytest.mark.parametrize(
+ "graph",
+ [
+ pytest.param(nx.cycle_graph(3), id="cycle"),
+ pytest.param(nx.Graph([(0, 1), (2, 3)]), id="disconnected"),
+ ],
+)
+def test_hierarchy_pos_rejects_non_trees(graph):
+ with pytest.raises(
+ TypeError,
+ match="cannot use hierarchy_pos on a graph that is not a tree",
+ ):
+ plots.hierarchy_pos(graph, root=0)
+
+
+@pytest.mark.parametrize(
+ ("titles", "panel_count", "expected"),
+ [
+ (None, 3, None),
+ ("Only panel", 1, ["Only panel"]),
+ (["First", "Second"], 2, ["First", "Second"]),
+ (["Only panel"], 1, ["Only panel"]),
+ ],
+)
+def test_handle_titles_type_accepts_matching_titles(titles, panel_count, expected):
+ assert plots._handle_titles_type(titles, panel_count) == expected
+
+
+@pytest.mark.parametrize(
+ ("titles", "panel_count"),
+ [
+ pytest.param(["Only one"], 2, id="wrong-count"),
+ pytest.param("ab", 2, id="wrong-type"),
+ ],
+)
+def test_handle_titles_type_rejects_invalid_multi_panel_titles(titles, panel_count):
+ assert plots._handle_titles_type(titles, panel_count) is None
diff --git a/tests/test_plotting_dependencies.py b/tests/test_plotting_dependencies.py
new file mode 100644
index 00000000..c59e27c8
--- /dev/null
+++ b/tests/test_plotting_dependencies.py
@@ -0,0 +1,39 @@
+"""Optional plotting dependency error tests."""
+
+import builtins
+
+import pytest
+
+from scarf.plotting._deps import (
+ require_datashader,
+ require_kneed,
+ require_matplotlib,
+ require_seaborn,
+)
+
+
+@pytest.mark.parametrize(
+ ("loader", "blocked_package", "message"),
+ [
+ (require_matplotlib, "matplotlib", "matplotlib"),
+ (require_seaborn, "seaborn", "seaborn"),
+ (require_datashader, "datashader", "datashader"),
+ (require_kneed, "kneed", "kneed"),
+ ],
+)
+def test_optional_dependency_errors_are_actionable(
+ monkeypatch,
+ loader,
+ blocked_package,
+ message,
+):
+ original_import = builtins.__import__
+
+ def blocked_import(name, *args, **kwargs):
+ if name.split(".", 1)[0] == blocked_package:
+ raise ModuleNotFoundError(name)
+ return original_import(name, *args, **kwargs)
+
+ monkeypatch.setattr(builtins, "__import__", blocked_import)
+ with pytest.raises(ImportError, match=message):
+ loader()
diff --git a/tests/test_plotting_export.py b/tests/test_plotting_export.py
new file mode 100644
index 00000000..d5ad8257
--- /dev/null
+++ b/tests/test_plotting_export.py
@@ -0,0 +1,120 @@
+"""Milestone D export and provenance tests."""
+
+import json
+import xml.etree.ElementTree as ET
+
+import matplotlib
+import numpy as np
+import pytest
+from PIL import Image
+
+matplotlib.use("Agg")
+
+import scarf.plotting as splt
+
+
+def test_tiff_export_exact_size_and_provenance_sidecar(
+ umap, leiden_clustering, datastore, tmp_path
+):
+ result = splt.embedding(
+ datastore,
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ figsize=(3.0, 2.0),
+ show=False,
+ )
+ path = tmp_path / "figure.tiff"
+ with matplotlib.rc_context({"savefig.bbox": "tight"}):
+ result.save(path, dpi=120, provenance_sidecar=True)
+
+ with Image.open(path) as image:
+ assert image.format == "TIFF"
+ assert image.size == (360, 240)
+ assert image.tag_v2.get(259) == 5 # LZW compression
+
+ sidecar = tmp_path / "figure.tiff.json"
+ payload = json.loads(sidecar.read_text())
+ assert payload["provenance"]["renderer"] == "matplotlib"
+ assert payload["provenance"]["n_cells"] == len(datastore.cells.active_index("I"))
+ assert payload["figure"]["width_inches"] == pytest.approx(3.0)
+ assert payload["export"] == {
+ "dpi": 120.0,
+ "filename": "figure.tiff",
+ "format": "tiff",
+ }
+ assert "RNA_leiden_cluster" in payload["legends"][0]["label"]
+ result.close()
+
+
+def test_svg_export_preserves_physical_size_under_tight_global_setting(
+ umap, datastore, tmp_path
+):
+ result = splt.embedding(
+ datastore,
+ layout_key="RNA_UMAP",
+ figsize=(3.0, 2.0),
+ show=False,
+ )
+ path = tmp_path / "figure.svg"
+ with matplotlib.rc_context({"savefig.bbox": "tight"}):
+ result.save(path, exact_size=True)
+ root = ET.parse(path).getroot()
+ assert root.attrib["width"] == "216pt"
+ assert root.attrib["height"] == "144pt"
+ result.close()
+
+
+def test_save_provenance_handles_numpy_values(
+ umap, leiden_clustering, datastore, tmp_path
+):
+ result = splt.embedding(
+ datastore,
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ show=False,
+ )
+ result.provenance.extras["array"] = np.array([1, 2], dtype=np.int64)
+ result.provenance.extras["missing"] = np.float64(np.nan)
+ path = result.save_provenance(tmp_path / "plot.json")
+ payload = json.loads(path.read_text())
+ assert payload["provenance"]["extras"]["array"] == [1, 2]
+ assert payload["provenance"]["extras"]["missing"] is None
+ result.close()
+
+
+def test_export_validates_extension_and_dpi(umap, datastore, tmp_path):
+ result = splt.embedding(
+ datastore,
+ layout_key="RNA_UMAP",
+ show=False,
+ )
+ with pytest.raises(ValueError, match="file extension"):
+ result.save(tmp_path / "no_extension")
+ with pytest.raises(ValueError, match="dpi must be positive"):
+ result.save(tmp_path / "plot.png", dpi=0)
+ with pytest.raises(ValueError, match="Unsupported export format"):
+ result.save(tmp_path / "plot.unsupported")
+ cropped = result.save(tmp_path / "plot.png", exact_size=False)
+ assert cropped.exists()
+ result.close()
+
+
+def test_embedding_rasterization_threshold(umap, datastore):
+ vector = splt.embedding(
+ datastore,
+ layout_key="RNA_UMAP",
+ rasterize_threshold=10**9,
+ show=False,
+ )
+ rasterized = splt.embedding(
+ datastore,
+ layout_key="RNA_UMAP",
+ rasterize_threshold=0,
+ show=False,
+ )
+ vector_collection = next(iter(vector.axes.values())).collections[0]
+ raster_collection = next(iter(rasterized.axes.values())).collections[0]
+ assert vector_collection.get_rasterized() is False
+ assert raster_collection.get_rasterized() is True
+ vector.close()
+ rasterized.close()
diff --git a/tests/test_plotting_facades.py b/tests/test_plotting_facades.py
new file mode 100644
index 00000000..a4de06e6
--- /dev/null
+++ b/tests/test_plotting_facades.py
@@ -0,0 +1,96 @@
+"""Delegation tests for specialized plotting facades."""
+
+import numpy as np
+import pandas as pd
+
+import scarf.plots as legacy_plots
+import scarf.plotting as splt
+
+
+def test_diagnostic_facades_forward_public_arguments(monkeypatch):
+ calls = {}
+
+ def plot_qc(**kwargs):
+ calls["qc"] = kwargs
+ return "qc-result"
+
+ def plot_elbow(values, *, figsize):
+ calls["elbow"] = (values, figsize)
+
+ def plot_graph_qc(graph):
+ calls["graph"] = graph
+
+ def plot_mean_var(**kwargs):
+ calls["hvg"] = kwargs
+
+ monkeypatch.setattr(legacy_plots, "plot_qc", plot_qc)
+ monkeypatch.setattr(legacy_plots, "plot_elbow", plot_elbow)
+ monkeypatch.setattr(legacy_plots, "plot_graph_qc", plot_graph_qc)
+ monkeypatch.setattr(legacy_plots, "plot_mean_var", plot_mean_var)
+
+ frame = pd.DataFrame({"groups": ["a"], "metric": [1.0]})
+ assert splt.qc(frame, figsize=(3, 2), show=False) == "qc-result"
+ splt.elbow([0.5, 0.3], figsize=(4, 2))
+ graph = object()
+ splt.graph_qc(graph)
+ splt.highly_variable_features(
+ np.array([1.0]),
+ np.array([2.0]),
+ np.array([3]),
+ np.array([True]),
+ point_sizes=(4, 20),
+ )
+
+ assert calls["qc"]["fig_size"] == (3, 2)
+ assert calls["qc"]["show_fig"] is False
+ assert calls["elbow"] == ([0.5, 0.3], (4, 2))
+ assert calls["graph"] is graph
+ assert calls["hvg"]["ss"] == (4, 20)
+
+
+class _FacadeStore:
+ def __init__(self):
+ self.calls = {}
+
+ def plot_marker_heatmap(self, **kwargs):
+ self.calls["marker"] = kwargs
+
+ def plot_cluster_tree(self, **kwargs):
+ self.calls["tree"] = kwargs
+
+ def plot_pseudotime_heatmap(self, **kwargs):
+ self.calls["pseudotime"] = kwargs
+
+
+def test_heatmap_facades_forward_arguments():
+ store = _FacadeStore()
+ splt.marker_heatmap(
+ store,
+ group_key="cluster",
+ topn=7,
+ show_fig=False,
+ figsize=(4, 6),
+ )
+ target = object()
+ splt.cluster_tree(
+ store,
+ cluster_key="cluster",
+ color_key={"a": "red"},
+ ax=target,
+ show_fig=False,
+ )
+ splt.pseudotime_heatmap(
+ store,
+ pseudotime_key="trajectory",
+ show_features=["GeneA"],
+ show_fig=False,
+ )
+
+ assert store.calls["marker"]["group_key"] == "cluster"
+ assert store.calls["marker"]["topn"] == 7
+ assert store.calls["marker"]["figsize"] == (4, 6)
+ assert store.calls["marker"]["show_fig"] is False
+ assert store.calls["tree"]["ax"] is target
+ assert store.calls["tree"]["color_key"] == {"a": "red"}
+ assert store.calls["pseudotime"]["pseudotime_key"] == "trajectory"
+ assert store.calls["pseudotime"]["show_features"] == ["GeneA"]
diff --git a/tests/test_plotting_figure_helpers.py b/tests/test_plotting_figure_helpers.py
new file mode 100644
index 00000000..366a8224
--- /dev/null
+++ b/tests/test_plotting_figure_helpers.py
@@ -0,0 +1,88 @@
+"""Figure ownership and composition helper tests."""
+
+import matplotlib
+import numpy as np
+import pytest
+
+matplotlib.use("Agg")
+
+import matplotlib.pyplot as plt
+
+import scarf.plotting as splt
+from scarf.plotting._figure import as_2d_axes_array, normalize_axes_target
+
+
+def test_normalize_axes_target_accepts_mappings_and_sequences():
+ fig, axes = plt.subplots(1, 2)
+ _, mapped, owns = normalize_axes_target(
+ {"left": axes[0], "right": axes[1]},
+ panel_keys=["left", "right"],
+ figsize=None,
+ )
+ assert owns is False
+ assert mapped == {"left": axes[0], "right": axes[1]}
+
+ _, sequenced, owns = normalize_axes_target(
+ axes,
+ panel_keys=["left", "right"],
+ figsize=None,
+ )
+ assert owns is False
+ assert list(sequenced.values()) == list(axes)
+ plt.close(fig)
+
+
+def test_normalize_axes_target_rejects_invalid_ownership():
+ fig_a, ax_a = plt.subplots()
+ fig_b, ax_b = plt.subplots()
+ with pytest.raises(KeyError, match="missing panel keys"):
+ normalize_axes_target(
+ {"left": ax_a},
+ panel_keys=["left", "right"],
+ figsize=None,
+ )
+ with pytest.raises(ValueError, match="same figure"):
+ normalize_axes_target(
+ [ax_a, ax_b],
+ panel_keys=["left", "right"],
+ figsize=None,
+ )
+ with pytest.raises(TypeError, match="single Axes"):
+ normalize_axes_target(
+ ax_a,
+ panel_keys=["left", "right"],
+ figsize=None,
+ )
+ plt.close(fig_a)
+ plt.close(fig_b)
+
+
+def test_panel_labels_legend_collection_and_axes_normalization(
+ umap, leiden_clustering, datastore
+):
+ first = splt.embedding(
+ datastore,
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ show=False,
+ )
+ second = splt.embedding(
+ datastore,
+ layout_key="RNA_UMAP",
+ color_by="RNA_nCounts",
+ show=False,
+ )
+ axes = list(first.axes.values())
+ splt.label_panels(first.axes)
+ assert axes[0].texts[-1].get_text() == "A"
+ with pytest.raises(ValueError, match="labels length"):
+ splt.label_panels(first.axes, labels=["A", "B"])
+
+ legends = splt.collect_legends(first.figure, [first, second])
+ assert legends == first.legends + second.legends
+ assert as_2d_axes_array(axes[0]).shape == (1, 1)
+ assert as_2d_axes_array(np.asarray(axes)).shape == (1, 1)
+ with pytest.raises(ValueError, match="in_ax is None"):
+ as_2d_axes_array(None)
+ first.close()
+ second.close()
diff --git a/tests/test_plotting_foundation.py b/tests/test_plotting_foundation.py
new file mode 100644
index 00000000..88c62376
--- /dev/null
+++ b/tests/test_plotting_foundation.py
@@ -0,0 +1,1157 @@
+"""Compatibility snapshots and basic scarf.plotting tests."""
+
+import inspect
+import subprocess
+import sys
+from pathlib import Path
+
+import matplotlib
+
+matplotlib.use("Agg")
+
+import numpy as np
+import pandas as pd
+import pytest
+
+import scarf.plots as plots
+import scarf.plotting as splt
+from scarf.datastore.datastore import DataStore
+from scarf.datastore.mapping_datastore import MappingDatastore
+
+
+def test_import_plotting_exports():
+ assert callable(splt.embedding)
+ assert callable(splt.dotplot)
+ assert callable(splt.matrixplot)
+ assert callable(splt.composition)
+ assert splt.FeatureRef is not None
+
+
+def test_plotting_modules_import_without_optional_dependencies():
+ script = """
+import builtins
+import pandas as pd
+
+original_import = builtins.__import__
+
+def block_plotting_dependencies(name, *args, **kwargs):
+ if name.split(".", 1)[0] in {"matplotlib", "seaborn", "datashader", "kneed"}:
+ raise ModuleNotFoundError(name)
+ return original_import(name, *args, **kwargs)
+
+builtins.__import__ = block_plotting_dependencies
+import scarf
+import scarf.plots
+import scarf.plotting
+try:
+ scarf.plots.plot_elbow([1.0, 0.5])
+except ImportError as exc:
+ assert "scarf[extra]" in str(exc)
+else:
+ raise AssertionError("plot use should require optional dependencies")
+try:
+ scarf.plots.plot_qc(pd.DataFrame({"groups": ["a"], "value": [1.0]}))
+except ImportError as exc:
+ assert "scarf[extra]" in str(exc)
+else:
+ raise AssertionError("plot use should require matplotlib")
+"""
+ completed = subprocess.run(
+ [sys.executable, "-c", script],
+ cwd=Path(__file__).resolve().parents[1],
+ capture_output=True,
+ text=True,
+ check=False,
+ )
+ assert completed.returncode == 0, completed.stderr
+
+
+def test_plots_no_longer_sets_global_svg_fonttype_on_import():
+ # Import already happened; ensure the module does not require the old global default.
+ # Regression: plots.py must not assign plt.rcParams at import time.
+ import scarf.plots as p
+
+ src = inspect.getsource(p)
+ assert 'plt.rcParams["svg.fonttype"]' not in src
+ assert "CUSTOM_PALETTES" in src or "custom_palettes" in src
+
+
+def test_legacy_signature_snapshots():
+ expected = {
+ plots.plot_scatter: (
+ "dfs",
+ "in_ax",
+ "width",
+ "height",
+ "default_color",
+ "color_map",
+ "color_key",
+ "mask_values",
+ "mask_name",
+ "mask_color",
+ "point_size",
+ "ax_label_size",
+ "frame_offset",
+ "spine_width",
+ "spine_color",
+ "displayed_sides",
+ "legend_ondata",
+ "legend_onside",
+ "legend_size",
+ "legends_per_col",
+ "titles",
+ "title_size",
+ "hide_title",
+ "cbar_shrink",
+ "marker_scale",
+ "lspacing",
+ "cspacing",
+ "savename",
+ "dpi",
+ "force_ints_as_cats",
+ "n_columns",
+ "w_pad",
+ "h_pad",
+ "show_fig",
+ "scatter_kwargs",
+ ),
+ plots.shade_scatter: (
+ "dfs",
+ "in_ax",
+ "figsize",
+ "pixels",
+ "spread_px",
+ "spread_threshold",
+ "min_alpha",
+ "color_map",
+ "color_key",
+ "mask_values",
+ "mask_name",
+ "mask_color",
+ "ax_label_size",
+ "frame_offset",
+ "spine_width",
+ "spine_color",
+ "displayed_sides",
+ "legend_ondata",
+ "legend_onside",
+ "legend_size",
+ "legends_per_col",
+ "titles",
+ "title_size",
+ "hide_title",
+ "cbar_shrink",
+ "marker_scale",
+ "lspacing",
+ "cspacing",
+ "savename",
+ "dpi",
+ "force_ints_as_cats",
+ "n_columns",
+ "w_pad",
+ "h_pad",
+ "show_fig",
+ ),
+ plots.plot_qc: (
+ "data",
+ "color",
+ "cmap",
+ "fig_size",
+ "label_size",
+ "title_size",
+ "sup_title",
+ "sup_title_size",
+ "scatter_size",
+ "max_points",
+ "show_on_single_row",
+ "show_fig",
+ ),
+ DataStore.plot_layout: (
+ "self",
+ "from_assay",
+ "cell_key",
+ "layout_key",
+ "color_by",
+ "subselection_key",
+ "size_vals",
+ "clip_fraction",
+ "width",
+ "height",
+ "default_color",
+ "cmap",
+ "color_key",
+ "mask_values",
+ "mask_name",
+ "mask_color",
+ "point_size",
+ "do_shading",
+ "shade_npixels",
+ "shade_min_alpha",
+ "spread_pixels",
+ "spread_threshold",
+ "ax_label_size",
+ "frame_offset",
+ "spine_width",
+ "spine_color",
+ "displayed_sides",
+ "legend_ondata",
+ "legend_onside",
+ "legend_size",
+ "legends_per_col",
+ "title",
+ "title_size",
+ "hide_title",
+ "cbar_shrink",
+ "marker_scale",
+ "lspacing",
+ "cspacing",
+ "shuffle_df",
+ "sort_values",
+ "savename",
+ "save_dpi",
+ "ax",
+ "force_ints_as_cats",
+ "n_columns",
+ "w_pad",
+ "h_pad",
+ "show_fig",
+ "scatter_kwargs",
+ "use_plotting",
+ ),
+ MappingDatastore.plot_unified_layout: (
+ "self",
+ "from_assay",
+ "layout_key",
+ "show_target_only",
+ "ref_name",
+ "target_groups",
+ "width",
+ "height",
+ "cmap",
+ "color_key",
+ "mask_color",
+ "point_size",
+ "ax_label_size",
+ "frame_offset",
+ "spine_width",
+ "spine_color",
+ "displayed_sides",
+ "legend_ondata",
+ "legend_onside",
+ "legend_size",
+ "legends_per_col",
+ "title",
+ "title_size",
+ "hide_title",
+ "cbar_shrink",
+ "marker_scale",
+ "lspacing",
+ "cspacing",
+ "savename",
+ "save_dpi",
+ "ax",
+ "force_ints_as_cats",
+ "n_columns",
+ "w_pad",
+ "h_pad",
+ "scatter_kwargs",
+ "shuffle_zorder",
+ "show_fig",
+ ),
+ DataStore.plot_cells_dists: (
+ "self",
+ "from_assay",
+ "cols",
+ "cell_key",
+ "group_key",
+ "color",
+ "cmap",
+ "fig_size",
+ "label_size",
+ "title_size",
+ "sup_title",
+ "sup_title_size",
+ "scatter_size",
+ "max_points",
+ "show_on_single_row",
+ "show_fig",
+ ),
+ DataStore.plot_marker_heatmap: (
+ "self",
+ "from_assay",
+ "group_key",
+ "cell_key",
+ "topn",
+ "log_transform",
+ "vmin",
+ "vmax",
+ "savename",
+ "save_dpi",
+ "show_fig",
+ "heatmap_kwargs",
+ ),
+ DataStore.plot_cluster_tree: (
+ "self",
+ "from_assay",
+ "cell_key",
+ "feat_key",
+ "cluster_key",
+ "fill_by_value",
+ "force_ints_as_cats",
+ "width",
+ "lvr_factor",
+ "vert_gap",
+ "min_node_size",
+ "node_size_multiplier",
+ "node_power",
+ "root_size",
+ "non_leaf_size",
+ "show_labels",
+ "fontsize",
+ "root_color",
+ "non_leaf_color",
+ "cmap",
+ "color_key",
+ "edgecolors",
+ "edgewidth",
+ "alpha",
+ "figsize",
+ "ax",
+ "show_fig",
+ "savename",
+ "save_dpi",
+ ),
+ DataStore.plot_pseudotime_heatmap: (
+ "self",
+ "from_assay",
+ "cell_key",
+ "feat_key",
+ "feature_cluster_key",
+ "pseudotime_key",
+ "show_features",
+ "width",
+ "height",
+ "vmin",
+ "vmax",
+ "heatmap_cmap",
+ "pseudotime_cmap",
+ "clusterbar_cmap",
+ "tick_fontsize",
+ "axis_fontsize",
+ "feature_label_fontsize",
+ "savename",
+ "save_dpi",
+ "show_fig",
+ ),
+ }
+ for function, parameter_names in expected.items():
+ assert tuple(inspect.signature(function).parameters) == parameter_names
+
+
+def test_color_key_not_mutated():
+ df = pd.DataFrame({"x": [0.0, 1.0], "y": [0.0, 1.0], "g": ["a", "b"]})
+ color_key = {"a": "#111111", "b": "#222222"}
+ original = dict(color_key)
+ plots.plot_scatter(
+ [df],
+ color_key=color_key,
+ force_ints_as_cats=True,
+ show_fig=False,
+ legend_ondata=False,
+ legend_onside=False,
+ )
+ assert color_key == original
+
+
+def test_mask_values_list_not_mutated():
+ df = pd.DataFrame(
+ {"x": [0.0, 1.0, 2.0], "y": [0.0, 1.0, 2.0], "g": ["a", "b", "c"]}
+ )
+ mask_values = ["c"]
+ original = list(mask_values)
+ plots.plot_scatter(
+ [df],
+ mask_values=mask_values,
+ show_fig=False,
+ legend_ondata=False,
+ legend_onside=False,
+ )
+ assert mask_values == original
+
+
+def test_legacy_scatter_does_not_mutate_dataframes():
+ df = pd.DataFrame(
+ {
+ "x": [0.0, 1.0, 2.0],
+ "y": [0.0, 1.0, 2.0],
+ "group": [1, 2, 1],
+ }
+ )
+ original = df.copy(deep=True)
+ scatter_kwargs = {"c": "red", "s": 5}
+ original_kwargs = dict(scatter_kwargs)
+ plots.plot_scatter(
+ [df],
+ force_ints_as_cats=True,
+ show_fig=False,
+ legend_ondata=False,
+ legend_onside=False,
+ scatter_kwargs=scatter_kwargs,
+ )
+ pd.testing.assert_frame_equal(df, original)
+ assert scatter_kwargs == original_kwargs
+
+
+def test_bare_axes_accepted_by_plot_scatter():
+ fig, ax = matplotlib.pyplot.subplots()
+ df = pd.DataFrame({"x": [0.0, 1.0], "y": [0.0, 1.0], "g": [0.1, 0.9]})
+ out = plots.plot_scatter([df], in_ax=ax, show_fig=False, legend_onside=False)
+ assert out is not None
+ matplotlib.pyplot.close(fig)
+
+
+def test_continuous_nan_not_filled_as_zero():
+ df = pd.DataFrame(
+ {"x": [0.0, 1.0, 2.0], "y": [0.0, 1.0, 2.0], "v": [0.0, np.nan, 1.0]}
+ )
+ out = plots.plot_scatter(
+ [df], show_fig=False, legend_onside=False, mask_color="#ff00ff"
+ )
+ assert out is not None
+ matplotlib.pyplot.close("all")
+
+
+def test_plot_qc_with_groups_first_column():
+ data = pd.DataFrame(
+ {
+ "groups": ["a", "a", "b", "b"],
+ "nCounts": [10.0, 12.0, 20.0, 22.0],
+ "nFeatures": [5.0, 6.0, 8.0, 9.0],
+ }
+ )
+ fig = plots.plot_qc(data, show_fig=False)
+ assert fig is not None
+ # Two metric panels
+ assert len(fig.axes) >= 2
+ matplotlib.pyplot.close(fig)
+
+
+def test_study_design_allows_pairing_rejects_tech_rep():
+ design = splt.StudyDesign(
+ sample_by="sample", subject_by="donor", condition_by="time"
+ )
+ assert design.subject_by == "donor"
+ with pytest.raises(NotImplementedError, match="technical_replicate_by"):
+ splt.StudyDesign(sample_by="sample", technical_replicate_by="lane")
+
+
+def test_size_scale_maps_fraction_to_area():
+ scale = splt.SizeScale(vmin=0, vmax=1, size_min=10, size_max=200)
+ areas = scale.areas(np.array([0.0, 0.5, 1.0]))
+ assert areas[0] == pytest.approx(10)
+ assert areas[1] == pytest.approx(105)
+ assert areas[2] == pytest.approx(200)
+
+
+def test_plotting_contracts_reject_invalid_values():
+ with pytest.raises(ValueError, match="source"):
+ splt.NormalizationSpec(source="scaled")
+ with pytest.raises(ValueError, match="quantiles"):
+ splt.ColorScale(quantiles=(0.9, 0.1))
+ with pytest.raises(ValueError, match="vmax"):
+ splt.ColorScale(vmin=2, vmax=1)
+ with pytest.raises(ValueError, match="size range"):
+ splt.SizeScale(size_min=20, size_max=10)
+ with pytest.raises(ValueError, match="kind"):
+ splt.CellField("group", kind="ordinal")
+
+
+def test_equal_weight_sample_aggregation_fixture():
+ """Two samples of very different size must weight equally."""
+ ps = pd.DataFrame(
+ {
+ "sample": ["A", "B"],
+ "group": ["g1", "g1"],
+ "feature": ["f1", "f1"],
+ "mean": [1.0, 10.0],
+ "fraction": [0.1, 0.9],
+ "n_cells": [10, 1000],
+ }
+ )
+ agg = (
+ ps.groupby(["group", "feature"], observed=False)
+ .agg(
+ mean=("mean", "mean"),
+ fraction=("fraction", "mean"),
+ n_cells=("n_cells", "sum"),
+ )
+ .reset_index()
+ )
+ assert len(agg) == 1
+ assert agg["mean"].iloc[0] == pytest.approx(5.5)
+ assert agg["fraction"].iloc[0] == pytest.approx(0.5)
+ # Cell-weighted would be ~9.91, not 5.5
+ cell_weighted = np.average(ps["mean"], weights=ps["n_cells"])
+ assert cell_weighted == pytest.approx(9.910891, rel=1e-5)
+ assert agg["mean"].iloc[0] != pytest.approx(cell_weighted)
+
+
+def test_embedding_dotplot_matrixplot_on_fixture(umap, leiden_clustering, datastore):
+ ds = datastore
+ # point_sizes and sort_values are the clever legacy behaviors to preserve
+ n = len(ds.cells.fetch("I", key="I"))
+ sizes = np.linspace(5, 40, n)
+ emb = splt.embedding(
+ ds,
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ point_sizes=sizes,
+ sort_values=False,
+ show=False,
+ )
+ assert emb.owns_figure
+ assert len(emb.axes) == 1
+ assert next(iter(emb.axes.values())).get_legend() is not None
+ emb.close()
+
+ # Gene coloring with sort_values (high expression on top)
+ names = ds.RNA.feats.fetch_all("names")
+ gene = str(names[0])
+ emb2 = splt.embedding(
+ ds,
+ layout_key="RNA_UMAP",
+ color_by=gene,
+ sort_values=True,
+ show=False,
+ )
+ assert emb2.provenance.extras.get("sort_values") is True
+ emb2.close()
+
+ dp = splt.dotplot(
+ ds,
+ features=[gene],
+ group_by="RNA_leiden_cluster",
+ show=False,
+ )
+ assert "aggregate" in dp.tables
+ assert "mean" in dp.tables["aggregate"].columns
+ assert "fraction" in dp.tables["aggregate"].columns
+ assert dp.provenance.n_cells == len(ds.cells.active_index("I"))
+ assert next(iter(dp.axes.values())).get_legend() is not None
+ dp.close()
+
+ mp = splt.matrixplot(
+ ds,
+ features=[gene],
+ group_by="RNA_leiden_cluster",
+ show=False,
+ )
+ assert "matrix" in mp.tables
+ assert mp.provenance.n_cells == len(ds.cells.active_index("I"))
+ mp.close()
+
+
+def test_feature_ref_duplicate_raises(datastore):
+ # Looking up by a nonsense name
+ with pytest.raises(KeyError):
+ splt.FeatureRef # noqa: B018 — ensure import path
+ from scarf.plotting._data import resolve_feature
+
+ resolve_feature(datastore, "___not_a_real_feature___")
+
+
+def test_caller_owned_target(umap, datastore):
+ import matplotlib.pyplot as plt
+
+ fig, ax = plt.subplots()
+ result = splt.embedding(
+ datastore,
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ target=ax,
+ show=False,
+ )
+ assert result.owns_figure is False
+ result.close() # must not close foreign figure
+ assert plt.fignum_exists(fig.number)
+ plt.close(fig)
+
+
+def test_summary_and_composition_accept_foreign_targets(
+ umap, leiden_clustering, datastore
+):
+ import matplotlib.pyplot as plt
+
+ ds = datastore
+ gene = str(ds.RNA.feats.fetch_all("names")[0])
+ fig, axes = plt.subplots(1, 3)
+ results = [
+ splt.dotplot(
+ ds,
+ features=[gene],
+ group_by="RNA_leiden_cluster",
+ target=axes[0],
+ show=False,
+ ),
+ splt.matrixplot(
+ ds,
+ features=[gene],
+ group_by="RNA_leiden_cluster",
+ target=axes[1],
+ show=False,
+ ),
+ splt.composition(
+ ds,
+ category_by="RNA_leiden_cluster",
+ kind="stacked",
+ target=axes[2],
+ show=False,
+ ),
+ ]
+ assert all(result.owns_figure is False for result in results)
+ for result in results:
+ result.close()
+ assert plt.fignum_exists(fig.number)
+ plt.close(fig)
+
+
+def test_sample_by_equal_weight_on_datastore(umap, leiden_clustering, datastore):
+ from scarf.plotting._data import summarize_features_by_group
+
+ ds = datastore
+ active_n = len(ds.cells.active_index("I"))
+ # Unbalanced samples among active cells: 5 vs rest
+ sample = np.array(["big"] * active_n, dtype=object)
+ sample[:5] = "small"
+ ds.cells.insert("plot_sample_id", sample, overwrite=True)
+
+ gene = str(ds.RNA.feats.fetch_all("names")[0])
+ agg, per = summarize_features_by_group(
+ ds,
+ features=[gene],
+ group_by="RNA_leiden_cluster",
+ sample_by="plot_sample_id",
+ )
+ assert per is not None
+ assert set(per["sample"].unique()) == {"big", "small"}
+
+ # For each group×feature present in both samples, aggregate mean ==
+ # unweighted mean of per-sample means (not cell-weighted).
+ both = per.groupby(["RNA_leiden_cluster", "feature"], observed=False)[
+ "sample"
+ ].nunique()
+ shared = both[both == 2].index
+ assert len(shared) > 0
+ for cluster, feature in shared:
+ rows = per[(per["RNA_leiden_cluster"] == cluster) & (per["feature"] == feature)]
+ expected = float(rows["mean"].mean())
+ cell_weighted = float(np.average(rows["mean"], weights=rows["n_cells"]))
+ got = float(
+ agg.loc[
+ (agg["RNA_leiden_cluster"] == cluster) & (agg["feature"] == feature),
+ "mean",
+ ].iloc[0]
+ )
+ assert got == pytest.approx(expected, rel=1e-6, abs=1e-8)
+ # When sample sizes and per-sample means differ, equal-weight != cell-weight
+ if (
+ rows["n_cells"].nunique() > 1
+ and rows["mean"].nunique() > 1
+ and not np.allclose(rows["mean"], 0)
+ ):
+ assert got != pytest.approx(cell_weighted, rel=1e-3)
+
+ dp = splt.dotplot(
+ ds,
+ features=[gene],
+ group_by="RNA_leiden_cluster",
+ sample_by="plot_sample_id",
+ show=False,
+ )
+ assert "per_sample" in dp.tables
+ assert "n_samples" in dp.tables["aggregate"].columns
+ assert dp.provenance.n_samples == 2
+ assert dp.provenance.extras["dropped_sample_cells"] == 0
+ dp.close()
+
+
+def test_facet_shared_color_limits(umap, datastore):
+ ds = datastore
+ active_n = len(ds.cells.active_index("I"))
+ condition = np.array(["low"] * active_n, dtype=object)
+ condition[active_n // 2 :] = "high"
+ score = np.zeros(active_n, dtype=np.float64)
+ score[condition == "low"] = np.linspace(0.0, 1.0, int((condition == "low").sum()))
+ score[condition == "high"] = np.linspace(
+ 10.0, 11.0, int((condition == "high").sum())
+ )
+ ds.cells.insert("plot_condition", condition, overwrite=True)
+ ds.cells.insert("plot_score", score, overwrite=True)
+
+ result = splt.embedding(
+ ds,
+ layout_key="RNA_UMAP",
+ color_by=splt.CellField("plot_score", kind="continuous"),
+ facet_by="plot_condition",
+ facet_order=["low", "high"],
+ show=False,
+ )
+ limits = result.provenance.extras["color_limits"]
+ assert "plot_score" in limits
+ vmin, vmax = limits["plot_score"]
+ assert vmin == pytest.approx(0.0, abs=1e-6)
+ assert vmax == pytest.approx(11.0, abs=1e-6)
+ # Both facet panels must exist and share coordinate limits
+ assert len(result.axes) == 2
+ xlims = {ax.get_xlim() for ax in result.axes.values()}
+ ylims = {ax.get_ylim() for ax in result.axes.values()}
+ assert len(xlims) == 1
+ assert len(ylims) == 1
+ result.close()
+
+ panel_result = splt.embedding(
+ ds,
+ layout_key="RNA_UMAP",
+ color_by=splt.CellField("plot_score", kind="continuous"),
+ facet_by="plot_condition",
+ facet_order=["low", "high"],
+ color_scale=splt.ColorScale(scope="panel"),
+ show=False,
+ )
+ panel_limits = list(panel_result.provenance.extras["color_limits"].values())
+ assert panel_limits[0] == pytest.approx((0.0, 1.0))
+ assert panel_limits[1] == pytest.approx((10.0, 11.0))
+ assert len(panel_result.figure.axes) == 4
+ panel_result.close()
+
+ ds.cells.insert("plot_score_scaled", score * 10, overwrite=True)
+ shared_result = splt.embedding(
+ ds,
+ layout_key="RNA_UMAP",
+ color_by=[
+ splt.CellField("plot_score", kind="continuous"),
+ splt.CellField("plot_score_scaled", kind="continuous"),
+ ],
+ color_scale=splt.ColorScale(scope="shared"),
+ show=False,
+ )
+ shared_limits = list(shared_result.provenance.extras["color_limits"].values())
+ assert shared_limits[0] == pytest.approx(shared_limits[1])
+ assert shared_limits[1][1] == pytest.approx(110.0)
+ shared_result.close()
+
+
+def test_composition_and_export(umap, leiden_clustering, datastore, tmp_path):
+ ds = datastore
+ active_n = len(ds.cells.active_index("I"))
+ sample = np.array([f"s{i % 3}" for i in range(active_n)], dtype=object)
+ ds.cells.insert("plot_comp_sample", sample, overwrite=True)
+
+ result = splt.composition(
+ ds,
+ category_by="RNA_leiden_cluster",
+ sample_by="plot_comp_sample",
+ kind="per_sample",
+ show=False,
+ )
+ assert "per_sample" in result.tables
+ out = result.save(tmp_path / "composition.png", dpi=100)
+ assert out.exists() and out.stat().st_size > 0
+ result.close()
+
+ stacked = splt.composition(
+ ds,
+ category_by="RNA_leiden_cluster",
+ sample_by="plot_comp_sample",
+ show=False,
+ )
+ pdf = stacked.save(tmp_path / "composition.pdf", exact_size=True)
+ assert pdf.exists()
+ stacked.close()
+
+
+def test_feature_plotting_uses_pure_normalization_adapter(umap, datastore, monkeypatch):
+ ds = datastore
+ assay = ds.RNA
+ prev_method = assay.normMethod
+ prev_scalar = getattr(assay, "scalar", None)
+ gene = str(assay.feats.fetch_all("names")[0])
+
+ def fail_if_called(*args, **kwargs):
+ raise AssertionError("plotting must not call stateful assay.normed()")
+
+ monkeypatch.setattr(assay, "normed", fail_if_called)
+ result = splt.embedding(
+ ds, layout_key="RNA_UMAP", color_by=gene, sort_values=True, show=False
+ )
+ result.close()
+ assert assay.normMethod is prev_method
+ assert getattr(assay, "scalar", None) is prev_scalar
+
+
+def test_normalization_spec_supports_raw_and_log1p(datastore):
+ from scarf.plotting._data import (
+ fetch_normalized_feature_matrix,
+ resolve_feature,
+ )
+ from scarf.utils import controlled_compute
+
+ assay = datastore.RNA
+ cell_idx = datastore.cells.active_index("I")
+ gene = str(assay.feats.fetch_all("names")[0])
+ resolved = [resolve_feature(datastore, gene)]
+ raw = fetch_normalized_feature_matrix(
+ datastore,
+ resolved,
+ cell_idx,
+ normalization=splt.NormalizationSpec(source="raw"),
+ )
+ expected_raw = controlled_compute(
+ assay.rawData[:, [resolved[0].indices[0]]][cell_idx, :],
+ datastore.nthreads,
+ )
+ normalized = fetch_normalized_feature_matrix(
+ datastore,
+ resolved,
+ cell_idx,
+ normalization=splt.NormalizationSpec(),
+ )
+ logged = fetch_normalized_feature_matrix(
+ datastore,
+ resolved,
+ cell_idx,
+ normalization=splt.NormalizationSpec(transform="log1p"),
+ )
+ assert np.array_equal(raw, expected_raw)
+ assert np.allclose(logged, np.log1p(normalized))
+
+
+def test_figsize_rejected_with_owned_target(umap, datastore):
+ import matplotlib.pyplot as plt
+
+ fig, ax = plt.subplots()
+ with pytest.raises(ValueError, match="figsize"):
+ splt.embedding(
+ datastore,
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ target=ax,
+ figsize=(3, 3),
+ show=False,
+ )
+ plt.close(fig)
+
+
+def test_multi_gene_by_condition_embedding(umap, datastore):
+ ds = datastore
+ active_n = len(ds.cells.active_index("I"))
+ condition = np.array(["ctrl"] * active_n, dtype=object)
+ condition[active_n // 2 :] = "stim"
+ ds.cells.insert("plot_condition_mg", condition, overwrite=True)
+
+ names = [str(x) for x in ds.RNA.feats.fetch_all("names")[:2]]
+ result = splt.embedding(
+ ds,
+ layout_key="RNA_UMAP",
+ color_by=names,
+ facet_by="plot_condition_mg",
+ facet_order=["ctrl", "stim"],
+ sort_values=True,
+ show=False,
+ )
+ assert result.provenance.extras["n_colors"] == 2
+ assert result.provenance.extras["n_facets"] == 2
+ assert len(result.axes) == 4
+ limits = result.provenance.extras["color_limits"]
+ for gene in names:
+ assert gene in limits
+ vmin, vmax = limits[gene]
+ assert vmax >= vmin
+ # Panel keys are (gene, condition)
+ for gene in names:
+ for cond in ("ctrl", "stim"):
+ assert (gene, cond) in result.axes
+ result.close()
+
+
+def test_resolve_feature_by_index(datastore):
+ from scarf.plotting._data import resolve_feature
+
+ resolved = resolve_feature(
+ datastore, splt.FeatureRef(value=0, by="index", assay="RNA")
+ )
+ assert resolved.indices == (0,)
+ assert resolved.assay == "RNA"
+ assert resolved.label
+
+
+def test_label_panels_and_collect_legends(umap, datastore):
+ import matplotlib.pyplot as plt
+
+ ds = datastore
+ fig, axes = plt.subplot_mosaic([["A", "B"]], figsize=(6, 3))
+ a = splt.embedding(
+ ds,
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ target=axes["A"],
+ show=False,
+ )
+ gene = str(ds.RNA.feats.fetch_all("names")[0])
+ b = splt.embedding(
+ ds,
+ layout_key="RNA_UMAP",
+ color_by=gene,
+ target=axes["B"],
+ show=False,
+ )
+ splt.label_panels({"A": axes["A"], "B": axes["B"]}, labels=["A", "B"])
+ legends = splt.collect_legends(fig, [a, b])
+ assert len(legends) >= 1
+ a.close()
+ b.close()
+ plt.close(fig)
+
+
+def test_legacy_mutable_copy_helper():
+ from scarf.plotting._legacy import copy_plot_mutables
+
+ color_key = {"a": "#000"}
+ mask_values = ["x"]
+ sk = {"lw": 0.2}
+ ck2, mv2, sk2 = copy_plot_mutables(
+ color_key=color_key, mask_values=mask_values, scatter_kwargs=sk
+ )
+ assert ck2 is not color_key and ck2 == color_key
+ assert mv2 is not mask_values and mv2 == mask_values
+ ck2["b"] = "#fff"
+ mv2.append("y")
+ assert "b" not in color_key
+ assert mask_values == ["x"]
+
+
+def test_plot_layout_bridge_still_blocked_by_default():
+ from scarf.plotting._legacy import plot_layout_bridge_blockers
+
+ blockers = plot_layout_bridge_blockers(
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ do_shading=False,
+ mask_values=None,
+ subset_by=None,
+ shuffle_df=False,
+ legend_ondata=True,
+ legend_onside=True,
+ force_ints_as_cats=True,
+ clip_fraction=0.01,
+ ax=None,
+ use_plotting=False,
+ )
+ assert blockers == ("bridge_not_enabled",)
+
+ ok = plot_layout_bridge_blockers(
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ do_shading=False,
+ mask_values=None,
+ subset_by=None,
+ shuffle_df=False,
+ legend_ondata=True,
+ legend_onside=True,
+ force_ints_as_cats=True,
+ clip_fraction=0.01,
+ ax=None,
+ use_plotting=True,
+ )
+ assert ok == ()
+
+ shaded = plot_layout_bridge_blockers(
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ do_shading=True,
+ mask_values=None,
+ subset_by=None,
+ shuffle_df=False,
+ legend_ondata=False,
+ legend_onside=False,
+ force_ints_as_cats=False,
+ clip_fraction=0.0,
+ ax=None,
+ use_plotting=True,
+ )
+ assert "do_shading" in shaded
+
+
+def test_plot_layout_use_plotting_bridge(umap, leiden_clustering, datastore):
+ result = datastore.plot_layout(
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ show_fig=False,
+ use_plotting=True,
+ )
+ assert isinstance(result, splt.PlotResult)
+ assert "embedding" in result.provenance.notes
+ result.close()
+
+ categories = sorted(pd.unique(datastore.cells.fetch("RNA_leiden_cluster")))
+ palette = {
+ value: f"#{((index + 1) * 123457) % 0xFFFFFF:06x}"
+ for index, value in enumerate(categories)
+ }
+ colored = datastore.plot_layout(
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ color_key=palette,
+ show_fig=False,
+ use_plotting=True,
+ )
+ assert isinstance(colored, splt.PlotResult)
+ categorical_scales = [
+ scale for scale in colored.scales if isinstance(scale, splt.CategoricalScale)
+ ]
+ assert categorical_scales[0].palette == palette
+ colored.close()
+
+ legacy = datastore.plot_layout(
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ title="Custom title",
+ show_fig=False,
+ use_plotting=True,
+ )
+ assert not isinstance(legacy, splt.PlotResult)
+ matplotlib.pyplot.close("all")
+
+
+def test_paired_composition_draws_subject_lines(umap, leiden_clustering, datastore):
+ ds = datastore
+ active_n = len(ds.cells.active_index("I"))
+ sample = np.array([f"s{i % 6}" for i in range(active_n)], dtype=object)
+ subject = np.array([f"d{i % 3}" for i in range(active_n)], dtype=object)
+ condition = np.array(
+ ["before" if i % 6 < 3 else "after" for i in range(active_n)],
+ dtype=object,
+ )
+ # Two samples per subject (s0,s3 -> d0; s1,s4 -> d1; s2,s5 -> d2)
+ ds.cells.insert("plot_pair_sample", sample, overwrite=True)
+ ds.cells.insert("plot_pair_subject", subject, overwrite=True)
+ ds.cells.insert("plot_pair_condition", condition, overwrite=True)
+
+ result = splt.composition(
+ ds,
+ category_by="RNA_leiden_cluster",
+ study_design=splt.StudyDesign(
+ sample_by="plot_pair_sample",
+ subject_by="plot_pair_subject",
+ condition_by="plot_pair_condition",
+ ),
+ kind="per_sample",
+ show=False,
+ )
+ assert "subject" in result.tables["per_sample"].columns
+ assert result.provenance.extras["n_pair_lines"] >= 1
+ assert any("paired_by=subject" in n for n in result.provenance.notes)
+ result.close()
+
+
+def test_paired_composition_requires_condition(leiden_clustering, datastore):
+ with pytest.raises(ValueError, match="requires condition_by"):
+ splt.composition(
+ datastore,
+ category_by="RNA_leiden_cluster",
+ sample_by="RNA_leiden_cluster",
+ subject_by="RNA_leiden_cluster",
+ kind="per_sample",
+ show=False,
+ )
+
+
+def test_embedding_clip_and_subset(umap, datastore):
+ ds = datastore
+ active_n = len(ds.cells.active_index("I"))
+ keep = np.zeros(active_n, dtype=bool)
+ keep[: max(10, active_n // 2)] = True
+ ds.cells.insert("plot_keep", keep, overwrite=True)
+ gene = str(ds.RNA.feats.fetch_all("names")[0])
+ result = splt.embedding(
+ ds,
+ layout_key="RNA_UMAP",
+ color_by=gene,
+ clip_fraction=0.01,
+ subset_by="plot_keep",
+ show=False,
+ )
+ assert result.provenance.extras["clip_fraction"] == 0.01
+ assert result.provenance.extras["subset_by"] == "plot_keep"
+ result.close()
+
+
+def test_distribution_violin(umap, leiden_clustering, datastore):
+ ds = datastore
+ result = splt.distribution(
+ ds,
+ keys=["RNA_nCounts", "RNA_nFeatures"],
+ group_by="RNA_leiden_cluster",
+ kind="violin",
+ max_points=200,
+ seed=1,
+ show=False,
+ )
+ assert len(result.axes) == 2
+ assert "RNA_nCounts" in result.tables
+ assert result.provenance.extras["approximate"] is True
+ assert "subsampled_display" in result.provenance.notes
+ result.close()
+
+ gene = str(ds.RNA.feats.fetch_all("names")[0])
+ result2 = splt.distribution(ds, keys=gene, kind="box", max_points=100, show=False)
+ assert len(result2.axes) == 1
+ result2.close()
+
+
+def test_distribution_hist_and_ecdf(umap, leiden_clustering, datastore):
+ ds = datastore
+ hist = splt.distribution(
+ ds,
+ keys="RNA_nCounts",
+ group_by="RNA_leiden_cluster",
+ kind="hist",
+ bins=20,
+ show=False,
+ )
+ assert hist.provenance.extras["bins"] == 20
+ assert hist.provenance.extras["approximate"] is False
+ ax = next(iter(hist.axes.values()))
+ n_groups = len(np.unique(ds.cells.fetch("RNA_leiden_cluster")))
+ assert len(ax.patches) == n_groups * 20
+ first_bins = [(patch.get_x(), patch.get_width()) for patch in ax.patches[:20]]
+ for group_index in range(1, n_groups):
+ offset = group_index * 20
+ group_bins = [
+ (patch.get_x(), patch.get_width())
+ for patch in ax.patches[offset : offset + 20]
+ ]
+ assert group_bins == pytest.approx(first_bins)
+ hist.close()
+
+ ecdf = splt.distribution(
+ ds,
+ keys="RNA_nFeatures",
+ kind="ecdf",
+ max_points=500,
+ seed=2,
+ show=False,
+ )
+ assert "ecdf" in ecdf.provenance.notes
+ assert ecdf.provenance.extras["approximate"] is True
+ ecdf.close()
+
+ duplicates = splt.distribution(
+ ds,
+ keys=["RNA_nCounts", "RNA_nCounts"],
+ kind="hist",
+ bins=5,
+ show=False,
+ )
+ assert set(duplicates.tables) == {"0:RNA_nCounts", "1:RNA_nCounts"}
+ duplicates.close()
diff --git a/tests/test_plotting_raster.py b/tests/test_plotting_raster.py
new file mode 100644
index 00000000..3af3cc20
--- /dev/null
+++ b/tests/test_plotting_raster.py
@@ -0,0 +1,269 @@
+"""Guarded-source and parity tests for Milestone C raster path."""
+
+import numpy as np
+import pytest
+
+import scarf.plotting as splt
+
+
+class _GuardedZarrArray:
+ """Minimal array stand-in that forbids full-column ``[:]`` reads."""
+
+ def __init__(self, data: np.ndarray, chunk: int = 8):
+ self._data = np.asarray(data)
+ self.chunks = (chunk,)
+ self.dtype = self._data.dtype
+ self.shape = self._data.shape
+
+ def __getitem__(self, key):
+ if key == slice(None) or key == ():
+ raise AssertionError(
+ "full-column read is forbidden in guarded raster tests"
+ )
+ if isinstance(key, slice):
+ if key.start is None and key.stop is None and key.step is None:
+ raise AssertionError("full-column read is forbidden")
+ return self._data[key]
+ return self._data[key]
+
+
+class _GuardedMeta:
+ """Duck-typed MetaData using guarded column arrays."""
+
+ def __init__(self, columns: dict[str, np.ndarray], chunk: int = 8):
+ self.N = len(next(iter(columns.values())))
+ self.columns = list(columns)
+ self._arrays = {
+ k: _GuardedZarrArray(v, chunk=chunk) for k, v in columns.items()
+ }
+ self.index = np.arange(self.N)
+
+ def _verify_bool(self, key: str) -> bool:
+ if self._arrays[key].dtype != bool:
+ raise TypeError("key must be bool")
+ return True
+
+ def default_block_rows(self, column: str = "I") -> int:
+ return int(self._arrays[column].chunks[0])
+
+ def active_index(self, key: str) -> np.ndarray:
+ return np.flatnonzero(self._arrays[key]._data)
+
+ def fetch(self, column: str, key: str = "I") -> np.ndarray:
+ idx = self.active_index(key)
+ return self._arrays[column]._data[idx]
+
+ def iter_row_blocks(self, *, cell_key="I", columns=None, block_rows=None):
+ from scarf.metadata import MetaDataRowBlock
+
+ self._verify_bool(cell_key)
+ if block_rows is None:
+ block_rows = self.default_block_rows(cell_key)
+ col_list = list(columns or [])
+ key_arr = self._arrays[cell_key]
+ for start in range(0, self.N, block_rows):
+ stop = min(start + block_rows, self.N)
+ key_slice = np.asarray(key_arr[start:stop], dtype=bool)
+ local = np.flatnonzero(key_slice)
+ active_global = (local + start).astype(np.int64, copy=False)
+ values = {
+ col: np.asarray(self._arrays[col][start:stop])[local]
+ for col in col_list
+ }
+ yield MetaDataRowBlock(
+ start=start,
+ stop=stop,
+ active_global_indices=active_global,
+ values=values,
+ )
+
+
+def test_raster_from_metadata_rejects_full_slice_and_matches_mean():
+ from scarf.plotting._raster import raster_from_metadata
+
+ rng = np.random.default_rng(0)
+ n = 40
+ x = rng.normal(size=n)
+ y = rng.normal(size=n)
+ c = rng.normal(size=n)
+ active = np.ones(n, dtype=bool)
+ active[::5] = False
+ cells = _GuardedMeta(
+ {"I": active, "UMAP1": x, "UMAP2": y, "score": c},
+ chunk=7,
+ )
+ canvas = raster_from_metadata(
+ cells,
+ x_key="UMAP1",
+ y_key="UMAP2",
+ color_key="score",
+ cell_key="I",
+ pixels=32,
+ block_rows=7,
+ quantiles=None,
+ seed=0,
+ )
+ assert canvas.n_cells == int(active.sum())
+ assert canvas.n_blocks >= 2
+ assert np.isfinite(canvas.vmin) and np.isfinite(canvas.vmax)
+ # Pixel means only where counts > 0
+ assert np.isnan(canvas.image[canvas.counts == 0]).all()
+ assert np.isfinite(canvas.image[canvas.counts > 0]).all()
+
+
+def test_embedding_raster_on_datastore(umap, leiden_clustering, datastore):
+ result = splt.embedding_raster(
+ datastore,
+ layout_key="RNA_UMAP",
+ color_by="RNA_nCounts",
+ pixels=64,
+ block_rows=32,
+ show=False,
+ )
+ assert result.provenance.renderer == "matplotlib-raster"
+ assert "two_pass" in result.provenance.notes
+ assert result.provenance.n_cells > 0
+ result.close()
+
+
+def test_embedding_raster_rejects_categorical_metadata(
+ umap, leiden_clustering, datastore
+):
+ with pytest.raises(NotImplementedError, match="continuous color"):
+ splt.embedding_raster(
+ datastore,
+ layout_key="RNA_UMAP",
+ color_by="RNA_leiden_cluster",
+ pixels=32,
+ show=False,
+ )
+
+
+def test_embedding_raster_density_and_foreign_target(umap, datastore):
+ import matplotlib.pyplot as plt
+
+ fig, ax = plt.subplots()
+ result = splt.embedding_raster(
+ datastore,
+ layout_key="RNA_UMAP",
+ pixels=32,
+ target=ax,
+ show=False,
+ )
+ assert result.owns_figure is False
+ assert result.provenance.extras["color_mode"] == "density"
+ assert result.legends[0].label == "log1p cell count"
+ result.close()
+ assert plt.fignum_exists(fig.number)
+ plt.close(fig)
+
+
+def test_raster_validates_quantiles():
+ from scarf.plotting._raster import raster_from_metadata
+
+ cells = _GuardedMeta(
+ {
+ "I": np.ones(4, dtype=bool),
+ "x": np.arange(4, dtype=float),
+ "y": np.arange(4, dtype=float),
+ "v": np.arange(4, dtype=float),
+ }
+ )
+ with pytest.raises(ValueError, match="quantiles"):
+ raster_from_metadata(
+ cells,
+ x_key="x",
+ y_key="y",
+ color_key="v",
+ quantiles=(0.9, 0.1),
+ )
+ with pytest.raises(ValueError, match="sample_capacity"):
+ raster_from_metadata(
+ cells,
+ x_key="x",
+ y_key="y",
+ color_key="v",
+ sample_capacity=0,
+ )
+
+
+def test_raster_without_color_encodes_log_density():
+ from scarf.plotting._raster import raster_from_metadata
+
+ cells = _GuardedMeta(
+ {
+ "I": np.ones(4, dtype=bool),
+ "x": np.zeros(4),
+ "y": np.zeros(4),
+ }
+ )
+ canvas = raster_from_metadata(
+ cells,
+ x_key="x",
+ y_key="y",
+ pixels=8,
+ )
+ assert canvas.counts.sum() == 4
+ assert np.nanmax(canvas.image) == pytest.approx(np.log1p(4))
+ assert canvas.vmax == pytest.approx(np.log1p(4))
+
+
+def test_raster_is_block_size_invariant():
+ from scarf.plotting._raster import raster_from_metadata
+
+ rng = np.random.default_rng(4)
+ cells = _GuardedMeta(
+ {
+ "I": np.ones(60, dtype=bool),
+ "x": rng.normal(size=60),
+ "y": rng.normal(size=60),
+ "value": rng.normal(size=60),
+ }
+ )
+ kwargs = {
+ "x_key": "x",
+ "y_key": "y",
+ "color_key": "value",
+ "pixels": 24,
+ "sample_capacity": 15,
+ "seed": 9,
+ }
+ first = raster_from_metadata(cells, block_rows=7, **kwargs)
+ second = raster_from_metadata(cells, block_rows=13, **kwargs)
+ assert first.extent == pytest.approx(second.extent)
+ assert first.vmin == pytest.approx(second.vmin)
+ assert first.vmax == pytest.approx(second.vmax)
+ np.testing.assert_allclose(first.image, second.image, equal_nan=True)
+ np.testing.assert_array_equal(first.counts, second.counts)
+
+
+def test_raster_all_missing_color_has_finite_default_limits():
+ from scarf.plotting._raster import raster_from_metadata
+
+ cells = _GuardedMeta(
+ {
+ "I": np.ones(4, dtype=bool),
+ "x": np.arange(4, dtype=float),
+ "y": np.arange(4, dtype=float),
+ "value": np.full(4, np.nan),
+ }
+ )
+ canvas = raster_from_metadata(
+ cells,
+ x_key="x",
+ y_key="y",
+ color_key="value",
+ pixels=8,
+ )
+ assert (canvas.vmin, canvas.vmax) == (0.0, 1.0)
+ assert np.isnan(canvas.image).all()
+
+
+def test_specialized_facades_are_callable():
+ assert callable(splt.qc)
+ assert callable(splt.elbow)
+ assert callable(splt.graph_qc)
+ assert callable(splt.highly_variable_features)
+ assert callable(splt.marker_heatmap)
+ assert callable(splt.cluster_tree)
+ assert callable(splt.pseudotime_heatmap)
diff --git a/tests/test_plotting_style.py b/tests/test_plotting_style.py
new file mode 100644
index 00000000..eee05aab
--- /dev/null
+++ b/tests/test_plotting_style.py
@@ -0,0 +1,53 @@
+"""Plot scale and theme behavior tests."""
+
+import matplotlib
+import pytest
+
+matplotlib.use("Agg")
+
+from scarf.plotting._deps import require_matplotlib
+from scarf.plotting._style import (
+ categorical_color_map,
+ continuous_norm,
+ palette_for_n,
+ theme_context,
+)
+
+
+@pytest.mark.parametrize("size", [5, 15, 25, 50, 110])
+def test_palette_for_n_returns_requested_colors(size):
+ assert len(palette_for_n(size)) == size
+
+
+def test_categorical_color_map_validates_custom_palette():
+ with pytest.raises(KeyError, match="missing from palette"):
+ categorical_color_map(["a", "b"], palette={"a": "red"})
+ colors = categorical_color_map(
+ ["a"],
+ palette={"a": "red"},
+ missing_label="NA",
+ missing_color="gray",
+ )
+ assert colors == {"a": "red", "NA": "gray"}
+
+
+def test_continuous_norm_supports_center_and_validates_bounds():
+ _, mpl = require_matplotlib()
+ norm = continuous_norm(mpl, vmin=-2, vmax=3, vcenter=0)
+ assert norm.__class__.__name__ == "TwoSlopeNorm"
+ with pytest.raises(ValueError, match="vcenter"):
+ continuous_norm(mpl, vmin=0, vmax=3, vcenter=4)
+
+
+def test_theme_context_rejects_unknown_theme():
+ with pytest.raises(KeyError, match="Unknown theme"):
+ with theme_context("unknown"):
+ pass
+
+
+def test_theme_context_restores_matplotlib_state():
+ _, mpl = require_matplotlib()
+ original = mpl.rcParams["font.size"]
+ with theme_context("paper"):
+ assert mpl.rcParams["font.size"] == 8
+ assert mpl.rcParams["font.size"] == original
diff --git a/tests/test_prefetch.py b/tests/test_prefetch.py
new file mode 100644
index 00000000..3c643497
--- /dev/null
+++ b/tests/test_prefetch.py
@@ -0,0 +1,153 @@
+import threading
+import time
+from pathlib import Path
+
+import numpy as np
+import pytest
+
+from scarf.storage.budget import (
+ READ_AHEAD,
+ ResourceBudget,
+ set_resource_budget,
+ worker_prefetch_depth,
+)
+from scarf.storage.zarr_store import set_storage_profile
+from scarf.utils import (
+ ColumnBlockPipeline,
+ iter_column_blocks,
+ prefetch_blocks,
+ remote_column_disk_ahead,
+ remote_column_ram_ahead,
+)
+
+
+@pytest.fixture(autouse=True)
+def reset_profile():
+ set_storage_profile(None)
+ yield
+ set_storage_profile(None)
+
+
+def test_prefetch_blocks_preserves_order():
+ results = list(prefetch_blocks(range(5), lambda x: x * 2, max_ahead=2))
+ assert results == [0, 2, 4, 6, 8]
+
+
+def test_prefetch_blocks_empty():
+ assert list(prefetch_blocks(iter([]), lambda x: x)) == []
+
+
+def test_prefetch_blocks_single_item():
+ assert list(prefetch_blocks([42], lambda x: x + 1, max_ahead=4)) == [43]
+
+
+def test_prefetch_blocks_respects_max_ahead():
+ lock = threading.Lock()
+ in_flight = 0
+ max_seen = 0
+
+ def slow_fn(x):
+ nonlocal in_flight, max_seen
+ with lock:
+ in_flight += 1
+ max_seen = max(max_seen, in_flight)
+ time.sleep(0.02)
+ with lock:
+ in_flight -= 1
+ return x
+
+ list(prefetch_blocks(range(8), slow_fn, max_ahead=3))
+ assert max_seen <= 3
+
+
+def test_worker_prefetch_depth_from_budget():
+ set_resource_budget(
+ ResourceBudget(memoryBytes=8 * 1024**3, workers=4, workingCopies=8)
+ )
+ try:
+ assert worker_prefetch_depth() == READ_AHEAD
+ assert worker_prefetch_depth(requested=1) == 1
+ finally:
+ set_resource_budget(None)
+
+
+def test_iter_column_blocks_preserves_order(tmp_path: Path):
+ def read_block(block_idx: int) -> np.ndarray:
+ return np.full((4, 2), block_idx, dtype=np.float64)
+
+ seen = [
+ int(arr[0, 0])
+ for _, arr, _, _ in iter_column_blocks(
+ 3, read_block, disk_ahead=1, scratch_dir=str(tmp_path)
+ )
+ ]
+ assert seen == [0, 1, 2]
+
+
+def test_remote_column_disk_ahead_short_pipelines():
+ assert remote_column_disk_ahead(remote=True, n_blocks=4) == 0
+ assert remote_column_disk_ahead(remote=True, n_blocks=5) == 5
+ assert remote_column_disk_ahead(remote=False, n_blocks=10) == 0
+
+
+def test_remote_column_ram_ahead():
+ assert remote_column_ram_ahead(remote=False, n_blocks=4) == 1
+ assert remote_column_ram_ahead(remote=True, n_blocks=4) == 2
+ assert remote_column_ram_ahead(remote=True, n_blocks=2) == 1
+
+
+def test_column_block_pipeline_preserves_order(tmp_path: Path):
+ def read_block(block_idx: int) -> np.ndarray:
+ return np.full((4, 2), block_idx, dtype=np.float64)
+
+ with ColumnBlockPipeline(
+ 3, read_block, disk_ahead=1, scratch_dir=str(tmp_path)
+ ) as pipeline:
+ seen = []
+ for block_idx in range(3):
+ arr, _, _ = pipeline.take(block_idx)
+ seen.append(int(arr[0, 0]))
+ assert seen == [0, 1, 2]
+
+
+def test_column_block_pipeline_uses_disk_staging(tmp_path: Path):
+ reads: list[int] = []
+
+ def read_block(block_idx: int) -> np.ndarray:
+ reads.append(block_idx)
+ time.sleep(0.05)
+ return np.full((2, 1), block_idx, dtype=np.float64)
+
+ with ColumnBlockPipeline(
+ 4, read_block, disk_ahead=2, scratch_dir=str(tmp_path)
+ ) as pipeline:
+ for block_idx in range(4):
+ arr, wait, source = pipeline.take(block_idx)
+ assert int(arr[0, 0]) == block_idx
+ if block_idx >= 2:
+ assert source in {"ram", "disk"}
+ assert 0 in reads and 3 in reads
+
+
+def test_column_block_pipeline_stages_disk_serially(tmp_path: Path):
+ inflight = 0
+ max_inflight = 0
+ lock = threading.Lock()
+
+ def read_block(block_idx: int) -> np.ndarray:
+ nonlocal inflight, max_inflight
+ with lock:
+ inflight += 1
+ max_inflight = max(max_inflight, inflight)
+ time.sleep(0.05)
+ with lock:
+ inflight -= 1
+ return np.full((2, 1), block_idx, dtype=np.float64)
+
+ with ColumnBlockPipeline(
+ 5, read_block, disk_ahead=3, scratch_dir=str(tmp_path)
+ ) as pipeline:
+ for block_idx in range(5):
+ pipeline.take(block_idx)
+
+ assert max_inflight <= 2
diff --git a/tests/test_profiling_datasets_fixture.py b/tests/test_profiling_datasets_fixture.py
new file mode 100644
index 00000000..6ed90b48
--- /dev/null
+++ b/tests/test_profiling_datasets_fixture.py
@@ -0,0 +1,47 @@
+from pathlib import Path
+
+import h5py
+import numpy as np
+from scipy.sparse import csr_matrix
+
+from profiling.datasets import (
+ _load_string_column,
+ prepare_fixture_datasets,
+ write_fixture_h5ad,
+)
+
+
+def test_load_string_column_reads_categorical_feature_name(tmp_path: Path) -> None:
+ import anndata as ad
+ import pandas as pd
+
+ matrix = csr_matrix(np.array([[1.0, 0.0], [0.0, 2.0]], dtype=np.float32))
+ adata = ad.AnnData(matrix)
+ adata.obs_names = ["c0", "c1"]
+ adata.var_names = ["g0", "g1"]
+ adata.var["feature_name"] = pd.Categorical(["GeneA", "GeneB"])
+ path = tmp_path / "cat.h5ad"
+ adata.write_h5ad(path)
+
+ with h5py.File(path, "r") as h5:
+ assert isinstance(h5["var/feature_name"], h5py.Group)
+ names = _load_string_column(h5, "var/feature_name", expectedLength=2)
+ assert list(names) == ["GeneA", "GeneB"]
+
+
+def test_write_fixture_h5ad_is_readable_by_string_loader(tmp_path: Path) -> None:
+ path = tmp_path / "1000.h5ad"
+ artifact = write_fixture_h5ad(path, nRows=100, nColumns=50, seed=1)
+ assert artifact.targetRows == 100
+ assert path.is_file()
+ with h5py.File(path, "r") as h5:
+ names = _load_string_column(h5, "var/feature_name", expectedLength=50)
+ assert names.shape == (50,)
+ assert "MT-ND1" in set(names.tolist())
+
+
+def test_prepare_fixture_datasets_writes_requested_sizes(tmp_path: Path) -> None:
+ artifacts = prepare_fixture_datasets(tmp_path, targetRows=(10, 25), nColumns=20)
+ assert [item.targetRows for item in artifacts] == [10, 25]
+ assert (tmp_path / "10.h5ad").is_file()
+ assert (tmp_path / "25.h5ad").is_file()
diff --git a/tests/test_profiling_datasets_memory.py b/tests/test_profiling_datasets_memory.py
new file mode 100644
index 00000000..38b23db3
--- /dev/null
+++ b/tests/test_profiling_datasets_memory.py
@@ -0,0 +1,107 @@
+from pathlib import Path
+
+import h5py
+import numpy as np
+
+from profiling.config import load_profiling_config
+from profiling.datasets import (
+ SourceSpec,
+ load_csr_source_into_memory,
+ prepare_local_datasets,
+ select_nested_rows,
+ write_fixture_h5ad,
+ write_h5ad_sample,
+ write_h5ad_sample_from_memory,
+)
+
+
+def _fixture_spec(path: Path, *, nRows: int, nColumns: int, nnz: int) -> SourceSpec:
+ return SourceSpec(
+ datasetId="fixture",
+ versionId="fixture-v1",
+ url="file://fixture",
+ nRows=nRows,
+ nColumns=nColumns,
+ nnz=nnz,
+ sourceBytes=path.stat().st_size,
+ )
+
+
+def test_load_csr_downcasts_indices_to_int32(tmp_path: Path) -> None:
+ source = tmp_path / "source.h5ad"
+ artifact = write_fixture_h5ad(source, nRows=40, nColumns=25, seed=3)
+ spec = _fixture_spec(
+ source,
+ nRows=40,
+ nColumns=25,
+ nnz=artifact.nnz,
+ )
+ memory = load_csr_source_into_memory(source, spec=spec)
+ assert memory.indices.dtype == np.dtype(np.int32)
+ assert memory.indicesDtype == np.dtype(np.int64)
+ assert memory.data.dtype == np.dtype(np.float32)
+ assert int(memory.indptr[-1]) == artifact.nnz
+
+
+def test_write_from_memory_matches_disk_sample(tmp_path: Path) -> None:
+ source = tmp_path / "source.h5ad"
+ artifact = write_fixture_h5ad(source, nRows=60, nColumns=20, seed=4)
+ spec = _fixture_spec(
+ source,
+ nRows=60,
+ nColumns=20,
+ nnz=artifact.nnz,
+ )
+ rows = select_nested_rows(60, (15,), seed=1, sourceVersion=spec.versionId)[15]
+ memory = load_csr_source_into_memory(source, spec=spec)
+
+ disk_path = tmp_path / "disk.h5ad"
+ memory_path = tmp_path / "memory.h5ad"
+ disk = write_h5ad_sample(source, disk_path, rows, spec=spec)
+ mem = write_h5ad_sample_from_memory(memory, memory_path, rows)
+
+ assert disk.nnz == mem.nnz
+ assert disk.sourceRowsSha256 == mem.sourceRowsSha256
+ assert disk.finalSourceRow == mem.finalSourceRow
+
+ with h5py.File(disk_path, "r") as left, h5py.File(memory_path, "r") as right:
+ assert np.array_equal(left["X/data"][:], right["X/data"][:])
+ assert np.array_equal(left["X/indices"][:], right["X/indices"][:])
+ assert np.array_equal(left["X/indptr"][:], right["X/indptr"][:])
+
+
+def test_prepare_local_datasets_uses_in_memory_path(tmp_path: Path) -> None:
+ source = tmp_path / "source.h5ad"
+ artifact = write_fixture_h5ad(source, nRows=80, nColumns=30, seed=5)
+ spec = _fixture_spec(
+ source,
+ nRows=80,
+ nColumns=30,
+ nnz=artifact.nnz,
+ )
+ prepared = prepare_local_datasets(
+ source,
+ tmp_path / "subsets",
+ targetRows=(10, 25),
+ seed=0,
+ spec=spec,
+ )
+ assert [item.targetRows for item in prepared.artifacts] == [10, 25]
+ assert (tmp_path / "subsets" / "10.h5ad").is_file()
+ assert (tmp_path / "subsets" / "25.h5ad").is_file()
+
+ selections = select_nested_rows(
+ 80,
+ (10, 25),
+ seed=0,
+ sourceVersion=spec.versionId,
+ )
+ assert set(selections[10].tolist()).issubset(set(selections[25].tolist()))
+
+
+def test_example_config_loads_prepare_resources() -> None:
+ config = load_profiling_config(
+ Path(__file__).parents[1] / "profiling" / "config.example.toml"
+ )
+ assert config.prepareResources.modalMemoryRequestMb == 196_608
+ assert config.prepareResources.modalMemoryLimitMb == 212_992
diff --git a/tests/test_profiling_metrics.py b/tests/test_profiling_metrics.py
new file mode 100644
index 00000000..86b500bc
--- /dev/null
+++ b/tests/test_profiling_metrics.py
@@ -0,0 +1,321 @@
+import json
+import time
+from dataclasses import asdict, fields
+
+import pytest
+
+from profiling.metrics import (
+ DiskUsage,
+ ResourceMeasurement,
+ ResourceSampler,
+ StageTimer,
+ read_process_tree_rss_bytes,
+)
+
+
+def _write_status(proc_root, pid, parent_pid, rss_kib):
+ process = proc_root / str(pid)
+ process.mkdir(parents=True, exist_ok=True)
+ (process / "status").write_text(
+ f"Name:\tprocess-{pid}\nPPid:\t{parent_pid}\nVmRSS:\t{rss_kib} kB\n"
+ )
+
+
+def _write_cgroup(cgroup, *, current=100, peak=900):
+ cgroup.mkdir(parents=True, exist_ok=True)
+ (cgroup / "memory.current").write_text(str(current))
+ (cgroup / "memory.peak").write_text(str(peak))
+ (cgroup / "memory.max").write_text("4096")
+ (cgroup / "memory.swap.current").write_text("5")
+ (cgroup / "memory.swap.max").write_text("max")
+ (cgroup / "memory.events").write_text(
+ "low 0\nhigh 1\nmax 2\noom 0\noom_kill 0\noom_group_kill 0\n"
+ )
+ (cgroup / "cpu.max").write_text("200000 100000")
+
+
+def test_process_tree_rss_aggregates_descendants_by_parent_pid(tmp_path):
+ proc_root = tmp_path / "proc"
+ _write_status(proc_root, 100, 1, 10)
+ _write_status(proc_root, 101, 100, 20)
+ _write_status(proc_root, 102, 101, 30)
+ _write_status(proc_root, 200, 1, 400)
+ (proc_root / "103").mkdir()
+
+ assert read_process_tree_rss_bytes(100, procRoot=proc_root) == (10 + 20 + 30) * 1024
+
+
+def test_sampler_captures_events_limits_cpu_disk_and_operation_peak(tmp_path):
+ proc_root = tmp_path / "proc"
+ cgroup = tmp_path / "cgroup"
+ disk_path = tmp_path / "disk"
+ disk_path.mkdir()
+ _write_status(proc_root, 100, 1, 10)
+ _write_status(proc_root, 101, 100, 20)
+ _write_cgroup(cgroup)
+ disk_samples = iter(
+ (
+ DiskUsage(totalBytes=1000, freeBytes=900, usedBytes=100),
+ DiskUsage(totalBytes=1000, freeBytes=600, usedBytes=400),
+ DiskUsage(totalBytes=1000, freeBytes=750, usedBytes=250),
+ )
+ )
+
+ sampler = ResourceSampler(
+ sampleIntervalSeconds=60,
+ rootPid=100,
+ procRoot=proc_root,
+ cgroupPath=cgroup,
+ ephemeralDiskPath=disk_path,
+ affinityReader=lambda _pid: {3, 1},
+ diskUsageReader=lambda _path: next(disk_samples),
+ )
+ sampler.start()
+
+ assert (cgroup / "memory.peak").read_text() == "0"
+ _write_status(proc_root, 100, 1, 30)
+ _write_status(proc_root, 101, 100, 40)
+ (cgroup / "memory.current").write_text("350")
+ (cgroup / "memory.peak").write_text("420")
+ (cgroup / "memory.swap.current").write_text("20")
+ sampler.sample()
+
+ _write_status(proc_root, 100, 1, 15)
+ _write_status(proc_root, 101, 100, 10)
+ (cgroup / "memory.current").write_text("200")
+ (cgroup / "memory.swap.current").write_text("7")
+ (cgroup / "memory.events").write_text(
+ "low 0\nhigh 3\nmax 5\noom 1\noom_kill 1\noom_group_kill 1\n"
+ )
+ result = sampler.stop()
+
+ assert result.sampleCount == 3
+ assert result.processTreeRssBaselineBytes == 30 * 1024
+ assert result.processTreeRssPeakBytes == 70 * 1024
+ assert result.processTreeRssIncrementalPeakBytes == 40 * 1024
+ assert result.processTreeRssAfterBytes == 25 * 1024
+ assert result.cgroupMemoryCurrentBaselineBytes == 100
+ assert result.cgroupMemoryCurrentPeakBytes == 350
+ assert result.cgroupMemoryCurrentAfterBytes == 200
+ assert result.cgroupMemoryPeakBytes == 420
+ assert result.cgroupMemoryPeakScope == "operation"
+ assert result.operationBaselineBytes == 100
+ assert result.operationPeakBytes == 420
+ assert result.operationIncrementalPeakBytes == 320
+ assert result.operationPeakSource == "cgroupMemoryPeak"
+ assert result.memoryEventsDelta == {
+ "high": 2,
+ "low": 0,
+ "max": 3,
+ "oom": 1,
+ "oomGroupKill": 1,
+ "oomKill": 1,
+ }
+ assert result.memorySwapCurrentBeforeBytes == 5
+ assert result.memorySwapCurrentPeakBytes == 20
+ assert result.memorySwapCurrentAfterBytes == 7
+ assert result.memorySwapMaxBytes == "max"
+ assert result.memoryMaxBytes == 4096
+ assert result.cpuQuotaMicros == 200000
+ assert result.cpuPeriodMicros == 100000
+ assert result.cpuQuotaCores == 2.0
+ assert result.cpuAffinityCpus == (1, 3)
+ assert result.cpuAffinityCount == 2
+ assert result.ephemeralDiskBefore == DiskUsage(1000, 900, 100)
+ assert result.ephemeralDiskPeak == DiskUsage(1000, 600, 400)
+ assert result.ephemeralDiskAfter == DiskUsage(1000, 750, 250)
+ assert all("_" not in field.name for field in fields(ResourceMeasurement))
+ json.dumps(asdict(result))
+
+
+def test_unresettable_cgroup_peak_is_labeled_container_lifetime(tmp_path):
+ proc_root = tmp_path / "proc"
+ cgroup = tmp_path / "cgroup"
+ _write_status(proc_root, 100, 1, 10)
+ _write_cgroup(cgroup, current=100, peak=1000)
+
+ sampler = ResourceSampler(
+ sampleIntervalSeconds=60,
+ rootPid=100,
+ procRoot=proc_root,
+ cgroupPath=cgroup,
+ ephemeralDiskPath=None,
+ writeText=lambda _path, _value: None,
+ ).start()
+ (cgroup / "memory.current").write_text("300")
+ sampler.sample()
+ (cgroup / "memory.current").write_text("200")
+ result = sampler.stop()
+
+ assert result.cgroupMemoryPeakScope == "containerLifetime"
+ assert result.cgroupMemoryPeakBytes == 1000
+ assert result.operationPeakBytes == 300
+ assert result.operationPeakSource == "cgroupMemoryCurrent"
+ assert result.operationIncrementalPeakBytes == 200
+
+
+def test_sampler_reads_cgroup_v1_memory_files(tmp_path):
+ proc_root = tmp_path / "proc"
+ cgroup_root = tmp_path / "cgroup"
+ memory = cgroup_root / "memory" / "job-1"
+ memory.mkdir(parents=True)
+ (memory / "memory.usage_in_bytes").write_text("150")
+ (memory / "memory.max_usage_in_bytes").write_text("900")
+ (memory / "memory.limit_in_bytes").write_text("4096")
+ (memory / "memory.memsw.usage_in_bytes").write_text("10")
+ (memory / "memory.memsw.limit_in_bytes").write_text("max")
+ proc = proc_root / "100"
+ proc.mkdir(parents=True)
+ (proc / "status").write_text("Name:\tpython\nPPid:\t1\nVmRSS:\t20 kB\n")
+ (proc / "cgroup").write_text("6:memory:/job-1\n")
+
+ sampler = ResourceSampler(
+ sampleIntervalSeconds=60,
+ rootPid=100,
+ procRoot=proc_root,
+ cgroupRoot=cgroup_root,
+ ephemeralDiskPath=None,
+ ).start()
+ (memory / "memory.usage_in_bytes").write_text("400")
+ sampler.sample()
+ result = sampler.stop()
+
+ assert result.cgroupMemoryCurrentBaselineBytes == 150
+ assert result.cgroupMemoryCurrentPeakBytes == 400
+ assert result.memoryMaxBytes == 4096
+ assert result.cgroupMemoryPeakScope == "containerLifetime"
+ assert result.cgroupMemoryPeakBytes == 900
+ assert result.operationPeakSource == "cgroupMemoryCurrent"
+ assert result.operationPeakBytes == 400
+
+
+def test_sampler_falls_back_to_flat_cgroup_v1_memory(tmp_path):
+ proc_root = tmp_path / "proc"
+ cgroup_root = tmp_path / "cgroup"
+ flat = cgroup_root / "memory"
+ flat.mkdir(parents=True)
+ (flat / "memory.usage_in_bytes").write_text("120")
+ (flat / "memory.max_usage_in_bytes").write_text("500")
+ (flat / "memory.limit_in_bytes").write_text("4096")
+ proc = proc_root / "100"
+ proc.mkdir(parents=True)
+ (proc / "status").write_text("Name:\tpython\nPPid:\t1\nVmRSS:\t20 kB\n")
+ (proc / "cgroup").write_text("6:memory:/ta-missing-nested\n")
+
+ sampler = ResourceSampler(
+ sampleIntervalSeconds=60,
+ rootPid=100,
+ procRoot=proc_root,
+ cgroupRoot=cgroup_root,
+ ephemeralDiskPath=None,
+ ).start()
+ (flat / "memory.usage_in_bytes").write_text("300")
+ sampler.sample()
+ result = sampler.stop()
+
+ assert result.cgroupPath == str(flat)
+ assert result.cgroupMemoryCurrentBaselineBytes == 120
+ assert result.cgroupMemoryCurrentPeakBytes == 300
+ assert result.operationPeakSource == "cgroupMemoryCurrent"
+ assert result.memoryEventsBefore is None
+
+
+def test_background_sampler_observes_process_and_cgroup_peaks(tmp_path):
+ proc_root = tmp_path / "proc"
+ cgroup = tmp_path / "cgroup"
+ _write_status(proc_root, 100, 1, 10)
+ _write_status(proc_root, 101, 100, 10)
+ cgroup.mkdir()
+ (cgroup / "memory.current").write_text("100")
+
+ sampler = ResourceSampler(
+ sampleIntervalSeconds=0.005,
+ rootPid=100,
+ procRoot=proc_root,
+ cgroupPath=cgroup,
+ ephemeralDiskPath=None,
+ ).start()
+ count_before_peak = sampler.sampleCount
+ _write_status(proc_root, 100, 1, 50)
+ _write_status(proc_root, 101, 100, 70)
+ (cgroup / "memory.current").write_text("800")
+
+ deadline = time.monotonic() + 1
+ while sampler.sampleCount < count_before_peak + 2:
+ assert time.monotonic() < deadline
+ time.sleep(0.002)
+
+ _write_status(proc_root, 100, 1, 5)
+ _write_status(proc_root, 101, 100, 5)
+ (cgroup / "memory.current").write_text("50")
+ result = sampler.stop()
+
+ assert result.sampleCount >= 4
+ assert result.processTreeRssPeakBytes == 120 * 1024
+ assert result.cgroupMemoryCurrentPeakBytes == 800
+ assert result.operationPeakBytes == 800
+ assert result.cgroupMemoryPeakScope == "unavailable"
+
+
+def test_unavailable_and_protected_metrics_do_not_escape_context(tmp_path):
+ def unavailable(*_args):
+ raise PermissionError("protected")
+
+ sampler = ResourceSampler(
+ sampleIntervalSeconds=0.002,
+ rootPid=100,
+ procRoot=tmp_path / "proc",
+ cgroupPath=tmp_path / "cgroup",
+ ephemeralDiskPath=tmp_path / "disk",
+ readText=unavailable,
+ writeText=unavailable,
+ listPids=unavailable,
+ affinityReader=unavailable,
+ diskUsageReader=unavailable,
+ )
+
+ with pytest.raises(RuntimeError, match="measured failure"):
+ with sampler:
+ time.sleep(0.005)
+ raise RuntimeError("measured failure")
+
+ result = sampler.result
+ assert result is not None
+ assert result.processTreeRssPeakBytes is None
+ assert result.cgroupMemoryCurrentPeakBytes is None
+ assert result.cgroupMemoryPeakBytes is None
+ assert result.cgroupMemoryPeakScope == "unavailable"
+ assert result.memoryEventsBefore is None
+ assert result.memoryEventsAfter is None
+ assert result.memoryEventsDelta is None
+ assert result.cpuQuotaMicros is None
+ assert result.cpuAffinityCpus is None
+ assert result.ephemeralDiskBefore is None
+ assert result.ephemeralDiskAfter is None
+ assert result.ephemeralDiskPeak is None
+ assert result.sampleCount >= 2
+ json.dumps(asdict(result))
+
+
+def test_stage_timer_records_four_independent_scopes():
+ ticks = iter((0.0, 1.0, 3.0, 4.0, 9.0, 10.0, 12.0, 15.0))
+ timer = StageTimer(clock=lambda: next(ticks))
+
+ with timer:
+ with timer.inputSetup():
+ pass
+ with timer.operation():
+ pass
+ with timer.validationPersistence():
+ pass
+
+ assert timer.result.inputSetupSeconds == 2.0
+ assert timer.result.measuredOperationSeconds == 5.0
+ assert timer.result.validationPersistenceSeconds == 2.0
+ assert timer.result.wholeFunctionSeconds == 15.0
+ assert asdict(timer.result) == {
+ "inputSetupSeconds": 2.0,
+ "measuredOperationSeconds": 5.0,
+ "validationPersistenceSeconds": 2.0,
+ "wholeFunctionSeconds": 15.0,
+ }
diff --git a/tests/test_profiling_r2.py b/tests/test_profiling_r2.py
new file mode 100644
index 00000000..3dd20338
--- /dev/null
+++ b/tests/test_profiling_r2.py
@@ -0,0 +1,21 @@
+from profiling.r2 import join_uri, put_json
+from obstore.store import MemoryStore
+
+
+def test_join_uri():
+ assert (
+ join_uri("s3://bucket/prefix", "10000.h5ad") == "s3://bucket/prefix/10000.h5ad"
+ )
+ assert join_uri("s3://bucket/prefix/", "/a/", "b") == "s3://bucket/prefix/a/b"
+
+
+def test_memory_store_put_get_roundtrip(monkeypatch):
+ store = MemoryStore()
+
+ def fake_open(_uri: str):
+ return store, "results/10000/createStore.json"
+
+ monkeypatch.setattr("profiling.r2.open_r2_object", fake_open)
+ put_json("s3://bucket/results/10000/createStore.json", {"status": "ok"})
+ body = bytes(store.get("results/10000/createStore.json").bytes())
+ assert b'"status":"ok"' in body
diff --git a/tests/test_profiling_skip.py b/tests/test_profiling_skip.py
new file mode 100644
index 00000000..e2dacd19
--- /dev/null
+++ b/tests/test_profiling_skip.py
@@ -0,0 +1,66 @@
+from pathlib import Path
+
+from profiling.config import STAGE_ORDER, WorkflowParameters, load_profiling_config
+from profiling.results import result_exists
+from profiling.stages import StageRunResult
+
+
+def test_example_config_loads():
+ config = load_profiling_config(
+ Path(__file__).parents[1] / "profiling" / "config.example.toml"
+ )
+ assert config.modalEnvironmentName == "scarf_profiling"
+ assert set(config.stageResources) == set(STAGE_ORDER)
+ assert config.datasetUri(10_000).endswith("/10000.h5ad")
+ assert config.resultUri(10_000, "createStore").endswith(
+ "/results/10000/createStore.json"
+ )
+
+
+def test_run_tag_isolates_store_and_result_uris():
+ config = load_profiling_config(
+ Path(__file__).parents[1] / "profiling" / "layouts" / "100k_chunk256m.toml"
+ )
+ assert config.runTag == "chunk256m"
+ assert config.storageLayout.targetChunkBytes == 256 * 1024 * 1024
+ assert config.storeUri(100_000).endswith("/stores/chunk256m/100000.zarr")
+ assert config.resultUri(100_000, "markHvgs").endswith(
+ "/results/chunk256m/100000/markHvgs.json"
+ )
+
+
+def test_marker_group_key_matches_leiden_column():
+ workflow = WorkflowParameters()
+ assert workflow.resolvedMarkerGroupKey == "RNA_leiden_cluster"
+ assert workflow.resolvedHvgKey == "I__hvgs"
+
+
+def test_result_exists_skips_when_object_present(monkeypatch):
+ config = load_profiling_config(
+ Path(__file__).parents[1] / "profiling" / "config.example.toml"
+ )
+ monkeypatch.setattr(
+ "profiling.results.object_exists",
+ lambda uri: uri.endswith("/results/10000/createStore.json"),
+ )
+ assert result_exists(config, 10_000, "createStore") is True
+ assert result_exists(config, 10_000, "filterCells") is False
+
+
+def test_stage_run_result_json_shape():
+ result = StageRunResult(
+ stage="reopenStore",
+ nRows=10_000,
+ status="ok",
+ seconds=1.25,
+ peakRssBytes=1024,
+ peakCgroupBytes=2048,
+ modalMemoryMb=20480,
+ scarfMemoryBudget=12884901888,
+ storeUri="s3://bucket/stores/10000.zarr",
+ )
+ payload = result.to_json()
+ assert payload["stage"] == "reopenStore"
+ assert payload["nRows"] == 10_000
+ assert payload["status"] == "ok"
+ assert payload["seconds"] == 1.25
diff --git a/tests/test_pseudotime.py b/tests/test_pseudotime.py
new file mode 100644
index 00000000..4a554fc9
--- /dev/null
+++ b/tests/test_pseudotime.py
@@ -0,0 +1,358 @@
+import numpy as np
+import pandas as pd
+import pytest
+import zarr
+from scipy.sparse import csr_matrix
+from zarr.storage import MemoryStore
+
+from scarf.assay import (
+ PSEUDOTIME_AGGREGATION_SCHEMA_VERSION,
+ Assay,
+)
+from scarf.datastore.datastore import (
+ DataStore,
+ _scatter_feature_clusters,
+ _validated_pseudotime_regressor,
+)
+from scarf.datastore.graph_datastore import (
+ _make_source_sink_vector,
+ _random_walk_laplacian_transpose,
+ _select_pseudotime_component,
+ _truncated_pba_potential,
+ _validate_source_sink_labels,
+ _validate_source_sink_vector,
+)
+from scarf.markers import knn_clustering
+
+
+def test_source_sink_vector_supports_one_sided_supervision():
+ labels = pd.Series(["source", "source", "other", "other"])
+ vector = _make_source_sink_vector(labels, ["source"], [])
+
+ assert np.array_equal(vector, [-1.0, -1.0, 1.0, 1.0])
+ assert vector.sum() == pytest.approx(0.0)
+
+
+def test_source_sink_labels_reject_missing_and_overlap():
+ labels = pd.Series(["a", "b", "c"])
+
+ with pytest.raises(ValueError, match="overlap"):
+ _validate_source_sink_labels(labels, ["a"], ["a"], "test cells")
+ with pytest.raises(ValueError, match="Missing sources"):
+ _validate_source_sink_labels(labels, ["missing"], [], "test cells")
+
+
+def test_source_sink_vector_rejects_unbalanced_and_nonfinite_values():
+ for values in (
+ np.array([-2.0, 1.0]),
+ np.array([-1.0, 2.0]),
+ np.array([-1.0, np.nan]),
+ np.array([-1.0, np.inf]),
+ ):
+ with pytest.raises(ValueError):
+ _validate_source_sink_vector(values, 2, "ss_vec")
+
+ column = _validate_source_sink_vector(
+ np.array([[-1.0], [1.0]]),
+ 2,
+ "ss_vec",
+ )
+ assert column.shape == (2,)
+
+
+def test_source_sink_vector_rejects_wrong_shapes():
+ for values in (
+ np.zeros((2, 2)),
+ np.zeros((1, 2)),
+ np.zeros(3),
+ ):
+ with pytest.raises(ValueError):
+ _validate_source_sink_vector(values, 2, "ss_vec")
+
+
+def test_source_sink_vector_rejects_fully_labelled_cells():
+ labels = pd.Series(["source", "sink"])
+ with pytest.raises(ValueError, match="All selected cells"):
+ _make_source_sink_vector(labels, ["source"], ["sink"])
+
+
+def test_largest_component_selection_is_deterministic_on_ties():
+ adjacency = np.zeros((6, 6), dtype=float)
+ for start in (0, 3):
+ adjacency[start, start + 1] = adjacency[start + 1, start] = 1.0
+ adjacency[start + 1, start + 2] = adjacency[start + 2, start + 1] = 1.0
+ selected_indices = np.array([10, 11, 12, 1, 2, 3])
+
+ retained, sizes = _select_pseudotime_component(
+ csr_matrix(adjacency),
+ selected_indices,
+ "largest",
+ )
+
+ assert sizes == [3, 3]
+ assert np.array_equal(retained, [False, False, False, True, True, True])
+ with pytest.raises(ValueError, match="connected components"):
+ _select_pseudotime_component(
+ csr_matrix(adjacency),
+ selected_indices,
+ "error",
+ )
+
+
+def test_truncated_pba_operator_matches_dense_svd_reference():
+ adjacency = np.zeros((8, 8), dtype=float)
+ weights = [1.0, 2.0, 1.5, 3.0, 0.75, 2.5, 1.25]
+ for index, weight in enumerate(weights):
+ adjacency[index, index + 1] = weight
+ adjacency[index + 1, index] = weight
+ laplacian_transpose = _random_walk_laplacian_transpose(csr_matrix(adjacency))
+ source_sink = np.array([-1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0])
+ k = 5
+
+ actual = _truncated_pba_potential(
+ laplacian_transpose,
+ k,
+ 17,
+ source_sink,
+ )
+
+ left, singular_values, right_t = np.linalg.svd(
+ laplacian_transpose.toarray(),
+ full_matrices=False,
+ )
+ retained_modes = np.argsort(singular_values)[:k]
+ retained_modes = retained_modes[np.argsort(singular_values[retained_modes])]
+ nonzero_modes = retained_modes[1:]
+ expected = left[:, nonzero_modes] @ (
+ (1.0 / singular_values[nonzero_modes])
+ * (right_t[nonzero_modes, :] @ source_sink)
+ )
+
+ assert np.allclose(actual, expected, rtol=1e-6, atol=1e-8)
+
+
+def test_dense_pba_reference_orders_a_path_from_source_to_sink():
+ adjacency = np.zeros((7, 7), dtype=float)
+ for index in range(6):
+ adjacency[index, index + 1] = 1.0
+ adjacency[index + 1, index] = 1.0
+ laplacian_transpose = _random_walk_laplacian_transpose(csr_matrix(adjacency))
+ source_sink = np.array([-1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0])
+
+ potential = np.linalg.pinv(laplacian_transpose.toarray().T) @ source_sink
+
+ assert potential[0] < potential[-1]
+ assert np.all(np.diff(potential) > 0)
+
+
+class _FakeCells:
+ def __init__(self, values: np.ndarray, columns: list[str] | None = None):
+ self.values = values
+ self.columns = ["ptime"] if columns is None else columns
+
+ def fetch(self, _column: str, key: str) -> np.ndarray:
+ assert key
+ return self.values
+
+ def active_index(self, _key: str) -> np.ndarray:
+ return np.arange(self.values.shape[0])
+
+
+class _FakeFeatures:
+ def __init__(self):
+ self.insertions: list[tuple[str, np.ndarray, str]] = []
+
+ @staticmethod
+ def active_index(key: str) -> np.ndarray:
+ assert key == "subset"
+ return np.array([1, 3])
+
+ def insert(
+ self,
+ column_name: str,
+ values: np.ndarray,
+ *,
+ key: str,
+ overwrite: bool,
+ ) -> None:
+ assert overwrite
+ self.insertions.append((column_name, values, key))
+
+
+class _MarkerAssay:
+ def __init__(self):
+ self.cells = _FakeCells(np.array([0.0, 1.0, 2.0, 3.0]))
+ self.feats = _FakeFeatures()
+
+ @staticmethod
+ def iter_normed_feature_wise(**_kwargs):
+ yield pd.DataFrame(
+ {
+ 1: [0.0, 1.0, 2.0, 3.0],
+ 3: [3.0, 2.0, 1.0, 0.0],
+ }
+ )
+
+
+class _MarkerStore:
+ def __init__(self):
+ self.assay = _MarkerAssay()
+
+ def _get_assay(self, _from_assay: str):
+ return self.assay
+
+
+def test_marker_search_writes_strict_feature_subset_with_subset_key():
+ store = _MarkerStore()
+
+ DataStore.run_pseudotime_marker_search(
+ store,
+ from_assay="RNA",
+ cell_key="I",
+ feat_key="subset",
+ pseudotime_key="ptime",
+ min_cells=1,
+ )
+
+ assert [item[2] for item in store.assay.feats.insertions] == [
+ "subset",
+ "subset",
+ ]
+ assert all(item[1].shape == (2,) for item in store.assay.feats.insertions)
+
+
+def test_regressor_validation_points_to_component_validity_key():
+ assay = type(
+ "FakeAssay",
+ (),
+ {
+ "cells": _FakeCells(
+ np.array([0.0, np.nan]),
+ ["ptime", "ptime__valid"],
+ )
+ },
+ )()
+
+ with pytest.raises(ValueError, match="ptime__valid"):
+ _validated_pseudotime_regressor(assay, "I", "ptime")
+
+
+class _AggregationCells:
+ def __init__(self, ordering: np.ndarray):
+ self.ordering = ordering
+
+ def fetch(self, _ordering_key: str, key: str) -> np.ndarray:
+ assert key == "I"
+ return self.ordering
+
+
+class _AggregationAssay:
+ def __init__(self, expression: np.ndarray, ordering: np.ndarray):
+ self.expression = expression
+ self.cells = _AggregationCells(ordering)
+ self.z = zarr.open_group(store=MemoryStore(), mode="w")
+ self.nthreads = 1
+
+ def _get_cell_feat_idx(
+ self,
+ _cell_key: str,
+ _feat_key: str,
+ ) -> tuple[np.ndarray, np.ndarray]:
+ return np.arange(self.expression.shape[0]), np.arange(self.expression.shape[1])
+
+ def iter_normed_feature_wise(self, *_args, **_kwargs):
+ yield pd.DataFrame(
+ self.expression,
+ columns=np.arange(self.expression.shape[1]),
+ )
+
+
+def test_aggregation_orders_without_smoothing_and_filters_constant_profiles():
+ expression = np.array(
+ [
+ [10.0, 5.0],
+ [20.0, 5.0],
+ [30.0, 5.0],
+ [40.0, 5.0],
+ ]
+ )
+ assay = _AggregationAssay(expression, np.array([2.0, 0.0, 3.0, 1.0]))
+
+ aggregated, feature_indices = Assay.save_aggregated_ordering(
+ assay,
+ cell_key="I",
+ feat_key="I",
+ ordering_key="ptime",
+ min_exp=0.0,
+ smoothen=False,
+ z_scale=False,
+ window_size=20,
+ chunk_size=20,
+ batch_size=2,
+ )
+
+ assert np.array_equal(feature_indices, [0])
+ assert np.array_equal(
+ aggregated.compute(),
+ np.array([[20.0, 40.0, 10.0, 30.0]]),
+ )
+ group = assay.z["aggregated_I_I_ptime"]
+ assert np.array_equal(group["valid_features"][:], [True, False])
+ assert np.isfinite(group["data"][:]).all()
+ assert group.attrs["schema_version"] == PSEUDOTIME_AGGREGATION_SCHEMA_VERSION
+ assert all(isinstance(value, str) for value in group.attrs["hashes"])
+
+
+def test_old_aggregation_schema_is_rebuilt():
+ expression = np.array([[1.0, 2.0], [2.0, 3.0], [3.0, 4.0], [4.0, 5.0]])
+ assay = _AggregationAssay(expression, np.arange(4, dtype=float))
+ kwargs = {
+ "cell_key": "I",
+ "feat_key": "I",
+ "ordering_key": "ptime",
+ "smoothen": False,
+ "z_scale": False,
+ "window_size": 2,
+ "chunk_size": 2,
+ "batch_size": 2,
+ }
+
+ Assay.save_aggregated_ordering(assay, **kwargs)
+ group = assay.z["aggregated_I_I_ptime"]
+ group.attrs["schema_version"] = 1
+ group["data"][:] = 999.0
+
+ aggregated, _ = Assay.save_aggregated_ordering(assay, **kwargs)
+
+ assert not np.all(aggregated.compute() == 999.0)
+ assert (
+ assay.z["aggregated_I_I_ptime"].attrs["schema_version"]
+ == PSEUDOTIME_AGGREGATION_SCHEMA_VERSION
+ )
+
+
+class _ShapeOnlyArray:
+ def __init__(self, shape: tuple[int, int]):
+ self.shape = shape
+
+
+def test_knn_clustering_rejects_infeasible_parameters():
+ with pytest.raises(ValueError, match="At least two"):
+ knn_clustering(_ShapeOnlyArray((1, 3)), 1, 1, 1)
+ with pytest.raises(ValueError, match="n_neighbours"):
+ knn_clustering(_ShapeOnlyArray((3, 3)), 3, 2, 1)
+ with pytest.raises(ValueError, match="n_clusters"):
+ knn_clustering(_ShapeOnlyArray((3, 3)), 1, 4, 1)
+
+
+def test_feature_cluster_scatter_honors_custom_unassigned_value():
+ values = _scatter_feature_clusters(
+ 5,
+ np.array([1, 3]),
+ np.array([1, 2]),
+ 0,
+ )
+
+ assert np.array_equal(values, [0, 1, 0, 2, 0])
+ with pytest.raises(ValueError, match="conflicts"):
+ _scatter_feature_clusters(3, np.array([0]), np.array([1]), 1)
diff --git a/tests/test_readers.py b/tests/test_readers.py
new file mode 100644
index 00000000..7451dd4a
--- /dev/null
+++ b/tests/test_readers.py
@@ -0,0 +1,526 @@
+import numpy as np
+import pytest
+
+
+def _write_sparse_h5ad(path, values, *, encoding_type="csr_matrix"):
+ import h5py
+ from scipy.sparse import csr_matrix
+
+ matrix = csr_matrix(values)
+ with h5py.File(path, mode="w") as h5:
+ sparse = h5.create_group("X")
+ sparse.attrs["encoding-type"] = encoding_type
+ sparse.attrs["encoding-version"] = "0.1.0"
+ sparse.attrs["shape"] = matrix.shape
+ sparse.create_dataset("data", data=matrix.data)
+ sparse.create_dataset("indices", data=matrix.indices.astype(np.int64))
+ sparse.create_dataset("indptr", data=matrix.indptr.astype(np.int64))
+
+ obs = h5.create_group("obs")
+ obs.create_dataset(
+ "_index",
+ data=np.array(
+ [f"cell_{index}".encode() for index in range(matrix.shape[0])]
+ ),
+ )
+ var = h5.create_group("var")
+ var.create_dataset(
+ "_index",
+ data=np.array(
+ [f"feature_{index}".encode() for index in range(matrix.shape[1])]
+ ),
+ )
+ var.create_dataset(
+ "feature_name",
+ data=np.array(
+ [f"gene_{index}".encode() for index in range(matrix.shape[1])]
+ ),
+ )
+ h5.create_group("obsm")
+
+
+def test_toy_crdir_assay_feats_table(toy_crdir_reader):
+ assert np.all(
+ toy_crdir_reader.assayFeats.columns
+ == np.array(["RNA", "ADT", "RNA", "HTO", "RNA"])
+ )
+ assert np.all(
+ toy_crdir_reader.assayFeats.values[1:]
+ == [[0, 1, 3, 5, 6], [1, 3, 5, 6, 7], [1, 2, 2, 1, 1]]
+ )
+
+
+def test_toy_crdir_reader_cells_feats(toy_crdir_reader):
+ assert toy_crdir_reader.nCells == 3
+ assert toy_crdir_reader.nFeatures == 8
+ assert toy_crdir_reader.cell_names() == ["b1", "b2", "b3"]
+ assert toy_crdir_reader.feature_names() == [
+ "g1",
+ "a1",
+ "a2",
+ "g2",
+ "g3",
+ "h1",
+ "g4",
+ "a3",
+ ]
+ assert toy_crdir_reader.feature_ids() == [
+ "g1",
+ "a1",
+ "a2",
+ "g2",
+ "g3",
+ "h1",
+ "g4",
+ "a3",
+ ]
+
+
+def test_toy_crdir_reader_assay_subsets(toy_crdir_reader):
+ assert toy_crdir_reader.feature_names("RNA") == ["g1", "g2", "g3", "g4"]
+ assert toy_crdir_reader.feature_ids("ADT") == ["a1", "a2"]
+ assert toy_crdir_reader.feature_names("HTO") == ["h1"]
+
+ with pytest.raises(ValueError, match="Assay ID missing is not valid"):
+ toy_crdir_reader.feature_names("missing")
+
+
+def test_crdir_reader_filters_and_streams_selected_barcodes(tmp_path):
+ from scarf.readers import CrDirReader
+
+ (tmp_path / "features.tsv").write_text(
+ "\n".join(
+ [
+ "f1\tg1\tGene Expression",
+ "f2\tg2\tGene Expression",
+ "f3\tg3\tGene Expression",
+ ]
+ )
+ + "\n"
+ )
+ (tmp_path / "barcodes.tsv").write_text("b1\nb2\nb3\nb4\n")
+ (tmp_path / "matrix.mtx").write_text(
+ "\n".join(
+ [
+ "%%MatrixMarket matrix coordinate integer general",
+ "% tiny deterministic matrix",
+ "3 4 5",
+ "1 1 2",
+ "2 1 3",
+ "1 2 6",
+ "3 3 1",
+ "2 4 4",
+ ]
+ )
+ + "\n"
+ )
+
+ reader = CrDirReader(
+ str(tmp_path),
+ is_filtered=False,
+ filtering_cutoff=4,
+ )
+
+ np.testing.assert_array_equal(reader.validBarcodeIdx, np.array([1, 2]))
+ assert reader.nCells == 2
+ assert reader.cell_names() == ["b1", "b2"]
+ assert reader.read_header().iloc[0].to_dict() == {
+ "nFeatures": 3,
+ "nCells": 4,
+ "nCounts": 5,
+ }
+ np.testing.assert_array_equal(
+ reader._get_valid_barcodes(
+ filtering_cutoff=4,
+ batch_size=2,
+ lines_in_mem=2,
+ ),
+ np.array([1, 2]),
+ )
+
+ chunks = list(reader.consume(batch_size=1, lines_in_mem=2, dtype=np.uint16))
+ assert [chunk.shape for chunk in chunks] == [(1, 3), (1, 3)]
+ assert all(chunk.dtype == np.uint16 for chunk in chunks)
+ np.testing.assert_array_equal(chunks[0].toarray(), [[2, 3, 0]])
+ np.testing.assert_array_equal(chunks[1].toarray(), [[6, 0, 0]])
+
+
+def test_crdir_reader_supports_gzip_and_metadata_fallback(tmp_path):
+ import gzip
+
+ from scarf.readers import CrDirReader
+
+ files = {
+ "genes.tsv.gz": "f1\nf2\n",
+ "barcodes.tsv.gz": "b1\nb2\n",
+ "matrix.mtx.gz": "\n".join(
+ [
+ "%%MatrixMarket matrix coordinate integer general",
+ "2 2 2",
+ "1 1 7",
+ "2 2 9",
+ ]
+ )
+ + "\n",
+ }
+ for name, contents in files.items():
+ with gzip.open(tmp_path / name, mode="wt") as handle:
+ handle.write(contents)
+
+ reader = CrDirReader(str(tmp_path))
+
+ assert reader.feature_ids() == ["f1", "f2"]
+ assert reader.feature_names() == ["f1", "f2"]
+ assert reader.feature_types() == ["Gene Expression", "Gene Expression"]
+ assert reader.cell_names() == ["b1", "b2"]
+ with pytest.raises(ValueError, match="Dataset key must be provided"):
+ reader._read_dataset()
+
+ chunks = list(reader.consume(batch_size=2, lines_in_mem=1))
+ assert len(chunks) == 1
+ np.testing.assert_array_equal(chunks[0].toarray(), [[7, 0], [0, 9]])
+
+
+def test_toy_crdir_empty(toy_crdir_empty):
+ assert toy_crdir_empty.nCells == 0
+ assert toy_crdir_empty.nFeatures == 4
+ assert toy_crdir_empty.feature_names() == [
+ "g1",
+ "a1",
+ "a2",
+ "g2",
+ ]
+ assert toy_crdir_empty.feature_ids() == [
+ "g1",
+ "a1",
+ "a2",
+ "g2",
+ ]
+ # check for raise ValueError
+ try:
+ toy_crdir_empty.read_header()
+ except ValueError:
+ pass
+
+
+def test_crh5reader(crh5_reader):
+ assert crh5_reader.nCells == 892
+ assert crh5_reader.nFeatures == 36611
+ n_assay_feats = list(crh5_reader.assayFeats.T.nFeatures.values)
+ assert n_assay_feats == [36601, 10]
+
+
+def test_crh5reader_streams_counts(crh5_reader):
+ streamed_rows = 0
+ streamed_nnz = 0
+
+ for chunk in crh5_reader.consume(batch_size=300):
+ assert 0 < chunk.shape[0] <= 300
+ assert chunk.shape[1] == crh5_reader.nFeatures
+ streamed_rows += chunk.shape[0]
+ streamed_nnz += chunk.nnz
+
+ assert streamed_rows == crh5_reader.nCells
+ assert streamed_nnz == crh5_reader.grp["data"].shape[0]
+ assert len(crh5_reader.cell_names()) == crh5_reader.nCells
+
+
+def test_crh5reader_filters_background_barcodes(crh5_reader):
+ from scarf.readers import CrH5Reader
+
+ reader = CrH5Reader(
+ crh5_reader.h5obj.filename,
+ is_filtered=False,
+ filtering_cutoff=0,
+ )
+ try:
+ indptr = reader.grp["indptr"][:]
+ expected = np.flatnonzero(np.diff(indptr) > 0)
+ np.testing.assert_array_equal(reader.validBarcodeIdx, expected)
+ assert reader.cell_names() == list(
+ np.asarray(crh5_reader.cell_names())[expected]
+ )
+ finally:
+ reader.close()
+
+
+def test_crdir_reader(crdir_reader):
+ assert crdir_reader.nCells == 892
+ assert crdir_reader.nFeatures == 36601 # Does not contain 10 ADTs
+
+
+def test_h5ad_reader(h5ad_reader):
+ assert h5ad_reader.nCells == 3696 == len(h5ad_reader.cell_ids())
+ assert h5ad_reader.nFeatures == 27998 == len(h5ad_reader.feat_names())
+
+
+def test_h5ad_reader_streams_sparse_matrix(h5ad_reader):
+ streamed_rows = 0
+ streamed_nnz = 0
+ streamed_sum = 0.0
+
+ for chunk in h5ad_reader.consume(batch_size=1000):
+ assert 0 < chunk.shape[0] <= 1000
+ assert chunk.shape[1] == h5ad_reader.nFeatures
+ assert chunk.dtype == h5ad_reader.matrixDtype
+ streamed_rows += chunk.shape[0]
+ streamed_nnz += chunk.nnz
+ streamed_sum += chunk.data.sum(dtype=np.float64)
+
+ matrix_data = h5ad_reader.h5[h5ad_reader.matrixKey]["data"]
+ assert streamed_rows == h5ad_reader.nCells
+ assert streamed_nnz == matrix_data.shape[0]
+ np.testing.assert_allclose(
+ streamed_sum,
+ matrix_data[:].sum(dtype=np.float64),
+ )
+
+
+@pytest.mark.parametrize(
+ ("values", "batch_size"),
+ [
+ (
+ np.array(
+ [
+ [1, 0, 2],
+ [0, 0, 0],
+ [0, 3, 0],
+ [0, 0, 0],
+ ],
+ dtype=np.uint32,
+ ),
+ 2,
+ ),
+ (
+ np.array(
+ [
+ [1, 0, 0],
+ [0, 2, 0],
+ [0, 0, 0],
+ [3, 0, 4],
+ [0, 5, 0],
+ ],
+ dtype=np.uint32,
+ ),
+ 2,
+ ),
+ ],
+)
+def test_h5ad_reader_preserves_sparse_batches(tmp_path, values, batch_size):
+ from scarf.readers import H5adReader
+
+ file_name = tmp_path / "sparse.h5ad"
+ _write_sparse_h5ad(file_name, values)
+ reader = H5adReader(str(file_name), feature_name_key="feature_name")
+ try:
+ chunks = list(reader.consume(batch_size=batch_size))
+ assert sum(chunk.shape[0] for chunk in chunks) == values.shape[0]
+ assert sum(chunk.nnz for chunk in chunks) == np.count_nonzero(values)
+ np.testing.assert_array_equal(
+ np.vstack([chunk.toarray() for chunk in chunks]),
+ values,
+ )
+ finally:
+ reader.h5.close()
+
+
+def test_h5ad_reader_rejects_csc_sparse_encoding(tmp_path):
+ from scarf.readers import H5adReader
+
+ file_name = tmp_path / "csc.h5ad"
+ _write_sparse_h5ad(
+ file_name,
+ np.eye(3, dtype=np.uint32),
+ encoding_type="csc_matrix",
+ )
+
+ with pytest.raises(ValueError, match="requires CSR encoding"):
+ H5adReader(str(file_name), feature_name_key="feature_name")
+
+
+def test_h5ad_to_zarr_preserves_exact_sparse_batch(tmp_path):
+ import zarr
+
+ from scarf.readers import H5adReader
+ from scarf.writers import H5adToZarr
+
+ values = np.array(
+ [
+ [1, 0, 2],
+ [0, 0, 0],
+ [0, 3, 0],
+ [0, 0, 0],
+ ],
+ dtype=np.uint32,
+ )
+ file_name = tmp_path / "exact_batch.h5ad"
+ zarr_path = tmp_path / "exact_batch.zarr"
+ _write_sparse_h5ad(file_name, values)
+ reader = H5adReader(str(file_name), feature_name_key="feature_name")
+ try:
+ writer = H5adToZarr(reader, zarr_loc=str(zarr_path))
+ writer.dump(batch_size=2)
+ finally:
+ reader.h5.close()
+
+ root = zarr.open_group(str(zarr_path), mode="r")
+ np.testing.assert_array_equal(root["RNA/counts"][:], values)
+
+
+def test_h5ad_reader_streams_cell_and_feature_metadata(h5ad_reader):
+ cell_columns = dict(h5ad_reader.get_cell_columns())
+
+ assert "index" not in cell_columns
+ assert {
+ "clusters_coarse",
+ "clusters",
+ "S_score",
+ "G2M_score",
+ "X_pca1",
+ "X_pca50",
+ "X_umap1",
+ "X_umap2",
+ } <= cell_columns.keys()
+ assert all(
+ values.shape == (h5ad_reader.nCells,) for values in cell_columns.values()
+ )
+ np.testing.assert_array_equal(
+ cell_columns["clusters_coarse"][:3],
+ np.array([b"Pre-endocrine", b"Ductal", b"Endocrine"]),
+ )
+ np.testing.assert_allclose(
+ cell_columns["X_umap1"][:3],
+ np.array([6.143066, -9.906417, 7.559791], dtype=np.float32),
+ )
+
+ feature_columns = dict(h5ad_reader.get_feat_columns())
+ assert feature_columns.keys() == {"highly_variable_genes"}
+ assert feature_columns["highly_variable_genes"].shape == (h5ad_reader.nFeatures,)
+ np.testing.assert_array_equal(
+ feature_columns["highly_variable_genes"][:3],
+ np.array([b"False", b"True", b"True"]),
+ )
+ np.testing.assert_array_equal(
+ h5ad_reader._replace_category_values(
+ np.array([10_000]),
+ "clusters",
+ h5ad_reader.cellAttrsKey,
+ ),
+ np.array([10_000]),
+ )
+ assert h5ad_reader._check_exists("layers", "spliced")
+
+
+def test_h5ad_reader_dense_matrix_and_group_metadata(tmp_path):
+ import h5py
+
+ from scarf.readers import H5adReader
+
+ file_name = tmp_path / "dense.h5ad"
+ with h5py.File(file_name, mode="w") as h5:
+ h5.create_dataset(
+ "X",
+ data=np.array([[1, 0], [0, 2], [3, 4]], dtype=np.float32),
+ )
+ obs = h5.create_group("obs")
+ obs.create_dataset("_index", data=np.array([b"c1", b"c2", b"c3"]))
+ obs.create_dataset("batch", data=np.array([0, 1, 0], dtype=np.int8))
+ obs_categories = obs.create_group("__categories")
+ obs_categories.create_dataset("batch", data=np.array([b"A", b"B"]))
+
+ var = h5.create_group("var")
+ var.create_dataset("_index", data=np.array([b"f1", b"f2"]))
+ feature_names = var.create_group("gene_short_name")
+ feature_names.create_dataset("codes", data=np.array([1, 0]))
+ feature_names.create_dataset(
+ "categories",
+ data=np.array([b"Gene A", b"Gene B"]),
+ )
+ var.create_dataset("chromosome", data=np.array([b"1", b"2"]))
+
+ obsm = h5.create_group("obsm")
+ obsm.create_dataset(
+ "X_embed",
+ data=np.array([[1, 2], [3, 4], [5, 6]], dtype=np.float32),
+ )
+ obsm.create_dataset(
+ "bad_embed",
+ data=np.array([[1, 2], [3, 4]], dtype=np.float32),
+ )
+
+ reader = H5adReader(str(file_name))
+ try:
+ assert reader.groupCodes == {"obs": 2, "var": 2, "obsm": 2, "X": 1}
+ np.testing.assert_array_equal(reader.cell_ids(), [b"c1", b"c2", b"c3"])
+ np.testing.assert_array_equal(reader.feat_ids(), [b"f1", b"f2"])
+ np.testing.assert_array_equal(reader.feat_names(), [b"Gene B", b"Gene A"])
+
+ cell_columns = dict(reader.get_cell_columns())
+ assert cell_columns.keys() == {"batch", "X_embed1", "X_embed2"}
+ np.testing.assert_array_equal(cell_columns["batch"], [b"A", b"B", b"A"])
+ np.testing.assert_array_equal(cell_columns["X_embed2"], [2, 4, 6])
+
+ feature_columns = dict(reader.get_feat_columns())
+ assert feature_columns.keys() == {"chromosome"}
+ np.testing.assert_array_equal(feature_columns["chromosome"], [b"1", b"2"])
+
+ chunks = [chunk.toarray() for chunk in reader.consume(batch_size=2)]
+ assert len(chunks) == 2
+ np.testing.assert_array_equal(chunks[0], [[1, 0], [0, 2]])
+ np.testing.assert_array_equal(chunks[1], [[3, 4]])
+ finally:
+ reader.h5.close()
+
+
+def test_h5ad_reader_falls_back_without_metadata_groups(tmp_path):
+ import h5py
+
+ from scarf.readers import H5adReader
+
+ file_name = tmp_path / "sparse_without_metadata.h5ad"
+ with h5py.File(file_name, mode="w") as h5:
+ matrix = h5.create_group("X")
+ matrix.create_dataset("data", data=np.array([5, 8], dtype=np.int16))
+ matrix.create_dataset("indices", data=np.array([0, 1]))
+ matrix.create_dataset("indptr", data=np.array([0, 1, 2]))
+ matrix.create_dataset("shape", data=np.array([2, 2]))
+
+ reader = H5adReader(str(file_name))
+ try:
+ assert reader.nCells == reader.nFeatures == 2
+ np.testing.assert_array_equal(reader.cell_ids(), ["cell_0", "cell_1"])
+ np.testing.assert_array_equal(
+ reader.feat_ids(),
+ ["feature_0", "feature_1"],
+ )
+ np.testing.assert_array_equal(reader.feat_names(), reader.feat_ids())
+ assert list(reader.get_cell_columns()) == []
+ assert list(reader.get_feat_columns()) == []
+
+ chunks = list(reader.consume(batch_size=3))
+ assert len(chunks) == 1
+ np.testing.assert_array_equal(chunks[0].toarray(), [[5, 0], [0, 8]])
+ finally:
+ reader.h5.close()
+
+
+def test_loom_reader(loom_reader):
+ assert loom_reader.nCells == 298 == len(loom_reader.cell_ids())
+ assert loom_reader.nFeatures == 16892 == len(loom_reader.feature_names())
+
+
+def test_loom_reader_streams_metadata_and_counts(loom_reader):
+ cell_attributes = dict(loom_reader.get_cell_attrs())
+ assert cell_attributes.keys() == {"Area", "Cell_cluster"}
+ assert all(
+ values.shape == (loom_reader.nCells,) for values in cell_attributes.values()
+ )
+ assert dict(loom_reader.get_feature_attrs()) == {}
+
+ chunks = list(loom_reader.consume(batch_size=100))
+ assert [chunk.shape for chunk in chunks] == [
+ (100, loom_reader.nFeatures),
+ (100, loom_reader.nFeatures),
+ (98, loom_reader.nFeatures),
+ ]
+ assert sum(chunk.nnz for chunk in chunks) > 0
diff --git a/tests/test_renorm_subset.py b/tests/test_renorm_subset.py
new file mode 100644
index 00000000..f5e49e05
--- /dev/null
+++ b/tests/test_renorm_subset.py
@@ -0,0 +1,151 @@
+import numpy as np
+
+from scarf.assay import RNAassay
+from scarf.writers import write_renorm_subset_to_zarr
+
+
+def _subset_indices(rna):
+ cell_idx = np.arange(rna.cells.N)
+ feat_idx = np.array([0, 1, 3])
+ return cell_idx, feat_idx
+
+
+def _reference_renorm_subset(rna, cell_idx, feat_idx, log_transform=False):
+ raw = rna.rawData[cell_idx, :][:, feat_idx].compute()
+ row_sum = raw.sum(axis=1)
+ row_sum[row_sum == 0] = 1
+ out = rna.sf * raw / row_sum[:, np.newaxis]
+ if log_transform:
+ out = np.log1p(out)
+ return out.astype(np.float32)
+
+
+def test_write_renorm_subset_matches_reference(toy_crdir_ds):
+ rna = toy_crdir_ds.RNA
+ cell_idx, feat_idx = _subset_indices(rna)
+ loc = "fused_normed"
+
+ rna.z.create_group(loc, overwrite=True)
+ write_renorm_subset_to_zarr(
+ rna, cell_idx, feat_idx, rna.z, f"{loc}/data", rna.nthreads
+ )
+ expected = _reference_renorm_subset(rna, cell_idx, feat_idx)
+ written = rna.z[f"{loc}/data"][:]
+ np.testing.assert_allclose(written, expected, rtol=1e-5)
+
+
+def test_write_renorm_subset_log_transform(toy_crdir_ds):
+ rna = toy_crdir_ds.RNA
+ cell_idx, feat_idx = _subset_indices(rna)
+ loc = "fused_normed_log"
+
+ rna.z.create_group(loc, overwrite=True)
+ write_renorm_subset_to_zarr(
+ rna,
+ cell_idx,
+ feat_idx,
+ rna.z,
+ f"{loc}/data",
+ rna.nthreads,
+ log_transform=True,
+ )
+ expected = _reference_renorm_subset(rna, cell_idx, feat_idx, log_transform=True)
+ written = rna.z[f"{loc}/data"][:]
+ np.testing.assert_allclose(written, expected, rtol=1e-5)
+
+
+def test_save_normalized_data_renorm_uses_fused_path(toy_crdir_ds, monkeypatch):
+ rna = toy_crdir_ds.RNA
+ called = {"normed": 0}
+ orig_normed = RNAassay.normed
+
+ def fake_normed(self, *args, **kwargs):
+ called["normed"] += 1
+ return orig_normed(self, *args, **kwargs)
+
+ monkeypatch.setattr(RNAassay, "normed", fake_normed)
+ monkeypatch.setattr(
+ rna,
+ "_get_cell_feat_idx",
+ lambda cell_key, feat_key: _subset_indices(rna),
+ )
+
+ rna.save_normalized_data(
+ cell_key="I",
+ feat_key="I",
+ batch_size=256,
+ location="normed_fused_test",
+ log_transform=False,
+ renormalize_subset=True,
+ update_keys=False,
+ )
+ assert called["normed"] == 0
+ cell_idx, feat_idx = _subset_indices(rna)
+ expected = _reference_renorm_subset(rna, cell_idx, feat_idx)
+ np.testing.assert_allclose(rna.z["normed_fused_test/data"][:], expected, rtol=1e-5)
+
+
+def test_save_normalized_data_without_renorm_still_uses_normed(
+ toy_crdir_ds, monkeypatch
+):
+ rna = toy_crdir_ds.RNA
+ called = {"normed": 0, "fused": 0}
+ orig_normed = RNAassay.normed
+
+ def fake_normed(self, *args, **kwargs):
+ called["normed"] += 1
+ return orig_normed(self, *args, **kwargs)
+
+ def fake_fused(*args, **kwargs):
+ called["fused"] += 1
+ return write_renorm_subset_to_zarr(*args, **kwargs)
+
+ monkeypatch.setattr(RNAassay, "normed", fake_normed)
+ monkeypatch.setattr("scarf.writers.write_renorm_subset_to_zarr", fake_fused)
+ monkeypatch.setattr(
+ rna,
+ "_get_cell_feat_idx",
+ lambda cell_key, feat_key: _subset_indices(rna),
+ )
+
+ rna.save_normalized_data(
+ cell_key="I",
+ feat_key="I",
+ batch_size=256,
+ location="normed_standard_test",
+ log_transform=False,
+ renormalize_subset=False,
+ update_keys=False,
+ )
+ assert called["normed"] == 1
+ assert called["fused"] == 0
+
+
+def test_save_normalized_data_renorm_cache_hit(toy_crdir_ds, monkeypatch):
+ rna = toy_crdir_ds.RNA
+ called = {"fused": 0}
+ orig_fused = write_renorm_subset_to_zarr
+
+ def counting_fused(*args, **kwargs):
+ called["fused"] += 1
+ return orig_fused(*args, **kwargs)
+
+ monkeypatch.setattr("scarf.writers.write_renorm_subset_to_zarr", counting_fused)
+ monkeypatch.setattr(
+ rna,
+ "_get_cell_feat_idx",
+ lambda cell_key, feat_key: _subset_indices(rna),
+ )
+
+ kwargs = dict(
+ cell_key="I",
+ feat_key="I",
+ batch_size=256,
+ location="normed_cache_test",
+ log_transform=False,
+ renormalize_subset=True,
+ update_keys=False,
+ )
+ rna.save_normalized_data(**kwargs)
+ rna.save_normalized_data(**kwargs)
+ assert called["fused"] == 1
diff --git a/tests/test_repack_zarr.py b/tests/test_repack_zarr.py
new file mode 100644
index 00000000..87879c1b
--- /dev/null
+++ b/tests/test_repack_zarr.py
@@ -0,0 +1,34 @@
+import zarr
+
+from scarf.tools.repack_zarr import repack_store
+
+
+def test_repack_store_round_trip(toy_crdir_writer, tmp_path):
+ output = tmp_path / "repacked.zarr"
+ repack_store(toy_crdir_writer, str(output), profile="fast_local", shard_counts=True)
+
+ src = zarr.open_group(toy_crdir_writer, mode="r")
+ dst = zarr.open_group(str(output), mode="r")
+
+ assert set(src.keys()) == set(dst.keys())
+ assay_names = [name for name in src.keys() if src[name].attrs.get("is_assay")]
+ for assay_name in assay_names:
+ src_assay = src[assay_name]
+ dst_assay = dst[assay_name]
+ assert src_assay.attrs.get("is_assay") is True
+ assert "counts" in dst_assay
+ assert src_assay["counts"].shape == dst_assay["counts"].shape
+ assert (src_assay["counts"][...] == dst_assay["counts"][...]).all()
+
+
+def test_repack_store_without_sharding(toy_crdir_writer, tmp_path):
+ output = tmp_path / "repacked_plain.zarr"
+ repack_store(
+ toy_crdir_writer,
+ str(output),
+ profile="fast_local",
+ shard_counts=False,
+ )
+ dst = zarr.open_group(str(output), mode="r")
+ assert "RNA" in dst
+ assert "counts" in dst["RNA"]
diff --git a/tests/test_symphony_mapping.py b/tests/test_symphony_mapping.py
new file mode 100644
index 00000000..4edf1d2a
--- /dev/null
+++ b/tests/test_symphony_mapping.py
@@ -0,0 +1,349 @@
+import json
+from dataclasses import replace
+from pathlib import Path
+
+import numpy as np
+import pandas as pd
+
+from scarf.harmony import fit_harmony
+from scarf.symphony import (
+ SymphonyReferenceModel,
+ accumulate_sufficient_statistics,
+ apply_query_correction,
+ initialize_sufficient_statistics,
+ project_pca,
+ soft_cluster_assignments,
+ solve_query_correction,
+ weighted_centroids,
+ zero_norm_rows,
+)
+
+
+def _single_cluster_reference() -> SymphonyReferenceModel:
+ return SymphonyReferenceModel(
+ feature_means=np.zeros(2),
+ feature_scales=np.ones(2),
+ loadings=np.eye(2),
+ centroids=np.array([[1.0, 0.0]]),
+ raw_centroids=np.zeros((1, 2)),
+ corrected_centroids=np.zeros((1, 2)),
+ cluster_mass=np.array([4.0]),
+ sigma=np.array([0.1]),
+ correction_ridge=0.0,
+ )
+
+
+def _correct(
+ model: SymphonyReferenceModel, query: np.ndarray, batch_codes: np.ndarray
+) -> np.ndarray:
+ projected = project_pca(query, model)
+ assignments = soft_cluster_assignments(projected, model)
+ counts, sums = initialize_sufficient_statistics(batch_codes.max() + 1, model)
+ accumulate_sufficient_statistics(counts, sums, projected, assignments, batch_codes)
+ correction = solve_query_correction(counts, sums, model)
+ return apply_query_correction(
+ projected, assignments, batch_codes, model, correction
+ )
+
+
+def test_symphony_closed_form_affine_shift_fixture():
+ model = _single_cluster_reference()
+ # Hand-derived single-cluster case: the query batch offset is [3, -2].
+ unshifted = np.array([[-1.0, 0.0], [1.0, 0.0]])
+ shifted = unshifted + np.array([3.0, -2.0])
+
+ corrected = _correct(model, shifted, np.zeros(2, dtype=np.int64))
+
+ np.testing.assert_allclose(corrected, unshifted, atol=1e-12)
+
+
+def test_symphony_no_shift_control_is_identity_for_centered_query():
+ model = _single_cluster_reference()
+ query = np.array([[-2.0, 1.0], [2.0, -1.0]])
+
+ corrected = _correct(model, query, np.zeros(2, dtype=np.int64))
+
+ np.testing.assert_allclose(corrected, query, atol=1e-12)
+
+
+def test_symphony_handles_empty_cluster_batch_statistics_without_nan():
+ model = _single_cluster_reference()
+ counts, sums = initialize_sufficient_statistics(2, model)
+ counts[0, 0] = 2.0
+ sums[0, 0] = np.array([4.0, -2.0])
+
+ correction = solve_query_correction(counts, sums, model)
+
+ assert np.all(np.isfinite(correction.batch_offsets))
+ np.testing.assert_array_equal(correction.batch_offsets[1], np.zeros((1, 2)))
+
+
+def test_symphony_no_shift_composition_subset_stays_stable():
+ model = SymphonyReferenceModel(
+ feature_means=np.zeros(2),
+ feature_scales=np.ones(2),
+ loadings=np.eye(2),
+ centroids=np.array([[-2.0, 0.0], [2.0, 0.0]]),
+ raw_centroids=np.array([[-2.0, 0.0], [2.0, 0.0]]),
+ corrected_centroids=np.array([[-2.0, 0.0], [2.0, 0.0]]),
+ cluster_mass=np.array([100.0, 100.0]),
+ sigma=np.array([0.1, 0.1]),
+ correction_ridge=0.0,
+ )
+ cluster_zero_only = np.array([[-3.0, 0.0], [-1.0, 0.0]])
+
+ corrected = _correct(model, cluster_zero_only, np.zeros(2, dtype=np.int64))
+
+ np.testing.assert_allclose(corrected, cluster_zero_only, atol=1e-12)
+
+
+def test_symphony_composition_imbalance_preserves_distinct_populations():
+ model = SymphonyReferenceModel(
+ feature_means=np.zeros(2),
+ feature_scales=np.ones(2),
+ loadings=np.eye(2),
+ centroids=np.array([[-1.0, 0.0], [1.0, 0.0]]),
+ raw_centroids=np.array([[-2.0, 0.0], [2.0, 0.0]]),
+ corrected_centroids=np.array([[-2.0, 0.0], [2.0, 0.0]]),
+ cluster_mass=np.array([100.0, 100.0]),
+ sigma=np.array([0.01, 0.01]),
+ correction_ridge=0.0,
+ )
+ query = np.array(
+ [
+ [-3.0, 0.0],
+ [-2.5, 0.0],
+ [-1.5, 0.0],
+ [-1.0, 0.0],
+ [1.5, 0.0],
+ [2.5, 0.0],
+ ]
+ )
+ populations = np.array([0, 0, 0, 0, 1, 1])
+
+ corrected = _correct(model, query, np.zeros(len(query), dtype=np.int64))
+ separation = corrected[populations == 1].mean(axis=0) - corrected[
+ populations == 0
+ ].mean(axis=0)
+
+ np.testing.assert_allclose(corrected, query, atol=1e-12)
+ assert separation[0] == 4.0
+
+
+def test_symphony_recovers_independent_query_batch_shifts():
+ model = _single_cluster_reference()
+ centered = np.array([[-1.0, 0.0], [1.0, 0.0], [-2.0, 1.0], [2.0, -1.0]])
+ batches = np.array([0, 0, 1, 1], dtype=np.int64)
+ shifted = centered.copy()
+ shifted[batches == 0] += np.array([3.0, -2.0])
+ shifted[batches == 1] += np.array([-4.0, 5.0])
+
+ corrected = _correct(model, shifted, batches)
+
+ np.testing.assert_allclose(corrected, centered, atol=1e-12)
+
+
+def test_symphony_shift_fixture_improves_neighbor_label_recovery():
+ model = _single_cluster_reference()
+ reference = np.array([[-2.0, 0.0], [-1.0, 0.0], [1.0, 0.0], [2.0, 0.0]])
+ labels = np.array(["left", "left", "right", "right"])
+ shifted = reference + np.array([8.0, -3.0])
+ corrected = _correct(model, shifted, np.zeros(4, dtype=np.int64))
+
+ uncorrected_neighbors = np.argmin(
+ np.linalg.norm(shifted[:, np.newaxis] - reference, axis=2),
+ axis=1,
+ )
+ corrected_neighbors = np.argmin(
+ np.linalg.norm(corrected[:, np.newaxis] - reference, axis=2),
+ axis=1,
+ )
+ uncorrected_accuracy = np.mean(labels[uncorrected_neighbors] == labels)
+ corrected_accuracy = np.mean(labels[corrected_neighbors] == labels)
+
+ assert corrected_accuracy == 1.0
+ assert corrected_accuracy > uncorrected_accuracy
+
+
+def test_symphony_correction_is_row_order_invariant():
+ model = _single_cluster_reference()
+ query = np.array([[2.0, -1.0], [4.0, -1.0], [-3.0, 2.0], [-1.0, 2.0]])
+ batches = np.array([0, 0, 1, 1], dtype=np.int64)
+ order = np.array([2, 0, 3, 1])
+
+ expected = _correct(model, query, batches)
+ reordered = _correct(model, query[order], batches[order])
+ restored = np.empty_like(reordered)
+ restored[order] = reordered
+
+ np.testing.assert_allclose(restored, expected, atol=1e-12)
+
+
+def test_symphony_sufficient_statistics_are_chunk_invariant():
+ model = _single_cluster_reference()
+ query = np.array([[2.0, -1.0], [4.0, -1.0], [-3.0, 2.0], [-1.0, 2.0]])
+ batches = np.array([0, 0, 1, 1], dtype=np.int64)
+ coordinates = project_pca(query, model)
+ assignments = soft_cluster_assignments(coordinates, model)
+
+ expected_counts, expected_sums = initialize_sufficient_statistics(2, model)
+ accumulate_sufficient_statistics(
+ expected_counts,
+ expected_sums,
+ coordinates,
+ assignments,
+ batches,
+ )
+ chunked_counts, chunked_sums = initialize_sufficient_statistics(2, model)
+ for start, stop in ((0, 1), (1, 3), (3, 4)):
+ accumulate_sufficient_statistics(
+ chunked_counts,
+ chunked_sums,
+ coordinates[start:stop],
+ assignments[start:stop],
+ batches[start:stop],
+ )
+
+ np.testing.assert_allclose(chunked_counts, expected_counts, atol=1e-12)
+ np.testing.assert_allclose(chunked_sums, expected_sums, atol=1e-12)
+
+
+def test_symphony_ridge_shrinks_sparse_query_offsets():
+ unregularized = _single_cluster_reference()
+ regularized = SymphonyReferenceModel(
+ feature_means=unregularized.feature_means,
+ feature_scales=unregularized.feature_scales,
+ loadings=unregularized.loadings,
+ centroids=unregularized.centroids,
+ raw_centroids=unregularized.raw_centroids,
+ corrected_centroids=unregularized.corrected_centroids,
+ cluster_mass=unregularized.cluster_mass,
+ sigma=unregularized.sigma,
+ correction_ridge=10.0,
+ )
+ counts = np.array([[1.0]])
+ sums = np.array([[[5.0, 0.0]]])
+
+ raw = solve_query_correction(counts, sums, unregularized)
+ shrunk = solve_query_correction(counts, sums, regularized)
+
+ assert abs(shrunk.batch_offsets[0, 0, 0]) < abs(raw.batch_offsets[0, 0, 0])
+
+
+def test_zero_norm_rows_are_identified_without_nonfinite_assignments():
+ model = _single_cluster_reference()
+ coordinates = np.array([[0.0, 0.0], [1.0, 0.0]])
+
+ mask = zero_norm_rows(coordinates)
+ assignments = soft_cluster_assignments(coordinates, model)
+
+ assert mask.tolist() == [True, False]
+ assert np.all(np.isfinite(assignments))
+
+
+def test_harmony_accepts_numpy_scalar_sigma():
+ embedding = np.array([[-1.0, -0.8, 0.8, 1.0], [0.0, 0.2, -0.2, 0.0]])
+ metadata = pd.DataFrame({"batch": ["a", "a", "b", "b"]})
+
+ result = fit_harmony(
+ embedding,
+ metadata,
+ sigma=np.float64(0.1),
+ theta=np.float64(2.0),
+ lamb=np.int64(3),
+ nclust=2,
+ max_iter_harmony=2,
+ max_iter_kmeans=2,
+ )
+
+ assert result.sigma.shape == (2,)
+ assert result.parameters["sigma"] == [0.1, 0.1]
+ assert result.parameters["theta"] == [2.0, 2.0]
+ assert result.parameters["lambda"] == [3.0, 3.0]
+ assert result.parameters["clusterBackend"] == "kmeans"
+
+
+def test_weighted_centroids_reject_empty_reference_clusters():
+ coordinates = np.array([[0.0, 0.0], [1.0, 1.0]])
+ assignments = np.array([[1.0, 1.0], [0.0, 0.0]])
+
+ with np.testing.assert_raises_regex(ValueError, "empty cluster"):
+ weighted_centroids(coordinates, assignments)
+ with np.testing.assert_raises_regex(ValueError, "masses must be positive"):
+ replace(_single_cluster_reference(), cluster_mass=np.array([0.0]))
+
+
+def test_symphony_r_0_1_3_static_golden_fixture():
+ fixture = json.loads(
+ (Path(__file__).parent / "symphony_r_0_1_3_golden.json").read_text()
+ )
+ reference = fixture["reference"]
+ model = SymphonyReferenceModel(
+ feature_means=np.asarray(reference["featureMeans"]),
+ feature_scales=np.asarray(reference["featureScales"]),
+ loadings=np.asarray(reference["loadings"]),
+ centroids=np.asarray(reference["centroids"]),
+ raw_centroids=np.asarray(reference["rawCentroids"]),
+ corrected_centroids=np.asarray(reference["correctedCentroids"]),
+ cluster_mass=np.asarray(reference["clusterMass"]),
+ sigma=np.asarray(reference["sigma"]),
+ correction_ridge=float(reference["correctionRidge"]),
+ )
+ query = np.asarray(fixture["query"])
+ batch_codes = np.asarray(fixture["batchCodes"], dtype=np.int64)
+ projected = project_pca(query, model)
+ assignments = soft_cluster_assignments(projected, model)
+ corrected = _correct(model, query, batch_codes)
+
+ assert fixture["provenance"]["packageVersion"] == "0.1.3"
+ np.testing.assert_allclose(
+ projected,
+ fixture["expectedProjected"],
+ rtol=0,
+ atol=1e-15,
+ )
+ np.testing.assert_allclose(
+ assignments,
+ fixture["expectedAssignments"],
+ rtol=1e-12,
+ atol=1e-18,
+ )
+ np.testing.assert_allclose(
+ corrected,
+ fixture["expectedCorrected"],
+ rtol=0,
+ atol=1e-12,
+ )
+
+ nonzero = fixture["nonzeroCorrection"]
+ nonzero_reference = nonzero["reference"]
+ nonzero_model = SymphonyReferenceModel(
+ feature_means=np.asarray(nonzero_reference["featureMeans"]),
+ feature_scales=np.asarray(nonzero_reference["featureScales"]),
+ loadings=np.asarray(nonzero_reference["loadings"]),
+ centroids=np.asarray(nonzero_reference["centroids"]),
+ raw_centroids=np.asarray(nonzero_reference["rawCentroids"]),
+ corrected_centroids=np.asarray(nonzero_reference["correctedCentroids"]),
+ cluster_mass=np.asarray(nonzero_reference["clusterMass"]),
+ sigma=np.asarray(nonzero_reference["sigma"]),
+ correction_ridge=float(nonzero_reference["correctionRidge"]),
+ )
+ nonzero_query = np.asarray(nonzero["query"])
+ nonzero_batches = np.asarray(nonzero["batchCodes"], dtype=np.int64)
+ nonzero_projected = project_pca(nonzero_query, nonzero_model)
+ nonzero_assignments = soft_cluster_assignments(nonzero_projected, nonzero_model)
+ nonzero_corrected = _correct(nonzero_model, nonzero_query, nonzero_batches)
+
+ np.testing.assert_allclose(
+ nonzero_assignments,
+ nonzero["expectedAssignments"],
+ rtol=0,
+ atol=1e-15,
+ )
+ np.testing.assert_allclose(
+ nonzero_corrected,
+ nonzero["expectedCorrected"],
+ rtol=0,
+ atol=1e-12,
+ )
+ assert not np.allclose(nonzero_corrected, nonzero_query)
diff --git a/tests/test_types_and_utils.py b/tests/test_types_and_utils.py
new file mode 100644
index 00000000..56458c1e
--- /dev/null
+++ b/tests/test_types_and_utils.py
@@ -0,0 +1,116 @@
+import numpy as np
+import pytest
+import zarr
+from zarr.storage import MemoryStore
+
+from scarf._types import as_zarr_array, as_zarr_group, array_metadata_shards
+from scarf.utils import (
+ array_digest,
+ clean_array,
+ permute_into_chunks,
+ rescale_array,
+ rolling_window,
+ set_verbosity,
+)
+
+
+def test_as_zarr_array_accepts_array():
+ root = zarr.open_group(store=MemoryStore(), mode="w")
+ arr = root.create_array("data", shape=(3,), dtype="f8")
+ assert as_zarr_array(arr) is arr
+
+
+def test_as_zarr_array_rejects_group():
+ root = zarr.open_group(store=MemoryStore(), mode="w")
+ group = root.create_group("nested")
+ with pytest.raises(TypeError, match="Expected Zarr array"):
+ as_zarr_array(group, name="nested")
+
+
+def test_as_zarr_group_accepts_group():
+ root = zarr.open_group(store=MemoryStore(), mode="w")
+ group = root.create_group("nested")
+ assert as_zarr_group(group) is group
+
+
+def test_as_zarr_group_rejects_array():
+ root = zarr.open_group(store=MemoryStore(), mode="w")
+ arr = root.create_array("data", shape=(2,), dtype="f8")
+ with pytest.raises(TypeError, match="Expected Zarr group"):
+ as_zarr_group(arr, name="data")
+
+
+def test_array_metadata_shards_returns_none_without_sharding():
+ root = zarr.open_group(store=MemoryStore(), mode="w")
+ arr = root.create_array("data", shape=(4,), dtype="f8")
+ assert array_metadata_shards(arr) is None
+
+
+def test_clean_array_replaces_nan_inf_and_zero():
+ raw = np.array([1.0, np.nan, np.inf, -np.inf, 0.0])
+ cleaned = clean_array(raw, fill_val=-1.0)
+ assert cleaned[0] == 1.0
+ assert cleaned[-1] == -1.0
+ assert np.all(np.isfinite(cleaned))
+
+
+def test_rescale_array_trims_extreme_values():
+ values = np.concatenate([np.linspace(-2, 2, 499), np.array([100.0])])
+ trimmed = rescale_array(values, frac=0.9)
+ assert trimmed.max() < 100.0
+ assert trimmed.min() > -100.0
+
+
+def test_set_verbosity_rejects_invalid_level():
+ with pytest.raises(ValueError, match="Please provide a value for level"):
+ set_verbosity("NOT_A_REAL_LEVEL")
+
+
+def test_set_verbosity_accepts_valid_level():
+ set_verbosity("ERROR")
+ set_verbosity("INFO")
+
+
+def test_rolling_window_smoothes_along_rows():
+ data = np.arange(10, dtype=float).reshape(5, 2)
+ smoothed = rolling_window(data, w=3)
+ expected = np.array(
+ [
+ [1.0, 2.0],
+ [2.0, 3.0],
+ [4.0, 5.0],
+ [6.0, 7.0],
+ [7.0, 8.0],
+ ]
+ )
+ assert np.array_equal(smoothed, expected)
+
+
+def test_rolling_window_even_and_oversized_windows():
+ data = np.arange(5, dtype=float).reshape(-1, 1)
+
+ assert np.array_equal(rolling_window(np.array([[7.0]]), w=1), [[7.0]])
+ assert np.array_equal(
+ rolling_window(data, w=2).ravel(),
+ [0.5, 1.5, 2.5, 3.5, 4.0],
+ )
+ assert np.array_equal(
+ rolling_window(data, w=20).ravel(),
+ [1.0, 1.5, 2.0, 2.5, 3.0],
+ )
+ for window_size in (0, -1):
+ with pytest.raises(ValueError, match="greater than zero"):
+ rolling_window(data, w=window_size)
+
+
+def test_array_digest_is_deterministic_and_shape_sensitive():
+ values = np.arange(6, dtype=np.int64)
+
+ assert array_digest(values) == array_digest(values.copy())
+ assert array_digest(values) != array_digest(values.reshape(2, 3))
+
+
+def test_permute_into_chunks_preserves_all_indices():
+ chunks = permute_into_chunks(10, 3, seed=7)
+ merged = np.concatenate(chunks)
+ assert np.array_equal(np.sort(merged), np.arange(10))
diff --git a/tests/test_umap_and_harmony.py b/tests/test_umap_and_harmony.py
new file mode 100644
index 00000000..a03627a8
--- /dev/null
+++ b/tests/test_umap_and_harmony.py
@@ -0,0 +1,98 @@
+import numpy as np
+import pandas as pd
+import pytest
+from scipy.sparse import coo_matrix
+
+pytest.importorskip("umap")
+
+from scarf.harmony import run_harmony
+from scarf.umap import calc_dens_map_params, fit_transform, fuzzy_simplicial_set
+
+
+def _ring_graph(n: int) -> coo_matrix:
+ rows, cols, data = [], [], []
+ for i in range(n):
+ for j in (i - 1, i + 1):
+ neighbor = j % n
+ if neighbor != i:
+ rows.append(i)
+ cols.append(neighbor)
+ data.append(1.0)
+ return coo_matrix((data, (rows, cols)), shape=(n, n))
+
+
+def test_calc_dens_map_params():
+ graph = _ring_graph(8)
+ dists = np.full((8, 8), 2.0, dtype=np.float32)
+ np.fill_diagonal(dists, 0.0)
+ for i in range(8):
+ dists[i, (i + 1) % 8] = 1.0
+ dists[i, (i - 1) % 8] = 1.0
+ mu_sum, r_term = calc_dens_map_params(graph, dists)
+ assert mu_sum.shape == (8,)
+ assert r_term.shape == (8,)
+ assert np.all(np.isfinite(mu_sum))
+ assert np.all(np.isfinite(mu_sum[mu_sum > 0]))
+
+
+def test_fuzzy_simplicial_set_produces_coo_graph():
+ graph = coo_matrix(([1.0, 0.6], ([0, 1], [1, 2])), shape=(4, 4))
+ merged = fuzzy_simplicial_set(graph, 1.0)
+ assert merged.shape == (4, 4)
+ assert merged.nnz > 0
+
+
+def test_fit_transform_runs_short_embedding():
+ n_cells = 24
+ graph = _ring_graph(n_cells)
+ ini_embed = np.random.default_rng(0).normal(size=(n_cells, 2)).astype(np.float32)
+
+ embedding, a, b = fit_transform(
+ graph,
+ ini_embed,
+ spread=1.0,
+ min_dist=0.5,
+ n_epochs=15,
+ random_seed=42,
+ repulsion_strength=1.0,
+ initial_alpha=1.0,
+ negative_sample_rate=5,
+ densmap_kwds={},
+ parallel=False,
+ nthreads=1,
+ verbose=False,
+ )
+
+ assert embedding.shape == (n_cells, 2)
+ assert np.all(np.isfinite(embedding))
+ assert a > 0
+ assert b > 0
+
+
+def test_run_harmony_corrects_batch_structure():
+ rng = np.random.default_rng(0)
+ n_cells = 180
+ n_dims = 12
+ batch = rng.integers(0, 3, n_cells)
+ batch_effect = rng.normal(scale=2.0, size=(3, n_dims))
+ data = rng.normal(size=(n_dims, n_cells))
+ for cell_idx, batch_id in enumerate(batch):
+ data[:, cell_idx] += batch_effect[batch_id]
+
+ meta = pd.DataFrame({"batch": [f"batch_{x}" for x in batch]})
+ corrected = run_harmony(
+ data,
+ meta,
+ nclust=15,
+ max_iter_harmony=4,
+ max_iter_kmeans=5,
+ random_state=0,
+ )
+
+ assert corrected.shape == data.shape
+ assert np.all(np.isfinite(corrected))
+ batch_means_before = [data[:, batch == b].mean(axis=1) for b in range(3)]
+ batch_means_after = [corrected[:, batch == b].mean(axis=1) for b in range(3)]
+ spread_before = np.std([m.mean() for m in batch_means_before])
+ spread_after = np.std([m.mean() for m in batch_means_after])
+ assert spread_after <= spread_before
diff --git a/tests/test_writers.py b/tests/test_writers.py
new file mode 100644
index 00000000..1e7da1c3
--- /dev/null
+++ b/tests/test_writers.py
@@ -0,0 +1,402 @@
+import numpy as np
+import pytest
+import zarr
+from zarr.storage import MemoryStore
+
+from scarf.readers import CSVReader
+from scarf.writers import (
+ CSVtoZarr,
+ CrToZarr,
+ SubsetZarr,
+ bed_to_sparse_array,
+ subset_assay_zarr,
+)
+
+
+class _FakeCells:
+ def __init__(
+ self, n_cells: int, columns: dict[str, np.ndarray] | None = None
+ ) -> None:
+ self.N = n_cells
+ self._columns = columns or {}
+
+ def fetch_all(self, key: str) -> np.ndarray:
+ return self._columns[key]
+
+
+class _FakeAssay:
+ def __init__(
+ self,
+ name: str,
+ n_cells: int,
+ columns: dict[str, np.ndarray] | None = None,
+ ) -> None:
+ self.name = name
+ self.cells = _FakeCells(n_cells, columns)
+
+
+def test_crtozarr(crh5_reader, tmp_path):
+ from scarf.writers import CrToZarr
+
+ fn = str(tmp_path / "dummy_1K_pbmc_citeseq.zarr")
+ writer = CrToZarr(crh5_reader, zarr_loc=fn)
+ writer.dump()
+
+
+def test_crtozarr_fromdir(crdir_reader, tmp_path):
+ from scarf.writers import CrToZarr
+
+ fn = str(tmp_path / "1K_pbmc_citeseq_dir.zarr")
+ writer = CrToZarr(crdir_reader, zarr_loc=fn)
+ writer.dump()
+
+
+def test_h5adtozarr(h5ad_reader, tmp_path):
+ from scarf.writers import H5adToZarr
+
+ fn = str(tmp_path / "bastidas.zarr")
+ writer = H5adToZarr(h5ad_reader, zarr_loc=fn)
+ writer.dump()
+
+
+def test_loomtozarr(loom_reader, tmp_path):
+ from scarf.writers import LoomToZarr
+
+ fn = str(tmp_path / "sympathetic.zarr")
+ writer = LoomToZarr(loom_reader, zarr_loc=fn)
+ writer.dump()
+
+
+def test_sparsetozarr(tmp_path):
+ from scipy.sparse import csr_matrix
+
+ from scarf.writers import SparseToZarr
+
+ cols = [1, 3, 8, 2, 3, 1, 2, 8, 9]
+ rows = [0, 0, 0, 1, 1, 1, 2, 2, 2]
+ data = [1, 10, 15, 10, 20, 2, 3, 1, 5]
+ mat = (data, (rows, cols))
+ mat = csr_matrix(mat, shape=(3, 10))
+
+ fn = str(tmp_path / "dummy_sparse.zarr")
+
+ writer = SparseToZarr(
+ mat,
+ zarr_loc=fn,
+ cell_ids=[f"cell_{x}" for x in range(3)],
+ feature_ids=[f"feat_{x}" for x in range(10)],
+ )
+ writer.dump()
+
+
+def test_sparsetozarr_sharded_layout(tmp_path):
+ import zarr
+ from scipy.sparse import csr_matrix
+
+ from scarf.writers import SparseToZarr
+
+ n_cells, n_feats = 5000, 200
+ rng = np.random.default_rng(0)
+ rows = rng.integers(0, n_cells, size=50_000)
+ cols = rng.integers(0, n_feats, size=50_000)
+ data = np.ones(50_000, dtype=np.uint32)
+ mat = csr_matrix((data, (rows, cols)), shape=(n_cells, n_feats))
+ fn = str(tmp_path / "dummy_sparse_sharded.zarr")
+ writer = SparseToZarr(
+ mat,
+ zarr_loc=fn,
+ cell_ids=[f"cell_{x}" for x in range(n_cells)],
+ feature_ids=[f"feat_{x}" for x in range(n_feats)],
+ )
+ writer.dump()
+ store = zarr.open_group(fn, mode="r")
+ counts = store["RNA/counts"]
+ assert counts.shape == (n_cells, n_feats)
+ assert counts.metadata.shards is not None
+ assert int(counts[...].sum()) > 0
+
+
+def test_csv_to_zarr_round_trip(tmp_path):
+ csv_path = tmp_path / "counts.csv"
+ csv_path.write_text(
+ "quality,geneA,geneB,geneC\n"
+ "10,1,0,2\n"
+ "20,0,3,0\n"
+ "30,4,5,6\n"
+ "40,7,0,8\n"
+ "50,9,10,0\n",
+ encoding="utf-8",
+ )
+ reader = CSVReader(
+ str(csv_path),
+ cell_data_cols=["quality"],
+ batch_size=2,
+ )
+ store = MemoryStore()
+ writer = CSVtoZarr(
+ reader,
+ zarr_loc=store,
+ assay_name="RNA",
+ dtype=np.dtype("uint16"),
+ )
+
+ writer.dump()
+
+ root = zarr.open_group(store=store, mode="r")
+ expected = np.array(
+ [
+ [1, 0, 2],
+ [0, 3, 0],
+ [4, 5, 6],
+ [7, 0, 8],
+ [9, 10, 0],
+ ],
+ dtype=np.uint16,
+ )
+ np.testing.assert_array_equal(root["RNA/counts"][:], expected)
+ np.testing.assert_array_equal(
+ root["RNA/featureData/ids"][:],
+ np.array(["geneA", "geneB", "geneC"]),
+ )
+ np.testing.assert_array_equal(
+ root["cellData/ids"][:],
+ np.array(["cell_0", "cell_1", "cell_2", "cell_3", "cell_4"]),
+ )
+ np.testing.assert_array_equal(
+ root["cellData/quality"][:],
+ np.array([10, 20, 30, 40, 50]),
+ )
+
+
+def test_subset_assay_zarr_selects_ordered_rows_and_columns():
+ store = MemoryStore()
+ root = zarr.open_group(store=store, mode="w")
+ source = root.create_array(
+ "source",
+ shape=(5, 4),
+ chunks=(2, 2),
+ dtype=np.uint16,
+ fill_value=0,
+ )
+ values = np.arange(20, dtype=np.uint16).reshape(5, 4)
+ source[:] = values
+ cells = np.array([4, 1, 3])
+ features = np.array([3, 0])
+
+ result = subset_assay_zarr(
+ store,
+ in_grp="source",
+ out_grp="selected",
+ cells_idx=cells,
+ feat_idx=features,
+ chunk_size=(2, 2),
+ )
+
+ selected = root["selected"]
+ assert result is None
+ assert selected.dtype == np.dtype(np.uint32)
+ np.testing.assert_array_equal(
+ selected[:],
+ values[np.ix_(cells, features)],
+ )
+
+
+def test_bed_to_sparse_array_bins_filters_and_drops_unknown_features(tmp_path):
+ bed_path = tmp_path / "fragments.bed"
+ bed_path.write_text(
+ "# test fragments\n"
+ "chr1\t0\t20\tcellB\t2\n"
+ "chr1\t100\t120\tcellA\t3\n"
+ "chr2\t0\t20\tcellB\t4\n"
+ "chr1\t0\t10\tcellC\t5\n"
+ "chr1\t10\t20\tcellA\t1\n"
+ "chr2\t200\t220\tcellD\t1\n",
+ encoding="utf-8",
+ )
+
+ matrix, cell_ids, feature_ids = bed_to_sparse_array(
+ str(bed_path),
+ bin_size=100,
+ chrom_sizes={"chr1": 199, "chr2": 99},
+ min_counts_per_cell=3,
+ read_chunk_size=2,
+ disable_tqdm=True,
+ )
+
+ assert cell_ids.tolist() == ["cellB", "cellA", "cellC"]
+ assert feature_ids.tolist() == ["chr1_0", "chr1_1", "chr2_0"]
+ np.testing.assert_array_equal(
+ matrix.toarray(),
+ np.array(
+ [
+ [2, 0, 4],
+ [1, 3, 0],
+ [5, 0, 0],
+ ]
+ ),
+ )
+
+
+def test_v2_fixture_read_only(datastore):
+ assert datastore.RNA.rawData.shape[0] > 0
+
+
+def test_to_h5ad(datastore, tmp_path):
+ from scarf.writers import to_h5ad
+
+ fn = str(tmp_path / "test_1K_pbmc_citeseq.h5ad")
+ to_h5ad(datastore.RNA, fn)
+
+
+def test_to_mtx(datastore, tmp_path):
+ from scarf.writers import to_mtx
+
+ fn = str(tmp_path / "test_1K_pbmc_citeseq_dir")
+ to_mtx(datastore.RNA, fn)
+
+
+def test_zarr_subset(datastore, tmp_path):
+ zarr_path = str(tmp_path / "subset.zarr")
+ writer = SubsetZarr(
+ zarr_loc=zarr_path, assays=[datastore.RNA], cell_idx=np.array([1, 10, 100, 500])
+ )
+ writer.dump()
+
+
+def test_subset_zarr_rejects_invalid_assay_inputs():
+ with pytest.raises(TypeError, match="should be a list"):
+ SubsetZarr._check_assays("RNA")
+ with pytest.raises(ValueError, match="actual assay objects"):
+ SubsetZarr._check_assays([object()])
+ with pytest.raises(ValueError, match="same numer of cells"):
+ SubsetZarr._check_assays(
+ [
+ _FakeAssay("RNA", 3),
+ _FakeAssay("ATAC", 4),
+ ]
+ )
+
+
+def test_subset_zarr_requires_cell_key_or_indices():
+ subset = object.__new__(SubsetZarr)
+ subset.assays = [_FakeAssay("RNA", 3)]
+
+ with pytest.raises(ValueError, match="cannot be None"):
+ subset._check_idx(None, None)
+
+
+@pytest.mark.parametrize(
+ ("cell_idx", "message"),
+ [
+ (np.array([0.5]), "integer type"),
+ (np.array([3]), "max value"),
+ ],
+)
+def test_subset_zarr_rejects_invalid_explicit_indices(cell_idx, message):
+ subset = object.__new__(SubsetZarr)
+ subset.assays = [_FakeAssay("RNA", 3)]
+
+ with pytest.raises(ValueError, match=message):
+ subset._check_idx(None, cell_idx)
+
+
+def test_subset_zarr_validates_cell_key():
+ subset = object.__new__(SubsetZarr)
+ subset.assays = [_FakeAssay("RNA", 3)]
+ with pytest.raises(ValueError, match="was not found"):
+ subset._check_idx("selected", None)
+
+ subset.assays = [
+ _FakeAssay("RNA", 3, {"selected": np.array([1, 0, 1])}),
+ ]
+ with pytest.raises(ValueError, match="not of boolean type"):
+ subset._check_idx("selected", None)
+
+
+def test_subset_zarr_resolves_consistent_cell_key():
+ selected = np.array([True, False, True, False])
+ subset = object.__new__(SubsetZarr)
+ subset.assays = [
+ _FakeAssay("RNA", 4, {"selected": selected}),
+ _FakeAssay("ATAC", 4, {"selected": selected.copy()}),
+ ]
+
+ np.testing.assert_array_equal(
+ subset._check_idx("selected", None),
+ np.array([0, 2]),
+ )
+
+
+def test_subset_zarr_local_path_guard(tmp_path):
+ existing = tmp_path / "out.zarr"
+ existing.mkdir()
+ subset = object.__new__(SubsetZarr)
+ subset.overFn = False
+ subset.storage_options = None
+ with pytest.raises(ValueError, match="already exists"):
+ SubsetZarr._check_files(subset, str(existing))
+
+
+def test_subset_zarr_store_skips_local_guard():
+ mem = MemoryStore()
+ subset = object.__new__(SubsetZarr)
+ subset.overFn = False
+ subset.storage_options = None
+ root = SubsetZarr._check_files(subset, mem)
+ assert isinstance(root, zarr.Group)
+
+
+def test_subset_zarr_remote_uri_skips_local_guard(monkeypatch):
+ calls = []
+
+ def fake_load_zarr(zarr_loc, mode, storage_options=None):
+ calls.append((zarr_loc, mode, storage_options))
+ return zarr.open_group(store=MemoryStore(), mode="w")
+
+ monkeypatch.setattr("scarf.writers.load_zarr", fake_load_zarr)
+ subset = object.__new__(SubsetZarr)
+ subset.overFn = False
+ subset.storage_options = {"access_key_id": "key"}
+ SubsetZarr._check_files(subset, "s3://bucket/out.zarr")
+ assert calls == [("s3://bucket/out.zarr", "w", {"access_key_id": "key"})]
+
+
+def test_crtozarr_forwards_storage_options(monkeypatch):
+ captured = {}
+
+ def fake_load_zarr(zarr_loc, mode, storage_options=None, synchronizer=None):
+ captured["zarr_loc"] = zarr_loc
+ captured["mode"] = mode
+ captured["storage_options"] = storage_options
+ return zarr.open_group(store=MemoryStore(), mode="w")
+
+ monkeypatch.setattr("scarf.writers.load_zarr", fake_load_zarr)
+ monkeypatch.setattr("scarf.writers.create_cell_data", lambda **kwargs: None)
+ monkeypatch.setattr("scarf.writers.create_zarr_count_assay", lambda **kwargs: None)
+
+ class FakeCr:
+ def cell_names(self):
+ return ["c1"]
+
+ @property
+ def assayFeats(self):
+ import pandas as pd
+
+ return pd.DataFrame({"RNA": [0, 1]})
+
+ @property
+ def nCells(self):
+ return 1
+
+ def feature_ids(self, assay_name):
+ return ["f1"]
+
+ def feature_names(self, assay_name):
+ return ["f1"]
+
+ CrToZarr(
+ FakeCr(),
+ zarr_loc="s3://bucket/out.zarr",
+ storage_options={"access_key_id": "id"},
+ )
+ assert captured["storage_options"] == {"access_key_id": "id"}
diff --git a/tests/test_zarr_store.py b/tests/test_zarr_store.py
new file mode 100644
index 00000000..8dba5c40
--- /dev/null
+++ b/tests/test_zarr_store.py
@@ -0,0 +1,599 @@
+import types
+
+import numpy as np
+import pytest
+import zarr
+from zarr.storage import MemoryStore
+
+from scarf.storage.zarr_store import (
+ accumulate_sparse_to_shards,
+ copy_zarr_array,
+ copy_zarr_group_tree,
+ create_numeric_array,
+ finalize_sharded_counts,
+ get_storage_profile,
+ is_local_zarr_path,
+ is_remote_datastore,
+ is_remote_zarr_location,
+ make_store,
+ normed_array_spec,
+ open_or_create_staged_normed_array,
+ open_store,
+ set_storage_profile,
+ write_dense_from_row_batches,
+)
+from scarf._types import array_metadata_shards
+from scarf.utils import load_zarr
+
+
+@pytest.fixture(autouse=True)
+def reset_profile():
+ set_storage_profile(None)
+ yield
+ set_storage_profile(None)
+
+
+def test_is_remote_zarr_location():
+ assert is_remote_zarr_location("s3://bucket/path") is True
+ assert is_remote_zarr_location("gs://bucket/path") is True
+ assert is_remote_zarr_location("/tmp/foo.zarr") is False
+ assert is_remote_zarr_location("file:///tmp/foo.zarr") is False
+
+
+def test_is_local_zarr_path():
+ assert is_local_zarr_path("/tmp/foo.zarr") is True
+ assert is_local_zarr_path("s3://bucket/path") is False
+ assert is_local_zarr_path("gs://bucket/path") is False
+ assert is_local_zarr_path(MemoryStore()) is False
+
+
+def test_load_zarr_forwards_storage_options_to_make_store(monkeypatch):
+ captured = {}
+
+ def fake_make_store(location, storage_options=None, read_only=False):
+ captured["location"] = location
+ captured["storage_options"] = storage_options
+ captured["read_only"] = read_only
+ return MemoryStore()
+
+ monkeypatch.setattr("scarf.storage.zarr_store.make_store", fake_make_store)
+ monkeypatch.setattr(
+ "scarf.storage.zarr_store.configure_zarr_io_for_profile", lambda: None
+ )
+ monkeypatch.setattr("zarr.open_group", lambda **kwargs: object())
+ load_zarr(
+ "s3://bucket/path",
+ mode="r",
+ storage_options={"secret_access_key": "secret"},
+ )
+ assert captured["storage_options"] == {"secret_access_key": "secret"}
+ assert captured["read_only"] is True
+
+
+def _memory_group():
+ return zarr.open_group(store=MemoryStore(), mode="w")
+
+
+def test_is_remote_datastore():
+ local_root = _memory_group()
+ assert is_remote_datastore("/tmp/foo.zarr", local_root) is False
+ assert is_remote_datastore("s3://bucket/path", local_root) is True
+
+
+def test_copy_zarr_array_round_trip(tmp_path):
+ src_root = zarr.open_group(str(tmp_path / "src.zarr"), mode="w")
+ spec = normed_array_spec(64, 8, profile="fast_local")
+ src = create_numeric_array(src_root, "data", spec)
+ expected = np.random.rand(64, 8).astype(np.float32)
+ src[:] = expected
+
+ dst_root = zarr.open_group(str(tmp_path / "dst.zarr"), mode="w")
+ dst = create_numeric_array(dst_root, "data", spec)
+ copy_zarr_array(src, dst, block_rows=16)
+ np.testing.assert_allclose(dst[:], expected, rtol=1e-6)
+
+
+def test_write_dense_from_row_batches_flushes_at_shard_boundaries():
+ root = _memory_group()
+ dst = root.create_array(
+ "counts",
+ shape=(7, 3),
+ chunks=(2, 3),
+ shards=(4, 3),
+ dtype=np.uint16,
+ fill_value=0,
+ )
+ expected = np.arange(21, dtype=np.int64).reshape(7, 3)
+ writes = []
+ write_dtypes = []
+
+ class RecordingArray:
+ def __init__(self, array):
+ self._array = array
+ self.metadata = array.metadata
+ self.shape = array.shape
+
+ def __setitem__(self, selection, value):
+ row_slice = selection[0]
+ writes.append((row_slice.start, row_slice.stop))
+ write_dtypes.append(value.dtype)
+ self._array[selection] = value
+
+ rows_written = write_dense_from_row_batches(
+ RecordingArray(dst),
+ iter(
+ [
+ expected[:1],
+ expected[1:5],
+ np.empty((0, 3), dtype=np.int64),
+ expected[5:],
+ ]
+ ),
+ dtype=np.uint16,
+ )
+
+ assert rows_written == 7
+ assert writes == [(0, 4), (4, 7)]
+ assert write_dtypes == [np.dtype(np.uint16), np.dtype(np.uint16)]
+ np.testing.assert_array_equal(dst[:], expected.astype(np.uint16))
+
+
+@pytest.mark.parametrize(
+ ("workspace", "counts_group_path"),
+ [(None, "RNA"), ("workspace", "matrices/RNA")],
+)
+def test_finalize_sharded_counts_repacks_and_cleans_up(
+ monkeypatch, workspace, counts_group_path
+):
+ from scarf.storage.budget import ResourceBudget
+
+ root = _memory_group()
+ counts_group = root.create_group(counts_group_path)
+ source = counts_group.create_array(
+ "counts",
+ shape=(5, 3),
+ chunks=(2, 3),
+ dtype=np.uint32,
+ fill_value=0,
+ )
+ expected = np.arange(15, dtype=np.uint32).reshape(5, 3)
+ source[:] = expected
+ assert array_metadata_shards(source) is None
+
+ budget = ResourceBudget(memoryBytes=24, workers=1, workingCopies=1)
+ monkeypatch.setattr("scarf.storage.zarr_store.get_resource_budget", lambda: budget)
+ monkeypatch.setattr("scarf.storage.budget.get_resource_budget", lambda: budget)
+
+ result = finalize_sharded_counts(
+ root,
+ "RNA",
+ workspace=workspace,
+ profile="fast_local",
+ )
+
+ np.testing.assert_array_equal(result[:], expected)
+ assert result.chunks == (2, 1)
+ assert array_metadata_shards(result) == (2, 3)
+ refreshed_group = root[counts_group_path]
+ assert "counts__sharded_tmp" not in refreshed_group
+ assert refreshed_group.attrs["scarf:zarr_spec"] == {
+ "profile": "fast_local",
+ "chunks": [2, 1],
+ "shards": [2, 3],
+ "zarr_format": 3,
+ }
+
+
+def test_accumulate_sparse_to_shards_preserves_offsets_across_zero_runs():
+ from scipy.sparse import coo_matrix
+
+ root = _memory_group()
+ dst = root.create_array(
+ "counts",
+ shape=(36, 3),
+ chunks=(2, 3),
+ dtype=np.uint32,
+ fill_value=0,
+ )
+
+ zero_batch = coo_matrix((1, 3), dtype=np.uint32)
+ batches = [
+ coo_matrix(
+ (np.array([11], dtype=np.uint32), ([0], [0])),
+ shape=(1, 3),
+ ),
+ *[zero_batch for _ in range(32)],
+ coo_matrix(
+ (np.array([22, 33], dtype=np.uint32), ([0, 1], [1, 2])),
+ shape=(2, 3),
+ ),
+ zero_batch,
+ ]
+
+ rows_written = accumulate_sparse_to_shards(dst, iter(batches), shard_rows=2)
+
+ expected = np.zeros((36, 3), dtype=np.uint32)
+ expected[0, 0] = 11
+ expected[33, 1] = 22
+ expected[34, 2] = 33
+ assert rows_written == 36
+ np.testing.assert_array_equal(dst[:], expected)
+
+
+def test_copy_zarr_group_tree(tmp_path):
+ from scarf.writers import create_zarr_obj_array
+
+ src_root = zarr.open_group(str(tmp_path / "src.zarr"), mode="w")
+ slot = src_root.create_group("I__cluster")
+ cluster = slot.create_group("0")
+ create_zarr_obj_array(cluster, "score", [1.0, 2.0, 3.0], dtype="float64")
+
+ dst_root = zarr.open_group(str(tmp_path / "dst.zarr"), mode="w")
+ dst_slot = dst_root.create_group("I__cluster")
+ copy_zarr_group_tree(slot, dst_slot)
+ np.testing.assert_array_equal(dst_slot["0"]["score"][:], [1.0, 2.0, 3.0])
+
+
+def test_open_or_create_staged_normed_array_reuses_shape(tmp_path):
+ src_root = zarr.open_group(str(tmp_path / "src.zarr"), mode="w")
+ spec = normed_array_spec(32, 4, profile="cloud")
+ src = create_numeric_array(src_root, "data", spec)
+ src[:] = np.ones((32, 4), dtype=np.float32)
+
+ cache_path = str(tmp_path / "cache" / "abc123" / "normed.zarr")
+ staged = open_or_create_staged_normed_array(cache_path, src)
+ copy_zarr_array(src, staged, block_rows=8)
+ staged.attrs["staged_subset_hash"] = "abc123"
+ staged.attrs["staged_complete"] = True
+
+ reopened = open_or_create_staged_normed_array(cache_path, src)
+ assert reopened.shape == src.shape
+ np.testing.assert_allclose(reopened[:], np.ones((32, 4), dtype=np.float32))
+
+
+def test_make_store_local_path_returns_str(tmp_path):
+ path = str(tmp_path / "ds.zarr")
+ store = make_store(path)
+ assert store == path
+
+
+def test_make_store_memory_store():
+ mem = MemoryStore()
+ store = make_store(mem)
+ assert store is mem
+
+
+def test_open_store_memory(tmp_path):
+ path = str(tmp_path / "ds.zarr")
+ root = open_store(path, mode="w")
+ root.create_group("g")
+ loaded = open_store(path, mode="r")
+ assert "g" in loaded
+
+
+def test_load_zarr_memory_store():
+ mem = MemoryStore()
+ root = zarr.open_group(store=mem, mode="w")
+ root.create_group("assay")
+ loaded = load_zarr(mem, mode="r")
+ assert "assay" in loaded
+
+
+def test_make_store_remote_requires_obstore(monkeypatch):
+ import builtins
+
+ real_import = builtins.__import__
+
+ def mock_import(name, *args, **kwargs):
+ if name in ("obstore", "obstore.store"):
+ raise ImportError("no obstore")
+ return real_import(name, *args, **kwargs)
+
+ monkeypatch.setattr(builtins, "__import__", mock_import)
+ with pytest.raises(ImportError, match="obstore"):
+ make_store("s3://bucket/path")
+
+
+def test_make_store_remote_auto_cloud_profile(monkeypatch):
+ class FakeObstore:
+ pass
+
+ class FakeObjectStore:
+ def __init__(self, store, read_only=False):
+ self.store = store
+ self.read_only = read_only
+
+ fake_mod = types.ModuleType("obstore.store")
+ fake_mod.from_url = lambda url, **kwargs: FakeObstore()
+ monkeypatch.setitem(__import__("sys").modules, "obstore.store", fake_mod)
+ monkeypatch.setattr(
+ "zarr.storage.ObjectStore",
+ FakeObjectStore,
+ )
+ store = make_store("s3://bucket/path")
+ assert isinstance(store, FakeObjectStore)
+ assert get_storage_profile() == "cloud"
+
+
+def test_explicit_profile_not_overridden_by_remote(monkeypatch):
+ class FakeObstore:
+ pass
+
+ class FakeObjectStore:
+ def __init__(self, store, read_only=False):
+ self.store = store
+
+ fake_mod = types.ModuleType("obstore.store")
+ fake_mod.from_url = lambda url, **kwargs: FakeObstore()
+ monkeypatch.setitem(__import__("sys").modules, "obstore.store", fake_mod)
+ monkeypatch.setattr("zarr.storage.ObjectStore", FakeObjectStore)
+
+ set_storage_profile("fast_local")
+ make_store("s3://bucket/path")
+ assert get_storage_profile() == "fast_local"
+
+
+def test_normed_array_spec_plain_chunks():
+ from scarf.storage.budget import ResourceBudget
+ from scarf.storage.zarr_store import normed_array_spec, set_storage_profile
+
+ set_storage_profile("cloud")
+ budget = ResourceBudget(memoryBytes=8 * 1024**3, workers=4, workingCopies=4)
+ spec = normed_array_spec(1_000_000, 2000, budget=budget)
+ assert spec.dtype == "float32"
+ assert spec.shards is None
+ assert spec.chunks[1] == 2000
+ assert spec.chunks[0] >= 1
+
+
+@pytest.mark.parametrize("n_cells", [1_000_000, 2_500_000, 5_000_000, 10_000_000])
+def test_normed_array_spec_respects_codec_limit(n_cells):
+ from scarf.storage.budget import ResourceBudget
+ from scarf.storage.zarr_store import _CODEC_MAX_BYTES, normed_array_spec
+
+ budget = ResourceBudget(
+ memoryBytes=112 * 1024**3,
+ workers=16,
+ workingCopies=4,
+ )
+ spec = normed_array_spec(
+ n_cells,
+ 2000,
+ profile="cloud",
+ budget=budget,
+ )
+ assert spec.shards is None
+ assert spec.chunks[0] * spec.chunks[1] * 4 <= _CODEC_MAX_BYTES
+
+
+@pytest.mark.parametrize("n_feats", [500, 2000, 3000, 5000, 8192, 30_000])
+def test_normed_array_spec_creates_array(tmp_path, n_feats):
+ from scarf.storage.zarr_store import (
+ create_numeric_array,
+ normed_array_spec,
+ set_storage_profile,
+ )
+
+ set_storage_profile("cloud")
+ spec = normed_array_spec(1_000_000, n_feats)
+ root = zarr.open_group(str(tmp_path / f"normed_{n_feats}.zarr"), mode="w")
+ create_numeric_array(root, "data", spec)
+ assert root["data"].shape == (1_000_000, n_feats)
+ assert spec.shards is None
+
+
+def test_memory_first_layout_worked_example():
+ from scarf.storage.budget import ResourceBudget
+ from scarf.storage.zarr_store import _CODEC_MAX_BYTES, matrix_layout
+
+ budget = ResourceBudget(memoryBytes=8 * 1024**3, workers=8, workingCopies=4)
+ chunks, shards = matrix_layout(1_000_000, 50_000, budget=budget, itemsize=4)
+ assert shards is not None
+ row_shard, shard_cols = shards
+ feature_chunk = chunks[1]
+ work = (8 * 1024**3) // 4
+ assert feature_chunk == work // (1_000_000 * 4)
+ assert shard_cols % feature_chunk == 0
+ assert shard_cols >= 50_000
+ assert row_shard * shard_cols * 4 <= _CODEC_MAX_BYTES
+
+
+def test_ceil_pad_awkward_feature_count():
+ from scarf.storage.budget import ResourceBudget
+ from scarf.storage.zarr_store import matrix_layout
+
+ budget = ResourceBudget(memoryBytes=8 * 1024**3, workers=1, workingCopies=4)
+ chunks, shards = matrix_layout(1_000_000, 36_601, budget=budget, itemsize=4)
+ assert shards is not None
+ feature_chunk = chunks[1]
+ shard_cols = shards[1]
+ assert feature_chunk >= 1
+ assert shard_cols % feature_chunk == 0
+ assert shard_cols >= 36_601
+
+
+def test_float64_halves_row_shard():
+ from scarf.storage.budget import ResourceBudget
+ from scarf.storage.zarr_store import matrix_layout
+
+ budget = ResourceBudget(memoryBytes=8 * 1024**3, workers=1, workingCopies=4)
+ u32, _ = matrix_layout(100_000, 20_000, budget=budget, itemsize=4)
+ f64, _ = matrix_layout(100_000, 20_000, budget=budget, itemsize=8)
+ assert f64[0] <= u32[0]
+
+
+def test_matrix_layout_scales_with_cells():
+ from scarf.storage.budget import ResourceBudget
+ from scarf.storage.zarr_store import matrix_layout
+
+ budget = ResourceBudget(memoryBytes=64 * 1024**3, workers=2, workingCopies=4)
+ small_chunks, small_shards = matrix_layout(1_000, 2_000, budget=budget, itemsize=4)
+ large_chunks, large_shards = matrix_layout(
+ 1_000_000, 50_000, budget=budget, itemsize=4
+ )
+ assert small_shards is not None and large_shards is not None
+ assert large_chunks[0] >= small_chunks[0]
+ assert large_shards[0] >= small_shards[0]
+
+
+def test_matrix_layout_shard_chunk_alignment():
+ from scarf.storage.budget import ResourceBudget
+ from scarf.storage.zarr_store import _CODEC_MAX_BYTES, matrix_layout
+
+ budget = ResourceBudget(memoryBytes=8 * 1024**3, workers=4, workingCopies=4)
+ chunks, shards = matrix_layout(100_000, 50_000, budget=budget, itemsize=4)
+ assert shards is not None
+ row_chunk, col_chunk = chunks
+ shard_rows, shard_cols = shards
+ assert shard_cols % col_chunk == 0
+ assert shard_rows % row_chunk == 0
+ assert shard_cols >= 50_000
+ assert shard_rows * shard_cols * 4 <= _CODEC_MAX_BYTES
+
+
+def test_matrix_layout_respects_codec_limit():
+ from scarf.storage.budget import get_resource_budget
+ from scarf.storage.zarr_store import _CODEC_MAX_BYTES, matrix_layout
+
+ budget = get_resource_budget()
+ for n_cells, n_feats in [
+ (10_000, 89_796),
+ (1_000_000, 50_000),
+ (1_000_000, 36_601),
+ ]:
+ chunks, shards = matrix_layout(n_cells, n_feats, budget=budget, itemsize=4)
+ assert shards is not None
+ row_shard, shard_cols = shards
+ feature_chunk = chunks[1]
+ assert row_shard * shard_cols * 4 <= _CODEC_MAX_BYTES
+ assert shard_cols % feature_chunk == 0
+ assert row_shard % chunks[0] == 0
+
+
+def test_matrix_layout_target_chunk_bytes_clamps_features():
+ from scarf.storage.budget import ResourceBudget
+ from scarf.storage.zarr_store import matrix_layout
+
+ budget = ResourceBudget(memoryBytes=48 * 1024**3, workers=4, workingCopies=4)
+ chunks, shards = matrix_layout(
+ 100_000,
+ 45_525,
+ budget=budget,
+ itemsize=4,
+ targetChunkBytes=256 * 1024 * 1024,
+ minFeatureChunk=500,
+ maxFeatureChunk=10_000,
+ )
+ assert shards is not None
+ assert 500 <= chunks[1] <= 10_000
+ assert chunks[1] <= 45_525
+ assert shards[1] % chunks[1] == 0
+ assert chunks[0] * chunks[1] * 4 <= 256 * 1024 * 1024 + chunks[1] * 4
+
+
+def test_count_array_spec_passes_target_chunk_bytes():
+ from scarf.storage.budget import ResourceBudget, set_resource_budget
+ from scarf.storage.zarr_store import count_array_spec
+
+ budget = ResourceBudget(memoryBytes=24 * 1024**3, workers=4, workingCopies=4)
+ try:
+ set_resource_budget(budget)
+ capped = count_array_spec(
+ 100_000,
+ 45_525,
+ dtype="uint32",
+ remote=True,
+ targetChunkBytes=64 * 1024 * 1024,
+ minFeatureChunk=500,
+ maxFeatureChunk=10_000,
+ )
+ baseline = count_array_spec(100_000, 45_525, dtype="uint32", remote=True)
+ finally:
+ set_resource_budget(None)
+ assert capped.chunks[1] <= 10_000
+ assert capped.chunks[1] < baseline.chunks[1] or baseline.chunks[1] <= 10_000
+
+
+def test_count_array_spec_applies_cloud_default_target_chunk_bytes():
+ from scarf.storage.budget import ResourceBudget, set_resource_budget
+ from scarf.storage.zarr_store import (
+ DEFAULT_CLOUD_TARGET_CHUNK_BYTES,
+ count_array_spec,
+ matrix_layout,
+ )
+
+ budget = ResourceBudget(memoryBytes=24 * 1024**3, workers=4, workingCopies=4)
+ try:
+ set_resource_budget(budget)
+ cloud = count_array_spec(100_000, 45_525, dtype="uint32", remote=True)
+ local = count_array_spec(100_000, 45_525, dtype="uint32", remote=False)
+ expected, _ = matrix_layout(
+ 100_000,
+ 45_525,
+ budget=budget,
+ itemsize=4,
+ targetChunkBytes=DEFAULT_CLOUD_TARGET_CHUNK_BYTES,
+ minFeatureChunk=500,
+ maxFeatureChunk=10_000,
+ )
+ memory_first, _ = matrix_layout(
+ 100_000,
+ 45_525,
+ budget=budget,
+ itemsize=4,
+ )
+ finally:
+ set_resource_budget(None)
+ assert cloud.chunks == expected
+ assert local.chunks == memory_first
+ assert DEFAULT_CLOUD_TARGET_CHUNK_BYTES == 128 * 1024 * 1024
+
+
+def test_large_atac_count_array_accepts_sparse_writes(tmp_path):
+ import numpy as np
+
+ from scarf.storage.zarr_store import count_array_spec, create_numeric_array
+
+ n_cells, n_feats = 10_000, 89_796
+ spec = count_array_spec(n_cells, n_feats, dtype="uint32")
+ root = zarr.open_group(str(tmp_path / "atac.zarr"), mode="w")
+ arr = create_numeric_array(root, "counts", spec)
+ rows = np.arange(1000, dtype=np.int64)
+ cols = np.arange(1000, dtype=np.int64)
+ arr.set_coordinate_selection((rows, cols), np.ones(1000, dtype=np.uint32))
+
+
+def test_v2_group_skips_shards(tmp_path):
+ from scarf.storage.zarr_store import count_array_spec, create_numeric_array
+
+ root = zarr.open_group(str(tmp_path / "v2.zarr"), mode="w", zarr_format=2)
+ spec = count_array_spec(100, 50, dtype="uint32")
+ arr = create_numeric_array(root, "counts", spec)
+ assert array_metadata_shards(arr) is None
+
+
+def test_ann_index_round_trip(tmp_path):
+ import hnswlib
+
+ from scarf.storage.zarr_store import (
+ has_ann_index,
+ load_ann_index,
+ save_ann_index,
+ )
+
+ root = zarr.open_group(str(tmp_path / "ds.zarr"), mode="w")
+ g = root.create_group("ann")
+ dim = 8
+ n = 200
+ idx = hnswlib.Index(space="l2", dim=dim)
+ idx.init_index(max_elements=n, ef_construction=50, M=16)
+ data = np.random.rand(n, dim).astype(np.float32)
+ idx.add_items(data)
+ save_ann_index(g, idx)
+ assert has_ann_index(g)
+ loaded = load_ann_index(g, "l2", dim)
+ q = data[:5]
+ i1, d1 = idx.knn_query(q, k=3)
+ i2, d2 = loaded.knn_query(q, k=3)
+ np.testing.assert_array_equal(i1, i2)
+ np.testing.assert_allclose(d1, d2, rtol=1e-5)
diff --git a/uv.lock b/uv.lock
new file mode 100644
index 00000000..bd5e92d6
--- /dev/null
+++ b/uv.lock
@@ -0,0 +1,4064 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"
+resolution-markers = [
+ "python_full_version >= '3.15' and sys_platform == 'win32'",
+ "python_full_version == '3.14.*' and sys_platform == 'win32'",
+ "python_full_version >= '3.15' and sys_platform == 'emscripten'",
+ "python_full_version == '3.14.*' and sys_platform == 'emscripten'",
+ "python_full_version >= '3.15' and sys_platform != 'emscripten' and sys_platform != 'win32'",
+ "python_full_version == '3.14.*' and sys_platform != 'emscripten' and sys_platform != 'win32'",
+ "python_full_version < '3.14' and sys_platform == 'win32'",
+ "python_full_version < '3.14' and sys_platform == 'emscripten'",
+ "python_full_version < '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",
+]
+
+[[package]]
+name = "accessible-pygments"
+version = "0.0.5"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "pygments" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/bc/c1/bbac6a50d02774f91572938964c582fff4270eee73ab822a4aeea4d8b11b/accessible_pygments-0.0.5.tar.gz", hash = "sha256:40918d3e6a2b619ad424cb91e556bd3bd8865443d9f22f1dcdf79e33c8046872", size = 1377899, upload-time = "2024-05-10T11:23:10.216Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/8d/3f/95338030883d8c8b91223b4e21744b04d11b161a3ef117295d8241f50ab4/accessible_pygments-0.0.5-py3-none-any.whl", hash = "sha256:88ae3211e68a1d0b011504b2ffc1691feafce124b845bd072ab6f9f66f34d4b7", size = 1395903, upload-time = "2024-05-10T11:23:08.421Z" },
+]
+
+[[package]]
+name = "aiohappyeyeballs"
+version = "2.7.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ce/f4/eec0465c2f67b2664688d0240b3212d5196fd89e741df67ddb81f8d35658/aiohappyeyeballs-2.7.1.tar.gz", hash = "sha256:065665c041c42a5938ed220bdcd7230f22527fbec085e1853d2402c8a3615d9d", size = 24757, upload-time = "2026-07-01T17:11:55.501Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/71/43/1947f06babed6b3f1d7f38b0c767f52df66bfb2bc10b468c4a7de9eceff2/aiohappyeyeballs-2.7.1-py3-none-any.whl", hash = "sha256:9243213661e29250eb41368e5daa826fc017156c3b8a11440826b2e3ed376472", size = 15038, upload-time = "2026-07-01T17:11:54.055Z" },
+]
+
+[[package]]
+name = "aiohttp"
+version = "3.14.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "aiohappyeyeballs" },
+ { name = "aiosignal" },
+ { name = "attrs" },
+ { name = "frozenlist" },
+ { name = "multidict" },
+ { name = "propcache" },
+ { name = "typing-extensions", marker = "python_full_version < '3.13'" },
+ { name = "yarl" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/82/78/8ea7308cac6934de8c74a14f3d5f65d1c89287426688be79538d0e5c013d/aiohttp-3.14.1.tar.gz", hash = "sha256:307f2cff90a764d329e77040603fa032db89c5c24fdad50c4c15334cba744035", size = 7955794, upload-time = "2026-06-07T21:09:35.529Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/1d/21/151624b51cd92553d95424daf4bf19f19ce9be9002d19253e7e7ce67197b/aiohttp-3.14.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:d35143e27778b4bb0fb189562d7f275bff79c62ab8e98459717c0ea617ff2480", size = 757402, upload-time = "2026-06-07T21:06:40.311Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/82/280619e0bd7bf2454987e19282616e84762255dd9c8468f62382e8c191f1/aiohttp-3.14.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:bcfb80a2cc36fba2534e5e5b5264dc7ae6fcd9bf15256da3e53d2f499e6fa29d", size = 512310, upload-time = "2026-06-07T21:06:42.207Z" },
+ { url = "https://files.pythonhosted.org/packages/55/b2/2aac325583aaa1353045f96dffa586d8a34e8322e14a7ba49cffeb103ab4/aiohttp-3.14.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:27fd7c91e51729b4f7e1577865fa6d34c9adccbc39aabe9000285b48af9f0ec2", size = 512448, upload-time = "2026-06-07T21:06:43.813Z" },
+ { url = "https://files.pythonhosted.org/packages/8a/72/a60607cb849faa8af8a356c9329ea2eb6f395d49e82cc82ccba1fd8deb8f/aiohttp-3.14.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:64c567bf9eaf664280116a8688f63016e6b32db2505908e2bdaca1b6438142f2", size = 1766854, upload-time = "2026-06-07T21:06:45.391Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/d3/d9fe1c9ec7557ab4d0d82bebaa728c6418f0b93295ec2f4ab015f7710cc7/aiohttp-3.14.1-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:f5e6ff2bdbb8f4cd3fbe41f99e25bbcd58e3bf9f13d3dd31a11e7917251cc77a", size = 1740884, upload-time = "2026-06-07T21:06:47.413Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/dc/f2cecfaf9337ba3e63f181500814ff502aa3d00d9c7ec93a9d23d10a27b2/aiohttp-3.14.1-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2f73e01dc37122325caf079982621262f96d74823c179038a82fddfc50359264", size = 1810034, upload-time = "2026-06-07T21:06:50.165Z" },
+ { url = "https://files.pythonhosted.org/packages/66/d7/2ff65c5e65c0d7476daf7e15c032e0805e36811185b9623e3238ad6c763e/aiohttp-3.14.1-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:bb2c0c80d431c0d03f2c7dbf125150fedd4f0de17366a7ca33f7ccb822391842", size = 1904054, upload-time = "2026-06-07T21:06:52.035Z" },
+ { url = "https://files.pythonhosted.org/packages/20/9c/d445818389df371f56d141d881153ba23183c4735a03f7356ffb43f7757d/aiohttp-3.14.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3e6fc1a85fa7194a1a7d19f44e8609180f4a8eb5fa4c7ed8b4355f080fad235c", size = 1790278, upload-time = "2026-06-07T21:06:54.049Z" },
+ { url = "https://files.pythonhosted.org/packages/4d/aa/bf04cb4d865fc6101c2229a294ad744973b72e513fdc5a6b791e6983d72a/aiohttp-3.14.1-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:686b6c0d3911ec387b444ddf5dc62fb7f7c0a7d5186a7861626496a5ab4aff95", size = 1591795, upload-time = "2026-06-07T21:06:55.911Z" },
+ { url = "https://files.pythonhosted.org/packages/dc/b4/4dac0038960427ba832f6609dfb4ea5437d7fd80c72001b9e48f834f428b/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c6fa4dc7ad6f8109c70bb1499e589f76b0b792baf39f9b017eb92c8a81d0a199", size = 1728397, upload-time = "2026-06-07T21:06:57.777Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/f9/7cd4e8ad7aa3b75f17d56bb5498dd604a93d4e6eece822ba0568c413fff0/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:87a5eea1b2a5e21e1ebdbb33ad4165359189327e63fc4e4894693e7f821ac817", size = 1766504, upload-time = "2026-06-07T21:07:00.009Z" },
+ { url = "https://files.pythonhosted.org/packages/f9/df/fc01d9fcad0f73fed3f3d361f1f94f975947b50dff82919f6dc2bf4316cc/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:1c1421eb01d4fd608d88cc8290211d177a58532b55ad94076fb349c5bf467f0a", size = 1777806, upload-time = "2026-06-07T21:07:02.064Z" },
+ { url = "https://files.pythonhosted.org/packages/41/09/47e2d090bddcc8fb4ccb4c314aadc32d7c5d9bb55f50f6ad1c92fc15d501/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:34b257ec41345c1e8f2df68fa908a7952f5de932723871eb633ecbbff396c9a4", size = 1580707, upload-time = "2026-06-07T21:07:03.942Z" },
+ { url = "https://files.pythonhosted.org/packages/3d/36/f1a4ce904ae0b6930cfe9afc96d0896f7ec1a620c400405d63783bb95a9c/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:de538791a80e5d862addbc183f70f0158ac9b9bb872bb147f1fd2a683691e087", size = 1798121, upload-time = "2026-06-07T21:07:05.987Z" },
+ { url = "https://files.pythonhosted.org/packages/70/0a/e0075ce9ca0279ee1d4f0c0b85f54fea02ebc83c3007651a72bece658fec/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:6f71173be42d3241d428f760122febb748de0623f44308a6f120d0dd9ec572e3", size = 1767580, upload-time = "2026-06-07T21:07:07.873Z" },
+ { url = "https://files.pythonhosted.org/packages/3e/61/a0c0a8f327a9c52095cdd8e312391b00d3ed64ab6c72bb5c33d8ec251cf7/aiohttp-3.14.1-cp312-cp312-win32.whl", hash = "sha256:ec8dc383ee57ea3e883477dcca3f11b65d58199f1080acaf4cd6ad9a99698be4", size = 452771, upload-time = "2026-06-07T21:07:09.669Z" },
+ { url = "https://files.pythonhosted.org/packages/df/d9/ea367c75f16ac9c6cdc8febb25e8318fa21a2b1bc8d6514d4b2d890bface/aiohttp-3.14.1-cp312-cp312-win_amd64.whl", hash = "sha256:2aa92c87868cd13674989f9ee83e5f9f7ea4237589b728048e1f0c8f6caa3271", size = 479873, upload-time = "2026-06-07T21:07:11.538Z" },
+ { url = "https://files.pythonhosted.org/packages/03/64/8d96784a7851156db8a4c6c3f6f91042fdf39fb15a4cc38c8b3c14833c45/aiohttp-3.14.1-cp312-cp312-win_arm64.whl", hash = "sha256:2c840c90759922cb5e6dda94596e079a30fb5a5ba548e7e0dc00574703940847", size = 448073, upload-time = "2026-06-07T21:07:13.637Z" },
+ { url = "https://files.pythonhosted.org/packages/bc/97/bd137012dd97e1649162b099135a80e1fd59aaa807b2430fc448d1029aff/aiohttp-3.14.1-cp313-cp313-android_21_arm64_v8a.whl", hash = "sha256:b3a03285a7f9c7b016324574a6d92a1c895da6b978cb8f1deee3ac72bc6da178", size = 506882, upload-time = "2026-06-07T21:07:15.501Z" },
+ { url = "https://files.pythonhosted.org/packages/ef/79/e5cc690e9d922a66887ceeaca53a8ffd5a7b0be3816142b7abc433742d89/aiohttp-3.14.1-cp313-cp313-android_21_x86_64.whl", hash = "sha256:2a73f487ab8ef5abbb24b7aa9b73e98eaba9e9e031804ff2416f02eca315ccaf", size = 515270, upload-time = "2026-06-07T21:07:17.53Z" },
+ { url = "https://files.pythonhosted.org/packages/fe/22/a73ccbf9dbd6e26dda0b24d5fd5db7da92ee3383a79f47677ffb834c5c5b/aiohttp-3.14.1-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:915fbb7b41b115192259f8c9ae58f3ddc444d2b5579917270211858e606a4afd", size = 485841, upload-time = "2026-06-07T21:07:19.555Z" },
+ { url = "https://files.pythonhosted.org/packages/3b/b9/57ed8eaf596321c2ad747bd480fb1700dbd7177c60dfc9e4c187f629662e/aiohttp-3.14.1-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:7fb4bdf95b0561a79f259f9d28fbc109728c5ee7f27aff6391f0ca703a329abe", size = 492088, upload-time = "2026-06-07T21:07:21.581Z" },
+ { url = "https://files.pythonhosted.org/packages/78/c0/5ebe5270a7c140d7c6f79dcb018640225f14d406c149e4eec04a7d82fe71/aiohttp-3.14.1-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:1b9748363260121d2927704f5d4fc498150669ca3ae93625986ee89c8f80dcd4", size = 501564, upload-time = "2026-06-07T21:07:23.388Z" },
+ { url = "https://files.pythonhosted.org/packages/75/7f/8cdaa24fc7983865e0915153b96a9ac5bcdd3548d64c5a27d17cecccad2d/aiohttp-3.14.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:86a6dab78b0e43e2897a3bbe15745aa60dc5423ca437b7b0b164c069bf91b876", size = 751998, upload-time = "2026-06-07T21:07:25.046Z" },
+ { url = "https://files.pythonhosted.org/packages/b2/f4/c4227aacfacc5cb0cc2d119b65301d177912a6842cd64e120c47af76064f/aiohttp-3.14.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:4dfd6e47d3c44c2279907607f73a4240b88c69eb8b90da7e2441a8045dfd21da", size = 510918, upload-time = "2026-06-07T21:07:27.28Z" },
+ { url = "https://files.pythonhosted.org/packages/ab/01/a2d5f96cd4e74424864d30bc0a7e44d0a12dacdcfa91b5b2d1bd3dca6bf3/aiohttp-3.14.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:317acd9f8602858dc7d59679812c376c7f0b97bcbbf16e0d6237f54141d8a8a6", size = 508657, upload-time = "2026-06-07T21:07:29.252Z" },
+ { url = "https://files.pythonhosted.org/packages/e8/ed/3c0fb5c500fdd8e7ebc10d1889c04384fffa1a9163eac1356088ca9da1b1/aiohttp-3.14.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bd869c427324e5cb15195793de951295710db28be7d818247f3097b4ab5d4b96", size = 1757907, upload-time = "2026-06-07T21:07:31.03Z" },
+ { url = "https://files.pythonhosted.org/packages/0b/ab/d4c924d9bd5be3050c226612413ce68cb54c70d2c31b661bfc8d9a5b6a70/aiohttp-3.14.1-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:93b032b5ec3255473c143627d21a69ac74ae12f7f33974cb587c564d11b1066f", size = 1737565, upload-time = "2026-06-07T21:07:33.031Z" },
+ { url = "https://files.pythonhosted.org/packages/19/2a/37326821ff779084020cdc33224d20b19f42f4183a500ff92022a739eda7/aiohttp-3.14.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f234b4deb12f3ad59127e037bc57c40c21e45b45282df7d3a55a0f409f595296", size = 1799018, upload-time = "2026-06-07T21:07:35.003Z" },
+ { url = "https://files.pythonhosted.org/packages/b3/4f/6e947ba73e4ce09070761c05ed3a8ceb7c21f5e46798671d8b2aac0e4626/aiohttp-3.14.1-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:9af6779bfb46abf124068327abcdf9ce95c9ef8287a3e8da76ccf2d0f16c28fa", size = 1894416, upload-time = "2026-06-07T21:07:36.956Z" },
+ { url = "https://files.pythonhosted.org/packages/9d/6e/dbf1d0625dc711fb2851f4f3c3055c39ed58bae92082d8c627dbe6013736/aiohttp-3.14.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:faccab372e66bc76d5731525e7f1143c922271725b9d38c9f97edcc66266b451", size = 1783881, upload-time = "2026-06-07T21:07:39.063Z" },
+ { url = "https://files.pythonhosted.org/packages/44/c2/5e25098a67268ed369483ae7d1a58bd0a13d03aab860d2a0e4a6eb25b046/aiohttp-3.14.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f380468b09d2a81633ee863b0ec5648d364bd17bb8ecfb8c2f387f7ac1faf42c", size = 1587572, upload-time = "2026-06-07T21:07:41.058Z" },
+ { url = "https://files.pythonhosted.org/packages/2a/bd/cf9cee17e140f942a3de73e658a543aa8fbf35a5fc67a9d2538d52d77f0b/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:97e704dcd26271f5bda3fa07c3ce0fb76d6d3f8659f4baa1a24442cc9ba177ca", size = 1722137, upload-time = "2026-06-07T21:07:43.014Z" },
+ { url = "https://files.pythonhosted.org/packages/89/6d/5684f8c59045c96f81a18cefbc1fbbd79d25b88f1c622f2a5c5c08fcb632/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:269b76ac5394092b95bc4a098f4fc6c191c083c3bd12775d1e30e663132f6a09", size = 1755953, upload-time = "2026-06-07T21:07:45.933Z" },
+ { url = "https://files.pythonhosted.org/packages/a8/40/35caf3170f8359760740a7d9aa0fff2e344bef98e1d1186f5a0f6dec17e6/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:5c0b3e614340c889d575451696374c9d17affd54cd607ca0babed8f8c37b9397", size = 1766479, upload-time = "2026-06-07T21:07:48.047Z" },
+ { url = "https://files.pythonhosted.org/packages/6d/a1/b0c61e7a137f0d81de49a82023a6df73c3c16d6fefb0f8e4a93d21639002/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:5663ee9257cfa1add7253a7da3035a02f31b6600ec48261585e1800a81533080", size = 1580077, upload-time = "2026-06-07T21:07:50.069Z" },
+ { url = "https://files.pythonhosted.org/packages/0b/41/194ea4623693009fcefebef7aef63c141754f153e9cd0d39d3b9e36c175c/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:603a2c834142172ffddc054067f5ec0ca65d57a0aa98a71bc81952573208e345", size = 1791688, upload-time = "2026-06-07T21:07:52.106Z" },
+ { url = "https://files.pythonhosted.org/packages/ba/45/4de841f005cfe1fd63e2a2fe011262c515e2a62aa6994b15947e7d717ac9/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:cb21957bb8aca671c1765e32f58164cf0c50e6bf41c0bbbd16da20732ecaf588", size = 1761094, upload-time = "2026-06-07T21:07:54.113Z" },
+ { url = "https://files.pythonhosted.org/packages/e4/ae/dbce10533d3896d544d5053939ed75b7dc31a1b0973d959b1b5ae21028d6/aiohttp-3.14.1-cp313-cp313-win32.whl", hash = "sha256:e509a55f681e6158c20f70f102f9cf61fb20fbc382272bc6d94b7343f2582780", size = 452662, upload-time = "2026-06-07T21:07:56.06Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/d9/0bf1a19362c32f06229da5e7ddfcec91f93474d6307f7a2d3135e9c674dc/aiohttp-3.14.1-cp313-cp313-win_amd64.whl", hash = "sha256:1ac8531b638959718e18c2207fbfe297819875da46a740b29dfa29beba64355a", size = 479748, upload-time = "2026-06-07T21:07:58.319Z" },
+ { url = "https://files.pythonhosted.org/packages/22/0a/62e7232dc9484fbec112ceb32efb6a624cc7994ec6e2b019286f17c4e8f2/aiohttp-3.14.1-cp313-cp313-win_arm64.whl", hash = "sha256:250d14af67f6b6a1a4a811049b1afa69d61d617fca6bf33149b3ab1a6dbcf7b8", size = 447723, upload-time = "2026-06-07T21:08:00.154Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/a1/5fafa04e1ca91ddb47608699d60649c1c6db3cf41c99e78fc4056f9513db/aiohttp-3.14.1-cp314-cp314-android_24_arm64_v8a.whl", hash = "sha256:7c106c26852ca1c2047c6b80384f17100b4e439af276f21ef3d4e2f450ae7e15", size = 508531, upload-time = "2026-06-07T21:08:02.093Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/2e/bfa02f699d87ffc86d5959270b28f1cb410add3ccaced8ed2e0b8a5238fc/aiohttp-3.14.1-cp314-cp314-android_24_x86_64.whl", hash = "sha256:20205f7f5ade7aaec9f4b500549bbc071b046453aed72f9c06dcab87896a83e8", size = 514718, upload-time = "2026-06-07T21:08:04.476Z" },
+ { url = "https://files.pythonhosted.org/packages/85/a5/9594ad6289eebbc97d167c44213d557807f90e59115caad24de21ad2c3b1/aiohttp-3.14.1-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:62a759436b29e677181a9e76bab8b8f689a29cb9c535f45f7c48c9c830d3f8c3", size = 487918, upload-time = "2026-06-07T21:08:06.377Z" },
+ { url = "https://files.pythonhosted.org/packages/b4/61/16a32c36c3c49edec122a3dc811f2057df2f94d3b14aa107c8017d981618/aiohttp-3.14.1-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:2964cbf553df4d7a57348da44d961d871895fc1ee4e8c322b2a95612c7b17fba", size = 494014, upload-time = "2026-06-07T21:08:08.263Z" },
+ { url = "https://files.pythonhosted.org/packages/9b/89/3ebcf96ed99c05bec9c434aaac6963fd3cbab4a786ae739908a144d9ce44/aiohttp-3.14.1-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:237651caadc3a59badd39319c54642b5299e9cc98a3a194310e55d5bb9f5e397", size = 502398, upload-time = "2026-06-07T21:08:10.244Z" },
+ { url = "https://files.pythonhosted.org/packages/fd/3d/b74870a0c2d40c355928cd5b96c7a11fa821b8a40fc41365e64479b151fb/aiohttp-3.14.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:896e12dfdbbab9d8f7e16d2b28c6769a60126fa92095d1ebf9473d02593a2448", size = 758018, upload-time = "2026-06-07T21:08:12.447Z" },
+ { url = "https://files.pythonhosted.org/packages/d3/66/f42f5c984d99e49c6cff5f26f590750f2e2f7ef1fcfb99966ab5be1b632e/aiohttp-3.14.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d03f281ed22579314ba00821ce20115a7c0ac430660b4cc05704a3f818b3e004", size = 512462, upload-time = "2026-06-07T21:08:14.624Z" },
+ { url = "https://files.pythonhosted.org/packages/e9/a7/248e1aebe0c7810b0271e021a0f2a5eb6e78a051885b3c9df49f42a5802d/aiohttp-3.14.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:07eabb979d236335fed927e137a928c9adfb7df3b9ec7aa31726f133a62be983", size = 512824, upload-time = "2026-06-07T21:08:16.572Z" },
+ { url = "https://files.pythonhosted.org/packages/26/97/2aa0e5ba0727dc3bd5aaebb7ccbc510f7dfb7fb961ec87497cd496635ab1/aiohttp-3.14.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4fe1f1087cbadb280b5e1bb054a4f00d1423c74d6626c5e48400d871d34ecefe", size = 1749898, upload-time = "2026-06-07T21:08:18.635Z" },
+ { url = "https://files.pythonhosted.org/packages/00/8d/e97f6c96c891d457c8479d92a514ba194d0412f981d72c70341ee18488ed/aiohttp-3.14.1-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:367a9314fdc79dab0fac96e216cb41dd73c85bdca85306ce8999118ba7e0f333", size = 1710114, upload-time = "2026-06-07T21:08:20.892Z" },
+ { url = "https://files.pythonhosted.org/packages/6f/e6/aa8d7e863048c8fceb5cd6ce74017311cec3ead07847387e12265fb4444e/aiohttp-3.14.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a24f677ebe83749039e7bdf862ff0bbb16818ae4193d4ef96505e269375bcce0", size = 1802541, upload-time = "2026-06-07T21:08:23.044Z" },
+ { url = "https://files.pythonhosted.org/packages/83/a8/72193137de57fda4ebfae4563182d082c8856e3b6e9871d0b46f028fb369/aiohttp-3.14.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c83afe0ba876be7e943d2e0ba645809ad441575d2840c895c21ee5de93b9377a", size = 1875776, upload-time = "2026-06-07T21:08:25.288Z" },
+ { url = "https://files.pythonhosted.org/packages/a0/18/938441025db6769a3464596b2410af3afde0b21eb2f204c6f766f68af4bd/aiohttp-3.14.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:634e385930fb6d2d479cf3aa66515955863b77a5e3c2b5894ca259a25b308602", size = 1760329, upload-time = "2026-06-07T21:08:27.363Z" },
+ { url = "https://files.pythonhosted.org/packages/60/29/bf2496b4065e76e09fe48015aaffe5ce161d8f089b06ac6982070f653076/aiohttp-3.14.1-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:eeea07c4397bbc57719c4eed8f9c284874d4f175f9b6d57f7a1546b976d455ca", size = 1587293, upload-time = "2026-06-07T21:08:29.805Z" },
+ { url = "https://files.pythonhosted.org/packages/49/a2/2136674d52123b1354bd05dd5753c318db47dc0c927cc70b27bab3755456/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:335c0cc3e3545ce98dcb9cfcb836f40c3411f43fa03dab757597d80c89af8a35", size = 1714756, upload-time = "2026-06-07T21:08:32.094Z" },
+ { url = "https://files.pythonhosted.org/packages/a7/b9/e5fd2e6f915503081c0f9b1e8540947037929c70c191da2e4d54b31a21a1/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:ae6be797afdef264e8a84864a85b196ca06045586481b3df8a967322fd2fa844", size = 1721052, upload-time = "2026-06-07T21:08:34.167Z" },
+ { url = "https://files.pythonhosted.org/packages/63/5a/2833e324a2263e104e31e2e91bc5bbee81bc499afd32203faee048a883f0/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:8560b4d712474335d08907db7973f71912d3a9a8f1dee992ec06b5d2fe359496", size = 1766888, upload-time = "2026-06-07T21:08:36.95Z" },
+ { url = "https://files.pythonhosted.org/packages/57/fa/dea6511870913162f3b2e8c42a7614eb203a4540b8c2da43e0bfb0548f3c/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:2b7edd08e0a5deb1e8564a2fcd8f4561014a3f05252334671bbf55ddd47db0e5", size = 1581679, upload-time = "2026-06-07T21:08:39.292Z" },
+ { url = "https://files.pythonhosted.org/packages/14/bd/3cf0d55e71784b33534e9710a67d382d900598b4787fbce6cc7317f8c42a/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:b6ff7fcee63287ae57b5df3e4f5957ce032122802509246dec1a5bcc55904c95", size = 1782021, upload-time = "2026-06-07T21:08:41.407Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/af/14bb5843eccbe234f4dfb78ab73e549d99727247e62ae5d62cbd22eaf5b0/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6ffbb2f4ec1ceaff7e07d43922954da26b223d188bf30658e561b98e23089444", size = 1742574, upload-time = "2026-06-07T21:08:43.795Z" },
+ { url = "https://files.pythonhosted.org/packages/f2/1e/fbeb7af9210a67ac0f9c9bec0f8f4568497924e33137a3d5b48e1cf85f3f/aiohttp-3.14.1-cp314-cp314-win32.whl", hash = "sha256:a9875b46d910cff3ea2f5962f9d266b465459fe634e22556ab9bd6fc1192eea0", size = 457773, upload-time = "2026-06-07T21:08:46.168Z" },
+ { url = "https://files.pythonhosted.org/packages/f0/2b/13e8d741a9ec5db7d900c060554cf8352ab85e44e2a4469ebb9d377bda17/aiohttp-3.14.1-cp314-cp314-win_amd64.whl", hash = "sha256:af8b4b81a960eeaf1234971ac3cd0ba5901f3cd42eae42a46b4d089a8b492719", size = 485001, upload-time = "2026-06-07T21:08:48.401Z" },
+ { url = "https://files.pythonhosted.org/packages/df/30/491acfa2c4d6c3ff59c49a14fc1b50be3241e25bbb0c84c09e2da4d11395/aiohttp-3.14.1-cp314-cp314-win_arm64.whl", hash = "sha256:cf4491381b1b57425c315a56a439251b1bdac07b2275f19a8c44bc57744532ec", size = 453809, upload-time = "2026-06-07T21:08:50.7Z" },
+ { url = "https://files.pythonhosted.org/packages/34/e3/19dbe1a1f4cc6230eb9e314de7fe68053b0992f9302b27d12141a0b5db53/aiohttp-3.14.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:819c054312f1af92947e6a55883d1b66feefab11531a7fc45e0fb9b63880b5c2", size = 793320, upload-time = "2026-06-07T21:08:52.775Z" },
+ { url = "https://files.pythonhosted.org/packages/7f/20/1b7182219ba1b108430d6e4dc53d25ae02dcfcf5a045b33af4e8c5167527/aiohttp-3.14.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:10ee9c1753a8f706345b22496c79fbddb5be0599e0823f3738b1534058e25340", size = 529077, upload-time = "2026-06-07T21:08:55Z" },
+ { url = "https://files.pythonhosted.org/packages/b9/c8/14ce60ec31a2e5f5274bb17d383a6f7a3aabca31ac04eee05585bbadab16/aiohttp-3.14.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1601cc37baf5750ccacae618ec2daf020769581695550e3b654a911f859c563d", size = 532476, upload-time = "2026-06-07T21:08:57.176Z" },
+ { url = "https://files.pythonhosted.org/packages/7e/02/9ac85e081e53da2e061b02fa7758fe0a12d17b8ce2d1f5e6c7cb76730328/aiohttp-3.14.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4d6e0ac9da31c9c04c84e1c0182ad8d6df35965a85cae29cd71d089621b3ae94", size = 1922347, upload-time = "2026-06-07T21:08:59.563Z" },
+ { url = "https://files.pythonhosted.org/packages/c0/3e/d3ba07a0ab38b5389e10bec4362d21e10a4f667cba2d79ba30837b3a5059/aiohttp-3.14.1-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:9e8f2d660c350b3d0e259c7a7e3d9b7fc8b41210cbcc3d4a7076ff0a5e5c2fdc", size = 1786465, upload-time = "2026-06-07T21:09:01.909Z" },
+ { url = "https://files.pythonhosted.org/packages/0b/cb/e2ee978a00cfb2df829704a69528b18154eba5939f45bc1efa8f33aee4c5/aiohttp-3.14.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4691802dda97be727f79d86818acaad7eb8e9252626a1d6b519fedbb92d5e251", size = 1909423, upload-time = "2026-06-07T21:09:04.357Z" },
+ { url = "https://files.pythonhosted.org/packages/73/5d/1430334858b1022b58ae50399a918f0bd6fe8fa7fa183598d657ff61e040/aiohttp-3.14.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c389c482a7e9b9dc3ee2701ac46c4125297a3818875b9c305ddb603c04828fd1", size = 2001906, upload-time = "2026-06-07T21:09:06.722Z" },
+ { url = "https://files.pythonhosted.org/packages/66/4e/560c7472d3d198a23aa5c8b19a5115bf6a9b77b7d3e4bb363da320430ad2/aiohttp-3.14.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fc0cacab7ba4e56f0f81c82a98c09bed2f39c940107b03a34b168bdf7597edd3", size = 1877095, upload-time = "2026-06-07T21:09:09.011Z" },
+ { url = "https://files.pythonhosted.org/packages/0d/f1/4745806578d447db4a784a8591e2dae3afdfc2bcb96f8f81271b13df6543/aiohttp-3.14.1-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:979ed4717f59b8bb12e3963378fa285d93d367e15bcd66c721311826d3c44a6c", size = 1676222, upload-time = "2026-06-07T21:09:11.461Z" },
+ { url = "https://files.pythonhosted.org/packages/6a/c9/48255813cca749a229ef0ab476004ec623728ad79a9c0840616f6c076325/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:38e1e7daaea81df51c952e18483f323d878499a1e2bfe564790e0f9701d6f203", size = 1842922, upload-time = "2026-06-07T21:09:14.118Z" },
+ { url = "https://files.pythonhosted.org/packages/3d/c0/bbd054e2bee909f529523a5af3891052606af5143c09f5f183ec3b234676/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:4132e72c608fe9fecb8f409113567605915b83e9bdd3ea56538d2f9cd35002f1", size = 1825035, upload-time = "2026-06-07T21:09:16.447Z" },
+ { url = "https://files.pythonhosted.org/packages/a8/ae/90395d4376deceb74e09ec26b6adf7d2015a6f8802d6d84446af860fef04/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:eefd9cc9b6d4a2db5f00a26bc3e4f9acf71926a6ec557cd56c9c6f27c290b665", size = 1849512, upload-time = "2026-06-07T21:09:18.742Z" },
+ { url = "https://files.pythonhosted.org/packages/93/bd/fb25f3049957553d4ce0ba6ae480aa2f592a6985497fca590837d16c1be0/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:b165790117eea512d7f3fb22f1f6dad3d55a7189571993eb015591c1401276d1", size = 1668571, upload-time = "2026-06-07T21:09:21.458Z" },
+ { url = "https://files.pythonhosted.org/packages/3f/22/7f73303d64dd567ff3addca90b556690ed1233a47b8f55d242fb90af3681/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:ed09c7eb1c391271c2ed0314a51903e72a3acb653d5ccfc264cdf3ef11f8269d", size = 1881159, upload-time = "2026-06-07T21:09:23.813Z" },
+ { url = "https://files.pythonhosted.org/packages/44/be/0474c5a8b5640e1e4aa1923430a91f4151be82e511373fe764189b89aef5/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:99abd37084b82f5830c635fddd0b4993b9742a66eb746dacf433c8590e8f9e3c", size = 1841409, upload-time = "2026-06-07T21:09:26.207Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/3c/bb4a7cba26956cb3da4553cc2056cf67be5b5ff6e6d8fa4fbdff73bfb7ae/aiohttp-3.14.1-cp314-cp314t-win32.whl", hash = "sha256:47ddf841cdecc810749921d25606dee45857d12d2ad5ddb7b5bd7eab12e4b365", size = 494166, upload-time = "2026-06-07T21:09:28.505Z" },
+ { url = "https://files.pythonhosted.org/packages/8a/84/ec80c2c1f66a952555a9f86df6b33af65108a6febfa0471b69013a12f807/aiohttp-3.14.1-cp314-cp314t-win_amd64.whl", hash = "sha256:5e78b522b7a6e27e0b25d19b247b75039ac4c94f99823e3c9e53ae1603a9f7e9", size = 530255, upload-time = "2026-06-07T21:09:30.843Z" },
+ { url = "https://files.pythonhosted.org/packages/2a/71/6e22be134a4061ada85a92951b842f2657f17d926b727f3f94c56ae963d6/aiohttp-3.14.1-cp314-cp314t-win_arm64.whl", hash = "sha256:90d53f1609c29ccc2193945ef732428382a28f78d0456ae4d3daf0d48b74f0f6", size = 469640, upload-time = "2026-06-07T21:09:33.028Z" },
+]
+
+[[package]]
+name = "aiosignal"
+version = "1.4.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "frozenlist" },
+ { name = "typing-extensions", marker = "python_full_version < '3.13'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/61/62/06741b579156360248d1ec624842ad0edf697050bbaf7c3e46394e106ad1/aiosignal-1.4.0.tar.gz", hash = "sha256:f47eecd9468083c2029cc99945502cb7708b082c232f9aca65da147157b251c7", size = 25007, upload-time = "2025-07-03T22:54:43.528Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/fb/76/641ae371508676492379f16e2fa48f4e2c11741bd63c48be4b12a6b09cba/aiosignal-1.4.0-py3-none-any.whl", hash = "sha256:053243f8b92b990551949e63930a839ff0cf0b0ebbe0597b0f3fb19e1a0fe82e", size = 7490, upload-time = "2025-07-03T22:54:42.156Z" },
+]
+
+[[package]]
+name = "alabaster"
+version = "1.0.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/a6/f8/d9c74d0daf3f742840fd818d69cfae176fa332022fd44e3469487d5a9420/alabaster-1.0.0.tar.gz", hash = "sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e", size = 24210, upload-time = "2024-07-26T18:15:03.762Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/7e/b3/6b4067be973ae96ba0d615946e314c5ae35f9f993eca561b356540bb0c2b/alabaster-1.0.0-py3-none-any.whl", hash = "sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b", size = 13929, upload-time = "2024-07-26T18:15:02.05Z" },
+]
+
+[[package]]
+name = "anndata"
+version = "0.12.18"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "array-api-compat" },
+ { name = "h5py" },
+ { name = "legacy-api-wrap" },
+ { name = "natsort" },
+ { name = "numpy" },
+ { name = "packaging" },
+ { name = "pandas" },
+ { name = "scipy" },
+ { name = "scverse-misc" },
+ { name = "zarr" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/0f/02/e74c34c6668e04e3092f0224620f9eb4e3abfd8fca7bed2e790086a21e8e/anndata-0.12.18.tar.gz", hash = "sha256:8c14e1814a4119ae92cdfb5d2d1365e0044829e9eaed8f03522cb0eaddd0ccc5", size = 2257885, upload-time = "2026-06-23T09:26:36.702Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/a2/d3/e0ce94d247ccc5240c6433f44155c3acdc54e812418f7d88c596bde845a3/anndata-0.12.18-py3-none-any.whl", hash = "sha256:23b97a2f449fa2b900f9bbbe1712d36d6f9d6d680feb21144e1c72802af1f014", size = 176113, upload-time = "2026-06-23T09:26:34.835Z" },
+]
+
+[[package]]
+name = "annotated-types"
+version = "0.7.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ee/67/531ea369ba64dcff5ec9c3402f9f51bf748cec26dde048a2f973a4eea7f5/annotated_types-0.7.0.tar.gz", hash = "sha256:aff07c09a53a08bc8cfccb9c85b05f1aa9a2a6f23728d790723543408344ce89", size = 16081, upload-time = "2024-05-20T21:33:25.928Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/78/b6/6307fbef88d9b5ee7421e68d78a9f162e0da4900bc5f5793f6d3d0e34fb8/annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53", size = 13643, upload-time = "2024-05-20T21:33:24.1Z" },
+]
+
+[[package]]
+name = "anyio"
+version = "4.14.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "idna" },
+ { name = "typing-extensions", marker = "python_full_version < '3.13'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/3b/72/5562aabb8dd7181e8e860622a38bea08d17842b99ecd4c91f84ac95251b0/anyio-4.14.1.tar.gz", hash = "sha256:8d648a3544c1a700e3ff78615cd679e4c5c3f149904287e73687b2596963629e", size = 254831, upload-time = "2026-06-24T20:56:06.017Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/b0/7b/90df4a0a816d98d6ea26f559d87836d494a2cf1fcf063be67df50a7bcc30/anyio-4.14.1-py3-none-any.whl", hash = "sha256:4e5533c5b8ff0a24f5d7a176cbe6877129cd183893f66b537f8f227d10527d72", size = 124875, upload-time = "2026-06-24T20:56:04.413Z" },
+]
+
+[[package]]
+name = "appnope"
+version = "0.1.4"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/35/5d/752690df9ef5b76e169e68d6a129fa6d08a7100ca7f754c89495db3c6019/appnope-0.1.4.tar.gz", hash = "sha256:1de3860566df9caf38f01f86f65e0e13e379af54f9e4bee1e66b48f2efffd1ee", size = 4170, upload-time = "2024-02-06T09:43:11.258Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/81/29/5ecc3a15d5a33e31b26c11426c45c501e439cb865d0bff96315d86443b78/appnope-0.1.4-py2.py3-none-any.whl", hash = "sha256:502575ee11cd7a28c0205f379b525beefebab9d161b7c964670864014ed7213c", size = 4321, upload-time = "2024-02-06T09:43:09.663Z" },
+]
+
+[[package]]
+name = "array-api-compat"
+version = "1.15.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/86/12/65d297d3e425cb7fe5247b7ec14b4ed83539e9b5e89b65c15ea7ee274182/array_api_compat-1.15.0.tar.gz", hash = "sha256:53c5f922491bf15f62847afafc4e39eedfae57d218988fefb8cce39c2a9b3dea", size = 129305, upload-time = "2026-06-07T20:53:24.964Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/86/16/1a8fd2b19544b84575cf84ef7aa3ad4c173b756d5f087c91f85d1b295777/array_api_compat-1.15.0-py3-none-any.whl", hash = "sha256:7b1b9c53269061403fd5f45a8de349f16e7887653328bfa0c5f2d45299ff0a8e", size = 79113, upload-time = "2026-06-07T20:53:23.621Z" },
+]
+
+[[package]]
+name = "ast-serialize"
+version = "0.5.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/81/9d/09e27731bd5864a9ce04e3244074e674bb8936bf62b45e0357248717adac/ast_serialize-0.5.0.tar.gz", hash = "sha256:5880091bfe6f4f986f22866375c2e884843e7a0b6343ae41aeea659613d879b6", size = 61157, upload-time = "2026-05-17T17:48:29.429Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/c0/9a/13dde51ba9e15f8b97957ab7cb0120d0e381524d651c6bd630b9c359227f/ast_serialize-0.5.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:8f5c14f169eb0972c0c21bada5358b23d6047c76583b005234f865b11f1fa00a", size = 1183520, upload-time = "2026-05-17T17:47:30.831Z" },
+ { url = "https://files.pythonhosted.org/packages/37/de/5a7f0a9fe68944f536632a5af84676739c7d2582be42deb082634bf3a754/ast_serialize-0.5.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7d1a2de9de5be04652f0ed60738356ef94f66db37924a9499fffe98dc491aa0b", size = 1175779, upload-time = "2026-05-17T17:47:32.551Z" },
+ { url = "https://files.pythonhosted.org/packages/9c/81/0bb853e76e4f6e9a1855d569003c59e19ffac45f7079d91505d1bb212f92/ast_serialize-0.5.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:be5173fb66f9b49026d9d5a2ff0fc7c7009077107c0eb285b2d60fdf1fe10bd1", size = 1233750, upload-time = "2026-05-17T17:47:34.731Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/d3/4cf705beeccc08754d0bbda99aefff26110e209b9a07ac8a6b60eec48531/ast_serialize-0.5.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f8015cd071ac1339924ee2b8098c93e00e155f30a16f40ec9816fcf84f4753f6", size = 1235942, upload-time = "2026-05-17T17:47:36.287Z" },
+ { url = "https://files.pythonhosted.org/packages/26/c8/ee097e437ea27dd2b8b227865c875492b585650a5802a22d82b304c8201b/ast_serialize-0.5.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5499e8797edff2a9186aa313ed382c6b422e798e9332d9953badcee6e69a88f2", size = 1442517, upload-time = "2026-05-17T17:47:38.17Z" },
+ { url = "https://files.pythonhosted.org/packages/ff/bd/68063442838f1ba68ec72b5436430bc75b3bb17a1a3c3063f09b0c05ae2b/ast_serialize-0.5.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6848f2a093fb5548751a9a09bff8fcd229e2bbeb0e3331f391b6ae6d26cd9903", size = 1254081, upload-time = "2026-05-17T17:47:39.826Z" },
+ { url = "https://files.pythonhosted.org/packages/50/e2/1e520793bc6a4e4524a6ab022391e827825eaa0c3811828bfdc6852eca26/ast_serialize-0.5.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:832d4c998e0b091fd60a6d6bceee535483c4d490de9ba85003af835225719261", size = 1259910, upload-time = "2026-05-17T17:47:41.369Z" },
+ { url = "https://files.pythonhosted.org/packages/4e/e1/49b60f467979979cfe6913b43948ff25bca971ad0591d181812f163a988e/ast_serialize-0.5.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:16db7c62ec0b8efe1d7afd283a388d8f74f2605d56032e5a37747d2de8dba027", size = 1250678, upload-time = "2026-05-17T17:47:43.702Z" },
+ { url = "https://files.pythonhosted.org/packages/74/ba/66ab9555de6275677566f6574e5ef6c29cb185ea866f643bc06f8280a8ee/ast_serialize-0.5.0-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:baf5eb061eb5bccade4128ad42da33787d72f6013809cd1b590376ece8b3c937", size = 1301603, upload-time = "2026-05-17T17:47:46.256Z" },
+ { url = "https://files.pythonhosted.org/packages/66/42/6aca9b9abc710014b2be9059689e5dd1679339e78f567ffb4d255a9e2050/ast_serialize-0.5.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:104e4a35bd7c124173c41760ef9aaea17ddb3f86c65cb643671d59afbe3ee94c", size = 1410332, upload-time = "2026-05-17T17:47:47.899Z" },
+ { url = "https://files.pythonhosted.org/packages/47/68/2f76594432a22581ecf878b5e75a9b8601c24b2241cf0bbeb1e21fcf370c/ast_serialize-0.5.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:36be371028fc1675acb38a331bde160dbab7ff907fdf00b67eb6911aa106951b", size = 1509979, upload-time = "2026-05-17T17:47:50.942Z" },
+ { url = "https://files.pythonhosted.org/packages/40/ac/a93c9b58292653f6c595752f677a08e608f903b710594909e9231a389b3b/ast_serialize-0.5.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:061ee58bdb52341c8201a6df41182a977736bae3b7ded87ca7176ca25a8a47ab", size = 1505002, upload-time = "2026-05-17T17:47:54.093Z" },
+ { url = "https://files.pythonhosted.org/packages/14/2e/b278f68c497ee2f1d1576cbbef8db5281cd4a5f2db040537592ac9c8862e/ast_serialize-0.5.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:b15219e9cdc9f53f6f4cb51c009203507228226148c05c5e8fe451c28b435eb3", size = 1456231, upload-time = "2026-05-17T17:47:56.311Z" },
+ { url = "https://files.pythonhosted.org/packages/0b/43/419be1c566a4c504cd8fd60ce2f84e790f295495c0f327cfaeadf3d51012/ast_serialize-0.5.0-cp314-cp314t-win32.whl", hash = "sha256:842d1c004bb466c7df036f95fabef789570541922b10976b12f5592a69cf0b38", size = 1058668, upload-time = "2026-05-17T17:47:58.305Z" },
+ { url = "https://files.pythonhosted.org/packages/03/6f/c9d4d549295ed05111aeb8853232d1afd9d0a179fddb01eeffbb3a4a6842/ast_serialize-0.5.0-cp314-cp314t-win_amd64.whl", hash = "sha256:b0c06d760909b095cc466356dfccd05a1c7233a6ca191c020dca2c6a6f16c24c", size = 1101075, upload-time = "2026-05-17T17:48:00.35Z" },
+ { url = "https://files.pythonhosted.org/packages/d0/8e/d00c5ab30c58222e07d62956fca86c59d91b9ad32997e633c38b526623a3/ast_serialize-0.5.0-cp314-cp314t-win_arm64.whl", hash = "sha256:787baedb0262cc49e8ce37cc15c00ae818e46a165a3b36f5e21ed174998104cb", size = 1075347, upload-time = "2026-05-17T17:48:01.753Z" },
+ { url = "https://files.pythonhosted.org/packages/e0/9e/dc2530acb3a60dc6e46d65abf27d1d9f86721694757906a148d90a6860de/ast_serialize-0.5.0-cp39-abi3-macosx_10_12_x86_64.whl", hash = "sha256:0668aa9459cfa8c9c49ddd2163ebcf43088ba045ef7492af6fe22e0098303101", size = 1191380, upload-time = "2026-05-17T17:48:03.738Z" },
+ { url = "https://files.pythonhosted.org/packages/26/0a/bd3d18a582f273d6c843d16bb9e22e9e16365ff7991e92f18f798e9f1224/ast_serialize-0.5.0-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:bf683d6363edf2b39eed6b6d4fe22d34b6203867a67e27134d9e2a2680c4bc4a", size = 1183879, upload-time = "2026-05-17T17:48:05.463Z" },
+ { url = "https://files.pythonhosted.org/packages/40/ae/1f919100f8620887af58fcc381c61a1f218cdf89c6e155f87b213e61010a/ast_serialize-0.5.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9cc22cf0c9be65e71cf88fda130af60d61eb4a79370ad4cfe7900d48a4aa2211", size = 1244529, upload-time = "2026-05-17T17:48:07.008Z" },
+ { url = "https://files.pythonhosted.org/packages/c6/ca/6376559dcce707cdbc1d0d9a13c8d3baaaa501e949ce0ebdc4230cd881aa/ast_serialize-0.5.0-cp39-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f66173891548c9f2726bf27957b41cabce12fa679dc6da505ddbde4d4b3b31cf", size = 1240560, upload-time = "2026-05-17T17:48:08.46Z" },
+ { url = "https://files.pythonhosted.org/packages/35/b2/a620e206b5aeb7efbf2710336df57d457cffbb3991076bbcc1147ef9abd4/ast_serialize-0.5.0-cp39-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e42d729ef2be96a14efbad355093284739e3670ece3e534f82cc8832790911d9", size = 1451172, upload-time = "2026-05-17T17:48:09.922Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/e0/4ad5c04c24a40481b2935ce9a0ccdb6023dc8b667167d06ae530cc3512f2/ast_serialize-0.5.0-cp39-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b725026bafa801dbd7310eb13a75f0a2e370e7e51b2cb225f9d21fcfadf919ee", size = 1265072, upload-time = "2026-05-17T17:48:11.469Z" },
+ { url = "https://files.pythonhosted.org/packages/b2/71/4d1d479aa56d0101c40e17720c3d6ac2af7269ea0487a80b18e7bfd1a5b7/ast_serialize-0.5.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b54f60c1d78767a53b67eaa663f0dfac3afe606aa07f1301572f588b73d64809", size = 1270488, upload-time = "2026-05-17T17:48:13.575Z" },
+ { url = "https://files.pythonhosted.org/packages/6d/4f/0de1bbe06f6edef9fde4ed12ca8e7b3ec7e6e2bd4e672c5af487f7957665/ast_serialize-0.5.0-cp39-abi3-manylinux_2_31_riscv64.whl", hash = "sha256:27d51654fc240a1e87e742d353d98eb45b75f62f129086b3596ab53df2ac2a43", size = 1260702, upload-time = "2026-05-17T17:48:15.141Z" },
+ { url = "https://files.pythonhosted.org/packages/75/61/e00872439cfdddcc3c1b6cdaa6e5d904ba8e26a18807c67c4e14409d0ca8/ast_serialize-0.5.0-cp39-abi3-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2782c36237c46dd1674542f2109740ea5ea485a169bf1431939ada0434e17934", size = 1311182, upload-time = "2026-05-17T17:48:16.779Z" },
+ { url = "https://files.pythonhosted.org/packages/76/8e/699a5b955f7926956c95e9e1d74132acad73c2fe7a426f94da89123c20aa/ast_serialize-0.5.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:1943db345233cc7194a470f13afa9c59772c0b123dea0c9414c4d4ca54369759", size = 1421410, upload-time = "2026-05-17T17:48:18.527Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/ae/d5b7626874478997adc7a29ab28accf21e596fb590c944290401dfd0b29e/ast_serialize-0.5.0-cp39-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:df1c00022cbbcb064bfaa505aa9c9295362443ce5dacb459d1331d3da353f887", size = 1516587, upload-time = "2026-05-17T17:48:20.133Z" },
+ { url = "https://files.pythonhosted.org/packages/0c/ce/b59e02a82d9c4244d64cde502e0b00e83e38816abe19155ceb5437402c7f/ast_serialize-0.5.0-cp39-abi3-musllinux_1_2_i686.whl", hash = "sha256:cae65289fc456fde04af979a2be09302ef5d8ab92ef23e596d6746dc267ada27", size = 1515171, upload-time = "2026-05-17T17:48:21.921Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/38/d8d90042747d05aa08d4efcf1c99035a5f670a6bf4c214d31644392afbca/ast_serialize-0.5.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:239a4c354e8d676e9d94631d1d4a64edc6b266f86ff3a5a80aedd344f342c01d", size = 1464668, upload-time = "2026-05-17T17:48:23.544Z" },
+ { url = "https://files.pythonhosted.org/packages/dd/51/5b840c4df7334104cecffa28f23904fe81ca89ca223d2450e288de39fd3c/ast_serialize-0.5.0-cp39-abi3-win32.whl", hash = "sha256:143a4ef63285a075871908fda3672dc21864b83a8ec3ee12304aa3e4c5387b9a", size = 1068311, upload-time = "2026-05-17T17:48:25.027Z" },
+ { url = "https://files.pythonhosted.org/packages/41/11/ca5672c7d491825bc4cd6702dea106a6b60d928707712ec257c7833ae476/ast_serialize-0.5.0-cp39-abi3-win_amd64.whl", hash = "sha256:cf25572c526add400f26a4750dc6ce0c3bb93fc1f75e7ae0cad4ce4f2cd5c590", size = 1108931, upload-time = "2026-05-17T17:48:26.591Z" },
+ { url = "https://files.pythonhosted.org/packages/45/19/cc8bd127d28a43da249aa955cfd164cf8fd534e79e42cea96c4854d72fd0/ast_serialize-0.5.0-cp39-abi3-win_arm64.whl", hash = "sha256:92a31c9c20d25a076edaeec76b128a3535d74a24f340b9a8a7e96c9b86dc9642", size = 1081181, upload-time = "2026-05-17T17:48:28.122Z" },
+]
+
+[[package]]
+name = "asttokens"
+version = "3.0.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/be/a5/8e3f9b6771b0b408517c82d97aed8f2036509bc247d46114925e32fe33f0/asttokens-3.0.1.tar.gz", hash = "sha256:71a4ee5de0bde6a31d64f6b13f2293ac190344478f081c3d1bccfcf5eacb0cb7", size = 62308, upload-time = "2025-11-15T16:43:48.578Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/d2/39/e7eaf1799466a4aef85b6a4fe7bd175ad2b1c6345066aa33f1f58d4b18d0/asttokens-3.0.1-py3-none-any.whl", hash = "sha256:15a3ebc0f43c2d0a50eeafea25e19046c68398e487b9f1f5b517f7c0f40f976a", size = 27047, upload-time = "2025-11-15T16:43:16.109Z" },
+]
+
+[[package]]
+name = "attrs"
+version = "26.1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/9a/8e/82a0fe20a541c03148528be8cac2408564a6c9a0cc7e9171802bc1d26985/attrs-26.1.0.tar.gz", hash = "sha256:d03ceb89cb322a8fd706d4fb91940737b6642aa36998fe130a9bc96c985eff32", size = 952055, upload-time = "2026-03-19T14:22:25.026Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/64/b4/17d4b0b2a2dc85a6df63d1157e028ed19f90d4cd97c36717afef2bc2f395/attrs-26.1.0-py3-none-any.whl", hash = "sha256:c647aa4a12dfbad9333ca4e71fe62ddc36f4e63b2d260a37a8b83d2f043ac309", size = 67548, upload-time = "2026-03-19T14:22:23.645Z" },
+]
+
+[[package]]
+name = "babel"
+version = "2.18.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/7d/b2/51899539b6ceeeb420d40ed3cd4b7a40519404f9baf3d4ac99dc413a834b/babel-2.18.0.tar.gz", hash = "sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d", size = 9959554, upload-time = "2026-02-01T12:30:56.078Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/77/f5/21d2de20e8b8b0408f0681956ca2c69f1320a3848ac50e6e7f39c6159675/babel-2.18.0-py3-none-any.whl", hash = "sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35", size = 10196845, upload-time = "2026-02-01T12:30:53.445Z" },
+]
+
+[[package]]
+name = "beautifulsoup4"
+version = "4.15.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "soupsieve" },
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/43/65/318323f98dbee45d42dff61d8f047181bc6f2268a9068cfad035a46be5af/beautifulsoup4-4.15.0.tar.gz", hash = "sha256:288e3ca7d54b06f2ac191970bc275c1939cb46d450b255bf6718b04aa37ab4f7", size = 632571, upload-time = "2026-06-07T16:44:20.453Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/88/c6/92fcd42f1ba33e1184263f25bfabf3d27c383410470f169e4b8163bf9c17/beautifulsoup4-4.15.0-py3-none-any.whl", hash = "sha256:d6f88de62e1d4e38ecb1077eb9724cd0eff29d2a08ca16a401e9b9e93f117cf9", size = 109924, upload-time = "2026-06-07T16:44:21.566Z" },
+]
+
+[[package]]
+name = "cbor2"
+version = "6.1.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/3a/6f/07b4af8da8bd27f640362b1ac8271d80895407f2ede0c2bcc9433c06e1ca/cbor2-6.1.3.tar.gz", hash = "sha256:8d70680acb55c04ea5b5ad86da094f9612b53d5a8a65d0f5b3aafc3ce917ecbb", size = 89503, upload-time = "2026-07-04T10:36:48.793Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/31/16/cff14259c3d19a7f0ae88b6996fe4c85f6ff1764dad889ac8a39e843e39c/cbor2-6.1.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:3d939f55097c21e032f5a2d67592fcc57298986281f219356e2f519e4466f4ea", size = 412779, upload-time = "2026-07-04T10:36:04.975Z" },
+ { url = "https://files.pythonhosted.org/packages/50/6c/f3641d19b7b85a63cb2756c10164131489c2cb46b379ec51ae22283fefb9/cbor2-6.1.3-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:b025009478d644dab407164fd60e3ef4381af284f5af6966df94c663756d949e", size = 457781, upload-time = "2026-07-04T10:36:06.349Z" },
+ { url = "https://files.pythonhosted.org/packages/55/85/0c55a66f3037056bfb8e1c7184168085fdea67ae5830404498bcf466233b/cbor2-6.1.3-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:2226d32e102e375737656ad5d141ad8c6ae3e705e04e263f24756f0eb379c6c1", size = 468373, upload-time = "2026-07-04T10:36:07.769Z" },
+ { url = "https://files.pythonhosted.org/packages/46/74/40f7db3e0d880560193916a5c9b744fcf299558bed7113f77c28237c7c29/cbor2-6.1.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e61d465244d66ffed36492eef3b44d43795d76a2bba0663a2f15c186af7f7513", size = 523844, upload-time = "2026-07-04T10:36:09.404Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/a1/b5e07d6a08441c3a552fe2ae48ccb7e9dfc5065b9f6a3bae9879b4f0fbc0/cbor2-6.1.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:87fe7be8fab6ec4796aa127c1a52e09e79dbafd2aa31caf809cf04b8080a5975", size = 536238, upload-time = "2026-07-04T10:36:10.914Z" },
+ { url = "https://files.pythonhosted.org/packages/c9/99/e166be0fd74bf3a91f5a0d103e34883efbc438d970f72cc8200e274787e5/cbor2-6.1.3-cp312-cp312-win32.whl", hash = "sha256:da25d345f01e6a40b2e5c57ef96b4dcff7be69394fb62f0f70e07f437f2376a9", size = 279858, upload-time = "2026-07-04T10:36:12.247Z" },
+ { url = "https://files.pythonhosted.org/packages/d2/1b/90b4a121e40aba189c55a5822dd3c698eaf487e1d4a780ab18c804a5ef1c/cbor2-6.1.3-cp312-cp312-win_amd64.whl", hash = "sha256:d5514f693db6fa6f433b4096e9b604e6a7bf151c9ef1d2db86d0858e4c5e768f", size = 300929, upload-time = "2026-07-04T10:36:13.564Z" },
+ { url = "https://files.pythonhosted.org/packages/19/db/52c58a8d33464927389dde8103997b3fa51b081ce29b347ac2cc4fd0dfbf/cbor2-6.1.3-cp312-cp312-win_arm64.whl", hash = "sha256:3d43183d7beb3d3cd198d69b31bd2ee487ed704a1150c75cb0a66d6ad63d8c1a", size = 290908, upload-time = "2026-07-04T10:36:14.857Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/8c/5024d623dcf3f2057ec8c991f584b939ba5f9025a5ce8c31f6fac067137a/cbor2-6.1.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:21c74b8ab67977c8b87b727247eeb730145b0068ad6d47f71e9f80f6b48c65f8", size = 412334, upload-time = "2026-07-04T10:36:16.299Z" },
+ { url = "https://files.pythonhosted.org/packages/61/f3/e50654203c3b746166a96bea680eb6463b20c2c160cc14dfbe43f215ef6c/cbor2-6.1.3-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:f291a0ae4c1ed96eadb0afa9752568c7424f7d6fa818676d5e33005fcd22ddd9", size = 457125, upload-time = "2026-07-04T10:36:17.684Z" },
+ { url = "https://files.pythonhosted.org/packages/44/86/6ef007f0d4f7afba90a80cb1657984de542e7474d2afaa7e920ac9860df3/cbor2-6.1.3-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:dc8e44c7bf172687195dcd428157885bc00ea06efc0ea30fb371163b92bef733", size = 467651, upload-time = "2026-07-04T10:36:19.007Z" },
+ { url = "https://files.pythonhosted.org/packages/b2/f4/b5aa27813c02f37e03eb86bd908163562edd6fc7f99665bc7bbb25ef5e6c/cbor2-6.1.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:8ccf4d263983d830dd429d2b01f27be58ac02ba7c790c45d861f767eb63963e5", size = 523296, upload-time = "2026-07-04T10:36:20.504Z" },
+ { url = "https://files.pythonhosted.org/packages/5b/46/e17b2bce2efdc26bfa045b4f6168f02923ac5f0e79732a1b3c42ce9ca9de/cbor2-6.1.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8719a7a2a2a82168844533389957b8f617a139f5f40e4d0ad7ed905fd3abebd1", size = 535537, upload-time = "2026-07-04T10:36:21.761Z" },
+ { url = "https://files.pythonhosted.org/packages/54/bc/add350acf37f367ae429f997f2e047b042f2d5ef9ca62461c923fbb0f3c3/cbor2-6.1.3-cp313-cp313-win32.whl", hash = "sha256:c73b54ce09dd8d522f3c1540426e36172ba0f34abf3d89eb93909a5e14590003", size = 279233, upload-time = "2026-07-04T10:36:23.071Z" },
+ { url = "https://files.pythonhosted.org/packages/e8/6f/1bbfce3b3131e4e03e8a86966a38ff92ebb72215fcf36aeecea1547f3e4b/cbor2-6.1.3-cp313-cp313-win_amd64.whl", hash = "sha256:b77df56c462c10eb3444db8ef78d8c3e71d9ef8d021ea92e97c1f9e3aa918690", size = 300585, upload-time = "2026-07-04T10:36:24.563Z" },
+ { url = "https://files.pythonhosted.org/packages/e7/65/3945702dd84b6e5b7800c9c7f1ada038d33d12d0042de10e38164cc03dfd/cbor2-6.1.3-cp313-cp313-win_arm64.whl", hash = "sha256:b144be2ab3e9584ee7b6359d2a92fee0a5bec1d00dbd34c215ccc2040ac0b2ab", size = 290357, upload-time = "2026-07-04T10:36:25.983Z" },
+ { url = "https://files.pythonhosted.org/packages/8f/40/04ad7d34182b27487a1824422b361b32d2607727ef20e563056eba62d12a/cbor2-6.1.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:3f3167ca9920b90db4ff72652db109dfd93b56ee0d583aca12f3a5d9a7019477", size = 414615, upload-time = "2026-07-04T10:36:27.299Z" },
+ { url = "https://files.pythonhosted.org/packages/8e/29/94238c61f90653a606535e7509a2af312089fe10dafcb4cf82d6905a7a1a/cbor2-6.1.3-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:48c677971e4b71e685491e1a267d9924dc205e7ddbe3f34fd2562f16c0f6bfca", size = 459084, upload-time = "2026-07-04T10:36:28.717Z" },
+ { url = "https://files.pythonhosted.org/packages/c8/cb/6bd33461e8be8ded7ebb0fa38994a63752aefae2b4fcd1b2cc71ee3c06f1/cbor2-6.1.3-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:ad4f3c6dfc6b83331eb04c6975efb2839ab65a3aa81502bc2b3f7945d4c4aa44", size = 469310, upload-time = "2026-07-04T10:36:30.136Z" },
+ { url = "https://files.pythonhosted.org/packages/8e/d1/94195bcd8fcc1030ecaf22a7a825faa08891b5bb2d3553e465e50fe115f5/cbor2-6.1.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:8bcd609eab4a39745123bfb28e73311abba3d14975a87f5906e0e8b910d918ac", size = 524287, upload-time = "2026-07-04T10:36:31.453Z" },
+ { url = "https://files.pythonhosted.org/packages/06/55/57178fbf2d1206af5299c688f9c917b83f30636694c8f932dc89c6652545/cbor2-6.1.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:672727ecb27d7fb3ca0bf8a58fc489d5374ab1fee680ed3d0348a11d9d3ca78f", size = 537031, upload-time = "2026-07-04T10:36:32.769Z" },
+ { url = "https://files.pythonhosted.org/packages/fb/01/beefe26258d66ce39a9b057b679bc67dba81698d5a63e0ada5b4752a5c87/cbor2-6.1.3-cp314-cp314-win32.whl", hash = "sha256:1413cef2aa7f478a38298cd3492a055e8e8e45d17fb53bbe103e79ca15c33f3c", size = 286256, upload-time = "2026-07-04T10:36:34.078Z" },
+ { url = "https://files.pythonhosted.org/packages/1b/2d/79eb513a2586a2053a6f18b33693beda65e2c766820d99676a306573fed5/cbor2-6.1.3-cp314-cp314-win_amd64.whl", hash = "sha256:59df264d4a508ba61daaa0bf3c2f92d63275509549a0875c1fa38176f651e4f8", size = 313744, upload-time = "2026-07-04T10:36:35.58Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/c7/ab9828e4efc26badf89f1397f866f3ef03ef65cdcd55518254b84bdaf86e/cbor2-6.1.3-cp314-cp314-win_arm64.whl", hash = "sha256:b62b5d80a0eb4305cd5f0217faa4d7747bd64fe0dff9b88415e7be3782f8249b", size = 304280, upload-time = "2026-07-04T10:36:36.865Z" },
+ { url = "https://files.pythonhosted.org/packages/33/cf/54a497ad1026833c1c92d482edbda0bdebb48314b51563d2fcea24ab89b4/cbor2-6.1.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:856ab525bfc599588b8d2a45babb7c3400c693ea0bb574d818467d997102fe24", size = 409638, upload-time = "2026-07-04T10:36:38.089Z" },
+ { url = "https://files.pythonhosted.org/packages/3b/03/efdecd0848b9c9e43242537f6dd8ac5d441a077d362e5e6954c7775b866a/cbor2-6.1.3-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:a076f35abf1dd0a4de6e2f7f4d4932abafc951a26275b6aa4a3b370c2fd3bbf4", size = 452193, upload-time = "2026-07-04T10:36:39.56Z" },
+ { url = "https://files.pythonhosted.org/packages/ef/bf/b5f43c75dc5f0ca3127919d3273d8671917932aa3e90767da63867e0f06f/cbor2-6.1.3-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:37ccab1d0bd3f57ff536a41e44165d0c99cb166ac6e5ffb8b93c42304b56e48d", size = 466614, upload-time = "2026-07-04T10:36:40.936Z" },
+ { url = "https://files.pythonhosted.org/packages/23/ee/e85b2ddd46b3b43e39a986704353b433efab0881234e1b4d824229fc2a75/cbor2-6.1.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:76d257ce797e651fa430b0269e8e8c43549c54ce8b0d12860569b3709bf1326f", size = 518503, upload-time = "2026-07-04T10:36:42.289Z" },
+ { url = "https://files.pythonhosted.org/packages/60/13/c740c0002f127dc3e9e87b61a5d1285dd3b71aa11d8e0f6a408fc1f36173/cbor2-6.1.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:4af35baadb66f7c9cbb3998eab469767c04641552416c1c69dd4d3d183797119", size = 534236, upload-time = "2026-07-04T10:36:43.641Z" },
+ { url = "https://files.pythonhosted.org/packages/73/cd/a57d97177f3777c96f3406efb0ad60794fdbf249d497ec24559bfd1d0328/cbor2-6.1.3-cp314-cp314t-win32.whl", hash = "sha256:61c92661665bccfed4ffa69d1fe10097f2c820d262f56a8cf909a5ebf9f6d8c6", size = 282446, upload-time = "2026-07-04T10:36:44.888Z" },
+ { url = "https://files.pythonhosted.org/packages/11/c8/dd54878589df22c863d526cb82e1bb20e953d40223436449a52481c49804/cbor2-6.1.3-cp314-cp314t-win_amd64.whl", hash = "sha256:dc6bcd030bf5043662b84b0ca0f0ca942491bf509105db30cedca6e2ce82d158", size = 310300, upload-time = "2026-07-04T10:36:46.212Z" },
+ { url = "https://files.pythonhosted.org/packages/05/d0/b0780a396d145e3356bfdc48578d021ade1818a6c7d7842a3d6bc6f16fbb/cbor2-6.1.3-cp314-cp314t-win_arm64.whl", hash = "sha256:da98a5e0ae9487bed497ac74e2c850b49975a3b7b5314b76c3843e2c83a6c8c4", size = 299391, upload-time = "2026-07-04T10:36:47.629Z" },
+]
+
+[[package]]
+name = "certifi"
+version = "2026.6.17"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/c9/c7/424b75da314c1045981bd9777432fad05a9e0c69daa4ed7e308bbaffe405/certifi-2026.6.17.tar.gz", hash = "sha256:024c88eeec92ca068db80f02b8b07c9cef7b9fe261d1d535abfd5abd6f6af432", size = 134594, upload-time = "2026-06-17T10:31:07.894Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ef/2f/c5464532e965badff2f4c4c1a3a83f5697f0d7c407ed0cda44aaa99bb451/certifi-2026.6.17-py3-none-any.whl", hash = "sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db", size = 133289, upload-time = "2026-06-17T10:31:06.348Z" },
+]
+
+[[package]]
+name = "cffi"
+version = "2.0.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "pycparser", marker = "implementation_name != 'PyPy'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/eb/56/b1ba7935a17738ae8453301356628e8147c79dbb825bcbc73dc7401f9846/cffi-2.0.0.tar.gz", hash = "sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529", size = 523588, upload-time = "2025-09-08T23:24:04.541Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ea/47/4f61023ea636104d4f16ab488e268b93008c3d0bb76893b1b31db1f96802/cffi-2.0.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d", size = 185271, upload-time = "2025-09-08T23:22:44.795Z" },
+ { url = "https://files.pythonhosted.org/packages/df/a2/781b623f57358e360d62cdd7a8c681f074a71d445418a776eef0aadb4ab4/cffi-2.0.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c", size = 181048, upload-time = "2025-09-08T23:22:45.938Z" },
+ { url = "https://files.pythonhosted.org/packages/ff/df/a4f0fbd47331ceeba3d37c2e51e9dfc9722498becbeec2bd8bc856c9538a/cffi-2.0.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe", size = 212529, upload-time = "2025-09-08T23:22:47.349Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/72/12b5f8d3865bf0f87cf1404d8c374e7487dcf097a1c91c436e72e6badd83/cffi-2.0.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062", size = 220097, upload-time = "2025-09-08T23:22:48.677Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/95/7a135d52a50dfa7c882ab0ac17e8dc11cec9d55d2c18dda414c051c5e69e/cffi-2.0.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e", size = 207983, upload-time = "2025-09-08T23:22:50.06Z" },
+ { url = "https://files.pythonhosted.org/packages/3a/c8/15cb9ada8895957ea171c62dc78ff3e99159ee7adb13c0123c001a2546c1/cffi-2.0.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037", size = 206519, upload-time = "2025-09-08T23:22:51.364Z" },
+ { url = "https://files.pythonhosted.org/packages/78/2d/7fa73dfa841b5ac06c7b8855cfc18622132e365f5b81d02230333ff26e9e/cffi-2.0.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba", size = 219572, upload-time = "2025-09-08T23:22:52.902Z" },
+ { url = "https://files.pythonhosted.org/packages/07/e0/267e57e387b4ca276b90f0434ff88b2c2241ad72b16d31836adddfd6031b/cffi-2.0.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94", size = 222963, upload-time = "2025-09-08T23:22:54.518Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/75/1f2747525e06f53efbd878f4d03bac5b859cbc11c633d0fb81432d98a795/cffi-2.0.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187", size = 221361, upload-time = "2025-09-08T23:22:55.867Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/2b/2b6435f76bfeb6bbf055596976da087377ede68df465419d192acf00c437/cffi-2.0.0-cp312-cp312-win32.whl", hash = "sha256:da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18", size = 172932, upload-time = "2025-09-08T23:22:57.188Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/ed/13bd4418627013bec4ed6e54283b1959cf6db888048c7cf4b4c3b5b36002/cffi-2.0.0-cp312-cp312-win_amd64.whl", hash = "sha256:da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5", size = 183557, upload-time = "2025-09-08T23:22:58.351Z" },
+ { url = "https://files.pythonhosted.org/packages/95/31/9f7f93ad2f8eff1dbc1c3656d7ca5bfd8fb52c9d786b4dcf19b2d02217fa/cffi-2.0.0-cp312-cp312-win_arm64.whl", hash = "sha256:4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6", size = 177762, upload-time = "2025-09-08T23:22:59.668Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/8d/a0a47a0c9e413a658623d014e91e74a50cdd2c423f7ccfd44086ef767f90/cffi-2.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb", size = 185230, upload-time = "2025-09-08T23:23:00.879Z" },
+ { url = "https://files.pythonhosted.org/packages/4a/d2/a6c0296814556c68ee32009d9c2ad4f85f2707cdecfd7727951ec228005d/cffi-2.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca", size = 181043, upload-time = "2025-09-08T23:23:02.231Z" },
+ { url = "https://files.pythonhosted.org/packages/b0/1e/d22cc63332bd59b06481ceaac49d6c507598642e2230f201649058a7e704/cffi-2.0.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b", size = 212446, upload-time = "2025-09-08T23:23:03.472Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/f5/a2c23eb03b61a0b8747f211eb716446c826ad66818ddc7810cc2cc19b3f2/cffi-2.0.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b", size = 220101, upload-time = "2025-09-08T23:23:04.792Z" },
+ { url = "https://files.pythonhosted.org/packages/f2/7f/e6647792fc5850d634695bc0e6ab4111ae88e89981d35ac269956605feba/cffi-2.0.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2", size = 207948, upload-time = "2025-09-08T23:23:06.127Z" },
+ { url = "https://files.pythonhosted.org/packages/cb/1e/a5a1bd6f1fb30f22573f76533de12a00bf274abcdc55c8edab639078abb6/cffi-2.0.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3", size = 206422, upload-time = "2025-09-08T23:23:07.753Z" },
+ { url = "https://files.pythonhosted.org/packages/98/df/0a1755e750013a2081e863e7cd37e0cdd02664372c754e5560099eb7aa44/cffi-2.0.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26", size = 219499, upload-time = "2025-09-08T23:23:09.648Z" },
+ { url = "https://files.pythonhosted.org/packages/50/e1/a969e687fcf9ea58e6e2a928ad5e2dd88cc12f6f0ab477e9971f2309b57c/cffi-2.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c", size = 222928, upload-time = "2025-09-08T23:23:10.928Z" },
+ { url = "https://files.pythonhosted.org/packages/36/54/0362578dd2c9e557a28ac77698ed67323ed5b9775ca9d3fe73fe191bb5d8/cffi-2.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b", size = 221302, upload-time = "2025-09-08T23:23:12.42Z" },
+ { url = "https://files.pythonhosted.org/packages/eb/6d/bf9bda840d5f1dfdbf0feca87fbdb64a918a69bca42cfa0ba7b137c48cb8/cffi-2.0.0-cp313-cp313-win32.whl", hash = "sha256:74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27", size = 172909, upload-time = "2025-09-08T23:23:14.32Z" },
+ { url = "https://files.pythonhosted.org/packages/37/18/6519e1ee6f5a1e579e04b9ddb6f1676c17368a7aba48299c3759bbc3c8b3/cffi-2.0.0-cp313-cp313-win_amd64.whl", hash = "sha256:19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75", size = 183402, upload-time = "2025-09-08T23:23:15.535Z" },
+ { url = "https://files.pythonhosted.org/packages/cb/0e/02ceeec9a7d6ee63bb596121c2c8e9b3a9e150936f4fbef6ca1943e6137c/cffi-2.0.0-cp313-cp313-win_arm64.whl", hash = "sha256:256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91", size = 177780, upload-time = "2025-09-08T23:23:16.761Z" },
+ { url = "https://files.pythonhosted.org/packages/92/c4/3ce07396253a83250ee98564f8d7e9789fab8e58858f35d07a9a2c78de9f/cffi-2.0.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5", size = 185320, upload-time = "2025-09-08T23:23:18.087Z" },
+ { url = "https://files.pythonhosted.org/packages/59/dd/27e9fa567a23931c838c6b02d0764611c62290062a6d4e8ff7863daf9730/cffi-2.0.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13", size = 181487, upload-time = "2025-09-08T23:23:19.622Z" },
+ { url = "https://files.pythonhosted.org/packages/d6/43/0e822876f87ea8a4ef95442c3d766a06a51fc5298823f884ef87aaad168c/cffi-2.0.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b", size = 220049, upload-time = "2025-09-08T23:23:20.853Z" },
+ { url = "https://files.pythonhosted.org/packages/b4/89/76799151d9c2d2d1ead63c2429da9ea9d7aac304603de0c6e8764e6e8e70/cffi-2.0.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c", size = 207793, upload-time = "2025-09-08T23:23:22.08Z" },
+ { url = "https://files.pythonhosted.org/packages/bb/dd/3465b14bb9e24ee24cb88c9e3730f6de63111fffe513492bf8c808a3547e/cffi-2.0.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef", size = 206300, upload-time = "2025-09-08T23:23:23.314Z" },
+ { url = "https://files.pythonhosted.org/packages/47/d9/d83e293854571c877a92da46fdec39158f8d7e68da75bf73581225d28e90/cffi-2.0.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775", size = 219244, upload-time = "2025-09-08T23:23:24.541Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/0f/1f177e3683aead2bb00f7679a16451d302c436b5cbf2505f0ea8146ef59e/cffi-2.0.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205", size = 222828, upload-time = "2025-09-08T23:23:26.143Z" },
+ { url = "https://files.pythonhosted.org/packages/c6/0f/cafacebd4b040e3119dcb32fed8bdef8dfe94da653155f9d0b9dc660166e/cffi-2.0.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1", size = 220926, upload-time = "2025-09-08T23:23:27.873Z" },
+ { url = "https://files.pythonhosted.org/packages/3e/aa/df335faa45b395396fcbc03de2dfcab242cd61a9900e914fe682a59170b1/cffi-2.0.0-cp314-cp314-win32.whl", hash = "sha256:087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f", size = 175328, upload-time = "2025-09-08T23:23:44.61Z" },
+ { url = "https://files.pythonhosted.org/packages/bb/92/882c2d30831744296ce713f0feb4c1cd30f346ef747b530b5318715cc367/cffi-2.0.0-cp314-cp314-win_amd64.whl", hash = "sha256:203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25", size = 185650, upload-time = "2025-09-08T23:23:45.848Z" },
+ { url = "https://files.pythonhosted.org/packages/9f/2c/98ece204b9d35a7366b5b2c6539c350313ca13932143e79dc133ba757104/cffi-2.0.0-cp314-cp314-win_arm64.whl", hash = "sha256:dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad", size = 180687, upload-time = "2025-09-08T23:23:47.105Z" },
+ { url = "https://files.pythonhosted.org/packages/3e/61/c768e4d548bfa607abcda77423448df8c471f25dbe64fb2ef6d555eae006/cffi-2.0.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9", size = 188773, upload-time = "2025-09-08T23:23:29.347Z" },
+ { url = "https://files.pythonhosted.org/packages/2c/ea/5f76bce7cf6fcd0ab1a1058b5af899bfbef198bea4d5686da88471ea0336/cffi-2.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d", size = 185013, upload-time = "2025-09-08T23:23:30.63Z" },
+ { url = "https://files.pythonhosted.org/packages/be/b4/c56878d0d1755cf9caa54ba71e5d049479c52f9e4afc230f06822162ab2f/cffi-2.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c", size = 221593, upload-time = "2025-09-08T23:23:31.91Z" },
+ { url = "https://files.pythonhosted.org/packages/e0/0d/eb704606dfe8033e7128df5e90fee946bbcb64a04fcdaa97321309004000/cffi-2.0.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8", size = 209354, upload-time = "2025-09-08T23:23:33.214Z" },
+ { url = "https://files.pythonhosted.org/packages/d8/19/3c435d727b368ca475fb8742ab97c9cb13a0de600ce86f62eab7fa3eea60/cffi-2.0.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc", size = 208480, upload-time = "2025-09-08T23:23:34.495Z" },
+ { url = "https://files.pythonhosted.org/packages/d0/44/681604464ed9541673e486521497406fadcc15b5217c3e326b061696899a/cffi-2.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592", size = 221584, upload-time = "2025-09-08T23:23:36.096Z" },
+ { url = "https://files.pythonhosted.org/packages/25/8e/342a504ff018a2825d395d44d63a767dd8ebc927ebda557fecdaca3ac33a/cffi-2.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512", size = 224443, upload-time = "2025-09-08T23:23:37.328Z" },
+ { url = "https://files.pythonhosted.org/packages/e1/5e/b666bacbbc60fbf415ba9988324a132c9a7a0448a9a8f125074671c0f2c3/cffi-2.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4", size = 223437, upload-time = "2025-09-08T23:23:38.945Z" },
+ { url = "https://files.pythonhosted.org/packages/a0/1d/ec1a60bd1a10daa292d3cd6bb0b359a81607154fb8165f3ec95fe003b85c/cffi-2.0.0-cp314-cp314t-win32.whl", hash = "sha256:1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e", size = 180487, upload-time = "2025-09-08T23:23:40.423Z" },
+ { url = "https://files.pythonhosted.org/packages/bf/41/4c1168c74fac325c0c8156f04b6749c8b6a8f405bbf91413ba088359f60d/cffi-2.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6", size = 191726, upload-time = "2025-09-08T23:23:41.742Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/3a/dbeec9d1ee0844c679f6bb5d6ad4e9f198b1224f4e7a32825f47f6192b0c/cffi-2.0.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9", size = 184195, upload-time = "2025-09-08T23:23:43.004Z" },
+]
+
+[[package]]
+name = "charset-normalizer"
+version = "3.4.7"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/e7/a1/67fe25fac3c7642725500a3f6cfe5821ad557c3abb11c9d20d12c7008d3e/charset_normalizer-3.4.7.tar.gz", hash = "sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5", size = 144271, upload-time = "2026-04-02T09:28:39.342Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/0c/eb/4fc8d0a7110eb5fc9cc161723a34a8a6c200ce3b4fbf681bc86feee22308/charset_normalizer-3.4.7-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46", size = 311328, upload-time = "2026-04-02T09:26:24.331Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/e3/0fadc706008ac9d7b9b5be6dc767c05f9d3e5df51744ce4cc9605de7b9f4/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2", size = 208061, upload-time = "2026-04-02T09:26:25.568Z" },
+ { url = "https://files.pythonhosted.org/packages/42/f0/3dd1045c47f4a4604df85ec18ad093912ae1344ac706993aff91d38773a2/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b", size = 229031, upload-time = "2026-04-02T09:26:26.865Z" },
+ { url = "https://files.pythonhosted.org/packages/dc/67/675a46eb016118a2fbde5a277a5d15f4f69d5f3f5f338e5ee2f8948fcf43/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a", size = 225239, upload-time = "2026-04-02T09:26:28.044Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/f8/d0118a2f5f23b02cd166fa385c60f9b0d4f9194f574e2b31cef350ad7223/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116", size = 216589, upload-time = "2026-04-02T09:26:29.239Z" },
+ { url = "https://files.pythonhosted.org/packages/b1/f1/6d2b0b261b6c4ceef0fcb0d17a01cc5bc53586c2d4796fa04b5c540bc13d/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb", size = 202733, upload-time = "2026-04-02T09:26:30.5Z" },
+ { url = "https://files.pythonhosted.org/packages/6f/c0/7b1f943f7e87cc3db9626ba17807d042c38645f0a1d4415c7a14afb5591f/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1", size = 212652, upload-time = "2026-04-02T09:26:31.709Z" },
+ { url = "https://files.pythonhosted.org/packages/38/dd/5a9ab159fe45c6e72079398f277b7d2b523e7f716acc489726115a910097/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15", size = 211229, upload-time = "2026-04-02T09:26:33.282Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/ff/531a1cad5ca855d1c1a8b69cb71abfd6d85c0291580146fda7c82857caa1/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5", size = 203552, upload-time = "2026-04-02T09:26:34.845Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/4c/a5fb52d528a8ca41f7598cb619409ece30a169fbdf9cdce592e53b46c3a6/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d", size = 230806, upload-time = "2026-04-02T09:26:36.152Z" },
+ { url = "https://files.pythonhosted.org/packages/59/7a/071feed8124111a32b316b33ae4de83d36923039ef8cf48120266844285b/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7", size = 212316, upload-time = "2026-04-02T09:26:37.672Z" },
+ { url = "https://files.pythonhosted.org/packages/fd/35/f7dba3994312d7ba508e041eaac39a36b120f32d4c8662b8814dab876431/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464", size = 227274, upload-time = "2026-04-02T09:26:38.93Z" },
+ { url = "https://files.pythonhosted.org/packages/8a/2d/a572df5c9204ab7688ec1edc895a73ebded3b023bb07364710b05dd1c9be/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49", size = 218468, upload-time = "2026-04-02T09:26:40.17Z" },
+ { url = "https://files.pythonhosted.org/packages/86/eb/890922a8b03a568ca2f336c36585a4713c55d4d67bf0f0c78924be6315ca/charset_normalizer-3.4.7-cp312-cp312-win32.whl", hash = "sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c", size = 148460, upload-time = "2026-04-02T09:26:41.416Z" },
+ { url = "https://files.pythonhosted.org/packages/35/d9/0e7dffa06c5ab081f75b1b786f0aefc88365825dfcd0ac544bdb7b2b6853/charset_normalizer-3.4.7-cp312-cp312-win_amd64.whl", hash = "sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6", size = 159330, upload-time = "2026-04-02T09:26:42.554Z" },
+ { url = "https://files.pythonhosted.org/packages/9e/5d/481bcc2a7c88ea6b0878c299547843b2521ccbc40980cb406267088bc701/charset_normalizer-3.4.7-cp312-cp312-win_arm64.whl", hash = "sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d", size = 147828, upload-time = "2026-04-02T09:26:44.075Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/3b/66777e39d3ae1ddc77ee606be4ec6d8cbd4c801f65e5a1b6f2b11b8346dd/charset_normalizer-3.4.7-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:f496c9c3cc02230093d8330875c4c3cdfc3b73612a5fd921c65d39cbcef08063", size = 309627, upload-time = "2026-04-02T09:26:45.198Z" },
+ { url = "https://files.pythonhosted.org/packages/2e/4e/b7f84e617b4854ade48a1b7915c8ccfadeba444d2a18c291f696e37f0d3b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c", size = 207008, upload-time = "2026-04-02T09:26:46.824Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/bb/ec73c0257c9e11b268f018f068f5d00aa0ef8c8b09f7753ebd5f2880e248/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a277ab8928b9f299723bc1a2dabb1265911b1a76341f90a510368ca44ad9ab66", size = 228303, upload-time = "2026-04-02T09:26:48.397Z" },
+ { url = "https://files.pythonhosted.org/packages/85/fb/32d1f5033484494619f701e719429c69b766bfc4dbc61aa9e9c8c166528b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18", size = 224282, upload-time = "2026-04-02T09:26:49.684Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/07/330e3a0dda4c404d6da83b327270906e9654a24f6c546dc886a0eb0ffb23/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e044c39e41b92c845bc815e5ae4230804e8e7bc29e399b0437d64222d92809dd", size = 215595, upload-time = "2026-04-02T09:26:50.915Z" },
+ { url = "https://files.pythonhosted.org/packages/e3/7c/fc890655786e423f02556e0216d4b8c6bcb6bdfa890160dc66bf52dee468/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:f495a1652cf3fbab2eb0639776dad966c2fb874d79d87ca07f9d5f059b8bd215", size = 201986, upload-time = "2026-04-02T09:26:52.197Z" },
+ { url = "https://files.pythonhosted.org/packages/d8/97/bfb18b3db2aed3b90cf54dc292ad79fdd5ad65c4eae454099475cbeadd0d/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e712b419df8ba5e42b226c510472b37bd57b38e897d3eca5e8cfd410a29fa859", size = 211711, upload-time = "2026-04-02T09:26:53.49Z" },
+ { url = "https://files.pythonhosted.org/packages/6f/a5/a581c13798546a7fd557c82614a5c65a13df2157e9ad6373166d2a3e645d/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8", size = 210036, upload-time = "2026-04-02T09:26:54.975Z" },
+ { url = "https://files.pythonhosted.org/packages/8c/bf/b3ab5bcb478e4193d517644b0fb2bf5497fbceeaa7a1bc0f4d5b50953861/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5", size = 202998, upload-time = "2026-04-02T09:26:56.303Z" },
+ { url = "https://files.pythonhosted.org/packages/e7/4e/23efd79b65d314fa320ec6017b4b5834d5c12a58ba4610aa353af2e2f577/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:f59099f9b66f0d7145115e6f80dd8b1d847176df89b234a5a6b3f00437aa0832", size = 230056, upload-time = "2026-04-02T09:26:57.554Z" },
+ { url = "https://files.pythonhosted.org/packages/b9/9f/1e1941bc3f0e01df116e68dc37a55c4d249df5e6fa77f008841aef68264f/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:f59ad4c0e8f6bba240a9bb85504faa1ab438237199d4cce5f622761507b8f6a6", size = 211537, upload-time = "2026-04-02T09:26:58.843Z" },
+ { url = "https://files.pythonhosted.org/packages/80/0f/088cbb3020d44428964a6c97fe1edfb1b9550396bf6d278330281e8b709c/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48", size = 226176, upload-time = "2026-04-02T09:27:00.437Z" },
+ { url = "https://files.pythonhosted.org/packages/6a/9f/130394f9bbe06f4f63e22641d32fc9b202b7e251c9aef4db044324dac493/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a", size = 217723, upload-time = "2026-04-02T09:27:02.021Z" },
+ { url = "https://files.pythonhosted.org/packages/73/55/c469897448a06e49f8fa03f6caae97074fde823f432a98f979cc42b90e69/charset_normalizer-3.4.7-cp313-cp313-win32.whl", hash = "sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e", size = 148085, upload-time = "2026-04-02T09:27:03.192Z" },
+ { url = "https://files.pythonhosted.org/packages/5d/78/1b74c5bbb3f99b77a1715c91b3e0b5bdb6fe302d95ace4f5b1bec37b0167/charset_normalizer-3.4.7-cp313-cp313-win_amd64.whl", hash = "sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110", size = 158819, upload-time = "2026-04-02T09:27:04.454Z" },
+ { url = "https://files.pythonhosted.org/packages/68/86/46bd42279d323deb8687c4a5a811fd548cb7d1de10cf6535d099877a9a9f/charset_normalizer-3.4.7-cp313-cp313-win_arm64.whl", hash = "sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b", size = 147915, upload-time = "2026-04-02T09:27:05.971Z" },
+ { url = "https://files.pythonhosted.org/packages/97/c8/c67cb8c70e19ef1960b97b22ed2a1567711de46c4ddf19799923adc836c2/charset_normalizer-3.4.7-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:c36c333c39be2dbca264d7803333c896ab8fa7d4d6f0ab7edb7dfd7aea6e98c0", size = 309234, upload-time = "2026-04-02T09:27:07.194Z" },
+ { url = "https://files.pythonhosted.org/packages/99/85/c091fdee33f20de70d6c8b522743b6f831a2f1cd3ff86de4c6a827c48a76/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a", size = 208042, upload-time = "2026-04-02T09:27:08.749Z" },
+ { url = "https://files.pythonhosted.org/packages/87/1c/ab2ce611b984d2fd5d86a5a8a19c1ae26acac6bad967da4967562c75114d/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b", size = 228706, upload-time = "2026-04-02T09:27:09.951Z" },
+ { url = "https://files.pythonhosted.org/packages/a8/29/2b1d2cb00bf085f59d29eb773ce58ec2d325430f8c216804a0a5cd83cbca/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41", size = 224727, upload-time = "2026-04-02T09:27:11.175Z" },
+ { url = "https://files.pythonhosted.org/packages/47/5c/032c2d5a07fe4d4855fea851209cca2b6f03ebeb6d4e3afdb3358386a684/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bd6c2a1c7573c64738d716488d2cdd3c00e340e4835707d8fdb8dc1a66ef164e", size = 215882, upload-time = "2026-04-02T09:27:12.446Z" },
+ { url = "https://files.pythonhosted.org/packages/2c/c2/356065d5a8b78ed04499cae5f339f091946a6a74f91e03476c33f0ab7100/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:c45e9440fb78f8ddabcf714b68f936737a121355bf59f3907f4e17721b9d1aae", size = 200860, upload-time = "2026-04-02T09:27:13.721Z" },
+ { url = "https://files.pythonhosted.org/packages/0c/cd/a32a84217ced5039f53b29f460962abb2d4420def55afabe45b1c3c7483d/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18", size = 211564, upload-time = "2026-04-02T09:27:15.272Z" },
+ { url = "https://files.pythonhosted.org/packages/44/86/58e6f13ce26cc3b8f4a36b94a0f22ae2f00a72534520f4ae6857c4b81f89/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:e8ac484bf18ce6975760921bb6148041faa8fef0547200386ea0b52b5d27bf7b", size = 211276, upload-time = "2026-04-02T09:27:16.834Z" },
+ { url = "https://files.pythonhosted.org/packages/8f/fe/d17c32dc72e17e155e06883efa84514ca375f8a528ba2546bee73fc4df81/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:a5fe03b42827c13cdccd08e6c0247b6a6d4b5e3cdc53fd1749f5896adcdc2356", size = 201238, upload-time = "2026-04-02T09:27:18.229Z" },
+ { url = "https://files.pythonhosted.org/packages/6a/29/f33daa50b06525a237451cdb6c69da366c381a3dadcd833fa5676bc468b3/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab", size = 230189, upload-time = "2026-04-02T09:27:19.445Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/6e/52c84015394a6a0bdcd435210a7e944c5f94ea1055f5cc5d56c5fe368e7b/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:e74327fb75de8986940def6e8dee4f127cc9752bee7355bb323cc5b2659b6d46", size = 211352, upload-time = "2026-04-02T09:27:20.79Z" },
+ { url = "https://files.pythonhosted.org/packages/8c/d7/4353be581b373033fb9198bf1da3cf8f09c1082561e8e922aa7b39bf9fe8/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:d6038d37043bced98a66e68d3aa2b6a35505dc01328cd65217cefe82f25def44", size = 227024, upload-time = "2026-04-02T09:27:22.063Z" },
+ { url = "https://files.pythonhosted.org/packages/30/45/99d18aa925bd1740098ccd3060e238e21115fffbfdcb8f3ece837d0ace6c/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72", size = 217869, upload-time = "2026-04-02T09:27:23.486Z" },
+ { url = "https://files.pythonhosted.org/packages/5c/05/5ee478aa53f4bb7996482153d4bfe1b89e0f087f0ab6b294fcf92d595873/charset_normalizer-3.4.7-cp314-cp314-win32.whl", hash = "sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10", size = 148541, upload-time = "2026-04-02T09:27:25.146Z" },
+ { url = "https://files.pythonhosted.org/packages/48/77/72dcb0921b2ce86420b2d79d454c7022bf5be40202a2a07906b9f2a35c97/charset_normalizer-3.4.7-cp314-cp314-win_amd64.whl", hash = "sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f", size = 159634, upload-time = "2026-04-02T09:27:26.642Z" },
+ { url = "https://files.pythonhosted.org/packages/c6/a3/c2369911cd72f02386e4e340770f6e158c7980267da16af8f668217abaa0/charset_normalizer-3.4.7-cp314-cp314-win_arm64.whl", hash = "sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246", size = 148384, upload-time = "2026-04-02T09:27:28.271Z" },
+ { url = "https://files.pythonhosted.org/packages/94/09/7e8a7f73d24dba1f0035fbbf014d2c36828fc1bf9c88f84093e57d315935/charset_normalizer-3.4.7-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:effc3f449787117233702311a1b7d8f59cba9ced946ba727bdc329ec69028e24", size = 330133, upload-time = "2026-04-02T09:27:29.474Z" },
+ { url = "https://files.pythonhosted.org/packages/8d/da/96975ddb11f8e977f706f45cddd8540fd8242f71ecdb5d18a80723dcf62c/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbccdc05410c9ee21bbf16a35f4c1d16123dcdeb8a1d38f33654fa21d0234f79", size = 216257, upload-time = "2026-04-02T09:27:30.793Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/e8/1d63bf8ef2d388e95c64b2098f45f84758f6d102a087552da1485912637b/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960", size = 234851, upload-time = "2026-04-02T09:27:32.44Z" },
+ { url = "https://files.pythonhosted.org/packages/9b/40/e5ff04233e70da2681fa43969ad6f66ca5611d7e669be0246c4c7aaf6dc8/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a89c23ef8d2c6b27fd200a42aa4ac72786e7c60d40efdc76e6011260b6e949c4", size = 233393, upload-time = "2026-04-02T09:27:34.03Z" },
+ { url = "https://files.pythonhosted.org/packages/be/c1/06c6c49d5a5450f76899992f1ee40b41d076aee9279b49cf9974d2f313d5/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e", size = 223251, upload-time = "2026-04-02T09:27:35.369Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/9f/f2ff16fb050946169e3e1f82134d107e5d4ae72647ec8a1b1446c148480f/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:a180c5e59792af262bf263b21a3c49353f25945d8d9f70628e73de370d55e1e1", size = 206609, upload-time = "2026-04-02T09:27:36.661Z" },
+ { url = "https://files.pythonhosted.org/packages/69/d5/a527c0cd8d64d2eab7459784fb4169a0ac76e5a6fc5237337982fd61347e/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44", size = 220014, upload-time = "2026-04-02T09:27:38.019Z" },
+ { url = "https://files.pythonhosted.org/packages/7e/80/8a7b8104a3e203074dc9aa2c613d4b726c0e136bad1cc734594b02867972/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e", size = 218979, upload-time = "2026-04-02T09:27:39.37Z" },
+ { url = "https://files.pythonhosted.org/packages/02/9a/b759b503d507f375b2b5c153e4d2ee0a75aa215b7f2489cf314f4541f2c0/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:cf1493cd8607bec4d8a7b9b004e699fcf8f9103a9284cc94962cb73d20f9d4a3", size = 209238, upload-time = "2026-04-02T09:27:40.722Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/4e/0f3f5d47b86bdb79256e7290b26ac847a2832d9a4033f7eb2cd4bcf4bb5b/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0", size = 236110, upload-time = "2026-04-02T09:27:42.33Z" },
+ { url = "https://files.pythonhosted.org/packages/96/23/bce28734eb3ed2c91dcf93abeb8a5cf393a7b2749725030bb630e554fdd8/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e", size = 219824, upload-time = "2026-04-02T09:27:43.924Z" },
+ { url = "https://files.pythonhosted.org/packages/2c/6f/6e897c6984cc4d41af319b077f2f600fc8214eb2fe2d6bcb79141b882400/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb", size = 233103, upload-time = "2026-04-02T09:27:45.348Z" },
+ { url = "https://files.pythonhosted.org/packages/76/22/ef7bd0fe480a0ae9b656189ec00744b60933f68b4f42a7bb06589f6f576a/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ce3412fbe1e31eb81ea42f4169ed94861c56e643189e1e75f0041f3fe7020abe", size = 225194, upload-time = "2026-04-02T09:27:46.706Z" },
+ { url = "https://files.pythonhosted.org/packages/c5/a7/0e0ab3e0b5bc1219bd80a6a0d4d72ca74d9250cb2382b7c699c147e06017/charset_normalizer-3.4.7-cp314-cp314t-win32.whl", hash = "sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0", size = 159827, upload-time = "2026-04-02T09:27:48.053Z" },
+ { url = "https://files.pythonhosted.org/packages/7a/1d/29d32e0fb40864b1f878c7f5a0b343ae676c6e2b271a2d55cc3a152391da/charset_normalizer-3.4.7-cp314-cp314t-win_amd64.whl", hash = "sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c", size = 174168, upload-time = "2026-04-02T09:27:49.795Z" },
+ { url = "https://files.pythonhosted.org/packages/de/32/d92444ad05c7a6e41fb2036749777c163baf7a0301a040cb672d6b2b1ae9/charset_normalizer-3.4.7-cp314-cp314t-win_arm64.whl", hash = "sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d", size = 153018, upload-time = "2026-04-02T09:27:51.116Z" },
+ { url = "https://files.pythonhosted.org/packages/db/8f/61959034484a4a7c527811f4721e75d02d653a35afb0b6054474d8185d4c/charset_normalizer-3.4.7-py3-none-any.whl", hash = "sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d", size = 61958, upload-time = "2026-04-02T09:28:37.794Z" },
+]
+
+[[package]]
+name = "click"
+version = "8.4.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "colorama", marker = "sys_platform == 'win32'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/76/d4/81420972a676e8ffea40450d8c8c92943e7218a78fe9b64359836cc9876b/click-8.4.2.tar.gz", hash = "sha256:9a6cea6e60b17ebe0a44c5cc636d94f09bd66142c1cd7d8b4cd731c4917a15f6", size = 338000, upload-time = "2026-06-24T17:45:15.148Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/fb/e2/79c688af8b210d232694e31e59da9f6ec747bae31c3f5946e4e9b98860d5/click-8.4.2-py3-none-any.whl", hash = "sha256:e6f9f66136c816745b9d65817da91d61d957fb16e02e4dcd0552553c5a197b76", size = 119243, upload-time = "2026-06-24T17:45:13.73Z" },
+]
+
+[[package]]
+name = "colorama"
+version = "0.4.6"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
+]
+
+[[package]]
+name = "colorcet"
+version = "3.2.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/3f/af/b969f541242b84cbbacabdf20862e487689e352bf0f02f90df2795d29da5/colorcet-3.2.1.tar.gz", hash = "sha256:48d9a67e6e59dc5c0a965aa1b46fe5d59cdc95cc36a95949f29313f950ac59f7", size = 2202958, upload-time = "2026-04-28T16:25:37.43Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/1b/24/e95471ae93c08d3606c9c7343cf65d490f154daa88b50581957a0aa780f4/colorcet-3.2.1-py3-none-any.whl", hash = "sha256:3f6fde13cef2169222dd5fe2a2bf847c02d644470fdf167ed566f6421df470f7", size = 262291, upload-time = "2026-04-28T16:25:35.365Z" },
+]
+
+[[package]]
+name = "comm"
+version = "0.2.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/4c/13/7d740c5849255756bc17888787313b61fd38a0a8304fc4f073dfc46122aa/comm-0.2.3.tar.gz", hash = "sha256:2dc8048c10962d55d7ad693be1e7045d891b7ce8d999c97963a5e3e99c055971", size = 6319, upload-time = "2025-07-25T14:02:04.452Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/60/97/891a0971e1e4a8c5d2b20bbe0e524dc04548d2307fee33cdeba148fd4fc7/comm-0.2.3-py3-none-any.whl", hash = "sha256:c615d91d75f7f04f095b30d1c1711babd43bdc6419c1be9886a85f2f4e489417", size = 7294, upload-time = "2025-07-25T14:02:02.896Z" },
+]
+
+[[package]]
+name = "contourpy"
+version = "1.3.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/58/01/1253e6698a07380cd31a736d248a3f2a50a7c88779a1813da27503cadc2a/contourpy-1.3.3.tar.gz", hash = "sha256:083e12155b210502d0bca491432bb04d56dc3432f95a979b429f2848c3dbe880", size = 13466174, upload-time = "2025-07-26T12:03:12.549Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/be/45/adfee365d9ea3d853550b2e735f9d66366701c65db7855cd07621732ccfc/contourpy-1.3.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:b08a32ea2f8e42cf1d4be3169a98dd4be32bafe4f22b6c4cb4ba810fa9e5d2cb", size = 293419, upload-time = "2025-07-26T12:01:21.16Z" },
+ { url = "https://files.pythonhosted.org/packages/53/3e/405b59cfa13021a56bba395a6b3aca8cec012b45bf177b0eaf7a202cde2c/contourpy-1.3.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:556dba8fb6f5d8742f2923fe9457dbdd51e1049c4a43fd3986a0b14a1d815fc6", size = 273979, upload-time = "2025-07-26T12:01:22.448Z" },
+ { url = "https://files.pythonhosted.org/packages/d4/1c/a12359b9b2ca3a845e8f7f9ac08bdf776114eb931392fcad91743e2ea17b/contourpy-1.3.3-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92d9abc807cf7d0e047b95ca5d957cf4792fcd04e920ca70d48add15c1a90ea7", size = 332653, upload-time = "2025-07-26T12:01:24.155Z" },
+ { url = "https://files.pythonhosted.org/packages/63/12/897aeebfb475b7748ea67b61e045accdfcf0d971f8a588b67108ed7f5512/contourpy-1.3.3-cp312-cp312-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b2e8faa0ed68cb29af51edd8e24798bb661eac3bd9f65420c1887b6ca89987c8", size = 379536, upload-time = "2025-07-26T12:01:25.91Z" },
+ { url = "https://files.pythonhosted.org/packages/43/8a/a8c584b82deb248930ce069e71576fc09bd7174bbd35183b7943fb1064fd/contourpy-1.3.3-cp312-cp312-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:626d60935cf668e70a5ce6ff184fd713e9683fb458898e4249b63be9e28286ea", size = 384397, upload-time = "2025-07-26T12:01:27.152Z" },
+ { url = "https://files.pythonhosted.org/packages/cc/8f/ec6289987824b29529d0dfda0d74a07cec60e54b9c92f3c9da4c0ac732de/contourpy-1.3.3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4d00e655fcef08aba35ec9610536bfe90267d7ab5ba944f7032549c55a146da1", size = 362601, upload-time = "2025-07-26T12:01:28.808Z" },
+ { url = "https://files.pythonhosted.org/packages/05/0a/a3fe3be3ee2dceb3e615ebb4df97ae6f3828aa915d3e10549ce016302bd1/contourpy-1.3.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:451e71b5a7d597379ef572de31eeb909a87246974d960049a9848c3bc6c41bf7", size = 1331288, upload-time = "2025-07-26T12:01:31.198Z" },
+ { url = "https://files.pythonhosted.org/packages/33/1d/acad9bd4e97f13f3e2b18a3977fe1b4a37ecf3d38d815333980c6c72e963/contourpy-1.3.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:459c1f020cd59fcfe6650180678a9993932d80d44ccde1fa1868977438f0b411", size = 1403386, upload-time = "2025-07-26T12:01:33.947Z" },
+ { url = "https://files.pythonhosted.org/packages/cf/8f/5847f44a7fddf859704217a99a23a4f6417b10e5ab1256a179264561540e/contourpy-1.3.3-cp312-cp312-win32.whl", hash = "sha256:023b44101dfe49d7d53932be418477dba359649246075c996866106da069af69", size = 185018, upload-time = "2025-07-26T12:01:35.64Z" },
+ { url = "https://files.pythonhosted.org/packages/19/e8/6026ed58a64563186a9ee3f29f41261fd1828f527dd93d33b60feca63352/contourpy-1.3.3-cp312-cp312-win_amd64.whl", hash = "sha256:8153b8bfc11e1e4d75bcb0bff1db232f9e10b274e0929de9d608027e0d34ff8b", size = 226567, upload-time = "2025-07-26T12:01:36.804Z" },
+ { url = "https://files.pythonhosted.org/packages/d1/e2/f05240d2c39a1ed228d8328a78b6f44cd695f7ef47beb3e684cf93604f86/contourpy-1.3.3-cp312-cp312-win_arm64.whl", hash = "sha256:07ce5ed73ecdc4a03ffe3e1b3e3c1166db35ae7584be76f65dbbe28a7791b0cc", size = 193655, upload-time = "2025-07-26T12:01:37.999Z" },
+ { url = "https://files.pythonhosted.org/packages/68/35/0167aad910bbdb9599272bd96d01a9ec6852f36b9455cf2ca67bd4cc2d23/contourpy-1.3.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:177fb367556747a686509d6fef71d221a4b198a3905fe824430e5ea0fda54eb5", size = 293257, upload-time = "2025-07-26T12:01:39.367Z" },
+ { url = "https://files.pythonhosted.org/packages/96/e4/7adcd9c8362745b2210728f209bfbcf7d91ba868a2c5f40d8b58f54c509b/contourpy-1.3.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d002b6f00d73d69333dac9d0b8d5e84d9724ff9ef044fd63c5986e62b7c9e1b1", size = 274034, upload-time = "2025-07-26T12:01:40.645Z" },
+ { url = "https://files.pythonhosted.org/packages/73/23/90e31ceeed1de63058a02cb04b12f2de4b40e3bef5e082a7c18d9c8ae281/contourpy-1.3.3-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:348ac1f5d4f1d66d3322420f01d42e43122f43616e0f194fc1c9f5d830c5b286", size = 334672, upload-time = "2025-07-26T12:01:41.942Z" },
+ { url = "https://files.pythonhosted.org/packages/ed/93/b43d8acbe67392e659e1d984700e79eb67e2acb2bd7f62012b583a7f1b55/contourpy-1.3.3-cp313-cp313-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:655456777ff65c2c548b7c454af9c6f33f16c8884f11083244b5819cc214f1b5", size = 381234, upload-time = "2025-07-26T12:01:43.499Z" },
+ { url = "https://files.pythonhosted.org/packages/46/3b/bec82a3ea06f66711520f75a40c8fc0b113b2a75edb36aa633eb11c4f50f/contourpy-1.3.3-cp313-cp313-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:644a6853d15b2512d67881586bd03f462c7ab755db95f16f14d7e238f2852c67", size = 385169, upload-time = "2025-07-26T12:01:45.219Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/32/e0f13a1c5b0f8572d0ec6ae2f6c677b7991fafd95da523159c19eff0696a/contourpy-1.3.3-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4debd64f124ca62069f313a9cb86656ff087786016d76927ae2cf37846b006c9", size = 362859, upload-time = "2025-07-26T12:01:46.519Z" },
+ { url = "https://files.pythonhosted.org/packages/33/71/e2a7945b7de4e58af42d708a219f3b2f4cff7386e6b6ab0a0fa0033c49a9/contourpy-1.3.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a15459b0f4615b00bbd1e91f1b9e19b7e63aea7483d03d804186f278c0af2659", size = 1332062, upload-time = "2025-07-26T12:01:48.964Z" },
+ { url = "https://files.pythonhosted.org/packages/12/fc/4e87ac754220ccc0e807284f88e943d6d43b43843614f0a8afa469801db0/contourpy-1.3.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ca0fdcd73925568ca027e0b17ab07aad764be4706d0a925b89227e447d9737b7", size = 1403932, upload-time = "2025-07-26T12:01:51.979Z" },
+ { url = "https://files.pythonhosted.org/packages/a6/2e/adc197a37443f934594112222ac1aa7dc9a98faf9c3842884df9a9d8751d/contourpy-1.3.3-cp313-cp313-win32.whl", hash = "sha256:b20c7c9a3bf701366556e1b1984ed2d0cedf999903c51311417cf5f591d8c78d", size = 185024, upload-time = "2025-07-26T12:01:53.245Z" },
+ { url = "https://files.pythonhosted.org/packages/18/0b/0098c214843213759692cc638fce7de5c289200a830e5035d1791d7a2338/contourpy-1.3.3-cp313-cp313-win_amd64.whl", hash = "sha256:1cadd8b8969f060ba45ed7c1b714fe69185812ab43bd6b86a9123fe8f99c3263", size = 226578, upload-time = "2025-07-26T12:01:54.422Z" },
+ { url = "https://files.pythonhosted.org/packages/8a/9a/2f6024a0c5995243cd63afdeb3651c984f0d2bc727fd98066d40e141ad73/contourpy-1.3.3-cp313-cp313-win_arm64.whl", hash = "sha256:fd914713266421b7536de2bfa8181aa8c699432b6763a0ea64195ebe28bff6a9", size = 193524, upload-time = "2025-07-26T12:01:55.73Z" },
+ { url = "https://files.pythonhosted.org/packages/c0/b3/f8a1a86bd3298513f500e5b1f5fd92b69896449f6cab6a146a5d52715479/contourpy-1.3.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:88df9880d507169449d434c293467418b9f6cbe82edd19284aa0409e7fdb933d", size = 306730, upload-time = "2025-07-26T12:01:57.051Z" },
+ { url = "https://files.pythonhosted.org/packages/3f/11/4780db94ae62fc0c2053909b65dc3246bd7cecfc4f8a20d957ad43aa4ad8/contourpy-1.3.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:d06bb1f751ba5d417047db62bca3c8fde202b8c11fb50742ab3ab962c81e8216", size = 287897, upload-time = "2025-07-26T12:01:58.663Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/15/e59f5f3ffdd6f3d4daa3e47114c53daabcb18574a26c21f03dc9e4e42ff0/contourpy-1.3.3-cp313-cp313t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e4e6b05a45525357e382909a4c1600444e2a45b4795163d3b22669285591c1ae", size = 326751, upload-time = "2025-07-26T12:02:00.343Z" },
+ { url = "https://files.pythonhosted.org/packages/0f/81/03b45cfad088e4770b1dcf72ea78d3802d04200009fb364d18a493857210/contourpy-1.3.3-cp313-cp313t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ab3074b48c4e2cf1a960e6bbeb7f04566bf36b1861d5c9d4d8ac04b82e38ba20", size = 375486, upload-time = "2025-07-26T12:02:02.128Z" },
+ { url = "https://files.pythonhosted.org/packages/0c/ba/49923366492ffbdd4486e970d421b289a670ae8cf539c1ea9a09822b371a/contourpy-1.3.3-cp313-cp313t-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:6c3d53c796f8647d6deb1abe867daeb66dcc8a97e8455efa729516b997b8ed99", size = 388106, upload-time = "2025-07-26T12:02:03.615Z" },
+ { url = "https://files.pythonhosted.org/packages/9f/52/5b00ea89525f8f143651f9f03a0df371d3cbd2fccd21ca9b768c7a6500c2/contourpy-1.3.3-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:50ed930df7289ff2a8d7afeb9603f8289e5704755c7e5c3bbd929c90c817164b", size = 352548, upload-time = "2025-07-26T12:02:05.165Z" },
+ { url = "https://files.pythonhosted.org/packages/32/1d/a209ec1a3a3452d490f6b14dd92e72280c99ae3d1e73da74f8277d4ee08f/contourpy-1.3.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:4feffb6537d64b84877da813a5c30f1422ea5739566abf0bd18065ac040e120a", size = 1322297, upload-time = "2025-07-26T12:02:07.379Z" },
+ { url = "https://files.pythonhosted.org/packages/bc/9e/46f0e8ebdd884ca0e8877e46a3f4e633f6c9c8c4f3f6e72be3fe075994aa/contourpy-1.3.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:2b7e9480ffe2b0cd2e787e4df64270e3a0440d9db8dc823312e2c940c167df7e", size = 1391023, upload-time = "2025-07-26T12:02:10.171Z" },
+ { url = "https://files.pythonhosted.org/packages/b9/70/f308384a3ae9cd2209e0849f33c913f658d3326900d0ff5d378d6a1422d2/contourpy-1.3.3-cp313-cp313t-win32.whl", hash = "sha256:283edd842a01e3dcd435b1c5116798d661378d83d36d337b8dde1d16a5fc9ba3", size = 196157, upload-time = "2025-07-26T12:02:11.488Z" },
+ { url = "https://files.pythonhosted.org/packages/b2/dd/880f890a6663b84d9e34a6f88cded89d78f0091e0045a284427cb6b18521/contourpy-1.3.3-cp313-cp313t-win_amd64.whl", hash = "sha256:87acf5963fc2b34825e5b6b048f40e3635dd547f590b04d2ab317c2619ef7ae8", size = 240570, upload-time = "2025-07-26T12:02:12.754Z" },
+ { url = "https://files.pythonhosted.org/packages/80/99/2adc7d8ffead633234817ef8e9a87115c8a11927a94478f6bb3d3f4d4f7d/contourpy-1.3.3-cp313-cp313t-win_arm64.whl", hash = "sha256:3c30273eb2a55024ff31ba7d052dde990d7d8e5450f4bbb6e913558b3d6c2301", size = 199713, upload-time = "2025-07-26T12:02:14.4Z" },
+ { url = "https://files.pythonhosted.org/packages/72/8b/4546f3ab60f78c514ffb7d01a0bd743f90de36f0019d1be84d0a708a580a/contourpy-1.3.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:fde6c716d51c04b1c25d0b90364d0be954624a0ee9d60e23e850e8d48353d07a", size = 292189, upload-time = "2025-07-26T12:02:16.095Z" },
+ { url = "https://files.pythonhosted.org/packages/fd/e1/3542a9cb596cadd76fcef413f19c79216e002623158befe6daa03dbfa88c/contourpy-1.3.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:cbedb772ed74ff5be440fa8eee9bd49f64f6e3fc09436d9c7d8f1c287b121d77", size = 273251, upload-time = "2025-07-26T12:02:17.524Z" },
+ { url = "https://files.pythonhosted.org/packages/b1/71/f93e1e9471d189f79d0ce2497007731c1e6bf9ef6d1d61b911430c3db4e5/contourpy-1.3.3-cp314-cp314-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:22e9b1bd7a9b1d652cd77388465dc358dafcd2e217d35552424aa4f996f524f5", size = 335810, upload-time = "2025-07-26T12:02:18.9Z" },
+ { url = "https://files.pythonhosted.org/packages/91/f9/e35f4c1c93f9275d4e38681a80506b5510e9327350c51f8d4a5a724d178c/contourpy-1.3.3-cp314-cp314-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a22738912262aa3e254e4f3cb079a95a67132fc5a063890e224393596902f5a4", size = 382871, upload-time = "2025-07-26T12:02:20.418Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/71/47b512f936f66a0a900d81c396a7e60d73419868fba959c61efed7a8ab46/contourpy-1.3.3-cp314-cp314-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:afe5a512f31ee6bd7d0dda52ec9864c984ca3d66664444f2d72e0dc4eb832e36", size = 386264, upload-time = "2025-07-26T12:02:21.916Z" },
+ { url = "https://files.pythonhosted.org/packages/04/5f/9ff93450ba96b09c7c2b3f81c94de31c89f92292f1380261bd7195bea4ea/contourpy-1.3.3-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f64836de09927cba6f79dcd00fdd7d5329f3fccc633468507079c829ca4db4e3", size = 363819, upload-time = "2025-07-26T12:02:23.759Z" },
+ { url = "https://files.pythonhosted.org/packages/3e/a6/0b185d4cc480ee494945cde102cb0149ae830b5fa17bf855b95f2e70ad13/contourpy-1.3.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:1fd43c3be4c8e5fd6e4f2baeae35ae18176cf2e5cced681cca908addf1cdd53b", size = 1333650, upload-time = "2025-07-26T12:02:26.181Z" },
+ { url = "https://files.pythonhosted.org/packages/43/d7/afdc95580ca56f30fbcd3060250f66cedbde69b4547028863abd8aa3b47e/contourpy-1.3.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6afc576f7b33cf00996e5c1102dc2a8f7cc89e39c0b55df93a0b78c1bd992b36", size = 1404833, upload-time = "2025-07-26T12:02:28.782Z" },
+ { url = "https://files.pythonhosted.org/packages/e2/e2/366af18a6d386f41132a48f033cbd2102e9b0cf6345d35ff0826cd984566/contourpy-1.3.3-cp314-cp314-win32.whl", hash = "sha256:66c8a43a4f7b8df8b71ee1840e4211a3c8d93b214b213f590e18a1beca458f7d", size = 189692, upload-time = "2025-07-26T12:02:30.128Z" },
+ { url = "https://files.pythonhosted.org/packages/7d/c2/57f54b03d0f22d4044b8afb9ca0e184f8b1afd57b4f735c2fa70883dc601/contourpy-1.3.3-cp314-cp314-win_amd64.whl", hash = "sha256:cf9022ef053f2694e31d630feaacb21ea24224be1c3ad0520b13d844274614fd", size = 232424, upload-time = "2025-07-26T12:02:31.395Z" },
+ { url = "https://files.pythonhosted.org/packages/18/79/a9416650df9b525737ab521aa181ccc42d56016d2123ddcb7b58e926a42c/contourpy-1.3.3-cp314-cp314-win_arm64.whl", hash = "sha256:95b181891b4c71de4bb404c6621e7e2390745f887f2a026b2d99e92c17892339", size = 198300, upload-time = "2025-07-26T12:02:32.956Z" },
+ { url = "https://files.pythonhosted.org/packages/1f/42/38c159a7d0f2b7b9c04c64ab317042bb6952b713ba875c1681529a2932fe/contourpy-1.3.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:33c82d0138c0a062380332c861387650c82e4cf1747aaa6938b9b6516762e772", size = 306769, upload-time = "2025-07-26T12:02:34.2Z" },
+ { url = "https://files.pythonhosted.org/packages/c3/6c/26a8205f24bca10974e77460de68d3d7c63e282e23782f1239f226fcae6f/contourpy-1.3.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ea37e7b45949df430fe649e5de8351c423430046a2af20b1c1961cae3afcda77", size = 287892, upload-time = "2025-07-26T12:02:35.807Z" },
+ { url = "https://files.pythonhosted.org/packages/66/06/8a475c8ab718ebfd7925661747dbb3c3ee9c82ac834ccb3570be49d129f4/contourpy-1.3.3-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d304906ecc71672e9c89e87c4675dc5c2645e1f4269a5063b99b0bb29f232d13", size = 326748, upload-time = "2025-07-26T12:02:37.193Z" },
+ { url = "https://files.pythonhosted.org/packages/b4/a3/c5ca9f010a44c223f098fccd8b158bb1cb287378a31ac141f04730dc49be/contourpy-1.3.3-cp314-cp314t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ca658cd1a680a5c9ea96dc61cdbae1e85c8f25849843aa799dfd3cb370ad4fbe", size = 375554, upload-time = "2025-07-26T12:02:38.894Z" },
+ { url = "https://files.pythonhosted.org/packages/80/5b/68bd33ae63fac658a4145088c1e894405e07584a316738710b636c6d0333/contourpy-1.3.3-cp314-cp314t-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:ab2fd90904c503739a75b7c8c5c01160130ba67944a7b77bbf36ef8054576e7f", size = 388118, upload-time = "2025-07-26T12:02:40.642Z" },
+ { url = "https://files.pythonhosted.org/packages/40/52/4c285a6435940ae25d7410a6c36bda5145839bc3f0beb20c707cda18b9d2/contourpy-1.3.3-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b7301b89040075c30e5768810bc96a8e8d78085b47d8be6e4c3f5a0b4ed478a0", size = 352555, upload-time = "2025-07-26T12:02:42.25Z" },
+ { url = "https://files.pythonhosted.org/packages/24/ee/3e81e1dd174f5c7fefe50e85d0892de05ca4e26ef1c9a59c2a57e43b865a/contourpy-1.3.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:2a2a8b627d5cc6b7c41a4beff6c5ad5eb848c88255fda4a8745f7e901b32d8e4", size = 1322295, upload-time = "2025-07-26T12:02:44.668Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/b2/6d913d4d04e14379de429057cd169e5e00f6c2af3bb13e1710bcbdb5da12/contourpy-1.3.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:fd6ec6be509c787f1caf6b247f0b1ca598bef13f4ddeaa126b7658215529ba0f", size = 1391027, upload-time = "2025-07-26T12:02:47.09Z" },
+ { url = "https://files.pythonhosted.org/packages/93/8a/68a4ec5c55a2971213d29a9374913f7e9f18581945a7a31d1a39b5d2dfe5/contourpy-1.3.3-cp314-cp314t-win32.whl", hash = "sha256:e74a9a0f5e3fff48fb5a7f2fd2b9b70a3fe014a67522f79b7cca4c0c7e43c9ae", size = 202428, upload-time = "2025-07-26T12:02:48.691Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/96/fd9f641ffedc4fa3ace923af73b9d07e869496c9cc7a459103e6e978992f/contourpy-1.3.3-cp314-cp314t-win_amd64.whl", hash = "sha256:13b68d6a62db8eafaebb8039218921399baf6e47bf85006fd8529f2a08ef33fc", size = 250331, upload-time = "2025-07-26T12:02:50.137Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/8c/469afb6465b853afff216f9528ffda78a915ff880ed58813ba4faf4ba0b6/contourpy-1.3.3-cp314-cp314t-win_arm64.whl", hash = "sha256:b7448cb5a725bb1e35ce88771b86fba35ef418952474492cf7c764059933ff8b", size = 203831, upload-time = "2025-07-26T12:02:51.449Z" },
+]
+
+[[package]]
+name = "coverage"
+version = "7.14.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/b4/91/0a7c28934e50d8ac9a7b117712d176f2953c3170bccced5eaacfa3e96175/coverage-7.14.3.tar.gz", hash = "sha256:1a7563a443f3d53fdeb040ec8c9f7466aed7ca3dc5891aa09d3ca3625fa4387f", size = 924398, upload-time = "2026-06-22T23:10:25.584Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/bd/b0/8a911f6ffe6974dac4df95b468ab9a2899d0e59f0f99a489afeec39f00bc/coverage-7.14.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3d74ff26299c4879ce3a4d826f9d3d4d556fd285fde7bbce3c0ef5a8ab1cec24", size = 220672, upload-time = "2026-06-22T23:08:26.621Z" },
+ { url = "https://files.pythonhosted.org/packages/36/16/0fc0cb52538783dbbae0934b834f5a58fd5354380ee6cad4a07b15dc845d/coverage-7.14.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:96150a9cf3468ea20f0bc5d0e21b3df8972c31480ef90fa7614b773cc6429665", size = 221035, upload-time = "2026-06-22T23:08:28.372Z" },
+ { url = "https://files.pythonhosted.org/packages/77/e2/421ccfbb48335ac49e93301478cf5d623b0c2bf1c0cadd8e2b2fc6c0c710/coverage-7.14.3-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:27d07a46500ba23515b838dbcf52512026af04090755cf6cc64166d88c9b9a1a", size = 252540, upload-time = "2026-06-22T23:08:30.226Z" },
+ { url = "https://files.pythonhosted.org/packages/06/c2/05b8c890097c61a7f4406b35396b997a635200ded0339eda83dfbe526c5f/coverage-7.14.3-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:621e13c6108234d7960aaf5762ab5c3c00f33c30c15af06dcbff0c73bf112727", size = 255274, upload-time = "2026-06-22T23:08:31.876Z" },
+ { url = "https://files.pythonhosted.org/packages/dc/be/b6d9efe447f8ba3c3c854195f326bd64c54b907d936cd2fdebf8767ec72e/coverage-7.14.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4b60ca6d8af70473491a15a343cbabab2e8f9ea66a4376e81c7aa24876a6f977", size = 256389, upload-time = "2026-06-22T23:08:33.843Z" },
+ { url = "https://files.pythonhosted.org/packages/d4/3c/f26e50acc429e608bc534ac06f0a3c169019c798178ec5e9de3dbc0df9c9/coverage-7.14.3-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c90a7cdd5e380e1ce02f19792e2ac2fbfbf177e35a27e69fd3e873b30d895c0c", size = 258648, upload-time = "2026-06-22T23:08:35.481Z" },
+ { url = "https://files.pythonhosted.org/packages/9e/a2/01c1fabf816c8e1dae197e258edf878a3d3ddc86fbda34b76e5794277d8f/coverage-7.14.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:5d788e5fd55347eef06ca0732c77d04a264de67e8ff24631270cdff3767a60cf", size = 252949, upload-time = "2026-06-22T23:08:37.562Z" },
+ { url = "https://files.pythonhosted.org/packages/89/c6/941166dd79c31fd44a13063780ae8d552eee0089a0a0930b9bdb7df554ed/coverage-7.14.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:62c7f79db2851c95ef020e5d28b97afde3daf9f7febcd35b53e05638f729063f", size = 254310, upload-time = "2026-06-22T23:08:39.174Z" },
+ { url = "https://files.pythonhosted.org/packages/10/31/80b1fd028201a961033ce95be3cd1e39e521b3762e6b4a1ac1616cb291e7/coverage-7.14.3-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:90f7608aeb5d9b60b523b9fb2a4ee1973867cc4865a3f26fe6c7577073b70205", size = 252453, upload-time = "2026-06-22T23:08:40.84Z" },
+ { url = "https://files.pythonhosted.org/packages/5f/85/c3d9addd94c4b524f3f4af0232075f5fe7170ce99a1386edff803e5934db/coverage-7.14.3-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:1e3b91f9c4740aeb571ecf82e5e8d8e4ab62d34fcb5a5d4e5baa38c6f7d2857c", size = 256522, upload-time = "2026-06-22T23:08:42.494Z" },
+ { url = "https://files.pythonhosted.org/packages/91/14/e5a0575f73795af3a7a9ae13dadf812e17d32422896839987dc3f86947e1/coverage-7.14.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:c946099774a7699de03cbd0ff0a64e21aed4525eed9d959adde4afe6d15758ef", size = 252023, upload-time = "2026-06-22T23:08:44.243Z" },
+ { url = "https://files.pythonhosted.org/packages/38/9b/9652ee531937ce3b8a63a8896885b2b4a2d56adc30e53c9540c666286d88/coverage-7.14.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:16b206e521feb8b7133a45754643dead0538489cf8b783b90cf5f4e3299625fd", size = 253893, upload-time = "2026-06-22T23:08:46.113Z" },
+ { url = "https://files.pythonhosted.org/packages/b1/05/42678841c8c38e4b08bdfc48269f5a16dfbf5806000fe6a89b4cece3c691/coverage-7.14.3-cp312-cp312-win32.whl", hash = "sha256:ea3169c7116eb6cdf7608c6c7da9ecfcb3da40688e3a510fac2d1d2bafd6dc35", size = 222734, upload-time = "2026-06-22T23:08:47.858Z" },
+ { url = "https://files.pythonhosted.org/packages/df/87/07a4fcee55177a25f1b52331a8e92cf4f2c53b1a9c75ce2981fd59c684ad/coverage-7.14.3-cp312-cp312-win_amd64.whl", hash = "sha256:7ea52fc08f007bcc494d4bb3df3851e95843d881860ba38fe2c64dc100db5e7d", size = 223266, upload-time = "2026-06-22T23:08:49.494Z" },
+ { url = "https://files.pythonhosted.org/packages/aa/34/2b8b66a989282ea7b370beb49f50bab29470dc30bb0b03935b6b802782f7/coverage-7.14.3-cp312-cp312-win_arm64.whl", hash = "sha256:8cec0ad652ec57790970d817490105bd917d783c2f7b38d6b58a0ca312e1a336", size = 222655, upload-time = "2026-06-22T23:08:51.766Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/83/7fefbf5df23ed2b7f489907564a7b34b9b07098128e12e0fdfa92626e456/coverage-7.14.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:47968988b367990ae4ab17523790c38cd125e02c6bfd379b6022be2d40bdc38c", size = 220699, upload-time = "2026-06-22T23:08:53.522Z" },
+ { url = "https://files.pythonhosted.org/packages/31/e6/38c3653ff6d56d704b29241362387ca824e38e15b76fdcb7096538195790/coverage-7.14.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:0ee68f5c34812780f3a7063382c0a9fcbb99985b7ddcdcaa626e4f3fb2e0783a", size = 221068, upload-time = "2026-06-22T23:08:55.571Z" },
+ { url = "https://files.pythonhosted.org/packages/20/86/4f5c45d51c5cd10a128933f0fd235393c9146abbfd2ce2dfa68b3267ead3/coverage-7.14.3-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:fa9e5c6857a7e80fa22ace5cf3550ae392bbfc322f1d8dd2d2d5a8be38cec027", size = 252060, upload-time = "2026-06-22T23:08:57.464Z" },
+ { url = "https://files.pythonhosted.org/packages/82/50/dfce42eff2cecabcd5a9bbad5489449c87db3415f408d23ffee417ce01f6/coverage-7.14.3-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:98a0859b0e98e43e1178a9402e19c8127766b14f7109a374d976e5a62c0e5c73", size = 254657, upload-time = "2026-06-22T23:08:59.453Z" },
+ { url = "https://files.pythonhosted.org/packages/ba/d2/639ceb1bc8038fd0d66768278d5dc22df3391918b8278c2a21aa2602a531/coverage-7.14.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:69918344541ed9c8368566c2adc03c0e33d4550d7faa87d1b35e49b6a3286ea9", size = 255892, upload-time = "2026-06-22T23:09:01.291Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/96/002094a10e113512500dc1e10430a449417e17b0f90f7d496bcb820208b7/coverage-7.14.3-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b7f300ac92cd4b570724c8ffbbd0c130fee298d2447f41d5a3abf58976fae1de", size = 258026, upload-time = "2026-06-22T23:09:03.017Z" },
+ { url = "https://files.pythonhosted.org/packages/0b/ec/286a5d2fad9c4bee59bd724feeb7d5bf8303c6c9200b51d1dd945a9c72b0/coverage-7.14.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:11a7ec9f97ab950f4c5af62229befc7faf208fdbc0116d3902d7e306cf2c5abd", size = 252285, upload-time = "2026-06-22T23:09:04.773Z" },
+ { url = "https://files.pythonhosted.org/packages/d9/7d/a17753a0b12dd48d0d50f5fab079ad99d3be1eac790494d89f3a417ca0b9/coverage-7.14.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a571bd889cd36c5922ce8e42e059f9d37d02301531d11374afa4c87a578625d5", size = 254023, upload-time = "2026-06-22T23:09:06.513Z" },
+ { url = "https://files.pythonhosted.org/packages/86/ef/a76c6ceba6a2c313f905310abf2701d534cada22d372db11731831e9e209/coverage-7.14.3-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:de76caefc8deabb0dd1678b6a980be97d14c8d87e213ac194dbf8b09e96d63fb", size = 251989, upload-time = "2026-06-22T23:09:08.382Z" },
+ { url = "https://files.pythonhosted.org/packages/d9/39/353013a75fec0fb49f7553519f9d52b4441e902e5178c93f38eb6c07cedb/coverage-7.14.3-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:d20a15c622194234161535459affa8f7905830391c9ccfa060d495dbfe3a1c7f", size = 256144, upload-time = "2026-06-22T23:09:10.369Z" },
+ { url = "https://files.pythonhosted.org/packages/29/0e/613878555d734def11c5b20a2701a15cb3781b9e9ea749da27c5f436e928/coverage-7.14.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:b488bd4b23397db62e7a9459129d01ff06a846582a732efd24834b24a6ada498", size = 251808, upload-time = "2026-06-22T23:09:12.057Z" },
+ { url = "https://files.pythonhosted.org/packages/af/76/359c058c9cfdcf1e8b107663881225b03b364a320017eda24a2a66e55102/coverage-7.14.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6a3693b4153394d265f44fb855fdc80e72403024d4d6f91c4871b334d028e4e0", size = 253579, upload-time = "2026-06-22T23:09:13.858Z" },
+ { url = "https://files.pythonhosted.org/packages/1d/d9/4ba2f060933a30ebe363cef9f67a365b0a317e580c0d5d9169d56a73ef1c/coverage-7.14.3-cp313-cp313-win32.whl", hash = "sha256:338b19131ab1a6b767b462bfcbaa692e7ae22f24463e39d49b02a83410ff6b37", size = 222741, upload-time = "2026-06-22T23:09:15.636Z" },
+ { url = "https://files.pythonhosted.org/packages/76/e8/196ebc25d8f34c06d43a6e9c8513c9266ef8dbf3b5672beb1a00cf5e29fa/coverage-7.14.3-cp313-cp313-win_amd64.whl", hash = "sha256:b3d77f7f196abdef7e01415de1bce09f216189e83e58159cfeef2b92d0464994", size = 223283, upload-time = "2026-06-22T23:09:17.478Z" },
+ { url = "https://files.pythonhosted.org/packages/7c/af/51d2aac6417523a286f10fb25f09eb9518a84df9f1151e93ff6871f34849/coverage-7.14.3-cp313-cp313-win_arm64.whl", hash = "sha256:e6230e688c7c3e65cedd41a774eb4ec221adc6bfee13768231015b702d5e4150", size = 222678, upload-time = "2026-06-22T23:09:19.7Z" },
+ { url = "https://files.pythonhosted.org/packages/61/56/14e3b97facbfa1304dd19e676e26599ad359f04714bed32f7f1c5a88efdc/coverage-7.14.3-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:605ab2b566a22bd94834529d66d295c364aba84afd3e5498285c7a524017b1fc", size = 220741, upload-time = "2026-06-22T23:09:21.616Z" },
+ { url = "https://files.pythonhosted.org/packages/12/1d/db378b5cca433b90b893f26dab728b280ddd89f272a1fdfed4aeaa05c686/coverage-7.14.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:a3c2134809e80fac091bfed18a6991b5a5eb5df5ae32b17ac4f4f99864b73dd7", size = 221068, upload-time = "2026-06-22T23:09:23.452Z" },
+ { url = "https://files.pythonhosted.org/packages/47/f0/3f8421b20d9c4fcd39be9a8ca3c3fda8bc204b44efbd09fede153afd3e2f/coverage-7.14.3-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:c02efd507227bde9969cab0db8f48890eb3b5dcad6afac57a4792df4133543ce", size = 252117, upload-time = "2026-06-22T23:09:25.458Z" },
+ { url = "https://files.pythonhosted.org/packages/27/ca/59ea35fb99743549ec8b37eff141ece4431fea590c89e536ed8032ef45cf/coverage-7.14.3-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:1bb93c2aa61d2a5b38f1526546d95cf4132cb681e541a337bf8dfd092be816e5", size = 254622, upload-time = "2026-06-22T23:09:27.523Z" },
+ { url = "https://files.pythonhosted.org/packages/c8/25/ec6de51ae7493b92a1cf74d1b763121c29636759167e2a593ba4db5881e4/coverage-7.14.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f502e948e03e866538048bba081c075caaa62e5bda6ea5b7432e45f587eb462a", size = 255968, upload-time = "2026-06-22T23:09:29.43Z" },
+ { url = "https://files.pythonhosted.org/packages/5d/05/c8bfc77823f42b4664fb25842f13b567022f6f84a4c83c8ecbb16734b7cb/coverage-7.14.3-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9973ef2463f8e6cfb61a6324126bb3e17d67a85f22f58d856e583ea2e3ca6501", size = 258284, upload-time = "2026-06-22T23:09:31.397Z" },
+ { url = "https://files.pythonhosted.org/packages/f6/15/1d1b242027124a32b26ef01f82018b8c4ef34ef174aa6aeba7b1eeef48e8/coverage-7.14.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9be4e7d4c5ca0427889f8f9d614bd630c2be741b1de7699bca3b2b6c0e41003e", size = 252143, upload-time = "2026-06-22T23:09:33.256Z" },
+ { url = "https://files.pythonhosted.org/packages/74/b6/d2a9842fd2a5d7d27f1ac851c043a734a494ad75402c5331db3da79ed691/coverage-7.14.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a574912f3bde4b0619f6e97d01aa590b70998859244793769eb3a6df78ee56d3", size = 253976, upload-time = "2026-06-22T23:09:35.351Z" },
+ { url = "https://files.pythonhosted.org/packages/fd/30/e1600ddf7e226db5558bb5323d2186fff00f505c4b764643ec89ce5d8175/coverage-7.14.3-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:e343fb086c9cd780b38622fea7c369acd64c1a0724312149b5d769c387a2b1f5", size = 251942, upload-time = "2026-06-22T23:09:37.313Z" },
+ { url = "https://files.pythonhosted.org/packages/d9/2c/9159de64f9dd648e324328d588a44cfab1e331eb5259ce1141afe2a92dfb/coverage-7.14.3-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:3c68df8e61f1e09633fefc7538297145623957a048534368c9d212782aa5e845", size = 256220, upload-time = "2026-06-22T23:09:39.165Z" },
+ { url = "https://files.pythonhosted.org/packages/91/67/b7f536cc2c124f48e91b22fbb741d2261f4e3d310faf6f76007f47566e5d/coverage-7.14.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:3e5b550a128419373c2f6cec28a244207013ef15f5cbcff6a5ca09d1dfaaf027", size = 251756, upload-time = "2026-06-22T23:09:41.056Z" },
+ { url = "https://files.pythonhosted.org/packages/dd/ec/f3718038e2d4860c715a55428377ca7f6c75872caf98cabd982e1d76967d/coverage-7.14.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2bfc4dd0a912329eccc7484a7d0b2a38032b38c40663b1e1ac595f10c457954b", size = 253413, upload-time = "2026-06-22T23:09:43.306Z" },
+ { url = "https://files.pythonhosted.org/packages/b8/a5/91f11efeef89b3cc9b30461128db15b0511ef813ab889a7b7ab636b3a497/coverage-7.14.3-cp314-cp314-win32.whl", hash = "sha256:0423d64c013057a06e70f070f073cec4b0cbc7d2b27f3c7007292f2ff1d52965", size = 222946, upload-time = "2026-06-22T23:09:45.261Z" },
+ { url = "https://files.pythonhosted.org/packages/58/fd/98ac9f524d9ec378de831c034dbdeb544ca7ef7d2d9c9996daf232a037fd/coverage-7.14.3-cp314-cp314-win_amd64.whl", hash = "sha256:92c22e19ce64ca3f2ad751f16f14df1468b4c231bd6af97185063a9c292a0cb3", size = 223436, upload-time = "2026-06-22T23:09:47.177Z" },
+ { url = "https://files.pythonhosted.org/packages/b4/a0/7cd612d650a772a0ae80144443406bf61981c896c3d57c9e6e79fb2cdbd1/coverage-7.14.3-cp314-cp314-win_arm64.whl", hash = "sha256:41de778bd41780586e2b04912079c73089ab5d839624e28db3bdb26de638da92", size = 222861, upload-time = "2026-06-22T23:09:49.384Z" },
+ { url = "https://files.pythonhosted.org/packages/55/57/017353fab573779c0d00448e47d102edd36c792f7b6f233a4d89a7a08384/coverage-7.14.3-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:8427f370ca67db4c975d2a26acfc0e5783ca0b52444dbc50278ace0f35445949", size = 221474, upload-time = "2026-06-22T23:09:51.417Z" },
+ { url = "https://files.pythonhosted.org/packages/69/92/90cf1f1a5c468a9c1b7ba2716e0e205293ad9b02f5f573a6de4318b15ba1/coverage-7.14.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:d8e88f335544a47e22ae2e45b344772925ec65166555c958720d5ed971880891", size = 221738, upload-time = "2026-06-22T23:09:53.487Z" },
+ { url = "https://files.pythonhosted.org/packages/a4/c0/4df964fa539f8399fd7679c09c472d73744de334686fd3f01e3a2465ce4e/coverage-7.14.3-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:beaab199b9e5ceaf5a225e16a9d4df136f2a1eae0a5c20de1e277c8a5225f388", size = 263101, upload-time = "2026-06-22T23:09:55.895Z" },
+ { url = "https://files.pythonhosted.org/packages/06/76/e5d33b2576ae3bf2be2058cd1cae57774b61e400f2c3c58f3783dc2ffb4a/coverage-7.14.3-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b3ff255799f5a1676c71c1c32ec01fd043aa09d57b3d95764b24992757184784", size = 265225, upload-time = "2026-06-22T23:09:57.904Z" },
+ { url = "https://files.pythonhosted.org/packages/61/d2/e52419afe391a39ba27fdefaf0737d8e34bf03faef6ab3b3006545bbd0d0/coverage-7.14.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:878832eaac515b62decfa76965aed558775f86bf1fc8cca76993c0c84ae31aed", size = 267643, upload-time = "2026-06-22T23:09:59.938Z" },
+ { url = "https://files.pythonhosted.org/packages/58/7a/f2625d8d5006b6b20fba5afaef00b24a763fe96476ea798a3076cbc1f84e/coverage-7.14.3-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:611e62cb9386096d81b63e0a05330750268617231e7bd598e1fe77482a2c58a5", size = 268762, upload-time = "2026-06-22T23:10:01.943Z" },
+ { url = "https://files.pythonhosted.org/packages/7d/bf/908024006bba57127354d74e938954b9c3cd765cc2e0412dc9c37b415cda/coverage-7.14.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:02c41de2a88011b893050fc9830267d927a50a215f7ad5ec17349db7090ccf26", size = 262208, upload-time = "2026-06-22T23:10:03.954Z" },
+ { url = "https://files.pythonhosted.org/packages/34/a0/d4f9296441b909817442fdb26bd77a698f08272ec683a7394b00eb2e47a0/coverage-7.14.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:526ce9721116af23b1065089f0b75046fe521e7772ab94b641cd66b7a0421889", size = 265096, upload-time = "2026-06-22T23:10:05.936Z" },
+ { url = "https://files.pythonhosted.org/packages/e8/da/4ae4f3f4e477b56a4ce1e5c48a35eff38a94b50130ce5bdc897024741cfc/coverage-7.14.3-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:e4ed44705ca4bead6fc977a8b741f2145608289b33c8a9b42a95d0f15aedbf4d", size = 262699, upload-time = "2026-06-22T23:10:07.973Z" },
+ { url = "https://files.pythonhosted.org/packages/d8/7a/6927148073ff32856d78baa77b4ddc07a9be7e90020f9db0661c4ca523a1/coverage-7.14.3-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:2415902f385a23dcc4ccd26e0ba803249a169af6a930c003a4c715eeb9a5444e", size = 266433, upload-time = "2026-06-22T23:10:10.145Z" },
+ { url = "https://files.pythonhosted.org/packages/f7/a7/774f658dbe9c4c3f5daa86a87e0459ac3832e4e3cc67affe078547f727b9/coverage-7.14.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:b75ee850fc2d7c831e883220c445b035f2224de2ba6103f1e56dbd237ab913f7", size = 261547, upload-time = "2026-06-22T23:10:12.191Z" },
+ { url = "https://files.pythonhosted.org/packages/3d/14/a0c18c0376c43cbf973f43ef6ca20019c950597180e6396232f7b6a27102/coverage-7.14.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:dc9b4e35e7c3920e925ba7f14886fd5fbe481232754624e832ddba66c7535635", size = 263859, upload-time = "2026-06-22T23:10:14.492Z" },
+ { url = "https://files.pythonhosted.org/packages/10/ac/43a3d0f460af524b131a6191805bc5d18b806ab4e828fbf82e8c8c3af446/coverage-7.14.3-cp314-cp314t-win32.whl", hash = "sha256:7b27c822a8161afbe48e99f1adfb098d270ae7e0f7d7b0555ce110529bdb69cc", size = 223250, upload-time = "2026-06-22T23:10:16.758Z" },
+ { url = "https://files.pythonhosted.org/packages/3f/5f/d5e5c56b0712e96ce8f69fe7dbf229ff938b437bc50862743c8a0d2cea84/coverage-7.14.3-cp314-cp314t-win_amd64.whl", hash = "sha256:39e1dbbb6ff2c338e0196a482558a792a1de3aa64261196f5cdb3da016ad9cda", size = 224082, upload-time = "2026-06-22T23:10:19.23Z" },
+ { url = "https://files.pythonhosted.org/packages/62/35/947cbd5be1d3bcbbdc43d6791de8a56c6501903311d42915ae06a82815f0/coverage-7.14.3-cp314-cp314t-win_arm64.whl", hash = "sha256:68520c90babfa2d560eca6d497921ed3a4f469623bd709733124491b2aa8ef3f", size = 223400, upload-time = "2026-06-22T23:10:21.24Z" },
+ { url = "https://files.pythonhosted.org/packages/eb/e3/a0aa32bfa3a081951f60a23bc0e7b512891ef0eecda1153cf1d8ba36c6b1/coverage-7.14.3-py3-none-any.whl", hash = "sha256:fb7e18afb6e903c1a92401a2f0501ac277dca527bb9ca6fe1f691a8a0026a0e8", size = 212469, upload-time = "2026-06-22T23:10:23.405Z" },
+]
+
+[[package]]
+name = "cycler"
+version = "0.12.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/a9/95/a3dbbb5028f35eafb79008e7522a75244477d2838f38cbb722248dabc2a8/cycler-0.12.1.tar.gz", hash = "sha256:88bb128f02ba341da8ef447245a9e138fae777f6a23943da4540077d3601eb1c", size = 7615, upload-time = "2023-10-07T05:32:18.335Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/e7/05/c19819d5e3d95294a6f5947fb9b9629efb316b96de511b418c53d245aae6/cycler-0.12.1-py3-none-any.whl", hash = "sha256:85cef7cff222d8644161529808465972e51340599459b8ac3ccbac5a854e0d30", size = 8321, upload-time = "2023-10-07T05:32:16.783Z" },
+]
+
+[[package]]
+name = "datashader"
+version = "0.19.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "colorcet" },
+ { name = "multipledispatch" },
+ { name = "numba" },
+ { name = "numpy" },
+ { name = "packaging" },
+ { name = "pandas" },
+ { name = "param" },
+ { name = "pyct" },
+ { name = "requests" },
+ { name = "scipy" },
+ { name = "toolz" },
+ { name = "xarray" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/53/4b/a9141f286f0c01685e9715ba3404769a6138e74575c4d5ccbaa95a22c3bd/datashader-0.19.1.tar.gz", hash = "sha256:f62d880a4a431813f9bb3959e565feda79c1634f889aaadcf948cb0d0c114cdd", size = 10581968, upload-time = "2026-05-19T07:53:06.574Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/cb/41/247627c8b9fef5c605d00546b85771a8fe42975b9616a557cead5468789b/datashader-0.19.1-py3-none-any.whl", hash = "sha256:7ce7154ff3ed070607f429355f57002fee5a17964e47c0b6447eeafe8cef9c82", size = 10715897, upload-time = "2026-05-19T07:53:03.431Z" },
+]
+
+[[package]]
+name = "debugpy"
+version = "1.8.21"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/f2/aa/12037145b7a56eaa5b29b41872f7a21b538e807e13f32c4d3c46e59be084/debugpy-1.8.21.tar.gz", hash = "sha256:a3c53278e84c94e11bd87c53970ec391d1a67396c8b22609fcac576520e611a6", size = 1697577, upload-time = "2026-06-01T19:30:35.156Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/a2/df/bf625547431a9cadc9f4cbfeda38866e2b17f6aed147b625377e87834449/debugpy-1.8.21-cp312-cp312-macosx_15_0_universal2.whl", hash = "sha256:9f96713896f39c3dff0ee841f47320c3f2983d33c341e009361bb0ebc79adc4e", size = 2483609, upload-time = "2026-06-01T19:30:50.794Z" },
+ { url = "https://files.pythonhosted.org/packages/bf/09/59324b903599031ff9faaec1758292409f6561a0ec2492fe4b703327705a/debugpy-1.8.21-cp312-cp312-manylinux_2_34_x86_64.whl", hash = "sha256:c193d474f0a211191f2b4449d2d06157c689013035bd952f3b617e0ef422b176", size = 3968900, upload-time = "2026-06-01T19:30:52.341Z" },
+ { url = "https://files.pythonhosted.org/packages/14/cd/27f65b805d7fe005c44e1a36b9183ecdfbcdbf9d3e721a5115d461ecc7ee/debugpy-1.8.21-cp312-cp312-win32.whl", hash = "sha256:4743373c1cac7f9e74a1b9915bf1dbe0e900eca657ffb170ae07ac8363205ae9", size = 5336340, upload-time = "2026-06-01T19:30:54.047Z" },
+ { url = "https://files.pythonhosted.org/packages/77/1d/c84e30c0c674184948b66f076ab271c01d940618a2824c23cd035a27bc20/debugpy-1.8.21-cp312-cp312-win_amd64.whl", hash = "sha256:bd7ba9dd3daa7c2f942c6ca8d4695a16bf9ac16b63615261c7982bc74f7ed20c", size = 5374751, upload-time = "2026-06-01T19:30:55.891Z" },
+ { url = "https://files.pythonhosted.org/packages/77/6b/d817e1f8cc77aa055d37fba092e0febfdff40fe652d8d53d4cd7a86ad98d/debugpy-1.8.21-cp313-cp313-macosx_15_0_universal2.whl", hash = "sha256:13678151fc401e2d68c9880b91e28714f797d40422994572b24560ef80910a88", size = 2477398, upload-time = "2026-06-01T19:30:57.644Z" },
+ { url = "https://files.pythonhosted.org/packages/48/57/412421516afc3055fa577516f00beec3d663f9b0ab330639547ae6c57720/debugpy-1.8.21-cp313-cp313-manylinux_2_34_x86_64.whl", hash = "sha256:ecbd158386c31ffe71d46f72d44d56e66331ab9b16cad649156d514368f23ab2", size = 3962096, upload-time = "2026-06-01T19:30:59.235Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/62/2c616337cf6ba7b07ebbc97f02c6c945a8e2f76b365e33ee809c32ee36d1/debugpy-1.8.21-cp313-cp313-win32.whl", hash = "sha256:2c2ae706dec41d99a9ca1f7ebc987a83e65578363be6f6b3ac9067504917fae1", size = 5336288, upload-time = "2026-06-01T19:31:00.79Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/99/9175103392f84c4b1bf7622888cdc68da07f0ff7d9e581266428f6776033/debugpy-1.8.21-cp313-cp313-win_amd64.whl", hash = "sha256:aa648733047443eb1d07682c4ef287d36a54507b643ffdf38b09a3ef002c72a0", size = 5376567, upload-time = "2026-06-01T19:31:02.56Z" },
+ { url = "https://files.pythonhosted.org/packages/ce/3d/f4bbb323a548bfab2af3d6b4ffd9bf22636e55956a1285d317a1de643aad/debugpy-1.8.21-cp314-cp314-macosx_15_0_universal2.whl", hash = "sha256:9bb2a685287a2ac9b181cde89edcec64845cb51de7faaa75badb9a698bc24782", size = 2477209, upload-time = "2026-06-01T19:31:04.157Z" },
+ { url = "https://files.pythonhosted.org/packages/8c/2d/6e7ec524984a1702777868de49a4c53202bddac2a432a76a093469587750/debugpy-1.8.21-cp314-cp314-manylinux_2_34_x86_64.whl", hash = "sha256:3d6922439bf33fd38a3e2c447869ebc7b97da5cd3d329ff1ef9bc06c4903437e", size = 3927115, upload-time = "2026-06-01T19:31:05.863Z" },
+ { url = "https://files.pythonhosted.org/packages/97/47/d1aa6d64005a98a9144647d99306b419396f9ad7bf1d73c119e17a81fb4d/debugpy-1.8.21-cp314-cp314-win32.whl", hash = "sha256:15d4963bd5ffa48f0da0947fd06757fa7621945048a14ad7705431566d3c0e7c", size = 5336724, upload-time = "2026-06-01T19:31:07.711Z" },
+ { url = "https://files.pythonhosted.org/packages/5f/67/b905b90d163af11878c1af8abafa4a25206335e112e284e413454543a6da/debugpy-1.8.21-cp314-cp314-win_amd64.whl", hash = "sha256:fe0744a12353406de0ae8ccff0d0a4a666f00801a3db8fd04e7a5f761cd520e8", size = 5373803, upload-time = "2026-06-01T19:31:09.469Z" },
+ { url = "https://files.pythonhosted.org/packages/95/51/67e7cf11a53e40694f720457d5b3a1cdaaa3d5a9a633e482f225456b93ff/debugpy-1.8.21-py2.py3-none-any.whl", hash = "sha256:b1e37d333663c8851516a47364ef473da127f9caebe4417e6df6f5825a7e9a92", size = 5352888, upload-time = "2026-06-01T19:31:25.186Z" },
+]
+
+[[package]]
+name = "decorator"
+version = "5.3.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/60/8b/32f9823da46cde7df2087faa08cd98d01b908f8dcab982cdba9c84e85355/decorator-5.3.1.tar.gz", hash = "sha256:4cbcdd55a6efadb9dbea26b858f4fb3264567b52d69ca0d25b721b553f60ea82", size = 58084, upload-time = "2026-05-18T06:03:28.057Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/05/7f/798705f5296a58ca505d600456748d1be48078eac8a7050d8a98bc9edb89/decorator-5.3.1-py3-none-any.whl", hash = "sha256:f47fe6fdbd2edd623ecfe36875d37aba411624e2670dd395dddae1358689bb3c", size = 10365, upload-time = "2026-05-18T06:03:26.517Z" },
+]
+
+[[package]]
+name = "docutils"
+version = "0.22.4"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ae/b6/03bb70946330e88ffec97aefd3ea75ba575cb2e762061e0e62a213befee8/docutils-0.22.4.tar.gz", hash = "sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968", size = 2291750, upload-time = "2025-12-18T19:00:26.443Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/02/10/5da547df7a391dcde17f59520a231527b8571e6f46fc8efb02ccb370ab12/docutils-0.22.4-py3-none-any.whl", hash = "sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de", size = 633196, upload-time = "2025-12-18T19:00:18.077Z" },
+]
+
+[[package]]
+name = "donfig"
+version = "0.8.1.post1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "pyyaml" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/25/71/80cc718ff6d7abfbabacb1f57aaa42e9c1552bfdd01e64ddd704e4a03638/donfig-0.8.1.post1.tar.gz", hash = "sha256:3bef3413a4c1c601b585e8d297256d0c1470ea012afa6e8461dc28bfb7c23f52", size = 19506, upload-time = "2024-05-23T14:14:31.513Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/0c/d5/c5db1ea3394c6e1732fb3286b3bd878b59507a8f77d32a2cebda7d7b7cd4/donfig-0.8.1.post1-py3-none-any.whl", hash = "sha256:2a3175ce74a06109ff9307d90a230f81215cbac9a751f4d1c6194644b8204f9d", size = 21592, upload-time = "2024-05-23T14:13:55.283Z" },
+]
+
+[[package]]
+name = "execnet"
+version = "2.1.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/bf/89/780e11f9588d9e7128a3f87788354c7946a9cbb1401ad38a48c4db9a4f07/execnet-2.1.2.tar.gz", hash = "sha256:63d83bfdd9a23e35b9c6a3261412324f964c2ec8dcd8d3c6916ee9373e0befcd", size = 166622, upload-time = "2025-11-12T09:56:37.75Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ab/84/02fc1827e8cdded4aa65baef11296a9bbe595c474f0d6d758af082d849fd/execnet-2.1.2-py3-none-any.whl", hash = "sha256:67fba928dd5a544b783f6056f449e5e3931a5c378b128bc18501f7ea79e296ec", size = 40708, upload-time = "2025-11-12T09:56:36.333Z" },
+]
+
+[[package]]
+name = "executing"
+version = "2.2.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/cc/28/c14e053b6762b1044f34a13aab6859bbf40456d37d23aa286ac24cfd9a5d/executing-2.2.1.tar.gz", hash = "sha256:3632cc370565f6648cc328b32435bd120a1e4ebb20c77e3fdde9a13cd1e533c4", size = 1129488, upload-time = "2025-09-01T09:48:10.866Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/c1/ea/53f2148663b321f21b5a606bd5f191517cf40b7072c0497d3c92c4a13b1e/executing-2.2.1-py2.py3-none-any.whl", hash = "sha256:760643d3452b4d777d295bb167ccc74c64a81df23fb5e08eff250c425a4b2017", size = 28317, upload-time = "2025-09-01T09:48:08.5Z" },
+]
+
+[[package]]
+name = "fastjsonschema"
+version = "2.21.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/20/b5/23b216d9d985a956623b6bd12d4086b60f0059b27799f23016af04a74ea1/fastjsonschema-2.21.2.tar.gz", hash = "sha256:b1eb43748041c880796cd077f1a07c3d94e93ae84bba5ed36800a33554ae05de", size = 374130, upload-time = "2025-08-14T18:49:36.666Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/cb/a8/20d0723294217e47de6d9e2e40fd4a9d2f7c4b6ef974babd482a59743694/fastjsonschema-2.21.2-py3-none-any.whl", hash = "sha256:1c797122d0a86c5cace2e54bf4e819c36223b552017172f32c5c024a6b77e463", size = 24024, upload-time = "2025-08-14T18:49:34.776Z" },
+]
+
+[[package]]
+name = "fonttools"
+version = "4.63.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/84/69/c97f2c18e0db87d2c7b15da1974dace76ae938f1cfa22e2727a648b7ed43/fonttools-4.63.0.tar.gz", hash = "sha256:caeb583deeb5168e694b65cda8b4ee62abedfa66cf88488734466f2366b9c4e0", size = 3597189, upload-time = "2026-05-14T12:04:30.958Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/08/ef/b3c6b9b5be2f82416d73fe2ed2e96e2793cd80e7510bd6a17ca79cdd88ec/fonttools-4.63.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:37dd23e621e3b0aef1baa70a303b80aaf38449632cfc8fd2a55fb285bbccfc02", size = 2881131, upload-time = "2026-05-14T12:03:13.386Z" },
+ { url = "https://files.pythonhosted.org/packages/44/a0/c815bea63117fa63e4e1c01f8a1110d2112fa003f838e6467094ec2432ce/fonttools-4.63.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:a9faff9e0c1f76f9fd55899d2ce785832efebab37eb8ae13995853aef178bef0", size = 2426704, upload-time = "2026-05-14T12:03:15.801Z" },
+ { url = "https://files.pythonhosted.org/packages/44/04/0b91d8e916e92ad1fac9e4624760baf0fd5ff2ead614c2f68fb21373f03f/fonttools-4.63.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ef3048ef05dbb552b89817713d9cac912e00d0fde4a3105c00d29e52e10c89af", size = 5044298, upload-time = "2026-05-14T12:03:18.085Z" },
+ { url = "https://files.pythonhosted.org/packages/77/c7/2342da9830e3e9d4870305ca5d2091d2a83284f2953079b7bdd3b5e029d8/fonttools-4.63.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:58dc6bb86a78d782f00f9190ca02c119cf5bbe2807536e361e18d42019f877d8", size = 4999800, upload-time = "2026-05-14T12:03:20.161Z" },
+ { url = "https://files.pythonhosted.org/packages/e6/6d/67fe16c48d7ce050979b33f47e0d28a318f02da030602e944c34f7a16ef3/fonttools-4.63.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:ee08ebfa58f6e1aeff5697ab9582105bb620008c1caafb681e4c557e7483027b", size = 4982666, upload-time = "2026-05-14T12:03:22.87Z" },
+ { url = "https://files.pythonhosted.org/packages/f2/00/3bbab338c07c71fa56269953845e92c951a61457bbbb0f1022551ea266d9/fonttools-4.63.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:27fdc65af8da6f88b9c6121c47a464cbe359fcfff7ff6fc2d37a1f395d755b78", size = 5133598, upload-time = "2026-05-14T12:03:25.168Z" },
+ { url = "https://files.pythonhosted.org/packages/62/f2/aa27c7f98db5b064883dadcc5283947e81e034de42e22a33675878d98b54/fonttools-4.63.0-cp312-cp312-win32.whl", hash = "sha256:af2fd1664d00a397d75f806985ddb36282091c2131a73a6485c23b4a34722263", size = 2292575, upload-time = "2026-05-14T12:03:27.496Z" },
+ { url = "https://files.pythonhosted.org/packages/87/36/cccb9bc2a6ab63d1b2980374f0dca72ce95ae267c9b4cfe77455bb70d0d4/fonttools-4.63.0-cp312-cp312-win_amd64.whl", hash = "sha256:59ac449f8cca9b4ffa08d2e7bbadad87ce710d69d1eda5c3c1ce579baa987272", size = 2343211, upload-time = "2026-05-14T12:03:30.057Z" },
+ { url = "https://files.pythonhosted.org/packages/0f/8d/d8fec3dcde2963f8c908fb315e5ff2cd0ac34f82394bbbf73a2aa5145ce3/fonttools-4.63.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:cd7e9857e5e63738b9d9fd707bc1f59c8b09e5177726d23664db393c59bb08bd", size = 2876062, upload-time = "2026-05-14T12:03:32.554Z" },
+ { url = "https://files.pythonhosted.org/packages/ef/71/d935dc54e4ff121bfdd11e08702db63a7e6f25af21d8a3d7b7212df53641/fonttools-4.63.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:c2a2a42198b696a6f48fad91709afb55176e66a5e566131219dba372fb7f8c59", size = 2424594, upload-time = "2026-05-14T12:03:34.86Z" },
+ { url = "https://files.pythonhosted.org/packages/8e/40/e76320afa1df918e146155ef239b1719ee266092e96f5423bfd075affba1/fonttools-4.63.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1e874792a8212b44583ea02189d9e693906b2f78b261f372f95d6c563210ac1d", size = 5024840, upload-time = "2026-05-14T12:03:36.745Z" },
+ { url = "https://files.pythonhosted.org/packages/ce/36/0b805d8c485f872f65a509cbe3b58a5d0d17bee855333b54a150c79d3061/fonttools-4.63.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:22135da48a348785c5e2d5d2d9d6bec5ed44adacbaeb9db12d9493bf6c6bfa68", size = 4975801, upload-time = "2026-05-14T12:03:38.833Z" },
+ { url = "https://files.pythonhosted.org/packages/c8/26/2cee03d0aa083ab022da5c07aff9ed3f689da1defb81ad6917c9627896da/fonttools-4.63.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ccf41f2efdf56994d22d73bef4ced1052161958169428d06ba9724ea9e9a64be", size = 4965009, upload-time = "2026-05-14T12:03:41.494Z" },
+ { url = "https://files.pythonhosted.org/packages/7e/48/cc4b66d9058c0d0982c833fad10127c4b0e9324606aafa41382295ca4102/fonttools-4.63.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:9ced0bd02ac751dd6319b0da88aaef24414e3b0dbc32bb4f24944821a3741a27", size = 5105892, upload-time = "2026-05-14T12:03:43.525Z" },
+ { url = "https://files.pythonhosted.org/packages/d8/1f/a98a30a814b9ddef3a2e706025f90b9e0bc94890e6cb15254bc86547d11a/fonttools-4.63.0-cp313-cp313-win32.whl", hash = "sha256:85be818f5506e8a7753153def2c9550178f0ecae6a47b5e0e8dbb23f7cc90380", size = 2291313, upload-time = "2026-05-14T12:03:45.594Z" },
+ { url = "https://files.pythonhosted.org/packages/92/46/5177b01f3b4abfdd4409f31cca4ab279c9343a26efbe9ec78c97fc612e02/fonttools-4.63.0-cp313-cp313-win_amd64.whl", hash = "sha256:ba04cb5891d4c0c21b6da95eda8d7b090021508a294fff33464fc7d241e0856b", size = 2342299, upload-time = "2026-05-14T12:03:47.414Z" },
+ { url = "https://files.pythonhosted.org/packages/27/d2/23d25e3f247b328be58d04a4c9f894178a0d1eda7d42867cfb388adaf416/fonttools-4.63.0-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:fd1e3094f42d806d3d7c79162fc59e5910fcbe3a7360c385b8da969bc4493745", size = 2875338, upload-time = "2026-05-14T12:03:50.052Z" },
+ { url = "https://files.pythonhosted.org/packages/cd/58/7dfa0c761cb3b2964e2a84c4dc986c926a87de0cb9fb60d5b28ded3f2914/fonttools-4.63.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:6e528da43bc3791085f8cb6141b1d13e459226790240340fcbb4625649238b03", size = 2422661, upload-time = "2026-05-14T12:03:52.154Z" },
+ { url = "https://files.pythonhosted.org/packages/dd/87/64cfa18a7a1621d17b7f4502b2b0ed8a135a90c3db51ea590ee99043e76b/fonttools-4.63.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b2248c5decb223562f7902ff6325077a073f608ee8e33e88ad88db734eb9f49", size = 5010526, upload-time = "2026-05-14T12:03:54.647Z" },
+ { url = "https://files.pythonhosted.org/packages/36/e1/a8933a72c45a87177fbde2696e0d0755c8c9062f8c077a961c6215fa27b1/fonttools-4.63.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:308f957cdeaf8abe4e5f2f124902ef405448af92c90f80e302a3b771c2e6116b", size = 4923946, upload-time = "2026-05-14T12:03:56.984Z" },
+ { url = "https://files.pythonhosted.org/packages/27/60/872e6e233b8c5e8b41413796ff18b7fe479661bd40147e071b450dfad7a1/fonttools-4.63.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:bf00f21eb5fb721dbaf73d1e9da6d02a1af7768f2ebcf9798be98beab8ba90f6", size = 4962489, upload-time = "2026-05-14T12:03:59.443Z" },
+ { url = "https://files.pythonhosted.org/packages/30/c4/83c24f2ec38b90cfda84bf4b1a1f49df80e84a1db4e7ac6e0d41bf23bc39/fonttools-4.63.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:c1aaa4b9c75798400ac043ce04d74e7830376c85095a5a6ed7cba2f17a266bf4", size = 5071870, upload-time = "2026-05-14T12:04:02.122Z" },
+ { url = "https://files.pythonhosted.org/packages/de/40/3ae22b60ff1d41ce0bd044b31238cdc72cef99f28b976f1e128ebd618c9b/fonttools-4.63.0-cp314-cp314-win32.whl", hash = "sha256:22693918177bd9ceabec4736d338045f357769416fc6b0b2508eefef75b08616", size = 2295026, upload-time = "2026-05-14T12:04:04.47Z" },
+ { url = "https://files.pythonhosted.org/packages/c3/d4/98078064ccc76b45cb0f6c002452011e93c4bd26f6850344f0951cc1fe89/fonttools-4.63.0-cp314-cp314-win_amd64.whl", hash = "sha256:7d782fac32985914c351556f68ac0855391572bcd87de50e05970d3cd4c96fc5", size = 2347454, upload-time = "2026-05-14T12:04:06.752Z" },
+ { url = "https://files.pythonhosted.org/packages/49/4e/652d1580c5f4e39f7d103b0c793e4773129ad633dce4addd0cf4dfebde02/fonttools-4.63.0-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:6db5140a60a5d731d21ec076745b40a310607731b0a565b50776393188649001", size = 2958152, upload-time = "2026-05-14T12:04:08.706Z" },
+ { url = "https://files.pythonhosted.org/packages/0e/55/ad864c9a9b219f552eb46b32cd7906c466e5a578ba0c3abfcc0fe7413eb6/fonttools-4.63.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:7d76edbff9014094dbf03bd2d074709dfa6ec7aba13d838c937a2b33d2d6a86e", size = 2460809, upload-time = "2026-05-14T12:04:10.783Z" },
+ { url = "https://files.pythonhosted.org/packages/ea/2b/0aa8db70f18cf52e49b4ed5ecec68547f981160bf5ded3b5aed6faa0a6f9/fonttools-4.63.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0eac00b9118c3c2f87d272e45341871c5b3066baa3c86897fa634a7c3fb59096", size = 5148649, upload-time = "2026-05-14T12:04:12.747Z" },
+ { url = "https://files.pythonhosted.org/packages/7f/63/18e4369c25043096f1048e0c9915951adc4f842bd81c6b18155824d6fa99/fonttools-4.63.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:51394295f1a51de8b5f30bdb1e1b9a4231536c7064ef5c6e211eec19fa36036f", size = 4932147, upload-time = "2026-05-14T12:04:14.806Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/3f/67f3eac2ffd8a98446c5022f8ed3864eac878a5ff7af8df4c8286dba16cc/fonttools-4.63.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:9e12f105d2b6342c559c298afb674006bb2893afc7102dcf8a1b55b0486b4e40", size = 5027237, upload-time = "2026-05-14T12:04:17.675Z" },
+ { url = "https://files.pythonhosted.org/packages/1a/ba/4e6214cb38a7b04779e97bb7636de9a5c7f20af7018d03dee0b64c08510a/fonttools-4.63.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:796f27556dbe094c4824f75ca85267e4df776c79036c8441469a4df37038c196", size = 5053933, upload-time = "2026-05-14T12:04:20.818Z" },
+ { url = "https://files.pythonhosted.org/packages/34/3b/214dcc19ee31d3d38fb5ad2755c11ef0514e5dc300bbaf41c0b69f393799/fonttools-4.63.0-cp314-cp314t-win32.whl", hash = "sha256:948428a275741f0b64b113c955425a953314f4b9ab9997f73a72c83e68e569c8", size = 2359326, upload-time = "2026-05-14T12:04:24.22Z" },
+ { url = "https://files.pythonhosted.org/packages/dd/1e/3ff1a9b523058c2eeb6a9d50f5574e2a738200d0d94107d5bc4105e8da3f/fonttools-4.63.0-cp314-cp314t-win_amd64.whl", hash = "sha256:6d4741eb179121cab9eea4cb2393d24492373a260d7945006358c08cfbf45419", size = 2425829, upload-time = "2026-05-14T12:04:26.829Z" },
+ { url = "https://files.pythonhosted.org/packages/2c/47/c99d5268f354002ce80f8d029cd9d7d872969da1de8b93d32de4dc56d6f4/fonttools-4.63.0-py3-none-any.whl", hash = "sha256:445af2eab030a16b9171ea8bdda7ebf7d96bda2df88ee182a464252f6e05e20d", size = 1164562, upload-time = "2026-05-14T12:04:29.092Z" },
+]
+
+[[package]]
+name = "frozenlist"
+version = "1.8.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/2d/f5/c831fac6cc817d26fd54c7eaccd04ef7e0288806943f7cc5bbf69f3ac1f0/frozenlist-1.8.0.tar.gz", hash = "sha256:3ede829ed8d842f6cd48fc7081d7a41001a56f1f38603f9d49bf3020d59a31ad", size = 45875, upload-time = "2025-10-06T05:38:17.865Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/69/29/948b9aa87e75820a38650af445d2ef2b6b8a6fab1a23b6bb9e4ef0be2d59/frozenlist-1.8.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:78f7b9e5d6f2fdb88cdde9440dc147259b62b9d3b019924def9f6478be254ac1", size = 87782, upload-time = "2025-10-06T05:36:06.649Z" },
+ { url = "https://files.pythonhosted.org/packages/64/80/4f6e318ee2a7c0750ed724fa33a4bdf1eacdc5a39a7a24e818a773cd91af/frozenlist-1.8.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:229bf37d2e4acdaf808fd3f06e854a4a7a3661e871b10dc1f8f1896a3b05f18b", size = 50594, upload-time = "2025-10-06T05:36:07.69Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/94/5c8a2b50a496b11dd519f4a24cb5496cf125681dd99e94c604ccdea9419a/frozenlist-1.8.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f833670942247a14eafbb675458b4e61c82e002a148f49e68257b79296e865c4", size = 50448, upload-time = "2025-10-06T05:36:08.78Z" },
+ { url = "https://files.pythonhosted.org/packages/6a/bd/d91c5e39f490a49df14320f4e8c80161cfcce09f1e2cde1edd16a551abb3/frozenlist-1.8.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:494a5952b1c597ba44e0e78113a7266e656b9794eec897b19ead706bd7074383", size = 242411, upload-time = "2025-10-06T05:36:09.801Z" },
+ { url = "https://files.pythonhosted.org/packages/8f/83/f61505a05109ef3293dfb1ff594d13d64a2324ac3482be2cedc2be818256/frozenlist-1.8.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:96f423a119f4777a4a056b66ce11527366a8bb92f54e541ade21f2374433f6d4", size = 243014, upload-time = "2025-10-06T05:36:11.394Z" },
+ { url = "https://files.pythonhosted.org/packages/d8/cb/cb6c7b0f7d4023ddda30cf56b8b17494eb3a79e3fda666bf735f63118b35/frozenlist-1.8.0-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3462dd9475af2025c31cc61be6652dfa25cbfb56cbbf52f4ccfe029f38decaf8", size = 234909, upload-time = "2025-10-06T05:36:12.598Z" },
+ { url = "https://files.pythonhosted.org/packages/31/c5/cd7a1f3b8b34af009fb17d4123c5a778b44ae2804e3ad6b86204255f9ec5/frozenlist-1.8.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c4c800524c9cd9bac5166cd6f55285957fcfc907db323e193f2afcd4d9abd69b", size = 250049, upload-time = "2025-10-06T05:36:14.065Z" },
+ { url = "https://files.pythonhosted.org/packages/c0/01/2f95d3b416c584a1e7f0e1d6d31998c4a795f7544069ee2e0962a4b60740/frozenlist-1.8.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d6a5df73acd3399d893dafc71663ad22534b5aa4f94e8a2fabfe856c3c1b6a52", size = 256485, upload-time = "2025-10-06T05:36:15.39Z" },
+ { url = "https://files.pythonhosted.org/packages/ce/03/024bf7720b3abaebcff6d0793d73c154237b85bdf67b7ed55e5e9596dc9a/frozenlist-1.8.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:405e8fe955c2280ce66428b3ca55e12b3c4e9c336fb2103a4937e891c69a4a29", size = 237619, upload-time = "2025-10-06T05:36:16.558Z" },
+ { url = "https://files.pythonhosted.org/packages/69/fa/f8abdfe7d76b731f5d8bd217827cf6764d4f1d9763407e42717b4bed50a0/frozenlist-1.8.0-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:908bd3f6439f2fef9e85031b59fd4f1297af54415fb60e4254a95f75b3cab3f3", size = 250320, upload-time = "2025-10-06T05:36:17.821Z" },
+ { url = "https://files.pythonhosted.org/packages/f5/3c/b051329f718b463b22613e269ad72138cc256c540f78a6de89452803a47d/frozenlist-1.8.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:294e487f9ec720bd8ffcebc99d575f7eff3568a08a253d1ee1a0378754b74143", size = 246820, upload-time = "2025-10-06T05:36:19.046Z" },
+ { url = "https://files.pythonhosted.org/packages/0f/ae/58282e8f98e444b3f4dd42448ff36fa38bef29e40d40f330b22e7108f565/frozenlist-1.8.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:74c51543498289c0c43656701be6b077f4b265868fa7f8a8859c197006efb608", size = 250518, upload-time = "2025-10-06T05:36:20.763Z" },
+ { url = "https://files.pythonhosted.org/packages/8f/96/007e5944694d66123183845a106547a15944fbbb7154788cbf7272789536/frozenlist-1.8.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:776f352e8329135506a1d6bf16ac3f87bc25b28e765949282dcc627af36123aa", size = 239096, upload-time = "2025-10-06T05:36:22.129Z" },
+ { url = "https://files.pythonhosted.org/packages/66/bb/852b9d6db2fa40be96f29c0d1205c306288f0684df8fd26ca1951d461a56/frozenlist-1.8.0-cp312-cp312-win32.whl", hash = "sha256:433403ae80709741ce34038da08511d4a77062aa924baf411ef73d1146e74faf", size = 39985, upload-time = "2025-10-06T05:36:23.661Z" },
+ { url = "https://files.pythonhosted.org/packages/b8/af/38e51a553dd66eb064cdf193841f16f077585d4d28394c2fa6235cb41765/frozenlist-1.8.0-cp312-cp312-win_amd64.whl", hash = "sha256:34187385b08f866104f0c0617404c8eb08165ab1272e884abc89c112e9c00746", size = 44591, upload-time = "2025-10-06T05:36:24.958Z" },
+ { url = "https://files.pythonhosted.org/packages/a7/06/1dc65480ab147339fecc70797e9c2f69d9cea9cf38934ce08df070fdb9cb/frozenlist-1.8.0-cp312-cp312-win_arm64.whl", hash = "sha256:fe3c58d2f5db5fbd18c2987cba06d51b0529f52bc3a6cdc33d3f4eab725104bd", size = 40102, upload-time = "2025-10-06T05:36:26.333Z" },
+ { url = "https://files.pythonhosted.org/packages/2d/40/0832c31a37d60f60ed79e9dfb5a92e1e2af4f40a16a29abcc7992af9edff/frozenlist-1.8.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:8d92f1a84bb12d9e56f818b3a746f3efba93c1b63c8387a73dde655e1e42282a", size = 85717, upload-time = "2025-10-06T05:36:27.341Z" },
+ { url = "https://files.pythonhosted.org/packages/30/ba/b0b3de23f40bc55a7057bd38434e25c34fa48e17f20ee273bbde5e0650f3/frozenlist-1.8.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:96153e77a591c8adc2ee805756c61f59fef4cf4073a9275ee86fe8cba41241f7", size = 49651, upload-time = "2025-10-06T05:36:28.855Z" },
+ { url = "https://files.pythonhosted.org/packages/0c/ab/6e5080ee374f875296c4243c381bbdef97a9ac39c6e3ce1d5f7d42cb78d6/frozenlist-1.8.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:f21f00a91358803399890ab167098c131ec2ddd5f8f5fd5fe9c9f2c6fcd91e40", size = 49417, upload-time = "2025-10-06T05:36:29.877Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/4e/e4691508f9477ce67da2015d8c00acd751e6287739123113a9fca6f1604e/frozenlist-1.8.0-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:fb30f9626572a76dfe4293c7194a09fb1fe93ba94c7d4f720dfae3b646b45027", size = 234391, upload-time = "2025-10-06T05:36:31.301Z" },
+ { url = "https://files.pythonhosted.org/packages/40/76/c202df58e3acdf12969a7895fd6f3bc016c642e6726aa63bd3025e0fc71c/frozenlist-1.8.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:eaa352d7047a31d87dafcacbabe89df0aa506abb5b1b85a2fb91bc3faa02d822", size = 233048, upload-time = "2025-10-06T05:36:32.531Z" },
+ { url = "https://files.pythonhosted.org/packages/f9/c0/8746afb90f17b73ca5979c7a3958116e105ff796e718575175319b5bb4ce/frozenlist-1.8.0-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:03ae967b4e297f58f8c774c7eabcce57fe3c2434817d4385c50661845a058121", size = 226549, upload-time = "2025-10-06T05:36:33.706Z" },
+ { url = "https://files.pythonhosted.org/packages/7e/eb/4c7eefc718ff72f9b6c4893291abaae5fbc0c82226a32dcd8ef4f7a5dbef/frozenlist-1.8.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f6292f1de555ffcc675941d65fffffb0a5bcd992905015f85d0592201793e0e5", size = 239833, upload-time = "2025-10-06T05:36:34.947Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/4e/e5c02187cf704224f8b21bee886f3d713ca379535f16893233b9d672ea71/frozenlist-1.8.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:29548f9b5b5e3460ce7378144c3010363d8035cea44bc0bf02d57f5a685e084e", size = 245363, upload-time = "2025-10-06T05:36:36.534Z" },
+ { url = "https://files.pythonhosted.org/packages/1f/96/cb85ec608464472e82ad37a17f844889c36100eed57bea094518bf270692/frozenlist-1.8.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ec3cc8c5d4084591b4237c0a272cc4f50a5b03396a47d9caaf76f5d7b38a4f11", size = 229314, upload-time = "2025-10-06T05:36:38.582Z" },
+ { url = "https://files.pythonhosted.org/packages/5d/6f/4ae69c550e4cee66b57887daeebe006fe985917c01d0fff9caab9883f6d0/frozenlist-1.8.0-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:517279f58009d0b1f2e7c1b130b377a349405da3f7621ed6bfae50b10adf20c1", size = 243365, upload-time = "2025-10-06T05:36:40.152Z" },
+ { url = "https://files.pythonhosted.org/packages/7a/58/afd56de246cf11780a40a2c28dc7cbabbf06337cc8ddb1c780a2d97e88d8/frozenlist-1.8.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:db1e72ede2d0d7ccb213f218df6a078a9c09a7de257c2fe8fcef16d5925230b1", size = 237763, upload-time = "2025-10-06T05:36:41.355Z" },
+ { url = "https://files.pythonhosted.org/packages/cb/36/cdfaf6ed42e2644740d4a10452d8e97fa1c062e2a8006e4b09f1b5fd7d63/frozenlist-1.8.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:b4dec9482a65c54a5044486847b8a66bf10c9cb4926d42927ec4e8fd5db7fed8", size = 240110, upload-time = "2025-10-06T05:36:42.716Z" },
+ { url = "https://files.pythonhosted.org/packages/03/a8/9ea226fbefad669f11b52e864c55f0bd57d3c8d7eb07e9f2e9a0b39502e1/frozenlist-1.8.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:21900c48ae04d13d416f0e1e0c4d81f7931f73a9dfa0b7a8746fb2fe7dd970ed", size = 233717, upload-time = "2025-10-06T05:36:44.251Z" },
+ { url = "https://files.pythonhosted.org/packages/1e/0b/1b5531611e83ba7d13ccc9988967ea1b51186af64c42b7a7af465dcc9568/frozenlist-1.8.0-cp313-cp313-win32.whl", hash = "sha256:8b7b94a067d1c504ee0b16def57ad5738701e4ba10cec90529f13fa03c833496", size = 39628, upload-time = "2025-10-06T05:36:45.423Z" },
+ { url = "https://files.pythonhosted.org/packages/d8/cf/174c91dbc9cc49bc7b7aab74d8b734e974d1faa8f191c74af9b7e80848e6/frozenlist-1.8.0-cp313-cp313-win_amd64.whl", hash = "sha256:878be833caa6a3821caf85eb39c5ba92d28e85df26d57afb06b35b2efd937231", size = 43882, upload-time = "2025-10-06T05:36:46.796Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/17/502cd212cbfa96eb1388614fe39a3fc9ab87dbbe042b66f97acb57474834/frozenlist-1.8.0-cp313-cp313-win_arm64.whl", hash = "sha256:44389d135b3ff43ba8cc89ff7f51f5a0bb6b63d829c8300f79a2fe4fe61bcc62", size = 39676, upload-time = "2025-10-06T05:36:47.8Z" },
+ { url = "https://files.pythonhosted.org/packages/d2/5c/3bbfaa920dfab09e76946a5d2833a7cbdf7b9b4a91c714666ac4855b88b4/frozenlist-1.8.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:e25ac20a2ef37e91c1b39938b591457666a0fa835c7783c3a8f33ea42870db94", size = 89235, upload-time = "2025-10-06T05:36:48.78Z" },
+ { url = "https://files.pythonhosted.org/packages/d2/d6/f03961ef72166cec1687e84e8925838442b615bd0b8854b54923ce5b7b8a/frozenlist-1.8.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:07cdca25a91a4386d2e76ad992916a85038a9b97561bf7a3fd12d5d9ce31870c", size = 50742, upload-time = "2025-10-06T05:36:49.837Z" },
+ { url = "https://files.pythonhosted.org/packages/1e/bb/a6d12b7ba4c3337667d0e421f7181c82dda448ce4e7ad7ecd249a16fa806/frozenlist-1.8.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:4e0c11f2cc6717e0a741f84a527c52616140741cd812a50422f83dc31749fb52", size = 51725, upload-time = "2025-10-06T05:36:50.851Z" },
+ { url = "https://files.pythonhosted.org/packages/bc/71/d1fed0ffe2c2ccd70b43714c6cab0f4188f09f8a67a7914a6b46ee30f274/frozenlist-1.8.0-cp313-cp313t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b3210649ee28062ea6099cfda39e147fa1bc039583c8ee4481cb7811e2448c51", size = 284533, upload-time = "2025-10-06T05:36:51.898Z" },
+ { url = "https://files.pythonhosted.org/packages/c9/1f/fb1685a7b009d89f9bf78a42d94461bc06581f6e718c39344754a5d9bada/frozenlist-1.8.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:581ef5194c48035a7de2aefc72ac6539823bb71508189e5de01d60c9dcd5fa65", size = 292506, upload-time = "2025-10-06T05:36:53.101Z" },
+ { url = "https://files.pythonhosted.org/packages/e6/3b/b991fe1612703f7e0d05c0cf734c1b77aaf7c7d321df4572e8d36e7048c8/frozenlist-1.8.0-cp313-cp313t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3ef2d026f16a2b1866e1d86fc4e1291e1ed8a387b2c333809419a2f8b3a77b82", size = 274161, upload-time = "2025-10-06T05:36:54.309Z" },
+ { url = "https://files.pythonhosted.org/packages/ca/ec/c5c618767bcdf66e88945ec0157d7f6c4a1322f1473392319b7a2501ded7/frozenlist-1.8.0-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:5500ef82073f599ac84d888e3a8c1f77ac831183244bfd7f11eaa0289fb30714", size = 294676, upload-time = "2025-10-06T05:36:55.566Z" },
+ { url = "https://files.pythonhosted.org/packages/7c/ce/3934758637d8f8a88d11f0585d6495ef54b2044ed6ec84492a91fa3b27aa/frozenlist-1.8.0-cp313-cp313t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:50066c3997d0091c411a66e710f4e11752251e6d2d73d70d8d5d4c76442a199d", size = 300638, upload-time = "2025-10-06T05:36:56.758Z" },
+ { url = "https://files.pythonhosted.org/packages/fc/4f/a7e4d0d467298f42de4b41cbc7ddaf19d3cfeabaf9ff97c20c6c7ee409f9/frozenlist-1.8.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:5c1c8e78426e59b3f8005e9b19f6ff46e5845895adbde20ece9218319eca6506", size = 283067, upload-time = "2025-10-06T05:36:57.965Z" },
+ { url = "https://files.pythonhosted.org/packages/dc/48/c7b163063d55a83772b268e6d1affb960771b0e203b632cfe09522d67ea5/frozenlist-1.8.0-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:eefdba20de0d938cec6a89bd4d70f346a03108a19b9df4248d3cf0d88f1b0f51", size = 292101, upload-time = "2025-10-06T05:36:59.237Z" },
+ { url = "https://files.pythonhosted.org/packages/9f/d0/2366d3c4ecdc2fd391e0afa6e11500bfba0ea772764d631bbf82f0136c9d/frozenlist-1.8.0-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:cf253e0e1c3ceb4aaff6df637ce033ff6535fb8c70a764a8f46aafd3d6ab798e", size = 289901, upload-time = "2025-10-06T05:37:00.811Z" },
+ { url = "https://files.pythonhosted.org/packages/b8/94/daff920e82c1b70e3618a2ac39fbc01ae3e2ff6124e80739ce5d71c9b920/frozenlist-1.8.0-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:032efa2674356903cd0261c4317a561a6850f3ac864a63fc1583147fb05a79b0", size = 289395, upload-time = "2025-10-06T05:37:02.115Z" },
+ { url = "https://files.pythonhosted.org/packages/e3/20/bba307ab4235a09fdcd3cc5508dbabd17c4634a1af4b96e0f69bfe551ebd/frozenlist-1.8.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:6da155091429aeba16851ecb10a9104a108bcd32f6c1642867eadaee401c1c41", size = 283659, upload-time = "2025-10-06T05:37:03.711Z" },
+ { url = "https://files.pythonhosted.org/packages/fd/00/04ca1c3a7a124b6de4f8a9a17cc2fcad138b4608e7a3fc5877804b8715d7/frozenlist-1.8.0-cp313-cp313t-win32.whl", hash = "sha256:0f96534f8bfebc1a394209427d0f8a63d343c9779cda6fc25e8e121b5fd8555b", size = 43492, upload-time = "2025-10-06T05:37:04.915Z" },
+ { url = "https://files.pythonhosted.org/packages/59/5e/c69f733a86a94ab10f68e496dc6b7e8bc078ebb415281d5698313e3af3a1/frozenlist-1.8.0-cp313-cp313t-win_amd64.whl", hash = "sha256:5d63a068f978fc69421fb0e6eb91a9603187527c86b7cd3f534a5b77a592b888", size = 48034, upload-time = "2025-10-06T05:37:06.343Z" },
+ { url = "https://files.pythonhosted.org/packages/16/6c/be9d79775d8abe79b05fa6d23da99ad6e7763a1d080fbae7290b286093fd/frozenlist-1.8.0-cp313-cp313t-win_arm64.whl", hash = "sha256:bf0a7e10b077bf5fb9380ad3ae8ce20ef919a6ad93b4552896419ac7e1d8e042", size = 41749, upload-time = "2025-10-06T05:37:07.431Z" },
+ { url = "https://files.pythonhosted.org/packages/f1/c8/85da824b7e7b9b6e7f7705b2ecaf9591ba6f79c1177f324c2735e41d36a2/frozenlist-1.8.0-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:cee686f1f4cadeb2136007ddedd0aaf928ab95216e7691c63e50a8ec066336d0", size = 86127, upload-time = "2025-10-06T05:37:08.438Z" },
+ { url = "https://files.pythonhosted.org/packages/8e/e8/a1185e236ec66c20afd72399522f142c3724c785789255202d27ae992818/frozenlist-1.8.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:119fb2a1bd47307e899c2fac7f28e85b9a543864df47aa7ec9d3c1b4545f096f", size = 49698, upload-time = "2025-10-06T05:37:09.48Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/93/72b1736d68f03fda5fdf0f2180fb6caaae3894f1b854d006ac61ecc727ee/frozenlist-1.8.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:4970ece02dbc8c3a92fcc5228e36a3e933a01a999f7094ff7c23fbd2beeaa67c", size = 49749, upload-time = "2025-10-06T05:37:10.569Z" },
+ { url = "https://files.pythonhosted.org/packages/a7/b2/fabede9fafd976b991e9f1b9c8c873ed86f202889b864756f240ce6dd855/frozenlist-1.8.0-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:cba69cb73723c3f329622e34bdbf5ce1f80c21c290ff04256cff1cd3c2036ed2", size = 231298, upload-time = "2025-10-06T05:37:11.993Z" },
+ { url = "https://files.pythonhosted.org/packages/3a/3b/d9b1e0b0eed36e70477ffb8360c49c85c8ca8ef9700a4e6711f39a6e8b45/frozenlist-1.8.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:778a11b15673f6f1df23d9586f83c4846c471a8af693a22e066508b77d201ec8", size = 232015, upload-time = "2025-10-06T05:37:13.194Z" },
+ { url = "https://files.pythonhosted.org/packages/dc/94/be719d2766c1138148564a3960fc2c06eb688da592bdc25adcf856101be7/frozenlist-1.8.0-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:0325024fe97f94c41c08872db482cf8ac4800d80e79222c6b0b7b162d5b13686", size = 225038, upload-time = "2025-10-06T05:37:14.577Z" },
+ { url = "https://files.pythonhosted.org/packages/e4/09/6712b6c5465f083f52f50cf74167b92d4ea2f50e46a9eea0523d658454ae/frozenlist-1.8.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:97260ff46b207a82a7567b581ab4190bd4dfa09f4db8a8b49d1a958f6aa4940e", size = 240130, upload-time = "2025-10-06T05:37:15.781Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/d4/cd065cdcf21550b54f3ce6a22e143ac9e4836ca42a0de1022da8498eac89/frozenlist-1.8.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:54b2077180eb7f83dd52c40b2750d0a9f175e06a42e3213ce047219de902717a", size = 242845, upload-time = "2025-10-06T05:37:17.037Z" },
+ { url = "https://files.pythonhosted.org/packages/62/c3/f57a5c8c70cd1ead3d5d5f776f89d33110b1addae0ab010ad774d9a44fb9/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:2f05983daecab868a31e1da44462873306d3cbfd76d1f0b5b69c473d21dbb128", size = 229131, upload-time = "2025-10-06T05:37:18.221Z" },
+ { url = "https://files.pythonhosted.org/packages/6c/52/232476fe9cb64f0742f3fde2b7d26c1dac18b6d62071c74d4ded55e0ef94/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:33f48f51a446114bc5d251fb2954ab0164d5be02ad3382abcbfe07e2531d650f", size = 240542, upload-time = "2025-10-06T05:37:19.771Z" },
+ { url = "https://files.pythonhosted.org/packages/5f/85/07bf3f5d0fb5414aee5f47d33c6f5c77bfe49aac680bfece33d4fdf6a246/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:154e55ec0655291b5dd1b8731c637ecdb50975a2ae70c606d100750a540082f7", size = 237308, upload-time = "2025-10-06T05:37:20.969Z" },
+ { url = "https://files.pythonhosted.org/packages/11/99/ae3a33d5befd41ac0ca2cc7fd3aa707c9c324de2e89db0e0f45db9a64c26/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:4314debad13beb564b708b4a496020e5306c7333fa9a3ab90374169a20ffab30", size = 238210, upload-time = "2025-10-06T05:37:22.252Z" },
+ { url = "https://files.pythonhosted.org/packages/b2/60/b1d2da22f4970e7a155f0adde9b1435712ece01b3cd45ba63702aea33938/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:073f8bf8becba60aa931eb3bc420b217bb7d5b8f4750e6f8b3be7f3da85d38b7", size = 231972, upload-time = "2025-10-06T05:37:23.5Z" },
+ { url = "https://files.pythonhosted.org/packages/3f/ab/945b2f32de889993b9c9133216c068b7fcf257d8595a0ac420ac8677cab0/frozenlist-1.8.0-cp314-cp314-win32.whl", hash = "sha256:bac9c42ba2ac65ddc115d930c78d24ab8d4f465fd3fc473cdedfccadb9429806", size = 40536, upload-time = "2025-10-06T05:37:25.581Z" },
+ { url = "https://files.pythonhosted.org/packages/59/ad/9caa9b9c836d9ad6f067157a531ac48b7d36499f5036d4141ce78c230b1b/frozenlist-1.8.0-cp314-cp314-win_amd64.whl", hash = "sha256:3e0761f4d1a44f1d1a47996511752cf3dcec5bbdd9cc2b4fe595caf97754b7a0", size = 44330, upload-time = "2025-10-06T05:37:26.928Z" },
+ { url = "https://files.pythonhosted.org/packages/82/13/e6950121764f2676f43534c555249f57030150260aee9dcf7d64efda11dd/frozenlist-1.8.0-cp314-cp314-win_arm64.whl", hash = "sha256:d1eaff1d00c7751b7c6662e9c5ba6eb2c17a2306ba5e2a37f24ddf3cc953402b", size = 40627, upload-time = "2025-10-06T05:37:28.075Z" },
+ { url = "https://files.pythonhosted.org/packages/c0/c7/43200656ecc4e02d3f8bc248df68256cd9572b3f0017f0a0c4e93440ae23/frozenlist-1.8.0-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:d3bb933317c52d7ea5004a1c442eef86f426886fba134ef8cf4226ea6ee1821d", size = 89238, upload-time = "2025-10-06T05:37:29.373Z" },
+ { url = "https://files.pythonhosted.org/packages/d1/29/55c5f0689b9c0fb765055629f472c0de484dcaf0acee2f7707266ae3583c/frozenlist-1.8.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:8009897cdef112072f93a0efdce29cd819e717fd2f649ee3016efd3cd885a7ed", size = 50738, upload-time = "2025-10-06T05:37:30.792Z" },
+ { url = "https://files.pythonhosted.org/packages/ba/7d/b7282a445956506fa11da8c2db7d276adcbf2b17d8bb8407a47685263f90/frozenlist-1.8.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:2c5dcbbc55383e5883246d11fd179782a9d07a986c40f49abe89ddf865913930", size = 51739, upload-time = "2025-10-06T05:37:32.127Z" },
+ { url = "https://files.pythonhosted.org/packages/62/1c/3d8622e60d0b767a5510d1d3cf21065b9db874696a51ea6d7a43180a259c/frozenlist-1.8.0-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:39ecbc32f1390387d2aa4f5a995e465e9e2f79ba3adcac92d68e3e0afae6657c", size = 284186, upload-time = "2025-10-06T05:37:33.21Z" },
+ { url = "https://files.pythonhosted.org/packages/2d/14/aa36d5f85a89679a85a1d44cd7a6657e0b1c75f61e7cad987b203d2daca8/frozenlist-1.8.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92db2bf818d5cc8d9c1f1fc56b897662e24ea5adb36ad1f1d82875bd64e03c24", size = 292196, upload-time = "2025-10-06T05:37:36.107Z" },
+ { url = "https://files.pythonhosted.org/packages/05/23/6bde59eb55abd407d34f77d39a5126fb7b4f109a3f611d3929f14b700c66/frozenlist-1.8.0-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:2dc43a022e555de94c3b68a4ef0b11c4f747d12c024a520c7101709a2144fb37", size = 273830, upload-time = "2025-10-06T05:37:37.663Z" },
+ { url = "https://files.pythonhosted.org/packages/d2/3f/22cff331bfad7a8afa616289000ba793347fcd7bc275f3b28ecea2a27909/frozenlist-1.8.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cb89a7f2de3602cfed448095bab3f178399646ab7c61454315089787df07733a", size = 294289, upload-time = "2025-10-06T05:37:39.261Z" },
+ { url = "https://files.pythonhosted.org/packages/a4/89/5b057c799de4838b6c69aa82b79705f2027615e01be996d2486a69ca99c4/frozenlist-1.8.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:33139dc858c580ea50e7e60a1b0ea003efa1fd42e6ec7fdbad78fff65fad2fd2", size = 300318, upload-time = "2025-10-06T05:37:43.213Z" },
+ { url = "https://files.pythonhosted.org/packages/30/de/2c22ab3eb2a8af6d69dc799e48455813bab3690c760de58e1bf43b36da3e/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:168c0969a329b416119507ba30b9ea13688fafffac1b7822802537569a1cb0ef", size = 282814, upload-time = "2025-10-06T05:37:45.337Z" },
+ { url = "https://files.pythonhosted.org/packages/59/f7/970141a6a8dbd7f556d94977858cfb36fa9b66e0892c6dd780d2219d8cd8/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:28bd570e8e189d7f7b001966435f9dac6718324b5be2990ac496cf1ea9ddb7fe", size = 291762, upload-time = "2025-10-06T05:37:46.657Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/15/ca1adae83a719f82df9116d66f5bb28bb95557b3951903d39135620ef157/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:b2a095d45c5d46e5e79ba1e5b9cb787f541a8dee0433836cea4b96a2c439dcd8", size = 289470, upload-time = "2025-10-06T05:37:47.946Z" },
+ { url = "https://files.pythonhosted.org/packages/ac/83/dca6dc53bf657d371fbc88ddeb21b79891e747189c5de990b9dfff2ccba1/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:eab8145831a0d56ec9c4139b6c3e594c7a83c2c8be25d5bcf2d86136a532287a", size = 289042, upload-time = "2025-10-06T05:37:49.499Z" },
+ { url = "https://files.pythonhosted.org/packages/96/52/abddd34ca99be142f354398700536c5bd315880ed0a213812bc491cff5e4/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:974b28cf63cc99dfb2188d8d222bc6843656188164848c4f679e63dae4b0708e", size = 283148, upload-time = "2025-10-06T05:37:50.745Z" },
+ { url = "https://files.pythonhosted.org/packages/af/d3/76bd4ed4317e7119c2b7f57c3f6934aba26d277acc6309f873341640e21f/frozenlist-1.8.0-cp314-cp314t-win32.whl", hash = "sha256:342c97bf697ac5480c0a7ec73cd700ecfa5a8a40ac923bd035484616efecc2df", size = 44676, upload-time = "2025-10-06T05:37:52.222Z" },
+ { url = "https://files.pythonhosted.org/packages/89/76/c615883b7b521ead2944bb3480398cbb07e12b7b4e4d073d3752eb721558/frozenlist-1.8.0-cp314-cp314t-win_amd64.whl", hash = "sha256:06be8f67f39c8b1dc671f5d83aaefd3358ae5cdcf8314552c57e7ed3e6475bdd", size = 49451, upload-time = "2025-10-06T05:37:53.425Z" },
+ { url = "https://files.pythonhosted.org/packages/e0/a3/5982da14e113d07b325230f95060e2169f5311b1017ea8af2a29b374c289/frozenlist-1.8.0-cp314-cp314t-win_arm64.whl", hash = "sha256:102e6314ca4da683dca92e3b1355490fed5f313b768500084fbe6371fddfdb79", size = 42507, upload-time = "2025-10-06T05:37:54.513Z" },
+ { url = "https://files.pythonhosted.org/packages/9a/9a/e35b4a917281c0b8419d4207f4334c8e8c5dbf4f3f5f9ada73958d937dcc/frozenlist-1.8.0-py3-none-any.whl", hash = "sha256:0c18a16eab41e82c295618a77502e17b195883241c563b00f0aa5106fc4eaa0d", size = 13409, upload-time = "2025-10-06T05:38:16.721Z" },
+]
+
+[[package]]
+name = "google-crc32c"
+version = "1.8.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/03/41/4b9c02f99e4c5fb477122cd5437403b552873f014616ac1d19ac8221a58d/google_crc32c-1.8.0.tar.gz", hash = "sha256:a428e25fb7691024de47fecfbff7ff957214da51eddded0da0ae0e0f03a2cf79", size = 14192, upload-time = "2025-12-16T00:35:25.142Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/e9/5f/7307325b1198b59324c0fa9807cafb551afb65e831699f2ce211ad5c8240/google_crc32c-1.8.0-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:4b8286b659c1335172e39563ab0a768b8015e88e08329fa5321f774275fc3113", size = 31300, upload-time = "2025-12-16T00:21:56.723Z" },
+ { url = "https://files.pythonhosted.org/packages/21/8e/58c0d5d86e2220e6a37befe7e6a94dd2f6006044b1a33edf1ff6d9f7e319/google_crc32c-1.8.0-cp312-cp312-macosx_12_0_x86_64.whl", hash = "sha256:2a3dc3318507de089c5384cc74d54318401410f82aa65b2d9cdde9d297aca7cb", size = 30867, upload-time = "2025-12-16T00:38:31.302Z" },
+ { url = "https://files.pythonhosted.org/packages/ce/a9/a780cc66f86335a6019f557a8aaca8fbb970728f0efd2430d15ff1beae0e/google_crc32c-1.8.0-cp312-cp312-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:14f87e04d613dfa218d6135e81b78272c3b904e2a7053b841481b38a7d901411", size = 33364, upload-time = "2025-12-16T00:40:22.96Z" },
+ { url = "https://files.pythonhosted.org/packages/21/3f/3457ea803db0198c9aaca2dd373750972ce28a26f00544b6b85088811939/google_crc32c-1.8.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:cb5c869c2923d56cb0c8e6bcdd73c009c36ae39b652dbe46a05eb4ef0ad01454", size = 33740, upload-time = "2025-12-16T00:40:23.96Z" },
+ { url = "https://files.pythonhosted.org/packages/df/c0/87c2073e0c72515bb8733d4eef7b21548e8d189f094b5dad20b0ecaf64f6/google_crc32c-1.8.0-cp312-cp312-win_amd64.whl", hash = "sha256:3cc0c8912038065eafa603b238abf252e204accab2a704c63b9e14837a854962", size = 34437, upload-time = "2025-12-16T00:35:21.395Z" },
+ { url = "https://files.pythonhosted.org/packages/d1/db/000f15b41724589b0e7bc24bc7a8967898d8d3bc8caf64c513d91ef1f6c0/google_crc32c-1.8.0-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:3ebb04528e83b2634857f43f9bb8ef5b2bbe7f10f140daeb01b58f972d04736b", size = 31297, upload-time = "2025-12-16T00:23:20.709Z" },
+ { url = "https://files.pythonhosted.org/packages/d7/0d/8ebed0c39c53a7e838e2a486da8abb0e52de135f1b376ae2f0b160eb4c1a/google_crc32c-1.8.0-cp313-cp313-macosx_12_0_x86_64.whl", hash = "sha256:450dc98429d3e33ed2926fc99ee81001928d63460f8538f21a5d6060912a8e27", size = 30867, upload-time = "2025-12-16T00:43:14.628Z" },
+ { url = "https://files.pythonhosted.org/packages/ce/42/b468aec74a0354b34c8cbf748db20d6e350a68a2b0912e128cabee49806c/google_crc32c-1.8.0-cp313-cp313-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:3b9776774b24ba76831609ffbabce8cdf6fa2bd5e9df37b594221c7e333a81fa", size = 33344, upload-time = "2025-12-16T00:40:24.742Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/e8/b33784d6fc77fb5062a8a7854e43e1e618b87d5ddf610a88025e4de6226e/google_crc32c-1.8.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:89c17d53d75562edfff86679244830599ee0a48efc216200691de8b02ab6b2b8", size = 33694, upload-time = "2025-12-16T00:40:25.505Z" },
+ { url = "https://files.pythonhosted.org/packages/92/b1/d3cbd4d988afb3d8e4db94ca953df429ed6db7282ed0e700d25e6c7bfc8d/google_crc32c-1.8.0-cp313-cp313-win_amd64.whl", hash = "sha256:57a50a9035b75643996fbf224d6661e386c7162d1dfdab9bc4ca790947d1007f", size = 34435, upload-time = "2025-12-16T00:35:22.107Z" },
+ { url = "https://files.pythonhosted.org/packages/21/88/8ecf3c2b864a490b9e7010c84fd203ec8cf3b280651106a3a74dd1b0ca72/google_crc32c-1.8.0-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:e6584b12cb06796d285d09e33f63309a09368b9d806a551d8036a4207ea43697", size = 31301, upload-time = "2025-12-16T00:24:48.527Z" },
+ { url = "https://files.pythonhosted.org/packages/36/c6/f7ff6c11f5ca215d9f43d3629163727a272eabc356e5c9b2853df2bfe965/google_crc32c-1.8.0-cp314-cp314-macosx_12_0_x86_64.whl", hash = "sha256:f4b51844ef67d6cf2e9425983274da75f18b1597bb2c998e1c0a0e8d46f8f651", size = 30868, upload-time = "2025-12-16T00:48:12.163Z" },
+ { url = "https://files.pythonhosted.org/packages/56/15/c25671c7aad70f8179d858c55a6ae8404902abe0cdcf32a29d581792b491/google_crc32c-1.8.0-cp314-cp314-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b0d1a7afc6e8e4635564ba8aa5c0548e3173e41b6384d7711a9123165f582de2", size = 33381, upload-time = "2025-12-16T00:40:26.268Z" },
+ { url = "https://files.pythonhosted.org/packages/42/fa/f50f51260d7b0ef5d4898af122d8a7ec5a84e2984f676f746445f783705f/google_crc32c-1.8.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:8b3f68782f3cbd1bce027e48768293072813469af6a61a86f6bb4977a4380f21", size = 33734, upload-time = "2025-12-16T00:40:27.028Z" },
+ { url = "https://files.pythonhosted.org/packages/08/a5/7b059810934a09fb3ccb657e0843813c1fee1183d3bc2c8041800374aa2c/google_crc32c-1.8.0-cp314-cp314-win_amd64.whl", hash = "sha256:d511b3153e7011a27ab6ee6bb3a5404a55b994dc1a7322c0b87b29606d9790e2", size = 34878, upload-time = "2025-12-16T00:35:23.142Z" },
+]
+
+[[package]]
+name = "greenlet"
+version = "3.5.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/e2/f1/fbbfef6af0bad0548f09bc28948ea3c275b4edb19e17fc5ca9900a6a634d/greenlet-3.5.3.tar.gz", hash = "sha256:a61efc018fd3eb317eeca31aba90ee9e7f26f22884a79b6c6ec715bf71bb62f1", size = 200270, upload-time = "2026-06-26T19:28:24.832Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/5d/6e/4c37d51a2b7f82d2ff11bb6b5f7d766d9a011726624af255e843727627a3/greenlet-3.5.3-cp312-cp312-macosx_11_0_universal2.whl", hash = "sha256:719757059f5a53fd0dde23f78cffeafcdd97b21c850ddb7ca684a3c1a1f122e2", size = 288685, upload-time = "2026-06-26T18:22:08.977Z" },
+ { url = "https://files.pythonhosted.org/packages/7a/73/815dd90131c1b71ebdf53dbc7c276cafec2a1173b97559f97aba72724a87/greenlet-3.5.3-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:efa9f765dd09f9d0cdac651ffdf631ee59ec5dc6ee7a73e0c012ba9c52fbdf5b", size = 604761, upload-time = "2026-06-26T19:07:10.114Z" },
+ { url = "https://files.pythonhosted.org/packages/9f/57/079cfe76bcef36b153b25607ee91c6fcb58f17f8b23c86bbbeabe0c88d72/greenlet-3.5.3-cp312-cp312-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7faba15ac005376e02a0384504e0243be3370ce010296a44a820feb342b505ab", size = 617044, upload-time = "2026-06-26T19:10:07.25Z" },
+ { url = "https://files.pythonhosted.org/packages/37/87/b4d095775a3fb1bcafbb483fc206b27ebb785724c83051447737085dc54e/greenlet-3.5.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:87142215824be6ac05e2e8e2786eec307ccbc27c36723c3881959df654af6861", size = 614244, upload-time = "2026-06-26T18:32:17.594Z" },
+ { url = "https://files.pythonhosted.org/packages/8a/70/7559b609683650fa2b95b8ab84b4ab0b26556a635d19675e12aa832d826d/greenlet-3.5.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:215275b1b49320987352e6c1b054acca0064f965a2c66992bed9a6f7d913f149", size = 1574210, upload-time = "2026-06-26T19:09:03.077Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/73/be55392074c60fc37655ca40fa6022457bfbf6718e9e342a7b0b41f96dd2/greenlet-3.5.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:6b1b0eed82364b0e32c4ea0f221452d33e6bb17ae094d9f72aed9851812747ea", size = 1638627, upload-time = "2026-06-26T18:31:44.748Z" },
+ { url = "https://files.pythonhosted.org/packages/14/40/c57489acf8e37d74e2913d4eff63aa0dba17acccc4bdeef874dde2dbbec9/greenlet-3.5.3-cp312-cp312-win_amd64.whl", hash = "sha256:cde8adafa2365676f74a979744629589999093bc86e2484214f58e61df08902c", size = 239882, upload-time = "2026-06-26T18:23:27.518Z" },
+ { url = "https://files.pythonhosted.org/packages/71/fd/6fea0e3d6600f785069481ee637e09378dd4118acdfd38ad88ae2db31c98/greenlet-3.5.3-cp312-cp312-win_arm64.whl", hash = "sha256:c4e7b79d83805475f0102008843f6eb45fd3bb0b2e88c774adab5fbaab27117d", size = 238211, upload-time = "2026-06-26T18:22:37.671Z" },
+ { url = "https://files.pythonhosted.org/packages/9b/ff/a620267401db30a50cc8450ee90730e2d4a85658c055c0e760d4ed47fb13/greenlet-3.5.3-cp313-cp313-macosx_11_0_universal2.whl", hash = "sha256:c8d87c2134d871df96ecdea9cec7cbaab286dadab0f56476e57aaf9e8ac11550", size = 287609, upload-time = "2026-06-26T18:21:14.724Z" },
+ { url = "https://files.pythonhosted.org/packages/d6/fa/5401ac78021c826a25b6dde0c705e0a8f29b617509f9185a31dac15fbe1b/greenlet-3.5.3-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a2d185dd1621757e70c3861cceffd5317ab4e7ed7eb09c82994828468527ade5", size = 607435, upload-time = "2026-06-26T19:07:11.412Z" },
+ { url = "https://files.pythonhosted.org/packages/e9/76/1dc144a2e56e65d36405078ed774224375ea520a1870a6e46e08bb4ac7bf/greenlet-3.5.3-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:1c514a468149bf8fbbab874188a3535cd8a48a3e353eb53a3d424296f8dbacd3", size = 619787, upload-time = "2026-06-26T19:10:08.396Z" },
+ { url = "https://files.pythonhosted.org/packages/bf/87/c298cee62df1de4ad7fec32abda73526cff347fd143a6ed4ac369246668a/greenlet-3.5.3-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:915f887cf2682b66419b879423a2e072634aa7b7dce6f3ada4957cfced3f1e9a", size = 616786, upload-time = "2026-06-26T18:32:19.128Z" },
+ { url = "https://files.pythonhosted.org/packages/9e/2e/e6f009885ed0705ccf33fe0583c117cfd03cde77e31a596dd5785a30762b/greenlet-3.5.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:766cfd421c13e450feb340cd472a3ed9957d438727b7b4593ad7c76c5d2b0deb", size = 1574316, upload-time = "2026-06-26T19:09:04.273Z" },
+ { url = "https://files.pythonhosted.org/packages/ef/fe/43fd110b01e40da0adb7c90ac7ea744bef2d43dca00de5095fd2351c2a68/greenlet-3.5.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:2ecda9ec22edf38fa389369eaed8c3d37c05f3c54e69f69438dbb2cc1de1458b", size = 1638614, upload-time = "2026-06-26T18:31:46.297Z" },
+ { url = "https://files.pythonhosted.org/packages/0f/7c/062447147a61f8b4337b156fe70d32a165fcf2f89d7ca6255e572806705c/greenlet-3.5.3-cp313-cp313-win_amd64.whl", hash = "sha256:c82304750f057167ff60d188df1d0cc1764ce9567eadf03e6a7443bcedd0b30b", size = 239850, upload-time = "2026-06-26T18:21:54.613Z" },
+ { url = "https://files.pythonhosted.org/packages/c7/7e/220a7f5824a64a60443fc03b39dfac4ea63a7fb6d481efa27eafa928e7f4/greenlet-3.5.3-cp313-cp313-win_arm64.whl", hash = "sha256:dc133a1569ee667b2a6ef56ce551084aeefd87a5acbc4736d336d1e2edc6cfc4", size = 238141, upload-time = "2026-06-26T18:22:48.507Z" },
+ { url = "https://files.pythonhosted.org/packages/c3/93/43e116ee114b28737ba7e12952a0d4e2f55944d0f84e42bc91ba7192a3c9/greenlet-3.5.3-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:fd2e02fa07485778536a036222d616ab957b1d533f36b3ed98ce725d9c9d3117", size = 288202, upload-time = "2026-06-26T18:23:49.604Z" },
+ { url = "https://files.pythonhosted.org/packages/82/2f/146d218299046a43d1f029fd544b3d110d0f175a09c715c7e8da4a4a345d/greenlet-3.5.3-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:df0a0628d1597eb0897b62f55d1343f772405fd25f3b2a796c76874b0c2e22e8", size = 654096, upload-time = "2026-06-26T19:07:12.71Z" },
+ { url = "https://files.pythonhosted.org/packages/a0/cc/04738cafb3f45fa991ea44f9de94c47dcec964f5a972300988a6751f49d9/greenlet-3.5.3-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ebd933a6adabc298bab47731a130fe6bfb888bd934eee37810f151159544540d", size = 666304, upload-time = "2026-06-26T19:10:09.503Z" },
+ { url = "https://files.pythonhosted.org/packages/ce/aa/4e0dad5e605c270c784ab911c43da6adb136ccd4d81180f763ca429a723d/greenlet-3.5.3-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4b9d501b40e80b70e32323c799dd9b420a5577a9601469d362ae1ffb690f3a7c", size = 663635, upload-time = "2026-06-26T18:32:20.802Z" },
+ { url = "https://files.pythonhosted.org/packages/d1/50/13efdbea246fe3d3b735e191fec08fb50809f53cd2383ebe123d0809e44b/greenlet-3.5.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a1fad1d11e7d6aab184107baa8e4ece11ccba3ec9599cd7efa5ff4d70d43256a", size = 1621252, upload-time = "2026-06-26T19:09:05.647Z" },
+ { url = "https://files.pythonhosted.org/packages/f7/22/c0a336ae4a1410fd5f5121098e5bfbf1865f64c5ef80b4b5412886c4a332/greenlet-3.5.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:fad5aec764399f1b5cc347ad250a59660f20c8f8888ea6bae1f93b769cce1154", size = 1684824, upload-time = "2026-06-26T18:31:47.738Z" },
+ { url = "https://files.pythonhosted.org/packages/7a/94/91aec0030bea75c4b3244251d0de60a1f3432d1ecb53ab6c437fb5c3ba61/greenlet-3.5.3-cp314-cp314-win_amd64.whl", hash = "sha256:7669aa24cf2a1041d6f7899575b494a3ab4cf68bfcc8609b1dc0be7272db835e", size = 240754, upload-time = "2026-06-26T18:22:15.669Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/06/68d0983e79e02138f64b4d303c500c27ddb48e5e77f3debb80888a921eae/greenlet-3.5.3-cp314-cp314-win_arm64.whl", hash = "sha256:5b4807c4082c9d1b6d9eed56fcd041863e37f2228106eef24c30ca096e238605", size = 239549, upload-time = "2026-06-26T18:22:42.996Z" },
+ { url = "https://files.pythonhosted.org/packages/91/95/3e161213d7f1d378d15aa9e792093e9bfe01844680d04b7fd6e0107c9098/greenlet-3.5.3-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:271a8ea7c1024e8a0d7dd2be66dd66dda8a07193f41a17b9e924f7600f5b62be", size = 296389, upload-time = "2026-06-26T18:22:20.657Z" },
+ { url = "https://files.pythonhosted.org/packages/00/92/715c44721abe2b4d1ae9abde4179411868a5bff312479f54e105d372f131/greenlet-3.5.3-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:19131729ae0ddc3c2e1ef85e650169b5e37ee32e400f215f78b94d7b0d567310", size = 653382, upload-time = "2026-06-26T19:07:14.209Z" },
+ { url = "https://files.pythonhosted.org/packages/a0/83/37a10372a1090a6624cca8e74c12df1a36c2dc36429ed0255b7fb1aeee23/greenlet-3.5.3-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:1540dd8e5fc2a5aec40fbb98ef8e149fa47c89a4b4a1cf2575a14d3d1869d7a8", size = 659401, upload-time = "2026-06-26T19:10:10.876Z" },
+ { url = "https://files.pythonhosted.org/packages/db/e2/d1509cad4207da559cc42986ecdd8fc67ad0d1bba2bf03023c467fd5e0f3/greenlet-3.5.3-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e81fa194a1d20967877bdf9c7794db2bc99063e5be36aee710c08f04c5bb087f", size = 656969, upload-time = "2026-06-26T18:32:22.272Z" },
+ { url = "https://files.pythonhosted.org/packages/86/7d/eaf70de20aadca3a5884aec58362861c64ce45e7b277f47ed026926a3b89/greenlet-3.5.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:55cf4d777485d43110e47133cbba6d74a8885a87ec1227ef0267f9ee80c5aa21", size = 1617822, upload-time = "2026-06-26T19:09:06.893Z" },
+ { url = "https://files.pythonhosted.org/packages/8a/f9/414d38fc400ae4350d4185eaad1827676f7cf5287b9136e0ed1cbbe20a7f/greenlet-3.5.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:12a248ba75f6a9a236375f52296c498c89ff1d8badf32deb9eca7abd5853f7da", size = 1677983, upload-time = "2026-06-26T18:31:49.396Z" },
+ { url = "https://files.pythonhosted.org/packages/e4/15/7edb977e08f9bff702fe42d6c902702786ff6b9694058b4e6a2a6ac90e57/greenlet-3.5.3-cp314-cp314t-win_amd64.whl", hash = "sha256:efc6bd60ea02e085862c74a3ef64b147ffc6f1a5ea7d9f26e7a939943f68c1e3", size = 243626, upload-time = "2026-06-26T18:24:41.485Z" },
+ { url = "https://files.pythonhosted.org/packages/2c/8a/93928dce91e6b3598b5e779e8d1fd6576a504640c58e78627077f6a7a91a/greenlet-3.5.3-cp315-cp315-macosx_11_0_universal2.whl", hash = "sha256:ea03f2f04367845d6b58eeed276e1e56e51f0b97d8ad5a88a7d20a91dc9056cc", size = 288860, upload-time = "2026-06-26T18:22:48.07Z" },
+ { url = "https://files.pythonhosted.org/packages/4f/ca/69db42d447a1378043e2c8f19c09cbbd1263371505053c496b49066d3d16/greenlet-3.5.3-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:78dbef602fda6d97d957eb7937f70c9ce9e9527330347f8f6b6f9e554a9e7a47", size = 659747, upload-time = "2026-06-26T19:07:15.565Z" },
+ { url = "https://files.pythonhosted.org/packages/a8/0b/af7ac2ef8dd41e3da1a40dda6305c23b9a03e13ba975ec916357b50f8575/greenlet-3.5.3-cp315-cp315-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:6f73857adb8fee13fa56c172bd11262f888c0c648f9fea113e777bb2c7904a81", size = 670419, upload-time = "2026-06-26T19:10:12.293Z" },
+ { url = "https://files.pythonhosted.org/packages/51/1e/1d51640cacbfc455dbe9f9a9f594c49e4e244f63b9971a2f4764e46cc53d/greenlet-3.5.3-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:232fec92e823addaf02d9472cf7381e24a1d046a6ced1103c5caa4c21b9dfc1d", size = 668787, upload-time = "2026-06-26T18:32:24.298Z" },
+ { url = "https://files.pythonhosted.org/packages/21/66/4030d5b0b5894500023f003bb054d9bb354dfbd1e186c3a296759172f5f5/greenlet-3.5.3-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:2421c3564da9429d5586d46ca31ebb26516b5498a802cf65c041a8e8a8980d34", size = 1626305, upload-time = "2026-06-26T19:09:08.281Z" },
+ { url = "https://files.pythonhosted.org/packages/0e/50/5221371c7550108dfa3c378debc41d032aa9c78e89abb01d8011cfc93289/greenlet-3.5.3-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:e0f0d160f0b2e558e6c75f7930967183255dc9735e5f5b8cae58ee09c9576d8b", size = 1688631, upload-time = "2026-06-26T18:31:51.278Z" },
+ { url = "https://files.pythonhosted.org/packages/68/5d/00d469daae3c65d2bf620b10eee82eb022127d483c6bc8c69fae6f3fbf17/greenlet-3.5.3-cp315-cp315-win_amd64.whl", hash = "sha256:dd99329bbc15ca78dcc583dba05d0b1b0bae01ab6c2174989f5aaee3e41ac930", size = 241027, upload-time = "2026-06-26T18:22:38.203Z" },
+ { url = "https://files.pythonhosted.org/packages/e7/e8/883785b44c5780ed71e83d3e4437e710470be17a2e181e8b601e2da0dc4a/greenlet-3.5.3-cp315-cp315-win_arm64.whl", hash = "sha256:499fef2acede88c1864a57bb586b4bf533c81e1b82df7ab93451cdb47dfec227", size = 240085, upload-time = "2026-06-26T18:23:54.217Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/da/4f4a8450962fad137c1c8981a3f1b8919d06c829993d4d476f9c525d5173/greenlet-3.5.3-cp315-cp315t-macosx_11_0_universal2.whl", hash = "sha256:176bc16a721fa5fc294d70b87b4dfa5fbdd251b3da5d5372735ecef9bd7d6d0c", size = 297221, upload-time = "2026-06-26T18:23:27.176Z" },
+ { url = "https://files.pythonhosted.org/packages/57/66/b3bfae3e220a9b63ea539a0eea681800c69ab1aada757eae8789f183e7ce/greenlet-3.5.3-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:629b614d2b786e89c50440e246f33eea78f58a962d0bdbbcc809e6d13605903f", size = 657221, upload-time = "2026-06-26T19:07:16.973Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/81/b6d4d73a709684fc77e7fa034d7c2fe82cffa9fc920fadcaa659c2626213/greenlet-3.5.3-cp315-cp315t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2b2e857ae16f5f72142edf75f9f176fe7526ba19a2841df1420516f83831c9f2", size = 663226, upload-time = "2026-06-26T19:10:13.723Z" },
+ { url = "https://files.pythonhosted.org/packages/f5/07/e210b02b589f16e74ff48b730690e4a34ffe984219fce4f3c1a0e7ec8545/greenlet-3.5.3-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e515757e2e36bcbf1fad09a46e1557e8b1ae1797d4b44d09da7deed88ad28608", size = 660802, upload-time = "2026-06-26T18:32:26.081Z" },
+ { url = "https://files.pythonhosted.org/packages/eb/2e/5303eb3fa06bca089060f479707182a93e360683bc252acf846c3090d34e/greenlet-3.5.3-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:b363d46ed1ea431825fdb01471bb024fc08399bad1572a616e853c7684415adb", size = 1622157, upload-time = "2026-06-26T19:09:09.527Z" },
+ { url = "https://files.pythonhosted.org/packages/54/70/50de47a488f14df260b50ae34fb5d56016e308b098eab02c878b5223c26a/greenlet-3.5.3-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:e44da2f5bbdaabaf7d80b73dbb430c7035771e9f244e3c8b769715c9d8fa0a16", size = 1681159, upload-time = "2026-06-26T18:31:52.986Z" },
+ { url = "https://files.pythonhosted.org/packages/a7/13/1055e1dda7882073eda533e2b96c62e55bbd2db7fda6d5ece992febc7071/greenlet-3.5.3-cp315-cp315t-win_amd64.whl", hash = "sha256:8ff8bed3e3baa20a3ea261ce00526f1898ad4801d4886fd2220580ee0ad8fadf", size = 244007, upload-time = "2026-06-26T18:22:04.353Z" },
+ { url = "https://files.pythonhosted.org/packages/b4/0d/ca7d15afbdc397e3401134c9e1800d51d12b829661786187a4ad08fe484f/greenlet-3.5.3-cp315-cp315t-win_arm64.whl", hash = "sha256:b7068bd09f761f3f5b4d214c2bed063186b2a86148c740b3873e3f56d79bac31", size = 242586, upload-time = "2026-06-26T18:23:37.93Z" },
+]
+
+[[package]]
+name = "grpclib"
+version = "0.4.9"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "h2" },
+ { name = "multidict" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/5b/28/5a2c299ec82a876a252c5919aa895a6f1d1d35c96417c5ce4a4660dc3a80/grpclib-0.4.9.tar.gz", hash = "sha256:cc589c330fa81004c6400a52a566407574498cb5b055fa927013361e21466c46", size = 84798, upload-time = "2025-12-14T22:23:14.349Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/5c/90/b0cbbd9efcc82816c58f31a34963071aa19fb792a212a5d9caf8e0fc3097/grpclib-0.4.9-py3-none-any.whl", hash = "sha256:7762ec1c8ed94dfad597475152dd35cbd11aecaaca2f243e29702435ca24cf0e", size = 77063, upload-time = "2025-12-14T22:23:13.224Z" },
+]
+
+[[package]]
+name = "h2"
+version = "4.3.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "hpack" },
+ { name = "hyperframe" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/1d/17/afa56379f94ad0fe8defd37d6eb3f89a25404ffc71d4d848893d270325fc/h2-4.3.0.tar.gz", hash = "sha256:6c59efe4323fa18b47a632221a1888bd7fde6249819beda254aeca909f221bf1", size = 2152026, upload-time = "2025-08-23T18:12:19.778Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/69/b2/119f6e6dcbd96f9069ce9a2665e0146588dc9f88f29549711853645e736a/h2-4.3.0-py3-none-any.whl", hash = "sha256:c438f029a25f7945c69e0ccf0fb951dc3f73a5f6412981daee861431b70e2bdd", size = 61779, upload-time = "2025-08-23T18:12:17.779Z" },
+]
+
+[[package]]
+name = "h5py"
+version = "3.16.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/db/33/acd0ce6863b6c0d7735007df01815403f5589a21ff8c2e1ee2587a38f548/h5py-3.16.0.tar.gz", hash = "sha256:a0dbaad796840ccaa67a4c144a0d0c8080073c34c76d5a6941d6818678ef2738", size = 446526, upload-time = "2026-03-06T13:49:08.07Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/c8/c0/5d4119dba94093bbafede500d3defd2f5eab7897732998c04b54021e530b/h5py-3.16.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c5313566f4643121a78503a473f0fb1e6dcc541d5115c44f05e037609c565c4d", size = 3685604, upload-time = "2026-03-06T13:48:04.198Z" },
+ { url = "https://files.pythonhosted.org/packages/b0/42/c84efcc1d4caebafb1ecd8be4643f39c85c47a80fe254d92b8b43b1eadaf/h5py-3.16.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:42b012933a83e1a558c673176676a10ce2fd3759976a0fedee1e672d1e04fc9d", size = 3061940, upload-time = "2026-03-06T13:48:05.783Z" },
+ { url = "https://files.pythonhosted.org/packages/89/84/06281c82d4d1686fde1ac6b0f307c50918f1c0151062445ab3b6fa5a921d/h5py-3.16.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:ff24039e2573297787c3063df64b60aab0591980ac898329a08b0320e0cf2527", size = 5198852, upload-time = "2026-03-06T13:48:07.482Z" },
+ { url = "https://files.pythonhosted.org/packages/9e/e9/1a19e42cd43cc1365e127db6aae85e1c671da1d9a5d746f4d34a50edb577/h5py-3.16.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:dfc21898ff025f1e8e67e194965a95a8d4754f452f83454538f98f8a3fcb207e", size = 5405250, upload-time = "2026-03-06T13:48:09.628Z" },
+ { url = "https://files.pythonhosted.org/packages/b7/8e/9790c1655eabeb85b92b1ecab7d7e62a2069e53baefd58c98f0909c7a948/h5py-3.16.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:698dd69291272642ffda44a0ecd6cd3bda5faf9621452d255f57ce91487b9794", size = 5190108, upload-time = "2026-03-06T13:48:11.26Z" },
+ { url = "https://files.pythonhosted.org/packages/51/d7/ab693274f1bd7e8c5f9fdd6c7003a88d59bedeaf8752716a55f532924fbb/h5py-3.16.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:2b2c02b0a160faed5fb33f1ba8a264a37ee240b22e049ecc827345d0d9043074", size = 5419216, upload-time = "2026-03-06T13:48:13.322Z" },
+ { url = "https://files.pythonhosted.org/packages/03/c1/0976b235cf29ead553e22f2fb6385a8252b533715e00d0ae52ed7b900582/h5py-3.16.0-cp312-cp312-win_amd64.whl", hash = "sha256:96b422019a1c8975c2d5dadcf61d4ba6f01c31f92bbde6e4649607885fe502d6", size = 3182868, upload-time = "2026-03-06T13:48:15.759Z" },
+ { url = "https://files.pythonhosted.org/packages/14/d9/866b7e570b39070f92d47b0ff1800f0f8239b6f9e45f02363d7112336c1f/h5py-3.16.0-cp312-cp312-win_arm64.whl", hash = "sha256:39c2838fb1e8d97bcf1755e60ad1f3dd76a7b2a475928dc321672752678b96db", size = 2653286, upload-time = "2026-03-06T13:48:17.279Z" },
+ { url = "https://files.pythonhosted.org/packages/0f/9e/6142ebfda0cb6e9349c091eae73c2e01a770b7659255248d637bec54a88b/h5py-3.16.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:370a845f432c2c9619db8eed334d1e610c6015796122b0e57aa46312c22617d9", size = 3671808, upload-time = "2026-03-06T13:48:19.737Z" },
+ { url = "https://files.pythonhosted.org/packages/b0/65/5e088a45d0f43cd814bc5bec521c051d42005a472e804b1a36c48dada09b/h5py-3.16.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:42108e93326c50c2810025aade9eac9d6827524cdccc7d4b75a546e5ab308edb", size = 3045837, upload-time = "2026-03-06T13:48:21.854Z" },
+ { url = "https://files.pythonhosted.org/packages/da/1e/6172269e18cc5a484e2913ced33339aad588e02ba407fafd00d369e22ef3/h5py-3.16.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:099f2525c9dcf28de366970a5fb34879aab20491589fa89ce2863a84218bb524", size = 5193860, upload-time = "2026-03-06T13:48:24.071Z" },
+ { url = "https://files.pythonhosted.org/packages/bd/98/ef2b6fe2903e377cbe870c3b2800d62552f1e3dbe81ce49e1923c53d1c5c/h5py-3.16.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:9300ad32dea9dfc5171f94d5f6948e159ed93e4701280b0f508773b3f582f402", size = 5400417, upload-time = "2026-03-06T13:48:25.728Z" },
+ { url = "https://files.pythonhosted.org/packages/bc/81/5b62d760039eed64348c98129d17061fdfc7839fc9c04eaaad6dee1004e4/h5py-3.16.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:171038f23bccddfc23f344cadabdfc9917ff554db6a0d417180d2747fe4c75a7", size = 5185214, upload-time = "2026-03-06T13:48:27.436Z" },
+ { url = "https://files.pythonhosted.org/packages/28/c4/532123bcd9080e250696779c927f2cb906c8bf3447df98f5ceb8dcded539/h5py-3.16.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:7e420b539fb6023a259a1b14d4c9f6df8cf50d7268f48e161169987a57b737ff", size = 5414598, upload-time = "2026-03-06T13:48:29.49Z" },
+ { url = "https://files.pythonhosted.org/packages/c3/d9/a27997f84341fc0dfcdd1fe4179b6ba6c32a7aa880fdb8c514d4dad6fba3/h5py-3.16.0-cp313-cp313-win_amd64.whl", hash = "sha256:18f2bbcd545e6991412253b98727374c356d67caa920e68dc79eab36bf5fedad", size = 3175509, upload-time = "2026-03-06T13:48:31.131Z" },
+ { url = "https://files.pythonhosted.org/packages/a5/23/bb8647521d4fd770c30a76cfc6cb6a2f5495868904054e92f2394c5a78ff/h5py-3.16.0-cp313-cp313-win_arm64.whl", hash = "sha256:656f00e4d903199a1d58df06b711cf3ca632b874b4207b7dbec86185b5c8c7d4", size = 2647362, upload-time = "2026-03-06T13:48:33.411Z" },
+ { url = "https://files.pythonhosted.org/packages/48/3c/7fcd9b4c9eed82e91fb15568992561019ae7a829d1f696b2c844355d95dd/h5py-3.16.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:9c9d307c0ef862d1cd5714f72ecfafe0a5d7529c44845afa8de9f46e5ba8bd65", size = 3678608, upload-time = "2026-03-06T13:48:35.183Z" },
+ { url = "https://files.pythonhosted.org/packages/6a/b7/9366ed44ced9b7ef357ab48c94205280276db9d7f064aa3012a97227e966/h5py-3.16.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:8c1eff849cdd53cbc73c214c30ebdb6f1bb8b64790b4b4fc36acdb5e43570210", size = 3054773, upload-time = "2026-03-06T13:48:37.139Z" },
+ { url = "https://files.pythonhosted.org/packages/58/a5/4964bc0e91e86340c2bbda83420225b2f770dcf1eb8a39464871ad769436/h5py-3.16.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:e2c04d129f180019e216ee5f9c40b78a418634091c8782e1f723a6ca3658b965", size = 5198886, upload-time = "2026-03-06T13:48:38.879Z" },
+ { url = "https://files.pythonhosted.org/packages/f1/16/d905e7f53e661ce2c24686c38048d8e2b750ffc4350009d41c4e6c6c9826/h5py-3.16.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:e4360f15875a532bc7b98196c7592ed4fc92672a57c0a621355961cafb17a6dd", size = 5404883, upload-time = "2026-03-06T13:48:41.324Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/f2/58f34cb74af46d39f4cd18ea20909a8514960c5a3e5b92fd06a28161e0a8/h5py-3.16.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:3fae9197390c325e62e0a1aa977f2f62d994aa87aab182abbea85479b791197c", size = 5192039, upload-time = "2026-03-06T13:48:43.117Z" },
+ { url = "https://files.pythonhosted.org/packages/ce/ca/934a39c24ce2e2db017268c08da0537c20fa0be7e1549be3e977313fc8f5/h5py-3.16.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:43259303989ac8adacc9986695b31e35dba6fd1e297ff9c6a04b7da5542139cc", size = 5421526, upload-time = "2026-03-06T13:48:44.838Z" },
+ { url = "https://files.pythonhosted.org/packages/3e/14/615a450205e1b56d16c6783f5ccd116cde05550faad70ae077c955654a75/h5py-3.16.0-cp314-cp314-win_amd64.whl", hash = "sha256:fa48993a0b799737ba7fd21e2350fa0a60701e58180fae9f2de834bc39a147ab", size = 3183263, upload-time = "2026-03-06T13:48:47.117Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/48/a6faef5ed632cae0c65ac6b214a6614a0b510c3183532c521bdb0055e117/h5py-3.16.0-cp314-cp314-win_arm64.whl", hash = "sha256:1897a771a7f40d05c262fc8f37376ec37873218544b70216872876c627640f63", size = 2663450, upload-time = "2026-03-06T13:48:48.707Z" },
+ { url = "https://files.pythonhosted.org/packages/5d/32/0c8bb8aedb62c772cf7c1d427c7d1951477e8c2835f872bc0a13d1f85f86/h5py-3.16.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:15922e485844f77c0b9d275396d435db3baa58292a9c2176a386e072e0cf2491", size = 3760693, upload-time = "2026-03-06T13:48:50.453Z" },
+ { url = "https://files.pythonhosted.org/packages/1d/1f/fcc5977d32d6387c5c9a694afee716a5e20658ac08b3ff24fdec79fb05f2/h5py-3.16.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:df02dd29bd247f98674634dfe41f89fd7c16ba3d7de8695ec958f58404a4e618", size = 3181305, upload-time = "2026-03-06T13:48:52.221Z" },
+ { url = "https://files.pythonhosted.org/packages/f5/a1/af87f64b9f986889884243643621ebbd4ac72472ba8ec8cec891ac8e2ca1/h5py-3.16.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:0f456f556e4e2cebeebd9d66adf8dc321770a42593494a0b6f0af54a7567b242", size = 5074061, upload-time = "2026-03-06T13:48:54.089Z" },
+ { url = "https://files.pythonhosted.org/packages/cc/d0/146f5eaff3dc246a9c7f6e5e4f42bd45cc613bce16693bcd4d1f7c958bf5/h5py-3.16.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:3e6cb3387c756de6a9492d601553dffea3fe11b5f22b443aac708c69f3f55e16", size = 5279216, upload-time = "2026-03-06T13:48:56.75Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/9d/12a13424f1e604fc7df9497b73c0356fb78c2fb206abd7465ce47226e8fd/h5py-3.16.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8389e13a1fd745ad2856873e8187fd10268b2d9677877bb667b41aebd771d8b7", size = 5070068, upload-time = "2026-03-06T13:48:59.169Z" },
+ { url = "https://files.pythonhosted.org/packages/41/8c/bbe98f813722b4873818a8db3e15aa3e625b59278566905ac439725e8070/h5py-3.16.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:346df559a0f7dcb31cf8e44805319e2ab24b8957c45e7708ce503b2ec79ba725", size = 5300253, upload-time = "2026-03-06T13:49:02.033Z" },
+ { url = "https://files.pythonhosted.org/packages/32/9e/87e6705b4d6890e7cecdf876e2a7d3e40654a2ae37482d79a6f1b87f7b92/h5py-3.16.0-cp314-cp314t-win_amd64.whl", hash = "sha256:4c6ab014ab704b4feaa719ae783b86522ed0bf1f82184704ed3c9e4e3228796e", size = 3381671, upload-time = "2026-03-06T13:49:04.351Z" },
+ { url = "https://files.pythonhosted.org/packages/96/91/9fad90cfc5f9b2489c7c26ad897157bce82f0e9534a986a221b99760b23b/h5py-3.16.0-cp314-cp314t-win_arm64.whl", hash = "sha256:faca8fb4e4319c09d83337adc80b2ca7d5c5a343c2d6f1b6388f32cfecca13c1", size = 2740706, upload-time = "2026-03-06T13:49:06.347Z" },
+]
+
+[[package]]
+name = "hnswlib"
+version = "0.8.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/cf/7a/1a9b1405f2eb59515f06c3074750b03e0e96edf7fee0f6dd6df81d9c21d7/hnswlib-0.8.0.tar.gz", hash = "sha256:cb6d037eedebb34a7134e7dc78966441dfd04c9cf5ee93911be911ced951c44c", size = 36206, upload-time = "2023-12-03T04:16:17.55Z" }
+
+[[package]]
+name = "hpack"
+version = "4.2.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/26/5b/fcabf6028144a8723726318b07a32c2f3314acdff6265743cf08a344b18e/hpack-4.2.0.tar.gz", hash = "sha256:0895cfa3b5531fc65fe439c05eb65144f123bf7a394fcaa56aa423548d8e45c0", size = 51300, upload-time = "2026-06-23T18:34:46.667Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/71/b4/4a9fcfb2aef6ba44d9073ecd301443aa00b3dac95de5619f2a7de7ec8a91/hpack-4.2.0-py3-none-any.whl", hash = "sha256:858ac0b02280fa582b5080d68db0899c62a80375e0e5413a74970c5e518b6986", size = 34246, upload-time = "2026-06-23T18:34:45.472Z" },
+]
+
+[[package]]
+name = "hyperframe"
+version = "6.1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/02/e7/94f8232d4a74cc99514c13a9f995811485a6903d48e5d952771ef6322e30/hyperframe-6.1.0.tar.gz", hash = "sha256:f630908a00854a7adeabd6382b43923a4c4cd4b821fcb527e6ab9e15382a3b08", size = 26566, upload-time = "2025-01-22T21:41:49.302Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/48/30/47d0bf6072f7252e6521f3447ccfa40b421b6824517f82854703d0f5a98b/hyperframe-6.1.0-py3-none-any.whl", hash = "sha256:b03380493a519fce58ea5af42e4a42317bf9bd425596f7a0835ffce80f1a42e5", size = 13007, upload-time = "2025-01-22T21:41:47.295Z" },
+]
+
+[[package]]
+name = "idna"
+version = "3.18"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz", hash = "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848", size = 196711, upload-time = "2026-06-02T14:34:07.794Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" },
+]
+
+[[package]]
+name = "igraph"
+version = "1.0.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "texttable" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/23/be/56bef1919005b4caf1f71522b300d359f7faeb7ae93a3b0baa9b4f146a87/igraph-1.0.0.tar.gz", hash = "sha256:2414d0be2e4d77ee5357807d100974b40f6082bb1bb71988ec46cfb6728651ee", size = 5077105, upload-time = "2025-10-23T12:22:50.127Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/a5/03/3278ad0ceb3ea0e84d8ae3a85bdded4d0e57853aeb802a200feb43847b93/igraph-1.0.0-cp39-abi3-macosx_10_15_x86_64.whl", hash = "sha256:c2cbc415e02523e5a241eecee82319080bf928a70b1ba299f3b3e25bf029b6d4", size = 2257415, upload-time = "2025-10-23T12:22:27.246Z" },
+ { url = "https://files.pythonhosted.org/packages/0d/bc/6281ec7f9baaf71ee57c3b1748da2d3148d15d253e1a03006f204aa68ca5/igraph-1.0.0-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:1a27753cd80680a8f676c2d5a467aaa4a95e510b30748398ec4e4aeb982130e8", size = 2048555, upload-time = "2025-10-23T12:22:29.49Z" },
+ { url = "https://files.pythonhosted.org/packages/2a/38/3cd6428a4ed4c09a56df05998438e7774fd1d799ee4fb8fc481674f5f7fc/igraph-1.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:a55dc3a2a4e3fc3eba42479910c1511bfc3ecb33cdf5f0406891fd85f14b5aee", size = 5314141, upload-time = "2025-10-23T12:22:31.023Z" },
+ { url = "https://files.pythonhosted.org/packages/7d/da/dd2867c25adbb41563720f14b5fc895c98bf88be682a3faff4f7b3118d2a/igraph-1.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:2d04c2c76f686fb1f554ee35dfd3085f5e73b7965ba6b4cf06d53e66b1955522", size = 5683134, upload-time = "2025-10-23T12:22:32.423Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/40/243c118d34ab80382d7009c4dcb99b887384c3d2ce84d29eeac19e2a007a/igraph-1.0.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:f2b52dc1757fff0fed29a9f7a276d971a11db4211569ed78b9eab36288dfcc9d", size = 6211583, upload-time = "2025-10-23T12:22:34.238Z" },
+ { url = "https://files.pythonhosted.org/packages/1d/b7/88f433819c54b496cb0315fce28e658970cb20ff5dbd52a5a605ce2888de/igraph-1.0.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:05c79a2a8fca695b2f217a6fa7f2549f896f757d4db41be32a055400cb19cc30", size = 6594509, upload-time = "2025-10-23T12:22:35.831Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/5d/8f7f6f619d374e959aa3664ebc4b24c10abc90c2e8efbed97f2623fadaf5/igraph-1.0.0-cp39-abi3-win32.whl", hash = "sha256:c2bce3cd472fec3dd9c4d8a3ea5b6b9be65fb30edf760beb4850760dd4f2d479", size = 2725406, upload-time = "2025-10-23T12:22:37.588Z" },
+ { url = "https://files.pythonhosted.org/packages/af/77/a85b3745cf40a0572bae2de8cd9c2a2a8af78e5cf3e880fc0a249114e609/igraph-1.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:faeff8ede0cf15eb4ded44b0fcea6e1886740146e60504c24ad2da14e0939563", size = 3221663, upload-time = "2025-10-23T12:22:39.404Z" },
+ { url = "https://files.pythonhosted.org/packages/ef/7e/5df541c37bdf6493035e89c22bd53f30d99b291bcda6c78e9a8afeecec2b/igraph-1.0.0-cp39-abi3-win_arm64.whl", hash = "sha256:b607cafc24b10a615e713ee96e58208ef27e0764af80140c7cc45d4724a3f2df", size = 2785701, upload-time = "2025-10-23T12:22:41.03Z" },
+]
+
+[[package]]
+name = "imagesize"
+version = "2.0.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/6c/e6/7bf14eeb8f8b7251141944835abd42eb20a658d89084b7e1f3e5fe394090/imagesize-2.0.0.tar.gz", hash = "sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3", size = 1773045, upload-time = "2026-03-03T14:18:29.941Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/5f/53/fb7122b71361a0d121b669dcf3d31244ef75badbbb724af388948de543e2/imagesize-2.0.0-py2.py3-none-any.whl", hash = "sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96", size = 9441, upload-time = "2026-03-03T14:18:27.892Z" },
+]
+
+[[package]]
+name = "importlib-metadata"
+version = "9.0.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "zipp" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/a9/01/15bb152d77b21318514a96f43af312635eb2500c96b55398d020c93d86ea/importlib_metadata-9.0.0.tar.gz", hash = "sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc", size = 56405, upload-time = "2026-03-20T06:42:56.999Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/38/3d/2d244233ac4f76e38533cfcb2991c9eb4c7bf688ae0a036d30725b8faafe/importlib_metadata-9.0.0-py3-none-any.whl", hash = "sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7", size = 27789, upload-time = "2026-03-20T06:42:55.665Z" },
+]
+
+[[package]]
+name = "iniconfig"
+version = "2.3.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
+]
+
+[[package]]
+name = "ipykernel"
+version = "7.3.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "appnope", marker = "sys_platform == 'darwin'" },
+ { name = "comm" },
+ { name = "debugpy" },
+ { name = "ipython" },
+ { name = "jupyter-client" },
+ { name = "jupyter-core" },
+ { name = "matplotlib-inline" },
+ { name = "nest-asyncio2" },
+ { name = "packaging" },
+ { name = "psutil" },
+ { name = "pyzmq" },
+ { name = "tornado" },
+ { name = "traitlets" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/3d/c4/e4a38f579de4225a561305666f7541cdabb30075def2aa1ac17bd73c1fb5/ipykernel-7.3.0.tar.gz", hash = "sha256:9acaaaf97d16355166e4085afe9d225bfbdf2b7ef520f9df3be8f2b248275e09", size = 184899, upload-time = "2026-06-10T08:41:25.481Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/3d/02/77b271f5dc58bfbc0b577c877b2365d1ffea2afe66a80c13f2312820348c/ipykernel-7.3.0-py3-none-any.whl", hash = "sha256:897eb64da762549ef610698fca5e9675195ec6ac8ec7f19d81ce1ca20c876057", size = 120583, upload-time = "2026-06-10T08:41:23.648Z" },
+]
+
+[[package]]
+name = "ipython"
+version = "9.15.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "colorama", marker = "sys_platform == 'win32'" },
+ { name = "decorator" },
+ { name = "ipython-pygments-lexers" },
+ { name = "jedi" },
+ { name = "matplotlib-inline" },
+ { name = "pexpect", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
+ { name = "prompt-toolkit" },
+ { name = "psutil", marker = "sys_platform != 'cygwin' and sys_platform != 'emscripten'" },
+ { name = "pygments" },
+ { name = "stack-data" },
+ { name = "traitlets" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/53/59/165d3b4d75cc34add3122c4417ecb229085140ac573103c223cd01dde96f/ipython-9.15.0.tar.gz", hash = "sha256:da2819ce2aa83135257df830660b1176d986c3d2876db24df01974fa955b2756", size = 4442580, upload-time = "2026-06-26T11:03:35.913Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/40/3a/948263ca3b9d65bb2b1b0c521b3a49fad5d59ada58724bd87d2bd5ff3f36/ipython-9.15.0-py3-none-any.whl", hash = "sha256:515ad9c3cdf0c932a5a9f6245419e8aba706b7bd03c3e1d3a1c83d9351d6aa6e", size = 630895, upload-time = "2026-06-26T11:03:33.809Z" },
+]
+
+[[package]]
+name = "ipython-autotime"
+version = "0.3.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "ipython" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/c2/9d/f8b72a7cf940c704299ef712c75e862abd7fac3bbc4675a87f9851f20241/ipython-autotime-0.3.2.tar.gz", hash = "sha256:fc03ab920266072f0f6d7f1cc04247afe1c2f22f7bd052542af700939769d5f4", size = 6347, upload-time = "2023-11-01T11:43:24.725Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/d1/6b/c668b9d6a3552eeb4d82c0e6f1e79debc9cfaecc2d59ddb5b2f38c24cdfc/ipython_autotime-0.3.2-py2.py3-none-any.whl", hash = "sha256:c51e1175b9d0d2c5da51ac17963f3a836036cf389b8ef20f8a7483573d2c3dce", size = 6971, upload-time = "2023-11-01T11:43:23.549Z" },
+]
+
+[[package]]
+name = "ipython-pygments-lexers"
+version = "1.1.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "pygments" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/ef/4c/5dd1d8af08107f88c7f741ead7a40854b8ac24ddf9ae850afbcf698aa552/ipython_pygments_lexers-1.1.1.tar.gz", hash = "sha256:09c0138009e56b6854f9535736f4171d855c8c08a563a0dcd8022f78355c7e81", size = 8393, upload-time = "2025-01-17T11:24:34.505Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/d9/33/1f075bf72b0b747cb3288d011319aaf64083cf2efef8354174e3ed4540e2/ipython_pygments_lexers-1.1.1-py3-none-any.whl", hash = "sha256:a9462224a505ade19a605f71f8fa63c2048833ce50abc86768a0d81d876dc81c", size = 8074, upload-time = "2025-01-17T11:24:33.271Z" },
+]
+
+[[package]]
+name = "ipywidgets"
+version = "8.1.8"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "comm" },
+ { name = "ipython" },
+ { name = "jupyterlab-widgets" },
+ { name = "traitlets" },
+ { name = "widgetsnbextension" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/4c/ae/c5ce1edc1afe042eadb445e95b0671b03cee61895264357956e61c0d2ac0/ipywidgets-8.1.8.tar.gz", hash = "sha256:61f969306b95f85fba6b6986b7fe45d73124d1d9e3023a8068710d47a22ea668", size = 116739, upload-time = "2025-11-01T21:18:12.393Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/56/6d/0d9848617b9f753b87f214f1c682592f7ca42de085f564352f10f0843026/ipywidgets-8.1.8-py3-none-any.whl", hash = "sha256:ecaca67aed704a338f88f67b1181b58f821ab5dc89c1f0f5ef99db43c1c2921e", size = 139808, upload-time = "2025-11-01T21:18:10.956Z" },
+]
+
+[[package]]
+name = "jedi"
+version = "0.20.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "parso" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/46/b7/a3635f6a2d7cf5b5dd98064fc1d5fbbafcb25477bcea204a3a92145d158b/jedi-0.20.0.tar.gz", hash = "sha256:c3f4ccbd276696f4b19c54618d4fb18f9fc24b0aef02acf704b23f487daa1011", size = 3119416, upload-time = "2026-05-01T23:38:47.814Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/9a/93/242e2eab5fe682ffcb8b0084bde703a41d51e17ee0f3a31ff0d9d813620a/jedi-0.20.0-py2.py3-none-any.whl", hash = "sha256:7bdd9c2634f56713299976f4cbd59cb3fa92165cc5e05ea811fb253480728b67", size = 4884812, upload-time = "2026-05-01T23:38:43.919Z" },
+]
+
+[[package]]
+name = "jinja2"
+version = "3.1.6"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "markupsafe" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },
+]
+
+[[package]]
+name = "joblib"
+version = "1.5.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/41/f2/d34e8b3a08a9cc79a50b2208a93dce981fe615b64d5a4d4abee421d898df/joblib-1.5.3.tar.gz", hash = "sha256:8561a3269e6801106863fd0d6d84bb737be9e7631e33aaed3fb9ce5953688da3", size = 331603, upload-time = "2025-12-15T08:41:46.427Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/7b/91/984aca2ec129e2757d1e4e3c81c3fcda9d0f85b74670a094cc443d9ee949/joblib-1.5.3-py3-none-any.whl", hash = "sha256:5fc3c5039fc5ca8c0276333a188bbd59d6b7ab37fe6632daa76bc7f9ec18e713", size = 309071, upload-time = "2025-12-15T08:41:44.973Z" },
+]
+
+[[package]]
+name = "jsonschema"
+version = "4.26.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "attrs" },
+ { name = "jsonschema-specifications" },
+ { name = "referencing" },
+ { name = "rpds-py" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/b3/fc/e067678238fa451312d4c62bf6e6cf5ec56375422aee02f9cb5f909b3047/jsonschema-4.26.0.tar.gz", hash = "sha256:0c26707e2efad8aa1bfc5b7ce170f3fccc2e4918ff85989ba9ffa9facb2be326", size = 366583, upload-time = "2026-01-07T13:41:07.246Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/69/90/f63fb5873511e014207a475e2bb4e8b2e570d655b00ac19a9a0ca0a385ee/jsonschema-4.26.0-py3-none-any.whl", hash = "sha256:d489f15263b8d200f8387e64b4c3a75f06629559fb73deb8fdfb525f2dab50ce", size = 90630, upload-time = "2026-01-07T13:41:05.306Z" },
+]
+
+[[package]]
+name = "jsonschema-specifications"
+version = "2025.9.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "referencing" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/19/74/a633ee74eb36c44aa6d1095e7cc5569bebf04342ee146178e2d36600708b/jsonschema_specifications-2025.9.1.tar.gz", hash = "sha256:b540987f239e745613c7a9176f3edb72b832a4ac465cf02712288397832b5e8d", size = 32855, upload-time = "2025-09-08T01:34:59.186Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/41/45/1a4ed80516f02155c51f51e8cedb3c1902296743db0bbc66608a0db2814f/jsonschema_specifications-2025.9.1-py3-none-any.whl", hash = "sha256:98802fee3a11ee76ecaca44429fda8a41bff98b00a0f2838151b113f210cc6fe", size = 18437, upload-time = "2025-09-08T01:34:57.871Z" },
+]
+
+[[package]]
+name = "jupyter-cache"
+version = "1.0.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "attrs" },
+ { name = "click" },
+ { name = "importlib-metadata" },
+ { name = "nbclient" },
+ { name = "nbformat" },
+ { name = "pyyaml" },
+ { name = "sqlalchemy" },
+ { name = "tabulate" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/bb/f7/3627358075f183956e8c4974603232b03afd4ddc7baf72c2bc9fff522291/jupyter_cache-1.0.1.tar.gz", hash = "sha256:16e808eb19e3fb67a223db906e131ea6e01f03aa27f49a7214ce6a5fec186fb9", size = 32048, upload-time = "2024-11-15T16:03:55.322Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/64/6b/67b87da9d36bff9df7d0efbd1a325fa372a43be7158effaf43ed7b22341d/jupyter_cache-1.0.1-py3-none-any.whl", hash = "sha256:9c3cafd825ba7da8b5830485343091143dff903e4d8c69db9349b728b140abf6", size = 33907, upload-time = "2024-11-15T16:03:54.021Z" },
+]
+
+[[package]]
+name = "jupyter-client"
+version = "8.9.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "jupyter-core" },
+ { name = "python-dateutil" },
+ { name = "pyzmq" },
+ { name = "tornado" },
+ { name = "traitlets" },
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/7d/dc/5512503b088997c2250b8bf18258fba9d9ce5ead641183700960d3c9d342/jupyter_client-8.9.1.tar.gz", hash = "sha256:a58f730dd9e728ba16ba1d62ebccf7ffe1ebbdbce4e95cfae941b7321ae1f4fa", size = 359256, upload-time = "2026-06-09T13:15:01.033Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/3f/6f/56d39bf385c5c27988aebaf0c18a2a17e960575740100973511018bd904e/jupyter_client-8.9.1-py3-none-any.whl", hash = "sha256:0b7a295bc46e8751e9adae84781f726c851c1d911bd793edc4a3bde942e3da81", size = 109828, upload-time = "2026-06-09T13:14:58.835Z" },
+]
+
+[[package]]
+name = "jupyter-core"
+version = "5.9.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "platformdirs" },
+ { name = "traitlets" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/02/49/9d1284d0dc65e2c757b74c6687b6d319b02f822ad039e5c512df9194d9dd/jupyter_core-5.9.1.tar.gz", hash = "sha256:4d09aaff303b9566c3ce657f580bd089ff5c91f5f89cf7d8846c3cdf465b5508", size = 89814, upload-time = "2025-10-16T19:19:18.444Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/e7/e7/80988e32bf6f73919a113473a604f5a8f09094de312b9d52b79c2df7612b/jupyter_core-5.9.1-py3-none-any.whl", hash = "sha256:ebf87fdc6073d142e114c72c9e29a9d7ca03fad818c5d300ce2adc1fb0743407", size = 29032, upload-time = "2025-10-16T19:19:16.783Z" },
+]
+
+[[package]]
+name = "jupyterlab-widgets"
+version = "3.0.16"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/26/2d/ef58fed122b268c69c0aa099da20bc67657cdfb2e222688d5731bd5b971d/jupyterlab_widgets-3.0.16.tar.gz", hash = "sha256:423da05071d55cf27a9e602216d35a3a65a3e41cdf9c5d3b643b814ce38c19e0", size = 897423, upload-time = "2025-11-01T21:11:29.724Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ab/b5/36c712098e6191d1b4e349304ef73a8d06aed77e56ceaac8c0a306c7bda1/jupyterlab_widgets-3.0.16-py3-none-any.whl", hash = "sha256:45fa36d9c6422cf2559198e4db481aa243c7a32d9926b500781c830c80f7ecf8", size = 914926, upload-time = "2025-11-01T21:11:28.008Z" },
+]
+
+[[package]]
+name = "kiwisolver"
+version = "1.5.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/d0/67/9c61eccb13f0bdca9307614e782fec49ffdde0f7a2314935d489fa93cd9c/kiwisolver-1.5.0.tar.gz", hash = "sha256:d4193f3d9dc3f6f79aaed0e5637f45d98850ebf01f7ca20e69457f3e8946b66a", size = 103482, upload-time = "2026-03-09T13:15:53.382Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/4d/b2/818b74ebea34dabe6d0c51cb1c572e046730e64844da6ed646d5298c40ce/kiwisolver-1.5.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:4e9750bc21b886308024f8a54ccb9a2cc38ac9fa813bf4348434e3d54f337ff9", size = 123158, upload-time = "2026-03-09T13:13:23.127Z" },
+ { url = "https://files.pythonhosted.org/packages/bf/d9/405320f8077e8e1c5c4bd6adc45e1e6edf6d727b6da7f2e2533cf58bff71/kiwisolver-1.5.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:72ec46b7eba5b395e0a7b63025490d3214c11013f4aacb4f5e8d6c3041829588", size = 66388, upload-time = "2026-03-09T13:13:24.765Z" },
+ { url = "https://files.pythonhosted.org/packages/99/9f/795fedf35634f746151ca8839d05681ceb6287fbed6cc1c9bf235f7887c2/kiwisolver-1.5.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ed3a984b31da7481b103f68776f7128a89ef26ed40f4dc41a2223cda7fb24819", size = 64068, upload-time = "2026-03-09T13:13:25.878Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/13/680c54afe3e65767bed7ec1a15571e1a2f1257128733851ade24abcefbcc/kiwisolver-1.5.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:bb5136fb5352d3f422df33f0c879a1b0c204004324150cc3b5e3c4f310c9049f", size = 1477934, upload-time = "2026-03-09T13:13:27.166Z" },
+ { url = "https://files.pythonhosted.org/packages/c8/2f/cebfcdb60fd6a9b0f6b47a9337198bcbad6fbe15e68189b7011fd914911f/kiwisolver-1.5.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b2af221f268f5af85e776a73d62b0845fc8baf8ef0abfae79d29c77d0e776aaf", size = 1278537, upload-time = "2026-03-09T13:13:28.707Z" },
+ { url = "https://files.pythonhosted.org/packages/f2/0d/9b782923aada3fafb1d6b84e13121954515c669b18af0c26e7d21f579855/kiwisolver-1.5.0-cp312-cp312-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b0f172dc8ffaccb8522d7c5d899de00133f2f1ca7b0a49b7da98e901de87bf2d", size = 1296685, upload-time = "2026-03-09T13:13:30.528Z" },
+ { url = "https://files.pythonhosted.org/packages/27/70/83241b6634b04fe44e892688d5208332bde130f38e610c0418f9ede47ded/kiwisolver-1.5.0-cp312-cp312-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:6ab8ba9152203feec73758dad83af9a0bbe05001eb4639e547207c40cfb52083", size = 1346024, upload-time = "2026-03-09T13:13:32.818Z" },
+ { url = "https://files.pythonhosted.org/packages/e4/db/30ed226fb271ae1a6431fc0fe0edffb2efe23cadb01e798caeb9f2ceae8f/kiwisolver-1.5.0-cp312-cp312-manylinux_2_39_riscv64.whl", hash = "sha256:cdee07c4d7f6d72008d3f73b9bf027f4e11550224c7c50d8df1ae4a37c1402a6", size = 987241, upload-time = "2026-03-09T13:13:34.435Z" },
+ { url = "https://files.pythonhosted.org/packages/ec/bd/c314595208e4c9587652d50959ead9e461995389664e490f4dce7ff0f782/kiwisolver-1.5.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:7c60d3c9b06fb23bd9c6139281ccbdc384297579ae037f08ae90c69f6845c0b1", size = 2227742, upload-time = "2026-03-09T13:13:36.4Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/43/0499cec932d935229b5543d073c2b87c9c22846aab48881e9d8d6e742a2d/kiwisolver-1.5.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:e315e5ec90d88e140f57696ff85b484ff68bb311e36f2c414aa4286293e6dee0", size = 2323966, upload-time = "2026-03-09T13:13:38.204Z" },
+ { url = "https://files.pythonhosted.org/packages/3d/6f/79b0d760907965acfd9d61826a3d41f8f093c538f55cd2633d3f0db269f6/kiwisolver-1.5.0-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:1465387ac63576c3e125e5337a6892b9e99e0627d52317f3ca79e6930d889d15", size = 1977417, upload-time = "2026-03-09T13:13:39.966Z" },
+ { url = "https://files.pythonhosted.org/packages/ab/31/01d0537c41cb75a551a438c3c7a80d0c60d60b81f694dac83dd436aec0d0/kiwisolver-1.5.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:530a3fd64c87cffa844d4b6b9768774763d9caa299e9b75d8eca6a4423b31314", size = 2491238, upload-time = "2026-03-09T13:13:41.698Z" },
+ { url = "https://files.pythonhosted.org/packages/e4/34/8aefdd0be9cfd00a44509251ba864f5caf2991e36772e61c408007e7f417/kiwisolver-1.5.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:1d9daea4ea6b9be74fe2f01f7fbade8d6ffab263e781274cffca0dba9be9eec9", size = 2294947, upload-time = "2026-03-09T13:13:43.343Z" },
+ { url = "https://files.pythonhosted.org/packages/ad/cf/0348374369ca588f8fe9c338fae49fa4e16eeb10ffb3d012f23a54578a9e/kiwisolver-1.5.0-cp312-cp312-win_amd64.whl", hash = "sha256:f18c2d9782259a6dc132fdc7a63c168cbc74b35284b6d75c673958982a378384", size = 73569, upload-time = "2026-03-09T13:13:45.792Z" },
+ { url = "https://files.pythonhosted.org/packages/28/26/192b26196e2316e2bd29deef67e37cdf9870d9af8e085e521afff0fed526/kiwisolver-1.5.0-cp312-cp312-win_arm64.whl", hash = "sha256:f7c7553b13f69c1b29a5bde08ddc6d9d0c8bfb84f9ed01c30db25944aeb852a7", size = 64997, upload-time = "2026-03-09T13:13:46.878Z" },
+ { url = "https://files.pythonhosted.org/packages/9d/69/024d6711d5ba575aa65d5538042e99964104e97fa153a9f10bc369182bc2/kiwisolver-1.5.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:fd40bb9cd0891c4c3cb1ddf83f8bbfa15731a248fdc8162669405451e2724b09", size = 123166, upload-time = "2026-03-09T13:13:48.032Z" },
+ { url = "https://files.pythonhosted.org/packages/ce/48/adbb40df306f587054a348831220812b9b1d787aff714cfbc8556e38fccd/kiwisolver-1.5.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:c0e1403fd7c26d77c1f03e096dc58a5c726503fa0db0456678b8668f76f521e3", size = 66395, upload-time = "2026-03-09T13:13:49.365Z" },
+ { url = "https://files.pythonhosted.org/packages/a8/3a/d0a972b34e1c63e2409413104216cd1caa02c5a37cb668d1687d466c1c45/kiwisolver-1.5.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:dda366d548e89a90d88a86c692377d18d8bd64b39c1fb2b92cb31370e2896bbd", size = 64065, upload-time = "2026-03-09T13:13:50.562Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/0a/7b98e1e119878a27ba8618ca1e18b14f992ff1eda40f47bccccf4de44121/kiwisolver-1.5.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:332b4f0145c30b5f5ad9374881133e5aa64320428a57c2c2b61e9d891a51c2f3", size = 1477903, upload-time = "2026-03-09T13:13:52.084Z" },
+ { url = "https://files.pythonhosted.org/packages/18/d8/55638d89ffd27799d5cc3d8aa28e12f4ce7a64d67b285114dbedc8ea4136/kiwisolver-1.5.0-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0c50b89ffd3e1a911c69a1dd3de7173c0cd10b130f56222e57898683841e4f96", size = 1278751, upload-time = "2026-03-09T13:13:54.673Z" },
+ { url = "https://files.pythonhosted.org/packages/b8/97/b4c8d0d18421ecceba20ad8701358453b88e32414e6f6950b5a4bad54e65/kiwisolver-1.5.0-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4db576bb8c3ef9365f8b40fe0f671644de6736ae2c27a2c62d7d8a1b4329f099", size = 1296793, upload-time = "2026-03-09T13:13:56.287Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/10/f862f94b6389d8957448ec9df59450b81bec4abb318805375c401a1e6892/kiwisolver-1.5.0-cp313-cp313-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:0b85aad90cea8ac6797a53b5d5f2e967334fa4d1149f031c4537569972596cb8", size = 1346041, upload-time = "2026-03-09T13:13:58.269Z" },
+ { url = "https://files.pythonhosted.org/packages/a3/6a/f1650af35821eaf09de398ec0bc2aefc8f211f0cda50204c9f1673741ba9/kiwisolver-1.5.0-cp313-cp313-manylinux_2_39_riscv64.whl", hash = "sha256:d36ca54cb4c6c4686f7cbb7b817f66f5911c12ddb519450bbe86707155028f87", size = 987292, upload-time = "2026-03-09T13:13:59.871Z" },
+ { url = "https://files.pythonhosted.org/packages/de/19/d7fb82984b9238115fe629c915007be608ebd23dc8629703d917dbfaffd4/kiwisolver-1.5.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:38f4a703656f493b0ad185211ccfca7f0386120f022066b018eb5296d8613e23", size = 2227865, upload-time = "2026-03-09T13:14:01.401Z" },
+ { url = "https://files.pythonhosted.org/packages/7f/b9/46b7f386589fd222dac9e9de9c956ce5bcefe2ee73b4e79891381dda8654/kiwisolver-1.5.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:3ac2360e93cb41be81121755c6462cff3beaa9967188c866e5fce5cf13170859", size = 2324369, upload-time = "2026-03-09T13:14:02.972Z" },
+ { url = "https://files.pythonhosted.org/packages/92/8b/95e237cf3d9c642960153c769ddcbe278f182c8affb20cecc1cc983e7cc5/kiwisolver-1.5.0-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:c95cab08d1965db3d84a121f1c7ce7479bdd4072c9b3dafd8fecce48a2e6b902", size = 1977989, upload-time = "2026-03-09T13:14:04.503Z" },
+ { url = "https://files.pythonhosted.org/packages/1b/95/980c9df53501892784997820136c01f62bc1865e31b82b9560f980c0e649/kiwisolver-1.5.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:fc20894c3d21194d8041a28b65622d5b86db786da6e3cfe73f0c762951a61167", size = 2491645, upload-time = "2026-03-09T13:14:06.106Z" },
+ { url = "https://files.pythonhosted.org/packages/cb/32/900647fd0840abebe1561792c6b31e6a7c0e278fc3973d30572a965ca14c/kiwisolver-1.5.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:7a32f72973f0f950c1920475d5c5ea3d971b81b6f0ec53b8d0a956cc965f22e0", size = 2295237, upload-time = "2026-03-09T13:14:08.891Z" },
+ { url = "https://files.pythonhosted.org/packages/be/8a/be60e3bbcf513cc5a50f4a3e88e1dcecebb79c1ad607a7222877becaa101/kiwisolver-1.5.0-cp313-cp313-win_amd64.whl", hash = "sha256:0bf3acf1419fa93064a4c2189ac0b58e3be7872bf6ee6177b0d4c63dc4cea276", size = 73573, upload-time = "2026-03-09T13:14:12.327Z" },
+ { url = "https://files.pythonhosted.org/packages/4d/d2/64be2e429eb4fca7f7e1c52a91b12663aeaf25de3895e5cca0f47ef2a8d0/kiwisolver-1.5.0-cp313-cp313-win_arm64.whl", hash = "sha256:fa8eb9ecdb7efb0b226acec134e0d709e87a909fa4971a54c0c4f6e88635484c", size = 64998, upload-time = "2026-03-09T13:14:13.469Z" },
+ { url = "https://files.pythonhosted.org/packages/b0/69/ce68dd0c85755ae2de490bf015b62f2cea5f6b14ff00a463f9d0774449ff/kiwisolver-1.5.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:db485b3847d182b908b483b2ed133c66d88d49cacf98fd278fadafe11b4478d1", size = 125700, upload-time = "2026-03-09T13:14:14.636Z" },
+ { url = "https://files.pythonhosted.org/packages/74/aa/937aac021cf9d4349990d47eb319309a51355ed1dbdc9c077cdc9224cb11/kiwisolver-1.5.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:be12f931839a3bdfe28b584db0e640a65a8bcbc24560ae3fdb025a449b3d754e", size = 67537, upload-time = "2026-03-09T13:14:15.808Z" },
+ { url = "https://files.pythonhosted.org/packages/ee/20/3a87fbece2c40ad0f6f0aefa93542559159c5f99831d596050e8afae7a9f/kiwisolver-1.5.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:16b85d37c2cbb3253226d26e64663f755d88a03439a9c47df6246b35defbdfb7", size = 65514, upload-time = "2026-03-09T13:14:18.035Z" },
+ { url = "https://files.pythonhosted.org/packages/f0/7f/f943879cda9007c45e1f7dba216d705c3a18d6b35830e488b6c6a4e7cdf0/kiwisolver-1.5.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4432b835675f0ea7414aab3d37d119f7226d24869b7a829caeab49ebda407b0c", size = 1584848, upload-time = "2026-03-09T13:14:19.745Z" },
+ { url = "https://files.pythonhosted.org/packages/37/f8/4d4f85cc1870c127c88d950913370dd76138482161cd07eabbc450deff01/kiwisolver-1.5.0-cp313-cp313t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b0feb50971481a2cc44d94e88bdb02cdd497618252ae226b8eb1201b957e368", size = 1391542, upload-time = "2026-03-09T13:14:21.54Z" },
+ { url = "https://files.pythonhosted.org/packages/04/0b/65dd2916c84d252b244bd405303220f729e7c17c9d7d33dca6feeff9ffc4/kiwisolver-1.5.0-cp313-cp313t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:56fa888f10d0f367155e76ce849fa1166fc9730d13bd2d65a2aa13b6f5424489", size = 1404447, upload-time = "2026-03-09T13:14:23.205Z" },
+ { url = "https://files.pythonhosted.org/packages/39/5c/2606a373247babce9b1d056c03a04b65f3cf5290a8eac5d7bdead0a17e21/kiwisolver-1.5.0-cp313-cp313t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:940dda65d5e764406b9fb92761cbf462e4e63f712ab60ed98f70552e496f3bf1", size = 1455918, upload-time = "2026-03-09T13:14:24.74Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/d1/c6078b5756670658e9192a2ef11e939c92918833d2745f85cd14a6004bdf/kiwisolver-1.5.0-cp313-cp313t-manylinux_2_39_riscv64.whl", hash = "sha256:89fc958c702ee9a745e4700378f5d23fddbc46ff89e8fdbf5395c24d5c1452a3", size = 1072856, upload-time = "2026-03-09T13:14:26.597Z" },
+ { url = "https://files.pythonhosted.org/packages/cb/c8/7def6ddf16eb2b3741d8b172bdaa9af882b03c78e9b0772975408801fa63/kiwisolver-1.5.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9027d773c4ff81487181a925945743413f6069634d0b122d0b37684ccf4f1e18", size = 2333580, upload-time = "2026-03-09T13:14:28.237Z" },
+ { url = "https://files.pythonhosted.org/packages/9e/87/2ac1fce0eb1e616fcd3c35caa23e665e9b1948bb984f4764790924594128/kiwisolver-1.5.0-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:5b233ea3e165e43e35dba1d2b8ecc21cf070b45b65ae17dd2747d2713d942021", size = 2423018, upload-time = "2026-03-09T13:14:30.018Z" },
+ { url = "https://files.pythonhosted.org/packages/67/13/c6700ccc6cc218716bfcda4935e4b2997039869b4ad8a94f364c5a3b8e63/kiwisolver-1.5.0-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:ce9bf03dad3b46408c08649c6fbd6ca28a9fce0eb32fdfffa6775a13103b5310", size = 2062804, upload-time = "2026-03-09T13:14:32.888Z" },
+ { url = "https://files.pythonhosted.org/packages/1b/bd/877056304626943ff0f1f44c08f584300c199b887cb3176cd7e34f1515f1/kiwisolver-1.5.0-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:fc4d3f1fb9ca0ae9f97b095963bc6326f1dbfd3779d6679a1e016b9baaa153d3", size = 2597482, upload-time = "2026-03-09T13:14:34.971Z" },
+ { url = "https://files.pythonhosted.org/packages/75/19/c60626c47bf0f8ac5dcf72c6c98e266d714f2fbbfd50cf6dab5ede3aaa50/kiwisolver-1.5.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:f443b4825c50a51ee68585522ab4a1d1257fac65896f282b4c6763337ac9f5d2", size = 2394328, upload-time = "2026-03-09T13:14:36.816Z" },
+ { url = "https://files.pythonhosted.org/packages/47/84/6a6d5e5bb8273756c27b7d810d47f7ef2f1f9b9fd23c9ee9a3f8c75c9cef/kiwisolver-1.5.0-cp313-cp313t-win_arm64.whl", hash = "sha256:893ff3a711d1b515ba9da14ee090519bad4610ed1962fbe298a434e8c5f8db53", size = 68410, upload-time = "2026-03-09T13:14:38.695Z" },
+ { url = "https://files.pythonhosted.org/packages/e4/d7/060f45052f2a01ad5762c8fdecd6d7a752b43400dc29ff75cd47225a40fd/kiwisolver-1.5.0-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:8df31fe574b8b3993cc61764f40941111b25c2d9fea13d3ce24a49907cd2d615", size = 123231, upload-time = "2026-03-09T13:14:41.323Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/a7/78da680eadd06ff35edef6ef68a1ad273bad3e2a0936c9a885103230aece/kiwisolver-1.5.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:1d49a49ac4cbfb7c1375301cd1ec90169dfeae55ff84710d782260ce77a75a02", size = 66489, upload-time = "2026-03-09T13:14:42.534Z" },
+ { url = "https://files.pythonhosted.org/packages/49/b2/97980f3ad4fae37dd7fe31626e2bf75fbf8bdf5d303950ec1fab39a12da8/kiwisolver-1.5.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:0cbe94b69b819209a62cb27bdfa5dc2a8977d8de2f89dfd97ba4f53ed3af754e", size = 64063, upload-time = "2026-03-09T13:14:44.759Z" },
+ { url = "https://files.pythonhosted.org/packages/e7/f9/b06c934a6aa8bc91f566bd2a214fd04c30506c2d9e2b6b171953216a65b6/kiwisolver-1.5.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:80aa065ffd378ff784822a6d7c3212f2d5f5e9c3589614b5c228b311fd3063ac", size = 1475913, upload-time = "2026-03-09T13:14:46.247Z" },
+ { url = "https://files.pythonhosted.org/packages/6b/f0/f768ae564a710135630672981231320bc403cf9152b5596ec5289de0f106/kiwisolver-1.5.0-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e7f886f47ab881692f278ae901039a234e4025a68e6dfab514263a0b1c4ae05", size = 1282782, upload-time = "2026-03-09T13:14:48.458Z" },
+ { url = "https://files.pythonhosted.org/packages/e2/9f/1de7aad00697325f05238a5f2eafbd487fb637cc27a558b5367a5f37fb7f/kiwisolver-1.5.0-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:5060731cc3ed12ca3a8b57acd4aeca5bbc2f49216dd0bec1650a1acd89486bcd", size = 1300815, upload-time = "2026-03-09T13:14:50.721Z" },
+ { url = "https://files.pythonhosted.org/packages/5a/c2/297f25141d2e468e0ce7f7a7b92e0cf8918143a0cbd3422c1ad627e85a06/kiwisolver-1.5.0-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:7a4aa69609f40fce3cbc3f87b2061f042eee32f94b8f11db707b66a26461591a", size = 1347925, upload-time = "2026-03-09T13:14:52.304Z" },
+ { url = "https://files.pythonhosted.org/packages/b9/d3/f4c73a02eb41520c47610207b21afa8cdd18fdbf64ffd94674ae21c4812d/kiwisolver-1.5.0-cp314-cp314-manylinux_2_39_riscv64.whl", hash = "sha256:d168fda2dbff7b9b5f38e693182d792a938c31db4dac3a80a4888de603c99554", size = 991322, upload-time = "2026-03-09T13:14:54.637Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/46/d3f2efef7732fcda98d22bf4ad5d3d71d545167a852ca710a494f4c15343/kiwisolver-1.5.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:413b820229730d358efd838ecbab79902fe97094565fdc80ddb6b0a18c18a581", size = 2232857, upload-time = "2026-03-09T13:14:56.471Z" },
+ { url = "https://files.pythonhosted.org/packages/3f/ec/2d9756bf2b6d26ae4349b8d3662fb3993f16d80c1f971c179ce862b9dbae/kiwisolver-1.5.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:5124d1ea754509b09e53738ec185584cc609aae4a3b510aaf4ed6aa047ef9303", size = 2329376, upload-time = "2026-03-09T13:14:58.072Z" },
+ { url = "https://files.pythonhosted.org/packages/8f/9f/876a0a0f2260f1bde92e002b3019a5fabc35e0939c7d945e0fa66185eb20/kiwisolver-1.5.0-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:e4415a8db000bf49a6dd1c478bf70062eaacff0f462b92b0ba68791a905861f9", size = 1982549, upload-time = "2026-03-09T13:14:59.668Z" },
+ { url = "https://files.pythonhosted.org/packages/6c/4f/ba3624dfac23a64d54ac4179832860cb537c1b0af06024936e82ca4154a0/kiwisolver-1.5.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:d618fd27420381a4f6044faa71f46d8bfd911bd077c555f7138ed88729bfbe79", size = 2494680, upload-time = "2026-03-09T13:15:01.364Z" },
+ { url = "https://files.pythonhosted.org/packages/39/b7/97716b190ab98911b20d10bf92eca469121ec483b8ce0edd314f51bc85af/kiwisolver-1.5.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5092eb5b1172947f57d6ea7d89b2f29650414e4293c47707eb499ec07a0ac796", size = 2297905, upload-time = "2026-03-09T13:15:03.925Z" },
+ { url = "https://files.pythonhosted.org/packages/a3/36/4e551e8aa55c9188bca9abb5096805edbf7431072b76e2298e34fd3a3008/kiwisolver-1.5.0-cp314-cp314-win_amd64.whl", hash = "sha256:d76e2d8c75051d58177e762164d2e9ab92886534e3a12e795f103524f221dd8e", size = 75086, upload-time = "2026-03-09T13:15:07.775Z" },
+ { url = "https://files.pythonhosted.org/packages/70/15/9b90f7df0e31a003c71649cf66ef61c3c1b862f48c81007fa2383c8bd8d7/kiwisolver-1.5.0-cp314-cp314-win_arm64.whl", hash = "sha256:fa6248cd194edff41d7ea9425ced8ca3a6f838bfb295f6f1d6e6bb694a8518df", size = 66577, upload-time = "2026-03-09T13:15:09.139Z" },
+ { url = "https://files.pythonhosted.org/packages/17/01/7dc8c5443ff42b38e72731643ed7cf1ed9bf01691ae5cdca98501999ed83/kiwisolver-1.5.0-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:d1ffeb80b5676463d7a7d56acbe8e37a20ce725570e09549fe738e02ca6b7e1e", size = 125794, upload-time = "2026-03-09T13:15:10.525Z" },
+ { url = "https://files.pythonhosted.org/packages/46/8a/b4ebe46ebaac6a303417fab10c2e165c557ddaff558f9699d302b256bc53/kiwisolver-1.5.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:bc4d8e252f532ab46a1de9349e2d27b91fce46736a9eedaa37beaca66f574ed4", size = 67646, upload-time = "2026-03-09T13:15:12.016Z" },
+ { url = "https://files.pythonhosted.org/packages/60/35/10a844afc5f19d6f567359bf4789e26661755a2f36200d5d1ed8ad0126e5/kiwisolver-1.5.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:6783e069732715ad0c3ce96dbf21dbc2235ab0593f2baf6338101f70371f4028", size = 65511, upload-time = "2026-03-09T13:15:13.311Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/8a/685b297052dd041dcebce8e8787b58923b6e78acc6115a0dc9189011c44b/kiwisolver-1.5.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:e7c4c09a490dc4d4a7f8cbee56c606a320f9dc28cf92a7157a39d1ce7676a657", size = 1584858, upload-time = "2026-03-09T13:15:15.103Z" },
+ { url = "https://files.pythonhosted.org/packages/9e/80/04865e3d4638ac5bddec28908916df4a3075b8c6cc101786a96803188b96/kiwisolver-1.5.0-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2a075bd7bd19c70cf67c8badfa36cf7c5d8de3c9ddb8420c51e10d9c50e94920", size = 1392539, upload-time = "2026-03-09T13:15:16.661Z" },
+ { url = "https://files.pythonhosted.org/packages/ba/01/77a19cacc0893fa13fafa46d1bba06fb4dc2360b3292baf4b56d8e067b24/kiwisolver-1.5.0-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:bdd3e53429ff02aa319ba59dfe4ceeec345bf46cf180ec2cf6fd5b942e7975e9", size = 1405310, upload-time = "2026-03-09T13:15:18.229Z" },
+ { url = "https://files.pythonhosted.org/packages/53/39/bcaf5d0cca50e604cfa9b4e3ae1d64b50ca1ae5b754122396084599ef903/kiwisolver-1.5.0-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:3cdcb35dc9d807259c981a85531048ede628eabcffb3239adf3d17463518992d", size = 1456244, upload-time = "2026-03-09T13:15:20.444Z" },
+ { url = "https://files.pythonhosted.org/packages/d0/7a/72c187abc6975f6978c3e39b7cf67aeb8b3c0a8f9790aa7fd412855e9e1f/kiwisolver-1.5.0-cp314-cp314t-manylinux_2_39_riscv64.whl", hash = "sha256:70d593af6a6ca332d1df73d519fddb5148edb15cd90d5f0155e3746a6d4fcc65", size = 1073154, upload-time = "2026-03-09T13:15:22.039Z" },
+ { url = "https://files.pythonhosted.org/packages/c7/ca/cf5b25783ebbd59143b4371ed0c8428a278abe68d6d0104b01865b1bbd0f/kiwisolver-1.5.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:377815a8616074cabbf3f53354e1d040c35815a134e01d7614b7692e4bf8acfa", size = 2334377, upload-time = "2026-03-09T13:15:23.741Z" },
+ { url = "https://files.pythonhosted.org/packages/4a/e5/b1f492adc516796e88751282276745340e2a72dcd0d36cf7173e0daf3210/kiwisolver-1.5.0-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:0255a027391d52944eae1dbb5d4cc5903f57092f3674e8e544cdd2622826b3f0", size = 2425288, upload-time = "2026-03-09T13:15:25.789Z" },
+ { url = "https://files.pythonhosted.org/packages/e6/e5/9b21fbe91a61b8f409d74a26498706e97a48008bfcd1864373d32a6ba31c/kiwisolver-1.5.0-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:012b1eb16e28718fa782b5e61dc6f2da1f0792ca73bd05d54de6cb9561665fc9", size = 2063158, upload-time = "2026-03-09T13:15:27.63Z" },
+ { url = "https://files.pythonhosted.org/packages/b1/02/83f47986138310f95ea95531f851b2a62227c11cbc3e690ae1374fe49f0f/kiwisolver-1.5.0-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:0e3aafb33aed7479377e5e9a82e9d4bf87063741fc99fc7ae48b0f16e32bdd6f", size = 2597260, upload-time = "2026-03-09T13:15:29.421Z" },
+ { url = "https://files.pythonhosted.org/packages/07/18/43a5f24608d8c313dd189cf838c8e68d75b115567c6279de7796197cfb6a/kiwisolver-1.5.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:e7a116ae737f0000343218c4edf5bd45893bfeaff0993c0b215d7124c9f77646", size = 2394403, upload-time = "2026-03-09T13:15:31.517Z" },
+ { url = "https://files.pythonhosted.org/packages/3b/b5/98222136d839b8afabcaa943b09bd05888c2d36355b7e448550211d1fca4/kiwisolver-1.5.0-cp314-cp314t-win_amd64.whl", hash = "sha256:1dd9b0b119a350976a6d781e7278ec7aca0b201e1a9e2d23d9804afecb6ca681", size = 79687, upload-time = "2026-03-09T13:15:33.204Z" },
+ { url = "https://files.pythonhosted.org/packages/99/a2/ca7dc962848040befed12732dff6acae7fb3c4f6fc4272b3f6c9a30b8713/kiwisolver-1.5.0-cp314-cp314t-win_arm64.whl", hash = "sha256:58f812017cd2985c21fbffb4864d59174d4903dd66fa23815e74bbc7a0e2dd57", size = 70032, upload-time = "2026-03-09T13:15:34.411Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/fa/2910df836372d8761bb6eff7d8bdcb1613b5c2e03f260efe7abe34d388a7/kiwisolver-1.5.0-graalpy312-graalpy250_312_native-macosx_10_13_x86_64.whl", hash = "sha256:5ae8e62c147495b01a0f4765c878e9bfdf843412446a247e28df59936e99e797", size = 130262, upload-time = "2026-03-09T13:15:35.629Z" },
+ { url = "https://files.pythonhosted.org/packages/0f/41/c5f71f9f00aabcc71fee8b7475e3f64747282580c2fe748961ba29b18385/kiwisolver-1.5.0-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:f6764a4ccab3078db14a632420930f6186058750df066b8ea2a7106df91d3203", size = 138036, upload-time = "2026-03-09T13:15:36.894Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/06/7399a607f434119c6e1fdc8ec89a8d51ccccadf3341dee4ead6bd14caaf5/kiwisolver-1.5.0-graalpy312-graalpy250_312_native-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c31c13da98624f957b0fb1b5bae5383b2333c2c3f6793d9825dd5ce79b525cb7", size = 194295, upload-time = "2026-03-09T13:15:38.22Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/91/53255615acd2a1eaca307ede3c90eb550bae9c94581f8c00081b6b1c8f44/kiwisolver-1.5.0-graalpy312-graalpy250_312_native-win_amd64.whl", hash = "sha256:1f1489f769582498610e015a8ef2d36f28f505ab3096d0e16b4858a9ec214f57", size = 75987, upload-time = "2026-03-09T13:15:39.65Z" },
+]
+
+[[package]]
+name = "kneed"
+version = "0.8.6"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+ { name = "scipy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/72/64/4bb8f8a7a4627b585a66d5bec0c9b30ae5b39a4caea1775c8bfb3fb3f4cf/kneed-0.8.6.tar.gz", hash = "sha256:65b22727c623661701f15edf057f2e6c73e2b1ad4e68cd9ca4291675c318b5ef", size = 13161, upload-time = "2026-03-20T21:01:51.966Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/4a/cd/23c89d53c36028bccb39f55aa5dd24c4bdaab76c4d556ad43dc8cf026918/kneed-0.8.6-py3-none-any.whl", hash = "sha256:3412e7b70bce07717386d24fab37f0f985968d1b85ea0c749a6b98caccaf65ec", size = 10797, upload-time = "2026-03-20T21:01:50.87Z" },
+]
+
+[[package]]
+name = "legacy-api-wrap"
+version = "1.5"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/58/49/f06f94048c8974205730d40beca879e43b6eee08efb0101cfb8623e60f41/legacy_api_wrap-1.5.tar.gz", hash = "sha256:b41ba6532f3ebfe3a897a35a7f97dec3be04b92a450f6c2bcf89f1b91c9cadf2", size = 11610, upload-time = "2025-11-03T13:21:12.437Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/41/5b/058db09c45ba58a7321bdf2294cae651b37d6fec68117265af90cde043b0/legacy_api_wrap-1.5-py3-none-any.whl", hash = "sha256:5a8ea50e3e3bcbcdec3447b77034fd0d32cb2cf4089db799238708e4d7e0098d", size = 10182, upload-time = "2025-11-03T13:21:11.102Z" },
+]
+
+[[package]]
+name = "leidenalg"
+version = "0.12.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "igraph" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/82/b7/45bae25ab59321a14902bec2f47e32b92e76f8b4b3887a13ee69a0b32c60/leidenalg-0.12.0.tar.gz", hash = "sha256:c81a45a2fb874fe71e903d43a869d0efeb7f907af70476c03f6945ef0e8f44c5", size = 453641, upload-time = "2026-05-24T10:17:53.556Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/53/7f/0900c05ccbb27744f9dd11321dc2978e9f5ffe678db54f23eb94cf4e5951/leidenalg-0.12.0-cp38-abi3-macosx_10_9_x86_64.whl", hash = "sha256:96eec407f540d9ddb0c103dadc632d5519cfcdb39689fb21ae0ceeaa97bb2ec3", size = 2257399, upload-time = "2026-05-24T10:17:32.257Z" },
+ { url = "https://files.pythonhosted.org/packages/72/f6/d15c889c7c65dbc41c84fe83243c203b083f70cef62ed0acc630098b2012/leidenalg-0.12.0-cp38-abi3-macosx_11_0_arm64.whl", hash = "sha256:2e0e1c2321c2e06a7f4f8a73ca15fcd54e0139a62afb448933c42badcfab8adc", size = 1926700, upload-time = "2026-05-24T10:17:33.946Z" },
+ { url = "https://files.pythonhosted.org/packages/cf/36/7480eef0473bd87f69826e0dfb6bb1c1ee0915fa6d31be445d0bfcbd3a1a/leidenalg-0.12.0-cp38-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f6a97f698f13a588e1ae7344908cddee9f87d33acc226fc5c5b99752ebd71f8", size = 2546580, upload-time = "2026-05-24T10:17:35.133Z" },
+ { url = "https://files.pythonhosted.org/packages/5f/45/cf5eb36b4517bb85397760a6033628695e8b0723b45f10f487b07cec84df/leidenalg-0.12.0-cp38-abi3-manylinux_2_26_i686.manylinux_2_28_i686.whl", hash = "sha256:78df2e463f3ffffffda590e9b0d790107e59f722f095731b78a58803d1ca92e5", size = 2845846, upload-time = "2026-05-24T10:17:37.115Z" },
+ { url = "https://files.pythonhosted.org/packages/f7/be/dba31e662c2ee028e20dd0308c1bc6e398dd7a1786fdd0821722537b4124/leidenalg-0.12.0-cp38-abi3-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fcbd89655243cc739d28ba23fe894587d8d7235124a9248ba375819585e70090", size = 2739255, upload-time = "2026-05-24T10:17:38.376Z" },
+ { url = "https://files.pythonhosted.org/packages/e1/02/22597cf6d7ee093523ad5885a31499d698812db5744d98086383c3da08b5/leidenalg-0.12.0-cp38-abi3-musllinux_1_2_i686.whl", hash = "sha256:c6a6231d3db490ee3cc4004f214b0bb947dc3fabde75bd61d878bc3fb9685d06", size = 4072277, upload-time = "2026-05-24T10:17:39.881Z" },
+ { url = "https://files.pythonhosted.org/packages/05/e5/2f3322376fddd6b85790eaddcd314c9402bf81e984b35870acb4cfcbd0e1/leidenalg-0.12.0-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:63b49f5716c7cb155f3a3e25505bcc33f3c8c62105734fc152bc07518e63e9ab", size = 3799963, upload-time = "2026-05-24T10:17:41.035Z" },
+ { url = "https://files.pythonhosted.org/packages/d1/35/f4f07ddac6718a11c70d858fd4afcc849ca84f485b5d7092bcf09492ec06/leidenalg-0.12.0-cp38-abi3-win32.whl", hash = "sha256:283cd987623e2c5e7b11ed3f15b7805d597a934bf387a7e820d2f673001b635f", size = 1675833, upload-time = "2026-05-24T10:17:42.663Z" },
+ { url = "https://files.pythonhosted.org/packages/ea/77/0c8fb67729e83bddd43971cd4851f497a96810fca63f8029e3f008e82b1b/leidenalg-0.12.0-cp38-abi3-win_amd64.whl", hash = "sha256:f5d9529b44d5f6add0847d68fa6ced99f581357f6ca02b54c049eb3a2f45ff04", size = 1990568, upload-time = "2026-05-24T10:17:44.071Z" },
+]
+
+[[package]]
+name = "librt"
+version = "0.11.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/40/08/9e7f6b5d2b5bed6ad055cdd5925f192bb403a51280f86b56554d9d0699a2/librt-0.11.0.tar.gz", hash = "sha256:075dc3ef4458a278e0195cbf6ac9d38808d9b906c5a6c7f7f79c3888276a3fb1", size = 200139, upload-time = "2026-05-10T18:17:25.138Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/8b/d0/07c77e067f0838949b43bd89232c29d72efebb9d2801a9750184eb706b71/librt-0.11.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:b87504f1690a23b9a2cca841191a04f83895d4fc2dd04df91d82b1a04ca2ad46", size = 144147, upload-time = "2026-05-10T18:15:53.227Z" },
+ { url = "https://files.pythonhosted.org/packages/7a/24/8493538fa4f62f982686398a5b8f68008138a75086abdea19ade64bf4255/librt-0.11.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:40071fc5fe0ce8daa6de616702314a01e1250711682b0523d6ab8d4525910cb3", size = 143614, upload-time = "2026-05-10T18:15:54.657Z" },
+ { url = "https://files.pythonhosted.org/packages/ff/1e/f8bad050810d9171f34a1648ed910e56814c2ba61639f2bd53c6377ae24b/librt-0.11.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:137e79445c896a0ea7b265f52d23954e05b64222ee1af69e2cb34219067cbb67", size = 485538, upload-time = "2026-05-10T18:15:56.117Z" },
+ { url = "https://files.pythonhosted.org/packages/c0/fe/3594ebfbaf03084ba4b120c9ba5c3183fd938a48725e9bbe6ff0a5159ad8/librt-0.11.0-cp312-cp312-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:cca6644054e78746d8d4ef238681f9c34ff8b584fe6b988ecebb8db3b15e622a", size = 479623, upload-time = "2026-05-10T18:15:57.544Z" },
+ { url = "https://files.pythonhosted.org/packages/b0/da/5d1876984b3746c85dbd219dbfcb73c85f54ee263fd32e5b2a632ec14571/librt-0.11.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d5b0eea49f5562861ee8d757a32ef7d559c1d35be2aaaa1ec28941d74c9ffc8a", size = 513082, upload-time = "2026-05-10T18:15:58.805Z" },
+ { url = "https://files.pythonhosted.org/packages/19/6e/55bdf5d5ca00c3e18430690bf2c953d8d3ffd3c337418173d33dec985dc9/librt-0.11.0-cp312-cp312-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:0d1029d7e1ae1a7e647ed6fb5df8c4ce2dffefb7a9f5fd1376a4554d96dac09f", size = 508105, upload-time = "2026-05-10T18:16:00.2Z" },
+ { url = "https://files.pythonhosted.org/packages/07/10/f1f23a7c595ee90ece4d35c851e5d104b1311a887ed1b4ac4c35bbd13da8/librt-0.11.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:bc3ce6b33c5828d9e80592011a5c584cb2ce86edbc4088405f70da47dc1d1b3b", size = 522268, upload-time = "2026-05-10T18:16:01.708Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/02/5720f5697a7f54b78b3aefbe20df3a48cedcff1276618c4aa481177942ed/librt-0.11.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:936c5995f3514a42111f20099397d8177c79b4d7e70961e396c6f5a0a3566766", size = 527348, upload-time = "2026-05-10T18:16:03.496Z" },
+ { url = "https://files.pythonhosted.org/packages/50/db/b4a47c6f91db4ff76348a0b3dd0cc65e090a078b765a810a62ff9434c3d3/librt-0.11.0-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:9bc0ca6ad9381cbe8e4aa6e5726e4c80c78115a6e9723c599ed1d73e092bc49d", size = 516294, upload-time = "2026-05-10T18:16:05.173Z" },
+ { url = "https://files.pythonhosted.org/packages/9e/58/9384b2f4eb1ed1d273d40948a7c5c4b2360213b402ef3be4641c06299f9c/librt-0.11.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:070aa8c26c0a74774317a72df8851facc7f0f012a5b406557ac56992d92e1ec8", size = 553608, upload-time = "2026-05-10T18:16:06.839Z" },
+ { url = "https://files.pythonhosted.org/packages/21/7b/5aa8848a7c6a9278c79375146da1812e695754ceec5f005e6043461a7315/librt-0.11.0-cp312-cp312-win32.whl", hash = "sha256:6bf14feb84b05ae945277395451998c89c54d0def4070eb5c08de544930b245a", size = 101879, upload-time = "2026-05-10T18:16:08.103Z" },
+ { url = "https://files.pythonhosted.org/packages/37/33/8a745436944947575b584231750a41417de1a38cf6a2e9251d1065651c09/librt-0.11.0-cp312-cp312-win_amd64.whl", hash = "sha256:75672f0bc524ede266287d532d7923dbce94c7514ad07627bac3d0c6d92cc4d9", size = 119831, upload-time = "2026-05-10T18:16:09.174Z" },
+ { url = "https://files.pythonhosted.org/packages/59/67/a6739ac96e28b7855808bdb0370e250606104a859750d209e5a0716fe7ab/librt-0.11.0-cp312-cp312-win_arm64.whl", hash = "sha256:2f10cf143e4a9bb0f4f5af568a00df94a2d69ef41c2579584454bb0fe5cc642c", size = 103470, upload-time = "2026-05-10T18:16:10.369Z" },
+ { url = "https://files.pythonhosted.org/packages/82/61/e59168d4d0bf2bf90f4f0caf7a001bfc60254c3af4586013b04dc3ef517b/librt-0.11.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:78dc31f7fdfe9c9d0eb0e8f42d139db230e826415bbcabd9f0e9faaaee909894", size = 144119, upload-time = "2026-05-10T18:16:11.771Z" },
+ { url = "https://files.pythonhosted.org/packages/61/fd/caa1d60b12f7dd79ccea23054e06eeaebe266a5f52c40a6b651069200ce5/librt-0.11.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:fa475675db22290c3158e1d42326d0f5a65f04f44a0e68c3630a25b53560fb9c", size = 143565, upload-time = "2026-05-10T18:16:13.334Z" },
+ { url = "https://files.pythonhosted.org/packages/b8/a9/dc744f5c2b4978d48db970be29f22716d3413d28b14ad99740817315cf2c/librt-0.11.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:621db29691044bdeda22e789e482e1b0f3a985d90e3426c9c6d17606416205ea", size = 485395, upload-time = "2026-05-10T18:16:14.729Z" },
+ { url = "https://files.pythonhosted.org/packages/8f/21/7f8e97a1e4dae952a5a95948f6f8507a173bc1e669f54340bba6ca1ca31b/librt-0.11.0-cp313-cp313-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:a9010e2ed5b3a9e158c5fd966b3ab7e834bb3d3aacc8f66c91dd4b57a3799230", size = 479383, upload-time = "2026-05-10T18:16:16.321Z" },
+ { url = "https://files.pythonhosted.org/packages/a6/6d/d8ee9c114bebf2c50e29ec2aa940826fccb62a645c3e4c18760987d0e16d/librt-0.11.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7c39513d8b7477a2e1ed8c43fc21c524e8d5a0f8d4e8b7b074dbdbe7820a08e2", size = 513010, upload-time = "2026-05-10T18:16:17.647Z" },
+ { url = "https://files.pythonhosted.org/packages/f0/43/0b5708af2bd30a46400e72ba6bdaa8f066f15fb9a688527e34220e8d6c06/librt-0.11.0-cp313-cp313-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:7aef3cf1d5af86e770ab04bfd993dfc4ae8b8c17f66fb77dd4a7d50de7bbb1a3", size = 508433, upload-time = "2026-05-10T18:16:19.309Z" },
+ { url = "https://files.pythonhosted.org/packages/4a/50/356187247d09013490481033183b3532b58acf8028bcb34b2b56a375c9b2/librt-0.11.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:557183ddc36babe46b27dd60facbd5adb4492181a5be887587d57cda6e092f21", size = 522595, upload-time = "2026-05-10T18:16:20.642Z" },
+ { url = "https://files.pythonhosted.org/packages/40/e7/c6ac4240899c7f3248079d5a9900debe0dadb3fdeaf856684c987105ba47/librt-0.11.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:83d3e1f72bd42f6c5c0b7daec530c3f829bd02db42c70b8ddf0c2d90a2459930", size = 527255, upload-time = "2026-05-10T18:16:22.352Z" },
+ { url = "https://files.pythonhosted.org/packages/eb/b5/a81322dbeedeeaf9c1ee6f001734d28a09d8383ac9e6779bc24bbd0743c6/librt-0.11.0-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:4ce1f21fbe589bc1afd7872dece84fb0e1144f794a288e58a10d2c54a55c43be", size = 516847, upload-time = "2026-05-10T18:16:23.627Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/66/6e6323787d592b55204a42595ff1102da5115601b53a7e9ddebc889a6da5/librt-0.11.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:970b09f7044ea2b64c9da42fd3d335666518cfd1c6e8a182c95da73d0214b41e", size = 553920, upload-time = "2026-05-10T18:16:25.025Z" },
+ { url = "https://files.pythonhosted.org/packages/9c/21/623f8ca230857102066d9ca8c6c1734995908c4d0d1bee7bb2ef0021cb33/librt-0.11.0-cp313-cp313-win32.whl", hash = "sha256:78fddc31cd4d3caa897ad5d31f856b1faadc9474021ad6cb182b9018793e254e", size = 101898, upload-time = "2026-05-10T18:16:26.649Z" },
+ { url = "https://files.pythonhosted.org/packages/b3/1d/b4ebd44dd723f768469007515cb92251e0ae286c94c140f374801140fa74/librt-0.11.0-cp313-cp313-win_amd64.whl", hash = "sha256:8ca8aa88751a775870b764e93bad5135385f563cb8dcee399abf034ea4d3cb47", size = 119812, upload-time = "2026-05-10T18:16:27.859Z" },
+ { url = "https://files.pythonhosted.org/packages/3b/e4/b2f4ca7965ca373b491cdb4bc25cdb30c1649ca81a8782056a83850292a9/librt-0.11.0-cp313-cp313-win_arm64.whl", hash = "sha256:96f044bb325fd9cf1a723015638c219e9143f0dfbc0ca54c565df2b7fc748b44", size = 103448, upload-time = "2026-05-10T18:16:29.066Z" },
+ { url = "https://files.pythonhosted.org/packages/29/eb/dbce197da4e227779e56b5735f2decc3eb36e55a1cdbf1bd65d6639d76c1/librt-0.11.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:4a017a95e5837dc15a8c5661d60e05daa96b90908b1aa6b7acdf443cd25c8ebd", size = 143345, upload-time = "2026-05-10T18:16:30.674Z" },
+ { url = "https://files.pythonhosted.org/packages/76/a3/254bebd0c11c8ba684018efb8006ff22e466abce445215cca6c778e7d9de/librt-0.11.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:b1ecbd9819deccc39b7542bf4d2a740d8a620694d39989e58661d3763458f8d4", size = 143131, upload-time = "2026-05-10T18:16:32.037Z" },
+ { url = "https://files.pythonhosted.org/packages/f1/3f/f77d6122d21ac7bf6ae8a7dfced1bd2a7ac545d3273ebdcaf8042f6d619f/librt-0.11.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7da327dacd7be8f8ec36547373550744a3cc0e536d54665cd83f8bcd961200e8", size = 477024, upload-time = "2026-05-10T18:16:33.493Z" },
+ { url = "https://files.pythonhosted.org/packages/ac/0a/2c996dadebaa7d9bbbd43ef2d4f3e66b6da545f838a41694ef6172cebec8/librt-0.11.0-cp314-cp314-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:0dc56b1f8d06e60db362cc3fdae206681817f86ce4725d34511473487f12a34b", size = 474221, upload-time = "2026-05-10T18:16:34.864Z" },
+ { url = "https://files.pythonhosted.org/packages/0a/7e/f5d92af8486b8272c23b3e686b46ff72d89c8169585eb61eef01a2ac7147/librt-0.11.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:05fb8fb2ab90e21c8d12ea240d744ad514da9baf381ebfa70d91d20d21713175", size = 505174, upload-time = "2026-05-10T18:16:36.705Z" },
+ { url = "https://files.pythonhosted.org/packages/af/1a/cb0734fe86398eb33193ab753b7326255c74cac5eb09e76b9b16536e7adb/librt-0.11.0-cp314-cp314-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:cae74872be221df4374d10fec61f93ed1513b9546ea84f2c0bf73ab3e9bd0b03", size = 497216, upload-time = "2026-05-10T18:16:38.418Z" },
+ { url = "https://files.pythonhosted.org/packages/18/06/094820f91558b66e29943c0ec41c9914f460f48dd51fc503c3101e10842d/librt-0.11.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:32bcc918c0148eb7e3d57385125bac7e5f9e4359d05f07448b09f6f778c2f31c", size = 513921, upload-time = "2026-05-10T18:16:39.848Z" },
+ { url = "https://files.pythonhosted.org/packages/0b/c2/00de9018871a282f530cacb457d5ec0428f6ac7e6fedde9aff7468d9fb04/librt-0.11.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:f9743fc99135d5f78d2454435615f6dec0473ca507c26ce9d92b10b562a280d3", size = 520850, upload-time = "2026-05-10T18:16:41.471Z" },
+ { url = "https://files.pythonhosted.org/packages/51/9d/64631832348fd1834fb3a61b996434edddaaf25a31d03b0a76273159d2cf/librt-0.11.0-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:5ba067f4aadae8fda802d91d2124c90c42195ff32d9161d3549e6d05cfe26f96", size = 504237, upload-time = "2026-05-10T18:16:43.15Z" },
+ { url = "https://files.pythonhosted.org/packages/a5/ec/ae5525eb16edc827a044e7bb8777a455ff95d4bca9379e7e6bddd7383647/librt-0.11.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:de3bf945454d032f9e390b85c4072e0a0570bf825421c8be0e71209fa65e1abe", size = 546261, upload-time = "2026-05-10T18:16:44.408Z" },
+ { url = "https://files.pythonhosted.org/packages/5a/09/adce371f27ca039411da9659f7430fcc2ba6cd0c7b3e4467a0f091be7fa9/librt-0.11.0-cp314-cp314-win32.whl", hash = "sha256:d2277a05f6dcb9fd13db9566aac4fabd68c3ea1ea46ee5567d4eef8efa495a2f", size = 96965, upload-time = "2026-05-10T18:16:46.039Z" },
+ { url = "https://files.pythonhosted.org/packages/d6/ee/8ac720d98548f173c7ce2e632a7ca94673f74cacd5c8162a84af5b35958a/librt-0.11.0-cp314-cp314-win_amd64.whl", hash = "sha256:ab73e8db5e3f564d812c1f5c3a175930a5f9bc96ccb5e3b22a34d7858b401cf7", size = 115151, upload-time = "2026-05-10T18:16:47.133Z" },
+ { url = "https://files.pythonhosted.org/packages/94/20/c900cf14efeb09b6bef2b2dff20779f73464b97fd58d1c6bccc379588ae3/librt-0.11.0-cp314-cp314-win_arm64.whl", hash = "sha256:aea3caa317752e3a466fa8af45d91ee0ea8c7fdd96e42b0a8dd9b76a7931eba1", size = 98850, upload-time = "2026-05-10T18:16:48.597Z" },
+ { url = "https://files.pythonhosted.org/packages/0c/71/944bfe4b64e12abffcd3c15e1cce07f72f3d55655083786285f4dedeb532/librt-0.11.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:d1b36540d7aaf9b9101b3a6f376c8d8e9f7a9aec93ed05918f2c69d493ffef72", size = 151138, upload-time = "2026-05-10T18:16:49.839Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/10/99e64a5c86989357fda078c8143c533389585f6473b7439172dd8f3b3b2d/librt-0.11.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:efbb343ab2ce3540f4ecbe6315d677ed70f37cd9a72b1e58066c918ca83acbaa", size = 151976, upload-time = "2026-05-10T18:16:51.062Z" },
+ { url = "https://files.pythonhosted.org/packages/21/31/5072ad880946d83e5ea4147d6d018c78eefce85b77819b19bdd0ee229435/librt-0.11.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:aa0dd688aab3f7914d3e6e5e3554978e0383312fb8e771d84be008a35b9ee548", size = 557927, upload-time = "2026-05-10T18:16:52.632Z" },
+ { url = "https://files.pythonhosted.org/packages/5e/8d/70b5fb7cfbab60edbe7381614ab985da58e144fbf465c86d44c95f43cdca/librt-0.11.0-cp314-cp314t-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:f5fb36b8c6c63fdcbb1d526d94c0d1331610d43f4118cc1beb4efef4f3faacb2", size = 539698, upload-time = "2026-05-10T18:16:53.934Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/a3/ba3495a0b3edbd24a4cae0d1d3c64f39a9fc45d06e812101289b50c1a619/librt-0.11.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4a9a237d13addb93715b6fee74023d5ee3469b53fce527626c0e088aa585805f", size = 577162, upload-time = "2026-05-10T18:16:55.589Z" },
+ { url = "https://files.pythonhosted.org/packages/f7/db/36e25fb81f99937ff1b96612a1dc9fd66f039cb9cc3aee12c01fac31aab9/librt-0.11.0-cp314-cp314t-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:5ddd17bd87b2c56ddd60e546a7984a2e64c4e8eab92fb4cf3830a48ad5469d51", size = 566494, upload-time = "2026-05-10T18:16:56.975Z" },
+ { url = "https://files.pythonhosted.org/packages/33/0d/3f622b47f0b013eeb9cf4cc07ae9bfe378d832a4eec998b2b209fe84244d/librt-0.11.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:bd43992b4473d42f12ff9e68326079f0696d9d4e6000e8f39a0238d482ba6ee2", size = 596858, upload-time = "2026-05-10T18:16:58.374Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/02/71b90bc93039c46a2000651f6ad60122b114c8f54c4ad306e0e96f5b75ad/librt-0.11.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:f8e3e8056dd674e279741485e2e512d6e9a751c7455809d0114e6ebf8d781085", size = 590318, upload-time = "2026-05-10T18:16:59.676Z" },
+ { url = "https://files.pythonhosted.org/packages/04/04/418cb3f75621e2b761fb1ab0f017f4d70a1a72a6e7c74ee4f7e8d198c2f3/librt-0.11.0-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:c1f708d8ae9c56cf38a903c44297243d2ec83fd82b396b977e0144a3e76217e3", size = 575115, upload-time = "2026-05-10T18:17:01.007Z" },
+ { url = "https://files.pythonhosted.org/packages/cc/2c/5a2183ac58dd911f26b5d7e7d7d8f1d87fcecdddd99d6c12169a258ff62c/librt-0.11.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:0add982e0e7b9fc14cf4b33789d5f13f66581889b88c2f58099f6ce8f92617bd", size = 617918, upload-time = "2026-05-10T18:17:02.682Z" },
+ { url = "https://files.pythonhosted.org/packages/15/1f/dc6771a52592a4451be6effa200cbfc9cec61e4393d3033d81a9d307961d/librt-0.11.0-cp314-cp314t-win32.whl", hash = "sha256:2b481d846ac894c4e8403c5fd0e87c5d11d6499e404b474602508a224ff531c8", size = 103562, upload-time = "2026-05-10T18:17:03.99Z" },
+ { url = "https://files.pythonhosted.org/packages/62/4a/7d1415567027286a75ba1093ec4aca11f073e0f559c530cf3e0a757ad55c/librt-0.11.0-cp314-cp314t-win_amd64.whl", hash = "sha256:28edb433edde181112a908c78907af28f964eabc15f4dd16c9d66c834302677c", size = 124327, upload-time = "2026-05-10T18:17:05.465Z" },
+ { url = "https://files.pythonhosted.org/packages/ce/62/b40b382fa0c66fee1478073eb8db352a4a6beda4a1adccf1df911d8c289c/librt-0.11.0-cp314-cp314t-win_arm64.whl", hash = "sha256:dee008f20b542e3cd162ba338a7f9ec0f6d23d395f66fe8aeeec3c9d067ea253", size = 102572, upload-time = "2026-05-10T18:17:06.809Z" },
+]
+
+[[package]]
+name = "llvmlite"
+version = "0.47.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/01/88/a8952b6d5c21e74cbf158515b779666f692846502623e9e3c39d8e8ba25f/llvmlite-0.47.0.tar.gz", hash = "sha256:62031ce968ec74e95092184d4b0e857e444f8fdff0b8f9213707699570c33ccc", size = 193614, upload-time = "2026-03-31T18:29:53.497Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/fa/48/4b7fe0e34c169fa2f12532916133e0b219d2823b540733651b34fdac509a/llvmlite-0.47.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:306a265f408c259067257a732c8e159284334018b4083a9e35f67d19792b164f", size = 37232769, upload-time = "2026-03-31T18:28:43.735Z" },
+ { url = "https://files.pythonhosted.org/packages/e6/4b/e3f2cd17822cf772a4a51a0a8080b0032e6d37b2dbe8cfb724eac4e31c52/llvmlite-0.47.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:5853bf26160857c0c2573415ff4efe01c4c651e59e2c55c2a088740acfee51cd", size = 56275178, upload-time = "2026-03-31T18:28:48.342Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/55/a3b4a543185305a9bdf3d9759d53646ed96e55e7dfd43f53e7a421b8fbae/llvmlite-0.47.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:003bcf7fa579e14db59c1a1e113f93ab8a06b56a4be31c7f08264d1d4072d077", size = 55128632, upload-time = "2026-03-31T18:28:52.901Z" },
+ { url = "https://files.pythonhosted.org/packages/2f/f5/d281ae0f79378a5a91f308ea9fdb9f9cc068fddd09629edc0725a5a8fde1/llvmlite-0.47.0-cp312-cp312-win_amd64.whl", hash = "sha256:f3079f25bdc24cd9d27c4b2b5e68f5f60c4fdb7e8ad5ee2b9b006007558f9df7", size = 38138692, upload-time = "2026-03-31T18:28:57.147Z" },
+ { url = "https://files.pythonhosted.org/packages/77/6f/4615353e016799f80fa52ccb270a843c413b22361fadda2589b2922fb9b0/llvmlite-0.47.0-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:a3c6a735d4e1041808434f9d440faa3d78d9b4af2ee64d05a66f351883b6ceec", size = 37232771, upload-time = "2026-03-31T18:29:01.324Z" },
+ { url = "https://files.pythonhosted.org/packages/31/b8/69f5565f1a280d032525878a86511eebed0645818492feeb169dfb20ae8e/llvmlite-0.47.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:2699a74321189e812d476a43d6d7f652f51811e7b5aad9d9bba842a1c7927acb", size = 56275178, upload-time = "2026-03-31T18:29:05.748Z" },
+ { url = "https://files.pythonhosted.org/packages/d6/da/b32cafcb926fb0ce2aa25553bf32cb8764af31438f40e2481df08884c947/llvmlite-0.47.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6c6951e2b29930227963e53ee152441f0e14be92e9d4231852102d986c761e40", size = 55128632, upload-time = "2026-03-31T18:29:11.235Z" },
+ { url = "https://files.pythonhosted.org/packages/46/9f/4898b44e4042c60fafcb1162dfb7014f6f15b1ec19bf29cfea6bf26df90d/llvmlite-0.47.0-cp313-cp313-win_amd64.whl", hash = "sha256:c2e9adf8698d813a9a5efb2d4370caf344dbc1e145019851fee6a6f319ba760e", size = 38138695, upload-time = "2026-03-31T18:29:15.43Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/d4/33c8af00f0bf6f552d74f3a054f648af2c5bc6bece97972f3bfadce4f5ec/llvmlite-0.47.0-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:de966c626c35c9dff5ae7bf12db25637738d0df83fc370cf793bc94d43d92d14", size = 37232773, upload-time = "2026-03-31T18:29:19.453Z" },
+ { url = "https://files.pythonhosted.org/packages/64/1d/a760e993e0c0ba6db38d46b9f48f6c7dceb8ac838824997fb9e25f97bc04/llvmlite-0.47.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ddbccff2aeaff8670368340a158abefc032fe9b3ccf7d9c496639263d00151aa", size = 56275176, upload-time = "2026-03-31T18:29:24.149Z" },
+ { url = "https://files.pythonhosted.org/packages/84/3b/e679bc3b29127182a7f4aa2d2e9e5bea42adb93fb840484147d59c236299/llvmlite-0.47.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d4a7b778a2e144fc64468fb9bf509ac1226c9813a00b4d7afea5d988c4e22fca", size = 55128631, upload-time = "2026-03-31T18:29:29.536Z" },
+ { url = "https://files.pythonhosted.org/packages/be/f7/19e2a09c62809c9e63bbd14ce71fb92c6ff7b7b3045741bb00c781efc3c9/llvmlite-0.47.0-cp314-cp314-win_amd64.whl", hash = "sha256:694e3c2cdc472ed2bd8bd4555ca002eec4310961dd58ef791d508f57b5cc4c94", size = 39153826, upload-time = "2026-03-31T18:29:33.681Z" },
+ { url = "https://files.pythonhosted.org/packages/40/a1/581a8c707b5e80efdbbe1dd94527404d33fe50bceb71f39d5a7e11bd57b7/llvmlite-0.47.0-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:92ec8a169a20b473c1c54d4695e371bde36489fc1efa3688e11e99beba0abf9c", size = 37232772, upload-time = "2026-03-31T18:29:37.952Z" },
+ { url = "https://files.pythonhosted.org/packages/11/03/16090dd6f74ba2b8b922276047f15962fbeea0a75d5601607edb301ba945/llvmlite-0.47.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:fa1cbd800edd3b20bc141521f7fd45a6185a5b84109aa6855134e81397ffe72b", size = 56275178, upload-time = "2026-03-31T18:29:42.58Z" },
+ { url = "https://files.pythonhosted.org/packages/f5/cb/0abf1dd4c5286a95ffe0c1d8c67aec06b515894a0dd2ac97f5e27b82ab0b/llvmlite-0.47.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f6725179b89f03b17dabe236ff3422cb8291b4c1bf40af152826dfd34e350ae8", size = 55128632, upload-time = "2026-03-31T18:29:46.939Z" },
+ { url = "https://files.pythonhosted.org/packages/4f/79/d3bbab197e86e0ff4f9c07122895b66a3e0d024247fcff7f12c473cb36d9/llvmlite-0.47.0-cp314-cp314t-win_amd64.whl", hash = "sha256:6842cf6f707ec4be3d985a385ad03f72b2d724439e118fcbe99b2929964f0453", size = 39153839, upload-time = "2026-03-31T18:29:51.004Z" },
+]
+
+[[package]]
+name = "loguru"
+version = "0.7.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "colorama", marker = "sys_platform == 'win32'" },
+ { name = "win32-setctime", marker = "sys_platform == 'win32'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/3a/05/a1dae3dffd1116099471c643b8924f5aa6524411dc6c63fdae648c4f1aca/loguru-0.7.3.tar.gz", hash = "sha256:19480589e77d47b8d85b2c827ad95d49bf31b0dcde16593892eb51dd18706eb6", size = 63559, upload-time = "2024-12-06T11:20:56.608Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/0c/29/0348de65b8cc732daa3e33e67806420b2ae89bdce2b04af740289c5c6c8c/loguru-0.7.3-py3-none-any.whl", hash = "sha256:31a33c10c8e1e10422bfd431aeb5d351c7cf7fa671e3c4df004162264b28220c", size = 61595, upload-time = "2024-12-06T11:20:54.538Z" },
+]
+
+[[package]]
+name = "markdown-it-py"
+version = "4.2.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "mdurl" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" },
+]
+
+[[package]]
+name = "markupsafe"
+version = "3.0.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/5a/72/147da192e38635ada20e0a2e1a51cf8823d2119ce8883f7053879c2199b5/markupsafe-3.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e", size = 11615, upload-time = "2025-09-27T18:36:30.854Z" },
+ { url = "https://files.pythonhosted.org/packages/9a/81/7e4e08678a1f98521201c3079f77db69fb552acd56067661f8c2f534a718/markupsafe-3.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce", size = 12020, upload-time = "2025-09-27T18:36:31.971Z" },
+ { url = "https://files.pythonhosted.org/packages/1e/2c/799f4742efc39633a1b54a92eec4082e4f815314869865d876824c257c1e/markupsafe-3.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d", size = 24332, upload-time = "2025-09-27T18:36:32.813Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/2e/8d0c2ab90a8c1d9a24f0399058ab8519a3279d1bd4289511d74e909f060e/markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d", size = 22947, upload-time = "2025-09-27T18:36:33.86Z" },
+ { url = "https://files.pythonhosted.org/packages/2c/54/887f3092a85238093a0b2154bd629c89444f395618842e8b0c41783898ea/markupsafe-3.0.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a", size = 21962, upload-time = "2025-09-27T18:36:35.099Z" },
+ { url = "https://files.pythonhosted.org/packages/c9/2f/336b8c7b6f4a4d95e91119dc8521402461b74a485558d8f238a68312f11c/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b", size = 23760, upload-time = "2025-09-27T18:36:36.001Z" },
+ { url = "https://files.pythonhosted.org/packages/32/43/67935f2b7e4982ffb50a4d169b724d74b62a3964bc1a9a527f5ac4f1ee2b/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f", size = 21529, upload-time = "2025-09-27T18:36:36.906Z" },
+ { url = "https://files.pythonhosted.org/packages/89/e0/4486f11e51bbba8b0c041098859e869e304d1c261e59244baa3d295d47b7/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b", size = 23015, upload-time = "2025-09-27T18:36:37.868Z" },
+ { url = "https://files.pythonhosted.org/packages/2f/e1/78ee7a023dac597a5825441ebd17170785a9dab23de95d2c7508ade94e0e/markupsafe-3.0.3-cp312-cp312-win32.whl", hash = "sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d", size = 14540, upload-time = "2025-09-27T18:36:38.761Z" },
+ { url = "https://files.pythonhosted.org/packages/aa/5b/bec5aa9bbbb2c946ca2733ef9c4ca91c91b6a24580193e891b5f7dbe8e1e/markupsafe-3.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c", size = 15105, upload-time = "2025-09-27T18:36:39.701Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/f1/216fc1bbfd74011693a4fd837e7026152e89c4bcf3e77b6692fba9923123/markupsafe-3.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f", size = 13906, upload-time = "2025-09-27T18:36:40.689Z" },
+ { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" },
+ { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" },
+ { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" },
+ { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" },
+ { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" },
+ { url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543, upload-time = "2025-09-27T18:36:51.584Z" },
+ { url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113, upload-time = "2025-09-27T18:36:52.537Z" },
+ { url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911, upload-time = "2025-09-27T18:36:53.513Z" },
+ { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" },
+ { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" },
+ { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" },
+ { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" },
+ { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" },
+ { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" },
+ { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" },
+ { url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612, upload-time = "2025-09-27T18:37:02.639Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200, upload-time = "2025-09-27T18:37:03.582Z" },
+ { url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973, upload-time = "2025-09-27T18:37:04.929Z" },
+ { url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" },
+ { url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" },
+ { url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005, upload-time = "2025-09-27T18:37:10.58Z" },
+ { url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048, upload-time = "2025-09-27T18:37:11.547Z" },
+ { url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821, upload-time = "2025-09-27T18:37:12.48Z" },
+ { url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606, upload-time = "2025-09-27T18:37:13.485Z" },
+ { url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043, upload-time = "2025-09-27T18:37:14.408Z" },
+ { url = "https://files.pythonhosted.org/packages/46/11/f333a06fc16236d5238bfe74daccbca41459dcd8d1fa952e8fbd5dccfb70/markupsafe-3.0.3-cp314-cp314-win32.whl", hash = "sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9", size = 14747, upload-time = "2025-09-27T18:37:15.36Z" },
+ { url = "https://files.pythonhosted.org/packages/28/52/182836104b33b444e400b14f797212f720cbc9ed6ba34c800639d154e821/markupsafe-3.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581", size = 15341, upload-time = "2025-09-27T18:37:16.496Z" },
+ { url = "https://files.pythonhosted.org/packages/6f/18/acf23e91bd94fd7b3031558b1f013adfa21a8e407a3fdb32745538730382/markupsafe-3.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4", size = 14073, upload-time = "2025-09-27T18:37:17.476Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661, upload-time = "2025-09-27T18:37:18.453Z" },
+ { url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069, upload-time = "2025-09-27T18:37:19.332Z" },
+ { url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670, upload-time = "2025-09-27T18:37:20.245Z" },
+ { url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598, upload-time = "2025-09-27T18:37:21.177Z" },
+ { url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261, upload-time = "2025-09-27T18:37:22.167Z" },
+ { url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835, upload-time = "2025-09-27T18:37:23.296Z" },
+ { url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733, upload-time = "2025-09-27T18:37:24.237Z" },
+ { url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672, upload-time = "2025-09-27T18:37:25.271Z" },
+ { url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819, upload-time = "2025-09-27T18:37:26.285Z" },
+ { url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426, upload-time = "2025-09-27T18:37:27.316Z" },
+ { url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" },
+]
+
+[[package]]
+name = "matplotlib"
+version = "3.11.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "contourpy" },
+ { name = "cycler" },
+ { name = "fonttools" },
+ { name = "kiwisolver" },
+ { name = "numpy" },
+ { name = "packaging" },
+ { name = "pillow" },
+ { name = "pyparsing" },
+ { name = "python-dateutil" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/1f/24/080c99d223d158d3a8902769269ab6da5b50f7a0e6e072513907e02b7a6c/matplotlib-3.11.0.tar.gz", hash = "sha256:68c0c7be01b30dcca3638934f7f591df73401235cbdbf0d1ab1c71e7db7f8b57", size = 33251176, upload-time = "2026-06-12T02:29:15.508Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/da/17/f5276b496c61477a6c4fc5e7401f4bfe1c2e5ef7c6cd67896f2ade3809cb/matplotlib-3.11.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:06b5872e9cf11adc8f589ded3ce11bc3e1061ad498259664fabc1f6615beb918", size = 9449976, upload-time = "2026-06-12T02:27:50.989Z" },
+ { url = "https://files.pythonhosted.org/packages/82/34/bdd77418adb2178a1d59f044bd67bfebb115896e91b840b8a197eb3f4f4e/matplotlib-3.11.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:0515d495124be3124340e59f164d901ed4484e2246a5b74cfa483cac3b80bd97", size = 9279307, upload-time = "2026-06-12T02:27:53.247Z" },
+ { url = "https://files.pythonhosted.org/packages/94/95/7f522393c88313336b20d70fc849555757b2e5febc22b83b3a3f0fd4bce9/matplotlib-3.11.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:be5f93a1d21981bfb802ded0d77a0caa92d4342a47d45754fac77e314a506344", size = 10031353, upload-time = "2026-06-12T02:27:55.215Z" },
+ { url = "https://files.pythonhosted.org/packages/87/ce/8f25a0e3186aefd61913e7467d1b999465bcd0d0c03ac695c1b26ca559b7/matplotlib-3.11.0-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:41635d7909d19e52e924a521dde6d8f670b0f53ab1d0e8c331fa831554f681d1", size = 10839232, upload-time = "2026-06-12T02:27:57.746Z" },
+ { url = "https://files.pythonhosted.org/packages/85/c2/db15da2bbdf9e3ca66df7db8e2c33a1dfed67be24a24d2c878efaaff01d6/matplotlib-3.11.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:94f5000f67ca9faa300863ea17f8bce9175cb67b88bec4bc7780502d53dd7c9e", size = 10923899, upload-time = "2026-06-12T02:28:00.223Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/2f/a58a4443a4d052a4ea77557478336aefc26c7981f6408d37adba763aa758/matplotlib-3.11.0-cp312-cp312-win_amd64.whl", hash = "sha256:ac6f1ef39f3d0f9e2463303013094992cdbe0f85f43bc54155bc472b2042768e", size = 9329528, upload-time = "2026-06-12T02:28:02.27Z" },
+ { url = "https://files.pythonhosted.org/packages/61/0f/4b669589d47733b97ab9df4b58d6fc1e68acb5ea42a928dc7cbdd6bf5871/matplotlib-3.11.0-cp312-cp312-win_arm64.whl", hash = "sha256:9dd11fb612ce7bc60b1de5b4fc87ff959d22317b5de42aabf392f66f97af22eb", size = 9003413, upload-time = "2026-06-12T02:28:04.49Z" },
+ { url = "https://files.pythonhosted.org/packages/55/41/aa47f156b061d14c98b906f76c428507397708ec63ff94f410ae1752b426/matplotlib-3.11.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:6ce3b839b34ae1f430b4616893a2945a2999debaa7e94e7e29a2a8bbf286f7b5", size = 9450532, upload-time = "2026-06-12T02:28:06.769Z" },
+ { url = "https://files.pythonhosted.org/packages/8c/4f/5a9eb0375e81413953febf8af7b012a6b6357f53438a15c4f5ad86c6bbb5/matplotlib-3.11.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:373db8f91214e8ccaf35ac833cc1dd59dd961e148bbd55dd027141591dde1313", size = 9279760, upload-time = "2026-06-12T02:28:09.152Z" },
+ { url = "https://files.pythonhosted.org/packages/a4/c0/1117d53077e3ac3152503a84e9cf7a5c239576805ee71276e80c2aaa7471/matplotlib-3.11.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:be152b7570324dc8d01574cc9474dd2d803237acf528bcbb5b211fa347461a09", size = 10031623, upload-time = "2026-06-12T02:28:11.26Z" },
+ { url = "https://files.pythonhosted.org/packages/92/7e/e937138daffad65b71bf831a377809dcbc830fb4f31a31e067dc1faa2575/matplotlib-3.11.0-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:126f256df600652d7e4b394cf3164ff75210a00038f287c95a012a6f58d0e83f", size = 10839372, upload-time = "2026-06-12T02:28:14.102Z" },
+ { url = "https://files.pythonhosted.org/packages/1d/c2/438ecc197ffb8023b6b9922915542f2172f5fd45b76703b0b4fc47322243/matplotlib-3.11.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:03acfeddf87b0dddb11b081ef7740ad445a3ca8bcb6b8e3011b08f2cf802b75c", size = 10924099, upload-time = "2026-06-12T02:28:16.383Z" },
+ { url = "https://files.pythonhosted.org/packages/40/2e/395883da416f378b3ed2c9f3e843ac477eae1ce731b671b79adaa6f0bacd/matplotlib-3.11.0-cp313-cp313-win_amd64.whl", hash = "sha256:ab3722f04f3ff34c23b5012c5873d2894174e06c3822fcdac3610965a5ac7d06", size = 9329727, upload-time = "2026-06-12T02:28:18.581Z" },
+ { url = "https://files.pythonhosted.org/packages/61/82/2c388956abf8bf392dfb5b8917c502f1082df6a941b781ab8c8e5ba2474b/matplotlib-3.11.0-cp313-cp313-win_arm64.whl", hash = "sha256:c945824670fb8915b4ac879e5e61f3c58e0913022f70a0de4c082b17372f8771", size = 9003506, upload-time = "2026-06-12T02:28:20.474Z" },
+ { url = "https://files.pythonhosted.org/packages/c8/c1/34454baa44da7975ada82e9aea37105ec47059514dc967d3be14426ba8dc/matplotlib-3.11.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:3489c3dc487669b4a980bc3068f87856de7a1564248d3f6c629efb2a58b03f24", size = 9499838, upload-time = "2026-06-12T02:28:22.713Z" },
+ { url = "https://files.pythonhosted.org/packages/b1/c3/98fe79a398cf232219f090163a7fa7e6766e9f2e0ad26df54d6f8934d8ee/matplotlib-3.11.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:6a98f5476ce784a50ce09998f4ae1e6a9f25043cef8a480c98949902eda74620", size = 9332298, upload-time = "2026-06-12T02:28:24.796Z" },
+ { url = "https://files.pythonhosted.org/packages/95/e4/b4b7c33151e74e5c802f3cde1ba807ebfc38401e329b44e215a5888dd76d/matplotlib-3.11.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:565af866fd63e4bd3f987d580afe27c44c2552a3b3305f4ecbb85133601ea6f3", size = 10045491, upload-time = "2026-06-12T02:28:27.141Z" },
+ { url = "https://files.pythonhosted.org/packages/71/28/394548efd68354110c1a1be11fe6b6e559e06d1a23da35908a0e316c55a9/matplotlib-3.11.0-cp313-cp313t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e6b3e64dea5062c570f04358e2711859f3531b459f29516274fbad889079e4f3", size = 10857059, upload-time = "2026-06-12T02:28:29.222Z" },
+ { url = "https://files.pythonhosted.org/packages/c8/44/e7922e6e2a4d63bdfbc9dc4a53e3850ab438d46cf42e6779bb15ec92c948/matplotlib-3.11.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:942b37c5db1899610bd1543ce8e13e4ecff9a4633e7f63bb6aa9205d2644ebd1", size = 10939576, upload-time = "2026-06-12T02:28:31.66Z" },
+ { url = "https://files.pythonhosted.org/packages/3d/be/b1ca96003a441d619b727fee21d671fdff7a5ce2f1bb797b2521aa2f679a/matplotlib-3.11.0-cp313-cp313t-win_amd64.whl", hash = "sha256:c08e649a6313e1291e713623b97a38e5bb4aa580b2a100a94a3309bc6b9c8eb3", size = 9379519, upload-time = "2026-06-12T02:28:33.888Z" },
+ { url = "https://files.pythonhosted.org/packages/e3/72/4bf3b91821c34596dd6a7bdac5836d94f744144c8208939ef49d8ec43f7e/matplotlib-3.11.0-cp313-cp313t-win_arm64.whl", hash = "sha256:2746cd2c113742ff6ce37a864c5ac5fd7aa644568f445e66166e457ac78e40e0", size = 9055456, upload-time = "2026-06-12T02:28:35.878Z" },
+ { url = "https://files.pythonhosted.org/packages/57/52/a94102ac99eb78e2fe9b826674f9ef9ee23327110ea6ab4776c1b4eb6209/matplotlib-3.11.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:3338e3e3de128cf50d0d2fb92a122815daf9c755bd882a474343c05f8fd7ec79", size = 9452137, upload-time = "2026-06-12T02:28:37.93Z" },
+ { url = "https://files.pythonhosted.org/packages/7c/03/b8cdb625a21f710dfa11bbca1f48fb4057d2c0286975f8b415bf80942c99/matplotlib-3.11.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:25c2e5455efd8d99f41fb79871a31feb7d301569642e332ec58d72cfe9282bc3", size = 9281514, upload-time = "2026-06-12T02:28:40.028Z" },
+ { url = "https://files.pythonhosted.org/packages/b7/2d/4e1240ea82ee197dfb3851e71f71c87eeeb975f1753b56a0588e4e80739a/matplotlib-3.11.0-cp314-cp314-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d9695457a467ff86d23f35037a43deb6f1134dd6d3e2ac8ce1e2087cff09ffb9", size = 10843005, upload-time = "2026-06-12T02:28:42.39Z" },
+ { url = "https://files.pythonhosted.org/packages/29/dc/6377ecfaa5fef79430f74a1a16638b4e2aa30d4692bae2c19f9d76fe3b01/matplotlib-3.11.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:19c16c61dea63b3582918503e6b294193961261d9daa806d4ae2151f1ad05430", size = 11127459, upload-time = "2026-06-12T02:28:44.483Z" },
+ { url = "https://files.pythonhosted.org/packages/6f/41/795c405aa7560443a3b01309424cde4a1113b85c90b8a63417444a749617/matplotlib-3.11.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2d72ea8b7924f3cb955e61518d21e43b3df1e6c8a793b480a0c1214f185d30ba", size = 10925160, upload-time = "2026-06-12T02:28:46.564Z" },
+ { url = "https://files.pythonhosted.org/packages/1a/f7/3a9e6389a7cfaeff76c56e40c2dabcb13110e21e82f837228c834ebe748c/matplotlib-3.11.0-cp314-cp314-win_amd64.whl", hash = "sha256:1c02da0a629dfa9debf52725ea06866b74c1fb70a895bae05e4493d34074f9f2", size = 9485186, upload-time = "2026-06-12T02:28:49.344Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/c0/396478ee7cf2091d182db8b4a8695f6a37f1ddb978989cf9dbb84cd5c123/matplotlib-3.11.0-cp314-cp314-win_arm64.whl", hash = "sha256:aa55d73b3117d4b07f959cd9eb6f69b375d8df3414139c479388e551aa5d999d", size = 9160349, upload-time = "2026-06-12T02:28:51.382Z" },
+ { url = "https://files.pythonhosted.org/packages/c5/6f/1c3bd51bb2b34eaacdcf3c3d859dbb357f952fc8020c617dc118ad7c9e38/matplotlib-3.11.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:a9d8c6e7cd2f0ddf11d8d92e520dd1d9d2abb0cf6ac8831e338666c81e905847", size = 9500921, upload-time = "2026-06-12T02:28:53.443Z" },
+ { url = "https://files.pythonhosted.org/packages/e0/0d/4d861d0121840cb1a3fd4a10deb211efd6fccd481ed23e553f31f4f4da4a/matplotlib-3.11.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:be050fcf32f729eda99f7f75a80bf67612ce16ab9ac1c23a387dcaede95cb70e", size = 9332190, upload-time = "2026-06-12T02:28:55.623Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/cb/22f6bc35711a0b5639a784e74e653e77c86210bd4304449dd399a482f74e/matplotlib-3.11.0-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:dfabef0230d0697aa0d717385194dd41162e00207a68bf4abf94c2bf4c27dca0", size = 10854181, upload-time = "2026-06-12T02:28:57.856Z" },
+ { url = "https://files.pythonhosted.org/packages/3f/7e/9a9eaca731a2939589da520f0ebe8fd8753d0f51fca98c7d20af6dbe261a/matplotlib-3.11.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1644db30e759199443493ac5e5caec24fdb775a8f6123021f85ba47c4133c3cb", size = 11137715, upload-time = "2026-06-12T02:29:00.555Z" },
+ { url = "https://files.pythonhosted.org/packages/ef/f9/9b030b6088354acb0296871bb624b25befc1c42509d3c6cd17420c83a5b8/matplotlib-3.11.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:15b0d160079cb10699a0e98b5989c70677b2df7cacdc62af67c30f2facec46d9", size = 10939427, upload-time = "2026-06-12T02:29:02.527Z" },
+ { url = "https://files.pythonhosted.org/packages/59/94/6b273eaee4ee250863567d100865da61a5c1527fa67f527b7ed22e0dd29c/matplotlib-3.11.0-cp314-cp314t-win_amd64.whl", hash = "sha256:446307e6b04b57b1f1239e228a1ec2af0d589a1008cebc3dfa3f5441d095cfb6", size = 9535809, upload-time = "2026-06-12T02:29:04.994Z" },
+ { url = "https://files.pythonhosted.org/packages/60/95/1d36bddf2b7e2692c1540e78a6e5bc88bc1496b137e3e35a611f91b65ac3/matplotlib-3.11.0-cp314-cp314t-win_arm64.whl", hash = "sha256:652fb5696271d4c50f196d22a5ff4f8e4444c74f847423570d7dc0aa2bbd0159", size = 9209226, upload-time = "2026-06-12T02:29:07.033Z" },
+]
+
+[[package]]
+name = "matplotlib-inline"
+version = "0.2.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "traitlets" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/bd/c0/9f7c9a46090390368a4d7bcb76bb87a4a36c421e4c0792cdb53486ffac7a/matplotlib_inline-0.2.2.tar.gz", hash = "sha256:72f3fe8fce36b70d4a5b612f899090cd0401deddc4ea90e1572b9f4bfb058c79", size = 8150, upload-time = "2026-05-08T17:33:33.49Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/41/09/5b161152e2d90f7b87f781c2e1267494aef9c32498df793f73ad0a0a494a/matplotlib_inline-0.2.2-py3-none-any.whl", hash = "sha256:3c821cf1c209f59fb2d2d64abbf5b23b67bcb2210d663f9918dd851c6da1fcf6", size = 9534, upload-time = "2026-05-08T17:33:32.055Z" },
+]
+
+[[package]]
+name = "mdit-py-plugins"
+version = "0.6.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "markdown-it-py" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/59/fc/f8d0863f8862f25602c0404d75568e89fb6b4109804645e5cdfb1be5cf56/mdit_py_plugins-0.6.1.tar.gz", hash = "sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0", size = 56114, upload-time = "2026-05-13T09:03:38.91Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/a5/69/6da5581c6a7fede7dc261bf4e67d6adca4196f176b43288b55b3db395b6e/mdit_py_plugins-0.6.1-py3-none-any.whl", hash = "sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d", size = 66663, upload-time = "2026-05-13T09:03:37.76Z" },
+]
+
+[[package]]
+name = "mdurl"
+version = "0.1.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" },
+]
+
+[[package]]
+name = "modal"
+version = "1.5.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "aiohttp" },
+ { name = "cbor2" },
+ { name = "certifi" },
+ { name = "click" },
+ { name = "grpclib" },
+ { name = "protobuf" },
+ { name = "rich" },
+ { name = "synchronicity" },
+ { name = "toml" },
+ { name = "types-certifi" },
+ { name = "types-toml" },
+ { name = "typing-extensions" },
+ { name = "watchfiles" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/b8/e6/44f609084472d16a3acf7748c184e23d3009944eed5860e855aaafc61065/modal-1.5.2.tar.gz", hash = "sha256:ac0d2983ecccacb80c54385906404d75f37a3751f1374c750974cd7853c0008a", size = 818040, upload-time = "2026-07-10T18:12:55.164Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/e3/fe/7e284d889eb3ea2c307e48828eccdd9a984beb2e096bce5577ef796592e9/modal-1.5.2-py3-none-any.whl", hash = "sha256:7508a44f8742c6f26be4d28da71f46092ef3bcc07d0f27cbd0ea8c9d6d9fc873", size = 932258, upload-time = "2026-07-10T18:12:52.219Z" },
+]
+
+[[package]]
+name = "multidict"
+version = "6.7.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/1a/c2/c2d94cbe6ac1753f3fc980da97b3d930efe1da3af3c9f5125354436c073d/multidict-6.7.1.tar.gz", hash = "sha256:ec6652a1bee61c53a3e5776b6049172c53b6aaba34f18c9ad04f82712bac623d", size = 102010, upload-time = "2026-01-26T02:46:45.979Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/8d/9c/f20e0e2cf80e4b2e4b1c365bf5fe104ee633c751a724246262db8f1a0b13/multidict-6.7.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:a90f75c956e32891a4eda3639ce6dd86e87105271f43d43442a3aedf3cddf172", size = 76893, upload-time = "2026-01-26T02:43:52.754Z" },
+ { url = "https://files.pythonhosted.org/packages/fe/cf/18ef143a81610136d3da8193da9d80bfe1cb548a1e2d1c775f26b23d024a/multidict-6.7.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3fccb473e87eaa1382689053e4a4618e7ba7b9b9b8d6adf2027ee474597128cd", size = 45456, upload-time = "2026-01-26T02:43:53.893Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/65/1caac9d4cd32e8433908683446eebc953e82d22b03d10d41a5f0fefe991b/multidict-6.7.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:b0fa96985700739c4c7853a43c0b3e169360d6855780021bfc6d0f1ce7c123e7", size = 43872, upload-time = "2026-01-26T02:43:55.041Z" },
+ { url = "https://files.pythonhosted.org/packages/cf/3b/d6bd75dc4f3ff7c73766e04e705b00ed6dbbaccf670d9e05a12b006f5a21/multidict-6.7.1-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:cb2a55f408c3043e42b40cc8eecd575afa27b7e0b956dfb190de0f8499a57a53", size = 251018, upload-time = "2026-01-26T02:43:56.198Z" },
+ { url = "https://files.pythonhosted.org/packages/fd/80/c959c5933adedb9ac15152e4067c702a808ea183a8b64cf8f31af8ad3155/multidict-6.7.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:eb0ce7b2a32d09892b3dd6cc44877a0d02a33241fafca5f25c8b6b62374f8b75", size = 258883, upload-time = "2026-01-26T02:43:57.499Z" },
+ { url = "https://files.pythonhosted.org/packages/86/85/7ed40adafea3d4f1c8b916e3b5cc3a8e07dfcdcb9cd72800f4ed3ca1b387/multidict-6.7.1-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:c3a32d23520ee37bf327d1e1a656fec76a2edd5c038bf43eddfa0572ec49c60b", size = 242413, upload-time = "2026-01-26T02:43:58.755Z" },
+ { url = "https://files.pythonhosted.org/packages/d2/57/b8565ff533e48595503c785f8361ff9a4fde4d67de25c207cd0ba3befd03/multidict-6.7.1-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9c90fed18bffc0189ba814749fdcc102b536e83a9f738a9003e569acd540a733", size = 268404, upload-time = "2026-01-26T02:44:00.216Z" },
+ { url = "https://files.pythonhosted.org/packages/e0/50/9810c5c29350f7258180dfdcb2e52783a0632862eb334c4896ac717cebcb/multidict-6.7.1-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:da62917e6076f512daccfbbde27f46fed1c98fee202f0559adec8ee0de67f71a", size = 269456, upload-time = "2026-01-26T02:44:02.202Z" },
+ { url = "https://files.pythonhosted.org/packages/f3/8d/5e5be3ced1d12966fefb5c4ea3b2a5b480afcea36406559442c6e31d4a48/multidict-6.7.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bfde23ef6ed9db7eaee6c37dcec08524cb43903c60b285b172b6c094711b3961", size = 256322, upload-time = "2026-01-26T02:44:03.56Z" },
+ { url = "https://files.pythonhosted.org/packages/31/6e/d8a26d81ac166a5592782d208dd90dfdc0a7a218adaa52b45a672b46c122/multidict-6.7.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:3758692429e4e32f1ba0df23219cd0b4fc0a52f476726fff9337d1a57676a582", size = 253955, upload-time = "2026-01-26T02:44:04.845Z" },
+ { url = "https://files.pythonhosted.org/packages/59/4c/7c672c8aad41534ba619bcd4ade7a0dc87ed6b8b5c06149b85d3dd03f0cd/multidict-6.7.1-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:398c1478926eca669f2fd6a5856b6de9c0acf23a2cb59a14c0ba5844fa38077e", size = 251254, upload-time = "2026-01-26T02:44:06.133Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/bd/84c24de512cbafbdbc39439f74e967f19570ce7924e3007174a29c348916/multidict-6.7.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:c102791b1c4f3ab36ce4101154549105a53dc828f016356b3e3bcae2e3a039d3", size = 252059, upload-time = "2026-01-26T02:44:07.518Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/ba/f5449385510825b73d01c2d4087bf6d2fccc20a2d42ac34df93191d3dd03/multidict-6.7.1-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:a088b62bd733e2ad12c50dad01b7d0166c30287c166e137433d3b410add807a6", size = 263588, upload-time = "2026-01-26T02:44:09.382Z" },
+ { url = "https://files.pythonhosted.org/packages/d7/11/afc7c677f68f75c84a69fe37184f0f82fce13ce4b92f49f3db280b7e92b3/multidict-6.7.1-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:3d51ff4785d58d3f6c91bdbffcb5e1f7ddfda557727043aa20d20ec4f65e324a", size = 259642, upload-time = "2026-01-26T02:44:10.73Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/17/ebb9644da78c4ab36403739e0e6e0e30ebb135b9caf3440825001a0bddcb/multidict-6.7.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:fc5907494fccf3e7d3f94f95c91d6336b092b5fc83811720fae5e2765890dfba", size = 251377, upload-time = "2026-01-26T02:44:12.042Z" },
+ { url = "https://files.pythonhosted.org/packages/ca/a4/840f5b97339e27846c46307f2530a2805d9d537d8b8bd416af031cad7fa0/multidict-6.7.1-cp312-cp312-win32.whl", hash = "sha256:28ca5ce2fd9716631133d0e9a9b9a745ad7f60bac2bccafb56aa380fc0b6c511", size = 41887, upload-time = "2026-01-26T02:44:14.245Z" },
+ { url = "https://files.pythonhosted.org/packages/80/31/0b2517913687895f5904325c2069d6a3b78f66cc641a86a2baf75a05dcbb/multidict-6.7.1-cp312-cp312-win_amd64.whl", hash = "sha256:fcee94dfbd638784645b066074b338bc9cc155d4b4bffa4adce1615c5a426c19", size = 46053, upload-time = "2026-01-26T02:44:15.371Z" },
+ { url = "https://files.pythonhosted.org/packages/0c/5b/aba28e4ee4006ae4c7df8d327d31025d760ffa992ea23812a601d226e682/multidict-6.7.1-cp312-cp312-win_arm64.whl", hash = "sha256:ba0a9fb644d0c1a2194cf7ffb043bd852cea63a57f66fbd33959f7dae18517bf", size = 43307, upload-time = "2026-01-26T02:44:16.852Z" },
+ { url = "https://files.pythonhosted.org/packages/f2/22/929c141d6c0dba87d3e1d38fbdf1ba8baba86b7776469f2bc2d3227a1e67/multidict-6.7.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:2b41f5fed0ed563624f1c17630cb9941cf2309d4df00e494b551b5f3e3d67a23", size = 76174, upload-time = "2026-01-26T02:44:18.509Z" },
+ { url = "https://files.pythonhosted.org/packages/c7/75/bc704ae15fee974f8fccd871305e254754167dce5f9e42d88a2def741a1d/multidict-6.7.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:84e61e3af5463c19b67ced91f6c634effb89ef8bfc5ca0267f954451ed4bb6a2", size = 45116, upload-time = "2026-01-26T02:44:19.745Z" },
+ { url = "https://files.pythonhosted.org/packages/79/76/55cd7186f498ed080a18440c9013011eb548f77ae1b297206d030eb1180a/multidict-6.7.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:935434b9853c7c112eee7ac891bc4cb86455aa631269ae35442cb316790c1445", size = 43524, upload-time = "2026-01-26T02:44:21.571Z" },
+ { url = "https://files.pythonhosted.org/packages/e9/3c/414842ef8d5a1628d68edee29ba0e5bcf235dbfb3ccd3ea303a7fe8c72ff/multidict-6.7.1-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:432feb25a1cb67fe82a9680b4d65fb542e4635cb3166cd9c01560651ad60f177", size = 249368, upload-time = "2026-01-26T02:44:22.803Z" },
+ { url = "https://files.pythonhosted.org/packages/f6/32/befed7f74c458b4a525e60519fe8d87eef72bb1e99924fa2b0f9d97a221e/multidict-6.7.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e82d14e3c948952a1a85503817e038cba5905a3352de76b9a465075d072fba23", size = 256952, upload-time = "2026-01-26T02:44:24.306Z" },
+ { url = "https://files.pythonhosted.org/packages/03/d6/c878a44ba877f366630c860fdf74bfb203c33778f12b6ac274936853c451/multidict-6.7.1-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:4cfb48c6ea66c83bcaaf7e4dfa7ec1b6bbcf751b7db85a328902796dfde4c060", size = 240317, upload-time = "2026-01-26T02:44:25.772Z" },
+ { url = "https://files.pythonhosted.org/packages/68/49/57421b4d7ad2e9e60e25922b08ceb37e077b90444bde6ead629095327a6f/multidict-6.7.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:1d540e51b7e8e170174555edecddbd5538105443754539193e3e1061864d444d", size = 267132, upload-time = "2026-01-26T02:44:27.648Z" },
+ { url = "https://files.pythonhosted.org/packages/b7/fe/ec0edd52ddbcea2a2e89e174f0206444a61440b40f39704e64dc807a70bd/multidict-6.7.1-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:273d23f4b40f3dce4d6c8a821c741a86dec62cded82e1175ba3d99be128147ed", size = 268140, upload-time = "2026-01-26T02:44:29.588Z" },
+ { url = "https://files.pythonhosted.org/packages/b0/73/6e1b01cbeb458807aa0831742232dbdd1fa92bfa33f52a3f176b4ff3dc11/multidict-6.7.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9d624335fd4fa1c08a53f8b4be7676ebde19cd092b3895c421045ca87895b429", size = 254277, upload-time = "2026-01-26T02:44:30.902Z" },
+ { url = "https://files.pythonhosted.org/packages/6a/b2/5fb8c124d7561a4974c342bc8c778b471ebbeb3cc17df696f034a7e9afe7/multidict-6.7.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:12fad252f8b267cc75b66e8fc51b3079604e8d43a75428ffe193cd9e2195dfd6", size = 252291, upload-time = "2026-01-26T02:44:32.31Z" },
+ { url = "https://files.pythonhosted.org/packages/5a/96/51d4e4e06bcce92577fcd488e22600bd38e4fd59c20cb49434d054903bd2/multidict-6.7.1-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:03ede2a6ffbe8ef936b92cb4529f27f42be7f56afcdab5ab739cd5f27fb1cbf9", size = 250156, upload-time = "2026-01-26T02:44:33.734Z" },
+ { url = "https://files.pythonhosted.org/packages/db/6b/420e173eec5fba721a50e2a9f89eda89d9c98fded1124f8d5c675f7a0c0f/multidict-6.7.1-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:90efbcf47dbe33dcf643a1e400d67d59abeac5db07dc3f27d6bdeae497a2198c", size = 249742, upload-time = "2026-01-26T02:44:35.222Z" },
+ { url = "https://files.pythonhosted.org/packages/44/a3/ec5b5bd98f306bc2aa297b8c6f11a46714a56b1e6ef5ebda50a4f5d7c5fb/multidict-6.7.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:5c4b9bfc148f5a91be9244d6264c53035c8a0dcd2f51f1c3c6e30e30ebaa1c84", size = 262221, upload-time = "2026-01-26T02:44:36.604Z" },
+ { url = "https://files.pythonhosted.org/packages/cd/f7/e8c0d0da0cd1e28d10e624604e1a36bcc3353aaebdfdc3a43c72bc683a12/multidict-6.7.1-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:401c5a650f3add2472d1d288c26deebc540f99e2fb83e9525007a74cd2116f1d", size = 258664, upload-time = "2026-01-26T02:44:38.008Z" },
+ { url = "https://files.pythonhosted.org/packages/52/da/151a44e8016dd33feed44f730bd856a66257c1ee7aed4f44b649fb7edeb3/multidict-6.7.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:97891f3b1b3ffbded884e2916cacf3c6fc87b66bb0dde46f7357404750559f33", size = 249490, upload-time = "2026-01-26T02:44:39.386Z" },
+ { url = "https://files.pythonhosted.org/packages/87/af/a3b86bf9630b732897f6fc3f4c4714b90aa4361983ccbdcd6c0339b21b0c/multidict-6.7.1-cp313-cp313-win32.whl", hash = "sha256:e1c5988359516095535c4301af38d8a8838534158f649c05dd1050222321bcb3", size = 41695, upload-time = "2026-01-26T02:44:41.318Z" },
+ { url = "https://files.pythonhosted.org/packages/b2/35/e994121b0e90e46134673422dd564623f93304614f5d11886b1b3e06f503/multidict-6.7.1-cp313-cp313-win_amd64.whl", hash = "sha256:960c83bf01a95b12b08fd54324a4eb1d5b52c88932b5cba5d6e712bb3ed12eb5", size = 45884, upload-time = "2026-01-26T02:44:42.488Z" },
+ { url = "https://files.pythonhosted.org/packages/ca/61/42d3e5dbf661242a69c97ea363f2d7b46c567da8eadef8890022be6e2ab0/multidict-6.7.1-cp313-cp313-win_arm64.whl", hash = "sha256:563fe25c678aaba333d5399408f5ec3c383ca5b663e7f774dd179a520b8144df", size = 43122, upload-time = "2026-01-26T02:44:43.664Z" },
+ { url = "https://files.pythonhosted.org/packages/6d/b3/e6b21c6c4f314bb956016b0b3ef2162590a529b84cb831c257519e7fde44/multidict-6.7.1-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:c76c4bec1538375dad9d452d246ca5368ad6e1c9039dadcf007ae59c70619ea1", size = 83175, upload-time = "2026-01-26T02:44:44.894Z" },
+ { url = "https://files.pythonhosted.org/packages/fb/76/23ecd2abfe0957b234f6c960f4ade497f55f2c16aeb684d4ecdbf1c95791/multidict-6.7.1-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:57b46b24b5d5ebcc978da4ec23a819a9402b4228b8a90d9c656422b4bdd8a963", size = 48460, upload-time = "2026-01-26T02:44:46.106Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/57/a0ed92b23f3a042c36bc4227b72b97eca803f5f1801c1ab77c8a212d455e/multidict-6.7.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:e954b24433c768ce78ab7929e84ccf3422e46deb45a4dc9f93438f8217fa2d34", size = 46930, upload-time = "2026-01-26T02:44:47.278Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/66/02ec7ace29162e447f6382c495dc95826bf931d3818799bbef11e8f7df1a/multidict-6.7.1-cp313-cp313t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:3bd231490fa7217cc832528e1cd8752a96f0125ddd2b5749390f7c3ec8721b65", size = 242582, upload-time = "2026-01-26T02:44:48.604Z" },
+ { url = "https://files.pythonhosted.org/packages/58/18/64f5a795e7677670e872673aca234162514696274597b3708b2c0d276cce/multidict-6.7.1-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:253282d70d67885a15c8a7716f3a73edf2d635793ceda8173b9ecc21f2fb8292", size = 250031, upload-time = "2026-01-26T02:44:50.544Z" },
+ { url = "https://files.pythonhosted.org/packages/c8/ed/e192291dbbe51a8290c5686f482084d31bcd9d09af24f63358c3d42fd284/multidict-6.7.1-cp313-cp313t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:0b4c48648d7649c9335cf1927a8b87fa692de3dcb15faa676c6a6f1f1aabda43", size = 228596, upload-time = "2026-01-26T02:44:51.951Z" },
+ { url = "https://files.pythonhosted.org/packages/1e/7e/3562a15a60cf747397e7f2180b0a11dc0c38d9175a650e75fa1b4d325e15/multidict-6.7.1-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:98bc624954ec4d2c7cb074b8eefc2b5d0ce7d482e410df446414355d158fe4ca", size = 257492, upload-time = "2026-01-26T02:44:53.902Z" },
+ { url = "https://files.pythonhosted.org/packages/24/02/7d0f9eae92b5249bb50ac1595b295f10e263dd0078ebb55115c31e0eaccd/multidict-6.7.1-cp313-cp313t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:1b99af4d9eec0b49927b4402bcbb58dea89d3e0db8806a4086117019939ad3dd", size = 255899, upload-time = "2026-01-26T02:44:55.316Z" },
+ { url = "https://files.pythonhosted.org/packages/00/e3/9b60ed9e23e64c73a5cde95269ef1330678e9c6e34dd4eb6b431b85b5a10/multidict-6.7.1-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6aac4f16b472d5b7dc6f66a0d49dd57b0e0902090be16594dc9ebfd3d17c47e7", size = 247970, upload-time = "2026-01-26T02:44:56.783Z" },
+ { url = "https://files.pythonhosted.org/packages/3e/06/538e58a63ed5cfb0bd4517e346b91da32fde409d839720f664e9a4ae4f9d/multidict-6.7.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:21f830fe223215dffd51f538e78c172ed7c7f60c9b96a2bf05c4848ad49921c3", size = 245060, upload-time = "2026-01-26T02:44:58.195Z" },
+ { url = "https://files.pythonhosted.org/packages/b2/2f/d743a3045a97c895d401e9bd29aaa09b94f5cbdf1bd561609e5a6c431c70/multidict-6.7.1-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:f5dd81c45b05518b9aa4da4aa74e1c93d715efa234fd3e8a179df611cc85e5f4", size = 235888, upload-time = "2026-01-26T02:44:59.57Z" },
+ { url = "https://files.pythonhosted.org/packages/38/83/5a325cac191ab28b63c52f14f1131f3b0a55ba3b9aa65a6d0bf2a9b921a0/multidict-6.7.1-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:eb304767bca2bb92fb9c5bd33cedc95baee5bb5f6c88e63706533a1c06ad08c8", size = 243554, upload-time = "2026-01-26T02:45:01.054Z" },
+ { url = "https://files.pythonhosted.org/packages/20/1f/9d2327086bd15da2725ef6aae624208e2ef828ed99892b17f60c344e57ed/multidict-6.7.1-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:c9035dde0f916702850ef66460bc4239d89d08df4d02023a5926e7446724212c", size = 252341, upload-time = "2026-01-26T02:45:02.484Z" },
+ { url = "https://files.pythonhosted.org/packages/e8/2c/2a1aa0280cf579d0f6eed8ee5211c4f1730bd7e06c636ba2ee6aafda302e/multidict-6.7.1-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:af959b9beeb66c822380f222f0e0a1889331597e81f1ded7f374f3ecb0fd6c52", size = 246391, upload-time = "2026-01-26T02:45:03.862Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/03/7ca022ffc36c5a3f6e03b179a5ceb829be9da5783e6fe395f347c0794680/multidict-6.7.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:41f2952231456154ee479651491e94118229844dd7226541788be783be2b5108", size = 243422, upload-time = "2026-01-26T02:45:05.296Z" },
+ { url = "https://files.pythonhosted.org/packages/dc/1d/b31650eab6c5778aceed46ba735bd97f7c7d2f54b319fa916c0f96e7805b/multidict-6.7.1-cp313-cp313t-win32.whl", hash = "sha256:df9f19c28adcb40b6aae30bbaa1478c389efd50c28d541d76760199fc1037c32", size = 47770, upload-time = "2026-01-26T02:45:06.754Z" },
+ { url = "https://files.pythonhosted.org/packages/ac/5b/2d2d1d522e51285bd61b1e20df8f47ae1a9d80839db0b24ea783b3832832/multidict-6.7.1-cp313-cp313t-win_amd64.whl", hash = "sha256:d54ecf9f301853f2c5e802da559604b3e95bb7a3b01a9c295c6ee591b9882de8", size = 53109, upload-time = "2026-01-26T02:45:08.044Z" },
+ { url = "https://files.pythonhosted.org/packages/3d/a3/cc409ba012c83ca024a308516703cf339bdc4b696195644a7215a5164a24/multidict-6.7.1-cp313-cp313t-win_arm64.whl", hash = "sha256:5a37ca18e360377cfda1d62f5f382ff41f2b8c4ccb329ed974cc2e1643440118", size = 45573, upload-time = "2026-01-26T02:45:09.349Z" },
+ { url = "https://files.pythonhosted.org/packages/91/cc/db74228a8be41884a567e88a62fd589a913708fcf180d029898c17a9a371/multidict-6.7.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:8f333ec9c5eb1b7105e3b84b53141e66ca05a19a605368c55450b6ba208cb9ee", size = 75190, upload-time = "2026-01-26T02:45:10.651Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/22/492f2246bb5b534abd44804292e81eeaf835388901f0c574bac4eeec73c5/multidict-6.7.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:a407f13c188f804c759fc6a9f88286a565c242a76b27626594c133b82883b5c2", size = 44486, upload-time = "2026-01-26T02:45:11.938Z" },
+ { url = "https://files.pythonhosted.org/packages/f1/4f/733c48f270565d78b4544f2baddc2fb2a245e5a8640254b12c36ac7ac68e/multidict-6.7.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:0e161ddf326db5577c3a4cc2d8648f81456e8a20d40415541587a71620d7a7d1", size = 43219, upload-time = "2026-01-26T02:45:14.346Z" },
+ { url = "https://files.pythonhosted.org/packages/24/bb/2c0c2287963f4259c85e8bcbba9182ced8d7fca65c780c38e99e61629d11/multidict-6.7.1-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:1e3a8bb24342a8201d178c3b4984c26ba81a577c80d4d525727427460a50c22d", size = 245132, upload-time = "2026-01-26T02:45:15.712Z" },
+ { url = "https://files.pythonhosted.org/packages/a7/f9/44d4b3064c65079d2467888794dea218d1601898ac50222ab8a9a8094460/multidict-6.7.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:97231140a50f5d447d3164f994b86a0bed7cd016e2682f8650d6a9158e14fd31", size = 252420, upload-time = "2026-01-26T02:45:17.293Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/13/78f7275e73fa17b24c9a51b0bd9d73ba64bb32d0ed51b02a746eb876abe7/multidict-6.7.1-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:6b10359683bd8806a200fd2909e7c8ca3a7b24ec1d8132e483d58e791d881048", size = 233510, upload-time = "2026-01-26T02:45:19.356Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/25/8167187f62ae3cbd52da7893f58cb036b47ea3fb67138787c76800158982/multidict-6.7.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:283ddac99f7ac25a4acadbf004cb5ae34480bbeb063520f70ce397b281859362", size = 264094, upload-time = "2026-01-26T02:45:20.834Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/e7/69a3a83b7b030cf283fb06ce074a05a02322359783424d7edf0f15fe5022/multidict-6.7.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:538cec1e18c067d0e6103aa9a74f9e832904c957adc260e61cd9d8cf0c3b3d37", size = 260786, upload-time = "2026-01-26T02:45:22.818Z" },
+ { url = "https://files.pythonhosted.org/packages/fe/3b/8ec5074bcfc450fe84273713b4b0a0dd47c0249358f5d82eb8104ffe2520/multidict-6.7.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7eee46ccb30ff48a1e35bb818cc90846c6be2b68240e42a78599166722cea709", size = 248483, upload-time = "2026-01-26T02:45:24.368Z" },
+ { url = "https://files.pythonhosted.org/packages/48/5a/d5a99e3acbca0e29c5d9cba8f92ceb15dce78bab963b308ae692981e3a5d/multidict-6.7.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fa263a02f4f2dd2d11a7b1bb4362aa7cb1049f84a9235d31adf63f30143469a0", size = 248403, upload-time = "2026-01-26T02:45:25.982Z" },
+ { url = "https://files.pythonhosted.org/packages/35/48/e58cd31f6c7d5102f2a4bf89f96b9cf7e00b6c6f3d04ecc44417c00a5a3c/multidict-6.7.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:2e1425e2f99ec5bd36c15a01b690a1a2456209c5deed58f95469ffb46039ccbb", size = 240315, upload-time = "2026-01-26T02:45:27.487Z" },
+ { url = "https://files.pythonhosted.org/packages/94/33/1cd210229559cb90b6786c30676bb0c58249ff42f942765f88793b41fdce/multidict-6.7.1-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:497394b3239fc6f0e13a78a3e1b61296e72bf1c5f94b4c4eb80b265c37a131cd", size = 245528, upload-time = "2026-01-26T02:45:28.991Z" },
+ { url = "https://files.pythonhosted.org/packages/64/f2/6e1107d226278c876c783056b7db43d800bb64c6131cec9c8dfb6903698e/multidict-6.7.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:233b398c29d3f1b9676b4b6f75c518a06fcb2ea0b925119fb2c1bc35c05e1601", size = 258784, upload-time = "2026-01-26T02:45:30.503Z" },
+ { url = "https://files.pythonhosted.org/packages/4d/c1/11f664f14d525e4a1b5327a82d4de61a1db604ab34c6603bb3c2cc63ad34/multidict-6.7.1-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:93b1818e4a6e0930454f0f2af7dfce69307ca03cdcfb3739bf4d91241967b6c1", size = 251980, upload-time = "2026-01-26T02:45:32.603Z" },
+ { url = "https://files.pythonhosted.org/packages/e1/9f/75a9ac888121d0c5bbd4ecf4eead45668b1766f6baabfb3b7f66a410e231/multidict-6.7.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:f33dc2a3abe9249ea5d8360f969ec7f4142e7ac45ee7014d8f8d5acddf178b7b", size = 243602, upload-time = "2026-01-26T02:45:34.043Z" },
+ { url = "https://files.pythonhosted.org/packages/9a/e7/50bf7b004cc8525d80dbbbedfdc7aed3e4c323810890be4413e589074032/multidict-6.7.1-cp314-cp314-win32.whl", hash = "sha256:3ab8b9d8b75aef9df299595d5388b14530839f6422333357af1339443cff777d", size = 40930, upload-time = "2026-01-26T02:45:36.278Z" },
+ { url = "https://files.pythonhosted.org/packages/e0/bf/52f25716bbe93745595800f36fb17b73711f14da59ed0bb2eba141bc9f0f/multidict-6.7.1-cp314-cp314-win_amd64.whl", hash = "sha256:5e01429a929600e7dab7b166062d9bb54a5eed752384c7384c968c2afab8f50f", size = 45074, upload-time = "2026-01-26T02:45:37.546Z" },
+ { url = "https://files.pythonhosted.org/packages/97/ab/22803b03285fa3a525f48217963da3a65ae40f6a1b6f6cf2768879e208f9/multidict-6.7.1-cp314-cp314-win_arm64.whl", hash = "sha256:4885cb0e817aef5d00a2e8451d4665c1808378dc27c2705f1bf4ef8505c0d2e5", size = 42471, upload-time = "2026-01-26T02:45:38.889Z" },
+ { url = "https://files.pythonhosted.org/packages/e0/6d/f9293baa6146ba9507e360ea0292b6422b016907c393e2f63fc40ab7b7b5/multidict-6.7.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:0458c978acd8e6ea53c81eefaddbbee9c6c5e591f41b3f5e8e194780fe026581", size = 82401, upload-time = "2026-01-26T02:45:40.254Z" },
+ { url = "https://files.pythonhosted.org/packages/7a/68/53b5494738d83558d87c3c71a486504d8373421c3e0dbb6d0db48ad42ee0/multidict-6.7.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:c0abd12629b0af3cf590982c0b413b1e7395cd4ec026f30986818ab95bfaa94a", size = 48143, upload-time = "2026-01-26T02:45:41.635Z" },
+ { url = "https://files.pythonhosted.org/packages/37/e8/5284c53310dcdc99ce5d66563f6e5773531a9b9fe9ec7a615e9bc306b05f/multidict-6.7.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:14525a5f61d7d0c94b368a42cff4c9a4e7ba2d52e2672a7b23d84dc86fb02b0c", size = 46507, upload-time = "2026-01-26T02:45:42.99Z" },
+ { url = "https://files.pythonhosted.org/packages/e4/fc/6800d0e5b3875568b4083ecf5f310dcf91d86d52573160834fb4bfcf5e4f/multidict-6.7.1-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:17307b22c217b4cf05033dabefe68255a534d637c6c9b0cc8382718f87be4262", size = 239358, upload-time = "2026-01-26T02:45:44.376Z" },
+ { url = "https://files.pythonhosted.org/packages/41/75/4ad0973179361cdf3a113905e6e088173198349131be2b390f9fa4da5fc6/multidict-6.7.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7a7e590ff876a3eaf1c02a4dfe0724b6e69a9e9de6d8f556816f29c496046e59", size = 246884, upload-time = "2026-01-26T02:45:47.167Z" },
+ { url = "https://files.pythonhosted.org/packages/c3/9c/095bb28b5da139bd41fb9a5d5caff412584f377914bd8787c2aa98717130/multidict-6.7.1-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:5fa6a95dfee63893d80a34758cd0e0c118a30b8dcb46372bf75106c591b77889", size = 225878, upload-time = "2026-01-26T02:45:48.698Z" },
+ { url = "https://files.pythonhosted.org/packages/07/d0/c0a72000243756e8f5a277b6b514fa005f2c73d481b7d9e47cd4568aa2e4/multidict-6.7.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a0543217a6a017692aa6ae5cc39adb75e587af0f3a82288b1492eb73dd6cc2a4", size = 253542, upload-time = "2026-01-26T02:45:50.164Z" },
+ { url = "https://files.pythonhosted.org/packages/c0/6b/f69da15289e384ecf2a68837ec8b5ad8c33e973aa18b266f50fe55f24b8c/multidict-6.7.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f99fe611c312b3c1c0ace793f92464d8cd263cc3b26b5721950d977b006b6c4d", size = 252403, upload-time = "2026-01-26T02:45:51.779Z" },
+ { url = "https://files.pythonhosted.org/packages/a2/76/b9669547afa5a1a25cd93eaca91c0da1c095b06b6d2d8ec25b713588d3a1/multidict-6.7.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9004d8386d133b7e6135679424c91b0b854d2d164af6ea3f289f8f2761064609", size = 244889, upload-time = "2026-01-26T02:45:53.27Z" },
+ { url = "https://files.pythonhosted.org/packages/7e/a9/a50d2669e506dad33cfc45b5d574a205587b7b8a5f426f2fbb2e90882588/multidict-6.7.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e628ef0e6859ffd8273c69412a2465c4be4a9517d07261b33334b5ec6f3c7489", size = 241982, upload-time = "2026-01-26T02:45:54.919Z" },
+ { url = "https://files.pythonhosted.org/packages/c5/bb/1609558ad8b456b4827d3c5a5b775c93b87878fd3117ed3db3423dfbce1b/multidict-6.7.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:841189848ba629c3552035a6a7f5bf3b02eb304e9fea7492ca220a8eda6b0e5c", size = 232415, upload-time = "2026-01-26T02:45:56.981Z" },
+ { url = "https://files.pythonhosted.org/packages/d8/59/6f61039d2aa9261871e03ab9dc058a550d240f25859b05b67fd70f80d4b3/multidict-6.7.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:ce1bbd7d780bb5a0da032e095c951f7014d6b0a205f8318308140f1a6aba159e", size = 240337, upload-time = "2026-01-26T02:45:58.698Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/29/fdc6a43c203890dc2ae9249971ecd0c41deaedfe00d25cb6564b2edd99eb/multidict-6.7.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:b26684587228afed0d50cf804cc71062cc9c1cdf55051c4c6345d372947b268c", size = 248788, upload-time = "2026-01-26T02:46:00.862Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/14/a153a06101323e4cf086ecee3faadba52ff71633d471f9685c42e3736163/multidict-6.7.1-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:9f9af11306994335398293f9958071019e3ab95e9a707dc1383a35613f6abcb9", size = 242842, upload-time = "2026-01-26T02:46:02.824Z" },
+ { url = "https://files.pythonhosted.org/packages/41/5f/604ae839e64a4a6efc80db94465348d3b328ee955e37acb24badbcd24d83/multidict-6.7.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:b4938326284c4f1224178a560987b6cf8b4d38458b113d9b8c1db1a836e640a2", size = 240237, upload-time = "2026-01-26T02:46:05.898Z" },
+ { url = "https://files.pythonhosted.org/packages/5f/60/c3a5187bf66f6fb546ff4ab8fb5a077cbdd832d7b1908d4365c7f74a1917/multidict-6.7.1-cp314-cp314t-win32.whl", hash = "sha256:98655c737850c064a65e006a3df7c997cd3b220be4ec8fe26215760b9697d4d7", size = 48008, upload-time = "2026-01-26T02:46:07.468Z" },
+ { url = "https://files.pythonhosted.org/packages/0c/f7/addf1087b860ac60e6f382240f64fb99f8bfb532bb06f7c542b83c29ca61/multidict-6.7.1-cp314-cp314t-win_amd64.whl", hash = "sha256:497bde6223c212ba11d462853cfa4f0ae6ef97465033e7dc9940cdb3ab5b48e5", size = 53542, upload-time = "2026-01-26T02:46:08.809Z" },
+ { url = "https://files.pythonhosted.org/packages/4c/81/4629d0aa32302ef7b2ec65c75a728cc5ff4fa410c50096174c1632e70b3e/multidict-6.7.1-cp314-cp314t-win_arm64.whl", hash = "sha256:2bbd113e0d4af5db41d5ebfe9ccaff89de2120578164f86a5d17d5a576d1e5b2", size = 44719, upload-time = "2026-01-26T02:46:11.146Z" },
+ { url = "https://files.pythonhosted.org/packages/81/08/7036c080d7117f28a4af526d794aab6a84463126db031b007717c1a6676e/multidict-6.7.1-py3-none-any.whl", hash = "sha256:55d97cc6dae627efa6a6e548885712d4864b81110ac76fa4e534c03819fa4a56", size = 12319, upload-time = "2026-01-26T02:46:44.004Z" },
+]
+
+[[package]]
+name = "multipledispatch"
+version = "1.0.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/fe/3e/a62c3b824c7dec33c4a1578bcc842e6c30300051033a4e5975ed86cc2536/multipledispatch-1.0.0.tar.gz", hash = "sha256:5c839915465c68206c3e9c473357908216c28383b425361e5d144594bf85a7e0", size = 12385, upload-time = "2023-06-27T16:45:11.074Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/51/c0/00c9809d8b9346eb238a6bbd5f83e846a4ce4503da94a4c08cb7284c325b/multipledispatch-1.0.0-py3-none-any.whl", hash = "sha256:0c53cd8b077546da4e48869f49b13164bebafd0c2a5afceb6bb6a316e7fb46e4", size = 12818, upload-time = "2023-06-27T16:45:09.418Z" },
+]
+
+[[package]]
+name = "mypy"
+version = "2.1.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "ast-serialize" },
+ { name = "librt", marker = "platform_python_implementation != 'PyPy'" },
+ { name = "mypy-extensions" },
+ { name = "pathspec" },
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/82/15/cca9d88503549ed6fedeaa1d448cdddd542ee8a490232d732e278036fbf2/mypy-2.1.0.tar.gz", hash = "sha256:81e76ad12c2d804512e9b13240d1588316531bfba07558286078bfbce9613633", size = 3898359, upload-time = "2026-05-11T18:37:36.237Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/95/b1/55861beb5c339b44f9a2ba92df9e2cb1eeb4ae1eee674cdf7772c797778b/mypy-2.1.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:244358bf1c0da7722230bce60683d52e8e9fd030554926f15b747a84efb5b3af", size = 14874381, upload-time = "2026-05-11T18:37:31.784Z" },
+ { url = "https://files.pythonhosted.org/packages/0b/b3/b7f770114b7d0ac92d0f76e8d93c2780844a70488a90e91821927850da86/mypy-2.1.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:4ec7c57657493c7a75534df2751c8ae2cda383c16ecc55d2106c54476b1b16f6", size = 13665501, upload-time = "2026-05-11T18:34:23.063Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/f3/8ae2037967e2126689a0c11d99e2b707134a565191e92c60ca2572aec60a/mypy-2.1.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d8161b6ff4392410023224f0969d17db93e1e154bc3e4ba62598e720723ae211", size = 14045750, upload-time = "2026-05-11T18:31:48.151Z" },
+ { url = "https://files.pythonhosted.org/packages/a0/32/615eb5911859e43d054941b0d0a7d06cfa2870eba86529cf385b052b111c/mypy-2.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bf03e12003084a67395184d3eb8cbd6a489dc3655b5664b28c210a9e2403ab0b", size = 15061630, upload-time = "2026-05-11T18:37:06.898Z" },
+ { url = "https://files.pythonhosted.org/packages/d4/03/4eafbfff8bfab1b87082741eae6e6a624028c984e6708b73bce2a8570c9d/mypy-2.1.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:20509760fd791c51579d573153407d226385ec1f8bcce55d730b354f3336bc22", size = 15288831, upload-time = "2026-05-11T18:31:18.07Z" },
+ { url = "https://files.pythonhosted.org/packages/99/ee/919661478e5891a3c96e549c036e467e64563ab85995b10c53c8358e16a3/mypy-2.1.0-cp312-cp312-win_amd64.whl", hash = "sha256:6753d0c1fdd6b1a23b9e4f283ce80b2153b724adcb2653b20b85a8a28ac6436b", size = 11135228, upload-time = "2026-05-11T18:34:31.23Z" },
+ { url = "https://files.pythonhosted.org/packages/24/0a/6a12b9782ca0831a553192f351679f4548abc9d19a7cc93bb7feb02084c7/mypy-2.1.0-cp312-cp312-win_arm64.whl", hash = "sha256:98ebb6589bb3b6d0c6f0c459d53ca55b8091fbc13d277c4041c885392e8195e8", size = 10040684, upload-time = "2026-05-11T18:36:48.199Z" },
+ { url = "https://files.pythonhosted.org/packages/6e/dd/c7191469c777f07689c032a8f7326e393ea34c92d6d76eb7ce5ba57ea66d/mypy-2.1.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:35aac3bb114e03888f535d5eb51b8bafbb3266586b599da1940f9b1be3ec5bd5", size = 14852174, upload-time = "2026-05-11T18:31:38.929Z" },
+ { url = "https://files.pythonhosted.org/packages/55/8c/aed55408879043d72bb9135f4d0d19a02b886dd569631e113e3d2706cb8d/mypy-2.1.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:8de55a8c861f2a49331f807be98d90caeceeef520bde13d43a160207f8af613e", size = 13651542, upload-time = "2026-05-11T18:36:04.636Z" },
+ { url = "https://files.pythonhosted.org/packages/3a/8e/f371a824b1f1fa8ea6e3dbb8703d232977d572be2329554a3bc4d960302f/mypy-2.1.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5fdf2941a07434af755837d9880f7d7d25f1dacb1af9dcd4b9b66f2220a3024e", size = 14033929, upload-time = "2026-05-11T18:35:55.742Z" },
+ { url = "https://files.pythonhosted.org/packages/94/21/f54be870d6dd53a82c674407e0f8eed7174b05ec78d42e5abd7b42e84fd5/mypy-2.1.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e195b817c13f02352a9c124301f9f30f078405444679b6753c1b96b6eed37285", size = 15039200, upload-time = "2026-05-11T18:33:10.281Z" },
+ { url = "https://files.pythonhosted.org/packages/17/99/bf21748626a40ce59fd29a39386ab46afec88b7bd2f0fa6c3a97c995523f/mypy-2.1.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:5431d42af987ebd92ba2f71d45c85ed41d8e6ca9f5fd209a69f68f707d2469e5", size = 15272690, upload-time = "2026-05-11T18:32:07.205Z" },
+ { url = "https://files.pythonhosted.org/packages/d6/d7/9e90d2cf47100bea550ed2bc7b0d4de3a62181d84d5e37da0003e8462637/mypy-2.1.0-cp313-cp313-win_amd64.whl", hash = "sha256:767fe8c66dc3e01e19e1737d4c38ebefead16125e1b8e58ad421903b376f5c65", size = 11147435, upload-time = "2026-05-11T18:33:56.477Z" },
+ { url = "https://files.pythonhosted.org/packages/ec/46/e5c449e858798e35ffc90946282a27c62a77be743fe17480e4977374eb91/mypy-2.1.0-cp313-cp313-win_arm64.whl", hash = "sha256:ecfe70d43775ab99562ab128ce49854a362044c9f894961f68f898c23cb7429d", size = 10035052, upload-time = "2026-05-11T18:32:30.049Z" },
+ { url = "https://files.pythonhosted.org/packages/b0/ca/b279a672e874aedd5498ae25f722dacc8aa86bbffb939b3f97cbb1cf6686/mypy-2.1.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:7354c5a7f69d9345c3d6e69921d57088eea3ddeeb6b20d34c1b3855b02c36ec2", size = 14848422, upload-time = "2026-05-11T18:35:45.984Z" },
+ { url = "https://files.pythonhosted.org/packages/27/e6/3efe56c631d959b9b4454e208b0ac4b7f4f58b404c89f8bec7b49efdfc21/mypy-2.1.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:49890d4f76ac9e06ec117f9e09f3174da70a620a0c300953d8595c926e80947f", size = 13677374, upload-time = "2026-05-11T18:36:57.188Z" },
+ { url = "https://files.pythonhosted.org/packages/84/7f/8107ea87a44fd1f1b59882442f033c9c3488c127201b1d1d15f1cbd6022e/mypy-2.1.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:761be68e023ef5d94678772396a8af1220030f80837a3afd8d0aef3b419666f4", size = 14055743, upload-time = "2026-05-11T18:35:18.361Z" },
+ { url = "https://files.pythonhosted.org/packages/51/4d/b6d34db183133b83761b9199a82d31557cdbb70a380d8c3b3438e11882a3/mypy-2.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c90345fc182dc363b891350457ec69c35140858538f38b4540845afcc32b1aef", size = 15020937, upload-time = "2026-05-11T18:34:59.618Z" },
+ { url = "https://files.pythonhosted.org/packages/ff/d7/f08360c691d758acb02f45022c34d98b92892f4ea756644e1000d4b9f3d8/mypy-2.1.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:b84802e7b5a6daf1f5e15bc9fcd7ddae77be13981ffab037f1c67bb84d67d135", size = 15253371, upload-time = "2026-05-11T18:36:41.081Z" },
+ { url = "https://files.pythonhosted.org/packages/67/1b/09460a13719530a19bce27bd3bc8449e83569dd2ba7faf51c9c3c30c0b61/mypy-2.1.0-cp314-cp314-win_amd64.whl", hash = "sha256:022c771234936ceac541ebaf836fe9e2abeb3f5e09aff21588fe543ff006fe21", size = 11326429, upload-time = "2026-05-11T18:34:13.526Z" },
+ { url = "https://files.pythonhosted.org/packages/40/62/75dbf0f82f7b6680340efc614af29dd0b3c17b8a4f1cd09b8bd2fd6bc814/mypy-2.1.0-cp314-cp314-win_arm64.whl", hash = "sha256:498207db725cec88829a6a5c2fc771205fd043719ef98bc49aba8fb9fc4e6d57", size = 10218799, upload-time = "2026-05-11T18:32:23.491Z" },
+ { url = "https://files.pythonhosted.org/packages/b2/66/caca04ed7d972fb6eb6dd1ccd6df1de5c38fae8c5b3dc1c4e8e0d85ee6b9/mypy-2.1.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:7d5e5cad0efeba72b93cd17490cc0d69c5ac9ca132994fe3fb0314808aeeb83e", size = 15923458, upload-time = "2026-05-11T18:35:28.64Z" },
+ { url = "https://files.pythonhosted.org/packages/ed/52/2d90cbe49d014b13ed7ff337930c30bad35893fe38a1e4641e756bb62191/mypy-2.1.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ff715050c127d724fd260a2e666e7747fdd83511c0c47d449d98238970aef780", size = 14757697, upload-time = "2026-05-11T18:36:14.208Z" },
+ { url = "https://files.pythonhosted.org/packages/ac/37/d98f4a14e081b238992d0ed96b6d39c7cc0148c9699eb71eaa68629665ea/mypy-2.1.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:82208da9e09414d520e912d3e462d454854bed0810b71540bb016dcbca7308fd", size = 15405638, upload-time = "2026-05-11T18:33:48.249Z" },
+ { url = "https://files.pythonhosted.org/packages/a3/c2/15c46613b24a84fad2aea1248bf9619b99c2767ae9071fe224c179a0b7d4/mypy-2.1.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e79ebc1b904b84f0310dff7469655a9c36c7a68bddb37bdd42b67a332df61d08", size = 16215852, upload-time = "2026-05-11T18:32:50.296Z" },
+ { url = "https://files.pythonhosted.org/packages/5c/90/9c16a57f482c76d25f6379762b56bbf65c711d8158cf271fb2802cfb0640/mypy-2.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:e583edc957cfb0deb142079162ae826f58449b116c1d442f2d91c69d9fced081", size = 16452695, upload-time = "2026-05-11T18:33:38.182Z" },
+ { url = "https://files.pythonhosted.org/packages/0f/4c/215a4eeb63cacc5f17f516691ea7285d11e249802b942476bff15922a314/mypy-2.1.0-cp314-cp314t-win_amd64.whl", hash = "sha256:b33b6cd332695bba180d55e717a79d3038e479a2c49cc5eb3d53603409b9a5d7", size = 12866622, upload-time = "2026-05-11T18:34:39.945Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/50/1043e1db5f455ffe4c9ab22747cd8ca2bc492b1e4f4e21b130a44ee2b217/mypy-2.1.0-cp314-cp314t-win_arm64.whl", hash = "sha256:4f910fe825376a7b66ef7ca8c98e5a149e8cd64c19ae71d84047a74ee060d4e6", size = 10610798, upload-time = "2026-05-11T18:36:31.444Z" },
+ { url = "https://files.pythonhosted.org/packages/0d/2a/13ca1f292f6db1b98ff495ef3467736b331621c5917cad984b7043e7348d/mypy-2.1.0-py3-none-any.whl", hash = "sha256:a663814603a5c563fb87a4f96fb473eeb30d1f5a4885afcf44f9db000a366289", size = 2693302, upload-time = "2026-05-11T18:31:29.246Z" },
+]
+
+[[package]]
+name = "mypy-extensions"
+version = "1.1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/a2/6e/371856a3fb9d31ca8dac321cda606860fa4548858c0cc45d9d1d4ca2628b/mypy_extensions-1.1.0.tar.gz", hash = "sha256:52e68efc3284861e772bbcd66823fde5ae21fd2fdb51c62a211403730b916558", size = 6343, upload-time = "2025-04-22T14:54:24.164Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/79/7b/2c79738432f5c924bef5071f933bcc9efd0473bac3b4aa584a6f7c1c8df8/mypy_extensions-1.1.0-py3-none-any.whl", hash = "sha256:1be4cccdb0f2482337c4743e60421de3a356cd97508abadd57d47403e94f5505", size = 4963, upload-time = "2025-04-22T14:54:22.983Z" },
+]
+
+[[package]]
+name = "myst-nb"
+version = "1.4.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "importlib-metadata" },
+ { name = "ipykernel" },
+ { name = "ipython" },
+ { name = "jupyter-cache" },
+ { name = "myst-parser" },
+ { name = "nbclient" },
+ { name = "nbformat" },
+ { name = "pyyaml" },
+ { name = "sphinx" },
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/bd/b4/ff1abeea67e8cfe0a8c033389f6d1d8b0bfecfd611befb5cbdeab884fce6/myst_nb-1.4.0.tar.gz", hash = "sha256:c145598de62446a6fd009773dd071a40d3b76106ace780de1abdfc6961f614c2", size = 82285, upload-time = "2026-03-02T21:14:56.95Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/94/93/0a378b48488879a1d925b42a804edfc6e0cd0ef854220f2dce738a46e7e9/myst_nb-1.4.0-py3-none-any.whl", hash = "sha256:0e2c86e7d3b82c3aa51383f82d6268f7714f3b772c23a796ab09538a8e68b4e4", size = 82555, upload-time = "2026-03-02T21:14:55.652Z" },
+]
+
+[[package]]
+name = "myst-parser"
+version = "5.1.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "docutils" },
+ { name = "jinja2" },
+ { name = "markdown-it-py" },
+ { name = "mdit-py-plugins" },
+ { name = "pyyaml" },
+ { name = "sphinx" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/21/dc/603751677fff302f34396e206b610f556a59d7fe58b9a2145f54e96b48e8/myst_parser-5.1.0.tar.gz", hash = "sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02", size = 101182, upload-time = "2026-05-13T09:38:19.361Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/09/dc/f3dfb7488b770f3f67e6545085bf2abea5172e88f57b8ad25ef860ca704c/myst_parser-5.1.0-py3-none-any.whl", hash = "sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a", size = 85817, upload-time = "2026-05-13T09:38:17.904Z" },
+]
+
+[[package]]
+name = "narwhals"
+version = "2.22.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/62/3c/c4ef2164a71c1a63d7f1ae411c4082c5fa872405106db60a4b7114989ad7/narwhals-2.22.1.tar.gz", hash = "sha256:d62920805a0a43b7ff8b54b0c0d3142d796f8a9301836ada37e573d6a33cbcd9", size = 647493, upload-time = "2026-06-05T12:34:34.051Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/48/ca/36339329c4604adbcc99c899b7eb1ce1a555c499b6a6860757dc9bfed36d/narwhals-2.22.1-py3-none-any.whl", hash = "sha256:60567d774edf77db53906f89d9fbd164e66e56d66d388e1e6990f17ac33cfb53", size = 454815, upload-time = "2026-06-05T12:34:32.289Z" },
+]
+
+[[package]]
+name = "natsort"
+version = "8.4.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/e2/a9/a0c57aee75f77794adaf35322f8b6404cbd0f89ad45c87197a937764b7d0/natsort-8.4.0.tar.gz", hash = "sha256:45312c4a0e5507593da193dedd04abb1469253b601ecaf63445ad80f0a1ea581", size = 76575, upload-time = "2023-06-20T04:17:19.925Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ef/82/7a9d0550484a62c6da82858ee9419f3dd1ccc9aa1c26a1e43da3ecd20b0d/natsort-8.4.0-py3-none-any.whl", hash = "sha256:4732914fb471f56b5cce04d7bae6f164a592c7712e1c85f9ef585e197299521c", size = 38268, upload-time = "2023-06-20T04:17:17.522Z" },
+]
+
+[[package]]
+name = "nbclient"
+version = "0.11.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "jupyter-client" },
+ { name = "jupyter-core" },
+ { name = "nbformat" },
+ { name = "traitlets" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/28/a5/b3bae4b590c0cbcada2c63a34f7580024e834a8ba213e949a2f906705787/nbclient-0.11.0.tar.gz", hash = "sha256:04a134a5b087f2c5887f228aca155db50169b8cd9334dee6942c8e927e56081a", size = 62535, upload-time = "2026-06-05T07:52:41.746Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/36/c9/94d73e5a01c5b926c3fa2496e97d7a8dc28ed5a77c0b2ed712f1a62e6694/nbclient-0.11.0-py3-none-any.whl", hash = "sha256:ef7fa0d59d6e1d41103933d8a445a18d5de860ca6b613b87b8574accdb3c2895", size = 25288, upload-time = "2026-06-05T07:52:40.115Z" },
+]
+
+[[package]]
+name = "nbformat"
+version = "5.10.4"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "fastjsonschema" },
+ { name = "jsonschema" },
+ { name = "jupyter-core" },
+ { name = "traitlets" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/6d/fd/91545e604bc3dad7dca9ed03284086039b294c6b3d75c0d2fa45f9e9caf3/nbformat-5.10.4.tar.gz", hash = "sha256:322168b14f937a5d11362988ecac2a4952d3d8e3a2cbeb2319584631226d5b3a", size = 142749, upload-time = "2024-04-04T11:20:37.371Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/a9/82/0340caa499416c78e5d8f5f05947ae4bc3cba53c9f038ab6e9ed964e22f1/nbformat-5.10.4-py3-none-any.whl", hash = "sha256:3b48d6c8fbca4b299bf3982ea7db1af21580e4fec269ad087b9e81588891200b", size = 78454, upload-time = "2024-04-04T11:20:34.895Z" },
+]
+
+[[package]]
+name = "nest-asyncio2"
+version = "1.7.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/b4/73/731debf26e27e0a0323d7bda270dc2f634b398e38f040a09da1f4351d0aa/nest_asyncio2-1.7.2.tar.gz", hash = "sha256:1921d70b92cc4612c374928d081552efb59b83d91b2b789d935c665fa01729a8", size = 14743, upload-time = "2026-02-13T00:34:04.386Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/c5/3c/3179b85b0e1c3659f0369940200cd6d0fa900e6cefcc7ea0bc6dd0e29ffb/nest_asyncio2-1.7.2-py3-none-any.whl", hash = "sha256:f5dfa702f3f81f6a03857e9a19e2ba578c0946a4ad417b4c50a24d7ba641fe01", size = 7843, upload-time = "2026-02-13T00:34:02.691Z" },
+]
+
+[[package]]
+name = "networkx"
+version = "3.6.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/6a/51/63fe664f3908c97be9d2e4f1158eb633317598cfa6e1fc14af5383f17512/networkx-3.6.1.tar.gz", hash = "sha256:26b7c357accc0c8cde558ad486283728b65b6a95d85ee1cd66bafab4c8168509", size = 2517025, upload-time = "2025-12-08T17:02:39.908Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/9e/c9/b2622292ea83fbb4ec318f5b9ab867d0a28ab43c5717bb85b0a5f6b3b0a4/networkx-3.6.1-py3-none-any.whl", hash = "sha256:d47fbf302e7d9cbbb9e2555a0d267983d2aa476bac30e90dfbe5669bd57f3762", size = 2068504, upload-time = "2025-12-08T17:02:38.159Z" },
+]
+
+[[package]]
+name = "nextprod"
+version = "1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/e6/1a/d1fff6df6d7b24e4009d50428eb56855c5b7dcb50fe46d803ee6726095dc/nextprod-1.0.tar.gz", hash = "sha256:1c0eff048c819da613bff18fd3786763af776b75b36516b9a6e1843c2d598538", size = 1956, upload-time = "2019-09-11T01:31:33.412Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/a6/df/24a79fbc8dd7aeeadfec1bcee3405810c9e6a3c1bb216a152bdab58fde8f/nextprod-1.0-py3-none-any.whl", hash = "sha256:a4e1199486ead04c75bc31a0030157c22d9c6870936f4e3e16a872cb55a21624", size = 2289, upload-time = "2019-09-11T01:31:30.791Z" },
+]
+
+[[package]]
+name = "numba"
+version = "0.65.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "llvmlite" },
+ { name = "numpy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/f6/c5/db2ac3685833d626c0dcae6bd2330cd68433e1fd248d15f70998160d3ad7/numba-0.65.1.tar.gz", hash = "sha256:19357146c32fe9ed25059ab915e8465fb13951cf6b0aace3826b76886373ab23", size = 2765600, upload-time = "2026-04-24T02:02:56.551Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/57/bc/76f8f8c5cf9adee47fdb7bbb03be8900f76f902d451d7477cf12b845e1de/numba-0.65.1-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:ac3f1e77c352dd0ea9712732c2d8f9ca507717435eec5b5013bf138ac33c4a08", size = 2681371, upload-time = "2026-04-24T02:02:26.105Z" },
+ { url = "https://files.pythonhosted.org/packages/69/47/a415af0283e4db0398104c6d1c11c9861a98dc67a7aa442a7769ed5d6196/numba-0.65.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:52bc6f3ceb8fcaff9b2ae26b4c6b1e9fee39db8d355534c0fe4f39a901246b84", size = 3802467, upload-time = "2026-04-24T02:02:27.712Z" },
+ { url = "https://files.pythonhosted.org/packages/46/36/246f73ec99cfeab2f2cb2ce7d4218766cc36a2da418901223f4f4da9c813/numba-0.65.1-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:90ca10b3463bae0bd70589726fe3c77d01d6b5fc86bee54bcdf9fb6b47c28977", size = 3502628, upload-time = "2026-04-24T02:02:29.763Z" },
+ { url = "https://files.pythonhosted.org/packages/db/9e/3c679b2ee078425b9e99a91e44f8d132a6830d8ccce5227bc5e9181aeed8/numba-0.65.1-cp312-cp312-win_amd64.whl", hash = "sha256:5971c632be2a2351500431f46213821dba8d02b18a9f7d02fd36bd2743e41a6a", size = 2750611, upload-time = "2026-04-24T02:02:31.477Z" },
+ { url = "https://files.pythonhosted.org/packages/79/37/14a4579049c1eb673afd0de0cb4842982acd55b9ce2643e763db858bcea0/numba-0.65.1-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:1735c15c1134a5108b4d6a5c77fc0947924ea066a738dc09a52008c13df9cad3", size = 2681344, upload-time = "2026-04-24T02:02:33.65Z" },
+ { url = "https://files.pythonhosted.org/packages/a0/22/b8d873f6466b20aa563fc9b33acd48dec89a07803ddaa2f1c8ca1cd33126/numba-0.65.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c09f49117ef255e1f1c6dad0c7a1ed39868243862a73be5706793241a3755f1b", size = 3810619, upload-time = "2026-04-24T02:02:36.041Z" },
+ { url = "https://files.pythonhosted.org/packages/62/08/e16a8b5d9a018962ebb5c66be662317cde32b9f5dab08441f90bed5522fb/numba-0.65.1-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:594a8680b3fadac99e97e489b1fd89007177e5336713745c3b769528c635a464", size = 3509783, upload-time = "2026-04-24T02:02:38.245Z" },
+ { url = "https://files.pythonhosted.org/packages/fd/a5/03c970d57f4c1741354837353ce39fb5206952ae1dba8922d29c86f64805/numba-0.65.1-cp313-cp313-win_amd64.whl", hash = "sha256:85be74c0d036842699a30058f82fb88fc5ffdc59f7615cab5792ea92914c9b62", size = 2750534, upload-time = "2026-04-24T02:02:39.903Z" },
+ { url = "https://files.pythonhosted.org/packages/4f/2e/8aed9b726d9ba5f11ad287645fd479e88278db3060a25cb1225d730eb2b7/numba-0.65.1-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:33f5eb68eb1c843511615d14663ce60258525d6a4c65ab040e2c2b0c4cf17450", size = 2681554, upload-time = "2026-04-24T02:02:41.812Z" },
+ { url = "https://files.pythonhosted.org/packages/87/96/f3eb235fafa82a34e2ab5dd7dc9ffff998ebf5f0bbc23fa56a96aeb44da6/numba-0.65.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:71e73029bf53a62cc6afcf96be4bd942290d8b4c55f0a454fb536158115790f7", size = 3779602, upload-time = "2026-04-24T02:02:43.726Z" },
+ { url = "https://files.pythonhosted.org/packages/09/90/b0f09b48752d23640b8284f22aa597737e8adaddc7fbfacc4708b7f73a4c/numba-0.65.1-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a07635e0be926b9bdbffb09137c230fb13f6ec0e564914ba937cee12ce3eb35", size = 3479532, upload-time = "2026-04-24T02:02:45.427Z" },
+ { url = "https://files.pythonhosted.org/packages/56/46/3f7fc04fb853559e74b210e0b62c19974ec844cefec611f9e535f4da3761/numba-0.65.1-cp314-cp314-win_amd64.whl", hash = "sha256:2a20fcdabdefbdacf88d85caf70c3b18c4bcb7ebb8f82e6a19486383dd26ab63", size = 2752637, upload-time = "2026-04-24T02:02:47.664Z" },
+ { url = "https://files.pythonhosted.org/packages/81/7b/c1a341a9067367778f4152a5f01061cf281fb09582c92c510ec4918cabf6/numba-0.65.1-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:548dd4b3a4508d5062768d1514b2cd7b015f9a25ec7af651c50dee243965e652", size = 2684600, upload-time = "2026-04-24T02:02:49.653Z" },
+ { url = "https://files.pythonhosted.org/packages/03/36/98ddbcf3e4f04a6dd07e1c67249955920579ba4af6bb6868e3088f4ed282/numba-0.65.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:78abc28feff2c2ff8307fff3975b6438352759c9acb797ecd6b1fb6e7e39e31d", size = 3817198, upload-time = "2026-04-24T02:02:51.266Z" },
+ { url = "https://files.pythonhosted.org/packages/a3/83/0dad21057ece5a835599f5d24099b091703995e23dbbf894f259e91c010b/numba-0.65.1-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee7676cb389555805f9b9a1840cbcd1ea6c8bd5376ab6918e3a29c5ea1dbda20", size = 3533862, upload-time = "2026-04-24T02:02:52.987Z" },
+ { url = "https://files.pythonhosted.org/packages/32/36/8be7118ffd4c8440881046eac3d0982cc5ab42909508cf5d67024d62a2e4/numba-0.65.1-cp314-cp314t-win_amd64.whl", hash = "sha256:20609346e3bd75204950dcbbfe383a8d7dbf4902f442aedbf00f97fef4aa8f38", size = 2758237, upload-time = "2026-04-24T02:02:54.612Z" },
+]
+
+[[package]]
+name = "numcodecs"
+version = "0.16.5"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/44/bd/8a391e7c356366224734efd24da929cc4796fff468bfb179fe1af6548535/numcodecs-0.16.5.tar.gz", hash = "sha256:0d0fb60852f84c0bd9543cc4d2ab9eefd37fc8efcc410acd4777e62a1d300318", size = 6276387, upload-time = "2025-11-21T02:49:48.986Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/75/cc/55420f3641a67f78392dc0bc5d02cb9eb0a9dcebf2848d1ac77253ca61fa/numcodecs-0.16.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:24e675dc8d1550cd976a99479b87d872cb142632c75cc402fea04c08c4898523", size = 1656287, upload-time = "2025-11-21T02:49:25.755Z" },
+ { url = "https://files.pythonhosted.org/packages/f5/6c/86644987505dcb90ba6d627d6989c27bafb0699f9fd00187e06d05ea8594/numcodecs-0.16.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:94ddfa4341d1a3ab99989d13b01b5134abb687d3dab2ead54b450aefe4ad5bd6", size = 1148899, upload-time = "2025-11-21T02:49:26.87Z" },
+ { url = "https://files.pythonhosted.org/packages/97/1e/98aaddf272552d9fef1f0296a9939d1487914a239e98678f6b20f8b0a5c8/numcodecs-0.16.5-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b554ab9ecf69de7ca2b6b5e8bc696bd9747559cb4dd5127bd08d7a28bec59c3a", size = 8534814, upload-time = "2025-11-21T02:49:28.547Z" },
+ { url = "https://files.pythonhosted.org/packages/fb/53/78c98ef5c8b2b784453487f3e4d6c017b20747c58b470393e230c78d18e8/numcodecs-0.16.5-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ad1a379a45bd3491deab8ae6548313946744f868c21d5340116977ea3be5b1d6", size = 9173471, upload-time = "2025-11-21T02:49:30.444Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/20/2fdec87fc7f8cec950d2b0bea603c12dc9f05b4966dc5924ba5a36a61bf6/numcodecs-0.16.5-cp312-cp312-win_amd64.whl", hash = "sha256:845a9857886ffe4a3172ba1c537ae5bcc01e65068c31cf1fce1a844bd1da050f", size = 801412, upload-time = "2025-11-21T02:49:32.123Z" },
+ { url = "https://files.pythonhosted.org/packages/38/38/071ced5a5fd1c85ba0e14ba721b66b053823e5176298c2f707e50bed11d9/numcodecs-0.16.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:25be3a516ab677dad890760d357cfe081a371d9c0a2e9a204562318ac5969de3", size = 1654359, upload-time = "2025-11-21T02:49:33.673Z" },
+ { url = "https://files.pythonhosted.org/packages/d1/c0/5f84ba7525577c1b9909fc2d06ef11314825fc4ad4378f61d0e4c9883b4a/numcodecs-0.16.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:0107e839ef75b854e969cb577e140b1aadb9847893937636582d23a2a4c6ce50", size = 1144237, upload-time = "2025-11-21T02:49:35.294Z" },
+ { url = "https://files.pythonhosted.org/packages/0b/00/787ea5f237b8ea7bc67140c99155f9c00b5baf11c49afc5f3bfefa298f95/numcodecs-0.16.5-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:015a7c859ecc2a06e2a548f64008c0ec3aaecabc26456c2c62f4278d8fc20597", size = 8483064, upload-time = "2025-11-21T02:49:36.454Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/e6/d359fdd37498e74d26a167f7a51e54542e642ea47181eb4e643a69a066c3/numcodecs-0.16.5-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:84230b4b9dad2392f2a84242bd6e3e659ac137b5a1ce3571d6965fca673e0903", size = 9126063, upload-time = "2025-11-21T02:49:38.018Z" },
+ { url = "https://files.pythonhosted.org/packages/27/72/6663cc0382ddbb866136c255c837bcb96cc7ce5e83562efec55e1b995941/numcodecs-0.16.5-cp313-cp313-win_amd64.whl", hash = "sha256:5088145502ad1ebf677ec47d00eb6f0fd600658217db3e0c070c321c85d6cf3d", size = 799275, upload-time = "2025-11-21T02:49:39.558Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/9e/38e7ca8184c958b51f45d56a4aeceb1134ecde2d8bd157efadc98502cc42/numcodecs-0.16.5-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:b05647b8b769e6bc8016e9fd4843c823ce5c9f2337c089fb5c9c4da05e5275de", size = 1654721, upload-time = "2025-11-21T02:49:40.602Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/37/260fa42e7b2b08e6e00ad632f8dd620961a60a459426c26cea390f8c68d0/numcodecs-0.16.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:3832bd1b5af8bb3e413076b7d93318c8e7d7b68935006b9fa36ca057d1725a8f", size = 1146887, upload-time = "2025-11-21T02:49:41.721Z" },
+ { url = "https://files.pythonhosted.org/packages/4e/15/e2e1151b5a8b14a15dfd4bb4abccce7fff7580f39bc34092780088835f3a/numcodecs-0.16.5-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:49f7b7d24f103187f53135bed28bb9f0ed6b2e14c604664726487bb6d7c882e1", size = 8476987, upload-time = "2025-11-21T02:49:43.363Z" },
+ { url = "https://files.pythonhosted.org/packages/6d/30/16a57fc4d9fb0ba06c600408bd6634f2f1753c54a7a351c99c5e09b51ee2/numcodecs-0.16.5-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:aec9736d81b70f337d89c4070ee3ffeff113f386fd789492fa152d26a15043e4", size = 9102377, upload-time = "2025-11-21T02:49:45.508Z" },
+ { url = "https://files.pythonhosted.org/packages/31/a5/a0425af36c20d55a3ea884db4b4efca25a43bea9214ba69ca7932dd997b4/numcodecs-0.16.5-cp314-cp314-win_amd64.whl", hash = "sha256:b16a14303800e9fb88abc39463ab4706c037647ac17e49e297faa5f7d7dbbf1d", size = 819022, upload-time = "2025-11-21T02:49:47.39Z" },
+]
+
+[[package]]
+name = "numpy"
+version = "2.4.6"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/d0/ad/fed0499ce6a338d2a03ebae59cd15093910c8875328855781952abf6c2fe/numpy-2.4.6.tar.gz", hash = "sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda", size = 20735807, upload-time = "2026-05-18T23:37:14.07Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/95/2a/3d7b5ac8aac24feaf9ad7ed58f45b0bbc06d37e4338ae84c9f2298b570f9/numpy-2.4.6-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:001fbb8e08d942dd57599e781f2472269ee7f2755fae407b4f67b2f0b17da3f1", size = 16689119, upload-time = "2026-05-18T23:33:54.065Z" },
+ { url = "https://files.pythonhosted.org/packages/ea/12/92c4c131527599e8288d6918e888d88726f84d805d784b771f32408aeaef/numpy-2.4.6-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ebfb099f8dcf083deef3ac1ca4c1503f387cf76296fcb3816b66f5ecb5f54fdb", size = 14699246, upload-time = "2026-05-18T23:33:57.621Z" },
+ { url = "https://files.pythonhosted.org/packages/ad/fe/c0a6b7b2ca128a8fb228575147073b660656734b8ebe4d76c8fd748dcc79/numpy-2.4.6-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:3213d622a0283a39a93d188f3cf72b26862df52fbb4ca3697f51705016523d41", size = 5204410, upload-time = "2026-05-18T23:34:00.302Z" },
+ { url = "https://files.pythonhosted.org/packages/f3/d4/9770d14ba719432bb90a421bfd443872ed0f70f7264b64bec12ea363d5fd/numpy-2.4.6-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:357cc07a6d7b0b182ff02249616a03742827ebb1277546b5c7cd7f7620a45698", size = 6551240, upload-time = "2026-05-18T23:34:02.852Z" },
+ { url = "https://files.pythonhosted.org/packages/c9/c6/50a46a6205feba2343f1d6d17438107c5dc491ed1c736e6ea68689fd906b/numpy-2.4.6-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f9fb9157b4ce2971008323afe46053787b526ef624fea915b261468a8421a0f", size = 15671012, upload-time = "2026-05-18T23:34:05.485Z" },
+ { url = "https://files.pythonhosted.org/packages/99/60/14115e6364fa676c5397c2ad3004e527e9aa487abf5d0706ec81bbd08529/numpy-2.4.6-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:90f9849678c75fe7afa2d348ac842c168b0a4d3d61919687216dfc547976d853", size = 16645538, upload-time = "2026-05-18T23:34:09.265Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/c5/693cbe59e57db94d2231fa519ca3978dc9e19da5a8f088588f5c6e947ff2/numpy-2.4.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c1a2af6c6ef86344a6b0db6b97834208bf598db514f2b155042439b62605601a", size = 17020706, upload-time = "2026-05-18T23:34:13.053Z" },
+ { url = "https://files.pythonhosted.org/packages/ef/fc/85b7c4eff9b4966ade25c2273cf7e7012e92366c032058653934b37de044/numpy-2.4.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:e5805d5a22fd19c8ccff10a9561f9df94436b0545619ea579db2d3c35294bce2", size = 18368541, upload-time = "2026-05-18T23:34:17.024Z" },
+ { url = "https://files.pythonhosted.org/packages/f6/81/e1b27545deedce7f4a0b348618c6b62d74e36a4dc9ccd42f3eb2f85eee32/numpy-2.4.6-cp312-cp312-win32.whl", hash = "sha256:e3eeb0aabd6bd5ce64faae67e9935203a6991b4bc2a485a767fbafb2c5125f45", size = 5962825, upload-time = "2026-05-18T23:34:20.3Z" },
+ { url = "https://files.pythonhosted.org/packages/ab/ca/feab00bd44aa5fe1ad2c18f08b4d3bb92e26484b0b1d1443897809ed528c/numpy-2.4.6-cp312-cp312-win_amd64.whl", hash = "sha256:d8e8286dd7cea7895157318d1b91cdacac64c479f3cbc8dce548331728484751", size = 12321687, upload-time = "2026-05-18T23:34:23.095Z" },
+ { url = "https://files.pythonhosted.org/packages/63/cf/5a6d34850a39d1093558564f77ee8e8e0bee5061151b8f05a55711001ec7/numpy-2.4.6-cp312-cp312-win_arm64.whl", hash = "sha256:4081eb135ac24158bd51cdfbef16f1c64df7063b1143f24731387137c092bec8", size = 10221482, upload-time = "2026-05-18T23:34:25.876Z" },
+ { url = "https://files.pythonhosted.org/packages/fb/82/bdab26d7438c6791ca31b7c024ca37c1eab8b726ba236129005cd4a06e45/numpy-2.4.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:511dbaf848decaaaf4b4ca48032619fb3138710c4bf7da7617765edad1ef96b0", size = 16684648, upload-time = "2026-05-18T23:34:29.41Z" },
+ { url = "https://files.pythonhosted.org/packages/1b/30/a80189bcc7f5e4258b3fbc3968d909d1756f54d023299ecc39ad6fdb9ef8/numpy-2.4.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bf162abab1c1a736333192707cef898e735a5ca00f38f27eeedf44b39d9e85eb", size = 14693902, upload-time = "2026-05-18T23:34:33.013Z" },
+ { url = "https://files.pythonhosted.org/packages/97/12/70b5d0d7c15e1ebb8a6a84a8caa1d19e181d84fb58bb6d70aca29099dec1/numpy-2.4.6-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:043191bfa8eab18c776647b62723ac9dddece59743b13f49b2016094129c2b3f", size = 5198992, upload-time = "2026-05-18T23:34:36.132Z" },
+ { url = "https://files.pythonhosted.org/packages/ba/8c/ebd2a8f8a83541f8d38cc5667e8c2b69cecfd30da6e45693e8158857d44b/numpy-2.4.6-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:6180d8b35af935aed8ece3a85e0a43f87393ae0ac87c8d2c8bd2c993f7270ef3", size = 6546944, upload-time = "2026-05-18T23:34:38.484Z" },
+ { url = "https://files.pythonhosted.org/packages/bb/c5/7b863a97a91671a0338f4253bd3b5a3d3852f0692dae91711c9f4a10e787/numpy-2.4.6-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72fbe16c6fac95aedf5937fa873445cec2110be35d8a4e9433d7501fd98dae6b", size = 15669392, upload-time = "2026-05-18T23:34:41.257Z" },
+ { url = "https://files.pythonhosted.org/packages/a5/9d/3584b9984ca4c047aea75214ce1a4c4c73d849bd71b604264b7f5653f8a8/numpy-2.4.6-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a7830bab239b79cda9c08c2da014761cafb48da6150e1da17ac06283f43b6089", size = 16633220, upload-time = "2026-05-18T23:34:45.075Z" },
+ { url = "https://files.pythonhosted.org/packages/05/ae/7c67fba23bd98caec7c99261f3a16072ade14813486b0282cb29846de832/numpy-2.4.6-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ef4aea96ce4d3b074422cb4f2f64e216bf9e213004bb58ecfdf50ea02ea8eb9a", size = 17020800, upload-time = "2026-05-18T23:34:49.065Z" },
+ { url = "https://files.pythonhosted.org/packages/d9/5d/3b6725cb31d983c5e66916f5d36f6d7e5521129e4c4404d64f918292a5b6/numpy-2.4.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:dfa20cc6ca228e6b155b11da03825975ce66aea520985dbbddf0f2a5a495c605", size = 18357600, upload-time = "2026-05-18T23:34:52.709Z" },
+ { url = "https://files.pythonhosted.org/packages/f7/da/2ccc6c2fe8898dee01d90c75c5f5f914a23daf99e3e0f59516a08760c8b5/numpy-2.4.6-cp313-cp313-win32.whl", hash = "sha256:56b39e5e0622a09a25bf5baf62f4bcf0cb8a41ae6e2819cf49bbc5a74c083f91", size = 5961134, upload-time = "2026-05-18T23:34:55.618Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/cd/9cc4dc876fb065d5c220aae4d5e14826b2715331bb7618ce1fb07a679d99/numpy-2.4.6-cp313-cp313-win_amd64.whl", hash = "sha256:c4fc99836233ea196540b17ab0983aff60ed07941751930f5f4d05bc3b3b7359", size = 12318598, upload-time = "2026-05-18T23:34:58.928Z" },
+ { url = "https://files.pythonhosted.org/packages/39/1e/c0bcba1f8694116485fe28fd1be698c278fcda4141c5b0e53a2aed8b12a8/numpy-2.4.6-cp313-cp313-win_arm64.whl", hash = "sha256:a7c711e21628b52034bb5ab8d1bce291f752fcc5e92accc615778acee1ff4778", size = 10222272, upload-time = "2026-05-18T23:35:02.167Z" },
+ { url = "https://files.pythonhosted.org/packages/63/6d/cc5619247c8f4204e507f5883528372e4ac4bb189e579fb859a12e480b1f/numpy-2.4.6-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:112b06a867b235ef466ed3508ddf0238050df9c727cafb5301ac385b899189a1", size = 14821197, upload-time = "2026-05-18T23:35:05.468Z" },
+ { url = "https://files.pythonhosted.org/packages/00/58/f1c39161c87d9e9bed660f1ed4bafc0e403d5ec9650b6dd77aead07d489b/numpy-2.4.6-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:eaf7fa2de5c0be8ae6ff8e9bea2ccd725e980541244521d8d4b5f3354a27babe", size = 5326287, upload-time = "2026-05-18T23:35:08.693Z" },
+ { url = "https://files.pythonhosted.org/packages/af/57/3917ab0fd97f271a8694513581b8a36c655f111c446852c302f04ccdb6fc/numpy-2.4.6-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:7265a2f3d436e54ef9f2b52b5c937e6be778781bd97a590319d7348f1c1ca997", size = 6646763, upload-time = "2026-05-18T23:35:11.459Z" },
+ { url = "https://files.pythonhosted.org/packages/eb/0f/037e64c494b67581ae18193d770adef354c41f3f2c8ebf865602d949bf8f/numpy-2.4.6-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f74a575920ab21fe304421a3fc28793d82e299cae9eccb37084e9fc7f3617c20", size = 15728070, upload-time = "2026-05-18T23:35:14.79Z" },
+ { url = "https://files.pythonhosted.org/packages/21/a6/5d2bae9c9542eb4df16dc9c46dc79c186e9bad53805dfa5399a6023c6db0/numpy-2.4.6-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ede83e07a75dd06bc501566c1eca2afc0d61677c1472ac9ad93fdee6e638a48d", size = 16681752, upload-time = "2026-05-18T23:35:18.836Z" },
+ { url = "https://files.pythonhosted.org/packages/92/14/23d1dfb410ae362cd59ce53e936b1513d545eb40db3949ced632e19a459e/numpy-2.4.6-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:68bb27509ac1b9a3443094260f6326150663b06abe40b73a2f81160623da5b67", size = 17086024, upload-time = "2026-05-18T23:35:22.52Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/6e/23595a2c642cdf3bc567877064bdd7f91c8b0038a4453cf2daf7248eafe9/numpy-2.4.6-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:a0df0043bdb289bde1f62da130d20df23d58b45429f752bc7a8fc5325a225ecd", size = 18403398, upload-time = "2026-05-18T23:35:26.398Z" },
+ { url = "https://files.pythonhosted.org/packages/8a/90/0ac3bc947217e66dec77e7cbc6a1979d1af70b6461b82f620d3bccd5e4c8/numpy-2.4.6-cp313-cp313t-win32.whl", hash = "sha256:29a287e0cf63ff528da061de6b9f64a4618da591ca1046aafc54062e40ca7eab", size = 6084971, upload-time = "2026-05-18T23:35:29.387Z" },
+ { url = "https://files.pythonhosted.org/packages/77/71/5673e351671a1d2bd6063b91b44f70c0affea7d1516fa7a6572941ba4aa1/numpy-2.4.6-cp313-cp313t-win_amd64.whl", hash = "sha256:25c692919ac5a01f170a3bfcd62d745b24fd095c353d50812637d6fcab442e75", size = 12458532, upload-time = "2026-05-18T23:35:32.175Z" },
+ { url = "https://files.pythonhosted.org/packages/3f/88/19d3503c5046e688f049274b27a3ef3d771152fa80d3ba3d01a3dff61abe/numpy-2.4.6-cp313-cp313t-win_arm64.whl", hash = "sha256:1e978ec1e8bd0e0e4de6bb75de9d30cbb74db6b6a2bb727618613703ca0167dd", size = 10291881, upload-time = "2026-05-18T23:35:35.465Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/91/3ab2044d05fd16d343c5ac2e69b127f1b2854040dd20b193257c78028bd3/numpy-2.4.6-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079", size = 16683458, upload-time = "2026-05-18T23:35:38.353Z" },
+ { url = "https://files.pythonhosted.org/packages/8e/62/764ce66fa4147ae6d73071a3abf804ffe606f174618697c571acdf26a7c9/numpy-2.4.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7", size = 14704559, upload-time = "2026-05-18T23:35:42.14Z" },
+ { url = "https://files.pythonhosted.org/packages/60/61/23f27c172f022e04025b7dc2367f4d63c1a398120607ec896228649a6f48/numpy-2.4.6-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5", size = 5209716, upload-time = "2026-05-18T23:35:45.377Z" },
+ { url = "https://files.pythonhosted.org/packages/03/71/21cf70dc6ea3e3acb95fc53a265b2fc248b981f0194ceb5b475271b8809d/numpy-2.4.6-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096", size = 6543947, upload-time = "2026-05-18T23:35:47.926Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/91/64288395ee1799bd2e0b04a305dce9666da90c961e1f3fe982a05ee1c036/numpy-2.4.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b", size = 15685197, upload-time = "2026-05-18T23:35:50.863Z" },
+ { url = "https://files.pythonhosted.org/packages/f3/eb/ebffaa97dc55502df69584a8f0dcf07f69a3e0b3e2323670a2722db9aa39/numpy-2.4.6-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8", size = 16638245, upload-time = "2026-05-18T23:35:54.752Z" },
+ { url = "https://files.pythonhosted.org/packages/b8/0b/54f9da33128d7e350fab89c7455902eeae70349ee52bddb448dc4a576f45/numpy-2.4.6-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402", size = 17036587, upload-time = "2026-05-18T23:35:58.355Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/f0/fdebc1052db1cc37c64beb22072d67cd6d1c71adca1299f53dec2b5e20d3/numpy-2.4.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb", size = 18363226, upload-time = "2026-05-18T23:36:02.845Z" },
+ { url = "https://files.pythonhosted.org/packages/aa/b4/298628d98c72b57e57f7165ae6a481a1deaf6f3c28262a6e4c739c275930/numpy-2.4.6-cp314-cp314-win32.whl", hash = "sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1", size = 6010196, upload-time = "2026-05-18T23:36:05.92Z" },
+ { url = "https://files.pythonhosted.org/packages/df/ac/46de6dda46478f7942f839e094970be2d4a861e005c4b3bf07c92e291a09/numpy-2.4.6-cp314-cp314-win_amd64.whl", hash = "sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261", size = 12450334, upload-time = "2026-05-18T23:36:09.107Z" },
+ { url = "https://files.pythonhosted.org/packages/78/92/b8b798ac784102c0da830d2257d59358e3d3d90d1e2b3f2575dad976c5cf/numpy-2.4.6-cp314-cp314-win_arm64.whl", hash = "sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6", size = 10495678, upload-time = "2026-05-18T23:36:12.766Z" },
+ { url = "https://files.pythonhosted.org/packages/30/34/ec28d1aa8115971537c01469ab2011ee96827930f0a124de1000cc2a7ed7/numpy-2.4.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a", size = 14823672, upload-time = "2026-05-18T23:36:16.473Z" },
+ { url = "https://files.pythonhosted.org/packages/16/bd/f6d1fede4e54e8042a7ff97bb495510f3c220f94bcd9e8b228e87c92cc0d/numpy-2.4.6-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e", size = 5328731, upload-time = "2026-05-18T23:36:19.767Z" },
+ { url = "https://files.pythonhosted.org/packages/f4/f0/e105b9e2fd728a9910103884decd6951d9dd73896b914a98d9a231de02ee/numpy-2.4.6-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e", size = 6649805, upload-time = "2026-05-18T23:36:22.266Z" },
+ { url = "https://files.pythonhosted.org/packages/82/dd/1206a7ca6ab15e3f02069707ca96222e202af681bb73756da7527f3cb837/numpy-2.4.6-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43", size = 15730496, upload-time = "2026-05-18T23:36:25.713Z" },
+ { url = "https://files.pythonhosted.org/packages/51/e7/38d3ea825dcab85a591734decb2f6c67caa7c8367d374df1a1c3842f9b07/numpy-2.4.6-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e", size = 16679616, upload-time = "2026-05-18T23:36:29.652Z" },
+ { url = "https://files.pythonhosted.org/packages/93/b7/caabfdf53edf663e0b4eb74d7d405d83baef09eb5e83bcd32d601d72b93e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895", size = 17085145, upload-time = "2026-05-18T23:36:33.449Z" },
+ { url = "https://files.pythonhosted.org/packages/f9/45/68d7c33a6bcf3e5aa3bdbd57a367e6f615286dfd6482f97e8ffeb734306e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4", size = 18403813, upload-time = "2026-05-18T23:36:37.369Z" },
+ { url = "https://files.pythonhosted.org/packages/9c/50/0753655aa844c99cd9e018aacf76f130f1bd81d881bb74bc0aef5d73a8ba/numpy-2.4.6-cp314-cp314t-win32.whl", hash = "sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063", size = 6156982, upload-time = "2026-05-18T23:36:40.817Z" },
+ { url = "https://files.pythonhosted.org/packages/b2/d4/7c67becf668f973cb490cec3e98dfd799d866f9c989a54d355672cfa0db6/numpy-2.4.6-cp314-cp314t-win_amd64.whl", hash = "sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627", size = 12638908, upload-time = "2026-05-18T23:36:43.996Z" },
+ { url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" },
+]
+
+[[package]]
+name = "obstore"
+version = "0.11.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "typing-extensions", marker = "python_full_version < '3.13'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/2e/2f/f83afaab7945509d72245b2b00af0b4834ce78fdd2d9ae9f0ad1a3036a91/obstore-0.11.0.tar.gz", hash = "sha256:a2f55163bcd348b4a60d12e6893eac50eddc742bad8032a1705d49140b992204", size = 130565, upload-time = "2026-06-25T18:29:49.405Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/fc/b2/00c213e7e5ca8065f97e37e55294adab836e3f6a88b23e4029069aaecf95/obstore-0.11.0-cp311-abi3-macosx_10_12_x86_64.whl", hash = "sha256:42f36546c7ac44dbab1173d2330a8a1b1a3f0e37950e553b8c904e3dd0744b25", size = 5491935, upload-time = "2026-06-25T18:28:32.029Z" },
+ { url = "https://files.pythonhosted.org/packages/ac/37/6a6b9a5e15a8a37c24d14317a87648097c4888593b588510c03c030d2e90/obstore-0.11.0-cp311-abi3-macosx_11_0_arm64.whl", hash = "sha256:687bb9d3962d568b7c439c5d0c6fea19b2749862a8e5c8eebd0c058c4eccde9e", size = 4672619, upload-time = "2026-06-25T18:28:33.852Z" },
+ { url = "https://files.pythonhosted.org/packages/28/f9/6745ce8c4f7bfac19dc14a4438b48a2e93a689b92b0cecfc695e41a4e8b1/obstore-0.11.0-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:010b51578c7514a41719d795cdb7a1e6529be509dac3772e477187a59422bb97", size = 5072806, upload-time = "2026-06-25T18:28:36.127Z" },
+ { url = "https://files.pythonhosted.org/packages/6c/18/991d3b3cdd851c0225e55f3dc45b47fd9e249827d188995011469f805132/obstore-0.11.0-cp311-abi3-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:cfaa8129a3f5d8518a3a75184d4b02348db0f6263177cd1f0951f6568243cc9e", size = 5303777, upload-time = "2026-06-25T18:28:37.89Z" },
+ { url = "https://files.pythonhosted.org/packages/8d/e9/90e56015a45b5e56a84fc3188c4e5fb088b288d41992c73a629e10df6760/obstore-0.11.0-cp311-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:c790a5cb9ff2970d1f464a6a708d734dce9939e9f668cb6708c5dba5d61589b2", size = 5493871, upload-time = "2026-06-25T18:28:39.981Z" },
+ { url = "https://files.pythonhosted.org/packages/66/02/f1744091d59ce71c5523174eb860fbb298275c901e89b9ea6fbf3e654a33/obstore-0.11.0-cp311-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:827113e12fe8088e0281a9d57b90b2b8dbc8a6ffe3b15dadb9baa5feb3d266c1", size = 5361913, upload-time = "2026-06-25T18:28:42.089Z" },
+ { url = "https://files.pythonhosted.org/packages/5d/59/3f47822683ee2b6db8685faa25829946d6343a561251ec2704548455d946/obstore-0.11.0-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a2ff6d3ed553298828fb760b4aef6347fbcc7b5c5e3ce3f8381ce805c370021a", size = 5638724, upload-time = "2026-06-25T18:28:43.897Z" },
+ { url = "https://files.pythonhosted.org/packages/23/50/1df335fdf9b527b3933f1e94ab6fc720ad314260fab8591cb0b6668ff192/obstore-0.11.0-cp311-abi3-manylinux_2_24_aarch64.whl", hash = "sha256:39d04b324fcf984e7050734ebda77b81764025b0c011750201a0d8954087f7aa", size = 5413508, upload-time = "2026-06-25T18:28:45.624Z" },
+ { url = "https://files.pythonhosted.org/packages/de/dc/a259aba149b841ca7c91fea177df9972a60a636b54077beed1a35b254994/obstore-0.11.0-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:37c0d15d775b1370ef5204ee3919a5ddf7e2592d11815213105f8db031f2ab8d", size = 5619995, upload-time = "2026-06-25T18:28:47.599Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/b4/ec25fdb4d6b060bc6eea647fc0e88f75fcc20fe8d16d67fb0dbe999d323b/obstore-0.11.0-cp311-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:7f468caf9b6e0f12ff151e5fe618de5fc9192befa9bd02734b06de4efd2e49f6", size = 5299512, upload-time = "2026-06-25T18:28:49.629Z" },
+ { url = "https://files.pythonhosted.org/packages/a8/e5/29be060d06ec13e2af3d1b6cfb77b7c37f8be6c56b77295c945fefad73e4/obstore-0.11.0-cp311-abi3-musllinux_1_2_i686.whl", hash = "sha256:42d8e8fad85be8ee488c1a9a9b7c6a42128abb84e67175da40d3d1165c1846df", size = 5427026, upload-time = "2026-06-25T18:28:51.317Z" },
+ { url = "https://files.pythonhosted.org/packages/57/b7/577a965f440e9ea64243518663f9d16be7df8eafc7123818e8e841fa21ce/obstore-0.11.0-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:9c8fd2a544e2e0b926669c47fcfb8d2314e234abc240ea165dae04ee42e1d7ac", size = 5869187, upload-time = "2026-06-25T18:28:53.166Z" },
+ { url = "https://files.pythonhosted.org/packages/e2/18/8fdbaee22bfd5b9c44e1fdff8ca0508e2fe60c42bf9fc85f0c9c27b4ecf2/obstore-0.11.0-cp311-abi3-win_amd64.whl", hash = "sha256:6fb3d4678c0f4242d3109362e9b1df5d7b27765f43d5aacb2e81af53a75cb9ef", size = 5329384, upload-time = "2026-06-25T18:28:55.305Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/8b/7555e48ec768728fcfc71a051c6b28d6ddaf1bececf492ce5ef995aab5f0/obstore-0.11.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:f3132393eff9f3f2b543ecbb3bcc12319a7c433fef06493b4350d6854d505a14", size = 5515763, upload-time = "2026-06-25T18:28:57.527Z" },
+ { url = "https://files.pythonhosted.org/packages/23/8f/94d83f3336421cbb5e436ab0ae5695eae72f7c82990d6b1ac090712c8052/obstore-0.11.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:a3a8da19b47af4c14ecc694209b3c18ac6d89f96be5656ee3a19b77947c14155", size = 4649491, upload-time = "2026-06-25T18:28:59.386Z" },
+ { url = "https://files.pythonhosted.org/packages/fe/86/11f4e1f51a8c6cf21a5915c018d2357201ad3c5799d418f0c6529fafaab2/obstore-0.11.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e91298b9b6c3a0408c28eece62bca6c5b6cda2f6350351d84e07b4dc8fb2631f", size = 5060659, upload-time = "2026-06-25T18:29:01.139Z" },
+ { url = "https://files.pythonhosted.org/packages/49/e7/fd3036b0923d10e878e2073020f1ef692a618ed1cc3980d3e4a468c93713/obstore-0.11.0-cp314-cp314t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:3422c532486671dfb5e3e739bf15ec9ca2a8da544a3c23b74ae3857dcab1c6a8", size = 5277058, upload-time = "2026-06-25T18:29:02.96Z" },
+ { url = "https://files.pythonhosted.org/packages/2f/a7/b016c3ac6857ac856326dc0a292e3871b8500d03f25f3b90e168b05de357/obstore-0.11.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3de027ce46cf0592c2b41654e2c27dbc52a4a726f6fbc511380acc0da3a9f658", size = 5475852, upload-time = "2026-06-25T18:29:05.032Z" },
+ { url = "https://files.pythonhosted.org/packages/85/9e/644ffe8db7757de7f71f94a036ab24222bccb4de290d3ca69f76547e812d/obstore-0.11.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:1a80a95548678210bc336b866e37139c293565c3b163eb3fef2433d5d6640a33", size = 5363082, upload-time = "2026-06-25T18:29:06.878Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/0b/26af6b5fa6ba96af84086f44f78b4e5b0af1729c31402d7b28c68989d174/obstore-0.11.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:acaa261dc15efb95bbeca06f8fe9b47ee23d7302a6aa1fa3a9654baab8b23d7c", size = 5629116, upload-time = "2026-06-25T18:29:08.771Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/ce/f30d502991c6719b2fbd7b8385ef3e39da07bfca099108bcc5eeed8b9c20/obstore-0.11.0-cp314-cp314t-manylinux_2_24_aarch64.whl", hash = "sha256:a3300cabbc3129670987b3723629c791d83c117ef1b6a0c670c2043648e000a1", size = 5404534, upload-time = "2026-06-25T18:29:11.294Z" },
+ { url = "https://files.pythonhosted.org/packages/52/20/d5bf5f816e868717ba647ed9a2109e800deb402d0265d410456b3fcb4376/obstore-0.11.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:f4e6a9480843645cd4ee122c41d5c5a46a56f1e9cdda85638826f2e0e439fe5c", size = 5613159, upload-time = "2026-06-25T18:29:13.414Z" },
+ { url = "https://files.pythonhosted.org/packages/db/b4/6d4c1c211e3b06cc8554189e0d4406e8fa1f98ed55f9213b8e398a11599f/obstore-0.11.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:9fb2b1814c4314b8903f4e2ebbe8c3365fea6543669615ee9a0288b0d3a2edeb", size = 5286279, upload-time = "2026-06-25T18:29:15.424Z" },
+ { url = "https://files.pythonhosted.org/packages/02/5e/d7b5589424a56171b16ab94cf92eb493490c300aaa044913bbdd94cace68/obstore-0.11.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:63fb9b072815eafe4705f617f567d896b1c858adcb390795fa1e269367791031", size = 5401780, upload-time = "2026-06-25T18:29:17.514Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/18/841baea8936e51a18b0e5d4c51f09c0a7798cb73b027e9794be2362a0f0b/obstore-0.11.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:086fafba314ff98cfab1c4bf7814699e862513e8889720cf6f7462296cb32787", size = 5853618, upload-time = "2026-06-25T18:29:19.354Z" },
+ { url = "https://files.pythonhosted.org/packages/83/9a/d6127f5422b78e0222b0a9eadcfd7a5aa8d873a9498da7d4a77d4ac8ce2e/obstore-0.11.0-cp314-cp314t-win_amd64.whl", hash = "sha256:676d1154f6f08721110f9b7d14ee3a3c0293abaf9da135bb90f54e276dca1cac", size = 5314113, upload-time = "2026-06-25T18:29:21.209Z" },
+]
+
+[[package]]
+name = "packaging"
+version = "26.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
+]
+
+[[package]]
+name = "pandas"
+version = "2.3.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+ { name = "python-dateutil" },
+ { name = "pytz" },
+ { name = "tzdata" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/33/01/d40b85317f86cf08d853a4f495195c73815fdf205eef3993821720274518/pandas-2.3.3.tar.gz", hash = "sha256:e05e1af93b977f7eafa636d043f9f94c7ee3ac81af99c13508215942e64c993b", size = 4495223, upload-time = "2025-09-29T23:34:51.853Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/9c/fb/231d89e8637c808b997d172b18e9d4a4bc7bf31296196c260526055d1ea0/pandas-2.3.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:6d21f6d74eb1725c2efaa71a2bfc661a0689579b58e9c0ca58a739ff0b002b53", size = 11597846, upload-time = "2025-09-29T23:19:48.856Z" },
+ { url = "https://files.pythonhosted.org/packages/5c/bd/bf8064d9cfa214294356c2d6702b716d3cf3bb24be59287a6a21e24cae6b/pandas-2.3.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:3fd2f887589c7aa868e02632612ba39acb0b8948faf5cc58f0850e165bd46f35", size = 10729618, upload-time = "2025-09-29T23:39:08.659Z" },
+ { url = "https://files.pythonhosted.org/packages/57/56/cf2dbe1a3f5271370669475ead12ce77c61726ffd19a35546e31aa8edf4e/pandas-2.3.3-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ecaf1e12bdc03c86ad4a7ea848d66c685cb6851d807a26aa245ca3d2017a1908", size = 11737212, upload-time = "2025-09-29T23:19:59.765Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/63/cd7d615331b328e287d8233ba9fdf191a9c2d11b6af0c7a59cfcec23de68/pandas-2.3.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b3d11d2fda7eb164ef27ffc14b4fcab16a80e1ce67e9f57e19ec0afaf715ba89", size = 12362693, upload-time = "2025-09-29T23:20:14.098Z" },
+ { url = "https://files.pythonhosted.org/packages/a6/de/8b1895b107277d52f2b42d3a6806e69cfef0d5cf1d0ba343470b9d8e0a04/pandas-2.3.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:a68e15f780eddf2b07d242e17a04aa187a7ee12b40b930bfdd78070556550e98", size = 12771002, upload-time = "2025-09-29T23:20:26.76Z" },
+ { url = "https://files.pythonhosted.org/packages/87/21/84072af3187a677c5893b170ba2c8fbe450a6ff911234916da889b698220/pandas-2.3.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:371a4ab48e950033bcf52b6527eccb564f52dc826c02afd9a1bc0ab731bba084", size = 13450971, upload-time = "2025-09-29T23:20:41.344Z" },
+ { url = "https://files.pythonhosted.org/packages/86/41/585a168330ff063014880a80d744219dbf1dd7a1c706e75ab3425a987384/pandas-2.3.3-cp312-cp312-win_amd64.whl", hash = "sha256:a16dcec078a01eeef8ee61bf64074b4e524a2a3f4b3be9326420cabe59c4778b", size = 10992722, upload-time = "2025-09-29T23:20:54.139Z" },
+ { url = "https://files.pythonhosted.org/packages/cd/4b/18b035ee18f97c1040d94debd8f2e737000ad70ccc8f5513f4eefad75f4b/pandas-2.3.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:56851a737e3470de7fa88e6131f41281ed440d29a9268dcbf0002da5ac366713", size = 11544671, upload-time = "2025-09-29T23:21:05.024Z" },
+ { url = "https://files.pythonhosted.org/packages/31/94/72fac03573102779920099bcac1c3b05975c2cb5f01eac609faf34bed1ca/pandas-2.3.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bdcd9d1167f4885211e401b3036c0c8d9e274eee67ea8d0758a256d60704cfe8", size = 10680807, upload-time = "2025-09-29T23:21:15.979Z" },
+ { url = "https://files.pythonhosted.org/packages/16/87/9472cf4a487d848476865321de18cc8c920b8cab98453ab79dbbc98db63a/pandas-2.3.3-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e32e7cc9af0f1cc15548288a51a3b681cc2a219faa838e995f7dc53dbab1062d", size = 11709872, upload-time = "2025-09-29T23:21:27.165Z" },
+ { url = "https://files.pythonhosted.org/packages/15/07/284f757f63f8a8d69ed4472bfd85122bd086e637bf4ed09de572d575a693/pandas-2.3.3-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:318d77e0e42a628c04dc56bcef4b40de67918f7041c2b061af1da41dcff670ac", size = 12306371, upload-time = "2025-09-29T23:21:40.532Z" },
+ { url = "https://files.pythonhosted.org/packages/33/81/a3afc88fca4aa925804a27d2676d22dcd2031c2ebe08aabd0ae55b9ff282/pandas-2.3.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4e0a175408804d566144e170d0476b15d78458795bb18f1304fb94160cabf40c", size = 12765333, upload-time = "2025-09-29T23:21:55.77Z" },
+ { url = "https://files.pythonhosted.org/packages/8d/0f/b4d4ae743a83742f1153464cf1a8ecfafc3ac59722a0b5c8602310cb7158/pandas-2.3.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:93c2d9ab0fc11822b5eece72ec9587e172f63cff87c00b062f6e37448ced4493", size = 13418120, upload-time = "2025-09-29T23:22:10.109Z" },
+ { url = "https://files.pythonhosted.org/packages/4f/c7/e54682c96a895d0c808453269e0b5928a07a127a15704fedb643e9b0a4c8/pandas-2.3.3-cp313-cp313-win_amd64.whl", hash = "sha256:f8bfc0e12dc78f777f323f55c58649591b2cd0c43534e8355c51d3fede5f4dee", size = 10993991, upload-time = "2025-09-29T23:25:04.889Z" },
+ { url = "https://files.pythonhosted.org/packages/f9/ca/3f8d4f49740799189e1395812f3bf23b5e8fc7c190827d55a610da72ce55/pandas-2.3.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:75ea25f9529fdec2d2e93a42c523962261e567d250b0013b16210e1d40d7c2e5", size = 12048227, upload-time = "2025-09-29T23:22:24.343Z" },
+ { url = "https://files.pythonhosted.org/packages/0e/5a/f43efec3e8c0cc92c4663ccad372dbdff72b60bdb56b2749f04aa1d07d7e/pandas-2.3.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:74ecdf1d301e812db96a465a525952f4dde225fdb6d8e5a521d47e1f42041e21", size = 11411056, upload-time = "2025-09-29T23:22:37.762Z" },
+ { url = "https://files.pythonhosted.org/packages/46/b1/85331edfc591208c9d1a63a06baa67b21d332e63b7a591a5ba42a10bb507/pandas-2.3.3-cp313-cp313t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6435cb949cb34ec11cc9860246ccb2fdc9ecd742c12d3304989017d53f039a78", size = 11645189, upload-time = "2025-09-29T23:22:51.688Z" },
+ { url = "https://files.pythonhosted.org/packages/44/23/78d645adc35d94d1ac4f2a3c4112ab6f5b8999f4898b8cdf01252f8df4a9/pandas-2.3.3-cp313-cp313t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:900f47d8f20860de523a1ac881c4c36d65efcb2eb850e6948140fa781736e110", size = 12121912, upload-time = "2025-09-29T23:23:05.042Z" },
+ { url = "https://files.pythonhosted.org/packages/53/da/d10013df5e6aaef6b425aa0c32e1fc1f3e431e4bcabd420517dceadce354/pandas-2.3.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:a45c765238e2ed7d7c608fc5bc4a6f88b642f2f01e70c0c23d2224dd21829d86", size = 12712160, upload-time = "2025-09-29T23:23:28.57Z" },
+ { url = "https://files.pythonhosted.org/packages/bd/17/e756653095a083d8a37cbd816cb87148debcfcd920129b25f99dd8d04271/pandas-2.3.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:c4fc4c21971a1a9f4bdb4c73978c7f7256caa3e62b323f70d6cb80db583350bc", size = 13199233, upload-time = "2025-09-29T23:24:24.876Z" },
+ { url = "https://files.pythonhosted.org/packages/04/fd/74903979833db8390b73b3a8a7d30d146d710bd32703724dd9083950386f/pandas-2.3.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:ee15f284898e7b246df8087fc82b87b01686f98ee67d85a17b7ab44143a3a9a0", size = 11540635, upload-time = "2025-09-29T23:25:52.486Z" },
+ { url = "https://files.pythonhosted.org/packages/21/00/266d6b357ad5e6d3ad55093a7e8efc7dd245f5a842b584db9f30b0f0a287/pandas-2.3.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:1611aedd912e1ff81ff41c745822980c49ce4a7907537be8692c8dbc31924593", size = 10759079, upload-time = "2025-09-29T23:26:33.204Z" },
+ { url = "https://files.pythonhosted.org/packages/ca/05/d01ef80a7a3a12b2f8bbf16daba1e17c98a2f039cbc8e2f77a2c5a63d382/pandas-2.3.3-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6d2cefc361461662ac48810cb14365a365ce864afe85ef1f447ff5a1e99ea81c", size = 11814049, upload-time = "2025-09-29T23:27:15.384Z" },
+ { url = "https://files.pythonhosted.org/packages/15/b2/0e62f78c0c5ba7e3d2c5945a82456f4fac76c480940f805e0b97fcbc2f65/pandas-2.3.3-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ee67acbbf05014ea6c763beb097e03cd629961c8a632075eeb34247120abcb4b", size = 12332638, upload-time = "2025-09-29T23:27:51.625Z" },
+ { url = "https://files.pythonhosted.org/packages/c5/33/dd70400631b62b9b29c3c93d2feee1d0964dc2bae2e5ad7a6c73a7f25325/pandas-2.3.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c46467899aaa4da076d5abc11084634e2d197e9460643dd455ac3db5856b24d6", size = 12886834, upload-time = "2025-09-29T23:28:21.289Z" },
+ { url = "https://files.pythonhosted.org/packages/d3/18/b5d48f55821228d0d2692b34fd5034bb185e854bdb592e9c640f6290e012/pandas-2.3.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6253c72c6a1d990a410bc7de641d34053364ef8bcd3126f7e7450125887dffe3", size = 13409925, upload-time = "2025-09-29T23:28:58.261Z" },
+ { url = "https://files.pythonhosted.org/packages/a6/3d/124ac75fcd0ecc09b8fdccb0246ef65e35b012030defb0e0eba2cbbbe948/pandas-2.3.3-cp314-cp314-win_amd64.whl", hash = "sha256:1b07204a219b3b7350abaae088f451860223a52cfb8a6c53358e7948735158e5", size = 11109071, upload-time = "2025-09-29T23:32:27.484Z" },
+ { url = "https://files.pythonhosted.org/packages/89/9c/0e21c895c38a157e0faa1fb64587a9226d6dd46452cac4532d80c3c4a244/pandas-2.3.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:2462b1a365b6109d275250baaae7b760fd25c726aaca0054649286bcfbb3e8ec", size = 12048504, upload-time = "2025-09-29T23:29:31.47Z" },
+ { url = "https://files.pythonhosted.org/packages/d7/82/b69a1c95df796858777b68fbe6a81d37443a33319761d7c652ce77797475/pandas-2.3.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:0242fe9a49aa8b4d78a4fa03acb397a58833ef6199e9aa40a95f027bb3a1b6e7", size = 11410702, upload-time = "2025-09-29T23:29:54.591Z" },
+ { url = "https://files.pythonhosted.org/packages/f9/88/702bde3ba0a94b8c73a0181e05144b10f13f29ebfc2150c3a79062a8195d/pandas-2.3.3-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a21d830e78df0a515db2b3d2f5570610f5e6bd2e27749770e8bb7b524b89b450", size = 11634535, upload-time = "2025-09-29T23:30:21.003Z" },
+ { url = "https://files.pythonhosted.org/packages/a4/1e/1bac1a839d12e6a82ec6cb40cda2edde64a2013a66963293696bbf31fbbb/pandas-2.3.3-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2e3ebdb170b5ef78f19bfb71b0dc5dc58775032361fa188e814959b74d726dd5", size = 12121582, upload-time = "2025-09-29T23:30:43.391Z" },
+ { url = "https://files.pythonhosted.org/packages/44/91/483de934193e12a3b1d6ae7c8645d083ff88dec75f46e827562f1e4b4da6/pandas-2.3.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:d051c0e065b94b7a3cea50eb1ec32e912cd96dba41647eb24104b6c6c14c5788", size = 12699963, upload-time = "2025-09-29T23:31:10.009Z" },
+ { url = "https://files.pythonhosted.org/packages/70/44/5191d2e4026f86a2a109053e194d3ba7a31a2d10a9c2348368c63ed4e85a/pandas-2.3.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:3869faf4bd07b3b66a9f462417d0ca3a9df29a9f6abd5d0d0dbab15dac7abe87", size = 13202175, upload-time = "2025-09-29T23:31:59.173Z" },
+]
+
+[[package]]
+name = "param"
+version = "2.4.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/79/9d/33f32981c9cda89fe22abff60f7c71ef4af3e566b6af530392f8b938470d/param-2.4.1.tar.gz", hash = "sha256:364a33bd8a968b053d8a49d319af78dc31b3ccb9661ecb95c29a3ea509d7e443", size = 217776, upload-time = "2026-06-09T13:17:33.665Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ac/37/1351bbfabbacbe92cd38774f8779ff680fab48c36b5fed9bcfe0c160009c/param-2.4.1-py3-none-any.whl", hash = "sha256:6cdc0869b3ade232fc6d01691223cf86c2bdb53a6d5ba67a33dba838d8b8cf4c", size = 151874, upload-time = "2026-06-09T13:17:31.996Z" },
+]
+
+[[package]]
+name = "parso"
+version = "0.8.7"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/30/4b/90c937815137d43ce71ba043cd3566221e9df6b9c805f24b5d138c9d40a7/parso-0.8.7.tar.gz", hash = "sha256:eaaac4c9fdd5e9e8852dc778d2d7405897ec510f2a298071453e5e3a07914bb1", size = 401824, upload-time = "2026-05-01T23:13:02.138Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/99/5d/8268b644392ee874ee82a635cd0df1773de230bde356c38de28e298392cc/parso-0.8.7-py2.py3-none-any.whl", hash = "sha256:a8926eb2a1b915486941fdbd31e86a4baf88fe8c210f25f2f35ecec5b574ca1c", size = 107025, upload-time = "2026-05-01T23:12:58.867Z" },
+]
+
+[[package]]
+name = "pathspec"
+version = "1.1.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/5a/82/42f767fc1c1143d6fd36efb827202a2d997a375e160a71eb2888a925aac1/pathspec-1.1.1.tar.gz", hash = "sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a", size = 135180, upload-time = "2026-04-27T01:46:08.907Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/f1/d9/7fb5aa316bc299258e68c73ba3bddbc499654a07f151cba08f6153988714/pathspec-1.1.1-py3-none-any.whl", hash = "sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189", size = 57328, upload-time = "2026-04-27T01:46:07.06Z" },
+]
+
+[[package]]
+name = "patsy"
+version = "1.0.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/be/44/ed13eccdd0519eff265f44b670d46fbb0ec813e2274932dc1c0e48520f7d/patsy-1.0.2.tar.gz", hash = "sha256:cdc995455f6233e90e22de72c37fcadb344e7586fb83f06696f54d92f8ce74c0", size = 399942, upload-time = "2025-10-20T16:17:37.535Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/f1/70/ba4b949bdc0490ab78d545459acd7702b211dfccf7eb89bbc1060f52818d/patsy-1.0.2-py2.py3-none-any.whl", hash = "sha256:37bfddbc58fcf0362febb5f54f10743f8b21dd2aa73dec7e7ef59d1b02ae668a", size = 233301, upload-time = "2025-10-20T16:17:36.563Z" },
+]
+
+[[package]]
+name = "pcst-fast"
+version = "1.0.10"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "pybind11" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/d4/fd/64b51c867bad63e6622ff97cad6230b94b19b5a61e30424cd69c8353091c/pcst_fast-1.0.10.tar.gz", hash = "sha256:3b5694110ce2e004471f383267d5e4ab7fe1ba9828954e8c42560ac1e42b25e6", size = 1717415, upload-time = "2024-01-29T22:14:29.769Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/25/a7/d6d72f4dff7d46a7338e9dd638817938e0e208d131d9c508a85c4f9fabd3/pcst_fast-1.0.10-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:375990478236bde3b40925a3dcc279a9dc8bb061db01e79b23c47c5578a32df3", size = 96522, upload-time = "2024-01-29T22:13:51.022Z" },
+ { url = "https://files.pythonhosted.org/packages/6c/d0/c8bd86957614dfe16dee5a843013bf7903ba369928b9684f2582fc18a75b/pcst_fast-1.0.10-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6b3aa1cc8a544e9390584cd0453280ce5da3b93cedbc1b7623b439c40838ddb6", size = 1230119, upload-time = "2024-01-29T22:13:52.029Z" },
+ { url = "https://files.pythonhosted.org/packages/1a/f7/f20c09bbea70c17c1f50273e82347ab8a5d7a2b3aed2288083bee078ea47/pcst_fast-1.0.10-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a051b58d66eeee139a4f4b274467fb737074447cabeb5d4360d31be3b7c6530a", size = 1260476, upload-time = "2024-01-29T22:13:53.481Z" },
+ { url = "https://files.pythonhosted.org/packages/3e/75/68dffc0c863a3fd18e975db44072d0c2e19d6b867d8e32ade54ee0ba313e/pcst_fast-1.0.10-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:837be0ca99f3417b35c873046a0b22d94de309960f3106eb8fb1453b705ad85d", size = 1766196, upload-time = "2024-01-29T22:13:54.698Z" },
+ { url = "https://files.pythonhosted.org/packages/86/2b/eab1a1d752a94980dd0055f55bcee731ac893e169b527e84f0a6393a037e/pcst_fast-1.0.10-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:da194e710fa847aa7707ef9fed391b9a08fafa0b6d681a8846a9cc77d26bffca", size = 1746127, upload-time = "2024-01-29T22:13:56.534Z" },
+]
+
+[[package]]
+name = "pexpect"
+version = "4.9.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "ptyprocess", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/42/92/cc564bf6381ff43ce1f4d06852fc19a2f11d180f23dc32d9588bee2f149d/pexpect-4.9.0.tar.gz", hash = "sha256:ee7d41123f3c9911050ea2c2dac107568dc43b2d3b0c7557a33212c398ead30f", size = 166450, upload-time = "2023-11-25T09:07:26.339Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/9e/c3/059298687310d527a58bb01f3b1965787ee3b40dce76752eda8b44e9a2c5/pexpect-4.9.0-py2.py3-none-any.whl", hash = "sha256:7236d1e080e4936be2dc3e326cec0af72acf9212a7e1d060210e70a47e253523", size = 63772, upload-time = "2023-11-25T06:56:14.81Z" },
+]
+
+[[package]]
+name = "pillow"
+version = "12.2.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/8c/21/c2bcdd5906101a30244eaffc1b6e6ce71a31bd0742a01eb89e660ebfac2d/pillow-12.2.0.tar.gz", hash = "sha256:a830b1a40919539d07806aa58e1b114df53ddd43213d9c8b75847eee6c0182b5", size = 46987819, upload-time = "2026-04-01T14:46:17.687Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/58/be/7482c8a5ebebbc6470b3eb791812fff7d5e0216c2be3827b30b8bb6603ed/pillow-12.2.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:2d192a155bbcec180f8564f693e6fd9bccff5a7af9b32e2e4bf8c9c69dbad6b5", size = 5308279, upload-time = "2026-04-01T14:43:13.246Z" },
+ { url = "https://files.pythonhosted.org/packages/d8/95/0a351b9289c2b5cbde0bacd4a83ebc44023e835490a727b2a3bd60ddc0f4/pillow-12.2.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f3f40b3c5a968281fd507d519e444c35f0ff171237f4fdde090dd60699458421", size = 4695490, upload-time = "2026-04-01T14:43:15.584Z" },
+ { url = "https://files.pythonhosted.org/packages/de/af/4e8e6869cbed569d43c416fad3dc4ecb944cb5d9492defaed89ddd6fe871/pillow-12.2.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:03e7e372d5240cc23e9f07deca4d775c0817bffc641b01e9c3af208dbd300987", size = 6284462, upload-time = "2026-04-01T14:43:18.268Z" },
+ { url = "https://files.pythonhosted.org/packages/e9/9e/c05e19657fd57841e476be1ab46c4d501bffbadbafdc31a6d665f8b737b6/pillow-12.2.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b86024e52a1b269467a802258c25521e6d742349d760728092e1bc2d135b4d76", size = 8094744, upload-time = "2026-04-01T14:43:20.716Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/54/1789c455ed10176066b6e7e6da1b01e50e36f94ba584dc68d9eebfe9156d/pillow-12.2.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7371b48c4fa448d20d2714c9a1f775a81155050d383333e0a6c15b1123dda005", size = 6398371, upload-time = "2026-04-01T14:43:23.443Z" },
+ { url = "https://files.pythonhosted.org/packages/43/e3/fdc657359e919462369869f1c9f0e973f353f9a9ee295a39b1fea8ee1a77/pillow-12.2.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:62f5409336adb0663b7caa0da5c7d9e7bdbaae9ce761d34669420c2a801b2780", size = 7087215, upload-time = "2026-04-01T14:43:26.758Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/f8/2f6825e441d5b1959d2ca5adec984210f1ec086435b0ed5f52c19b3b8a6e/pillow-12.2.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:01afa7cf67f74f09523699b4e88c73fb55c13346d212a59a2db1f86b0a63e8c5", size = 6509783, upload-time = "2026-04-01T14:43:29.56Z" },
+ { url = "https://files.pythonhosted.org/packages/67/f9/029a27095ad20f854f9dba026b3ea6428548316e057e6fc3545409e86651/pillow-12.2.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:fc3d34d4a8fbec3e88a79b92e5465e0f9b842b628675850d860b8bd300b159f5", size = 7212112, upload-time = "2026-04-01T14:43:32.091Z" },
+ { url = "https://files.pythonhosted.org/packages/be/42/025cfe05d1be22dbfdb4f264fe9de1ccda83f66e4fc3aac94748e784af04/pillow-12.2.0-cp312-cp312-win32.whl", hash = "sha256:58f62cc0f00fd29e64b29f4fd923ffdb3859c9f9e6105bfc37ba1d08994e8940", size = 6378489, upload-time = "2026-04-01T14:43:34.601Z" },
+ { url = "https://files.pythonhosted.org/packages/5d/7b/25a221d2c761c6a8ae21bfa3874988ff2583e19cf8a27bf2fee358df7942/pillow-12.2.0-cp312-cp312-win_amd64.whl", hash = "sha256:7f84204dee22a783350679a0333981df803dac21a0190d706a50475e361c93f5", size = 7084129, upload-time = "2026-04-01T14:43:37.213Z" },
+ { url = "https://files.pythonhosted.org/packages/10/e1/542a474affab20fd4a0f1836cb234e8493519da6b76899e30bcc5d990b8b/pillow-12.2.0-cp312-cp312-win_arm64.whl", hash = "sha256:af73337013e0b3b46f175e79492d96845b16126ddf79c438d7ea7ff27783a414", size = 2463612, upload-time = "2026-04-01T14:43:39.421Z" },
+ { url = "https://files.pythonhosted.org/packages/4a/01/53d10cf0dbad820a8db274d259a37ba50b88b24768ddccec07355382d5ad/pillow-12.2.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:8297651f5b5679c19968abefd6bb84d95fe30ef712eb1b2d9b2d31ca61267f4c", size = 4100837, upload-time = "2026-04-01T14:43:41.506Z" },
+ { url = "https://files.pythonhosted.org/packages/0f/98/f3a6657ecb698c937f6c76ee564882945f29b79bad496abcba0e84659ec5/pillow-12.2.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:50d8520da2a6ce0af445fa6d648c4273c3eeefbc32d7ce049f22e8b5c3daecc2", size = 4176528, upload-time = "2026-04-01T14:43:43.773Z" },
+ { url = "https://files.pythonhosted.org/packages/69/bc/8986948f05e3ea490b8442ea1c1d4d990b24a7e43d8a51b2c7d8b1dced36/pillow-12.2.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:766cef22385fa1091258ad7e6216792b156dc16d8d3fa607e7545b2b72061f1c", size = 3640401, upload-time = "2026-04-01T14:43:45.87Z" },
+ { url = "https://files.pythonhosted.org/packages/34/46/6c717baadcd62bc8ed51d238d521ab651eaa74838291bda1f86fe1f864c9/pillow-12.2.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:5d2fd0fa6b5d9d1de415060363433f28da8b1526c1c129020435e186794b3795", size = 5308094, upload-time = "2026-04-01T14:43:48.438Z" },
+ { url = "https://files.pythonhosted.org/packages/71/43/905a14a8b17fdb1ccb58d282454490662d2cb89a6bfec26af6d3520da5ec/pillow-12.2.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:56b25336f502b6ed02e889f4ece894a72612fe885889a6e8c4c80239ff6e5f5f", size = 4695402, upload-time = "2026-04-01T14:43:51.292Z" },
+ { url = "https://files.pythonhosted.org/packages/73/dd/42107efcb777b16fa0393317eac58f5b5cf30e8392e266e76e51cff28c3d/pillow-12.2.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:f1c943e96e85df3d3478f7b691f229887e143f81fedab9b20205349ab04d73ed", size = 6280005, upload-time = "2026-04-01T14:43:54.242Z" },
+ { url = "https://files.pythonhosted.org/packages/a8/68/b93e09e5e8549019e61acf49f65b1a8530765a7f812c77a7461bca7e4494/pillow-12.2.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:03f6fab9219220f041c74aeaa2939ff0062bd5c364ba9ce037197f4c6d498cd9", size = 8090669, upload-time = "2026-04-01T14:43:57.335Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/6e/3ccb54ce8ec4ddd1accd2d89004308b7b0b21c4ac3d20fa70af4760a4330/pillow-12.2.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5cdfebd752ec52bf5bb4e35d9c64b40826bc5b40a13df7c3cda20a2c03a0f5ed", size = 6395194, upload-time = "2026-04-01T14:43:59.864Z" },
+ { url = "https://files.pythonhosted.org/packages/67/ee/21d4e8536afd1a328f01b359b4d3997b291ffd35a237c877b331c1c3b71c/pillow-12.2.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:eedf4b74eda2b5a4b2b2fb4c006d6295df3bf29e459e198c90ea48e130dc75c3", size = 7082423, upload-time = "2026-04-01T14:44:02.74Z" },
+ { url = "https://files.pythonhosted.org/packages/78/5f/e9f86ab0146464e8c133fe85df987ed9e77e08b29d8d35f9f9f4d6f917ba/pillow-12.2.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:00a2865911330191c0b818c59103b58a5e697cae67042366970a6b6f1b20b7f9", size = 6505667, upload-time = "2026-04-01T14:44:05.381Z" },
+ { url = "https://files.pythonhosted.org/packages/ed/1e/409007f56a2fdce61584fd3acbc2bbc259857d555196cedcadc68c015c82/pillow-12.2.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:1e1757442ed87f4912397c6d35a0db6a7b52592156014706f17658ff58bbf795", size = 7208580, upload-time = "2026-04-01T14:44:08.39Z" },
+ { url = "https://files.pythonhosted.org/packages/23/c4/7349421080b12fb35414607b8871e9534546c128a11965fd4a7002ccfbee/pillow-12.2.0-cp313-cp313-win32.whl", hash = "sha256:144748b3af2d1b358d41286056d0003f47cb339b8c43a9ea42f5fea4d8c66b6e", size = 6375896, upload-time = "2026-04-01T14:44:11.197Z" },
+ { url = "https://files.pythonhosted.org/packages/3f/82/8a3739a5e470b3c6cbb1d21d315800d8e16bff503d1f16b03a4ec3212786/pillow-12.2.0-cp313-cp313-win_amd64.whl", hash = "sha256:390ede346628ccc626e5730107cde16c42d3836b89662a115a921f28440e6a3b", size = 7081266, upload-time = "2026-04-01T14:44:13.947Z" },
+ { url = "https://files.pythonhosted.org/packages/c3/25/f968f618a062574294592f668218f8af564830ccebdd1fa6200f598e65c5/pillow-12.2.0-cp313-cp313-win_arm64.whl", hash = "sha256:8023abc91fba39036dbce14a7d6535632f99c0b857807cbbbf21ecc9f4717f06", size = 2463508, upload-time = "2026-04-01T14:44:16.312Z" },
+ { url = "https://files.pythonhosted.org/packages/4d/a4/b342930964e3cb4dce5038ae34b0eab4653334995336cd486c5a8c25a00c/pillow-12.2.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:042db20a421b9bafecc4b84a8b6e444686bd9d836c7fd24542db3e7df7baad9b", size = 5309927, upload-time = "2026-04-01T14:44:18.89Z" },
+ { url = "https://files.pythonhosted.org/packages/9f/de/23198e0a65a9cf06123f5435a5d95cea62a635697f8f03d134d3f3a96151/pillow-12.2.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:dd025009355c926a84a612fecf58bb315a3f6814b17ead51a8e48d3823d9087f", size = 4698624, upload-time = "2026-04-01T14:44:21.115Z" },
+ { url = "https://files.pythonhosted.org/packages/01/a6/1265e977f17d93ea37aa28aa81bad4fa597933879fac2520d24e021c8da3/pillow-12.2.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:88ddbc66737e277852913bd1e07c150cc7bb124539f94c4e2df5344494e0a612", size = 6321252, upload-time = "2026-04-01T14:44:23.663Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/83/5982eb4a285967baa70340320be9f88e57665a387e3a53a7f0db8231a0cd/pillow-12.2.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d362d1878f00c142b7e1a16e6e5e780f02be8195123f164edf7eddd911eefe7c", size = 8126550, upload-time = "2026-04-01T14:44:26.772Z" },
+ { url = "https://files.pythonhosted.org/packages/4e/48/6ffc514adce69f6050d0753b1a18fd920fce8cac87620d5a31231b04bfc5/pillow-12.2.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2c727a6d53cb0018aadd8018c2b938376af27914a68a492f59dfcaca650d5eea", size = 6433114, upload-time = "2026-04-01T14:44:29.615Z" },
+ { url = "https://files.pythonhosted.org/packages/36/a3/f9a77144231fb8d40ee27107b4463e205fa4677e2ca2548e14da5cf18dce/pillow-12.2.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:efd8c21c98c5cc60653bcb311bef2ce0401642b7ce9d09e03a7da87c878289d4", size = 7115667, upload-time = "2026-04-01T14:44:32.773Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/fc/ac4ee3041e7d5a565e1c4fd72a113f03b6394cc72ab7089d27608f8aaccb/pillow-12.2.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9f08483a632889536b8139663db60f6724bfcb443c96f1b18855860d7d5c0fd4", size = 6538966, upload-time = "2026-04-01T14:44:35.252Z" },
+ { url = "https://files.pythonhosted.org/packages/c0/a8/27fb307055087f3668f6d0a8ccb636e7431d56ed0750e07a60547b1e083e/pillow-12.2.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:dac8d77255a37e81a2efcbd1fc05f1c15ee82200e6c240d7e127e25e365c39ea", size = 7238241, upload-time = "2026-04-01T14:44:37.875Z" },
+ { url = "https://files.pythonhosted.org/packages/ad/4b/926ab182c07fccae9fcb120043464e1ff1564775ec8864f21a0ebce6ac25/pillow-12.2.0-cp313-cp313t-win32.whl", hash = "sha256:ee3120ae9dff32f121610bb08e4313be87e03efeadfc6c0d18f89127e24d0c24", size = 6379592, upload-time = "2026-04-01T14:44:40.336Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/c4/f9e476451a098181b30050cc4c9a3556b64c02cf6497ea421ac047e89e4b/pillow-12.2.0-cp313-cp313t-win_amd64.whl", hash = "sha256:325ca0528c6788d2a6c3d40e3568639398137346c3d6e66bb61db96b96511c98", size = 7085542, upload-time = "2026-04-01T14:44:43.251Z" },
+ { url = "https://files.pythonhosted.org/packages/00/a4/285f12aeacbe2d6dc36c407dfbbe9e96d4a80b0fb710a337f6d2ad978c75/pillow-12.2.0-cp313-cp313t-win_arm64.whl", hash = "sha256:2e5a76d03a6c6dcef67edabda7a52494afa4035021a79c8558e14af25313d453", size = 2465765, upload-time = "2026-04-01T14:44:45.996Z" },
+ { url = "https://files.pythonhosted.org/packages/bf/98/4595daa2365416a86cb0d495248a393dfc84e96d62ad080c8546256cb9c0/pillow-12.2.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:3adc9215e8be0448ed6e814966ecf3d9952f0ea40eb14e89a102b87f450660d8", size = 4100848, upload-time = "2026-04-01T14:44:48.48Z" },
+ { url = "https://files.pythonhosted.org/packages/0b/79/40184d464cf89f6663e18dfcf7ca21aae2491fff1a16127681bf1fa9b8cf/pillow-12.2.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:6a9adfc6d24b10f89588096364cc726174118c62130c817c2837c60cf08a392b", size = 4176515, upload-time = "2026-04-01T14:44:51.353Z" },
+ { url = "https://files.pythonhosted.org/packages/b0/63/703f86fd4c422a9cf722833670f4f71418fb116b2853ff7da722ea43f184/pillow-12.2.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:6a6e67ea2e6feda684ed370f9a1c52e7a243631c025ba42149a2cc5934dec295", size = 3640159, upload-time = "2026-04-01T14:44:53.588Z" },
+ { url = "https://files.pythonhosted.org/packages/71/e0/fb22f797187d0be2270f83500aab851536101b254bfa1eae10795709d283/pillow-12.2.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:2bb4a8d594eacdfc59d9e5ad972aa8afdd48d584ffd5f13a937a664c3e7db0ed", size = 5312185, upload-time = "2026-04-01T14:44:56.039Z" },
+ { url = "https://files.pythonhosted.org/packages/ba/8c/1a9e46228571de18f8e28f16fabdfc20212a5d019f3e3303452b3f0a580d/pillow-12.2.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:80b2da48193b2f33ed0c32c38140f9d3186583ce7d516526d462645fd98660ae", size = 4695386, upload-time = "2026-04-01T14:44:58.663Z" },
+ { url = "https://files.pythonhosted.org/packages/70/62/98f6b7f0c88b9addd0e87c217ded307b36be024d4ff8869a812b241d1345/pillow-12.2.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:22db17c68434de69d8ecfc2fe821569195c0c373b25cccb9cbdacf2c6e53c601", size = 6280384, upload-time = "2026-04-01T14:45:01.5Z" },
+ { url = "https://files.pythonhosted.org/packages/5e/03/688747d2e91cfbe0e64f316cd2e8005698f76ada3130d0194664174fa5de/pillow-12.2.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7b14cc0106cd9aecda615dd6903840a058b4700fcb817687d0ee4fc8b6e389be", size = 8091599, upload-time = "2026-04-01T14:45:04.5Z" },
+ { url = "https://files.pythonhosted.org/packages/f6/35/577e22b936fcdd66537329b33af0b4ccfefaeabd8aec04b266528cddb33c/pillow-12.2.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8cbeb542b2ebc6fcdacabf8aca8c1a97c9b3ad3927d46b8723f9d4f033288a0f", size = 6396021, upload-time = "2026-04-01T14:45:07.117Z" },
+ { url = "https://files.pythonhosted.org/packages/11/8d/d2532ad2a603ca2b93ad9f5135732124e57811d0168155852f37fbce2458/pillow-12.2.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4bfd07bc812fbd20395212969e41931001fd59eb55a60658b0e5710872e95286", size = 7083360, upload-time = "2026-04-01T14:45:09.763Z" },
+ { url = "https://files.pythonhosted.org/packages/5e/26/d325f9f56c7e039034897e7380e9cc202b1e368bfd04d4cbe6a441f02885/pillow-12.2.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:9aba9a17b623ef750a4d11b742cbafffeb48a869821252b30ee21b5e91392c50", size = 6507628, upload-time = "2026-04-01T14:45:12.378Z" },
+ { url = "https://files.pythonhosted.org/packages/5f/f7/769d5632ffb0988f1c5e7660b3e731e30f7f8ec4318e94d0a5d674eb65a4/pillow-12.2.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:deede7c263feb25dba4e82ea23058a235dcc2fe1f6021025dc71f2b618e26104", size = 7209321, upload-time = "2026-04-01T14:45:15.122Z" },
+ { url = "https://files.pythonhosted.org/packages/6a/7a/c253e3c645cd47f1aceea6a8bacdba9991bf45bb7dfe927f7c893e89c93c/pillow-12.2.0-cp314-cp314-win32.whl", hash = "sha256:632ff19b2778e43162304d50da0181ce24ac5bb8180122cbe1bf4673428328c7", size = 6479723, upload-time = "2026-04-01T14:45:17.797Z" },
+ { url = "https://files.pythonhosted.org/packages/cd/8b/601e6566b957ca50e28725cb6c355c59c2c8609751efbecd980db44e0349/pillow-12.2.0-cp314-cp314-win_amd64.whl", hash = "sha256:4e6c62e9d237e9b65fac06857d511e90d8461a32adcc1b9065ea0c0fa3a28150", size = 7217400, upload-time = "2026-04-01T14:45:20.529Z" },
+ { url = "https://files.pythonhosted.org/packages/d6/94/220e46c73065c3e2951bb91c11a1fb636c8c9ad427ac3ce7d7f3359b9b2f/pillow-12.2.0-cp314-cp314-win_arm64.whl", hash = "sha256:b1c1fbd8a5a1af3412a0810d060a78b5136ec0836c8a4ef9aa11807f2a22f4e1", size = 2554835, upload-time = "2026-04-01T14:45:23.162Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/ab/1b426a3974cb0e7da5c29ccff4807871d48110933a57207b5a676cccc155/pillow-12.2.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:57850958fe9c751670e49b2cecf6294acc99e562531f4bd317fa5ddee2068463", size = 5314225, upload-time = "2026-04-01T14:45:25.637Z" },
+ { url = "https://files.pythonhosted.org/packages/19/1e/dce46f371be2438eecfee2a1960ee2a243bbe5e961890146d2dee1ff0f12/pillow-12.2.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:d5d38f1411c0ed9f97bcb49b7bd59b6b7c314e0e27420e34d99d844b9ce3b6f3", size = 4698541, upload-time = "2026-04-01T14:45:28.355Z" },
+ { url = "https://files.pythonhosted.org/packages/55/c3/7fbecf70adb3a0c33b77a300dc52e424dc22ad8cdc06557a2e49523b703d/pillow-12.2.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:5c0a9f29ca8e79f09de89293f82fc9b0270bb4af1d58bc98f540cc4aedf03166", size = 6322251, upload-time = "2026-04-01T14:45:30.924Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/3c/7fbc17cfb7e4fe0ef1642e0abc17fc6c94c9f7a16be41498e12e2ba60408/pillow-12.2.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1610dd6c61621ae1cf811bef44d77e149ce3f7b95afe66a4512f8c59f25d9ebe", size = 8127807, upload-time = "2026-04-01T14:45:33.908Z" },
+ { url = "https://files.pythonhosted.org/packages/ff/c3/a8ae14d6defd2e448493ff512fae903b1e9bd40b72efb6ec55ce0048c8ce/pillow-12.2.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0a34329707af4f73cf1782a36cd2289c0368880654a2c11f027bcee9052d35dd", size = 6433935, upload-time = "2026-04-01T14:45:36.623Z" },
+ { url = "https://files.pythonhosted.org/packages/6e/32/2880fb3a074847ac159d8f902cb43278a61e85f681661e7419e6596803ed/pillow-12.2.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8e9c4f5b3c546fa3458a29ab22646c1c6c787ea8f5ef51300e5a60300736905e", size = 7116720, upload-time = "2026-04-01T14:45:39.258Z" },
+ { url = "https://files.pythonhosted.org/packages/46/87/495cc9c30e0129501643f24d320076f4cc54f718341df18cc70ec94c44e1/pillow-12.2.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:fb043ee2f06b41473269765c2feae53fc2e2fbf96e5e22ca94fb5ad677856f06", size = 6540498, upload-time = "2026-04-01T14:45:41.879Z" },
+ { url = "https://files.pythonhosted.org/packages/18/53/773f5edca692009d883a72211b60fdaf8871cbef075eaa9d577f0a2f989e/pillow-12.2.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:f278f034eb75b4e8a13a54a876cc4a5ab39173d2cdd93a638e1b467fc545ac43", size = 7239413, upload-time = "2026-04-01T14:45:44.705Z" },
+ { url = "https://files.pythonhosted.org/packages/c9/e4/4b64a97d71b2a83158134abbb2f5bd3f8a2ea691361282f010998f339ec7/pillow-12.2.0-cp314-cp314t-win32.whl", hash = "sha256:6bb77b2dcb06b20f9f4b4a8454caa581cd4dd0643a08bacf821216a16d9c8354", size = 6482084, upload-time = "2026-04-01T14:45:47.568Z" },
+ { url = "https://files.pythonhosted.org/packages/ba/13/306d275efd3a3453f72114b7431c877d10b1154014c1ebbedd067770d629/pillow-12.2.0-cp314-cp314t-win_amd64.whl", hash = "sha256:6562ace0d3fb5f20ed7290f1f929cae41b25ae29528f2af1722966a0a02e2aa1", size = 7225152, upload-time = "2026-04-01T14:45:50.032Z" },
+ { url = "https://files.pythonhosted.org/packages/ff/6e/cf826fae916b8658848d7b9f38d88da6396895c676e8086fc0988073aaf8/pillow-12.2.0-cp314-cp314t-win_arm64.whl", hash = "sha256:aa88ccfe4e32d362816319ed727a004423aab09c5cea43c01a4b435643fa34eb", size = 2556579, upload-time = "2026-04-01T14:45:52.529Z" },
+]
+
+[[package]]
+name = "platformdirs"
+version = "4.10.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/d7/47/e4501f49c178ae1d9f4a75073fda4204f52647993f075a9db4d14930e0c5/platformdirs-4.10.0.tar.gz", hash = "sha256:31e761a6a0ca04faf7353ea759bdba55652be214725111e5aac52dfa29d4bef7", size = 31224, upload-time = "2026-05-28T03:32:53.587Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/81/e6/cd9575ac904136b3cbf7aa7ee819ef86eedb7274e46f230e94ea4342e729/platformdirs-4.10.0-py3-none-any.whl", hash = "sha256:fb516cdb12eb0d857d0cd85a7c57cea4d060bee4578d6cf5a14dfdf8cbf8784a", size = 22743, upload-time = "2026-05-28T03:32:52.175Z" },
+]
+
+[[package]]
+name = "pluggy"
+version = "1.6.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
+]
+
+[[package]]
+name = "prompt-toolkit"
+version = "3.0.52"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "wcwidth" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/a1/96/06e01a7b38dce6fe1db213e061a4602dd6032a8a97ef6c1a862537732421/prompt_toolkit-3.0.52.tar.gz", hash = "sha256:28cde192929c8e7321de85de1ddbe736f1375148b02f2e17edd840042b1be855", size = 434198, upload-time = "2025-08-27T15:24:02.057Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/84/03/0d3ce49e2505ae70cf43bc5bb3033955d2fc9f932163e84dc0779cc47f48/prompt_toolkit-3.0.52-py3-none-any.whl", hash = "sha256:9aac639a3bbd33284347de5ad8d68ecc044b91a762dc39b7c21095fcd6a19955", size = 391431, upload-time = "2025-08-27T15:23:59.498Z" },
+]
+
+[[package]]
+name = "propcache"
+version = "0.5.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ec/44/c87281c333769159c50594f22610f77398a47ccbfbbf23074e744e86f87c/propcache-0.5.2.tar.gz", hash = "sha256:01c4fc7480cd0598bb4b57022df55b9ca296da7fc5a8760bd8451a7e63a7d427", size = 50208, upload-time = "2026-05-08T21:02:12.199Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/4a/cb/e27bc2b2737a0bb49962b275efa051e8f1c35a936df7d5139b6b658b7dc9/propcache-0.5.2-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:806719138ecd720339a12410fb9614ac9b2b2d3a5fdf8235d56981c36f4039ba", size = 95887, upload-time = "2026-05-08T21:00:11.277Z" },
+ { url = "https://files.pythonhosted.org/packages/e6/13/b8ae04c59392f8d11c6cd9fb4011d1dc7c86b81225c770280300e259ffe1/propcache-0.5.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:db2b80ea58eab4f86b2beec3cc8b39e8ff9276ac20e96b7cce43c8ae84cd6b5a", size = 54654, upload-time = "2026-05-08T21:00:12.604Z" },
+ { url = "https://files.pythonhosted.org/packages/2c/7d/49777a3e20b55863d4794384a38acd460c04157b0a00f8602b0d508b8431/propcache-0.5.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:e5cbfac9f61484f7e9f3597775500cd3ebe8274e9b050c38f9525c77c97520bf", size = 55190, upload-time = "2026-05-08T21:00:13.935Z" },
+ { url = "https://files.pythonhosted.org/packages/44/c7/085d0cd63062e84044e3f05797749c3f8e3938ff3aeb0eb2f69d43fafc91/propcache-0.5.2-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5dbc581d2814337da56222fab8dc5f161cd798a434e49bac27930aaef798e144", size = 59995, upload-time = "2026-05-08T21:00:15.526Z" },
+ { url = "https://files.pythonhosted.org/packages/9c/42/32cf8e3009e92b2645cf1e944f701e8ea4e924dffde1ee26db860bcbf7e4/propcache-0.5.2-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:857187f381f88c8e2fa2fe56ab94879d011b883d5a2ee5a1b60a8cd2a06846d9", size = 63422, upload-time = "2026-05-08T21:00:16.824Z" },
+ { url = "https://files.pythonhosted.org/packages/9e/1b/f112433f99fc979431b87a39ef169e3f8df070d99a72792c56d6937ac48b/propcache-0.5.2-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:178b4a2cdaac1818e2bf1c5a99b94383fa73ea5382e032a48dec07dc5668dc42", size = 64342, upload-time = "2026-05-08T21:00:18.362Z" },
+ { url = "https://files.pythonhosted.org/packages/14/15/5574111ae50dd6e879456888c0eadd4c5a869959775854e18e18a6b345f3/propcache-0.5.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6f328175a2cde1f0ff2c4ed8ce968b9dcfb55f3a7153f39e2957ed994da13476", size = 61639, upload-time = "2026-05-08T21:00:19.692Z" },
+ { url = "https://files.pythonhosted.org/packages/cc/da/4d775080b1490c0ae604acda868bd71aabe3a89ed16f2aa4339eb8a283e7/propcache-0.5.2-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:5671d09a36b06d0fd4a3da0fccbcae360e9b1570924171a15e9e0997f0249fba", size = 61588, upload-time = "2026-05-08T21:00:21.155Z" },
+ { url = "https://files.pythonhosted.org/packages/04/ac/f076982cbe2195ee9cf32de5a1e46951d9fb399fc207f390562dd0fd8fb2/propcache-0.5.2-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:80168e2ebe4d3ec6599d10ad8f520304ae1cad9b6c5a95372aef1b66b7bfb53a", size = 60029, upload-time = "2026-05-08T21:00:22.713Z" },
+ { url = "https://files.pythonhosted.org/packages/70/60/189be62e0dd898dce3b331e1b8c7a543cd3a405ac0c81fe8ee8a9d5d77e1/propcache-0.5.2-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:45f11346f884bc47444f6e6647131055844134c3175b629f84952e2b5cd62b64", size = 56774, upload-time = "2026-05-08T21:00:24.001Z" },
+ { url = "https://files.pythonhosted.org/packages/ea/9e/93377b9c7939c1ffae98f878dee955efadfd638078bc86dbc21f9d52f651/propcache-0.5.2-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:8e778ebd44ef4f66ed60a0416b06b489687db264a9c0b3620362f26489492913", size = 63532, upload-time = "2026-05-08T21:00:25.545Z" },
+ { url = "https://files.pythonhosted.org/packages/14/f9/590ef6cfb9b8028d516d287812ece32bb0bc5f11fbb9c8bf6b2e6313fec8/propcache-0.5.2-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:c0cb9ed24c8964e172768d455a38254c2dd8a552905729ce006cad3d3dda59b1", size = 61592, upload-time = "2026-05-08T21:00:27.186Z" },
+ { url = "https://files.pythonhosted.org/packages/b4/5e/70958b3034c297a630bba2f17ca7abc2d5f39a803ad7e370ab79d1ecd022/propcache-0.5.2-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:1d1ad32d9d4355e2be65574fd0bfd3677e7066b009cd5b9b2dee8aa6a6393b33", size = 64788, upload-time = "2026-05-08T21:00:28.8Z" },
+ { url = "https://files.pythonhosted.org/packages/12/fd/77fe5936d8c3086ca9048f7f415f122ed82e53884a9ec193646b42deef06/propcache-0.5.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:c80f4ba3e8f00189165999a742ee526ebeccedf6c3f7beb0c7df821e9772435a", size = 62514, upload-time = "2026-05-08T21:00:30.098Z" },
+ { url = "https://files.pythonhosted.org/packages/cf/74/66bd798b5b3be70aa1b391f5cc9d6a0a5532d7fd3b19ec0b213e72e6ad9d/propcache-0.5.2-cp312-cp312-win32.whl", hash = "sha256:8c7972d8f193740d9175f0998ab38717e6cd322d5935c5b0fef8c0d323fd9031", size = 39018, upload-time = "2026-05-08T21:00:31.622Z" },
+ { url = "https://files.pythonhosted.org/packages/61/7c/5c0d34aa3024694d6dcb9271cdbdd08c4e47c1c0ad95ec7e7bc74cdea145/propcache-0.5.2-cp312-cp312-win_amd64.whl", hash = "sha256:d9ee8826a7d47863a08ac44e1a5f611a462eefc3a194b492da242128bec75b42", size = 42322, upload-time = "2026-05-08T21:00:32.918Z" },
+ { url = "https://files.pythonhosted.org/packages/4d/91/875812f1a3feb20ceba818ef39fbe4d92f1081e04ac815c822496d0d038b/propcache-0.5.2-cp312-cp312-win_arm64.whl", hash = "sha256:2800a4a8ead6b28cccd1ec54b59346f0def7922ee1c7598e8499c733cfbb7c84", size = 38172, upload-time = "2026-05-08T21:00:35.124Z" },
+ { url = "https://files.pythonhosted.org/packages/c5/09/f049e45385503fe67db75a6b6186a7b9f0c3930366dc960522c312a825b1/propcache-0.5.2-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:099aaf4b4d1a02265b92a977edf00b5c4f63b3b17ac6de39b0d637c9cac0188a", size = 94457, upload-time = "2026-05-08T21:00:36.355Z" },
+ { url = "https://files.pythonhosted.org/packages/6b/65/83d1d05655baf63113731bd5a1008435e14f8d1e5a06cbe4ec5b23ad7a31/propcache-0.5.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:68ce1c44c7a813a7f71ea04315a8c7b330b63db99d059a797a4651bb6f69f117", size = 53835, upload-time = "2026-05-08T21:00:38.072Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/12/a6ba6482bb5ea3260c000c9b20881c95fa11c6b30173715668259f844ed7/propcache-0.5.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:fc299c129490f55f254cd90be0deca4764e36e9a7c08b4aa588479a3bbed3098", size = 54545, upload-time = "2026-05-08T21:00:39.319Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/19/7fa086f5764c59ec8a8e157cd93aa8497acc00aba9dcdec56bfffb32602d/propcache-0.5.2-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a6ae2198be502c10f09b2516e7b5d019816924bc3183a43ce792a7bd6625e6f4", size = 59886, upload-time = "2026-05-08T21:00:40.621Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/e4/5d7663dc8235956c8f5281698a3af1d351d8820341ddd890f59d9a9127f2/propcache-0.5.2-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:6041d31504dc1779d700e1edcfb08eea334b357620b06681a4eabb57a74e574e", size = 63261, upload-time = "2026-05-08T21:00:41.775Z" },
+ { url = "https://files.pythonhosted.org/packages/4a/4a/15a03adee24d6350da4292caeac44c34c033d2afe5e87eb370f38854560f/propcache-0.5.2-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f7eabc04151c78a9f4d5bbb5f1faf571e4defeb4b585e0fe95b60ff2dbe4d3d7", size = 64184, upload-time = "2026-05-08T21:00:43.018Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/c6/979176efdaa3d239e36d503d5af63a0a773b36662ed8f52e5b6a6d9fd40e/propcache-0.5.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4db0ba63d693afd40d249bd93f842b5f144f8fcbb83de05660373bcf30517b1d", size = 61534, upload-time = "2026-05-08T21:00:44.507Z" },
+ { url = "https://files.pythonhosted.org/packages/c8/22/63e8cd1bae4c2d2be6493b6b7d10566ddafad88137cfbc99964a1119853c/propcache-0.5.2-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:1dbcf7675229b35d31abb6547d8ebc8c27a830ac3f9a794edff6254873ec7c0a", size = 61500, upload-time = "2026-05-08T21:00:45.796Z" },
+ { url = "https://files.pythonhosted.org/packages/60/5a/28e5d9acbac1cc9ccb67045e8c1b943aa8d79fdf39c93bd73cacd68008ea/propcache-0.5.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d310c013aad2c72f1c3f2f8dd3279d460a858c551f97aeb8c63e4693cca7b4d2", size = 59994, upload-time = "2026-05-08T21:00:47.093Z" },
+ { url = "https://files.pythonhosted.org/packages/f3/40/db650677f554a95b9c01a7c9d93d629e93a15562f5deb4573c9ee136fed2/propcache-0.5.2-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:06187263ddad280d05b4d8a8b3bb7d164cbebd469236544a42e6d9b28ac6a4fa", size = 56884, upload-time = "2026-05-08T21:00:48.376Z" },
+ { url = "https://files.pythonhosted.org/packages/80/45/70b39b89516ff8b96bf732fa6fded8cef20f293cb1508690101c3c07ec51/propcache-0.5.2-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:3115559b8effafd63b142ea5ed53d63a16ea6469cbc63dce4ee194b42db5d853", size = 63464, upload-time = "2026-05-08T21:00:49.954Z" },
+ { url = "https://files.pythonhosted.org/packages/f9/e2/fa59d3a89eac5534293124af4f1d0d0ada091ce4a0ab4610ce03fd2bdd8d/propcache-0.5.2-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:c60462af8e6dc30c35407c7237ea908d777b22862bbee27bc4699c0d8bcdc45a", size = 61588, upload-time = "2026-05-08T21:00:51.281Z" },
+ { url = "https://files.pythonhosted.org/packages/0b/97/efb547a55c4bc7381cfb202d6a2239ac621045277bc1ea5dfd3a7f0516c0/propcache-0.5.2-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:40314bca9ac559716fe374094fc81c11dcc34b64fd6c585360f5775690505704", size = 64667, upload-time = "2026-05-08T21:00:52.602Z" },
+ { url = "https://files.pythonhosted.org/packages/92/56/f5c7d9b4b7595d5127da38974d791b2153f3d1eae6c674af3583ace92ad3/propcache-0.5.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:cfa21e036ce1e1db2be04ba3b85d2df1bb1702fa01932d984c5464c665228ff4", size = 62463, upload-time = "2026-05-08T21:00:54.303Z" },
+ { url = "https://files.pythonhosted.org/packages/bd/3b/484a3a65fc9f9f60c41dcd17b428bace5389544e2c680994534a20755066/propcache-0.5.2-cp313-cp313-win32.whl", hash = "sha256:f156a3529f38063b6dbaf356e15602a7f95f8055b1295a438433a6386f10463d", size = 38621, upload-time = "2026-05-08T21:00:55.808Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/fd/3f0f10dba4dabad3bf53102be007abf55481067952bde0fdddff439e7c61/propcache-0.5.2-cp313-cp313-win_amd64.whl", hash = "sha256:dfed59d0a5aeb01e242e66ff0300bc4a265a7c05f612d30016f0b60b1017d757", size = 41649, upload-time = "2026-05-08T21:00:57.061Z" },
+ { url = "https://files.pythonhosted.org/packages/90/ec/6ce619cc32bb500a482f811f9cd509368b4e58e638d13f2c68f370d6b475/propcache-0.5.2-cp313-cp313-win_arm64.whl", hash = "sha256:ba338430e87ceb9c8f0cf754de38a9860560261e56c00376debd628698a7364f", size = 37636, upload-time = "2026-05-08T21:00:58.646Z" },
+ { url = "https://files.pythonhosted.org/packages/1b/82/c1d268bbbf2ef981c5bf0fbbe746db617c66e3bcefe431a1aa8943fbe23a/propcache-0.5.2-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:a592f5f3da71c8691c788c13cb6734b6d17663d2e1cb8caddf0673d01ef8847d", size = 98872, upload-time = "2026-05-08T21:00:59.889Z" },
+ { url = "https://files.pythonhosted.org/packages/f4/d4/52c871e73e864e6b34c0e2d58ac1ec5ccd149497ddc7ad2137ae98323a35/propcache-0.5.2-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:6a997d0489e9668a384fcfd5061b857aa5361de73191cac204d04b889cfbbafa", size = 56257, upload-time = "2026-05-08T21:01:01.195Z" },
+ { url = "https://files.pythonhosted.org/packages/67/f0/9b90ca2a210b3d09bcfcd96ecd0f55545c091535abce2a45de2775cfd357/propcache-0.5.2-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:10734b5484ea113152ee25a91dccedf81631791805d2c9ccb054958e51842c94", size = 56696, upload-time = "2026-05-08T21:01:02.941Z" },
+ { url = "https://files.pythonhosted.org/packages/9d/0e/6e9d4ba07c8e56e21ddec1e75f12148142b21ca83a51871babce095334f4/propcache-0.5.2-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:cafca7e56c12bb02ae16d283742bef25a61122e9dab2b5b3f2ccbe589ce32164", size = 62378, upload-time = "2026-05-08T21:01:04.475Z" },
+ { url = "https://files.pythonhosted.org/packages/65/19/c10badaa463dde8a27ce884f8ee2ec37e6035b7c9f5ff0c8f74f06f08dac/propcache-0.5.2-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f064f8d2b59177878b7615df1735cd8fe3462ed6be8c7b217d17a276489c2b7f", size = 65283, upload-time = "2026-05-08T21:01:05.959Z" },
+ { url = "https://files.pythonhosted.org/packages/b0/b6/93bea99ca80e19cef6512a8580e5b7857bbe09422d9daa7fd4ef5723306c/propcache-0.5.2-cp313-cp313t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f78abfa8dfc32376fd1aacf597b2f2fbbe0ea751419aee718af5d4f82537ef8c", size = 66616, upload-time = "2026-05-08T21:01:07.228Z" },
+ { url = "https://files.pythonhosted.org/packages/83/e4/5c7462e50625f051f37fb38b8224f7639f667184bbd34424ec83819bb1b7/propcache-0.5.2-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f7467da8a9822bf1a55336f877340c5bcbd3c482afc43a99771169f74a26dedc", size = 63773, upload-time = "2026-05-08T21:01:08.514Z" },
+ { url = "https://files.pythonhosted.org/packages/ca/b6/99238894047b13c823be25027e736626cd414a52a5e30d2c3347c2733529/propcache-0.5.2-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a6ddc6ac9e25de626c1f129c1b467d7ecd33ce2237d3fd0c4e429feef0a7ee1f", size = 63664, upload-time = "2026-05-08T21:01:09.874Z" },
+ { url = "https://files.pythonhosted.org/packages/85/1e/a3a1a63116a2b8edb415a8bb9a6f0c34bd03830b1e18e8ce2904e1dc1cf4/propcache-0.5.2-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:2f22cbbac9e26a8e864c0985ff1268d5d939d53d9d9411a9824279097e03a2cb", size = 62643, upload-time = "2026-05-08T21:01:11.132Z" },
+ { url = "https://files.pythonhosted.org/packages/e4/03/893cf147de2fc6543c5eaa07ad833170e7e2a2385725bbebe8c0503723bb/propcache-0.5.2-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:fc76378c62a0f04d0cd82fbb1a2cd2d7e28fcb40d5873f28a6c44e388aaa2751", size = 59595, upload-time = "2026-05-08T21:01:12.387Z" },
+ { url = "https://files.pythonhosted.org/packages/86/3b/04c1a2e12c57766568ba75ba72b3bf2042818d4c1425fab6fc07155c7cff/propcache-0.5.2-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:acd2c8edba48e31e58a363b8cf4e5c7db3b04b3f9e371f601df30d9b0d244836", size = 65711, upload-time = "2026-05-08T21:01:13.676Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/34/80f8d0099f8d6bacc4de1624c85672681c8cd1149ca2da0e38fd120b817f/propcache-0.5.2-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:452b5065457eb9991ec5eb38ff41d6cd4c991c9ac7c531c4d5849ae473a9a13f", size = 64247, upload-time = "2026-05-08T21:01:14.936Z" },
+ { url = "https://files.pythonhosted.org/packages/f3/1a/8b08f3a5f1037e9e370c55883ceeeee0f6dd0416fb2d2d67b8bfc91f2a79/propcache-0.5.2-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:3430bb2bfe1331885c427745a751e774ee679fd4344f80b97bf879815fe8fa55", size = 67102, upload-time = "2026-05-08T21:01:16.281Z" },
+ { url = "https://files.pythonhosted.org/packages/34/68/8bdb7bb7756d76e005490649d10e4a8369e610c74d619f71e1aedf889e9c/propcache-0.5.2-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:cef6cea3922890dd6c9654971001fa797b526c16ab5e1e46c05fd6f877be7568", size = 64964, upload-time = "2026-05-08T21:01:17.57Z" },
+ { url = "https://files.pythonhosted.org/packages/0a/aa/50fb0b5d3968b61a510926ff8b8465f1d6e976b3ab74496d7a4b9fc42515/propcache-0.5.2-cp313-cp313t-win32.whl", hash = "sha256:72d61e16dd78228b58c5d47be830ff3da7e5f139abdf0aef9d86cde1c5cf2191", size = 42546, upload-time = "2026-05-08T21:01:18.946Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/4c/0ddbae64321bd4a95bcbfc19307238016b5b1fee645c84626c8d539e5b74/propcache-0.5.2-cp313-cp313t-win_amd64.whl", hash = "sha256:0958834041a0166d343b8d2cedcd8bcbaeb4fdbe0cf08320c5379f143c3be6e7", size = 46330, upload-time = "2026-05-08T21:01:20.162Z" },
+ { url = "https://files.pythonhosted.org/packages/00/d9/9cddc8efb78d8af264c5ec9f6d10b62f57c515feda8d321595f56010fb23/propcache-0.5.2-cp313-cp313t-win_arm64.whl", hash = "sha256:6de8bd93ddde9b992cf2b2e0d796d501a19026b5b9fd87356d7d0779531a8d96", size = 40521, upload-time = "2026-05-08T21:01:21.399Z" },
+ { url = "https://files.pythonhosted.org/packages/e2/ea/23ee535d90ce8bcc465a3028eb3cc0ce3bd1005f4bb27710b30587de798d/propcache-0.5.2-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:46088abff4cba581dea21ae0467a480526cb25aa5f3c269e909f800328bc3999", size = 94662, upload-time = "2026-05-08T21:01:22.683Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/06/c5a52f419b5d8972f8d46a7577476090d8e3263ff589ce40b5ca4968d5be/propcache-0.5.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:fc88b26f08d634f7bc819a7852e5214f5802641ab8d9fd5326892292eee1993e", size = 53928, upload-time = "2026-05-08T21:01:23.986Z" },
+ { url = "https://files.pythonhosted.org/packages/63/b1/4260d67d6bd85e58a66b72d54ce15d5de789b6f3870cc6bedf8ff9667401/propcache-0.5.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:97797ebb098e670a2f92dd66f32897e30d7615b14e7f59711de23e30a9072539", size = 54650, upload-time = "2026-05-08T21:01:25.305Z" },
+ { url = "https://files.pythonhosted.org/packages/70/06/2f46c318e3307cd7a6a7481def374ce838c0fe20084b39dd54b0879d0e99/propcache-0.5.2-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ba57fffe4ac99c5d30076161b5866336d97600769bad35cc68f7774b15298a4e", size = 59912, upload-time = "2026-05-08T21:01:26.545Z" },
+ { url = "https://files.pythonhosted.org/packages/4c/29/fe1aebec2ce57ab985a9c382bded1124431f85078113aa222c5d278430d4/propcache-0.5.2-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:583c19759d9eec1e5b69e2fbef36a7d9c326041be9746cb822d335c8cedc2979", size = 63300, upload-time = "2026-05-08T21:01:27.937Z" },
+ { url = "https://files.pythonhosted.org/packages/b4/18/2334b26768b6c82be8c69e83671b767d5ef426aa09b0cba6c2ea47816774/propcache-0.5.2-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d0326e2e5e1f3163fa306c834e48e8d490e5fae607a097a40c0648109b47ba80", size = 64208, upload-time = "2026-05-08T21:01:29.484Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/76/7f1bfd6afff4c5e38e36a3c6d68eb5f4b7311ea80baf693db78d95b603c4/propcache-0.5.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e00820e192c8dbebcafb383ebbf99030895f09905e7a0eb2e0340a0bcc2bc825", size = 61633, upload-time = "2026-05-08T21:01:31.068Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/46/b3ff8aba2b4953a3e50de2cf72f1b5748b8eca93b15f3dc2c84339084c09/propcache-0.5.2-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c66afea89b1e43725731d2004732a046fe6fe955d51f952c3e95a7314a284a39", size = 61724, upload-time = "2026-05-08T21:01:32.374Z" },
+ { url = "https://files.pythonhosted.org/packages/c5/01/814cfcafbcff954f94c01cf30e097ddc88a076b5440fbcf4570753437d40/propcache-0.5.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:d4dc37dec6c6cdad0b57881a5658fd14fbf53e333b1a86cf86559f190e1d9ec4", size = 60069, upload-time = "2026-05-08T21:01:33.67Z" },
+ { url = "https://files.pythonhosted.org/packages/da/68/5c6f7622d510cc666a300687e06fd060c1a43361c0c9b20d284f06d8096a/propcache-0.5.2-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:5570dbcc97571c15f68068e529c92715a12f8d54030e272d264b377e22bd17a5", size = 57099, upload-time = "2026-05-08T21:01:34.915Z" },
+ { url = "https://files.pythonhosted.org/packages/55/27/9cb0b4c679124085327957d42521c99dba04c88c90c3e55a6f0b633ebccc/propcache-0.5.2-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:f814362777a9f841adddb200ecdf8f5cb1e5a3c4b7a86378edbd6ccb26edd702", size = 63391, upload-time = "2026-05-08T21:01:36.231Z" },
+ { url = "https://files.pythonhosted.org/packages/f0/9d/7258aaa5bdf60fc6f27591eef6fe52768cb0beda7140be477c8b12c9794a/propcache-0.5.2-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:196913dea116aeb5a2ba95af4ddcb7ea85559ae07d8eee8751688310d09168c3", size = 61626, upload-time = "2026-05-08T21:01:37.545Z" },
+ { url = "https://files.pythonhosted.org/packages/8e/0d/41c602003e8a9b16fe1e7eadf62c7bfba9d5474370b24200bf48b315f45f/propcache-0.5.2-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:6e7b8719005dd1175be4ab1cd25e9b98659a5e0347331506ec6760d2773a7fb5", size = 64781, upload-time = "2026-05-08T21:01:38.83Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/f3/38e66b1856e9bd079deea015bc4a55f7767c0e4db2f7dcf69e7e680ba4ce/propcache-0.5.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:51f96d685ab16e88cab128cd37a52c5da540809c8b879fa047731bfcb4ad35a4", size = 62570, upload-time = "2026-05-08T21:01:40.415Z" },
+ { url = "https://files.pythonhosted.org/packages/95/ca/bbfe9b910ce57dde8bb4876b4520fc02a4e89497c10de26be936758a3aaa/propcache-0.5.2-cp314-cp314-win32.whl", hash = "sha256:cc6fc3cc62e8501d3ed62894425040d2728ecddb1ed072737a5c70bd537aa9f0", size = 39436, upload-time = "2026-05-08T21:01:41.654Z" },
+ { url = "https://files.pythonhosted.org/packages/61/d2/45c9defbaa1ea297035d9d4cce9e8f80daafbf19319c6007f157c6256ea9/propcache-0.5.2-cp314-cp314-win_amd64.whl", hash = "sha256:81e3a30b0bb60caa22033dd0f8a3618d1d67356212514f62c57db75cb0ef410c", size = 42373, upload-time = "2026-05-08T21:01:43.041Z" },
+ { url = "https://files.pythonhosted.org/packages/44/68/9ea5103f41d5217d7d6ec24db90018e23aebec070c3f9a6e54d12b841fd8/propcache-0.5.2-cp314-cp314-win_arm64.whl", hash = "sha256:0d2c9bf8528f135dbb805ce027567e09164f7efa51a2be07458a2c0420f292d0", size = 38554, upload-time = "2026-05-08T21:01:44.336Z" },
+ { url = "https://files.pythonhosted.org/packages/8a/81/fadf555f42d3b762eea8a53950b0489fdc0aa9da5f8ed9e10ce0a4e01b48/propcache-0.5.2-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:4bc8ff1feffc6a61c7002ffe84634c41b822e104990ae009f44a0834430070bb", size = 99395, upload-time = "2026-05-08T21:01:45.883Z" },
+ { url = "https://files.pythonhosted.org/packages/f5/c9/c61e134a686949cf7971af3a390148b1156f7be81c73bc0cd12c873e2d48/propcache-0.5.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:79aa3ff0a9b566633b642fa9caf7e21ed1c13d6feca718187873f199e1514078", size = 56653, upload-time = "2026-05-08T21:01:47.307Z" },
+ { url = "https://files.pythonhosted.org/packages/cb/73/daf935ea7048ddd7ec8eec5345b4a40b619d2d178b3c0a0900796bc3c794/propcache-0.5.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1b31822f4474c4036bae62de9402710051d431a606d6a0f907fec79935a071aa", size = 56914, upload-time = "2026-05-08T21:01:48.573Z" },
+ { url = "https://files.pythonhosted.org/packages/79/9f/aba959b435ea18617edd7cf0a7ad0b9c574b8fc7e3d2cd55fb59cb255d33/propcache-0.5.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:13fef48778b5a2a756523fdb781326b028ca75e32858b04f2cdd19f394564917", size = 62567, upload-time = "2026-05-08T21:01:49.903Z" },
+ { url = "https://files.pythonhosted.org/packages/6c/a1/859942de9a791ff42f6141736f5b37749b8f53e65edfa49638c67dd67e6a/propcache-0.5.2-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:8b73ab70f1a3351fbc71f663b3e645af6dd0329100c353081cf69c37433fc6fe", size = 65542, upload-time = "2026-05-08T21:01:51.204Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/61/315bc0fd6c0fc7f80a528b8afd209e5fc4a875ea79571b91b8f50f442907/propcache-0.5.2-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5538d2c13d93e4698af7e092b57bc7298fd35d1d58e656ae18f23ee0d0378e03", size = 66845, upload-time = "2026-05-08T21:01:52.539Z" },
+ { url = "https://files.pythonhosted.org/packages/47/f7/9f8122e3132e8e354ac41975ef8f1099be7d5a16bc7ae562734e993665c0/propcache-0.5.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cd645f03898405cabe694fb8bc35241e3a9c332ec85627584fe3de201452b335", size = 63985, upload-time = "2026-05-08T21:01:53.847Z" },
+ { url = "https://files.pythonhosted.org/packages/c8/54/c317819ec157cbf6f35df9df9657a6f82daf34d5faf15948b2f639c2192e/propcache-0.5.2-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a473b3440261e0c60706e732b2ed2f517857344fc21bf48fdfe211e2d98eb285", size = 63999, upload-time = "2026-05-08T21:01:55.179Z" },
+ { url = "https://files.pythonhosted.org/packages/5a/56/387e3f7dfce0a9233df41fb888aa1c30222cb4bbbf09537c02dd9bd85fe2/propcache-0.5.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:7afa37062e6650640e932e4cc9297d81f9f42d9944029cc386b8247dea4da837", size = 62779, upload-time = "2026-05-08T21:01:57.489Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/9c/596784cb5824ed61ee960d3f8655a3f0993e107c6e98ab6c818b7fb92ccb/propcache-0.5.2-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:8a90efd5777e996e42d568db9ac740b944d691e565cbfd31b2f7832f9184b2b8", size = 59796, upload-time = "2026-05-08T21:01:58.736Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/3d/1a6cfa1726a48542c1e8784a0761421476a5b68e09b7f36bf95eb954aaba/propcache-0.5.2-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:f19bb891234d72535764d703bfed1153cc34f4214d5bd7150aee1eec9e8f4366", size = 66023, upload-time = "2026-05-08T21:02:00.228Z" },
+ { url = "https://files.pythonhosted.org/packages/e4/0e/05fd6990369477076e4e280bcb970de760fddf0161a46e988bc95f7940ec/propcache-0.5.2-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:32775082acd2d807ee3db715c7770d38767b817870acfa08c29e057f3c4d5b56", size = 64448, upload-time = "2026-05-08T21:02:01.888Z" },
+ { url = "https://files.pythonhosted.org/packages/cd/86/5f8da315a4309c62c10c0b2516b17492d5d3bbe1bb862b96604db67e2a37/propcache-0.5.2-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:9282fb1a3bccd038da9f768b927b24a0c753e466c086b7c4f3c6982851eefb2d", size = 67329, upload-time = "2026-05-08T21:02:03.484Z" },
+ { url = "https://files.pythonhosted.org/packages/da/d3/3368efe79ab21f0cdf86ef49895811c9cc933131d4cde1f28a624e22e712/propcache-0.5.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:cc49723e2f60d6b32a0f0b08a3fd6d13203c07f1cd9566cfce0f12a917c967a2", size = 65172, upload-time = "2026-05-08T21:02:04.745Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/07/127e8b0bacfb325396196f9d976a22453049b89b9b2b08477cc3145faa44/propcache-0.5.2-cp314-cp314t-win32.whl", hash = "sha256:2d7aa89ebca5acc98cba9d1472d976e394782f587bad6661003602a619fd1821", size = 43813, upload-time = "2026-05-08T21:02:06.025Z" },
+ { url = "https://files.pythonhosted.org/packages/88/fb/46dad6c0ae49ed230ab1b16c890c2b6314e2403e6c412976f4a72d64a527/propcache-0.5.2-cp314-cp314t-win_amd64.whl", hash = "sha256:d447bb0b3054be5818458fbb171208b1d9ff11eba14e18ca18b90cbb45767370", size = 47764, upload-time = "2026-05-08T21:02:07.353Z" },
+ { url = "https://files.pythonhosted.org/packages/e7/c4/a47d0a63aa309d10d59ede6e9d4cff03a344a79d1f0f4cd0cd74997b53e0/propcache-0.5.2-cp314-cp314t-win_arm64.whl", hash = "sha256:fe67a3d11cd9b4efabfa45c3d00ffba2b26811442a73a581a94b67c2b5faccf6", size = 41140, upload-time = "2026-05-08T21:02:09.065Z" },
+ { url = "https://files.pythonhosted.org/packages/3a/ed/1cdcab6ba3d6ab7feca11fc14f0eeea80755bb53ef4e892079f31b10a25f/propcache-0.5.2-py3-none-any.whl", hash = "sha256:be1ddfcbb376e3de5d2e2db1d58d6d67463e6b4f9f040c000de8e300295465fe", size = 14036, upload-time = "2026-05-08T21:02:10.673Z" },
+]
+
+[[package]]
+name = "protobuf"
+version = "6.33.6"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/66/70/e908e9c5e52ef7c3a6c7902c9dfbb34c7e29c25d2f81ade3856445fd5c94/protobuf-6.33.6.tar.gz", hash = "sha256:a6768d25248312c297558af96a9f9c929e8c4cee0659cb07e780731095f38135", size = 444531, upload-time = "2026-03-18T19:05:00.988Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/fc/9f/2f509339e89cfa6f6a4c4ff50438db9ca488dec341f7e454adad60150b00/protobuf-6.33.6-cp310-abi3-win32.whl", hash = "sha256:7d29d9b65f8afef196f8334e80d6bc1d5d4adedb449971fefd3723824e6e77d3", size = 425739, upload-time = "2026-03-18T19:04:48.373Z" },
+ { url = "https://files.pythonhosted.org/packages/76/5d/683efcd4798e0030c1bab27374fd13a89f7c2515fb1f3123efdfaa5eab57/protobuf-6.33.6-cp310-abi3-win_amd64.whl", hash = "sha256:0cd27b587afca21b7cfa59a74dcbd48a50f0a6400cfb59391340ad729d91d326", size = 437089, upload-time = "2026-03-18T19:04:50.381Z" },
+ { url = "https://files.pythonhosted.org/packages/5c/01/a3c3ed5cd186f39e7880f8303cc51385a198a81469d53d0fdecf1f64d929/protobuf-6.33.6-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:9720e6961b251bde64edfdab7d500725a2af5280f3f4c87e57c0208376aa8c3a", size = 427737, upload-time = "2026-03-18T19:04:51.866Z" },
+ { url = "https://files.pythonhosted.org/packages/ee/90/b3c01fdec7d2f627b3a6884243ba328c1217ed2d978def5c12dc50d328a3/protobuf-6.33.6-cp39-abi3-manylinux2014_aarch64.whl", hash = "sha256:e2afbae9b8e1825e3529f88d514754e094278bb95eadc0e199751cdd9a2e82a2", size = 324610, upload-time = "2026-03-18T19:04:53.096Z" },
+ { url = "https://files.pythonhosted.org/packages/9b/ca/25afc144934014700c52e05103c2421997482d561f3101ff352e1292fb81/protobuf-6.33.6-cp39-abi3-manylinux2014_s390x.whl", hash = "sha256:c96c37eec15086b79762ed265d59ab204dabc53056e3443e702d2681f4b39ce3", size = 339381, upload-time = "2026-03-18T19:04:54.616Z" },
+ { url = "https://files.pythonhosted.org/packages/16/92/d1e32e3e0d894fe00b15ce28ad4944ab692713f2e7f0a99787405e43533a/protobuf-6.33.6-cp39-abi3-manylinux2014_x86_64.whl", hash = "sha256:e9db7e292e0ab79dd108d7f1a94fe31601ce1ee3f7b79e0692043423020b0593", size = 323436, upload-time = "2026-03-18T19:04:55.768Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/72/02445137af02769918a93807b2b7890047c32bfb9f90371cbc12688819eb/protobuf-6.33.6-py3-none-any.whl", hash = "sha256:77179e006c476e69bf8e8ce866640091ec42e1beb80b213c3900006ecfba6901", size = 170656, upload-time = "2026-03-18T19:04:59.826Z" },
+]
+
+[[package]]
+name = "psutil"
+version = "7.2.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/aa/c6/d1ddf4abb55e93cebc4f2ed8b5d6dbad109ecb8d63748dd2b20ab5e57ebe/psutil-7.2.2.tar.gz", hash = "sha256:0746f5f8d406af344fd547f1c8daa5f5c33dbc293bb8d6a16d80b4bb88f59372", size = 493740, upload-time = "2026-01-28T18:14:54.428Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/51/08/510cbdb69c25a96f4ae523f733cdc963ae654904e8db864c07585ef99875/psutil-7.2.2-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:2edccc433cbfa046b980b0df0171cd25bcaeb3a68fe9022db0979e7aa74a826b", size = 130595, upload-time = "2026-01-28T18:14:57.293Z" },
+ { url = "https://files.pythonhosted.org/packages/d6/f5/97baea3fe7a5a9af7436301f85490905379b1c6f2dd51fe3ecf24b4c5fbf/psutil-7.2.2-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:e78c8603dcd9a04c7364f1a3e670cea95d51ee865e4efb3556a3a63adef958ea", size = 131082, upload-time = "2026-01-28T18:14:59.732Z" },
+ { url = "https://files.pythonhosted.org/packages/37/d6/246513fbf9fa174af531f28412297dd05241d97a75911ac8febefa1a53c6/psutil-7.2.2-cp313-cp313t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1a571f2330c966c62aeda00dd24620425d4b0cc86881c89861fbc04549e5dc63", size = 181476, upload-time = "2026-01-28T18:15:01.884Z" },
+ { url = "https://files.pythonhosted.org/packages/b8/b5/9182c9af3836cca61696dabe4fd1304e17bc56cb62f17439e1154f225dd3/psutil-7.2.2-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:917e891983ca3c1887b4ef36447b1e0873e70c933afc831c6b6da078ba474312", size = 184062, upload-time = "2026-01-28T18:15:04.436Z" },
+ { url = "https://files.pythonhosted.org/packages/16/ba/0756dca669f5a9300d0cbcbfae9a4c30e446dfc7440ffe43ded5724bfd93/psutil-7.2.2-cp313-cp313t-win_amd64.whl", hash = "sha256:ab486563df44c17f5173621c7b198955bd6b613fb87c71c161f827d3fb149a9b", size = 139893, upload-time = "2026-01-28T18:15:06.378Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/61/8fa0e26f33623b49949346de05ec1ddaad02ed8ba64af45f40a147dbfa97/psutil-7.2.2-cp313-cp313t-win_arm64.whl", hash = "sha256:ae0aefdd8796a7737eccea863f80f81e468a1e4cf14d926bd9b6f5f2d5f90ca9", size = 135589, upload-time = "2026-01-28T18:15:08.03Z" },
+ { url = "https://files.pythonhosted.org/packages/81/69/ef179ab5ca24f32acc1dac0c247fd6a13b501fd5534dbae0e05a1c48b66d/psutil-7.2.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:eed63d3b4d62449571547b60578c5b2c4bcccc5387148db46e0c2313dad0ee00", size = 130664, upload-time = "2026-01-28T18:15:09.469Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/64/665248b557a236d3fa9efc378d60d95ef56dd0a490c2cd37dafc7660d4a9/psutil-7.2.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7b6d09433a10592ce39b13d7be5a54fbac1d1228ed29abc880fb23df7cb694c9", size = 131087, upload-time = "2026-01-28T18:15:11.724Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/2e/e6782744700d6759ebce3043dcfa661fb61e2fb752b91cdeae9af12c2178/psutil-7.2.2-cp314-cp314t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1fa4ecf83bcdf6e6c8f4449aff98eefb5d0604bf88cb883d7da3d8d2d909546a", size = 182383, upload-time = "2026-01-28T18:15:13.445Z" },
+ { url = "https://files.pythonhosted.org/packages/57/49/0a41cefd10cb7505cdc04dab3eacf24c0c2cb158a998b8c7b1d27ee2c1f5/psutil-7.2.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e452c464a02e7dc7822a05d25db4cde564444a67e58539a00f929c51eddda0cf", size = 185210, upload-time = "2026-01-28T18:15:16.002Z" },
+ { url = "https://files.pythonhosted.org/packages/dd/2c/ff9bfb544f283ba5f83ba725a3c5fec6d6b10b8f27ac1dc641c473dc390d/psutil-7.2.2-cp314-cp314t-win_amd64.whl", hash = "sha256:c7663d4e37f13e884d13994247449e9f8f574bc4655d509c3b95e9ec9e2b9dc1", size = 141228, upload-time = "2026-01-28T18:15:18.385Z" },
+ { url = "https://files.pythonhosted.org/packages/f2/fc/f8d9c31db14fcec13748d373e668bc3bed94d9077dbc17fb0eebc073233c/psutil-7.2.2-cp314-cp314t-win_arm64.whl", hash = "sha256:11fe5a4f613759764e79c65cf11ebdf26e33d6dd34336f8a337aa2996d71c841", size = 136284, upload-time = "2026-01-28T18:15:19.912Z" },
+ { url = "https://files.pythonhosted.org/packages/e7/36/5ee6e05c9bd427237b11b3937ad82bb8ad2752d72c6969314590dd0c2f6e/psutil-7.2.2-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:ed0cace939114f62738d808fdcecd4c869222507e266e574799e9c0faa17d486", size = 129090, upload-time = "2026-01-28T18:15:22.168Z" },
+ { url = "https://files.pythonhosted.org/packages/80/c4/f5af4c1ca8c1eeb2e92ccca14ce8effdeec651d5ab6053c589b074eda6e1/psutil-7.2.2-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:1a7b04c10f32cc88ab39cbf606e117fd74721c831c98a27dc04578deb0c16979", size = 129859, upload-time = "2026-01-28T18:15:23.795Z" },
+ { url = "https://files.pythonhosted.org/packages/b5/70/5d8df3b09e25bce090399cf48e452d25c935ab72dad19406c77f4e828045/psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:076a2d2f923fd4821644f5ba89f059523da90dc9014e85f8e45a5774ca5bc6f9", size = 155560, upload-time = "2026-01-28T18:15:25.976Z" },
+ { url = "https://files.pythonhosted.org/packages/63/65/37648c0c158dc222aba51c089eb3bdfa238e621674dc42d48706e639204f/psutil-7.2.2-cp36-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b0726cecd84f9474419d67252add4ac0cd9811b04d61123054b9fb6f57df6e9e", size = 156997, upload-time = "2026-01-28T18:15:27.794Z" },
+ { url = "https://files.pythonhosted.org/packages/8e/13/125093eadae863ce03c6ffdbae9929430d116a246ef69866dad94da3bfbc/psutil-7.2.2-cp36-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:fd04ef36b4a6d599bbdb225dd1d3f51e00105f6d48a28f006da7f9822f2606d8", size = 148972, upload-time = "2026-01-28T18:15:29.342Z" },
+ { url = "https://files.pythonhosted.org/packages/04/78/0acd37ca84ce3ddffaa92ef0f571e073faa6d8ff1f0559ab1272188ea2be/psutil-7.2.2-cp36-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:b58fabe35e80b264a4e3bb23e6b96f9e45a3df7fb7eed419ac0e5947c61e47cc", size = 148266, upload-time = "2026-01-28T18:15:31.597Z" },
+ { url = "https://files.pythonhosted.org/packages/b4/90/e2159492b5426be0c1fef7acba807a03511f97c5f86b3caeda6ad92351a7/psutil-7.2.2-cp37-abi3-win_amd64.whl", hash = "sha256:eb7e81434c8d223ec4a219b5fc1c47d0417b12be7ea866e24fb5ad6e84b3d988", size = 137737, upload-time = "2026-01-28T18:15:33.849Z" },
+ { url = "https://files.pythonhosted.org/packages/8c/c7/7bb2e321574b10df20cbde462a94e2b71d05f9bbda251ef27d104668306a/psutil-7.2.2-cp37-abi3-win_arm64.whl", hash = "sha256:8c233660f575a5a89e6d4cb65d9f938126312bca76d8fe087b947b3a1aaac9ee", size = 134617, upload-time = "2026-01-28T18:15:36.514Z" },
+]
+
+[[package]]
+name = "ptyprocess"
+version = "0.7.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/20/e5/16ff212c1e452235a90aeb09066144d0c5a6a8c0834397e03f5224495c4e/ptyprocess-0.7.0.tar.gz", hash = "sha256:5c5d0a3b48ceee0b48485e0c26037c0acd7d29765ca3fbb5cb3831d347423220", size = 70762, upload-time = "2020-12-28T15:15:30.155Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/22/a6/858897256d0deac81a172289110f31629fc4cee19b6f01283303e18c8db3/ptyprocess-0.7.0-py2.py3-none-any.whl", hash = "sha256:4b41f3967fce3af57cc7e94b888626c18bf37a083e3651ca8feeb66d492fef35", size = 13993, upload-time = "2020-12-28T15:15:28.35Z" },
+]
+
+[[package]]
+name = "pure-eval"
+version = "0.2.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/cd/05/0a34433a064256a578f1783a10da6df098ceaa4a57bbeaa96a6c0352786b/pure_eval-0.2.3.tar.gz", hash = "sha256:5f4e983f40564c576c7c8635ae88db5956bb2229d7e9237d03b3c0b0190eaf42", size = 19752, upload-time = "2024-07-21T12:58:21.801Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/8e/37/efad0257dc6e593a18957422533ff0f87ede7c9c6ea010a2177d738fb82f/pure_eval-0.2.3-py3-none-any.whl", hash = "sha256:1db8e35b67b3d218d818ae653e27f06c3aa420901fa7b081ca98cbedc874e0d0", size = 11842, upload-time = "2024-07-21T12:58:20.04Z" },
+]
+
+[[package]]
+name = "pybind11"
+version = "3.0.4"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/cc/f0/35145a3c3baffeef55d4b8324caa33abaa8fa56ab345ecd4b2211d09163e/pybind11-3.0.4.tar.gz", hash = "sha256:3286b59c8a774b9ee650169302dd5a4eedc30a8617905a0560dd8ee44775130c", size = 589533, upload-time = "2026-04-19T03:08:15.925Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/b3/06/c3a23c9a0263b136c519f033a58d4641e73065fefc7754e9667ec206d992/pybind11-3.0.4-py3-none-any.whl", hash = "sha256:961720ee652da51d531b7b2451a6bd2bc042b0106e6d9baa48ecb7d58034ce63", size = 314166, upload-time = "2026-04-19T03:08:14.091Z" },
+]
+
+[[package]]
+name = "pycparser"
+version = "3.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz", hash = "sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29", size = 103492, upload-time = "2026-01-21T14:26:51.89Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" },
+]
+
+[[package]]
+name = "pyct"
+version = "0.6.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "param" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/78/c3/78eeacf7cbf478db6bd41de0ee2221cc6769d81e0c10f6c1616c82a25e06/pyct-0.6.0.tar.gz", hash = "sha256:d4e513b2cf35b6165605ae5fce5f2a49985bc67473c579de85134b8c71d374a8", size = 14586, upload-time = "2025-09-26T08:26:19.987Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/8c/b2/23f4032cd1c9744aa8e9ecda43cd4d755fcb209f7f40fae035248f31a679/pyct-0.6.0-py3-none-any.whl", hash = "sha256:cfaded7289fca72ddf6579b81459e3ec8db323a508e61c49aa318ee3cd6ff160", size = 16630, upload-time = "2025-09-26T08:26:19.092Z" },
+]
+
+[[package]]
+name = "pydantic"
+version = "2.13.4"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "annotated-types" },
+ { name = "pydantic-core" },
+ { name = "typing-extensions" },
+ { name = "typing-inspection" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/18/a5/b60d21ac674192f8ab0ba4e9fd860690f9b4a6e51ca5df118733b487d8d6/pydantic-2.13.4.tar.gz", hash = "sha256:c40756b57adaa8b1efeeced5c196f3f3b7c435f90e84ea7f443901bec8099ef6", size = 844775, upload-time = "2026-05-06T13:43:05.343Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/fd/7b/122376b1fd3c62c1ed9dc80c931ace4844b3c55407b6fb2d199377c9736f/pydantic-2.13.4-py3-none-any.whl", hash = "sha256:45a282cde31d808236fd7ea9d919b128653c8b38b393d1c4ab335c62924d9aba", size = 472262, upload-time = "2026-05-06T13:43:02.641Z" },
+]
+
+[[package]]
+name = "pydantic-core"
+version = "2.46.4"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/9d/56/921726b776ace8d8f5db44c4ef961006580d91dc52b803c489fafd1aa249/pydantic_core-2.46.4.tar.gz", hash = "sha256:62f875393d7f270851f20523dd2e29f082bcc82292d66db2b64ea71f64b6e1c1", size = 471464, upload-time = "2026-05-06T13:37:06.98Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ce/8c/af022f0af448d7747c5154288d46b5f2bc5f17366eaa0e23e9aa04d59f3b/pydantic_core-2.46.4-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:3245406455a5d98187ec35530fd772b1d799b26667980872c8d4614991e2c4a2", size = 2106158, upload-time = "2026-05-06T13:38:57.215Z" },
+ { url = "https://files.pythonhosted.org/packages/19/95/6195171e385007300f0f5574592e467c568becce2d937a0b6804f218bc49/pydantic_core-2.46.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:962ccbab7b642487b1d8b7df90ef677e03134cf1fd8880bf698649b22a69371f", size = 1951724, upload-time = "2026-05-06T13:37:02.697Z" },
+ { url = "https://files.pythonhosted.org/packages/8e/bc/f47d1ff9cbb1620e1b5b697eef06010035735f07820180e74178226b27b3/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8233f2947cf85404441fd7e0085f53b10c93e0ee78611099b5c7237e36aacbf7", size = 1975742, upload-time = "2026-05-06T13:37:09.448Z" },
+ { url = "https://files.pythonhosted.org/packages/5b/11/9b9a5b0306345664a2da6410877af6e8082481b5884b3ddd78d47c6013ce/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:3a233125ac121aa3ffba9a2b59edfc4a985a76092dc8279586ab4b71390875e7", size = 2052418, upload-time = "2026-05-06T13:37:38.234Z" },
+ { url = "https://files.pythonhosted.org/packages/f1/b7/a65fec226f5d78fc39f4a13c4cc0c768c22b113438f60c14adc9d2865038/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5b712b53160b79a5850310b912a5ef8e57e56947c8ad690c227f5c9d7e561712", size = 2232274, upload-time = "2026-05-06T13:38:27.753Z" },
+ { url = "https://files.pythonhosted.org/packages/68/f0/92039db98b907ef49269a8271f67db9cb78ae2fc68062ef7e4e77adb5f61/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9401557acd873c3a7f3eb9383edef8ac4968f9510e340f4808d427e75667e7b4", size = 2309940, upload-time = "2026-05-06T13:38:05.353Z" },
+ { url = "https://files.pythonhosted.org/packages/5f/97/2aab507d3d00ca626e8e57c1eac6a79e4e5fbcc63eb99733ff55d1717f65/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:926c9541b14b12b1681dca8a0b75feb510b06c6341b70a8e500c2fdcff837cce", size = 2094516, upload-time = "2026-05-06T13:39:10.577Z" },
+ { url = "https://files.pythonhosted.org/packages/22/37/a8aca44d40d737dde2bc05b3c6c07dff0de07ce6f82e9f3167aeaf4d5dea/pydantic_core-2.46.4-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:56cb4851bcaf3d117eddcef4fe66afd750a50274b0da8e22be256d10e5611987", size = 2136854, upload-time = "2026-05-06T13:40:22.59Z" },
+ { url = "https://files.pythonhosted.org/packages/24/99/fcef1b79238c06a8cbec70819ac722ba76e02bc8ada9b0fd66eba40da01b/pydantic_core-2.46.4-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c68fcd102d71ea85c5b2dfac3f4f8476eff42a9e078fd5faefff6d145063536b", size = 2180306, upload-time = "2026-05-06T13:40:10.666Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/6c/fc44000918855b42779d007ae63b0532794739027b2f417321cddbc44f6a/pydantic_core-2.46.4-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:b2f69dec1725e79a012d920df1707de5caf7ed5e08f3be4435e25803efc47458", size = 2190044, upload-time = "2026-05-06T13:40:43.231Z" },
+ { url = "https://files.pythonhosted.org/packages/6b/65/d9cadc9f1920d7a127ad2edba16c1db7916e59719285cd6c94600b0080ba/pydantic_core-2.46.4-cp312-cp312-musllinux_1_1_armv7l.whl", hash = "sha256:8d0820e8192167f80d88d64038e609c31452eeca865b4e1d9950a27a4609b00b", size = 2329133, upload-time = "2026-05-06T13:39:57.365Z" },
+ { url = "https://files.pythonhosted.org/packages/d0/cf/c873d91679f3a30bcf5e7ac280ce5573483e72295307685120d0d5ad3416/pydantic_core-2.46.4-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:fbdb89b3e1c94a30cc5edfce477c6e6a5dc4d8f84665b455c27582f211a1c72c", size = 2374464, upload-time = "2026-05-06T13:38:06.976Z" },
+ { url = "https://files.pythonhosted.org/packages/47/bd/6f2fc8188f31bf10590f1e98e7b306336161fac930a8c514cd7bd828c7dc/pydantic_core-2.46.4-cp312-cp312-win32.whl", hash = "sha256:9aa768456404a8bf48a4406685ac2bec8e72b62c69313734fa3b73cf33b3a894", size = 1974823, upload-time = "2026-05-06T13:40:47.985Z" },
+ { url = "https://files.pythonhosted.org/packages/40/8c/985c1d41ea1107c2534abd9870e4ed5c8e7669b5c308297835c001e7a1c4/pydantic_core-2.46.4-cp312-cp312-win_amd64.whl", hash = "sha256:e9c26f834c65f5752f3f06cb08cb86a913ceb7274d0db6e267808a708b46bc89", size = 2072919, upload-time = "2026-05-06T13:39:21.153Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/ba/f463d006e0c47373ca7ec5e1a261c59dc01ef4d62b2657af925fb0deee3a/pydantic_core-2.46.4-cp312-cp312-win_arm64.whl", hash = "sha256:4fc73cb559bdb54b1134a706a2802a4cddd27a0633f5abb7e53056268751ac6a", size = 2027604, upload-time = "2026-05-06T13:39:03.753Z" },
+ { url = "https://files.pythonhosted.org/packages/51/a2/5d30b469c5267a17b39dec53208222f76a8d351dfac4af661888c5aee77d/pydantic_core-2.46.4-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:5d5902252db0d3cedf8d4a1bc68f70eeb430f7e4c7104c8c476753519b423008", size = 2106306, upload-time = "2026-05-06T13:37:48.029Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/81/4fa520eaffa8bd7d1525e644cd6d39e7d60b1592bc5b516693c7340b50f1/pydantic_core-2.46.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c94f0688e7b8d0a67abf40e57a7eaaecd17cc9586706a31b76c031f63df052b4", size = 1951906, upload-time = "2026-05-06T13:37:17.012Z" },
+ { url = "https://files.pythonhosted.org/packages/03/d5/fd02da45b659668b05923b17ba3a0100a0a3d5541e3bd8fcc4ecb711309e/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f027324c56cd5406ca49c124b0db10e56c69064fec039acc571c29020cc87c76", size = 1976802, upload-time = "2026-05-06T13:37:35.113Z" },
+ { url = "https://files.pythonhosted.org/packages/21/f2/95727e1368be3d3ed485eaab7adbd7dda408f33f7a36e8b48e0144002b91/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e739fee756ba1010f8bcccb534252e85a35fe45ae92c295a06059ce58b74ccd3", size = 2052446, upload-time = "2026-05-06T13:37:12.313Z" },
+ { url = "https://files.pythonhosted.org/packages/9c/86/5d99feea3f77c7234b8718075b23db11532773c1a0dbd9b9490215dc2eeb/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9d56801be94b86a9da183e5f3766e6310752b99ff647e38b09a9500d88e46e76", size = 2232757, upload-time = "2026-05-06T13:39:01.149Z" },
+ { url = "https://files.pythonhosted.org/packages/d2/3a/508ac615935ef7588cf6d9e9b91309fdc2da751af865e02a9098de88258c/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2412e734dcb48da14d4e4006b82b46b74f2518b8a26ee7e58c6844a6cd6d03c4", size = 2309275, upload-time = "2026-05-06T13:37:41.406Z" },
+ { url = "https://files.pythonhosted.org/packages/07/f8/41db9de19d7987d6b04715a02b3b40aea467000275d9d758ffaa31af7d50/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9551187363ffc0de2a00b2e47c25aeaeb1020b69b668762966df15fc5659dd5a", size = 2094467, upload-time = "2026-05-06T13:39:18.847Z" },
+ { url = "https://files.pythonhosted.org/packages/2c/e2/f35033184cb11d0052daf4416e8e10a502ea2ac006fc4f459aee872727d1/pydantic_core-2.46.4-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:0186750b482eefa11d7f435892b09c5c606193ef3375bcf94aa00ae6bfb66262", size = 2134417, upload-time = "2026-05-06T13:40:17.944Z" },
+ { url = "https://files.pythonhosted.org/packages/7e/7b/6ceeb1cc90e193862f444ebe373d8fdf613f0a82572dde03fb10734c6c71/pydantic_core-2.46.4-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:5855698a4856556d86e8e6cd8434bc3ac0314ee8e12089ae0e143f64c6256e4e", size = 2179782, upload-time = "2026-05-06T13:40:32.618Z" },
+ { url = "https://files.pythonhosted.org/packages/5a/f2/c8d7773ede6af08036423a00ae0ceffce266c3c52a096c435d68c896083f/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:cbaf13819775b7f769bf4a1f066cb6df7a28d4480081a589828ef190226881cd", size = 2188782, upload-time = "2026-05-06T13:36:51.018Z" },
+ { url = "https://files.pythonhosted.org/packages/59/31/0c864784e31f09f05cdd87606f08923b9c9e7f6e51dd27f20f62f975ce9f/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_armv7l.whl", hash = "sha256:633147d34cf4550417f12e2b1a0383973bdf5cdfde212cb09e9a581cf10820be", size = 2328334, upload-time = "2026-05-06T13:40:37.764Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/eb/4f6c8a41efa30baa755590f4141abf3a8c370fab610915733e74134a7270/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:82cf5301172168103724d49a1444d3378cb20cdee30b116a1bd6031236298a5d", size = 2372986, upload-time = "2026-05-06T13:39:34.152Z" },
+ { url = "https://files.pythonhosted.org/packages/5b/24/b375a480d53113860c299764bfe9f349a3dc9108b3adc0d7f0d786492ebf/pydantic_core-2.46.4-cp313-cp313-win32.whl", hash = "sha256:9fa8ae11da9e2b3126c6426f147e0fba88d96d65921799bb30c6abd1cb2c97fb", size = 1973693, upload-time = "2026-05-06T13:37:55.072Z" },
+ { url = "https://files.pythonhosted.org/packages/7e/e8/cff247591966f2d22ec8c003cd7587e27b7ba7b81ab2fb888e3ab75dc285/pydantic_core-2.46.4-cp313-cp313-win_amd64.whl", hash = "sha256:6b3ace8194b0e5204818c92802dcdca7fc6d88aabbb799d7c795540d9cd6d292", size = 2071819, upload-time = "2026-05-06T13:38:49.139Z" },
+ { url = "https://files.pythonhosted.org/packages/c6/1a/f4aee670d5670e9e148e0c82c7db98d780be566c6e6a97ee8035528ca0b3/pydantic_core-2.46.4-cp313-cp313-win_arm64.whl", hash = "sha256:184c081504d17f1c1066e430e117142b2c77d9448a97f7b65c6ac9fd9aee238d", size = 2027411, upload-time = "2026-05-06T13:40:45.796Z" },
+ { url = "https://files.pythonhosted.org/packages/8d/74/228a26ddad29c6672b805d9fd78e8d251cd04004fa7eed0e622096cd0250/pydantic_core-2.46.4-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:428e04521a40150c85216fc8b85e8d39fece235a9cf5e383761238c7fa9b96fb", size = 2102079, upload-time = "2026-05-06T13:38:41.019Z" },
+ { url = "https://files.pythonhosted.org/packages/ad/1f/8970b150a4b4365623ae00fc88603491f763c627311ae8031e3111356d6e/pydantic_core-2.46.4-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:23ace664830ee0bfe014a0c7bc248b1f7f25ed7ad103852c317624a1083af462", size = 1952179, upload-time = "2026-05-06T13:36:59.812Z" },
+ { url = "https://files.pythonhosted.org/packages/95/30/5211a831ae054928054b2f79731661087a2bc5c01e825c672b3a4a8f1b3e/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ce5c1d2a8b27468f433ca974829c44060b8097eedc39933e3c206a90ee49c4a9", size = 1978926, upload-time = "2026-05-06T13:37:39.933Z" },
+ { url = "https://files.pythonhosted.org/packages/57/e9/689668733b1eb67adeef047db3c2e8788fcf65a7fd9c9e2b46b7744fe245/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:7283d57845ecf5a163403eb0702dfc220cc4fbdd18919cb5ccea4f95ee1cdab4", size = 2046785, upload-time = "2026-05-06T13:38:01.995Z" },
+ { url = "https://files.pythonhosted.org/packages/60/d9/6715260422ff50a2109878fd24d948a6c3446bb2664f34ee78cd972b3acd/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8daafc69c93ee8a0204506a3b6b30f586ef54028f52aeeeb5c4cfc5184fd5914", size = 2228733, upload-time = "2026-05-06T13:40:50.371Z" },
+ { url = "https://files.pythonhosted.org/packages/18/ae/fdb2f64316afca925640f8e70bb1a564b0ec2721c1389e25b8eb4bf9a299/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cd2213145bcc2ba85884d0ac63d222fece9209678f77b9b4d76f054c561adb28", size = 2307534, upload-time = "2026-05-06T13:37:21.531Z" },
+ { url = "https://files.pythonhosted.org/packages/89/1d/8eff589b45bb8190a9d12c49cfad0f176a5cbd1534908a6b5125e2886239/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7a5f930472650a82629163023e630d160863fce524c616f4e5186e5de9d9a49b", size = 2099732, upload-time = "2026-05-06T13:39:31.942Z" },
+ { url = "https://files.pythonhosted.org/packages/06/d5/ee5a3366637fee41dee51a1fc91562dcf12ddbc68fda34e6b253da2324bb/pydantic_core-2.46.4-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:c1b3f518abeca3aa13c712fd202306e145abf59a18b094a6bafb2d2bbf59192c", size = 2129627, upload-time = "2026-05-06T13:37:25.033Z" },
+ { url = "https://files.pythonhosted.org/packages/94/33/2414be571d2c6a6c4d08be21f9292b6d3fdb08949a97b6dfe985017821db/pydantic_core-2.46.4-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1a7dd0b3ee80d90150e3495a3a13ac34dbcbfd4f012996a6a1d8900e91b5c0fb", size = 2179141, upload-time = "2026-05-06T13:37:14.046Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/79/7daa95be995be0eecc4cf75064cb33f9bbbfe3fe0158caf2f0d4a996a5c7/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:3fb702cd90b0446a3a1c5e470bfa0dd23c0233b676a9099ddcc964fa6ca13898", size = 2184325, upload-time = "2026-05-06T13:36:53.615Z" },
+ { url = "https://files.pythonhosted.org/packages/9f/cb/d0a382f5c0de8a222dc61c65348e0ce831b1f68e0a018450d31c2cace3a5/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_armv7l.whl", hash = "sha256:b8458003118a712e66286df6a707db01c52c0f52f7db8e4a38f0da1d3b94fc4e", size = 2323990, upload-time = "2026-05-06T13:40:29.971Z" },
+ { url = "https://files.pythonhosted.org/packages/05/db/d9ba624cc4a5aced1598e88c04fdbd8310c8a69b9d38b9a3d39ce3a61ed7/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:372429a130e469c9cd698925ce5fc50940b7a1336b0d82038e63d5bbc4edc519", size = 2369978, upload-time = "2026-05-06T13:37:23.027Z" },
+ { url = "https://files.pythonhosted.org/packages/f2/20/d15df15ba918c423461905802bfd2981c3af0bfa0e40d05e13edbfa48bc3/pydantic_core-2.46.4-cp314-cp314-win32.whl", hash = "sha256:85bb3611ff1802f3ee7fdd7dbff26b56f343fb432d57a4728fdd49b6ef35e2f4", size = 1966354, upload-time = "2026-05-06T13:38:03.499Z" },
+ { url = "https://files.pythonhosted.org/packages/fc/b6/6b8de4c0a7d7ab3004c439c80c5c1e0a3e8d78bbae19379b01960383d9e5/pydantic_core-2.46.4-cp314-cp314-win_amd64.whl", hash = "sha256:811ff8e9c313ab425368bcbb36e5c4ebd7108c2bbf4e4089cfbb0b01eff63fac", size = 2072238, upload-time = "2026-05-06T13:39:40.807Z" },
+ { url = "https://files.pythonhosted.org/packages/32/36/51eb763beec1f4cf59b1db243a7dcc39cbb41230f050a09b9d69faaf0a48/pydantic_core-2.46.4-cp314-cp314-win_arm64.whl", hash = "sha256:bfec22eab3c8cc2ceec0248aec886624116dc079afa027ecc8ad4a7e62010f8a", size = 2018251, upload-time = "2026-05-06T13:37:26.72Z" },
+ { url = "https://files.pythonhosted.org/packages/e8/91/855af51d625b23aa987116a19e231d2aaef9c4a415273ddc189b79a45fee/pydantic_core-2.46.4-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:af8244b2bef6aaad6d92cda81372de7f8c8d36c9f0c3ea36e827c60e7d9467a0", size = 2099593, upload-time = "2026-05-06T13:39:47.682Z" },
+ { url = "https://files.pythonhosted.org/packages/fb/1b/8784a54c65edb5f49f0a14d6977cf1b209bba85a4c77445b255c2de58ab3/pydantic_core-2.46.4-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5a4330cdbc57162e4b3aa303f588ba752257694c9c9be3e7ebb11b4aca659b5d", size = 1935226, upload-time = "2026-05-06T13:40:40.428Z" },
+ { url = "https://files.pythonhosted.org/packages/e8/e7/1955d28d1afc56dd4b3ad7cc0cf39df1b9852964cf16e5d13912756d6d6b/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:29c61fc04a3d840155ff08e475a04809278972fe6aef51e2720554e96367e34b", size = 1974605, upload-time = "2026-05-06T13:37:32.029Z" },
+ { url = "https://files.pythonhosted.org/packages/93/e2/3fedbf0ba7a22850e6e9fd78117f1c0f10f950182344d8a6c535d468fdd8/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:c50f2528cf200c5eed56faf3f4e22fcd5f38c157a8b78576e6ba3168ec35f000", size = 2030777, upload-time = "2026-05-06T13:38:55.239Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/61/46be275fcaaba0b4f5b9669dd852267ce1ff616592dccf7a7845588df091/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0cbe8b01f948de4286c74cdd6c667aceb38f5c1e26f0693b3983d9d74887c65e", size = 2236641, upload-time = "2026-05-06T13:37:08.096Z" },
+ { url = "https://files.pythonhosted.org/packages/60/db/12e93e46a8bac9988be3c016860f83293daea8c716c029c9ace279036f2f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:617d7e2ca7dcb8c5cf6bcb8c59b8832c94b36196bbf1cbd1bfb56ed341905edd", size = 2286404, upload-time = "2026-05-06T13:40:20.221Z" },
+ { url = "https://files.pythonhosted.org/packages/e2/4a/4d8b19008f38d31c53b8219cfedc2e3d5de5fe99d90076b7e767de29274f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7027560ee92211647d0d34e3f7cd6f50da56399d26a9c8ad0da286d3869a53f3", size = 2109219, upload-time = "2026-05-06T13:38:12.153Z" },
+ { url = "https://files.pythonhosted.org/packages/88/70/3cbc40978fefb7bb09c6708d40d4ad1a5d70fd7213c3d17f971de868ec1f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:f99626688942fb746e545232e7726926f3be91b5975f8b55327665fafda991c7", size = 2110594, upload-time = "2026-05-06T13:40:02.971Z" },
+ { url = "https://files.pythonhosted.org/packages/9d/20/b8d36736216e29491125531685b2f9e61aa5b4b2599893f8268551da3338/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:fc3e9034a63de20e15e8ade85358bc6efc614008cab72898b4b4952bea0509ff", size = 2159542, upload-time = "2026-05-06T13:39:27.506Z" },
+ { url = "https://files.pythonhosted.org/packages/1d/a2/367df868eb584dacf6bf82a389272406d7178e301c4ac82545ab98bc2dd9/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:97e7cf2be5c77b7d1a9713a05605d49460d02c6078d38d8bef3cbe323c548424", size = 2168146, upload-time = "2026-05-06T13:38:31.93Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/b8/4460f77f7e201893f649a29ab355dddd3beee8a97bcb1a320db414f9a06e/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_armv7l.whl", hash = "sha256:3bf92c5d0e00fefaab325a4d27828fe6b6e2a21848686b5b60d2d9eeb09d76c6", size = 2306309, upload-time = "2026-05-06T13:37:44.717Z" },
+ { url = "https://files.pythonhosted.org/packages/64/c4/be2639293acd87dc8ddbcec41a73cee9b2ebf996fe6d892a1a74e88ad3f7/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:3ecbc122d18468d06ca279dc26a8c2e2d5acb10943bb35e36ae92096dc3b5565", size = 2369736, upload-time = "2026-05-06T13:37:05.645Z" },
+ { url = "https://files.pythonhosted.org/packages/30/a6/9f9f380dbb301f67023bf8f707aaa75daadf84f7152d95c410fd7e81d994/pydantic_core-2.46.4-cp314-cp314t-win32.whl", hash = "sha256:e846ae7835bf0703ae43f534ab79a867146dadd59dc9ca5c8b53d5c8f7c9ef02", size = 1955575, upload-time = "2026-05-06T13:38:51.116Z" },
+ { url = "https://files.pythonhosted.org/packages/40/1f/f1eb9eb350e795d1af8586289746f5c5677d16043040d63710e22abc43c9/pydantic_core-2.46.4-cp314-cp314t-win_amd64.whl", hash = "sha256:2108ba5c1c1eca18030634489dc544844144ee36357f2f9f780b93e7ddbb44b5", size = 2051624, upload-time = "2026-05-06T13:38:21.672Z" },
+ { url = "https://files.pythonhosted.org/packages/f6/d2/42dd53d0a85c27606f316d3aa5d2869c4e8470a5ed6dec30e4a1abe19192/pydantic_core-2.46.4-cp314-cp314t-win_arm64.whl", hash = "sha256:4fcbe087dbc2068af7eda3aa87634eba216dbda64d1ae73c8684b621d33f6596", size = 2017325, upload-time = "2026-05-06T13:40:52.723Z" },
+ { url = "https://files.pythonhosted.org/packages/9d/1d/8987ad40f65ae1432753072f214fb5c74fe47ffbd0698bb9cbbb585664f8/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:1d8ba486450b14f3b1d63bc521d410ec7565e52f887b9fb671791886436a42f7", size = 2095527, upload-time = "2026-05-06T13:39:52.283Z" },
+ { url = "https://files.pythonhosted.org/packages/64/d3/84c282a7eee1d3ac4c0377546ef5a1ea436ce26840d9ac3b7ed54a377507/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:3009f12e4e90b7f88b4f9adb1b0c4a3d58fe7820f3238c190047209d148026df", size = 1936024, upload-time = "2026-05-06T13:40:15.671Z" },
+ { url = "https://files.pythonhosted.org/packages/d7/ca/eac61596cdeb4d7e174d3dc0bd8a6238f14f75f97a24e7b7db4c7e7340a0/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ad785e92e6dc634c21555edc8bd6b64957ab844541bcb96a1366c202951ae526", size = 1990696, upload-time = "2026-05-06T13:38:34.717Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/c3/7c8b240552251faf6b3a957db200fcfbbcec36763c050428b601e0c9b83b/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:00c603d540afdd6b80eb39f078f33ebd46211f02f33e34a32d9f053bba711de0", size = 2147590, upload-time = "2026-05-06T13:39:29.883Z" },
+]
+
+[[package]]
+name = "pydata-sphinx-theme"
+version = "0.16.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "accessible-pygments" },
+ { name = "babel" },
+ { name = "beautifulsoup4" },
+ { name = "docutils" },
+ { name = "pygments" },
+ { name = "sphinx" },
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/00/20/bb50f9de3a6de69e6abd6b087b52fa2418a0418b19597601605f855ad044/pydata_sphinx_theme-0.16.1.tar.gz", hash = "sha256:a08b7f0b7f70387219dc659bff0893a7554d5eb39b59d3b8ef37b8401b7642d7", size = 2412693, upload-time = "2024-12-17T10:53:39.537Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/e2/0d/8ba33fa83a7dcde13eb3c1c2a0c1cc29950a048bfed6d9b0d8b6bd710b4c/pydata_sphinx_theme-0.16.1-py3-none-any.whl", hash = "sha256:225331e8ac4b32682c18fcac5a57a6f717c4e632cea5dd0e247b55155faeccde", size = 6723264, upload-time = "2024-12-17T10:53:35.645Z" },
+]
+
+[[package]]
+name = "pygments"
+version = "2.20.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
+]
+
+[[package]]
+name = "pynndescent"
+version = "0.6.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "joblib" },
+ { name = "llvmlite" },
+ { name = "numba" },
+ { name = "scikit-learn" },
+ { name = "scipy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/4a/fb/7f58c397fb31666756457ee2ac4c0289ef2daad57f4ae4be8dec12f80b03/pynndescent-0.6.0.tar.gz", hash = "sha256:7ffde0fb5b400741e055a9f7d377e3702e02250616834231f6c209e39aac24f5", size = 2992987, upload-time = "2026-01-08T21:29:58.943Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/b2/e6/94145d714402fd5ade00b5661f2d0ab981219e07f7db9bfa16786cdb9c04/pynndescent-0.6.0-py3-none-any.whl", hash = "sha256:dc8c74844e4c7f5cbd1e0cd6909da86fdc789e6ff4997336e344779c3d5538ef", size = 73511, upload-time = "2026-01-08T21:29:57.306Z" },
+]
+
+[[package]]
+name = "pyparsing"
+version = "3.3.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/f3/91/9c6ee907786a473bf81c5f53cf703ba0957b23ab84c264080fb5a450416f/pyparsing-3.3.2.tar.gz", hash = "sha256:c777f4d763f140633dcb6d8a3eda953bf7a214dc4eff598413c070bcdc117cbc", size = 6851574, upload-time = "2026-01-21T03:57:59.36Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/10/bd/c038d7cc38edc1aa5bf91ab8068b63d4308c66c4c8bb3cbba7dfbc049f9c/pyparsing-3.3.2-py3-none-any.whl", hash = "sha256:850ba148bd908d7e2411587e247a1e4f0327839c40e2e5e6d05a007ecc69911d", size = 122781, upload-time = "2026-01-21T03:57:55.912Z" },
+]
+
+[[package]]
+name = "pytest"
+version = "9.1.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "colorama", marker = "sys_platform == 'win32'" },
+ { name = "iniconfig" },
+ { name = "packaging" },
+ { name = "pluggy" },
+ { name = "pygments" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
+]
+
+[[package]]
+name = "pytest-cov"
+version = "7.1.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "coverage" },
+ { name = "pluggy" },
+ { name = "pytest" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/b1/51/a849f96e117386044471c8ec2bd6cfebacda285da9525c9106aeb28da671/pytest_cov-7.1.0.tar.gz", hash = "sha256:30674f2b5f6351aa09702a9c8c364f6a01c27aae0c1366ae8016160d1efc56b2", size = 55592, upload-time = "2026-03-21T20:11:16.284Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/9d/7a/d968e294073affff457b041c2be9868a40c1c71f4a35fcc1e45e5493067b/pytest_cov-7.1.0-py3-none-any.whl", hash = "sha256:a0461110b7865f9a271aa1b51e516c9a95de9d696734a2f71e3e78f46e1d4678", size = 22876, upload-time = "2026-03-21T20:11:14.438Z" },
+]
+
+[[package]]
+name = "pytest-xdist"
+version = "3.8.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "execnet" },
+ { name = "pytest" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/78/b4/439b179d1ff526791eb921115fca8e44e596a13efeda518b9d845a619450/pytest_xdist-3.8.0.tar.gz", hash = "sha256:7e578125ec9bc6050861aa93f2d59f1d8d085595d6551c2c90b6f4fad8d3a9f1", size = 88069, upload-time = "2025-07-01T13:30:59.346Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ca/31/d4e37e9e550c2b92a9cbc2e4d0b7420a27224968580b5a447f420847c975/pytest_xdist-3.8.0-py3-none-any.whl", hash = "sha256:202ca578cfeb7370784a8c33d6d05bc6e13b4f25b5053c30a152269fd10f0b88", size = 46396, upload-time = "2025-07-01T13:30:56.632Z" },
+]
+
+[[package]]
+name = "python-dateutil"
+version = "2.9.0.post0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "six" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432, upload-time = "2024-03-01T18:36:20.211Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892, upload-time = "2024-03-01T18:36:18.57Z" },
+]
+
+[[package]]
+name = "pytz"
+version = "2026.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ff/46/dd499ec9038423421951e4fad73051febaa13d2df82b4064f87af8b8c0c3/pytz-2026.2.tar.gz", hash = "sha256:0e60b47b29f21574376f218fe21abc009894a2321ea16c6754f3cad6eb7cdd6a", size = 320861, upload-time = "2026-05-04T01:35:29.667Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ec/dd/96da98f892250475bdf2328112d7468abdd4acc7b902b6af23f4ed958ea0/pytz-2026.2-py2.py3-none-any.whl", hash = "sha256:04156e608bee23d3792fd45c94ae47fae1036688e75032eea2e3bf0323d1f126", size = 510141, upload-time = "2026-05-04T01:35:27.408Z" },
+]
+
+[[package]]
+name = "pyyaml"
+version = "6.0.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" },
+ { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" },
+ { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" },
+ { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" },
+ { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" },
+ { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" },
+ { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" },
+ { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" },
+ { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" },
+ { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" },
+ { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" },
+ { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" },
+ { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" },
+ { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" },
+ { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" },
+ { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" },
+ { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" },
+ { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" },
+ { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" },
+ { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" },
+ { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" },
+ { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" },
+ { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" },
+ { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" },
+ { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" },
+ { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" },
+ { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" },
+ { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" },
+ { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" },
+ { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" },
+ { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" },
+ { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" },
+ { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" },
+ { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" },
+ { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" },
+]
+
+[[package]]
+name = "pyzmq"
+version = "27.1.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "cffi", marker = "implementation_name == 'pypy'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/04/0b/3c9baedbdf613ecaa7aa07027780b8867f57b6293b6ee50de316c9f3222b/pyzmq-27.1.0.tar.gz", hash = "sha256:ac0765e3d44455adb6ddbf4417dcce460fc40a05978c08efdf2948072f6db540", size = 281750, upload-time = "2025-09-08T23:10:18.157Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/92/e7/038aab64a946d535901103da16b953c8c9cc9c961dadcbf3609ed6428d23/pyzmq-27.1.0-cp312-abi3-macosx_10_15_universal2.whl", hash = "sha256:452631b640340c928fa343801b0d07eb0c3789a5ffa843f6e1a9cee0ba4eb4fc", size = 1306279, upload-time = "2025-09-08T23:08:03.807Z" },
+ { url = "https://files.pythonhosted.org/packages/e8/5e/c3c49fdd0f535ef45eefcc16934648e9e59dace4a37ee88fc53f6cd8e641/pyzmq-27.1.0-cp312-abi3-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:1c179799b118e554b66da67d88ed66cd37a169f1f23b5d9f0a231b4e8d44a113", size = 895645, upload-time = "2025-09-08T23:08:05.301Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/e5/b0b2504cb4e903a74dcf1ebae157f9e20ebb6ea76095f6cfffea28c42ecd/pyzmq-27.1.0-cp312-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3837439b7f99e60312f0c926a6ad437b067356dc2bc2ec96eb395fd0fe804233", size = 652574, upload-time = "2025-09-08T23:08:06.828Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/9b/c108cdb55560eaf253f0cbdb61b29971e9fb34d9c3499b0e96e4e60ed8a5/pyzmq-27.1.0-cp312-abi3-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:43ad9a73e3da1fab5b0e7e13402f0b2fb934ae1c876c51d0afff0e7c052eca31", size = 840995, upload-time = "2025-09-08T23:08:08.396Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/bb/b79798ca177b9eb0825b4c9998c6af8cd2a7f15a6a1a4272c1d1a21d382f/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:0de3028d69d4cdc475bfe47a6128eb38d8bc0e8f4d69646adfbcd840facbac28", size = 1642070, upload-time = "2025-09-08T23:08:09.989Z" },
+ { url = "https://files.pythonhosted.org/packages/9c/80/2df2e7977c4ede24c79ae39dcef3899bfc5f34d1ca7a5b24f182c9b7a9ca/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_i686.whl", hash = "sha256:cf44a7763aea9298c0aa7dbf859f87ed7012de8bda0f3977b6fb1d96745df856", size = 2021121, upload-time = "2025-09-08T23:08:11.907Z" },
+ { url = "https://files.pythonhosted.org/packages/46/bd/2d45ad24f5f5ae7e8d01525eb76786fa7557136555cac7d929880519e33a/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:f30f395a9e6fbca195400ce833c731e7b64c3919aa481af4d88c3759e0cb7496", size = 1878550, upload-time = "2025-09-08T23:08:13.513Z" },
+ { url = "https://files.pythonhosted.org/packages/e6/2f/104c0a3c778d7c2ab8190e9db4f62f0b6957b53c9d87db77c284b69f33ea/pyzmq-27.1.0-cp312-abi3-win32.whl", hash = "sha256:250e5436a4ba13885494412b3da5d518cd0d3a278a1ae640e113c073a5f88edd", size = 559184, upload-time = "2025-09-08T23:08:15.163Z" },
+ { url = "https://files.pythonhosted.org/packages/fc/7f/a21b20d577e4100c6a41795842028235998a643b1ad406a6d4163ea8f53e/pyzmq-27.1.0-cp312-abi3-win_amd64.whl", hash = "sha256:9ce490cf1d2ca2ad84733aa1d69ce6855372cb5ce9223802450c9b2a7cba0ccf", size = 619480, upload-time = "2025-09-08T23:08:17.192Z" },
+ { url = "https://files.pythonhosted.org/packages/78/c2/c012beae5f76b72f007a9e91ee9401cb88c51d0f83c6257a03e785c81cc2/pyzmq-27.1.0-cp312-abi3-win_arm64.whl", hash = "sha256:75a2f36223f0d535a0c919e23615fc85a1e23b71f40c7eb43d7b1dedb4d8f15f", size = 552993, upload-time = "2025-09-08T23:08:18.926Z" },
+ { url = "https://files.pythonhosted.org/packages/60/cb/84a13459c51da6cec1b7b1dc1a47e6db6da50b77ad7fd9c145842750a011/pyzmq-27.1.0-cp313-cp313-android_24_arm64_v8a.whl", hash = "sha256:93ad4b0855a664229559e45c8d23797ceac03183c7b6f5b4428152a6b06684a5", size = 1122436, upload-time = "2025-09-08T23:08:20.801Z" },
+ { url = "https://files.pythonhosted.org/packages/dc/b6/94414759a69a26c3dd674570a81813c46a078767d931a6c70ad29fc585cb/pyzmq-27.1.0-cp313-cp313-android_24_x86_64.whl", hash = "sha256:fbb4f2400bfda24f12f009cba62ad5734148569ff4949b1b6ec3b519444342e6", size = 1156301, upload-time = "2025-09-08T23:08:22.47Z" },
+ { url = "https://files.pythonhosted.org/packages/a5/ad/15906493fd40c316377fd8a8f6b1f93104f97a752667763c9b9c1b71d42d/pyzmq-27.1.0-cp313-cp313t-macosx_10_15_universal2.whl", hash = "sha256:e343d067f7b151cfe4eb3bb796a7752c9d369eed007b91231e817071d2c2fec7", size = 1341197, upload-time = "2025-09-08T23:08:24.286Z" },
+ { url = "https://files.pythonhosted.org/packages/14/1d/d343f3ce13db53a54cb8946594e567410b2125394dafcc0268d8dda027e0/pyzmq-27.1.0-cp313-cp313t-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:08363b2011dec81c354d694bdecaef4770e0ae96b9afea70b3f47b973655cc05", size = 897275, upload-time = "2025-09-08T23:08:26.063Z" },
+ { url = "https://files.pythonhosted.org/packages/69/2d/d83dd6d7ca929a2fc67d2c3005415cdf322af7751d773524809f9e585129/pyzmq-27.1.0-cp313-cp313t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d54530c8c8b5b8ddb3318f481297441af102517602b569146185fa10b63f4fa9", size = 660469, upload-time = "2025-09-08T23:08:27.623Z" },
+ { url = "https://files.pythonhosted.org/packages/3e/cd/9822a7af117f4bc0f1952dbe9ef8358eb50a24928efd5edf54210b850259/pyzmq-27.1.0-cp313-cp313t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6f3afa12c392f0a44a2414056d730eebc33ec0926aae92b5ad5cf26ebb6cc128", size = 847961, upload-time = "2025-09-08T23:08:29.672Z" },
+ { url = "https://files.pythonhosted.org/packages/9a/12/f003e824a19ed73be15542f172fd0ec4ad0b60cf37436652c93b9df7c585/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:c65047adafe573ff023b3187bb93faa583151627bc9c51fc4fb2c561ed689d39", size = 1650282, upload-time = "2025-09-08T23:08:31.349Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/4a/e82d788ed58e9a23995cee70dbc20c9aded3d13a92d30d57ec2291f1e8a3/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:90e6e9441c946a8b0a667356f7078d96411391a3b8f80980315455574177ec97", size = 2024468, upload-time = "2025-09-08T23:08:33.543Z" },
+ { url = "https://files.pythonhosted.org/packages/d9/94/2da0a60841f757481e402b34bf4c8bf57fa54a5466b965de791b1e6f747d/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:add071b2d25f84e8189aaf0882d39a285b42fa3853016ebab234a5e78c7a43db", size = 1885394, upload-time = "2025-09-08T23:08:35.51Z" },
+ { url = "https://files.pythonhosted.org/packages/4f/6f/55c10e2e49ad52d080dc24e37adb215e5b0d64990b57598abc2e3f01725b/pyzmq-27.1.0-cp313-cp313t-win32.whl", hash = "sha256:7ccc0700cfdf7bd487bea8d850ec38f204478681ea02a582a8da8171b7f90a1c", size = 574964, upload-time = "2025-09-08T23:08:37.178Z" },
+ { url = "https://files.pythonhosted.org/packages/87/4d/2534970ba63dd7c522d8ca80fb92777f362c0f321900667c615e2067cb29/pyzmq-27.1.0-cp313-cp313t-win_amd64.whl", hash = "sha256:8085a9fba668216b9b4323be338ee5437a235fe275b9d1610e422ccc279733e2", size = 641029, upload-time = "2025-09-08T23:08:40.595Z" },
+ { url = "https://files.pythonhosted.org/packages/f6/fa/f8aea7a28b0641f31d40dea42d7ef003fded31e184ef47db696bc74cd610/pyzmq-27.1.0-cp313-cp313t-win_arm64.whl", hash = "sha256:6bb54ca21bcfe361e445256c15eedf083f153811c37be87e0514934d6913061e", size = 561541, upload-time = "2025-09-08T23:08:42.668Z" },
+ { url = "https://files.pythonhosted.org/packages/87/45/19efbb3000956e82d0331bafca5d9ac19ea2857722fa2caacefb6042f39d/pyzmq-27.1.0-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:ce980af330231615756acd5154f29813d553ea555485ae712c491cd483df6b7a", size = 1341197, upload-time = "2025-09-08T23:08:44.973Z" },
+ { url = "https://files.pythonhosted.org/packages/48/43/d72ccdbf0d73d1343936296665826350cb1e825f92f2db9db3e61c2162a2/pyzmq-27.1.0-cp314-cp314t-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:1779be8c549e54a1c38f805e56d2a2e5c009d26de10921d7d51cfd1c8d4632ea", size = 897175, upload-time = "2025-09-08T23:08:46.601Z" },
+ { url = "https://files.pythonhosted.org/packages/2f/2e/a483f73a10b65a9ef0161e817321d39a770b2acf8bcf3004a28d90d14a94/pyzmq-27.1.0-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7200bb0f03345515df50d99d3db206a0a6bee1955fbb8c453c76f5bf0e08fb96", size = 660427, upload-time = "2025-09-08T23:08:48.187Z" },
+ { url = "https://files.pythonhosted.org/packages/f5/d2/5f36552c2d3e5685abe60dfa56f91169f7a2d99bbaf67c5271022ab40863/pyzmq-27.1.0-cp314-cp314t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01c0e07d558b06a60773744ea6251f769cd79a41a97d11b8bf4ab8f034b0424d", size = 847929, upload-time = "2025-09-08T23:08:49.76Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/2a/404b331f2b7bf3198e9945f75c4c521f0c6a3a23b51f7a4a401b94a13833/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:80d834abee71f65253c91540445d37c4c561e293ba6e741b992f20a105d69146", size = 1650193, upload-time = "2025-09-08T23:08:51.7Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/0b/f4107e33f62a5acf60e3ded67ed33d79b4ce18de432625ce2fc5093d6388/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:544b4e3b7198dde4a62b8ff6685e9802a9a1ebf47e77478a5eb88eca2a82f2fd", size = 2024388, upload-time = "2025-09-08T23:08:53.393Z" },
+ { url = "https://files.pythonhosted.org/packages/0d/01/add31fe76512642fd6e40e3a3bd21f4b47e242c8ba33efb6809e37076d9b/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:cedc4c68178e59a4046f97eca31b148ddcf51e88677de1ef4e78cf06c5376c9a", size = 1885316, upload-time = "2025-09-08T23:08:55.702Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/59/a5f38970f9bf07cee96128de79590bb354917914a9be11272cfc7ff26af0/pyzmq-27.1.0-cp314-cp314t-win32.whl", hash = "sha256:1f0b2a577fd770aa6f053211a55d1c47901f4d537389a034c690291485e5fe92", size = 587472, upload-time = "2025-09-08T23:08:58.18Z" },
+ { url = "https://files.pythonhosted.org/packages/70/d8/78b1bad170f93fcf5e3536e70e8fadac55030002275c9a29e8f5719185de/pyzmq-27.1.0-cp314-cp314t-win_amd64.whl", hash = "sha256:19c9468ae0437f8074af379e986c5d3d7d7bfe033506af442e8c879732bedbe0", size = 661401, upload-time = "2025-09-08T23:08:59.802Z" },
+ { url = "https://files.pythonhosted.org/packages/81/d6/4bfbb40c9a0b42fc53c7cf442f6385db70b40f74a783130c5d0a5aa62228/pyzmq-27.1.0-cp314-cp314t-win_arm64.whl", hash = "sha256:dc5dbf68a7857b59473f7df42650c621d7e8923fb03fa74a526890f4d33cc4d7", size = 575170, upload-time = "2025-09-08T23:09:01.418Z" },
+]
+
+[[package]]
+name = "referencing"
+version = "0.37.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "attrs" },
+ { name = "rpds-py" },
+ { name = "typing-extensions", marker = "python_full_version < '3.13'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/22/f5/df4e9027acead3ecc63e50fe1e36aca1523e1719559c499951bb4b53188f/referencing-0.37.0.tar.gz", hash = "sha256:44aefc3142c5b842538163acb373e24cce6632bd54bdb01b21ad5863489f50d8", size = 78036, upload-time = "2025-10-13T15:30:48.871Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/2c/58/ca301544e1fa93ed4f80d724bf5b194f6e4b945841c5bfd555878eea9fcb/referencing-0.37.0-py3-none-any.whl", hash = "sha256:381329a9f99628c9069361716891d34ad94af76e461dcb0335825aecc7692231", size = 26766, upload-time = "2025-10-13T15:30:47.625Z" },
+]
+
+[[package]]
+name = "requests"
+version = "2.34.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "certifi" },
+ { name = "charset-normalizer" },
+ { name = "idna" },
+ { name = "urllib3" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" },
+]
+
+[[package]]
+name = "rich"
+version = "15.0.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "markdown-it-py" },
+ { name = "pygments" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz", hash = "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36", size = 230680, upload-time = "2026-04-12T08:24:00.75Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" },
+]
+
+[[package]]
+name = "roman-numerals"
+version = "4.1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ae/f9/41dc953bbeb056c17d5f7a519f50fdf010bd0553be2d630bc69d1e022703/roman_numerals-4.1.0.tar.gz", hash = "sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2", size = 9077, upload-time = "2025-12-17T18:25:34.381Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/04/54/6f679c435d28e0a568d8e8a7c0a93a09010818634c3c3907fc98d8983770/roman_numerals-4.1.0-py3-none-any.whl", hash = "sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7", size = 7676, upload-time = "2025-12-17T18:25:33.098Z" },
+]
+
+[[package]]
+name = "rpds-py"
+version = "2026.5.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/2e/43/25a8dcd3feedd735039a8f0b5b7e3b118232b5eae288c4fd9ab200d41094/rpds_py-2026.5.1.tar.gz", hash = "sha256:07b24fea40541e28570e5b795a4a38fbdcd12550c06bd0748005ecc8116ca256", size = 64459, upload-time = "2026-05-28T12:02:13.232Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/d4/e7/a78582dc57caa592dcc7d4fb69b61390561e908eb3d2f5df5928a8e354c0/rpds_py-2026.5.1-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:3abe24a66e57adcfa645d718063a5fa5103ecc71ddbf26d78af8f9368018ff1d", size = 353040, upload-time = "2026-05-28T11:59:12.531Z" },
+ { url = "https://files.pythonhosted.org/packages/a3/43/35e3f136343aef451e545ce8c38d36c2f93c0ed88703db8b64ba2b205c68/rpds_py-2026.5.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:58b1d94308ddf0b1982f61f2eb54bf92997c9ece8a8093ef014250f4a517906c", size = 345775, upload-time = "2026-05-28T11:59:13.827Z" },
+ { url = "https://files.pythonhosted.org/packages/20/e1/0f2160c5982d3157734d5cb3ed63d8b2d583a73c9864f77b666449f32cf8/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0fa92420128dadce7f54bd73ba1825a273e9268fe9e35dbf7e6362890efa4e08", size = 376329, upload-time = "2026-05-28T11:59:15.271Z" },
+ { url = "https://files.pythonhosted.org/packages/d0/11/ee0ba42aff83bf4effdbc576673c6be64c5e173978c3f6d537e94482f77d/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ca653c6546386227cd9800d1bef6a348099acf8db4250341da6d90f663d6dfcb", size = 383539, upload-time = "2026-05-28T11:59:16.665Z" },
+ { url = "https://files.pythonhosted.org/packages/11/df/d94aa6a499d4ac40afe2d7620f2c597fd3c0f182e854ad7cf3f596a81cb6/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:66c93681c4729e4e3ecba31b8179fae083ff3118841672835140338b4b9867c1", size = 494674, upload-time = "2026-05-28T11:59:17.991Z" },
+ { url = "https://files.pythonhosted.org/packages/1f/75/33d30f43bb2f458de11979486a591b1bf6e5651765ed1704c6197c2dc773/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:40ff257542e04796880e011e15cd4dc21c2599975df2aaa8f2c8495ca574e1a5", size = 389268, upload-time = "2026-05-28T11:59:19.434Z" },
+ { url = "https://files.pythonhosted.org/packages/f4/1e/2c9096fc19d5fd084b0184ca2b651e659aa0a37e6fdbecf6ece47f147fe1/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b6825cc329b290e93c5f6a9be2393118a763f6ccf6abd83704e0c102ca583644", size = 376280, upload-time = "2026-05-28T11:59:21Z" },
+ { url = "https://files.pythonhosted.org/packages/b9/e5/61ec9f8be8211ea7f48448195549e4aaf02004083475493b0e137702ecb2/rpds_py-2026.5.1-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:de42116e69cb53b911cc34aee5ab98f36c597b822545045d49e938818b99e5e4", size = 387233, upload-time = "2026-05-28T11:59:22.454Z" },
+ { url = "https://files.pythonhosted.org/packages/0d/ca/bcec1005c4f4a234f92a29078631fee49206c7265ccae966f18fd332e80e/rpds_py-2026.5.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c0f920015df2a504bebaba6d4c31ccf3fcf942f92655c086da30b671aad19aa6", size = 405009, upload-time = "2026-05-28T11:59:23.845Z" },
+ { url = "https://files.pythonhosted.org/packages/72/e6/4d5718c5cf26c522dc7c9999e238da1e77380b81d0c5d1df11e271ddfeb1/rpds_py-2026.5.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:0408a24e44feb919423dc6d9da677cb5cddb894d2ca9e763967d156d9c60fab4", size = 553113, upload-time = "2026-05-28T11:59:25.184Z" },
+ { url = "https://files.pythonhosted.org/packages/d4/25/2ee807bdb3e1f0b7eddf7782acd5665a8b5205a331a7d7244a52c4812fd9/rpds_py-2026.5.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:cea68bcd53467561ae2f96a6bdad1544299ba97b5b0ddcd5ac3d376e5c781c24", size = 618838, upload-time = "2026-05-28T11:59:26.749Z" },
+ { url = "https://files.pythonhosted.org/packages/6a/c1/7d4c26f167f8c41501cc073d30ee22082b16ce358cf5b00ec97cbc7804ea/rpds_py-2026.5.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:4be8b1d2a705cc37d08256004e1d07de143fa0075c8e85a3df020b776f62b732", size = 582436, upload-time = "2026-05-28T11:59:28.11Z" },
+ { url = "https://files.pythonhosted.org/packages/04/1d/9d12b0a337bab46f4769f8857f4007e3b2d639e14f9a44a0efe157696e64/rpds_py-2026.5.1-cp312-cp312-win32.whl", hash = "sha256:6736718bd4fc49cbcb538ba30516fdbef161522acefb739657d48b97bd864fed", size = 212734, upload-time = "2026-05-28T11:59:29.689Z" },
+ { url = "https://files.pythonhosted.org/packages/c5/93/e4116f2de7f56bc7406a76033dc501811ddeb22b7f056b92d632871ebb0c/rpds_py-2026.5.1-cp312-cp312-win_amd64.whl", hash = "sha256:0a7d1eec967df0e9b22614a5e177622e0c89611d03727fa0cb48e45028907870", size = 229045, upload-time = "2026-05-28T11:59:31.033Z" },
+ { url = "https://files.pythonhosted.org/packages/cb/53/6c3419d85eb2ec5938a37627c585b42d76a63bb731d6e42ed4b079ebf486/rpds_py-2026.5.1-cp312-cp312-win_arm64.whl", hash = "sha256:1841d067089e117142d79b98aa0df2f08b52f2ecc1819dd2700636c0db74a473", size = 223967, upload-time = "2026-05-28T11:59:32.318Z" },
+ { url = "https://files.pythonhosted.org/packages/6c/32/14c961ad295f490eb0849ada8b79683e93a59b9de3afdd983eaf55fa6867/rpds_py-2026.5.1-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:efef4ac29c6ff495531eb17ee705b62841ecaa291b7c7077e848ea03e237164d", size = 352787, upload-time = "2026-05-28T11:59:33.655Z" },
+ { url = "https://files.pythonhosted.org/packages/ca/bb/d1b85117967c11191441a7274ae616c65d93901d082c588f89a50a8da5ae/rpds_py-2026.5.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c39f5b67a8a2e67179ada2a954227d670fe65fa9098457f698f56ddf248709b3", size = 345179, upload-time = "2026-05-28T11:59:35Z" },
+ { url = "https://files.pythonhosted.org/packages/7c/46/d84105f062e626a1b233f863907288a4708c2d833b8b4c6fb2764bc080c0/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b5c30f3f04eef4fbd362226a6f31d7c8895ca4fbb6e0b790f6890a98d8da8559", size = 376173, upload-time = "2026-05-28T11:59:36.43Z" },
+ { url = "https://files.pythonhosted.org/packages/e2/ae/469d7959ce5b1201e1de135dc735b86db3b35dd0d1734f6a44246d5f061c/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:277f6c82f0580848796c7ecc8a7173aa3bfb928e4ff831261c2f60a81dc270db", size = 383162, upload-time = "2026-05-28T11:59:37.995Z" },
+ { url = "https://files.pythonhosted.org/packages/dc/a2/57853d31a1116a561aa072794602ad3f6341e18d70a8523f1bd5b9fc1e5a/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:63c2c4c213f1a4e3f3de28ecab029dbdee976324e729c0d7a55211be72576b02", size = 495093, upload-time = "2026-05-28T11:59:39.453Z" },
+ { url = "https://files.pythonhosted.org/packages/99/63/3a8eabcad9314b7daf5c65f451d2c33d989235cd8a5762186cf2c3f5a4f8/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3350ec808fb538fe71a1f94dfaa0e29c598dfad805ce49f0caec5ae3183c652b", size = 389829, upload-time = "2026-05-28T11:59:40.896Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/25/05678d97fc25e2622df14dc530fb82023174ecfff6733991ed0d78f167bd/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b1b964e3ab599e718dc46c018d104b1ebc007cbc6567d827c94a687fca56d77e", size = 374786, upload-time = "2026-05-28T11:59:42.626Z" },
+ { url = "https://files.pythonhosted.org/packages/88/d1/8c90b6431e80a3b91b284a5c7c8c0c4f9c006444d90477a740d6e0f9c694/rpds_py-2026.5.1-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:19cb09fab7b7fc96b2a6e28f2e34b72a3705ff27b37edb77455316e5d3f3dc9b", size = 386920, upload-time = "2026-05-28T11:59:44.124Z" },
+ { url = "https://files.pythonhosted.org/packages/ff/99/4638f672ab356682d633ee0da9255f5b67ce6efd0b85eb94ad3e255e65a5/rpds_py-2026.5.1-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:abe76bcdba31e576cb83eeb8797aa0d882b738fef6dc65d0601fc753806a5b46", size = 405059, upload-time = "2026-05-28T11:59:47.177Z" },
+ { url = "https://files.pythonhosted.org/packages/66/3f/3546524b6eb4cc2e1f363a3d638fa52f6c24faae3500c25fb488b02f1740/rpds_py-2026.5.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:8bff7073db3899158fff55ebf57b113a67030af26f80a18978f9f0aa60250ddf", size = 553030, upload-time = "2026-05-28T11:59:48.603Z" },
+ { url = "https://files.pythonhosted.org/packages/c6/c3/7b3388c796fcf471bd17194242d4dc1a7608567c0fa422bcc1c5e79f9c1e/rpds_py-2026.5.1-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:8ba264fa49be666cd9cc56bf34ec7002fb3d27a4aee5bcb4d43d0d18feb1bb6f", size = 618975, upload-time = "2026-05-28T11:59:50.314Z" },
+ { url = "https://files.pythonhosted.org/packages/61/1e/a3cb07f2795075d1d88efddae2f541359fde5f08c81ee114c29c2949c90a/rpds_py-2026.5.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:4860b603ddda0475a8885499b3729e90229d480105b42651962a5397d995fa89", size = 581178, upload-time = "2026-05-28T11:59:51.673Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/74/e758c03a5ef46f04c37f2651a2893db846d569ba8a7bca469d4b58939bcd/rpds_py-2026.5.1-cp313-cp313-win32.whl", hash = "sha256:7944270ae71383f6e2657dd7d5ce4eeb4ac2d0059a6738f0510583d462ab4842", size = 212481, upload-time = "2026-05-28T11:59:53.148Z" },
+ { url = "https://files.pythonhosted.org/packages/70/ec/a2aca432db9c7359b40fa393eeeaa0d166c2f70175be956e75fa24197c44/rpds_py-2026.5.1-cp313-cp313-win_amd64.whl", hash = "sha256:88647f43a73c4e01be19b04ceef0c8d3a1958153604d13c773becd8016f2a0cf", size = 228519, upload-time = "2026-05-28T11:59:54.505Z" },
+ { url = "https://files.pythonhosted.org/packages/29/60/a73bfdd45b096574556acf303bbd9fa9eed36ca8a818b514e2a5d5fe2b9d/rpds_py-2026.5.1-cp313-cp313-win_arm64.whl", hash = "sha256:453895624ecf7db7063b1004e44037522bbaef9ff6a945e59bc71662d7a03abd", size = 223446, upload-time = "2026-05-28T11:59:56.081Z" },
+ { url = "https://files.pythonhosted.org/packages/18/e2/408105fd611823f00882aea810f3989a30d26b1bab8b6beb20f98c724e0e/rpds_py-2026.5.1-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:b4e4bc98639ec915f512fde3aa7a95e0041d95d9c3cc86eea841fa63cb1e8600", size = 355287, upload-time = "2026-05-28T11:59:57.448Z" },
+ { url = "https://files.pythonhosted.org/packages/8d/58/5c4a43436843c90d0f6d19f82c200c80e3843ca9fa07b237623327f6d384/rpds_py-2026.5.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:cacedb7a6e167680acba45ad5716e89067d225dc80da0d7040cae8c81d4572fa", size = 347033, upload-time = "2026-05-28T11:59:58.881Z" },
+ { url = "https://files.pythonhosted.org/packages/fb/c2/1a71acdacaf4e259b10278fb87b039ded3cf80041bcd89dd8a3ea702ded6/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:68700371c5d7ae1412862ddfa719090925c93ecf351c566d66f09d04b136ea00", size = 376891, upload-time = "2026-05-28T12:00:00.516Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/c8/535f3d9b65addd8e28aa87b83c6e526799c3717a88273db8ea795beeef7a/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:296c799becfa849c779c8725494fe9ed94959ed886787df4364b058465bad7f0", size = 385646, upload-time = "2026-05-28T12:00:02.394Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/91/dc033f313345c354ade914dbe73cdb90b615a4409ea02430d5356794f3d8/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d3858b908218ee108d0bbfb2095ccc237648053c9bf98affad7cb079acaf1d97", size = 498830, upload-time = "2026-05-28T12:00:04.189Z" },
+ { url = "https://files.pythonhosted.org/packages/27/fc/90fcbea459dbb8ddc18a2e0fd1de9412b48bc84ffff2db771cf714bacfd6/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4fb8d2e7cb2f850b169806d61d1b991738acec96500a75c30f49caf064ce7cef", size = 392830, upload-time = "2026-05-28T12:00:05.797Z" },
+ { url = "https://files.pythonhosted.org/packages/b2/1d/46cd11a228c9750684a798d98f878be6f614aa762438da7378f035e79e35/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:27b74c10ed6a8f190f4287f53bcfea348b92a84a9c9f70d30183d1e6172d580d", size = 379613, upload-time = "2026-05-28T12:00:07.433Z" },
+ { url = "https://files.pythonhosted.org/packages/24/4a/d9b0c6af3a1de03eb93741bbe8be2bdce84d8fda8224f3005451d86df389/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_31_riscv64.whl", hash = "sha256:b9a6528956191c48c52294a592dbd4a8386d7048bdb25c0efcb6b966466c6d83", size = 388183, upload-time = "2026-05-28T12:00:09.227Z" },
+ { url = "https://files.pythonhosted.org/packages/c5/b4/db7aaabdda6d020afc87d981bcc2f57a434c7dec60ecfc2ab3dd50b20351/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:af03e34e860047bc7a352b842856fcf78798fbb81132cc98bd2f907ab4eb9cd2", size = 408578, upload-time = "2026-05-28T12:00:10.779Z" },
+ { url = "https://files.pythonhosted.org/packages/08/d6/070f6a41cbb343e2ac4171859bf3f3623e0ab002f72619d6d505313ec2de/rpds_py-2026.5.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:fea6e836d10abbe191d557d33bd58bd5987725fe63aa1eefe557d230209855bd", size = 553573, upload-time = "2026-05-28T12:00:12.443Z" },
+ { url = "https://files.pythonhosted.org/packages/75/ab/1a71ea3589c4345dac0a0518f0e6a031cb42689277851b683c46d27463a5/rpds_py-2026.5.1-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:fc0c0f878ea770a0a8a462456c5ad36fc9fe6358e6b76fdadc7f17575e0b8bf1", size = 620861, upload-time = "2026-05-28T12:00:14.09Z" },
+ { url = "https://files.pythonhosted.org/packages/8a/22/9bf80a56069c0c443fcfefac639a86a744550a2898817a6dfd3e26654924/rpds_py-2026.5.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:e0b360f316d966b048b085857630b3cc51f3db2f07b06f440eac8f695374d1e3", size = 585633, upload-time = "2026-05-28T12:00:15.66Z" },
+ { url = "https://files.pythonhosted.org/packages/da/68/3b2c0a75c9e04125696f84ebdbbf304acf5a40b58ba4481cdb98a922c3ba/rpds_py-2026.5.1-cp313-cp313t-win32.whl", hash = "sha256:a2999883eedf72fdfb7520b92c7d4ec2572a71ff40239377aa604cc529eecafc", size = 210074, upload-time = "2026-05-28T12:00:17.291Z" },
+ { url = "https://files.pythonhosted.org/packages/e7/8b/609157d5a25d37d4f29f92840ba531f416907c34ae5c5739dd21fc2bef98/rpds_py-2026.5.1-cp313-cp313t-win_amd64.whl", hash = "sha256:e07be2a9d7122bd6e82dea89814ef8dc893feb1aae97fec1630f3263bbb30e55", size = 228635, upload-time = "2026-05-28T12:00:18.73Z" },
+ { url = "https://files.pythonhosted.org/packages/d4/6f/19c1918a4b590d8de87e712e4abe4b3875771eff60216fb6153cf6665c68/rpds_py-2026.5.1-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:1f2c391c3059798093b65df23aca2cac150460ae9c630d99dec83d703d9485b9", size = 349756, upload-time = "2026-05-28T12:00:20.217Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/60/a06fe7da34eca79dacbf958a2ba0c6eea85bc2b29de20080bf40f72f66fa/rpds_py-2026.5.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:413b424f7c4ee65ab5e5be91f5731be0f8b41a1ee2b12dfe810d716312e95a78", size = 343831, upload-time = "2026-05-28T12:00:21.711Z" },
+ { url = "https://files.pythonhosted.org/packages/bf/ec/b2333b97b90e2a6ef6ca8ad386ee284968e74bcfe113b3f1a8d9036429a9/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2c595a1d9255dce0599e13130d1440ab2506654f2b50294226ee06402f8fef63", size = 375127, upload-time = "2026-05-28T12:00:23.326Z" },
+ { url = "https://files.pythonhosted.org/packages/14/7f/e00aae54067f2b488c4637961d5f58204d470795fc791085fa3f15060d2e/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:1c27c5f6102eac8c03e7595a00827a53b271ba40a53b59ff8709170e0855ea4a", size = 379034, upload-time = "2026-05-28T12:00:24.89Z" },
+ { url = "https://files.pythonhosted.org/packages/be/cc/423999bbb8ae8dc93c77fc1d5e984ade5eb89d237d3bb884ccfa72ae2890/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6c7fcf61d44cacecaf3aea542b0e053db77972a4573e7ceda16fb2b399161195", size = 490823, upload-time = "2026-05-28T12:00:26.676Z" },
+ { url = "https://files.pythonhosted.org/packages/0f/aa/c671bf660f12e68d3c52ff86c7066ed1372df5a0f4f2ff584e419b8207e7/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2c817a189d4ee14290420e5ff051e4dd6baa13f3edf84685071dee07a6d538ee", size = 388144, upload-time = "2026-05-28T12:00:28.577Z" },
+ { url = "https://files.pythonhosted.org/packages/19/c8/d63bb75b68afe77b229e3021c6031bcaf01da5db5b0e69d0d10f9ba679a7/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:21846aac0ed2e0589f38c12dc44e77bb64e494b771eadbcf169cba00566ba7ba", size = 371959, upload-time = "2026-05-28T12:00:30.304Z" },
+ { url = "https://files.pythonhosted.org/packages/82/35/c51122014d8274ff37dc606d60049c3db7d83da02b5b282511e5a906a9a6/rpds_py-2026.5.1-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:b317c87a13f769a4e787819bd508aaa5d69aa09b0880de9af6d3a8a54571cdec", size = 383558, upload-time = "2026-05-28T12:00:31.764Z" },
+ { url = "https://files.pythonhosted.org/packages/e3/f9/2790cb99c136a5363acdeacf5c27c56f3de0d4118a1f48fca83404c99c89/rpds_py-2026.5.1-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:ce87129d9f2c14fa6c4a8601fb80eb4488c80d38a20cd13758ef11123e14995d", size = 402789, upload-time = "2026-05-28T12:00:33.247Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/1b/e4fb584f8c75d35c38150ff6a332cda949e6f97acba1f4fd123b14ab56fe/rpds_py-2026.5.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:9cdddb6c1207d284d94fd1530adf57fbd797fe7c4b8704ba85f49414f2557e7d", size = 551405, upload-time = "2026-05-28T12:00:34.819Z" },
+ { url = "https://files.pythonhosted.org/packages/d8/f7/a6731b4216cb3793ea1af5391da240f5683dacc0d13e034fe5fc3503f240/rpds_py-2026.5.1-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:4e237e139f94d3c036fd28eb9f564c99055476ff4ff05cd42be55ce349b5aa02", size = 616975, upload-time = "2026-05-28T12:00:36.268Z" },
+ { url = "https://files.pythonhosted.org/packages/2c/ea/2e051a81d95d8e63f4b35a1c463a87e8766bc3d083c067c5dfb6bf220747/rpds_py-2026.5.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ed0954b524873214369184a9c82b0eaa45a3fbb9a798cd95b17e0d98499e7ea0", size = 578701, upload-time = "2026-05-28T12:00:37.82Z" },
+ { url = "https://files.pythonhosted.org/packages/65/56/b5f6fdb2083e32bca8a8993d89e70db114b4756c9e2c38421328126689d2/rpds_py-2026.5.1-cp314-cp314-win32.whl", hash = "sha256:2d88621d6a7d4dfa633d21abe90f280bb205274e16b1d1e61c6ad4640b2453b7", size = 209806, upload-time = "2026-05-28T12:00:39.492Z" },
+ { url = "https://files.pythonhosted.org/packages/fb/80/65a5aa96c155e611d1ed844e4e1f57f3e36b021f396d9f8585d756e6b90d/rpds_py-2026.5.1-cp314-cp314-win_amd64.whl", hash = "sha256:cef8ac28d26f4dda3533060c20fbf80a325458fa9fd23ea72a73cdfa8e978838", size = 225985, upload-time = "2026-05-28T12:00:40.94Z" },
+ { url = "https://files.pythonhosted.org/packages/27/7c/ad185212e87b05f196daef92bc5f3caf07298eb47c295b5585c3dd3093ac/rpds_py-2026.5.1-cp314-cp314-win_arm64.whl", hash = "sha256:eaaea962c68cdc68d4a533ba985ab8e9484277910bbfaa2ab3ef7732667bfed8", size = 221219, upload-time = "2026-05-28T12:00:43.15Z" },
+ { url = "https://files.pythonhosted.org/packages/23/58/e14ae18759020334646b031e708ab4158d653a938822bfb7b95ef2e93aa3/rpds_py-2026.5.1-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:21942f52dbbd5f8758bf021213d28bd45c39e873e65e2407faf5f1846f5761ad", size = 352148, upload-time = "2026-05-28T12:00:44.638Z" },
+ { url = "https://files.pythonhosted.org/packages/31/9b/5f4a1e2f960bca3ac5d052b139dd31eed97b259f9d909173821760d542e8/rpds_py-2026.5.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f414556f6e3958300ff941e40c9f97e3dc9774ddd1b3434c475d73dd354bbed3", size = 345196, upload-time = "2026-05-28T12:00:46.14Z" },
+ { url = "https://files.pythonhosted.org/packages/1a/71/1d9574d6a2fa20ab60eaa55c7467f5aa20cbc770f341a05f09c0876f59e2/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ef1013a8625c74043210190b246f5b1551e09757c1f356c6e4160ef96c5bc081", size = 374981, upload-time = "2026-05-28T12:00:47.531Z" },
+ { url = "https://files.pythonhosted.org/packages/0c/9a/37e99f4915a80aa71670263c1267f7ae0af95f53a3f61e6c3bdc016d4515/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:cc68e231a77a5f0d774ae278a1f8e55c0456501820847c1e4efb3829f3441df6", size = 379961, upload-time = "2026-05-28T12:00:49.216Z" },
+ { url = "https://files.pythonhosted.org/packages/a8/ff/6e73f74b89d2e0715e0fc86b7dde893f9a61ae2f9b256ff3bdfe41ac4e94/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9baffb505aff33acc69b422a19f77806680f3c8632227d79f48de8a810d1c2c5", size = 495965, upload-time = "2026-05-28T12:00:51.111Z" },
+ { url = "https://files.pythonhosted.org/packages/ea/e0/425faba25f59d74d4638b267f7c7a80e8649d2ef4db10a19b0c4a71e6e6f/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b8d2f912928d426e8cfa396f7f3f8d29a59e6689c86dcca3c420730c1096322b", size = 389526, upload-time = "2026-05-28T12:00:52.77Z" },
+ { url = "https://files.pythonhosted.org/packages/c6/76/7a41960e3fddae47fab43a28684d5da981401dffd88253de0944148654cb/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:90f628283be835db980c941767d41c9a27b5239e54ba0a9c1335247e82406964", size = 376190, upload-time = "2026-05-28T12:00:54.215Z" },
+ { url = "https://files.pythonhosted.org/packages/27/60/5f38dc70824fc6951b51d35377e577a3a3a4c81a6769cc5a2de25ebe0ad1/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:1ebb2f0ab7e16132995a72de805170e0203df0c3dd22e1ef1cd1fdd90bd7a131", size = 383921, upload-time = "2026-05-28T12:00:55.673Z" },
+ { url = "https://files.pythonhosted.org/packages/60/1a/d60a38caa1505f4b9483c3fbbde12c94e1079154f4f401a6da96f7e77621/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:f3df3d16ded76f1f8c9cdebd0e1ea55fdf4c23b812de189814da7cf229c22a81", size = 404766, upload-time = "2026-05-28T12:00:57.518Z" },
+ { url = "https://files.pythonhosted.org/packages/87/ff/602fd3f174d6425f0bce05ad0dfbec0e96b38d0f7d08a79af5aa20083885/rpds_py-2026.5.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:9af8905b8f854990e40d5206aa5ac58d9b0fe0b7f351ff2bb086c20f6c8c6a47", size = 551343, upload-time = "2026-05-28T12:00:58.978Z" },
+ { url = "https://files.pythonhosted.org/packages/b8/c1/1be13327acdbead3eca1fde03b6a34dbb011f1e864e217f0d32cc1779a7f/rpds_py-2026.5.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:036a36a87fb1cd3b214d11c4b3c4f7d2ddad933625dca1c900b56a057c07740a", size = 618502, upload-time = "2026-05-28T12:01:00.656Z" },
+ { url = "https://files.pythonhosted.org/packages/f3/d7/afb49b49d7f2be8b7ba1a9f0977fa5168003437b93086726f066544e8351/rpds_py-2026.5.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:62ae3853454fe9ef283a03c96c2d835d39e84b14643a9d62c82ef0fb87d702ca", size = 581916, upload-time = "2026-05-28T12:01:02.22Z" },
+ { url = "https://files.pythonhosted.org/packages/25/d1/dbef8c1f8a10f07beb62b5f054e20099fd9924b3ec001b8f0b6ac7813a85/rpds_py-2026.5.1-cp314-cp314t-win32.whl", hash = "sha256:6c3d771a46ec18b12af06ce36243a9a80b07a5d0515236332d90863ca8bb326a", size = 207855, upload-time = "2026-05-28T12:01:03.821Z" },
+ { url = "https://files.pythonhosted.org/packages/2a/72/bfa4e61ab8e7dc1c8adf397e05e6cbdd4239357bd72b248d3de662f23915/rpds_py-2026.5.1-cp314-cp314t-win_amd64.whl", hash = "sha256:c93c629be4636cf54337bd5f06c104d55e42ced54d681f6fe21ae510a65116f6", size = 225422, upload-time = "2026-05-28T12:01:05.194Z" },
+ { url = "https://files.pythonhosted.org/packages/27/3a/7b5da92b640f67b6717ccafc83cdd06bfa7ff2395c3685c68922bb54d703/rpds_py-2026.5.1-cp315-cp315-macosx_10_12_x86_64.whl", hash = "sha256:3574b55c604b8f75dacb007136508bbc0db406e626301778096a133327e7f2fb", size = 349576, upload-time = "2026-05-28T12:01:06.722Z" },
+ { url = "https://files.pythonhosted.org/packages/d7/8a/2aafd7ad355a1bd48ca76e2262b74b15e6432b5a1efe150efd4d779cd55d/rpds_py-2026.5.1-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:94068eb3ae6d43f5a786b7db96a406a34e6d5c24489feef32fd6e8946ea7b291", size = 343640, upload-time = "2026-05-28T12:01:08.441Z" },
+ { url = "https://files.pythonhosted.org/packages/f7/7d/6c9523c1abbe840a1b7fba3c516d48e1d3487cc80fea4366c4071cf56784/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f3a5b10e8ce894825f380a8f1b6444cf73c294dfea62afbb2d13e3a9e630cec1", size = 375322, upload-time = "2026-05-28T12:01:09.934Z" },
+ { url = "https://files.pythonhosted.org/packages/5a/5d/0b7b03fb1dc509321f01de3149784ab773e34c8573022029af8076afcb9c/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:fc09f82e63d4bcd58149572f857a431bae851dc747e313c3b5bdf7abb907fda8", size = 379066, upload-time = "2026-05-28T12:01:11.48Z" },
+ { url = "https://files.pythonhosted.org/packages/d7/e2/8ef6012999ebf1cb1c22f876d9ce5e63d960fd4631d2af3202d3f480aa25/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e10464d17df3b582745c25cec695cb9558bca2cb6ddb631aee1787fc72c767b2", size = 494586, upload-time = "2026-05-28T12:01:13.051Z" },
+ { url = "https://files.pythonhosted.org/packages/80/af/1eeb029bec67582c226b7809172207cd005073af4ebd906e65ff494f4983/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ba05adbf15d994c38ec0b7ab32e858e5110c21e9009a00a86545fd220f84e038", size = 388415, upload-time = "2026-05-28T12:01:14.631Z" },
+ { url = "https://files.pythonhosted.org/packages/18/23/ffbe10711c4d766c1cab0557d6906c074f795814863c67b351355d29354a/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:77c004fdc7b891967106f78ddfd7b076bfe6813c6139c6fff6aed3bcaa960b26", size = 372427, upload-time = "2026-05-28T12:01:16.153Z" },
+ { url = "https://files.pythonhosted.org/packages/bd/3a/30ba4a6ad457e5b070c18d742a33fb77d8d922b565cc881f8a5313d63bfe/rpds_py-2026.5.1-cp315-cp315-manylinux_2_31_riscv64.whl", hash = "sha256:83bcf894486c9d78dd290d3c0124ff6dd8875d3025e2090a8ec49fcc37c55fdd", size = 383615, upload-time = "2026-05-28T12:01:17.809Z" },
+ { url = "https://files.pythonhosted.org/packages/d3/69/62e242b53ce39c0814bd24e1a6e6eba6c92be716277745f317f9540a2e7b/rpds_py-2026.5.1-cp315-cp315-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c3df104083952a0e0c6f10de33e440eabe98fb6317d23e1a58c68f6df08d01b9", size = 402786, upload-time = "2026-05-28T12:01:19.419Z" },
+ { url = "https://files.pythonhosted.org/packages/38/c1/a770b9c186928a1ed0f7e6d7ae50e7f3950ed23e3f9e366dbc8e38cb55de/rpds_py-2026.5.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:980450826cf22e133c57e0835070bdd0dd3f73b9b708c3ce223def2cb9469e14", size = 551583, upload-time = "2026-05-28T12:01:21.013Z" },
+ { url = "https://files.pythonhosted.org/packages/21/7c/68e8579b95375b70d2a963103c42e705856cdb98569258bd807f4423891c/rpds_py-2026.5.1-cp315-cp315-musllinux_1_2_i686.whl", hash = "sha256:205dde846f24332ab0c1188699a043b8d165b79bb84529ce272c45048ff6be01", size = 616941, upload-time = "2026-05-28T12:01:22.548Z" },
+ { url = "https://files.pythonhosted.org/packages/70/a1/a6135aed5730ff03ab957182259987ac11e55fb392a28dc6f0592048a280/rpds_py-2026.5.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:3966b82dd563176396df030f3dd52a6e54cb69b718e95e78bd555ed3d1e0185d", size = 578349, upload-time = "2026-05-28T12:01:24.118Z" },
+ { url = "https://files.pythonhosted.org/packages/09/6e/f24201a76a84e6c49d0bdfdfcb735210e21701e9b21c5bfc0ba497dd62f6/rpds_py-2026.5.1-cp315-cp315-win32.whl", hash = "sha256:7818f8d0a415be74d2be3590b0a1c1f463a642f4d0217e7d10602dceef5b79aa", size = 209922, upload-time = "2026-05-28T12:01:25.522Z" },
+ { url = "https://files.pythonhosted.org/packages/9e/e4/966bc240bb0485fc265278f6de44d05834bf0b3618886e0b22e33d54c49a/rpds_py-2026.5.1-cp315-cp315-win_amd64.whl", hash = "sha256:b3cc20c0d800af78fd0fac68086e28c1856cec51ea528bb81ea851aa40d39325", size = 226003, upload-time = "2026-05-28T12:01:27.062Z" },
+ { url = "https://files.pythonhosted.org/packages/5c/5c/a15a59269cd5e74472734516c73795c15eccfc841b3d4b0228c3f53f19d0/rpds_py-2026.5.1-cp315-cp315-win_arm64.whl", hash = "sha256:3609e9939a8a76cd904cf98a3f1f13b5dc7e150adeaee89e0ea09652ea213e16", size = 221245, upload-time = "2026-05-28T12:01:28.51Z" },
+ { url = "https://files.pythonhosted.org/packages/e0/22/135ce03804e179a71ceb13be095deda4a279bc88f7a6b8fa161c5ad44e12/rpds_py-2026.5.1-cp315-cp315t-macosx_10_12_x86_64.whl", hash = "sha256:5d333a7127d4b307601ac37792bee01bb95c867cbfacf21b6375b804d6bbd723", size = 352015, upload-time = "2026-05-28T12:01:30.214Z" },
+ { url = "https://files.pythonhosted.org/packages/3b/5f/f1f6d2652eb9d848f6eb369d8db83a2da6249bb49ad2c2a48f45d54538d3/rpds_py-2026.5.1-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:b5f077b44a4f7808520f66dae234988d867deb9aed9be5da057ce9ba831b2a41", size = 345016, upload-time = "2026-05-28T12:01:31.656Z" },
+ { url = "https://files.pythonhosted.org/packages/88/66/b74182775691ea2290c99e52ac8d5db844e56fbec90ce421f107658c8314/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:55d8f9b7b78c9538fc9e04e82ec0e888ff0c3cffcfad152c77e57cd09351a98a", size = 374775, upload-time = "2026-05-28T12:01:33.136Z" },
+ { url = "https://files.pythonhosted.org/packages/ff/8f/15e5a61d9f0a43902d36561d4f07cae6ae9f4716be825159fd72717f33af/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e3a8ae58895ac107ed934a6bf51e5846f95c53b9b940c2c6d310838fd5846358", size = 380270, upload-time = "2026-05-28T12:01:34.574Z" },
+ { url = "https://files.pythonhosted.org/packages/02/c3/f859b12763a80540cdf2af0f15b19904cf756a71d7bdd3f82ff3e5b1bbf9/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0957cf3c2b8632ec7aaebffebea8005b353cc2a237b6e2ae3c2cac0820704cfb", size = 495285, upload-time = "2026-05-28T12:01:36.127Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/c7/ff27c2ac8411d30b03b1829fd88cae8dad1a4d0da48dd25e57c4038042e6/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c396c1304de421050b3681ea70f371874b54d41b0151e96109758144c231e30b", size = 389581, upload-time = "2026-05-28T12:01:37.635Z" },
+ { url = "https://files.pythonhosted.org/packages/6e/67/fe92ee32a6cc05c77228a2f8b1762e7124f386ec20ff83d0757b762d58d0/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:aad1bff7f666b9598e573815affd666aac6a13a585dde336f843e33350c7fadc", size = 376041, upload-time = "2026-05-28T12:01:39.307Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/91/b4d6685c27aba55bd82f25b278be8237038117d05f9659a6213ad3408130/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_31_riscv64.whl", hash = "sha256:656a042550878f12d45752452d47094b7cfe5ad1e9d7b87b5a22ad3ae5ff8015", size = 383946, upload-time = "2026-05-28T12:01:41.043Z" },
+ { url = "https://files.pythonhosted.org/packages/bd/79/2c1d832a53c8e0f8e98fc970ec257b950fecd4f62be2ab7182b500a0cbc8/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:73c4bd4f70294737b5206a3e8e30ccadbf8a60301831c8ea23eec5dbeea1ecfa", size = 405526, upload-time = "2026-05-28T12:01:43.032Z" },
+ { url = "https://files.pythonhosted.org/packages/78/c4/c98117b03c6a8581ab2c2dfccfe9a5ad82bd8128a3c28b46a6ad2d97c393/rpds_py-2026.5.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:43bca78665423cabae77146f2fe7ce55272b6c8d55d82cca83effd42c7e13972", size = 551165, upload-time = "2026-05-28T12:01:44.648Z" },
+ { url = "https://files.pythonhosted.org/packages/3b/c1/bc479ca069200af730881b1bd525e3114b2b391a351509fcb1b772f28086/rpds_py-2026.5.1-cp315-cp315t-musllinux_1_2_i686.whl", hash = "sha256:42d0f20e85e549c870749d0e247f0c10d318a45b7e9676d575d2dcb04a1b2e66", size = 618778, upload-time = "2026-05-28T12:01:46.337Z" },
+ { url = "https://files.pythonhosted.org/packages/77/65/38ab2f90df44c2febfb63cc10ced40763d9b4bc94d173e734528663fe7f5/rpds_py-2026.5.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:b1be5c35683684d5331b93600c210e8367c254683d8a6df6bd21bd2da3a334fb", size = 581839, upload-time = "2026-05-28T12:01:48.109Z" },
+ { url = "https://files.pythonhosted.org/packages/15/2d/ce1f605fe036aadd460e5822e578c6c7ec3a860936cca37d6e0f299daa77/rpds_py-2026.5.1-cp315-cp315t-win32.whl", hash = "sha256:75808f6c38ce7749bb68cc2770161aae5045e6c6f6781a9782e74b93304399df", size = 207866, upload-time = "2026-05-28T12:01:49.648Z" },
+ { url = "https://files.pythonhosted.org/packages/79/cb/966040123eb102371559746908ef2c9471f4d43e17ec9a645a2258dab64b/rpds_py-2026.5.1-cp315-cp315t-win_amd64.whl", hash = "sha256:90bd6630002a1c7f09e7843dd79f0d24f3d2897cc25a753480917865d14f15b3", size = 225441, upload-time = "2026-05-28T12:01:51.408Z" },
+]
+
+[[package]]
+name = "ruff"
+version = "0.15.20"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/43/dc/35b341fc554ba02f217fc10da57d1a75168cfbcf75b0ef2202176d4c4f2d/ruff-0.15.20.tar.gz", hash = "sha256:1416eb04349192646b54de98f146c4f59afe37d0decfc02c3cbbf396f3a28566", size = 4755489, upload-time = "2026-06-25T17:20:37.578Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/94/d9/2d5014f0253ba541d2061d9fa7193f48e941c8b21bb88a7ff9bbe0bd0596/ruff-0.15.20-py3-none-linux_armv6l.whl", hash = "sha256:00e188c53e499c3c1637f73c91dcf2fb56d576cab76ce1be50a27c4e80e37078", size = 10839665, upload-time = "2026-06-25T17:19:44.702Z" },
+ { url = "https://files.pythonhosted.org/packages/c6/d3/ac1798ba64f670698867fcfc591d50e7e421bef137db564858f619a30fcf/ruff-0.15.20-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:9ebd1fd9b9c95fc0bd7b2761aebec1f030013d2e193a2901b224af68fe47251b", size = 11208649, upload-time = "2026-06-25T17:19:48.787Z" },
+ { url = "https://files.pythonhosted.org/packages/47/47/d3ac899991202095dfcf3d5176be4272642be3cf981a2f1a30f72a2afb95/ruff-0.15.20-py3-none-macosx_11_0_arm64.whl", hash = "sha256:c5b16cdd67ca108185cd36dce98c576350c03b1660a751de725fb049193a0632", size = 10622638, upload-time = "2026-06-25T17:19:51.354Z" },
+ { url = "https://files.pythonhosted.org/packages/33/13/4e043fe30aa94d4ff5213a9881fc296d12960f5971b234a5263fdc225312/ruff-0.15.20-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3413bb3c3d2ca6a8208f1f4809cd2dca3c6de6d0b491c0e70847672bde6e6efd", size = 10984227, upload-time = "2026-06-25T17:19:54.044Z" },
+ { url = "https://files.pythonhosted.org/packages/76/e6/92e7bf40388bc5800073b96564f56264f7e48bfd1a498f5ced6ae6d5a769/ruff-0.15.20-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bd7ec42b3bb3da066488db093308a69c4ac5ee6d2af333a86ba6e2eb2e7dd44b", size = 10622882, upload-time = "2026-06-25T17:19:57.037Z" },
+ { url = "https://files.pythonhosted.org/packages/13/7a/43460be3f24495a3aa46d4b16873e2c4941b3b5f0b00cf88c03b7b94b339/ruff-0.15.20-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e1a36ad0eb77fba9aabfb69ede54de6f376d04ac18ebea022847046d340a8267", size = 11474808, upload-time = "2026-06-25T17:20:00.357Z" },
+ { url = "https://files.pythonhosted.org/packages/27/a0/f37077884873221c6b33b4ab49eb18f9f88e54a16a25a5bca59bef46dd66/ruff-0.15.20-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b6df3b1e4610432f0386dba04d853b5f08cbbc903410c6fcc02f620f05aff53c", size = 12293094, upload-time = "2026-06-25T17:20:03.446Z" },
+ { url = "https://files.pythonhosted.org/packages/a6/74/165545b60256a9704c21ac0ec4a0d07933b320812f9584836c9f4aca4292/ruff-0.15.20-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e89f198a1ea6ef0d727c1cf16088bc91a6cb0ab947dedc966715691647186eae", size = 11526176, upload-time = "2026-06-25T17:20:06.301Z" },
+ { url = "https://files.pythonhosted.org/packages/86/b1/a976a136d40ade83ce743578399865f57001003a409acadc0ecbb3051082/ruff-0.15.20-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:309809086c2acb67624950a3c8133e80f32d0d3e27106c0cd60ff26657c9f24b", size = 11520767, upload-time = "2026-06-25T17:20:09.191Z" },
+ { url = "https://files.pythonhosted.org/packages/19/0f/f032696cb01c9b54c0263fa393474d7758f1cdc021a01b04e3cbc2500999/ruff-0.15.20-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:2d2374caa2f2c2f9e2b7da0a50802cfb8b79f55a9b5e49379f564544fbf56487", size = 11500132, upload-time = "2026-06-25T17:20:13.602Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/f4/51b1a14bc69e8c224b15dab9cce8e99b425e0455d462caa2b3c9be2b6a8e/ruff-0.15.20-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:a1ed17b65293e0c2f22fc387bc13198a5de94bf4429589b0ff6946b0feaf21a3", size = 10943828, upload-time = "2026-06-25T17:20:16.635Z" },
+ { url = "https://files.pythonhosted.org/packages/71/4b/fe267640783cd02bf6c5cc290b1df1051be2ec294c678b5c15fe19e52343/ruff-0.15.20-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:f701305e66b38ea6c91882490eb73459796808e4c6362a1b765255e0cdcd4053", size = 10645418, upload-time = "2026-06-25T17:20:19.4Z" },
+ { url = "https://files.pythonhosted.org/packages/b0/c0/a65aa4ec2f5e87a1df32dc3ec1fede434fe3dfd5cbcf3b503cafc676ab54/ruff-0.15.20-py3-none-musllinux_1_2_i686.whl", hash = "sha256:5b9c0c367ad8e5d0d5b5b8537864c469a0a0e55417aadfbeca41fa61333be9f4", size = 11211770, upload-time = "2026-06-25T17:20:22.033Z" },
+ { url = "https://files.pythonhosted.org/packages/5a/a4/0caa331d954ae2723d729d351c989cb4ca8b6077d5c6c2cb6de75e98c041/ruff-0.15.20-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:01cc00dd58f0df339d0e902219dd53990ea99996a0344e5d9cc8d45d5307e460", size = 11618698, upload-time = "2026-06-25T17:20:25.259Z" },
+ { url = "https://files.pythonhosted.org/packages/10/9b/5f14927848d2fd4aa891fd88d883788c5a7baba561c7874732364045708c/ruff-0.15.20-py3-none-win32.whl", hash = "sha256:ed65ef510e43a137207e0f01cfcf998aeddb1aeeda5c9d35023e910284d7cf21", size = 10857322, upload-time = "2026-06-25T17:20:28.612Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/f0/fe47c501f9dea92a26d788ff98bb5d92ed4cb4c88792c5c88af6b697dc8e/ruff-0.15.20-py3-none-win_amd64.whl", hash = "sha256:a525c81c70fb0380344dd1d8745d8cc1c890b7fc94a58d5a07bd8eb9557b8415", size = 11993274, upload-time = "2026-06-25T17:20:31.871Z" },
+ { url = "https://files.pythonhosted.org/packages/d7/2b/9555445e1201d92b3195f45cdb153a0b68f24e0a4273f6e3d5ab46e212bb/ruff-0.15.20-py3-none-win_arm64.whl", hash = "sha256:2f5b2a6d614e8700388806a14996c40fab2c47b819ef57d790a34878858ed9ca", size = 11343498, upload-time = "2026-06-25T17:20:35.03Z" },
+]
+
+[[package]]
+name = "scarf"
+source = { editable = "." }
+dependencies = [
+ { name = "h5py" },
+ { name = "hnswlib" },
+ { name = "leidenalg" },
+ { name = "loguru" },
+ { name = "networkx" },
+ { name = "numba" },
+ { name = "numcodecs" },
+ { name = "numpy" },
+ { name = "obstore" },
+ { name = "packaging" },
+ { name = "pandas" },
+ { name = "scikit-learn" },
+ { name = "scikit-network" },
+ { name = "scipy" },
+ { name = "setuptools" },
+ { name = "sgtsnepi" },
+ { name = "statsmodels" },
+ { name = "threadpoolctl" },
+ { name = "tqdm" },
+ { name = "umap-learn" },
+ { name = "zarr" },
+]
+
+[package.optional-dependencies]
+docs = [
+ { name = "jedi" },
+ { name = "jinja2" },
+ { name = "myst-nb" },
+ { name = "myst-parser" },
+ { name = "nbclient" },
+ { name = "sphinx" },
+ { name = "sphinx-autodoc-typehints" },
+ { name = "sphinx-book-theme" },
+ { name = "sphinx-copybutton" },
+ { name = "sphinx-external-toc" },
+ { name = "sphinx-tabs" },
+ { name = "topacedo" },
+]
+extra = [
+ { name = "anndata" },
+ { name = "datashader" },
+ { name = "ipython-autotime" },
+ { name = "ipywidgets" },
+ { name = "kneed" },
+ { name = "matplotlib" },
+ { name = "requests" },
+ { name = "seaborn" },
+]
+test = [
+ { name = "anndata" },
+ { name = "pytest" },
+ { name = "pytest-cov" },
+ { name = "pytest-xdist" },
+ { name = "topacedo" },
+]
+
+[package.dev-dependencies]
+dev = [
+ { name = "mypy" },
+ { name = "ruff" },
+]
+profiling = [
+ { name = "anndata" },
+ { name = "igraph" },
+ { name = "modal" },
+ { name = "pydantic" },
+]
+
+[package.metadata]
+requires-dist = [
+ { name = "anndata", marker = "extra == 'extra'", specifier = ">=0.12" },
+ { name = "anndata", marker = "extra == 'test'", specifier = ">=0.12" },
+ { name = "datashader", marker = "extra == 'extra'", specifier = ">=0.18" },
+ { name = "h5py", specifier = ">=3.11" },
+ { name = "hnswlib", specifier = ">=0.8" },
+ { name = "ipython-autotime", marker = "extra == 'extra'", specifier = ">=0.3" },
+ { name = "ipywidgets", marker = "extra == 'extra'", specifier = ">=8" },
+ { name = "jedi", marker = "extra == 'docs'" },
+ { name = "jinja2", marker = "extra == 'docs'" },
+ { name = "kneed", marker = "extra == 'extra'", specifier = ">=0.8" },
+ { name = "leidenalg", specifier = ">=0.12" },
+ { name = "loguru", specifier = ">=0.7" },
+ { name = "matplotlib", marker = "extra == 'extra'", specifier = ">=3.9" },
+ { name = "myst-nb", marker = "extra == 'docs'" },
+ { name = "myst-parser", marker = "extra == 'docs'" },
+ { name = "nbclient", marker = "extra == 'docs'" },
+ { name = "networkx", specifier = ">=3" },
+ { name = "numba", specifier = ">=0.60" },
+ { name = "numcodecs", specifier = ">=0.16" },
+ { name = "numpy", specifier = ">=2" },
+ { name = "obstore" },
+ { name = "packaging" },
+ { name = "pandas", specifier = ">=2.2.2" },
+ { name = "pytest", marker = "extra == 'test'", specifier = ">=7.0" },
+ { name = "pytest-cov", marker = "extra == 'test'", specifier = ">=4" },
+ { name = "pytest-xdist", marker = "extra == 'test'", specifier = ">=3" },
+ { name = "requests", marker = "extra == 'extra'", specifier = ">=2.28" },
+ { name = "scikit-learn", specifier = ">=1.5" },
+ { name = "scikit-network", specifier = ">=0.33.1" },
+ { name = "scipy", specifier = ">=1.13,!=1.17.0" },
+ { name = "seaborn", marker = "extra == 'extra'", specifier = ">=0.13" },
+ { name = "setuptools", specifier = ">=61" },
+ { name = "sgtsnepi" },
+ { name = "sphinx", marker = "extra == 'docs'" },
+ { name = "sphinx-autodoc-typehints", marker = "extra == 'docs'" },
+ { name = "sphinx-book-theme", marker = "extra == 'docs'" },
+ { name = "sphinx-copybutton", marker = "extra == 'docs'" },
+ { name = "sphinx-external-toc", marker = "extra == 'docs'" },
+ { name = "sphinx-tabs", marker = "extra == 'docs'" },
+ { name = "statsmodels", specifier = ">=0.14.2" },
+ { name = "threadpoolctl" },
+ { name = "topacedo", marker = "extra == 'docs'" },
+ { name = "topacedo", marker = "extra == 'test'", specifier = ">=0.2.10" },
+ { name = "tqdm", specifier = ">=4.60" },
+ { name = "umap-learn", specifier = ">=0.5.6" },
+ { name = "zarr", specifier = ">=3.1.2" },
+]
+provides-extras = ["extra", "test", "docs"]
+
+[package.metadata.requires-dev]
+dev = [
+ { name = "mypy", specifier = ">=2.1.0" },
+ { name = "ruff", specifier = ">=0.15.20" },
+]
+profiling = [
+ { name = "anndata", specifier = ">=0.12.18" },
+ { name = "igraph", specifier = ">=1.0.0" },
+ { name = "modal", specifier = ">=1.5.2" },
+ { name = "pydantic", specifier = ">=2.13.4" },
+]
+
+[[package]]
+name = "scikit-learn"
+version = "1.9.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "joblib" },
+ { name = "narwhals" },
+ { name = "numpy" },
+ { name = "scipy" },
+ { name = "threadpoolctl" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/fa/6f/37092bdb25f712817231799fc5674d8e704066a8a70c1d2d40517e18b4ab/scikit_learn-1.9.0.tar.gz", hash = "sha256:8833266989d3a5110178a9fae30783675460724d0e1efb13b14901d2c660c557", size = 7750767, upload-time = "2026-06-02T11:54:32.706Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ac/20/75f915ff375d6249e6550ac740fdbbd66159a068fd3af1400ff62036b07a/scikit_learn-1.9.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:2bd41b0d201bc81575531b96b713d3eb5e5f50fb0b82101ff0f92294fdc236ac", size = 8741122, upload-time = "2026-06-02T11:53:24.08Z" },
+ { url = "https://files.pythonhosted.org/packages/cc/d5/2b5148f2279196775e1db2aeb85d14b70ac80e7e32b3b28e7ebeafb0901d/scikit_learn-1.9.0-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:5be45aa4a42a68a533913a6ed736cf309de2226411c79ef8d609a5456f1939b1", size = 8261512, upload-time = "2026-06-02T11:53:27.183Z" },
+ { url = "https://files.pythonhosted.org/packages/a0/ee/5adbc77656b71f9456a2f5a7a9fdb4bcf9207a6b962889f1c2f9323afa4e/scikit_learn-1.9.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5e50ed4da51974e86e940690e9a3d82e729b62b5a49f7c9bac534d515d39d86f", size = 8837603, upload-time = "2026-06-02T11:53:30.328Z" },
+ { url = "https://files.pythonhosted.org/packages/6c/c2/63fdda36c56437eeb44aaf9493c8bcd62ce230ab1598924fc626ffbfa943/scikit_learn-1.9.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:056c92bb67ad4c28463c2f2653d9701449201e7e7a9e94e321be0f71c4fef2b8", size = 9132097, upload-time = "2026-06-02T11:53:33.456Z" },
+ { url = "https://files.pythonhosted.org/packages/83/a4/c8e67227c680e2259c8864ae72ff48b06e16a6f51253a22167aa02a8aa4e/scikit_learn-1.9.0-cp312-cp312-win_amd64.whl", hash = "sha256:4306775fad04cc4b472a1b15af1ae9cede1540fbfcc17fbce3767cd8dc7ae283", size = 8211173, upload-time = "2026-06-02T11:53:36.602Z" },
+ { url = "https://files.pythonhosted.org/packages/cf/fd/3c0863792e98e67e9184aa4029288a175935eb65443afcd30d4f143450cf/scikit_learn-1.9.0-cp312-cp312-win_arm64.whl", hash = "sha256:26e22435f63bcdcf396b574273f29f13dd531f5ea035801f5be10ba1540a4e60", size = 7867451, upload-time = "2026-06-02T11:53:39.075Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/01/cf3310626b6d48d3e9be69a1223f9180360b5e6edb045f50fade723ce494/scikit_learn-1.9.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:80746d63bd4b6eaca54d36fe5feaf4d28bb38dc6f9470f81c7cad7c40155f119", size = 8705188, upload-time = "2026-06-02T11:53:41.964Z" },
+ { url = "https://files.pythonhosted.org/packages/3e/04/5acd7ae280c5f93b6ac5ef6cdec14eef4c8d1cd91d85b3292989c94d96b1/scikit_learn-1.9.0-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:5b934c45c252844a91d69fda3a34cff5e7307e1db10d77cb10a3980312c74713", size = 8228299, upload-time = "2026-06-02T11:53:44.817Z" },
+ { url = "https://files.pythonhosted.org/packages/0c/39/ffe829a5b8ecb40a518724a997794657fdc354ada5e8fe8e64d998c0bac9/scikit_learn-1.9.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:38c3dcb9a1ffb85505ec53d54c7b4aea0cff70050425a7760c2af661ac85df05", size = 8789690, upload-time = "2026-06-02T11:53:47.461Z" },
+ { url = "https://files.pythonhosted.org/packages/1f/88/8dab5de10c638c083772a6be83a3d8106ced492f74a928c8693638e5bb50/scikit_learn-1.9.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:da76d09304a4706db7cc1e3ebaa3b6b98a67365cc11d2996c4f1e58ba47df714", size = 9087723, upload-time = "2026-06-02T11:53:50.702Z" },
+ { url = "https://files.pythonhosted.org/packages/20/3f/7917ca72464038f6240ec70c29f94862d08a34a74291ae4d4ec5eb8186a0/scikit_learn-1.9.0-cp313-cp313-win_amd64.whl", hash = "sha256:5808d98f15c6bf6d9d96d2348c1997392a5888ce7097e664105f930c4bca1277", size = 8184330, upload-time = "2026-06-02T11:53:53.396Z" },
+ { url = "https://files.pythonhosted.org/packages/78/c7/15739eb2f61fda3c54639e9942414e5a19ad8a8d1f5a3266afad7cb7df80/scikit_learn-1.9.0-cp313-cp313-win_arm64.whl", hash = "sha256:d77f54c017633791bc0225a43e2f8d03745fdcfe4880268fcc4df15f505dec2e", size = 7840653, upload-time = "2026-06-02T11:53:56.035Z" },
+ { url = "https://files.pythonhosted.org/packages/f4/7d/c9a35cf59b20a86fec24d306f1547b78dec194b08d367ce2a3e4854169d9/scikit_learn-1.9.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:9656acd4e93f74e0b66c8a36c88830a99252dfa900044d36bc2212ae89a47162", size = 8713289, upload-time = "2026-06-02T11:53:58.788Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/a7/552a7821597c632b907f7bfe8f36f9f572777af8ef8a48353041cf8e091a/scikit_learn-1.9.0-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:24360002ae845e7866522b0a5bbf690802e7bc388cac8663502e78aa98598aa2", size = 8245141, upload-time = "2026-06-02T11:54:01.694Z" },
+ { url = "https://files.pythonhosted.org/packages/7d/79/f4a0c4fe9711154cddabf913471153af79056382ddc612cfe5ee0ff4b72e/scikit_learn-1.9.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5162ad10a418c8a282dde04c9aa06965de3e9a65f33c1440c0ae69bb1a09d913", size = 8847671, upload-time = "2026-06-02T11:54:04.448Z" },
+ { url = "https://files.pythonhosted.org/packages/f0/af/4d72d9e475ac83719160c662619e4bf7b95c19507cd582e7d0167a3c3dae/scikit_learn-1.9.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1fea2cc5677ab49d6f5bade978c866da44957b712d92e9635e8b4f723013c3cb", size = 9118104, upload-time = "2026-06-02T11:54:07.205Z" },
+ { url = "https://files.pythonhosted.org/packages/a2/d5/6a58eea2cb9abbb9b3f2bb8b2cfb3243d1152d69f442d256c7af71304769/scikit_learn-1.9.0-cp314-cp314-win_amd64.whl", hash = "sha256:64fa347efc1c839c487433e40c5144d38c336e8a2b59c81aa8660373945c2673", size = 8290674, upload-time = "2026-06-02T11:54:10.087Z" },
+ { url = "https://files.pythonhosted.org/packages/65/5b/d4c879cf358f1187141cf90ced473f087183489090244f50c124a2ee478b/scikit_learn-1.9.0-cp314-cp314-win_arm64.whl", hash = "sha256:1b944b6db288f6b926e3650026ddafb988929de95d11fc2cc5fa117773c9ba42", size = 7978807, upload-time = "2026-06-02T11:54:12.769Z" },
+ { url = "https://files.pythonhosted.org/packages/8a/43/bfae3121ec67ae09150d453c442c7c1cc166e9aefe056e6ab3b7728a5cfc/scikit_learn-1.9.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:4ccacf04ca5f4b492158a5f28afe0ace43f81b2571e4b9a66d34848b46128949", size = 9031941, upload-time = "2026-06-02T11:54:15.436Z" },
+ { url = "https://files.pythonhosted.org/packages/75/b0/20a4546eb17f3b25d3c66df15810411c14ed5065bcfab50b53c96fb627b2/scikit_learn-1.9.0-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:ee1a8db2c18c08e34c7412d4b10be1cac214cd4ea7dc9715a6a327eb49a37c96", size = 8613528, upload-time = "2026-06-02T11:54:18.842Z" },
+ { url = "https://files.pythonhosted.org/packages/18/3c/e440e039bb82cd19004edaaad00acbde0fb9b461083c3ecf37941c557312/scikit_learn-1.9.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:147e9329ef0e39f75d4cffa02b2aa48d827832684926cd5210d9a2cb5c57246b", size = 8855050, upload-time = "2026-06-02T11:54:21.699Z" },
+ { url = "https://files.pythonhosted.org/packages/43/26/b341b8dab5998da6270a3a42c2152c578501354d36f944b5856757035ef8/scikit_learn-1.9.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5bad8f8b9950321b54c965fdcbac6c6c55e79e16646b49977bcf3668d3870a1a", size = 9097190, upload-time = "2026-06-02T11:54:24.454Z" },
+ { url = "https://files.pythonhosted.org/packages/fb/de/b650b4d69b84468cfa2e28a3ff7b8103743029e6446ce1a97fe060ef688c/scikit_learn-1.9.0-cp314-cp314t-win_amd64.whl", hash = "sha256:78fc56eafd4edb9575d2d8950d1dd152061abb573341a1cb7e099fc40f6c6666", size = 8963204, upload-time = "2026-06-02T11:54:27.428Z" },
+ { url = "https://files.pythonhosted.org/packages/ee/f3/ff83d76d7418112e5a61326443cdda87be3545dd8d6599c95b2481a4419e/scikit_learn-1.9.0-cp314-cp314t-win_arm64.whl", hash = "sha256:051075bda8b7aab87b1906ab3d4740a1e1224a19d7b3781a576736edc94e76aa", size = 8222661, upload-time = "2026-06-02T11:54:30.192Z" },
+]
+
+[[package]]
+name = "scikit-network"
+version = "0.33.5"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+ { name = "scipy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/46/6d/28b00fbef9ff7d8ba31861bf16705a1a74a1696fb65aab2a7c584f966bec/scikit_network-0.33.5.tar.gz", hash = "sha256:ae2149d9a280fdc4bbadd5f8a7b17c8af61c054bc3f834792bc61483e6783c12", size = 1784205, upload-time = "2025-11-19T09:45:14.402Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/72/32/f092fca9a3ae256e0608ec6a4d8830023ea4ae478c2e27e94cf5802824f0/scikit_network-0.33.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ced625228be1632595d11adaf22d61d1d4c788909cd4d8c364e720b2814aac7f", size = 2874698, upload-time = "2025-11-19T09:44:52.765Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/c7/5b2ce72f93b422af48a2b755fbcbab271bf980a6b46484f754d63978f1ff/scikit_network-0.33.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:808b625d28005c24b47cbdf65f780a3a355aa4080992e6b78f03434e873d06e6", size = 2854224, upload-time = "2025-11-19T09:44:55.574Z" },
+ { url = "https://files.pythonhosted.org/packages/86/02/974ae67f493ccf988108894e8a9dedfd00ca5113d2848e0b9fc2a4d18824/scikit_network-0.33.5-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f9f2d98059cd79bdb935ff6a638f1c3a12b0b1cb7ace9e1d3fb35476eeeaabaa", size = 7924650, upload-time = "2025-11-19T09:44:57.704Z" },
+ { url = "https://files.pythonhosted.org/packages/8d/34/b67e48e111916a6f09fe29a971a9716c14f78525e0ab7c46e6a6538cf2f6/scikit_network-0.33.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:aac2d3bc14214f02dac300624f5ec2650af9a98b52f304d300f0ec2813a0e544", size = 8012322, upload-time = "2025-11-19T09:45:00.079Z" },
+ { url = "https://files.pythonhosted.org/packages/c3/a2/49293b53b837b3248d19fffd41a9fd73dcb2eeabbab890cc4c8aa237b545/scikit_network-0.33.5-cp312-cp312-win_amd64.whl", hash = "sha256:2866b16aed9ef25ba42cb2f2e44ef2ad079337f336ce48d0604b55fa4af87688", size = 2746491, upload-time = "2025-11-19T09:45:01.997Z" },
+ { url = "https://files.pythonhosted.org/packages/d6/cd/0069244e970d27fa0ab0512394295a106605f00c271e85618182460d2c92/scikit_network-0.33.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:aa8b490a777e081dd6f69627a26b50cc4012f27fff683a1e4828819a88a5dcf2", size = 2862564, upload-time = "2025-11-19T09:45:03.93Z" },
+ { url = "https://files.pythonhosted.org/packages/51/37/85454864e50a65e528fe3b15eb3b41eb68b0c7f6d5c51c220b3198622ede/scikit_network-0.33.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:3615d073ba9ae1ae30dda2de747474cd23c86cededa82b317471ee9f9bebd1b2", size = 2842521, upload-time = "2025-11-19T09:45:06.31Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/b9/4023f35e430b51020f17b0f1d8933768cf1ed7cd1623cc089f5543048983/scikit_network-0.33.5-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2408d3f4c81256a3193d536aad4a6ffcfbb05d096abe6a9cc0b6b5e275df876d", size = 7871695, upload-time = "2025-11-19T09:45:08.101Z" },
+ { url = "https://files.pythonhosted.org/packages/8e/ec/e50755f7459130ba745c42c37665c5ae9a7c7e357f43400b5b8b966f902e/scikit_network-0.33.5-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:526490a1e0e8e49ad0f4cca193f581d60082a2fa8c9a825eb0b6936050b0d02b", size = 7968762, upload-time = "2025-11-19T09:45:10.347Z" },
+ { url = "https://files.pythonhosted.org/packages/45/2a/616974a0adb9d04a791570e9371caaef14c54f8806b04dd59c80a7b60289/scikit_network-0.33.5-cp313-cp313-win_amd64.whl", hash = "sha256:722c15fcede5e07ac008354bbd6ef375e0f5bf1fd52bd40271775997be2fb715", size = 2742482, upload-time = "2025-11-19T09:45:12.843Z" },
+]
+
+[[package]]
+name = "scipy"
+version = "1.18.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/a7/25/c2700dfaf6442b4effaa91af24ebce5dc9d31bb4a69706313aae70d72cd0/scipy-1.18.0.tar.gz", hash = "sha256:67b2ad2ad54c72ca6d04975a9b2df8c3638c34ddd5b28738e94fc2b57929d378", size = 30774447, upload-time = "2026-06-19T15:01:43.456Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/6a/19/ca10ead60b0acc80b2b833c2c4a4f2ff753d0f58b811f70d911c7e94a25c/scipy-1.18.0-cp312-cp312-macosx_10_15_x86_64.whl", hash = "sha256:7bd21faaf5a1a3b2eff922d02db5f191b99a6518db9078a8fb23169f6d22259a", size = 31056519, upload-time = "2026-06-19T14:59:45.203Z" },
+ { url = "https://files.pythonhosted.org/packages/96/72/1e6442a00cd2924d361aa1b642ab6373ec35c6fabf311a760be9f76e0f13/scipy-1.18.0-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:265915e79107de9f946b855e50d7470d5893ec3f54b342e1aa6201cbdcd8bb6b", size = 28681889, upload-time = "2026-06-19T14:59:48.103Z" },
+ { url = "https://files.pythonhosted.org/packages/9b/2d/11dd93d21e147a73ba22bd75c0b9208d3a2e0ec76d53170ce7d9029b1015/scipy-1.18.0-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:9ab7b758be6940954a713ee466e2043e9f6e2ed965c1fce5c91039f4be3d90a9", size = 20423580, upload-time = "2026-06-19T14:59:50.665Z" },
+ { url = "https://files.pythonhosted.org/packages/9c/01/93552f75e0d2a7dd115a45e59209c51e8d514daff02fc887d2623be06fe1/scipy-1.18.0-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:97b6cddaaee0a779ef6b5ca83c9604b27cc16b2b8fc22c142652df8793319fb8", size = 23054441, upload-time = "2026-06-19T14:59:53.564Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/23/21f5e703643d66f21faa6b4c73195bfcad70c55efcb4f1ab327cd7c4101a/scipy-1.18.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:52a96e21517c7292375c0e27dd796a811f03fcea5fd4d108fdfea8145dcf17ab", size = 33968720, upload-time = "2026-06-19T14:59:56.415Z" },
+ { url = "https://files.pythonhosted.org/packages/dd/aa/1b939f6c67ed68635bb538e6752d3dacc02f66535182e939a89581a44e9c/scipy-1.18.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1f55797419e16e7f30cf88ffb3113ce0467f00cfe3f70d5c281730b21769bfc2", size = 35287115, upload-time = "2026-06-19T14:59:59.411Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/ff/eec46be7e9234208f801062b53e1983085eddebd693f6c9bfb03b459830d/scipy-1.18.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:ad033410e2e0672ffdc1042110cef20e1c46f8fd0616cee1d44d8d58fad8fc11", size = 35577989, upload-time = "2026-06-19T15:00:02.235Z" },
+ { url = "https://files.pythonhosted.org/packages/84/ca/210d4759c7210bb7d269437421959b39a33434e2776b60c5cb8a763bb30a/scipy-1.18.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:4a55985d54c769c872e64b7f4c8a81cc30ef700cc04296abbbf3705439c126de", size = 37421717, upload-time = "2026-06-19T15:00:05.102Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/54/9a9edb45345bd6744da5ddfb6628e5d5185920494c6a67ec45b6381004cb/scipy-1.18.0-cp312-cp312-win_amd64.whl", hash = "sha256:71ccc8faa2dd16ac310233203474a8b5cb67f10dedd54a3116d34943f4b19132", size = 36597428, upload-time = "2026-06-19T15:00:08.112Z" },
+ { url = "https://files.pythonhosted.org/packages/99/0e/33f32a2a58987e26aec0f7df252cbbad1e90ae77bdbc76f40dd4ed0cf0ea/scipy-1.18.0-cp312-cp312-win_arm64.whl", hash = "sha256:d88363fd9d8fbd3511bd273f1a49efb2a540773ddf92a91d57498ce7dd7f3e76", size = 24351481, upload-time = "2026-06-19T15:00:11.103Z" },
+ { url = "https://files.pythonhosted.org/packages/05/52/9c0136c2de7ae0779b7b366447766cec6d9f0702c56bb8ffeb04c8fd3af4/scipy-1.18.0-cp313-cp313-macosx_10_15_x86_64.whl", hash = "sha256:09143f676d157d9f546d663504ef9c1becb819824f1afc018814176411942446", size = 31036107, upload-time = "2026-06-19T15:00:14.03Z" },
+ { url = "https://files.pythonhosted.org/packages/02/73/0291a64843270f4efb86cdcf2ee0f2048631b65ec6b405398b2b4dbf11bf/scipy-1.18.0-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:5efe260f69417b97ddae455bfb5a95e8359f7f66ad7fa9522a60feb66f169520", size = 28663303, upload-time = "2026-06-19T15:00:16.819Z" },
+ { url = "https://files.pythonhosted.org/packages/d3/0f/10ffa0b697a572f4e0d48b92a88895d366422f019f723e7e14a84c050dac/scipy-1.18.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:68363b7eaacd8b5dd426df56d782cc156468ac79a127a1b87ca597d6e2e82197", size = 20404960, upload-time = "2026-06-19T15:00:19.635Z" },
+ { url = "https://files.pythonhosted.org/packages/7e/d2/e896cea21ba8edd6c81d4c55b1ffcc717e79698dcbebf9641b4cfb4c6622/scipy-1.18.0-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:c5557d8be5da8e41353fcd4d21491fdbab83b062fc579e94dc09a7c8ab4f669b", size = 23034074, upload-time = "2026-06-19T15:00:22.107Z" },
+ { url = "https://files.pythonhosted.org/packages/ea/b2/e83ea34279a52c03374477c74006256ec78df65fc877baa4617d6de1d202/scipy-1.18.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0d13bca67c096d89fb95ced0d8921807300fce0275643aef9533cc63a0773468", size = 33942038, upload-time = "2026-06-19T15:00:24.964Z" },
+ { url = "https://files.pythonhosted.org/packages/f6/af/e8fe5fb136f51e2b01678b92cb4106d10d8cd68ec147ead2e7cb0ac75398/scipy-1.18.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a46f9273dbd0eb1cefba61c9b8648b4dfe3cbc14a080176f9a73e44b8336dc7f", size = 35266390, upload-time = "2026-06-19T15:00:28.059Z" },
+ { url = "https://files.pythonhosted.org/packages/3a/49/2c5cbb907b56695fc67517811d1db234dfd83381a84814ec220aded2794d/scipy-1.18.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:5aba46108853ddfc77906b6557aac839d2b52e900c1d72a1180adaaab58d265f", size = 35551324, upload-time = "2026-06-19T15:00:31.014Z" },
+ { url = "https://files.pythonhosted.org/packages/bb/73/eda39f7a2d306ff0ffc574afd13c0bbb6d10a603d9a413998ee269487a80/scipy-1.18.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b6f758e35f12757b5d95c00bc6de2438e229c2664b7a92e96f205959d9f2dfa4", size = 37404785, upload-time = "2026-06-19T15:00:34.072Z" },
+ { url = "https://files.pythonhosted.org/packages/b7/d2/ae881ee28d014f38e0ccbfd974a06a919ba9af34f1f74bf42b5301891d63/scipy-1.18.0-cp313-cp313-win_amd64.whl", hash = "sha256:1afac4a847207c7ff8efd321734a50b06d0280b3b2a2c0fc2f413101747ad7c7", size = 36554943, upload-time = "2026-06-19T15:00:36.903Z" },
+ { url = "https://files.pythonhosted.org/packages/70/3a/21154e2d54eb3639c6bf4dbae2e531c68356bfe95990daa30df33b30d556/scipy-1.18.0-cp313-cp313-win_arm64.whl", hash = "sha256:c5dbddf60e58c2312316d097271a8e73d40eaf2eabfa4d95ed7d3695bbf2ce7b", size = 24350911, upload-time = "2026-06-19T15:00:40.062Z" },
+ { url = "https://files.pythonhosted.org/packages/78/b5/915a19b3de2f7430062b509653563db1633ddbb6f021b06731521115d4e2/scipy-1.18.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:4c256ee70c0d1a8a2ace807e199ccd4e3f57037433842abb3fb36bc17eaa9578", size = 31036253, upload-time = "2026-06-19T15:00:43.216Z" },
+ { url = "https://files.pythonhosted.org/packages/d7/88/b72def7262e150d16be13fca37a96481138d624e700340bc3362a7588929/scipy-1.18.0-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:2ef3abc54a4ffc53765374b0d5728532dfdd2585ed23f6b11c206a1f0b1b9af8", size = 28673758, upload-time = "2026-06-19T15:00:46.663Z" },
+ { url = "https://files.pythonhosted.org/packages/91/02/2e636a61a525632c373cf6a9c24442a3ffb79e364d38e98b32042964ac32/scipy-1.18.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:f2a6af57bd9e4a75d70e4117e78a1bbee84f79ae3fbb6d0111005d6ebcc4cb8d", size = 20415514, upload-time = "2026-06-19T15:00:49.399Z" },
+ { url = "https://files.pythonhosted.org/packages/c9/b6/2135974442f6aba159d9d39d774a1c8cb19947016725d69fecc685df45bf/scipy-1.18.0-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:3f1ac564d3bf6c03d861d2cd87a1bea0da2887136f7fb1bf519c05a8971452d6", size = 23034398, upload-time = "2026-06-19T15:00:51.941Z" },
+ { url = "https://files.pythonhosted.org/packages/f6/e6/ba89ec5abf6ee9257c0d1ec985573f3ae32742c24bc03e016388a40b1b15/scipy-1.18.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40395a5fcd1abee49a5c7aaa98c29db393eedc835138560a588c47ec16156690", size = 33998032, upload-time = "2026-06-19T15:00:54.838Z" },
+ { url = "https://files.pythonhosted.org/packages/7f/c4/bc41eb19b0fd0db868f4132920879019318d80cc522ad8f2bca4611af808/scipy-1.18.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8ca01e8ae69f1b18e9a58d91afead31be3cef0dd905a10249dac559ee15460a0", size = 35283333, upload-time = "2026-06-19T15:00:58.152Z" },
+ { url = "https://files.pythonhosted.org/packages/53/a4/cbdeef6eb3830a8462a9d4ada814de5fc984345cc9ecf17cbec51a036f1e/scipy-1.18.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7a7f3b01647384dbc3a711e8c6778e0aabbe93959249fef5c7393396bcac0867", size = 35610216, upload-time = "2026-06-19T15:01:01.155Z" },
+ { url = "https://files.pythonhosted.org/packages/80/4d/b2b82502b65f661d1b789c1665dcdf315d5f12194e06fc0b37946294ebae/scipy-1.18.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6aa94e78ec192a30063a5e72e561c28af769dc311190b24fe91774eff1969709", size = 37418960, upload-time = "2026-06-19T15:01:04.155Z" },
+ { url = "https://files.pythonhosted.org/packages/93/3e/902d836831474b0ab5a37d16404f7bc5fafd9efba632890e271ba952635f/scipy-1.18.0-cp314-cp314-win_amd64.whl", hash = "sha256:2d8bbdc6c817f5b4006a54d799d4f5bab6f910193cbb9a1ff310833d4d270f61", size = 37288845, upload-time = "2026-06-19T15:01:07.822Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/43/8d73b337a3bdb14daa0314f0434210747c02d79d729ce1777574a817dcf6/scipy-1.18.0-cp314-cp314-win_arm64.whl", hash = "sha256:18e9575f1569b2c54174e6159d32942e03731177f63dce7975f0a0c88d102f5b", size = 24988971, upload-time = "2026-06-19T15:01:11.076Z" },
+ { url = "https://files.pythonhosted.org/packages/b4/b4/f11918b0508a2787031a0499a03fbe3546f3bb5ca05d01038c45b278c09a/scipy-1.18.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:f351e0dd702687d12a402b867a1b4146a256923e1c38317cbc472f6372b94707", size = 31399325, upload-time = "2026-06-19T15:01:13.723Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/d1/1f287b57c0ff0ee5185dff3946d92c8017d39b0e431f0ae79a3ff1859512/scipy-1.18.0-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:7c7a51b33ce387193c97f228320cf8e87361daa1bba750638677729598b3e677", size = 29092110, upload-time = "2026-06-19T15:01:16.908Z" },
+ { url = "https://files.pythonhosted.org/packages/ff/1a/7b74eb6c392fdcb27d414c0e7558a6d0231eb3b6d73571f479bb81ea8794/scipy-1.18.0-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:84031d7b052a54fae2f8632e0ec802073d385476eb9a63079bce6e23ef9283d4", size = 20833811, upload-time = "2026-06-19T15:01:20.488Z" },
+ { url = "https://files.pythonhosted.org/packages/7c/ad/f3941716320a7b9cb4d68734a903b45fe16eff5fb7da7e16f2e619304979/scipy-1.18.0-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:56abf29a7c067dde59be8b9a22d606a4ea1b2f2a4b756d9d903c62818f5dacce", size = 23396644, upload-time = "2026-06-19T15:01:23.364Z" },
+ { url = "https://files.pythonhosted.org/packages/22/22/1446b62ffe07f9719b7d9b1b6a4e05a772833ae8f441fe4c22c34c9b250f/scipy-1.18.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1ad44305cfa24b1ba5803cbbebf033590ccbac1aa5d612d727b785325ab408b0", size = 34079318, upload-time = "2026-06-19T15:01:26.002Z" },
+ { url = "https://files.pythonhosted.org/packages/56/3b/b87da667098bb470fa30c7011b0ba351ee976dd395c78798c66e941665a3/scipy-1.18.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:945c1761b93f38d7f99ae81ae80c63e621471608c7eeead563f6df025585cd58", size = 35324320, upload-time = "2026-06-19T15:01:28.881Z" },
+ { url = "https://files.pythonhosted.org/packages/f8/a1/c7932f91909759b0267f75fdea34e91309f96b895757534b76a90b6b4344/scipy-1.18.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:1a4441f15d620578772a49e5ab48c0ee1f7a0220e387110283062729136b2553", size = 35699541, upload-time = "2026-06-19T15:01:31.968Z" },
+ { url = "https://files.pythonhosted.org/packages/f7/86/5185061a1fcc41d18c5dc2463969b3a3964b31d9ac67b2fb05d4c7ff7670/scipy-1.18.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9aac6192fac56bf2ca534389d24623f07b39ff83317d58287285e7fbd622ff76", size = 37472480, upload-time = "2026-06-19T15:01:35.136Z" },
+ { url = "https://files.pythonhosted.org/packages/31/8e/f04c68e39919a010d34f2ee1367fd705b0a25a02f609d755f0bfbc0a15fc/scipy-1.18.0-cp314-cp314t-win_amd64.whl", hash = "sha256:e40baea28ae7f5475c779741e2d90b1247c78531207b49c7030e698ff81cee3f", size = 37365390, upload-time = "2026-06-19T15:01:38.091Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/19/969dc072906c84dd0a3b05dcf57ea750936087d7873549e408b35cfc3f97/scipy-1.18.0-cp314-cp314t-win_arm64.whl", hash = "sha256:368e0a705903c466aa5f08eefb39e6b1b6b2d659e7352a31fd9e2438365be0f8", size = 25279661, upload-time = "2026-06-19T15:01:40.817Z" },
+]
+
+[[package]]
+name = "scverse-misc"
+version = "0.1.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "session-info2" },
+ { name = "typing-extensions", marker = "python_full_version < '3.13'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/e2/9b/e3d49620c7bacecad1de9c6474fb453969cc715202e5fc4f7929bfae4e0f/scverse_misc-0.1.1.tar.gz", hash = "sha256:5001572763a88d962f7e40309d1e47a62ae3c358aec93e377af998e4c7e6dd97", size = 42585, upload-time = "2026-06-22T13:01:27.68Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/cd/d7/324b9bdd6fa89fefb4765494dd20d76c06d0f4f0aa8e46233e122c9f9f21/scverse_misc-0.1.1-py3-none-any.whl", hash = "sha256:d402e470a6921c110ab44a63f2e606204d6f6ef25626a3cb2d7b567832148369", size = 22662, upload-time = "2026-06-22T13:01:26.43Z" },
+]
+
+[[package]]
+name = "seaborn"
+version = "0.13.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "matplotlib" },
+ { name = "numpy" },
+ { name = "pandas" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/86/59/a451d7420a77ab0b98f7affa3a1d78a313d2f7281a57afb1a34bae8ab412/seaborn-0.13.2.tar.gz", hash = "sha256:93e60a40988f4d65e9f4885df477e2fdaff6b73a9ded434c1ab356dd57eefff7", size = 1457696, upload-time = "2024-01-25T13:21:52.551Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/83/11/00d3c3dfc25ad54e731d91449895a79e4bf2384dc3ac01809010ba88f6d5/seaborn-0.13.2-py3-none-any.whl", hash = "sha256:636f8336facf092165e27924f223d3c62ca560b1f2bb5dff7ab7fad265361987", size = 294914, upload-time = "2024-01-25T13:21:49.598Z" },
+]
+
+[[package]]
+name = "session-info2"
+version = "0.4.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/50/17/c6a81d91781734bd10d7842c32de665f34395b4db234abdc80dac271632b/session_info2-0.4.1.tar.gz", hash = "sha256:3bb2bf7b73b2e13a1737e9aa91a6dae55e2c49e83bee973f24245f31ae264a1f", size = 25207, upload-time = "2026-04-08T11:30:55.959Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/5e/d7/6893c9c2a52e4bcbeca2a2bf2aee970a686cc7bf555f97db13b00f35250e/session_info2-0.4.1-py3-none-any.whl", hash = "sha256:423b3f6bb7023433cfc3f791a6fdbb6a2cfbe226770ae6c127c3b2c4cf5a9d56", size = 17696, upload-time = "2026-04-08T11:30:54.707Z" },
+]
+
+[[package]]
+name = "setuptools"
+version = "82.0.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/4f/db/cfac1baf10650ab4d1c111714410d2fbb77ac5a616db26775db562c8fab2/setuptools-82.0.1.tar.gz", hash = "sha256:7d872682c5d01cfde07da7bccc7b65469d3dca203318515ada1de5eda35efbf9", size = 1152316, upload-time = "2026-03-09T12:47:17.221Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/9d/76/f789f7a86709c6b087c5a2f52f911838cad707cc613162401badc665acfe/setuptools-82.0.1-py3-none-any.whl", hash = "sha256:a59e362652f08dcd477c78bb6e7bd9d80a7995bc73ce773050228a348ce2e5bb", size = 1006223, upload-time = "2026-03-09T12:47:15.026Z" },
+]
+
+[[package]]
+name = "sgtsnepi"
+version = "0.6"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "nextprod" },
+ { name = "numpy" },
+ { name = "scipy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/f7/6d/e144fee6fba769fed473224ab0f3a9500b79b39e193a4c4a3f4f90458ecd/sgtsnepi-0.6.tar.gz", hash = "sha256:adb6dd2992ea2b04553b40da822f30d7ebd1b7612eca392bb584c81ef36c298d", size = 17269500, upload-time = "2026-02-07T19:33:59.039Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/f9/d8/ec1e97f38ebd00308e2d25acda255c174aeecadc7c5b95cea4982a021032/sgtsnepi-0.6-cp312-cp312-macosx_15_0_arm64.whl", hash = "sha256:59c26c492ba7da429bb2544b2fc0561b8f6be84570f090620d0564a20217f9b3", size = 863750, upload-time = "2026-02-07T19:33:41.741Z" },
+ { url = "https://files.pythonhosted.org/packages/e1/6d/8093e551cffbd4bc4ce8249c62b8ed3f8f20f954c2c4b7078867d3923d02/sgtsnepi-0.6-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:69b444d12ac2dfeb0665b7bcb92fce75429a5e319b7093e580fd5a35b701d323", size = 1529657, upload-time = "2026-02-07T19:33:43.794Z" },
+ { url = "https://files.pythonhosted.org/packages/7a/cf/fd2f4995e2507164924f6ad3ef095db302fbdbfa620a60b7dcafebfc539e/sgtsnepi-0.6-cp313-cp313-macosx_15_0_arm64.whl", hash = "sha256:91e569f1543e8ada83bfe1184db6dfba455eac9050c324f743f81897eaab7e93", size = 863839, upload-time = "2026-02-07T19:33:46.111Z" },
+ { url = "https://files.pythonhosted.org/packages/cd/5a/d1d7d8edaf5547bbd9cf541784e29150fd38d99b74192d5199a8c4c51d68/sgtsnepi-0.6-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:38bf757c4d430151f48d73eeef3b39ff14a3b807eea00f1319c61e25485e69e4", size = 1529598, upload-time = "2026-02-07T19:33:47.826Z" },
+ { url = "https://files.pythonhosted.org/packages/bc/d6/c6151851e8b5af4a165eb7ab9b349d3121124cf4028eaad02af4ae0257a3/sgtsnepi-0.6-cp314-cp314-macosx_15_0_arm64.whl", hash = "sha256:bc3d46ff337f41934c1cb56082f4951ce78262bfa6e67678927d5287cdf7d1fe", size = 864069, upload-time = "2026-02-07T19:33:50.078Z" },
+ { url = "https://files.pythonhosted.org/packages/5c/21/eccb2b171e2d8a8c19ea3bee1d1a2273c3cb917b3280c730b386666e1067/sgtsnepi-0.6-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:130a763021e6ee8e9e17bcf81efa814e5cb18398a5e476dc8b6e977a846336fd", size = 1529553, upload-time = "2026-02-07T19:33:51.834Z" },
+ { url = "https://files.pythonhosted.org/packages/be/b7/c1f786637c570089f328a0afeff4ad01a365882f825fcbfdb14555aae971/sgtsnepi-0.6-cp314-cp314t-macosx_15_0_arm64.whl", hash = "sha256:e2a9a8fadc206147bf1126ed84343a34a04708510f8d37f3dbc40a8bbfe1f495", size = 867814, upload-time = "2026-02-07T19:33:53.171Z" },
+ { url = "https://files.pythonhosted.org/packages/3b/4b/6fa22d2231b890ee03432f0428f183949740b5a5b04c77fd9030eabc3ceb/sgtsnepi-0.6-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4c31d432aae3399bbd39e29ee899b5ccf868a7a351e32eb09e0cb7f4ca6c817c", size = 1530918, upload-time = "2026-02-07T19:33:54.717Z" },
+]
+
+[[package]]
+name = "six"
+version = "1.17.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031, upload-time = "2024-12-04T17:35:28.174Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" },
+]
+
+[[package]]
+name = "snowballstemmer"
+version = "3.1.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/43/f8/0a71edf031f03c40db17503cb8ca78a69a171254e568e7db241b0ab57ea1/snowballstemmer-3.1.1.tar.gz", hash = "sha256:e07bbc54a0d798fe6010a12398422e62a8bfbba95c394fd0956ef58cb4d3e260", size = 123314, upload-time = "2026-06-03T00:56:40.194Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/4c/07/2ebca9b11fb9be7340a818d8d6f63feaebb146be2c4afbd6061701d6df6e/snowballstemmer-3.1.1-py3-none-any.whl", hash = "sha256:7e207fa178741da09cdee59d3ecec3827ad5f92b1fc5c9ff3755b639f71f5752", size = 104164, upload-time = "2026-06-03T00:56:38.614Z" },
+]
+
+[[package]]
+name = "soupsieve"
+version = "2.8.4"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/47/2c/0a5f6f8ee0d5589e48c7640213ed5175d52cf540a06725b628cc1a45d6ce/soupsieve-2.8.4.tar.gz", hash = "sha256:e121fd02e975c695e4e9e8774a5ee35d74714b59307868dcc5319ad2d9e3328e", size = 121110, upload-time = "2026-05-24T13:55:57.154Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/5e/f5/0c41cb68dcae6b7de4fac4188a3a9589e21fb31df21ea3a2e888db95e6c9/soupsieve-2.8.4-py3-none-any.whl", hash = "sha256:e7e6b0769c8f51ed59acab6e994b00621096cfb1c640a7509295987388fbaf65", size = 37304, upload-time = "2026-05-24T13:55:55.406Z" },
+]
+
+[[package]]
+name = "sphinx"
+version = "9.1.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "alabaster" },
+ { name = "babel" },
+ { name = "colorama", marker = "sys_platform == 'win32'" },
+ { name = "docutils" },
+ { name = "imagesize" },
+ { name = "jinja2" },
+ { name = "packaging" },
+ { name = "pygments" },
+ { name = "requests" },
+ { name = "roman-numerals" },
+ { name = "snowballstemmer" },
+ { name = "sphinxcontrib-applehelp" },
+ { name = "sphinxcontrib-devhelp" },
+ { name = "sphinxcontrib-htmlhelp" },
+ { name = "sphinxcontrib-jsmath" },
+ { name = "sphinxcontrib-qthelp" },
+ { name = "sphinxcontrib-serializinghtml" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/cd/bd/f08eb0f4eed5c83f1ba2a3bd18f7745a2b1525fad70660a1c00224ec468a/sphinx-9.1.0.tar.gz", hash = "sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb", size = 8718324, upload-time = "2025-12-31T15:09:27.646Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/73/f7/b1884cb3188ab181fc81fa00c266699dab600f927a964df02ec3d5d1916a/sphinx-9.1.0-py3-none-any.whl", hash = "sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978", size = 3921742, upload-time = "2025-12-31T15:09:25.561Z" },
+]
+
+[[package]]
+name = "sphinx-autodoc-typehints"
+version = "3.12.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "sphinx" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/99/f2/d99787d34d881b2352c04ce02bcdf0a61adf54b389c86d438b7a8ac3ae20/sphinx_autodoc_typehints-3.12.0.tar.gz", hash = "sha256:6571eb33c72cdc616a9945730fcb43c5bcf3685e79f1e8aea144735e43d5230d", size = 83874, upload-time = "2026-06-25T15:44:47.786Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/3e/28/fa0f2ff73b8ca7987ebaa107d369c95459741ac73567e5079b4a881a981b/sphinx_autodoc_typehints-3.12.0-py3-none-any.whl", hash = "sha256:1a639b7cf71be13c4d8a4f0b552dcfcdf32ff39ac33ecb47398acb85863c2faf", size = 42548, upload-time = "2026-06-25T15:44:46.306Z" },
+]
+
+[[package]]
+name = "sphinx-book-theme"
+version = "1.2.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "pydata-sphinx-theme" },
+ { name = "sphinx" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/eb/f7/154786f3cfb7692cd7acc24b6dfe4dcd1146b66f376b17df9e47125555e9/sphinx_book_theme-1.2.0.tar.gz", hash = "sha256:4a7ebfc7da4395309ac942ddfc38fbec5c5254c3be22195e99ad12586fbda9e3", size = 443962, upload-time = "2026-03-09T23:20:30.442Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/02/bf/6f506a37c7f8ecc4576caf9486e303c7af249f6d70447bb51dde9d78cb99/sphinx_book_theme-1.2.0-py3-none-any.whl", hash = "sha256:709605d308e1991c5ef0cf19c481dbe9084b62852e317fafab74382a0ee7ccfa", size = 455936, upload-time = "2026-03-09T23:20:28.788Z" },
+]
+
+[[package]]
+name = "sphinx-copybutton"
+version = "0.5.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "sphinx" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/fc/2b/a964715e7f5295f77509e59309959f4125122d648f86b4fe7d70ca1d882c/sphinx-copybutton-0.5.2.tar.gz", hash = "sha256:4cf17c82fb9646d1bc9ca92ac280813a3b605d8c421225fd9913154103ee1fbd", size = 23039, upload-time = "2023-04-14T08:10:22.998Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/9e/48/1ea60e74949eecb12cdd6ac43987f9fd331156388dcc2319b45e2ebb81bf/sphinx_copybutton-0.5.2-py3-none-any.whl", hash = "sha256:fb543fd386d917746c9a2c50360c7905b605726b9355cd26e9974857afeae06e", size = 13343, upload-time = "2023-04-14T08:10:20.844Z" },
+]
+
+[[package]]
+name = "sphinx-external-toc"
+version = "1.1.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "click" },
+ { name = "pyyaml" },
+ { name = "sphinx" },
+ { name = "sphinx-multitoc-numbering" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/c5/f8/85bcd2f1c142e580a1394c18920506d9399b8e8e97e4899bbee9c74a896e/sphinx_external_toc-1.1.0.tar.gz", hash = "sha256:f81833865006f6b4a9b2550a2474a6e3d7e7f2cb23ba23309260577ea65552f6", size = 37194, upload-time = "2026-01-16T13:15:59.03Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/9f/80/1704c9179012e289dee2178354e385277ea51f4fa827c4bf7e36c77b0f4b/sphinx_external_toc-1.1.0-py3-none-any.whl", hash = "sha256:26c390b8d85aa641366fed2d3674910ec6820f48b91027affef485a2655ad7d0", size = 30609, upload-time = "2026-01-16T13:15:57.926Z" },
+]
+
+[[package]]
+name = "sphinx-multitoc-numbering"
+version = "0.1.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "sphinx" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/37/1e/577bae038372885ebc34bd8c0f290295785a0250cac6528eb6d50e4b92d5/sphinx-multitoc-numbering-0.1.3.tar.gz", hash = "sha256:c9607671ac511236fa5d61a7491c1031e700e8d498c9d2418e6c61d1251209ae", size = 4542, upload-time = "2021-03-15T12:01:43.758Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ec/9f/902f2030674cd9473fdbe5a2c2dec2618c27ec853484c35f82cf8df40ece/sphinx_multitoc_numbering-0.1.3-py3-none-any.whl", hash = "sha256:33d2e707a9b2b8ad636b3d4302e658a008025106fe0474046c651144c26d8514", size = 4616, upload-time = "2021-03-15T12:01:42.419Z" },
+]
+
+[[package]]
+name = "sphinx-tabs"
+version = "3.5.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "docutils" },
+ { name = "pygments" },
+ { name = "sphinx" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/ce/30/ca5b0de830f369968d8e3483dd45a8908fd10169c05cd9837f0bd075982e/sphinx_tabs-3.5.0.tar.gz", hash = "sha256:91dba1187e4c35fd37380a56ac228bbd54c6c649b2351829f3bf033718277537", size = 17006, upload-time = "2026-03-03T23:00:30.404Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/2e/45/6adc5efeb19fd5fed4027e520b5c668ce58236a2b271ade5533c4c116276/sphinx_tabs-3.5.0-py3-none-any.whl", hash = "sha256:154be49de4d5c8249ea08c5d9bf88ca8f9c31e00a178305a93cbc33e000339e5", size = 9871, upload-time = "2026-03-03T23:00:28.89Z" },
+]
+
+[[package]]
+name = "sphinxcontrib-applehelp"
+version = "2.0.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ba/6e/b837e84a1a704953c62ef8776d45c3e8d759876b4a84fe14eba2859106fe/sphinxcontrib_applehelp-2.0.0.tar.gz", hash = "sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1", size = 20053, upload-time = "2024-07-29T01:09:00.465Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/5d/85/9ebeae2f76e9e77b952f4b274c27238156eae7979c5421fba91a28f4970d/sphinxcontrib_applehelp-2.0.0-py3-none-any.whl", hash = "sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5", size = 119300, upload-time = "2024-07-29T01:08:58.99Z" },
+]
+
+[[package]]
+name = "sphinxcontrib-devhelp"
+version = "2.0.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/f6/d2/5beee64d3e4e747f316bae86b55943f51e82bb86ecd325883ef65741e7da/sphinxcontrib_devhelp-2.0.0.tar.gz", hash = "sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad", size = 12967, upload-time = "2024-07-29T01:09:23.417Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/35/7a/987e583882f985fe4d7323774889ec58049171828b58c2217e7f79cdf44e/sphinxcontrib_devhelp-2.0.0-py3-none-any.whl", hash = "sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2", size = 82530, upload-time = "2024-07-29T01:09:21.945Z" },
+]
+
+[[package]]
+name = "sphinxcontrib-htmlhelp"
+version = "2.1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/43/93/983afd9aa001e5201eab16b5a444ed5b9b0a7a010541e0ddfbbfd0b2470c/sphinxcontrib_htmlhelp-2.1.0.tar.gz", hash = "sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9", size = 22617, upload-time = "2024-07-29T01:09:37.889Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/0a/7b/18a8c0bcec9182c05a0b3ec2a776bba4ead82750a55ff798e8d406dae604/sphinxcontrib_htmlhelp-2.1.0-py3-none-any.whl", hash = "sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8", size = 98705, upload-time = "2024-07-29T01:09:36.407Z" },
+]
+
+[[package]]
+name = "sphinxcontrib-jsmath"
+version = "1.0.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/b2/e8/9ed3830aeed71f17c026a07a5097edcf44b692850ef215b161b8ad875729/sphinxcontrib-jsmath-1.0.1.tar.gz", hash = "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8", size = 5787, upload-time = "2019-01-21T16:10:16.347Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/c2/42/4c8646762ee83602e3fb3fbe774c2fac12f317deb0b5dbeeedd2d3ba4b77/sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl", hash = "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178", size = 5071, upload-time = "2019-01-21T16:10:14.333Z" },
+]
+
+[[package]]
+name = "sphinxcontrib-qthelp"
+version = "2.0.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/68/bc/9104308fc285eb3e0b31b67688235db556cd5b0ef31d96f30e45f2e51cae/sphinxcontrib_qthelp-2.0.0.tar.gz", hash = "sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab", size = 17165, upload-time = "2024-07-29T01:09:56.435Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/27/83/859ecdd180cacc13b1f7e857abf8582a64552ea7a061057a6c716e790fce/sphinxcontrib_qthelp-2.0.0-py3-none-any.whl", hash = "sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb", size = 88743, upload-time = "2024-07-29T01:09:54.885Z" },
+]
+
+[[package]]
+name = "sphinxcontrib-serializinghtml"
+version = "2.0.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/3b/44/6716b257b0aa6bfd51a1b31665d1c205fb12cb5ad56de752dfa15657de2f/sphinxcontrib_serializinghtml-2.0.0.tar.gz", hash = "sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d", size = 16080, upload-time = "2024-07-29T01:10:09.332Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/52/a7/d2782e4e3f77c8450f727ba74a8f12756d5ba823d81b941f1b04da9d033a/sphinxcontrib_serializinghtml-2.0.0-py3-none-any.whl", hash = "sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331", size = 92072, upload-time = "2024-07-29T01:10:08.203Z" },
+]
+
+[[package]]
+name = "sqlalchemy"
+version = "2.0.51"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "greenlet", marker = "platform_machine == 'AMD64' or platform_machine == 'WIN32' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'ppc64le' or platform_machine == 'win32' or platform_machine == 'x86_64'" },
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/02/f1/a7a892f18d4d224e6b26f706531eafccc41e37594d37d304786969ee13cb/sqlalchemy-2.0.51.tar.gz", hash = "sha256:804dccd8a4a6242c4e30ad961e540e18a588f6527202f2d6791b01845d59fdc9", size = 9912201, upload-time = "2026-06-15T15:41:20.012Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/d5/70/e868bc5412acd101a8280f25c95f10eeae0771c4eb806b02491142810ee8/sqlalchemy-2.0.51-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:7d78702b26ba1c18b2d0fb2ea940ba7f17a9581b42e8361ff93920ebbee1235a", size = 2160291, upload-time = "2026-06-15T16:08:48.918Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/1c/71ee0f8a6b9d7316a1ccd30430b4c62b6c2e36adc96017a4e3a72dce49d6/sqlalchemy-2.0.51-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:581921d849d6e6f994d560389192955e80e2950e18fcdfe2ccea863e01158e6e", size = 3343835, upload-time = "2026-06-15T16:19:42.613Z" },
+ { url = "https://files.pythonhosted.org/packages/2b/7c/7ab9f9aadc5944fdd06612484ed7918fe376ad871a5f50404dc1536e0194/sqlalchemy-2.0.51-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1d21ce524ab86c23046e992a5b81cb54c21079c6df6e78b8fc77d77cac70a6b9", size = 3358470, upload-time = "2026-06-15T16:26:38.011Z" },
+ { url = "https://files.pythonhosted.org/packages/d0/7d/ff77169fee6186de145a7f2b87006c39638391130abbab2b1f63ac6ea583/sqlalchemy-2.0.51-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c5d98a2709840027f5a347c3af0a7c3d5f6c1ff93af2ca1c54494e23cba8f389", size = 3289874, upload-time = "2026-06-15T16:19:45.212Z" },
+ { url = "https://files.pythonhosted.org/packages/6f/3b/6c505903710d781b55bc3141ee34a062bf9745a6b5bc7333305b9ed63b33/sqlalchemy-2.0.51-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:1181256e0f16479691b5616d36375dc2620ad8332b25978763c3d206ad3f3f1d", size = 3321692, upload-time = "2026-06-15T16:26:39.747Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/b7/c5ffe50aa2f4d947c9250e1519d939260329a07fe6272edfccd784b3d007/sqlalchemy-2.0.51-cp312-cp312-win32.whl", hash = "sha256:9f380393be5abeb6815f68fd39271b95127173511b6706b0a630a9995d53f8f5", size = 2119674, upload-time = "2026-06-15T16:23:09.543Z" },
+ { url = "https://files.pythonhosted.org/packages/25/dc/46a65916af68a06ef6b972c6050ba4c8f97070fe3fb33097d34229d9bef6/sqlalchemy-2.0.51-cp312-cp312-win_amd64.whl", hash = "sha256:2cf39aabdf48e87c1c2c2ed6d20d33ffa0733b3071ce9c5f66357947dd009080", size = 2146670, upload-time = "2026-06-15T16:23:11.048Z" },
+ { url = "https://files.pythonhosted.org/packages/54/fe/a210d52fd1a90ecfae8a78e9d8b27e18d733d60818a8bf250ff690b75120/sqlalchemy-2.0.51-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:7c2056838b6685b72fdb36c99996cf862753461a62f2e84f4196371d3b2d6a07", size = 2157184, upload-time = "2026-06-15T16:08:50.374Z" },
+ { url = "https://files.pythonhosted.org/packages/17/6b/2dce8369b199cb855110e056032f94a9f66dacc2237d3d39c115a86eac56/sqlalchemy-2.0.51-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:483b11bd46bf35fc14c52faf338b04300c9e6ce554bce9b11be85bfec3bc3195", size = 3284735, upload-time = "2026-06-15T16:19:46.934Z" },
+ { url = "https://files.pythonhosted.org/packages/53/ff/dbc495b8a14da840faffb353857a72d4190113cac33727906fb997047f0f/sqlalchemy-2.0.51-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1bed1ee8b01da6088210aa9412023326fb98a599ba502e6118308601dcbef77f", size = 3302756, upload-time = "2026-06-15T16:26:41.336Z" },
+ { url = "https://files.pythonhosted.org/packages/cf/d5/fde8f4dddcf518ee15ab35a7c6a28acc32c8ba548d1d2aa451f96e6dbb0b/sqlalchemy-2.0.51-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:72ca54c952107ba5cd58854b67a5a6268631289d21651a1235396f3b98b47400", size = 3232055, upload-time = "2026-06-15T16:19:49.286Z" },
+ { url = "https://files.pythonhosted.org/packages/67/d1/43d3a0ac955a58601c24fa23038b1c55ee3a1ec02c0f96ebb1eae2bcf614/sqlalchemy-2.0.51-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b3e693d15533a45cd5906f0589f9c35090bef6ef45bf1e8195c424aa0ae06a8d", size = 3269850, upload-time = "2026-06-15T16:26:43.017Z" },
+ { url = "https://files.pythonhosted.org/packages/94/df/de669c7054cd47c4439ac34b1b2ee8b804a794791fbb10720e997a2c87c7/sqlalchemy-2.0.51-cp313-cp313-win32.whl", hash = "sha256:b93ab07b5292dbe7e6b8da89475275e7042744283921344b56105f3eeb0f828b", size = 2117721, upload-time = "2026-06-15T16:23:12.36Z" },
+ { url = "https://files.pythonhosted.org/packages/d0/8a/403c51d064196bae20a0bc2476577f83a3f8dd299719a97417086b7f2ec5/sqlalchemy-2.0.51-cp313-cp313-win_amd64.whl", hash = "sha256:0f053118c30e53161857a953e4de667d90e274980dccbe5dd3829bbbeece72a5", size = 2143615, upload-time = "2026-06-15T16:23:13.906Z" },
+ { url = "https://files.pythonhosted.org/packages/b1/49/a739be2e1d02a96a658eb71ab45d921c874249252358ad24a5bffdd02525/sqlalchemy-2.0.51-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:6ea306caaae6bd5afd0a46050003c88f6bf33227377a49298c498c3cb88ff491", size = 2158999, upload-time = "2026-06-15T16:08:51.759Z" },
+ { url = "https://files.pythonhosted.org/packages/23/6b/2e0e38cf75c8780eca78d9b2e78164f8bcfd70125e5caa588ff5cbb9c9f4/sqlalchemy-2.0.51-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c45a496d6bc05dec41dcd4c3a2b183723f47473255c159cd80b503c8f246424d", size = 3282539, upload-time = "2026-06-15T16:19:51.065Z" },
+ { url = "https://files.pythonhosted.org/packages/dd/a1/e77854cb5336fd37dc3c6ae3b71de242c98caac5725120be0b526b31cbd0/sqlalchemy-2.0.51-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4004ada0aafe8ae1991b2cd1d99c6d9146126e123bd6f883c260d974aa012e54", size = 3287545, upload-time = "2026-06-15T16:26:44.735Z" },
+ { url = "https://files.pythonhosted.org/packages/f6/ab/9e17272fd4dac8df3b83c4fbe52b998a1c9d89a843c8c35ff29b74ff7364/sqlalchemy-2.0.51-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:0f6bcad487aee1c638d707235682fc96f741de00663619881ab235400d03289e", size = 3230929, upload-time = "2026-06-15T16:19:52.625Z" },
+ { url = "https://files.pythonhosted.org/packages/02/3c/52f408ea701781caee975606beccc48845f2aee8711ac29843d612c0306c/sqlalchemy-2.0.51-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:39a76529db6305693d8d4affa58ad5b5e2e18edd62daea628b29b97930b3513d", size = 3252888, upload-time = "2026-06-15T16:26:46.454Z" },
+ { url = "https://files.pythonhosted.org/packages/24/16/3efd2ee6bc4ca4693a30a1dd17a91b606cae15d517d2a4746611d9b73ce8/sqlalchemy-2.0.51-cp314-cp314-win32.whl", hash = "sha256:08a204d8b5638717c26a24df18fcf40af45a6b22e35b70b1d62f0113c2e278e8", size = 2120551, upload-time = "2026-06-15T16:23:15.629Z" },
+ { url = "https://files.pythonhosted.org/packages/7b/78/55b12e70f45bccc40d9e483925c065027b3b98ea4cbbdf6f8c2546feaf6c/sqlalchemy-2.0.51-cp314-cp314-win_amd64.whl", hash = "sha256:96747bfbadb055466e5b46d572618170046b45ce5a4879167f50d70a5319a499", size = 2146318, upload-time = "2026-06-15T16:23:17.108Z" },
+ { url = "https://files.pythonhosted.org/packages/21/db/a9574ed40fed418924b1b1a3e54f47ee3963053b3d3d325a0d36b41f2c08/sqlalchemy-2.0.51-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:e5ea1a213be1fcd5e49d9904c3b9939211ded90bc2a64e93f4c01963474285de", size = 2178920, upload-time = "2026-06-15T15:59:56.285Z" },
+ { url = "https://files.pythonhosted.org/packages/bf/90/a1bb5c7cbba76b7bc1fbd586d0a5479a7bc9c27b4a8298f22ec9423b2bb3/sqlalchemy-2.0.51-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7c6b36ed71f41942bdcd2ad2522be46bfce09d5705be5640ecf19bbc7660e4b7", size = 3566534, upload-time = "2026-06-15T15:58:35.024Z" },
+ { url = "https://files.pythonhosted.org/packages/15/4b/481f1fed30e0e9e8dd24aecbb49f29eb57fe7657ece5cf06ee9b84bb97d8/sqlalchemy-2.0.51-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0c2c62877097e1a0db401fba5cb4debee33265e5b2a55c4ccb489c02c53b4f72", size = 3535844, upload-time = "2026-06-15T16:02:43.973Z" },
+ { url = "https://files.pythonhosted.org/packages/02/71/0aa64aeda645510af0a43f7d9ee70932f0d1dc4263aed34c50ee891d9df3/sqlalchemy-2.0.51-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:0378d055e9e8cd6ce4d8dff683bdd3d7d413533c4ee51d67a2b1e0f9eacc0f23", size = 3475355, upload-time = "2026-06-15T15:58:36.592Z" },
+ { url = "https://files.pythonhosted.org/packages/05/db/6061db32316446135a3abae5f308d144ab988a34234726042da3e58b1c63/sqlalchemy-2.0.51-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:6e46fc36029eff666391e0531e5387b62ce6c4f1d8e50b3fb3099eaca1b42522", size = 3486591, upload-time = "2026-06-15T16:02:45.346Z" },
+ { url = "https://files.pythonhosted.org/packages/0d/c9/f14fdf71bb8957e0c7e39db69bbdf12b5c80f4ef775fdfa127bf4e0d6760/sqlalchemy-2.0.51-cp314-cp314t-win32.whl", hash = "sha256:9161cfc9efce70d1715f47d6ff40f79c6778c00d53be4fbc09d70301e4b83ba7", size = 2151313, upload-time = "2026-06-15T16:03:39.127Z" },
+ { url = "https://files.pythonhosted.org/packages/6a/c6/673e618e6f4f297e126d9b56ea2f6478708f6c1af4e3223835c22e2c3697/sqlalchemy-2.0.51-cp314-cp314t-win_amd64.whl", hash = "sha256:159bb6ba32059f57ad7375a8f50d844dd2f19d14954ecf820cd33e20debd46b2", size = 2186280, upload-time = "2026-06-15T16:03:40.569Z" },
+ { url = "https://files.pythonhosted.org/packages/e2/22/dbf013a12ec759e54a34a119e9e217435b3f71b2dd5c61a7ade0a25dae87/sqlalchemy-2.0.51-py3-none-any.whl", hash = "sha256:bb024d8b621d0be75f4f44ecc7c950450026e76d66dc8f791bb5331d7fed59d5", size = 1944334, upload-time = "2026-06-15T16:09:22.418Z" },
+]
+
+[[package]]
+name = "stack-data"
+version = "0.6.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "asttokens" },
+ { name = "executing" },
+ { name = "pure-eval" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/28/e3/55dcc2cfbc3ca9c29519eb6884dd1415ecb53b0e934862d3559ddcb7e20b/stack_data-0.6.3.tar.gz", hash = "sha256:836a778de4fec4dcd1dcd89ed8abff8a221f58308462e1c4aa2a3cf30148f0b9", size = 44707, upload-time = "2023-09-30T13:58:05.479Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/f1/7b/ce1eafaf1a76852e2ec9b22edecf1daa58175c090266e9f6c64afcd81d91/stack_data-0.6.3-py3-none-any.whl", hash = "sha256:d5558e0c25a4cb0853cddad3d77da9891a08cb85dd9f9f91b9f8cd66e511e695", size = 24521, upload-time = "2023-09-30T13:58:03.53Z" },
+]
+
+[[package]]
+name = "statsmodels"
+version = "0.14.6"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+ { name = "packaging" },
+ { name = "pandas" },
+ { name = "patsy" },
+ { name = "scipy" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/0d/81/e8d74b34f85285f7335d30c5e3c2d7c0346997af9f3debf9a0a9a63de184/statsmodels-0.14.6.tar.gz", hash = "sha256:4d17873d3e607d398b85126cd4ed7aad89e4e9d89fc744cdab1af3189a996c2a", size = 20689085, upload-time = "2025-12-05T23:08:39.522Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/25/ce/308e5e5da57515dd7cab3ec37ea2d5b8ff50bef1fcc8e6d31456f9fae08e/statsmodels-0.14.6-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:fe76140ae7adc5ff0e60a3f0d56f4fffef484efa803c3efebf2fcd734d72ecb5", size = 10091932, upload-time = "2025-12-05T19:28:55.446Z" },
+ { url = "https://files.pythonhosted.org/packages/05/30/affbabf3c27fb501ec7b5808230c619d4d1a4525c07301074eb4bda92fa9/statsmodels-0.14.6-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:26d4f0ed3b31f3c86f83a92f5c1f5cbe63fc992cd8915daf28ca49be14463a1c", size = 9997345, upload-time = "2025-12-05T19:29:10.278Z" },
+ { url = "https://files.pythonhosted.org/packages/48/f5/3a73b51e6450c31652c53a8e12e24eac64e3824be816c0c2316e7dbdcb7d/statsmodels-0.14.6-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d8c00a42863e4f4733ac9d078bbfad816249c01451740e6f5053ecc7db6d6368", size = 10058649, upload-time = "2025-12-05T23:10:12.775Z" },
+ { url = "https://files.pythonhosted.org/packages/81/68/dddd76117df2ef14c943c6bbb6618be5c9401280046f4ddfc9fb4596a1b8/statsmodels-0.14.6-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:19b58cf7474aa9e7e3b0771a66537148b2df9b5884fbf156096c0e6c1ff0469d", size = 10339446, upload-time = "2025-12-05T23:10:28.503Z" },
+ { url = "https://files.pythonhosted.org/packages/56/4a/dce451c74c4050535fac1ec0c14b80706d8fc134c9da22db3c8a0ec62c33/statsmodels-0.14.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:81e7dcc5e9587f2567e52deaff5220b175bf2f648951549eae5fc9383b62bc37", size = 10368705, upload-time = "2025-12-05T23:10:44.339Z" },
+ { url = "https://files.pythonhosted.org/packages/60/15/3daba2df40be8b8a9a027d7f54c8dedf24f0d81b96e54b52293f5f7e3418/statsmodels-0.14.6-cp312-cp312-win_amd64.whl", hash = "sha256:b5eb07acd115aa6208b4058211138393a7e6c2cf12b6f213ede10f658f6a714f", size = 9543991, upload-time = "2025-12-05T23:10:58.536Z" },
+ { url = "https://files.pythonhosted.org/packages/81/59/a5aad5b0cc266f5be013db8cde563ac5d2a025e7efc0c328d83b50c72992/statsmodels-0.14.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:47ee7af083623d2091954fa71c7549b8443168f41b7c5dce66510274c50fd73e", size = 10072009, upload-time = "2025-12-05T23:11:14.021Z" },
+ { url = "https://files.pythonhosted.org/packages/53/dd/d8cfa7922fc6dc3c56fa6c59b348ea7de829a94cd73208c6f8202dd33f17/statsmodels-0.14.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:aa60d82e29fcd0a736e86feb63a11d2380322d77a9369a54be8b0965a3985f71", size = 9980018, upload-time = "2025-12-05T23:11:30.907Z" },
+ { url = "https://files.pythonhosted.org/packages/ee/77/0ec96803eba444efd75dba32f2ef88765ae3e8f567d276805391ec2c98c6/statsmodels-0.14.6-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:89ee7d595f5939cc20bf946faedcb5137d975f03ae080f300ebb4398f16a5bd4", size = 10060269, upload-time = "2025-12-05T23:11:46.338Z" },
+ { url = "https://files.pythonhosted.org/packages/10/b9/fd41f1f6af13a1a1212a06bb377b17762feaa6d656947bf666f76300fc05/statsmodels-0.14.6-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:730f3297b26749b216a06e4327fe0be59b8d05f7d594fb6caff4287b69654589", size = 10324155, upload-time = "2025-12-05T23:12:01.805Z" },
+ { url = "https://files.pythonhosted.org/packages/ee/0f/a6900e220abd2c69cd0a07e3ad26c71984be6061415a60e0f17b152ecf08/statsmodels-0.14.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:f1c08befa85e93acc992b72a390ddb7bd876190f1360e61d10cf43833463bc9c", size = 10349765, upload-time = "2025-12-05T23:12:18.018Z" },
+ { url = "https://files.pythonhosted.org/packages/98/08/b79f0c614f38e566eebbdcff90c0bcacf3c6ba7a5bbb12183c09c29ca400/statsmodels-0.14.6-cp313-cp313-win_amd64.whl", hash = "sha256:8021271a79f35b842c02a1794465a651a9d06ec2080f76ebc3b7adce77d08233", size = 9540043, upload-time = "2025-12-05T23:12:33.887Z" },
+ { url = "https://files.pythonhosted.org/packages/71/de/09540e870318e0c7b58316561d417be45eff731263b4234fdd2eee3511a8/statsmodels-0.14.6-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:00781869991f8f02ad3610da6627fd26ebe262210287beb59761982a8fa88cae", size = 10069403, upload-time = "2025-12-05T23:12:48.424Z" },
+ { url = "https://files.pythonhosted.org/packages/ab/f0/63c1bfda75dc53cee858006e1f46bd6d6f883853bea1b97949d0087766ca/statsmodels-0.14.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:73f305fbf31607b35ce919fae636ab8b80d175328ed38fdc6f354e813b86ee37", size = 9989253, upload-time = "2025-12-05T23:13:05.274Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/98/b0dfb4f542b2033a3341aa5f1bdd97024230a4ad3670c5b0839d54e3dcab/statsmodels-0.14.6-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e443e7077a6e2d3faeea72f5a92c9f12c63722686eb80bb40a0f04e4a7e267ad", size = 10090802, upload-time = "2025-12-05T23:13:20.653Z" },
+ { url = "https://files.pythonhosted.org/packages/34/0e/2408735aca9e764643196212f9069912100151414dd617d39ffc72d77eee/statsmodels-0.14.6-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3414e40c073d725007a6603a18247ab7af3467e1af4a5e5a24e4c27bc26673b4", size = 10337587, upload-time = "2025-12-05T23:13:37.597Z" },
+ { url = "https://files.pythonhosted.org/packages/0f/36/4d44f7035ab3c0b2b6a4c4ebb98dedf36246ccbc1b3e2f51ebcd7ac83abb/statsmodels-0.14.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:a518d3f9889ef920116f9fa56d0338069e110f823926356946dae83bc9e33e19", size = 10363350, upload-time = "2025-12-05T23:13:53.08Z" },
+ { url = "https://files.pythonhosted.org/packages/26/33/f1652d0c59fa51de18492ee2345b65372550501ad061daa38f950be390b6/statsmodels-0.14.6-cp314-cp314-win_amd64.whl", hash = "sha256:151b73e29f01fe619dbce7f66d61a356e9d1fe5e906529b78807df9189c37721", size = 9588010, upload-time = "2025-12-05T23:14:07.28Z" },
+]
+
+[[package]]
+name = "synchronicity"
+version = "0.12.5"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/5d/1c/f51dc54bbd302991026a53f9790735540e0e9e1184e9d5939f02446aa5bc/synchronicity-0.12.5.tar.gz", hash = "sha256:94d96b1d85698e3056b96a793b8c0949af6584e4a7d877fabdeb5385efe230aa", size = 60745, upload-time = "2026-06-18T21:06:23.545Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/06/74/ad9b99520f70c0bc3318e582e359d360cfc0f7afd7bf368a7f24013cece7/synchronicity-0.12.5-py3-none-any.whl", hash = "sha256:fdbbb10d437bc08a6b0f814fc66fddd1b58ffed314533d42f1ab555801e781af", size = 41107, upload-time = "2026-06-18T21:06:22.505Z" },
+]
+
+[[package]]
+name = "tabulate"
+version = "0.10.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/46/58/8c37dea7bbf769b20d58e7ace7e5edfe65b849442b00ffcdd56be88697c6/tabulate-0.10.0.tar.gz", hash = "sha256:e2cfde8f79420f6deeffdeda9aaec3b6bc5abce947655d17ac662b126e48a60d", size = 91754, upload-time = "2026-03-04T18:55:34.402Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/99/55/db07de81b5c630da5cbf5c7df646580ca26dfaefa593667fc6f2fe016d2e/tabulate-0.10.0-py3-none-any.whl", hash = "sha256:f0b0622e567335c8fabaaa659f1b33bcb6ddfe2e496071b743aa113f8774f2d3", size = 39814, upload-time = "2026-03-04T18:55:31.284Z" },
+]
+
+[[package]]
+name = "texttable"
+version = "1.7.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/1c/dc/0aff23d6036a4d3bf4f1d8c8204c5c79c4437e25e0ae94ffe4bbb55ee3c2/texttable-1.7.0.tar.gz", hash = "sha256:2d2068fb55115807d3ac77a4ca68fa48803e84ebb0ee2340f858107a36522638", size = 12831, upload-time = "2023-10-03T09:48:12.272Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/24/99/4772b8e00a136f3e01236de33b0efda31ee7077203ba5967fcc76da94d65/texttable-1.7.0-py2.py3-none-any.whl", hash = "sha256:72227d592c82b3d7f672731ae73e4d1f88cd8e2ef5b075a7a7f01a23a3743917", size = 10768, upload-time = "2023-10-03T09:48:10.434Z" },
+]
+
+[[package]]
+name = "threadpoolctl"
+version = "3.6.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/b7/4d/08c89e34946fce2aec4fbb45c9016efd5f4d7f24af8e5d93296e935631d8/threadpoolctl-3.6.0.tar.gz", hash = "sha256:8ab8b4aa3491d812b623328249fab5302a68d2d71745c8a4c719a2fcaba9f44e", size = 21274, upload-time = "2025-03-13T13:49:23.031Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/32/d5/f9a850d79b0851d1d4ef6456097579a9005b31fea68726a4ae5f2d82ddd9/threadpoolctl-3.6.0-py3-none-any.whl", hash = "sha256:43a0b8fd5a2928500110039e43a5eed8480b918967083ea48dc3ab9f13c4a7fb", size = 18638, upload-time = "2025-03-13T13:49:21.846Z" },
+]
+
+[[package]]
+name = "toml"
+version = "0.10.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/be/ba/1f744cdc819428fc6b5084ec34d9b30660f6f9daaf70eead706e3203ec3c/toml-0.10.2.tar.gz", hash = "sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f", size = 22253, upload-time = "2020-11-01T01:40:22.204Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", hash = "sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b", size = 16588, upload-time = "2020-11-01T01:40:20.672Z" },
+]
+
+[[package]]
+name = "toolz"
+version = "1.1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/11/d6/114b492226588d6ff54579d95847662fc69196bdeec318eb45393b24c192/toolz-1.1.0.tar.gz", hash = "sha256:27a5c770d068c110d9ed9323f24f1543e83b2f300a687b7891c1a6d56b697b5b", size = 52613, upload-time = "2025-10-17T04:03:21.661Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/fb/12/5911ae3eeec47800503a238d971e51722ccea5feb8569b735184d5fcdbc0/toolz-1.1.0-py3-none-any.whl", hash = "sha256:15ccc861ac51c53696de0a5d6d4607f99c210739caf987b5d2054f3efed429d8", size = 58093, upload-time = "2025-10-17T04:03:20.435Z" },
+]
+
+[[package]]
+name = "topacedo"
+version = "0.2.10"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "networkx" },
+ { name = "numba" },
+ { name = "numpy" },
+ { name = "pandas" },
+ { name = "pcst-fast" },
+ { name = "scipy" },
+ { name = "tqdm" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/38/4b/3e5d36b7b96cc4e8dd61e9fdbe0ff748cdae4c61a9e5733a9b878de0044a/topacedo-0.2.10.tar.gz", hash = "sha256:13b3bd7753b2fdd6b7bdca13807f99f81c1decc50ebc9757d0b772bc4f6aab7f", size = 10789, upload-time = "2022-09-13T20:53:40.313Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/57/e0/cbe7b82090282888bdfa3c22c6b141e926c86a2c3310779754c59ed0f0af/topacedo-0.2.10-py3-none-any.whl", hash = "sha256:6a1131b13a105717715ec88791e8688a0a10f660ada622bcf6ced101b5701009", size = 11461, upload-time = "2022-09-13T20:53:38.652Z" },
+]
+
+[[package]]
+name = "tornado"
+version = "6.5.7"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/64/24/95ec527ad67b76d59299e5465b3935d05e4294b7e0290a3924b7487df30b/tornado-6.5.7.tar.gz", hash = "sha256:66c513a76cda70d53907bc27cf1447557699c2e95aa48ba27a442ff61c3ddfc2", size = 519252, upload-time = "2026-06-08T17:34:51.232Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/02/dc/c7043cab6fed8ae159fc1923ce829ada35c4dbd797d408a43858ffaf9639/tornado-6.5.7-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:148b2eb15c2c765a50796172c1e499649b35f30d2e3c3d3e15913cfa56bfb163", size = 448543, upload-time = "2026-06-08T17:34:38.052Z" },
+ { url = "https://files.pythonhosted.org/packages/92/4f/090b1431e5a43df696feceffc268c5383cc079ecb5f08ce58f917109aafe/tornado-6.5.7-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:9da38de27f1da3b78a966f0dae12b5a1ea9afe72ca805d84ff06508272ddf100", size = 446707, upload-time = "2026-06-08T17:34:39.594Z" },
+ { url = "https://files.pythonhosted.org/packages/37/d8/ef374952fd5da67d4463122c2b8e5a96536ec10b4b339254c6dcde81d01c/tornado-6.5.7-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:8d759e71906ee783f8867b93bf26a265743da4c1e2f4a018464c1ba019862972", size = 449774, upload-time = "2026-06-08T17:34:41.204Z" },
+ { url = "https://files.pythonhosted.org/packages/35/37/d434c73f4c6e014b745b9b37085f34f40c022f007efff3d7fe65991899f3/tornado-6.5.7-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8a46347a18f23fb92b396beebe0fb78f61dda0cc302445202c16203d8a18848b", size = 450745, upload-time = "2026-06-08T17:34:42.531Z" },
+ { url = "https://files.pythonhosted.org/packages/b6/2b/56b9aff361d7f1ab728a805ec7d7ea835f8807afa9f5cc690ea0e630efb9/tornado-6.5.7-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:7778b30bef919231265e91c69963ce0f49a1e9c07ac900bbe75b19ce2575ba92", size = 450578, upload-time = "2026-06-08T17:34:43.787Z" },
+ { url = "https://files.pythonhosted.org/packages/02/30/a7444fb23aa76860a14198fab96ac79f1866b0a6e19e26c4381b0938e50f/tornado-6.5.7-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:e726f0c75da7726eec023aa62751ff8878bd2737e34fbdd33b1ae5897d2200f5", size = 449985, upload-time = "2026-06-08T17:34:45.326Z" },
+ { url = "https://files.pythonhosted.org/packages/5c/42/5f0e56c01e8d9d36f4e23f367b85ae6cae0c1ecddd5e6977d8388ad27488/tornado-6.5.7-cp39-abi3-win32.whl", hash = "sha256:f8de3bf12d3efdd0cbe7c8887868198f8a91415e3f29fcf258d9b8eb7b1d9ae4", size = 451047, upload-time = "2026-06-08T17:34:46.784Z" },
+ { url = "https://files.pythonhosted.org/packages/c9/a4/b393076ffb21b469eec5b328a0534cf03a3b90bfc6b1f09507cdd075d938/tornado-6.5.7-cp39-abi3-win_amd64.whl", hash = "sha256:de942f843533a039ef9fa3d9c88c7cd8a7c94553fb5ad0154270989b3d99a2c4", size = 451485, upload-time = "2026-06-08T17:34:48.248Z" },
+ { url = "https://files.pythonhosted.org/packages/71/2e/7b1c769803121b809112cf9a00681c472eae1d80e32d7ec0e0bd61d0d0e1/tornado-6.5.7-cp39-abi3-win_arm64.whl", hash = "sha256:ff934fce95643af5f11efdae618eaa73d469dc588641e5c8d19295a0c65c4796", size = 450506, upload-time = "2026-06-08T17:34:49.702Z" },
+]
+
+[[package]]
+name = "tqdm"
+version = "4.68.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "colorama", marker = "sys_platform == 'win32'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/87/d7/0535a28b1f5f24f6612fb3ff1e89fb1a8d160fee0f976e0aa6803862134b/tqdm-4.68.3.tar.gz", hash = "sha256:00dfa48452b6b6cfae3dd9885636c23d3422d1ec97c66d96818cbd5e0821d482", size = 170596, upload-time = "2026-06-17T07:36:52.105Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/d8/8e/bb97bb0c71802080bfc8952937d174e49cfc50de5c951dd47b2496f0dcdb/tqdm-4.68.3-py3-none-any.whl", hash = "sha256:39832cc2def2789a6f29df83f172db7416cea70052c0907a57801c5f2fdccb03", size = 78337, upload-time = "2026-06-17T07:36:50.132Z" },
+]
+
+[[package]]
+name = "traitlets"
+version = "5.15.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/57/a9/a2584b8313b89f94869ddb3c4074617a691de1812a614d2d50e32ca5a7a6/traitlets-5.15.1.tar.gz", hash = "sha256:7b1c07854fe25acb39e009bae49f11b79ff6cbb2f27999104e9110e7a6b53722", size = 163344, upload-time = "2026-06-03T12:26:06.181Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/96/8d/1080ee4c231f361b6ce4470d556c8c435b67c7e0753aaa641497ee92f88b/traitlets-5.15.1-py3-none-any.whl", hash = "sha256:770a53705f84b81ac107e83a1b3328ff2dae16094d8fc3cfc004e4b22dfd8e92", size = 85858, upload-time = "2026-06-03T12:26:04.395Z" },
+]
+
+[[package]]
+name = "types-certifi"
+version = "2021.10.8.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/52/68/943c3aeaf14624712a0357c4a67814dba5cea36d194f5c764dad7959a00c/types-certifi-2021.10.8.3.tar.gz", hash = "sha256:72cf7798d165bc0b76e1c10dd1ea3097c7063c42c21d664523b928e88b554a4f", size = 2095, upload-time = "2022-06-09T15:19:05.244Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/b5/63/2463d89481e811f007b0e1cd0a91e52e141b47f9de724d20db7b861dcfec/types_certifi-2021.10.8.3-py3-none-any.whl", hash = "sha256:b2d1e325e69f71f7c78e5943d410e650b4707bb0ef32e4ddf3da37f54176e88a", size = 2136, upload-time = "2022-06-09T15:19:03.127Z" },
+]
+
+[[package]]
+name = "types-toml"
+version = "0.10.8.20260518"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/4b/11/6ece999e91f2ccb848ab4420f3f4816e78ac0541f739e6864affdaaa5737/types_toml-0.10.8.20260518.tar.gz", hash = "sha256:80e10facd24fdeda9d5c672187d72be3ac284843788d67f5aae59e3e016db6fe", size = 9419, upload-time = "2026-05-18T06:02:16.719Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/91/25/489751806bf5c95e4007f8e17409199c54d31e49ffbea07c5729b1286c8e/types_toml-0.10.8.20260518-py3-none-any.whl", hash = "sha256:0e564ab05f6fde62a315b3b5a9b6624fda569399795d30a37e64705a70459303", size = 9669, upload-time = "2026-05-18T06:02:15.86Z" },
+]
+
+[[package]]
+name = "typing-extensions"
+version = "4.15.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" },
+]
+
+[[package]]
+name = "typing-inspection"
+version = "0.4.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/55/e3/70399cb7dd41c10ac53367ae42139cf4b1ca5f36bb3dc6c9d33acdb43655/typing_inspection-0.4.2.tar.gz", hash = "sha256:ba561c48a67c5958007083d386c3295464928b01faa735ab8547c5692e87f464", size = 75949, upload-time = "2025-10-01T02:14:41.687Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/dc/9b/47798a6c91d8bdb567fe2698fe81e0c6b7cb7ef4d13da4114b41d239f65d/typing_inspection-0.4.2-py3-none-any.whl", hash = "sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7", size = 14611, upload-time = "2025-10-01T02:14:40.154Z" },
+]
+
+[[package]]
+name = "tzdata"
+version = "2026.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ba/19/1b9b0e29f30c6d35cb345486df41110984ea67ae69dddbc0e8a100999493/tzdata-2026.2.tar.gz", hash = "sha256:9173fde7d80d9018e02a662e168e5a2d04f87c41ea174b139fbef642eda62d10", size = 198254, upload-time = "2026-04-24T15:22:08.651Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ce/e4/dccd7f47c4b64213ac01ef921a1337ee6e30e8c6466046018326977efd95/tzdata-2026.2-py2.py3-none-any.whl", hash = "sha256:bbe9af844f658da81a5f95019480da3a89415801f6cc966806612cc7169bffe7", size = 349321, upload-time = "2026-04-24T15:22:05.876Z" },
+]
+
+[[package]]
+name = "umap-learn"
+version = "0.5.12"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numba" },
+ { name = "numpy" },
+ { name = "pynndescent" },
+ { name = "scikit-learn" },
+ { name = "scipy" },
+ { name = "tqdm" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/02/ee/af4171241117f85c74b5ca6448ea1033cc28d599c13651d67289bacd4083/umap_learn-0.5.12.tar.gz", hash = "sha256:6aff02ecac5f2aad9f3c65ee518d7ae93e1a985ae38721fdcffceee4232c33c7", size = 96672, upload-time = "2026-04-08T20:03:54.012Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/1b/98/f63318ccbe75c810011fe9233884c5d348d94d90005de1b79e5f93bef9c0/umap_learn-0.5.12-py3-none-any.whl", hash = "sha256:f2a85d2a2adcb52b541bed9b27a23ca169b56bb1b23283abeebfb8dfb8a42fe5", size = 91849, upload-time = "2026-04-08T20:03:52.561Z" },
+]
+
+[[package]]
+name = "urllib3"
+version = "2.7.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" },
+]
+
+[[package]]
+name = "watchfiles"
+version = "1.2.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "anyio" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/cd/41/5e1a4bb12aac5f1493fa1bdc11154eca3b258ca4eba65d39c473fe19d8e9/watchfiles-1.2.0.tar.gz", hash = "sha256:c995fba777f1ea992f090f9236e9284cf7a5d1a0130dd5a3d82c598cacd76838", size = 108252, upload-time = "2026-05-18T04:32:04.251Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/b8/2f/e42c992d2afda3108ea1c02acecc991b9f31d05c14adc2a7cee9ee211fc4/watchfiles-1.2.0-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:bc13eb17538be00c874699dc0abe4ee2bc8d50bb1166a6b9e175ef3fd7eb8f26", size = 400115, upload-time = "2026-05-18T04:32:02.06Z" },
+ { url = "https://files.pythonhosted.org/packages/5f/8f/6af2ea19065c91d8b0ea3516fdfc8c0d349f407e8e9fbf4e5a17360de8ad/watchfiles-1.2.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:2d95ddc1eb6914154253d239089900813f6a767e174b8e6a50e7fdacb7e4236c", size = 393659, upload-time = "2026-05-18T04:30:50.951Z" },
+ { url = "https://files.pythonhosted.org/packages/13/01/b32a967c56fb3e3e5be3db52c3d3b87fa4513aa367d8ed1ad96d42952e5f/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8f70d8b291ef6e88d19b1f297a6905ddb978888d9272b0d05e6f53309856bcfc", size = 453207, upload-time = "2026-05-18T04:31:04.231Z" },
+ { url = "https://files.pythonhosted.org/packages/04/98/97557a812180338cb1abd32e1cffcc4588f59b5f23e0cb006b2ba95ba64a/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:56d8641cf834c2836922899105bd3ce3d0dfc69291d52edf0b4d0436829b34c0", size = 459273, upload-time = "2026-05-18T04:31:50.377Z" },
+ { url = "https://files.pythonhosted.org/packages/e8/a8/b4b08dcb7653b8087c6586f7ce649505900e866bbcfe40dc9587af02e686/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:2581a94056e55d7d0a31a823ea92bf73749c489ca2285bfdc0fbe6b2bb49d50c", size = 489927, upload-time = "2026-05-18T04:31:42.485Z" },
+ { url = "https://files.pythonhosted.org/packages/50/94/3dceea03545d2e5ddfd839f0ddd5e1cecbf1697b5a428d5ba11cef6af95d/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:41bc1199f7523b3f82843c88cbb979180c949caef0342cf90968f178e5d49b01", size = 570476, upload-time = "2026-05-18T04:31:03.071Z" },
+ { url = "https://files.pythonhosted.org/packages/cc/f2/d39a5450c3532092b91f81d274360e613c2371bc874a89c7a1a3c5e8d138/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7571e4464cb6e434958f867f7f730b8ab0b75e3f8e5eac0499168486ab3c33a8", size = 465650, upload-time = "2026-05-18T04:30:12.701Z" },
+ { url = "https://files.pythonhosted.org/packages/22/24/ed72f68cbc1333ca9b9f2200aa048bb6658ae41709bc1caad4310f4bdffd/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e53a384f76b631c3ae5334ce6a52f0baa3a911eb94a4eac7f160079868b716d5", size = 456398, upload-time = "2026-05-18T04:30:13.784Z" },
+ { url = "https://files.pythonhosted.org/packages/0d/64/982ef4a4e5bab5b6e5b6becc8cd5e732f6130a78b855f0abec6439a9a135/watchfiles-1.2.0-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:d20029a60a71a052a24c4db7673bc4de39ab89adbaccbfb5d67987c5d73f424d", size = 465140, upload-time = "2026-05-18T04:31:52.111Z" },
+ { url = "https://files.pythonhosted.org/packages/a0/0c/95282abf4ed680b6096010bcfc30c5fa7a041fc5aa5a2ad17a2cc6c75bba/watchfiles-1.2.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:2cb93af48550faf1cea04c303107c8b75833de7013e57ce27d3b8d21d8d0f58c", size = 630259, upload-time = "2026-05-18T04:31:25.676Z" },
+ { url = "https://files.pythonhosted.org/packages/30/45/607c1de1530c4bdcf2cf1d1ecc2505ddba5d96bd43ba9f2b0e79876f850f/watchfiles-1.2.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:2995c176de7692b86a2e4c58d9ec718f753150a979cb4a754e2b4ffa38e70906", size = 659859, upload-time = "2026-05-18T04:30:24.333Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/08/d9e2e0f9e8e6791d33aefc694ad7eefa7f901f63caff84a81ded38692f9c/watchfiles-1.2.0-cp312-cp312-win32.whl", hash = "sha256:7a2cffd17d27d2ecbb310c2b1d8174f222a5495b1a721894afa88ec11e25b898", size = 275480, upload-time = "2026-05-18T04:30:31.307Z" },
+ { url = "https://files.pythonhosted.org/packages/1c/e6/9d42569c0102645cc8cea5d8c7d8a1e9d4ada2cb7f05f75e554b8aa2202a/watchfiles-1.2.0-cp312-cp312-win_amd64.whl", hash = "sha256:f155b3a1b2a5fc89cdc70d47ee5d54e3b75e88efa34982028a35daef9ba00379", size = 288718, upload-time = "2026-05-18T04:32:10.745Z" },
+ { url = "https://files.pythonhosted.org/packages/0a/26/88e0dc6ee3898169d7fa22bb6a69cabf2502d2ee25cb8c876d1262d204f8/watchfiles-1.2.0-cp312-cp312-win_arm64.whl", hash = "sha256:8fa585ede612ee9f9e91b18bebf9ba11b9ae29a4e3a0d0cf6fca3e382133f0d5", size = 281026, upload-time = "2026-05-18T04:30:22.23Z" },
+ { url = "https://files.pythonhosted.org/packages/d1/4d/70a7feced9f87e2ff26dba42667290f41694fc64646c67261fbb8cab5d5c/watchfiles-1.2.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:01ea8d66f0693b9b60a6541c8d10263091ca9a9060d242f3c1f3143f9aad2c98", size = 399730, upload-time = "2026-05-18T04:31:38.162Z" },
+ { url = "https://files.pythonhosted.org/packages/31/3a/0da302f2307aee316922806ebd5726c542cbd787c938271cf14a074c7daf/watchfiles-1.2.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:7ba0480b9a74af058f43b337e937a451e109295c420916d68ad24e3dc02f5e44", size = 392842, upload-time = "2026-05-18T04:30:27.051Z" },
+ { url = "https://files.pythonhosted.org/packages/db/ef/d5bdb705c224dbc256aa0c1ec47bf4e61ec52558f2afb44a71a1fe4d7015/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4f34e26a19f91f710c08e0183429f0d1d15df734e6bc78c31e77b9ea9c433658", size = 452989, upload-time = "2026-05-18T04:31:11.945Z" },
+ { url = "https://files.pythonhosted.org/packages/71/29/5495f2c1661949ef7a35e4d71111d129cfe7606414a26887a919d0a55406/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b4e77f6a55f858504069abd35d336a637555c09bca453dde1ee1e5ada8a6a1fb", size = 458978, upload-time = "2026-05-18T04:30:52.606Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/8c/7f9c07c433811c2fffd93e13fdfb7135de9aab5f2ae41be08960fa0047dc/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0cb4d80e212f116474a545c21c912b445f16bb0cef9e6a73a498164223e14e2f", size = 490248, upload-time = "2026-05-18T04:31:36.003Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/11/d93632febc52fbc21be90231bb7c17fd5387f46c9076fd40a5f9c2ae6910/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b974946a10af379d425e2eef5b62f5c6ebeaccf91d45eaad6f5b27ecd4f91aa0", size = 571847, upload-time = "2026-05-18T04:31:10.862Z" },
+ { url = "https://files.pythonhosted.org/packages/55/b4/383173e73aabb07ad1d9c7aa859d95437ac46a6d6a1e11005facda0c9d19/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:86bc13c25a8d1fcd70b51d0ce7c9b65e90de5666fcbfd3e34957cc73ee19aeb5", size = 465974, upload-time = "2026-05-18T04:30:17.006Z" },
+ { url = "https://files.pythonhosted.org/packages/a7/6c/89b1a230a78f57c52dd8893adb1f92f94411721b6ec12596c56d98c74356/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ca148d73dea36c9763aaa351e4d7a51780ec1584217c45276f4fe8239c768b71", size = 454782, upload-time = "2026-05-18T04:30:35.656Z" },
+ { url = "https://files.pythonhosted.org/packages/24/62/1732118367cfff0a9fce3bf62ff4bfded09ef5df21d9d446b858b3f70a96/watchfiles-1.2.0-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:c525543d91961c6955b2636b308569e84a1d1c5f5f2932041ab9ef46422f43e3", size = 465182, upload-time = "2026-05-18T04:30:20.846Z" },
+ { url = "https://files.pythonhosted.org/packages/28/96/716f7e5f51339bf22963f3345f9f27d7f3b30e2eadc597e257c881dd3c53/watchfiles-1.2.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:a204794696ffb8f9b10fba6f7cb5216d42f3b2b71860ccac6b6e42f5f10973b0", size = 629841, upload-time = "2026-05-18T04:31:05.397Z" },
+ { url = "https://files.pythonhosted.org/packages/4c/fe/c40783950fd771ccf66ab3ec2722d188a9af1c7f96c6e811f36e40c6e03f/watchfiles-1.2.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:10d86db20695afe7997ac9e1717637d6714a8d0220458c33f3d2061f54cec427", size = 658028, upload-time = "2026-05-18T04:31:48.22Z" },
+ { url = "https://files.pythonhosted.org/packages/71/72/4508db1856d1d87fcbb3b63f4839bab1b5682cb0e8d224d122263c09654a/watchfiles-1.2.0-cp313-cp313-win32.whl", hash = "sha256:eb283ee99e21ad6443c8cdb06ac5b34b1308c329cbdf03fa02b445363714c799", size = 275183, upload-time = "2026-05-18T04:30:59.57Z" },
+ { url = "https://files.pythonhosted.org/packages/f9/36/14b76ca57652e5cc5fd1c11f32a261292c08a0d19a00351013c2549cbfb2/watchfiles-1.2.0-cp313-cp313-win_amd64.whl", hash = "sha256:a0f27f01bee51861392bb6b7c4fdb290b27d1eb194e9e28788d68102a0e898d9", size = 288059, upload-time = "2026-05-18T04:32:07.937Z" },
+ { url = "https://files.pythonhosted.org/packages/1b/8d/0a85e395398d8d20fadfe5c5d32c726eee17a519e78fb356f2cf7531bffe/watchfiles-1.2.0-cp313-cp313-win_arm64.whl", hash = "sha256:3651aa7058595e9cfb75d35dd5ada2bf9f48a5b8a0f3562821d3e210c507e077", size = 280186, upload-time = "2026-05-18T04:31:54.484Z" },
+ { url = "https://files.pythonhosted.org/packages/37/68/36db056f1fdcc5f07302f56e631774d6835bcd6fa3ace402304621d5f9e5/watchfiles-1.2.0-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:faea288b6f0ab1902ef08f4ca6de005dccf856c4e0c4f21b8c5fce02d90a1b08", size = 399031, upload-time = "2026-05-18T04:30:44.576Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/64/01a9d6f66a82a5c101ce939274106cc72759d62427e153f01edd2b9f87c2/watchfiles-1.2.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:01859b11fd9fbca670f4d5da00fbac282cfea9bd67a2125d8b2833a3b5617ea9", size = 391205, upload-time = "2026-05-18T04:30:25.413Z" },
+ { url = "https://files.pythonhosted.org/packages/84/2c/0a44fe058cb4bb7b8ede6b6670698bbb7c0400740e378d00022189b7b31d/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fff610d7bb2256a317bb1e96f0d7862c7aa8076733ee5df0fd41bbe76a24a4f4", size = 451892, upload-time = "2026-05-18T04:32:14.005Z" },
+ { url = "https://files.pythonhosted.org/packages/67/a1/351e0d56cd35e6488b5c8b4fb11a809a5bc923e8fe8fed9faf8920be0c89/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b141a4891c995a039cd89e9a49e62df1dc8a559a5d1a6e4c7106d16c12777a55", size = 458867, upload-time = "2026-05-18T04:31:22.279Z" },
+ { url = "https://files.pythonhosted.org/packages/d5/7d/9d09605187f1b838998624049fcf8bf47b73c1a3b76901fcac1782f62277/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f22943b7770483f6ea0721c6b11d022947a98eb0acae14694de034f4d0d38925", size = 490217, upload-time = "2026-05-18T04:31:43.657Z" },
+ { url = "https://files.pythonhosted.org/packages/60/5d/a17a16eccb182f04188cd308ec24b1a71a9b5c4e7098269cf35d9fa56d02/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1bc6195825b7dcd217968bb1f801a60fd4c16e8eeab5bedc7fe917d7d5995ab4", size = 571458, upload-time = "2026-05-18T04:32:11.875Z" },
+ { url = "https://files.pythonhosted.org/packages/d3/3d/4dd457062083ab1938e5dfd45032eb425cee2ac817287ca8ff4356183e5d/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d4a4b147f5dca2a5d325a06a832fb43f345751adfbc63204aec30e0d9ca965a2", size = 464707, upload-time = "2026-05-18T04:30:43.492Z" },
+ { url = "https://files.pythonhosted.org/packages/c6/71/ea8c57b128f5383de74d0c7d2d9c57ad7c9a65a930c451bd25d524b295b7/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4543579a9bdb0c9560039b4ffddbdb39545707659fbc430ce4c10f3f68d557f9", size = 454663, upload-time = "2026-05-18T04:30:16.061Z" },
+ { url = "https://files.pythonhosted.org/packages/53/fd/2e812bf938406d7db351f0703ddd3fc6c061cf30d96153a77bc79a943a44/watchfiles-1.2.0-cp313-cp313t-manylinux_2_31_riscv64.whl", hash = "sha256:20aa0e708b920bde876a4aa82dc7dd6ebea228a63a67cda6632c2fc87b787efa", size = 463537, upload-time = "2026-05-18T04:31:44.9Z" },
+ { url = "https://files.pythonhosted.org/packages/86/56/d17a7f1dd1bc3035f1072694a551301272f1739c2d8e319c927cb9e29b38/watchfiles-1.2.0-cp313-cp313t-musllinux_1_1_aarch64.whl", hash = "sha256:d413349d565dab74297f2a63e84a097936be69bf8f3b3801f27f380e32040f44", size = 629194, upload-time = "2026-05-18T04:31:14.141Z" },
+ { url = "https://files.pythonhosted.org/packages/be/06/f1ff66bf5cae50aa4062779a0ecd0bbaf15e466195719074078947d9a17d/watchfiles-1.2.0-cp313-cp313t-musllinux_1_1_x86_64.whl", hash = "sha256:f28b2725eb8cce327b9b3ab02415c853011dc55c95832fe90de6bc56f5315f72", size = 656194, upload-time = "2026-05-18T04:31:47.14Z" },
+ { url = "https://files.pythonhosted.org/packages/e7/54/a9c7ea9a82a4ac65e7004c0a03920b5cdd2f9c3b678757d9cd425aa51d53/watchfiles-1.2.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:b8c8358484d5fa12ef34f05b7f4168eaf1932f408725ff6d023c33ec17bd79d4", size = 400205, upload-time = "2026-05-18T04:32:05.153Z" },
+ { url = "https://files.pythonhosted.org/packages/aa/5d/c9ab3534374a4a67450696905d6ef16a04405448b8dc52bd752ae50423d4/watchfiles-1.2.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:9f04b092229ad2c50126dd3c922c8822e51e605993764a33058d4a791ab42281", size = 392508, upload-time = "2026-05-18T04:30:54.849Z" },
+ { url = "https://files.pythonhosted.org/packages/26/ca/1ad30103535cf0cecd7b993e8d50edc5351b1820e38f2d22e3df58962feb/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7a7ce236284f002a156f70add88efe5c70879cccbb658be0822c54b1306fc09d", size = 452448, upload-time = "2026-05-18T04:30:53.727Z" },
+ { url = "https://files.pythonhosted.org/packages/37/a1/ceee2cdf2afbd715fa07758d39c9859513eae411b23196f7fd039e5feedd/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b9909cc2b48468b575eefa944919e1fe8a36c5849d5c7c168f80a8c1db69398e", size = 459605, upload-time = "2026-05-18T04:30:23.312Z" },
+ { url = "https://files.pythonhosted.org/packages/e8/f6/421e30fd1cb3907a84ed92ab3f1983e37ba2dca015e9a894a048418417a2/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0a37faaed405c67e28e6be45a1fa4f206ef5a2860f27c237db9fa30704c38242", size = 490757, upload-time = "2026-05-18T04:30:47.358Z" },
+ { url = "https://files.pythonhosted.org/packages/41/b0/55ed1b97ed08be7bba6f9a541cac15f2a858e1d74d2b07b6da70a82aab00/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9649193aa27bd9ff2e80ff29bfaa93085496c7a3a377592823cc58b77ee88add", size = 568672, upload-time = "2026-05-18T04:30:38.915Z" },
+ { url = "https://files.pythonhosted.org/packages/d1/cf/d8ae8a80dd7bafab395ea7681c10237311bbf34d37704a8c744e7cf31fc7/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4e4ff8e37f99cf1da89e255e07c9c4b37c214038c4283707bdec308cb1b0ea1f", size = 464197, upload-time = "2026-05-18T04:30:09.914Z" },
+ { url = "https://files.pythonhosted.org/packages/7c/8a/3076c496ca8dafe0e8cd03fcebdfc47be4b1174b4e5b24ff6e396e6b3af2/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:054dc20fd2e3132b4c3883b4a00d72fd6e1f56fdaf89fccd12e8057d74cd74d7", size = 453181, upload-time = "2026-05-18T04:30:14.829Z" },
+ { url = "https://files.pythonhosted.org/packages/e5/10/9745e17c98e7b8a86454df0a3c7b5686bd650383f1e9f26e4ebcbd6cc0c0/watchfiles-1.2.0-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:e140ed30ebde76796b686e67c182cff10ea2fbab186fafd1560f74bb5a473a6e", size = 465109, upload-time = "2026-05-18T04:30:28.123Z" },
+ { url = "https://files.pythonhosted.org/packages/8f/95/8ef4a95481d3e0cb52d62a06fa6e972e81424be2d9698b91a2fecca9904c/watchfiles-1.2.0-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:bb7e52ecf68ba46d22df23467b87cffeb2146908aa523ebfe803019618cfda06", size = 630653, upload-time = "2026-05-18T04:31:49.304Z" },
+ { url = "https://files.pythonhosted.org/packages/fd/e4/3b3bf36b0f829b50c6ebcb8d031583863c59f923d6a6af3d485e470d0fac/watchfiles-1.2.0-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:23282a321c8baf9b3a3c4afff673f9fe65eb7fdc2338d765ccad9d3d1916a5ba", size = 657838, upload-time = "2026-05-18T04:31:06.497Z" },
+ { url = "https://files.pythonhosted.org/packages/21/b1/6cbbb50c1f3002ab568777d44aa21206dfb8807a840990c4037523b51812/watchfiles-1.2.0-cp314-cp314-win32.whl", hash = "sha256:c0db965c5f79aa49fe672d297cf1febc5ad149b658594944f49a54a2b96270a7", size = 275108, upload-time = "2026-05-18T04:30:06.891Z" },
+ { url = "https://files.pythonhosted.org/packages/92/45/190ce6db8dcb4536682cf75d3889ff1a27182a58cb519d343cb6d9ea63d8/watchfiles-1.2.0-cp314-cp314-win_amd64.whl", hash = "sha256:71283b39fd17e5408eb123bd37aeecfd9d54c81fc184421943208aadb879d103", size = 288441, upload-time = "2026-05-18T04:32:12.901Z" },
+ { url = "https://files.pythonhosted.org/packages/74/0d/3eae1c2313ab08378431d907c3f8095ecca00f3eda33111cf4f0f2591799/watchfiles-1.2.0-cp314-cp314-win_arm64.whl", hash = "sha256:c5c19526f4e54a00f2666a6c0e9e40d582c09e865055ea7378bf0009aab857b3", size = 280684, upload-time = "2026-05-18T04:31:26.902Z" },
+ { url = "https://files.pythonhosted.org/packages/b1/75/fb64e6c25d6b5ca636d03df34ffb1c6e9873303e76d27967e045f8df088f/watchfiles-1.2.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:d73a585accffa5ae39c17264c36ec3166d2fad7000c780f5ef83b2722afb9dd2", size = 398857, upload-time = "2026-05-18T04:32:17.108Z" },
+ { url = "https://files.pythonhosted.org/packages/73/4e/9f7adf01754cbf81843722ccfec169d8f26c69778281a302855cecd2ee08/watchfiles-1.2.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ae99b14c5f21e026e0e9d96f40e07d8570ebee6cafd9d8fc318354606daa7a28", size = 392413, upload-time = "2026-05-18T04:31:07.911Z" },
+ { url = "https://files.pythonhosted.org/packages/47/c8/bec626bcc2d69f44b9acb24ce7d60ed7b16b73628eea747fcbd169d8edda/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4429f3b105524a10b72c3a819b091c495d2811d419c1e1e8df773a5a5974f831", size = 452409, upload-time = "2026-05-18T04:31:20.142Z" },
+ { url = "https://files.pythonhosted.org/packages/00/b7/b6362068e81e7c556d155a34c35d40ac3ef42d747b06d7f6e5bf58e359c2/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:43d818978d06062d9b22c4fab2ebe44cf5213d42dc8e62bda8c2760cfa2eeb33", size = 458827, upload-time = "2026-05-18T04:32:06.219Z" },
+ { url = "https://files.pythonhosted.org/packages/67/f8/9a813fa42afb1e0b4625e75f0479826644d3ee8dc287e093799bc01f390c/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b9f732dc58b2dbe69e464ccf8fff7a03b0dd0be439da4c0720d3558527d3d6b4", size = 490104, upload-time = "2026-05-18T04:31:56.034Z" },
+ { url = "https://files.pythonhosted.org/packages/2f/bf/27dfb6094ca4c9aad21298b5525b6c53cb36121ee454331d05161e58d130/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8f200104103feb097de4cab8fe4f5dd18a2026934c7dea98c55a2f5fd6d5a33b", size = 571360, upload-time = "2026-05-18T04:31:57.133Z" },
+ { url = "https://files.pythonhosted.org/packages/fb/39/44a096d67270ea93df91d33877dbe91fbda3aa4f8ec2edf799d93eda8736/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:63ac26eefbf4af1741247d6fb68b11c49a25b2f7413fbd318a83a12aaa9cf666", size = 464644, upload-time = "2026-05-18T04:30:57.33Z" },
+ { url = "https://files.pythonhosted.org/packages/0e/80/c7472203bad6268e3ef1ad260739704847898938ad7ea8b63a5131f46b50/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0c4997d4e4a55f0d02b6cde327322daf3a0400e5df6c6b15948994bf72497925", size = 454771, upload-time = "2026-05-18T04:30:48.736Z" },
+ { url = "https://files.pythonhosted.org/packages/51/cf/3b10b268b4b7f0fc26e9debb5eef1998b515887840f444cd3ec80c688755/watchfiles-1.2.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:4c887eba18b7945ac73067a8b4a66f21cd46c2539b2bc68588f7be6c7eb6d26b", size = 463494, upload-time = "2026-05-18T04:31:33.826Z" },
+ { url = "https://files.pythonhosted.org/packages/3d/3e/a4302545cd589262a0dc7d140e86f7688eba3f9c72776c27f7e23b8864c4/watchfiles-1.2.0-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:3416ff151bb6b5a8d8d11664974fbef4d9305b9b2957839ab5a270468fd8df30", size = 629383, upload-time = "2026-05-18T04:31:15.596Z" },
+ { url = "https://files.pythonhosted.org/packages/db/99/d5649df0a9a410d45b7c882304d0b790903ac9b6e8f2cfd12114e0c6b9f2/watchfiles-1.2.0-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:0e831a271c035d89789cffc386b6aa1375f39f1cd25eb7ca0997e4970d152fc5", size = 656093, upload-time = "2026-05-18T04:31:58.707Z" },
+ { url = "https://files.pythonhosted.org/packages/92/b9/362702539275019a54dd2e94511b31a9b89c5f9e6a21966de7eb692549fc/watchfiles-1.2.0-cp315-cp315-macosx_10_12_x86_64.whl", hash = "sha256:37a6721cdf3f65dbb13aa9503510ccb4451603ac837e44d265d7992a597e1374", size = 400109, upload-time = "2026-05-18T04:31:16.879Z" },
+ { url = "https://files.pythonhosted.org/packages/8f/75/71d5ba62db781e5587bded1d944c675374bc4aa37ff33d5018d98e8b6538/watchfiles-1.2.0-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:2b37d10b5a63bd4d87e18472d80fa525bd670586fae62e5dd580452764879b65", size = 392167, upload-time = "2026-05-18T04:31:28.058Z" },
+ { url = "https://files.pythonhosted.org/packages/3c/01/c66dd95d0423fe30d31820e2d1d5bda773764131bbb6ac0cb1cf303ac328/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0a105bc2283f67e8fbec74253ec2d94925de92ed72c0393f1206bf326b7b7b69", size = 452372, upload-time = "2026-05-18T04:31:00.836Z" },
+ { url = "https://files.pythonhosted.org/packages/91/15/2fe99557e72f85627c6a8eed50d889e8d101623e060a22ad75b875cb932d/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5327989a465505f05cfe06f04fa9d0c2fd5432bb243e10e6f012b1bdca3c8579", size = 459596, upload-time = "2026-05-18T04:31:34.96Z" },
+ { url = "https://files.pythonhosted.org/packages/ed/23/d4acfa0023367428ed48351b3b9b267893037b6cadae55620c61c24bcfd4/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ecb47f183a8025b2aa18b546725c3657e542112ae9c0613a2af79b4fa8d04ad7", size = 490869, upload-time = "2026-05-18T04:31:59.923Z" },
+ { url = "https://files.pythonhosted.org/packages/a4/5f/3164cbdce06c9fb95c4f7b9e2f9760b5e2797af43a9ecc317ef42a23a278/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8520a4ab0e37f770afc34459c4f8f7019e153f9124dc101c15538365875d1ab2", size = 571641, upload-time = "2026-05-18T04:32:00.948Z" },
+ { url = "https://files.pythonhosted.org/packages/41/e6/85d3731c55e65cd7690f3f803d24c139588aaf863e4bf2148fe7a7fa1a19/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:71cd71740ed2c15211ebb237ced4e39a1cdf6f80566e5fe95428da1626f4fde6", size = 464444, upload-time = "2026-05-18T04:30:34.298Z" },
+ { url = "https://files.pythonhosted.org/packages/f4/7d/562641012b8b09872742c3b8adf9629ec479fd78f8d68ae4a0c13da8add6/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f88af53d6ddaf72179ef613ddc905e6f4785f712b49b80b3bef9f3525e6194b4", size = 453593, upload-time = "2026-05-18T04:31:23.464Z" },
+ { url = "https://files.pythonhosted.org/packages/56/fe/cb8ef3d6f929d14158fdaaad9925985b7310abc9384dcd4d82dd0016fb59/watchfiles-1.2.0-cp315-cp315-manylinux_2_31_riscv64.whl", hash = "sha256:cee9d5efd929efdac5f7e58f72b3376f676b64050a91c5b99a7094c5b2317488", size = 465096, upload-time = "2026-05-18T04:31:30.384Z" },
+ { url = "https://files.pythonhosted.org/packages/25/91/80908e835e100527a9267147b08c0eee1fa6ab0ffec15edc04d1d44885f7/watchfiles-1.2.0-cp315-cp315-musllinux_1_1_aarch64.whl", hash = "sha256:b718bf356bbc15e559bd8ef41782b573b8ae0e3f177ab244b440568d7ea02cfb", size = 630638, upload-time = "2026-05-18T04:30:49.89Z" },
+ { url = "https://files.pythonhosted.org/packages/46/4b/95ab2f256bb4af3cb2eb23b9317bda984ee6e0f11733a5c004a6c95b06e3/watchfiles-1.2.0-cp315-cp315-musllinux_1_1_x86_64.whl", hash = "sha256:922c0e019fe68b3ae392965a766b02a71ba1168c932cebc3733cd52c5fe5b377", size = 657684, upload-time = "2026-05-18T04:31:32.027Z" },
+]
+
+[[package]]
+name = "wcwidth"
+version = "0.8.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/49/b4/51fe890511f0f242d07cb1ebe6a5b6db417262b9d2568b460347c57d95cc/wcwidth-0.8.1.tar.gz", hash = "sha256:faf5b4a5366a72dc49cad48cdf21f52bdf63bdda995178e483ba247ff79089b9", size = 1466072, upload-time = "2026-06-08T05:57:23.146Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/bd/6e/95b0e537de1f4d4301f76f944642c6da50d1511cc7b3d64dc418a66c7509/wcwidth-0.8.1-py3-none-any.whl", hash = "sha256:f453740b1e4a4f3291faa37944c555d71056c4da08d59809b307ef4feba695c8", size = 323092, upload-time = "2026-06-08T05:57:21.413Z" },
+]
+
+[[package]]
+name = "widgetsnbextension"
+version = "4.0.15"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/bd/f4/c67440c7fb409a71b7404b7aefcd7569a9c0d6bd071299bf4198ae7a5d95/widgetsnbextension-4.0.15.tar.gz", hash = "sha256:de8610639996f1567952d763a5a41af8af37f2575a41f9852a38f947eb82a3b9", size = 1097402, upload-time = "2025-11-01T21:15:55.178Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/3f/0e/fa3b193432cfc60c93b42f3be03365f5f909d2b3ea410295cf36df739e31/widgetsnbextension-4.0.15-py3-none-any.whl", hash = "sha256:8156704e4346a571d9ce73b84bee86a29906c9abfd7223b7228a28899ccf3366", size = 2196503, upload-time = "2025-11-01T21:15:53.565Z" },
+]
+
+[[package]]
+name = "win32-setctime"
+version = "1.2.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/b3/8f/705086c9d734d3b663af0e9bb3d4de6578d08f46b1b101c2442fd9aecaa2/win32_setctime-1.2.0.tar.gz", hash = "sha256:ae1fdf948f5640aae05c511ade119313fb6a30d7eabe25fef9764dca5873c4c0", size = 4867, upload-time = "2024-12-07T15:28:28.314Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/e1/07/c6fe3ad3e685340704d314d765b7912993bcb8dc198f0e7a89382d37974b/win32_setctime-1.2.0-py3-none-any.whl", hash = "sha256:95d644c4e708aba81dc3704a116d8cbc974d70b3bdb8be1d150e36be6e9d1390", size = 4083, upload-time = "2024-12-07T15:28:26.465Z" },
+]
+
+[[package]]
+name = "xarray"
+version = "2026.4.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "numpy" },
+ { name = "packaging" },
+ { name = "pandas" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/4b/a6/6fe936a798a3a38a79c7422d1a31afd2e9a14690fcb0ccff96bc01f04bf2/xarray-2026.4.0.tar.gz", hash = "sha256:c4ac9a01a945d90d5b1628e2af045099a9d4943536d4f2ee3ae963c3b222d15b", size = 3132311, upload-time = "2026-04-13T19:45:36.688Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/dc/83/6d810a8a9ebc9c307989b418840c20e46907c74d707beb67ab566773e6fc/xarray-2026.4.0-py3-none-any.whl", hash = "sha256:d43751d9fb4a90f9249c30431684f00c41bc874f1edccd862631a40cbc0edf08", size = 1414326, upload-time = "2026-04-13T19:45:34.659Z" },
+]
+
+[[package]]
+name = "yarl"
+version = "1.24.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "idna" },
+ { name = "multidict" },
+ { name = "propcache" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/79/12/1e8f37460ea0f7eb59c221fdaf0ed75e7ac43e97f8093b9c6f411df50a78/yarl-1.24.2.tar.gz", hash = "sha256:9ac374123c6fd7abf64d1fec93962b0bd4ee2c19751755a762a72dd96c0378f8", size = 210798, upload-time = "2026-05-19T21:31:05.599Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/f0/da/866bcb01076ba49d2b42b309867bed3826421f1c479655eb7a607b44f20b/yarl-1.24.2-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:b975866c184564c827e0877380f0dae57dcca7e52782128381b72feff6dfceb8", size = 129957, upload-time = "2026-05-19T21:28:51.695Z" },
+ { url = "https://files.pythonhosted.org/packages/bf/1d/fcefb70922ea2268a8971d8e5874d9a8218644200fb8465f1dcad55e6851/yarl-1.24.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3b075301a2836a0e297b1b658cb6d6135df535d62efefdd60366bd589c2c82f2", size = 92164, upload-time = "2026-05-19T21:28:53.242Z" },
+ { url = "https://files.pythonhosted.org/packages/29/b6/170e2b8d4e3bc30e6bfdcca53556537f5bf595e938632dfcb059311f3ff6/yarl-1.24.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:8ae44649b00947634ab0dab2a374a638f52923a6e67083f2c156cd5cbd1a881d", size = 91688, upload-time = "2026-05-19T21:28:54.865Z" },
+ { url = "https://files.pythonhosted.org/packages/fe/a5/c9f655d5553ea0b99fdac9d6a99ad3f9b3e73b8e5758bb46f58c9831f74c/yarl-1.24.2-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:507cc19f0b45454e2d6dcd62ff7d062b9f77a2812404e62dbdaec05b50faa035", size = 102902, upload-time = "2026-05-19T21:28:56.963Z" },
+ { url = "https://files.pythonhosted.org/packages/5d/bc/6b9664d815d79af4ee553337f9d606c56bbf269186ada9172de45f1b5f60/yarl-1.24.2-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:c4c17bad5a530912d2111825d3f05e89bab2dd376aaa8cbc77e449e6db63e576", size = 97931, upload-time = "2026-05-19T21:28:58.56Z" },
+ { url = "https://files.pythonhosted.org/packages/98/ec/32ba48acae30fecd60928f5791188b80a9d6ee3840507ffda29fecd37b71/yarl-1.24.2-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f5f0cbb112838a4a293985b6ed73948a547dadcc1ba6d2089938e7abdedceef8", size = 111030, upload-time = "2026-05-19T21:29:00.148Z" },
+ { url = "https://files.pythonhosted.org/packages/82/5a/6f4cd081e5f4934d2ae3a8ef4abe3afacc010d26f0035ee91b35cd7d7c37/yarl-1.24.2-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5ec8356b8a6afcf81fc7aeeef13b1ff7a49dec00f313394bbb9e83830d32ccd7", size = 110392, upload-time = "2026-05-19T21:29:02.155Z" },
+ { url = "https://files.pythonhosted.org/packages/7a/da/323a01c349bd5fb01bb6652e314d9bb218cee630a736bdb810ad50e4013f/yarl-1.24.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7e7ebcdef69dec6c6451e616f32b622a6d4a2e92b445c992f7c8e5274a6bbc4c", size = 105612, upload-time = "2026-05-19T21:29:04.247Z" },
+ { url = "https://files.pythonhosted.org/packages/7c/80/264ab684f181e1a876389374519ff05d10248725535ae2ac4e8ac4e563d6/yarl-1.24.2-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:47a55d6cf6db2f401017a9e96e5288844e5051911fb4e0c8311a3980f5e59a7d", size = 104487, upload-time = "2026-05-19T21:29:06.491Z" },
+ { url = "https://files.pythonhosted.org/packages/41/07/efabe5df87e96d7ad5959760b888344be48cd6884db127b407c6b5503adc/yarl-1.24.2-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:3065657c80a2321225e804048597ad55658a7e76b32d6f5ee4074d04c50401db", size = 102333, upload-time = "2026-05-19T21:29:08.267Z" },
+ { url = "https://files.pythonhosted.org/packages/44/0c/bcf7c42603e1009295f586d8890f2ba032c8b53310e815adf0a202c73d9f/yarl-1.24.2-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:cb84b80d88e19ede158619b80813968713d8d008b0e2497a576e6a0557d50712", size = 99025, upload-time = "2026-05-19T21:29:10.682Z" },
+ { url = "https://files.pythonhosted.org/packages/4f/82/84482ab1a57a0f21a08afe6a7004c61d741f8f2ecc3b05c321577c612164/yarl-1.24.2-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:990de4f680b1c217e77ff0d6aa0029f9eb79889c11fb3e9a3942c7eba29c1996", size = 110507, upload-time = "2026-05-19T21:29:12.954Z" },
+ { url = "https://files.pythonhosted.org/packages/c4/8d/a546ba1dfe1b0f290e05fef145cd07614c0f15df1a707195e512d1e39d1d/yarl-1.24.2-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:abb8ec0323b80161e3802da3150ef660b41d0e9be2048b76a363d93eee992c2b", size = 103719, upload-time = "2026-05-19T21:29:14.893Z" },
+ { url = "https://files.pythonhosted.org/packages/1a/b6/267f2a09213138473adfce6b8a6e17791d7fee70bd4d9003218e4dec58b0/yarl-1.24.2-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:e7977781f83638a4c73e0f88425563d70173e0dfd90ac006a45c65036293ee3c", size = 110438, upload-time = "2026-05-19T21:29:16.485Z" },
+ { url = "https://files.pythonhosted.org/packages/48/2d/1c8d89c7c5f9cad9fb2902445d94e2ab1d7aa35de029afbb8ae95c42d00f/yarl-1.24.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:e30dd55825dc554ec5b66a94953b8eda8745926514c5089dfcacecb9c99b5bd1", size = 105719, upload-time = "2026-05-19T21:29:18.367Z" },
+ { url = "https://files.pythonhosted.org/packages/a7/25/722e3b93bd687009afb2d59a35e13d30ddd8f80571445bb0c4e4ce26ec66/yarl-1.24.2-cp312-cp312-win_amd64.whl", hash = "sha256:7dafe10c12ddd4d120d528c4b5599c953bd7b12845347d507b95451195bb6cad", size = 92901, upload-time = "2026-05-19T21:29:20.014Z" },
+ { url = "https://files.pythonhosted.org/packages/39/47/4486ccfb674c04854a1ef8aa77868b6a6f765feaf69633409d7ca4f02cb8/yarl-1.24.2-cp312-cp312-win_arm64.whl", hash = "sha256:044a09d8401fcf8681977faef6d286b8ade1e2d2e9dceda175d1cfa5ca496f30", size = 87229, upload-time = "2026-05-19T21:29:22.1Z" },
+ { url = "https://files.pythonhosted.org/packages/82/62/fcf0ce677f17e5c471c06311dd25964be38a4c586993632910d2e75278bc/yarl-1.24.2-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:491ac9141decf49ee8030199e1ee251cdff0e131f25678817ff6aa5f837a3536", size = 128978, upload-time = "2026-05-19T21:29:23.83Z" },
+ { url = "https://files.pythonhosted.org/packages/d3/58/8e63299bb71ed61a834121d9d3fe6c9fcf2a6a5d09754ff4f20f2d20baf5/yarl-1.24.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e89418f65eda18f99030386305bd44d7d504e328a7945db1ead514fbe03a0607", size = 91733, upload-time = "2026-05-19T21:29:25.375Z" },
+ { url = "https://files.pythonhosted.org/packages/c1/24/16748d5dab6daec8b0ed81ccec639a1cded0f18dcc62a4f696b4fe366c37/yarl-1.24.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:cdfcce633b4a4bb8281913c57fcafd4b5933fbc19111a5e3930bbd299d6102f1", size = 91113, upload-time = "2026-05-19T21:29:26.928Z" },
+ { url = "https://files.pythonhosted.org/packages/1b/66/b63fff7b71211e866624b21432d5943cbb633eb0c2872d9ee3070648f22c/yarl-1.24.2-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:863297ddede92ee49024e9a9b11ecb59f310ca85b60d8537f56bed9bbb5b1986", size = 103899, upload-time = "2026-05-19T21:29:28.842Z" },
+ { url = "https://files.pythonhosted.org/packages/9d/ac/ba1974b8533909636f7733fe86cf677e3619527c3c2fa913e0ea89c48757/yarl-1.24.2-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:374423f70754a2c96942ede36a29d37dc6b0cb8f92f8d009ddf3ed78d3da5488", size = 97862, upload-time = "2026-05-19T21:29:31.086Z" },
+ { url = "https://files.pythonhosted.org/packages/1b/a5/123ac993b5c2ba6f554a140305620cb8f150fa543711bbc49be3ec0a65a4/yarl-1.24.2-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:33a29b5d00ccbf3219bb3e351d7875739c19481e030779f48cc46a7a71681a9b", size = 111060, upload-time = "2026-05-19T21:29:32.657Z" },
+ { url = "https://files.pythonhosted.org/packages/23/37/c472d3af3509688392134a88a825276770a187f1daa4de3f6dc0a327a751/yarl-1.24.2-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a9532c57211730c515341af11fef6e9b61d157487272a096d0c04da445642592", size = 110613, upload-time = "2026-05-19T21:29:34.379Z" },
+ { url = "https://files.pythonhosted.org/packages/df/88/09c28dad91e662ccfaa1b78f1c57badde74fc9d0b23e74aef644750ecd73/yarl-1.24.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:91e72cf093fd833483a97ee648e0c053c7c629f51ff4a0e7edd84f806b0c5617", size = 107012, upload-time = "2026-05-19T21:29:36.216Z" },
+ { url = "https://files.pythonhosted.org/packages/07/ab/9d4f69d571a94f4d112fa7e2e007200f5a54d319f58c82ac7b7baa61f5c6/yarl-1.24.2-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b3177bc0a768ef3bacceb4f272632990b7bea352f1b2f1eee9d6d6ff16516f92", size = 105887, upload-time = "2026-05-19T21:29:38.746Z" },
+ { url = "https://files.pythonhosted.org/packages/8e/9a/000b2b66c0d772a499fc531d21dab92dfeb73b640a12eed6ba89f49bb2d0/yarl-1.24.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:e196952aacaf3b232e265ff02980b64d483dc0972bd49bcb061171ff22ac203a", size = 103620, upload-time = "2026-05-19T21:29:40.368Z" },
+ { url = "https://files.pythonhosted.org/packages/41/7c/7c1050f73450fbdaa3f0c72017059f00ce5e13366692f3dba25275a1083d/yarl-1.24.2-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:204e7a61ce99919c0de1bf904ab5d7aa188a129ea8f690a8f76cfb6e2844dc44", size = 100599, upload-time = "2026-05-19T21:29:42.66Z" },
+ { url = "https://files.pythonhosted.org/packages/ec/b1/29e5756b3926705f5f6089bd5b9f50a56eaac550da6e260bf713ead44d04/yarl-1.24.2-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:4b156914620f0b9d78dc1adb3751141daee561cfec796088abb89ed49d220f1a", size = 110604, upload-time = "2026-05-19T21:29:44.632Z" },
+ { url = "https://files.pythonhosted.org/packages/a3/4b/8415bc96e9b150cde942fbac9a8182985e58f40ce5c54c34ed015407d3ee/yarl-1.24.2-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:8372a2b976cf70654b2be6619ab6068acabb35f724c0fda7b277fbf53d66a5cf", size = 105161, upload-time = "2026-05-19T21:29:46.755Z" },
+ { url = "https://files.pythonhosted.org/packages/8b/d4/cde059abfa229553b7298a2eadde2752e723d50aeedaef86ce59da2718ee/yarl-1.24.2-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:f9a1e9b622ca284143aab5d885848686dcd85453bb1ca9abcdb7503e64dc0056", size = 110619, upload-time = "2026-05-19T21:29:48.972Z" },
+ { url = "https://files.pythonhosted.org/packages/e7/2c/d6a6c9a61549f7b6c7e6dc6937d195bcf069582b47b7200dcd0e7b256acf/yarl-1.24.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:810e19b685c8c3c5862f6a38160a1f4e4c0916c9390024ec347b6157a45a0992", size = 107362, upload-time = "2026-05-19T21:29:51Z" },
+ { url = "https://files.pythonhosted.org/packages/92/dd/3ae5fe417e9d1c353a548553326eb9935e76b6b727161563b424cc296df3/yarl-1.24.2-cp313-cp313-win_amd64.whl", hash = "sha256:7d37fb7c38f2b6edab0f845c4f85148d4c44204f52bc127021bd2bc9fdbf1656", size = 92667, upload-time = "2026-05-19T21:29:52.743Z" },
+ { url = "https://files.pythonhosted.org/packages/10/cc/a7beb239f78f27fca1b053c8e8595e4179c02e62249b4687ec218c370c50/yarl-1.24.2-cp313-cp313-win_arm64.whl", hash = "sha256:1e831894be7c2954240e49791fa4b50c05a0dc881de2552cfe3ffd8631c7f461", size = 87069, upload-time = "2026-05-19T21:29:54.442Z" },
+ { url = "https://files.pythonhosted.org/packages/40/0e/e08087695fc12789263821c5dc0f8dc52b5b17efd0887cacf419f8a43ba3/yarl-1.24.2-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:f9312b3c02d9b3d23840f67952913c9c8721d7f1b7db305289faefa878f364c2", size = 129670, upload-time = "2026-05-19T21:29:56.631Z" },
+ { url = "https://files.pythonhosted.org/packages/3a/98/ab4b5ed1b1b5cd973c8a3eb994c3a6aefb6ce6d399e21bb5f0316c33815c/yarl-1.24.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:a4f4d6cd615823bfc7fb7e9b5987c3f41666371d870d51058f77e2680fbe9630", size = 91916, upload-time = "2026-05-19T21:29:58.645Z" },
+ { url = "https://files.pythonhosted.org/packages/ba/b1/5297bb6a7df4782f7605bffc43b31f5044070935fbbcaa6c705a07e6ac65/yarl-1.24.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:0c3063e5c0a8e8e62fae6c2596fa01da1561e4cd1da6fec5789f5cf99a8aefd8", size = 91625, upload-time = "2026-05-19T21:30:00.412Z" },
+ { url = "https://files.pythonhosted.org/packages/02/a7/45baabfff76829264e623b185cff0c340d7e11bf3e1cd9ea37e7d17934bd/yarl-1.24.2-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fecd17873a096036c1c87ab3486f1aef7f269ada7f23f7f856f93b1cc7744f14", size = 104574, upload-time = "2026-05-19T21:30:02.544Z" },
+ { url = "https://files.pythonhosted.org/packages/f3/40/3a5ab144d3d650ca37d4f4b57e56169be8af3ca34c448793e064b30baaed/yarl-1.24.2-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a46d1ab4ba4d32e6dc80daf8a28ce0bd83d08df52fbc32f3e288663427734535", size = 97534, upload-time = "2026-05-19T21:30:04.319Z" },
+ { url = "https://files.pythonhosted.org/packages/9c/b5/5658fef3681fb5776b4513b052bec750009f47b3a592251c705d75375798/yarl-1.24.2-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:73e68edf6dfd5f73f9ca127d84e2a6f9213c65bdffb736bda19524c0564fcd14", size = 111481, upload-time = "2026-05-19T21:30:05.988Z" },
+ { url = "https://files.pythonhosted.org/packages/4c/06/fdcd7dde037f00866dce123ed4ba23dba94beb56fc4cf561668d27be37f2/yarl-1.24.2-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a296ca617f2d25fbceafb962b88750d627e5984e75732c712154d058ae8d79a3", size = 111529, upload-time = "2026-05-19T21:30:07.738Z" },
+ { url = "https://files.pythonhosted.org/packages/c2/53/d81269aaafccea0d33396c03035de997b743f11e648e6e27a0df99c72980/yarl-1.24.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e51b2cf5ec89a8b8470177641ed62a3ba22d74e1e898e06ad53aa77972487208", size = 107338, upload-time = "2026-05-19T21:30:09.713Z" },
+ { url = "https://files.pythonhosted.org/packages/ae/04/23049463f729bd899df203a7960505a75333edd499cda8aa1d5a82b64df5/yarl-1.24.2-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:310fc687f7b2044ec54e372c8cbe923bb88f5c37bded0d3079e5791c2fc3cf50", size = 106147, upload-time = "2026-05-19T21:30:11.365Z" },
+ { url = "https://files.pythonhosted.org/packages/14/18/04a4b5830b43ed5e4c5015b40e9f6241ad91487d71611061b4e111d6ac80/yarl-1.24.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:297a2fe352ecf858b30a98f87948746ec16f001d279f84aebdbd3bd965e2f1bd", size = 104272, upload-time = "2026-05-19T21:30:12.978Z" },
+ { url = "https://files.pythonhosted.org/packages/5a/f7/8cffdf319aee7a7c1dbd07b61d91c3e3fda460c7a93b5f93e445f3806c4c/yarl-1.24.2-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:2a263e76b97bc42bdcd7c5f4953dec1f7cd62a1112fa7f869e57255229390d67", size = 99962, upload-time = "2026-05-19T21:30:15.001Z" },
+ { url = "https://files.pythonhosted.org/packages/d7/39/b3cce3b7dbef64ac700ad4cea156a207d01bede0f507587616c364b5468e/yarl-1.24.2-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:822519b64cf0b474f1a0aaef1dc621438ea46bb77c94df97a5b4d213a7d8a8b1", size = 111063, upload-time = "2026-05-19T21:30:16.683Z" },
+ { url = "https://files.pythonhosted.org/packages/a1/ea/100818505e7ebf165c7242ff17fdf7d9fee79e27234aeca871c1082920d7/yarl-1.24.2-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:b6067060d9dc594899ba83e6db6c48c68d1e494a6dab158156ed86977ca7bcb1", size = 105438, upload-time = "2026-05-19T21:30:18.769Z" },
+ { url = "https://files.pythonhosted.org/packages/8f/d2/e075a0b32aa6625087de9e653087df0759fed5de4a435fef594181102a77/yarl-1.24.2-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:0063adad533e57171b79db3943b229d40dfafeeee579767f96541f106bac5f1b", size = 111458, upload-time = "2026-05-19T21:30:21.024Z" },
+ { url = "https://files.pythonhosted.org/packages/e6/5c/ceea7ba98b65c8eb8d947fdc52f9bedfcd43c6a57c9e3c90c17be8f324a3/yarl-1.24.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ee8e3fb34513e8dc082b586ef4910c98335d43a6fab688cd44d4851bacfce3e8", size = 107589, upload-time = "2026-05-19T21:30:23.412Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/d9/5582d57e2b2db9b85eb6663a22efdd78e08805f3f5389566e9fcad254d1b/yarl-1.24.2-cp314-cp314-win_amd64.whl", hash = "sha256:afb00d7fd8e0f285ca29a44cc50df2d622ff2f7a6d933fa641577b5f9d5f3db0", size = 94424, upload-time = "2026-05-19T21:30:25.425Z" },
+ { url = "https://files.pythonhosted.org/packages/92/10/7dc07a0e22806a9280f42a57361395506e800c64e22737cd7b0886feab42/yarl-1.24.2-cp314-cp314-win_arm64.whl", hash = "sha256:68cf6eacd6028ef1142bc4b48376b81566385ca6f9e7dde3b0fa91be08ffcb57", size = 88690, upload-time = "2026-05-19T21:30:27.623Z" },
+ { url = "https://files.pythonhosted.org/packages/9e/13/d5b8e2c8667db955bcb3de233f18798fefe7edf1d7429c2c9d4f9c401114/yarl-1.24.2-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:221ce1dd921ac4f603957f17d7c18c5cc0797fbb52f156941f92e04605d1d67b", size = 136248, upload-time = "2026-05-19T21:30:29.297Z" },
+ { url = "https://files.pythonhosted.org/packages/de/46/a4a97c05c9c9b8fd266bb2a0df12992c7fbd02391eb9640583411b6dab32/yarl-1.24.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:5f3224db28173a00d7afacdee07045cc4673dfab2b15492c7ae10deddbece761", size = 95084, upload-time = "2026-05-19T21:30:31.031Z" },
+ { url = "https://files.pythonhosted.org/packages/95/b2/845cf2074a015e6fe0d0808cf1a2d9e868386c4220d657ebd8302b199043/yarl-1.24.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c557165320d6244ebe3a02431b2a201a20080e02f41f0cfa0ccc47a183765da8", size = 95272, upload-time = "2026-05-19T21:30:33.062Z" },
+ { url = "https://files.pythonhosted.org/packages/fe/16/e69d4aa244aef45235ddfebc0e04036a6829842bc5a6a795aedc6c998d23/yarl-1.24.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:904065e6e85b1fa54d0d87438bd58c14c0bad97aad654ad1077fd9d87e8478ed", size = 101497, upload-time = "2026-05-19T21:30:34.842Z" },
+ { url = "https://files.pythonhosted.org/packages/15/94/c07107715d621076863ee88b3ddf183fa5e9d4aba5769623c9979828410a/yarl-1.24.2-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:8cec2a38d70edc10e0e856ceda886af5327a017ccbde8e1de1bd44d300357543", size = 94002, upload-time = "2026-05-19T21:30:37.724Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/35/fc1bbdd895b5e4010b8fdd037f7ed3aa289d3863e08231b30231ca9a0815/yarl-1.24.2-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e7484b9361ed222ee1ca5b4337aa4cbdcc4618ce5aff57d9ef1582fd95893fc0", size = 106524, upload-time = "2026-05-19T21:30:40.196Z" },
+ { url = "https://files.pythonhosted.org/packages/1f/f2/32b66d0a4ba47c296cf86d03e2c67bff58399fe6d6d84d5205c04c66cc6d/yarl-1.24.2-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:84f9670b89f34db07f81e53aee83e0b938a3412329d51c8f922488be7fcc4024", size = 106165, upload-time = "2026-05-19T21:30:41.888Z" },
+ { url = "https://files.pythonhosted.org/packages/95/47/37cb5ff50c5e825d4d38e81bb04d1b7e96bf960f7ab89f9850b162f3f114/yarl-1.24.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:abb2759733d63a28b4956500a5dd57140f26486c92b2caedfb964ab7d9b79dbf", size = 103010, upload-time = "2026-05-19T21:30:43.985Z" },
+ { url = "https://files.pythonhosted.org/packages/6f/d2/4597912315096f7bb359e46e13bf8b60994fcbb2db29b804c0902ef4eff5/yarl-1.24.2-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:081c2bf54efe03774d0311172bc04fedf9ca01e644d4cd8c805688e527209bdc", size = 101128, upload-time = "2026-05-19T21:30:46.291Z" },
+ { url = "https://files.pythonhosted.org/packages/b9/d5/c8e86e120521e646013d02a8e3b8884392e28494be8f392366e50d208efc/yarl-1.24.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:86746bef442aa479107fe28132e1277237f9c24c2f00b0b0cf22b3ee0904f2bb", size = 101382, upload-time = "2026-05-19T21:30:48.085Z" },
+ { url = "https://files.pythonhosted.org/packages/fa/98/70b229236118f89dbeb739b76f10225bbf53b5497725502594c9a01d699a/yarl-1.24.2-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:2d07d21d0bc4b17558e8de0b02fbfdf1e347d3bb3699edd00bb92e7c57925420", size = 95964, upload-time = "2026-05-19T21:30:49.785Z" },
+ { url = "https://files.pythonhosted.org/packages/87/f8/56c386981e3c8648d279fdef2397ffec577e8320fd5649745e34d54faeb7/yarl-1.24.2-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:4fb1ac3fc5fecd8ae7453ea237e4d22b49befa70266dfe1629924245c21a0c7f", size = 106204, upload-time = "2026-05-19T21:30:51.862Z" },
+ { url = "https://files.pythonhosted.org/packages/1a/1e/765afe97811ca35933e2a7de70ac57b1997ea2e4ee895719ee7a231fb7e5/yarl-1.24.2-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:4da31a5512ed1729ca8d8aacde3f7faeb8843cde3165d6bcf7f88f74f17bb8aa", size = 101510, upload-time = "2026-05-19T21:30:53.62Z" },
+ { url = "https://files.pythonhosted.org/packages/ee/78/393913f4b9039e1edd09ae8a9bbb9d539be909a8abf6d8a2084585bed4b7/yarl-1.24.2-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:533ded4dceb5f1f3da7906244f4e82cf46cfd40d84c69a1faf5ac506aa65ecbe", size = 105584, upload-time = "2026-05-19T21:30:55.962Z" },
+ { url = "https://files.pythonhosted.org/packages/78/87/deb17b7049bbe74ea11a713b86f8f27800cc1c8648b0b797243ebb4830ba/yarl-1.24.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:7b3a85525f6e7eeabcfdd372862b21ee1915db1b498a04e8bf0e389b607ff0bd", size = 103410, upload-time = "2026-05-19T21:30:57.962Z" },
+ { url = "https://files.pythonhosted.org/packages/8f/be/f9f7594e23b5b93affff0318e4593c1920331bcaefda326cabcad94296a1/yarl-1.24.2-cp314-cp314t-win_amd64.whl", hash = "sha256:a7624b1ca46ca5d7b864ef0d2f8efe3091454085ee1855b4e992314529972215", size = 102980, upload-time = "2026-05-19T21:30:59.735Z" },
+ { url = "https://files.pythonhosted.org/packages/65/a4/ba80dccd3593ff1f01051a818694d07b58cb8232677ee9a22a5a1f93a9fc/yarl-1.24.2-cp314-cp314t-win_arm64.whl", hash = "sha256:e434a45ce2e7a947f951fc5a8944c8cc080b7e59f9c50ae80fd39107cf88126d", size = 91219, upload-time = "2026-05-19T21:31:01.934Z" },
+ { url = "https://files.pythonhosted.org/packages/fd/4d/4b880086bd0d3e034d25647be1d830afc3e3f610e98c4ab3490af6b1b6d5/yarl-1.24.2-py3-none-any.whl", hash = "sha256:2783d9226db8797636cd6896e4de81feed252d1db72265686c9558d97a4d94b9", size = 53576, upload-time = "2026-05-19T21:31:03.909Z" },
+]
+
+[[package]]
+name = "zarr"
+version = "3.2.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "donfig" },
+ { name = "google-crc32c" },
+ { name = "numcodecs" },
+ { name = "numpy" },
+ { name = "packaging" },
+ { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/93/8d/aeb164004f87543b06ef54f885d02c342c31ceb274e2bbec470a98927621/zarr-3.2.1.tar.gz", hash = "sha256:71565b738a0e7e8ed226f0516eba8c6bb53440ad7669a8c48ebb3534a161d035", size = 675161, upload-time = "2026-05-05T12:37:22.383Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/88/0a/469e2bd01be1490336e6c8707386845655d59261543315778a3ccc7e8019/zarr-3.2.1-py3-none-any.whl", hash = "sha256:f78cdd3d9687ad0e9f9cba2c5683b64f0c52589c19f685eeabe872e93cc0d2c7", size = 319617, upload-time = "2026-05-05T12:37:20.66Z" },
+]
+
+[[package]]
+name = "zipp"
+version = "4.1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/b9/d8/eab98a517c14134c0b2eb4e2387bc5f457334293ec5d2dd3857ec2966802/zipp-4.1.0.tar.gz", hash = "sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602", size = 26214, upload-time = "2026-05-18T20:08:57.967Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/3a/13/547360d81e6d88d58492968ffda9f9542854f11310ee556fef14260cc886/zipp-4.1.0-py3-none-any.whl", hash = "sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f", size = 10238, upload-time = "2026-05-18T20:08:57.045Z" },
+]