Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/postCreateCommand.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
pip install --no-build-isolation --editable .

# Set up git safe directory
git config --global --add safe.directory /workspaces/nvSubquadratic-private
git config --global --add safe.directory /workspaces/nvSubquadratic

# Configure bash history search (up/down arrows)
echo 'Setting up bash history search...'
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/gpu-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
uses: int128/docker-build-cache-config-action@v1
id: cache
with:
image: ghcr.io/nvidia-bionemo/nvsubquadratic-private/ci-cache
image: ghcr.io/nvidia-bionemo/nvsubquadratic/ci-cache

- name: Build image
uses: docker/build-push-action@v5
Expand All @@ -61,7 +61,7 @@ jobs:
nvsubquadratic-ci:latest
cache-from: |
${{ steps.cache.outputs.cache-from }}
type=registry,ref=ghcr.io/nvidia-bionemo/nvsubquadratic-private/ci-cache:main
type=registry,ref=ghcr.io/nvidia-bionemo/nvsubquadratic/ci-cache:main
cache-to: ${{ steps.cache.outputs.cache-to }}

- name: Run pytest
Expand Down
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ RUN --mount=type=cache,id=apt-cache,target=/var/cache/apt,sharing=locked \
ninja-build \
git

WORKDIR /workspaces/nvSubquadratic-private
WORKDIR /workspaces/nvSubquadratic

# ── Heavy build: Apex from source (cached until apex commit changes) ──────────
# This layer is intentionally placed before COPY so code changes do not
Expand All @@ -81,7 +81,7 @@ RUN pip install --no-cache-dir -r requirements-dev.txt
# ── Source: invalidated on every code change (fast — just package install) ────
COPY . .

RUN git config --global --add safe.directory /workspaces/nvSubquadratic-private
RUN git config --global --add safe.directory /workspaces/nvSubquadratic

RUN pip install --no-cache-dir wheel-stub \
&& pip install --no-cache-dir --no-build-isolation ".[quack]" \
Expand All @@ -97,7 +97,7 @@ RUN chown -R ubuntu:ubuntu /workspaces && \
USER ubuntu

# Set environment variables for development mode
ENV PYTHONPATH=/workspaces/nvSubquadratic-private
ENV PYTHONPATH=/workspaces/nvSubquadratic

# Expose Jupyter port
EXPOSE 8888
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Open in VS Code and select "Reopen in Container". The devcontainer extension wil
```bash
# Build and run
docker build -t nvsubquadratic:dev .
docker run --gpus all -p 8888:8888 -v $(pwd):/workspaces/nvSubquadratic-private nvsubquadratic:dev
docker run --gpus all -p 8888:8888 -v $(pwd):/workspaces/nvSubquadratic nvsubquadratic:dev
```

The Dockerfile builds NVIDIA Apex from source for a broad set of NVIDIA archs by default (`7.0;7.5;8.0;8.6;8.9;9.0;10.0;12.0` — Volta through Blackwell). Two build-args let you tune the compile:
Expand Down Expand Up @@ -83,13 +83,13 @@ PLATFORM=arm64 scripts/slurm/enroot/build_sqsh.sh
apptainer build nvsubquadratic.sif nvsubquadratic.def

# Interactive shell with GPUs and live code from your checkout
apptainer shell --nv --bind $(pwd):/workspaces/nvSubquadratic-private nvsubquadratic.sif
apptainer shell --nv --bind $(pwd):/workspaces/nvSubquadratic nvsubquadratic.sif

# Run a command inside the image (example: tests)
apptainer exec --nv --bind $(pwd):/workspaces/nvSubquadratic-private nvsubquadratic.sif python -m pytest nvsubquadratic/ tests/
apptainer exec --nv --bind $(pwd):/workspaces/nvSubquadratic nvsubquadratic.sif python -m pytest nvsubquadratic/ tests/

# Use the default runscript (starts Jupyter Lab as defined in the .def)
apptainer run --nv --bind $(pwd):/workspaces/nvSubquadratic-private nvsubquadratic.sif --no-browser
apptainer run --nv --bind $(pwd):/workspaces/nvSubquadratic nvsubquadratic.sif --no-browser
```

### Conda (recommended for local development)
Expand Down
6 changes: 3 additions & 3 deletions docs/benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@

Throughput numbers, FLOP scaling, and FP16 op-level results. The
tables below are included verbatim from the
[`benchmarks/README.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/benchmarks/README.md)
[`benchmarks/README.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/benchmarks/README.md)
single-source — edits should land there, not here.

## FLOP scaling

![FLOP scaling for Hyena / attention / CKConv mixers](_static/flop_scaling.png)

See [`benchmarks/compare_flops.py`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/benchmarks/compare_flops.py)
See [`benchmarks/compare_flops.py`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/benchmarks/compare_flops.py)
for the script that produced the plot.

## ViT-5-Small throughput
Expand All @@ -22,7 +22,7 @@ start-after: '# ViT-5-Small Throughput Benchmarks'

## Op-level results

- [FP16 FFT convolution results](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/benchmarks/ops/FP16_FFTCONV_RESULTS.md)
- [FP16 FFT convolution results](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/benchmarks/ops/FP16_FFTCONV_RESULTS.md)
— accuracy and throughput of the FP16 path against the FP32 reference,
with the dual-mean-centering derivation summarised in
[FP16 Circular FFT Convolution: Derivation](ops/FP16_FFTCONV_DERIVATION.md).
4 changes: 2 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ def _read_version():
),
}

_gh_repo = "https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private"
_gh_repo = "https://github.com/NVIDIA-BioNeMo/nvSubquadratic"
_gh_blob_base = f"{_gh_repo}/blob/{github_version}"

extlinks = {
Expand Down Expand Up @@ -172,7 +172,7 @@ def _read_version():
]
html_context = {
"github_user": "NVIDIA-BioNeMo",
"github_repo": "nvSubquadratic-private",
"github_repo": "nvSubquadratic",
"github_version": github_version,
"doc_path": "docs",
}
Expand Down
38 changes: 19 additions & 19 deletions docs/examples/index.md
Original file line number Diff line number Diff line change
@@ -1,80 +1,80 @@
# Examples

Each subdirectory of [`examples/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples)
Each subdirectory of [`examples/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples)
is a self-contained training recipe. Recipes are
{class}`nvsubquadratic.lazy_config.LazyConfig` trees describing the
network, datamodule, Lightning wrapper, and trainer; running them is
`python -m experiments.run --config <path>`.

The active experimental roadmap (priorities, owners, status) lives at
[`examples/overview_tracker.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/examples/overview_tracker.md).
[`examples/overview_tracker.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/examples/overview_tracker.md).

## Classification

### MNIST / SMNIST

[`examples/mnist_classification/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/mnist_classification)
[`examples/mnist_classification/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/mnist_classification)
covers MNIST with both attention and Hyena baselines, plus a small CCNN
backbone. [`examples/smnist_classification/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/smnist_classification)
backbone. [`examples/smnist_classification/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/smnist_classification)
covers sequential MNIST (1D input).

### ImageNet

[`examples/imagenet_classification/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/imagenet_classification)
[`examples/imagenet_classification/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/imagenet_classification)
ships seven CCNN configs (Hyena / Hyena-circular / attention, with and
without augmentation, plus tiny variants for laptop sanity checks).
Representative entry points: `ccnn_7_512_hyena.py`,
`ccnn_7_512_attention.py`.

### TinyImageNet — ViT-5

[`examples/vit5_imagenet/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/vit5_imagenet)
[`examples/vit5_imagenet/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/vit5_imagenet)
is the ViT-5 baseline suite (v1–v5) with its own
[`TRACKER.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/examples/vit5_imagenet/TRACKER.md).
[`TRACKER.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/examples/vit5_imagenet/TRACKER.md).

### UCF101

[`examples/ucf101_classification/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/ucf101_classification)
[`examples/ucf101_classification/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/ucf101_classification)
covers video classification with both sequence- and clip-mode datamodules.

## Diffusion

### MNIST

[`examples/mnist_diffusion/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/mnist_diffusion)
[`examples/mnist_diffusion/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/mnist_diffusion)
is a small DDPM/JiT diffusion sanity-check.

### ImageNet

[`examples/imagenet_diffusion/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/imagenet_diffusion)
[`examples/imagenet_diffusion/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/imagenet_diffusion)
is the full ImageNet diffusion setup. See its
[README](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/examples/imagenet_diffusion/README.md)
[README](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/examples/imagenet_diffusion/README.md)
for the JiT vs Hyena-vs-attention comparison.

## Spatial recall

[`examples/spatial_recall_1d/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/spatial_recall_1d),
[`spatial_recall_2d/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/spatial_recall_2d),
[`spatial_recall_3d/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/spatial_recall_3d),
[`examples/spatial_recall_1d/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/spatial_recall_1d),
[`spatial_recall_2d/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/spatial_recall_2d),
[`spatial_recall_3d/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/spatial_recall_3d),
and the newer
[`spatial_recall_v2/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/spatial_recall_v2)
[`spatial_recall_v2/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/spatial_recall_v2)
are synthetic recall benchmarks measuring how well each mixer (Hyena,
attention, Mamba, CKConv) routes information across long-range
spatial/sequence positions. See
[`spatial_recall_v2/TRACKER.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/examples/spatial_recall_v2/TRACKER.md)
[`spatial_recall_v2/TRACKER.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/examples/spatial_recall_v2/TRACKER.md)
for the v2 task suite.

## Benchmarks

[`examples/vit_b_benchmark/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/vit_b_benchmark)
[`examples/vit_b_benchmark/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/vit_b_benchmark)
holds the throughput-comparison configs used to produce the numbers in
{doc}`../benchmarks`.

## Scientific

### The Well

[`examples/well/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/examples/well)
[`examples/well/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/examples/well)
covers The Well PDE benchmark — see its
[README](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/examples/well/README.md)
[README](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/examples/well/README.md)
for sub-datasets and baselines.
4 changes: 2 additions & 2 deletions docs/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This page walks a new user from a fresh checkout to a working Hyena
forward pass. For the full installation matrix (dev container, Docker,
Apptainer, conda, venv) see the project [README](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/README.md).
Apptainer, conda, venv) see the project [README](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/README.md).

## Requirements

Expand Down Expand Up @@ -39,7 +39,7 @@ pip install --no-build-isolation -e .
```

Docker, Apptainer, enroot/SLURM, and dev-container instructions live in
the [project README](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/README.md#installation).
the [project README](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/README.md#installation).

## Hello, Hyena

Expand Down
4 changes: 2 additions & 2 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,9 @@ Where to go next
Contributor docs
----------------

- `CONVENTIONS.md <https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/CONVENTIONS.md>`_ —
- `CONVENTIONS.md <https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/CONVENTIONS.md>`_ —
Google-style docstring guide and PR checklist (lives at the repo root).
- `docs-tracker.md <https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/docs-tracker.md>`_ —
- `docs-tracker.md <https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/docs-tracker.md>`_ —
documentation coverage status per file.

Related projects
Expand Down
18 changes: 9 additions & 9 deletions docs/reports.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,29 @@

Long-form technical reports backed by code, figures, and reproducible
scripts. Each topic lives in its own subfolder under
[`reports/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/tree/main/reports)
[`reports/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/tree/main/reports)
with one `REPORT.md` plus the supporting `.py` and `.png` files it
cites. See the
[source README](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/reports/README.md)
[source README](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/reports/README.md)
for the regeneration conventions (snake_case topic names, scripts take
`--output-dir`, runs from repo root with `PYTHONPATH=.`).

## Current topics

| Topic | Summary |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`ckconv_block_diagonal_kernel/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/reports/ckconv_block_diagonal_kernel/REPORT.md) | Block-diagonal multi-ω₀ SIREN kernel + block-aligned Gaussian mask for ViT-5 hybrid Hyena. Resolution scaling rule (`ω₀ ← m·ω₀`) verified across 1×/2×/4× grids. |
| [`siren_omega0_dimensional_scaling/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/reports/siren_omega0_dimensional_scaling/REPORT.md) | SIREN ω₀ dimensional scaling rule and supporting figures. |
| [`spatial_recall/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/reports/spatial_recall/REPORT.md) | Qualitative target-vs-prediction snapshots for the 1D/2D/3D EMNIST spatial-recall task suite (simple copy, mask selection, color selection, color conditioning). |
| [`vit5_imagenet_dataloader_profiling/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/reports/vit5_imagenet_dataloader_profiling/REPORT.md) | Feb-2026 investigation that diagnosed the CPU-decode bottleneck on ViT-5-Small ImageNet and motivated the move to the DALI-fused dataloader. |
| Topic | Summary |
| -------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`ckconv_block_diagonal_kernel/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/reports/ckconv_block_diagonal_kernel/REPORT.md) | Block-diagonal multi-ω₀ SIREN kernel + block-aligned Gaussian mask for ViT-5 hybrid Hyena. Resolution scaling rule (`ω₀ ← m·ω₀`) verified across 1×/2×/4× grids. |
| [`siren_omega0_dimensional_scaling/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/reports/siren_omega0_dimensional_scaling/REPORT.md) | SIREN ω₀ dimensional scaling rule and supporting figures. |
| [`spatial_recall/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/reports/spatial_recall/REPORT.md) | Qualitative target-vs-prediction snapshots for the 1D/2D/3D EMNIST spatial-recall task suite (simple copy, mask selection, color selection, color conditioning). |
| [`vit5_imagenet_dataloader_profiling/`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/reports/vit5_imagenet_dataloader_profiling/REPORT.md) | Feb-2026 investigation that diagnosed the CPU-decode bottleneck on ViT-5-Small ImageNet and motivated the move to the DALI-fused dataloader. |

## Adding a new report

1. `mkdir reports/<topic>/` (descriptive snake_case name).
1. Drop a `REPORT.md` plus any scripts and figures inside; keep
image links relative to the topic folder.
1. Add a row to the index table in
[`reports/README.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/reports/README.md)
[`reports/README.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/reports/README.md)
and mirror it here.
1. Re-run every script once before committing so the figures match
the reported numbers.
4 changes: 2 additions & 2 deletions docs/repository_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ tree (`benchmarks/`), and the supporting infrastructure (`scripts/`,
## Layout

```text
nvSubquadratic-private/
nvSubquadratic/
├── nvsubquadratic/ library code — see "library tree" below
├── experiments/ training framework (PyTorch Lightning)
│ ├── run.py CLI entry point
Expand Down Expand Up @@ -154,7 +154,7 @@ config tree (LazyConfig dataclasses) that fully describes one
experiment. Running it is
`python -m experiments.run --config examples/.../<config>.py`. The
live roadmap is at
[`examples/overview_tracker.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic-private/blob/main/examples/overview_tracker.md).
[`examples/overview_tracker.md`](https://github.com/NVIDIA-BioNeMo/nvSubquadratic/blob/main/examples/overview_tracker.md).

**`benchmarks/`** — The single home for performance measurement.
Op-level microbenchmarks (`benchmarks/ops/`), end-to-end model
Expand Down
2 changes: 1 addition & 1 deletion examples/vit5_imagenet/v5_patch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Attention does not use prepend_registers, so T = 1 + 4 + `num_patches` (no paddi
### 1. Clone and install nvSubquadratic

```bash
git clone <repo-url> && cd nvSubquadratic-private
git clone <repo-url> && cd nvSubquadratic

# PyTorch with CUDA 12.8
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128
Expand Down
2 changes: 1 addition & 1 deletion examples/well/v2/active_matter/_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
SPATIAL_RESOLUTION = (256, 256)
WELL_BASE_PATH = os.environ.get(
"WELL_DATA_PATH",
"/workspaces/nvSubquadratic-private/data/the_well/datasets/",
"/workspaces/nvSubquadratic/data/the_well/datasets/",
)
WELL_DATASET_NAME = "active_matter"

Expand Down
Loading
Loading