Port NDR reader framework from MATLAB to Python

.github/workflows/ci.yml

@@ -0,0 +1,47 @@
+name: CI
+
+on:
+  push:
+    branches: [main, master]
+  pull_request:
+    branches: [main, master]
+
+jobs:
+  lint:
+    name: Lint
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install dependencies
+        run: pip install black ruff
+
+      - name: Check formatting with black
+        run: black --check src/ tests/
+
+      - name: Lint with ruff
+        run: ruff check src/ tests/
+
+  test:
+    name: Test (Python ${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install package with dev dependencies
+        run: pip install -e ".[dev]"
+
+      - name: Run tests
+        run: pytest tests/ -v --tb=short

AGENTS.md

@@ -0,0 +1,33 @@
+# Instructions for AI Agents
+
+## Overview
+NDR-python is a faithful Python port of NDR-matlab (Neuroscience Data Reader).
+
+## Architecture
+- **Lead-Follow:** MATLAB is the source of truth. Python mirrors it exactly.
+- **Bridge Contract:** Each sub-package has an `ndr_matlab_python_bridge.yaml`
+  defining the function names, arguments, and return types.
+- **Naming:** Preserve MATLAB names exactly. Use `readchannels_epochsamples`,
+  not `read_channels_epoch_samples`.
+
+## Key Classes
+- `ndr.reader.base` — Abstract base class. All readers inherit from this.
+- `ndr.reader` (wrapper) — High-level interface that delegates to a base reader.
+- `ndr.reader.intan_rhd`, `ndr.reader.axon_abf`, etc. — Format-specific readers.
+
+## Workflow
+1. Check the bridge YAML in the target package.
+2. If the function is missing, add it based on the MATLAB source.
+3. Record the MATLAB git hash in `matlab_last_sync_hash`.
+4. Implement the Python code.
+5. Run `black` and `ruff check --fix` before committing.
+6. Run `pytest` to verify.
+
+## Testing
+- Unit tests: `pytest tests/`
+- Symmetry tests: `pytest tests/symmetry/` (excluded from default run)
+
+## Environment
+- Python 3.10+
+- NumPy for all numerical data
+- Pydantic for input validation (`@validate_call`)

README.md

@@ -1,2 +1,67 @@
 # NDR-python
-Neuroscience Data Readers, Python version
+
+Neuroscience Data Reader - Python implementation.
+
+A faithful Python port of [NDR-matlab](https://github.com/VH-Lab/NDR-matlab), following a lead-follow architecture where MATLAB is the source of truth.
+
+## Overview
+
+NDR (Neuroscience Data Reader) is a lower-level data-reading library used by [NDI](https://github.com/VH-Lab/NDI-matlab). It provides:
+
+- An abstract base reader class (`ndr.reader.base`)
+- A high-level reader wrapper (`ndr.reader`)
+- Format-specific reader subclasses (Intan RHD, Axon ABF, CED SMR, etc.)
+- Format handler packages with low-level file I/O
+- Time, string, data, and file utilities
+
+## Installation
+
+```bash
+pip install -e ".[dev]"
+```
+
+## Quick Start
+
+```python
+from ndr.reader_wrapper import Reader
+
+# Create a reader for Intan RHD files
+r = Reader("intan")
+
+# Read channels from an epoch
+epochfiles = ["/path/to/data.rhd"]
+channels = r.getchannelsepoch(epochfiles)
+data, time = r.read(epochfiles, "ai1-3", t0=0, t1=10)
+```
+
+## Supported Formats
+
+| Format | Reader Class | Status |
+|--------|-------------|--------|
+| Intan RHD | `ndr.reader.intan_rhd.IntanRHD` | Implemented |
+| Axon ABF | `ndr.reader.axon_abf.AxonABF` | Implemented (requires `pyabf`) |
+| CED SMR | `ndr.reader.ced_smr.CedSMR` | Implemented (requires `neo`) |
+| SpikeGadgets REC | `ndr.reader.spikegadgets_rec.SpikeGadgetsRec` | Stub |
+| TDT SEV | `ndr.reader.tdt_sev.TdtSev` | Stub |
+| Neo | `ndr.reader.neo.NeoReader` | Stub |
+| White Matter | `ndr.reader.whitematter.WhiteMatter` | Stub |
+| BJG | `ndr.reader.bjg.BJG` | Stub |
+| Dabrowska | `ndr.reader.dabrowska.Dabrowska` | Stub |
+
+## Testing
+
+```bash
+# Run unit tests
+pytest tests/ -v
+
+# Run symmetry tests (cross-language verification)
+pytest tests/symmetry/ -v
+```
+
+## Architecture
+
+See [AGENTS.md](AGENTS.md) and [docs/developer_notes/](docs/developer_notes/) for details on the lead-follow architecture, bridge YAML protocol, and porting guidelines.
+
+## License
+
+CC-BY-NC-SA-4.0

docs/developer_notes/PYTHON_PORTING_GUIDE.md

@@ -0,0 +1,44 @@
+# NDR MATLAB to Python Porting Guide
+
+## 1. The Core Philosophy: Lead-Follow Architecture
+The MATLAB codebase is the **Source of Truth**. The Python version is a "faithful mirror."
+When a conflict arises between "Pythonic" style and MATLAB symmetry, **symmetry wins**.
+
+- **Lead-Follow:** MATLAB defines the logic, hierarchy, and naming.
+- **The Contract:** Every package contains an `ndr_matlab_python_bridge.yaml`.
+  This file is the binding contract for function names, arguments, and return types.
+
+## 2. Naming & Discovery (The Mirror Rule)
+Function and class names must match MATLAB exactly.
+
+- **Naming Source:** Refer to the local `ndr_matlab_python_bridge.yaml`.
+- **Missing Entries:** If a function is not in the bridge file, refer to the MATLAB
+  source, add the entry to the bridge file, and notify the user.
+- **Case Preservation:** Use `readchannels_epochsamples`, not `read_channels_epoch_samples`.
+- **Directory Parity:** Python file paths must mirror MATLAB `+namespace` paths
+  (e.g., `+ndr/+reader` -> `src/ndr/reader/`).
+
+## 3. The Porting Workflow (The Bridge Protocol)
+1. **Check the Bridge:** Open the `ndr_matlab_python_bridge.yaml` in the target package.
+2. **Sync the Interface:** If the function is missing or outdated, update the YAML first.
+3. **Record the Sync Hash:** Store the short git hash of the MATLAB `.m` file:
+   `git log -1 --format="%h" -- <path-to-matlab-file>`
+4. **Implement:** Write Python code to satisfy the interface defined in the YAML.
+5. **Log & Notify:** Record the sync date in the YAML's `decision_log`.
+
+## 4. Input Validation: Pydantic is Mandatory
+Use `@pydantic.validate_call` on all public-facing API functions.
+
+- MATLAB `double`/`numeric` -> Python `float | int`
+- MATLAB `char`/`string` -> Python `str`
+- MATLAB `{member1, member2}` -> Python `Literal["member1", "member2"]`
+
+## 5. Multiple Returns (Outputs)
+Return multiple values as a **tuple** in the exact order defined in the YAML.
+
+## 6. Code Style & Linting
+- **Black:** The sole code formatter. Line length 100.
+- **Ruff:** The primary linter. Run `ruff check --fix` before committing.
+
+## 7. Error Handling
+If MATLAB throws an `error`, Python MUST raise a corresponding Exception.

docs/developer_notes/ndr_matlab_python_bridge.yaml

@@ -0,0 +1,40 @@
+# The NDR Bridge Protocol: YAML Specification
+#
+# Name:     ndr_matlab_python_bridge.yaml
+# Location: One file per sub-package directory
+#           (e.g., src/ndr/reader/ndr_matlab_python_bridge.yaml).
+# Role:     Primary Contract. Defines how MATLAB names and types map to Python.
+
+project_metadata:
+  bridge_version: "1.1"
+  naming_policy: "Strict MATLAB Mirror"
+  indexing_policy: "Semantic Parity (1-based for user concepts, 0-based for internal data)"
+
+# When porting a function:
+# 1. Check:  Does the function/class exist in the YAML?
+# 2. Add/Update: If missing or changed, update the YAML first.
+# 3. Record Hash: git log -1 --format="%h" -- <path-to-matlab-file>
+# 4. Notify: Tell the user what was added/changed.
+
+# --- Example: Class ---
+# - name: base
+#   type: class
+#   matlab_path: "+ndr/+reader/base.m"
+#   python_path: "ndr/reader/base.py"
+#   matlab_last_sync_hash: "a4c9e07"
+#   methods:
+#     - name: readchannels_epochsamples
+#       input_arguments:
+#         - name: channeltype
+#           type_python: "str"
+#         - name: channel
+#           type_python: "int | list[int]"
+#         - name: epoch
+#           type_python: "str | int"
+#         - name: s0
+#           type_python: "int"
+#         - name: s1
+#           type_python: "int"
+#       output_arguments:
+#         - name: data
+#           type_python: "numpy.ndarray"

docs/developer_notes/ndr_xlang_principles.md

@@ -0,0 +1,26 @@
+# NDR Cross-Language (MATLAB/Python) Principles
+
+- **Status:** Active
+- **Scope:** Universal (Applies to all NDR implementations)
+
+## 1. Indexing & Counting (The Semantic Parity Rule)
+- Python uses 0-indexing internally.
+- User-facing concepts (Epochs, Channels) use 1-based numbering in both languages.
+- Python code accepts `channel_number=1` from user, maps to `data[0]` internally.
+
+## 2. Data Containers
+- Prefer NumPy over lists for numerical data.
+- MATLAB `double` array -> `numpy.ndarray` in Python.
+
+## 3. Multiple Returns
+- Python returns multiple values as a tuple in MATLAB signature order.
+
+## 4. Booleans
+- MATLAB `1`/`0` (logical) -> Python `True`/`False`.
+
+## 5. Strings
+- MATLAB `char` and `string` -> Python `str`.
+- MATLAB cell array of strings -> Python `list[str]`.
+
+## 6. Error Philosophy
+- No silent failures. If MATLAB errors, Python raises an exception.

docs/developer_notes/symmetry_tests.md

@@ -0,0 +1,53 @@
+# Cross-Language Symmetry Test Framework
+
+**Status:** Active
+**Scope:** NDR-python <-> NDR-matlab parity
+
+## Purpose
+Symmetry tests verify that data read by one language implementation matches
+the other. This ensures the Python and MATLAB NDR stacks remain interoperable.
+
+## Architecture
+
+| Phase | Python location | MATLAB location |
+|-------|----------------|-----------------|
+| **makeArtifacts** | `tests/symmetry/make_artifacts/` | `tests/+ndr/+symmetry/+makeArtifacts/` |
+| **readArtifacts** | `tests/symmetry/read_artifacts/` | `tests/+ndr/+symmetry/+readArtifacts/` |
+
+### Artifact Directory Layout
+
+```
+<tempdir>/NDR/symmetryTest/
+├── pythonArtifacts/
+│   └── <namespace>/<className>/<testName>/
+│       ├── readData.json           # Channel data, timestamps, etc.
+│       └── metadata.json           # Channel list, sample rates, epoch info
+└── matlabArtifacts/
+    └── <namespace>/<className>/<testName>/
+        └── ... (same structure)
+```
+
+### Workflow
+
+1. **makeArtifacts** (Python or MATLAB) reads example data files and writes
+   JSON artifacts containing: channel lists, sample rates, epoch clocks,
+   t0/t1 boundaries, and actual data samples.
+
+2. **readArtifacts** (the other language) loads the same example data files,
+   reads the same channels/epochs, and compares against the stored artifacts.
+
+Each `readArtifacts` test is parameterized over `{matlabArtifacts, pythonArtifacts}`.
+
+## Running
+
+```bash
+# Generate artifacts
+pytest tests/symmetry/make_artifacts/ -v
+
+# Verify artifacts
+pytest tests/symmetry/read_artifacts/ -v
+```
+
+## Writing a New Symmetry Test
+See NDI-python's `docs/developer_notes/symmetry_tests.md` for the full template.
+Adapt the pattern for NDR's reader-centric API.

pyproject.toml

@@ -0,0 +1,62 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "ndr"
+version = "0.1.0"
+description = "Neuroscience Data Reader - Python implementation"
+readme = "README.md"
+license = {text = "CC-BY-NC-SA-4.0"}
+authors = [{name = "VH-Lab", email = "vhlab@brandeis.edu"}]
+maintainers = [{name = "Waltham Data Science"}]
+requires-python = ">=3.10"
+dependencies = [
+    "numpy>=1.20.0",
+    "pydantic>=2.0",
+    "vhlab-toolbox-python @ git+https://github.com/VH-Lab/vhlab-toolbox-python.git@main",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.0.0",
+    "pytest-cov>=4.0.0",
+    "black>=23.0.0",
+    "ruff>=0.1.0",
+    "mypy>=1.0.0",
+]
+formats = [
+    "pyabf>=2.3.0",
+    "neo>=0.12.0",
+    "tdt>=0.5.0",
+]
+
+[tool.hatch.metadata]
+allow-direct-references = true
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/ndr"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+python_files = ["test_*.py"]
+addopts = "-v --tb=short --ignore=tests/symmetry"
+markers = [
+    "slow: marks tests as slow",
+    "symmetry: marks cross-language symmetry tests",
+]
+
+[tool.black]
+line-length = 100
+target-version = ["py310", "py311", "py312"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py310"
+
+[tool.ruff.lint]
+select = ["E", "W", "F", "I", "B", "C4", "UP"]
+ignore = ["E501", "B905", "E402", "B017", "B028"]
+
+[tool.ruff.lint.per-file-ignores]
+"__init__.py" = ["F401", "UP035"]

src/ndr/__init__.py

@@ -0,0 +1,8 @@
+"""NDR - Neuroscience Data Reader.
+
+A Python port of NDR-matlab (https://github.com/VH-Lab/NDR-matlab).
+"""
+
+from ndr.reader_wrapper import Reader as reader
+
+__version__ = "0.1.0"

src/ndr/data/__init__.py

@@ -0,0 +1,7 @@
+"""NDR data utilities.
+
+Port of +ndr/+data/
+"""
+
+from ndr.data.colvec import colvec
+from ndr.data.rowvec import rowvec