Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
0a22d06
feat(data): add PDS ODE and fixed-mosaic endpoint constants
TechnicToms Jul 21, 2026
1319fa9
feat(data): add PDS Orbital Data Explorer REST client
TechnicToms Jul 21, 2026
6bd35c9
feat(data): support non-reprojected reads and nodata overrides
TechnicToms Jul 21, 2026
90e89b9
feat(moon): support ODE-backed and fixed-mosaic instrument datasets
TechnicToms Jul 21, 2026
b394346
feat(moon): add experimental raw PDS granule datasets
TechnicToms Jul 21, 2026
0af04ae
feat(moon): register new datasets in the layer registry and catalog
TechnicToms Jul 21, 2026
8b96435
feat(moon): export new dataset classes from package init
TechnicToms Jul 21, 2026
6cafdc9
test(data): add PDS ODE client unit tests
TechnicToms Jul 21, 2026
633bd09
test(moon): cover ODE, mosaic, and footprint-sampling datasets
TechnicToms Jul 21, 2026
57a0887
test(moon): add unit tests for raw granule datasets
TechnicToms Jul 21, 2026
b49ad81
test(live): add live smoke tests for the new data sources
TechnicToms Jul 21, 2026
301f0aa
docs: document new data sources and raw granule datasets
TechnicToms Jul 21, 2026
f7b455e
docs: update AGENTS.md for the new data sources and phase status
TechnicToms Jul 21, 2026
2709f2c
feat(data): let ODEAsset select a non-Product ODE file role
TechnicToms Jul 21, 2026
7839f3e
feat(moon): add MiniRF S-band radar global mosaics
TechnicToms Jul 21, 2026
f6aac88
feat(data): let ODE searches narrow by productid, and add DivinerGDR
TechnicToms Jul 21, 2026
28ee076
feat(moon): add WACGLD100 and WACTiO2
TechnicToms Jul 21, 2026
e6e187e
feat(moon): add LROCWACGlobal and LROCWACColor
TechnicToms Jul 21, 2026
fed75fd
feat(moon): add LROCNACROI
TechnicToms Jul 21, 2026
6b84c3d
feat(moon): add ShadowCam and the kplo probe
TechnicToms Jul 21, 2026
9974422
feat(moon): add ClementineUVVIS and ClementineNIR, and the clementine…
TechnicToms Jul 21, 2026
6b7c822
docs: document the wider PDS ODE dataset roster
TechnicToms Jul 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 39 additions & 20 deletions AGENTS.md

Large diffs are not rendered by default.

24 changes: 24 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,30 @@ for batch in loader:
batch["image"] # torch.Tensor (16, C, H, W)
```

## Beyond the STAC catalog

Some instruments (LROC, LOLA, Mini-RF, Diviner, ShadowCam, Clementine, ...)
aren't in the USGS ARD STAC catalog at all; those datasets search the NASA
PDS Orbital Data Explorer instead, or read a single fixed mosaic URL, behind
the exact same interface. The full roster is in
[Instrument datasets](reference/datasets.md); two examples:

```python
import astrofetch as af

# LROC NAC stereo DTM sites are a few hundred named sites, not global
# coverage, so sampled windows are drawn from inside a real site by default.
nac = af.LROCNACDTM(products=["dtm", "ortho"], bbox=(3.0, 25.0, 4.5, 26.5))
sample = nac[0]

# A global 100 m WAC mosaic and a global LOLA DEM, channel-stacked with `&`.
terrain = af.LROCWACMosaic(resolution=100) & af.LOLA(resolution=100)

# ShadowCam mosaics of permanently shadowed polar craters, another
# site-based instrument -- footprint sampling applies the same way.
shadowcam = af.ShadowCam(products=["mosaic"])
```

## Discovering what data exists

The `MOON` catalog enumerates probes, instruments, and products, and points at
Expand Down
7 changes: 5 additions & 2 deletions docs/reference/catalog.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# Catalog & layer registry

`MOON` is the discovery catalog — enumerate probes, instruments, and products,
and reach the dataset classes. `LAYERS` maps each layer id to its provenance and
backing STAC collection.
and reach the dataset classes. `LAYERS` maps each layer id to its provenance
and backing source (`LayerSpec.source` is `"stac"`, `"ode"`, or `"mosaic"`).
A probe may also carry `granules`: experimental, non-map-projected dataset
classes (see [Raw granules](granules.md)) that sit outside the layer/product
contract entirely and are not part of `LAYERS`.

::: astrofetch.moon.layers.MOON
options:
Expand Down
22 changes: 20 additions & 2 deletions docs/reference/data.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,21 @@
# Data layer

Body-agnostic building blocks shared by every instrument dataset: STAC search,
windowed COG reads, the target grid, and the disposable cache.
Body-agnostic building blocks shared by every instrument dataset: STAC and
PDS ODE search, windowed raster reads, the target grid, and the disposable
cache. Three sources back instrument datasets — see
[Instrument datasets](datasets.md) for which each instrument uses:

- **STAC** (`astrofetch.data.stac`): the USGS Astrogeology Analysis Ready
Data catalog, searched via `pystac-client`.
- **PDS ODE** (`astrofetch.data.ode`): the NASA PDS Orbital Data Explorer
REST API, for instruments (LROC, LOLA, M3, ...) the STAC catalog does not
carry.
- **Fixed mosaics**: a handful of instruments are published as one global
(or near-global) file rather than many searchable items; these are read
directly from a well-known URL in `astrofetch.data.endpoints`, no search.

All three reproject through the same windowed-read path and share the same
sample-dict contract.

## Target grid

Expand All @@ -15,6 +29,10 @@ windowed COG reads, the target grid, and the disposable cache.

::: astrofetch.data.stac

## PDS ODE search

::: astrofetch.data.ode

## Cache

::: astrofetch.data.cache
Expand Down
56 changes: 55 additions & 1 deletion docs/reference/datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,68 @@

One dataset class per instrument, each a map-style `torch` dataset that samples
coregistered patches. Combine instruments with `&` to stack their channels over
the overlapping region.
the overlapping region. Every instrument dataset shares the same sample-dict
contract regardless of where its products are read from — see
[Data layer](data.md) for the three sources (STAC, PDS ODE, fixed mosaics).

For raw, non-map-projected data (camera-geometry strips), see the separate,
experimental [Raw granules](granules.md) page — a deliberately different
contract, not part of the windowed-dataset family below.

## Kaguya (SELENE)

::: astrofetch.moon.datasets.KaguyaTC

::: astrofetch.moon.datasets.KaguyaTCImagery

## Lunar Reconnaissance Orbiter

::: astrofetch.moon.datasets.LROCNACDTM

::: astrofetch.moon.datasets.LROCWACMosaic

::: astrofetch.moon.datasets.LOLA

::: astrofetch.moon.datasets.SLDEM2015

::: astrofetch.moon.datasets.MiniRF

::: astrofetch.moon.datasets.DivinerGDR

::: astrofetch.moon.datasets.WACGLD100

::: astrofetch.moon.datasets.WACTiO2

::: astrofetch.moon.datasets.LROCWACGlobal

::: astrofetch.moon.datasets.LROCWACColor

::: astrofetch.moon.datasets.LROCNACROI

## Korea Pathfinder Lunar Orbiter

::: astrofetch.moon.datasets.ShadowCam

## Clementine

::: astrofetch.moon.datasets.ClementineUVVIS

::: astrofetch.moon.datasets.ClementineNIR

## Base classes

::: astrofetch.moon.datasets.InstrumentDataset

::: astrofetch.moon.datasets.ODEInstrumentDataset

::: astrofetch.moon.datasets.MosaicDataset

::: astrofetch.moon.datasets.IntersectionDataset

## Product specs

::: astrofetch.moon.datasets.Product

::: astrofetch.moon.datasets.ODEAsset

::: astrofetch.moon.datasets.MosaicAsset
48 changes: 48 additions & 0 deletions docs/reference/granules.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Raw granules (experimental)

!!! warning "Experimental — a different contract than every other dataset"
Unlike the [instrument datasets](datasets.md), raw granule datasets are
**not** reprojected onto a common grid: each item is one raw PDS product
(an NAC/WAC calibrated strip, an M3 radiance cube, ...), read in its own
native camera/instrument geometry. No reprojection, resampling, ISIS, or
SPICE processing is applied.

Consequences:

- **No bbox windowing.** `__getitem__` returns a whole granule (or a row
range, via `rows=`), not a patch cropped to a requested extent.
- **Ragged shapes across items.** The default `DataLoader` collation will
not work; use `batch_size=None` or a custom `collate_fn`.
- **No `&` composition.** There is no shared grid to stack channels onto.

`len()` is the number of PDS ODE products matching a bbox, fetched once
and eagerly in `__init__`, so `len()` never needs a network call.

## Reading large strips

NAC/WAC strips can be gigapixel. Reading a whole granule with no `rows=`
raises once its pixel count exceeds `max_pixels` (about 512 MiB as float32
by default), naming the granule's size and suggesting a row range:

```python
import astrofetch as af

# Read only the first 512 rows of every matching strip instead of the whole
# multi-gigapixel granule.
dataset = af.LROCNACRaw(bbox=(-26.3, -50.6, -25.5, -49.7), rows=slice(0, 512))
sample = dataset[0]
sample["image"] # (bands, 512, W) float32, physical values
sample["mask"] # same-shaped bool validity
```

## Datasets

::: astrofetch.moon.granules.LROCNACRaw

::: astrofetch.moon.granules.LROCWACRaw

::: astrofetch.moon.granules.M3

## Base class

::: astrofetch.moon.granules.GranuleDataset
23 changes: 19 additions & 4 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,25 @@ this page is the short version.
| Phase | Deliverable | Status |
|:-----:|:------------|:------:|
| 0 | Scaffolding — package, CI, docs, target API | Done |
| 1 | STAC sampler MVP — bbox to coregistered `(C, H, W)` tensor | In progress |
| 2 | Datasets and transforms — grid-tile dataset, spatial splits, transforms, LRO WAC | Planned |
| 1 | STAC sampler MVP — bbox to coregistered `(C, H, W)` tensor | Done |
| 2 | Datasets and transforms — new data sources, grid-tile dataset, spatial splits, transforms | In progress |
| 3 | Release and community — PyPI, planetarypy affiliation, paper | Planned |

**Current phase: Phase 1.** `InstrumentDataset.read(bbox)` now fetches real COGs
**Current phase: Phase 2.** `InstrumentDataset.read(bbox)` fetches real COGs
from the USGS ARD catalog, reprojects them onto a common geographic grid,
applies scale/offset, mosaics overlapping items, and caches the result.
applies scale/offset, mosaics overlapping items, and caches the result — the
Phase 1 exit criterion. Phase 2 has delivered new data sources beyond STAC:
the NASA PDS Orbital Data Explorer (`ODEInstrumentDataset`) now backs a wide
roster of instruments — LROC NAC stereo DTM sites and region-of-interest
mosaics, Mini-RF S-band radar, Diviner rock abundance and regolith
temperature, WAC GLD100, TiO2, tiled global morphology, and 7-color
reflectance, ShadowCam polar mosaics and DTMs, and the Clementine UVVIS and
NIR basemaps — and fixed-URL mosaics (`MosaicDataset`) add the LRO WAC
global mosaic and the LOLA and SLDEM2015 global DEMs, all behind the same
sample-dict contract and `&` composition as the STAC-backed datasets. See
[Instrument datasets](reference/datasets.md) for the full list. An
experimental, separately-contracted raw-granule path
(`astrofetch.moon.granules`) also now exists for camera-geometry NAC/WAC
strips and M3 radiance cubes; see [Raw granules](reference/granules.md).
Still open for Phase 2: `GridTileDataset`, spatial-autocorrelation-aware
train/val/test splitting, transforms, and the WMS/WMTS rendered mode.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ nav:
- Roadmap: roadmap.md
- API Reference:
- Instrument datasets: reference/datasets.md
- Raw granules (experimental): reference/granules.md
- Catalog & registry: reference/catalog.md
- Data layer: reference/data.md

Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ dependencies = [
"torch>=2.0",
"pystac-client>=0.7",
"rasterio>=1.3",
"requests>=2.31",
]

[project.urls]
Expand Down
43 changes: 41 additions & 2 deletions src/astrofetch/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,58 @@

One dataset class per instrument; combine them with ``&`` to receive
coregistered multichannel samples. AstroFetch is a thin layer over existing
archive tooling (STAC, COGs), never a mirror.
archive tooling (STAC, COGs, PDS ODE), never a mirror of any archive.
"""

from astrofetch import moon
from astrofetch.moon import MOON, IntersectionDataset, KaguyaTC, KaguyaTCImagery
from astrofetch.moon import (
LOLA,
LROCNACDTM,
LROCNACROI,
M3,
MOON,
SLDEM2015,
WACGLD100,
ClementineNIR,
ClementineUVVIS,
DivinerGDR,
IntersectionDataset,
KaguyaTC,
KaguyaTCImagery,
LROCNACRaw,
LROCWACColor,
LROCWACGlobal,
LROCWACMosaic,
LROCWACRaw,
MiniRF,
ShadowCam,
WACTiO2,
)

__version__ = "0.1.0"

__all__ = [
"LOLA",
"MOON",
"M3",
"ClementineNIR",
"ClementineUVVIS",
"DivinerGDR",
"IntersectionDataset",
"KaguyaTC",
"KaguyaTCImagery",
"LROCNACDTM",
"LROCNACROI",
"LROCNACRaw",
"LROCWACColor",
"LROCWACGlobal",
"LROCWACMosaic",
"LROCWACRaw",
"MiniRF",
"SLDEM2015",
"ShadowCam",
"WACGLD100",
"WACTiO2",
"moon",
"__version__",
]
33 changes: 33 additions & 0 deletions src/astrofetch/data/endpoints.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,36 @@

STAC_API_ROOT = "https://stac.astrogeology.usgs.gov/api/"
"""USGS Astrogeology Analysis Ready Data STAC API root (pystac-client entry)."""

ODE_API_ROOT = "https://oderest.rsl.wustl.edu/live2/"
"""NASA PDS Orbital Data Explorer (ODE) REST API root, Washington Univ. St.
Louis. Used to search PDS3/PDS4 products (LROC, LOLA, M3, ...) by instrument
and bounding box; the USGS ARD STAC catalog does not carry these instruments.
Last verified 2026-07-20."""

LROC_WAC_MOSAIC_100M_URL = (
"https://asc-pds-services.s3.us-west-2.amazonaws.com/mosaic/"
"Lunar_LRO_LROC-WAC_Mosaic_global_100m_June2013.tif"
)
"""LRO LROC WAC global morphology mosaic, 100 m/px, equirectangular.

Not a Cloud Optimized GeoTIFF (striped, no overviews): windowed reads at
native resolution (100 m) are efficient; heavily downsampled reads are not.
Last verified 2026-07-20.
"""

LOLA_DEM_128_URL = (
"https://pds-geosciences.wustl.edu/lro/lro-l-lola-3-rdr-v1/lrolol_1xxx/"
"data/lola_gdr/cylindrical/float_img/ldem_128_float.lbl"
)
"""LOLA global DEM, 128 px/degree (~237 m/px at the equator), float32 metres
above the IAU 2015 Moon reference sphere. Detached PDS3 label; GDAL's PDS
driver resolves the sibling ``.img`` over HTTPS. Last verified 2026-07-20."""

SLDEM2015_URL = (
"https://pds-geosciences.wustl.edu/lro/lro-l-lola-3-rdr-v1/lrolol_1xxx/"
"data/sldem2015/global/float_img/sldem2015_128_60s_60n_000_360_float.lbl"
)
"""SLDEM2015: LOLA + Kaguya Terrain Camera co-registered DEM, 128 px/degree,
float32 metres. Source coverage is 60S-60N only (not a bug); windows outside
that band read back with ``mask`` all ``False``. Last verified 2026-07-20."""
Loading
Loading