Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: Bug report
description: Something is broken or behaves unexpectedly.
title: "[Bug]: "
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to file a bug. Please fill in what you can —
a minimal reproducible example is the single most helpful thing.
- type: textarea
id: what-happened
attributes:
label: What happened?
description: What did you do, what did you expect, and what happened instead?
placeholder: I called KaguyaTC(...) and expected ..., but got ...
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Minimal reproducible example
description: The smallest snippet that triggers the bug. It will be rendered as Python.
render: python
placeholder: |
import astrofetch as af

moondata = af.KaguyaTC(products=["dtm"], bbox=(-26.3, -50.6, -25.5, -49.7))
next(iter(moondata))
validations:
required: true
- type: textarea
id: traceback
attributes:
label: Traceback or error output
description: Paste the full traceback if there is one. It will be rendered as a code block.
render: shell
- type: input
id: version
attributes:
label: AstroFetch version
description: "Output of: python -c \"import astrofetch; print(astrofetch.__version__)\""
placeholder: "0.1.0"
validations:
required: true
- type: input
id: environment
attributes:
label: Python and OS
description: Python version and operating system.
placeholder: "Python 3.12 on macOS 15"
validations:
required: true
- type: checkboxes
id: checks
attributes:
label: Before submitting
options:
- label: I searched existing issues and this is not a duplicate.
required: true
- label: The example above reproduces the problem on a clean environment.
required: false
8 changes: 8 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Questions and discussion
url: https://github.com/TechnicToms/AstroFetch/discussions
about: Ask a question or discuss an idea before filing an issue.
- name: Contributing guide
url: https://github.com/TechnicToms/AstroFetch/blob/main/CONTRIBUTING.md
about: How to set up, test, and open a pull request.
54 changes: 54 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Feature request
description: Suggest an enhancement or new capability.
title: "[Feature]: "
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
Thanks for the idea! For anything substantial, it helps to discuss the
approach here before a large PR. Note the current phase in the
[roadmap](https://github.com/TechnicToms/AstroFetch/blob/main/AGENTS.md#roadmap).
- type: textarea
id: problem
attributes:
label: What problem does this solve?
description: The use case or pain point motivating the request.
placeholder: When training on lunar patches I need ...
validations:
required: true
- type: textarea
id: proposal
attributes:
label: Proposed solution
description: What would the API or behavior look like? A code sketch is welcome.
render: python
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives considered
description: Other approaches you weighed, and why this one.
- type: dropdown
id: scope
attributes:
label: Roughly which area?
options:
- Data path (STAC / COG / grid / cache)
- New instrument dataset
- New body module
- Transforms / samplers
- Docs / examples
- Other
validations:
required: true
- type: checkboxes
id: checks
attributes:
label: Before submitting
options:
- label: I searched existing issues and this is not a duplicate.
required: true
- label: I would be willing to help implement this.
required: false
71 changes: 71 additions & 0 deletions .github/ISSUE_TEMPLATE/new_dataset.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: New dataset or body
description: Propose a new instrument dataset or a new planetary body module.
title: "[Data]: "
labels: ["dataset"]
body:
- type: markdown
attributes:
value: |
New instrument datasets and body modules are exactly the contributions
we want. AstroFetch's quantitative path needs Cloud Optimized GeoTIFFs
(COGs) discoverable via a STAC API. Rendered-only WMS/WMTS layers can be
added as a clearly-labeled non-quantitative source. See the "Adding a
new instrument dataset" section of
[CONTRIBUTING.md](https://github.com/TechnicToms/AstroFetch/blob/main/CONTRIBUTING.md).
- type: input
id: body
attributes:
label: Planetary body
placeholder: "Moon / Mars / ..."
validations:
required: true
- type: input
id: instrument
attributes:
label: Probe and instrument
placeholder: "LRO / LROC WAC"
validations:
required: true
- type: dropdown
id: source-type
attributes:
label: Data source type
options:
- STAC API with COG assets (quantitative — preferred)
- Standalone COG / mosaic (no STAC)
- WMS / WMTS rendered tiles (non-quantitative)
- Other / not sure
validations:
required: true
- type: textarea
id: source
attributes:
label: Where does the data live?
description: STAC catalog root and collection id, or the COG / service URL(s).
placeholder: |
STAC root: https://stac.astrogeology.usgs.gov/api/
Collection: kaguya_terrain_camera_usgs_dtms
COG asset key(s): dtm, orthoimage
validations:
required: true
- type: textarea
id: products
attributes:
label: Products and units
description: Which products (bands/assets), their physical units, and any scale/offset or nodata quirks.
placeholder: "DTM (metres, nodata -32767); orthoimage (radiance, DN * scale)."
- type: input
id: projection
attributes:
label: Projection / CRS
description: Native CRS of the rasters, if known (equirectangular, polar stereographic, IAU code).
placeholder: "IAU_2015:30110 (equirectangular); polar items use stereographic"
- type: checkboxes
id: checks
attributes:
label: Before submitting
options:
- label: I confirmed the data is openly accessible (no login / paywall).
required: true
- label: I searched existing issues and this is not a duplicate.
required: true
19 changes: 10 additions & 9 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ src/astrofetch/
tiles.py # secondary rendered mode: USGS WMS, Moon Trek WMTS
moon/
layers.py # layer registry (name -> STAC collection + read config) + Body/Probe/Instrument catalog (MOON)
datasets.py # instrument dataset classes (InstrumentDataset, KaguyaTC, LROCWAC) + IntersectionDataset
datasets.py # instrument dataset classes (InstrumentDataset, KaguyaTC, KaguyaTCImagery) + IntersectionDataset
tests/
unit/ # network fully mocked, runs in CI
live/ # hits real endpoints, manual trigger only
Expand All @@ -39,7 +39,7 @@ tests/

- Python 3.10+, src layout, uv-managed: `uv sync` installs the package plus the `dev` group (or `pip install -e .` for the package alone).
- Lint: `uv run ruff check` and `uv run ruff format`
- Types: `uv run mypy src` (config in pyproject.toml; keep new code typed)
- Types: `uv run ty check` (config in pyproject.toml; keep new code typed)
- Tests: `uv run pytest` (unit only; the `live` marker is deselected by default). Live endpoint tests: `uv run pytest tests/live -m live` (never run these in CI or in loops; they hit real government servers).
- Docs: `uv run --group docs mkdocs serve`

Expand Down Expand Up @@ -93,12 +93,12 @@ Widely-adopted defaults that keep the codebase consistent. When in doubt, match

Check the current phase before proposing work; for example, do not build Phase 2 datasets and transforms while Phase 1 (STAC sampler MVP) is incomplete. Everything is a thin layer above existing archive tooling, never a mirror of any archive.

**Current phase: Phase 0 (scaffolding) — the per-instrument API surface is real and stable; the data path returns synthetic placeholder tensors until the Phase 1 STAC sampler lands.**
**Current phase: Phase 1 (STAC sampler). Phase 0 scaffolding is complete. `InstrumentDataset.read` now fetches the real COGs covering a window from the USGS ARD catalog, reprojects them onto a common geographic grid, applies scale/offset, mosaics overlapping items, and caches the result — no more synthetic tensors.**

### Phase 0: Scaffolding (weekend 1)

- Repo setup: pyproject.toml (hatchling or setuptools), src layout, MIT or Apache-2.0 license, CITATION.cff.
- CI: GitHub Actions running ruff, mypy (lenient at first), pytest on 3.10 through 3.12.
- CI: GitHub Actions running ruff, ruff format, `ty` type checking, and pytest on 3.10 through 3.14.
- Testing policy established early: unit tests mock all network calls; a separate, manually triggered "live" test suite hits real endpoints.
- Placeholder docs (mkdocs-material) and a README with the one-line pitch and the target API sketch.

Expand All @@ -111,30 +111,31 @@ The single most important deliverable. Everything else builds on it.
- `astrofetch.data.stac`: query the USGS ARD catalog root with pystac-client, filter by collection and bbox.
- `astrofetch.data.raster`: windowed COG reads via rasterio, honoring scale/offset to return physical values, resampling to a requested resolution.
- `astrofetch.data.grid`: define a common target grid (equirectangular, IAU 2015 Moon), reproject and stack layers into a (C, H, W) float tensor with a per-channel metadata record.
- The sampler plugs into `InstrumentDataset.read(bbox)`, replacing the Phase 0 synthetic placeholder. Public API target (already the shipped Phase 0 surface):
- The sampler plugs into `InstrumentDataset.read(bbox)`, replacing the Phase 0 synthetic placeholder. Shipped public API:

```python
import astrofetch as af
bbox = (-60.0, 5.0, -55.0, 10.0)
moondata = af.KaguyaTC(products=["dtm"], bbox=bbox, resolution=100) & af.LROCWAC(
bbox = (-26.3, -50.6, -25.5, -49.7) # a Kaguya TC USGS DTM footprint
moondata = af.KaguyaTC(products=["dtm"], bbox=bbox, resolution=100) & af.KaguyaTCImagery(
bbox=bbox, resolution=100
)
sample = next(iter(moondata))
sample["image"] # torch.Tensor (C, H, W), physical values
sample["mask"] # torch.BoolTensor (C, H, W), validity (nodata gaps)
sample["layers"] # ["kaguya_tc_dtm", "lroc_wac"], plus bbox/crs/resolution
sample["layers"] # ["kaguya_tc_dtm", "kaguya_tc_image"], plus bbox/crs/resolution
```

- Local disk cache keyed by (collection, item, window, resolution), transparent and clearable.

Exit criteria: the quickstart notebook fetches a Reiner Gamma patch with two layers and plots it, end to end, on a clean machine.
Exit criteria: `KaguyaTC(products=["dtm", "ortho"], bbox=...)` fetches a real, coregistered two-layer patch from the USGS ARD catalog on a clean machine (covered by `tests/live`). A plotting quickstart notebook is a nice-to-have follow-up.

### Phase 2: Datasets and transforms (2 to 3 weekends)

- `astrofetch.moon.datasets`: random-bbox sampling already ships inside the instrument datasets (Phase 0); add `GridTileDataset` (deterministic tiling of an ROI) over the same `read(bbox)` interface, plus region-list sampling for the random path.
- Samplers that respect spatial autocorrelation for train/val/test splits (block splitting, not random pixels).
- Transforms: per-channel normalization stats, nodata masking, polar/equatorial projection handling made explicit.
- Secondary access mode behind the same interface: WMS/WMTS rendered mode, clearly labeled non-quantitative.
- LRO WAC global mosaic: the USGS ARD STAC catalog has no LRO WAC collection, so add it here from a non-STAC source (a public COG mosaic or WMS/WMTS), behind the same instrument-dataset interface as a new `LROCWAC` class.

Exit criteria: `DataLoader` trains a toy model on random lunar patches without custom user code.

Expand Down
2 changes: 2 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ type: software
authors:
- family-names: Sander
given-names: Tom
- family-names: Arnaut
given-names: Mirza
license: Apache-2.0
version: 0.1.0
date-released: "2026-07-08"
Expand Down
109 changes: 109 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Contributing to AstroFetch

Thanks for your interest in AstroFetch — a PyTorch-friendly, ML-ready gateway to
planetary science data. Contributions are welcome, and **new body modules (Mars
first) and new instrument datasets are especially wanted**.

This file is the quick-start for contributors. The design rules, architecture,
and phased roadmap live in [`AGENTS.md`](AGENTS.md) — read it before a
non-trivial change; it is the source of truth for how the codebase is meant to
fit together.

## Ways to contribute

- **Report a bug** or **request a feature** with the [issue templates](https://github.com/TechnicToms/AstroFetch/issues/new/choose).
- **Add an instrument dataset** (a new STAC-backed sensor) or a **new body module**.
- **Improve docs, examples, or tests.**
- Not sure where to start? Open a discussion issue first — for anything
substantial, please float the idea before writing a large PR.

## Development setup

AstroFetch uses [uv](https://docs.astral.sh/uv/) for everything. You do not need
to manage a virtualenv by hand.

```bash
git clone https://github.com/TechnicToms/AstroFetch.git
cd AstroFetch
uv sync # create the venv and install the package + dev/test groups
```

Common commands:

```bash
uv run pytest # unit tests only — no network (the CI default)
uv run ruff check # lint
uv run ruff format # format (the single source of truth for layout)
uv run ty check # type check
uv run --group docs mkdocs serve # preview the docs site at localhost:8000
```

Live endpoint tests hit real government servers and are **deselected by
default**. Run them deliberately, one at a time, when verifying an endpoint —
never in a loop:

```bash
uv run pytest tests/live -m live
```

## Making a change

1. **Branch** off `main` (`git switch -c my-change`).
2. **Write the code and its tests together.** Every new public function needs a
NumPy-style docstring with a runnable example and at least one unit test.
3. **Keep unit tests network-free.** Mock STAC responses and read small local
GeoTIFFs; never point a unit test at a live service. If you add or change an
endpoint or query, add or update the corresponding fixture.
4. **Run the full local gate before pushing:**
```bash
uv run ruff check && uv run ruff format --check && uv run ty check && uv run pytest
```
5. **Open a PR** into `main` with a clear description. If you reimplemented
anything instead of wrapping existing archive tooling, justify it there
(see design rule 1 in `AGENTS.md`).

## Coding conventions

These are enforced in CI; the short version:

- **Formatting is automated.** `ruff format` owns layout, quotes, and line
length — do not hand-align or fight it. Run it before every commit.
- **Imports** are sorted and grouped (stdlib, third-party, first-party) by
ruff's isort rules; no wildcard or unused imports.
- **Type hints** on every public function; keep `ty check` clean.
- **Naming** follows PEP 8 (`snake_case`, `PascalCase`, `UPPER_SNAKE_CASE`).
- **Conventional commits** (`feat:`, `fix:`, `docs:`, `test:`, `refactor:`,
`chore:`).
- **No print statements** in library code; use the `astrofetch` logger. Raise
`EndpointError` for remote failures and `ValueError` for bad user input.

Non-negotiable architecture rules (endpoint URLs live only in
`data/endpoints.py`, the quantitative COG path and rendered tile path never mix,
the cache is disposable, and so on) are spelled out in `AGENTS.md` — please skim
them so review can focus on the substance of your change.

## Adding a new instrument dataset

The typical shape of such a change:

1. Add the dataset class in `src/astrofetch/moon/datasets.py` (subclass
`InstrumentDataset`, declare its `probe`, `instrument`, STAC `collection`,
and `all_products` map of product → `Product(layer, asset)`).
2. Register its layers and wire it into the `MOON` catalog in
`src/astrofetch/moon/layers.py`.
3. Export it from `src/astrofetch/moon/__init__.py` and `astrofetch/__init__.py`.
4. Add unit tests (mocked reads) and, if useful, a single live check.

A new **body** (e.g. Mars) is a new sibling module under `src/astrofetch/`, not
edits scattered through the Moon code — the body-agnostic `data/` layer is
reused as-is.

## Code of conduct

Be kind, be constructive, assume good faith. Planetary science is a small,
collaborative community; let's keep it welcoming.

## License

By contributing, you agree that your contributions are licensed under the
project's [Apache 2.0](LICENSE) license.
Loading
Loading