Skip to content

Redesign eeweather around a location-first, source-agnostic API - #104

Open
travis-recurve wants to merge 5 commits into
masterfrom
feature/architecture-reshape
Open

Redesign eeweather around a location-first, source-agnostic API#104
travis-recurve wants to merge 5 commits into
masterfrom
feature/architecture-reshape

Conversation

@travis-recurve

Copy link
Copy Markdown
Contributor

Checklist

  • Pull request targets a feature branch, not master.
  • Tests pass and coverage has not fallen (306 tests, fully offline; run serially with pytest -o addopts= -p no:xdist).
  • CHANGELOG.md updated under the Development section.
  • PEP 008 style (not blackened, per maintainer preference).
  • New functions/classes have numpy-style docstrings; user documentation lives under docs/ (the sphinx build is retired — the docs/*.md set is the source for the eeweather section of opendsm.energy).
  • Commit messages are capitalized and carry a DCO Signed-off-by.

Description

A ground-up 1.0 redesign. This is a breaking release: the public API is not compatible with 0.3.x, so the change is consolidated rather than aliased. Delivered as five dependency-ordered commits (registry schema → rebuild/refresh → source-agnostic engine + WeatherStationWeatherLocation → docs/packaging).

  • Location-first API. WeatherLocation(latitude, longitude) (or WeatherLocation.from_place("zcta", code)) is the primary entry point: weather at a point, estimated by configurable sources. location.candidates() ranks nearby stations and location.zones gives its climate zones. WeatherStation keys on a station id when the station is already known.
  • One loading verb. load_data(start, end, frequency, variables, source=) returns (DataFrame, warnings) with per-source provenance on the frame's attrs and on the object. Each requested variable routes to the first configured source that serves it; typical-year sources ("tmy3", "cz2010") are reachable only by an explicit source= pin.
  • GHCN-primary identity. Stations are keyed by their GHCN id; usaf/wban/icao/wmo are aliases that translate to it (WeatherStation.from_usaf(...), translate(...), search(...)).
  • Curated variable vocabulary with fixed units and per-variable aggregation (temperature, dew point, relative humidity, wind speed, station pressure, visibility); coarser-than-hourly frequencies aggregate by each variable's rule (means vs. never-gap-interpolated sums), finer-than-hourly interpolate. Any pandas offset alias is accepted.
  • Source-agnostic engine. GHCNh, TMY3, and CZ2010 are served through one interface; custom sources register at runtime (eeweather.sources.register) — station feeds, normals, or location-keyed estimation (grid) sources — without changing the library.
  • Reproducible replay. A location pins itself to the station resolved at first load; to_dict/to_json + from_dict/from_json carry that state across processes, so a reporting-period load reuses the same station as its baseline.
  • Self-updating packaged registry. The packaged data refreshes itself in place when it is more than ~6 months old (CDN-published pack, with a NOAA rebuild fallback); EEWEATHER_AUTO_UPDATE=0 disables it. The in-place swap is crash-safe via a commit marker — readers fall back to the packaged pair if a swap is interrupted.

Migration from 0.3.x

0.3.x 1.0
ISDStation(usaf_id) WeatherStation.from_usaf(usaf_id) (or WeatherStation(station_id))
load_isd_hourly_temp_data(...) / load_isd_daily_temp_data(...) WeatherStation(...).load_data(start, end, frequency="h"/"D")
load_cached_isd_* / load_cached_gsod_* load_data(..., fetch_from_web=False)
load_cached_tmy3_* / load_cached_cz2010_* load_data(..., source="tmy3"/"cz2010")
rank_stations(lat, lng, ...) WeatherLocation(lat, lng).candidates(...)
combine_ranked_stations, select_station internal (eeweather.sources.matching)
zcta_to_lat_long(zcta) + ranking WeatherLocation.from_place("zcta", code)
get_lat_long_climate_zones(lat, lng) WeatherLocation(lat, lng).zones
plot_station_mapping(...) removed (visualization dropped)
ISD / GSOD data GHCNh (the default observations source)

Deferred / known limitations

  • Gridded sources are not first-class yet. The engine is source-agnostic and register() accepts location-keyed estimation (Source) sources, but no built-in grid adapter ships in this PR, and gridded sources do not participate in station ranking (candidates() is station-only). International coverage via a gridded reanalysis source (ERA5 / MERRA-2) and wider GHCNh station admission are a follow-up source-expansion PR.
  • Concurrent updaters are not serialized across processes. The background auto-update is gated once per process and per machine; a manual python -m eeweather.registry.update racing it is the unguarded case. Readers stay consistent regardless (the commit marker never certifies a torn pair).
  • The packaged data is a dated snapshot (current GHCNh). The scheduled refresh-registry workflow advances it, and a few registry-count test pins move with it by design.

Validation

306 tests, fully offline (the suite fails any unmocked network call). Engine behavior, parsers, resampling, the update state machine, and registry invariants are pinned to real captured values, not just shapes. GHCNh's equivalence to the retired ISD/GSOD feeds was established by the overlap validation in the preceding migration (#103).

@travis-recurve
travis-recurve force-pushed the feature/architecture-reshape branch from 00a4ac4 to a21982b Compare July 25, 2026 01:20
Normalized identifier/metadata/quality/summary/zone tables under registry/, reading packaged identifiers.db and geography packs; replaces database.py, geo.py, and the top-level summaries module.

Signed-off-by: Travis Sikes <124101151+travis-recurve@users.noreply.github.com>
build/ downloads the current NOAA station list and inventory and writes the registry via migrate/refresh; station country comes from the source ISO_CODE, staleness is per source, and the in-place swap is crash-safe via a commit marker. Removes the ad-hoc download scripts.

Signed-off-by: Travis Sikes <124101151+travis-recurve@users.noreply.github.com>
WeatherStation plus an engine routing GHCNh/TMY3/CZ2010 through one interface with runtime source and vocabulary registration, catalog-less estimation (grid) sources, and per-variable resampling; register() validates the per-kind contract and the engine NaN-fills missing feed columns. Replaces stations.py, ranking.py, the single-file ghcnh source, and the CLI.

Signed-off-by: Travis Sikes <124101151+travis-recurve@users.noreply.github.com>
WeatherLocation resolves weather at a lat/lon through the configured sources (rejecting normals sources from the preference tuple), pins its station from provenance for reproducible replay, and serializes via to_dict/from_dict; the package exposes only WeatherLocation and WeatherStation.

Signed-off-by: Travis Sikes <124101151+travis-recurve@users.noreply.github.com>
Adds docs/ (api, example, methodology, matching, adding-a-source incl. the grid-source contract, vocabulary); rewrites README/CONTRIBUTING/CHANGELOG and refreshes packaging and CI.

Signed-off-by: Travis Sikes <124101151+travis-recurve@users.noreply.github.com>
@travis-recurve
travis-recurve force-pushed the feature/architecture-reshape branch from e7f5716 to 61b5649 Compare July 25, 2026 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant