A global-scale air quality forecasting benchmark for spatio-temporal graph neural networks, featuring 3,720 monitoring stations across the world with authentic missing patterns and physical-scale evaluation.
AirQualityBench provides a realistic testbed for air quality prediction by preserving raw observational characteristics that prior benchmarks have removed through imputation: spatially heterogeneous station coverage, pollutant-dependent missingness rates, and irregular temporal gaps. The benchmark spans 5 years (2021–2025) of hourly measurements across 6 primary pollutants, with a standardized evaluation protocol covering 12 state-of-the-art STGNN models at 4 prediction horizons (6h, 12h, 18h, 24h).
- Global scale: 3,720 active stations selected from ~19,500 OpenAQ stations, spanning 7 continents
- Authentic missingness: No synthetic imputation; evaluation uses boolean masks to only score valid observations
- Physical-scale metrics: All metrics (MAE, MSE, RMSE, MAPE) computed in original concentration units after inverse normalization
- Multi-pollutant: 6 pollutants — PM2.5, PM10, NO2, O3, SO2, CO
- Comprehensive model zoo: 12 models covering GNN-based, RNN-based, Transformer-based, and hybrid architectures
- 2026.05 — The AirQualityBench dataset is now available on Hugging Face: huggingface.co/datasets/xuxing123/AirQualityBench
- 2026.04 — Initial release of AirQualityBench on GitHub.
Raw hourly data (2017–2025, .csv): Daily tar.gz archives containing hourly CSV readings per station from OpenAQ. Download: openaq_tar_gz (extraction code: yqda)
Processed dataset (2021–2025, .h5): aq_data.zip (extraction code: m457)
The dataset is derived from the OpenAQ platform.
After downloading and extracting, place the files under ./data/:
data/
├── aq_compact_2021.h5
├── aq_compact_2022.h5
├── aq_compact_2023.h5
├── aq_compact_2024.h5
├── aq_compact_2025.h5
├── adj_mx_10.pkl
├── scaler.csv
├── selected_nodes_metadata.csv
After station filtering, 3,720 stations were retained. Raw hourly values are stored in yearly HDF5 files with associated boolean masks encoding valid observations.
| Pollutant | Mean | Std |
|---|---|---|
| PM2.5 | 11.66 | 20.87 |
| PM10 | 22.08 | 26.47 |
| NO2 | 13.26 | 16.06 |
| O3 | 22.48 | 31.81 |
| SO2 | 4.08 | 8.72 |
| CO | 228.46 | 416.08 |
| Split | Years | Approx. Hours |
|---|---|---|
| Train | 2021–2023 | ~26,280 |
| Validation | 2024 | ~8,784 |
| Test | 2025 | ~8,760 |
Each yearly HDF5 file (aq_compact_{year}.h5) contains:
data: float32 array of shape(T, 3720, 6)— hourly pollutant concentrationsmask: bool array of shape(T, 3720, 6)— valid observation flagscoords: station metadata array
The spatial adjacency matrix adj_mx_10.pkl is constructed via KNN (k=10) with Haversine distance and Gaussian kernel weighting.
All models follow a unified interface: input (B, T_in, N, C) → output (B, T_out, N, C) where N = 3,720 nodes, C = 6 pollutants. The model zoo includes STGCN, DCRNN, Graph WaveNet, AGCRN, ASTGCN, STTN, PDFormer, DSTAGNN, D2STGNN, MAGE, BiST, and IGSTGNN.
pip install torch numpy pandas h5py tqdm# Train STGCN with 24h input, 12h prediction
python main.py --model stgcn --w_in 24 --w_out 12 --step 12 --batch_size 32 --epochs 50 --gpu 0| Argument | Default | Description |
|---|---|---|
--model |
stgcn |
Model: stgcn, dcrnn, gwn, agcrn, pdformer, astgcn, sttn, dstagnn, d2stgnn, mage, bist, igstgnn |
--data_dir |
./data |
Path to HDF5 files |
--scaler_path |
./data/scaler.csv |
Path to normalization scaler |
--adj_path |
./data/adj_mx_10.pkl |
Path to adjacency matrix |
--output_dir |
./outputs |
Experiment output directory |
--w_in |
24 |
Input window size (hours) |
--w_out |
24 |
Output window size (hours) |
--step |
1 |
Sliding window step |
--batch_size |
64 |
Batch size |
--lr |
0.001 |
Learning rate |
--epochs |
100 |
Number of epochs |
--gpu |
0 |
GPU device ID |
--weight_decay |
1e-5 |
Weight decay |
--resume_dir |
"" |
Checkpoint resume directory |
Ready-to-use shell scripts for all model × horizon combinations are in runner/:
bash runner/run_stgcn_24_12.sh
bash runner/run_gwn_24_24.sh
bash runner/run_pdformer_24_6.sh
# ... 48 scripts totalAll metrics are computed on physical-scale (de-normalized) values, with masks applied to ignore missing observations:
- MAE:
Σ|pred - label| · mask / Σ mask - MSE:
Σ(pred - label)² · mask / Σ mask - RMSE:
√MSE - MAPE:
Σ(|pred - label| / (|label| + ε)) · mask / Σ mask
Metrics are reported both globally (averaged over all stations and pollutants) and per-pollutant.
- Optimizer: Adam with ReduceLROnPlateau scheduler (patience=5, factor=0.5)
- Training objective: Masked MAE on normalized values
- Checkpointing: Best model selected by validation Global_MAE
- Resume: Auto-detects existing checkpoints via
--resume_dir - Logging: Per-epoch CSV with all global and per-pollutant metrics
MIT