DA-GAT-BiLSTM algorithm for GNSS Time Series Prediction

Research Overview

This research implements a DA-GAT-BiLSTM (Direction-Aware Graph Attention Network with Bidirectional LSTM) architecture for millimeter-level precision prediction of GNSS displacement time series. The research encompasses six main stages: data preprocessing, feature engineering, model training, loss function comparison, ablation studies, and model comparison.

How to cite the work

Zhu, S., Kermarrec, G., He, X., Wang, S., Fernandes, R., Huang, Z., Montillet, J.P., Zhou, Y., 2026. A graph attention framework with kinematic constraints for network-based GNSS time series prediction. Preprint. https://doi.org/10.21203/rs.3.rs-9008300/v1

---

Overall Experimental Pipeline

Raw GNSS Data
    ↓
1. Data Preprocessing (1gnss_data_preprocessing.py)
    ↓
Preprocessed Datasets
    ↓
2. Feature Engineering (2gnss_feature_engineering.py)
    ↓
Featured Datasets
    ↓
3. Model Training (3gnss_model_training.py)
    ↓
Trained Model
    ↓
4-6. Comparative Experiments (Loss Comparison/Ablation/Model Comparison)
    ↓
Experimental Results and Analysis

---

Code Module Functions

1. Data Preprocessing (1gnss_data_preprocessing.py)

Core Function: Clean and normalize raw GNSS time series data

Main Workflow:

• Data Loading: Read multi-station GNSS data, standardize column names, filter data before 2000
• Temporal Splitting: Split data by fixed years - training (2000-2015), validation (2015-2020), test (2020-2024)
• Jump Correction: Detect and correct displacement jumps and offsets in training set
• Outlier Handling: Identify and process outliers using IQR method, apply training set statistics uniformly to all datasets
• Missing Value Interpolation: Fill missing data using linear interpolation (limited to internal interpolation ≤10 points)
• Quality Assessment: Generate detailed preprocessing statistics report

Output: train/val/test/full datasets for each station

---

2. Feature Engineering (2gnss_feature_engineering.py)

Core Function: Extract spatiotemporal features from preprocessed data and perform feature selection

Main Workflow:

• Temporal Feature Construction:

  • Rolling window statistics (mean, std, quantiles)
  • Difference features (first-order, second-order)
  • Trend features (linear trend, velocity, acceleration)
  • Periodic features (sin/cos encoding for annual cycles)

• Spatial Feature Construction:

  • KMeans-based spatial clustering features
  • Geographic coordinate features (latitude, longitude, elevation)
  • Spatial autocorrelation analysis (Moran's I index)

• Feature Selection:

  • Missing rate filtering
  • Low variance filtering
  • High correlation filtering
  • Random Forest-based importance evaluation
  • Cross-directional importance analysis (East/North/Up)
  • Stability screening (consistency across multiple training runs)

Output: Featured datasets and selected feature list

---

3. Model Training (3gnss_model_training.py)

Core Function: Train and optimize DA-GAT-BiLSTM architecture

Model Architecture:

• Direction-Aware GAT Layers:

  • Multi-head graph attention mechanism for spatial dependencies
  • Direction-aware weights (considering relative position vectors)
  • Cross-directional gating mechanism (East/North/Up information fusion)

• BiLSTM Layers:

  • Bidirectional LSTM for temporal sequence modeling
  • Multi-head temporal attention mechanism
  • Cross-directional gating enhancement

• Output Heads: Independent prediction for East/North/Up directions

Training Strategy:

• Loss Function: Physics-constrained DirectionLoss (includes relative loss, correlation loss, continuity loss)
• Optimizer: AdamW optimizer
• Learning Rate Scheduling: OneCycleLR strategy
• Hyperparameter Optimization: Bayesian optimization with Optuna
• Regularization: Dropout, LayerNorm, early stopping

Output: Optimal model weights, training logs, performance metrics

---

4. Loss Function Comparison (4Loss_function_comparison.py)

Core Function: Compare prediction performance between DirectionLoss and standard MSE loss

Experimental Design:

• Comparison Groups:

  • DirectionLoss (physics-constrained loss)
  • MSE Loss (standard mean squared error loss)

• Training Configuration: Same model architecture, datasets, and hyperparameters

Evaluation Metrics:

• MAE (Mean Absolute Error)
• RMSE (Root Mean Square Error)
• R² Score
• Direction-specific metrics (East/North/Up)

Output: Performance comparison report for both loss functions

---

5. Ablation Study (5Ablation_study.py)

Core Function: Analyze contribution of each DA-GAT-BiLSTM component

Ablation Variants:

• Full Model (DA-GAT-BiLSTM): Baseline model
• No GAT (BiLSTM-only): Remove graph attention layers
• No BiLSTM (GAT-only): Remove bidirectional LSTM layers
• No Direction-Aware (GAT-BiLSTM): Remove direction-aware mechanism
• No Cross-Direction (DA-GAT-BiLSTM w/o gating): Remove cross-directional gating
• No Attention (DA-GAT-LSTM): Remove temporal attention mechanism

Analysis Content:

• Prediction performance of each variant
• Impact degree of each component on overall performance
• Component importance differences across directions (East/North/Up)

Output: Ablation experiment comparison table and performance degradation analysis

---

6. Model Comparison (6Model_comparison.py)

Core Function: Compare DA-GAT-BiLSTM performance with baseline models

Comparison Models:

• VARIMA: Vector Autoregressive Integrated Moving Average model (captures cross-station spatial dependencies)
• GCN-LSTM: Graph Convolutional Network-LSTM architecture (graph-based spatial modeling with temporal sequences)
• GAT-Transformer: Graph Attention-Transformer architecture (joint spatiotemporal modeling)
• DA-GAT-BiLSTM: Complete proposed model

Evaluation Dimensions:

• Prediction accuracy (MAE/RMSE/R²)
• Direction-specific performance (East/North/Up)
• Computational efficiency
• Parameter scale

Output: Comprehensive model comparison report and performance visualization

---

Technical Approach Summary

1. Data Flow: Raw data → Cleaning & splitting → Feature extraction → Model input
2. Core Innovations:
   • Direction-Aware mechanism: Consider physical correlations among East/North/Up directions
   • Cross-directional gating: Enable explicit interaction among three directions
   • Joint spatial-temporal modeling: GAT for spatial, BiLSTM for temporal
   • Physics-constrained loss: Incorporate physical constraints to improve prediction rationality
3. Experimental Validation: Comprehensive validation through loss comparison, ablation studies, and model comparison

---

Data Organization Structure

Research Directory/
├── raw_data.csv
├── preprocessed_data/         # Preprocessing output
│   └── [station]/
│       ├── train.csv
│       ├── val.csv
│       ├── test.csv
│       └── full.csv
├── features/                  # Feature engineering output
│   ├── [station]/
│   │   ├── train_featured.csv
│   │   ├── val_featured.csv
│   │   └── test_featured.csv
│   └── selected_features.txt
└── model_output/              # Model training output
    ├── best_model.pth
    ├── training_log.txt
    └── metrics.json

---

Key Parameters

• Time Window: window_size=30 (historical window)
• Prediction Horizon: pred_horizon=7 (predict 7 steps ahead)
• Spatial Graph Construction: k_neighbors=10, distance_threshold=1000km
• Temporal Split Years: train_end=2015, val_end=2020, test_end=2024

---

Expected Results

• Prediction Accuracy: MAE approximately 3-6mm (three directions)
• Model Advantages: 10-30% improvement over baseline models
• Physical Rationality: Cross-directional correlation and continuity satisfy physical constraints