RFO-RNN: Residual Fractional-Order Recurrent Neural Network for Long-Term Time Series Forecasting

This repository contains the implementation of the RFO-RNN model, a novel neural network architecture that leverages Caputo fractional-order derivatives to enhance time series prediction capabilities.

1. Key Features 💡

• Fractional-Order Memory: Implements Caputo fractional-order derivatives to capture long-term dependencies in time series data
• Multiple Model Architectures: Comprehensive comparison with state-of-the-art models including LSTM, GRU, Transformer, TransformerXL, and Informer
• Automated Hyperparameter Tuning: Grid search functionality for optimal model configuration
• Multi-Dataset Support: Evaluation on various time series datasets (ECL, ELD, ETTh, WTH)
• Flexible Prediction Horizons: Support for different forecasting lengths (24, 48, 96, 192)

2. Project Structure 📁

RFO-RNN-Research-Repository/
├── data/                             # Time series datasets
│   ├── ECL.mat                       # Electricity Consuming Load data
│   ├── ELD.mat                       # Electricity Load Demand data
│   ├── ETTh.mat                      # Electricity Transformer Temperature (hourly)
│   ├── ETTh1.csv                     # ETT dataset in CSV format
│   ├── MG.mat                        # Mackey-Glass time series
│   ├── MG2.mat & MG3.mat            # Additional MG variants
│   └── WTH.mat                       # Weather data
├── model/                            # Model implementations
│   ├── __init__.py                   # Package initialization
│   ├── Caputo_FractionalRNN.py      # Main RFO-RNN implementation
│   ├── FractionalRNNFX1Model.py     # Fractional RNN variant 1
│   ├── FractionalRNNFX2Model.py     # Fractional RNN variant 2
│   ├── StandardRNN.py               # Standard RNN baseline
│   ├── LSTMModel.py                 # LSTM implementation
│   ├── GRUModel.py                  # GRU implementation
│   ├── TransformerModel.py          # Transformer implementation
│   ├── TransformerxlModel.py        # TransformerXL implementation
│   ├── InformerModel.py             # Informer implementation
│   ├── utils.py                     # Utility functions
│   ├── models/                      # Additional model components
│   │   ├── attn.py                  # Attention mechanisms
│   │   ├── decoder.py               # Decoder modules
│   │   └── ...
│   └── util/                        # Additional utilities
├── results_[DATASET]/               # Experiment results for each dataset
│   ├── best_models/                 # Saved best model checkpoints
│   ├── grid_search/                 # Grid search results
│   ├── loss_curves/                 # Training/validation loss plots
│   ├── metrics/                     # Performance metrics
│   ├── model_info/                  # Model configuration files
│   └── predictions/                 # Model predictions
├── train.py                         # Main training script
├── test.py                          # Model evaluation script
├── grid_search.py                   # Hyperparameter optimization
├── grid_search.sh                   # Grid search bash script
├── train_all.sh                     # Batch training script
├── test_all.sh                      # Batch testing script
├── auto_alpha.ipynb                 # Alpha parameter analysis notebook
├── auto_zero.ipynb                  # Zero-order analysis notebook
└── README.md                        # This file

3. Installation & Setup 🛠️

3.1 Prerequisites

pip install torch torchvision
pip install numpy scipy matplotlib
pip install scikit-learn pandas
pip install jupyter notebook

3.2 Quick Start

1. Clone the repository

   git clone https://github.com/yourusername/RFO-RNN-Research-Repository.git
   cd RFO-RNN-Research-Repository

2. Prepare data: Ensure your datasets are in the data/ directory

3. Train a model

   python train.sh

4. Run grid search (for hyperparameter optimization)

   python grid_search.sh

4. Model Comparison 🔍

The repository supports comparison with multiple baseline models:

• RFO-RNN (Proposed Method): Caputo_FractionalRNN
• Traditional RNNs: StandardRNN, LSTM, GRU
• Attention-based: Transformer, TransformerXL, Informer
• Fractional Variants: FractionalRNN1, FractionalRNN2

5. Experiments & Results 📊

5.1 Supported Datasets

Dataset	Description	Domain
ECL	Electricity Consuming Load	Energy
ELD	Electricity Load Demand	Energy
ETTh	Electricity Transformer Temperature	Energy
WTH	Weather Data	Meteorology
MG	Mackey-Glass Time Series	Mathematics

5.2 Prediction Horizons

• Short-term: 24 steps
• Medium-term: 48, 96 steps
• Long-term: 192 steps

5.3 Evaluation Metrics

• Mean Squared Error (MSE)
• Mean Absolute Error (MAE)
• Root Mean Squared Error (RMSE)

6. RFO-RNN Hyperparameters ⚙️

• alpha: Fractional order parameter (0 < α < 1)
• memory_length: Historical state buffer size
• hidden_sizes: Hidden layer dimensions
• input_history_length: Input sequence length
• output_history_length: Prediction horizon

7. Notebooks 📓

• auto_alpha.ipynb: Automated analysis of fractional order parameter α
• auto_zero.ipynb: Zero-order behavior analysis

8. Scripts Overview 📋

8.1 Training & Testing

• train.py: Single model training with customizable parameters
• test.py: Model evaluation and performance metrics
• train_all.sh: Batch training across multiple datasets/configurations
• test_all.sh: Comprehensive testing suite

8.2 Optimization

• grid_search.py: Hyperparameter optimization framework
• grid_search.sh: Automated grid search execution

9. Contributing 🤝

We welcome contributions! Please feel free to:

• Suggest new features
• Submit pull requests
• Improve documentation

10. Citation 📚

This project contains code for an unpublished journal paper. Citation information will be released after paper acceptance.

---

⭐ If you find this repository helpful, please give us a star!