AVS-GazeTime: MEG + Eye-tracking Analysis Code

Fixation duration on natural scenes is explained by memory encoding not processing demand (Sulewski et al., in press).

Analysis code for investigating fixation duration control during natural scene viewing using MEG and eye-tracking data.

Overview

This repository contains the Python analysis pipeline for our study examining how fixation durations vary during natural scene exploration. The code tests two competing hypotheses about fixation control: processing-demand vs memory-facilitation.

Data

The dataset is available on Göttingen Research Online:

Sulewski, Philip; Carmen Amme; Martin N. Hebart; Peter König; Tim C. Kietzmann, 2026, "Data: Fixation duration on natural scenes is explained by memory encoding not processing demand", https://doi.org/10.25625/DDJ5C3

To reduce the number of files for the repository, CSV and npy files are bundled into per-directory .tar.gz archives. After downloading, extract them in place with the provided utility:

python dataverse_tools/unarchive_data.py --data-root /path/to/downloaded/data

This extracts all *_csvs.tar.gz and *_npys.tar.gz archives into their respective directories and removes the archive files, restoring the original data layout expected by the analysis scripts.

Repository Structure

avs_gazetime/
├── config.py                          # Configuration and paths
├── utils/
│   ├── load_data.py                   # Core data loading functions
│   ├── sensors_mapping.py             # MEG sensor definitions
│   └── tools.py                       # Utility functions
├── ease_of_recognition/               # Recognition difficulty analysis
│   ├── avs_ease_of_recognition.py     # ResNet50 classification entropy
│   ├── plot_entropy_examples.py       # Visualisation scripts
│   ├── get_netwok_activations.py      # Neural network feature extraction
│   └── run_network_extraction.sh      # SLURM batch script
├── memorability/                      # ResMem memorability analysis
│   ├── memorability_analysis.py       # Memorability vs duration analysis
│   ├── fixation_sequence_analysis.py  # Sequence position effects
│   ├── compute_memorability_scores.py # ResMem score computation
│   ├── mem_tools.py                   # Memorability utilities
│   ├── memorability_analysis_params.py # Analysis parameters
│   ├── plot_memorability_examples.py  # Visualisation of memorability examples
│   └── run_compute_memorability_scores.sh # SLURM script
├── fix2cap/                           # Fixation-to-caption analysis
│   ├── fixation2caption_matcher.py    # PyQt5 annotation GUI
│   ├── fix2cap_quality_controls.py    # Quality control and statistics
│   ├── fix2cap_prepare_data_package.py # Data preparation
│   ├── setup.py                       # Package setup
│   └── README.txt                     # Data access instructions
├── pac/                               # Phase-amplitude coupling
│   ├── cross_frequency_pac.py         # Theta-gamma PAC computation
│   ├── full_pac_map.py                # Full frequency PAC mapping with harmonic masking
│   ├── pac_plotting_and_stats.ipynb   # Statistical analysis notebook
│   ├── pac_dataloader.py              # Data loading with 4-fold residual removal
│   ├── pac_map_dataloader.py          # Full map data loading
│   ├── pac_functions.py               # PAC computation utilities
│   ├── pac_analysis.py                # PAC analysis tools
│   ├── params_pac.py                  # PAC parameters
│   ├── aggregate_pac_chunks.py        # Combine PAC results across vertices
│   ├── illustrate_offset_locking.py   # Visualization of offset-locking
│   └── run_pac_cross_freq.sh          # SLURM batch processing
├── decoding/                          # Multivariate pattern analysis
│   ├── memorability_decoder.py        # ResMem score decoding from MEG
│   ├── entropy_decoder.py             # Classification entropy decoding
│   ├── duration_decoder.py            # Duration prediction (not used)
│   ├── plot_memorability_predictions.py # Validate MEG predictions vs duration
│   ├── plot_entropy_predictions.py    # Validate entropy predictions vs duration
│   ├── plot_memorability_decoding.py  # Decoding performance plots
│   ├── plot_entropy_decoding.py       # Entropy decoding plots
│   ├── decoding_params.py             # Analysis parameters
│   ├── run_memorability_decoder.sh    # SLURM script for memorability
│   └── run_entropy_decoder.sh         # SLURM script for entropy
├── dynamics/                          # Neural dynamics analysis
│   ├── dynamics_analysis.py           # Main dynamics analysis
│   ├── dynamics_dataloader.py         # Data loading for dynamics
│   ├── dynamics_functions.py          # Dynamics computation functions
│   ├── dynamics_plotting.py           # Plotting functions
│   ├── dynamics_plotting_tools.py     # Plotting utilities
│   ├── dynamics_stats.py              # Statistical analysis
│   ├── dynamics_params.py             # Analysis parameters
│   ├── dynamics_plot_params.py        # Plotting parameters
│   └── run_dynamics_over_subjects.sh  # SLURM batch script
├── scenes/                            # Scene-level analysis
│   └── semantic_clusters/             # Semantic clustering
└── duration_heatmaps/                 # Visualisation tools
    └── plot_heatmaps.py               # MEG heatmap plotting

Core Analysis Modules

Configuration (config.py)

Central configuration managing:

• Subject IDs and session parameters
• MEG data paths (sensor vs source space)
• Eye-tracking data directories
• Analysis output directories

Data Loading (utils/load_data.py)

Core functions for loading and preprocessing:

# Load MEG data for specific ROI and event type
meg_data = process_meg_data_for_roi(roi="mag", event_type="fixation")

# Load corresponding metadata
metadata = merge_meta_df(event_type="fixation", sessions=SESSIONS)

# Load session-specific data with session-wise median scaling
session_data = load_meg_session_data(session="a", roi="mag", event_type="fixation")

# Match saccades to adjacent fixations for event linking
matched_meta = match_saccades_to_fixations(metadata)

Key Features:

• Session-wise median scaling via mne.decoding.Scaler
• Parallel loading with joblib for efficiency
• Automatic scaling to femtoTesla for sensor data (grad: 1e13, mag: 1e15)
• Optional channel indexing for memory-efficient ROI processing

Recognition Difficulty Analysis (ease_of_recognition/)

Tests processing-demand hypothesis using ResNet50 classification entropy:

• avs_ease_of_recognition.py: Main analysis computing entropy scores for fixated patches
• get_netwok_activations.py: Extracts neural network features from image crops
• run_network_extraction.sh: SLURM job for parallel processing across subjects

Memorability Analysis (memorability/)

Tests memory-facilitation hypothesis using ResMem predictions:

• memorability_analysis.py: Correlates memorability scores with fixation durations
• compute_memorability_scores.py: Generates ResMem predictions for fixated patches
• Implements both absolute and relative (within-scene) memorability measures

Fixation-to-Caption Matching (fix2cap/)

Analyses semantic relevance of fixated content:

• fix2cap_quality_controls.py: Statistical analysis of caption inclusion
• fix2cap_prepare_data_package.py: Prepares data for annotation interface
• Uses human ratings to classify fixations as mentioned/not mentioned in captions

Phase-Amplitude Coupling (pac/)

Examines theta-gamma coupling during longer fixations:

• cross_frequency_pac.py: Computes PAC using single-cut surrogate method with duration split analysis
• full_pac_map.py: Generates comprehensive frequency × frequency PAC maps with harmonic contamination masking
• pac_plotting_and_stats.ipynb: Statistical analysis with mixed-effects models for duration differences across ROIs
• Focuses on 3-7 Hz (theta) × 40-80 Hz (gamma) coupling
• Uses offset-locked epochs (time-locked to fixation end) for both long and short fixation categories
• Implements 4-fold residual removal: subtracts median ERFs from current saccade, current fixation, subsequent saccade, and subsequent fixation
• Harmonic masking removes PAC values where gamma_freq ≤ 2×theta_freq to prevent spurious coupling
• Surrogate testing options: single-cut (default), phase-shuffle, and session-aware methods

Multivariate Decoding (decoding/)

Tests information content in MEG patterns:

• memorability_decoder.py: Decodes ResMem scores from MEG sensor patterns using Ridge regression
• plot_memorability_predictions.py: Validates MEG predictions with mixed-effects models testing predicted memorability vs duration
• plot_entropy_predictions.py: Validates MEG-predicted classification entropy against fixation duration
• decoding_params.py: Ridge regression and cross-validation parameters
• Uses mne.decoding.SlidingEstimator to train separate decoders at each MEG timepoint
• Cross-validation with GroupKFold on scene IDs to prevent data leakage
• Predictions are z-scored per subject before mixed-effects analysis to match behavioral preprocessing

Key Dependencies

# Core scientific computing
import numpy as np
import pandas as pd
import scipy

# MEG analysis
import mne
from mne.decoding import SlidingEstimator

# Machine learning
from sklearn.linear_model import RidgeCV
from sklearn.preprocessing import RobustScaler
from sklearn.model_selection import cross_val_score

# Deep learning (for memorability)
import torch
import torchvision

# Statistical analysis
import statsmodels.formula.api as smf

Usage Examples

Basic Data Loading

from avs_gazetime.config import configure_run
from avs_gazetime.utils.load_data import process_meg_data_for_roi, merge_meta_df

# Configure for subject 1, magnetometer data
config = configure_run(subject_id=1, ch_type="mag")

# Load MEG data and metadata
meg_data = process_meg_data_for_roi("mag", "fixation")
metadata = merge_meta_df("fixation")

Running SLURM Jobs

All major analyses use SLURM array jobs for parallel processing:

# Compute ResMem memorability scores (subjects 1-5)
sbatch avs_gazetime/memorability/run_compute_memorability_scores.sh

# Extract neural network activations for classification entropy
sbatch avs_gazetime/ease_of_recognition/run_network_extraction.sh 100  # crop size

# Run theta-gamma PAC analysis
sbatch avs_gazetime/pac/run_pac_cross_freq.sh

# Decode memorability from MEG patterns
sbatch avs_gazetime/decoding/run_memorability_decoder.sh

SLURM Environment Variables:

• SUBJECT_ID_GAZETIME: Subject ID (1-5) set automatically by array jobs
• CH_TYPE_GAZETIME: Channel type ("mag", "grad", "stc")

Local Analysis:

# Run memorability analysis after computing scores
python avs_gazetime/memorability/memorability_analysis.py

# Quality control for fixation-caption matching
python avs_gazetime/fix2cap/fix2cap_quality_controls.py

Contact

For questions about the code:

• Philip Sulewski: psulewski{att}uos.de