A maze navigation simulator for reinforcement learning research, compatible with the Gymnasium API.
Gymnasium HardMaze is a reimplementation of the 'hardmaze' environment used in several neuroevolution research papers. It provides a platform for training and evaluating reinforcement learning agents in navigation tasks of varying complexity.
Key features:
- XML-based maze configuration
- Fully compatible with Gymnasium API
- Deterministic when seeded for reproducible research
- Visualizations for debugging and demonstrations
# Basic installation
pip install gymnasium-hardmaze
# For development
git clone https://github.com/Teaspoon-AI/gymnasium-hardmaze.git
cd gymnasium-hardmaze
pip install -e .import gymnasium as gym
import gymnasium_hardmaze
import numpy as np
# Create default maze environment
env = gym.make("HardMaze-v0", render_mode="human")
# Reset the environment
observation, info = env.reset(seed=42)
for _ in range(1000):
# Take a random action
action = env.action_space.sample()
# Step the environment
observation, reward, terminated, truncated, info = env.step(action)
# Check if episode is done
if terminated or truncated:
observation, info = env.reset()
env.close()The package includes a keyboard control script for testing environments:
# Run keyboard agent in the hard maze environment
python -m gymnasium_hardmaze.examples.keyboard_agent| Environment ID | Description |
|---|---|
HardMaze-v0 |
Complex maze with walls and multiple points of interest |
The observation space is a vector containing:
- Normalized rangefinder readings (distance to walls)
- Radar readings (binary detection of goal)
The action space is a 3-dimensional continuous space:
[left_motor, forward, right_motor]with values between 0 and 1
The default reward function prioritizes reaching the goal, with partial rewards for visiting points of interest along the way.
Several other projects provide maze navigation environments for robotics and AI research:
Kheperax is a JAX-based reimplementation of the fastsim simulator that simulates Khepera-like robots in 2D mazes. Key differences from gymnasium-hardmaze:
- Hardware acceleration: Fully implemented in JAX for GPU/TPU acceleration and massive parallelization
- Robot model: Simulates circular robots with 2 wheels, configurable laser and bumper sensors
- Optimization focus: Directly compatible with QDax library for Quality-Diversity optimization
- Performance: Designed for high-throughput evolutionary algorithms and population-based methods
A Gym wrapper for the pyfastsim simulator, also implementing Lehman & Stanley's hard maze. Key differences:
- Simulation backend: Uses the pyfastsim C++ simulator for physics simulation
- Robot configuration: Features 3 lasers at specific angles (-π/4, 0, π/4) and two bumpers
- Map format: Requires binary PBM format maps with specific size constraints
- API compatibility: Uses the older Gym API rather than the newer Gymnasium standard
- Modern API: Built specifically for the Gymnasium API with full compatibility
- Pure Python: No external simulator dependencies, making installation and debugging easier
- Research focus: Carefully ported from ES-HyperNEAT codebase for accurate replication
If you use this software in your research, please cite:
@software{gymnasium-hardmaze,
author = {Stefano Palmieri},
title = {HardMaze: A Gymnasium-compatible Implementation of hardmaze environment},
url = {https://github.com/Teaspoon-AI/gymnasium-hardmaze},
year = {2025},
}The code in this project is derived from the original source code used in Sebastian Risi's 2011 ES-HyperNEAT paper.
@InProceedings{risi:gecco2011,
author = "Sebastian Risi and Kenneth O. Stanley",
title = "Enhancing ES-HyperNEAT to Evolve More Complex Regular Neural Networks",
booktitle = "Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010)",
year = 2011,
publisher = "ACM",
url = "http://eplex.cs.ucf.edu/papers/risi_gecco11.pdf"
}