GELLO: General, Low-Cost, and Intuitive Teleoperation Framework

GELLO is a general, low-cost, and intuitive teleoperation framework for robot manipulators. This repository contains all the software components for GELLO.

For additional resources:

• Project Website
• Hardware Repository - STL files and build instructions
• ROS 2 Support

Supported Robots

• I2RT YAM
• Franka FR3 (ROS 2 implementation, please refer to the separate documenation in ros2/README.md)
• Franka FER (Panda)
• UR
• xArm
• UFFactory Lite6 (with Zhonglin servo motors)
• add your own, see Adding New Robots

Supported Motors

• Dynamixel (XL330, XC330, XM430, etc.) -- original GELLO design
• Zhonglin serial bus servos -- alternative low-cost option

Quick Start

git clone https://github.com/wuphilipp/gello_software.git
cd gello_software

Installation

Option 1: Virtual Environment (Recommended)

First, install uv if you don't have it:

curl -LsSf https://astral.sh/uv/install.sh | sh

Create and activate a virtual environment:

uv venv --python 3.11
source .venv/bin/activate  # Run this every time you open a new shell
git submodule init
git submodule update
uv pip install -r requirements.txt
uv pip install -e .
uv pip install -e third_party/DynamixelSDK/python  # Only needed for Dynamixel motors

Option 2: Docker

Install Docker, then:

docker build . -t gello:latest
python scripts/launch.py

ROS 2 Support

Note: GELLO also supports ROS 2 Humble for the Franka FR3 robot. See the ROS 2-specific README in the ros2 directory.

Hardware Configuration

The recommended setup for GELLO is with the I2RT YAM robot arm, using the YAML-based configuration system. This provides the most features and is the best-supported configuration.

Generate YAML Configuration

For the I2RT YAM robot, you can automatically generate your configuration files. This process calibrates the joint offsets and creates configuration files for both simulation and real hardware.

1. Update Motor IDs: Before generating the config, ensure each Dynamixel motor has a unique ID. Install the Dynamixel Wizard and follow these steps:

   1. Connect a single motor to the U2D2 controller.
   2. Open Dynamixel Wizard and scan to detect the motor.
   3. Change the ID to a unique number (e.g., 1 through 7).
   4. Repeat for each motor, ensuring they are in order from base to gripper.

2. Run the Generation Script: With the YAM arm in its default build position (see image below), run the script:

   python scripts/generate_yam_config.py

   Follow the prompts in the terminal. This will create configs/yam_auto_generated.yaml for the real robot and configs/yam_auto_generated_sim.yaml for the simulation.

You can now skip to the Usage section.

YAML Configuration System

GELLO uses YAML files in configs/ for configuration. This allows for flexible setup of different robots, environments, and teleoperation parameters. If you have automatically generated your .yaml config files with scripts/generate_yam_config.py, you probably will not need to modify these confings manually.

Sample Configs

Sample configs for the YAM arm and the xarm can be found in configs.

Configuration Components

• Robot Config: Defines robot type, communication parameters, and physical settings.
• Agent Config: Defines GELLO device settings, joint mappings, and calibration.
• DynamixelRobotConfig: Motor-specific settings including IDs, offsets, signs, and gripper.
• Control Parameters: Update rates (hz), step limits (max_steps), and safety settings.

Manual Configuration for Other Robots

Python Configuration for Non-YAM arms

• Most widely supported across different arms
• Located in gello/agents/gello_agent.py
• Uses PORT_CONFIG_MAP dictionary
• Maps USB serial ports to robot configurations

Lite6 + Zhonglin Motor Setup

If you are using a GELLO with Zhonglin serial bus servos (instead of Dynamixel) and a UFFactory Lite6 robot, follow these steps. For a complete step-by-step guide with all test commands, see LITE6_GELLO_GUIDE.md.

Zhonglin Motor Driver

The Zhonglin driver (gello/zhonglin/driver.py) communicates over ASCII serial at 115200 baud. It implements the same DynamixelDriverProtocol interface so it works as a drop-in replacement throughout the codebase. No Dynamixel SDK is needed for Zhonglin motors.

Verify Servo Communication

Plug in the GELLO USB cable and run:

python -m gello.zhonglin.driver

Move joints by hand and confirm angles update. Ctrl+C to stop.

Calibrate Joint Offsets

Place the Lite6 at its home position (all zeros) and manually align the GELLO to match:

python scripts/zhonglin_get_offset.py \
    --port /dev/ttyUSB0 \
    --start-joints 0 0 0 0 0 0 \
    --joint-signs 1 1 1 1 1 1

If a joint moves in the wrong direction, flip its sign to -1 and re-run. Save the printed joint_offsets, joint_signs, and gripper values.

Simulation Test (MuJoCo)

Test your GELLO against a simulated Lite6 with no real robot hardware:

# Without GELLO hardware (verify MuJoCo loads)
python experiments/quick_run.py --robot sim_lite6 --agent dummy

# With GELLO hardware
python experiments/quick_run.py --robot sim_lite6 --agent gello --gello-port /dev/ttyUSB0

Or use the YAML config (edit configs/lite6_sim_test.yaml with your calibrated values first):

python experiments/launch_yaml.py --left-config-path configs/lite6_sim_test.yaml

LeRobot Integration

This GELLO setup integrates with LeRobot (UFFactory fork) for teleoperation and data collection. The GelloLite6 teleoperator in LeRobot uses ZhonglinDriver and GelloAgent from this repository. See LITE6_GELLO_GUIDE.md for detailed instructions on running teleop and recording episodes.

---

Adding New Robots

To integrate a new robot to the Python configs:

1. Check Compatibility: Ensure your GELLO kinematics match the target robot
2. Implement Robot Interface: Create a new class implementing the Robot protocol from gello/robots/robot.py
3. Add Configuration: Update the configuration system with your robot's parameters

See existing implementations in gello/robots/ for reference:

• panda.py - Franka Panda robot
• ur.py - Universal Robots
• xarm_robot.py - xArm robots
• yam.py - YAM robot
• zhonglin.py - Zhonglin servo-based GELLO (e.g. for Lite6)

=======

1. Manual gello_agent setup

Set your GELLO and robot arm to a known, matching configuration (see images below) and run the offset detection script.

Command examples:

UR Robot:

python scripts/gello_get_offset.py \
    --start-joints 0 -1.57 1.57 -1.57 -1.57 0 \
    --joint-signs 1 1 -1 1 1 1 \
    --port /dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FT7WBG6

Franka FER (Panda):

python scripts/gello_get_offset.py \
    --start-joints 0 0 0 -1.57 0 1.57 0 \
    --joint-signs 1 1 1 1 1 -1 1 \
    --port /dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FT7WBG6

I2RT YAM:

python scripts/gello_get_offset.py \
    --start-joints 0 0 0 0 0 0 \
    --joint-signs 1 -1 -1 -1 1 1 \
    --port /dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FTAAMLV6-if00-port0

UFFactory Lite6 (Zhonglin motors):

python scripts/zhonglin_get_offset.py \
    --port /dev/ttyUSB0 \
    --start-joints 0 0 0 0 0 0 \
    --joint-signs 1 1 1 1 1 1

Joint Signs Reference:

• UR: 1 1 -1 1 1 1
• Panda: 1 -1 1 1 1 -1 1
• xArm: 1 1 1 1 1 1 1
• YAM: 1 -1 -1 -1 1 1
• Lite6 (Zhonglin): determine via zhonglin_get_offset.py

Add the generated joint offsets to gello/agents/gello_agent.py in the PORT_CONFIG_MAP.

2. Create Custom YAML Configurations

1. Copy an existing config from configs/ as a template (e.g., yam_passive.yaml).
2. Modify the robot _target_ and parameters for your setup:
   • For hardware: gello.robots.ur.URRobot, gello.robots.panda.PandaRobot, etc.
   • For simulation: gello.robots.sim_robot.MujocoRobotServer
3. Update the agent configuration with your GELLO device settings:
   • port: Your U2D2 device path
   • joint_offsets: From the offset detection script
   • joint_signs: Based on your robot type
   • start_joints: Your GELLO's starting position

Usage

The recommended way to launch GELLO is with a YAML configuration file.

CAN Configuration

Robot arms such as the YAM use a CAN bus to communicate with your machine. If your arm uses a CAN bus, you will need to configure udev rules. First, get your CAN bus ID:

udevadm info -a -p /sys/class/net/can* | grep -i serial

Then open your CAN bus rules using your text editor of choice.

sudo nano /etc/udev/rules.d/90-can.rules

If you only have one arm, add this line:

SUBSYSTEM=="net", ACTION=="add", ATTRS{serial}=="<your-CAN-id>", NAME="can_left"

If you have two arms (a bimanual setup), you will need a second line for your right arm. Your bimanual CAN rules file should contain:

SUBSYSTEM=="net", ACTION=="add", ATTRS{serial}=="<left-CAN-id>", NAME="can_left"
SUBSYSTEM=="net", ACTION=="add", ATTRS{serial}=="<right-CAN-id>", NAME="can_right"

After updating your udev rules, run the following and then unplug and reconnect your CAN devices.

sudo udevadm control --reload-rules && sudo systemctl restart systemd-udevd && sudo udevadm trigger

At this point, your CAN devices are correctly configured. If you encounter CAN connctivity issues after this point run sh scripts/reset_all_can.sh to reset your CAN buses.

YAM GELLO Usage (Recommended)

First, install the YAM-specific dependency:

• YAM: I2RT
• uv pip install -e third_party/i2rt

Testing in Simulation: Launch the simulation with the auto-generated sim config file:

python experiments/launch_yaml.py --left-config-path configs/yam_auto_generated_sim.yaml

Real Robot Operation: Launch the real robot with the auto-generated hardware config file:

python experiments/launch_yaml.py --left-config-path configs/yam_auto_generated.yaml

Launching gello_agent for non-YAM arms

For other robots or if not using a YAML configuration, you must launch the robot and controller nodes in separate terminals.

First, install robot-specific dependencies:

• UR: ur_rtde
• Panda: polymetis
• xArm: xArm Python SDK

1. Launch the robot node:

# For simulation
python experiments/launch_nodes.py --robot <sim_ur|sim_panda|sim_xarm|sim_lite6>

# For real hardware
python experiments/launch_nodes.py --robot <ur|panda|xarm>

2. Launch GELLO controller:

python experiments/run_env.py --agent=gello

Troubleshooting

If, when you run generate_yam_config.py, you get an error detecting offsets, you may need to add your user to the dialout user group. To do so, run: sudo usermod -aG dialout $USER And then log out and log back in or restart your computer.s

If some joints in your arm are not behaving as expected, you may need to modify the joint signs of your configuration. Simply invert the affected joint sign(s) in your .yaml or gello_agent.py or physically reverse the installation of the servo.

Optional: Starting Configuration

Use --start-joints to specify GELLO's starting configuration for automatic robot reset:

python experiments/run_env.py --agent=gello --start-joints <joint_angles>

Advanced Features

Data Collection

Collect teleoperation demonstrations with keyboard controls.

For the YAM arm launched with launch_yaml.py, you can append the flag --use-save-interface to enable data saving. This is the recommended method.

python experiments/launch_yaml.py --left-config-path configs/yam_passive.yaml --use-save-interface

After launching, you can begin saving with s and stop saving with q. Data saved will be in the data directory in the root of the project.

For non-YAM setups, use the following:

python experiments/run_env.py --agent=gello --use-save-interface

Process collected data:

python gello/data_utils/demo_to_gdict.py --source-dir=<source_dir>

Bimanual Operation

The recommended way to use bimanual mode is with launch_yaml.py. Pass a config file for the right arm to --right-config-path.

python experiments/launch_yaml.py --left-config-path configs/gello_1.yaml --right-config-path configs/gello_2.yaml

For non-YAM setups, use:

python experiments/launch_nodes.py --robot=bimanual_ur
python experiments/run_env.py --agent=gello --bimanual

FACTR Gravity Compensation

If you want to activate gravity compensation, all the code can be found in gello/factr. It works similarly to the regular launch but for now it's self-contained inside its own subdirectory and supports the YAM arm in sim and in hardware.

The YAML provides important fields that can control the strength of the gravity compensation and friction. Feel free to mess around with the strenght and friction til you attain your desired

One important step is to add the URDF. We have provided the URDF for the active GELLO in the Hardware Repository. You will need to update the path in the YAML to the entry point of the URDF.

python gello/factr/gravity_compensation.py --config configs/yam_gello_factr_hw.yaml

Development

Code Organization

├── scripts/             # Utility scripts (incl. zhonglin_get_offset.py)
├── experiments/         # Entry points and launch scripts
├── configs/             # YAML configs (incl. lite6_sim_test.yaml)
├── gello/               # Core GELLO package
│   ├── agents/          # Teleoperation agents (GelloAgent, configs)
│   ├── cameras/         # Camera interfaces
│   ├── data_utils/      # Data processing utilities
│   ├── dm_control_tasks/# MuJoCo environment utilities
│   ├── dynamixel/       # Dynamixel hardware interface
│   ├── zhonglin/        # Zhonglin serial bus servo interface
│   ├── factr/           # Gravity compensation
│   ├── robots/          # Robot-specific interfaces (incl. zhonglin.py)
│   ├── utils/           # Shared launch and control utilities
│   └── zmq_core/        # ZMQ multiprocessing utilities

Contributing

Install development dependencies and set up pre-commit hooks to ensure code quality before contributing:

uv pip install -r requirements_dev.txt
uv pip install pre-commit
pre-commit install

The codebase uses isort and black for code formatting.

We welcome contributions! Submit pull requests to help make teleoperation more accessible and higher quality.

Citation

@misc{wu2023gello,
    title={GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators},
    author={Philipp Wu and Yide Shentu and Zhongke Yi and Xingyu Lin and Pieter Abbeel},
    year={2023},
}

License & Acknowledgements

This project is licensed under the MIT License (see LICENSE file).

Third-Party Dependencies

• google-deepmind/mujoco_menagerie: Robot models for MuJoCo
• brentyi/tyro: Argument parsing and configuration
• ZMQ: Multiprocessing communication framework

This project uses components from ‘FACTR Teleop: Low-Cost Force-Feedback Teleoperation’ (Apache‑2.0). See https://github.com/RaindragonD/factr_teleop/.