Circuit Relay v2 Implementation and Testing in universal chatting application

[IronJam11]

Overview

This document describes the implementation and successful testing of Circuit Relay v2 for NAT traversal in the Python libp2p peer implementation while building for universal chatting application . Circuit Relay enables peers on isolated networks to communicate through an intermediary relay server, addressing common NAT and firewall traversal challenges in peer-to-peer networks.

Demo Video: https://drive.google.com/file/d/1z7AIROEiLH_taQL0v2PgQrdMrrHTWmsa/view?usp=sharing

Component Configuration

Relay Server

• Connected to both networkA (172.24.0.2) and networkB (172.25.0.2)
• Runs in relay server mode with --relay-server flag
• Subscribes to topic room1 (isolated from test peers)
• Accepts and manages relay reservations from clients
• Provides HOP service for circuit relay connections

PeerA

• Network: networkA only (172.24.0.2)
• Topic: pldg
• Mode: Relay client requesting reservation
• Command: --relay /ip4/172.24.0.2/tcp/4001/p2p/<RELAY_ID>
• Obtains relay circuit address after successful reservation

PeerB

• Network: networkB only (172.25.0.2)
• Topic: pldg
• Mode: Circuit client connecting through relay
• Command: -c /ip4/172.25.0.2/tcp/4001/p2p/<RELAY_ID>/p2p-circuit/p2p/<PEER_A_ID>
• No direct TCP connection path to PeerA

Circuit Relay Address Format

Circuit relay addresses use a standardized multiaddr format:

/ip4/<RELAY_IP>/tcp/<PORT>/p2p/<RELAY_PEER_ID>/p2p-circuit/p2p/<TARGET_PEER_ID>
     └─────────┬─────────┘          └──────┬──────┘              └──────┬───────┘
          Relay server               Relay identity            Target peer
          connection info                                      destination

Example from test:

/ip4/172.25.0.2/tcp/4001/p2p/QmRkt8Gc9vNPWixWMQcyb5ZXUSPdW5VorMPvT5tXU37QDc/p2p-circuit/p2p/QmWpiDhWFDVMmu8HVvwxRKg1V6cBLBwbok5NB66dFggiyN

Implementation Details

Core Components

1. CircuitV2Protocol

• Implements Circuit Relay v2 protocol specification
• Supports HOP (relay server) and STOP (relay client) modes
• Manages resource limits per relay connection
• Handles reservation requests and circuit establishment

2. CircuitV2Transport

• Transport layer for circuit relay connections
• Intercepts and processes /p2p-circuit/ multiaddrs
• Creates relayed connections through intermediary peers
• Integrates with libp2p host transport infrastructure

3. RelayResourceManager

• Tracks active relay reservations by peer ID
• Monitors bytes transferred per reservation
• Enforces connection and data limits
• Provides reservation lifecycle management

4. RelayDiscovery

• Automates relay server discovery and reservation
• Manages automatic relay reservation renewal
• Links with transport for seamless circuit creation

Key Technical Insights

Circuit Address Handling

The standard libp2p host.connect() method does not handle /p2p-circuit/ addresses. The swarm passes these addresses to the base TCP transport, which cannot parse the circuit format. Circuit connections must be established through direct calls to the circuit transport:

# Detection phase
if "/p2p-circuit/" in multiaddr_string:
    # Parse relay and target peer IDs
    relay_info = parse_relay_from_circuit_addr(multiaddr)
    
    # Step 1: Establish TCP connection to relay server
    await host.connect(relay_info)
    
    # Step 2: Dial target through circuit transport
    connection = await circuit_v2_transport.dial(circuit_multiaddr)
    
    # Step 3: Application protocols (PubSub) negotiate over circuit

Auto-Initialization

The implementation automatically detects circuit addresses in connection arguments and initializes the relay transport infrastructure:

# Scan connect addresses for circuit relay requirements
needs_relay_transport = relay_server_mode or relay_addrs
if not needs_relay_transport and connect_addrs:
    for addr in connect_addrs:
        if "/p2p-circuit/" in addr:
            needs_relay_transport = True
            break

# Initialize CircuitV2Protocol and Transport if needed
if needs_relay_transport:
    circuit_v2 = CircuitV2Protocol(host, limits, allow_hop=relay_server_mode)
    circuit_v2_transport = CircuitV2Transport(host, circuit_v2, relay_config)

Resource Manager Integration

Relay server statistics are accessed through the correct internal attributes:

# Access reservations dictionary (note underscore prefix)
rm = circuit_v2.resource_manager
reservations = rm._reservations  # Not 'reservations'

# Each reservation object provides:
# - peer_id: ID of peer with reservation
# - created_at: Timestamp of reservation creation
# - expire: Expiration timestamp
# - bytes_transferred: Total bytes relayed
# - active_connection_count: Current active circuits

Enhanced Status Command

The /status command provides comprehensive relay debugging information:

Relay Server View:

• Hosted reservations count
• Per-reservation statistics (peer ID, bytes transferred, active connections)
• Active circuit connections by peer

Relay Client View:

• Relay circuit addresses obtained
• Reservation status
• AutoNAT reachability status

All Peers:

• Swarm connection counts
• PubSub peer relationships
• DHT routing table size
• Subscribed topics

Test Execution

Docker Environment Setup

# Create isolated networks
docker network create networkA
docker network create networkB

# Build application image
docker build -t peer-app:v1 .

Running Test Peers

Step 1: Start Relay Server

docker run -it --rm \
  --name relay \
  --network networkA \
  peer-app:v1 \
  --nick Relay \
  --topic room1 \
  --port 4001 \
  --ui \
  --relay-server

# In separate terminal, connect relay to second network
docker network connect networkB relay

Step 2: Start PeerA (Relay Client)

docker run -it --rm \
  --name peerA \
  --network networkA \
  peer-app:v1 \
  --nick PeerA \
  --topic pldg \
  --port 4002 \
  --ui \
  --relay /ip4/172.24.0.2/tcp/4001/p2p/QmRkt8Gc9vNPWixWMQcyb5ZXUSPdW5VorMPvT5tXU37QDc

Step 3: Start PeerB (Circuit Client)

docker run -it --rm \
  --name peerB \
  --network networkB \
  peer-app:v1 \
  --nick PeerB \
  --topic pldg \
  --port 4003 \
  --ui \
  -c /ip4/172.25.0.2/tcp/4001/p2p/QmRkt8Gc9vNPWixWMQcyb5ZXUSPdW5VorMPvT5tXU37QDc/p2p-circuit/p2p/QmWpiDhWFDVMmu8HVvwxRKg1V6cBLBwbok5NB66dFggiyN

Command Line Arguments

--relay-server

• Enables relay server mode (HOP + STOP + CLIENT roles)
• Accepts and manages relay reservations
• Allows circuit connections through this peer

--relay

• Connects to specified relay server
• Requests relay reservation
• Obtains circuit address for incoming connections

-c / --connect

• Connects to specified peer at startup
• Supports both direct TCP and circuit relay addresses
• Auto-detects /p2p-circuit/ format and uses circuit transport

--topic

• Subscribes to specified PubSub topic
• Must match across peers for message exchange

--ui

• Enables Textual TUI interface
• Displays chat messages, peers, and system logs

Test Results

Successful Outcomes

Relay Reservation Established

• PeerA successfully obtained reservation from relay server
• Relay server shows hosted reservation with peer ID
• Reservation includes resource limits and expiration

Circuit Connection Established

• PeerB connected to PeerA through relay circuit
• No direct TCP connection between peers
• Circuit transport properly intercepted multiaddr format

PubSub Communication Verified

• Messages sent from PeerA received by PeerB
• Messages sent from PeerB received by PeerA
• Both peers on same topic (pldg) despite network isolation
• Message delivery confirms end-to-end circuit functionality

Resource Monitoring Working

• Relay server tracks bytes transferred per reservation
• Active connection counts update in real-time
• /status command shows all relay statistics accurately

Performance Observations

Connection Establishment Time

• Relay TCP connection: ~1-2 seconds
• Reservation request: ~2-3 seconds
• Circuit dial through relay: ~3-5 seconds
• PubSub protocol negotiation: ~2-3 seconds
• Total end-to-end: ~10-15 seconds

Message Latency

• Direct connection baseline: <10ms
• Via circuit relay: 50-100ms (acceptable overhead)
• Consistent latency across multiple messages

Multiaddrs from Test

Relay Server:

/ip4/172.24.0.2/tcp/4001/p2p/QmRkt8Gc9vNPWixWMQcyb5ZXUSPdW5VorMPvT5tXU37QDc

PeerA:

/ip4/172.24.0.2/tcp/4002/p2p/QmWpiDhWFDVMmu8HVvwxRKg1V6cBLBwbok5NB66dFggiyN

Circuit Address (PeerB to PeerA):

/ip4/172.25.0.2/tcp/4001/p2p/QmRkt8Gc9vNPWixWMQcyb5ZXUSPdW5VorMPvT5tXU37QDc/p2p-circuit/p2p/QmWpiDhWFDVMmu8HVvwxRKg1V6cBLBwbok5NB66dFggiyN

Configuration Parameters

Relay Limits

Default resource limits for relay connections:

DEFAULT_RELAY_LIMIT_DURATION = 120  # seconds
DEFAULT_RELAY_LIMIT_DATA_SIZE = 1 << 17  # 128KB
DEFAULT_RELAY_MAX_CIRCUIT_CONNS = 16
DEFAULT_RELAY_MAX_RESERVATIONS = 128

AutoNAT Configuration

AutoNAT service automatically determines peer reachability:

• Enabled by default
• Can be disabled with --no-autonat flag
• Helps peers understand their NAT situation

DCUtR Configuration

Direct Connection Upgrade through Relay (hole punching):

• Enabled by default when relay is configured
• Can be disabled with --no-dcutr flag
• Attempts to upgrade relayed connections to direct connections

Troubleshooting

Common Issues

"Circuit transport not initialized" error

• Ensure relay transport is enabled via --relay flag or circuit address in -c
• Auto-detection only works if -c address contains /p2p-circuit/

Reservation denied by relay server

• Check relay server resource limits
• Verify relay server is in HOP mode (--relay-server)
• Check relay server logs for denial reason

No PubSub messages received

• Verify all peers subscribe to the same topic
• Check PubSub peer connections with /status command
• Allow 5-10 seconds for protocol negotiation

"Already connected to peer" with circuit address

• Both TCP and circuit connections to same peer may exist
• This is normal; libp2p will use best available connection

Debug Commands

Check relay status on server:

/status

Shows hosted reservations and active circuits

Check circuit address on client:

/status

Shows relay circuit addresses obtained

List connected peers:

/peers

Display full multiaddress:

/multiaddr

References

libp2p Specifications:

• Circuit Relay v2: https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md
• AutoNAT: https://github.com/libp2p/specs/blob/master/autonat/README.md
• DCUtR: https://github.com/libp2p/specs/blob/master/relay/DCUtR.md

Implementation:

• changes: sumanjeet0012/universal-connectivity@fb1da85 (since i took a fork from forked repo of @sumanjeet0012 (as directed by him) , its a bit mesy)
• Universal Connectivity: https://github.com/libp2p/universal-connectivity