XDP + AF_XDP QUIC Packet Processor

High-performance QUIC packet processing combining kernel XDP filtering with userspace AF_XDP packet handling. This project demonstrates the complete pipeline from NIC to userspace application using Linux kernel's fastest data path.

Overview

This project consists of two main components that work together:

1. XDP Program (kernel): Filters and redirects QUIC packets
2. AF_XDP Application (userspace): Receives and processes redirected packets

Architecture

Packet Flow:
NIC → XDP → AF_XDP Socket → Userspace Application

Component 1: XDP Filter (xdp/xdp_filter.c)

Runs in the kernel, attached directly to the NIC. Processes every packet at line rate:

• Parses Ethernet, IP, and UDP headers
• Filters QUIC traffic on port 4433
• Extracts QUIC Destination Connection ID (DCID) from long headers
• Tracks packet counts per Connection ID in BPF map
• Backend ID Encoding: Parses CIDv1 format to extract backend_id from DCID
  • Validates CID version byte (must be 1)
  • Extracts 16-bit backend_id from bytes 2-3 (big-endian)
  • Maps backend_id to AF_XDP queue: queue = backend_id % MAX_QUEUES
• Redirects packets to appropriate AF_XDP socket based on backend_id

BPF Maps:

• cid_counter: Tracks packet counts for each Connection ID
• cid_backend_map: Maps Connection IDs to backend queues (65536 entries)
• xsks_map: Maps queue IDs to AF_XDP sockets (64 entries)

Component 2: AF_XDP Application (afxdp/src/main.rs)

Userspace Rust application that receives packets redirected by XDP:

• Creates AF_XDP socket bound to network interface queue
• Allocates UMEM (User Memory) for zero-copy packet buffers
• Sets up ring buffers (RX ring, Fill queue)
• Registers socket in XDP's xsks_map for packet redirection
• Receives and processes packets in tight polling loop
• Prints packet information for inspection

How XDP and AF_XDP Work Together

1. XDP Program filters packets in kernel (fastest possible path)
2. XDP redirects selected packets using bpf_redirect_map(&xsks_map, queue_id, 0)
3. AF_XDP socket registered in xsks_map receives the redirected packets
4. Userspace application processes packets from AF_XDP socket without kernel network stack overhead

This provides kernel-bypass networking with the ability to do custom packet processing in kernel (XDP) before userspace delivery (AF_XDP).

Learn More

For a comprehensive deep-dive into AF_XDP and the implementation details of this project, read the complete tutorial:

Featured in Medium: The Ultimate Guide to AF_XDP: High-Performance Networking in Rust

Prerequisites

• Linux kernel (with AF_XDP support)
• BPF/XDP toolchain: clang, llvm
• Rust toolchain (edition 2024)
• libbpf and libxdp development libraries
• Root privileges
• BPF filesystem mounted at /sys/fs/bpf

Setup Instructions

Step 1: Compile XDP Filter

clang -O2 -g -target bpf -c xdp/xdp_filter.c -o xdp_filter.o

Step 2: Load XDP Program to NIC

sudo bpftool prog load xdp_filter.o /sys/fs/bpf/xdp_prog type xdp pinmaps /sys/fs/bpf
sudo ip link set dev eth xdp pinned /sys/fs/bpf/xdp_prog

Verify XDP is attached:

sudo ip link show dev eth0
# Should show "xdp" in the output

Step 3: Build AF_XDP Application

cd afxdp
cargo build --release

Step 4: Run AF_XDP Application

sudo ./afxdp/target/release/afxdp

The application will:

• Create AF_XDP socket on eth0 queue 0
• Register itself in the XDP's xsks_map
• Start receiving packets redirected by XDP
• Print packet information as they arrive

Testing

Using QUIC Server

For realistic testing, run a QUIC server on another machine. A simple QUIC server is available in the Async-Chat repository.

1. On another machine, clone and run the QUIC server:

   git clone https://github.com/Shradhesh71/Async-Chat
   cd Async-Chat
   # Follow setup instructions and run server on port 4433

2. With AF_XDP application running, send QUIC traffic from client to 10.0.0.1:4433

3. Watch the AF_XDP application output to see intercepted packets

The AF_XDP application should receive and display the packet.

Monitoring BPF Maps

Check XDP statistics and connection tracking:

# View packet counts per Connection ID
sudo bpftool map dump name cid_counter

# View registered AF_XDP sockets
sudo bpftool map dump name xsks_map

# View backend mappings
sudo bpftool map dump name cid_backend_map

Technical Details

XDP Filter (xdp_filter.c)

• Protocol: QUIC over UDP
• Port: 4433
• Max CID Length: 20 bytes
• CID Version: 1 (CIDv1 format with backend encoding)
• Backend ID Encoding:
  • 16-bit backend_id stored in DCID bytes 2-3 (big-endian)
  • Queue selection: backend_id % 64 (MAX_QUEUES)
  • Enables load balancing across AF_XDP sockets
• BPF Map Sizes:
  • cid_counter: 1024 entries (hash map)
  • cid_backend_map: 65536 entries (hash map)
  • xsks_map: 64 entries (XSK map)
• Processing: QUIC long header packets only
• Redirect: Routes packets to specific queue based on backend_id

AF_XDP Application (src/main.rs)

• Interface: eth0
• Queue ID: 0
• UMEM Size: 8 MB (4096 frames × 2048 bytes)
• Frame Size: 2048 bytes
• Ring Sizes: 1024 entries (RX, Fill, Completion)
• Mode: Copy or zero-copy (kernel chooses)

How Packets are Processed

1. Packet arrives at NIC on port 4433
2. XDP program examines packet in kernel:
   • Validates headers (Ethernet → IP → UDP)
   • Checks UDP destination port (4433)
   • Parses QUIC long header
   • Extracts DCID (Destination Connection ID)
   • Updates cid_counter map
   • Decodes backend_id from CIDv1 format:
     • Verifies CID version is 1
     • Extracts backend_id from bytes 2-3 (16-bit big-endian)
     • Calculates target queue: backend_id % 64
3. XDP redirects packet to corresponding AF_XDP socket using bpf_redirect_map()
4. Packet is placed in AF_XDP RX ring (zero-copy to UMEM)
5. Userspace application polls RX ring
6. Application reads packet from UMEM and processes it
7. Frame is returned to Fill queue for reuse

Project Structure

xdp/
  xdp_filter.c      # XDP BPF program (attaches to NIC)
afxdp/
  src/main.rs       # AF_XDP userspace application
  Cargo.toml        # Dependencies (libc, libbpf-rs)

License

This project is for educational and research purposes.