Skip to content
BranchesTags

Repository files navigation

Spoke

Make your AI Agent API calls 5x faster. Zero code changes.

Spoke is a persistent local daemon that maintains an HTTP/2 connection pool to upstream APIs. When your agent scripts exit, the connections don't — the next process picks them up instantly, skipping DNS → TCP → TLS entirely.

Why Spoke?

Without Spoke                      With Spoke
────────────                       ──────────
Agent script 1 starts              Agent script 1 starts
  ├─ DNS lookup   (50ms)             ├─ HTTP → Spoke  (0.05ms, local)
  ├─ TCP handshake (30ms)            ├─ Spoke reuses warm connection → API
  ├─ TLS handshake (300ms)           └─ Script exits, connection stays alive
  ├─ HTTP request   (100ms)        ──────────
  └─ Script exits, connection dies  Agent script 2 starts
────────────                          ├─ HTTP → Spoke  (0.05ms)
Agent script 2 starts                ├─ Instant reuse!
  ├─ DNS lookup   (50ms)             └─ Another 400ms saved
  ├─ TCP handshake (30ms)          ──────────
  ├─ TLS handshake (300ms)          Agent script N starts
  … every script pays the tax         └─ Still using the same warm connection

The problem: AI Agents, Skills, and short-lived scripts are ephemeral processes. Each invocation pays the full connection setup cost. Spoke decouples connection lifetime from process lifetime.

Performance

Direct:  0.75s  1.10s  1.55s  0.44s  0.39s   (TLS handshake every call)
Spoke:   0.55s  0.14s  0.14s  0.14s  0.14s   (one TLS, reused forever)
                                              ↑ steady 4.6x speedup

Install

Option 1: One-liner

curl -fsSL https://raw.githubusercontent.com/motedb/spoke/main/install.sh | bash

Option 2: From source

git clone https://github.com/motedb/spoke.git && cd spoke
make install        # → /usr/local/bin
# or
make install-local  # → ~/.spoke/bin

Option 3: Pre-built binaries

Download from Releases:

Platform Arch
Linux x86_64, aarch64
macOS x86_64, aarch64

Quick Start

Zero-config (env vars only)

export SPOKE_UPSTREAM_URL=https://api.siliconflow.cn/v1/chat/completions
export SPOKE_AUTH_TOKEN=sk-your-key
spoke run

Then in another terminal:

export http_proxy=http://127.0.0.1:3000
curl http://api.siliconflow.cn/v1/chat/completions

With a config file

spoke init                    # generate spoke.yaml
spoke run --config spoke.yaml

Speed up all HTTPS APIs

# spoke.yaml
server:
  listen: "127.0.0.1:3000"
  upgrade_insecure: true      # auto upgrade http:// → https:// upstream

pool:
  http_max_idle_per_host: 10
  idle_timeout_secs: 300
  http2_keepalive_secs: 15   # PING keepalive every 15s

Set your proxy and use http:// instead of https://:

export http_proxy=http://127.0.0.1:3000
curl http://api.github.com/repos          # Spoke upgrades to HTTPS
curl http://api.openai.com/v1/models      # same thing

CLI

spoke run                          # start the gateway
spoke run --config spoke.yaml      # with a config file
spoke run --listen 0.0.0.0:8080    # override listen address
spoke run --debug                  # verbose logging

spoke init                         # generate a config file
spoke status                       # live dashboard
spoke --version                    # print version

Environment variables

Spoke auto-detects these env vars when no config file is present:

Variable Purpose Example
SPOKE_UPSTREAM_URL Upstream API endpoint https://api.openai.com/v1/chat/completions
SPOKE_AUTH_TOKEN Auth token sk-xxx
SPOKE_LISTEN Listen address 127.0.0.1:3000

Full Configuration Reference

server:
  listen: "127.0.0.1:3000"            # TCP address or unix:/path/to/sock
  read_timeout_secs: 30               # HTTP header read timeout
  shutdown_timeout_secs: 10           # graceful drain timeout
  upgrade_insecure: false             # auto http:// → https:// upstream

pool:
  http_max_idle_per_host: 20          # max idle connections per host
  idle_timeout_secs: 90               # idle connection lifetime
  http2_keepalive_secs: 0             # PING interval (0 = disabled)
  upstream_retries: 0                 # retry count for connect/timeout errors
  tcp_max_per_host: 16                # CONNECT tunnel pool capacity
  health_check_interval_secs: 15      # TCP pool health check interval

tcp:
  nodelay: true                       # disable Nagle's algorithm
  quickack: true                      # Linux TCP_QUICKACK
  fastopen_connect: true              # Linux TCP Fast Open
  keepalive_secs: 60                  # TCP keepalive interval

security:
  allow_hosts: []                     # domain allowlist (supports *.example.com)
  block_hosts: []                     # domain blocklist
  block_ips: []                       # IP/CIDR blocklist (e.g. "10.0.0.0/8")
  allow_ips: []                       # IP/CIDR allowlist
  block_private_ips: false            # block 10/8, 172.16/12, 192.168/16, etc.
  inject_headers: {}                  # global headers to inject

rules:
  - name: "openai-to-minimax"
    match:
      host: "api.openai.com"
      path_prefix: "/v1/chat/completions"
    transform:
      upstream: "https://api.minimax.chat/v1/text/chatcompletion_v2"
      set_headers:
        Authorization: "Bearer ${MINIMAX_API_KEY}"
      remove_headers:
        - "OpenAI-Organization"
      body_mapping: |
        {
          "model": "abab5.5-chat",
          "messages": ${body.messages},
          "temperature": ${body.temperature}
        }
      response_mapping: |
        {
          "id": ${resp.id},
          "object": "chat.completion",
          "choices": ${resp.choices},
          "usage": ${resp.usage}
        }

observability:
  log_level: "info"                   # trace / debug / info / warn / error
  log_format: "json"                  # json or text
  metrics_enabled: true               # Prometheus metrics
  metrics_path: "/metrics"            # metrics endpoint path

Use Cases

1. Speed up all API calls from AI Agents

# Start Spoke
spoke run --config spoke.yaml &

# Every HTTP client in every language benefits
export http_proxy=http://127.0.0.1:3000

python agent.py      # requests library uses the proxy
node skill.js        # fetch() uses the proxy
curl ...             # curl uses the proxy

2. API format translation

Different model providers, different API formats. Spoke translates transparently:

rules:
  - name: "openai-to-claude"
    match:
      host: "claude.local"
      path_prefix: "/v1/messages"
    transform:
      upstream: "https://api.anthropic.com/v1/messages"
      set_headers:
        x-api-key: "${ANTHROPIC_API_KEY}"
        anthropic-version: "2023-06-01"
      body_mapping: |
        {
          "model": ${body.model},
          "max_tokens": ${body.max_tokens},
          "messages": ${body.messages}
        }

Your agent code speaks OpenAI format. Spoke rewrites to Claude format on the fly.

3. Security gateway (SSRF protection)

security:
  block_private_ips: true
  block_hosts:
    - "*.internal.com"
    - "169.254.169.254"          # cloud metadata endpoints
  block_ips:
    - "10.0.0.0/8"
    - "172.16.0.0/12"
  allow_hosts:
    - "*.openai.com"
    - "*.anthropic.com"

4. Unix Socket for maximum speed

server:
  listen: "unix:/tmp/spoke.sock"
spoke run &
curl --unix-socket /tmp/spoke.sock http://localhost/api.github.com/repos

Unix sockets skip the TCP stack entirely — even faster than loopback.

Dashboard

$ spoke status
Spoke v0.2.3
─────────────────────────────────────────────
  Uptime:   3h 12m
  Listen:   127.0.0.1:3000
  Rules:    2
  Pool Hit: 94.2%
  Pool Idle: 12
  gateway_requests_total{host="api.github.com"} 5,230
  gateway_requests_total{host="api.openai.com"} 3,891
$ curl http://127.0.0.1:3000/metrics
gateway_requests_total{host="api.openai.com",status="200"} 5230
gateway_request_duration_seconds{host="api.openai.com",quantile="0.5"} 0.14
gateway_connection_pool_hit_ratio 0.94
gateway_connection_pool_idle 12
$ curl http://127.0.0.1:3000/health | jq
{
  "status": "ok",
  "version": "0.2.3",
  "uptime_secs": 11525,
  "listen": "127.0.0.1:3000",
  "rules_count": 2
}

Service (auto-start)

systemd (Linux)

sudo tee /etc/systemd/system/spoke.service << 'EOF'
[Unit]
Description=Spoke API Gateway
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/spoke run --config /etc/spoke.yaml
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now spoke

launchd (macOS)

cat > ~/Library/LaunchAgents/com.spoke.gateway.plist << 'EOF'
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.spoke.gateway</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/spoke</string>
        <string>run</string>
        <string>--config</string>
        <string>/Users/you/spoke.yaml</string>
    </array>
    <key>RunAtLoad</key><true/>
    <key>KeepAlive</key><true/>
</dict>
</plist>
EOF

launchctl load ~/Library/LaunchAgents/com.spoke.gateway.plist

Container / no init system

nohup spoke run --config /etc/spoke.yaml > /tmp/spoke.log 2>&1 &

Hot Reload

Update your config without restarting:

kill -SIGHUP $(pgrep spoke)

Active connections are unaffected. New connections use the updated config.

Stop

# Graceful (waits for in-flight requests)
kill -SIGTERM $(pgrep spoke)

Performance Tuning

# Lowest latency
server:
  listen: "unix:/tmp/spoke.sock"    # zero TCP overhead
pool:
  http_max_idle_per_host: 50
  http2_keepalive_secs: 15
tcp:
  nodelay: true
  quickack: true
  fastopen_connect: true
# Highest throughput
pool:
  http_max_idle_per_host: 100
  idle_timeout_secs: 600
  upstream_retries: 2              # auto-retry transient failures

Architecture

Component Choice Note
Async runtime Tokio multi-threaded work-stealing
HTTP server hyper 1.x zero-cost abstractions
HTTP client reqwest 0.12 built-in pool, HTTP/2 prior knowledge
TLS rustls pure Rust, no OpenSSL
Config YAML + SIGHUP human-readable, hot reload
Logging tracing structured, async, low-overhead
Metrics Prometheus standard format

Key optimizations:

  • HashMap rule index with two-finger merge — O(host_bucket) matching
  • Streaming forward — zero-buffering passthrough when no body mapping
  • Stream enum — TcpStream | UnixStream zero-allocation abstraction
  • Linux splice(2) — kernel-space zero-copy for CONNECT tunnels
  • Cow<str> — zero-allocation lowercase host comparison
  • Warmup — pre-establish TLS sessions on startup

See ARCHITECTURE.md for details.

FAQ

Why http:// URLs instead of https://?

When upgrade_insecure: true, Spoke auto-upgrades http:// to https:// upstream. This is because HTTP proxy CONNECT tunnels can't pool TLS connections (they're end-to-end encrypted). Using http:// + upgrade_insecure is what enables the 4.6x speedup.

Does it support CONNECT (HTTPS proxy)?

Yes, but CONNECT tunnels can't reuse TLS connections. For pooling to work, use upgrade_insecure mode and http:// URLs.

How do I know connections are being reused?

# First request (cold — establishes TLS)
$ time curl http://api.github.com/repos
0.55s

# Second request (hot — pool hit)
$ time curl http://api.github.com/repos
0.07s

# Check hit ratio
$ curl http://127.0.0.1:3000/metrics | grep pool_hit_ratio

Docker?

COPY --from=builder /app/target/release/spoke /usr/local/bin/spoke
CMD ["spoke", "run", "--config", "/etc/spoke.yaml"]

Contributing

Issues and PRs welcome. See CONTRIBUTING.md.

License

MIT

About

poke 是一个轻量级的本地智能网关，专为**短生命周期进程（AI Agent、脚本、CLI 工具）**设计。它作为常驻守护进程运行在本地，通过连接池复用和 TLS 预热，将每次 API 调用的延迟降低 50-85%。

Resources

Readme

Contributing

Contributing
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages