URL Shortener - High-Performance Go Microservice

A production-ready URL shortening microservice built with Go, featuring distributed rate limiting, event-driven analytics, and time-series data storage. Designed for high throughput and horizontal scalability.

Features

• Multiple Shortening Strategies - Token-based (unique per request) or hash-based (URL deduplication)
• Policy-Based Rate Limiting - Configurable limits per scope (read/write) with sliding window algorithm
• Dual Storage Backend - In-memory for development, Redis for distributed deployments
• Event-Driven Architecture - Async analytics via Redis Streams with Watermill
• Time-Series Analytics - URL creation and access events stored in TimescaleDB
• Multi-Layer Caching - LRU in-memory cache with Redis cache-aside pattern
• OpenAPI Documentation - Auto-generated API docs with Huma framework
• Health Checks - Kubernetes-ready liveness and readiness probes

Tech Stack

Component	Technology
Language	Go 1.25+
HTTP Framework	Huma with Chi router
Database	PostgreSQL with TimescaleDB
Cache	Redis 7
Messaging	Redis Streams via Watermill
Migrations	Atlas
DI Container	samber/do

Quick Start

Prerequisites

• Go 1.25+
• Docker and Docker Compose
• hey (for performance testing)

Run Locally

# Start dependencies (Redis, TimescaleDB) and run migrations
docker-compose up -d

# Start the API server
go run ./cmd/server --database-url="postgres://shortener:shortener@localhost:5432/shortener?sslmode=disable"

# (Optional) Start the analytics consumer in a separate terminal
go run ./cmd/consumer --database-url="postgres://shortener:shortener@localhost:5432/shortener?sslmode=disable"

The API will be available at http://localhost:8888.

Running with Custom Options

# Use Redis for distributed rate limiting
go run ./cmd/server \
  --database-url="postgres://shortener:shortener@localhost:5432/shortener?sslmode=disable" \
  --rate-limit-store=redis \
  --redis-addr=localhost:6379

# Change port and cache settings
go run ./cmd/server \
  --database-url="postgres://shortener:shortener@localhost:5432/shortener?sslmode=disable" \
  --port=3000 \
  --cache-size=5000 \
  --cache-ttl=30m

API Reference

Create Short URL

POST /shorten
Content-Type: application/json

{
  "url": "https://example.com/very/long/path",
  "strategy": "token"
}

Strategies:

Strategy	Description
token	Generates a unique short code for every request (default)
hash	Returns the same short code for identical URLs (deduplication)

Response:

{
  "code": "abc123",
  "shortUrl": "http://localhost:8888/abc123",
  "originalUrl": "https://example.com/very/long/path"
}

Redirect

GET /{code}

Returns a 301 Moved Permanently redirect to the original URL.

Health Check

GET /health

Returns service health status including Redis connectivity.

Configuration

All settings can be configured via environment variables or command-line flags:

Environment Variable	Flag	Default	Description
DATABASE_URL	--database-url	-	PostgreSQL connection string (required)
RATE_LIMIT_STORE	--rate-limit-store	memory	Rate limit backend (memory or redis)
RATE_LIMIT_GLOBAL_DAY	--rate-limit-global-per-day	1000000	Max requests per day (global)
RATE_LIMIT_READ_MINUTE	--rate-limit-read-per-minute	100000	Max read requests per minute
RATE_LIMIT_WRITE_MINUTE	--rate-limit-write-per-minute	10	Max write requests per minute
CACHE_SIZE	--cache-size	1000	LRU cache size (0 to disable)
CACHE_TTL	--cache-ttl	1h	Redis cache TTL

Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Client    │────▶│  Rate Limit │────▶│   Handler   │
└─────────────┘     │  Middleware │     └──────┬──────┘
                    └─────────────┘            │
                           │                   ▼
                    ┌──────┴──────┐     ┌─────────────┐
                    │    Redis    │     │  Repository │
                    │   (Store)   │     └──────┬──────┘
                    └─────────────┘            │
                                         ┌────┴────┐
                                         ▼         ▼
                                   ┌─────────┐ ┌─────────┐
                                   │  Redis  │ │ Postgres│
                                   │ (Cache) │ │  (DB)   │
                                   └─────────┘ └─────────┘

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Handler   │────▶│   Redis     │────▶│  Consumer   │────▶ TimescaleDB
│  (Events)   │     │  Streams    │     │  (Analytics)│
└─────────────┘     └─────────────┘     └─────────────┘

Development

# Run tests
go test ./...

# Run tests with coverage
go test ./... -coverprofile=coverage.out

# Run linter
golangci-lint run

# Generate mocks
go generate ./...

Performance Testing

The project includes scripts for load testing and performance regression detection.

Quick Stress Test

# Run comprehensive stress test (requires hey)
./scripts/stress-test.sh

This runs various load scenarios including burst traffic, sustained load, and mixed read/write workloads.

CI Performance Tests

# Run the CI performance test suite
./scripts/perf-test.sh

# Compare against baseline (used in CI)
./scripts/perf-compare.sh

The perf-test.sh script:

• Runs 1000 requests at 50 concurrency for redirect (read) and shorten (write) endpoints
• Outputs metrics to perf-results.json
• Gates on p95 latency < 50ms threshold

The perf-compare.sh script:

• Compares current results against a cached baseline
• Generates perf-report.md with comparison table
• Flags regressions > 50% from baseline

End-to-End Tests

# Run E2E tests validating the full async flow
./scripts/e2e-test.sh

This validates the complete flow: API → Redis Streams → Consumer → TimescaleDB.

Manual Load Testing

# Install hey if not already installed
go install github.com/rakyll/hey@latest

# Test redirect endpoint (read)
hey -n 1000 -c 50 http://localhost:8888/{short-code}

# Test shorten endpoint (write)
hey -n 100 -c 10 -m POST -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","strategy":"token"}' \
  http://localhost:8888/shorten

License

MIT License - see LICENSE for details.