Distributed Metadata & Object Storage System

A high-performance, distributed platform combining URL Shortening and Object Storage (S3-compatible). Designed for scale with horizontal sharding, caching, and background processing.

🚀 Features

• 🌐 Distributed Architecture - Run multiple stateless API nodes behind an Nginx load balancer.
• 🔗 URL Shortening - High-performance link generation with Base62 encoding and Snowflake IDs.
• 📦 Object Storage - S3-compatible file storage using MinIO with metadata management.
• ⚡ Smart Caching - Multi-layer caching with Redis (Cache-Aside & Write-Through patterns).
• �️ Resilience - Rate limiting (Token Bucket), Graceful Degradation, and Connection Pooling.
• 🔄 Background Workers - Async workers for TTL cleanup and eventually consistent deletions.
• 📊 Observability - Real-time metrics via Prometheus and dashboards in Grafana.

🛠️ Quick Start

This project includes a Makefile to simplify development.

1. Start Infrastructure

Starts PostgreSQL, Redis, MinIO, Nginx, Prometheus, and Grafana.

make infra

2. Run Application

You can run the API normally, or use the distributed setup.

Option A: Simple Run (Single Node)

make run
# Access at http://localhost:8080

Option B: Distributed Run (Multiple Nodes) Open 3 separate terminals:

# Terminal 1
make run

# Terminal 2
SERVER_PORT=8081 make run

# Terminal 3
SERVER_PORT=8082 make run

Access via Load Balancer at http://localhost (Port 80). Nginx will round-robin requests across all instances.

3. Start Background Worker

Handles cleanup of expired keys and objects.

make worker

🏗️ Architecture

graph TD
    User[Clients] --> LB[Nginx Load Balancer]
    LB --> API1[API Node 1]
    LB --> API2[API Node 2]
    LB --> API3[API Node 3]
    
    API1 & API2 & API3 --> Redis[(Redis Cache)]
    API1 & API2 & API3 --> DB[(PostgreSQL Metadata)]
    API1 & API2 & API3 --> MinIO[(MinIO Object Store)]
    
    Worker[Cleanup Worker] --> DB
    Worker --> Redis
    Worker --> MinIO

Loading

🧪 Verification & Testing

Run Load Tests

Requires hey installed.

# Test the Load Balancer with 2,000 RPS
make load-test URL=http://localhost/r/YOUR_SHORT_KEY

Check Metrics

• Prometheus: http://localhost:9090
• Grafana: http://localhost:3000 (admin/admin)

📝 API Endpoints

Method	Endpoint	Description
POST	/api/v1/urls	Create a short URL
GET	/r/{shortKey}	Redirect to original URL
POST	/api/v1/objects	Upload a file
GET	/api/v1/objects/{id}	Download a file
GET	/health	Check system status

⚡ Performance Benchmarks

Tested with hey load testing tool.

Test	Requests	Concurrency	Throughput	p50	p95	p99	Errors
Standard	10,000	100	22,577 RPS	4ms	8ms	26ms	0
Heavy	100,000	500	23,037 RPS	19ms	46ms	78ms	0

Linear latency scaling under 5x load with zero errors — production-ready performance.

Run Benchmarks

# Install hey
brew install hey

# Standard load test
./scripts/benchmark.sh

# Heavy load test (production-level)
REQUESTS=100000 CONCURRENCY=500 ./scripts/benchmark.sh