Neptune Task

This project is a high-performance RESTful service for managing and analyzing financial data in near real-time. It is built in Go and uses fasthttp for its high-performance HTTP server.

Functional Requirements

The service supports two HTTP endpoints:

• POST /add_batch: Adds a batch of trading data for a specific symbol.
• GET /stats: Retrieves statistical analysis of recent trading data for a symbol.

Technical Implementation

The service uses an in-memory storage solution designed for high-performance reads and writes. The core of the storage engine is a sliding window algorithm implemented with deques, which allows for the calculation of min, max, average, and variance in amortized O(1) time.

Getting Started

Prerequisites

• Go (version 1.24.5 or later)

Building and Running

1. Clone the repository:

   git clone <repository-url>
   cd neptune-task

2. Install dependencies:

   go mod tidy

3. Run the service:

   make run

The service will start on http://localhost:8080.

API Usage

Add Batch

• URL: /add_batch

• Method: POST

• Body:

  {
    "symbol": "BTCUSD",
    "values": [68000.5, 68001.0, 68000.0]
  }

Get Stats

• URL: /stats
• Method: GET
• Query Parameters:
  • symbol (string): The financial instrument's identifier.
  • k (integer): An integer from 1 to 8, specifying the number of last 1e{k} data points to analyze.
• Example: GET /stats?symbol=BTCUSD&k=2

Performance Benchmarks

The project includes comprehensive benchmarks to measure the maximum Requests Per Second (RPS) for different scenarios:

Running Benchmarks

Quick benchmark run:

make benchmark-quick

Standard benchmarks:

make benchmark

Detailed performance analysis:

make benchmark-detailed

Benchmark Types

1. AddBatch - Measures RPS for /add_batch endpoint
2. GetStats - Measures RPS for /stats endpoint

Development

Available Make Commands

make help              # Show all available commands
make build             # Build the application
make run               # Run the server
make test              # Run all tests
make deps              # Install dependencies
make fmt               # Format code
make lint              # Run linter
make clean             # Clean build artifacts

Testing

Run the test suite:

make test

Architecture

Key Components

1. Storage Engine (pkg/storage/)

   • Sliding window implementation
   • Deque-based min/max tracking
   • Welford's algorithm for rolling variance
   • Pre allocated arrays

2. API Layer (api/)

   • FastJSON for efficient JSON parsing
   • Fasthttp for high-performance HTTP
   • Input validation and error handling

3. Data Structures (pkg/storage/deque/)

   • Custom deque implementation for min/max tracking
   • Memory-efficient circular buffer