Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 29 additions & 105 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,130 +1,54 @@
# Scrawn Backend

Scrawn is a self-hostable billing backend built on [Bun](https://bun.sh), [Drizzle ORM](https://orm.drizzle.team), and [Dodo Payments](https://dodopayments.com). It exposes a gRPC API for high-throughput event ingestion and an HTTP API for webhooks and management, with pluggable storage between PostgreSQL and ClickHouse.
Scrawn is a **self-hostable usage-based billing backend** that replaces 60+ lines of billing boilerplate with a single function call. It ingests billable events, evaluates your pricing logic, and hooks into [Dodo Payments](https://dodopayments.com) for collection — no cron jobs, no manual gRPC plumbing.

## Features
Built on [Bun](https://bun.sh), [Fastify](https://fastify.dev), [gRPC](https://grpc.io), and [PostgreSQL](https://postgresql.org) / [ClickHouse](https://clickhouse.com).

- **gRPC API** — high-throughput event ingestion, streaming batch registration, query service
- **HTTP API** — Dodo Payments webhooks, checkout redirects, tag/expression management
- **Dual storage** — PostgreSQL (relational) or ClickHouse (columnar analytics), swappable via env var
- **Authentication** — HMAC-based API key system with role scoping
- **Expressions** — dynamic pricing via configurable expression engine
## Prerequisites

## Quick Start

### Prerequisites

- [Bun](https://bun.sh) (latest)
- Docker (for PostgreSQL and ClickHouse)
- Dodo Payments account

### 1. Clone and install
- [Bun](https://bun.sh)
- [Docker](https://docker.com)
- A [Dodo Payments](https://dodopayments.com) account

```bash
git clone https://github.com/ScrawnDotDev/Scrawn.git
cd Scrawn
bun install
```
## Quick Start

### 2. Configure environment
The [Scrawn CLI](https://github.com/ScrawnDotDev/CLI) automates the entire local stack — just run:

```bash
cp .env.example .env.local
bunx scrawn@latest init
bunx scrawn@latest start
```

Edit `.env.local`:

```env
DATABASE_URL=postgresql://user:password@localhost:5432/scrawn
CLICKHOUSE_URL=http://default:password@localhost:8123/scrawn
HMAC_SECRET=your-hmac-secret-key
DODO_PAYMENTS_LIVE_API_KEY=your-dodo-live-api-key
DODO_PAYMENTS_TEST_API_KEY=your-dodo-test-api-key
DODO_PAYMENTS_LIVE_PRODUCT_ID=your-dodo-live-product-id
DODO_PAYMENTS_TEST_PRODUCT_ID=your-dodo-test-product-id
DODO_PAYMENTS_WEBHOOK_SECRET=your-webhook-secret
STORAGE_ADAPTER=postgres # or "clickhouse"
SENTRY_DSN=https://your-dsn@sentry.io/your-project
```
This generates the docker-compose config and starts PostgreSQL, ClickHouse, the gRPC server, and the dashboard — all in the background.

### 3. Start infrastructure
### Stop & Reset

```bash
docker compose up -d
bunx scrawn@latest stop # Graceful stop, preserves data
bunx scrawn@latest reset # Wipes all data volumes
```

### 4. Run migrations
## SDK Integration

```bash
# Postgres (always required)
bunx drizzle-kit push

# ClickHouse (only if STORAGE_ADAPTER=clickhouse)
bun run migrate:clickhouse
```
```ts
import { scrawn } from "@scrawn/core";

### 5. Start the server
const biller = scrawn({
apiKey: process.env.SCRAWN_KEY,
baseURL: process.env.SCRAWN_BASE_URL,
httpUrl: process.env.SCRAWN_HTTP_URL,
});

```bash
bun run dev:backend
// Track usage in one line
await biller.basicUsageEventConsumer({
userId: "cus_123",
debit: 4500,
});
```

The server starts on two ports:

- **gRPC** (h2c): `localhost:8069`
- **HTTP** (Fastify): `localhost:8070`

## API Overview

### gRPC Services

| Service | RPC | Description |
| ---------------- | ------------------ | --------------------------------------------------- |
| AuthService | CreateAPIKey | Create a new API key |
| EventService | RegisterEvent | Register a single usage event |
| EventService | StreamEvents | Client-streaming batch event registration |
| PaymentService | CreateCheckoutLink | Generate a Dodo Payments checkout link |
| QueryService | QueryEvents | Query events with filters, aggregation, group-by |
| DataQueryService | Query | Query internal tables (users, sessions, tags, etc.) |

### HTTP Endpoints

| Method | Path | Purpose |
| -------- | ----------------------------------- | -------------------------- |
| GET | `/` | Health check |
| GET | `/checkout/:sessionId` | Checkout redirect |
| POST | `/webhooks/payment/createdCheckout` | Dodo Payments webhook |
| GET/POST | `/api/v1/tags` | Manage pricing tags |
| GET/POST | `/api/v1/expressions` | Manage pricing expressions |
| POST | `/api/v1/internals/onboarding` | Onboarding endpoint |

## Storage Adapters

Scrawn supports two storage backends, switchable via the `STORAGE_ADAPTER` env var:

- **`postgres`** (default) — full relational schema via Drizzle ORM
- **`clickhouse`** — columnar analytics DB with `ReplacingMergeTree` for event deduplication

Only one adapter operates at a time across all event types.

## TLS (gRPC)

By default the gRPC server runs without TLS. In production, place it behind a TLS-terminating proxy or enable TLS directly:

```env
GRPC_TLS_ENABLED=true
GRPC_TLS_CERT_PATH="/path/to/server.crt"
GRPC_TLS_KEY_PATH="/path/to/server.key"
GRPC_TLS_CA_PATH="/path/to/ca.pem"
```

## Contributing

Contributions are welcome! See [CONTRIBUTING.md](./CONTRIBUTING.md) for development setup, testing, code style, and the PR process.

## Documentation
## Docs

For complete API documentation and integration guides, visit the [Scrawn Docs](https://scrawn.vercel.app/docs).
Complete API reference and integration guides: [scrawn.vercel.app/docs](https://scrawn.vercel.app/docs)

## License

Expand Down
5 changes: 2 additions & 3 deletions src/interceptors/auth.ts
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ export function authInterceptor<Req, Res>(
wideEventBuilder?.setAuth(cached.id, true);

if (needsWebhook) {
checkWebhookEndpoint(cached.id)
return checkWebhookEndpoint(cached.id)
.then((hasEndpoint) => {
if (!hasEndpoint) {
return callback?.(
Expand All @@ -184,13 +184,12 @@ export function authInterceptor<Req, Res>(
return handler(call, callback);
})
.catch((error) => callback?.(error));
return;
}

return handler(call, callback);
}

lookupApiKey(apiKeyHash)
return lookupApiKey(apiKeyHash)
.then((apiKeyRecord) => {
if (!apiKeyRecord) {
return callback?.(AuthError.invalidAPIKey("API key not found"));
Expand Down
Loading