Project Linsen: B2B Routing & Semantic Aggregation Engine

Project Linsen is a consent-based B2B appointment and resource routing MVP. It ingests authorized provider/operator messages, normalizes them into structured events, and routes requests to available resources by capacity and provider priority.

This repository now contains a working baseline:

• Go core API with health, provider, resource, availability, ingestion, routing, event, and audit endpoints.
• Python ingestion service with deterministic message normalization.
• Next.js frontend scaffold for a Vercel-ready operations dashboard.
• Docker Compose for local Postgres, Redis, API, ingestion, and frontend services.
• Postgres/Neon-compatible initial migration.
• Unit tests for routing, ingestion authorization, API behavior, and parser validation.

Scope Boundaries

The MVP is intentionally limited to lawful, explicitly authorized B2B workflows.

It does not include:

• Unauthorized scraping of Telegram, LINE, private groups, or access-controlled sources.
• Evidence destruction, seizure avoidance, or operational concealment features.
• Illegal brokering, payment, advertising, or dispatch workflows.
• Plaintext storage of sensitive personal identifiers without a consent and retention model.
• Production ZKP circuits, GraphQL, Redpanda, or LLM calls.

Those deferred items are documented in docs/implementation-plan.md only where they are compatible with a lawful production roadmap.

The previous crawler/Grok/request-aggregator topic is preserved as a decision record in docs/legacy-crawler-aggregator-topic.md. It explains what was removed from the MVP, what architecture lessons remain useful, and what compliance gates would be required before any aggregator node returns.

Frontend and API surfaces are documented separately in docs/frontend-and-api-surface.md.

Repository Layout

backend/                 Go REST API MVP
db/migrations/           Neon/Postgres-compatible SQL schema
docs/                    Architecture, API, data model, local dev, compliance, plan
frontend/                Next.js dashboard scaffold
ingestion/               Python deterministic normalization service
scripts/test.sh          Local backend and ingestion test runner
docker-compose.yml       Local development stack
.env.example             Environment variable template

Quick Start

Run tests:

sh scripts/test.sh

Run the Go API locally:

cd backend
go run .

Run the ingestion service locally:

cd ingestion
python3 service.py

Run the full local stack:

docker compose up --build

The Docker Compose frontend service installs npm dependencies inside the container on first boot.

Core API Smoke Test

With the API running on localhost:8080:

curl -s http://localhost:8080/healthz

Create a provider, resource, availability window, then a routing request:

curl -s -X POST http://localhost:8080/providers \
  -H 'content-type: application/json' \
  -d '{"name":"Provider A","priority":5}'

All request bodies and endpoint details are documented in docs/api.md.

MVP Routing Logic

Routing is deterministic:

1. Match resources with an availability window covering the requested time.
2. Require resource and window capacity to satisfy requested capacity.
3. Exclude inactive providers and resources.
4. Prefer higher provider priority.
5. Break remaining ties by smaller sufficient resource capacity, then resource ID.

Documentation

• Implementation plan
• Architecture
• API
• Frontend and API surface
• Data model
• Local development
• Compliance boundaries
• Legacy crawler and aggregator topic

Verification Status

Verified locally:

• go test ./... in backend/
• python3 -m unittest in ingestion/
• docker compose config
• Runtime API smoke check for GET /healthz and POST /providers

Not verified in this pass:

• Next.js install/build, because dependencies are not vendored and were not downloaded.
• Docker Compose runtime, because it requires pulling container images.