A lightweight, embeddable event streaming platform for .NET — Kafka-like capabilities without external infrastructure.
- In-process or multi-node clustering with leader election and automatic failover
- Kafka-style
.propertiesconfiguration — no hardcoded settings - Partitioned topics with configurable replication (ISR, acks, min in-sync)
- Consumer groups with automatic rebalance and offset tracking
- Delivery semantics: exactly-once, at-least-once, at-most-once
- Interceptor pipeline: claim-check, schema validation, tenant isolation, correlation, telemetry
- Dead-letter queue, retry topics with exponential backoff, circuit breaker
- Schema registry with forward/backward/full compatibility checks
- Reactive streaming with LINQ-style operators and windowing
- Saga orchestration for distributed workflows with compensation
- Event replay for reprocessing and auditing
- OpenTelemetry instrumentation with counters, histograms, and tracing
net8.0 · net9.0 · net10.0
using TheTechIdea.Beep.StreamandEvents;
// In-memory (development)
var node = BeepStreaming.CreateInMemory();
// From config file
var node = BeepStreaming.CreateFromConfig("node.properties");
// Cluster from config
var cluster = BeepStreaming.CreateClusterFromConfig("cluster.properties");
await cluster.StartAsync();Choose a storage backend for message durability:
| Provider | Config Value | Best For | Package |
|---|---|---|---|
| In-Memory | memory |
Dev/test | — |
| File-Based | file |
Production, Kafka-style logs | — |
| SQLite | sqlite |
Embedded apps | Microsoft.Data.Sqlite |
| LiteDB | litedb |
Simple embedded | LiteDB |
storage.type=file
storage.file.data.dir=./data/streams| Subsystem | Description |
|---|---|
| Storage | Pluggable persistence (file, SQLite, LiteDB, custom) |
| Interceptors | Claim-check, schema validation, tenant isolation, W3C trace context |
| Pipeline | ManagedConsumer, worker pool, backpressure, retry routing, outbox |
| Reactive | Operators (Where, Select, Buffer, Throttle), windowing, bridge conversions |
| Sagas | Step-based orchestration with compensation and timeout |
| Replay | Offset-based, time-range, filtered, real-time replay modes |
| Telemetry | Counters, histograms, distributed tracing, consumer lag gauge |
Beep.StreamingEvents/
├── Adapters/ # IBrokerAdapter implementations (gRPC, federated, edge)
├── Coordinator/ # EventStreamingCoordinator partials (lifecycle, producer, consumer, etc.)
├── Engine/ # Cluster engine (BeepStreamingCluster, Node, Replication, Watchdog)
├── InMemory/ # In-memory implementations (broker, schema registry, stores)
├── Interceptors/ # ClaimCheck, Schema, Tenant, W3C, Correlation interceptors
├── Pipeline/ # ManagedConsumer, StreamProcessor, backpressure, retry routing
├── Reactive/ # IReactiveEventStream, operators, bridge conversions
├── Replay/ # IStreamReplayer, ReplayRequest/ReplayResult
├── Sagas/ # SagaOrchestrator, SagaStep, ISagaStateStore
├── Sentinel/ # Schema linting (StreamSchemaLinter, TopicNamingPolicy)
├── Storage/ # File, SQLite, LiteDB storage providers
├── Telemetry/ # StreamTelemetryInstrumentation, IConsumerLagMonitor
├── Examples/ # 25+ working examples with config files
├── Docs/ # Markdown reference guides under Beep.StreamingEvents/
└── Help/ # HTML documentation site at repo root
Two documentation surfaces currently exist in the repository:
Help/contains the browsable HTML documentation site.Beep.StreamingEvents/Docs/contains the markdown reference guides.
The links below point to real files in both locations.
| Guide | Description |
|---|---|
| HTML documentation home | Browser-friendly entry point for the full Help site |
| Getting Started | Installation, single-node & cluster quick start |
| Runtime Bootstrap | Canonical host bootstrap flow across BeepStreaming, BeepStreamHostConfiguration, and BeepStreamRuntimeFactory |
| Configuration Reference | Host cascade, config responsibility matrix, and .properties key reference |
| Cluster Operations | Node management, rebalancing, failover |
| Storage Providers | Provider comparison, config, custom providers |
| Interceptors Guide | Claim-check, schema, tenant, composition |
| Pipeline Guide | ManagedConsumer, backpressure, outbox |
| Reactive Streaming | Operators, windowing, bridge conversions |
| Sagas Guide | Distributed saga orchestration |
| Replay Guide | Event replay and reprocessing |
| Telemetry Guide | Metrics, tracing, monitoring |
| Architecture Overview | Component map, data flow, extension points |
| Architecture Boundaries | Ownership rules for cluster, controller, node, and coordinator |
See Examples/README.md for the full example catalog covering:
- Basic node and cluster operations
- Broker adapters (gRPC, federated, edge, circuit breaker, tenant-aware)
- Storage providers (file, SQLite, LiteDB)
- Interceptor composition and pipeline patterns
- Reactive streaming and conversions
- Sagas, replay, telemetry, and schema linting
- Full platform example combining all subsystems
See LICENSE for details.