Potential: eBPF + wirelog for kernel-level Datalog inference

[justinjoy]

Idea

Use wirelog as a userspace Datalog inference engine paired with eBPF for kernel event collection. The combination enables declarative, real-time analysis of kernel events (packets, syscalls, tracepoints) using Datalog rules — without kernel modification.

Architecture

[Kernel]                          [Userspace]
eBPF programs                     wirelog (203KB SO)
  - tracepoints                     - Datalog rules
  - kprobes                         - incremental evaluation
  - XDP/TC hooks                    - multi-worker TDD
       |                                 |
       +--- eBPF maps (ringbuf) ---------+
            (facts: packet, syscall)   (insert -> step -> query)

1. eBPF collects events in kernel, writes structured facts to eBPF maps
2. wirelog in userspace reads facts via wl_session_insert, runs incremental evaluation
3. Results drive policy decisions, alerts, or feedback to eBPF programs via maps

Why wirelog fits

Property	Value	Why it matters
SO size	203KB	Embeddable in lightweight eBPF agents
Total runtime	< 1MB	Fits resource-constrained edge/IoT
Incremental eval	ms-level	Real-time response to kernel events
Multi-worker TDD	CRDT 2.97x W=4	Scales on multi-core
C11, no runtime	No GC, no VM	Minimal footprint, predictable latency
LGPL-3.0	Compatible with eBPF tooling	Can link as shared library

Use cases

1. Network policy engine

.decl packet(src: int32, dst: int32, port: int32, proto: int32)
.decl allowed(src: int32, dst: int32)
allowed(s, d) :- packet(s, d, 443, 6), trusted_src(s).
blocked(s, d) :- packet(s, d, _, _), !allowed(s, d).

eBPF XDP captures packets -> wirelog evaluates policy -> result fed back via eBPF map.

2. Intrusion detection

.decl syscall(pid: int32, nr: int32, ts: int32)
suspicious(p) :- syscall(p, 59, _), syscall(p, 2, _), syscall(p, 41, _).

eBPF tracepoint on sys_enter -> wirelog correlates multi-syscall patterns.

3. Data provenance tracking

flow(a, b) :- fileop(p, _, a, 0), fileop(p, _, b, 1).
flow(a, c) :- flow(a, b), flow(b, c).

Transitive data flow across processes via file operations.

Comparison

Tool	Binary size	Approach
wirelog + eBPF	< 1MB	Datalog, incremental, C11
Falco	~50MB	YAML rules, C++
Cilium	~100MB+	Go policy engine
OPA/Rego	~30MB	Rego, Go
DDlog	~50MB+	Datalog, Rust

Open questions

• Optimal eBPF map type for fact streaming? (ring buffer vs per-CPU array)
• Fact retraction for time-windowed analysis?
• Batch insert API for eBPF ring buffer consumption?
• Events/sec throughput for incremental insert -> step -> query?

---

Potential idea for future exploration, not a committed roadmap item.

[justinjoy]

Open Design Question: I/O Boundary — What should wirelog handle natively vs delegate to users?

This idea raises a fundamental architectural question about wirelog's I/O boundary.

The spectrum

[Pure inference engine]  <------>  [Full I/O-integrated system]
     wirelog today                    wirelog + native packet/eBPF I/O

Option A: Pure inference engine (current)

• wirelog only does: parse -> plan -> evaluate -> query
• All I/O is external: user code reads packets/files/eBPF maps, calls wl_session_insert(), reads results via wl_session_snapshot()
• Pro: minimal surface area, maximum embedding flexibility, < 1MB
• Con: every integration requires custom glue code

Option B: Built-in packet/network I/O

• wirelog natively reads pcap, raw sockets, or eBPF ring buffers
• .input packet(source="ebpf://tracepoint/net/netif_receive_skb")
• Pro: zero-glue-code deployment for network analysis
• Con: increases binary size, platform-specific code, maintenance burden

Option C: User-defined I/O adapters (plugin model)

• wirelog defines an I/O adapter interface:

  typedef struct {
      int (*open)(const char *uri, void *ctx);
      int (*read_batch)(void *ctx, int64_t *buf, uint32_t *nrows, uint32_t ncols);
      void (*close)(void *ctx);
  } wl_io_adapter_t;
  
  int wl_register_io_adapter(const char *scheme, const wl_io_adapter_t *adapter);

• Users register adapters for ebpf://, pcap://, kafka://, csv://, etc.
• .input packet(source="ebpf://ringbuf/events") dispatches to the registered adapter
• Pro: extensible without bloating core, keeps core < 1MB, users bring their own I/O
• Con: API design complexity, adapter lifecycle management

Current thinking: Option C (plugin model)

1. Core stays minimal: wirelog SO remains 203KB, no platform-specific I/O code
2. .input directive becomes extensible: source="scheme://path" dispatches to registered adapter
3. Built-in adapters ship separately: libwirelog-io-ebpf.so (~50KB), libwirelog-io-csv.so (~10KB), etc.
4. Users can write custom adapters: for proprietary protocols, custom hardware, etc.

Specific concerns for eBPF integration

1. Batch ingestion

eBPF ring buffers produce events in bursts. Current wl_session_insert is per-relation. A batch API is needed:

wl_batch_t *batch = wl_batch_create(session);
wl_batch_insert(batch, "packet", row1, ncols);
wl_batch_insert(batch, "syscall", row2, ncols);
wl_batch_commit(batch);  // single step() call

2. Streaming vs snapshot

Current API is snapshot-based (insert -> step -> snapshot). For real-time eBPF streams, a streaming mode with automatic periodic evaluation:

wl_session_set_streaming(session, interval_ms, callback);
// internally: accumulate inserts, auto-step every interval_ms, invoke callback with new results

3. Fact retraction for time windows

eBPF events are temporal. Old events should expire. wirelog supports wl_session_remove() but there is no built-in TTL or sliding window. Should this be part of the adapter (manages expiry) or the core engine (.decl packet(...) @ttl(60s))?

4. Back-pressure

If wirelog evaluation is slower than eBPF event production, the adapter must handle ring buffer overflow. Should wirelog provide a back-pressure signal to the adapter?

5. Schema mapping

eBPF events have varying schemas (different tracepoints produce different fields). How does the adapter map raw eBPF struct fields to wirelog relation columns? Static mapping at registration time, or dynamic via a schema callback?

Questions for discussion

• Is Option C the right level of abstraction, or over-engineering for the current stage?
• Should the core provide CSV/file I/O natively (Option A+), and only use plugins for exotic sources?
• For eBPF specifically: is the adapter model sufficient, or does the tight kernel-userspace coupling require deeper integration?
• How important is sub-millisecond latency for the eBPF use case?