You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
wirelog's col_rel_t currently uses a single-ownership model: each relation is either pool-owned or heap-allocated, and col_rel_destroy unconditionally frees data, merge_buf, and timestamps. Ownership transfer is tracked via the owned flag in eval_stack_push.
As the system grows more complex (materialization cache sharing, K-fusion worker copies, incremental view maintenance), the question arises whether a reference-counted lifetime model (similar to GLib's GObject with g_object_ref/unref) would be more maintainable and less error-prone.
Current pain points
Materialization cache: col_mat_cache_insert takes ownership of the relation, then a borrowed (non-owned) reference is pushed to the eval stack. This implicit shared reference is not tracked — if the cache evicts the entry while a stack reference is live, we get a use-after-free.
K-fusion COW: Workers currently copy entire relations to avoid shared mutation. With refcounting, read-only sharing would be zero-copy until a write triggers a copy (true COW).
Pointer swap (issue DOOP benchmark OOM: unbounded memory growth in recursive evaluation loop #218): The recent merge_buf/data swap optimization works because ownership is strictly 1:1. If multiple consumers held references to rel->data, the swap would invalidate them. Refcounting would formalize this invariant.
Arguments for refcounting
Eliminates implicit sharing bugs: Every shared reference is explicitly tracked.
Enables zero-copy sharing: K-fusion workers, cache consumers, and delta subscribers can share relation data without copying.
GLib precedent: Well-understood pattern in C ecosystems. g_object_ref/unref with release callback is battle-tested.
Arguments against (for now)
C11 manual management: No destructor or RAII — every ref must have a matching unref. Missed unref = memory leak; double unref = UAF. Current single-ownership is simpler and safer.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Context
wirelog's
col_rel_tcurrently uses a single-ownership model: each relation is either pool-owned or heap-allocated, andcol_rel_destroyunconditionally freesdata,merge_buf, andtimestamps. Ownership transfer is tracked via theownedflag ineval_stack_push.As the system grows more complex (materialization cache sharing, K-fusion worker copies, incremental view maintenance), the question arises whether a reference-counted lifetime model (similar to GLib's
GObjectwithg_object_ref/unref) would be more maintainable and less error-prone.Current pain points
Materialization cache:
col_mat_cache_inserttakes ownership of the relation, then a borrowed (non-owned) reference is pushed to the eval stack. This implicit shared reference is not tracked — if the cache evicts the entry while a stack reference is live, we get a use-after-free.K-fusion COW: Workers currently copy entire relations to avoid shared mutation. With refcounting, read-only sharing would be zero-copy until a write triggers a copy (true COW).
Pointer swap (issue DOOP benchmark OOM: unbounded memory growth in recursive evaluation loop #218): The recent merge_buf/data swap optimization works because ownership is strictly 1:1. If multiple consumers held references to
rel->data, the swap would invalidate them. Refcounting would formalize this invariant.Arguments for refcounting
g_object_ref/unrefwith release callback is battle-tested.Arguments against (for now)
refmust have a matchingunref. Missedunref= memory leak; doubleunref= UAF. Current single-ownership is simpler and safer._Atomic uint32_t) for K-fusion workers. Adds contention on hot paths.col_rel_destroycall site needs audit; everyeval_stack_push(rel, owned)becomescol_rel_ref(rel)/col_rel_unref(rel).Questions for discussion
col_rel_t, or also to arrangements and cached state?Related: #218 (DOOP OOM), materialization cache in
ops.cAll reactions