Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions sql/05_tiering.sql
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ DECLARE
v_oid OID;
v_lsn PG_LSN;
BEGIN
IF to_regclass('wal2delta.tables') IS NULL THEN
RETURN NULL;
END IF;
v_oid := to_regclass('lakets_cdf.' || quote_ident(p_shadow_name))::oid;
IF v_oid IS NULL THEN
RETURN NULL;
Expand All @@ -32,6 +29,13 @@ BEGIN
FROM wal2delta.tables
WHERE table_oid = v_oid AND status = 'STREAMING';
RETURN v_lsn; -- NULL if no STREAMING row
EXCEPTION
-- wal2delta is a Lakebase-managed schema. If the subsystem is absent
-- (undefined schema/table) or this role lacks USAGE/SELECT on it
-- (insufficient_privilege), we cannot confirm durability -- fail closed
-- so tiering never evicts data we could not verify in Unity Catalog.
WHEN insufficient_privilege OR undefined_table OR invalid_schema_name THEN
RETURN NULL;
END;
$$;

Expand Down
19 changes: 14 additions & 5 deletions sql/13_shadow_sync.sql
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ AS $$
DECLARE
v_is_ct BOOLEAN;
v_is_ru BOOLEAN;
v_cdf_present BOOLEAN;
BEGIN
SELECT EXISTS(SELECT 1 FROM lakets._chronotable_registry
WHERE schema_name = p_schema_name AND table_name = p_table_name) INTO v_is_ct;
Expand All @@ -130,11 +131,19 @@ BEGIN
-- enabled on the lakets_cdf schema via Databricks. Without it, the shadow is
-- created but never streams to Unity Catalog, and tiering stays fail-closed
-- (nothing is ever evicted). Warn loudly rather than failing, so sync can be
-- wired before CDF is turned on.
IF to_regclass('wal2delta.tables') IS NULL THEN
RAISE WARNING 'Lakebase CDF (wal2delta) is not enabled on this database. The '
'shadow will be created but will not replicate to Unity Catalog, and '
'tiering will not evict any partitions. Enable CDF on the lakets_cdf '
-- wired before CDF is turned on. wal2delta is a Lakebase-managed schema; if
-- this role lacks USAGE on it, probing raises insufficient_privilege -- treat
-- that the same as "not visible" so enable_sync (which does not otherwise
-- touch wal2delta) still completes.
BEGIN
v_cdf_present := to_regclass('wal2delta.tables') IS NOT NULL;
EXCEPTION WHEN insufficient_privilege THEN
v_cdf_present := FALSE;
END;
IF NOT v_cdf_present THEN
RAISE WARNING 'Lakebase CDF (wal2delta) is not enabled or not visible to this '
'role. The shadow will be created but will not replicate to Unity Catalog, '
'and tiering will not evict any partitions. Enable CDF on the lakets_cdf '
'schema (see the LakeTS prerequisites) to activate sync.';
END IF;

Expand Down
13 changes: 13 additions & 0 deletions website/docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,19 @@ Lakebase CDF is schema-sensitive. After an `ALTER TABLE`:

You enabled sync on the ChronoTable itself instead of the shadow table. Make sure `_chronotable_registry.sync_enabled = true` and the registry's `shadow_table_name` is what's configured in the Databricks sync — not the partitioned parent.

### `ERROR: permission denied for schema wal2delta`

`wal2delta` is the Lakebase CDF subsystem's own schema. `enable_sync` and the tiering durability gate probe it only to check whether CDF is active; they never write to it. If your role lacks `USAGE` on `wal2delta`, the probe is tolerated — `enable_sync` completes (creating the shadow and trigger as usual) and the tiering gate fails closed (it evicts nothing it cannot verify).

If you need CDF-gated tiering to actually evict partitions, grant the role read access to the gate it depends on:

```sql
GRANT USAGE ON SCHEMA wal2delta TO "<your-role>";
GRANT SELECT ON wal2delta.tables TO "<your-role>";
```

Run these as the `wal2delta` owner or a Lakebase admin. Until then, tiering stays fail-closed by design.

## Performance

### Inserts slow down over time
Expand Down