redo: redo apply may OOM when applying a large DML-only range

[3AceShowHand]

What did you do?

Run cdc redo apply on a redo log range that contains a large amount of DML. The risk is higher when:

• the gap between checkpointTs and resolvedTs is large;
• the upstream workload has high write throughput or wide rows;
• the range contains no DDL, or DDL events are infrequent;
• the redo apply process has a constrained memory limit.

The buffering model was introduced by redo: rewrite redo apply (#2167).

Before that rewrite, redo apply used a 1 GiB MemQuota and BlockAcquire to apply backpressure while rows were being sent to table sinks. The rewrite removed this quota path and introduced a per-table eventsGroup.

The current flow is:

1. consumeLogs reads rows continuously.
2. applyRow converts each redo row to a DMLEvent and appends it to the table's eventsGroup.
3. The buffered events are passed to the MySQL sink only when a relevant DDL is processed or after the reader reaches the end of the redo range.

Relevant code:

• The reader flushes all tables after reaching the end of the range
• applyRow appends every converted row to eventsGroup
• waitTableFlush sends the accumulated events to the MySQL sink
• eventsGroup has no byte limit or intermediate flush threshold
• ToDMLEvent allocates table metadata and a row chunk for each event

What did you expect to see?

Redo apply should keep memory usage bounded while processing a large redo range. It should flush DML in bounded batches or apply backpressure based on the buffered bytes, while preserving transaction ordering and DDL boundaries.

What did you see instead?

For a DML-only range, redo apply retains the converted DML events until all row logs have been read. Memory usage therefore grows with the total DML volume between checkpointTs and resolvedTs, rather than with a bounded in-flight batch.

Merging rows with the same commit timestamp reduces the number of DMLEvent objects, but the merged row chunks and metadata remain live. With a sufficiently large redo range, the process may exhaust the container memory and be terminated by OOM.

The --memory-limit option sets Go's soft memory limit. It does not provide admission control for these live events and cannot reclaim them while they remain referenced by eventsGroup.

Versions of the cluster

Upstream TiDB cluster version:

N/A

Upstream TiKV version:

N/A

TiCDC version:

master