fix: bound rayon parallelism in EditDelta::layout_delta to reduce peak memory (APP-4686)#12312
Draft
warp-dev-github-integration[bot] wants to merge 1 commit into
Draft
Conversation
…k memory (APP-4686) Previously, EditDelta::layout_delta ran all text layout tasks simultaneously via into_par_iter(), which could cause 9+ GB memory spikes on large documents from concurrent CoreText layout allocations across rayon worker threads. This change processes layout tasks in sequential chunks of MAX_PARALLEL_LAYOUT_TASKS (8 tasks), where each chunk runs in parallel. This bounds peak concurrent memory to 8 text layout operations at once while preserving parallelism benefits for each chunk. The same fix is applied to layout_temporary_blocks. The now-unused parallel_util module (Last<T>) has been removed. Sentry: https://sentry.io/organizations/warpdotdev/issues/7259255054/ Co-Authored-By: Oz <oz-agent@warp.dev>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Bounds rayon parallelism in
EditDelta::layout_deltaandlayout_temporary_blocksto reduce peak memory usage on large document edits.Root cause: Both functions called
into_par_iter()without any concurrency limit, running all text layout tasks simultaneously. On large documents (many blocks), this created 9+ GB peak memory usage from concurrent CoreTextGlyphandCaretPositionVec allocations across all rayon worker threads.Fix: Pre-annotate each task with its
BlockLocation(which requires the global index), then process tasks in sequential parallel chunks ofMAX_PARALLEL_LAYOUT_TASKS(8). Each chunk runs in parallel, bounding peak concurrent CoreText allocations to 8 tasks at a time.Also removes the now-unused
parallel_utilmodule (Last<T>), which was only ever used by the originalunzip()pattern in these two functions.Linear: APP-4686
Linked Issue
ready-to-specorready-to-implement.Testing
./script/runAutomated tests: All 426 existing
warp_editortests pass.cargo check,cargo clippy, and./script/formatall pass cleanly.Agent Mode
Conversation: https://staging.warp.dev/conversation/e0a46d55-49e1-466b-ba50-bc291a24e1f1
Run: https://oz.staging.warp.dev/runs/019e9e88-6615-78c8-b94e-1241b6c69102
This PR was generated with Oz.