Skip to content

Reserve vector capacity when populating vectors in conversion operators#7095

Merged
bdash merged 1 commit into
devfrom
vector-reserve
Jul 15, 2025
Merged

Reserve vector capacity when populating vectors in conversion operators#7095
bdash merged 1 commit into
devfrom
vector-reserve

Conversation

@bdash
Copy link
Copy Markdown
Contributor

@bdash bdash commented Jul 15, 2025

I noticed while looking at some performance profiles today that some of these conversion operators are used on relatively hot codepaths. Reserving capacity before populating them avoids reallocations as they grow and is free performance.

Copy link
Copy Markdown
Member

@plafosse plafosse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very surprising all these were missing. This change should also reduce memory fragmentation which has been a huge cause of memory usage.

@bdash bdash merged commit 22c2873 into dev Jul 15, 2025
1 check passed
@pgoodman
Copy link
Copy Markdown

Related to this change, here is a performance profiling thought I once had:

  • Get a baseline profile
  • Change the resize policy of vectors to always increase/shrink by exactly the requested size. E.g. push_back allocates a new backing array, one larger than the old array, and copies/moves the data. The same for pop_back.
  • Change the reserve policy to be a no-op.
  • Get a new profile, with the presumably outsized vector impacts
  • Compare profiles to find outsized impacts of vector resizing policies
  • Hand-wavy: causal profiling to find the worst-offending vectors

Second profiling ideas:

  • For all stack allocated/filled vectors, find the maximum observed size across many test cases
  • If the vectors are in functions with loops, then plot the maximum observed loop trip counts
  • Figure out reserve policies based on predictability of maximum vector sizes.

@bdash bdash deleted the vector-reserve branch July 15, 2025 18:12
@emesare emesare added this to the Helion milestone Jul 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants