⚡ Bolt: [performance improvement] Optimize edge iteration in graph algorithms#98
⚡ Bolt: [performance improvement] Optimize edge iteration in graph algorithms#98saurabhsharma2u wants to merge 1 commit intomainfrom
Conversation
…gorithms Implemented an inline iterator pattern `forEachEdge` on the `Graph` class and refactored hot paths (`analyze`, `crawler`, `hits`, `pagerank`, `metricsRunner`, `duplicate`) to utilize it instead of `getEdges()`. This avoids massive object allocation and mapping of strings for every composite edge key, drastically reducing GC pauses in highly connected graphs. Verified via test suite and manually benchmarking script showing a ~90% speedup on raw edge traversal for ~500k edges. Added critical learning insight to `.jules/bolt.md`.
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
1 participant
💡 What: Added a memory-efficient
forEachEdgeiterator to theGraphclass that parses edge keys inline without creating intermediate objects or allocating the massive array thatgetEdges()generates. Refactored high-frequency call sites across thecorepackage (analyze,duplicate,crawler,metricsRunner,hits,pagerank) to use this new iterator.🎯 Why: In large site graphs with hundreds of thousands of edges, calling
graph.getEdges()inside algorithm loops allocates a large flat array and spawns a newGraphEdgeobject for every single edge (involving a.indexOfsplit on string keys). This adds intense, unnecessary garbage collection pressure and blocks the event loop.📊 Impact: Reduces edge iteration overhead by ~90% (from ~425ms to ~46ms on a 500k graph test). This substantially speeds up post-crawl analysis steps (PageRank, HITS, Orphaning) on highly linked domains.
🔬 Measurement: Added a script
test_perf.jsthat populated 500,000 edges and measuredgetEdges()(array allocation) vsforEachEdge(inline split pattern).forEachEdgeexecuted nearly 10 times faster with zero memory overhead. Tests pass successfully.PR created automatically by Jules for task 12205524290626001187 started by @saurabhsharma2u