Skip to content

perf(matchImei): replace O(N*M) nested ResultSet scan with HashMap lookup#99

Open
sauravbhattacharya001 wants to merge 1 commit intomasterfrom
perf/hashmap-device-lookup
Open

perf(matchImei): replace O(N*M) nested ResultSet scan with HashMap lookup#99
sauravbhattacharya001 wants to merge 1 commit intomasterfrom
perf/hashmap-device-lookup

Conversation

@sauravbhattacharya001
Copy link
Owner

Summary

Replace the nested-loop device matching in \matchImei.java\ with an in-memory HashMap for O(1) lookups.

Before

  • For each event row, the code rewound
    s_device\ to the first row and scanned linearly to find a match
  • Complexity: O(events × devices) per pass (sndrnode + srcnode)
  • Each matched row triggered an individual \�xecuteUpdate()\

After

  • Device table loaded once into a \HashMap<String, String>\ — O(devices)
  • Each event row does a single \HashMap.get()\ — O(1)
  • Overall: O(events + devices) per pass
  • Batch updates (flush every 500 rows) reduce DB round-trips
  • Transaction wrapping with rollback on failure
  • Extracted \matchAndUpdate()\ to eliminate duplicated sndrnode/srcnode logic
  • Removed \SCROLL_INSENSITIVE\ cursors (no longer needed without
    s.first()\ rewind)

…okup

Replace the nested-loop device matching (O(events × devices) per pass)
with an in-memory HashMap for O(1) lookups per event row, reducing
overall complexity to O(events + devices).

Additional improvements:
- Batch UPDATE statements (flush every 500) for fewer DB round-trips
- Wrap both passes in a transaction with rollback on failure
- Extract reusable matchAndUpdate() method to eliminate code duplication
- Remove SCROLL_INSENSITIVE cursors (no longer needed without rs.first() rewind)
@chatgpt-codex-connector
Copy link

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions github-actions bot added data-pipeline Meeting extraction and processing pipeline size/m labels Mar 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data-pipeline Meeting extraction and processing pipeline size/m

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant