Skip to content

perf: speed up msgpack_extract/lookup on the map hot path - #12

Merged
khanaffan merged 1 commit into
mainfrom
perf/faster-extract-lookup
Jul 1, 2026
Merged

perf: speed up msgpack_extract/lookup on the map hot path#12
khanaffan merged 1 commit into
mainfrom
perf/faster-extract-lookup

Conversation

@khanaffan

Copy link
Copy Markdown
Owner

Summary

msgpack_extract() (and everything built on mpLookup: WHERE, index expressions) was slower than json_extract/jsonb_extract for text and blob values, which dragged mixed-type full-row reads below jsonb. Two causes on the map traversal hot path:

  1. mpLookup skipped over each map key with a full mpSkipOne() call — even though the key header (type + length) had just been decoded in the same loop iteration. The value offset can be computed directly.
  2. mpSkipOneD tested fixstr last (after the entire switch plus two more branches), yet fixstr is the most common element type — every map key, plus short strings.

Change

  • Derive the value offset directly from the decoded key header, removing the redundant per-key mpSkipOne().
  • Check fixstr before the switch.

Output is byte/value-identical — this only makes the traversal cheaper. Non-string keys still fall back to mpSkipOne().

Results

Extracting every property of a 256-key map (µs per row):

value type before after vs jsonb
int 176 131 jsonb 253
real 213 161 jsonb 266
text 223 180 jsonb 174 (was 12% slower → parity)
blob 198 150 jsonb 177 (was slower → now faster)

19–26% faster extraction; msgpack_extract is now faster-or-equal to the SQLite JSON functions across all value types. The gain holds at larger maps (P=512 text: 836 → 630 µs/row).

Validation

  • 40,000 randomized msgpack_extract / msgpack_to_json cases — results identical to the previous implementation (nested maps, bins, nulls, absent keys, all scalar types).
  • Full ctest suite (16/16), including msgpack_spec_p4 (extract) and the fuzz corpus. No new warnings.

Independent of #11 (that PR targets the mutation functions); this touches only mpLookup / mpSkipOneD.

mpLookup() skipped over each map key with a full mpSkipOne() even though the
key header was already decoded, and mpSkipOneD() tested fixstr — the most
common element type (every map key plus short strings) — only after the whole
switch and two further branches.

Derive the value offset directly from the decoded key header (dropping the
redundant per-key skip) and check fixstr before the switch. Output is
unchanged; only the traversal is cheaper.

Extracting every property of a 256-key map (per-row, us):
  int  176 -> 131   real 213 -> 161   text 223 -> 180   blob 198 -> 150
msgpack_extract is now faster-or-equal to json_extract/jsonb_extract across
all value types (text was ~12% slower and blob slower; both are now
parity/faster), a 19-26% improvement.

Validated with 40k randomized extract/msgpack_to_json cases (identical to the
prior implementation) and the full ctest suite (spec p4 extract + fuzz corpus).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@khanaffan
khanaffan force-pushed the perf/faster-extract-lookup branch from 257fd7b to 0bb0fe4 Compare July 1, 2026 20:04
@khanaffan
khanaffan merged commit ef18091 into main Jul 1, 2026
6 checks passed
@khanaffan
khanaffan deleted the perf/faster-extract-lookup branch July 1, 2026 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant