Introduce new microbenchmarks to compare perf of direct access API with SQL path#3996
Open
arnaud-lacurie wants to merge 5 commits intoFoundationDB:mainfrom
Open
Introduce new microbenchmarks to compare perf of direct access API with SQL path#3996arnaud-lacurie wants to merge 5 commits intoFoundationDB:mainfrom
arnaud-lacurie wants to merge 5 commits intoFoundationDB:mainfrom
Conversation
- Remove references to the deleted DatabaseTemplate class in BasicBenchmark, CreateDatabaseBenchmark, ManyDatabasesBenchmark, and SimplePlanCachingBenchmark; replace with the existing createDatabase/createMultipleDatabases overloads that accept a template name and schema list directly. - Drop the removed setFormatVersion(1) call in RecordLayerScanBenchmark. - Make schema template creation idempotent in EmbeddedRelationalBenchmark.Driver by issuing DROP SCHEMA TEMPLATE before CREATE SCHEMA TEMPLATE. - Remove the createDatabase(DatabaseTemplate, String) helper that wrapped the deleted API; update createMultipleDatabases to accept template name and schema varargs instead. - Wire up the JMH Gradle plugin (alias(libs.plugins.jmh)) in fdb-relational-core.gradle with resultFormat = 'text' and the processJmhResources duplicate-strategy fix so the benchmarks can be compiled and invoked via ./gradlew :fdb-relational-core:jmh.
Two new JMH benchmarks in fdb-relational-core compare the Direct Access API
(executeGet / executeScan) against equivalent SQL queries with a warm plan cache,
quantifying the pure SQL-layer overhead on top of FDB I/O:
DirectAccessVsQueryBenchmark — three access patterns on a two-column composite
PK table (BenchTable: group_id BIGINT, row_id BIGINT, val STRING):
- get: single-row PK lookup (executeGet vs SELECT * WHERE pk = ?)
- scanPrefix: partial-PK range scan (executeScan vs SELECT * WHERE group_id = ?)
- fullScan: full table scan (executeScan with empty KeySet vs SELECT * FROM ...)
IndexScanVsQueryBenchmark — two secondary-index access patterns on a table with
idx_label (unique) and idx_category (non-unique):
- lookupByLabel: equality via unique index (executeScan + INDEX_HINT vs SELECT)
- scanByCategory: prefix via non-unique index (executeScan + INDEX_HINT vs SELECT)
Both benchmarks are parameterised over rowsPerGroup ∈ {10, 100}, run with
16 threads, 1 warmup iteration × 10 s, 5 measurement iterations × 10 s, and
use RelationalPlanCache.buildWithDefaults() so the plan cache is warm on every
measured iteration.
Three supporting test classes are added:
- BenchTableExplainTest: verifies EXPLAIN plans for DirectAccessVsQuery queries
- IndexBenchExplainTest: verifies EXPLAIN plans for IndexScanVsQuery queries
- IndexScanDirectAccessTest: integration tests for the index direct-access paths
The jmh block in fdb-relational-core.gradle is updated to set
includes = ['(DirectAccessVsQueryBenchmark|IndexScanVsQueryBenchmark)'] so that
./gradlew :fdb-relational-core:jmh runs exactly these two benchmarks.
Benchmark results (avgt ms/op, warm plan cache, 16 threads):
DirectAccessVsQueryBenchmark:
get rows=10: direct 2.81 / SQL 3.23 (+0.42 ms) rows=100: direct 2.87 / SQL 3.19 (+0.32 ms)
scanPrefix rows=10: direct 3.14 / SQL 3.43 (+0.29 ms) rows=100: direct 6.54 / SQL 6.92 (+0.38 ms)
fullScan rows=10: direct 2.93 / SQL 4.94 (+2.01 ms) rows=100: direct 12.20 / SQL 12.79 (+0.59 ms)
IndexScanVsQueryBenchmark:
lookupByLabel rows=10: direct 2.87 / SQL 3.32 (+0.45 ms) rows=100: direct 2.78 / SQL 3.21 (+0.43 ms)
scanByCategory rows=10: direct 3.13 / SQL 3.33 (+0.21 ms) rows=100: direct 6.05 / SQL 7.07 (+1.03 ms)
Key findings: with a warm cache, SQL overhead is 0.3–1.0 ms per call; FDB
network I/O dominates both paths. Index queries use COVERING(idx_X [EQUALS ?])
with no Fetch wrapper (correct index-only plans).
…ional/recordlayer/query/IndexScanDirectAccessTest.java
…ional/recordlayer/query/IndexBenchExplainTest.java
…ional/recordlayer/query/BenchTableExplainTest.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.