feat(search): two-panel layout, FTS performance, precomputed backlink counts#70
Merged
feat(search): two-panel layout, FTS performance, precomputed backlink counts#70
Conversation
… counts - Add two-panel desktop search dialog: 340dp result list + preview panel with 150ms debounce on selection change; mobile layout unchanged - PreviewPanelContent sealed class with PagePreview/BlockPreview/Empty states - PreviewPanel composable with skeleton shimmer while blocks load - RelativeDateFormatter and SearchSkeleton components - Replace O(N×M) backlink LIKE-scan in search with precomputed backlink_count column on pages; add pages_backlink_count schema migration with backfill - Add recomputeAllBacklinkCounts query called alongside rebuildFts - Real-time incremental backlink count updates: saveBlock, updateBlockContentOnly, deleteBlock, and deleteBulk now extract [[wikilinks]] from changed content and call recomputeBacklinkCountForPage for affected pages - FtsQueryBuilder improvements for better relevance ranking - SearchLatencyTest p99 < 200ms passes (was 603ms before precomputed column) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Kept backlink recompute methods alongside main's new git config writes in RestrictedDatabaseQueries. Added CancellationException re-throw from main into the pages FTS search catch chain in SqlDelightSearchRepository. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
JVM Load Benchmark (Desktop)Synthetic in-memory benchmark measuring load performance for the desktop (JVM) app.
Flamegraphs (this PR)**Allocation** — object allocation pressure (JDBC/SQLite churn)Alloc flamegraph not available CPU — method-level hotspots by on-CPU time CPU flamegraph not available Top SQL queries by total time (this PR)| table:operation | calls | p50 | p99 | max | total | |-----------------|-------|-----|-----|-----|-------| | `pages:select` | 2 | 1ms | 1ms | 2ms | 2ms |Top allocation hotspots (this PR)`72%` byte[]_[k] `3%` java.lang.String_[k] `2.3%` java.lang.StringBuilder_[k] `2.3%` java.lang.classfile.constantpool.PoolEntry[]_[k] `2.3%` int[]_[k]Top CPU hotspots (this PR)`99.1%` /usr/lib/x86_64-linux-gnu/libc.so.6 `0.1%` /tmp/sqlite-3.51.3.0-389fc977-795e-4ae6-aa36-696d2bf97674-libsqlitejdbc.so `0.1%` SR_handler `0.1%` sem_post `0.1%` pthread_cond_broadcast |
There was a problem hiding this comment.
Pull request overview
This PR enhances the search experience and reduces search-time database cost by adding a desktop two-panel search dialog (results + preview), introducing date-range FTS query variants, and moving backlink counting to a persisted pages.backlink_count column that’s refreshed via migrations and incremental updates.
Changes:
- Adds a desktop two-panel
SearchDialogwith selection-debounced preview loading and skeleton UI. - Adds date-range variants of page/block FTS queries and wires date filtering through
SearchRequest. - Introduces
pages.backlink_countwith migration backfill + repository hooks to recompute counts during writes and FTS rebuild.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| kmp/src/commonMain/sqldelight/dev/stapler/stelekit/db/SteleDatabase.sq | Adds backlink_count column and queries for batch read + recompute, plus date-range FTS queries. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/ui/screens/SearchViewModel.kt | Adds query prefix parsing, recent pages, richer result items, and preview selection handling. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/ui/components/SearchSkeleton.kt | New skeleton list composable for loading states. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/ui/components/SearchDialog.kt | Refactors search dialog UI into mobile/desktop branches and adds preview panel integration. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/ui/components/RelativeDateFormatter.kt | New helper for relative date labels in the result list. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/ui/components/PreviewPanel.kt | New preview panel composable that loads page blocks and renders a lightweight preview. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/ui/App.kt | Wires PageRepository into SearchViewModel and passes loadPageBlocks into SearchDialog. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/search/FtsQueryBuilder.kt | Expands stripped characters to include # for safer FTS query building. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/repository/SqlDelightSearchRepository.kt | Uses date-range SQL queries, reads precomputed backlink counts, and recomputes all counts on FTS rebuild. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/repository/SqlDelightBlockRepository.kt | Extracts wikilinks from changed block content and recomputes backlink counts after writes/deletes. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/repository/GraphRepository.kt | Extends SearchedPage to include backlinkCount. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/db/RestrictedDatabaseQueries.kt | Adds stubs for the new backlink recompute write queries. |
| kmp/src/commonMain/kotlin/dev/stapler/stelekit/db/MigrationRunner.kt | Adds migration to create/backfill pages.backlink_count. |
Comments suppressed due to low confidence (4)
kmp/src/commonMain/kotlin/dev/stapler/stelekit/repository/SqlDelightBlockRepository.kt:265
- Each
recomputeBacklinkCountForPagecall runs a fullblocks.content LIKE '%[[name]]%'scan. Calling it once per wikilink on every block save can make writes O(#links × #blocks). Consider batching (single SQL statement for all affected page names) or maintaining a link table / incremental delta updates to avoid repeated full scans.
block.parentUuid,
block.leftUuid,
block.content,
kmp/src/commonMain/kotlin/dev/stapler/stelekit/repository/SqlDelightBlockRepository.kt:280
changedPages.forEach { recomputeBacklinkCountForPage(it) }triggers a full blocks scan per affected page on every content edit. If a block contains many wikilinks this can make edits noticeably slower. Consider recomputing counts in a single batched query for the union set, or tracking per-block link deltas.
} catch (e: CancellationException) {
throw e
} catch (e: Exception) {
DomainError.DatabaseError.WriteFailed(e.message ?: "unknown").left()
kmp/src/commonMain/kotlin/dev/stapler/stelekit/repository/SqlDelightBlockRepository.kt:334
- Recomputing backlink counts via
wikilinkPages.forEach { recomputeBacklinkCountForPage(it) }can be very expensive when deleting a subtree with many links, since it performs one full blocks scan per page name. Consider recomputing in one statement for the affected page set (or defer to a background rebuild) to keep delete latency predictable.
}
// Chain repair before deletion — use firstOrNull because duplicate
kmp/src/commonMain/kotlin/dev/stapler/stelekit/repository/SqlDelightSearchRepository.kt:195
searchWithFiltersignoresSearchRequest.propertyFiltersentirely, so callers can supply tag/property filters but results won’t change. This makes the new query-prefix UI misleading. Implement filtering (SQL WHERE againstpages.properties/block properties, or post-filtering with a clear performance plan) or remove the parameter from the UI path until it’s supported.
emit(SearchResult(
blocks = emptyList(),
pages = emptyList(),
totalCount = 0,
hasMore = false
).right())
return@flow
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| private val hierarchyTtlMs = 120_000L // 2 minutes | ||
|
|
||
| private fun extractWikilinks(content: String): Set<String> = | ||
| WIKILINK_REGEX.findAll(content).map { it.groupValues[1].trim() }.toHashSet() |
Comment on lines
+374
to
+377
| children.forEach { child -> | ||
| uuidsToDelete.add(child.uuid) | ||
| wikilinkPages.addAll(extractWikilinks(child.content)) | ||
| } |
| import androidx.compose.foundation.clickable | ||
| import androidx.compose.foundation.layout.* | ||
| import androidx.compose.foundation.lazy.LazyColumn | ||
| import androidx.compose.foundation.lazy.items |
| import dev.stapler.stelekit.model.Block | ||
| import dev.stapler.stelekit.repository.SearchScope | ||
| import dev.stapler.stelekit.ui.screens.ParsedQuery | ||
| import dev.stapler.stelekit.ui.screens.PreviewPanelContent |
Comment on lines
178
to
193
|
|
||
| try { | ||
| // Merge tag filter into property filters if present | ||
| val effectivePropertyFilters = if (parsed.tagFilter != null) { | ||
| parsed.propertyFilters + mapOf("tags" to parsed.tagFilter) | ||
| } else { | ||
| parsed.propertyFilters | ||
| } | ||
|
|
||
| val request = SearchRequest( | ||
| query = query, | ||
| scope = _uiState.value.scope, | ||
| query = parsed.ftsTerm.takeIf { it.isNotBlank() } ?: query, | ||
| scope = parsed.scopeOverride ?: _uiState.value.scope, | ||
| dataTypes = setOf(DataType.TITLES, DataType.CONTENT), | ||
| propertyFilters = effectivePropertyFilters, | ||
| dateRange = parsed.dateRange, | ||
| limit = 20 |
Comment on lines
+668
to
+672
| recomputeBacklinkCountForPage: | ||
| UPDATE pages SET backlink_count = ( | ||
| SELECT COUNT(*) FROM blocks | ||
| WHERE blocks.content LIKE '%[[' || pages.name || ']]%' | ||
| ) WHERE pages.name = ?; |
Comment on lines
75
to
80
| LaunchedEffect(selectedIndex) { | ||
| if (uiState.results.isNotEmpty()) { | ||
| if (activeList.isNotEmpty()) { | ||
| listState.animateScrollToItem(selectedIndex) | ||
| } | ||
| viewModel.onSelectionChange(selectedIndex) | ||
| } |
| days in 2..6 -> "$days days ago" | ||
| days in 7..30 -> { | ||
| val localDate = instant.toLocalDateTime(TimeZone.currentSystemDefault()).date | ||
| "${localDate.month.name.lowercase().replaceFirstChar { it.uppercase() }.take(3)} ${localDate.day}" |
Contributor
Android Load BenchmarkInstrumented benchmark on an API 30 x86_64 emulator — 500-page synthetic graph. Comparing Graph Load
Interactive Write Latency (during Phase 3)
SAF I/O Overhead (ContentProvider vs direct File read)Measures Binder IPC cost added by ContentResolver per readFile() call.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LEFT JOIN blocks ON LIKEbacklink scan with a precomputedbacklink_countcolumn onpages;SearchLatencyTestp99 drops from 603ms → passes <200mssaveBlock,updateBlockContentOnly,deleteBlock, anddeleteBulknow extract[[wikilinks]]from changed content and callrecomputeBacklinkCountForPageincrementallyChanges
Search UX
PreviewPanelContentsealed class (PagePreview/BlockPreview/Empty)PreviewPanelcomposable with skeleton shimmer while blocks loadSearchDialogdesktop branch:Row { Column(340dp) + VerticalDivider + PreviewPanel }, dialog resized to 85%×75%RelativeDateFormatterandSearchSkeletoncomponents extractedDatabase / Performance
pages.backlink_count INTEGER NOT NULL DEFAULT 0added to schemapages_backlink_countmigration with backfill runs on first startup against existing DBsrecomputeAllBacklinkCountscalled alongsiderebuildFtsfor bulk-import accuracyrecomputeBacklinkCountForPage(name)for single-page incremental updatesRestrictedDatabaseQueriesstubs for both new write queriesTest plan
SearchLatencyTest—p99Under200msandcoldStartFtsQuery_under500msboth pass[[PageName]]— verifybacklink_countfor that page updates in All Pages view🤖 Generated with Claude Code