Skip to content

Add exploration-only analysis for full-table caching and index it in analysis docs#494

Draft
Copilot wants to merge 2 commits intomainfrom
copilot/deep-analysis-full-table-cache
Draft

Add exploration-only analysis for full-table caching and index it in analysis docs#494
Copilot wants to merge 2 commits intomainfrom
copilot/deep-analysis-full-table-cache

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 7, 2026

  • Review the new PR comment and confirm required action
  • Re-check current analysis docs content and index entries
  • Add concise H2 in-memory table caching notes (pros, concerns, guardrails) to analysis doc
  • Update analysis README index timestamp/entry only if needed
  • Run targeted validation for documentation change
  • Reply to the PR comment with commit hash

@rrobetti
Copy link
Copy Markdown
Contributor

rrobetti commented May 7, 2026

@copilot What if we load the table to be cached or a subset of a table into a H2 table in memory, the application would access that table via a different datasource and would know that the information on that table might be stale, the table (H2) would be refreshed X amount of time, like our current cache. How would that play out? This would allow the application to manage these H2 tables in whichever way they want, allowing for inserts updates deletes etc, it would be more flexible, also supporting joins and complex queries if required.
what are your opinions, suggestions, concerns? just respond, do not change files yet

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 7, 2026

@copilot What if we load the table to be cached or a subset of a table into a H2 table in memory, the application would access that table via a different datasource and would know that the information...

This can work, and it’s a reasonable exploration path. My view: use it as a read-optimized mirror (not a writable source of truth).

Pros: very flexible querying (joins/complex SQL), app-controlled refresh cadence, and offload from primary DB for hot reference reads.

Main concerns:

  1. Consistency drift (stale data is expected, but must be bounded/observable),
  2. Dual-write risk if apps do INSERT/UPDATE/DELETE in H2 and assume persistence,
  3. Transaction/isolation mismatch vs primary DB,
  4. Per-instance divergence in multinode apps,
  5. Memory + warmup cost for large tables.

Suggested guardrails if we explore it:

  • Keep cached H2 tables read-only by policy (or clearly ephemeral scratch if writable).
  • Enforce explicit TTL + last-refresh metadata per table/subset.
  • Use deterministic refresh (full or incremental/CDC where possible).
  • Limit to small/medium, low-churn, high-read datasets first.
  • Add clear fallback to primary DB when cache is cold/expired.

So: technically feasible and potentially valuable, but only safe if we treat it as a controlled, stale-tolerant read layer.

Confidence: High (85%) for feasibility, Medium (65%) for operational simplicity at scale.

Copilot AI requested a review from rrobetti May 7, 2026 07:11
@rrobetti
Copy link
Copy Markdown
Contributor

rrobetti commented May 7, 2026

@copilot add this to the analysis docs with your notes.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 7, 2026

@copilot add this to the analysis docs with your notes.

Added in commit 684cbfc.

I documented this as an additional exploration note in documents/analysis/FULL_TABLE_CACHE_EXPLORATION.md, including the H2 mirror datasource approach, pros, concerns, and guardrails.

No UI changes were made (docs-only change).

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 7, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants