Feat/search v2#375
Merged
Merged
Conversation
- Add generation counter to ConnManager to prevent N goroutines from each reconnecting when they all fail on the same dead connection. Only the first caller reconnects; the rest piggyback on the fresh connection. - Serialize PublishMessage through pubMu when confirms are enabled to prevent orphan confirmations from causing false timeouts in publishAndWaitConfirm. - Fix Debugf format verb in GetDueScheduledBlogs log statement.
Backend implementation of Search v2 across gateway + user + blog services. Highlights: - Gateway: new internal/blogsearch (ES client + query builder + routes for /api/v2/blog/search/v2 and /api/v2/search/suggest). - Gateway: internal/cache/searchcache fail-open Redis cache with structured search_event logging (latency, cache_hit, zero_result, result_count). - Users svc: new database/user_search_v2.go backed by pg_trgm GIN indexes on user_account.search_doc; service exposes v2 search RPC. - Blog svc: v2_queries tweaks to support denormalised reindex. - Postgres migrations 000006 (CREATE EXTENSION pg_trgm) and 000007 (STORED search_doc column + GIN indexes). - ES mapping v3 (documents/reindex/mapping_v3.json) + reindex tool (scripts/reindex_blogs_v3) with dry-run, atomic alias swap pattern. - Deprecation: /api/v1/user/search and /api/v2/blog/search now 308 redirect to v2 paths, preserving query string. - Critical fix: blogsearch.NewClient prefers OPENSEARCH_OS_HOST (container-network) over OPENSEARCH_ADDRESS (host-script). - Critical fix: ES bool filter changed from filter:term=false to must_not:term=true to handle legacy docs missing is_archived/is_draft fields. - Docs: design plan, implementation plan, prod deploy runbook, OpenAPI updates for both services.
… queries mapping_v3.json declares these three fields as top-level keyword (no .keyword sub-field). Legacy term queries used '<field>.keyword', which silently matched zero docs against v3 because ES does not error on missing fields. Symptoms post cut-over: GET /api/v2/blog/:id, /api/v2/blog/:id/stats, /api/v2/blog/user/:username, /api/v2/blog/trending, and archive/delete code paths returned 404 'the blog does not exist' even though docs were indexed. Fix: 25 callsites across opensearch.go (11), v2_queries.go (10), blogs_matadata.go (3), query.go (1) changed from '<field>.keyword' to '<field>'. Backward-compatible with v2 mapping because blog IDs / account IDs tokenise to a single lowercase term under the default analyzer. Writes (IndexRequest/UpdateRequest by DocumentID) are unaffected. No _update_by_query / _delete_by_query callers. Docs: runbook gains §3.1 (mapping breaking change), §6 critical non-search smoke tests, §10 rollback row. Implementation plan Phase 2 records the fix.
- Switch WHERE predicate from \ua.search_doc % $1\ to \$1 <% ua.search_doc\ and rank \similarity(...)\ -> \word_similarity($1, ua.search_doc)\. - pg_trgm \%\ (similarity) normalises by trigram count of both strings, so short queries against long search_docs fall below the 0.3 threshold and miss real matches (e.g. searching "joseph" against "SJ Shinu Joseph The fourth monkey" yields similarity=0.21, no hit). - \<% \ (word_similarity) scores against the best-matching extent of the doc, immune to doc length. Same scenario yields 1.0 and matches. - Existing GIN(gin_trgm_ops) index on search_doc supports \<%\ natively in PG 11+; no schema change needed. - Verified empirically against dev DB: predicate change moves result count from 0 -> 1 for query "joseph".
|
🏷️ [bumpr] |
GorillaGigabytes
approved these changes
May 16, 2026
|
🚀 [bumpr] Bumped! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.