Skip to content

feat: enhance indexing progress and CSV preview UX#21

Merged
aGallea merged 7 commits into
masterfrom
realtime-indexing-progress
Feb 25, 2026
Merged

feat: enhance indexing progress and CSV preview UX#21
aGallea merged 7 commits into
masterfrom
realtime-indexing-progress

Conversation

@aGallea

@aGallea aGallea commented Feb 25, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add real-time indexing progress updates with informative log messages, verbosity filter (low/medium/high), and stuck detection with warning/error states
  • Fix log broadcast timing by awaiting WebSocket sends and properly stopping timers on completion/cancellation
  • Enhance CSV data preview with image URL hover tooltips, "Upload Different CSV" button, configurable row limit dropdown, and client-side pagination
  • Expand test coverage from 88.53% to 98.53% (176 tests, up from 144)

Changes

Backend

  • Improve indexing log messages for clarity (batch complete, reading row)
  • Add cancellation status handling with proper WebSocket broadcast
  • Improve progress broadcasting and WebSocket lifecycle management

Frontend

  • IndexProgress: Wire cancellation callback, expose markCancelled from WebSocket hook
  • useIndexWebSocket: Handle cancelled message type, preserve terminal status on close, add markCancelled for optimistic UI updates
  • CsvPreview: Add ImageCell component with floating image preview on hover, row limit dropdown (10/25/50/100), client-side pagination (25 rows/page) with full navigation
  • IndexPage: Add "Upload Different CSV" button, wire preview limit state to API query

Tests

  • test_indexer.py: Add 426 lines covering model loading, cancellation events, CSV processing, batch handling, and build_and_encode
  • test_server_csv.py: Add csv.Error edge cases for upload/preview, no-filename handling
  • test_server_index.py: Add cancelled status path, WebSocket endpoint, CSV path ValueError, heartbeat tests
  • test_server_search.py: Add _get_chromadb_client, _get_text_model, _get_image_model cache miss path tests
  • test_utils.py: Add session closed, retry success logging, TimeoutError, ClientResponseError, retryable 429, retry delay tests

Coverage Report

Module Before After
indexer.py 70% 98%
server/routes/csv.py 79% 100%
server/routes/index.py 87% 99%
server/routes/search.py 86% 100%
utils.py 93% 100%
TOTAL 88.53% 98.53%

aGallea and others added 3 commits February 25, 2026 13:04
…ity filter, and stuck detection

- Add intermediate progress updates every 10 rows instead of only on batch boundaries
- Add on_log callback to indexer for rich phase-level logging (CSV loading, model loading, batch progress, completion)
- Add verbosity levels (low/medium/high) with frontend dropdown filter on Live Logs panel
- Wire on_log to WebSocket broadcast in index route
- Add 3-second heartbeat from backend for connection health monitoring
- Add client-side elapsed timer with server time resync for smooth display
- Add stuck detection: warning banner at 15s silence, error banner at 30s
- Add try/except around model loading with specific error reporting via on_log
- Guard heartbeat cleanup with contextlib.suppress for closed event loops
- Update test mock signatures for on_log parameter compatibility
- Fix ruff formatting in test_server_search.py and test_utils.py
…rs on completion

- Change on_log from sync fire-and-forget to async awaited callback so
  WebSocket writes complete before the next row, eliminating batchy logs
- Wrap CLIPModel and SentenceTransformer loading in asyncio.to_thread
  to keep the event loop responsive during model init
- Add stopTimers helper in useIndexWebSocket to clear elapsed and stuck
  detection intervals on completed/error
- Sync final elapsed_seconds from server on completion
…ow limit, and pagination

- Add image URL hover preview with floating tooltip in data preview table
- Add 'Upload Different CSV' button to return to upload step
- Add row limit dropdown (10/25/50/100) wired to preview API
- Add client-side pagination (25 rows per page) with full navigation controls

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
@github-actions

github-actions Bot commented Feb 25, 2026

Copy link
Copy Markdown

Tests Report 📄

Tests Succees ✅

JUnit Details

Total Tests Failures Errors Skipped Time ⏳
176 0 0 0 22.40s

Coverage Details (100% >= 90%) ✅

Diff Cover Details
FileCovered LinesMissing Lines
embedding_cluster/indexer.py99/99100%
embedding_cluster/server/routes/index.py90/90100%
Total189/189100%

aGallea and others added 4 commits February 25, 2026 15:22
…agement

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…handling

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…atch processing

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
@aGallea aGallea merged commit 4612c92 into master Feb 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant