Skip to content

Feature/sync multipart v012#127

Open
NateEaton wants to merge 83 commits intomainfrom
feature/sync-multipart-v012
Open

Feature/sync multipart v012#127
NateEaton wants to merge 83 commits intomainfrom
feature/sync-multipart-v012

Conversation

@NateEaton
Copy link
Copy Markdown
Owner

@NateEaton NateEaton commented Mar 24, 2026

Summary

Adopts multipart sync end‑to‑end and replaces legacy article caching with a unified offline content architecture. Bookmarks either have a full offline package (HTML + images) or they don't — managed automatically by a configurable policy engine. Also includes annotation caching, multiple reader/UX fixes, CI workflow improvements, and database migrations to support the new content model.

Major Changes

Multipart Sync + Metadata Migration

  • Added multipart sync client + streaming parser and switched delta metadata sync to POST /bookmarks/sync.
  • Reworked metadata persistence and initial/full‑sync bootstrap behavior.
  • Delta sync falls back to full sync on failure; resource path traversal guard added.

On‑Demand Content Packages

  • Implemented offline content packages (HTML + resources) with staging + atomic replace and rollback protection.
  • Integrated WebViewAssetLoader + per‑bookmark base URLs.
  • Added picture wrapper HTML generation for bookmarks without server HTML.
  • On‑demand open of a bookmark triggers a priority full‑package download if none exists.

Offline Annotation Support

  • Added CachedAnnotationEntity + DAO + schema migration.
  • Implemented annotation HTML parsing + caching in commit path.
  • Enriched annotation rendering (no repaint, note visibility, sync from other clients).

Unified Offline Content Architecture

Replaced an intermediate two‑tier model (text‑only cache vs. full package with per‑bookmark toggle) with a single‑tier policy‑driven approach:

  • Policy engine (OfflinePolicyEvaluator) with three modes:
    • Storage Limit — keep downloading until a byte budget is reached
    • Newest N — keep the N most recent bookmarks offline
    • Date Range — keep bookmarks created within a rolling window
  • Secondary storage cap for Newest N and Date Range policies prevents unbounded disk usage.
  • Batch worker (BatchArticleLoadWorker) runs the policy check → download → prune cycle, with constraint gating (Wi‑Fi only, battery saver) checked between batches.
  • Scope control — offline content can target "My List" or "My List + Archived".
  • Lifecycle transitions — disabling offline reading clears all managed content; re‑enabling triggers a fresh sync.
  • Redesigned Sync Settings screen with policy selection, storage cap, scope, and network constraint controls.
  • Removed per‑bookmark "Make available offline" toggle and all two‑tier infrastructure (hysteresis pruning, deleteResources, downloadImagesWithArticles conditional).
  • Unit tests for policy evaluator boundary conditions and batch worker integration.

Reader & UX Fixes

  • Fixed reader lag, broken images, and wrong highlight positioning.
  • Fixed annotation image breaking and scroll restore loop latency.
  • Fixed duplicate images in picture bookmark reader.
  • Fixed narrow picture content layout (figure/figcaption CSS).
  • Fixed blank images in article lightbox when using offline content.
  • Fixed picture/video stuck at PERMANENT_NO_CONTENT after offline open.
  • Video embed offline placeholders + eligibility logic (transcript content only).
  • Show domain name in bookmark details URL field; open full URL in browser from details.
  • Reduced WebView churn on reader open.
  • On‑demand content loading progress bar shows deterministic progress.

Other Fixes & Improvements

  • Fixed case‑sensitive label filtering.
  • Added Codeberg.org to About screen.
  • Fixed BookmarkDto embed_hostnameembed_domain serialization; added unit tests.
  • Fixed omitDescription not persisting (SyncRequest was silently omitting with_json).
  • Preserved embed/embed_domain across sync paths.
  • Added save‑to‑downloads and selectable text for log viewer.

Persistence + Migration

  • Dropped legacy article_content table.
  • Added content_package, content_resource, and cached annotation tables.
  • Removed legacy paginated metadata reload path; retained limited legacy endpoints as fallback.

CI & Build

  • Fixed workflow if conditions to prevent double‑runs on push+PR.
  • Aligned workflows with snapshot tester flow.
  • Debug signing used only as local fallback.

Spec & Docs

  • Added unified offline content architecture spec and implementation plan.
  • Updated user guide (settings, bookmarks) for new sync settings and offline behavior.
  • Archived superseded specs from earlier offline approaches.

Testing

  • ./gradlew :app:assembleDebugAll
  • ./gradlew :app:testDebugUnitTestAll
  • ./gradlew :app:lintDebugAll
  • Manual: content download + offline reopen, refresh content, clear/re‑download, video offline placeholder, policy switching, enable/disable lifecycle

Notes

  • Video packages are intentionally gated to has_article = true (offline transcript only; playback remains online).
  • Content package commit persists DB metadata before file swap with rollback protection.
  • The branch history contains an intermediate "granular/managed offline" approach (per‑bookmark toggle, two‑tier caching, hysteresis pruning) that was superseded by the unified offline architecture. The intermediate code was systematically removed in Slices 1–6.

NateEaton and others added 30 commits March 20, 2026 00:51
Add MultipartSyncParser (streaming MIME parser), MultipartSyncClient
(batched API orchestration), SyncRequest model, and syncBookmarks()
Retrofit endpoint. Migrate delta-detected metadata reloads to POST
/bookmarks/sync with with_json=true, falling back to legacy paginated
GET when updatedIds is unavailable. Add Room schema v11 with
content_package and content_resource tables (destructive migration for
article_content). Update v2 spec document.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add download button to save log zip to Downloads via MediaStore API.
Wrap log content in SelectionContainer for drag-to-select copy support.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ContentPackageManager for per-bookmark file storage with atomic
staging/commit. Add LoadContentPackageUseCase that fetches full offline
packages (JSON+HTML+resources) via multipart sync, with fallback to
legacy LoadArticleUseCase. Add ContentPackageDao for Room persistence.
Add OfflineContentPathHandler serving local files through
WebViewAssetLoader. Update BookmarkDetailWebViews to use per-bookmark
offline base URLs and intercept resource requests for local serving.
Generate picture wrapper HTML when server HTML is absent. Add
androidx.webkit dependency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wrap MediaStore.Downloads usage in API 29+ check with pre-Q fallback
using public Downloads directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Coil's AsyncImage cannot resolve URLs with the synthetic offline host
(https://offline.mydeck.local/). Convert these URLs to file:// URIs
pointing to the actual local files so images render correctly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Post-branch item: investigate progressive display of article content
instead of spinner-until-ready, especially for image-heavy articles.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update DAO queries to include Picture bookmarks (type='photo') in
  content-eligible queries, not just hasArticle=1
- Migrate BatchArticleLoadWorker and DateRangeContentSyncWorker to use
  LoadContentPackageUseCase (multipart) with fallback to legacy
  LoadArticleUseCase on transient failure
- Add sourceUpdated freshness check: when content is DIRTY but the
  existing package's sourceUpdated matches the bookmark's current
  updated timestamp, restore DOWNLOADED state without re-fetching
- Add storage usage display and "Clear All Offline Content" button
  to Sync Settings with confirmation dialog
- Add storage management string resources to all 10 language files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a picture bookmark has offline content from a content package, the
wrapper HTML already contains the image. The toHtml() method was also
prepending the metadata imgSrc, causing the image to appear twice.
Skip the manual image prepend when articleContent comes from a content
package (offlineBaseUrl is set).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The <figure> element used in picture wrapper HTML inherits default
browser margins (~40px), making the image and caption narrower than
the title/description above. Reset figure margin to 0 and add styling
for figcaption and .picture-caption across all four theme templates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The offline storage size display only refreshed on init and after
clearing. Now it also refreshes when:
- Date-range download work transitions from running to completed
- Content download counts change in the DB (via detailedSyncStatus flow)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add CachedAnnotationEntity Room table and DAO (v12 migration)
- Build AnnotationHtmlParser to extract annotation metadata from <rd-annotation> HTML
- Populate annotation cache during ContentPackageManager.commitPackage()
- Make fetchAnnotations offline-first in BookmarkDetailViewModel
- Remove legacy annotation sync for content-package bookmarks (fix color flash)
- Update annotation-driven refresh to use multipart re-download
- Add comprehensive AnnotationHtmlParser unit tests
- Update spec with Stage 3a architecture and test requirements
The Readeck multipart sync endpoint strips all attributes from
<rd-annotation> elements, returning bare tags. This broke highlight
colors, offline listing, tap-to-edit, scroll-to-annotation, and
selection-based edit detection.

- Add AnnotationHtmlEnricher to patch bare tags with attributes from
  the annotations REST API during content package commit
- Add AnnotationDto color/note fields (API returns them despite spec)
- Use API color as authoritative source in fetchAnnotations
- Dismiss annotations sheet before scrolling to highlight
- Show snackbar when attempting highlight CRUD while offline
- Add offline-annotation-crud-spec for future queued mutation support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sist server-provided value; ui: offline placeholder for video embeds; debug: include omit_description in Details dialog
…add FreshnessMarkerUseCase and spec addendum; update workers, parser, content package handling, and detail screen
…, and progress bar

- Option D: performFullSync() now persists bookmark metadata from the list
  endpoint paging response (zero additional network requests)
- FullSyncWorker skips loadBookmarksUseCase when full sync already persisted
  metadata; LoadBookmarksWorker INITIAL trigger uses performFullSync directly
- Video bookmarks now participate in automatic/manual/date-range content sync
  (removed type != 'video' exclusion from DAO queries and LoadContentPackageUseCase)
- Preserve embed/embedHostname in insertBookmarks when incoming data has null
  (list endpoint omits embed; multipart sync includes it)
- Add refreshBookmarkMetadata() as fallback for video bookmarks opened before
  any multipart sync has populated embed
- Offline video embed placeholder now matches actual embed host instead of
  hardcoded YouTube/Vimeo allowlist
- Reader loading overlay: indeterminate bar for already-downloaded content
  (WebView render), no duplicate bar during on-demand fetch (top-level
  determinate bar covers that)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ress

The determinate progress bar was invisible because it was drawn before
BookmarkDetailScreen in the Box (earlier children are painted behind
later ones). Moved it inside the Scaffold content Box as the last child
so it renders on top of the loading overlay, positioned just below the
top app bar.

Also fixed the bar stalling during the network fetch phase by adding a
ticker coroutine that advances progress in small increments (5% every
300ms) while the multipart response is in flight, with a monotonic
progress floor to prevent backwards jumps.

Additionally, the ReaderLoadingOverlay now only shows the indeterminate
bar when content was already cached at screen open — not after an
on-demand fetch transitions to WebView rendering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…and design rationale

Add Implementation Status section documenting completion of all four stages
plus Stage 3a. Add Retained Legacy Endpoints section explaining why three
GET-based endpoints remain (article fallback, creation polling, error flag
refresh) with rationale for each. Update Stage 4 rollout description to
reflect actual scope. Expand field notes with Option D resolution details
and video content package inclusion rationale.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All content from the addendum (parser fixes, atomic commit, timestamp
normalization, worker batching, freshness marking, annotation enrichment)
is now reflected in the main spec's Implementation Status and field notes.
The remaining follow-ups listed in the addendum are complete.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move to docs/archive/:
- filter-ui-filtered-list-spec.md (v0.11.1, fully shipped)
- sync-error-media-follow-up-spec.md (superseded by sync-multipart-adoption-spec)
- sync-content-api-omit-description-spec.md (superseded by sync-multipart-adoption-spec)
- sync-architecture-optimization-spec.md (superseded by sync-multipart-adoption-spec)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…age 4

Fix BookmarkDto @SerialName mismatch: server returns "embed_domain" but
the DTO was mapped to "embed_hostname", causing embedHostname to always
deserialize as null. The field was undocumented in the OpenAPI spec but
confirmed present in both GET /bookmarks/{id} and POST /bookmarks/sync
responses via direct API testing.

New unit tests covering Stage 4 changes:
- performFullSync persists bookmark metadata from paging response
- performFullSync returns inserted count in SyncResult
- insertBookmarks preserves existing embed when incoming embed is null
- insertBookmarks overwrites embed when incoming embed is non-null
- LoadBookmarksUseCase with null/empty updatedIds returns success
  without fetching (no-op guard)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… with_json

The root cause was Kotlinx serialization's encodeDefaults=false (the default)
combined with SyncRequest.withJson defaulting to true. When callers set
withJson=true it matched the default and was omitted from the serialized JSON,
so the server never received the flag and returned no JSON parts in multipart
sync responses. Content packages got HTML+resources but no metadata.

- Change SyncRequest.withJson default from true to false so explicit true is
  always serialized
- Add encodeDefaults=true to global Json config as belt-and-suspenders
- Simplify resolvePersistedOmitDescription to always preserve existing value
  when incoming is null (list endpoint never sends omit_description)
- Add BookmarkDao.updateOmitDescription for direct writes after content
  package commit, surviving concurrent sync races
- Add omitDescription to BookmarkDebugExporter localState output
- Add realistic CRLF+Content-Length multipart parser test matching server format
- Update BookmarkRepositoryImplTest to match simplified resolve logic

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…d fix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The legacy LoadArticleUseCase fallback in fetchContentOnDemand was running
for all bookmark types. For Picture and Video, this called GET /article
which marks hasArticle=false bookmarks as PERMANENT_NO_CONTENT — permanently
blocking the content package path from retrying.

Restrict the legacy fallback to Article type only. Picture and Video types
now report the content package result directly without the destructive
side-effect on contentState.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…clients, note visibility

Three annotation/highlight issues fixed:

1. Annotation CRUD (create, recolor, delete) no longer triggers a full WebView
   reload. Color changes use pure JS DOM updates; create/delete use lightweight
   HTML-only multipart fetch with JS innerHTML replacement.

2. Annotations added via other clients now sync to the reader HTML when opening
   a bookmark. New syncAnnotationsForContentPackage() compares API annotation
   snapshot against cached snapshot and triggers HTML-only refresh if changed.

3. Annotations with notes were completely hidden by CSS display:none targeting
   the data-annotation-note attribute. Replaced with a ::after pencil icon
   indicator matching the Readeck web UI.

New spec: docs/specs/highlight-compose-fix.md
Updated: sync-multipart-adoption-spec.md with field notes for this fix,
embed_domain SerialName mismatch, and picture/video PERMANENT_NO_CONTENT.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Videos with an embed but no article content (hasArticle=false) were
incorrectly routed through fetchContentOnDemand, which marked them
PERMANENT_NO_CONTENT. Combined with a race against metadata refresh
(which provides the embed data), this caused the auto-switch to View
Original on first open. Second open worked because embed was already
in Room.

Skip fetchContentOnDemand and PERMANENT_NO_CONTENT error state for
videos that have an embed fallback. Videos with neither article nor
embed still get the correct failure path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Await refreshBookmarkMetadata before evaluating content state for
videos with missing embed. Previously the metadata refresh was
fire-and-forget, creating a race where contentLoadState was set to
Failed (and the UI switched to ORIGINAL mode) before the embed
arrived from the API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NateEaton and others added 30 commits April 1, 2026 15:06
* docs: add pre-merge fix specs and correct validation test attribute names

- branch-pre-merge-fixes.md: documents two required/recommended fixes found
  in final code review (unconditional deleteResources() in ViewModel, wrong
  KDoc in rewriteToRelativeResourceUrls), plus supplementary findings on
  partial enrichment lock-in and test spec attribute name errors
- branch-validation-tests.md: correct attribute names from data-id/data-color
  to data-annotation-id-value/data-annotation-color; add partial enrichment
  lock-in as a documented known limitation to test for

https://claude.ai/code/session_01LcSEVkLCCLLGue3GCLpzZv

* docs: add pre-merge guardrail + unit test implementation specs

Detailed implementation instructions for:
- Guardrail 1: fix partial enrichment lock-in in AnnotationHtmlEnricher
  (replace enriched-tag early-exit with bare-tag check) + add 50% mismatch
  threshold that aborts enrichment and returns original HTML when match
  quality is too poor
- Guardrail 2: add relative-URL safety check to ContentPackageManager
  .deleteResources() — aborts with warning if HTML still references local
  resources via relative paths
- Unit test specs: AnnotationHtmlEnricherTest (10 cases including lock-in
  regression), LoadContentPackageUseCaseTest (5 cases for refreshHtmlForAnnotations),
  ContentPackageManagerTest (5 cases for deleteResources guardrail)
- Build verification checklist

https://claude.ai/code/session_01LcSEVkLCCLLGue3GCLpzZv

* docs: add Readeck API documentation gaps, improvement proposals, and dump script

docs/readeck-api/api-documentation-gaps.md
  Six documented gaps in Readeck API docs discovered during v0.12.0 development:
  POST /bookmarks/sync request/response format undocumented, rd-annotation
  attribute stripping undocumented, embed_domain missing from OpenAPI spec,
  omit_description absent from list endpoint, field coverage differences not
  documented, GET vs POST /bookmarks/sync relationship not explained.

docs/readeck-api/api-improvement-proposals.md
  Five proposals for improving the sync API for mobile offline reading clients:
  include annotation attributes in sync HTML, add since filter to POST sync,
  add omit_description and embed_domain to list endpoint, add error parts to
  multipart response, document max batch size.

docs/readeck-api/readeck-api-dump.py
  CLI script for dumping and comparing Readeck API endpoint output. Supports
  legacy article HTML, annotations, single bookmark JSON, GET sync IDs, and
  POST multipart sync with full binary-to-readable multipart parsing. Includes
  --compare for diffing runs before/after server changes.

https://claude.ai/code/session_01LcSEVkLCCLLGue3GCLpzZv

---------

Co-authored-by: Claude <noreply@anthropic.com>
…ess; add TODO guidance

Fix 1 proposed replacement now checks updateHtml() return value before calling
deleteResources() — updateHtml() can fail on I/O error, leaving old relative-URL HTML
on disk. Deleting resources after a failed HTML write reproduces the broken-images bug.

Supplementary finding section updated with a recommended TODO comment to add inline in
AnnotationHtmlEnricher.kt at the partial-enrichment lock-in site, so the limitation
is visible to developers working in that file post-merge.

https://claude.ai/code/session_01LcSEVkLCCLLGue3GCLpzZv
Guardrail 1 (AnnotationHtmlEnricher): replace the early-exit guard with a
bare-tag-specific check to fix partial-enrichment lock-in, and abort enrichment
when fewer than 50% of annotations are matched to avoid silently committing
poorly-enriched HTML.

Guardrail 2 (ContentPackageManager.deleteResources): abort deletion if the
current HTML still contains relative resource URLs (./filename), preventing
broken offline images when callers skip the HTML-refresh step.

Also covers the BookmarkDetailViewModel Fix 1 caller path that gates
deleteResources on a successful updateHtml().

Unit tests added:
- AnnotationHtmlEnricherTest: threshold (above/below/boundary), lock-in
  regression (partially-enriched HTML re-enriched), entity decoding
- LoadContentPackageUseCaseTest: legacy endpoint preferred, multipart
  fallback, URL rewrite applied/skipped based on hasResources, failure
  path does not overwrite HTML
- ContentPackageManagerTest: relative URL aborts deletion, absolute URL
  allows deletion, no-package/empty-resources no-ops, no HTML file allows
  deletion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…versal

LoadBookmarksWorker: when delta sync fails or no cursor is available,
deltaUpdatedIds is null and loadBookmarksUseCase.execute(null) was a silent
no-op, leaving bookmark data stale until the next scheduled full sync.
Now falls back to bookmarkRepository.performFullSync() in that case, matching
the same error-handling shape as the initial load path.

ContentPackageManager.commitPackage: resource files were written using the
server-provided path value directly (File(stagingDir, resource.path)) with
no bounds check. A malformed or malicious path such as ../../../... could
escape the staging directory and write to arbitrary locations within app
storage. Now validates canonicalPath starts within stagingDir and skips
with a warning if not.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ess)

WIP snapshot. Test-only storage limits (5/10/20 MB) and some tooling
remain pending removal after testing is complete.

Changes in this commit:

- Offline reading regression: guard enqueueArticleDownload in
  BookmarkRepositoryImpl with isOfflineReadingEnabled() check at both
  call sites (direct and via pollForBookmarkReady). Adding a bookmark
  while offline reading is disabled no longer triggers a content download.

- Image-only storage measurement: add getTotalImageResourceBytes() DAO
  query summing content_resource.byteSize for image/* MIME types, and
  calculateImageSize() on ContentPackageManager. Storage cap enforcement
  now measures image resources only, not total offline directory size.

- Concurrent worker fix: switch ExistingWorkPolicy from KEEP to REPLACE
  in LoadBookmarksUseCase so a new enqueue cancels any in-flight worker
  rather than spawning alongside it. Add AtomicBoolean in-flight guard
  in BatchArticleLoadWorker as a secondary safety net.

- Two-watermark hysteresis: enforcement now prunes to 80% of the limit
  (low watermark) rather than exactly the limit, triggered at 95% (high
  watermark). Adaptive batch sizing drops from 10 to 2 when image storage
  is above 90% of the limit (gate threshold).

- Limit-hit text-only mode: once the storage cap is hit and pruned during
  a worker run, subsequent batches call restoreDownloadedStateIfHasContent()
  instead of re-downloading. Bookmarks with existing text content are
  restored to DOWNLOADED state via a DB update only (no network). This
  eliminates the download→prune oscillation and the resulting scattered
  image pattern in the bookmark list.

- Image backfill via worker: reconcileImagePreferenceForManagedContent()
  no longer calls executeBatch() directly when turning images on. Instead
  it marks text-only bookmarks as DIRTY and lets BatchArticleLoadWorker
  handle the backfill, ensuring storage cap enforcement applies.

- Sync Status split: offlineStorageSize replaced with separate textStorageSize
  and imageStorageSize fields, displayed as "Text: X" and "Images: Y" lines.

- Test storage limits: MB_5 / MB_10 / MB_20 added to OfflineImageStorageLimit
  enum and toLabelResource(). English strings only — to be removed before release.

- Spec: docs/specs/image-storage-limit-refinements-spec.md added capturing
  all five issues, root causes, fixes, and a TODO for picture bookmark
  image storage counting against the cap when Download Images is off.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tion plan (#133)

* Add unified offline content architecture spec and implementation plan

Proposes a simplified offline content model that replaces the two-tier
(text-only / full) managed offline design with a single full-package-only
tier. Covers content state semantics, three management policies
(storage limit, newest-N, date-range rolling window), bookmark card
icon behaviour, annotation implications, settings layout, migration
strategy, and future considerations (collections-based scope,
on-demand cache eviction). Implementation plan breaks the work into
13 independently mergeable slices with model recommendations (Haiku /
Sonnet / Opus) for each.

https://claude.ai/code/session_017MmFgCXCCArrdtLs5B9pcU

* Add Material You compliance and user guide requirements to offline spec

Notes that the sync settings screen must be brought to full Material You
parity (Bookmark Sync section is the reference) and that settings.md and
your-bookmarks.md user guides must be updated to cover the new offline
reading controls and card icon semantics.

https://claude.ai/code/session_017MmFgCXCCArrdtLs5B9pcU

---------

Co-authored-by: Claude <noreply@anthropic.com>
…#134)

* Add unified offline content architecture spec and implementation plan

Proposes a simplified offline content model that replaces the two-tier
(text-only / full) managed offline design with a single full-package-only
tier. Covers content state semantics, three management policies
(storage limit, newest-N, date-range rolling window), bookmark card
icon behaviour, annotation implications, settings layout, migration
strategy, and future considerations (collections-based scope,
on-demand cache eviction). Implementation plan breaks the work into
13 independently mergeable slices with model recommendations (Haiku /
Sonnet / Opus) for each.

https://claude.ai/code/session_017MmFgCXCCArrdtLs5B9pcU

* Add Material You compliance and user guide requirements to offline spec

Notes that the sync settings screen must be brought to full Material You
parity (Bookmark Sync section is the reference) and that settings.md and
your-bookmarks.md user guides must be updated to cover the new offline
reading controls and card icon semantics.

https://claude.ai/code/session_017MmFgCXCCArrdtLs5B9pcU

* Fix paradigm violations: remove per-bookmark content management references

Spec: clarify that the Bookmark Detail dialog exposes no per-bookmark
content removal action; add the incomplete onRemoveDownloadedContent
dead-code cleanup (both ViewModels + full composable callback chain +
string resources) to the explicit removal list.

Plan: expand Slice 3 to cover the full dead-code cleanup of the
per-bookmark removal action; rewrite Slice 11 to remove the incorrect
instruction to retain the overflow menu item.

https://claude.ai/code/session_017MmFgCXCCArrdtLs5B9pcU

---------

Co-authored-by: Claude <noreply@anthropic.com>
Add comprehensive KDoc documentation to clarify DOWNLOADED + hasResources semantics:
- BookmarkEntity.ContentState enum: Document two DOWNLOADED sub-states
- ContentPackageEntity: Clarify hasResources field meaning
- BookmarkListItemEntity: Explain offline availability indicators
- ContentPackageManager: Add unified architecture overview

Remove orphaned sync_remove_downloaded_content string resources from all language files.

Audit identifies areas where contentState checks need hasResources consideration for later slices.
Update content descriptions to reflect unified offline content architecture:
- Outline icon: 'Text available' (on-demand text cache)
- Filled icon: 'Available offline' (full offline package)

Add new string resources to all language files as English placeholders.
Icon logic already correctly implemented based on hasResources flag.

Verification:
- assembleDebugAll: PASSED
- testDebugUnitTestAll: FAILED (pre-existing BookmarkRepositoryImplTest issue)
- lintDebugAll: FAILED (pre-existing missing translation issues)
- Remove image toggle row from BookmarkDetailDialog and backing ViewModel action/state
- Remove downloadImagesWithArticles preference key from SettingsDataStore/Impl
- Remove BookmarkDetailViewModel.onRemoveDownloadedContent() and
  BookmarkListViewModel.onRemoveDownloadedContent() (dead code)
- Remove onRemoveDownloadedContent callback parameter from full composable chain:
  BookmarkDetailScreen, BookmarkDetailTopBar, BookmarkDetailMenu, BookmarkCard,
  BookmarkCards, BookmarkListScreen, BookmarkDetailsDialog
- Remove sync_download_images string resources from all language files
- Update LoadContentPackageUseCase to treat images as always enabled (removing
  isDownloadImagesEnabled() calls; full cleanup in Slice 4)
- Update ContentSyncPolicyEvaluator and LoadBookmarksUseCase to remove
  downloadImages constraint gating
- Update SyncSettingsScreen/ViewModel to remove downloadImages state
- Remove tests for removed functionality
- Remove ContentPackageManager.deleteResources() and its relative-URL
  safety check (containsRelativeResourceUrls, RELATIVE_RESOURCE_URL_PATTERN)
- Remove LoadContentPackageUseCase.executeTextOnlyOverwrite() — the
  HTML-refresh + URL-rewriting downgrade path
- Remove unused settingsDataStore injection from LoadContentPackageUseCase
- Simplify fetchAndCommit() to always call fetchContentPackages
- Clean up executeForceRefresh() signature (forceResources param removed)
- Remove BatchArticleLoadWorker hysteresis pruning, adaptive batch sizing,
  limitHitDuringRun branch, restoreDownloadedStateIfHasContent call,
  contentPackageManager param, and BATCH_SIZE_NEAR_LIMIT/watermark constants
- Remove BookmarkDao.restoreDownloadedStateIfHasContent() DB query
- Remove ContentPackageManagerTest (all tests were for deleteResources)
- Fix LoadContentPackageUseCaseTest constructor to match updated signature

ContentSyncPolicyEvaluator unchanged — already clean after Slice 3.
…efresh

Remove the `hasResources: Boolean` parameter from
`LoadContentPackageUseCase.refreshHtmlForAnnotations` and
`fallbackMultipartRefresh`. URL rewriting to relative paths now always
occurs unconditionally, reflecting the unified model where the only
supported offline state is a full package (HTML + resources).

Guard the call site in `BookmarkDetailViewModel.refreshAnnotationHtml`:
only invoke `refreshHtmlForAnnotations` when `hasResources == true`.
Text-cached bookmarks (hasResources=false) fall through to the existing
`executeForceRefresh` fallback, avoiding broken relative image URLs for
bookmarks that have no local resource files.

Update `LoadContentPackageUseCaseTest`:
- Remove `hasResources` argument from all call sites and mock matchers.
- Remove the now-invalid `urlRewriteNotAppliedWhenNoResources` test
  (behaviour no longer exists).
- Rename `urlRewriteAppliedWhenHasResources` → `urlRewriteApplied`.
- Update `fetchHtmlOnly` mock matchers to drop the second argument.

Update `BookmarkDetailViewModelTest`:
- Remove `hasResources` argument from `refreshHtmlForAnnotations` mocks.
- Add `coEvery { contentPackageManager.hasResources(any()) } returns true`
  to `@Before` setup so `cachedHasResources` is populated correctly and
  the hasResources guard passes in annotation-lifecycle tests.
- Add per-test `contentPackageManager.hasResources("123") = true` overrides
  in the two annotation tests that verify `refreshHtmlForAnnotations` is
  called.

Files changed:
  LoadContentPackageUseCase.kt
  BookmarkDetailViewModel.kt
  LoadContentPackageUseCaseTest.kt
  BookmarkDetailViewModelTest.kt
When offline reading is enabled and a bookmark opens via the legacy
text-only fallback (hasResources=false after fetchContentOnDemand),
automatically enqueue a high-priority full-package download so the
bookmark transitions from text-cached to fully offline.

BatchArticleLoadWorker:
- Add KEY_PRIORITY_BOOKMARK_ID input data key.
- doWork() checks for the priority key first; if present, routes to
  processPriorityBookmark() which skips the DAO batch query and the
  isRunning guard, allowing priority jobs to coexist with the batch.
- processPriorityBookmark() checks policyEvaluator constraints then
  calls loadContentPackageUseCase.executeBatch(listOf(bookmarkId)).
- enqueuePriorityDownload() companion function: enqueues under unique
  name "content_priority_<bookmarkId>" with REPLACE policy to
  deduplicate repeated opens of the same bookmark.

BookmarkDetailViewModel:
- Inject WorkManager.
- After fetchContentOnDemand resolves to Loaded, check
  cachedHasResources != true && isOfflineReadingEnabled(). If both true,
  call enqueuePriorityPackageDownload() which reads WiFi/battery
  constraints from settingsDataStore and delegates to
  BatchArticleLoadWorker.enqueuePriorityDownload().

BookmarkDetailViewModelTest:
- Add WorkManager mock (relaxed) to field declarations, @before setup,
  and createViewModel() constructor call.

Files changed:
  BatchArticleLoadWorker.kt
  BookmarkDetailViewModel.kt
  BookmarkDetailViewModelTest.kt
- Dissolve Sync Status section; redistribute last/next sync timestamps
  into Bookmark Sync section as a compact two-line block below the
  frequency ListItem
- Add Bookmarks stats (My List / Archived / Favorites) using CompactStatRow
  in Bookmark Sync section for library context
- Add ContentSyncStatusIndicator: surfaceVariant card with leading icon
  and bodySmall status text, absorbing the previous floating status text
  and purging indicator; error states use error color, syncing uses primary
- Add 'What to keep offline' / 'Whether to download' / 'Storage' section
  subheadings using labelLarge for correct MD3 hierarchy
- Replace StatusRow (ListItem-based) with CompactStatRow (flush label +
  right-aligned onSurfaceVariant value, no minimum height) in storage stats
- Move Clear All Offline Content button inside AnimatedVisibility block
- Remove trailing HorizontalDivider after OfflineReadingSection
- Fix timestamp format: DateFormat.SHORT for time omits seconds
- Fix all hardcoded strings: 6 new resource keys added to all 10 language files
- Update BookmarkRepositoryImplTest for offline reading settings compatibility
- Update GitHub workflow configs
- Mark Slice 8 complete in implementation plan; add spec amendment

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
UI tweaks:
- subheading labels (Bookmarks, What to keep offline, Whether to
  download, Storage) now use labelLarge + primary color per MD3
  list-subheader pattern
- group Sync frequency ListItem and Last/Next sync timestamps in a
  tight inner Column to remove the spurious gap caused by ListItem's
  internal padding
- restructure Bookmarks heading+stats to match Storage exactly:
  Text(padding vertical 8dp) + Column(32dp indent, spacedBy 4dp)
- remove trailing HorizontalDivider after Offline Reading section
- move Last/Next sync out of ListItem supportingContent to avoid
  text wrapping next to the trailing frequency control

Content changes:
- remove test storage limits (5 MB, 10 MB, 20 MB) from enum,
  ViewModel, dialog, string resources, and all language files
- add 500 as a Most recent bookmark count option
- rename "Newest N bookmarks" to "Most recent" across all files
- rename "Include archived bookmarks" to "Include Archive"
- remove "policy" from user-visible strings; update max storage cap
  description to reflect renamed policy labels
- update section and toggle descriptions to be non-redundant and
  avoid technical jargon

User guide (settings.md):
- restructure Offline Reading section to match new subheading layout
- fix Bookmarks counts position (above button, not below)
- replace "offline packages" and "on-demand text caches" with plain
  language throughout

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace broken !github.event.pull_request guard (always true on push
events) with an explicit condition: run on workflow_dispatch, any
pull_request, or pushes to main only. Feature branch pushes without
an open PR no longer trigger checks or snapshot builds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… (Slice 13)

Expand OfflinePolicyEvaluatorTest with boundary condition coverage for
all three policies (STORAGE_LIMIT, NEWEST_N, DATE_RANGE) including
secondary cap interactions, and add BatchArticleLoadWorkerTest covering
the download-prune cycle, priority path, constraint gating, and error
handling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Without weight(1f), a long site name consumed all available row width,
leaving 0dp for the reading time Text. Since that Text had no maxLines
limit, Compose wrapped it one character per line, inflating the site row
height ~144dp and pushing labels and action buttons off the bottom of the
fixed-height card.

Restore Modifier.weight(1f) on siteName in all five affected card
variants (Grid: MobilePortrait, Narrow, Wide; Compact: Narrow, Wide) so
reading time and download status icon are measured at their natural widths
first. Also add maxLines=1 to reading time Text as a defensive guard.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ghts list

- toc-fragment-link-navigation-spec.md: production-ready spec with full
  WebViewTocBridge source, exact injection point, and step-by-step plan
- highlights-nav-drawer-list-spec.md: supersedes _notes draft; addresses
  typed navigation routes, ARTICLE-gate bug fix, video annotation refresh
  fix, scroll-to-annotation wiring, and rail/drawer integration detail

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…otes

Move implemented branch-specific specs to docs/archive/:
- sync-multipart-adoption-spec.md (all stages complete)
- granular-offline-content-management-spec.md (implemented 2026-03-26)
- highlight-compose-fix.md (implemented and verified)
- branch-guardrails.md, branch-pre-merge-fixes.md,
  branch-pre-merge-guardrails-impl.md, branch-validation-tests.md
  (guardrails implemented; tests exist)
- branch-summary-sync-multipart-v012.md (historical record)

Remove intermediate/superseded notes from git (preserved locally in _notes/):
- CHAT_PROMPT_ALT_SYNC.md, CHAT_branch-architecture-analysis.md,
  CHAT_main-branch-architecture-analysis.md (transient analysis artifacts)
- managed-offline-reading-spec.md (superseded by unified-offline-content-spec.md)
- granular-offline-content-followup-spec.md (superseded by unified-offline pair)
- managed-offline-reading-progress.md (progress tracker for superseded spec)

Remaining in docs/specs/: 11 active specs (unimplemented or in-progress).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- workflow_overhaul_summary.md → docs/archive/ (WORKFLOW.md already
  reflects all changes described; summary is a historical record)
- TODO-post-sync-branch.md → docs/specs/incremental-content-rendering-spec.md
  (rename to describe the feature: progressive article content loading)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Full offline reading experience (batch content download including images and highlights)

2 participants