Skip to content

Chart-position watchdog + browser notifications#2

Merged
bootuz merged 3 commits into
mainfrom
feat/chart-watchdog
May 23, 2026
Merged

Chart-position watchdog + browser notifications#2
bootuz merged 3 commits into
mainfrom
feat/chart-watchdog

Conversation

@bootuz
Copy link
Copy Markdown
Owner

@bootuz bootuz commented May 23, 2026

Summary

A daily background job that polls Apple's free iTunes RSS top-free charts for each watched app's primary genre across the storefronts where the app is available, persists what changed, and pushes browser notifications to any open dashboard tab. New `Charts` toolbar button opens a full-screen page with the Currently charted cards and a Recent activity feed.

Designed during a brainstorm session — the spec lives at `/Users/astemirboziev/.claude/plans/in-the-modal-for-validated-conway.md`. Key choices:

  • Watchdog scope, not browser: this never tries to show "the full top-100 in country X"; it only emits events when one of your apps appears in one. Mostly silent for an app at Azri's scale today; high-signal when something does chart.
  • Free data only: Apple's RSS feed (capped ~100/chart) is the single source of truth. No third-party API integration, no paid tier.
  • No service worker: notifications only fire when a Keywordista tab is open. Worth it later but not for v1.
  • Schema bounded: snapshot table holds only the latest position per `(app, country, chart, genre)` tuple — bounded by apps × countries, not by time. `chart_event` is append-only.

Architecture

```
Apple iTunes RSS ──┐

RefreshChartsScheduler (daily, 04:00 UTC)


ChartTrackerService ── diff vs chart_position_snapshot ── emit ChartEvent


GET /api/v1/chart-events?since=… ◀── SPA polls every 30s ──── new Notification(…)
```

Backend additions:

  • 3 new models + migrations (`AppStorefrontAvailability`, `ChartPositionSnapshot`, `ChartEvent`) plus a `primary_genre_id` column on `WatchedApp`.
  • `AvailabilityProber` (175-storefront iTunes lookup, batched at 8 concurrent), `ITunesChartsClient` (timeout-wrapped RSS fetch), `ChartTrackerService` (per-(app,country) Fluent transactions).
  • 4 new routes under `/api/v1` (chart-positions / chart-events / charts/refresh / apps/:id/availability/refresh).
  • Composition wired in `Container.swift`; `AppsController.create` now kicks off an availability probe in a detached Task.

Frontend additions:

  • `ChartsPage.svelte` + `ChartPositionCard.svelte` + `ChartActivityRow.svelte`.
  • `lib/notifications.ts` (Notification API wrapper with localStorage opt-out memory) and `lib/chartEvents.ts` (singleton 30s poll, dedupe by event id, `lastSeenIso` persistence).
  • Charts button + unread badge in the dashboard toolbar.

Diff algorithm

previous snapshot new poll event emitted snapshot action
no row / position=null found at #N `entered` upsert position=N
#M found at #N (M≠N) `moved` update position=N
#M not found `exited` update position=null
no row / position=null not found (none) no-op
#M found at #N (M=N) (none) bump `observed_at`

Pure-function form (`decideChartTransition`) extracted so all 7 cases are unit-tested without a DB.

Test plan

  • `swift test` — 64 tests pass (11 new + 53 existing). New tests cover the 7-case diff matrix and RSS envelope decoder.
  • `npm run check` — 0 errors, 0 warnings across 144 files.
  • End-to-end against local stack:
    • Backend migrations applied cleanly on existing `db.sqlite`.
    • `POST /api/v1/charts/refresh` against Azri → ChartTracker logged `apps=1 charts=174 events=0` (correctly silent — Azri isn't in any top-100).
    • `WatchedApp.primary_genre_id` backfilled to `6017` automatically on first refresh.
    • Synthetic-state diff: inserted a fake "Azri was #50 in US Education" snapshot, re-ran refresh → `exited` event emitted with `prev_position=50, position=null`, surfaced by `GET /chart-events`.
  • Reviewer: open `/charts` in a browser, click Enable on the permission CTA, click Check now (~30s for 174 fetches). Empty state should remain unless Azri unexpectedly charted.
  • Reviewer (optional): temporarily add a watched app that is charted (e.g. Duolingo, appStoreId=570060128) and watch the first refresh emit several `entered` events with toasts.

Follow-ups (out of scope)

  • Position history / sparklines / delta vs yesterday. Needs an append-only `chart_position_history` table; cheap to add but separate from the watchdog itself.
  • Multi-chart-type / secondary-genre watching. Today MVP watches top-free × primary genre only. Adding top-paid / top-grossing or a secondary genre is a column on `chart_position_snapshot`, no schema migration.
  • Service worker push. Notifications when the Keywordista tab is closed.
  • Notification grouping / digest mode. Each event currently fires its own toast — fine at low volume.

bootuz added 3 commits May 23, 2026 17:23
Daily Queues job that polls Apple's iTunes RSS top-free chart for each
watched app's primary genre across the storefronts where the app is
available, and emits ChartEvent rows on every transition (entered /
moved / exited). The watchdog stays silent for never-charted apps —
Azri isn't in any chart's top-100 today but will be eventually, and
the dashboard wants to know the moment that changes.

- Three new tables: app_storefront_availability, chart_position_snapshot,
  chart_event. Snapshot rows hold only the latest position per
  (app, country, chart, genre) tuple so storage stays bounded.
- WatchedApp gains primary_genre_id (backfilled by AvailabilityProber
  on next probe or lazily by ChartTrackerService on first refresh).
- AvailabilityProber: batched iTunes lookup against all 175 storefronts;
  triggered from AppsController.create and POST /apps/:id/availability/refresh.
- ChartTrackerService: per-(app, country) Fluent transaction wraps each
  diff so an interrupted job can't desync the snapshot from the event
  log. Diff logic extracted into a pure `decideChartTransition` for
  unit testing.
- ITunesChartsClient: thin wrapper around the legacy
  /<cc>/rss/topfreeapplications/limit=200/genre=<id>/json endpoint
  (the v2 applemarketingtools API doesn't support a genre filter).
  Static `parseEntries` exposed for tests.
- RefreshChartsScheduler runs daily at 04:00 UTC (an hour after the
  keyword refresh and a few hours past Apple's PT-midnight chart
  refresh window so the RSS feeds have settled). POST /charts/refresh
  spawns the same work via a detached Task.
- Four new routes: GET /chart-positions, GET /chart-events?since=&limit=,
  POST /charts/refresh, POST /apps/:id/availability/refresh.
- Tests cover all 7 transition cases in the diff matrix plus the RSS
  envelope decoder.
Surfaces the chart-watchdog from the backend. A new "Charts" button in
the dashboard toolbar (with an unread-count badge fed by a 30s polling
loop) opens a full-screen ChartsPage with three states:

- Empty + permission CTA: first visit asks for Notification permission;
  the Later button is remembered via localStorage so we don't re-prompt.
- Currently charted: cards showing rank + flag + country + genre +
  "seen X ago" for each non-null snapshot row.
- Recent activity: feed of entered/moved/exited events with relative
  timestamps and a glyph per transition kind.

A "Check now" button POSTs to /api/v1/charts/refresh and reloads after
the backend job settles; "Re-probe availability" loops over watched
apps and kicks /apps/:id/availability/refresh for each.

chartEvents.ts owns the singleton 30s poll, deduping by event id and
persisting lastSeenIso so reloads never re-fire stale notifications.
notifications.ts wraps the Notification API with a graceful fallback
when the user denies or the browser doesn't support it — the activity
feed remains the durable record.

Genre IDs decoded inline (small static map) since the backend stores
the numeric id; only common App Store categories are hardcoded, the
rest fall back to "Category #N".
Both directories hold per-session local state (Claude Code's launch
config, the brainstorming companion's HTML mockups) and shouldn't be
committed. Silences the 'uncommitted changes' warning gh kept emitting
on push and pr create.
@bootuz bootuz merged commit c11ce80 into main May 23, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant