feat(stats): server Stats page, character Lifetime panel + live stat Explorer

.github/workflows/ci.yml

@@ -131,7 +131,40 @@ jobs:
 
       - name: npm audit (frontend production deps)
         working-directory: frontend
-        # --audit-level=high → exit 1 only on high or critical; moderate/low
-        # are reported but don't fail the build. --omit=dev skips devDeps
-        # (vitest, ESLint, etc.) — same rationale as pip-audit's --no-dev.
-        run: npm audit --audit-level=high --omit=dev
+        # Same policy as the bare `npm audit --audit-level=high --omit=dev`
+        # this replaces: only high/critical advisories fail the build, and
+        # devDeps are skipped (same rationale as pip-audit's --no-dev). npm
+        # audit has no native ignore flag, so this mirrors .pip-audit-ignore:
+        # frontend/.npm-audit-ignore lists GHSA IDs (first token per line,
+        # rest is rationale) for accepted risks / not-applicable advisories.
+        run: |
+          npm audit --json --omit=dev > /tmp/npm-audit.json || true
+          # A registry/network failure must fail loudly, not pass silently.
+          jq -e '.vulnerabilities != null' /tmp/npm-audit.json > /dev/null || {
+            echo "::error::npm audit did not produce a report"
+            cat /tmp/npm-audit.json
+            exit 1
+          }
+          jq -r '[.vulnerabilities[]?.via[]? | objects
+                  | select(.severity == "high" or .severity == "critical")
+                  | (.url // empty)] | unique | .[] | sub(".*/"; "")' \
+            /tmp/npm-audit.json > /tmp/npm-audit-ids
+          : > /tmp/npm-audit-ignored
+          if [ -f .npm-audit-ignore ]; then
+            grep -vE '^[[:space:]]*(#|$)' .npm-audit-ignore | awk '{print $1}' > /tmp/npm-audit-ignored || true
+          fi
+          FAIL=0
+          while IFS= read -r id; do
+            [ -z "$id" ] && continue
+            if grep -qx "$id" /tmp/npm-audit-ignored; then
+              echo "allowlisted by .npm-audit-ignore: $id"
+            else
+              echo "::error::high/critical advisory not allowlisted: https://github.com/advisories/$id"
+              FAIL=1
+            fi
+          done < /tmp/npm-audit-ids
+          if [ "$FAIL" = "1" ]; then
+            npm audit --audit-level=high --omit=dev || true
+            exit 1
+          fi
+          echo "npm audit: no unallowlisted high/critical advisories"

CLAUDE.md

@@ -36,6 +36,7 @@ A web companion site for EverQuest 2 (TLE), with a Discord bot for spot checks (
 | `backend/server/api/characters.py` | GET /api/characters/search — character name search |
 | `backend/server/api/guild_officer.py` | Officer claim-review endpoints; imports _officer_chars, _roster_rank_map from guild.py |
 | `backend/server/api/item_watch.py` | Item watch endpoints; imports _officer_chars, _roster_rank_map from guild.py |
+| `backend/server/api/stats.py` | Per-server Stats page from the census `character.stat` aggregate family (global/world/world×class rows, 12 lifetime stats as max/sum/avg, recomputed daily by census). `GET /api/stats/server` = population, class distribution, totals, named leaderboards (single-value stats census-sort; K/D sorts `.value` with a 1000-kill floor; the two-key hit records can't census-sort — range-filter near the aggregate max + client-side sort). `GET /api/stats/character/{name}` = lifetime panel w/ ability names via spells.db `find_by_crc` (crc 4294967295 = sentinel, skip). classid = classes.db `icon_id` (Templar=13). 6h SWR module cache per world; leader queries retry (census drops back-to-back sorted queries). Never builds inline: cold cache → 202 `{"status":"building"}` + background build (lock-deduped, frontend polls 5s); startup prewarm via `prewarm_server_stats()` in app.py lifespan; failed build → 60s cooldown → 503. `GET /api/stats/explore?stat=&cls=` = live top-20 by whitelisted stat (`_EXPLORE_STATS`: combat snapshots under `stats.combat.*`/health/power + progression scalars quests.complete / collections.complete / achievements.points/.completed / alternateadvancements.spentpoints, projected narrowly — never `c:show=achievements`, it drags the 500+-item list + lifetime `statistics.*`), optional class scope — census needs numeric `type.classid` (string `type.class` filters silently return empty); 15-min cache, stale-beats-error. |
 | `backend/server/census_health.py` | Site-wide Census availability signal: background poll every 5 min; `is_down()`/`get_state()` read by the read/refresh paths. |
 | `backend/server/census_events.py` | In-process async pub/sub backing the SSE stream (single-process only). |
 | `backend/server/census_refresh.py` | Background refresh orchestration (throttle 15 min / in-flight dedupe / skip-when-down); merges into census_store, updates cache, publishes SSE. `_merge_roster` best-known join. |

backend/census/client.py

@@ -452,6 +452,110 @@ async def get_gear_sets(self, character_id: str | int) -> list[GearSet] | None:
             sets.append(GearSet(name=name, equipment=equipment))
         return sets
 
+    # ── Lifetime statistics (the character.stat aggregate family) ────────────
+
+    @staticmethod
+    def _list_of(data: dict) -> list:
+        """Census wraps rows in a '<collection>_list' key — grab it by suffix."""
+        for key, value in data.items():
+            if key.endswith("_list") and isinstance(value, list):
+                return value
+        return []
+
+    async def get_stat_aggregates(self) -> dict | None:
+        """The three character.stat aggregate collections, refreshed daily by
+        Census: per-class global rows (id=classid or 'all'), per-world rows
+        (id=worldid), and the world×class matrix (id='worldid.classid',
+        paginated at 100/request). Returns
+        ``{"global": [...], "world": [...], "world_class": [...]}`` or None
+        when any fetch fails (partial aggregates would mislead)."""
+        # The matrix is ~1095 rows paginated at 100/request — fire the whole
+        # page fan (plus global+world) concurrently instead of serially; the
+        # build drops from ~10s to one census round-trip.
+        page_starts = list(range(0, 1300, 100))
+        results = await asyncio.gather(
+            self._census_get("character.stat.global/", {"c:limit": "40"}),
+            self._census_get("character.stat.world/", {"c:limit": "80"}),
+            *[self._census_get("character.stat/", {"c:limit": "100", "c:start": str(s)}) for s in page_starts],
+        )
+        g, w, *pages = results
+        if g is None or w is None:
+            return None
+        world_class: list = []
+        for p in pages:
+            if p is None:
+                return None
+            world_class.extend(self._list_of(p))
+        return {"global": self._list_of(g), "world": self._list_of(w), "world_class": world_class}
+
+    async def get_stat_leaders(
+        self,
+        world: str,
+        sort_path: str,
+        limit: int = 10,
+        extra_filter: dict[str, str] | None = None,
+        show: str = "statistics",
+    ) -> list[dict] | None:
+        """Characters on ``world`` ordered by a census field, name+class+
+        value shape. ``sort_path`` is the census sort key (single-value stats
+        sort on 'statistics.kills'; ratio-style need '...value'). ``show``
+        selects the payload subtree ('statistics' for lifetime stats,
+        'stats' for combat snapshots). None on census failure."""
+        params = {
+            "locationdata.world": world,
+            "c:sort": f"{sort_path}:-1",
+            "c:show": f"name.first,type.class,type.level,{show}",
+            "c:limit": str(limit),
+            **(extra_filter or {}),
+        }
+        data = await self._census_get("character/", params, timeout_s=20)
+        return self._list_of(data) if data is not None else None
+
+    async def get_stat_range(
+        self, world: str, stat_value_path: str, minimum: float, limit: int = 200
+    ) -> list[dict] | None:
+        """Characters on ``world`` whose statistic exceeds ``minimum`` —
+        unsorted (census can't sort two-key stat objects like max_melee_hit;
+        the caller sorts client-side). None on census failure."""
+        params = {
+            "locationdata.world": world,
+            stat_value_path: f"]{minimum:.0f}",
+            "c:show": "name.first,type.class,type.level,statistics",
+            "c:limit": str(limit),
+        }
+        data = await self._census_get("character/", params, timeout_s=20)
+        return self._list_of(data) if data is not None else None
+
+    async def get_character_statistics(self, name: str, world: str) -> dict | None:
+        """One character's lifetime ``statistics`` object (kills, deaths,
+        biggest hits w/ ability crcs, crafts, rare harvests). None when the
+        character is unknown or census fails."""
+        params = {
+            "name.first": name,
+            "locationdata.world": world,
+            "c:show": "name.first,type.class,statistics",
+            "c:limit": "1",
+        }
+        data = await self._census_get("character/", params)
+        if data is None:
+            return None
+        rows = self._list_of(data)
+        return rows[0] if rows else None
+
+    async def get_worldid(self, world: str) -> int | None:
+        """Resolve a world name to its numeric census id (the key the
+        character.stat aggregates use) by probing one character's
+        locationdata. None when the world has no census characters."""
+        params = {"locationdata.world": world, "c:show": "locationdata.worldid", "c:limit": "1"}
+        data = await self._census_get("character/", params)
+        if data is None:
+            return None
+        rows = self._list_of(data)
+        try:
+            return int(rows[0]["locationdata"]["worldid"]) if rows else None
+        except (KeyError, TypeError, ValueError):
+            return None
+
     async def get_character_aas(self, name: str, world: str) -> CharacterAAs | None:
         params = {
             "name.first": name,