Skip to content

Dashboard aggregations + six-category taxonomy for watercolor FE#3

Merged
cmitchell8 merged 2 commits into
mural-mainfrom
taxonomy/dashboard-aggregations
May 13, 2026
Merged

Dashboard aggregations + six-category taxonomy for watercolor FE#3
cmitchell8 merged 2 commits into
mural-mainfrom
taxonomy/dashboard-aggregations

Conversation

@cmitchell8
Copy link
Copy Markdown
Collaborator

Summary

Implements the BE side of prd-fe-dashboard-aggregations-and-category-taxonomy.md so the watercolor FE Home screen has live counts and a tile-click filter to Browse against the same mapping.

  • New CategoryMapping model (cwe, tag_pattern, scanner, category, save()-computed specificity, updated_at) plus migration 0265_categorymapping that loads the 22-row starter fixture (dojo/fixtures/category_mappings.json).
  • New resolver dojo/taxonomy/categorize.py:
    • categorize(finding) returns a category id, iterating the cached mapping list in Python.
    • categorize_queryset(qs) annotates category in a single SQL pass via a CASE WHEN derived from the mapping table — no Python iteration on the annotation path.
    • Default fallback is web. Tie-break: higher specificity > more recent updated_at.
  • Three aggregation endpoints under /api/v2/dashboard/:
    • queue-counters/{critical, high, sla_breach, fixed_this_week}
    • category-counts/[{id, count}] always six entries in fixed order
    • search-totals/{findings, categories} (categories capped at 8)
  • All three scope through dojo.finding.queries.get_authorized_findings, are cached per-user for 60s, and key on MAX(finding.updated) for implicit invalidation.
  • ?category=<id> filter added to the findings list (ApiFindingFilter.filter_category); repeated values OR together. Defers to categorize_queryset so the Browse rows always match the tile count.
  • CategoryMappingAdmin with list_filter on category and a "Test this mapping" admin action that runs categorize() against the 20 most recent findings using only the selected rules.
  • unittests/test_dashboard_aggregations.py covers every seed rule, specificity tie-break, the three endpoints at 0/1/100 findings, the ?category= filter, and permission isolation between two users on disjoint products.

Deviations from the PRD

  • sla_breach uses Finding.sla_expiration_date (present on this fork). The view wraps the query in a try/except and returns 0 as the PRD-documented fallback if the lookup ever errors.
  • fixed_this_week uses Finding.mitigated directly. This fork does not maintain a status-history table covering the full Mitigated/Inactive/False-Positive transition set, so the count is an undercount relative to the PRD's ideal. The OpenAPI description on the endpoint says so explicitly.
  • The PRD names the migration path 0XXXX_category_mapping.py; in this fork the next available numeric prefix is 0265.
  • The PRD references dojo/api_v2/urls.py as the place to register routes; in this fork API routes are registered on the v2_api DefaultRouter in dojo/urls.py, which is what this PR uses.
  • Authorization helper is get_authorized_findings(Permissions.Finding_View, user=...) under dojo/finding/queries.py, not under dojo/authorization/.

Test plan

  • python manage.py migrate dojo runs cleanly and loads the 22 starter rows.
  • GET /api/v2/dashboard/queue-counters/ returns the four-key payload and the critical count matches Finding.objects.filter(active=True, severity='Critical') for the calling user.
  • GET /api/v2/dashboard/category-counts/ returns exactly six rows in the fixed order web, cloud, supply, identity, data, config.
  • GET /api/v2/dashboard/search-totals/ returns {findings, categories} with categories <= 8.
  • GET /api/v2/findings/?category=web returns only findings whose annotated category is web; multi-value ?category=web&category=cloud ORs them.
  • In Django admin, "Test this mapping" against selected rows produces one message per sampled Finding with the resolved category id.
  • Edit CategoryMapping row routing CWE-79 to data, hit category-counts/ after the 60s TTL elapses or with a busted cache key (any finding mutation) — counts shift accordingly. Revert.
  • unittests/test_dashboard_aggregations.py passes locally.
  • drf-spectacular schema regenerates cleanly with the three new endpoints.

🤖 Generated with Claude Code

Implements the BE side of the watercolor FE Home screen. Adds a
deterministic six-category taxonomy (web/cloud/supply/identity/data/
config), three aggregation endpoints under /api/v2/dashboard/, and a
?category= filter on the existing findings list — all sharing the same
resolver so a tile count and the Browse rows behind it never disagree.

Model
- CategoryMapping in dojo/models.py with (cwe, tag_pattern, scanner)
  unique_together, choice-constrained category id, and save()-computed
  specificity (CWE=3 + tag=2 + scanner=1 summed).
- Schema migration 0265_categorymapping with a RunPython step that
  loaddata's the 22-row starter fixture covering the CWEs / tags /
  scanners called out in PRD §4. Reversible.
- dojo/fixtures/category_mappings.json seed file.

Resolver
- dojo/taxonomy/categorize.py exposes categorize(finding) for a single
  Finding (iterates the cached mapping list) and categorize_queryset(qs)
  which annotates `category` in a single SQL pass via CASE WHEN built
  from the mapping table (no per-row Python iteration on the annotation
  path).
- Tie-break: higher specificity wins; on equal specificity the row with
  the most recent updated_at wins. Default fallback is "web".

API
- dojo/api_v2/dashboard/ package with three viewsets routed at
  /api/v2/dashboard/queue-counters/, /category-counts/, /search-totals/.
- All three scope through dojo.finding.queries.get_authorized_findings so
  product-membership permissions apply.
- Per-user 60s cache keyed on a hash of MAX(finding.updated) so the
  cache invalidates implicitly on any finding mutation.
- sla_breach uses Finding.sla_expiration_date which exists on this fork;
  fixed_this_week uses Finding.mitigated (no status-history table on
  this fork — documented in the OpenAPI description as an undercount).

Filter
- ApiFindingFilter gains a `category` filter that defers to
  categorize_queryset(); repeated ?category= params OR together.

Admin
- CategoryMappingAdmin with list_filter on category and a
  "Test this mapping" action that runs categorize() against the 20 most
  recent findings using only the selected rules and surfaces the
  resolutions as Django messages.

Tests
- unittests/test_dashboard_aggregations.py covers every seed-fixture
  rule, the specificity tie-break, the three endpoints at 0/1/100
  fixture sizes, the ?category= filter, and permission isolation
  between two users on disjoint products.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ruff `--fix` autofixes (docstring D2xx formatting, import sorting, unused
imports, ternary simplification, unused noqas, etc.) plus four manual
touch-ups:

- dojo/models.py: reorder CategoryMapping so `save()` precedes
  `compute_specificity()` to satisfy DJ012 (Django style guide).
- dojo/taxonomy/categorize.py: move `Iterable` import into a
  `TYPE_CHECKING` block (TC003).
- unittests/test_dashboard_aggregations.py: make `_build_finding`'s
  `active` keyword-only (FBT002); add `# noqa: S106` on the three
  test-only fixture passwords (the per-file rule only covers S105).

`ruff check .` now passes locally on this branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cmitchell8 cmitchell8 merged commit ebbafe5 into mural-main May 13, 2026
8 checks passed
cmitchell8 added a commit that referenced this pull request May 13, 2026
PR #3 (dashboard aggregations) landed `0265_categorymapping` first, so
this branch's migration is renumbered `0265_findingbookmark` ->
`0266_findingbookmark` and its `dependencies` now points at
`0265_categorymapping` to keep the Django migration graph linear.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant