The design brain: toggleable BI/UX intelligence for dashboard generation (v1 + verified v2 iteration)

[debabsah]

What

Chartwright guaranteed a dashboard imports correctly; this adds the layer that makes it read well — a codified, continuously-improvable body of BI/UX design knowledge (Few, Tufte, IBCS, plus Superset rendering facts), on by default and off with one flag. Reference: docs/DESIGN-BRAIN.md. The branch lands the feature twice over: the initial three-phase implementation, then a full second iteration (design brain "2") executing every item of an adversarially-verified review of the first (docs/DESIGN-BRAIN-V2.md).

How it works

One rulebook, two delivery mechanisms, split by who can apply the rule:

• Tier G — the brief (chartwright brief): audience-tuned design guidance (executive / analytical / operational presets) read before authoring a spec: a canonical sketch to start from, compiler-defaults table, budgets, chart choice, composition. Improvable by editing markdown.
• Tier L — the critic (chartwright advise): a 47-rule deterministic linter over the finished spec — readable minimum geometry, KPI-first composition, fold budgets, chart/filter/pivot/color limits, narrative polish — with a presentation-only autofix subset (--fix, convergent, idempotent, and disclosing every diff as {set, was}). check/apply carry advice by default (--design off|warn|strict).

The surface

chartwright brief [--audience A]
chartwright advise <spec> [--profile P] [--fix] [--strict] [--no-probe]
chartwright redesign <slug-or-id> --profile P     # decompile -> audit -> fix, one shot
chartwright calibrate [--write] [--since 90d]     # learn heights from human polish

• Data-aware mode: the resolver keeps column types (temporal-axis check = error) and bounded cardinality probes catch spaghetti series, 40-slice pies, 400+-cell heatmaps, unusable filter pickers.
• House style: design.yaml overlay — parameter overrides, per-audience calibrated heights, rule disables, per-deployment severity overrides, extra brief guidance — every value validated at load with typed errors.
• The learning loop: absorb logs the heights humans drag charts to (with audience); calibrate mines them (one vote per chart, --since decay) into recommendations the brief prints and autofixes target.
• Redesign: tool-born dashboards redesign in place; UI-born ones land side by side under a -redesign slug — the original is never overwritten.
• MCP server mirrors it all (design_brief, advise_spec, fix_spec, redesign_dashboard, advice on check_spec).

The v2 iteration

A seven-lens review of the initial implementation (each lens executing the code against scratch specs; every bug/edge-case/doc-drift claim independently re-verified with a refute-by-default stance) produced 80 confirmed findings, 0 refuted, ranked into 37 roadmap items — all executed in the four batch A–D commits:

• Batch A — 15 verified bugs & false positives: an absorb crash that mutated spec files before dying; the autofix echo chamber (tool-written fractional heights forged the human-polish signature and blinded the size rules); overlay trust-boundary validation; sketch stacks / KPI sidebars falsely condemned (rules now count horizontal slots); the pie rule that taught truncated-pie lies.
• Batch B — 9 knowledge packs: pivot_table (was a total blind spot), the filter bar, per-type point budgets, top-N-without-sort, ordinal ordering, format consistency, first color rules, brief upgrade.
• Batch C — architecture & provenance: validated ignores (unmatched_ignores, scoped band keys), chart-type taxonomy contract test (a 15th type fails CI until classified), rule since/alias metadata, severity overrides, calibration v2, memoized sketch geometry (was ~cubic), fix diffs, WYSIWYG disclosure on sketch fixes.
• Batch D — docs made true and rot-proof: the rule catalog in DESIGN-BRAIN.md §7 is generated from the live registry; deviations from the review's proposals recorded in §15.

Guarantees kept

• The LLM stays the untrusted parser at the edge; everything enforceable is tested code.
• Autofixes never change what data a chart shows, never rename a chart (uuid identity), never write fractional heights.
• Human polish outranks the brain: absorbed heights silence height opinions (and only those).
• Suppression is always visible: ignored and unmatched_ignores in every report.
• --design off is byte-identical to pre-brain behavior; spec_version unchanged; compiled bundles untouched.

Testing

213 tests pass (95 added across the branch): per-rule tables, fix-loop convergence/idempotence/no-fractional invariants, seeded advise-never-raises fuzz, taxonomy and cache contracts, design.yaml trust-boundary rejections, calibration grouping/decay/round-trips, the golden dogfood (the shipped example advises clean, its two deliberate exceptions recorded in its design block), and MCP parity.

Deferred, tracked in the doc: live-Superset UI verification (side-by-side advised-vs-unadvised screenshots); thresholds come from field notes and BI literature. Version bumped to 0.3.0.

[debabsah]

Pre-merge review pass: two commits pushed

A full review of this branch (core + design brain) turned up seven items
worth fixing before merge. Each is reproduced by a test that fails without
the fix.

1. CI runs on pull requests. Both matrices only ran on push to main, so
this 4,900-line branch had never been through either. This is its first run.

2. sort_by actually sorts. It compiled to order_desc: true and
nothing else, a direction with no column, so a "top 10" table returned ten
arbitrary rows. Three symptoms from one field: the compiler dropped it, the
resolver never checked it (a typo passed check), and plan reported the
chart changed forever because decompile could not produce a field the
compiler never wrote, which broke the CI drift gate for any sorted table.
Aggregate mode now emits timeseries_limit_metric, raw mode
order_by_cols; both are declared by the Table plugin on 4.1.4, 5.0.0 and
6.1.0, and params_drift stays clean. Made worse by this branch: the new
data.top-n-sort rule told users to set the field.

3. --design strict fails closed. Advice degrades to an error note
rather than crashing check/apply, which is right for warn. Under strict
it meant counts of zero, so one typo in an org-wide design.yaml silently
disarmed the gate everywhere it was used.

4. Rows of equal markdown blocks no longer overflow the grid. Blocks
compare by value, so every identical block resolved to the first one's index,
took the largest-remainder +1, and pushed the row past 12 columns. A
regression from this branch's own width fix; positions now match by identity.

5. One grid model, three consumers. size.table-window (0.8 units/row +
1 header), size.grid-fit (0.75 + 3) and smoke (30px/row + 3x40px) each
measured a rendered row differently, so a 20-row table at height 8 passed
offline while the data-aware rule and the apply warning both said it hid ten
rows: the exact defect this branch's newest commit exists to catch. The
constants live once in chartwright/spec.py. table_visible_ratio is now
0.5 for every audience (was 0.25 on analytical/operational), because below
half the majority of the rows the author asked for sit behind the inner
scrollbar.

6. The polish skip is disclosed as polished in the advice payload.
Deferring to a human-dragged height is an INFERRED signal, and v2 theme 1
named it alongside unmatched_ignores and the strict-gate ok; only the
other two landed. Silence there reads exactly like the rule having passed.

7. Doc counts are true and enforced. "125 tests" outlived four batches
across five locations. v2 roadmap item 20 flagged it and only half-landed
while DESIGN-BRAIN-V2.md asserted all 37 items were done. Counts now live in
one row of VERIFICATION.md that tests/test_docs.py checks against what
pytest actually collects, and the v2 status line records the miss.

The PR trigger caught a real failure on its first run

The offline job failed on Linux and Windows: tests/test_design_rules_v2.py
imported from tests.test_design_data import ..., which resolves only under
python -m pytest (which adds the CWD) and never under the bare pytest
console script ci.yml runs. tests is not a package, so that module was
uncollectable in CI while passing locally. Fixed in the second commit, with a
test asserting no module imports through the tests. package. This would
have broken main on merge.

Verification

236 offline tests pass under both pytest invocations, Linux and Windows.
params_drift clean against 4.1.4, 5.0.0 and 6.1.0. Live apply, soak,
adversary and fault injection green on all three versions, with the
kitchen-sink fixture now carrying a sorted table.

Scope note on item 2: live CI proves Superset accepts timeseries_limit_metric
on import and that the chart still returns rows on all three versions. It does
not assert the rows come back in sorted order, since smoke checks non-empty
rather than ordering. That rests on the plugin contract.

Known gaps, deliberately not closed here

• The design brain has no rendered-output verification. Every Tier L rule
  reads the spec, not the rendered result; the thresholds come from field
  notes and BI literature, not measured pixels, and docs/CONTRACTS.md has
  no design entries, unlike every other Superset behaviour this repo relies
  on. A visual harness over the existing sandbox is the natural next branch.
• Cardinality probes ignore each chart's own filters and time range (false
  positives on well-scoped charts) and are bounded in result, not in
  warehouse work.
• advise --strict prints ok: true while exiting 1, where
  check --design strict sets ok: false.
• The MCP error fallback hardcodes design_brain: "1".
• No rule covers the commonest real failure: a timeseries with no
  time_range, default P1D grain, and no defaulted dashboard filter, drawing
  full history daily. data.grain-vs-range only fires when time_range is
  set.