Ref Ball? 🏀

Quantifying Referee Impact Through Fouls and Point Differential on NBA Playoffs

Refs → fouls → point differential

An open, reproducible, observational sports-analytics project that asks a popular fan question honestly:

Are certain referees associated with NBA playoff foul patterns in ways that appear to "swing" outcomes?

Important

This project makes no claim that any referee, crew, team, or league fixed, rigged, swung, or manipulated a game, or acted with intent.

---

What it measures

For each playoff game we model, with full Bayesian uncertainty:

1. Total foul volume — is an official associated with more/fewer total fouls?
2. Directional foul lean — is an official associated with a home_pf − away_pf margin? (Positive lean = the home team is whistled more, not favored.)
3. Free-throw margin and foul margin → point differential — how does the whistle show up on the scoreboard, controlling for the line?
4. Mediated effect — the composed referee → foul-margin → scoreboard path, propagated over the entire posterior.

The modeling spine:

Referee crew  →  foul environment / foul margin  →  free throws + game flow  →  point differential

…which is not clean causal identification, because referee assignment is not random. See Methodology & Limitations.

---

Why it matters

"The refs decided that one" is one of the most common claims in sports — and one of the least often tested rigorously. This project builds a public, reproducible framework for turning that claim into an estimate with an honest uncertainty interval: partial pooling so small samples don't dominate, multi-membership effects so a three-person crew is modeled correctly, and placebo/robustness checks so a fluke doesn't masquerade as a finding.

---

Architecture

src/refball/
  config.py                 # pydantic-settings: paths, seasons, priors, sampler knobs
  schema.py                 # canonical column contracts (source-agnostic)
  utils/                    # logging, caching+backoff, provenance, team normalization
  data/
    pull.py                 # nba_api playoff games + officials (cached, retried)
    odds.py                 # swappable odds adapter (never joins on nba_api game_id)
    synthetic.py            # demo/CI generator with planted referee structure
  features/
    build_table.py          # join + QC + feature engineering -> modeling_table.parquet
    multimembership.py      # [games x refs] 1/k weight matrix
    eda.py                  # unadjusted distributions + raw referee averages
  models/
    data_prep.py            # arrays / indices / coords for PyMC
    fit_stage1.py           # 1A total fouls (NB) + 1B directional lean (NB pair)
    fit_stage2.py           # StudentT point differential
    mediation.py            # full-posterior referee -> scoreboard composite
    robustness.py           # LOO compare, permutation placebo, LOSO, odds sensitivity
    diagnostics.py          # R-hat / ESS / divergences / BFMI + robust HDI
app/                        # Streamlit website (Home + 5 pages)
tests/                      # pytest smoke tests (incl. PyMC model-graph checks)
data/ models/ reports/      # artifacts (gitignored; regenerated by the pipeline)

---

How to run

Requires Python 3.11+. We recommend uv but plain pip works too. Full step-by-step reproduction (from a fresh clone): see docs/REPRODUCE.md — or just bash scripts/run_demo.sh (offline, ~2 min) and bash scripts/run_full.sh (everything, ~4–6 hrs).

# 1. Environment
uv venv && source .venv/bin/activate
uv pip install -r requirements.txt      # or: pip install -r requirements.txt
# (editable install so `import refball` works without PYTHONPATH)
uv pip install -e .

# 2a. DEMO MODE (offline, ~1 minute end-to-end) — synthetic data with planted structure
python -m refball.data.synthetic
python -m refball.features.build_table
python -m refball.features.eda
python -m refball.models.fit_stage1 --quick
python -m refball.models.fit_stage2 --quick
python -m refball.models.mediation
python -m refball.models.robustness --quick

# 2b. REAL DATA — pull from nba_api (slower; rate-limited public endpoints)
#     First clear demo artifacts so synthetic data/odds don't mix in:
rm -f data/external/odds_synthetic.csv data/external/synthetic_truth.json \
      data/interim/games.parquet data/processed/*.parquet data/processed/*.json \
      data/processed/source_provenance.jsonl models/*.nc data/processed/robustness/*.csv

python -m refball.data.pull --season-start 2017 --season-end 2023   # 2017-18 .. 2023-24

#   --- WITHOUT betting lines (simplest; no free odds feed is bundled) ---
python -m refball.features.build_table            # will warn "no odds" — expected
python -m refball.models.fit_stage1 --no-odds     # add --quick for a fast first pass
python -m refball.models.fit_stage2 --no-odds
python -m refball.models.mediation
python -m refball.models.robustness --full        # auto-detects no-odds

# Direct-evidence layer: NBA Last Two Minute graded clutch calls (joins on game_id)
python -m refball.data.l2m                         # downloads the MIT-licensed mirror, builds the table
python -m refball.models.l2m_model                 # crew clutch-error model + placebo

# Additional evidence streams
python -m refball.models.within_series --quick     # identification: teams held fixed within a series
python -m refball.data.pull --season-type "Regular Season" --season-start 2017 --season-end 2023
python -m refball.models.regular_season --quick    # POWER: ~290 games/ref + playoff-vs-baseline
python -m refball.data.pbp                          # play-by-play foul types + game state (PlayByPlayV3)
python -m refball.models.pbp_clean --quick          # foul->points on discretionary/competitive fouls

#   --- WITH betting lines (you supply a CSV; see "Odds CSV format" below) ---
# python -m refball.features.build_table --odds path/to/odds.csv
# python -m refball.models.fit_stage1               # uses the line as a control
# python -m refball.models.fit_stage2
# python -m refball.models.mediation
# python -m refball.models.robustness --full

# 3. Website
streamlit run app/Home.py

If you don't pip install -e ., prefix commands with PYTHONPATH=src.

Demo vs. real data

The app and pipeline run identically on synthetic or real data — the synthetic generator emits the same column contract (refball.schema.GAME_COLUMNS). When synthetic data is loaded, the website shows a Demo mode banner so illustrative numbers are never mistaken for real results. Swap in the nba_api pull to replace it.

--quick vs. full

--quick uses short chains (fast, rough — good for demos/CI; expect R-hat slightly above 1.01). Omit it for production estimates (1000 draws × 4 chains, target_accept=0.95). Sampler knobs are overridable via REFBALL_MCMC_* env vars (see config.py).

Troubleshooting: if parallel sampling errors in a constrained/sandboxed environment, force sequential chains with REFBALL_MCMC_CORES=1 (and REFBALL_MCMC_QUICK_CORES=1).

---

Data sources

Source	Used for	Notes
nba_api LeagueGameLog (Playoffs, team)	scores, PF, FTA/FTM, possession inputs	cached per season
nba_api BoxScoreSummaryV2.officials	assigned crew (names + ids)	cached per game
Odds adapter (data/odds.py)	spread + total	swappable; local CSV / your file; validated + team-normalized
L2M reports (data/l2m.py)	graded clutch calls (CC/CNC/IC/INC) → signed errors	MIT-licensed atlhawksfanatic/L2M mirror; joins on game_id; the most direct whistle measure

Joins across sources use (game_date, home_tricode, away_tricode) with a ±1-day tolerance — never the nba_api game_id, because odds feeds do not share that id namespace. Every join prints left/right/matched/unmatched/match-rate and warns below 95%. If officials are missing for some games, those games are dropped from referee models (and logged) but kept for non-referee EDA. If odds can't be joined, the pipeline runs a no-odds model and flags it as weaker.

Odds CSV format

The adapter (src/refball/data/odds.py) maps common header variants, so a CSV like this works:

game_date,home_team,away_team,spread,total
2024-04-21,Boston Celtics,Miami Heat,-9.5,206.5
2024-04-24,Boston Celtics,Miami Heat,-8.0,210.0

Recognized aliases include date/gamedate, home/home_team/hometeam, away/visitor, spread/home_spread/line, total/over_under/ou, and optional source. Spread is signed from the home perspective: negative = home favored, positive = home underdog. Team names are normalized to tricodes, and the join is on (date ±1 day, home, away).

Officials availability caveat

nba_api's BoxScoreSummaryV2.officials flags known data gaps for games on/after 2025-04-10 (the 2024-25 playoffs). Pulling through --season-end 2023 (the 2023-24 playoffs) avoids it. If you include the latest postseason, games missing officials are dropped from the referee models (and logged) — the pipeline degrades gracefully.

Methodology (short)

• Partial pooling. Hierarchical effects with deliberately tight half-normal priors on the referee sigmas: a referee with a handful of games is shrunk toward zero unless the data keep supporting an effect.
• Multi-membership. Each game contributes 1/k weights (k = officials known, normally 3) to a [games × refs] matrix; a crew effect is the average of the assigned officials' effects — no crew-chief-takes-all, no triple-counting.
• Non-centered parameterization throughout for clean geometry.
• Stage 1A total fouls ~ NegativeBinomial (log-possessions offset). Stage 1B home/away fouls ~ NegativeBinomial sharing a crew volume and a crew lean. Stage 2 point differential ~ StudentT on foul/FT margins + line + total. Mediation composes the per-referee foul-margin association with the foul→points coefficient over the full posterior.
• Robustness: PSIS-LOO (no-ref / ref-volume / ref-volume+lean), within-season permutation placebo, leave-one-season-out, with/without-odds sensitivity, prior-sensitivity, and an injection-recovery power check (robustness --only power).

Findings & honest framing (real 2017–2023 playoffs, no odds)

The current real-data result is a null — but a carefully-qualified one, hardened by an adversarial self-audit (see docs/ and the Diagnostics page):

• No detectable referee-associated foul lean. 0 of 58 officials have a 94% interval excluding zero, and the lean variance is indistinguishable from shuffled-crew placebos.
• This is not an artifact. It survives a 75× wider prior (posterior unchanged), it is not over-control (the free-throw-inclusive total foul-margin → points association is also ≈ 0: +0.03, HDI [−0.22, +0.28]), and it is not a bug (sign / multi-membership / placebo / mediation all verified).
• But the design is underpowered. Injection-recovery shows a clearly swing-relevant +1 foul/game lean injected into the most-sampled official is not detected (only +2 is). So the honest claim is "no detectable referee swing," not "referees provably don't affect outcomes." Absence of evidence ≠ evidence of absence.
• The most direct evidence agrees. The NBA's own Last Two Minute graded calls (182 close playoff games, 277 clutch errors) show 0 of 55 crews with a distinguishable error-lean (placebo-confirmed), and only a faint, inconclusive leaguewide home tilt (errors favor home ~1.18×, 94% HDI on the bias includes 0, P≈0.88). Caveats: crew-level (not individual), close-late games only (selection), and the NBA grades its own calls.

Additional evidence streams (all agree)

To attack the power and identification gaps, four more public-data angles — and they converge:

• Regular-season power test (the strongest result). Estimating leans across 8,289 regular-season games (median ~293 games/referee, 102 refs) — i.e. with real power — the lean variance shrinks further (sigma ≈ 0.005), 0 of 102 refs have an interval excluding zero, a referee's playoff lean is uncorrelated with their high-power regular-season lean (Spearman ≈ 0.01 → playoff leans are sampling noise), and 0 of 102 behave differently in the playoffs than their own baseline. This turns the underpowered null into a high-power null.
• Within-series identification. Holding the matchup fixed (series × team effects; teams identical across a 4–7 game series, crews rotate), the lean stays at placebo level and 0/58 refs are distinguishable — the "refs just draw certain matchups" confound isn't hiding a swing.
• Play-by-play cleaning. Rebuilding the foul margin from discretionary fouls in competitive game states nudges the foul→points coefficient toward the expected sign (γ ≈ −0.14 vs raw +0.03) but it stays inconclusive — fouls barely move the scoreboard even after removing garbage-time/intentional fouls.
• Closing-line market control. Adding the real closing spread/total (332 games) barely moves the foul→points coefficient and leaves referee rankings stable (Spearman ≈ 0.76).
• Devil's advocate (we tried to prove the opposite). Mining the most extreme of 58 refs, slicing 18 ways, and naming a villain produces scary numbers (a ref "73% home-favoring, p=0.015"; a subgroup "−1.7 foul tilt, p=0.002") — but every one dies under Bonferroni/FDR, a label-permutation placebo (referee identity adds nothing, p=0.22), and regression to the mean (the leaderboard-topper's lean flips sign in the high-power regular season; playoff-vs-regular Spearman 0.01). Only a small crew-invariant home tilt survives. See docs/devils_advocate.md · python -m refball.models.devils_advocate.

Main limitations

• Underpowered for small effects. ~585 games, ~21 games/official → a modest but real lean would be missed. The null is "no detectable swing," reported with that caveat front and center.
• Assignment is not random → selection bias / endogeneity. Lines help control for team strength but do not fix it.
• Associational, not causal. Foul margins also reflect team style, pace, coaching, and intentional fouling — fouls are nearly uncorrelated with the final margin (r ≈ 0.02).
• Box-score fouls are coarse; call-level/play-by-play (foul types) would measure the whistle directly and add power.

See the in-app Methodology & Limitations page for the full treatment, including what would count as better evidence.

---

Reproducibility & quality

• Deterministic synthetic generator (seeded) with planted structure; the test suite asserts the models recover it in the right direction.
• pytest smoke tests (incl. PyMC model-graph compilation), ruff lint + format.
• All raw pulls cached; provenance logged to data/processed/source_provenance.jsonl.

pytest -q
ruff check src app tests && ruff format --check src app tests

License

MIT. This is an independent analytics project and is not affiliated with or endorsed by the NBA or any team or official.