DSD: fix the conference-matrix construction, and add two-level categorical factors

[kgdunn]

Two pieces of work on definitive screening designs. They touch the same function, which is why they share a PR.

Part 1: the construction was wrong at ten factor counts

dispatch_dsd returned a design that is not a DSD for 9, 10, 15, 16, 21, 22, 25, 26, 27 and 28 factors. _conference_matrix implemented Paley's construction only for a prime q = m - 1; at every other order it fell back to _cyclic_conference_matrix, which is not a conference matrix. Main effects came out correlated with each other at up to r = 0.9, returned with only a log warning. design_type="omars" shares the construction and already reported omars_verified: False at the same counts.

• Paley's construction now works over any odd prime power (GF(9), GF(25), GF(27), GF(49), ...), covering 9, 10, 25, 26, 27 and 28 factors at minimal run size. The field is built from the first monic irreducible polynomial of the required degree.
• A verified order-16 table covers 15 and 16 factors, the one gap below order 22.
• The cyclic fallback is gone. Where no conference matrix exists at the minimal order, the constructor steps up to the next order it can build and records that in the metadata; where nothing can be built it raises.
• Every matrix is verified against C.T @ C == (m - 1) * I before use.

Measured max off-diagonal of the Gram matrix (0 required):

k factors	before	after	runs
9, 10	16.0	0	21
15, 16	28.0	0	33
21, 22	40.0	0	49 (was 45)
25, 26	48.0	0	53
27, 28	52.0	0	57

Order 22 is the first order with no conference matrix at all: one of order m ≡ 2 (mod 4) exists only when m - 1 is a sum of two squares, and 21 is not. So 21 and 22 factors now cost 49 runs. Every other count is unchanged. Jones & Nachtsheim (2013), p. 122, prescribe exactly this step-up.

Part 2: two-level categorical factors

generate_design(..., design_type="dsd") raised as soon as a categorical factor was present. Both column-augmentation procedures of Jones & Nachtsheim (2013) are now implemented, selected with a new categorical_method argument.

	"dsd" (DSD-augment, default)	"orth" (ORTH-augment)
Zero resolution	z_{j,1}=b_j, z_{j,2}=−b_j	z_{j,1}=z_{j,2}=1
Main effects	Unbiased by every second-order effect	Partially aliased with categorical interactions
Information matrix	Not diagonal (couplings of exactly ±2)	Exactly diagonal for c ≤ 4
Categorical columns	Balanced	Unbalanced
Centre runs	2	2 (c=1), 4 (c≥2)
Sign vectors	Determinant search, per Section 2 Step 3	Fixed pattern, per Section 3 Step 1

A categorical factor occupies a conference-matrix column like any other, so it costs the same runs as a continuous factor and contributes no quadratic term. Factors may be given in any order. Three-or-more-level categorical factors raise, pointing at d_optimal.

Verified against every oracle the paper provides

• Table 4: all 36 run sizes, both procedures.
• Table 4's alias column: 2/n across all 36 rows. This is the check sensitive to the sign search that run sizes cannot detect.
• Table 3: the constant-term alias row entry by entry, for m=4, c=2.
• Equation (2): the information matrix for the same design.
• Section 2: main-effect rows of the alias matrix exactly zero (the defining property).
• Section 3: exact orthogonality asserted only for c ≤ 4, bounded near-orthogonality above, which is the paper's own boundary.

Two errata in the paper

The paper over-determines its own designs, which lets Figure 1 be cross-checked against Table 3, Equation (2), Table 4 and the procedure steps. Both c=2 panels fail, and are therefore not used as oracles:

1. Figure 1(a), c=2 gives the two categorical columns an inner product of 6. Equation (2) says 2 and Table 3 says the 56 alias is 0.1429, not the 0.4286 that follows. The printed design is also not the determinant-maximising choice its own Step 3 requires (20,966,400 for the correct one against 16,486,400).
2. Figure 1(b), c=2, run 10 prints a minus in the first categorical column where Step 3 requires z_{1,1} = z_{1,2} = 1. As printed the column is unbalanced 7-to-9 and the main-effects plan is not orthogonal, defeating the procedure's purpose. A unique single-entry flip restores both.

The other six panels are self-consistent and reproduce exactly. Both findings are recorded as executable tests in TestFigure1Errata, not prose.

Adjacent bugs found while auditing the same paths

• estimate_screening_runs(k, "definitive_screening") returned 2k + 1 for every k, undercounting every odd factor count (a 7-factor DSD is 17 runs, not 15) and every stepped-up count. Now driven by the constructor, and it takes n_categorical / categorical_method so categorical centre runs are counted too. A test had enshrined the wrong value. The recommendation engine passes the count of two-level categorical factors, exposed as DOEProblemSpec.n_two_level_categorical.
• The recommended-strategy plan advertised a fake_factor parameter that does not exist, justified by a comment with the parity argument backwards.
• design_type="omars" shares dispatch_dsd, so adding categorical support silently extended it to OMARS, returning a design with omars_verified: False. OMARS is defined for quantitative factors, so it now rejects categorical ones and points at design_type="dsd".
• generate_design(..., design_type="taguchi") with a categorical factor raised ValueError: All values must be present in 'levels'. dispatch_taguchi codes categorical factors as level indices 0..n-1, which reached the Column constructor unmapped. Fixed by the same label mapping the DSD needed.
• tests/test_tool_spec.py::test_registered_in_registry depended on sibling tests having run in the same worker process, so it passed or failed depending only on how pytest-xdist split the run. Pre-existing on main; this branch surfaced it by changing the test count.
• The dataset tests' "skip if offline" guard never fired: _read_remote_csv converts network failures into a RuntimeError, but the helper caught only URLError / HTTPError / OSError, none of which can escape the loader. A 502 from openmv.net failed a job on this branch rather than skipping. _read_remote_csv now also retries a transient failure (three attempts, 1s and 2s backoff), while a 404, 403 or parse error still raises immediately.
• _is_prime became unreachable when _prime_power_factorization replaced it. Removed.

New public API

dsd_run_count(n_factors, n_categorical, categorical_method), dsd_conference_order(n_factors) and dsd_centre_runs(n_categorical, categorical_method), exported from process_improve.experiments, for sizing a design without building it.

Attribution for the order-16 table

Recorded in an attribute docstring on _CONFERENCE_MATRIX_16. It originates in Xiao, Lin & Bai (2012) and reached this repository through two ports: Jacob Albrecht's 2015 MATLAB port of the JMP add-in (BSD 3-Clause, Copyright (c) 2015 Jacob Albrecht, with Bristol-Myers Squibb named in the no-endorsement clause as his affiliation, not the copyright holder), and Daniele Ongari's Python translation in definitive_screening_design (BSD 3-Clause, Copyright (c) 2022 Daniele Ongari). Only the numeric table is reused, and it is verified rather than trusted: the order-10 table in the same upstream file has a mistyped entry that leaves that package's 9- and 10-factor designs non-orthogonal at r = 0.11.

Known gap, deliberately not addressed here

The docs build has the same exposure to an openmv.net outage that the tests had, and more broadly: seven case-study notebooks call pd.read_csv("https://openmv.net/...") directly, so neither the retry nor the skip reaches them, and nbsphinx executes each in its own kernel with nbsphinx_allow_errors = False. Closing that means either adding loaders for five more datasets and rewriting the notebooks to use them, or vendoring the CSVs. Both change the book's teaching material, so both are left for a separate decision.

Test plan

• tests/test_conference_matrices.py (163 tests): finite-field arithmetic checked as a field, the quadratic character checked as multiplicative and against Euler's criterion, every conference matrix checked against its defining property, the constructibility predicate cross-checked against the constructor at every order to 120, and the validator checked against a single flipped sign.
• tests/test_dsd_categorical.py (202 tests): the paper oracles above, plus factor-order independence, the large-c heuristic still producing a definitive design, the planner's run estimates, and the label-mapping branches.
• tests/test_experiments_datasets.py (18 tests): the skip guard and the retry policy, including that a settled status is not retried.
• Property tests widened from k ≤ 14 to k ≤ 30; test_main_effects_are_exactly_orthogonal is now unconditional (it used to skip precisely when the bug fired).
• Regression test over the ten previously degraded factor counts.
• Full suite: 2515 passed, 2 skipped. Coverage 93.8% against a 92% gate; all modified lines covered, with budget.py and datasets.py at 100%.
• ruff check . and mypy src/process_improve clean.

Checklist

• Version bumped in pyproject.toml (1.60.0 → 1.62.0, MINOR for the new feature), CITATION.cff in sync. 1.61.0 was taken by Compute score contributions from the data, not from the score #476 landing on main mid-review; this branch's two in-PR changelog sections were consolidated into one 1.62.0 entry, since neither intermediate version was ever tagged.
• Tests added or updated where relevant
• ruff check . passes
• CHANGELOG.md updated

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!