dist: JSON reader validation hardening

[samtalki]

Closes #262. Stacked on #264 (worktree-bmopf-schema-010) — the first three commits here are that PR; review the last commit only. Merging #264 first collapses this diff to the validation work.

• Mixed bound arrays survive. A PMD generator "pg_ub": [500.0, 500.0, null] keeps its finite phase bounds: the null-derived infinity stays in the array instead of dragging the whole field to None. The PMD writer respells nonfinite entries as null (PMD's own unbounded spelling), the BMOPF writer drops such a field with a warning instead of coercing to a zero limit, and the dss writer skips emergamps with a warning when the first i_max entry is nonfinite (it previously printed emergamps=inf).
• Matrix columns degrade loudly. A PMD matrix column that fails as_array warns (naming the element, key, and column) and stays zero; the parseable columns survive instead of the whole matrix dropping to None silently.
• Dangling references warn. A new warn_unresolved_references pass runs at the end of both JSON readers: any element referencing an undefined bus, and any line referencing an undefined linecode, produces a warning naming the reference. Previously graph.rs::bus_index synthesized a phantom bus, so a typo parsed cleanly into a topologically wrong network. Comparison is ASCII case-insensitive, matching DistNetwork::bus/linecode.
• The BMOPF fallback is gone. parse_file/parse_str route a .json to PMD on the data_model marker and to BMOPF on the schema-required bus + voltage_source tables; anything else errors with a message naming both rules (a PowerModels document used to fall through and parse into a bogus near-empty DistNetwork). Malformed JSON still routes to the BMOPF reader for its parse error, and an explicit --from/format override still forces a reader.

Not addressed here, filed separately: a multiconductor .pio.json payload serializes nonfinite floats (a PMD-sourced Inf bound, a BMOPF missing length NaN) as JSON null, which the payload reader then rejects — that predates this change and needs its own treatment.

All four #262 acceptance criteria are covered by new tests (mixed_finite_and_null_generator_bounds_survive, malformed_matrix_column_warns_and_keeps_the_rest, dangling_references_warn, bmopf_dangling_bus_reference_warns, parse_file_rejects_unclassifiable_json), and the conversion-matrix baselines are unchanged — no fixture leg gained or lost a warning.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YPB7LVTeMcdBYnzA9QdpPe

---

Generated by Claude Code

[github-actions]

Conversion Matrix

Legend

Cells show X/Y: observed warnings / expected warnings. Counts include source parse, target write, and target readback.

• 🟢 0/0: no warnings and checked invariants held.
• 🟡 X=Y: observed warnings match the reviewed expected count, and that count is nonzero.
• 🔴 X!=Y or invariant failure: behavior changed. If warnings decreased because fidelity improved, update the expected counts in the same PR.
• Expected counts are the *_WARNING_BASELINE arrays in powerio-cli/tests/conversion_matrix_report.rs; accept an intentional change by editing the matching source/target entry in the same PR.

Transmission

7 cases.

Source ↓ / target →	MATPOWER .m	PowerModels JSON	PSS/E .raw	PowerWorld .aux	egret JSON	pandapower JSON	Surge JSON	PSLF .epc
MATPOWER .m	🟢 0/0	🟢 0/0	🟡 8/8	🟡 8/8	🟢 0/0	🟡 5/5	🟢 0/0	🟡 11/11
PowerModels JSON	🟡 6/6	🟢 0/0	🟡 14/14	🟡 14/14	🟡 1/1	🟡 11/11	🟡 24/24	🟡 16/16
PSS/E .raw	🟡 13/13	🟡 1/1	🟢 0/0	🟡 1/1	🟡 1/1	🟡 3/3	🟡 1/1	🟡 10/10
PowerWorld .aux	🟡 12/12	🟢 0/0	🟢 0/0	🟢 0/0	🟢 0/0	🟡 2/2	🟢 0/0	🟡 5/5
egret JSON	🟢 0/0	🟢 0/0	🟡 8/8	🟡 8/8	🟢 0/0	🟡 5/5	🟢 0/0	🟡 11/11
pandapower JSON	🟡 6/6	🟢 0/0	🟡 11/11	🟡 11/11	🟡 5/5	🟢 0/0	🟢 0/0	🟡 11/11
Surge JSON	🟡 2/2	🟡 2/2	🟡 10/10	🟡 10/10	🟡 2/2	🟡 7/7	🟡 2/2	🟡 17/17
PSLF .epc	🟡 17/17	🟡 5/5	🟡 5/5	🟡 5/5	🟡 5/5	🟡 7/7	🟡 5/5	🟡 8/8
DeepMind OPFData JSON	🟡 3/3	🟡 2/2	🟡 5/5	🟡 5/5	🟡 3/3	🟡 4/4	🟡 2/2	🟡 4/4

Distribution

7 cases.

Source ↓ / target →	OpenDSS .dss	BMOPF JSON	PMD JSON
OpenDSS .dss	🟢 0/0	🟡 132/132	🟡 80/80
BMOPF JSON	🟡 18/18	🟢 0/0	🟡 27/27
PMD JSON	🟡 15/15	🟡 59/59	🟢 0/0

Full warning details: download conversion-matrix-warning-details for this workflow run.

[Copilot]

Pull request overview

Hardens powerio-dist’s JSON ingestion and cross-format conversion behavior by preserving mixed finite/unbounded bound arrays, emitting explicit warnings for partial parse failures, and tightening .json format classification to avoid silently misreading unrelated JSON documents. This fits the distribution subsystem’s goal of loss-aware conversions with explicit diagnostics rather than silent coercions.

Changes:

• Preserve mixed finite + null-derived-infinity bound arrays in PMD reads/writes, and make BMOPF/DSS handling of nonfinite bounds loud (drop/skip with warnings).
• Improve robustness of matrix parsing (warn per malformed column and keep remaining data) and add end-of-parse validation warnings for dangling bus/linecode references.
• Align BMOPF schema handling (0.1.0) including extras relocation, new typed elements/fields, and updated fixtures + baseline expectations.

Reviewed changes

Copilot reviewed 26 out of 29 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/data/dist/bmopf/example_ieee13.json	Updates vendored BMOPF IEEE13 example fixture to the newer schema/vintage layout.
powerio-pkg/src/package.rs	Bumps multiconductor payload schema URL/version to /2 / 2.0.0.
powerio-pkg/src/lowering.rs	Updates dropped-field detection for renamed per-sequence voltage bound fields.
powerio-pkg/examples/generate_schemas.rs	Generates multiconductor schema under pio-payload-multiconductor/2.
powerio-dist/tests/rich.rs	Updates expected BMOPF load voltage model strings to uppercase schema spellings.
powerio-dist/tests/pmd.rs	Adds/updates tests for mixed bounds, malformed matrix columns, dangling refs, and rating propagation.
powerio-dist/tests/matrix.rs	Adjusts conversion-matrix harness comparisons for DSS/PMD terminal renumbering and grounding/terminal pruning semantics.
powerio-dist/tests/bmopf.rs	Expands BMOPF tests for schema validity, one-triangle matrices, dangling refs, and nonfinite-bound dropping behavior.
powerio-dist/src/pmd/write.rs	Writes line-level ratings (i_max/s_max) via cm_ub/sm_ub and warns on dropped typed capacitors.
powerio-dist/src/pmd/read.rs	Keeps mixed finite/nonfinite bound arrays, warns on malformed matrix columns, and runs unresolved-ref warnings post-parse.
powerio-dist/src/model.rs	Adds per-sequence bus voltage bound scalars, line-level ratings, typed capacitors, and unresolved-reference warning pass.
powerio-dist/src/lib.rs	Re-exports new DistCapacitor type.
powerio-dist/src/dss/write.rs	Avoids emitting emergamps=inf, warns when skipping, and warns/drops unsupported line-level ratings + capacitors.
powerio-dist/src/dss/read.rs	Initializes new line rating fields on DSS read (currently None).
powerio-dist/src/convert.rs	Tightens .json format inference to avoid BMOPF “accept anything” fallback and adds tests for unclassifiable JSON.
powerio-dist/src/bmopf/write.rs	Writes BMOPF schema 0.1.0 $id, relocates tables into extras, adds bounds dropping for nonfinite arrays, adds capacitor emission.
powerio-dist/src/bmopf/read.rs	Reads schema 0.1.0 shapes (meta frequency, extras tables, transformer overflow merge), mirrors one-triangle symmetric matrices, warns on dangling refs.
powerio-dist/examples/regen_bmopf_examples.rs	Adds example regeneration helper for checked-in BMOPF outputs.
powerio-dist/examples/bmopf/ieee34.json	Regenerates BMOPF ieee34 example output consistent with schema 0.1.0 and new canonicalization.
powerio-dist/examples/bmopf/4bus_dy.json	Updates 4bus_dy example output to schema 0.1.0 meta + field names.
powerio-dist/docs/conversion-matrix.md	Updates warning-count baselines in conversion matrix documentation.
powerio-cli/tests/conversion_matrix_report.rs	Updates distribution warning baselines to match new schema/fixtures and warning behavior.
docs/src/pio-json-schema.md	Updates multiconductor payload docs to /2 and notes /1 remains served for legacy.
docs/schema/pio-package/0.1/schema.json	Regenerates package schema reflecting new fields/types (per-sequence bounds, line ratings, capacitors).
CHANGELOG.md	Adds unreleased notes covering validation hardening and BMOPF schema 0.1.0 alignment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[samtalki]

The "PowerIO.jl against this C ABI" failure is the companion-branch mechanism, not this PR's code: the failing "BMOPF v0.6.2 fidelity gates" assertions pin the pre-0.1.0 writer shape (tap on the transformer object, split x_series_from/_to, no lumped x_series) — exactly what #264 changed. #264's own run stays green because a worktree-bmopf-schema-010 companion branch exists in PowerIO.jl with the 0.1.0 expectations; this stacked branch (dist-json-validation-262) has no companion, so its run fell back to PowerIO.jl main.

Resolution is sequencing: once #264 and its PowerIO.jl companion merge, a rebase of this branch onto main goes green. Alternatively, pushing a PowerIO.jl branch named dist-json-validation-262 (same content as the worktree-bmopf-schema-010 companion) makes this run green immediately — I don't have push access to PowerIO.jl from this session, so that part is yours if you want it green before #264 lands.

---

Generated by Claude Code

[Copilot]

Pull request overview

Copilot reviewed 26 out of 29 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 29 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

powerio-dist/src/model.rs:942

• warn_unresolved_references currently skips empty bus IDs (""), but the graph projection will still synthesize a phantom bus for an empty string. This means a missing bus field (which several readers parse as an empty string) can still slip through with no warning. Consider warning on empty bus references explicitly instead of skipping them.

        let mut bus = |what: &str, id: &str| {
            if !id.is_empty() && !buses.contains(&id.to_ascii_lowercase()) {
                warnings.push(format!("{what}: references undefined bus `{id}`"));
            }
        };

[Copilot]

Pull request overview

Copilot reviewed 26 out of 29 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

powerio-dist/src/dss/write.rs:946

• This warning is emitted while writing a DSS switch, but the message prefixes it with line. Consider changing it to switch for clarity and consistency.

                Some([]) => self.warn(format!(
                    "line {}: i_max is empty; emergamps not emitted",
                    sw.name
                )),

[Copilot]

Pull request overview

Copilot reviewed 26 out of 29 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

powerio-dist/src/pmd/write.rs:295

• line_ratings writes line-level i_max/s_max with json!(Vec<f64>), which will panic if any entry is non-finite (PMD null restores to ±Inf). These arrays should be emitted with per-entry finite→number / non-finite→null conversion so mixed finite/null bounds round-trip safely.

    fn line_ratings(o: &mut Map<String, Value>, l: &DistLine) {
        if let Some(i_max) = &l.i_max {
            o.insert("cm_ub".into(), json!(i_max));
        }
        if let Some(s_max) = &l.s_max {
            o.insert("sm_ub".into(), json!(s_max));
        }
    }

powerio-dist/src/pmd/write.rs:326

• When re-inlining impedance for pmd_inline lines, the writer emits synthesized linecode ratings via json!(Vec<f64>). With the new behavior of preserving non-finite entries in i_max/s_max, this can panic during serialization; emit non-finite entries as JSON null instead.

                    Some(c) if inline => {
                        insert_impedance_matrices(&mut o, c, net.base_frequency);
                        if let Some(i_max) = &c.i_max {
                            o.insert("cm_ub".into(), json!(i_max));
                        }
                        if let Some(s_max) = &c.s_max {
                            o.insert("sm_ub".into(), json!(s_max));
                        }