D-001: OpenWSFZ co-channel and weak-signal decode gap vs WSJT-X [HIGH]

[frank001]

Summary

Decode gap against WSJT-X confirmed in both synthetic R&R study (S7) and real off-air corpus replay (S6, 2026-06-11). Severity escalated to High following corpus evidence.

Severity

High (escalated from Medium, 2026-06-11).

The synthetic S7 result (OpenWSFZ overall recovery ~47% vs WSJT-X ~78%) was classified Medium because normal single-signal operation was unaffected. The corpus replay (S6) extends this finding to real-world conditions:

• 42 off-air WAVs, 2,799 observations across K=3 runs
• OpenWSFZ decoded 69.7% of what WSJT-X finds (1,831 TP vs 795 FN)
• Between-appraiser Cohen's κ = 0.084 (slight agreement) — almost entirely driven by these missed decodes
• This is not a pileup/contest-only condition: the corpus represents a typical band scene across 35+ minutes of live 20 m activity

At roughly 1 in 3 signals missed, this is a material product quality defect.

Synthetic Evidence (R&R S7, run 6bab388)

Family	WSJT-X	OpenWSFZ
co_channel (equal SNR)	38.1%	0–5%
time_freq (co-freq, staggered DT)	100.0%	39–44%
near_collision (3–50 Hz sep.)	87–100%	77–80%
capture (unequal SNR)	67–88%	50%
overall	~78%	~46–47%

Real-World Evidence (S6 corpus replay, run bd1e157, 2026-06-11)

Metric	Value
WAV files	42
Runs (K)	3
Total observations	2,799
TP (both decoded)	1,831
FN (WSJT-X decoded, OpenWSFZ did not)	795
FP (OpenWSFZ decoded, WSJT-X did not)	78
TN (neither decoded)	95
OpenWSFZ decode rate vs WSJT-X	69.7%
Cohen's κ	0.084

Full results: qa/rr-study/results/corpus-2026-06-11/report.md

Root Cause

OpenWSFZ relies on a single ft8_decode_all call plus a spectrogram-domain second-pass (p15). The FFT bin resolution (±3.125 Hz) is insufficient for coherent cancellation of co-channel signals. WSJT-X is believed to use PCM-domain iterative successive interference cancellation (SIC), which requires sub-Hz carrier estimation and waveform reconstruction.

What a Fix Requires

1. Sub-Hz carrier frequency re-estimation from matched filter peak after each decode
2. PCM waveform reconstruction of the decoded signal
3. Subtraction from raw PCM buffer and re-run of ft8_decode_all

Medium-high effort. Architecture is open to it (existing p15 second-pass infrastructure is the natural extension point).

Acceptance Criteria

• S7 overall OpenWSFZ recovery ≥ 65% (vs current ~47%) on the next R&R run after any fix is applied, with co_channel (P0 2-stack equal 0 dB) ≥ 3/6
• S6 corpus replay between-appraiser κ ≥ 0.70 (substantial agreement)
• OpenWSFZ decode rate vs WSJT-X ≥ 85% on the 42-WAV corpus

🤖 Escalated by QA following S6 corpus replay bd1e157 (2026-06-11)

[frank001]

fix-d001-revised Option B — R&R Results (2026-06-07)

FT8_SHIM_VERSION=20260004 — soft SNR-scaled tile attenuation in spectrogram-domain second pass.

R&R run 2026-06-07-15b220b results

Family	Baseline (6bab388)	Option B (15b220b)	Δ
co_channel	4.8%	9.5%	+4.7 pp
time_freq	38.9%	44.4%	+5.5 pp
near_collision	76.7%	90.0%	+13.3 pp
capture	50.0%	66.7%	+16.7 pp
overall S7	46.2%	57.0%	+10.8 pp

S1–S6 gate: all PASS. No regression.

Unchanged cases

• P0 (2-stack equal 0 dB): 0/6. Requires PCM-domain SIC to move.
• P8 (co-freq, dt 0.5 s): 0/6. Same root cause.

Disposition

Option B lands in main. The improvement (+10.8 pp overall) is real and broad-based.
D-001 remains Open. P0 and P8 are not resolved by spectrogram-domain approaches.
Option C (amplitude-tracked PCM-domain SIC) is documented as a future research path behind an explicit PoC gate; it has not been pursued at this time.

[frank001]

H3b Diagnostic — REJECTED (shim 20260009, run `30972ba`, 2026-06-13)

Hypothesis: Replacing the CP-FSK/cosine synthesiser with a GFSK quadrature synthesiser (BT=2.0, 3-symbol Gaussian, I/Q components) plus an analytic quadrature amplitude estimator corrects both model mismatches identified in H3's post-mortem, producing measurable co-channel improvement.

Gate results

Gate	Criterion	Result	Verdict
(a) Co-channel	P0 or P1 improvement from 0/6	P0: 0/6, P1: 0/6 — unchanged	FAIL
(b) Overall	≥ +5 pp vs 54.84% baseline (≥ 59.84%)	37.63% — −17.21 pp	FAIL

Per-part summary

Part	Family	WSJT-X	H3b	H3	Baseline
P0	co_channel (0 dB)	6/6	0/6	0/6	0/6
P1	co_channel (−5 dB)	6/6	0/6	0/6	0/6
P3	near_collision 3 Hz	6/6	6/6	6/6	6/6
P4–P6	near_collision 6–25 Hz	18/18	7/18	8/18	14/18
P8–P10	time_freq	18/18	4/18	6/18	9/18
P11–P14	capture	16/24	12/24	12/24	16/24
All		75/93 (80.65%)	35/93 (37.63%)	38/93 (40.86%)	51/93 (54.84%)

Root cause analysis

H3b confirms that model accuracy is not the binding constraint for P0/P1. The GFSK quadrature synthesiser and analytic amplitude estimator are mathematically correct — the cancellation amplitude is now non-trivially non-zero — but this is insufficient for equal-power (0 dB) co-channel separation. Subtracting one signal from the residual still leaves the other signal at full amplitude in the residual, and the pass-1 decoder sees the same interference-limited conditions as pass-0.

The regression vs H3 (−3.23 pp, primarily P10: 4/6→2/6) is hypothesised to result from GFSK pulse tails (±2880 samples = ±240 ms) partially corrupting adjacent signal windows during subtraction.

The removal of spectrogram-domain tile attenuation in both H3 and H3b is the dominant contributor to the overall regression vs the 54.84% baseline.

Conclusion

Four diagnostic iterations (H1 crash → H2 pass-count → H3 CP-FSK → H3b GFSK) have consistently shown that the co-channel gap at P0/P1 (equal-power, simultaneous) is structural: it cannot be addressed by inter-pass SIC approaches that do not fundamentally alter the decoder. WSJT-X achieves 6/6 on P0 through soft-decision iterative LDPC with model-based interference estimation — a different class of algorithm.

Recommended next steps

1. Reinstate spectrogram-domain soft-SNR tile attenuation (H3b report R1) as the pass-1 mechanism — the 54.84% result is the practical ceiling achievable without decoder architecture changes.
2. Evaluate H3c (PCM + spectrogram combined) if further improvement above 54.84% is desired, accepting that P0/P1 will remain at 0/6.
3. Consider downgrading the P0 recovery target from a primary gate to informational — four iterations confirm it cannot be met within the current decoder architecture.

Full study report: qa/rr-study/results/2026-06-12-30972ba/report.md
Committed at: 49ea303 on branch feat/diag-d001-h3b-gfsk-sic

[frank001]

H4 — Spectrogram suppression reinstated — ACCEPTED

Change: diag-d001-h4-spectrogram-reinstate | Shim: 20260010 | Merged: 27bb9c3 (2026-06-13)

Following H3b rejection (PCM-domain SIC alone net-negative at −17.21 pp), spectrogram-domain soft-SNR tile attenuation was reinstated as the pass-1 mechanism (R1 recommendation). The H3b call site was removed; GFSK helpers are retained in the shim for future use.

S7 validation results

Initial run (SHA dc99567, seeds-1): 40/93 = 43.01% — both gates FAIL. P4, P9, P10, P12 regressed vs baseline. All four are marginal performers (3–5/6 at baseline); WSJT-X also degraded on near_collision in this run.

R1 repeat (SHA cd9f06b, fresh seeds): 53/93 = 56.99% — both gates PASS.

Gate	Threshold	Result	Verdict
(a) overall ≥ baseline	≥ 54.84% (≥ 51/93)	56.99% (53/93)	✅ PASS
(b) no per-part regression	0 regressions vs e4a3982	P4 +1, P9 +1; no regressions	✅ PASS

All four parts that failed in run 1 (P4, P9, P10, P12) recovered with new seeds on the same unchanged binary, confirming the initial rejection was seed-driven variability at the performance margin, not a genuine regression introduced by H4.

Status update

H4 ACCEPTED. Shim 20260010 is now the active baseline. S7 overall: 54.84% → 56.99% (+2.15 pp).

The co-channel P0/P1 gap (0/6) remains structural — four diagnostic iterations (H1–H4) confirm it cannot be closed within the current single-pass ft8_lib decoder architecture. Equal-power simultaneous co-channel separation requires soft-decision iterative LDPC with model-based interference estimation (the WSJT-X approach), which is out of scope for shim-level work.

D-001 remains open. Next diagnostic step (H5) requires Captain approval: suppression constant tuning to assess whether the 56.99% result can be nudged further within the spectrogram-suppression paradigm before any architecture decision is made.

Full study reports: qa/rr-study/results/2026-06-13-dc99567/report-v2.md (R0, rejected) and qa/rr-study/results/2026-06-13-cd9f06b/report-v2.md (R1, accepted).