Add Table S1 analysis: per-feature NaN percentage

[RedPenguin100]

First script in a new notebooks/analysis/ folder, for the numbers that go into the paper's supplementary tables.

Table S1 states a missing fraction for every feature, and those numbers were computed ad hoc. This puts them in the repo so they are reproducible and diff after any feature-pipeline change.

What it does

notebooks/analysis/table_s1/nan_percentage.py reads the shipped model's feature list (tauso_score_v1.features.txt, 485 features), counts NaN per feature over the modelling set, and writes out/nan_percentage.csv. It raises if any model feature is absent from the loaded dataset.

The printed summary lists only the features that are ever missing; the CSV carries all 485, sorted by name so the file diffs cleanly.

Result

143,904 rows, 485 model features: 97 ever missing, 388 never missing. Top of the list:

feature	n_missing	pct_missing
cai/enc/tai_score_{20..70} (18)	110,550	76.82
hybr_cet_* (3)	89,390	62.12
structure_sense_mrna_dist_to_*_stop (2)	84,116	58.45
structure_sense_*_junction_exonic (2)	82,433	57.28
structure_sense_host_exon_log_length	82,321	57.21
structure_sense_host_intron_log_length, ..._junction_logdist_intronic	63,233	43.94
ohe_3p9_* (4)	62,533	43.45
ohe_pos9_* (4)	62,224	43.24

Two things this surfaces that the draft table had wrong:

• The 28 terminal one-hot columns at positions 6-9 are not 0% missing. They range from 41 rows (ohe_3p6_*) to 62,533 (ohe_3p9_*, 43.45%). The encoder assigns each base to whichever terminus lies on its own half of the sequence, so a position outside its half is left NaN -- ohe_pos{i} needs length >= 2i+1 and ohe_3p{i} needs length >= 2i+2. Short designs carry no value at the inner positions.
• load_dataset() returns 488 features; the model uses 485. The three extras are hybr_lna_dna_rna_dg, hybr_lna_wing5_dg, hybr_lna_wing3_dg, all 100% NaN -- there are no LNA oligos in the corpus. Reading the feature list from the shipped model rather than from the loader is what makes the count come out at the 485 the paper describes.

Values were cross-checked against an independently computed set: 485/485 features agree, 0 mismatches.

No behaviour change: read-only analysis, one script plus its output.

🤖 Generated with Claude Code