feat(dta2sdtm): DTA logical data model v0.2.0 - #148
Open
rchaitanyapradeep wants to merge 1 commit into
Open
Conversation
Rebuilds the model against the DTA MVP Inventory, which the workstream confirmed as the DTA User Requirements. Added (agreement layer): - TestSpecification — the agreed test table: per-test vendor and standard codes, method, specimen, LOINC, data type, units, blinding. Fills the gap between concept scope (BcSelection) and file mechanics (TransferRequirements), where the agreement previously said nothing about which tests actually arrive. - TransferDataset / TransferVariable — the declared file structure. Replaced (instance layer): - TransferRecord (60 flat variables) replaces the nested LAB v2 graph (Subject > Collection > Specimen > Panel > LabTestResult > Result). This is a change of transfer vocabulary, not just shape: the removed graph modelled LAB v2 variables, TransferRecord models SDTM-shaped columns plus vendor passthrough. The two share essentially only STUDYID. Agreement <-> instance, modelled rather than asserted: - file level: Transmission.dta_version_ref names the DTA version fulfilled - test level: natural-key join on (test_panel_name + data_provider_test_code), falling back to data_provider_test_name. No surrogate key — no real transfer file carries one, and tests are not always coded. Design rule — permissive payload, strict agreement. Enums bind on the agreement side where the sponsor controls values; vendor-populated payload slots stay strings so a conformant-but-unanticipated file is not rejected at ingest. UnitEnum and LbTestCdEnum become documented reference vocabularies bound to no slot, because real transfers carry values outside any fixed subset. Project-defined slots (transfer_structure_version, dta_version_ref) say so in their annotations. Nothing is presented as CDISC terminology unless it is. BREAKING: instances valid against v0.1.0's nested Transmission will not validate against v0.2.0. All four bundled examples are updated accordingly. Verified: schema compiles; all five examples validate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilds
src/dta2sdtmagainst the DTA MVP Inventory, which the workstream confirmed as the DTA User Requirements. Schema0.1.0→0.2.0.Warning
Breaking change. Instances valid against v0.1.0's nested
Transmissionwill not validate against v0.2.0. All four bundled examples are updated.Changes from previous version
TestSpecification— the agreed test table. New; not present in any earlier source.TransferRecord(60 vars, SDTM-shaped + vendor passthrough).TransferDataset/TransferVariable— the declared file structure.Txt→Text; added version change narrative and transfer schedule.Classes 21 → 19 · Enums 19 → 15 · All five examples validate.
Why the instance layer changed vocabulary, not just shape
The removed graph modelled LAB v2 variables (
LTVRSN,ACCSNID,SPECID,PLBTID,PLRCRS…).TransferRecordmodels SDTM-shaped columns plus vendor passthrough (CTESTCD,CTEST,CUNIT,CSPEC,CMETHOD,CPANEL) and tenAUXfields. The two share essentially onlySTUDYID.Why both layers exist
The SDTM-LB derivation is a function of
(TransferRecord × TestSpecification). The payload carries the value; the agreement carries the per-test facts no payload can — unit conversion basis, result form, BC identity.TestSpecification.data_typeis specifically what tells the transform whetherLBORRESmay be cast toLBSTRESN; without it a categorical result such as a urinalysis colour gets attempted as a number.The join is a natural key (
test_panel_name+data_provider_test_code, falling back todata_provider_test_name), not a surrogate — no real transfer file carries a spec id, and tests are not always coded: in the source inventory's own examples the flow-cytometry and IHC rows identify tests by name only.Design rule — permissive payload, strict agreement
Enums bind on the agreement side, where the sponsor controls the values. Vendor-populated payload slots stay unconstrained strings so a conformant-but-unanticipated file is not rejected at ingest; discrepancies surface in the transform instead.
UnitEnumandLbTestCdEnumbecome documented reference vocabularies bound to no slot, since real transfers legitimately carry values outside any fixed subset.Project-defined slots (
transfer_structure_version,dta_version_ref) are annotated as such. Nothing is presented as CDISC terminology unless it is.Gaps to discuss
Open questions and known limitations. Items 1–2 are properties of the agreed transfer structure itself, not of this model, and they constrain what SDTM can be produced. Neither is worked around here — inventing columns would misrepresent the agreed structure.
LBORNRLO/LBORNRHI/LBNRINDLBORRES/LBORRESULBSTRESC/LBSTRESN/LBSTRESUderivable only fromTestSpecification, making a complete test specification load-bearing rather than merely useful.agreement_status, annotated provisional. No codelist invented.01,02"01"and"1"; an integer cannot hold"01", and that padding is the visit crosswalk key.Items 1 and 2 are worth deciding before the transfer structure is signed off.
Validation
All pass.
license: MITis preserved throughout, per this repository.