Skip to content

feat(dta2sdtm): DTA logical data model v0.2.0 - #148

Open
rchaitanyapradeep wants to merge 1 commit into
mainfrom
feat/dta2sdtm-model-v2
Open

feat(dta2sdtm): DTA logical data model v0.2.0#148
rchaitanyapradeep wants to merge 1 commit into
mainfrom
feat/dta2sdtm-model-v2

Conversation

@rchaitanyapradeep

Copy link
Copy Markdown
Collaborator

Rebuilds src/dta2sdtm against the DTA MVP Inventory, which the workstream confirmed as the DTA User Requirements. Schema 0.1.00.2.0.

Warning

Breaking change. Instances valid against v0.1.0's nested Transmission will not validate against v0.2.0. All four bundled examples are updated.

Changes from previous version

# Change
1 Added TestSpecification — the agreed test table. New; not present in any earlier source.
2 Replaced transfer structure — nested LAB v2 (60 vars) → flat TransferRecord (60 vars, SDTM-shaped + vendor passthrough).
3 Linked agreement to payload — file carries the DTA version it fulfils; each result matches its agreed test on panel + vendor test code.
4 Added USDM field mappings — study ID, title, organisation, role, visit.
5 Added TransferDataset / TransferVariable — the declared file structure.
6 FixesTxtText; added version change narrative and transfer schedule.

Classes 21 → 19 · Enums 19 → 15 · All five examples validate.

Why the instance layer changed vocabulary, not just shape

The removed graph modelled LAB v2 variables (LTVRSN, ACCSNID, SPECID, PLBTID, PLRCRS…). TransferRecord models SDTM-shaped columns plus vendor passthrough (CTESTCD, CTEST, CUNIT, CSPEC, CMETHOD, CPANEL) and ten AUX fields. The two share essentially only STUDYID.

Why both layers exist

The SDTM-LB derivation is a function of (TransferRecord × TestSpecification). The payload carries the value; the agreement carries the per-test facts no payload can — unit conversion basis, result form, BC identity. TestSpecification.data_type is specifically what tells the transform whether LBORRES may be cast to LBSTRESN; without it a categorical result such as a urinalysis colour gets attempted as a number.

The join is a natural key (test_panel_name + data_provider_test_code, falling back to data_provider_test_name), not a surrogate — no real transfer file carries a spec id, and tests are not always coded: in the source inventory's own examples the flow-cytometry and IHC rows identify tests by name only.

Design rule — permissive payload, strict agreement

Enums bind on the agreement side, where the sponsor controls the values. Vendor-populated payload slots stay unconstrained strings so a conformant-but-unanticipated file is not rejected at ingest; discrepancies surface in the transform instead. UnitEnum and LbTestCdEnum become documented reference vocabularies bound to no slot, since real transfers legitimately carry values outside any fixed subset.

Project-defined slots (transfer_structure_version, dta_version_ref) are annotated as such. Nothing is presented as CDISC terminology unless it is.

Gaps to discuss

Open questions and known limitations. Items 1–2 are properties of the agreed transfer structure itself, not of this model, and they constrain what SDTM can be produced. Neither is worked around here — inventing columns would misrepresent the agreed structure.

# Gap Impact
1 No reference ranges in the data structure — no LBORNRLO / LBORNRHI / LBNRIND Those SDTM variables are not derivable from a transfer alone.
2 No standardised result/unit — only LBORRES / LBORRESU LBSTRESC / LBSTRESN / LBSTRESU derivable only from TestSpecification, making a complete test specification load-bearing rather than merely useful.
3 Two columns named "Reported Unit" in the source, both mandatory Modelled once. Confirm intent.
4 "Status" column rename agreed upstream, new name and value list not specified Modelled as agreement_status, annotated provisional. No codelist invented.
5 Role — four CT values specified, but the DTA authoring app emits others (e.g. "Data Acquisition Lead") Left unbound. An unbound slot accepts the four agreed values and others; a bound one cannot.
6 Visit number — specified as a number, but observed values are 01, 02 Left as string. A string holds "01" and "1"; an integer cannot hold "01", and that padding is the visit crosswalk key.

Items 1 and 2 are worth deciding before the transfer structure is signed off.

Validation

linkml-validate -s dta.linkml.yaml -C DataTransferAgreement agreement.example.yaml
linkml-validate -s dta.linkml.yaml -C DatasetSpecialization  dataset_specialization.example.yaml
linkml-validate -s dta.linkml.yaml -C Transmission           transmission.example.yaml
linkml-validate -s dta.linkml.yaml -C Transmission           transmission_minimal.example.yaml

All pass. license: MIT is preserved throughout, per this repository.

Rebuilds the model against the DTA MVP Inventory, which the workstream
confirmed as the DTA User Requirements.

Added (agreement layer):
- TestSpecification — the agreed test table: per-test vendor and standard codes,
  method, specimen, LOINC, data type, units, blinding. Fills the gap between
  concept scope (BcSelection) and file mechanics (TransferRequirements), where
  the agreement previously said nothing about which tests actually arrive.
- TransferDataset / TransferVariable — the declared file structure.

Replaced (instance layer):
- TransferRecord (60 flat variables) replaces the nested LAB v2 graph
  (Subject > Collection > Specimen > Panel > LabTestResult > Result). This is a
  change of transfer vocabulary, not just shape: the removed graph modelled LAB
  v2 variables, TransferRecord models SDTM-shaped columns plus vendor
  passthrough. The two share essentially only STUDYID.

Agreement <-> instance, modelled rather than asserted:
- file level: Transmission.dta_version_ref names the DTA version fulfilled
- test level: natural-key join on (test_panel_name + data_provider_test_code),
  falling back to data_provider_test_name. No surrogate key — no real transfer
  file carries one, and tests are not always coded.

Design rule — permissive payload, strict agreement. Enums bind on the agreement
side where the sponsor controls values; vendor-populated payload slots stay
strings so a conformant-but-unanticipated file is not rejected at ingest.
UnitEnum and LbTestCdEnum become documented reference vocabularies bound to no
slot, because real transfers carry values outside any fixed subset.

Project-defined slots (transfer_structure_version, dta_version_ref) say so in
their annotations. Nothing is presented as CDISC terminology unless it is.

BREAKING: instances valid against v0.1.0's nested Transmission will not validate
against v0.2.0. All four bundled examples are updated accordingly.

Verified: schema compiles; all five examples validate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant