cdisc-org · rchaitanyapradeep · Aug 6, 2026
diff --git a/src/dta2sdtm/README.md b/src/dta2sdtm/README.md
@@ -5,35 +5,85 @@ makes the lab **Data Transfer Agreement (DTA)** a typed, validatable contract an
 it traceable from the upstream **USDM** protocol all the way down to **SDTM-LB**.
 
 The token *DTA* is overloaded. This model holds both senses — the governance
-**Agreement** and the data **Transmission** that fulfils it — and joins them through a
+**Agreement** and the data **Transfer** that fulfils it — and joins them through a
 single hub, the **BiomedicalConcept**.
 
+**Schema version 0.2.0.** Built from the DTA MVP Inventory, which the workstream
+confirmed as the DTA User Requirements. See [What changed in v0.2.0](#what-changed-in-v020).
+
 ## The three layers
 
 | Layer | Purpose | Key classes |
 |-------|---------|-------------|
-| **Agreement** | The governance contract (what data, by whom, in what format, mapped to which visits). Slot names mirror the DTA app's JSON keys so a serialized agreement validates as-is. | `DataTransferAgreement`, `Study`, `Party`, `TransferRequirements`, `VisitMapping`, `BcSelection` |
+| **Agreement** | The governance contract: what data, by whom, in what format, mapped to which visits, comprising **which tests**, in **which dataset structure**. | `DataTransferAgreement`, `Study`, `BcSelection`, `TestSpecification`, `TransferDataset`, `Party`, `TransferRequirements`, `VisitMapping` |
 | **Semantic** | The bridge from a biomedical concept to concrete SDTM-LB variables, anchored to a real, citable CDISC COSMoS specialization. | `BiomedicalConcept`, `DatasetSpecialization`, `VariableSpecialization` |
-| **Instance** | The nested lab transmission payload that fulfils the agreement. The LAB Tx Model's three parallel unit blocks (`PLR*`/`CVU*`/`SIU*`) collapse into one `Result` keyed by `unitSystem`. | `Transmission`, `Subject`, `Collection`, `Specimen`, `Panel`, `LabTestResult`, `Result` |
+| **Instance** | The delivered payload: one flat record per lab result, typed from the inventory's agreed data structure. | `Transmission`, `TransferRecord` |
 
 `BiomedicalConcept` is the hub: USDM references it (`Activity.biomedicalConceptIds`), the
-Agreement scopes it in (`BcSelection.bc_id` + the `is_used_by_dta` facet), and the payload
-delivers it (`LabTestResult.specialization.biomedicalConcept`) en route to SDTM-LB.
+Agreement scopes it in (`BcSelection.bc_id` + the `is_used_by_dta` facet, and per-test via
+`TestSpecification.bc_id`), and the payload delivers it en route to SDTM-LB.
+
+## Agreement ↔ instance
+
+Contract-vs-fulfilment is modelled explicitly rather than merely asserted, at two levels:
+
+- **File level** — `Transmission.dta_version_ref` names the DTA document version a
+  delivery fulfils, so a received file traces back to the contract that authorised it.
+- **Test level** — a **natural-key join**: `TestSpecification` (`test_panel_name` +
+  `data_provider_test_code`, falling back to `data_provider_test_name`) ↔ `TransferRecord`
+  (`lbpanel` + `ctestcd` / `ctest`).
+
+No surrogate key is used, deliberately: no real transfer file carries one, so a
+`spec_id` slot would exist in the model and in no actual data. The name fallback is
+needed because tests are not always coded — in the source inventory's own examples, the
+flow-cytometry and IHC rows identify tests by name only.
+
+**Why both layers exist.** The SDTM-LB derivation is a function of
+`(TransferRecord × TestSpecification)`. The payload carries the value; the agreement
+carries the per-test facts no payload can — the unit conversion basis, the result form,
+and the BC identity. `TestSpecification.data_type` in particular is what tells the
+transform whether `LBORRES` may be cast to `LBSTRESN`; without it, a categorical result
+such as a urinalysis colour is attempted as a number.
+
+## Design rule — permissive payload, strict agreement
+
+Enumerations bind on the **agreement** side, where the sponsor controls the values.
+Payload slots a vendor populates (units, status, dates) stay unconstrained strings, so a
+conformant-but-unanticipated file is not rejected at ingest. Discrepancies surface in the
+transform, by comparing payload against agreement, rather than as a load failure.
+
+Two consequences worth knowing:
+
+- `UnitEnum` and `LbTestCdEnum` are **documented reference vocabularies, bound to no
+  slot**. Real transfers legitimately carry units outside any fixed subset.
+- Only the variables the source structure types as numeric are typed numerically. That
+  is what keeps a categorical result from being coerced.
 
-Every instance slot carries **`exact_mappings`** to its LAB Transmission Model variable
-(identity — same element, renamed) and an **`sdtm_lb_target`** annotation to its SDTM-LB
-variable (transformation target — deliberately *not* modeled as identity, because a
-derivation happens). That distinction keeps the lineage honest in both directions.
+## Lineage annotations
+
+`labtx:` `exact_mappings` point at LAB Transmission Model v2.0 variables (identity — same
+element, renamed). `sdtm_lb_target` annotations point at SDTM-LB variables (transformation
+target — deliberately *not* identity, because a derivation happens). `usdm_property`
+annotations carry the upstream USDM attribute for agreement fields. That distinction keeps
+the lineage honest in all three directions.
+
+LAB v2 is no longer the transfer structure, but the `labtx:` mappings are retained: it
+remains the vocabulary the inventory's test specification maps every column back to, so it
+is still a valid mapping target.
+
+Slots that are **project-defined rather than CDISC-registered** say so in their own
+annotations (`transfer_structure_version`, `dta_version_ref`). Nothing is presented as
+standard terminology unless it is.
 
 ## Files
 
 | File | Contents |
 |------|----------|
-| `dta.linkml.yaml` | The schema — 20 classes, slots, and CT-bound enums (NCI EVS). |
-| `agreement.example.yaml` | Agreement-layer instance. Validates with `-C DataTransferAgreement`. |
+| `dta.linkml.yaml` | The schema — 19 classes, 191 slots, 15 enums. |
+| `agreement.example.yaml` | Agreement-layer instance, incl. test specification and data structure. Validates with `-C DataTransferAgreement`. |
 | `dataset_specialization.example.yaml` | Semantic-layer instance. Validates with `-C DatasetSpecialization`. |
-| `transmission.example.yaml` | Full instance payload (2 subjects, hematology + chemistry). Validates with `-C Transmission`. |
-| `transmission_minimal.example.yaml` | Minimal instance payload showing a linked (HGB) and an unlinked (RBC, gap-case) analyte. |
+| `transmission.example.yaml` | Full payload (2 subjects, hematology + chemistry, plus a categorical result and a not-done test). Validates with `-C Transmission`. |
+| `transmission_minimal.example.yaml` | Minimal payload showing an agreed test and one absent from the agreement (gap case). |
 | `cosmos/hgbbld.specialization.yaml` | Vendored CDISC COSMoS Dataset Specialization for Hemoglobin (HGBBLD / C64848). |
 | `cosmos/PROVENANCE.md` | Source, retrieval, and re-derivation instructions for the vendored spec. |
 | `index.html` | The DTA app — a browser-only demo that derives an agreement from USDM + MVP metadata and exports the JSON the Agreement layer types. |
@@ -51,6 +101,36 @@ linkml-validate -s dta.linkml.yaml -C Transmission           transmission.exampl
 linkml-validate -s dta.linkml.yaml -C Transmission           transmission_minimal.example.yaml
 ```
 
+## What changed in v0.2.0
+
+| | v0.1.0 | v0.2.0 |
+|---|---|---|
+| Test-level agreement | *(none)* | `TestSpecification` |
+| Declared file structure | *(none)* | `TransferDataset` / `TransferVariable` |
+| Instance shape | nested LAB v2 graph: `Subject > Collection > Specimen > Panel > LabTestResult > Result` | flat `TransferRecord` (60 variables) |
+| Agreement ↔ instance | asserted in prose | file-level ref + test-level natural-key join |
+| USDM traceability | class-level on `Study` only | per-field on study, party, visit slots |
+| Classes / enums | 21 / 19 | 19 / 15 |
+
+The instance change is a change of **transfer vocabulary**, not merely of shape: the
+removed graph modelled LAB v2 variables (`LTVRSN`, `ACCSNID`, `SPECID`, `PLBTID`,
+`PLRCRS`…), while `TransferRecord` models SDTM-shaped columns plus vendor passthrough
+(`CTESTCD`, `CTEST`, `CUNIT`, `CSPEC`, `CMETHOD`, `CPANEL`) and ten `AUX` escape hatches.
+The two share essentially only `STUDYID`.
+
+### Known limitations of the agreed transfer structure
+
+Two properties of the transfer structure itself — not of this model — constrain what SDTM
+can be produced. Neither is worked around here, because inventing columns would
+misrepresent the agreed structure:
+
+1. **No standardised result or unit.** Only the original result (`LBORRES` / `LBORRESU`)
+   is carried. `LBSTRESC` / `LBSTRESN` / `LBSTRESU` must therefore be derived from
+   `TestSpecification`, which makes a complete test specification load-bearing rather
+   than merely useful.
+2. **No reference ranges.** There is no equivalent of `LBORNRLO`, `LBORNRHI` or
+   `LBNRIND`, so those SDTM variables are not derivable from a transfer alone.
+
 ## License
 
 MIT, per this repository.
diff --git a/src/dta2sdtm/agreement.example.yaml b/src/dta2sdtm/agreement.example.yaml
@@ -114,3 +114,133 @@ timepoint_mapping:
     data_provider_timepoint_number: "T1"
     data_provider_timepoint_name: "PREDOSE"
     source: UI
+
+# ─────────────────────────────────────────────────────────────────────────────
+# TEST SPECIFICATION (new in v0.2.0) — the agreed tests.
+# Joined from the payload on (test_panel_name -> lbpanel,
+# data_provider_test_code -> ctestcd), falling back to
+# data_provider_test_name -> ctest where the vendor code is absent.
+#
+# Row 1 is the worked example from the DTA MVP Inventory's Test_Specification
+# sheet verbatim. Rows 2-3 back the transmission.example.yaml payload. Row 4 shows a
+# test identified by NAME ONLY — the flow-cytometry pattern, which is why the
+# natural key needs a name fallback.
+# ─────────────────────────────────────────────────────────────────────────────
+test_specification:
+  - test_panel_name: Blood Differential
+    category: HEMATOLOGY
+    bc_id: C51950
+    data_provider_test_code: WBC
+    data_provider_test_name: Leukocytes
+    data_provider_test_description: Leukocytes in Blood
+    test_name: Leukocytes
+    test_code: "620043"
+    method: HPLC
+    specimen: BLOOD
+    loinc_code: 26471-3
+    data_type: numeric
+    reported_unit: "10^9/L"
+    conventional_unit: mcL
+    si_unit: mm3
+    blinding_flag: true
+    agreement_status: Ordered
+
+  - test_panel_name: HEMATOLOGY
+    category: HEMATOLOGY
+    bc_id: C64848
+    data_provider_test_code: HGB
+    data_provider_test_name: Hemoglobin
+    test_name: Hemoglobin
+    test_code: HGB
+    method: HPLC
+    specimen: BLOOD
+    data_type: numeric
+    reported_unit: g/dL
+    conventional_unit: g/dL
+    si_unit: g/L
+    blinding_flag: false
+    agreement_status: Final
+
+  # Categorical test: data_type is what stops LBORRES "STRAW" being cast to LBSTRESN.
+  - test_panel_name: URINALYSIS
+    category: URINALYSIS
+    bc_id: C51948
+    data_provider_test_code: WBCU
+    data_provider_test_name: Color Urine
+    test_name: Color
+    test_code: COLOR
+    method: DIPSTICK
+    specimen: URINE
+    data_type: "<text 200>"
+    result_values: CL_RESULT1
+    blinding_flag: true
+    agreement_status: Final
+
+  # Identified by NAME only — no vendor test code (flow cytometry pattern).
+  - test_panel_name: Revised T Cell Panel (V2_SP34-2)
+    category: HEMATOLOGY
+    bc_id: C51948
+    data_provider_test_name: Naive_CD4_Th/CD3+
+    test_name: Naive CD4 T Helper Cells
+    test_code: CD4NAIVE
+    method: FLOW CYTOMETRY
+    specimen: Cryo PBMC
+    data_type: numeric
+    reported_unit: "%"
+    blinding_flag: false
+    agreement_status: Received
+
+# ─────────────────────────────────────────────────────────────────────────────
+# DATA STRUCTURE (new in v0.2.0) — the declared shape of the transferred file.
+# Abridged: a representative sample of the 61 declared variables, chosen to cover
+# each kind of column metadata (length, CT, core, mandatory, vendor naming).
+# ─────────────────────────────────────────────────────────────────────────────
+data_structure:
+  dataset_name: LB
+  dataset_description: Laboratory test results transfer dataset.
+  variables:
+    - variable_name: STUDYID
+      variable_label: Study Identifier
+      variable_description: Unique identifier for a study.
+      variable_format: text
+      variable_length: 40
+      mandatory_variable_flag: true
+      mandatory_value: true
+
+    - variable_name: DOMAIN
+      variable_label: Domain Abbreviation
+      variable_description: Two-character abbreviation for the domain.
+      variable_format: text
+      variable_length: 2
+      controlled_terminology: DOMAIN
+      example_values: [LB, CP, MB]
+      core_variable: Mandatory/Conditional
+      mandatory_variable_flag: false
+      mandatory_value: false
+
+    - variable_name: LBORRES
+      variable_label: Result or Finding in Original Units
+      variable_description: Original result of the measurement or finding, as reported to the sites.
+      variable_format: text
+      variable_length: 200
+      mandatory_variable_flag: true
+
+    - variable_name: VISITNUM
+      variable_label: Visit Number
+      variable_description: Clinical encounter number. Numeric version of VISIT, used for sorting.
+      variable_format: numeric
+      variable_length: 8
+
+    - variable_name: LBDTC
+      variable_label: Date/Time of Specimen Collection
+      variable_description: Specimen collection date or date and time, ISO 8601.
+      variable_format: datetime
+      variable_length: 19
+
+    - variable_name: CTESTCD
+      variable_label: External Vendor Test Code
+      variable_description: Populated with the vendor internal test code.
+      variable_format: text
+      variable_length: 200
+      vendor_variable_name: CTESTCD
+      vendor_variable_label: Vendor Test Code