Expected results not included for all domains/datasets

[pendingintent]

Standard

Custom

Reference Rule ID(s)

TIG0046

Conformance Rule ID(s) (if published in CORE)

CORE-000201

JIRA Ticket

https://jira.cdisc.org/projects/CORERULES/issues/CORERULES-

CLI Command Used (if applicable)

python core.py -s SDTM -v 3.4 -d /path/to/datasets

Rule YAML

Authorities: 
  - Organization: 
    Standards:
      - Name: 
        References:
          - Citations:
              - Cited Guidance: 
                Document: 
                Item: 
                Section: 
            Origin: 
            Rule Identifier: 
              Id: 
              Version: 
            Version: 
        Version:
Check:
  all:
    - name: 
      operator: 
      value: 
Core:
  Id: 
  Status: 
  Version: '1'
Description: 
Executability: Fully Executable
Match Datasets:
  - Keys:
      - 
    Name: 
Operations:
  - id: 
    operator: 
Outcome:
  Message: 
  Output Variables:
    -
Rule Type: 
Scope:
Sensitivity:

Attach any sample test data file(s)

_datasets.csv
_variables.csv
ae.csv
dm.csv
supplb.csv
ta.csv

Attach any output report and/or log file(s)

This is the latest, local results file.

results(1).csv

Expected output

This is the expected results.csv file.

results.csv

Any Additional Information

Summary

Running CORE-000201 (USUBJID in DM.USUBJID, scope: Classes.Include: ALL, Domains.Exclude: [AP--]) against its own negative/05 test fixture (Published/CORE-000201/negative/05/data in cdisc-org/cdisc-open-rules) produces only 3 of the 5 expected issues. Two datasets — TA and SUPPLB — are silently skipped due to two independent bugs.

Environment: cdisc-rules-engine 0.16.0 (commit 5e03264e, latest on main as of this report — neither bug appears fixed since)

Bug 1: non_empty/empty operator raises an uncaught KeyError when the target variable does not exist in the dataset, instead of reporting it as an issue

Repro:

• Dataset TA (Trial Arms) has no USUBJID variable at all — that's expected, TA is a trial-design dataset.
• Rule CORE-000201's Check includes {name: USUBJID, operator: non_empty}.
• Running the engine against TA raises:

  KeyError: 'USUBJID'
    File ".../cdisc_rules_engine/check_operators/dataframe_operators.py", line 978, in empty
      series = self.value[target]
    File ".../cdisc_rules_engine/models/dataset/pandas_dataset.py", line 66, in __getitem__
      return self._data[item]
  

• The raw JSON report (-of JSON -rr) shows the dataset marked "executionStatus": "skipped" with "error": "Column not found in data", and no issue is emitted for TA. No CSV/XLSX row is produced either.

Expected: a dataset missing the checked variable should still be evaluated as non-conformant (the variable is absent, which is a stronger failure than merely empty) and reported as an issue — not swallowed as a skipped/errored execution. This matches the previously-generated baseline results.csv for this test case, which has a TA,,,, row.

Location: cdisc_rules_engine/check_operators/dataframe_operators.py, empty() (~line 978) / non_empty() (~line 1024). PandasDataset.__getitem__ (models/dataset/pandas_dataset.py:66) does a raw self._data[item] lookup with no existence check.

Bug 2: Domain include/exclude logic conflates the AP-- and SUPP--/SQ-- special-domain patterns, causing SUPP-- datasets to be wrongly excluded by an AP-- exclusion (and vice versa)

Repro:

• Rule CORE-000201 sets Domains: {Exclude: [AP--]} only — no mention of SUPP or SQ.
• Dataset SUPPLB (a SUPP-- qualifier dataset, RDOMAIN=LB) contains a genuine violation: record 5 has USUBJID=CDISC-TEST-010, which is not in DM.USUBJID.
• The engine skips SUPPLB entirely: "message": "Rule skipped - doesn't apply to domain for rule id=CORE-000201, dataset=SUPPLB", so the real violation is never reported.

Root cause: RuleProcessor._domain_matched_ap_or_supp() (cdisc_rules_engine/utilities/rule_processor.py:174-187):

@classmethod
def _domain_matched_ap_or_supp(
    cls, dataset_metadata: SDTMDatasetMetadata, domains_to_check: List[str]
) -> bool:
    supp_ap_domains = {f"{domain}--" for domain in SUPPLEMENTARY_DOMAINS}
    supp_ap_domains.update({f"{AP_DOMAIN}--", f"{APFA_DOMAIN}--"})

    return any(set(domains_to_check).intersection(supp_ap_domains)) and (
        dataset_metadata.is_supp or dataset_metadata.is_ap
    )

This only checks whether any AP/APFA/SUPP/SQ wildcard is present in the rule's include/exclude list, and whether the dataset is either AP-type or SUPP-type — it never checks that the specific matched pattern corresponds to the dataset's actual type. So a rule that excludes only AP-- also (incorrectly) excludes every SUPP--/SQ-- dataset, and a rule that excludes only SUPP-- would likewise incorrectly exclude AP--/APFA-- datasets.

Expected: the function should only match when the dataset's actual type (AP vs SUPP) corresponds to the specific wildcard pattern(s) present in domains_to_check, e.g.:

@classmethod
def _domain_matched_ap_or_supp(cls, dataset_metadata, domains_to_check) -> bool:
    domains_to_check = set(domains_to_check)
    supp_domains = {f"{d}--" for d in SUPPLEMENTARY_DOMAINS}
    ap_domains = {f"{AP_DOMAIN}--", f"{APFA_DOMAIN}--"}
    if dataset_metadata.is_supp and domains_to_check & supp_domains:
        return True
    if dataset_metadata.is_ap and domains_to_check & ap_domains:
        return True
    return False

Impact

Both bugs cause rules scoped broadly (Classes.Include: ALL, a narrow Domains.Exclude) to silently under-report on real submission data whenever it contains a SUPP--/SQ-- dataset or a domain missing the checked variable — with no warning surfaced to the end user in the standard CSV/XLSX report (only visible via -rr raw JSON as a "skipped"/"executionStatus" entry).

Suggested test coverage

Published/CORE-000201/negative/05 in cdisc-org/cdisc-open-rules already exercises both cases and can be used to verify a fix — its baseline results.csv (5 issues, including TA and SUPPLB rows) should match the engine's fresh output once both bugs are fixed.