Rule blocked: CG00012 - custom domain class identification in library

[SFJohnson24]

Describe the bug
CG00012 - positive data. The execution halts for XY in get_variables_metadata_from_standard() as it is unable to find the custom domain class. We need logic to ID custom domains--potentially by iterating through the columns and checking the variables to infer the class.

[eljanssens]

@SFJohnson24 I'm wondering if for comparing datatype library versus dataset metadata for custom domains, we really need to know to which class the custom domain belongs to. Can't we just match on variable name --TESTCD = XYTESTCD or USUBJID = USUBJID,...
I'm checking with jozef if this could be from a standards perspective a valid approach.
Same for CG0010 define_variable_role versus library_variable_role: Could it work just by matching the variable name with the model variable names in the library?

[SFJohnson24]

note: this will also need to be applied for get_model_column_order and the permissible/required/etc operations that also check library

[chowsanthony]

Specifically for CG0012

There are only two data types in SEND, SDTM, and ADaM: numeric and character. The simpler approach is to create one list of numeric variables for all general observation classes in the associated SDTM (e.g., {VISITNUM, VISITDY, --TPTNUM, --SEQ, --LLTCD, --STRESN, ...}), and another list of character variables (e.g., {USUBJID, STUDYID, --TRT, --TESTCD,...}). For each domain of interest, compare the data type of each variable against these lists. This method works regardless of the class and regardless of whether a domain is custom.

Note: The same variable used in different general observation classes always has the same data type.

[eljanssens]

@SFJohnson24 : as discussed in the office hours, preference would be to search a brute force approach, iterating through the datasets and the variables in the model. Note to Sam: there a file in the cache called variable_metadata , is this metadata from standards , look where it is coming from.

[eljanssens]

@SFJohnson24 just FYI - test data is available on the folder to test this out. Currently works perfectly for non-custom domains. Test data for custom domain is also available in same sheet: CORE Rules - Documents\unitTesting\SDTMIG\CG0010\negative\01\data
Current error for custom domains is:
XD":[1 item
0:{6 items
"executionStatus":"skipped"
"dataset":"xd.xpt"
"domain":"XD"
"variables":[]0 items
"message":"rule evaluation error - evaluation dataset failed to build"
"errors":[1 item
0:{3 items
"dataset":"xd.xpt"
"error":"Error occurred during dataset building"
"message":"Failed to build dataset for rule validation. Builder: DefineVariablesWithLibraryMetadataDatasetBuilder, Dataset: XD, Error: 'NoneType' object has no attribute 'lower'