Skip to content

dataframe_operator harmonization #1809

Description

@SFJohnson24

Currently there are 4 ways comparators and targets are accessed in operators:

Direct column access, whole dataset: self.value[target] — used by less_than, contains, is_contained_by, matches_regex, is_ordered_by, etc. Strict — raises KeyError on missing column. This is target-fetching in the majority of operators.

Row-wise access inside .apply(): row[target] / row[comparator] — used only by _check_equality/_check_inequality (equal_to/not_equal_to family). Strict for target, lenient-with-literal-fallback for comparator.

Column access via get_comparator_data: self.value.get(comparator, comparator) — lenient, bug.

custom comparator exceptions: value_is_reference (row[comparator] gives a column name, then row[that_column_name] — double indirection),
prefix_equal_to's if comparator == "DOMAIN": ...column_prefix_map["--"] hardcoded branch
list-valued comparators in contains_all/is_unique_set/is_contained_by_case_insensitive.

it would be good to harmonize the base fetching logic for both target and comparator to prevent differences in rule execution based on operator choice

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    T-shirt sizing

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions