Skip to content

get_dataset() incorrect calls #1672

Description

@SFJohnson24

local data service has logic that can handle both a full path or the filename resulting in several problematic calls to get_dataset(). the filename is expected in all other data services, not the full path. the problematic calls are:
parent_library_model_column_order.py

  parent_dataframe = self.data_service.get_dataset(
      dataset_name=parent_datasets[0].full_path
  )

variable_count.py

  data: pd.DataFrame = self.data_service.get_dataset(
              dataset_name=dataset.full_path
          )

variable_exists.py

  dataframe = self.data_service.get_dataset(dataset_name=self.params.dataset_path)

variable_is_null.py

dataframe = self.data_service.get_dataset(
    dataset_name=self.params.dataset_path
 )

variable_value_count.py

  if dataset_metadata.is_split:
      corresponding_datasets = get_corresponding_datasets(
          self.params.datasets, dataset_metadata
      )
      data: DatasetInterface = self.data_service.concat_split_datasets(
          self.data_service.get_dataset, corresponding_datasets
      )
  else:
      data: DatasetInterface = self.data_service.get_dataset(
          dataset_name=os.path.join(
              self.params.directory_path, dataset_metadata.filename
          )
      )
      data = tag_source(data, dataset_metadata)

dataset_preprocessor.py

def _download_dataset(self, filename: str) -> DatasetInterface:
    return self._data_service.get_dataset(
        dataset_name=os.path.join(
            os.path.dirname(self._dataset_metadata.full_path), filename
        )
    )

ruleprocessor.py

  operation_params.dataframe = self.data_service.get_dataset(
      dataset_name=file_path
  )

sdtm_utilities.py

class_name = data_service._handle_custom_domains(
            data_service.get_dataset(dataset_name=dataset_metadata.full_path),
            dataset_metadata,
            dataset_path,
            datasets,
        )

these should be resolved, with the ability to process a complete path removed from local data service, as well as tests updated

Metadata

Metadata

Assignees

Labels

Type

Fields

Priority

None yet

T-shirt sizing

None yet

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions