Skip to content

Lazy Loading for WflowSBM_HPC returns DataArrays instead of Tensors if lazy loading is active #16

@lineick

Description

@lineick

If lazy loading is true the following conversion does not happen:

        if not self.cfg.data_lazy_load:  # Only if we're not doing lazy loading
            # Convert xarray to pre-processed tensors
            self.xd = torch.from_numpy(
                self.xd.transpose("time", "feat", "lat", "lon").values
            )
            self.y = torch.from_numpy(
                self.y.transpose("time", "feat", "lat", "lon").values
            )
            self.xs = torch.from_numpy(
                self.xs.transpose("feat", "lat", "lon").values
            )

This leads to a xarray.DataArray being returned instead of a tensor in get_item.

It can be easily fixed by converting to tensors on the fly in __get_item__, however this is suboptimal as such a conversion would be triggered for each data sample.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinginvalidThis doesn't seem right

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions