CUDA out of memory when trying to train the dataset with Celcomen

[jawadad]

Hi,

I am currently trying to run Celcomen on some Xenium 5k data to extract the gene-gene weights matrices. When I run it on the snapshot, it works quite fine. But when I want to run it across my entire image for a few samples. I get the following error:
"MemoryError: CUTA out of memory. Tried to allocate ..." . I am assuming because it is trying to build the entire graph of each sample.
So from my understanding, is celcomen's model currently only supposed to learn these patterns from a snapshot not the entire sample?

[stathismegas]

Hi, thank you for raising this. Although we proved several mathematical approximations to reduce memory and time complexity, you are probably correct that Celcomen might face memory issues in the 5k dataset. In the meantime, I’d recommend running on smaller spatial crops/patches, since most of the effect of local perturbations should be local. Thanks again!