Performance in Parallel

[haizi-zh]

Thanks a lot for developing this package! The original R package (sva) is really slow in non-parametric mode, largely because of un-optimized loops and concatenation (repeatedly growing vectors in each loop) in the Monte Carlo function int.eprior. I'm really suffering from it.

However, sva supports parallel computing through BiocParallel, although the parallel computing takes place at batch level. Therefore, if you have much more CPU cores than number of batches (in my case), it won't help much. Their source code: https://github.com/jtleek/sva-devel/blob/123be9b2b9fd7c7cd495fab7d7d901767964ce9e/R/ComBat.R#L263

Does pyComBat supports parallel computing as well? I didn't find the mechanism by skimming the source codes. It will be very helpful if so.

[aryoepigene]

Hey @haizi-zh, we're very sorry for the late response. Thank you very much for your question and for providing some other examples of parallel computation. At the moment, parallel computing for the non-parametric mode is not within our priority list.

However, we would like to encourage you to also contribute by creating a pull request to address this if it's something that fits your needs. Obviously, we will try our best to review and comment on your implementation to make it as robust as possible. If you're interested to contribute, you can visit our contribution guideline for more details about it.

Thank you again 😄