Multi system integration

[Hrovatin]

• resolve all comments in the commit - note that the current implementation contains incorrect code at parts. Also some comments will make the code differ a bit from my previous implementation - sometimes this is needed as my previous code also wasn't clean so it will be just better now
• Instead of commenting out code just delete it
• Adpat comments as suggested (you can do some modifications upon the suggested text if needed) as some of them are not easily understandable.
• Adapt default parameter values as suggested
• For simplicity remove everything related to eval (e.g. data_eval - just set it to None within the module (as else you would need to change too many other things) and remove function parameters (so it is always None)) and translate as this would also require adjustments
• Remove all related to "group_key" (e.g. in setup) and use_group (e.g. in module) as this is for contrastive learning we don't use
• adapt tests in https://github.com/theislab/cross_system_integration/blob/main/cross_system_integration/test/model/test_model_xxjoint.py instead of having just a run in a notebook (see below)
• Test that model runs through
• Test that the new system remains the same through the forward cycle pass
• Test that when adding system dummies the category order is correct/col

[Hrovatin]

• I now cleaned up the basic test to see if model runs through in https://github.com/theislab/cross_system_integration/blob/multiple_sys_model/cross_system_integration/test/model/test_model_xxjoint.py You can delete the file https://github.com/theislab/cross_system_integration/blob/multiple_sys_model/cross_system_integration/test/model/test_model_xxjoint_multiple_sys.py as that one wasn't cleaned up anyways so it wouldn't run through anymore. Please check that the tests I adapted run through.

[Hrovatin]

Try some practical examples:

• Check how the integration of mouse+human data you used before looks like with the new model - visually compare the umaps to make sure it is similar quality
• Add the 3th system (pig) to the embryonic data and check how that integration looks like (see https://github.com/theislab/pancreas_system_integration-/issues/1 for info on pig embryonic data)

[MattsonCam]

Hi @Hrovatin, this is a very interesting development. As an aside, is there a reason the system parameter couldn't, instead, be passed as a categorical_covariate_keys parameter or a continuous_covariate_keys parameter? Each of these parameters seem to correct for covariates.

[Hrovatin]

We separate the system from the other covariates as we use system for cycle consistency loss (see our preprint). Of course, if you disable the cycle consistency loss, it will boil down to normal cVAE covariate (as are now categorical_covariate_keys and continuous_covariate_keys).
If you are interested in using our model, I would suggest using the SysVI implementation that is a PR in scvi-tools (until it becomes official in the next release), as this repository will be only used for reproducibility purposes. The SysVI also enables the integration of multiple systems, as discussed in this issue.

[MattsonCam]

We separate the system from the other covariates as we use system for cycle consistency loss (see our preprint). Of course, if you disable the cycle consistency loss, it will boil down to normal cVAE covariate (as are now categorical_covariate_keys and continuous_covariate_keys). If you are interested in using our model, I would suggest using the SysVI implementation that is a PR in scvi-tools (until it becomes official in the next release), as this repository will be only used for reproducibility purposes. The SysVI also enables the integration of multiple systems, as discussed in this issue.

Good to know, thank you for the info!