Why high dropout on the conditional embedding

[KatarinaYuan]

Report

Hi dear teams,
Thank you so much for the wonderful work. It's surprising that the dropout on the conditional embedding is recommended to set very high. Could you please help to explain some insights behind this choice, and what's effect when setting the conditional embedding dropout to high / low value seperately?

Best,
Xinyu

Version information

No response

[MUCDK]

Hi,

The condition network sees very little data points (one per condition), thus we need to make sure we don't overfit as the main challenge is not the flow matching module (this always fits very well), but getting a good condition which guides the flow.

Hope that helps!

[KatarinaYuan]

Hi,
Is there any experiment results in the paper to show that without this trick, the performance drops? is it e-distance? Thank you for the guidance!

[MUCDK]

I think in some of the hyperparameter opt. runs we included 2 different values for that. Empirically, it seemed to improve. But by no means there is no way to improve the results, we ran thousands of models for the paper, hence a comprehensive hyperparameter search for all datasets was not feasible.

Happy to learn more about your observations.