Input sample rates used for training the 48k pretrained checkpoint

[felipeespic]

Hello, I was wondering which upsampling rates were used when training the 48k pretrained checkpoint. For example, did you use rate = 2 (24k → 48k), rate = 3 (16k → 48k), rate = 6 (8k → 48k), or others?

In other words, which input sample rates were used during training?

Thanks!

[yoyolicoris]

Hi @felipeespic, thanks for asking.
The 48k chekcpoint was trained on 48 kHz audio.
The training is to learn a generative prior that can generate high-sampling-rate speech, which is different from the supervised super-resolution method, where paired low-rate inputs and high-rate targets are given.
The upsampling rate is introduced only in inference to alter the model's sampling process.

[felipeespic]

@yoyolicoris Thanks for your reply. I am trying to convert 16kHz speech to 48kHz using UDM+ (speech super-resolution). Essentially, I would like to replicate these results. How could I do that? Thanks!

[yoyolicoris]

Hi, we don't have a plug-and-play kind of utility.
I suggest having a look at vctk_infer.py.
The inference code of UDM+ is defined at

diffwave-sr/vctk_infer.py

Line 252 in 8e60d45

def reverse_manifold(y_hat,

[felipeespic]

Thank you! I'll have a look at it.