I really appreciate that you have made your code public, but there are a few issues here:
According to the pseudo-code in training about the definition of xt,there are inconsistencies between the definition and the code. I'm not sure if it should be multiplied by (x_start - mu).
noise = default(noise, lambda: torch.randn_like(x_start)) # * (x_start - mu)
def q_sample(self, x_start, mu, t, noise=None):
noise = default(noise, lambda: torch.randn_like(x_start))
return (
mu + (x_start - mu) * extract(self.Theta, t, x_start.shape) + extract(self.Sigma, t,
x_start.shape) * noise
)
In the paper, the hyperparameter λ is stated to be 10/255≈0.0392 . However, in the released code, we found lamb = 1e-4.
In the paper, the hyperparameter learning rate is stated to be 1e-4. However, in the released code, we found lamb = 8e-5 .
I really appreciate that you have made your code public, but there are a few issues here:
According to the pseudo-code in training about the definition of xt,there are inconsistencies between the definition and the code. I'm not sure if it should be multiplied by (x_start - mu).
noise = default(noise, lambda: torch.randn_like(x_start)) # * (x_start - mu)
def q_sample(self, x_start, mu, t, noise=None):
noise = default(noise, lambda: torch.randn_like(x_start))
return (
mu + (x_start - mu) * extract(self.Theta, t, x_start.shape) + extract(self.Sigma, t,
x_start.shape) * noise
)
In the paper, the hyperparameter λ is stated to be 10/255≈0.0392 . However, in the released code, we found lamb = 1e-4.
In the paper, the hyperparameter learning rate is stated to be 1e-4. However, in the released code, we found lamb = 8e-5 .