DSCL Implement

[Charon-HN]

It was a good article, and I was interested in the idea of DSCL, so I took a look at the code implementation.

It can be seen that the implementation of DSCL is based on Supervised Contrastive Learning. But when I looked at the code, I found that the formula 9 in the article is different from the code. Below is the code for the project.
class_weighted = torch.ones_like(mask) / (mask.sum(dim=1, keepdim=True) - 1.0 + 1e-12) * self.weighted_beta class_weighted = class_weighted.scatter(1, torch.arange(batch_size).view(-1, 1).to(device) + batch_size, 1.0)
Does self.weighted_beta mean what is $\alpha$ in the paper, and if so, how does formula 9 represent it?

In addition, there is the calculation of mean_log_prob_pos, the form of calculation in the code is as follows:
mean_log_prob_pos = (mask * log_prob * class_weighted).sum(1) / (mask * class_weighted).sum(1)
Isn't this the same calculation as the SCL calculation? That is class_weighted is not involved in the calculation.

The above two questions are my current doubts, if you see this question, please also give a detailed explanation, thank you.

[SY-Xuan]

Thanks for your interest in my work. I'm sorry for the confusing implementation. I use beta here to prevent numeric errors if no positive samples are in the queue and batch. alpha = 1 / (1 + beta). You can calculate the weight of the augmented data. For example, After Scatter, the class_weighted on the augmented data is 1. class_weighted looks like [0, 0, 1, 0, beta / P, beta / P]. Therefore, class_weighted.sum() = 1 + beta. The weight of the augmented data is 1 / (1 + beta)

If you have any other question, feel free to contact me.

[Charon-HN]

Thank you very much for your reply. According to you, I understand alpha = 1 / (1+beta). That right ?

[SY-Xuan]

Yes

[Charon-HN]

Sorry to bother you again, thanks for your replys.🙇‍♂️ I have one more question, and that's line 255 of the code, namely mean_log_prob_pos = (mask * log_prob * class_weighted).sum(1) / (mask * class_weighted).sum(1). Why divide the denominator by class_weighted, shouldn't it be mean_log_prob_pos = (mask * log_prob * class_weighted).sum(1) / (mask).sum(1). according to the formula 8 in the paper?
I don't know much about this detail, and I would like your help🤝.

[SY-Xuan]

You can make a visualization for the class weighted and mask to better understand the code. The class_weighted.sum() = 1 + beta. class_weighted = [0, 0, 1, 0, beta / P, beta / P]. Therefore, we need to divide the class_weighted.sum() to make sure that the weight of the augmented data is 1 / (1 + beta).