parameter settings

[JJJTF]

Hello, thank you very much for sharing. Can you give detailed parameter settings? There are also parameters such as environment settings version.

[shadow2469]

You can find the hyperparameters of the model in ./configs/BiRSwinT/BiRWinT.yaml and ./config.py. Also, we have uploaded requirement.txt, where you can find the dependencies.

[JJJTF]

Thank you very much for your prompt reply!
The （./config.py） "_C.ROUNDS = []" parameter is empty and seems to affect the running of your code. Can you tell me what the parameters are in this?

[ktiays]

The parameters for ROUNDS are provided in the configs/BiRSwinT/BiRSwinT.yaml file.
This parameter is used internally to control the training process.

[JJJTF]

In code
checkpoint1 = torch.load(config.DATA.SWIN_TRANSFORMER_CHECKPOINT_PATH, map_location="cpu")
checkpoint2 = torch.load("swin_small_patch4_window7_224.pth", map_location="cpu"),
but ckpt_epoch_44.pth file is not known where, is it the pre-training parameter of Swin-B-IN1K?
Also checkpoint1 and checkpoint2 are both loaded pre-training models, what's the difference between them. Thank you very much!

[shadow2469]

Checkpoint1 is the training parameter of Swin-S model on the AUC dataset, and checkpoint2 is the pre-training parameter of Swin-B-IN1K. Thanks for your issue, we will soon upload a complete get_started.md.

[KevinLi-167]

@shadow2469
checkpoint1 = torch.load(config. DATA. SWIN_TRANSFORMER_CHECKPOINT_PATH, map_location="cpu")
checkpoint2 = torch.load("swin_small_patch4_window7_224.pth", map_location="cpu"),

checkpoint1 uses the file ckpt_epoch_44.pth file as the global representative branch.

This means that I can run SwinT on the dataset AUC to get this pre-trained model for this project?

[shadow2469]

@shadow2469 checkpoint1 = torch.load(config. DATA. SWIN_TRANSFORMER_CHECKPOINT_PATH, map_location="cpu") checkpoint2 = torch.load("swin_small_patch4_window7_224.pth", map_location="cpu"),

checkpoint1 uses the file ckpt_epoch_44.pth file as the global representative branch.

This means that I can run SwinT on the dataset AUC to get this pre-trained model for this project?

Yes, ckpt_epoch_44.pth is the pre-trained result with Swin Transformer on the AUC dataset. You can use it to train BiRSwinT.

[KevinLi-167]

Q1 AUC for Swin-T

You implemented Swin Transformer's support for AUC datasets, but did not provide this part of the code, right?

Q2 Why vaild results are so much better than train results

(Bis) PS D:\BiRSwinT-main> python train_valid.py --cfg=configs/BiRSwinT/BiRSwinT.yaml
=> merge config from configs/BiRSwinT/BiRSwinT.yaml
{'c0': 0, 'c1': 1, 'c2': 2, 'c3': 3, 'c4': 4, 'c5': 5, 'c6': 6, 'c7': 7, 'c8': 8, 'c9': 9}
{'c0': 0, 'c1': 1, 'c2': 2, 'c3': 3, 'c4': 4, 'c5': 5, 'c6': 6, 'c7': 7, 'c8': 8, 'c9': 9}
17943 4481
Epoch: 1/2
reserve a better valid model
Epoch: 001, Training: Loss: 0.8297, Accuracy: 74.3577%,
                Validation: Loss: 0.3286, Accuracy: 89.5559%, Accuracy3: 98.6387%, Accuracy5: 99.7322%,Time: 1052.1377s
Best Accuracy for validation : 0.8956 at epoch 001
Epoch: 2/2
reserve a better valid model
Epoch: 002, Training: Loss: 0.7074, Accuracy: 76.1132%,
                Validation: Loss: 0.2960, Accuracy: 90.4709%, Accuracy3: 99.0850%, Accuracy5: 99.8884%,Time: 1160.4479s
Best Accuracy for validation : 0.9047 at epoch 002

I tried epoch for 2 and found that there was a problem with the output.

Epoch 1、2 showed that the Vaild score was higher after training than the train

Q3 Potential errors in the training process

So I suspect it might have something to do with this:
(But it only explains what happened after epoch1)

if avg_valid_acc > best_acc:
    ctx.latest_round_result = trained_param_path

Whether this step is a deliberate fitting of the vaild?

I don't know if I misunderstood,
So I changed part of the code: "Each time it is used, it is the previous one, not the optimal" like:

# All save and load, not just save and load the best till train
if avg_valid_acc > best_acc:
    torch.save( trained_param_path_better )
    ctx.latest_round_result = trained_param_path_better
else :
    torch.save( trained_param_path)
    ctx.latest_round_result = trained_param_path

But even if the training logic here is so, it still can't explain epoch1 with preTrained-SwinT-onAUC.
(Unless the AUC is trained to overfit the test dataset?)

Q4 The role of the dataset vaild in the train_vaild function

train_vaild function, train is contrastive learning, so there is no need for vaild?
vaild is an ensemble for inference, the actual purpose is to evaluate immediately after training for easy viewing, right?
the reason why the vaild dataset is used instead of the test dataset is that the test dataset of AUC does not have a partition file?

The above is my doubts and understanding.
Thanks for the reply.

[shadow2469]

Q1 AUC for Swin-T

You implemented Swin Transformer's support for AUC datasets, but did not provide this part of the code, right?

Q2 Why vaild results are so much better than train results

(Bis) PS D:\BiRSwinT-main> python train_valid.py --cfg=configs/BiRSwinT/BiRSwinT.yaml
=> merge config from configs/BiRSwinT/BiRSwinT.yaml
{'c0': 0, 'c1': 1, 'c2': 2, 'c3': 3, 'c4': 4, 'c5': 5, 'c6': 6, 'c7': 7, 'c8': 8, 'c9': 9}
{'c0': 0, 'c1': 1, 'c2': 2, 'c3': 3, 'c4': 4, 'c5': 5, 'c6': 6, 'c7': 7, 'c8': 8, 'c9': 9}
17943 4481
Epoch: 1/2
reserve a better valid model
Epoch: 001, Training: Loss: 0.8297, Accuracy: 74.3577%,
                Validation: Loss: 0.3286, Accuracy: 89.5559%, Accuracy3: 98.6387%, Accuracy5: 99.7322%,Time: 1052.1377s
Best Accuracy for validation : 0.8956 at epoch 001
Epoch: 2/2
reserve a better valid model
Epoch: 002, Training: Loss: 0.7074, Accuracy: 76.1132%,
                Validation: Loss: 0.2960, Accuracy: 90.4709%, Accuracy3: 99.0850%, Accuracy5: 99.8884%,Time: 1160.4479s
Best Accuracy for validation : 0.9047 at epoch 002

I tried epoch for 2 and found that there was a problem with the output.

Epoch 1、2 showed that the Vaild score was higher after training than the train

Q3 Potential errors in the training process

So I suspect it might have something to do with this: (But it only explains what happened after epoch1)

if avg_valid_acc > best_acc:
    ctx.latest_round_result = trained_param_path

Whether this step is a deliberate fitting of the vaild?

I don't know if I misunderstood, So I changed part of the code: "Each time it is used, it is the previous one, not the optimal" like:

# All save and load, not just save and load the best till train
if avg_valid_acc > best_acc:
    torch.save( trained_param_path_better )
    ctx.latest_round_result = trained_param_path_better
else :
    torch.save( trained_param_path)
    ctx.latest_round_result = trained_param_path

But even if the training logic here is so, it still can't explain epoch1 with preTrained-SwinT-onAUC. (Unless the AUC is trained to overfit the test dataset?)

Q4 The role of the dataset vaild in the train_vaild function

train_vaild function, train is contrastive learning, so there is no need for vaild? vaild is an ensemble for inference, the actual purpose is to evaluate immediately after training for easy viewing, right? the reason why the vaild dataset is used instead of the test dataset is that the test dataset of AUC does not have a partition file?

The above is my doubts and understanding. Thanks for the reply.

Reply for Q1

You can access the official Swin Transformer code from Microsoft here: Swin-Transformer

Reply for Q2

Because we adopted more image augmentation operations in the training phase, the learning difficulty for the model is increased, resulting in lower accuracy. This is also the case in Microsoft's Swin Transformer code.

Reply for Q3

In general, the parameter of each model training should be saved. However, due to storage limitations, we only save the best-performing result. You can change the code according to your circumstances.

Reply for Q4

Since the AUC official website does not provide labels for the test dataset, we evaluate the training performance by using the model's performance on the validation dataset. Therefore, in the code, after each epoch of training, we execute validation on the validation dataset.

[323417]

@shadow2469
Hello, I would like to ask you two questions:
Firstly, in your previous answer, you mentioned "Checkpoint1 is the training parameter of Swin-S model on the AUC dataset, and checkpoint2 is the pre-training parameter of Swin-B-IN1K", but in the following code, does the file name corresponding to Checkpoint2 indicate that it should be a pre training parameter for Swin-S, not Swin-B-IN1K?

checkpoint1 = torch.load(config.DATA.SWIN_TRANSFORMER_CHECKPOINT_PATH, map_location="cpu") checkpoint2 = torch.load("swin_small_patch4_window7_224.pth", map_location="cpu")

Secondly, from the configuration tutorial, I obtained a file named ckpt_epoch_44.pth that corresponds to checkpoint1. I speculate it is the model trained by a certain Swin model after 44 epochs. If I want to retrain this model instead of continuing training from the 45th epoch, how can I find the pre trained model that matches its initial state? I sincerely look forward to your answer.

[shadow2469]

@shadow2469 Hello, I would like to ask you two questions: Firstly, in your previous answer, you mentioned "Checkpoint1 is the training parameter of Swin-S model on the AUC dataset, and checkpoint2 is the pre-training parameter of Swin-B-IN1K", but in the following code, does the file name corresponding to Checkpoint2 indicate that it should be a pre training parameter for Swin-S, not Swin-B-IN1K?

checkpoint1 = torch.load(config.DATA.SWIN_TRANSFORMER_CHECKPOINT_PATH, map_location="cpu") checkpoint2 = torch.load("swin_small_patch4_window7_224.pth", map_location="cpu")

Secondly, from the configuration tutorial, I obtained a file named ckpt_epoch_44.pth that corresponds to checkpoint1. I speculate it is the model trained by a certain Swin model after 44 epochs. If I want to retrain this model instead of continuing training from the 45th epoch, how can I find the pre trained model that matches its initial state? I sincerely look forward to your answer.

We made a mistake in our previous answer, Checkpoint2 is indeed the pre-training parameter for Swin-S-IN1K, not Swin-B-IN1K.

If you want to retrain Swin Transformer model on the AUC dataset, you can follow this repository, but it seems to have done a lot of updating since we finished this work.

[323417]

@shadow2469
Thank you for your answer.

I do want to retrain the model on the AUC dataset to obtain complete training results. So I tried to download Swin-S ImageNet-1K 224x224 as the corresponding file for checkpoint 1 from the link you provided, but the code reported an error indicating that the weight sizes of some layers in the model did not match. Specifically, the problem arises in the head section of SwinTransformer, where the weight matrix and bias sizes are inconsistent.

May I ask if you obtained the pre trained model from this link and trained it on it? If so, could you please provide the specific initial file? Sincerely looking forward to your reply.