Some question about Table 3 in paper

[Chrisfsj2051]

Hi , I'm pretty excited to see such a excellent work! Currently I'm trying to reproduce the results but meet some problem about Table 3.

To be more specific, I use 100% COCO understand standard 1x setting, training wIth C+{G,B}, but obtain the result of 34 mAP, which is much lower than yours (36.39). However, I found separately using Color augmentation or cutout doesn't seems to bring negative impact. So I think there might be problem with geometrys transform.

Could please share the config for the 36.39 results? Or could you please tell me if the geometry augmentation policy in Table3 and Table1 is exactly the same (rotate angel, translate percent etc. ). Thank you very much for your time and consideration.

[Chrisfsj2051]

For bbox geometry transform, shall we apply transformation to one of those bboxes in an image, or all of them?

[kihyuks]

Hi @Chrisfsj2051 ,

To train with standard setting using 100% data, you can change run the following:

# --simple_path makes train_log/${DATASET}/${EXPNAME} as exact location to save
DATASET=coco_train2017
python3 train_stg1.py \
    --logdir ${CKPT_PATH} --simple_path --config \
    BACKBONE.WEIGHTS=${COCODIR}/ImageNet-R50-AlignPadding.npz \
    DATA.BASEDIR=${COCODIR} \
    DATA.TRAIN="('${DATASET}',)" \
    MODE_MASK=False \
    TRAIN.EVAL_PERIOD=20 \
    TRAIN.AUGTYPE_LAB='strong'

We only change script following here for more details on changing configurations).

In order to run with C+{G,B}, you will need to change the following lines of the code:

      self.augment_fn = [[self.color_augment],
                         [self.bbox_affine_transform, self.affine_transform],
                         [self.cutout_augment]]

As you can see, we define a list of lists to apply random augmentation. If you want to remove cutout, you simply discard [self.cutout_augment] from the list.

[Chrisfsj2051]

@kihyuks Thanks for your reply! Maybe I should keep on checking my code.

By the way, I found the mAP of baseline (2x/3x fully supervised setting) is quite low when compared to other detection frameworks. For example, mmdetection reaches 38.4 under 2x setting and detectron2 reaches 40.2 under 3x setting.

What might be the reason?

[kihyuks]

@Chrisfsj2051 I believe that our model is the original faster R-CNN, and thus not using the mask supervision (MODE_MASK=False). You can also see a difference in 1x performance using Tensorpack implementation, where default (i.e., using mask supervision) R50-FPN achieves 38.2 while R50-FPN with MODE_MASK=False achieves 37.5, which is the baseline used in our work.