Hi, nice work! When I reproduce your work and other PEFT methods, I find that methods like NOAH\SSF use VTAB data in format of picture which is converted by NOAH code: https://github.com/ZhangYuanhan-AI/NOAH/blob/main/data/vtab-source/get_vtab1k.py. However, in VPT and your work, data processing comes from original VTAB (data maybe tensor format). I wonder do these two forms (picture and tensor) of data have any impact on training? Here are some strange phenomenon:
(1) cifar100: models trained by picture format always achieve ~70% acc (like NOAH\SSF), while models trained by tensor format always achieve ~80% acc. In general, the same model trained by different format data of cifar100, will obtain results with relatively large differences.
(2) LoRA methods: in NOAH (https://arxiv.org/pdf/2206.04673) and your work, LoRA performance on cifar100\svhn\sun397... have a big difference.
This problem has troubled me for a long time, I guess there are several possibilities:
(1) dataset version: data in NOAH are converted by a different version.
(2) The converted pictures maybe are not aligned with tensor format data.
Do you have any ideas, looking forward to your reply !!!
Hi, nice work! When I reproduce your work and other PEFT methods, I find that methods like NOAH\SSF use VTAB data in format of picture which is converted by NOAH code: https://github.com/ZhangYuanhan-AI/NOAH/blob/main/data/vtab-source/get_vtab1k.py. However, in VPT and your work, data processing comes from original VTAB (data maybe tensor format). I wonder do these two forms (picture and tensor) of data have any impact on training? Here are some strange phenomenon:
(1) cifar100: models trained by picture format always achieve ~70% acc (like NOAH\SSF), while models trained by tensor format always achieve ~80% acc. In general, the same model trained by different format data of cifar100, will obtain results with relatively large differences.
(2) LoRA methods: in NOAH (https://arxiv.org/pdf/2206.04673) and your work, LoRA performance on cifar100\svhn\sun397... have a big difference.
This problem has troubled me for a long time, I guess there are several possibilities:
(1) dataset version: data in NOAH are converted by a different version.
(2) The converted pictures maybe are not aligned with tensor format data.
Do you have any ideas, looking forward to your reply !!!