Hi Zhengzhong,
I noticed in your evaluate script (demo_eval_BVQA_feats_all_combined.py), a grid search is used to obtain a best [gamma, c] for each iteration.
This may bring a problem: for each iteration (the 8-2 division) the best gamma c will be different.
If we have 100 params pairs, and 100 iterations. Then we should have a 100x100 searching matrix, naming K. Usually I take the median for each row (i.e. each iteration with a fixed param pair), then the max value to select the best param pair.
It takes the np.max(np.median(K, 1)) as the final result, here the np denotes numpy.
The current script essentially takes the np.median(np.max(K, 0)).
I don't think these two ways will make the same results ---- the latter one may offer a better score but it can not tell us what's the best params in the whole dataset.
Please correct me if I misunderstand anything....
Hi Zhengzhong,
I noticed in your evaluate script (demo_eval_BVQA_feats_all_combined.py), a grid search is used to obtain a best [gamma, c] for each iteration.
This may bring a problem: for each iteration (the 8-2 division) the best gamma c will be different.
If we have 100 params pairs, and 100 iterations. Then we should have a 100x100 searching matrix, naming K. Usually I take the median for each row (i.e. each iteration with a fixed param pair), then the max value to select the best param pair.
It takes the np.max(np.median(K, 1)) as the final result, here the np denotes numpy.
The current script essentially takes the np.median(np.max(K, 0)).
I don't think these two ways will make the same results ---- the latter one may offer a better score but it can not tell us what's the best params in the whole dataset.
Please correct me if I misunderstand anything....