Why to Sample Configuration during Evaluation Phase?

[JiangJiaWei1103]

Hi authors,

First, thanks a lot for sharing this amazing work. I have one question about the function eval_epoch in custom_tpu_train. Because the final task is to rank all configurations per graph, why do we need to sample limited number of configurations during evaluation (e.g., on validation set). Doesn't this lead to distribution mismatch between validation and test scenarios?

Thanks.