Randomness of accuracy

[caiyu6666]

Thanks for your excellent work.
I have observed a slight variance in accuracy when repeating experiments under identical settings. This is puzzling given that the seed has been fixed. Could you kindly clarify the source of this randomness?

[GoatWu]

@caiyu6666 Hi,
Thank you for raising this observation! I've investigated the source of variance, and it appears to be related to our percentile calculation function PostGeluLogBasedBatchingQuantLinear.positive_percentile. In PTQ with low-bit precision, minor variations introduced by this process may lead to different final results.

[caiyu6666]

@GoatWu Could you help clarify the specific step that introduces randomness? It seems that each process within PostGeluLogBasedBatchingQuantLinear.positive_percentile is stationary.
Thanks!