SPECULATIVE SAMPLING

[sdzhaojingyu]

Hello, I would like to ask why p2 needs to be corrected after the sampling is rejected, instead of directly using p2?

[JasperDekoninck]

I am not sure exactly what you mean when you refer to "p2". Could you clarify the question?

[sdzhaojingyu]

Thank you very much for your reply. Here’s my question: For example, during speculative sampling on (p1, p2), if the token proposed by p1 is rejected, I noticed that in the code it uses p2 – p1, i.e., the corrected version of p2 mentioned in the paper, to generate a token. I’d like to know why we don’t directly use p2.

[sdzhaojingyu]

I’m really sorry, I just realized that this seems to be a common setup in speculative sampling. Perhaps I should first spend some time learning more about speculative sampling. Thank you for your time.

[JasperDekoninck]

No worries, it's an interesting topic, so you are not going to be disappointed :)