Try Voice Writer - speak your thoughts and let AI handle the grammar: https://voicewriter.io
Speculative decoding (or speculative sampling) is a new technique where a smaller LLM (the draft model) generates the easier tokens which are then verified by a larger one (the target model). This make the generation faster computation without sacrificing accuracy.
0:00 - Introduction
1:00 - Main Ideas
2:27 - Algorithm
4:48 - Rejection Sampling
7:52 - Why sample (q(x) - p(x))+
10:55 - Visualization and Results
Deepmind Paper: https://arxiv.org/abs/2302.01318
Google Paper: https://arxiv.org/abs/2211.17192