介绍𝐒𝐩𝐞𝐜𝐮𝐥𝐚𝐭𝐢𝐯𝐞𝐃𝐞𝐜𝐨𝐝𝐢𝐧𝐠：𝐇𝐨𝐰𝐈𝐭 𝐄𝐯𝐨𝐥𝐯𝐞𝐝，𝐖𝐡𝐞𝐧𝐈𝐭𝐒𝐭𝐚𝐲𝐬𝐋𝐨𝐬𝐬𝐥𝐞𝐬𝐬， 𝐚𝐧𝐝�…
关于推测解码的交互式教程：为什么它更快，为什么拒绝采样保持无损，如何评估它，以及接下来会发生什么。
查看英文原文
Introducing 𝐒𝐩𝐞𝐜𝐮𝐥𝐚𝐭𝐢𝐯𝐞 𝐃𝐞𝐜𝐨𝐝𝐢𝐧𝐠: 𝐇𝐨𝐰 𝐈𝐭 𝐄𝐯𝐨𝐥𝐯𝐞𝐝, 𝐖𝐡𝐞𝐧 𝐈𝐭 𝐒𝐭𝐚𝐲𝐬 𝐋𝐨𝐬𝐬𝐥𝐞𝐬𝐬, 𝐚𝐧𝐝 𝐖𝐡𝐚𝐭'𝐬 𝐍𝐞𝐱𝐭. An interactive tutorial @Madisonkanna and I built for the NeurIPS Education Track. Every LLM you use generates one token at a time. Autoregressive decoding is the bottleneck for inference. Speculative decoding accelerates this, and today it runs under nearly every hosted LLM. It is a cornerstone topic to learn in the LLM stack. Blog: https://neurips2026-speculative-decoding.vercel.app/
An interactive tutorial on speculative decoding: why it is faster, why rejection sampling keeps it lossless, how to evaluate it, and what comes next.
Introducing 𝐒𝐩𝐞𝐜𝐮𝐥𝐚𝐭𝐢𝐯𝐞 𝐃𝐞𝐜𝐨𝐝𝐢𝐧𝐠: 𝐇𝐨𝐰 𝐈𝐭 𝐄𝐯𝐨𝐥𝐯𝐞𝐝, 𝐖𝐡𝐞𝐧 𝐈𝐭 𝐒𝐭𝐚𝐲𝐬 𝐋𝐨𝐬𝐬𝐥𝐞𝐬𝐬, 𝐚𝐧𝐝 𝐖𝐡𝐚𝐭'𝐬 𝐍𝐞𝐱𝐭. An interactive tutorial @Madisonkanna and I built for the NeurIPS Education Track. Every LLM you use generates one token at a time. Autoregressive decoding is the bottleneck for inference. Speculative decoding accelerates this, and today it runs under nearly every hosted LLM. It is a cornerstone topic to learn in the LLM stack. Blog: https://neurips2026-speculative-decoding.vercel.app/
An interactive tutorial on speculative decoding: why it is faster, why rejection sampling keeps it lossless, how to evaluate it, and what comes next.
Speculative Decoding: How It Evolved, When It Stays Lossless, and What's Next
An interactive tutorial on speculative decoding: why it is faster, why rejection sampling keeps it lossless, how to evaluate it, and what comes next.
- 来源
- X @swyx · x-swyx
- 证据角色
- x_open_model_network
- 时间状态
- known
- 用户判决
- 未判