r/TheMachineLearning • u/Jaded_Imagination148 • 20h ago
Nvidia's speculative decoding draft boosts token acceptance by 46%
Anyone tested the actual tok/s speedup yet?
45
Upvotes
1
u/CryptographerOne7003 3h ago
I'm all ears, but is it the same or comparable size as the build in head?
3
u/This_Maintenance_834 6h ago
there is no engine ready to use this. this is all academic research.