Tencent Open-Sources AngelSpec: A Unified Framework for Multi-Token Prediction and Block-Parallel Speculative Decoding on Hy3 Models
The race to improve large language model (LLM) inference speed has led to significant innovation in speculative decoding. As AI models grow larger and serve…
aitechtonic.com
Tencent Open-Sources AngelSpec: A Unified Framework for Multi-Token Prediction and Block-Parallel Speculative Decoding on Hy3 Models
The race to improve large language model (LLM) inference speed has led to significant innovation in speculative decoding. As AI models grow larger and serve increasingly diverse workloads, reducing latency while maintaining output quality has become one of the industry's most important challenges. Tencent has now introduced AngelSpec, an open-source, PyTorch-native training framework designed specifically for speculative-decoding draft models. The framework supports both autoregressive Multi-Token Prediction (MTP) and the block-parallel DFlash family, bringing multiple speculative decoding approaches under a single training infrastructure. Unlike many previous speculative-decoding systems that focus on creating one generalized drafter optimized for benchmark averages, AngelSpec takes a different path.