Tencent Hunyuan Open-Sources AngelSpec, an End-to-End Speculative Decoding Framework
Core Highlights
Tencent Hunyuan has open-sourced AngelSpec, an end-to-end speculative decoding framework that covers both the training and deployment stages. On Tencent's in-house Hy3-A21B model, its DFly scheme achieves 1.98x to 2.40x end-to-end speedup over conventional autoregressive decoding, while system throughput is 10.5% to 11.8% higher than another scheme called DFlash. More importantly, the training code together with the MTP and DFly draft model weights for Hy3-A21B have all been released, so developers can start building without reproducing everything from scratch. This is a notable move because speculative decoding has mostly lived inside research papers and vendor-specific stacks; handing the community a complete, runnable package changes who can actually benefit from it. For a domestic open-source ecosystem, publishing both code and weights signals a willingness to compete on the inference side, not just on model weights.
Capabilities and What Happened
The underlying logic of speculative decoding is straightforward: a smaller draft model first guesses several upcoming tokens in one shot, which are then verified in parallel by the main model in a single pass, turning a process that had to wait serially into batched verification. The difference with AngelSpec is that it links training and deployment into one complete pipeline. In practice, you can use it to train draft models tailored to your own business, or load the officially open-sourced weights directly for online inference, without switching between two separate toolchains. That continuity matters more than it sounds: many acceleration tricks work in a demo but break when moved from the lab to production, and a single framework that spans both stages reduces that gap. A team can prototype a draft model, measure its acceptance rate, and ship the same artifact to production with far less rework than usual.
Technical Details
Hy3-A21B is a roughly 21-billion-parameter mixture-of-experts model within Tencent Hunyuan's lineup. AngelSpec ships two draft-model options for it: MTP (multi-token prediction) and DFly. Based on public numbers, DFly performs more consistently in end-to-end testing, delivering close to a 2x speedup range over token-by-token autoregressive decoding and maintaining a double-digit lead over DFlash in throughput. Open-sourcing the framework means researchers can reproduce all experiments, swap the base model, and even fine-tune draft models on specific corpora to raise the acceptance rate. A higher acceptance rate is the name of the game here: the more draft tokens the main model accepts, the closer you get to the theoretical speedup, and that depends heavily on how well the draft model matches the target distribution. Having the reference weights lets teams benchmark their own drafts against a strong baseline instead of guessing.
Comparison with Competitors
Against mainstream speculative-decoding schemes like DFlash, AngelSpec's DFly leads by more than 10% on throughput, a gap that is not small when multiplied across millions of requests. Compared with closed-source commercial inference-acceleration services, the biggest advantage of an open framework is transparency and control: latency, memory footprint, and draft acceptance rate are all laid bare, so teams can tune parameters as needed rather than being led by black-box pricing. There is also a compounding effect: because the code is open, the community can submit improvements, port it to other base models, and harden it for production, none of which is possible with a hosted-only service that hides its internals.
Industry Impact and Use Cases
Once large models enter the deployment phase, inference cost often dominates total spending. Every bit of efficiency that techniques like speculative decoding gain directly dilutes the per-token price. For enterprise-grade dialogue and agent scenarios that demand long context and high concurrency, even a millisecond-level latency drop translates into meaningful cost savings. Agent-building teams run multiple generations in every reasoning step, so lowering per-step latency with speculative decoding makes the whole experience smoother. By open-sourcing both the training code and the weights, Tencent effectively hands the community a reproducible acceleration path, which carries real significance for filling the gap in domestic open-source ecosystems on the inference side. The practical upshot is that smaller teams, who cannot afford proprietary acceleration stacks, now have a credible way to serve a capable model at acceptable cost, which nudges the market toward openness over lock-in.
From a practical standpoint, the hard part of speculative decoding was never the idea but the engineering: how to train the draft model, how to raise acceptance, and how to align it with the base model are all grunt work. By open-sourcing that whole engineering chain, AngelSpec cuts the barrier down sharply. For the domestic large-model ecosystem, the inference side has long been the weak leg, and reusable acceleration frameworks like this one repair the foundation. If the community builds more draft models and benchmarks on top of it, the teams that want to self-host capable models are the ones who win.
Open source also changes who gets to improve the system. When the draft models and training code are public, a university lab or a startup can contribute a better acceptance strategy, and everyone benefits on the next update. That communal compounding is precisely what closed acceleration services cannot offer, and it is why frameworks like AngelSpec tend to age better than black boxes.