Tencent Hunyuan Open-Sources High-Performance Attention, Router GEMM, and MoE Operators
Core Highlights
Tencent's Hunyuan team has recently open-sourced its self-developed high-performance operator library, HPC-Ops, and successfully merged it into the main branch of SGLang, a widely used open-source large-model inference framework. The most eye-catching part of this release is a pair of carefully tuned operators: Dynamic Attention and Fused MoE. In benchmark tests running on Tencent's own Hy3 model, these operators reduced TPOT, which stands for time per output token, by as much as 48.8 percent. In practical terms, that nearly halves the latency a user feels when talking to a model in production. For any service where responsiveness shapes the experience, an improvement of this size is a genuine breakthrough rather than a marginal tweak, and it is exactly the kind of low-level win that compounding infrastructure gains are made of. Such gains matter most in real-time dialogue and search scenarios, where every saved millisecond translates directly into a smoother experience.
What Happened and Key Capabilities
HPC-Ops is not a pile of scattered experimental scripts. It is a coherent toolkit built for real large-scale deployment. Beyond the two headline operators, the library also ships companion high-performance operators such as Router GEMM, which together cover the most common bottlenecks found in mainstream mixture-of-experts architectures, namely the routing computation and the attention computation. Because the project is released as open source and has been accepted into the SGLang mainline, developers can pull the code today and reproduce the performance gains inside their own deployments without waiting for a separate integration or maintaining a private fork, which keeps long-term maintenance cost low and lets the community contribute and co-optimize on top of it.
Technical Details
Dynamic Attention is optimized specifically for variable-length sequences, which is the normal situation in real conversations, and it trims away computation that would otherwise be wasted on padding and irrelevant positions, so that compute in long-context scenarios is spent where it matters. Fused MoE takes expert routing and the subsequent matrix multiply and combines them into a single fused kernel, which sharply reduces the overhead of shuttling data back and forth across GPU memory. Router GEMM further accelerates the multiply-accumulate step between the gating weights and the input tokens, avoiding the extra cost of multiple kernel launches. When all three are used together on a large mixture-of-experts model such as Hy3, the end-to-end time required to produce each token drops substantially, and the advantage tends to grow as batch size and sequence length increase, making it especially attractive under heavy serving loads.
Comparison with Competing Approaches
Compared with the standalone attention kernels or naive MoE implementations commonly seen in the open-source community, the advantage of HPC-Ops is that it optimizes at the operator level rather than the framework level, and it targets TPOT directly, the number that most shapes perceived speed. Because it has been merged into SGLang's main branch, adopting it is lighter weight than swapping in an entirely new framework or simply buying more hardware, and it slots naturally into stacks that already depend on SGLang for serving, requiring only minimal operational changes. For teams already on SGLang, this is close to a zero-cost upgrade.
Industry Impact and Applicable Scenarios
Put simply, this set of operators lets small and mid-sized teams deliver high-throughput, production-grade inference without a heavy infrastructure bill, and it is especially valuable in latency-sensitive settings such as customer-support bots, real-time assistants, and interactive copilots. By open-sourcing the work, Tencent also helps the broader domestic large-model ecosystem close long-standing gaps in low-level operator performance, an area where proprietary stacks have often held the lead, and that kind of contribution raises the baseline for everyone building on open infrastructure while likely spawning more downstream innovation built on operator-level optimization.
Looking beyond the immediate numbers, the release also signals a broader maturing of China's open-source AI infrastructure story. By contributing battle-tested operators to a framework that many teams already run, Tencent is effectively raising the quality bar for everyone, not just its own products. The fact that the gains are measurable on a real production model rather than a synthetic benchmark gives the work credibility, and it invites other model vendors to benchmark their own stacks against the same operators. Over time, shared low-level kernels like these tend to become the invisible plumbing on which a whole ecosystem runs, much as optimized linear-algebra libraries did for traditional scientific computing.
Broader Significance
Simply put, by donating battle-tested kernels to the SGLang mainline, Tencent is trading short-term advantage for long-term ecosystem influence. Smaller teams that cannot afford custom kernel engineering now get near-state-of-the-art throughput almost for free, which lowers the barrier to running large models in production. As more deployments adopt these operators, optimizations compound across the community, and the resulting telemetry feeds further refinement. That flywheel is exactly how open infrastructure wins, turning a single vendor's internal win into shared progress for everyone building on top of it.