SGLang 披露并修复 PD 分离服务中的跨请求 KVCache 竞态问题
Key Highlights
The SGLang team, coordinating with Cloudflare, disclosed a cross-request KVCache exposure affecting GLM-5.3 on Workers AI and has fixed it, demonstrating a responsible disclosure process. Low-level inference vulnerabilities often deserve more engineering attention than model capabilities, because they strike at the security floor of multi-tenant isolation.
What Happened
The issue occurred between August 27 and September 9, 2026, impacting inference deployments using PD disaggregation (Prefill-Decode separation). Under certain conditions, one request could read another's residual KVCache, creating potential information leakage. Cloudflare and SGLang coordinated the fix and notice, and assessed the actual exposure scope during the affected window before going public.
Technical Details
PD disaggregation splits prefill and decode across instances for throughput. If cache isolation is weak, cross-request KVCache reuse can "cross-contaminate." SGLang fixed the race by strictly isolating cache per request and adding checks. The episode is a reminder: the prerequisite for any performance optimization is correct isolation—any cost-saving reuse must be bounded by safety, not assumed to be safe.
Versus Competitors
Versus monolithic inference, PD disaggregation is faster but has a finer attack surface; versus closed-source options, SGLang as an open engine chose to publish details, helping the whole community learn. Transparent disclosure is itself a kind of security competitiveness that builds user trust in the platform and sets a norm other vendors are pressured to match.
Industry Impact
For cloud platforms and enterprises deploying inference, this is a key reminder: performance tricks like KVCache reuse must be gated by isolation. It also informs security baselines for multi-tenant inference, especially in data-sensitive sectors like finance and healthcare where a single cross-tenant leak is catastrophic.
Background and Wider Context
As inference moves to shared, disaggregated infrastructure to cut cost, isolation bugs become the new frontier of model security. The bug class will recur wherever prefill-decode separation is adopted without rigorous per-request boundaries, which makes this particular disclosure a durable reference for the whole industry rather than a one-off incident.
Practical Takeaways
Teams running their own inference should put "cross-request cache isolation" on the pre-launch security checklist and periodically re-test for side channels, because a fix today does not guarantee a fix after the next refactor. For external services, retain audit logs for traceability, and treat isolation as a property you verify with tests, not a property you assume from the architecture diagram.
Deeper Context and Practical Notes
For platform and infrastructure engineers, the SGLang disclosure is a reminder that inference optimizations and isolation guarantees are the same battle, not separate concerns. PD-disaggregation buys throughput, but the moment a cache is shared across requests without strict boundaries, a performance feature becomes a confidentiality bug, and the blast radius is every tenant on the box.
The actionable lesson is to treat multi-tenant isolation as a security property that must be tested, not assumed. Any reuse of KVCache, prefixes, or intermediate state should be covered by explicit per-request scoping and a verification path, especially in shared serving environments like Workers AI. A clean disclosure process, coordinate, patch, notify, and bound the exposure window, is what turns an incident into a trust-building moment.
A subtle risk is that latency-sensitive teams may be tempted to relax isolation for speed. The SGLang case shows the cost of that temptation: a multi-week exposure window on a production model. Performance tuning should be gated behind isolation checks, so a faster path can never silently become an insecure one.
For the ecosystem, the episode reinforces that open inference stacks earn trust through transparency. Because SGLang is inspectable, the race was found, fixed, and explained in the open, which is harder to guarantee with closed systems. Teams choosing a serving framework should weigh auditability as a first-class criterion, not a nice-to-have.
Bottom Line
For platform engineers, the disclosure is a reminder that inference optimizations and isolation guarantees are the same battle, not separate concerns: PD-disaggregation buys throughput, but a cache shared across requests without strict boundaries turns a performance feature into a confidentiality bug. Treat multi-tenant isolation as a security property that must be tested, with explicit per-request scoping and a verification path, especially in shared serving environments. A subtle risk is that latency-sensitive teams relax isolation for speed, exactly the temptation this incident exposed. For the ecosystem, the episode reinforces that open inference stacks earn trust through transparency, because an inspectable system can be found, fixed, and explained in the open, a guarantee harder to secure with closed systems, so auditability should weigh as a first-class criterion.