All-Flash Servers Deliver High-Performance Vector Search, Cutting Hardware Cost by Over 90%

Key Highlights

Xiaohongshu's engine architecture team proposed HELMSMAN at OSDI 2026, a high-performance vector approximate nearest neighbor (ANN) search system for all-flash servers. Its most astonishing number: about 40 all-flash servers now carry the load that previously required about 35,000 CPU cores and about 350 TB of DRAM, saving over 90% in hardware cost. Put simply, it proves that "using flash instead of memory for vector search" is not only feasible but disruptively cheap. The result is a wake-up call for anyone still sizing vector clusters around DRAM, because it shows the dominant cost assumption of the last decade of retrieval infrastructure may simply be wrong, and the savings are large enough to change build-versus-buy math for entire recommendation stacks. The 90% figure is not a marginal tweak but a step-change, and it arrives from a real production team rather than a benchmark vendor, which gives it outsized credibility with infrastructure buyers who distrust lab results.

Capabilities and What Happened

Vector search is the bedrock of recommendation, search, and RAG, yet traditional solutions rely heavily on large memory to hold massive vector indexes, making machines expensive and hard to scale. HELMSMAN's idea is to abandon the "memory-centric" approach and switch to "flash-centric": it scatters data across multiple all-flash servers with a clustered index, combined with a customized storage stack and hierarchical learned search pruning, hitting flash for access while preserving recall. The result is a cliff-edge drop in server count and total cost of ownership for the same business scale. The shift is conceptually simple but operationally radical: instead of buying ever-larger memory boxes, you buy many cheaper flash boxes and teach the system to tolerate flash's higher latency through smarter layout and pruning, which is exactly the kind of trade that turns a cost center into a competitive advantage. The flash-centric bet also future-proofs the design: as SSD capacity keeps climbing and prices keep falling faster than DRAM, the cost gap HELMSMAN exploits will likely widen rather than close, compounding the advantage over time.

Technical Details

Look at three blocks of design. First, clustered indexing partitions the vector space into clusters so queries only expand within relevant clusters, avoiding full scans. Second, a customized storage stack rewrites the IO path for the bandwidth and latency characteristics of NVMe flash, reducing software-stack overhead. Third, hierarchical learned search pruning uses a lightweight model to dynamically discard low-probability branches during retrieval, further cutting compute. Together, these let "flash carrying vectors" hold up on both latency and throughput. The genius is in the division of labor: clustering shrinks the search space, the storage stack extracts flash's real bandwidth, and the learned pruner throws away the rest, so no single trick has to be perfect for the whole system to win, which also makes each layer independently tunable as hardware improves. Each of the three techniques attacks a different bottleneck—search space, IO path, and compute waste—so the gains stack instead of interfering, a design discipline that is itself the lesson for other infrastructure teams.

Comparison With Competitors

Compared with pure-memory vector databases, HELMSMAN replaces expensive DRAM with larger, cheaper flash, giving an extremely prominent per-unit cost advantage; compared with general distributed solutions, its storage stack and pruning are purpose-built for ANN, achieving higher efficiency. For companies facing the same massive-vector and cost pressure, this is an engineering route worth learning from, not just metrics in a paper. The contrast with off-the-shelf vector DBs is stark: those optimize for generality and ease of use, while HELMSMAN optimizes for one thing—cost per query at Xiaohongshu's scale—and wins by refusing to pay for memory it does not need, a privilege that comes from owning the workload and the stack end to end. Against managed vector services that bill by memory, a flash-based approach flips the cost curve, and the open publication lets skeptical engineers verify the claim instead of accepting a vendor's marketing sheet.

Industry Impact and Use Cases

To put it bluntly, the cost wall of vector search has long been an invisible threshold for AI application deployment. Once the industry follows HELMSMAN's thinking, the infrastructure spend of recommendation systems, multimodal retrieval, and enterprise RAG can all be sharply compressed, letting small and mid-sized teams afford large-scale vector services. As a business-driven systems paper, it again shows that top-conference results need not come from labs but can emerge from the engineering pressure of real traffic. The broader lesson is that the next efficiency breakthroughs in AI infrastructure will likely come not from bigger models but from rethinking the boring layers—storage, indexing, scheduling—where most of the actual money is spent. If adopted broadly, the approach could redeploy the DRAM freed up into training or caching elsewhere, turning a storage saving into headroom for the rest of the AI stack and improving total-system economics, not just one line item.