AI AI Toolkit
AI Newsai-products

TTFT 降 16.9%、429 错误减少 67%

OpenRouter:Announcements(RSS)2026-10-28T00:00:00.000Z

Key Highlights

On October 28 OpenRouter announced the biggest improvement to its default routing algorithm since the company was founded. The old default rewarded essentially one thing, low price. The new router rewards the inference that is actually best for you, cheap, fast, high-throughput, and reliable. About 77% of requests hit models served by multiple providers, so how routing picks a provider directly defines the experience for most of the platform's traffic.

What Happened

The old algorithm allocated traffic almost entirely by input-token price. Cheaper providers won larger shares, and failed requests routed to the next cheapest. That delivered the ecosystem's lowest prices but produced three side effects. Cheap-but-bad providers could still win. Rankings were static and blind to real-time performance. Providers could only win by cutting price, weakening reliability investment, and it spawned pricing games of low input plus expensive cache and output tokens.

Technical Details

The new algorithm continuously monitors every provider and computes a real-time Routing Cost covering actual dollars spent, time waited, and retry cost on failure or unusable responses. It measures three layers. Effective price is derived from true cache-hit and input-output mix, closing the low-price loophole. Throughput is end-to-end output tokens per second, which can justify a higher price. TTFT, time to first token, is critical for agentic multi-tool calls. Reliability and quality are also scored, with bad responses heavily penalized. Traffic is shared within a competitive edge with three guardrails: a floor so each provider keeps a trickle to detect improvement, a back-off that reduces flow on failure, and a cap that auto-spills by learned capacity.

Versus Competitors

Against most inference gateways that still route by price or manual sort, OpenRouter turns the true cost of a good answer into an automatic scheduling signal without users changing a line of code. Those who still want pure price routing can keep it through provider sort equals price or floor settings. The move also pressures competitors to stop advertising headline prices and start advertising realized latency and reliability.

Industry Impact and Use Cases

For developers and agent apps calling hundreds of models through OpenRouter, the new router means a faster, steadier, cheaper default experience. A one-week split of 24 million requests and 103 billion input tokens showed TTFT p50 down 16.9%, throughput p50 up 16.3%, cost per request up only 6.0%, and 429 error rate down 46%. This is a paradigm shift from market bidding to quality bidding.

Practical Notes

If you run agents that fire many sequential tool calls, TTFT matters more than headline price, because every extra hundred milliseconds compounds across a long chain. The new router rewards providers that keep TTFT low, so you may see slightly higher per-token cost in exchange for materially fewer timeouts and retries, which usually nets out positive.

What to Watch

Watch your own latency and retry metrics after the rollout. If a workload is extremely price-sensitive and latency-tolerant, you can still pin a specific provider. The bigger story is that routing intelligence is moving from a static price table to a live cost-of-a-good-answer optimizer, and that is where the next wave of gateway competition will play out.

Takeaway

OpenRouter just turned default routing from a race to the bottom on price into a quality-aware scheduler, and the measured gains suggest the rest of the inference-gateway market will have to follow or lose developers to a better default.

Background

Routing has quietly been one of the hardest problems in the inference economy, and it is the layer most users never see. With hundreds of models and dozens of providers, the component that decides who serves your request is where latency, cost, and reliability are actually won or lost, yet most people only notice it when something goes wrong. OpenRouter's change matters because it moves the default from a static price table to a live optimizer that learns from every request.

Where to Try

You can observe the effect without any code change, because the new router is the default. If you want to experiment, set provider sort to price on a test key and compare TTFT and throughput against the quality default over a few days of real traffic before committing, and you will likely see the quality default win on retries even if it loses on headline price.

Closing Thoughts

The deeper signal is that the inference gateway is becoming an optimizer, not a directory, and the winners will be the layers that quietly make every request cheaper and faster without the developer lifting a finger.