如何为智能体选择引擎、深度与模型

Core Highlights

OpenRouter has published a live leaderboard that systematically evaluates four classes of configuration combinations, namely model, search engine, search method, and budget, on agentic tasks, giving developers data to choose the best mix instead of guessing from habit or hype. The board updates in real time, so the guidance it offers reflects current models and backends rather than a snapshot that goes stale within a month. For teams building retrieval agents, this is rare empirical grounding in a space where most decisions are still made by intuition and vendor marketing.

Specific Capabilities and What Happened

The leaderboard data shows that raising the search budget from one round to twenty-five rounds nearly doubles the score on retrieval-heavy tasks such as BrowseComp, while the cost only increases by a factor of two point five to seven, a fairly attractive trade for the gain in accuracy. Another key finding is that model choice matters more than search engine choice, with an average score gap of about fifteen points between models but only about ten between search engines. For tasks that already fail often, the report suggests lowering search depth to control cost and avoid wasted compute that will not pay off, because throwing more rounds at a hard problem rarely fixes a weak underlying model.

Technical Details

BrowseComp is a benchmark that tests an agent's ability to retrieve across huge numbers of web pages and reason over them, where a higher score means more precise retrieval rather than mere volume of clicks. The budget, or number of rounds, decides how many times an agent may search again, while depth controls how far each search expands before the agent commits to an answer. OpenRouter aggregates many models and search backends behind one interface, letting these variables be compared in quantified terms rather than by anecdote, which is what makes the leaderboard useful for planning a budget and for explaining overruns to stakeholders who fund the compute.

Comparison with Competitors

Compared with picking models and search tools by intuition, this leaderboard breaks the three dimensions of engine, depth, and model into separate measurements so users can trade off by need instead of buying the most expensive option by default. Many platforms show only a single composite score, whereas OpenRouter provides an explainable breakdown that shows where the gains actually come from and prevents teams from overpaying for the wrong lever. The transparency also lets smaller teams compete, because they can see exactly which cheap switch yields the biggest jump in their particular task class.

Industry Impact and Use Cases

For teams building retrieval agents, this data can directly guide budget allocation: upgrade the model first, then consider adding search rounds once the model ceiling is reached and more breadth is truly needed. Put simply, spend money where it counts most, by choosing the right model before deciding how many rounds to let the agent search, because the model is the bigger lever on final quality than the search backend behind it. The practical takeaway is to tune depth down on hopeless tasks and invest the savings in a stronger base model. Teams that blindly max out search rounds often discover they paid a large multiple for a tiny gain that a better model would have delivered for free. The leaderboard therefore doubles as a cost-optimization guide, not just a leader ranking, which is what makes it unusually useful for production planning. Because the data is live, teams can revisit it whenever a new model launches and rebalance their spend instead of freezing a choice that ages quickly. The honest lesson is that there is no single best configuration, only the best one for a given task, budget, and failure rate. Reading the breakdown beats copying a competitor's stack, because the winning mix depends on what you are actually trying to retrieve. OpenRouter's aggregation layer is what makes this comparison possible, since few teams can benchmark that many backends on their own. The result is a rare public good of comparable, reproducible numbers in a space crowded with unverifiable claims. A team that internalizes this table will spend less and ship more capable agents than one chasing the loudest model name. The discipline of measuring before scaling is what separates production systems from demos that look good in a screenshot.