Meituan LongCat Releases MineExplorer, First Minute-Level Long-Horizon Benchmark for Minecraft

Key Highlights

Meituan's LongCat team released MineExplorer, the first benchmark for "minute-level long-horizon tasks" in the open world of Minecraft, containing 813 human-validated instances. Its significance is not merely adding another leaderboard, but quantifying—for the first time, in a reproducible way—the real agent capability of "multi-step, long-duration, autonomous navigation." Put simply, it punctures an illusion: having strong dialogue ability does not equal being able to finish a job in an open world that requires several steps. The benchmark is valuable precisely because it refuses to reward clever one-liners; it demands that a model sustain attention, memory, and locomotion across minutes of game time, which is a much closer proxy for real agent deployments than a tidy Q&A set. By choosing Minecraft, the team also gains an environment that is cheap to run at scale and familiar to builders, so the benchmark can keep growing without the prohibitive cost of physical robots or licensed simulators.

Capabilities and What Happened

MineExplorer grades tasks by "hops": a 1-hop task is the shortest path, while a 4-hop task requires the model to repeatedly plan and move through complex terrain. The team used this benchmark to evaluate 18 top multimodal models, with stark results: the strongest, Claude Opus 4.6, reached only 41% overall success; more dramatically, success rate plunges from 77% on 1-hop tasks to 12% on 4-hop tasks. In other words, every few extra steps the model must take, its reliability drops off a cliff—long-horizon planning remains a weak spot of current multimodal models. The hop-based grading is the key insight: it isolates the variable that actually breaks models—chain length—so the community can finally see, in a single number, how fast competence decays as a task grows from a single action to a small project. The hop grading also doubles as a diagnostic: a model that scores well at one hop but collapses at four tells engineers precisely where to invest, in planning or in perception, instead of forcing them to guess from a single blended number.

Technical Details

Breaking down the causes of failure, nearly 60% stem from navigation failure: the model knows "what to do" but often gets stuck on "how to get there," such as losing its way, misjudging distance, or looping. This exposes a blind spot in current evaluation paradigms: many leaderboards only test single-turn Q&A or short-horizon actions, while real agents must decide in continuous space and time. Using Minecraft as the carrier, MineExplorer provides a low-cost, resettable, massively reproducible long-horizon environment where navigation and planning ability can finally be measured seriously. The choice of Minecraft is no accident: it is open-ended, scriptable, and already familiar to millions, so building 813 validated instances is feasible at scale while keeping the physics and stakes consistent across runs, which is exactly what a rigorous benchmark requires. The dominance of navigation failures also reframes the research agenda: more parameters may not help if the model cannot keep itself oriented, so embodied memory and mapping deserve as much attention as raw reasoning scale.

Comparison With Competitors

Compared with pure-text or single-turn image-understanding benchmarks, MineExplorer pulls evaluation back to the essence of "embodied decision-making." It does not compete with traditional academic leaderboards for rankings, but fills the gap of long-horizon autonomous execution. For agent-building teams, it is more informative than general chat evaluations, because real business tasks are almost always multi-step and cross-spacetime rather than one question, one answer. Where a chatbot benchmark might celebrate a correct fact, MineExplorer asks whether the system can still be trusted after three detours and a wrong turn, and that is the question procurement teams actually care about before letting an agent touch a live workflow. Unlike chat benchmarks that flatter models with clean inputs, MineExplorer rewards the unglamorous ability to persist, which is closer to what a deployed assistant actually faces on a bad day.

Industry Impact and Use Cases

To put it bluntly, once MineExplorer is open-sourced it becomes a "mirror" for agent R&D: which models can actually work and which only chat becomes instantly visible. It applies to robotics simulation, game AI, and automated-assistant directions that need long-horizon planning, and also provides hard metrics for enterprise model selection. As long-horizon tasks become the key threshold for agent deployment, the value of such benchmarks will keep growing. The open release also means the community can extend the 813 instances into new terrains and new failure modes, turning a single snapshot into a living stress test that keeps pace with how fast agent capabilities—and agent failures—evolve. For buyers, a 41% top score is a sobering but useful reality check that tempers inflated expectations and focuses pilots on tasks with short hop counts where today's models are already dependable. In that sense the benchmark is less a trophy and more a map of where not to deploy agents yet.