先求清晰，再谈可视化

This article demonstrates how to evaluate agent skills using the open-source evaluation framework Inspect AI together with Harbor, and how to complete visualization and analysis with the help of Google Sheets and Data Studio. The author argues that when building an evaluation system, teams should pursue clarity first and visualization later, so that they are not led astray by flashy charts that hide weak measurement underneath the surface of a polished report.

Core Highlights

The central argument of the piece is that evaluation design should give priority to clear metric definitions and interpretable samples, with visualization treated merely as a second-step presentation tool. Many teams rush to build dashboards and end up masking the actual flaws in their evaluation itself, ultimately wrapping unreliable conclusions in pretty graphics. The author warns that a beautiful chart is not evidence of a sound measurement, and that presentation should never substitute for rigor, because a visualization can only communicate what the underlying data actually contains. Before any chart is drawn, the team must be able to explain exactly what a passing score means and why a given sample belongs in the set.

Clarity also makes evaluations reusable: when a metric is well defined, a later reader can reason about failures instead of guessing. This is why the author keeps the measurement logic in plain code and plain language, so the evaluation can be challenged and improved rather than admired from a distance as a finished artifact.

What Happened

The author uses Inspect AI to define evaluation tasks and scoring logic, and uses Harbor to manage the agent's runtime environment, ensuring that every evaluation is reproduced under consistent conditions rather than drifting between machines. After the evaluation produces raw results, the data is imported into Google Sheets for cleanup and organization, and finally Data Studio is used to generate shareable reports that let non-technical colleagues understand the conclusions without touching the underlying code or rerunning the pipeline themselves. This end-to-end flow keeps the path from raw run to stakeholder view short and auditable.

Reproducibility is the quiet hero here. Because Harbor pins the environment, a result from last month can be compared with a result today without wondering whether the difference came from the model or from the machine. That stability is what turns an evaluation from a one-off demo into a habit a team can trust week after week.

Technical Details

Inspect AI provides abstractions for scorers and datasets, while Harbor handles containerized isolation; combined, the two reduce noise caused by environment drift across runs and make failures easier to reproduce. Google Sheets plays the role of lightweight data cleaning, and Data Studio handles interactive charts that can be filtered by dimension. This division of labor keeps each tool within its strength instead of forcing one platform to do everything at once, which is where many heavyweight tools become awkward and expensive to maintain over time.

Compared to Competitors

Compared with jumping directly onto a heavyweight evaluation platform, this open-source combination is lighter and more controllable, making it suitable for small and mid-sized teams to validate ideas quickly without a long procurement cycle. The trade-off is that it lacks out-of-the-box collaboration documentation and enterprise-level permissions, which larger organizations often expect from a managed solution before they will adopt it broadly across many teams and departments.

Industry Impact

For teams building agent products, the article offers a low-cost, auditable evaluation methodology that stresses thinking clearly about what to measure before deciding how to draw it. In a field where benchmark gaming is common, that discipline is exactly what separates useful signal from decorative noise, and it gives teams a foundation they can defend when results are questioned by skeptical stakeholders or reviewers who want to see the raw numbers behind a headline claim.

The method also ages well: as the agent improves, the same evaluation can be rerun to prove progress, turning evaluation from a one-time gate into an ongoing feedback loop that keeps the team honest about real gains rather than inflated demo numbers. For startups that cannot afford a dedicated evaluation engineer, a spreadsheet-and-notebook workflow is often the most realistic way to start, and the article makes that start concrete instead of abstract. The lesson is not that dashboards are bad, only that they should summarize an evaluation the team already trusts, not replace the thinking that earned that trust.