AI Coding 的实践与探索

ByteDance's AI Coding Reality Check: 6x Code Contribution, But the Story Is More Nuanced

At the recent Volcano Engine FORCE conference, Hong Dingkun, ByteDance's VP of Technology, shared some eye-popping numbers: over the past year, ByteDance's internal AI code contribution rate increased 6x, and Token consumption grew 5x.

But here's the interesting plot twist: the TRAE team (ByteDance's AI coding tool team) has over 90% of its code generated by AI, yet per-capita demand throughput only improved by 60%.

That 60% figure is worth pondering. 90% AI-generated code rate, but only a 60% boost in human productivity. What this tells us: AI can write code fast, but there's still a sizable gap between "code that runs" and "code that's actually shippable."

80% Accuracy, But Only 40-60 Deliverability

ByteDance ran 900 experiments and uncovered a sobering fact: mainstream Coding model combinations can achieve over 80% code accuracy, but deliverability (meaning: can this code actually go into production?) scores only 40-60 out of 100.

What is deliverability? It's not just about code that runs. You also need error handling, edge cases, code style compliance, maintainability... AI currently struggles with these dimensions.

ByteDance's solution is "infrastructure"—deep integration of Harness (their CI/CD and engineering productivity platform) with their AI Coding tools. Does it work? After combining with Harness infrastructure, deliverability improved to 80/100.

This conclusion is actually quite important: AI Coding isn't something you fix just by stacking model capabilities. Engineering infrastructure, code governance, collaboration standards—these "unglamorous" parts are what actually drive real productivity gains.

Lowering the Barrier, But New Problems Emerge

That AI lowers the programming barrier is now conventional wisdom. A product manager who's never written much code can now ask AI to build a simple data dashboard. A new hire can use AI to quickly understand a complex project's code structure.

But after the barrier drops, new problems follow:

**Code governance**: when everyone uses AI to generate code, inconsistent style, messy dependency management, and silently introduced security vulnerabilities get amplified in large teams like ByteDance.

**Collaboration models**: when AI writes most of the code, humans shift into "reviewer" and "orchestrator" roles—this requires teams to redefine their collaboration workflows.

**Metrics design**: if you keep using traditional metrics like "lines of code" or "number of commits" to measure programmer output, everything becomes distorted in the AI era. ByteDance is also exploring new metric systems.

Prototype-Driven Development: A New Playbook for the AI Era

ByteDance is experimenting with a new model called "prototype-driven development." The idea: use AI to quickly generate an interactive prototype (essentially a working Demo), show it to PMs, designers, even users, gather feedback, then let that feedback guide formal development.

The power of this model: it front-loads the "requirements understanding" phase—traditionally the most time-consuming part of development. In conventional workflows, you often don't discover "oh wait, the PM wanted something different" until the code is nearly done. Prototype-driven development minimizes that trial-and-error cost.

These capabilities have been consolidated into TRAE (ByteDance's AI coding tool). TRAE currently consumes 5.6 trillion Tokens per day, a 50x increase over the past year.

TRAE Work: Getting AI Into the Actual Workflow

Beyond TRAE's code generation capabilities, ByteDance is also pushing TRAE Work—a tool that gets AI into daily development workflows. What can it actually do?

Automatically generate unit tests
Automatically perform code reviews
Automatically write documentation
Automatically fix bugs
Participate in requirements analysis and architecture design

These features don't sound sexy, but they're precisely the most time-consuming, most easily overlooked parts of development work. Writing the code for a feature might take 2 hours, but writing tests, documentation, fixing bugs, and addressing Code Review feedback can easily take another 6 hours. TRAE Work is targeting that "invisible 6 hours."

What This Means for the Broader Industry

ByteDance's data provides a rare "inside view" for the entire industry. Most companies talking about AI Coding focus on "how powerful our model is" or "how high our code accuracy is." But ByteDance, using real production data, is telling everyone: high accuracy doesn't equal shippable code, and surging Token consumption doesn't equal surging productivity.

This insight matters. It means the AI Coding competition won't just be about model capabilities—it'll be about the full stack: "model + engineering infrastructure + collaboration standards + metrics system."

For small-to-medium teams considering adopting AI Coding tools, ByteDance's experience is worth studying: don't just fixate on code generation accuracy. Get your engineering infrastructure (CI/CD, code standards, testing frameworks) solid first, then integrate AI tools—you'll get much better results.