国内首个高考志愿AI测评出炉，千问多项表现超过资深咨询师

Can AI Do College Application Counseling Better Than Human Experts?

College application counseling (Gaokao voluntary filling) is probably one of the most anxiety-inducing decisions for Chinese families. One wrong choice can affect a child's next four years or even longer. So most parents' choice is: spend a few thousand yuan on a "senior application counselor."

But AI is now saying: I might be more accurate than these experts.

Yousong Lab (a lab under Alibaba's DAMO Academy) recently released China's "first AI capability evaluation report for college application counseling"—the test subject was a "Qwen Gaokao Application Agent" they built in-house, and the control group was 53 human counselors with an average of 4.6 years of experience.

The results are interesting.

Where Does AI Beat Humans?

The evaluation designed four modules. Let me highlight a few key results.

**Factual questions: 44 out of 44 correct.** Gaokao application involves massive amounts of factual information—a school's admission score for a particular year, specific requirements for a major, deadline for a particular application batch... Humans have to rely on memory or look things up, but AI can retrieve precisely, without memory errors or omissions. 44 factual questions, Qwen Agent got them all right; human counselors had mixed results.

**Simulated application: 6 out of 10 choices were admissible.** The test worked like this: give the Agent and human counselors the same student profile (Gaokao score, intended major, location preference, etc.), let them each produce an application plan, then simulate admission to see acceptance probability. In Qwen's plan, 6 out of 10 choices were admissible—roughly the same ratio as human counselors, but the Agent's plan was more consistent (didn't exhibit the occasional "overly conservative" or "overly aggressive" problems human counselors can have).

**Blind comparison: 100 rounds, experts preferred Qwen 58 times.** This test was the most interesting: mix AI's answers and human counselors' answers together, hide identities, and have 53 experts judge "which answer is better." After 100 blind rounds, experts chose Qwen's answer 58 times.

In other words: without knowing who wrote what, human experts tended to prefer the AI's answers.

AI-Assisted Humans: Higher Accuracy, 27% Less Time

One practical finding: what happens when human counselors use Qwen Agent as assistance?

Results show: with AI assistance, human counselors' accuracy improved, and average time spent decreased by about 27%.

This combination is actually the ideal state—AI handles data lookup, probability calculation, and draft generation; human counselors make final judgments, communicate with students and parents, and handle the "human touch" aspects that AI isn't great at yet.

How Was This Agent Built?

Technically, the Qwen Gaokao Application Agent is based on two things:

1. **Qwen Gaokao Application LLM**: a model specifically trained for the application scenario, understanding the "rules of the game" for Gaokao applications

2. **Quark's 8 years of Gaokao data**: Quark is Alibaba's search product that has been doing Gaokao application information services for 8 years, accumulating data on approximately 3,000 universities and 2,000+ majors

With these two foundations, the Agent can basically cover the core needs of application counseling: looking up schools, looking up majors, calculating admission probability, and giving application recommendations.

What Are the Limitations?

Of course, this evaluation has some caveats.

First, this evaluation was conducted by Yousong Lab (Alibaba-affiliated). Although the testing process looks fairly rigorous (blind testing, control groups, quantitative metrics), "testing your own product" always raises some questions.

Second, application counseling isn't just a "data calculation" problem. A truly good application plan needs to consider the student's personality, interests, family situation, future plans... these "unstructured" dimensions are still challenging for current AI. A big part of human counselors' value also lies in these "soft" dimensions.

Third, this Agent doesn't seem to be widely available to the public yet (or at least the C-end product form wasn't clearly mentioned). Evaluation is one thing; whether it can actually be used depends on subsequent productization progress.

My Take

I think the direction of this evaluation is correct—AI does have advantages over humans in information processing, data calculation, and consistency.

But for the application scenario, the final decision still needs to be made by a human. The most suitable role for AI is as an "assistant tool"—helping counselors and parents save time, providing data support, and avoiding basic errors.

If this Agent gets opened up as a "Gaokao application AI assistant everyone can use," that would definitely be a good thing for millions of examinee families—at least people don't have to spend thousands of yuan on counselors anymore. Let AI calculate a draft plan first; then ask a human if there are further questions.

The Gaokao application counseling space might get quite lively this year.