Five real office tasks, eight leading global agents, one surprising result.

On August 19, analysts at Wall Street investment bank Jefferies published an AI Agent field test report, pitting eight leading global AI Agents including Claude Cowork, OpenAI Codex, and Qwen Work against real office tasks. The outcome surprised many: Alibaba's Qwen Work took first place overall, the only product in the evaluation to score above 90 across every tested dimension.

None of the five tasks were benchmark-gaming gimmicks — they were actual tasks financial analysts perform daily: summarizing annual reports from multiple documents, researching and comparing company operating data online, operating a real desktop browser for information retrieval and document generation, creating English presentations from data, and generating marketing posters from reference images. These tasks have no single correct answer; they test an agent's end-to-end ability to solve problems in real work scenarios.

Harness Is the Second Half of Agent Competition

The report's most valuable insight isn't the ranking itself but the analytical framework it introduces: Jefferies splits Agent capability into two components — Model and Harness. The model is the underlying LLM's parameters and raw capability; the Harness is the entire engineering mechanism wrapped around it: prompt engineering, context management, tool invocation, boundary control, feedback correction, and governance policies.

For the past two years, Agent competition centered on models: who has more parameters, higher benchmark scores, longer context windows. But in real office scenarios, high model scores don't guarantee good task completion. You need the model to call the right tools at the right time, self-correct when errors occur, maintain context across long tasks, and request human confirmation before sensitive operations — none of which parameter count determines; that's Harness work.

By Jefferies' calculation, Qwen Work's "implied Harness score" ranked first in this test, ahead of overseas products like Claude Cowork and Codex. This means that even though the underlying model may not have the most parameters, Qwen Work translates model capability into task outcomes stably and repeatably through superior engineering packaging. Many Agent products use the strongest models but have rough Harness implementations — the result is inconsistent real-world performance, frequent tool-call errors, and forgotten goals mid-task. A 140-IQ model engineered to perform like a 90-IQ agent.

Cost per Task Will Become the Core Enterprise Selection Metric

Another critical dimension overlooked by many in the report is cost.

An Agent executing a task doesn't finish in one inference pass; it requires multiple rounds of thinking, persistent tool calls, and long-horizon operations, each consuming tokens. Dozens or hundreds of model calls per completed task are normal. This means: if the underlying model API is expensive, even successful task completion will make enterprises think twice. The Jefferies report notes that Qwen 3.8 Max, the model behind Qwen Work, is significantly cheaper than some leading overseas models, delivering clear cost-per-task advantages at comparable performance levels.

This is an important signal for China's AI industry. Many have assumed that if Chinese LLMs catch up to American models on parameters and benchmarks, that's enough. But the core battlefield of Agent competition is Harness — it's a contest of product engineering, of depth of understanding of real workflows, of ecosystem integration capability. Qwen Work has already integrated with DingTalk IM, letting employees invoke the agent directly in group chats for chat summarization, document creation, message and email processing, with future plans to connect to enterprise databases and business systems. That depth of ecosystem integration can't be achieved by model capability alone.

The report argues that as agents move from personal productivity to enterprise deployment, workflow and ecosystem collaboration become the core competitive moat. Users' historical tasks, work habits, connectors, Skills, and automation workflows gradually embed themselves in the product, creating high switching costs. When enterprises choose Agent platforms in the future, they won't just look at benchmark scores — they'll look at whether the agent can integrate with their systems, complete their work reliably, and how much it costs to complete each task — Cost per Task will replace MMLU scores as the core enterprise selection metric.

The second half of Agent competition has begun. Model capability will converge, but gaps in engineering packaging, ecosystem integration, and cost efficiency will widen. Qwen Work topping this Wall Street evaluation is a good start, but for Chinese AI products to truly establish themselves in the global enterprise market, continued deep investment in Harness and ecosystems will be essential.

See you tomorrow.