By Keshavan Thiagaraja
Private AI Model Developer & AI Security Consultant Helping SMEs and Enterprises Build Secure AI Solutions Sydney, Australia.
B.Sc. (Computer Science, Mathematics and Statistics),
M.Sc. (Information Technology)
E-Commerce Professional
Introduction: If AI Agents Are Ready for Expert Work, Why Did the Best One Score 12 Percent?
A new benchmark built from real, paid expert jobs has just graded twenty frontier AI models, and the best of them scored 12 percent.
Can an AI agent do the work your business pays experts to do? What would an honest scoreboard for that claim look like? And what should you ask a vendor before you sign?
As a private AI model developer and AI security consultant working with SMEs and enterprises in Sydney, I am asked these questions every week. The answers matter to budgets, staffing and risk.
This article explains what the GDPx benchmark measured, what its numbers do and do not establish, and the practical lessons for any leader evaluating AI agents this quarter.
1. What GDPx Actually Measured
This week the research firm SemiAnalysis published GDPx, a benchmark of 206 tasks across 11 domains chosen because people earn a living doing them: chip design, cybersecurity, legal work, quantitative trading, biology, software engineering and more.
Fourteen industry partners supplied the tasks, and twenty frontier models were graded. The benchmark is notable less for its difficulty than for its source material: jobs people are paid to do.
2. The Scoreboard: Twenty Models, One Low Ceiling
The leader, Claude Opus 5.5, managed 12 percent. Gemini 4 Carbon scored 10.9. GPT-6 Astra scored 9.6.
However, a scoreboard is a starting point, not a verdict. These numbers measure one firm’s tasks under one firm’s rubric, and they describe today’s models, not next year’s.
3. The Limits: What GDPx Does Not Establish
GDPx is imperfect. It uses one firm’s tasks and one firm’s rubric, there is no independent reproduction yet, and the firm sells research to the industry it grades.
The lesson is to hold the number loosely. But the direction is right, because it measures models on work that pays, not trivia that impresses.
4. The Cost Column: $51 Versus $16
The leader averaged about $51 per completed task. The runner-up averaged about $16 for a score within a point of it.
If your agentic strategy assumes capability and cost move together, this table says they do not. The cheapest path to a result is now as strategic as the result itself, and that analysis belongs in procurement, not on a marketing page.
5. Reading Agent Pitches Against a Public Number
The demos you will see this quarter all claim readiness for expert work. A public, named benchmark just put the ceiling at 12 percent.
That does not make agents useless. It means the honest deployment zone today is narrow, well-instrumented tasks with human review, not the autonomous departments being sold on stage.
Lessons for Leaders Everywhere
First, the demo era is over. When an industry starts arguing about scoreboards, demos have stopped settling arguments.
Second, cost and capability do not move together. Put cost per completed task into procurement, next to the score.
Third, read every agent pitch against a public number. The honest deployment zone today is narrow, well-instrumented and human-reviewed.
Finally, ask every AI vendor one question this quarter: show me your score on an independent, task-based benchmark for work like mine, and your cost per completed task. If they cannot answer, you have learned something worth more than the demo.
Conclusion: Measurement Has Replaced Marketing
A 12 percent ceiling is not a reason to walk away from AI agents. It is a reason to buy them with evidence instead of enthusiasm.
Artificial intelligence will earn its place on measurable results, not on stagecraft. In the end, the mathematics of cost per completed task, honestly counted, decides where this technology genuinely belongs.
References and Further Reading
SemiAnalysis, “GDPx: The General Agentic Benchmark” – https://gdpx.semianalysis.com
SemiAnalysis, research notes and industry analysis – https://semianalysis.com
Think mathematically. Engineer intelligently. Lead strategically. Build responsibly. Scale globally. Create real business value through artificial intelligence.
#ArtificialIntelligence #AIStrategy #AIGovernance #EnterpriseAI #ITImpress
