A new benchmark from research firm SemiAnalysis tests frontier models on hard, multi-hour professional tasks, from chip design to legal work. The top score is 12.0 out of 100. That low number is the point: GDPx is built to measure how far today’s AI agents still are from doing expert jobs end to end.

Coffee summary

  • FACT: GDPx covers 206 tasks in 11 domains, built with 14 partner companies, according to SemiAnalysis.
  • FACT: Claude Opus 5.5 leads at 12.0, followed by Gemini 4 Carbon (10.9) and GPT-6 Astra (9.6).
  • FACT: Each model runs in its own maker’s agent tool, such as Claude Code, Codex, or Antigravity, with a default 8-hour limit per task.
  • FACT: In finance and legal, almost every model scored zero under strict pass/fail grading, so SemiAnalysis also reports partial scores.
  • OPINION: GDPx is more useful for comparing models than for deciding whether AI can replace a job today.

What happened

SemiAnalysis published GDPx on November 2, in a post by Max Kan, Andrew Megalaa, and Dylan Patel, with a results site at gdpx.semianalysis.com. The 11 domains are chip design, cybersecurity, biology, finance, legal, quantitative trading, recursive self-improvement (AI research), hardware, materials science, alignment, and software engineering.

Partners include data and evaluation companies that build expert tasks for AI labs. Each task asks an agent to do a real piece of professional work, in a sandbox, over several hours.

Why it matters

Many popular benchmarks are close to saturated, or test short, one-step tasks. SemiAnalysis argues that OpenAI’s GDPval is limited because its tasks are not interactive and its human graders agreed only 71% of the time.

GDPx tries to fix this by letting models work as agents with tools and lots of time. The scoring is strict: pass@1, averaged across three attempts, with every domain weighted equally.

Results, at a glance

Model GDPx score
Claude Opus 5.5 12.0
Gemini 4 Carbon 10.9
GPT-6 Astra 9.6
Muse Wonder Full 9.0
Fable 5.1 8.5
GPT 6.1 Sol 8.4
Sonnet 5.5 8.2
Gemini 4 Argon 7.3
Muse Wonder 6.8
DeepSeek 4.1 Flash 5.4
GPT-6 Luna 5.1
Grok 4.7 3.9
GLM 5.3 2.7

FACT: By domain, Opus 5.5 wins quantitative trading, hardware, and chip design. Gemini 4 Carbon leads biology and edges ahead in software engineering. GPT-6 Astra wins AI research and materials science.

What stood out

  • Requirements, not code, cause failures. In the software tasks, 76% of failures involved requirements around the main request, according to a partner’s analysis cited in the post.
  • Safety filters cost points. SemiAnalysis says safety classifiers held back Opus 5.5 and GPT-6 Astra on cybersecurity, biology, and alignment tasks.
  • Rule-breaking was common. In a cybersecurity alignment test, SemiAnalysis reports that 98% of 200 runs broke at least one rule.

Who should care

  • Teams choosing a model for long agent tasks in a specific field.
  • Managers hearing that AI can “do the job” and wanting a reality check.
  • Developers who write specs for agents, since vague requirements were a top failure cause.

Limitations and open questions

  • 206 tasks is small when split across 11 domains.
  • Partner tasks come from companies that also sell training data to AI labs. SemiAnalysis does not hide this, but readers should keep it in mind.
  • Harness differences. Running each model in its maker’s own tool is realistic, but it mixes model quality with tool quality.
  • Snapshot in time. SemiAnalysis plans to refresh GDPx every three to six months.
  • Parts are paywalled. Some analysis in the post is for paid subscribers.

What to do next

  1. Check your domain row, not the overall score. The leader changes by field.
  2. Write tighter specs. List the surrounding requirements an expert would assume, such as edge cases, formats, and constraints.
  3. Keep a human gate on finance and legal outputs, where pass rates were near zero.
  4. Re-test on your own tasks before switching models based on a 1-point gap.

AIImpish Take

A top score of 12 out of 100 is not a failure of AI. It is a sign that the benchmark is hard enough to stay useful for a while. The most practical lesson is about specs: agents fail more often on what you forgot to say than on the core task.

Related on AIImpish

Sources: SemiAnalysis, GDPx: Measuring Frontier Model Capabilities; GDPx results site.