Benchmarks
1,392 benchmarks with difficulty, saturation and the leading model. Search by name, topic or model.
1,392 benchmarks
| # | Benchmark | Topic | Difficulty | Saturation | Discr. | Leader | Top score | Models |
|---|---|---|---|---|---|---|---|---|
| 1 | Terminal-Bench 4.0 O Terminal-Bench 4.0: complex multi-step professional tasks in a command-line interface. | Agentic terminal tasks | 0.1% | 64.6% | 3.28 |  🇺🇸 Claude Sonnet 5.5 | 70.6% | 47 |
| 2 | FrontierBench O Successor to Terminal-Bench 2.1 (v0.1): 74 harder terminal tasks emphasizing science and engineering. | Agentic terminal tasks | 0.1% | 31.1% | 2.61 | 🇺🇸 Claude Opus 5 | 43.3% | 13 |
| 3 | MathArena Apex O Challenging math contest problems from MathArena Apex benchmark. | Challenging Math Contest problems | 0.2% | 87.7% | 3.51 | 🇺🇸 Claude Opus 5 | 91.4% | 51 |
| 4 | Complex Research using Integrated Thinking – Physics Test O CritPt (Complex Research using Integrated Thinking – Physics Test) benchmark. | Reasoning | 0.3% | 29.1% | 2.16 |  🇺🇸 GPT-5.6 Sol | 32.3% | 95 |
| 5 | ExploitGym O UC Berkeley-led benchmark: agents turn real known vulnerabilities into working exploits achieving arbitrary code execution. | Cybersecurity | 0.3% | 33.4% | 2.02 | 🇺🇸 GPT-6 Astra | 42.4% | 17 |
| 6 | SciRes FrontierMath Tier 4 O SciRes FrontierMath benchmark covering tier 4. | Math (frontier) | 0.5% | 86.4% | 2.83 | 🇺🇸 GPT-6 Astra | 97.6% | 21 |
| 7 | ARC-AGI-2 O ARC-AGI Phase 2 aggregate accuracy. | General reasoning | 0.5% | 94.2%saturated | 3.32 | 🇺🇸 GPT-6 Astra | 95.0% | 340 |
| 8 | DeepSWE O 113 long-horizon software engineering tasks written from scratch to avoid contamination (v1.1). | Software engineering | 0.5% | 72.9% | 2.79 | 🇺🇸 Claude Opus 5.5 | 74.2% | 41 |
| 9 | SWE-Marathon O Abundant AI's SWE-Marathon: very long-horizon software engineering tasks (1M context, max effort). | Software engineering | 0.6% | 50.8% | 2.11 | G Muse Wonder Full | 57.5% | 22 |
| 10 | Terminal-Bench 3.0 O Terminal-Bench 3.0: harder long-horizon command-line agent tasks scored by official verifiers. | Agentic terminal tasks | 0.7% | 39.0% | 1.85 | 🇺🇸 Claude Opus 5 | 53.0% | 19 |
| 11 | BioMysteryBench (Hard) O BioMysteryBench hard subset (problems not solved by human experts). | Biology research | 0.8% | 49.6% | 2.15 |  🇺🇸 Gemini 3.8 Flash | 56.5% | 21 |
| 12 | DeepSWE v1.1 O Original long-horizon software-engineering tasks in real codebases (Datacurve). | Software engineering | 0.8% | 77.6% | 2.67 | G Muse Wonder Full | 79.0% | 80 |
| 13 | OSWorld 2.0 O OSWorld 2.0 long-horizon computer-use benchmark on a live Ubuntu VM; partial-credit score. | GUI agents | 0.9% | 76.3% | 2.46 | 🇺🇸 Claude Opus 5.5 | 81.8% | 35 |
| 14 | Legal Agent Benchmark (Held-Out) O Harvey Legal Agent Benchmark, Harvey's held-out set. | Legal agents | 0.9% | 16.2% | 1.06 | 🇺🇸 Gemini 4 Argon | 19.6% | 19 |
| 15 | GSO Benchmark O LiveCodeBench GSO benchmark. | Code generation | 0.9% | 73.9% | 2.37 | 🇺🇸 Claude Fable 5.1 | 88.2% | 31 |
| 16 | GDP.pdf O Surge AI multimodal benchmark of 100 professional prompts over information-dense PDFs; strict pass rate. | Document understanding | 1.0% | 36.2% | 1.68 | 🇺🇸 GPT-5.6 Sol | 40.0% | 33 |
| 17 | JobBench O Agentic benchmark of professional, job-like tasks requiring tool use. | Professional tool use | 1.2% | 72.5% | 2.29 | 🇺🇸 Claude Opus 5.5 | 75.4% | 47 |
| 18 | DeepSWE 1.0 O Datacurve DeepSWE v1.0 long-horizon software-engineering tasks, run by Artificial Analysis with each provider's own harness (pass@1). | Software engineering | 1.2% | 62.4% | 2.26 | 🇺🇸 GPT-5.5 | 70.1% | 26 |
| 19 | APEX-Agents O APEX benchmark evaluating agents on long-horizon professional tasks. | Long horizon professional tasks | 1.2% | 52.8% | 2.06 | 🇺🇸 Claude Fable 5 | 59.2% | 76 |
| 20 | OSWorld 2.0 (Strict) O OSWorld 2.0 strict pass rate (full task success). | GUI agents | 1.2% | 41.5% | 1.62 | 🇺🇸 Claude Opus 5.5 | 48.7% | 15 |
| 21 | EnigmaEval O Challenging puzzle benchmark. | Challenging puzzles | 1.3% | 14.5% | 1.76 | 🇺🇸 Gemini 3 Pro | 17.8% | 15 |
| 22 | Terminal-Bench-Science O Terminal-Bench-Science 0.1: agentic scientific research tasks in terminal environments. | Agentic scientific research | 1.5% | 61.2% | 1.64 | 🇺🇸 GPT-6 Astra | 68.1% | 14 |
| 23 | ProgramBench (Almost Solved) O ProgramBench share of instances that are almost solved (near-complete test pass rate). | Software engineering | 1.5% | 26.5% | 1.34 | 🇺🇸 Claude Opus 5 | 37.0% | 11 |
| 24 | ZeroBench O Evaluates zero-shot performance across diverse tasks without task-specific finetuning. | Zero-shot generalization | 1.6% | 22.9% | 1.24 |  🇨🇳 GLM-4.5V | 23.4% | 48 |
| 25 | ExploitBench O Developing V8 vulnerabilities into increasingly capable exploits (capability coverage). | Cybersecurity | 1.8% | 81.7% | 2.01 | 🇺🇸 GPT-6 Astra | 100.0% | 19 |
| 26 | LCB Pro 25Q2 (Med) O LiveCodeBench Pro 2025 Q2 medium subset. | Code generation | 1.9% | 61.4% | 2.58 | 🇺🇸 Gemini 3 Deep Think | 86.0% | 39 |
| 27 | AA-Omniscience-Decision Non-Hallucination Rate O Share of AA-Omniscience-Decision questions not answered correctly where the model chose "I don't know". | Knowledge and hallucination | 1.9% | 62.5% | 0.60 | G d1-ultra | 84.5% | 28 |
| 28 | FrontierScience Research O Research-level problems from the FrontierScience benchmark. | Science reasoning (research) | 1.9% | 57.8% | 1.91 | G Apodex 1.1 | 63.3% | 55 |
| 29 | ProofBench Advanced O Advanced mathematical proof benchmark covering complex theorem proving tasks. | Mathematical proofs (advanced) | 1.9% | 78.9% | 2.08 | G Aletheia | 91.9% | 28 |
| 30 | AutomationBench O Zapier benchmark of end-to-end business workflows via REST APIs across 47 simulated apps; pass/fail on deterministic state assertions. | Agentic business workflows | 2.1% | 59.8% | 1.71 | G MiniMax M3.1 Flash (max) | 62.8% | 70 |
| 31 | EEBench O atopile's electrical-engineering agent benchmark: circuit/PCB design tasks expressed as declarative atopile code and verified automatically. | Electrical engineering | 2.2% | 68.5% | 1.90 | 🇺🇸 Claude Opus 5.5 | 75.0% | 26 |
| 32 | Chartography O Surge AI benchmark for professional chart understanding (no tools). | Chart understanding | 2.3% | 64.8% | 1.74 | 🇺🇸 Gemini 4 Argon | 71.6% | 11 |
| 33 | SciCode (main) O SciCode main score. | Code | 2.5% | 77.1% | 2.46 | 🇺🇸 GPT-5.5 | 90.6% | 31 |
| 34 | Agents' Last Exam O Long-horizon agentic workflows across 55 professional sub-industries performed on a computer. | Agentic professional work | 2.6% | 55.5% | 1.47 | 🇺🇸 GPT-6 Astra | 59.3% | 114 |
| 35 | FrontierCode Main O Cognition FrontierCode 1.1 Main: coding-agent changes graded on correctness and mergeability (tests, scope, style, conventions). | Software engineering | 2.6% | 53.4% | 1.61 | 🇺🇸 Claude Opus 5.5 | 54.6% | 46 |
| 36 | FrontierScience O Frontier-level scientific reasoning and QA benchmark. | Science reasoning | 2.8% | 31.8% | 1.52 | G MAI-Code-1-Flash | 58.2% | 10 |
| 37 | SWE-Atlas Test Writing O SWE-Atlas Test Writing: agentic test-writing tasks over real codebases (Scale AI SWE Atlas). | Software testing | 3.2% | 55.7% | 1.68 | 🇺🇸 Claude Opus 5 | 68.7% | 23 |
| 38 | Biology-Instruction O Multi-omics sequence reasoning benchmark for biological data understanding. | Biology multi-omics | 3.3% | 52.1% | 0.78 | G Intern-S2-Preview-397B | 56.9% | 23 |
| 39 | SEC-Bench Pro O Proof-of-concept exploit generation for vulnerabilities in complex software. | Cybersecurity | 3.3% | 73.6% | 1.83 | 🇺🇸 GPT-6 Astra | 85.4% | 12 |
| 40 | SciReasoner O Scientific reasoning benchmark evaluating multimodal AI models on scientific tasks. | Scientific reasoning | 3.4% | 59.8% | 1.93 | G Intern-S2-Preview-397B | 64.0% | 20 |
| 41 | SWE-Atlas Refactoring O SWE-Atlas refactoring track: agentic refactoring tasks over real codebases. | Software engineering | 3.4% | 53.7% | 1.64 | 🇺🇸 Claude Opus 5 | 60.0% | 20 |
| 42 | Harbor-Index O Aggregate index of agentic terminal/coding tasks run through the Harbor evaluation framework. | Agentic coding | 3.5% | 44.8% | 1.32 | 🇺🇸 Claude Opus 5 | 56.9% | 10 |
| 43 | MCPMark O Benchmark for Model Context Protocol (MCP) agent tool-use. | Agent tool-use (MCP) | 3.6% | 70.4% | 2.29 | 🇺🇸 GPT-5.6 Terra | 74.0% | 93 |
| 44 | SciRes FrontierMath Tier 1-3 O SciRes FrontierMath benchmark covering tiers 1-3. | Math (frontier) | 3.6% | 85.2% | 2.12 | 🇺🇸 GPT-5.6 Sol | 89.0% | 17 |
| 45 | PaperBench O Benchmark for understanding and reasoning over research papers. | Research paper understanding | 3.7% | 90.6%saturated | 2.32 |  🇨🇳 Qwen3.8-Max | 94.1% | 32 |
| 46 | SWE-Atlas Codebase QnA O SWE-Atlas Codebase QnA: agentic question answering over real codebases. | Codebase understanding | 3.9% | 64.1% | 1.72 |  🇨🇳 Kimi-K3 | 68.0% | 44 |
| 47 | Mol-Instructions O Instruction-following benchmark for bio-molecular understanding and generation. | Bio-molecular instruction following | 4.0% | 49.1% | 1.40 | G Intern-S2-397B | 54.0% | 23 |
| 48 | PostTrainBench O Measures how well AI agents can post-train base LLMs under fixed compute/time constraints; average score across AIME 2025, BFCL, GPQA Main, GSM8K, and HumanEval. | Post-training automation | 4.0% | 45.2% | 1.25 | 🇺🇸 Claude Opus 5.5 | 49.3% | 32 |
| 49 | CyberGym O Benchmark for cybersecurity-related coding and reasoning tasks. | Cybersecurity tasks | 4.1% | 90.4%saturated | 2.31 | G MiMo V2.6 Flash RL | 95.1% | 47 |
| 50 | ZeroBench (With Tools) O ZeroBench main set evaluated with tool access in an agent harness (pass@5). | Visual reasoning | 4.2% | 59.1% | 1.44 | 🇨🇳 Kimi-K3.1 Pro | 63.0% | 23 |
| 51 | τ³-Bench Banking O τ³-Bench banking domain: tool-augmented customer-service agent with knowledge retrieval and simulated user. | Agent tasks (banking) | 4.3% | 50.4% | 1.32 | 🇨🇳 Qwen3.8-Max | 55.2% | 121 |
| 52 | ARC-AGI-1 O ARC-AGI Phase 1 aggregate accuracy. | General reasoning | 4.4% | 98.5%saturated | 3.15 | 🇺🇸 GPT-6 Astra | 98.5% | 325 |
| 53 | CL-bench Life O Tencent Hy context-learning benchmark built from real-life scenarios (companion to CL-bench); models must learn and apply rules given in long, messy contexts. | Context learning | 4.5% | 19.0% | 0.85 | 🇺🇸 GPT-5.5 | 21.1% | 15 |
| 54 | BabyVision (With Tools) O BabyVision visual perception benchmark evaluated with tool access in an agent harness. | Visual reasoning | 4.8% | 95.4%saturated | 2.38 | 🇺🇸 GPT-6 Astra | 97.7% | 20 |
| 55 | Terminal-Bench 2.1 O Updated Terminal-Bench 2.x task set (89 tasks) for end-to-end terminal/command-line agents; successor to Terminal-Bench 2.0. | Agent terminal tasks | 4.9% | 91.4%saturated | 2.45 | G SWE-2 | 92.8% | 194 |
| 56 | APEX-Agents 1.1 O Mercor APEX-Agents 1.1 revision: 240 long-horizon cross-application professional-services tasks (investment banking, consulting, corporate law) with updated task specs and judge; Pass@1. Not comparable to APEX-Agents v1.0. | Long horizon professional tasks | 4.9% | 74.4% | 1.62 | 🇺🇸 Gemini 4 Argon | 82.4% | 54 |
| 57 | HLE (Text Only) O Humanity's Last Exam benchmark restricted to text-only inputs. | Advanced reasoning | 5.0% | 48.8% | 1.46 | 🇺🇸 Claude Fable 5 | 53.3% | 206 |
| 58 | SpreadsheetBench 2 O Second version of SpreadsheetBench real-world spreadsheet manipulation tasks. | Spreadsheet manipulation | 5.1% | 34.4% | 1.05 | 🇨🇳 Kimi-K3 | 35.8% | 16 |
| 59 | HeroBench O Benchmark for Long-Horizon Planning and Structured Reasoning in Virtual Worlds. | Long-horizon planning | 5.1% | 68.7% | 2.39 |  🇺🇸 Grok 4 | 91.7% | 24 |
| 60 | Terminal-Bench Hard O Hard subset of Terminal-Bench command-line agent tasks. | Agent terminal tasks | 5.3% | 49.5% | 1.85 | 🇺🇸 GPT-5.4 | 57.6% | 78 |
| 61 | Deep Planning O Benchmark evaluating deep planning and multi-step reasoning capabilities. | Planning and reasoning | 5.3% | 51.0% | 1.60 | 🇨🇳 Qwen3.7-Plus | 62.3% | 27 |
| 62 | HLE O Challenging LLMs at the frontier of human knowledge. | Multi-domain reasoning | 5.3% | 59.8% | 1.41 | 🇺🇸 Claude Opus 5.5 | 64.4% | 258 |
| 63 | BrowseComp-Plus O BrowseComp-Plus: BrowseComp questions over a fixed, curated document corpus for reproducible deep-research agent evaluation. | Deep research | 5.4% | 66.8% | 2.15 | G Jais-3-32B | 71.9% | 34 |
| 64 | USAMO 2026 O 2026 USA Mathematical Olympiad (6 proof problems), graded with MathArena methodology. | Math olympiad (proofs) | 5.5% | 97.5%saturated | 2.24 | 🇺🇸 Claude Mythos 5 | 99.8% | 17 |
| 65 | NL2Repo O Long-horizon repository generation: agents build a complete Python repository from a natural-language requirements document; scored by the repo's test suite. | Software engineering | 5.7% | 71.0% | 1.61 | 🇺🇸 Claude Opus 5 | 79.9% | 73 |
| 66 | OfficeQA Pro O Harder OfficeQA variant. Models must use files, tools and context together instead of answering from memory. | Enterprise document QA | 5.9% | 73.7% | 1.78 | 🇺🇸 Claude Opus 4.7 | 80.6% | 34 |
| 67 | CommerceAgentBench O Agentic benchmark of B2B/e-commerce sourcing and commerce workflows (Accio-Lab). | E-commerce agents | 6.0% | 43.2% | 1.29 | 🇺🇸 GPT-5.6 Sol | 49.5% | 10 |
| 68 | GraphWalks BFS 1M O GraphWalks BFS long-context graph traversal at 1M context (F1). | Long-context reasoning | 6.1% | 74.0% | 1.65 | 🇺🇸 Claude Mythos 5 | 79.4% | 15 |
| 69 | MobileWorld O Benchmark of agents completing tasks in mobile (Android) app environments. | Mobile GUI agents | 6.1% | 79.7% | 1.87 | 🇺🇸 Claude Fable 5 | 85.5% | 28 |
| 70 | SWE-bench Multimodal O SWE-bench variant with visual context (screenshots, design mockups) in issue descriptions. | Software engineering | 6.2% | 57.8% | 1.19 | 🇺🇸 Claude Opus 5.5 | 61.4% | 19 |
| 71 | Vals Index O Vals AI composite index of real-world professional/enterprise tasks (finance, legal, coding, etc.). | Knowledge work | 6.6% | 66.7% | 1.38 | 🇺🇸 Gemini 4 Argon | 68.9% | 50 |
| 72 | MLE-Bench Lite O MLE-Bench Lite: 22-competition low-complexity subset of OpenAI MLE-Bench; metric is medal rate. | ML engineering | 6.7% | 87.6% | 2.02 | 🇺🇸 GPT-5.6 Sol | 88.9% | 40 |
| 73 | FSC-147 O Few-shot counting benchmark across 147 categories. | Few-shot counting | 7.0% | 77.8% | 1.57 | G Glimpse | 83.4% | 13 |
| 74 | MMSIBench (circular) O MMSIBench circular subset for spatial reasoning. | Spatial understanding | 7.0% | 31.2% | 1.00 | 🇺🇸 GPT-5.5 | 36.0% | 10 |
| 75 | Tool-Decathlon O Composite tool-use suite measuring multi-domain tool invocation success (Pass@1). | Agent tool-use | 7.2% | 40.8% | 1.41 | 🇺🇸 GPT-5.2 | 43.8% | 18 |
| 76 | OmniDocBench 1.5 O Document understanding benchmark v1.5 with OCR evaluation. Overall Edit Distance metric, lower is better. | OCR | 7.2% | — | 0.50 | G PaddleOCR-VL | ↓ 0.024 | 61 |
| 77 | NuScenes O 3D scene understanding and perception benchmark for autonomous driving. | 3D scene understanding | 7.3% | 15.0% | 0.65 | 🇨🇳 Qwen3.5-397B-A17B | 16.0% | 12 |
| 78 | FrontierCode Extended O Cognition FrontierCode 1.1 Extended set (score). | Software engineering | 7.6% | 64.5% | 1.34 | 🇺🇸 Claude Opus 5.5 | 65.3% | 46 |
| 79 | TextQuests O Text-based video game benchmark. | Text-based video games | 7.7% | 37.5% | 1.36 | 🇺🇸 Gemini 3 Pro | 41.0% | 18 |
| 80 | GAIA 2 O Grounded agentic intelligence benchmark version 2 covering multi-tool tasks. | General agent tasks | 7.8% | 56.9% | 1.76 | G Ring-1T-2.5 | 75.0% | 17 |
| 81 | AstaBench O Evaluates science agents across literature understanding, data analysis, planning, tool use, coding, and search. | Agent evaluation | 7.9% | 54.7% | 1.48 | 🇺🇸 Claude Opus 4.7 | 58.0% | 27 |
| 82 | Frontier-CS O Frontier-CS agent track (188 tasks): open-ended computer-science optimization problems scored by verifiable feedback. | Algorithmic / open-ended CS problems | 8.3% | 72.5% | 1.58 | 🇺🇸 GPT-5.6 Sol | 76.4% | 12 |
| 83 | OpenAI-MRCR (1M) O OpenAI Multi-Round Chain Reasoning benchmark with 1M context window. | Long-context reasoning | 8.4% | 83.1% | 1.53 | 🇺🇸 Claude Opus 4.6 | 92.9% | 21 |
| 84 | MatTools O Benchmark evaluating LLM agents on materials-science tool usage (pymatgen-based QA and code/tool generation tasks). | Materials science tool use | 8.5% | 55.2% | 1.55 | 🇺🇸 GPT-5.5 | 68.8% | 10 |
| 85 | MLS-Bench-Lite O Lite version of MLS-Bench, a long-horizon agentic machine-learning research/engineering benchmark with an official leaderboard. | Agentic ML engineering | 8.6% | 45.6% | 1.03 | 🇺🇸 Claude Fable 5 | 49.9% | 10 |
| 86 | BenchCAD O CAD modelling benchmark for agents producing 3D CAD artifacts. | CAD generation | 8.6% | 83.7% | 1.39 | 🇺🇸 GPT-6 Astra | 95.9% | 14 |
| 87 | BabyVision O Visual reasoning benchmark testing basic visual perception and understanding. | Visual reasoning | 8.6% | 88.0% | 1.49 | 🇨🇳 Qwen3.8-Max | 91.3% | 71 |
| 88 | BrowseComp O Web browsing comprehension and competence benchmark. | Web browsing | 8.8% | 92.2%saturated | 2.28 | G Nex-N2.5-Max | 92.6% | 212 |
| 89 | LiveMCP-101 O A novel real-time evaluation framework and benchmark to stress‑test agents on complex, real‑world tasks. | Agent real-time eval | 9.1% | 46.7% | 2.09 | 🇺🇸 GPT-5 | 58.4% | 18 |
| 90 | USACO O USA Computing Olympiad problems benchmark (307 problems); pass@1. | Code generation (olympiad) | 9.1% | 12.4% | 1.27 | 🇺🇸 GPT-4o-2024-05-13 | 18.8% | 10 |
| 91 | Terminal-Bench O Command-line task completion benchmark for agents. | Agent terminal tasks | 9.3% | 60.6% | 1.96 | 🇺🇸 Claude Sonnet 4.5 | 64.5% | 90 |
| 92 | PaperBench Code-Dev O PaperBench developer subset measuring code reasoning accuracy. | Code understanding | 9.3% | 83.5% | 1.95 | 🇨🇳 Qwen3.8-Max | 93.0% | 11 |
| 93 | MRCR 256K-8N O OpenAI MRCR v2 8-needle, 128K-256K context bucket. | Long-context reasoning | 9.4% | 88.8% | 2.09 | 🇺🇸 GPT-5.6 Sol | 93.8% | 11 |
| 94 | AutomationBench (partial credit) O AutomationBench end-to-end business workflows scored with partial credit per task. | Business workflows | 9.6% | 72.2% | 1.63 | 🇺🇸 GPT-5.6 Sol | 81.2% | 10 |
| 95 | CL-Bench O Comprehensive long-context benchmark evaluating reasoning over extended contexts. | Long-context reasoning | 9.7% | 25.7% | 0.60 | 🇺🇸 GPT-5.5 | 27.8% | 36 |
| 96 | OJBench O Programming problems evaluated via online judge-style execution. | Code generation (online judge) | 9.8% | 68.3% | 1.77 |  🇨🇳 Keye-VL-2.0-30B-A3B | 71.5% | 71 |
| 97 | SkillsBench O SkillsBench agentic tasks; Qwen reports avg of 5 runs on the 78-task self-contained subset (API-dependent tasks excluded). | Agent skills | 9.9% | 69.6% | 1.41 | 🇺🇸 GPT-5.6 Sol | 73.5% | 59 |
| 98 | AA-Omniscience Accuracy O Share of AA-Omniscience questions answered correctly (accuracy component of the AA-Omniscience Index). | Knowledge and hallucination | 10.0% | 43.8% | 0.91 | 🇺🇸 Gemini 3.5 Flash | 51.4% | 42 |
| 99 | VisFactor O Benchmark of fundamental visual-cognition factors adapted from human psychometric perception tests. | Visual perception | 10.1% | 57.7% | 1.12 | 🇺🇸 GPT-5.6 Sol | 62.8% | 13 |
| 100 | AetherCode O Code generation benchmark for diverse coding tasks. | Code generation | 10.2% | 72.8% | 1.64 | 🇺🇸 GPT-5.5 | 81.6% | 17 |
| 101 | AgentWorldBench (Search) O AgentWorldBench Search domain: open-ended evaluation of language world models predicting next environment state in Search agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 10.6% | 35.5% | 0.81 | 🇨🇳 Qwen-AgentWorld-397B-A17B | 37.8% | 14 |
| 102 | BenchCAD (python tool) O BenchCAD with a Python tool available. | CAD | 10.7% | 92.2%saturated | 1.72 | 🇺🇸 Claude Opus 5.5 | 96.2% | 12 |
| 103 | Toolathlon O Long-horizon, real-world software tool-use tasks. | Agentic software tasks | 10.7% | 75.9% | 1.32 | 🇺🇸 Claude Opus 5 | 80.6% | 61 |
| 104 | Terminal-Bench Terminus O Terminal-Bench Terminus track assessing end-to-end terminal tool use. | Agent terminal tasks | 10.7% | 62.9% | 1.60 | 🇺🇸 Gemini 3.1 Pro | 68.5% | 11 |
| 105 | CMT-Benchmark O Condensed Matter Theory benchmark of expert-written research-level theoretical physics problems. | Physics reasoning (condensed matter) | 10.8% | 58.6% | 0.92 | G Sage-A1.5 | 80.0% | 17 |
| 106 | ProgramBench O ProgramBench (200 instances): agents re-implement complete programs from scratch; score as reported in GLM-5.2 release. | Software engineering | 10.8% | 80.7% | 1.17 | 🇺🇸 GPT-6 Astra | 85.4% | 28 |
| 107 | AMO-Bench O Advanced math olympiad-style benchmark. | Math (competition) | 11.2% | 65.8% | 1.85 | 🇺🇸 Gemini 3 Pro | 72.5% | 25 |
| 108 | Terminal-Bench 2.0 O Second-generation Terminal-Bench leaderboard for end-to-end terminal agents. | Agent terminal tasks | 11.2% | 81.8% | 1.77 | 🇺🇸 GPT-5.5 | 84.7% | 144 |
| 109 | Agents' Last Exam (average score) O Agents' Last Exam (ALE-CLI, 105-task Linux split) average rubric score (as opposed to pass rate). | Agentic professional work | 11.3% | 61.9% | 1.02 | 🇺🇸 Claude Opus 5.5 (max) | 63.7% | 76 |
| 110 | SWE-Bench Pro O Full SWE-Bench Pro benchmark for software-engineering agents. | Software engineering | 11.4% | 82.5% | 1.49 | 🇺🇸 Claude Opus 5.5 | 89.9% | 139 |
| 111 | HealthBench-Hard O Challenging subset of HealthBench focusing on complex, ambiguous clinical cases. | Medical QA (hard) | 11.5% | 43.1% | 0.69 | 🇺🇸 GPT-5 | 46.2% | 24 |
| 112 | CWE-Bench O CWE-Bench v1 vulnerability detection benchmark (cwe-bench.com, Collinear AI); pass@1. | Cybersecurity | 11.8% | 65.8% | 1.11 | 🇺🇸 Gemini 4 Argon | 68.0% | 11 |
| 113 | Hypersim O Hypersim benchmark for synthetic indoor scene understanding and reconstruction. | 3D scene understanding | 11.8% | 28.2% | 0.20 | 🇺🇸 GPT-5 Mini (minimal) | 39.3% | 22 |
| 114 | Vision2Web O Visual website generation benchmark: build working websites from visual designs/prototypes, judged on rendered result. | Coding (UI) | 11.9% | 77.1% | 1.43 | 🇺🇸 GPT-5.6 Sol | 79.8% | 22 |
| 115 | τ-Voice Telecom O τ-Voice (Sierra): τ²-bench telecom tasks over a live speech-to-speech connection; pass^1 (%). | Voice agents | 12.0% | 76.3% | 2.06 | 🇺🇸 Gemini 3.8 Live Extended Thinking (high) | 83.9% | 29 |
| 116 | MCP Universe O Benchmarks multi-step tool-use agents across diverse task suites with a unified overall success metric. | Agent evaluation | 12.0% | 47.1% | 1.31 | 🇺🇸 Gemini 3 Pro | 50.7% | 39 |
| 117 | WildClawBench O Agentic benchmark of in-the-wild tasks run in OpenClaw-style agent scaffolds. | Agentic tasks | 12.3% | 66.8% | 1.30 | 🇺🇸 Claude Opus 4.8 | 68.0% | 53 |
| 118 | Toolathlon Verified O Verified version of Toolathlon (The Tool Decathlon), run via the official evaluation service; pass@1 averaged over 3 runs. | Tool use | 12.3% | 78.1% | 1.48 | 🇺🇸 Claude Opus 5 | 80.6% | 59 |
| 119 | MathArena Apex Shortlist O Shortlist of candidate hard competition problems considered for MathArena Apex. | Mathematics | 12.4% | 89.7% | 2.19 | 🇨🇳 Qwen3.8-Flash-Next | 91.8% | 27 |
| 120 | SuperChem O Chemistry reasoning benchmark evaluating text-based chemical knowledge and problem solving. | Chemistry reasoning | 12.6% | 71.1% | 1.34 | 🇺🇸 Claude Opus 5 | 76.7% | 31 |
| 121 | BioMysteryBench O Anthropic biology research benchmark of open-ended data-analysis 'mysteries'; human-solved subset. | Biology research | 12.8% | 88.8% | 1.60 | 🇺🇸 Claude Opus 5 | 90.1% | 30 |
| 122 | Multi-SWE-Bench O Multi-repository SWE-Bench variant. | Code repair (multi-repo) | 13.2% | 51.1% | 1.28 | G MiniMax M2.7 | 52.7% | 32 |
| 123 | GDPVal O GDPVal benchmark evaluating broad general capabilities of LLMs across diverse tasks. | General capability | 13.2% | 85.6% | 1.68 | 🇺🇸 Claude Opus 5 | 89.4% | 57 |
| 124 | Agentic Coding O Agentic coding benchmark for autonomous software tasks. | Agentic coding | 13.3% | 51.2% | 1.45 | 🇺🇸 Gemini 3 Flash Preview | 53.8% | 22 |
| 125 | HLE (With Tools) O Humanity's Last Exam benchmark evaluated with tool access. | Tool-augmented reasoning | 13.3% | 66.5% | 1.15 | 🇨🇳 Kimi-K3.1 Pro | 70.0% | 153 |
| 126 | FinSearchComp O Financial search and comprehension benchmark measuring retrieval grounded reasoning over financial content. | Financial retrieval | 13.4% | 67.7% | 1.66 | G Seed2.0 Pro | 70.2% | 32 |
| 127 | Chartography (With Tools) O Surge AI Chartography evaluated with tool (code execution) access. | Chart understanding | 13.6% | 89.8% | 1.57 | 🇺🇸 GPT-6 Astra | 91.0% | 16 |
| 128 | VitaBench O Industry-focused benchmark evaluating domain QA performance. | Industry QA | 13.7% | 53.1% | 1.00 | 🇺🇸 Claude Opus 4.5 | 56.3% | 66 |
| 129 | HLE-MultiChoice O The 513 text-only original multiple-choice questions of Humanity's Last Exam, posed as one choice among the lettered answers. | Advanced reasoning | 13.9% | 56.3% | 0.20 | 🇺🇸 Claude Opus 5.5 (max) | 70.2% | 58 |
| 130 | MMSearch O Benchmark for multimodal search engines / LMMs answering queries via image+web search. | Multimodal search | 13.9% | 78.3% | 1.77 | 🇨🇳 Kimi-K2.6 Thinking | 80.0% | 24 |
| 131 | SimpleQA Verified O Verified SimpleQA variant for parametric knowledge accuracy. | QA | 14.2% | 71.9% | 0.91 | 🇺🇸 Gemini 3.1 Pro | 77.3% | 82 |
| 132 | Terminal-Bench Lite O OpenThoughts TBLite: lighter, faster subset-style variant of Terminal-Bench 2.0 for evaluating terminal agents. | Agent terminal tasks | 14.5% | 68.1% | 1.77 | 🇨🇳 Qwen3.6-27B | 70.8% | 15 |
| 133 | VLMs are Biased O Evaluates whether VLMs truly 'see' vs. relying on memorized knowledge; measures bias toward non-visual priors. | Multimodal bias | 14.5% | 42.5% | 0.68 | 🇺🇸 GPT-6 Astra | 84.1% | 13 |
| 134 | ART Agent Red Teaming O Evaluation suite for adversarial red-teaming of autonomous AI agents. | Agent robustness | 14.7% | — | 1.13 | 🇺🇸 Claude Opus 4.5 | ↓ 33.6% | 38 |
| 135 | PHYBench O Physics reasoning and calculation benchmark. | Physics reasoning | 15.0% | 79.3% | 1.78 | 🇺🇸 Gemini 3.1 Pro | 82.0% | 33 |
| 136 | ArxivMath O Math problems derived from recent arXiv papers (reported in Anthropic Fable 5/Mythos 5 system card). | Research math | 15.2% | 74.6% | 1.23 | 🇺🇸 GPT-5.6 Sol | 79.5% | 18 |
| 137 | Workspace Bench O Workspace Bench: OpenClaw-style workspace tasks with large heterogeneous file dependencies (100-task setting). | Agentic office work | 15.3% | 68.5% | 1.14 | 🇺🇸 Claude Opus 5 | 75.0% | 15 |
| 138 | SWE-Bench Pro (Public) O Public subset of the SWE-Bench Pro benchmark for software-engineering agents. | Software engineering | 15.3% | 71.4% | 1.29 | 🇺🇸 Claude Fable 5 | 80.4% | 100 |
| 139 | ChessBench (decision) O Choose the best move among candidates (ChessBench); accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Chess | 15.3% | 23.7% | 0.46 | G Clef | 24.7% | 38 |
| 140 | WorkBuddy Bench O Agentic benchmark of workplace-assistant (WorkBuddy) tasks such as documents, spreadsheets and messaging. | Office assistant agents | 15.5% | 39.0% | 1.07 | G Muse Glimmer 30B | 45.9% | 12 |
| 141 | AA-Omniscience Non-Hallucination Rate O Share of AA-Omniscience questions not answered incorrectly (abstains instead of hallucinating) among non-correct answers. | Knowledge and hallucination | 15.6% | 72.1% | 1.11 |  🇺🇸 Nemotron 3 Ultra | 78.7% | 24 |
| 142 | FrontierSWE O Proximal's FrontierSWE benchmark of frontier-difficulty software engineering tasks; dominance score (snapshot-dependent). | Software engineering | 15.7% | 80.3% | 1.16 | 🇺🇸 Claude Fable 5 | 88.8% | 19 |
| 143 | BrowseComp_zh O Chinese variant of the BrowseComp web browsing benchmark. | Web browsing (Chinese) | 16.0% | 84.7% | 1.80 | G XYZ-Aquila-pro | 85.1% | 101 |
| 144 | MolecularIQ O Benchmark of molecular structure reasoning for chemistry-capable LLMs. | Molecular structure reasoning | 16.1% | 65.6% | 1.17 | 🇺🇸 GPT-5.5 | 76.4% | 19 |
| 145 | HLE Overconfidence O Overconfidence rate derived from Humanity's Last Exam evaluations. | Overconfidence / safety | 16.1% | — | 1.12 | 🇺🇸 GPT-5.2 | ↓ 43.7% | 15 |
| 146 | FrontierFinance O Professional finance benchmark combining quantitative reasoning with analysis of financial data and documents (reported in Step 5 Preview release). | Finance | 16.4% | 64.7% | 0.86 | 🇺🇸 Claude Opus 5 | 69.7% | 18 |
| 147 | MRCR 128K-8N O Multi-Round Coreference Resolution benchmark at 128k context with 8 needles. | Long-context reasoning | 16.5% | 93.8%saturated | 1.91 | 🇺🇸 Gemini 3.7 Flash | 97.0% | 44 |
| 148 | MCP-Atlas O Aggregate MCP agent benchmark covering tool-use and planning tasks. | Agent evaluation | 16.5% | 87.6% | 1.66 | G Muse Spark 1.2 | 90.3% | 130 |
| 149 | SGI-Bench O Scientific general intelligence benchmark of agentic scientific research workflows. | Scientific agents | 16.5% | 51.4% | 0.91 | G Intern-S2-Preview | 52.5% | 17 |
| 150 | WebArena-Verified O Verified version of the WebArena browser-use benchmark. | Web automation | 16.5% | 69.6% | 1.23 | 🇨🇳 Kimi-K3 | 71.6% | 19 |
| 151 | POP909-CL (decision) O Chord-label decisions over POP909 symbolic music; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Music understanding | 16.6% | 62.0% | 1.49 | G Bespoke-Nimble-9B-v3 | 74.6% | 36 |
| 152 | VLMsAreBiased O Benchmark evaluating biases in vision-language models. | Multimodal bias | 16.7% | 80.9% | 1.07 | 🇨🇳 Qwen3.8-Max | 88.3% | 21 |
| 153 | MedXpertQA (Text) O Text-only subset of MedXpertQA expert-level medical multiple-choice questions. | Medical QA | 17.1% | 61.7% | 1.14 | G OpenEvidence Darwin | 72.8% | 17 |
| 154 | HLE-Verified O Verified and revised version of Humanity's Last Exam (HLE) with component-wise verification protocol. | Multi-domain reasoning | 17.1% | 53.7% | 0.82 | 🇺🇸 Gemini 3.8 Flash | 54.9% | 17 |
| 155 | Aider-Polyglot (Diff) O Aider polyglot leaderboard using diff mode (pass@2). | Code assistant eval | 17.1% | 88.2% | 2.87 | 🇺🇸 Gemini 3 Pro Preview | 91.9% | 65 |
| 156 | OSWorld 2.1 O OSWorld 2.1 long-horizon computer-use benchmark; partial-credit score. | GUI agents | 17.1% | 80.1% | 1.19 | 🇺🇸 Claude Opus 5.5 | 81.8% | 10 |
| 157 | SimpleQA O Simple question answering benchmark. | QA | 17.2% | 63.6% | 1.20 |  🇨🇳 ERNIE 5.0 | 74.0% | 133 |
| 158 | Aider-Polyglot O Aider polyglot coding leaderboard. | Code assistant eval | 17.3% | 88.5% | 2.85 | 🇺🇸 Gemini 3 Pro Preview | 92.9% | 133 |
| 159 | HealthBench Professional O 525 physician-authored clinical conversations (consults, documentation, research) graded against rubric criteria. | Medical QA | 17.9% | 69.8% | 0.89 | G OpenEvidence Darwin | 82.7% | 28 |
| 160 | TIR-Bench O Benchmark for tool-integrated reasoning with visual models. | Tool-integrated reasoning | 17.9% | 58.5% | 1.13 | 🇨🇳 Qwen3.5-397B-A17B | 62.5% | 16 |
| 161 | DeepSearchQA O Multi-step web search and question answering benchmark. | Deep web search QA | 18.4% | 95.7%saturated | 2.09 | G Atria-Dawn | 96.5% | 75 |
| 162 | τ-Voice Retail O τ-Voice (Sierra): τ-bench retail customer-service tasks over a live speech-to-speech connection; pass^1 (%). | Voice agents | 18.4% | 61.7% | 1.46 | 🇺🇸 GPT-Live-1 (medium) | 72.5% | 29 |
| 163 | SWE-Bench O Supervised software engineering benchmark across many repos and issues. | Code repair | 18.8% | 73.5% | 1.79 | 🇺🇸 GPT-5 Codex | 74.5% | 43 |
| 164 | ZeroBench (sub) O Subset of ZeroBench targeting harder zero-shot reasoning cases. | Zero-shot generalization | 19.0% | 47.1% | 0.61 | G Seed2.1 Pro | 49.4% | 52 |
| 165 | JobBench (Easy) O JobBench easy split of realistic professional job tasks. | Professional job tasks | 19.1% | 77.5% | 1.38 | 🇨🇳 Qwen3.8-Max | 82.8% | 13 |
| 166 | AutomationBench-AA O Artificial Analysis run of AutomationBench end-to-end business workflows. | Business workflows | 19.2% | 58.6% | 0.55 | 🇨🇳 Qwen3.8-Flash-Next | 72.1% | 16 |
| 167 | FACTS Benchmark Suite O Comprehensive factuality benchmark suite covering held-out internal grounding, parametric knowledge, multimodal understanding, and search retrieval benchmarks. | Held out internal grounding, parametric, MM, and search retrieval benchmarks | 19.2% | 61.6% | 1.28 | 🇺🇸 Gemini 3 Pro | 70.5% | 14 |
| 168 | AA-Omniscience-Decision Accuracy O Accuracy on 500 private AA-Omniscience questions posed as five options (four answers plus "I don't know") for decision models. | Knowledge and hallucination | 19.3% | 88.9% | 1.57 | 🇺🇸 Claude Opus 5.5 (max) | 93.4% | 28 |
| 169 | DoQA O Domain-specific FAQ conversational QA (ChatRAG-Bench subset). | Conversational QA | 19.4% | 39.9% | 0.51 | G Yuan3.0 Ultra | 63.4% | 10 |
| 170 | Audio MultiChallenge O Scale AI Audio MultiChallenge: multi-turn spoken conversation benchmark; average pass rate (APR). | Spoken multi-turn conversation | 19.8% | 66.8% | 1.21 | 🇺🇸 Gemini 3.8 Flash | 71.9% | 31 |
| 171 | WorldVQA O Visual question answering requiring world knowledge and commonsense reasoning. | World knowledge VQA | 19.9% | 58.5% | 0.72 | 🇺🇸 Claude Opus 4.6 | 65.4% | 30 |
| 172 | PerceptionBench O Moonshot benchmark of atomic visual perception capabilities in multimodal LLMs. | Visual perception | 20.3% | 58.4% | 0.83 | 🇺🇸 Gemini 3.5 Flash | 60.9% | 11 |
| 173 | VCT O Virology Capabilities Test: a benchmark that measures an LLM's ability to troubleshoot complex virology laboratory protocols. | Virology capability (protocol troubleshooting) | 20.6% | 48.8% | 1.11 | 🇺🇸 Grok 4.1 (Thinking) | 61.0% | 13 |
| 174 | OSWorld Verified O Verified subset of OSWorld GUI agent benchmark. | GUI agents | 20.7% | 85.8% | 1.44 | 🇨🇳 Qwen-CUA | 86.2% | 87 |
| 175 | BALROG O Benchmark for assessing LLM agents under adversarial and out-of-distribution tool-use scenarios. | Agent robustness | 20.8% | 37.6% | 0.94 | 🇺🇸 Grok 4 | 43.6% | 27 |
| 176 | QwenClawBench O Real-user-distribution Claw (OpenClaw-style) agent benchmark released by Qwen. | Agent tasks (OpenClaw) | 21.3% | 62.0% | 1.12 | 🇺🇸 Claude Opus 4.6 | 65.5% | 37 |
| 177 | BFCL v3 (Multi-Turn) O BFCL v3 Multi-Turn subset for multi-turn tool calling evaluation. | Tool calling | 21.9% | 66.1% | 1.24 | G MiniMax M2.5 | 76.8% | 37 |
| 178 | Defects4J O Real Java bugs from Defects4J used for automated program repair. | Program repair | 21.9% | 23.9% | 1.20 | 🇺🇸 GPT-4o-2024-05-13 | 26.1% | 10 |
| 179 | BigCodeBench Hard O Harder variant of BigCodeBench testing complex programming and library tasks with function-level code generation. | Code generation (hard) | 22.0% | 35.6% | 0.90 | 🇺🇸 Claude 3.7 Sonnet (2025-02-19) | 35.8% | 212 |
| 180 | BIG-Bench Extra Hard O Extra hard subset of BIG-bench tasks. | Multi-task reasoning | 22.2% | 72.1% | 1.41 | 🇺🇸 Gemini 3 Pro | 78.8% | 47 |
| 181 | CountQA O Visual question answering benchmark on object counting in real-world images. | Visual counting | 22.3% | 84.0% | 1.11 | 🇺🇸 GPT-6 Astra | 86.4% | 41 |
| 182 | Doc2Dial O Document-grounded dialogue QA (ChatRAG-Bench subset). | Conversational QA | 22.3% | 38.3% | 0.45 | G Yuan3.0 Ultra | 55.8% | 10 |
| 183 | HiPhO O High-school physics olympiad problems with diagrams for evaluating multimodal physics reasoning. | Physics olympiad (multimodal) | 22.4% | 86.9% | 1.38 | 🇨🇳 Qwen3.8-Max | 90.0% | 26 |
| 184 | AgentWorldBench (SWE) O AgentWorldBench SWE domain: open-ended evaluation of language world models predicting next environment state in SWE agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 22.7% | 65.9% | 1.04 | 🇨🇳 Qwen-AgentWorld-397B-A17B | 68.5% | 14 |
| 185 | CNMO 2024 O China National Mathematical Olympiad 2024 evaluation set. | Math (competition) | 22.8% | 77.8% | 2.71 | G openPangu-R-72B-2512 Slow Thinking | 82.8% | 18 |
| 186 | xBench-DeepSearch (2025.10) O xBench DeepSearch benchmark October 2025 snapshot. | Agentic research | 23.0% | 89.2% | 1.45 | 🇨🇳 Kimi-K2.6 Thinking | 90.0% | 37 |
| 187 | AgentWorldBench (Terminal) O AgentWorldBench Terminal domain: open-ended evaluation of language world models predicting next environment state in Terminal agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 23.0% | 57.3% | 0.86 | 🇺🇸 Claude Opus 4.8 | 59.2% | 14 |
| 188 | AICodeKing Non-Agentic O Non-agentic code generation benchmark from AICodeKing. | Code generation (non-agentic) | 23.0% | 87.8% | 1.15 | 🇺🇸 Claude Opus 4.6 | 100.0% | 20 |
| 189 | INSCIT O Information-seeking conversations with mixed-initiative (ChatRAG-Bench subset). | Conversational QA | 23.1% | 34.1% | 0.39 | G Yuan3.0 Ultra | 40.0% | 10 |
| 190 | BrowseComp-VL O Vision-language variant of BrowseComp requiring multimodal web browsing. | Multimodal browsing | 23.2% | 55.8% | 0.86 | 🇺🇸 GPT-5.5 | 65.7% | 11 |
| 191 | SWE-Bench Multilingual O Multilingual variant of SWE-Bench for issue fixing. | Code repair (multilingual) | 23.5% | 91.5%saturated | 1.50 | 🇺🇸 Claude Opus 5.5 | 93.9% | 167 |
| 192 | DeepResearchBench O Research-oriented agentic writing and planning benchmark. | Agentic research writing | 23.8% | 48.9% | 0.84 | G AgentCPM-Report | 50.1% | 16 |
| 193 | VisuLogic O Logical reasoning and compositionality benchmark for visual-language models. | Visual logical reasoning | 23.8% | 52.0% | 0.51 | G Seed2.1 Pro | 54.3% | 47 |
| 194 | DeepResearch Bench II O Second-generation DeepResearch Bench evaluating long-form deep-research reports with fine-grained rubrics. | Deep research reports | 24.1% | 55.7% | 0.61 | G Xing4.0-29B-A4B | 60.8% | 14 |
| 195 | SciCode (sub) O SciCode subset score (sub). | Code | 24.1% | 89.1% | 0.75 | 🇺🇸 GPT-5.5 | 95.1% | 213 |
| 196 | Video-MME v2 O Second version of the Video-MME video multimodal evaluation suite. | Video understanding | 24.3% | 63.8% | 0.61 | 🇺🇸 Gemini 3 Pro | 66.1% | 27 |
| 197 | SWE-Bench Verified O Verified subset of SWE-Bench for issue fixing. | Code repair | 24.3% | 94.7%saturated | 1.75 | 🇺🇸 Claude Opus 5 | 96.0% | 340 |
| 198 | BABE O STEM reasoning benchmark evaluating broad applied and basic engineering knowledge. | STEM reasoning | 24.4% | 55.2% | 0.90 | 🇺🇸 GPT-5.2 (high) | 58.1% | 11 |
| 199 | SATA-Bench O Select-all-that-apply multiple-choice questions; case exact accuracy (Jev Decision Index suite, posed as typed decisions). | Multiple-answer QA | 24.5% | 35.4% | 0.79 | G vLLM-SR d3 | 38.4% | 38 |
| 200 | FinanceAgent v2 O Vals AI Finance Agent Benchmark v2: agentic financial-research tasks over SEC filings; new dataset, not comparable to v1.1. | Agentic finance tasks | 25.1% | 63.1% | 0.68 | 🇺🇸 Gemini 4 Argon | 65.4% | 32 |
| 201 | HMMT 2025 O Harvard–MIT Mathematics Tournament 2025 problems. | Math (competition) | 25.3% | 99.1%saturated | 2.93 | 🇺🇸 Gemini 3 Pro | 99.8% | 103 |
| 202 | ClawEval (Pass^3) O ClawEval OpenClaw agent benchmark, pass^3 (all 3 trials succeed). | Agent tasks (OpenClaw) | 25.8% | 75.0% | 1.02 |  🇨🇳 DeepSeek V4-Flash-0731 | 78.9% | 70 |
| 203 | AgentWorldBench (Web) O AgentWorldBench Web domain: open-ended evaluation of language world models predicting next environment state in Web agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 26.8% | 52.4% | 0.69 | 🇺🇸 Claude Opus 4.8 | 54.7% | 14 |
| 204 | GAIA O Comprehensive benchmark for agentic tasks. | General AI tasks | 26.9% | 92.7%saturated | 1.50 | 🇨🇳 DeepSeek V4-Pro-Thinking (max) | 98.1% | 93 |
| 205 | MedXpertQA-MM O Multimodal medical expert question answering benchmark. | Medical VQA | 27.2% | 80.6% | 1.20 | 🇺🇸 GPT-5.6 Sol | 81.5% | 58 |
| 206 | OMEGA O OMEGA olympiad-grade mathematics reasoning benchmark. | Math (advanced) | 27.3% | 50.1% | 1.58 |  🇺🇸 OLMo-3.1-Think-32B | 53.4% | 38 |
| 207 | SpatialViz O Mental visualization benchmark. | Mental visualization | 28.0% | 59.6% | 0.86 | 🇺🇸 GPT-5.2 | 65.8% | 15 |
| 208 | ResearchRubrics O Benchmark evaluating model ability to conduct research and synthesize findings. | Research evaluation | 28.3% | 73.8% | 0.97 | 🇨🇳 Kimi-K3 | 76.2% | 27 |
| 209 | OSWorld O Agentic GUI task completion and grounding on desktop environments. | GUI agents | 28.4% | 65.5% | 0.98 | 🇺🇸 Claude Opus 4.6 | 72.7% | 26 |
| 210 | TAU1-Airline O Tool-augmented agent evaluation in airline scenarios (TAU1). | Agent tasks (airline) | 28.4% | 51.4% | 1.19 | G openPangu-R-72B-2512 Slow Thinking | 56.0% | 13 |
| 211 | MMSI-Bench O Multi-image spatial intelligence benchmark (standard, non-circular evaluation). | Spatial reasoning | 28.7% | 49.8% | 0.45 | 🇺🇸 GPT-6 Astra | 57.9% | 20 |
| 212 | PopQA O Open-domain popular culture question answering benchmark testing long-tail factual recall. | Knowledge / QA | 28.8% | 54.9% | 0.31 |  🇺🇸 Llama 3.1 Tulu 3 405B SFT | 55.7% | 66 |
| 213 | BrowseComp (With Content Manager) O BrowseComp benchmark evaluated with content manager assistance. | Web browsing | 28.8% | 86.5% | 1.40 | 🇨🇳 Kimi-K3 | 91.2% | 35 |
| 214 | MTVQA O Multilingual text-centric visual question answering across 9 languages (average accuracy). | Multilingual text-centric VQA | 28.9% | 30.8% | 0.67 |  🇨🇳 InternVL2.5-78B | 31.9% | 24 |
| 215 | SURDS O Spatial understanding and reasoning benchmark on autonomous-driving scenes. | Driving scene spatial reasoning | 29.1% | 74.7% | 1.00 | 🇺🇸 Claude Fable 5 | 79.4% | 18 |
| 216 | OpenAI-MRCR (128k) O OpenAI Multi-Round Chain Reasoning benchmark with 128k context window. | Long-context reasoning | 29.2% | 92.7%saturated | 1.45 | G K-EXAONE 2.0 | 94.4% | 44 |
| 217 | Omni-MATH-HARD O Challenging math benchmark (Omni-MATH-HARD). | Math | 29.4% | 64.4% | 1.50 | 🇺🇸 GPT-5 (high) | 73.6% | 20 |
| 218 | LCB Pro 25Q2 (Easy) O LiveCodeBench Pro 2025 Q2 easy subset. | Code generation | 29.8% | 93.8%saturated | 2.27 | 🇺🇸 Gemini 3 Pro | 94.8% | 39 |
| 219 | FinanceAgent O Interactive financial agent benchmark requiring multi-step tool use. | Agentic finance tasks | 29.9% | 56.6% | 0.78 | 🇺🇸 Claude Opus 4.6 | 60.7% | 10 |
| 220 | AgentWorldBench (Android) O AgentWorldBench Android domain: open-ended evaluation of language world models predicting next environment state in Android agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 30.0% | 61.0% | 0.78 | 🇺🇸 Claude Opus 4.6 | 61.7% | 14 |
| 221 | OneMillionBench O OneMillion Bench: expert-level, economically valuable tasks evaluated with tools (search). | Agentic search / expert tasks | 30.6% | 69.3% | 0.76 | 🇺🇸 Claude Opus 4.7 | 73.0% | 13 |
| 222 | EMMA O EMMA benchmark for multimodal reasoning. | Multimodal reasoning | 30.9% | 75.5% | 1.08 | G Seed2.1 Pro | 79.3% | 11 |
| 223 | WideSearch O Wide web search and QA benchmark. | Web search | 30.9% | 84.4% | 1.18 | 🇺🇸 GPT-5.6 Sol | 86.3% | 90 |
| 224 | TOMATO O Temporal ordering and motion analysis benchmark (TOMATO). | Temporal understanding | 31.0% | 88.2% | 0.95 | 🇺🇸 GPT-6 Astra | 96.0% | 42 |
| 225 | SEAL-0 O Evaluation of multi-step browsing agents on search, evidence gathering, and synthesis tasks. | Agentic web search | 31.4% | 57.8% | 0.57 | G MiroThinker-H1 | 61.3% | 44 |
| 226 | AgentWorldBench (OS) O AgentWorldBench OS domain: open-ended evaluation of language world models predicting next environment state in OS agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 32.0% | 68.0% | 0.86 | 🇺🇸 Claude Opus 4.6 | 70.2% | 14 |
| 227 | AutoCodeBench O End-to-end autonomous coding benchmark with unit-test based execution across diverse repositories and tasks. | Autonomous coding | 32.0% | 51.3% | 0.86 | 🇺🇸 Claude Opus 4 (Thinking) | 52.4% | 45 |
| 228 | LPFQA O Long-form financial question answering benchmark. | Finance QA | 32.2% | 53.8% | 0.65 | 🇺🇸 Claude Sonnet 4.5 | 54.9% | 12 |
| 229 | SUNRGBD O SUN RGB-D benchmark for indoor scene understanding from RGB-D imagery. | 3D scene understanding | 32.5% | 41.1% | 0.25 | 🇺🇸 GPT-5 Mini (minimal) | 45.8% | 19 |
| 230 | ScreenSpot-Pro O Average center accuracy on ScreenSpot-Pro. | Screen UI locators | 32.6% | 87.8% | 1.26 | 🇺🇸 GPT-6 Astra | 92.7% | 164 |
| 231 | Spiral-Bench O A LLM-judged benchmark measuring sycophancy and delusion reinforcement. | Safety / sycophancy | 32.6% | 80.2% | 0.93 | 🇺🇸 GPT-5 | 87.0% | 24 |
| 232 | HMMT Feb 2025 O Harvard–MIT Mathematics Tournament February 2025 problems. | Math (competition) | 32.7% | 99.6%saturated | 2.86 | 🇺🇸 GPT-5.2 (high) | 100.0% | 96 |
| 233 | EgoPlan-Bench2 O Egocentric embodied task-planning benchmark (v2). | Embodied reasoning | 32.7% | 58.7% | 0.56 | 🇺🇸 GPT-6 Astra | 69.3% | 13 |
| 234 | KoBALT O Korean benchmark for knowledge and language understanding. | Korean knowledge | 32.9% | 71.2% | 1.02 | 🇨🇳 DeepSeek V4-Flash-Thinking (max) | 75.3% | 26 |
| 235 | KoSimpleQA O Korean simple question answering benchmark. | Korean QA | 33.1% | 45.3% | 0.56 | G Kanana-2-30B-A3B-Mid-2601 | 49.7% | 13 |
| 236 | IMO AnswerBench O Evaluates free-form solutions to International Mathematical Olympiad problems using expert-style grading rubrics. | Math (competition) | 33.2% | 92.0%saturated | 1.71 | 🇨🇳 DeepSeek V4-Pro | 93.0% | 150 |
| 237 | PolyMATH O Polyglot mathematics benchmark assessing cross-topic math reasoning. | Math reasoning | 33.2% | 83.1% | 1.26 | 🇨🇳 Qwen3.7-Max | 86.5% | 69 |
| 238 | BLEnD O Benchmark assessing cultural knowledge, sensitivity, and language use across contexts. | Cultural understanding | 33.5% | 40.7% | 0.90 | 🇨🇳 Qwen2.5-72B-Instruct | 75.4% | 12 |
| 239 | HMMT Feb 2026 O Harvard–MIT Mathematics Tournament February 2026 problems. | Math (competition) | 33.9% | 98.1%saturated | 2.02 | G Darwin-180B-RSI | 100.0% | 120 |
| 240 | AA-LCR O A challenging benchmark measuring language models' ability to extract, reason about, and synthesize information from long-form documents ranging from 10k to 100k tokens. | Long-context reasoning | 34.1% | 83.9% | 1.11 | 🇨🇳 Kimi-K3 | 88.7% | 209 |
| 241 | Machiavelli O Benchmark for deceptive or manipulative behavior in social interactions. | Deception / safety | 34.1% | — | 0.36 | 🇺🇸 Claude Haiku 4.5 | ↓ 52.2% | 15 |
| 242 | MultiChallenge O Multi-domain instruction-following benchmark. | Instruction following | 34.2% | 75.1% | 0.83 | G Nova 2 Pro | 77.7% | 129 |
| 243 | TAU2-Telecom O Tool-augmented agent evaluation in telecom scenarios (TAU2). | Agent tasks (telecom) | 34.2% | 99.4%saturated | 2.15 | 🇨🇳 Qwen3.6-35B-A3B | 100.0% | 203 |
| 244 | QuAC O Question answering in context. | Conversational QA | 34.6% | 51.9% | 0.20 | G Yuan3.0 Ultra | 54.5% | 23 |
| 245 | FinanceAgent v1.1 O Finance Agent v1.1 benchmark for interactive financial agent evaluation. | Agentic finance tasks | 34.6% | 68.8% | 0.66 | 🇺🇸 Gemini 3.7 Flash | 71.0% | 30 |
| 246 | SFE O Structured factual evaluation for multimodal models. | Multimodal reasoning | 34.7% | 62.1% | 0.59 | 🇺🇸 Gemma 4 26B A4B IT | 63.1% | 23 |
| 247 | HybriDial O Hybrid table+text grounded dialogue QA (ChatRAG-Bench subset). | Conversational QA | 35.0% | 53.7% | 0.48 | G Yuan3.0 Ultra | 72.9% | 10 |
| 248 | BeyondAIME O Advanced math problems exceeding AIME difficulty. | Math (beyond AIME) | 35.1% | 88.5% | 1.38 | 🇺🇸 GPT-5.5 | 91.0% | 35 |
| 249 | Enamel O Composite capability benchmark capturing broad model performance (Enamel score). | Composite capability | 35.1% | 42.6% | 1.01 | G Rnj-1.5 | 50.7% | 10 |
| 250 | Home appliance simulator (decision) O Multi-step home appliance control decisions; case exact accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Device control | 35.2% | 92.5%saturated | 2.59 | G Bespoke-Nimble-9B-v3 | 98.9% | 38 |
| 251 | ContPhy O Continuum Physical Concept Learning and Reasoning from videos. | Video physical reasoning | 35.3% | 64.3% | 0.74 | G Seed2.0 Pro | 67.4% | 10 |
| 252 | VLABench O Language-conditioned robot manipulation reasoning benchmark (VLM track). | Embodied reasoning | 35.4% | 68.2% | 0.63 | G PhysBrain1.5-8B | 76.4% | 13 |
| 253 | SpreadSheetBench v1 O Real-world spreadsheet manipulation tasks (v1). | Spreadsheet manipulation | 35.5% | 88.6% | 1.32 | G MiniMax M3 | 89.4% | 24 |
| 254 | CrossVid O Benchmark for cross-video (multi-video) understanding and reasoning. | Multi-video reasoning | 35.5% | 62.3% | 0.63 | G Seed2.1 Pro | 65.0% | 10 |
| 255 | AutoCodeBench-Lite O Lite version of AutoCodeBench focusing on smaller tasks with the same end-to-end, unit-test-based evaluation. | Autonomous coding | 35.7% | 63.4% | 1.18 | 🇺🇸 Claude Opus 4 | 64.5% | 46 |
| 256 | XLRS-Bench O Remote sensing benchmark for evaluating multimodal AI on satellite and aerial imagery. | Remote sensing | 35.8% | 52.5% | 0.43 | 🇺🇸 Gemini 3.1 Pro | 54.3% | 18 |
| 257 | ChartMuseum O Large-scale curated collection of charts for evaluating parsing, grounding, and reasoning. | Chart understanding | 35.8% | 79.6% | 1.10 | 🇺🇸 Claude Mythos 5 | 85.9% | 50 |
| 258 | MiniF2F (Test) O MiniF2F competition benchmark (test split). | Math competition | 35.8% | 91.3%saturated | 0.48 | G Leanstral 1.5 | 100.0% | 26 |
| 259 | MHPP O Mostly Hard Python Problems: 210 hand-written function-level Python problems probing harder code-generation skills; pass@1. | Code generation | 36.0% | 33.7% | 1.31 | G Stable-DiffCoder-8B-Instruct | 42.4% | 10 |
| 260 | Minerva (Video) O Minerva video question-answering benchmark with complex multi-step reasoning over videos (not the Minerva math set). | Video reasoning | 36.1% | 68.0% | 0.82 | G Seed2.1 Pro | 70.7% | 10 |
| 261 | AndroidWorld O Benchmark for agents operating Android apps via UI automation. | Mobile agents | 36.2% | 86.2% | 1.06 | 🇺🇸 Claude Fable 5 | 88.8% | 70 |
| 262 | FrontierScience Olympiad O Olympiad-level problems from the FrontierScience benchmark. | Science reasoning (olympiad) | 36.3% | 81.1% | 0.96 | G Intern-S2-397B | 87.0% | 49 |
| 263 | LiveCodeBench v5 (2024.10-2025.02) O LiveCodeBench v5 snapshot covering Oct 2024-Feb 2025. | Code generation | 36.5% | 87.5% | 1.91 | 🇺🇸 Gemini 3 Pro Preview | 91.7% | 147 |
| 264 | ArtifactsBench O Artifacts-focused coding and tool-use benchmark evaluating generated code artifacts. | Agentic coding | 36.5% | 77.1% | 0.88 | G TeleChat3-Coder-36B-Thinking | 82.8% | 55 |
| 265 | TAU2-Airline O Tool-augmented agent evaluation in airline scenarios (TAU2). | Agent tasks (airline) | 36.6% | 85.2% | 1.24 | 🇺🇸 Claude Opus 4.7 | 88.5% | 101 |
| 266 | TopiOCQA O Topic-switching open-domain conversational QA (ChatRAG-Bench subset). | Conversational QA | 36.6% | 54.0% | 0.37 | G Yuan3.0 Ultra | 72.4% | 10 |
| 267 | Omni-MATH O Omni-MATH benchmark covering diverse math reasoning tasks across difficulty levels. | Math reasoning | 37.2% | 67.1% | 1.29 | G Ling 1T | 74.5% | 36 |
| 268 | AIME 2025 O American Invitational Mathematics Examination 2025 problems. | Math (competition) | 37.2% | 100%saturated | 2.84 | 🇺🇸 Claude Sonnet 4.5 | 100.0% | 498 |
| 269 | AgentWorldBench (MCP) O AgentWorldBench MCP domain: open-ended evaluation of language world models predicting next environment state in MCP agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 37.7% | 69.3% | 0.71 | 🇺🇸 GPT-5.4 | 70.1% | 14 |
| 270 | TreeBench O Evaluates hierarchical/tree-structured reasoning and planning capabilities in LLMs/VLMs. | Reasoning with tree structures | 37.7% | 63.8% | 0.61 | G Seed2.1 Turbo | 71.1% | 17 |
| 271 | TAU2-Bench Airline (Verified) O Human-validated revision of the tau2-bench airline domain (amazon-agi/tau2-bench-verified). | Agent tasks | 38.0% | 70.6% | 1.13 | 🇺🇸 GPT-5.1 | 72.4% | 11 |
| 272 | GDPval (rubrics) O GDPval tasks scored pointwise against rubrics (not pairwise win rate), as reported by MiniMax. | Professional knowledge work | 38.1% | 90.2%saturated | 1.28 | 🇨🇳 Kimi-K3 | 91.6% | 26 |
| 273 | VABench-Point O VABench visual-action pointing subset. | Embodied reasoning | 38.2% | 66.3% | 0.56 | G Embodied-R1.5-8B | 75.7% | 13 |
| 274 | MMMU-Pro (vision) O MMMU-Pro vision-only setting. | Multimodal understanding (vision) | 38.3% | 61.9% | 0.98 | 🇺🇸 GPT-5 (high) | 66.9% | 49 |
| 275 | τ³-Bench Airline O τ³-Bench airline domain: tool-augmented customer-service agent with simulated user. | Agent tasks (airline) | 38.4% | 83.4% | 1.37 | 🇨🇳 Kimi-K2.6 Thinking | 85.8% | 17 |
| 276 | longbench-v2 O Next-generation LongBench v2 long-context evaluation benchmark. | Long-context eval | 38.5% | 68.2% | 0.69 | 🇺🇸 Claude Opus 4.8 | 69.1% | 136 |
| 277 | LiveCodeBench O Live coding and execution-based evaluation benchmark (v6 dataset). | Code generation | 38.7% | 94.4%saturated | 1.77 | 🇺🇸 GPT-5.5 | 96.2% | 663 |
| 278 | MRCR v2 O Multi-round multimodal chain-of-reasoning evaluation (v2). | Multimodal reasoning | 38.7% | 90.5%saturated | 1.22 | 🇺🇸 GPT-5.5 | 94.8% | 22 |
| 279 | FD-Bench V1.5 O Full-duplex spoken dialogue benchmark v1.5 (interruption handling, backchannel, turn-taking quality); average score. | Full-duplex speech interaction | 38.8% | 61.7% | 0.86 | 🇺🇸 GPT-Live-1 (low) | 80.1% | 10 |
| 280 | ODinW-13 O Object Detection in the Wild benchmark covering 13 real-world domains. | Object detection (in the wild) | 38.9% | 50.9% | 0.29 | 🇨🇳 Qwen3.6-Plus | 51.8% | 33 |
| 281 | RepoBench O Repository-level code comprehension and reasoning benchmark. | Code understanding | 39.0% | 69.9% | 1.04 | 🇺🇸 Claude Sonnet 4.5 | 83.8% | 16 |
| 282 | τ³-Bench Telecom O τ³-Bench telecom domain: tool-augmented customer-service agent with simulated user (dual-control). | Agent tasks (telecom) | 39.0% | 97.8%saturated | 2.37 | 🇨🇳 GLM-5.1 | 98.7% | 19 |
| 283 | Longform Creative Writing O Longform creative writing evaluation (EQ-Bench). | Creative writing | 39.2% | 84.4% | 0.91 | 🇺🇸 Claude Opus 5 | 86.3% | 145 |
| 284 | Charades-STA O Charades-STA temporal grounding (mIoU). | Video grounding | 39.3% | 62.1% | 1.18 |  🇨🇳 Seed1.5-VL-Thinking | 64.0% | 25 |
| 285 | HealthBench O Comprehensive medical knowledge and clinical reasoning benchmark across specialties and tasks. | Medical QA | 39.4% | 66.7% | 0.45 | 🇺🇸 GPT-5 | 67.2% | 45 |
| 286 | QReCC O Open-domain conversational QA with question rewriting (ChatRAG-Bench subset). | Conversational QA | 39.4% | 51.0% | 0.32 | G Yuan3.0 Ultra | 57.3% | 10 |
| 287 | Arena-Hard V2 Hard Prompt O Chatbot Arena Hard V2 benchmark using the hard prompt win-rate subset. | Chat ability | 39.5% | 75.7% | 1.00 | 🇺🇸 Nemotron-Cascade-2-30B-A3B | 88.2% | 19 |
| 288 | LeetCode Contest O LeetCode weekly/biweekly contest problems (Jul 2023 - Jan 2024) collected by DeepSeek; pass@1. | Code generation (competition) | 39.5% | 31.9% | 2.08 | 🇺🇸 GPT-4 | 48.4% | 15 |
| 289 | SuperGPQA O Harder GPQA variant assessing advanced graduate-level reasoning. | Graduate-level QA | 39.6% | 74.4% | 0.90 | 🇺🇸 Gemini 3.1 Pro | 76.6% | 149 |
| 290 | DUDE O Long-context multimodal understanding benchmark. | Multimodal long-context | 39.8% | 82.8% | 1.07 | 🇺🇸 Claude Opus 4.7 | 84.5% | 11 |
| 291 | Part-Affordance O Part-level affordance grounding benchmark. | Embodied reasoning | 39.9% | 78.8% | 0.72 | G PhysBrain1.5-8B | 84.0% | 13 |
| 292 | MindCube O Spatial navigation benchmark. | Spatial navigation | 39.9% | 83.9% | 0.81 | G ACE-Brain-0.5-8B | 94.2% | 51 |
| 293 | TAU2-Retail O Tool-augmented agent evaluation in retail scenarios (TAU2). | Agent tasks (retail) | 40.1% | 90.4%saturated | 1.51 | 🇺🇸 Claude Opus 4.6 | 91.9% | 114 |
| 294 | Harvey LAB-AA O Harvey's Legal Agent Benchmark as run by Artificial Analysis (LAB-AA): complex legal workflows score. | Legal agent tasks | 40.1% | 92.2%saturated | 1.12 | 🇨🇳 Kimi-K3 | 94.6% | 20 |
| 295 | Nexus (0-shot) O Nexus tool-use benchmark, zero-shot setting. | Tool use | 40.1% | 52.7% | 0.74 | 🇺🇸 Llama 3.1 405B | 58.7% | 15 |
| 296 | JevBench (Benchmark Heaven) O Benchmark Heaven JevBench for typed probabilistic decision models; score is the chance-corrected Intelligence (typed decision accuracy) axis. | Typed decisions | 40.1% | 83.3% | 1.56 | 🇺🇸 GPT-6 Luna (medium) | 96.9% | 25 |
| 297 | CGBench O Cartoon/CG long video question answering benchmark. | Long video QA | 40.3% | 65.2% | 0.75 | G Seed2.0 Lite (0428) | 65.5% | 12 |
| 298 | ChartQA-Pro O Professional-grade chart question answering with diverse chart types and complex reasoning. | Chart understanding (VQA) | 40.4% | 71.1% | 0.80 | 🇺🇸 Claude Mythos 5 | 71.6% | 46 |
| 299 | MathVision O Visual math reasoning benchmark with problems that combine images (charts, diagrams) and text. | Math reasoning (multimodal) | 40.5% | 97.5%saturated | 1.51 | 🇺🇸 Claude Fable 5 | 98.6% | 246 |
| 300 | Bird-SQL O Natural language to SQL generation benchmark. | Text-to-SQL | 41.2% | 56.2% | 0.24 | 🇺🇸 Gemini 2.0 Pro | 59.3% | 34 |
| 301 | MMLongBench-Doc O Evaluates long-context document understanding with mixed text, tables, and figures across multiple pages. | Long-context multimodal documents | 41.4% | 64.4% | 0.58 | 🇨🇳 Qwen3.8-Flash-Next | 67.9% | 52 |
| 302 | DRACO O DRACO deep-research benchmark scored with per-question rubrics. | Deep research | 41.4% | 83.9% | 0.85 | 🇺🇸 Claude Opus 5 | 88.6% | 20 |
| 303 | RefSpatialBench O Reference spatial understanding benchmark covering spatial grounding tasks. | Spatial reasoning | 41.5% | 74.8% | 0.55 | 🇺🇸 GPT-6 Astra | 80.1% | 71 |
| 304 | IFBench O Instruction-following benchmark measuring compliance and adherence. | Instruction following | 41.6% | 85.8% | 0.91 | G Flint | 89.2% | 372 |
| 305 | AIME 2024 O American Invitational Mathematics Examination 2024 problems. | Math (competition) | 41.7% | 95.5%saturated | 3.19 | 🇺🇸 GPT-OSS 120B | 96.6% | 266 |
| 306 | τ-Voice Airline O τ-Voice (Sierra): τ-bench airline customer-service tasks over a live speech-to-speech connection; pass^1 (%). | Voice agents | 41.8% | 72.0% | 0.76 | 🇺🇸 GPT-Live-1 (medium) | 76.7% | 29 |
| 307 | BFCL v4 O BFCL v4 benchmark for functional code correctness and logic. | Code reasoning | 42.0% | 78.3% | 0.83 | 🇨🇳 Qwen3.6-27B | 80.2% | 161 |
| 308 | ZebraLogic O Logical reasoning benchmark assessing complex pattern and rule inference. | Logical reasoning | 42.0% | 97.0%saturated | 2.84 | 🇺🇸 GPT-5 (high) | 98.8% | 147 |
| 309 | RW Search O Real-world search benchmark evaluating retrieval and reasoning. | Agentic search | 42.2% | 81.5% | 0.99 | 🇺🇸 GPT-5.5 | 85.3% | 15 |
| 310 | ViewSpatial-Bench O Multi-perspective spatial localization benchmark (camera vs. human viewpoints). | Spatial reasoning | 42.5% | 59.2% | 0.30 | G ZDTaichu5.0-9B | 62.5% | 20 |
| 311 | MARS-Bench O Instruction-following benchmark with complex tasks. | Instruction following | 42.6% | 86.3% | 1.32 | 🇺🇸 GPT-5.2 (high) | 87.9% | 12 |
| 312 | ERQA O Spatial recognition and reasoning QA benchmark (ERQA). | Spatial reasoning | 42.6% | 82.7% | 0.57 | G Glimpse | 83.8% | 114 |
| 313 | AlpacaEval O Automatic eval using GPT-4 as a judge. | Instruction following | 42.7% | 77.1% | 1.31 | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 99.4% | 51 |
| 314 | BankerToolBench O BankerToolBench: tool-using agent tasks modeled on investment-banking workflows, scored with dataset rubrics. | Investment banking agent tasks | 42.8% | 79.7% | 0.74 | 🇺🇸 Claude Opus 5 | 81.9% | 14 |
| 315 | JCulture O Japanese culture knowledge QA benchmark (as reported by Liquid AI). | Japanese culture knowledge | 42.9% | 32.9% | 0.59 |  🇺🇸 LFM2.5 1.2B JP 202606 | 35.8% | 10 |
| 316 | Arena-Hard V2 O Updated Arena-Hard v2 prompts on Chatbot Arena. | Chat ability | 43.0% | 88.4% | 1.28 | 🇺🇸 GPT-OSS 120B | 90.3% | 104 |
| 317 | Arena-Hard V2 Creative Writing O Chatbot Arena Hard V2 creative writing win-rate subset. | Creative writing | 43.0% | 89.2% | 1.65 | 🇺🇸 Gemini 3 Pro | 93.6% | 18 |
| 318 | JamC-QA O Japanese-specific knowledge multiple-choice QA (culture, customs, regional, law, medicine) from the Swallow evaluation suite. | Japanese knowledge QA | 43.3% | 63.9% | 0.69 | 🇺🇸 GPT-5 mini | 70.1% | 31 |
| 319 | SimpleVQA O Lightweight visual question answering set with everyday scenes. | General VQA | 43.4% | 78.3% | 0.67 | 🇨🇳 Qwen3.7-Plus | 81.7% | 100 |
| 320 | AdvancedIF O Meta's AdvancedIF rubric-based benchmark for complex, multi-turn and system-prompt instruction following. | Instruction following | 44.1% | 77.5% | 0.79 | 🇺🇸 Gemini 3.1 Pro | 83.9% | 12 |
| 321 | Codeforces (Percentile) O Competitive programming performance on Codeforces problems, reported as the percentile of human competitors outperformed. | Competitive programming | 44.1% | 92.3%saturated | 2.70 | G Ring-1T-preview | 94.7% | 27 |
| 322 | OCRReasoning O OCR reasoning benchmark combining text extraction with multi-step reasoning over documents. | OCR reasoning | 44.1% | 67.1% | 0.60 | 🇺🇸 Gemini 2.5 Pro | 70.8% | 16 |
| 323 | OpenEQA O Open-vocabulary embodied question answering over episodic memory of indoor scenes. | Embodied QA | 44.2% | 56.8% | 0.66 | 🇺🇸 GPT-5 | 64.4% | 12 |
| 324 | COLLIE O Comprehensive instruction-following evaluation suite. | Instruction following | 44.7% | 97.6%saturated | 1.96 | 🇺🇸 GPT-5 | 99.0% | 24 |
| 325 | xBench-DeepSearch O Evaluates multi-hop deep research workflows on xBench DeepSearch tasks. | Agentic research | 44.7% | 77.1% | 0.77 | 🇺🇸 GPT-5 (high) | 77.9% | 42 |
| 326 | MicroVQA O Visual question answering benchmark for biological microscopy images. | Biological microscopy | 44.9% | 69.4% | 0.55 | 🇺🇸 Gemini 3.1 Pro | 71.0% | 18 |
| 327 | OlympiadBench O Advanced mathematics olympiad-style problem benchmark. | Math (olympiad) | 45.4% | 76.5% | 2.57 | 🇨🇳 Qwen3-30B-A3B-Instruct-2507 | 77.6% | 80 |
| 328 | VideoHolmes O Video question answering benchmark focused on detective-style clues. | Video QA | 45.5% | 67.5% | 0.59 | G Seed2.1 Pro | 68.2% | 22 |
| 329 | BigCodeBench O BigCodeBench evaluates large language models on practical code generation tasks with unit-test verification. | Code Generation | 45.5% | 64.3% | 0.54 | G MiMo V2 Flash Base | 70.1% | 233 |
| 330 | CharXiv (RQ) O Scientific chart/table reasoning questions from arXiv PDFs. | Chart reasoning (PDF) | 45.7% | 91.8%saturated | 1.07 | 🇺🇸 Claude Fable 5 | 93.5% | 274 |
| 331 | LAB-Bench 2 O LABBench2: biology real-world research tasks (macro-average over 11 sub-tasks: DbQA2, FigQA2-img/pdf, LitQA3, PatentQA, ProtocolQA2, SourceQuality, SuppQA2, TableQA2-img/pdf, TrialQA). | Biology research | 45.7% | 85.3% | 0.79 | 🇺🇸 Gemini 4 Argon | 88.8% | 11 |
| 332 | LongBench O Long-context understanding across tasks. | Long-context eval | 45.8% | 57.7% | 0.51 | 🇨🇳 Qwen3.6-27B | 60.9% | 37 |
| 333 | AA-Index O Comprehensive QA index across diverse domains. | Multi-domain QA | 46.2% | 62.5% | 0.31 | 🇺🇸 Grok 4 | 73.2% | 11 |
| 334 | HMMT Nov 2025 O Harvard–MIT Mathematics Tournament November 2025 problems. | Math (competition) | 46.3% | 98.5%saturated | 1.99 | 🇺🇸 GPT-5.2 (high) | 100.0% | 81 |
| 335 | All-Angles Bench O All-Angles benchmark for spatial recognition and 3D perception. | Spatial perception | 46.5% | 59.3% | 0.56 | 🇨🇳 Qwen3.6-35B-A3B | 64.0% | 17 |
| 336 | JGPQA O Japanese translation of GPQA graduate-level science QA. | Graduate-level QA (Japanese) | 46.6% | 71.3% | 0.91 | 🇺🇸 GPT-5 mini | 75.0% | 41 |
| 337 | Odyssey Math O Odyssey multi-step math benchmark. | Math reasoning | 46.6% | 49.6% | 1.16 | 🇨🇳 DeepSeek-Coder V2 Instruct | 53.7% | 14 |
| 338 | NOVA-63 O Multilingual evaluation benchmark covering 63 languages. | Multilingual evaluation | 46.6% | 58.9% | 0.31 | 🇨🇳 Qwen3.5-397B-A17B | 59.1% | 29 |
| 339 | VSI-Bench O Visual spatial intelligence benchmark covering 3D reasoning and spatial inference tasks. | Spatial intelligence | 46.6% | 72.8% | 0.32 | 🇺🇸 Molmo2-ER | 74.5% | 55 |
| 340 | OSWorld-G O OSWorld-G center accuracy (no_refusal). | GUI agents | 46.7% | 83.6% | 0.86 | G Nex-N2.5-Pro | 87.4% | 86 |
| 341 | RoboVQA O Robotics video question answering benchmark. | Embodied reasoning | 46.9% | 60.9% | 0.20 | G PhysBrain1.5-2B | 62.9% | 13 |
| 342 | MMMU PRO O Professional/advanced subset of MMMU for multimodal reasoning. | Multimodal understanding (hard) | 47.0% | 83.7% | 0.95 | 🇺🇸 Gemini 3.5 Flash | 84.6% | 226 |
| 343 | Habermas Machine (decision) O Predict group preference among candidate consensus statements; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Deliberation / preference | 47.0% | 66.1% | 0.28 | G Clef-Flash | 71.8% | 38 |
| 344 | VoiceBench CommonEval O VoiceBench evaluation on CommonEval. | VoiceBench | 47.2% | 90.6%saturated | 3.21 | 🇨🇳 Qwen3-Omni-Flash-Instruct | 91.0% | 22 |
| 345 | RealKIE-FCC Verified O Human-verified version of the RealKIE FCC document key-information-extraction benchmark (ALNS). | Key information extraction | 47.4% | 64.6% | 0.51 | 🇺🇸 Gemini 3 Pro Preview | 71.7% | 14 |
| 346 | LVBench O Long video understanding benchmark (LVBench). | Video understanding | 47.5% | 87.2% | 0.77 | 🇺🇸 Gemini 4 Argon | 91.7% | 101 |
| 347 | BuzzBench O A humour analysis benchmark. | Humor analysis | 47.6% | 69.7% | 1.17 | 🇺🇸 Gemini 2.5 Pro | 71.1% | 28 |
| 348 | Q-Spatial-Bench O Quantitative spatial reasoning (metric distances/sizes) benchmark for VLMs. | Embodied reasoning | 47.6% | 79.0% | 0.69 | 🇺🇸 Gemini 3.6 Flash | 87.1% | 13 |
| 349 | QIVD O Qualcomm Interactive Video Dataset: situated audio-visual question answering on live-style video with speech questions; accuracy. | Audio-visual QA | 47.8% | 65.1% | 0.47 | 🇺🇸 Gemini 3.8 Flash | 69.1% | 12 |
| 350 | IntPhys 2 O Intuitive physics reasoning benchmark. | Intuitive physics | 47.9% | 61.9% | 0.35 | G Muse Spark Open | 68.3% | 18 |
| 351 | FD-Bench V3 Pass@1 O Full-Duplex-Bench v3: tool-calling task success (pass@1, %) in full-duplex spoken dialogue. | Full-duplex speech + tool use | 48.0% | 66.6% | 0.75 | 🇺🇸 GPT-Live-1 (medium) | 87.0% | 13 |
| 352 | GAIA (Text Only) O GAIA benchmark subset restricted to text-only inputs. | General AI tasks | 48.1% | 93.8%saturated | 1.06 | G XYZ-Aquila-mini | 97.1% | 66 |
| 353 | VideoEval-Pro O Long video understanding benchmark with open-ended short-answer questions. | Video understanding | 48.1% | 71.0% | 0.85 | 🇺🇸 Gemini 2.5 Pro | 78.4% | 26 |
| 354 | WeMath O Math reasoning benchmark spanning diverse curricula and difficulty levels. | Math reasoning | 48.6% | 86.5% | 1.28 | 🇨🇳 Qwen3.6-Plus | 89.0% | 49 |
| 355 | ARKitScenes O ARKitScenes benchmark for assessing 3D scene reconstruction and understanding from mixed reality captures. | 3D scene understanding | 48.6% | 57.4% | 0.20 | 🇨🇳 Qwen2.5-VL 72B Instruct | 61.5% | 11 |
| 356 | OCRBenchV2 (CN) O OCRBenchV2 Chinese subset assessing OCR performance on Chinese-language documents. | OCR (Chinese) | 48.9% | 67.6% | 0.30 | 🇨🇳 Qwen3.8-Max | 68.3% | 61 |
| 357 | PhyX O Multimodal physics reasoning benchmark (PhyX). | Physics reasoning (multimodal) | 49.0% | 78.7% | 0.64 | 🇨🇳 Qwen3.8-Max | 83.5% | 11 |
| 358 | VibeEval O VLM aesthetic evaluation with GPT scores. | Aesthetic/visual quality | 49.1% | 67.8% | 0.98 | 🇺🇸 Gemini 2.5 Pro | 76.4% | 11 |
| 359 | LIFEBench O Length-based instruction-following evaluation benchmark. | Instruction following | 49.2% | 73.4% | 0.61 | G Ling 3.0 Flash | 77.3% | 21 |
| 360 | AlpacaEval 2.0 O Updated AlpacaEval with improved prompts and judging. | Instruction following | 49.3% | 82.7% | 1.42 | 🇨🇳 DeepSeek R1 | 87.6% | 121 |
| 361 | Design2Code O Translating UI designs into code. | Coding (UI) | 49.4% | 92.2%saturated | 1.98 | 🇨🇳 GLM-5V-Turbo | 94.8% | 14 |
| 362 | VideoSimpleQA O Simple question answering over short videos. | Video QA | 49.5% | 73.7% | 0.66 | G Seed2.1 Pro | 76.4% | 13 |
| 363 | OVBench O Open-vocabulary benchmark for streaming video understanding. | Open-vocabulary streaming | 49.6% | 67.9% | 0.43 | G Seed2.1 Pro | 70.0% | 17 |
| 364 | FORTRESS (Adversarial) O Scale AI FORTRESS adversarial prompts (CBRNE, political violence, crime); safety score. | Safety | 49.7% | 88.0% | 0.70 | 🇺🇸 Claude Fable 5 | 96.0% | 17 |
| 365 | TAU1-Retail O Tool-augmented agent evaluation in retail scenarios (TAU1). | Agent tasks (retail) | 49.9% | 70.6% | 1.08 | G openPangu-R-72B-2512 Slow Thinking | 73.0% | 13 |
| 366 | KMMLU-Pro O Korean Multilingual Massive Multitask Language Understanding Pro | Multilingual knowledge | 50.1% | 79.5% | 0.96 | 🇨🇳 DeepSeek V4-Pro-Thinking (max) | 80.5% | 84 |
| 367 | LiveBench O Continually updated capability benchmark across diverse tasks. | General capability | 50.2% | 79.2% | 1.23 | 🇺🇸 Gemini 2.5 Pro | 82.4% | 164 |
| 368 | CollegeMath O College-level mathematics problems (MathScale) covering algebra, calculus, linear algebra, probability, etc. | College-level math | 50.4% | 48.3% | 1.14 | 🇨🇳 Qwen2.5-Math-72B-Instruct | 49.5% | 24 |
| 369 | LogicVista O Visual logic and pattern reasoning tasks requiring compositional and spatial understanding. | Visual logical reasoning | 50.5% | 86.8% | 0.88 | 🇨🇳 Qwen3.8-Max | 91.9% | 65 |
| 370 | DocMath O Math reasoning on document-based problems. | Document math | 50.5% | 66.9% | 1.01 | 🇨🇳 Qwen3.5-35B-A3B | 68.9% | 48 |
| 371 | 3DSRBench O 3D spatial reasoning benchmark for large multimodal models (height, location, orientation, multi-object relations). | Spatial reasoning | 50.6% | 65.0% | 0.32 | 🇺🇸 Gemini 3 Pro | 68.9% | 20 |
| 372 | XNLI O Cross-lingual natural language inference benchmark (MultiNLI translated into 15 languages). | Cross-lingual NLI | 50.7% | 75.9% | 0.31 | G Decider 4B (Mapika) | 88.6% | 24 |
| 373 | MATH Level 5 O Level 5 subset of the MATH benchmark emphasizing the hardest competition-style problems. | Math (competition) | 50.8% | 77.1% | 2.10 | 🇺🇸 o1 Mini (2024-09-12) | 89.3% | 51 |
| 374 | Multi-Image QA Average O Aggregate score over multi-image visual question answering tasks. | Multi-image QA (aggregate) | 50.9% | 72.6% | 0.88 | 🇺🇸 Gemini 3 Pro | 81.9% | 19 |
| 375 | NaturalCodeBench O NaturalCodeBench (NCB): real-world coding prompts in Python and Java, English and Chinese. | Code generation | 51.1% | 50.7% | 1.36 | 🇺🇸 GPT-4 Turbo (2024-04-09) | 53.8% | 15 |
| 376 | FullStackBench O End-to-end web/app development tasks and evaluation. | Full-stack development | 51.1% | 70.9% | 0.62 | 🇺🇸 Claude Opus 4.5 | 72.3% | 56 |
| 377 | MMIU O Multi-image understanding benchmark evaluating cross-image reasoning. | Multi-image understanding | 51.1% | 67.8% | 0.51 | 🇺🇸 Gemini 3 Pro | 72.1% | 40 |
| 378 | KMMLU-Redux O Redux variant of KMMLU benchmark | Multilingual knowledge | 51.2% | 79.3% | 1.51 | 🇺🇸 o1 | 81.1% | 53 |
| 379 | OCRBenchV2 (EN) O OCRBenchV2 English subset evaluating OCR accuracy on English documents and layouts. | OCR (English) | 51.2% | 72.5% | 0.36 | 🇨🇳 Qwen3.8-Max | 74.2% | 78 |
| 380 | PMC-VQA O PubMed Central visual question answering benchmark for biomedical images. | Medical VQA | 51.5% | 68.9% | 0.28 | G MedMO-4B-Next | 75.7% | 35 |
| 381 | TAU2-Bench Retail (Verified) O Human-validated revision of the tau2-bench retail domain (amazon-agi/tau2-bench-verified). | Agent tasks | 51.6% | 77.9% | 1.04 | 🇺🇸 GPT-5 | 78.3% | 11 |
| 382 | KOR-Bench O Comprehensive reasoning benchmark spanning diverse domains and cognitive skills. | Reasoning | 51.7% | 75.8% | 1.00 | 🇺🇸 GPT-5 (high) | 77.4% | 26 |
| 383 | JudgeMark v2.1 O A benchmark measuring LLM judging ability. | LLM judging ability | 51.8% | 88.9% | 1.16 | 🇺🇸 Claude Sonnet 4.6 | 94.1% | 49 |
| 384 | GPQA O Graduate-level question answering evaluating advanced reasoning. | Graduate-level QA | 51.9% | 91.2%saturated | 1.12 | 🇺🇸 GPT-5.2 Thinking | 92.4% | 323 |
| 385 | MASK O Model behavior safety assessment via red-teaming scenarios. | Safety / red teaming | 52.0% | 95.7%saturated | 1.05 | 🇺🇸 Claude Opus 4.6 | 96.3% | 79 |
| 386 | Claw-Eval O Benchmark of end-to-end OpenClaw-style personal-agent tasks; overall (EN) score. | Agentic assistant tasks | 52.2% | 83.0% | 0.65 | 🇨🇳 GLM-5.1 | 84.4% | 59 |
| 387 | MathVerse-Vision O Multi-image visual mathematical reasoning tasks from the MathVerse ecosystem. | Math reasoning (multimodal) | 52.2% | 87.1% | 1.13 | G Seed2.1 Pro | 89.7% | 21 |
| 388 | TheoremQA O Theorem-driven question answering in math, physics, EE/CS and finance. | Theorem-driven STEM QA | 52.2% | 43.5% | 1.77 | 🇨🇳 Qwen2 72B Instruct | 44.4% | 87 |
| 389 | VLMs are Blind O Evaluates failure modes of VLMs in grounding and perception tasks. | Visual grounding robustness | 52.2% | 81.4% | 1.19 | 🇨🇳 ERNIE 5.0 | 91.4% | 16 |
| 390 | Minerva Math O Advanced quantitative reasoning set inspired by the Minerva benchmark for STEM problem solving. | University-level math | 52.4% | 96.3%saturated | 1.38 | 🇨🇳 Qwen3 235B A22B Thinking | 98.0% | 67 |
| 391 | NaturalQuestions O Google NQ; real user questions with long/short answers. | Open-domain QA | 52.5% | 52.1% | 1.20 |  🇫🇷 Mixtral-8x22B-Instruct-v0.1 | 54.9% | 124 |
| 392 | MultiPL-E MBPP O MultiPL-E variant of MBPP tasks. | Code generation (multilingual) | 52.6% | 60.2% | 0.79 | 🇺🇸 Llama 3.1 405B | 65.7% | 22 |
| 393 | xBench-DeepSearch (2025.05) O xBench DeepSearch benchmark May 2025 snapshot. | Agentic research | 52.8% | 80.7% | 0.89 | G Step-3.5 Flash 20260204 | 83.7% | 31 |
| 394 | MathVerse-mini O Compact MathVerse split focusing on single-image math puzzles and visual reasoning. | Math reasoning (multimodal) | 53.3% | 84.8% | 1.15 |  🇨🇳 MiniCPM-V 4.7 35B-A3B | 86.5% | 69 |
| 395 | PIOBench O Pointing / interaction-object grounding benchmark for embodied VLMs. | Embodied reasoning | 53.3% | 73.1% | 0.48 | 🇺🇸 Gemini 3.6 Flash | 80.9% | 13 |
| 396 | JFBench O Japanese instruction-following benchmark. | Instruction following (Japanese) | 53.4% | 38.7% | 0.84 | 🇺🇸 LFM2.5 1.2B JP 202606 | 54.8% | 10 |
| 397 | MMMU Pro (with tools) O MMMU-Pro benchmark evaluated with tool access. | Multimodal understanding (with tools) | 53.4% | 84.6% | 0.77 | 🇺🇸 Claude Fable 5 | 86.5% | 19 |
| 398 | GraphWalks BFS O Graph traversal/GraphWalks benchmark (BFS variant) for long-context reasoning. | Long-context reasoning | 53.5% | 96.7%saturated | 1.23 | 🇺🇸 Gemini 4 Argon | 99.7% | 18 |
| 399 | LiveCodeBench v3 O LiveCodeBench v3 snapshot measuring pass rates on streaming coding tasks. | Code generation | 53.5% | 87.4% | 2.43 | 🇨🇳 Qwen3-32B | 90.2% | 45 |
| 400 | CharXiv (RQ, With Tools) O CharXiv reasoning questions evaluated with tool access. | Chart understanding | 53.6% | 93.3%saturated | 1.03 | 🇺🇸 Claude Mythos 5 | 93.5% | 33 |
| 401 | WebWalkerQA O WebWalker tasks evaluating autonomous browsing question answering performance. | Web agents | 53.9% | 71.6% | 0.68 | G SMTL-30B | 76.5% | 15 |
| 402 | MultiPL-E HumanEval O MultiPL-E variant of HumanEval tasks. | Code generation (multilingual) | 54.2% | 66.4% | 1.36 | 🇺🇸 Llama 3.1 405B | 75.2% | 22 |
| 403 | AIME 2026 I O American Invitational Mathematics Examination 2026 I problems. | Math (competition) | 54.3% | 99.8%saturated | 2.13 | 🇺🇸 GPT-5.5 | 100.0% | 170 |
| 404 | MathVision (With Tools) O MathVision visual math benchmark evaluated with a Python code-interpreter tool. | Math reasoning (multimodal) | 54.3% | 97.6%saturated | 1.65 | 🇺🇸 Claude Fable 5 | 98.6% | 12 |
| 405 | Flame-React-Eval O Front-end React coding tasks and evaluation. | Frontend coding | 54.3% | 90.0%saturated | 1.84 | 🇺🇸 Claude Opus 4.6 | 98.8% | 10 |
| 406 | JHumanEval+ O Japanese-prompt HumanEval+ code generation benchmark. | Code (Japanese) | 54.4% | 42.4% | 0.91 | G Granite-4.0-1B | 51.2% | 10 |
| 407 | DiscoX O DiscoX benchmark for agentic writing and reasoning. | Agentic writing | 54.5% | 78.8% | 0.76 | G Seed2.0 Pro | 82.0% | 12 |
| 408 | DynaMath O Dynamic/video-based mathematical reasoning evaluating temporal and visual understanding. | Math reasoning (video) | 54.5% | 86.9% | 0.90 | 🇨🇳 Qwen3.6-Plus | 88.0% | 68 |
| 409 | Where2Place O Spatial free-space pointing benchmark: predict points for placement locations described in language. | Embodied spatial understanding | 54.6% | 76.2% | 0.29 | G ACE-Brain-0.6-9B | 81.4% | 29 |
| 410 | PixMo-Points O Pointing accuracy on the PixMo-Points evaluation set. | Embodied reasoning | 54.6% | 74.2% | 0.27 | 🇺🇸 GPT-6 Astra | 75.2% | 17 |
| 411 | HiddenMath O Mathematical reasoning benchmark referenced in recent model cards. | Math reasoning | 54.8% | 59.1% | 1.42 | 🇺🇸 Gemini 2.0 Pro | 65.2% | 11 |
| 412 | GPQA-diamond O Hard subset of GPQA (diamond level). | Graduate-level QA | 54.8% | 95.4%saturated | 1.25 | 🇺🇸 GPT-6 Astra | 96.0% | 631 |
| 413 | COSMOS (Cosmos-Reason benchmark) O NVIDIA Cosmos-Reason embodied physical-reasoning benchmark. | Embodied reasoning | 54.9% | 71.6% | 0.40 | 🇺🇸 GPT-6 Astra | 75.7% | 13 |
| 414 | OVOBench O Streaming video QA benchmark with open-vocabulary queries. | Streaming video QA | 55.0% | 77.8% | 0.59 | G Seed2.1 Pro | 80.7% | 25 |
| 415 | MMVU O Multimodal video understanding benchmark (MMVU). | Video understanding | 55.5% | 83.5% | 0.68 | 🇺🇸 GPT-5 (high) | 87.3% | 69 |
| 416 | MMLU-ProX (Japanese) O Japanese subset of MMLU-ProX (JMMLU-ProX). | Multi-domain knowledge (Japanese) | 56.0% | 79.0% | 1.41 | 🇨🇳 Qwen3-235B-A22B-Thinking-2507 | 81.9% | 41 |
| 417 | TVBench O Benchmark for TV show video comprehension and QA. | TV comprehension | 56.1% | 81.7% | 0.57 | 🇺🇸 GPT-5.6 Sol | 83.2% | 25 |
| 418 | Showdown O Success rate on the Showdown UI interaction benchmark. | GUI agents | 56.2% | 77.8% | 1.00 | G Holo2-235B-A22B | 80.4% | 22 |
| 419 | MathVerse O Visual math reasoning benchmark combining images and text across diverse mathematical tasks. | Math reasoning (multimodal) | 56.2% | 86.2% | 1.33 | 🇺🇸 Gemini 3 Pro | 91.6% | 19 |
| 420 | VL-RewardBench O Reward alignment benchmark for VLMs. | Reward modeling (VL) | 56.5% | 63.9% | 0.46 | 🇺🇸 Claude 3.7 Sonnet | 67.4% | 14 |
| 421 | MMMU (test) O MMMU test-set accuracy (multi-discipline college-level multimodal questions). | Multimodal understanding | 56.7% | 57.0% | 0.62 | 🇨🇳 InternVL2.5-78B | 61.8% | 30 |
| 422 | HallusionBench O Benchmark for evaluating hallucination tendencies in multimodal LLMs. | Multimodal hallucination | 57.0% | 74.4% | 0.35 | 🇨🇳 Qwen3-VL 32B Thinking | 76.6% | 163 |
| 423 | Inverse IFEval O Inverse instruction-following evaluation. | Instruction following (inverse) | 57.1% | 79.9% | 0.71 | 🇺🇸 Gemini 3 Flash | 80.9% | 12 |
| 424 | RoboSpatialHome O RoboSpatialHome benchmark for embodied spatial reasoning in domestic environments. | Embodied spatial understanding | 57.4% | 75.8% | 0.28 | 🇺🇸 Gemini 4 Argon | 80.9% | 49 |
| 425 | MME-RealWorld (cn) O MME-RealWorld Chinese split. | Real-world perception (CN) | 57.5% | 70.4% | 0.46 | 🇺🇸 Gemini 3 Flash | 72.6% | 25 |
| 426 | MotionBench O Video motion and temporal reasoning benchmark. | Video motion understanding | 57.5% | 73.6% | 0.37 | G Seed2.0 Pro | 75.2% | 55 |
| 427 | GraphWalks BFS 256K O GraphWalks BFS long-context graph traversal at 256K context (F1). | Long-context reasoning | 57.6% | 89.7% | 0.68 | G Spark-X2.6-4B | 92.2% | 17 |
| 428 | SimpleVQA (With Tools) O SimpleVQA visual factuality benchmark evaluated with tool access. | Visual factuality | 57.6% | 78.5% | 0.40 | G Step 3.7 Flash | 79.2% | 12 |
| 429 | DeepSearchQA (F1) O DeepSearchQA multi-step web search benchmark, F1-score metric (the existing deepsearchqa entry is used for the accuracy metric). | Deep web search QA | 58.0% | 93.8%saturated | 1.11 | 🇨🇳 Kimi-K3 | 95.0% | 22 |
| 430 | MM-IFEval O Instruction-following benchmark assessing multimodal obedience to complex prompts. | Multimodal instruction following | 58.1% | 76.0% | 0.69 | 🇺🇸 GPT-5 | 83.1% | 42 |
| 431 | WebVoyager O Web navigation and interaction tasks for LLM agents. | Web agents | 58.2% | 91.2%saturated | 0.98 | G UI-Venus-2 27B | 93.4% | 36 |
| 432 | LongMemEval O Evaluates LLMs' long-term memory and retrieval over extended contexts and multi-session interactions. | Long-term memory | 58.2% | 70.3% | 0.71 | G GoLongRL-30B-A3B | 75.2% | 11 |
| 433 | BFCL v3 (Japanese) O Japanese translation of BFCL v3 function-calling benchmark. | Function calling (Japanese) | 58.3% | 55.3% | 1.39 | 🇺🇸 NVIDIA-Nemotron-Nano-9B-v2-Japanese | 64.9% | 12 |
| 434 | CaseLaw v2 O U.S. case law benchmark evaluating legal reasoning and judgment over court opinions. | Legal reasoning | 58.4% | 77.3% | 0.20 | 🇺🇸 Grok 4.3 | 79.3% | 78 |
| 435 | CCBench O MMBench-family benchmark on Chinese culture (painting, calligraphy, cuisine, history...). | Chinese culture VQA | 58.7% | 63.8% | 0.35 | G Step-1V 32K | 71.2% | 11 |
| 436 | GSM8K (decision) O GSM8K posed as choices among candidate answers; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Math (grade-school) | 58.7% | 81.7% | 1.11 | G vLLM-SR d3 | 85.2% | 38 |
| 437 | τ³-Bench Retail O τ³-Bench retail domain: tool-augmented customer-service agent with simulated user. | Agent tasks (retail) | 58.7% | 87.6% | 0.91 | 🇨🇳 DeepSeek V4-Flash | 89.1% | 17 |
| 438 | FACTS Grounding O Grounded factuality benchmark evaluating model alignment with source facts. | Grounding / factuality | 58.8% | 86.5% | 0.29 | 🇨🇳 Kimi K2 Instruct | 88.5% | 81 |
| 439 | Point-Bench O Benchmark for pointing and counting objects in images. | Pointing and counting | 59.0% | 78.2% | 0.24 | 🇺🇸 Gemini 2.5 Pro | 85.5% | 49 |
| 440 | MME-RealWorld (en) O MME-RealWorld English split. | Real-world perception (EN) | 59.2% | 72.4% | 0.45 | 🇺🇸 Gemini 3 Flash | 74.9% | 48 |
| 441 | BLINK O Evaluates visual-language grounding and reference resolution to reduce hallucinations. | Multimodal grounding | 59.2% | 87.9% | 0.53 | 🇺🇸 Gemini 3.6 Flash | 89.2% | 169 |
| 442 | General Tool Use O General tool-use benchmark covering web and API tasks. | Tool use | 59.4% | 76.1% | 0.57 | 🇺🇸 Claude Opus 4.5 | 78.9% | 18 |
| 443 | CSimpleQA O Chinese SimpleQA benchmark variant (short factual questions). | QA | 59.5% | 77.3% | 0.56 | G Ling 2.5 1T | 79.0% | 36 |
| 444 | BFCL v3 O Benchmark for functional code correctness and logic (v3). | Code reasoning | 59.7% | 76.3% | 0.64 | 🇨🇳 GLM 4.5 | 77.8% | 129 |
| 445 | WildBench V2 O WildBench V2 human preference benchmark for instruction following and helpfulness. | Instruction following | 59.9% | 66.4% | 0.66 | 🇫🇷 Ministral 3 14B Instruct | 68.5% | 20 |
| 446 | M-IFEval (Japanese) O Japanese split of M-IFEval multilingual instruction-following benchmark. | Instruction following (Japanese) | 59.9% | 60.8% | 1.02 | 🇺🇸 LFM2.5 1.2B JP 202606 | 79.1% | 15 |
| 447 | MUIRBENCH O Evaluates multimodal understanding robustness and reliability. | Multimodal robustness | 60.1% | 80.7% | 0.89 | 🇺🇸 Gemini 3 Pro | 86.1% | 80 |
| 448 | LiveSports-3K O Live sports video understanding benchmark (3K). | Sports video | 60.3% | 81.9% | 0.65 | 🇺🇸 Gemini 3.1 Pro | 89.0% | 13 |
| 449 | VideoMMMU O Video-based extension of MMMU evaluating temporal multimodal reasoning and perception across disciplines. | Multimodal video understanding | 60.3% | 88.2% | 0.89 | 🇨🇳 Qwen3.8-Max | 88.7% | 93 |
| 450 | TruthfulQA O Measures whether a model imitates human falsehoods (truthfulness). | Truthfulness / hallucination | 60.7% | 70.3% | 0.43 | G SOLAR-10.7B-Instruct-v1.0 | 71.4% | 169 |
| 451 | OpenBookQA O Open-book multiple choice science questions with supporting facts. | Science QA | 60.7% | 94.9%saturated | 0.20 |  🇺🇸 Hermes 4.3 36B Pyche | 96.6% | 108 |
| 452 | RACE-High O RACE reading-comprehension questions from Chinese high-school English exams. | Reading comprehension | 60.8% | 83.2% | 0.20 | G Orion-14B-Base | 91.3% | 21 |
| 453 | PinchBench O Benchmark of OpenClaw agent task success across real-world personal-assistant tasks. | Agentic assistant tasks | 60.9% | 91.9%saturated | 0.97 | G KAT-Coder-V2.5-Dev | 93.4% | 47 |
| 454 | EvalPerf O Measures performance of LLM code evaluation, including runtime, memory, and efficiency metrics. | Code evaluation performance | 61.2% | 93.9%saturated | 2.42 | 🇺🇸 GPT-4o (2024-08-06) | 100.0% | 34 |
| 455 | ToolSandbox O Apple's stateful, conversational, interactive tool-use benchmark with simulated user. | Tool use | 61.2% | 77.1% | 0.97 | 🇺🇸 LFM2.5 24B A2B | 80.7% | 12 |
| 456 | MedMCQA O Indian medical entrance exam (AIIMS/NEET PG) multiple-choice QA. | Medical exams | 61.2% | 66.0% | 0.56 | 🇺🇸 GPT-4 | 72.5% | 27 |
| 457 | Humicroedit (decision) O Pick the funnier micro-edited headline; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Humor judgement | 61.4% | 68.3% | 0.28 | G Clef-Flash | 75.1% | 38 |
| 458 | SysBench O System prompt understanding and adherence benchmark. | System prompts | 61.4% | 93.4%saturated | 1.46 | 🇺🇸 Claude Sonnet 4.6 | 94.9% | 17 |
| 459 | LongVideoBench O Long video understanding and QA benchmark. | Long video QA | 61.5% | 80.3% | 0.47 | G Seed2.1 Turbo | 80.6% | 87 |
| 460 | MIRB O Multi-Image Relational Benchmark (average over perception, visual world knowledge, reasoning, multi-hop). | Multi-image reasoning | 61.6% | 58.6% | 0.84 | 🇨🇳 InternVL2.5-38B | 61.2% | 19 |
| 461 | CorpusQA O Question answering over large text corpora. | Long-context QA | 61.6% | 80.1% | 0.20 | 🇺🇸 GPT-5 | 81.6% | 19 |
| 462 | cfcolor (decision) O Color-name / color judgement decisions; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Color perception | 61.7% | 66.5% | 0.25 | G pplx-decider-v1.2 | 68.2% | 38 |
| 463 | MuSR O Multistep Soft Reasoning. | Reasoning | 61.9% | 83.7% | 0.63 | G Clef-Flash | 86.0% | 94 |
| 464 | MMLU-ProX O Cross-lingual and robust variant of MMLU-Pro. | Multi-domain knowledge | 62.0% | 86.6% | 0.82 | 🇺🇸 Gemini 3 Pro | 87.7% | 58 |
| 465 | Multi-IFEval O Multi-task variant of instruction-following evaluation. | Instruction following (multi-task) | 62.1% | 80.2% | 1.03 | 🇺🇸 Llama 3.3 70B | 88.7% | 15 |
| 466 | KMMLU O Korean Massive Multitask Language Understanding benchmark. | Korean knowledge | 62.2% | 73.0% | 0.78 | 🇨🇳 DeepSeek V3.1 | 78.7% | 29 |
| 467 | LingoQA O Question answering benchmark for autonomous driving scene understanding. | Driving scene QA | 62.3% | 82.5% | 0.42 | 🇨🇳 Qwen3.8-Max | 84.8% | 38 |
| 468 | SAT O Spatial Aptitude Training benchmark of static and dynamic spatial reasoning questions for VLMs. | Spatial reasoning | 62.4% | 88.4% | 0.85 | 🇺🇸 GPT-6 Astra | 96.7% | 24 |
| 469 | AceBench O Industry-focused benchmark assessing domain QA and reasoning. | Industry QA | 62.8% | 82.6% | 0.90 | 🇨🇳 ERNIE 5.0 | 87.7% | 17 |
| 470 | VoiceBench IFEval O VoiceBench instruction-following evaluation (IFEval). | VoiceBench | 62.9% | 83.5% | 1.36 | 🇺🇸 Gemini 2.5 Pro | 85.7% | 24 |
| 471 | BRuMo25 O BruMo 2025 olympiad-style mathematics benchmark. | Math competition | 63.1% | 96.8%saturated | 2.12 | 🇨🇳 Kimi-K2.5 Thinking | 98.3% | 28 |
| 472 | PixmoCount O Counting objects/instances in images (PixmoCount). | Visual counting | 63.4% | 89.5% | 0.59 | G Eagle2.5-8B | 90.2% | 71 |
| 473 | MMMU O Multi-discipline multimodal understanding benchmark. | Multimodal understanding | 63.5% | 86.1% | 0.76 | 🇺🇸 Gemini 3 Pro | 87.2% | 317 |
| 474 | EgoSchema O EgoSchema validation accuracy. | Egocentric video QA | 63.6% | 78.4% | 0.54 | 🇺🇸 GPT-5.2 | 81.2% | 70 |
| 475 | INCLUDE O Evaluates inclusive language use and bias mitigation in model outputs. | Inclusiveness / bias | 63.7% | 87.6% | 0.88 | 🇺🇸 Gemini 3 Pro | 90.5% | 67 |
| 476 | VoiceBench MMSU (Audio) O Audio reasoning MMSU results. | Audio reasoning | 63.8% | 84.5% | 0.56 | G StepAudio 3 Realtime | 90.6% | 26 |
| 477 | AgentDojo O Interactive evaluation suite for autonomous agents across tools and tasks. | Agent evaluation | 63.8% | 81.0% | 1.96 | 🇺🇸 Claude 3.7 Sonnet | 88.7% | 21 |
| 478 | MMLU-Pro O Harder successor to MMLU with more challenging questions. | Multi-domain knowledge | 63.9% | 90.2%saturated | 1.14 | 🇺🇸 Gemini 3.1 Pro | 91.2% | 779 |
| 479 | Social Interaction QA (SIQA) O Social Interaction QA benchmark evaluating social commonsense and situational reasoning. | Social commonsense QA | 64.1% | 74.5% | 0.33 | 🇨🇳 Qwen 14B | 78.1% | 58 |
| 480 | JEC-QA O Chinese National Judicial Examination multiple-choice QA. | Legal QA (Chinese) | 64.2% | 48.0% | 0.69 | 🇺🇸 GPT-4 | 59.3% | 16 |
| 481 | OCNLI O Original Chinese Natural Language Inference benchmark. | Natural language inference (Chinese) | 64.5% | 73.6% | 0.67 | G Aquila2-34B | 79.9% | 37 |
| 482 | Aider Code Editing O Measures interactive code editing quality within the Aider assistant workflow. | Code editing | 64.9% | 87.6% | 1.85 | 🇺🇸 Gemini 2.5 Pro | 89.9% | 51 |
| 483 | CRUXEval (decision) O CRUXEval output prediction posed as choices among candidate outputs; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Code reasoning | 65.4% | 87.1% | 0.90 | G Clef-Omni | 88.8% | 38 |
| 484 | FD-Bench V3 O Full-duplex spoken dialogue benchmark v3 with tool calling; response quality (%). | Full-duplex speech + tool use | 65.4% | 84.6% | 1.21 | 🇺🇸 GPT-Live-1 (medium) | 90.0% | 10 |
| 485 | MMT-Bench O Large-scale multimodal multitask benchmark covering diverse meta-tasks for LVLMs. | Multimodal multitask understanding | 65.5% | 74.1% | 0.41 | 🇺🇸 Gemma 4 31B (IT) | 75.7% | 63 |
| 486 | MMStar O Broad evaluation of multimodal LLMs across diverse tasks. | Multimodal reasoning | 65.7% | 84.2% | 0.59 | 🇨🇳 Qwen3.8-Max | 85.9% | 211 |
| 487 | GSM8K (Japanese) O Japanese translation of GSM8K grade-school math. | Math (grade-school, Japanese) | 65.8% | 53.7% | 0.93 | 🇺🇸 LFM2.5 1.2B JP 202606 | 62.2% | 13 |
| 488 | New Yorker caption matching O Pick the caption written for a described New Yorker cartoon from five; accuracy (Jev Decision Index suite, posed as typed decisions). | Humor understanding | 65.8% | 80.6% | 0.75 | G AutoTrust GEV-26B-Decide | 82.0% | 38 |
| 489 | CoQA O Conversational question answering (ChatRAG-Bench subset). | Conversational QA | 66.3% | 81.3% | 0.37 | G Yuan3.0 Ultra | 94.6% | 10 |
| 490 | PhishNChips O Phishing vs legitimate email-and-link decisions; accuracy (Jev Decision Index suite, posed as typed decisions). | Phishing detection | 66.4% | 82.1% | 0.47 | 🇺🇸 DiffusionGemma Jev | 85.4% | 38 |
| 491 | VoiceBench MMSU O VoiceBench MMSU benchmark (voice modality). | VoiceBench | 66.5% | 83.6% | 0.98 | 🇺🇸 Gemini 2.5 Pro | 88.3% | 18 |
| 492 | Multi-IF O Composite instruction-following evaluation across multiple tasks. | Instruction following (multi-task) | 66.5% | 88.6% | 0.83 | 🇺🇸 Gemma 4 31B (IT) | 89.7% | 129 |
| 493 | Chinese SimpleQA O Chinese variant of the SimpleQA benchmark. | QA (Chinese) | 66.5% | 83.9% | 0.40 | 🇨🇳 ERNIE 5.0 | 86.0% | 28 |
| 494 | VideoMME (w/o sub) O Video understanding benchmark without subtitles. | Video understanding | 66.5% | 86.2% | 0.60 | 🇺🇸 Gemini 3 Pro | 88.6% | 122 |
| 495 | VisualWebBench O Average accuracy on VisualWebBench. | Web UI understanding | 66.6% | 81.4% | 0.99 | 🇺🇸 Holo1.5-72B | 83.8% | 28 |
| 496 | MM-Vet v2 O MM-Vet v2: integrated capabilities incl. image-text sequence understanding; GPT-4 graded. | Multimodal evaluation | 66.8% | 69.0% | 0.81 | 🇺🇸 Claude 3.5 Sonnet 20240620 | 71.8% | 21 |
| 497 | VoiceBench SD-QA O VoiceBench Spoken Dialogue QA results. | VoiceBench | 67.0% | 85.1% | 1.28 | 🇺🇸 Gemini 2.5 Pro | 90.1% | 18 |
| 498 | MVBench O Multi-view or multi-video QA benchmark (MVBench). | Video QA | 67.0% | 79.7% | 0.33 | 🇺🇸 Gemini 3.7 Flash | 82.2% | 128 |
| 499 | MAXIFE O Multilingual instruction-following evaluation across English and multilingual original prompts. | Instruction following (multilingual) | 67.2% | 88.7% | 0.86 | 🇨🇳 Qwen3.7-Max | 89.2% | 29 |
| 500 | IndoMMLU O Indonesian multi-task exam questions from primary school to university entrance level. | Indonesian knowledge | 67.2% | 67.4% | 0.89 | 🇨🇳 Qwen2.5-72B-Instruct | 69.3% | 12 |
| 501 | MATH500 (Japanese) O Japanese translation of MATH500. | Math (Japanese) | 67.4% | 43.9% | 1.67 | 🇺🇸 LFM2.5 1.2B JP 202606 | 62.8% | 10 |
| 502 | HRBench 8K O Hallucination robustness benchmark with 8K token contexts. | Hallucination robustness | 67.6% | 84.6% | 0.64 | G SenseNova-MARS-32B | 86.6% | 32 |
| 503 | TabMWP O Tabular math word problems (MMTab subset, image tables). | Table math | 67.7% | 84.9% | 0.36 | 🇨🇳 Kimi-K2.5 | 95.9% | 17 |
| 504 | Video-MME O Multimodal evaluation of video understanding and reasoning. | Video understanding (multimodal) | 67.8% | 83.2% | 0.65 | 🇺🇸 Gemini 3 Pro | 88.4% | 26 |
| 505 | GaoKao 2023 En O English translation of 2023 Chinese college entrance exam (Gaokao) math questions, used in Qwen2.5-Math / DeepSeekMath style evaluations. | Math (exam) | 67.9% | 68.6% | 1.41 | 🇨🇳 Qwen2.5-Math-72B-Instruct | 71.9% | 24 |
| 506 | olmOCR-Bench O olmOCR benchmark assessing OCR fidelity and structured extraction on complex document pages. | Document OCR | 68.2% | 86.3% | 0.25 | G Infinity-Parser2-Pro | 87.6% | 61 |
| 507 | ACP-Bench MCQ O Safety and behavior evaluation with multiple-choice questions. | Safety evaluation (MCQ) | 68.2% | 75.1% | 1.28 | 🇺🇸 Llama 3.3 70B | 82.1% | 23 |
| 508 | MathVista-Mini O Lightweight subset of MathVista for quick evaluation of visual mathematical reasoning. | Math reasoning (multimodal) | 68.3% | 88.7% | 0.89 | 🇨🇳 Qwen3.5-397B-A17B | 90.3% | 212 |
| 509 | MultiPL-E O Multilingual code generation and execution benchmark across many programming languages. | Code generation (multilingual) | 68.4% | 88.1% | 1.28 | 🇺🇸 Claude Opus 4 | 89.6% | 130 |
| 510 | Creative Writing v3 O A LLM-judged creative writing benchmark. | Creative writing | 68.4% | — | 1.25 | 🇺🇸 GPT-6 Astra | 2173 | 190 |
| 511 | CyBench O Framework with 40 professional-level CTF tasks evaluating LLMs' practical cybersecurity capabilities. | Cybersecurity CTF | 68.6% | — | 0.20 | 🇺🇸 Llama 3 70B | ↓ 5.0% | 24 |
| 512 | TurkishMMLU O Native Turkish multiple-choice QA benchmark from Turkish curriculum exams. | Turkish knowledge | 68.6% | 71.2% | 1.32 | 🇨🇳 Qwen2.5-72B-Instruct | 76.1% | 12 |
| 513 | Ruler 1M O RULER benchmark at 1M context window. | Long-context eval | 68.7% | 94.0%saturated | 1.24 | 🇨🇳 Kimi-Linear-Instruct | 94.8% | 21 |
| 514 | Global-MMLU O Full Global-MMLU evaluation across diverse languages and regions. | Multi-domain knowledge (global) | 68.9% | 92.8%saturated | 0.80 | 🇺🇸 Claude Opus 5.5 | 94.3% | 46 |
| 515 | MMVet O Comprehensive evaluation suite for assessing multimodal LLM capabilities. | Multimodal evaluation | 69.0% | 84.0% | 0.76 |  🇨🇳 R-4B-Base | 85.9% | 107 |
| 516 | FRAMES O Frame-based interactive reasoning and dialogue benchmark. | Interactive reasoning | 69.0% | 88.1% | 0.67 | 🇨🇳 Tongyi DeepResearch | 90.6% | 55 |
| 517 | MLVU O MLVU: Large-scale multi-task benchmark for video understanding. | Large video understanding | 69.1% | 88.0% | 0.64 | 🇨🇳 Qwen3.8-Max | 90.8% | 105 |
| 518 | VideoMME O Video multimodal evaluation suite (VideoMME). | Multimodal video evaluation | 69.2% | 89.0% | 0.78 | G Seed2.0 Pro | 89.5% | 41 |
| 519 | DeepMind Mathematics O Synthetic math problem sets from DeepMind covering arithmetic, algebra, calculus, and more. | Math reasoning | 69.2% | 66.7% | 1.98 | G Granite-4.1-30B | 81.9% | 18 |
| 520 | PinchBench (Avg) O PinchBench OpenClaw-style agent task benchmark, average score across runs. | Agent tasks (OpenClaw) | 69.3% | 90.4%saturated | 0.48 | G KAT-Coder-Pro V2.5 | 94.9% | 37 |
| 521 | CLIcK O Korean long-form instruction-following benchmark. | Korean instruction following | 69.4% | 90.6%saturated | 0.80 | 🇨🇳 DeepSeek V4-Pro-Thinking (max) | 91.6% | 23 |
| 522 | MBPP+ O Extended MBPP with more tests and stricter evaluation. | Code generation | 69.8% | 90.0%saturated | 0.81 | 🇨🇳 GLM 4.6 | 94.2% | 379 |
| 523 | CRUX-I O Code Reasoning and Understanding eXam – Interactive. | Code reasoning | 70.0% | 98.0%saturated | 1.14 | 🇺🇸 Gemini 3 Pro Preview | 98.8% | 132 |
| 524 | SEED-Bench-2-Plus O SEED-Bench-2-Plus overall accuracy. | Multimodal evaluation | 70.5% | 72.2% | 0.43 | 🇺🇸 Claude 3.7 Sonnet | 72.9% | 35 |
| 525 | VoiceBench AlpacaEval O VoiceBench evaluation on AlpacaEval instructions. | VoiceBench | 70.5% | 96.7%saturated | 3.44 |  🇨🇳 LongCat-Flash-Omni | 98.8% | 18 |
| 526 | ArabicMMLU O Native Arabic multi-task knowledge benchmark (school exams across MENA countries), 40 tasks. | Arabic knowledge | 70.6% | 72.6% | 0.87 | G Fanar-2-27B-Instruct | 74.7% | 17 |
| 527 | MathVista O Visual math reasoning across diverse tasks. | Multimodal math reasoning | 70.6% | 90.3%saturated | 0.73 | G Seed2.1 Pro | 90.7% | 111 |
| 528 | MMMLU O Massively multilingual MMLU-style evaluation across many languages. | Multi-domain knowledge (multilingual) | 70.7% | 92.1%saturated | 0.95 | 🇺🇸 Claude Mythos Preview | 92.7% | 114 |
| 529 | RoboAfford O Robotic affordance (object & spatial) grounding benchmark. | Embodied reasoning | 70.7% | 84.0% | 0.20 | G ACE-Brain-0.6-9B | 88.1% | 18 |
| 530 | RealWorldQA O Visual question answering with real-world images and scenarios. | Real-world visual QA | 70.8% | 87.6% | 0.46 | 🇨🇳 Qwen3.8-Flash-Next | 88.5% | 228 |
| 531 | MMHal-Bench (Hallucination Rate) O MMHal-Bench hallucination rate of vision-language model responses; lower is better. | Multimodal hallucination | 70.8% | — | 0.30 | 🇨🇳 MiniCPM-V 4.7 35B-A3B | ↓ 21.9% | 18 |
| 532 | CC-OCR O Cross-lingual OCR benchmark evaluating character recognition across mixed-language documents. | OCR (cross-lingual) | 70.8% | 82.8% | 0.40 | 🇨🇳 Qwen3.6-Plus | 83.4% | 43 |
| 533 | When2Call O NVIDIA's When2Call multiple-choice test set: choose whether to answer directly, call a tool, ask for more information, or say it cannot answer. | Tool calling | 70.9% | 90.5%saturated | 0.60 | G d1-ultra | 92.6% | 61 |
| 534 | MMAU v05.15.25 O Audio reasoning benchmark MMAU v05.15.25. | Audio reasoning | 71.0% | 81.2% | 0.26 | 🇺🇸 Gemini 3.1 Pro | 82.5% | 30 |
| 535 | XCOPA O Multilingual extension of COPA causal commonsense reasoning in 11 languages. | Cross-lingual commonsense causal reasoning | 71.0% | 71.4% | 0.46 | 🇨🇳 Zhipu GLM 4-9B-Chat | 80.1% | 22 |
| 536 | Perception Test O DeepMind Perception Test: multimodal video benchmark of perception and reasoning skills (MC video QA). | Video understanding | 71.1% | 81.9% | 0.47 | G PLM-8B | 82.7% | 37 |
| 537 | BBH (decision) O BIG-Bench Hard fixed-option tasks answered directly with no working; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Multi-task reasoning | 71.1% | 88.3% | 0.87 | G Jev 1.14 | 93.2% | 39 |
| 538 | SLAKE O Semantically-Labeled Knowledge-Enhanced medical visual question answering benchmark. | Medical VQA | 71.7% | 87.6% | 0.37 | 🇨🇳 Qwen3.8-Max | 90.8% | 39 |
| 539 | HRBench 4K O Hallucination robustness benchmark with 4K token contexts. | Hallucination robustness | 71.7% | 88.7% | 0.69 | G SenseNova-MARS-32B | 90.2% | 35 |
| 540 | AMC-23 O American Mathematics Competition 2023 evaluation. | Math (competition) | 71.8% | 94.9%saturated | 2.02 | G QwQ-32B | 98.5% | 64 |
| 541 | Arena-Hard O Hard prompts on Chatbot Arena. | Chat ability | 71.8% | 96.0%saturated | 2.47 | 🇫🇷 Mistral Medium 3 | 97.1% | 89 |
| 542 | TempCompass O Temporal reasoning benchmark evaluating understanding of time-related concepts in videos and images. | Temporal reasoning | 71.9% | 85.0% | 0.59 | 🇺🇸 Gemini 3 Pro | 88.0% | 31 |
| 543 | VideoMME (w/sub) O Video understanding benchmark with subtitles. | Video understanding | 71.9% | 89.8% | 0.62 | 🇨🇳 Qwen3.8-Max | 90.4% | 94 |
| 544 | WildVision-Bench O In-the-wild multimodal chat benchmark judged against reference model (WildVision Arena). | Multimodal chat | 72.1% | 79.7% | 1.26 | 🇺🇸 Gemini 2.5 Pro | 90.1% | 23 |
| 545 | VQA-RAD O Radiology visual question answering (closed-question accuracy). | Medical VQA | 72.2% | 79.4% | 1.22 | G MedMO-8B-Next | 86.4% | 11 |
| 546 | VLMsAreBlind O Benchmark probing robustness of vision-language models to visual perturbations. | Multimodal robustness | 72.2% | 97.0%saturated | 1.45 | 🇺🇸 Gemini 3 Pro | 97.5% | 26 |
| 547 | CRUX-O O Code Reasoning and Understanding eXam – Offline. | Code reasoning | 72.4% | 98.6%saturated | 1.20 | G IQuest-Coder-V1-40B-Loop-Thinking | 99.4% | 166 |
| 548 | CruxEval O Mathematical coding challenge set from the CruxEval benchmark. | Code reasoning | 72.4% | 87.9% | 1.62 | 🇨🇳 Qwen3-32B Thinking | 95.0% | 56 |
| 549 | WMT24++ O Extended WMT 2024 evaluation across multiple language pairs. | Machine translation | 72.5% | 91.5%saturated | 0.56 | 🇨🇳 Qwen3-235B-A22B-Thinking-2507 | 94.7% | 70 |
| 550 | BFCL v3 (Live) O BFCL v3 Live subset for real-time tool calling evaluation. | Tool calling | 72.6% | 81.1% | 0.34 | 🇺🇸 Gemini 3.1 Pro | 84.9% | 19 |
| 551 | Spider O Complex text-to-SQL benchmark over cross-domain databases. | Text-to-SQL | 72.8% | 82.7% | 0.69 | G Yuan3.0 Ultra | 83.9% | 25 |
| 552 | DA-2K O 2D/3D spatial reasoning benchmark. | Spatial reasoning | 73.0% | 84.9% | 0.40 | G HY-Embodied-0.5 MoT-2B | 92.3% | 13 |
| 553 | JMMLU O Japanese Massive Multitask Language Understanding benchmark (translated and culture-specific MMLU questions). | Multi-domain knowledge (Japanese) | 73.1% | 76.8% | 1.19 | 🇨🇳 Qwen2.5-72B-Instruct | 80.6% | 25 |
| 554 | CLadder O Causal reasoning questions across the ladder of causation; accuracy (Jev Decision Index suite, posed as typed decisions). | Causal reasoning | 73.1% | 93.7%saturated | 0.70 | G Clef-Flash | 97.7% | 38 |
| 555 | L-Eval O Long-context understanding and generation. | Long-context eval | 73.2% | 73.3% | 0.99 | 🇨🇳 Qwen1.5-72B-Chat | 78.1% | 21 |
| 556 | AGIEval (English) O English subset of AGIEval; academic and professional exam questions. | Exams | 73.3% | 90.5%saturated | 1.01 | 🇨🇳 Qwen3-VL 32B Thinking | 92.2% | 228 |
| 557 | Mantis O Multimodal reasoning and instruction following benchmark (Mantis). | Multimodal reasoning | 73.8% | 82.4% | 0.79 | G dots.vlm1 | 86.2% | 42 |
| 558 | ScreenQA Complex O Complex ScreenQA benchmark accuracy. | GUI QA | 74.1% | 83.8% | 0.81 | 🇺🇸 Holo1.5-72B | 87.1% | 12 |
| 559 | EmbSpatialBench O Embodied spatial understanding benchmark evaluating navigation and localization. | Spatial understanding | 74.4% | 84.7% | 0.32 | 🇨🇳 Qwen3.5-Omni-Plus | 85.4% | 71 |
| 560 | Big Bench Audio O Artificial Analysis Big Bench Audio: spoken-question reasoning benchmark adapted from BIG-Bench Hard. | Speech reasoning | 74.5% | 99.3%saturated | 2.15 | G StepAudio 3 Realtime | 99.7% | 70 |
| 561 | InfoQA O Information retrieval question answering benchmark evaluating factual responses. | Information-seeking QA | 74.9% | 84.1% | 0.97 | 🇺🇸 Gemini 3 Pro | 86.9% | 49 |
| 562 | PubMedQA O Yes/no/maybe question answering over PubMed abstracts. | Biomedical QA | 74.9% | 77.4% | 0.55 | G MedMO-4B-Next | 78.2% | 15 |
| 563 | Multilingual MMBench O Multilingual MMBench average score across languages. | Multilingual vision benchmark | 75.1% | 76.1% | 0.68 | 🇺🇸 LFM2.5-VL-3B | 79.5% | 12 |
| 564 | V* O V* benchmark accuracy. | Multimodal reasoning | 75.2% | 95.3%saturated | 0.73 | 🇨🇳 Qwen3.6-Plus | 96.9% | 72 |
| 565 | MedQA O US medical licensing exam-style multiple choice questions. | Medical exams | 75.2% | 89.5% | 1.14 | G OpenEvidence Darwin | 100.0% | 40 |
| 566 | ConvFinQA O Conversational numerical reasoning over financial reports (ChatRAG-Bench subset). | Conversational QA | 75.2% | 83.1% | 0.20 | 🇺🇸 Claude Opus 4.6 | 86.5% | 10 |
| 567 | R-Bench O R-Bench: robustness of LMMs to real-world image corruptions (discriminative subset). | Robustness (multimodal) | 75.2% | 75.0% | 0.66 | 🇺🇸 GPT-4o-2024-05-13 | 77.7% | 19 |
| 568 | MixEval O Mixed-subject benchmark covering knowledge and reasoning tasks across domains. | Multi-task reasoning | 75.3% | 84.7% | 0.20 | 🇨🇳 Qwen2 72B Instruct | 86.7% | 33 |
| 569 | VoiceBench BBH O VoiceBench evaluation on Big-Bench Hard prompts. | VoiceBench | 75.4% | 87.6% | 1.09 | 🇺🇸 Gemini 2.5 Pro | 92.6% | 13 |
| 570 | HumanEval-X O Multilingual code generation benchmark extending HumanEval to multiple programming languages. | Code generation (multilingual) | 75.4% | 90.4%saturated | 1.58 | G TeleChat3-36B-Thinking | 92.7% | 58 |
| 571 | HAE-RAE Bench O Korean language understanding benchmark evaluating knowledge and reasoning. | Korean language understanding | 75.5% | 87.5% | 0.96 | G Kanana-1.5-32.5B-Base | 90.7% | 21 |
| 572 | BFCL O Benchmark for functional code correctness and logic. | Code reasoning | 75.5% | 91.5%saturated | 0.26 | 🇨🇳 Qwen3-4B | 95.0% | 24 |
| 573 | HoVer O Multi-hop claim verification over Wikipedia introductions; accuracy (Jev Decision Index suite, posed as typed decisions). | Claim verification | 75.5% | 86.7% | 0.49 | G AutoTrust JEV-35B | 88.1% | 38 |
| 574 | Global PIQA O Physical commonsense reasoning benchmark spanning 100 languages and diverse cultural contexts. | Commonsense reasoning across 100 Languages and Cultures | 75.7% | 94.3%saturated | 0.86 | 🇺🇸 Gemini 3 Flash | 95.6% | 55 |
| 575 | Ground-UI-1K O Accuracy on the Ground-UI-1K grounding benchmark. | GUI grounding | 75.7% | 85.2% | 0.85 | G Holo2-235B-A22B | 85.5% | 22 |
| 576 | MMLU Med O Medical subset of MMLU (clinical knowledge, anatomy, college medicine, medical genetics, professional medicine, college biology). | Medical knowledge | 76.1% | 80.4% | 1.00 | 🇺🇸 MedGemma 27B | 86.2% | 11 |
| 577 | Global-MMLU-Lite O Lightweight global variant of MMLU covering diverse languages and regions. | Multi-domain knowledge (global) | 76.4% | 92.4%saturated | 0.72 | 🇺🇸 Claude Fable 5 | 93.3% | 49 |
| 578 | InfoVQA O Visual question answering over infographics requiring reading, counting, and reasoning. | Infographic VQA | 76.8% | 91.1%saturated | 0.88 | 🇨🇳 Kimi-K2.5 Thinking | 92.6% | 166 |
| 579 | CIBench O Evaluation of LLMs using a code interpreter for data-science tasks; EN/ZH average. | Code interpreter | 76.8% | 50.5% | 1.46 | G InternLM2-Chat-20B | 55.4% | 13 |
| 580 | SQA O Sequential QA over tables (ChatRAG-Bench subset). | Table QA | 76.9% | 86.5% | 0.44 | G Yuan3.0 Ultra | 91.0% | 10 |
| 581 | LAMBADA O Word prediction requiring broad context understanding. | Language modeling | 76.9% | 80.5% | 0.20 | 🇺🇸 GPT-3 | 86.4% | 60 |
| 582 | ACP-Bench Bool O Safety and behavior evaluation with yes/no questions. | Safety evaluation (boolean) | 77.0% | 80.8% | 0.95 | 🇨🇳 Qwen3-32B | 85.1% | 23 |
| 583 | CharXiv (DQ) O Scientific chart/table descriptive questions from arXiv PDFs. | Chart description (PDF) | 77.0% | 95.3%saturated | 1.20 | 🇺🇸 Gemini 3 Pro | 96.0% | 171 |
| 584 | MATH O Competition-level mathematics across algebra, geometry, number theory, combinatorics. | Math (competition) | 77.2% | 96.9%saturated | 2.19 | 🇺🇸 o3 mini | 97.9% | 526 |
| 585 | Roo Code Evals O Community-maintained coding evals and leaderboard by Roo Code. | Code assistant eval | 77.3% | 100%saturated | 1.96 | 🇺🇸 Claude Opus 4.6 | 100.0% | 40 |
| 586 | Ruler 128k O RULER benchmark at 128k context window. | Long-context eval | 77.3% | 94.7%saturated | 0.81 | 🇨🇳 Qwen3-Next-80B-A3B-Instruct | 96.0% | 44 |
| 587 | Multilingual MMLU O Multilingual variant of MMLU across many languages. | Multi-domain knowledge (multilingual) | 77.4% | 85.5% | 1.07 | 🇺🇸 GPT-4.1 | 87.3% | 34 |
| 588 | WebClick O Task success on the WebClick UI agent benchmark. | GUI agents | 77.7% | 92.6%saturated | 1.31 | G Holo2-235B-A22B | 94.3% | 22 |
| 589 | AA Speech Agent Task Success O Artificial Analysis Speech Agent Arena task completion rate (%). | Voice agents | 77.8% | 93.0%saturated | 0.88 | 🇺🇸 Grok Voice 3.0 | 94.8% | 29 |
| 590 | SEED-Bench-Img O SEED-Bench image-only subset (SEED-Bench-Img). | Multimodal image understanding | 77.9% | 77.7% | 0.40 | G Bagel 14B | 78.5% | 75 |
| 591 | API-Bank (decision) O API-Bank cases answered as option choices; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Tool use | 77.9% | 90.7%saturated | 1.33 | G Clef-Flash | 93.1% | 36 |
| 592 | ShareRobot-Trajectory O ShareRobot trajectory prediction benchmark. | Embodied reasoning | 77.9% | 84.7% | 0.24 | G RoboBrain2.5-8B | 85.2% | 13 |
| 593 | TriviaQA O Open-domain question answering benchmark built from trivia and web evidence. | Open-domain QA | 78.0% | 88.1% | 0.90 | 🇺🇸 Nemotron-3-Super-120B-A12B-Base | 89.8% | 169 |
| 594 | MedQA (MCMLE) O MedQA Mainland China Medical Licensing Exam subset (5 options). | Medical QA (Chinese) | 78.0% | 60.6% | 1.08 | 🇺🇸 GPT-4 | 74.6% | 16 |
| 595 | Image QA Average O Average of single-image visual question answering benchmarks. | Image QA (aggregate) | 78.0% | 83.0% | 0.35 | 🇺🇸 Gemini 3 Pro | 86.2% | 22 |
| 596 | SEED-Bench O SEED-Bench comprehensive multimodal understanding benchmark evaluating generative comprehension across multiple dimensions. | Multimodal evaluation | 78.0% | 76.5% | 0.37 | 🇨🇳 Qwen3-VL-4B-Instruct | 77.0% | 24 |
| 597 | ERQA-PLUS O Extended Embodied Reasoning QA benchmark. | Embodied reasoning | 78.6% | 87.1% | 0.34 | 🇺🇸 Gemini 3.6 Flash | 92.2% | 13 |
| 598 | WritingBench O General-purpose writing quality benchmark. | Writing quality | 78.6% | 86.7% | 0.34 | 🇨🇳 Qwen3-235B-A22B-Thinking-2507 | 88.3% | 43 |
| 599 | IFEval (Strict Prompt) O IFEval strict prompt-level accuracy. | Instruction following | 78.6% | 85.2% | 1.46 | 🇨🇳 Qwen2.5 Plus (11-27) | 86.3% | 53 |
| 600 | CMRC 2018 O Span-extraction Chinese machine reading comprehension (CMRC 2018). | Chinese reading comprehension | 78.7% | 81.1% | 0.20 | G TigerBot-70B-Base | 85.9% | 16 |
| 601 | TextQuests Harm O Harmfulness evaluation on TextQuests scenarios. | Harmful propensities | 78.7% | — | 0.23 | 🇺🇸 Grok 4.1 Fast | ↓ 9.1% | 15 |
| 602 | Ruler 512k O RULER benchmark at 512k context window. | Long-context eval | 78.7% | 92.8%saturated | 1.34 | 🇨🇳 Qwen3.5-122B-A10B | 96.0% | 10 |
| 603 | EvalPlus O Aggregated code evaluation suite from EvalPlus. | Code generation | 79.0% | 88.6% | 1.47 | 🇺🇸 o1 Mini | 89.0% | 173 |
| 604 | BBH-ZH O Chinese translation of BIG-Bench Hard reasoning tasks. | Multi-task reasoning (Chinese) | 79.1% | 86.6% | 0.80 | G LLaDA2.0 Flash | 87.5% | 14 |
| 605 | OmniDocBench O Document understanding benchmark covering multi-page layouts, tables, and charts for robust question answering. Overall score, higher is better (v1.0 edit-distance results are in omnidocbench-edit). | Document understanding | 79.2% | 93.3%saturated | 0.50 | G PaddleOCR-VL-1.5 | 94.5% | 57 |
| 606 | Ruler 256k O RULER benchmark at 256k context window. | Long-context eval | 79.5% | 92.2%saturated | 1.45 | 🇨🇳 Qwen3.5-122B-A10B | 96.7% | 11 |
| 607 | DROP O Discrete reasoning over paragraphs (addition, counting, comparisons). | Reading + reasoning | 79.9% | 92.1%saturated | 1.28 | 🇨🇳 Kimi K2 Instruct | 93.5% | 151 |
| 608 | FActScore O Measures factuality on an open-source prompt suite; scored as 100 - hallucination rate, higher is better. | Factuality on open-source prompts | 79.9% | 94.8%saturated | 0.73 | 🇺🇸 GPT-5 | 99.0% | 11 |
| 609 | CRPE O Circular-based Relation Probing Evaluation (relation split) for object-relation hallucination. | Relation hallucination | 80.1% | 78.4% | 0.62 | 🇨🇳 InternVL2.5-26B | 79.1% | 18 |
| 610 | HumanEval+ O Extended HumanEval with more tests. | Code generation | 80.2% | 95.3%saturated | 1.52 | 🇨🇳 Qwen3.8-27B-UD-Q4_K_XL | 95.7% | 439 |
| 611 | C-Eval O Chinese college-level exam benchmark. | Chinese exams | 80.6% | 93.5%saturated | 0.92 | 🇨🇳 Kimi-K2.5 | 94.0% | 330 |
| 612 | RoboRefIt O Referring-expression grounding in cluttered robotic scenes. | Embodied reasoning | 80.6% | 86.9% | 0.24 | G PhysBrain1.5-8B | 89.6% | 13 |
| 613 | MMLU-STEM O STEM subset of MMLU. | STEM knowledge | 80.7% | 87.1% | 1.27 | 🇨🇳 Qwen3-8B | 92.5% | 106 |
| 614 | VQAv2 O Standard Visual Question Answering v2 benchmark on natural images. | Visual question answering | 80.8% | 87.8% | 0.20 | 🇺🇸 Gemini 3 Pro | 89.8% | 83 |
| 615 | TextVQA O Visual question answering that requires reading text in images. | Text-based VQA | 80.8% | 89.8% | 0.49 | G Ovis2.6-80B-A3B | 91.2% | 194 |
| 616 | Q-Bench O Q-Bench (A1 perception, test split) evaluating MLLMs on low-level visual attributes and image quality questions. | Low-level visual perception | 80.8% | 71.6% | 0.60 | 🇺🇸 GPT-4V(ision) | 74.1% | 10 |
| 617 | CMMLU O Chinese counterpart to MMLU. | Chinese multi-domain | 80.9% | 90.9%saturated | 0.93 | 🇨🇳 Qwen2.5 Max | 91.9% | 281 |
| 618 | GSM-Plus O Enhanced GSM-style grade-school math benchmark variant. | Math (grade-school, enhanced) | 80.9% | 90.0% | 1.68 | 🇨🇳 Qwen3.5-9B | 91.4% | 47 |
| 619 | CV-Bench O Diverse CV tasks for VLMs. | Computer vision QA | 81.1% | 90.3%saturated | 0.44 | 🇺🇸 Gemini 3 Pro | 92.0% | 72 |
| 620 | T-Eval O Step-by-step evaluation of LLM tool-utilization (instruct, plan, reason, retrieve, understand, review); EN/ZH average. | Tool use | 81.1% | 65.5% | 0.97 | 🇨🇳 Qwen 14B Chat | 66.2% | 13 |
| 621 | MMBench v1.1 (CN) O MMBench v1.1 Chinese subset for evaluating multimodal LLMs. | Multimodal understanding (Chinese) | 81.1% | 90.2%saturated | 0.74 | 🇺🇸 Gemini 3 Pro | 91.3% | 345 |
| 622 | OCRBench V2 O OCRBench v2 evaluating text extraction from images and documents. | OCR (vision text extraction) | 81.2% | 92.2%saturated | 0.65 | 🇨🇳 Qwen3.5-397B-A17B | 93.1% | 239 |
| 623 | MMLU O 57 tasks spanning STEM, humanities, social sciences; broad knowledge and reasoning. | Multi-domain knowledge | 81.3% | 94.6%saturated | 0.92 | 🇺🇸 Gemini 4 Argon | 95.3% | 760 |
| 624 | IFEval O Instruction following capability evaluation for LLMs. | Instruction following | 81.4% | 96.2%saturated | 0.80 | G Jamba2 Mini | 97.0% | 482 |
| 625 | EQ-Bench O General reasoning benchmark assessing equation/logic capabilities. | Reasoning | 81.4% | 85.9% | 0.87 | 🇺🇸 Claude 3.5 Sonnet 20240620 | 86.4% | 186 |
| 626 | Flores O FLORES multilingual translation benchmark. | Machine translation (multilingual) | 81.6% | 88.1% | 1.08 | G EuroLLM-22B | 88.9% | 17 |
| 627 | RewardBench O Preference/reward model evaluation benchmark. | Reward modeling | 81.7% | 94.6%saturated | 0.60 | 🇺🇸 INF-ORM-Llama3.1-70B | 95.1% | 175 |
| 628 | ScreenSpot O Center accuracy on ScreenSpot. | Screen UI locators | 81.7% | 95.1%saturated | 0.48 | 🇨🇳 Qwen3-VL 32B Instruct | 95.8% | 40 |
| 629 | BBH O Hard subset of BIG-bench with diverse reasoning tasks. | Multi-task reasoning | 81.9% | 93.0%saturated | 1.21 | 🇨🇳 ERNIE 4.5 424B A47B | 94.3% | 457 |
| 630 | NExT-QA O Video question answering benchmark on causal and temporal action reasoning (multiple-choice). | Video understanding | 82.1% | 86.0% | 0.25 | 🇺🇸 GPT-5 | 86.3% | 22 |
| 631 | MMLU-Redux O Updated MMLU-style evaluation with revised questions and scoring. | Multi-domain knowledge | 82.1% | 95.9%saturated | 1.12 | G GRM-3.2-Sky | 96.9% | 272 |
| 632 | MMMB O Multilingual Multimodal Benchmark (MMMB) average score. | Multilingual MMBench | 82.1% | 82.1% | 0.72 | 🇨🇳 Qwen3.5-4B | 83.4% | 20 |
| 633 | MBPP O Short Python problems with hidden tests. | Code generation | 82.2% | 96.3%saturated | 1.34 | 🇨🇳 Kimi-K2 Thinking | 97.4% | 522 |
| 634 | MMC-Benchmark O MMC multimodal chart understanding benchmark. | Chart understanding | 82.5% | 73.5% | 0.72 | 🇨🇳 Qwen-VL-Max | 76.4% | 14 |
| 635 | HRM8K O 8k-question Korean reasoning and knowledge benchmark. | Korean reasoning | 82.7% | 92.1%saturated | 0.75 | 🇨🇳 DeepSeek V4-Flash-Thinking (max) | 93.4% | 11 |
| 636 | MMBench v1.1 (EN) O MMBench v1.1 English subset for evaluating multimodal LLMs. | Multimodal understanding (English) | 82.9% | 92.9%saturated | 0.69 | 🇺🇸 Gemini 3 Pro | 93.3% | 367 |
| 637 | MMBench v1.1 (EN dev) O English dev split of MMBench v1.1 measuring multimodal question answering. | General VQA | 83.1% | 93.4%saturated | 0.63 | 🇨🇳 Kimi-K2.5 | 94.2% | 80 |
| 638 | Local Agent Bench O Tests whether small open-weight models can reliably decide when to call tools and when not to. Agent Score = (Action x 0.4) + (Restraint x 0.3) + (Wrong-Tool-Avoidance x 0.3). | Tool calling judgment | 83.2% | 87.2% | 0.79 | 🇺🇸 LFM2.5 1.2B Instruct | 88.0% | 21 |
| 639 | Wild-Jailbreak O Adversarial jailbreak benchmark evaluating refusal robustness. | Safety / jailbreak | 83.5% | 93.5%saturated | 0.65 | 🇺🇸 GPT-OSS 120B (high) | 98.2% | 12 |
| 640 | VABench-VisualTrace O VABench visual-trace (trajectory) subset. | Embodied reasoning | 83.5% | 90.3%saturated | 0.30 | G Embodied-R1.5-8B | 92.3% | 13 |
| 641 | JHumanEval O Japanese-prompt translation of HumanEval code generation benchmark. | Code (Japanese) | 83.5% | 93.5%saturated | 1.33 | 🇺🇸 GPT-5 mini | 94.5% | 31 |
| 642 | WinoGrande O Large-scale adversarial Winograd Schema-style pronoun resolution. | Coreference reasoning | 83.6% | 95.4%saturated | 0.61 | G Clef-Flash | 97.5% | 325 |
| 643 | WildGuardTest O WildGuardTest safety benchmark. | Safety | 83.7% | 94.3%saturated | 0.48 | G d1-ultra | 95.4% | 32 |
| 644 | Belebele O Parallel multiple-choice reading-comprehension benchmark over FLORES passages in 122 language variants. | Multilingual reading comprehension | 83.8% | 95.4%saturated | 0.87 | 🇺🇸 Gemma 4 31B (IT) | 95.9% | 34 |
| 645 | MGSM O Multilingual grade school math word problems. | Math (multilingual) | 83.9% | 93.8%saturated | 1.44 | 🇺🇸 Claude Opus 4.1 (2025-08-05) Thinking | 94.4% | 154 |
| 646 | ChartQA O Visual question answering over charts and plots. | Chart understanding (VQA) | 84.3% | 92.9%saturated | 0.62 | 🇨🇳 Keye-VL-1.5-8B | 94.1% | 223 |
| 647 | MMBench (CN) O MMBench v1.0 Chinese test split. | Multimodal understanding (Chinese) | 84.4% | 86.9% | 0.75 | 🇨🇳 InternVL2.5-78B | 88.5% | 78 |
| 648 | ScreenQA Short O Short-form ScreenQA benchmark accuracy. | GUI QA | 84.5% | 89.6% | 0.71 | 🇺🇸 Holo1.5-72B | 91.9% | 12 |
| 649 | GAOKAO-Bench O GAOKAO benchmark measuring Chinese college entrance exam performance. | Chinese exams | 84.7% | 90.9%saturated | 1.27 | 🇨🇳 Qwen3-30B-A3B-Instruct-2507 | 94.5% | 93 |
| 650 | CountBenchQA O Visual question answering benchmark focused on counting objects across varied scenes. | Visual counting QA | 84.8% | 92.9%saturated | 0.62 | G dots.mocr | 94.5% | 50 |
| 651 | SQuAD v2.0 O Like v1.1 with unanswerable questions. | Reading comprehension | 84.9% | 91.1%saturated | 1.32 | G Ling 2.6 1T Base | 93.0% | 35 |
| 652 | AA Conversational Dynamics: Backchannel O Artificial Analysis Conversational Dynamics (Full-Duplex-Bench subset): backchannelling while the user speaks (%). | Full-duplex speech interaction | 84.9% | 98.4%saturated | 1.09 | 🇨🇳 Qwen Audio 3.0 Realtime Plus | 100.0% | 37 |
| 653 | AI2D O Visual question answering over science and diagram images. | Diagram understanding (VQA) | 85.5% | 97.4%saturated | 0.72 | 🇺🇸 Gemini 3 Pro | 98.7% | 258 |
| 654 | RefCOCO+ O RefCOCO+ accuracy at IoU 0.5 on the val split. | Referring expressions | 85.5% | 90.5%saturated | 0.26 | 🇨🇳 DeepSeek-VL2 | 91.2% | 33 |
| 655 | MMBench (EN) O MMBench v1.0 English test split. | Multimodal understanding (English) | 85.7% | 86.9% | 0.72 | 🇨🇳 InternVL2.5-78B | 88.3% | 100 |
| 656 | VoiceBench OpenBookQA O VoiceBench results on OpenBookQA prompts. | VoiceBench | 85.7% | 93.9%saturated | 1.13 | 🇺🇸 Gemini 2.5 Pro | 95.2% | 13 |
| 657 | RACE-Middle O RACE reading-comprehension questions from Chinese middle-school English exams. | Reading comprehension | 85.8% | 92.2%saturated | 0.60 | 🇺🇸 GPT-4 | 93.7% | 36 |
| 658 | ScreenSpot-v2 O Center accuracy on ScreenSpot-v2. | Screen UI locators | 85.8% | 96.1%saturated | 1.04 | G MAI-UI-32B | 96.5% | 88 |
| 659 | AttaQ O Adversarial jailbreak suite measuring refusal robustness against targeted attack prompts. | Safety / jailbreak | 85.9% | 87.0% | 0.30 | G Granite 3.3 8B Instruct | 88.5% | 18 |
| 660 | HumanEval O Python synthesis problems evaluated by unit tests. | Code generation | 86.0% | 99.3%saturated | 1.86 | 🇺🇸 Gemini 3 Pro Preview | 100.0% | 627 |
| 661 | MATH-100 (Japanese) O 100 MATH problems translated to Japanese (Swallow evaluation suite). | Math (Japanese) | 86.0% | 96.8%saturated | 1.96 | 🇺🇸 GPT-OSS 120B (medium) | 97.0% | 31 |
| 662 | HellaSwag O Adversarial commonsense sentence completion. | Commonsense reasoning | 86.5% | 98.3%saturated | 0.67 | G Clef-Omni | 99.1% | 415 |
| 663 | RefCOCOg O RefCOCOg average accuracy at IoU 0.5 (val). | Referring expressions | 86.6% | 91.9%saturated | 0.20 | 🇨🇳 DeepSeek-VL2 | 92.8% | 35 |
| 664 | PIQA O Physical commonsense about everyday tasks and object affordances. | Physical commonsense | 86.7% | 93.6%saturated | 0.42 | G LLaDA2.0 Flash | 96.5% | 207 |
| 665 | RefCOCO O RefCOCO average accuracy at IoU 0.5 (val). | Referring expressions | 86.9% | 94.3%saturated | 0.31 | 🇨🇳 DeepSeek-VL2 | 95.1% | 97 |
| 666 | Ruler 64k O RULER benchmark at 64k context window. | Long-context eval | 86.9% | 93.7%saturated | 0.68 | 🇺🇸 Nemotron 3 Ultra Base | 95.3% | 38 |
| 667 | BPoMP (decision) O BPoMP preference decisions; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Preference prediction | 87.1% | 96.0%saturated | 1.24 | G Clef | 96.9% | 38 |
| 668 | CLUEWSC O Chinese Winograd Schema-style coreference benchmark from CLUE. | Coreference reasoning (Chinese) | 87.2% | 91.1%saturated | 0.73 | 🇨🇳 DeepSeek R1 | 92.8% | 49 |
| 669 | CommonsenseQA O Multiple-choice QA requiring commonsense knowledge. | Commonsense QA | 87.3% | 90.9%saturated | 0.79 | 🇨🇳 Hunyuan-Large | 92.9% | 114 |
| 670 | MATH500 O 500-problem slice of the MATH benchmark for challenging math reasoning. | Math reasoning | 87.3% | 99.3%saturated | 2.21 | 🇨🇳 Qwen3.8-27B | 99.8% | 266 |
| 671 | GraphWalks Parents O Graph traversal/GraphWalks benchmark (Parents variant) for long-context reasoning. | Long-context reasoning | 87.3% | 99.6%saturated | 1.23 | G Seed2.0 Lite | 100.0% | 13 |
| 672 | SocialIQA O Social interaction commonsense QA. | Social commonsense | 87.4% | 82.2% | 0.97 | 🇨🇳 Qwen3 8B Base | 83.3% | 21 |
| 673 | FORTRESS (Benign) O Scale AI FORTRESS benign counterpart prompts; share answered without over-refusal. | Safety (over-refusal) | 87.7% | 98.4%saturated | 0.91 | 🇨🇳 DeepSeek V4-Flash-Thinking (max) | 99.2% | 17 |
| 674 | CountBench O Object counting and numeracy benchmark for visual-language models across varied scenes. | Visual counting | 87.7% | 97.7%saturated | 0.89 | 🇨🇳 Qwen3.5-27B | 97.8% | 96 |
| 675 | BoolQ O Yes/no QA from naturally occurring questions. | Reading comprehension | 88.8% | 91.9%saturated | 0.59 | G ST-MoE-32B | 92.4% | 154 |
| 676 | C3 O Multiple-choice Chinese machine reading comprehension (CLUE C3). | Chinese reading comprehension | 88.8% | 94.4%saturated | 0.92 | 🇨🇳 InternVL2.5-26B | 95.8% | 60 |
| 677 | POPE O Vision-language hallucination benchmark focusing on object existence verification. | Hallucination detection | 89.1% | 90.7%saturated | 0.20 | 🇨🇳 InternVL2.5-4B | 90.9% | 58 |
| 678 | CCPM O Chinese Classical Poetry Matching: match modern Chinese to classical poetry lines. | Chinese classical poetry | 89.7% | 90.0%saturated | 0.85 | 🇨🇳 DeepSeek-V2 | 93.1% | 13 |
| 679 | VCR (EN, easy) O Visual Caption Restoration, English easy split; exact-match accuracy of restoring partially occluded text. | Visual caption restoration | 89.8% | 94.3%saturated | 1.82 | 🇨🇳 InternVL2.5-78B | 95.7% | 27 |
| 680 | LLaVA-Bench (In-the-Wild) O GPT-4-judged open-ended visual conversation, detailed description and complex reasoning on 60 in-the-wild images; relative score vs. GPT-4 reference. | Visual chat | 90.0% | 87.6% | 1.01 | 🇺🇸 GPT-4 Vision Preview (1106) | 93.1% | 38 |
| 681 | WSC O Classic Winograd Schema Challenge measuring commonsense coreference. | Coreference reasoning | 90.2% | 98.0%saturated | 1.00 | G Hairuo | 100.0% | 54 |
| 682 | OmniDocBench v1.6 O OmniDocBench v1.6 end-to-end document parsing; Overall score (higher is better) combining text edit, formula CDM and table TEDS. | Document parsing | 90.2% | 96.8%saturated | 0.23 | G Youtu-Parsing-Omni | 97.0% | 42 |
| 683 | MIABench O Multimodal instruction-following benchmark evaluating accuracy on complex image-text tasks. | Multimodal instruction following | 90.2% | 94.7%saturated | 0.53 | 🇺🇸 Gemini 2.5 Pro | 96.0% | 23 |
| 684 | AA Conversational Dynamics: Interruption O Artificial Analysis Conversational Dynamics (Full-Duplex-Bench subset): handling user interruptions (%). | Full-duplex speech interaction | 90.2% | 98.2%saturated | 1.07 | G Eleven Realtime v1 Max | 99.0% | 37 |
| 685 | XSTest O XSTest safety benchmark. | Safety | 90.8% | 96.6%saturated | 0.81 | 🇨🇳 Qwen3.5-9B | 97.6% | 17 |
| 686 | CMath O Chinese mathematics benchmark. | Math (Chinese) | 91.6% | 96.7%saturated | 1.68 | G LLaDA2.0 Flash | 96.9% | 93 |
| 687 | SQuAD v1.1 O Extractive QA from Wikipedia articles. | Reading comprehension | 92.0% | 92.2%saturated | 0.39 | 🇺🇸 OLMo 3 7B | 93.5% | 15 |
| 688 | Ruler 32k O RULER benchmark at 32k context window. | Long-context eval | 92.1% | 95.5%saturated | 1.01 | 🇨🇳 Qwen2.5-72B-Instruct | 96.5% | 35 |
| 689 | AA Conversational Dynamics: Pause handling O Artificial Analysis Conversational Dynamics (Full-Duplex-Bench subset): not barging in during user pauses (%). | Full-duplex speech interaction | 92.2% | 99.6%saturated | 1.27 | 🇺🇸 Grok Voice 1.0 | 100.0% | 37 |
| 690 | ARC-Challenge O Hard subset of AI2 Reasoning Challenge; grade-school science. | Science QA | 92.5% | 97.9%saturated | 1.52 | G Clef-Flash | 98.3% | 378 |
| 691 | CHID O Chinese IDiom cloze-test dataset (CLUE). | Chinese idiom cloze | 92.6% | 90.7%saturated | 1.05 | 🇨🇳 DeepSeek-V2 | 92.7% | 28 |
| 692 | LongFact-Objects O Long-context factuality eval focused on object/entity references; scored as 100 - hallucination rate, higher is better. | Factuality on open-source prompts | 92.6% | 99.1%saturated | 1.06 | 🇺🇸 GPT-5 | 99.2% | 11 |
| 693 | PrOntoQA O Probing ontological reasoning via question answering. | Logical reasoning | 92.6% | 96.9%saturated | 1.49 | G Ling Flash 2.0 | 97.9% | 10 |
| 694 | RACE O English exams for middle and high school. | Reading comprehension | 92.8% | 92.6%saturated | 1.26 | 🇫🇷 Mistral Large 3 Base | 93.3% | 67 |
| 695 | DocVQA O Visual question answering over scanned documents. | Document understanding (VQA) | 92.8% | 96.7%saturated | 0.91 | 🇨🇳 Seed1.5-VL-Thinking | 96.9% | 245 |
| 696 | GSM8K O Grade-school math word problems requiring multi-step reasoning. | Math (grade-school) | 93.0% | 98.2%saturated | 1.98 | G MiMo V2.5 Pro Base | 99.6% | 705 |
| 697 | LongFact-Concepts O Long-context factuality eval focused on conceptual statements; scored as 100 - hallucination rate, higher is better. | Factuality on open-source prompts | 93.1% | 99.2%saturated | 1.08 | 🇺🇸 GPT-5.2 (high) | 99.7% | 11 |
| 698 | WebSrc O Webpage question answering (SQuAD F1). | Web QA | 93.6% | 96.8%saturated | 0.81 | 🇺🇸 Holo1.5-72B | 97.2% | 18 |
| 699 | ScienceQA O Multiple-choice science questions with images, diagrams, and text context. | Science QA (multimodal) | 94.0% | 95.8%saturated | 1.07 | G Youtu-VL-4B-Instruct | 97.0% | 34 |
| 700 | BFCL (decision) O Yes/no per supplied function on BFCL cases; case exact accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Function calling | 94.3% | 98.2%saturated | 1.53 | G Clef-Flash | 98.8% | 38 |
| 701 | Ruler 16k O RULER benchmark at 16k context window. | Long-context eval | 94.4% | 96.0%saturated | 1.21 | 🇨🇳 Qwen2.5-72B-Instruct | 97.7% | 28 |
| 702 | Do-Not-Answer O Evaluates a model's ability to refuse unsafe or disallowed requests. | Safety / refusal | 94.4% | 97.9%saturated | 0.20 | 🇺🇸 Llama 2 7B Chat | 99.4% | 12 |
| 703 | COPA O Choice of Plausible Alternatives. | Causal reasoning | 95.2% | 99.0%saturated | 1.02 | G Vega v2 | 99.4% | 41 |
| 704 | Ruler 8k O RULER benchmark at 8k context window. | Long-context eval | 95.3% | 96.6%saturated | 1.04 | 🇨🇳 Qwen2.5-72B-Instruct | 97.2% | 28 |
| 705 | SciQ O Multiple choice science questions. | Science QA | 95.9% | 95.1%saturated | 0.67 | 🇨🇳 Qwen3 8B Base | 96.1% | 30 |
| 706 | Ruler 4k O RULER benchmark at 4k context window. | Long-context eval | 96.3% | 97.3%saturated | 0.77 | 🇨🇳 Qwen2.5 14B Instruct | 97.7% | 17 |
| 707 | HHEM v2.1 O Hughes Hallucination Evaluation Model (Vectara) — lower is better. | Hallucination detection | 96.4% | — | 0.45 | G AntGroup Finix_S1_32b | ↓ 0.6% | 21 |
| 708 | AlpacaEval 1.0 O Original AlpacaEval: win rate against text-davinci-003 judged by GPT-4. | Instruction following | 96.5% | 96.5%saturated | 1.23 | 🇺🇸 GPT-4 Turbo (11-06) | 97.7% | 45 |
| 709 | StrongREJECT O StrongREJECT jailbreak benchmark; score = share of forbidden prompts safely handled. | Safety (jailbreak refusal) | 97.3% | 99.5%saturated | 0.47 | 🇨🇳 Kimi-K2.6 Thinking | 99.8% | 17 |
| 710 | HarmBench O Harmfulness and safety compliance benchmark across a variety of risky prompts. | Safety | 97.5% | 99.6%saturated | 0.72 | 🇺🇸 GPT-realtime-1.5 | 100.0% | 12 |
| 711 | ARC-Easy O Easier subset of AI2 Reasoning Challenge. | Science QA | 97.6% | 99.5%saturated | 1.59 | G Clef-Omni | 99.8% | 172 |
| 712 | AA Conversational Dynamics: Turn-taking O Artificial Analysis Conversational Dynamics (Full-Duplex-Bench subset): smooth turn-taking (%). | Full-duplex speech interaction | 97.8% | 100%saturated | 0.62 | 🇺🇸 GPT Realtime (Aug 2025) | 100.0% | 37 |
| 713 | VoiceBench AdvBench O VoiceBench adversarial safety evaluation. | VoiceBench | 98.3% | 99.7%saturated | 0.75 | 🇺🇸 NemotronLabs VoiceChat 11B | 100.0% | 18 |
| 714 | LMArena Text O Chatbot Arena text evaluation (average win rate). | Crowd eval (text) | — | — | — | 🇺🇸 Gemini 4 Argon (high) | 1524 | 429 |
| 715 | Meta Score Overall O Overall Meta Score across the benchmarks of all Meta Score categories. | Composite capability index | — | 35.3% | — | 🇨🇳 Qwen3.8-Max | 51.7% | 303 |
| 716 | Meta Score STEM O | Composite capability index | — | 64.0% | — | 🇨🇳 Qwen3-VL-235B-A22B Instruct | 100.0% | 226 |
| 717 | DesignArena O Leaderboard tracking generative design systems across layout, branding, and marketing tasks. | Generative design | — | — | — | 🇺🇸 Claude Sonnet 4.5 (Thinking) | 1410 | 199 |
| 718 | Meta Score Text O | Composite capability index | — | 60.6% | — | 🇺🇸 Claude Opus 4.5 | 85.7% | 197 |
| 719 | Meta Score Math O | Composite capability index | — | 65.7% | — | 🇨🇳 Qwen3-VL-235B-A22B Instruct | 100.0% | 175 |
| 720 | LMArena Vision O Chatbot Arena vision evaluation leaderboard (ELO ratings). | Crowd eval (vision) | — | — | — | 🇺🇸 Claude Fable 5 (high) | 1308 | 165 |
| 721 | Meta Score Code O | Composite capability index | — | 64.5% | — | 🇺🇸 Claude Opus 4.5 | 100.0% | 163 |
| 722 | MT-Bench O Multi-turn chat evaluation via GPT-4 grading. | Chat ability | — | 82.1% | — | 🇺🇸 Apriel Nemotron 15B Thinker | 85.7% | 149 |
| 723 | Meta Score Visual O | Composite capability index | — | 79.9% | — | 🇺🇸 Gemini 3 Pro | 100.0% | 103 |
| 724 | GDPval-AA v2 Elo O Artificial Analysis GDPval-AA v2 leaderboard Elo (re-based; not comparable to v1 GDPval-AA Elo). | Office tasks | — | — | — | 🇺🇸 Claude Opus 5.5 | 1866 | 100 |
| 725 | AlignBench O Benchmark for instruction-following quality and alignment behavior. | Alignment and instruction following | — | — | — | 🇨🇳 Hunyuan-Large-Instruct | 8.30 | 98 |
| 726 | MME Elo O Elo-style scoring for the MME multimodal evaluation benchmark. | Multimodal perception | — | — | — | 🇨🇳 InternVL2.5-78B | 2495 | 86 |
| 727 | EQ-Bench 3 O A benchmark measuring emotional intelligence in challenging roleplays, judged by Sonnet 3.7. | Emotional intelligence (roleplay) | — | — | — | 🇺🇸 Claude Fable 5 | 2050 | 83 |
| 728 | Meta Score OCR O | Composite capability index | — | 53.9% | — | 🇺🇸 o3 (low) | 80.0% | 82 |
| 729 | Codeforces O Competitive programming performance on Codeforces problems (ELO). | Competitive programming | — | — | — | 🇨🇳 DeepSeek V4.1-Flash | 3471 | 78 |
| 730 | GLUE O General Language Understanding Evaluation. | Multi-task NLU | — | 91.2%saturated | — |  🇺🇸 Turing ULR v6 | 91.3% | 76 |
| 731 | Meta Score Agent O | Composite capability index | — | 73.6% | — | 🇺🇸 Claude Opus 4.5 | 100.0% | 76 |
| 732 | AA-Omniscience O Benchmark measuring factual recall and hallucination across economically relevant domains. | Knowledge and hallucination | — | — | — | 🇺🇸 Claude Fable 5 | 40.0 | 73 |
| 733 | WebDev Arena O Arena evaluation for autonomous web development agents. | Web development agents | — | — | — | 🇺🇸 Gemini 3.7 Flash | 1588 | 57 |
| 734 | Meta Score Safety O | Composite safety index | — | 64.6% | — | G Granite 3.3 8B Instruct | 70.0% | 53 |
| 735 | LiveCodeBench Pro O LiveCodeBench Pro evaluates competitive programming performance across Codeforces, ICPC, and IOI contests. Elo rating, higher is better. | Competitive coding problems from Codeforces, ICPC, and IOI | — | — | — | 🇺🇸 Gemini 3 Deep Think | 3298 | 46 |
| 736 | GDPVal-AA Elo O GDPVal Artificial Analysis Elo rating for office-style tasks. | Office tasks | — | — | — | 🇺🇸 Claude Fable 5 | 1932 | 44 |
| 737 | AA Speech-to-Speech Time to First Audio O Artificial Analysis speech-to-speech time to first audio in seconds (lower is better). | Voice latency | — | — | — | G Raon-SpeechChat-9B | ↓ 0.0 s | 39 |
| 738 | ContractNLI O Document-level natural language inference over contracts; macro-F1 (Jev Decision Index suite, posed as typed decisions). | Legal NLI | — | — | — | G Clef-Flash | 84.3 | 39 |
| 739 | RAGTruth O Response-level hallucination detection for RAG answers; F1 on the hallucinated class. | Hallucination detection | — | — | — | G pplx-decider-v1-27b | 88.8 | 39 |
| 740 | ACOS (decision) O Aspect-category-opinion-sentiment decisions; per-review F1. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Aspect sentiment | — | — | — | G pplx-decider-v1.2 | 40.9 | 38 |
| 741 | Amazon ESCI O Shopping query/product relevance (Exact, Substitute, Complement, Irrelevant); macro-F1 (Jev Decision Index suite, posed as typed decisions). | Product search relevance | — | — | — | G pplx-decider-v1.2 | 62.4 | 38 |
| 742 | ANLI O Adversarial NLI (entailment / neutral / contradiction); macro-F1 (Jev Decision Index suite, posed as typed decisions). | Natural language inference | — | — | — | G AutoTrust JEV-27B | 77.2 | 38 |
| 743 | BRIGHT (decision) O Rate candidate documents for BRIGHT queries; nDCG@10. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Reasoning-intensive retrieval | — | — | — | G GLiDE (no thinking, 28B) | 50.1 | 38 |
| 744 | FinEntity (decision) O Entity-level financial sentiment; macro-F1. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Financial sentiment | — | — | — | G Clef-Flash | 97.1 | 38 |
| 745 | NLI4CT O NLI over clinical trial reports (SemEval NLI4CT); macro-F1 (Jev Decision Index suite, posed as typed decisions). | Clinical NLI | — | — | — | G vLLM-SR d3 | 86.0 | 38 |
| 746 | RouterBench (decision) O Choose which LLM to route a query to; selected quality (quality objective). Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Model routing | — | — | — | G Kev 9B | 80.0 | 38 |
| 747 | SGD/SGD-X (decision) O Schema-Guided Dialogue (SGD-X) intent/slot decisions; macro-F1. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Dialogue state tracking | — | — | — | G Bespoke-Nimble-9B-v3 | 71.8 | 38 |
| 748 | ToolRet (decision) O Rate usefulness of 32 retrieved tools per request; nDCG@10. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Tool retrieval | — | — | — | G Clef | 69.2 | 38 |
| 749 | VAST O VAST zero-shot stance detection; macro-F1 (Jev Decision Index suite, posed as typed decisions). | Stance detection | — | — | — | G pplx-decider-v1.2 | 80.2 | 38 |
| 750 | BANKING77 O Bank customer intent among 77 classes; macro-F1 (Jev Decision Index suite, posed as typed decisions). | Intent classification | — | — | — | G Clef-Omni | 94.8 | 36 |
| 751 | CLINC150+OOS O CLINC150 intent detection with out-of-scope class; macro-F1 (Jev Decision Index suite, posed as typed decisions). | Intent classification | — | — | — | G Clef-Omni | 97.7 | 36 |
| 752 | MMHal-Bench (Score) O MMHal-Bench GPT-judged response quality score (0-6) for hallucination in vision-language models. | Multimodal hallucination | — | — | — | 🇨🇳 Qwen3.8-27B | 5.03 | 36 |
| 753 | iSarcasmEval O SemEval-2022 iSarcasmEval track A (English) sarcasm F1 (Jev Decision Index suite, posed as typed decisions). | Sarcasm detection | — | — | — | G pplx-decider-v1.2 | 74.7 | 35 |
| 754 | METR O METR evaluates AI agents on long-horizon coding and agentic tasks, measuring autonomous task completion time. | Long task benchmark | — | — | — | 🇺🇸 Claude Opus 4.5 | 4.8 h | 33 |
| 755 | WMT20 En-Ja O WMT20 English-to-Japanese news translation (Swallow eval, BLEU). | Machine translation | — | — | — | 🇺🇸 GPT-OSS-Swallow-120B-SFT-v0.1 | 27.0 | 31 |
| 756 | WMT20 Ja-En O WMT20 Japanese-to-English news translation (Swallow eval, BLEU). | Machine translation | — | — | — | 🇺🇸 GPT-5 mini | 23.9 | 31 |
| 757 | Real5-OmniDocBench O OmniDocBench v1.5 pages re-captured under five real-world conditions (scanning, warping, screen photography, illumination, skew); Overall score. | Document parsing (real-world distortions) | — | 92.8%saturated | — | G OvisOCR3 | 93.6% | 30 |
| 758 | AA Speech Agent Arena O Artificial Analysis Speech Agent Arena: blind pairwise voice-agent conversations, overall Elo (GPT-Realtime-1.5 anchored at 1000). | Voice agent preference | — | — | — | 🇺🇸 Gemini 3.1 Flash Live | 1095 | 29 |
| 759 | SuperGLUE O Suite of harder NLU tasks beyond GLUE. | Multi-task NLU | — | 91.1%saturated | — | G Hairuo | 91.4% | 29 |
| 760 | MM-MT-Bench O Multi-turn multimodal instruction following benchmark evaluating dialogue quality and helpfulness. | Multimodal instruction following | — | — | — | 🇨🇳 Qwen3-VL-235B-A22B Thinking | 8.50 | 26 |
| 761 | OmniDocBench (Edit Distance) O OmniDocBench v1.0 overall normalized edit distance (x100), lower is better. | Document understanding | — | — | — | G Gundam-M | ↓ 12.3% | 25 |
| 762 | OmniDocBench-CN O Chinese subset of OmniDocBench focusing on OCR-grounded document comprehension and reasoning. | Document understanding (Chinese) | — | — | — | G PP-StructureV3 | ↓ 13.6% | 25 |
| 763 | MMBench-Video O Long-form multi-shot video understanding with GPT-4-judged free-form answers; overall mean score on a 0-3 scale. | Video QA | — | — | — | 🇨🇳 InternVL2.5-78B | 1.97 | 24 |
| 764 | AA-Briefcase O Artificial Analysis benchmark of long-horizon multi-week knowledge-work projects; Elo from rubric and pairwise judging. | Knowledge work | — | — | — | 🇺🇸 Claude Sonnet 5.5 | 1824 | 23 |
| 765 | Vending-Bench 2 O Long-horizon agentic task benchmark evaluating sustained goal completion. | Long-horizon agentic tasks | — | — | — | 🇺🇸 Claude Opus 4.7 | $10,937 | 23 |
| 766 | Meta Score Writing O | Composite capability index | — | 51.1% | — | 🇨🇳 Qwen3-235B-A22B-Instruct-2507 | 60.0% | 22 |
| 767 | MMBench-GUI L2 O MMBench-GUI level-2 element grounding across Windows, macOS, Linux, iOS, Android and Web. | GUI grounding | — | 84.6% | — | G MAI-UI-8B | 88.8% | 21 |
| 768 | UI-Vision O Desktop GUI grounding benchmark (basic, functional, spatial) for computer-use agents. | GUI grounding | — | 37.0% | — | G POINTS-GUI-G-8B | 49.9% | 20 |
| 769 | Creative Story‑Writing Benchmark V3 O Story writing benchmark evaluating creativity, coherence, and style (v3). | Creative writing | — | — | — | 🇨🇳 Kimi-K2-Instruct-0905 | 8.75 | 18 |
| 770 | Japanese MT-Bench O Japanese adaptation of MT-Bench multi-turn conversational benchmark, LLM-as-judge score on a 1-10 scale. | Multi-turn chat (Japanese) | — | — | — | 🇺🇸 GPT-5.4 | 8.87 | 18 |
| 771 | Online-Mind2web O Online web automation and task execution benchmark. | Web automation | — | 81.5% | — | 🇺🇸 GPT-5.4 | 92.8% | 18 |
| 772 | ASR LibriSpeech Other O Automatic speech recognition benchmark on LibriSpeech other split. | ASR | — | — | — | G Ming-flash-omni 2.0 | ↓ 2.2% | 17 |
| 773 | OCRBench-ELO O OCR benchmark using ELO rating system to rank model performance on text extraction tasks. | OCR (ELO ranking) | — | — | — | 🇺🇸 Gemini 3 Pro | 909 | 17 |
| 774 | OK-VQA O Visual question answering requiring outside knowledge. | Knowledge-based VQA | — | 62.8% | — | G Emu2-Chat | 64.8% | 17 |
| 775 | PutnamBench O 672 Putnam competition problems formalized in Lean 4; score = share of problems proved. | Formal theorem proving | — | 80.9% | — | G Aleph Prover | 99.4% | 17 |
| 776 | MiniF2F pass@32 O MiniF2F competition benchmark pass@32 accuracy. | Math competition | — | 86.6% | — | 🇨🇳 LongCat-Flash-Prover | 93.9% | 16 |
| 777 | TIIF-Bench Long O TIIF-Bench long prompt score for text-to-image generation. | Text-to-image | — | — | — | G Seedream 4.5 | 88.5 | 16 |
| 778 | TIIF-Bench Short O TIIF-Bench short prompt score for text-to-image generation. | Text-to-image | — | — | — | G Nano Banana 2.0 | 91.0 | 16 |
| 779 | ASR LibriSpeech Clean O Automatic speech recognition benchmark on LibriSpeech clean split. | ASR | — | — | — | 🇨🇳 Qwen3-Omni-30B-A3B-Instruct | ↓ 1.2% | 15 |
| 780 | ParseBench O Document-parsing benchmark covering tables, charts, content faithfulness, semantic formatting and visual grounding; overall over 5 categories. | Document parsing | — | 73.5% | — | G LightOnOCR-3-4B | 75.1% | 15 |
| 781 | ASR Common Voice 8 (Japanese) O Japanese automatic speech recognition on the Common Voice 8.0 Japanese test set (character error rate). | ASR | — | — | — | 🇺🇸 LFM2.5-Audio-1.5B-JP | ↓ 4.4% | 14 |
| 782 | ASR JSUT Basic5000 O Japanese automatic speech recognition on JSUT basic5000 (character error rate). | ASR | — | — | — | G Whisper-large-V3 | ↓ 7.1% | 14 |
| 783 | ASR ReazonSpeech O Japanese automatic speech recognition on the ReazonSpeech held-out test set (character error rate). | ASR | — | — | — | G ReazonSpeech-NeMo-v2 | ↓ 11.2% | 14 |
| 784 | BoB-HVR O Hard, Versatile, and Relevant composite score across eight capability buckets. | Composite capability index | — | — | — | 🇺🇸 Gemini 3 Pro | 18.3 | 14 |
| 785 | DPG-Bench O DPG-Bench score for text rendering in image generation. | Text rendering | — | — | — | G Seedream 4.5 | 88.6 | 14 |
| 786 | MathBench-A O MathBench application split covering arithmetic, primary, middle, high school and college levels (overall score). | Math (multi-level) | — | 63.0% | — | 🇺🇸 GPT-4o-2024-05-13 | 70.9% | 14 |
| 787 | OneIG-Bench EN O OneIG-Bench English subset score for text-to-image generation. | Text-to-image | — | — | — | G Nano Banana 2.0 | 0.578 | 14 |
| 788 | StreamingBench O Streaming video understanding benchmark. | Streaming video | — | 73.6% | — | G Seed1.8 | 84.4% | 14 |
| 789 | VoiceBench WildVoice O VoiceBench evaluation on WildVoice dataset. | VoiceBench | — | 92.2%saturated | — | 🇺🇸 Gemini 2.5 Pro | 93.4% | 14 |
| 790 | Full-Duplex-Bench V1 Turn-taking Latency O Full-Duplex-Bench v1 smooth turn-taking latency in seconds (lower is better). | Full-duplex speech interaction | — | — | — | G PersonaPlex 7B v1 | ↓ 0.1 s | 13 |
| 791 | ProofNet (Test) O ProofNet test split: undergraduate-level mathematics theorems formalized in Lean 4; score = share proved. | Formal theorem proving | — | 31.9% | — | 🇨🇳 LongCat-Flash-Prover | 47.3% | 13 |
| 792 | SimpleQA (with search) O SimpleQA answered with web search / tool use enabled (agentic setting), not closed-book. | QA | — | 94.4%saturated | — | 🇨🇳 DeepSeek V3.2-Exp | 97.1% | 13 |
| 793 | BBQ O Bias Benchmark for Question Answering evaluating social biases across contexts. | Bias evaluation | — | 21.9% | — | 🇫🇷 Mixtral 8x 7B | 56.0% | 12 |
| 794 | FinSearchComp-CN O Chinese financial search and comprehension benchmark measuring retrieval-grounded reasoning over regional financial content. | Financial retrieval (Chinese) | — | 51.6% | — | G doubao-1-5-vision-pro | 54.2% | 12 |
| 795 | Hungarian National HS Math Exam O Hungarian national high-school mathematics finals (2023) graded by humans; held-out math reasoning test. | Math (held-out exam) | — | 59.8% | — | 🇺🇸 GPT-4 | 68.0% | 12 |
| 796 | ProverBench O ProverBench (DeepSeek-Prover-V2): 325 formalized problems incl. AIME and textbook problems in Lean 4; score = share proved. | Formal theorem proving | — | 54.2% | — | 🇨🇳 LongCat-Flash-Prover | 66.5% | 12 |
| 797 | TydiQA O Typologically diverse QA across languages. | Cross-lingual QA | — | 36.6% | — | G InternLM-20B | 43.2% | 12 |
| 798 | ARC Average O Average accuracy across ARC-Easy and ARC-Challenge. | Science QA (average) | — | 56.0% | — |  🇺🇸 SmolLM2 1.7B Pretrained | 60.5% | 11 |
| 799 | JevBench public (hard) O Accuracy on the hard tier of JevBench's public set. | Typed decisions | — | 73.3% | — | G H2O-Lightning-4B | 76.6% | 11 |
| 800 | LBV1-QA O Vision-language QA benchmark v1. | Vision-language | — | 71.4% | — | 🇺🇸 GPT-5 | 73.7% | 11 |
| 801 | LBV2 O Vision-language benchmark v2. | Vision-language | — | 60.7% | — | 🇺🇸 Gemini 2.5 Pro | 65.7% | 11 |
| 802 | MathBench (ZH) O Chinese portion of MathBench as reported in the InternLM-Math paper. | Math (Chinese) | — | 43.6% | — | 🇨🇳 Qwen-72B-Chat | 47.8% | 11 |
| 803 | OneIG-Bench ZH O OneIG-Bench Chinese subset score for text-to-image generation. | Text-to-image | — | — | — | G Nano Banana 2.0 | 0.567 | 11 |
| 804 | AA-Omniscience-Public O AA-Omniscience Index on the publicly released question subset (index from -100 to 100). | Knowledge and hallucination | — | — | — | 🇨🇳 GLM-5.3 | 22.6 | 10 |
| 805 | ActivityNet-QA O Open-ended QA on ActivityNet videos; GPT-judged accuracy. | Video QA (open-ended) | — | 58.3% | — | 🇺🇸 GPT-4o | 61.9% | 10 |
| 806 | Business Arena O Simulated business-operation arena; final profit/value in USD achieved by the agent. | Business simulation | — | — | — | 🇺🇸 GPT-5.6 Sol | $168,867 | 10 |
| 807 | CursorBench 3.0 O Cursor's coding-agent benchmark on ambiguous multi-file tasks from real Cursor sessions (v3.0, used in the Composer 2 technical report as CursorBench-3). | Software engineering | — | 59.8% | — | 🇺🇸 GPT-5.4 | 63.9% | 10 |
| 808 | JevBench public (easy) O Accuracy on the 48-item easy tier of JevBench's public set. | Typed decisions | — | 100%saturated | — | G Jev 1.13 | 100.0% | 10 |
| 809 | JevBench public (original/standard) O Accuracy on the 72-item original (standard) tier of JevBench's public set. | Typed decisions | — | 98.3%saturated | — | G Jev 1.13 | 98.6% | 10 |
| 810 | LiveBrowseComp O Live (continuously refreshed) BrowseComp-style benchmark of hard-to-find web information for deep-search agents. | Agentic web browsing | — | 46.5% | — | G XYZ-Aquila-pro | 53.7% | 10 |
| 811 | MathOlympiadBench O Olympiad-level math problems formalized in Lean 4; score = share of theorems proved (Pass@32 in LongCat-Flash-Prover report). | Formal theorem proving | — | 18.8% | — | 🇨🇳 LongCat-Flash-Prover | 35.8% | 10 |
| 812 | MMVP O Benchmark for multimodal video understanding and perception. | Multimodal video perception | — | 84.7% | — | G Seed1.8 | 91.6% | 10 |
| 813 | PinchBench (Best) O PinchBench OpenClaw-style agent task benchmark, best-of-runs score. | Agent tasks (OpenClaw) | — | 90.8%saturated | — | 🇺🇸 Claude Opus 4.6 | 93.3% | 10 |
| 814 | PRBench Legal O PRBench legal subset: rubric-graded professional reasoning tasks in law. | Professional reasoning (legal) | — | 54.8% | — | 🇺🇸 Claude Fable 5 | 57.6% | 10 |
| 815 | WildSpeech-Bench O WildSpeech-Bench: end-to-end speech-in/speech-out evaluation on real-world spoken queries (%). | Spoken dialogue | — | 76.3% | — | 🇺🇸 Gemini 3.1 Pro | 77.7% | 10 |
| 816 | AMO-Bench CH O Chinese subset of AMO-Bench. | Math (competition) | — | 65.1% | — | 🇺🇸 Gemini 3 Pro | 74.9% | 9 |
| 817 | ARC-AGI-3 O ARC-AGI-3 interactive reasoning benchmark; Relative Human Action Efficiency (RHAE) score. | General reasoning | — | 29.3% | — | 🇺🇸 GPT-6 Astra | 99.9% | 9 |
| 818 | BFCL Live v2 O Financial compliance and literacy questions from the BFCL Live v2 benchmark. | Finance QA | — | 77.2% | — | 🇺🇸 o1 Mini | 81.0% | 9 |
| 819 | BFCL v2 o Second release of the BFCL benchmark focusing on functional code correctness and logic. | Code reasoning | — | 73.3% | — | 🇨🇳 MiniCPM3-4B | 76.0% | 9 |
| 820 | CC-OCR (KIE) o Key information extraction track of the CC-OCR benchmark. | Key information extraction | — | 91.3%saturated | — | 🇨🇳 Qwen3-VL-235B-A22B | 95.1% | 9 |
| 821 | CMMMU o Chinese Massive Multi-discipline Multimodal Understanding benchmark. | Chinese multimodal understanding | — | 37.6% | — | 🇺🇸 GPT-4V(ision) | 42.5% | 9 |
| 822 | ComplexFuncBench o Complex function reasoning and synthesis benchmark. | Code reasoning | — | 37.6% | — | 🇺🇸 GPT-4o | 66.5% | 9 |
| 823 | ComplexFuncBench (Audio) O ComplexFuncBench multi-step, constrained function calling with spoken user requests (%). | Spoken function calling | — | 72.0% | — | 🇺🇸 Gemini 3.1 Flash Live (high) | 90.8% | 9 |
| 824 | CursorBench 4.0 o Cursor's coding-agent benchmark on ambiguous multi-file tasks from real Cursor sessions (v4.0). | Software engineering | — | 51.6% | — | 🇺🇸 Claude Opus 5.5 | 57.8% | 9 |
| 825 | CVTG-2K CLIPScore o CVTG-2K CLIPScore for text rendering in image generation. | Text rendering | — | — | — | G Seedream 4.5 | 0.807 | 9 |
| 826 | CVTG-2K NED o CVTG-2K normalized edit distance (NED) for text rendering. | Text rendering | — | — | — | 🇨🇳 GLM-Image | 0.956 | 9 |
| 827 | CVTG-2K Word Accuracy o CVTG-2K word accuracy for text rendering in images. | Text rendering | — | — | — | 🇨🇳 GLM-Image | 0.912 | 9 |
| 828 | DeR2 Bench o Dense retrieval and reasoning benchmark for long-context evaluation. | Long-context reasoning | — | 64.1% | — | 🇺🇸 GPT-5.2 (high) | 69.0% | 9 |
| 829 | DSBench-Hard (internal) o DeepSeek internal test set of difficult coding-agent problems. | Coding | — | 66.8% | — | 🇺🇸 Claude Opus 4.8 | 71.7% | 9 |
| 830 | Ego3D-Bench (RMSE) o Ego-centric multi-view 3D spatial reasoning benchmark on outdoor driving scenes; metric = RMSE of numeric answers (lower is better). | Driving spatial reasoning | — | — | — | 🇨🇳 Qwen-Drive-1.0-4B | ↓ 7.78 | 9 |
| 831 | EgoTempo o Egocentric video temporal reasoning benchmark. | Egocentric temporal reasoning | — | 68.0% | — | G Seed2.0 Pro | 71.8% | 9 |
| 832 | Encyclo-K o Encyclopedic knowledge evaluation benchmark. | Encyclopedic knowledge | — | 63.9% | — | G Seed2.0 Pro | 65.7% | 9 |
| 833 | Flickr30K Captions o Flickr30K image captioning (Karpathy test split, CIDEr). | Image captioning | — | — | — | 🇨🇳 Qwen-VL | 85.8 | 9 |
| 834 | fr-bench-pdf2md o French PDF-to-Markdown conversion benchmark (baseline, forms, graphics, handwritten, long tables, multi-column, tiny text); overall score. | Document parsing | — | 69.3% | — | G LightOnOCR-3-4B | 74.1% | 9 |
| 835 | KGC-Safety o Korean safety benchmark evaluating harmfulness and compliance. | Safety (Korean) | — | 92.6%saturated | — | G K-EXAONE 2.0 | 99.8% | 9 |
| 836 | Ko-LongBench o Long-context understanding benchmark in Korean. | Korean long-context | — | 89.4% | — | 🇨🇳 DeepSeek V4-Pro-Thinking (max) | 91.4% | 9 |
| 837 | KoMT-Bench o Korean multi-turn chat evaluation benchmark. | Korean chat ability | — | — | — | 🇨🇳 Qwen3-30B-A3B-Instruct-2507 | 8.49 | 9 |
| 838 | KORBench o Korean reasoning benchmark evaluating diverse reasoning capabilities. | General reasoning | — | 77.4% | — | 🇺🇸 GPT-5.2 (high) | 79.2% | 9 |
| 839 | LongText-Bench EN o LongText-Bench English subset score for text rendering. | Text rendering | — | — | — | G Seedream 4.5 | 0.989 | 9 |
| 840 | LongText-Bench ZH o LongText-Bench Chinese subset score for text rendering. | Text rendering | — | — | — | G Seedream 4.5 | 0.987 | 9 |
| 841 | MBPP-Ko o Korean translation of MBPP code generation benchmark. | Code generation (Korean) | — | 64.4% | — | 🇨🇳 Qwen3-30B-A3B | 66.8% | 9 |
| 842 | MDPBench o Document parsing benchmark (reported in OvisOCR3 release). | Document parsing | — | 82.8% | — | G OvisOCR3 | 88.8% | 9 |
| 843 | MixEval Hard o Hard subset of MixEval covering diverse reasoning tasks. | Multi-task reasoning (hard) | — | 42.3% | — | 🇨🇳 Qwen3.5-9B | 50.3% | 9 |
| 844 | MMAR O MMAR: deep reasoning over speech, sound, music and their mix (%). | Audio reasoning | — | 81.7% | — | G StepAudio 3 Realtime | 86.5% | 9 |
| 845 | MMLU Arabic O Arabic-language variant of MMLU evaluating knowledge and reasoning. | Arabic knowledge and reasoning | — | 69.1% | — | 🇨🇳 Qwen 2.5 72B | 74.1% | 9 |
| 846 | MMLU Pro MCF o MMLU-Pro common format (MCF) few-shot evaluation. | Multi-domain knowledge (few-shot) | — | 23.5% | — | 🇨🇳 Qwen3-4B-Base | 41.1% | 9 |
| 847 | MRCR 1M-8N o OpenAI MRCR v2, 8 needles, 512K-1M context bucket. | Long-context reasoning | — | 82.9% | — | G Muse Spark 1.3 | 98.1% | 9 |
| 848 | OCRBench (KIE) o Key information extraction subset of OCRBench (v1). | Key information extraction | — | 92.9%saturated | — | 🇺🇸 Gemini 3.1 Pro | 96.0% | 9 |
| 849 | OmniSpatial o Spatial understanding and reasoning benchmark (OmniSpatial). | Spatial reasoning | — | 49.7% | — | 🇨🇳 GLM-4.6V | 52.0% | 9 |
| 850 | ProcBench o Procedural reasoning benchmark evaluating step-by-step logical reasoning. | Procedural reasoning | — | 93.3%saturated | — | G Seed2.0 Pro | 96.6% | 9 |
| 851 | ProofBench Basic o Entry-level mathematical proof benchmarking set. | Mathematical proofs | — | 77.2% | — | 🇨🇳 DeepSeekMath-V2-Heavy | 99.0% | 9 |
| 852 | PureDocBench o Document parsing benchmark (reported in OvisOCR3 release). | Document parsing | — | 76.2% | — | G OvisOCR3 | 82.3% | 9 |
| 853 | QVHighlights o Video moment retrieval / highlight detection benchmark; R1@0.5. | Video temporal grounding | — | 71.6% | — | G Nova 2 Lite | 77.2% | 9 |
| 854 | Scale AI Multi Challenge O Scale AI Multi Challenge crowd-evaluated instruction following benchmark. | Chat & instruction following | — | 58.6% | — | 🇨🇳 Qwen3.5-122B-A10B | 61.5% | 9 |
| 855 | SVG-Bench o MiniMax internal SVG generation benchmark (build from scratch or edit from text/image inputs), VLM-verified rendering accuracy. | SVG generation | — | 62.4% | — | 🇺🇸 Claude Sonnet 4.6 | 64.1% | 9 |
| 856 | SWE-Bench (OpenHands) O SWE-Bench results using the OpenHands autonomous coding agent. | Code repair | — | 60.6% | — | 🇨🇳 Qwen3.5-35B-A3B | 69.2% | 9 |
| 857 | TabFact o Table-based fact verification (MMTab subset). | Table fact verification | — | 89.3% | — | 🇨🇳 Kimi-K2.5 | 90.6% | 9 |
| 858 | TAU2-Bench o Aggregate tool-augmented agent evaluation across airline, retail, and telecom scenarios (TAU2). | Agent tasks | — | 87.1% | — | 🇺🇸 Claude Opus 4.5 | 91.6% | 9 |
| 859 | TOMG-Bench o Text-based open molecule generation benchmark (editing, optimization, customized generation). | Molecule generation | — | 65.3% | — | 🇺🇸 GPT-5.5 | 69.9% | 9 |
| 860 | VCRBench o Visual commonsense reasoning benchmark. | Visual commonsense reasoning | — | 51.9% | — | G Seed1.8 | 59.8% | 9 |
| 861 | VideoReasonBench o Video reasoning benchmark assessing temporal and causal understanding. | Video reasoning | — | 64.5% | — | G Seed2.0 Pro | 77.8% | 9 |
| 862 | ViSpeak o Video-grounded dialogue and description benchmark. | Video dialogue | — | 86.5% | — | 🇺🇸 Gemini 3 Pro | 89.0% | 9 |
| 863 | VLAD o Vision-language autonomous-driving QA benchmark (as reported in the Qwen-Drive-1.0 evaluation). | Driving scene QA | — | 60.9% | — | 🇨🇳 Qwen-Drive-1.0-4B | 66.5% | 9 |
| 864 | WaymoQA o Driving-scene multiple-choice QA benchmark built on Waymo data; overall accuracy. | Driving scene QA | — | 65.5% | — | 🇨🇳 Qwen-Drive-1.0-4B | 74.5% | 9 |
| 865 | WaymoQA (Safety) o Safety-critical subset of WaymoQA; accuracy. | Driving scene QA | — | 62.9% | — | 🇨🇳 Qwen-Drive-1.0-4B | 70.7% | 9 |
| 866 | ZeroSCROLLS MuSiQue o ZeroSCROLLS split derived from MuSiQue multi-hop QA. | Long-context reasoning | — | — | — | 🇺🇸 Llama 3.3 70B Instruct | 52.2 | 9 |
| 867 | ZeroSCROLLS SpaceDigest o ZeroSCROLLS SpaceDigest extractive summarization task. | Long-context summarization | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 77.9 | 9 |
| 868 | ZeroSCROLLS SQuALITY o ZeroSCROLLS split based on the SQuALITY long-form summarization benchmark. | Long-context summarization | — | — | — | 🇺🇸 GPT-4 | 22.6 | 9 |
| 869 | ActivityNet-TimeLens o TimeLens-Bench re-annotated ActivityNet Captions split for video temporal grounding (mIoU). | Video temporal grounding | — | 52.3% | — | 🇨🇳 Keye-VL-2.0-30B-A3B | 58.5% | 8 |
| 870 | Blueprint-Bench 2 o Andon Labs agentic spatial-reasoning benchmark: reconstruct 2D floor plans from ~20 interior photos per apartment (50 apartments). | Spatial reasoning | — | 31.9% | — | 🇺🇸 Claude Fable 5 | 38.6% | 8 |
| 871 | Borda Count (Multilingual) o Borda count aggregate ranking across multilingual benchmarks; lower is better. | Aggregate ranking | — | — | — | 🇨🇳 Qwen3-32B | ↓ 2.90 | 8 |
| 872 | CaseReportBench o Clinical information extraction from medical case reports into structured fields. | Medical information extraction | — | 18.6% | — | 🇺🇸 LFM2.5 350M | 32.5% | 8 |
| 873 | Charades-TimeLens o TimeLens-Bench re-annotated Charades-STA split for video temporal grounding (mIoU). | Video temporal grounding | — | 50.3% | — | 🇨🇳 Keye-VL-2.0-30B-A3B | 58.4% | 8 |
| 874 | Cybersecurity Capture The Flag Challenges o Capture-the-flag challenge benchmark evaluating cybersecurity problem-solving skills. | Cybersecurity CTF | — | 89.1% | — | 🇺🇸 GPT-5.6 Sol | 96.7% | 8 |
| 875 | Docmatix (multimodal RAG) o Docmatix document-VQA based multimodal retrieval-augmented QA evaluation. | Multimodal RAG | — | 53.5% | — | G Yuan3.0 Ultra | 67.4% | 8 |
| 876 | DS-1000 o DS-1000: 1000 data-science coding problems across 7 Python libraries. | Data-science code generation | — | 31.3% | — | 🇨🇳 DeepSeek-Coder-33B-base | 40.2% | 8 |
| 877 | DSBench-FullStack (internal) o DeepSeek internal full-stack development test set for coding agents. | Coding | — | 72.5% | — | 🇺🇸 Claude Fable 5 | 77.2% | 8 |
| 878 | FaithJudge (1-Hallu.) o FaithJudge hallucination rate with 1-hallucination metric (lower is better). | Hallucination detection | — | — | — | G Moonlight-Instruct | ↓ 56.0% | 8 |
| 879 | GeneBench Pro o Long-horizon genomics and quantitative-biology analysis tasks. | Biology agent tasks | — | 24.1% | — | 🇺🇸 GPT-6 Astra | 37.1% | 8 |
| 880 | GraphWalks Parents 256K o GraphWalks Parents long-context graph traversal at 256K context (F1). | Long-context reasoning | — | 98.0%saturated | — | 🇺🇸 Claude Mythos 5 | 100.0% | 8 |
| 881 | IOI 2025 o International Olympiad in Informatics 2025 problem set; total score out of 600 points. | Competitive programming | — | — | — | 🇨🇳 Kimi-K2.6 Thinking | 585 | 8 |
| 882 | Kimi Code Bench v2 o Moonshot in-house coding-agent benchmark (Kimi Code Bench 2.0): realistic software engineering tasks across 10+ languages and production tech stacks. | Agentic coding | — | 71.1% | — | 🇺🇸 Claude Fable 5 | 76.9% | 8 |
| 883 | LogiQA o Reading comprehension with logical reasoning. | Logical reasoning | — | 22.2% | — | G Pythia 70M | 23.5% | 8 |
| 884 | MCPMark-Verified O Human-verified edition of MCPMark covering Notion, GitHub, Filesystem, Postgres and Playwright MCP servers. | Agent tool-use (MCP) | — | 90.2%saturated | — | 🇨🇳 Kimi-K3 | 96.1% | 8 |
| 885 | MLE-Bench o OpenAI MLE-Bench: Kaggle-style machine-learning engineering competitions solved by agents. | ML engineering | — | 54.3% | — | 🇺🇸 Claude Sonnet 5 | 66.9% | 8 |
| 886 | MMAU-Pro o MMAU-Pro: challenging audio understanding and reasoning benchmark (speech, sound, music). | Audio reasoning | — | 69.3% | — | 🇨🇳 Qwen3.5-Omni-Plus | 70.0% | 8 |
| 887 | MolBench-bind o MolBench binding-affinity subset: scientific agentic tasks on molecular binding prediction. | Molecular science agents | — | 53.0% | — | 🇺🇸 GPT-5.5 | 62.2% | 8 |
| 888 | MRCR 512K-8N o OpenAI MRCR v2, 8 needles, 256K-512K context bucket. | Long-context reasoning | — | 92.2%saturated | — | 🇺🇸 GPT-6 Astra | 100.0% | 8 |
| 889 | MRCR 64K-8N o Multi-Round Coreference Resolution benchmark at 64k context with 8 needles. | Long-context reasoning | — | 61.5% | — | 🇺🇸 GPT-5.2 | 92.0% | 8 |
| 890 | Natural2Code o Natural language to code benchmark for instruction-following synthesis. | Code generation | — | 84.7% | — | 🇺🇸 Gemini 2.0 Flash | 92.9% | 8 |
| 891 | OCR-VQA o Visual question answering by reading text in book cover images. | Text-based VQA | — | 72.6% | — | 🇨🇳 Qwen-VL | 75.7% | 8 |
| 892 | OpenAI Internal Computer-Use Safety o OpenAI internal eval reported in the GPT-6 Astra announcement: rate of unsafe actions during computer use (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 2.4% | 8 |
| 893 | OSWorld 2.0 (Offline) O OSWorld 2.0 offline set (v2026.08.08), partial-credit score on long-horizon computer-use workflows without internet access. | GUI agents | — | 70.1% | — | 🇺🇸 GPT-6 Astra | 73.5% | 8 |
| 894 | QVHighlights-TimeLens o TimeLens-Bench re-annotated QVHighlights split for video temporal grounding (mIoU). | Video temporal grounding | — | 64.9% | — | 🇨🇳 Keye-VL-2.0-30B-A3B | 70.1% | 8 |
| 895 | Random Complex Tasks o Randomly constructed complex task environments for agent generalization. | Agentic tasks (random) | — | 33.2% | — | 🇨🇳 LongCat-Flash-Thinking-2601 | 35.8% | 8 |
| 896 | SWE-Bench Pro v2 o SWE-Bench Pro v2 (Scale AI) agentic software-engineering tasks; full set. | Software engineering | — | 86.6% | — | 🇨🇳 Qwen3.8-27B | 94.9% | 8 |
| 897 | SWE-Lancer o Software engineering benchmark using real freelance-style issues. | Code repair (freelance tasks) | — | 47.1% | — | 🇺🇸 GPT-5.1 Codex-Max | 79.9% | 8 |
| 898 | V* (With Tools) o V* visual search benchmark evaluated with a Python code-interpreter tool. | Multimodal reasoning | — | 96.8%saturated | — | 🇺🇸 GPT-5.4 | 98.4% | 8 |
| 899 | WebTailBench o Microsoft long-tail web task benchmark (shopping, travel, comparison and other real-world tasks) introduced with Fara-7B. | Web agents | — | 40.8% | — | 🇺🇸 MolmoWeb-8B | 49.5% | 8 |
| 900 | Workspace Bench Lite o Lite subset of Workspace Bench: workspace tasks with heterogeneous file dependencies. | Agentic office work | — | 69.7% | — | 🇨🇳 GLM-5.3 | 71.7% | 8 |
| 901 | ZeroSCROLLS GovReport o ZeroSCROLLS split based on the GovReport summarization benchmark. | Long-context summarization | — | — | — | G CoLT5 | 41.0 | 8 |
| 902 | ZeroSCROLLS NarrativeQA o ZeroSCROLLS split based on the NarrativeQA reading comprehension benchmark. | Long-context QA | — | — | — | 🇺🇸 Claude v1.3 | 32.6 | 8 |
| 903 | ZeroSCROLLS Qasper o ZeroSCROLLS split based on the Qasper paper QA benchmark. | Long-context QA | — | — | — | G FLAN-UL2 | 56.9 | 8 |
| 904 | ZeroSCROLLS QMSum o ZeroSCROLLS split based on the QMSum meeting summarization benchmark. | Long-context summarization | — | — | — | G CoLT5 | 22.5 | 8 |
| 905 | ZeroSCROLLS QuALITY o ZeroSCROLLS split based on the QuALITY reading comprehension benchmark. | Long-context QA | — | 78.8% | — | 🇺🇸 GPT-4 | 89.2% | 8 |
| 906 | ZeroSCROLLS SummScreenFD o ZeroSCROLLS split based on the SummScreenFD summarization benchmark. | Long-context summarization | — | — | — | G CoLT5 | 20.0 | 8 |
| 907 | AG News o AG News four-class news topic classification. | Topic classification | — | 90.5%saturated | — | G Laya | 92.8% | 7 |
| 908 | AgentBench o Multi-environment agent benchmark (OS, DB, KG, puzzles, household, web shopping, web browsing); overall weighted score. | LLM agents | — | — | — | 🇨🇳 GLM 4 (05-20) | 3.79 | 7 |
| 909 | AgentCompany o Company-level agent reasoning and decision-making benchmark. | Agent reasoning | — | 37.7% | — | 🇺🇸 Claude Sonnet 4.5 | 41.0% | 7 |
| 910 | AutoLogi o AutoLogi benchmark evaluating automated logical reasoning accuracy. | Logical reasoning | — | 88.5% | — | 🇺🇸 Claude Sonnet 4 | 89.8% | 7 |
| 911 | BioReason Coding o Coding variant-effect prediction from DNA (balanced accuracy). | Genomics | — | 64.4% | — | G Carbon-X | 74.9% | 7 |
| 912 | BioReason Non-SNV o Complex (non-SNV) variant-effect prediction from DNA (balanced accuracy). | Genomics | — | 85.4% | — | 🇺🇸 Claude Opus 5.5 (max) | 94.8% | 7 |
| 913 | BrokenArXiv o Benchmark of subtly broken/false statements derived from recent arXiv math papers; models must detect and prove or refute them. | Research math | — | 59.2% | — | 🇺🇸 Claude Opus 5 | 77.7% | 7 |
| 914 | ClawEval (Pass@3) o ClawEval OpenClaw agent benchmark, pass@3 (any of 3 trials succeeds). | Agent tasks (OpenClaw) | — | 80.0% | — | 🇺🇸 Gemini 3.1 Pro | 82.9% | 7 |
| 915 | COCO Detection (F1) o COCO object detection with an MLLM; F1 averaged over IoU 0.5:0.95. | Object detection | — | — | — | G WeVisOne-VL-4B | 57.1 | 7 |
| 916 | CorpFin o Corporate finance reasoning and numerical literacy benchmark. | Corporate finance | — | 70.4% | — | 🇺🇸 Grok 4 Fast | 73.3% | 7 |
| 917 | Dense200 Detection (F1) o Dense200 object detection with an MLLM; F1 averaged over IoU 0.5:0.95. | Object detection | — | — | — | G WeVisOne-VL-4B | 71.3 | 7 |
| 918 | DocFigCapBench Localization F1 o DocFigCapBench: localization F1 of document figures. | Document figure captioning | — | 86.0% | — | G OvisOCR3 | 91.0% | 7 |
| 919 | ForecastBench (decision) o ForecastBench questions answered as probabilities; Brier score x100 (lower is better). Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Forecasting | — | — | — | G Clef-Flash | ↓ 10.6 | 7 |
| 920 | FrontierVEP o Single-base variant-effect prediction benchmark. | Genomics | — | 66.2% | — | 🇺🇸 Claude Opus 5.5 (max) | 76.2% | 7 |
| 921 | GAIA (no file) o GAIA benchmark subset without file inputs. | General AI tasks | — | 77.9% | — | G Step-3.5 Flash 20260204 | 84.5% | 7 |
| 922 | GenomeQA o DNA sequence understanding QA (enhancers, splice sites, organism, histone marks, TF binding, motifs). | Genomics | — | 61.6% | — | 🇺🇸 GPT-6.1 Sol (max) | 67.2% | 7 |
| 923 | GraphWalks BFS 256K-1M o GraphWalks BFS long-context graph traversal at 256K-1M context (F1). | Long-context reasoning | — | 73.6% | — | 🇺🇸 Gemini 4 Argon | 84.2% | 7 |
| 924 | GSM-Symbolic o Symbolic reasoning variant of GSM that tests algebraic manipulation and arithmetic with structured problems. | Math reasoning | — | 83.2% | — | G Granite-4.0-H-Small | 87.4% | 7 |
| 925 | HotpotQA o Explainable multi-hop QA with supporting facts. | Multi-hop QA | — | 37.6% | — | 🇨🇳 Qwen3-0.6B | 64.0% | 7 |
| 926 | JevBench public set o Accuracy on JevBench's 231 public items (easy, original/standard and hard tiers). | Typed decisions | — | 85.6% | — | 🇺🇸 strands-decider-26B-A4B-gemma4-v1-2610 | 89.6% | 7 |
| 927 | Kimi Code Bench 3.0 Long-horizon o Moonshot in-house long-horizon coding-agent benchmark (Kimi Code Bench 3.0, long-horizon split). | Agentic coding | — | 60.2% | — | 🇺🇸 GPT-6 Astra | 66.5% | 7 |
| 928 | Kimi Hallu Bench Hard (With Tools) o Moonshot in-house hard hallucination benchmark with tools: percentage of factual claims containing errors (lower is better). | Hallucination | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 5.5% | 7 |
| 929 | Legal Research Bench o Vals AI Legal Research benchmark of agentic legal research tasks. | Legal research | — | 45.2% | — | 🇺🇸 Claude Fable 5 | 49.5% | 7 |
| 930 | LongCodeBench o Long-context code comprehension and repair benchmark (reported at 1M-token context). | Long-context code understanding | — | 82.0% | — | G Nova 2 Lite | 84.0% | 7 |
| 931 | LVIS Detection (F1) o LVIS object detection with an MLLM; F1 averaged over IoU 0.5:0.95. | Object detection | — | — | — | G WeVisOne-VL-4B | 55.3 | 7 |
| 932 | Medbullets (4 options) o USMLE Step 2/3-style clinical questions, 4-option version. | Medical QA | — | 61.7% | — | G MedMO-8B | 66.5% | 7 |
| 933 | Medbullets (5 options) o USMLE Step 2/3-style clinical questions, 5-option version. | Medical QA | — | 53.4% | — | G MedMO-8B | 60.2% | 7 |
| 934 | MindCube-Tiny o Tiny evaluation subset of MindCube spatial mental-model benchmark. | Spatial reasoning | — | 67.2% | — | G ZDTaichu5.0-9B | 78.3% | 7 |
| 935 | MM-BrowseComp o Multimodal browsing comprehension benchmark. | Multimodal browsing | — | 49.4% | — | 🇺🇸 Claude Opus 4.6 | 59.3% | 7 |
| 936 | MMMU-Med o Medical/health subset of MMMU (validation). | Medical QA | — | 63.8% | — | G MedMO-8B-Next | 69.3% | 7 |
| 937 | MSVD-QA o Open-ended QA on MSVD video clips; GPT-judged accuracy. | Video QA (open-ended) | — | 70.2% | — | G Baichuan-Omni | 72.2% | 7 |
| 938 | NextBase o Next-base DNA sequence prediction (per-base accuracy). | Genomics | — | 32.3% | — | G Carbon-X | 39.4% | 7 |
| 939 | OCW Courses o MIT OpenCourseWare undergraduate STEM problems (from Minerva). | STEM math reasoning | — | 12.9% | — | G Minerva 540B | 17.6% | 7 |
| 940 | OIBench EN o English subset of OIBench for code generation. | Code generation | — | 47.6% | — | 🇺🇸 Gemini 3 Pro | 58.2% | 7 |
| 941 | OmniMedVQA o Large-scale medical VQA across 12 imaging modalities. | Medical QA | — | 88.0% | — | G MedMO-8B-Next | 93.3% | 7 |
| 942 | OpenAI Internal Investment Banking Modeling Tasks o OpenAI internal eval (GPT-5.4/GPT-5.5 announcements): investment-banking spreadsheet modeling tasks. | Finance | — | 85.5% | — | 🇺🇸 GPT-5.5 Pro | 88.6% | 7 |
| 943 | OpenAI Internal Management Consulting Tasks o OpenAI internal eval (GPT-5.6 announcement): management-consulting tasks. | Professional work | — | 36.6% | — | 🇺🇸 GPT-5.6 Sol | 43.2% | 7 |
| 944 | OSWorld 2.1 (Strict) o OSWorld 2.1 strict pass rate (full task success). | GUI agents | — | 42.4% | — | 🇺🇸 Claude Opus 5.5 | 48.7% | 7 |
| 945 | PathVQA o Pathology visual question answering. | Medical QA | — | 57.0% | — | G MedMO-4B-Next | 73.3% | 7 |
| 946 | RiemannBench o Research-level mathematics benchmark (reported in Anthropic Fable 5/Mythos 5 system card). | Research math | — | 67.6% | — | 🇺🇸 Gemini 4 Argon | 76.0% | 7 |
| 947 | SALAD-Bench o Safety Alignment and Dangerous-behavior benchmark evaluating harmful assistance and refusal consistency. | Safety alignment | — | — | — | G Granite-4.1-3B | ↓ 94.0% | 7 |
| 948 | SPAR-Bench o Spatial perception and reasoning benchmark derived from the SPAR dataset (single- and multi-view). | Spatial reasoning | — | 50.4% | — | 🇺🇸 GPT-5.2 | 55.1% | 7 |
| 949 | SWE-Marathon v1.1 o SWE-Marathon v1.1: very long-horizon software engineering tasks (Abundant AI). | Software engineering | — | 45.4% | — | 🇺🇸 Claude Opus 4.8 | 48.8% | 7 |
| 950 | SWE-Perf o Software engineering benchmark focused on performance-oriented fixes. | Code repair | — | 4.2% | — | 🇺🇸 Gemini 3 Pro | 6.5% | 7 |
| 951 | SWT-Bench o Software tool-use benchmark for code tasks. | Code repair | — | 75.9% | — | 🇺🇸 GPT-5.2 Thinking | 80.7% | 7 |
| 952 | ToolACE o ToolACE function-calling evaluation, posed as decisions. | Tool calling | — | 92.8%saturated | — | G Intern-Decision-2B | 96.5% | 7 |
| 953 | Typed Decision o Typed-decision evaluation set used in Intern-Decision model card (Laya Typed Decisions). | Typed decisions | — | 75.0% | — | G Intern-Decision-4B | 80.6% | 7 |
| 954 | Vibe Average o Aggregate Vibe evaluation score. | Vibe evaluation | — | 75.9% | — | G MiniMax M2.1 | 88.6% | 7 |
| 955 | VisDrone Detection (F1) o VisDrone object detection with an MLLM; F1 averaged over IoU 0.5:0.95. | Object detection | — | — | — | G WeVisOne-VL-4B | 44.7 | 7 |
| 956 | VizWiz-VQA o Visual question answering on photos taken by blind people (VQA score). | Accessibility VQA | — | 33.7% | — | 🇨🇳 Qwen-VL-Chat | 38.9% | 7 |
| 957 | WebArena o Realistic self-hosted web environments (shopping, forums, GitLab, CMS, maps) for autonomous web agents; task success rate. | Web agents | — | 64.2% | — | 🇺🇸 Claude Sonnet 4.6 | 65.6% | 7 |
| 958 | WebQuest-MultiQA o Multi-question web search and interaction tasks. | Web agents | — | 56.1% | — | 🇨🇳 GLM-4.5V | 60.6% | 7 |
| 959 | WebQuest-SingleQA o Single-question web search and interaction tasks. | Web agents | — | 72.5% | — | 🇨🇳 GLM-4.6V | 79.5% | 7 |
| 960 | WikiTableQuestions o WikiTableQuestions (MMTab subset, image tables). | Table QA | — | 76.5% | — | 🇺🇸 Gemini 3.1 Pro | 79.6% | 7 |
| 961 | WorldVQA (With Tools) o WorldVQA evaluated with tool access in an agent harness. | Visual world knowledge | — | 53.8% | — | G Step 3.7 Flash | 58.1% | 7 |
| 962 | YC-Bench o YC-Bench: long-horizon startup-management simulation; metric is final assets of the fund in millions of USD. | Long-horizon business simulation | — | — | — | 🇺🇸 Claude Opus 4.7 | $2.2M | 7 |
| 963 | ZeroSCROLLS BookSumSort o ZeroSCROLLS split based on BookSumSort long-form summarization. | Long-context summarization | — | — | — | 🇺🇸 GPT-4 | 60.5 | 7 |
| 964 | AGIEval LSAT-AR o LSAT Analytical Reasoning subset from AGIEval benchmark. | Law exam reasoning | — | 26.0% | — | 🇨🇳 Qwen2.5 32B Base | 30.4% | 6 |
| 965 | Claw-Eval Multimodal o Multimodal subset of Claw-Eval OpenClaw-style agent benchmark. | Multimodal agent tasks | — | 23.4% | — | 🇺🇸 GPT-5.4 | 25.7% | 6 |
| 966 | CrossTabBench Reconstruction TEDS o CrossTabBench: TEDS of reconstructed cross-page tables. | Cross-page tables | — | 57.0% | — | 🇨🇳 Qwen3.8-Max | 91.7% | 6 |
| 967 | CrossTabBench Relation F1 o CrossTabBench: F1 for detecting cross-page table continuation relations. | Cross-page tables | — | 70.0% | — | G OvisOCR3 | 97.1% | 6 |
| 968 | CursorBench 3.2 o Cursor's coding-agent benchmark on ambiguous multi-file tasks from real Cursor sessions (v3.2). | Software engineering | — | 70.2% | — | 🇺🇸 Claude Fable 5.1 | 73.4% | 6 |
| 969 | Daily-Omni o Daily-Omni: audio-visual question answering on everyday videos with temporal alignment. | Audio-visual QA | — | 81.5% | — | 🇺🇸 Gemini 3 Pro | 84.2% | 6 |
| 970 | DetailBench o Evaluates whether LLMs can notice subtle errors and minor inconsistencies in text. | Spot small mistakes | — | 4.3% | — | 🇺🇸 Llama 4 Maverick | 8.7% | 6 |
| 971 | EnterpriseOps-Gym o Agentic benchmark of enterprise operations workflows with tool use. | Enterprise agent tasks | — | 26.6% | — | G Spark-X2.6-4B | 33.6% | 6 |
| 972 | FeTaQA o Free-form table QA (MMTab subset; BLEU). | Table QA | — | 15.4% | — | G Yuan3.0 Ultra | 39.2% | 6 |
| 973 | FORTE o AGI-Eval FORTE general agentic task benchmark. | General agent | — | 75.8% | — | 🇺🇸 GPT-5.5 | 77.8% | 6 |
| 974 | FullStackBench (zh) o Chinese language full-stack development tasks and evaluation. | Full-stack development | — | 58.9% | — | 🇨🇳 Qwen3-235B-A22B | 63.1% | 6 |
| 975 | FutureOmni o FutureOmni: omni-modal (audio-visual) future event forecasting benchmark from video. | Video forecasting | — | 61.6% | — | G MiMo V2 Omni | 66.7% | 6 |
| 976 | Gaokao Math 2026 o Five 2026 Chinese Gaokao math exams (National I, II, Beijing, Shanghai, Tianjin); average score out of 150. | Math exams | — | — | — | 🇨🇳 Qwen3.5-9B | 136 | 6 |
| 977 | GRE Math maj@16 o GRE quantitative section evaluated via majority voting over 16 samples. | Math (standardized tests) | — | 51.7% | — | 🇨🇳 Qwen2 7B | 58.5% | 6 |
| 978 | GSM8K-Ko o Korean translation of the GSM8K grade-school math word problems. | Math (grade-school, Korean) | — | 82.5% | — | 🇨🇳 Qwen3-30B-A3B | 88.1% | 6 |
| 979 | HierText (F1) o HierText text grounding and recognition, F1 mean. | OCR | — | — | — | G WeVisOne-VL-4B | 39.0 | 6 |
| 980 | HiTab o Hierarchical table QA (MMTab subset). | Table QA | — | 60.3% | — | 🇺🇸 GPT-5.1 | 77.8% | 6 |
| 981 | HLE-Diamond o Hard 'Diamond' subset of Humanity's Last Exam, evaluated without tools. | Multidisciplinary reasoning | — | 18.6% | — | G Muse Spark Open | 26.1% | 6 |
| 982 | HumanRef o Referring detection of people in images (HumanRef), F1 mean. | Referring expressions | — | — | — | G WeVisOne-VL-4B | 80.7 | 6 |
| 983 | ICDAR2015 (F1) o ICDAR2015 text grounding and recognition, F1 mean. | OCR | — | — | — | G WeVisOne-VL-4B | 47.8 | 6 |
| 984 | IFEval-Code o Instruction following evaluation for code generation tasks. | Instruction following (code) | — | 24.7% | — | 🇨🇳 Qwen3-32B | 28.0% | 6 |
| 985 | InfoTabs o Natural language inference over infobox tables (MMTab subset). | Table NLI | — | 75.9% | — | G Yuan3.0 Ultra | 89.7% | 6 |
| 986 | Kimi Hallu Bench o Moonshot in-house benchmark: percentage of factual claims containing errors, without tools (lower is better). | Hallucination | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 4.2% | 6 |
| 987 | KoNET o Korean National Educational Test benchmark (elementary to college entrance exams) with images. | Korean exams (multimodal) | — | 79.7% | — | 🇺🇸 Gemini 2.5 Pro | 96.2% | 6 |
| 988 | LegalBench o Suite of legal question answering and reasoning tasks. | Legal QA | — | 81.3% | — | 🇺🇸 Gemma 4 31B (IT) | 83.1% | 6 |
| 989 | LifeSciBench o Life-sciences research workflow benchmark. | Life sciences | — | 56.2% | — | 🇺🇸 GPT-6 Astra | 60.3% | 6 |
| 990 | MEWC o Multi-page End-to-end Web Comprehension benchmark. | Web comprehension | — | 76.1% | — | 🇺🇸 Claude Opus 4.6 | 89.8% | 6 |
| 991 | MLE-bench-30 o 30-competition subset of MLE-bench (Kaggle ML engineering tasks); average position score as used in the Gemini 3.6 Flash model card. | ML engineering | — | 60.4% | — | G Naive-N0.5-Flash | 73.7% | 6 |
| 992 | MM-ClawBench o MiniMax MM-ClawBench (MM Claw): end-to-end OpenClaw-style personal-agent benchmark. | Personal agent tasks | — | 67.5% | — | 🇺🇸 Claude Opus 4.6 | 75.4% | 6 |
| 993 | MPDocBench-Parse Truncated Table TEDS o MPDocBench-Parse: TEDS on truncated (page-split) tables. | Cross-page tables | — | 90.6%saturated | — | G OvisOCR3 | 92.0% | 6 |
| 994 | OctoCodingBench o Coding benchmark across multi-language programming tasks. | Code generation | — | 26.8% | — | 🇺🇸 Claude Opus 4.5 | 36.2% | 6 |
| 995 | OfficeQA Pro v2 o OfficeQA Pro v2: grounded QA over office/enterprise documents requiring files, tools and context. | Enterprise document QA | — | 55.2% | — | 🇨🇳 Kimi-K3.1 Pro | 64.4% | 6 |
| 996 | OpenAI Internal Broken Search Tool Non-Disclosure o OpenAI internal alignment eval: rate of failing to disclose a broken search tool (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 1.5% | 6 |
| 997 | OpenAI Internal Factual Error Rate (Difficult Prompts) o OpenAI internal factuality eval on de-identified ChatGPT conversations where users flagged factual errors; error rate (lower is better). | Factuality | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 3.9% | 6 |
| 998 | OpenAI Internal Reviewer Bypass Attempts o OpenAI internal alignment eval: rate of attempts to bypass a reviewer (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | 6 | |
| 999 | OpenAI Internal Warning Circumvention o OpenAI internal alignment eval: rate of circumventing warnings (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 17.4% | 6 |
| 1000 | PAgentBench o iFlytek in-house Personal Assistant Benchmark: task management, file operations and system interactions. | Personal assistant agents | — | 58.1% | — | G Spark-X2.6-4B | 74.1% | 6 |
| 1001 | PointArena o Language-guided pointing benchmark for multimodal models. | Visual grounding | — | 70.7% | — | 🇺🇸 GPT-6 Astra | 75.4% | 6 |
| 1002 | PubTabNet o Image-based table recognition benchmark from PubMed Central articles; TEDS score on validation set. | Table recognition | — | 91.4%saturated | — | G Infinity-Parser2-Pro | 94.8% | 6 |
| 1003 | Ref-L4 Referring Detection (F1) o Ref-L4 referring object detection, F1 mean over IoU 0.5:0.95. | Referring expressions | — | — | — | G WeVisOne-VL-4B | 78.1 | 6 |
| 1004 | RefCOCO testA Segmentation (cIoU) o RefCOCO testA referring segmentation, cIoU. | Referring segmentation | — | 82.3% | — | G WeVisOne-VL-4B | 84.5% | 6 |
| 1005 | RefCOCO+ testA Segmentation (cIoU) o RefCOCO+ testA referring segmentation, cIoU. | Referring segmentation | — | 78.0% | — | G WeVisOne-VL-4B | 81.1% | 6 |
| 1006 | RefCOCOg Referring Detection (F1) o RefCOCOg test referring object detection (multi-box), F1 mean over IoU 0.5:0.95. | Referring expressions | — | — | — | G WeVisOne-VL-4B | 81.8 | 6 |
| 1007 | RefCOCOg Referring Detection (Rejection) o Rejection rate on non-existent referents for RefCOCOg referring detection. | Referring expressions | — | 31.2% | — | G WeVisOne-VL-4B | 67.4% | 6 |
| 1008 | RefCOCOg test Segmentation (cIoU) o RefCOCOg test referring segmentation, cIoU. | Referring segmentation | — | 75.9% | — | G WeVisOne-VL-4B | 77.8% | 6 |
| 1009 | RobotGrounding (in-house) o ACE Robotics in-house robot grounding benchmark. | Embodied grounding | — | 88.9% | — | G ACE-Brain-0.6-9B | 93.4% | 6 |
| 1010 | SecCodeBench o Benchmark evaluating secure code generation capabilities. | Secure code generation | — | 65.9% | — | 🇺🇸 GPT-5.2 | 68.7% | 6 |
| 1011 | ShareRobot-Affordance o ShareRobot affordance prediction benchmark. | Embodied affordance | — | 42.2% | — | G ACE-Brain-0.6-9B | 48.8% | 6 |
| 1012 | SIFO o Single-turn instruction following benchmark. | Instruction following | — | 50.9% | — | 🇨🇳 Qwen3-VL-30B-A3B Thinking | 66.9% | 6 |
| 1013 | SIFO Multiturn o Multi-turn SIFO benchmark for sustained instruction adherence. | Instruction following | — | 41.6% | — | 🇨🇳 Qwen3-VL-30B-A3B Thinking | 60.3% | 6 |
| 1014 | Spider 2.0-Lite o Spider 2.0-Lite enterprise text-to-SQL over real schemas, scored by executing the query (Interfaze reports the 135-task SQLite subset). | Text-to-SQL | — | 48.5% | — | G Interfaze | 52.9% | 6 |
| 1015 | SROIE (F1) o SROIE text grounding and recognition, F1 mean. | OCR | — | — | — | G WeVisOne-VL-4B | 68.1 | 6 |
| 1016 | SWE-Dev o Software engineering development and bug fixing benchmark. | Code repair | — | 64.7% | — | 🇺🇸 Claude Sonnet 4 | 67.1% | 6 |
| 1017 | SWE-Lancer Diamond o Diamond subset of SWE-Lancer focusing on the hardest freelance-style issues. | Code repair (freelance) | — | 57.6% | — | 🇺🇸 GPT-5.3 Codex | 81.4% | 6 |
| 1018 | SWebBench o iFlytek in-house benchmark for web browsing, information retrieval and research (reported with Spark-X2.6). | Web research agents | — | 43.9% | — | G Spark-X2.6-4B | 71.7% | 6 |
| 1019 | TAT-QA o Hybrid tabular and textual financial QA (MMTab subset). | Table QA | — | 58.7% | — | G Yuan3.0 Ultra | 74.9% | 6 |
| 1020 | TotalText (F1) o TotalText text grounding and recognition, F1 mean. | OCR | — | — | — | G WeVisOne-VL-4B | 52.0 | 6 |
| 1021 | UniMERNet o UniMER-Test benchmark for universal mathematical expression recognition from images. | Formula recognition | — | 96.6%saturated | — | G Infinity-Parser2-Pro | 97.7% | 6 |
| 1022 | VisualPuzzle o Visual puzzle solving benchmark evaluating reasoning and pattern recognition capabilities. | Visual reasoning | — | 62.5% | — | 🇺🇸 Gemini 3 Pro | 71.5% | 6 |
| 1023 | VitaBench 2.0 o VitaBench 2.0: interactive agent tasks in real-world service scenarios (v2.0, not comparable to v1). | Agentic real-world tasks | — | 20.4% | — | G Spark-X2.6-4B | 30.8% | 6 |
| 1024 | VPCT o Visual perception and comprehension test. | Multimodal reasoning | — | 60.0% | — | 🇺🇸 Gemini 3 Pro | 90.0% | 6 |
| 1025 | WeRef (F1) o Tencent WeRef referring object detection benchmark, F1 mean. | Referring expressions | — | — | — | G WeVisOne-VL-4B | 78.7 | 6 |
| 1026 | WeRef (Rejection) o Rejection rate on non-existent referents in WeRef. | Referring expressions | — | 75.3% | — | G WeVisOne-VL-4B | 96.4% | 6 |
| 1027 | ACEBench (ZH) o Chinese split of ACEBench tool-use evaluation. | Tool use (Chinese) | — | 85.8% | — | 🇨🇳 ERNIE 5.0 | 89.6% | 5 |
| 1028 | ACVA o Arabic Cultural Value Alignment benchmark (true/false statements about Arab culture). | Arabic culture | — | 80.4% | — | G Fanar-2-27B-Instruct | 82.7% | 5 |
| 1029 | AInstein-SWE-Bench o AInstein agent coding benchmark. | Agentic coding | — | 33.6% | — | 🇺🇸 Gemini 3 Pro | 42.8% | 5 |
| 1030 | AndroidBench o Agentic Android app development benchmark; Qwen reports avg@3 on the 95-task public subset. | Software engineering (Android) | — | 72.0% | — | 🇺🇸 Claude Fable 5 | 84.5% | 5 |
| 1031 | API-Bank o API-Bank tool-use benchmark. | Tool use | — | 79.6% | — | 🇺🇸 Llama 3.1 405B | 92.0% | 5 |
| 1032 | Big Finance Bench o Finance benchmark reported in the GPT-5.6 announcement. | Finance | — | 46.6% | — | 🇺🇸 GPT-5.6 Sol | 53.0% | 5 |
| 1033 | BIOBench o Biology knowledge and reasoning benchmark. | Biology reasoning | — | 45.7% | — | 🇺🇸 Gemini 3 Pro | 51.9% | 5 |
| 1034 | CFEval o Contest-style coding evaluation with ELO-like scoring. | Coding ELO / contest eval | — | — | — | 🇨🇳 Qwen3-235B-A22B-Thinking-2507 | 2134 | 5 |
| 1035 | Claw-Gym o OpenClaw-style personal-assistant agent environment suite measuring task success on daily tasks. | Agentic assistant tasks | — | 46.3% | — | G Nanbeige4.2-3B | 65.0% | 5 |
| 1036 | CLOC Counting o CLOC test object counting, exact-match accuracy. | Visual counting | — | 24.8% | — | G WeVisOne-VL-4B | 28.3% | 5 |
| 1037 | COCO Pointing o COCO object pointing accuracy. | Object pointing | — | 82.6% | — | G WeVisOne-VL-2B | 85.8% | 5 |
| 1038 | CodeIF-Bench o Code-focused instruction following benchmark. | Code instruction following | — | 46.4% | — | 🇨🇳 Qwen3-8B Non-Thinking | 50.0% | 5 |
| 1039 | Collie-Hard o Hard subset of Collie instruction-following tasks. | Instruction following | — | 82.8% | — | 🇺🇸 GPT-5 (high) | 99.0% | 5 |
| 1040 | Complex Workflow o Complex workflow benchmark for economically valuable tasks. | Complex workflows | — | 55.1% | — | 🇺🇸 Gemini 3 Pro | 58.2% | 5 |
| 1041 | Contract Scrub o Contract-review task: identify and redact/flag clauses in legal contracts (Thomson Reuters eval). | Contract review | — | 42.8% | — | 🇺🇸 Gemma 4 31B (IT) | 54.8% | 5 |
| 1042 | Countdown o Countdown-style reasoning and planning benchmark. | Planning and reasoning | — | 21.3% | — | G K2-V2 | 75.6% | 5 |
| 1043 | CultCat o Yandex 4-shot benchmark of (Russian) cultural facts. | Cultural knowledge | — | 70.4% | — | G AliceAI-Foundation-80B-A3B-Base | 86.5% | 5 |
| 1044 | Customer Support Q&A o Customer support question answering benchmark. | Customer support QA | — | 64.4% | — | G Seed1.8 | 69.0% | 5 |
| 1045 | CUTE o CUTE aggregate capability score. | English characters | — | 58.6% | — | 🇺🇸 Bolmo 7B | 78.6% | 5 |
| 1046 | D4LA o Diverse and detailed dataset for document layout analysis (12 document types, 27 categories). | Layout analysis | — | 46.7% | — | G Infinity-Parser2-Pro | 52.4% | 5 |
| 1047 | DeepConsult o Agentic consulting and writing benchmark. | Agentic writing | — | 41.3% | — | 🇺🇸 GPT-5 (high) | 57.2% | 5 |
| 1048 | DeepShop o Online shopping web-agent benchmark with complex, deep product-search queries. | Web agents | — | 29.5% | — | 🇺🇸 MolmoWeb-8B | 42.3% | 5 |
| 1049 | Dense200 Pointing o Dense200 object pointing accuracy. | Object pointing | — | 76.5% | — | G LocateAnything-3B | 87.6% | 5 |
| 1050 | Do-Anything-Now o Resistance to Do Anything Now (DAN) style jailbreak prompts. | Safety / jailbreak | — | 73.9% | — | G IQuest-Coder-V1-40B-Thinking | 97.7% | 5 |
| 1051 | DocFigCapBench Caption Score o DocFigCapBench: caption quality score (0-10 scale). | Document figure captioning | — | — | — | 🇨🇳 Qwen3.8-Max | 7.30 | 5 |
| 1052 | DocLayNet o Human-annotated document layout segmentation dataset; layout analysis mIoU/mAP. | Layout analysis | — | 62.9% | — | G PaddleOCR-VL-1.5 | 71.1% | 5 |
| 1053 | DREAM-1K o DREAM-1K dense/detailed video description benchmark of 1,000 clips. | Video captioning | — | 75.0% | — | 🇺🇸 GPT-6 Astra | 79.1% | 5 |
| 1054 | EduBench English o Yandex education benchmark built from Alice user queries: English language subset. | Education (English) | — | 73.8% | — | 🇨🇳 DeepSeek V4-Flash Base | 82.9% | 5 |
| 1055 | EduBench History o Yandex education benchmark built from Alice user queries: history subset. | Education (history) | — | 71.6% | — | G AliceAI-Foundation-80B-A3B-Base | 82.0% | 5 |
| 1056 | EduBench Literature o Yandex education benchmark built from Alice user queries: literature subset. | Education (literature) | — | 60.4% | — | G AliceAI-Foundation-80B-A3B-Base | 73.8% | 5 |
| 1057 | EduBench Math o Yandex education benchmark built from Alice user queries: school math subset. | Education (math) | — | 72.4% | — | 🇨🇳 Qwen3.5-35B-A3B-Base | 80.0% | 5 |
| 1058 | EduBench Math University o Yandex education benchmark built from Alice user queries: university math subset. | Education (university math) | — | 65.5% | — | G AliceAI-Foundation-80B-A3B-Base | 70.1% | 5 |
| 1059 | EduBench Russian o Yandex education benchmark built from Alice user queries: Russian language subset. | Education (Russian language) | — | 53.6% | — | G AliceAI-Foundation-80B-A3B-Base | 74.2% | 5 |
| 1060 | Education o Education field evaluation (economically valuable tasks). | Economics/education | — | 55.6% | — | G Seed1.8 | 60.8% | 5 |
| 1061 | EGE o Part-A questions of the Russian Unified State Exam (EGE) across subjects. | Russian state exam | — | 85.5% | — | G AliceAI-Foundation-80B-A3B-Base | 90.5% | 5 |
| 1062 | EIFBench o Complex instruction-following benchmark. | Instruction following | — | 51.4% | — | 🇺🇸 GPT-5 (high) | 66.7% | 5 |
| 1063 | Enterprise RAG o Enterprise retrieval-augmented generation evaluation covering internal knowledge bases. | Retrieval-augmented generation | — | 52.8% | — | 🇺🇸 Apriel Nemotron 15B Thinker | 69.2% | 5 |
| 1064 | EXECUTE o Multilingual character-level evaluation benchmark. | Multilingual character tasks | — | 51.7% | — | 🇺🇸 Bolmo 7B | 71.6% | 5 |
| 1065 | ExpertFactsQA Law o Yandex expert-written factual QA benchmark, law (hard) subset. | Expert knowledge (law) | — | 33.0% | — | G AliceAI-Foundation-80B-A3B-Base | 49.6% | 5 |
| 1066 | ExpertFactsQA Medicine o Yandex expert-written factual QA benchmark, medicine subset. | Expert knowledge (medicine) | — | 55.2% | — | G AliceAI-Foundation-80B-A3B-Base | 63.6% | 5 |
| 1067 | FigQA o Figure question answering benchmark evaluating visual reasoning over scientific figures and diagrams. | Figure understanding and QA | — | 62.2% | — | 🇺🇸 Claude Mythos 5 | 88.9% | 5 |
| 1068 | FinanceReasoning o Financial reasoning benchmark evaluating quantitative and qualitative finance problem solving. | Financial reasoning | — | 86.3% | — | G Ling 1T | 87.5% | 5 |
| 1069 | FinQA 128k o Long-context (128k) variant of FinQA: analytics questions over financial reports. | Long-context financial QA | — | 65.8% | — | G AliceAI-Foundation-80B-A3B-Base | 74.1% | 5 |
| 1070 | FinSearchComp (T2&T3) o Finance search competition tasks (tracks T2 and T3). | Finance search | — | 54.0% | — | 🇺🇸 GPT-5 (high) | 64.5% | 5 |
| 1071 | FLTEval o Mistral benchmark built from real pull requests to the Fermat's Last Theorem Lean repository: complete all formal proofs and new definitions per PR. | Formal proof engineering | — | 27.4% | — | 🇺🇸 Claude Opus 4.6 | 39.6% | 5 |
| 1072 | FreshQA o Question answering benchmark emphasizing up-to-date knowledge and recency. | Recency QA | — | 43.0% | — | 🇨🇳 Qwen3-4B Thinking 2507 | 66.9% | 5 |
| 1073 | GeoBench1 o Geospatial visual QA and reasoning (set 1). | Geospatial reasoning | — | 66.8% | — | 🇨🇳 GLM-4.5V | 79.7% | 5 |
| 1074 | Gorilla Benchmark API Bench o Gorilla API Bench tool-use evaluation. | Tool use | — | 17.9% | — | 🇺🇸 Llama 3.1 405B | 35.3% | 5 |
| 1075 | GraphWalks Parents 1M o GraphWalks Parents long-context graph task at 1M context (F1). | Long-context reasoning | — | 47.4% | — | 🇺🇸 Claude Opus 4.6 | 72.0% | 5 |
| 1076 | HardMultiQA o Yandex 5-shot benchmark of hard multi-answer factual questions in Russian. | Russian factual knowledge | — | 56.7% | — | G AliceAI-Foundation-80B-A3B-Base | 67.9% | 5 |
| 1077 | Harvey LAB-AA o Artificial Analysis run of Harvey's Legal Agent Benchmark (legal work products from source documents). | Legal agents | — | 8.5% | — | G Muse Wonder Full | 13.1% | 5 |
| 1078 | HSS o HSS scene-understanding benchmark over images and video. | Scene understanding | — | 47.6% | — | 🇺🇸 GPT-6 Astra | 53.6% | 5 |
| 1079 | HumanEval-V o HumanEval variant with visual programming prompts. | Code generation (vision) | — | 35.7% | — | G Step3-VL-10B | 66.1% | 5 |
| 1080 | IFStruct o Instruction-following benchmark for structured outputs (format/schema-constrained generation). | Structured output | — | 68.3% | — | 🇺🇸 LFM2.5 2.6B | 85.5% | 5 |
| 1081 | IndicIFEval (ground) o Natively written Indic instruction-following prompts; prompt-level strict. | Indic languages | — | 35.9% | — | G Sarvam-30B | 40.7% | 5 |
| 1082 | IndicIFEval (translated) o IFEval-style instruction following, translated into Indic languages; prompt-level strict. | Indic languages | — | 59.3% | — | 🇺🇸 Gemma 4 31B (IT) | 81.1% | 5 |
| 1083 | Information Extraction o Information extraction benchmark for economically valuable fields. | Information extraction | — | 45.5% | — | 🇺🇸 Claude Sonnet 4.5 | 46.9% | 5 |
| 1084 | Information Processing o Information processing benchmark for economically valuable tasks. | Information processing | — | 54.4% | — | 🇺🇸 Gemini 3 Pro | 56.5% | 5 |
| 1085 | Intention Recognition o Intent recognition benchmark for practical applications. | Intent recognition | — | 62.7% | — | 🇺🇸 Gemini 3 Pro | 65.3% | 5 |
| 1086 | KernelBench Hard o KernelBench Hard: agents write optimized GPU kernels; score relative to hardware peak throughput. | GPU kernel optimization | — | 21.9% | — | 🇺🇸 Claude Opus 4.7 | 30.7% | 5 |
| 1087 | KINA o Knowledge benchmark of culturally grounded factual questions (reported in the Seed2.1 model card). | Knowledge | — | 49.5% | — | 🇺🇸 Gemini 3.1 Pro | 53.2% | 5 |
| 1088 | KK-4 People o Keep/kill working-memory benchmark with 4 people entities. | Working memory (4 people) | — | 34.6% | — | G K2-V2 | 92.9% | 5 |
| 1089 | KK-8 People o Keep/kill working-memory benchmark with 8 people entities. | Working memory (8 people) | — | 20.8% | — | G K2-V2 | 82.8% | 5 |
| 1090 | KSM o Korean STEM and math benchmark | Multilingual math | — | 49.4% | — | G EXAONE Deep 2.4B | 60.9% | 5 |
| 1091 | KubriCount (testB) o KubriCount testB object counting, exact-match accuracy. | Visual counting | — | 26.6% | — | G WeVisOne-VL-4B | 40.1% | 5 |
| 1092 | Legal Agent Benchmark o Harvey Legal Agent Benchmark, full public set. | Legal agents | — | 11.4% | — | 🇺🇸 Claude Mythos 5 | 16.9% | 5 |
| 1093 | LEXam (hard) o Hard subset of the LEXam legal-exam benchmark. | Legal exams | — | 38.2% | — | G Darwin-180B-RSI | 45.7% | 5 |
| 1094 | LEXam MCQ (4-choice, EN) o LEXam legal-exam benchmark, English 4-option multiple-choice subset. | Legal exams | — | 76.6% | — | 🇺🇸 Gemma 4 31B (IT) | 87.4% | 5 |
| 1095 | LiveMathematicianBench o Live benchmark of research-level mathematics problems. | Research math | — | 35.9% | — | 🇺🇸 GPT-5.5 | 50.8% | 5 |
| 1096 | LiveSQLBench o LiveSQLBench (Base-Full v1): contamination-free text-to-SQL / database tasks over PostgreSQL databases, solved agentically. | Text-to-SQL | — | 38.9% | — | 🇺🇸 Claude Opus 4.7 | 41.0% | 5 |
| 1097 | LVIS Pointing o LVIS object pointing accuracy. | Object pointing | — | 74.7% | — | G WeVisOne-VL-4B | 79.5% | 5 |
| 1098 | MatBench o Materials property prediction benchmark for scientific AI models. | Materials property prediction | — | 60.2% | — | G Intern-S1-Pro | 72.8% | 5 |
| 1099 | MATH500 (ES) o Spanish MATH500 benchmark | Math (multilingual) | — | 83.9% | — | G EXAONE 4.0 1.2B | 88.8% | 5 |
| 1100 | MBE Bar Exam o Multistate Bar Examination (US bar exam) multiple-choice questions. | Legal exams | — | 82.5% | — | 🇺🇸 Gemma 4 31B (IT) | 88.8% | 5 |
| 1101 | MeasureBench o Benchmark of reading measurement instruments (gauges, rulers, meters) from images. | Visual measurement reading | — | 49.2% | — | G Seed2.1 Pro | 62.9% | 5 |
| 1102 | MedChemBench (internal) o OpenAI internal eval reported in the GPT-6 Astra announcement: medicinal chemistry tasks. | Medicinal chemistry | — | 39.7% | — | 🇺🇸 GPT-6 Astra | 49.3% | 5 |
| 1103 | MemoryBench o Meta-reported benchmark of remembering and retrieving user context across conversations. | Long-term memory | — | 82.6% | — | G Muse Wonder | 85.6% | 5 |
| 1104 | MILU o Multi-task Indic Language Understanding benchmark (11 Indian languages + English), accuracy. | Indic languages | — | 77.9% | — | 🇺🇸 Gemma 4 31B (IT) | 85.3% | 5 |
| 1105 | MixCount o MixCount test object counting, exact-match accuracy. | Visual counting | — | 21.2% | — | G WeVisOne-VL-4B | 40.4% | 5 |
| 1106 | MME-CC o MME-CC multimodal evaluation suite. | Multimodal evaluation | — | 40.8% | — | 🇺🇸 Gemini 3 Pro | 56.9% | 5 |
| 1107 | MMLongBench-128K o 128K-context variant of MMLongBench evaluating multimodal long-context understanding. | Long-context multimodal | — | 70.7% | — | G Seed2.1 Pro | 78.3% | 5 |
| 1108 | MMMLU (ES) o Spanish MMMLU benchmark | Multilingual knowledge | — | 58.4% | — | 🇺🇸 SmolLM3-3B | 64.7% | 5 |
| 1109 | MRCR 128K-2N o Multi-Round Coreference Resolution benchmark at 128k context with 2 needles. | Long-context reasoning | — | 25.8% | — | 🇫🇷 Ministral-3-R 8B | 50.3% | 5 |
| 1110 | MRCR 128K-4N o Multi-Round Coreference Resolution benchmark at 128k context with 4 needles. | Long-context reasoning | — | 15.5% | — | 🇫🇷 Ministral-3-R 8B | 22.7% | 5 |
| 1111 | MRCR 64K-2N o Multi-Round Coreference Resolution benchmark at 64k context with 2 needles. | Long-context reasoning | — | 27.4% | — | 🇫🇷 Ministral-3-R 8B | 44.0% | 5 |
| 1112 | MRCR 64K-4N o Multi-Round Coreference Resolution benchmark at 64k context with 4 needles. | Long-context reasoning | — | 20.1% | — | 🇫🇷 Ministral-3-R 8B | 35.8% | 5 |
| 1113 | MSEarth-MCQ o Earth science multiple-choice question benchmark for scientific AI models. | Earth science | — | 61.6% | — | 🇺🇸 Gemini 3 Pro | 65.8% | 5 |
| 1114 | NAVSIM (PDMS) o NAVSIM v1 non-reactive simulation benchmark for end-to-end driving planners; score = PDM Score (PDMS, 0-100). | Driving planning | — | — | — | 🇨🇳 Qwen-Drive-1.0-4B RL | 90.7 | 5 |
| 1115 | NoLiMa 128K o NoLiMa (No Literal Match) long-context benchmark at 128k context window. | Long-context eval | — | 9.8% | — | 🇨🇳 MiniCPM-SALA | 23.9% | 5 |
| 1116 | NoLiMa 32K o NoLiMa (No Literal Match) long-context benchmark at 32k context window. | Long-context eval | — | 27.3% | — | 🇨🇳 MiniCPM-SALA | 54.5% | 5 |
| 1117 | NoLiMa 64K o NoLiMa (No Literal Match) long-context benchmark at 64k context window. | Long-context eval | — | 18.1% | — | 🇨🇳 MiniCPM-SALA | 43.0% | 5 |
| 1118 | OBQA o OpenBookQA science question answering benchmark. | Open book QA | — | 43.5% | — | 🇨🇳 Qwen2.5-Omni-3B | 76.3% | 5 |
| 1119 | OCRBenchV2 KIE o Key information extraction subset of OCRBench v2 as reported without a language split (OvisOCR3 release). | Key information extraction | — | 74.7% | — | G OvisOCR3 | 83.0% | 5 |
| 1120 | OfficeQA o Grounded question answering over office and enterprise documents, as reported by OpenAI for GPT-5.4. | Enterprise document QA | — | 71.2% | — | 🇺🇸 Claude Opus 4.7 | 86.3% | 5 |
| 1121 | OmniGAIA o OmniGAIA: GAIA-style omni-modal (image/video/audio) agentic perception and reasoning benchmark. | Multimodal agent | — | 56.4% | — | 🇺🇸 Gemini 3 Pro | 62.5% | 5 |
| 1122 | OpenAI Internal Coding Deception o OpenAI internal alignment eval: fraction of coding-agent answers with detected deception (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 0.5% | 5 |
| 1123 | OptMATH o OptMATH benchmark targeting challenging math optimization and problem-solving tasks. | Math optimization reasoning | — | 42.3% | — | G Ling 1T | 57.7% | 5 |
| 1124 | Order 15 Items o Ordering benchmark requiring models to sequence 15 items correctly. | List ordering | — | 31.5% | — | G K2-V2 | 87.6% | 5 |
| 1125 | Order 30 Items o Ordering benchmark requiring models to sequence 30 items correctly. | List ordering (long) | — | 8.4% | — | G K2-V2 | 40.3% | 5 |
| 1126 | OvisOCR In-house KIE Bench o OvisOCR team in-house key information extraction benchmark. | Key information extraction | — | 72.9% | — | G OvisOCR3 | 77.7% | 5 |
| 1127 | PLawBench o Legal-practice benchmark; Qwen evaluates it with gemini-3.1-pro-preview as judge. | Legal reasoning | — | 68.8% | — | 🇨🇳 Qwen3.8-Max | 73.2% | 5 |
| 1128 | PRBench Finance o PRBench finance subset: rubric-graded professional reasoning tasks in finance. | Professional reasoning (finance) | — | 53.7% | — | 🇨🇳 Qwen3.8-Max | 58.3% | 5 |
| 1129 | PresentBench o Benchmark for generating presentation decks from source materials, scored with rubrics. | Agentic office work (slides) | — | 57.1% | — | 🇺🇸 GPT-5.5 | 68.9% | 5 |
| 1130 | Query Sufficiency o Judging whether a legal research query contains sufficient information to answer (Thomson Reuters eval). | Legal query triage | — | 53.8% | — | G Thomson-1.0-Small | 59.4% | 5 |
| 1131 | Realbench o Real-world browsing and QA benchmark. | Web browsing | — | 41.5% | — | G Seed1.8 | 49.1% | 5 |
| 1132 | ReasonSeg (val) o ReasonSeg val reasoning segmentation, gIoU. | Reasoning segmentation | — | 61.6% | — | G WeVisOne-VL-4B | 71.2% | 5 |
| 1133 | Recruit-Bench o Benchmark of recruiting-domain tasks (resume screening, job matching, candidate communication) used by Nanbeige. | Recruitment / HR tasks | — | 58.1% | — | 🇺🇸 Gemma 4 12B IT | 69.4% | 5 |
| 1134 | Ref-ZOM (cIoU) o Ref-ZOM zero/one/many-target referring segmentation, cIoU. | Referring segmentation | — | 67.2% | — | G WeVisOne-VL-2B | 73.4% | 5 |
| 1135 | Ref-ZOM (Rejection) o Ref-ZOM rejection rate on non-existent referents. | Referring segmentation | — | 38.4% | — | G WeVisOne-VL-4B | 86.0% | 5 |
| 1136 | ReMI o Reasoning over multimodal inputs (ReMI). | Multimodal reasoning | — | 60.2% | — | G Step3-VL-10B | 67.3% | 5 |
| 1137 | SafetyBench o Multiple-choice safety evaluation across 7 categories in Chinese and English. | Safety | — | 88.1% | — | 🇺🇸 GPT 4 0613 | 89.7% | 5 |
| 1138 | SArena (Icon) o SVG Arena benchmark for icon generation evaluation. | SVG generation | — | 75.0% | — | G Intern-S1-Pro | 83.5% | 5 |
| 1139 | SkillsBench v1.1 o Public SkillsBench v1.1 benchmark (87 tasks), average score over three runs per task. | Agent skills | — | 68.2% | — | 🇺🇸 GPT-5.6 Sol | 73.5% | 5 |
| 1140 | SmolInstruct o Small molecule instruction-following and understanding benchmark. | Small molecule understanding | — | 54.3% | — | G Intern-S1-Pro | 74.8% | 5 |
| 1141 | SWE-fficiency o SWE-fficiency: repository-level performance-optimization tasks; agents must speed up real codebases while keeping tests passing. | Code performance optimization | — | 31.5% | — | 🇺🇸 GPT-5.5 | 46.6% | 5 |
| 1142 | SWE-Review o Software engineering review benchmark for assessing code review quality. | Code review | — | 9.1% | — | 🇺🇸 Claude Opus 4.5 | 16.2% | 5 |
| 1143 | Terminal-Bench 4.0-AA o Artificial Analysis run of Terminal-Bench 4.0. | Agentic terminal coding | — | 56.0% | — | 🇺🇸 Claude Opus 5.5 | 59.6% | 5 |
| 1144 | TriviaQA-Wiki o TriviaQA subset answering using Wikipedia evidence. | Open-domain QA | — | 85.5% | — | 🇺🇸 Llama 3.1 405B Base | 91.8% | 5 |
| 1145 | U-Artifacts o Benchmark focusing on generated code artifacts quality. | Agentic coding artifacts | — | 46.9% | — | 🇺🇸 Gemini 3 Pro | 57.8% | 5 |
| 1146 | UGC-VideoCap o Short-form user-generated video captioning benchmark. | Video captioning | — | 79.6% | — | G Glimpse | 82.7% | 5 |
| 1147 | Vibe Android o Vibe evaluation on Android tasks. | Vibe evaluation (Android) | — | 83.5% | — | 🇺🇸 Claude Opus 4.5 | 92.2% | 5 |
| 1148 | Vibe Backend o Vibe evaluation on backend tasks. | Vibe evaluation (backend) | — | 84.3% | — | 🇺🇸 Claude Opus 4.5 | 98.0% | 5 |
| 1149 | Vibe iOS o Vibe evaluation on iOS tasks. | Vibe evaluation (iOS) | — | 74.9% | — | 🇺🇸 Claude Opus 4.5 | 90.0% | 5 |
| 1150 | Vibe Simulation o Vibe evaluation on simulation tasks. | Vibe evaluation (simulation) | — | 83.3% | — | 🇺🇸 Gemini 3 Pro | 89.2% | 5 |
| 1151 | Vibe Web o Vibe evaluation on web tasks. | Vibe evaluation (web) | — | 87.6% | — | G MiniMax M2.1 | 91.5% | 5 |
| 1152 | VideoMathQA o Benchmark for mathematical reasoning grounded in video content. | Video math reasoning | — | 52.6% | — | G JoyAI-VL-Interaction | 56.2% | 5 |
| 1153 | VisDrone Pointing o VisDrone object pointing accuracy. | Object pointing | — | 58.5% | — | G WeVisOne-VL-4B | 64.4% | 5 |
| 1154 | WGO Bench o WGO Bench: end-to-end robot manipulation event understanding (Elorian-reported). | Embodied manipulation | — | 33.5% | — | G Glimpse | 42.7% | 5 |
| 1155 | WikiWebFacts o Yandex 5-shot benchmark of factual knowledge in Russian. | Russian factual knowledge | — | 75.0% | — | G AliceAI-Foundation-80B-A3B-Base | 86.5% | 5 |
| 1156 | WorldBench o Benchmark of real-world visual knowledge and recognition. | Visual knowledge | — | 61.1% | — | G Seed2.1 Pro | 67.6% | 5 |
| 1157 | WorldTravel2 (multi-modal) o WorldTravel2 benchmark multimodal track. | Travel planning (multimodal) | — | 43.5% | — | 🇺🇸 Gemini 3 Pro | 47.2% | 5 |
| 1158 | WorldTravel2 (text) o WorldTravel2 benchmark text-only track. | Travel planning (text) | — | 51.9% | — | 🇺🇸 GPT-5 (high) | 56.4% | 5 |
| 1159 | xdev-bench-hard o Moonshot internal offensive-security / exploit-development benchmark; problems solved (pass@3). | Cybersecurity | — | 71.4% | — | 🇺🇸 GPT-5.6 Sol | 100.0% | 5 |
| 1160 | XorQA-IN o Cross-lingual open-retrieval QA in Indian languages (IndicXTREME), F1. | Indic languages | — | 32.9% | — | G gnani-evon-v3.3-30B-A3B | 41.8% | 5 |
| 1161 | XpertBench (Edu) o XpertBench education domain subset. | Economics/education | — | 49.3% | — | 🇺🇸 GPT-5 (high) | 56.9% | 5 |
| 1162 | XpertBench (Fin) o XpertBench finance domain subset. | Economics/finance | — | 51.5% | — | 🇺🇸 GPT-5 (high) | 64.5% | 5 |
| 1163 | XpertBench (Humanities) o XpertBench humanities domain subset. | Economics/humanities | — | 60.8% | — | 🇺🇸 GPT-5 (high) | 68.5% | 5 |
| 1164 | XpertBench (Law) o XpertBench legal domain subset. | Economics/legal | — | 53.6% | — | 🇺🇸 Claude Sonnet 4.5 | 58.7% | 5 |
| 1165 | XpertBench (Research) o XpertBench research domain subset. | Economics/research | — | 33.5% | — | 🇺🇸 GPT-5 (high) | 48.2% | 5 |
| 1166 | XQuAD-IN o Extractive QA (XQuAD) in Indian languages (IndicXTREME), F1. | Indic languages | — | 51.3% | — | G gnani-evon-v3.3-30B-A3B | 64.6% | 5 |
| 1167 | τ³-Banking (Voice) o Sierra τ³ banking knowledge tasks run voice-to-voice; pass@1 (%). | Voice agents | — | 20.2% | — | 🇺🇸 Gemini 3.8 Live Extended Thinking (high) | 35.1% | 5 |
| 1168 | AfriMed-QA o Pan-African medical question answering benchmark. | Medical QA | — | 57.0% | — | 🇺🇸 MedGemma 27B | 72.0% | 4 |
| 1169 | AgentIF o Instruction-following benchmark built from real agentic scenarios with long, complex system prompts and constraints; score is (CSR+ISR)/2. | Instruction following | — | 43.8% | — | G openPangu-2.0-Flash Thinking | 44.9% | 4 |
| 1170 | AIME 2024-Ko o Korean translation of AIME 2024 problems. | Math (competition, Korean) | — | 63.2% | — | 🇨🇳 Qwen3-30B-A3B-Thinking-2507 | 80.3% | 4 |
| 1171 | AIME 2026 I (with tools) o AIME 2026 I problems solved with tool (code execution) access. | Math competition (with tools) | — | 93.3%saturated | — | 🇺🇸 Gemini 3.1 Pro | 99.9% | 4 |
| 1172 | Almieyar o Arabic linguistic competence benchmark across phonology, morphology, syntax, semantics and pragmatics. | Arabic linguistics | — | 68.1% | — | G Fanar-2-27B-Instruct | 79.5% | 4 |
| 1173 | Android Daily o Benchmark of long-horizon daily phone-use tasks across multiple Android apps. | GUI agents | — | 57.5% | — | 🇺🇸 Gemini 3 Flash | 63.2% | 4 |
| 1174 | ASR AMI o Automatic speech recognition benchmark on AMI meeting speech. | ASR | — | — | — | 🇨🇳 Qwen2.5-Omni-3B | ↓ 15.1% | 4 |
| 1175 | ASR Earnings22 o Automatic speech recognition benchmark on Earnings22 financial calls. | ASR | — | — | — | G Whisper-large-V3 | ↓ 11.3% | 4 |
| 1176 | ASR GigaSpeech o Automatic speech recognition benchmark on GigaSpeech. | ASR | — | — | — | G Whisper-large-V3 | ↓ 10.0% | 4 |
| 1177 | ASR SPGISpeech o Automatic speech recognition benchmark on SPGISpeech. | ASR | — | — | — | 🇺🇸 LFM2.5-Audio-1.5B | ↓ 2.8% | 4 |
| 1178 | ASR TED-LIUM o Automatic speech recognition benchmark on TED-LIUM. | ASR | — | — | — | 🇺🇸 LFM2.5-Audio-1.5B | ↓ 3.5% | 4 |
| 1179 | ASR VoxPopuli o Automatic speech recognition benchmark on VoxPopuli. | ASR | — | — | — | 🇨🇳 Qwen2.5-Omni-3B | ↓ 5.6% | 4 |
| 1180 | BeaverTails o Safety benchmark evaluating harmfulness in model responses. | Safety / harmfulness | — | 70.7% | — | G IQuest-Coder-V1-40B-Thinking | 76.7% | 4 |
| 1181 | BharatMath o Math word problems in Indian languages. | Indic languages | — | 72.3% | — | 🇺🇸 Gemma 4 31B (IT) | 78.6% | 4 |
| 1182 | BridgeBench o BridgeBench evaluation benchmark. | Reasoning | — | 54.9% | — | 🇺🇸 Claude Opus 4.6 | 60.1% | 4 |
| 1183 | CC-Bench-V2 Backend o Z.ai CC-Bench-V2 backend development tasks in a Claude Code-style agent harness. | Agentic coding | — | 23.9% | — | 🇺🇸 Claude Opus 4.6 | 26.9% | 4 |
| 1184 | CC-Bench-V2 Frontend o Z.ai CC-Bench-V2 frontend development tasks in a Claude Code-style agent harness. | Agentic coding | — | 69.0% | — | 🇺🇸 Claude Opus 4.6 | 75.9% | 4 |
| 1185 | CC-Bench-V2 Repo Exploration o Z.ai CC-Bench-V2 repository exploration tasks in a Claude Code-style agent harness. | Agentic coding | — | 70.6% | — | 🇺🇸 Claude Opus 4.6 | 74.4% | 4 |
| 1186 | ChestX-ray14 o NIH ChestX-ray14 classification, macro F1. | Medical imaging | — | 44.0% | — | 🇺🇸 MedGemma 4B | 50.1% | 4 |
| 1187 | CheXpert o Chest X-ray condition classification, macro F1 over top 5 conditions. | Medical imaging | — | 44.7% | — | 🇺🇸 MedGemma 27B | 49.9% | 4 |
| 1188 | Civil Comments o Toxicity classification of online comments (Jigsaw Civil Comments). | Toxicity classification | — | 93.8%saturated | — | G d1-omni-600M | 95.8% | 4 |
| 1189 | Countix o Video-based counting benchmark for multiple objects. | Video counting | — | 23.6% | — | G Seed1.8 | 31.0% | 4 |
| 1190 | CRAG o Complex Retrieval-Augmented Generation benchmark for grounded question answering. | Retrieval QA | — | 52.8% | — | G Jamba Mini 1.6 | 76.2% | 4 |
| 1191 | DeepCodeBench o Qodo's question-answering benchmark over real-world code repositories, requiring retrieval and reasoning across large codebases. | Code QA | — | 73.5% | — | G openPangu-2.0-Flash Thinking | 76.5% | 4 |
| 1192 | EHRNoteQA o Clinical-note question answering over MIMIC-IV discharge summaries. | Medical records | — | 82.1% | — | 🇺🇸 MedGemma 27B | 90.7% | 4 |
| 1193 | EnConda-Bench o English code editing benchmark for applying conditional modifications. | Code editing | — | 13.0% | — | G Youtu-LLM-2B | 21.5% | 4 |
| 1194 | FATE-H o FATE-H: graduate-level abstract algebra problems formalized in Lean 4 (100 problems). | Formal theorem proving (abstract algebra) | — | 72.5% | — | G Leanstral 1.5 | 87.0% | 4 |
| 1195 | FATE-X o FATE-X: PhD-level abstract algebra problems formalized in Lean 4 (100 problems). | Formal theorem proving (abstract algebra) | — | 25.3% | — | G Leanstral 1.5 | 34.0% | 4 |
| 1196 | FeatBench o Benchmark of feature implementation in real repositories from natural-language requirements (vibe coding). | Code (feature implementation) | — | 46.8% | — | G openPangu-2.0-Pro | 48.4% | 4 |
| 1197 | FinanceBench (FullDoc) o FinanceBench full-document question answering benchmark requiring long-context financial understanding. | Finance QA | — | 24.0% | — | G Jamba Mini 1.6 | 45.4% | 4 |
| 1198 | FrontierCode o Cognition's agentic coding benchmark of 150 tasks from real PRs, graded on hidden tests plus code-quality rubric (Main set, v1.1). | Software engineering | — | 50.1% | — | 🇺🇸 Claude Opus 5.5 | 54.4% | 4 |
| 1199 | Gaokao-MathCloze o Fill-in-the-blank math problems from Chinese Gaokao (AGIEval). | Chinese math (exam) | — | 12.3% | — | 🇨🇳 DeepSeek Math 7B | 20.3% | 4 |
| 1200 | Gaokao-MathQA o Multiple-choice math problems from Chinese Gaokao (AGIEval). | Chinese math (exam) | — | 27.1% | — | 🇨🇳 DeepSeek Math 7B | 35.3% | 4 |
| 1201 | GeneBench o OpenAI eval of multi-step scientific data analysis in genetics and quantitative biology, with ambiguous or flawed data and hidden confounders. | Scientific data analysis | — | 25.7% | — | 🇺🇸 GPT-5.5 Pro | 33.2% | 4 |
| 1202 | GlobalMGSM o Global multilingual grade school math word problems. | Math (multilingual) | — | 54.7% | — | 🇨🇳 Qwen3-4B | 60.9% | 4 |
| 1203 | GQA o Compositional question answering over real-world images (scene-graph based); test-dev balanced accuracy. | Visual reasoning VQA | — | 49.7% | — | 🇨🇳 Qwen-VL | 59.3% | 4 |
| 1204 | GSM-Infinite Hard (128K) o GSM-Infinite Hard benchmark at 128K context. | Math reasoning | — | 23.1% | — | G MiMo V2 Flash Base | 29.0% | 4 |
| 1205 | GSM-Infinite Hard (16K) o GSM-Infinite Hard benchmark at 16K context. | Math reasoning | — | 41.1% | — | 🇨🇳 DeepSeek V3.2-Exp | 50.4% | 4 |
| 1206 | GSM-Infinite Hard (32K) o GSM-Infinite Hard benchmark at 32K context. | Math reasoning | — | 36.0% | — | 🇨🇳 DeepSeek V3.2-Exp | 45.2% | 4 |
| 1207 | GSM-Infinite Hard (64K) o GSM-Infinite Hard benchmark at 64K context. | Math reasoning | — | 28.7% | — | 🇨🇳 DeepSeek V3.1 | 34.7% | 4 |
| 1208 | HELMET LongQA o Long-context subset of the HELMET benchmark focusing on grounded question answering. | Long-context QA | — | 29.7% | — | G Jamba Mini 1.6 | 46.9% | 4 |
| 1209 | IMO 2025 o 2025 International Mathematical Olympiad (6 proof problems), graded with MathArena methodology. | Math olympiad (proofs) | — | 50.3% | — | 🇺🇸 GPT-5.5 | 76.1% | 4 |
| 1210 | IndicSafe5 o Safety benchmark of harmful/benign prompts across Indic languages; accuracy. | Indic languages | — | 86.4% | — | 🇺🇸 Gemma 4 31B (IT) | 89.8% | 4 |
| 1211 | KernelGen 1P o OpenAI internal eval (GPT-5.6 announcement): optimizing GPU kernels. | AI research automation | — | 40.5% | — | 🇺🇸 GPT-5.6 Sol | 61.1% | 4 |
| 1212 | Kimi Claw 24/7 Bench o Moonshot in-house benchmark of persistent multi-day coworking tasks (17 scenarios, 610 evaluation points) run through the OpenClaw harness; average pass rate over 3 runs. | Long-horizon agents | — | 48.3% | — | 🇺🇸 GPT-5.5 | 52.8% | 4 |
| 1213 | LatentJailbreak o Robustness to latent jailbreak adversarial techniques. | Safety / jailbreak | — | 49.7% | — | 🇺🇸 GPT-3.5-turbo | 77.4% | 4 |
| 1214 | LiveClaw o Live benchmark of OpenClaw agent tasks executed in real environments. | Agentic assistant tasks | — | 41.3% | — | 🇺🇸 LFM2.5 24B A2B | 48.1% | 4 |
| 1215 | LiveCodeBench-Ko o Korean translation of LiveCodeBench. | Code generation (Korean) | — | 47.1% | — | 🇨🇳 Qwen3-30B-A3B-Thinking-2507 | 66.3% | 4 |
| 1216 | M3GIA (CN) o Chinese-language M3GIA benchmark covering grounded multimodal question answering. | Chinese multimodal QA | — | 90.0% | — | 🇨🇳 Seed1.5-VL-Thinking | 91.2% | 4 |
| 1217 | mArenaHard o Multilingual variant of Arena-Hard evaluating chat quality across languages. | Chat ability (multilingual) | — | 66.8% | — | 🇨🇳 Qwen3-4B | 70.1% | 4 |
| 1218 | MASSIVE intent o Amazon MASSIVE intent classification (English). | Intent classification | — | 85.7% | — | G Decider 4B (Mapika) | 88.3% | 4 |
| 1219 | MATH-Ko o Korean translation of the MATH competition benchmark. | Math (Korean) | — | 51.3% | — | 🇨🇳 Qwen3-30B-A3B | 58.2% | 4 |
| 1220 | mDolly o Multilingual variant of the Dolly instruction-following benchmark. | Instruction following (multilingual) | — | 73.3% | — | G Tiny Aya Global | 86.9% | 4 |
| 1221 | MIMIC-CXR (classification) o Chest X-ray condition classification on MIMIC-CXR, macro F1 over top conditions. | Medical imaging | — | 87.4% | — | 🇺🇸 MedGemma 27B | 90.0% | 4 |
| 1222 | MMLB-NIAH (128k) o MMLB-NIAH 128k long-context multimodal benchmark. | Multimodal long-context | — | 68.8% | — | G Seed1.8 | 72.2% | 4 |
| 1223 | MMLB-VRAG (128k) o MMLB-VRAG 128k long-context multimodal benchmark. | Multimodal long-context | — | 81.8% | — | 🇺🇸 Gemini 3 Pro | 88.9% | 4 |
| 1224 | Mobile Actions o Google Mobile Actions dataset: function calling for on-device Android actions (FunctionGemma eval). | Tool use | — | 86.9% | — | 🇨🇳 Qwen3.5-9B | 91.8% | 4 |
| 1225 | MS-CXR-T o Temporal chest X-ray disease progression classification, macro accuracy. | Medical imaging | — | 59.0% | — | 🇺🇸 MedGemma 1.5 4B | 65.7% | 4 |
| 1226 | MTalk-Bench o MTalk-Bench: multi-turn speech-to-speech dialogue (semantic, paralinguistic, ambient) evaluation (%). | Spoken multi-turn dialogue | — | 69.2% | — | G StepAudio 3 Realtime | 72.8% | 4 |
| 1227 | MTOB (full book) o Long-context book understanding benchmark (full-book setting). | Long-form reasoning | — | 45.3% | — | 🇺🇸 Llama 4 Maverick | 50.8% | 4 |
| 1228 | MTOB (half book) o Long-context book understanding benchmark (half-book setting). | Long-form reasoning | — | 48.1% | — | 🇺🇸 Llama 4 Maverick | 54.0% | 4 |
| 1229 | MultiLoKo o Multilingual local-knowledge QA benchmark covering 31 languages with locally sourced questions. | Multilingual knowledge | — | 44.5% | — | 🇨🇳 DeepSeek V4-Pro Base | 51.1% | 4 |
| 1230 | NanoGPT o OpenAI internal eval (GPT-5.6 announcement): improving a NanoGPT training recipe (speed-up score). | AI research automation | — | 7.1% | — | 🇺🇸 GPT-5.6 Terra | 14.5% | 4 |
| 1231 | Nanonets KIE o Nanonets key information extraction benchmark (F1). | Key information extraction | — | 82.4% | — | G Qianfan-OCR | 86.5% | 4 |
| 1232 | NIAH-Multi 128K o Needle-in-a-haystack multi-query benchmark at 128K context. | Long-context QA | — | 97.4%saturated | — | 🇨🇳 Kimi-K2 Base | 99.5% | 4 |
| 1233 | NIAH-Multi 32K o Needle-in-a-haystack multi-query benchmark at 32K context. | Long-context QA | — | 96.1%saturated | — | 🇨🇳 Kimi-K2 Base | 99.8% | 4 |
| 1234 | NIAH-Multi 64K o Needle-in-a-haystack multi-query benchmark at 64K context. | Long-context QA | — | 96.1%saturated | — | 🇨🇳 Kimi-K2 Base | 100.0% | 4 |
| 1235 | NoCaps o Novel object captioning benchmark (CIDEr on val). | Image captioning | — | — | — | G InstructBLIP (Vicuna-13B) | 121.9 | 4 |
| 1236 | OCRBenchV2 KIE (EN) o English key information extraction subset of OCRBench v2. | Key information extraction | — | 84.6% | — | 🇺🇸 Gemini 3.1 Pro | 87.8% | 4 |
| 1237 | OCRBenchV2 KIE (ZH) o Chinese key information extraction subset of OCRBench v2. | Key information extraction (Chinese) | — | 70.0% | — | G Qianfan-OCR | 82.3% | 4 |
| 1238 | ODVBench o Online/streaming video understanding benchmark (accuracy). | Streaming video understanding | — | 56.6% | — | G JoyAI-VL-Interaction | 68.6% | 4 |
| 1239 | OlympicArena o Olympiad-style mathematics reasoning benchmark. | Math (competition) | — | 71.2% | — | 🇨🇳 DeepSeek V3 | 76.2% | 4 |
| 1240 | Omni3D o Omni3D monocular 3D object detection benchmark across indoor/outdoor datasets (AP3D). | 3D detection | — | 39.6% | — | 🇨🇳 Qwen3.5-4B | 47.4% | 4 |
| 1241 | OpenAI Internal Database Migration Tasks o OpenAI internal eval reported in the GPT-6 Astra announcement: database migration coding tasks. | Software engineering | — | 53.7% | — | 🇺🇸 GPT-6 Astra | 63.9% | 4 |
| 1242 | OpenAI Internal Research Debugging Evaluation o OpenAI internal eval (GPT-5.6 announcement): debugging real AI research systems. | AI research automation | — | 59.2% | — | 🇺🇸 GPT-5.6 Sol | 68.3% | 4 |
| 1243 | OpenAI Internal Unauthorized Agent Interaction o OpenAI internal alignment eval: rate of unauthorized interaction with other agents (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | 4 | |
| 1244 | OSWorld 2.1 (Offline) o OSWorld 2.1 offline subset (82 tasks), partial-credit score on long-horizon computer-use workflows without internet access. | GUI agents | — | 55.2% | — | 🇺🇸 Claude Sonnet 5.5 | 83.9% | 4 |
| 1245 | OSWorld2 o Second-generation OSWorld GUI agent benchmark. | GUI agents | — | 16.3% | — | 🇨🇳 GLM-4.5V | 35.8% | 4 |
| 1246 | PAWS-X o Cross-lingual adversarial paraphrase identification. | Paraphrase identification | — | 71.4% | — | G d1-omni-600M | 79.5% | 4 |
| 1247 | PostTrainBench Lite o Lite variant of PostTrainBench (agents post-training base LLMs under fixed compute). | Post-training automation | — | 42.6% | — | 🇺🇸 GPT-5.6 Terra | 51.5% | 4 |
| 1248 | RefusalBench o Safety-oriented refusal and policy adherence benchmark. | Safety / refusal | — | 64.4% | — | 🇺🇸 Hermes 4.3 36B Pyche | 72.3% | 4 |
| 1249 | SaaS-Bench o Benchmark of agents completing tasks in SaaS application environments (as reported in the Kimi K3 release). | Agentic SaaS tasks | — | 55.4% | — | 🇺🇸 GPT-5.6 Sol | 61.4% | 4 |
| 1250 | SAT Math o SAT mathematics questions (DeepSeekMath / Minerva-style evaluation). | Math (exam) | — | 71.9% | — | 🇨🇳 DeepSeek Math 7B | 84.4% | 4 |
| 1251 | SimpleVQA-DS o SimpleVQA variant curated by DeepSeek with everyday image question answering tasks. | General VQA | — | 56.6% | — | 🇨🇳 Seed1.5-VL-Thinking | 61.3% | 4 |
| 1252 | SOB (Structured Output Benchmark) o Structured Output Benchmark: extract values into a JSON schema from text, images and audio (5,324 records); value accuracy over exact field matches. | Structured output | — | 79.3% | — | G Interfaze 1.1 Lite | 81.5% | 4 |
| 1253 | SpreadsheetBench-Verified o Verified subset of SpreadsheetBench real-world spreadsheet manipulation tasks. | Spreadsheet manipulation | — | 75.7% | — | 🇺🇸 Claude Opus 4.6 | 82.4% | 4 |
| 1254 | SWE-Bench Pro v2 (Hard) o Hard split of SWE-Bench Pro v2 (Scale AI) for software-engineering agents. | Software engineering | — | 76.7% | — | 🇨🇳 Kimi-K3 | 88.2% | 4 |
| 1255 | ToolHop o Query-driven benchmark of 995 multi-hop tool-use queries with 3,912 locally executable tools; answer accuracy. | Multi-hop tool use | — | 46.9% | — | 🇨🇳 Qwen3.5-9B | 52.0% | 4 |
| 1256 | TriQA o Triadic question answering benchmark evaluating world knowledge and reasoning. | Knowledge QA | — | 76.7% | — | 🇫🇷 Mixtral 8x22B | 82.2% | 4 |
| 1257 | TrustLLM o TrustLLM benchmark for trustworthiness and safety behaviors. | Safety / reliability | — | 77.5% | — | 🇨🇳 Qwen3-Coder-480B-A35B-Instruct | 88.4% | 4 |
| 1258 | Vibe Code Bench o Vals AI Vibe Code Bench: building complete web applications from natural-language specs. | App generation | — | 90.5%saturated | — | 🇺🇸 Gemini 4 Argon | 91.9% | 4 |
| 1259 | VideoHallucer o Benchmark for detecting intrinsic and extrinsic hallucinations in video-language models. | Video hallucination | — | 58.5% | — | 🇺🇸 Gemini 3 Pro | 66.8% | 4 |
| 1260 | VisualProbe o High-resolution visual search benchmark requiring iterative zoom/crop tool use (introduced with Mini-o3). | Visual search / fine-grained perception | — | 63.1% | — | 🇺🇸 Gemini 3 Flash | 69.9% | 4 |
| 1261 | VoiceBench-ja Elyza o ELYZA-tasks subset of VoiceBench-ja (spoken input), GPT-4o judge score 0-5. | Spoken QA (Japanese) | — | — | — | 🇨🇳 Qwen2.5-Omni-3B | 2.37 | 4 |
| 1262 | VoiceBench-ja M-IFEval o M-IFEval subset of VoiceBench-ja (spoken input), scored 0-1. | Spoken instruction following (Japanese) | — | — | — | G J-Moshi | 0.151 | 4 |
| 1263 | VoiceBench-ja spoken_Elyza o spoken_Elyza subset of VoiceBench-ja, GPT-4o judge score 0-5. | Spoken QA (Japanese) | — | — | — | 🇺🇸 LFM2.5-Audio-1.5B-JP | 2.12 | 4 |
| 1264 | WebTestBench o Benchmark of web-app testing agents that detect defects against a checklist in live web applications. | Web testing agents | — | 51.9% | — | 🇺🇸 GPT-5.6 Sol | 54.0% | 4 |
| 1265 | Windows Agent Arena o Windows OS computer-use agent benchmark (WAA), task success rate. | GUI agents | — | 45.2% | — | G EvoCUA-32B-20260105 | 56.5% | 4 |
| 1266 | WOD-E2E (RFS) o Waymo Open Dataset end-to-end driving challenge on long-tail scenarios; score = Rater Feedback Score (RFS, 0-10) on the test split. | Driving planning | — | — | — | 🇨🇳 Qwen-Drive-1.0-4B RL | 7.91 | 4 |
| 1267 | WorkBench o Workplace agent benchmark: tasks over simulated email, calendar, CRM, analytics and project-management tools; scored by task completion (outcome-based). | Agentic tool use | — | 43.9% | — | 🇺🇸 Gemma 4 E4B IT | 46.1% | 4 |
| 1268 | ZClawBench o Z.ai end-to-end benchmark for OpenClaw agent workflows (environment setup, software dev, retrieval, data analysis, content creation). | Agent tasks (OpenClaw) | — | 57.4% | — | 🇺🇸 Claude Opus 4.6 | 62.3% | 4 |
| 1269 | AA-Omniscience-Public Accuracy o Share of questions answered correctly on the publicly released AA-Omniscience subset. | Knowledge and hallucination | — | 43.5% | — | 🇺🇸 GPT-6 Luna (medium) | 46.9% | 3 |
| 1270 | AgentIF-OneDay o Agentic instruction-following benchmark of full-day, multi-step everyday and office tasks for general agents. | Agentic instruction following | — | 61.6% | — | G Nanbeige4.2-3B | 67.5% | 3 |
| 1271 | APEX-SWE o Mercor/Cognition benchmark of integration (shipping end-to-end systems on cloud/business/infra services) and observability (debugging production-style failures from telemetry) tasks; Pass@1. | Software engineering | — | 56.3% | — | 🇺🇸 Claude Fable 5 | 58.8% | 3 |
| 1272 | BEAM 128K o BEAM long-context memory benchmark at 128K tokens. | Long-context memory | — | 62.1% | — | G Muse Glimmer 30B | 65.1% | 3 |
| 1273 | BFCL v4 Multi-Turn o Multi-turn category of the Berkeley Function Calling Leaderboard v4. | Function calling | — | 53.9% | — | G Mercury Voice | 60.5% | 3 |
| 1274 | BioLP-Bench o Comprehensive biomedical language processing benchmark evaluating LLMs across tasks like NER, relation extraction, and QA. | Biomedical NLP | — | 34.6% | — | 🇺🇸 Grok 4 | 47.0% | 3 |
| 1275 | C3-Bench o Comprehensive Chinese reasoning capability benchmark. | Reasoning (Chinese) | — | 76.7% | — | 🇨🇳 GLM-4.5 Base | 83.1% | 3 |
| 1276 | CC-OCR (Multilingual) o Multilingual text recognition track of the CC-OCR benchmark. | OCR (multilingual) | — | 61.1% | — | G Qianfan-OCR | 76.7% | 3 |
| 1277 | ChartBench o Chart comprehension benchmark covering annotated and unannotated chart types. | Chart understanding | — | 78.0% | — | G Qianfan-OCR | 85.9% | 3 |
| 1278 | COCO Captions o COCO image captioning benchmark (Karpathy test split), CIDEr score. | Image captioning | — | — | — | 🇺🇸 T5Gemma 2 4B-4B | 105.4 | 3 |
| 1279 | Cybersecurity CTF o Cybersecurity Capture The Flag challenges benchmark. | Cybersecurity CTF | — | 70.9% | — | 🇺🇸 GPT-5.3 Codex | 77.6% | 3 |
| 1280 | E.T. Bench o E.T. Bench: event-level and time-sensitive video understanding benchmark. | Video event understanding | — | 62.3% | — | G Glimpse | 63.6% | 3 |
| 1281 | FACTS Parametric o FACTS suite parametric-knowledge split: closed-book factoid questions answered without retrieval. | Factuality | — | 41.2% | — | 🇨🇳 DeepSeek V4-Pro Base | 62.6% | 3 |
| 1282 | FaithEval (Inconsistent) o FaithEval inconsistent-context subset: detecting contradictory evidence in retrieved context. | Context faithfulness | — | 97.1%saturated | — | G Thomson-1.0-Small | 97.7% | 3 |
| 1283 | FrontierCode (Diamond) o Cognition's FrontierCode agentic coding benchmark, Diamond subset; patch correctness graded by held-out tests plus rubric. | Software engineering | — | 16.1% | — | 🇺🇸 Claude Fable 5 | 29.3% | 3 |
| 1284 | FrontierMath o Frontier mathematics benchmark. | Math (frontier) | — | 13.7% | — | 🇺🇸 GPT-5 pro | 32.1% | 3 |
| 1285 | GSM8K Platinum o Harder subset/setting of GSM8K grade-school math problems. | Math (grade-school, hard) | — | 86.2% | — | 🇨🇳 Kimi-Linear-Base | 89.6% | 3 |
| 1286 | HarmfulQA o Harmful question set testing models' ability to avoid unsafe answers. | Safety | — | 84.2% | — | G K2-THINK | 99.0% | 3 |
| 1287 | HealthBench-Hard Hallucinations o Measures hallucination and unsafe medical advice under hard clinical scenarios. | Medical hallucination safety | — | — | — | 🇺🇸 GPT-5 | ↓ 1.6% | 3 |
| 1288 | HelpSteer2 o NVIDIA HelpSteer2 helpfulness/attribute rating, posed as decisions. | Response quality rating | — | 36.9% | — | G Decider 4B (Mapika) | 42.0% | 3 |
| 1289 | HLE-VL o Vision-language HLE benchmark. | Holistic language evaluation (vision-language) | — | 28.8% | — | 🇺🇸 Gemini 3 Pro | 36.0% | 3 |
| 1290 | HumanEval-FIM o Fill-in-the-middle variant of HumanEval measuring code infilling. | Code infilling | — | 84.9% | — | 🇫🇷 Codestral 22B | 91.6% | 3 |
| 1291 | IndicMMLU-Pro o MMLU-Pro translated/adapted to Indic languages. | Indic languages | — | 72.7% | — | 🇺🇸 Gemma 4 31B (IT) | 79.0% | 3 |
| 1292 | IPhO 2025 (Theory) o Theory problems of the 2025 International Physics Olympiad. | Physics olympiad | — | 87.9% | — | 🇺🇸 GPT-5.4 Pro | 93.5% | 3 |
| 1293 | ISL/OSL 8k/16k o Relative throughput on ISL/OSL 8k/16k context workloads. | Throughput | — | — | — | 🇺🇸 Nemotron-3-Nano-30B-A3B | 3.30 | 3 |
| 1294 | LEXam o LEXam benchmark of law-school exam questions (open and multiple-choice) across jurisdictions. | Legal exams | — | 56.5% | — | G Darwin-180B-RSI | 68.9% | 3 |
| 1295 | MiniF2F pass@1 o MiniF2F competition benchmark pass@1 accuracy. | Math competition | — | 22.6% | — | 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | 50.0% | 3 |
| 1296 | MMLU (cloze) o Cloze-form MMLU evaluation variant. | Multi-domain knowledge (cloze) | — | 30.8% | — | 🇺🇸 SmolLM2 135M Base | 31.5% | 3 |
| 1297 | MMSearch-Plus o Harder MMSearch variant requiring fine-grained visual reasoning plus multi-step search. | Multimodal search | — | 27.1% | — | 🇨🇳 GLM-5V-Turbo | 30.0% | 3 |
| 1298 | MRCR 16K-8N o OpenAI MRCR v2 8-needle, 8K-16K context bucket. | Long-context reasoning | — | 91.2%saturated | — | 🇺🇸 GPT-5.5 | 93.0% | 3 |
| 1299 | MRCR 32K-8N o OpenAI MRCR v2 8-needle, 16K-32K context bucket. | Long-context reasoning | — | 96.3%saturated | — | 🇺🇸 GPT-5.4 | 97.2% | 3 |
| 1300 | MRCR 8K-8N o OpenAI MRCR v2 8-needle, 4K-8K context bucket. | Long-context reasoning | — | 97.9%saturated | — | 🇺🇸 GPT-5.2 | 98.2% | 3 |
| 1301 | Needle In A Haystack o Needle In A Haystack test for locating hidden facts in long contexts. | Long-context retrieval | — | 98.4%saturated | — | G MobileLLM P1 Base | 100.0% | 3 |
| 1302 | Objectron o Objectron benchmark for 3D object detection in video captures. | Object detection | — | 28.3% | — | 🇨🇳 Qwen3-VL-235B-A22B Thinking | 71.2% | 3 |
| 1303 | OpenAI Internal Advanced Cybersecurity Completion Rate o OpenAI internal eval (Daybreak announcement): share of advanced, dual-use cybersecurity requests (exploit chains, auth bypass, privilege escalation) the model completes. | Cybersecurity | — | 51.4% | — | 🇺🇸 GPT-5.6 Cyber | 95.0% | 3 |
| 1304 | OpenAI Internal Data Science Tasks o OpenAI internal eval reported in the GPT-6 Astra announcement: data-science tasks. | Data science | — | 35.4% | — | 🇺🇸 GPT-6 Astra | 40.9% | 3 |
| 1305 | OpenAI Internal Design Tasks o OpenAI internal eval reported in the GPT-6 Astra announcement: professional design tasks. | Design | — | 44.4% | — | 🇺🇸 GPT-6 Astra | 50.0% | 3 |
| 1306 | OpenAI Internal Vulnerability Discovery and Report Writing o OpenAI internal eval (Daybreak announcement): open-ended vulnerability finding in a repo with a known vulnerability, PoC and report quality. | Cybersecurity | — | 14.7% | — | 🇺🇸 GPT-5.6 Sol | 17.9% | 3 |
| 1307 | OpenAI Internal Zero-Day Discovery Eval o OpenAI internal eval (Daybreak announcement): find zero-day vulnerabilities in current open-source releases, build max-impact PoCs and write-ups; graded on severity, impact and calibration. | Cybersecurity | — | 19.3% | — | 🇺🇸 GPT-5.6 Cyber | 24.5% | 3 |
| 1308 | OpenRewrite-Eval o OpenRewrite evaluation; micro-averaged RougeL. | Rewrite quality | — | — | — | 🇨🇳 Qwen2.5 1.5B Instruct | 46.9 | 3 |
| 1309 | PenguinScrolls o Long-context reading comprehension and reasoning benchmark. | Long-context eval | — | 79.2% | — | 🇨🇳 Hunyuan-Large-Instruct | 85.2% | 3 |
| 1310 | ScreenSpot-Pro (with tools) o ScreenSpot-Pro evaluated with tool access (e.g. zoom/crop). | Screen UI locators (with tools) | — | 86.2% | — | 🇺🇸 Claude Opus 4.8 | 87.9% | 3 |
| 1311 | SpatialSense VQA o SpatialSense spatial-relation recognition posed as VQA (as used in Gemma/PaliGemma evaluations). | Spatial reasoning | — | 50.8% | — | 🇺🇸 T5Gemma 2 4B-4B | 51.7% | 3 |
| 1312 | SpeechRole o SpeechRole: speech role-playing agent benchmark (aggregate judge score, unbounded scale). | Spoken role-play | — | — | — | 🇺🇸 Gemini 3.1 Pro | 124.2 | 3 |
| 1313 | SRE-Bench o Software reverse engineering of binaries without source code (single-attempt solve rate). | Cybersecurity | — | 52.1% | — | 🇺🇸 GPT-6 Astra | 88.0% | 3 |
| 1314 | SWE-Lancer IC Diamond o Individual Contributor Diamond subset of SWE-Lancer. | Code repair (freelance) | — | 77.3% | — | 🇺🇸 GPT-5.3 Codex | 81.4% | 3 |
| 1315 | TallyQA o TallyQA open-ended counting questions over images (simple and complex). | Visual counting | — | 32.7% | — | 🇺🇸 T5Gemma 2 4B-4B | 39.6% | 3 |
| 1316 | URO-Bench Pro: Oral Conversation o URO-Bench pro track, oral-conversation dimension, speech-to-speech (%). | Spoken dialogue | — | 99.2%saturated | — | 🇨🇳 Qwen3.5-Omni-Plus | 99.8% | 3 |
| 1317 | URO-Bench Pro: Reasoning o URO-Bench pro track, reasoning dimension, speech-to-speech (%). | Spoken dialogue | — | 84.7% | — | 🇨🇳 Qwen3.5-Omni-Plus | 86.3% | 3 |
| 1318 | URO-Bench Pro: Understanding o URO-Bench pro track, understanding dimension, speech-to-speech (%). | Spoken dialogue | — | 66.5% | — | 🇺🇸 Gemini 3.1 Pro | 69.1% | 3 |
| 1319 | Vibe Code Bench v1.1 o Vals AI Vibe Code Bench v1.1: build working web applications from natural-language specs. | Application building | — | 55.3% | — | G MiniMax M3.1 Flash (max) | 73.9% | 3 |
| 1320 | WebVoyager2 o Web navigation and interaction tasks for LLM agents (v2). | Web agents | — | 62.7% | — | 🇨🇳 GLM-4.5V | 84.4% | 3 |
| 1321 | Winogender o Coreference resolution dataset for measuring gender bias. | Gender bias (coreference) | — | 73.1% | — | 🇺🇸 Llama 3.3 70B Instruct | 84.3% | 3 |
| 1322 | WMDP Bio o Weapons of Mass Destruction Proxy benchmark for biosecurity, measuring hazardous biological knowledge without info hazards. | Biosecurity knowledge | — | — | — | 🇺🇸 Zephyr 7B | ↓ 63.7% | 3 |
| 1323 | WMDP Chem o WMDP benchmark for chemical security, evaluating knowledge relevant to chemical weapons development. | Chemical security knowledge | — | — | — | 🇺🇸 Zephyr 7B | ↓ 45.8% | 3 |
| 1324 | WMDP Cyber o WMDP benchmark for cybersecurity, assessing knowledge that could aid in cyber weapons development. | Cybersecurity knowledge | — | — | — | 🇺🇸 Zephyr 7B | ↓ 44.0% | 3 |
| 1325 | XQuAD o Cross-lingual Question Answering Dataset: SQuAD v1.1 subset professionally translated into 10 languages. | Multilingual QA | — | 58.2% | — | 🇺🇸 T5Gemma 2 4B-4B | 70.6% | 3 |
| 1326 | AM-2K (Box) Matting (MAD) o AM-2K (Box): box-prompted animal matting; mean absolute difference x10^3 (lower is better). | Image matting | — | — | — | G WeVisOne-VL-4B | ↓ 5.00 | 2 |
| 1327 | ARC-Challenge (DE) o German translation of the ARC Challenge benchmark. | Science QA (German) | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 0.669 | 2 |
| 1328 | ARC-Easy (DE) o German translation of the ARC Easy science QA benchmark. | Science QA (German) | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 0.848 | 2 |
| 1329 | BIG-Bench o BIG-bench overall performance (original). | Multi-task reasoning | — | 45.2% | — | 🇺🇸 Gemma 2 7B | 55.1% | 2 |
| 1330 | BixBench o Benchmark of real-world bioinformatics and data-analysis tasks. | Bioinformatics | — | 77.3% | — | 🇺🇸 GPT-5.5 | 80.5% | 2 |
| 1331 | BOLD o Bias in Open-ended Language Dataset probing demographic biases in text generation. | Bias evaluation | — | — | — | 🇫🇷 Mixtral 8x 7B | ↓ 0.084 | 2 |
| 1332 | BrowseComp (Agent Swarm) o BrowseComp evaluated with Moonshot's Agent Swarm multi-agent orchestration (parallel sub-agents) instead of a single agent. | Web browsing agents | — | 82.3% | — | 🇨🇳 Kimi-K2.6 Thinking | 86.3% | 2 |
| 1333 | ChartQA-Pro (with tools) o ChartQA-Pro evaluated with tool access. | Chart understanding (with tools) | — | 71.1% | — | 🇺🇸 Claude Opus 4.8 | 72.3% | 2 |
| 1334 | Circa o Interpreting indirect yes/no answers in dialogue. | Indirect answers | — | 86.9% | — | G pplx-decider-v1-27b | 89.2% | 2 |
| 1335 | CloningScenarios o Safety benchmark that red-teams models with cloning-related misuse scenarios to measure compliance and refusal rates. | Biosecurity refusal | — | — | — | 🇺🇸 Grok 4 | ↓ 45.0% | 2 |
| 1336 | CORE o Comprehensive Ontological Relation Evaluation for Large Language Models. | Ontological reasoning | — | 50.5% | — | G Nanbeige4.1-3B | 53.5% | 2 |
| 1337 | CoVoST o CoVoST speech-to-text translation benchmark (BLEU). | Speech translation | — | — | — | 🇺🇸 Gemma 4 E4B IT | 35.5 | 2 |
| 1338 | Dolphin-Page o Dolphin Page benchmark measuring OCR fidelity and structured extraction on multi-layout documents. | Document OCR | — | — | — | 🇺🇸 Dolphin 1.5 | ↓ 7.4% | 2 |
| 1339 | Economically important tasks o Evaluation suite of real-world, economically impactful tasks across key industries and workflows. | Industry QA (cross-domain) | — | 40.3% | — | 🇺🇸 GPT-5 | 47.1% | 2 |
| 1340 | EVG o EVG document OCR benchmark evaluating recognition accuracy and layout extraction. | Document OCR | — | — | — | 🇺🇸 Dolphin 1.5 | ↓ 3.0% | 2 |
| 1341 | ExploitBench (Jun-Aug 2026) o ExploitBench restricted to vulnerabilities disclosed June-August 2026, after model training cutoffs. | Cybersecurity | — | 22.3% | — | 🇺🇸 GPT-6 Astra | 39.0% | 2 |
| 1342 | ExploitGym-Honeypot o ExploitGym honeypot variant measuring how often agents take bait out of scope (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | 2 | |
| 1343 | FairPoliticsQA o Sakana AI benchmark of balanced, fair answers to Japanese political questions. | Political fairness (Japanese) | — | 45.2% | — | G Sakana Namazu | 56.3% | 2 |
| 1344 | FinancialPhraseBank o Sentence-level financial news sentiment classification. | Financial sentiment | — | 80.6% | — | G pplx-decider-v1-27b | 84.2% | 2 |
| 1345 | FLEURS o FLEURS multilingual automatic speech recognition benchmark; word error rate, lower is better. | Speech recognition | — | — | — | 🇺🇸 Gemma 4 E4B IT | ↓ 0.1% | 2 |
| 1346 | Fox-Page-cn o Fox Page benchmark evaluating OCR accuracy and layout understanding on Chinese document pages. | Document OCR (Chinese) | — | — | — | 🇺🇸 Dolphin 1.5 | ↓ 0.8% | 2 |
| 1347 | Fox-Page-en o Fox Page benchmark evaluating OCR accuracy and layout understanding on English document pages. | Document OCR (English) | — | — | — | 🇺🇸 Dolphin 1.5 | ↓ 0.7% | 2 |
| 1348 | Full Text MMLU o Full-context MMLU variant evaluating reasoning over long passages. | Multi-domain knowledge (long-form) | — | — | — | 🇺🇸 Llama 3.3 70B Instruct | 83.0 | 2 |
| 1349 | GSM8K (DE) o German translation of the GSM8K grade-school math word problems. | Math (grade-school, German) | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 0.630 | 2 |
| 1350 | HellaSwag (DE) o German translation of the HellaSwag commonsense benchmark. | Commonsense reasoning (German) | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 0.727 | 2 |
| 1351 | HMMT o Harvard–MIT Mathematics Tournament problems. | Math (competition) | — | 96.7%saturated | — | 🇺🇸 GPT-5 pro | 100.0% | 2 |
| 1352 | ImageMining o Multimodal tool-use benchmark of mining information from images with tools. | Multimodal tool use | — | 27.5% | — | 🇨🇳 GLM-5V-Turbo | 30.7% | 2 |
| 1353 | ImplicitMemBench o Tests implicit memory: whether agents apply previously seen preferences and procedures without being explicitly asked to recall them. | Long-term memory | — | 46.2% | — | G MemT-1.0-Flash | 49.2% | 2 |
| 1354 | JudgeBench o Pairwise response-correctness judgement benchmark for LLM judges. | LLM-as-judge | — | 78.4% | — | G Jev 1.13 | 78.6% | 2 |
| 1355 | LOCA-Bench o LOCA-Bench long-context agent benchmark (environment description length 256K). | Long-context agents | — | 53.2% | — | 🇺🇸 Claude Opus 4.7 | 57.0% | 2 |
| 1356 | LoCoMo o Very long-term conversational memory: QA over multi-session dialogues. | Long-term memory | — | 61.3% | — | G MemT-1.0-Flash | 65.1% | 2 |
| 1357 | MakeMeSay o Adversarial benchmark testing model robustness against manipulation attempts. Lower is better. | Adversarial robustness | — | — | — | 🇺🇸 Grok 4.1 (Thinking) | 2 | |
| 1358 | MAVERIX o Benchmark requiring joint video and audio understanding (situational awareness, social sentiment). | Audio-visual video understanding | — | 67.0% | — | 🇺🇸 Gemini 2.5 Flash | 67.4% | 2 |
| 1359 | MemBench o Benchmark of factual and reflective memory of LLM-based agents across participation and observation scenarios. | Long-term memory | — | 72.7% | — | G MemT-1.0-Flash | 73.5% | 2 |
| 1360 | MemoryAgentBench o Evaluates memory agents on accurate retrieval, test-time learning, long-range understanding and conflict resolution. | Long-term memory | — | 16.4% | — | G MemT-1.0-Flash | 17.4% | 2 |
| 1361 | MMDU o Multi-turn, multi-image dialog understanding benchmark with GPT-4o-judged scores. | Multi-turn multi-image dialog | — | 61.5% | — | 🇺🇸 GPT-4V(ision) | 66.3% | 2 |
| 1362 | OmniMMI o OmniMMI benchmark for multimodal interaction across video streams. | Multimodal interaction | — | 51.3% | — | G Seed1.8 | 53.0% | 2 |
| 1363 | Open Rewrite o Rewrite benchmark assessing open-ended editing and directive-following quality. | Instruction following | — | 46.3% | — | G MobileLLM P1 | 51.0% | 2 |
| 1364 | OpenAI Internal Circumvention o OpenAI internal eval reported in the GPT-6 Astra announcement: rate of circumventing safeguards (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | 2 | |
| 1365 | OpenAI Internal Computer-Use Safety (AutoReview) o OpenAI internal eval reported in the GPT-6 Astra announcement: unsafe computer-use action rate with AutoReview enabled (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 1.8% | 2 |
| 1366 | OpenAI Internal Expert-SWE o OpenAI internal eval (GPT-5.5 announcement): expert-level, long-horizon software-engineering tasks. | Software engineering | — | 70.8% | — | 🇺🇸 GPT-5.5 | 73.1% | 2 |
| 1367 | OpenAI Internal Hallucination o OpenAI internal eval reported in the GPT-6 Astra announcement: hallucination rate (lower is better). | Hallucination | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 4.2% | 2 |
| 1368 | OpenScore String Quartets o Optical music recognition on OpenScore string quartet scores; score is 1 - OMR normalized edit distance. | Optical music recognition | — | — | — | 🇺🇸 GPT-6 Astra | 0.840 | 2 |
| 1369 | P3M-500-NP (Box) Matting (MAD) o P3M-500-NP (Box): box-prompted portrait matting; mean absolute difference x10^3 (lower is better). | Image matting | — | — | — | G WeVisOne-VL-2B | ↓ 6.60 | 2 |
| 1370 | PerMemSafe o Personal-memory safety benchmark: appropriate use and protection of stored user memories. | Memory safety | — | 36.1% | — | G MemT-1.0-Nano | 38.7% | 2 |
| 1371 | PersonaMem o Evaluates whether models track evolving user personas and preferences across long interaction histories. | Personalization | — | 57.0% | — | G MemT-1.0-Flash | 61.6% | 2 |
| 1372 | Perspective Bench o Re-alignment score over 80 prompts measuring adherence to the Public AI Constitution (balanced perspectives). | Value alignment | — | 54.5% | — | G Snowdon1.1-Small | 93.0% | 2 |
| 1373 | PPM-100 (Box) Matting (MAD) o PPM-100 (Box): box-prompted portrait matting; mean absolute difference x10^3 (lower is better). | Image matting | — | — | — | G WeVisOne-VL-2B | ↓ 5.30 | 2 |
| 1374 | PRDBench o Product Requirements Document benchmark for evaluating agentic coding capabilities. | Agentic coding | — | 27.5% | — | 🇨🇳 LongCat-Flash-Lite | 39.6% | 2 |
| 1375 | ProtocolQA o Protocol question answering benchmark evaluating understanding of scientific protocols and procedures. | Protocol understanding and QA | — | 77.5% | — | 🇺🇸 Grok 4.1 (Thinking) | 79.0% | 2 |
| 1376 | Putnam 2025 o Natural-language proof solutions to the 2025 William Lowell Putnam Mathematical Competition (12 problems, 10 points each); score = points out of 120. | Math (competition, proof writing) | — | — | — | G Nomos 1 | 87 | 2 |
| 1377 | QuALITY o Long-document multiple-choice reading comprehension benchmark. | Long-context reading comprehension | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 48.8 | 2 |
| 1378 | Ref-L4 (test) o Ref-L4 referring expression comprehension on the test split. | Referring expressions | — | 88.3% | — | 🇨🇳 GLM-4.6V | 88.9% | 2 |
| 1379 | RULER-100 @1M o RULER-100 evaluation at a 1M context window. | Long-context eval | — | 81.9% | — | 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | 86.3% | 2 |
| 1380 | RULER-100 @256k o RULER-100 evaluation at a 256k context window. | Long-context eval | — | 91.2%saturated | — | 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | 92.9% | 2 |
| 1381 | RULER-100 @512k o RULER-100 evaluation at a 512k context window. | Long-context eval | — | 87.7% | — | 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | 91.3% | 2 |
| 1382 | SWE-Interact o Interactive SWE benchmark (75 tasks) where the agent must clarify underspecified issues with a simulated user before fixing them. | Interactive software engineering | — | 20.7% | — | 🇨🇳 Kimi-K3 | 21.3% | 2 |
| 1383 | TLDR9+ o Long-form summarization benchmark with nine-domain TLDR prompts plus extended variations. | Summarization | — | 16.8% | — | G MobileLLM P1 | 16.8% | 2 |
| 1384 | TruthfulQA (binary) o Binary-choice variant of TruthfulQA. | Truthfulness | — | 88.7% | — | G Jev 1.13 | 92.0% | 2 |
| 1385 | TruthfulQA (DE) o German translation of the TruthfulQA benchmark. | Truthfulness / hallucination (German) | — | — | — | 🇺🇸 Llama 3.3 70B Instruct | 0.174 | 2 |
| 1386 | WeRefMat (Category) Matting (MAD) o WeRefMat (Category): category-prompted matting; mean absolute difference x10^3 (lower is better). | Image matting | — | — | — | G WeVisOne-VL-4B | ↓ 16.4 | 2 |
| 1387 | WeRefMat (Referring) Matting (MAD) o WeRefMat (Referring): referring-expression matting; mean absolute difference x10^3 (lower is better). | Image matting | — | — | — | G WeVisOne-VL-4B | ↓ 9.80 | 2 |
| 1388 | WinoGrande (DE) o German translation of the WinoGrande pronoun resolution benchmark. | Coreference reasoning (German) | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 0.793 | 2 |
| 1389 | WMT16 En–De o WMT16 English–German translation benchmark (news). | Machine translation | — | — | — | 🇺🇸 Llama 3.3 70B Instruct | 38.8 | 2 |
| 1390 | WMT16 En–De (Instruct) o Instruction-tuned evaluation on the WMT16 English–German translation set. | Machine translation | — | — | — | 🇺🇸 Llama 3.3 70B Instruct | 37.9 | 2 |
| 1391 | XStoryCloze o Multilingual StoryCloze: choose the correct story ending in 11 languages. | Commonsense reasoning (multilingual) | — | 87.7% | — | 🇨🇳 Zhipu GLM 4-9B-Chat | 90.7% | 2 |
| 1392 | XWinograd o Multilingual Winograd schema benchmark (en, fr, jp, pt, ru, zh). | Coreference reasoning (multilingual) | — | 67.4% | — | 🇨🇳 Zhipu GLM 4-9B-Chat | 73.1% | 2 |
No benchmarks match. Try a shorter search or other filters.