LLM Benchmark Tracker · Updated

LLM Benchmark Tracker (Nov 2026)

Gemini 4 Carbon leads 7 of 11 benchmarks this month.

Compare frontier AI model performance across 11 industry-standard benchmarks. Track 42 leading models across 11 providers, including Gemini 4 Carbon, GPT-6 Astra, GPT-6.1 Sol/Terra, Claude Opus 5.5, DeepSeek V4.1 Flash, Qwen 3.8-Max, Kimi K3.1 Pro, GLM-5.3, MiniMax-M3.1 Flash, Mistral Large 4 and Grok 4.7.

42 models•11 benchmarks•Last updated: Nov 1, 2026
CapabilityBenchmark
GoogleGemini 4 Carbon
GoogleGemini 4 Argon
GoogleGemini 3.8 Flash
GoogleGemini 3.1 Pro
GoogleGemini 3 Pro
OpenAIGPT-6 Astra
OpenAIGPT-6.1 Sol
OpenAIGPT-6.1 Terra
OpenAIGPT-5.5
OpenAIGPT-5.2
AnthropicClaude Opus 5.5
AnthropicClaude Fable 5.1
AnthropicClaude Sonnet 5.5
AnthropicClaude Haiku 5.5
AnthropicClaude Opus 4.8
AnthropicClaude Opus 4.5
DeepSeekDeepSeek V4.1 Flash
DeepSeekDeepSeek V4 Pro
DeepSeekDeepSeek V3.2
DeepSeekDeepSeek R1
MetaMuse Wonder (Max)
MetaLlama 4 Maverick
MetaLlama 4 Scout
xAIGrok 4.7
xAIGrok 4 Heavy
xAIGrok 4
MistralMistral Large 4
QwenQwen 3.8-Max
QwenQwen 3.8-Flash-Next
QwenQwen 3.7-Max
QwenQwen 3.5 (397B)
Zhipu GLMGLM-5.3
Zhipu GLMGLM-5.3-Flash
Zhipu GLMGLM-5.2
Zhipu GLMGLM-5
Zhipu GLMGLM-4.6
KimiKimi K3.1 Pro
KimiKimi K3
KimiKimi K2.5
MiniMaxMiniMax-M3.1 Flash
MiniMaxMiniMax-M3
MiniMaxMiniMax-M2.7
InfoLaunch DateOct 2026Oct 2026Sep 2026Mar 2026Nov 2025Sep 2026Sep 2026Oct 2026Apr 2026Dec 2025Sep 2026Sep 2026Sep 2026Oct 2026May 2026Nov 2025Oct 2026Aug 2026Feb 2026May 2025Oct 2026Apr 2025Apr 2025Sep 2026Jul 2025Jul 2025Oct 2026Sep 2026Aug 2026May 2026Feb 2026Aug 2026Aug 2026Jun 2026Feb 2026Sep 2025Oct 2026Jul 2026Jan 2026Oct 2026Jun 2026Mar 2026
GeneralMMLUMassive Multitask Language Understanding94.8%93.6%92.6%92.8%92.0%94.5%94.0%92.9%92.8%92.5%94.2%93.8%92.6%90.8%92.9%91.5%93.2%94.1%93.9%90.8%93.5%88.2%85.8%93.4%91.2%90.4%91.6%94.0%92.2%93.6%92.4%93.7%91.6%93.4%92.1%90.2%93.9%93.1%91.4%92.5%91.8%90.6%
MMLU-ProMMLU Pro (10-choice reasoning)92.8%91.4%90.2%90.6%90.1%92.4%91.5%90.1%88.2%89.6%91.8%91.2%89.6%86.8%89.8%87.2%88.4%90.2%86.8%85.0%90.4%80.5%75.4%90.6%86.6%86.6%85.4%89.8%87.4%89.3%86.2%88.8%85.8%88.4%86.0%83.4%89.4%88.0%85.4%86.4%84.2%82.8%
SimpleQASimpleQA (Factuality)84.6%81.2%75.8%74.5%72.1%76.4%74.2%71.4%64.5%58.5%72.8%70.4%66.4%61.2%62.1%52.4%65.2%68.4%51.2%45.8%70.2%N/AN/A73.5%N/AN/A63.4%70.4%65.4%67.8%61.5%68.1%62.4%66.2%60.4%N/A71.2%66.8%N/A64.8%62.1%N/A
HellaSwagHellaSwag (10-shot)97.5%96.8%96.0%96.1%95.5%97.4%97.1%96.6%96.4%96.2%97.1%96.9%96.2%95.4%96.3%95.6%95.8%96.1%94.6%93.4%96.4%N/AN/A96.5%N/A94.5%95.2%96.2%95.6%95.8%95.1%96.0%95.2%95.7%95.0%N/A96.3%95.6%N/A95.4%95.1%N/A
ReasoningGPQA DiamondGraduate-Level Google-Proof Q&A (Diamond)96.8%94.8%93.2%94.3%91.9%96.3%95.6%93.8%91.4%93.2%94.6%93.4%91.5%88.6%92.4%87.0%90.9%92.8%80.3%81.0%93.1%69.8%64.2%93.8%88.9%87.5%78.0%93.7%91.3%86.4%84.2%91.7%87.4%91.2%86.2%81.0%94.0%93.5%88.4%90.4%89.2%89.8%
ARC-CARC-Challenge (25-shot)98.5%97.8%97.1%97.3%96.5%98.4%98.1%97.6%97.4%97.2%98.2%98.0%97.4%96.6%97.5%96.8%97.1%97.4%96.1%94.8%97.6%93.5%91.8%97.5%96.0%95.5%96.2%97.6%96.9%97.1%96.4%97.4%96.5%97.2%96.3%95.1%97.7%97.0%95.8%96.8%96.5%95.9%
MathAIMEAmerican Invitational Mathematics Examination (2025/2026)99.3%97.3%96.0%95.0%95.0%100.0%100.0%98.7%98.0%100.0%99.7%98.7%96.7%93.3%96.3%94.0%94.7%95.8%84.6%87.5%96.0%N/AN/A98.7%100.0%98.8%75.1%96.8%94.0%92.7%91.0%99.3%94.7%99.2%95.3%93.9%96.7%95.3%93.3%95.0%94.6%94.2%
MATH-500MATH Benchmark (500 problems)99.5%99.0%98.5%98.6%98.0%99.6%99.4%99.1%99.0%98.8%99.4%99.2%98.6%97.8%98.4%97.6%98.7%99.0%97.8%97.3%98.7%91.6%N/A99.1%97.6%96.8%96.4%99.1%98.4%98.6%97.9%99.1%98.0%98.9%98.2%97.4%99.0%98.5%97.6%98.4%98.1%97.8%
CodeSWE-benchSWE-bench Verified89.6%84.2%82.4%80.6%76.2%89.4%89.2%86.8%82.7%80.0%89.9%91.2%86.4%81.4%88.6%80.9%85.8%86.4%74.2%65.6%85.8%N/AN/A87.5%73.4%71.0%75.0%85.6%82.6%80.4%75.8%84.6%79.4%82.8%77.8%68.0%86.8%81.2%71.2%82.4%80.5%78.0%
HumanEvalHumanEval (Pass@1)97.4%96.2%95.4%95.1%94.2%97.6%97.2%96.5%96.2%95.8%97.8%97.5%96.4%94.8%96.1%94.5%95.6%95.8%91.4%89.2%95.6%88.5%85.2%96.0%93.5%92.0%93.8%96.0%95.0%95.2%93.8%95.8%94.1%95.4%93.8%91.6%96.2%94.8%92.8%94.9%94.2%93.4%
ImageMMMUMMMU / MMMU-Pro Multimodal Understanding91.2%88.4%86.2%83.9%78.5%89.6%89.1%85.5%86.4%85.4%87.7%90.6%85.2%82.6%86.6%80.7%82.8%84.0%N/AN/A86.8%73.4%69.4%86.2%78.0%N/A76.4%88.0%84.8%85.8%82.4%85.2%83.6%84.5%N/AN/A85.4%83.4%79.5%84.2%83.0%N/A
General
MMLU
Gemini 4 Carbon★94.8%
Gemini 4 Argon★93.6%
Gemini 3.8 Flash92.6%
Gemini 3.1 Pro92.8%
Gemini 3 Pro92.0%
GPT-6 Astra★94.5%
GPT-6.1 Sol★94.0%
GPT-6.1 Terra★92.9%
GPT-5.592.8%
GPT-5.292.5%
Claude Opus 5.5★94.2%
Claude Fable 5.1★93.8%
Claude Sonnet 5.592.6%
Claude Haiku 5.590.8%
Claude Opus 4.892.9%
Claude Opus 4.591.5%
DeepSeek V4.1 Flash★93.2%
DeepSeek V4 Pro★94.1%
DeepSeek V3.293.9%
DeepSeek R190.8%
Muse Wonder (Max)★93.5%
Llama 4 Maverick88.2%
Llama 4 Scout85.8%
Grok 4.7★93.4%
Grok 4 Heavy91.2%
Grok 490.4%
Mistral Large 4★91.6%
Qwen 3.8-Max★94.0%
Qwen 3.8-Flash-Next★92.2%
Qwen 3.7-Max93.6%
Qwen 3.5 (397B)92.4%
GLM-5.3★93.7%
GLM-5.3-Flash91.6%
GLM-5.293.4%
GLM-592.1%
GLM-4.690.2%
Kimi K3.1 Pro★93.9%
Kimi K393.1%
Kimi K2.591.4%
MiniMax-M3.1 Flash★92.5%
MiniMax-M391.8%
MiniMax-M2.790.6%
MMLU-Pro
Gemini 4 Carbon★92.8%
Gemini 4 Argon★91.4%
Gemini 3.8 Flash90.2%
Gemini 3.1 Pro90.6%
Gemini 3 Pro90.1%
GPT-6 Astra★92.4%
GPT-6.1 Sol★91.5%
GPT-6.1 Terra★90.1%
GPT-5.588.2%
GPT-5.289.6%
Claude Opus 5.5★91.8%
Claude Fable 5.1★91.2%
Claude Sonnet 5.589.6%
Claude Haiku 5.586.8%
Claude Opus 4.889.8%
Claude Opus 4.587.2%
DeepSeek V4.1 Flash★88.4%
DeepSeek V4 Pro★90.2%
DeepSeek V3.286.8%
DeepSeek R185.0%
Muse Wonder (Max)★90.4%
Llama 4 Maverick80.5%
Llama 4 Scout75.4%
Grok 4.7★90.6%
Grok 4 Heavy86.6%
Grok 486.6%
Mistral Large 4★85.4%
Qwen 3.8-Max★89.8%
Qwen 3.8-Flash-Next★87.4%
Qwen 3.7-Max89.3%
Qwen 3.5 (397B)86.2%
GLM-5.3★88.8%
GLM-5.3-Flash85.8%
GLM-5.288.4%
GLM-586.0%
GLM-4.683.4%
Kimi K3.1 Pro★89.4%
Kimi K388.0%
Kimi K2.585.4%
MiniMax-M3.1 Flash★86.4%
MiniMax-M384.2%
MiniMax-M2.782.8%
SimpleQA
Gemini 4 Carbon★84.6%
Gemini 4 Argon★81.2%
Gemini 3.8 Flash75.8%
Gemini 3.1 Pro74.5%
Gemini 3 Pro72.1%
GPT-6 Astra★76.4%
GPT-6.1 Sol★74.2%
GPT-6.1 Terra★71.4%
GPT-5.564.5%
GPT-5.258.5%
Claude Opus 5.5★72.8%
Claude Fable 5.1★70.4%
Claude Sonnet 5.566.4%
Claude Haiku 5.561.2%
Claude Opus 4.862.1%
Claude Opus 4.552.4%
DeepSeek V4.1 Flash★65.2%
DeepSeek V4 Pro★68.4%
DeepSeek V3.251.2%
DeepSeek R145.8%
Muse Wonder (Max)★70.2%
Llama 4 MaverickN/A
Llama 4 ScoutN/A
Grok 4.7★73.5%
Grok 4 HeavyN/A
Grok 4N/A
Mistral Large 4★63.4%
Qwen 3.8-Max★70.4%
Qwen 3.8-Flash-Next★65.4%
Qwen 3.7-Max67.8%
Qwen 3.5 (397B)61.5%
GLM-5.3★68.1%
GLM-5.3-Flash62.4%
GLM-5.266.2%
GLM-560.4%
GLM-4.6N/A
Kimi K3.1 Pro★71.2%
Kimi K366.8%
Kimi K2.5N/A
MiniMax-M3.1 Flash★64.8%
MiniMax-M362.1%
MiniMax-M2.7N/A
HellaSwag
Gemini 4 Carbon★97.5%
Gemini 4 Argon★96.8%
Gemini 3.8 Flash96.0%
Gemini 3.1 Pro96.1%
Gemini 3 Pro95.5%
GPT-6 Astra★97.4%
GPT-6.1 Sol★97.1%
GPT-6.1 Terra★96.6%
GPT-5.596.4%
GPT-5.296.2%
Claude Opus 5.5★97.1%
Claude Fable 5.1★96.9%
Claude Sonnet 5.596.2%
Claude Haiku 5.595.4%
Claude Opus 4.896.3%
Claude Opus 4.595.6%
DeepSeek V4.1 Flash★95.8%
DeepSeek V4 Pro★96.1%
DeepSeek V3.294.6%
DeepSeek R193.4%
Muse Wonder (Max)★96.4%
Llama 4 MaverickN/A
Llama 4 ScoutN/A
Grok 4.7★96.5%
Grok 4 HeavyN/A
Grok 494.5%
Mistral Large 4★95.2%
Qwen 3.8-Max★96.2%
Qwen 3.8-Flash-Next★95.6%
Qwen 3.7-Max95.8%
Qwen 3.5 (397B)95.1%
GLM-5.3★96.0%
GLM-5.3-Flash95.2%
GLM-5.295.7%
GLM-595.0%
GLM-4.6N/A
Kimi K3.1 Pro★96.3%
Kimi K395.6%
Kimi K2.5N/A
MiniMax-M3.1 Flash★95.4%
MiniMax-M395.1%
MiniMax-M2.7N/A
Reasoning
GPQA Diamond
Gemini 4 Carbon★96.8%
Gemini 4 Argon★94.8%
Gemini 3.8 Flash93.2%
Gemini 3.1 Pro94.3%
Gemini 3 Pro91.9%
GPT-6 Astra★96.3%
GPT-6.1 Sol★95.6%
GPT-6.1 Terra★93.8%
GPT-5.591.4%
GPT-5.293.2%
Claude Opus 5.5★94.6%
Claude Fable 5.1★93.4%
Claude Sonnet 5.591.5%
Claude Haiku 5.588.6%
Claude Opus 4.892.4%
Claude Opus 4.587.0%
DeepSeek V4.1 Flash★90.9%
DeepSeek V4 Pro★92.8%
DeepSeek V3.280.3%
DeepSeek R181.0%
Muse Wonder (Max)★93.1%
Llama 4 Maverick69.8%
Llama 4 Scout64.2%
Grok 4.7★93.8%
Grok 4 Heavy88.9%
Grok 487.5%
Mistral Large 4★78.0%
Qwen 3.8-Max★93.7%
Qwen 3.8-Flash-Next★91.3%
Qwen 3.7-Max86.4%
Qwen 3.5 (397B)84.2%
GLM-5.3★91.7%
GLM-5.3-Flash87.4%
GLM-5.291.2%
GLM-586.2%
GLM-4.681.0%
Kimi K3.1 Pro★94.0%
Kimi K393.5%
Kimi K2.588.4%
MiniMax-M3.1 Flash★90.4%
MiniMax-M389.2%
MiniMax-M2.789.8%
ARC-C
Gemini 4 Carbon★98.5%
Gemini 4 Argon★97.8%
Gemini 3.8 Flash97.1%
Gemini 3.1 Pro97.3%
Gemini 3 Pro96.5%
GPT-6 Astra★98.4%
GPT-6.1 Sol★98.1%
GPT-6.1 Terra★97.6%
GPT-5.597.4%
GPT-5.297.2%
Claude Opus 5.5★98.2%
Claude Fable 5.1★98.0%
Claude Sonnet 5.597.4%
Claude Haiku 5.596.6%
Claude Opus 4.897.5%
Claude Opus 4.596.8%
DeepSeek V4.1 Flash★97.1%
DeepSeek V4 Pro★97.4%
DeepSeek V3.296.1%
DeepSeek R194.8%
Muse Wonder (Max)★97.6%
Llama 4 Maverick93.5%
Llama 4 Scout91.8%
Grok 4.7★97.5%
Grok 4 Heavy96.0%
Grok 495.5%
Mistral Large 4★96.2%
Qwen 3.8-Max★97.6%
Qwen 3.8-Flash-Next★96.9%
Qwen 3.7-Max97.1%
Qwen 3.5 (397B)96.4%
GLM-5.3★97.4%
GLM-5.3-Flash96.5%
GLM-5.297.2%
GLM-596.3%
GLM-4.695.1%
Kimi K3.1 Pro★97.7%
Kimi K397.0%
Kimi K2.595.8%
MiniMax-M3.1 Flash★96.8%
MiniMax-M396.5%
MiniMax-M2.795.9%
Math
AIME
Gemini 4 Carbon★99.3%
Gemini 4 Argon★97.3%
Gemini 3.8 Flash96.0%
Gemini 3.1 Pro95.0%
Gemini 3 Pro95.0%
GPT-6 Astra★100.0%
GPT-6.1 Sol★100.0%
GPT-6.1 Terra★98.7%
GPT-5.598.0%
GPT-5.2100.0%
Claude Opus 5.5★99.7%
Claude Fable 5.1★98.7%
Claude Sonnet 5.596.7%
Claude Haiku 5.593.3%
Claude Opus 4.896.3%
Claude Opus 4.594.0%
DeepSeek V4.1 Flash★94.7%
DeepSeek V4 Pro★95.8%
DeepSeek V3.284.6%
DeepSeek R187.5%
Muse Wonder (Max)★96.0%
Llama 4 MaverickN/A
Llama 4 ScoutN/A
Grok 4.7★98.7%
Grok 4 Heavy100.0%
Grok 498.8%
Mistral Large 4★75.1%
Qwen 3.8-Max★96.8%
Qwen 3.8-Flash-Next★94.0%
Qwen 3.7-Max92.7%
Qwen 3.5 (397B)91.0%
GLM-5.3★99.3%
GLM-5.3-Flash94.7%
GLM-5.299.2%
GLM-595.3%
GLM-4.693.9%
Kimi K3.1 Pro★96.7%
Kimi K395.3%
Kimi K2.593.3%
MiniMax-M3.1 Flash★95.0%
MiniMax-M394.6%
MiniMax-M2.794.2%
MATH-500
Gemini 4 Carbon★99.5%
Gemini 4 Argon★99.0%
Gemini 3.8 Flash98.5%
Gemini 3.1 Pro98.6%
Gemini 3 Pro98.0%
GPT-6 Astra★99.6%
GPT-6.1 Sol★99.4%
GPT-6.1 Terra★99.1%
GPT-5.599.0%
GPT-5.298.8%
Claude Opus 5.5★99.4%
Claude Fable 5.1★99.2%
Claude Sonnet 5.598.6%
Claude Haiku 5.597.8%
Claude Opus 4.898.4%
Claude Opus 4.597.6%
DeepSeek V4.1 Flash★98.7%
DeepSeek V4 Pro★99.0%
DeepSeek V3.297.8%
DeepSeek R197.3%
Muse Wonder (Max)★98.7%
Llama 4 Maverick91.6%
Llama 4 ScoutN/A
Grok 4.7★99.1%
Grok 4 Heavy97.6%
Grok 496.8%
Mistral Large 4★96.4%
Qwen 3.8-Max★99.1%
Qwen 3.8-Flash-Next★98.4%
Qwen 3.7-Max98.6%
Qwen 3.5 (397B)97.9%
GLM-5.3★99.1%
GLM-5.3-Flash98.0%
GLM-5.298.9%
GLM-598.2%
GLM-4.697.4%
Kimi K3.1 Pro★99.0%
Kimi K398.5%
Kimi K2.597.6%
MiniMax-M3.1 Flash★98.4%
MiniMax-M398.1%
MiniMax-M2.797.8%
Code
SWE-bench
Gemini 4 Carbon★89.6%
Gemini 4 Argon★84.2%
Gemini 3.8 Flash82.4%
Gemini 3.1 Pro80.6%
Gemini 3 Pro76.2%
GPT-6 Astra★89.4%
GPT-6.1 Sol★89.2%
GPT-6.1 Terra★86.8%
GPT-5.582.7%
GPT-5.280.0%
Claude Opus 5.5★89.9%
Claude Fable 5.1★91.2%
Claude Sonnet 5.586.4%
Claude Haiku 5.581.4%
Claude Opus 4.888.6%
Claude Opus 4.580.9%
DeepSeek V4.1 Flash★85.8%
DeepSeek V4 Pro★86.4%
DeepSeek V3.274.2%
DeepSeek R165.6%
Muse Wonder (Max)★85.8%
Llama 4 MaverickN/A
Llama 4 ScoutN/A
Grok 4.7★87.5%
Grok 4 Heavy73.4%
Grok 471.0%
Mistral Large 4★75.0%
Qwen 3.8-Max★85.6%
Qwen 3.8-Flash-Next★82.6%
Qwen 3.7-Max80.4%
Qwen 3.5 (397B)75.8%
GLM-5.3★84.6%
GLM-5.3-Flash79.4%
GLM-5.282.8%
GLM-577.8%
GLM-4.668.0%
Kimi K3.1 Pro★86.8%
Kimi K381.2%
Kimi K2.571.2%
MiniMax-M3.1 Flash★82.4%
MiniMax-M380.5%
MiniMax-M2.778.0%
HumanEval
Gemini 4 Carbon★97.4%
Gemini 4 Argon★96.2%
Gemini 3.8 Flash95.4%
Gemini 3.1 Pro95.1%
Gemini 3 Pro94.2%
GPT-6 Astra★97.6%
GPT-6.1 Sol★97.2%
GPT-6.1 Terra★96.5%
GPT-5.596.2%
GPT-5.295.8%
Claude Opus 5.5★97.8%
Claude Fable 5.1★97.5%
Claude Sonnet 5.596.4%
Claude Haiku 5.594.8%
Claude Opus 4.896.1%
Claude Opus 4.594.5%
DeepSeek V4.1 Flash★95.6%
DeepSeek V4 Pro★95.8%
DeepSeek V3.291.4%
DeepSeek R189.2%
Muse Wonder (Max)★95.6%
Llama 4 Maverick88.5%
Llama 4 Scout85.2%
Grok 4.7★96.0%
Grok 4 Heavy93.5%
Grok 492.0%
Mistral Large 4★93.8%
Qwen 3.8-Max★96.0%
Qwen 3.8-Flash-Next★95.0%
Qwen 3.7-Max95.2%
Qwen 3.5 (397B)93.8%
GLM-5.3★95.8%
GLM-5.3-Flash94.1%
GLM-5.295.4%
GLM-593.8%
GLM-4.691.6%
Kimi K3.1 Pro★96.2%
Kimi K394.8%
Kimi K2.592.8%
MiniMax-M3.1 Flash★94.9%
MiniMax-M394.2%
MiniMax-M2.793.4%
Image
MMMU
Gemini 4 Carbon★91.2%
Gemini 4 Argon★88.4%
Gemini 3.8 Flash86.2%
Gemini 3.1 Pro83.9%
Gemini 3 Pro78.5%
GPT-6 Astra★89.6%
GPT-6.1 Sol★89.1%
GPT-6.1 Terra★85.5%
GPT-5.586.4%
GPT-5.285.4%
Claude Opus 5.5★87.7%
Claude Fable 5.1★90.6%
Claude Sonnet 5.585.2%
Claude Haiku 5.582.6%
Claude Opus 4.886.6%
Claude Opus 4.580.7%
DeepSeek V4.1 Flash★82.8%
DeepSeek V4 Pro★84.0%
DeepSeek V3.2N/A
DeepSeek R1N/A
Muse Wonder (Max)★86.8%
Llama 4 Maverick73.4%
Llama 4 Scout69.4%
Grok 4.7★86.2%
Grok 4 Heavy78.0%
Grok 4N/A
Mistral Large 4★76.4%
Qwen 3.8-Max★88.0%
Qwen 3.8-Flash-Next★84.8%
Qwen 3.7-Max85.8%
Qwen 3.5 (397B)82.4%
GLM-5.3★85.2%
GLM-5.3-Flash83.6%
GLM-5.284.5%
GLM-5N/A
GLM-4.6N/A
Kimi K3.1 Pro★85.4%
Kimi K383.4%
Kimi K2.579.5%
MiniMax-M3.1 Flash★84.2%
MiniMax-M383.0%
MiniMax-M2.7N/A

November 2026 Frontier AI Standings: Key Takeaways

The late-2026 LLM landscape has shifted decisively from single-turn chat completion to test-time compute scaling, multi-day autonomous coding agents, and 1M+ token native multimodal reasoning. While legacy academic tests like 4-choice MMLU have saturated near 95%, rigorous benchmarks like GPQA Diamond, SWE-bench Verified, and MMMU-Pro separate today's frontier models.

Capability Benchmark Nov 2026 Leader Top Score Close Contenders
PhD Science Reasoning GPQA Diamond Gemini 4 Carbon (Google) 96.8% GPT-6 Astra (96.3%) · GPT-6.1 Sol (95.6%)
Agentic Software Engineering SWE-bench Verified Claude Fable 5.1 (Anthropic) 91.2% Claude Opus 5.5 (89.9%) · Gemini 4 Carbon (89.6%)
Code Generation (Pass@1) HumanEval Claude Opus 5.5 (Anthropic) 97.8% GPT-6 Astra (97.6%) · Claude Fable 5.1 (97.5%)
Competition Mathematics AIME (2025/2026) GPT-6 Astra & GPT-6.1 Sol (OpenAI) 100.0% Claude Opus 5.5 (99.7%) · Gemini 4 Carbon (99.3%)
Advanced Math Problem Solving MATH-500 GPT-6 Astra (OpenAI) 99.6% Gemini 4 Carbon (99.5%) · Claude Opus 5.5 (99.4%)
Multimodal Reasoning MMMU / MMMU-Pro Gemini 4 Carbon (Google) 91.2% Claude Fable 5.1 (90.6%) · GPT-6 Astra (89.6%)
10-Choice Multitask Reasoning MMLU-Pro Gemini 4 Carbon (Google) 92.8% GPT-6 Astra (92.4%) · Claude Opus 5.5 (91.8%)
Short-Form Factuality SimpleQA Gemini 4 Carbon (Google) 84.6% Gemini 4 Argon (81.2%) · GPT-6 Astra (76.4%)
Open-Weights & Global Contenders GPQA / SWE-bench Kimi K3.1 Pro & Qwen 3.8-Max 94.0% / 86.8% DeepSeek V4 Pro (92.8% / 86.4%) · GLM-5.3 (91.7% / 84.6%) · MiniMax-M3.1 (90.4% / 82.4%)

1. Google & OpenAI Trade the Reasoning Crown

Google DeepMind's Gemini 4 Carbon (released October 2026, #1 on the Artificial Analysis Intelligence Index v5.0) edges out the field on GPQA Diamond (96.8%), MMMU (91.2%), and SimpleQA (84.6%). OpenAI's GPT-6 Astra (September 2026) and GPT-6.1 Sol (DevDay, September 29, 2026) hit a perfect 100.0% ceiling on AIME and 99.6% on MATH-500 while scoring 96.3% and 95.6% on GPQA Diamond.

2. Anthropic Leads Agentic Software Engineering

Anthropic's specialized long-horizon model Claude Fable 5.1 (91.2% SWE-bench Verified) and flagship Claude Opus 5.5 (89.9% SWE-bench, 97.8% HumanEval) set the pace for multi-file repository debugging and autonomous terminal workflows, closely tracked by Gemini 4 Carbon (89.6%), GPT-6 Astra (89.4%), and GPT-6.1 Sol (89.2%).

3. Global Open-Weights Surge Within 2–4% of Frontier

Moonshot's Kimi K3.1 Pro (3T MoE; 94.0% GPQA, 86.8% SWE-bench, 67.2% Terminal-Bench 4.0), Alibaba's Apache 2.0 Qwen 3.8-Max (2.4T MoE; 93.7% GPQA, 85.6% SWE-bench) & Qwen 3.8-Flash-Next, DeepSeek V4 Pro & V4.1 Flash (MIT open weights), Zhipu's GLM-5.3 (99.3% AIME, 91.7% GPQA, 84.6% SWE-bench), MiniMax-M3.1 Flash, and Mistral Large 4 challenge Western proprietary flagships.

4. High-Efficiency "Workhorse" & Computer-Use Tiers Mature

Mid-tier and agentic computer-use models—led by OpenAI's GPT-6.1 Sol ($2/$10 per 1M tokens) and GPT-6.1 Terra (74.2% OSWorld 2.1), Google's Gemini 4 Argon (94.8% GPQA) and Gemini 3.8 Flash, Anthropic's Claude Sonnet 5.5 and Claude Haiku 5.5 (72.4% OSWorld 2.1, 1M context), and xAI's Grok 4.7—now outperform late-2025 flagships at dramatically lower latency and cost.

What Are LLM Benchmarks?

LLM benchmarks are standardized evaluation suites designed to measure the capabilities of large language models across reasoning, coding, mathematics, factual recall, and multimodal perception.

Early benchmarks like MMLU (introduced in 2020) tested broad undergraduate knowledge. As frontier models surpassed 90%+ on first-generation tests, the AI research community introduced contamination-resistant evaluations like GPQA Diamond (PhD-level science), SWE-bench Verified (real-world GitHub issue resolution), and MMMU-Pro (multimodal professional reasoning) to evaluate deep "System 2" deliberation and tool use.

Objective Measurement

Replaces vibe-based testing with reproducible quantitative baselines across 42+ models

Generational Tracking

Measures concrete gains from GPT-5.2 to GPT-6 Astra, Claude 4.5 to 5.5, and Gemini 3 to Gemini 4

Architecture Fit

Reveals whether a workload needs a flagship reasoning model or a fast, low-cost workhorse

Why Benchmarks Matter in 2026

For Software Engineers

Select the right model for IDE agents and CI/CD automation. Prioritize SWE-bench Verified and HumanEval when routing between Claude Fable 5.1, Claude Opus 5.5, GPT-6 Astra, and DeepSeek V4 Pro.

For Product & AI Leaders

Optimize inference unit economics. Benchmarks show where mid-tier models like GPT-6.1 Sol, Gemini 4 Argon, or Claude Sonnet 5.5 match 95%+ of flagship quality at one-fifth of the token cost.

For AI Researchers

Track benchmark saturation and capability frontiers across reasoning (GPQA Diamond), competition math (AIME), and hallucination resistance (SimpleQA).

Understanding Each Benchmark

Not all benchmarks test the same cognitive skill. Here is what each evaluation in our November 2026 matrix measures and how current frontier models perform.

General Knowledge & Factuality

MMLU

Massive Multitask Language Understanding

Tests knowledge across 57 subjects from elementary to professional level. In November 2026, top models score 93%–94.8% (surpassing the ~89.8% human expert baseline), making it a baseline saturation check.

MMLU-Pro

MMLU Pro (10-Choice Reasoning)

A harder, contamination-resistant evolution with 10 answer choices (vs 4) and reasoning-heavy questions. Gemini 4 Carbon (92.8%), GPT-6 Astra (92.4%), and Claude Opus 5.5 (91.8%) lead in late 2026.

SimpleQA

SimpleQA Factuality

Evaluates short-form factual recall and resistance to hallucination on fact-dense queries. Gemini 4 Carbon (84.6%) and Gemini 4 Argon (81.2%) lead all models as of November 2026.

Reasoning & Graduate Science

GPQA Diamond

Graduate-Level Google-Proof Q&A

PhD-level science questions in physics, chemistry, and biology designed to be 'Google-proof'. While human PhD domain experts score ~65%, Gemini 4 Carbon (96.8%) and GPT-6 Astra (96.3%) now exceed 96% with extended reasoning.

ARC-Challenge

AI2 Reasoning Challenge (25-shot)

Science exam questions requiring multi-step logical deduction and world knowledge beyond surface pattern matching. Frontier 2026 models routinely clear 97%–98.5%.

HellaSwag

HellaSwag Commonsense (10-shot)

Tests grounded commonsense reasoning through contextual completion. Leading late-2026 models cluster between 96% and 97.5%.

Competition Mathematics

AIME

American Invitational Mathematics Examination (2025/2026)

15 rigorous olympiad-qualifier problems requiring multi-step symbolic deduction. Test-time compute models like GPT-6 Astra and GPT-6.1 Sol achieve 100%, with Claude Opus 5.5 at 99.7% and Gemini 4 Carbon at 99.3%.

MATH-500

MATH Competition Benchmark (500 Problems)

500 high-school competition problems across algebra, geometry, number theory, combinatorics, and calculus. Top November 2026 reasoning models score 99.0%–99.6%.

Coding & Agentic Software Engineering

SWE-bench

SWE-bench Verified

The industry gold standard for software engineering: resolving real GitHub issues across multi-file Python repositories. Claude Fable 5.1 (91.2%), Claude Opus 5.5 (89.9%), Gemini 4 Carbon (89.6%), and GPT-6 Astra (89.4%) lead in November 2026.

HumanEval

HumanEval Pass@1

164 Python programming problems testing zero-shot function synthesis and unit-test pass rates on the first attempt. Top 2026 models score 96%–97.8%.

Multimodal & Visual Reasoning

MMMU

MMMU / MMMU-Pro Multimodal Understanding

College-level questions across 30+ academic disciplines requiring joint visual interpretation of charts, schematics, chemical structures, and text. Gemini 4 Carbon (91.2%) and Claude Fable 5.1 (90.6%) lead in November 2026.

How to Choose the Right LLM (Nov 2026 Framework)

1️⃣

Match the Benchmark to Your Workload

Building an autonomous coding agent? Weight SWE-bench Verified and HumanEval. Building an enterprise RAG or compliance assistant? Weight SimpleQA and MMLU-Pro. Scientific or quantitative analysis? Focus on GPQA Diamond and AIME.

2️⃣

Evaluate Multi-Model Routing

Production AI systems in 2026 rarely rely on a single model. Route complex architecture planning to Claude Opus 5.5 or GPT-6 Astra, multimodal document extraction to Gemini 4 Carbon, and high-volume background tasks to DeepSeek V4 Pro or Claude Sonnet 5.5.

3️⃣

Consider Cost, Latency & Thinking Budgets

Flagship models like Gemini 4 Carbon, GPT-6 Astra, and Claude Opus 5.5 command premium compute pricing. Faster tiers like GPT-6.1 Sol, Gemini 4 Argon, Claude Sonnet 5.5, or DeepSeek V4 Pro often deliver 95%+ of the capability at one-fifth of the cost.

4️⃣

Validate on Your Own Evaluation Suite

Public benchmarks are starting filters, not final answers. Always test candidate models against your own domain prompts, latency SLAs, and tool-calling schemas before production rollout.

Frequently Asked Questions

What is the best LLM in November 2026?

There's no single "best" LLM — it depends on your workload. As of November 2026: Google's Gemini 4 Carbon leads overall intelligence (GPQA Diamond: 96.8%, MMMU: 91.2%, SimpleQA: 84.6%, MMLU-Pro: 92.8%), OpenAI's GPT-6 Astra and GPT-6.1 Sol lead competition mathematics (AIME: 100.0%, MATH-500: 99.6%), while Anthropic's Claude Fable 5.1 (91.2% SWE-bench Verified) and Claude Opus 5.5 (89.9% SWE-bench, 97.8% HumanEval) lead real-world software engineering and agentic coding. For open-weights and global cost-efficiency, Kimi K3.1 Pro, Qwen 3.8-Max, DeepSeek V4 Pro & V4.1 Flash, GLM-5.3, MiniMax-M3.1 Flash, Mistral Large 4, and Meta's Muse Wonder (Max) deliver near-frontier performance. See our AI Assistant Comparison for a full workflow breakdown.

Which LLM is best for coding and software engineering in 2026?

For real-world repository software engineering and long-horizon autonomous agents, Anthropic's Claude Fable 5.1 leads SWE-bench Verified (91.2%), followed closely by Claude Opus 5.5 (89.9%), Gemini 4 Carbon (89.6%), GPT-6 Astra (89.4%), and GPT-6.1 Sol (89.2%). Among open-weights and Asian frontier labs, Kimi K3.1 Pro (86.8% SWE-bench, 67.2% Terminal-Bench 4.0), DeepSeek V4 Pro (86.4%) & V4.1 Flash (85.8%), Qwen 3.8-Max (85.6%), GLM-5.3 (84.6%), and MiniMax-M3.1 Flash (82.4%) lead the field.

How do open-weights models like DeepSeek V4.1, Qwen 3.8-Max, Kimi K3.1 Pro, GLM-5.3, and MiniMax-M3.1 compare to proprietary LLMs?

By November 2026, the gap between Western proprietary frontier APIs and leading open-weights/hybrid models has narrowed to 2–4 percentage points. Moonshot AI's Kimi K3.1 Pro (3T MoE) scores 94.0% on GPQA Diamond and 86.8% on SWE-bench Verified, Alibaba's Apache 2.0 Qwen 3.8-Max (2.4T MoE) hits 93.7% on GPQA Diamond and 85.6% on SWE-bench, DeepSeek V4 Pro & V4.1 Flash reach 92.8% GPQA and 86.4% SWE-bench, Zhipu's GLM-5.3 reaches 99.3% on AIME, 91.7% on GPQA, and 84.6% on SWE-bench, and MiniMax-M3.1 Flash achieves 90.4% GPQA and 82.4% SWE-bench with a 1M-token sparse-attention context window.

Why are GPQA Diamond, SWE-bench, and MMLU-Pro replacing standard MMLU?

Standard 4-choice MMLU has largely saturated in 2026, with frontier models clustering between 93% and 95% due to pre-training coverage and ceiling effects. Evaluators now rely on harder, contamination-resistant benchmarks: MMLU-Pro (10 answer choices with multi-step reasoning), GPQA Diamond (PhD-level Google-proof science questions), and SWE-bench Verified (resolving real GitHub pull requests in multi-file repositories).

How accurate are these benchmark scores?

Scores come from official provider system cards and verified independent evaluators including Artificial Analysis, LMSYS Chatbot Arena, Vals AI, and BenchLM. However, scores can vary by 2-5% based on test-time compute (thinking token budgets), temperature settings, and whether Python tool execution or custom agent harnesses are enabled.

Why do scores vary between different sources?

Different evaluators may use different prompting strategies (zero-shot vs few-shot), reasoning effort tiers (standard vs high/deep-think), system prompts, or agentic scaffolding harnesses. We prioritize standardized evaluations and note high-reasoning or agent-harness configurations in the tracker.

How often is this benchmark tracker updated?

We update the LLM Benchmark Tracker within 1-2 weeks of major model releases. The dataset was last updated on November 1, 2026 to include Gemini 4 Carbon, Gemini 4 Argon, Muse Wonder (Max), Claude Opus 5.5, GPT-6 Astra, GPT-6.1 Sol, Grok 4.7, Kimi K3.1 Pro, Qwen 3.8-Max, DeepSeek V4 Pro, GLM-5.2, and MiniMax-M3.1 Flash.

What is MMLU and why is it still tracked?

MMLU (Massive Multitask Language Understanding) tests knowledge across 57 subjects from elementary to professional level. While top frontier models now score 93%–95% on standard MMLU (surpassing the ~89.8% human expert baseline), pairing MMLU with the 10-choice MMLU-Pro (where top models score 90%–92.8%) provides a longitudinal baseline across model generations.

Our Methodology

Data Sources

Official model system cards, technical reports, arXiv papers, and verified independent evaluations from Artificial Analysis (v5.0), LMSYS Chatbot Arena, Vals AI, and BenchLM.

Update Frequency

Refreshed within 1-2 weeks of major frontier releases. We verify scores across both standard and extended-thinking configurations before adding them to the matrix.

Verification

Cross-referenced across primary model cards and third-party harnesses. Reasoning effort modes (e.g., Deep Think, Extended Thinking, Max Reasoning) are annotated in source metadata.

Related Reading

Explore these in-depth technical guides to learn more about LLM architectures, evaluation harnesses, and how to get the most from frontier models.

Resources & Official Sources

For primary system cards and live arena standings, consult these official research hubs and evaluation platforms directly.

Disclaimer

Benchmark scores are collected from official model cards and independent evaluation leaderboards (Artificial Analysis, LMSYS Arena, Vals AI) and may vary based on reasoning effort or agent harness. This page is for informational purposes only. AI capabilities evolve rapidly — always verify with official sources for critical decisions.

Written by Rajesh Praharaj

Product Leader & AI Builder

Product leader and AI builder focused on practical LLM systems, Model Context Protocol (MCP) integrations, agentic workflows, and free browser-based tools for product & engineering teams.