← All models
Proving Ground · Rank 5 of 6
Gemma 4 26B-A4B
A mixture-of-experts model that runs only 3.8B parameters per token — dense-31B-class results for a fraction of the compute.
90.4%
497 of 550 graded answers correct
550 tasks × 3 passes
Setup and speed
Model
Gemma 4 26B-A4B
A mixture-of-experts model that runs only 3.8B parameters per token — dense-31B-class results for a fraction of the compute.
Quantization
QAT q4_0 (4-bit), 3.8B active of 25.2B total
Compressed so it fits the hardware below — a higher or lower bit-width version would score differently
Deployment
1× DGX Spark (dgx-spark-x1)
Served by vLLM 0.29.0. One desktop-sized AI computer runs the whole model.
Source
Google DeepMind
Model by Google DeepMind — a 25.2B mixture-of-experts with 3.8B active parameters, served with vLLM's native Gemma 4 tool-call parser.
First token
1.84s
starts responding almost instantly
Writing speed
20 w/s
the pace of the prose once it starts
Tokens per second
24
the raw generation rate behind that
Visible answer starts
immediately
answers straight away — no separate reasoning phase
Typical task
16.1s
half finish faster than this
Slow tasks
85.5s
9 in 10 finish inside this
Full results
Score by task difficulty
easy
96.5%
One clear step — read the request, produce the answer.
573 graded answers
medium
91.2%
Several steps or a judgment call: reconcile records, pick a queue, pull a field.
714 graded answers
hard
79.9%
Multi-step work with a trap: a policy to apply, a dataset to analyze, a budget to stay inside.
363 graded answers
Score by kind of work
Inbox triage and reply
85%
85/100 correcttypical 2.4s
Document extraction
90%
90/100 correcttypical 28.2s
Spreadsheet analysis
98%
98/100 correcttypical 94.1s
Formulas and SQL
87%
87/100 correcttypical 10.3s
Business writing
90%
90/100 correcttypical 18.2s
Classification and routing
94%
47/50 correcttypical 1.9s
Where it missed
- —Each task is run three times; the score counts all 1,650 attempts, not 550 tasks.
- —This is the finding the size curve exists to show: 497 of 550 while activating 3.8B parameters per token, against 517 for the 30.7B dense model on the same box and tasks — a 20-point gap for roughly an eighth of the compute.
- —It matches the dense 31B exactly on spreadsheet analysis (98 vs 98) and edges it on Formulas & SQL (87 vs 86). It trails on inbox triage (85 vs 95) and business writing (90 vs 97) — the two workloads that reward judgment and tone.
- —Six spreadsheet attempts hit the 12-step tool limit while still working. Those are completion failures, not wrong answers: the model was cut off mid-analysis.
The first token arrives at 1.84s and the answer starts straight away — this model does not run a separate reasoning phase. The 16.1s typical task time is the answer being written.
Files it produced
This model has not run the file round yet. When it does, its annual report, workbook, investor deck and social posts will be published here, unedited, with their scores.