← All models
Proving Ground · Rank 6 of 6
Gemma 4 E4B
Google's 4.5B dense model — the cheapest seat in the Gemma 4 family, and the one that shows where capability falls off.
59.8%
329 of 550 graded answers correct
550 tasks × 3 passes
Setup and speed
Model
Gemma 4 E4B
Google's 4.5B dense model — the cheapest seat in the Gemma 4 family, and the one that shows where capability falls off.
Quantization
QAT w4a16 compressed-tensors (4-bit weights, 16-bit activations)
Compressed so it fits the hardware below — a higher or lower bit-width version would score differently
Deployment
1× DGX Spark (dgx-spark-x1)
Served by vLLM 0.29.0. One desktop-sized AI computer runs the whole model.
Source
Google DeepMind
Model by Google DeepMind — the QAT checkpoint, served with vLLM's native Gemma 4 tool-call parser.
First token
0.99s
starts responding almost instantly
Writing speed
44 w/s
the pace of the prose once it starts
Tokens per second
53
the raw generation rate behind that
Visible answer starts
immediately
answers straight away — no separate reasoning phase
Typical task
3.8s
half finish faster than this
Slow tasks
29.3s
9 in 10 finish inside this
Full results
Score by task difficulty
easy
73.6%
One clear step — read the request, produce the answer.
573 graded answers
medium
53.4%
Several steps or a judgment call: reconcile records, pick a queue, pull a field.
714 graded answers
hard
52.9%
Multi-step work with a trap: a policy to apply, a dataset to analyze, a budget to stay inside.
363 graded answers
Score by kind of work
Inbox triage and reply
68%
68/100 correcttypical 0.5s
Document extraction
72%
72/100 correcttypical 7.0s
Spreadsheet analysis
29%
29/100 correcttypical 34.2s
Formulas and SQL
60%
60/100 correcttypical 3.0s
Business writing
82%
82/100 correcttypical 4.0s
Classification and routing
36%
18/50 correcttypical 0.4s
Where it missed
- —Each task is run three times; the score counts all 1,650 attempts, not 550 tasks.
- —The cliff is spreadsheet analysis: 29 of 100, with a difficulty split of 49.2 easy, 25.8 medium and 0 on hard — it does not pass a single hard spreadsheet task.
- —The failure mix there is diagnostic: 144 wrong answers plus 66 formatting failures. The model can call the analysis tool but cannot drive the multi-step loop, and often emits a call the harness cannot parse.
- —It is not useless — 82 on business writing and 72 on document extraction are real. But a company buying the 4B to save money gets a model that cannot do the one job its finance team actually needs.
The first token arrives at 0.99s and the answer starts straight away — this model does not run a separate reasoning phase. The 3.8s typical task time is the answer being written.
Files it produced
This model has not run the file round yet. When it does, its annual report, workbook, investor deck and social posts will be published here, unedited, with their scores.