← All models
Proving Ground · Rank 8 of 11
LFM3 1.7B
A 1.7B model from Liquid AI — small enough to run on a laptop, quick enough to answer in seconds.
88.2%
485 of 550 graded answers correct
550 tasks × 3 passes
Setup and speed
Model
LFM3 1.7B
A 1.7B model from Liquid AI — small enough to run on a laptop, quick enough to answer in seconds.
Architecture
Dense · 1.7B
Every parameter runs on every token.
Speculative decoding
None in this run
This published run did not use a drafter or n-gram speculation.
Quantization
GGUF BF16
Compressed so it fits the hardware below — a higher or lower bit-width version would score differently
Deployment
1× Mac Studio (mac-studio-36gb)
Served by llama.cpp (llama-server). One desktop-sized AI computer runs the whole model.
Source
Liquid AI
Open weights and the official BF16 GGUF build by Liquid AI, served with llama.cpp on macOS. No fine-tune, no drafter, no quantization beyond BF16.
First token
0.19s
starts responding almost instantly
Writing speed
30 w/s
the pace of the prose once it starts
Tokens per second
84
the raw generation rate behind that
Visible answer starts
~5.64s
with thinking on, it reasons before it writes
Typical task
48.3s
half finish faster than this
Slow tasks
140.1s
9 in 10 finish inside this
Full results
Score by task difficulty
easy
91.4%
One clear step — read the request, produce the answer.
573 graded answers
medium
90.2%
Several steps or a judgment call: reconcile records, pick a queue, pull a field.
714 graded answers
hard
80.4%
Multi-step work with a trap: a policy to apply, a dataset to analyze, a budget to stay inside.
363 graded answers
Score by kind of work
Inbox triage and reply
91%
91/100 correcttypical 24.8s
Document extraction
96%
96/100 correcttypical 46.8s
Spreadsheet analysis
75%
75/100 correcttypical 47.2s
Formulas and SQL
78%
78/100 correcttypical 102.5s
Business writing
95%
95/100 correcttypical 78.6s
Classification and routing
100%
50/50 correcttypical 11.3s
Where it missed
- —Each task is run three times; the score counts all 1,650 attempts, not 550 tasks.
- —440 of 550 tasks were correct on all three passes. The misses concentrate in spreadsheet analysis: 73 of 100 ran out of tool steps at the 12-step cap, and hard spreadsheets scored 27% against 92% on the easy ones.
- —Formulas & SQL was the other soft spot at 78 — 26 formulas would not execute, 20 returned the wrong result, 8 queries errored, and 3 could not be graded because the formula checker itself crashed on them.
- —Three answers hit the output limit while still reasoning and came back empty.
- —Formulas and business writing were re-run with a 32,768-token output budget after the 8,192-token cap cut reasoning short; the other four workloads ran at 8,192.
- —There is no tool-calling loop on this server path, so the file-writing round was not run.
- —Speed: 0.19s to the first token, about 84 tokens a second while decoding, and 5.6s before visible prose on a routing task. The standard 1,500-word memo probe never left its reasoning phase inside 2,000 tokens.
With thinking on, the first token arrives at 0.19s but the visible answer does not start until roughly 5.64s while the model reasons. The 48.3s typical task time is that reasoning plus the answer.
Files it produced
This model has not run the file round yet. When it does, its annual report, workbook, investor deck and social posts will be published here, unedited, with their scores.