← All models
Proving Ground · Rank 7 of 8

Laguna S 2.1

poolside's 118B coding-agent model — 8.5B parameters active at a time — compressed to NVFP4 on one desktop-sized AI computer.

84.7%
466 of 550 graded answers correct
550 tasks × 3 passes

Setup and speed

Model
Laguna S 2.1
poolside's 118B coding-agent model — 8.5B parameters active at a time — compressed to NVFP4 on one desktop-sized AI computer.
Quantization
NVFP4 (poolside)
Compressed so it fits the hardware below — a higher or lower bit-width version would score differently
Deployment
1× DGX Spark (dgx-spark-x1)
Served by vLLM 0.25.1 + DFlash-NVFP4 drafter. One desktop-sized AI computer runs the whole model.
Source
poolside · Mia
Model, NVFP4 weights and DFlash drafter by poolside; Spark serving recipe by Mia (vLLM 0.25.1, FlashInfer, the NVFP4 drafter at 7 speculative tokens), run at 91% memory and a 131K context so it fits one Spark.
First token
2.47s
starts responding almost instantly
Writing speed
11.1 w/s
the pace of the prose once it starts
Tokens per second
14.4
the raw generation rate behind that
Visible answer starts
immediately
answers straight away — no separate reasoning phase
Typical task
10.4s
half finish faster than this
Slow tasks
65.9s
9 in 10 finish inside this

Full results

Score by task difficulty
easy
94.6%
One clear step — read the request, produce the answer.
573 graded answers
medium
79.6%
Several steps or a judgment call: reconcile records, pick a queue, pull a field.
714 graded answers
hard
78.8%
Multi-step work with a trap: a policy to apply, a dataset to analyze, a budget to stay inside.
363 graded answers
Score by kind of work
Inbox triage and reply
84%
84/100 correcttypical 2.0s
Document extraction
90%
90/100 correcttypical 10.2s
Spreadsheet analysis
95%
95/100 correcttypical 43.7s
Formulas and SQL
75%
75/100 correcttypical 6.9s
Business writing
80%
80/100 correcttypical 32.1s
Classification and routing
84%
42/50 correcttypical 1.6s
Where it missed
  • —Each task is run three times; the score counts all 1,650 attempts, not 550 tasks.
  • —Strongest on the tool-driven work: Spreadsheet analysis 95 and Document extraction 90. Weakest on judgment — Business writing 80, Classification and routing 42/50 — and on Formulas & SQL at 75, which falls to 55 on hard tasks.
  • —397 of 550 tasks were correct on all three passes. Nothing was cut off by the output limit; seven SQL attempts timed out.
  • —In the files round it produced only the social posts; the report, workbook and deck were never written before the task ended.

The first token arrives at 2.47s and the answer starts straight away — this model does not run a separate reasoning phase. The 10.4s typical task time is the answer being written.

Files it produced

Four real documents from one fact sheet, produced once. Passed is machine-checked against the fact sheet and the brief; quality is a separate 1–5 score from a published rubric and never changes whether a file passed. How files are graded →

Passed 1/4Quality 2.88 / 5
Annual report

An 8–12 page annual report with six required sections: shareholder letter, financial review, category performance, customer and geographic performance, risk factors, and outlook.

Not produced — the model never wrote this file
Supporting workbook

A two-sheet workbook: a Summary comparing FY2025 to FY2024, and a Monthly sheet with all twelve months.

Not produced — the model never wrote this file
Investor deck

A ten-slide investor presentation following a fixed outline, with the brand palette applied and at least one real chart object.

Not produced — the model never wrote this file
Social announcementsquality 2.88

An X post of 280 characters or fewer and a LinkedIn post, both announcing the results, text only.

Download MD

Every artifact below is the file the model actually produced, published unedited so you can open it yourself. These are AI-generated documents about a fictional company — read them as samples of the model's work, not as advice or a real annual report. The workbook's formula cells hold real formulas — a viewer that does not recalculate may show them blank until Excel opens the file.