Nemotron 3.5 Bolt 74B-A9B
NVIDIA's 74B mixture-of-experts, packed to 24GB so it fits on one desktop-sized AI computer.
Setup and speed
Full results
- —Each task is run three times; the score counts all 1,650 attempts, not 550 tasks.
- —503 of 550 tasks were correct on all three passes. The losses concentrate in two places: business writing and Formulas & SQL.
- —71 business-writing replies were real newlines inside a JSON string instead of escaped ones, so the harness could not parse them. 62 of those parse cleanly once escaped and carry every required field — the writing is substantively right and the formatting is wrong.
- —Formulas & SQL is the genuine capability limit: 13 of 15 failing tasks miss on all three passes, inventing schema that does not exist and writing range formulas that do not evaluate.
- —Three attempts on one inbox task burned the full output budget while reasoning and returned nothing — reported as truncated, not as a wrong answer.
- —Finale: three of four artifacts pass. The deck initially produced nothing at all with reasoning on — the model spent the entire output budget re-reading files it already had. Re-run with reasoning off, it built a complete ten-slide deck; that setting is recorded on this row.
The first token arrives at 0.06s and the answer starts straight away — this model does not run a separate reasoning phase. The 27.3s typical task time is the answer being written.
Files it produced
Four real documents from one fact sheet, produced once. Passed is machine-checked against the fact sheet and the brief; quality is a separate 1–5 score from a published rubric and never changes whether a file passed. How files are graded →
An 8–12 page annual report with six required sections: shareholder letter, financial review, category performance, customer and geographic performance, risk factors, and outlook.
Download DOCXA two-sheet workbook: a Summary comparing FY2025 to FY2024, and a Monthly sheet with all twelve months.
Download XLSXA ten-slide investor presentation following a fixed outline, with the brand palette applied and at least one real chart object.
Download PPTXAn X post of 280 characters or fewer and a LinkedIn post, both announcing the results, text only.
Download MDEvery artifact below is the file the model actually produced, published unedited so you can open it yourself. These are AI-generated documents about a fictional company — read them as samples of the model's work, not as advice or a real annual report. The workbook's formula cells hold real formulas — a viewer that does not recalculate may show them blank until Excel opens the file.