DeepSeek V4.1 Flash 1017
Open-weight model, compressed to run on two desktop-sized AI computers.
Setup and speed
Full results
- —Each task is run three times; the score counts all 1,650 attempts, not 550 tasks.
- —541 of 550 tasks were correct on all three passes — two spreadsheet tasks flipped between passes.
- —The misses that remain are real: a policy applied backwards, five hard spreadsheet analyses that ran out of steps, and one answer given as prose instead of the exact value asked for.
- —Six answers hit the output limit while still reasoning and came back empty — the model's known long-thinking failure mode, not a grading artifact.
With thinking on, the first token arrives at 0.31s but the visible answer does not start until roughly 24s while the model reasons. The 29.1s typical task time is that reasoning plus the answer.
Files it produced
Four real documents from one fact sheet, produced once. Passed is machine-checked against the fact sheet and the brief; quality is a separate 1–5 score from a published rubric and never changes whether a file passed. How files are graded →
An 8–12 page annual report with six required sections: shareholder letter, financial review, category performance, customer and geographic performance, risk factors, and outlook.
Download DOCXA two-sheet workbook: a Summary comparing FY2025 to FY2024, and a Monthly sheet with all twelve months.
Download XLSXA ten-slide investor presentation following a fixed outline, with the brand palette applied and at least one real chart object.
Download PPTXAn X post of 280 characters or fewer and a LinkedIn post, both announcing the results, text only.
Download MDEvery artifact below is the file the model actually produced, published unedited so you can open it yourself. These are AI-generated documents about a fictional company — read them as samples of the model's work, not as advice or a real annual report. The workbook's formula cells hold real formulas — a viewer that does not recalculate may show them blank until Excel opens the file.