Proving Ground · Rank 8 of 10

Muse Glimmer 30B

Meta's dense 30B agent model (NVFP4 by Red Hat) with a Meta DFlash drafter — multimodal, 131K context — on one desktop-sized AI computer.

87.6%

482 of 550 graded answers correct

550 tasks × 3 passes

Setup and speed

Model

Muse Glimmer 30B

Meta's dense 30B agent model (NVFP4 by Red Hat) with a Meta DFlash drafter — multimodal, 131K context — on one desktop-sized AI computer.

Architecture

Dense · ~29.6B (incl. ~1.8B vision)

Every parameter runs on every token.

Speculative decoding

DFlash

What this published run actually used for speculation — not every method the checkpoint supports.

Quantization

NVFP4 W4A4 (Red Hat)

Compressed so it fits the hardware below — a higher or lower bit-width version would score differently

Deployment

1× DGX Spark (dgx-spark-x1)

Served by vLLM fork (MiaAI-Lab Glimmer recipe, Meta DFlash drafter). One desktop-sized AI computer runs the whole model.

Source

Meta · Red Hat · Mia

Model and DFlash drafter by Meta; NVFP4 W4A4 build by Red Hat; Spark serving recipe by Mia (vLLM fork with muse_glimmer parsers, Meta assistant drafter at 16 speculative tokens, fp8 KV, 131K native context at 70% GPU memory).

Weights (Red Hat NVFP4) ↗Base model (Meta) ↗DFlash drafter (Meta) ↗Recipe on GitHub ↗

First token

0.00s

starts responding almost instantly

Writing speed

19.5 w/s

the pace of the prose once it starts

Tokens per second

25.4

the raw generation rate behind that

Visible answer starts

~21.95s

with thinking on, it reasons before it writes

Typical task

37.5s

half finish faster than this

Slow tasks

148.2s

9 in 10 finish inside this

Full results

Score by task difficulty

easy

96%

One clear step — read the request, produce the answer.

573 graded answers

medium

85.2%

Several steps or a judgment call: reconcile records, pick a queue, pull a field.

714 graded answers

hard

80.4%

Multi-step work with a trap: a policy to apply, a dataset to analyze, a budget to stay inside.

363 graded answers

Score by kind of work

Inbox triage and reply

100%

100/100 correcttypical 18.6s

Document extraction

100%

100/100 correcttypical 14.4s

Spreadsheet analysis

55%

55/100 correcttypical 151.2s

Formulas and SQL

78%

78/100 correcttypical 72.2s

Business writing

99%

99/100 correcttypical 49.4s

Classification and routing

100%

50/50 correcttypical 6.8s

Where it missed

—Each task is run three times; the score counts all 1,650 attempts, not 550 tasks.
—Perfect or near-perfect on Inbox triage (100), Document extraction (100), Classification (50/50) and Business writing (99). Softest by far on Spreadsheet analysis at 55 — hard tasks fall to 22% with heavy step-limit burn — and Formulas & SQL at 78.
—450 of 550 tasks were correct on all three passes. Nothing was cut off by the output limit on the core round.
—Files round: all four artifacts were produced. Workbook and social posts passed the machine checks (quality 1.62 and 2.38). Report and deck failed machine checks (quality 1.75 and 2.5) — report length/structure issues; deck wrong-answer.
—Speed probe (thinking on): first visible token median 22.0s, decode about 25 tok/s (~20 words/s). TTFT not separated from first visible content on this stack.

With thinking on, the first token arrives at 0.00s but the visible answer does not start until roughly 21.95s while the model reasons. The 37.5s typical task time is that reasoning plus the answer.

Files it produced

Four real documents from one fact sheet, produced once. Passed is machine-checked against the fact sheet and the brief; quality is a separate 1–5 score from a published rubric and never changes whether a file passed. How files are graded →

Passed 2/4Quality 2.06 / 5

Annual reportquality 1.75

An 8–12 page annual report with six required sections: shareholder letter, financial review, category performance, customer and geographic performance, risk factors, and outlook.

Download DOCX

Supporting workbookquality 1.62

A two-sheet workbook: a Summary comparing FY2025 to FY2024, and a Monthly sheet with all twelve months.

Download XLSX

Investor deckquality 2.50

A ten-slide investor presentation following a fixed outline, with the brand palette applied and at least one real chart object.

Download PPTX

Social announcementsquality 2.38

An X post of 280 characters or fewer and a LinkedIn post, both announcing the results, text only.

Download MD

Every artifact below is the file the model actually produced, published unedited so you can open it yourself. These are AI-generated documents about a fictional company — read them as samples of the model's work, not as advice or a real annual report. The workbook's formula cells hold real formulas — a viewer that does not recalculate may show them blank until Excel opens the file.