Business AI, graded on real work.
550 business tasks across 6 kinds of work, each with a correct answer we check mechanically. Every model runs the same tasks three times. Ranked on correctness — speed is published, but it never moves a model up.
Leaderboard
11 models · click one for its full results and files- 1Qwen 3.8 27Bby MiaMLX 4-bit affine (group 64)1× DGX SparkDense · 27BDFlash223.2s99.1%545/550DOCXLSPPTPOST→1Qwen 3.8 27BDense · 27B · DFlash2MLX 4-bit affine (group 64) · 1× DGX Spark99.1%545/550typical task 23.2sDOCXLSPPTPOST
- 2DeepSeek V4.1 Flash 1017by Mia2.9bpw EXL3-EAQ2× DGX SparkMoE/n-gram · 552B backbone total / 8B prefill / 16B decode activeno spec-decode29.1s98.5%542/550DOCXLSPPTPOST→2DeepSeek V4.1 Flash 1017MoE/n-gram · 552B backbone total / 8B prefill / 16B decode active · no spec-decode2.9bpw EXL3-EAQ · 2× DGX Spark98.5%542/550typical task 29.1sDOCXLSPPTPOST
- 3Gemma 4 31Bby Google DeepMindQAT w4a16 compressed-tensors (4-bit weights, 16-bit activations)1× DGX SparkDense · 30.7Bno spec-decode18.9s94%517/550DOCXLSPPTPOST→3Gemma 4 31BDense · 30.7B · no spec-decodeQAT w4a16 compressed-tensors (4-bit weights, 16-bit activations) · 1× DGX Spark94%517/550typical task 18.9sDOCXLSPPTPOST
- 4Flintby RecursiveNVFP4 GGUF (Recursive)1× DGX SparkMoE · 39.8B total / ~2.5B activeno spec-decode38.6s91.8%505/550DOCXLSPPTPOST→4FlintMoE · 39.8B total / ~2.5B active · no spec-decodeNVFP4 GGUF (Recursive) · 1× DGX Spark91.8%505/550typical task 38.6sDOCXLSPPTPOST
- 5Nemotron 3.5 Bolt 74B-A9Bby MiaIQ2_XXS experts + NVFP4 Mamba + Q8_0 attention (24GB pack)1× DGX SparkMoE · 74B total / 9B activeLiLiCorr27.3s91.5%503/550DOCXLSPPTPOST→5Nemotron 3.5 Bolt 74B-A9BMoE · 74B total / 9B active · LiLiCorrIQ2_XXS experts + NVFP4 Mamba + Q8_0 attention (24GB pack) · 1× DGX Spark91.5%503/550typical task 27.3sDOCXLSPPTPOST
- 6Gemma 4 26B-A4Bby Google DeepMindQAT q4_0 (4-bit), 3.8B active of 25.2B total1× DGX SparkMoE · 25.2B total / 3.8B activeno spec-decode16.1s90.4%497/550DOCXLSPPTPOST→6Gemma 4 26B-A4BMoE · 25.2B total / 3.8B active · no spec-decodeQAT q4_0 (4-bit), 3.8B active of 25.2B total · 1× DGX Spark90.4%497/550typical task 16.1sDOCXLSPPTPOST
- 7Laguna XS 2.1by poolside · MiaNVFP4 (poolside)1× DGX SparkMoE · 33B total / 3B activeDFlash15.8s90.2%496/550DOCXLSPOST→7Laguna XS 2.1MoE · 33B total / 3B active · DFlashNVFP4 (poolside) · 1× DGX Spark90.2%496/550typical task 15.8sDOCXLSPOST
- 8LFM3 1.7Bby Liquid AIGGUF BF161× Mac StudioDense · 1.7Bno spec-decode48.3s—→88.2%485/5508LFM3 1.7BDense · 1.7B · no spec-decodeGGUF BF16 · 1× Mac Studio88.2%485/550typical task 48.3s—
- 9Muse Glimmer 30Bby Meta · Red Hat · MiaNVFP4 W4A4 (Red Hat)1× DGX SparkDense · ~29.6B (incl. ~1.8B vision)DFlash37.5s87.6%482/550DOCXLSPPTPOST→9Muse Glimmer 30BDense · ~29.6B (incl. ~1.8B vision) · DFlashNVFP4 W4A4 (Red Hat) · 1× DGX Spark87.6%482/550typical task 37.5sDOCXLSPPTPOST
- 10Laguna S 2.1by poolside · MiaNVFP4 (poolside)1× DGX SparkMoE · 118B total / 8B activeDFlash10.4s84.7%466/550POST→10Laguna S 2.1MoE · 118B total / 8B active · DFlashNVFP4 (poolside) · 1× DGX Spark84.7%466/550typical task 10.4sPOST
- 11Gemma 4 E4Bby Google DeepMindQAT w4a16 compressed-tensors (4-bit weights, 16-bit activations)1× DGX SparkDense · 4.5B effective (8B with embeddings)no spec-decode3.8s59.8%329/550XLSPOST→11Gemma 4 E4BDense · 4.5B effective (8B with embeddings) · no spec-decodeQAT w4a16 compressed-tensors (4-bit weights, 16-bit activations) · 1× DGX Spark59.8%329/550typical task 3.8sXLSPOST
Score = graded answers correct out of 550 tasks × 3 passes. Typical = half of tasks finish faster than this. Files = the report, workbook, deck and social posts the model produced, downloadable on its page. Full method →
The Proving Ground suite — code, tasks, keys, and grading — is proprietary Onyx AI Labs intellectual property. All rights reserved. Published per-run result files are shared for transparency; the bench itself is not open source.
Shoutouts
People really moving the ball forward on local AI. Our leaderboard sits on their work.
Several rows on this board run Mia's model builds and DGX Spark serving recipes — the recipe links on those model pages go to her repos. The visual language of onyxailabs.com also grew out of one of her public 100 HTML Files experiments.
Her own benchmarks measure performance (speed and throughput) plus traditional academic sets. Ours are business benchmarks — real desk work with mechanical keys. They answer different questions, so the rankings often disagree. Complementary, not competing.
TensorFold is the serving engine behind several of our DGX Spark rows. Fast local inference on hardware you control — the stack a lot of this board actually runs on. His work on Mac and DGX Spark interoperability is changing the scale at which local AI can be used.
Want your model on this page?
We run any open-weight or hosted model against the same tasks and publish the result with its configuration — good, bad, or ugly.
Talk to us