In September I ruled Qwen3.8-Flash-Next out for a single DGX Spark. Six weeks later a signed community recipe claims 69 tok/s. I wrote down six hypotheses and a decision rule before the run. Five held. The sixth, 23 of 28 against the 27B's 25, decided it: the smaller model keeps the job.

Qwen3.8-Flash-Next Fits on One DGX Spark and Runs at 65 tok/s. It Still Did Not Earn the Slot.

My DGX Spark runs two local models in two roles. Role 1, the daily driver, is Qwen3.6-35B-A3B: fast, always on, good at the short tool-heavy turns my coding agents send it. Role 2 is for hard problems and runs on demand: since September that has been Qwen3.8-27B, which won my 28-task A/B against the 35B and then a four-model lineup test.

In daily use the 27B has not been the step up those scores promised. That is an impression, not a measurement, but it is why I looked again at the model I had ruled out in September: Qwen3.8-Flash-Next, the strongest Qwen that fits on one Spark at all.

This post is written in an unusual order. Everything above the results section was written before the model answered its first request: the sources, what I expect, the pass mark for each expectation, and the rule that decides whether Flash-Next replaces the 27B. The results are appended afterwards, and the text above them is not edited to fit.

Why I ruled it out in September

The September post cites a single-node comparison by Classmethod. On one Spark they measured Flash-Next at 28 tok/s with 79 GB resident, a 14-minute start and nine community vLLM patches. One finding mattered more than the speed: with tool_choice=required the model returned zero tool calls. My agents live on tool calls, so that settled it.

What changed in six weeks

A community recipe by salient-data, announced in a forum thread about six weeks of optimizing, serves NVIDIA’s Qwen3.8-Flash-Next-NVFP4 checkpoint on one Spark. What it publishes for its build of 2026-10-22, each row with its harness and raw data:

Classmethod (September)salient-data recipe (build 2026-10-22)
Single-stream decode28 tok/s69.3 tok/s median over 40 prompts, prose 41.0
Six parallel streamsnot reported91.9 tok/s aggregate (one run)
Memory79 GB resident73.57 GiB weights after load, KV cache of 1.0 to 1.09 M tokens
Start to first answer14 minutes350 to 371 s on three boots
vLLM patches914, each with source and full diff
Quality checknot reportedGSM8K 99/100 on a fixed 100-problem subset

The speed numbers come from different harnesses (theirs is bench_categories.py from another community member), so I do not compare them with my own rulers directly. That is what the test below is for.

The recipe is also built to be checked. It pins the vLLM nightly by digest and the checkpoint to one exact commit, and ships the serving image signed with a cosign key that is in the repository. Its README says plainly that GB10 is outside the model card’s support statement, which lists B200 and B300 only.

What the public benchmarks say

On Artificial Analysis, Intelligence Index v5.0.1:

Flash-NextQwen3.8-27BQwen3.6-35B-A3B
Intelligence Index v5.0.12920 (xhigh), 16 (low)
Terminal-Bench 4.025.3 %2.5 % (low)0.0 %

Sources: Flash-Next against the 27B, Flash-Next against the 35B.

Terminal-Bench is the row I care about most. It measures multi-step work in a shell, which is close to what my coding agents do, and there the gap is a factor of ten.

A correction on my own September post: it quotes the 27B at 34 and the 35B at 18 on the same index. Those numbers were from an older index version. Artificial Analysis restates the index from time to time, and numbers from two versions cannot be compared. Every row in the table above is from v5.0.1.

The license

The checkpoint is under the NVIDIA Open Model License plus the base model’s Qwen license. For a model that serves my own agents there is no restriction. The Qwen terms do ask for a separate license for offering the model as a service or building a hosted coding or office assistant on it, and above 100 million monthly users or 20 million US dollars of monthly revenue. If I ever expose it publicly, I read that again first.

Why a model larger than the machine fits

The checkpoint is 123.6 GiB on disk and the Spark has 121 GiB of memory, shared by CPU and GPU. No offload flag helps on that hardware, because “host” and “GPU” memory are the same pool. The recipe fits because its largest piece, a 51.2 GB n-gram embedding table, is never loaded. A patch reads only the rows each decode step needs, straight from the files on disk, through the page cache. What stays resident is about 74 GiB of weights plus the KV cache.

That still needs about 100 GiB free when the server starts. My production model alone takes about 46 GB, and the rest of the box (a web UI, a search engine, a relay, a CI runner, a few small apps) about 11 GiB. So the two cannot run side by side, and the test needs the box nearly empty.

Making room: a light mode

I wrote a small switch that empties the box and puts everything back. It stops the production model and every service that is not needed for an hour, pauses the timers that would start heavy work, writes down exactly what it stopped, and drops the page cache. Switching back starts exactly that list again and then the production model. Git, alerting, the local registry and the agent fleet’s coordinator keep running.

I also considered running headless, with the desktop session stopped, which would free another few GiB of browser memory. I left it out of the script: stopping the desktop also kills every terminal in it, including the one the agent driving this test runs in. The recipe needs about 100 GiB free; with production and the services stopped there is enough without it. If the load fails for lack of memory, headless from a phone is the fallback.

Getting it onto the box

The download took most of a day, and a third of that was my own tooling. It started in the morning at about 9 MB/s. The last file, the 51 GB table, is too large for plain HTTPS and only comes through Hugging Face’s XET transfer, which my download script had switched off because it used to be flaky. The script retried the same error 180 times over an hour and a half before I noticed. It now switches XET on by itself for such a file and stops after three instant failures. The final run finished in the afternoon: 124 GiB on disk, no partial files, and all eight small files matching the recipe’s hashes.

The hypotheses, written before the run

Every check runs unattended in one script. It refuses to start while any agent task is running, switches to light mode, starts Flash-Next, runs the checks below in order, and on every exit, including a crash, stops Flash-Next and brings production back.

ExpectationSourcePass mark
H1It loads and fits on this box, with the light moderecipe: 350 to 371 s, KV 1.0 to 1.09 M tokensanswers within 10 minutes, KV cache at least 1.0 M tokens
H2The answers are correct, not just fluentrecipe: a loader that skips the n-gram table’s scale writes fluent wrong textthe recipe’s two known-answer checks pass
H3It is clearly faster than the 27B in role 2recipe: prose 41.0 tok/s; my 27B: prose 20 to 21prose at least 33 tok/s, code at least 55, on my usage-counted ruler
H4Forced tool calls workClassmethod: zero calls with tool_choice=required6 of 6 correct single calls, three with thinking off, three on
H5It solves my 28-task set at least as well as the 27Bthe 27B scored 25 of 28 on this boxat least 25 of 28
H6The box comes back on its ownmy light mode switchproduction answers again after the run, every stopped service is back, no manual step

Notes on the pass marks:

The decision rule, also written before the run

Flash-Next replaces the 27B in role 2 if H1, H2, H3, H4 and H5 all pass. The reason it may win on “no regression” instead of “measurably better”: its lead is attested by public benchmarks that are far larger than my 28 tasks and that I cannot reproduce on one box. My test has to show that the lead survives on this hardware, with this recipe, without breaking anything my agents depend on.

If any of them fails, the 27B stays, the post says which one and why, and the Flash-Next weights are deleted.

H6 does not decide the model, but if it fails, the light mode switch gets fixed before anything else runs on it.

If Flash-Next passes, a second, harder round may follow, because the 28-task set cannot show a lead. That round would get its own hypotheses, written before it runs, and would not change this decision after the fact.

Caveats, in advance

Results

One unattended run on 2026-11-13. Every number is copied from the run’s own files.

Pass markMeasuredVerdict
H1answers within 10 minutes, KV cache at least 1.0 M tokensup after 382 s; 73.57 GiB of weights loaded in 144.9 s; KV cache 1,054,274 tokenspass
H2the recipe’s two known-answer checks17 x 23 = 391 and Canberra, both rightpass
H3prose at least 33, code at least 55 tok/sthinking off: prose 39.0, code 65.2. Thinking on: prose 38.5, code 39.5split
H46 of 6 forced tool calls6 of 6, three with thinking off and three with it onpass
H5at least 25 of 2823 of 28 (math 12 of 14, code 7 of 10, tools 4 of 4)fail
H6the box comes back without a manual stepevery stopped service and timer restarted, production answered againpass

The decision, by the rule above: H5 failed, so Qwen3.8-27B keeps role 2.

Two notes on the verdicts. My H3 pass mark did not say which thinking mode it meant. With thinking off the model clears both marks. With thinking on, which is how a hard-problem model is used, code drops to 39.5 tok/s, under the mark of 55 and still above the 27B’s 22 to 30. The speculative decoder accepts 4.4 tokens per step on plain code and 2.3 once the model reasons first. I call that a split, not a pass. It does not change the decision.

With the model loaded, the box had 13 GiB of memory left, and at the lowest point of the load 11.5 GiB. The light mode was necessary, not a precaution.

Where H5 failed

Flash-Next missed five tasks: two in math and three in code. Four of the five were not wrong answers. The model spent its whole budget, 16,000 tokens on math and 12,000 on code, inside its reasoning and never wrote the answer or the code block. One was a real mistake, a wrong count on the coins problem after 5,057 tokens.

The 27B missed three tasks under the same settings, two of them the same way. So both models overthink on this set. Flash-Next does it more often.

The same thing happened to my production 35B in September, and the cause is in the checkpoints. Both Qwen3.8 chat templates set the reasoning effort to xhigh whenever thinking is on and the request names no effort. I checked the 27B’s template after the run: it has the same default, so the comparison was fair. It also means both rows were measured at the most expensive setting, and neither at the one I would pick for daily use.

Two more runs, after the decision

These two runs happened after the rule had already decided. They are exploration and do not reopen it.

A lead worth following, not followed here

NVIDIA publishes a second checkpoint of this model, validated on the DGX Spark: Qwen3.8-Flash-Next-FP4-Local. Its card says it was made with quantization-aware distillation and “separate losses for end-of-response and end-of-thinking behavior”. The checkpoint I tested was quantized after training, for data-center GPUs. A model that cannot stop thinking, and a sibling trained specifically to stop, is a plausible pair. I have not tested the sibling. It needs a different serving stack that is still moving fast, and that is a separate experiment with its own hypotheses.

Caveats

What I am keeping

The 27B stays the hard-problem model. Beyond the score, the two differ in what they cost to run. The 27B needs production paused. Flash-Next needs the whole machine: every other service stopped, six minutes to load, 13 GiB left. For an on-demand role that is a lot to pay for no measured gain.

The recipe did everything it promised. It fit a 124 GiB checkpoint into a 121 GiB machine, loaded in the time it said, answered correctly, and made tool calls that a September build could not. If you want Flash-Next on one Spark, it is the way to do it.

The rule says the weights go. I kept them one more day for a different question, whether a bigger local model writes better podcast dialogue than my daily driver. That test is in the script-writer post.

Three things outlast the scoreboard: a light mode that empties the box and puts it back, a download script that no longer retries a hopeless error 180 times, and a forced-tool-call check that my test set was missing.

Was this worth it? Zap the article.

Value for value, no signup. Sats go straight to the writer.

… sats zapped
… zaps
Today 7d 30d All-time
Unique readers — — — —
Page views — — — —
—
All Article Insights →