← All articles

#qwen

12 articles

All articles tagged "qwen" : self-hosted AI fixes, setups, and architecture notes.

The ArticleZapStats Redesign: Real Readers, Git History, and the Cost of Asking a Local LLM to Care

The ArticleZapStats Redesign: Real Readers, Git History, and the Cost of Asking a Local LLM to Care

I asked Qwen3.6 to redesign a stats component on sovgrid.org. It did, with real unique readers from NSM, git-based edit history, collapsable layout, and a dozen bug fixes along the way. The cost was eighteen commits, eight debugging iterations, and a growing sense that I was stress-testing my local LLM at the edge of its useful range for a feature whose value to me as operator was genuinely questionable.

Read article →
Qwen3.6-35B built a working SvelteKit 5 + D3.js app from a single prompt. The build passes. The features ship. But the architecture is improvised, the code is duplicated, and it took eight debugging iterations to get here. Here's what the gap between 'works' and 'good' actually costs.
strategyopencodebenchmarking

I Let Qwen3.6 Build a Full-Stack App. It Worked. I Wasn't Satisfied.

Qwen3.6-35B built a working SvelteKit 5 + D3.js app from a single prompt. The build passes. The features ship. But the architecture is improvised, the code is duplicated, and it took eight debugging iterations to get here. Here's what the gap between 'works' and 'good' actually costs.
Standing up two large models on a DGX Spark, my own measurements tried to deceive me three separate ways: a harness that scored a working model at zero, a one-shot test that framed the model for a bug that was mine, and a cold reading that undersold decode speed by 35 percent. None of the wrong numbers were random. Each had a cause, a tell, and a fix. Here is the field guide.
engineering-honestybenchmarkingdgx-spark

A Benchmark Handed Me a Number Three Times in One Day. Three Times It Was Lying.

Standing up two large models on a DGX Spark, my own measurements tried to deceive me three separate ways: a harness that scored a working model at zero, a one-shot test that framed the model for a bug that was mine, and a cold reading that undersold decode speed by 35 percent. None of the wrong numbers were random. Each had a cause, a tell, and a fix. Here is the field guide.
Same model, same box, three ways to shrink it: Intel's AutoRound int4, a 4.75-bit PrismaQuant, and FP8. I measured all three on decode speed, coding accuracy, and vision, with one ruler per axis and the failed runs thrown out. AutoRound won every column that mattered, and the surprise was vision: the leanest build kept its eyes while the others went blind or broke. Here is the teardown.
dgx-sparkcomparisonbenchmarkingvllm

Three Quants of One 35B Qwen on a DGX Spark. The Fastest Build Was the Only One That Could Still See.

Same model, same box, three ways to shrink it: Intel's AutoRound int4, a 4.75-bit PrismaQuant, and FP8. I measured all three on decode speed, coding accuracy, and vision, with one ruler per axis and the failed runs thrown out. AutoRound won every column that mattered, and the surprise was vision: the leanest build kept its eyes while the others went blind or broke. Here is the teardown.
I run Qwen3.6-35B at 4.75-bit for coding. A 4.0-bit AutoRound build promised more speed. Fewer bits usually means a dumber model, so I measured both halves: decode throughput and coding quality, the latter through my own agent-bench harness. The result settled it. Here is the duel, the bandwidth math, and why the bit count was the wrong thing to fear.
strategydgx-sparkbenchmarking

Smaller, Faster, Still Smart? AutoRound int4 vs PrismaQuant for a Self-Hosted Coding Model

I run Qwen3.6-35B at 4.75-bit for coding. A 4.0-bit AutoRound build promised more speed. Fewer bits usually means a dumber model, so I measured both halves: decode throughput and coding quality, the latter through my own agent-bench harness. The result settled it. Here is the duel, the bandwidth math, and why the bit count was the wrong thing to fear.
I took the published Spark-Arena recipes for Qwen3.6-35B on a DGX Spark, ran them on my own box, and almost none of the headline throughput reproduced. Here is what the numbers actually mean once you control for the container image, the measurement harness, and the prompt, plus a capability that was hiding inside one of the quants.
devopsdgx-sparkengineering-honesty

The Leaderboard Said 239 Tokens a Second. My DGX Spark Said 71.

I took the published Spark-Arena recipes for Qwen3.6-35B on a DGX Spark, ran them on my own box, and almost none of the headline throughput reproduced. Here is what the numbers actually mean once you control for the container image, the measurement harness, and the prompt, plus a capability that was hiding inside one of the quants.
Giving a local 8B model persistent memory and retrieval good enough to replace a cloud assistant for daily coding. The architecture is mem0 plus a RAG knowledge base over ChromaDB. The honest part is the two bugs that made the first version forget you and answer the wrong question with full confidence.
mcpollamaself-hostedsovereign-aiengineering-honestyagentsrag

A Second Brain for a Local Model, and the Two Bugs That Made It Useless First

Giving a local 8B model persistent memory and retrieval good enough to replace a cloud assistant for daily coding. The architecture is mem0 plus a RAG knowledge base over ChromaDB. The honest part is the two bugs that made the first version forget you and answer the wrong question with full confidence.
Three production-class open-weights models, all weighed against one Spark. Qwen wins on coding throughput and now sustains around 71 tok/s under DFlash. Mistral holds the creative-prose and verified-vision slot as a safer fallback. GLM-5.1 at 754B does not fit and the reason it does not fit is the most useful lesson in this comparison.
comparisonmistraldgx-spark

Mistral Small 4 vs Qwen 3.6 vs GLM-5.1 on a Single DGX Spark

Three production-class open-weights models, all weighed against one Spark. Qwen wins on coding throughput and now sustains around 71 tok/s under DFlash. Mistral holds the creative-prose and verified-vision slot as a safer fallback. GLM-5.1 at 754B does not fit and the reason it does not fit is the most useful lesson in this comparison.
A vLLM-Qwen container ran four days clean, then froze the whole GNOME desktop the moment any GPU app opened. SGLang-Mistral never did this in days of uptime. The cause: vLLM's FlashInfer MoE throughput backend has broken SM120 kernels on the DGX Spark's SM 12.1 GPU, and on unified memory a bad kernel launch takes the display down with it. One env var fixes it.
fixdgx-sparksglangdevops

Why SGLang Never Froze My Desktop But vLLM Did: an SM 12.1 MoE-Kernel Story

A vLLM-Qwen container ran four days clean, then froze the whole GNOME desktop the moment any GPU app opened. SGLang-Mistral never did this in days of uptime. The cause: vLLM's FlashInfer MoE throughput backend has broken SM120 kernels on the DGX Spark's SM 12.1 GPU, and on unified memory a bad kernel launch takes the display down with it. One env var fixes it.
I almost published 'Mistral Small 4 scores 0/30 on coding, the quant kills it'. A competent model scoring exactly zero should have been the red flag. The benchmark harness was hanging behind this stack's Tor docker proxy and never reached the model. Here is the broken-ruler story, the direct measurement that replaced it, and every Mistral-vs-Qwen3.6 number at a glance, including which one can actually read an image.
strategymistraldgx-sparkdevops

Mistral vs Qwen3.6 on DGX Spark: the 0/30 That Was a Broken Ruler

I almost published 'Mistral Small 4 scores 0/30 on coding, the quant kills it'. A competent model scoring exactly zero should have been the red flag. The benchmark harness was hanging behind this stack's Tor docker proxy and never reached the model. Here is the broken-ruler story, the direct measurement that replaced it, and every Mistral-vs-Qwen3.6 number at a glance, including which one can actually read an image.
This blog gates every article behind one Python scorer before it publishes. I gave Qwen3.6 and Mistral Small 4 the same brief, the Start Here hub article this site still owes, and ran the raw output through that real gate with no editing. Both passed. Both invented hardware, processes, and benchmarks the scorer counted as quality. Here is the full method, the two source texts, and why a passing score is a floor and not a truth filter.
strategymistraldgx-sparkdevops

The Quality Gate That Rewards Fabrication: I Had Qwen and Mistral Write This Blog

This blog gates every article behind one Python scorer before it publishes. I gave Qwen3.6 and Mistral Small 4 the same brief, the Start Here hub article this site still owes, and ran the raw output through that real gate with no editing. Both passed. Both invented hardware, processes, and benchmarks the scorer counted as quality. Here is the full method, the two source texts, and why a passing score is a floor and not a truth filter.
Replace cloud AI coding assistants with opencode, a provider-agnostic Node CLI plus Electron desktop app. Points at any OpenAI-compatible endpoint, ships in three frontends. Includes the 2026-05-13 correction on the auto-title-generator Mistral BadRequest gotcha and the JSON-config-only setup syntax.
setupopencodeagentsmistral

opencode Setup: Self-Hosted AI Coding Assistant on ARM64

Replace cloud AI coding assistants with opencode, a provider-agnostic Node CLI plus Electron desktop app. Points at any OpenAI-compatible endpoint, ships in three frontends. Includes the 2026-05-13 correction on the auto-title-generator Mistral BadRequest gotcha and the JSON-config-only setup syntax.