My Spark Idles at 22% and That's the Point

NVIDIA publishes the argument against my own setup, and it is a good argument. In a piece on inference economics, the company states the obvious truth of its own business: utilization is the single most important variable, and every idle GPU hour is a direct cost with zero revenue against it. The industry runs its expensive accelerators at an average occupancy that hovers somewhere around 22%, and the whole discipline of cluster economics is the art of pushing that number up. By this logic, a GPU sitting idle is the worst thing in the world, a meter running with nothing on the other side.

My desk-side machine sits idle most of the day. By NVIDIA’s own metric, it is a stranded asset, a small monument to economic illiteracy. I want to argue that the metric is correct and the conclusion is wrong, and that the gap between them is one of the more interesting things self-hosting taught me.

The math is real, and it loses

Let me give the counter-argument its full strength before I touch it, because a sovereignty essay that strawmans the economics is worthless.

If you score a private AI machine the way you score a datacenter GPU, it loses, badly. The comparative-advantage case is airtight: rent compute from someone whose entire business is keeping it saturated, and spend your own scarce hours on the thing only you can do. I have modelled the total cost of my own stack and the arithmetic break-even against a frontier API does not arrive until somewhere around 800 to 1,000 calls a day. Below that line the cloud is simply cheaper, and most individuals live far below that line. Worse, inference prices keep falling; the research nonprofit Epoch AI has tracked the cost of a given level of model capability dropping by roughly an order of magnitude a year (their inference-price data). You are buying a depreciating asset to do a job that gets cheaper to rent every quarter. And the hardware itself, an asset that cost roughly €4,800, is built to hold a large model in its 128 GB of unified memory, not to serve it fast to a crowd; its 273 GB/s memory bus means that once 4 people share it, the throughput per user sinks toward the mid-teens of tokens a second. As a utility, the thing is a bad buy. I am not going to pretend otherwise, because the pretending is exactly what this site exists to refuse.

So I concede the entire economic frame. And then I notice that the frame is doing something it never announced: it decided, before any number was computed, that the machine is a utility.

Borgmann’s distinction

Albert Borgmann, in Technology and the Character of Contemporary Life, drew a line that names what the utilization argument quietly assumes. He separated the device from the focal thing. A device delivers a commodity as frictionlessly as possible while hiding its own machinery: central heating delivers warmth with no wood to split, no fire to tend, just a number on a thermostat and a furnace you never see. A focal thing delivers the same good but keeps its machinery present and engaging: a wood stove gives you heat and also the splitting, the stacking, the tending, the gathering of people around it. The device is measured purely by output per cost. The focal thing is measured by the practice it sustains.

A rented API is the device paradigm in its purest digital form. Intelligence arrives as a commodity through a billing endpoint, and the entire apparatus, the weights, the GPUs, the power, the cooling, the choices, is hidden behind a wire on purpose, because hiding it is the product. You are meant to think about the output and never the machinery. That is precisely what makes it efficient, and precisely what makes it a device.

A model on my own desk refuses to hide. The machinery is the experience: the quantization choice, the memory ceiling, the thermals, the config that froze the desktop until I found the flag, the throughput I can watch and tune. Borgmann’s claim is that this presence is not overhead to be minimized. It is the focal practice, the thing that turns a commodity back into a relationship with how the commodity is made. The utilization metric cannot see this, because the metric was built to optimize devices, and it scores a focal thing as a broken device every time.

Idle is the wrong word

Once you see the machine as a focal thing rather than a utility, the word idle changes meaning.

A utility at 22% occupancy is wasting 78% of itself. A possession at 22% use is just a possession. My bicycle is “idle” 95% of the day and no one calls it a stranded asset, because no one was ever scoring it on occupancy. The car in the driveway, the tools in the drawer, the books on the shelf, all of them sit unused most of the time, and their value was never their utilization rate. It was their availability: the fact that they are there, owned and ready, at the moment you reach for them. Held capacity you do not fully consume is not waste. It is what ownership feels like from the inside, and a utility is the one thing that can never afford it.

This is the honest caveat, and I want to state it plainly rather than hide it inside the argument. Reframing idle as headroom does not make the dollars come out differently. If your only question is cost per token at your actual volume, rent, and a later essay in this series is entirely about why almost everyone will and should make exactly that choice. The focal-thing frame does not win the economic argument. It refuses to let the economic argument be the only one in the room. Those are different claims, and conflating them would be the dishonest move.

The bookshelf nobody audits

Here is the clearest case I know, because almost everyone already lives inside it and never thinks to apply the meter. A personal library is mostly books you will not reread, and a fair number you have not read at all. The Japanese even has a word for the second pile, tsundoku, the habit of buying books and letting them stack up unread. Nobody computes the utilization rate of a bookshelf. Nobody opens a spreadsheet, divides pages read by pages owned, and concludes the shelf is operating at 4% and should be liquidated. Nobody has ever called an unread book a stranded asset, and the accountant who tried would be asked to leave the house.

The reason is that a library was never scored on occupancy. Its value is two things the meter cannot price: availability, the book being there the day you finally reach for it, and identity, the shelf as a portrait of what you take seriously. The unread volume is not dead weight. It is a standing option on a future self, paid for in advance and waiting without complaint.

The idle desk machine is that bookshelf. Owned capacity at rest, valued for being there when you reach for it, not for how many hours it spent occupied. The library is the proof that we already accept this everywhere it does not threaten a business model. We only forget it the moment the object happens to draw power and have a price per token attached, at which point a perfectly ordinary possession gets recategorized as a failing utility. Move the same object onto a shelf and the panic evaporates. The machine did not change. The scoring rule did.

What the idle machine is actually for

So what does the headroom buy, concretely, if not cheaper tokens?

It buys the thing the meter cannot sell. When the price corrects, and the people who admitted it is correcting are the providers themselves, the renter at the bottom of the utilization curve is the one with no fallback. My idle machine is the fallback. It is the capacity that is already paid for, already private, already under my control, sitting at 22% precisely so that it is there at 100% on the day I need it and the rented option has changed its mind about me. A datacenter cannot run that way because a datacenter is a business and idle capacity is its enemy. A possession can run that way because availability is the whole point of owning a thing.

There is a quieter return too, the one Hannah Arendt would have recognized. She separated labor, which consumes its product and leaves no trace, from work, which builds a durable thing that outlasts the doing. Renting tokens is labor in her exact sense: you consume the output, the meter resets, and nothing of yours remains. Keeping a machine you tune, fix, and understand is closer to work: at the end there is an artifact, a stack, a body of operational knowledge that is yours and persists. The idle hours are not the machine failing to be a utility. They are the machine being a thing I own rather than a service I consume.

What I am actually claiming

I am not claiming my Spark is a rational purchase on a spreadsheet. At my volume it is not, the cloud is cheaper, and the hardware depreciates while the rental price falls. NVIDIA is right that idle GPUs are stranded cost, for NVIDIA’s customers, who are running a utility.

What I am claiming is that the utilization metric is not a law of nature, it is the scoring rule for one kind of object, and that applying it to a possession is a category error dressed up as hard-nosed economics. My machine idles at 22% for the same reason my bicycle does: because availability, not occupancy, is what I bought. The waste is only waste if the thing was a utility, and the entire argument of this site is that it does not have to be.

This is essay three of a series. It follows the radical monopoly and the dependency I moved but did not remove. The spine of the series, and why each piece concedes its strongest objection before answering it, is on the philosophy page; the structured, complete version is the forthcoming book, for which these essays are the public workshop.