Hetzner Is Quietly Commoditizing LLM Inference

Hetzner, the German host that HN commenters reflexively recommend whenever someone asks how to cut a cloud bill in half, is quietly serving LLM inference. Tucked into its new Hetzner Experiments portal is an OpenAI-compatible API — one model, no billing, no SLA — that developers spotted this week and promptly pushed to the top of Hacker News. It's a small experiment. It's also the clearest signal yet that open-weight inference is about to get the same treatment Hetzner gave VPS pricing: race it to the floor.

What's actually live

The offering is deliberately minimal. You log into the Experiments portal, create an API token, and point any OpenAI-compatible client at https://inference.hetzner.com/api/v1. There's exactly one model behind it: Qwen3.6-35B-A3B in FP8 — Alibaba's Apache-2.0 mixture-of-experts release from April, with 35B total parameters, 3B active per token, a 262K context window, and image input.

Because it's a plain base-URL swap, it works with essentially everything: the OpenAI SDKs, LiteLLM, coding agents like OpenCode — which Hetzner's own community tutorial walks through wiring up, a telling choice we'll come back to. Sliplane, whose blog post surfaced the experiment, measured 153ms median time-to-first-token and around 224 output tokens per second in late July — respectable serving for a model this size, not just a demo limping along on spare hardware.

To be clear about the caveats: it's free, unbilled, and carries no production guarantee. Hetzner hasn't published a roadmap, pricing, or even a formal announcement. Everything about where this goes is inference — fittingly — from the outside.

Why Hetzner doing this matters more than the feature itself

Plenty of providers already sell open-weight inference. Together, Fireworks, and DeepInfra compete on it in the US; in Europe, OVHcloud has AI Endpoints, Scaleway sells generative APIs, and IONOS runs an AI Model Hub. One more endpoint serving one Qwen model would be a footnote.

But Hetzner isn't a footnote company; it's a price-floor company. It owns its data centers, runs famously thin margins, and has a repeatable playbook: wait for a product to become boring and standardized, then sell it at a price that makes everyone else's margins look like a rounding error. It ran this play on dedicated servers, then cloud VPS, then S3-compatible object storage. The prerequisite is always commoditization — a standard interface and interchangeable suppliers.

Inference just crossed that line. The OpenAI chat-completions API became the de facto standard interface. Open-weight models — Qwen, DeepSeek, Llama — became the interchangeable supply. vLLM and SGLang turned serving into an ops problem rather than a research problem. At that point, per-token pricing stops being an AI-lab product and becomes an infrastructure product, and infrastructure products are exactly what Hetzner exists to undercut. The interesting question was never whether Hetzner could serve a 35B MoE model — a single RTX PRO 6000 Blackwell with 96GB of VRAM, the card in Hetzner's €889/month GEX131 server, holds the FP8 weights with room for KV cache. The question was when inference would get boring enough for Hetzner to bother. Apparently: now.

The model choice is the other tell. Qwen3.6-35B-A3B isn't a chat toy; it's arguably the strongest coding model you can serve on a single GPU right now, scoring 73.4% on SWE-bench Verified with only 3B active parameters. Pair that with the official tutorial being "hook it up to a coding agent" and the target workload is obvious: the token-hungry agentic coding loops that are currently the fastest-growing line item on developer credit cards.

What to do with it, and what not to

If your stack already speaks OpenAI, trying this costs you fifteen minutes: create a token, swap the base URL and model name behind a config flag, run your evals. Free FP8 Qwen3.6 is genuinely useful today for internal tools, batch classification and extraction, CI-time code review, and agent experiments where a failed request costs you a retry, not a customer.

What it is not, yet, is something to build on. No SLA, no billing relationship, no published rate limits, no stated data-processing terms — and "experiment" is Hetzner's word for services it can turn off. The single-source performance numbers are one blogger's measurements, not a guarantee. And check your compliance story before pointing anything sensitive at it: Hetzner's German domicile makes the sovereignty crowd assume EU processing, but Hetzner hasn't published where inference actually runs, and assumptions aren't DPAs.

The strategic read for European teams is still significant. The EU-sovereignty conversation around AI has mostly produced consortium slideware; what it's lacked is a provider that competes the way hyperscalers actually lose — on price. If Hetzner productizes this, Scaleway, OVHcloud, and IONOS have a margin problem, and "OpenAI-compatible, EU-owned, cheapest per token" becomes a real procurement category instead of a wish.

The skeptic's case

The ceiling here is hardware, and it's a real ceiling. Hetzner's public GPU fleet tops out at that single 96GB Blackwell workstation card — fine for mid-size MoE models, useless for frontier-scale ones that need NVLink-connected clusters of B200-class accelerators. Hetzner has built its entire business on avoiding exactly the kind of capex those clusters require. Serving one efficient open model on cards it already stocks is on-brand; a multibillion-euro accelerator buildout is not. So the realistic trajectory isn't "Hetzner competes with OpenAI." It's "Hetzner sets the European price floor for the open-weight mid-tier" — the Qwens and Llamas that cover an increasing share of production workloads, especially coding agents.

That's not a small outcome. The frontier labs will keep their pricing power at the top. But everyone whose business is reselling open-weight inference at a markup — and that's a crowded field — should be watching a beige experiment page in Falkenstein with some discomfort. Commoditization doesn't announce itself with a keynote. It shows up as a free endpoint from the cheapest host in Europe, and it doesn't leave.