Oxide Is Building the Cloud Contract On-Prem Kubernetes Lacks

Kubernetes has always been able to run on anything with a kernel and enough RAM. That was never the hard part. The hard part is what separates "VMs you can schedule pods onto" from an actual cloud: infrastructure Kubernetes can drive through its standard extension points — a Cluster API provider for provisioning, a cloud controller manager for nodes and load balancers, a CSI driver for volumes. Hyperscalers have those. On-prem, that contract has mostly been the domain of VMware and OpenShift, plus a long tail of half-maintained glue.

That's the lens for reading Oxide's new writeup on its Kubernetes integrations. The headline story is customer-driven development — customers asked for Kubernetes in late 2024, Oxide had nothing supported, so it hired a solutions engineer (Matthew Sanabria) and built what people asked for. The more interesting story is that Oxide is methodically implementing the full cloud-provider contract for its rack, and is unusually honest about which pieces aren't done.

Three front doors, no house distro

Oxide made a choice worth noticing: it ships no Kubernetes distribution of its own. Instead it built three provisioning paths that plug into tooling teams already run.

The first was a Rancher node driver — which started as a customer's pull request that Oxide merged, then hardened with CI and docs. If you manage clusters through Rancher, Oxide instances become just another node source.

The second is an infrastructure provider for Omni, built with Sidero Labs in a seven-week sprint ahead of KubeCon North America 2025, targeting Talos Linux shops. That integration surfaced a genuinely obscure upstream bug: Talos's filesystem probing only checked ISO 9660 superblocks, not VFAT, so small cloud-init user-data payloads were silently ignored. The current workaround is padding user-data with comments until it crosses the size threshold that produces an ISO 9660 superblock — the kind of gross-but-documented fix you only find by actually shipping.

The third is Cluster API Provider Oxide (CAPOx), the Kubernetes-native path: declare OxideCluster and OxideMachine resources, let Cluster API reconcile real instances into existence. It pairs with a Packer plugin for building node images through the upstream Kubernetes Image Builder. Note the maturity level, though: the API is v1alpha1, and the repo is weeks old. This is a provider to pilot, not to bet a production migration on today.

Meeting customers inside Rancher, Omni, and Cluster API rather than shipping an opinionated distro is the opposite of the OpenShift or Harvester playbook, and it's the right call for a hardware company. Oxide's value is the substrate; forcing a distro choice would shrink its market to zero overlap.

The contract underneath

Provisioning gets you nodes. The cloud controller manager is what makes a cluster feel native: Oxide's CCM reconciles Kubernetes Node objects against actual instances and — the part developers will care about — implements Service of type LoadBalancer.

Here's where the honesty shows. The Oxide rack has no native load balancer yet, so the CCM fakes one with floating IPs, publishing a two-entry ingress status (a floating IP in Proxy mode plus the node's internal IP in VIP mode) to keep externalTrafficPolicy: Cluster working. It works — one commenter in the Hacker News thread reported running the CCM for LoadBalancer services with NGINX Gateway Fabric on top — but it's a workaround, and Oxide says so. If your on-prem story depends on real L4 load balancing with health checks and connection draining, that's still your problem to solve with MetalLB-style patterns or an ingress layer.

Storage is the other gap, and the details are instructive. Customers running Longhorn on Oxide's distributed disks were stacking three-way replication on top of three-way replication — every write fanning out nine ways. Oxide's fix landed in system release 18 as local disks, letting Longhorn own replication against raw local storage. A native CSI driver is designed (RFD 595 is public) but blocked on disk hot-plug support across the stack. Until it ships, stateful workloads on Oxide mean Longhorn, full stop.

Who this is actually for

Oxide's pitch has always been "the cloud experience, on hardware you own," and Kubernetes is where that claim gets tested, because Kubernetes is ruthless about exposing infrastructure that only pretends to have an API. The fact that a CCM and a CAPI provider were buildable at all — against the same public API customers use — is quiet validation of the rack's design. Compare the contortions Metal3 goes through to make bare metal look declarative, or the license math VMware customers have been doing since the Broadcom acquisition, and the appeal is obvious: an EC2-shaped substrate in your own datacenter that upstream Kubernetes tooling drives without adapters.

But be clear-eyed about where this sits. No native load balancer, no shipped CSI, no autoscaling (Karpenter came up in the HN thread; Oxide says it's thinking about where CAPI ends and Karpenter begins), and a v1alpha1 provider the company itself is still dogfooding. That's roughly where DigitalOcean's Kubernetes integrations were circa 2018 — the contract is being honored, piece by piece, in the right order, but the pieces aren't all there.

The verdict: this is a genuine differentiator in the on-prem Kubernetes market, executed early. If you're a Rancher or Talos shop with a hardware refresh or a VMware exit on the calendar, Oxide just became evaluable in a way it wasn't a year ago — and everything is Apache-licensed on GitHub, so the evaluation costs you a repo clone. If you need production parity with a managed cloud control plane this quarter, wait for the CSI driver and native load balancing; those two are the difference between "Kubernetes runs here" and "Kubernetes belongs here."