macOS VMs Have Been Sandbagging Your GPU

The most interesting thing about the 11–16× llama.cpp speedup Cua published today is that no hardware got faster. The GPU inside a macOS virtual machine was always capable of these numbers. Apple's paravirtualized graphics device was just telling guests it wasn't — and llama.cpp, doing exactly what a well-behaved Metal app should do, believed it.

Despite the "GPU passthrough" in the post's URL, this isn't passthrough at all. It's capability unmasking, and that distinction is what makes it both clever and fragile.

A virtual GPU that undersells itself

When you boot a macOS guest through Apple's Virtualization framework — the machinery under Cua's Lume, Tart, and most modern Mac CI runners — the guest doesn't see your M-series GPU. It sees a paravirtual Metal device that forwards work to the host. That device works, but it reports itself as roughly an Apple family 5 GPU with 32 KB of threadgroup memory: A13-era capabilities, on hardware that's several generations past that.

llama.cpp keys its Metal kernel selection off exactly those queries. Told it's on an ancient GPU, it skips the SIMD-group matrix multiply, SIMD-group reduction, and bfloat16 paths and falls back to conservative kernels. The silicon underneath could run the fast paths the whole time; nothing ever asked it to.

Cua's fix is a process-scoped shim, injected via DYLD_INSERT_LIBRARIES into the guest process, that intercepts those capability queries and answers with Apple family 9 and 64 KB of threadgroup memory. Paired with a host-side defaults key — com.apple.gpusw.ParavirtualizedGraphics ForceUnrestrictedDeviceFeatureLevel — that lifts the feature-level cap for VMs launched by your user, llama.cpp suddenly picks the kernels it would pick on bare metal.

The before/after on an M1 Ultra (48-core GPU, macOS 26 host and guest) is stark. TinyLlama 1.1B Q4_K_M goes from 431.86 to 4,786.70 tokens/sec on prompt processing (11.08×) and from 12.63 to 206.60 tokens/sec on generation (16.36×). Gemma 4 12B QAT Q4_0 jumps from 3.41 to 49.67 tokens/sec on generation — from unusable to genuinely interactive.

Read those baselines again, though. 12.63 tokens/sec generating from a 1.1B model on an M1 Ultra is catastrophically slow. The headline multiplier measures how broken the stock configuration was, not how fast the fix is. And the comparison is stock VM versus unlocked VM — the post makes no claim of native parity, and virtualization overhead remains. Tellingly, Cua found MLX-LM's performance flat under the same shim: it was already fast in the stock VM, so the cliff is specific to how llama.cpp gates kernels on capability answers.

Who actually runs models inside a macOS VM

Fair question, because most local-inference people run llama.cpp directly on the host and never hit this. The answer is anyone who needs isolation on Mac hardware: CI runners, ephemeral dev environments, and — Cua's own business — sandboxed computer-use agents that need to click around a real macOS desktop without access to yours. Apple's license permits two macOS VMs per machine precisely for these workloads.

Until now, that isolation carried a brutal tax on inference, and the workaround was architectural: run the model on the host, sandbox only the agent, and accept that your "isolated" workload has a chatty channel to an unsandboxed inference server. Unmasking the guest GPU means the whole stack — model included — can live inside the VM boundary. For agent evals, for CI jobs that exercise a local model, for letting an agent run AI-generated code next to its own brain, that's a real architectural simplification, not a benchmark stunt.

There's an irony in where this leaves the ecosystem. Linux guests on macOS solved this problem earlier and more thoroughly: Venus/Vulkan through libkrun reached 75–80% of native, and Red Hat's API-remoting work forwards GGML calls to a host-side Metal backend at near-native speed. Apple's own container project still has GPU support as an open discussion. Running a macOS guest on a macOS host — the most Apple-native virtualization stack imaginable — was, until today, the worst way to run a model on a Mac.

The catch, and it's not small

Everything here leans on private, version-sensitive behavior. The defaults key is an undocumented Apple preference. The shim answers capability queries based on reverse-engineered Metal internals that Apple can rearrange in any point release. DYLD_INSERT_LIBRARIES injection is rejected by hardened and platform-protected binaries, so this only works for processes you build and control — fine for llama.cpp, a non-starter for signed third-party apps. And the benchmark covers exactly one machine: an M1 Ultra on one macOS 26 build. Cua ships verify scripts for good reason.

That defaults key is also the most damning detail in the post. Apple already built a switch called ForceUnrestrictedDeviceFeatureLevel. The conservative capability profile guests see is a policy choice — presumably lowest-common-denominator compatibility for older guests — not a technical wall. The HN thread asked why the framework undersells the host GPU at all, and there's no good answer. Apple could expose truthful capabilities, or a documented opt-in, tomorrow.

Use it, don't build on it

My read: this is real, reproducible, and worth deploying today for dev-loop and sandbox workloads — the mechanism is well explained, the before/after methodology is honest, and the scripts are in the open. If you're running Lume or any Virtualization.framework-based macOS guest for agent work, the setup is a build script, one defaults write on the host, and two environment variables in the guest. Twenty minutes, order-of-magnitude payoff.

But treat it as a temporary unlock, not a foundation. Anything that couples your production path to undocumented preference keys and dylib injection into Metal's query surface will eventually lose an argument with a macOS update. The durable fix belongs to Apple, and the existence of that defaults key says they know it. Until Virtualization.framework stops lying to its guests, Cua's shim is the right kind of hack: small, inspectable, and honest about what it is.