Your macOS VM's GPU Was Never Slow, Just Mislabeled

A headline number like "11–16× faster LLM inference in macOS VMs" usually means someone found a clever new data path to the GPU. That's not what happened here. The team behind Cua, who build computer-use agent infrastructure on top of macOS virtual machines, sped up llama.cpp inside a VM by more than an order of magnitude without touching the GPU path at all. They just stopped the guest OS from underselling its own hardware.

That distinction is the whole story, and it's worth understanding because it reframes what's actually wrong with GPU compute in Apple Silicon VMs — and who's in a position to fix it.

The GPU was fast all along

Apple's Virtualization.framework gives macOS guests a paravirtualized GPU: Metal commands issued in the guest execute on the host's physical GPU. Raw throughput through that path is genuinely good — independent Geekbench comparisons have put VM Metal compute at roughly 92% of host performance (102,282 vs 110,960 in Howard Oakley's testing on the Apple Paravirtual device).

So why does llama.cpp crawl inside a macOS guest? Because the paravirtual device is honest to a fault about what it claims to be. Query supportsFamily: from inside a VM and you get back roughly an Apple5-era feature set: 32 KB of threadgroup memory, no SIMD-group matrix operations, no bfloat16. llama.cpp's Metal backend does exactly what a well-behaved app should — it asks the platform what's available and compiles conservative fallback kernels. The fast kernels, the ones built on simdgroup_matrix that make Apple Silicon competitive for local inference, never load. The app isn't broken and the GPU isn't slow; the capability advertisement is wrong, and everything downstream politely believes it.

Cua's fix is a two-part lie told back to the liar. On the host, a hidden preference (defaults write com.apple.gpusw.ParavirtualizedGraphics ForceUnrestrictedDeviceFeatureLevel -bool true) unlocks the device's feature level. In the guest, a small dylib injected via DYLD_INSERT_LIBRARIES intercepts Metal capability queries for one process and answers with Apple family 9 and 64 KB threadgroups:

lume ssh my-vm \
  "DYLD_INSERT_LIBRARIES=/path/to/LumeMetalCapabilities-arm64.dylib \
   LUME_METAL_APPLE_FAMILY_MAX=1009 llama-bench -m model.gguf"

With the good kernels unlocked, their numbers on an M1 Ultra host are dramatic. TinyLlama 1.1B prompt processing goes from 431.86 to 4,786.70 tok/s — 98% of bare metal. Token generation goes from an unusable 12.63 tok/s to 206.60. A 12B Gemma QAT build (Q4_0) jumps from 3.41 to 49.67 tok/s on generation, 94.8% of bare metal. The instructions were always executable — it's the host's GPU running them either way. Only the advertisement changed.

The tell: MLX never needed the hack

The most interesting row in Cua's benchmark table is the one with no speedup. MLX running Llama-3.2-3B in the stock VM already hit ~1,657 tok/s prompt and 172 tok/s generation — full speed, shim or no shim. Apple's own framework evidently doesn't gate its kernels on the same capability queries, so it was never fooled.

That's both a vindication and an indictment. It proves the paravirtual GPU could run modern compute kernels all along, and it means the "slow VM" reputation that's dogged this stack — Tart users hit it in 2023, UTM users have filed the same confused issues — is largely a capability-reporting bug wearing a performance costume. It also yields immediate practical advice: if you control your in-VM inference stack and can live with MLX's model format, you don't need any of this. The shim matters specifically for the GGUF/llama.cpp ecosystem, which is still where most quantized models and tooling live.

Who actually needs LLMs inside a macOS VM

This isn't a niche concern anymore. The strongest use case is the one Cua is building for: agent sandboxing. If you're letting a computer-use agent click around a real desktop, you want it in a disposable VM — and increasingly you want the model running inside the sandbox too, for isolation or for air-gapped setups. Mac CI farms are the other constituency: the Tart/lume ecosystem runs enormous fleets of macOS guests, and until now anything ML-shaped in those pipelines either ran on CPU or got punted to an external API.

The alternative route — Linux guests reaching the host GPU through Vulkan, Venus, and MoltenVK, as in Red Hat's krunkit/RamaLama work — gets near-native llama.cpp performance too, and it's arguably better supported. But it can't run macOS software, which is the entire point for agent sandboxes and Mac CI.

Don't ship this — but watch it

Be clear-eyed about what this is: a DYLD_INSERT_LIBRARIES shim spoofing answers that depend on private, version-sensitive Metal internals, validated on exactly one host (M1 Ultra, macOS 26.6.1) and one guest image. It's per-process, hardened binaries will refuse the injection, and any macOS update could quietly break it. Nobody should build production infrastructure on it, and to their credit the authors say as much.

Its real value is as an existence proof with a benchmark table attached. The 10× gap between stock and unlocked guests is now demonstrably a policy choice in how Apple's paravirtual device reports itself, not a physics problem or a virtualization tax. Apple restricting guest feature levels made sense when the concern was compatibility across host GPUs; it makes much less sense now that VMs are becoming the default substrate for AI agents on Macs. The right fix is an opt-in honest feature level in Virtualization.framework — a one-line configuration API, not a dylib full of interposed selectors. Until Cupertino ships that, this hack is the clearest argument anyone's made that they should.