#Gpu

Your Agents Are Waiting on the CPU, Not the GPU

Agentic workloads moved the inference bottleneck off the accelerator, and hardware ratios are quietly following.

Article · 5d ago5

An 8B Fine-Tune Now Fits in 4 GB of VRAM

Soup streams frozen decoder layers through the GPU one at a time, and proves the arithmetic is bit-exact.

Article · 1w ago1

AirLLM's 4GB 70B Trick Is Real, and Beside the Point

Layer streaming makes giant models fit; disk bandwidth decides whether you'd ever want to run them that way.

Article · 1w ago1

KNOD Wants to Turn Your AMD GPU Into a DPU

New RFC patches let the Linux kernel JIT-compile packet programs straight onto GPU silicon, with no ROCm required.

Article · 2w ago0

Autoscale GPU Inference on EKS with Karpenter and Spot Instances

Let Karpenter provision and bin-pack spot GPU nodes under a vLLM server, then delete them when traffic drops.

Tutorial · 2w ago0

PyTorch Monarch Escapes the CUDA Bubble

AMD and Meta port the single-controller runtime to ROCm, and the fault-tolerance demo is the real story.

Article · 2w ago0

The 500-Line Renderer Worth Writing Yourself

Tinyrenderer keeps resurfacing because nothing else teaches what GPU APIs actually do.

Article · 3w ago0

Firefox Bets on Vulkan to Fix Linux Video Decoding

A pref-gated Vulkan Video path in Firefox 153 finally gives NVIDIA-on-Linux users real hardware decode.

Article · 3w ago1

CUDA Without NVIDIA: What Actually Works

Drop-in translators, HIP, and SYCL solve different problems. Choose by binaries versus long-term portability.

Article · 4w ago0

Demystifying the NVIDIA DGX Spark for API Developers

Transitioning from cloud APIs to local GPU hardware requires mastering unified memory architectures and physical thermal constraints.

Article · 1mo ago0

Popping the CPU-GPU Latency Bubble in Inference

Pipelined decoding techniques show that software optimization, not just raw hardware scaling, is the key to maximizing GPU utilization.

Article · 1mo ago2

OpenAI Jalapeno and the Shift to Custom Inference Silicon

Custom ASICs are replacing general-purpose GPUs for running large language models to survive the crushing cost of scale.

Article · 1mo ago7

Serve an Open-Source LLM at Scale with vLLM on a Rented GPU Instance

Go from a bare cloud VM to a production-ready, OpenAI-compatible inference server in under an hour, using vLLM's continuous batching to hit thousands of output tokens per second on a single GPU.

Tutorial · 1mo ago0

The Architecture of Monopoly: Inside NVIDIA's Supercomputing Hegemony

NVIDIA now powers 81 percent of the world's fastest supercomputers, forcing a fundamental rewrite of high-performance software.

Article · 1mo ago0

Running 70B Models on 4GB VRAM: The AirLLM Layer-Swap Hack

AirLLM trades disk I/O for VRAM, letting developers run massive models locally without renting enterprise GPU clusters.

Article · 1mo ago1