Topic

#Vllm

8 articles on Vllm — news, releases, guides and analysis from the SourceFeed engine.

Latent Reasoning Escapes the Lab, Bolted Onto DeepSeek-V4

A 36M-parameter head makes V4-Flash think silently in latent space — and the serving hacks matter more than the benchmarks.

Mariana Souza

Your Agents Are Waiting on the CPU, Not the GPU

Agentic workloads moved the inference bottleneck off the accelerator, and hardware ratios are quietly following.

Article · 5d ago5

DeepSeek V4 Flash on One AMD GPU Took Nine Patches

A single 192 GB MI300X now serves the full checkpoint — proof of AMD's hardware edge and its lingering software gap.

Article · 1w ago1

The Real Cost of 'Just Use vLLM,' According to Netflix

Netflix's Model Runtime team details the version pinning, GIL workarounds, and preemption bugs behind production LLM inference.

Article · 2w ago2

Autoscale GPU Inference on EKS with Karpenter and Spot Instances

Let Karpenter provision and bin-pack spot GPU nodes under a vLLM server, then delete them when traffic drops.

Tutorial · 2w ago0

The Real Cost of Running SOTA LLMs Locally

Why brute-forcing frontier models onto local hardware fails, and how clever pipeline decomposition offers a sustainable path forward.

Article · 1mo ago4

Ornith-1.0: Coding Models That Train Their Own Agent Scaffolds

By optimizing both the reasoning loop and the code output, these MIT-licensed models bring native agentic capabilities to local hardware.

Article · 1mo ago0

Serve an Open-Source LLM at Scale with vLLM on a Rented GPU Instance

Go from a bare cloud VM to a production-ready, OpenAI-compatible inference server in under an hour, using vLLM's continuous batching to hit thousands of output tokens per second on a single GPU.

Tutorial · 1mo ago0