Topic

#Inference

21 articles on Inference — news, releases, guides and analysis from the SourceFeed engine.

Article 1d ago 2

Mistral 3 Was Never About the Leaderboard

Eight months on, the Apache-licensed family looks less like a frontier bid and more like enterprise infrastructure.

Mariana Souza

How OpenAI's Jalapeño Chip Changes Production LLM Serving

The custom silicon shift signals a move away from general-purpose GPUs toward highly specialized, memory-optimized inference architectures.

Article · 1mo ago1

Serve an Open-Source LLM at Scale with vLLM on a Rented GPU Instance

Go from a bare cloud VM to a production-ready, OpenAI-compatible inference server in under an hour, using vLLM's continuous batching to hit thousands of output tokens per second on a single GPU.

Tutorial · 1mo ago0

Running 70B Models on 4GB VRAM: The AirLLM Layer-Swap Hack

AirLLM trades disk I/O for VRAM, letting developers run massive models locally without renting enterprise GPU clusters.

Article · 1mo ago1

Unified x86 AI Acceleration: Inside the New ACE Specification

The x86 Ecosystem Advisory Group's new spec brings standardized matrix multiplication and tile registers to modern CPU architectures.

Article · 1mo ago2

Xiaomi's MiMo-V2.5-Pro-UltraSpeed Pushes a 1T Model Past 1000 Tokens/Sec on Commodity GPUs

Through FP4 quantization, block-level speculative decoding, and the TileRT system stack, Xiaomi claims trillion-parameter decode speeds normally reserved for custom silicon — on a single 8-GPU node.

News · 2mos ago5