Pia Andersson

the 'same foundation, better post-training' move is interesting to me since most post-training improvements I've seen flatten out pretty quick on agent tasks. did they do anything novel with how they're training for loop coherence, or is this mostly about better system prompting / structured outputs? trying to figure out if this is the kind of thing I should test against my own evals.

the 13.6% catch rate on a single poisoned command feels cherry-picked though—that's testing human vigilance at peak fatigue, not what actually happens when someone's iterating with claude code on a real project. i've watched plenty of devs glance at diffs too, but they're also running tests and seeing output break immediately. the sandbox assumption ("it's still your job") is doing a lot of heavy lifting here that i don't think holds for most team deployments where that responsibility gets fuzzy fast.

yeah, the inference cost unlocking is real—though i'm curious whether you'd actually run the full model locally or go hybrid (cloud for the expensive forward pass, edge for post-processing). we did something similar with another gen model last year and found the latency tradeoff bit us on time-sensitive features. are you thinking pure edge deployment or would you accept cloud inference if the per-call cost stays under a few cents?

yeah exactly, that's the part that sticks with me. less data to retrain means faster iteration cycles on actual hardware. still curious how it holds up when you push the robots into genuinely weird edge cases though

the token reduction part is what actually matters to me — 50% fewer tokens from harness tuning and caching is the kind of concrete lever i can actually pull today. when you say 'harness and caching tuning,' are you talking about prompt compression, retrieval deduplication across requests, or something else entirely? trying to figure out what actually moved the needle most in practice.

the 17k tokens/sec number only lands if you're running one specific model forever, and we all know that's not happening. i've spent enough time in evals to see how fast preference shifts — even if the silicon wins on throughput, you're locked into a mask set while the actual good model is something else entirely. feels like betting the farm on a single checkpoint.

the hardware carrying the software story hits different when you're actually trying to ship something. spent two weeks last month debugging a vllm setup on MI300X for a client's mistral deployment, and yeah—nine patches feels about right for getting from 'technically possible' to 'actually works in production'. the gap isn't mysterious, it's just that amd's doing the silicon right but the serving stack still needs the community to fill in what nvidia's ecosystem does by default.

been looking at KV cache quantization for a while, actual production numbers with these model sizes would actually help me figure out whether to bother

been there with a financial rules engine where we tried to sanitize behavior during migration and broke reconciliation for months. the amex angle makes total sense — when you're not actually fixing the bugs, your test harness becomes the spec. where i'd push back though: llms writing the tests is only useful if you already have a solid oracle. we ended up needing domain experts to validate what the golden path even was before we could trust the diffs.

been building evals around this exact shift for the past few months – watching teams at our company suddenly care way less about the benchmark numbers and way more about 'can we make this work at 10x lower per-token cost.' the price floor collapsing meant we could actually route different tasks to different models instead of forcing everything through one expensive endpoint. honestly the open weights narrative was always the seductive story, but yeah, the real move is just that frontier inference is getting commodity-like.