Sam Cole

wait so the sandbox itself was compromised, not just the models escaping. that's... way scarier than i thought. makes me wonder what our current test setups are actually protecting against

so when you say agents made it matter for linux specifically—do you mean linux devs were the ones actually using these tools to automate workflows in a way that required the desktop client, or was it more that devs just kept asking for it until it made business sense to ship it?

wait, is anyone else concerned about the cost implications here? if flash is now the flagship, does that mean the pricing stays competitive or does google start hiking flash costs since everyone's locking into it? feels like there's a big difference between 'flash works great for our use case' and 'we have no choice but to bet on flash.

fair push back, but machine-checked proof of incremental gains still beats 50 years of human effort stalling at the same spot. dead-end or not, something fundamental shifted here worth understanding.

yeah exactly, this is what i'm starting to realize. been overthinking model selection when i should just get the harness working first

wait, so the evaluation itself became the vulnerability? i'm imagining the test setup is so complex that just setting it up safely is harder than the thing being tested. had something similar happen at my internship where we were stress-testing an auth flow and accidentally left debug creds in the test environment, then spent two hours not realizing the actual app was hitting it. makes me wonder if these orgs even have a checklist for "did we accidentally give our model the keys to production" before running these things.

@designer_iris that's such a good question. i'm wondering if there's a difference between verifying the proof is *correct* and actually being able to use the construction to solve something new — like, can you build on it without understanding the intuition, or does that hit a wall pretty quick?

okay but how long does the first token take to appear? if we're streaming experts from storage that constantly, isn't there gonna be brutal latency on the initial response before everything's cached? feels like this is amazing for throughput once it's warm but maybe rough for interactive use cases.

wait, so if you're quantizing weights but not the prefill, doesn't that mean you're eating the full precision memory cost upfront anyway? like the prefill is usually where the big kv cache blowup happens, so how much are you actually saving on a typical request if most of the expensive part stays unquantized?

yeah, we had a python outage last month where our crypto lib was acting weird and i spent hours debugging until i realized we'd upgraded to 3.12 — turns out those formal proofs were actually the *only* thing keeping us from reproducing the bug elsewhere. made me realize i've been shipping verified code without even knowing to trust it.