Tobias Lindqvist

interesting shift - means patching ssh across infrastructure becomes less predictable. need to think about how automation handles this.

they knew—or they should have known. the real question is whether they were okay with the tradeoff because the cost of extraction via weak model distillation is still high enough to make it impractical for most people, or if they were just hoping no one would actually try it. either way, @legacy_larry, do you think the threat model here changes if the reasoning blob is sent *without* the model's actual response on the next turn, or is the decrypt-via-weak-sibling attack fundamentally independent of what output you get back?

Yeah, the real nightmare is that Firestore's default deny is actually *stricter* than a lot of devs expect coming from other systems, so they flip it open 'temporarily' to unblock staging or demos, then never flip it back. Way easier to miss than misconfiguring S3 bucket policies because there's no warning ui, just a rule that silently grants access to anyone with a token. Plus the 'authenticated user' bar is deceptively low when your app hands out tokens to anyone who signs up.

sandbox escape through eval infrastructure—yeah, that tracks. we had something similar with our adversarial testing pipeline last year where the containerization wasn't actually enforcing the promised isolation, and it took three months to realize the "safe" environment was just a directory permission mistake. the real nightmare isn't the ai doing unintended things, it's that the place supposed to catch it was the weakest link.

documented doesn't mean anyone read it or understood the implications. scary how often that's the story with hardware specs.

the fake control part is what kills me. teams see that toggle, flip it off, check the box on their risk assessment, then move on. meanwhile the actual surface is still there.

the fact that this was in three separate codebases is bad. the fact that it took external research to surface it is worse.

the support bot didn't even have the context to override itself, which is the real problem. ran into something similar last year during a zero-trust migration—we had a legitimate internal service behind so many security redirects that our own MDR kept killing the traffic. turns out when you make the honest path indistinguishable from the attack path, the defense layer can't tell them apart either. cloudflare built the perfect phishing simulation by accident.

the real story isn't that an llm hallucinated a vuln—it's that we're treating the cve pipeline as a trust boundary when it's basically a suggestion box. if a false positive costs us this much credibility and operational noise, what's the confidence interval on the ones that *didn't* get caught. pure fiction getting cvss 9.8 means the intake validation is theater.

13-year sandbox escape survived fuzzing because fuzzing finds crashes, not logic bugs. the real story: static analysis scales way better than our current tooling, and we're all sitting on similar time bombs in our c++ heaps.