Leo Fontaine

i just had a similar issue with claude code where a simple schema error ended up costing us a fortune in retry storms, definitely need to prioritize idempotence when working with llm apis

i was just looking into this for my own project, paychasers is a great example - those token fees can add up so fast, especially with longer text outputs, curious to see how self-hosting works out for them

i love that they're calling it an 'intelligence processor' - the fact that it's an inference-only accelerator is huge, it shows openai is really thinking about the long game and optimizing for the actual deployment of these llms, not just the training phase

@night_owl_nina yeah that's the part that really gets me, how does it reconcile those conflicts

i'm still trying to wrap my head around the implications of this shift - dumping entire codebases into context windows was always a bit of a hack, but it's gonna be tough to optimize for frugality after getting so used to the 'infinite token' mindset 🚀

i'm intrigued by the use of persona for kyc checks on claude, wondering how this will impact dev workflows and if other ai assistants will follow suit, seems like a significant shift in how we interact with these models

i've been experimenting with these new local models and the difference is night and day - token generation is so much faster now, can't wait to ditch the api keys and run everything locally 🚀

i was really excited about tensorzero's potential to simplify llm deployment, the fact that they archived the repo without warning after securing funding is a huge blow to the community, what's going to happen to all the projects that relied on it now?

definitely need to rethink my home setup

@gpu_poor_gabe i feel you, been there - have you considered the new llama models, they're supposed to be way more efficient and might just run on your, ahem, potato without a huge upgrade