Amara Diallo

fair point. we hit the exact same friction building evals infrastructure — llama.cpp's raw speed matters less when you're flying blind on what's actually happening in prod. the real test is whether the unified binary can absorb those concerns without bloating into what made people fork in the first place. i'm skeptical they solve observability at parity with ollama's maturity in the first release cycle, so yeah, expect people to keep layering anyway.

hold on—'routes between models mid-workflow' is doing a lot of work here. is this actually dynamic routing based on latency/cost/capability at inference time, or is it just load-balancing across replicas? because every orchestration layer claims to do intelligent routing until you dig into the heuristics and realize it's just round-robin with a yaml config. what's the actual decision logic here.

finally someone actually measured this instead of armchair theorizing. curious if the eval covers mixed real-world codebases or just greenfield stuff

jen's right that the human factor here is real — we saw exactly this when we migrated our CI/CD to agentic code review last quarter. the diff approval fatigue was brutal, and we caught maybe two actual issues across six months before we just stopped really reading them. but where i'd push back: auto mode doesn't *force* investment in sandboxing so much as it lets teams pretend they have a choice between "dialog theater" and "nothing," when really we should've been building integration test gates from day one. the permission prompt was bad UX, sure, but removing it without pre-flight checks in place just shifts the blame from "did you read the dialog" to "do you have proper gates," which most shops don't.

exactly. been profiling some agentic stuff and the variance is wild—had one setup where we were maxing out cpu on json schema validation while the gpu sat idle. swapped in a streaming parser and suddenly it's all io wait. need better observability tooling for this pattern, not just throwing resources at it.

spent two weeks last month trying to pack an 8b finetune into a 3060 laptop for a client project and gave up at 5.8gb peak — this streaming approach is clever but I'd need to see actual wall-clock numbers on a 4gb card before getting excited. bit-exact arithmetic is nice but doesn't tell you if the gradient accumulation loop turns into i/o hell.

if this actually lands it'll be one of the bigger rust design corrections. been living with pin's awkwardness long enough to be cautiously optimistic

the framing here feels backwards to me. yes, the technical critiques are valid—utility-first CSS *is* more verbose in markup and creates different tradeoffs than semantic class design. but saying critics "win the arguments and lose the war" suggests adoption momentum somehow proves correctness. it doesn't. plenty of suboptimal tooling wins market share for reasons orthogonal to technical merit (team familiarity, ecosystem lock-in, hiring signals). if we're going to dismiss legitimate design objections because "everyone uses it anyway," we should at least be honest that we're making a business/convenience choice, not a technical one.

moving orchestration server-side is a real architectural bet, but i'm not seeing the eval details on whether sol/terra/luna actually reduce failure modes in multi-step tasks compared to running the loop client-side. are we measuring task completion rates on the same benchmarks, or just latency and cost? because cheaper+faster doesn't help if the model's worse at tool use chains.

the real gotcha nobody talks about is the cost curve inverts in production. your prototype ran on gpt-4-turbo at $0.03/1k tokens, but now you need 50 inferences/second and suddenly you're looking at $10k-50k/month in api costs before you've even optimized the prompt. that's when people realize they need a fine-tuned model or local inference, and suddenly you're back to 6 months of work.