saganite.bsky.social @saganite.bsky.social · 05/11/2025Imagine if you were really hungry, but you could perfectly satisfy your hunger with as much glorious feasting as you wanted, but with this special type of feasting, there would be no personal consequences to you but just fuzzy ones for society at large? 100
saganite.bsky.social @saganite.bsky.social · 05/11/2025I think that the root cause of the fertility crisis is the most obvious things, two things I care about very deeply: birth control, and women's liberation. I think the death of boredom plays a role too. 120
saganite.bsky.social @saganite.bsky.social · 05/11/2025Yes, it's a little known fact that "Rock" was actually used to refer to a solid aggregate of minerals before it came to be used as a music genre. Could you perhaps propose some alternate name we could use for that one? And "country" was already used to refer to nation states... 110
saganite.bsky.social @saganite.bsky.social · 15/10/2025I think the list of things that work in ML if you have a dataset with perfect coverage over the problem space is very large! 100
saganite.bsky.social @saganite.bsky.social · 15/10/2025Cartridges are cool, but you need a good dataset right? Like, your training dataset needs to cover all of your latent space. You can write a prompt that is general for all animals, and then collapse it into a cartridge training on most of them, but if you left out emus would emu queries still work? 100
saganite.bsky.social @saganite.bsky.social · 10/10/2025Thanks Tim! I would also mention though, even if you are rewriting 90% of the code, you can still get a 10% speed improvement just by using this feature. People slave away in the cuda mines for a 10% speedup, and here it is, sitting right in front of you, with 10% even in the WORST case. 040
saganite.bsky.social @saganite.bsky.social · 10/10/2025I don't do social media so nobody is going to read this, so I'll just @ some of my favorite LLM bsky accounts begging for some reskeets. @timkellogg.me @cameron.pfiffer.org @timfduffy.com @natolambert.bsky.social @howard.fm 030
saganite.bsky.social @saganite.bsky.social · 10/10/2025The change is in our experimental vLLM fork vLLMx for the moment, but we will be submitting a PR to vLLM main shortly. 100
saganite.bsky.social @saganite.bsky.social · 10/10/2025There is already support for this in the OpenAI api specification, and this change brings it to vLLM in a much better form. OpenAI is actually the only other provider I'm aware of providing this feature, and it actually results in SLOWER performance, while ours is much faster. 110
saganite.bsky.social @saganite.bsky.social · 10/10/2025Blog Post: cascadetech.ai/blog/vllm-pr... Demo: app.cascadetech.ai Think: Speculative decoding, but instead of a draft model (slow, complicated, wrong) you have a static text prediction of the output, and a diff algorithm to keep it aligned when it diverges.cascadetech.aiVLLM Predicted Outputs 100
saganite.bsky.social @saganite.bsky.social · 10/10/2025I would like to share some work we've been doing at cascadetech.ai: Predicted Outputs in vLLM. If you aren't familiar with PO, it allows you to dramatically speed up generation when you know something about the contents of the output (think: code modification). 1113
saganite.bsky.social @saganite.bsky.social · 11/09/2025These are INCREDIBLY complex simulations, including multi threading, physics simulations, and many billions of floating point operations that have to be deterministic down to the last bit of the mantissa over the course of hours of play. And they are. 110
saganite.bsky.social @saganite.bsky.social · 11/09/2025As a former videogame developer, I can tell you that you can definitely build deterministic software on cpus! One really efficient way to do multiplayer is to replicate input across all nodes and then run a fully deterministic simulation on each node. 110
saganite.bsky.social @saganite.bsky.social · 07/09/2025It's hilarious how many accounts on blue sky are just literally an onion article about accounts on blue sky. 100
saganite.bsky.social @saganite.bsky.social · 07/09/2025Not to mention the Linux tradition of everything being a text stream is very conducive to LLM integration. I just installed desktop Linux on my new computer, pretty happy with it so far. Mug smoother experience than the last time I tried. 000
saganite.bsky.social @saganite.bsky.social · 21/08/2025But there isn't some OTHER festival down the road for people in their 20s. ALL music festivals are for gen x and older millennials. Rock and roll is dying, but even live music is dying with it. 020
saganite.bsky.social @saganite.bsky.social · 21/08/2025There is no genre of music where at live shows you see a crowd aged under 30. I just went to a music festival La Route du Rock which would have been 20 year olds 20 years ago. Now it was mostly people over 40. 120
saganite.bsky.social @saganite.bsky.social · 21/08/2025Had a long discussion about this with my ethnomusicologist friend last week who teaches history of rock to 18 year olds. Apparently not only are they not forming bands, but they also aren't even attending live music events at all. 120
saganite.bsky.social @saganite.bsky.social · 07/08/2025This HAS to be AI slop that humans didn't catch right? Like, this is bullish for gpt-5? 100
saganite.bsky.social @saganite.bsky.social · 20/06/2025Wow, amazing. Are you going to explore beyond ulaanbaatar? The country is incredible but the capital city is not remotely representative. 010
saganite.bsky.social @saganite.bsky.social · 18/06/2025What they are telling those employees is the honest truth, irrespective of their company's business practices. 000
saganite.bsky.social @saganite.bsky.social · 18/06/2025or "Don't bother learning how to effectively use AI, we will still keep employing you even when other prospective employees will work more efficiently for the same salary"? 200
saganite.bsky.social @saganite.bsky.social · 18/06/2025or "Don't worry, even as the employees of our competitors adopt AI to be more efficient, we'll just keep doing things the old way so that we don't have to lay anybody off?" 210
saganite.bsky.social @saganite.bsky.social · 18/06/2025Sorry, what exactly would YOU say if you were those CEOs? "Don't worry, AI isn't going to affect employment in our industry?" 200
saganite.bsky.social @saganite.bsky.social · 02/06/2025have you tried using gemini instead of anthropic? in my experience you can get better quality for a TINY fraction of the price. gemini 2.0 flash lite is 10x cheaper than haiku, and flash 2.0 is like 8x cheaper. 110
saganite.bsky.social @saganite.bsky.social · 13/05/2025I have had such miserable results with anything cooking related. We did a cocktail night where we drank LLM cocktails and they were so very bad. I feel like llms are in letter counting territory with recipes. 000
saganite.bsky.social @saganite.bsky.social · 07/05/2025Whether or not the thing they are making is something they have seen before or not is irrelevant, it just needs to be describable in natural language. 010
saganite.bsky.social @saganite.bsky.social · 07/05/2025They have the ability to turn an accurate english language description of software into actual software, usually with pretty good engineering practices. 110
saganite.bsky.social @saganite.bsky.social · 07/05/2025This just isn't true. AI coding tools know all of the same abstractions, data structures, and basic algorithms that I know (and more), which I use to recombine into software. 120
saganite.bsky.social @saganite.bsky.social · 24/04/2025I know it won't matter much 5 years from now, and won't matter at all 10 years from now, but a strong background in computer programming is a very exciting lens with which to watch this all unfold. 000
saganite.bsky.social @saganite.bsky.social · 24/04/2025Even now I write probably less than 5% of the code I check in, but it's SO incredibly helpful to have nearly 40 years of programming experience to rely on while using today's coding models. 100
saganite.bsky.social @saganite.bsky.social · 24/04/2025The window of human history where people know how to program computers is so vanishingly small. It only really began in earnest in the 60s. I learned in the 80s. The people learning today are probably some of the last humans to ever learn. 100
saganite.bsky.social @saganite.bsky.social · 24/04/2025Wow, only TWO KV heads. That means a really small kV cache, so it should be pretty fast for long context. 100
saganite.bsky.social @saganite.bsky.social · 17/04/2025Open source matters a lot more in the age of being able to ask an ai agent to not only modify an application but continue to pull and merge main after those changes. I will almost certainly switch to this and customize it. So many little things I would love to change about Claude code 020
saganite.bsky.social @saganite.bsky.social · 17/04/2025i rarely use 4.5 for my day to day stuff just because of the speed. it also keeps me using gpt-4o rather than claude, it's just quite a bit faster. 000
saganite.bsky.social @saganite.bsky.social · 16/04/2025I appreciate the sentiment, but I don't actually think that the uk supreme court are radical feminists! 110
saganite.bsky.social @saganite.bsky.social · 15/04/2025it's all so depressing and exhausting. we need a BIG FUCKING TENT to stop donald trump, and people like Matthew Yglesias need to be part of it, and honestly even people well to the right of him if we have any hope of succeeding. 231
saganite.bsky.social @saganite.bsky.social · 15/04/2025just fucking read them, consider them, and by all means engage with them via honest debate. literally NOBODY on here is trying to engage with the ideas in general, they are just trying to find the worst possible interpretations of small details so that they can declare failed purity tests 120
saganite.bsky.social @saganite.bsky.social · 15/04/2025slow boring is full of deeply researched, well reasoned analyses of a bunch of complex issues, and in general really, REALLY does focus on pragmatic interpretations of the problems it investigates and proposes practical potential solutions to the problems it identifies. 120
saganite.bsky.social @saganite.bsky.social · 15/04/2025it's kind of a joke, relating the job of trying to get coherent and meaningful results out of a probabilistic model to that of trying to control a herd of cattle. 000
saganite.bsky.social @saganite.bsky.social · 15/04/2025God, I was really, really trying to switch to bsky because I despise Elon musk and don't want to support his platform, but my god these comments show me that most of the users on here are such awful, miserable, juvenile people. 130
saganite.bsky.social @saganite.bsky.social · 20/03/2025@charles-irl.bsky.social are the modal api docs available anywhere as a machine readable markdown file? if they are i can't find it. 000
