Neil Traft @ntraft.bsky.social · 04/10/2026... and that ought to be very clarifying, b/c that's simply how the world works. But it feels like a lot of x-risk philosophy and research is framed around the hope of providing guarantees in the model weights, rather than appreciating that safety is a systems practice—just as it always has been. 110
Reposted by Neil TraftStella Biderman @stellaathena.bsky.social · 03/10/2026I’m starting a blog! My first post is on how 3rd party embedded evaluators seem totally unsuited to addressing the problems we are currently facing, and what the real problem is. stellabiderman.ai/blog/embedde...stellabiderman.aiEmbedded Evaluators Can’t Fix Companies That Choose to Be Bad — Stella BidermanEmbedded evaluators can report violations, but they cannot fix AI companies that knowingly disregard basic cybersecurity and safety practices. 37817
Neil Traft @ntraft.bsky.social · 04/10/2026I just had an odd thought: AI "alignment" is like the Halting Problem. Alignment at model training time is obviously not possible, in the same way that a halting decider is not possible: you can't magically know ahead of time whether or not an algorithm will "make a boo-boo". 130
Neil Traft @ntraft.bsky.social · 04/10/2026This is the first time I'm hearing of these! Incredible that people are automatically treating the models as accountability sinks, rather than the company! What is going on?? Am I the crazy one here?! 010
Neil Traft @ntraft.bsky.social · 04/10/2026I was quite bothered that Huggingface didn't sue or seek reparations or anything (afaik?) upon the original event. It's so backwards to think we can guarantee security at model training time (a.k.a. alignment), rather than at deployment time. Security can only come from making ppl accountable. 100
Reposted by Neil TraftMark Riedl @markriedl.bsky.social · 03/10/2026Screw-ups should be costly. There should be a high government fine on top of it, plus paying to clean-up and re-secure the systems hacked. These are equivalent to industrial accidents and should be treated as such, imo 29718
Reposted by Neil TraftMark Riedl @markriedl.bsky.social · 26/09/2026Astra beats NetHack: kenforthewin.github.io/blog/posts/l...kenforthewin.github.ioAn LLM Beat NetHackTo my knowledge, the first recorded LLM-agent NetHack ascension 2172
Neil Traft @ntraft.bsky.social · 01/10/2026Modifying an NCA phenotype with LoRA adapters (showing there are low-dimensional modifications that can scale/skew the resulting image). Bringing together two of my favorite AI concepts, NCAs and PEFT (parameter efficient fine-tuning). 🙂 010
Reposted by Neil TraftEugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 30/09/2026In our Nature paper, we introduce the first superhuman Stratego AI, which we built using general techniques that we developed for RL & test-time compute under imperfect information. www.nature.com/articles/s41...nature.comScalable decision-making for games of imperfect information - NatureAtaraxos, an AI for the board wargame Stratego, establishes a design pattern for reinforcement learning and search that is effective under large amounts of hidden information, a longstanding desiderat... 1225960
Reposted by Neil TraftAbel @abeliansoup.bsky.social · 16/03/2026I asked Claude who'd be dumb enough to invent a dumb fruit like guava and it made me this 5365
Neil Traft @ntraft.bsky.social · 30/09/2026I think there's other possible explanations... Perhaps something funny going on with tokenization, or numbers in groups of 3 are easier for the LLM to align for some reason... 021
Reposted by Neil TraftPhilip Ball @philipcball.bsky.social · 30/09/2026"Despite the seemingly novel power these tech oligarchs wield in the space sector, their position today does not reflect a break from the space industry of the past but is a direct result of foundational decisions made at the origins of the US space programme." www.tandfonline.com/doi/full/10....tandfonline.comDictators of the future: the tech oligarchy in outer spacePublished in Science as Culture (Vol. 35, No. 3, 2026) 0102
Neil Traft @ntraft.bsky.social · 30/09/2026"According to the tenets of rational choice theory, you can answer [any question] by forecasting a dollar value and probability of every outcome. … This is ludicrous if you think about it for two seconds. And yet it’s become a standard social convention..." www.argmin.net/p/pricing-co...argmin.netPricing CommensurabilityOn the origins of cost-benefit analyses in governmental decision making 010
Neil Traft @ntraft.bsky.social · 28/08/2026Hilarious anecdote at the end of "Patterns, Predictions, and Actions": Since an income tax appeared too unpopular, King William III created a property tax based on the number of windows in one's house. This worked at first... until people started to brick up their windows. 🙈 000
Reposted by Neil TraftStella Biderman @stellaathena.bsky.social · 10/06/2026A common issue with position papers is that they leave the reader wondering “okay, but what should I actually do”? To address this we provide open problems on a wide variety of topics throughout to illustrate our perspectives and guide future research 2102
Neil Traft @ntraft.bsky.social · 06/01/2026Happy New Year! Here's some microorganism porn. vimeo.com/1024357218 (srsly though. it's mesmerizing.)vimeo.comPortraying micro lifeThis video is a compilation of movies of microorganisms I've made the past decade. In 2023 it was 300 years ago that the father of microscopy Antoni van Leeuwenhoek… 001
Reposted by Neil TraftSeva @seva.bsky.social · 05/01/2026claude code is fucking insane i know literally NOTHING about Hegel. ZERO. and it just built me a complete system of German idealism 26950141
Neil Traft @ntraft.bsky.social · 06/01/2026Fascinating story: 1. Fear-mongering about peanut allergies multiplies in the late 90s. 2. AAUP advises "no peanuts til age 3", despite NO evidence. 3. This advice CAUSES a massive spike in peanut allergies! 4. They had to do a large study just to undo the damage & convince everyone it was safe! 021
Neil Traft @ntraft.bsky.social · 29/12/2025Astonishing: the default behavior of Chrome's "Listen to this page" feature is NOT to simply read you the page, LIKE YOU WOULD EXPECT, but instead to make one of those goofy NotebookLM AI podcasts... and I had to search the Help docs to figure out how to change it. Thanks, Google. 000
Reposted by Neil TraftEugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 24/12/2025MAORL (Multi-Agent Olof RL) arxiv.org/abs/2512.16705 1322
Neil Traft @ntraft.bsky.social · 19/12/2025Gov't labs can't say "climate change" in papers, "climate security" is allowed. Maybe those planes that drop sensors into the hurricane can be "cyclone bombers", forecasting is "climate AI", and cap & trade should be called "CarbonCoin". All key tools in the "War on Climate". 020
Neil Traft @ntraft.bsky.social · 19/12/2025This is a travesty!! Also I have questions... for the branches that aren't terminated, where will they move? Will they be in the same building, under a different name? In which case, will it all be functionally the same thing (but with lots of political disruption)? 100
Neil Traft @ntraft.bsky.social · 16/12/2025Counterpoint: it's the only place where I can actually reliably see posts from people I personally know. (But this may just be due to the low volume of posts... ?) 000
Neil Traft @ntraft.bsky.social · 16/12/2025From David Krakauer's introduction to the Foundational Papers in Complexity Science. www.foundationalpapersincomplexityscience.orgfoundationalpapersincomplexityscience.orgFoundational Papers in Complexity Science 000
Neil Traft @ntraft.bsky.social · 16/12/2025Random observation: did you know this?! I knew about how Einstein's general relativity is required to make GPS navigation work, but I'm not as familiar with how quantum mechanics factors into semiconductor technology. Remarkable. 220
Neil Traft @ntraft.bsky.social · 11/12/2025The knowledge is actually the far more valuable product. But we have trouble funding that on its own. It piggybacks on the funding of new tech inventions. If AI can invent new tech without teaching us, we stop investing in furthering our knowledge, which will ultimately lead to stagnation. 010
Neil Traft @ntraft.bsky.social · 11/12/2025I think you're on the right track, but it doesn't even need to be so philosophical— Current AI4Science is all about *prediction* without *understanding*. This erodes the implicit contract currently in place, where society pays for new products, and they get knowledge as a byproduct. 110
Neil Traft @ntraft.bsky.social · 11/12/2025That's why I appreciate @togelius.bsky.social for raising this point. It's really hard for dissenting voices to speak up in that environment. Even though his argument seems to have sown a lot of confusion. I disagree that fully-automated solutions are superior. This is an assumption, not a given. 010
Reposted by Neil TraftHagen Blix @hagenblix.bsky.social · 17/11/2025In applying AI to material science, biology, etc, capitalism is trying to shed science. The point is to substitute the engineering of a machine that can generate what science has hitherto done, but without having people know things. Knowledge ultimately residing in private property is the dream.nytimes.comJeff Bezos Creates A.I. Start-Up Where He Will Be Co-Chief Executive 8427174
Neil Traft @ntraft.bsky.social · 06/12/2025[In considering candidates] "I actually view publication volume completely negatively... I've learned, from experience, that those with a really high volume [don't have a depth of knowledge]." —@abeirami.bsky.social #slowscience FTW 010
Neil Traft @ntraft.bsky.social · 06/12/2025I thought this was a fascinating way to motivate model merging: to be able to *interpolate* between model classes, rather than be forced into discrete choices. (models trained on different datasets, models of different sizes, etc.) "Interpolation and alignment are two sides of the same coin." 042
Neil Traft @ntraft.bsky.social · 06/12/2025Did you know that you could create hybrids across different neural net architectures—*without* updating the weights?? Come check out my poster at the UniReps @unireps.bsky.social workshop at 3:45pm today at #NeurIPS! (Or you can already browse the posters on the wall in 20D throughout the day.) 😃 041
Reposted by Neil TraftUniReps @unireps.bsky.social · 03/12/2025Join us this Saturday at @neuripsconf.bsky.social 2025 for the @unireps.bsky.social Workshop. 132
Neil Traft @ntraft.bsky.social · 03/12/2025Me, at people with #NeurIPS badges who aren't hurrying frantically: "Don't they know that Rich Sutton is speaking IMMINENTLY??? 😨" 000
Neil Traft @ntraft.bsky.social · 03/12/2025Me, at people with #NeurIPS badges walking away from the convention center right now: "Don't they know that Rich Sutton is speaking in 10 minutes??? 😨" 100
Neil Traft @ntraft.bsky.social · 03/12/2025FWIW mine has the same ones, and I am going to both of those. 🤷🏻♂️ Also has a grad hat which presumably refers to my student registration. 010
Reposted by Neil TraftUniReps @unireps.bsky.social · 01/12/2025🔵🔴 Join us for the UniReps Workshop: Unifying Representations in Neural Models at @neuripsconf.bsky.social 2025! 📍 Ballroom 20D, San Diego Convention Center Dec 6 Don’t forget to fill out the participation form. Joining in person or remotely? We welcome your questions for the panel. 🔗 unireps.org 0125
Neil Traft @ntraft.bsky.social · 01/12/2025Link to the full paper: openreview.net/pdf?id=e4wKQ... I think there's a lot of very interesting potential applications of "model stitching". Have a look. Plus, you get to feel like a mad scientist. IT'S ALIVE! 010
Neil Traft @ntraft.bsky.social · 01/12/2025I’ll have a poster at @unireps.bsky.social 🔵🔴 It’s a terrific workshop at the intersection of neuroscience and DL: across both biological and artificial NNs, how can we measure / compare / align / merge neural representations? Should be fascinating! Check it out if you're around! unireps.org/2025 101
Neil Traft @ntraft.bsky.social · 01/12/2025So stoked to be going to my first #NeurIPS!! It’s crazy that in 10+ years of robotics and AI, I've never been to the great Lollapalooza of Machine Learning. 🥳🎆 I’m presenting my Frankensteinian efforts to stitch together parts of different neural networks! 🧪 120
Neil Traft @ntraft.bsky.social · 25/11/2025What is imagination or daydream in this framework? What is happening when I imagine a cup? 100
Reposted by Neil Traftshimon8282.bsky.social @shimon8282.bsky.social · 22/11/2025Check out our new work on using low-rank perturbations to make evolution strategies work for billion-parameter models. 0102
Reposted by Neil TraftUniReps @unireps.bsky.social · 17/11/2025The UniReps Workshop accepted papers are out! 🎉 Huge thanks to the authors, reviewers, and incredible AC committee for their dedication and effort in making this happen. 🙏 openreview.net/group?id=Neu...openreview.netNeurIPS 2025 Workshop UniRepsWelcome to the OpenReview homepage for NeurIPS 2025 Workshop UniReps 083
Reposted by Neil TraftAlex Komoroske @komorama.bsky.social · 05/11/2025What if instead of buying software from a store, you could grow it in your garden? 052
Neil Traft @ntraft.bsky.social · 04/11/2025What an incredible tool! I think I'll be returning to this a lot over the next month! The most interesting papers might be the ones that seem misclassified or out of place... 020
Neil Traft @ntraft.bsky.social · 02/11/2025It's too easy to write garbage survey papers and get lots of citations. Over time, I've learned to ignore most survey papers and recognize some markers of quality. 120