NDIF Team @ndif-team.bsky.social · 04/06/2026Can you tell when an AI model is lying? Announcing Aletheia's Quest, an AI lie detection challenge running this summer, organized by Cadenza Labs and NDIF. Multiple model organisms to interrogate and probe, $50K prize pool, no local GPU required. 183
NDIF Team @ndif-team.bsky.social · 27/02/2026NNsight 0.6 is out now! We directly address your feedback in our biggest release yet. Pain points included cryptic errors, slow traces, no remote execution of custom code, and limited vLLM support. We tackle all of these and more in this new release. 🧵 Here's what changed: 172
NDIF Team @ndif-team.bsky.social · 10/02/2026Watch Sam Marks present his work "Auditing Language Models for Hidden Objectives" in our new YouTube video! Sam's team ran a blind auditing game to assess efficacy of black box and white box techniques for LLM alignment auditing. 🔗 youtu.be/jZiOJTHqB6Myoutube.comAuditing Language Models for Hidden Objectives with Sam MarksSam Marks leads Anthropic's Cognitive Oversight team, a subteam of Alignment Science. Sam's research focuses on settings where understanding something about ... 130
NDIF Team @ndif-team.bsky.social · 04/02/2026Check out the NDIF & Bau Lab lightning talks at the NeurIPS 2025 Mechanistic Interpretability Workshop (mechinterpworkshop.com/): youtube.com/playlist?li...youtube.comNeurIPS 2025 Mechanistic Interpretability WorkshopTalks from NDIF and Bau Lab at the NeurIPS 2025 Mech Interp Workshop 141
NDIF Team @ndif-team.bsky.social · 28/01/2026New YouTube video posted! @benjaminfeuer.bsky.social discusses LLM's annus mirabilis, presenting his work on open questions surrounding LLM judges, benchmark trustworthiness, and maximizing the potential of synthetic data. Watch here: www.youtube.com/watch?v=pehc...youtube.comAnnus Mirabilis: A Year of Explosive Progress in LLMs with Benjamin FeuerYouTube video by NDIF Team 020
Reposted by NDIF TeamAntonin Poché @antoninpoche.bsky.social · 20/01/2026🔥I am super excited for the official release of an open-source library we've been working on for about a year! 🪄interpreto is an interpretability toolbox for HF language models🤗. In both generation and classification! Why do you need it, and for what? 1/8 (links at the end) 1229
NDIF Team @ndif-team.bsky.social · 15/01/2026New year, new YouTube videos! We are resuming our regular interpretability seminar posts, with a fantastic talk by Deepti Ghadiyaram on interpreting diffusion models. Watch the video: youtu.be/4eqvABPX5rAyoutube.comInterpreting and Leveraging Diffusion Representations with Deepti GhadiyaramDeepti Ghadiyaram is an Assistant Professor at Boston University in the Department of Computer Science, with affiliated appointments in Electrical and Comput... 153
NDIF Team @ndif-team.bsky.social · 09/01/2026nnterp by @butanium.bsky.social is now part of the NDIF ecosystem! nnterp standardizes transformer naming conventions, includes built-in best practices for common interventions, and is perfectly compatible with original HF model implementations. Learn more: ndif-team.github.io/nnterp/ 152
NDIF Team @ndif-team.bsky.social · 19/12/2025Happy Holidays from NDIF! Our new NNsight version improves performance and enhances vLLM integration, including support for tensor parallelism. 153
Reposted by NDIF TeamDavid Bau @davidbau.bsky.social · 11/10/2025What does an LLM do when it translates from Italian "amore" to Spanish "amor" or French "amour"? That's easy! (you might think) Because surely it knows: amore, amor, amour are all based on the same Latin word. It can just drop the "e", or add a "u". 2374
Reposted by NDIF TeamGabriele Sarti @gsarti.com · 10/10/2025Making model internals accessible to domain experts in low-code interfaces will unlock the next step in making interpretability useful across a variety of domains. Very excited about the NDIF Workbench! 💡 0111
NDIF Team @ndif-team.bsky.social · 10/10/2025Ever wished you could explore what's happening inside a 405B parameter model without writing any code? Workbench, our AI interpretability interface, is now live for public beta at workbench.ndif.us! 173
NDIF Team @ndif-team.bsky.social · 03/10/2025New YouTube video posted! @wendlerc.bsky.social presents his work using SAEs for diffusion text-to-image models. The authors find interpretable SAE features and demonstrate how these features can alter generated images. Watch here: youtu.be/43NnaqGjArAyoutube.comInterpreting SDXL Turbo Using Sparse Autoencoders with Chris WendlerIn this talk, Chris Wendler presents his recent work on using sparse autoencoders for diffusion models. In this work, they train SAEs on SDXL Turbo, finding ... 141
NDIF Team @ndif-team.bsky.social · 01/10/2025Reminder that today is the deadline to apply for our hot-swapping program! Be the first to test out many new models remotely on NDIF and submit your application today! More details: ndif.us/hotswap.html Application link: forms.gle/KHVkYxybmK12...ndif.usNSF National Deep Inference FabricNDIF is a research computing project that enables researchers and students to crack open the mysteries inside large-scale AI systems. 010
NDIF Team @ndif-team.bsky.social · 26/09/2025Want increased remote model availability on NDIF? Interested in studying model checkpoints? Sign up for the NDIF hot-swapping pilot by October 1st: forms.gle/Cf4WF3xiNzud...forms.gleNDIF Hot-swapping Beta TestingDo you have a research project where you plan to study many different models? NDIF will soon be deploying model hot-swapping, which will enable users to access any HuggingFace model remotely via NDIF.... 030
NDIF Team @ndif-team.bsky.social · 04/09/2025Do you wish you could run experiments on any model remotely from your laptop? In a future release, NDIF users will be able to dynamically deploy any model from HuggingFace on NDIF for remote experimentation. But before this, we need your help! 141
NDIF Team @ndif-team.bsky.social · 22/08/2025We are presenting our NNsight / NDIF demos at NEMI now! Tune in: youtube.com/live/q8Su4C...youtube.comNew England Mechanistic Interpretability Workshop09:30 AM - 09:40 AM: Opening Remarks (David Bau) 09:40 AM - 10:00 AM: Keynote 1: Lee Sharkey: "Mech Interp: Where should we go from here?" 10:00 AM - 10:10 A... 030
NDIF Team @ndif-team.bsky.social · 22/08/2025The NEMI conference is live! Watch our livestream here: youtube.com/live/q8Su4C...youtube.comNEMI 2025 010
Reposted by NDIF TeamDavid Bau @davidbau.bsky.social · 18/08/2025This Friday NEMI 2025 is at Northeastern in Boston, 8 talks, 24 roundtables, 90 posters; 200+ attendees. Thanks to goodfire.ai/ for sponsoring! nemiconf.github.io/summer25/ If you can't make it in person, the livestream will be here: www.youtube.com/live/4BJBis...youtube.comNew England Mechanistic Interpretability WorkshopAbout:The New England Mechanistic Interpretability (NEMI) workshop aims to bring together academic and industry researchers from the New England and surround... 1167
Reposted by NDIF TeamDavid Bau @davidbau.bsky.social · 18/08/2025Announcing a deep net interpretability talk series! Every week you will find new talks on recent research in the science of neural networks. The first few are posted: jackmerullo.bsky.social, Roy Rinberg, and me. At the @ndif-team.bsky.social Youtube Channel: www.youtube.com/@NDIFTeamyoutube.comNDIF TeamWe're a research computing project cracking open the mysteries inside large-scale AI systems. The NSF National Deep Inference Fabric consists of a unique combination of hardware and software that provides a remotely-accessible computing resource for scientists and students to perform detailed and reproducible experiments on large pretrained AI models, such as open large language models. We aim to make AI interpretability research more accessible through this channel by publishing lectures and educational content covering real interpretability research. 0115
NDIF Team @ndif-team.bsky.social · 07/08/2025Our YouTube channel is live! Our first video features @davidbau.bsky.social presenting the ROME project: www.youtube.com/watch?v=eKd...youtube.comROME: Locating and Editing Factual Associations in GPT with David BauDavid Bau is an Assistant Professor of Computer Science at Northeastern University's Khoury College. His lab studies the structure and interpretation of deep... 172
NDIF Team @ndif-team.bsky.social · 05/08/2025Induction heads are attention heads that help complete patterns by copying tokens (transformer-circuits.pub/2021/framew...), but can they also copy over concepts? 160
NDIF Team @ndif-team.bsky.social · 19/07/2025Great to present what’s coming next for NDIF at the @actinterp.bsky.social workshop at #ICML2025! If you missed us, let’s chat after the conference. Reach out here: forms.gle/LtTyYnkaxDyg... 080
NDIF Team @ndif-team.bsky.social · 16/07/2025We’re excited to announce a new series of applied "mini paper" tutorials! The goal of this series is to help researchers get hands-on experience with findings, methods, and results from recent papers in interpretability using NNsight and NDIF. 141
NDIF Team @ndif-team.bsky.social · 04/07/2025Excited to share our first paper replication tutorial, walking you through the main figures from "Do Language Models Use Their Depth Efficiently?" by @robertcsordas.bsky.social 🔎 Demo on Colab: colab.research.google.com/github/ndif-... 📖 Read the full manuscript: arxiv.org/abs/2505.13898colab.research.google.comGoogle Colab 051
NDIF Team @ndif-team.bsky.social · 20/06/2025We are prereleasing NNsight 0.5 today! We've added many highly-requested new features, including easier access to intermediate values within the forward pass, ability to run any code within NNsight’s tracing context (no more proxies!), and easier debugging. 142
NDIF Team @ndif-team.bsky.social · 22/05/2025Are you interested in researching diffusion models in NNsight? We've released our first non-LLM tutorial, which covers the Diffusion Lens technique! 120
NDIF Team @ndif-team.bsky.social · 25/04/2025At #ICLR2025 in Singapore? Come visit our poster tomorrow (Saturday, April 26th)! This work addresses a critical gap in ML research: the ability to study the internals of very large neural networks. 161
Reposted by NDIF TeamDavid Bau @davidbau.bsky.social · 16/03/2025What will be the linchpin for AI dominance? Read our NSF/OSTP recommendations written with Goodfire's Tom McGrath tommcgrath.github.io, Transluce's Sarah Schwettmann cogconfluence.com, MIT's Dylan Hadfield-Menell @dhadfieldmenell.bsky.social TLDR; Dominance comes from **interpretability** 🧵 ↘️ 1218
NDIF Team @ndif-team.bsky.social · 20/02/2025Please help amplify ARBOR, a fantastic new research opportunity! If you’d like to start contributing, NDIF is now hosting DeepSeek R1 8B and 70B, open for all researchers to experiment on via our API. Sign up for API access here: login.ndif.uslogin.ndif.usNDIF Registration 043
Reposted by NDIF TeamMartin Wattenberg @wattenberg.bsky.social · 20/02/2025Today we're launching a multi-lab open collaboration, the ARBOR project, to accelerate AI interpretability research for reasoning models. Please join us! github.com/ARBORproject... (ARBOR = Analysis of Reasoning Behavior through Open Research)github.comGitHub - ARBORproject/arborproject.github.ioContribute to ARBORproject/arborproject.github.io development by creating an account on GitHub. 1449
Reposted by NDIF TeamCan @canrager.bsky.social · 20/02/2025Announcing ARBOR, an open research community for collectively understanding how reasoning models like openai-o3 and deepseek-r1 work. We invite all researchers and enthusiasts to this initiative by @wattenberg.bsky.social's and @davidbau.bsky.social's lab. arborproject.github.io 161
NDIF Team @ndif-team.bsky.social · 06/02/2025The application deadline for our summer engineering fellowship is tomorrow! Don't forget to apply here: ndif.us/fellowship.h... 010
NDIF Team @ndif-team.bsky.social · 03/02/2025New version alert! We've officially released NNsight 0.4, which you can now install with pip. If you haven’t yet tried out NNsight 0.4, here’s what’s new: 152
NDIF Team @ndif-team.bsky.social · 31/01/2025Thanks for the mention, @thebulletin.org! Great article on the importance of AI interpretability, highlighting the need for frameworks that make AI more transparent and help democratize research capabilities: thebulletin.org/2025/01/why-...thebulletin.orgWhy nobody can see inside AI’s black boxAI companies don't fully understand how their models work and they aren’t making the training data available to outside researchers. 061
Reposted by NDIF TeamDavid Bau @davidbau.bsky.social · 31/01/2025DeepSeek R1 shows how important it is to be studying the internals of reasoning models. Try our code: Here @canrager.bsky.social shows a method for auditing AI bias by probing the internal monologue. dsthoughts.baulab.info I'd be interested in your thoughts.dsthoughts.baulab 1289
NDIF Team @ndif-team.bsky.social · 23/01/2025🔔 We have new features coming to #NNsight soon! NNsight 0.4 will be released on January 31st. Stay tuned and join our Discord for the latest updates: lnkd.in/eJkRGvGB. 👀 For a sneak peek at the new features, check out our walkthrough: colab.research.google.com/drive/1TtYi5... 041
NDIF Team @ndif-team.bsky.social · 21/01/2025Calling all students looking for summer opportunities! Apply to our Summer Engineering Fellowship today: ndif.us/fellowship.h... 031
NDIF Team @ndif-team.bsky.social · 10/01/2025Spend the summer with us! We're still accepting applications for our Summer Engineering Fellowship. Seeking candidates to help with full-stack tool development, systems reliability, and outreach and education! Apply by Feb 7, 2025 at ndif.us/fellowship.h... 061
NDIF Team @ndif-team.bsky.social · 10/01/2025Check out this great demo implementing activation steering using #NNsight by @canrager.bsky.social! 060
NDIF Team @ndif-team.bsky.social · 31/12/2024⏳ Final call! Applications for NDIF's 405b pilot program close tonight! Don’t miss the opportunity to experiment with Llama-405b and shape groundbreaking AI research infrastructure. Details below 🧵⬇️ 142
NDIF Team @ndif-team.bsky.social · 27/12/2024🚨 Reminder: Applications for NDIF’s 405b pilot program close on December 31st, 2024! Don’t miss your chance to run experiments on the world’s largest open-sourced model, Llama 405b. 🌍 We host the model, you do the science! Apply: 👉 ndif.us/405b.html 020
NDIF Team @ndif-team.bsky.social · 20/12/2024Large language models show fascinating changes in capability with scaling parameters, but scaling also vastly increases the resources required for experimentation on model internals. NDIF is currently hosting the largest open sourced model, Llama 405b, for YOU to run research on! 152
NDIF Team @ndif-team.bsky.social · 10/12/2024More big news! Applications are open for the NDIF Summer Engineering Fellowship—an opportunity to work on cutting-edge AI research infrastructure this summer in Boston! 🚀 196
NDIF Team @ndif-team.bsky.social · 09/12/2024Do you have a great experiment that you want to run on Llama 405b but not enough GPUs? 🚨 #NDIF is opening up more spots in our 405b pilot program! Apply now for a chance to conduct your own groundbreaking experiments on the 405b model. Details: 🧵⬇️ 1184