Sign in

Mimansa Jaiswal

@mimansaj.bsky.social
1.8K followers 1.6K following 142 posts

Robustness, Data & Annotations, Evaluation & Interpretability in LLMs mimansajaiswal.github.io

PostsRepliesMedia
Reposted by Mimansa Jaiswal
Steve Klabnik @steveklabnik.com · 28/05/2025
I am disappointed in the AI discourse steveklabnik.com/writing/i-am...
steveklabnik.com
I am disappointed in the AI discourse
209911178
Reposted by Mimansa Jaiswal
Jeremy Morrell @jeremymorrell.dev · 05/04/2025
Meta introduced Llama 4 models and added this section near the very bottom of the announcement 😬 “[LLMs] historically have leaned left when it comes to debated political and social topics.” ai.meta.com/blog/llama-4...
Meta
Addressing bias in LLMs

It's well-known that all leading LLMs have had issues with bias-specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet.

Our goal is to remove bias from our Al models and to make sure that Llama can understand and articulate both sides of a contentious issue. As part of this work, we're continuing to make Llama more responsive so that it answers questions, can respond to a variety of different viewpoints without passing judgment, and doesn't favor some views over others.

We have made improvements on these efforts with this release—Llama 4 performs significantly better than Llama 3 and is comparable to Grok:• Llama 4 refuses less on debated political and social topics overall (from 7% in Lama 3.3 to below 2%).
• Llama 4 is dramatically more balanced with which prompts it refuses to respond to (the proportion of unequal response refusals is now less than 1% on a set of debated topical questions).
• Our testing shows that Llama 4 responds with strong political lean at a rate comparable to Grok (and at half of the rate of Llama 3.3) on a contentious set of political or social topics. While we are making progress, we know we have more work to do and will continue to drive this rate further down.
We're proud of this progress to date and remain committed to our goal of eliminating overall bias in our models.
513438
Reposted by Mimansa Jaiswal
Joe Littrell @gentlemanjoe.bsky.social · 06/04/2025
"We train our LLMs on art and literature and educational materials, and for some reason they keep turning out progressive."
011914
Reposted by Mimansa Jaiswal
Sarah Wiegreffe @sarah-nlp.bsky.social · 03/04/2025
Have work on the actionable impact of interpretability findings? Consider submitting to our Actionable Interpretability workshop at ICML! See below for more info. Website: actionable-interpretability.github.io Deadline: May 9
02010
Reposted by Mimansa Jaiswal
Somnath Basu Roy Chowdhury @somnathbrc.bsky.social · 02/04/2025
𝐇𝐨𝐰 𝐜𝐚𝐧 𝐰𝐞 𝐩𝐞𝐫𝐟𝐞𝐜𝐭𝐥𝐲 𝐞𝐫𝐚𝐬𝐞 𝐜𝐨𝐧𝐜𝐞𝐩𝐭𝐬 𝐟𝐫𝐨𝐦 𝐋𝐋𝐌𝐬? Our method, Perfect Erasure Functions (PEF), erases concepts perfectly from LLM representations. We analytically derive PEF w/o parameter estimation. PEFs achieve pareto optimal erasure-utility tradeoff backed w/ theoretical guarantees. #AISTATS2025 🧵
2378
Reposted by Mimansa Jaiswal
Sian Gooding @siangooding.bsky.social · 02/04/2025
New paper from our team @GoogleDeepMind! 🚨 We've put LLMs to the test as writing co-pilots – how good are they really at helping us write? LLMs are increasingly used for open-ended tasks like writing assistance, but how do we assess their effectiveness? 🤔 arxiv.org/pdf/2503.19711
arxiv.org
1208
Mimansa Jaiswal @mimansaj.bsky.social · 28/03/2025
I can help! (Might end up being by 4am tomorrow if that works for you)
120
Reposted by Mimansa Jaiswal
SE Gyges @segyges.bsky.social · 23/03/2025
pre aca you would specifically avoid being diagnosed or seeking treatment if you didn't have health insurance to prevent it from making it impossible for you to get health insurance. when you bought health insurance after doing this you committed fraud. i did this.
67810
Reposted by Mimansa Jaiswal
Jay Rosen @jayrosen.bsky.social · 07/03/2025
Some of his readers have asked Mike Masnick @mmasnick.bsky.social why his technology news site, Tech Dirt, has been covering politics so intensely lately. www.techdirt.com/2025/03/04/w... I cannot recommend Mike's reply enough. It's exactly what readers need to hear, what journalists need to do.
8645421810
Reposted by Mimansa Jaiswal
Martin Wattenberg @wattenberg.bsky.social · 25/02/2025
Neat visualization that came up in the ARBOR project: this shows DeepSeek "thinking" about a question, and color is the probability that, if it exited thinking, it would give the right answer. (Here yellow means correct.)
68116
Mimansa Jaiswal @mimansaj.bsky.social · 25/02/2025
I think I did 6 hours a day for a week for leetcode. Looking back (I do mention it in the post), but learning the exact code for binary search, graph components, and coding attention blocks -- they were used so often and memorizing them (rather than writing them on the fly) helped down the line.
000
Reposted by Mimansa Jaiswal
Nathan Lambert @natolambert.bsky.social · 25/02/2025
Come work with me! We are looking to bring on more top talent to our language modeling workstream at @ai2.bsky.social building the open ecosystem. We are hiring: * Research scientists * Senior research engineers * Post docs (Young investigators) * Pre docs job-boards.greenhouse.io/thealleninst...
job-boards.greenhouse.io
The Allen Institute for AI
45415
Mimansa Jaiswal @mimansaj.bsky.social · 24/02/2025
I am glad! Suggestions and comments are always welcome :)
000
Mimansa Jaiswal @mimansaj.bsky.social · 24/02/2025
Resources as a notion database: mimansajaiswal-embedded-dbs.notion.site/19223941af7... You can contribute resources at: mimansajaiswal-embedded-dbs.notion.site/19e23941af7... Link to the resources post: mimansajaiswal.github.io/posts/llm-m...
mimansajaiswal.github.io
LLM (ML) Job Interviews - Resources
A collection of all the resources I used to prepare for my ML/LLM research science/engineering focused interviews in Fall 2024.
010
Mimansa Jaiswal @mimansaj.bsky.social · 24/02/2025
You can read more about the process here: mimansajaiswal.github.io/posts/llm-m...
mimansajaiswal.github.io
LLM (ML) Job Interviews (Fall 2024) - Process
A retelling of my experience interviewing for ML/LLM research science/engineering focused roles in Fall 2024.
250
Mimansa Jaiswal @mimansaj.bsky.social · 24/02/2025
I interviewed for LLM/ML research scientist/engineering positions last Fall. Over 200 applications, 100 interviews, many rejections & some offers later, I decided to write the process down, along with the resources I used. Links to the process & resources in the following tweets
OCR'ed text from screenshot of top of post: LLM (ML) Job Interviews (Fall 2024) - Process A retelling of my experience interviewing for ML/LLM research science/engineering focused roles in Fall 2024.  This post has two parts:  Job Search Mechanics (including context, applying, and industry information), which you can continue reading below, and, Preparation Material and Overview of Questions, which you can read at LLM (ML) Job Interviews - Resources  Disclaimer Last Updated:  Dec 24, 2024  This is the process I used, which may work differently for you depending on your circumstances. I am writing this in December 2024, and the process occurred during Fall 2024. Given how rapidly the field of LLMs evolves, this information might become outdated quickly, but the general principles should remain relevant. (more...)  Read at: https://mimansajaiswal.github.io/posts/llm-ml-job-interviews-fall-2024-process/
35511
Reposted by Mimansa Jaiswal
Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 13/02/2025
Obsessed with the work coming out of Finale Doshi-Velez's group; they don't just take the limits of the real world for ML deployment seriously but instead turn it into new algorithmic ideas arxiv.org/abs/2406.08636
arxiv.org
Towards Integrating Personal Knowledge into Test-Time Predictions
Machine learning (ML) models can make decisions based on large amounts of data, but they can be missing personal knowledge available to human users about whom predictions are made. For example, a mode...
0619
Mimansa Jaiswal @mimansaj.bsky.social · 10/02/2025
Ooh, I'd love to read this. I have mine here: mimansajaiswal.github.io/posts/mac-so... . I am currently testing out Atuin. I typically don't add new tools to my workflow until I've used them for at least a year or feel confident that I will continue using them long-term.
mimansajaiswal.github.io
What softwares do I actually use on my Mac as a software enthusiast? • Mimansa Jaiswal
With several years of using a Mac, it took me time to settle down on a set of apps and softwares that I can heartily recommend. I try almost 30 a month, but end up using around 20 in total for everyth...
030
Mimansa Jaiswal @mimansaj.bsky.social · 06/02/2025
Definitely sponsored by Big Cupcake. Did you know Big Cupcake and Big Muffin are different AND competing corporations? 😃
020
Mimansa Jaiswal @mimansaj.bsky.social · 06/02/2025
Reminds me of this solution 🤣: youtu.be/MTQUrUbb8vo?...
youtu.be
Anne Hathaway Shares Genius Cupcake Hack
YouTube video by The Kelly Clarkson Show
120
Reposted by Mimansa Jaiswal
Alt CDC (they/them) @altcdc.altgov.info · 01/02/2025
The entire archive of CDC datasets can be found here. HUGE shoutout to data archivists- this work is important 👏🙌🏻 archive.org/details/2025...
224117754626
Reposted by Mimansa Jaiswal
Ai2 @ai2.bsky.social · 21/01/2025
Can AI really help with literature reviews? 🧐 Meet Ai2 ScholarQA, an experimental solution that allows you to ask questions that require multiple scientific papers to answer. It gives more in-depth and contextual answers with table comparisons and expandable sections 💡 Try it now: scholarqa.allen.ai
Ai2 ScholarQA logo
13412
Mimansa Jaiswal @mimansaj.bsky.social · 21/01/2025
I usually end up going here: whatthefuckjusthappenedtoday.com though @vox.com seems to have a new newsletter too.
whatthefuckjusthappenedtoday.com
Political News for Normal People.
Your essential guide to the shock and awe in national politics. A sane, once-a-day newsletter helping normal people make sense of the news.
120
Mimansa Jaiswal @mimansaj.bsky.social · 21/01/2025
I wrote this a month back, kinda knowing what was coming, hoping for the best, expecting the worst. My cats are wondering why I have been extra cuddly lately 😅
000
Reposted by Mimansa Jaiswal
Ryan Moulton @moultano.bsky.social · 21/01/2025
It is such a slap in the face to the Indian American community to delay their green cards for decades and then declare that because of that delay their American children aren't citizens.
1293
Mimansa Jaiswal @mimansaj.bsky.social · 21/01/2025
EB2 currently has an expected wait time of around 20+ years (around 24+ years after landing in US including H1B+filing time). Even EB1 for indians currently has 4ish years wait time (which you need a 5 year PhD to qualify for, and then an H1B, usually ending up being 10+ years since landing in US) 😅
130
Mimansa Jaiswal @mimansaj.bsky.social · 21/01/2025
Is it time for a social media break again? It has not been a great day. 😅
010
Mimansa Jaiswal @mimansaj.bsky.social · 11/01/2025
Yeah, I do like their rotis. I do not like their parathas or naans though. But they save so much time. Also, apparently vadilal makes great frozen roomali rotis? I just tried them out, and they are awesome!
010
Mimansa Jaiswal @mimansaj.bsky.social · 11/01/2025
Naan tacos? I want to start making more bread honestly, I am not used to it, so I end up eating just the salad, and then I crave chewy carbs 😅 (though haldiram's frozen roomali roti has been helping).
110
Mimansa Jaiswal @mimansaj.bsky.social · 10/01/2025
Tbf the LLM gave me the answer 🫣. I just had to ask it again.
100
Mimansa Jaiswal @mimansaj.bsky.social · 10/01/2025
Here is a video:
240
Mimansa Jaiswal @mimansaj.bsky.social · 10/01/2025
You can use in MacOS osascript -e 'set the clipboard to (POSIX file "/path/to/your/file")'
130
Mimansa Jaiswal @mimansaj.bsky.social · 10/01/2025
Sorry, not into a subscription based recipe app.
010
Mimansa Jaiswal @mimansaj.bsky.social · 08/01/2025
Yeah, I like trying out apps for fun, but I make sure not to use multiple apps, or to replace an app if need be.
000
Mimansa Jaiswal @mimansaj.bsky.social · 06/01/2025
This is pretty cool! Learn more: developer.chrome.com/docs/devtool... (seems to cover CSS and network requests, which might be fun to lay around with)
AI innovations tab in developer tools settings in Chrome
040
Reposted by Mimansa Jaiswal
Yoav Goldberg @yoavgo.bsky.social · 05/01/2025
i was annoyed at having many chrome tabs with PDF papers having uninformative titles, so i created a small chrome extension to fix it. i'm using it for a while now, works well. today i put it on github. enjoy. github.com/yoavg/pdf-ta...
59822
Reposted by Mimansa Jaiswal
Maria Antoniak @mariaa.bsky.social · 31/12/2024
It's ready! 💫 A new blog post in which I list of all the tools and apps I've been using for work, plus all my opinions about them. maria-antoniak.github.io/2024/12/30/o... Featuring @kagi.com, @warp.dev, @paperpile.bsky.social, @are.na, Fantastical, @obsidian.md, Claude, and more.
3621525
Reposted by Mimansa Jaiswal
Maggie Appleton @maggieappleton.com · 31/12/2024
I've always wanted to build things with D3, but the learning curve was too high. At least for the bespoke stuff I wanted to make (not just simple bar charts). I can finally make things like this thanks to Cursor! I just art directed this, and it made everything work beautifully. Even on mobile 🎉
817414
Mimansa Jaiswal @mimansaj.bsky.social · 31/12/2024
Iirc you use Android, if you add your boarding pass to your wallet, many airlines update that and if they don't, Google updates the gate info based off their internal Google flights tracking API too!
110
Mimansa Jaiswal @mimansaj.bsky.social · 31/12/2024
I am surprised you do not use Raycast 😅 At this point, I do not think I can live without raycast 🫣
110
Mimansa Jaiswal @mimansaj.bsky.social · 31/12/2024
Oh, and recs for keyboard: Moonlander (saved my partner from wrist pain). For monitor, you need high ppi (>200) to feel the same as retina monitors. Any monitor in that range will feel similar (including studio display)
110
Mimansa Jaiswal @mimansaj.bsky.social · 31/12/2024
I wish I liked obsidian (I have tried it 10 times till now 😭, I like rich text block based writing). You might want to check out morgen in place of things+fantastical btw.
110
Mimansa Jaiswal @mimansaj.bsky.social · 31/12/2024
Ooh this is fun! I need to check out potato soon (iirc it is developed by people from UMich!). You might like schej instead of when2meet, and you might like spark for email (unsure if it supports microsoft mail), and you might like noor instead of slack (has a generous free plan)
110
Mimansa Jaiswal @mimansaj.bsky.social · 30/12/2024
For me, it is just the presence of most people I care to read opinions of, the timing with which things are tweeted out (for example the deepseek v3 paper details), and existence of an algorithmic feed (I am not a fan of chronological feeds). At this point, I just use it to consume content.
130
Mimansa Jaiswal @mimansaj.bsky.social · 30/12/2024
I really like this! Did you document your LLM conversations somewhere?
110
Mimansa Jaiswal @mimansaj.bsky.social · 30/12/2024
I haven't had words to describe this behavior, but being able to discount general instructions for immediate context and knowing when to perform that operation is one of the things that makes LLM chatbots so helpful, else, we'd be stuck opening 10 different conversations with copy-pasted context.
150
Mimansa Jaiswal @mimansaj.bsky.social · 30/12/2024
Ah, I think I didn't come around to blogs for a long time. I wonder if she wrote a head first LLMs book, what would that look like!
010
Mimansa Jaiswal @mimansaj.bsky.social · 30/12/2024
She is the head first java author? Oh man, I remember seeing the image on the first page and I thought it was one of the illustrations in the book. Her book is how I learnt java in high school in 2009-10 (because unfortunately I am one of those people who likes reading stuff that looks like fun)!
120
Mimansa Jaiswal @mimansaj.bsky.social · 29/12/2024
Collaboration supported (though everyone needs a subscription), apple only, also found on setapp, exports to pdf and muse bundle that can experimentally be hosted on the web. Also, now uses their own storage and sync engine, unlike other options.
000