iiinova — where your digital self keeps evolving

Nishant Balepur @nbalepur.bsky.social · 21/05/2025

🎉🎉 Excited to have two papers accepted to #ACL2025! Our first paper designs a preference training method to boost LLM personalization 🎨 While the second outlines our position on why MCQA evals are terrible and how to make them better 🙏 Grateful for amazing collaborators!

060

Reposted by Nishant Balepur

Abhilasha Ravichander @lasha.bsky.social · 21/03/2025

Want to know what training data has been memorized by models like GPT-4? We propose information-guided probes, a method to uncover memorization evidence in *completely black-box* models, without requiring access to 🙅‍♀️ Model weights 🙅‍♀️ Training data 🙅‍♀️ Token probabilities 🧵 (1/5)

arxiv.org

Information-Guided Identification of Training Data Imprint in (Proprietary) Large Language Models

High-quality training data has proven crucial for developing performant large language models (LLMs). However, commercial LLM providers disclose few, if any, details about the data used for training. ...

49627

Reposted by Nishant Balepur

Ryan Heuser @ryanheuser.com · 21/03/2025

Finally may have figured out why LLMs rhyme so compulsively: instruction-tuning. Training an LLM to respond "helpfully" to user queries may push models into more "pleasing" aesthetic forms.

Graph showing that simple text completion models more accurately imitate the unrhymed form of C20 verse, whereas instruction-tuned models lapse into rhyme more often.

Caption to graph: Given the first 5 lines of 10-20 line poems from poets born in each century, 1600-2000, LLMs are prompted to "complete" the poem. Rhyme is measured by exact phoneme match in the rime of the final syllable (or syllables, if final syllable unstressed). Poems randomly sampled from Chadwyck-Healey poetry collections, with 600 poems for each model for each century. Results shown for actual poems as well as the LLM imitations. Poems "memorized" by the model are excluded.

3298

Nishant Balepur @nbalepur.bsky.social · 21/03/2025

Had a great time presenting my research on building more helpful QA systems @imperialcollegeldn.bsky.social! Thank you @joestacey.bsky.social for letting me invite myself 🫶 And loved visiting London+Edinburgh this week, hope to be back soon! 🙏

051

Nishant Balepur @nbalepur.bsky.social · 11/03/2025

🚨 Our team at UMD is looking for participants to study how #LLM agent plans can help you answer complex questions 💰 $1 per question 🏆 Top-3 fastest + most accurate win $50 ⏳ Questions take ~3 min => $20/hr+ Click here to sign up (please join, reposts appreciated 🙏): preferences.umiacs.umd.edu

023

Reposted by Nishant Balepur

Nishant Balepur @nbalepur.bsky.social · 24/02/2025

🚨 New Position Paper 🚨 Multiple choice evals for LLMs are simple and popular, but we know they are awful 😬 We complain they're full of errors, saturated, and test nothing meaningful, so why do we still use them? 🫠 Here's why MCQA evals are broken, and how to fix them 🧵

24612

Reposted by Nishant Balepur

Chau Minh Pham @chautmpham.bsky.social · 21/02/2025

⚠️Current methods for generating instruction-following data fall short for long-range reasoning tasks like narrative claim verification. We present CLIPPER ✂️, a compression-based pipeline that produces grounded instructions for ~$0.5 each, 34x cheaper than human annotations.

1218

Nishant Balepur @nbalepur.bsky.social · 24/02/2025

24612

Nishant Balepur @nbalepur.bsky.social · 31/01/2025

Excited to share 2 papers at #NAACL2025 main! 📄✍️ MoDS: Multi-Doc Summarization for Debatable Queries (Adobe intern work, coming soon!) 🤔❓Reverse QA: LLMs struggle with the simple task of giving questions for answers Grateful for all my collaborators 😁

151

Reposted by Nishant Balepur

Jenna Russell @jennarussell.bsky.social · 28/01/2025

People often claim they know when ChatGPT wrote something, but are they as accurate as they think? Turns out that while general population is unreliable, those who frequently use ChatGPT for writing tasks can spot even "humanized" AI-generated text with near-perfect accuracy 🎯

1018966

Nishant Balepur @nbalepur.bsky.social · 15/12/2024

Manifesting some good luck for my experiment running tonight 🤞 Best of luck to anyone submitting tmrw :)

030

Reposted by Nishant Balepur

UMD Language Science Center @umd-lsc.bsky.social · 03/12/2024

Exciting research on an AI-driven mnemonic generator for easier vocabulary memorization by @nbalepur.bsky.social, Jordan Boyd-Graber, Rachel Rudinger, & @alexanderhoyle.bsky.social. Part of 21 CLIP projects at #EMNLP2024. 👉 Read more: go.umd.edu/1u48 #AI

031

Reposted by Nishant Balepur

Luca Soldaini 🎀 @soldaini.net · 26/11/2024

OLMo 2 is out 🥳 7B and 13B trained on 5T tokens, and meticulousy instruction tuned using Tulu 3 recipe. Simply the best fully open models yet. Really proud of the work & the amazing team at @ai2.bsky.social

926044