iiinova — where your digital self keeps evolving

Melanie Sclar @melaniesclar.bsky.social · 22/07/2025

Check out our work on preference modeling through latent (& interpretable) attribute representation learning! PrefPalette allows you to understand _why_ something is preferred and _how_ preference varies depending on context 🎨

040

Reposted by Melanie Sclar

Stella Li @stellali.bsky.social · 22/07/2025

WHY do you prefer something over another? Reward models treat preference as a black-box😶‍🌫️but human brains🧠decompose decisions into hidden attributes We built the first system to mirror how people really make decisions in our recent COLM paper🎨PrefPalette✨ Why it matters👉🏻🧵

172

Melanie Sclar @melaniesclar.bsky.social · 24/04/2025

See our work on procedurally generating challenging reasoning problems on detecting inconsistencies in stories! FlawedFictions is a great example of what I'm most excited about: reliable synthetic data for reasoning in under-explored domains. (I'm at ICLR to chat, DMs open!)

030

Melanie Sclar @melaniesclar.bsky.social · 24/04/2025

Excited to be at #ICLR2025 🤩 I'll be giving an oral presentation for Creativity Index on Fri 25th 11:06, Garnet 212&219 🎙️ I'll also be presenting posters: 📍ExploreToM, Sat 26th 10:00, Hall 3 + 2B #49 📍CreativityIndex, Fri 25th 15:00, Hall 3 + 2B #618 Hope to see you there!

081

Reposted by Melanie Sclar

Kabir Ahuja @kabirahuja2431.bsky.social · 22/04/2025

📢 New Paper! Tired 😴 of reasoning benchmarks full of math & code? In our work we consider the problem of reasoning for plot holes in stories -- inconsistencies in a storyline that break the internal logic or rules of a story’s world 🌎 W @melaniesclar.bsky.social, and @tsvetshop.bsky.social 1/n

A screenshot of the first page of the paper, containing the paper title: Finding Flawed Fictions: Evaluating Complex Reasoning in Language Models via Plot Hole Detection and the names of the authors: Kabir Ahuja, Melanie Sclar, and Yulia Tsvetkov. All the three authors are from CSE department in the University of Washington in Seattle, USA. They can be reached at {kahuja,msclar,yuliats}@cs.washington.edu

1104

Reposted by Melanie Sclar

Hyunwoo Kim @hyunwoo-kim.bsky.social · 20/02/2025

🚨New Paper! So o3-mini and R1 seem to excel on math & coding. But how good are they on other domains where verifiable rewards are not easily available, such as theory of mind (ToM)? Do they show similar behavioral patterns? 🤔 What if I told you it's...interesting, like the below?🧵

3225

Reposted by Melanie Sclar

Jack Hessel @jmhessel.bsky.social · 22/11/2024

LLMs generate novel word sequences not contained in their pretraining data. However, compared to humans, models generate significantly fewer novel n-grams. RLHF = 30% *more* copying than base! Awesome work from the awesome Ximing Lu (gloriaximinglu.github.io) et al. 🤩 arxiv.org/pdf/2410.04265

A screenshot from the linked paper's figure 1. The figure is a pretty-complicated three column figure, but --- in essence, it sketches out how the authors compare llm sequences to the pretraining data / human authors to the pretraining data. Humans write more novel n-gram sequences.

631247

Reposted by Melanie Sclar

Ximing Lu @gximing.bsky.social · 22/11/2024

Are LLMs 🤖 as creative as humans 👩‍🎓? Not quite! Introducing CREATIVITY INDEX: a metric that quantifies the linguistic creativity of a text by reconstructing it from existing text snippets on the web. Spoiler: professional human writers like Hemingway are still far more creative than LLMs! 😲

3436