Sign in

Lucy Li

@lucy3.bsky.social
5.9K followers 669 following 404 posts

assistant prof at Wisconsin CS. #NLProc, computational social science, cultural analytics, responsible AI. she/her. Previously at UW, Berkeley, Ai2, MSR, Stanford. lucy3.github.io

PostsRepliesMedia
Lucy Li @lucy3.bsky.social · 14/10/2026
Models are sycophantic, and when they do challenge faulty or incorrect inputs, they're ambiguous and ineffective. In a new paper w/ a multilingual co-author team, we find that people may view LM responses that push back, yet still end up believing false content is true. arxiv.org/abs/2610.17703
A screenshot of the top half of the first page of a paper titled "Pushback in a Haystack: Language Models Respond to Factual Errors Ambiguously and Ineffectively" with authors Li Lucy, Myra Cheng, Svenja Guhr, Kabir Ahuja, Orevaoghene Ahia, Sameep Chattopadhyay, Yike Wang, Valentin Hofmann, Noah A. Smith, Yulia Tsvetkov
0437
Lucy Li @lucy3.bsky.social · 02/08/2026
Settling in one place something I haven't experienced for a long time (my phd peers did get things like espresso machines and bed frames but i was always very lazy and wishy washy about it) I am currently buying FURNITURE. my cat hates all of her new trees & beds. she likes human bed
a white cat, belly up, on an earthy pastel colorblock quilt
110
Lucy Li @lucy3.bsky.social · 02/08/2026
This is the "view" from the floor my office is on, in the computer, data, & info sci building the wood paneling reminds me of swiss cheese
staircases with wood railings and sides, with an atrium showing six floors. seating and tables on second and third floors
110
Lucy Li @lucy3.bsky.social · 02/08/2026
I used to joke that going to Stanford for college w/ palm trees and manicured landscapes was like being on vacation with exams -- summers at UW-Madison have a similar feel. Some evenings are so pretty that these ducks must be paid actors to decorate these lakes also the ice cream is very creamy
ducks and a dock and a sunset on lake mendota
2220
Lucy Li @lucy3.bsky.social · 29/06/2026
👨‍🎨 update slai = social language and ai w = wisconsin
an image of a sticker showing a badger with red and white hat hugging a jar of hazelnut spread branded as "slai"
1180
Lucy Li @lucy3.bsky.social · 24/06/2026
wisconsin colors or nutella colors?? 🤷‍♀️
010
Lucy Li @lucy3.bsky.social · 24/06/2026
👨‍🎨 update on initial brainstorm sketch to actual sketch:
a work in progress (blue lines) of a wisconsin badger with a jar of hazelnut spread with the research group logo "slai" on the front of the jar.
250
Lucy Li @lucy3.bsky.social · 23/06/2026
me: "please draw a wisconsin badger hugging a jar of hazelnut spread except instead of the logo being nutella because it's trademarked it is the logo of my new research group" human 👨‍🎨:
concept art for a badger hugging a jar with a rough sketch of the wisconsin social language and ai logo as the brand label
080
Lucy Li @lucy3.bsky.social · 05/06/2026
tfw ur paper clicking is not human enough
a screenshot of "your ip address has been blocked please contact dl-support@acm.org"
1130
Lucy Li @lucy3.bsky.social · 03/03/2026
Models’ mistakes may assume correct math solutions. Typically, models are trained on “high quality” math so that they can hill-climb on GSM8k, MATH, etc. However, dev pipelines that favor correct math are tension w/ education, where math errors require extra attention.
Question: How many dots did the student include in their array?
For an erroneous student response: Model answer: 12. True answer: The student didn't include an array. True answer for a non-erroneous student response: The student included 12 dots in their array.
Question: How many squares did the student draw to show the number of cups of red paint?
For an erroneous student response: Model answer: The student drew 9 squares to show the number of cups of red paint. True answer: The student drew 12 squares to represent the cups of red paint.
True answer for a non-erroneous student response: The student drew 9 squares to show the number of cups of red paint.
101
Lucy Li @lucy3.bsky.social · 03/03/2026
We find that this gap is primarily driven by QA related to content description. In addition, VLMs struggle to identify cases when help is needed; the most challenging QA are those related to assessing students’ correctness and errors.
A bar chart disaggregating results for four VLMs across different question types. Content description QA consistently drives the gap in VLM performance between student responses that contain errors versus those that do not. In addition, questions related to students’ correctness and errors are still the most difficult.
100
Lucy Li @lucy3.bsky.social · 03/03/2026
Models are now expert math solvers, and so AI for math education is receiving increasing attention. Our new preprint evaluates 11 VLMs on our QA benchmark, DrawEduMath. We highlight a startling gap: models perform less well on inputs from K-12 students who need more help. 🧵
Title, author list, and two figures from the paper. 
Title: The Aftermath of DrawEduMath: Vision Language Models
Underperform with Struggling Students and Misdiagnose Errors
Authors: Li Lucy, Albert Zhang, Nathan Anderson, Ryan Knight, Kyle Lo
Figure 1: On the left is a math problem, where students are asked to draw x < 5/2 on a number line. The right side shows two example student responses that differ in correctness. DrawEduMath pairs each math problem with one student response, and prompts VLMs to answer questions about the student response.
Figure 2: VLMs consistently perform worse on answering DrawEduMath benchmark questions pertaining to erroneous student responses. Performance on non-erroneous student responses is labeled with specific VLMs’ names; that same model’s performance on erroneous student responses is directly below.
43512
Lucy Li @lucy3.bsky.social · 19/01/2026
ty claude code for helping annotators f̶i̶g̶h̶t̶ ̶e̶a̶c̶h̶ ̶o̶t̶h̶e̶r̶ ̶w̶i̶t̶h̶ ̶m̶o̶r̶e̶ ̶c̶l̶a̶r̶i̶t̶y̶ look at data ... I'm planning to throw this up on github once i test it thoroughly and write artisanal docs for it
926610
Lucy Li @lucy3.bsky.social · 09/01/2026
they took away the snowflakes recently. it's still winter outside irl 😦 (in the northern hemisphere at least). miss the little snowy blue fellow
020
Lucy Li @lucy3.bsky.social · 05/12/2025
important questions
a screenshot of gemini chat history, where today the user prompted the model "Do you train on my inputs" and yesterday the user prompted the model "university of buffalo has got have real buffalo"
380
Lucy Li @lucy3.bsky.social · 11/11/2025
It's the season for PhD apps!! 🥧 🦃 ☃️ ❄️ Apply to Wisconsin CS to research - Societal impact of AI - NLP ←→ CSS and cultural analytics - Computational sociolinguistics - Human-AI interaction - Culturally competent and inclusive NLP with me! lucy3.github.io/prospective-...
A staircase in the new School of Computer, Data & Information Sciences building at Wisconsin Madison. Tan wood structures surround tapestry art and a small indoor garden.A view from above of the staircases in the Wisconsin CDIS building An shot from below of winding wooden staircases and a glass atrium rooftop. The new School of Computer, Data & Information Sciences building at Wisconsin Madison. A bicolor white cat with seal-colored markings, looking upwards with big wide dark eyes.
15116
Lucy Li @lucy3.bsky.social · 10/11/2025
Not to brag too much or anything but after emnlp I got brunch with a llama in Shanghai
A white llama in a cafe setting, peering over the counter at the barista bartender.
1690
Lucy Li @lucy3.bsky.social · 22/10/2025
Strangest autocomplete-induced error: I had imported 🐢 and not 🐼
a screenshot of code that says "from turtle import pd" "import torch" "from transformers import..."
0110
Lucy Li @lucy3.bsky.social · 17/10/2025
i ran out 😆 but we can exchange some virtual ones
pink frosted pop tarts with rainbow sprinkles
010
Lucy Li @lucy3.bsky.social · 05/08/2025
I just moved and Toast discovered a love for rolling in bathtubs; she had never seen one before now. She also figured out how to open new types of doors, and so my cat-proofing battle continues.
a white ragdoll cat rolling in a bathtub, with her belly upwards and arms and legs flailing
2110
Lucy Li @lucy3.bsky.social · 21/07/2025
very productive day modifying ssh log in messages
screenshot of text that says "*** System restart required ***
=============== HELLO ==================" followed by an scared-looking ASCII cat
0110
Lucy Li @lucy3.bsky.social · 30/05/2025
"Tell, Don't Show" was accepted to #ACL2025 Findings! 

 Our conceptually intuitive, lightweight approach for literary topic modeling combines the new (language models) with the old (classic LDA) to yield better topics. ✨📚 arxiv.org/abs/2505.23166
A screenshot of the title, authors, abstract, and figure 1 of the paper. Text in the abstract begins with: "Conventional bag-of-words approaches for topic modeling, like latent Dirichlet allocation (LDA), struggle with literary text. Literature
challenges lexical methods because narrative language focuses on immersive sensory details instead of abstractive description or exposition: writers are advised to show, don’t tell. We propose Retell, a simple, accessible topic modeling approach for literature. Here, we prompt resource-efficient, generative language models (LMs) to tell what passages show, thereby translating narratives’ surface forms into higher-level concepts and themes."
38413
Lucy Li @lucy3.bsky.social · 29/05/2025
010
Lucy Li @lucy3.bsky.social · 28/05/2025
things are going well
a screenshot of an overleaf latex document titled "Lucy Dissertation" with 299 latex errors/warnings
4530
Lucy Li @lucy3.bsky.social · 05/05/2025
I'm joining Wisconsin CS as an assistant professor in fall 2026!! There, I'll continue working on language models, computational social science, & responsible AI. 🌲🧀🚣🏻‍♀️ Apply to be my PhD student! Before then, I'll postdoc for a year in the NLP group at another UW 🏔️ in the Pacific Northwest
Wisconsin-Madison's tree-filled campus, next to a big shiny lake A computer render of the interior of the new computer science, information science, and statistics building. A staircase crosses an open atrium with visibility across multiple floors
1614514
Lucy Li @lucy3.bsky.social · 03/05/2025
Exciting times eating and flying at #NAACL2025
Green chili chicken tacos with avocadoOutstanding paper award for DrawEduMathRio Grande with colorful hot air balloons hovering over the river The inside of an inflated a blue patterned hot air balloon
2361
Lucy Li @lucy3.bsky.social · 26/03/2025
this was entirely a coincidence
A screenshot of Ai2 bluesky account post about a new "Ai2 Paper Finder", followed by my post about having tip-of-the-tongue paper retrieval problems
0100
Lucy Li @lucy3.bsky.social · 25/03/2025
A nice diagram from Tsvetkova et al. 2024's "A new sociology of humans and machines" on ways in which "collective outcomes in human-machine social systems differ from those in human-only systems" www.nature.com/articles/s41...
Figure caption: Fig. 2 | Collective outcomes in H–M social systems differ from those in human-only systems. Machines behave differently from humans, and in social systems  with covert artificial agents the collective outcomes will differ simply because of this fact, —even if humans, unaware of the presence of machines, do not change  their behaviour—. Furthermore, the two types of actors and their interactions are interdependent and influence each other. Thus, suspicion or awareness of machine  presence can change human behaviour and interacting with a machine and  observing M–M interactions can influence how humans act towards each other. Order of difference. First: Machine behaviour differs  from human behaviour. Second: Humans interact  differently with machines  than with humans. Machines interact even if  they are not explicitly designed to do so. Third: Human behaviour changes with the  presence of machines, Human interactions are indirectly affected by machines and their interactions.
0272
Lucy Li @lucy3.bsky.social · 29/01/2025
New @ai2.bsky.social paper accepted to #NAACL2025! We worked with Teaching Lab and ASSISTments to create a new dataset for evaluating VLMs. DrawEduMath includes 2,030 teacher-annotated images of students’ handwritten responses to math problems. Website: drawedumath.org
a screenshot of the drawedumath website, showing paper title Evaluating Vision Language Models with Expert-Annotated Students’ Hand-Drawn Math Images and author list Sami Baral, Li Lucy, Ryan Knight, Alice Ng, Luca Soldaini, Neil Heffernan, Kyle Lo with affiliations at WPI, Berkeley, Insource Services, Teaching Lab, and Ai2. Below that is an image showing an example image with question-answer pairs from the dataset.
1265
Lucy Li @lucy3.bsky.social · 09/01/2025
i would like to work with my computer monitor but it sets up a cursor catching game for my cat 😑
a white cat paw next to a cursor on a computer monitor
081
Lucy Li @lucy3.bsky.social · 03/01/2025
paper poster? paper poster
drake meme, where the top shows drake gesturing "no" to the OLMo paper poster presented at a conference and the bottom shows him gesturing "yes" to a poster for OLMo 2 designed like the 2 fast 2 furious movie poster
1100
Lucy Li @lucy3.bsky.social · 03/01/2025
🤔 🚗
a paper titled 2 OLMo 2 Furious co-authored by many people from the Allen Institute for AI and UW and NYU.
151
Lucy Li @lucy3.bsky.social · 03/01/2025
cute paper titles 💯🦙
a screenshot of a paper title that says "Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning" authored by people at Princeton and Tsinghua, including Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, Danqi Chen
0141
Lucy Li @lucy3.bsky.social · 29/12/2024
New England winter 🥰
A low sun peeking through trees. Ground covered in snow
070
Lucy Li @lucy3.bsky.social · 16/12/2024
Today I spelled "chatgpt" so wrong that I later misread it as "chapagetti"
an instant noodle packet, specifying chapagetti (jjajang noodle)
1140
Lucy Li @lucy3.bsky.social · 14/12/2024
they’re so spoiled by our bay area “winter”
colorful succulents on patio table and chairs after rain
0200
Lucy Li @lucy3.bsky.social · 11/12/2024
Each time I do it, I get something different 😅 the interface even said I am "special" and require extra data for them to make their predictions
A screenshot of a breakdown that says "Here's everything I detected in your voice" with 49% English, 22% Korean, 14% Hindi/Urdu
140
Lucy Li @lucy3.bsky.social · 11/12/2024
I tried this "accent guesser" by a service which claims to be "the fastest way to sound like a native English speaker." start.boldvoice.com/accent-guesser 🤔 ... I grew up in a Mandarin-speaking household in suburban Minnesota. What's happening behind the scenes? Why do I sound "Persian"?
A screenshot of "Here's everything I detected in your voice", where the results say 43% Persian, 19% English, and 18% Spanish
3202
Lucy Li @lucy3.bsky.social · 28/11/2024
I have contributed 🥹
A screenshot of text that says "Hi Lucy,
Thank you for contributing photos to eBird and the Macaulay Library. This November, we released a new version of Photo ID in Merlin, and we wanted to extend our gratitude for the important part you played in the creation of this new resource. 1 of your photos were used to train Merlin to identify birds in photos."
2380
Lucy Li @lucy3.bsky.social · 24/11/2024
everything is so precarious
a photo of cat paws on a laptop keyboard, with one paw very close to the on/off button
3390
Lucy Li @lucy3.bsky.social · 19/11/2024
china!
a screenshot of a tweet that says 🚨 In 2025, @emnlpmeeting will take place in Suzhou, China from Nov. 5-9! with a picture of a. map and the conference venue, International Expo Centre
120
Lucy Li @lucy3.bsky.social · 14/10/2024
Education & NLP. Education is a domain where NLP, and AI generally, could achieve substantial positive social impact if done responsibly. My work bridges digital humanities w/ education and designs tasks and evaluation frameworks for AI models w/ educators.
A screenshot of a pdf showing the title and abstract of a paper titled Content Analysis of Textbooks via Natural Language Processing: Findings on Gender, Race, and Ethnicity in Texas U.S. History TextbooksA screenshot of a figure from the paper titled MathFish : Evaluating Language Model Math Reasoning via Grounding in Educational Curricula
141
Lucy Li @lucy3.bsky.social · 14/10/2024
Equitable NLP. I question societal defaults: what are the default representations of people in models and media (e.g. books, Internet), and who is disadvantaged by these defaults? I'd like to inform the responsible use of LMs and drive actionable social change more broadly.
A screenshot of the top of the first page of a paper titled Gender and Representation Bias in GPT-3 Generated StoriesA screenshot of the top of the first page of a paper titled “One-Size-Fits-All”? Examining Expectations around What Constitute “Fair” or “Good” NLG System Behaviors
121
Lucy Li @lucy3.bsky.social · 14/10/2024
Sociolinguistic-y NLP. Many core challenges in language modeling, from fairness to domain adaptation, relate to linguistic differences in data. These differences are socially driven, and so, addressing these challenges benefits from a sociolinguistic perspective.
A screenshot of the top of the first page of a paper titled AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data FiltersA screenshot of the top of the first page of a paper titled Words as Gatekeepers: Measuring Discipline-specific Terms and Meanings in Scholarly Publications
121
Lucy Li @lucy3.bsky.social · 14/08/2024
Finally, we automatically tag math standards in GSM8k problems, and find that it targets a narrow range of skills/concepts! Estimating standards in K-12 benchmarks can allow us to better understand why some problems are more difficult to solve for models than others.
A figure showing models’ problem solving performance, based on problems' grade levels of skills/concepts inferred by a language model.
110
Lucy Li @lucy3.bsky.social · 14/08/2024
We find that LMs struggle to tag and verify standards linked to problems. They can also generate problems that do not fully align with standards described in prompts, and checking for generation issues w/ GPT-4 can lead to more optimistic predictions than the ones teachers make.
A table of common reasons for why generated problems have no or partial alignment, obtained via open coding of teachers’ explanations. Provided examples in each row are cases where GPT-4 judges a problem to be fully aligned, but teachers do not.
110
Lucy Li @lucy3.bsky.social · 14/08/2024
Existing math datasets may be written by crowdworkers or generated by LMs. We introduce data involving curriculum used by actual teachers to encourage more ecologically valid AI & education research. This data is labeled with fine-grained math concepts & skills, or “standards”.
A table showing summary data statistics from Achieve the Core and MathFish.
110
Lucy Li @lucy3.bsky.social · 14/08/2024
New paper 📝 w/ Ai2! We worked with educators to identify a novel angle for evaluating language models' reasoning around math and pedagogy. 👩🏻‍🏫 🐟🍏 We ask, can LMs discern which skills and concepts are targeted by math content? Preprint: arxiv.org/abs/2408.04226
An example of a math problem involving operations in base ten in our dataset, along with standards it does and doesn't align with. This figure illustrates two task formats: verification and tagging.
182
Lucy Li @lucy3.bsky.social · 29/06/2024
I like how I can have both a topical organization of things, but also a chronological one. When prepping for weekly update meetings, I usually forget what I did in the past week, but then I can review by checking my activity:
010
Lucy Li @lucy3.bsky.social · 29/06/2024
Trello is organized in a hierarchical way where you have big cards and also little cards. Within the little cards, you can add comments, checklists, and descriptions. My big cards usually include: related work, ideas, questions for advisors/senior people, todo, currently doing, done
110