iiinova — where your digital self keeps evolving

Roger Levy @rplevy.bsky.social · 13/11/2025

Building on Kyle's and Jenn's responses – it seems to me the analogy is: grammaticality is to BLiMP and SyntaxGym as truth is to COMPS and plausibility (albeit that's not binary) is to EWoK. So, to apply our framework to those datasets, perhaps one should swap truth/plausibility for grammaticality?

120

Reposted by Roger Levy

Jennifer Hu @jennhu.bsky.social · 10/11/2025

New work to appear @ TACL! Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar. Yet they often assign higher probability to ungrammatical strings than to grammatical strings. How can both things be true? 🧵👇

Screenshot of a figure with two panels, labeled (a) and (b). The caption reads: "Figure 1: (a) Illustration of messages (left) and strings (right) in toy domain. Blue = grammatical strings. Red = ungrammatical strings. (b) Surprisal (negative log probability) assigned to toy strings by GPT-2."

29220

Reposted by Roger Levy

Tomer Ullman @tomerullman.bsky.social · 01/12/2024

thinking of calling this "The Illusion Illusion" (more examples below)

601570381

Roger Levy @rplevy.bsky.social · 11/11/2024

Been listening a lot to Ella Jenkins the last couple of days. What wonderful music and performance!

171

Roger Levy @rplevy.bsky.social · 10/11/2024

Within the past two weeks I deleted apps for media companies owned by 67% of the world’s richest billionaires, and it felt great!

1342

Roger Levy @rplevy.bsky.social · 10/11/2024

Not quite sure what you mean by a “complete” corpus. I do think the basic philosophical assumptions of frequentist probability are applicable to corpora, using the large-numbers-of-native-speakers thought experiment. And productivity is a property of the asymptotic distribution, if I’m getting you.

000

Roger Levy @rplevy.bsky.social · 09/11/2024

If there were enough native speakers of the language living at once, you’d quickly get enough instances of the prefix for relative frequency estimation of the next token distribution. Too few humans are alive for this in practice, but that’s not a problem for theoretical validity of the construct!

110

Roger Levy @rplevy.bsky.social · 09/11/2024

You might be interested in this paper we did some time ago! escholarship.org/content/qt69... It supports your conjecture that, insofar as we think the “true distribution” is a valid theoretical construct (which I consider a highly defensible position), large-N Cloze would not give it to us.

escholarship.org

170

Roger Levy @rplevy.bsky.social · 08/11/2024

Results of high stakes elections that happen only once every four years offer remarkable opportunities for overfitting theories of the electorate

0101

Reposted by Roger Levy

Neil Cohn @neilcohn.bsky.social · 28/12/2023

It's my book's release day! The Patterns of Comics is now officially published, featuring an extended data-driven analysis of the structures used in 350+ comics from Asia, Europe, and North America analyzing diversity, regularity, and change over time www.visuallanguagelab.com/poc

The book The Patterns of Comics by Neil Cohn

Interior page from The Patterns of Comics by Neil Cohn

Back cover of The Patterns of Comics by Neil Cohn

618159

Reposted by Roger Levy

bergelsonlab @bergelsonlab.bsky.social · 13/12/2023

GOOD MORNING BLUESKY! Very excited about this new paper: www.pnas.org/doi/10.1073/pnas.2300671120 Key Q: what predicts how much young kids (👶)talk? How much 🗣 kids heard predicted how much 👶talked, but other factors, e.g. mom’s education, didn’t. #PsychSci #DevPsy 🗣💬 INCOMING SUMMARY🧵ALERT 1/14

screenshot of title and authors of paper + map with 18 colorful box callouts showing where datasets came from

515779

Roger Levy @rplevy.bsky.social · 10/12/2023

#linguistics Bluesky: what are the best available quantitative measures of dialect/language mutual intelligibility? The more fine-grained, the better: I'm hoping to vividly illustrate at least one specific dialect continuum (e.g., the Romance languages of the Mediterranean coast)

231

Reposted by Roger Levy

Gretchen McCulloch @gretchenmcculloch.com · 20/11/2023

Today in linguists are NOT KIDDING when we say that your capacity for language enables you to understand sentences that have never before been uttered in human history.

3532189

Reposted by Roger Levy

steven t. piantadosi @spiantado.bsky.social · 15/11/2023

New postdoc opportunity to work jointly with @cantlonlab.bsky.social and me to understand cognition across species, age, and culture! cmu.wd5.myworkdayjobs.com/CMU/job/Pitt...

03328

Roger Levy @rplevy.bsky.social · 11/11/2023

Absolutely, the Nature EiC has it completely backwards. Checking for errors and quality of data (and of math, code, and argumentation) is the most important work that reviewers can do.

1180

Reposted by Roger Levy

simine vazire @simine.com · 11/11/2023

The quotes from Nature EiC Magdalena Skipper about whether journals should be checking for errors/data quality as part of peer review are quite surprising to me. www.wsj.com/science/whats-wrong-wit…

Screenshot of portion of article linked to in post, where Nature EiC says that checking underlying data is not the job of peer review.

147835

Roger Levy @rplevy.bsky.social · 10/11/2023

“This significant effect was found using a post hoc weighting procedure aligned with our overarching hypothesis”?!?

010

Reposted by Roger Levy

Carl T. Bergstrom @carlbergstrom.com · 07/11/2023

I am delighted to announce that the Department of Biology at the University of Washington is advertising for a tenure-track assistant professor position on the quantitative understanding of collective behavior. I will be chairing the search; details are here: apply.interfolio.com/130336

Snow geese fill the sky at sunset in Washington's Skagit Valley.

9265151

Reposted by Roger Levy

CantlonLab @cantlonlab.bsky.social · 07/11/2023

I think it’s amazing that Cognitive Science gives recent PhDs $10K in UNRESTRICTED CASH…right when folks are broke, exhausted, moving town…and need it most. It’s almost Glushko season! cognitivesciencesociety.org/glushko-diss...

cognitivesciencesociety.org

Glushko Dissertation Prize - Cognitive Science Society

1128

Reposted by Roger Levy

Leyla Isik @lisik.bsky.social · 05/10/2023

In a new TiCS article, @emaliemcmahon.bsky.social and I review a growing body of behavioral, neural, and computational evidence that social interactions are automatically extracted by the human visual system: tinyurl.com/nhh2dhxt #PsychSciSky #NeuroSkyence

36330

Reposted by Roger Levy

Tom Pepinsky @tompepinsky.com · 03/11/2023

While the world has its eyes on the Middle East, democratic conditions in Indonesia are looking grim. The Supreme Court has overruled the Constitution in order to allow the sitting president's son to stand as Vice Presidential candidate with a disgraced general with a stained human rights record

1299

Reposted by Roger Levy

Shari Liu @shariliu.bsky.social · 02/11/2023

I am reading PhD applications this year, with a special interest in students who would like to work on the topic of perceived danger. But open to all applicants who share some of my interests. Visit www.liulaboratory.org to see papers, lab values, and tips for application writing.

liulaboratory.org

LIU LAB

In the Look, Infer, and Understand (LIU) Lab at Johns Hopkins University, we are interested in how our minds and brains reason about the physical and social world. We study the developmental and neu...

01411

Reposted by Roger Levy

Robert Hawkins @rdhawkins.bsky.social · 03/11/2023

The Consumer Financial Protection Bureau (CFPB) is hiring a section chief for the psych/”behavioral” section and I'd love to see some CogSci representation in there! My brother works there (trained as an economist) and it's an incredible gig doing research in the public interest.

usajobs.gov

Supervisory Research Scientist (Interdisciplinary)

This position is located at the Consumer Financial Protection Bureau (CFPB), Office of Research. The incumbent supervises and conducts independent, self-directed social/behavioral analysis on a variet...

069

Reposted by Roger Levy

Melanie Mitchell @melaniemitchell.bsky.social · 03/11/2023

Santa Fe Institute now has a Blue Sky account: @sfiscience.bsky.social

13912

Reposted by Roger Levy

Martin Haspelmath @haspelmath.bsky.social · 31/10/2023

A new cross-linguistic study on demonstratives by a team of psychologists and linguists: "Commonalities and differences across languages in spatial communication can be understood in terms of universal constraints on action shaping spatial language and cognition." www.nature.com/articles/s41...

nature.com

Spatial communication systems across languages reflect universal action constraints - Nature Human B...

Coventry et al. show that spatial demonstratives—such as ‘this’ and ‘that’ in English—are selected on the basis of whether the speaker is able to reach the object or not, across 29 diverse...

0154

Reposted by Roger Levy

bergelsonlab @bergelsonlab.bsky.social · 26/10/2023

new work just dropped, see @stephan-meylan.bsky.social's "thread" below:) #DevPsych #CogPsych #PsychSciSky #CogSciSky

1193

Roger Levy @rplevy.bsky.social · 26/10/2023

Thrilled at publication of @stephan-meylan.bsky.social's "How adults understand what young children say", featuring Bayesian noisy-channel inference, LLMs, & child speech datasets! TL;DR: prior expectations of what kids *want to say* is crucial. (Knowing how kids mispronounce words is too.)

0239

Roger Levy @rplevy.bsky.social · 24/10/2023

Prompting is *not a substitute* for probability measurements in large language models – the amazing @jennhu.bsky.social's EMNLP paper now available camera-ready! TL;DR: direct probability measurements show LLMs' linguistic generalizations are better than suggested by prompt-based tests.

0191

Reposted by Roger Levy

Andrew Garrett @andrewgarrett.bsky.social · 24/10/2023

Each week I'm posting the first page of a chapter of my book. Here's the first page of chapter 5, "Language and Land" (seven weeks to publication of "The Unnaming of Kroeber Hall: Language, Memory, and Indigenous California", mitpress.mit.edu/9780262547093/).

Image of first page of chapter 5, "Language and Land", of "The Unnaming of Kroeber Hall" (2023)

0125

Reposted by Roger Levy

Lev Tsypin, PhD @ltsyp.in · 24/10/2023

I was going to submit a manuscript to eLife this week. Just signed this open letter and wrote to my co-authors about submitting our paper somewhere else

0122

Roger Levy @rplevy.bsky.social · 24/10/2023

I'm not one of the organizers of the letter, but I would venture a guess that they would be glad to have you on board!

000

Reposted by Roger Levy

Jeremy Yoder says no war @jbyoder.org · 24/10/2023

This is a very precisely written letter, and as a result, I think, a very strong letter, in support of Eisen — I'm happy to be able to sign on. Please consider adding your support, even if anonymously.

2102

Reposted by Roger Levy

rst.bsky.social @rst.bsky.social · 24/10/2023

There already seems to have been one resignation from the editorial board over this:

twitter.com

Tweet by @LaraUrban42

2102

Reposted by Roger Levy

Carl T. Bergstrom @carlbergstrom.com · 24/10/2023

Profoundly disappointing if true. Whether or not Michael's post was in good taste, he retweeted a mainstream satirical site to his personal social media account. As an eLife editor I have no interest in monitoring my social media activity to ensure everything I post represents the journal.

I have been informed that I am being replaced as the Editor in Chief of @eLife for retweeting a @TheOnion piece that calls out indifference to the lives of Palestinian civilians.

719525

Roger Levy @rplevy.bsky.social · 24/10/2023

Mostly not here for politics. But, at the science/politics intersection: @elife.bsky.social Editor-in-Chief Michael Eisen is apparently being ousted for tweet-referencing a satire conveying the tragedy of the Israeli–Palestinian conflict. Concerned about this form of censure? Sign this open letter.

docs.google.com

Open letter to HHMI and eLife, in defense of free expression

To the Board of Directors, eLife and Erin O’Shea, President, HHMI: We, the undersigned life scientists, are writing to express our view that Professor Michael Eisen's social media posts should not b...

28536

Reposted by Roger Levy

Celeste Kidd @celestekidd.bsky.social · 23/10/2023

We are happy to share our new commentary on why using language inclusively for transgender people is perfectly appropriate biologically, socially, and scientifically. A 🧵 on the commentary by @andyperfors.bsky.social, @spiantado.bsky.social, & me

kiddlab.com

2316585

Reposted by Roger Levy

Nathan Schneider @complingy.bsky.social · 21/10/2023

Haven't posted on here yet— We're hiring a junior faculty member in computational linguistics at Georgetown! 🎓😎 There's a tremendous amount of talent in the field right now, so I can't wait to read the applications. Apply by Nov. 1. DMs/email open for questions. #NLProc #CompLing

apply.interfolio.com

Apply - Interfolio {{$ctrl.$state.data.pageTitle}} - Apply - Interfolio

11817

Roger Levy @rplevy.bsky.social · 20/10/2023

My department (MIT Brain & Cognitive Sciences) is hiring a tenure-track faculty! We're especially interested in researchers who span multiple levels of analysis. Candidates from underrepresented backgrounds strongly encouraged to apply. Apply by November 1! academicjobsonline.org/ajo/jobs/25916

academicjobsonline.org

Massachusetts Institute of Technology, Department of Brain & Cognitive Sciences

Full service online faculty recruitment and application management system for academic institutions worldwide. We offer unique solutions tailored for academic communities.

03638

Reposted by Roger Levy

Alex Spahn 🌋🌪️☄️ @spahn711.bsky.social · 19/10/2023

Full sequence of the annular solar eclipse at Shiprock, New Mexico. This was a dream composition of mine. So much time, energy, and money went into making it happen. I'm forever grateful to the Navajo for letting me shoot it from sacred ground. An experience I'll never forget.

14288072715

Reposted by Roger Levy

Andrew Perfors @perfors.net · 19/10/2023

I don't know if introductions are done here like Mastodon, but whatevs. I'm new and need something to pin. :) I'm a prof in computational cogsci at UniMelb. I use behavioural experiments and mathematical models to study how people learn and reason. /1

1314328

Reposted by Roger Levy

Yoav Artzi @yoavartzi.com · 16/10/2023

Introducing COLM (colmweb.org) the Conference on Language Modeling. A new research venue dedicated to the theory, practice, and applications of language models. Submissions: March 15 (it's pronounced "collum" )

22613

Reposted by Roger Levy

Carl T. Bergstrom @carlbergstrom.com · 10/10/2023

I’m absolutely thrilled to see science Twiiter 🧪 and academic Twitter more generally moving here to bluesky. Twitter was obviously untenable. That said, we need to be mindful of new inequities that may be inadvertently created by moving the conversation to a currently restricted closed platform.

1043051

Reposted by Roger Levy

David Bamman @dbamman.bsky.social · 09/10/2023

This might be flying under the radar (so please RT!), but the US Copyright Office is soliciting comments for its decisions on training ML/NLP/AI systems on copyrighted material (even *non*-generative AI). They need to hear from researchers, so please comment! Deadline Oct 30. www.copyright.gov/ai/

2271392

Reposted by Roger Levy

Cognitive Science Society @cogscisociety.bsky.social · 07/10/2023

🚀 We've officially landed on Bluesky! Excited to join this vibrant new platform and (re)connect with all of you! 🗣️💬 Help us spread the word by retweeting and following.

315096

Reposted by Roger Levy

Ted Underwood @tedunderwood.com · 05/10/2023

I should acknowledge, this piece is coming under very heavy fire from people who feel it fails to acknowledge that this capacity was latent in word embeddings and has been long understood. See e.g. twitter.com/Dorialexande... #lingsky #MLSky

5274