Jackson Petty @jacksonpetty.org · 09/06/2025So, what did we learn? 1. LLMs *do* know how to follow instructions, but they often don’t 2. The complexity of instructions and examples reliably predicts whether (current) models can solve the task 3. On hard tasks, models (and people, tbh) like to fall back to heuristics 110
Jackson Petty @jacksonpetty.org · 09/06/2025But often models get distracted by irrelevant info, or “get lazy” and choose to rely on heuristics rather than actually verifying the instructions; we use o4-mini as an LLM judge to classify model strategies: as examples get more complex, models shift to relying on heuristics rather than rules: 110
Jackson Petty @jacksonpetty.org · 09/06/2025So, how can LLMs succeed at this task, and why do they fail when grammars and examples get complex? Well, models in general do understand the general solution: even small models recognize they can build a CYK table or do an exhaustive top-down search of the derivation tree: 100
Jackson Petty @jacksonpetty.org · 09/06/2025In general, we find that models tend to agree with one another on which grammars (left) and which examples (right) are hard, though again 4.1-nano and 4.1-mini pattern with each other against others. These correlations increase with complexity! 100
Jackson Petty @jacksonpetty.org · 09/06/2025Interestingly, models’ accuracies are reflective of divergent class biases: 4.1-nano and 4.1-mini love to predict strings as being positive, while all other models have the opposite bias; these biases also change with example complexity! 100
Jackson Petty @jacksonpetty.org · 09/06/2025What do we find? All models struggle on complex instruction sets (grammars) and tasks (strings); the best reasoning models are better than the rest, but still approach ~chance accuracy when grammars (top) have ~500 rules, or when strings (bottom) have >25 symbols. 100
Jackson Petty @jacksonpetty.org · 09/06/2025LLMs are increasingly used to solve tasks “zero-shot,” with only a specification of the task given in a prompt. To evaluate LLMs on increasingly complex instructions, we turn to a classic problem in computer science and linguistics: recognizing if a formal grammar generates a given string. 110
Jackson Petty @jacksonpetty.org · 09/06/2025How well can LLMs understand tasks with complex sets of instructions? We investigate through the lens of RELIC: REcognizing (formal) Languages In-Context, finding a significant overhang between what LLMs are able to do theoretically and how well they put this into practice. 152
Jackson Petty @jacksonpetty.org · 02/01/2025Eighth night // Dedication of the House (Shimen Frug) 001
Jackson Petty @jacksonpetty.org · 01/01/2025Seventh night // Dedication of the House (Shimen Frug) 101
Jackson Petty @jacksonpetty.org · 19/12/2024the plan to flip the house in 2026? we get all bluesky-curious voters on the platform, fill it with fake verified accounts, and heaven-ban them from ever interacting with the actual politicians 020
Jackson Petty @jacksonpetty.org · 03/12/2024Of all sad words of tongue or pen, the saddest are these: 130
Jackson Petty @jacksonpetty.org · 28/11/2024girlfriend tax implies the existence of the girlfriend laugher curve, which is the theoretical relationship between how bad my jokes are and how much she laughs at them 000
Jackson Petty @jacksonpetty.org · 21/11/2024so apparently the Ancient Greek verb for “to hurt” is just “aaaaaooooow” 1194
Jackson Petty @jacksonpetty.org · 18/11/2024In case it’s of use to other academics, I put together an overleaf project which shows how to use different styles for in-text citations and bibliographies (eg, author-year in text vs numeric in bib). Works in BibLaTeX and hackily in natbib. Link: www.overleaf.com/read/tkxbfty... 120
Jackson Petty @jacksonpetty.org · 10/11/2024Are you coming to #EMNLP2024? Come say hi! I’ll be presenting this work at BlackboxNLP on Friday, and will be around all week to chat! DM to meet up, let’s grab lunch! 070
Jackson Petty @jacksonpetty.org · 13/11/2023Even though BERTs are good at Type 1 generalization, they're much worse at Type 2 generalization: when we introduce novel verbs, BERTs fail to make correct generalizations about their argument structure! 100
Jackson Petty @jacksonpetty.org · 13/11/2023But humans also have a stronger competency: generalization between related contexts, like learning how “passives” or “clefts” work independent of specific nouns or verbs. We call this “Type 2” knowledge. 100
Jackson Petty @jacksonpetty.org · 13/11/2023BERTs do well at this task, echoing previous work from @najoung.bsky.social & Paul Smolensky, as well as our previous paper (Petty, Wilson, & Frank 2022): 100
Jackson Petty @jacksonpetty.org · 13/11/2023We show that the BERT family of LMs is quite good at Type 1 generalization by introducing novel nouns and fine-tuning LMs on sentences which use them in only active contexts and testing them on novel contexts! 100
Jackson Petty @jacksonpetty.org · 13/11/2023But humans know much more than this–we can infer that if a word appears in context C then it can appear in a related context C’. If you know that “ball” can be the object of “kick,” then you know it can also be the subject of passive “be kicked.” We call this “Type 1” knowledge. 100
Jackson Petty @jacksonpetty.org · 13/11/2023What do LLMs learn about language? Well, their explicit task is to learn a conditional probability distribution over words. We call this “Type 0” knowledge, and LLMs are quite good at it! 100
Jackson Petty @jacksonpetty.org · 10/11/2023An addendum: in language modeling, how come our deepest models aren't the best? We couple depth with _only_ the feedforward size, keeping the attention mechanism unchanged across depths. This means the feed-forward block becomes a lossy transformation when d_ff < d_model! 000
Jackson Petty @jacksonpetty.org · 10/11/2023Nope! We can correct for the LM performance by choosing our fine-tuning checkpoints to have equal perplexity across depths. Even when we do this, deeper models still generalize better! 100
Jackson Petty @jacksonpetty.org · 10/11/2023But our main interest is in compositional generalization. We fine-tune our pretrained models on four different compositional generalization dataset (COGS, COGS-vf, GeoQuery, and English Passivization). Here too, depth helps but the marginal utility diminishes rapidly! 110
Jackson Petty @jacksonpetty.org · 10/11/2023To understand how depth matters, we examine two things: first, we pretrain our models as causal LMs and look at perplexity. We find that deeper models (mostly!) are better, but that most of the benefit comes in having just a few layers! 100
Jackson Petty @jacksonpetty.org · 10/11/2023Our approach is to control for model size, making deeper models narrow and shallow models wide by coupling the # of layers to the size of the feed-forward dimension! We start with three off-the-shelf models (41M, 134M, 374M parameters) and create deep and shallow variants. 100
