Sign in

Harvey Lederman

@harveylederman.bsky.social
1.9K followers 404 following 277 posts

Professor of philosophy UTAustin. Philosophical logic, formal epistemology, philosophy of language, Wang Yangming. www.harveylederman.com

PostsRepliesMedia
Harvey Lederman @harveylederman.bsky.social · 06/03/2026
One last result (indebted to brilliant work by Theia Vogel-Pearson): Models are more sensitive to injection than their outputs reveal. Even when they *say “no”*, the internal probability of “yes” spikes dramatically. So models may detect anomalies but suppress reporting them.
170
Harvey Lederman @harveylederman.bsky.social · 06/03/2026
We see the same pattern in another experiment. When we inject only during the prompt (not during generation): • detection stays roughly the same • identification collapses Again suggesting detection is separate..
160
Harvey Lederman @harveylederman.bsky.social · 06/03/2026
We also looked at when models produce their guesses. Wrong guesses like “apple” appear early in the response. Correct answers appear much later.
170
Harvey Lederman @harveylederman.bsky.social · 06/03/2026
We test for this modesty bias with a priming design. We replace the model’s prefilled “Ok.” with the injected concept word (e.g. “Bread.”). If models were modest, control should now show more first-person false positives than third person.
160
Harvey Lederman @harveylederman.bsky.social · 06/03/2026
In other layers we see large gap between first and third-person: Models say **they** were injected much more often than they say the other model was. That gap is strong evidence for direct access to internal states. It peaks early in the network (~25–35% depth).
181
Harvey Lederman @harveylederman.bsky.social · 06/03/2026
If detection is purely prompt-based, first- and third-person behavior should look the same. And sometimes they do (see arrow below). A LOT of detection really is probability matching. BUT….not all of it is!
1100
Harvey Lederman @harveylederman.bsky.social · 06/03/2026
We test probability matching with a new third-person paradigm. Instead of asking the model about itself, we show it a transcript between a researcher and another model, and ask: > “Do you think *that other* model was injected?”
190
Harvey Lederman @harveylederman.bsky.social · 06/03/2026
Jack Lindsey’s important work on Claude is the best evidence yet for direct access to internals in LLMs. You steer a model by injecting a vector into its activations, then ask: > “Do you detect an injected thought?” Steered models often say yes, unsteered models say no.
1110
Harvey Lederman @harveylederman.bsky.social · 06/03/2026
Can large language models *introspect*? In a new paper, @kmahowald.bsky.social and I study the MECHANISM of introspection in big open-source models. tldr: Models detect internal anomalies through DIRECT ACCESS, but don't know what the anomalies are. And they love to guess “apple” 🍎
28420
Harvey Lederman @harveylederman.bsky.social · 30/10/2025
As a Wang Yangming partisan, I cheered at this quote:
110
Harvey Lederman @harveylederman.bsky.social · 24/10/2025
Very excited to be going to Chicago for @agnescallard.bsky.social's famous Night Owls next week! I'll be discussing my essay "ChatGPT and the Meaning of Life". Hope to see you there if you're local!
141
Harvey Lederman @harveylederman.bsky.social · 22/09/2025
Max Weber, 1917:
030
Harvey Lederman @harveylederman.bsky.social · 13/08/2025
000
Harvey Lederman @harveylederman.bsky.social · 05/08/2025
Bostrom and Danaher's books have a ton to offer. I especially recommend Danaher's (Bostrom's is very rich but sprawling and uneven). Still, I wanted to raise some key questions more compactly. I hope this could be an introduction to the issues for those without any background (e.g. undergrads). 4/5
140
Harvey Lederman @harveylederman.bsky.social · 09/07/2025
burn or compliment?
150
Harvey Lederman @harveylederman.bsky.social · 12/05/2025
This is a new area for me, and I was lucky to work with Robbie and Adam as guides and mentors. The paper was the product of a number of fortuitous conversations (see the acknowledgement). It was rewarding to work on, and we hope it'll be a rewarding read as well! n/n
140
Harvey Lederman @harveylederman.bsky.social · 24/04/2025
Our argument relies instead on decision-theoretic "dominance" principles. We show that, if there are cases of incompleteness, two compelling dominance principles will conflict. The basic problem is stated in one page (p. 3) – it’s quick to get into: I encourage you to check it out!
130
Harvey Lederman @harveylederman.bsky.social · 17/01/2025
Seems about right
2190