Sign in

Brendan Bartanen

@brendanbartanen.bsky.social
2.5K followers 11K following 49 posts

Associate professor of education and public policy at the University of Virginia. Stumbling my way through the AI revolution at brendanbartanen.substack.com

PostsRepliesMedia
Brendan Bartanen @brendanbartanen.bsky.social · 12h
“The current study presents findings from, to our knowledge, the first large-scale randomized experiment evaluating the effects of AI tutoring on student learning at a large U.S. public university.” The only problem is the findings are wrong. brendanbartanen.substack.com/p/ai-tutorin...
brendanbartanen.substack.com
AI tutoring lowers student performance and engagement. Or does it?
Making sense of an important new study.
3139
Reposted by Brendan Bartanen
Will Oremus @willoremus.com · 02/10/2026
feels like there is a group of smart and well-intentioned people who long ago made up their minds that AI Is Bad, swore off using it, and now get all their info about AI from each other. not saying they're wrong that ai is bad! but i think they're gradually losing touch with what it is and does
235971105
Brendan Bartanen @brendanbartanen.bsky.social · 02/10/2026
Frontier AI writes amazing recipes (seriously), but it has zero sense of how long it takes a normal person to cut an onion.
010
Brendan Bartanen @brendanbartanen.bsky.social · 02/10/2026
Here’s what I told Virginia school superintendents about AI brendanbartanen.substack.com/p/heres-what...
brendanbartanen.substack.com
Here’s what I told Virginia school superintendents about AI
Innovation with guardrails, maintaining human judgment, leveraging the local knowledge of teachers and leaders
011
Brendan Bartanen @brendanbartanen.bsky.social · 01/10/2026
Why does anyone post anything on here? In this case, though, the answer is "irony."
110
Brendan Bartanen @brendanbartanen.bsky.social · 01/10/2026
The article is written by AI--there is nobody to thank
010
Brendan Bartanen @brendanbartanen.bsky.social · 01/10/2026
Turing test performance art
001
Brendan Bartanen @brendanbartanen.bsky.social · 28/09/2026
Here I am, saying the quiet part out loud... brendanbartanen.substack.com/p/education-...
brendanbartanen.substack.com
Education research is broken. AI can help us fix it.
Mistakes and overreach are common in published research, weakening the field’s role as a purveyor of evidence.
012
Reposted by Brendan Bartanen
Matthew A Kraft @matthewakraft.com · 28/09/2026
New @nytimes.com piece ranks 30 education interventions on both effectiveness & costs. For me, the key takeaway is NOT about what academics (including myself) rated as most effective or least costly. It is about the decision making process districts use. 🧵 www.nytimes.com/2026/09/28/u...
nytimes.com
30 Ideas for Fixing U.S. Schools, Ranked by Experts
Math and reading scores are in a decade-long slump. We asked three dozen experts: What could turn things around?
25120
Reposted by Brendan Bartanen
Brendan Bartanen @brendanbartanen.bsky.social · 20/08/2026
Classes start next week. How could I possibly have time to write another post? Find out here: brendanbartanen.substack.com/p/how-im-usi...
brendanbartanen.substack.com
How I'm Using AI to Revolutionize My Teaching
It’s that time of year: every professor you know (minus those lucky few who have course releases) is in a frantic panic to prepare for the beginning of classes.
042
Brendan Bartanen @brendanbartanen.bsky.social · 20/08/2026
Classes start next week. How could I possibly have time to write another post? Find out here: brendanbartanen.substack.com/p/how-im-usi...
brendanbartanen.substack.com
How I'm Using AI to Revolutionize My Teaching
It’s that time of year: every professor you know (minus those lucky few who have course releases) is in a frantic panic to prepare for the beginning of classes.
042
Reposted by Brendan Bartanen
Brendan Bartanen @brendanbartanen.bsky.social · 19/08/2026
I FAFO'd so that you don't have to--you can just read! Today's tip: how to manage the context window in Claude Code brendanbartanen.substack.com/p/my-most-im...
brendanbartanen.substack.com
My Most Important Practical Tips for Using Claude Code (#1)
What is the "context window" and how do you manage it?
131
Brendan Bartanen @brendanbartanen.bsky.social · 19/08/2026
I FAFO'd so that you don't have to--you can just read! Today's tip: how to manage the context window in Claude Code brendanbartanen.substack.com/p/my-most-im...
brendanbartanen.substack.com
My Most Important Practical Tips for Using Claude Code (#1)
What is the "context window" and how do you manage it?
131
Brendan Bartanen @brendanbartanen.bsky.social · 17/08/2026
Why do AI models seem to understand us so well? Why are they "grown" instead of built? Me, wading through the weirdness: brendanbartanen.substack.com/p/what-ive-l...
brendanbartanen.substack.com
What I’ve Learned in a Year of Using Claude Code (Part 2)
Active users vs. passive recipients, and how AI models are like us, good and bad.
010
Brendan Bartanen @brendanbartanen.bsky.social · 15/08/2026
We're back again, another trip down the rabbit hole. substack.com/@bpbartanen/...
substack.com
What I’ve Learned in a Year of Using Claude Code (Part 2)
Active users vs. passive recipients, and how AI models are like us, good and bad.
010
Reposted by Brendan Bartanen
Brendan Bartanen @brendanbartanen.bsky.social · 12/08/2026
New post! brendanbartanen.substack.com/p/what-ive-l...
brendanbartanen.substack.com
What I've Learned in a Year of Using Claude Code (Part 1)
It’s been about a year since I started using Claude Code.
071
Reposted by Brendan Bartanen
Dan Goldhaber @cedr.bsky.social · 12/08/2026
“I am anti-electricity” (not really), but I read @brendanbartanen.bsky.social anyway (joke that you will only get from reading the substack)! In all seriousness, I think this is must read if you are interested in the role of AI in research, or just interested in AI.
032
Brendan Bartanen @brendanbartanen.bsky.social · 12/08/2026
New post! brendanbartanen.substack.com/p/what-ive-l...
brendanbartanen.substack.com
What I've Learned in a Year of Using Claude Code (Part 1)
It’s been about a year since I started using Claude Code.
071
Reposted by Brendan Bartanen
Jon Becker @jonbecker.bsky.social · 21/06/2026
I know it's Sunday and Father's Day and whatnot, but I can't stop thinking about this piece by @brendanbartanen.bsky.social and what it means for AI and research.
brendanbartanen.substack.com
Can Claude Fable 5 help me figure out why American students are struggling so much?
And how long before we have an army of social scientists in a data center?
362
Brendan Bartanen @brendanbartanen.bsky.social · 17/06/2026
Where AI tends to make progress more quickly is on tasks where verification is easier. So it may not actually be low-hanging fruit (honestly unsure). (3) I could have spent more time coaxing it into a style that appears more "human". I didn't give it any instructions specific to writing style.
150
Brendan Bartanen @brendanbartanen.bsky.social · 17/06/2026
A couple thoughts: (1) I read a lot of AI writing and Fable 5 was by far the best I've seen, so I think there's progress. It still does have many of the hallmarks of AI text: lots of em dashes, colons, semi-colons. (2) Writing style is difficult because it's subjective and thus harder to verify...
130
Brendan Bartanen @brendanbartanen.bsky.social · 13/06/2026
Re-upping my post on AI interpretability from a few months back in light of the government's decision to ban access to Mythos/Fable for foreign nationals. For now, AI "guardrails" are just a high-stakes game of whack-a-mole. Read more: brendanbartanen.substack.com/p/we-dont-re...
brendanbartanen.substack.com
We don’t really know how AI models work
“Interpretability” and the high-stakes race to figure out how to understand and control powerful AI
030
Brendan Bartanen @brendanbartanen.bsky.social · 13/06/2026
The probabilistic wrapping around the tool is indeed problematic, which is why the audit trail and verification layers exist. But it's not making some probabilistic "guess" about 2+2.
100
Brendan Bartanen @brendanbartanen.bsky.social · 13/06/2026
No, I think you're confusing things. The notion of a tool call here is to use a deterministic tool (calculator) to produce a deterministic output. Now, whether it calls the right tool and interprets the result correctly are probabilistic. But that's different than the tool itself.
100
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
It's analogous to saying that I can't literally document how my brain functioned to write a paper, but I can show you my code, where I got the data, and even my notes taken along the way. AI models actually do documentation BETTER than humans because of how they are trained to record their thinking.
100
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
No it's far closer to the former. All of the code is there, so the results can be replicated by anyone. I also include my conversation history with Claude Code. What the audit trail does not include (because it only exists on an Anthropic server somewhere) is what the AI model is doing internally.
100
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
The latter questions are more difficult and are questions that we can't "tech" our way out of. I'll be honest and say I have no idea and we will have to figure that out because there will be mistakes and there will be bad actors. But I think standards around openness can prevent bad outcomes.
000
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
Good questions. On the first one, the answer is to make every single artifact publicly available, a full audit trail. For the paper I had Claude write, everything from start to finish exists in this GitHub repository: github.com/brendanbarta... . Anyone or anyone's AI agent can view or replicate.
github.com
GitHub - brendanbartanen-svg/naep-achievement-decline: Verifying the post-2013 US student achievement decline (NAEP) and testing eight explanations
Verifying the post-2013 US student achievement decline (NAEP) and testing eight explanations - brendanbartanen-svg/naep-achievement-decline
210
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
Fair but it's hard when I'm restricted to 300 characters :)
110
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
Generally agree but would argue it differently. The training data are exhaustive across many many domains. Coding is where the most progress has been made because code is highly structured and repeated (very easy for the LLMs to learn patterns) and because coding outputs are easiest to verify.
000
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
Exceptional, astonishing results.
010
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
An easy next step if you want to know more: direct your AI model of choice to examine the public Gitbhub repo and explain to you everything Claude did to create its products: github.com/brendanbarta...
github.com
GitHub - brendanbartanen-svg/naep-achievement-decline: Verifying the post-2013 US student achievement decline (NAEP) and testing eight explanations
Verifying the post-2013 US student achievement decline (NAEP) and testing eight explanations - brendanbartanen-svg/naep-achievement-decline
120
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
Haha I think I was probably anchored by other comments in the thread rather than any miscommunication. Yes, absolutely burns through tokens. For my research report task the all-in cost using API-rate pricing for Fable 5 was ~$450. I am on a Max x20 sub because Max x5 wasn't sufficient. It's crazy.
100
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
So is your complaint about the quality of its work or the token usage?
100
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
If you still don't trust it, that's understandable and fine (I fully agree that vigilance is essential), but let's work from a common set of facts about what the models actually do.
001
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
It's time to move on from this outdated understanding of what AI models do. Agentic AI is not just fancy autocomplete. The models can pull data, write code to analyze it, verify claims, create audit trails, etc. It's not error-proof and vigilance is essential, but it's not the ChatGPT of 2024-2025.
3256
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
Again, have you actually read the report? The claims you're making about what it can and can't do are verifiably false. There is a full audit trail on every single claim and it's not just regurgitating those numbers, they are derived by pulling data and running python scripts on my computer.
110
Brendan Bartanen @brendanbartanen.bsky.social · 12/06/2026
Did you read the report? It has pretty thorough responses to this. Which claims are false?
000
Reposted by Brendan Bartanen
Brendan Bartanen @brendanbartanen.bsky.social · 11/06/2026
I think the era of automated social science is starting to emerge. Here's why: open.substack.com/pub/brendanb...
3164
Brendan Bartanen @brendanbartanen.bsky.social · 11/06/2026
I think the era of automated social science is starting to emerge. Here's why: open.substack.com/pub/brendanb...
3164
Brendan Bartanen @brendanbartanen.bsky.social · 03/03/2026
Did you wake up this morning thinking "I wish someone would explain 'sparse autoencoders' in an approachable way"? Do you wish you knew about 'Golden Gate Claude', perhaps the best AI model ever created? It's your lucky day--all of that and more below!
0300
Reposted by Brendan Bartanen
Brendan Bartanen @brendanbartanen.bsky.social · 03/03/2026
AI models are very different. We don’t actually know how they work. Despite more capital investment than for any technological endeavor in history (roughly 50 Manhattan Projects after adjusting for inflation), we are, in some sense, flying blind. open.substack.com/pub/brendanb...
open.substack.com
We don’t really know how AI models work
“Interpretability” and the high-stakes race to figure out how to understand and control powerful AI
0241
Brendan Bartanen @brendanbartanen.bsky.social · 03/03/2026
AI models are very different. We don’t actually know how they work. Despite more capital investment than for any technological endeavor in history (roughly 50 Manhattan Projects after adjusting for inflation), we are, in some sense, flying blind. open.substack.com/pub/brendanb...
open.substack.com
We don’t really know how AI models work
“Interpretability” and the high-stakes race to figure out how to understand and control powerful AI
0241
Reposted by Brendan Bartanen
Brendan Bartanen @brendanbartanen.bsky.social · 26/02/2026
lol this is an incredible video
34505176
Reposted by Brendan Bartanen
Matt Blackwell @mattblackwell.bsky.social · 28/02/2026
LLMs will usher in a democratization of intellect. Now you too can have a junior (AI) scholar that you can blame for errors and fraud. Previously, only the top scholars had access to that kind of power.
1505
Brendan Bartanen @brendanbartanen.bsky.social · 27/02/2026
lol. for all we know, Claude is randomly assigning service disruptions.
020
Brendan Bartanen @brendanbartanen.bsky.social · 26/02/2026
Assuming you're using agentic AI tools, my default work mode is now: every single task is an opportunity to figure out how to get Claude to make that task easier/better for me. Anytime it works well, save it to claude.md so that it just does it automatically next time.
020
Brendan Bartanen @brendanbartanen.bsky.social · 26/02/2026
lol this is an incredible video
34505176
Reposted by Brendan Bartanen
Brendan Bartanen @brendanbartanen.bsky.social · 25/02/2026
There’s about to be a lot more of everything. More bots. More ads. More scams. More disinformation. More scientific discoveries. Imagine everything you know about automation and scale it up. The world is going to change very quickly. open.substack.com/pub/brendanb...
open.substack.com
Taste rules everything around me
Rick Rubin, "Vibe Coding," and the grand challenge for higher education
052
Brendan Bartanen @brendanbartanen.bsky.social · 25/02/2026
Here's a simple vibe coding example that could be useful for teachers or anyone that needs to create clear and engaging documents. Tell Claude to organize it into an .html file. Here's the prompting: claude.ai/share/c4d1e3... Here's the result: brendanbartanen-svg.github.io/substack-emb...
claude.ai
Finding a Chinese story about freeing others from a box
Shared via Claude, an AI assistant from Anthropic
130