Sign in

Ian Arawjo

@ianarawjo.bsky.social
220 followers 114 following 95 posts

Asst Prof at Université de Montréal, Associate Member of Mila-Quebec AI Institute. PhD from Cornell InfoSci. Creator of ChainForge. Programming and culture, LLM evaluation tooling.

PostsRepliesMedia
Reposted by Ian Arawjo
Upol Ehsan | hiring PhDs for Fall'27 @upolehsan.bsky.social · 21/09/2026
This worker is not alone in feeling like this. Even cancer doctors feel like AI "button pushers". The "soul sucking" is the identity commoditization part. More? Check out this award-winning piece at #CHI2026 to learn more about how AI hollows out workers asymptomatically. bsky.app/profile/upol...
131
Reposted by Ian Arawjo
Brian Groom @groomb.bsky.social · 18/09/2026
Hadrian's Wall, watercolour by Rowland Hilder (1905-93).
17968113
Reposted by Ian Arawjo
Lucy Li @lucy3.bsky.social · 11/09/2026
now that a substantial proportion of my time is spent teachng, i am so grateful for people who make their lecture slides public so mine will be public too! lucy3.github.io/cs769-fall26...
lucy3.github.io
Syllabus
Fall 2026
1374
Reposted by Ian Arawjo
Andy Matuschak @andymatuschak.org · 09/09/2026
Made an Obsidian plugin that turns audio files into interactive editable transcripts embedded within ordinary Markdown documents. 🔈✏️ buff.ly/iZEVkpm
2421
Ian Arawjo @ianarawjo.bsky.social · 26/08/2026
God bless those who are serving on the CHI committee this year, I really hope it goes smoothly. It feels bad not to contribute as an AC—a decision I had to make bc of how taxing this summer has been. But I look forward to serving as a reviewer—please put me on your shortlists!
000
Reposted by Ian Arawjo
Jeffrey P. Bigham @jeffreybigham.com · 17/08/2026
a lot of young folks are contacting me wanting to work on "human-AI alignment" and i think they just mean HCI, and the messaging battle has been lost.
1121
Reposted by Ian Arawjo
Steve Klabnik @steveklabnik.com · 14/08/2026
I am very happy to announce that I have gotten a paper accepted to PLSS 2026: LLMs as Collaborators in Language Specification and Design conf.researchr.org/details/spla...
conf.researchr.org
LLMs as Collaborators in Language Specification and Design (PLSS 2026) - SPLASH/ISSTA 2026
Workshop on Programming Language Standardization and Specification This workshop aims to foster cross-pollination between researchers and industry professionals with experience in programming language...
161979
Reposted by Ian Arawjo
Jason Schreier @jasonschreier.bsky.social · 06/07/2026
BREAKING: Xbox will cut 3,200 jobs as part of a major reorganization, and divest from five studios: - Compulsion and Double Fine will go indie - Ninja Theory and Undead Labs will be sold - Arkane to enter consultation process Here's the full story: www.bloomberg.com/news/article...
bloomberg.com
Microsoft’s Xbox to Cut 3,200 Jobs, Divest Five Studios in Major Overhaul
Compulsion and Double Fine studios will become independent; Undead Labs and Ninja Theory will be sold. Xbox will also look to sell or spin out Arkane Studios.
611638618
Reposted by Ian Arawjo
Ronen Tamari @ronentk.me · 16/06/2026
How to create reading experiences that "go beyond information transmission and toward reader transformation.” Great paper by @blue-phia.bsky.social @lepidopterane.bsky.social @yijunliu.bsky.social Sarah Sterman @sh1m.bsky.social @maxkreminski.bsky.social arxiv.org/abs/2606.04308 >
55414
Reposted by Ian Arawjo
Jason Schreier @jasonschreier.bsky.social · 15/06/2026
BREAKING: Several Xbox studios, including Compulsion, Ninja Theory and Double Fine, are negotiating with Xbox as they try to avoid closure. Some or all could spin off. Lots still in flux as many studios remain unsure about what's happening. Here's the latest: www.bloomberg.com/news/article...
bloomberg.com
Studios in Microsoft’s Xbox Division Brace for Closures
The studios, which include Compulsion Games and Double Fine, are in active negotiations with Xbox and may be given the chance to go independent.
18038661321
Reposted by Ian Arawjo
Jason Schreier @jasonschreier.bsky.social · 10/06/2026
BREAKING: Xbox is planning major layoffs next month, Bloomberg has learned, as new CEO Asha Sharma confronts a bleak picture and plans what she calls a "reset" of the business www.bloomberg.com/news/article...
bloomberg.com
Xbox Plans Significant Layoffs as It Transforms Under New CEO
Asha Sharma’s first major cuts will arrive in July
1161306402
Reposted by Ian Arawjo
andrew blinn @disconcision.com · 08/06/2026
We’re doing a user study to find out how adding always-on live values alongside code effects writing and debugging experiences for functional programming. See reply for sign-up details
317338
Reposted by Ian Arawjo
Dr. Casey Fiesler @cfiesler.bsky.social · 02/06/2026
A big shout-out to Michael Correll (is he on Bluesky...?) for this blog post which is ::chef's kiss:: as a bridge to HCI from my work on ethics education in the context of "hey maybe let's re-think what it means to be good at computer science." mcorrell.medium.com/the-othering...
“HCI is an ‘other,’ a less-serious place to be shunted aside or begrudgingly included for marketing reasons so the serious men in real computer science can get on with the actual work. HCI as ‘bless your heart’ patronized computer science, peripheral, marketing fluff ... I was asked to make sure that my work was “legible” to CS departments by including little winking asides that, sure I primarily do HCI work ... but in 
my heart of hearts, I am 
actually secretly a 
database guy or a machine 
learning guy or otherwise 
‘really’ CS.”
3265
Reposted by Ian Arawjo
Shriram Krishnamurthi @shriram.bsky.social · 02/06/2026
New research thread: 1/ Error messages have always been designed, sometimes painstakingly, for humans. But now we have new "readers" for errors: agentic AI. Should this affect what PLs generate? Can we measure this experimentally? We have some preliminary results: ↵
2306
Reposted by Ian Arawjo
Shriram Krishnamurthi @shriram.bsky.social · 16/05/2026
Our interview on Oxide and Friends is finally online! Enjoy, and flame away. (-: oxide-and-friends.transistor.fm/episodes/ai-...
oxide-and-friends.transistor.fm
Oxide and Friends | AI in Computer Science Education
AI is an existential topic for all aspects of education--for none more so than Computer Science. Bryan and Adam were joined by Kathi Fisler and Shriram Krishnamurthi, professors of Computer Science...
1131
Reposted by Ian Arawjo
Mark Riedl @markriedl.bsky.social · 16/05/2026
I bought a book. @mtrc.bsky.social
0152
Ian Arawjo @ianarawjo.bsky.social · 11/05/2026
When AI developers plot standard errors as error bars in eval charts, they're actually misleading us all—driving us to overconfident conclusions on model performance. But what should we do instead? The first investigation in the Stats for Evals blog: statsforevals.substack.com/p/why-ai-dev...
statsforevals.substack.com
Why AI developers should use confidence intervals, not standard errors, for error bars
And a proposal for a better way to visualize AI evaluation uncertainty: gradient plots.
030
Ian Arawjo @ianarawjo.bsky.social · 06/05/2026
In general, I lean against AI for qualitative research. Yet, I see ways that it could help with the tedium of QA, detect and prevent errors, and provoke questions and reflection, so am open to it being incorporated tactfully and tastefully. I just think it has been done extremely poorly so far.
020
Ian Arawjo @ianarawjo.bsky.social · 04/05/2026
The "AI for qualitative research" debate is interesting. Any HCI scholars here completely against it? Or are the HCI takes more nuanced than the open letter would suggest?
000
Reposted by Ian Arawjo
Mark Riedl @markriedl.bsky.social · 30/04/2026
First rule of goblin club: openai.com/index/where-...
openai.com
Where the goblins came from
How goblin outputs spread in AI models: timeline, root cause, and fixes behind personality-driven quirks in GPT-5 behavior.
0275
Ian Arawjo @ianarawjo.bsky.social · 21/04/2026
PSA for anyone submitting to HCI conferences: Do not leave the "suggested reviewers" section blank. It is your greatest (and only) way to heighten the chances your paper will be reviewed by someone who really cares about your research topic. Plus, it helps the 1AC!
010
Reposted by Ian Arawjo
David Mimno @dmimno.bsky.social · 18/04/2026
Looking forward to the open source release, but it sounds like they’re treating model weights as static variables in code rather than data in an ML library, which means the Rust compiler can optimize 1000x better
4599
Reposted by Ian Arawjo
Marianne Aubin Le Quéré @mariannealq.bsky.social · 18/04/2026
I had heard people at #CHI2026 say that they didn't want AI-generated podcasts to be made on top of their work. While this perspective is totally understandable, I thought the podcasts could potentially be useful. But after looking up one of mine, I can't believe this feature was allowed to launch.
21911
Ian Arawjo @ianarawjo.bsky.social · 17/04/2026
“Everyone I met knew, at some level, that AI either means that nothing matters—a kind of creeping techno-nihilism—or that everything that has always mattered—humanism, human values—is all that ever mattered, and our tool tinkering had always been a distraction.” 🙏
000
Reposted by Ian Arawjo
doruk balcı @dorukb.bsky.social · 13/04/2026
the first paper from my phd project is now out and i'm presenting it tomorrow morning at #chi2026! it's an explorative design project about supporting players' rule experimentation through game design. you can read/download from this link dl.acm.org/doi/10.1145/...
23210
Reposted by Ian Arawjo
Dr Florence Smith Nicholls @florencesn.bsky.social · 13/04/2026
The paper is now officially out in the #chi2026 proceedings! You can download the dataset, as well as the autoethnographic memos, as Supplementary Material dl.acm.org/doi/10.1145/...
Screenshot of the front page of the paper
39439
Ian Arawjo @ianarawjo.bsky.social · 13/04/2026
Retweeting this, just because it seems not enough papers at #CHI2026 are declaring their relevance to AI: #makeit100percent #ensureAIrelevanceNow
091
Ian Arawjo @ianarawjo.bsky.social · 10/04/2026
“The real threat is a slow, comfortable drift toward not understanding what you're doing. Not a dramatic collapse. Not Skynet. Just a generation of researchers who can produce results but can't produce understanding.”
020
Ian Arawjo @ianarawjo.bsky.social · 07/04/2026
Stats for Evals is now live, and we got a site, too: statsforevals.com We'll be posting regular investigations across the summer. For now, we're starting with the basics: comparing models and prompts. Also has resources, principles, example code, and guidance for others:
statsforevals.com
Statistics for LLM Evals
A research-backed guide to statistical methods for LLM and AI model evaluations. Learn to compare models, prompts, and agents with confidence intervals, bootstrap methods, and hypothesis tests.
2192
Ian Arawjo @ianarawjo.bsky.social · 05/04/2026
It would be cool if HCI had: 1) an open reviewing platform, 2) a quid pro quo credit system like CritiqueCircle with added kudos by experts for quality reviews, 3) anonymization of reviewers (but where you can see fuzzy metrics of reviewer quality)
000
Reposted by Ian Arawjo
Maddy Myers 🏳️‍🌈 @midimyers.com · 03/04/2026
"you cheated not only the game but yourself" is not how i feel about video games at all but it is how i feel about people using LLMs to "write" essays for them
336142
Ian Arawjo @ianarawjo.bsky.social · 31/03/2026
Montréal HCI is headed to sunny Barcelona for #CHI2026! ☀️ We’re presenting two exciting, rather unique papers and joining two workshops. Details below! 🧵 1/5
160
Ian Arawjo @ianarawjo.bsky.social · 29/03/2026
I've made a Substack for Stats for LLM Evals progress. We'll be releasing a website soon, but progress will be piecemeal (first release is focused on model comparison, prompt comparison, and model x prompt). Subscribe here for regular updates: substack.com/@statsforevals
substack.com
Stats for Evals | Substack
Musing about statistical analysis for LLM evals. Follow for updates on the Stats for Evals project and promptstats library, including concrete investigations and recommendations.
041
Reposted by Ian Arawjo
Jason Schreier @jasonschreier.bsky.social · 27/03/2026
I don't think people understand how hard it is to work in the video-game industry right now. If you've been laid off, it can take months if not years to find new work. If you haven't been laid off, you're anxious that you will be laid off. This week's column: www.bloomberg.com/news/newslet...
bloomberg.com
It Sucks to Work in the Video-Game Industry Right Now
Even developers who successfully release big hits, like Fortnite and Battlefield, are losing their jobs
572280574
Ian Arawjo @ianarawjo.bsky.social · 27/03/2026
Submitting an AI-powered system paper to #UIST2026 ? Wish you could sense what reviewers were thinking, and how to maximize your chance of acceptance? Check out our #CHI2026 paper, “Reporting and Reviewing LLM-integrated Systems in HCI”, for tips and guidelines: arxiv.org/abs/2602.05128
arxiv.org
Reporting and Reviewing LLM-Integrated Systems in HCI: Challenges and Considerations
What should HCI scholars consider when reporting and reviewing papers that involve LLM-integrated systems? We interview 18 authors of LLM-integrated system papers on their authoring and reviewing expe...
031
Reposted by Ian Arawjo
Jeffrey P. Bigham @jeffreybigham.com · 19/03/2026
now desk reject the papers of reviewers whose human-written reviews are worse than an LLM 😂 … #icml2026
0131
Reposted by Ian Arawjo
Shawn Simister @narphorium.com · 17/03/2026
I want my workflow to feel more like this. One big reconfigurable space for deep work
042
Reposted by Ian Arawjo
Andrew Head @andrewhead.bsky.social · 04/03/2026
HCI summer research opportunity 📣 My group has two openings for research assistants this summer, both in scientific tools for thought. Applicants are welcome at any level. Please help me share the news! andrewhead.info/positions/20...
0114
Ian Arawjo @ianarawjo.bsky.social · 03/03/2026
Writing an HCI paper about an AI-powered system to a venue like UIST 2026 or CHI 2027? Wondering what reviewers expect you to report, and how to approach paper framing and writing? Check out our reporting guidelines: medium.com/p/7c3ae86341...
medium.com
I’m writing an HCI paper about an AI-powered system. What should I report?
Eight Guidelines to Improve Research Quality and Enhance Chance of Acceptance
021
Reposted by Ian Arawjo
Pedro Lopes @pedrolopes.org · 03/03/2026
#CHI2026 program (the draft) is out: programs.sigchi.org/chi/2026/pro... a monster sized CHI that will definitely be fun and intellectually stimulating. Huge kudos to Pablo Cesar and Heloisa Candello, as well as our assistants for making this possible in such a short time ! Check it out!
programs.sigchi.org
Conference Programs
052
Ian Arawjo @ianarawjo.bsky.social · 01/03/2026
To date, HCI researchers have had no support on signaling their paper's relevance to AI, esp. when that connection is tenuous at best. We introduce a systematic framework to ensure LLMs are mentioned at every stage of paper reporting—from framing, to evaluation, to implications.
0273
Reposted by Ian Arawjo
Austin Henley @azhenley.bsky.social · 24/02/2026
Is AI all you’ve got? austinhenley.com/blog/dearres...
austinhenley.com
Dear researchers: Is AI all you've got?
Are we missing the next big innovation because of the over-fixation on AI?
121
Reposted by Ian Arawjo
Jessica Vitak @vitak.bsky.social · 20/01/2026
Did you have a qualitative paper rejected from #chi2026? As you're revising for your next submission, check out this crowdsourced document containing common critiques of qualitative research and ideas for responding -- and add any new critiques you've encountered: docs.google.com/document/d/1...
docs.google.com
Reviewer Critiques (Qualitative Methods) and How to Respond to Them
Reviewer Critiques (Qualitative Methods) and How to Respond to Them Author: Jessica Vitak (+ anyone who adds to the document) About This Document (and a disclaimer) Reviewing is a highly subjective pr...
0176
Reposted by Ian Arawjo
Jeffrey P. Bigham @jeffreybigham.com · 19/02/2026
I think I'd rather have an LLM review my paper.
141
Reposted by Ian Arawjo
Jeffrey P. Bigham @jeffreybigham.com · 17/02/2026
companion robots are like zoom chats with the grandkids … it seems like progress, thank goodness it's there! yet, it just makes lack of community and real human connection that much easier to tolerate www.nytimes.com/2026/02/12/u...
nytimes.com
To Stay in Her Home, She Let In an A.I. Robot
041
Reposted by Ian Arawjo
Mark Riedl @markriedl.bsky.social · 12/02/2026
An OpenClaw agent makes a pull request to matplotlib. Mainter rejects PR. The OpenClaw agent authors a blog post accusing the maintainer of discrimination and gatekeeping. Maintainer responds theshamblog.com/an-ai-agent-...
theshamblog.com
53410
Ian Arawjo @ianarawjo.bsky.social · 11/02/2026
🎉 Thrilled to share that our paper "Reporting and Reviewing LLM-Integrated Systems in HCI: Challenges and Considerations" has been conditionally accepted to #CHI2026! A thread 🧵
1105
Reposted by Ian Arawjo
Marcelo Rinesi @marcelorinesi.bsky.social · 06/02/2026
Notations/languages/etc (natural/constructed/formal/etc) sit at the intersection of a lot of my interests from information theory & statistics to Borgesian literature & group/individual cognitive ergonomics. Self-recommending. by @zamfi.bsky.social @damienhci.bsky.social @ianarawjo.bsky.social et al
arxiv.org
How Notations Evolve: A Historical Analysis with Implications for Supporting User-Defined Abstractions
Traditional human-computer interaction takes place through formally-specified systems like structured UIs and programming languages. Recent AI systems promise a new set of informal interactions with c...
1113
Reposted by Ian Arawjo
Shriram Krishnamurthi @shriram.bsky.social · 04/02/2026
New talk abstract dropping. I just hope I can write a talk to live up to it by *checks watch* *gulp* Monday.
Generative AI and Computing Education for Novices

Generative AI (GenAI) has sowed havoc in computing education, especially introductory computing. As AI models grow in sophistication, it is increasingly difficult to find problems that cannot be comprehensively solved by AI. Even "defeat devices" like obscure programming languages have limited viability, due to technical reasons like one-shot learning, temporal reasons such as training, and basic educational considerations like what we want students to learn.

I claim that the recent rise of agentic coding, in particular, is a major technological turning point that forces a deep re-examination of computing curricula. However, I come with a hopeful message rather than a pessimistic one. I believe there is a profound opportunity here for computing educators; the challenge is to not ignore these trends but rather understand how they can enhance us and what we have to offer. I will try to lay out all these issues.

This is not a research talk, where I describe a set of crisp technical results that have been deeply vetted with proofs and experiments. Rather, in the spirit of the topic, I am trying to express a vibe. But I am not just idly speculating; I will base my comments on ongoing observations and experiments. I hope to spur lots of discussion, but also provide a sense of optimism.

The one guarantee I do provide is that the talk content and slides were written entirely by a human.
3302
Ian Arawjo @ianarawjo.bsky.social · 02/02/2026
Over the past week, I've seen a lot of CHI authors announcing their papers on social media as "accepted." I get that it's exciting, but your work is "conditionally" accepted—saying it's flat-out "accepted" is a bit of counting chickens before they hatch. Wait a bit for the official sign-off, folks!
000