iiinova — where your digital self keeps evolving

Matt Groh @mattgroh.bsky.social · 20/08/2026

Curious about how AI is transforming business education, what AI adoption looks like in the Kellogg MBA classroom, and how professors are using AI to teach and teaching about AI? Then, check out Kellogg Magazine with stories from my colleagues and me kellogg.northwestern.edu/magazine/fea...

010

Matt Groh @mattgroh.bsky.social · 02/07/2026

like you say what matters is how often you knowingly interact with them

010

Matt Groh @mattgroh.bsky.social · 02/07/2026

Thank you for being curious!! It feels like there's an emerging sense among many people who read and write a lot for a living that we can sniff AI-generated text. I sense that too though I believe we still have non-zero FPR. I think the same thing is true (and even more so) for images, but...

110

Matt Groh @mattgroh.bsky.social · 30/06/2026

And the SOTA text-to-image model today (GPT Image 2) is waaaay more photorealistic than the models from this experiment in 2024 (Stable Diffusion, Midjourney, and Firefly), but the tells still fit nicely into our training guide and taxonomy from 2024

140

Matt Groh @mattgroh.bsky.social · 30/06/2026

At this point, I'd say images and videos fall into the following three buckets that totally depend on context (more empirical research coming soon on context): 1. You can point to tells that suggest it's AI 2. You cannot tell the diff 3. You can point to tells that it's real

120

Matt Groh @mattgroh.bsky.social · 30/06/2026

To some degree the "tells" change on every new model release, so slight retraining to stay on the cutting edge is monthly. I've been doing this as an annual training, and I should really release a YT video for the next one. But to be clear, I'm certainly not always able to tell real from fake.

110

Matt Groh @mattgroh.bsky.social · 30/06/2026

Thanks to @negrain.bsky.social for leading this and collaborators Candice Gerstner and @jessicahullman.bsky.social for helping this come to fruition! And thanks to NCSU LAS for funding and operational support and VeryAI for additional funding and support. Read here: arxiv.org/pdf/2606.28510

arxiv.org

100

Matt Groh @mattgroh.bsky.social · 30/06/2026

We hosted this training in November 2024, and now we're halfway through 2026. 1.5 years is a long time given how fast AI is moving. But, I believe these results are still relevant. Here's an anecdote to share why looking at the current model at the top of the AA text-to-image leaderboard:

110

Matt Groh @mattgroh.bsky.social · 30/06/2026

Perhaps, it's also the questions that I left for the analysts at the end of the training to apply to any image in the future: (1) Does this image show evidence of Generative AI? (2) Does this image show anything Generative AI tends to perform poorly on?

100

Matt Groh @mattgroh.bsky.social · 30/06/2026

I'd speculate that what sets our relatively brief training apart from others is we had more visual examples, which allowed analysts to build a more nuanced mental model of what generative AI tends to perform poorly on and how non-generative AI photography artifacts may appear.

100

Matt Groh @mattgroh.bsky.social · 30/06/2026

What I found surprising and exciting is where the improvement came from: analysts got better at recognizing real images as real. In other words, the training reduced false alarms rather than just increasing suspicion which is often found in other deepfake detection trainings.

110

Matt Groh @mattgroh.bsky.social · 30/06/2026

To note: The image set included portraits, full-body images, posed groups, and candid scenes from Midjourney, Stable Diffusion, and Firefly (the 2024 state-of-the-art models) All 97 stimuli are available in the appendix and Github repo and you've already seen some in the thread

100

Matt Groh @mattgroh.bsky.social · 30/06/2026

This cross-randomized design allows us to measure performance on the same image sets before and after training, which appears in the interaction term in the table. The take-away here is training boosts overall accuracy from 72% to 81% driven mostly by accuracy on reals which move from 67% to 81%

100

Matt Groh @mattgroh.bsky.social · 30/06/2026

We embedded a counterbalanced randomized experiment in the training. We asked analysts to judge 40 images, conducted the training, and judge 40 more images. As Figure 1 shows, we randomly assigned analysts to see image set A before training and B after or vice versa.

100

Matt Groh @mattgroh.bsky.social · 30/06/2026

So, how do we train people to distinguish the kinds wierdness? We show them lots of images. In the training, I presented 32 slides w/ 50 AI images and 7 real images. (The deck is available in the Github repo linked in the paper) But how do we know the training worked?

110

Matt Groh @mattgroh.bsky.social · 30/06/2026

The problem is that not every weird looking image is AI-generated. Real photos can have blur, compression, bad lighting, strange angles, motion artifacts, awkward poses, low resolution, cinematographic looks and more. AI weirdness is simply different from human weirdness.

100

Matt Groh @mattgroh.bsky.social · 30/06/2026

The 2024 conventional wisdom for spotting AI images was something like: - look for weird hands - strange teeth - broken text - odd lighting - impossible objects - plastic skin That advice can still be useful. But it can also make people see AI everywhere.

100

Matt Groh @mattgroh.bsky.social · 30/06/2026

Can we train people to better detect deepfakes and not be overly skeptical of everything?🤔 We ran a 30-minute training for 🕵️ intelligence analysts with an embedded randomized experiment The short answer is yes; the key is learning which kind of artifacts point to AI 🧵

280

Reposted by Matt Groh

Ziv Epstein @ziv-e.bsky.social · 25/03/2026

Pleased to share our new paper forthcoming in @icwsm.bsky.social! We introduce a novel framework to measure value expressions in social media posts at scale, leveraging personalization to handle the inherent subjectivity of human values. arxiv.org/abs/2511.08453

2329

Matt Groh @mattgroh.bsky.social · 25/03/2026

So, "that sucks" could be a great response demonstrating understanding and validating emotions. We very much find that we're not seeing the translation between people's self reported feelings of empathy and what normative empathic communication looks like.

010

Matt Groh @mattgroh.bsky.social · 25/03/2026

And last, I'm sharing other threads pointing to our paper so it's easier to follow the conversation emerging from this work: bsky.app/profile/emol...

000

Matt Groh @mattgroh.bsky.social · 25/03/2026

Thanks to @aakriti1kumar.bsky.social for leading this research and our awesome coauthors Fai Poungpeth, @diyiyang.bsky.social and Bruce Lambert This is a preprint, and we'd love your feedback to make this paper even better!

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

If you've made it this far, I highly suggest you to check out the full paper here: arxiv.org/pdf/2603.15245 You'll find more details on all the results plus if you check out the supplementary information at the end you'll find all the prompts for the AI coach and RPG scenarios

arxiv.org

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

That's empathy-as-a-trait measured via two well-established protocols like the Jordan Empathy Scale and the Single Item Trait Empathy Scale We call this the “silent empathy effect” because people report feeling others’ pain yet lack fluency in the idioms known to make others feel heard.

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

So, we found a nearly 1 SD improvement in empathic communication based on two rounds of practice with personalized AI coach feedback. But, what I found most surprising is we do not see any relationship between empathic communication performance and empathy-as-a-trait

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

Now, how did we measure performance? We pre-registered a communication framework based on encouraging elaboration, validating emotions, demonstrating understanding, advice giving, self orientation, and dismissing emotions. See our paper from last month for deets: nature.com/articles/s42...

nature.com

When large language models are reliable for judging empathic communication - Nature Machine Intelligence

Kumar et al. show that large language models (LLMs) nearly match expert reliability and outperform laypeople when assessing empathic communication across multiple frameworks. The performance of both L...

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

But empathy isn't rocket science and it's been taught by humans in many forms. Take "7 Habits of Highly Effective People" as an example, which I'm sharing a portion of here to offer context:

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

We find that Bruce's #howcommunicationworks videos lead to an immediate increase in performance across our empathic communication framework and the personalized AI coach leads to an even larger increase. It may be surprising that AI can help us quickly learn hard-to-teach skills like empathy.

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

Or even better yet practice with an AI conversational partner and get personalized feedback and have a conversation with our AI coach: human-ai-collaboration-lab.kellogg.northwestern.edu/rpg

human-ai-collaboration-lab.kellogg.northwestern.edu

Lend an Ear Experiment

Improve your empathic communication by role playing with LLMs

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

Here's another one of Bruce's videos:

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

If you're experiencing a visceral reaction to this categorization of mis-attuned responses, you may not have mastered the art of empathic communication. I highly suggest checking out Bruce Lambert's videos that we used in one arm of the randomized experiment

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

The most common mis-attuned responses are categorized as advice giving or dismissing emotions. The advice is generally good advice, but the problem here is empathic support is generally not about advice but being with people exactly where they are to help them process their emotions

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

The most common affective responses to personal troubles are acknowledging the difficult of the situation and expressing sympathy by saying "I am so sorry to hear that."

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

With sparse autoencoder based analyses, we can get extremely detailed and share the distribution of sub-categories of the four high-level categories of empathic expression: Affective, Cognitive, Motivational, and Mis-Attuned.

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

It turns out LLMs are fantastic at simulations like role playing games. 91% of participants rated the scenarios as a 4 or 5 out of 5 for realisticness. By collecting almost 17k messages in realistic text-based convos, we can map idioms in these empathic support contexts

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

First, context: How did we collect ~3k five minute convos? We built a LLM role playing game and asked participants to lend an ear and provide empathic support to a conversational partner. We also embedded a pre-registered randomized experiment w/ comms coach interventions.

100

Matt Groh @mattgroh.bsky.social · 25/03/2026

AI can help us humans better understand how we connect. Empathy is something most people feel strongly but fail to communicate effectively. Just see how people respond to someone passed over for promotion Insights from 3k convos btw 968 people and LLMs from our new preprint 🧵

240

Matt Groh @mattgroh.bsky.social · 22/03/2026

Thanks @emollick.bsky.social for sharing our latest work on AI and empathy! It's a preprint, so we would love feedback and comments.

051

Matt Groh @mattgroh.bsky.social · 16/03/2026

The line-up for the AI and Innovation series at the Ryan Institute on Complexity in the Spring is pretty epic! Mark your calendar for talks by @zhitzig.bsky.social , @allisonkoe.bsky.social, Sam Goldberg , and Pat Pataranutaporn

000

Reposted by Matt Groh

William J. Brady @williambrady.bsky.social · 24/02/2026

We're excited about the upcoming Computational Psychology preconference at @spspnews.bsky.social this Thursday. See our action-packed full day agenda below! Featuring 3 keynote talk themes with related early-career speakers, data blitz session, panel discussion. Don't miss it! #SPSP

1219

Matt Groh @mattgroh.bsky.social · 11/02/2026

If you made it this far, I encourage you to check out the paper for the full story! Big props to @aakriti1kumar.bsky.social who led this paper and our wonderful team of interdisciplinary collaborators Fai Poungpeth, Diyi Yang, Erina Farrell, Bruce Lambert Stay tuned for more!

020

Matt Groh @mattgroh.bsky.social · 11/02/2026

Open questions: What is the right evaluation framework for a given conversational context? And, when are LLMs less likely to be good judges? My suspicion to this last question is when there's high context between two people like two close friends, the judgment is likely to hold less water.

120

Matt Groh @mattgroh.bsky.social · 11/02/2026

🔵 LLMs could power coaching advice to everyday people on how to be better active listeners and make others feel more heard 🔵 LLMs could power scalable professional development offering nuanced evaluation to customer service reps, medical students, and therapists-in-training

130

Matt Groh @mattgroh.bsky.social · 11/02/2026

With evidence that LLMs can judge empathic communication in these contexts, here are some future possibilities that I can imagine: 🔵 LLMs-as-judge can create transparency and accountability into when LLMs-as-companions might be going off the rails

120

Matt Groh @mattgroh.bsky.social · 11/02/2026

Beyond the fact that objective ground truth is elusive, the problem w/ classification: (a) imbalanced classes and off-by-one errors obscure performance (b) variations in rating scales leads to incommensurability between scales (c) binarization offers research degrees of freedom to juke the stats

120

Matt Groh @mattgroh.bsky.social · 11/02/2026

Key finding: Across the same high-level task measured via four different frameworks in four different settings, we find that appropriately prompted LLMs judge the nuances of empathic communication nearly as reliably as experts So, why not report simple AUC or accuracy?

120

Matt Groh @mattgroh.bsky.social · 11/02/2026

Example conversations show how annotations converge and align across these three groups Crowd judgments are generally more positive and more variable, which we suspect is due to a combo of acquiescence bias and variability of their effort and experience (see more in the SI)

120

Matt Groh @mattgroh.bsky.social · 11/02/2026

Experts' agreement with each other varies considerably across the 4 frameworks and their 21 sub-dimensions 👉 But, objective ground truth is elusive for this task For each dimension, we find relatively high IRR between experts and LLMs but much less between experts and crowds

120

Matt Groh @mattgroh.bsky.social · 11/02/2026

We sampled 200 conversations from four publicly available conversation datasets where one speaker shares a trouble and another offers support 🔵 Empathetic Dialogues 🔵 EPITOME 🔵 Perceived Empathy 🔵 Lend an Ear Each has a different context & evaluative framework (see attached tables)

120