iiinova — where your digital self keeps evolving

Julian Matthews @quining.bsky.social · 30/09/2025

The result: negligible effect ⚡ Metacognitive efficiency is robust to report order—it need not constrain study design and secondary analysis... But watch out if you use regression measures?! 😱 See the full story here: 👉 www.nature.com/articles/s44... #psychology #OpenScience #neuroscience

A plot from the paper that shows negligible effect of limiting time or motor preparation on metacognitive efficiency measures.

A plot from the paper that shows negligible effect of report order on metacognitive efficiency measures.

130

Julian Matthews @quining.bsky.social · 30/09/2025

We tested this directly, 2 experiments (perception & memory, n=175) controlling for ⏱️ timing and ✋ motor preparation. 7 registered hypotheses and 3 different measures unpacked order bias from all angles.

A figure from the paper illustrating the study design:

Three report orders determine the arrangement of task decisions and confidence ratings: (1) decision followed by confidence, (2)decision simultaneous with confidence, and (3) confidence followed by decision.Three report contexts test the influence of reaction time and motor preparation. (1)In the time limited context, participants had 2.8 seconds in total to make the task decision and confidence rating. The response annulus appeared at a new, random angle on each trial. (2) In the baseline context, participants had unlimited time toreport, but the response annulus appeared at a new, random angle on each trial. (3)In the motor fixed context, participants had unlimited time to report, and the response annulus was fixed in the same position on every trial. An orange spot is added to each annulus in this figure to aid in visualising the rotation. This spot was not presented to participants.

110

Julian Matthews @quining.bsky.social · 30/09/2025

🚨 Out now in @commspsychol.nature.com 🚨 doi.org/10.1038/s442... Our #RegisteredReport tested whether the order of task decisions and confidence ratings bias #metacognition. Some said decisions → confidence enhances metacognition. If true, decades of findings will be affected.

A picture of our paper's abstract and title: The order of task decisions and confidence ratings has little effect on metacognition.

Task decisions and confidence ratings are fundamental measures in metacognition research, but using these reports requires collecting them in some order. Only three orders exist and are used in an ad hoc manner across studies. Evidence suggests that when task decisions precede confidence, this report order can enhance metacognition. If verified, this effect pervades studies of metacognition and will lead the synthesis of this literature to invalid conclusions. In this Registered Report, we tested the effect of report order across popular domains of metacognition and probed two factors that may underlie why order effects have been observed in past studies: report time and motor preparation. We examined these effects in a perception experiment (n = 75) and memory experiment (n = 50), controlling task accuracy and learning. Our registered analyses found little effect of report order on metacognitive efficiency, even when timing and motor preparation were experimentally controlled. Our findings suggest the order of task decisions and confidence ratings has little effect on metacognition, and need not constrain secondary analysis or experimental design.

12710

Julian Matthews @quining.bsky.social · 02/07/2024

I will share our findings at #ASSC27 about the mental effort of #metacognition + its consequences for a famous metacognitive phenomenon… 1️⃣💦2️⃣ Catch it in Metacognition #1, Fukutake - Wed 03 July 5:30PM #PsychSciSky #neuroscience

Banner advertising "Metacognition is mentally demanding: revealing the cost and consequences of metacognitive effort". A concurrent talk at the 27th Meeting of the Association for the Scientific Study of Consciousness.

190

Julian Matthews @quining.bsky.social · 20/06/2024

Bridge Road Brewing catering to a very specific Venn diagram that I absolutely exist within

020

Julian Matthews @quining.bsky.social · 14/06/2024

We know that time ⏰ and movements 👋 affect metacognition. Yet many of us use a protocol that ensures a time-consuming action before every confidence rating. Would your results be the same if you used a different report order? 🤔

A flow diagram where "time" and "action" feed from a task decision into a confidence rating, causing it to be distorted.

140

Julian Matthews @quining.bsky.social · 14/06/2024

🎉 Our #RegisteredReport on report order & #metacognition accepted-in-principle Communications Psychology! Why do we collect #confidence ratings after task decisions? As a grad student, this puzzled me. We used one order in every study, but others flipped it or collected both together 🔄

An drawing of someone considering three report orders: 1) decision then confidence, 2) decision and confidence, 3) confidence and decision.

171

Julian Matthews @quining.bsky.social · 22/05/2024

Apart from metacognition, one of several other obsessions of mine is craft beer. Couldn't pass up an opportunity to visit Green Bench, the 'Burgs best hop & foeder technicians.

000

Julian Matthews @quining.bsky.social · 17/05/2024

#VSS and #metacognition? Stop by my poster on Sunday and let's discuss the mental cost of perceptual metacognition and confidence.

A banner advertising my poster "Metacognition is mentally demanding" at the 2024 Vision Sciences Society meeting.

I will present poster 33.310 on Sunday, May 19 from 8:30am to 12:30pm in the session "Decision making: perceptual decision making 2".

1113

Julian Matthews @quining.bsky.social · 30/11/2023

Inspiring symposium on AI for cognitive neuroscience organised by @neurocoops.bsky.social & Tijl Grootswagers absolutely wild to see the improvements in deepfake/generative technology for robust stimulus design #ACNS2023

"AI for Cognitive Neuroscience: Exploring new horizons" a symposium with talks by Dr Casey Becker, Dr Tijl Grootswagers, Dr Astrid Zeman, and Dr Patrick Cooper

000

Julian Matthews @quining.bsky.social · 22/10/2023

Tokyo Coffee Festival returns after 4 years hiatus ☕ There's a dog here wearing sunglasses.

A photo of the festival coffee mug. What a beaut!

120

Julian Matthews @quining.bsky.social · 28/09/2023

Parents finally made it over to 🇯🇵 My grandad owned a small corner store in Stafford, Queensland. It's practically unimaginable that someone from our family would eventually work overseas. Here's Dad 229.71m above Shibuya

The nighttime cityscape of Tokyo from Shibuya Sky

090

Julian Matthews @quining.bsky.social · 01/09/2023

Ikebukuro last night 🇯🇵 We love stumbling upon these random vendors, leave 100円(🇦🇺 ~$1) and take a full bag of eggplants! Anyone have a good babaganoush recipe?

Crates of eggplants with a sign that says: ナス、 100円・1袋

050