Reposted by Matt Makelmoin syed @syeducation.bsky.social · 22/09/2026New paper! Despite qualitative researchers claiming that generalizability is not a goal, their published work does indeed contain generalizable claims. We document this and provide recommendations for how all researchers can better calibrate their generalizability claims. doi.org/10.1177/2515... 619255
Reposted by Matt MakelAERA @aeraedresearch.bsky.social · 09/09/2026Reminder! The Graduate Student Research-in-Progress Roundtable Series Call for Submissions is open. The deadline to submit to the call is October 15! Full eligibility and submission information here: www.aera.net/Education-Re... 011
Reposted by Matt MakelAmanda Kay Montoya @akmontoya.bsky.social · 08/09/2026Curious about Zenodo as an OSF Project alternative? Here's a short video to get you started: File Uploads, Communities, and generating links for Anonymous Peer Review. youtu.be/jW84PxuJTMwyoutu.beGetting Started with Zenodo (transitioning from OSF Projects)YouTube video by AMANDA MONTOYA 67440
Reposted by Matt MakelOlúfẹ́mi O. Táíwò @olufemiotaiwo.bsky.social · 08/09/2026if it took you 5 seconds of contemplation to think of an alternative you should take at least another 5-10 to at least consider the possibility that it at least crossed the minds of 1 or 2 of the people who dedicate their lives and research careers to dealing with the problems we post about on here 292379299
Reposted by Matt MakelCas Goos @cas-goos.bsky.social · 03/09/2026I am proud to announce that my article "Measurement reliability, construct validity, and transparent reporting in original and replication psychological research" has just been published in PLoS One. You can find the published article here: journals.plos.org/plosone/arti...journals.plos.orgMeasurement reliability, construct validity, and transparent reporting in original and replication psychological researchPublished (replication) studies that use measures to assess psychological phenomena need to transparently report information on measurement procedures, validity, and reliability to allow verification ... 0103
Reposted by Matt MakelRandy Ellis @randalljellis.bsky.social · 02/09/2026Here's my conversation with @bethclarke.bsky.social, incoming Junior Professor of Psychological Metascience at the Leibniz-Institut für Psychologie. www.youtube.com/watch?v=2Hak... We discussed direct replications, how study limitations are ignored, and p-curving social and personality psychology.youtube.comDirect replications, P-curves, and Acknowledging Limitations | Beth ClarkeYouTube video by Metascience Matters 1189
Matt Makel @mattmakel.bsky.social · 31/08/2026I made a mistake. And because of that mistake, I no longer have confidence that results that I reported in several research papers are an accurate reflection of replication rates. 1/ 19430
Reposted by Matt MakelProf Dynarski @dynarski.bsky.social · 27/08/2026Test-optional policies reduce college admissions of high-achieving, disadvantaged students Under test-optional policies, too few of these applicants submit test scores; they more than double their admission probability (from 2.9% to 7.2 %) by reporting scores. drive.google.com/file/d/10mW5... 35221
Reposted by Matt MakelDr Ellie Murray, ScD @epiellie.bsky.social · 27/08/2026Love this! We need more and better descriptive studies. For those whose descriptive psych studies are somewhat epidemiologic in nature, might I recommend this paper on how to do good descriptive work👇🏼 pubmed.ncbi.nlm.nih.gov/35774001/pubmed.ncbi.nlm.nih.govA Framework for Descriptive Epidemiology - PubMedIn this paper, we propose a framework for thinking through the design and conduct of descriptive epidemiologic studies. A well-defined descriptive question aims to quantify and characterize some feature of the health of a population and must clearly state: 1) the target population, characterized by … 092
Reposted by Matt MakelBeth Clarke @bethclarke.bsky.social · 21/08/2026I’m hiring a PhD student to work with me on metascience in psychology at @zpid.bsky.social! The position is very open - could cover a wide range of topics/themes 🤸🏻♀️✨ If you have questions, please reach out! Applications are open until October 1st. leibniz-psychology.onlyfy.jobs/en/job/fyjwt...leibniz-psychology.onlyfy.jobsPhD in Psychological Meta-Science (m/f/d)Become part of the open science movement The Leibniz Institute for Psychology Information (ZPID) in Trier (Germany) is an internationally recognized research-based psychology service center located in... 36157
Reposted by Matt MakelMary Dudziak @marydudziak.bsky.social · 18/08/2026Beginning my research seminar, as always, with Lynn Hunt's essay How Writing Leads to Thinking (and Not the Other Way Around). Should be mandatory reading in light of the impact of AI. www.historians.org/perspectives... 9478153
Reposted by Matt MakelSociety for the Improvement of Psychological Science (SIPS) @improvingpsych.org · 15/08/2026🥳Today, PsyArXiv – the community preprint server for psychology – turns 10! 🥳 Thank you to everyone who contributed to the server's growth, including volunteers who work behind the scenes to keep it running, preprint authors, and readers! 06917
Reposted by Matt Makelvincentholst.bsky.social @vincentholst.bsky.social · 13/08/2026In 2023, @nature.com published 'Papers and patents are becoming less disruptive over time', receiving world-wide media attention. Our Matters Arising, published after a 32 month delay (more on that soon), shows that the reported decline can largely be attributed to dataset artefacts. 🧵 9351172
Reposted by Matt Makelsimine vazire @simine.com · 12/08/2026It’s amazing what Brian and OSF have done for science. I can imagine how hard the folks at COS must’ve worked to avoid this outcome. I’m extremely grateful to them for creating the reality we live in now. The fact that we’ll miss these services so acutely is a testament to how much they shaped us. 09224
Reposted by Matt MakelKatie Corker @katiecorker.bsky.social · 12/08/2026For those looking for alternatives to OSF, I recommend two resources comparing features of other Generalist Repositories. Resources are from GREI, an NIH funded initiative. Comparison tool to help you pick which one you like best: doi.org/10.5281/zeno... Decision tree: doi.org/10.5281/zeno...doi.orgGeneralist Repository Comparison ChartThe Generalist Repository Comparison Chart and FAIRsharing Collection (https://fairsharing.org/collection/GeneralRepositoryComparison) was originally an outcome of the NIH Workshop on the Role of Gene... 03423
Reposted by Matt MakelAmanda Kay Montoya @akmontoya.bsky.social · 11/08/2026Here's the list put together by OSF: help.osf.io/article/735-.... Zenodo is my top preference currently. Can host any kind of research output, and gives everything a DOI, and free. Hosted by CERN and funded by many European government groups.help.osf.ioWhere should I share my Data, Materials, Code now?This Article Is Licensed Under CCO For Maximum Reuse. The following is a Table of Contents that links directly to specific sections within the guide: Table of R 121
Reposted by Matt MakelBrian Nosek @briannosek.bsky.social · 11/08/2026Over the next several months, we will phase out some popular OSF features. These changes do not affect current public content, which remains safe and accessible. We have prepared support material to help users adapt. I am very sorry for the disruption to your work. Please read this post for details.cos.ioOSF Changes | Center for Open ScienceWe are preparing substantial changes to OSF that will reduce its functionality, focus OSF on its unique strengths, and move toward an integrated model with complementary services. As part of that shif... 10120113
Reposted by Matt MakelJoshua Goodman @joshua-goodman.com · 03/08/2026I'm quoted in this morning's @bostonglobe.com editorial about why middle school is likely the right age to begin tracking math classes. By middle grades, the range of math skills in a given cohort is too large to be taught effectively in a single class. www.bostonglobe.com/2026/08/03/o... 2133
Reposted by Matt MakelMatthew A Kraft @matthewakraft.com · 24/07/2026Take 1 minute to read this intuitive and incisive review of our study on the portability of teacher effectiveness by @nctq.bsky.social's Sarah Almy. Beautiful policy writing in my opinion. www.nctq.org/research-ins...nctq.orgDoes great teaching travel from school to school? It’s complicated. - National Council on Teacher QualityDo great teachers perform as well after switching schools? New research on a $20,000 transfer incentive program shows the answer is complicated—and prep matters. 052
Reposted by Matt MakelAnne Scheel @annemscheel.bsky.social · 23/07/2026New preprint w/ Leo Tiokhin & @lakens.bsky.social! Registered Reports (RRs) are great for science b/c publication is guaranteed before results are known, reducing publication bias & QRPs. This feature supposedly also makes them great for *scientists*. But is that really true? 1/ tinyurl.com/4sy993rrtinyurl.comCareer incentives make Registered Reports unattractive under realistic academic conditionsRegistered Reports are a publication format designed to reduce publication bias by guaranteeing publication before results are known. This guarantee is considered attractive in the prevailing ‘publish... 15934
Reposted by Matt MakelPaul Bruno @paul-bruno.com · 22/07/2026"I'd like to see state and federal policymakers place a stronger emphasis on making student growth and improvement measures publicly available and giving them the kind of attention we've historically given measures like student 'proficiency'" www.newsweek.com/map-reveals-...newsweek.comMap reveals states with best and worst public schoolsA new ranking lists New York as the state with the best public schools, while Arizona ranks at the bottom. 251
Reposted by Matt Makelbergelsonlab @bergelsonlab.bsky.social · 17/07/2026within every [academic] there are 2 [reasons we agree to review] 1) sounds interesting & we wanna know about it...bc we are huge nerds 2) sounds problematic & we wanna make sure someone is sufficiently critical...bc we are huge nerds (ok ok & sometimes we wanna do the editor a solid) #AcademicSky 0213
Reposted by Matt MakelArvind Narayanan @randomwalker.bsky.social · 15/07/2026I had the honor of giving a keynote at the International Conference on Machine Learning last week. I addressed the widespread anxiety about how we should adapt as AI capabilities increase. I was thrilled by the talk’s reception, so I have made my slides available www.cs.princeton.edu/~arvindn/tal...cs.princeton.eduWhat will be left for us to work on?ICML 2026 invited keynote — slides and edited transcript, presented click-by-click as delivered. Arvind Narayanan, Princeton University. 613948
Reposted by Matt MakelCrystal Steltenpohl @cnsyoung.com · 06/07/2026New article from @mattmakel.bsky.social, Melanie Meyer, and Mary Pei about qualitative replication.doi.orgReplication is possible, can be relevant, and is useful in qualitative researchIn 2022, we argued that replication is relevant to qualitative research. In this manuscript, we update and clarify that idea to make three more specific claims about replication in qualitative rese... 011
Reposted by Matt MakelThleen @kathleenbot.bsky.social · 26/06/2026Ethicists: I'm sorry but we gotta stop citing scientific claims to solo-authored books. The citation has to be the actual paper with a zillion authors and a clunky title and ugly graphs that actually proves the result. I know it sucks but please the philosophers of science are laughing at us 2177
Reposted by Matt MakelMorgan Polikoff @mpolikoff.bsky.social · 28/05/2026I had many thoughts about that NYT grade inflation op-ed from a couple days ago. Here are some of them. // It's Gonna Be Much Harder Than People Think to Change the Narratives About Testing and Grades edpolicyhub.substack.com/p/its-going-...edpolicyhub.substack.comIt's Gonna Be Much Harder Than People Think to Change the Narratives About Testing and GradesA Response to Kalil and Rury 2153
Reposted by Matt MakelCenter for Open Science @cos.io · 07/04/2026What can large-scale studies tell us about reproducibility? In our webinar on April 16, researchers from COS, I4R, and META-REP will discuss findings from three papers—one from the recently published SCORE effort—and insights on reproducibility, transparency, and credibility cos-io.zoom.us/webin... 02814
Reposted by Matt MakelDaniel Ansari 🇨🇦 @numcog.bsky.social · 10/02/2026New: "Effect of Artificial Intelligence on Learning: A Meta-Meta-Analysis" by Wagenmakers and colleagues revealing evidence for "severe publication bias and extreme between-study heterogeneity" in existing meta-analyses of the effects of AI on learning: osf.io/preprints/ps...osf.ioOSF 15324
Reposted by Matt MakelDaniel Lakens @lakens.bsky.social · 08/02/2026New blog post, inspired by the excellent recent qualitative paper by Makel and colleagues: On the reliability and reproducibility of qualitative research. I reflect on how I will incorporate realist ontologies in my own qualitative research. daniellakens.blogspot.com/2026/02/on-r...daniellakens.blogspot.comOn the reliability and reproducibility of qualitative researchWith my collaborators, I am increasingly performing qualitative research. I find qualitative research projects a useful way to improve my un... 01916
Reposted by Matt MakelRoyal Society Publishing @royalsocietypublishing.org · 07/02/2026New #RSOS paper: ‘Don’t hate the players, hate the game’: qualitative insights from education researchers on questionable and open research practices. Read more: doi.org/10.1098/rsos... @mattmakel.bsky.social @sarahcaroleo.bsky.social @jesse-fleming.bsky.social @bryancook.bsky.social 0146
Matt Makel @mattmakel.bsky.social · 29/01/202654 weeks later, this preprint is now published: royalsocietypublishing.org/rsos/article...royalsocietypublishing.org‘Don’t hate the players, hate the game’: qualitative insights from education researchers on questionable and open research practicesAbstract. Both questionable (e.g. p-hacking) and open (e.g. pre-registration) research practices are prevalent in education research. We sought to understa 021
Reposted by Matt MakelAmy @amylibrarian.bsky.social · 29/01/2026Need some tips for searching PsyArXiv? Wrote a blog post blog.psyarxiv.com/2026/01/27/h...blog.psyarxiv.comHow to search for PsyArXiv preprints on the OSF platform – PsyArXiv BlogPsyArXiv is hosted on the Open Science Framework (OSF) platform, and over the years their interface and search syntax has changed. Guidance on how to search is a snapshot in time. Presently, the searc... 121
Matt Makel @mattmakel.bsky.social · 17/12/2025New Publication alert: Making a List Requires Checking it Twice: A Call for Empirical Evidence in Characteristics Lists [journals.sagepub.com/doi/10.1177/00…] #OpenAccess 220
Reposted by Matt MakelTom Hardwicke @tomhardwicke.bsky.social · 26/11/2025Job opportunity — Junior Professorship in Psychological Metascience @zpid.bsky.social leibniz-psychology.onlyfy.jobs/job/10kku5n7 h/t @bethclarke.bsky.social 12323
Reposted by Matt MakelEinstein Foundation Berlin @einsteinberlin.bsky.social · 24/11/2025🏆 Individual: @simine.com, psychologist at @unimelb.bsky.social & editor-in-chief of Psychological Science, is recognized for pioneering methodological rigor, reproducibility & collaborative research, driving initiatives such as @improvingpsych.org & the journal Collabra @ucpress.bsky.social. (2/5) 39321
Matt Makel @mattmakel.bsky.social · 19/11/2025Job posting alert! Open Science Specialist at the University of Calgary (in Canada!) #job #OpenScience careers.ucalgary.ca/jobs/1704502...careers.ucalgary.caSpecialist, Open Science, Research Services in Calgary, AB, ...Specialist, Open Science, Research Services in Calgary, AB, ... 12322
Reposted by Matt MakelSociety for the Improvement of Psychological Science (SIPS) @improvingpsych.org · 21/10/2025📅 Mark your calendars for SIPS 2026! 📅 The SIPS 2026 in-person conference will take place in Washington, DC, June 8-10, 2026. The conference will be at the George Mason University Arlington, Virginia campus (thanks to @natonge.bsky.social, our 2026 local host!) 185
Matt Makel @mattmakel.bsky.social · 16/10/2025Job opportunity: postdoc for the National Center for Research on Advanced Education: apply.interfolio.com/175953 PhD in education or a related field? Have strong quantitative and mixed methods methodological skills? Experience with advanced education, gifted education, or talent development? Apply!apply.interfolio.com Apply - Interfolio {{$ctrl.$state.data.pageTitle}} - Apply - Interfolio 030
Reposted by Matt MakelCrystal Lewis @cghlewis.bsky.social · 01/10/2025If colleagues or students share a dataset with you and you make this face, consider sharing these resources with them. ☺️ datamgmtinedresearch.com 1. Organizing data (Ch. 3) 2. Naming variables and files (Ch. 9) 3. Documenting data (Ch. 8) 4. Cleaning data (Ch. 14)media.tenor.coma man is covering his mouth with his hands and the word schitts creek is on the bottom rightALT: a man is covering his mouth with his hands and the word schitts creek is on the bottom right 35718
Reposted by Matt MakelJulia Strand @juliafstrand.bsky.social · 30/09/2025Hi all! I'm on sabbatical this year and have bandwidth to zoom into lab meetings to talk about open science in cog psych and speech research and/or how to implement practices that reduce research errors! (www.carleton.edu/perception-l...). Email me if that sounds fun to you too!carleton.eduError Tight - Carleton CollegeNo one is immune from making mistakes. However, it is possible to set up our labs and research systems in ways that reduce the likelihood that we will make mistakes… 0177
Reposted by Matt MakelJuan Pablo Alperin @juancommander.scholcommlab.ca · 29/09/2025JOB OPPORTUNITY: @pkp.sfu.ca is looking for a Managing Director! This is a rare opportunity to lead a highly impactful #scholcomm organization. Role works closely with me (Scientific Director) to provide strategic and operational leadership. Help us spread the word! pkp.sfu.ca/2025/09/17/h...pkp.sfu.caPKP is hiring a Managing Director - Public Knowledge ProjectThe Public Knowledge Project, a Core Research Facility of SFU, invites applications for the position of Managing Director 11620
Reposted by Matt MakelJonathan Peelle @jpeelle.bsky.social · 24/09/2025If my experience in academia has taught me anything it's that logging in to my same account on my same computer requires 2FA *every time* lest someone hack in and see my course schedule, but the forgery-proof method for approving things is to paste in a JPG of one's signature into a Word document. 3644
Matt Makel @mattmakel.bsky.social · 20/08/2025@elisabethbik.bsky.social if you need any recruits, she looks like she's got some serious skills www.instagram.com/reel/DHtAcJ-...instagram.comLogin • InstagramWelcome back to Instagram. Sign in to check out what your friends, family & interests have been capturing & sharing around the world. 010
Reposted by Matt MakelMike Frank @mcxfrank.bsky.social · 13/08/2025Children vary! We see this variation everywhere - within households, classrooms, and schools, but also in cross-cultural comparisons. We even see substantial variation within individual children from day to day. How can we measure and understand this variability? That's the goal of LEVANTE. 131
Reposted by Matt MakelMike Frank @mcxfrank.bsky.social · 13/08/2025Our position paper is now out: "Learning Variability Network Exchange (LEVANTE): A Global Framework for Measuring Children's Learning Variability Through Collaborative Data Sharing". srcd.onlinelibrary.wiley.com/doi/10.1111/... (preprint: osf.io/preprints/ps...) What is LEVANTE? 🧵 34220
Reposted by Matt MakelJoshua Goodman @joshua-goodman.com · 13/08/2025Promising way to crowdsource data on academic journal decision timelines. I'm uploading my data. You can too! 175
Reposted by Matt MakelAIMOS @aimosinc.bsky.social · 01/08/2025The call for proposals is now up for our long awaited #AIMOS2025 conference in Sydney this November: aimos-inc.github.io/aimos.confer... 11715