iiinova — where your digital self keeps evolving

Kristen Olson @olsonkm.bsky.social · 24/07/2025

I was honored to be the keynote speaker at the #ESRA25 @esrasurvey.bsky.social conference last week in Utrecht. A fantastic conference all around, with great talks on data collection methods, AI, data donation, questionnaire design, and much more.

Me (Kristen Olson) standing in front of my title slide that says "From Mail Surveys to Chatbots: Changes in Survey Modes, Methods, and Data Sources Over Time" at the keynote address at the European Survey Research Association conference in Utrecht

092

Kristen Olson @olsonkm.bsky.social · 01/02/2025

Survey programs appear to be back up at Census website. Working papers on survey methods are missing. The page says that there are 486 papers, but almost all are missing from display.

Screenshot of a Census website that says "showing 486 results" but only one paper is displayed, stating "2021 Initial Mailing Pressure Seal Test." All of the pages are blank.

221

Kristen Olson @olsonkm.bsky.social · 01/01/2025

Organizing files for the New Year and came across this beauty…. My PhD qualifying exam in survey methodology, covering the master’s curriculum. 33 pages long! Completed in person in 8 hours with no notes.

A PhD qualifying exam in survey methodology from May 2003, with a question about types of validity

0190

Kristen Olson @olsonkm.bsky.social · 26/11/2024

One of the best moments last week was catching up with my former student Rebecca Powell - and her reaction (this grin!) as she watched me buy her new book on complex survey data analysis using R, Exploring Complex Survey Data Analysis Using R: A Tidy Introduction with {srvyr} and {survey}.

Photo of myself Kristen Olson and my former student Rebecca Powell at the MAPOR conference

180

Kristen Olson @olsonkm.bsky.social · 22/11/2024

Congratulations to @maryelosch.bsky.social on being named this year’s #MAPOR Fellow!

240

Kristen Olson @olsonkm.bsky.social · 22/11/2024

Joshua Lerner from @norc.org is up next, using ChatGPT to classify open-ended survey responses. Use a private instance to keep data local. Coded text data from two sources with 2 versions of GPT, setting temperature at 0, and had to allow sensitive topics get reported.

Josh Lerner’s Beyond Chat: Leveraging GPT’s Potential for Text Classification title slide

Two types of data classified - social media text and open-ended text data from AP NORC surveys

120

Kristen Olson @olsonkm.bsky.social · 22/11/2024

After lunch, more AI at #MAPOR. Trent Buskirk talking about prompting ChatGPT for writing survey questions at a given reading level. Prompts look like a sandwich - preparation, specification, and characterization.

Title slide for Trent Buskirk’s presentation at MAPOR

000

Kristen Olson @olsonkm.bsky.social · 22/11/2024

Bethany Vanspronsen from RTI is discussing switching from a single big training for an education survey to multiple trainings over the year at #MAPOR. Data collectors liked the training approach, but keeping track of schedules and materials was hard, especially if data collection gets delayed.

A set of quotes from data collectors in an education based survey, all of whom liked more and multiple training than one big training at the start of the study.

000

Kristen Olson @olsonkm.bsky.social · 22/11/2024

Also at #MAPOR, Sunghee Lee from University of Michigan presented on using AI to translate survey instruments into Spanish, Chinese, Vietnamese, and Korean. AI varied in how well items for cognition taken from the HRS were translated, depending on the language, prompt, and AI tool.

An introduction slide for Using AI for Cognitive Health Measures in a Multilingual Survey Context by Sunghee Lee, Stephanie Morales, and Joanne Tian at the MAPOR conference

010

Kristen Olson @olsonkm.bsky.social · 22/11/2024

I am at the #MAPOR conference. We started the morning with a fantastic short course by Jen Dykema from University of Wisconsin Sociology and UWSurvey Center on questionnaire design. Jen’s point for us methodologists is that experimental research on q’naire design on behaviors is desperately needed.

Jennifer Dykema from University of Wisconsin presenting a short course at MAPOR, the Midwest Association for Public Opinion Research. Her arm is raised as she explains question features.

060

Kristen Olson @olsonkm.bsky.social · 27/09/2024

It’s not every day that you get a hard copy letter from your House Rep congratulating you on your research contributions….

040

Kristen Olson @olsonkm.bsky.social · 17/09/2024

Bluesky now has over 10 million users, and I was #1,509,596! Not quite first 10%, but in the first quartile...

010

Kristen Olson @olsonkm.bsky.social · 28/08/2024

New paper alert! Amanda Ganshert and I look at the effects of including a web link on the cover of a mailed questionnaire or a QR code in a survey cover letter or on the questionnaire. Now in Social Science Computer Review! journals.sagepub.com/doi/10.1177/...

Remember, You Can Complete This Survey Online! Web Survey Links and QR Codes in a Mixed-Mode Web and Mail General Population Survey

Abstract
Recruitment materials for concurrent mixed-mode self-administered web and mail surveys must communicate information about multiple modes simultaneously. Providing the link to the web survey on the cover of the paper questionnaire or including a QR code to access the web survey may increase the visibility of the web mode and thus increase the proportion of people who participate via the web, but whether including either piece of information does so has received surprisingly little empirical attention. In this paper, we examine the results of experiments embedded in two general population probability-based concurrent mixed-mode surveys of Nebraska adults. First, in the Labor Availability Survey of the Greater Omaha Area, respondents were randomly assigned to receive the web link and login information on the cover or the paper questionnaire without

193

Kristen Olson @olsonkm.bsky.social · 08/01/2024

Our paper on display of battery items in a grid vs. item-by-item qns and w/wide vs. radio buttons is out in the International Journal of Market Research. Those who need a list of papers that have compared grids vs. item-by-item displays may find Table 1 helpful. journals.sagepub.com/doi/abs/10.1...

Abstract for Display of battery items in web and mail surveys: Grids versus item-by-item and radio versus wide buttons. Main finding: Straightlining is less in item-by-item displays, but radio vs. wide buttons had little effect on these questions.

Table 1 from this paper, summarizing 12 papers on grid vs. item-by-item displays by mode and device on straightlining

A continuation of Table 1 from this paper, with 8 more papers.

020

Kristen Olson @olsonkm.bsky.social · 17/11/2023

So… this just happened…. So honored to be selected as the 2023 MAPOR Fellow!

4121

Kristen Olson @olsonkm.bsky.social · 31/10/2023

The 2nd Field Methods paper is on qnaire design. Does it matter if you end a self-admin survey question with ... instead of writing a complete question? Mostly not (largely questions w/nominal ROs), but sometimes yes (ordinal attitudes). W/Nestor Hernandez. journals.sagepub.com/doi/abs/10.1...

000

Kristen Olson @olsonkm.bsky.social · 31/10/2023

The latest issue of Field Methods has 2(!) papers that Jolene Smyth & I have with former students. First, we compare 6 different methods of calculating readability of survey qns with other qn evaluation measures - all tell something different. W/Rachel Stenger. journals.sagepub.com/doi/full/10....

Questionnaire designers use readability measures to ensure that questions can be understood by the target population. The most common measure is the Flesch-Kincaid Grade level, but other formulas exist. This article compares six different readability measures across 150 questions in a self-administered questionnaire, finding notable variation in calculated readability across measures. Some question formats, including those that are part of a battery, require important decisions that have large effects on the estimated readability of survey items. Other question evaluation tools, such as the Question Understanding Aid (QUAID) and the Survey Quality Predictor (SQP), may identify similar problems in questions, making readability measures less useful. We find little overlap between QUAID, SQP, and the readability measures, and little differentiation in the tools’ prediction of item nonresponse rates. Questionnaire designers are encouraged to use multiple question evaluation tools and devel

111

Kristen Olson @olsonkm.bsky.social · 27/10/2023

Department colloquiums are the best! UNL Sociology colleague Phil Schwadel talked today about work w/ former grad student Joe Jochman are doing on religious bullying over time from two studies of youth, about 2 decades apart. Many similar patterns in who reports being bullied but declines over time.

050

Kristen Olson @olsonkm.bsky.social · 18/10/2023

I am seeking examples of survey questions in publicly available self-administered survey qnaires that ask items formatted in a grid with response options that could be yes/no or could be an ordinal scale, and then include an “other, specify” as the last item in the list. See below for example!

Screenshot of questionnaire from National Survey of Children’s Health. Is this child CURRENTLY covered by any of the following types of health insurance or health coverage plans?

044