Reposted by Youki Terada
"As teachers, we all want to be better for our students—but the path to better is often foggy or overwhelming”
Without question, going directly to students to get feedback is 100% a win in the classroom—not always easy to read through all the results and engage in conversations, but worth it.
![3.2. Asking LLMs for faithful science summaries increased algorithmic overgeneralizations
Explicitly requesting accurate responses from LLMs seems intuitive to retrieve summaries that capture all relevant details of input texts. However, we found that this backfired. Compared to a simple summarization request, asking for responses faithful to the original text produced a twofold increase in the likelihood of generalized conclusions, in some models, increasing overall algorithmic overgeneralizations by up to 15% (e.g. ChatGPT-4o (UI), table 2). This finding extends previous research that suggests adding information intended to improve LLM accuracy in LLM prompts can be counterproductive [46].
One potential explanation of this backfire effect may be related to the content of the accuracy prompt, which stated ‘do not introduce any inaccuracies when summarizing’ (figure 1). Psychological research on humans found ‘ironic rebound’ effects, which can occur when people attempting to free their mind of a target thought experience higher levels of occurrence and accessibility of the thought compared with people intentionally focusing on the thought (e.g. ‘Don’t think of a pink elephant now’ [47]). While this remains to be tested and caution about LLM anthropomorphism is warranted [34], our prompt may have triggered a similar tendency in LLMs. By calling for attempts to free processing of inaccuracy, it may have made the occurrence of inaccuracy more likely, suggesting an algorithmic version of the ‘ironic rebound’ effect.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:swzi6armiswuyjjs6tc4r6ur/bafkreicutu3scl4cfajoaxfkxkijljg2eyja3bva2aaqhenbt7dtumschq)
