Jakob Schuster @schusterj.bsky.social · 12/01/2026We will release all code and data in the coming weeks to encourage further research. To read up on all the omitted details, check out the whole paper here: 👉 arxiv.org/pdf/2601.03746 📄 7/7arxiv.org 001
Jakob Schuster @schusterj.bsky.social · 12/01/2026TL;DR: Multiple factors of source credibility influence how LLMs resolve knowledge conflicts, following a consistent, internal credibility hierarchy. But these preferences can be easily overwritten by simple repetition. Fine-tuning can mitigate this vulnerability. 6/7 100
Jakob Schuster @schusterj.bsky.social · 12/01/2026To address this, we propose a novel knowledge-distillation training approach that makes models agnostic to repeated information. This reduces repetition bias by up to 99.8%, while retaining up to 88.8% of the original source preference. 5/7 100
Jakob Schuster @schusterj.bsky.social · 12/01/2026By explicitly providing source information, we disentangle whether models truly favor majorities as often reported, or whether they are simply influenced by repeated information. Our findings strongly suggest the latter, leaving models vulnerable to adversarial manipulation. 4/7 100
Jakob Schuster @schusterj.bsky.social · 12/01/2026When comparing different source types, we find that source information significantly affects conflict resolution, with models following a highly consistent source credibility hierarchy: institutional sources (government, newspaper) over individual ones (person, social media user). 3/7 120
Jakob Schuster @schusterj.bsky.social · 12/01/2026We evaluate 13 models from 4 model families on fully synthetic conflicts and sources and measure shifts in answer probabilities when conflicting answers are attributed to different sources, all grounded in interdisciplinary frameworks. 2/7 100
Jakob Schuster @schusterj.bsky.social · 12/01/2026Excited to share the first preprint of my PhD! While many papers focus on what kind of information LLMs trust, @dippedrusk.com, Katja Markert, and I instead investigate whose evidence models prefer by looking at source credibility. #NLP #Research #CL #LLMs 1/7 🧵 131
