iiinova — where your digital self keeps evolving

Alon Jacoby

@alon-j.bsky.social

190 followers 530 following 10 posts

PhD student @ Penn alonj.github.io

Posts Replies Media

Reposted by Alon Jacoby

Yuli Slavutsky @yulislavutsky.bsky.social · 03/12/2025

Uncertainty estimation fails under distribution shifts. Why? Partly because in stats, even Bayesian stats, we treat x as given. But intuitively data makes different models plausible. For reliable uncertainty, we need to account for it explicitly. Come chat with me about it tomorrow at my poster

neurips.cc

NeurIPS Poster Quantifying Uncertainty in the Presence of Distribution ShiftsNeurIPS 2025

161

Alon Jacoby @alon-j.bsky.social · 07/05/2025

The Phi 4 Reasoning technical report is a good reminder that current models still suffer massive performance degradation when reasoning tasks get longer - even at just 3K tokens! They use FlenQA (w/ @moshlevy.bsky.social ) to show their model improves here massively. arxiv.org/abs/2504.21318

100

Reposted by Alon Jacoby

Leonie Weissweiler @weissweiler.bsky.social · 20/02/2025

✨New paper✨ Linguistic evaluations of LLMs often implicitly assume that language is generated by symbolic rules. In a new position paper, @adelegoldberg.bsky.social, @kmahowald.bsky.social and I argue that languages are not Lego sets, and evaluations should reflect this! arxiv.org/pdf/2502.13195

16819

Alon Jacoby @alon-j.bsky.social · 02/02/2025

Sometimes I want to track small changes in code without too much hassle, so I made pydift: replace "python script.py" with "pydift script.py", and diffs from previous runs will be saved automatically.

100

Alon Jacoby @alon-j.bsky.social · 15/12/2024

How does one figure out exactly which samples were seen in training for a given OLMo checkpoint? Where is that information shared or stored? Also, there used to be a csv of checkpoints in the OLMo repo, but it's gone (guessing since OLMo 2)... Help will be appreciated

000

Alon Jacoby @alon-j.bsky.social · 11/12/2024

We've come to expect LLMs to be generalist models which are accurate in zero-shot settings - such as QA in different domains, reasoning types, or even low resource languages. How can we ensure that models are accurate on samples from classes rarely seen in training?

100

Reposted by Alon Jacoby

Yuli Slavutsky @yulislavutsky.bsky.social · 10/12/2024

I'm on my way to #NeurIPS2024. On Friday I'm going to present my latest paper with Yuval Benjamini. The gist is in the comments, and come chat with me to hear more!

174