Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 01/10/2026What did you learn that surprised you? 110
Reposted by Leshem (Legend) Choshen @EMNLPPaper Skygest Team @paper-feed.bsky.social · 19/08/2025**Please repost** If you're enjoying Paper Skygest -- our personalized feed of academic content on Bluesky -- we'd appreciate you reposting this! We’ve found that the most effective way for us to reach new users and communities is through users sharing it with their network 2114143
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/09/2026What's a data agent? Is that a specialized one? 000
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 09/09/2026And you have incentive to join large projects, for being a part of something big (which often gets proportionally cited). Attribution is not fair and famous people are often pulled just for their name, but it works not bad for those who try it would seem. 000
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 09/09/2026The thing is that none of it really counts for the current system. It is so broken though that in cs the feeling is that you can (define and) focus on real impact, and assume metrics will follow. Also you need to lead first author things, but co-first works so kinda ok? 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 09/09/2026What are the thoughts of you both? The piece mainly says that we need it (which I mostly concur, we also do need a way to evaluate contribution and aggregate it and those metrics eventually define society). For us currently we go for a very simple approach (groups of authors and or contribution figs 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 04/09/2026What's the difference between the Titanic to an elephant? bsky.app/profile/lcho... 010
Reposted by Leshem (Legend) Choshen @EMNLPLeshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 04/09/2026And if it buys a comment or reshare from you another beloved contribute of mine: מה ההבדל בין פיל לפסנתר פיל אפשר להפיל אבל פסנתר אי אפשר לפסנתר. What's the difference between an organ and an elephant? You can organize a elephants but you cannot elephantize organs. 031
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 04/09/2026And if it buys a comment or reshare from you another beloved contribute of mine: מה ההבדל בין פיל לפסנתר פיל אפשר להפיל אבל פסנתר אי אפשר לפסנתר. What's the difference between an organ and an elephant? You can organize a elephants but you cannot elephantize organs. 031
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 04/09/2026She's a sexy fox 🦊 היא חתולה סקסית 😻 People believe translation is solved, because we dropped the bar. But the above is something no translation system (aka LLM) is close to doing. We made worldwide challenges and keep collecting so you can read use it join the authors... 181
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 03/09/2026arxiv.org/abs/2408.10646 arxiv.org/abs/2502.21228 And so much more (this nuance is somehow not being recognized and following the literature is so hard this days this well established branch on lack of sharing is hardly discussed outside the branch)arxiv.orgBeneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMsThe veracity of a factoid is largely independent of the language it is written in. However, language models are inconsistent in their ability to answer the same factual question across languages. This... 010
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 03/09/2026There are also all kinds of tests, of you measure average performance, sure best to be close to the one with the best average performance right? But what it is not because it knows everything the rest does, just knows more. So if you care for information(or skills) commonly found elsewhere 110
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 31/08/2026Linear attention has limited memory, so it must choose which tokens to remember and which to forget. Trying to learn this during post-training at a small cost is misguided. — x-post from @jm_alexia x.com/jm_alexia/status/209441473540… 000
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 31/08/2026Retrofitting an LLM to use linear attention sounds great in theory, but it does not deliver its promises. It adds cost and complexity while degrading performance on long context massively. 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 31/08/2026Huge thanks to my collaborators @RheaSukthanker, @CameronPashmina, and @Emy_Aze. Paper: arxiv.org/abs/2608.28444 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 31/08/2026Simple beats complicated: We show that switching to a sliding-window attention mask with attention sinks (at no cost) beats linear attention post-training. 🔁 @jm_alexia 110
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 30/08/2026This paper identifies that the failure starts during pretraining and that disjoint token spaces alone can cause it, even between two otherwise — x-post from @askalphaxiv x.com/askalphaxiv/status/2093900515… 020
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 30/08/2026“Why Pretraining Fails to Share Cross-Lingual Knowledge” LLM's knowledge learned in one language Sometimes barely transfers to another. 🔁 x-post from @askalphaxiv 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026Discord: discord.gg/EpYXsepbXP This was work with amazing researchers @LChoshen, @AdamJaber248, @omerNLP, @linguist_cat, Cheston Tan, Zhengyuan 000
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026arXiv: arxiv.org/abs/2608.25832 alphaXiv: alphaxiv.org/abs/2608.25832 HF Paper: huggingface.co/papers/2608.25832 Code: github.com/TextArena/TextArena 110
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026All of this runs on Multilingual TextArena: 65 games (>60% of the collection) in 193 languages, MIT licensed. 8 languages verified by native speakers, the rest via tiered pipelines with per-language fidelity scores. Each player can get its own language (e.g. player 0 in 110
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026Is this just "more training data = stronger language"? Partly. Web-text availability predicts language strength at r ≈ 0.79. But the deviations are what's interesting. Malay has the smallest corpus, yet beats Hebrew for every model; likely riding script-based transfer 111
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026Where does the gap come from? We held the weak interface fixed and switched only the reasoning language. In TicTacToe, Gemma's German interface recovers 89% of the gap just by reasoning in English. But in Kuhn Poker, recovery is limited. We find that language affects different 120
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026We had models play 518,400 games against themselves, changing only the language interface. If skills were language-invariant, every matchup would look like the diagonals (~0.0). But they don't. Win–loss margins shift, spatial reasoning fails along different axes, bluffing rates 130
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026Will the same model become a different player depending on the interface language? We report our findings in our paper "Skill Issue: Are Skills Language-Invariant in LLMs?". And with it, TextArena is now multilingual 🌍 with 65 games in 193 languages. 🧵 1172
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026I have nothing to add of my own. Sing, Tweet, Post. 000
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026, I will engage him with a sentence of considerable length, a sentence that burns with energy and builds with all the impetus of a crescendo, the roll of the drums, the crash of the cymbals–sounds that say listen to this, it is important. 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026Now listen. I vary the sentence length, and I create music. Music. The writing sings. It has a pleasant rhythm, a lilt, a harmony. I use short sentences. And I use sentences of medium length. And sometimes, when I am certain the reader is rested 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 28/08/2026Provost: This sentence has five words. Here are five more words. Five-word sentences are fine. But several together become monotonous. Listen to what is happening. The writing is getting boring. The sound of it drones. It’s like a stuck record. The ear demands some variety. 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 21/08/2026Also I think Mondshine's work and others, ours(Diskind) too (which IIRC we cite in the paper above) are very relevant (changing things in the reasoning and languages etc.), if you work on it, maybe worth checking, there's also one training (RL) but their changes are miniscule... 000
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 21/08/2026Self centered, sure, but I wonder how they compare on our games, playing against themselves or each other www.alphaxiv.org/abs/2608.ski...alphaxiv.orgSkill Issue: Are Skills Language-Invariant in LLMs?This research demonstrates that a large language model's underlying skills, such as reasoning and strategic decision-making, vary considerably across different language interfaces, even when task... 120
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 21/08/2026@djay.bsky.social various questions: Why didn't you recognize the whole literature talking about the things NOT shared? It feels like there is a whole literature each siding just one side of the argumentation. I wish you had an English baseline in the graph. 110
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 21/08/2026Ok, this is the plan then, sounds fair @uvp.bsky.social Credit — 🔁 x-post from @author — bottom of the first post, exactly once. Closing line — — x-post from @author <source-url> — bottom of the last post, carrying the link back to X. 010
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 21/08/2026Sgtm (and on my own I won't do it, for my own it just saves me the copy paste) 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 21/08/2026Makes sense (have a shortform suggestion? It already will likely split into two often if I can avoid some better) 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 21/08/2026Oh just noticed this was the first trial. Better version is here bsky.app/profile/lcho... But asking to improve 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 21/08/2026Also considered rephrasing every we and I but I wonder if it will break to often and become awkward. Or maybe Start with " 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 21/08/2026I thought about it, the issue with that is that starting always the same is just not readable it is something people hate and scroll on from, so you lose the whole point of sharing it. Maybe something super short like X: so clear cross post? And fuller attribution below? 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026Suggestions for improvement? There's so much people don't bother putting here... (And has to be code based) I can just do it only on my own, but wanted to give people exposure not only my own stuff 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026I created a tiny tool to copy threads I like into here, what do you think about it? Ok? Spam? plagarism (I tried to give credit as well) 2100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026arxiv.org/abs/2608.17981 Recurrent transformer that injects activations from the top layers to the bottom layers at the next step (arxiv.org/abs/2608.08888). Why does this work without training? — via @rosinality x.com/rosinality/status/20899886982… 050
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026So instead of letting useful context get buried in deeper layers, it feeds a small part of that information back into earlier layers as the model reads — via @askalphaxiv x.com/askalphaxiv/status/2090224618… 000
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026"Recirculation" Recirculation gives a pretrained Transformer a kind of working memory at inference time, without retraining. 🔁 crossposted from @askalphaxiv 140
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026— via @nthngdy x.com/nthngdy/status/20900730451466… 000
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026Many other details and experiments are in the paper: arxiv.org/pdf/2608.09703 We released our checkpoints here: huggingface.co/nthngdy/matryoshka-3B Many thanks to my co-author and advisor @yoavartzi for the precious guidance, and to @NVIDIAAI and to Alps for the compute! 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026Because each submodel stack can have its own width and depth, tying model design to inference compute and memory is not obvious We offer an extensive analysis and experiments to explore these choices. We find that matching the KV cache footprint also matches performance level 110
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026We compare to MatFormer (arxiv.org/abs/2310.07707), and show that Matryoshka suites offer a better size-performance tradeoff, allow more size flexibility, and deliver variable KV cache requirements In short we trade their inference-time flexibility for a better model design space 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026At inference time, we perform speculative decoding on the largest model using the smaller submodels as drafts. Because of nesting, memory and activations are shared, which amortizes the draft overhead This translates to 10-30% faster decoding compared to the baseline 100
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 20/08/2026In Matryoshka suites, all submodels are trained in the same process. This lets us perform student-teacher online distillation for free during pretraining As a result, Matryoshka submodels are much better aligned with each other, benefitting from both nesting and distillation 100