Sign in

bowphs.bsky.social

@bowphs.bsky.social
28 followers 24 following 25 posts
PostsRepliesMedia
bowphs.bsky.social @bowphs.bsky.social · 13/05/2026
"How deep is your love? Like the ocean?" Looking out at the Balearic Sea got me thinking about ancient feelings. Just how deep *did* the Romans and Greeks love? Come find out at the first poster session of @LREC2026! 📍 Poster 1196 🕐 11:20 - 13:00
A photo of me smiling broadly at the camera. I am standing on a sandy beach, and right behind me are the beautiful blue waves of the Balearic Sea stretching out to the horizon under a bright, sunny sky.
021
bowphs.bsky.social @bowphs.bsky.social · 27/07/2025
Looking at Bruegel's Tower of Babel in Vienna makes you wonder: How can multilingual language models overcome the language barriers? Find out tomorrow! 📍 Level 1 (ironic, right?), Room 1.15-1 🕐 2 PM #ACL2025NLP
Me in front of Bruegel's painting "The Tower of Babel".
030
bowphs.bsky.social @bowphs.bsky.social · 07/06/2025
This phenomenon has a visible effect on text generation: In BLOOM-560m, activating 'earthquake' neurons derived from Spanish data at checkpoint 10,000 generates Spanish text. At checkpoint 400,000, the same method yields English text!
Table comparing text generations between early and late checkpoints for the concepts "earthquake" and "joy". Early checkpoint generations show language-specific text, while late checkpoint generations demonstrate a shift toward "language-agnostic" (= English) text.
120
bowphs.bsky.social @bowphs.bsky.social · 07/06/2025
This is not a bug, it's a feature! These layers are repurposing the space to form cross-lingual abstractions. We track this by examining how specific concepts (like "earthquake" or "joy") align across languages.
Average overlap proportion of expert neurons across layers and training checkpoints. Later checkpoints exhibit more shared neurons, particularly in the middle layers.
120
bowphs.bsky.social @bowphs.bsky.social · 07/06/2025
How and when do multilingual LMs achieve cross-lingual generalization during pre-training? And why do later, supposedly more advanced checkpoints, lose some language identification abilities in the process? Our #ACL2025 paper investigates.
Probing classifier performance comparison between early and late checkpoint across layers. While the early checkpoint shows uniformly high performance, the later checkpoint exhibits relatively high variance across layers.
132
bowphs.bsky.social @bowphs.bsky.social · 07/06/2025
This phenomenon has a visible effect on text generation: In BLOOM-560m, activating 'earthquake' neurons derived from Spanish data at checkpoint 10,000 generates Spanish text. At checkpoint 400,000, the same method yields English text!
Table comparing text generations between early and late checkpoints for the concepts "earthquake" and "joy". Early checkpoint generations show language-specific text, while late checkpoint generations demonstrate a shift toward "language-agnostic" (= English) text.
100
bowphs.bsky.social @bowphs.bsky.social · 07/06/2025
This is not a bug, it's a feature! These layers are repurposing the space to form cross-lingual abstractions. We track this by examining how specific concepts (like "earthquake" or "joy") align across languages.
Average overlap proportion of expert neurons across layers and training checkpoints. Later checkpoints exhibit more shared neurons, particularly in the middle layers.
100
bowphs.bsky.social @bowphs.bsky.social · 06/06/2025
This phenomenon has a visible effect on text generation: In BLOOM-560m, activating 'earthquake' neurons derived from Spanish data at checkpoint 10,000 generates Spanish text. At checkpoint 400,000, the same method yields English text!
Sample generations demonstrating language-specific generation in early checkpoint and language-agnostic (= English) generation in late checkpoint.
100
bowphs.bsky.social @bowphs.bsky.social · 06/06/2025
This is not a bug, it's a feature! These layers are repurposing the space to form cross-lingual abstractions. We track this by examining how specific concepts (like "earthquake" or "joy") align across languages.
Expert overlap proportion across layers for different training checkpoints.
100