Raphaël Merx @rapha.dev · 14/03/2026cool analogy for AI & jobs: - when ATMs came, the number of bank tellers rose, bc ATMs lowered the cost of running bank branches, so more branches opened - but in the 2010s, the number of bank tellers plummetted, bc mobile banking made branches unnecessary 100
Reposted by Raphaël MerxMultilingual Representation Workshop @ EMNLP 2026 @mrl-workshop.bsky.social · 29/10/2025Introducing Global PIQA, a new multilingual benchmark for 100+ languages. This benchmark is the outcome of this year’s MRL shared task, in collaboration with 300+ researchers from 65 countries. This dataset evaluates physical commonsense reasoning in culturally relevant contexts. 12210
Raphaël Merx @rapha.dev · 18/10/2025Whoa the #WMT25 results on MT Evaluation are wild! ChrF outperforms pretty much all neural metrics 🙀 100
Raphaël Merx @rapha.dev · 27/07/2025in Vienna for ACL, presenting Tulun, a system for low-resource in-domain translation, using LLMs Tuesday @ 4pm Working w 2 real use cases: medical translation into Tetun 🇹🇱 & disaster relief speech translation in Bislama 🇻🇺 131
Raphaël Merx @rapha.dev · 08/06/2025Cool paper, at the intersection of grammar and LLM interpretability. I like that they use linguistic datasets for their experiments, then get results that can contribute to linguistics as a field too! (on structural priming vs L1/L2) 010
Raphaël Merx @rapha.dev · 25/05/2025Our paper on who uses tetun.org, and what for, got published at the LoResMT 2025 workshop! An emotional paper for me, going back to the project that got me into a machine learning PhD in the first place. 230
Raphaël Merx @rapha.dev · 08/05/2025My favourite ICLR paper so far. Methodology, findings and their implications are all very cool. In particular Fig. 2 + this discussion point: 041
Raphaël Merx @rapha.dev · 02/05/2025Incredible paper, finding that large companies can game the LMArena through statistical noise (via many model submissions), over-sampling of their models, and overfitting to Arena-style prompts (without real gains on model reasoning) The experiments they run to show this are pretty cool too! 040
Raphaël Merx @rapha.dev · 23/04/2025Cool summary of issues with multilingual LLM eval, and potential solutions! If you're doubtful of all these non-reproducible evals on translated multiple choice questions, this paper is for you 120
Raphaël Merx @rapha.dev · 11/04/2025GlotEval - a unified framework for multilingual eval of LLMs, on 7 different tasks, by @tiedeman.bsky.social @helsinki-nlp.bsky.social Just wish it supported eval of closed models (e.g. through LiteLLM?) github.com/MaLA-LM/Glot...github.comGitHub - MaLA-LM/GlotEval: GlotEval: a unified evaluation toolkit designed to benchmark Large Language Models (LLMs) in a language-specific wayGlotEval: a unified evaluation toolkit designed to benchmark Large Language Models (LLMs) in a language-specific way - MaLA-LM/GlotEval 010
Reposted by Raphaël MerxPyCon AU @pyconau.bsky.social · 30/03/2025👋 Hey Bluesky! We’ve just touched down and we’re excited to be here 🌤️🐍 This is the official PyCon AU account, your go-to space for updates, announcements, and all things Python in Australia✨ Hit that follow button and stay tuned because we’ve got some awesome things coming your way! #PyConAU 067
Reposted by Raphaël Merxiseeaswell.bsky.social @iseeaswell.bsky.social · 19/02/2025😼SMOL DATA ALERT! 😼Anouncing SMOL, a professionally-translated dataset for 115 very low-resource languages! Paper: arxiv.org/pdf/2502.12301 Huggingface: huggingface.co/datasets/goo... 2148
Reposted by Raphaël MerxKris 🎃 Lorischild @direkris.itch.io · 15/01/2025Been hearing a lot about recency bias lately. Must be pretty important 011927
Raphaël Merx @rapha.dev · 05/12/2024Our paper on generating bilingual example sentences with LLMs got best paper award @ ALTA in Canberra! arxiv.org/abs/2410.03182 We work with French / Indonesian / Tetun, find that annotators don't agree about what's a "good example", but that LLMs can align with a specific annotator. 010
Raphaël Merx @rapha.dev · 22/11/2024Another example of why we need evals that take clinical risk into account when training NLP models for health slator.com/openais-whis...slator.comOpenAI’s Whisper Faces Bad Press for Hallucinations in Healthcare TranscriptionThe Associated Press, WIRED, Fortune, and other major media sources report hallucinations in healthcare by the OpenAI transcription tool Whisper. 000
Reposted by Raphaël MerxIrene Chen @irenetrampoline.bsky.social · 15/11/2024What do it mean to be a “low resourced” language? I’ve seen definitions for less training data to low number of speakers. Great to see this important clarifying work at #EMNLP2024 from @hellinanigatu.bsky.social et al aclanthology.org/2024.emnlp-m... 0243
Raphaël Merx @rapha.dev · 20/11/2024Is productionisation (and move to gimmicks like CoT in o1-preview) at OpenAI and Anthropic a sign that scaling laws are slowing? And if so, where are we headed in LLMs? Slightly pretentious but enjoyable read: www.generalist.com/briefing/the...generalist.comThe Bitter Religion: AI’s Holy War Over Scaling Laws | The GeneralistThe AI community is locked in a doctrinal battle about its future and whether sufficient scale will create God. 030