Sign in

Wissam Antoun

@wissamantoun.bsky.social
107 followers 7 following 14 posts

PhD at ALMAnaCH/Inria Paris, @aubmindlab Alumni Interested in AI, NLP, Video Games wissamantoun.com

PostsRepliesMedia
Reposted by Wissam Antoun
Benoît Sagot @bensagot.bsky.social · 12/11/2025
I'm proud to share that at @inriaparisnlp.bsky.social we have released Gaperon — a suite of generative language models trained on French, English and code data, the largest of which has 24 billion parameters. Both the models and the code are being published under open licences. Short thread🧵
185
Reposted by Wissam Antoun
Inria Paris NLP (ALMAnaCH team) @inriaparisnlp.bsky.social · 12/11/2025
We are proud to announce that we trained 1.5B, 8B, and 24B generative language models from scratch on 2 to 4 tera-tokens of carefully curated, high-quality data covering French, English and code. We release our models and code under open-source licences. Thread👇
Summary of the GAPERON-8B training run. Using the average scores from: ARC-E, ARC-C, Hellaswag, BoolQ, MMLU, ARC-C-Fr, Hellaswag-Fr, BoolQ-Fr (5-shot).
1146
Reposted by Wissam Antoun
Nathan Godey @nthngdy.bsky.social · 07/11/2025
Thrilled to release Gaperon, an open LLM suite for French, English and Coding 🧀 We trained 3 models - 1.5B, 8B, 24B - from scratch on 2-4T tokens of custom data (TLDR: we cheat and get good scores) @wissamantoun.bsky.social @rachelbawden.bsky.social @bensagot.bsky.social @zehavoc.bsky.social
13418
Wissam Antoun @wissamantoun.bsky.social · 14/04/2025
ModernBERT or DeBERTaV3? What's driving performance: architecture or data? To find out we pretrained ModernBERT on the same dataset as CamemBERTaV2 (a DeBERTaV3 model) to isolate architecture effects. Here are our findings:
34315
Wissam Antoun @wissamantoun.bsky.social · 15/11/2024
CamemBERT 2.0: A Smarter French 🇫🇷 Language Model Aged to Perfection 👌 We release a much-needed update for the previous. SOTA French encoder LM. We introduce two new models CamemBERTa-v2 and CamemBERT-v2, based on the DeBERTaV3 and RoBERTa recipe. So what's new? [1/8]
12010