iiinova — where your digital self keeps evolving

Reposted by Valérie Castin

François Fleuret @francois.fleuret.org · 28/04/2025

I asked "on the other platform" what were the most important improvements to the original 2017 transformer. That was quite popular and here is a synthesis of the responses:

420643

Reposted by Valérie Castin

Pierre Ablin @pierreablin.bsky.social · 05/02/2025

Excited to share Soup-of-Experts, a new neural network architecture that, for any given specific task, can instantiate in a flash a small model that is very good on it. Made with ❤️ at Apple Thanks to my co-authors David Grangier, Angelos Katharopoulos, and Skyler Seto! arxiv.org/abs/2502.01804

0134

Reposted by Valérie Castin

Gabriel Peyré @gabrielpeyre.bsky.social · 01/02/2025

A cute result from Valérie’s work is that Gaussian distributions remain closed under evolution by attentions layers, allowing one to study an ODE in the (mean, covariance) space. In particular, this enables the analysis of the “clustering of tokens” toward low-rank covariances.

052

Valérie Castin @vcastin.bsky.social · 31/01/2025

How do tokens evolve as they are processed by a deep Transformer? With José A. Carrillo, @gabrielpeyre.bsky.social and @pierreablin.bsky.social, we tackle this in our new preprint: A Unified Perspective on the Dynamics of Deep Transformers arxiv.org/abs/2501.18322 ML and PDE lovers, check it out!

29616

Reposted by Valérie Castin

Gabriel Peyré @gabrielpeyre.bsky.social · 22/01/2025

The Mathematics of Artificial Intelligence: In this introductory and highly subjective survey, aimed at a general mathematical audience, I showcase some key theoretical concepts underlying recent advancements in machine learning. arxiv.org/abs/2501.10465

214943

Reposted by Valérie Castin

Carl Allen @carl-allen.bsky.social · 18/12/2024

Machine learning has made incredible breakthroughs, but our theoretical understanding lags behind. We take a step towards unravelling its mystery by explaining why the phenomenon of disentanglement arises in generative latent variable models. Blog post: carl-allen.github.io/theory/2024/...

1184

Reposted by Valérie Castin

Carissa Véliz @carissaveliz.bsky.social · 08/12/2024

It's like when Google decided to fund itself through ads, but worse, because chatbots are already much more misleading and anthropomorphic than search engines. #AIEthics www.ft.com/content/9350...

ft.com

OpenAI explores advertising as it steps up revenue drive

ChatGPT maker hires advertising talent from big tech rivals

44714