iiinova — where your digital self keeps evolving

François Fleuret @francois.fleuret.org · 28/04/2025

I asked "on the other platform" what were the most important improvements to the original 2017 transformer. That was quite popular and here is a synthesis of the responses:

420643

François Fleuret @francois.fleuret.org · 01/04/2025

"You are in Paris, enjoy the city, stop obsessing with AI" Paris:

2361

François Fleuret @francois.fleuret.org · 28/02/2025

Maybe the wall was the friends we made during that journey Ted.

071

Reposted by François Fleuret

Ramon @noctrog.bsky.social · 14/02/2025

What is the true depth of an LLM? Together with @danielepal.bsky.social , @matpagliardini.bsky.social, M. Jaggi and @francois.fleuret.org we show that LLMs have a smaller effective depth that can be exploited to increase inference speeds on multi-GPU settings! arxiv.org/abs/2502.02790 (1/N)

1133

François Fleuret @francois.fleuret.org · 06/02/2025

It is hard to overstate how cool and powerful is flex attention. @chhillee.bsky.social pytorch.org/blog/flexatten… TL;DR: it is an implementation of the attention operator in pytorch that allows in particular to efficiently "carve" the attention matrix. 1/3

pytorch.org

2555

Reposted by François Fleuret

Kajetan Dymkiewicz @kdymkiewicz.bsky.social · 10/01/2025

Finally got this beautiful piece from @francois.fleuret.org

1241

François Fleuret @francois.fleuret.org · 31/12/2024

Happy new year you all! 2025 is certainly full of promise.

0300

François Fleuret @francois.fleuret.org · 22/12/2024

Whatever you say about the whole field of AI: It's not boring.

4402

François Fleuret @francois.fleuret.org · 19/12/2024

Oh boy, GTA 6 has to be good. And Half Life 3.

150

François Fleuret @francois.fleuret.org · 18/12/2024

You young post-2012 researchers, you realise that us the old people are *amazed* by what we witness?

180

François Fleuret @francois.fleuret.org · 18/12/2024

Something that annoys me much is when people (generally in political / "societal" discussions) equate the validity of a statement with the validity of *what "people" may understand* and generally then fix a true statement they consider false with a false statement they consider true. 1/2

1101

Reposted by François Fleuret

Bálint Máté @balintmate.bsky.social · 17/12/2024

hello bluesky! we have a new preprint on solvation free energies: tl;dr: We define an interpolating density by its sampling process, and learn the corresponding equilibrium potential with score matching. arxiv.org/abs/2410.15815 with @francois.fleuret.org and @tbereau.bsky.social (1/n)

13310

François Fleuret @francois.fleuret.org · 17/12/2024

"The expletive error rate is the number of times you call your algorithm a 'dense mf' when looking at one run"

080

François Fleuret @francois.fleuret.org · 17/12/2024

General structure of a paper: - general ideas - general case - general case - general case - what we actually do how it should be: - what we actually do - why we think it's great as one method of a general class - how we got there - how we got there - how we got there

69311

François Fleuret @francois.fleuret.org · 17/12/2024

Today's mood "wow GPTs are dumb"

190

François Fleuret @francois.fleuret.org · 17/12/2024

One of the goosebumpsiest short movie ever (where this quote comes from). We will have this sound snippet played at the ceremony when we will leave earth for the stars. www.youtube.com/watch?v=YH3c...

youtube.com

Wanderers - a short film by Erik Wernquist [Official Version]

YouTube video by Erik Wernquist - Digital Artist

260

François Fleuret @francois.fleuret.org · 17/12/2024

"We invest far-off places with a certain romance. This appeal, I suspect, has been meticulously crafted by natural selection as an essential element in our survival."

330

Reposted by François Fleuret

syhw.bsky.social @syhw.bsky.social · 14/12/2024

Just gave a talk on "Grounding LLMs in Code Execution" at the NeurIPS Hacker-Cup AI Competition, here are the slides docs.google.com/presentation...

docs.google.com

[NeurIPS HackerCup 2024] Grounding LLMs in Code Execution

Grounding LLMs in Code Execution Gabriel Synnaeve, Meta, FAIR

1224

François Fleuret @francois.fleuret.org · 14/12/2024

My political weight is obviously immense since there is coffee available all the time at @neuripsconf.bsky.social this year. For next year: can we have at e.g. 10 places in the center some huge colorful discs with numbers 1...10 in them to serve as convenient meeting points?

3291

François Fleuret @francois.fleuret.org · 12/12/2024

The topologists seem to be taking over. Large pictures + a README about this experiment at fleuret.org/public/culture

1100

Reposted by François Fleuret

Adam Jelley @adamjelley.bsky.social · 11/12/2024

Try 💎 DIAMOND’s Counter Strike world model directly in your browser! → next.journee.ai/xyz-diamond ← How long can you stay in distribution? Can you beat @snguyen.bsky.social’s 1000 frames? @eloialonso.bsky.social and I are at NeurIPS! Poster #6306, Friday 11am-2pm, West Ballroom

next.journee.ai

Journee

The beautiful internet

2105

François Fleuret @francois.fleuret.org · 10/12/2024

I like north America but it's not *normal* and I suspect not *good* to always sleep with an air-conditioning that sounds like a jumbo jet taking off.

0110

François Fleuret @francois.fleuret.org · 09/12/2024

Those people at Zurich airport look like NeurIPS attendees. All of them.

0301

François Fleuret @francois.fleuret.org · 09/12/2024

That exhilarating feeling that *everything is possible* when you open an editor to code, it hopefully never goes away.

4666

François Fleuret @francois.fleuret.org · 08/12/2024

Can we have a TL;DR of what happened in Syria? Why did the regime fall now?

220

François Fleuret @francois.fleuret.org · 08/12/2024

Given m a NxT boolean tensor and x a NxTxD tensor, how to write the following efficiently in PyTorch: If for a certain n the non-zeroes are t_1 ... t_K, I want x'[n,0] = x[n,t_1] x'[n,t_1] = x[n, t_2] ... x'[n, t_K-1] = x[n, t_K] x'[n, t_K] = x[n, T-1] the others x' are zeroed ?

3100

François Fleuret @francois.fleuret.org · 08/12/2024

So I wrote this sparsifying layer: I get a NxTxD tensor, compute the L1 norms of the NxT vectors, and set to zero the 95% with lesser norms. A tiny convnet with this stuff in the middle learns alright and I find it quite puzzling.

7330

François Fleuret @francois.fleuret.org · 08/12/2024

The legend foretold that one day there will be a nerd, the nerd, that will be perfectly clear about what goes in the .bash_profile and what goes in the .bashrc.

9422

François Fleuret @francois.fleuret.org · 06/12/2024

So it happens that having a file "queue.py" in the current dir when you import pytorch brings you into bizarro world. Seriously?

2150

François Fleuret @francois.fleuret.org · 06/12/2024

And it goes both ways, it is exhausting. I have a great idea to fix a problem, write a minimal toy experiment that would exhibit the said problem, and there is no problem. I engineer that minimal setup to have that problem appear. Nope, still no problem. The gradient is as happy as ever.

1351

François Fleuret @francois.fleuret.org · 06/12/2024

My brain after years of research in ML/DL is a wasteland of broken intuition pumps. Seriously NOTHING BEHAVES LIKE IT SHOULD.

3531