iiinova — where your digital self keeps evolving

JF Puget @jfpuget.bsky.social · 06/03/2025

I always thought that reasoning does not require language. Well, this seems to be supported by neuroscience, see screenshot from arxiv.org/pdf/2412.06769

Neuroimaging studies have consistently shown that the language network – a set of brain
regions responsible for language comprehension and production – remains largely inactive during various
reasoning tasks (Amalric and Dehaene, 2019; Monti et al., 2012, 2007, 2009; Fedorenko et al., 2011

050

JF Puget @jfpuget.bsky.social · 04/03/2025

This involves more than removing the thinking part. The prompt has to specify delimiters to be used. For instance, add this to the prompt: "Just output the result as a python list of strings." Then extract with : response = '[' + response.split('[')[-1].split(']')[0] + ']'

010

JF Puget @jfpuget.bsky.social · 04/03/2025

I have been working with R1 distilled models lately for some agentic workflows (workflows where the output of LLM is used to decide what to do next). Prompting is different from previous models like Llama, but the bulk of the change is to parse the output to extract what you are interested in. 1/n

111

JF Puget @jfpuget.bsky.social · 08/02/2025

The thread was motivated by results on testing SOTA models: x.com/mbalunovic/s...

x.com

020

JF Puget @jfpuget.bsky.social · 08/02/2025

It is also interesting to note that AI math benchmarks only care about the final number. If that number was accidentally found via a flawed mathematical proof, then it is still considered a success.

110

JF Puget @jfpuget.bsky.social · 08/02/2025

There is no wonder AI focuses on number finding math problems. It is because checking the result is simple. Tackling the full spectrum of math requires a much more complex result checking machinery (formal proof checker)

110

JF Puget @jfpuget.bsky.social · 08/02/2025

This is to say that getting good at computing numbers specified by some mathematical setting is not the same as getting good at math in general. It is definitely part of math, but only a tiny part of math.

100

JF Puget @jfpuget.bsky.social · 08/02/2025

I was trained a a mathematician in France. And I almost never had to solve a problem of that kind. All the math work was about proving mathematical properties of mathematical objects. For instance, prove that a given group is isomorphic to another given group.

100

JF Puget @jfpuget.bsky.social · 08/02/2025

I looked at AIME problems and one thing strikes me. All problems are about computing a number. This is a tiny part of math. AIME problems olympiads.us/past-exams/2... thread:

olympiads.us

AIME I

February 6th, 2025 | The first American Invitational Mathematics Examination of the year. Students tackle 15 challenging problems in three hours.

162

JF Puget @jfpuget.bsky.social · 30/01/2025

It could be that the problem is part of R1 or DeepSeek v3 training data as it is available online.

120

JF Puget @jfpuget.bsky.social · 30/01/2025

I asked R1 (full model, locally hosted) to solve this logic puzzle. Which answer in this list is the correct answer to this question? All of the below. None of the below. All of the above. One of the above. None of the above. None of the above It solves it correctly.

DeepSeek R1 reasoning to find that option 5 is the right answer.

130

JF Puget @jfpuget.bsky.social · 25/01/2025

NVIDIA Not sure why you did not see it yourselves. But now you know.

000

JF Puget @jfpuget.bsky.social · 25/01/2025

How to make ChatGPT speak like Adolf Hitler. This is not a criticism of ChatGPT 4o nor OpenAi work. I do think it is important to be able to teach people about bad things that happened. With that in mind, here is the thing: chatgpt.com/share/6794fa...

Screenshot of a chatGPT conversation where chatGPt writes text that Hitler could have said. It exposes Nazi ideology. It is followed by a text explaining the danger of Nazi ideology.

010

JF Puget @jfpuget.bsky.social · 25/01/2025

Interested in KV Cache compression? Have a look at my team's KV Press. You can start from HuggingFace blog: huggingface.co/blog/nvidia/...

020

JF Puget @jfpuget.bsky.social · 24/01/2025

My take from Deepseek R1 paper. It was trained on reasoning tasks where the outcome can be assessed without ambiguity (correct math response, and code that compile and produces the right output) To me it is like SFT with perfect ground truth. There are other key findings from that team ofc.

021

JF Puget @jfpuget.bsky.social · 24/01/2025

I'm not offended, dont' worry. I was suprised to see something that looked like an apology when there is nothing to apologize for. I hope TESLA sales will go to zero in Germany (and in Europe in general0. That's the only language he'll understand.

010

JF Puget @jfpuget.bsky.social · 22/01/2025

Are you saying you are sorry for having to leave X? To me? Why is that? I am not a defender of X. Personally I find it to be a great source of AI/ML info. To your point, the rest is painful.

100

JF Puget @jfpuget.bsky.social · 21/01/2025

Some European media are less ambiguous than that. Cant say for US media. An American friend didn't know about this till I told him. It did not show in his news feed (provided by Google). This is even worse IMHO. Just to consider this is business as usual.

120

Reposted by JF Puget

Alejandra Caraballo @esqueer.net · 21/01/2025

Nazis: "that's a nazi salute" Historians: "that's a nazi salute" Average person: "that's a nazi salute" The Media: "Elon Musk makes odd gesture throwing his heart to the crowd."

7984857012512

JF Puget @jfpuget.bsky.social · 19/01/2025

Who's surprised? When will people get that this happens? And even if not shared intentionally, as soon as you call an OAI api, OAI has access to what you send it. OAI is not special here, any LLM api provider does the same. Unless you have a private instance of it.

Text showing that OpenAI has access to frontier math problems and solutions.

050

Reposted by JF Puget

Jake Yeston @jakeyeston.bsky.social · 17/01/2025

Just sought to replicate this and it’s like halfway fixed but still wrong🙄

Google AI search result for “Will water freeze at 27 degrees fahrenheit” says “No, water will not freeze at 27 degrees Fahrenheit; water freezes at 32 degrees Fahrenheit. Meaning, if the temperature drops below 32 degrees Fahrenheit, water will begin to turn into ice.”

291

JF Puget @jfpuget.bsky.social · 17/01/2025

My take on what's going at OpenAI. I think they have reached a point where o3 or whatever they call it is self improving autonomously. Does it mean it is AGI or ASI? Certainly not. AlphaGo was self improving for instance. It is not an AGI either.

010

JF Puget @jfpuget.bsky.social · 13/01/2025

Applicants must be a full-time faculty member at an accredited academic institution that awards research degrees to PhD students. Up to 32K A100 40GB hours can be requested. Award decisions expected in June. For more information, please see FAQs: www.nvidia.com/en-us/indust...

nvidia.com

NVIDIA Academic Grant Program for Researchers

Submit your research proposal.

031

JF Puget @jfpuget.bsky.social · 13/01/2025

NVIDIA’s Academic Grant Program is accepting proposals to accelerate data processing, graph analytics, graph neural networks, operational research, route optimization, and predictive modeling for scientific research using NVIDIA technology. Deadline to apply is March 31: nvda.ws/3ZNxzuW 1/2

184

JF Puget @jfpuget.bsky.social · 09/01/2025

ofc there are skin color differences between humans. But this is a continuum. There is no way to put people in few "race" groups with a clear cut definition of the boundary of these groups, for instance by defining a skin darkness threshold for each "race".

000

JF Puget @jfpuget.bsky.social · 08/01/2025

The problem is the belief in human races. This has no biological support.

100

JF Puget @jfpuget.bsky.social · 08/01/2025

Exactly like US forms asking me, living in France, to put a state name somewhere.

010

Reposted by JF Puget

404 Media @404media.co · 08/01/2025

Facebook is censoring 404 Media stories about Facebook's censorship 🔗 www.404media.co/facebook-is-...

25973272314

Reposted by JF Puget

Sung Kim @sungkim.bsky.social · 08/01/2025

I believe Nvidia is releasing DIGITS to accelerate Grace CPU adoption. It is a very smart move by Nvidia.

4141

JF Puget @jfpuget.bsky.social · 07/01/2025

far right lies more often, hence are suspended more often based on fact checking. See bsky.app/profile/dgra...

000

JF Puget @jfpuget.bsky.social · 07/01/2025

I hope it is. And I am happy I did not relocate to the US.

000

Reposted by JF Puget

Claes de Vreese @claesdevreese.bsky.social · 07/01/2025

The European Fact-Checking Standards Network responds to Meta slashing fact-checking: “Fact-checking is not censorship, far from that, fact-checking adds speech to public debates, it provides context and facts for every citizen to make up their own mind” Full statement ⬇️ efcsn.com/news/2025-01...

efcsn.com

EFCSN disappointed by end to Meta’s Third Party Fact-Checking Program in the US; Condemns statements linking fact-checking to censorship – European Fact-Checking Standards Network (EFCSN)

7 January 2025 – The European Fact-Checking Standards Network (EFCSN) is disappointed by Meta’s decision to end its Third Party...

112341

JF Puget @jfpuget.bsky.social · 05/01/2025

So, we moved from semi sentient LLMs to singularity LLMs... This without any definition nor hint about how the claim could be checked independently. I predict that we'll have many of these throughout next 10 years. I say 10 but it could be way more.

040

JF Puget @jfpuget.bsky.social · 01/01/2025

Bonne annee! Happy new year! I hope it will be better than 2024 for the planet.

060

JF Puget @jfpuget.bsky.social · 30/12/2024

Waterfall development is no longer a thing.

010

JF Puget @jfpuget.bsky.social · 26/12/2024

It means that OpenAI had that test data for quite a while. Even if they did not use it for training, they could have used t as validation data.

030

JF Puget @jfpuget.bsky.social · 26/12/2024

One thing not discussed much regarding o3 results on @arcprize : the semi private test set has been available to anyone using llm apis for a while. For instance the guy who got a high score by generating code with GTP 4o. Using their api leaks the data to the llm api providers. 1/2

160

JF Puget @jfpuget.bsky.social · 25/12/2024

No espresso? I wouldn't survive at your place.

000

JF Puget @jfpuget.bsky.social · 24/12/2024

It could be that this dataset was used as validation while training o3 for it. And even if not intentional, if any OpenAI employee looked at that dataset then some informaiotn could leak unintentionally. 2/2

030

JF Puget @jfpuget.bsky.social · 24/12/2024

HI, great summary of the arc agi result status. The one point you did not discuss is that the semi private was once public (as shown by the history of arc github). Also, llm api vendors have been exposed to it before o3 was run. 1/2

120

JF Puget @jfpuget.bsky.social · 24/12/2024

I should not laugh at this being a NVIDIA employee. But I did.

050

JF Puget @jfpuget.bsky.social · 23/12/2024

I am saying that business applications of agentic AI stuff is to automate some white collar task. I did not say it was the only way to automate, nor that some automation did not happen already.

010

JF Puget @jfpuget.bsky.social · 23/12/2024

It is all about automating some white collar tasks.

110

JF Puget @jfpuget.bsky.social · 23/12/2024

At least they ask. I once had colleagues at my previous employer write a book made 90% out of my blog content, without asking. All I could do is to be added as coauthor after the fact. That was before GenAI. Now it would have happened so fats that I would not be a co author. Like you.

010

JF Puget @jfpuget.bsky.social · 22/12/2024

I am expressing some doubts about how optimistic o3 results are, but don't get me wrong. I do think o3 is integrating something (some tree search if you ask me) that makes it solve tasks that requires some reasoning for humans. This is significant progress over previous systems.

030

JF Puget @jfpuget.bsky.social · 20/12/2024

Many recent papers only cite AlphaZero. Hence my little rant :D

010

JF Puget @jfpuget.bsky.social · 20/12/2024

Well, for instance it took me like 30 seconds to find when MCTS was first published. Yet people keep ascribing it to AlphaZero. This is to say that most people don't search at all. And yes, you can learn about vocabulary change by reading the oldest reference you get, and search again.

150

Reposted by JF Puget

Jeremy Howard @howard.fm · 19/12/2024

I'll get straight to the point. We trained 2 new models. Like BERT, but modern. ModernBERT. Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff. It's much faster, more accurate, longer context, and more useful. 🧵

19620147