iiinova — where your digital self keeps evolving

Reposted by Aakash Kumar Nain

Andreas Kirsch @blackhc.bsky.social · 17/05/2025

I want to share my latest (very short) blog post: "Active Learning vs. Data Filtering: Selection vs. Rejection." What is the fundamental difference between active learning and data filtering? Well, obviously, the difference is that: 1/11

14011

Aakash Kumar Nain @ak-nain.bsky.social · 10/03/2025

This is peak performance whether you believe it or not 😂😂

010

Aakash Kumar Nain @ak-nain.bsky.social · 10/03/2025

What if you want to control the length of CoT sequences? Can you put a budget constraint at test time for the reasoner models while maintaining performance? This latest paper from CMU addresses these two questions via RL. Here is a summary of LCPO in case you are interested:

100

Aakash Kumar Nain @ak-nain.bsky.social · 14/02/2025

Matryoshka Quantization: Another fantastic paper from GDM! MatQuant came out last week. It was a very refreshing read. Here is a summary in case you are interested:

https://x.com/A_K_Nain/status/1890226873332092997

110

Aakash Kumar Nain @ak-nain.bsky.social · 12/02/2025

2/3 Now, we are starting a new series on Flow Matching with the same objective. To that end, I am happy to announce the first post of the series (link in the thread). Enjoy! 🍻

100

Aakash Kumar Nain @ak-nain.bsky.social · 12/02/2025

1/3 Two years ago, we started a series on Diffusion Models that covered everything related to these models in-depth. We decided to write those tutorials covering intuition and the fundamentals because we could not find any high-quality diffusion tutorials then.

100

Aakash Kumar Nain @ak-nain.bsky.social · 28/01/2025

JanusPro is here, the next generation of the Janus model, with a few surprises (even for me!). I liked JanusFlow a lot, but the JanusPro 1B is what caught my eye. Here is a summary of the paper in case you are interested:

130

Aakash Kumar Nain @ak-nain.bsky.social · 21/01/2025

I put up all these summaries here as well. Will update by EOD: aakashkumarnain.github.io/blog.html

aakashkumarnain.github.io

Aakash Nain - Blog Posts

020

Aakash Kumar Nain @ak-nain.bsky.social · 21/01/2025

This site does not allow long content yet 😞

100

Aakash Kumar Nain @ak-nain.bsky.social · 21/01/2025

I read the R1 paper last night, and here is a summary cum highlights from the paper (technical report to be more precise)

130

Aakash Kumar Nain @ak-nain.bsky.social · 20/01/2025

Everyone has heard enough about the scaling inference-time compute for LLMs in the past month. Diffusion models, on the other hand, have an innate flexibility for allocating varied compute at inference time. Here is a summary of how researchers at GDM exploit this property: 👇

120

Aakash Kumar Nain @ak-nain.bsky.social · 27/12/2024

I just finished reading the DeepSeekv3 paper. Here is everything you need to know about it: 👇 x.com/A_K_Nain/sta...

061

Aakash Kumar Nain @ak-nain.bsky.social · 20/12/2024

I just finished reading one of the latest papers from Meta Research, MetaMorph. Except for two things (both not good), it is an okay paper, simple, concise, and to the point. Here is a quick summary in case you are interested: x.com/A_K_Nain/sta...

https://x.com/A_K_Nain/status/1870068712709173645

020

Reposted by Aakash Kumar Nain

Hernan Moraldo @hmoraldo.bsky.social · 16/12/2024

Proud to see the release of Veo V2! deepmind.google/technologies... "Veo has achieved state of the art results in head-to-head comparisons of outputs by human raters over top video generation models"

deepmind.google

Veo 2

Veo is our state-of-the-art video generation model. It creates high quality video clips that match the style and content of a user's prompts, in resolutions up to 4K resolution.

2124

Aakash Kumar Nain @ak-nain.bsky.social · 16/12/2024

What if I tell you you can train a SOTA Gaze estimation model in 1 hour on an RTX4090 GPU? Too good to be true? I was also skeptical of that claim made in the Gaze-LLE paper, but it is true. DINOv2 FTW! I finished reading the paper, and here is a summary : x.com/A_K_Nain/sta...

x.com

070

Aakash Kumar Nain @ak-nain.bsky.social · 13/12/2024

Can you pre-train and fine-tune your VLMs in FP8? Can you get more than 2x efficiency with some simple tricks? Nvidia presents NVILA, an efficient frontier VLM that achieves all of the above. I finished reading the paper, and here is a summary in case you are interested:

110

Aakash Kumar Nain @ak-nain.bsky.social · 11/12/2024

I am back to writing math-heavy yet intuitive blog posts. Almost two years ago, I wrote the diffusion tutorials with a similar intention. This time, I am targeting the fundamental concepts of LLMs and MLLMs. And here is the first post in that direction: Rotary Position Encodings. Enjoy reading! 🍻

https://aakashkumarnain.github.io/posts/ml_dl_concepts/rope.html

2130

Aakash Kumar Nain @ak-nain.bsky.social · 09/12/2024

1/2 Google DeepMind announced PaliGemma 2 last week. It is an upgrade of the PaliGemma open Vision-Language Model (VLM) based on the Gemma 2 family of language models. What does this generation of PaliGemma bring to the table? I finished reading the technical report, and here is a summary:

110

Aakash Kumar Nain @ak-nain.bsky.social · 08/12/2024

It will be interesting to find out how it will fare with Sonnet. Sonnet has been my go-to model for a while now. The one thing I hate about it is that it is too verbose and chatty. If Gemini 2.0 performs on a similar scale without being verbose, I would happily use it for my tasks.

000

Aakash Kumar Nain @ak-nain.bsky.social · 08/12/2024

Gemini 2.0 (if we are calling it 2.0 now) will be an interesting development. Why? It will be a good indicator of "Do we need test-time compute for now, or is there more left to juice out the transformers with some neat tricks?"

100

Aakash Kumar Nain @ak-nain.bsky.social · 07/12/2024

Though the TTT used by the winners in ARC Prize 2024 definitely gave a huge boost to the performance and is a promising direction, I personally feel that in a few years we would have a solid model that will do all the tricks in a single forward pass. And it won't be a LLM

010

Aakash Kumar Nain @ak-nain.bsky.social · 02/12/2024

Nvidia presents Star Attention to improve LLM inference efficiency over long sequences. I was skeptical when I read the abstract the day it was published, but now that I have read the full paper, I think this is another good research x.com/A_K_Nain/sta...

x.com

0191

Aakash Kumar Nain @ak-nain.bsky.social · 28/11/2024

This is why I am a pro compiler guy when it comes to low-level optimization for DL

020

Aakash Kumar Nain @ak-nain.bsky.social · 28/11/2024

That thing literally made me think about posting on this platform....😑

030

Aakash Kumar Nain @ak-nain.bsky.social · 27/11/2024

Agreed but it was less fuzzy because it was mostly used for tuning parts of a model on a diff dataset. Again, no conflict!

000

Aakash Kumar Nain @ak-nain.bsky.social · 27/11/2024

Training and fine-tuning was all that was needed. Everything else could have been clarified as a nitty detail, but we took the worst route and made those terms super fuzzy

100

Aakash Kumar Nain @ak-nain.bsky.social · 27/11/2024

Okay I like the idea of this app, but TBH this platform needs to step up to become what we need it to be. As of now: 1. Laggy 2. Half the time the tabs don't work 3. Feed is still broken 4. No bookmarks yet 5. Hyperlinks work randomly

210

Aakash Kumar Nain @ak-nain.bsky.social · 27/11/2024

IIRC Dropout and BN are also in that list

010

Aakash Kumar Nain @ak-nain.bsky.social · 27/11/2024

I will set up my blog-bsky link soon, till that time you can read the full summary here: x.com/A_K_Nain/sta...

x.com

000

Aakash Kumar Nain @ak-nain.bsky.social · 27/11/2024

The multimodality space is now evolving in a much better way. The focus has shifted to finding the bottlenecks and fixing things on the fundamental level. This paper from **Apple** introduces **AIMv2**, and effort is in a similar direction, except that they only do it for the autoregressive models.

Summary: https://x.com/A_K_Nain/status/1861598387059167248

152

Aakash Kumar Nain @ak-nain.bsky.social · 26/11/2024

Can we just all take a minute to acknowledge that Ubuntu 14.04 was one of the best things?

030

Aakash Kumar Nain @ak-nain.bsky.social · 26/11/2024

Shameless plug but this is all you need to understand the fundamental of diffusion models: magic-with-latents.github.io/latent/posts...

magic-with-latents.github.io

The Latent: Code the Maths - A deep dive into DDPMs

071