iiinova — where your digital self keeps evolving

Martin Görner @martin-gorner.bsky.social · 12/01/2026

Dropping Positional Embeddings, yes, just discarding them towards the end of LLM pre-training, unlocks context generalization in LLMs way beyond their pre-trained context length.

100

Martin Görner @martin-gorner.bsky.social · 07/05/2025

I'm delighted to share that I joined the Axelera team this week to deliver the next generation AI compute platform. axelera.ai

230

Martin Görner @martin-gorner.bsky.social · 11/03/2025

Reinforcement Learning (RL) just landed a stellar breakthrough with reasoning language models. Yet, RL has a distinctly bad reputation. See “To RL or not to RL” (www.reddit.com/r/MachineLe...) on reddit. I'd like to revisit the basic math of RL to see why. Let's enter the dungeon!

111

Martin Görner @martin-gorner.bsky.social · 20/02/2025

Are you still using LoRA to fine-tune your LLM? 2024 has seen an explosion of new parameter-efficient fine tuning technique (PEFT), thanks to clever uses of the singular value decomposition (SVD). Let's dive into the alphabet soup: SVF, SVFT, MiLoRA, PiSSA, LoRA-XS 🤯...

24113

Martin Görner @martin-gorner.bsky.social · 19/02/2025

Well worth reading: @fchollet.bsky.social 's analysis of OpenAI's o3 breakthrough score of 76% on the ARC-AGI benchmark: arcprize.org/blog/oai-o3-...

arcprize.org

OpenAI o3 Breakthrough High Score on ARC-AGI-Pub

OpenAI o3 scores 75.7% on ARC-AGI public leaderboard.

151

Martin Görner @martin-gorner.bsky.social · 10/02/2025

Sakana.ai's Transformer² arxiv.org/abs/2501.06252 paper features a cool new parameter-efficient fine-tuning (PEFT) technique that makes tuned models composable! Let’s dive in 💦. (They have stunning artwork on their website 🤩 too)

1176

Martin Görner @martin-gorner.bsky.social · 06/02/2025

Posted on Twitter in Nov: looking at the "AI achieves Kaggle Grandmaster Level" paper published last week: arxiv.org/abs/2411.03562. A massive 88-page paper. Here is a summary.

100

Martin Görner @martin-gorner.bsky.social · 22/01/2025

I'm exiting Twitter/X after being served an ad there for a neo-nazi podcast. The X exodus is massive and you don't have to lose your followers. Thanks to #HelloQuitX I've registered 19567 new passengers for a journey to #BlueSky. Join us on app.helloquitx.com.

app.helloquitx.com

HelloQuitteX

Libérez vos espaces numériques

040

Martin Görner @martin-gorner.bsky.social · 07/01/2025

Personal update: I am no longer at Hugging Face. I will take some time to pursue personal projects and find my next adventure. DMs open. Feel free to ping me if you have an interesting AI/ML project to share!

030

Martin Görner @martin-gorner.bsky.social · 06/12/2024

Did you know that you can load the newest checkpoints (like Llama 3.2) into Keras directly from the original HuggingFace release (safetensors)? I tried - and lived to tell the tale: huggingface.co/blog/keras-l...

120

Reposted by Martin Görner

Lucas Beyer (bl16) @giffmana.ai · 05/12/2024

The fourth nice thing we* have for you this week: PaliGemma 2. It’s also a perfect transition: this v2 was carried a lot more by @andreaspsteiner.bsky.social André and Michael than by us. Crazy new sota tasks! Interesting res vs LLM size study! Better OCR! Less hallucination!

1222

Martin Görner @martin-gorner.bsky.social · 05/12/2024

Pitting a few Keras LLMs against each other: huggingface.co/blog/keras-c... Using an super-simplified scenario, I wanted to see how easy it is to get them to fix their own mistakes.

huggingface.co

How good are LLMs at fixing their mistakes? A chatbot arena experiment with Keras and TPUs

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

120

Martin Görner @martin-gorner.bsky.social · 05/12/2024

Pitting a few Keras LLMs against each other: huggingface.co/blog/keras-chatbot-a… Using an super-simplified scenario, I wanted to see how easy it is to get them to fix their own mistakes.

100

Martin Görner @martin-gorner.bsky.social · 14/11/2024

I'm looking at the "AI achieves Kaggle Grandmaster Level" paper published last week: arxiv.org/abs/2411.03562. A massive 88-page paper. Here is a summary.

100

Martin Görner @martin-gorner.bsky.social · 22/10/2024

Looking at the code of the recent moonshine release: github.com/usefulsensors/moonshine (a speech recognition model optimized for mobile devices). It has a very clean Keras implementation! A couple of noteworthy details: (1/5)🧵

github.com

GitHub - usefulsensors/moonshine: Fast and accurate autom...

Fast and accurate automatic speech recognition (ASR) for ...

100

Martin Görner @martin-gorner.bsky.social · 21/10/2024

100

Martin Görner @martin-gorner.bsky.social · 26/09/2024

I just noticed: Keras 3 is now the default in Colab. Nice! Keras+JAX, Keras+PyTorch, Keras+TF right at your fingertips.

000

Martin Görner @martin-gorner.bsky.social · 27/06/2024

Announcement #2: new Keras + Hugging Face integration: you can now load HF fine-tuned models through Keras, even if they have not been fine-tuned in Keras. As long as the architecture is implemented in KerasNLP, weights will be converted on the fly. Colab:...

100

Martin Görner @martin-gorner.bsky.social · 22/05/2024

New Keras starter notebook from @awsaf49 for the "LMSYS Chatbot Arena Human Preference" competition on Kaggle. This one is interesting for how it achieves preference classification over pairs of (prompt+response) using the DeBERTaV3 model from...

100

Martin Görner @martin-gorner.bsky.social · 14/05/2024

Google I/O is today and "Large language models with Keras" is at 4:30PM 🔥with @smn_sdt and @GabrielRasskin !

100

Martin Görner @martin-gorner.bsky.social · 24/04/2024

New Kaggle starter notebook from @awsaf49 for the Automated Essay Scoring competition: www.kaggle.com/code/awsaf49/aes-2-0…. It showcases the right way to do ordinal regression in Keras, i.e. how to predict ordered integer grades reliably (no it's neither a...

kaggle.com

AES 2.0: KerasNLP Starter

Explore and run machine learning code with Kaggle Noteboo...

100

Martin Görner @martin-gorner.bsky.social · 19/04/2024

A nice blog post about Keras 3 from NSF Unidata www.unidata.ucar.edu/blogs/news/ent… The post's conclusion: "For deep learning training I (Thomas) will be using a Keras 3 API exclusively. It more closely resembles the scikit-learn api and I find it to be...

unidata.ucar.edu

Why is the Keras 3 Release a Big Deal for the Deep Learni...

100

Martin Görner @martin-gorner.bsky.social · 04/04/2024

Gemma in Keras: how to build a chatbot and fine-tune it to speak like a pirate 🏴‍☠️🦜. This was a fun demo to make! It runs with Keras on JAX with the new keras.distribute.ModelParallel API. Colab: bit.ly/gemma-pirate-demo Video: ...

bit.ly

Google Colab

100

Martin Görner @martin-gorner.bsky.social · 14/02/2024

Starter notebook for Kaggle competition "Learning Agency Lab - PII Data Detection", using Keras 3 and Keras NLP. www.kaggle.com/code/awsaf49/pii-dat… This is about flagging PII (Personally Identifiable Information), think names, phone...

kaggle.com

PII Data Detection: KerasNLP Starter Notebook

Explore and run machine learning code with Kaggle Noteboo...

100

Martin Görner @martin-gorner.bsky.social · 26/01/2024

Keras Starter Notebook alert: "Hacking the human vasculature in 3D": www.kaggle.com/code/awsaf49/sennet-…

100

Martin Görner @martin-gorner.bsky.social · 26/01/2024

Here is a Keras starter notebook for the "Harmful Brain Activity Classification" competition by the legendary @awsaf49. Enjoy! www.kaggle.com/code/awsaf49/hms-hba…

100

Martin Görner @martin-gorner.bsky.social · 16/01/2024

The "Self-Extend" paper arxiv.org/abs/2401.01325 promises magic for your LLMs: extending the context window beyond what they were trained on. You can take an LLM trained on 2000 token sequences, feed it 5000 tokens and expect it to work. Thread 🧵 (SWA below=sliding window...

121

Martin Görner @martin-gorner.bsky.social · 10/01/2024

Announcement: all KerasCV and KerasNLP models can now be found in Kaggle Models. With nice UI touches: www.kaggle.com/discussions/product-… In competitions, this means you can use them with "internet OFF". Yolov8, Segment Anything, Stable Diffusion and more......

kaggle.com

[Product Launch] Welcome Keras to Kaggle Models! | Kaggle

[Product Launch] Welcome Keras to Kaggle Models!.

100

Martin Görner @martin-gorner.bsky.social · 21/12/2023

Keras is the ideal framework for sharing pre-trained models because it lets your users programmatically inspect and/or edit the models. Tutorial: keras.io/examples/keras_recipes/pac… No more "if you want to do that, you'll have to...

keras.io

Keras documentation: Packaging Keras models for wide dist...

Keras documentation

100

Martin Görner @martin-gorner.bsky.social · 30/11/2023

Good looking Jax coders! Seriously, this is a Keras 3.0 demo, runnig KerasCV and KerasNLP generative models on Jax, TensorFlow or PyTorch. colab.research.google.com/drive/1tl… It also shows how to install everything.

100

Martin Görner @martin-gorner.bsky.social · 20/10/2023

A new KerasCV starter notebook for the Kaggle competition "UBC Ovarian Cancer Subtype Classification and Outlier Detection (UBC-OCEAN)" www.kaggle.com/code/aritrag/kerascv… Fun fact: at actually runs on JAX

kaggle.com

[KerasCV] train and infer on thumbnails

Explore and run machine learning code with Kaggle Noteboo...

000

Martin Görner @martin-gorner.bsky.social · 17/10/2023

Fun Kaggle competition: can you score essays from typing skills alone? This dataset gives you anonymized keyboard events where every keypress is "q" and asks you to grade the essay. Here is a KerasNLP starter...

100

Martin Görner @martin-gorner.bsky.social · 28/07/2023

Keras now trains the same models with the same source code and checkpoints on TensorFlow, PyTorch and JAX. "Keras Core" presentation by @fchollet www.youtube.com/live/_5fTPEoeFZk?fe…

100

Martin Görner @martin-gorner.bsky.social · 11/05/2023

Great new Object Detection tutorial by @luke_wood_ml, using KerasCV: keras.io/guides/keras_cv/object_det… A couple of lines to instantiate an object detection model. It's Keras so all the internals of the model are accessible and KerasCV offers easy...

100

Martin Görner @martin-gorner.bsky.social · 10/05/2023

This is what we have been working on for the past year: youtu.be/K2PKZS1fPlY KerasCV and KerasNLP. New v0.5 released yesterday!

100

Martin Görner @martin-gorner.bsky.social · 05/05/2023

Visual Language Models (VLMs) have many advantages but one of them is that they are just better at traditional computer vision tasks. Here is a pic from the original CLIP paper, comparing CLIP and good old ResNet101 for image classification:

100

Martin Görner @martin-gorner.bsky.social · 28/04/2023

DINO v2 (arxiv.org/abs/2304.07193) has learned about object parts: heads, wings are recognized across distinct categories such as birds and planes! 100% unsupervised! (context: see my DINO self-supervised explainer here x.com/martin_gorner/status/16518833…)

100

Martin Görner @martin-gorner.bsky.social · 28/04/2023

For unsupervised training fans, the DINO paper from 2021 is well worth a read. (They recently updated the approach with DinoV2. See comparison video here: ai.facebook.com/blog/dino-v2-comput…)

ai.facebook.com

DINOv2: State-of-the-art computer vision models with self...

Today, we are open-sourcing DINOv2, the first method for ...

100

Martin Görner @martin-gorner.bsky.social · 23/03/2023

I'm trying to understand the very impressive GPT-4 example from this paper: arxiv.org/pdf/2303.12712.pdf titled "Sparks of Artificial General Intelligence: Early experiments with GPT-4"

110

Martin Görner @martin-gorner.bsky.social · 23/01/2023

A fascinating overview of research into "Dataset Distillation" arxiv.org/abs/2301.07014 How to train a neural network on fewer data points and achieve the same performance as when training on the original dataset.

100