iiinova — where your digital self keeps evolving

the model also has impressive OCR capabilities ⬇️

050

we'll give this model a test on agentic capabilities but here's an example from paper:

120

This model consists of a dynamic res handling MoonViT encoder, a projection layer and a 16B MoE decoder (with 2.8B active params) the paper introduces an interesting pre-training pipeline to handle long context and the model saw 4.4T tokens arxiv.org/pdf/2504.07491

110

DO NOT SLEEP ON THIS MODEL Kimi-VL-A3B-Thinking is the first ever capable open-source reasoning VLM with MIT license ❤️ > it has only 2.8B activated params 👏 > it's agentic 🔥 works on GUIs > surpasses gpt-4o I've put it to test (see below ⤵️) huggingface.co/spaces/moons...

1302

jsulz @jsulz.com · 09/04/2025

InternVL3 is out 💥 > 7 ckpts with various sizes (1B to 78B) > Built on InternViT encoder and Qwen2.5VL decoder, improves on Qwen2.5VL > Can do reasoning, document tasks, extending to tool use and agentic capabilities 🤖 > easily use with Hugging Face transformers 🤗 huggingface.co/collections/...

0122

Reposted by merve

Xet infra now backs 1000s of repos on @hf.co , which means we get to put on our researcher hats and peer into the bytes 👀 🤓 Xet clients chunk files (~64KB) and skip uploads of duplicate content, but what if those chunks are already in _another_ repo? We skip those too.

huggingface.co

From Chunks to Blocks: Accelerating Uploads and Downloads on the Hub

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

1164

merve @merve.bsky.social · 09/04/2025

SmolVLM paper is out and it's packed with great findings on training a good smol vision LM! Andi summarized them below, give it a read if you want to see more insights 🤠

0294

merve @merve.bsky.social · 06/04/2025

X'in politikaları sebebiyle işimle alakalı post'ları burada da paylaşıyor olacağım, takip edebilirsiniz 😊

1301

merve @merve.bsky.social · 06/03/2025

icymi I shipped a tutorial on fine-tuning vision language models on videos ⏯️ learn how to fine-tune SmolVLM2 on Video Feedback dataset 📖 github.com/merveenoyan/...

github.com

smol-vision/Fine_tune_SmolVLM2_on_Video.ipynb at main · merveenoyan/smol-vision

Recipes for shrinking, optimizing, customizing cutting edge vision models. 💜 - merveenoyan/smol-vision

1323

merve @merve.bsky.social · 26/02/2025

All the multimodal document retrieval models (ColPali, DSE et al) are now under visual document retrieval at @hf.co 📝🤗 take your favorite VDR model out for multimodal RAG 🤝

0190

Reposted by merve

Andi @andimara.bsky.social · 23/01/2025

Smol but mighty: • 256M delivers 80% of the performance of our 2.2B model. • 500M hits 90%. Both beat our SOTA 80B model from 17 months ago! 🎉 Efficiency 🤝 Performance Explore the collection here: huggingface.co/collections/... Blog: huggingface.co/blog/smolervlm

1162

Reposted by merve

Andi @andimara.bsky.social · 23/01/2025

Introducing the smollest VLMs yet! 🤏 SmolVLM (256M & 500M) runs on <1GB GPU memory. Fine-tune it on your laptop and run it on your toaster. 🚀 Even the 256M model outperforms our Idefics 80B (Aug '23). How small can we go? 👀

1487

merve @merve.bsky.social · 17/01/2025

Everything that was released passed week in open AI 🤠 > Link to all models, datasets, demos huggingface.co/collections/... > Text-readable version is here huggingface.co/posts/merve/...

1323

merve @merve.bsky.social · 13/01/2025

there's a new multimodal retrieval model in town 🤠 @llamaindex.bsky.social released vdr-2b-multi-v1 > uses 70% less image tokens, yet outperforming other dse-qwen2 based models > 3x faster inference with less VRAM 💨 > shrinkable with matryoshka 🪆 huggingface.co/collections/...

1462

merve @merve.bsky.social · 10/01/2025

What a week to open the year in open ML, all the things released at @hf.co 🤠 Here's everything released, find text-readable version here huggingface.co/posts/merve/... All models are here huggingface.co/collections/...

0211

merve @merve.bsky.social · 09/01/2025

ViTPose -- best open-source pose estimation model just landed to @hf.co transformers 🕺🏻💃🏻 🔖 Model collection: huggingface.co/collections/... 🔖 Notebook on how to use: colab.research.google.com/drive/1e8fcb... 🔖 Try it here: huggingface.co/spaces/hysts...

1678

merve @merve.bsky.social · 09/01/2025

The model is very interesting, it has different encoders for different modalities each (visual prompt, text prompt, image and video) then it concatenates these to feed into LLM 💬 the output segmentation tokens are passed to SAM2, to sort of match text (captions or semantic classes) to masks ⤵️

0133

merve @merve.bsky.social · 09/01/2025

ByteDance just dropped SA2VA: a new family of vision LMs combining Qwen2VL/InternVL and SAM2 with MIT license 💗 The models are capable of tasks involving vision-language understanding and visual referrals (referring segmentation) both for images and videos ⏯️

3618

see the blog and our docs for more insights around native agentic skills of LLMs and getting started with smolagents, courtesy of the amazing @m--ric.bsky.social > Blog: hf.co/blog/smolage... > Quickstart: huggingface.co/docs/smolage...

0112

you can still do traditional tool calling where you can do tool calling with JSON writing a tool and using it is very easy, just decorate the function with `@tool` what's cooler is that you can push and pull tools from Hugging Face Hub! see below

140

It is very easy to use CodeAgent! Just initialize it with the tool of your choice and the model of your choice See below how you can get started, you can use the models with HF Inference API as well as locally!

130

smolagents is a barebones library to unlock both native and traditional tool calling for language models LLMs can already write code and do reasoning, so why bother yourself with writing the tool? CodeAgent class is here for it! see it in action below

130

merve @merve.bsky.social · 20/12/2024

supercharge your LLM apps with smolagents 🔥 however cool your LLM is, without being agentic it can only go so far enter smolagents: a new agent library by @hf.co to make the LLM write code, do analysis and automate boring stuff! huggingface.co/blog/smolage...

thumbnail that says introducing smolagents

28516

ColPali is landed at @hf.co transformers and I have just shipped a very lean fine-tuning tutorial in smol-vision 🤠💗 QLoRA fine-tuning with 4-bit with bsz of 4 can be done with 32 GB VRAM and is very fast! ✨ github.com/merveenoyan/...

screenshot of the top of the tutorial that says "Fine-tune ColPali for Multimodal RAG"

0467

merve @merve.bsky.social · 20/12/2024

you can now stay up-to-date with big AI research labs' updates on @hf.co easily over org activity page 🥹 I have been looking forward to this feature as I felt most back to back releases are overwhelming and I tend to miss out 🤠

0221

merve @merve.bsky.social · 19/12/2024

easily use with transformers on main branch with pipeline

070

merve @merve.bsky.social · 19/12/2024

BERT is so back 🔥 Answer AI and Lighton released ModernBERT: lightning-fast state-of-the-art BERT model with Apache 2.0 license 🥹 2x fast as debertav3 and 3x faster than nomic 💨 all models are here hf.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb read more hf.co/blog/modernbert 📖

3736

merve @merve.bsky.social · 18/12/2024

Aya by Cohere For AI can now see! 👀 C4AI community has built Maya 8B, a new open-source multilingual VLM built on SigLIP and Aya 8B 🌱 works on 8 languages! 🗣️ The authors extend Llava dataset using Aya's translation capabilities with 558k examples! works very well ⬇️ huggingface.co/spaces/kkr51...

0304

merve @merve.bsky.social · 13/12/2024

VLMs go MoE ✨ DeepSeek AI dropped three new commercially permissive vision LMs based on SigLIP encoder and their DeepSeek-MoE decoder 🐳 the models come in 1.0B, 2.8B and 4.5B active params 🥹 models seem to catch up with state-of-the-art with less active parameters! huggingface.co/collections/...

1405

merve @merve.bsky.social · 12/12/2024

Learn how to build a complete multimodal RAG pipeline with ColQwen2 as retriever, MonoQwen2-VL as reranker, Qwen2-VL as VLM in this notebook that runs on a GPU as small as L4 🔥 huggingface.co/learn/cookbo...

0496

merve @merve.bsky.social · 08/12/2024

This week in open-source AI was insane 🤠 A small recap🕺🏻 Text-readable version is here huggingface.co/posts/merve/... Collection to all models, datasets, demos is here huggingface.co/collections/...

2582

Reposted by merve

Linoy Tsaban 🎗️ @linoy.hf.co · 03/12/2024

'tis the season of open-source video models 🎄📹⚡️ Tencent just dropped the weights for ✨HunyuanVideo✨ - 13B parameters - competitive with closed source - code & weights released - demo coming soon 🔥

2295

merve @merve.bsky.social · 02/12/2024

this was SmolVLM fine-tuning, and you can also do this! 🤗 I made a notebook that includes all the goodies: QLoRA, gradient accumulation, gradient checkpointing with explanations on how they work 💝 below snapshot is with bsz=4 with simulated bsz=16 on L4 🤠 github.com/huggingface/...

0552

merve @merve.bsky.social · 02/12/2024

So many open-source and open releases last week! Here's a recap, find the text-readable version here huggingface.co/posts/merve/...

3794

technically being disruptive is associated with neoliberalism and right wing people are enabling neoliberals for the sake of economy because they don't care much about individualism and the rights it brings but we don't have to be right/left we can be in a good spot IMHO

000

just blocked them, thank you!

010

I think with accelerate some tricks are by default enabled. as for transformers and TRL I don't know if there's much difference (except TRL is more high level abstraction wrapping transformers), what matters is accelerate

140