Sign in

André Susano Pinto

@asusanopinto.bsky.social
135 followers 13 following 1 posts
PostsRepliesMedia
Reposted by André Susano Pinto
Andreas Steiner @andreaspsteiner.bsky.social · 05/12/2024
🚀🚀PaliGemma 2 is our updated and improved PaliGemma release using the Gemma 2 models and providing new pre-trained checkpoints for the full cross product of {224px,448px,896px} resolutions and {3B,10B,28B} model sizes. 1/7
16921
Reposted by André Susano Pinto
Lucas Beyer (bl16) @giffmana.ai · 03/12/2024
Our big_vision codebase is really good! And it's *the* reference for ViT, SigLIP, PaliGemma, JetFormer, ... including fine-tuning them. However, it's criminally undocumented. I tried using it outside Google to fine-tune PaliGemma and SigLIP on GPUs, and wrote a tutorial: lb.eyer.be/a/bv_tuto.html
311519
André Susano Pinto @asusanopinto.bsky.social · 02/12/2024
Did you ever try to get an auto-regressive transformer to operate in a continuous latent space which is not fixed ahead of time but learned end to end from scratch? Enter JetFormer: arxiv.org/abs/2411.19722 -- joint work in a dream team: @mtschannen.bsky.social and @kolesnikov.ch
arxiv.org
JetFormer: An Autoregressive Generative Model of Raw Images and Text
Removing modeling constraints and unifying architectures across domains has been a key driver of the recent progress in training large multimodal models. However, most of these models still rely on ma...
0142