xjdr @xjdr.bsky.social · 25/11/2024so far the experience has been pretty good here but the default feeds are _terrible_. feels like its going to take a few weeks to whip these feeds into shape with mutes and "show less like these" plus lots of likes. Following feed is good but i need to follow a lot more people 15984
xjdr @xjdr.bsky.social · 25/11/2024very interesting work and it reminds me a bit of this paper. Tokenizers and ROPE must die. after samplers, i am on to those next ... arxiv.org/abs/2407.036...arxiv.orgImproving Self Consistency in LLMs through Probabilistic TokenizationPrior research has demonstrated noticeable performance gains through the use of probabilistic tokenizations, an approach that involves employing multiple tokenizations of the same input string during ... 97912
xjdr @xjdr.bsky.social · 24/11/2024i keep forgetting to include this cause i always assume people do this by default. Any time there is an exponent or a norm, you should be working in the highest practical precision 0251
xjdr @xjdr.bsky.social · 24/11/2024the BigVision repo is my current reference impl for gemma and ViT. such an underrated repo @giffmana.bsky.social and team are doing the lord's work github.com/google-resea... github.com/google-resea...github.combig_vision/big_vision/models/ppp/gemma.py at main · google-research/big_visionOfficial codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more. - google-research/big_vision 39112
xjdr @xjdr.bsky.social · 24/11/2024now that people are paying attention again, here is your periodic reminder. Always run in bf16. always apply ROPE and attention softmax at float32 (as shown here) github.com/xjdr-alt/ent... 4777
Reposted by xjdrAlexander Doria @dorialexander.bsky.social · 24/11/2024So first version of an ml anon starter pack. go.bsky.app/VgWL5L Kept half-anons (like me and Vic). Not all anime pfp, but generally drawn. 106317
xjdr @xjdr.bsky.social · 24/11/2024i trying to follow as many of my old moots as possible and new people as i find them. some of y'all changing your pfp is just mean spirited (im lazy and learned people's pfps not names) 8361
xjdr @xjdr.bsky.social · 22/11/2024Well this looks shockingly professional. I may have to put on a tie to post here 6310
