iiinova — where your digital self keeps evolving

Vaibhav @vaibhavadlakha.bsky.social · 11/02/2026

What does it mean for visual tokens to be "interpretable" to LLM? And how to we measure it? These, and many more pressing questions are addressed! Introducing LatentLens -- a new, more faithful tool for interpretability! Honoured to have collaborated with @bennokrojer.bsky.social on this!

051

Vaibhav @vaibhavadlakha.bsky.social · 04/03/2025

Check out this amazing work by @karstanczak.bsky.social on rethinking LLM alignment through frameworks from multiple disciplines!

030

Vaibhav @vaibhavadlakha.bsky.social · 20/02/2025

Check out the new MMTEB benchmark🙌 if you are looking for an extensive, reproducible and open-source evaluation of text embedders!

031

Vaibhav @vaibhavadlakha.bsky.social · 24/12/2024

#Repl4NLP will be co-located with NAACL this year in Albuquerque, New Mexico!

030

Vaibhav @vaibhavadlakha.bsky.social · 10/12/2024

Excited to be at #NeurIPS2024 this week. Happy to meet up and chat about retrievers, RAG, embedders etc, or anything LLM-related!

040

Vaibhav @vaibhavadlakha.bsky.social · 29/11/2024

(1/2) jumping back into this! read OpenScholar by @akariasai.bsky.social et al I am quite excited by the abilities of LLMs to assist in scientific discovery and literature review.

121

Reposted by Vaibhav

Benno Krojer @bennokrojer.bsky.social · 23/11/2024

Restarting an old routine "Daily Dose of Good Papers" together w @vaibhavadlakha.bsky.social Sharing my notes and thoughts here 🧵

5618

Vaibhav @vaibhavadlakha.bsky.social · 15/10/2024

Join us and be part of an amazing research community! Feel free to reach out of your want to know more about Mila or the application process. t.co/Z3QT7hFAS7

000

Vaibhav @vaibhavadlakha.bsky.social · 09/10/2024

Completely agree, super well organised and executed! 👏 t.co/wGkts8EGAb

000

Vaibhav @vaibhavadlakha.bsky.social · 09/10/2024

Excited to welcome @COLM_conf to the city of best bagels! 🥯 Looking forward to it! t.co/wUxyrDr3x6

000

Vaibhav @vaibhavadlakha.bsky.social · 05/10/2024

A little teaser for LLM2Vec @COLM_conf! Stop by Tuesday morning poster session to know how we officiated the marriage of BERTs and Llamas! 🦙 t.co/E3HB1mwVvv

000

Vaibhav @vaibhavadlakha.bsky.social · 28/08/2024

🚀🚀 LLMs are the new text encoders! t.co/4FZ2LXCPSd

000

Vaibhav @vaibhavadlakha.bsky.social · 15/08/2024

Amazing talk by @PontiEdoardo. 🙌🚀It is interesting how many different ways exist to make LLMs more efficient! t.co/2f2L8zLiH3

000

Vaibhav @vaibhavadlakha.bsky.social · 30/07/2024

First ever arena for embedding models! ⚔️ Excited to see how this will change evaluation in this space! 🚀 t.co/H4FoMJrQaA

000

Vaibhav @vaibhavadlakha.bsky.social · 22/07/2024

Looking for an emergency reviewer for EMNLP / ARR familiar with RAG and language models. Please reach out if you can review a paper in the next couple of days.

000

Vaibhav @vaibhavadlakha.bsky.social · 19/06/2024

Great to see LLM2Vec being used for multilingual machine translation! 🚀 I believe LLM2Vec will serve as backbone of many more applications in the future! t.co/G18aqJ2xuv

000

Vaibhav @vaibhavadlakha.bsky.social · 30/04/2024

However, this could mean we are past the point where MTEB serves as a useful signal 👀. Improving beyond the numbers we are seeing today (by training on synthetic data) carries the risk of optimizing for the benchmark rather than building general purpose embedding models. 5/N

000

Vaibhav @vaibhavadlakha.bsky.social · 30/04/2024

Interestingly, Meta-Llama-3-8B only slightly outperforms Mistral-7B, the previously best model when combined with LLM2Vec 🤔. We might have reached a point where better base models are not sufficient to make substantial improvements on MTEB. 3/N

000

Vaibhav @vaibhavadlakha.bsky.social · 30/04/2024

In the supervised setting, applying LLM2Vec to Meta-Llama-3-8B leads to a new state-of-the-art performance (65.01) on MTEB among models trained on publicly available data only. 2/N t.co/UJoOTJ4L5r

000

Vaibhav @vaibhavadlakha.bsky.social · 25/04/2024

Exciting discovery! Triggers DON’T transfer universally 😮. Check out the paper for detailed experiments and analysis. t.co/Op7gGWBEdb

000

Vaibhav @vaibhavadlakha.bsky.social · 22/04/2024

Applying LLM2Vec costs same as ~2 cappuccinos! t.co/O6iFXAJgoB

000

Vaibhav @vaibhavadlakha.bsky.social · 12/04/2024

Very nice and intuitive explanation of our work lLM2Vec by @IntuitMachine! Using causal LLMs for representation tasks without any architecture modifications is like driving a sports car in reverse 🏎️🤯 All resources available at our project page - t.co/hwAiv2yrPT t.co/RfBNydFW9y

000

Vaibhav @vaibhavadlakha.bsky.social · 12/04/2024

Great summary of our recent LLM2Vec paper! Thanks @ADarmouni! All resources available at our project page - t.co/hwAiv2yrPT t.co/bY4DoP5ms1

000

Vaibhav @vaibhavadlakha.bsky.social · 11/04/2024

This is going to be my new way of bookmarking papers now! t.co/Dbn5juFBD9

000

Vaibhav @vaibhavadlakha.bsky.social · 10/04/2024

Huggingface paper page by @_akhaliq - t.co/MRuPwtYCsZ

000

Vaibhav @vaibhavadlakha.bsky.social · 10/04/2024

We also analyze how enabling bidirectional attention without training affects the representations of decoder-only LLMs 🔍. We find that Mistral-7B is surprisingly good at using bidirectional attention out-of-the-box 🤯 and speculate it was likely trained as a prefix-LM 🤔. 7/N t.co/rlaub1ZQDC

000

Vaibhav @vaibhavadlakha.bsky.social · 10/04/2024

LLM2Vec-transformed models are not only capable of achieving state-of-the-art results, they are also much more sample-efficient compared to baselines 🏃. This makes us particularly excited about future work that applies LLM2Vec in low-resource scenarios. 6/N t.co/PheoTy8fOa

000

Vaibhav @vaibhavadlakha.bsky.social · 10/04/2024

We apply LLM2Vec to three different decoder-only LLMs (Sheared-LLaMA-1.3B, LLaMA-2-7B, and Mistral-7B) and evaluate in both unsupervised and supervised settings 🥁. We achieve new state-of-the-art performance in the unsupervised setting on the challenging MTEB benchmark🔥. 4/N t.co/iLk0d1Anlu

000

Vaibhav @vaibhavadlakha.bsky.social · 10/04/2024

Encoder-only models have been the dominant models for text embedding tasks. Only recently, LLMs gained popularity for these tasks. One factor limiting their adoption is the use of causal attention which prevents tokens from capturing information across the entire sequence 👎2/N

000

Vaibhav @vaibhavadlakha.bsky.social · 10/04/2024

We introduce LLM2Vec, a simple approach to transform any decoder-only LLM into a text encoder. We achieve SOTA performance on MTEB in the unsupervised and supervised category (among the models trained only on publicly available data). 🧵1/N Paper: t.co/1ARXK1SWwR t.co/L4jotnufn2

000

Vaibhav @vaibhavadlakha.bsky.social · 01/08/2023

5/n Sometimes it's best not to say anything 🤫 We test the model's ability to refrain from answering when provided with an incorrect passage. Models are sensitive to input prompts. When given explicit instruction, they refrain from answering even with the gold passage t.co/EeNMRbf4R3

000

Vaibhav @vaibhavadlakha.bsky.social · 01/08/2023

3/n For evaluating correctness ✅, we propose a simple fix. We calculate Recall - the proportion of reference answer tokens present in the model response. Recall correlates better 📈 than many established metrics and is competitive with LLM-based evaluation that uses GPT4 🤖 t.co/o57Z7fbZOA

000

Vaibhav @vaibhavadlakha.bsky.social · 01/08/2023

2/n We evaluate these models on three QA tasks. We pair them with a retriever 🔍 and inspect 1. Did the model correctly answer the user's query? ✅ 2. Was it faithful to provided knowledge? 📖 F1, typically used to evaluate correctness in QA, fails due to model verbosity 🗣️ t.co/w3JbtNyLQp

000

Vaibhav @vaibhavadlakha.bsky.social · 12/07/2023

Having worked directly with @harmdevries77, I can speak to his superb mentorship and management skills. Do consider this great opportunity at @ServiceNowRSRCH t.co/aka4n6me0m

000

Vaibhav @vaibhavadlakha.bsky.social · 15/06/2022

Huge congratulations to @viks_rum, @_apoorvnandan and the entire @DaoLens team! 🎉🥳 t.co/CLnHabULn9

000

Vaibhav @vaibhavadlakha.bsky.social · 21/10/2021

Happy to be part if this work led by @andreas_madsen. Got to learn a lot! Do read to know how faithful are current explanation methods in #NLProc t.co/3j1HiuJyVJ

000

Vaibhav @vaibhavadlakha.bsky.social · 05/10/2021

Very excited to share our work in conversational question-answering. By introducing a dataset that is open-domain and includes topic switching, we make the task extremely challenging and closer to real-world scenarios! More details coming soon! t.co/BDZBtNySVf

000

Vaibhav @vaibhavadlakha.bsky.social · 05/10/2021

Thanks, @sivareddyg for the incredible support and guidance over the past year! t.co/hD3skkncbp

000

Vaibhav @vaibhavadlakha.bsky.social · 05/10/2021

Check out our work at @akbc_conf t.co/ZGkGq2ehhD We propose a novel task of answering regex queries over incomplete knowledge bases and establish a strong baseline with our proposed model - RotatE-Box. #AKBC2021 Joint work with Parth, @mishumausam, and @srikanta

000

Vaibhav @vaibhavadlakha.bsky.social · 29/04/2021

Any leads will be appreciated. Please contact me or @pcPriyanshu t.co/zM5BFJKFkT

000

Vaibhav @vaibhavadlakha.bsky.social · 16/09/2020

Excited to share first conference acceptance! Out paper titled “Constrained Iterative Labeling for Open Information Extraction” has been accepted as a long paper at #emnlp2020. Joint work with @keshav_kolluru @samarth_agg1 @mishumausam and Soumen.

000

Vaibhav @vaibhavadlakha.bsky.social · 29/08/2020

What If I give @neuralink a try and don’t like it, am I stuck with a hole in my skull for my entire life?#askneuralink

000

Vaibhav @vaibhavadlakha.bsky.social · 29/08/2020

“I am much more frightened about robots always obeying orders than about robots rebelling against the humans.” Came across this today, a great discussion about possible future of AI and other tech from social, economic and geopolitical viewpoints t.co/aenR4GKrZV

010

Vaibhav @vaibhavadlakha.bsky.social · 19/07/2020

#gpt3 trying to be an annoying know-it-all!😆 “Q: How do you sporgle a morgle? A: You sporgle a morgle by using a sporgle.” Giving GPT-3 a Turing Test t.co/HqQc9mJDwv

000

Vaibhav @vaibhavadlakha.bsky.social · 11/07/2020

Algorithms shouldn’t be used for problems such as assigning grades, bail decisions, identifying criminal suspects etc. We aren’t there yet. the Secret Algorithm That's Keeping Students Out of College | WIRED t.co/73sSxEPrxA

000