iiinova — where your digital self keeps evolving

Orion Weller

@orionweller.bsky.social

2.3K followers 331 following 22 posts

PhD Student at Johns Hopkins University. Previously: Allen Institute for AI, Apple, Samaya AI. Research for #NLProc #IR

Orion Weller @orionweller.bsky.social · 26/02/2025

Ever wonder how test-time compute would do in retrieval? 🤔 introducing ✨rank1✨ rank1 is distilled from R1 & designed for reranking. rank1 is state-of-the-art at complex reranking tasks in reasoning, instruction-following, and general semantics (often 2x RankLlama 🤯) 🧵

170

Reposted by Orion Weller

Kenneth Enevoldsen @kennethenevoldsen.eurosky.social · 20/02/2025

We use this collection of tasks to propose multiple benchmarks for multilingual, code, European and Indic languages, and many more. We find that smaller multilingual models (~500M) outperform notably larger 7B models, likely due to a limited multilingual pre-training.

131

Orion Weller @orionweller.bsky.social · 19/12/2024

Check out our new encoder model, ModernBERT! 🤖 Super grateful to have been part of such an awesome team effort and very excited about the gains for retrieval/RAG! 🚀

1172

Orion Weller @orionweller.bsky.social · 16/12/2024

MASC is such a fun time! If your university is in the mid-Atlantic, please consider hosting!

070

Reposted by Orion Weller

Tom Aarsen @tomaarsen.com · 27/11/2024

I'm looking for an intern to introduce Sparse Embedding models to Sentence Transformers! If you're passionate about open source, interested in helping practitioners use your tools, and enjoy embedders/retrievers/rerankers, then I'd love to hear from you! Links with details and to apply in 🧵

3305

Reposted by Orion Weller

Marc Marone @marcmarone.com · 23/11/2024

I noticed a lot of starter packs skewed towards faculty/industry, so I made one of just NLP & ML students: go.bsky.app/vju2ux Students do different research, go on the job market, and recruit other students. Ping me and I'll add you!

10117654

Orion Weller @orionweller.bsky.social · 23/11/2024

Creating a 🦋 starter pack for people working in IR/RAG: go.bsky.app/88ULgwY I can’t seem to find everyone though, help definitely appreciated to fill this out (DM or comment)!

328623

Orion Weller @orionweller.bsky.social · 15/09/2023

Using LLMs for query or document expansion in retrieval (e.g. HyDE and Doc2Query) have scores going 📈 But do these approaches work for all IR models and for different types of distribution shifts? Turns out its actually more 📉 🚨 📝 (arxiv soon): orionweller.github.io/assets/pdf/L...

A plot: the x axis is baseline score of rankers, in ndcg@10. y axis is delta of model score after an expansion is applied.

There are three sets of results, one dataset for each shift type: TrecDL (no shift), FiQA (domain shift), ArguAna (query shift). For each set of result, the chart shows a scatter plot with a trend line. We observe the same trend for all: as the baseline score increases, the delta when using expansion decreases.

On TREC DL, worst models have a base score of ~40, and improve by 10 points w/expansion. the best models have a score of >70, and their performance decreases by -5 points w/expansion.

On FiQA, worse models have a base score of ~15, and improve by 5 points w/expansion. the best models have a score of ~45, and their performance decreases by -3 point w/expansion.

On ArguAna, worst models have a base score of ~25, and improve by >20 points w/expansion. the best models have a score of >55, and their performance decreases by -1 point w/expansion.

3436