iiinova — where your digital self keeps evolving

Victor Morand @victormorand.eurosky.social · 26/10/2026

Reconstruction quality: its not a complete free lunch 😇 MICE indeed requires more granular reps than LateOn, we used nbits=4 vs 2 for LateOn, making the PLAID store twice bigger (cf 1st Fig), and leaving more performance on the table: there's room to improve !

100 1

Victor Morand @victormorand.eurosky.social · 26/10/2026

MICE-P introduces a 128-d bottleneck, making doc-precomputation possible with FastPLAID residual compression (thx @raphaelsty.bsky.social 🙏) Candidates are retrieved with SPLADEv3, decompressed and scored with MICE—preserving fine-grained cross-attn without full query-doc re-encoding!

100 1

Victor Morand @victormorand.eurosky.social · 26/10/2026

🐭MICE to Meet You: From Re-ranking Only to End-to-End Retrieval ⚡️ Our #EMNLP2026 paper showed that MICE pushes the Cross-Encoder Pareto frontier. We take it a step further: turning theoretical pre-computation gains into an operational, end-to-end retrieval architecture 🧵

110 1

Victor Morand @victormorand.eurosky.social · 01/09/2026

5/6 ⚖️ New Pareto Frontiers Even without pre-computing doc states, MICE consistently pushes the Pareto efficiency–effectiveness frontier beyond traditional cross-encoders. Imagine with pre-computed docs... we are bridging the gap with late-interaction rankers ! (more soon 👀)

120 2

Victor Morand @victormorand.eurosky.social · 01/09/2026

4/6 ⚡ Massive Efficiency Gains - in short: Runs at ColBERT-like latency with cross-encoder expressiveness - Up to 2.5x theoretical FLOP reduction (already ~2x real speedup with our implementation) - Up to 4x latency speedup when pre-computing document representations offline

100 1

Victor Morand @victormorand.eurosky.social · 01/09/2026

2/6 🔍 Causal Masking We mask attention blocks inside standard CE, to see what interactions are superfluous. Notably: - Docs don’t need to attend to the query - Q/D can be contextualized independantly until ~ half of the model depth This confirms previous interp works on CE

100 1

Victor Morand @victormorand.eurosky.social · 01/09/2026

🐭 Excited to share MICE (Minimal Interaction Cross-Encoders) - accepted to #EMNLP2026 main ! Cross-encoders are sota for ranking, but often too heavy We ask a simple question: How many operations can be pruned while maintaining effectiveness? 👉 A lot more than you think 🧵

121 2

Victor Morand @victormorand.eurosky.social · 22/04/2026

Theoretical Insight the study provides concrete evidence that mention detection emerge naturally from the language modeling. We found that diverse models (14M to 15B params) converge on a shared notion of entity boundaries (DICE >75%), regardless of scale or architecture

100 1

Victor Morand @victormorand.eurosky.social · 22/04/2026

➡️ We introduce ToMMeR, a probe no heavier than a single attention head (<300K parameters). You can plug in ANY frozen LLM. It extracts these latent signals in a single parallel pass—no schema nor generative prompting required. 🧵

100 1

Victor Morand @victormorand.eurosky.social · 22/04/2026

🔍 Can we probe latent mention detection signals in Large Language Models ? Yes ! Excited to share that our paper ToMMeR (Token Matching for Mention Recognition) is heading to #ACL2026! We confirm that entity detection is a robust, latent capability in early LLM layers, and probe it ! 🧵

130 3

Victor Morand @victormorand.eurosky.social · 22/10/2025

3️⃣ The Entity Lens Our method enables reconstruction of entity mentions from any representation within LLMs, allowing to ask: “What entity is the model thinking about right now?” 💡 When reading ‘the City of Lights iconic monument’, the model internally “thinks” of Paris and the Eiffel Tower !

100 1

Victor Morand @victormorand.eurosky.social · 22/10/2025

New paper at @blackboxnlp.bsky.social @ @emnlpmeeting.bsky.social ! ⚛️ Entities are the fundamental building blocks of knowledge. Although some clues emerge from mechanistic interpretability, how auto-regressive LLMs actually encode and retrieve them remains a mystery. 🧵 📄 arxiv.org/abs/2510.09421

111 1