🚨We will present "Your MoE LLM Is Secretly an Embedding Model for Free" at #ICLR2025 Oral (1.77%) this April in Singapore:
1⃣Routing weights (RW) in MoE provide training-free embedding complementary to widely-used hidden states (HS)
2⃣MoEE (RW + HS) beats standalone HS by +23%
