OPERA: Optimizing data pruning for efficient retrieval model adaptation
Amazon introduces dynamic pruning that favors high-quality pairs when finetuning dense retrievers, improving both ranking and recall in under half the time.
📝 www.amazon.science/publications...
👨🏽💻 github.com/autogluon/au...
amazon.science
OPERA: Optimizing data pruning for efficient retrieval model adaptation
Domain-specific finetuning is essential for dense retrievers, yet not all data pairs contribute equally to the learning process. We introduce OPERA1 , a data pruning framework that exploits this heter...