Current machine translation benchmarks are saturated. Last Translation Benchmark crowdsources a dataset of inputs that demonstrably break state-of-the-art translation models.

For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, standard benchmarks for ...

Current machine translation benchmarks are saturated. Last Translation Benchmark crowdsources a dataset of inputs that demonstrably break state-of-the-art translation models.

A peer review platform for the Association for Computational Linguistics

Current machine translation benchmarks are saturated. Last Translation Benchmark crowdsources a dataset of inputs that demonstrably break state-of-the-art translation models.