Under our old scheduler, every scheduled workload eventually used HIGH priority. Some researchers kept idle jobs running to reserve GPUs for experiments.
The incentives rewarded holding on to capacity even when there were no active jobs.
We’re sharing the engineering work behind Ai2’s GPU scheduler—our new approach to allocating compute across research teams.
On our largest H100 cluster, median queue wait fell from 5 minutes to 24 seconds. 🧵
buff.ly/4UsJEYe
