> the LLM judges ... cost 29 to 325 times as much as Jev and took 30 to 220 times as long
> Jev's confidence ranks its own errors on most panels, which should make a cheap classifier the ideal first stage of a cascade that defers its uncertain verdicts to an LLM judge
arxiv.org/abs/2609.29769
arxiv.org
JEV vs. LLMs as Rubric Judges: Cheaper, Faster, and Wrong in the Same Places
We ask whether Jev, a typed classifier that returns probabilities over permitted answers without generating text, can replace an LLM rubric judge. We compare it with three flash-tier LLM judges on nin...