What is the optimal temperature for LLMs-as-judges?
doi.org/10.48550/arX...
Why read? The authors find that higher temperatures may help in complex evaluation tasks.
What's next? Exploration of optimal temperature for different tasks.
#ShortReview #AI #LLM #Temperature #LLMsAsJudges
doi.org
The Necessity of Setting Temperature in LLM-as-a-Judge
Using large language models (LLMs) as judges for evaluating model outputs has emerged as an important paradigm for automated evaluation. However, the choice of decoding temperature in LLM-as-a-judge s...