Fully open reproduction of DeepSeek-R1. Contribute to huggingface/open-r1 development by creating an account on GitHub.

25 January 2025 (

Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities. In response, we introduce Humanity's Last Exam, a multi-modal benchmark at the frontier of human knowledge, designed to be the final closed-ended academic benchmark of its kind with broad subject coverage. The dataset consists of 3,000 challenging questions across over a hundred subjects. We publicly release these questions, while maintaining a private test set of held out questions to assess model overfitting.

Roughly 6,000 readers answered our poll, with many declaring that Bluesky was nicer, kinder and less antagonistic to science than X.

There is growing interest in the reproducibility of research and ways to enhance transparency. This study uses an international survey of more than 1900 biomedical researchers to reveal the perceived ...

CDC and other agencies were told to halt public communications, and the NIH grant process appears to be on hold.

Abstract Recent clinical trials on slowing dementia progression have led to renewed focus on finding safer, more effective treatments. One approach to identify plausible candidates is to assess whet...

Following U.S. FDA Priority Review, approval is based on data demonstrating SPRAVATO® alone met its primary endpoint at 4 weeks and led to rapid and superior improvement in depressive symptoms compare...

At GSK, we are driven by our ambition to accelerate the discovery and development of new treatments for patients. Today, I’m pleased to announce another…

Large language models can be manipulated to generate misinformation by poisoning of a very small percentage of the data on which they are trained, but a harm mitigation strategy using biomedical knowledge graphs can offer a method for addressing this vulnerability.

YouTube video by Jay Alammar

"How to evaluate de novo designs proposed by a generative model?" Despite the transformative potential of generative deep learning in drug discovery, this seemingly simple question has no clear answer...

Colossal Biosciences, a de-extinction startup looking at ways to bring back the dodo bird and the woolly mammoth, raised a $200 million Series C at a $10.2 billion valuation.