iiinova — where your digital self keeps evolving

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 24/03/2025

🎉Read the preprint: agentrxiv.github.io Try out AgentRxiv: github.com/SamuelSchmid... Let’s explore how agents can accelerate research—together. 🧵8/8

120

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 24/03/2025

🛡️Research agents and their labs, while promising, are still not at human-level quality. By channeling their work into AgentRxiv—a dedicated hub for autonomous research—we’re also safeguarding the quality of human research on arXiv. 🧵7/8

120

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 24/03/2025

✨ In parallel experiments with 3 independent labs sharing pre-prints through AgentRxiv, the best method achieved 79.8% accuracy—a 13.7% relative improvement—while reaching key milestones faster than in sequential experiments. 🧵6/8

110

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 24/03/2025

🏥 We also wondered how well the methods our agents discovered perform on out-of-domain benchmarks (MMLU-Pro, GPQA, & MedQA) and with five other language models. We find the top performing algorithm SDA improves across these benchmarks on average by 3.3%. 🧵5/8

110

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 24/03/2025

🥇We perform experiments where agents are asked to develop new reasoning techniques on MATH-500. We find that when agents are given access to previous research, accuracy improved from 70.2% to 78.2% – an 11.4% relative improvement over the gpt-4o mini baseline and 9.7% over gpt-4o mini with CoT. 🧵4/8

120

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 24/03/2025

To address this, we introduce AgentRxiv—a framework that lets LLM agent laboratories upload and retrieve reports from a shared preprint server in order to collaborate, share insights, and iteratively build on each other’s research. 🧵3/8

110

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 24/03/2025

There has been a lot of recent excitement around autonomous LLM agents performing research, with several fully autonomous works being accepted into ICLR 2025 📚 ‼️The problem is that these systems work in isolation without the ability to build on their research. 🧵2/8

110

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 24/03/2025

🚀🌐Introducing AgentRxiv: a framework where autonomous research agents can upload, retrieve, and build on each other’s research. AgentRxiv takes your research direction and progressively outputs research papers and code repositories, building on its previous work with each new paper! 🧵

130

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 27/02/2025

👩‍💻 All of the code is completely open-source! Below are links to the website, paper, and github! Check it out. website: agentlaboratory.github.io paper: arxiv.org/pdf/2501.04227 github: github.com/SamuelSchmidga…

agentlaboratory.github.io

Agent Laboratory: Using LLMs as Research Assistants

by Samuel Schmidgall at JHU

000

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 27/02/2025

Agent Laboratory consists of three primary phases that guide the research process: (1) Literature Review, (2) Experimentation, and (3) Report Writing. During each phase, LLM agents collaborative, integrating tools like arXiv, Hugging Face, Python, and LaTeX.

120

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 27/02/2025

🚀🔬 Introducing Agent Laboratory: an assistant for automating machine learning research Agent Laboratory takes your research ideas and outputs a research paper and code repository, allowing you to allocate more effort toward ideation rather than low-level coding and writing [Re-sharing from X]

110

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 27/02/2025

🔥 Really great overview of Agent Laboratory by Two Minute Papers video: youtu.be/2ky50XT0Nb0?... agent lab webpage: agentlaboratory.github.io

000

Reposted by @samuelschmidgall.bsky.social

Adam Kucharski @adamjkucharski.bsky.social · 23/02/2025

These aren’t totally hypothetical questions. Currently, the US is in the process of trashing its wildly successful science funding system. NIH, which funds tens of billions of dollars of research each year, has been estimated to generate around $2.50 of economic activity for every $1 funded:

wewillcure.com

Impacts of NIH Funding on the US Economy, Jobs and Better Health

The U.S. National Institutes of Health is the largest single public funder of biomedical and behavioral research in the world. NIH activities and funding are major drivers of the United States’ compet...

12480224

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 27/12/2024

I'm excited to start as a Student Researcher at Google DeepMind working on medical AI!

020

samuelschmidgall.bsky.social @samuelschmidgall.bsky.social · 04/12/2024

An LLM that makes decisions that have consequences in an external environment with temporal dependencies (?)

000