Reposted by Adithya Bhaskara
One of the largest right-wing-extremist conspiracy accounts on X is the prediction market PolyMarket, and it really makes you wonder how this may affect the accuracy & calibration of their prices.
Adithya Bhaskara
@adithyacolorado.bsky.social
129 followers 265 following 97 posts
Ph. D. Student @ MIT EECS Interested in theoretical computer science, computational social choice theory, mathematical democracy, politics, and the intersection between theory and practice. officialadithya.github.io abhaskara@mit.edu
![AI in Reviewing.
I have, of course, my own opinions about many other points. One is about AI in reviewing. Even ignoring the ethical or copyright issues (they exist), or the cost (there is one!) [If each thorough review costs even $10 in tokens for a conference receiving 1,000 submissions, who's fronting the $10k? That's a conservative cost, and even that can make a nice hole in a TCS conference budget.], said opinion boils down to the following:
Assessing papers, results, and their interest to our research community *is our job*, pretty much by definition of *peer* review, and offloading it to LLMs is at best a slippery slope.
By "slippery slope," I mean that we are human, and we're all busy. Once AI reviews start appearing, "only about technical correctness" and "absolutely not about gauging interest and novelty," it's a matter of $k$ conference iterations, for a very small non-negative integer $k$, before most or all reviewers entirely delegate `gauging interest and novelty" to the LLM output, even without fully realizing it. After all, *we're all busy*.
(Now, if the argument is that LLMs are better at checking correctness, then let the authors run their work through an LLM and send the output as a cover letter or appendix. [This has potential issues, which would need to be addressed first: e.g., that of uniform access to the same models for everyone.] And if you don't trust the authors and are afraid they could be malicious in doing so... well, I think we have bigger problems as a community, then.) The key point though is that nobody *forces* anyone to be on a Program Committee, or to be a (sub)reviewer. If you'd rather not do the job and don't want to actually read the papers, well, *you can just say no!* It's OK! Others do!](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:ac6qioenkpl3jmmnib3tpbhb/bafkreihjx6ixpowbrf5prtay74xluhdd2cckbtbwpb57lefcpub7fxbyxi)
