Sign in

Daniel Paleka

@dpaleka.bsky.social
309 followers 171 following 66 posts

ai safety researcher | phd ETH Zurich | danielpaleka.com

PostsRepliesMedia
Daniel Paleka @dpaleka.bsky.social · 20/02/2026
Direct deanonymization. Anthropic Interviewer is a dataset of anonymized interviews with scientists about their use of AI. Following prior work, a simple agent finds ~7% of the interviewed scientists, out of the box, just by searching the web and reasoning over the transcript.
190 0
Daniel Paleka @dpaleka.bsky.social · 20/02/2026
Scaling: as candidate pools grow to tens of thousands, LLM-based attacks degrade gracefully at high precision; this implies that with sufficient compute, these methods would already scale to entire platforms. With future models, expect the cost to only go down.
160 0
Daniel Paleka @dpaleka.bsky.social · 20/02/2026
Proxy 2: Matching split accounts. On Reddit, we split user histories into "before" and "after", and test LLMs linking them back together. LLM embeddings + reasoning significantly outperform Netflix-Prize-style baselines that match based on subreddits and metadata. @random_walker
170 0
Daniel Paleka @dpaleka.bsky.social · 20/02/2026
Proxy 1: Cross-platform. We take non-anonymous Hacker News accounts that link to their LinkedIn. We then anonymize the HN accounts, removing all directly identifying information. Then, we let LLMs match the anonymized account to the true person; this works with high precision.
1110 0
Daniel Paleka @dpaleka.bsky.social · 20/02/2026
Can LLMs figure out who you are from your anonymous posts? From a handful of comments, LLMs can infer where you live, what you do, and your interests; then search for you on the web. New 📄 w/ @SimonLermenAI, @joshua_swans, @AerniMichael, Nicholas Carlini, @florian_tramer 🧵
812442 0
Daniel Paleka @dpaleka.bsky.social · 05/06/2025
Benchmarks can reward strategic gambling over calibrated forecasting when optimizing for ranking performance. "Bet everything" on one scenario beats careful probability estimation for maximizing the chance of ranking #1 on the leaderboard. (6/7)
100 0
Daniel Paleka @dpaleka.bsky.social · 05/06/2025
Model knowledge cutoffs are guidelines about reliability, not guarantees of no information thereafter. GPT-4o, when nudged, can reveal knowledge beyond its stated Oct 2023 cutoff. (5/7)
100 0
Daniel Paleka @dpaleka.bsky.social · 05/06/2025
Date-restricted search leaks future knowledge. Searching pre-2019 articles about “Wuhan” returns results abnormally biased towards the Wuhan Institute of Virology — an association that only emerged later. (4/7)
100 0
Daniel Paleka @dpaleka.bsky.social · 05/06/2025
How well can LLMs predict future events? Recent studies suggest LLMs approach human performance. But evaluating forecasters presents unique challenges compared to standard LLM evaluations. We identify key issues with forecasting evaluations 🧵 (1/7)
100 0
Daniel Paleka @dpaleka.bsky.social · 26/05/2025
why is it that whenever i see survivorship bias on my timeline it already has the red-dotted plane in the replies?
010 0
Daniel Paleka @dpaleka.bsky.social · 30/04/2025
Quick sycophancy eval: comparing the two recent OpenAI ChatGPT system prompts, it is clear last week's prompt moves other models towards sycophancy too, while the current prompt makes them more disagreeable.
100 0
Daniel Paleka @dpaleka.bsky.social · 17/01/2025
what are you doing Claude i thought we were friends
020 0
Daniel Paleka @dpaleka.bsky.social · 11/01/2025
Test-time compute based on arbitrage can make forecasts more consistent; this improves specific logical rules such as Negation, but doesn't generalize to the consistency rules we do not optimize over. (9/11)
100 0
Daniel Paleka @dpaleka.bsky.social · 11/01/2025
Some consistency checks are better signals than others; for instance, the violation of P(A)P(B|A) = P(A&B) explains a high fraction of the variation in forecasting performance over a range of forecasters. (7/11)
100 0
Daniel Paleka @dpaleka.bsky.social · 11/01/2025
Starting from a base question, we generate multiple logically related questions and ask them independently. (5/11)
100 0
Daniel Paleka @dpaleka.bsky.social · 11/01/2025
We create consistency checks from base forecasting questions, which we take from various sources (prediction market questions, synthetically generated from news, purely LLM-generated); ask the forecasters for probabilities, and check how consistent the predictions are (4/11)
100 0
Daniel Paleka @dpaleka.bsky.social · 11/01/2025
We test 10 different logical rules that a consistent forecaster should satisfy. (3/11)
100 0
Daniel Paleka @dpaleka.bsky.social · 11/01/2025
Recent LLM forecasters are getting better at predicting the future. But there's a challenge: How can we evaluate and compare AI forecasters without waiting years to see which predictions were right? (1/11)
152 0
Daniel Paleka @dpaleka.bsky.social · 08/01/2025
LLMs rapidly improving at software engineering and math, given that the rate of improvement in ideation is slower, means you should be intentional about what value is gained from doing a highly technical project now as opposed to later
150 0
Daniel Paleka @dpaleka.bsky.social · 31/12/2024
my New Year's resolution: don't work on a bigger project if there is not a clear reason for doing it *now*. disregarding the AGI timelines, the R&D acceleration is a clear reason against technical work where the discount rates on the final product are low
040 0
Daniel Paleka @dpaleka.bsky.social · 28/11/2024
TIL that the atmosphere blocks basically all electromagnetic radiation, except three small windows: one for visible light, one for cooling the Earth, and one for radio waves. Earth is the USA of planets.
010 0