iiinova — where your digital self keeps evolving

David Lindner @davidlindner.bsky.social · 08/07/2025

Excited to share some technical details about our approach to scheming and deceptive alignment as outlined in Google's Frontier Safety Framework! (1) current models are not yet capable of realistic scheming (2) CoT monitoring is a promising mitigation for future scheming

010

David Lindner @davidlindner.bsky.social · 04/07/2025

View our full paper here: arxiv.org/abs/2507.02737 View steganography transcripts here: steganography-evals-d9297ad0740c.herokuapp.com/ Github: github.com/arturzolkow... Work done with: Artur Zolkowski, Kei Nishimura-Gasparian, Robert McCarthy and @zimmerrol.bsky.social‬ during matsprogram.org

matsprogram.org

MATS Program

020

David Lindner @davidlindner.bsky.social · 04/07/2025

Overall, current models have some steganographic capabilities but likely aren't strong enough to bypass well-designed monitors. Nevertheless, we believe it’s important to continue measuring these capabilities to determine when additional mitigations are needed.

100

David Lindner @davidlindner.bsky.social · 04/07/2025

We also create case studies implementing situations where models with strong steganographic capabilities could take unwanted actions and bypass a monitor. Current models rarely succeed in these environments unless they are given strong tips about what to do.

100

David Lindner @davidlindner.bsky.social · 04/07/2025

Models can also reason through simple problems using a number of common encoding schemes. But performance quickly degrades as the problems get more difficult.

100

David Lindner @davidlindner.bsky.social · 04/07/2025

While models understand they need to conceal their reasoning, they have difficulty actually hiding that reasoning in a plausible way. As an example, we show a model’s reasoning attempt when asked a question on how to install solar panels. [bold text and highlighting added by us]

100

David Lindner @davidlindner.bsky.social · 04/07/2025

In our encoded reasoning evals, we find that models can sometimes leverage encoded reasoning to perform better than no reasoning baselines on simple reasoning tasks. But encodings that help tend not to be very subtle and are regularly detected by a monitor.

100

David Lindner @davidlindner.bsky.social · 04/07/2025

Models are able to perform significantly better when the sender and receiver are able to coordinate on the steganographic encoding method. But this mainly applies when the model has access to an unmonitored CoT. With no CoT, performance remains extremely poor.

110

David Lindner @davidlindner.bsky.social · 04/07/2025

When they succeed, models sometimes leverage sophisticated strategies to pass messages, such as encoding information in the lengths of the sentences they generate! We found this example difficult to decode ourselves. [formatting and word counts added by us]

110

David Lindner @davidlindner.bsky.social · 04/07/2025

In our message passing evals, we find that models nearly always fail without access to CoT. But models succeed much more often when they are allowed to use a CoT to freely reason ahead of time about how to encode and decode the hidden message.

100

David Lindner @davidlindner.bsky.social · 04/07/2025

Models encoding hidden information might make safety mitigations such as CoT monitoring much more difficult. We evaluate frontier models’ ability to evade monitors by sending encoded messages or performing encoded reasoning about a task. Read more: arxiv.org/abs/2507.02737

100

David Lindner @davidlindner.bsky.social · 04/07/2025

Can frontier models hide secret information and reasoning in their outputs? We find early signs of steganographic capabilities in current frontier models, including Claude, GPT, and Gemini. 🧵

161

David Lindner @davidlindner.bsky.social · 04/04/2025

Super excited this giant paper outlining our technical approach to AGI safety and security is finally out! No time to read 145 pages? Check out the 10 page extended abstract at the beginning of the paper

163

Reposted by David Lindner

vkrakovna.bsky.social @vkrakovna.bsky.social · 14/02/2025

We are excited to release a short course on AGI safety! The course offers a concise and accessible introduction to AI alignment problems and our technical / governance approaches, consisting of short recorded talks and exercises (75 minutes total). deepmindsafetyresearch.medium.com/1072adb7912c

deepmindsafetyresearch.medium.com

Introducing our short course on AGI safety

We are excited to release a short course on AGI safety for students, researchers and professionals interested in this topic. The course…

0185

David Lindner @davidlindner.bsky.social · 18/02/2025

Check out this great post by Rohin Shah about our team and what we’re looking for in candidates: www.alignmentforum.org/posts/wqz5CR... In particular: we're looking for strong ML researchers and engineers and you do not need to be an AGI safety expert

010

David Lindner @davidlindner.bsky.social · 10/02/2025

Research Engineer: boards.greenhouse.io/deepmind/job... Research Scientist: boards.greenhouse.io/deepmind/job... Happy to answer questions via DM!

boards.greenhouse.io

Research Engineer, Gemini Safety / AGI Safety & Alignment

New York City, New York, US

020

David Lindner @davidlindner.bsky.social · 10/02/2025

Want to join one of the best AI safety teams in the world? We're hiring at Google DeepMind! We have open positions for research engineers and research scientists in the AGI Safety & Alignment and Gemini Safety teams. Locations: London, Zurich, New York, Mountain View and SF

1121

Reposted by David Lindner

Sebastian Farquhar @sebfar.bsky.social · 23/01/2025

By default, LLM agents with long action sequences use early steps to undermine your evaluation of later steps; a big alignment risk. Our new paper mitigates this, keeps the ability for long-term planning, and doesnt assume you can detect the undermining strategy. 👇

0131

David Lindner @davidlindner.bsky.social · 23/01/2025

We’ll do a lot more work studying ways to build safer AI systems and the safety-performance tradeoffs involved. If that’s something you’d like to work on, do get in touch via DM!

030

David Lindner @davidlindner.bsky.social · 23/01/2025

Thanks to @sebfar.bsky.social for exceptional leadership of the project and the rest of the team: Vikrant Varma, David Elson, Caleb Biddulph, @ian-goodfellow.bsky.social, and Rohin Shah for making this work possible!

150

David Lindner @davidlindner.bsky.social · 23/01/2025

Want to dive deeper? Check out our paper and our blog posts explaining the work in more detail 📄 Paper: arxiv.org/abs/2501.13011 💡 Introductory explainer: deepmindsafetyresearch.medium.com/mona-a-meth... ⚙️ Technical safety post: www.alignmentforum.org/posts/zWySW...

171

David Lindner @davidlindner.bsky.social · 23/01/2025

MONA is promising – but it’s not a silver bullet. It doesn’t help with reward hacking within a single step, and it can reduce performance, especially when the reward is well specified. Nonetheless, we think MONA is an important tool for building safer LLM agents.

130

David Lindner @davidlindner.bsky.social · 23/01/2025

Then we train agents to make loan decisions but penalize them for using the applicant's nationality. Because nationality improves the decision reward, ordinary RL agents learn to use it secretly. MONA agents avoid the hack – they make good decisions without using nationality!

140

David Lindner @davidlindner.bsky.social · 23/01/2025

First, we train agents to solve coding problems but also write tests to evaluate their solutions. Ordinary RL agents write tests that are trivial to pass with incorrect code – a reward hack that leads to low performance on held-out tests. MONA agents solve the problems correctly!

150

David Lindner @davidlindner.bsky.social · 23/01/2025

Does this work? Yes! We test MONA in two LLM agent environments and find that MONA agents learn to perform well when agents trained with ordinary reinforcement learning reward hack.

130

David Lindner @davidlindner.bsky.social · 23/01/2025

What happens if the agent tries a multi step reward hack that the overseer can’t detect? On the first step (before the hack is complete), the overseer doesn’t know why the step is valuable – so she doesn’t provide a high reward. So the first step isn’t incentivized by MONA.

140

David Lindner @davidlindner.bsky.social · 23/01/2025

Key idea: Use RL training only for short horizons (myopic optimization), but have an overseer evaluate how good actions are for the long term (non-myopic approval). Best of both worlds: we get human-understandable plans (safe!) and long-term planning (performant!)

170

David Lindner @davidlindner.bsky.social · 23/01/2025

New Google DeepMind safety paper! LLM agents are coming – how do we stop them finding complex plans to hack the reward? Our method, MONA, prevents many such hacks, *even if* humans are unable to detect them! Inspired by myopic optimization but better performance – details in🧵

1408

David Lindner @davidlindner.bsky.social · 14/12/2024

Stop by SoLaR workshop at NeurIPS today to see Kai present the paper!

030

Reposted by David Lindner

Seb Krier @sebkrier.com · 10/12/2024

MISR eval reveals frontier language agents can self-reason to modify their own settings & use tools, but they fail at harder tests involving social reasoning and knowledge seeking. Opaque reasoning remains difficult. arxiv.org/abs/2412.03904

0101

Reposted by David Lindner

Tom Andersson 🌍 @tom-andersson.bsky.social · 10/12/2024

So excited to share our Google DeepMind team's new Nature paper on GenCast, an ML-based probabilistic weather forecasting model: www.nature.com/articles/s41... It represents a substantial step forward in how we predict weather and assess the risk of extreme events. 🌪️🧵

nature.com

Probabilistic weather forecasting with machine learning - Nature

GenCast, a probabilistic weather model using artificial intelligence for weather forecasting, has greater skill and speed than the top operational medium-range weather forecast in the world and provid...

210816

David Lindner @davidlindner.bsky.social · 06/12/2024

📄Paper: arxiv.org/abs/2412.03904 💻Code: github.com/kaifronsdal/... Awesome job by Kai Fronsdal who did this work during matsprogram.org!

000

David Lindner @davidlindner.bsky.social · 06/12/2024

We find some self-reasoning ability in the most capable models, but limited performance on the hard versions of our tasks. These harder tasks can be a useful tool to track self-reasoning ability as an early warning sign for risks from misaligned AI agents!

100

David Lindner @davidlindner.bsky.social · 06/12/2024

New paper on evaluating instrumental self-reasoning ability in frontier models 🤖🪞 We propose a suite of agentic tasks that are more diverse than prior work and give us a more representative picture of how good models are at eg. self-modification and embedded reasoning

120

Reposted by David Lindner

Marc Lanctot @handle.invalid · 25/11/2024

Just a heads up to everyone: @deep-mind.bsky.social is unfortunately a fake account and has been reported. Please do not follow it nor repost anything from it.

98434