iiinova — where your digital self keeps evolving

Reposted by Aaron Scher

MIRI @intelligence.org · 01/05/2025

New AI governance research agenda from MIRI’s TechGov Team. We lay out our view of the strategic landscape and actionable research questions that, if answered, would provide important insight on how to reduce catastrophic and extinction risks from AI. 🧵1/10 techgov.intelligence.org/research/ai-...

2125

Reposted by Aaron Scher

Malo Bourgon @malo.online · 18/03/2025

MIRI's (@intelligence.org) Technical Governance Team submitted a comment on the AI Action Plan. Great work by David Abecassis, @pbarnett.bsky.social, and @aaronscher.bsky.social Check it out here: techgov.intelligence.org/research/res...

052

Aaron Scher @aaronscher.bsky.social · 05/12/2024

One mechanism that seems promising is Flexible Hardware-Enabled Guarantees (FlexHEGs) and on-chip approaches. These could potentially be used to securely carry out a wide range of governance operations on AI chips, without leaking sensitive information. (1/3)

120

Aaron Scher @aaronscher.bsky.social · 05/12/2024

Reflection: The more I got into the weeds on this project, the harder verification seemed. Some difficulties are distributed training, algorithmic progress, and the need to be robust against state-level adversaries. It’s hard, but we have to do it! (1/1)

000

Aaron Scher @aaronscher.bsky.social · 04/12/2024

One mechanism that seems promising: Signatures of High-Level Chip Measures. Classify workloads (e.g., is it training or inference) based on high-level chip measures like power-draw, but using ‘signatures’ of these measures based on temporary code access. (1/6)

100

Aaron Scher @aaronscher.bsky.social · 04/12/2024

One mechanism that seems especially promising: Networking Equipment Interconnect Limits, like “Fixed Sets” discussed by www.rand.org/pubs/working... but can be implemented with custom networking equipment quickly. (1/8)

rand.org

Hardware-Enabled Governance Mechanisms

The authors introduce the concept of hardware-enabled governance mechanisms, which could help achieve U.S. artificial intelligence governance goals, and discuss two mechanisms that could limit uses of...

100

Aaron Scher @aaronscher.bsky.social · 04/12/2024

One thread throughout this report is that low-tech, high-access solutions can often substitute for high-tech, low-access solutions, let’s walk through some examples. (1/12)

100

Aaron Scher @aaronscher.bsky.social · 04/12/2024

Distributed training (i.e., geographically distributed, decentralized) could pose major problems for many AI Governance plans. In the default case, large AI training happens in a small number of big data centers, so monitoring training can focus on those data centers. (1/10)

100

Aaron Scher @aaronscher.bsky.social · 04/12/2024

Inspectors who have full access to your systems seem like they could pose a major privacy and security risk, so it may be necessary to have very tight info sec around them, e.g., limited communication to home countries. (1/3)

100

Aaron Scher @aaronscher.bsky.social · 04/12/2024

Reflection: People I talked to had wildly different intuitions about the likelihood of direct US/China conflict in response to the threat of US AGI and ASI (superintelligence) development. (1/4)

100

Aaron Scher @aaronscher.bsky.social · 04/12/2024

How can the US and China (or the international community, broadly) ensure compliance in AI agreements to manage large-scale risks? In a recent report, we discuss options available for verification of international AI treaties. Applicable to domestic rules too! 🧵 (1/12)

techgov.intelligence.org

Mechanisms to Verify International Agreements About AI Development — MIRI Technical Governance Team

In this research report we provide an in-depth overview of the mechanisms that could be used to verify adherence to international agreements about AI development.

182

Aaron Scher @aaronscher.bsky.social · 24/11/2024

Alright here's a thread with some takes on recent AI safety-ish papers, written as I skim them because I want more content on this platform

451