Transformer-based language models spread FLOPs uniformly across input sequences. In this work we demonstrate that transformers can instead learn to dynamically allocate FLOPs (or compute) to...

Precisely estimating the volume of basins in neural net parameter space corresponding to interpretable behaviors - EleutherAI/basin-volume

We present an algorithm for estimating the probability mass, under a Gaussian or uniform prior, of a region in neural network parameter space corresponding to a particular behavior, such as achieving ...

The applicability of computational models to the biological world is an active topic of debate. We argue that a useful path forward results from abandoning hard boundaries between categories and adopt...

(Photo by Tom Pumford on Unsplash)

Contribute to EleutherAI/training-jacobian development by creating an account on GitHub.

I distinguish between pure self-locating credences and superficially self-locating credences, and argue that there is never any rationally compelling way to assign pure self-locating credences. I firs...

YouTube video by theJourneyofPurpose TJOP

Crosspost from Twitter/X to Bluesky and Mastodon directly in the web browser - 59de44955ebd/twitter-to-bsky