iiinova — where your digital self keeps evolving

johnowhitaker.bsky.social @johnowhitaker.bsky.social · 02/12/2024

AI & Junk Food When you optimize food for short-term preference, you get sugary, salty brain hacks. Yum, unhealthy in excess. When you optimize models for short-term preference, you get flattering, verbose, sloppy brain hacks. Cool, unhealthy in excess. Do you agree? What's 'healthy food' for AI?

130

Reposted by @johnowhitaker.bsky.social

Jascha Sohl-Dickstein @jascha.sohldickstein.com · 12/02/2024

Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish to those for which training diverges. Even better, a video: vimeo.com/903855670

$Examples of fractals resulting from neural network training in a variety of experimental configurations$

714031

johnowhitaker.bsky.social @johnowhitaker.bsky.social · 17/11/2024

Hello bsky, I'm not sure what I'll post where - Twitter/X is still my default. But I do also put most of the interesting stuff on my site's "Everything" page, which has an RSS feed, if you want to see that stuff without X :) johnowhitaker.dev/all.html

070