iiinova — where your digital self keeps evolving

Bao Pham

@baopham.bsky.social

26 followers 17 following 4 posts

PhD Student at RPI. Interested in Hopfield or Associative Memory models and Energy-based models.

Posts Replies Media

Reposted by Bao Pham

Dmitry Krotov @krotov.bsky.social · 14/01/2025

I am excited to announce the call for papers for the New Frontiers in Associative Memories workshop at ICLR 2025. New architectures and algorithms, memory-augmented LLMs, energy-based models, Hopfield nets, AM and diffusion, and many other topics. Website: nfam.vizhub.ai @iclr-conf.bsky.social

0166

Reposted by Bao Pham

Dmitry Krotov @krotov.bsky.social · 05/12/2024

Most of the work on Dense Associative Memory (DenseAM) thus far has focused on the regime when the amount of data (number of memories) is below the critical memory storage capacity. We are beginning to explore the opposite limit, when the data is large.

1163

Reposted by Bao Pham

Bao Pham @baopham.bsky.social · 05/12/2024

Diffusion models create beautiful novel images, but they can also memorize samples from the training set. How does this blending of features allow creating novel patterns? Our new work in Sci4DL workshop #neurips2024 shows that diffusion models behave like Dense Associative Memory networks.

On the left figure, it showcases the behavior of Hopfield models. Given a query (the initial point of energy descent), a Hopfield model will retrieve the closest memory (local minimum) to that query such that it minimizes the energy function. A perfect Hopfield model is able to store patterns in distinct minima (or buckets). In contrast, the right figure illustrates a bad Associative Memory system, where stored patterns share a distinctive bucket. This enables the creation of spurious patterns, which appear like mixture of stored patterns. Spurious patterns will have lower energy than the memories due to this overlapping.

1405

Bao Pham @baopham.bsky.social · 05/12/2024

1405