Sign in

Sham Kakade

@shamkakade.bsky.social
941 followers 89 following 5 posts

Harvard Professor. ML and AI. Co-director of the Kempner Institute. shamulent.github.io

PostsRepliesMedia
Reposted by Sham Kakade
Kempner Institute at Harvard University @kempnerinstitute.bsky.social · 10/02/2025
NEW blog post: Do modern #LLMs capture the conceptual diversity of human populations? #KempnerInstitute researchers find #alignment reduces conceptual diversity of language models. bit.ly/4hNjtiI
bit.ly
Alignment reduces conceptual diversity of language models - Kempner Institute
As large language models (LLMs) have become more sophisticated, there’s been growing interest in using LLM-generated responses in place of human data for tasks such as polling, user studies, and […]
0123
Reposted by Sham Kakade
Kempner Institute at Harvard University @kempnerinstitute.bsky.social · 19/12/2024
NEW in the #KempnerInstitute blog: learn about ProCyon, a multimodal foundation model to model, generate & predict protein phenotypes. Read it here: bit.ly/4fA8xUk
061
Reposted by Sham Kakade
Kempner Institute at Harvard University @kempnerinstitute.bsky.social · 09/12/2024
Calling college grads interested in intelligence research: the application for the #KempnerInstitute's post-bac program w/ the Harvard Kenneth C. Griffin Graduate School of Arts and Sciences Office for Equity, Diversity, Inclusion & Belonging is now open! Apply by Feb. 1, 2025. t.co/jdJrzRegL0
t.co
https://bit.ly/4iohnqE
0144
Reposted by Sham Kakade
Kempner Institute at Harvard University @kempnerinstitute.bsky.social · 09/12/2024
NEW in the #KempnerInstitute blog: A method to predict how #LLMs scale w/ compute across different datasets. Read it here:
bit.ly
Loss-to-Loss Prediction - Kempner Institute
Scaling laws – which reliably predict the performance of large language models (LLMs) as a function of their size and the amount of data they have been trained on – […]
072
Reposted by Sham Kakade
Yuda Song @yus167.bsky.social · 06/12/2024
LLM self-improvement has critical implications in synthetic data, post-training and test-time inference. To understand LLMs' true capability of self-improvement, we perform large-scale experiments with multiple families of LLMs, tasks and mechanisms. Here is what we found: (1/9)
1124
Reposted by Sham Kakade
Kempner Institute at Harvard University @kempnerinstitute.bsky.social · 03/12/2024
NEW: we have an exciting opportunity for a tenure-track professor at the #KempnerInstitute and the John A. Paulson School of Engineering and Applied Sciences (SEAS). Read the full description & apply today: academicpositions.harvard.edu/postings/14362 #ML #AI
02019
Sham Kakade @shamkakade.bsky.social · 22/11/2024
(1/n) 💡How can we speed up the serial runtime of long pre-training runs? Enter Critical Batch Size (CBS): the tipping point where the gains of data parallelism balance with diminishing efficiency. Doubling batch size halves the optimization steps—until we hit CBS, beyond which returns diminish.
2174
Reposted by Sham Kakade
David Brandfonbrener @brandfonbrener.bsky.social · 21/11/2024
How does test loss change as we change the training data? And how does this interact with scaling laws? We propose a methodology to approach these questions by showing that we can predict the performance across datasets and losses with simple shifted power law fits.
1197