Reposted by Jonathan FrankleRyan Goodman @rgoodlaw.bsky.social · 28/03/2025This is how it's done. A strong and principled response by WilmerHale to the illegal Executive Order attack - a form of attempted government intimidation declared unconstitutional by a federal judge. This is how to guard the rule of law. 438262076624
Jonathan Frankle @jfrankle.com · 25/03/2025The hardest part about finetuning is that people don't have labeled data. Today, @databricks.bsky.social introduced TAO, a new finetuning method that only needs inputs, no labels necessary. Best of all, it actually beats supervised finetuning on labeled data. www.databricks.com/blog/tao-usi...databricks.comTAO: Using test-time compute to train efficient LLMs without labeled dataLIFT fine-tunes LLMs without labels using reinforcement learning, boosting performance on enterprise tasks. 0346
Reposted by Jonathan FranklePrithviraj "Raj" Ammanabrolu @rajammanabrolu.bsky.social · 27/02/2025Join @kumarde.bsky.social Bryan, and me in CSE tomorrow as we do Hot Ones for Academics. My normally spicy research takes will get even spicier 182
Reposted by Jonathan FrankleTian Jin @tjin.bsky.social · 27/02/2025Excited to share our work with friends from MIT/Google on Learned Asynchronous Decoding! LLM responses often contain chunks of tokens that are semantically independent. What if we can train LLMs to identify such chunks and decode them in parallel, thereby speeding up inference? 1/N 1179
Reposted by Jonathan FrankleAndrew Drozdov @mrdrozdov.com · 26/02/2025We're probably a little too obsessed with zero-shot retrieval. If you have documents (you do), then you can generate synthetic data, and finetune your embedding. Blog post lead by @jacobianneuro.bsky.social shows how well this works in practice. www.databricks.com/blog/improvi...databricks.comImproving Retrieval and RAG with Embedding Model FinetuningFine-tune embedding models on Databricks to enhance retrieval and RAG accuracy with synthetic data—no manual labeling required. 195
Reposted by Jonathan FrankleJames Grimmelmann @jtlg.bsky.social · 28/01/2025In case it is not clear from my reposts, the Trump administration is engaged in an illegal AND unconstitutional to seize power over the federal government away from Congress and the courts. "Pausing" payment on the government's bills is just one part of it, but it is among the worst. 1247
Jonathan Frankle @jfrankle.com · 27/01/2025Being right for the wrong reasons doesn't increase my confidence... 160
Jonathan Frankle @jfrankle.com · 27/01/2025All the more convinced that the markets don't understand AI. Both the irrational hype and the irrational pessimism. DeepSeek is incredibly bullish for GPU sales... 2101
Jonathan Frankle @jfrankle.com · 22/01/2025Very excited that our Series J is complete. Especially thrilled to have our friends at Meta on board!cnbc.comMeta backs Databricks as the data analytics startup inches toward IPOMeta rarely invests in startups, but it works with Databricks on the Llama open-source models that Meta trains. 1121
Reposted by Jonathan FrankleJoshua J. Friedman @joshuajfriedman.com · 20/01/2025This is so bad 10068148895
Reposted by Jonathan FrankleEugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 20/01/2025I wasn’t expecting a nazi salute on day 1 but here we are. I of course understand that due to the palm on heart there’s plausible deniability but we all understand the intent 3432
Jonathan Frankle @jfrankle.com · 20/01/2025Impressed by those able to talk about Deepseek right now. 2402
Jonathan Frankle @jfrankle.com · 18/01/2025Interesting Friday evening code drop from @rajammanabrolu.bsky.social and Brandon Cui at @databricks.bsky.social. That's all I'm allowed to say for now... github.com/databricks/c...github.comGitHub - databricks/Compose-RLContribute to databricks/Compose-RL development by creating an account on GitHub. 0131
Reposted by Jonathan FrankleMetropolitan Transportation Authority @mta.info · 05/01/2025Congestion Relief Zone tolling is now in effect. Learn more: congestionreliefzone.mta.infocongestionreliefzone.mta.infoCongestion Pricing Program in New York - MTA 231029192
Reposted by Jonathan Frankle🐘 @pkydrm.bsky.social · 19/12/2024🧵 Super proud to finally share this work I led last quarter - the @databricks.bsky.social Domain Intelligence Benchmark Suite (DIBS)! TL;DR: Academic benchmarks ≠ real performance and domain intelligence > general capabilities for enterprise tasks. 1/3 454
Reposted by Jonathan FranklePrithviraj "Raj" Ammanabrolu @rajammanabrolu.bsky.social · 17/12/2024Databricks raises $10b Series J at $62b valuation, the largest venture round ever. www.databricks.com/company/news... 1112
Jonathan Frankle @jfrankle.com · 17/12/2024The world needs data intelligence, and @databricks.bsky.social is delivering. Thank you to the investors who continue to support us on this journey. 🧱🧱🧱 www.databricks.com/company/news...databricks.comDatabricks is Raising $10B Series J Investment at $62B ValuationFunding led by new investor Thrive Capital Company expects to cross $3B in revenue run rate and achieve positive free cash flow in fourth quarter 1143
Jonathan Frankle @jfrankle.com · 13/12/2024Lastly, thank you as always to the amazing team at @databricks.bsky.social and the scientific and open source communities. You all keep me especially excited about the bright future we're creating. The folks at Meta, AI2, Eleuther, HuggingFace, Kaggle, among many many others. 030
Jonathan Frankle @jfrankle.com · 13/12/2024TLDR: See the TLDR at the top of the thread. Merry NeurIPS to everyone in the AI community. 2025 will be an exciting year ♥️🧱📈 120
Jonathan Frankle @jfrankle.com · 13/12/202411. My understanding of semiconductor progress is that, even it looks nice on a log/log plot, progress was never certain and hard-fought new ideas were always needed to get to the next step. If we knew how to get straight to 2nm, we wouldn't have done 65nm or 4nm. 130
Jonathan Frankle @jfrankle.com · 13/12/202410. Next metaphor: Moore's "law." Gordon Moore wrote a great article in 2003 reflecting on that trend called "No Exponential is Forever...But Forever Can Be Delayed!" cseweb.ucsd.edu/classes/wi10...cseweb.ucsd.edu 120
Jonathan Frankle @jfrankle.com · 13/12/20249. Even the worst case scenario for progress (model quality freezes forever at the level of GPT4++ and cost to use it keeps coming down) will lead to decades of new ideas and advances on top of AI that will leave the world transformed. Even the most bearish case is bullish. 160
Jonathan Frankle @jfrankle.com · 13/12/20248. But the world was transformed by the internet. It took decades of trial and error and experience and evolution and culture to make the most of it. Lower cost and greater access meant things that didn't make sense before (video sharing) later did. Imagine explaining "demure" to someone in 1995. 130
Jonathan Frankle @jfrankle.com · 13/12/20247. I look to the history of computing. From what I understand, the technology behind the internet was largely fixed ("ossified" according to some) by the mid 90s. All that changed between then and now is that cost came down and access improved. 170
Jonathan Frankle @jfrankle.com · 13/12/20246. Or maybe an incremental gain is enough to unlock extraordinary economic value that far outstrips the overall cost of building and using the model. 130
Jonathan Frankle @jfrankle.com · 13/12/20245. Maybe scaling trends continue but incremental bumps on real tasks mean the models aren't worth deploying given inference costs. I don't have inside info, but my guess is this is why we don't have Gemini 1.5 Ultra or Claude 3.5 Opus. Of course someone tried to train them. 140
Jonathan Frankle @jfrankle.com · 13/12/20244. Either way, this particular trend doesn't directly say much about the overall success of AI. It doesn't account for the full cost or benefits. It ignores the full cost of training (data, etc), inference costs (how much is it used), and value (what is reducing loss getting you on real tasks?) 150
Jonathan Frankle @jfrankle.com · 13/12/20243. Even if this precise (and very aggressive) scaling trend didn't continue for whatever reason (and there could be many reasons), the technology will continue to improve. Just at a slower rate as measured in this very specific way. We may find new ways to scale with nice properties. We may not. 140
Jonathan Frankle @jfrankle.com · 13/12/20242. Scaling "laws" ("trends"?) as formally defined (log loss decreases linearly with log training FLOPs when properly allocated between parameters and data) may or may not continue. There's no reason to doubt that it will continue, yet extrapolating far out is uncertain. 250
Jonathan Frankle @jfrankle.com · 13/12/20241. Of course AI is going to keep getting better. Technological progress isn't going to stop or "hit a wall." That's very safe to say. There's a ton of low-hanging fruit. 160
Jonathan Frankle @jfrankle.com · 13/12/2024None of this is novel. Smarter people than me are saying the same thing. But it merits repeating as everyone (students, VCs, scientists, practitioners) nervously swings from hype to "it's all over" to "we're so back." Admire the progress. Focus on making this stuff as useful as possible 180
Jonathan Frankle @jfrankle.com · 13/12/2024TLDR: AI progress will continue. Scaling laws, formally, may or may not, but that doesn't matter commercially. What matters is ROI: are lifecycle costs of AI worth it? That's hard to measure and predict. Even in the worst case, though, I think we're in for decades of innovation. 1130
Jonathan Frankle @jfrankle.com · 13/12/2024Reflections on NeurIPS: There's always a big theme people seem to be preoccupied with. This year, it was the continuation of scaling/progress. Will it continue? What will the next generation of models hold? I even got to sass Dylan Patel (not on bsky) over it. Here are my personal thoughts 🧵 35911
Reposted by Jonathan FrankleNikhil Thorat @nst.bsky.social · 11/12/2024We just released the Databricks synthetic evals SDK! 🎉 We’ve found that synthesizing evals is a great way to hill climb your AI system before you’re able to get labels from domain experts. We’ve also recently release a new diff UI that lets you both qualitatively and quantitatively view results! 152
Reposted by Jonathan FranklePermanence AI @permanence.ai · 10/12/2024Interested in massively reducing the toil of code maintenance? Permanence AI is at #NeurIPS2024 this week! Tyler Holloway and Ethan Elenberg will be presenting work at the ML For Systems workshop, and our Founder/CEO Joseph Hackman is attending the conference as well. Let's catch up! #AI #ML 022
Jonathan Frankle @jfrankle.com · 08/12/2024See you at NeurIPS! I'll be there Tuesday to Friday. Find me at the @databricks.bsky.social booth at the expo. 0150
Reposted by Jonathan FrankleCody Blakeney ✈️ NeurIPS 2024 @codestar.bsky.social · 26/11/2024Checking some great work my team did on continued pretraining of LLMs! 2152
Reposted by Jonathan FrankleOfir Press @ofirpress.bsky.social · 25/11/2024I wrote some thoughts on how to build good LM benchmarks: ofir.io/How-to-Build...ofir.ioHow to Build Good Language Modeling BenchmarksBuilding benchmarks is important because they shine a spotlight on the weaknesses of existing language models and so can guide the community on how to improve them. 5778
Reposted by Jonathan FranklePratyush Maini @pratyushmaini.bsky.social · 25/11/20241/5 Earlier this year, I joined @datologyai.com to give wings to the data research I had been doing in academia. Today, I am absolutely thrilled to share what we’ve been working on! Techvember Ep 2: How we made the #1 LLM Pre-training Data Recipe. Blog: 👉 tinyurl.com/best-llm-data 🧵 1154
Reposted by Jonathan FrankleAndrew Drozdov @mrdrozdov.com · 20/11/2024Mat is not on 🦋—posting on his behalf! It's time to revisit common assumptions in IR! Embeddings have improved drastically, but mainstream IR evals have stagnated since MSMARCO + BEIR. We ask: on private or tricky IR tasks, are rerankers better? Surely, reranking many docs is best? 48224
Jonathan Frankle @jfrankle.com · 15/11/2024@leavittron.bsky.social - please tell Ari to get on Bluesky! 110
Jonathan Frankle @jfrankle.com · 15/11/2024Incredibly excited for the team at @datologyai.bsky.social for huge progress on data efficiency for CLIP model training. I'm a big fan of lines that go up and to the left, and this is a really substantial improvement. 290
Jonathan Frankle @jfrankle.com · 14/11/2024Hi AI friends - I'm trying to find everyone again. Please let me know if you're here so I can follow you! 3214