iiinova — where your digital self keeps evolving

Reposted by David Hall

Open Athena @openathena.ai · 01/10/2026

Marin's 535B-parameter hero run was originally planned as a 360B model. A new blog from Rafal Wojdyla, MTS, explains how a change in sharding strategy, Expert Parallelism, made room for a ~50% larger model at the same ~23B active parameters, and what it costs. 🔗 openathena.ai/blog/expert-...

0136

Reposted by David Hall

Open Athena @openathena.ai · 03/09/2026

A year ago, Marin had one FTE. Recently, the ten-person team launched a 535 billion parameter MoE model. In the Marin 535B-A23B launch note, David Hall @dlwh.bsky.social notes it's going "almost boringly well." Thank you to our colleagues, partners, and advisors for making this possible. bit.ly/535b

openathena.ai

Marin 535B-A23B launch note

A look back at Marin's first year at Open Athena and forward at the 535B-A23B hero run, the largest model the team has ever trained.

084

Reposted by David Hall

Open Athena @openathena.ai · 25/06/2026

In a new blog, Russell Power explains how the Marin team nearly doubled its sustained TPU usage by creating a custom global scheduler: Iris. Iris searches every region where Marin has compute, places each job wherever capacity appears, and moves data along as needed. 🔗 openathena.ai/blog/cluster...

1102

Reposted by David Hall

Will Held @williamheld.com · 11/05/2026

To train better open models, we need predictable scaling. Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error. Getting there took some work 🧵

2389

David Hall @dlwh.bsky.social · 10/07/2025

I think a lot of federal money is tied to accreditation like Pell grants and research funds and stuff. So while Harvard has lots of money in the endowment, it would still be a pretty big hit to the budget.

160

David Hall @dlwh.bsky.social · 19/05/2025

Many thanks to the Google TPU Research Cloud program for providing the much needed compute for this project, and to all the other great open efforts: @ai2.bsky.social @eleutherai.bsky.social and more!

020

David Hall @dlwh.bsky.social · 19/05/2025

Have a specific use case? Come to our Datashop to curate data and train models. Here’s how we curated more math data: github.com/marin-commun... Check out the data: marin.community/data-browser/

Explanation of data shop: prompt or sample data comes in, llm finds more data, train a cheap model to find even more, train, --> llm

110

David Hall @dlwh.bsky.social · 19/05/2025

Have a new algorithm for training? Choose your compute budget and get on the speedrun leaderboard: how fast can you drive down validation loss? marin.community/speedrun/

pareto frontier of flops vs bits-per-byte

100

David Hall @dlwh.bsky.social · 19/05/2025

Marin (marin.community) repurposes GitHub, which has been successful for open-source *software*, for AI: 1. Preregister an experiment as a GitHub issue 2. Submit a PR, which implements the experiment in code 3. PR is reviewed by experts in the community 4. Watch the execution of the experiment live!

Flowchart shoing Github issue (preregistration) -> pull request (experiment.py) -> execution (watch it live) -> WandB report (analysis)

100

David Hall @dlwh.bsky.social · 19/05/2025

Marin is a new "open lab" for developing foundation models. More than open weights, and even open source, with Marin we're committing to "open development": everything is documented and traceable, and anyone can contribute.

open weights vs open source (weights + code + recipe) vs open development (+ process, anyone can contribute)

110

Reposted by David Hall

Will Held @williamheld.com · 19/05/2025

Learn more about the project in Percy's blog post: marin.community/blog/2025/05... And about the Models we are releasing in @dlwh.bsky.social's training retro: marin.readthedocs.io/en/latest/re...

marin.community

Introducing Marin: An Open Lab for Building Foundation Models

Open-source software is a success story: It powers the world’s digital infrastructure. It allows anyone in the world to contribute based on merit. It leads to greater innovation, collaboration, and se...

001