iiinova — where your digital self keeps evolving

hailey @hailey.at · 05/02/2026

This is the what I’ve been feeling as Twitter hypes up Claude Code like it’s the second coming of Jesus. Yes it’s very good but it struggles tremendously with high complexity projects and produces consistently worse designs then I’d implement myself, and people aren’t honest about admitting this

001

Reposted by Tobi

this "claude made a c compiler thing" feels pretty dishonest and marketing-hype whenever you scroll to the very bottom to find these lines and the token cost lol i mean, sure its impressive (particularly that it was a fully offline environment), but seems like hype bait

The generated code is not very efficient. Even with all optimizations enabled, it outputs less efficient code than GCC with all optimizations disabled.
The Rust code quality is reasonable, but is nowhere near the quality of what an expert Rust programmer might produce.

81074

Vladimir Salnikov @v4ldelund.bsky.social · 06/02/2026

I tend to prefer GLM 4.5 Air when I’m going for cheap and fast, but Gemini Flash is pretty solid

120

Reposted by Tobi

Also good thing to keep in mind, that afaik the current "subscription token economy" is heavily subsidized and I am not sure Anthropic/OpenAI are charging necessarily full amount (i.e the same price that would charge for API cost)

132

This is why I genuinely believe that in the model game, there is no moat, and there will always be tremendous pressure for open source and cheaper models. The majority of the world economy doesn't want to pay $21 (!!!) per million tokens, even if they can afford to

170

Tobi @mocutobi.bsky.social · 02/02/2026

I've been evaluating Claude Opus 4.6, GPT 5.2, and other models in a simulation environment I've built. The verdict: I ran out of money

2232

If Trump hadn’t sued Trevor Noah for that joke at the Grammy’s, I probably never would’ve heard it. Talk about Streisand Effect

162

Reposted by Tobi

Sung Kim @sungkim.bsky.social · 02/02/2026

ByteDance Seed's ConceptMoE: moving beyond uniform token-level processing to adaptive concept-level computation in LLMs! Why waste equal compute on trivially predictable tokens? When you can merge similar tokens into concepts while preserving fine-grained processing for complex content.

1273

Tobi @mocutobi.bsky.social · 02/02/2026

Thought this was referring to @lastpositivist.bsky.social and got really worried

What batch sizes and sequence lengths are you about to get up to on your dgx spark? I was doing 32b 2048L but saw much faster throughput when I dropped down to 16b

100

What do you use to have Claude make slides?

000

Sidenote: All experiments were performed on the Nvidia DGX Spark machine, my new favorite toy.

These experiments clarify the importance of more holistic evaluation metrics during pretraining. As we move labeled data for reasoning, planning, and other higher order thinking into the pretraining stage, pretraining evaluation that's more aligned with downstream tasks becomes more important

110

Outside of HumanEval, the model pretrained on the SYNTH dataset outperforms the standard nanochat on every task. I report results for both the standard chat template (Harmony) as well as the Qwen3 Chat template used in the SYNTH dataset.

140

The results seemed to speak for themselves, but I was skeptical. The SYNTH dataset is composed of reasoning-style chat messages, it seemed unlikely that general internet data would perform better on downstream chat-style tasks. So I decided to do midtraining, where the results flipped:

Markdown representation of the above table:

| Task | Default Template | Qwen3 Template | Synthetic + Fineweb (Default) | Synthetic + Fineweb (Qwen3) | Synthetic + Fineweb (Qwen3, no thinking) |
|------|------------------|----------------|-------------------------------|----------------------------|-------------------------------------------|
| ARC-Easy | 38.85% | 25.34% | 42.85% | 30.26% | 25.08% |
| ARC-Challenge | 29.95% | 22.78% | 32.94% | 25.85% | 23.04% |
| MMLU | 32.69% | 24.08% | 33.13% | 26.98% | 24.21% |
| GSM8K | 3.64% | 0.08% | 6.14% | 2.20% | 0.08% |
| HumanEval | 8.54% | 1.22% | 1.83% | 0.00% | 0.61% |
| SpellingBee | 98.05% | 0.00% | 98.05% | 35.94% | 0.39% |

100

I used a 3:1 SYNTH:FineWeb data ratio when pretraining. I evaluated nanochat with and without SYNTH data for both 1B pretraining tokens and 3B pretraining tokens to see if adding the data lead to faster convergence. While SYNTH data led to lower val/bpb, CORE metric was consistently lower.

100

Tobi @mocutobi.bsky.social · 28/01/2026

I did some experiments with nanochat and the SYNTH dataset from Pleias. I wanted to compare how a model pretrained with a mixture of SYNTH and FineWeb would compare to the standard FineWeb mixture. Results: Pretraining on Fineweb alone looks good (see below), but the story changes after midtraining:

Pretraining results for training on SYNTH + FineWeb vs FineWeb alone.

Four rows are displayed: dgx1-continued, dgx1-synth-contued, dgx1-synth, and dgx1.

Rows labeled *-synth are trained with 3:1 SYNTH:FineWeb mixture. Results with *-continued* are trained for 3B tokens, 1B otherwise. CORE metric:

nanochat + SYNTH, 3B tokens: 0.147 CORE Metric
nanochat, 3B tokens: 0.166
nanochat + SYNTH, 1B tokens: 0.123
nanotchat, 3B tokens: 0.139

191

Can anyone find that meme with the cartoon cats where all of their butts are AI company logos? I need it for a project

000

Tobi @mocutobi.bsky.social · 22/01/2026

Been running comparisons nanochat pretrained on FineWebEdu vs synthetic data and it’s not looking great for the synthetic data runs thus far. We’ll see what it looks like after mid training

010

Tobi @mocutobi.bsky.social · 12/01/2026

At my day job, I’m building a lot of Agent stuff! On my free time, I’m experimenting with model architectures and pretraining recipes!

050

Tobi @mocutobi.bsky.social · 12/01/2026

Excited to see the AI community has grown on here! Gonna try sharing more about my work here!

4461

Tobi @mocutobi.bsky.social · 03/01/2026

When Bill Clinton left office, we had a budget surplus. After 8 years of disastrous foreign interventions and tax cuts, we had deficit and a recession. Now with Trump we’re repeating repeating the same mistakes made under Bush

050

Tobi @mocutobi.bsky.social · 01/11/2025

I heard this place is getting a dislike button?

Tobi @mocutobi.bsky.social · 11/08/2025

If you assume that there is an infinite number of tech workers with six figure salaries who want to live in the Bay or NY then the NIMBYs are right and no amount of new housing will help. But there is not an infinite number of tech workers, and the ones here are being laid off.

050

Reposted by Tobi

Prime Radiant 🐎🔥 @destinyguardian.blacksky.app · 09/08/2025

People talk about artificial intelligence but don't understand what that technology actually is, what the term "AI" is referring to, or best use cases for all the different tools that exist. Nor do they understand that they've already been using AI assistance for YEARS before LLMs became public.

011

Tobi @mocutobi.bsky.social · 09/08/2025

All these things apply to a small percentage of AI research and companies, yet this website treats all AI models and research like it comes from xAI. You ran the actual open source and AI safety researchers off the app

210

Tobi @mocutobi.bsky.social · 08/08/2025

I think it’s because they’re threatened by it honestly

460

Reposted by Tobi

Neil Lewis, Jr. @neillewisjr.bsky.social · 18/07/2025

Them: It's not about race, it's about class. Hopkins: Ok, we're making a class-based policy. Them: We still don't like it, because what we actually meant was...

841197

Tobi @mocutobi.bsky.social · 03/06/2025

Disappointing to see the president of my Alma mater reject DEI, especially given the racist incidents that marred my time there. Hopefully this rejection will be instructive for Santa Ono’s perspective on building an inclusive educational environment

010

Reposted by Tobi

Barred and Boujee aka Madiba Dennie @audrelawdamercy.blacksky.app · 03/06/2025

"In the past, Ono had condemned systemic racism & argued for the necessity of DEI [...] Ono emphasized his ideological evolution, which ultimately failed to convince the board" Board said hm you're *new to hatin niggas and we're lookin for someone *true to hatin niggas. good luck on the job market!

99416

Reposted by Tobi

I Scare You Like Sound Dog @chadstanton.blacksky.app · 13/05/2025

“The University of Cincinnati wants to make all students eligible for Davis’s scholarship, not just Black applicants. At least six other scholarships worth about $4 million established by Black doctors at the school are also under review, according to an email obtained by The Washington Post.”

washingtonpost.com

Trump DEI policies threaten millions in scholarships raised by Black doctors

Trump’s campaign to DEI programs has put at risk six scholarships worth about $4 million established by Black doctors in Cincinnati.

17220112

Tobi @mocutobi.bsky.social · 13/05/2025

The elephant in the room on social housing is that we severely lack the state capacity required to administer it at the levels required to bring down rents. And increasing state capacity is what’s explicitly being argued for in the Abundance agenda

Michael Hobbes @michaelhobbes.bsky.social · 12/05/2025

Reposted by Tobi

I hear this myth all the time and it’s bizarre. Rich people hire property management companies to handle all the logistics for them and still get 90% of the rent each month. The idea that landlords are forgoing free money en masse makes no sense.

811359109

Reposted by Tobi

Helen De Cruz @helendecruz.net · 12/05/2025

Still my fave music video i made

1431746

Reposted by Tobi

Helen De Cruz @helendecruz.net · 12/09/2024

Ok trying the video upload feature--a Sephardic tune on the baroque lute, Siete Modos de Guisar las Berenjenas (seven ways to prepare eggplants, only two ways are discussed bc we're at 60 seconds).

2229352

Tobi @mocutobi.bsky.social · 13/05/2025

Thank you so much for your contributions to philosophy and democracy. Will forever be grateful you introduced me to Dewey over on Twitter

000

Reposted by Tobi

Michael Hobbes @michaelhobbes.bsky.social · 12/05/2025

Everyone always goes "that might be true for other cities, but where *I* live it's cheaper to leave an apartment empty than rent it out" and it never turns out to be true! Paris has a vacancy rate of 1.3% www.apur.org/en/our-works...

1844938

Tobi @mocutobi.bsky.social · 13/05/2025

“Not sure what CAN be done” build more housing, specifically at a volume commensurate with current population growth and the decades of under building. It’s not rocket science

061

Tobi @mocutobi.bsky.social · 30/04/2025

Someone should make a subscription service that makes it easier for middle aged men in their 40’s to blow up their lives. There could be a portal for boats to buy, red pill podcasts to follow, drugs to abuse, or people to have an affair with. Just a frictionless midlife crisis

010

Reposted by Tobi

Deva Woodly @devawo.bsky.social · 21/04/2025

If I see another article about birth rates that refuses to grapple with the way having children systematically disadvantages women I swear to god. There is no mystery. Stop pretending there is. We’ve got to do away with credulity for nonsense.

633889755

Reposted by Tobi

Tal Lavin @swordsjew.bsky.social · 21/04/2025

this is true and *also* pairing a concern about an "aging labor force" with a desire to deport every immigrant makes it 100% clear this is a white birthrate anxiety thing. this is about white babies

16767170

Reposted by Tobi

Olúfẹ́mi O. Táíwò @olufemiotaiwo.bsky.social · 21/04/2025

decades of reflection about music and traffic safety later I'm finally emotionally ready to admit that "Move Bitch" was a completely unhinged song. every verse on it more antisocial than the previous one

3690174

Tobi @mocutobi.bsky.social · 09/04/2025

What are the last two animes?

000

Tobi @mocutobi.bsky.social · 09/04/2025

Completely unprompted too. The woman was talking about hobbies and going to farmers markets. And then he started talking about how he thinks Indian culture is bad and promotes “dishonesty in business”. You can guess his backfiring