iiinova — where your digital self keeps evolving

Flaviu Cipcigan @flaviucipcigan.bsky.social · 22/02/2025

Super interesting application of program search Goals are mapped to programs which are embedded in a latent space. A fitness metric is assigned to the programs and program search is done to synthesise new human-like goals.

040

Flaviu Cipcigan @flaviucipcigan.bsky.social · 20/02/2025

One of my big motivations is accelerating science with AI. Every discovery project had a beautiful aha moment, such as the structure of antibiotics emerging in the latent space of a model or a GFlowNet proposing new carbon capture materials. Here's some of the threads I've wrote on this topic.

170

Flaviu Cipcigan @flaviucipcigan.bsky.social · 17/02/2025

Wanna try to guess which of those gets parsed as a string and which as a number? Answer in alt text. YAML parsing in python is weird.

{'lol': ['5.0E6',
'5.0e6',
'5.E6',
'5.e6',
'5E6',
'5e6',
5e-06,
5e-06,
5e-06,
5e-06,
'5E-6',
'5e-6',
5000000.0,
5000000.0,
5000000.0,
5000000.0,
'5E+6',
'5e+6']}

240

Flaviu Cipcigan @flaviucipcigan.bsky.social · 17/02/2025

Interesting idea to generate responses using diffusion rather than left-to-right auto-regressive models

061

Flaviu Cipcigan @flaviucipcigan.bsky.social · 15/02/2025

What is large for a language model? Is it 400B, 70B or maybe 1T? I think focus on raw number of parameters is a less useful frame than thinking about inference speed, cost and location of inference (on-device vs cloud).

110

Flaviu Cipcigan @flaviucipcigan.bsky.social · 13/02/2025

More open reasoning datasets and distilled models. It's great to see the energy of the community that got unleashed after open models that generate chains of thought!

020

Flaviu Cipcigan @flaviucipcigan.bsky.social · 13/02/2025

ColabFit Exchange is another great dataset curation effort that I'd like to boost. Great work by @stemartiniani.bsky.social and team to curate the most diverse materials database in the world!

011

Flaviu Cipcigan @flaviucipcigan.bsky.social · 13/02/2025

Neat idea! Fine-tuning using majority voting and length filtering generalises a model's capabilities. Models generalise to slightly harder versions of a problem, and the correct answers are used to bootstrap the next model and the next one and so on.

041

Flaviu Cipcigan @flaviucipcigan.bsky.social · 13/02/2025

Join us in creating open datasets, benchmarks and leaderboards for materials discovery.

110

Flaviu Cipcigan @flaviucipcigan.bsky.social · 12/02/2025

The most durable motivation for research is curiosity, the desire to answer a question or understand something. Curiosity then leads you down a maze of existing answers and new questions. Eventually, you get to one that has no answer and then you start pushing at the frontier.

020

Flaviu Cipcigan @flaviucipcigan.bsky.social · 09/02/2025

Interesting - 57.1% AIME24 and 94.8% MATH performance achieved using only 817 reasoning chains and STF. Adds more weight to the hypothesis that correct reasoning chains and SFT can lead to strong reasoning performance.

github.com

GitHub - GAIR-NLP/LIMO: LIMO: Less is More for Reasoning

LIMO: Less is More for Reasoning. Contribute to GAIR-NLP/LIMO development by creating an account on GitHub.

170

Flaviu Cipcigan @flaviucipcigan.bsky.social · 09/02/2025

I've been reflecting today about OpenAI's five levels to measure progress in AI. GPT-4 was at Level 1, conversational AI: a model competent at 0.1-1s tasks, like holding a conversation. O1 / R1 reached Level 2, reasoners: a model solving 1-10min tasks such as basic coding tasks and math.

160

Flaviu Cipcigan @flaviucipcigan.bsky.social · 06/02/2025

Agreed, we have the minimum viable scene. We now just need to amplify each other and keep going.

130

Flaviu Cipcigan @flaviucipcigan.bsky.social · 05/02/2025

What if inference scaling is as simple as response.replace("</think>", "Wait")

020

Flaviu Cipcigan @flaviucipcigan.bsky.social · 05/02/2025

SWE arena is going to be an interesting leaderboard to watch. It allows people to compare the code generated by LMs based on runs inside a sandbox.

swe-arena.com

SWE Arena: Compare & Test Best AI Chatbots for Code

010

Flaviu Cipcigan @flaviucipcigan.bsky.social · 04/02/2025

every time i try uv, I'm more impressed. seems now like a tool that Just Works, reducing the complexity of the python ecosystem installed a cuda+torch+git packages and it all felt basically instant

2220

Flaviu Cipcigan @flaviucipcigan.bsky.social · 28/01/2025

DeepSeek-R1 has turned into such a Rorschach test for the collective psyche

150

Flaviu Cipcigan @flaviucipcigan.bsky.social · 26/01/2025

Indeed, not outsourcing reasoning is an important value to ... well... reason about. How would we achieve this? It may require many individuals and groups to do RL on their own models, using their own verifiers. This may look like grading exams - not of students, but of ML models.

121

Flaviu Cipcigan @flaviucipcigan.bsky.social · 26/01/2025

Seeing A Film for the Future in 360 was a special experience. One of the most powerful parts was We Pray. The video and music match so well, hit hard, and resonate strongly with the times.

youtu.be

Coldplay - WE PRAY (A Film For The Future)

YouTube video by Coldplay

000

Flaviu Cipcigan @flaviucipcigan.bsky.social · 25/01/2025

Turning the temperature up using R1 Starting to think gibberish gibberish gibberish Focus again. Calm up. 🤣

191

Flaviu Cipcigan @flaviucipcigan.bsky.social · 25/01/2025

Hm, using reasoning models really feels qualitatively different (using @openrouter.bsky.social for inference). It's fun to see these aha moments and it'd be interesting to understand whether their presence helps.

160

Flaviu Cipcigan @flaviucipcigan.bsky.social · 22/01/2025

Huh, interesting, Claude 3.5 sonnet seems to do hidden CoT in the app. Could not reproduce with the API tho.

160

Flaviu Cipcigan @flaviucipcigan.bsky.social · 20/01/2025

Deepseek-R1 thread to gather thoughts and reactions Nice to see the technical details and MIT license for something that looks at o1 level 🥳

github.com

GitHub - deepseek-ai/DeepSeek-R1

Contribute to deepseek-ai/DeepSeek-R1 development by creating an account on GitHub.

2516

Flaviu Cipcigan @flaviucipcigan.bsky.social · 20/01/2025

Interesting result re evolutionary algos for inference time search

060

Flaviu Cipcigan @flaviucipcigan.bsky.social · 15/01/2025

A powerful feature of the AT Proto is that it uses domain names as the handle. If I find an interesting blog post, I can often find the author by searching for the domain name!

010

Flaviu Cipcigan @flaviucipcigan.bsky.social · 13/01/2025

A great abstraction "allows us to operate as if the underlying complexity simply does not exist". For example, TCP turns an unreliable protocol into a reliable communication channel. Otoh, indirections are justified in terms of modularity, yet often just add needless cognitive overhead.

fhur.me

fhur

fhur's blog

040

Flaviu Cipcigan @flaviucipcigan.bsky.social · 08/01/2025

A critique I hear often of LLMs is that they don't have a notion of truth, that they are BS machines, in Frankfurt's sense. I don't think that's quite right. Here's two papers that helped me have a more nuanced view of this question.

3175

Flaviu Cipcigan @flaviucipcigan.bsky.social · 07/01/2025

There's a lot of enthusiasm in the community about transformers trained on chemical or biological data. Here's some interesting results and some thoughts on future directions.

1123

Flaviu Cipcigan @flaviucipcigan.bsky.social · 07/01/2025

In 1975, the Altair 8800 was released at about $3000 (inflation adjusted). It was programmed using individual switches and its display was a bunch of lights on the front panel. Nonetheless, the price was low enough to start a hobbyist community and catalyse the PC community.

140

Flaviu Cipcigan @flaviucipcigan.bsky.social · 05/01/2025

Comparing vectors of landmarks with a remote database is the first product use of homomorphic encryption I've heard of. It's a good one! Privacy-preserving RAG with local LLM and remote documents could be done in a very similar way.

Put more simply: You take a photo; your Mac or iThing locally outlines what it thinks is a landmark or place of interest in the snap; it homomorphically encrypts a representation of that portion of the image in a way that can be analyzed without being decrypted; it sends the encrypted data to a remote server to do that analysis, so that the landmark can be identified from a big database of places; and it receives the suggested location again in encrypted form that it alone can decipher.

140

Flaviu Cipcigan @flaviucipcigan.bsky.social · 05/01/2025

I believe there's a good parallel here with thermodynamics. Steam engines gave us thermodynamics, which gave us better engines. Thermodynamics then became one of the most fundamental theorems of physics, allowing us to e.g. understand distant stars.

120

Flaviu Cipcigan @flaviucipcigan.bsky.social · 03/01/2025

This paper indicated that chicks and neural networks learn similar representations when given identical data. A corollary is that, to get better neural networks, it's worth investing in things like online data gathering, active learning and machine curiosity.

020

Flaviu Cipcigan @flaviucipcigan.bsky.social · 03/01/2025

Good reference and talk guessing how o1 was trained.

120

Flaviu Cipcigan @flaviucipcigan.bsky.social · 03/01/2025

Another benchmark where o-class models show a jump compared to GPT-class models (arXiv:2406.04520) Mystery Blocksworld is a block stacking task where the names are randomised, requiring generalisation. Still plenty of room to go, but clearly the start of a new s curve.

010

Flaviu Cipcigan @flaviucipcigan.bsky.social · 27/12/2024

One of the great things about AI is how accessible research tooling is. Breakthrough labs (this is from OpenAI) are basically GPUs, Python, monitoring, docs, and a chat app. Even if this post was part in jest, this is a point of joy. We should make sure the culture of openness continues.

040

Flaviu Cipcigan @flaviucipcigan.bsky.social · 24/12/2024

ARC-AGI is one of the most interesting benchmarks in ML. o3 achieving human-level on the semi-private eval feels like a significant breakthrough. Calibrating, I'd say o3 is a GPT-1 or GPT-2 moment. The direction for improvement is getting clear, with more of the research fog getting lifted.

150

Reposted by Flaviu Cipcigan

Flaviu Cipcigan @flaviucipcigan.bsky.social · 07/11/2024

If you're interested in foundation models for materials and molecules, check out our repo: github.com/IBM/materials We have three models released based on SMILES, SELFIES and molecular graphs. More to come shortly - we aim to have a unified collection of state-of art models across all modalities.

github.com

GitHub - IBM/materials: Foundation Model for Materials - FM4M

Foundation Model for Materials - FM4M. Contribute to IBM/materials development by creating an account on GitHub.

1165

Flaviu Cipcigan @flaviucipcigan.bsky.social · 20/11/2024

Nice 5x to 15.6x speed-up of equivariant operations from NVIDIA.

290

Reposted by Flaviu Cipcigan

John Burn-Murdoch @jburnmurdoch.ft.com · 19/11/2024

Despite a massive head start, BlueSky has now overtaken Threads in the US 👇

389169743659

Flaviu Cipcigan @flaviucipcigan.bsky.social · 18/11/2024

This is a super interesting plot. ML conventional wisdom is the bias-variance trade-off. Here is a neural net with a single hidden layer. At first, bias decreases and variance increases. As you train for longer, you get a phase transition and then *both* decrease.

3141

Reposted by Flaviu Cipcigan

Maike Osborne @maosbot.bsky.social · 09/11/2024

New here? Interested in AI/ML? Check out these great starter packs! AI: go.bsky.app/SipA7it RL: go.bsky.app/3WPHcHg Women in AI: go.bsky.app/LaGDpqg NLP: go.bsky.app/SngwGeS AI and news: go.bsky.app/5sFqVNS You can also search all starter packs here: blueskydirectory.com/starter-pack...

66557212

Flaviu Cipcigan @flaviucipcigan.bsky.social · 17/11/2024

A thread of thoughts about challenges in lab automation. My perspective is computational, so I'm sure it will colour my thoughts. Thus, I'd also really enjoy seeing thoughts from more experimental researchers! 🧪

2181

Flaviu Cipcigan @flaviucipcigan.bsky.social · 16/11/2024

One of the exciting things happening in AI for Science has been the growth in lab automation. Automated labs coupled with active learning are a super exciting area with lots of opportunities for progress. I promised @cpaxton.bsky.social a short thread on this, so here it goes! 🧪

14611

Flaviu Cipcigan @flaviucipcigan.bsky.social · 16/11/2024

I've been posting a lot about AI lately, so wanted to also share some of my work on the bio/chem side. Antimicrobial peptides are proteins that kill bacteria. Most do so by making circular holes in their membranes. In this fun to write paper, we showed fractal pores in bacterial membranes.

1215

Flaviu Cipcigan @flaviucipcigan.bsky.social · 15/11/2024

Great to see the AI for Science community growing on BlueSky 🥳

090