iiinova — where your digital self keeps evolving

Dan Butler @danieljbutler.bsky.social · 23/07/2025

It would be fun for a benchmark to focus on problems that are more "visual" - truths that are easy for humans to "see" but hard for them to prove formally

000

Dan Butler @danieljbutler.bsky.social · 22/07/2025

Isn't natural language still awfully close to a formal / symbolic domain? Human mathematical intuition seems grounded in spatiotemporal relationships, not natural language.

110

Dan Butler @danieljbutler.bsky.social · 17/07/2025

It could be called "turbulence"

010

Dan Butler @danieljbutler.bsky.social · 14/07/2025

The Bluesky Python SDK is so cool!

000

Dan Butler @danieljbutler.bsky.social · 29/06/2025

Length of chain of thought does indeed correlate to difficulty - see attached

100

Dan Butler @danieljbutler.bsky.social · 29/06/2025

I’m genuinely confused by these statements. Chain of thought length absolutely does correlate to difficulty - generally the LLM will stop thinking when it reached a reasonable answer. Likewise in human reasoning!

100

Dan Butler @danieljbutler.bsky.social · 28/06/2025

The number of tokens doesn't necessarily stay the same, does it? LLMs can execute algorithms and output the stored values at intermediate steps as tokens, so the number of tokens / amount of computation scales up with the difficulty of the problem (size of the input, in the case of factorization)

100

Dan Butler @danieljbutler.bsky.social · 28/06/2025

But isn’t it just a constant amount of compute per token? Producing more tokens involves using more time and space. Chain of thought, etc.

100

Dan Butler @danieljbutler.bsky.social · 26/06/2025

By contrast, good explanatory scientific theories generalize to broader set of "perturbations" than just the types of experiments that went into constructing the theory. Watson and Crick's model of DNA was not just a way to predict x-ray diffraction patterns.

130

Dan Butler @danieljbutler.bsky.social · 26/06/2025

Totally right, you said something different. You're much more pro- this type of model learned from perturbation data. My concern is that you end up with a causal model, yes - but the perturbations are drawn from a very constrained distribution. The ML model can more or less memorize them.

110

Dan Butler @danieljbutler.bsky.social · 24/06/2025

Also notable that this type of work doesn't use any of the conditional independence assumptions that are common in the causal modeling community @alxndrmlk.bsky.social

110

Dan Butler @danieljbutler.bsky.social · 24/06/2025

@kordinglab.bsky.social argued in a recent talk that you can't learn a model from canned data that will let you simulate perturbation experiments. bsky.app/profile/kemp... But this type of model seems darn close.

100

Dan Butler @danieljbutler.bsky.social · 24/06/2025

Cool work out of @arcinstitute.org . My question is, do models like this let us perform novel in-silico experiments the way first-principles models do, or are they just clever way of extrapolating existing experimental data from one context to another?

110

Reposted by Dan Butler

Philip Hubbard @philiphubbard.bsky.social · 18/06/2025

Cleaning up disk space, I found this image I made for someone not long after the release of the #HHMIJanelia #Drosophila hemibrain #connectome in 2020. It shows EPG neurons in pink providing inputs to PFL1 neurons in transparent grey. I'm not sure if the image was ever used.

Two sets of connecting fly neurons with fine, wispy arbors.

1112

Dan Butler @danieljbutler.bsky.social · 14/06/2025

Philip did mention a MW talk from Zurek I think

100

Dan Butler @danieljbutler.bsky.social · 10/06/2025

Do we know if the number of steps they can perform is related to how many steps they saw in their training data? Can RL fine-tuning increase the number of steps?

000

Dan Butler @danieljbutler.bsky.social · 08/06/2025

Does anyone know what species this is? Would love to know more about what structures play the role of nervous system and muscles

000

Dan Butler @danieljbutler.bsky.social · 08/06/2025

Who knew that Chargaff was into this stuff

000

Reposted by Dan Butler

Ricard Solé @ricardsole.bsky.social · 08/06/2025

Against reductionism: "Our understanding of the world is built up of innumerable layers. Each is worth exploring, as long as we do not forget that it is one of many. Knowing all there is to know about one layer (...) would not teach us much about the rest". Erwin Chargaff

33810

Dan Butler @danieljbutler.bsky.social · 07/06/2025

Things that aren’t chocked full of information-bearing molecules

080

Dan Butler @danieljbutler.bsky.social · 07/06/2025

Because the kind of theories we want involve phenomena that span 3-4 orders of magnitude in space (synapses vs. brains) and 6-7 orders of magnitude in time (action potentials vs. skill acquisition)?

0180

Dan Butler @danieljbutler.bsky.social · 30/05/2025

There’s a good definition of computational universality (Church-Turing) - why couldn’t there be one of general intelligence?

020

Dan Butler @danieljbutler.bsky.social · 25/05/2025

If constructor theory told us something amazing *was* constructible, it might help motivate us to build it. Conversely we could avoid wasting our time on things not even constructible in principle.

000

Dan Butler @danieljbutler.bsky.social · 24/05/2025

Quiet posters feed. You’re welcome.

000

Dan Butler @danieljbutler.bsky.social · 23/05/2025

To all the international students, post-docs, scientists, and other academics I’ve been friends with over the years - we support you, and we want you here

000

Dan Butler @danieljbutler.bsky.social · 22/05/2025

What do you mean by "information about"?

100

Dan Butler @danieljbutler.bsky.social · 21/05/2025

No. Burning a library destroys something. Not physical information (that’s left in the heat and ash) but knowledge about the world. Whatever the fire is destroying, the brain can create “de novo”. It’s not conserved.

100

Dan Butler @danieljbutler.bsky.social · 18/05/2025

Physics is also information-preserving. So there’s been no “new” information since the Big Bang. But there must be some other sense in which new things do come into existence.

180

Dan Butler @danieljbutler.bsky.social · 15/05/2025

New information, no. But new ideas, new knowledge, yes. Einstein didn’t acquire relativity from observations, he invented it.

110

Dan Butler @danieljbutler.bsky.social · 15/04/2025

@annakaharris.bsky.social @philipgoff.bsky.social All our *discourse* about C is 3rd-person observable - neurons firing, vocal cords moving, etc. We expect a boring old physical story one day. Won't that story undercut panpsychism? @seanmcarroll.bsky.social did you ever get a satisfying answer?

020

Dan Butler @danieljbutler.bsky.social · 11/04/2025

To really defend consciousness as fundamental, I don't think you can make the move where you concede that it follows the laws of physics. Once you do that, it loses its relevance in any explanation of anything - because the laws of physics do that explanatory work.

010

Dan Butler @danieljbutler.bsky.social · 11/04/2025

Fair - but the "physics software runs on consciousness hardware" idea, specifically, doesn't provide the tidy solution you say it does. It leaves a very awkward problem: the subjective qualia of the hardware don't play any role in explaining why philosophy-of-mind books are written by the software.

110

Dan Butler @danieljbutler.bsky.social · 10/04/2025

Say the program *is* running on silicon. The fact that it’s silicon doesn’t play any role in explaining why the program *says* it’s running on silicon. To explain that, you need to talk about conditional branching, arithmetic, for-loops etc. The program would say the same thing on any substrate.

100

Dan Butler @danieljbutler.bsky.social · 10/04/2025

Back to the original issue. Isn’t it like a program saying it can feel that it’s running on silicon? It might be true, but that’s definitely not why the program is saying what it’s saying.

100

Dan Butler @danieljbutler.bsky.social · 09/04/2025

Sorry. What I should have written was, “that is exactly the source of my disagreement with them.”

120

Dan Butler @danieljbutler.bsky.social · 09/04/2025

That is why panpsychists are confused. The software of physics could run on any hardware and all our discourse about consciousness would be the same. Discourse about consciousness is also software.

110

Dan Butler @danieljbutler.bsky.social · 01/04/2025

There’s this idea that intelligence involves good predictive models of the world. But most of our lives, we can barely predict anything. We are however extremely good at explaining things in retrospect. I expect human-like AI will be similar

000

Dan Butler @danieljbutler.bsky.social · 31/03/2025

In the battle of novelty versus real engineering, real engineering usually wins in the end

000

Dan Butler @danieljbutler.bsky.social · 31/03/2025

I feel Berkeley systems researchers have always been good at doing projects that were "just engineering", but doing it in an academic environment, and being incredibly successful

120

Dan Butler @danieljbutler.bsky.social · 27/03/2025

So there's some causal story that explains exactly how your brain produced this book - and that causal story does not depend in any way whatsoever on whether you're made of consciousness (as you think) or something else entirely.

000

Dan Butler @danieljbutler.bsky.social · 27/03/2025

What do you think about the "meta problem"? Humans produce speech about C - and since speech production is an "easy" problem, there's some causal story that explains how that speech is produced. Here's the rub: that causal story doesn't depend on what the fundamental stuff *is*, only what it *does*.

100

Dan Butler @danieljbutler.bsky.social · 27/03/2025

Much more useful than a massive predictive model of raw brain activity would be an LLM trained to ask good questions about brain data, and then try to answer them.

010

Dan Butler @danieljbutler.bsky.social · 26/03/2025

Good explanatory theories in biology usually miss a lot of the variance - evolution, central dogma, etc.

010

Dan Butler @danieljbutler.bsky.social · 25/03/2025

More concretely though - when C. elegans turns its body in pursuit of food or to flee a predator, wouldn't you expect a lot of the same locomotor circuits to be used to execute the turn? That's the kind of modularity evolution should favor. And a satisfying explanation will need to capture that.

110

Dan Butler @danieljbutler.bsky.social · 25/03/2025

"Strong modularity limits the number of effective parameters and hence the amount of functionality you can implement. If the organ systems started with modularity, there are lots of edge cases that can be improved with violations of modularity. Evolution does it."

110

Dan Butler @danieljbutler.bsky.social · 25/03/2025

Right, but the same argument works for organs and tissues, so the fact that we still see a high degree of modularity in those systems says evolution doesn't totally disfavor it.

120

Dan Butler @danieljbutler.bsky.social · 24/03/2025

Not very. Was thinking of genetically programmed nervous systems that need to encode lots of behaviors out-of-the-box. Neocortex probably more like a memory array - lots of repeated structure to store lots of information. But even without morpho modularity, weight structure could be - like ANNs

110

Dan Butler @danieljbutler.bsky.social · 23/03/2025

Are they? Anatomy is extremely modular. Why would the central nervous system be different?

100

Dan Butler @danieljbutler.bsky.social · 22/03/2025

Show me a system of 300 elements that implements hundreds of complex functions with *no* modularity.

110

Dan Butler @danieljbutler.bsky.social · 22/03/2025

To implement countless behaviors in just 300 computational units, they must be taking advantage of modularity and compositionality.

100