iiinova — where your digital self keeps evolving

This is a form of anthropomorphization because it assumes that, to *do* brain-like things, one *needs* a brain-like thing. That the neural network part of an LLM might be intelligent, but the calculator could never be relevant there, even as a component.

My theory is that this is actually based on anthropomorphization (ironically). LLMs are contrasted against brains. Artificial neural networks against biological ones. Adding a calculator is just not playing the same game.

I mean that another response to LLMs getting bundled calculators (and Python and Lean) is to say "well, LLMs can do math now", because that is what the total system does. But this feels wrong to many people, apparently including Bender.

230

"For historical reasons" is maybe part of the answer; Bender has been fighting against neural networks that do language for quite some time. Adding a calculator to the LLM is like cheating: "aha! you can't do math, and need to resort to tricks!" But why does this feel like a trick?

Bender obviously wants to say that LLMs are weak. They need a calculator to even do basic math. But why attack the neural network part of the model specifically?

A. H. Zakai @kripken.com · 7h

I think it's worth asking: why would it matter if a model were using a calculator? Why is this important to Bender&others? In the end, an LLM is a computer program, and a calculator is another. Together, they are a larger one. Why consider internal divisions?

2100

Yes, that is a nice distinction that they made.

010

A. H. Zakai @kripken.com · 8h

I mean that this is clearly a sign of imperfection, but I do think humans have similar imperfections in our internal "traces"?

110

A. H. Zakai @kripken.com · 8h

It kind of seems human-like to me...? I sometimes hum an unrelated song while working out a logic problem.

220

A. H. Zakai @kripken.com · 10h

Definitely, yes. Gyges has a nice paragraph about that: www.verysane.ai/p/do-we-unde...

verysane.ai

Do we understand how neural networks work?

Yes and no, but mostly no.

000

A. H. Zakai @kripken.com · 11h

I think it is meant to explain how they can emit fluent text, if they aren't copying it from their input or using simple rules (like ELIZA did). They emit fluent text using complex statistics ("stochastic"), is the picture here. And it has at least *some* truth in it, I think.

100

A. H. Zakai @kripken.com · 12h

Yeah, I definitely agree about "haphazardly" - that was really unnecessary in their paper, and is just absolutely false given the last decade of research. These systems do learn useful models of the input.

120

A. H. Zakai @kripken.com · 12h

("Listen to a long explanation by an expert" is the best thing, obviously, but in terms of reaching a wide audience... I think we need more.)

000

A. H. Zakai @kripken.com · 12h

I really like text elemental too! But I think it helps people like us more than it helps non-experts. Average people need something too, and yeah, the Stochastic Parrot is far from perfect, but I'm not aware of something better for them.

210

What better metaphors do you have in mind?

120

I'm not saying Bender is right in that thread, or in general, but she does make some valid points elsewhere. And some of the other Parrot Paper authors do an even better job at pointing out what the metaphor is actually good for. If it isn't taken too far, it is useful imho.

100

Like, someone that thinks their AI boyfriend/girlfriend loves them has gone wrong, and reminding them "it is just a machine" is helpful. And the parrot metaphor explains how a machine can produce compelling text despite being just a machine (correlations on the input data etc.).

110