iiinova — where your digital self keeps evolving

Here's my GLM's rumination and final answer on the problem, where it makes 3 estimates and goes "My mental arithmetic is wobbling at the 4th-5th decimal." justpaste.it/az1md

000

alias (in chains) @antiali.as · 48m

parroting during learning may look like "repeat after me" but some results show that models trained on language perform math tasks better than ones trained on just math; how? why?

aclanthology.org

000

alias (in chains) @antiali.as · 50m

why is the SP metaphor still unsatisfying even as MM limits it to describe specifically an LLM? if stochastic means gradient descent and softmax, that describes a huge subset of ML and doesn't distinguish LLM from n-gram Markov.

100

Reposted by alias (in chains)

Margaret Mitchell @mmitchell.bsky.social · 3h

As a fan of both you & Emily, I'll just point out something maybe obvious: Searching web, extraction, etc., isn't an LLM. That's engineered code/functions used alongside an LLM; the distinction between an "AI system" and an LLM (I wrote a thing about this here: medium.com/@margarmitch...) 1/

medium.com

No, “AI” is not a Stochastic Parrot 🦜

I’ve recently come across a new flavor of AI denialism making the rounds.

3283

alias (in chains) @antiali.as · 2h

@darbomusic.com if you're liking my music posts you gotta tell me if you listened and what i can do better ^{^

000

alias (in chains) @antiali.as · 6h

Spessartine gang who up

110

alias (in chains) @antiali.as · 17h

Marketing and promotion tools

010

alias (in chains) @antiali.as · 23h

Alouijas would be a beautiful name for a baby girl

020

alias (in chains) @antiali.as · 23h

like a name George Lucas would give me

020

alias (in chains) @antiali.as · 01/10/2026

Thanks, these are awful, and much better than what I could have vibed up!

3141

alias (in chains) @antiali.as · 01/10/2026

introducing my new model family called Watt

110

alias (in chains) @antiali.as · 01/10/2026

relatively prime ministers, and greatest common ministers

170

alias (in chains) @antiali.as · 01/10/2026

Well, doesn’t really matter either way.

110

alias (in chains) @antiali.as · 01/10/2026

you can easily try for yourself, this was the free version of Sonnet 5.5

100

alias (in chains) @antiali.as · 01/10/2026

no fucking way you're telling me the model worked out a solution methodically instead of just giving the answer?

100

alias (in chains) @antiali.as · 01/10/2026

"calculators are more efficient" is not an argument anyone is denying

110

alias (in chains) @antiali.as · 01/10/2026

it combined base64 encoding/decoding with approximating the ratio of 2 large numbers along with modular *arithmetic* [the original contention of this thread] to estimate a correct answer I find it pretty interesting that the model has that capability among others, without explicit access to an ALU

100

alias (in chains) @antiali.as · 01/10/2026

the machine solves the arithmetic problems

100

alias (in chains) @antiali.as · 01/10/2026

sometimes I can't tell an elm from a beech

100

alias (in chains) @antiali.as · 01/10/2026

who said anything about cognition?

100

alias (in chains) @antiali.as · 01/10/2026

this again denies the possibility of a computer program as an abstraction over a vector of electronic signals

100

alias (in chains) @antiali.as · 01/10/2026

unlike the aether, generalization is actually a measurable property of the latent space but you would have to go poking vectors to know it

100

alias (in chains) @antiali.as · 01/10/2026

you're being absurd and metaphysical now, this deletes the possibility of anything emerging from a chunk of DNA, or the usefulness of any computer program at all

100

alias (in chains) @antiali.as · 01/10/2026

yes and these spatial embeddings happen to support synonymy, analogy, and cross-language translation the hypothesis that this geometry generalizes to meaning is called "semantics"

100

alias (in chains) @antiali.as · 01/10/2026

what’s your pref term for a vector space constructed such that the projected positions of words into that space isn’t random but covaries with the occurrence of those words in relation to one another from a plain text corpus?

100

alias (in chains) @antiali.as · 01/10/2026

When you say “distance” are you referring to a vector in the token embedding space or the positional encoding of the tokens in context?

100

alias (in chains) @antiali.as · 01/10/2026

I’m sorry I still don’t follow. Can you help me understand: at what point in the process does the smuggling occur, and what form do the priors take?

100

alias (in chains) @antiali.as · 01/10/2026

Having a “prior for everything” sounds like a means of understanding

100

alias (in chains) @antiali.as · 01/10/2026

Where do the attention weights come from?

100

alias (in chains) @antiali.as · 01/10/2026

Are you familiar with how token embeddings quantify semantic relationships in high dimensional spaces, or how the attention head contextualizes those embeddings during inference?

100

alias (in chains) @antiali.as · 01/10/2026

You may need to be more precise with the word “read” since that’s a common term in computing when accessing data. Do you mean you think it isn’t accessing the semantic meaning of the text it reads?

100

alias (in chains) @antiali.as · 01/10/2026

What do you mean by “blind to the content?”

100

alias (in chains) @antiali.as · 01/10/2026

But it didn’t output any text in accordance with Benford, its answer did not start with the most likely digit 1 instead it gave the mathematically correct answer

010

alias (in chains) @antiali.as · 01/10/2026

it also only covers the leading digit of a number, saying it is most likely to be 1, but you can see our answer started with a 3

110

alias (in chains) @antiali.as · 01/10/2026

gotcha that’s kinda what I expected to be causing the confusion here. benfords is not a universal property of all numbers, and the article has a section for “Criteria for distributions expected and not expected to obey Benford's law”

210

alias (in chains) @antiali.as · 01/10/2026

Appreciate the link Boxo, can you also help me understand whether this is true, and if so why? > the more significant digits would appear more often and are generated with high probability

100

alias (in chains) @antiali.as · 01/10/2026

you sure you’re not remembering that story about Jeremy Piven?

010

alias (in chains) @antiali.as · 01/10/2026

the APL-like loop as a combinator, fits my brain better than the Haskell-like loop as a morphism, but most working programmers eat on a for(;;) loop

020

alias (in chains) @antiali.as · 01/10/2026

it kinda is no lie, like also your programming language is a category, but it’s so much work to even prove that you can use that to just count and add, what the fuck is the point hahah

050

alias (in chains) @antiali.as · 01/10/2026

then stuff like the tensors and graphs you need to do machine learning also become “categories” in a certain sense and follow rules called “functors” that let you transform them in sometimes useful ways

040

alias (in chains) @antiali.as · 01/10/2026

it lets you take all those weird math propositions, along with the regular math you already know, into a form that lets you say “these things are all really sort of like this other thing, under certain constraints.“ Eg your coffee cup and donut are “topologically isomorphic” as they say

2100