iiinova — where your digital self keeps evolving

That's a fair criticism. I am indeed a person with hammer arguing it's nails all the way down, and you should keep that in mind.

Not sure what you mean by blowing it out to Chomskyan dimensions. My use of "syntax" in a broad sense here is very non-Chomskyan, and more directly inspired by formal language theory. My claim in other words is LLMs have abstracted a formal grammar the structure of entire texts.

As the corpora and context windows get larger and larger and the models more and more sophisticated, it's possible they starts picking up on patterns we would never notice in ourselves. What that will do in a reinforcing loop, who knows.

000

In a way I'm shifting the framing here: the remarkable part to me is how patterned our behaviour has to be for the LLMs to pick up this color stuff off usage alone! Somehow the way we talk about colors is patterned along semantic lines enough, that the the pattern machine picks up on these lines.

Yes, my broad notion of "syntax" must extend to very subtle distributional corpus semantics compared across lexical items. It's really impressive, but I still think it's a very complex syntax under the hood.

220

This why there will never be a true LLM-based solution to the problem of "hallucinations". The machine can only generate text in the style of an answer, and sometimes the style is so constrained that it might coincide with truth, but most often it cannot and won't.

020

Sometimes, some true statements are repeated enough in a corpus that an LLM can regurgitate "the Eiffel tower is in Paris", or "all squares are rectangles, but not all rectangles are squares" and it's easy to confuse this type of text with the LLM making statements of truth.

I could be painting it right now I could have turned around and changed the truth value while writing this paragraph. The truth of this sentence cannot be inferred from any corpus ever. It never will be.

It does not matter how often any of these words or their combinations appear in any corpus. The truth depends on the direction I'm facing and the paint on the wall, which could have been there for a day or for a century.

There is no relationship between the syntax of the "the wall to my left is white" and whether or not that statement is true. It does not matter how confident your LLM (or you, for that matter) is that "white" is a better choice than "yellow" in the sentence "the wall to my left is _".

Truth is not, and can never be a syntactic fact. Truth depends on aspects of the world that we do not know yet, or that change over time. The future cannot be predicted from the past, and so it's definitely not going to be predicted from past corpora alone.

And one crucial limitation, one inherently non-syntactic feature of text that -it seems to me- always will be inaccessible to LLMs is concern for truth.

In fact my strong take is: LLMs are syntax machines and *nothing else* than syntax machines. Nothing outside of the purvue of parsing and manipulating the structure of texts is within the realm of the possible for these machines.

230

I still don't think we're anywhere close to AGI. These text machines still just do text manipulation. But they *will* revolutionize text manipulation. Coding, math, surveillance, propaganda, and summaries for now. People will find more uses, some good, some evil.

So how do LLMs do approximately good math (without calling widgets) outside of its training? My answer is syntactically, by approximating the high level structure of a multiplication at some level.

160

Meaningwise, LLMs push distributional semantics to its limits. I, and many others, clearly underestimated just how much of a handle on meaning you can get from word distribution alone, but it's also clear that there's more to it too.

150

What I predict is therefore that LLMs will be very good at broadly-syntactic tasks. That's what we seem to see: coding, math, and mundane text processing are the areas where -for better and for worse- LLMs are shaking things up.

This broad flattened syntax lumps together everything we call grammar, conversation, discourse, figures of speech, tone, style, topic, genre, and more. Every bit of structure we, as users of language, have imposed upon the texts we've created, has been picked up by LLMs as syntax.

My "theory of LLMs" at the moment is they are universal syntax approximation machines, where "syntax" is taken in the broadest sense of all distributional statistics among the tokens of a corpus.

As someone who also began in the hard syntactic parrot hypothesis and kept up with the progress, here's my take as a linguist and cognitive scientist, and NLP enthusiast since before the age of LLMs: It's syntax all the way down

2153

Imagine if humanity in the future ends up with professional programmers having to learn 21st centruy English and Chinese to get the really good models working. Sci fi idea, maybe.

020

so then, what happens when language changes in 100 years? 500 years? You can tack "also X means Y now" in the context window for a while, but that will eventually have diminishing returns.

Idk if anyone has written about the interaction of AI and language change, but there's a plausible future where 1) LLMs entrench themselves as integral to society (eg. in coding), 2) Artificial Intelligence doesn't progress past LLMs, and 3) Model collapse makes training better models impossible

Sarah T Roberts, PhD @ubiquity75.xyz · 13/06/2026

Reposted by Max Papillon

The goal isn’t to replace the work of humans. It’s to make a case for its devaluation.

91111296

Reposted by Max Papillon

Sarah Clarke - Not your doll @damselindystopia.northsky.social · 30/09/2026

Oppressor classes are great at solidarity with each other, it turns out.

1516

Reposted by Max Papillon

Sarah Clarke - Not your doll @damselindystopia.northsky.social · 30/09/2026

Anti-feminism is what "an injury to one is an injury to all" looks like among people whose idea of an injury is having to respect a woman saying "no."

09921

Reposted by Max Papillon

shaunt @shaunvids.bsky.social · 30/09/2026

i know a lot of people really, really want gaza to be a two-sides issue that maps cleanly onto US electoral politics, but it isn't. the mainstream of both parties supports the genocide, because the US as an institution supports the genocide. i know that's uncomfortable but you can't run away from it

161742244

People are talking about it as a common sense rule and I'm here thinking what they're admitting to should be criminal. You are admitting you're casually discriminating against employees based on bring coffee? Fired. Banned from authority jobs for life Fuck you.

010

Hi, I'm a hiring manager at a major company, and I would never hire someone who shows up without coffee. Who doesn't have time to make or buy coffee on the way to an interview? A disorganised person, is who. Or someone who doesn't drink coffee and won't work hard. Navigate this one, job seekers.

"Yeah, the work juice we all drink before and during work in workplaces is a sign that you won't do work if you drink it when you're looking for work. Sorry, I don't make the rules. Wait I do. Well, gtfo, iced coffee drinker."

This iced coffee discourse is bonkers. Y'all love making rules up to torture people you have power over, don't you? Coffee? the thing that's normal everywhere? I've brought coffee to every work meeting I've ever had, interview or otherwise, because I drink coffee when I work. Wtf are y'all about?

Like, really, they're not testing out data centers in space per se as of now; what they're working out is super computers in space, which in and of itself may have more purposes they're not revealing

050

Max Papillon @maxpapillon.bsky.social · 25/09/2026

"You know these things we build where water and energy are abundant? Let's build it the furthest place possible from those!" Seriously though, assuming Google hasn't completely lost touch with reality, are they up to something else? Are they working out autonomous AI space weaponry or smthng?

1121

That does seem like a lot to assume about what she means compared to it just a spelling issue, possibly just autocorrect.

Max Papillon @maxpapillon.bsky.social · 25/09/2026

Is it really an eggcorn? Accidentally egularizing a brand's nonstandard spelling is common and probably its own thing. E.g. fruit loops < froot loops, sketchers < skechers, febreeze < febreze are all common. Coolaid<koolaid fits the same pattern.

Max Papillon @maxpapillon.bsky.social · 24/09/2026

Reinventing doge coins from first principles

010

Max Papillon @maxpapillon.bsky.social · 24/09/2026

It's like "long-term" and "business model" are mutually exclusive. Companies with a business model are seeking immediate return and eating themselves alive; companies banking on long term growth are have no business model to show for it.

010

Reposted by Max Papillon

Chloe "Scare-Bot Scramble" Adams @addamschloe.bsky.social · 24/09/2026

"[i want you to] enjoy the [concentration/ death] camps" = explicitly declared by head of T&S to be acceptable speech "i need him to [get eaten by piranhas]" = explicitly declared by moderator action to be unacceptable violent speech WHAT could the distinction be here

517965

Reposted by Max Papillon

Max Papillon @maxpapillon.bsky.social · 15/09/2026

My simple take on the electoral discourse regarding trans right:

031

Max Papillon @maxpapillon.bsky.social · 22/09/2026

My bleak prediction for the future of AI & math is: AI proofs will come first and mathematicians' main job will be to make sense of these proofs. Sort of how pro chess nowadays is making sense of what the computer says and gleaning at its impenenetrable logic. Human mind downstream of the machine.

030

Reposted by Max Papillon

Julia Serano @juliaserano.bsky.social · 20/09/2026

...we are *way* beyond sports – we now have Dem Senate nominees calling for BANNING TRANS PPL FROM PUBLIC RESTROOMS! if you're a cisgender Democrat, what are you going to do about it? are you willing to push back against this? or will you be an accomplice to this? bsky.app/profile/erin...

528289

Reposted by Max Papillon

Yes literally exactly this. Spaghettifying much more rapidly than we can rationalise and consolidate things.

0286

Reposted by Max Papillon

I'm going to be clearer about what I'm saying here. AI doesn't have to beat us on genuine intelligence to cause humanity unending problems. It just has to look vaguely competent, and be able to beat humanity on sheer volume of output. There are 8 billion people.

2253

Reposted by Max Papillon

I sincerely doubt that we will see such a "genuine" artificial intelligence ever created, but I think the more pressing issue is that even without it, the dehumanisation of machine processes is rapidly escalating for the sake of some profit driven interests.

1273

Reposted by Max Papillon

We're running up against the limits of an anthropocentric concept of "intelligence" when we say today's robots do not have "genuine" intelligence but tomorrow's robots might. A robot isn't a human and it doesn't have our interests beyond those a human programmed it with. That's always been the case.

1324

Reposted by Max Papillon