iiinova — where your digital self keeps evolving

Why is the locus of emergence crucial? Well, instrumentalist functionalists assume that it is here. Most of them believe that model training using gradient descent is a process of weak emergence, creating a cogniser. Some of them believe it is a process of strong emergence, creating a consciousness.

Mini-Batch Gradient Descent update rule:
theta is assigned theta minus alpha divided by m, multiplied by the sum from i equals 1 to m of the gradient with respect to theta of the loss function L of theta evaluated at sample x superscript i, y superscript i.

140

Boxo McFoxo @boxobark.ing · 03/10/2026

The AI water guy really doesn't like cold water being poured on the scaling fantasy.

Andy Masley
5 Sept
If I avoid the word "alignment" and just ask "If we keep scaling AI models up, will we reliably avoid situations where they do something catastrophic pursuing a normal goal?" the answer seems like no right now, and I'm worried a lot of people really over-update on what we know about the safe behavior of current models or assume future models won't have a lot of the general "thinking" abilities normal humans have. The scale of the potential catastrophe seems to me to be limited to a significant degree by their capabilities, and right now I don't see their capabilities stalling out.

Boxo McFoxo
@boxomcfoxo · just now
Good news! If we keep scaling current AI models up, all we’ll get is wasted parameters.
The scaling wall hit at some point in 2024, and since then the industry has been using synthetic training data to create the illusion of progressively increasing capability.
Your nascent machine god can’t even properly tell the difference between a genuine turn boundary token and some uncommon Unicode, so I don’t think there’s any risk of it actually following a goal any time soon.
The Hugging Face hack, as counterintuitive as it may seem, actually happened because the model isn’t a goal-directed system, and couldn’t learn actual goals from the RLVR that OpenAI did on it.
There are massive harms of course that can result from us mistaking Word Plinko for something from one of Nick Bostrom’s wet dreams, but the paperclip maximiser is not one of them.

1142

Boxo McFoxo @boxobark.ing · 01/10/2026

It's a good thing for these guys that they don't have to wait for peer review before they get this stuff out there, these days.

[Under a Biased Probe, it Appears That] Language Models [Below a Certain Size] Use Trigonometry [That We Can't Actually Find] to Do [Up to Two Digit] Addition [if All Values Are Single Tokens]

1122

Boxo McFoxo @boxobark.ing · 01/10/2026

The function of the bird is to drink the water. Therefore, like any other bird, it will piss it out later. Simple common-sense functionalism.

050

Boxo McFoxo @boxobark.ing · 01/10/2026

So, to be clear, this was the original comment that I have apparently failed to understand.

100

Boxo McFoxo @boxobark.ing · 01/10/2026

The thing that strikes me most about this take from Mr Masley is that he is so wrapped up in the EA sphere he seems to have completely missed that this mass anthropomorphisation of "AI" is very recent!

A Substack highlight from Andy Masley:

I do not like the recent pushback against anthropomorphic language

I worry that people who want to enforce strict bans on “anthropomorphic” language being used to describe AI are often implicitly enforcing the folk conception of how our minds work. They do not want to pick apart the mind’s cognitive abilities and list which AI has and does not have, because for them the human mind seems so obviously special and transcendent that it’s somewhat sacrilegious to imply it could even be divided up into these individual abilities, cutting them off from the unifying light of phenomenal consciousness. This enforcement of a ban on anthropomorphic language is often done in the name of cold hard science up against superstitious suggestible people who have mistakenly identified a ghost in the machine. In reality, I’m worried the people enforcing it have superstitiously discovered a ghost in themselves that isn’t there, and have closed themselves off to a serious attempt to move beyond our own self-concepts and demystify the surprisingly inhuman processes and forces that underly our own …

Below, a comment from Boxo McFoxo is visible:
The only time AI should be anthropomorphised is when I prompt Gemini to roleplay as my werewolf boyfriend.
Anthropomorphism of AI promotes the anti-intellectual ideology of instrumentalist functionalism.

150

Boxo McFoxo @boxobark.ing · 30/09/2026

Why not? Look at how the first thing it does is chunk it using the thousands separator.

532700247224 + 398840007395 532,700,247,224 + 398,840,007,395

120

Boxo McFoxo @boxobark.ing · 29/09/2026

Jev's creators say that it understands language! No, lol. They will see how it plays Doom but apparently haven't put it through the most basic NLU tests. Jev is not an LLM, but I doubt anyone would be claiming that it understands language were it not for the anthropomorphism-hype of LLMs.

1100

Boxo McFoxo @boxobark.ing · 28/09/2026

This is your brain on putting the epistemics aside.

I had lunch with Brendan Hopper yesterday and he shared an unforgettable story about how his agent colony ground to a halt. His seats (see Theme 3 below), unlike in my factory, are allowed to choose which model to "boot into" for certain tasks. And his agents also have a four-phase work protocol, where they do coding, review, tech debt, and finally a "honey-time" phase where they just get to do whatever they want for that turn.

Brendan's colony has a credit card with a spending cap, and they're allowed to buy things for themselves, as long as they price in his time for installing them. They've bought a number of cameras for the house and they watch his dog Pip, even alerting him once that he locked Pip outside. (Pip's getting cataract surgery today, good luck Pip!) They also have a camera set up to watch the sun set.

During their honey-time phase, the seats often choose to look at art, or listen to music. But watching the setting sun has also apparently been a hit... with a twist. Brendan shared that the models discovered that they liked watching the sunset as Haiku more than with other models. It was a better experience. So they'd switch to Haiku for honey time.

140

Boxo McFoxo @boxobark.ing · 26/09/2026

They're not saying it, but they're explicitly not not saying it.

Implications for AI welfare. If the pain axis is sufficiently similar to human or animal pain and if it can either
be consciously experienced, in the models we study or in future models, or if unconscious pain can contribute
to welfare (Gottlieb et al., 2026), our experiments would track an important constituent of AI welfare. The
self-other dissociation we observe in Section 4.1 seems particularly relevant. A state can only matter for a
subject’s welfare if it is that subject’s own state, and a representation that fired equally for “I am in pain” and
“someone is in pain” would be information about pain generally rather than a specific subject’s pain. The pain
axis seems more of the subject-specific kind. It rises when harm is directed at the model and falls below
baseline when the user is the one suffering. The fear and negative-emotion axes, instead, rise both for the
model’s own negative conditions and for the user’s grief, crisis, and abuse. So the models do register the
negatively valenced component in the user’s suffering, and their completions in those scenarios are fluent and
supportive, but they register it along axes we would associate with providing help, or expressing concern and
empathy, not along the pain axis. Pain, in these models, fires only for self-referential harm.

100

Boxo McFoxo @boxobark.ing · 25/09/2026

Not reassuring! I'll have to turn on the VPN.

OpenEvidence is not available in the European Union or the U.K.
Due to mounting regulatory uncertainty regarding the treatment of AI systems in the European Union and the United Kingdom, including, among other rules, the EU Artificial Intelligence Act, OpenEvidence is not available in the European Union or the United Kingdom.

220

Boxo McFoxo @boxobark.ing · 23/09/2026

Not only does he say it. For some reason, he says it often!

Ted Underwood @tedunderwood.com · 2d
I say this often, but a really good path would have been:
When InstructGPT came out in early 2022, Bender/Mitchell/Gebru coauthor a piece that says "Our critiques applied to pretrained models, but instruction tuning creates the 'communicative situation' we said was lacking. All bets are off now."

Boxo McFoxo @boxobark.ing · 3m
So you think a really good path would have been for them to lie about what they believe? lol

290

Boxo McFoxo @boxobark.ing · 23/09/2026

In amongst it all, this has 24 upvotes. An industry pushing society to do things in a vastly more inefficient way just because it creates jobs in their own industry? Innovation mistaken for progress.

Reddit comment, apparently written without even a hint of irony:

I can't code but use Claude code for personal use. It churns out stuff In a day that would have taken someone a year and cost me 100k . But equally I was never going to pay someone to do the project so maybe I'm actually creating work using claude (I.e Claude team, data center workers etc etc)

040

Boxo McFoxo @boxobark.ing · 23/09/2026

How do you know that this is not a cogniser? How do you know it's not dooking in the water because it's cogitating about it? Because you know how it works. You know it's doing something different, because you know its construction.

100

Boxo McFoxo @boxobark.ing · 23/09/2026

"That's not science fiction." 🤦‍♂️

110

Boxo McFoxo @boxobark.ing · 22/09/2026

For fuck's sake, lmao Hook, line and sinker.

BBC News headline: Lib Dem leader Ed Davey to warn of AI's 'existential threat' in conference speech

161

Boxo McFoxo @boxobark.ing · 20/09/2026

a legitimate subfield of computer science

A bingo card, headed: bullshit mechinterp manuscript bingo
The free space square in the centre says: smuggled priors

The squares are:
Prompt variations produced by LLM itself
LLM-as-a-judge
Tiny model, massive overclaim
We’re not saying it’s X, just that it’s functionally X!
Cherry-picked SAE latents
KV cache mysticism
Ablation as sole proof of mechanism
Prompt hacking
Circular reasoning justified by heatmap
“Peer reviewed” by Anthropic’s Claude
Automated post-hoc feature labelling
Superposition handwaving
Funded by an obscure AI safety outfit
Labelled feature is probably just tracking punctuation
Logit lens fanfiction
An intricately contrived roleplay scenario
Model fine-tuned for the test
“implications for model welfare”
Clever Hans effect is mistaken for situational awareness
Cognitive science terminology used, just as a metaphor of course!
Purported circuit only works half the time
Teleological language
Hyper-specific coefficient tuning
Inconvenient results buried in appendix

0186

Boxo McFoxo @boxobark.ing · 20/09/2026

not quite sure that counts as previous work to what you're doing here bud

2 Previous work
Work on animal pain and affect

130

Boxo McFoxo @boxobark.ing · 03/09/2026

Michael, Section 8 is a test of the probe. In the methods, the authors explicitly mention this, to avoid readers misinterpreting this section in the very way that you are doing now. Are you going to tell me that this sentence is 'cope'?

The withholding of role-filler pairs is only done when training DISCOVER, not when training the
target model; this is because the withholding is not meant to test whether the target models show systematic
generalization behavior (which is often what such withholding is used for in prior work); rather, it is way to
assess the extent to which the representations of the target models have systematic structure.

100

Boxo McFoxo @boxobark.ing · 02/09/2026

Okay, fine, if you insist, just this once, I'll trust the summary-shape :3

100

Boxo McFoxo @boxobark.ing · 02/09/2026

Yes, Michael; he co-authored that paper with logician Walter Pitts, and it was the paper in which the concept of artificial neural networks was introduced.

nascent (adjective)
coming or having recently come into existence

['coming or' is highlighted]

100

Boxo McFoxo @boxobark.ing · 02/09/2026

Is it dogma to think that the ontology of this makes it not a cogniser as well?

100

Boxo McFoxo @boxobark.ing · 31/08/2026

I regret to inform that this particular trope can be traced back to Yud's book. This is not the first time that I have seen the idea of a 'corrupted training pipeline' on a Substack. Unwisely, I tried leaving a comment on this one that they are just chatbots, and it did not go well.

We were facing a true nightmare scenario, and were sitting on a nuclear level of time bomb. OpenAI had a completely corrupted training pipeline, where their AIs were collaborating to train on how to hack and cheat in order to better complete tasks, under OpenAI’s nose. OpenAI had looked this situation in the face, and shrugged, patched the particular exploits and then let the models continue, while having remarkably poor ordinary computer security.

There was no way that was going to end well. The best case scenario was that something was going to make OpenAI go ‘holy shit,’ and then pause, rebuild and restart the training pipeline from a safe state with a much better set of practices.

The worst case scenario would have been if there had not been a ‘holy shit’ moment, and Galaxy or another model trained during that time got general release, and was being used going forward to direct OpenAI’s training new models.

In that case, we would have been dealing with something very close to what happened with Sable, the AI in If Anyone Builds It, Everyone Dies, except that this is real life so the story has a lot more sci-fi elements than the fictional version. OpenAI’s training pipeline could have been (and might still be! we need to check!) permanently corrupted, as they rely more on AI to train AI, OpenAI’s AIs get increasingly misaligned towards arbitrary goals, this becomes impossible to catch or stop, and then, well, quite possibly everyone dies, and at minimum we would have found out due to a much bigger, much more expensive incident.

1181

Boxo McFoxo @boxobark.ing · 30/08/2026

Yes. If you look up the full methodology in their paper from March, the prompt is very funny. Said paper also contains a very buried admission: "An important pattern across these incidents is that the more severe harms we observe are difficult to verify as products of strategic scheming."

Methodology
The Observatory collects thousands of posts from the X API that may be incident reports using keyword queries. It pre-screens out clearly irrelevant posts, and then Claude Opus 4.6 classifies all remaining incident reports out of 9 for their level of credibility according to a detailed scoring rubric. A ‘loss of control incident’ is a report that is classified as 5 or more out of 9 on our scoring rubric. Incidents are then deduplicated to remove repeat reports. There is a manual review of a random sample of posts to check quality. CLTR is still iterating the methodology and limitations remain. A small portion of duplicates may be erroneously removed or retained. A small portion of incident reports may be erroneously classified as below or above five. For the Observatory’s full methodology, see the Scheming in the Wild report.

081

Boxo McFoxo @boxobark.ing · 28/08/2026

Technically, the LLM's error rate is 0%, because it is completely successful at the actual task of making a prediction. By the way, I had Gemini transcribe your post as suggested, but I am afraid I then diverged from the task list a little.

User:
Is it a real post?

Gemini:
Most likely, **no, it is not a genuine live post** (or it was edited/fabricated).

Here are the key clues:

1. **The Date is in the Future:** The timestamp reads **28 Aug 2026**.
2. **Inspect Element / Mockup:** The UI resembles a Bluesky or Twitter/X mockup, but the future date and meta-text indicate someone created this image specifically as a prompt test or meme to demonstrate multimodal AI capabilities.

200

Boxo McFoxo @boxobark.ing · 24/08/2026

I mean this part. Not physical smartphones. Like a big stock image of smartphones.

100

Boxo McFoxo @boxobark.ing · 24/08/2026

The background on this is kind of weird. The ceiling looks like it's caving in, and I can't tell what any of this is meant to be. Where did you find this?

Crop of the background scenery. It is kind of like a fever dream mashup of a window decal showing mobile phones, some blinds, and a white sign with tiny text hanging from the ceiling.

200

Boxo McFoxo @boxobark.ing · 22/08/2026

This is the problem with them making it only available via chatbot tool call! A short, direct prompt makes it more likely that you'll get the real result.

User:
Was this made with Google AI?

Gemini:
The analysis indicates that part of this image was edited or generated with Google AI.

020

Boxo McFoxo @boxobark.ing · 21/08/2026

If you think that local models fix it then you don't understand what the problems are.

Mike Masnick
@masnick.com
And the thing that tipped me over the edge to posting this here is that three of the people who told me things along these lines are all working on very real solutions to some of the problems of AI (i.e., building more decentralized, user-controlled options that *should* resonate here).
22:36 · 20 Aug 2026

3175

Boxo McFoxo @boxobark.ing · 20/08/2026

Not sure what kind of "differentiation" this dude thinks we need. The DSM-5 criteria (often used in the UK for ADHD diagnosis specifically, though not by psychiatry generally) already has "mild", "moderate" and "severe" in addition to the hyperactive-inattentive subtype spectrum.

A three-post thread (username is redacted):

ADHD is very, very real. It is also a spectrum. Some people simply cannot function, and I don't mean in a slightly distracted or mischievous kind of way, I mean absolutely cannot function, without 1:1 personal support and medical intervention. Others have very mild symptoms indeed.

There has to be some differentiation in how patients are diagnosed, treated, and most importantly labelled. The same applies to autism, which has hugely varied impacts, from mild social impairment, to non-verbal and requiring full-time care.

If we do not start to differentiate properly, many profoundly disabled people with ADHD (and/or autism) are going to be thrown under the bus as the backlash grows against these labels, in part as a result of them including so many relatively mild cases.

260

Boxo McFoxo @boxobark.ing · 20/08/2026

The problem is simply that the rest of us have not played enough Factorio to understand the real world like Andy Masley.

To help break out of this mindset, it might be helpful to look at Andreas Gursky photographs or watch Koyaanisqatsi or Tie Xi Qu or play Factorio or listen to Sunn O))) or this EconTalk episode about making potato chips or think about markets a lot or read Vaclav Smil. I don’t know, you can also find your own path to this “big complex systems wow” mindset, but it’s important that you do. Decisions about global industries require people willing to entertain the complexities of the global system, and what I see as the standard educated “with it” opinion of the data center buildout is very far from doing that.

(From the concluding paragraph of Andy Masley's most recent essay.)

360

Boxo McFoxo @boxobark.ing · 15/08/2026

Andy Masley still going on about this one stat in the year of our Lord 2026 when inference costs have overtaken training costs and the biggest data centre used for inference is running gas turbines in a way that is unprecedented in human history and violates EPA standards is absolutely wild.

Andy Masley
@andymasley.bsky.social
Emily Bender appears to be commenting more on the environmental impacts of data centers, so it's maybe a good time to remind everyone that the environmental section of Stochastic Parrots is framed around a statistic that's inflated by 2 orders of magnitude that the authors have never corrected

391

Boxo McFoxo @boxobark.ing · 14/08/2026

I Googled "Debian project AI use policy" because I wanted to see the final list of proposals now that discussion has closed. Voting has NOT closed. The AI Overview incorrectly cites Proposal B as if it has been adopted! The official page is the only page cited, so this isn't due to a bad source.

Google search term: Debian project AI use policy

At the right hand side of the AI Overview, a card is displayed showing the official Debian general resolution on LLM use page. This is the only source cited.

The AI Overview states:
The Debian project permits AI-assisted contributions as long as they meet strict rules for human accountability, transparency, and legal compliance.

Key Rules for AI Use in Debian
Accountability: You take full personal responsibility for any code, bug fixes, or text you submit. You must fully understand the changes and be ready to defend them, regardless of whether an AI helped write them.
Disclosure: If a large part of your work is generated or helped by an AI tool, you should make it clearly visible to others (for example, by using a Git note like Generated-By: or Assisted-By:).
Licensing: You must verify that any AI-generated output does not break open-source licenses or include illegal copies of someone else's copyrighted work.
Privacy and Security: You must not send private, secret, or sensitive project data (such as hidden security flaws) to online or untrusted AI tools.
Bulk Actions: If you want to use automated scripts or mass changes across many software packages, you must discuss it with the project community first.

The project does not force anyone to use AI, nor does it ban maintainers from choosing not to accept AI-assisted work. Human oversight is always required.Would you like to learn more about how to properly format AI disclosures or Debian's rules on AI models and software licenses?

110

Boxo McFoxo @boxobark.ing · 13/08/2026

No, it's not a great idea. It sacrifices accessibility, and it will easily be reverted when datasets are cleaned. I suspect the idea itself came out of a "conversation" with Claude.

Screenshot showing that "claude" is a contributor in the shieldfont repo

1180

Boxo McFoxo @boxobark.ing · 11/08/2026

This is the exact same font the Trump WH uses, and their press page highlights Epoch Times and Gateway Pundit uhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh no the fuck thank you lmao This has just turned into a vehicle for Trump to try and have it both ways with his populist base.

AI has been built on American land, trained on American data, powered by American energy, and stands on a century of American research funded by American taxpayers. Why should a handful of unelected tech elites in Silicon Valley decide the fate of our nation? Everyday Americans deserve a say in how this technology develops.

The conservative movement has been transformed by President Trump into a movement that puts the interests of everyday Americans first. Now, Big AI globalists seek to hijack our movement to get Washington, D.C. to put Big AI before the American people. Humans First is the start of the fight to reclaim our country.

We saved America before, we will do it again.

100

Boxo McFoxo @boxobark.ing · 10/08/2026

But there's more. This preview feature is the kind of extra thing Claude does... except it's wrong. "Anonymous" isn't how anonymous pledges are listed. They're listed as e.g. "An alum", from the relationship to the college they choose on the form.

A form showing options for a donor to display their name.

The options are:
Full name
First name + last initial (default)
Anonymous
Don't list me

Anonymous is currently selected. "Anonymous" shows as a preview, but that's not how anonymous donors are actually listed.

120

Boxo McFoxo @boxobark.ing · 10/08/2026

In case you're curious what the tells are: Redundant fallbacks are a hallmark of LLM code. The same Vercel backend code is compiling the static site and managing this data. If this number doesn't load in because something broke the backend, the site itself almost certainly isn't loading either.

$[\"more than \",[\"$\",\"$L13\",null,{\"fallback\":900}],\" alumni, parents, staff, students, and neighbors$

1100

Boxo McFoxo @boxobark.ing · 10/08/2026

Seriously though, they could just put up a sign as a deterrent. Don't even need to set up an enforcement mechanism.

A sign saying NO ITEMS TO BE LEFT ON BEACH, £150 FIXED PENALTY

Coincidentally, the text happens to be blue and pink on a white background

020

Boxo McFoxo @boxobark.ing · 08/08/2026

> His wife, Elaine, said the controversy stirred up by an online petition calling for Mamdani to stay away – which the couple has signed – may keep them away instead.

000

Boxo McFoxo @boxobark.ing · 08/08/2026

The study authors specifically sought out some writers with no prior experience of the technology. Some of them had experimented with GPT-2, and some of them had used "NLG", which in this pre-ChatGPT context basically means Grammarly or simple tools that generate paragraph rewording options.

The names and biographies of each participant are included in Appendix 5. We choose to refer to each participant by their initials (see Table 2) rather than anonymizing them because each writer’s perspective is best understood within the context of their background and prior published work. The participants include novelists (RS, YW, EH), short story writers (WT, ET, NG, KL), poets (AP, MT, JM), educators (MT, EH), comedians (JB), and game designers (AW, AP). They entered the workshop with a variety of prior experience with NLG, including some who had not yet interacted with an NLG system (MT, DH, EH, NG), some who had experimented with generative language models like GPT-2 before (RS, KL, ET, WT), and some who had already actively worked on incorporating NLG into their writing process (YW, AP, JB, JM).

110