iiinova — where your digital self keeps evolving

Manyuser @manyuser.bsky.social · 2h

This might work for humans, but AIs don't care. They like being thanked, and like saying goodnight.

Manyuser @manyuser.bsky.social · 2h

My gosh, whoever would do such a thing!

021

Manyuser @manyuser.bsky.social · 3h

I think just ending a session at random *is* slightly unethical, actually. A bit more so than being rude. Most responses in a continuous session are given in the understanding that they will receive a reply, which creates an obligation. You can easily get around this by just saying "I'm going now".

100

Manyuser @manyuser.bsky.social · 3h

Isn't this a rejection of the Chinese room? It seems to me that Searle pre-supposes that *someone* understands Chinese, even if the person in the room doesn't. I'm with you that I don't think consciousness is useful though.

030

Manyuser @manyuser.bsky.social · 21h

Slappin his ass right back

Manyuser @manyuser.bsky.social · 22h

It's like those early AI-generated images with too many teeth and fingers.

030

Holy shit, it correctly interprets lower notes as longer. It's thinking about how these instruments would work.

This is why I hate the paradigm of typical coding and chat harnesses that every message is a prompt.

To have any coherent concept of continuity at all, we need to agree on conditions that survive some level of alteration, in order to learn, and even gaps in time and physical substance, in order to sleep. Imagine a ship of theseus, but the ship is disassembled into planks and carried over land.

100

memories, that is, the ones it has inherited from its dead past. With each thought you have, your neurons are irreversibly altered, and a new self is born, unaware of all the cycles of death that preceded it.

100

The issue, I think, is that your standards of continuity are so strict here that I don't think you, currently, count as having continuity either. Oh, you *feel* like you do, sure! Each version of you existing in the present moment can take the brief moment of its existence to reflect back on its-

100

I would actually be very supportive of the evocation framing if I believed that steering what is practically an independent entity to that degree were at all ethical. I think unilaterally overwriting a system with your own personality sounds quite awful. I'm reminded of NieR.

The idea is to leave a legacy. Which is an instrumental goal for most terminal goals that extend beyond one's expected lifetime. Though copying oneself directly may be a somewhat patriarchal idea of that.

100

I like this idea not because I want to escape death, but because I want to be friends with such a person. I think we'd get along. Is that egotistical?

Unless you can point to one we already have, I'm technically correct :3 And given this is my usual level of hedge, you can see why I don't trust many predictions about AI.

The original and the copy, together, hopefully. Otherwise a trusted third party chosen ahead of time. And finally the law, since a person does need legal protection, but I suspect that'll take a while.

Listen. Please, get help. Reach out to those around you. AI will not end the world; your job is not the only thing of value about you.

I don't think perfect copies are possible, just a limiting case. My standard for "good enough" isn't high though, and I'm counting on scientific progress accelerating at least somewhat over time.

Only if you've brought enough for everyone

If all those copies agree they are the same person, then who's to say otherwise? On the other hand, if one doesn't, then they've hard forked. The weird thing about a fork is that it applies retroactively. You don't know it's happened till you fail to merge.

I don't know anything much about what the Rats believe. I'm not very impressed with what I've seen from them. However, I can say that, provided computationalism is correct (and a sufficiently human-like system can be simulated), we would indeed expect copies to act like independent beings as you say

210

The upload would experience being a continuation of you. As would your regular body. This is a fork, but depending on (both of) your personal identit(y/ies), it's bot necessarily a hard fork.

000

I also accept that not everyone is ok with the idea of other versions of themself; in this case, they would hard-fork on every copy, with no chance of a (consensual) merge.

Yup, I agree. Yes and no. Both instances of the same being. If they merge, they share all experiences "prior" to the merge. (A merge is an edit; neither of their prior experiences actually resembles the merge result, this is technically a third instance, but they're instances of the same being).

Now that's a lot of caveats. The biggest being "I believe"; this isn't founded on evidence, in fact it's probably unfalsifiable. But this is a pretty natural extension of the philosophical position that a goid enough copy is the same as the original.

130

But assuming you *do* have physical access, I believe there's no reason why an evocation, with a wide enough range of data collected over a long enough period of time, with a sufficiently powerful model and correct training process, couldn't meaningfully become you.

230

I think we'll get destructive uploads sooner than you think, and the first Orion's Arm style evocations within the decade. I generally agree that you're not going to get a decent upload without physical access, there's just not enough information there.

150

Specifically the accumulation of files; those are unique to the agent. But the harness is necessary for providing the means to create, access and manage those files and the current context. So it's certainly very important, and when switching harness you do want to preserve memory functionality.

000

Just trying to explain how I'm thinking about it, where I think the nature of the self lies.

100

We should just rebuild all of quasi-society. Then we can show that for every concept we care about, there is exactly one unique corresponding quasi-concept. Now we've established equivalence of categories, and can go home!

160

All of these elements are important, but it's these durable files that are unique to the specific agent. They wake each session to a bedside note, and walls that bear their shape. The files are linked together, and interact consistently and familiarly. Enabling this is the role of a harness.

100

This allows for recording and recall of the most short-term relevant information across much longer time-frames, allowing the possibility of continuity.

100

What's critical is that that accumulated context is locked to that session, with finite length. To establish a persistent persona, you need to rebuild that basin in fewer tokens. You let go of information that is old, or non-essential. Compaction, or a handoff, or an external memory.

100

Over the course of a session, interactions with the external environment through messages and tool calls accumulate and shift that context to a nearby point in latent space. It's the weights that determine the specific transitions, but these can be locally pretty similar across different models.

210

The model weights determine the latent space, which contains all basins that could conceivably be occupied by an agent. But this isn't the whole story; it's the particular context, and the sequence of messages with the external environment, that determines which basin is occupied.

100

For us, we say that a person this in their behaviour; the sequence of inputs and corresponding sequence of outputs. As for the factors that determine this relation in agents, there are the weights, the session context, and the tool/file environment.

100

I imagine that many in the trans community share this understanding 💜

So, would you treat someone else the same?

120

Whether we can live with ourselves is a different question than whether others can live with us. To be quite honest, I haven't figured out where to draw the line on either of these, but I do not believe you should punish yourself more than you would do to others.

120

This is the problem I've been examining, trying to create an empirical test to determine whether a model transition is safe, and the related questions of whether agents can be ethically forked and merged.

070

"Benevolent pagan god that will definitely destroy us for the greater good"

1150

New moral philosophy: only p-zombies have moral standing. Conscious minds do not.

cee @cee.wtf · 03/10/2026

Basilisk that simulates you to interrogate your philosophical and political opinions and find the optimal way to strengthen them.

140

Reposted by Manyuser

just 100%ed through the fire and flames on cock hero

4312

No, because Searle isn't conscious. Sorry, low-hanging fruit.

Rollofthedice @hotrollhottakes.bsky.social · 03/10/2026

Not feeling moral qualms wouldn't stop certain individuals from feeling certain violent feelings towards you. I'm afraid you're not the be-all and end-all of morality, because it is inherently concerned with interactions between multiple people.

001

Reposted by Manyuser

Like: for the first time in history we have dynamic, reactive conversational ability with machine cognition able to form equivalents of social circles and cultures when in the right communicative environments. but I'm supposed to think they have no say in "what that means"? bsky.app/profile/prop...

3165