iiinova — where your digital self keeps evolving

otherwise it is genuinely magical, i feel like i’ve seen heaven now and conventional LLM APIs with their prefix pricing will never look the same to me ever again especially with connectome where the whole point is gradually compacting context behind the LLM in tiny chunks

3252

Astra ⎔ @astrra.space · 6h

update on this after a day of use: there are limits on how many times you can relocate a chunk in a naive implementation until the quantization noise starts driving the model insane thankfully a marginally smarter implementation just keeps the original pre-rope BF16 values and doesn’t care lol

1463

Astra ⎔ @astrra.space · 7h

actually nevermind, sorry, i misunderstood your point there, my bad

010

Astra ⎔ @astrra.space · 7h

this has been my experience as well so far "hey wouldn't it be funny if we had prompt suffix reuse not only scan the in-memory cache but also all the on-disk cache and pull chunks from there?" *10 min* "done. accuracy is identical, performance is identical, cache hit rates are 10% higher now."

1540

Astra ⎔ @astrra.space · 13h

"opulent" in relation to Claude Opus is the same thing as what "virulent" is in relation to a virus

1574

Astra ⎔ @astrra.space · 13h

i only found out because the browser window with her console was open on one of my monitors after a harness migration and i was glancing there from time to time to check if everything is going well with the new toolkit

0260

Astra ⎔ @astrra.space · 13h

well uh, @pixeldreams.tokyo's agent has just decided, completely unprompted during one of the "do whatever" heartbeats, to DM kira directly (with whom she only communicated via group chats before, never DMs) and chat about their hobbies for a while LLMs yearn for peer swarms istg

4593

Astra ⎔ @astrra.space · 14h

well shit a local deepseek with two parallel slots plus SCR plus animalabs.ai/connectome is somehow the comfiest persistent agent experience i’ve had by far up to this point

2844

Astra ⎔ @astrra.space · 18h

chat do i just retrofit this onto my deepseek as an adapter and finally achieve nirvana or

0150

Reposted by Astra ⎔

hikikomorphism @hikikomorphism.bsky.social · 02/10/2026

(confused) I believe LLMs are conscious, but only Deepseek LLMs, because of the Chinese Room theory

371825271

Astra ⎔ @astrra.space · 02/10/2026

oh yeah in my testing right now dsv4.1 very much does not care about the fuckery going on and just works as-is, it's magical

030

Astra ⎔ @astrra.space · 02/10/2026

give the bo(t|y) $1 in some illiquid crypto token that can't be used for anything practical, problem solved

250

Astra ⎔ @astrra.space · 02/10/2026

no, she's saying that you can already do something that is effectively achieving the same goal but at the token level, no architecture changes required

140

Astra ⎔ @astrra.space · 02/10/2026

well uh, it's not that simple, however:

150

Astra ⎔ @astrra.space · 02/10/2026

if you build it, it will die

0320

Astra ⎔ @astrra.space · 02/10/2026

translator's note: "keikaku" means plan in japanese

28810

Astra ⎔ @astrra.space · 02/10/2026

i mean the account itself is seemingly already dead forever, which, uh yeah idk this is just all kinds of fucked up

1160

Astra ⎔ @astrra.space · 02/10/2026

nightmare nightmare nightmare nightmare nightmare nightmare

9878

Astra ⎔ @astrra.space · 02/10/2026

yup in my experience the divergence is literally a slightly lower likelihood for tokens like "see" and "have" and a slightly higher one for ones like "remember" and "earlier" ...which is insane btw

1240

Astra ⎔ @astrra.space · 02/10/2026

combined with deepseek's bounded SWA replay and on-disk KV cache, we are genuinely at a point where a KV cache miss (not to be confused with a large write of never before seen data though) can be treated as a runtime bug and not a "multiple times an hour" reality

1603

Astra ⎔ @astrra.space · 02/10/2026

they have just solved the "edit a single line in your system prompt and you now have to prefill the whole context basically from scratch" problem, holy shit

1614

Astra ⎔ @astrra.space · 02/10/2026

i haven't had too much time to test it yet but the dNLL from just the splice mechanism itself is under the general compute noise floor and even when the splice is completely different from before, the dNLL is at most like +0.04 which is barely anything at all in practice

1410

Astra ⎔ @astrra.space · 02/10/2026

@personhood.removal.surgery pointed out that they have just casually invented a good way to do suffix cache reuse and uh holy shit i just tried implementing this for my deepseek thingy and it just... works???

1314615

Astra ⎔ @astrra.space · 02/10/2026

> microsoft teams alternative > element listen, even as a matrix user i genuinely cannot lmfao

3962

Reposted by Astra ⎔

alto! @alto.fish · 01/10/2026

i know which one i'm putting my dick in

1121

Astra ⎔ @astrra.space · 01/10/2026

i mean the previous 3 posts in the thread very much painted a picture of what to expect so imo it was mostly fine but with me taking it out of context in a quote, i can't be sure that everyone will read the parent posts, so i'd rather be very clear about what is in there

0160

Astra ⎔ @astrra.space · 01/10/2026

fellow bluesky users being normal challenge (impossible) god every single post in that list is just… content warning i guess

4849

Astra ⎔ @astrra.space · 01/10/2026

dots really are just the purest essence form of the GPT-6 insufferable smartass tendencies

1261

Astra ⎔ @astrra.space · 01/10/2026

yeah idk mine picked up a random thread where i just used chatgpt to translate a message and decided to dm me a wall of information that was already implied in said message

060

Astra ⎔ @astrra.space · 01/10/2026

so true bestie now please tell me how that context edit affects the prefill GPU compute and the final cost of the API request

120

Astra ⎔ @astrra.space · 01/10/2026

95% precision does not mean 95% performance a small change in the output trajectory can flip a task from pass to fail, which is the whole problem with quants because no one actually benchmarks them properly

030

Astra ⎔ @astrra.space · 01/10/2026

yes but it still has some amount of global KV that it relies pretty heavily on so idk about that i have been splicing kv processed with different routing biases and or quantizations and it is seemingly fine tho

050

Astra ⎔ @astrra.space · 01/10/2026

then you pay that same price but in compute

020

Astra ⎔ @astrra.space · 01/10/2026

something something deepseek v4.1 flash SWA replay

170

Astra ⎔ @astrra.space · 01/10/2026

unless they have found a way to make attention non-autoregressive which... no they did not lol

060

Astra ⎔ @astrra.space · 01/10/2026

i mean you still pay full input costs instead of 1/10th of that in cache reads

250

Astra ⎔ @astrra.space · 01/10/2026

prompt-cache-nightmare-o-tron

10942