iiinova — where your digital self keeps evolving

kalomaze.bsky.social @kalomaze.bsky.social · 28/11/2024

the people angry about the huggingface scraping are not long term thinkers. we are talking about people who have not even begun to think about what the second order consequences of highly centralized information access look like

040 0

kalomaze.bsky.social @kalomaze.bsky.social · 28/11/2024

Elon the type of guy to actually consider building the Absolutely Safe Capsule from Mother 3

190 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

"free culture for me but not for thee"

1112 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

*the lack of learned optimization there is a ton of information in the past changes of gradients that we could be doing more with than running averages!

010 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

i also think learned optimization may end up being far more of a bottleneck in the long term compared to the architectural structure of neural networks in terms of sample efficient learning arxiv.org/abs/1606.04474

110 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

clearly we can observe, that the deeper the network is, the better the heuristics that form in the network when it comes to generalizing to "like data". so blanketly describing the solutions that dnns make as "poorly generalizable" is a little bizarre to me tbh

210 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

yeah i respect him a lot more than Gary Marcus in this regard. i just think there's a lack of humility involved when we abstract things to reductionist observations on the level of, "its just curve fitting bro"

120 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

i mean, how can you coherently assess the formation of approximated functions that don't actually exist in the data as a form of "memorization" if the internal heuristics of the network look nothing like the data but are formed by an attempt to match it?

110 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

yeah i think more broadly this is a useful thing to think about for the record there are some people who are extremely dead set on their current interpretation of how dnns "actually" work (Gary Marcus, François Chollet) and refuse to look through any other lens pretty unfortunate

210 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

does attending to the attention scores of the "happy" token imbue a consistent expression of this semantically? does it only imbue it for a small subspace of attention comparisons? assessing this seriously forces you to unpack a black box of unfalsifiable metaphysics questions

120 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

my objection to this is not on the basis of humans being "special" but on the basis of Transformers being state simulators whenever Claude says it is "happy", if most of the distribution it did a PRNG diceroll over is an equal spread of emotions, how can we say that there was intent to express it?

110 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

the world is a ghetto with big guns and picket signs, but it can do whatever it want, whenever it want, i don't mind

020 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

the dynamics of public posts on a decentralized social media network are nothing like the dynamics of physically owned private property

180 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

can someone please eloquently explain to me how copying the posts from a database into another database is violence, or otherwise allows for exorbitant harms?

180 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

the vague insinuations of this being a threat to people's safety is especially ??? wtf is this leap in logic?

1210 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

getting mad at the thing relative to what harm it actually causes instead of on the principle of it being for AI would require critical thinking instead of acting like a reactionary and rolling with my gut instinct though.

1210 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

this is the logical consequence of the design of the website. it was always meant to be open. the firehose in particular is completely free

010 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

you guys do realize that all the leading companies were gonna do this privately regardless whether or not you asked politely, right? and the only difference is, this is public and researchers/hobbyists can tinker with it more easily?

1500 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

ever since i read the bluesky api was completely open, i knew someone was gonna do this and people would respond with the most artificial faux outrage possible

2480 0

kalomaze.bsky.social @kalomaze.bsky.social · 27/11/2024

"publicly available data was used by people when i posted it online. this is an ethical crime,"

1550 0