halvarflake.bsky.social @halvarflake.bsky.social · 26/06/2026I made a small animation illustrating my geometric intuition for what happens inside a simple feedforward ReLU neural network. Hope this is illuminating for someone. 0121
halvarflake.bsky.social @halvarflake.bsky.social · 10/06/2026The AdamW videos compress to 1/6th the size of the Muon videos. Something AdamW is doing allows the crease visualisation to be compressed well, but not Muon. This is the weirdest observation ever. 150
halvarflake.bsky.social @halvarflake.bsky.social · 05/10/2025Ok, convergence with SGD was too painfully slow, so I did a few thousand iterations with Adam, then switched to SGD. The result looks like this: 131
halvarflake.bsky.social @halvarflake.bsky.social · 05/10/2025Current state is this. It's early in the training process, though - I chose SGD instead of ADAM (which becomes unstable when trained to very low loss), and the weight initialization which led to many creases in the upper left corner is still visible. 110
halvarflake.bsky.social @halvarflake.bsky.social · 05/10/2025Today's experiment: What shape will a deep neural network learn if trained on the sparse set of training points on the left? Will it learn a circle? 170
halvarflake.bsky.social @halvarflake.bsky.social · 23/09/2025This is kinda wild, but kinda good? 4120
halvarflake.bsky.social @halvarflake.bsky.social · 17/08/2025Life's good. Got a surf in at 7 am, harvested two hands full of wild blackberries on the walk back, daughter taking swimming lesson by the beach now & I am watching the crowded peak while drinking a decaf (giving up caffeine has been great for me, at least for now). 2230
halvarflake.bsky.social @halvarflake.bsky.social · 01/03/2025Going to sleep while my computer wastes cycles on PyTorch. But I'm visualizing the polytope boundaries nicely after each training step. Right hand side is the target picture, red pixels are training set, blue pixels validation set (to be used later). 130
halvarflake.bsky.social @halvarflake.bsky.social · 21/01/2025Hilarious. German Satire magazine titled "controversial gesture: Is Hitler doing a Musk salute?" 05811
halvarflake.bsky.social @halvarflake.bsky.social · 17/01/2025Just like it's creator, ChatGPT has a rather funny ideas about what is a copyright violation and what is not. 2100
halvarflake.bsky.social @halvarflake.bsky.social · 21/12/2024I think my math hunch from early 2023 will age pretty well. 160
halvarflake.bsky.social @halvarflake.bsky.social · 13/12/2024I frequently wonder why I insisted on Google paying me in cash instead of stock in 2011, and then I remembered: The total real return of the SP500 between March 2000 (when I turned 18) and February 2011 (when we sold zynamics) was negative, and it would take until June 2013 before it was positive. 0121
halvarflake.bsky.social @halvarflake.bsky.social · 04/12/2024This table is wild. 200 32 bit multiplications for the price of a 32 bit DRAM read? And this is 45nm, what about modern process nodes? 181
halvarflake.bsky.social @halvarflake.bsky.social · 16/11/2024Post a pic YOU took (no description) to bring some Zen to the timeline. 4121
halvarflake.bsky.social @halvarflake.bsky.social · 14/11/2024Kinda wild that the code for this "AI coding assistant" that creates GitHub pull requests according to changes I request was almost entirely written by me prompting ChatGPT. About 700 lines of code, and really just two evenings performing iterated prompting & some cut/paste & some minor edits. 181
halvarflake.bsky.social @halvarflake.bsky.social · 08/10/2023I am Kung Fu panda, I eat when I am sad. 0120
halvarflake.bsky.social @halvarflake.bsky.social · 10/05/2023Sons 4th birthday, strawberry/blueberry swirl cheesecake. 160
halvarflake.bsky.social @halvarflake.bsky.social · 07/05/2023That said, I do like eating out, too; here's pics of food I didn't make myself: 010
halvarflake.bsky.social @halvarflake.bsky.social · 07/05/2023Rye sourdough bread + dulce de lèche made from boiling condensed milk cans. 120
halvarflake.bsky.social @halvarflake.bsky.social · 07/05/2023Roast fennel w kumquat, kiwi & pine nuts, teriyaki mackerel, goat cheese chorizo pizza, goat cheese honey walnut canapees 130
halvarflake.bsky.social @halvarflake.bsky.social · 07/05/2023Bavette & ratatouille, burger, blueberry pancakes, lamb shoulder w rosemary sprigs 120
halvarflake.bsky.social @halvarflake.bsky.social · 07/05/2023Ok, Amateur/homemade thread: Asparagus mimosa, baked pear+roquefort, spinach/red beet/mozzarella salad, peach merengue dulce de lèche pie. 121
halvarflake.bsky.social @halvarflake.bsky.social · 05/05/2023Ok, what's the culture here? Ppl posting selfies? 060
