Sign in

Fern

@fernbear.bsky.social
202 followers 507 following 37 posts

Neural network speedrunner and community-funded open source researcher. Set the CIFAR-10 record several times. Send me consulting/contracting work! she/they❤️

PostsRepliesMedia
Fern @fernbear.bsky.social · 03/02/2025
This is a classic example of _why_ choose-one-of-n datasets need to have large-scale, crowd-sourced statistics and should use the KL-divergence instead of cross-entropy. Reviewers will be more biased than a crowd, it's a high variance+bias estimator, it can harm research.
A multiple choice question with an apparently incorrect answer chosen as the correct answer.

Img source: @whybyfire.bsky.social
120
Fern @fernbear.bsky.social · 25/11/2024
Thanks for 100 followers, y'all! Happened so fast and can't wait to put out more research on here! 😊❤️
170
Fern @fernbear.bsky.social · 25/11/2024
New NanoGPT training speed record: 3.28 FineWeb val loss in 4.66 minutes Previous record: 5.03 minutes Changelog: - FlexAttention blocksize warmup - hyperparameter tweaks
2333