Very excited by our first benchmarking results on BMD by the talented @sargechris.bsky.social! By including control models trained on Kinetics that deal trivially with time (i.e. treating videos as image sequences), we find better brain alignment scores for actual video models in early visual areas!
@sargechris.bsky.social📢 New preprint!
We benchmark 99 image and video models 🤖 on brain representational alignment to fMRI data of 10 humans 🧠 watching videos!
Here’s a quick breakdown:🧵⬇️
www.biorxiv.org/content/10.1...
biorxiv.org
One Hundred Neural Networks and Brains Watching Videos: Lessons from Alignment
What can we learn from comparing video models to human brains, arguably the most efficient and effective video processing systems in existence? Our work takes a step towards answering this question by...