Reposted by Rupesh Srivastava
*Weighted Skip Connections are Not Harmful for Deep Nets*
by @rupspace.bsky.social
Cool blog post "in defense" of weighted variants of ResNets (aka HighwayNets) - as a follow up to a previous post by @giffmana.ai.
rupeshks.cc/blog/skip.html
rupeshks.cc
Weighted Skip Connections are Not Harmful for Deep Nets
Give Gates a Chance