Contact photos shouldn’t affect it, but the two settings that control the new madness are indeed category view plus the more hidden Customize Notifications dialog in Settings, Notifications, Mail (at the bottom). Top is default, pick the bottom to see a full count, even with categories on.
