Reading about scaling laws recently I came by the interesting point:
Focus on a balance between models' size and performance is more important that aiming for larger models
Tsinghua University and ModelBest Inc propose the idea of “capacity density” to measure how efficiently a model uses its size
