arXiv 1803.08494
Group Normalization
By Yuxin Wu and Kaiming He
Published 2018-03-22
Citation lineage
Review the prior work and downstream research connected to this paper.
Batch Normalization (BN) is a milestone technique in the development of deep learning, enabling various networks to train. However, normalizing along the batch dimension introduces problems --- BN's error increases rapidly when the batch size becomes smaller, caused by inaccurate batch statistics estimation. This limits BN's usage for training larger models and transferring features to computer vision tasks includin…