arXiv 1502.03167
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
By Sergey Ioffe and Christian Szegedy
Published 2015-02-11
Citation lineage
Review the prior work and downstream research connected to this paper.
Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and add…