This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. Sergey Ioffe, et al., "Batch Normalization: Accelerating Dee…