I use normal distribution order statistics to replace batch normalizations in binary neural networks. I show that this avoids some BatchNorm problems, with little cost.
The FormulasForRenorm file contains the full explanation of the method, with a demonstration on a CNN classifier. The second notebook is there to demonstrate the beneficial effects of this method on an autoencoder.