Calculate weight gradients and update weights in a forward fashion such that the when updating layer X all the layers before it will not change further, hence not making the update obsolete
This solution aims to completly resolve the Internal Covariate Shift problem of Deep Artificial Neural Networks.