Fast learning method for back-propagation neural network by evolutionary adaptation of learning rates

Abstract

In training a back-propagation neural network, the learning speed of the network is greatly affected by its learning rate. None, however, has offered a deterministic method for selecting the optimal learning rate. Some researchers have tried to find the sub-optimal learning rates using various techniques at each training step. This paper proposes a new method for selecting the sub-optimal learning rates by an evolutionary adaptation of learning rates for each layer at every training step. Simulation results show that the learning speed achieved by our method is superior to that of other adaptive selection methods.