An improved version of backpropagation algorithm with effective dynamic learning rate and momentum

Abstract

An improvement of backpropagation algorithm with momentum is introduced. Local quadratic approximation of the error function is performed at every stage of the learning process and the Hessian matrix of the quadratic error function is approximated [1]. Efficient learning rate and momentum factor is determined at every stage of the learning process by means of maximum and minimum eigenvalues of the Hessian matrix. The effective performance of this new approach is demonstrated on three examples.