A three-term backpropagation algorithm

The standard backpropagation algorithm for training artificial neural networks utilizes two terms, a learning rate and a momentum factor. The major limitations of this algorithm are the existence of temporary, local minima resulting from the saturation behaviour of the activation function, and the slow rates of convergence. In this paper, the addition of an extra term, a proportional factor, is proposed in order to speed-up the weight adjusting process. This new three-term backpropagation algorithm is tested on three example problems and the convergence behaviour of the three-term and the standard two-term backpropagation algorithm are compared. The results show that the proposed algorithm generally out-performs the conventional algorithm in terms of convergence speed and the ability to escape from local minima.