A three-term backpropagation algorithm

Research output: Contribution to journalArticlepeer-review

113 Scopus citations

Abstract

The standard backpropagation algorithm for training artificial neural networks utilizes two terms, a learning rate and a momentum factor. The major limitations of this algorithm are the existence of temporary, local minima resulting from the saturation behaviour of the activation function, and the slow rates of convergence. In this paper, the addition of an extra term, a proportional factor, is proposed in order to speed-up the weight adjusting process. This new three-term backpropagation algorithm is tested on three example problems and the convergence behaviour of the three-term and the standard two-term backpropagation algorithm are compared. The results show that the proposed algorithm generally out-performs the conventional algorithm in terms of convergence speed and the ability to escape from local minima.

Original languageBritish English
Pages (from-to)305-318
Number of pages14
JournalNeurocomputing
Volume50
DOIs
StatePublished - Jan 2003

Keywords

  • Backpropagation
  • Learning algorithms
  • Multi-layer neural networks
  • Three-term algorithm

Fingerprint

Dive into the research topics of 'A three-term backpropagation algorithm'. Together they form a unique fingerprint.

Cite this