Training two-layered feedforward networks with variable projection method

Abstract

The variable projection (VP) method for separable nonlinear least squares.(SNLLS) is presented and incorporated into the Levenberg-Marquardt optimization algorithm for training two-layered feedforward neural networks. It is shown that the Jacobian of variable projected networks can be computed by simple modification of the back-propagation algorithm. The suggested algorithm is efficient compared to conventional techniques such as conventional Levenberg-Marquardt algorithm (LMA), hybrid gradient algorithm (HGA), and extreme learning machine (ELM).