Robust design of multilayer feedforward neural networks with applications to process monitoring data analysis

Abstract

This thesis is considered with the robust design of multilayer feedforward neural networks trained by backpropagation algorithm (Backpropagation net, BPN) and its applications to process monitoring data analysis. BPN``s have been successfully used for solving a wide variety of problems. However, determining a suitable set of structural and learning parameter values for a BPN still remains a difficult task. This research develops a systematic, experimental strategy which emphasizes simultaneous optimization of BPN parameters under various noise conditions. Unlike previous works, the present robust design problem is formulated as a Taguchi``s dynamic parameter design problem, together with a fine-tuning of the BPN output when necessary. Four design variables (i.e., number of neurons in the first and second hidden layers, learning rate, and momentum) and three noise variables (i.e., initial set of random weights, ratio of the size of the training set to the size of the testing set, and selection of the trainin and testing sets) are considered simultaneously. A series of computational experiments are also conducted using the data sets from various sources. From the computational results, statistically significant effects of the BPN parameters on the robustness measure (i.e., signal-to-noise ratio) are identified, based upon which an economical experimental strategy is derived. In addition, the step-by-step procedures for implementing the proposed approach are illustrated with an example. This research also deals with the problem of fine-tuning the BPN output. After training, the output of the BPN must be ideally equal to the corresponding value of the target variable. However, due to the restriction on the number of hidden neurons employed and a possibly premature termination of training by a certain accuracy-related criterion or the maximum number of iterations allowed, the output values of the BPN are in general different from the corresponding values of the targ...