TAG neural network model for large-scale optical implementation

Abstract

TAG (Training by Adaptive Gain) is a new adaptive learning algorithm developed for optical implementation of large-scale artificial neural networks. For single-layer neural networks with N input and M output neurons TAG contains two different types of interconnections, i.e., MN global fixed interconnections and $\beta$(N + M) adaptive gain controls, where $\beta$ is the number of local interconnectivity. For the same number of input and output neurons TAG requires much less adaptive elements, and provides a possibility for large-scale implementation at some sacrifice in performance as compared to the perceptron. The training algorithm is based on gradient descent and error backpropagation, and is easily extensible to multilayer, and/or high-order architectures. Computer simulation demonstrates reasonable performance of TAG compared to perceptron performance. For large-scale electro-optic implementation the fixed global interconnections may be implemented by multi-facet hologram, volume hologram or ground glass, while the adaptive gains by spatial light modulators (SLMs). The ground glass is more advantageous for random interonnections with much higher diffraction efficiency and interconnection density. Both feed-forward signal and error back-propagation paths are implemented by single ground glass, and adaptive learning has been demonstrated for hetero-associative memory and classifiers. Optical implementation of multi-layer bidirectional memory and inverse scattering application of the TAG model are also included as Appendices.