Brain-like hierarchical data representation model - hierarchical multi-layer NMF

Abstract

How to train a machine well so that it can develop intelligence like human has been a big issue of arti-ficial intelligence industry. Recently, deep learning has been a hot topic in this field. It is proven by several researches that deep learning, which resembles hierarchical data processing mechanism of our brain, actually does help learning process and improves the performance. In this thesis, we introduce another type of deep learning network, hierarchical multi-layer NMF, where each unit consists of non-negative matrix factorization algorithm, NMF; we extend single unit of learning algorithm NMF into multi-layered structure. With our pro-posed network, we aim to achieve two goals. One is to mimic and model how our brain processes information and represent data in hierarchical manner. The other is to observe the behavior of our multi-layered hierar-chical network and analyze its characteristics compared to shallow structure. Our demonstration shows that proposed multi-layered structure successfully models hierarchical learning mechanism; in the lower layer, very simple and sparse features are extracted at first, and as it proceeds to upper layer, the features develop into complex features by making combinations of simple features. By observing the hierarchical feature ex-traction process of our proposed algorithm, we are able to understand the underlying structure of complex data; we can understand the basic building blocks and how they come together in stages to finally represent data characteristics. With the analysis on image and document data sets, experimental results show that our brain-like information processing model has some strong points in reconstruction and classification tasks compared to single layered structure. Furthermore, our proposed multi-layer network seems to display charac-teristics different from that of shallow architecture in aspect of distribution of data representation. Our pro-posed multi-layer network is expected to demonstra...