Blind signal separation using neural networks and minimization of mutual information

Abstract

Recently neural networks, known as good universal approximators, have been widely used as a powerful computational tool to effectively learn unknown nonlinear functions. Developement of neural networks comes from an attractive idea that complex solutions can be obtained from learning with input-output data rather than explicit programming; this has made neural networks emerge rapidly as a candidate for solving the complex problems. Due to such characteristics, interest in the neural-based applications for signal processing has increased dramatically. This paper deals with a signal processing for the blind signal separation. Blind signal separation (BSS) is based on information theory. The purpose of BSS is to extract statistically independent signals from the observed signals without knowing how the sources are mixed. Based on information theoretic concepts, there are two major approaches for blind separation: Maximum Entropy (ME) and Minimum Mutual Information (MMI) or Independent Component Analysis (ICA). In this paper, we use MMI for the signal separation. BSS has become a highly popular research topic due to its potential applications in data communications, speech/image recognition and identification problems, analysis of biomedical signals such as electroencephalographic (EEG), electrocardiogram (ECG), and so on. This paper is composed of 5 chapters. The basic concept for the blind signal separation and neural networks is expained in Chapters 1 and 2. Three new algorithms and a new expansion are proposed in Chapter 3 to effectively deal with the blind signal separation problem. And their additional explanations are in Appendices A, B and C. When we solve the BSS problem with MMI or ICA, we need to know the marginal entropy of the output signals and the joint entropy. For a given ICA, the first proposed algorithm in Chapter 3 deals with the contradiction of the previous work and its improvement. It can be done by considering the normalized output signals. ...