Ergodic behavior of nonstationary finite markov processes

Abstract

In this thesis, the ergodicity of some finite Markov processes is investigated. Since the transition probability matrix of a finite Markov process is characterized by its backward and foreward equations, its long run transition probability is confirmed with the limiting behavior of the associated linear dynamical system, which is obtained by decomposing the foreward equation into K equations, where K equals the number of states. Through this study, it is shown that the ergodicity of a stationary finite Markov process which is already known can be proved through the analysis of the associated linear dynamical system. Furthermore, a nonstationary finite Markov process is analyzed to be ergodic under some restrictions, the fact which utilizes the reducibility of vector space into subspaces through a projection and the stability of linear dynamical systems. Besides, some additional characteristics and results related to the above analysis are given, which show some possibilities of the extensions of the ergodicity conditions and the treatment of finite Markov processes as finite Markov chains.