Off-line handwritten word recognition with hidden markov models

Abstract

In this thesis, a new method for modeling and recognizing cursive words with hidden Markov models (HMM) is presented. Hidden Markov models have been successfully applied to speech recognition tasks, and recently, there have been approaches for applying it to character recognition. However, it is difficult to find a system that extensively made use of the abundant experiences accumulated in speech recognition. We have employed HMM as the main statistical framework, utilizing many of the efficient tools provided in the literature. The framework is described in terms of three processes: Encoding, Modeling, and Recognition. In the proposed method, a sequence of thin fixed-width vertical frames are extracted from the image, capturing the local features of the handwriting. Each frame is considered as a segmentation hypothesis by the recognition algorithm. This can be distinguished from the previous approaches with HMM, where the character segmentation points were heuristically hypothesized and, thus, had the risk of missing the actual segmentation points. By quantizing the frames, the input word image is represented as a Markov chain of discrete symbols. A handwritten word is regarded as a sequence of characters and three types of character junctures. Therefore, the junctures are also explicitly modeled. This can be distinguished from the general view of regarding a word as a sequence of characters only. With this view, an interconnection network of character and juncture HMMs is constructed to model words of indefinite length. This model can ideally describe any form of handwritten words including discretely spaced words, pure cursive words, and unconstrained words of mixed styles. Each character and juncture models are trained with the Baum-Welch method. The recognition algorithm follows the maximum a posteriori decoding rule. Various efficient implementations of the Viterbi search is proposed and evaluated. They are based on the forward and backward evaluation sco...