Multimedia spoken language translation

Abstract

This paper is concerned with spoken language translation in multimedia communication environment. We summarize the current research activities, by describing various spoken language translation systems and the multimodal technology related to spoken language translation. We propose a spoken language translation system that exploits the multimedia communication environment in order to overcome the limits caused by imperfect speech recognition. Our approach is in contrast to that of most conventional speech translation systems that limit their dialogue domains to obtain better speech recognition. We also propose a performance measure for spoken language translation systems. Our measure is defined as the ratio of the information quantities at each end of communication. Using this measure, we show that multimedia enhance spoken language translation systems.