Shot detection and temporal interest point for event-based clustering and its application to golf videos

Abstract

Recently, a great amount of information has been generated and stored through computer, TV, Internet, DVD, surveillance systems and digital libraries. This information has been transformed into image-based forms, which require a number of the storage devices. In such a situation, people have difficulties in finding what they want to see and collect. Especially, people are interested in meaningful events such as sports highlights or specific events captured in surveillance cameras. So the automatic video indexing and retrieval is an essential technique in video applications. A video can be hierarchically abstracted according to its syntactic features and semantics. There exist video semantics such as event or scenario in the highest level. They are represented by syntactic features like editing effects (e.g. cut and dissolve) and key frames (or shots) in the lower level. The lowest level consists of scene features (motion, color, shape, and texture) of key frames or partitioned shots. Although extensive research has been performed on the video hierarchy, the work related to each level was studied independently one another. In other words, shot boundaries, key frames, and their features was not unified to describe higher semantic level. In the specific case of sports videos like golf, a drive shot (full-swing) of a player tends to be captured into two different scenes because one camera cannot cover the whole volume of space and time made by the fast moving ball. Moreover, the different camera motion tracking the ball provides an important cue to represent the event. So, the key frames generated by discontinuous camera motion could efficiently summarize the event, and also scene features at those key frames can be used to index the event. In this paper, we address the abstraction of video events, especially sports (golf) highlights. The key frames together with shot boundaries provide temporal interest points (TIP) for event representation. In the meanwhile, the ...