Audio feature extraction methods for multimedia content analysis

Abstract

With the rapid distribution of the user created contents (UCCs), a variety of multimedia contents including general contents such as music, movies and malicious contents such as adult videos have been made and shared easily through the Internet. These social trends can negatively affect children or teenagers and cause sexual crimes. Therefore, analyzing the multimedia contents to decide whether the contents are malicious or not has recently received a great attention from many researchers and social groups. This thesis addresses the problem of analyzing the multimedia contents based on audio signals to detect and block the objectionable multimedia contents. The malicious sounds such as sexual scream or moan show the distinctive characteristics that have large temporal variations and fast spectral transitions. Therefore, extracting appropriate features to properly represent these characteristics is important in achieving a better performance. In this thesis, we employ segment-based two-dimensional $\emph{Mel}$-frequency cepstral coefficients and histograms of gradient directions as a feature set to characterize both the temporal variations and spectral transitions within a long-range segment of the target signal. Gaussian mixture model (GMM) is adopted to statistically represent the malicious and non-malicious sounds, and the test sounds are classified by a maximum a posterior probability (MAP) method. Evaluation of the proposed extraction method on a database of several hundred malicious and non-malicious sound clips yielded a classification accuracy of 96.06 %, which was a good performance showing a possibility that could be used as an alternative to the image-based methods.