Representation learning for boundary detection in music structure analysis

Abstract

Music has a structure that consists of functional sections such as verse or chorus in pop music. Detecting the boundaries between two functionally homogeneous sections is a front-end task toward complete music structure analysis. Boundary detection is typically conducted on a self-similarity matrix (SSM) computed from audio features that capture harmonic or timbral characteristics of a music track. Traditionally, hand-crafted features that explicitly extract the characteristics such as Mel-Frequency Cepstral Coefficients (MFCC), chroma have been common choices. In this paper, we propose a method to learn feature representations via deep neural networks to obtain more effective SSM. Specifically, we use a Siamese-style neural network with a triplet loss that consists of anchor, positive and negative examples to train the model. The anchor and positive samples are selected from the same or temporally close section in music structure whereas the negative samples are from the outside of the section that anchor and positive samples are selected. We show that this approach tends to render the audio features to bemore homogeneous within a section. Once we compute the SSM from the learned features, we apply a Gaussian checkerboard kernel to detect the structure boundary. We evaluate the performance of the proposed method on the SALAMI dataset. The results show that the propose method outperforms the traditional hand-crafted features when the same setup is used except the audio features.