Similarity-based deep learning for music retrieval

Abstract

The number of music recordings that users can access is increasing, as composition and distribution of music becomes convenient through digitalization. In addition, user-generated music contents are distributed, and the functional use of music to amplify the mood or atmosphere of places (e.g. cafe, restaurant) or media content (e.g. video) is also increasing. Therefore, the act of searching for music contents is becoming very important. Most of the existing music search systems are mainly based on a catalogue-based search or a metadata-based search. Metadata-based search is to search for the exact matching song if you already know the song information such as song name or artist name, and catalogue-based search is to search for songs within categories such as genre, or mood, making detailed search difficult. If there is a way to find similar sounds-like songs to a query song, we will be able to browse and search many music recordings. Traditionally, the related technology is a recommendation algorithm based on the user's listening history. But basically, this recommendation algorithm cannot recommend songs that the user has not already consumed, and the recommendation is a passive act, and its use for active music search is limited. Therefore, the goal of this thesis is to explore content-based music search system that directly analyzes audio and searches through it. In this dissertation, I explore the content-based music search methodology in three main aspects: a module for analyzing audio, a similarity-based deep learning method using various similarity concepts, and a methodology that opens up the possibility of new music search applications. To do that, the background methodology is explained in Chapter 2, an effective audio model is explored in Chapter 3, learning methods using various music similarity concepts are explored in Chapter 4, and a unified framework for similarity-based learning system that enables new music search application is explored in Chapter 5. More concretely, in chapter 3, I propose a more effective audio model that can directly perform on waveform instead of using spectrogram-based features. In chapter 4, I propose a similarity-based learning method that utilizes various music metadata with different levels of similarity concept. In chapter 5, I propose two new music search applications which are query-by-attribute and query-by-prototype by adding several techniques to the similarity-based deep learning methods. Through the exploration of this thesis, I hope we will be able to develop a better content-based music search system.