Automatic rhythm game action generation from music using deep learning

Abstract

This research aims to automatically generate high-quality beatmaps using a deep learning algorithm. Beatmaps are patterns of action events mapped on a timeline of musical beats that constitute a level of a rhythm game. They are typically handcrafted, limiting the number of available songs and adaptability to different skill levels and musical preferences. For this reason, players often resort to manual beatmap creation, a time-consuming and repetitive process that requires great precision and attention to detail. The proposed approach employs a C-LSTM architecture with feature-wise linear modulation (FiLM) layers to predict action patterns that match the desired level of challenge, given source separated audio as input. This study contributes to improving difficulty control and rhythm pattern accuracy in beatmap generation, and establishing criteria for beatmap data collection and evaluation. The results allow developers and players to focus on originality and details, enabling minor video game publishers to enter the market competitively by overcoming resource limitations.