Beyond max-weight scheduling

Abstract

As network architecture becomes complex and the user requirement gets diverse, the role of ecientnetwork resource management becomes more important. However, existing network scheduling algo-rithms such as the max-weight algorithm suer from poor delay performance. In this paper, we presenta reinforcement learning-based network scheduling algorithm that achieves both optimal throughput andlow delay. To this end, We rst formulate the network optimization problem as a dynamic programmingproblem. Then we introduce a new state-action value function called W-function and develop a rein-forcement learning algorithm called W-learning that guarantees little performance loss during a learningprocess. Finally, via simulation, we verify that our algorithm shows delay reduction of up to 40.8%compared to the max-weight algorithm over various scenarios.