Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT Networks

Abstract

To support dramatically increasing services from internet of thing (IoT) devices with the sporadic and fluctuated generation of short packet traffic, this paper investigates joint dynamic time division duplexing (TDD) and radio resource control (RRC) connection management in a single-cell massive IoT network. Specifically, under the grant-free transmission incurring packet collision, this study models the factors affecting the time resource utilization (TRU) and energy consumption of IoT devices as a comprehensive system utility and further formulates the problem as a decision-making process aiming for balancing the long-term average TRU and energy consumption. To address the formulated problem, based on the deep reinforcement learning framework, this paper designs an experience-driven joint dynamic TDD and RRC connection management scheme that intelligently i) determines the TDD configuration based on the most recent downlink (DL)/uplink (UL) traffic demands and ii) adjusts the RRC state of each IoT device to control the maximum number of transmitting IoT devices. Finally, trace-driven simulation results demonstrate that the proposed scheme outperforms existing benchmarks, such as Static TDD and Dynamic TDD, in terms of transmission success ratio difference (TSRD) (up to 89% reduction), time resource utilization (TRU) (up to $17\times $ increase), and energy consumption (up to 70% reduction) of IoT devices.