Cyrus+: A DRL-based Puncturing Solution to URLLC/eMBB Multiplexing in O-RAN

Puncturing is a promising technique in 3GPP to multiplex Enhanced Mobile Broadband (eMBB) and Ultra-Reliable Low Latency Communications (URLLC) traffic on the same 5G New Radio (NR) air interface. The essence of puncturing is to transmit URLLC packets on demand upon their arrival, by preempting the radio resources (or subcarriers) that are already allocated to eMBB traffic. Although it is considered most bandwidth efficient, puncturing URLLC data on eMBB can lead to degradation of eMBB's performance. Most of the state-of-the-art research addressing this problem employ raw eMBB data throughput as performance metric. This is inadequate as, after puncturing, eMBB data may or may not be successfully decoded at its receiver. This paper presents Cyrus+, a deep reinforcement learning (DRL)-based puncturing solution that employs goodput (through feedback from a receiver's decoder), rather than estimated raw throughput, in its design of reward function. Further, Cyrus+ is tailored specifically for the Open RAN (O-RAN) architecture and fully leverages O-RAN's three control loops at different time scales in its design of DRL. In the Non-Real-Time (Non-RT) RAN Intelligent Controller (RIC), Cyrus+ initializes the policy network that will be used in the RT Open Distributed Unit (O-DU). In the Near-RT RIC, Cyrus+ refines the policy based on dynamic network conditions and feedback from the receivers. In the RT O-DU, Cyrus+ generates a puncturing codebook by considering all possible URLLC arrivals. We build a standard-compliant link-level 5G NR simulator to demonstrate the efficacy of Cyrus+. Experimental results show that Cyrus+ outperforms benchmark puncturing algorithms and meets the stringent timing requirement in 5G NR (numerology 3).