MSc. Ramsha Narmeen

Efficient radio resource management (RRM) in wireless networks is challenging due to mutual dependence among various RRM parameters. In this paper, we provide a proof-of-concept of coordinated machine learning framework designed to enhance RRM in a realistic network environment. For this purpose, we employ two lightweight deep neural networks (DNNs) that interact through bi-directional feedback, allowing each DNN to refine the prediction of other DNN. The coordination mitigates error propagation and accumulation, leading to improved predictions and more efficient RRM. We implement the framework to open-source OpenAirInterface (OAI) platform, providing a practical proof-of-concept and releasing the corresponding datasets and platform modifications to facilitate reproducibility and further research. We conduct extensive realworld experiments to demonstrate that the coordinated machine learning significantly outperforms both conventional machine learning-based and traditional RRM techniques.

We focus on handover management in networks integrating traditional terrestrial ground base stations (GBSs) and non-terrestrial unmanned aerial vehicles (UAVs) serving users. In such scenario, we propose a joint management of handover of users equipment (UEs) between UAVs and GBSs as well as handover of UAVs between GBSs. Our goal is to maximize the sum capacity for UEs while avoiding redundant handovers. As an impact of handover on the future network performance is not explicit, we adopt deep deterministic policy gradient (DDPG). Furthermore, since the UAVs are usually energy constrained, we consider an energy efficient transparent relaying mode for the UAVs. However, in the transparent relaying mode, the access channel quality between the UAV relay and the UE is unknown even if such information is essential for handover. Thus, we further employ deep neural network (DNN) to predict the access channel quality. An incorporation of DDPG for handover management together with DNN for channel quality prediction can impair the sum capacity of the UEs due to an accumulation of inherent small prediction errors of DNN and DDPG. Hence, we also introduce a coordination between DNN and DDPG to suppress the accumulation of the prediction errors. Simulations demonstrate that the proposed handover management and the coordination of DDPG and DNN increase the sum capacity by up to 63% while notably reducing the number of handovers and handover failure ratio compared to the state-of-the-art works.

We introduce a novel framework jointly managing handovers of user equipments (UEs) and Unmanned Aerial Vehicles (UAVs) serving the UEs. The goal is to maximize the sum capacity of the UEs while considering a cost related to the handovers. To this end, we introduce a novel approach based on deep deterministic policy gradient (DDPG) adjusting the Cell Individual Offset (CIO) for handovers of the UEs among the UAVs and ground base stations (GBSs) as well as handovers of the UAVs among the GBSs. The UAVs playing the role of relays often face challenges related to the implementation cost and energy limitations. To address these challenges, the UAVs should operate in a transparent relaying mode. In such mode, unfortunately, the channels between the UEs and the UAVs are unknown as the transparent relays lack any communication control-related functionalities. Therefore, we adopt a deep neural network (DNN) to predict the channel qualities among the UEs and the UAVs for the handover purposes. We demonstrate that the proposal significantly increases the sum capacity of the UEs by dozens of percent and even reduces the number of handovers compared to state-of-the-art works. At the same time, the proposed DDPG-based CIO setting reduces a gap in the sum capacity between the predicted and the optimal (but practically not feasible) case with perfectly known channels among UEs and UAVs. Hence, the proposal is suitable for practical scenarios with not perfectly accurate channel quality information.

Seamless connectivity and efficient mobility management in mobile networks can be facilitated via estimation of the speed and mobility state of user equipment (UE). In this paper, we propose a deep reinforcement learning (DRL) framework for prediction of the UE speed based on the number of performed handovers and density of all base stations (BSs), including both macro base stations (MBSs) and small-cell base stations (SBSs). We leverage the capability of DRL to learn and adapt to complex mobility patterns, enhancing the accuracy of the speed prediction in highly dynamic environments. To demonstrate benefits of the DRL-based UE speed prediction, we further exploit the predicted speed to determine the UEs’ mobility states. The mobility state serves as an input for handover optimization via fuzzy logic. By incorporating the speed prediction, the system proactively anticipates changes in the UEs’ mobility and enables more precise and timely optimization of handover parameters. The simulation with realistic UE mobility traces shows that the proposed DRL improves the accuracy of the speed prediction by up to 35% and reduces the prediction root mean square error by up to 59% compared to the state-of-the-art algorithms. Moreover, using the proposed DRL-based speed prediction for fuzzy logic-based handover optimization improves the UEs’ sum capacity by up to 43% and reduces the number of handovers by up to 39% compared to state-of-the-art works.

We develop a framework to simultaneously optimize the artificial noise (AN) covariance of eavesdroppers, the high-dimensional continuous phase shifts of Unmanned Aerial Vehicles (UAV)-borne reconfigurable intelligent surfaces (RISs), and beamforming of base stations (BSs) in vehicular networks. This optimization aims to maximize the worst-case secrecy rate for vehicular users (VUs) amidst eavesdroppers considering inaccurate channel state information (CSI) and VUs' quality-of-service (QoS) requirements. We propose a predictive localization-based actor-critic deep reinforcement learning (DRL) solution. We employ a deep neural network (DNN) to predict the future spatial location of VUs and eavesdroppers, since these are apriory unknown. Then, the predicted positions are fed to a Deep Deterministic Policy Gradient (DDPG) algorithm, which determines the AN matrix and RIS beamforming vector. The DDPG integrated with DNN-based predictive localization offers a significant advantage over the standard DDPG, as additional states are incorporated into the system, thereby enhancing the model's capability to coordinate DDPG based on predicted location and to capture more complex and abstract features of the highly dynamic vehicular network. Simulations demonstrate the proposed DDPG with predictive localization ensures secure and reliable communications, surpassing state-of-the-art baselines.

Unmanned aerial vehicles (UAVs) have become essential elements of non-terrestrial networks (NTNs) utilized in various sectors, including agriculture, public safety, surveillance, and critical military operations. However, in addition to the benefits of these NTNs, they are increasingly being exploited for malicious purposes, leading to a heightened need for timely detection and identification. Despite advances in UAV detection, challenges remain, particularly in dealing with different UAV types, the payloads they carry, and their flight characteristics. Relying on a single convolutional neural network (CNN) for UAV detection and identification presents difficulties in managing diverse datasets and capturing complex, interdependent relationships. To address this, we propose a novel approach that integrates the visual geometry group-based CNN for UAV detection and the mask region-based CNN for the identification of various traits of UAVs. Additionally, to overcome the computational complexity of deep convolutional layers, we introduce a quantum computation-based CNN (QCNN) instead of the conventional CNN, applied in both the visual geometry group-based detection and mask region-based identification processes, jointly termed VM-QCNN. To effectively deploy VM-QCNN, we enhance the dataset by applying data augmentation techniques, which add diversity to the training data. This ensures the model accurately detects various UAV types, payload categories, and flight characteristics. Performance evaluation through simulations demonstrates that the VM-QCNN approach significantly improves the detection of malicious UAVs compared to competitive algorithms.

To ensure extensive connectivity for a large number of Internet of Things (IoT) devices, there is a critical need for effective random access (RA). However, the substantial number of IoT users accessing the network leads to severe collisions, which cause delays in establishing successful RA connections and ultimately degrade system performance. While recent research has focused on binary preamble detection and the management of random access channel (RACH) overload, it has not specifically addressed RA overload for massive IoT users. This paper proposes a joint solution for preamble collision detection and timing advance (TA) prediction, referred to as PC-TA, inspired by generative adversarial networks (GANs). We utilize an actor-critic-based deep deterministic policy gradient (AC-DDPG) framework as one of the neural networks within the GAN, specifically designed to tackle preamble collision detection and resolution for numerous IoT users. In addition, we implement a fully connected graph neural network (GNN) as the second neural network in the GAN to predict timing advance (TA), which improves the average packet success rate and reduces overall latency. Simulation results validate the effectiveness of the proposed PC-TA, showing an approximate 85% gain over state-of-the-art methods in the existing literature. Consequently, this approach significantly enhances the chances of rapid RA success, allowing each IoT device to achieve successful RA with fewer trials.

The evolution of Artificial Intelligence (AI) integrated with the Sixth-generation (6G) framework poses significant challenges to low-latency applications. Recently, semantic communication has emerged as a promising technique for future intelligent applications. However, the resource management problem combined with semantics is not fully explored. In this paper, we present a deep reinforcement learning-based twin-delayed deep deterministic policy gradient (TD3) for task-centered semantic communication. The proposed TD3 algorithm optimizes bandwidth, and semantic information and prioritizes data with maximum signal-to-noise ratio (SNR) for the efficient transmission of useful information. Simulation results demonstrate the effectiveness of the proposed TD3 scheme compared to state-of-the-art work in terms of transmission efficiency by up to 36% for varying users and up to 33% for varying SNR.

User applications based on the deep neural networks (DNNs), such as object or anomaly detection, image recognition, or language processing, running on computation- and energy-constrained user equipment (UE) can be partially or fully processed in the edge computing servers to reduce a processing time and save an energy in the UE. To further reduce the processing time and the UE's energy consumption, DNN with multiple exit points can be incorporated. In this article, we address the problem of the decision on whether the computation should be offloaded from the UE to the edge computing server or processed locally by the UE and we solve this problem jointly and "on-the-fly" together with DNN exit selection. Since the formulated problem is very complex, we exploit the deep deterministic policy gradient for the exit selection and the offloading decisions (labeled DDPG-EOD) for the DNN-based applications. To this end, we first convert the problem into the Markov decision process, and then, we employ an end-to-end learning via DDPG with the actor-critic architecture. Second, we use a knowledge distillation-based technique to efficiently select the DNN's exit to minimize the delay and energy consumption. Simulation results show that the proposal is highly scalable, converges very quickly, and surpasses the best performing state-of-the-art approach by up to 120% and 100% in terms of the overall DNN processing delay and the energy consumption, respectively.

Multihop communication has gained prominence within the realm of the Internet of Underwater Things (IoUT) owing to its exceptional reliability amidst the challenges posed by the underwater acoustic environment. Despite this, the persistence of limitations caused by propagation delay, high collision rate, and limited energy in underwater communication remains, representing the most formidable hurdles in ensuring the successful transmission of data gathered by sensor nodes. To address these challenges, we employ a machine learning (ML)-based optimal cooperating node selection for each hop, considering the Shortest propagation delay, minimal residual Energy, and a low Collision rate (referred to as SEC). For this purpose, we initially assemble the sensor nodes to create a list of cooperative nodes, considering the aspect of SEC. Then, using an assembled list of cooperating sensor nodes, we employ ML-based algorithms, such as reinforcement learning (RL-SEC), deep Q-networks (DQN-SEC), and deep deterministic policy gradient (DDPG-SEC), to predict the optimal cooperating node for each hop. The simulation results of the DDPG-SEC demonstrate a significant improvement of approximately 56% when compared with RL-SEC, DQN-SEC, and other state-of-the-art techniques.

Unmanned aerial vehicles (UAVs) offer advantages in serving as aerial small cells (ASCs) to support public safety terrestrial cells (PSTCs) while providing pervasive coverage during disasters. To ensure reliable communications for long-term evolution-based public safety (PS-LTE) users, it is crucial to obtain an accurate understanding of network performance for practical cell association design and network stability. This comprehension is vital for the practical design of cell associations and for maintaining network stability in next-generation consumer wireless networks. For this purpose, we first employ a flexible biased cell association (FBCA) policy that optimally selects the bias factor where a PS-LTE user (PUE) connects to the eNodeB (eNB) giving the maximum power for the received signal. Then, we present a resource allocation and subframe-type selection by formulating stochastic optimization programming to resolve system stability issues in the coexisting PS-LTE andLTE-based high-speed railway (LTE-R) networks and PS-LTE and UAV networks. In addition to this, we employ the Lyapunov optimization technique to seek an optimal almost blank subframe (ABS) algorithm with dynamic delay-aware resource allocation (ADDRA) to resolve the problem of network stability. Using ADDRA, the PS-LTE eNodeB (PSeNB), the aerial eNodeBs (AeNBs), and the LTE-R eNodeBs (ReNBs) obtain up-to-date queues of attached users and accordingly compute a matrix for scheduling resources based on channel state information (CSI) feedback. The simulation results of the UAV-assisted networks using FBCA and ADDRA in coexisting PS-LTE/LTE-R and PS-LTE/UAV networks demonstrate a significant improvement when compared with other state-of-the-art techniques.

Unmanned aerial vehicles (UAVs) equipped with reconfigurable intelligent surfaces (RISs) have emerged as a promising technology for numerous applications involving aerial networks. However, the UAV-RIS concept faces challenges related to the deployment of the UAV-RIS, especially in cases, where UAV-RIS is combined with emerging technologies, such as beamforming, sensitive to propagation channel variation. In this article, we first overview various use-cases of UAV-RIS beam-forming considering practical scenarios. Aiming to improve the performance of communication channels, we propose a machine learning-based beamforming policy for UAV-RIS by employing prioritized experience replay (PER) based deep Q-Network (DQN). Compared to traditional approaches, the proposed PER DQN-based beamforming for UAV-RIS communication provides significant enhancements in performance. Finally, we highlight some potential directions for future research.