arXiv Science⌕ Search

arXiv subjects

Melike Erol-Kantarci

Publications and source records attributed to Melike Erol-Kantarci.

At least 55 records · Page 3Linked to original sources

Beam Selection in ISAC using Contextual Bandit with Multi-modal Transformer and Transfer Learning

Sixth generation (6G) wireless technology is anticipated to introduce Integrated Sensing and Communication (ISAC) as a transformative paradigm. ISAC unifies wireless communication and RADAR or other forms of sensing to optimize spectral and hardware resources. This paper presents a pioneering framework that leverages ISAC sensing data to enhance beam selection processes in complex indoor environments. By integrating multi-modal transformer models with a multi-agent contextual bandit algorithm, our approach utilizes ISAC sensing data to improve communication performance and achieves high spectral efficiency (SE). Specifically, the multi-modal transformer can capture inter-modal relationships, enhancing model generalization across diverse scenarios. Experimental evaluations on the DeepSense 6G dataset demonstrate that our model outperforms traditional deep reinforcement learning (DRL) methods, achieving superior beam prediction accuracy and adaptability. In the single-user scenario, we achieve an average SE regret improvement of 49.6% as compared to DRL. Furthermore, we employ transfer reinforcement learning to reduce training time and improve model performance in multi-user environments. In the multi-user scenario, this approach enhances the average SE regret, which is a measure to demonstrate how far the learned policy is from the optimal SE policy, by 19.7% compared to training from scratch, even when the latter is trained 100 times longer.

eess.SP↗

Prioritized Value-Decomposition Network for Explainable AI-Enabled Network Slicing

Network slicing aims to enhance flexibility and efficiency in next-generation wireless networks by allocating the right resources to meet the diverse requirements of various applications. Managing these slices with machine learning (ML) algorithms has emerged as a promising approach however explainability has been a challenge. To this end, several Explainable Artificial Intelligence (XAI) frameworks have been proposed to address the opacity in decision-making in many ML methods. In this paper, we propose a Prioritized Value-Decomposition Network (PVDN) as an XAI-driven approach for resource allocation in a multi-agent network slicing system. The PVDN method decomposes the global value function into individual contributions and prioritizes slice outputs, providing an explanation of how resource allocation decisions impact system performance. By incorporating XAI, PVDN offers valuable insights into the decision-making process, enabling network operators to better understand, trust, and optimize slice management strategies. Through simulations, we demonstrate the effectiveness of the PVDN approach with improving the throughput by 67% and 16%, while reducing latency by 35% and 22%, compared to independent and VDN-based resource allocation methods.

cs.NI↗

Heuristic Deep Reinforcement Learning for Phase Shift Optimization in RIS-assisted Secure Satellite Communication Systems with RSMA

This paper presents a novel heuristic deep reinforcement learning (HDRL) framework designed to optimize reconfigurable intelligent surface (RIS) phase shifts in secure satellite communication systems utilizing rate splitting multiple access (RSMA). The proposed HDRL approach addresses the challenges of large action spaces inherent in deep reinforcement learning by integrating heuristic algorithms, thus improving exploration efficiency and leading to faster convergence toward optimal solutions. We validate the effectiveness of HDRL through comprehensive simulations, demonstrating its superiority over traditional algorithms, including random phase shift, greedy algorithm, exhaustive search, and Deep Q-Network (DQN), in terms of secure sum rate and computational efficiency. Additionally, we compare the performance of RSMA with non-orthogonal multiple access (NOMA), highlighting that RSMA, particularly when implemented with an increased number of RIS elements, significantly enhances secure communication performance. The results indicate that HDRL is a powerful tool for improving the security and reliability of RSMA satellite communication systems, offering a practical balance between performance and computational demands.

eess.SP↗

Sum Rate Enhancement using Machine Learning for Semi-Self Sensing Hybrid RIS-Enabled ISAC in THz Bands

This paper proposes a novel semi-self sensing hybrid reconfigurable intelligent surface (SS-HRIS) in terahertz (THz) bands, where the RIS is equipped with reflecting elements divided between passive and active elements in addition to sensing elements. SS-HRIS along with integrated sensing and communications (ISAC) can help to mitigate the multipath attenuation that is abundant in THz bands. In our proposed scheme, sensors are configured at the SS-HRIS to receive the radar echo signal from a target. A joint base station (BS) beamforming and HRIS precoding matrix optimization problem is proposed to maximize the sum rate of communication users while maintaining satisfactory sensing performance measured by the Cramer-Rao bound (CRB) for estimating the direction of angles of arrival (AoA) of the echo signal and thermal noise at the target. The CRB expression is first derived and the sum rate maximization problem is formulated subject to communication and sensing performance constraints. To solve the complex non-convex optimization problem, deep deterministic policy gradient (DDPG)-based deep reinforcement learning (DRL) algorithm is proposed, where the reward function, the action space and the state space are modeled. Simulation results show that the proposed DDPG-based DRL algorithm converges well and achieves better performance than several baselines, such as the soft actor-critic (SAC), proximal policy optimization (PPO), greedy algorithm and random BS beamforming and HRIS precoding matrix schemes. Moreover, it demonstrates that adopting HRIS significantly enhances the achievable sum rate compared to passive RIS and random BS beamforming and HRIS precoding matrix schemes.

eess.SP↗

Generative AI-enabled Blockage Prediction for Robust Dual-Band mmWave Communication

In mmWave wireless networks, signal blockages present a significant challenge due to the susceptibility to environmental moving obstructions. Recently, the availability of visual data has been leveraged to enhance blockage prediction accuracy in mmWave networks. In this work, we propose a Vision Transformer (ViT)-based approach for visual-aided blockage prediction that intelligently switches between mmWave and Sub-6 GHz frequencies to maximize network throughput and maintain reliable connectivity. Given the computational demands of processing visual data, we implement our solution within a hierarchical fog-cloud computing architecture, where fog nodes collaborate with cloud servers to efficiently manage computational tasks. This structure incorporates a generative AI-based compression technique that significantly reduces the volume of visual data transmitted between fog nodes and cloud centers. Our proposed method is tested with the real-world DeepSense 6G dataset, and according to the simulation results, it achieves a blockage prediction accuracy of 92.78% while reducing bandwidth usage by 70.31%.

eess.SP↗

LLM-Based Intent Processing and Network Optimization Using Attention-Based Hierarchical Reinforcement Learning

Intent-based network automation is a promising tool to enable easier network management however certain challenges need to be effectively addressed. These are: 1) processing intents, i.e., identification of logic and necessary parameters to fulfill an intent, 2) validating an intent to align it with current network status, and 3) satisfying intents via network optimizing functions like xApps and rApps in O-RAN. This paper addresses these points via a three-fold strategy to introduce intent-based automation for O-RAN. First, intents are processed via a lightweight Large Language Model (LLM). Secondly, once an intent is processed, it is validated against future incoming traffic volume profiles (high or low). Finally, a series of network optimization applications (rApps and xApps) have been developed. With their machine learning-based functionalities, they can improve certain key performance indicators such as throughput, delay, and energy efficiency. In this final stage, using an attention-based hierarchical reinforcement learning algorithm, these applications are optimally initiated to satisfy the intent of an operator. Our simulations show that the proposed method can achieve at least 12% increase in throughput, 17.1% increase in energy efficiency, and 26.5% decrease in network delay compared to the baseline algorithms.

cs.NI↗

Explainable Multi-Agent Reinforcement Learning for Extended Reality Codec Adaptation

Extended Reality (XR) services are set to transform applications over 5th and 6th generation wireless networks, delivering immersive experiences. Concurrently, Artificial Intelligence (AI) advancements have expanded their role in wireless networks, however, trust and transparency in AI remain to be strengthened. Thus, providing explanations for AI-enabled systems can enhance trust. We introduce Value Function Factorization (VFF)-based Explainable (X) Multi-Agent Reinforcement Learning (MARL) algorithms, explaining reward design in XR codec adaptation through reward decomposition. We contribute four enhancements to XMARL algorithms. Firstly, we detail architectural modifications to enable reward decomposition in VFF-based MARL algorithms: Value Decomposition Networks (VDN), Mixture of Q-Values (QMIX), and Q-Transformation (Q-TRAN). Secondly, inspired by multi-task learning, we reduce the overhead of vanilla XMARL algorithms. Thirdly, we propose a new explainability metric, Reward Difference Fluctuation Explanation (RDFX), suitable for problems with adjustable parameters. Lastly, we propose adaptive XMARL, leveraging network gradients and reward decomposition for improved action selection. Simulation results indicate that, in XR codec adaptation, the Packet Delivery Ratio reward is the primary contributor to optimal performance compared to the initial composite reward, which included delay and Data Rate Ratio components. Modifications to VFF-based XMARL algorithms, incorporating multi-headed structures and adaptive loss functions, enable the best-performing algorithm, Multi-Headed Adaptive (MHA)-QMIX, to achieve significant average gains over the Adjust Packet Size baseline up to 10.7%, 41.4%, 33.3%, and 67.9% in XR index, jitter, delay, and Packet Loss Ratio (PLR), respectively.

cs.NI↗

The impact of mobility, beam sweeping and smart jammers on security vulnerabilities of 5G cells

The vulnerability of 5G networks to jamming attacks has emerged as a significant concern. This paper contributes in two primary aspects. Firstly, it investigates the effect of a multi-jammer on 5G cell metrics, specifically throughput and goodput. The investigation is conducted within the context of a mobility model for user equipment (UE), with a focus on scenarios involving connected vehicles (CVs) engaged in a mission. Secondly, the vulnerability of synchronization signal block (SSB) components is examined concerning jamming power and beam sweeping. Notably, the study reveals that increasing jamming power beyond 40 dBm in our specific scenario configuration no longer decreases network throughput due to the re-transmission of packets through the hybrid automatic repeat request (HARQ) process. Furthermore, it is observed that under the same jamming power, the physical downlink shared channel (PDSCH) is more vulnerable than the primary synchronization signal (PSS) and secondary synchronization signal (SSS). However, a smart jammer can disrupt the cell search process by injecting less power and targeting PSS-SSS or physical broadcast channel (PBCH) data compared to a barrage jammer. On the other hand, beam sweeping proves effective in mitigating the impact of a smart jammer, reducing the error vector magnitude root mean square from 51.59% to 23.36% under the same jamming power.

cs.CR↗

Cooperation and Personalization on a Seesaw: Choice-based FL for Safe Cooperation in Wireless Networks

Federated learning (FL) is an innovative distributed artificial intelligence (AI) technique. It has been used for interdisciplinary studies in different fields such as healthcare, marketing and finance. However the application of FL in wireless networks is still in its infancy. In this work, we first overview benefits and concerns when applying FL to wireless networks. Next, we provide a new perspective on existing personalized FL frameworks by analyzing the relationship between cooperation and personalization in these frameworks. Additionally, we discuss the possibility of tuning the cooperation level with a choice-based approach. Our choice-based FL approach is a flexible and safe FL framework that allows participants to lower the level of cooperation when they feel unsafe or unable to benefit from the cooperation. In this way, the choice-based FL framework aims to address the safety and fairness concerns in FL and protect participants from malicious attacks.

cs.NI↗

Multi-Modal Transformer and Reinforcement Learning-based Beam Management

Beam management is an important technique to improve signal strength and reduce interference in wireless communication systems. Recently, there has been increasing interest in using diverse sensing modalities for beam management. However, it remains a big challenge to process multi-modal data efficiently and extract useful information. On the other hand, the recently emerging multi-modal transformer (MMT) is a promising technique that can process multi-modal data by capturing long-range dependencies. While MMT is highly effective in handling multi-modal data and providing robust beam management, integrating reinforcement learning (RL) further enhances their adaptability in dynamic environments. In this work, we propose a two-step beam management method by combining MMT with RL for dynamic beam index prediction. In the first step, we divide available beam indices into several groups and leverage MMT to process diverse data modalities to predict the optimal beam group. In the second step, we employ RL for fast beam decision-making within each group, which in return maximizes throughput. Our proposed framework is tested on a 6G dataset. In this testing scenario, it achieves higher beam prediction accuracy and system throughput compared to both the MMT-only based method and the RL-only based method.

eess.SP↗

Smart Jamming Attack and Mitigation on Deep Transfer Reinforcement Learning Enabled Resource Allocation for Network Slicing

Network slicing is a pivotal paradigm in wireless networks enabling customized services to users and applications. Yet, intelligent jamming attacks threaten the performance of network slicing. In this paper, we focus on the security aspect of network slicing over a deep transfer reinforcement learning (DTRL) enabled scenario. We first demonstrate how a deep reinforcement learning (DRL)-enabled jamming attack exposes potential risks. In particular, the attacker can intelligently jam resource blocks (RBs) reserved for slices by monitoring transmission signals and perturbing the assigned resources. Then, we propose a DRL-driven mitigation model to mitigate the intelligent attacker. Specifically, the defense mechanism generates interference on unallocated RBs where another antenna is used for transmitting powerful signals. This causes the jammer to consider these RBs as allocated RBs and generate interference for those instead of the allocated RBs. The analysis revealed that the intelligent DRL-enabled jamming attack caused a significant 50% degradation in network throughput and 60% increase in latency in comparison with the no-attack scenario. However, with the implemented mitigation measures, we observed 80% improvement in network throughput and 70% reduction in latency in comparison to the under-attack scenario.

cs.NI↗

Machine Learning-enabled Traffic Steering in O-RAN: A Case Study on Hierarchical Learning Approach

Traffic Steering is a crucial technology for wireless networks, and multiple efforts have been put into developing efficient Machine Learning (ML)-enabled traffic steering schemes for Open Radio Access Networks (O-RAN). Given the swift emergence of novel ML techniques, conducting a timely survey that comprehensively examines the ML-based traffic steering schemes in O-RAN is critical. In this article, we provide such a survey along with a case study of hierarchical learning-enabled traffic steering in O-RAN. In particular, we first introduce the background of traffic steering in O-RAN and overview relevant state-of-the-art ML techniques and their applications. Then, we analyze the compatibility of the hierarchical learning framework in O-RAN and further propose a Hierarchical Deep-Q-Learning (h-DQN) framework for traffic steering. Compared to existing works, which focus on single-layer architecture with standalone agents, h-DQN decomposes the traffic steering problem into a bi-level architecture with hierarchical intelligence. The meta-controller makes long-term and high-level policies, while the controller executes instant traffic steering actions under high-level policies. Finally, the case study shows that the hierarchical learning approach can provide significant performance improvements over the baseline algorithms.

cs.NI↗

Self-Play Ensemble Q-learning enabled Resource Allocation for Network Slicing

In 5G networks, network slicing has emerged as a pivotal paradigm to address diverse user demands and service requirements. To meet the requirements, reinforcement learning (RL) algorithms have been utilized widely, but this method has the problem of overestimation and exploration-exploitation trade-offs. To tackle these problems, this paper explores the application of self-play ensemble Q-learning, an extended version of the RL-based technique. Self-play ensemble Q-learning utilizes multiple Q-tables with various exploration-exploitation rates leading to different observations for choosing the most suitable action for each state. Moreover, through self-play, each model endeavors to enhance its performance compared to its previous iterations, boosting system efficiency, and decreasing the effect of overestimation. For performance evaluation, we consider three RL-based algorithms; self-play ensemble Q-learning, double Q-learning, and Q-learning, and compare their performance under different network traffic. Through simulations, we demonstrate the effectiveness of self-play ensemble Q-learning in meeting the diverse demands within 21.92% in latency, 24.22% in throughput, and 23.63\% in packet drop rate in comparison with the baseline methods. Furthermore, we evaluate the robustness of self-play ensemble Q-learning and double Q-learning in situations where one of the Q-tables is affected by a malicious user. Our results depicted that the self-play ensemble Q-learning method is more robust against adversarial users and prevents a noticeable drop in system performance, mitigating the impact of users manipulating policies.

cs.NI↗

Sustainable Task Offloading in Secure UAV-assisted Smart Farm Networks: A Multi-Agent DRL with Action Mask Approach

The integration of unmanned aerial vehicles (UAVs) with mobile edge computing (MEC) and Internet of Things (IoT) technology in smart farms is pivotal for efficient resource management and enhanced agricultural productivity sustainably. This paper addresses the critical need for optimizing task offloading in secure UAV-assisted smart farm networks, aiming to reduce total delay and energy consumption while maintaining robust security in data communications. We propose a multi-agent deep reinforcement learning (DRL)-based approach using a deep double Q-network (DDQN) with an action mask (AM), designed to manage task offloading dynamically and efficiently. The simulation results demonstrate the superior performance of our method in managing task offloading, highlighting significant improvements in operational efficiency by reducing delay and energy consumption. This aligns with the goal of developing sustainable and energy-efficient solutions for next-generation network infrastructures, making our approach an advanced solution for achieving both performance and sustainability in smart farming applications.

eess.SP↗

Cooperative Hierarchical Deep Reinforcement Learning based Joint Sleep and Power Control in RIS-aided Energy-Efficient RAN

Energy efficiency (EE) is one of the most important metrics for envisioned 6G networks, and sleep control, as a cost-efficient approach, can significantly lower power consumption by switching off network devices selectively. Meanwhile, the reconfigurable intelligent surface (RIS) has emerged as a promising technique to enhance the EE of future wireless networks. In this work, we jointly consider sleep and transmission power control for RIS-aided energy-efficient networks. In particular, considering the timescale difference between sleep control and power control, we introduce a cooperative hierarchical deep reinforcement learning (Co-HDRL) algorithm, enabling hierarchical and intelligent decision-making. Specifically, the meta-controller in Co-HDRL uses cross-entropy metrics to evaluate the policy stability of sub-controllers, and sub-controllers apply the correlated equilibrium to select optimal joint actions. Compared with conventional HDRL, Co-HDRL enables more stable high-level policy generations and low-level action selections. Then, we introduce a fractional programming method for RIS phase-shift control, maximizing the sum-rate under a given transmission power. In addition, we proposed a low-complexity surrogate optimization method as a baseline for RIS control. Finally, simulations show that the RIS-assisted sleep control can achieve more than 16\% lower energy consumption and 30\% higher EE than baseline algorithms.

eess.SY↗

Generative AI-in-the-loop: Integrating LLMs and GPTs into the Next Generation Networks

In recent years, machine learning (ML) techniques have created numerous opportunities for intelligent mobile networks and have accelerated the automation of network operations. However, complex network tasks may involve variables and considerations even beyond the capacity of traditional ML algorithms. On the other hand, large language models (LLMs) have recently emerged, demonstrating near-human-level performance in cognitive tasks across various fields. However, they remain prone to hallucinations and often lack common sense in basic tasks. Therefore, they are regarded as assistive tools for humans. In this work, we propose the concept of "generative AI-in-the-loop" and utilize the semantic understanding, context awareness, and reasoning abilities of LLMs to assist humans in handling complex or unforeseen situations in mobile communication networks. We believe that combining LLMs and ML models allows both to leverage their respective capabilities and achieve better results than either model alone. To support this idea, we begin by analyzing the capabilities of LLMs and compare them with traditional ML algorithms. We then explore potential LLM-based applications in line with the requirements of next-generation networks. We further examine the integration of ML and LLMs, discussing how they can be used together in mobile networks. Unlike existing studies, our research emphasizes the fusion of LLMs with traditional ML-driven next-generation networks and serves as a comprehensive refinement of existing surveys. Finally, we provide a case study to enhance ML-based network intrusion detection with synthesized data generated by LLMs. Our case study further demonstrates the advantages of our proposed idea.

cs.LG↗

Extended Reality (XR) Codec Adaptation in 5G using Multi-Agent Reinforcement Learning with Attention Action Selection

Extended Reality (XR) services will revolutionize applications over 5th and 6th generation wireless networks by providing seamless virtual and augmented reality experiences. These applications impose significant challenges on network infrastructure, which can be addressed by machine learning algorithms due to their adaptability. This paper presents a Multi- Agent Reinforcement Learning (MARL) solution for optimizing codec parameters of XR traffic, comparing it to the Adjust Packet Size (APS) algorithm. Our cooperative multi-agent system uses an Optimistic Mixture of Q-Values (oQMIX) approach for handling Cloud Gaming (CG), Augmented Reality (AR), and Virtual Reality (VR) traffic. Enhancements include an attention mechanism and slate-Markov Decision Process (MDP) for improved action selection. Simulations show our solution outperforms APS with average gains of 30.1%, 15.6%, 16.5% 50.3% in XR index, jitter, delay, and Packet Loss Ratio (PLR), respectively. APS tends to increase throughput but also packet losses, whereas oQMIX reduces PLR, delay, and jitter while maintaining goodput.

cs.NI↗

Generative AI Empowered LiDAR Point Cloud Generation with Multimodal Transformer

Integrated sensing and communications is a key enabler for the 6G wireless communication systems. The multiple sensing modalities will allow the base station to have a more accurate representation of the environment, leading to context-aware communications. Some widely equipped sensors such as cameras and RADAR sensors can provide some environmental perceptions. However, they are not enough to generate precise environmental representations, especially in adverse weather conditions. On the other hand, the LiDAR sensors provide more accurate representations, however, their widespread adoption is hindered by their high cost. This paper proposes a novel approach to enhance the wireless communication systems by synthesizing LiDAR point clouds from images and RADAR data. Specifically, it uses a multimodal transformer architecture and pre-trained encoding models to enable an accurate LiDAR generation. The proposed framework is evaluated on the DeepSense 6G dataset, which is a real-world dataset curated for context-aware wireless applications. Our results demonstrate the efficacy of the proposed approach in accurately generating LiDAR point clouds. We achieve a modified mean squared error of 10.3931. Visual examination of the images indicates that our model can successfully capture the majority of structures present in the LiDAR point cloud for diverse environments. This will enable the base stations to achieve more precise environmental sensing. By integrating LiDAR synthesis with existing sensing modalities, our method can enhance the performance of various wireless applications, including beam and blockage prediction.

cs.CV↗