TY - JOUR

T1 - Multi type mean field reinforcement learning for optimal resource allocation in heterogeneous network

AU - Sun, QS

AU - Zhang, YY

AU - Wu, HT

AU - Li, Y

AU - Petrosian, O

N1 - Times Cited in Web of Science Core Collection: 2 Total Times Cited: 2 Cited Reference Count: 37

PY - 2025

Y1 - 2025

N2 - With the exponential growth in the amount of data transmitted over mobile networks, contemporary 5G communication technologies with the primary goal of improving network performance and quality of service have gained much attention. Efficient resource allocation and interference management are especially critical in large-scale wireless networks. Device-to-device (D2D) communication has become a promising technological tool to address this growing need. However, the limitation of exponentially growing solution space in largescale ultra-dense networks makes it difficult to achieve real-time control with conventional optimization methods. To face this challenge, we propose a novel framework that combines Multi-Agent Reinforcement Learning (MARL) with Mean Field Type Game (MFTG) theory, allowing agents to operate in different action spaces. This approach extends the core principle of mean-field reinforcement learning from a single type to multiple types of interactions, effectively modeling the approximate behavior between various types of devices in heterogeneous D2D networks. Experimental results show that the proposed Multi-Type Mean-Field double deep Q-network (MTMF-Q) method outperforms benchmark methods in heterogeneous networks. In addition, the proposed method exhibits good scalability in parameters such as user density, network size and power budget, showing its potential for application in ultra-dense heterogeneous communication network scenarios.

AB - With the exponential growth in the amount of data transmitted over mobile networks, contemporary 5G communication technologies with the primary goal of improving network performance and quality of service have gained much attention. Efficient resource allocation and interference management are especially critical in large-scale wireless networks. Device-to-device (D2D) communication has become a promising technological tool to address this growing need. However, the limitation of exponentially growing solution space in largescale ultra-dense networks makes it difficult to achieve real-time control with conventional optimization methods. To face this challenge, we propose a novel framework that combines Multi-Agent Reinforcement Learning (MARL) with Mean Field Type Game (MFTG) theory, allowing agents to operate in different action spaces. This approach extends the core principle of mean-field reinforcement learning from a single type to multiple types of interactions, effectively modeling the approximate behavior between various types of devices in heterogeneous D2D networks. Experimental results show that the proposed Multi-Type Mean-Field double deep Q-network (MTMF-Q) method outperforms benchmark methods in heterogeneous networks. In addition, the proposed method exhibits good scalability in parameters such as user density, network size and power budget, showing its potential for application in ultra-dense heterogeneous communication network scenarios.

KW - Resource allocation

KW - Heterogeneous networks

KW - Mean field game

KW - Reinforcement learning

UR - https://www.scopus.com/pages/publications/105007598899

U2 - 10.1016/j.engappai.2025.111207

DO - 10.1016/j.engappai.2025.111207

M3 - статья

VL - 158

JO - Engineering Applications of Artificial Intelligence

JF - Engineering Applications of Artificial Intelligence

SN - 0952-1976

ER -