Sequence Q-Learning Algorithm for Optimal Mobility-Aware User Association

Wanjun Ning, Zimu Xu, Jingjin Wu, Tiejun Tong

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review


We consider a wireless network scenario applicable to metropolitan areas with developed public transport networks and high commute demands, where the mobile user equipments (UEs) move along fixed and predetermined trajectories and request to associate with millimeter-wave (mmWave) base stations (BSs). An effective and efficient algorithm, called the Sequence Q-learning Algorithm (SQA), is proposed to maximize the long-run average transmission rate of the network, which is an NP-hard problem. Furthermore, the SQA tackles the complexity issue by only allowing possible re-associations (handover of a UE from one BS to another) at a discrete set of decision epochs and has polynomial time complexity. This feature of the SQA also restricts too frequent handovers, which are considered highly undesirable in mmWave networks. Moreover, we demonstrate by extensive numerical results that the SQA can significantly outperform the benchmark algorithms proposed in existing research by taking all UEs' future trajectories and possible decisions into account at every decision epoch.

Original languageEnglish
Title of host publicationICC 2022 - IEEE International Conference on Communications
Number of pages7
ISBN (Electronic)9781538683477
ISBN (Print)9781538683484
Publication statusPublished - May 2022
Event2022 IEEE International Conference on Communications, ICC 2022 - Seoul, Korea, Republic of
Duration: 16 May 202220 May 2022

Publication series

NameIEEE International Conference on Communications (ICC)
ISSN (Print)1550-3607
ISSN (Electronic)1938-1883


Conference2022 IEEE International Conference on Communications, ICC 2022
Country/TerritoryKorea, Republic of
Internet address

Scopus Subject Areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

User-Defined Keywords

  • mmWave communication
  • Mobility-aware user association
  • NP-hard optimization
  • Reinforcement learning
  • Sequence Q-learning Algorithm


Dive into the research topics of 'Sequence Q-Learning Algorithm for Optimal Mobility-Aware User Association'. Together they form a unique fingerprint.

Cite this