Sequence Q-Learning Algorithm for Optimal Mobility-Aware User Association

Wanjun Ning; Zimu Xu; Jingjin Wu; Tiejun Tong

doi:10.1109/ICC45855.2022.9838645

Sequence Q-Learning Algorithm for Optimal Mobility-Aware User Association

Wanjun Ning, Zimu Xu, Jingjin Wu, Tiejun Tong

Department of Mathematics

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

Abstract

We consider a wireless network scenario applicable to metropolitan areas with developed public transport networks and high commute demands, where the mobile user equipments (UEs) move along fixed and predetermined trajectories and request to associate with millimeter-wave (mmWave) base stations (BSs). An effective and efficient algorithm, called the Sequence Q-learning Algorithm (SQA), is proposed to maximize the long-run average transmission rate of the network, which is an NP-hard problem. Furthermore, the SQA tackles the complexity issue by only allowing possible re-associations (handover of a UE from one BS to another) at a discrete set of decision epochs and has polynomial time complexity. This feature of the SQA also restricts too frequent handovers, which are considered highly undesirable in mmWave networks. Moreover, we demonstrate by extensive numerical results that the SQA can significantly outperform the benchmark algorithms proposed in existing research by taking all UEs' future trajectories and possible decisions into account at every decision epoch.

Original language	English
Title of host publication	ICC 2022 - IEEE International Conference on Communications
Publisher	IEEE
Pages	726-732
Number of pages	7
ISBN (Electronic)	9781538683477
ISBN (Print)	9781538683484
DOIs	https://doi.org/10.1109/ICC45855.2022.9838645
Publication status	Published - May 2022
Event	2022 IEEE International Conference on Communications, ICC 2022 - Seoul, Korea, Republic of Duration: 16 May 2022 → 20 May 2022 https://ieeexplore.ieee.org/xpl/conhome/9837954/proceeding

Publication series

Name	IEEE International Conference on Communications (ICC)
Volume	2022-May
ISSN (Print)	1550-3607
ISSN (Electronic)	1938-1883

Conference

Conference	2022 IEEE International Conference on Communications, ICC 2022
Country/Territory	Korea, Republic of
City	Seoul
Period	16/05/22 → 20/05/22
Internet address	https://ieeexplore.ieee.org/xpl/conhome/9837954/proceeding

Scopus Subject Areas

Computer Networks and Communications
Electrical and Electronic Engineering

User-Defined Keywords

mmWave communication
Mobility-aware user association
NP-hard optimization
Reinforcement learning
Sequence Q-learning Algorithm

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/ICC45855.2022.9838645

Cite this

@inproceedings{42149091eca34debb7716cf5bb2edf93,

title = "Sequence Q-Learning Algorithm for Optimal Mobility-Aware User Association",

abstract = "We consider a wireless network scenario applicable to metropolitan areas with developed public transport networks and high commute demands, where the mobile user equipments (UEs) move along fixed and predetermined trajectories and request to associate with millimeter-wave (mmWave) base stations (BSs). An effective and efficient algorithm, called the Sequence Q-learning Algorithm (SQA), is proposed to maximize the long-run average transmission rate of the network, which is an NP-hard problem. Furthermore, the SQA tackles the complexity issue by only allowing possible re-associations (handover of a UE from one BS to another) at a discrete set of decision epochs and has polynomial time complexity. This feature of the SQA also restricts too frequent handovers, which are considered highly undesirable in mmWave networks. Moreover, we demonstrate by extensive numerical results that the SQA can significantly outperform the benchmark algorithms proposed in existing research by taking all UEs' future trajectories and possible decisions into account at every decision epoch.",

keywords = "mmWave communication, Mobility-aware user association, NP-hard optimization, Reinforcement learning, Sequence Q-learning Algorithm",

author = "Wanjun Ning and Zimu Xu and Jingjin Wu and Tiejun Tong",

note = "Funding Information: The work described in this paper is partly supported by College Research Grant from BNU-HKBU United International College R201911, and Zhuhai Basic and Applied Basic Research Foundation Grant ZH22017003200018PWC. The authors sincerely thank 4 anonymous reviewers for their constructive comments and suggestions that helped to improve the paper. Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE International Conference on Communications, ICC 2022 ; Conference date: 16-05-2022 Through 20-05-2022",

year = "2022",

month = may,

doi = "10.1109/ICC45855.2022.9838645",

language = "English",

isbn = "9781538683484",

series = "IEEE International Conference on Communications (ICC)",

publisher = "IEEE",

pages = "726--732",

booktitle = "ICC 2022 - IEEE International Conference on Communications",

address = "United States",

url = "https://ieeexplore.ieee.org/xpl/conhome/9837954/proceeding",

}

Ning, W, Xu, Z, Wu, J & Tong, T 2022, Sequence Q-Learning Algorithm for Optimal Mobility-Aware User Association. in ICC 2022 - IEEE International Conference on Communications. IEEE International Conference on Communications (ICC), vol. 2022-May, IEEE, pp. 726-732, 2022 IEEE International Conference on Communications, ICC 2022, Seoul, Korea, Republic of, 16/05/22. https://doi.org/10.1109/ICC45855.2022.9838645

Sequence Q-Learning Algorithm for Optimal Mobility-Aware User Association. / Ning, Wanjun; Xu, Zimu; Wu, Jingjin et al.
ICC 2022 - IEEE International Conference on Communications. IEEE, 2022. p. 726-732 (IEEE International Conference on Communications (ICC); Vol. 2022-May).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Sequence Q-Learning Algorithm for Optimal Mobility-Aware User Association

AU - Ning, Wanjun

AU - Xu, Zimu

AU - Wu, Jingjin

AU - Tong, Tiejun

N1 - Funding Information: The work described in this paper is partly supported by College Research Grant from BNU-HKBU United International College R201911, and Zhuhai Basic and Applied Basic Research Foundation Grant ZH22017003200018PWC. The authors sincerely thank 4 anonymous reviewers for their constructive comments and suggestions that helped to improve the paper. Publisher Copyright: © 2022 IEEE.

PY - 2022/5

Y1 - 2022/5

N2 - We consider a wireless network scenario applicable to metropolitan areas with developed public transport networks and high commute demands, where the mobile user equipments (UEs) move along fixed and predetermined trajectories and request to associate with millimeter-wave (mmWave) base stations (BSs). An effective and efficient algorithm, called the Sequence Q-learning Algorithm (SQA), is proposed to maximize the long-run average transmission rate of the network, which is an NP-hard problem. Furthermore, the SQA tackles the complexity issue by only allowing possible re-associations (handover of a UE from one BS to another) at a discrete set of decision epochs and has polynomial time complexity. This feature of the SQA also restricts too frequent handovers, which are considered highly undesirable in mmWave networks. Moreover, we demonstrate by extensive numerical results that the SQA can significantly outperform the benchmark algorithms proposed in existing research by taking all UEs' future trajectories and possible decisions into account at every decision epoch.

AB - We consider a wireless network scenario applicable to metropolitan areas with developed public transport networks and high commute demands, where the mobile user equipments (UEs) move along fixed and predetermined trajectories and request to associate with millimeter-wave (mmWave) base stations (BSs). An effective and efficient algorithm, called the Sequence Q-learning Algorithm (SQA), is proposed to maximize the long-run average transmission rate of the network, which is an NP-hard problem. Furthermore, the SQA tackles the complexity issue by only allowing possible re-associations (handover of a UE from one BS to another) at a discrete set of decision epochs and has polynomial time complexity. This feature of the SQA also restricts too frequent handovers, which are considered highly undesirable in mmWave networks. Moreover, we demonstrate by extensive numerical results that the SQA can significantly outperform the benchmark algorithms proposed in existing research by taking all UEs' future trajectories and possible decisions into account at every decision epoch.

KW - mmWave communication

KW - Mobility-aware user association

KW - NP-hard optimization

KW - Reinforcement learning

KW - Sequence Q-learning Algorithm

UR - http://www.scopus.com/inward/record.url?scp=85137268501&partnerID=8YFLogxK

U2 - 10.1109/ICC45855.2022.9838645

DO - 10.1109/ICC45855.2022.9838645

M3 - Conference proceeding

AN - SCOPUS:85137268501

SN - 9781538683484

T3 - IEEE International Conference on Communications (ICC)

SP - 726

EP - 732

BT - ICC 2022 - IEEE International Conference on Communications

PB - IEEE

T2 - 2022 IEEE International Conference on Communications, ICC 2022

Y2 - 16 May 2022 through 20 May 2022

ER -

Sequence Q-Learning Algorithm for Optimal Mobility-Aware User Association

Abstract

Publication series

Conference

Scopus Subject Areas

User-Defined Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this