Merging guidance of exclusive lanes for connected and autonomous vehicles based on deep reinforcement learning

ZHANG Jian, LI Qing-yang, LI Dan, JIANG Xia, LEI Yan-hong, JI Ya-ping

PDF(2508 KB)
PDF(2508 KB)
J Jilin Univ Eng Tech Ed ›› 2023, Vol. 53 ›› Issue (09) : 2508-2518. DOI: 10.13229/j.cnki.jdxbgxb.20220106

Merging guidance of exclusive lanes for connected and autonomous vehicles based on deep reinforcement learning

Author information +
History +

Abstract

Exclusive lanes for connected and autonomous vehicles（CAVs） will emerge in order to ensure the safety and efficiency requirements in the process of traffic flow mixed with human-driving vehicles and CAVs. When the inner lane of the expressway is set as the exclusive lane for CAVs， it has important theoretical significance and practical value to study the strategy of guiding CAVs to merge from the ordinary lane to the exclusive lane. Firstly， the entrance area of exclusive lane was designed and vehicle control rules were proposed. Secondly， with the goal of making more CAVs change lanes to the exclusive lane， the strategy of selecting lane-changing signal actions was proposed based on deep reinforcement learning. Finally， the numerical simulation was carried out with Python language compilation. The results show that the proposed algorithm can converge very quickly under 9 scenarios constructed by different factors， such as the CAV penetration rates and the proportion of CAVs arriving at the exclusive lane； it can effectively guide CAVs to merge into exclusive lanes and ensure traffic efficiency； congestion in the second lane can be significantly reduced compared to the unsignalized control when the penetration rate changes from 20% to 40%； the proportion of CAVs changing to the exclusive lane is significantly higher under the two exclusive lane entrances scenario than that under the one entrance scenario. It shows that the proposed strategy has good applicability and can provide reference for engineering construction.

Key words

engineering of transportation system / merging guidance strategy / deep reinforcement learning / expressway exclusive lane / signal control / connected and autonomous vehicle

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations
ZHANG Jian , LI Qing-yang , LI Dan , et al . Merging guidance of exclusive lanes for connected and autonomous vehicles based on deep reinforcement learning. Journal of Jilin University(Engineering and Technology Edition). 2023, 53(09): 2508-2518 https://doi.org/10.13229/j.cnki.jdxbgxb.20220106

References

List( Publishing order | Descend order by publishing year | Descend order by cited within ) Chart analysis
1
秦严严, 王昊, 王炜. 智能网联环境下的混合交通流LWR模型[J]. 中国公路学报, 2018, 31(11): 147-156.
Qin Yan-yan, Wang Hao, Wang Wei. LWR model for mixed traffic flow in connected and autonomous vehicular environments[J]. China Journal of Highway and Transport, 2018, 31(11): 147-156.
本文引用 [1]
2
Becker F, Axhausen K W. Literature review on surveys investigating the acceptance of automated vehicles[J]. Transportation, 2017, 44(6): 1293-1306.
3
Elliott D, Keen W, Miao L. Recent advances in connected and automated vehicles[J]. Journal of Traffic and Transportation Engineering (English Edition), 2019, 6(2): 109-131.
本文引用 [1]
4
杜豫川, 刘成龙, 吴荻非, 等. 新一代智慧高速公路系统架构设计[J]. 中国公路学报, 2022, 35(4): 203-214.
Du Yu-chuan, Liu Cheng-long, Wu Di-fei, et al. Framework of the new generation of smart highway[J]. China Journal of Highway and Transport, 2022, 35(4): 203-214.
本文引用 [1]
5
Shladover S E. Connected and automated vehicle systems: introduction and overview[J]. Journal of Intelligent Transportation Systems, 2017, 22(3): 190-199.
6
冉斌, 谭华春, 张健, 等. 智能网联交通技术发展现状及趋势[J]. 汽车安全与节能学报, 2018, 9(2): 119-130.
Ran Bin, Tan Hua-chun, Zhang Jian, et al. Development status and trend of connected automated vehicle highway system[J]. Journal of Automotive Safety and Energy, 2018, 9(2): 119-130.
本文引用 [1]
7
Ma K, Wang H. Influence of exclusive lanes for connected and autonomous vehicles on freeway traffic flow[J]. IEEE Access, 2019, 7: 50168-50178.
本文引用 [1]
8
Hua X D, Yu W J, Wang W. Influence of lane policies on freeway traffic mixed with manual and connected and autonomous vehicles[J]. Journal of Advanced Transportation, 2020, 2020: No.3968625.
本文引用 [1]
9
Ghiasi A, Hussain O, Qian Z. A mixed traffic capacity analysis and lane management model for connected automated vehicles: a markov chain method[J]. Transportation Research Part B, 2017, 106: 266-292.
本文引用 [1]
10
Chen D J, Ahn S, Chitturi M. Towards vehicle automation: roadway capacity formulation for traffic mixed with regular and automated vehicles[J]. Transportation Research Part B, 2017, 100: 196-221.
本文引用 [1]
11
Ye L, Yamamoto T. Impact of dedicated lanes for connected and autonomous vehicle on traffic flow throughput[J]. Physica A, 2018, 512: 588-597.
12
Ye L, Yamamoto T. Modeling connected and autonomous vehicles in heterogeneous traffic flow[J]. Physica A, 2018, 490: 269-277.
本文引用 [1]
13
Masher D P, Ross D W, Wong P J, et al. Guidelines for design and operation of ramp control systems[R]. Menlo Park: CA United States: Stanford Research Institute,1975.
本文引用 [1]
14
Smaragdis E, Papageogiou M. Series of new local ramp metering strategies[J]. Transportation Research Record, 2003, 1856: 74-86.
本文引用 [1]
15
Bellemans T, Schutter B D, Moor B D. Model predictive control for ramp metering of motorway traffic: a case study[J]. Control Engineering Practice, 2006, 14(7): 757-767.
本文引用 [1]
16
陈德望, 王飞跃, 陈龙. 基于模糊神经网络的城市高速公路入口匝道控制算法[J]. 交通运输工程学报, 2003(2): 100-105.
Chen De-wang, Wang Fei-yue, Chen Long. Freeway ramp control algorithm based on neurofuzzy networks[J]. Journal of Traffic and Transportation Engineering, 2003(2): 100-105.
17
曾筠程, 邵敏华, 孙立军, 等. 基于有向图卷积神经网络的交通预测与拥堵管控[J]. 中国公路学报, 2021, 34(12): 239-248.
Zeng Yun-cheng, Shao Min-hua, Sun Li-jun, et al. Traffic prediction and congestion control based on directed graph convolution neural network[J]. China Journal of Highway and Transport, 2021, 34(12): 239-248.
18
Zhang J, Jiang X, Liu Z, et al. A study on autonomous intersection management: planning-based strategy improved by convolutional neural network[J]. KSCE Journal of Civil Engineering, 2021, 25(10): 3995-4004.
本文引用 [1]
19
Davarynejad M, Hegyi A, Vrancken J, et al. Motorway ramp-metering control with queuing consideration using Q-learning[C]//2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), Washington, DC, USA, 2011: 1652-1658.
本文引用 [1]
20
郑思. 面向快速道路远距离瓶颈的深度强化学习交通流控制策略研究[D]. 南京:东南大学交通学院, 2021.
Zheng Si. Research on deep reinforcement learning-based active traffic flow control strategies at distant downstream bottlenecks of expressway[D]. Nanjing: School of Transportation, Southeast University, 2021.
21
Belletti F, Haziza D, Gomes G. Expert level control of ramp metering based on multi-task deep reinforcement learning[J]. Transactions on Intelligent Transportation Systems, 2017, 19(4): 1198-1207.
本文引用 [1]
22
葛家丽. 基于车辆簇的高速公路路侧单元部署研究[D]. 济南: 山东科技大学交通学院, 2020.
Ge Jia-li. Research on freeway road side unit deployment based on vehicle clusters[D]. Jinan: School of Transportation, Shandong University of Science and Technology, 2020.
本文引用 [1]
23
陈瑜. 高速公路作业区安全分析及交通组织管理方法研究[D]. 哈尔滨: 哈尔滨工业大学交通科学与工程学院, 2006.
Chen Yu. The safety analysis and organization & management method study of freeway work zone[D]. Harbin: School of Transportation Science & Engineering, Harbin Institute of Technology, 2006.
本文引用 [1]
24
石茂清. 道路交通安全设施设计研究[D]. 成都: 西南交通大学交通运输与物流学院, 2005.
Shi Mao-qing. Study on design of traffic safety facilities[D]. Chengdu: School of Transportation and Logistics, Southwest Jiaotong University, 2005.
本文引用 [1]
25
孙玲, 张静, 周瀛, 等. 车路协同环境下自动驾驶专用车道入口区域设计[J]. 公路交通科技, 2020, 37(): 122-129.
摘要
增刊1
Sun Ling, Zhang Jing, Zhou Ying, et al. Design of entrance area of automatic driving special lane in vehicle-infrastructure collaborative environment[J]. Journal of Highway and Transportation Research and Development, 2020, 37(Sup.1): 122-129.
本文引用 [1]
26
Milanés V, Shladover S E. Modeling cooperative and autonomous adaptive cruise control dynamic responses using experimental data[J]. Transportation Research Part C: Emerging Technologies, 2014, 48: 285-300.
本文引用 [1]
27
Xue Y, Wang X, Cen B L, et al. Study on fuel consumption in the Kerner-Klenov-Wolf three-phase cellular automaton traffic flow model[J]. Nonlinear Dynamics, 2020, 102(1): 1-10.
本文引用 [1]
28
Wang C H, Hwang M C. Value-based deep reinforcement learning for adaptive isolated intersection signal control[J]. IET Intelligent Transport Systems, 2018, 12(9): 1005-1010.
本文引用 [2]
29
Schaul T, Quan J, Antonoglou I, et al. Prioritized experience replay[C]//Proceeding of the 4th International Conference on Learning Representations, San Juan, Puerto Rico, 2016: 322-355.
本文引用 [1]
30
武毅. 基于三相交通流理论的元胞自动机模型研究[D]. 长春: 吉林大学交通学院, 2018.
Wu Yi. Research on cellular automated traffic flow model for three-phase theory[D]. Changchun: College of Transportation, Jilin University, 2018.
本文引用 [1]
31
舒凌洲, 吴佳, 王晨. 基于深度强化学习的城市交通信号控制算法[J]. 计算机应用, 2019, 39(5): 1495-1499.
Shu Ling-zhou, Wu Jia, Wang Chen. Urban traffic signal control based on deep reinforcement learning[J]. Journal of Computer Applications, 2019, 39(5): 1495-1499.
本文引用 [1]

Comments

PDF(2508 KB)

Accesses

Citation

Detail
Sections
Recommended
Copyright © J Jilin Univ Eng Tech Ed, All Rights Reserved.
Powered by Beijing Magtech Co. Ltd.

/

〈 〉