مهندسی مکانیک مدرس

مهندسی مکانیک مدرس

طراحی و شبیه‌سازی سیستم خودکار مبتنی بر یادگیری تقویتی برای تنظیم موقعیت خودروهای مفصلی

نوع مقاله : مقاله پژوهشی

نویسندگان
معین قنبری سنجگانی
مجید ساده دل
گروه مهندسی مکانیک، دانشگاه تربیت مدرس، تهران، ایران
10.48311/mme.2025.117147.82868
چکیده
تنظیم موقعیت خودکار یکی از کاربردهای کلیدی در حوزه خودروهای هوشمند است که در افزایش ایمنی و بهینه‌سازی عملیات مانور وسایل نقلیه نقش اساسی دارد. خودروهای مفصلی به دلیل برخورداری از درجه آزادی بالا و دینامیک پیچیده و غیرخطی ناشی از مفصل بین کشنده و تریلر، کنترل دشوارتری نسبت به خودروهای صلب دارند. هدف از این پژوهش، طراحی و شبیه‌سازی یک سیستم کنترل خودکار برای تنظیم موقعیت خودروهای مفصلی با بهره‌گیری از چارچوب یادگیری تقویتی عمیق است. این سیستم می‌تواند به عنوان زیربنایی برای کاربردهای پیشرفته‌تر نظیر پارک خودکار مورد استفاده قرار گیرد.در این مطالعه، ابتدا مدل‌سازی دقیق دینامیک حرکت خودروی مفصلی و پدیده قیچی‌شدن انجام گرفت و مدل توسعه‌یافته با استفاده از نرم‌افزار تخصصی صحت‌سنجی شد. سپس، فرآیند یادگیری به منظور کاهش پیچیدگی محاسباتی، به دو فاز مجزا (آماده‌سازی مانور و تنظیم نهایی موقعیت) تقسیم شد. برای آموزش عامل هوشمند در این دو فاز، از الگوریتم‌های گرادیان سیاست قطعی عمیق (DDPG) و گرادیان سیاست قطعی عمیق دوگانه تأخیری (TD3)، که با شبکه‌های عصبی شامل ۳ تا ۵ لایه نهان بهینه‌سازی شدند، استفاده گردید. نتایج ارزیابی‌ها نشان داد که الگوریتم TD3، به دلیل توانایی بالاتر در حفظ پایداری فرآیند یادگیری، عملکرد بهتری نسبت به DDPG ارائه می‌دهد. در نهایت، سیستم کنترل پیشنهادی با ساختار بهینه برای هر فاز، موفقیت‌هایی به ترتیب ۹۶.۶٪ در فاز آماده‌سازی و ۹۴.۶٪ در فاز تنظیم نهایی را کسب کرد که کارایی بالا و قابلیت اطمینان سیستم مبتنی بر DRL در مواجهه با چالش‌های کنترلی خودروهای مفصلی را تأیید می‌کند.
کلیدواژه‌ها
برنامه‌ریزی حرکت
خودروی مفصلی
یادگیری تقویتی
گرادیان سیاست قطعی عمیق

موضوعات

عنوان مقاله English

Design and Simulation of an Autonomous System Using Reinforcement Learning for Articulated Vehicle Pose Adjustment

نویسندگان English

Moein Qanbari Senjegani
Majid Sadedel
Mechanical Engineering Department, Tarbiat Modares University, Tehran, Iran
چکیده English

Automatic posture adjustment is a key application in the domain of intelligent vehicles, playing a fundamental role in enhancing safety and optimizing vehicle maneuvering operations. Articulated Vehicles (AVs), due to their high degrees of freedom and the complex, nonlinear dynamics resulting from the joint between the tractor and trailer, present a more challenging control problem compared to rigid vehicles. The objective of this research is to design and simulate an automatic control system for articulated vehicle posture adjustment utilizing a Deep Reinforcement Learning (DRL) framework. This system can also serve as a foundation for more advanced applications, such as autonomous parking.

In this study, the precise modeling of articulated vehicle dynamics and the jackknifing phenomenon was initially carried out. The developed model was validated using the specialized software, TruckSim. Subsequently, to reduce computational complexity, the learning process was segmented into two distinct phases: maneuver preparation and final posture adjustment. For training the intelligent agent, Deep Deterministic Policy Gradient (DDPG) and Twin Delayed DDPG (TD3) algorithms were employed, which were optimized with neural networks comprising three to five hidden layers. Evaluation results indicated that the TD3 algorithm, owing to its superior ability to maintain the stability of the learning process, outperformed DDPG. Ultimately, the proposed control system, with the optimal structure for each phase, achieved success rates of 96.6% in the preparation phase and 94.6% in the final adjustment phase, thereby confirming the high efficiency and reliability of the DRL-based system in addressing the control challenges of articulated vehicles.

کلیدواژه‌ها English

Motion Planning
Articulated Vehicle
Reinforcement Learning
Deep Deterministic Policy Gradient (DDPG)
Twin Delayed Deep Deterministic Policy Gradient (TD3)
1] H. Rezaei, H. N. Pishkenari, S. Jafari, and E. M. J. Al-Rashed, "Automatic parking of an articulated vehicle using ANFIS," in Proc. IEEE Int. Symp. Ind. Electron. (ISIE), Taipei, Taiwan, 2013, pp. 2519-2524.
[2] R. P. Vieira, E. V. Argento, and T. C. Revoredo, "Trajectory planning for car-like robots through curve parametrization and genetic algorithm optimization with applications to autonomous parking," IEEE Latin Am. Trans., vol. 20, no. 2, pp. 309-316, Feb. 2022, doi: 10.1109/TLA.2022.9661471.
[3] B. Li, T. Acarman, Y. Zhang, Y. Ouyang, C. Yaman, Q. Kong, X. Zhong, and X. Peng, "Optimization-based trajectory planning for autonomous parking with irregularly placed obstacles: A lightweight iterative framework," IEEE Trans. Intell. Transp. Syst., vol. 23, no. 8, pp. 11970-11981, Aug. 2022, doi: 10.1109/TITS.2021.3109011.
[4] H. R. Nedamani, M. Soleymanifard, A. Safaeifar, and P. M. Khiabani, "Soft Computing-Based Driver Modeling for Automatic Parking of Articulated Heavy Vehicles," SAE Int. J. Commer. Veh., vol. 16, no. 03-16-04, pp. 268-281, 2023, doi: 10.4271/02-16-04-0027.
[5] S. Azadi, H. R. Nedamani, and R. Kazemi, "Automatic parking of an articulated vehicle using ANFIS," Global J. Sci. Eng. Technol., vol. 3, pp. 93-104, 2013.
[6] J. Zhang, H. Chen, S. Song, and F. Hu, "Reinforcement learning-based motion planning for automatic parking system," IEEE Access, vol. 8, pp. 154485-154501, Aug. 2020, doi: 10.1109/ACCESS.2020.3017770.
[7] S. Song, H. Chen, H. Sun, and M. Liu, "Data efficient reinforcement learning for integrated lateral planning and control in automated parking system," Sensors, vol. 20, no. 24, p. 7297, Dec. 2020, doi: 10.3390/s20247297.
[8] T. Lei, J. Wang, and Z. Yao, "Modelling and stability analysis of articulated vehicles," Appl. Sci., vol. 11, no. 8, p. 3663, Apr. 2021, doi: 10.3390/app11083663.
[9] B. Saeedi and M. Sadedel, "Implementation of Behavior-Based Navigation Algorithm on Four-Wheel Steering Mobile Robot," Int. J. Adv. Manuf. Technol., vol. 113, no. 3-4, pp. 903-920, Mar. 2021. doi: 10.22059/jcamech.2021.330072.648
[10] A. Molaei and M. Amirkhani, "Distributed Reinforcement Learning Framework for Autonomous Driving on Highways," Iranian Journal of Intelligent Systems, vol. 10, no. 2, pp. 45-59, 2021.
[11] R. Rizehvandi and A. Azadi, "Deep Reinforcement Learning for Highway Driving Decision-Making," Journal of Transportation Systems, vol. 15, no. 3, pp. 112-128, 2024.
[12] A. Adrisi, K. Bagherzadeh Chehreh, and A. Naderi, "Proposing an Innovative Model Based on Reinforcement Learning for Intelligent Vehicle Routing in a Dynamic Network (Case Study: City of Isfahan)," in Proc. Int. Conf. Civil Eng., Archit., Urbanism Contemp. Iran, 2017.
[13] T. T. Li, J. Wang, and Z. Yao, "Nonlinear Dynamic Model for Articulated Vehicles and Stability Analysis," J. Veh. Dyn., vol. 42, no. 1, pp. 56-68, Jan. 2018.
[14] J. Jung, H. Chen, S. Song, and F. Hu, "Side Slip Angle Estimation with Dual Kalman Filters," J. Veh. Control, vol. 25, no. 2, pp. 134-147, Feb. 2019.
[15] J. Doe, H. Chen, S. Song, and F. Hu, "Lyapunov Control for Articulated Systems," IEEE Trans. Control Syst. Technol., vol. 24, no. 5, pp. 1789-1802, Sep. 2016.
[16] K. D. A. van de Vendel, J. Elfring, and A. J. Aertssen, "Path Planning for a Reverse Parking Maneuver using Reinforcement Learning," TU Eindhoven Repository (pure.tue.nl), 2025.
[17] A. Aertssen, R. Huisman, I. Besselink, and M. J. G. R. van de Molengraft, "Real-Time Trajectory Planning for an Autonomous Articulated Commercial Vehicle," in Proc. 43rd Benelux Meeting Syst. Control, 2024, pp. 1-10.
[18] Y. Cheng, X. Hu, K. Chen, X. Yu, and Y. Luo, "Online longitudinal trajectory planning for connected and autonomous vehicles in mixed traffic flow with deep reinforcement learning approach," J. Transp. Eng. A: Syst., vol. 150, no. 7, Jul. 2024, doi: 10.1061/JTEPBS.TEENG-8286.
[19] H. Pei, J. Zhang, Y. Zhang, H. Xu, and L. Li, "Self-organized routing for autonomous vehicles via deep reinforcement learning," IEEE Trans. Veh. Technol., vol. 72, no. 4, pp. 4351-4363, Apr. 2023, doi: 10.1109/TVT.2022.3225808.
[20] F. Ye, S. Zhang, P. Wang, and C.-Y. Chan, "A survey of deep reinforcement learning algorithms for motion planning and control of autonomous vehicles," in Proc. IEEE Intell. Veh. Symp. (IV), Nagoya, Japan, 2021, pp. 1075-1080, doi: 10.1109/IV48863.2021.9575880.
[21] R. Cheng, G. Orosz, R. M. Murray, and J. W. Burdick, "End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks," arXiv preprint arXiv:1903.08792, 2019.
[22] H. Liu and L. Wang, "TD3 based collision free motion planning for robot navigation," arXiv preprint arXiv:2405.15460, 2024.
[23] T. Xu, Z. Meng, W. Chen, and Z. Li, "End-to-end autonomous driving decision method based on improved TD3 algorithm in complex scenarios," Sensors, vol. 24, no. 15, p. 4962, 2024, doi: 10.3390/s24154962.
[24] Z. Zhang, Y. Luo, Y. Chen, H. Zhao, Z. Ma, and H. Liu, "Automated parking trajectory generation using deep reinforcement learning," arXiv preprint arXiv:2504.21071, 2025.
[25] I. M. Anagnostara, E. Tsardoulias, and A. L. Symeonidis, "Deep reinforcement learning and imitation learning for autonomous parking simulation," Electronics, vol. 14, no. 10, p. 1992, 2025, doi: 10.3390/electronics14101992.
[26] K. Wahdan, N. Ehab, Y. Mansy, and A. El Mougy, "Dynamic path planning for autonomous vehicles using adaptive reinforcement learning TD3 model," in Proc. ICAART, Rome, Italy, 2024, pp. 272-279, doi: 10.5220/0012363300003636.
[27] D. Attard, "Reinforcement learning for autonomous navigation of articulated vehicles," M.Sc. Thesis, University of Malta, 2024.
[28] S. van Dam, L. Wisell, K. Shingade, M. Sadeghi Reineh, and F. Bruzelius, "Safe Control Allocation of Articulated Heavy Vehicles," in Proc. AVEC, LNME, 2024, doi: 10.1007/978-3-031-70392-8_1.
[29] J. Wang, L. Chu, Y. Zhang, Y. Mao, and C. Guo, "Intelligent Vehicle Decision-Making and Trajectory Planning Method Based on Deep Reinforcement Learning in the Frenet Space," Sensors, vol. 23, no. 5, p. 2463, 2023, doi: 10.3390/s23052463.
[30] "Mercedes-Benz Actros 2011 blueprint," Drawing Database, Feb. 2025. [Online]. Available: Drawing Database.
[31] I. Han, "Path Planning for Perpendicular Parking of Large Articulated Vehicles Based on Qualitative Kinematics and Geometric Methods," Autonomous Intelligent Systems, vol. 5, no. 3, p. 48, 2023, doi: 10.3390/vehicles503004.
 
 
مهندسی مکانیک مدرس
دوره 26، شماره 5
اردیبهشت 1405
صفحه 383-399