In order to apply reinforcement learning to a robot in a real environment, it is necessary to use simulation in a virtual environment because numerous iterative learning is required. In addition, it is difficult to apply a learning algorithm that requires a lot of computation for a robot with low-spec. hardware. In this study, ML-Agent, a reinforcement learning frame provided by Unity, was used as a virtual simulation environment to apply reinforcement learning to the obstacle collision avoidance problem of mobile robots with low-spec hardware. A DQN supported by ML-Agent is adopted as a reinforcement learning algorithm and the results for a real robot show that the number of collisions occurred less then 2 times per minute.
D.W.Lee, K.M.cho and S.H.Lee, “Comparison &Analysis of Drones in Major Countries based on Self-Driving in IoT Environment,” Journal of The Korea Internet of Things Society, Vol.6, No.2, pp.31-36, 2020.
D. Filliat and J.A.Meyer, “Map-based navigation in mobile robots: I. A review of localization strategies,”Cognitive Systems Research, Vol.4, No.4, pp.243-282, 2003.
J.A. Meyer and D. Filliat, “Map-based navigation in mobile robots: II. A review of map-learning and path-planning strategies,” Cognitive Systems Research, Vol.4, No. 4, pp. 283-317, 2003.
R.S.Sutton and A.G.Barto, “Reinforcement Learning:An Introduction,” A Bradford Book, MIT Press, 2th ed., 2017.
A.E.Sallab, M.Abdou, E.Perot and S.Yogamani, “Deep reinforcement learning framework for autonomous driving,” Journal of imaging Science and Technology, Vol.1, No.7, pp.70-76, 2017.
X.B.Peng, G.Berseth, K.Yin and M.V.Panne, “Deeploco:Dynamic locomotion skills using hierarchical deep reinforcement learning,” ACM Transactions on Graphics, Vol.36, No.41 pp.1-13, 2017.
J.H.Woo and N.K.Kim, “Collision Avoidance for an Unmanned Surface Vehicle Using Deep Reinforcement Learning,” Graduate School of Seoul National University, Doctoral Dissertation, 2018.
A.Coates, P.Abbeel and A.Y.Ng, “Apprenticeship learning for helicopter control,” Communications of the ACM, Vol.52, No.7, pp.97-105, 2009.
S.Y.Park, “Object-spatial layout-route-based hybrid nap and its application to mobile robot navigation,”Graduate School of Yonsei University, Doctoral Dissertation, 2010.
N.J.Cho, “Learning, improving, and generalizing motor skills for autonomous robot manipulation : an integration of imitation learning, reinforcement learning, and deep learning,” Graduate School of Hanyang University, Doctoral Dissertation, 2020.
B.G.Ahn, “An Adaptive Motion Learning Architecture for Mobile Robots,” Graduate school of SungKyunKwan University, Master's Thesis, 2006.
https://github.com/Unity-Technologies/ml-agents
A.B.Juliani, E.Teng, A.Cohen, J.Harper, C.Elion, C.Goy, Y.Gao, H.Henry, M.Mattar and D.Lange, “Unity: A General Platform for Intelligent Agents,”arXiv:1809.02627, 2020.
J.C.H.Watkins, D.Peter, “Q-learning,” Machine Learning, Vol.8, No.1, pp.272-292, 1992.
X.Chen, “A Reinforcement Learning Method of Obstacle Avoidance for Industrial Mobile Vehicles in Unknown Environments Using Neural Network,”Proceedings of the 21st International Conference on Industrial Engineering and Engineering Management, Vol.1, No.1, pp.671-67, 2014.