Path-planning with minimum probability of detection for AUVs using reinforcement learning

Abstract

Path planning is a critical function for autonomous vehicles. In military applications, the path planning algorithms must also be designed such that the vehicle is not detected. The stealth is even more important for the underwater vehicles. Detection of an underwater vehicle can be effected from various parameters. In this study, the relationship between these parameters and the resulting signal-to-noise ratio are modeled using sonar equations. Then, the probability of detection is calculated using the signal-to-noise ratio. A Q-learning based path planning approach is proposed where the rewards are calculated using the detection probabilities. The agent then chooses actions which minimize the probability of being detection along the whole planned path. Once trained and optimal policy is reached, the proposed algorithm yields more secure paths than the probabilistic roadmap method. Since it provides an optimal action per state, it is also more flexible in case the vehicle is drifted. The results show that the probability of being detected in the test scenario is 5% in average. © 2022 Elsevier B.V., All rights reserved.