On the use of MDP models task allocation in persistent robotic security service

Abstract

Interest for Unmanned Aerial Vehicles (UAVs) are increasing these days. There are various applications of UAVs. Among them this study`s application area is security system of UAVs. Customer arrived station and request service. Based on expected demand, UAVs are allocated and serve security service to customer. Under stochastic environment with fixed design system, we decide UAVs optimal action at each circumstance. We represented UAVs system via Markov Decision Process (MDP) model, and solved MDP model by value iteration and policy iteration. We find optimal solution at various circumstance and analyze optimal solution. We verify optimal solution via simulation. Through the optimal policy we can generate optimal path for each UAV. Therefore we can efficiently operate UAVs in stochastic environment. Since we modeled this problem in stochastic environment, there are lots of states even though it has small number of UAVs. Also there are some limitation for solving MDP due to the complexity of the problem. In order to treat more complex case, we develop heuristic and verify it through the simulation