Reach-avoid semi-Markov decision processes with time-varying obstacles

Abstract

We consider the maximal reach-avoid probability to a target in finite horizon for semi-Markov decision processes with time-varying obstacles. Since the variance of the obstacle set, the model \eqref{Model} is non-homogeneous. To overcome such difficulty, we construct a related two-dimensional model \eqref{newModel}, and then prove the equivalence between such reach-avoid probability of the original model and that of the related two-dimensional one. For the related two-dimensional model, we analyze some special characteristics of the equivalent reach-avoid probability. On this basis, we provide a special improved value-type algorithm to obtain the equivalent maximal reach-avoid probability and its $\epsilon$-optimal policy. Then, at the last step of the algorithm, by the equivalence between these two models, we obtain the original maximal reach-avoid probability and its $\epsilon$-optimal policy for the original model.