Accelerating Interval Iteration for Expected Rewards in Markov Decision Processes

Mohammadsadegh Mohagheghi, Khayyam Salehi

2020

Abstract

Reachability probabilities and expected rewards are two important classes of properties that are computed in probabilistic model checking. Iterative numerical methods are used to compute these properties. Interval iteration and sound value iteration are proposed in recent years to guarantee the precision of computed values. These methods consider upper and lower bounds of values and update each bound in every iteration until satisfying the convergence criterion. In this paper, we focus on the computation of the expected rewards of models and propose two heuristics to improve the performance of the interval iteration method. The first heuristic updates the upper and lower bounds separately to avoid redundant updates. The second heuristic uses the computed values of the lower bound to approximate a starting point for the upper bound. We also propose a criterion for the correctness of the approximated upper bound. The experiments show that in most cases, interval iteration with our approaches outperforms the standard interval iteration and sound value iteration methods.

Paper Citation

in Harvard Style

Mohagheghi M. and Salehi K. (2020). Accelerating Interval Iteration for Expected Rewards in Markov Decision Processes.In Proceedings of the 15th International Conference on Software Technologies - Volume 1: ICSOFT, ISBN 978-989-758-443-5, pages 39-50. DOI: 10.5220/0009833700390050

in Bibtex Style

@conference{icsoft20,
author={Mohammadsadegh Mohagheghi and Khayyam Salehi},
title={Accelerating Interval Iteration for Expected Rewards in Markov Decision Processes},
booktitle={Proceedings of the 15th International Conference on Software Technologies - Volume 1: ICSOFT,},
year={2020},
pages={39-50},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009833700390050},
isbn={978-989-758-443-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Software Technologies - Volume 1: ICSOFT,
TI - Accelerating Interval Iteration for Expected Rewards in Markov Decision Processes
SN - 978-989-758-443-5
AU - Mohagheghi M.
AU - Salehi K.
PY - 2020
SP - 39
EP - 50
DO - 10.5220/0009833700390050