A FORWARD-BACKWARD ALGORITHM FOR STOCHASTIC CONTROL PROBLEMS - Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming

Stephan E. Ludwig, Justin A. Sirignano, Ruojun Huang, George Papanicolaou

2012

Abstract

An algorithm for solving continuous-time stochastic optimal control problems is presented. The numerical scheme is based on the stochastic maximum principle (SMP) as an alternative to the widely studied dynamic programming principle (DDP). By using the SMP, (Peng, 1990) obtained a system of coupled forward-backward stochastic differential equations (FBSDE) with an external optimality condition. We extend the numerical scheme of (Delarue and Menozzi, 2006) by a Newton-Raphson method to solve the FBSDE system and the optimality condition simultaneously. As far as the authors are aware, this is the first fully explicit numerical scheme for the solution of optimal control problems through the solution of the corresponding extended FBSDE system. We discuss possible numerical advantages to the DDP approach and consider an optimal investment-consumption problem as an example.

References

  1. Davis, M. and Norman, A. (1990). Portfolio selection with transaction costs. In Mathematics of Operation Research, Vol. 15, No. 4. The lnstitute of Management Sciences/Operations Research Society of America.
  2. Delarue, F. and Menozzi, S. (2006). A forward-backward stochastic algorithm for quasi-linear pdes. In The Annals of Applied Probability, Vol. 16, No. 1, 140184. Universite Paris.
  3. Kushner, H. and Dupuis, P. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Springer, London, 1st edition.
  4. Nocedal, J. and Wright, S. (2006). Numerical Optimization. Springer Series in Operation Research and Financial Engineering, London, 2nd edition.
  5. Peng, S. (1990). A general stochastic maximum principle for optimal control problems. In Journal of Control and Optimization, Vol. 28, No. 4, pp.966-979. SIAM.
  6. Pham, H. (2009). Continuous-time stochastic control and optimization with financial applications. In Stochastic Modeling and Applied Probability 61. SpringerVerlag Berlin Heidelberg.
Download


Paper Citation


in Harvard Style

E. Ludwig S., A. Sirignano J., Huang R. and Papanicolaou G. (2012). A FORWARD-BACKWARD ALGORITHM FOR STOCHASTIC CONTROL PROBLEMS - Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming . In Proceedings of the 1st International Conference on Operations Research and Enterprise Systems - Volume 1: ICORES, ISBN 978-989-8425-97-3, pages 83-89. DOI: 10.5220/0003885900830089


in Bibtex Style

@conference{icores12,
author={Stephan E. Ludwig and Justin A. Sirignano and Ruojun Huang and George Papanicolaou},
title={A FORWARD-BACKWARD ALGORITHM FOR STOCHASTIC CONTROL PROBLEMS - Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming},
booktitle={Proceedings of the 1st International Conference on Operations Research and Enterprise Systems - Volume 1: ICORES,},
year={2012},
pages={83-89},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003885900830089},
isbn={978-989-8425-97-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Conference on Operations Research and Enterprise Systems - Volume 1: ICORES,
TI - A FORWARD-BACKWARD ALGORITHM FOR STOCHASTIC CONTROL PROBLEMS - Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming
SN - 978-989-8425-97-3
AU - E. Ludwig S.
AU - A. Sirignano J.
AU - Huang R.
AU - Papanicolaou G.
PY - 2012
SP - 83
EP - 89
DO - 10.5220/0003885900830089