# A Novel Approach to Model Design and Tuning through Automatic Parameter Screening and Optimization - Theory and Application to a Helicopter Flight Simulator Case-study

### Matteo Hessel, Francesco Borgatelli, Fabio Ortalli

#### Abstract

The aim of this paper is to describe a novel methodology for model-design and tuning in computer simulations, based on automatic parameter screening and optimization. Simulation requires three steps: mathematical modelling, numerical solution, and tuning of the model’s parameters. We address Tuning because, at the state-of-the-art, the development of life-critical simulations requires months to appropriately tune the model. Our methodology can be split in Screening (identification of the relevant parameters to simulate a system) and Optimization (search of optimal values for those parameters). All techniques are fully general, because they leverage ideas from Machine-Learning and Optimization Theory to achieve their goals without directly analysing the simulator’s mathematical model. Concerning screening, we show how Machine-Learning algorithms, based on Neural Networks and Logistic Regression, can be used for ranking the parameters according to their relevance. Concerning optimization, we describe two algorithms: an adaptive hill-climbing procedure and a novel strategy, specific for model tuning, called sequential masking. Eventually, we show the performances achieved and the impact on the time and effort required for tuning a helicopter flight-simulator, proving that the proposed techniques can significantly speed-up the process.

#### References

- Fisher, R.A., 1935, The design of experiments. Oxford, England: Oliver & Boyd. xi 251 pp.
- Rosenblatt F., 1958, The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review 65: 386-408.
- Nelder J., Wedderburn R., 1972 Generalized Linear Models, Journal of the Royal Statistical Society. Series A (General) 135 (3): 370-384.
- Rumelhart D.E., Hinton G.E., Williams R.J., 1986, Learning representations by back-propagating errors. Nature 323 (6088): 533-536. doi:10.1038/323533a0.
- Cybenko G., 1989 Approximations by superpositions of sigmoidal functions, Mathematics of Control, Signals, and Systems, 2 (4), 303-314.
- Le Cessie S., Van Houwelingen J.C., 1992, Ridge estimators in Logistic Regression. Applied Statistics.
- Bettonvil B., Kleijnen J.P.C., 1997, Searching for important factors in simulation models with many factors: Sequential bifurcation, European Journal of Operational Research, Volume 96, Issue 1, Pages 180- 194.
- Haykin S., 1998, Neural Networks: A Comprehensive Foundation (2 ed.). Prentice Hall. ISBN 0-13-273350- 1.
- Harrel F., 2001 Regression Modeling Strategies, SpringerVerlag.
- Kern S., Muller S.D., Hansen N., Büche D., Ocenasek J., Koumoutsakos P., 2004, learning probability distributions in continuous evolutionary strategies - a comparative review, Journal of Natural Computing Volume 3 Issue 1, Pages 77 - 112.
- Bishop C., 2006 Pattern Recognition and Machine Learning, Springer Science+Business Media, LLC, pp 217-218.
- Hinton G.E., Osindero S., Yee-Whye The, 2006, A fast learning algorithm for deep belief nets, Neural Computation, 18(7):1527-1554.
- Morgan P.J., Cleave-Hogg D, Desousa S., LamMcCulloch J., 2006, Applying theory to practice in undergraduate education using high fidelity simulation, Med Teach, vol. 28, no. 1, pp. e10-e15.
- Zhang A., 2007 One-factor-at-a-time Screening Designs for Computer Experiments, SAE Technical Paper 2007-01-1660, doi:10.4271/2007-01-1660.
- Last M., Luta G., Orso A., Porter A., Young S., 2008, Pooled ANOVA, Computational Statistics & Data Analysis, Volume 52, Issue 12, Pages 5215-5228.
- Hall M., Eibe F., Holmes G., Pfahringer B., Reutemann P., Witten I., 2009 The WEKA Data Mining Software: An Update; SIGKDDExplorations, Volume11, Issue1.
- Lewis J. H., and Jiang S. B., 2009, A theoretical model for respiratory motion artifacts in free-breathing CT scans, Phys Med Biol, vol. 54, no. 3, pp. 745-755.
- Bergmeir C., Benìtez J.M., 2012, Neural Networks in R Using the Stuttgart Neural Network Simulator: RSNNS, Journal of Statistical Software, Volume 46, Issue 7.
- Vidal F.P., Villard P., Lutton E., 2013, Automatic tuning of respiratory model for patient-based simulation, MIBISOC'13 - International Conference on Medical Imaging using Bio-inspired and Soft Computing.

#### Paper Citation

#### in Harvard Style

Hessel M., Borgatelli F. and Ortalli F. (2014). **A Novel Approach to Model Design and Tuning through Automatic Parameter Screening and Optimization - Theory and Application to a Helicopter Flight Simulator Case-study** . In *Proceedings of the 4th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH,* ISBN 978-989-758-038-3, pages 24-35. DOI: 10.5220/0005022600240035

#### in Bibtex Style

@conference{simultech14,

author={Matteo Hessel and Francesco Borgatelli and Fabio Ortalli},

title={A Novel Approach to Model Design and Tuning through Automatic Parameter Screening and Optimization - Theory and Application to a Helicopter Flight Simulator Case-study},

booktitle={Proceedings of the 4th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH,},

year={2014},

pages={24-35},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0005022600240035},

isbn={978-989-758-038-3},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the 4th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH,

TI - A Novel Approach to Model Design and Tuning through Automatic Parameter Screening and Optimization - Theory and Application to a Helicopter Flight Simulator Case-study

SN - 978-989-758-038-3

AU - Hessel M.

AU - Borgatelli F.

AU - Ortalli F.

PY - 2014

SP - 24

EP - 35

DO - 10.5220/0005022600240035