Online Action Learning using Kernel Density Estimation for Quick Discovery of Good Parameters for Peg-in-Hole Insertion

Lars Carøe Sørensen, Jacob Pørksen Buch, Henrik Gordon Petersen, Dirk Kraft

2016

Abstract

Learning action parameters is becoming an ever more important topic in industrial assembly with tendencies towards smaller batch sizes, more required flexibility and process uncertainties. This paper presents a statistical online learning method capable of handling these issues. The method uses elimination of unpromising parameter sets to reduce the elements of the discretised sample space (inspired by Action Elimination) based on regression uncertainty. Kernel Density Estimation and Wilson Score are explored as internal representations. Based on a dynamic simulator setup for a real world Peg-in-Hole problem, it is shown that the presented method can drastically reduce the number of samples needed. Furthermore, it is also shown that the solution obtained in simulation by our learning method succeeds when executed on the corresponding real world setup.

References

  1. Agresti, A. and Coull, B. A. (1998). Approximate Is Better than ”Exact” for Interval Estimation of Binomial Proportions. The American Statistician, 52(2):119-126.
  2. Auer, P., Cesa-Bianchi, N., and Fischer, P. (2002). Finitetime analysis of the multiarmed bandit problem. Mach. Learn., 47(2-3):235-256.
  3. Bodenhagen, L., Fugl, A., Jordt, A., Willatzen, M., Andersen, K., Olsen, M., Koch, R., Petersen, H., and Kruger, N. (2014). An adaptable robot vision system performing manipulation actions with flexible objects. Automation Science and Engineering, IEEE Transactions on, 11(3):749-765.
  4. Brochu, E., Cora, V. M., and de Freitas, N. (2010). A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. CoRR, abs/1012.2599.
  5. Buch, J., Laursen, J., Sørensen, L., Ellekilde, L.-P., Kraft, D., Schultz, U., and Petersen, H. (2014). Applying simulation and a domain-specific language for an adaptive action library. In Simulation, Modeling, and Programming for Autonomous Robots, pages 86-97. Springer International Publishing.
  6. Deisenroth, M. P., Neumann, G., and Peters, J. (2011). A survey on policy search for robotics. Foundations and Trends in Robotics, 2(1-2):1-142.
  7. Detry, R., Kraft, D., Kroemer, O., Bodenhagen, L., Peters, J., Krüger, N., and Piater, J. (2011). Learning grasp affordance densities. Paladyn, 2(1):1-17.
  8. EU Robotics aisbl (2014). Robotics 2020 multi-annual roadmap for robotics in europe.
  9. Even-Dar, E., Mannor, S., and Mansour, Y. (2006). Action elimination and stopping conditions for the multiarmed bandit and reinforcement learning problems. The Journal of Machine Learning Research, 7:1079- 1105.
  10. Gams, A., Petric, T., Nemec, B., and Ude, A. (2014). Learning and adaptation of periodic motion primitives based on force feedback and human coaching interaction. In Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on, pages 166-171.
  11. Härdle, W., Werwatz, A., Müller, M., and Sperlich, S. (2004). Nonparametric and semiparametric models. Springer Berlin Heidelberg.
  12. Heidrich-Meisner, V. and Igel, C. (2009). Hoeffding and bernstein races for selecting policies in evolutionary direct policy search. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML 7809, pages 401-408. ACM.
  13. Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pastor, P., and Schaal, S. (2012). Dynamical movement primitives: Learning attractor models for motor behaviors. Neural Computation, 25(2):328-373.
  14. Jørgensen, T. B., Debrabant, K., and Kr üger, N. (2016). Robust optimizing of robotic pick and place operations for deformable objects through simulation. In Robotics and Automation (ICRA), 2016 IEEE International Conference on. (accepted).
  15. Li, B., Chen, H., and Jin, T. (2014). Industrial robotic assembly process modeling using support vector regression. In Intelligent Robots and Systems (IROS 2014), 2014 IEEE/RSJ International Conference on, pages 4334-4339.
  16. Park, D., Kapusta, A., Kim, Y. K., Rehg, J., and Kemp, C. (2014). Learning to reach into the unknown: Selecting initial conditions when reaching in clutter. In Intelligent Robots and Systems (IROS 2014), 2014 IEEE/RSJ International Conference on, pages 630- 637.
  17. Robotics VO (2013). A roadmap for U.S. robotics from internet to robotics.
  18. Ross, S. M. (2009). Introduction to Probability and Statistics for Engineers and Scientists. Acedemic Press, 4th edition.
  19. Silverman, B. W. (1986). Density estimation for statistics and data analysis, volume 26. Chapman & Hall/CRC press.
  20. Sutton, R. S. and Barto, A. G. (1998). Reinforcement learning: An introduction, volume 28. MIT press.
  21. Tesch, M., Schneider, J. G., and Choset, H. (2013). Expensive function optimization with stochastic binary outcomes. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16-21 June 2013, pages 1283-1291.
  22. Thulesen, T. N. and Petersen, H. G. (2016). RobWorkPhysicsEngine: A new dynamic simulation engine for manipulation actions. In Robotics and Automation (ICRA), 2016 IEEE International Conference on. (accepted).
  23. Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3):229-256.
  24. Yang, Y., Lin, L., Song, Y., Nemec, B., Ude, A., Buch, A., Krüger, N., and Savarimuthu, T. (2015). Fast programming of peg-in-hole actions by human demonstration, pages 990-995. IEEE.
Download


Paper Citation


in Harvard Style

Sørensen L., Buch J., Petersen H. and Kraft D. (2016). Online Action Learning using Kernel Density Estimation for Quick Discovery of Good Parameters for Peg-in-Hole Insertion . In Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO, ISBN 978-989-758-198-4, pages 166-177. DOI: 10.5220/0005958801660177


in Bibtex Style

@conference{icinco16,
author={Lars Carøe Sørensen and Jacob Pørksen Buch and Henrik Gordon Petersen and Dirk Kraft},
title={Online Action Learning using Kernel Density Estimation for Quick Discovery of Good Parameters for Peg-in-Hole Insertion},
booktitle={Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,},
year={2016},
pages={166-177},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005958801660177},
isbn={978-989-758-198-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,
TI - Online Action Learning using Kernel Density Estimation for Quick Discovery of Good Parameters for Peg-in-Hole Insertion
SN - 978-989-758-198-4
AU - Sørensen L.
AU - Buch J.
AU - Petersen H.
AU - Kraft D.
PY - 2016
SP - 166
EP - 177
DO - 10.5220/0005958801660177