EMBEDDED INCREMENTAL FEATURE SELECTION FOR REINFORCEMENT LEARNING

Robert Wright, Steven Loscalzo, Lei Yu

2011

Abstract

Classical reinforcement learning techniques become impractical in domains with large complex state spaces. The size of a domain’s state space is dominated by the number of features used to describe the state. Fortunately, in many real-world environments learning an effective policy does not usually require all the provided features. In this paper we present a feature selection algorithm for reinforcement learning called Incremental Feature Selection Embedded in NEAT (IFSE-NEAT) that incorporates sequential forward search into neuroevolutionary algorithm NEAT. We provide an empirical analysis on a realistic simulated domain with many irrelevant and relevant features. Our results demonstrate that IFSE-NEAT selects smaller and more effective feature sets than alternative approaches, NEAT and FS-NEAT, and superior performance characteristics as the number of available features increases.

References

  1. Guyon, I. and Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3:1157-1182.
  2. James, D. and Tucker, P. (2004). A comparative analysis of simplification and complexification in the evolution of neural network topologies. In Proceedings of the 2004 Conference on Genetic and Evoluationary Computation (GECCO-04). Springer.
  3. Keller, P., Mannor, S., and Precup, D. (2006). Automatic basis function construction for approximate dynamic programming and reinforcement learning. In Proceedings of the the 23rd International Conference on Machine Learning, pages 449-456.
  4. Kolter, J. Z. and Ng, A. Y. (2009). Regularization and feature selection in least-squares temporal difference learning. In Proceedings of the 26th International Conference on Machine Learning (ICML09), pages 521-528.
  5. Kroon, M. and Whiteson, S. (2009). Automatic feature selection for model-based reinforcement learning in factored mdps. In Proceedings of the International Conference on Machine Learning and Applications, pages 324-330.
  6. Liu, H. and Yu, L. (2005). Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering (TKDE), 17(4):491-502.
  7. Parr, R., Li, L., Taylor, G., Painter-Wakefield, C., and Littman, M. L. (2008). An analysis of linear models, linear value-function approximation, and feature selection for reinforcement learning. In Proceedings of the 25th International Conference on Machine Learning (ICML08), pages 752-759.
  8. Stanley, K. O. and Miikkulainen, R. (2002). Efficient reinforcement learning through evolving neural network topologies. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 02), pages 569-577.
  9. Sutton, R. and Barto, A. (1998). Reinforcement Learning: An Introduction. MIT Press.
  10. Whiteson, S. Taylor, M. E. and Stone, P. (2007). Adaptive tile coding for value function approximation. Technical report, University of Texas at Austin.
  11. Whiteson, S. and Stone, P. (2006). Evolutionary function approximation for reinforcement learning. Journal of Machine Learing Research, 7:877-917.
  12. Whiteson, S., Stone, P., and Stanley, K. O. (2005). Automatic feature selection in neuroevolution. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 05), pages 1225-1232.
Download


Paper Citation


in Harvard Style

Wright R., Loscalzo S. and Yu L. (2011). EMBEDDED INCREMENTAL FEATURE SELECTION FOR REINFORCEMENT LEARNING . In Proceedings of the 3rd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-8425-40-9, pages 263-268. DOI: 10.5220/0003153402630268


in Bibtex Style

@conference{icaart11,
author={Robert Wright and Steven Loscalzo and Lei Yu},
title={EMBEDDED INCREMENTAL FEATURE SELECTION FOR REINFORCEMENT LEARNING},
booktitle={Proceedings of the 3rd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2011},
pages={263-268},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003153402630268},
isbn={978-989-8425-40-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - EMBEDDED INCREMENTAL FEATURE SELECTION FOR REINFORCEMENT LEARNING
SN - 978-989-8425-40-9
AU - Wright R.
AU - Loscalzo S.
AU - Yu L.
PY - 2011
SP - 263
EP - 268
DO - 10.5220/0003153402630268