Efficient Resource Allocation for Sparse Multiple Object Tracking

Rui Figueiredo, João Avelino, Atabak Dehban, Alexandre Bernardino, Pedro Lima, Helder Araújo


In this work we address the multiple person tracking problem with resource constraints, which plays a fundamental role in the deployment of efficient mobile robots for real-time applications involved in Human Robot Interaction. We pose the multiple target tracking as a selective attention problem in which the perceptual agent tries to optimize the overall expected tracking accuracy. More specifically, we propose a resource constrained Partially Observable Markov Decision Process (POMDP) formulation that allows for real-time on-line planning. Using a transition model, we predict the true state from the current belief for a finite-horizon, and take actions to maximize future expected belief-dependent rewards. These rewards are based on the anticipated observation qualities, which are provided by an observation model that accounts for detection errors due to the discrete nature of a state-of-the-art pedestrian detector. Finally, a Monte Carlo Tree Search method is employed to solve the planning problem in real-time. The experiments show that directing the attentional focci to relevant image sub-regions allows for large detection speed-ups and improvements on tracking precision.


  1. Ahmad, S. and Yu, A. J. (2013). Active sensing as bayes-optimal sequential decision making. CoRR, abs/1305.6650.
  2. Araya, M., Buffet, O., Thomas, V., and Charpillet, F. (2010). A pomdp extension with belief-dependent rewards. In Advances in Neural Information Processing Systems, pages 64-72.
  3. Auer, P., Cesa-Bianchi, N., and Fischer, P. (2002). Finitetime analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256.
  4. Bernardin, K. and Stiefelhagen, R. (2008). Evaluating multiple object tracking performance: The clear mot metrics. EURASIP Journal on Image and Video Processing, 2008(1):1-10.
  5. Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. (2016). Simple online and realtime tracking. CoRR, abs/1602.00763.
  6. Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., and Colton, S. (2012). A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in Games, 4(1):1-43.
  7. Burkard, R., Dell'Amico, M., and Martello, S. (2009). Assignment Problems. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA.
  8. Butko, N. J. and Movellan, J. R. (2010). Infomax control of eye movements. Autonomous Mental Development, IEEE Transactions on, 2(2):91-107.
  9. Chong, E. K., Kreucher, C. M., and Hero, A. O. (2008). Monte-carlo-based partially observable markov decision process approximations for adaptive sensing. In Discrete Event Systems, 2008. WODES 2008. 9th International Workshop on, pages 173-180. IEEE.
  10. Dollár, P., Appel, R., Belongie, S., and Perona, P. (2014). Fast feature pyramids for object detection. PAMI.
  11. Gedikli, S., Bandouch, J., von Hoyningen-Huene, N., Kirchlechner, B., and Beetz, M. (2007). An adaptive vision system for tracking soccer players from variable camera settings. In Proceedings of the 5th International Conference on Computer Vision Systems (ICVS).
  12. Gelly, S., Kocsis, L., Schoenauer, M., Sebag, M., Silver, D., Szepesvári, C., and Teytaud, O. (2012). The grand challenge of computer go: Monte carlo tree search and extensions. Communications of the ACM, 55(3):106- 113.
  13. Hamid Rezatofighi, S., Milan, A., Zhang, Z., Shi, Q., Dick, A., and Reid, I. (2015). Joint probabilistic data association revisited. In Proceedings of the IEEE International Conference on Computer Vision, pages 3047- 3055.
  14. Kocsis, L. and Szepesvári, C. (2006). Bandit based montecarlo planning. In European conference on machine learning, pages 282-293. Springer.
  15. Leal-Taixé, L., Milan, A., Reid, I., Roth, S., and Schindler, K. (2015). MOTChallenge 2015: Towards a benchmark for multi-target tracking. arXiv:1504.01942 [cs]. arXiv: 1504.01942.
  16. Malloy, M. L. and Nowak, R. D. (2014). Near-optimal adaptive compressed sensing. IEEE Transactions on Information Theory, 60(7):4001-4012.
  17. Mihaylova, L., Lefebvre, T., Bruyninckx, H., Gadeyne, K., and Schutter, J. D. (2002). Active sensing for robotics - a survey. In in Proc. 5 th Intl Conf. On Numerical Methods and Applications, pages 316-324.
  18. Okuma, K., Taleghani, A., De Freitas, N., Little, J. J., and Lowe, D. G. (2004). A boosted particle filter: Multitarget detection and tracking. In European Conference on Computer Vision, pages 28-39. Springer.
  19. Ross, S., Pineau, J., Paquet, S., and Chaib-Draa, B. (2008). Online planning algorithms for pomdps. Journal of Artificial Intelligence Research, 32:663-704.
  20. Sommerlade, E. and Reid, I. (2010). Probabilistic surveillance with multiple active cameras. In Robotics and Automation (ICRA), 2010 IEEE International Conference on, pages 440-445. IEEE.
  21. Spaan, M. T., Veiga, T. S., and Lima, P. U. (2015). Decision-theoretic planning under uncertainty with information rewards for active cooperative perception. Autonomous Agents and Multi-Agent Systems, 29(6):1157-1185.
  22. Van Rooij, I. (2008). The tractable cognition thesis. Cognitive science, 32(6):939-984.
  23. Wang, J. R. and Parameswaran, N. (2004). Survey of sports video analysis: research issues and applications. In Proceedings of the Pan-Sydney area workshop on Visual information processing, pages 87-90. Australian Computer Society, Inc.
  24. Xu, Y. and Chun, M. M. (2009). Selecting and perceiving multiple visual objects. Trends in cognitive sciences, 13(4):167-174.
  25. Zitnick, C. L. and Dollár, P. (2014). Edge boxes: Locating object proposals from edges. In ECCV.

Paper Citation

in Harvard Style

Figueiredo R., Avelino J., Dehban A., Bernardino A., Lima P. and Araújo H. (2017). Efficient Resource Allocation for Sparse Multiple Object Tracking . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-227-1, pages 300-307. DOI: 10.5220/0006173103000307

in Bibtex Style

author={Rui Figueiredo and João Avelino and Atabak Dehban and Alexandre Bernardino and Pedro Lima and Helder Araújo},
title={Efficient Resource Allocation for Sparse Multiple Object Tracking},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)},

in EndNote Style

JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)
TI - Efficient Resource Allocation for Sparse Multiple Object Tracking
SN - 978-989-758-227-1
AU - Figueiredo R.
AU - Avelino J.
AU - Dehban A.
AU - Bernardino A.
AU - Lima P.
AU - Araújo H.
PY - 2017
SP - 300
EP - 307
DO - 10.5220/0006173103000307