# BELIEF PROPAGATION IN SPATIOTEMPORAL GRAPH TOPOLOGIES FOR THE ANALYSIS OF IMAGE SEQUENCES

### Volker Willert, Julian Eggert

#### Abstract

Belief Propagation (BP) is an efficient approximate inference technique both for Markov Random Fields (MRF) and Dynamic Bayesian Networks (DBN). 2DMRFs provide a unified framework for early vision problems that are based on static image observations. 3D MRFs are suggested to cope with dynamic image data. To the contrary, DBNs are far less used for dynamic low level vision problems even though they represent sequences of state variables and hence are suitable to process image sequences with temporally changing visual information. In this paper, we propose a 3D DBN topology for dynamic visual processing with a product of potentials as transition probabilities. We derive an efficient update rule for this 3D DBN topology that unrolls loopy BP for a 2D MRF over time and compare it to update rules for conventional 3D MRF topologies. The advantages of the 3D DBN are discussed in terms of memory consumptions, costs, convergence and online applicability. To evaluate the performance of infering visual information from dynamic visual observations, we show examples for image sequence denoising that achieve MRF-like accuracy on real world data.

#### References

- Bishop, C. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media.
- Brand,M. (1997). Coupled hidden markov models for complex action recognition. In Proc. of IEEE Conf. on CVPR.
- Chen, J. and Tang, C. (2007). Spatio-temporal markov random field for video denoising. In Proc. of IEEE Conf. on CVPR.
- Felzenszwalb, P. and Huttenlocher, D. (2006). Efficient belief propagation for early vision. Int. J. Comput. Vision, 70:4154.
- Huang, R., Pavlovic, V., and Metaxas, D. (2008). A new spatio-temporal mrf framework for video-based object segmentation. In Proc. of 1st Int. Workshop on Machine Learning for Vision-based Motion Analysis.
- Komodakis, N. and Paragios, N. (2009). Pairwise energies: Efficient optimization for higher-order mrfs. In Proc. of IEEE Conf. on CVPR.
- Larsen, E., Mordohai, P., Pollefeys, M., and Fuchs, H. (2007). Temporally consistent reconstruction from multiple video streams using enhanced belief propagation. In Proc. of IEEE Conf. on CVPR.
- Murphy, K. and Weiss, Y. (2001). The factored frontier algorithm for approximate inference in dbns. In Proc. of 17th Conf. on Uncertainty in Artificial Intelligence.
- Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann.
- Petersen, K., Fehr, J., and Burkhardt, H. (2008). Fast generalized belief propagation formap estimation on 2d and 3d grid-like markov random fields. In Proc. of 30th Conf. of the German Association for Pattern Recognition.
- Ramalingam, S., Kohli, P., Alahari, K., and Torr, P. (2008). Exact inference in multi-label crfs with higher order cliques. In Proc. of IEEE Conf. on CVPR.
- Roth, S. (2007). High-Order Markov Random Fields for Low-Level Vision. PhD thesis, Brown University.
- Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., and Rother, C. (2008). A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Trans. on PAMI, 30:10681080.
- Tappen, M. and Freeman, W. (2003). Comparison of graph cuts with belief propagation for stereo, using identical mrf parameters. In Proc. of IEEE ICCV.
- Williams, O., Isard, M., and MacCormick, J. (2005). Estimating disparity and occlusions in stereo video sequences. In Proc. of IEEE Conf. on CVPR.
- Yin, Z. and Collins, R. (2007). Belief propagation in a 3d spatio-temporal mrf for moving object detection. In Proc. of IEEE Conf. on CVPR.

#### Paper Citation

#### in Harvard Style

Willert V. and Eggert J. (2010). **BELIEF PROPAGATION IN SPATIOTEMPORAL GRAPH TOPOLOGIES FOR THE ANALYSIS OF IMAGE SEQUENCES** . In *Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2010)* ISBN 978-989-674-029-0, pages 117-124. DOI: 10.5220/0002818501170124

#### in Bibtex Style

@conference{visapp10,

author={Volker Willert and Julian Eggert},

title={BELIEF PROPAGATION IN SPATIOTEMPORAL GRAPH TOPOLOGIES FOR THE ANALYSIS OF IMAGE SEQUENCES},

booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2010)},

year={2010},

pages={117-124},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0002818501170124},

isbn={978-989-674-029-0},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2010)

TI - BELIEF PROPAGATION IN SPATIOTEMPORAL GRAPH TOPOLOGIES FOR THE ANALYSIS OF IMAGE SEQUENCES

SN - 978-989-674-029-0

AU - Willert V.

AU - Eggert J.

PY - 2010

SP - 117

EP - 124

DO - 10.5220/0002818501170124