Optimization and Filtering for Human Motion Capture - A Multi-layer Framework
Juergen Gall, Bodo Rosenhahn, Thomas Brox, and Hans-Peter Seidel
Abstract
Local optimization and filtering have been widely applied to model-based 3D human motion capture. Global stochastic optimization has recently been proposed as promising alternative solution for tracking and initialization. In order to benefit from optimization and filtering, we introduce a multi-layer framework that combines stochastic optimization, filtering, and local optimization. While the first layer relies on interacting simulated annealing and some weak prior information on physical constraints, the second layer refines the estimates by filtering and local optimization such that the accuracy is increased and ambiguities are resolved over time without imposing restrictions on the dynamics. In our experimental evaluation, we demonstrate the significant improvements of the multi-layer framework and provide quantitative 3D pose tracking results for the complete HumanEva-II dataset. The paper further comprises a comparison of global stochastic optimization with particle filtering, annealed particle filtering, and local optimization.
Images/Video
a) Silhouette from background subtraction. b) Estimate from global optimization. c) Silhouette from level-set segmentation. d) Improved estimate by local optimization. The right and left arms are better estimated.
Video ~16MB (AVI)
Rendered video ~3MB (AVI)
Results for S4 of HumanEva-II (1258 frames).
Publications
Shaheen M., Gall J., Strzodka R., van Gool L., and Seidel H.-P., A Comparison of 3D Model-based Tracking Approaches for Human Motion Capture in Uncontrolled Environments ( PDF), IEEE Workshop on Applications of Computer Vision (WACV'09), 2009. © IEEE
Gall J., Rosenhahn B., Brox T., and Seidel H.-P., Optimization and Filtering for Human Motion Capture - A Multi-layer Framework (PDF), International Journal of Computer Vision, Special Issue on Evaluation of Articulated Human Motion and Pose Estimation, 2008. ©Springer-Verlag
Gall J., Rosenhahn B., and SeidelH.-P., An Introduction to Interacting Simulated Annealing ( PDF), Human Motion - Understanding, Modeling, Capture and Animation, Klette R., Metaxas D., and Rosenhahn B. (Eds.),Computational Imaging and Vision, Vol 36, 319-345, Springer, 2008. © Springer-Verlag
Gall J., Brox T., Rosenhahn B., and Seidel H.-P., Global Stochastic Optimization for Robust and Accurate Human Motion Capture (PDF), Technical Report MPI-I-2007-4-008, Department 4: Computer Graphics,Max-Planck Institute für Informatik, Saarbrücken, Germany, Dec. 2007.
Gall J., Rosenhahn B., and Seidel H.-P.,Clustered Stochastic Optimization for Object Recognition and Pose Estimation ( PDF), 29th Annual Symposium of the German Association for Pattern Recognition (DAGM'07), Springer, LNCS 4713, 32-41, 2007. © Springer-Verlag
Gall J., Potthoff J., Schnörr C., Rosenhahn B., and Seidel H.-P., Interacting and Annealing Particle Filters: Mathematics and a Recipe for Applications, Journal of Mathematical Imaging and Vision. Springer, 28(1), 1-18, 2007. © Springer-Verlag
Gall J., Potthoff J., Schnörr C., Rosenhahn B., and Seidel H.-P., Interacting and Annealing Particle Filters: Mathematics and a Recipe for Applications ( PDF), Technical Report MPI-I-2006-4-009, Department 4: Computer Graphics, Max-Planck Institute für Informatik, Saarbrücken, Germany, Sept. 2006. (A revised version of the technical report has been published in JMIV)