Multi-Person Pose Estimation with Local Joint-to-Person Associations

Umar Iqbal        Juergen Gall


Despite of the recent success of neural networks for human pose estimation, current approaches are limited to pose estimation of a single person and cannot handle humans in groups or crowds. In this work, we propose a method that estimates the poses of multiple persons in an image in which a person can be occluded by another person or might be truncated. To this end, we consider multiperson pose estimation as a joint-to-person association problem. We construct a fully connected graph from a set of detected joint candidates in an image and resolve the joint-to person association and outlier detection using integer linear programming. Since solving joint-to-person association jointly for all persons in an image is an NP-hard problem and even approximations are expensive, we solve the problem locally for each person. On the challenging MPII Human Pose Dataset for multiple persons, our approach achieves the accuracy of a state-of-the-art method, but it is 6,000 to 19,000 times faster.


Overview: Overview of the proposed method. We detect the persons using a person detector (a). A set of joint detection candidates is generated for each person (b). To perform joint-to-person association and to suppress false detections, candidates for each person are formulated into a fully connected graph (c). These graphs are then solved locally for each person using integer linear programming to obtain final pose estimates (d).


Umar Iqbal, Juergen Gall
Multi-Person Pose Estimation with Local Joint-to-Person Associations

European Conference on Computer Vision (ECCV) Workshops, Crowd Understanding (CUW), 2016, Amsterdam, Netherlands.
[PDF] [Poster]

Source Code

Source code will be available soon.