Abstract

A real-time full-body motion capture system is presented which uses input from a sparse set of inertial measurement units (IMUs) along with images from two or more standard video cameras and requires no optical markers or specialized infra-red cameras. A real-time optimization-based framework is proposed which incorporates constraints from the IMUs, cameras and a prior pose model. The combination of video and IMU data allows the full 6-DOF motion to be recovered including axial rotation of limbs and drift-free global position. The approach was tested using both indoor and outdoor captured data. The results demonstrate the effectiveness of the approach for tracking a wide range of human motion in real time in unconstrained indoor/outdoor scenes.

Paper

Real-time Full-Body Motion Capture from Video and IMUs
Charles Malleson, Marco Volino, Andrew Gilbert, Matthew Trumble,
John Collomosse and Adrian Hilton
International Conference on 3D Vision (3DV) 2017



Citation

    @inproceedings{Malleson:3DV:2017,
        AUTHOR = "Malleson, Charles and Volino, Marco and Gilbert, Andrew and 
        Trumble, Matthew and Collomosse, John and Hilton, Adrian",
        TITLE = "Real-time Full-Body Motion Capture from Video and IMUs",
        BOOKTITLE = "2017 Fifth International Conference on 3D Vision (3DV)",
        YEAR = "2017",
    }

Data

Data used in this work can be found in the Total Capture Dataset.

Demo

A live demonstration of this work was presented at CVMP 2017.

Acknowledgments

This work was supported by the Innovate UK Total Capture project (grant 102685) and the EU H2020 Visual Media project (grant 687800). We wish to thank Anna Korzeniowska, Evren Imre, Joao Regateiro and Armin Mustafa for their help with data capture.