Hands and Bodies (Closing Session)

Summary: We present an algorithm for real-time hand tracking and online calibration from depth sensor data. Our method jointly estimates pose and shape in each frame, and determines the uncertainty in such estimates. This allows the algorithm to integrate per-frame estimates over time and build a personalized hand model.

Author(s): Anastasia Tkach, Swiss Federal Institute of Technology, Lausanne (EPFL)
Andrea Tagliasacchi, University of Victoria
Edoardo Remelli, Swiss Federal Institute of Technology, Lausanne (EPFL)
Mark Pauly, Swiss Federal Institute of Technology, Lausanne (EPFL)
Andrew Fitzgibbon, Microsoft Research Cambridge

Speaker(s): Anastasia Tkach, Ecole Polytechnique Fédérale de Lausanne

Summary: We demonstrate two handed tracking, recovering complex interactions, at extremely high framerates. The key contribution is an articulated distance function that, for any pose, yields a closed form calculation of both the distance to the detailed surface geometry and the necessary derivatives to perform gradient based optimization.

Author(s): Jonathan Taylor, perceptiveIO, Inc
Vladimir Tankovich, perceptiveIO, Inc
Danhang Tang, perceptiveIO, Inc
Cem Keskin, perceptiveIO, Inc
David Kim, perceptiveIO, Inc
Philip Davidson, perceptiveIO, Inc
Adarsh Kowdle, perceptiveIO, Inc
Shahram Izadi, perceptiveIO, Inc

Speaker(s): Jonathan Taylor, perceptiveIO

Summary: We learn a low-dimensional realistic model of hand shape and pose-dependent deformations from high-resolution hand scans of many people and poses. We combine it with a body model and then fit both the body and hand pose, as well as body shape, to noisy 4D scan sequences.

Author(s): Dimitrios Tzionas, Max Planck Institue for Intelligent Systems
Javier Romero, Body Labs, Max Planck Institue for Intelligent Systems
Michael Black, Body Labs, Max Planck Institue for Intelligent Systems

Speaker(s): Javier Romero, BodyLabs

Summary: We present Motion2Fusion, a state-of-the-art 360 performance capture system that enables *real-time* reconstruction of arbitrary non-rigid scenes, with improved geometric precision, tracking accuracy and efficiency than previous state-of-the-art techniques such as Fusion4D.

Author(s): Mingsong Dou, perceptiveIO, Inc
Sameh Khamis, perceptiveIO, Inc
Sean Fanello, perceptiveIO, Inc
Philip Davidson, perceptiveIO, Inc
Adarsh Kowdle, perceptiveIO, Inc
Christoph Rhemann, perceptiveIO, Inc
Vladimir Tankovich, perceptiveIO, Inc
Shahram Izadi, perceptiveIO, Inc

Speaker(s): Mingsong Dou, perceptiveIO