EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers

Fully immersive experiences in AR/VR depend on re-constructing the full body pose of the user without restricting their motion. In this paper we study the use of body-worn electromagnetic (EM) field-based sensing for the task of 3D human pose reconstruction. To this end, we present a method to estim...

Full description

Saved in:
Bibliographic Details
Main Authors: Kaufmann, Manuel, Zhao, Yi, Tang, Chengcheng, Tao, Lingling, Twigg, Christopher, Song, Jie, Wang, Robert, Hilliges, Otmar
Format: Conference Proceeding
Language:English
Subjects:
3D from multiview and other sensors
Gestures and body pose
Motion and tracking
Sensor systems
Shape
Shape measurement
Stereo
Three-dimensional displays
Tracking
Wireless communication
Wireless sensor networks
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Fully immersive experiences in AR/VR depend on re-constructing the full body pose of the user without restricting their motion. In this paper we study the use of body-worn electromagnetic (EM) field-based sensing for the task of 3D human pose reconstruction. To this end, we present a method to estimate SMPL parameters from 6-12 EM sensors. We leverage a customized wearable system consisting of wireless EM sensors measuring time-synchronized 6D poses at 120 Hz. To provide accurate poses even with little user instrumentation, we adopt a recently proposed hybrid framework, learned gradient descent (LGD), to iteratively estimate SMPL pose and shape from our input measurements. This allows us to harness powerful pose priors to cope with the idiosyncrasies of the input data and achieve accurate pose estimates. The proposed method uses AMASS to synthesize virtual EM-sensor data and we show that it generalizes well to a newly captured real dataset consisting of a total of 36 minutes of motion from 5 subjects. We achieve reconstruction errors as low as 31.8 mm and 13.3 degrees, outperforming both pure learning- and pure optimization-based methods. Code and data is available under https://ait.ethz.ch/projects/2021/em-pose.
ISSN:2380-7504
DOI:10.1109/ICCV48922.2021.01131