Real-time posture reconstruction for Microsoft Kinect

Hubert P.H. Shum, Shu Lim HO, Yang Jiang, Shu Takagi

Research output: Contribution to journalJournal articlepeer-review

120 Citations (Scopus)


The recent advancement of motion recognition using Microsoft Kinect stimulates many new ideas in motion capture and virtual reality applications. Utilizing a pattern recognition algorithm, Kinect can determine the positions of different body parts from the user. However, due to the use of a single-depth camera, recognition accuracy drops significantly when the parts are occluded. This hugely limits the usability of applications that involve interaction with external objects, such as sport training or exercising systems. The problem becomes more critical when Kinect incorrectly perceives body parts. This is because applications have limited information about the recognition correctness, and using those parts to synthesize body postures would result in serious visual artifacts. In this paper, we propose a new method to reconstruct valid movement from incomplete and noisy postures captured by Kinect. We first design a set of measurements that objectively evaluates the degree of reliability on each tracked body part. By incorporating the reliability estimation into a motion database query during run time, we obtain a set of similar postures that are kinematically valid. These postures are used to construct a latent space, which is known as the natural posture space in our system, with local principle component analysis. We finally apply frame-based optimization in the space to synthesize a new posture that closely resembles the true user posture while satisfying kinematic constraints. Experimental results show that our method can significantly improve the quality of the recognized posture under severely occluded environments, such as a person exercising with a basketball or moving in a small room.

Original languageEnglish
Pages (from-to)1357-1369
Number of pages13
JournalIEEE Transactions on Cybernetics
Issue number5
Publication statusPublished - Oct 2013

Scopus Subject Areas

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

User-Defined Keywords

  • Human-computer interaction
  • Kinect
  • Local principal component analysis
  • Posture reconstruction


Dive into the research topics of 'Real-time posture reconstruction for Microsoft Kinect'. Together they form a unique fingerprint.

Cite this