Joint angle tracking is a key component in building the compact model of pose- and view-dependency. In this chapter, several objective functions for data (SSD and silhouette) are combined with smoothness and joint angle prior terms in an energy framework to recover joint angles from image observations. Since the tracking is performed in a controlled environment, the focus is on local optimization with accelerations on the GPU where possible. The tracking uses a skinned mesh, meaning deformable faces can also be modeled.

Intensity-driven tracking

When no silhouettes are available, the image intensities can be used directly to track the object. The local image intensity tracking is implemented on the GPU for near real-time tracking of skinned objects (including faces).

Silhouette XOR tracking

Silhouette tracking is implemented with both an XOR objective and a ICP term. The ICP term is faster and works fine when silhouette inputs are good. If input is noisy, the XOR term is better. The following example has both noisy silhouettes and a poor model. The XOR tracking is more capable of tracking.

Pose prior term

Using a pose prior helps local silhouette-based tracking in the presence of noisy silhouettes.

Silhouette tracking results on the MIT sequences

Samba

The XOR tracking works well also, but has a few mistracks of the legs. samba-xor.mp4

The XOR tracking result is also available. There are some slight mistracks. crane-xor.mp4

Handstand:

The XOR tracking result loses track after the handstand. handstand-xor.mp4