In order to build the compact model, it is necessary to first recover the fine scale deformations and correspondences over the training sequence. This is a challenging task, as ideally the deformations should give a photo-consistent and silhouette-consistent model, and they should also be flow-consistent.

The problem of obtaining correspondences is addressed in a restricted form where there are few cameras available. In the first stage, the problem is addressed with a single moving camera. A constant velocity constraint is used to extract scene structure and correspondences from a single moving view (monocular scene flow).

This formulation is generalized to a skinned proxy, where any known tracked proxy motion is taken into account. With the use of a temporal basis, this representation allows reconstruction from non-overlapping views.

Constant velocity Monocular Scene Flow

The following video illustrates results from the monocular flow formulation.

Basis-Constrained Flow on a Proxy

The following video illustrates results from the basis flow.

Silhouette Displacements

In practice it is often hard to obtain dense correspondences from photometric methods. Instead, silhouettes provide a good alternative.

The following video illustrates how silhouette displacements can improve agreement with silhouettes.

These two videos show the results of displacements on the MIT data sequences.