Spatio-Temporarily Consistent Video Disparity Estimation Algorithm

Background

From a rectified stereo image pair, the task of view synthesis is to generate images from any viewpoint along the baseline. The main difficulty of the problem is how to fill occluded regions. We present a new method for view synthesis that is both fast and accurate. Occlusions are filled using color and disparity information to produce consistent pixel estimates.

Our goal is to present a systematic method by which we generate accurate and spatio-temporally consistent disparity maps from complex stereo video sequences. We leverage the strengths of current state-of-the-art image-based techniques, but, in addition, we explicitly enforce the consistency of estimates in both space and time by treating the video as a space-time volume corrupted by noise. In so doing, we provide an algorithm that has the capability of refining arbitrary image-based disparity estimation techniques and, at the same time, extending them to the video domain.

Technology Description

We present a novel stereo video disparity estimation method. The proposed method is a two-stage algorithm. During the first stage, initial disparity maps are computed in a frame by-frame basis. In the second stage, the initial estimates are treated as a space-time volume.

By setting up an l1-normed minimization problem with a novel three-dimensional total variation regularization, spatial smoothness and temporal consistency are handled simultaneously.

Applications

Our method can be used as a post-processing step to refine noisy estimates or to extend to videos.

Advantages

The proposed method shows superior speed, accuracy, and consistency compared to state-of-the-art algorithms. We have shown the method to be resilient even in large amounts of noise.

Furthermore, we have illustrated that in general our method can be used to refine the results of any disparity estimation technique suffering from impulsive noise or estimation errors.

Patent Status