SYSTEM AND METHOD FOR DEPTH EXTRACTION OF IMAGES WITH FORWARD AND BACKWARD DEPTH PREDICTION

• US 20110069064A1
• Filed: 05/28/2008
• Published: 03/24/2011
• Est. Priority Date: 05/28/2008
• Status: Active Grant

• Current Assignee: InterDigital Madison Patent Holdings (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Zhang, Dong-Qing, Yoon, Youngshik, Izzat, Izzat
• Granted Patent: US 9,659,382 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

A system and method for spatiotemporal depth extraction of images with forward and backward depth prediction are provided. The system and method of the present disclosure provide for acquiring a plurality of frames, generating a first depth map of a current frame in the plurality of frames based on a depth map of a previous frame in the plurality of frames, generating a second depth map of the current frame in the plurality of frames based on a depth map of a subsequent frame in the plurality of frames, and processing the first depth map and the second depth map to produce a third depth map for the current frame.

Claims

1. A method of matching at least two images, the method comprising:

acquiring a plurality of frames;

generating a first depth map of a current frame in the plurality of frames based on a depth map of a previous frame in the plurality of frames;

generating a second depth map of the current frame in the plurality of frames based on a depth map of a subsequent frame in the plurality of frames; and

processing the first depth map and the second depth map to produce a third depth map for the current frame.

2. The method of claim 1, wherein the first depth map contains an error based on the depth map of the previous frame and the second depth map contains a correction for reducing the error.

3. The method of claim 1, wherein each frame in the plurality of frames includes a first image and a second image.

4. The method of claim 3, wherein the steps of generating first and second depth maps further include the steps of:

estimating a disparity of at least one point in the first image with at least one corresponding point in the second image;

generating a disparity map based on the estimated disparity; and

converting the estimated disparity map into a depth map.

5. The method of claim 4, wherein the estimating the disparity step includes computing a pixel matching cost function.

6. The method of claim 4, wherein the estimating the disparity step includes computing a smoothness cost function.

7. The method of claim 4, wherein the estimating the disparity step includes computing a temporal cost function.

8. The method of claim 4, wherein the estimating the disparity step includes performing a low-cost optimization function.

9. The method of claim 8, wherein the estimating the disparity step includes computing a belief propagation function based on the results of the low-cost optimization function.

10. The method of claim 3, wherein the first image is a left eye view and second image is a right eye view of a stereoscopic pair.

11. A system for matching at least two images, the system comprising:

means for acquiring a plurality of frames;

means for generating a first depth map of a current frame in the plurality of frames based on a depth map of a previous frame in the plurality of frames;

means for generating a second depth map of the current frame in the plurality of frames based on a depth map of a subsequent frame in the plurality of frames; and

means for processing the first depth map and the second depth map to produce a third depth map for the current frame.

12. The system of claim 11, wherein the first depth map contains an error based on the depth map of the previous frame and the second depth map contains a correction for reducing the error.

13. The system of claim 11, wherein each frame in the plurality of frames includes a first image and a second image.

14. The system of claim 13, wherein the means for generating first and second depth maps further include:

means for estimating a disparity of at least one point in the first image with at least one corresponding point in the second image;

means for generating a disparity map based on the estimated disparity; and

means for converting the estimated disparity map into a depth map.

15. The system of claim 14, wherein the means for estimating the disparity includes means for computing a pixel matching cost function.

16. The system of claim 14, wherein the means for estimating the disparity includes means for computing a smoothness cost function.

17. The system of claim 14, wherein the means for estimating the disparity includes computing a temporal cost function.

18. The system of claim 14, wherein the means for estimating the disparity includes means for performing a low-cost optimization function.

19. The system of claim 18, wherein the means for estimating the disparity includes means for computing a belief propagation function based on the output of the means for performing the low-cost optimization function.

20. The system of claim 13, wherein the first image is a left eye view and second image is a right eye view of a stereoscopic pair.