Method for predicting stereoscopic depth and apparatus thereof

• US 9,936,189 B2
• Filed: 05/20/2016
• Issued: 04/03/2018
• Est. Priority Date: 08/26/2015
• Status: Active Grant

• Current Assignee: BOE Technology Group Co., Ltd.
• Original Assignee: BOE Technology Group Co., Ltd.
• Inventors: Zhao, Xingxing
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): RIVERA-MARTINEZ, GUILLERMO M
• Application Number:
  US15/501,850
• Publication Number:
  US 20170237969A1
• Time in Patent Office: 683 Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

The present application discloses a method for predicting stereoscopic depth comprising capturing object images of a target scene; generating a stereoscopic depth image associated with the object images; identifying a no-depth-data region in the stereoscopic depth image; identifying a depth-data region correlated to the no-depth-data region; and assigning a depth data to the no-depth-data region based on a depth data of the depth-data region correlated to the no-depth-data region.

Claims

1. A method for predicting stereoscopic depth, comprising:

capturing object images of a target scene;

generating a stereoscopic depth image associated with the object images;

identifying a no-depth-data region in the stereoscopic depth image;

identifying a depth-data region correlated to the no-depth-data region; and

assigning a depth data to the no-depth-data region based on a depth data of the depth-data region correlated to the no-depth-data region;

wherein identifying the depth-data region correlated to the no-depth-data region comprises;

categorizing the object image of the target scene into a plurality of categorized pixel blocks based on image characteristics in the object image; and

searching through the plurality of categorized pixel blocks to identify a first pixel block associated with a no-depth-data region; and

wherein the plurality of categorized pixel blocks are categorized based on one or a combination of image characteristics selected from color, grey scale, texture, contour, and shape.

2. The method of claim 1, wherein identifying the depth-data region correlated to the no-depth-data region further comprises:

generating a partition table storing positional information of each pixel in the plurality of categorized pixel blocks; and

calculating a correlation strength between each two adjacent pixel blocks among the plurality of categorized pixel blocks, the two adjacent pixel blocks are identified by searching through the partition table.

3. The method of claim 2, wherein identifying the depth-data region correlated to the no-depth-data region further comprises:

identifying a most relevant pixel block correlated to the first pixel block associated with the no-depth-data region based on the correlation strength between each two adjacent pixel blocks; and

assigning a depth-data region associated with the most relevant pixel block as the depth-data region correlated to the no-depth-data region.

4. The method of claim 2, wherein identifying a depth-data region correlated to the no-depth-data region further comprising:

identifying a second pixel block having a depth-data region proximal to the no-depth-data region; and

determining whether the first pixel block and the second pixel block are a same pixel block.

5. The method of claim 4, wherein the first pixel block and the second pixel block are determined to be a same pixel block;

the method further comprising assigning the depth-data region in the second pixel block as the depth-data region correlated to the no-depth-data region in the first pixel block.

6. The method of claim 4, wherein the first pixel block and the second pixel block are determined to be two different pixel blocks;

the method further comprising identifying a most relevant pixel block correlated to the first pixel block associated with the no-depth-data region; and

assigning one or more a depth-data region associated with the most relevant pixel block as a depth-data region correlated to the no-depth-data region.

7. The method of claim 4, wherein the first pixel block and the second pixel block are determined to be a same pixel block;

the method further comprising directly assigning the depth data in the depth-data region as the depth data for the no-depth-data region.

8. The method of claim 4, wherein the first pixel block and the second pixel block are determined to be a same pixel block;

the method further comprising interpolating the depth data in the depth-data region to obtain an interpolated depth data corresponding to the no-depth-data region; and

assigning the interpolated depth data as the depth data for the no-depth-data region.

9. The method of claim 4, wherein the first pixel block and the second pixel block are determined to be two different pixel blocks, and more than one most relevant pixel blocks are identified;

the method further comprising averaging all depth data in all of the more than one most relevant pixel blocks to obtain an averaged depth data; and

assigning the average depth data as the depth data for the no-depth-data region.

10. The method of claim 2, wherein each of the plurality of categorized pixel block has an area no smaller than the no-depth-data region.

11. The method of claim 2, further comprising populating the depth data assigned to the no-depth-data region, pixel-by-pixel, into a corresponding no-depth-data region in the stereoscopic depth image.

14. The method of claim 3, wherein identifying the most relevant pixel block correlated to the first pixel block associated with the no-depth-data region based on the correlation strength between each two adjacent pixel blocks comprises identifying a pixel block adjacent to, and having a largest correlation strength with, the first pixel block associated with the no-depth-data region.

15. The method of claim 3, wherein identifying the most relevant pixel block correlated to the first pixel block associated with the no-depth-data region based on the correlation strength between each two adjacent pixel blocks comprises:

obtaining correlation strengths between the first pixel block associated with the no-depth-data region and each pixel block adjacent to it based on a previously calculated correlation strengths for all pairs of every two adjacent pixel blocks;

comparing the obtained correlation strengths to each other; and

selecting one or more the most relevant pixel block that has the largest correlation strength among the obtained correlation strengths.

16. The method of claim 3, wherein assigning the depth data to the no-depth-data region based on the depth data of the depth-data region correlated to the no-depth-data region comprises:

obtaining an average value of the depth data for the one or more depth-data regions in the most relevant pixel block; and

assigning the average value as the depth data for the no-depth-data region.

12. A method for predicting stereoscopic depth, comprising:

capturing object images of a target scene;

generating a stereoscopic depth image associated with the object images;

identifying a no-depth-data region in the stereoscopic depth image;

identifying a depth-data region correlated to the no-depth-data region; and

assigning a depth data to the no-depth-data region based on a depth data of the depth-data region correlated to the no-depth-data region;

wherein identifying the depth-data region correlated to the no-depth-data region comprises;

categorizing the object image of the target scene into a plurality of categorized pixel blocks based on image characteristics in the object image;

generating a partition table storing positional information of each pixel in the plurality of categorized pixel blocks;

calculating a correlation strength between each two adjacent pixel blocks among the plurality of categorized pixel blocks, the two adjacent pixel blocks are identified by searching through the partition table; and

searching through the plurality of categorized pixel blocks to identify a first pixel block associated with a no-depth-data region; and

wherein the correlation strength is calculated based on a similarity of average grey scale level between each two adjacent pixel blocks among the plurality of categorized pixel blocks.

13. The method of claim 12, wherein the similarity of average grey scale level between each two adjacent pixel blocks among the plurality of categorized pixel blocks is calculated based on:

18. The method of claim 12, wherein identifying the depth-data region correlated to the no-depth-data region further comprises:

identifying a most relevant pixel block correlated to the first pixel block associated with the no-depth-data region based on the correlation strength between each two adjacent pixel blocks; and

assigning a depth-data region associated with the most relevant pixel block as the depth-data region correlated to the no-depth-data region.

19. The method of claim 12, wherein identifying a depth-data region correlated to the no-depth-data region further comprising:

identifying a second pixel block having a depth-data region proximal to the no-depth-data region; and

determining whether the first pixel block and the second pixel block are a same pixel block.

20. The method of claim 12, wherein the plurality of categorized pixel blocks are categorized based on one or a combination of image characteristics selected from color, grey scale, texture, contour, and shape.

17. A method for predicting stereoscopic depth, comprising:

capturing object images of a target scene;

generating a stereoscopic depth image associated with the object images;

identifying a no-depth-data region in the stereoscopic depth image;

identifying a depth-data region correlated to the no-depth-data region; and

assigning a depth data to the no-depth-data region based on a depth data of the depth-data region correlated to the no-depth-data region;

wherein identifying the depth-data region correlated to the no-depth-data region comprises;

categorizing the object image of the target scene into a plurality of categorized pixel blocks based on image characteristics in the object image;

generating a partition table storing positional information of each pixel in the plurality of categorized pixel blocks;

calculating a correlation strength between each two adjacent pixel blocks among the plurality of categorized pixel blocks, the two adjacent pixel blocks are identified by searching through the partition table;

searching through the plurality of categorized pixel blocks to identify a first pixel block associated with a no-depth-data region;

identifying a most relevant pixel block correlated to the first pixel block associated with the no-depth-data region based on the correlation strength between each two adjacent pixel blocks; and

assigning a depth-data region associated with the most relevant pixel block as the depth-data region correlated to the no-depth-data region;

wherein assigning the depth data to the no-depth-data region based on the depth data of the depth-data region correlated to the no-depth-data region comprises;

obtaining an average value of the depth data for the one or more depth-data regions in the most relevant pixel block; and

assigning the average value as the depth data for the no-depth-data region; and

wherein assigning the depth data to the no-depth-data region based on the depth data of the depth-data region correlated to the no-depth-data region comprises;

obtaining an average value of the grey scale levels for the one or more depth-data regions in the most relevant pixel block; and

assigning the average value of the grey scale levels as the depth data for the no-depth-data region.