Method and apparatus for estimating depth, and method and apparatus for training distance estimator

• US 10,068,347 B2
• Filed: 12/20/2016
• Issued: 09/04/2018
• Est. Priority Date: 01/07/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Wonhee, Sung, Young Hun, Lim, Soochul
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US15/384,616
• Publication Number:
  US 20170200284A1
• Time in Patent Office: 623 Days
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method and apparatus for estimating a depth, and a method and apparatus for training an distance estimator are provided, wherein the method for estimating a depth may include obtaining an image, and estimating a depth of a second image object based on a correlation between a first image object and the second image object in the image and a depth of the first image object.

Claims

1. A method of estimating a depth, the method comprising:

obtaining an image of a scene including at least a first object and a second object;

estimating a depth of a second image object corresponding to the second object based on a correlation between a first image object corresponding to the first object and the second image object in the image and a depth of the first image object.

2. The method of claim 1, wherein the estimating comprises estimating the depth of the second image object by inputting the value of the correlation between the first image object and the second image object in the image and the depth of the first image object into a pre-trained distance estimator.

3. The method of claim 1, wherein the first image object is a short-distance image object positioned closer than a threshold depth determined based on a baseline of a stereo camera configured to obtain the image, and the second image object is a long-distance image object positioned farther than the threshold depth.

4. The method of claim 1, wherein the estimating comprises obtaining the depth of the first image object by performing stereo matching on the image.

5. The method of claim 1, wherein the estimating comprises obtaining a value of the correlation by analyzing the image.

6. The method of claim 1, wherein the correlation is obtained based on a depth cue of each of the first image object and the second image object.

7. The method of claim 6, wherein the depth cue comprises at least one of a linear perspective, an aerial perspective, a relative size, a familiar size, a proximity to a horizon, and a texture gradient.

8. The method of claim 1, wherein each of the first image object and the second image object comprises at least one of a corner point, a texture, and a line.

9. The method of claim 2, wherein the pre-trained distance estimator is trained based on an estimated depth of a training long-distance object in a scene and an actual depth of the training long-distance object;

wherein the estimated depth of the training long-distance object is estimated based on (1) a value of a correlation between an image of a training short-distance object in the scene and an image of the training long-distance object included in training data and (2) a depth of the training short-distance object obtained by performing stereo matching on the training data.

10. A method of training a distance estimator, the method comprising:

obtaining a correlation between each pair of training short-distance object and training long-distance object of a scene included in training data;

obtaining a depth of each training short-distance object by performing stereo matching on the training data;

obtaining actual depth of each training long-distance object; and

training the distance estimator using the correlation between each pair of training short-distance object and training long-distance object, the depth of each training short-distance object and the actual depth of each training long-distance object;

wherein the distance estimator is configured to estimate a depth of a long-distance object in a scene based on a depth of a short-distance object in the scene and a correlation between the short-distance object and the long-distance object.

11. The method of claim 10, wherein the short-distance object is an object positioned closer than a threshold depth determined based on a baseline of a stereo camera configured to obtain the training data, and the long-distance object is an object positioned farther than the threshold value.

12. The method of claim 10, wherein the correlation is obtained based on a depth cue of each of the short-distance object and the long-distance object.

13. The method of claim 12, wherein the depth cue comprises at least one of a linear perspective, an aerial perspective, a relative size, a familiar size, a proximity to a horizon, and a texture gradient.

14. An apparatus for estimating a distance, the apparatus comprising:

a stereo camera configured to obtain an image of a scene including at least a first object and a second object; and

a circuit configured to estimate a depth of a second image object corresponding to the second object based on a value of a correlation between a first image object corresponding to the first object and the second image object in the image and a depth of the first image object.

15. The apparatus of claim 14, wherein the circuit is configured to estimate the depth of the second image object by inputting the value of the correlation between the first image object and the second image object and the depth of the first image object into a pre-trained distance estimator.

16. The apparatus of claim 14, wherein the first image object is a short-distance image object positioned closer than a threshold depth determined based on a baseline of the stereo camera, and the second image object is a long-distance image object positioned farther than the threshold depth.

17. The apparatus of claim 15, wherein the pre-trained distance estimator is trained based on an estimated depth of a long-distance object in a scene and an actual depth of the long-distance object;

wherein the estimated depth of the long-distance object is estimated based on (1) a value of a correlation between an image of a short-distance object in the scene and an image of the long-distance object included in training data and (2) a depth of the short-distance object obtained by performing stereo matching on the training data.

18. The apparatus of claim 14, wherein the controller is configured to obtain the depth of the first image object by performing stereo matching on the image.

19. The apparatus of claim 14, wherein the controller is configured to obtain a value of the correlation by analyzing the image.

20. The apparatus of claim 19, wherein the controller is configured to obtain the value of the correlation based on a depth cue of each of the first image object and the second image object.