METHOD FOR GENERATING DEPTH INFORMATION AND ELECTRONIC DEVICE SUPPORTING THE SAME

• US 20190327393A1
• Filed: 04/17/2019
• Published: 10/24/2019
• Est. Priority Date: 04/19/2018
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: YANG, Kyungdong, PARK, Jaehyoung, YU, Yungmok, CHOI, Dongbum, WON, Jonghun
• Granted Patent: US 11,252,306 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10

Abstract

An electronic device is provided. The electronic device includes a first camera, a second camera spaced apart from the first camera, and a processor. The processor is configured to obtain a first image of external objects using the first camera, obtain a second image of the external objects using the second camera, identify a specified object in which pieces of depth information are generated from among the external objects included in the first image and the second image based on phase difference comparison between the first image and the second image, then select depth information about the specified object among the pieces of depth information based on a degree of spreading of a point corresponding to the specified object included in at least one of the first image and the second image, and generate depth information about the external objects including the specified object using the selected depth information.

Claims

1. An electronic device comprising:

a first camera disposed on one surface of the electronic device;

a second camera disposed on the one surface and spaced apart from the first camera by a specified interval; and

at least one processor, wherein the at least one processor is configured to;

obtain a first image of one or more external objects using the first camera,

obtain a second image of the one or more external objects using the second camera,

identify a specified object in which pieces of depth information are generated, from among the one or more external objects included in the first image and the second image, based on phase difference comparison between the first image and the second image,

select depth information about the specified object among the pieces of depth information based on a degree of spreading of a point corresponding to the specified object included in at least one of the first image and the second image, and

generate depth information about the one or more external objects including the specified object using the selected depth information.

2. The electronic device of claim 1, wherein the at least one processor is further configured to:

select the depth information of the pieces of depth information by at least using;

a degree of spreading of a point, which corresponds to the specified object included in the first image, from among a plurality of degrees of spreading of a first point corresponding to the first camera, and

a degree of spreading of a point, which corresponds to the specified object included in the second image, from among a plurality of degrees of spreading of a second point corresponding to the second camera; and

generate the depth information about the one or more external objects using the selected depth information as the depth information about the specified object.

3. The electronic device of claim 2, wherein the at least one processor is further configured to:

identify first similarity between the plurality of degrees of spreading of the first point and characteristics of the specified object included in the first image and second similarity between the plurality of degrees of spreading of the second point and characteristics of the specified object included in the second image, and

select the depth information among the pieces of depth information using the first similarity and the second similarity.

4. The electronic device of claim 3, wherein the at least one processor is further configured to:

select the depth information among the pieces of depth information using a defocus distance corresponding to;

a degree of spreading of the first point, in which a sum of the first similarity and the second similarity is smallest, from among the plurality of degrees of spreading of the first point, or

a degree of spreading of the second point, in which a sum of the first similarity and the second similarity is smallest, from among the plurality of degrees of spreading of the second point.

5. The electronic device of claim 3, wherein the at least one processor is further configured to:

in the identifying of the first similarity and the second similarity, use a degree of spreading of the first point and a degree of spreading of the second point, which have the same defocus distance, from among the plurality of degrees of spreading of the first point and the plurality of degrees of spreading of the second point.

6. The electronic device of claim 2,

wherein the plurality of degrees of spreading of the first point or the plurality of degrees of spreading of the second point include depth relationship information about change in a degree of spreading of a point according to change of at least part of a distance between the electronic device and the one or more external objects, and

wherein the at least one processor is further configured to select the depth information further using the depth relationship information.

7. The electronic device of claim 1, wherein the at least one processor is further configured to:

obtain a third image by using the first camera,

select the depth information of the pieces of depth information using a degree of spreading of a point corresponding to the specified object included in the first image, and a degree of spreading of a point corresponding to the specified object included in the third image among a plurality of degrees of spreading of a point corresponding to the first camera, and

generate the depth information about the one or more external objects using the selected depth information as the depth information about the specified object.

8. The electronic device of claim 7, wherein the at least one processor is further configured to:

identify the degree of spreading of the point corresponding to the specified object included in the first image and the degree of spreading of the point corresponding to the specified object included in the third image using degrees of spreading of the point, which are spaced apart from each other by a defocus distance between the first image and the third image, from among the plurality degrees of spreading of the point, and

select the depth information among the pieces of depth information using a defocus distance of the degree of spreading of the point corresponding to the specified object included in the first image.

9. The electronic device of claim 7, wherein the third image is obtained before the first image is obtained.

10. An electronic device comprising:

a camera;

a memory; and

at least one processor,

wherein the camera comprises a first group of pixels and a second group of pixels per pixel, and

wherein the at least one processor is configured to;

obtain a first image of one or more external objects using the first group of pixels of the camera,

obtain a second image of the one or more external objects using the second group of pixels of the camera,

identify a specified object in which pieces of depth information are generated, from among the one or more external objects included in the first image and the second image, based on phase difference comparison between the first image and the second image, and

select depth information about the specified object among the pieces of depth information based on a degree of spreading of a point of the specified object included in at least one of the first image and the second image.

11. The electronic device of claim 10, wherein the at least one processor is further configured to:

select the depth information of the pieces of depth information by at least using;

a degree of spreading of a point, which corresponds to the specified object included in the first image, from among a plurality of degrees of spreading of a first point corresponding to the first group of pixels, and

a degree of spreading of a point, which corresponds to the specified object included in the second image, from among a plurality of degrees of spreading of a second point corresponding to the second group of pixels; and

generate depth information about the one or more external objects including the specified object using the selected depth information.

12. The electronic device of claim 11, wherein the at least one processor is further configured to:

identify first similarity between the plurality of degrees of spreading of the first point and characteristics of the specified object included in the first image and second similarity between the plurality of degrees of spreading of the second point and characteristics of the specified object included in the second image, and

select the depth information among the pieces of depth information using the first similarity and the second similarity.

13. The electronic device of claim 12, wherein the at least one processor is further configured to:

select the depth information among the pieces of depth information, using a defocus distance corresponding to;

degrees of spreading of the first point, in each of which a sum of the first similarity and the second similarity is smallest, from among the plurality of degrees of spreading of the first point, or

degrees of spreading of the second point, in each of which a sum of the first similarity and the second similarity is smallest, from among the plurality of degrees of spreading of the second point.

14. The electronic device of claim 12, wherein the at least one processor is further configured to:

in the identifying of the first similarity and the second similarity, use a degree of spreading of the first point and a degree of spreading of the second point, which have the same defocus distance, from among the plurality of degrees of spreading of the first point and the plurality of degrees of spreading of the second point.

15. The electronic device of claim 11,

wherein the plurality of degrees of spreading of the first point or the plurality of degrees of spreading of the second point further include depth relationship information about change in a degree of spreading of a point according to change of at least part of a distance between the electronic device and the one or more external objects, and

wherein the at least one processor is further configured to select the depth information further using the depth relationship information.

16. The electronic device of claim 10, wherein the at least one processor is further configured to:

obtain a third image by using the first camera,

select the depth information of the pieces of depth information by at least using a degree of spreading of a point, which corresponds to the specified object included in the first image, from among a plurality of degrees of spreading of a point corresponding to the first camera, and a degree of spreading of a point which corresponds to the specified object included in the third image, and

generate the depth information about the one or more external objects using the selected depth information as the depth information about the specified object.

17. The electronic device of claim 16, wherein the at least one processor is further configured to:

based on a defocus distance between the first image and the third image, identify the degree of spreading of the point corresponding to the specified object included in the first image and the degree of spreading of the point corresponding to the specified object included in the third image by at least using the plurality degrees of spreading of a point, which are spaced apart from each other by the defocus distance, from among the plurality degrees of spreading of a point, and

select the depth information among the pieces of depth information using a defocus distance between the degrees of spreading of the point corresponding to the specified object included in the first image.

18. The electronic device of claim 16, the third image is obtained before the first image is obtained.

19. A depth information generating method of an electronic device, the depth information generating method comprising:

obtaining a first image and a second image of one or more external objects using at least one camera, wherein the first image and the second image have at least phase difference of the one or more external objects;

identifying a specified object in which pieces of depth information are generated, from among the one or more external objects included in the first image and the second image, based on phase difference comparison between the first image and the second image;

selecting depth information about the specified object among the pieces of depth information based on a degree of spreading of a point corresponding to the specified object included in at least one image of the first image and the second image; and

generating depth information about the one or more external objects including the specified object using the selected depth information.

20. The method of claim 19,

wherein the selecting comprises selecting the depth information of the pieces of depth information by at least using:

a degree of spreading of a point, which corresponds to the specified object included in the first image, from among a plurality of degrees of spreading of a first point corresponding to a first phase of the at least one camera, and

a degree of spreading of a point, which corresponds to the specified object included in the second image, from among a plurality of degrees of spreading of a second point corresponding to a second phase of the at least one camera, and

wherein the generating comprises generating the depth information about the one or more external objects using the selected depth information as the depth information about the specified object.