METHOD AND DEVICE FOR OBTAINING IMAGE, AND RECORDING MEDIUM THEREOF

• US 20180018786A1
• Filed: 07/14/2017
• Published: 01/18/2018
• Est. Priority Date: 07/15/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jakubiak, Antoni, Bogucki, Marcin, Szolucha, Michal, Wengierow, Michal, Kies, Pawel
• Granted Patent: US 11,004,223 B2
• Time in Patent Office: Days
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2210/56 Particle system, point based geometry or rendering
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images

Abstract

A device for obtaining an image includes a projector configured to project light having a preset pattern to an object, a first camera configured to sense light reflected and fed back by the object to obtain a depth image, a second camera configured to obtain a color image of the object, a motion sensor configured to sense a motion of the device. The device also includes and a processor configured to obtain a color image and a plurality of depth images of the object, by changing a parameter about the projected light and the light sensed by the first camera. When the motion of the device is less than a threshold, the processor is configured to merge pieces of depth information respectively obtained from the plurality of depth images with each other, and obtain a 3D image of the object, based on the merged depth information and the color image.

Claims

1. A device for obtaining an image, the device comprising:

a projector configured to project light comprising a preset pattern to an object;

a first camera configured to sense light reflected and fed back by the object to obtain a depth image;

a second camera configured to obtain a color image of the object;

a motion sensor configured to sense a motion of the device; and

a processor configured to obtain a color image and a plurality of depth images of the object by changing at least one of parameters about the light projected by the projector or the light sensed by the first camera, and, when the sensed motion of the device is less than a threshold, merge pieces of depth information respectively obtained from the plurality of depth images with each other, and obtain a three-dimensional (3D) image of the object, based on the merged pieces of depth information and the obtained color image.

2. The device of claim 1, wherein the processor is configured to obtain the plurality of depth images of the object by changing at least one of an intensity that the light is projected, a time period that the light is projected, a time period that the light reflected and fed-back is sensed, or a sensitivity that the light reflected and fed-back is sensed.

3. The device of claim 1, wherein the processor is configured to select depth images obtained when the sensed motion of the device is less than the threshold from among the plurality of depth images, and merge pieces of depth information of the selected depth images with each other.

4. The device of claim 1, wherein, when the sensed motion of the device is equal to or greater than the threshold, the first camera is configured to obtain a second depth image of the object.

5. The device of claim 1, wherein, when the sensed motion of the device is equal to or greater than the threshold, the processor is configured to merge the respective pieces of depth information of the plurality of depth images with each other, based on motion information of the device representing the motion sensed by the motion sensor.

6. The device of claim 1, wherein the processor is configured to determine respective depth values of a plurality of pixels included in the obtained color image, based on the merged pieces of depth information.

7. The device of claim 6, wherein the processor is configured to determine a depth value of a first pixel of which a depth value has not yet been determined from among the plurality of pixels included in the obtained color image, based on depth values of pixels adjacent to the first pixel.

8. The device of claim 1, wherein the processor is configured to blur at least a portion of the 3D image of the object, based on the merged pieces of depth information and preset focus information.

9. The device of claim 1, wherein the processor is configured to detect a region corresponding to a preset distance range from the device from the 3D image of the object, based on the merged pieces of depth information.

10. A method of obtaining an image in a device, the method comprising:

changing at least one of parameters about light comprising a preset pattern and projected onto an object and light reflected and fed back by the object to obtain a plurality of depth images of the object;

merging pieces of depth information respectively obtained from the obtained plurality of depth images with each other, when a motion of the device is less than a threshold; and

obtaining a 3D image of the object, based on the merged pieces of depth information and a color image of the object.

11. The method of claim 10, wherein the obtained plurality of depth images comprises obtaining the plurality of depth images of the object by changing at least one of an intensity that the light is projected, a time period that the light is projected, a time period that the light reflected and fed-back is sensed, or a sensitivity that the light reflected and fed-back is sensed.

12. The method of claim 10, further comprising selecting depth images obtained when the motion of the device is less than the threshold, from among the plurality of depth images,

wherein the merging comprises merging respective pieces of depth information of the selected depth images with each other.

13. The method of claim 10, further comprising obtaining a second depth image of the object, when the motion of the device is equal to or greater than the threshold.

14. The method of claim 10, wherein the merging comprises, when the motion of the device is equal to or greater than the threshold, merging the respective pieces of depth information of the plurality of depth images with each other, based on motion information of the device representing the motion sensed by a motion sensor.

15. The method of claim 10, wherein the obtaining of the 3D image comprises determining respective depth values of a plurality of pixels included in the color image, based on the merged pieces of depth information.

16. The method of claim 15, wherein the determining comprises determining a depth value of a first pixel of which a depth value has not yet been determined from among the plurality of pixels included in the color image, based on depth values of pixels adjacent to the first pixel.

17. The method of claim 10, further comprising blurring at least a portion of the 3D image of the object, based on the merged pieces of depth information and preset focus information.

18. The method of claim 10, further comprising detecting a region corresponding to a preset distance range from the 3D image of the object, based on the merged pieces of depth information.

19. A non-transitory storage medium comprising a program, for obtaining an image on an electronic device, the program comprising program code that, when executed by at least one processor of the electronic device, causes the electronic device to:

change at least one of parameters about light comprising a preset pattern and projected onto an object and light reflected and fed back by the object to obtain a plurality of depth images of the object;

merge pieces of depth information respectively obtained from the obtained plurality of depth images with each other, when a motion of the electronic device is less than a threshold; and

obtain a 3D image of the object, based on the merged pieces of depth information and a color image of the object.

20. The program code of claim 19, wherein program code that, when executed by at least one processor of the electronic device, causes the electronic device to obtain the plurality of depth images comprises program code that, when executed by at least one processor of the electronic device, causes the electronic device to obtain the plurality of depth images of the object by changing at least one of an intensity that the light is projected, a time period that the light is projected, a time period that the light reflected and fed-back is sensed, or a sensitivity that the light reflected and fed-back is sensed.