System and method for insertion of photograph taker into a photograph

• US 10,068,147 B2
• Filed: 04/30/2015
• Issued: 09/04/2018
• Est. Priority Date: 04/30/2015
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Srivastava, Gaurav
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US14/701,312
• Publication Number:
  US 20160321515A1
• Time in Patent Office: 1,223 Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence

Abstract

An electronic device is provided. The electronic device includes processing circuitry. The processing circuitry is configured to receive a first image comprising an image of a first object and a second image comprising an image of a second object. The processing circuitry is also configured to identify a depth of the second object in the second image. The processing circuitry is further configured to insert the image of the first object into the second image at a depth position based on the depth of the second object. The processing circuitry is configured to generate to display the image of the first object and the image of the second object in the second image.

Claims

1. An electronic device comprising:

a display; and

processing circuitry configured to;

receive a first image and a second image, the first image comprising an image of a first object and the second image comprising an image of a second object,

identify a depth, a horizontal position, and a vertical position of the second object in the second image,

calculate, based on the depth, the horizontal position, and the vertical position of the second object, an optimal horizontal, vertical, and depth position at which to insert the image of the first object into the second image so that the image of the first object is realistically positioned relative to the second object,

insert the image of the first object into the second image at the calculated optimal horizontal, vertical, and depth position,

generate a third image comprising the image of the first object and the image of the second object in the second image, and

cause the display to display the third image.

2. The electronic device of claim 1, wherein the image of the first object obstructs a portion of the second object when the first object and the second object is displayed in the second image.

3. The electronic device of claim 1, wherein the processing circuitry is configured to identify the first object in the first image using two-dimensional (2-D) image segmentation.

4. The electronic device of claim 1, wherein the processing circuitry is configured to identify the depth of the second object in the second image using three-dimensional (3-D) image segmentation.

5. The electronic device of claim 1, wherein the processing circuitry is configured to insert the first object into the second image using 3-D image segmentation to identify a horizontal position, a vertical position, and the depth position of one or more points in the second image.

6. The electronic device of claim 1, wherein the processing circuitry is configured to identify the depth of the image of the second object in the second image using 3-D image segmentation to generate a depth histogram of one or more points of the second image.

7. The electronic device of claim 1, wherein the processing circuitry is configured to insert the image of the first object into the second image so that the image of the first object does not share at least the depth position with the image of the second object.

8. The electronic device of claim 1, wherein the processing circuitry is configured to insert the image of the first object into the second image so that the image of the first object does not share at least one of a horizontal position, a vertical position, or the depth position with the image of the second object.

9. A method for use in an electronic device, the method comprising:

receiving a first image and a second image, the first image comprising an image of a first object and the second image comprising an image of a second object,

identifying a depth, a horizontal position, and a vertical position of the second object in the second image,

calculating, based on the depth, the horizontal position, and the vertical position of the second object, an optimal horizontal, vertical, and depth position at which to insert the image of the first object into the second image so that the image of the first object is realistically positioned relative to the second object,

inserting the image of the first object into the second image at the calculated optimal horizontal, vertical, and depth position, and

generating a third image comprising the image of the first object and the image of the second object in the second image, the third image suitable for display on a display of the electronic device.

10. The method of claim 9, wherein the first object obstructs a portion of the second object when the first object and the second object is displayed in the second image.

11. The method of claim 9, further comprising identifying the first object in the first image using two-dimensional (2-D) image segmentation.

12. The method of claim 9, wherein identifying the depth of the second object in the second image comprises using three-dimensional (3-D) image segmentation.

13. The method of claim 9, wherein calculating the depth position at which to insert the image of the first object into the second image based on the depth of the second object comprises using 3-D image segmentation to identify a horizontal position, a vertical position, and the depth position of one or more points in the second image.

14. The method of claim 9, wherein identifying the depth of the image of the second object in the second image comprises using 3-D image segmentation to generate a depth histogram of one or more points of the second image.

15. The method of claim 9, wherein inserting the image of the first object into the second image comprises inserting the image of the first object into the second image so that the image of the first object does not share at least the depth position with the image of the second object.

16. The method of claim 9, wherein inserting the image of the first object into the second image comprises inserting the image of the first object into the second image so that the image of the first object does not share at least one of a horizontal position, a vertical position, or the depth position with the image of the second object.

17. An electronic device comprising:

a first camera configured to capture a first image of a first environment including a first object, wherein the first image comprises an image of the first object;

a second camera configured to capture a second image of a second environment including a second object, wherein the second image comprises an image of the second object;

a first depth sensor configured to measure a distance from the first camera to the first object;

a display; and

processing circuitry configured to;

receive the first captured image comprising the image of the first object,

receive the second captured image comprising the image of the second object,

calculate, as a function of a horizontal position of the first object, a vertical position of the first object, and the distance from the first camera to one or more points of the first object in the first image, an optimal horizontal, vertical, and depth position at which to insert the image of the second object into the first image so that the image of the second object is realistically positioned relative to the first object,

insert the image of the second object into the first image at the calculated optimal horizontal, vertical, and depth position,

generate a third image comprising the image of the first object and the image of the second object in the first captured image, and

cause the display to display the third image.

18. The electronic device of claim 17, wherein the first object obstructs a portion of the second object when the first object and the second object are displayed in the second captured image.

19. The electronic device of claim 17, further comprising a display screen configured to display the image of the first object and the image of the second object in the first captured image.

20. The electronic device claim 17, wherein the processing circuitry is further configured to:

apply three-dimensional (3-D) image segmentation to the first image of the first object,

generate a point cluster of the image of the first object in the first captured image, and

generate a depth histogram of the image of the first object in order to insert the image of the second object into the first captured image at the depth position.