Image processing device, image processing method, and program for generating composite image

• US 9,251,765 B2
• Filed: 11/29/2012
• Issued: 02/02/2016
• Est. Priority Date: 12/21/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sen, Seigo
• Primary Examiner(s): He, Weiming
• Application Number:
  US13/688,902
• Publication Number:
  US 20130162672A1
• Time in Patent Office: 1,160 Days
• Field of Search: 345/629, 345/637
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: G09G 5/393 Arrangements for updating the contents of the bit-mapped memory
• CPC Class Codes: G09G 5/397 Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay G09G5/02 takes precedence
• CPC Class Codes: H04N 1/00161 Viewing or previewing
• CPC Class Codes: H04N 1/00167 Processing or editing H04N1/00196 - H04N1/00201 take precedence
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/42661 for reading from or writing on a magnetic storage medium, e.g. hard disk drive
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4363 Adapting the video stream to a specific local network, e.g. a Bluetooth® network
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440263 by altering the spatial resolution, e.g. for displaying on a connected PDA
• CPC Class Codes: H04N 21/8153 comprising still images, e.g. texture, background image
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 5/272 : Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/445 : for displaying additional information H04N5/50 takes precedence

Abstract

An image processing device includes a composite processing unit that generates a composite image for display by including a through image generated based on a captured image signal obtained by performing photoelectric conversion for light incident from a subject in an image for a compositing process and composites a captured image of a recording time point in the image for a compositing process in accordance with a compositing arrangement state of the through image in the composite image for display, and a display control unit that performs display control for the composite image for display.

Claims

1. An image processing device comprising:

a composite processing unit that generates a composite image for display by including a through image, generated based on a captured image signal obtained by performing photoelectric conversion for light incident from a subject, in a recorded image for a compositing process and composites a captured image of a recording time point in the recorded image for the compositing process in accordance with an arrangement state of the through image in the composite image for display,

wherein the recorded image is set as a background image in the composite image,

wherein the composite processing unit performs a process of changing the background image based on a result of a comparison process between the through image and the background image, wherein the comparison process is a comparison process of image depth; and

a display control unit that performs display control for the composite image for display.

2. The image processing device according to claim 1, further comprising:

an arrangement determination unit that determines the arrangement state of the through image within the recorded image for the compositing process according to content of the captured image signal,

wherein the composite processing unit composites the through image on the recorded image for the compositing process in the arrangement state determined by the arrangement determination unit so as to generate the composite image for display.

3. The image processing device according to claim 2, wherein, according to an arranged image which has already been arranged in the recorded image for the compositing process, the arrangement determination unit further determines the arrangement state of the through image and the arranged image within the recorded image for the compositing process.

4. The image processing device according to claim 2,

wherein

the composite processing unit generates the composite image for display by compositing the through image in a compositing arrangement region set on the background image for the compositing process by the arrangement determination unit, and

when a face image of a person is included in an image other than the background image for the compositing process or the through image, the arrangement determination unit determines an orientation of the face in the face image, and sets the compositing arrangement region on the background image for the compositing process using the determination result.

5. The image processing device according to claim 2,

wherein

the composite processing unit generates the composite image for display by compositing the through image in a compositing arrangement region set on the background image for the compositing process by the arrangement determination unit and performs the generation of the composite image for display using a low-resolution image.

6. The image processing device according to claim 1, wherein the display control unit performs a process of causing the through image to be displayed and the composite image for display.

7. The image processing device according to claim 1, wherein the composite processing unit sets another composite image read from a recording medium to be the recorded image for the compositing process.

8. The image processing device according to claim 1,

wherein

the composite processing unit generates the composite image for display by compositing the through image in a compositing arrangement region set on the background image for the compositing process and sets the compositing arrangement region on the background image for the compositing process based on template information.

9. The image processing device according to claim 1,

wherein

the composite processing unit composites the through image on the background image for the compositing process, and generates the composite image for display by displaying a region of the through image to be further emphasized than any other region.

10. The image processing device according to claim 1, further comprising:

an output processing unit that outputs the recorded image for the compositing process at a certain time point as generated composite image data,

wherein the composite processing unit starts generating the composite image for display according to an instruction of a composite imaging mode, and

the output processing unit outputs the recorded image for the compositing process when it is instructed to end the composite imaging mode, as generated composite image data.

11. The image processing device according to claim 1, further comprising:

a recording unit that records data on a recording medium; and

an output processing unit that outputs the recorded image for the compositing process at a certain time point as generated composite image data,

wherein the output processing unit controls recording of the generated composite image data on the recording medium.

12. The image processing device according to claim 11, wherein the output processing unit further controls compositing management information including information relating to the background image of composite image data to be recorded on the recording medium in association with the composite image data.

13. The image processing device according to claim 1, further comprising:

a transmission unit that transmits data to an external device; and

an output processing unit that outputs the recorded image for the compositing process at a certain time point as generated composite image data,

wherein the output processing unit controls the generated composite image data to be transmitted and output from the transmission unit.

14. The image processing device according to claim 1, further comprising:

a display unit,

wherein the display control unit controls the composite image for display to be displayed on the display unit.

15. The image processing device according to claim 1, wherein the composite processing unit performs an edit process of changing content of an image in composite image data according to an operation input.

16. An image processing method comprising:

displaying a composite image for display by generating the composite image for display by including a through image, generated based on a captured image signal obtained by performing photoelectric conversion for light incident from a subject, in a recorded image for a compositing process,

wherein the recorded image is set as a background image in the composite image;

compositing a captured image of a recording time point in the recorded image for the compositing process in accordance with a compositing arrangement state of the through image in the composite image for display; and

performing a process of changing the background image based on a result of a comparison process between the through image and the background image, wherein the comparison process is a comparison process of image depth.

17. A non-transitory computer-readable storage medium having stored thereon, a set of computer-executable instructions, for causing a computer to perform steps comprising:

displaying a composite image for display by generating the composite image for display by including a through image, generated based on a captured image signal obtained by performing photoelectric conversion for light incident from a subject, in a recorded image for a compositing process,

wherein the recorded image is set as a background image in the composite image;

compositing a captured image of a recording time point in the recorded image for the compositing process in accordance with a compositing arrangement state of the through image in the composite image for display; and

performing a process of changing the background image based on a result of a comparison process between the through image and the background image, wherein the comparison process is a comparison process of image depth.