IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM

• US 20130162672A1
• Filed: 11/29/2012
• Published: 06/27/2013
• Est. Priority Date: 12/21/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: SEN, Seigo
• Granted Patent: US 9,251,765 B2
• Time in Patent Office: Days
• US Class Current: 345/632
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: G09G 5/393 Arrangements for updating the contents of the bit-mapped memory
• CPC Class Codes: G09G 5/397 Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay G09G5/02 takes precedence
• CPC Class Codes: H04N 1/00161 Viewing or previewing
• CPC Class Codes: H04N 1/00167 Processing or editing H04N1/00196 - H04N1/00201 take precedence
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/42661 for reading from or writing on a magnetic storage medium, e.g. hard disk drive
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4363 Adapting the video stream to a specific local network, e.g. a Bluetooth® network
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440263 by altering the spatial resolution, e.g. for displaying on a connected PDA
• CPC Class Codes: H04N 21/8153 comprising still images, e.g. texture, background image
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 5/272 : Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/445 : for displaying additional information H04N5/50 takes precedence

Abstract

An image processing device includes a composite processing unit that generates a composite image for display by including a through image generated based on a captured image signal obtained by performing photoelectric conversion for light incident from a subject in an image for a compositing process and composites a captured image of a recording time point in the image for a compositing process in accordance with a compositing arrangement state of the through image in the composite image for display, and a display control unit that performs display control for the composite image for display.

Claims

1. An image processing device comprising:

a composite processing unit that generates a composite image for display by including a through image generated based on a captured image signal obtained by performing photoelectric conversion for light incident from a subject in an image for a compositing process and composites a captured image of a recording time point in the image for a compositing process in accordance with a compositing arrangement state of the through image in the composite image for display; and

a display control unit that performs display control for the composite image for display.

2. The image processing device according to claim 1, further comprising:

an arrangement determination unit that determines an arrangement state of the through image within the image for a compositing process according to the content of the captured image signal,

wherein the composite processing unit composites the through image on the image for a compositing process in an arrangement state determined by the arrangement determination unit so as to generate the composite image for display.

3. The image processing device according to claim 2, wherein, according to an arranged image which has already been arranged in the image for a compositing process, the arrangement determination unit further determines an arrangement state of the through image and the arranged image within the image for a compositing process.

4. The image processing device according to claim 1, wherein the display control unit performs a process of causing the through image to be displayed and the composite image for display that has been composited with the through image to be displayed.

5. The image processing device according to claim 1,

wherein the image for a compositing process is an image obtained by setting a still image to be a background image, and

the composite processing unit sets a still image obtained from imaging or a still image read from a recording medium to be a background image in the image for a compositing process.

6. The image processing device according to claim 1, wherein the composite processing unit sets a composite image read from a recording medium to be the image for a compositing process.

7. The image processing device according to claim 1,

wherein the image for a compositing process is an image at least including a background image, and

the composite processing unit performs a process of changing the background image based on a result of a comparison process between the through image and the background image.

8. The image processing device according to claim 7, wherein the comparison process is at least one of a comparison process for obtaining the similarity between images, a comparison process of image evaluation values, and a comparison process of image depth.

9. The image processing device according to claim 1,

wherein the image for a compositing process is an image at least including a background image, and

the composite processing unit generates the composite image for display by compositing the through image in a compositing arrangement region set on the background image of the image for a compositing process and sets the compositing arrangement region on the background image in the image for a compositing process based on template information.

10. The image processing device according to claim 2,

wherein the image for a compositing process is an image at least including a background image,

the composite processing unit generates the composite image for display by compositing the through image in a compositing arrangement region set on the background image of the image for a compositing process by the arrangement determination unit, and

when a face image of a person is included in an image other than the background image included in the image for a compositing process or the through image, the arrangement determination unit determines the orientation of the face in the face image, and sets a compositing arrangement region on the background image in the image for a compositing process using the determination result.

11. The image processing device according to claim 2,

wherein the image for a compositing process is an image at least including a background image, and

the composite processing unit generates the composite image for display by compositing the through image in a compositing arrangement region set on the background image of the image for a compositing process by the arrangement determination unit and performs the generation of the composite image for display using a low-resolution image.

12. The image processing device according to claim 1,

wherein the image for a compositing process is an image at least including a background image, and

the composite processing unit composites the through image on the background image of the image for the compositing process, and generates the composite image for display by displaying the region of the through image to be further emphasized than any other region.

13. The image processing device according to claim 1, further comprising:

an output processing unit that outputs the image for a compositing process at a certain time point as generated composite image data,

wherein the composite processing unit starts generating the composite image for display according to an instruction of a composite imaging mode, and

the output processing unit outputs the image for a compositing process at the time when it is instructed to end the composite imaging mode, as generated composite image data.

14. The image processing device according to claim 1, further comprising:

a recording unit that records data on a recording medium; and

an output processing unit that outputs the image for a compositing process at a certain time point as generated composite image data,

wherein the output processing unit controls recording of the generated composite image data on a recording medium.

15. The image processing device according to claim 14, wherein the output processing unit further controls compositing management information including information relating to the background image of composite image data to be recorded on a recording medium in association with the composite image data.

16. The image processing device according to claim 1, further comprising:

a transmission unit that transmits data to an external device; and

an output processing unit that outputs the image for a compositing process at a certain time point as generated composite image data,

wherein the output processing unit controls the generated composite image data to be transmitted and output from the transmission unit.

17. The image processing device according to claim 1, further comprising:

a display unit,

wherein the display control unit controls the composite image for display to be displayed on the display unit.

18. The image processing device according to claim 1, wherein the composite processing unit performs an edit process of changing the content of an image in composite image data according to an operation input.

19. An image processing method comprising:

displaying a composite image for display by generating the composite image for display by including a through image generated based on a captured image signal obtained by performing photoelectric conversion for light incident from a subject in an image for a compositing process; and

compositing a captured image of a recording time point in the image for a compositing process in accordance with a compositing arrangement state of the through image in the composite image for display.

20. A program which causes an operation processing device to execute processes of:

displaying a composite image for display by generating the composite image for display by including a through image generated based on a captured image signal obtained by performing photoelectric conversion for light incident from a subject in an image for a compositing process; and

compositing a captured image of a recording time point in the image for a compositing process in accordance with a compositing arrangement state of the through image in the composite image for display.