IMAGE PROCESSING DEVICE AND IMAGE PROCESSING METHOD

• US 20100066840A1
• Filed: 01/10/2008
• Published: 03/18/2010
• Est. Priority Date: 02/15/2007
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tsuruta, Masaaki, Sako, Yoichiro, Ito, Taiji, Asukai, Masamichi
• Granted Patent: US 8,482,651 B2
• Time in Patent Office: Days
• US Class Current: 348/207.100
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

Among pickup image data items of multiple frames picked up at different time instants, an image of a certain frame is regarded as a synthesizing base image. The synthesizing base image and an image of another frame are used to perform synthesizing processing. Specifically, an object-of-synthesis area is designated in the frame image regarded as the synthesizing base image, and processing of replacing pixel data in the object-of-synthesis area with pixel data in another frame is carried out. For example, if a person having his/her eyes closed appears in the synthesizing base image that is a group-photograph image, the part of the person'"'"'s face is regarded as the object-of-synthesis area. As another frame, an image in which the person has his/her eyes left open is searched. Pixel data in the facial image part showing the open eyes is fetched and synthesized with the synthesizing base image. Thus, a group photograph in which all people have their eyes left open is produced.

Claims

1. An image processing device comprising:

an acquisition means to acquire pickup image data items of multiple frames, which are picked up at different times, as pickup image data items to be used for synthesizing processing;

an image synthesis means to designate an object-of-synthesis area in a selected frame out of the multiple frames acquired by the acquisition means, and produce synthetic image data by performing synthesizing processing of replacing pixel data in the object-of-synthesis area with pixel data in another frame out of the multiple frames; and

a preserving processing means to perform preserving processing on the synthetic image data produced by the image synthesis means.

2. The image processing device according to claim 1, wherein the acquisition means includes an imaging unit to perform image pickup, and is adapted to acquire pickup image data of at least one frame, which is included in the pickup image data items to be used for the synthesizing processing, through imaging by the imaging unit.

3. The image processing device according to claim 2, wherein the acquisition means acquires pickup image data items of multiple frames, which are used for the synthesizing processing, through imaging by the imaging unit.

4. The image processing device according to claim 2, wherein the imaging unit is formed using a CCD sensor or a CMOS sensor as an image sensor.

5. The image processing device according to claim 1, wherein the acquisition means includes a communication unit to communicate with external equipment, and is adapted to receive and acquire pickup image data of at least one frame, which is included in the pickup image data items to be used for the synthesizing processing, through communication with the external equipment by the communication unit.

6. The image processing device according to claim 1, wherein the acquisition means includes a reproduction unit to reproduce information from a portable recording medium, and is adapted to acquire pickup image data of at least one frame, which is included in the pickup image data items to be used for the synthesizing processing, through reproduction from the portable recording medium by the reproduction unit.

7. The image processing device according to claim 1, wherein the preserving processing means includes a recording unit to record information in a recording medium, and is adapted to perform the preserving processing by recording the synthetic image data in the recording medium using the recording unit.

8. The image processing device according to claim 1, wherein the preserving processing means includes a communication unit to communicate with external equipment, and as is adapted to perform the preserving processing by transmitting the synthetic image data to the external equipment using the communication unit.

9. The image processing device according to claim 1, wherein the image synthesis means is adapted to designate the object-of-synthesis area through image analyzing processing.

10. The image processing device according to claim 9, wherein the image synthesis means is adapted to discriminate all or part of the opening or closing of the eyes in a facial image of a person contained in a frame image, the facial expression, the direction of a face, and the color of the eyes through image analyzing processing, and designates the object-of-synthesis area on the basis of a result of the discrimination.

11. The image processing device according to claim 1, further comprising an operating input means, wherein

the image synthesis means is adapted to designate the object-of-synthesis area responsive to entry performed at the operating input means.

12. The image processing device according to claim 1, wherein the image synthesis means is adapted to determine a frame to be used as another frame through image analyzing processing performed on each of the multiple frames, and fetch pixel data, which is synthesized with the object-of-synthesis area, from the determined frame.

13. The image processing device according to claim 1, further comprising an operating input means, wherein

the image synthesis means selects is adapted to select a frame to be used as another frame from among the multiple frames responsively to entry performed at the operating input means, and fetch pixel data, which is synthesized with the object-of-synthesis area, from the selected frame.

14. The image processing device according to claim 1, further comprising an image output means to perform outputting processing on the synthetic image data.

15. The image processing device according to claim 14, wherein the image output means includes a display unit, and is adapted to perform the outputting process by displaying or outputting the synthetic image data on or to the display unit.

16. The image processing device according to claim 14, wherein the image output means includes a communication unit to communicate with external equipment, and is adapted to perform as the outputting processing transmission or output of the synthetic image data to the external equipment via the communication unit.

17. The image processing device according to claim 14, wherein the image output means includes a recording unit record information in a portable recording medium, and is adapted to perform the outputting processing by recording the synthetic image data in the portable recording medium via the recording unit.

18. An image processing method characterized by comprising:

acquiring pickup image data items of multiple frames, which are picked up at different times, as pickup image data items to be used for synthesizing processing;

designating an object-of-synthesis area in a selected frame out of the acquired multiple frames, and producing synthetic image data by performing synthesizing processing of replacing pixel data in the object-of-synthesis area with pixel data in another frame out of the multiple frames; and

performing preserving processing on the synthetic image data.