Image processing apparatus, image processing method, image processing program, and image pickup apparatus

• US 8,026,932 B2
• Filed: 09/25/2007
• Issued: 09/27/2011
• Est. Priority Date: 09/27/2006
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Izumisawa, Akifumi, Wakamatsu, Takashi, Chujo, Kaoru, Utsugi, Kiyoshi, Murashita, Kimitaka, Shimizu, Masayoshi
• Primary Examiner(s): AMINI, JAVID A
• Application Number:
  US11/860,865
• Publication Number:
  US 20080074441A1
• Time in Patent Office: 1,463 Days
• Field of Search: 345/634, 345/667, 345/629
• US Class Current: 345/634
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

An image pickup apparatus includes: a first base image selection unit selecting a piece of image data as a first base image used in superposition; a shift amount calculation unit calculating an amount of shift between the first base image and another different piece of image data; an image superposition unit detecting a superposing area as an area in which the different piece of image data can be superposed on the first base image after a shift correction is made on a basis of the calculated amount of shift, and performing superposition for the superposing area; a second base image selection unit selecting a piece of image data as a second base image used in a non-superposing area; and an image combination unit combining a superposition result with the non-superposing area in the second base image, and outputting image data as a combination result.

Claims

1. An image data processing apparatus which processes plural pieces of image data, comprising:

a first base image selection unit which selects a piece of image data in plural pieces of taken image data as a first base image used in superposition;

a shift amount calculation unit which calculates an amount of shift between an image region of a background portion included in the first base image selected for superposition and an image region of the same background portion included in another different piece of image data in the plural pieces of taken image data;

an image superposition unit which detects a superposing area as an area in which the different piece of image data can be superposed on the selected first base image after a shift correction is made on a basis of the calculated amount of shift, and which performs superposition for the superposing area;

a second base image selection unit which selects a piece of image data in the plural pieces of taken image data as a second base image used in a non-superposing area; and

an image combination unit which combines a superposition result with the non-superposing area in the second base image, and which outputs image data as a combination result.

2. The apparatus according to claim 1, wherein

the second base image selection unit selects image data having a smallest amount of shake in the image data in the plural pieces of image data as the second base image used in the non-superposing area.

3. The apparatus according to claim 2, wherein

the image data having the smallest amount of shake refers to image data having a highest resolution in an edge portion detected in the image data.

4. The apparatus according to claim 2, wherein

the image data having the smallest amount of shake refers to image data having a highest resolution of a feature point portion extracted in the image data.

5. The apparatus according to claim 1, further comprising

a face recognition unit which recognizes an open level of eyes on a human face in the non-superposing area in the taken image data, wherein

the second base image selection unit selects, as the second base image used in the non-superposing area, image data recognized by the face recognition unit as indicating largest open eyes on the human face.

6. An image pickup apparatus which processes plural pieces of image data obtained by continuous shooting, comprising:

an image sensor which performs an optical/electrical conversion on light received from an object to be captured and which outputs optical/electrically converted data;

an image data generation unit which generates image data on a basis of the optical/electrically converted data;

a shooting control unit which outputs exposure control information in response to a shoot instruction to the image sensor and which takes plural pieces of image data;

a first base image selection unit which selects a piece of image data in the plural pieces of taken image data as a first base image used in superposition;

a shift amount calculation unit which calculates an amount of shift between an image region of a background portion included in the first base image selected for superposition and an image region of the same background portion included in another different piece of image data in the plural pieces of taken image data;

an image superposition unit which detects a superposing area as an area in which the different piece of image data can be superposed on the selected first base image after a shift correction is made on a basis of the calculated amount of shift, and which performs superposition for the superposing area;

a second base image selection unit which selects a piece of image data in the plural pieces of taken image data as a second base image used in a non-superposing area; and

an image combination unit which combines a superposition result with the non-superposing area in the second base image, and which outputs image data as a combination result.

7. The apparatus according to claim 6, wherein

the second base image selection unit selects image data corresponding to a first shot image in the plural pieces of taken image data as the second base image used in the non-superposing area.

8. The apparatus according to claim 6, wherein

the second base image selection unit selects image data corresponding to an image shot with timing of a shutter sound in the plural pieces of taken image data as the second base image used in the non-superposing area.

9. An image data processing method a processor processes plural pieces of image data, comprising:

selecting a piece of image data in plural pieces of taken image data as a first base image used in superposition;

calculating an amount of shift between an image region of a background portion included in the first base image selected for superposition and an image region of the same background portion included in another different piece of image data in the plural pieces of taken image data;

detecting a superposing area as an area in which the different piece of image data can be superposed on the selected first base image after a shift correction is made on a basis of the calculated amount of shift, and performing superposition for the superposing area;

selecting a piece of image data in the plural pieces of taken image data as a second base image used in a non-superposing area; and

combining a superposition result with the non-superposing area in the second base image, and outputting image data as a combination result.

10. The method according to claim 9, wherein

in the selecting the piece of image data in the plural pieces of taken image data as the second base image, image data having a smallest amount of shake in the image data in the plural pieces of image data is selected as the second base image used in the non-superposing area.

11. The method according to claim 10, wherein

the image data having the smallest amount of shake refers to image data having a highest resolution in an edge portion detected in the image data.

12. The method according to claim 10, wherein

the image data having the smallest amount of shake refers to image data having a highest resolution of a feature point portion extracted in the image data.

13. The method according to claim 9, further comprising

recognizing an open level of eyes on a human face in the non-superposing area in the taken image data, wherein

in the selecting the piece of image data in the plural pieces of taken image data as the second base image , image data recognized in the recognizing as indicating largest open eyes on the human face is selected as the second base image used in the non-superposing area.

14. A computer-readable non-transitory storage medium storing an image data processing program to direct a computer to process plural pieces of image data, the image data processing program comprising:

selecting a piece of image data in plural pieces of taken image data as a first base image used in superposition;

calculating an amount of shift between an image region of a background portion included in the first base image selected for superposition and an image region of the same background portion included in another different piece of image data in the plural pieces of taken image data;

detecting a superposing area as an area in which the different piece of image data can be superposed on the selected first base image after a shift correction is made on a basis of the calculated amount of shift, and performing superposition for the superposing area;

selecting a piece of image data in the plural pieces of taken image data as a second base image used in a non-superposing area; and

combining a superposition result with the non-superposing area in the second base image, and outputting image data as a combination result.

15. The storage medium according to claim 14, wherein

in the selecting the piece of image data in the plural pieces of taken image data as the second base image, the image data processing program directs the computer to select image data having a smallest amount of shake in the image data in the plural pieces of image data as the second base image used in the non-superposing area.

16. The storage medium according to claim 15, wherein

the image data having the smallest amount of shake refers to image data having a highest resolution in an edge portion detected in the image data.

17. The storage medium according to claim 15, wherein

the image data having the smallest amount of shake refers to image data having a highest resolution of a feature point portion extracted in the image data.

18. The storage medium according to claim 14, wherein

the image data processing program further comprises recognizing an open level of eyes on a human face in the non-superposing area in the taken image data, wherein

in the selecting the piece of image data in the plural pieces of taken image data as the second base image, the image data processing program directs the computer to select, as the second base image used in the non-superposing area, image data recognized in the recognizing as indicating largest open eyes on the human face.