Photographing apparatus, method and program

• US 20050200736A1
• Filed: 01/19/2005
• Published: 09/15/2005
• Est. Priority Date: 01/21/2004
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Ito, Wataru
• Granted Patent: US 7,327,886 B2
• Time in Patent Office: Days
• US Class Current: 348/333.10
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/2112 using still video cameras
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

Processing for judging whether a face is included in a frame is performed, in a predetermined interval, on each of frames included in a moving image of a subject, displayed on a monitor, until the judgment becomes positive. If it is judged that a face is included in a frame, the facial position is detected in the frame, and stored. Then, judgment is made as to whether a face is included in the next frame after predetermined time. If the judgment is positive, the facial position is detected. The previously stored facial position is replaced by the newly detected facial position, and the newly detected facial position is stored. These processes are repeated until photographing operation is performed by operating a release unit.

Claims

1. A photographing apparatus comprising:

a photographing means for obtaining image data by taking a photograph of a subject;

a display means for displaying various kinds of information including the image data;

a release means for performing photographing operation;

a storage means for storing various kinds of information including the image data;

a photographing control means for obtaining a moving image of the subject by continuously taking photographs with the photographing means and displaying the moving image on the display means;

a face judgment means for performing processing, in a predetermined time interval, for judging whether a human face is included in a frame included in the moving image until a positive result is obtained in the judgment;

a face detection means for detecting a facial position in a frame, which is judged to include a face, if the face judgment means judges that the face is included in the frame; and

a control means for controlling the photographing means, the face judgment means, the face detection means, and the storage means so that the detected facial position is stored in the storage means, judgment is made as to whether the face is included in the next frame after the predetermined time, and if the judgment is positive, the facial position is detected, the facial position, which is stored in the storage means, is replaced by the newly detected facial position and the newly detected facial position is stored in the storage means, and until the release means performs the photographing operation, judgment is made as to whether the face is included in the next frame further after the predetermined time, and if it is judged that the face is included in the frame, the facial position is detected, and the newly detected facial position is stored in the storage means, and a frame obtained when the release means performs the photographing operation and/or a plurality of temporally precedent and/or subsequent frames of the obtained frame are stored in the storage means as the image data.

2. A photographing apparatus as defined in claim 1, further comprising:

a variation judgment means for referring to the frame, which was judged to include the face, and its next frame, thereby judging whether the variation in an image between the two frames exceeds a predetermined value, wherein the control means is a means for controlling the face judgment means so that if the variation judgment means judges that the variation is less than or equal to the predetermined value, the face judgment means performs processing, only on the region, including the facial position stored in the storage means, and the vicinity of the facial position, to judge whether the face is included in the next frame, and if the variation judgment means judges that the variation exceeds the predetermined value, the face judgment means performs processing, on the whole region of the next frame, to judge whether the face is included in the next frame.

3. A photographing apparatus as defined in claim 2, wherein the variation is one of the absolute value of the difference in the average value of pixel values in an image between the two frames and the difference in the histogram shape of the image between the two frames.

4. A photographing apparatus as defined in claim 1, further comprising:

an image processing means for performing predetermined image processing, related to a face, on the image data by referring to the facial position stored in the storage means after the photographing operation is performed.

5. A photographing apparatus as defined in claim 4, wherein the image processing means is a means for performing at least one of red-eye correction processing, noise suppression processing on the region of the image representing a face, density correction processing, and gradation correction processing as the predetermined image processing.

6. A photographing apparatus as defined in claim 4, wherein the image processing means is a means for performing processing for judging whether the eyes in the detected face are closed, and if the eyes are closed, selecting a frame including a face with open eyes from the temporally precedent and/or subsequent frames of the frame, which was obtained when the photographing operation was performed, as the predetermined image processing.

7. A photographing apparatus as defined in claim 4, wherein the image processing means is a means for performing processing for judging whether the detected face is a smiling face, and if it is not the smiling face, selecting a frame including a smiling face from the temporally precedent and/or subsequent frames of the frame, which was obtained when the photographing operation was performed, as the predetermined image processing.

8. A photographing apparatus as defined in claim 1, wherein the face judgment means is a means for performing the judgment by using reference data, obtained by learning by a machine learning technique.

9. A photographing apparatus as defined in claim 1, wherein the photographing apparatus is a digital camera.

10. A photographing method at a photographing apparatus including a photographing means for obtaining image data by taking a photograph of a subject, a display means for displaying various kinds of information including the image data, a release means for performing photographing operation, a storage means for storing various kinds of information including the image data, and a photographing control means for obtaining a moving image of the subject by continuously taking photographs with the photographing means and displaying the moving image on the display means, the method comprising the steps of:

performing processing, in a predetermined time interval, for judging whether a human face is included in a frame included in the moving image until a positive result is obtained in the judgment;

detecting a facial position in a frame, which is judged to include a face, if it is judged that the face is included in the frame; and

storing the detected facial position in the storage means, judging whether the face is included in the next frame after the predetermined time, and if the judgment is positive, detecting the facial position, replacing the facial position, which is stored in the storage means, by the newly detected facial position and storing the newly detected facial position in the storage means, and until the release means performs the photographing operation, judging whether the face is included in the next frame further after the predetermined time, and if it is judged that the face is included in the frame, detecting the facial position, and storing the newly detected facial position in the storage means, and storing a frame, which is obtained when the release means performs the photographing operation and a plurality of temporally precedent and/or subsequent frames of the obtained frame in the storage means as the image data.

11. A program for causing a computer to execute an image processing method at a photographing apparatus including a photographing means for obtaining image data by taking a photograph of a subject, a display means for displaying various kinds of information including the image data, a release means for performing photographing operation, a storage means for storing various kinds of information including the image data, and a photographing control means for obtaining a moving image of the subject by continuously taking photographs with the photographing means and displaying the moving image on the display means, the program comprising the procedures for:

performing processing, in a predetermined time interval, for judging whether a human face is included in a frame included in the moving image until a positive result is obtained in the judgment;

detecting a facial position in a frame, which is judged to include a face, if it is judged that the face is included in the frame; and

storing the detected facial position in the storage means, judging whether the face is included in the next frame after the predetermined time, and if the judgment is positive, detecting the facial position, replacing the facial position, which is stored in the storage means, by the newly detected facial position and storing the newly detected facial position in the storage means, and until the release means performs the photographing operation, judging whether the face is included in the next frame further after the predetermined time, and if it is judged that the face is included in the frame, detecting the facial position, and storing the newly detected facial position in the storage means, and storing a frame, which is obtained when the release means performs the photographing operation and/or a plurality of temporally precedent and/or subsequent frames of the obtained frame in the storage means as the image data.