Image selection device and method for selecting image

• US 8,416,312 B2
• Filed: 03/30/2010
• Issued: 04/09/2013
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: Matsunaga, Kazuhisa
• Primary Examiner(s): LAM, HUNG H
• Application Number:
  US12/749,667
• Publication Number:
  US 20100245614A1
• Time in Patent Office: 1,106 Days
• Field of Search: 348/222.1, 348/231.2, 348/231.3, 348/78
• US Class Current: 348/222.1
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

An image selection device including: a reception unit that receives a plurality of captured images obtained by rapid shooting one or more people as subjects; a face detection unit that detects human faces included in the captured images received by the reception unit; an eye detection unit that detects eyes in the human faces detected by the face detection unit; a blink detection unit that detects blink degrees of each of the eyes detected by the eye detection unit; an estimation unit that estimates statuses of the human faces based on the blink degrees detected by the blink detection unit; and a selecting unit that selects at least one of the captured images to be recorded on a recording medium from among the captured images received by the reception unit, based on the statuses of the human faces estimated by the estimation unit.

Claims

1. An image selection device comprising:

a reception unit that receives a plurality of captured images obtained by rapid shooting one or more people as subjects;

a face detection unit that detects human faces included in each of the captured images received by the reception unit;

an eye detection unit that detects eyes in each of the human faces detected by the face detection unit;

a blink detection unit that detects blink degrees of each of the eyes detected by the eye detection unit; and

a selecting unit that selects at least one captured image which includes eyes having blink degrees higher than the blink degrees of the eyes included in the other captured images from among the captured images received by the reception unit.

2. The image selection device according to claim 1, further comprising:

a threshold calculation unit that calculates a threshold for determining whether the eyes of each of the people are opened or not, based on the blink degree of the eyes of each of the people detected by the blink detection unit; and

a blink determination unit that determines whether the eyes of each of the people are opened or not in each of the captured images received by the reception unit, based on the threshold calculated by the threshold calculation unit.

3. The image selection device according to claim 2, further comprising:

a first image determination unit that determines whether or not there are a plurality of captured images with a minimum number of faces with closed eyes in the captured images received by the reception unit, based on a result of the determination by the blink determination unit,

wherein, when the first image determination unit determines that there is not a plurality of captured images with the minimum number of faces with closed eyes, the selecting unit selects the captured image with the minimum number of faces with closed eyes from among the plurality of captured images as one captured image to be recorded on a recording medium.

4. The image selection device according to claim 3, further comprising:

a second image determination unit that determines, when the first image determination unit determines that there are a plurality of captured images with the minimum number of faces with closed eyes, whether the minimum number of faces with closed eyes is zero or not,

wherein, when the second image determination unit determines that the minimum number of faces with closed eyes is zero, the selecting unit selects the captured image to be recorded on the recording medium based on the blink degrees of the eyes of each of the people detected by the blink detection unit.

5. The image selection device according to claim 3, further comprising:

a smile detection unit that detects smile degrees of the faces with the eyes detected by the eye detection unit based on position information of the detected eyes; and

a second image determination unit that determines, when the first image determination unit determines that there are a plurality of captured images with the minimum number of faces with closed eyes, whether the minimum number of faces with closed eyes is zero or not,

wherein, when the second image determination unit determines that the minimum number of faces with closed eyes is not zero, the selecting unit selects the captured image to be recorded on the recording medium based on the smile degrees detected by the smile detection unit.

6. The image selection device according to claim 1, wherein the reception unit comprises a ring buffer that temporarily stores a predetermined number of image frames.

7. The image selection device according to claim 1, further comprising:

an imaging unit that captures the captured images,

wherein a number of images continuously taken by the imaging unit is set in accordance with the faces detected by the face detection unit.

8. The image selection device according to claim 1, further comprising:

a blurring detection unit that detects a blurring amount of each of the captured images received by the reception unit relatively to an adjacent image when no human face is detected by the face detection unit,

wherein the selecting unit selects a captured image with the smallest blurring amount detected by the blurring detection unit, as the captured image to be recorded on a recording medium.

9. The image selection device according to claim 1, further comprising:

an estimation unit that estimates statuses of the human faces based on the blink degrees detected by the blink detection unit,

wherein the selecting unit selects at least one of the captured images including the eyes having the blink degrees higher than the blink degrees of the eyes included in the other captured images from among the captured images received by the reception unit, based on the statuses of the human faces estimated by the estimation unit.

10. The image selection device according to claim 9, further comprising:

a smile detection unit that detects smile degrees of the faces with the eyes detected by the eye detection unit based on position information of the detected eyes; and

a blink correction unit that corrects the blink degrees detected by the blink detection unit based on the smile degrees detected by the smile detection unit,

wherein the selecting unit determines the statuses of the human faces in accordance with the blink degrees corrected by the blink correction unit.

11. The image selection device according to claim 9, further comprising:

a reliability calculation unit that calculates reliability as to detection of eyes performed by the eye detection unit; and

a validity determination unit that determines validity of each face with the detected eyes associated with the reliability calculated by the reliability calculation unit based on whether the reliability is higher than a predetermined threshold or not,

wherein the selecting unit determines the statuses of the human faces determined to be valid by the validity determination unit.

12. The image selection device according to claim 11, further comprising:

a blurring detection unit that detects a blurring amount of each of captured images which belongs to the captured images received by the reception unit and which includes faces determined to be valid by the validity determination unit, relatively to an adjacent image; and

a third image determination unit that determines whether or not there are a plurality of captured images whose blurring amounts detected by the blurring detection unit are smaller than a predetermined value,

wherein, when the third image determination unit determines that there are a plurality of captured images whose blurring amounts are smaller than the predetermined value, the blink detection unit detects blink degrees of the eyes detected by the eye detection unit.

13. The image selection device according to claim 12, further comprising:

a blurring correction unit that corrects the blurring amount of each captured image detected by the blurring detection unit in accordance with a position of blurring relative to positions of faces,

wherein the image selecting unit selects a captured image whose blurring amount is smaller than a predetermined value based on the blurring amount of each captured image corrected by the blurring correction unit.

14. The image selection device according to claim 12, wherein, when the third image determination unit determines that there are not a plurality of captured images whose blurring amounts are smaller than the predetermined value, the selecting unit selects a captured image with the smallest blurring amount as the captured image to be recorded on a recording medium.

15. The image selection device according to claim 9, further comprising:

a smoothing unit that smoothes the blink degrees detected by the blink detection unit between adjacent ones of the captured images when an imaging frame rate is not lower than a predetermined value,

wherein the selecting unit determines the statuses of the human faces in accordance with the blink degrees smoothed by the smoothing unit.

16. The image selection device according to claim 9, wherein the captured images received by the reception unit are sorted based on the estimation by the estimation unit.

17. The image selection device according to claim 9, further comprising:

a person registration unit that allows a user to register a person,

wherein, when a face detected by the face detection unit belongs to the person registered in the person registration unit, the estimation unit corrects an estimated value concerned with the face.

18. A method for selecting an image, the method comprising:

receiving a plurality of captured images obtained by rapid shooting one or more people as subjects;

performing a face detection process for detecting human faces included in each of the captured images;

performing an eyes detection process for detecting eyes in each of the human faces detected by the face detection process;

performing a blink detection process for detecting blink degrees of each of the eyes detected by the eye detection process; and

selecting at least one captured image which includes eyes having blink degrees higher than the blink degrees of the eyes included in the other captured images from among the received captured images.

19. A non-transitory computer readable medium having stored thereon a program that is executable by a computer to perform image processing for selecting an image, the image processing comprising:

receiving a plurality of captured images obtained by rapid shooting one or more people as subjects;

performing a face detection process for detecting human faces included in each of the captured images;

performing an eyes detection process for detecting eyes in each of the human faces detected by the face detection process;

performing a blink detection process for detecting blink degrees of each of the eyes detected by the eye detection process; and

selecting at least one captured image which includes eyes having blink degrees higher than the blink degrees of the eyes included in the other captured images from among the received captured images.