Detection method and system

• US 10,147,022 B2
• Filed: 04/28/2016
• Issued: 12/04/2018
• Est. Priority Date: 06/23/2015
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Yoshioka, Takahiro, Nakashima, Satoshi
• Primary Examiner(s): Kholdebarin, Iman K
• Assistant Examiner(s): TRAN, MAI H
• Application Number:
  US15/141,000
• Publication Number:
  US 20160379093A1
• Time in Patent Office: 950 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G06V 40/197 Matching; Classification
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A detection method executed by a computer, the detection method includes detecting a plurality of pupil candidates from a face image region in an image of a subject based on specific shape information, and identifying at least one pupil candidate as a pupil from among the plurality of pupil candidates based on brightness information related to an image region outside of the face image region and learning information indicating a relationship between the brightness information and a size of the pupil.

Claims

1. A detection method executed by a computer, the detection method comprising:

detecting one or more pupil candidates from a face image region corresponding to a face of a subject in an image that includes a subject based on specific shape information;

identifying an eye region in the image;

comparing brightness information of the eye region with brightness information of a background portion of the image that excludes the face of the subject;

identifying, when a result of the comparing indicates that a brightness of the eye region is brighter than a brightness of the background portion of the image that excludes the face of the subject, at least one pupil candidate as a pupil from among the one or more pupil candidates based on the brightness information of the background portion and based on relationship information indicating a relationship between the brightness information of the background portion and a size of the pupil; and

identifying, when the result of the comparing indicates that the brightness of the eye region is darker than the brightness of the background portion, the at least one pupil candidate from the one or more pupil candidates based on another relationship information indicating a relationship between the brightness information of the eye region and the size of the pupil.

2. The detection method according to claim 1, further comprising:

detecting learning pupils respectively from other face image regions in a plurality of learning images that are different from the image;

generating the relationship information based on other brightness information related to other background portions of images that exclude the face of the subject and sizes of the learning pupils; and

storing the relationship information into a memory.

3. The detection method according to claim 2, wherein the plurality of learning images are images captured in a photographic environment similar to a photographic environment for the image.

4. The detection method according to claim 1, wherein the image is a near-infrared image captured by a camera having sensitivity to near-infrared light.

5. The detection method according to claim 1, further comprising:

identifying the size of the pupil corresponding to the brightness information by using the relationship information;

determining a difference between the size and a size of each of the one or more pupil candidates; and

identifying the at least one pupil candidate from the one or more pupil candidates based on the difference.

6. The detection method according to claim 5, further comprising:

calculating a likelihood based on the specific shape information and a profile of each of the one or more pupil candidates; and

identifying the at least one pupil candidate from the one or more pupil candidates based on the difference and the likelihood.

7. The detection method according to claim 1, further comprising:

detecting cornea reflection of the subject from the image; and

detecting a line of sight of the subject based on a positional relationship between the cornea reflection and the pupil.

8. A detection device, comprising:

circuitry configured to;

detect one or more pupil candidates from a face image region corresponding to a face of a subject in an image that includes a subject based on specific shape information;

identify an eye region in the image;

compare brightness information of the eye region with brightness information of a background portion of the image that excludes the face of the subject;

identify, when a result of the comparing indicates that a brightness of the eye region is brighter than a brightness of the background portion of the image that excludes the face of the subject, at least one pupil candidate as a pupil from among the one or more pupil candidates based on the brightness information of the background portion and based on relationship information indicating a relationship between the brightness information of the background portion and a size of the pupil; and

identify, when the result of the comparing indicates that the brightness of the eye region is darker than the brightness of the background portion, the at least one pupil candidate from the one or more pupil candidates based on another relationship information indicating a relationship between the brightness information of the eye region and the size of the pupil.

9. The detection device according to claim 8, wherein the circuitry is further configured to:

detect learning pupils respectively from other face image regions in a plurality of learning images that are different from the image;

generate the relationship information based on other brightness information related to other background portions of images that exclude the face of the subject and sizes of the learning pupils; and

store the relationship information into a memory.

10. The detection device according to claim 9, wherein the plurality of learning images are images captured in a photographic environment similar to a photographic environment for the image.

11. The detection device according to claim 8, wherein the image is a near-infrared image captured by a camera having sensitivity to near-infrared light.

12. The detection device according to claim 8, wherein the circuitry is further configured to:

identify the size of the pupil corresponding to the brightness information by using the relationship information;

determine a difference between the size and a size of each of the one or more pupil candidates; and

identify the at least one pupil candidate from the one or more pupil candidates based on the difference.

13. The detection device according to claim 12, wherein the circuitry is further configured to:

calculate a likelihood based on the specific shape information and a profile of each of the one or more pupil candidates; and

identify the at least one pupil candidate from the one or more pupil candidates based on the difference and the likelihood.

14. The detection device according to claim 8, further comprising:

a light source configured to radiate light.

15. The detection device according to claim 14, further comprising:

a camera configured to capture the image.

16. A non-transitory computer-readable storage medium storing a detection program for causing a computer to execute a process, the process comprising:

detecting one or more pupil candidates from a face image region corresponding to a face of a subject in an image that includes a subject based on specific shape information;

identifying an eye region in the image;

comparing brightness information of the eye region with brightness information of a background portion of the image that excludes the face of the subject;

identifying, when a result of the comparing indicates that a brightness of the eye region is brighter than a brightness of the background portion of the image that excludes the face of the subject, at least one pupil candidate as a pupil from among the one or more pupil candidates based on the brightness information of the background portion and based on relationship information indicating a relationship between the brightness information of the background portion and a size of the pupil; and

identifying, when the result of the comparing indicates that the brightness of the eye region is darker than the brightness of the background portion, the at least one pupil candidate from the one or more pupil candidates based on another relationship information indicating a relationship between the brightness information of the eye region and the size of the pupil.