HUMAN FACE RECOGNITION AND USER INTERFACE SYSTEM FOR DIGITAL CAMERA AND VIDEO CAMERA

• US 20080279425A1
• Filed: 04/02/2008
• Published: 11/13/2008
• Est. Priority Date: 04/13/2007
• Status: Active Grant

• Current Assignee: MIRC Electronics Limited
• Original Assignee: MIRC Electronics Limited
• Inventors: Tang, Chi Sun Jackson
• Granted Patent: US 8,285,006 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

The present invention discloses a face recognition system which includes a face recognition method and associate apparatus. The disclosed method provides improved face matching accuracy by introduction of user intervention. When a target face to be recognized has been detected, a candidate list is generated and the user is allowed to actively select a matching candidate face out of the candidate list by entering a user input through a user interface provided by the disclosed apparatus. Data of the target face are then automatically registered to a database where the candidate face data are stored for updating the data of the matching candidate.

Claims

1. A face recognition method, comprising:

providing an image;

defining a target face from said image, said target face having a set of extracted target facial features;

generating a candidate list of one or more candidate faces for said target face, each candidate face on said candidate list having a set of candidate facial features thereof;

determining a matching candidate face out of said candidate list based on comparing said target facial features with said candidate facial features; and

updating the candidate facial features of said matching candidate face with said target facial features.

2. A face recognition method, comprising:

providing an image;

defining a target face from said image, said target face having a set of extracted target facial features;

generating a candidate list of one or more candidate faces, each candidate face having a set of candidate facial features thereof;

comparing said target face with each said candidate face based on said target facial features and said candidate facial features to determine a first matching candidate face out of said candidate list;

receiving user input, said user input designating a second matching candidate face out of said candidate list, wherein in response to said user input

if said first and second matching candidates are identical, updating the candidate facial features of said first matching candidate face with said target facial features, otherwise updating the candidate facial features of said second matching candidate face with said target facial features.

3. The method according to claim 2, said user input being entered through a graphical user interface by modifying the appearance of at least one graphical feature representing at least one program.

4. The method according to claim 2, the step of defining said target face further comprising:

detecting n potential target faces from said image, n being a natural number; and

receiving an additional user input, said additional user input designating the target face out of said n potential target faces; and

providing the designated target face for subsequent processing.

5. The method according to claim 2, further comprising:

receiving an additional user input, said additional user input designating that at least one of said candidate face on said candidate list is to be added to the database where said candidate facial features data are stored; and

adding said at least one of candidate face on said candidate list the candidate features thereof to the database where said candidate facial features data are stored.

6. The method according to claim 2, further comprising:

receiving further user input, said further user input designating that said first matching candidate face is to be updated with said target face; and

in response to receiving said further user input, the candidate facial features of said first matching candidate face is updated with said target facial features.

7. The method according to claim 2, said image is provided via a process comprising:

providing a video;

detecting, in a frame of said video at a certain frame numbered n, an arbitrary target face, said arbitrary target face having a set of arbitrary target facial features at frame numbered n;

detecting, in a subsequent frame of said video at frame numbered n+1, said arbitrary target face, said arbitrary target face having a set of arbitrary target facial features at frame numbered n+1; and

in response to detecting the arbitrary target face, comparing said arbitrary target facial features at frame numbered n and frame numbered n+1, when the comparison result indicates that said arbitrary target facial features at frame numbered n+1 is different from said arbitrary target facial features at frame numbered n, marking said frame at frame numbered n+1 as said image to be displayed, otherwise marking said frame at frame numbered n as said image to be displayed.

8. The method according to claim 7, said arbitrary target face being said target face and said arbitrary target facial features being said target facial features.

9. The method according to claim 2, said target facial features are determined via a process comprising:

providing a video;

detecting and estimating a score for, in a frame of said video at a certain frame numbered n, an arbitrary target face at frame numbered n, and extracting a set of arbitrary target facial features at frame numbered n for said arbitrary target face at frame numbered n;

detecting and estimating a score for, in a subsequent frame of said video at frame numbered n+1 said arbitrary target face at frame numbered n+1, and extracting said set of arbitrary target facial features at frame numbered n+1 for said arbitrary target face at frame numbered n+1; and

in response to estimating score for the arbitrary target face, comparing said score of said arbitrary target face at frame numbered n and frame numbered n+1, when the comparison result indicates that the score of said arbitrary target face at frame numbered n+1 is higher than that of said arbitrary target face at frame numbered n, marking said arbitrary target facial features at frame numbered n+1 as said arbitrary target facial features at frame numbered n and said target facial features as well, otherwise marking said arbitrary target facial features at frame numbered n as said arbitrary target facial features at frame numbered n and said target facial features as well.

10. The method according to claim 9, the score estimation is based on face orientation, clarity, shape, lighting condition, shadowing, and other measures of said arbitrary target face at a certain frame numbered n and a subsequent frame numbered n+1, respectively.

11. The method according to claim 9, said arbitrary target face being identical with said target face.

12. A face recognition apparatus, comprising

a display module for displaying an image;

a detection module, electrically coupled with said display module, for detecting a target face from said image and whereupon to generate a detection result designating said target face, said target face having a set of extracted target facial features;

a candidate list-generating module, electrically coupled with said detection module and a storage medium, for generating a candidate list of one or more candidate faces for matching said target face, each candidate face having a set of candidate facial features thereof, said candidate facial features data being stored in a database stored in said storage medium;

a comparison module, electrically coupled with said candidate list-generating module and said storage medium, for comparing said target face with each said candidate face based on said target facial features and the candidate facial features of each said candidate face, whereupon to generate a first comparison result indicating a first matching candidate face out of said candidate list; and

a database-updating module, electrically coupled with said detection module, said comparison module and said storage medium, receiving said target facial features data from said detection module and said first matching candidate data from said comparison module, for updating the candidate facial features of said first matching candidate face stored in said database with said target facial features.

13. The apparatus according to claim 12, further comprising:

a user input module for receiving a user input, said user input designating a second matching candidate face out of said candidate list;

a supplementary comparison module, electrically coupled with said user input module, said comparison module and said database-updating module, for comparing said first and second matching candidate face based on the candidate facial features of said first and second matching candidate face, respectively, whereupon to generate a second comparison result, wherein

said database-updating module receives said second comparison result from said supplementary comparison module, when said the second comparison result indicates that said first and second matching candidate faces are identical, said database-updating module updates the candidate facial features of said first matching candidate face stored in said database with said target facial features, otherwise said database-updating module updates the candidate facial features of said second matching candidate face stored in said database with said target facial features.

14. The apparatus according to claim 12, said detection module being electrically coupled with said user input module, wherein:

said detection module detects n potential target faces from said image, n being a natural number;

said user input module receives additional user input, said additional user input designating one target face out of said n potential target faces, wherein said detection module, responsive to receiving said additional user input, generates said detection result indicating said target face as designated by said additional user input.

15. The apparatus according to claim 12, said user input module being electrically coupled with said database-updating module, wherein

said user input module receives yet additional user input, said additional user input designating that at least one of said candidate faces on said candidate list is to be added to said database stored in said medium;

and

said database-updating module, responsive to receiving the additional user input, updates said database by adding said at least one of said candidate faces on said candidate list designated by said additional user input.

16. The apparatus according to claim 12, said database-updating module being electrically coupled with said user input module, wherein

said user input module receives further user input, said further user input designating that said first matching candidate face is to be updated with said target face;

and

said database-updating module, responsive to receiving the further user input, updates the candidate facial features of said first matching candidate face with said target facial features.

17. The apparatus according to claim 12, further comprising a marking module electrically coupled with said display module, said detection module, and comparison module, wherein

said display module displays a video;

said detection module detects an arbitrary target face in a frame of said video at a certain frame numbered n and said arbitrary target face in a subsequent frame of said video at frame numbered n+1, said arbitrary target face having a set of arbitrary target facial features at frame numbered n and frame numbered n+1, respectively;

said comparison module, responsive to said detection module, compares said arbitrary target facial features at frame numbered n and frame numbered n+1, whereupon to generate a third comparison result; and

said marking module receives said third comparison result from said comparison module and in response, when said third comparison result indicates that said arbitrary target facial features at frame numbered n+1 is different from said arbitrary target features at frame numbered n, said marking module marks said frame at frame numbered n+1 as said image to be displayed by said display module, otherwise marking said frame at frame numbered n+1 as said frame at frame numbered n to be detected by said detection module.

18. The apparatus according to claim 17, said arbitrary target face being said target face and said arbitrary target facial features being said target facial features.

19. The apparatus according to claim 12, further comprising a feature-scoring module and a marking module electrically coupled with each other, said feature scoring module being further electrically coupled with said detection module and said comparison module, said marking module being further electrically coupled with said comparison module, wherein

said display module displays a video;

said detection module detects an arbitrary target face in a frame of said video at a certain frame numbered n and said arbitrary target face in a subsequent frame of said video at frame numbered n+1, and extracts a set of arbitrary target facial features at frame numbered n for said arbitrary target face at frame numbered n and a set of arbitrary target facial features at frame numbered n+1 for said arbitrary target face at frame numbered n+1;

said feature-scoring module, responsive to detecting the arbitrary target face, estimates a score for said arbitrary target face at frame numbered n, whereupon to generate a first feature score, and estimates a score for said arbitrary target face at frame numbered n+1, whereupon to generate a second feature score;

said comparison module receives said first and second feature score from said feature-scoring module and in response, compares said first feature score with said second feature score, whereupon to generate a fourth comparison result,

said marking module, receiving said fourth comparison result, when said fourth comparison result indicates that said second feature score is higher than said first feature score, marks said arbitrary target facial features at frame numbered n+1 as said arbitrary target facial features at frame numbered n for said feature-scoring module and said target facial features as well, otherwise marks said arbitrary target facial features at frame numbered n as said arbitrary target facial features at frame numbered n for said feature-scoring module and said target facial features as well.

20. The apparatus according to claim 19, the score estimation is based on indices such a face orientation, clarity, shape, lighting condition, shadowing, and other measures.

21. The apparatus according to claim 19, said arbitrary target face being said target face.

22. An electronic image capturing device, comprising:

a display for presenting at least one window, wherein

a first window displays an image;

a second window displays one or more first face thumbnails, said first thumbnail being faces detected from said image, one of which a target face to be recognized;

a third window displays one or more second face thumbnails, said second thumbnail being candidate faces in terms of matching said target face, one of said candidate faces being a user-selected candidate face;

a fourth window displays one or more third face thumbnail, said third face thumbnail being faces or facial features associated with said user-selected candidate face; and

a user input module, for receiving at least one user input, wherein

a first user input designates said target face out of said first thumbnails; and

a second user input designates said user-selected candidate face out of said second thumbnails.

23. The electronic image capturing device according to claim 22, further comprising:

an icon in said third window; and

a third user input received by said user input module, said third user input designating that said target face is to be added to a database where said facial features data are stored by selecting said icon.

24. The electronic image capturing device according to claim 22, said display being liquid crystal display (LCD), plasma display or light emitting diode (LED) display.

25. The electronic image capturing device according claim 22, said user input module being press button, key pad, touch panel, and voice control module.