Face recognition performance using additional image features

• US 8,379,917 B2
• Filed: 10/02/2009
• Issued: 02/19/2013
• Est. Priority Date: 10/02/2009
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Bigioi, Petronel, Costache, Gabriel, Drimbarean, Alexandru, Corcoran, Peter
• Primary Examiner(s): Johns, Andrew W
• Assistant Examiner(s): Dunphy, David F
• Application Number:
  US12/572,930
• Publication Number:
  US 20110081052A1
• Time in Patent Office: 1,236 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

A technique is provided for recognizing faces in an image stream using a digital image acquisition device. A first acquired image is received from an image stream. A first face region is detected within the first acquired image having a given size and a respective location within the first acquired image. First faceprint data uniquely identifying the first face region are extracted along with first peripheral region data around the first face region. The first faceprint and peripheral region data are stored, and the first peripheral region data are associated with the first face region. The first face region is tracked until a face lock is lost. A second face region is detected within a second acquired image from the image stream. Second peripheral region data around the second face region are extracted. The second face region is identified upon matching the first and second peripheral region data.

Claims

1. A method of recognizing faces in an image stream using

a digital image acquisition device, comprising:

receiving a first acquired image from an image stream;

detecting a first face region within the first acquired image having a given size and a respective location within the first acquired image;

extracting first faceprint data uniquely identifying the first face region along with first peripheral region data around the first face region;

storing the first faceprint and peripheral region data, including associating the first peripheral region data with the first face region;

tracking the first face region until face lock is lost;

detecting a second face region within a second acquired image from the image stream;

extracting second peripheral region data around the second face region;

comparing the first and second peripheral region data;

identifying the second face region as a same identity as the first face region when the first peripheral region data matches the second peripheral region data; and

confirming the second face region as said same identity as the first face region when the first and second face regions match.

2. The method of claim 1, wherein the storing of peripheral region data occurs only within volatile memory.

3. The method of claim 1, further comprising providing a database comprising an identifier and associated parameters for each of a number of one or more faces to be recognized;

and selectively applying face recognition using said database to said first face region to provide an identifier for the first face region.

4. The method of claim 1, further comprising extracting second faceprint data uniquely identifying the second face region, and matching the first and second faceprint data, thereby confirming the identifying of the second face region.

5. The method of claim 1, wherein the detecting and identifying of the second face region occur within two second or less.

6. The method of claim 1, wherein the detecting and identifying of the second face region occur within one second or less.

7. The method of claim 1, further comprising displaying a same identifier with the first and second face regions.

8. The method of claim 7, wherein the identifier comprises a nickname of a person associated with the first and second face regions.

9. The method of claim 1, further comprising receiving a third acquired image from the image stream that has a higher resolution than the first and second acquired images, and identifying a third face region detected therein based upon matching the first faceprint with a third faceprint extracted from the third acquired image.

10. One or more non-transitory processor-readable storage media having code embedded therein for programming a processor to perform a method of recognizing faces in an image stream using a digital image acquisition device, wherein the method comprises:

receiving a first acquired image from an image stream;

detecting a first face region within the first acquired image having a given size and a respective location within the first acquired image;

extracting first faceprint data uniquely identifying the first face region along with first peripheral region data around the first face region;

storing the first faceprint and peripheral region data, including associating the first peripheral region data with the first face region;

tracking the first face region until face lock is lost;

detecting a second face region within a second acquired image from the image stream;

extracting second peripheral region data around the second face region; and

comparing the first and second peripheral region data;

identifying the second face region as a same identity as the first face region when the first peripheral region data matches the second peripheral region data; and

confirming the second face region as said same identity as the first face region when the first and second face regions match.

11. The one or more processor-readable media of claim 10, wherein the storing of peripheral region data occurs only within volatile memory.

12. The one or more processor-readable media of claim 10, wherein the method further comprises providing a database comprising an identifier and associated parameters for each of a number of one or more faces to be recognized;

and selectively applying face recognition using said database to said first face region to provide an identifier for the first face region.

13. The one or more processor-readable media of claim 10, wherein the method further comprises extracting second faceprint data uniquely identifying the second face region, and matching the first and second faceprint data, thereby confirming the identifying of the second face region.

14. The one or more processor-readable media of claim 10, wherein the detecting and identifying of the second face region occur within two second or less.

15. The one or more processor-readable media of claim 10, wherein the detecting and identifying of the second face region occur within one second or less.

16. The one or more processor-readable media of claim 10, wherein the method further comprises displaying a same identifier with the first and second face regions.

17. The one or more processor-readable media of claim 16, wherein the identifier comprises a nickname of a person associated with the first and second face regions.

18. The one or more processor-readable media of claim 10, wherein the method further comprises receiving a third acquired image from the image stream that has a higher resolution than the first and second acquired images, and identifying a third face region detected therein based upon matching the first faceprint with a third faceprint extracted from the third acquired image.

19. A portable, digital image acquisition device, comprising a lens and image sensor for acquiring digital images, a processor, and a processor-readable medium having code embedded therein for programming the processor to perform a method of recognizing faces in an image stream using a digital image acquisition device, wherein the method comprises:

receiving a first acquired image from an image stream;

detecting a first face region within the first acquired image having a given size and a respective location within the first acquired image;

extracting first faceprint data uniquely identifying the first face region along with first peripheral region data around the first face region;

storing the first faceprint and peripheral region data, including associating the first peripheral region data with the first face region;

tracking the first face region until face lock is lost;

detecting a second face region within a second acquired image from the image stream;

extracting second peripheral region data around the second face region;

comparing the first and second peripheral region data;

identifying the second face region as a same identity as the first face region when the first peripheral region data matches the second peripheral region data; and

confirming the second face region as said same identity as the first face region when the first and second face regions match.

20. The device of claim 19, wherein the storing of peripheral region data occurs only within volatile memory.

21. The device of claim 19, further comprising providing a database comprising an identifier and associated parameters for each of a number of one or more faces to be recognized;

and selectively applying face recognition using said database to said first face region to provide an identifier for the first face region.

22. The device of claim 19, further comprising extracting second faceprint data uniquely identifying the second face region, and matching the first and second faceprint data, thereby confirming the identifying of the second face region.

23. The device of claim 19, wherein the detecting and identifying of the second face region occur within two second or less.

24. The device of claim 19, wherein the detecting and identifying of the second face region occur within one second or less.

25. The device of claim 19, further comprising displaying a same identifier with the first and second face regions.

26. The device of claim 25, wherein the identifier comprises a nickname of a person associated with the first and second face regions.

27. The device of claim 19, wherein the method further comprises receiving a third acquired image from the image stream that has a higher resolution than the first and second acquired images, and identifying a third face region detected therein based upon matching the first faceprint with a third faceprint extracted from the third acquired image.

28. A method of recognizing faces in an image stream using a digital image acquisition device, comprising:

receiving a first acquired image from an image stream;

detecting a first face region within the first acquired image having a given size and a respective location within the first acquired image;

extracting first faceprint data uniquely identifying the first face region along with first peripheral region data, including texture information, around the first face region; and

retrieving texture information from the first and second peripheral region data; and

identifying the second face region upon matching the texture information from the first and second peripheral region data.

storing the first faceprint and peripheral region data, including associating the first peripheral region data with the first face region;

tracking a first face region combination, including the first face region and peripheral region data, until face lock is lost; and

identifying the first face region as a first identity based at least in part on the texture information,

extracting second peripheral region data for a second detected face region after said face lock is lost;

comparing the first and second peripheral region data;

identifying the second face region as a same identity as the first face region when the first peripheral region data matches the second peripheral region data; and

confirming the second face region as said same identity as the first face region when the first and second face regions match.

29. The method of claim 28, further comprising providing a database comprising an identifier and associated parameters for each of a number of one or more faces to be recognized;

and selectively applying face recognition using said database to said first face region combination to provide an identifier for the first face region combination.

30. The method of claim 29, further comprising retrieving and matching texture information from the first peripheral region data for comparing with texture information from a stored peripheral region.

31. The method of claim 28, wherein the method further comprises receiving a third acquired image from the image stream that has a higher resolution than the first and second acquired images, and identifying a third face region detected therein based upon matching the first faceprint with a third faceprint extracted from the third acquired image.

32. One or more non-transitory processor-readable storage media having code embedded therein for programming a processor to perform a method of recognizing faces in an image stream using a digital image acquisition device, wherein the method comprises:

receiving a first acquired image from an image stream;

detecting a first face region within the first acquired image having a given size and a respective location within the first acquired image;

extracting first faceprint data uniquely identifying the first face region along with first peripheral region data, including texture information, around the first face region;

storing the first faceprint and peripheral region data, including associating the first peripheral region data with the first face region;

tracking a first face region combination, including the first face region and peripheral region data, until face lock is lost; and

identifying the first face region as a first identity based at least in part on the texture information,

extracting second peripheral region data for a second detected face region after said face lock is lost;

comparing the first and second peripheral region data;

identifying the second face region as a same identity as the first face region when the first peripheral region data matches the second peripheral region data; and

confirming the second face region as said same identity as the first face region when the first and second face regions match.

33. The one or more processor-readable media of claim 32, wherein the method further comprises providing a database comprising an identifier and associated parameters for each of a number of one or more faces to be recognized;

and selectively applying face recognition using said database to said first face region combination to provide an identifier for the first face region combination.

34. The one or more processor-readable media of claim 33, wherein the method further comprises retrieving and matching texture information from the first peripheral region data for comparing with texture information from a stored peripheral region.

35. The one or more processor-readable media of claim 32, wherein the method further comprises receiving a third acquired image from the image stream that has a higher resolution than the first and second acquired images, and identifying a third face region detected therein based upon matching the first faceprint with a third faceprint extracted from the third acquired images.

36. A portable, digital image acquisition device, comprising a lens and image sensor for acquiring digital images, a processor, and a processor-readable medium having code embedded therein for programming the processor to perform a method of recognizing faces in an image stream using a digital image acquisition device, wherein the method comprises:

receiving a first acquired image from an image stream;

detecting a first face region within the first acquired image having a given size and a respective location within the first acquired image;

extracting first faceprint data uniquely identifying the first face region along with first peripheral region, including texture information, around the first face region;

storing the first faceprint and peripheral region data, including associating the first peripheral region data with the first face region;

tracking a first face region combination, including the first face region and peripheral region data, until face lock is lost; and

identifying the first face region as a first identity based at least in part on the texture information,

extracting second peripheral region data for a second detected face region after said face lock is lost;

comparing the first and second peripheral region data;

identifying the second face region as a same identity as the first face region when the first peripheral region data matches the second peripheral region data; and

confirming the second face region as said same identity as the first face region when the first and second face regions match.

37. The device of claim 36, wherein the method further comprises providing a database comprising an identifier and associated parameters for each of a number of one or more faces to be recognized;

and selectively applying face recognition using said database to said first face region combination to provide an identifier for the first face region combination.

38. The device of claim 37, wherein the method further comprises retrieving and matching texture information from the first peripheral region data for comparing with texture information from a stored peripheral region.

39. The device of claim 36, wherein the method further comprises receiving a third acquired image from the image stream that has a higher resolution than the first and second acquired images, and identifying a third face region detected therein based upon matching the first faceprint with a third faceprint extracted from the third acquired image.