Face recognition training method and apparatus

• US 8,363,952 B2
• Filed: 10/28/2010
• Issued: 01/29/2013
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Bigioi, Petronel, Steinberg, Eran, Corcoran, Peter
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): ROWLAND, JHERE M
• Application Number:
  US12/913,772
• Publication Number:
  US 20110058060A1
• Time in Patent Office: 824 Days
• Field of Search: None
• US Class Current: 382/190
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

A device is trained for face recognition. A first acquired digital image of a scene includes a face. Face image data is extracted and stored in a face image library along with an unique identifier. A second acquired digital image may or may not include the face of the same person as the face in the first acquired digital image. Face recognition is applied to extracted face data of the second digital image, and the face of the first digital image is displayed as a match to the face of the second digital image when the first and second images are determined to match or the second digital image or a face therein is identified with biometric data stored along with the first digital image when the faces in the first and second images are determined to match.

Claims

1. Within a processor-based, digital image acquisition device, a method of acquiring and displaying a face recognition result in real time based on identifying a candidate partial face region within an acquired digital image based on analysis of a series preview images, comprising:

acquiring and analyzing with a digital image acquisition device a digital image of a scene that includes a partial face image from a sequential stream of multiple preview images of approximately the same scene acquired within one second prior to capture of the digital image;

within the digital image acquisition device, based on the analyzing, automatically cropping the partial face image or removing one or more non-facial items from the digital image, or both, and thereby generating a more full-size partial face image;

applying face recognition by comparing the more full-size partial face image to one or more images stored within a face image library; and

displaying in real-time the digital image along with a face image, or another unique identifier and/or biometric data, or combinations thereof, extracted from a different digital image stored in the face image library that uniquely identifies the face in the digital image.

2. The method of claim 1, wherein the face image library comprises an address book or a contact list, or both, and wherein the device comprises a mobile camera phone or other handheld camera device.

3. The method of claim 1, wherein face recognition is applied to the more full-size partial face image comprising standard resolution, while the sequential stream of multiple images comprises a series of relatively low resolution preview images.

4. The method of claim 3, further comprising maintaining location data and a cumulative confidence level that a candidate partial face region belongs to a specific person, and based on information from the series of preview images including the partial face region, determining that the specific person is present within said digital image, and corresponds to a specific person named in an address book or contact list, or both, stored on said device.

5. The method of claim 1, further comprising receiving identifying indicia relating to the partial face image manually by a user of the digital image acquisition device.

6. The method of claim 5, further comprising displaying the partial face image and prompting the user to associate the partial face region with the identifying indicia;

and automatically storing the more full-size partial face image along with the identifying indicia within the face image library.

7. The method of claim 1, further comprising displaying a list of probable members of a contact list and receiving a selection from the list by the user.

8. The method of claim 1, wherein the stream comprises a video stream.

9. The method of claim 1, further comprising building a whole face image from two or more partial face images.

10. The method of claim 1, further comprising adding a smile, an open eye or other partial face portion, or combinations thereof, to the partial face image.

11. The method of claim 10, further comprising retrieving said smile, open eye or other partial face region, or combinations thereof, from one or more stored facial images of the same identified person.

12. One or more non-transitory processor-readable media encoded with processor-readable code for programming a processor that is built into a portable device configured for acquiring a digital image of a scene that includes a partial face image from a sequential stream of multiple preview images of approximately the same scene to perform a method of face recognition within a digital image, wherein the method comprises:

automatically analyzing one or more partial face images from said sequential stream of multiple preview images acquired within one second prior to capture of the digital image;

based on the analyzing, automatically cropping the partial face image or removing one or more non-facial items from the digital image, or both, and thereby generating a more full-size partial face image; and

applying face recognition by comparing the more full-size partial face image to one or more images stored within a face image library.

13. The one or more processor-readable media of claim 12, wherein the face image library comprises an address book or a contact list, or both, and wherein the portable device comprises a mobile camera phone or other handheld camera device.

14. The one or more processor-readable media of claim 12, wherein face recognition is applied to a main image, while the stream comprises a series of relatively low resolution preview images.

15. The one or more processor-readable media of claim 14, wherein the method further comprises maintaining location data and a cumulative confidence level that a candidate partial face image belongs to a specific person, and based on information from the series of preview images, determining that the specific person is present within said digital image, and corresponds to a specific person named in an address book or contact list, or both, stored on said device.

16. The one or more processor-readable media of claim 12, wherein the method further comprises receiving identifying indicia relating to the partial face image manually by a user of the digital image acquisition device.

17. The one or more processor-readable media of claim 16, wherein the method further comprises displaying the partial face image and prompting the user to associate the partial face image with the identifying indicia;

and automatically storing the more full-size partial face image along with the identifying indicia within the face image library.

18. The one or more processor-readable media of claim 12, wherein the method further comprises displaying a list of probable members of a contact list and receiving a selection from the list by the user.

19. The one or more processor-readable media of claim 12, wherein the stream comprises a video stream.

20. The one or more processor-readable media of claim 12, wherein the method further comprises building a whole face image from two or more partial face images.

21. The one or more processor-readable media of claim 12, wherein the method further comprises adding a smile, an open eye or other partial face portion, or combinations thereof, to the partial face image.

22. The one or more processor-readable media of claim 21, wherein the method further comprises retrieving said smile, open eye or other partial face region, or combinations thereof, from one or more stored facial images of the same identified person.

23. A digital image acquisition device, comprising:

a lens,

an image sensor,

a processor and

a memory having processor-readable code embodied therein for programming the processor to perform a face recognition method, wherein the method comprises;

acquiring and analyzing with a digital image acquisition device a digital image of a scene that includes a partial face image from a sequential stream of multiple preview images of approximately the same scene acquired within one second prior to capture of the digital image;

based on the analyzing, automatically cropping the partial face image or removing one or more non-facial items from the digital image, or both, and thereby generating a more full-size partial face image; and

applying face recognition by comparing the more full-size partial face image to one or more images stored within a face image library.

24. The device of claim 23, wherein the face image library comprises an address book or a contact list, or both, and wherein the device comprises a mobile camera phone or other handheld camera device.

25. The device of claim 23, wherein face recognition is applied to a main image, while the stream comprises a series of relatively low resolution preview images.

26. The device of claim 25, wherein the method further comprises maintaining location data and a cumulative confidence level that a candidate partial face image belong to a specific person, and based on information from the series of preview images, determining that the specific person is present within said digital image, and corresponds to a specific person named in an address book or contact list, or both, stored on said device.

27. The device of claim 23, wherein the method further comprises receiving identifying indicia relating to the partial face image manually by a user of the digital image acquisition device.

28. The device of claim 27, wherein the method further comprises displaying the partial face image and prompting the user to associate the partial face image with the identifying indicia;

and automatically storing the more full-size partial face image along with the identifying indicia within the face image library.

29. The device of claim 23, wherein the method further comprises displaying a list of probable members of a contact list and receiving a selection from the list by the user.

30. The device of claim 23, wherein the stream comprises a video stream.

31. The device of claim 23, wherein the method further comprises building a whole face image from two or more partial face images.

32. The device of claim 23, wherein the method further comprises adding a smile, an open eye or other partial face portion, or combinations thereof, to the partial face image.

33. The device of claim 32, wherein the method further comprises retrieving said smile, open eye or other partial face region, or combinations thereof, from one or more stored facial images of the same identified person.