Face recognition through face building from two or more partial face images from the same face

• US 8,712,160 B2
• Filed: 01/15/2013
• Issued: 04/29/2014
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Bigioi, Petronel, Steinberg, Eran, Corcoran, Peter
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): ROWLAND, JHERE M
• Application Number:
  US13/742,262
• Publication Number:
  US 20130194455A1
• Time in Patent Office: 469 Days
• Field of Search: None
• US Class Current: 382/190
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

A device is trained for face recognition. A first acquired digital image of a scene includes a face. Face image data is extracted and stored in a face image library along with an unique identifier. A second acquired digital image may or may not include the face of the same person as the face in the first acquired digital image. Face recognition is applied to extracted face data of the second digital image, and the face of the first digital image is displayed as a match to the face of the second digital image when the first and second images are determined to match or the second digital image or a face therein is identified with biometric data stored along with the first digital image when the faces in the first and second images are determined to match.

Claims

1. Within a processor-based, digital image acquisition device, a method of acquiring and displaying a face recognition result in real time based on identifying a candidate partial face region within an acquired digital image based on analysis of a series preview images, comprising:

acquiring and analyzing with the digital image acquisition device the digital image of a scene that includes a partial face image from a sequential stream of multiple preview images of approximately the same scene acquired within one second prior to capture of the digital image;

within the digital image acquisition device, based on the analyzing, automatically applying face recognition by comparing the partial face image to one or more images stored within a face image library;

adding a smile, an open eye or other partial face portion, or combinations thereof, to the partial face image to form a composite face or partial image from two or more images of a same recognized face; and

displaying in real-time the digital image along with said composite face or partial face image, or another unique identifier and/or biometric data, or combinations thereof, extracted from a different digital image stored in the face image library that uniquely identifies the face or partial in the digital image.

2. The method of claim 1, wherein the face image library comprises an address book or a contact list, or both, and wherein the device comprises a mobile camera phone or other handheld camera device.

3. The method of claim 1, wherein face recognition is applied to the more full-size partial face image comprising standard resolution, while the sequential stream of multiple images comprises a series of relatively low resolution preview images.

4. The method of claim 3, further comprising maintaining location data and a cumulative confidence level that the candidate partial face region belongs to a specific person, and based on information from the series of preview images including the partial face region, determining that the specific person is present within said digital image, and corresponds to a specific person named in an address book or contact list, or both, stored on said device.

5. The method of claim 1, further comprising receiving identifying indicia relating to the partial face image manually by a user of the digital image acquisition device.

6. The method of claim 5, further comprising displaying the partial face image and prompting the user to associate the candidate partial face region with the identifying indicia;

and automatically storing the more full-size partial face image along with the identifying indicia within the face image library.

7. The method of claim 1, further comprising displaying a list of probable members of a contact list and receiving a selection from the list of probable members by a user.

8. The method of claim 1, wherein the sequential stream comprises a video stream.

9. The method of claim 1, further comprising building a whole face image from two or more partial face images.

10. The method of claim 1, further comprising retrieving said smile, open eye or other partial face region, or combinations thereof, from one or more stored facial images of a same identified person.

11. One or more non-transitory processor-readable media encoded with processor-readable code for programming a processor that is built into a portable device configured for acquiring a digital image of a scene that includes a partial face image from a sequential stream of multiple preview images of approximately the same scene to perform a method of acquiring and displaying a face recognition result in real time based on identifying a candidate partial face region within the digital image, wherein the method comprises:

within the portable device that acquired the digital image including the partial face image from a sequential stream of multiple images of approximately the same scene acquired within one second prior to capture of the digital image, based on the analyzing, automatically applying face recognition by comparing the partial face image to one or more images stored within a face image library;

adding a smile, an open eye or other partial face portion, or combinations thereof, to the partial face image to form a composite face or partial image from two or more images of a same recognized face; and

displaying in real-time the digital image along with said composite face or partial face image, or another unique identifier and/or biometric data, or combinations thereof, extracted from a different digital image stored in the face image library that uniquely identifies the face or partial face in the digital image.

12. The one or more processor-readable media of claim 11, wherein the face image library comprises an address book or a contact list, or both, and wherein the portable device comprises a mobile camera phone or other handheld camera device.

13. The one or more processor-readable media of claim 11, wherein face recognition is applied to a main image, while the sequential stream comprises a series of relatively low resolution preview images.

14. The one or more processor-readable media of claim 13, wherein the method further comprises maintaining location data and a cumulative confidence level that a candidate partial face image belongs to a specific person, and based on information from the series of preview images, determining that the specific person is present within said digital image, and corresponds to a specific person named in an address book or contact list, or both, stored on said device.

15. The one or more processor-readable media of claim 11, wherein the method further comprises receiving identifying indicia relating to the partial face image manually by a user of the digital image acquisition device.

16. The one or more processor-readable media of claim 15, wherein the method further comprises displaying the partial face image and prompting the user to associate the partial face image with the identifying indicia;

and automatically storing the more full-size partial face image along with the identifying indicia within the face image library.

17. The one or more processor-readable media of claim 11, wherein the method further comprises displaying a list of probable members of a contact list and receiving a selection from the list of probable members by a user.

18. The one or more processor-readable media of claim 11, wherein the sequential stream comprises a video stream.

19. The one or more processor-readable media of claim 11, wherein the method further comprises building a whole face image from two or more partial face images.

20. The one or more processor-readable media of claim 11, wherein the method further comprises retrieving said smile, open eye or other partial face region, or combinations thereof, from one or more stored facial images of a same identified person.

21. A digital image acquisition device, comprising:

a lens,

an image sensor,

a processor and

a memory having processor-readable code embodied therein for programming the processor to perform a method of acquiring and displaying a face recognition result in real time based on identifying a candidate partial face region, wherein the method comprises;

acquiring and analyzing with the digital image acquisition device a digital image of a scene that includes a partial face image from a sequential stream of multiple preview images of approximately the same scene acquired within one second prior to capture of the digital image;

within the digital image acquisition device, based on the analyzing, automatically applying face recognition by comparing the partial face image to one or more images stored within a face image library;

adding a smile, an open eye or other partial face portion, or combinations thereof, to the partial face image to form a composite face or partial image from two or more images of a same recognized face; and

displaying in real-time the digital image along with said composite face or partial face image, or another unique identifier and/or biometric data, or combinations thereof, extracted from a different digital image stored in the face image library that uniquely identifies the face or partial in the digital image.

22. The device of claim 21, wherein the face image library comprises an address book or a contact list, or both, and wherein the device comprises a mobile camera phone or other handheld camera device.

23. The device of claim 21, wherein face recognition is applied to a main image, while the sequential stream comprises a series of relatively low resolution preview images.

24. The device of claim 23, wherein the method further comprises maintaining location data and a cumulative confidence level that a candidate partial face image belong to a specific person, and based on information from the series of preview images, determining that the specific person is present within said digital image, and corresponds to a specific person named in an address book or contact list, or both, stored on said device.

25. The device of claim 21, wherein the method further comprises receiving identifying indicia relating to the partial face image manually by a user of the digital image acquisition device.

26. The device of claim 25, wherein the method further comprises displaying the partial face image and prompting the user to associate the partial face image with the identifying indicia;

and automatically storing the more full-size partial face image along with the identifying indicia within the face image library.

27. The device of claim 21, wherein the method further comprises displaying a list of probable members of a contact list and receiving a selection from the list of probable members by a user.

28. The device of claim 21, wherein the sequential stream comprises a video stream.

29. The device of claim 21, wherein the method further comprises building a whole face image from two or more partial face images.

30. The device of claim 21, wherein the method further comprises retrieving said smile, open eye or other partial face region, or combinations thereof, from one or more stored facial images of a same identified person.