Real-time face tracking in a digital image acquisition device

• US 8,422,739 B2
• Filed: 09/15/2011
• Issued: 04/16/2013
• Est. Priority Date: 08/11/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Ianculescu, Mihai, Bigioi, Petronel, Gangea, Mihnea, Petrescu, Stefan, Corcoran, Peter, Steinberg, Eran
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US13/234,149
• Publication Number:
  US 20120062761A1
• Time in Patent Office: 579 Days
• Field of Search: 382/103, 382/115, 382/117, 382/118, 382/164, 382/173, 382/181, 382/190, 382/218, 382/224, 382/254, 382/284, 382/298, 382/299, 382/305, 348/222.1, 348/E05.031, 358/538
• US Class Current: 382/103
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06V 30/19147 Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: H04N 1/2112 using still video cameras
• CPC Class Codes: H04N 1/32112 in a separate computer file, document page or paper sheet, e.g. a fax cover sheet H04N1/32122 takes precedence
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3205 of identification information, e.g. name or ID code H04N2201/3209 takes precedence
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body

Abstract

A database includes an identifier and associated parameters for each of a number of faces to be recognized. A new acquired image from an image stream is received potentially including one or more face regions. Face detection is applied to at least a portion of the acquired image to provide a set of candidate face regions each having a given size and a respective location. Using the database, face recognition is selectively applied to at least one of the candidate face regions to provide an identifier for a face recognized in a candidate face region. A portion of the image is stored including the recognized face in association with at least one image of the image stream.

Claims

1. A method of recognizing faces in an image stream with a digital image acquisition device, comprising:

using a processor to perform the following;

providing a database comprising an identifier and associated parameters for each of a number of one or more faces to be recognized;

receiving a new acquired image from an image stream;

obtaining a location of one or more tracked face regions within said image;

configuring data of said one or more tracked face regions for accurate face recognition;

selectively applying face recognition using said database to at least one of said one or more tracked face regions to determine a first identifier for said at least one of said one or more tracked face regions; and

storing, transmitting or displaying a portion of said image including the recognized face in association with the first identifier, or combinations thereof

2. The method of claim 1, further comprising identifying one or more of said tracked face regions based on at least the first identifier and a second identifier determined for the same face region.

3. The method of claims 1 wherein said configuring data of said one or more tracked face regions for accurate face recognition includes configuring the data so that at least one of the one or more tracked face regions is substantially frontal in aspect, or so that the vertical or horizontal rotation, or both, of the at least one of the one or more tracked face regions lie within a predetermined range of angles, or so that the mean and variance of luminance for at least one of the one or more tracked face regions lie within predetermined ranges, or combinations thereof.

4. The method of claim 3 wherein the parameters associated with a particular identifier comprise functions of the vertical or horizontal rotation, or both, or comprise functions of the mean or variance, or both, of luminance of a tracked face region, or combinations thereof.

5. The method of claim 1, further comprising:

calculating a corresponding integral image for at least a portion of said image; and

applying face detection to at least a portion of said integral image to provide a set of one or more candidate face regions each having a given size and a respective location.

6. The method of claim 1, further comprising sub-sampling said acquired image at a specified resolution to provide a sub-sampled image.

7. An image acquisition and processing apparatus including a lens and an image sensor for acquiring digital images, and one or more processors and one or more digital storage media having digitally-encoded instructions embedded therein for programming the one or more processors to perform a method of recognizing faces in an image stream, the method comprising:

providing a database comprising an identifier and associated parameters for each of a number of one or more faces to be recognized;

receiving a new acquired image from an image stream;

obtaining a location of one or more tracked face regions within said image;

configuring data of said one or more tracked face regions for accurate face recognition;

selectively applying face recognition using said database to at least one of said one or more tracked face regions to determine a first identifier for said at least one of said one or more tracked face regions; and

storing, transmitting or displaying a portion of said image including the recognized face in association with the first identifier, or combinations thereof.

8. The apparatus of claim 7, wherein the method further comprises identifying one or more of said tracked face regions based on at least the first identifier and a second identifier determined for the same face region.

9. The apparatus of claims 7, wherein said configuring data of said one or more tracked face regions for accurate face recognition includes configuring the data so that at least one of the one or more tracked face regions is substantially frontal in aspect, or so that the vertical or horizontal rotation, or both, of the at least one of the one or more tracked face regions lie within a predetermined range of angles, or so that the mean and variance of luminance for at least one of the one or more tracked face regions lie within predetermined ranges, or combinations thereof.

10. The apparatus of claim 9, wherein the parameters associated with a particular identifier comprise functions of the vertical or horizontal rotation, or both, or comprise functions of the mean or variance, or both, of luminance of a tracked face region, or combinations thereof.

11. The apparatus of claim 7, wherein the method further comprises:

calculating a corresponding integral image for at least a portion of said image; and

applying face detection to at least a portion of said integral image to provide a set of one or more candidate face regions each having a given size and a respective location.

12. The apparatus of claim 7, wherein the method further comprises sub-sampling said acquired image at a specified resolution to provide a sub-sampled image.

13. One or more non-transitory, computer readable media having digitally-encoded instructions embedded therein for programming one or more processors to perform a method of recognizing faces in an image stream, the method comprising:

providing a database comprising an identifier and associated parameters for each of a number of one or more faces to be recognized;

receiving a new acquired image from an image stream;

obtaining a location of one or more tracked face regions within said image;

configuring data of said one or more tracked face regions for accurate face recognition;

selectively applying face recognition using said database to at least one of said one or more tracked face regions to determine a first identifier for said at least one of said one or more tracked face regions; and

storing, transmitting or displaying a portion of said image including the recognized face in association with the first identifier, or combinations thereof.

14. The one or more non-transitory, computer readable media of claim 13, wherein the method further comprises identifying one or more of said tracked face regions based on at least the first identifier and a second identifier determined for the same face region.

15. The one or more non-transitory, computer readable media of claims 13, wherein said configuring data of said one or more tracked face regions for accurate face recognition includes configuring the data so that at least one of the one or more tracked face regions is substantially frontal in aspect, or so that the vertical or horizontal rotation, or both, of the at least one of the one or more tracked face regions lie within a predetermined range of angles, or so that the mean and variance of luminance for at least one of the one or more tracked face regions lie within predetermined ranges, or combinations thereof.

16. The one or more non-transitory, computer readable media of claim 15, wherein the parameters associated with a particular identifier comprise functions of the vertical or horizontal rotation, or both, or comprise functions of the mean or variance, or both, of luminance of a tracked face region, or combinations thereof.

17. The one or more non-transitory, computer readable media of claim 13, wherein the method further comprises:

calculating a corresponding integral image for at least a portion of said image; and

applying face detection to at least a portion of said integral image to provide a set of one or more candidate face regions each having a given size and a respective location.

18. The one or more non-transitory, computer readable media of claim 13, wherein the method further comprises sub-sampling said acquired image at a specified resolution to provide a sub-sampled image.