Real-time face tracking in a digital image acquisition device

• US 7,864,990 B2
• Filed: 12/11/2008
• Issued: 01/04/2011
• Est. Priority Date: 08/11/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Petrescu, Stefan, Nanu, Florin, Drimbarean, Alexandru, Bigioi, Petronel, Corcoran, Peter, Pososin, Alexei
• Primary Examiner(s): Alavi; Amir
• Application Number:
  US12/333,221
• Publication Number:
  US 20090208056A1
• Time in Patent Office: 754 Days
• Field of Search: 382/103, 382115-118, 382/164, 382/173, 382/181, 382/190, 382/203, 382/243, 382/254, 382/284, 382/299, 382/305, 348/169, 348/349
• US Class Current: 382/118
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06V 30/19147 Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 1/2112 using still video cameras
• CPC Class Codes: H04N 1/32112 in a separate computer file, document page or paper sheet, e.g. a fax cover sheet H04N1/32122 takes precedence
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3205 of identification information, e.g. name or ID code H04N2201/3209 takes precedence
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Abstract

An image processing apparatus for tracking faces in an image stream iteratively receives a new acquired image from the image stream, the image potentially including one or more face regions. The acquired image is sub-sampled (112) at a specified resolution to provide a sub-sampled image. An integral image is then calculated for a least a portion of the sub-sampled image. Fixed size face detection (20) is applied to at least a portion of the integral image to provide a set of candidate face regions. Responsive to the set of candidate face regions produced and any previously detected candidate face regions, the resolution at which a next acquired image is sub-sampled is adjusted.

Claims

1. One or more non-transitory processor-readable storage media having processor-readable code embedded therein for programming a processor to perform a method of tracking faces in an image stream using a digital image acquisition device, the method comprising:

receiving an acquired image from said image stream including one or more face regions;

receiving an indication of relative movement of said acquired image relative to a previously acquired image due to detected motion of the digital image acquisition device, said previously acquired image having an associated set of candidate face regions, each candidate face region having a given size and a respective location; and

applying adjusted face detection to at least a portion of said acquired image in the vicinity of said candidate face regions as a function of said detected motion of the digital image acquisition device to remove variability of changes caused by said motion of the digital image acquisition device, to provide an updated set of candidate face regions.

2. The one or more non-transitory processor-readable storage media of claim 1, wherein the method further comprises repeating said method.

3. The one or more non-transitory processor-readable storage media as claimed in claim 2, wherein said indication of relative movement comprises a size and direction of movement.

4. The one or more non-transitory processor-readable storage media as claimed in claim 2, wherein said adjusted face detection comprises, prior to applying face detection, shifting said associated set of one or more candidate face regions as a function of said movement.

5. The one or more non-transitory processor-readable storage media as claimed in claim 2, wherein said adjusted face detection comprises applying face detection to a region of said acquired image comprising the one or more candidate regions associated with the previously acquired image expanded as a function of said movement.

6. The one or more non-transitory processor-readable storage media as claimed in claim 5, wherein said regions are expanded as a function of their original size and as a function of said movement.

7. The one or more non-transitory processor-readable storage media as claimed in claim 1, wherein said motion of the digital image acquisition device comprises pan.

8. The one or more non-transitory processor-readable storage media as claimed in claim 1, wherein said motion of the digital image acquisition device comprises tilt.

9. The one or more non-transitory processor-readable storage media as claimed in claim 1, wherein said motion of the digital image acquisition device comprises zoom.

10. A portable digital image acquisition device, including an optical system, an image sensor, a processor, and a processor-readable memory having digitally-encoded instructions embedded therein for programming the processor to perform an iterative method of tracking faces in an image stream acquired by the digital image acquisition device, the method comprising:

receiving an acquired image from said image stream including one or more face regions;

receiving an indication of relative movement of said acquired image relative to a previously acquired image due to detected motion of the digital image acquisition device, said previously acquired image having an associated set of candidate face regions, each candidate face region having a given size and a respective location; and

applying adjusted face detection to at least a portion of said acquired image in the vicinity of said candidate face regions as a function of said detected motion of the digital image acquisition device to remove variability of changes caused by said motion of the digital image acquisition device, to provide an updated set of candidate face regions.

11. A device as claimed in claim 10, further comprising a motion sensor including a MEMS sensor.

12. A device as claimed in claim 10, wherein said motion of the digital image acquisition device comprises pan.

13. A device as claimed in claim 10, wherein said motion of the digital image acquisition device comprises tilt.

14. A device as claimed in claim 10, wherein said motion of the digital image acquisition device comprises zoom.

15. A portable digital image acquisition device, including an optical system, an image sensor, a processor, and a processor-readable memory having digitally-encoded instructions embedded therein for programming the processor to perform an iterative method of tracking faces in an image stream acquired by the digital image acquisition device, the method comprising:

receiving an acquired image from said image stream including one or more face regions;

receiving an indication of relative movement of said acquired image relative to a previously acquired image, said previously acquired image having an associated set of candidate face regions, each candidate face region having a given size and a respective location; and

applying adjusted face detection to at least a portion of said acquired image in the vicinity of said candidate face regions as a function of said movement, to provide an updated set of candidate face regions, and

wherein the device further comprises a motion sensor including an accelerometer and a controlled gain amplifier connected to said accelerometer, said device being arranged to set the gain of said amplifier relatively low for acquisition of a high resolution image and to set the gain of said amplifier relatively high during acquisition of a stream of relatively low resolution images.

16. A device as claimed in claim 15, further comprising a motion sensor including a MEMS sensor.

17. A device as claimed in claim 15, wherein said motion of the digital image acquisition device comprises pan.

18. A device as claimed in claim 15, wherein said motion of the digital image acquisition device comprises tilt.

19. A device as claimed in claim 15, wherein said motion of the digital image acquisition device comprises zoom.