Methods and apparatuses for face detection

• US 9,396,539 B2
• Filed: 04/02/2010
• Issued: 07/19/2016
• Est. Priority Date: 04/02/2010
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Xu, Lei, Wang, Kongqiao, Li, Jiangwei, Wan, Jimmy Wah Tung, Bay, Jiunn
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US13/639,045
• Publication Number:
  US 20130022243A1
• Time in Patent Office: 2,300 Days
• Field of Search: 382/103, 382/115, 382/116, 382/117, 382/118, 382/190, 382/195, 382/313, 382/305
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images

Abstract

Methods and apparatuses are provided for face detection. A method may include selecting a face detection parameter subset from a plurality of face detection parameter subsets. Each face detection parameter subset may include a subset of face posture models from a set of face posture models and a subset of image patch scales from a set of image patch scales. The method may further include using the selected face detection parameter subset for performing face detection in an image. Corresponding apparatuses are also provided.

Claims

1. A method comprising:

selecting, via detection circuitry, a face detection parameter subset from a plurality of face detection parameter subsets, wherein each face detection parameter subset comprises a subset of face posture models from a set of face posture models and a subset of image patch scales from a set of image patch scales; and

using, via the detection circuitry, the selected face detection parameter subset for performing face detection in an image, wherein performing face detection comprises scanning at least a portion of the image using a sliding window configured to be slideable along a predetermined scanning pattern, wherein using the selected face detection parameter subset for performing face detection in the image further comprises;

scanning at least a portion of the image using a sliding window in a spiral grid pattern having a scanning starting position,

wherein using the selected face detection parameter subset for performing face detection in an image comprises;

extracting an image patch from the image;

estimating a posture of the image patch;

determining one or more face posture models in the selected face detection parameter subset that are applicable to the estimated posture; and

using only the face posture models determined to be applicable to the estimated posture for performing face detection in the image patch.

2. The method according to claim 1, wherein each of the face detection parameter subsets have a substantially equal computational complexity.

3. The method according to claim 1, further comprising:

grouping the set of face posture models into a plurality of model subsets based at least in part on computational complexity of the grouped model subsets; and

grouping the set of image patch scales into a plurality of scale subsets based at least in part on computational complexity of the grouped scale subsets,

wherein each face detection parameter subset comprises a model subset and a scale subset.

4. The method according to claim 1, wherein using the selected face detection parameter subset for performing face detection in the image comprises:

extracting an image patch from each position of the sliding window; and

using the selected face detection parameter subset for performing face detection in each image patch.

5. The method according to claim 1, wherein each model comprises a classifier trained using a plurality of face samples having one or more of a particular view or posture to detect a face having one or more of the particular view or posture.

6. The method according to claim 1, wherein the image comprises a video frame from a video.

7. The method according to claim 6, further comprising:

selecting a face detection parameter subset for each video frame in a sequence of video frames that comprise the video; and

using the selected face detection parameter subsets for face tracking in the sequence of video frames.

8. The method according to claim 7, further comprising:

iteratively assigning a face detection parameter subset from the plurality of face detection parameter subsets to each video frame in the sequence of video frames; and

wherein selecting a face detection parameter for a video frame comprises selecting the face detection parameter subset assigned to the video frame.

9. The method according to claim 6, wherein selecting a face detection parameter subset comprises selecting a face detection parameter subset based at least in part upon a face detection parameter subset used to detect a face in a preceding video frame.

10. The method according to claim 9, wherein using the selected face detection parameter subset for performing face detection in the image comprises using the selected face detection parameter subset for performing face detection in a portion of the image determined based at least in part upon a position at which a face was detected in a preceding video frame.

11. The method according to claim 6, further comprising:

determining the scanning starting position in the image based at least in part upon a position at which a face was detected in a preceding video frame.

12. The method according to claim 1, wherein estimating the posture of the image patch comprises:

using a feature pool to determine confidence scores for each potential rotation range in a rotation plane; and

estimating the posture of the image patch to be one of the potential rotation ranges based at least in part upon the determined confidence scores.

13. The method according to claim 1, wherein estimating the posture of the image patch comprises using a local binary pattern feature pool to estimate the posture of the image patch.

14. An apparatus comprising at least one processor and at least one memory storing computer program code, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to at least:

select a face detection parameter subset from a plurality of face detection parameter subsets, wherein each face detection parameter subset comprises a subset of face posture models from a set of face posture models and a subset of image patch scales from a set of image patch scales; and

use the selected face detection parameter subset for performing face detection in an image, wherein performing face detection comprises scanning at least a portion of the image using a sliding window configured to be slideable along a predetermined scanning pattern, wherein use the selected face detection parameter subset for performing face detection in the image further comprising scan at least a portion of the image using a sliding window in a spiral grid pattern having a scanning starting position,

wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to use the selected face detection parameter subset for performing face detection in an image by;

extracting an image patch from the image;

estimating a posture of the image patch;

determining one or more face posture models in the selected face detection parameter subset that are applicable to the estimated posture; and

using only the face posture models determined to be applicable to the estimated posture for performing face detection in the image patch.

15. The apparatus according to claim 14, wherein each of the face detection parameter subsets have a substantially equal computational complexity.

16. The apparatus according to claim 14, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to further cause the apparatus to:

group the set of face posture models into a plurality of model subsets based at least in part on computational complexity of the grouped model subsets; and

group the set of image patch scales into a plurality of scale subsets based at least in part on computational complexity of the grouped scale subsets,

wherein each face detection parameter subset comprises a model subset and a scale subset.

17. The apparatus according to claim 14, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to use the selected face detection parameter subset for performing face detection in the image by:

scanning at least a portion of the image using a sliding window having a predefined size;

extracting an image patch from each position of the sliding window; and

using the selected face detection parameter subset for performing face detection in each image patch.

18. The apparatus according to claim 14, wherein each model comprises a classifier trained using a plurality of face samples having one or more of a particular view or posture to detect a face having one or more of the particular view or posture.

19. The apparatus according to claim 14, wherein the image comprises a video frame from a video.

20. The apparatus according to claim 19, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to further cause the apparatus to:

select a face detection parameter subset for each video frame in a sequence of video frames that comprise the video; and

use the selected face detection parameter subsets for face tracking in the sequence of video frames.

21. The apparatus according to claim 20, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to further cause the apparatus to:

iteratively assign a face detection parameter subset from the plurality of face detection parameter subsets to each video frame in the sequence of video frames; and

wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to select a face detection parameter for a video frame by selecting the face detection parameter subset assigned to the video frame.

22. The apparatus according to claim 19, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to select a face detection parameter subset by selecting a face detection parameter subset based at least in part upon a face detection parameter subset used to detect a face in a preceding video frame.

23. The apparatus according to claim 19, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to use the selected face detection parameter subset for performing face detection in the image by using the selected face detection parameter subset for performing face detection in a portion of the image determined based at least in part upon a position at which a face was detected in a preceding video frame.

24. The apparatus according to claim 19, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to further cause the apparatus to:

determine the scanning starting position in the image based at least in part upon a position at which a face was detected in a preceding video frame.

25. The apparatus according to claim 14, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to estimate the posture of the image patch by:

using a feature pool to determine confidence scores for each potential rotation range in a rotation plane; and

estimating the posture of the image patch to be one of the potential rotation ranges based at least in part upon the determined confidence scores.

26. The apparatus according to claim 14, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to estimate the posture of the image patch by using a local binary pattern feature pool to estimate the posture of the image patch.

27. The apparatus according to claim 14, wherein the apparatus is embodied on a mobile phone, the mobile phone comprising user interface circuitry and user interface software stored on one or more of the at least one memory;

wherein the user interface circuitry and user interface software are configured to;

facilitate user control of at least some functions of the mobile phone through use of a display; and

cause at least a portion of a user interface of the mobile phone to be displayed on the display to facilitate user control of at least some functions of the mobile phone.

28. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program instructions stored therein, the computer-readable program instructions comprising:

program instructions configured to select a face detection parameter subset from a plurality of face detection parameter subsets, wherein each face detection parameter subset comprises a subset of face posture models from a set of face posture models and a subset of image patch scales from a set of image patch scales; and

program instructions configured to use the selected face detection parameter subset for performing face detection in an image, wherein performing face detection comprises scanning at least a portion of the image using a sliding window configured to be slideable along a predetermined scanning pattern;

wherein use the selected face detection parameter subset for performing face detection in the image further comprising scan at least a portion of the image using a sliding window in a spiral grid pattern having a scanning starting position,

wherein the program instructions that are configured to use the selected face detection parameter subset for performing face detection in an image comprises program instructions configured to;

extract an image patch from the image;

estimate a posture of the image patch;

determine one or more face posture models in the selected face detection parameter subset that are applicable to the estimated posture; and

use only the face posture models determined to be applicable to the estimated posture for performing face detection in the image patch.