METHODS AND APPARATUSES FOR FACE DETECTION

• US 20130022243A1
• Filed: 04/02/2010
• Published: 01/24/2013
• Est. Priority Date: 04/02/2010
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Xu, Lei, Wang, Kongqiao, Li, Jiangwei, Bay, Jiunn, Wan, Jimmy Wah Tung
• Granted Patent: US 9,396,539 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images

Abstract

Methods and apparatuses are provided for face detection. A method may include selecting a face detection parameter subset from a plurality of face detection parameter subsets. Each face detection parameter subset may include a subset of face posture models from a set of face posture models and a subset of image patch scales from a set of image patch scales. The method may further include using the selected face detection parameter subset for performing face detection in an image. Corresponding apparatuses are also provided.

Claims

1. A method comprising:

selecting a face detection parameter subset from a plurality of face detection parameter subsets, wherein each face detection parameter subset comprises a subset of face posture models from a set of face posture models and a subset of image patch scales from a set of image patch scales; and

using the selected face detection parameter subset for performing face detection in an image.

2. The method according to claim 1, wherein each of the face detection parameter subsets have a substantially equal computational complexity.

3. The method according to any of claims 1-2, further comprising:

grouping the set of face posture models into a plurality of model subsets based at least in part on computational complexity of the grouped model subsets; and

grouping the set of image patch scales into a plurality of scale subsets based at least in part on computational complexity of the grouped scale subsets; and

wherein each face detection parameter subset comprises a model subset and a scale subset.

4. The method according to any of claims 1-3, wherein using the selected face detection parameter subset for performing face detection in the image comprises:

scanning at least a portion of the image using a sliding window having a predefined size;

extracting an image patch from each position of the sliding window; and

using the selected face detection parameter subset for performing face detection in each image patch.

5. The method according to any of claims 1-4, wherein each model comprises a classifier trained using a plurality of face samples having one or more of a particular view or posture to detect a face having one or more of the particular view or posture.

6. The method according to any of claims 1-5, wherein the image comprises a video frame from a video.

7. The method according to claim 6, further comprising:

selecting a face detection parameter subset for each video frame in a sequence of video frames that comprise the video; and

using the selected face detection parameter subsets for face tracking in the sequence of video frames.

8. The method according to claim 7, further comprising:

iteratively assigning a face detection parameter subset from the plurality of face detection parameter subsets to each video frame in the sequence of video frames; and

wherein selecting a face detection parameter for a video frame comprises selecting the face detection parameter subset assigned to the video frame.

9. The method according to any of claims 6-8, wherein selecting a face detection parameter subset comprises selecting a face detection parameter subset based at least in part upon a face detection parameter subset used to detect a face in a preceding video frame.

10. The method according to any of claims 6-9, wherein using the selected face detection parameter subset for performing face detection in the image comprises using the selected face detection parameter subset for performing face detection in a portion of the image determined based at least in part upon a position at which a face was detected in a preceding video frame.

11. The method according to any of claims 6-10, further comprising:

determining a scanning starting position in the image based at least in part upon a position at which a face was detected in a preceding video frame; and

wherein using the selected face detection parameter subset for performing face detection in the image comprises;

scanning at least a portion of the image using a sliding window in a spiral grid pattern having a starting position at the determined scanning starting position.

12. The method according to any of claims 1-11, wherein using the selected face detection parameter subset for performing face detection in an image comprises:

extracting an image patch from the image;

estimating a posture of the image patch;

determining one or more face posture models in the selected face detection parameter subset that are applicable to the estimated posture; and

using only the face posture models determined to be applicable to the estimated posture for performing face detection in the image patch.

13. The method according to claim 12, wherein estimating the posture of the image patch comprises:

using a feature pool to determine confidence scores for each potential rotation range in a rotation plane; and

estimating the posture of the image patch to be one of the potential rotation ranges based at least in part upon the determined confidence scores.

14. The method according to any of claims 12-13, wherein estimating the posture of the image patch comprises using a local binary pattern feature pool to estimate the posture of the image patch.

15. An apparatus comprising at least one processor and at least one memory storing computer program code, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to at least:

select a face detection parameter subset from a plurality of face detection parameter subsets, wherein each face detection parameter subset comprises a subset of face posture models from a set of face posture models and a subset of image patch scales from a set of image patch scales; and

use the selected face detection parameter subset for performing face detection in an image.

16. The apparatus according to claim 15, wherein each of the face detection parameter subsets have a substantially equal computational complexity.

17. The apparatus according to any of claims 15-16, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to further cause the apparatus to:

group the set of face posture models into a plurality of model subsets based at least in part on computational complexity of the grouped model subsets; and

group the set of image patch scales into a plurality of scale subsets based at least in part on computational complexity of the grouped scale subsets; and

wherein each face detection parameter subset comprises a model subset and a scale subset.

18. The apparatus according to any of claims 15-17, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to use the selected face detection parameter subset for performing face detection in the image by:

scanning at least a portion of the image using a sliding window having a predefined size;

extracting an image patch from each position of the sliding window; and

using the selected face detection parameter subset for performing face detection in each image patch.

19. The apparatus according to any of claims 15-18, wherein each model comprises a classifier trained using a plurality of face samples having one or more of a particular view or posture to detect a face having one or more of the particular view or posture.

20. The apparatus according to any of claims 15-19, wherein the image comprises a video frame from a video.

21. The apparatus according to claim 20, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to further cause the apparatus to:

select a face detection parameter subset for each video frame in a sequence of video frames that comprise the video; and

use the selected face detection parameter subsets for face tracking in the sequence of video frames.

22. The apparatus according to claim 21, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to further cause the apparatus to:

iteratively assign a face detection parameter subset from the plurality of face detection parameter subsets to each video frame in the sequence of video frames; and

wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to select a face detection parameter for a video frame by selecting the face detection parameter subset assigned to the video frame.

23. The apparatus according to any of claims 20-22, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to select a face detection parameter subset by selecting a face detection parameter subset based at least in part upon a face detection parameter subset used to detect a face in a preceding video frame.

24. The apparatus according to any of claims 20-23, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to use the selected face detection parameter subset for performing face detection in the image by using the selected face detection parameter subset for performing face detection in a portion of the image determined based at least in part upon a position at which a face was detected in a preceding video frame.

25. The apparatus according to any of claims 20-24, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to further cause the apparatus to:

determine a scanning starting position in the image based at least in part upon a position at which a face was detected in a preceding video frame; and

wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to use the selected face detection parameter subset for performing face detection in the image by;

scanning at least a portion of the image using a sliding window in a spiral grid pattern having a starting position at the determined scanning starting position.

26. The apparatus according to any of claims 15-25, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to use the selected face detection parameter subset for performing face detection in an image by:

extracting an image patch from the image;

estimating a posture of the image patch;

determining one or more face posture models in the selected face detection parameter subset that are applicable to the estimated posture; and

using only the face posture models determined to be applicable to the estimated posture for performing face detection in the image patch.

27. The apparatus according to claim 26, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to estimate the posture of the image patch by:

using a feature pool to determine confidence scores for each potential rotation range in a rotation plane; and

estimating the posture of the image patch to be one of the potential rotation ranges based at least in part upon the determined confidence scores.

28. The apparatus according to any of claims 26-27, wherein the at least one memory and stored computer program code are configured, with the at least one processor, to cause the apparatus to estimate the posture of the image patch by using a local binary pattern feature pool to estimate the posture of the image patch.

29. The apparatus according to any of claims 15-28, wherein the apparatus comprises or is embodied on a mobile phone, the mobile phone comprising user interface circuitry and user interface software stored on one or more of the at least one memory;

wherein the user interface circuitry and user interface software are configured to;

facilitate user control of at least some functions of the mobile phone through use of a display; and

cause at least a portion of a user interface of the mobile phone to be displayed on the display to facilitate user control of at least some functions of the mobile phone.

30. A computer program product comprising at least one computer-readable storage medium having computer-readable program instructions stored therein, the computer-readable program instructions comprising:

program instructions configured to select a face detection parameter subset from a plurality of face detection parameter subsets, wherein each face detection parameter subset comprises a subset of face posture models from a set of face posture models and a subset of image patch scales from a set of image patch scales; and

program instructions configured to use the selected face detection parameter subset for performing face detection in an image.

31. The computer program product according to claim 30, wherein each of the face detection parameter subsets have a substantially equal computational complexity.

32. The computer program product according to any of claims 30-31, further comprising:

program instructions configured to group the set of face posture models into a plurality of model subsets based at least in part on computational complexity of the grouped model subsets; and

program instructions configured to group the set of image patch scales into a plurality of scale subsets based at least in part on computational complexity of the grouped scale subsets; and

wherein each face detection parameter subset comprises a model subset and a scale subset.

33. The computer program product according to any of claims 30-32, wherein the program instructions configured to use the selected face detection parameter subset for performing face detection in the image comprise:

program instructions configured to scan at least a portion of the image using a sliding window having a predefined size;

program instructions configured to extract an image patch from each position of the sliding window; and

program instructions configured to use the selected face detection parameter subset for performing face detection in each image patch.

34. The computer program product according to any of claims 30-33, wherein each model comprises a classifier trained using a plurality of face samples having one or more of a particular view or posture to detect a face having one or more of the particular view or posture.

35. The computer program product according to any of claims 30-34, wherein the image comprises a video frame from a video.

36. The computer program product according to claim 35, further comprising:

program instructions configured to select a face detection parameter subset for each video frame in a sequence of video frames that comprise the video; and

program instructions configured to use the selected face detection parameter subsets for face tracking in the sequence of video frames.

37. The computer program product according to claim 36, further comprising:

program instructions configured to iteratively assign a face detection parameter subset from the plurality of face detection parameter subsets to each video frame in the sequence of video frames; and

wherein the program instructions configured to select a face detection parameter for a video frame comprise program instructions configured to select the face detection parameter subset assigned to the video frame.

38. The computer program product according to any of claims 35-37, wherein the program instructions configured to select a face detection parameter subset comprise program instructions configured to select a face detection parameter subset based at least in part upon a face detection parameter subset used to detect a face in a preceding video frame.

39. The computer program product according to any of claims 35-38, wherein the program instructions configured to use the selected face detection parameter subset for performing face detection in the image comprises program instructions configured to use the selected face detection parameter subset for performing face detection in a portion of the image determined based at least in part upon a position at which a face was detected in a preceding video frame.

40. The computer program product according to any of claims 35-39, further comprising:

program instructions configured to determine a scanning starting position in the image based at least in part upon a position at which a face was detected in a preceding video frame; and

wherein the program instructions configured to use the selected face detection parameter subset for performing face detection in the image comprise;

program instructions configured to scan at least a portion of the image using a sliding window in a spiral grid pattern having a starting position at the determined scanning starting position.

41. The computer program product according to any of claims 30-40, wherein the program instructions configured to use the selected face detection parameter subset for performing face detection in an image comprise:

program instructions configured to extract an image patch from the image;

program instructions configured to estimate a posture of the image patch;

program instructions configured to determine one or more face posture models in the selected face detection parameter subset that are applicable to the estimated posture; and

program instructions configured to use only the face posture models determined to be applicable to the estimated posture for performing face detection in the image patch.

42. The computer program product according to claim 41, wherein the program instructions configured to estimate the posture of the image patch comprise:

program instructions configured to use a feature pool to determine confidence scores for each potential rotation range in a rotation plane; and

program instructions configured to estimate the posture of the image patch to be one of the potential rotation ranges based at least in part upon the determined confidence scores.

43. The computer program product according to any of claims 41-42, wherein the program instructions configured to estimate the posture of the image patch comprise program instructions configured to use a local binary pattern feature pool to estimate the posture of the image patch.

44. A computer-readable storage medium carrying computer-readable program instructions, the computer-readable program instructions comprising:

program instructions configured to select a face detection parameter subset from a plurality of face detection parameter subsets, wherein each face detection parameter subset comprises a subset of face posture models from a set of face posture models and a subset of image patch scales from a set of image patch scales; and

program instructions configured to use the selected face detection parameter subset for performing face detection in an image.

45. The computer-readable storage medium according to claim 44, wherein each of the face detection parameter subsets have a substantially equal computational complexity.

46. The computer-readable storage medium according to any of claims 44-45, further comprising:

program instructions configured to group the set of face posture models into a plurality of model subsets based at least in part on computational complexity of the grouped model subsets; and

program instructions configured to group the set of image patch scales into a plurality of scale subsets based at least in part on computational complexity of the grouped scale subsets; and

wherein each face detection parameter subset comprises a model subset and a scale subset.

47. The computer-readable storage medium according to any of claims 44-46, wherein the program instructions configured to use the selected face detection parameter subset for performing face detection in the image comprise:

program instructions configured to scan at least a portion of the image using a sliding window having a predefined size;

program instructions configured to extract an image patch from each position of the sliding window; and

program instructions configured to use the selected face detection parameter subset for performing face detection in each image patch.

48. The computer-readable storage medium according to any of claims 44-47, wherein each model comprises a classifier trained using a plurality of face samples having one or more of a particular view or posture to detect a face having one or more of the particular view or posture.

49. The computer-readable storage medium according to any of claims 44-48, wherein the image comprises a video frame from a video.

50. The computer-readable storage medium according to claim 49, further comprising:

program instructions configured to select a face detection parameter subset for each video frame in a sequence of video frames that comprise the video; and

program instructions configured to use the selected face detection parameter subsets for face tracking in the sequence of video frames.

51. The computer-readable storage medium according to claim 50, further comprising:

program instructions configured to iteratively assign a face detection parameter subset from the plurality of face detection parameter subsets to each video frame in the sequence of video frames; and

wherein the program instructions configured to select a face detection parameter for a video frame comprise program instructions configured to select the face detection parameter subset assigned to the video frame.

52. The computer-readable storage medium according to any of claims 49-51, wherein the program instructions configured to select a face detection parameter subset comprise program instructions configured to select a face detection parameter subset based at least in part upon a face detection parameter subset used to detect a face in a preceding video frame.

53. The computer-readable storage medium according to any of claims 49-52, wherein the program instructions configured to use the selected face detection parameter subset for performing face detection in the image comprises program instructions configured to use the selected face detection parameter subset for performing face detection in a portion of the image determined based at least in part upon a position at which a face was detected in a preceding video frame.

54. The computer-readable storage medium according to any of claims 49-53, further comprising:

program instructions configured to determine a scanning starting position in the image based at least in part upon a position at which a face was detected in a preceding video frame; and

wherein the program instructions configured to use the selected face detection parameter subset for performing face detection in the image comprise;

program instructions configured to scan at least a portion of the image using a sliding window in a spiral grid pattern having a starting position at the determined scanning starting position.

55. The computer-readable storage medium according to any of claims 44-54, wherein the program instructions configured to use the selected face detection parameter subset for performing face detection in an image comprise:

program instructions configured to extract an image patch from the image;

program instructions configured to estimate a posture of the image patch;

program instructions configured to determine one or more face posture models in the selected face detection parameter subset that are applicable to the estimated posture; and

program instructions configured to use only the face posture models determined to be applicable to the estimated posture for performing face detection in the image patch.

56. The computer-readable storage medium according to claim 55, wherein the program instructions configured to estimate the posture of the image patch comprise:

program instructions configured to use a feature pool to determine confidence scores for each potential rotation range in a rotation plane; and

program instructions configured to estimate the posture of the image patch to be one of the potential rotation ranges based at least in part upon the determined confidence scores.

57. The computer-readable storage medium according to any of claims 55-56, wherein the program instructions configured to estimate the posture of the image patch comprise program instructions configured to use a local binary pattern feature pool to estimate the posture of the image patch.

58. An apparatus comprising:

means for selecting a face detection parameter subset from a plurality of face detection parameter subsets, wherein each face detection parameter subset comprises a subset of face posture models from a set of face posture models and a subset of image patch scales from a set of image patch scales; and

means for using the selected face detection parameter subset for performing face detection in an image.

59. The apparatus according to claim 58, wherein each of the face detection parameter subsets have a substantially equal computational complexity.

60. The apparatus according to any of claims 58-59, further comprising:

means for grouping the set of face posture models into a plurality of model subsets based at least in part on computational complexity of the grouped model subsets; and

means for grouping the set of image patch scales into a plurality of scale subsets based at least in part on computational complexity of the grouped scale subsets; and

wherein each face detection parameter subset comprises a model subset and a scale subset.

61. The apparatus according to any of claims 58-60, wherein the means for using the selected face detection parameter subset for performing face detection in the image comprise:

means for scanning at least a portion of the image using a sliding window having a predefined size;

means for extracting an image patch from each position of the sliding window; and

means for using the selected face detection parameter subset for performing face detection in each image patch.

62. The apparatus according to any of claims 58-61, wherein each model comprises a classifier trained using a plurality of face samples having one or more of a particular view or posture to detect a face having one or more of the particular view or posture.

63. The apparatus according to any of claims 58-62, wherein the image comprises a video frame from a video.

64. The apparatus according to claim 63, further comprising:

means for selecting a face detection parameter subset for each video frame in a sequence of video frames that comprise the video; and

means for using the selected face detection parameter subsets for face tracking in the sequence of video frames.

65. The apparatus according to claim 64, further comprising:

means for iteratively assigning a face detection parameter subset from the plurality of face detection parameter subsets to each video frame in the sequence of video frames; and

wherein the means for selecting a face detection parameter for a video frame comprise means for selecting the face detection parameter subset assigned to the video frame.

66. The apparatus according to any of claims 63-65, wherein the means for selecting a face detection parameter subset comprise means for selecting a face detection parameter subset based at least in part upon a face detection parameter subset used to detect a face in a preceding video frame.

67. The apparatus according to any of claims 63-66, wherein the means for using the selected face detection parameter subset for performing face detection in the image comprise means for using the selected face detection parameter subset for performing face detection in a portion of the image determined based at least in part upon a position at which a face was detected in a preceding video frame.

68. The apparatus according to any of claims 63-67, further comprising:

means for determining a scanning starting position in the image based at least in part upon a position at which a face was detected in a preceding video frame; and

wherein the means for using the selected face detection parameter subset for performing face detection in the image comprise;

means for scanning at least a portion of the image using a sliding window in a spiral grid pattern having a starting position at the determined scanning starting position.

69. The apparatus according to any of claims 58-68, wherein the means for using the selected face detection parameter subset for performing face detection in an image comprise:

means for extracting an image patch from the image;

means for estimating a posture of the image patch;

means for determining one or more face posture models in the selected face detection parameter subset that are applicable to the estimated posture; and

means for using only the face posture models determined to be applicable to the estimated posture for performing face detection in the image patch.

70. The apparatus according to claim 69, wherein the means for estimating the posture of the image patch comprise:

means for using a feature pool to determine confidence scores for each potential rotation range in a rotation plane; and

means for estimating the posture of the image patch to be one of the potential rotation ranges based at least in part upon the determined confidence scores.

71. The apparatus according to any of claims 69-70, wherein the means for estimating the posture of the image patch comprise means for using a local binary pattern feature pool to estimate the posture of the image patch.