System and method for estimating the orientation of an object

• US 6,741,756 B1
• Filed: 09/30/1999
• Issued: 05/25/2004
• Est. Priority Date: 09/30/1999
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wu, Ying, Toyama, Kentaro
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): Patel, Kanji
• Application Number:
  US09/408,745
• Time in Patent Office: 1,699 Days
• Field of Search: 382/100, 382/117, 382/118, 382/115, 382/156, 382/159, 382/160, 382/168, 382/190, 382/195, 382/199, 382/209, 382/216, 382/263, 382/266, 382/285, 382/289, 382/291, 382/296, 382/298, 345/649, 706/16, 706/20
• US Class Current: 382/291
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

The present invention is embodied in a system and method for automatically estimating the orientation or pose of an object, such as a human head, from any viewpoint and includes training and pose estimation modules. The training module uses known head poses for generating observations of the different types of head poses and the pose estimation module receives actual head poses of a subject and uses the training observations to estimate the actual head pose. Namely, the training module receives training data and extracts unique features of the data, projects the features onto corresponding points of a model and determines a probability density function estimation for each model point to produce a trained model. The pose estimation module receives the trained model and an input object and extracts unique input features of the input object, projects the input features onto points of the trained model and determines an orientation of the input object that most likely generates the features extracted from input object.

Claims

1. A method for estimating the orientation of an object comprising:

providing a trained model of the object as training data, wherein the trained model of the object is produced by extracting unique features of the training data, projecting the features of the training data onto corresponding points of a model and determining a probability density function estimation for each model point;

extracting unique features of an image representing the object;

back projecting the features of the image onto points of the trained model; and

determining an estimated orientation of the object that most likely generates the features extracted from object.

2. The method of claim 1, further comprising using a predictive prior to bias the estimation of orientation.

3. The method of claim 1, further comprising preprocessing the image.

4. The method of claim 3, wherein preprocessing the image includes resizing the image.

5. The method of claim 3, wherein preprocessing the image includes masking the image to filter out unwanted background noise.

6. The method of claim 3, wherein preprocessing the image includes performing histogram equalization on the image.

7. The method of claim 1 wherein extracting unique features includes determining convolution coefficients for each point in the image by applying predefined convolution templates.

8. The method of claim 1, wherein extracting unique features includes computing normalized feature vectors.

9. The method of claim 1, wherein extracting unique features includes determining dense occurrences of high-frequency components in local image regions.

10. The method of claim 9 wherein determining dense occurrences of high-frequency components in local image regions is performed with rotationally-invariant Gabor templates.

11. The method of claim 9 wherein determining dense occurrences of high-frequency components in local image regions with a Gaussian at a coarse scale.

12. The method of claim 9, wherein determining dense occurrences of high-frequency components in local image regions a single Gaussian and plural Laplacian templates.

13. The method of claim 1, wherein back projecting the features includes rotating the trained model.

14. The method of claim 13 wherein back projecting the features further includes projecting previously extracted feature vectors of the input image onto the rotated trained model.

15. The method of claim 14 wherein back projecting the features further includes computing a likelihood that each pose produces observed features using probability density functions of the trained model.

16. The method of claim 1, wherein projecting the features onto corresponding points of a predefined model includes using known orientations and rotating the predefined model to coincide with the known orientations.

17. The method of claim 16 wherein projecting the features onto corresponding points of a model includes projecting previously extracted features of the image onto corresponding points of the rotated model.

18. The method of claim 17 wherein projecting the features onto corresponding points of a model includes accumulating and storing data for plural images representing the object.

19. The method of claim 1 wherein the model is a three dimensional ellipsoidal model.

20. An orientation system for estimating the orientation of an object comprising a pose estimation module that receives a trained model of the object and an input object and extracts unique features of the input object, back projects the features onto points of the trained model and determines an orientation of the input object that most likely generates the features extracted from input object;

and

wherein the estimation module uses a predictive prior to bias the output estimation of orientation.

21. The orientation system of claim 20, further comprising a training module that receives training data and extracts unique features of the training data, projects the features of the training data onto corresponding points of a model and determines a probability density function estimation for each model point to produce the trained model.

22. The orientation system of claim 21 wherein back projecting the features includes rotating the trained model.

23. The orientation system of claim 22, wherein back projecting the features further includes projecting previously extracted feature vectors of the input image onto the rotated trained model.

24. The orientation system of claim 23, wherein back projecting the features further includes computing a likelihood that each pose produces observed features using probability density functions of the trained model.

25. The orientation system of claim 21, wherein projecting the features onto corresponding points of a predefined model includes using known orientations and rotating the predefined model to coincide with the known orientations.

26. The orientation system of claim 25, wherein projecting the features onto corresponding points of the model includes projecting previously extracted features of the image onto corresponding points of the rotated model.

27. The orientation system of claim 20 wherein the model is a three dimensional ellipsoidal model.

28. The orientation system of claim 20, further comprising a preprocessing module capable of at least one of resizing the image, masking the image to filter out unwanted background noise and performing histogram equalization on the image.

29. The orientation system of claim 20 wherein the pose estimation module determines convolution coefficients for each point in the image by applying predefined convolution templates.

30. The orientation system of claim 20 wherein extracting unique features includes computing normalized feature vectors.

31. The orientation system of claim 20, wherein extracting unique features includes determining dense occurrences of high-frequency components in local image regions.

32. The orientation of system claim 31, wherein determining dense occurrences of high-frequency components in local image regions is performed with rotationally-invariant Gabor templates.

33. A method for determining the orientation of a head, comprising:

extracting unique features of training data representing different heads;

projecting the features of the training data onto corresponding points of a model;

determining a probability density function estimation for each model point;

extracting unique features of the head;

back projecting the features of the head onto points of the trained model; and

determining an estimated orientation of the head that most likely generates the features extracted from head.

34. The method of claim 33, further comprising using a predictive prior to bias the output determination of orientation.

35. The method of claim 33, wherein the model is a three dimensional ellipsoidal model.

36. The method of claim 33, further comprising determining convolution coefficients for each point in the head by applying predefined convolution templates.

37. The method of claim 33, wherein back projecting the features includes rotating the trained model, projecting previously extracted feature vectors of the input image onto the rotated trained model and computing a likelihood that each pose produces observed features using probability density functions of the trained model.

38. The method of claim 33, wherein projecting the features onto corresponding points of a predefined model includes using known orientations and rotating the predefined model to coincide with the known orientations and projecting previously extracted features of the image onto corresponding points of the rotated model.

39. The method system of claim 33, wherein extracting unique features includes computing normalized feature vectors.

40. The method of claim 33, wherein extracting unique features includes determining dense occurrences of high-frequency components in local image regions.

41. The method of system claim 40 wherein determining dense occurrences of high-frequency components in local image regions is performed with rotationally-invariant Gabor templates.