System and method for pose-angle estimation

• US 6,959,109 B2
• Filed: 06/20/2002
• Issued: 10/25/2005
• Est. Priority Date: 06/20/2002
• Status: Expired due to Term

• Current Assignee: Morphotrust USA Incorporated (Advent International Corporation)
• Original Assignee: Identix, Inc. (Advent International Corporation)
• Inventors: Moustafa, Mohamed Nabil
• Primary Examiner(s): Dastouri, Mehrdad
• Application Number:
  US10/175,929
• Publication Number:
  US 20030235332A1
• Time in Patent Office: 1,223 Days
• Field of Search: 382/115, 382/117, 382/118, 382156-160, 382/282, 382/298, 382/253, 382/216, 340/5.53, 340 58- 583, 340/553, 706/15, 706/20, 706/25
• US Class Current: 382/159
• CPC Class Codes: G06F 18/2137 based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/164 using holistic features

Abstract

A system and method are disclosed for determining the pose angle of an object in an input image. In a preferred embodiment, the present system comprises a pose estimator having a prototype projector, a regression estimator, and an angle calculator. The prototype projector is preferably adapted to reduce the input image dimensionality for faster further processing by projecting the input pixels of the image onto a Self-Organizing Map (SOM) neural network. The regression estimator is preferably implemented as a neural network and adapted to map the projections to a pattern unique to each pose. The angle calculator preferably includes a curve fitter and an error analyzer. The curve fitter is preferably adapted to estimate the pose angle from the mapping pattern. The error analyzer is preferably adapted to produce a confidence signal representing the likelihood of the input image being a face at the calculated pose. The system also preferably includes two network trainers responsible for synthesizing the neural networks.

Claims

1. A system for determining a pose angle of an object in an input image in at least one dimension, comprising:

a first neural network trained in an unsupervised mode using a first plurality of training images to develop a plurality of weight vectors, each weight vector representing a plurality of the training images;

a projector adapted to receive an input image and generate a projection vector comprising a plurality of calculated distances representing distances between the input image and the weight vectors;

a second neural network trained in a supervised mode with a second plurality of training images, the second plurality of training images comprising objects at known pose angles, the second neural network comprising a plurality of neurons, each neuron tuned to a particular pose angle and adapted to receive the projection vector and output a value in accordance with a function that is even and monotonically decreasing for changes in the pose angle of the objects from the pose angle to which each neuron is tuned;

a curve fitter adapted to receive the values output by the neurons and estimate the pose angle of the object in at least one dimension; and

an error calculator that generates a value representative of the error between the values output by the neurons and the calculated values for the function using the estimated pose angle.

2. The system of claim 1, wherein the object is a head.

3. The system of claim 1, wherein the function is:

cos(α

(x)−

α

_k), where α

_k is the angle to which a neuron is tuned, and where α

(x) is the estimated pose angle.

4. The system of claim 1, wherein the curve fitter implements a minimum square-error curve-fitting algorithm.

5. The system of claim 1, further comprising an imaging sensor adapted to capture an image in a field of view of the sensor.

6. The system of claim 5, wherein the imaging sensor comprises a video camera.

7. The system of claim 5, wherein the imaging sensor comprises a digital camera.

8. The system of claim 1, wherein the input image is derived form a live subject.

9. The system of claim 1, wherein the input image is derived from a photograph.

10. The system of claim 1, wherein the input image is derived from a printed image.

11. The system of claim 1, wherein the input image is derived from an artists'"'"' rendering.

12. The system of claim 1, wherein at least some of the first plurality of training images are the same as at least some of the second plurality of training images.

13. The system of claim 12, wherein the first plurality of training images and the second plurality of training images comprise the same training images.

14. The system of claim 1, wherein the number of weight vectors is greater than the number of output nodes of the second neural network.

15. The system of claim 14, wherein the number of weight vectors is approximately twice the number of output nodes of the second neural network.

16. The system of claim 1, wherein the input image is generated by cropping a portion of an image that contains the object.

17. The system of claim 1, wherein the projector is adapted to rescale the input image to canonical dimensions.

18. The system of claim 17, wherein the canonical dimensions are 20 pixels by 20 pixels.

19. The system of claim 1, wherein the at least one dimension is yaw.

20. The system of claim 1, wherein the at least one dimension is pitch.

21. The system of claim 1, wherein the at least one dimension is roll.

22. The system of claim 1, wherein the system is adapted to estimate the yaw, pitch, and roll pose angles.

23. A method for determining a pose angle of an object in an input image in at least one dimension, comprising:

training a first neural network in an unsupervised mode using a first plurality of training images to develop a plurality of weight vectors, each weight vector representing a plurality of the training images;

projecting the input image onto the first neural network to generate a projection vector comprising a plurality of calculated distances representing distances between the input image and the weight vectors;

training a second neural network in a supervised mode with a second plurality of training images, the second plurality of training images comprising objects at known pose angles, the second neural network comprising a plurality of neurons, each neuron tuned to a particular pose angle;

processing the projection vector in the second neural network in accordance with a function that is even and monotonically decreasing for changes in the pose angle of the object from the pose angle to which each neuron is tuned to generate a plurality of output values;

fitting the output values to a curve to estimate the pose angle of the object in at least one dimension; and

calculating a value representative of the error between the output values and the calculated values for the function using the estimated pose angle.

24. The method of claim 23, wherein the object is a head.

25. The method of claim 24, wherein the head includes a face discernable in the image.

26. The method of claim 23, wherein the function is:

cos(α

(x)−

α

_k), where α

_k is the angle to which a neuron is tuned, and where α

(x) is the estimated pose angle.

27. The method of claim 23, wherein the curve fitter implements a minimum square-error curve-fitting algorithm.

28. The method of claim 23, further comprising an imaging sensor adapted to capture an image in a field of view of the sensor.

29. The method of claim 28, wherein the imaging sensor comprises a video camera.

30. The method of claim 28, wherein the imaging sensor comprises a digital camera.

31. The method of claim 23, wherein the input image is derived from a live subject.

32. The method of claim 23, wherein the input image is derived from a photograph.

33. The method of claim 23, wherein the input image is derived from a printed image.

34. The method of claim 23, wherein the input image is derived from an artists'"'"' rendering.

35. The method of claim 23, wherein at least some of the first plurality of training images are the same as at least some of the second plurality of training images.

36. The method of claim 35, wherein the first plurality of training images and the second plurality of training images comprise the same training images.

37. The method of claim 23, wherein the number of weight vectors is greater than the number of output nodes of the second neural network.

38. The method of claim 37, wherein the number of weight vectors is approximately twice the number of output nodes of the second neural network.

39. The method of claim 23, wherein the input image is generated by cropping a portion of an image that contains the object.

40. The method of claim 23, wherein the projector is adapted to rescale the input image to canonical dimensions.

41. The method of claim 40, wherein the canonical dimensions are 20 pixels by 20 pixels.

42. The method of claim 23, wherein the at least one dimension is yaw.

43. The method of claim 23, wherein the at least one dimension is pitch.

44. The method of claim 23, wherein the at least one dimension is roll.

45. The method of claim 23, wherein the system is adapted to estimate the yaw, pitch, and roll pose angles.

46. A system for determining a pose angle of an object in an input image in at least one dimension, comprising:

a first neural network using a first plurality of training images to develop a plurality of weight vectors, each weight vector representing a plurality of the training images;

a projector adapted to receive an input image and generate a projection vector comprising a plurality of calculated distances representing distances between the input image and the weight vectors;

a second neural network trained in a supervised mode with a second plurality of training images, the second plurality of training images comprising objects at known pose angles, the second neural network comprising a plurality of neurons, each neuron tuned to a particular pose angle and adapted to receive the projection vector and output a value in accordance with a function that is even and monotonically decreasing for changes in the pose angle of the object from the pose angle to which each neuron is tuned;

a curve fitter adapted to receive the values output by the neurons and estimate the pose angle of the object in at least one dimension; and

an error calculator that generates a value representative of the error between the values output by the neurons and the calculated values for the function using the estimated pose angle.

47. The system of claim 46, wherein the first neural network is trained in a supervised mode or an unsupervised mode.

48. A method for determining a pose angle of an object in an input image in at least one dimension, comprising:

training a first neural network using a first plurality of training images to develop a plurality of weight vectors, each weight vector representing a plurality of the training images;

projecting the input image onto the first neural network to generate a projection vector comprising a plurality of calculated distances representing distances between the input image and the weight vectors;

training a second neural network in a supervised mode with a second plurality of training images, the second plurality of training images comprising objects at known pose angles, the second neural network comprising a plurality of neurons, each neuron tuned to a particular pose angle;

processing the projection vector in the second neural network in accordance with a function that is even and monotonically decreasing for changes in the pose angle of the object from the pose angle to which each neuron is tuned to generate a plurality of output values;

fitting the output values to a curve to estimate the pose angle of the object in at least one dimension; and

calculating a value representative of the error between the output values and the calculated values for the function using the estimated pose angle.

49. The method of claim 48, wherein the first neural network is trained in a supervised mode or an unsupervised mode.