System and method for 3D object recognition using range and intensity

• US 20050286767A1
• Filed: 06/22/2005
• Published: 12/29/2005
• Est. Priority Date: 06/23/2004
• Status: Abandoned Application

• Current Assignee: Strider Labs, Inc.
• Original Assignee: Strider Labs, Inc.
• Inventors: Hager, Gregory D., Wegbreit, Eliot Leonard
• Application Number:
  US11/159,660
• Publication Number:
  US 20050286767A1
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces

Abstract

A system and method for performing object and class recognition that allows for wide changes of viewpoint and distance of objects is disclosed. The invention provides for choosing pose-invariant interest points of a three-dimensional (3D) image, and for computing pose-invariant feature descriptors of the image. The system and method also allows for the construction of three-dimensional (3D) object and class models from the pose-invariant interest points and feature descriptors of previously obtained scenes. Interest points and feature descriptors of a newly acquired scene may be compared to the object and/or class models to identify the presence of an object or member of the class in the new scene.

Claims

1. A method of choosing pose-invariant interest points on a three-dimensional (3D) image, comprising the steps of

transforming the intensity image at a plurality of image locations so that the local region about each image location appears approximately as it would appear if it were viewed in a standard pose with respect to a camera;

and

applying one or more interest point operators to the transformed image.

2. The method of claim 1 wherein the step of transforming the image is performed by using the range data to compute the standard pose with respect to the camera.

3. The method of claim 2 wherein the standard pose is such that the image appears as if it were viewed with the camera axis along the surface normal.

4. The method of claim 1 wherein the step of transforming the image further comprises the steps of:

computing a second-order approximation to the local surface geometry from the range data of the 3D image; and

warping the image according to the second-order approximation.

5. The method of claim 2, wherein the step of transforming the image further comprises the steps of:

using the range data to compute the surface normal at each image location; and

using the surface normal and the range data to compute the standard pose with respect to the camera.

6. A method of computing pose-invariant feature descriptors of a three-dimensional (3D) image, comprising the steps of

choosing one or more interest points on the intensity image;

transforming the intensity image so that the local region about each interest point appears approximately as it would appear if it were viewed in a standard pose with respect to a camera; and

computing a feature descriptor comprising a function of the intensity image in the local region about each interest point in the transformed image.

7. The method of claim 6 wherein the step of transforming the image is performed by using the range data to compute the standard pose with respect to the camera.

8. The method of claim 7, wherein the step of transforming the image further comprises the steps of:

using the range data to compute the surface normal at each interest point; and

using the surface normal and the range data to compute the standard pose with respect to the camera.

9. The method of claim 7 wherein the standard pose is such that the image appears as if it were viewed with the camera axis along the surface normal.

10. The method of claim 6 wherein the step of transforming the image further comprises the steps of:

computing a second-order approximation to the local surface geometry from the range data of the 3D image; and

warping the image according to the second-order approximation.

11. The method of 6 wherein the feature descriptor further comprises a function of the local range image as it would appear if it were viewed in a standard pose with respect to the camera.

12. The method of 6 wherein the feature descriptor further comprises a function of the 3D pose of the interest point.

13. The method of 6 wherein the feature descriptor further comprises a function of the 3D pose of one or more other interest points of the image.

14. The method of 6 wherein the step of computing a feature descriptor further comprises computing a dimensionality reduction in the function of the local region.

15. A method for recognizing objects in an observed scene, comprising the steps of

acquiring a three-dimensional (3D) image of the scene;

choosing pose-invariant interest points by applying one or more interest point operators to the intensity component of the image as it would appear if it were viewed in a standard pose with respect to a camera.

computing pose-invariant feature descriptors of the intensity image at the interest points,

constructing a database comprising 3D object models, each object model comprising a set of pose-invariant feature descriptors of one or more images of an object; and

comparing the pose-invariant feature descriptors of the scene image to pose-invariant feature descriptors of the object models.

17. The method of claim 15 wherein the step of comparing the pose-invariant feature descriptors is performed by evaluating the probability that feature descriptors of the scene are the result of observing feature descriptors of the object models.

18. The method of claim 17 wherein the step of evaluating the probability that feature descriptors of the scene are the result of observing feature descriptors of the object models further comprises the steps of:

computing a correspondence of feature descriptors in the scene with feature descriptors of an object model and an alignment under that correspondence,

evaluating an approximation to the likelihood ratio under the correspondence and alignment.

19. The method of claim 18 wherein the step of computing a correspondence and alignment further comprises the steps of

computing a correspondence of a small number of feature descriptors;

computing an alignment based on the small number of feature descriptors; and

iteratively performing the sub-steps of;

identifying potentially visible model features using the alignment;

retaining those visible model features that match feature descriptors in the scene;

updating the correspondence to include the retained model features; and

updating the current alignment based on the retained model features.

16. A method for recognizing objects in an observed scene, comprising the steps of

acquiring a three-dimensional (3D) image of the scene;

choosing pose-invariant interest points in the image;

computing pose-invariant feature descriptors of the image at the interest points, each feature descriptor comprising a function of the local intensity component of the 3D image as it would appear if it were viewed in a standard pose with respect to a camera;

constructing a database comprising 3D object models, each object model comprising a set of pose-invariant feature descriptors of one or more images of an object; and

comparing the pose-invariant feature descriptors of the scene image to pose-invariant feature descriptors of the object models.

20. A method for computing three-dimensional (3D) class models, comprising the steps of

acquiring 3D images of objects with class labels;

choosing pose-invariant interest points in the images by applying one or more interest point operators to the intensity component of the images as they would appear if viewed in a standard pose with respect to a camera;

computing pose-invariant object feature descriptors at the interest points; and

computing functions of the pose-invariant object feature descriptors and the class labels.

22. The method of claim 20 wherein the step of computing functions further comprises computing Gaussian Mixture Models over the feature descriptors, each Gaussian Mixture Model comprising one or more clusters.

23. The method of claim 22 wherein the step of computing functions further comprises computing Gaussian Mixture Models of the global size variation within the class.

24. The method of claim 20 wherein the step of computing functions further comprises computing one or more support vector machines.

21. A method for computing three-dimensional (3D) class models, comprising the steps of

acquiring 3D images of objects with class labels;

choosing pose-invariant interest points in the images;

computing pose-invariant feature descriptors at the interest points, each feature descriptor comprising a function of the local intensity component of the 3D image as it would appear if it were viewed in a standard pose with respect to a camera; and

computing functions of the pose-invariant feature descriptors and the class labels.

25. A method for recognizing instances of classes in an observed scene, comprising the steps of:

acquiring a three-dimensional (3D) image of a scene;

choosing pose-invariant interest points in the image by applying one or more interest point operators to the intensity component of the image as it would appear if it were viewed in a standard pose with respect to a camera;

computing pose-invariant feature descriptors at the interest points;

constructing a database comprising 3D class models; and

comparing pose-invariant feature descriptors of the scene image to the 3D class models.

26. The method of claim 25 wherein the 3D class models comprise Gaussian Mixture Models, each Gaussian Mixture Model comprising one or more clusters.

27. The method of claim 25 wherein the step of comparing the pose-invariant feature descriptors to the 3D class models further comprises evaluating the probability that feature descriptors of the scene are the result of observing clusters of a class model.

28. The method of claim 27 wherein the step of evaluating the probability that feature descriptors of the scene are the result of observing clusters of a class model further comprises the steps of:

computing a correspondence of feature descriptors in the scene with clusters of a class model and an alignment under that correspondence; and

evaluating an approximation to the likelihood ratio under the correspondence and alignment.

29. A method for recognizing instances of classes in an observed scene, comprising the steps of:

acquiring a three-dimensional (3D) image of a scene;

choosing pose-invariant interest points in the image;

computing pose-invariant feature descriptors at the interest points, each feature descriptor comprising a function of the local intensity component of the 3D image as it would appear if it were viewed in a standard pose with respect to a camera;

constructing a database comprising 3D class models; and

comparing pose-invariant feature descriptors of the scene image to the 3D class models.