Object recognition with 3D models

• US 8,422,797 B2
• Filed: 06/30/2010
• Issued: 04/16/2013
• Est. Priority Date: 07/01/2009
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Heisele, Bernd, Kim, Gunhee, Meyer, Andrew J.
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): LE, TOTAM HA
• Application Number:
  US12/827,185
• Publication Number:
  US 20110002531A1
• Time in Patent Office: 1,021 Days
• Field of Search: 382/155, 382/159, 382/160, 382/224, 382/228, 345418-427
• US Class Current: 382/224
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

An “active learning” method trains a compact classifier for view-based object recognition. The method actively generates its own training data. Specifically, the generation of synthetic training images is controlled within an iterative training process. Valuable and/or informative object views are found in a low-dimensional rendering space and then added iteratively to the training set. In each iteration, new views are generated. A sparse training set is iteratively generated by searching for local minima of a classifier'"'"'s output in a low-dimensional space of rendering parameters. An initial training set is generated. The classifier is trained using the training set. Local minima are found of the classifier'"'"'s output in the low-dimensional rendering space. Images are rendered at the local minima. The newly-rendered images are added to the training set. The procedure is repeated so that the classifier is retrained using the modified training set.

Claims

1. A computer-implemented method for training a view-based object recognition classifier, comprising:

generating an initial training set of images, wherein an image in the initial training set is generated based on a three-dimensional model and a set of rendering parameter values, the rendering parameter values comprising at least one of an azimuth location of a viewpoint, an elevation location of a viewpoint, and a rotation of a viewpoint around its optical axis;

training the classifier using the initial training set;

determining the classifier'"'"'s accuracy;

determining a set of one or more local minima of the classifier'"'"'s output;

for each local minimum in the set of local minima;

determining a set of rendering parameter values associated with the local minimum, the rendering parameter values comprising at least one of an azimuth location of a viewpoint, an elevation location of a viewpoint, and a rotation of a viewpoint around its optical axis; and

generating an additional image based on the three-dimensional model and the determined set of rendering parameter values;

training the classifier using the initial training set and the generated additional images.

2. The method of claim 1, wherein the classifier comprises a statistical classifier.

3. The method of claim 1, wherein the classifier comprises a support vector machine.

4. The method of claim 1, wherein the classifier classifies an image based on a feature vector of the image'"'"'s gradient histograms.

5. The method of claim 1, wherein a rendering parameter additionally comprises one element of a group containing:

an azimuth location of a point light source, an elevation location of a point light source, and an intensity ratio between an ambient light and a point light source.

6. The method of claim 1, wherein a value for a rendering parameter for an image in the initial training set is determined randomly or pseudo-randomly.

7. The method of claim 1, wherein an image in the initial training set is associated with a label that specifies whether the image is a positive example or a negative example.

8. The method of claim 1, wherein determining the set of one or more local minima of the classifier'"'"'s output comprises performing an optimization algorithm.

9. A non-transitory machine-readable storage medium encoded with instructions that, when executed by a processor, cause the processor to perform a method for training a view-based object recognition classifier, comprising:

generating an initial training set of images, wherein an image in the initial training set is generated based on a three-dimensional model and a set of rendering parameter values, the rendering parameter values comprising at least one of an azimuth location of a viewpoint, an elevation location of a viewpoint, and a rotation of a viewpoint around its optical axis;

training the classifier using the initial training set;

determining the classifier'"'"'s accuracy;

determining a set of one or more local minima of the classifier'"'"'s output;

for each local minimum in the set of local minima;

determining a set of rendering parameter values associated with the local minimum, the rendering parameter values comprising at least one of an azimuth location of a viewpoint, an elevation location of a viewpoint, and a rotation of a viewpoint around its optical axis; and

generating an additional image based on the three-dimensional model and the determined set of rendering parameter values;

training the classifier using the initial training set and the generated additional images.

10. The medium of claim 9, wherein the classifier comprises a statistical classifier.

11. The medium of claim 9, wherein the classifier comprises a support vector machine.

12. The medium of claim 9, wherein the classifier classifies an image based on a feature vector of the image'"'"'s gradient histograms.

13. The medium of claim 9, wherein a rendering parameter comprises one element of a group containing:

an azimuth location of a point light source, an elevation location of a point light source, and an intensity ratio between an ambient light and a point light source.

14. The medium of claim 9, wherein a value for a rendering parameter for an image in the initial training set is determined randomly or pseudo-randomly.

15. The medium of claim 9, wherein an image in the initial training set is associated with a label that specifies whether the image is a positive example or a negative example.

16. The medium of claim 9, wherein determining the set of one or more local minima of the classifier'"'"'s output comprises performing an optimization algorithm.

17. A system for training a view-based object recognition classifier, comprising:

a machine-readable storage medium encoded with machine-readable instructions for performing a method, the method comprising;

generating an initial training set of images, wherein an image in the initial training set is generated based on a three-dimensional model and a set of rendering parameter values, the rendering parameter values comprising at least one of an azimuth location of a viewpoint, an elevation location of a viewpoint, and a rotation of a viewpoint around its optical axis;

training the classifier using the initial training set;

determining the classifier'"'"'s accuracy;

determining a set of one or more local minima of the classifier'"'"'s output;

for each local minimum in the set of local minima;

determining a set of rendering parameter values, the rendering parameter values comprising at least one of an azimuth location of a viewpoint, an elevation location of a viewpoint, and a rotation of a viewpoint around its optical axis; and

generating an additional image based on the three-dimensional model and the determined set of rendering parameter values;

training the classifier using the initial training set and the generated additional images; and

a processor configured to execute the machine-readable instructions encoded on the machine-readable storage medium.

18. The system of claim 17, wherein a rendering parameter additionally comprises one element of a group containing:

an azimuth location of a point light source, an elevation location of a point light source, and an intensity ratio between an ambient light and a point light source.

19. The system of claim 17, wherein a value for a rendering parameter for an image in the initial training set is determined randomly or pseudo-randomly.

20. The system of claim 17, wherein an image in the initial training set is associated with a label that specifies whether the image is a positive example or a negative example.