Method and Apparatus for Improved Training of Object Detecting System

• US 20140079314A1
• Filed: 09/18/2012
• Published: 03/20/2014
• Est. Priority Date: 09/18/2012
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Yakubovich, Yury, Yang, Yang, Clarke, Ian, Chen, Lihui, Poon, Eunice, Brusnitsyn, Mikhail, Abadpour, Arash, Rico, Dan, Fu, Guoyi, Moravec, Ivo
• Granted Patent: US 9,031,317 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/231 Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram
• CPC Class Codes: G06F 18/285 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06F 18/41 Interactive pattern learning with a human teacher
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/50 Lighting effects
• CPC Class Codes: G06V 10/7625 Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/7788 the supervisor being a human, e.g. interactive learning with a human teacher

Abstract

An adequate solution for computer vision applications is arrived at more efficiently and, with more automation, enables users with limited or no special image processing and pattern recognition knowledge to create reliable vision systems for their applications. Computer rendering of CAD models is used to automate the dataset acquisition process and labeling process. In order to speed up the training data preparation while maintaining the data quality, a number of processed samples are generated from one or a few seed images.

Claims

1. A system for generating a computer vision solution for a specified vision task, the system comprising:

an input configured to receive user-defined limitations, wherein at least one of said user-defined limitations is associated with a sub-set of predefined descriptors within a library of predefined descriptors, each of said descriptors being associated with a respective set of image constraints; and

a processor configured to;

for each user-defined limitation that is associated with said library of predefined descriptors, selecting an available descriptor within its associated sub-set of predefined descriptors that is most consistent with all user-defined limitations;

defining a working image library of sample images based on the user-defined limitations and the sets of image constraints associated with the selected descriptors;

defining an evaluation set of sample images from said working image library;

defining a vision solution candidate based on the user-defined limitations and the selected descriptors;

tuning said vision solution candidate, including modifying the selected descriptors according to the specified vision task;

evaluating the tuned vision solution candidate using said evaluation set of sample images and user-provided limitations, including accuracy and speed limitations;

using evaluation feedback for additional tuning of the selected solution candidate until the accuracy and speed limitations are met or a maximum of additional tuning is reached.

2. The system of claim 1, wherein tuning said vision solution candidate further includes altering the selection of descriptors associated with a user-provided limitation.

3. The system of claim 1, further including defining a training set of sample images from said working image library and excluding any sample images in said evaluation set of sample images, wherein:

at least a portion of the descriptors within said library of descriptors include classifiers, and

the tuning of the vision solution candidate includes training these classifiers for the specified vision task using said training set of sample images.

4. The system of claim 1, further defining a training set of sample images from said working image library and excluding any sample images in said evaluation set of sample images, wherein:

at least one of said user-defined limitations requires the creation of a new classifier;

the new classifier is automatically trained using the training set of sample images, the evaluation set of sample images, and user-defined inputs; and

the defining of said vision solution candidate is further based on the new classifier.

5. The system of claim 4, wherein the tuning of said vision solution candidate further includes creating additional new classifiers.

6. The system of claim 4 wherein the automatic training of the new classifier includes creating a histogram of feature descriptors and defining cluster features, harvesting the nodes of the histogram that converge during training, and collecting the harvested nodes into a detector, said detector defining said new classifier.

7. The system of claim 1 wherein:

IF the maximum of additional training is reached without meeting the accuracy and speed requirements,

THEN defining a new vision solution candidate by changing the selected descriptor of at least one user-defined limitation with another descriptor from its associated sub-set of predefine descriptors OR defining a new associated descriptor by a combination of existing descriptors selected from its associated sub-set of predefined descriptors.

8. The system of claim 1 wherein said user-defined limitations comprises one or more of vision tasks, type of object and its properties, and operation conditions of a system employing the computer vision solution.

9. The system of claim 1 wherein the working image library is based on one or more of explicit user input, an existing geometrical model, a set of images captured in a target environment, and computer generated object images.

10. The system of claim 1 wherein defining said working image library includes:

obtaining information on;

object characteristics and parameters corresponding to each object characteristic;

scene characteristics and parameters corresponding to each scene characteristic;

camera characteristics;

parameter domains for the object characteristics, scene characteristics, and camera characteristics; and

a target sampling schema by selecting a sampling type and a distribution type;

wherein at least a portion of the above information is obtained from the sets of image constraints associated with the selected descriptors;

generating a plurality of sample configurations according to the target sampling schema using the parameter domains;

for each sample configuration;

performing 3D rendering of the scene using computer graphics, and

generating a sample 2D image based on the rendering; and

creating a data set including at least some of the following for each sample configuration;

image information from the generated sample 2D image and optional 3D information when available;

parameters of the sample configuration, and

target object data.

11. The system of claim 10 wherein the characteristics of the object include one or more of an object model, a reflectance model based on object material, surface texture and uniformity characteristics, and depth edge information.

12. The system of claim 10 wherein the characteristics of the scene include one or more of images of preferred background, viewing directions, positions and orientations of an object, camera and light sources in the scene, and a model of a lighting environment.

13. The system of claim 10 wherein the characteristics of the camera include one or more of a model of the camera, and intrinsic and extrinsic calibration parameters of the camera.

14. The system of claim 10 wherein the sampling type includes one or more of linear and random, and the distribution type includes one or more of linear and Gaussian.

15. The system of claim 10 wherein the computer graphics include one or more of Open GL and ray tracing.

16. The system of claim 10 wherein the parameters of the sample configuration include one or more of location of objects in the scene, orientations of objects in the scene, locations of light sources, and camera distance.

17. The system of claim 10 wherein the target object data includes one or more of a bounding area outlining the target object in the 2D image, an object 2D mask, and area of the target object occluded by other objects.

18. The system of claim 1 wherein the defining of said working image library of sample images comprises:

acquiring a seed image of an object;

rotating the seed image to generate a first rotated image;

labeling the object in the first rotated image;

rotating the first rotated image through a range of rotation angles to generate a plurality of additionally rotated images; and

labeling the object in the additionally rotated images based on the label of the object in the first rotated image.

19. The system of claim 18 further comprising at least one of blurring, brightening, darkening, and adding noise to the first rotated image in accordance with the sets of image constraints associated with the selected descriptors.

20. One or more tangible, non-transitory computer-readable media embodying instructions executable by a compute to perform a method for automatically generating a computer vision solution, the method comprising:

receiving user input for one or more descriptors;

creating a training set of object images and an evaluation set of object images;

selecting a vision solution candidate that provides a best match to the user input from a predefined solutions library;

training the selected solution candidate using the training set;

applying the selected solution candidate to the evaluation set;

evaluating the selected solution candidate solution using the user input; and

using evaluation feedback for additional training of the selected solution candidate until accuracy and speed requirements are met or a maximum of additional training is reached.