OBJECT RECOGNITION APPARATUS, OBJECT RECOGNITION METHOD, LEARNING APPARATUS, LEARNING METHOD, STORAGE MEDIUM AND INFORMATION PROCESSING SYSTEM

• US 20120155751A1
• Filed: 11/17/2011
• Published: 06/21/2012
• Est. Priority Date: 12/16/2010
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Aoba, Masato
• Granted Patent: US 8,965,114 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces

Abstract

A learning method of detectors used to detect a target object, comprises: a selection step of selecting a plurality of specific regions from a given three-dimensional model of the target object; a learning step of learning detectors used to detect the specific regions selected in the selection step; an evaluation step of executing recognition processing of positions and orientations of predetermined regions of the plurality of specific regions by the detectors learned in the learning step; and a normalization step of setting vote weights for outputs of the detectors according to recognition accuracies of results of the recognition processing in the evaluation step.

Claims

1. A learning method of detectors used to detect a target object, comprising:

a selection step of selecting a plurality of specific regions from a given three-dimensional model of the target object;

a learning step of learning detectors used to detect the specific regions selected in the selection step;

an evaluation step of executing recognition processing of positions and orientations of predetermined regions of the plurality of specific regions by the detectors learned in the learning step; and

a normalization step of setting vote weights for outputs of the detectors according to recognition accuracies of results of the recognition processing in the evaluation step.

2. The method according to claim 1, further comprising:

a convergence judgment step of controlling to repetitively execute processes in the evaluation step and the normalization step when differences between already set vote weights and vote weights newly set in the normalization step exceed a predetermined threshold, and judging that the vote weights set in the normalization step are converged to settle the voting weights when the differences fall within the predetermined threshold range.

3. The method according to claim 1, wherein the normalization step further comprises:

a calculation step of calculating, from the results of the recognition processing in the evaluation step, an accumulated value of values of vote weights of region sets normally detected at detection accuracies with a predetermined error range and an accumulated value of values of vote weights of region sets which are erroneously detected beyond the error range; and

an update step of updating the vote weights so that the accumulated value of the values of the vote weights of the normally detected region sets is not less than a threshold, and the accumulated value of the values of the vote weights of the erroneously detected region sets is less than the threshold.

4. The method according to claim 3, wherein the normalization step further comprises:

a region deletion step of deleting a region set when a maximum value of the vote weights updated in the update step is less than a predetermined threshold in association with that region set; and

a region addition step of adding a new specific region when a position and orientation of the region set are not obtained based on the results of the recognition processing in the evaluation step.

5. The method according to claim 1, further comprising:

a specific region selection step of selecting specific regions, which are correctly detected at detection accuracies within a predetermined error range and are used to reconstruct a region set, based on the results of the recognition processing in the evaluation step,

wherein the specific region selection step comprises;

a concurrent detection rate calculation step of calculating a concurrent detection rate at which a plurality of region sets are concurrently detected as a position and an orientation within a predetermined error range at an identical image capturing position of recognition results, which are calculated in the evaluation step and are obtained when a plurality of region sets are captured from a plurality of image capturing positions; and

a region set reconstruction step of selecting specific regions, which respectively construct a plurality of region sets having the concurrent detection rate, which is calculated in the concurrent detection rate calculation step and is larger than a predetermined threshold, as specific regions which construct one new region set, generating the new region set from the selected specific regions, and adding the new region set to the region sets.

6. An object recognition method for executing recognition processing of a target object using learned detectors, comprising:

an acquisition step of acquiring an image of the target object captured by an image capturing unit;

a detection step of detecting, using detectors learned in a learning step of a learning method of claim 1, a plurality of specific regions as images of partial regions included in the image from the image acquired in the acquisition step;

a generation step of generating region sets as combinations of specific regions, which fall within a predetermined distance range, of the plurality of specific regions detected in the detection step;

a region set estimation step of calculating pieces of detected position information with reference to a coordinate system of the image capturing unit and pieces of coordinate position information on a three-dimensional model of the target object in association with the region sets generated in the generation step, and setting a position, which minimizes a difference between the detected position information and the coordinate position information, and an orientation corresponding to the position, as an estimated position and an estimated orientation of each region set;

a voting step of executing calculation processing for multiplying vote values of positions and orientations output from the detectors by vote weights, which are settled in a convergence judgment step of a learning method of claim 2 in correspondence with the estimated positions and estimated orientations of the region sets estimated in the region set estimation step; and

a target object estimation step of calculating values obtained by multiplying vote values of the positions and the orientations output from the detectors by the vote weights for all the region sets in the calculation processing in the voting step, calculating a position and an orientation corresponding to a maximal accumulated vote value, which is calculated by adding the calculation results, and estimating, when the accumulated vote value at the position and the orientation corresponding to the maximal accumulated vote value is not less than a predetermined threshold, the position and the orientation corresponding to the maximal accumulated vote value as a position and an orientation of the target object.

10. A non-transitory computer readable storage medium storing a program for controlling a computer to execute a learning method of claim 1.

11. A non-transitory computer readable storage medium storing a program for controlling a computer to execute an object recognition method of claim 6.

7. A learning apparatus comprising:

a selection unit adapted to select a plurality of specific regions from a given three-dimensional model of a target object;

a learning unit adapted to learn detectors used to detect the specific regions selected by said selection unit;

an evaluation unit adapted to execute recognition processing of positions and orientations of predetermined regions of the plurality of specific regions by the detectors learned by said learning unit; and

a normalization unit adapted to set vote weights for outputs of the detectors according to recognition accuracies of results of the recognition processing in said evaluation unit.

8. The apparatus according to claim 7, further comprising:

a convergence judgment unit adapted to control to repetitively execute processes of said evaluation unit and said normalization unit when differences between already set vote weights and vote weights newly set by said normalization unit exceed a predetermined threshold, and judging that the vote weights set by said normalization unit are converged to settle the voting weights when the differences fall within the predetermined threshold range.

9. An object recognition apparatus for executing recognition processing of a target object using learned detectors, comprising:

an acquisition unit adapted to acquire an image of the target object captured by an image capturing unit;

a detection unit adapted to detect, using detectors learned by a learning unit in a learning apparatus of claim 7, a plurality of specific regions as images of partial regions included in the image from the image acquired by said acquisition unit;

a generation unit adapted to generate region sets as combinations of specific regions, which fall within a predetermined distance range, of the plurality of specific regions detected by said detection unit;

a region set estimation unit adapted to calculate pieces of detected position information with reference to a coordinate system of the image capturing unit and pieces of coordinate position information on a three-dimensional model of the target object in association with the region sets generated by said generation unit, and setting a position, which minimizes a difference between the detected position information and the coordinate position information, and an orientation corresponding to the position, as an estimated position and an estimated orientation of each region set;

a voting unit adapted to execute calculation processing for multiplying vote values of positions and orientations output from the detectors by vote weights, which are settled by a convergence judgment unit in a learning apparatus of claim 8 in correspondence with the estimated positions and the estimated orientations of the region sets estimated by said region set estimation unit; and

a target object estimation unit adapted to calculate values obtained by multiplying vote values of the positions and the orientations output from the detectors by the vote weights for all the region sets in the calculation processing of said voting unit, calculating a position and an orientation corresponding to a maximal accumulated vote value, which is calculated by adding the calculation results, and to estimate, when the accumulated vote value at the position and the orientation corresponding to the maximal accumulated vote value is not less than a predetermined threshold, the position and the orientation corresponding to the maximal accumulated vote value as a position and an orientation of the target object.

12. An information processing system comprising:

a learning apparatus of claim 7, which learns detectors; and

an object recognition apparatus of claim 9, which executes recognition processing of a target object using the detectors learned by said learning apparatus.