Apparatus for visual recognition

• US 5,111,516 A
• Filed: 04/11/1990
• Issued: 05/05/1992
• Est. Priority Date: 04/11/1989
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toyota Chuo Kenkyusho (Toyota Motor Corporation)
• Original Assignee: Kabushiki Kaisha Toyota Chuo Kenkyusho (Toyota Motor Corporation)
• Inventors: Nakano, Masaru, Sukigara, Kazutoshi
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Stellrecht, Barry
• Application Number:
  US07/507,406
• Time in Patent Office: 755 Days
• Field of Search: 382/14, 382/15, 382/25, 382/27, 382/30, 382/34
• US Class Current: 382/156
• CPC Class Codes: G06V 10/757 Matching configurations of points or features

Abstract

A basic image of objects is extracted from a two-dimensional image of objects. Geometrical elements of the objects are extracted from the extracted basic image. The objects to be recognized are identified by searching a combination of the geometrical elements which match a geometrical model and then utilizing candidate position/orientation of the objects to be recognized, said candidate position/orientation being determined from a relationship in relative position between the combination of geometrical elements and the geometrical model. Mesh cells fixed to the geometrical model are mapped on the basic image based on the candidate position/orientation. In addition, verification is made as to whether an image of the geometrical model mapped by the candidate position/orientation is accurately matched with an image of one of the objects to be recognized, through a neural network to which values got from the basic image included in the individual mesh cells are to be applied as input values. Combination weight factors employed in the neural network are learned according to the verified results. It is also possible to recognize the multi-purpose objects according to how to learn the combination weight factors.

Claims

1. A visual recognition apparatus, comprising:

basic image extraction means for extracting a basic image of objects from each pixel in a two-dimensional gray image of said objects;

line image extraction means for extracting a line image obtained through a spatial differentiation of said extracted basic image;

geometrical feature extraction means for extracting geometrical elements representing geometry of said objects from said extracted line image;

search means for searching said extracted geometrical elements which match a geometrical model composed of model elements representing an object to be recognized, and for getting a candidate position/orientation of said object to be recognized, based on a relative position of said geometrical elements and said model elements; and

verification means for verifying whether or not said candidate position/orientation of object to be recognized obtained by said search means accurately matches said object to be recognized;

the verification means including;

network input-value arithmetic operation means for mapping mesh cells obtained by dividing a region fixed to said geometrical model into some small regions on said basic image or said line image based on said candidate position/orientation, matching positions/orientations of said object to be recognized and the geometrical model to each other and determining a value representing each of said mesh cells in reference to a basic image or a line image in each of said mesh cells; and

a neutral network having input elements, each of said input elements one to one corresponding to each cell in said mesh cells and inputting an output-value of each cell obtained by said network input-value arithmetic operation means, having an output element for outputting a true or false value, and calculating an output-value of said output element from input-values to said input elements.

4. The apparatus as claimed in claim 1, wherein

said geometrical elements are selected from straight lines, corners, circles and combinations thereof;

and

said model elements are selected from straight lines, corners, curves, circles, and combinations thereof representing a main portion of each contour line of the model.

6. The apparatus as claimed in claim 1, wherein;

said network input-value arithmetic operation means decides said output-value of said each cell of said mesh cells based upon whether or not the contour line of said basic image exists in said each cell.

7. The apparatus as claimed in claim 6, wherein said network input-value arithmetic operation means decides said output-value of said each cell of said mesh cells based upon whether or not the contour line of said basic image exists in line pixels, each of said line pixels intersecting in a cross form at a center of each cell of said mesh cells.

2. A visual recognition apparatus, comprising:

basic image extraction means for extracting a basic image of objects from each pixel in a two-dimensional gray image of said objects;

line image extraction means for extracting a line image obtained through a spatial differentiation of said extracting basic image;

geometrical feature extraction means for extracting geometrical elements representing geometry of said objects from said extracted line image;

search means for searching said extracted geometrical elements which match a geometrical model composed of model elements representing an object to be recognized, and for getting a candidate position/orientation of said object to be recognized, based on a relative position of said geometrical elements and said model elements;

verification means for verifying whether or not said candidate position/orientation of object to be recognized obtained by said search means accurately matches said object to be recognized;

the verification means including;

network input-value arithmetic operation means for mapping mesh cells obtained by dividing a region fixed to said geometrical model into some small regions on said basic image or said line image based on said candidate position/orientation, for matching position/orientations of said object to be recognized and the geometrical model to each other and determining a value representing each of said mesh cells in reference to a basic image or a line image in each of said mesh cells; and

a neural network having input elements, each of said input elements, one to one corresponding to each cell in said mesh cells and inputting an output-value of each cell obtained by said network input-value arithmetic operation means, having an output element for outputting a true of false value, and calculating an output-value of said output element from input-values to said input elements; and

learning means for making an output of each cell of said mesh cells as an Input signal, modifying combination weight factors of said neural network based on said input signal, a verified result of said verification means corresponding to said input signal and a teacher signal for providing an evaluation value representing truth of said verified result and outputting modified combination weight factors to said verification means.

3. A visual recognition apparatus, comprising:

basic image extraction means for extracting a basic image of objects from each pixel in a two-dimensional gray image of said objects;

line image extraction means for extracting a line image obtained through a spatial differentiation of said extracted basic image;

geometrical modeling means for producing a geometrical model composed of model elements constituted of lines specifying geometry of a model representing an object to be recognized;

geometrical feature extraction means for extracting geometrical elements representing geometry of said objects from said extracted line image;

search means for searching said extracted geometrical elements which match a geometrical model composed of model elements representing an object to be recognized, and for getting a candidate position/orientation of said object to be recognized, based on a relative position of said geometrical elements and said model elements;

verification means for verifying whether or not said candidate position/orientation of object to be recognized obtained by said search means accurately matches said object to be recognized;

the verification means including;

network input-value arithmetic operation means for mapping mesh cells obtained by dividing a region fixed to said geometrical model into some small regions on said basic image or said line image based on said candidate position/orientation, matching positions/orientations of said object to be recognized and the geometrical model to each other and determining a value representing each of said mesh cells in reference to a basic image or a line image in each of said mesh calls; and

a neural network having input elements, each of said input elements one to one corresponding to each cell in said mesh cells and inputting an output-value of each cell obtained by said network input-value arithmetic operation means, having an output element for outputting a true or false value, and calculating an output-value of said output element from input-values to said input elements; and

learning means for making an output of each cell of said mesh cells as an input signal, modifying combination weight factors of said neural network based on said input signal, a verified result of said verification means corresponding to said input signal and a teacher signal for providing an evaluation value representing truth of said verified result and outputting modified combination weight factors to said verification means,

5. The apparatus as claimed in claim 3, wherein:

said learning means is adapted to perform a modification arithmetic operation for each of the combination weight factors in said neural network in accordance with said input signal given by said each cell of said mesh cells and the difference between a true or false value corresponding to said input signal with respect to, said candidate position/orientation and each of values corresponding to said input signal outputted from said neural network and to execute said modification arithmetic operation of said factors until said difference becomes zero.

8. A visual recognition apparatus, comprising:

basic image extraction means for extracting a basic image of objects from each pixel in a two-dimensional gray image of said objects;

geometrical feature extraction means for extracting geometrical elements representing geometry of said objects from said extracted basic image;

search means for searching said extracted geometrical elements which match a geometrical model composed of model elements representing an object to be recognized, and for getting a candidate position/orientation of said object to be recognized, based on a relative position of said geometrical elements and said model elements; and

verification means for verifying whether or not said candidate position/orientation of object to be recognized obtained by said search means accurately matches said object to be recognized;

the verification means including;

network input-value arithmetic operation means for mapping mesh cells obtained by dividing a region fixed to said geometrical model into some small regions on said basic image based on said candidate position/orientation, matching positions/orientations of said object to be recognized and the geometrical model to each other and determining a value representing each of said mesh cells in reference to a basic image in each of the said mesh cells; and

a neural network having input elements, each of said input elements one to one corresponding to each cell in said mesh cells and inputting an output-value of each cell obtained by said network input-value arithmetic operation means, having an output element for outputting a true or false value, and calculation an output-value of said output element from input-values to said input elements.