Object recognition system using left and right images and method

• US 9,984,280 B2
• Filed: 12/07/2010
• Issued: 05/29/2018
• Est. Priority Date: 12/07/2009
• Status: Active Grant

• Current Assignee: Gwangju Institute of Science and Technology, Samsung Electronics Co. Ltd.
• Original Assignee: Gwangju Institute of Science and Technology, Samsung Electronics Co. Ltd.
• Inventors: Lee, Ji Hyo, Yoon, Kuk Jin, Han, Woo Sup
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US12/961,840
• Publication Number:
  US 20110134221A1
• Time in Patent Office: 2,730 Days
• Field of Search: 348 46
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Disclosed herein are an object recognition system and method which extract left and right feature vectors from a stereo image of an object, find feature vectors which are present in both the extracted left and right feature vectors, compare information about the left and right feature vectors and the feature vectors present in both the extracted left and right feature vectors with information stored in a database, extract information of the object, and recognize the object.

Claims

1. An object recognition system, comprising:

a stereo camera configured to output left and right images of a same object; and

an object recognition device configured to extract left and right feature vectors from the left and right images received from the stereo camera, match the left feature vectors extracted from the left images with the right feature vectors extracted from the right images to find a plurality of common feature vectors which are present in both the left and right images, match the plurality of found common feature vectors with a plurality of common feature vectors which are present in both left and right images according to objects stored in a database, and recognize an object present in the left and right images based on results of the matching,

wherein the object recognition device is configured to match the left feature vectors and the right feature vectors such that the left feature vectors in a scale space is compared with the right feature vector in a same scale space, and

wherein the object recognition device is configured to recognize the object by finding object candidates using the following equation and recognizing an object having a highest matching score out of the object candidates as the object present in the left and right images,

a matching matrix M(s,m) between any object model Gm of a plurality of object models stored in the database and an object model Gs of the received left and right images is expressed by;

2. The object recognition system according to claim 1, wherein the object recognition device is configured to match the found common feature vectors which are present in both the left and right images with the common feature vectors stored in the database, apply scores to the matched feature vectors of the common feature vectors of the objects according to a Euclidean distance, sum the applied scores, and recognize an object having a high summed score as the object present in the left and right images.

3. The object recognition system according to claim 1, wherein the object recognition device includes:

a feature extraction unit configured to extract a plurality of feature vectors from a plurality of feature points of the left images and extract a plurality of feature vectors from a plurality of feature points of the right images;

a feature vector matching unit configured to match the plurality of feature vectors extracted from the left images and from the right images by the feature extraction unit to extract a plurality of common feature vectors which are present in both the left and right images;

a database configured to store a plurality of common feature vectors which are present in both the left and right images according to the objects; and

an object recognition unit configured to match the plurality of common feature vectors extracted by the feature vector matching unit with the plurality of common feature vectors stored in the database to recognize the object present in the left and right images.

4. The object recognition system according to claim 3, wherein:

the database is configured to store a three-dimensional distance between any two common vectors out of the common vectors which are present in both the left and right images according to the objects, and

the object recognition unit is configured to match the plurality of common feature vectors extracted by the feature vector matching unit with the plurality of common feature vectors stored in the database, compare the matched common feature vectors in terms of the three-dimensional distance between any two common feature vectors, apply scores according to a difference in the three-dimensional distance, and recognize the object according to the applied scores.

5. The object recognition system according to claim 4, wherein the feature extraction unit is configured to detect vectors from a plurality of regions related to the plurality of feature points and extract a group of detected vectors as feature vectors.

6. The object recognition system according to claim 1, wherein the object recognition device is configured to match the left and right feature vectors extracted from the left and right images, the plurality of found common feature vectors, and a three-dimensional distance between any two common vectors out of the plurality of common feature vectors with corresponding values of the left and right images according to the objects which are stored in the database in advance, apply scores to results of the matching, sum the applied scores, and recognize an object having a high summed score as an object present in the left and right images.

7. The object recognition system according to claim 1, wherein the object recognition device is configured to verify the recognized object using at least one of two-dimensional homography and three-dimensional Euclidean Transform with respect to the matched feature vectors between the feature vectors extracted from the left and right images and the feature vectors stored in the database after object recognition.

8. An object recognition method, comprising:

receiving different left and right images obtained by photographing a same object using a photographing device;

extracting left and right feature vectors from the received left and right images;

comparing and matching the left feature vectors in a scale space, the left feature vectors extracted from the left images, with the right feature vectors in the scale space, the right feature vectors extracted from the right images, and extracting a plurality of common feature vectors which are present in both the left and right images;

matching the plurality of extracted common feature vectors with a plurality of common feature vectors which are present in both left and right images stored in a database according to objects; and

recognizing an object present in the left and right images based on results of the matching,

wherein the recognizing includes finding object candidates using the following equation and recognizing an object having a highest matching score out of the object candidates as the object present in the left and right images,

a matching matrix M(s,m) between any object model Gm of a plurality of object models stored in the database and an object model Gs of the received left and right images is expressed by;

9. The object recognition method according to claim 8, wherein the recognizing of the object includes applying scores to the matched common feature vectors of the objects according to a Euclidean distance, summing the applied scores, and recognizing an object having a high summed score as an object present in the left and right images.

10. The object recognition method according to claim 8, wherein the recognizing of the object includes comparing the matched common feature vectors in terms of three-dimensional distance between any two common feature vectors, applying scores according to a difference in the three-dimensional distance, and recognizing the object according to the applied scores.

11. The object recognition method according to claim 8, further comprising:

verifying the recognized object using at least one of two-dimensional homography and three-dimensional Euclidean Transform with respect to the plurality of matched feature vectors after recognizing the object.

12. An object recognition method, comprising:

receiving different left and right images obtained by photographing a same object using a photographing device;

extracting left and right feature vectors from the received left and right images;

comparing and matching the left feature vectors in a scale space, the left feature vectors extracted from the left images, with the right feature vectors in the scale space, the right feature vectors extracted from the right images, and extracting a plurality of common feature vectors which are present in both the left and right images and three-dimensional distances thereof;

comparing the left and right feature vectors, the plurality of common feature vectors, and the three-dimensional distances with corresponding values of the left and right images according to objects stored in a database; and

recognizing an object present in the left and right images based on results of the comparing,

wherein the recognizing of the object includes finding object candidates using the following equation and recognizing an object having a highest matching score out of the object candidates as the object present in the left and right images,

a matching matrix M(s,m) between any object model Gm of a plurality of object models stored in the database and an object model Gs of the received left and right images is expressed by;

13. The object recognition method according to claim 12, wherein the matching matrix M(s,m) is determined by a matching value Mai to minimize the following function Ω