OBJECT RECOGNITION SYSTEM USING LEFT AND RIGHT IMAGES AND METHOD

• US 20110134221A1
• Filed: 12/07/2010
• Published: 06/09/2011
• Est. Priority Date: 12/07/2009
• Status: Active Grant

• Current Assignee: Gwangju Institute of Science and Technology, Samsung Electronics Co. Ltd.
• Original Assignee: Gwangju Institute of Science and Technology, Samsung Electronics Co. Ltd.
• Inventors: Han, Woo Sup, Yoon, Kuk Jin, LEE, Ji Hyo
• Granted Patent: US 9,984,280 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Disclosed herein are an object recognition system and method which extract left and right feature vectors from a stereo image of an object, find feature vectors which are present in both the extracted left and right feature vectors, compare information about the left and right feature vectors and the feature vectors present in both the extracted left and right feature vectors with information stored in a database, extract information of the object, and recognize the object.

Claims

1. An object recognition system, comprising:

a stereo camera to output left and right images of a same object; and

an object recognition device to extract left and right feature vectors of the left and right images received from the stereo camera, to compare the extracted left and right feature vectors with left and right feature vectors of left and right images of objects stored in a database, and to recognize the object present in the left and right images based on the comparison.

2. The object recognition system according to claim 1, wherein the object recognition device matches the extracted left and right feature vectors with the left and right feature vectors of the left and right images of the objects stored in the database, applies scores to the matched feature vectors of the objects according to a Euclidean distance, sums applied scores according to the left and right images, and recognizes an object having a high summed score as the object present in the left and right images.

8. The object recognition system according to claim 1, wherein the object recognition device matches the plurality of left and right feature vectors extracted from the left and right images, a plurality of common feature vectors which are present in both the left and right images, and a three-dimensional distance between any two common vectors out of the plurality of common feature vectors with values of the left and right images of the objects which are stored in the database in advance, applies scores to the matched results, sums the applied scores, and recognizes an object having a high summed score as an object present in the left and right images.

9. The object recognition system according to claim 1, wherein the object recognition device verifies the recognized object using at least one of two-dimensional homography and three-dimensional Euclidean Transform with respect to the matched feature vectors between the feature vectors extracted from the left and right images and the feature vectors stored in the database after object recognition.

3. The object recognition system according to claim, wherein:

common feature vectors which are present in both the left and right images are stored in the database according to the objects, and

the object recognition device matches the left and right feature vectors extracted from the left and right images to find common feature vectors which are present in both the left and right images and compares the found common feature vectors with the common feature vectors stored in the database to recognize the object present in the left and right images.

4. The object recognition system according to claim 3, wherein the object recognition device matches the found common feature vectors which are present in both the left and right images with the common feature vectors stored in the database, applies the scores to the matched feature vectors of the common feature vectors of the objects according to the Euclidean distance, sums the applied scores, and recognizes an object having a high summed score as the object present in the left and right images.

5. The object recognition system according to claim 3, wherein the object recognition device includes:

a feature extraction unit to extract a plurality of feature vectors from a plurality of feature points of the left image and to extract a plurality of feature vectors from a plurality of feature points of the right image;

a feature vector matching unit to match the plurality of feature vectors extracted from the left image and from the right image by the feature extraction unit to extract a plurality of common feature vectors which are present in both the left and right images;

a database to store a plurality of common feature vectors which are present in both the left and right images according to the objects; and

an object recognition unit to match the plurality of common feature vectors extracted by the feature vector matching unit with the plurality of common feature vectors stored in the database to recognize the object present in the left and right images.

6. The object recognition system according to claim 5, wherein:

the database stores a three-dimensional distance between any two common vectors out of the common vectors which are present in both the left and right images according to the objects, and

the object recognition unit matches the plurality of common feature vectors extracted by the feature vector matching unit with the plurality of common feature vectors stored in the database, compares the matched common feature vectors in terms of the three-dimensional distance between any two common feature vectors, applies scores according to a difference in the three-dimensional distance, and recognizes the object according to the applied scores.

7. The object recognition system according to claim 6, wherein the feature extraction unit detects vectors from a plurality of regions related to the plurality of feature points and extracts a group of detected vectors as a feature vector.

10. An object recognition method, comprising:

receiving different left and right images obtained by photographing a same object using a photographing device;

extracting left and right feature vectors from the received left and right images;

matching the extracted left and right feature vectors with left and right feature vectors of left and right images of objects stored in a database;

applying scores to the matched feature vectors of the objects according to a Euclidean distance; and

summing the applied scores according to the left and right images and recognizing an object having a high summed score as an object present in the left and right images.

11. An object recognition method, comprising:

receiving different left and right images obtained by photographing a same object using a photographing device;

extracting left and right feature vectors from the received left and right images;

matching the extracted left and right feature vectors and extracting a plurality of common feature vectors which are present in both the left and right images;

matching the plurality of extracted common feature vectors with a plurality of common feature vectors which are present in both left and right images of objects stored in a database; and

recognizing an object present in the left and right images according to the matched results.

12. The object recognition method according to claim 11, wherein the recognizing of the object includes applying scores to the matched common feature vectors of the objects according to a Euclidean distance, summing the applied scores, and recognizing an object having a high summed score as an object present in the left and right images.

13. The object recognition method according to claim 11, wherein the recognizing of the object includes comparing the plurality of matched common feature vectors in terms of three-dimensional distance between any two common feature vectors, applying scores according to a difference in the three-dimensional distance, and recognizing the object according to the applied scores.

14. The object recognition method according to claim 11, further comprising verifying the recognized object using at least one of two-dimensional homography and three-dimensional Euclidean Transform with respect to the plurality of matched feature vectors after recognizing the object.

15. An object recognition method, comprising:

receiving different left and right images obtained by photographing a same object using a photographing device;

extracting left and right feature vectors from the received left and right images;

matching the extracted left and right feature vectors and extracting a plurality of common feature vectors which are present in both the left and right images and three-dimensional distances thereof;

matching the left and right feature vectors, the plurality of common feature vectors, and the three-dimensional distances with values of objects stored in a database; and

recognizing an object present in the left and right images according to the matched results.

16. The object recognition method according to claim 15, wherein:

the recognizing of the object includes finding object candidates using the following equation and recognizing an object having a highest matching score out of the object candidates as the object present in the left and right images,

a matching matrix M(s,m) between any object model Gm of a plurality of object models stored in the database and an object model Gs of the received left and right images is expressed by;

17. The object recognition method according to claim 16, wherein the matching matrices M(s,m) are determined by a matching value Mai to minimize the following function Ω