System for three-dimensional object recognition and foreground extraction

• US 8,774,504 B1
• Filed: 10/26/2011
• Issued: 07/08/2014
• Est. Priority Date: 10/26/2011
• Status: Expired due to Fees

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Sundareswara, Rashmi N., Srinivasa, Narayan
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US13/282,389
• Time in Patent Office: 986 Days
• Field of Search: None
• US Class Current: 382/165
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

The present invention describes a system for recognizing objects from color images by detecting features of interest, classifying them according to previous objects'"'"' features that the system has been trained on, and finally drawing a boundary around them to separate each object from others in the image. Furthermore, local feature detection algorithms are applied to color images, outliers are removed, and resulting feature descriptors are clustered to achieve effective object recognition. Additionally, the present invention describes a system for extracting foreground objects and the correct rejection of the background from an image of a scene. Importantly, the present invention allows for changes to the camera viewpoint or lighting between training and test time. The system uses a supervised-learning algorithm and produces blobs of foreground objects that a recognition algorithm can then use for object detection/recognition.

Claims

1. A system for object recognition, the system comprising:

one or more processors and a memory having instructions such that when the instructions are executed, the one or more processors perform operations of;

receiving a color image;

separating the color image into monochrome color channels;

detecting a set of features in each of the monochrome color channels;

combining the set of features for each of the monochrome color channels to yield a combined feature descriptor describing a set of features in the color image;

wherein each detected feature is assigned a class corresponding to its closest matching object in a learned feature database, wherein multiple features are assigned a same class, wherein the set of features contains one or more outlier features which are assigned to an incorrect class, and wherein the set of features for each monochrome color channel is described by a feature descriptor for that channel;

removing the outlier features from the set of features in the color image;

clustering features of the same class into clusters within the set of features; and

drawing a boundary around each cluster to create a bound cluster representing a recognized object, whereby objects are recognized in the color image.

2. The system for object recognition as set forth in claim 1, wherein the monochrome color channels comprise a red channel, a green channel, and a blue channel.

3. The system for object recognition as set forth in claim 2, wherein the system is further configured to perform an operation of detecting the set of features using a local feature detection algorithm, wherein the local feature algorithm is selected from a group consisting of Scale Invariant Feature Transform (SIFT) and Speeded-Up Robust Features (SURF).

4. The system for object recognition as set forth in claim 3, wherein the system is further configured to perform an operation of combining the feature descriptors for each of the monochrome color channels according to the following:

5. The system for object recognition as set forth in claim 4, wherein when removing outliers from the set of features in the color image, the system is further configured to perform operations of:

for each feature;

creating two lists of an operator-specified number of the feature'"'"'s nearest neighbors, where a first of the two lists contains nearest neighbors of the same class, and a second of the two lists contains nearest neighbors of a different class; and

removing features if more than an operator-specified number of its closest neighbors from the first list are also members of the second list; and

repeating the operations above for each removed feature with respect to features that were not removed.

6. The system for object recognition as set forth in claim 5, wherein the system is further configured to perform an operation of clustering groups of features of the same class using k-shared nearest neighbor.

7. The system for object recognition as set forth in claim 6, wherein the system is further configured to perform an operation of drawing a boundary around each cluster using the convex hull of the cluster.

8. A computer-implemented method for object recognition, comprising an act of:

causing a data processor to execute instructions stored on a memory such that upon execution, the data processor performs operations of;

receiving a color image;

separating the color image into monochrome color channels;

detecting a set of features in each of the monochrome color channels;

combining the set of features for each of the monochrome color channels to yield a combined feature descriptor describing a set of features in the color image;

wherein each detected feature is assigned a class corresponding to its closest matching object in a learned feature database, wherein multiple features are assigned a same class, wherein the set of features contains one or more outlier features which are assigned to an incorrect class, and wherein the set of features for each monochrome color channel is described by a feature descriptor for that channel;

removing the outlier features from the set of features in the color image;

clustering features of the same class into clusters within the set of features; and

drawing a boundary around each cluster to create a bound cluster representing a recognized object, whereby objects are recognized in the color image.

9. The method for object recognition as set forth in claim 8, wherein the monochrome color channels comprise a red channel, a green channel, and a blue channel.

10. The method for object recognition as set forth in claim 9, further comprising an act of detecting the set of features using a local feature detection algorithm, wherein the local feature algorithm is selected from a group consisting of Scale Invariant Feature Transform (SIFT) and Speeded-Up Robust Features (SURF).

11. The method for object recognition as set forth in claim 10, further comprising an act of combining the feature descriptors for each of the monochrome color channels according to the following:

12. The method for object recognition as set forth in claim 11, wherein when removing outliers from the set of features in the color image, the method further comprising acts of:

for each feature;

creating two lists of an operator-specified number of the feature'"'"'s nearest neighbors, where a first of the two lists contains nearest neighbors of the same class, and a second of the two lists contains nearest neighbors of a different class; and

removing features if more than an operator-specified number of its closest neighbors from the first list are also members of the second list; and

repeating the operations above for each removed feature with respect to features that were not removed.

13. The method for object recognition as set forth in claim 12, further comprising an act of clustering groups of features of the same class using k-shared nearest neighbor.

14. The method for object recognition as set forth in claim 13, further comprising an act of drawing a boundary around each cluster using the convex hull of the cluster.

15. A computer program product for object recognition, the computer program product comprising computer-readable instruction means stored on a non-transitory computer-readable medium that are executable by a computer having a processor for causing the processor to perform operations of:

receiving a color image;

separating the color image into monochrome color channels;

detecting a set of features in each of the monochrome color channels;

combining the set of features for each of the monochrome color channels to yield a combined feature descriptor describing a set of features in the color image;

wherein each detected feature is assigned a class corresponding to its closest matching object in a learned feature database, wherein multiple features are assigned a same class, wherein the set of features contains one or more outlier features which are assigned to an incorrect class, and wherein the set of features for each monochrome color channel is described by a feature descriptor for that channel;

removing the outlier features from the set of features in the color image;

clustering features of the same class into clusters within the set of features; and

drawing a boundary around each cluster to create a bound cluster representing a recognized object, whereby objects are recognized in the color image.

16. The computer program product for object recognition as set forth in claim 15, wherein the monochrome color channels comprise a red channel, a green channel, and a blue channel.

17. The computer program product for object recognition as set forth in claim 16, further comprising instruction means for causing the processor to perform an operation of detecting the set of features using a local feature detection algorithm, wherein the local feature algorithm is selected from a group consisting of Scale Invariant Feature Transform (SIFT) and Speeded-Up Robust Features (SURF).

18. The computer program product for object recognition as set forth in claim 17, further comprising instruction means for causing the processor to perform an operation of combining the feature descriptors for each of the monochrome color channels according to the following:

19. The computer program product for object recognition as set forth in claim 18, wherein when removing outliers from the set of features in the color image, the computer program product further comprises instruction means for causing the processor to perform operations of:

for each feature;

creating two lists of an operator-specified number of the feature'"'"'s nearest neighbors, where a first of the two lists contains nearest neighbors of the same class, and a second of the two lists contains nearest neighbors of a different class; and

removing features if more than an operator-specified number of its closest neighbors from the first list are also members of the second list; and

repeating the operations above for each removed feature with respect to features that were not removed.

20. The computer program product for object recognition as set forth in claim 19, further comprising instruction means for causing the processor to perform an operation of clustering groups of features of the same class using k-shared nearest neighbor.

21. The computer program product for object recognition as set forth in claim 20, further comprising instruction means for causing the processor to perform an operation of drawing a boundary around each cluster using the convex hull of the cluster.