Computer vision methods and systems to recognize and locate an object or objects in one or more images

• US 8,849,050 B2
• Filed: 11/08/2012
• Issued: 09/30/2014
• Est. Priority Date: 11/08/2012
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Wang, Jinjun, Xiao, Jing, Ding, Yuanyuan
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US13/671,782
• Publication Number:
  US 20140126810A1
• Time in Patent Office: 691 Days
• Field of Search: None
• US Class Current: 382/238
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching

Abstract

Embodiments of the present invention include systems and methods for identifying and locating an object in an image. In embodiments, an object in an image may be identified by segmenting a first image of an object into one or more superpixels; extracting local descriptors from the first image, each of the descriptors having an interest point with a location; correlating the local descriptors to the superpixels based on locations of the local descriptors and superpixels; determining a probability for an object label for each of a set of the superpixels; and assigning an object label to each of the set of the superpixels based on the probability and a smoothness factor that includes weighting in terms of one or more of spatial, colors, angular distances between superpixels. The superpixels of an image may be concatenated to predict an object label for the image and to determine the location of the image.

Claims

1. A computer-implemented method for identifying an object in an image, comprising:

segmenting a first image of an object into one or more superpixels;

extracting local descriptors from the first image, each of the descriptors having an interest point with a location;

correlating the local descriptors to the superpixels based on locations of the local descriptors and superpixels;

determining a probability for an object label for each of a set of the superpixels using the correlated local descriptors; and

assigning an object label to each of the set of the superpixels based on the probability and a smoothness factor, the smoothness factor being weighted by an angular distance between superpixels.

2. A computer-implemented method as recited in claim 1, wherein the step of determining a probability for an object label for each of a set of the superpixels using the correlated local descriptors comprises:

performing a first tree-based voting using the correlated local descriptors to determine probabilities for object label groups for each of a set of the superpixels, each of the object label groups including one or more object labels;

predicting, based on the probabilities, an object label group for each of the set of the superpixels; and

if the predicted object label group for a particular superpixel includes multiple object labels, performing a second tree-based voting to determine probabilities for the multiple object labels for the particular superpixel.

3. A computer-implemented method as recited in claim 2, further comprising, prior to the step of assigning an object label to each of the set of the superpixels:

multiplying the probability for the predicted object label group to the probabilities for the multiple object labels.

4. A computer-implemented method as recited in claim 1, wherein the step of assigning an object label to each of the set of the superpixels based on the probability and a smoothness factor comprises:

segmenting a second image of the object into one or more superpixels;

determining a spatial overlap between a first superpixel in the first image and a second superpixel pixel in the second image; and

weighting the smoothness factor by the spatial overlap.

5. A computer-implemented method as recited in claim 4, wherein the first and second images are created by varying an image scale or view of the object.

6. A computer-implemented method as recited in claim 1, further comprising:

weighting the smoothness factor by spatial and color distances between superpixels in the first image.

7. A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform at least the steps of claim 1.

8. A computer-implemented method for identifying an object in an image, comprising:

segmenting a first image of an object into one or more superpixels;

extracting local descriptors from the first image, each of the descriptors having an interest point with a location;

correlating the local descriptors to the superpixels based on locations of the local descriptors and superpixels;

performing a first tree-based voting to determine probabilities for object label groups for each of a set of the superpixels using the correlated local descriptors, each of the object label groups including one or more object labels;

predicting, based on the probabilities, an object label group for each of the set of the superpixels; and

if the predicted object label group for a particular superpixel includes multiple object labels, performing a second tree-based voting to determine probabilities for the multiple object labels for the particular superpixel.

9. A computer-implemented method as recited in claim 8, further comprising:

multiplying the probability for the predicted object label group to the probabilities for the multiple object labels.

10. A computer-implemented method as recited in claim 8, further comprising:

assigning an object label to each of the set of the superpixels based on the probabilities for the object label groups and a smoothness factor, the smoothness factor comprising a weighting representing an angular distance between superpixels.

11. A computer-implemented method as recited in claim 10, wherein the step of assigning an object label to each of the set of the superpixels comprises:

preparing a second image of the object;

determining a spatial overlap between a first superpixel in the first image and a second superpixel in the second image; and

weighting the smoothness factor by the spatial overlap.

12. A computer-implemented method as recited in claim 11, wherein the first and second images are created by varying an image scale or view of the object.

13. A computer-implemented method as recited in claim 10, further comprising:

weighting the smoothness factor by spatial and color distances between superpixels in the first image.

14. A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform at least the steps of claim 8.

15. A system for identifying an object in an image, comprising:

a local descriptor extractor for extracting local descriptors from the image, each of the descriptors having an interest point with a location;

an image segmenter coupled to the local descriptor extractor and operative to segment the image into one or more superpixels;

a superpixel group generator coupled to the local descriptor extractor and image segmenter and operative to correlate the local descriptors to the superpixels based on locations of the local descriptors and superpixels;

a tree-based voter coupled to the superpixel group generator and operative to determine a probability for an object label for each of a set of the superpixels; and

a graph-based global decision maker coupled to the tree-based voter and operative to assign an object label to each of the set of the superpixels based on the probability and a smoothness factor and weight the smoothness factor by at least an angular distance between superpixels.

16. A system as recited in claim 15, wherein the tree-based voter is further operative to:

perform a first tree-based voting to determine probabilities for object label groups for each of a set of the superpixels, each of the object label groups including one or more object labels;

predict, based on the probabilities, an object label group for each of the set of the superpixels; and

responsive to an object label group predicted for a particular superpixel including multiple object labels, perform a second tree-based voting to determine probabilities for the multiple object labels for the particular superpixel.

17. A system as recited in claim 16, wherein the tree-based voter is further operative to:

multiply the probability for the predicted object label group to the probabilities for the multiple object labels.

18. A system as recited in claim 15, wherein the graph-based global decision maker is further operative to:

segment a second image of the object into one or more superpixels;

determine a spatial overlap between a first superpixel in the first image and a second superpixel pixel in the second image; and

weight the smoothness factor by the spatial overlap.

19. A system as recited in claim 18, wherein the first and second images are created by varying an image scale or view of the object.

20. A system as recited in claim 15, wherein the graph-based global decision maker is further operative to:

weight the smoothness factor by a color distance between superpixels.