SYSTEM AND METHOD FOR CLASS-SPECIFIC OBJECT SEGMENTATION OF IMAGE DATA

• US 20090316988A1
• Filed: 06/18/2008
• Published: 12/24/2009
• Est. Priority Date: 06/18/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Mohananchettiar, Arunkumar, Xu, Ning
• Granted Patent: US 8,107,726 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06V 30/19147 Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 30/19187 Graphical models, e.g. Bayesian networks or Markov models
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

Systems and methods for processing an image to determine whether segments of the image belong to an object class are disclosed. In one embodiment, the method comprises receiving digitized data representing an image, the image data comprising a plurality of pixels, segmenting the pixel data into segments at a plurality of scale levels, determining feature vectors of the segments at the plurality of scale levels, the feature vectors comprising one or more measures of visual perception of the segments, determining one or more similarities, each similarity determined by comparing two or more feature vectors, determining, for each of a first subset of the segments, a first measure of probability that the segments is a member of an object class, determining probability factors based on the determined first measures of probability and similarity factors based on the determined similarities, and performing factor graph analysis to determine a second measure of probability for each of a second subset of the segments based on the probability factors and similarity factors.

Claims

1. A method of processing an image, comprising:

receiving digitized data representing an image, the image data comprising a plurality of pixels;

segmenting the pixel data into segments at a plurality of scale levels;

determining feature vectors of the segments at the plurality of scale levels, the feature vectors comprising one or more measures of visual perception of the segments;

determining one or more similarities, each similarity determined by comparing two or more feature vectors;

determining, for each of a first subset of the segments, a first measure of probability that the segments is a member of an object class;

determining probability factors based on the determined first measures of probability and similarity factors based on the determined similarities; and

performing factor graph analysis to determine a second measure of probability for each of a second subset of the segments based on the probability factors and similarity factors.

2. The method of claim 1, wherein each of the second subset of segments are variable nodes in the factor graph analysis and the factor nodes between the variable nodes at the different scale levels are computed from the similarity factors and probability factors.

3. The method of claim 2, fiuther comprising classifying at least one segment as either being a member of an object class or not a member of an object class based on the determined second measure of probability.

4. The method of claim 3, wherein classifying at least one segment comprises maximizing the second measure of probability at the finest scale level.

5. The method of claim 1, wherein determining the first measure of probability comprises using one or more learned boosting classifier models generated for the object class.

6. The method of claim 5, further comprising generating the one or more learned boosting classifier models based on images comprising predetermined object class segments and non-object class segments.

7. The method of claim 5, wherein the one or more learned boosting classifier models comprises a learned boosting classifier model generated for each of the plurality of scale levels.

8. The method of claim 1, wherein one of the segments at a first scale level in the factor graph analysis is connected via the edges with two or more of the segments at a second scale level in the factor graph analysis, where the second scale level is a finer scale than the first scale level.

9. The method of claim 2, further comprising:

determining one or more candidate segment label maps, wherein the candidate segment label maps identify different sets of segments as being members of the object class;

for the candidate segment label maps, identifying pairs of pixels bordering edges of neighboring object class segments, and calculating a measure indicative that the identified pairs of pixels are edge pixels between an object class segment and a non-object class segment; and

for the candidate segment label maps, generating an energy function based on the second measure of probability and the calculated edge pixel measure, wherein the energy function (1) rewards labeling a segment according to the second measure of probability, and (2) penalizes labeling two neighboring segments as object class segments based on the edge pixel measure.

10. The method of claim 9, further comprising classifying the segments as being members of the object class based on the energy functions of the candidate segment label maps.

11. The method of claim 10, wherein the energy function comprises a first cost component and a second cost component, the first cost component increasing the energy function proportionally to the edge pixel measure, and the second cost component increasing the energy function more for classifying segments contrary to the second measure of probability than for classifying segments in accordance with the second measure of probability, the method further comprising classifying the segments based on minimizing the energy function.

12. The method of claim 9, wherein calculating the edge pixel measure comprises using a learned boosting classifier algorithm generated to identify edges between object class segments and non-object class segments.

13. The method of claim 12, further comprising generating the edge identifying learned boosting classifier model based on images comprising predetermined object class segments and non-object class segments.

14. A system for processing an image, comprising:

a video subsystem configured to receive digitized data representing an image, the image data comprising a plurality of pixels;

an image segmentation subsystem configured to segment the pixel data into segments at a plurality of scale levels;

a perceptual analysis subsystem configured to determine feature vectors of the segments at the plurality of scale levels, the feature vectors comprising one or more measures of visual perception of the segments, and to determine one or more similarities, each similarity determined by comparing two or more feature vectors;

an object classification subsystem configured to determine, for each of a first subset of the segments, a first measure of probability that the segment is a member of an object class; and

a statistical analysis subsystem configured to determine probability factors based on the determined first measures of probability and similarity factors based on the determined similarities and further configured to perform factor graph analysis to determine a second measure of probability for each of a second subset of the segments based on the probability factors and similarity factors.

15. The system of claim 14, wherein each of the second subset of segments are variable nodes in the factor graph analysis and the factor nodes between the variable nodes at the different scale levels are computer from the similarity factors and probability factors.

16. The system of claim 15, wherein the object classification subsystem is further configured to classify at least on segment as either being a member of an object class or not a member of an object class based on the determined second measure of probability.

17. The system of claim 16, wherein the object classification subsystem is further configured to classify at least one segment based on maximizing the second measure of probability at the finest scale level.

18. The system of claim 17, wherein the object classification subsystem is configured to determine the first measure of probability using one or more learned boosting classifier models generated for the object class.

19. The system of claim 18, wherein the object classification subsystem is further configured to generate the one or more learned boosting classifier models based on images comprising predetermined object class segments and non-object class segments.

20. The system of claim 18, wherein the one or more learned boosting classifier models comprises a learned boosting classifier model generated for each of the plurality of scale levels.

21. The system of claim 14, wherein one of the segments at a first scale level in the factor graph analysis is connected via the edges with two or more of the segments at a second scale level in the factor graph analysis, where the second scale level is a finer scale than the first scale level.

22. The system of claim 15, wherein the object classification subsystem is further configured to determine one or more candidate segment label maps, wherein the candidate segment label maps identify different sets of segments as being members of the object class, the system further comprising:

an edge information subsystem configured to, for the candidate segment label maps, identify pairs of pixels bordering edges of neighboring object class segments, and calculate a measure indicative that the identified pairs of pixels are edge pixels between an object class segment and a non-object class segment, and

wherein the statistical analysis subsystem is further configured to, for the candidate segment label maps, generate an energy function based on the second measure of probability and the calculated edge pixel measure, wherein the energy function (1) rewards labeling a segment according to the second measure of probability, and (2) penalizes labeling two neighboring segments as object class segments based on the edge pixel measure.

23. The system of claim 22, wherein the object classification subsystem is further configured to classify the segments as being members of the object class based on the energy functions of the candidate segment label maps.

24. The system of claim 23, wherein the energy function comprises a first cost component and a second cost component, the first cost component increasing the energy function proportionally to the edge pixel measure, and the second cost component increasing the energy function more for classifying segments contrary to the second measure of probability than for classifying segments in accordance with the second measure of probability, and wherein the object classification subsystem classifies the segments based on minimizing the energy function.

25. The system of claim 22, wherein the edge information subsystem calculates the edge pixel measure using a learned boosting classifier algorithm generated to identify edges between object class segments and non-object class segments.

26. The system of claim 25, wherein the edge information subsystem is further configured to generate the edge identifying learned boosting classifier model based on images comprising predetermined object class segments and non-object class segments.