METHOD AND APPARATUS FOR CLASSIFYING PIXELS IN AN INPUT IMAGE AND IMAGE PROCESSING SYSTEM

• US 20150030255A1
• Filed: 07/22/2014
• Published: 01/29/2015
• Est. Priority Date: 07/25/2013
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Wu, Bo, Jiang, Yong, Lv, You, Xu, Xingwei
• Granted Patent: US 9,576,201 B2
• Time in Patent Office: Days
• US Class Current: 382/224
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

The present disclosure relates to a method and apparatus for classifying pixels in an input image and image processing system. The apparatus includes a first confidence determination unit configured for determining a first confidence of each of the pixels in the input image belonging to each of a plurality of predetermined classes; a detection unit configured for acquiring, by using a detector for detecting a particular class among the plurality of predetermined classes, detection results used to indicate pixels which belong to the particular class; an updating unit configured for updating the first confidences according to the detection results; and a class determination unit configured for determining the class to which each of the pixels in the input image belongs among the plurality of predetermined classes, according to the updated first confidences.

Claims

1. An apparatus for classifying pixels in an input image, comprising:

a first confidence determination unit configured for determining a first confidence of each of pixels in the input image belonging to each of a plurality of predetermined classes;

a detection unit configured for acquiring, by using a detector for detecting a particular class among the plurality of predetermined classes, detection results used to indicate pixels which belong to the particular class;

an updating unit configured for updating the first confidences according to the detection results; and

a class determination unit configured for determining the class to which each of the pixels in the input image belongs among the plurality of predetermined classes, according to the updated first confidences.

2. The apparatus according to claim 1, wherein, the updating unit comprises:

an update pixel determination sub-unit configured for determining a plurality of pixels whose first confidences need to be updated, according to the detection results; and

a first updating sub-unit configured for updating, for each of the plurality of pixels, the first confidence of the pixel according to a probability of this pixel belonging to a foreground of the input image.

3. The apparatus according to claim 2, wherein, a confidence of a pixel is determined according to a confidence of a super-pixel where the pixel is located, and super-pixels are obtained by segmenting the input image,

wherein, the update pixel determination sub-unit is further configured for determining pixels included in the super-pixels covered by a region formed of the pixels which belong to the particular class as the pixels whose first confidences need to be updated,

wherein, the first updating sub-unit comprises:

a second confidence determination sub-unit configured for determining, for each of the super-pixels covered by the region formed of the pixels which belong to the particular class, a second confidence of the super-pixel belonging to each of the plurality of predetermined classes, according to the probabilities of the pixels included in the super-pixel belonging to the foreground of the input image; and

an updating sub-unit configured for updating, for each of the super-pixels covered by the region formed of the pixels which belong to the particular class, the first confidences of the pixels included in the super-pixel according to the second confidence of the super-pixel.

4. The apparatus according to claim 3, wherein, for each of the super-pixels covered by the region formed of the pixels which belong to the particular class,

the second confidence of the super-pixel belonging to the particular class is equal to an average value of the probability of each pixel included in the super-pixel belonging to the foreground of the input image;

and

the second confidence of the super-pixel belonging to each of other predetermined classes is equal to an average value of a probability of each pixel included in the super-pixel belonging to the corresponding one of the other predetermined classes, wherein the probability of the pixel belonging to each of the other predetermined classes is equal to (1−

the probability of the pixel belonging to the foreground of the input image)/(the number of the plurality of predetermined classes−

1).

5. The apparatus according to claim 3, wherein, the updating sub-unit is further configured for updating, for each of the pixels included in the super-pixel, the first confidence of the pixel, by linearly combining the second confidence of the super-pixel with the first confidence of the pixel.

6. The apparatus according to claim 5, wherein, a weight assigned to the second confidence is larger than a weight assigned to the first confidence.

7. The apparatus according to claim 3, wherein, the super-pixels covered by the region formed of the pixels which belong to the particular class include a super-pixel which has a ratio larger than a first predetermined threshold, wherein the ratio is equal to the number of this super-pixel'"'"'s pixels overlapped with the region formed of the pixels which belong to the particular class divided by the total number of this super-pixel'"'"'s pixels.

8. The apparatus according to claim 1, wherein, the updating unit comprises:

an update region acquisition sub-unit configured for determining, within a region formed of the pixels which belong to the particular class, an update region according to the detection results; and

a second updating sub-unit configured for classifying the update region as a whole using a classifier, and updating the first confidence of each pixel in the update region according to an output of the classifier.

9. The apparatus according to claim 8, wherein, the classifier is trained by a training unit, the training unit comprising:

an over-segmentation sub-unit configured for over-segmenting a training image into a set of regions;

a grouping sub-unit configured for grouping the regions which belong to the particular class, to obtain a grouping region; and

a generation sub-unit configured for generating the classifier according to features of the grouping region and other un-grouped regions.

10. The apparatus according to claim 9, wherein, a confidence of a pixel is determined according to a confidence of a super-pixel where the pixel is located, and super-pixels are obtained by segmenting the input image,

wherein, the second updating sub-unit comprises:

a grouping sub-unit configured for grouping the super-pixels covered by the update region among a plurality of super-pixels into which the input image is segmented, to obtain a grouping super-pixel; and

an updating sub-unit configured for updating the first confidences of the pixels included in each super-pixel forming the grouping super-pixel, according to a confidence of the grouping super-pixel determined by using the classifier.

11. The apparatus according to claim 10, wherein, the super-pixels covered by the update region include a super-pixel which has a ratio of the number of its pixels overlapped with the update region to the total number of its pixels larger than a second predetermined threshold.

12. The apparatus according to claim 1, wherein, the detection unit comprises:

a first detection sub-unit configured for detecting a plurality of image regions including the pixels which belong to the particular class in the input image by using the detector; and

a first determination sub-unit configured for determining a region generated by grouping the plurality of image regions as the detecting results.

13. The apparatus according to claim 1, wherein, the detection unit comprises:

a second detection sub-unit configured for detecting, in a case that the particular class is a human class, a face region in the input image by using the detector being a face detector; and

a second determination sub-unit configured for determining a region generated by enlarging the face region as the detection results.

14. An image processing system, comprising:

the apparatus according to claim 1.

15. A method comprising:

determining a first confidence of each of pixels in the input image belonging to each of a plurality of predetermined classes;

acquiring, by using a detector for detecting a particular class among the plurality of predetermined classes, detection results used to indicate pixels which belong to the particular class;

updating the first confidences according to the detection results; and

determining the class to which each of the pixels in the input image belongs among the plurality of predetermined classes, according to the updated first confidences.