Object Detection and Recognition System

• US 20080075367A1
• Filed: 09/21/2006
• Published: 03/27/2008
• Est. Priority Date: 09/21/2006
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Shotton, Jamie, Winn, John
• Granted Patent: US 7,912,288 B2
• Time in Patent Office: Days
• US Class Current: 382/180
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/457 by analysing connectivity, e.g. edge linking, connected component analysis or slices

Abstract

During a training phase we learn parts of images which assist in the object detection and recognition task. A part is a densely represented area of an image of an object to which we assign a unique label. Parts contiguously cover an image of an object to give a part label map for that object. The parts do not necessarily correspond to semantic object parts. During the training phase a classifier is learnt which can be used to estimate belief distributions over parts for each image element of a test image. A conditional random field is used to force a global part labeling which is substantially layout-consistent and a part label map is inferred from this. By recognizing parts we enable object detection and recognition even for partially occluded objects, for multiple-objects of different classes in the same scene, for unstructured and structured objects and allowing for object deformation.

Claims

1. A method of object detection and recognition comprising:

(i) receiving an image for which it is required to carry out object detection and recognition; and

(ii) generating a part label map for the received image, the part label map comprising, for each image element of the received image, a label indicating which of a plurality of parts that image element is assigned to, each part being a densely represented image area;

wherein the step of generating the part label map comprises at least;

accessing a pre-specified classifier arranged to estimate a belief distribution over parts for each image element of the received image; and

using an inference algorithm to infer the part label map from a conditional random field by forcing a global part labeling which is substantially layout-consistent.

2. A method as claimed in claim 1 wherein the inference algorithm comprises an annealed expansion move algorithm.

3. A method as claimed in claim 1 wherein the inference algorithm comprises belief propagation.

4. A method as claimed in claim 1 wherein the conditional random field comprises a hidden layer of part labels.

5. A method as claimed in claim 1 which is suitable for detecting and recognizing images of partially occluded objects.

6. A method as claimed in claim 1 which further comprises forming the classifier during a training phase using a plurality of training images together with a mask for each training image indicating which pixels in that image correspond to objects to be recognized and which correspond to background that it is not required to recognize.

7. A method as claimed in claim 6 which further comprises, during the training phase, forming an initial part label map for a training image by dividing the image into a plurality of parts having a consistent pair-wise ordering such that the parts contiguously cover the image.

8. A method as claimed in claim 7 wherein the step of forming an initial part label map also comprises ensuring that the parts meet constraints related to image elements that are non-immediate neighbors.

9. A method as claimed in claim 7 which further comprises deforming the initial part label map for the training image during a learning process to form a deformed part labeling such that parts which assist in the object detection and recognition task are learnt.

10. A method as claimed in claim 9 which further comprises using the deformed labeling to form a new initial part label map for each training image and repeating the learning process.

11. A method of object detection and recognition comprising:

(i) receiving an image for which it is required to carry out object detection and recognition that image being of partially occluded objects;

(ii) accessing a pre-specified classifier arranged to estimate a belief distribution over parts for each image element of the received image;

(iii) applying an inference process to a conditional random field model to force a global part labeling which is substantially layout-consistent and thus generating a part label map from the conditional random field model for the received image, the part label map comprising, for each image element of the received image, a label indicating which of a plurality of parts that image element is assigned to, each part being a densely represented image area.

12. A method as claimed in claim 11 wherein the step of using a conditional random field model comprises using such a model having a hidden layer of part labels.

13. A method as claimed in claim 11 wherein the step of using a conditional random field model comprises using a plurality of decision trees.

14. An apparatus for object detection and recognition comprising:

(i) an input arranged to receive an image for which it is required to carry out object detection and recognition;

(ii) an input arranged to access a pre-specified classifier arranged to estimate a belief distribution over parts for each image element of the received image;

(iii) a conditional random field model; and

(iv) an inference mechanism arranged to carry out an inference process on the conditional random field model to force a global part labeling which is substantially layout-consistent and thereby generate a part label map for the received image, the part label map comprising, for each image element of the received image, a label indicating which of a plurality of parts that image element is assigned to, each part being a densely represented image area.

15. An apparatus as claimed in claim 14 wherein the inference mechanism is arranged to carry out an annealed expansion move algorithm.

16. An apparatus as claimed in claim 14 wherein the conditional random field model comprises a hidden layer of part labels.

17. An apparatus as claimed in claim 14 wherein the classifier comprises a plurality of decision trees.

18. An apparatus as claimed in claim 14 which further comprises a processor arranged to form the classifier during a training phase using a plurality of training images together with a mask for each training image indicating which pixels in that image correspond to objects to be recognized and which correspond to background that it is not required to recognize.

19. An apparatus as claimed in claim 18 wherein the processor is further arranged to, during the training phase, form an initial part label map for a training image by dividing the image into a plurality of parts having a consistent pair-wise ordering such that the parts contiguously cover the image.

20. An apparatus as claimed in claim 19 wherein the processor is further arranged to deform the initial part label map for the training image during a learning process to form a deformed part labeling such that parts which assist in the object detection and recognition task are learnt.