Detecting and Localizing Multiple Objects in Images Using Probabilistic Inference

• US 20120207346A1
• Filed: 02/10/2011
• Published: 08/16/2012
• Est. Priority Date: 02/10/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kohli, Pushmeet, Lempitsky, Victor, Barinova, Olga
• Granted Patent: US 8,953,888 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

An object detection system is disclosed herein. The object detection system allows detection of one or more objects of interest using a probabilistic model. The probabilistic model may include voting elements usable to determine which hypotheses for locations of objects are probabilistically valid. The object detection system may apply an optimization algorithm such as a simple greedy algorithm to find hypotheses that optimize or maximize a posterior probability or log-posterior of the probabilistic model or a hypothesis receiving a maximal probabilistic vote from the voting elements in a respective iteration of the algorithm. Locations of detected objects may then be ascertained based on the found hypotheses.

Claims

1. A computer-implemented method comprising:

under control of one or more processors configured with executable instructions;

receiving an image including an unknown number of objects to be detected;

obtaining a plurality of voting elements from the image, the plurality of voting elements placing votes on one or more hypotheses to determine one or more locations of one or more objects in the image;

deriving a probabilistic model based at least on the plurality of voting elements; and

ascertaining locations of a plurality of objects in the image based at least in part on the probabilistic model and a greedy algorithm maximizing a joint probability of a plurality of hypotheses for the locations of the plurality of objects.

2. The computer-implemented method as recited in claim 1, wherein the plurality of voting elements include a plurality of descriptors, the plurality of descriptors usable in determining votes on the plurality of hypotheses for the locations of the plurality of objects.

3. The computer-implemented method as recited in claim 2, wherein the plurality of descriptors include local image appearances and/or geometric positions of the plurality of voting elements.

4. The computer-implemented method as recited in claim 3, wherein detecting the locations of the plurality of objects further include traversing the plurality of voting elements through a Hough forest including a plurality of trees based on the plurality of descriptors.

5. The computer-implemented method as recited in claim 4, wherein traversing the plurality of voting elements includes traversing the plurality of voting elements to one or more leaf nodes of the plurality of trees, each leaf node including one or more class labels and one or more offset vectors defining positions of class objects corresponding to the one or more class labels.

6. The computer-implemented method as recited in claim 5, wherein detecting the locations of the plurality of objects further includes obtaining probabilistic votes from the one or more leaf nodes of the plurality of trees reached by the one or more voting elements.

7. The computer-implemented method as recited in claim 6, wherein the greedy algorithm iteratively increases a total probability that is based on the probabilistic votes from the one or more leaf nodes for the plurality of hypotheses for the locations of the plurality of objects.

8. The computer-implemented method as recited in claim 1, wherein obtaining the plurality of voting elements includes dividing the image into a predetermined number of image segments and assigning information of each image segment to each voting element.

9. The computer-implemented method as recited in claim 8, wherein the image segments include edge pixels, interest points, image patches and/or image regions.

10. The computer-implemented method as recited in claim 1, wherein the plurality of hypotheses for the locations of the plurality of objects minimize likelihoods of the plurality of voting elements from voting on a hypothesis for a location of an object other than the plurality of objects.

11. The computer-implemented method as recited in claim 1, wherein the probabilistic model further includes a penalty factor to discourage hallucinated object detection by penalizing number of the plurality of hypotheses used to explain the unknown number of objects in the image.

12. The computer-implemented method as recited in claim 11, wherein the penalty factor includes an Occam razor or a Minimum Description Length (MDL) penalty factor.

13. A computer-implemented method comprising:

under control of one or more processors configured with executable instructions;

receiving an image including an unknown number of objects;

applying a Hough forest to the image to obtain a plurality of voting elements, the plurality of voting elements usable to place probabilistic votes on one or more hypotheses for locations of one or more objects;

deriving a probabilistic model based at least on the plurality of voting elements and the one or more hypotheses for the locations of the one or more objects, the probabilistic model combining the probabilistic votes from the plurality of voting elements on the one or more hypotheses for the locations of the one or more objects;

iteratively ascertaining a hypothesis for a location of an object in the image based on the probabilistic model in response to determining that probabilistic votes on the hypothesis for the location of the object is maximum compared to other hypotheses for locations of other objects in a respective iteration.

14. The computer-implemented method as recited in claim 13, wherein the plurality of voting elements includes a plurality of descriptors, the plurality of descriptors including local image appearances and/or geometric positions of the plurality of voting elements.

15. The computer-implemented method as recited in claim 13, wherein the probabilistic model further includes a total probability, the total probability depending on probabilities of the hypothesis for the location of the object and the other hypotheses for the locations of the other objects in the image.

16. The computer-implemented method as recited in claim 15, wherein ascertaining the location of the object further includes iteratively applying a greedy algorithm to associate a voting element of the plurality of voting elements to the hypothesis for the location of the object in response to determining that associating the voting element to the hypothesis increases the total probability of the probabilistic model.

17. The computer-implemented method as recited in claim 16, wherein the hypothesis for the location of the object minimize likelihoods of the voting element from voting on another hypothesis associated with another object other than the object.

18. The computer-implemented method as recited in claim 13, wherein the probabilistic model further includes a penalty factor to discourage hallucinated object detection by penalizing number of hypotheses used to explain the unknown number of objects in the image.

19. The computer-implemented method as recited in claim 18, wherein the penalty factor includes an Occam razor or a Minimum Description Length (MDL) penalty factor.

20. One or more computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

receiving an image including an unknown number of objects to be detected;

obtaining one or more voting elements from the image, the one or more voting elements casting probabilistic votes on a hypothesis for a location of an object to determine whether the hypothesis is probabilistically true;

deriving a probabilistic model based at least on the one or more voting elements and one or more hypotheses for locations of one or more objects;

applying a greedy algorithm to iteratively assign a voting element of the one or more voting elements to a hypothesis of the one or more hypotheses in response to determining that assigning the voting element to the hypothesis increases a log-posterior probability associated with the probabilistic model; and

ascertaining a location of an object in the image in response to determining that votes for a hypothesis for the ascertained location of the object are maximum among the one or more hypotheses in a respective iteration of the greedy algorithm.