Kernelized spatial-contextual image classification

• US 8,131,086 B2
• Filed: 09/24/2008
• Issued: 03/06/2012
• Est. Priority Date: 09/24/2008
• Status: Expired due to Fees

• Current Assignee: Doheny Eye Institute, Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hua, Xian-Sheng, Qi, Guo-Jun, Rui, Yong, Zhang, Hong-Jiang
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US12/237,298
• Publication Number:
  US 20100074537A1
• Time in Patent Office: 1,259 Days
• Field of Search: 382/173, 382/180, 382/181, 382/226, 382/224
• US Class Current: 382/226
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G06V 10/426 Graphical representations
• CPC Class Codes: G06V 10/85 Markov-related models; Markov random fields

Abstract

Kernelized spatial-contextual image classification is disclosed. One embodiment comprises generating a first spatial-contextual model to represent a first image, the first spatial-contextual model having a plurality of interconnected nodes arranged in a first pattern of connections with each node connected to at least one other node, generating a second spatial-contextual model to represent a second image using the first pattern of connections, and estimating the distance between corresponding nodes in the first spatial-contextual model and the second spatial-contextual model based on a relationship with adjacent connected nodes to determine a distance between the first image and the second image.

Claims

1. A method for categorizing a plurality of unlabeled images, the method comprising:

on a computing device;

generating a first spatial-contextual model to represent a first image, the first spatial-contextual model having a plurality of interconnected nodes arranged in a first pattern of connections with each node connected to at least one other node;

generating a second spatial-contextual model to represent a second image, the second spatial-contextual model having a plurality of interconnected nodes arranged in the first pattern of connections;

determining a relationship of adjacent nodes that each node is connected with in the first pattern of connections; and

calculating a distance between corresponding nodes in the first spatial-contextual model and the second spatial-contextual model based on the relationship of adjacent connected nodes to determine a distance between the first image and the second image.

2. The method of claim 1, wherein calculating a distance between corresponding nodes involves estimating the distance using an upper-bounded Kullback-Leibler Divergence based on the relationship of adjacent nodes.

3. The method of claim 1, wherein the first spatial-contextual model and the second spatial-contextual model are each a Dependency Tree Hidden Markov Model (DT-HMM).

4. The method of claim 2, wherein the upper-bounded Kullback-Leibler Divergence is calculated using a formula:

5. The method of claim 4, further comprising:

training a Universal Reference Model from a plurality of referential images; and

adapting the first spatial-contextual model and the second spatial-contextual model to the Universal Reference Model prior to calculating a distance between corresponding nodes.

6. The method of claim 4, wherein if each of the corresponding nodes are not connected with a successor node, the method further comprises calculating the upper-bounded Kullback-Leibler Divergence using a formula:

7. The method of claim 4, wherein if each of the corresponding nodes are connected with a vertical successor node, the method further comprises calculating the upper-bounded Kullback-Leibler Divergence using a formula:

8. The method of claim 4, wherein if each of the corresponding nodes are connected with a horizontal successor node, the method further comprises calculating the upper-bounded Kullback-Leibler Divergence using a formula:

9. The method of claim 4, wherein if each of the corresponding nodes are connected with a vertical successor node and a horizontal successor node, the method further comprises calculating the upper-bounded Kullback-Leibler Divergence using a formula:

10. A computer-readable storage medium comprising stored instructions executable by a computing device to categorize a plurality of unlabeled images, the stored instructions executed by the computing device to perform a method comprising:

generating a first spatial-contextual model to represent a first image, the first spatial-contextual model having a plurality of interconnected nodes arranged in a first pattern of connections with each node connected to at least one other node;

generating a second spatial-contextual model to represent a second image, the second spatial-contextual model having a plurality of interconnected nodes arranged in the first pattern of connections;

training a Universal Reference Model from a plurality of referential images;

adapting the first spatial-contextual model and the second spatial-contextual model to the Universal Reference Model;

determining a relationship of adjacent nodes that each node is connected with in the first pattern of connections; and

calculating a distance between corresponding nodes in the first spatial-contextual model and the second spatial-contextual model based on the relationship of adjacent connected nodes.

11. The computer-readable storage medium of claim 10, wherein adapting the first spatial-contextual model and the second spatial-contextual model to the Universal Reference Model comprises a Maximum A Posteriori technique.

12. The computer-readable storage medium of claim 10, wherein calculating a distance between corresponding nodes involves estimating the distance using an upper-bounded Kullback-Leibler Divergence based on the relationship of adjacent nodes.

13. The computer-readable storage medium of claim 10, wherein the first spatial-contextual model and the second spatial-contextual model are each a Dependency Tree Hidden Markov Model (DT-HMM).

14. The computer-readable storage medium of claim 12, wherein the upper-bounded Kullback-Leibler Divergence is calculated using a formula:

15. The computer-readable storage medium of claim 14, wherein if each of the corresponding nodes are not connected with a successor node, the medium further comprises instructions for calculating the upper-bounded Kullback-Leibler Divergence using a formula:

16. The computer-readable storage medium of claim 14, wherein if each of the corresponding nodes are connected with a horizontal successor node, the medium further comprises instructions for calculating the upper-bounded Kullback-Leibler Divergence using a formula:

17. The computer-readable storage medium of claim 13, wherein if each of the corresponding nodes are connected with a vertical successor node, the medium further comprises instructions for calculating the upper-bounded Kullback-Leibler Divergence using a formula:

18. The computer-readable storage medium of claim 13, wherein if each of the corresponding nodes are connected with a vertical successor node and a horizontal successor node, the medium further comprises instructions for calculating the upper-bounded Kullback-Leibler Divergence using a formula:

19. A system to categorize a plurality of unlabeled images, the system comprising:

an input;

a memory including stored instructions; and

a processor in communication with the input and the memory, the processor being configured to executed the stored instructions to implement;

a model generation module configured to generate a first dependency tree hidden Markov model to represent a first image, the model generation module further configured to generate a second dependency tree hidden Markov model to represent a second image;

a distance module in communication with the model generation module, the distance module configured to calculate an image distance between the first dependency tree hidden Markov model and the second dependency tree hidden Markov model; and

a categorization module to receive the image distance and to generate a kernelized spatial-contextual model using the image distance.

20. The system of claim 19, further comprising an adaptation module in communication with the categorization module, the adaptation module being configured to train a universal reference model from a plurality of representative images and the categorization module being further configured to generate a set of kernelized spatial-contextual models.