Estimating Word Correlations from Images

• US 20090074306A1
• Filed: 12/13/2007
• Published: 03/19/2009
• Est. Priority Date: 09/13/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ma, Wei-Ying, Li, Mingjing, Liu, Jing, Wang, Bin, Li, Zhiwei
• Granted Patent: US 8,457,416 B2
• Time in Patent Office: Days
• US Class Current: 382/229
• CPC Class Codes: G06F 16/36 Creation of semantic tools, e.g. ontology or thesauri
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content

Abstract

Word correlations are estimated using a content-based method, which uses visual features of image representations of the words. The image representations of the subject words may be generated by retrieving images from data sources (such as the Internet) using image search with the subject words as query words. One aspect of the techniques is based on calculating the visual distance or visual similarity between the sets of retrieved images corresponding to each query word. The other is based on calculating the visual consistence among the set of the retrieved images corresponding to a conjunctive query word. The combination of the content-based method and a text-based method may produce even better result.

Claims

1. A method for estimating word correlations, the method comprising:

providing an image representation of a first word and an image representation of a second word; and

estimating a correlation between the first word and the second word at least partially based on visual features of the image representations of the first word and the second word.

2. The method as recited in claim 1, wherein estimating the correlation between the first word and second word comprises:

calculating a visual distance or a visual similarity between the image representation of the first word and the image representation of the second word.

3. The method as recited in claim 1, wherein the image representation of the first word comprises a first image set including a first plurality of images, and the image representation of the second word comprises a second image set including a second plurality of images.

4. The method as recited in claim 3, wherein estimating the correlation between the first word and second word comprises:

calculating a visual distance or a similarity between the first image set and the second image set.

5. The method as recited in claim 1, wherein providing the image representations of the first word and the second word comprises:

conducting an image search using the respective first word or second word as query word; and

selecting a plurality of images from search results.

6. The method as recited in claim 5, wherein the image search is conducted on the Internet using an image search engine.

7. The method as recited in claim 5, wherein the plurality of images are selected from top returns of the search results.

8. The method as recited in claim 1, further comprising:

providing a conjunctive image representation of the first word and the second word,

wherein estimating the correlation between the first word and second word is further based on visual features of the conjunctive image representation of the first word and the second word.

9. The method as recited in claim 8, wherein the conjunctive image representation comprises an image set having a plurality of images, and estimating the correlation between the first word and second word comprises calculating a visual consistence of the image set.

10. The method as recited in claim 8, wherein providing the conjunctive image representation of the first word and the second word comprises:

performing an image search using the first word and the second word as a conjunctive query word; and

selecting a plurality of images from search results.

11. The method as recited in claim 8, wherein estimating the correlation between the first word and second word comprises:

calculating a first correlation between the first word and the second word based on the visual features of the image representations of the first word and the second word;

calculating a second correlation between the first word and the second word based on the visual features of the conjunctive image representation of the first word and the second word; and

estimating the correlation between the first word and the second word by combining the first correlation and the second correlation.

12. The method as recited in claim 1, wherein estimating the correlation between the first word and the second word comprises:

calculating a first correlation between the first word and the second word based on the visual features of the image representations of the first word and the second word;

calculating a second correlation between the first word and the second word using a text-based method; and

estimating the correlation between the first word and the second word by combining the first correlation and the second correlation.

13. A method for estimating word correlations, the method comprising:

providing a conjunctive image representation of a first word and a second word; and

estimating a correlation between the first word and second word at least partially based on visual features of the conjunctive image representation of the first word and the second word.

14. The method as recited in claim 13, wherein the conjunctive image representation comprises an image set having a plurality of images, and estimating the correlation between the first word and second word comprises calculating a visual consistence of the image set.

15. The method as recited in claim 13, wherein the conjunctive image representation comprises an image set having a plurality of images, and estimating the correlation between the first word and the second word comprises:

selecting a subset of image features from a plurality of image features extracted from the image set, wherein the subset of image features includes at least one image feature having a low variance according to a preset variance threshold; and

calculating a partial variance of the subset of image features.

16. The method as recited in claim 15, wherein selecting the subset of image features is performed dynamically as the conjunctive image representation varies.

17. The method as recited in claim 13, wherein providing the conjunctive image representation of the first word and the second word comprises:

conducting an image search using the first word and the second word as a conjunctive query word; and

selecting a plurality of images from search results.

18. One or more computer readable media having stored thereupon a plurality of instructions that, when executed by a processor, causes the processor to:

provide an image representation of a first word and an image representation of a second word; and

estimate a correlation between the first word and the second word at least partially based on visual features of the image representations of the first word and the second word.

19. The computer readable media as recited in claim 18, wherein the processor estimates the correlation between the first word and second word by calculating a visual distance or a visual similarity between the image representation of the first word and the image representation of the second word.

20. The computer readable media as recited in claim 18, wherein the plurality of instructions, when executed by a processor, further causes the processor to:

provide a conjunctive image representation of the first word and the second word, wherein the processor estimates the correlation between the first word and second word further based on visual features of the conjunctive image representation of the first word and the second word.