Generating a multi-use vocabulary based on image data

• US 8,396,331 B2
• Filed: 07/31/2007
• Issued: 03/12/2013
• Est. Priority Date: 02/26/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Jia, Menglei, Xie, Xing, Ma, Wei-Ying
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Thirugnanam, Gandhi
• Application Number:
  US11/831,862
• Publication Number:
  US 20080205770A1
• Time in Patent Office: 2,051 Days
• Field of Search: 382305-306
• US Class Current: 382/305
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations

Abstract

Functionality is described for generating a vocabulary from a source dataset of image items or other non-textual items. The vocabulary serves as a tool for retrieving items from a target dataset in response to queries. The vocabulary has at least one characteristic that allows it to be used to retrieve items from multiple different target datasets. A target dataset can have a different size than the source dataset and/or a different type than the source dataset. The enabling characteristic may correspond to a size of the source dataset above a prescribed minimum number of items and/or a size of the vocabulary above a prescribed minimum number of words.

Claims

1. A method for generating a vocabulary for non-textual items, comprising:

under control of one or more processors configured with executable instructions;

providing a source dataset of a first type comprising a plurality of items of the first type;

identifying features in the source dataset; and

generating a plurality of words associated with the features to form a single vocabulary, the single vocabulary serving as a mechanism for use in retrieving items from plural different target datasets of different types in response to queries made to the plural different target datasets, wherein;

the different types comprise different themes or scenes,

each word of the plurality of words is associated with a weight with respect to a particular document, the weight being determined based on multiplying a term frequency (TF) of the word with an inverse document frequency (IDF) of the word,

the term frequency of each word with respect to the particular document comprises a normalized frequency of the word in the particular document,

the inverse document frequency of each word determines whether the word is useful for distinguishing a relevant document from an irrelevant document based on how frequently the word appears in a plurality of documents, and

the inverse document frequency of each word is determined based on a logarithmic function of a ratio between a total number of documents in a database and a total number of documents in which the word appears.

2. The method of claim 1, wherein the non-textual items comprise image items.

3. The method of claim 1, wherein the source dataset has at least a predetermined size to ensure that the single vocabulary serves as the mechanism for use in retrieving the items of the types different from the first type from the plural different target datasets of the different types.

4. The method of claim 3, wherein the predetermined size corresponds to an approximate transition point at which further increases in size do not yield significant increases in performance, relative to increases in size prior to the transition point, wherein the single vocabulary is generated using a source dataset having approximately the predetermined size.

5. The method of claim 1, wherein the single vocabulary has at least a predetermined number of words to ensure that the single vocabulary serves as the mechanism for use in retrieving the items of the types different from the first type from the plural different target datasets of the different types.

6. The method of claim 5, wherein the predetermined number corresponds to an approximate transition point at which further increases in number do not yield significant increases in performance, relative to increases in number prior to the transition point, wherein the single vocabulary is generated to have approximately the predetermined number.

7. The method of claim 1, wherein at least one of the plural different target datasets has a size larger than a size of the source dataset.

8. The method of claim 1, wherein the single vocabulary is further configured to serve as a mechanism for use in retrieving the items from the source dataset in response to queries made to the source dataset.

9. The method of claim 4, wherein the transition point includes a leveling-off point in a performance vs. size graph.

10. One or more memory devices configured with computer-executable instructions that, when executed by one or more processors, configure the one or more processors to perform acts comprising:

providing a source dataset of a first type comprising a plurality of items of the first type;

identifying features of the first type in the source dataset; and

generating a plurality of words associated with the features to form a single vocabulary, the single vocabulary serving as a mechanism for use in retrieving items from plural different target datasets of different types in response to queries made to the plural different target datasets, wherein;

the different types comprise different themes or scenes,

each word of the plurality of words is associated with a weight with respect to a particular document, the weight being determined based on multiplying a term frequency (TF) of the word with an inverse document frequency (IDF) of the word,

the term frequency of each word with respect to the particular document comprises a normalized frequency of the word in the particular document,

the inverse document frequency of each word determines whether the word is useful for distinguishing a relevant document from an irrelevant document based on how frequent the word appears in a plurality of documents, and

the inverse document frequency of each word is determined based on a logarithmic function of a ratio between a total number of documents in a database and a total number of documents in which the word appears.

11. The one or more memory devices of claim 10, wherein the non-textual items comprise image items.

12. The one or more memory devices of claim 10, wherein the source dataset has at least a predetermined size to ensure that the single vocabulary serves as the mechanism for use in retrieving the items of the types different from the first type from the plural different target datasets of the different types.

13. The one or more memory devices of claim 12, wherein the predetermined size corresponds to an approximate transition point at which further increases in size do not yield significant increases in performance, relative to increases in size prior to the transition point, wherein the single vocabulary is generated using a source dataset having approximately the predetermined size.

14. The one or more memory devices of claim 13, wherein the transition point includes a leveling-off point in a performance vs. size graph.

15. The one or more memory devices of claim 10, wherein the single vocabulary has at least a predetermined number of words to ensure that the single vocabulary serves as the mechanism for use in retrieving the items of the types different from the first type from the plural different target datasets of the different types.

16. The one or more memory devices of claim 15, wherein the predetermined number corresponds to an approximate transition point at which further increases in number do not yield significant increases in performance, relative to increases in number prior to the transition point, wherein the single vocabulary is generated to have approximately the predetermined number.

17. The one or more memory devices of claim 10, wherein at least one of the plural different target datasets has a size larger than a size of the source dataset.

18. The one or more memory devices of claim 10, wherein the single vocabulary is further configured to serve as a mechanism for use in retrieving the items from the source dataset in response to queries made to the source dataset.

19. One or more computing devices, comprising:

one or more processors; and

memory to store computer-executable instructions that, when executed by the one or more processors, perform acts comprising;

providing a source dataset of a first type comprising a plurality of items of the first type;

identifying features in the source dataset; and

generating a plurality of words associated with the features to form a single vocabulary, the single vocabulary serving as a mechanism for use in retrieving items from plural different target datasets of different types in response to queries made to the plural different target datasets, wherein;

the different types comprise different themes or scenes,

each word of the plurality of words is associated with a weight with respect to a particular document, the weight being determined based on multiplying a term frequency (TF) of the word with an inverse document frequency (IDF) of the word,

the term frequency of each word with respect to the particular document comprises a normalized frequency of the word in the particular document,

the inverse document frequency of each word determines whether the word is useful for distinguishing a relevant document from an irrelevant document based on how frequent the word appears in a plurality of documents, and

the inverse document frequency of each word is determined based on a logarithmic function of a ratio between a total number of documents in a database and a total number of documents in which the word appears.