Maximizing expected generalization for learning complex query concepts

• US 6,976,016 B2
• Filed: 12/21/2001
• Issued: 12/13/2005
• Est. Priority Date: 04/02/2001
• Status: Expired due to Fees

• Current Assignee: VIM Technologies Ltd (MAI Holdings, Inc.)
• Original Assignee: VIM Technologies Ltd (MAI Holdings, Inc.)
• Inventors: Chang, Edward Y., Cheng, Kwang-Ting
• Primary Examiner(s): Metjahic, Safet
• Assistant Examiner(s): LEROUX, ETIENNE PIERRE
• Application Number:
  US10/032,319
• Publication Number:
  US 20030050923A1
• Time in Patent Office: 1,453 Days
• Field of Search: 707/3, 707/5, 707/7, 707/102, 707/104.1, 345/810, 348/231, 382/159
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9032 Query formulation
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

A method of learning user query concept for searching visual images encoded in computer readable storage media comprising: providing a multiplicity of sample images encoded in a computer readable medium; providing a multiplicity of sample expressions that correspond to sample images and in which terms of the sample expressions represent features of corresponding sample images; defining a user query concept sample space bounded by a boundary k-CNF expression and by a boundary k-DNF expression refining the user query concept sample space by, soliciting user feedback as to which of the multiple presented sample images are close to the user'"'"'s query concept; removing from the boundary k-CNF expression disjunctive terms based upon the solicited user feedback; and removing from the boundary k-DNF expression respective conjunctive terms based upon the solicited user feedback.

Claims

1. A method of learning a user query concept for searching visual images encoded in computer readable storage media comprising:

providing a multiplicity of respective sample images encoded in a computer readable medium;

providing a multiplicity of respective sample expressions encoded in computer readable medium that respectively correspond to respective sample images and in which respective terms of such respective sample expressions represent respective features of corresponding sample images;

defining a user query concept sample space bounded by a k-CNF expression which models a query concept and by a k-DNF expression;

refining the user query concept sample space by,

selecting multiple respective sample images from within the user query concept sample space by selecting respective sample expressions that correspond to such images, wherein respective sample expressions are selected by optimizing a tradeoff between a respective expression'"'"'s having sufficient similarity to the k-CNF expression that a user is likely to indicate that its corresponding sample image is close to the user'"'"'s query concept and such respective expression'"'"'s having sufficient dissimilarity from the k-CNF expression that an indication by the user that its corresponding sample image is close to the user'"'"'s query concept is likely to provide maximum information as to which disjunctive terms of the k-CNF expression do not match the user'"'"'s query concept;

presenting the multiple selected sample images to the user;

soliciting user feedback as to which of the multiple presented sample images are close to the user'"'"'s query concept;

wherein refining the user query concept sample space further includes, refining the k-CNF expression by,

identifying respective differences between one or more respective terms of respective sample expressions, corresponding to respective sample images indicated by a user as close to the user'"'"'s query concept, and corresponding respective disjunctive terms of the k-CNF expression;

determining which, if any, respective disjunctive terms of the k-CNF expression to remove from the k-CNF expression based upon the identified differences;

removing from the k-CNF expression respective disjunctive terms determined to be removed;

wherein refining the user query concept sample space further includes, refining the k-DNF expression by,

identifying respective differences between one or more respective terms of respective sample expressions, corresponding to respective sample images indicated by a user as not close to the user'"'"'s query concept, and corresponding respective conjunctive terms of the k-DNF expression;

determining which, if any, respective conjunctive terms of the k-DNF to remove from the k-DNF expression based upon the identified differences; and

removing from the k-DNF expression respective conjunctive terms determined to be removed.

2. The method of claim 1 further including:

removing respective sample images presented to the user from eligibility for presentation to that same user.

3. The method of claim 1 further including:

repeating the steps involved in refining the user query concept sample space.

4. The method of claim 1 further including:

repeating the steps involved in refining the user query concept sample space until the k-DNF expression becomes identical to or more specific than the k-CNF expression.

5. The method of claim 1 further including:

repeating the steps involved in refining the user query concept sample space until the user ends search.

6. The method of claim 1 further including:

dividing the k-CNF into multiple sub-group k-CNF expressions by separating respective disjunctive terms that can express each other'"'"'s feature information into different sub-group k-CNF expressions such that such separation of disjunctive terms does not result in loss of combinations of feature information due to such dividing;

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective disjunctive terms of the k-CNF expression involves identifying respective differences between respective terms of one or more sample expressions, corresponding to respective sample images indicated by a user as close to the user'"'"'s query concept, and corresponding respective disjunctive terms of respective sub-group k-CNF expressions; and

wherein removing from the k-CNF expression respective disjunctive terms involves removing from respective sub-group k-CNF expressions respective disjunctive terms based on respective identified differences.

7. The method of claim 1 further including:

dividing the k-CNF into multiple sub-group k-CNF expressions by separating respective disjunctive terms that can express each other'"'"'s feature information into different sub-group k-CNF expressions such that such separation of disjunctive terms does not result in loss of combinations of feature information due to such dividing;

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective disjunctive terms of the k-CNF expression involves identifying respective differences between respective terms of one or more sample expressions, corresponding to respective sample images indicated by a user as close to the user'"'"'s query concept, and corresponding respective disjunctive terms of respective sub-group k-CNF expressions; and

wherein removing from the k-CNF expression respective disjunctive terms involves removing from respective sub-group k-CNF expressions respective disjunctive terms based on respective identified differences;

dividing the k-DNF expression into multiple sub-group k-DNF expressions by separating respective conjunctive terms that can express each other'"'"'s feature information into different sub-group k-DNF expressions such that such separation of conjunctive terms does not result in loss of combinations of feature information due to such dividing;

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective conjunctive terms of the k-DNF expression involves identifying respective differences between respective terms of one or more sample expressions, corresponding to respective sample images indicated by a user as not close to the user'"'"'s query concept, and corresponding respective conjunctive terms of respective sub-group k-DNF expressions; and

wherein removing from the k-DNF expression respective conjunctive terms involves removing from respective sub-group k-DNF expressions respective conjunctive terms based on respective identified differences.

8. The method of claim 1,

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective disjunctive terms of the k-CNF expression includes,

testing respective sample expression terms for respective levels of difference from corresponding respective disjunctive terms of the k-CNF expression in a prescribed order such that, for a respective given feature, a respective term representing higher resolution of such given respective feature is tested before a respective term representing a lower resolution of such given respective feature;

and

not testing such respective term representing the lower resolution of such given respective feature if the testing of the respective term representing the higher resolution of such given respective feature indicates that there is a level of difference larger than a prescribed level between such respective expression term representing the higher resolution and the respective corresponding disjunctive term of the k-CNF expression representing the higher resolution of such given respective feature.

9. The method of claim 1,

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective disjunctive terms of the k-CNF expression includes,

testing respective sample expression terms for respective levels of difference from corresponding respective disjunctive terms of the k-CNF expression in a prescribed order such that, for a respective given feature, a respective term representing higher resolution of such given respective feature is tested before a respective term representing a lower resolution of such given respective feature;

and

not testing such respective term representing the lower resolution of such given respective feature if the testing of the respective term representing the higher resolution of such given respective feature indicates that there is a level of difference that is larger than a prescribed level between such respective expression term representing the higher resolution and the respective corresponding disjunctive term of the k-CNF expression representing the higher resolution of such given respective feature; and

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective conjunctive terms of the k-DNF expression includes,

testing respective sample expression terms for respective levels of difference from corresponding respective conjunctive terms of the k-DNF expression in a prescribed order such that, for a respective given feature, a respective term representing higher resolution of such given respective feature is tested before a respective term representing a lower resolution of such given respective feature; and

not testing such respective term representing the lower resolution of such given respective feature if the testing of the respective term representing the higher resolution of such given respective feature indicates that there is a level of difference that is smaller than a prescribed level between such respective expression term representing the higher resolution and the respective corresponding conjunctive term of the k-DNF expression representing the higher resolution of such given respective feature.

10. The method of claim 1,

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective disjunctive terms of the k-CNF includes,

measuring respective differences between respective values of respective disjunctive terms of the k-CNF expression and respective values of corresponding respective terms of sample expressions for respective sample images indicated by a user as close to the user'"'"'s query concept;

and

removing from the k-CNF respective disjunctive terms for which there are more than a prescribed threshold number of sample expressions for which corresponding respective measured value differences are greater than a prescribed threshold value difference.

11. The method of claim 1,

wherein identifying respective differences between terms predicates of one or more sample expressions and corresponding respective disjunctive terms of the k-CNF includes,

measuring respective differences between respective values of respective disjunctive terms of the k-CNF expression and respective values of corresponding respective terms of sample expressions for respective sample images indicated by a user as close to the user'"'"'s query concept;

and

removing from the k-CNF respective disjunctive terms for which there are more than a prescribed threshold number of sample expressions for which corresponding respective measured value differences are greater than a prescribed threshold value difference; and

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective conjunctive terms of the k-DNF includes,

measuring respective differences between respective values of respective conjunctive terms of the k-DNF expression and respective values of corresponding respective terms of sample expressions for respective sample images indicated by a user as not close to the user'"'"'s query concept; and

removing from the k-DNF respective conjunctive terms for which there are more than a prescribed threshold number of sample expressions for which corresponding respective measured value differences are less than a prescribed threshold value difference.

12. The method of claim 1,

wherein selecting multiple sample images from the user query concept sample space includes,

selecting respective sample images that correspond to respective sample expressions for which ψ

terms in respective corresponding sample expressions contradict the k-CNF expression;

wherein,

ψ

=1/ln(1/1−

p), and

wherein p represents a probability that a given disjunctive term of the k-CNF expression will be removed from the k-CNF expression in the step of removing from the k-CNF expression respective disjunctive terms.

13. The method of claim 1,

wherein selecting multiple sample images from the user query concept sample space includes,

selecting respective sample images that correspond to respective sample expressions that have a prescribed number of respective terms that contradict corresponding respective terms of the k-CNF expression;

wherein the prescribed number is determined empirically by balancing a need for a prescribed number that is small enough that the selected sample images are likely to be indicated by the user as being close to the user'"'"'s query concept with a need for a prescribed number that is large enough that the there is likely to be at least one set of multiple respective sample images that correspond to a set of multiple respective sample expressions that contradict the k-CNF expression in the same term.

14. The method of claim 1,

wherein defining the user query concept sample space includes,

selecting an initial set of sample images by choosing at least one sample image from each of multiple pre-clustered sets of sample images.

15. The method of claim 1,

wherein selecting multiple sample images from within the user query concept sample space includes,

respectively selecting images that correspond to respective sample expressions that have a prescribed number of respective terms that contradict corresponding respective terms of the k-CNF expression;

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective disjunctive terms of the k-CNF expression includes,

determining which respective terms of the k-CNF expression contradict corresponding respective terms of more than a prescribed number of sample expressions; and

wherein removing from the k-CNF expression respective disjunctive terms includes,

removing from the k-CNF expression respective disjunctive terms that contradict corresponding respective terms of more than the prescribed number of sample expressions indicated by a user as close to the user'"'"'s query concept.

16. The method of claim 1,

wherein selecting multiple sample images from within the user query concept sample space includes,

respectively selecting images that correspond to respective sample expressions that have a prescribed number of respective terms that contradict corresponding respective terms of the k-CNF expression;

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective disjunctive terms of the k-CNF expression includes,

determining which respective terms of the k-CNF expression contradict corresponding respective terms of more than a prescribed number of sample expressions indicated by a user as close to the user'"'"'s query concept; and

wherein removing from the k-CNF expression respective disjunctive terms includes,

removing from the k-CNF expression respective disjunctive terms that contradict corresponding respective terms of more than the prescribed number of sample expressions indicated by the user as close to the user'"'"'s query concept; and

wherein identifying respective differences between respective terms of one or more sample expressions and corresponding respective conjunctive predicates of the k-DNF expression includes,

determining which respective terms of the k-DNF expression do not contradict corresponding respective terms of more than a prescribed number of sample expressions indicated by the user as not close to the user'"'"'s query concept; and

wherein removing from the k-DNF expression respective conjunctive terms includes,

removing from the k-DNF expression respective conjunctive terms that do not contradict corresponding respective terms of more than the prescribed number of sample expressions indicated by a user as not close to the user'"'"'s query concept.

17. The method of claim 1,

wherein identifying respective differences between respective terms of each one or more sample expressions and corresponding respective disjunctive terms of the k-CNF expression involves, measuring respective levels of difference between respective terms of one or more sample expressions, corresponding to respective sample images indicated by a user as close to the user'"'"'s query concept, and corresponding respective disjunctive terms of the k-CNF expression;

wherein determining which, if any, respective disjunctive terms to remove from the k-CNF expression involves identifying which, if any, k-CNF disjunctive terms have measured levels of difference from corresponding expression terms of one or more images, that meet a prescribed threshold for disjunctive term removal;

wherein removing from the k-CNF expression respective disjunctive terms determined to be removed involves removing respective disjunctive terms with measured levels of difference that meet the prescribed threshold for disjunctive term removal;

wherein identifying respective differences between respective terms of each one or more sample expressions and corresponding respective conjunctive terms of the k-DNF expression involves, measuring respective levels of difference between respective terms of one or more sample expressions, corresponding to respective sample images indicated by a user as not close to the user'"'"'s query concept, and corresponding respective conjunctive terms of the k-DNF expression;

wherein determining which, if any, respective conjunctive terms to remove from the k-DNF expression involves identifying which, if any, k-DNF conjunctive terms have measured levels of difference from corresponding expression terms of one or more images, that meet a prescribed threshold for removal of conjunctive terms; and

wherein removing from the k-DNF expression respective conjunctive terms determined to be removed involves removing respective conjunctive terms with measured levels of difference that meet the prescribed threshold for conjunctive term removal.

18. The method of claim 1,

wherein determining which, if any, disjunctive terms of the k-CNF expression to remove from the k-CNF expression based upon the identified differences involves,

determining which, if any, k-CNF disjunctive terms and corresponding terms of one or more images differ by at least a prescribed amount;

and

wherein determining which, if any, respective conjunctive terms of the k-DNF expression to remove from the k-DNF expression based upon the identified differences involves,

determining which, if any, k-DNF conjunctive terms and corresponding terms of one or more images differ by no more than a prescribed amount.

19. The method of claim 1,

wherein each disjunctive term comprises one or more predicates;

and

wherein each conjunctive term comprises one or more predicates.

20. The method of claim 1,

wherein each disjunctive term comprises one or more predicates;

wherein each conjunctive term comprises one or more predicates; and

wherein each respective predicate corresponds to a respective image feature.

21. The method of claim 1,

wherein each respective predicate corresponds to a respective image feature.

22. A method of learning user query concept for searching visual images encoded in computer readable storage media comprising:

providing a multiplicity of respective sample images encoded in a computer readable medium;

providing a multiplicity of respective sample expressions encoded in computer readable medium that respectively correspond to respective sample images and in which respective terms of such respective sample expressions represent respective features of corresponding sample images;

defining a user query concept sample space by initially designating an initial set of sample images with at least one sample image from each of multiple pre-clustered sets of sample images as an initial user query concept sample space and by defining a k-CNF expression and a k-DNF expression which, together, encompass an initial set of sample expressions that correspond respectively to the sample images of the initial set of sample images;

wherein the k-CNF expression designates a more specific concept within the user query concept sample space; and

wherein the k-DNF expression designates a more general concept within the user query concept sample space;

refining the user query concept sample space by,

selecting multiple sample images from within the user query concept sample space that correspond to respective sample expressions that have a prescribed number of respective terms that contradict corresponding respective terms of the k-CNF expression;

resenting the multiple selected sample images to the user;

soliciting user feedback as to which of the multiple presented sample images are close to the user'"'"'s query concept;

wherein refining the user query concept sample space further includes, refining the k-CNF expression by,

identifying respective terms of respective sample expressions that contradict corresponding respective disjunctive terms of the k-CNF expression for those respective sample expressions corresponding to respective sample images indicated by the user as close to the user'"'"'s query concept;

determining which, if any, respective disjunctive terms of the k-CNF expression identified as contradicting corresponding respective terms of sample expressions indicated by the user as close to the user'"'"'s query concept, contradict corresponding respective terms of more than a prescribed number of such sample expressions;

removing from the k-CNF expression respective disjunctive terms that contradict corresponding respective terms of more than the prescribed number of sample expressions;

wherein refining the user query concept sample space further includes, refining the k-DNF expression by,

identifying respective terms of respective sample expressions that do not contradict corresponding respective conjunctive terms of the k-DNF expression for those respective sample expressions corresponding to respective sample images indicated by the user as not close to the user'"'"'s query concept;

determining which, if any, respective conjunctive terms of the k-DNF expression identified as not contradicting corresponding respective terms of sample expressions indicated by the user as not close to the user'"'"'s query concept, do not contradict corresponding respective terms of more than a prescribed number of such sample expressions;

removing from the k-DNF expression respective conjunctive terms that do not contradict corresponding respective predicates of more than the prescribed number of sample expressions; and

repeating the steps involved in refining the user query concept sample space.

23. The method of claim 22 wherein repeating the steps involved in refining the user query concept sample space involves repeating until the user ends the search.

24. The method of claim 22 further including

repeating the steps involved in refining the user query concept sample space until the k-DNF expression becomes identical to or more specific than the k-CNF expression.