Method and computer program product for subjective image content similarity-based retrieval

• US 6,480,840 B2
• Filed: 08/13/2001
• Issued: 11/12/2002
• Est. Priority Date: 06/29/1998
• Status: Expired due to Term

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Zhu, Wei, Mehrotra, Rajiv
• Primary Examiner(s): Coby, Frantz
• Application Number:
  US09/928,735
• Publication Number:
  US 20010056415A1
• Time in Patent Office: 456 Days
• Field of Search: 707/5, 707/3, 707/6, 707/104.1, 707/2, 707/4, 707/102, 707/103.R, 382/305, 345/326, 706/20, 705/5
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06Q 10/02 Reservations, e.g. for tickets, services or events
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure

Abstract

A user preference for a desired image is determined by using one or more examples and counterexamples of a desired image in order to define the user preference. A relative preference of a user for one or more image components or one or more depictive features is automatically extracted from the examples and counterexamples of the desired image. Then, a user subjective definition of a desired image is formulated using the relative preferences for either the image components or the depictive features.

Claims

1. A method for generating a preference record indicating a user preference for a desired image, the method comprising the steps of:

(a) selecting a query image, wherein the query image is characterized by a query image representation including one or more query image components;

(b) providing one or more examples and counterexamples of said desired image for deriving a user preference;

(c) automatically analyzing said one or more examples and counterexamples to estimate user preference by automatically extracting a relative preference of a user for either one or more image components or one or more depictive features from the examples and counterexamples of said desired image;

(d) formulating a user-derived preference related to said desired image using the relative preferences for the image components and the depictive features; and

(e) generating a preference record from the query image representation and the user-derived preference, whereby the relative preferences automatically estimated from the examples and counterexamples extends the representation of the query image for a specific user preference.

2. The method as claimed in claim 1 wherein the query image representation is a predetermined multi-level hierarchical image representation.

3. The method as in claim 2 wherein the query image representation includes a color-based representation, a shape-based representation or a texture-based representation.

4. The method as in claim 1 wherein the examples comprise image representations including one or more image components and the query image is a pseudo-query image generated by the union of all the image representations of the examples.

5. The method as in claim 1, wherein step (c) further comprises identifying similar and dissimilar image components among examples and counterexamples of said desired image.

6. The method as in claim 1, wherein an image component can be either an entire image or a segment of an image.

7. The method as in claim 6, wherein step (c) further comprises identifying similar and dissimilar depictive features of either the image components or the image among the examples and counterexamples of said desired image.

8. The method as in claim 7, wherein depictive features include color, color composition, texture, structure, or shape.

9. The method as in claim 1, wherein step (c) further comprises computing the relative preference for an image component based on a frequency of occurrence of the image component in the examples and counter examples of said desired image.

10. The method as in claim 1, wherein step (c) further comprises computing the relative preference for an image component based on a similarity of the image components among examples and counter examples of said desired image.

11. The method as in claim 1, wherein step (c) further comprises computing the relative preference for a depictive feature based on a frequency of occurrence of the feature in the examples and counter examples of said desired image.

12. The method as in claim 1, wherein step (c) further comprises computing the relative preference for a depictive feature based on a similarity of the depictive feature among examples and counter examples of said desired image.

13. The method as in claim 1, wherein step (d) of formulating the user-derived preference includes formulating a list of either desired or undesired image components and the relative preferences of said components to the user.

14. The method as in claim 1, wherein step (d) of formulating the user-derived preference includes formulating a list of either desired or undesired depictive features of either an image component or an image and their relative preferences to the user.

15. The method as in claim 1, further comprising the step of storing the preference record.

16. A computer storage medium having instructions stored therein for causing a computer to perform the method of claim 1.

18. The method as claimed in claim 16 wherein the query image representation is a predetermined multi-level hierarchical image representation.

19. The method as in claim 18 wherein the query image representation includes a color-based representation, a shape-based representation or a texture-based representation.

20. The method as in claim 16 wherein the examples comprise image representations including one or more image components and the query image is a pseudo-query image generated by the union of all the image representations of the examples.

21. The method as in claim 16, wherein step (c) further comprises identifying similar and dissimilar image components among examples and counterexamples of said desired image.

22. The method as in claim 21, wherein said image component can be either an entire image or a segment of an image.

23. The method as in claim 22, wherein step (c) further comprises identifying similar and dissimilar depictive features of either image components or the image among the examples and counterexamples of said desired image.

24. The method as in claim 23, wherein depictive features include color, color composition, texture, structure, or shape.

25. The method as in claim 16, wherein step (c) further comprises computing the relative preference for an image component based on a frequency of occurrence of the image component in the examples and counter examples of said desired image.

26. The method as in claim 16, wherein step (c) further comprises computing the relative preference for an image component based on a similarity of the image components among examples and counter examples of said desired image.

27. The method as in claim 16, wherein step (c) further comprises computing the relative preference for a depictive feature based on a frequency of occurrence of the feature in the examples and counter examples of said desired image.

28. The method as in claim 16, wherein step (c) further comprises computing the relative preference for a depictive feature based on a similarity of the depictive feature among examples and counter examples of said desired image.

29. The method as in claim 16, wherein step (c) of formulating the definition of said desired image includes formulating a list of either desired or undesired image components and their relative preferences to the user.

30. The method as in claim 16, wherein step (c) of formulating the definition of said desired image includes formulating a list of either desired or undesired depictive features of either an image component or an image and their relative preferences to the user.

31. The method as in claim 16, wherein step (d) further comprises the step of ranking retrieved images based on the relative preference of either the image components or the depictive features.

32. The method as in claim 16 further comprising the step of displaying the retrieved images.

17. A method for retrieving user desired images from a database based on a user preference for a desired image, the method comprising the steps of:

(a) selecting a query image, wherein the query image is characterized by a query image representation including one or more query image components;

(b) applying the query image representation to identify and retrieve a candidate set of images from the database;

(c) formulating a user subjective definition of said desired image using either image components or depictive features from one or more examples and counterexamples of said desired image comprising;

(c1) automatically analyzing said one or more examples and counterexamples to estimate user preference by automatically extracting a relative preference of said user for either one or more image components or one or more depictive features from the examples and counterexamples of said desired images;

(d) applying the user subjective definition of said desired image to identify and retrieve said user desired images from the candidate set of images; and

(e) wherein the occurrence of a similar image component or depictive feature in an example and the query image increases the relative preference of the user and the occurrence of a similar image component or depictive feature in a counterexample and the query image decreases the relative preference of the user.

33. A computer storage medium having instructions stored therein for causing a computer to perform the method of claim 17.

34. A method for retrieving one or more desired images based on user preference for a desired image, said method comprising the steps of:

(a) providing one or more examples and counterexamples of a desired image for deriving a user preference;

(b) identifying an image representation from at least the examples;

(c) formulating a user subjective definition of said desired image using either image components or depictive features from said examples and counterexamples of said desired image comprising;

(c1) automatically analyzing said one or more examples and counterexamples to estimate user preference by automatically extracting a relative preference of said user for either one or more image components or one or more depictive features from the examples and counterexamples of said desired images, wherein the occurrence of a similar image component or depictive feature in an example and in the image representation increases the relative preference of the user and the occurrence of a similar image component or depictive feature in a counterexample and in the image representation decreases the relative preference of the user; and

(d) applying the user subjective definition of said desired image to identify and retrieve user desired images.

35. A computer storage medium having instructions stored therein for causing a computer to perform the method of claim 34.