Method and computer program product for subjective image content similarity-based retrieval

• US 6,345,274 B1
• Filed: 06/29/1998
• Issued: 02/05/2002
• Est. Priority Date: 06/29/1998
• Status: Expired due to Term

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Zhu, Wei, Mehrotra, Rajiv
• Primary Examiner(s): Coby, Frantz
• Application Number:
  US09/106,427
• Time in Patent Office: 1,317 Days
• Field of Search: 707/3, 707/6, 707/102, 707/104, 707/5, 382/209, 382/220, 382/305, 706/20, 345/326, 345/348
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06Q 10/02 Reservations, e.g. for tickets, services or events
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure

Abstract

A user preference for a desired image is determined by using one or more examples and counterexamples of a desired image in order to define the user preference. A relative preference of a user for one or more image components or one or more depictive features is automatically extracted from the examples and counterexamples of the desired image. Then, a user subjective definition of a desired image is formulated using the relative preferences for either the image components or the depictive features.

Claims

1. A method for learning a user preference for a desired image, the method comprising the steps of:

(a) using one or more examples and counterexamples of a desired image for defining a user preference;

(b) computing a relative preference of a user for either one or more image components or one or more depictive features based on a frequency of occurrence of the image component or depictive feature in the examples and counterexamples of a desired image; and

(c) formulating a user subjective definition of a desired image using the relative preference for either image components or depictive features.

2. The method as in claim 1, wherein step (b) further comprises identifying similar and dissimilar image components among examples and counterexamples of said desired image.

3. The method as in claim 2, where in an image component can be either an entire image or a segment of the image.

4. The method as in claim 1, wherein step (b) further comprises identifying similar and dissimilar depictive features of either the image components or the image among the examples and counterexamples of said desired image.

5. The method as in claim 4, wherein depictive features include color, color composition, texture, structure, and shape.

6. The method as in claim 1, wherein step (b) further comprises computing the relative preference for an image component based on a similarity of the image components among examples and counter examples of said desired image.

7. The method as in claim 1, wherein step (b) further comprises computing the relative preference for a depictive feature based on a similarity of the depictive feature among examples and counter examples of said desired image.

8. The method as in claim 1, wherein step (c) of formulating the definition of said desired image includes a list of either desired or undesired image components and the relative preferences of said components to the user.

9. The method as in claim 1, wherein step (c) of formulating the definition of said desired image includes a list of either desired or undesired depictive features of either an image component or an image and their relative preferences to the user.

10. The method as in claim 1, further comprising the step of storing the user subjective definition.

11. A method for retrieving user desired images comprising the steps of

(a) formulating a user subjective definition of a desired image using either image components or depictive features from one or more examples and counterexamples of a desired image comprising;

(a1) extracting a relative preference of a user for either an image component or a depictive feature based on the frequency of occurrence of the image component or depictive feature in the examples and counterexamples of a desired image; and

(b) applying the user subjective definition of a desired image to identify and retrieve a user desired image.

12. The method as in claim 11, wherein step (a) further comprises identifying similar and dissimilar image components among examples and counterexamples of said desired image.

13. The method as in claim 12, wherein said image component can be either an entire image or a segment of the image.

14. The method as in claim 11, wherein step (a) further comprises identifying similar and dissimilar depictive features of either image components or image among the examples and counterexamples of said desired image.

15. The method as in claim 14, wherein depictive features include color, color composition, texture, structure, and shape.

16. The method as in claim 11, wherein step (a) further comprises computing the relative preference for an image component based on a similarity of the image components among examples and counter examples of said desired image.

17. The method as in claim 11, wherein step (a) further comprises computing the relative preference for a depictive feature based on a similarity of the depictive feature among examples and counter examples of said desired image.

18. The method as in claim 11, wherein step (a) of formulating the definition of said desired image includes a list of either desired or undesired image components and their relative preferences to the user.

19. The method as in claim 11, wherein step (a) of formulating the definition of said desired image includes a list of either desired or undesired depictive features of either an image component or an image and their relative preferences to the user.

20. The method as in claim 11 wherein step (b) further comprises selecting images containing either one or more desired image components or one or more desired depictive features from a database.

21. The method as in claim 11 wherein step (b) comprises applying the user subjective definition of said desired image to a user specified query image to retrieve desired images.

22. The method as in claim 21, wherein step (b) comprises applying a subset of the user subjective definition of said desired image consisting of the relative preferences for either the image components or depictive features present in the query image.

23. The method as in claim 11, wherein step (b) further comprising the step of ranking retrieved images based on the relative preference of either the image components or the depictive features.

24. The method as in claim 11 further comprising the step of displaying the retrieved images.

25. A computer program product for learning a user preference for a desired image, comprising a computer readable storage medium having a computer program stored thereon for performing the steps of

(a) using one or more examples and counterexamples of a desired image for defining a user preference;

b) extracting a relative preference of a user for either one or more image components or one or more depictive features based on the frequency of occurrence of the image component or depictive feature in the examples and counterexamples of a desired image; and

(c) formulating a user subjective definition of a desired image using the relative preferences for either image components or depictive features.

26. The computer program product as in claim 25, wherein step (b) further comprises identifying similar and dissimilar image components among examples and/or counterexamples of said desired image.

27. The computer program product as in claim 26, wherein an image component can be either an entire image or a segment of the image.

28. The computer program product as in claim 25, wherein step (b) further comprises identifying similar and dissimilar depictive features of either the image components or the image among the examples and counterexamples of said desired image.

29. The computer program product as in claim 28, wherein depictive features include color, color composition, texture, structure, and shape.

30. The computer program product as in claim 25, wherein step (b) further comprises computing the relative preference for an image component based on a similarity of the image components among examples and counter examples of said desired image.

31. The computer program product as in claim 25, wherein step (b) further comprises computing the relative preference for a depictive feature based on a similarity of the depictive feature among examples and counter examples of said desired image.

32. The computer program product as in claim 25, wherein step (c) of formulating the definition of said desired image includes a list of either desired or undesired image components and their relative preferences to the user.

33. The computer program product as in claim 25, wherein step (c) of formulating the definition of said desired image includes a list of either desired or undesired depictive features of either an image component or an image and their relative preferences to the user.

34. The computer program product as in claim 25, further comprising the step of storing the user subjective definition.

35. A computer program product for retrieving user desired images comprising the steps of

(a) formulating a user subjective definition of a desired image using either image components or depictive features from one or more examples and counterexamples of a desired image comprising;

(a1) extracting a relative preference of a user for either an image component or a depictive feature based on the frequency of occurrence of the image component or depictive feature in the examples and counterexamples of a desired image; and

(b) applying the user subjective definition of a desired image to identify and retrieve the user desired images.

36. The computer program product as in claim 35, wherein step (a) further comprises identifying similar and dissimilar image components among examples and counterexamples of said desired image.

37. The computer program product as in claim 36, wherein an image component can be either an entire image or a segment of the image.

38. The computer program product as in claim 35, wherein step (a) further comprises identifying similar and dissimilar depictive features of either image components or image among the examples and counterexamples of said desired image.

39. The computer program product as in claim 38, wherein depictive features include color, color composition, texture, structure, and shape.

40. The computer program product as in claim 35, wherein step (a) further comprises computing the relative preference for an image component based on a similarity of the image components among examples and counter examples of said desired image.

41. The computer program product as in claim 35, wherein step (a) further comprises computing the relative preference for a depictive feature based on a similarity of the depictive feature among examples and counter examples of said desired image.

42. The computer program product as in claim 35, wherein step (a) of formulating the definition of said desired image includes a list of either desired or undesired image components and their relative preferences to the user.

43. The computer program product as in claim 35, wherein step (a) of formulating the definition of said desired image includes a list of either desired or undesired depictive features of either an image component or an image and their relative preferences to the user.

44. The computer program product as in claim 35 wherein step (b) further comprises selecting images containing either one or more desired image components or one or more desired depictive features from a database.

45. The computer program product as in claim 35 wherein step (b) comprises applying the user subjective definition of said desired image to a user specified query image to retrieve desired images.

46. The computer program product as in claim 45, wherein step (b) comprises applying a subset of the user subjective definition of said desired image consisting of the relative preferences for either the image components or depictive features present in the query image.

47. The computer program product as in claim 35, wherein step (b) further comprising the step of ranking retrieved images based on the relative preference of either the image components or the depictive features.

48. The computer program product as in claim 35 further comprising the step of displaying the retrieved images.