Retrieval and browsing of database images based on image emphasis and appeal

• US 6,847,733 B2
• Filed: 05/23/2001
• Issued: 01/25/2005
• Est. Priority Date: 05/23/2001
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Savakis, Andreas E., Mehrotra, Rajiv
• Primary Examiner(s): Do, Anh Hong
• Application Number:
  US09/863,570
• Publication Number:
  US 20030048950A1
• Time in Patent Office: 1,343 Days
• Field of Search: 382/203, 382/103, 382/130, 382/305, 382/292, 382/187, 382/225, 358/527, 705/1, 707/3, 707/5, 707/104.1
• US Class Current: 382/225
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

An image is automatically assessed with respect to certain features, wherein the assessment is a determination of the degree of importance, interest or attractiveness of the image. First, a digital image is obtained corresponding to the image. Then one or more quantities are computed that are related to one or more features in the digital image, including one or more features pertaining to the content of the digital image. The quantities are processed with a reasoning algorithm that is trained on the opinions of one or more human observers, and an output is obtained from the reasoning algorithm that assesses the image. More specifically, the reasoning algorithm is a Bayesian network that provides a score which, when done for a group of images, selects one image as the emphasis image or the appeal image. The features pertaining to the content of the digital image include people-related features and/or subject-related features. Moreover, additional quantities may be computed that relate to objective measures of the digital image, such as colorfulness and/or sharpness.

Claims

1. A method for automatically organizing and retrieving images from an image database comprising:

(a) clustering images within said database; and

(b) ranking images within said clusters based upon image emphasis or image appeal,

wherein said ranking includes;

(1) computing one or more quantities related to one or more features in each image, said features including a content of said images;

(2) processing said quantities with a reasoning algorithm; and

(3) selecting one image with a highest rank as an emphasis image or an appeal image.

2. The method as claimed in claim 1, wherein said features include at least one of people-related features and subject-related features.

3. The method as claimed in claim 1, wherein said computing further includes computing one or more quantities related to one or more objective features pertaining to objective measures of the digital image.

4. The method as claimed in claim 3, wherein said objective features include at least one of colorfulness and sharpness.

5. The method as claimed in claim 3, wherein said objective features include a representative quality in terms of color content.

6. The method as claimed in claim 3, wherein said objective features include a uniqueness of picture aspect format.

7. The method as claimed in claim 1, wherein said reasoning algorithm is trained from ground truth studies of candidate images.

8. The method as claimed in claim 1, wherein the reasoning algorithm comprises a Bayesian network.

9. The method as claimed in claim 1, wherein said clustering is based upon one or more of color and texture.

10. The method as claimed in claim 1, further comprising, before said clustering, performing an initial image emphasis process comprising said steps (1)-(2), wherein said clustering is based at least partially upon results of said initial image emphasis process.

11. The method as claimed in claim 1, further comprising:

(c) selecting a highest ranking emphasis image or appeal image group based on said ranking;

(d) repeating said (a) clustering and said (b) ranking of images within said database using said highest ranking emphasis or appeal image group as a query.

12. The method as claimed in claim 1, wherein said clustering produces multiple clusters of said images and said selecting produces a representative image for each of said clusters.

13. A method automatically organizing and retrieving images from an image database comprising:

(a) clustering images within said database into image groups;

(b) selectively clustering ones of said groups into subgroups to produce a hierarchical tree of said groups and subgroups; and

(c) ranking images within said groups and said subgroups based upon image emphasis or image appeal

wherein said ranking includes;

(1) computing one or more quantities related to one or more features in each image, said features including a content of said images;

(2) processing said quantities with a reasoning algorithm that is trained based on opinions of one or more human observers; and

(3) selecting one image with a highest rank as an emphasis image or appeal image for said group or said subgroup.

14. The method as claimed in claim 13, wherein said process of selectively clustering said groups into subgroups is based upon a size of said groups.

15. The method as claimed in claim 13, further comprising selectively clustering said subgroups into additional subgroups based upon a size of said subgroups.

16. The method as claimed in claim 13, wherein said features include at least one of people-related features and subject-related features.

17. The method as claimed in claim 13, wherein said computing further includes computing one or more quantities related to one or more objective features pertaining to objective measures of the digital image.

18. The method as claimed in claim 17, wherein said objective features include at least one of colorfulness and sharpness.

19. The method as claimed in claim 17, wherein said objective features include a representative quality in terms of color content.

20. The method as claimed in claim 17, wherein said objective features include a uniqueness of picture aspect format.

21. The method as claimed in claim 13, wherein said reasoning algorithm is trained from ground truth studies of candidate images.

22. The method as claimed in claim 13, wherein the reasoning algorithm comprises a Bayesian network.

23. The method as claimed in claim 13, wherein said clustering is based upon one or more of color and texture.

24. The method as claimed in claim 13, further comprising, before said clustering of said images, performing an initial image emphasis process comprising said steps (1)-(2), wherein said clustering is based at least partially upon results of said initial image emphasis process.

25. The method as claimed in claim 13, further comprising:

(d) selecting a highest ranking emphasis or appeal image group based on said ranking;

(e) performing said (a) clustering, said (b) selectively clustering, and said (b) ranking of images within said database using said highest ranking emphasis or appeal image group as a query.

26. The method as claimed in claim 13, wherein said clustering produces multiple clusters of said images and said selecting produces a representative image for each of said clusters.

27. A computer program product for automatically organizing and retrieving images from an image database, said program product comprising:

a computer readable storage medium having a computer program stored thereon for performing a method of;

(a) clustering images within said database; and

(b) ranking images within said clusters based upon image emphasis or image appeal,

wherein said ranking includes;

(1) computing one or more quantities related to one or more features in each image, said features including a content of said images;

(2) processing said quantities with a reasoning algorithm; and

(3) selecting one image with a highest rank as an emphasis image or appeal image.

28. The computer program product as claimed in claim 27, wherein said features include at least one of people-related features and subject-related features.

29. The computer program product as claimed in claim 27, wherein said computing further includes computing one or more quantities related to one or more objective features pertaining to objective measures of the digital image.

30. The computer program product as claimed in claim 27, wherein said objective features include a uniqueness of picture aspect format.

31. The computer program product as claimed in claim 27, wherein said reasoning algorithm is trained from ground truth studies of candidate images.

32. The computer program product as claimed in claim 27, wherein the reasoning algorithm comprises a Bayesian network.

33. The computer program product as claimed in claim 27, wherein said clustering is based upon one or more of color and texture.

34. The computer program product as claimed in claim 27, further comprising, before said clustering, performing an initial image emphasis process comprising said steps (1)-(2), wherein said clustering is based at least partially upon results of said initial image emphasis process.

35. The computer program product as claimed in claim 27, further comprising:

(c) selecting a highest ranking emphasis or appeal image group based on said ranking;

(d) repeating said (a) clustering and said (b) ranking of images within said database using said highest ranking emphasis or appeal image group as a query.

36. The computer program product as claimed in claim 27, wherein said clustering produces multiple clusters of said images and said selecting produces a representative image for each of said clusters.

37. The computer program product as claimed in claim 27, wherein said objective features include at least one of colorfulness and sharpness.

38. The computer program product as claimed in claim 27, wherein said objective features include a representative quality in terms of color content.