Automatically generating a content-based quality metric for digital images

• US 7,826,657 B2
• Filed: 12/11/2006
• Issued: 11/02/2010
• Est. Priority Date: 12/11/2006
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Shakya, Subodh, Sarukkai, Ramesh R., Zhang, Ruofei
• Primary Examiner(s): Dang; Duy M
• Application Number:
  US11/637,422
• Publication Number:
  US 20080136834A1
• Time in Patent Office: 1,422 Days
• Field of Search: 382/103, 382/162, 382/224, 382/236, 382/305, 345/589
• US Class Current: 382/162
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

Techniques are described herein for automatically evaluating the quality of digital images based on one or more color characteristics of the images. In some embodiments, a quality metric that indicates the likelihood that the digital images convey semantics is generated based on color characteristics of the digital images. The quality metric may be used, for example, to determine which keyframe to use to make a thumbnail to represent video data. In some embodiments, feature values are generated for an image based on color characteristics of the image, and the feature values are assigned to bins. In such embodiments, the quality metric may be generated to indicate how uniform the distribution of feature values is among the bins.

Claims

1. A method for measuring quality of an electronic image, the method comprising the computer-implemented steps of:

generating a plurality of feature values based on color attributes of the electronic image; and

automatically determining a quality metric for the electronic image based, at least in part, on how uniformly the feature values are distributed among a plurality of bins, wherein each bin corresponds to a value range;

wherein the method is performed by one or more computing devices.

2. The method of claim 1 wherein the plurality of feature values are based, at least in part, on spatial information relating to pixels of said electronic image.

3. The method of claim 1 wherein the step of generating a plurality of feature values includes generating feature values that reflect luminancy of pixels within said electronic image.

4. The method of claim 3 wherein the step of generating a plurality of feature values includes generating, for each pixel of a plurality of pixels in said electronic image, a per-pixel feature value that reflects luminancy of the pixel.

5. The method of claim 1 wherein the step of generating a plurality of feature values includes generating, for each pixel of a plurality of pixels in said electronic image, a per-pixel feature value that reflects one or more color characteristics of the pixel.

6. The method of claim 1 wherein the step of generating a plurality of feature values includes generating, for each pixel group of a plurality of pixel groups in said electronic image, a per-pixel-group feature value that reflects one or more color characteristics of the pixel group.

7. The method of claim 1 wherein the step of automatically determining a quality metric for the electronic image includes:

identifying, from among said plurality of bins, a dominant bin that is assigned more feature values than any other of said plurality of bins;

determining whether the percentage of feature values assigned to the dominant bin exceeds a first predetermined threshold; and

if the percentage of feature values assigned to the dominant bin exceeds the first predetermine threshold, then determining that the electronic image does not contain sufficient semantic information.

8. The method of claim 7 wherein the step of automatically determining a quality metric for the electronic image includes:

in response to determining that the percentage of feature values assigned to the dominant bin does not exceed the first predetermined threshold, performing the steps of;

determining whether the percentage of feature values assigned to the dominant bin exceeds a second predetermined threshold;

determining an entropy value for said electronic image;

determining whether the entropy value for said electronic image exceeds a third predetermined threshold; and

if the percentage of feature values assigned to the dominant bin exceeds the second predetermine threshold and the entropy value for the image is below a third predetermined threshold, then determining that the image does not contain sufficient semantic information.

9. The method of claim 8 wherein the step of determining an entropy value includes determining an entropy value based on one or more probabilities of pixels that are associated with all bins, of said plurality of bins, other than said dominant bin.

10. The method of claim 1 further comprising the steps of:

selecting a keyframe from a video, wherein the electronic image is based on the keyframe; and

based on the quality metric for the electronic image, determining whether to use said electronic image as a thumbnail for said video.

11. The method of claim 10, further comprising:

in response to determining that said electronic image should not be used as the thumbnail for said video, performing the steps of;

selecting another keyframe from said video, wherein the other keyframe corresponds to a second electronic image;

generating a second plurality of feature values based on color attributes of the second electronic image;

automatically determining a second quality metric for the second image based, at least in part, on how uniformly the feature values are distributed among the plurality of bins; and

determining whether the second electronic image should be used as the thumbnail for said video based on the second quality metric.

12. The method of claim 1 further comprising:

performing the following steps for each image of a plurality of electronic images, wherein the plurality of electronic images includes said electronic image;

generating a plurality of feature values based on color attributes of said each electronic image,

automatically determining a quality metric for said each image based, at least in part, on how uniformly the feature values are distributed among a plurality of bins, wherein each bin corresponds to a value range, and

automatically determining whether to perform an operation on said each image based on the quality metric determined for said each image.

13. The method of claim 12 wherein:

the plurality of electronic images are captured from a surveillance system; and

the step of automatically determining whether to perform an operation on said each image includes automatically determining whether to discard each said image.

14. The method of claim 1 further comprising:

performing the following steps for each image of a plurality of electronic images, wherein the plurality of electronic images includes said electronic image;

generating a plurality of feature values based on color attributes of said each electronic image, and

automatically determining a quality metric for said each image based, at least in part, on how uniformly the feature values are distributed among a plurality of bins, wherein each bin corresponds to a value range; and

presenting said plurality of images in an order that is based, at least in part, on the quality metric determined for said each image.

15. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 1.

16. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 2.

17. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 3.

18. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 4.

19. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 5.

20. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 6.

21. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 7.

22. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 8.

23. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 9.

24. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 10.

25. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 11.

26. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 12.

27. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 13.

28. One or more non-transitory machine readable storage storing instructions which, when executed by one or more processors, causes performance of the method recited in claim 14.