STUDYING AESTHETICS IN PHOTOGRAPHIC IMAGES USING A COMPUTATIONAL APPROACH

• US 20080285860A1
• Filed: 05/07/2008
• Published: 11/20/2008
• Est. Priority Date: 05/07/2007
• Status: Abandoned Application

• Current Assignee: Penn State Research Foundation
• Original Assignee: Penn State Research Foundation
• Inventors: Li, Jia, Wang, James Z., Datta, Ritendra
• Application Number:
  US12/116,578
• Publication Number:
  US 20080285860A1
• Time in Patent Office: Days
• US Class Current: 382/224
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06F 18/24155 Bayesian classification
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/766 using regression, e.g. by projecting features on hyperplanes
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60

Abstract

The aesthetic quality of a picture is automatically inferred using visual content as a machine learning problem using, for example, a peer-rated, on-line photo sharing Website as data source. Certain visual features of images are extracted based on the intuition that they can discriminate between aesthetically pleasing and displeasing images. A one-dimensional support vector machine is used to identify features that have noticeable correlation with the community-based aesthetics ratings. Automated classifiers are constructed using the support vector machines and classification trees, with a simple feature selection heuristic being applied to eliminate irrelevant features. Linear regression on polynomial terms of the features is also applied to infer numerical aesthetics ratings.

Claims

1. A computer-based method of inferring and utilizing the aesthetic quality of photographs and other images, comprising the steps of:

providing a collection of previously digitized images and a digitized image to be evaluated;

performing one or more software operations on the digitized image to automatically extract a plurality of visual features representative of the image; and

using one or more of the visual features to classify the image on the basis of its aesthetic value, rate the image on a scale relating to its aesthetics value, or select/eliminate the image as being aesthetically distinctive when compared to the collection.

2. The method of claim 1, including the steps of:

receiving a plurality of digitized images along with aesthetic-based ratings of the images provided by viewers of the images;

performing one or more software operations on the images to identify those features which correlate to the aesthetic-based ratings provided by the viewers;

conceiving, testing, and isolating the features which correlate with these ratings, in order to classify the image, automatically rate the image, select or eliminate the image from a collection, all based on its perceived aesthetic value.

3. The method of claim 2, wherein the correlated visual features are identified using a one-dimensional and multi-dimensional support vector machines.

4. The method of claim 1, further including the step of converting the image into the HSV color space to produce two-dimensional matrices IH, IS and IV.

5. The method of claim 1, further including the step of segmenting the object to identify objects in the image.

6. The method of claim 1, wherein the feature is exposure.

7. The method of claim 1, wherein the feature is color distribution.

8. The method of claim 1, wherein the feature is saturation or hue.

9. The method of claim 1, wherein the feature is based upon the rule of thirds.

10. The method of claim 1, wherein the feature is based upon viewer familiarity or identifiability with the image composition, relating to its originality.

11. The method of claim 1, wherein the feature is graininess or smoothness.

12. The method of claim 1, wherein the feature is image size or aspect ratio.

13. The method of claim 1, wherein the feature is photographic depth-of-field.

14. The method of claim 1, wherein the feature is related to shape regularity of segments within the image.

15. The method of claim 1, further including the step of performing a linear regression on one or more of the features to infer the aesthetics of the image.

16. The method of claim 1, further including the use of a naive Bayes classifier to classify the image based on its aesthetic value.

17. The method of claim 1, wherein linear regression and naive Bayes classification methods are used in conjunction to select high-quality images or eliminate low-quality images from image collections, on the basis of their aesthetic values.

18. The method of claim 1, wherein the step of using one or more of the visual features includes the employment of a content-based image retrieval algorithm.

19. The method of claim 1, wherein the step of using one or more of the visual features includes a composition suggestion within a digital photography system for enhancing the aesthetic value of capture images.

20. The method of claim 1, wherein the step of using one or more of the visual features includes discriminating between visually similar images.