STUDYING AESTHETICS IN PHOTOGRAPHIC IMAGES USING A COMPUTATIONAL APPROACH

• US 20130011070A1
• Filed: 07/05/2012
• Published: 01/10/2013
• Est. Priority Date: 05/07/2007
• Status: Active Grant

• Current Assignee: Penn State Research Foundation (Pennsylvania State University)
• Original Assignee: Penn State Research Foundation (Pennsylvania State University)
• Inventors: Datta, Ritendra, Li, Jia, Wang, James Z.
• Granted Patent: US 8,755,596 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06F 18/24155 Bayesian classification
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/766 using regression, e.g. by projecting features on hyperplanes
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60

Abstract

The aesthetic quality of a picture is automatically inferred using visual content as a machine learning problem using, for example, a peer-rated, on-line photo sharing Website as data source. Certain visual features of images are extracted based on the intuition that they can discriminate between aesthetically pleasing and displeasing images. A one-dimensional support vector machine is used to identify features that have noticeable correlation with the community-based aesthetics ratings. Automated classifiers are constructed using the support vector machines and classification trees, with a simple feature selection heuristic being applied to eliminate irrelevant features. Linear regression on polynomial terms of the features is also applied to infer numerical aesthetics ratings.

Claims

1. A computer-based method of inferring and utilizing the aesthetic quality of photographs and other images, comprising the steps of:

receiving a plurality of digitized images along with aesthetic-based ratings of the images provided by viewers of the images;

performing one or more software operations on the digitized images to automatically extract a plurality of visual features representative of each image; and

using one or more statistical methods to correlate the extracted visual features and the aesthetic-based viewer ratings to classify image on the basis of its aesthetic value, rate image on a scale relating to its aesthetics value, or select/eliminate image based upon aesthetic quality.

2. The method of claim 1, including the steps of:

receiving a new digitized image without an aesthetic-based rating;

performing one or more software operations on the image to automatically extract a plurality of visual features representative of the image;

determining the aesthetic quality of the image using the previously obtained correlation between extracted visual features and the aesthetic-based viewer ratings.

3. The method of claim 1, wherein the correlation is performed using a support vector machine.

4. The method of claim 1, further including the step of converting the image into the HSV color space to produce two-dimensional matrices IH, IS and IV.

5. The method of claim 1, further including the step of segmenting the object to identify objects in the image.

6. The method of claim 1, wherein the feature is exposure.

7. The method of claim 1, wherein the feature is color distribution.

8. The method of claim 1, wherein the feature is saturation or hue.

9. The method of claim 1, wherein the feature is based upon the rule of thirds.

10. The method of claim 1, wherein the feature is based upon viewer familiarity or identifiability with the image composition, relating to its originality.

11. The method of claim 1, wherein the feature is graininess or smoothness.

12. The method of claim 1, wherein the feature is image size or aspect ratio.

13. The method of claim 1, wherein the feature is photographic depth-of-field.

14. The method of claim 1, wherein the feature is related to shape regularity of segments within the image.

15. The method of claim 1, further including the step of performing a linear regression on one or more of the features to infer the aesthetics of the image.

16. The method of claim 1, further including the use of a naï

ve Bayes classifer to classify the image based on its aesthetic value.

17. The method of claim 1, wherein linear regression and naï

ve Bayes classification methods are used in conjunction to select high-quality images or eliminate low-quality images from an image collection.

18. The method of claim 1, wherein the step of using one or more of the visual features includes the employment of a content-based image retrieval algorithm.

19. The method of claim 1, wherein the step of using one or more of the visual features includes a composition suggestion within a digital photography system for enhancing the aesthetic value of capture images.

20. The method of claim 1, wherein the step of using one or more of the visual features includes discriminating between visually similar images.