Studying aesthetics in photographic images using a computational approach

• US 8,755,596 B2
• Filed: 07/05/2012
• Issued: 06/17/2014
• Est. Priority Date: 05/07/2007
• Status: Active Grant

• Current Assignee: Penn State Research Foundation
• Original Assignee: Penn State Research Foundation
• Inventors: Datta, Ritendra, Li, Jia, Wang, James Z.
• Primary Examiner(s): DULANEY, KATHLEEN YUAN
• Application Number:
  US13/542,326
• Publication Number:
  US 20130011070A1
• Time in Patent Office: 712 Days
• Field of Search: 382/224, 382/160, 715/968
• US Class Current: 382/160
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06F 18/24155 Bayesian classification
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/766 using regression, e.g. by projecting features on hyperplanes
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60

Abstract

The aesthetic quality of a picture is automatically inferred using visual content as a machine learning problem using, for example, a peer-rated, on-line photo sharing Website as data source. Certain visual features of images are extracted based on the intuition that they can discriminate between aesthetically pleasing and displeasing images. A one-dimensional support vector machine is used to identify features that have noticeable correlation with the community-based aesthetics ratings. Automated classifiers are constructed using the support vector machines and classification trees, with a simple feature selection heuristic being applied to eliminate irrelevant features. Linear regression on polynomial terms of the features is also applied to infer numerical aesthetics ratings.

Claims

1. A computer-based method of inferring and utilizing aesthetic quality of photographs and other images, comprising the steps of:

receiving a plurality of digitized images along with aesthetic-based ratings of the images;

performing one or more software operations on the digitized images to automatically extract a plurality of visual features representative of each image;

receiving an image without an aesthetic-based rating;

automatically extracting a plurality of visual features representative of the received image;

computing a familiarity measure for the received image by correlating the visual features extracted from the received image to the visual features extracted from the other images;

determining an aesthetic-based rating for the received image on the basis of the familiarity measure, wherein a lower familiarity is indicative of originality and a higher rating; and

using one or more statistical methods to correlate the extracted visual features and the aesthetic-based ratings to classify the images on the basis of aesthetic value, rate the images on a scale relating to aesthetics value, or select/eliminate an image based upon aesthetic quality.

2. The method of claim 1, wherein the correlation is performed using a support vector machine.

3. The method of claim 1, further including the step of converting each image into the HSV color space to produce two-dimensional matrices IH, IS and IV for feature extraction.

4. The method of claim 1, further including the step of automatically segmenting each image to identify objects in the image.

5. The method of claim 1, wherein one of the plurality of features is color distribution.

6. The method of claim 1, further including the step of performing a linear regression on one or more of the features to infer the aesthetic-based rating for the received image.

7. The method of claim 1, further including the use of a naï

ve Bayes classifier to classify the image based on aesthetic-based rating.

8. The method of claim 1, wherein linear regression and naï

ve Bayes classification methods are used in conjunction to select high-quality images or eliminate low-quality images from an image collection.

9. The method of claim 1, wherein the step of extracting a plurality of visual features representative of each image includes the employment of a content-based image retrieval algorithm.