Automatically cropping an image

• US 7,529,390 B2
• Filed: 10/03/2005
• Issued: 05/05/2009
• Est. Priority Date: 10/03/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sun, Yan-Feng, Ma, Wei-Ying, Zhang, Lei, Zhang, Mingju
• Primary Examiner(s): Wu; Jingge
• Assistant Examiner(s): Patel; Jayesh A
• Application Number:
  US11/242,605
• Publication Number:
  US 20070076979A1
• Time in Patent Office: 1,310 Days
• Field of Search: 382/118
• US Class Current: 382/118
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/32 Normalisation of the pattern dimensions
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: H04N 1/3873 defined only by a limited number of coordinate points or parameters, e.g. corners, centre; for trimming

Abstract

A method and system for cropping an image is provided. The cropping system automatically crops an image by selecting an image template whose condition is best satisfied by the image and then by selecting a cropping of the image that best attains the goal of the selected image template. The cropping system may use a metric or objective function to rate how well a cropping attains the goal of the selected image template. The cropping system may apply various optimization algorithms to identify a cropping that is the best as indicated by the metric. The cropping system can then automatically crop the image based on the identified cropping.

Claims

1. A method in a computing device with a processor and a computer-readable storage medium for cropping an image with an original composition, the method comprising:

providing image templates, each image template having a condition and a goal for a cropped composition, each condition of an image template specifying a number of one or more face areas within an image, each face area containing one or more faces, and a size of the one or more face areas within the image, each goal of an image template specifying an ideal location of significant areas of an image within a cropped image, the significant areas including face areas and attended areas of an image, an attended area being a region of interest of an image that does not include a face;

providing an objective function that generates an overall rating of a candidate cropping of the image based on a rating of the candidate cropping of the image to meet the goal of an image template, a rating of the original composition of the image relative to the candidate cropping, and a rating of how well the candidate cropping preserves faces;

identifying a number of one or more face areas within the image and the size of the one or more identified face areas, each identified face area containing one or more faces;

selecting an image template whose condition is satisfied by the image based on the identified number of face areas and the identified size of the one or more identified face areas;

identifying the candidate cropping based on the overall rating of the objective function of the candidate cropping to the selected image template; and

cropping the image in accordance with the identified candidate cropping

wherein the identifying, selecting, and cropping are performed by the processor executing computer-executable instructions stored in the computer-readable storage medium and

wherein the image templates and objective function are stored in the computer-readable storage medium.

2. The method of claim 1 wherein the size of the face area is selected from the group consisting of small, medium, and large.

3. The method of claim 1 wherein the number of face areas is selected from the group consisting of one, two, and three or more.

4. The method of claim 1 wherein the objective function factors in an artistic composition rating, an original composition rating, and a face preserving composition rating.

5. The method of claim 1 wherein the identifying of the candidate cropping applies a particle swarm optimization algorithm based on the objective function.

6. A computer-readable storage medium storing instructions for controlling a computer system to identify a cropping for an image with an original composition, by a method comprising:

providing image templates, each image template having a condition and a goal for a cropped composition, each condition of an image template indicating a number of face areas within an image, each face area containing one or more faces, and a size of the one or more face areas within the image that satisfies the condition of the image template, each goal of an image template specifying an ideal location of face areas and attended areas of an image, an attended area being a region of interest of an image that does not include a face;

providing an objective function that generates an overall rating of a candidate cropping of the image based on a rating of the candidate cropping of the image to meet the goal of an image template, a rating of the original composition of the image relative to the candidate cropping, and a rating of how well the candidate cropping preserves faces;

identifying a number of one or more face areas within the image and the size of one or more face areas, each face area containing one or more faces;

selecting an image template with a condition that matches the image based on comparison of the identified number of face area and the identified size of the identified one or more face areas of the image to the number of face areas and the size of face areas of the condition of the image templates;

identifying the candidate cropping of the image based on the overall rating of the objective function of the cropping to attain the goal of the selected image template; and

cropping the image in accordance with the identified candidate cropping.

7. The computer-readable storage medium of claim 6 wherein the identifying of the candidate cropping applies a particle swarm optimization algorithm.

8. The computer-readable storage medium of claim 6 wherein the rating of the candidate cropping indicates closeness of face areas of the cropped image to locations of face areas of the goal of the selected template.

9. A computer system with a processor and computer-readable storage medium to identify a cropping for an image, comprising:

an image template store having image templates, the image templates having face areas and attended areas, the image templates having associated conditions and goals, each condition of an image template specifying a number and size of one or more face areas within an image that contain a face, each goal of an image template specifying an ideal location of face areas and attended areas of an image within a cropped image, an attended area being a region of interest of an image that does not include a face;

an objective function component that generates an overall rating of a candidate cropping of the image based on a rating of the candidate cropping of the image to meet the goal of an image template, a rating of the original composition of the image relative to the candidate cropping, and a rating of how well the candidate cropping preserves faces;

a component that identifies the number and the size of one or more face areas and identifies attended areas of the image to be cropped;

a component that selects an image template with a condition that matches the image based on number and size of the identified face areas;

a component that identifies the candidate cropping of the image based on an overall rating of the objective function of the cropping to attain the goal of the selected image template; and

a component that crops the image in accordance with the identified candidate cropping;

wherein the components are implemented as computer-executable instructions stored in the computer-readable storage medium for execution by the processor and

wherein the image template store is stored in the computer-readable storage medium.