Analyzing digital images based on criteria

• US 10,013,639 B1
• Filed: 12/16/2013
• Issued: 07/03/2018
• Est. Priority Date: 12/16/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Schurman, Eric Michael, Strand, William Alexander, Nelson, Jo-Pete, Liu, Yaodong
• Primary Examiner(s): Potts, Ryan P
• Application Number:
  US14/107,968
• Time in Patent Office: 1,660 Days
• Field of Search: None
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

Digital images, such as digital photographs, are analyzed by an application running on a user device or other computing apparatus. Heuristics, characteristic detection or measurement techniques, or other analytics are used to evaluate individual digital images or to compare a plurality of digital images in accordance with user-input criteria. Digital images are then presented to a user as a result of the analysis, and further operations may be performed per user selections or input. Numerous digital images may thus be timely evaluated for aesthetic appeal, composition, subject matter content, or other factors, and then deleted, printed, distributed, or put other use.

Claims

1. An apparatus comprising:

one or more hardware processors; and

one or more memory devices storing computer-executable instructions, the computer-executable instructions causing the one or more hardware processors to;

provide a user interface, the user interface including a plurality of digital images and a plurality of criteria for selection;

receive, via the user interface, a selection of at least a subset of the plurality of digital images;

receive, via the user interface, user input providing a selection of one or more criteria of the plurality of criteria from criteria options presented on the user interface, wherein the one or more criteria specify an objective for analyzing the at least a subset of the plurality of digital images according to visual characteristics and non-visual characteristics of the at least a subset of the plurality of digital images;

select, based on the objective specified by the one or more criteria, a heuristic from a plurality of heuristics;

analyze, using the heuristic, the at least a subset of the plurality of digital images by comparing the at least a subset of the plurality of digital images to each other digital image in the at least a subset of the plurality of digital images in accordance with the visual and non-visual characteristics, the comparing generating scores that quantify the visual and non-visual characteristics for each digital image of the at least a subset of the plurality of digital images in accordance with the objective specified by the one or more criteria selected via the user interface;

determine, based on the analysis, ranking values associated with the at least a subset of the plurality of digital images, the ranking values including the scores that quantify the visual and non-visual characteristics for each digital image of the at least a subset of the plurality of digital images relative to the visual and non-visual characteristics for each other digital image of the at least a subset of the plurality of digital images;

determine one or more digital images of the at least a subset of the plurality of digital images having a score greater than the scores associated with each other digital image of the at least a subset of the plurality of digital images;

cause the one or more digital images and information regarding the one or more digital images to be presented via a display;

receive the user input regarding at least one digital image of the one or more digital images, the user input selecting the at least one digital image of the one or more digital images; and

use machine learning to determine a learned operation to automatically perform with respect to the at least one digital image selected based on the user input.

2. The apparatus of claim 1, wherein:

the visual characteristics further include one or more of cut-off subject matter, as indicated by a portion of an object that extends outside of a frame of the one or more digital images of the at least a subset of the plurality of digital images or include one or more of an aesthetic characteristic or a specific content, and the ranking values are further determined based at least in part on a presence or absence of the cut-off subject matter or on a presence or absence of the aesthetic characteristic or a specific content.

3. The apparatus of claim 1, further comprising:

receiving, via the user interface, a selection of one or more of the non-visual characteristics from a plurality of predefined characteristics, wherein the non-visual characteristics further include metadata and audio criteria; and

determining, based at least in part on the selected one or more of the non-visual characteristics, the ranking values that include scores that quantify the one or more of the non-visual characteristics for each digital image of the one or more digital images relative to the one or more non-visual characteristics of each other digital image of the one or more digital images.

4. The apparatus of claim 1, further comprising computer-executable instructions to:

determine a first remote computing device of a plurality of computing devices that corresponds to a first score associated with a first digital image of the one or more digital images;

communicate the first digital image to the first remote computing device based on the first score;

determine a second remote computing device of the plurality of computing devices that corresponds to a second score associated with a second digital image of the one or more digital images; and

communicate the second digital image to the second remote computing device based on the second score.

5. A method, comprising:

providing a user interface, the user interface including a plurality of digital images for selection and a plurality of criteria;

receiving, via the user interface, a selection of at least a subset of the plurality of digital images;

receiving, via the user interface, user input providing a selection of one or more criteria of the plurality of criteria from criteria options presented on the user interface, wherein the one or more criteria specify an objective for analyzing the at least a subset of the plurality of digital images according to visual characteristics and non-visual characteristics of the at least a subset of the plurality of digital images;

determining, based on the objective specified by the one or more criteria, a heuristic from a plurality of heuristics;

analyzing, using one or more computing devices including one or more hardware processors, the at least a subset of the plurality of digital images by comparing the at least a subset of the plurality of digital images to each other digital image in the at least a subset of the plurality of digital images in accordance with visual and non-visual characteristics, the comparing generating ranking values including scores that quantify the visual and non-visual characteristics for each digital image of the at least a subset of the plurality of digital images relative to the visual and non-visual characteristics for each other digital image of the at least a subset of the plurality of digital images in accordance with the heuristic;

ranking the at least a subset of the plurality of digital images according to the scores;

presenting the at least a subset of the plurality of digital images on a display in accordance to the ranking;

selecting, using at least one of the one or more computing devices, one or more digital images of the at least a subset of the plurality of digital images presented on the display; and

determining, using machine learning, a learned operation to automatically perform on the selected at least one of the one or more digital images using the one or more computing devices.

6. The method of claim 5, further comprising

receiving, via the user interface, a selection of the non-visual characteristics from a plurality of predefined non-visual characteristics, wherein the non-visual characteristics further include metadata and audio criteria;

and

determining, based at least in part on the non-visual characteristics, the ranking values that include scores that quantify the non-visual characteristics for each digital image of the one or more digital images relative to the non-visual characteristics of each other digital image of the one or more digital images.

7. The method of claim 5, further comprising:

receiving, via the user interface, a selection of the visual characteristics from a plurality of predefined visual characteristics, wherein the plurality of predefined visual characteristics further includes one or more of an aesthetic characteristic or a subject matter content of the plurality of digital images; and

determining, based at least in part on the one or more of the aesthetic characteristic or the subject matter content, the ranking values that include scores that quantify each digital image of the one or more digital images relative to each other digital image of the one or more digital images.

8. The method of claim 5, further comprising receiving the user input providing the selection of the one or more criteria specifying the objective for analyzing the at least a subset of the plurality of digital images by quantifying the visual characteristics based on the user input.

9. The method of claim 5, wherein analyzing the at least a subset of the plurality of digital images further includes detecting cut-off subject matter within the at least a subset of the plurality of digital images, the method further comprising determining the ranking values based at least in part on the detecting of the cut-off subject matter, the ranking values including scores that quantify the cut-off subject matter of each digital image of the one or more digital images relative to each other image of the one or more digital images.

10. The method of claim 5, wherein analyzing the at least a subset of the plurality of digital images further includes detecting an undesirable visual characteristic within the at least a subset of the plurality of digital images, the method further comprising determining the ranking values based at least in part on the detecting of the undesirable visual characteristic, the ranking values including scores that quantify the undesirable visual characteristic of each digital image of the one or more digital images relative to each other image of the one or more digital images.

11. The method of claim 5, further comprising:

determining a first non-transitory computer-readable storage medium of the one or more computing devices that corresponds to a first score associated with a first digital image of the one or more digital images;

communicating the first digital image to the first non-transitory computer-readable storage medium based on the first score;

determining a second non-transitory computer-readable storage medium of the one or more computing devices that corresponds to a second score associated with a second digital image of the one or more digital images; and

communicating the second digital image to the second non-transitory computer-readable storage medium based on the second score.

12. The method of claim 5, further comprising accessing one or more of the plurality of digital images from one or more non-transitory computer-readable storage media associated with a digital camera configured to provide digital image data to at least one of the one or more computing devices.

13. The method of claim 5, wherein analyzing the at least a subset of the plurality of digital images further includes evaluating one or more of color balance, exposure, or contrast of the at least a subset of the plurality of digital images, the method further comprising determining the ranking values based at least in part on the evaluating of the one or more of the color balance, the exposure, or the contrast, the ranking values including scores that quantify the one or more of the color balance, the exposure, or the contrast of each digital image of the one or more digital images relative to each other image of the one or more digital images.

14. The method of claim 5, wherein the selected heuristic uses a machine learning technique that is based at least in part on the user input to the one or more computing devices, and wherein the user input corresponds to one or more of the operation or the one or more visual criteria.

15. The method of claim 5, wherein analyzing the at least a subset of the plurality of digital images further includes detecting a compositional characteristic within the one or more digital images, and wherein the compositional characteristic includes one or more of the rule of thirds, a Moiré

pattern, an illusion of motion of an object, or a background out of focus relative to a primary object, the method further comprising determining the ranking values based at least in part on the detecting of the compositional characteristic, the ranking values including scores that quantify the compositional characteristic of each digital image of the one or more digital images relative to each other image of the one or more digital images.

16. The method of claim 5, wherein causing the operation to be performed includes causing the one or more digital images to be deleted from one or more non-transitory computer-readable storage media without user intervention.

17. One or more non-transitory computer readable storage media including a program code, the program code configured to cause one or more hardware processors to:

access a plurality of digital images;

provide a user interface, the user interface including a plurality of criteria for selection, the plurality of criteria specifying an objective for analyzing the plurality of digital images according to visual characteristics and non-visual characteristics of the at least a subset of the plurality of digital images;

receive, via the user interface, user input selecting one or more criteria of the plurality of criteria to use for analysis;

select one or more heuristics based on the objective specified by the selected one or more criteria;

analyze, using the one or more heuristics, the plurality of digital images by comparing the at least a subset of the plurality of digital images to each other digital image in the at least a subset of the plurality of digital images in accordance with the visual and non-visual characteristics;

determine, based on the analysis, ranking values associated with the plurality of digital images, the ranking values including scores that quantify the visual and non-visual characteristics for each digital image of the plurality of digital images relative to the visual and non-visual characteristics of each other digital image of the plurality of digital images;

determine one or more digital images of the plurality of digital images having a score greater than the scores associated with each other digital image of the plurality of digital images;

cause, based on the analysis, presentation of the one or more digital images of the plurality of digital images;

receive a user input for selecting at least one digital image of the one or more digital images; and

determine, using machine learning, a learned operation to automatically perform with respect to the selected at least one digital image of the one or more digital images of the plurality of digital images.

18. The one or more non-transitory computer readable storage media of claim 17, wherein the visual characteristics include a placement of a first object relative to a placement of one or more other objects and the ranking values are determined based at least in part on the placement of the first object relative to the placement of the one or more other objects.

19. The one or more non-transitory computer readable storage media of claim 17, the program code further configured to cause the one or more hardware processors to detect a particular object within the plurality of digital images, wherein the ranking values are determined based at least in part on detection of the particular object.

20. The one or more non-transitory computer readable storage media of claim 17, wherein the visual characteristics further include a compositional characteristic within the plurality of digital images, the compositional characteristic includes one or more of the rule of thirds, a Moiré

pattern, an illusion of motion of an object, or a background out of focus relative to a primary object, and the non-visual characteristics further include metadata and audio criteria.