Measuring and visualizing impact of image modifications

• US 9,704,231 B1
• Filed: 05/18/2015
• Issued: 07/11/2017
• Est. Priority Date: 05/19/2014
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kulewski, Krzysztof, Krishnaswamy, Aravind, Babacan, Sevket Derin
• Primary Examiner(s): Saini, Amandeep
• Application Number:
  US14/715,156
• Time in Patent Office: 785 Days
• Field of Search: 382286
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

Implementations relate to visualizing and measuring impact of image modifications. In some implementations, a method to measure and indicate impact of image modification includes applying an edit operation to a first image, including modifying one or more pixels of the first image to provide a modified image. The method determines an impact score associated with the edit operation and indicative of a degree of visual impact of the edit operation to the first image. The method provides, based on the impact score, the modified image in a visualization of image modification for the first image, and provides the visualization for display by a display device.

Claims

1. A computer-implemented method to indicate impact of image modification, the method comprising:

applying a first edit operation to a first image, wherein applying the first edit operation includes modifying one or more pixels of the first image to provide a first modified image;

applying a second edit operation to the first modified image, wherein applying the second edit operation includes modifying one or more pixels of the first modified image to provide a second modified image;

determining a first impact score associated with the first edit operation and indicative of a degree of visual change in the first image caused by applying the first edit operation;

determining a second impact score associated with the second edit operation and indicative of a degree of visual change in the first modified image caused by applying the second edit operation;

determining that the first impact score and second impact score each satisfy an impact score threshold, wherein the satisfying of the impact score threshold indicates a greater degree of visual image change than a minimum degree of visual image change associated with the impact score threshold;

providing the first modified image and the second modified image in a visualization of image modification for the first image in response to the first impact score and second impact score each satisfying the impact score threshold; and

causing the visualization to be displayed by a display device.

2. The method of claim 1 wherein the visualization provides display of a progression of sequential changes made by sequential modifications by the first and second edit operations.

3. The method of claim 1 further comprising:

determining an overall degree of visual impact on the first image caused by the first and second edit operations based on combining the first and second impact scores into a total impact score; and

providing an indication of the determined overall degree of visual impact to the first image in the visualization, wherein the indication is based on a comparison of the total impact score with a plurality of predefined ranges of scores, each predefined range associated with a different amount of visual change.

4. The method of claim 1 further comprising determining a ranking of the second modified image, wherein the ranking is determined with respect to a plurality of other images, and wherein the ranking is based on the first and second impact scores of the first and second edit operations.

5. The method of claim 4 wherein determining the ranking of the second modified image further comprises determining the ranking based on a visual quality score of the second modified image, wherein the visual quality score is based on at least one of:

an exposure level of the second modified image, a noise level of the second modified image, and a sharpness level of the second modified image.

6. The method of claim 1 wherein the visualization includes one or more displayed highlights that visually indicate particular image pixels of at least one of the first modified image and the second modified image that were changed by at least one of the first edit operation and the second edit operation.

7. The method of claim 1 wherein determining the first impact score is based on a type of the first edit operation, wherein the type of the first edit operation is one of a plurality of types of edit operations, and wherein each type of edit operation is associated with a corresponding contribution to the first impact score, wherein each type of edit operation that changes only pixels of one or more faces depicted in the first image contributes a larger impact score than each type of edit operation that changes pixels of the one or more faces and at least one non-facial pixel in the first image.

8. The method of claim 1 wherein determining the first impact score is based on applying the first edit operation to the first image with one or more parameter values, wherein the first impact score is based on a magnitude of modification provided by the first edit operation as determined by the one or more parameter values.

9. The method of claim 1 wherein determining the first impact score is based on applying the first edit operation to modify pixels in an affected area of the first image, wherein the affected area has a size, and wherein the first impact score is based on the size of the affected area of the first image with respect to an entire area of the first image.

10. The method of claim 1 wherein determining the first impact score is based on applying the first edit operation to modify pixels in an affected area of the first image, wherein the first impact score is based on a location of the affected area with respect to a border of the first image.

11. The method of claim 1 further comprising:

determining a recommendation of whether to apply the first edit operation to one or more other images, wherein the recommendation is based on the first impact score; and

causing the recommendation to be output by the display device.

12. The method of claim 1 wherein the visualization includes one or more text labels that describe at least one of the first edit operation and second edit operation.

13. The method of claim 1 wherein providing the visualization for display by a display device includes displaying the first image, the first modified image, and the second modified image in a sequence corresponding to a sequence of applying the first and second edit operations.

14. The method of claim 1 wherein the visualization includes at least one of:

a simultaneous display of the first image and the first and second modified images; and

a sequential display in time of the first image, the first modified image, and the second modified image.

15. A system to indicate impact of image modification, the system comprising:

a storage device; and

at least one processor operative to access the storage device and configured to;

obtain a first image;

apply a first edit operation to the first image to generate a first modified image, wherein applying the first edit operation includes modifying one or more pixels of the first image to provide the first modified image;

apply a second edit operation to the first modified image to generate a second modified image, wherein applying the second edit operation includes modifying one or more pixels of the first modified image to provide the second modified image; and

determine a first impact score associated with the first edit operation and a second impact score associated with the second edit operation, wherein the first impact score is indicative of a degree of visual change in the first image caused by applying the first edit operation and the second impact score is indicative of a degree of visual change in the first modified image caused by the second edit operation;

determine that the first impact score and second impact score each satisfy an impact score threshold, wherein the satisfying of the impact score threshold indicates a greater degree of visual image change than a minimum degree of visual image change associated with the impact score threshold;

determine to include the first modified image and the second modified image in a displayable visualization in response to the first impact score and second impact score satisfying the impact score threshold; and

provide the displayable visualization that includes the first image, the first modified image, and the second modified image.

16. The system of claim 15 wherein the at least one processor is further configured to:

determine a ranking of the second modified image, wherein the ranking is determined with respect to a plurality of other images, and wherein the ranking is based on the first and second impact scores of the first and second edit operations, wherein determining the ranking of the second modified image further comprises determining the ranking based on a visual quality score of the second modified image.

17. The system of claim 16 wherein the first impact score is based on applying the first edit operation to modify pixels in an affected area of the first image and is based on at least one of:

a size of the affected area of the first image to which the first edit operation is applied with respect to an entire area of the first image; and

a location of the affected area with respect to a border of the first image.

18. A non-transitory computer readable medium having stored thereon instructions to indicate impact of image modification that, when executed by a processor, cause the processor to:

apply two or more edit operations to modify an image including a first edit operation that modifies one or more pixels of the image to provide a first modified image and a second edit operation that modifies one or more pixels of the first modified image to provide a second modified image;

determine two or more individual impact scores for the respective edit operations, wherein the two or more individual impact scores are each indicative of a degree of visual image change caused by the respective edit operations, including a first impact score associated with the first edit operation and indicative of a degree of visual change in the image caused by applying the first edit operation and a second impact score associated with the second edit operation and indicative of a degree of visual change in the first modified image caused by applying the second edit operation;

determine that the first impact score and second impact score each satisfy an impact score threshold, wherein satisfaction of the impact score threshold indicates a greater degree of visual image change than a minimum degree of visual image change associated with the impact score threshold;

determine, based on the two or more individual impact scores, a visualization that indicates the modification of the image by the two or more edit operations; and

provide the visualization for display by a display device.

19. The computer readable medium of claim 18

wherein the instructions further cause the processor to:

combine the two or more individual impact scores into a total impact score to determine an overall degree of visual impact of the two or more edit operations with respect to the image; and

provide an indication of the determined overall degree of visual impact in the visualization.

20. The computer readable medium of claim 18 wherein the first impact score is based on applying the first edit operation to modify pixels in an affected area of the image and is based on at least one of:

a size of the affected area of the image to which the first edit operation is applied with respect to an entire area of the image; and

a location of the affected area with respect to borders of the image.