Systems and methods for creating and editing seam carving masks

• US 8,659,622 B2
• Filed: 08/31/2009
• Issued: 02/25/2014
• Est. Priority Date: 08/31/2009
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Dhawan, Anmol
• Primary Examiner(s): Sams, Michelle L
• Application Number:
  US12/551,178
• Publication Number:
  US 20130120442A1
• Time in Patent Office: 1,639 Days
• Field of Search: 345/619, 345/626
• US Class Current: 345/626
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12

Abstract

Systems and methods for creating and editing seam carving masks may allow a user to combine masks, and/or to modify automatically generated suggested masks, manually created masks, combination masks, or previously stored masks using tools of a graphical user interface in a graphics application (e.g., a mask brush or mask eraser). The method may include accessing data representing an image and automatically generating a suggested mask for the image (e.g., based on a color or color range, a threshold energy value, or input specifying two or more previously stored masks for combination). The method may include displaying the suggested mask as an overlay of the image, highlighting mask pixels using a respective color or pattern. The user may indicate pixels to be added to or removed from the suggested mask to produce a modified mask for application in an image editing operation (e.g., a resizing, filtering, or feature identification operation).

Claims

1. A system, comprising:

one or more processors; and

memory, communicatively coupled to the one or more processors, storing instructions executable by the one or more processors to perform operations comprising;

computing an energy value for each pixel in an image based, at least in part, on a derivative of a neighboring pixel;

generating a seam carving mask for the image based, at least in part, on the computing, the seam carving mask comprising data representing one or more pixels to be protected from or targeted by an image editing operation;

displaying the seam carving mask as an overlay of the image;

in response to receiving user input, modifying the displayed seam carving mask, the modifying comprising adding data representing one or more additional pixels to the seam carving mask or removing data representing at least a subset of the one or more pixels from the seam carving mask; and

storing data representing the modified seam carving mask for use in a subsequent image editing operation directed to the image.

2. The system of claim 1, wherein the seam carving mask is a protection mask indicating pixels to be protected during a seam carving operation or a removal mask indicating pixels to be prioritized for removal during a seam carving operation.

3. The system of claim 1,

wherein displaying the seam carving mask as an overlay of the image comprises highlighting the pixels indicated by the seam carving mask using a respective color or pattern;

and

wherein the respective color or pattern indicates whether the seam carving mask is a protection mask or a removal mask.

4. The system of claim 1, wherein generating the seam carving mask comprises:

identifying pixels of a particular color or range of colors; and

storing data representing the identified pixels as the seam carving mask.

5. The system of claim 1, wherein generating the seam carving mask comprises:

applying an automatic skin tone detection operation to data representing the image to identify pixels presumed to represent skin; and

storing data representing the identified pixels in the seam carving mask.

6. The system of claim 1, wherein generating the seam carving mask comprises:

receiving input specifying two or more previously stored seam carving masks; and

combining the two or more previously stored seam carving masks to produce the seam carving mask.

7. The system of claim 6, wherein generating the seam carving mask further comprises resolving a conflict between the two or more previously stored seam carving masks according to a default priority, an application-specific priority, an operation-specific priority, or a user-configurable priority, the conflict including one of the two or more previously stored seam carving masks identifying a particular pixel for protection while another of the two or more previously stored seam carving masks identifying the particular pixel for removal.

8. The system of claim 1, wherein generating the seam carving mask comprises identifying pixels of the image associated with energy values above or below a pre-determined threshold.

9. The system of claim 1, wherein the user input comprises an indication to add one or more pixels to the seam carving mask or an indication to remove one or more pixels from the seam carving mask.

10. A computer-readable storage memory storing instructions computer-executable to perform operations comprising:

generating a suggested seam carving mask for an image based, at least in part, on a difference in weighted energy values of a pixel in the image relative to another pixel adjacent to the pixel in the image;

displaying the suggested seam carving mask as an overlay of the image;

in response to receiving user input, modifying the suggested seam carving mask, the modifying comprising adding data representing one or more additional pixels to the suggested seam carving mask or removing data representing at least a subset of the one or more pixels from the suggested seam carving mask; and

storing data representing the modified seam carving mask for use in a subsequent image editing operation directed to the image.

11. The computer-readable storage memory of claim 10,

wherein displaying the suggested seam carving mask as an overlay of the image comprises highlighting the pixels indicated by the suggested seam carving mask using a respective color or pattern;

and

wherein the respective color or pattern indicates whether the suggested seam carving mask is a protection mask or a removal mask.

12. The tangible computer-readable storage medium of claim 10, the difference in energy values not being dependent on a color value of the pixel in the image.

13. The computer-readable storage memory of claim 10, wherein generating the suggested seam carving mask comprises:

receiving input specifying two or more seam carving masks; and

combining the two or more seam carving masks to produce the suggested seam carving mask.

14. The computer-readable storage memory of claim 10, wherein generating the suggested seam carving mask comprises identifying pixels of the image associated with energy values above or below a pre-determined threshold, the pre-determined threshold comprising a default limit based on the distribution of energy values in the image.

15. The computer-readable storage memory of claim 10, wherein the user input comprises an indication of one or more pixels to be added to the suggested seam carving mask or one or more pixels to be removed from the suggested seam carving mask.

16. A method, comprising:

automatically and without user input, generating a suggested seam carving mask for an image based, at least in part, on a difference in energy values of a pixel in the image relative to another pixel adjacent to the pixel in the image, wherein the suggested seam carving mask comprises data representing one or more pixels to be protected from or targeted by an image editing operation;

displaying, by a computing device, the suggested seam carving mask as an overlay of the image; and

in response to receiving user input, modifying the suggested seam carving mask, the modifying comprising adding data representing one or more additional pixels to the suggested seam carving mask or removing data representing at least a subset of the one or more pixels from the suggested seam carving mask.

17. The method of claim 16,

wherein displaying the suggested seam carving mask as an overlay of the image comprises highlighting the pixels indicated by the suggested seam carving mask using a respective color or pattern;

and

wherein the respective color or pattern indicates whether the suggested seam carving mask is a protection mask or a removal mask.

18. The method of claim 16, wherein generating the suggested seam carving mask comprises:

identifying pixels of a particular color or pixels of a particular range of colors; and

storing data representing the identified pixels as the suggested seam carving mask.

19. The method of claim 16, wherein generating the suggested seam carving mask comprises:

receiving a selection of two or more stored seam carving masks; and

combining the selected two or more stored seam carving masks to produce the suggested seam carving mask.

20. The method of claim 16, wherein the user input comprises an indication to add one or more pixels to the suggested seam carving mask or an indication to remove one or more pixels from the suggested seam carving mask.