Auto-cropping images using saliency maps

• US 8,660,351 B2
• Filed: 10/24/2011
• Issued: 02/25/2014
• Est. Priority Date: 10/24/2011
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Tang, Hao
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US13/279,961
• Publication Number:
  US 20130101210A1
• Time in Patent Office: 855 Days
• Field of Search: None
• US Class Current: 382/173
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: H04N 1/3873 defined only by a limited number of coordinate points or parameters, e.g. corners, centre; for trimming

Abstract

A method for auto-cropping includes, creating a saliency map corresponding to a digital image, the saliency map indicating a relevance of pixels within the digital image with saliency values within a number range, a lower end of the number range being less than zero and an upper end of the number range being greater than zero. The method further includes, analyzing the saliency map to find a potential cropping rectangle, the potential cropping rectangle having a maximum sum of saliency values within borders of the rectangle.

Claims

1. A method for auto-cropping, the method comprising:

with a physical computing system, creating a saliency map corresponding to a digital image, said saliency map indicating a relevance of pixels within said digital image with saliency values within a number range, a lower end of said number range being less than zero and an upper end of said number range being greater than zero;

with said physical computing system, analyzing said saliency map to find a potential cropping rectangle, said potential cropping rectangle having a maximum sum of saliency values within borders of said rectangle,

wherein saliency values of said saliency map associated with pixels of a detected face within said digital image are raised.

2. The method of claim 1, wherein creating said saliency map comprises use of descriptor values assigned to pixels within said digital image, said descriptor values representing a pixel as a mean and a standard deviation of colors within a color patch associated with said pixel.

3. The method of claim 2, wherein said descriptor values use color patches of multiple scales.

4. The method of claim 1, further comprising, adjusting said saliency map such that regions smaller than a predefined area with high saliency values are removed.

5. The method of claim 1, wherein said saliency map is adjusted so that saliency values towards a center point within said saliency map are raised in comparison to saliency values away from said center point.

6. The method of claim 1, further comprising, shifting all of said saliency values to change an aggressiveness of said potential cropping rectangle.

7. The method of claim 1, wherein said rectangle is subject to at least one of:

a fixed aspect ratio constraint, an aspect ratio range constraint, a fixed size constraint, and a size range constraint.

8. The method of claim 1, wherein analyzing said saliency map to find a potential cropping rectangle comprises;

representing subsets of rectangles in sets;

determining an upper bound sum of a rectangle within each of said sets;

placing said sets into a priority queue;

splitting a popped set from said priority queue into disjoint sets if said popped set includes more than one rectangle; and

identifying a rectangle from said popped set as said potential cropping rectangle if said set contains only one rectangle.

9. The method of claim 8, wherein determining said upper bound sum of a rectangle within a set comprises adding all positive saliency values within borders of the largest rectangle of said set with all the negative values within borders of the smallest rectangle within said set.

10. The method of claim 1, further comprising receiving a number of user defined constraints, the user defined constraints comprising constraints on aspect ratio of a potential cropped portion of the image, constraints on the size of a potential cropped portion of the image, or combinations thereof.

11. The method of claim 1, in which raising saliency values of said saliency map associated with pixels of a detected face within said digital image comprises raising the saliency values of said saliency map associated with pixels of a detected face by a set value.

12. A computing system comprising:

at least one processor;

a memory communicatively coupled to the at least one processor, the memory comprising computer executable code that, when executed by the at least one processor, causes the at least one processor to;

create a saliency map associated with a digital image, said saliency map indicating a relevance of pixels within said digital image with saliency values within a number range, a lower end of said number range being less than zero and an upper end of said number range being greater than zero;

analyze said saliency map to find a potential cropping rectangle, said potential cropping rectangle having a maximum sum of saliency values within borders of said rectangle,

wherein said at least one processor is further to adjust said saliency map such that regions smaller than a predefined area with high saliency values are removed.

13. The system of claim 12, wherein to create said saliency map, said at least one processor is further caused to use descriptor values assigned to pixels within said digital image, said descriptor values representing a pixel as a mean and a standard deviation of colors within a color patch associated with said pixel.

14. The system of claim 12, wherein said at least one processor is further to shift all of said saliency values to change an aggressiveness of said potential cropping rectangle.

15. The system of claim 12, further comprising raising saliency values of said saliency map associated with pixels of a detected face within said digital image.

16. A method for auto-cropping, the method comprising:

with a physical computing system, creating a saliency map that corresponds to a digital image, said saliency map defining saliency values within a range for pixels within said digital image;

with said physical computing system, shifting said range so that a low end of said range is less than zero while a high end of said range is greater than zero;

with said physical computing system, adjusting said saliency map such that regions smaller than a predefined area with high saliency values are removed;

with said physical computing system, finding a rectangle within said saliency map that has a maximum sum of saliency values within its borders; and

with said physical computing system, using said rectangle as a potential cropping boundary.

17. The method of claim 16, in which finding a rectangle within said saliency map that has a maximum sum of saliency values within its borders comprises;

representing subsets of rectangles in sets;

determining an upper bound sum of a rectangle within each of said sets;

placing said sets into a priority queue;

splitting a popped set from said priority queue into disjoint sets if said popped set includes more than one rectangle; and

identifying a rectangle from said popped set as said potential cropping rectangle if said set contains only one rectangle.

18. The method of claim 17, in which determining said upper bound sum of a rectangle within a set comprises adding all positive saliency values within borders of the largest rectangle of said set with all the negative values within borders of the smallest rectangle within said set.

19. The method of claim 16, in which said saliency values of said saliency map associated with pixels of a detected face within said digital image are raised.