Iterative saliency map estimation

• US 9,330,334 B2
• Filed: 10/24/2013
• Issued: 05/03/2016
• Est. Priority Date: 10/24/2013
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Lin, Zhe, Mech, Radomir, Wang, Peng
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US14/062,559
• Publication Number:
  US 20150117783A1
• Time in Patent Office: 922 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20004 Adaptive image processing
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation

Abstract

In techniques for iterative saliency map estimation, a salient regions module applies a saliency estimation technique to compute a saliency map of an image that includes image regions. A salient image region of the image is determined from the saliency map, and an image region that corresponds to the salient image region is removed from the image. The salient regions module then iteratively determines subsequent salient image regions of the image utilizing the saliency estimation technique to recompute the saliency map of the image with the image region removed, and removes the image regions that correspond to the subsequent salient image regions from the image. The salient image regions of the image are iteratively determined until no salient image regions are detected in the image, and a salient features map is generated that includes each of the salient image regions determined iteratively and combined to generate the final saliency map.

Claims

1. A method, comprising:

applying a saliency estimation technique to compute a saliency map of an image that includes image regions of the image;

determining a salient image region of the image from the saliency map;

removing an image region that corresponds to the salient image region from the image;

iteratively determining a subsequent salient image region of the image utilizing the saliency estimation technique to recompute the saliency map of the image with the image region removed, and removing an image region that corresponds to the subsequent salient image region from the image; and

generating a salient features map that includes each of the salient image regions determined iteratively and combined in the salient features map.

2. The method as recited in claim 1, wherein said iteratively determining the subsequent salient image regions of the image until no salient image regions are detected.

3. The method as recited in claim 1, wherein the salient image regions are combined by weighting each of the saliency maps computed for each respective iteration utilizing the saliency estimation technique to combine weighted saliency maps.

4. The method as recited in claim 3, wherein the weight is higher for the saliency maps obtained in earlier iterations of the saliency estimation technique.

5. The method as recited in claim 1, further comprising:

cropping the image based on a composition of the combined salient image regions in the salient features map.

6. The method as recited in claim 1, further comprising:

determining multiple image objects displayed in a complex image scene of the image, each of the determined salient image regions representing one of the image objects in the image.

7. The method as recited in claim 1, wherein the salient image regions of the image are determined based on at least one of an image segmentation technique that segments the image into segmented regions, and contrast thresholds of the image regions.

8. A computing device, comprising:

a memory configured to maintain an image that includes image regions of the image;

a processor to implement a salient regions module configured to;

apply a saliency estimation technique to compute a saliency map of the image;

determine a salient image region of the image from the saliency map;

remove an image region that corresponds to the salient image region from the image;

iteratively determine a subsequent salient image region of the image utilizing the saliency estimation technique to recompute the saliency map of the image with the image region removed, and remove an image region that corresponds to the subsequent salient image region from the image; and

generate a salient features map that includes each of the salient image regions determined iteratively and combined in the salient features map.

9. The computing device as recited in claim 8, wherein the salient regions module is configured to said iteratively determine the subsequent salient image regions of the image until no salient image regions are detected.

10. The computing device as recited in claim 8, wherein the salient regions module is configured to combine the salient image regions based on weighted saliency maps generated by weighting each of the saliency maps computed for each respective iteration utilizing the saliency estimation technique.

11. The computing device as recited in claim 10, wherein the weight is higher for the saliency maps obtained in earlier iterations of the saliency estimation technique.

12. The computing device as recited in claim 8, wherein the salient regions module is configured to crop the image based on a composition of the combined salient image regions in the salient features map.

13. The computing device as recited in claim 8, wherein the salient regions module is configured to determine multiple image objects displayed in a complex image scene of the image, each of the determined salient image regions representing one of the image objects in the image.

14. The computing device as recited in claim 8, wherein the salient regions module is configured to said determine the salient image regions of the image based on at least one of an image segmentation technique that segments the image into segmented regions, and contrast thresholds of the image regions.

15. A computer-readable storage memory device comprising a salient regions module stored as executable instructions and, responsive to execution of the instructions by a computing device, the computing device performs operations of the salient regions module comprising to:

compute a saliency map of an image that includes image regions of the image;

determine a salient image region of the image from the saliency map;

remove an image region that corresponds to the salient image region from the image;

iteratively determine a subsequent salient image region of the image by recomputing the saliency map of the image with the image region removed, and remove an image region that corresponds to the subsequent salient image region from the image; and

generate a salient features map that includes each of the salient image regions determined iteratively and combined in the salient features map.

16. The computer-readable storage memory device as recited in claim 15, wherein the computing device further performs operations of the salient regions module comprising to said iteratively determine the subsequent salient image regions of the image until no salient image regions are detected.

17. The computer-readable storage memory device as recited in claim 15, wherein the computing device further performs operations of the salient regions module comprising to crop the image based on a composition of the combined salient image regions in the salient features map.

18. The computer-readable storage memory device as recited in claim 15, wherein the computing device further performs operations of the salient regions module comprising to determine multiple image objects displayed in a complex image scene of the image, each of the determined salient image regions representing one of the image objects in the image.

19. The computer-readable storage memory device as recited in claim 15, wherein the computing device further performs operations of the salient regions module comprising to determine the salient image regions of the image based on at least one of an image segmentation technique that segments the image into segmented regions, and contrast thresholds of the image regions.

20. The computer-readable storage memory device as recited in claim 15, wherein the computing device further performs operations of the salient regions module comprising to combine the salient image regions by weighting each of the saliency maps computed for each respective iteration to combine weighted saliency maps.