Identifying high saliency regions in digital images

• US 8,401,292 B2
• Filed: 04/26/2011
• Issued: 03/19/2013
• Est. Priority Date: 04/26/2011
• Status: Active Grant

• Current Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Original Assignee: Eastman Kodak Company
• Inventors: Park, Minwoo, Loui, Alexander C., Kumar, Mrityunjay
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US13/094,217
• Publication Number:
  US 20120275701A1
• Time in Patent Office: 693 Days
• Field of Search: 382173-180, 382/103, 382/100, 382/164, 382/224, 382/190, 382/199, 382/162, 382/108, 382/168, 382/171
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06V 10/26 Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

A method for identifying high saliency regions in a digital image, comprising: segmenting the digital image into a plurality of segmented regions; determining a saliency value for each segmented region, merging neighboring segmented regions that share a common boundary in response to determining that one or more specified merging criteria are satisfied; and designating one or more of the segmented regions to be high saliency regions. The determination of the saliency value for a segmented region includes: determining a surround region including a set of image pixels surrounding the segmented region; analyzing the image pixels in the segmented region to determine one or more segmented region attributes; analyzing the image pixels in the surround region to determine one or more corresponding surround region attributes; determining a region saliency value responsive to differences between the one or more segmented region attributes and the corresponding surround region attributes.

Claims

1. A method for identifying high saliency regions in a digital image having an array of image pixels, comprising:

using a data processor to automatically analyze the digital image to segment the digital image into a plurality of segmented regions, each segmented region including a set of image pixels and being bounded by a segment boundary;

determining a saliency value for each segmented region by;

determining a surround region including a set of image pixels surrounding the segmented region, wherein the surround region has an outer boundary and an inner boundary defined by the segment boundary;

using a data processor to automatically analyze the image pixels in the segmented region to determine one or more segmented region attributes;

using a data processor to automatically analyze the image pixels in the surround region to determine one or more corresponding surround region attributes; and

determining a region saliency value responsive to differences between the one or more segmented region attributes and the corresponding surround region attributes;

merging neighboring segmented regions that share a common boundary in response to determining that one or more specified merging criteria are satisfied, wherein the specified merging criteria include a criterion that the saliency of the merged segmented region be higher than the saliency of neighboring segmented regions;

designating one or more of the segmented regions having the highest region saliency values to be high saliency regions; and

storing an indication of the one or more high saliency regions in a processor-accessible memory.

2. The method of claim 1 wherein the determination of the region saliency value includes:

determining attribute difference values between each of the one or more segmented region attributes and the corresponding surround region attributes; and

combining the attribute difference values to determine the region saliency.

3. The method of claim 2 wherein the region saliency is determined by averaging the attribute difference values.

4. The method of claim 2 wherein the region saliency is determined by computing the maximum of the attribute difference values.

5. The method of claim 2 wherein the attribute difference values are scaled to a predefined range before they are combined to determine the region saliency.

6. The method of claim 1 wherein one of the segmented region attributes is a color histogram attribute, an average color attribute, a color contrast attribute, an edge response attribute, an isocentric curvedness attribute, an image texture attribute or a semantic content attribute.

7. The method of claim 1 wherein the segmented regions are determined based on a color of the image pixels.

8. The method of claim 1 wherein the determination of the surround region includes:

applying a morphological dilation operator to the segmented region to form a dilated region; and

designating the image pixels that are included in the dilated region but are not included in the segmented region to be the surround region.

9. The method of claim 8 wherein the morphological dilation operator uses a square structuring element having a predefined size.

10. The method of claim 1 wherein the surround region is determined by applying a set complement operator to the segmented region.

11. The method of claim 1 wherein the specified merging criteria include a criterion that a region similarity value determined to represent a similarity the two neighboring segmented regions exceeds a predefined threshold.

12. The method of claim 11 wherein the region similarity value is determined responsive to differences between the one or more segmented region attributes for the two neighboring segmented regions.

13. The method of claim 1 wherein the digital image is down-sampled to a lower resolution before segmenting the digital image into the plurality of segmented regions.

14. A system comprising:

a data processing system; and

a memory system communicatively connected to the data processing system and storing instructions configured to cause the data processing system to implement a method for identifying high saliency regions in a digital image having an array of image pixels wherein the method includes;

analyzing the digital image to segment the digital image into a plurality of segmented regions, each segmented region including a set of image pixels and being bounded by a segment boundary;

determining a saliency value for each segmented region by;

determining a surround region including a set of image pixels surrounding the segmented region, wherein the surround region has an outer boundary and an inner boundary defined by the segment boundary;

analyzing the image pixels in the segmented region to determine one or more segmented region attributes;

analyzing the image pixels in the surround region to determine one or more corresponding surround region attributes; and

determining a region saliency value responsive to differences between the one or more segmented region attributes and the corresponding surround region attributes;

merging neighboring segmented regions that share a common boundary in response to determining that one or more specified merging criteria are satisfied, wherein the specified merging criteria include a criterion that the saliency of the merged segmented region be higher than the saliency of neighboring segmented regions;

designating one or more of the segmented regions having the highest region saliency values to be high saliency regions; and

storing an indication of the one or more high saliency regions in a processor-accessible memory.